UC Davis logo banner small

Network 21 Router Pilot Test Results

Table of Contents

Vendor - Anixter / Bay Networks

Introduction

During the test period of March 16, 1998, to March 27, 1998 the Network 21 Router Pilot Test was conducted with routing platforms supplied by Vendor B - Bay Networks, Inc. in partnership with Anixter, Inc. This document provides a summary of the test results and problems encountered.

Test Purpose and Description

The purpose of the Router Pilot Test was to confirm satisfactory operation and to baseline the performance of Bay Networks BLN-2 routers within an ATM network comprised of 3Com ATM switches. The results of this testing will be used to score the Bay Networks products within the context of an RFP bid evaluation. Five tests were performed, categorized as follows:

Each test was divided into sub-tests to facilitate the testing and analysis process. Each sub-test will be individually assessed and scored based upon test results. Additional pass/fail thresholds were set at levels that would reject behavior or performance that in the University's judgment indicates actual malfunction, failure to meet the design intent of the devices or their application within the UC Davis campus network.

Test Participants

All tests were conducted and scored by UC Davis and INS personnel. 3Com Corporation, Bay Networks and Anixter personnel were present to provide technical support. The following individuals were present during the Router Pilot Testing:

 

Test Configuration

The Router Pilot Test Network consists of eight 3Com CoreBuilder 7000 switches, two 3Com SuperStack 3000 switches, ten 3Com SuperStack 1000 switches and two Bay Networks BLN-2 routers, all of which are interconnected via ATM links at OC-3 (155 Mbps) speed. Two Bay Networks BLN-2 routers were installed, each with the following configuration:

1 each BLN series chassis w/ dual power supplies and backplane revision 2

1 each FRE2 Processor Module (Rev. 17)

1 each Ethernet Interface

2 each ATM Routing Engine (ARE), processor revision 10, w/ OC-3 MMF Interface

Release 12.0 of system software and image files

Test equipment was inserted into this configuration at various points for the purpose of injecting traffic, observing device behavior, and measuring performance parameters. The logical configuration for the Router Pilot Test consisted of fifteen Emulated LANs (ELANs) simulating eight different configuration types expected in the final production network.

 

 

 

Network 21 Router Test 1A - Basic Configuration Test

Test Description and Purpose

This test is called the Basic Configuration Test. This is a part of the test suite "Router Test 1 – Device Configuration". This test satisfies portions of Item 7 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to demonstrate basic functional aspects of router device configuration.

This test will demonstrate fundamental configuration of a router attached to two ELANs before full deployment. Configuration tests will include interface addressing, starting up a routing protocol, and remote access.

Test Configuration

Test Topology

Network Topology: This test will use a portion of the fully deployed, fully configured test topology used in the 3Com tests. This test will use the router attached in Area 3.

Hosts: Three IP hosts with one in ELAN Test9, one in ELAN Test12, and one attached to the management ELAN. One serial terminal device will be needed for physical console access.

Testers: No testers are necessary unless problems are encountered.

Test Conditions

This is a pre-deployment scenario with the router in a bench-test environment for accessibility. The router is connected to an ATM port for access to ELANs Test5 and Test9. The router will also be connected to the management ELAN via an Ethernet interface. Test hosts will be pinged to determine connectivity.

Process

    1. Connect terminal device to the Area 3 router console port. Connect an IP host to a port in ELAN Test5 and another to a port in ELAN Test9.
    2. Power-up router.
    3. Configure the management network Ethernet interface on the router as per Attachment A from the router console port.
    4. Telnet from NMS1 on the management ELAN to the router and configure ELANs Test5 and Test9 on an ATM interface as per Attachment A.
    5. Configure the router for RIPv2. Configure RIPv2 to route networks 169.237.0.0 and 128.120.0.0.
    6. Ping all IP hosts as described in the Recorded Results section below from the NMS workstation to verify connectivity.

 

Items Under Test

Items Under Test:

    1. CoreBuilder Switch
    2. ELAN Services
    3. Network Management System
    4. Standards and Protocols
    5. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch
    2. Physical Media
    3. IP Hosts
    4. Test Equipment

Expected Results

The router will be configured with two ELANs on an ATM interface and one LAN on the Ethernet interface. The router will pass traffic once the routing protocol is configured.

Recorded Results

Test Parameter

Result

Console Access

Yes

Telnet Access

Yes

Ping Source

Ping Destination

Reachability

Router

Management Host

Yes

Router

ELAN Test9 Host

Yes

Router

ELAN Test5 Host

Yes

Management Host

Router

Yes

Management Host

ELAN Test9 Host

Yes

Management Host

ELAN Test5 Host

Yes

ELAN Test9 Host

Router

Yes

ELAN Test9 Host

Management Host

Yes

ELAN Test9 Host

ELAN Test5 Host

Yes

ELAN Test5 Host

Router

Yes

ELAN Test5 Host

Management Host

Yes

ELAN Test5 Host

ELAN Test9 Host

Yes

 

Analysis for Test 1A

The Area 3 router was configured from a telnet session following basic IP address programming from a console session. The router ATM interfaces were configured and successfully obtained the proper ATM network prefix from its associated ATM switch interface. LAN Emulation Clients (LEC) were configured on ATM sub-interfaces. The ATM Forum well known address for the LAN Emulation Configuration Server (LECS) was obtained by the router from the 3Com ATM switches. The LEC's successfully registered with the appropriate LES/BUS. At this point, LEC's for ELANs Test5 and Test9 were fully functional on the router. RIP v2 was configured on the router via a telnet session. Successful pings were performed between hosts on ELAN Test5, ELAN Test9 and the management subnet as well as between these hosts and various router interfaces.

 

Network 21 Router Test 1B - In-depth Configuration Test

Test Description and Purpose

This test is called the In-depth Configuration Test. This is a part of the test suite "Router Test 1 – Device Configuration". This test satisfies Item 7 of the Test Spec Matrix. This test is to be performed during the Router Pilot.

The purpose of this test is to demonstrate advanced configuration options of the routers.

This test will demonstrate advanced configurations of the devices by doing a complete router configuration in Area 3 and a partial configuration in the other router to accommodate further Network Management System configuration.

Test Configuration

Test Topology

Network Topology: This test requires the fully deployed, fully configured 3Com test topology with the addition of routers which will be configured as part of this test and following network management test.

Hosts: Two IP hosts; one on the Management ELAN and one to travel to each ELAN for connectivity tests.

Testers: No testers are necessary unless problems are encountered.

Test Conditions

This test involves doing advanced configuration on the router and verifying the results across the network.

Process

    1. Connect IP host to management ELAN. Use Telnet for configuration tasks.
    2. Configure Area 3 router ATM interface 1 and interface 2 subinterfaces 1-7 per Attachment A.
    3. Configure RIPv2 as the routing protocol. Do not configure for OSPF routing at this time.
    4. From the router console port, configure the Area 2 router Ethernet interface as per Attachment A.
    5. Configure the Area 2 router for RIPv2.
    6. Configure the Area 3 and Area 2 router for SNMP management.
    7. Verify connectivity to configured subnets from the management ELAN by pinging each router interface from NMS1.
    8. Verify connectivity from configured subnets to the management ELAN by connecting an IP host to each Area 3 ELAN and pinging NMS1 from each host.
    9. Verify configuration with vendor.
    10. Save configurations to non-volatile memory.
    11. Power off routers. Make a note of the time.
    12. After 10 seconds, power on the router in Area 3 and verify the correctness of the configuration from non-volatile memory.
    13. After 30 minutes, power on the router in Area 3 and verify the correctness of the configuration from non-volatile memory.
    14. From the NMS or a telnet session, reset/reload routers and verify correctness of configuration from non-volatile memory.
    15. Test connectivity to the management ELAN from every Area 3 configured ELAN as in step 8 above.

Items Under Test

Items Under Test:

    1. SuperStack Switch
    2. CoreBuilder Switch
    3. ELAN Services

Items NOT Under Test and NOT Influencing Results:

    1. Network Management System
    2. Standards and Protocols
    3. Router
    4. Physical Media
    5. IP Hosts
    6. Test Equipment

Expected Results

The router can be configured to the ELAN level using remote access. The router can save configuration to non-volatile memory and restore that configuration after a reset or power cycle lasting at least 30 minutes. The Area 3 router is fully configured and the Area 2 router is ready to accept further configuration via SNMP.

Recorded Results

Test Parameter

Passed?

Local Configuration

Yes

Save & Restore Configuration

Yes

Remote Configuration

Yes

Correct Configuration after RESET

Yes

Correct Configuration after Power Cycle

Yes

Correct configuration after 30 minute Power Cycle

Yes

Connectivity to each ELAN

Yes

 

Analysis for Test 1B

Additional LEC's were configured on interfaces A3 and A4 on the Area 3 router. These LEC's successfully registered with their appropriate LES/BUS and became active. The Area 2 and Area 3 routers were successfully configured for RIP v2 and SNMP management. All ping tests were completed successfully. The router under test retained its configuration after power cycle and reset procedures demonstrating that configurations can be restored from NVRAM after power outages.

 

 

Network 21 Router Test 1C - Authentication Test

Test Description and Purpose

This test is called the Authentication Test. This is a part of the test suite "Router Test 1 – Device Configuration". This test satisfies Item 8 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to demonstrate at least two levels of authentication/authorization.

This test will use an IP host connected to the management ELAN and initiate Telnet session to the routers and verify the functionality of multiple levels of authentication and authorization and the differences between read-only access and full access authorization. If convenient, dynamic authentication/authorization will be investigated in preparation for testing of value-added features.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully configured 3Com test topology with the addition of two routers.

Hosts: NMS1 acting as an IP host for Telnet access.

Testers: Testers are not necessary unless problems are encountered.

Test Conditions

This test involves the entire 3Com pilot with an IP host connected to the management ELAN. Routers are accessed by NMS1 via Telnet session.

Process

    1. From NMS1, Telnet to the Area 3 router and login as read-only. Verify the ability to display router interface status and statistics, router CPU information, and routing information.
    2. Attempt to view SNMP community strings, router configuration files, and router passwords.
    3. Attempt to disable ATM interface 1. Attempt to modify the IP address for ATM interface 1. Attempt to save configuration. Attempt to reload/restart router.
    4. Logout of router.
    5. From NMS1, Telnet to the Area 3 router and login with full access read-write privileges. . Verify the ability to display router interface status and statistics, router CPU information, and routing information.
    6. Verify access to sensitive commands such as SNMP community strings and router passwords. Verify access to configuration files.
    7. Disable ATM interface 1 and verify that the router reports that the interface is down. Enable ATM interface 1 and verify that the router reports that the interface is up. Change the IP address of ATM interface 1 and verify with the router that the IP address has been changed. Return to original configuration after verification. Save configuration and reload.
    8. Document additional features (except dynamic authentication/authorization) beyond two levels of access.

Items Under Test

Items Under Test:

    1. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch
    2. CoreBuilder Switch
    3. ELAN Services
    4. Network Management System
    5. Standards and Protocols
    6. Physical Media
    7. IP Hosts
    8. Test Equipment

Expected Results

The router will provide at least two levels of access with appropriate levels of access to the command interpreter.

Recorded Results

Test Parameter

Result

Read-Only Access

Yes

Full Access

Yes

Other - UNIX

Yes

 

Analysis for Test 1C

The routers provide two levels of access in the default configuration. Each level of access grants appropriate privileges to the switch command interpreter. Passwords cannot be viewed from the router command interpreter or from the Site Manager GUI. Password recovery is intrusive, but can be performed locally. Access to the Site Manager GUI can be restricted using conventional UNIX group and user restrictions.

 

 

Network 21 Router Test 2A - Router Basic Management Test

Test Description and Purpose

This test is called the Router Basic Management Test. This is a part of the test suite "Router Test 2 -- Management". This test satisfies Items 6 and 16 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to demonstrate the router's ability to interact with and be configured by the SNMP network management systems for basic functions.

This test will use a single SNMP network management station to configure all the ELAN interfaces associated with the Area 2 router. Host connections will be made across the router to verify the functionality of the configuration and compatibility with previous configurations done via serial or Telnet access to the command interpreter.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully configured 3Com test topology with one fully configured router and one partially configured router.

Hosts: One SNMP management station and two portable IP hosts.

Testers: No testers are necessary unless problems are encountered.

Test Conditions

The remaining router is fully configured using an SNMP network management station. Verification of connectivity to remote ELANs is accomplished between the network management station and a portable IP host.

Process

    1. Using NMS1, configure the router in Area 2 with all of its primary ELANs as per Attachment A. Configure both routers for OSPF routing. Remove all RIPv2 routing configured previously.
    2. Verify configuration with vendor. Save configuration.
    3. Telnet to the Area 2 router and reset/reload. From NMS1, verify router recovery. Manually compare the active configuration to the previously saved configuration file and insure that the configuration has not changed.
    4. Test connectivity to each ELAN as follows:

Items Under Test

Items Under Test:

    1. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch
    2. CoreBuilder Switch
    3. ELAN Services
    4. Network Management System
    5. Standards and Protocols
    6. Physical Media
    7. IP Hosts
    8. Test Equipment

Expected Results

The NMS will configure the router with all its associated ELANs. These ELANs are reachable via IP from any other ELAN on the test network.

Recorded Results

Test Parameter

Result

SNMP Configuration Process

Yes

Uses a Single Station

Yes

Configuration Permanence

Yes

ELAN Connectivity (same router)

Yes

ELAN Connectivity (between routers)

Yes

 

Analysis for Test 2A

Using the Site Manager GUI and Optivity Network Management Software (NMS), the test conductors successfully configured the routers per the test configurations. The router retained all configuration information after power-cycles were performed. OSPF routing was configured and route advertisements were successfully exchanged between the routers. All connectivity tests were successful.

 

 

Network 21 Router Test 2B - Management Functionality Test

Test Description and Purpose

This test is called the Management Functionality test. This is a part of the test suite "Router Test 2 -- Management". This test satisfies portions of Items 9 and 16 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to verify the support of SNMP and appropriate IETF MIBs as well as vendor provided MIBs.

This test will issue SNMP queries to the routers examine the results to insure support and accuracy. This test will also investigate results from SNMP queries using vendor provided MIBs. This test will also exercise the capability of the router to send a trap for a link down and the capability of the NMS to initiate an action when the trap is received.

Test Configuration

Test Topology

Network Topology: This test will use the fully deployed, fully configured router test topology.

Hosts: One network management station. Two IP hosts will be needed if no traffic has been passed on the network to pre-load the interface counters.

Testers: No testers are necessary unless problems are encountered.

Test Conditions

This test involves the full router test topology and an SNMP management station. If the network has not been exercised since the last reset, two IP hosts will be attached to ELANs Test9 and Test7 in Area 3; these two hosts will transfer files to load the interface counters on the two routers. The two IP hosts are disconnected after the transfers.

Process

    1. Connect SNMP network management station.
    2. Telnet to routers and check interface counters. Determine if hosts are needed to pre-load counters. Connect hosts to ELANs Test7 and Test9 and transfer files to set counters to non-zero.
    3. Using a Telnet connection, query the send and receive counters for router ATM subinterfaces associated with ELANs Test7 and Test9. Query the send and receive counters for ELAN Test15 (router backbone) on both routers.
    4. Using the router SNMP manager from NMS1, query octets that have gone in and out of the router ATM subinterfaces associated with ELANs Test7, Test9, and Test15. Verify the results with the numbers obtained in step 3.
    5. Using an "SNMP Walk" or "MIB Browser" type of utility, verify the functionality of the appropriate IETF MIBs for the router and its interfaces.
    6. Using an "SNMP Walk" or "MIB Browser" type of utility, verify and document the added features of the vendor provided proprietary MIBs.
    7. Configure NMS1 to accept a link down trap and generate a trap message, change GUI link status, and sound an audible alarm upon receipt of this trap.
    8. Configure both routers to send SNMP traps to NMS1.
    9. Disconnect ATM link r3a3 in Area 3. Verify that NMS1 received a link down trap, generated a trap message, changed the GUI link status, and sounded an audible alarm as configured in step 7.
    10. Reconnect link r3a3. Verify that NMS1 received a "link recovered" message, generated a link up trap message, and changed the GUI link status.

Items Under Test

Items Under Test:

    1. Network Management System
    2. Standards and Protocols
    3. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch
    2. CoreBuilder Switch
    3. ELAN Services
    4. Physical Media
    5. IP Hosts
    6. Test Equipment

Expected Results

The router and NMS will interact accurately with SNMP using both IETF and proprietary MIBs. The router is capable of sending traps and the NMS is capable of processing the trap and performing a pre-defined task.

Recorded Results

Test Parameter

Result

SNMP and Console Statistic Consistency

Yes

IETF MIB Functionality

Yes

Proprietary MIB Functionality

Yes

Router Sends Trap

Yes

NMS Responds to Trap

Yes

 

Analysis for Test 2B

All SNMP queries returned accurate information when compared to counters on the routers. Traffic counters cannot be reset or cleared on the router itself, but the Site manager GUI can query for the change in traffic counters over a given period of time. Management Information Base (MIB) functionality was confirmed via the HP OpenView MIB browser utility and the Site Manager GUI. Standard MIB-II objects are supported with the exception of ospf and atm. Extensive support for Bay Networks (Wellfleet) private MIBs are available and functional (including ospf and atm objects) from the Site Manager GUI. Trap messages for link failures were successfully transmitted by the routers and received by the NMS. The NMS successfully generated alert notifications. One bug was observed during the testing. The Optivity software will process a trap from the router first, then pass on a "meta-trap" to HP-OV. Optivity successfully issued a "link-down" meta-trap to HP-OV when a link failure was induced on a router interface. When the router interface was restored to normal operation, Optivity correctly displayed the "link-up" status, but passed a second "link-down" meta-trap to HP-OV.

 

 

Network 21 Router Test 2C - Multiple Workstation Management Distribution Test

Test Description and Purpose

This test is called the Multiple Workstation Management Distribution Test. This is a part of the test suite "Router Test 2 -- Management". This test satisfies Item 31 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to determine the functionality of the distributed network management, polling in this environment, and determine an adequate number of remote polling devices.

This test will exercise distributed network management in the NOC. This test will also exercise a distributed polling station to determine the load on a single station and the network by a given number of routers.

Test Configuration

Test Topology

Network Topology: This test will use the fully deployed, fully configured router test topology.

Hosts: Two SNMP network management stations, one remote polling IP host.

Testers: A Sniffer will be connected with the remote polling IP host to determine network load caused by polling.

Test Conditions

This test uses the full router test topology and adds two network management stations and one remote polling station. No other IP hosts are needed.

Process

    1. Connect NMS1 and NMS2 to the management ELAN.
    2. Exercise the network management systems with simultaneous, distributed tasks as follows:
    1. Configure NMS1 to use a remote polling station.
    2. Connect the Sniffer and the remote polling station to the management ELAN. Verify configuration and basic connectivity.
    3. Configure the remote polling station to query routers every 5 seconds.
    4. Use the Sniffer to record network load from polling station.
    5. Record resource utilization on the remote polling station if possible.
    6. Disconnect the Ethernet link between the Area 3 router and the management ELAN. Verify that appropriate traps are received by NMS1 and that the polling station notifies NMS1 of the link down condition.
    7. Reconnect the Ethernet link to the Area 3 router from the management ELAN and verify that appropriate traps and notifications are sent and received.
    8. Return the management systems to their pretest configurations.

Items Under Test

Items Under Test:

    1. Network Management System
    2. Standards and Protocols
    3. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch
    2. CoreBuilder Switch
    3. ELAN Services
    4. Physical Media
    5. IP Hosts
    6. Test Equipment

Expected Results

The two NOC network management stations will be able to perform distributed independent tasks on the routers. The remote polling station will operate with a reasonable network load to the devices in monitors. NMS1 will be able to function with the remote polling station.

Recorded Results

Test Parameter

Result

NMS Distributed Tasks to Routers.

No

Remote Polling Station Network Load

N/A

Remote Polling Station Resource Utilization

N/A

NOC to Remote Polling Workstation Functionality

No

 

 

Analysis for Test 2C

Remote polling capability is not supported by the Optivity software.

 

 

Network 21 Router Test 3A - Throughput and Capacity Test

Test Description and Purpose

This test is called the throughput and capacity test. This is a part of the test suite "Router Test 3 --Measurement". This test satisfies Items 3, 4, and 20 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to determine the capabilities of an ATM router interface and verify that specified equipment meets our throughput specifications.

This test will determine the Maximum Forwarding Rate (MFR) of an ATM router interface. The traffic load offered to the router will initiate from Ethernet at various sizes. Several calculations will be made to determine the aggregate forwarding rate of the routers (using 64 Byte Ethernet frames). The MFR with 256 Byte Frames will determine the suitability for use in a real life 9400 port network.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully configured router test topology.

Hosts: None.

Testers: This test requires the Netcom SmartBits ELAN Advanced Switch Test

Test Conditions

This test uses the full router test network with routers and testers; no other devices are needed. The testers will flood the network to determine the MFR at different frame sizes. Steps 3-6 are mathematical calculations used to determine MFR.

Process

    1. Connect SmartBits ports 1-10 to ELAN Test9 on IDF312. Connect SmartBits ports 21-30 to ELAN Test3 on IDF 221.

    2. Execute SmartBits VAST ELAN by IP Subnet test with frame sizes of 64, 128, 256, 512, 1024, and 1500 Bytes. Record the MFR for each frame size.

    3. Apply the largest MFR from the results above to the number of router interfaces specified by vendor for this solution and calculate the maximum aggregate forwarding rate of the router solution.

    4. Using MFR and frame sizes, calculate bandwidth for each frame size.

    5. Apply largest bandwidth to the number of router interfaces specified by vendor. Determine the maximum router aggregate bandwidth.

    6. Apply the MFR for 256 Byte frames and apply to the number of router interfaces specified by the vendor to determine the actual forwarding rate.

Items Under Test

Items Under Test:

    1. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch

    2. CoreBuilder Switch

    3. ELAN Services

    4. Network Management System

    5. Standards and Protocols

    6. Physical Media

    7. IP Hosts

    8. Test Equipment

Expected Results

The entire router solution as bid will forward an aggregate of at least 2.4 million 64-byte packets per second calculated by multiplying the forwarding rate of an interface by the total number of interfaces as bid. In addition, the entire router solution will forward at least 600,000 256-byte packets per second calculated by multiplying the forwarding rate of an interface by the total number of interfaces as bid. The entire router solution will have an aggregate bandwidth of at least 2.7 Gbps calculated by multiplying the per interface bandwidth by the total number of interfaces bid.

Recorded Results

Frame Size

Theoretical

Max. MFR

MFR

Percent

Packet Loss

Aggregate MFR

Aggregate

Bandwidth

64

144,810 pps

144,662 pps

0.102%*

2.604 Mpps

2.79 Gbps

128

84,446 pps

84,446 pps

0.000%

1.520 Mpps

2.79 Gbps

256

45,290 pps

45,290 pps

0.000%

815,220 pps

2.79 Gbps

512

23,500 pps

23,500 pps

0.000%

423,000 pps

2.79 Gbps

1024

11,970 pps

11,970 pps

0.000%

215,460 pps

2.79 Gbps

1518

8,130 pps

8,130 pps

0.000%

146,340 pps

2.79 Gbps

*Test was not conclusive in determining the BLN as the cause of the packet loss.

Analysis for Test 3A

The Netcom Smartbits test equipment was configured for the layer-3 packet loss test. Ten ports transmitted an aggregate 100 Mbps unidirectional data-stream through both routers (four ATM interfaces) with a test duration of 600 seconds. In order to mitigate a potential bottleneck at the Ethernet-to-ATM edge devices (3Com SuperStack 1000 switches), the SmartBits transmit and receive ports were distributed across four SuperStack 1000 switches. Testing was conducted with the router ATM interfaces configured as UNI 3.0 and UNI 3.1 interfaces. The test results were almost identical regardless of the UNI specification used.

During each packet-loss test, CPU utilization on each of the ARE cards was displayed and recorded. Each measurement reflects the CPU utilization during a 30 second period as reported by the ARE cards. In general, router ATM cards which were transmitting cells displayed a higher CPU utilization than cards which were receiving ATM cells. Cells flowed from the Smartbits Test Set (10 each 10 Mbps streams) into the ATM network and entered Area 3 router on interface A4. The Area3 router then transmitted the data stream out of interface A3 back into the ATM cloud. Area2 router received the data stream on interface A3 and forwarded the traffic out of interface A4. The data stream then traversed the remaining portion of the ATM network into the receive ports on the SmartBits Test Set. The CPU utilization measurements are displayed in the following table:

 

 

 

Frame Size

Theoretical

Max. MFR

CPU Util. Area3, Interface A3 (Tx)

CPU Util. Area3, Interface A4 (Rx)

CPU Util. Area2, Interface A3 (Rx)

CPU Util. Area2, Interface A4 (Tx)

64

144,810 pps

55%

47%

67%

66%

128

84,446 pps

60%

64%

55%

59%

256

45,290 pps

62%

57%

46%

65%

512

23,500 pps

53%

42%

40%

66%

1024

11,970 pps

44%

27%

35%

60%

1518

8,130 pps

42%

30%

33%

54%

 

 

Network 21 Router Test 3B - Latency Test

Test Description and Purpose

This test is called the Latency Test. This is a part of the test suite "Router Test 3 - Measurement". This test satisfies Item 15 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to determine the delay introduced by sending a frame through a router from one ELAN to another.

This test will place a high volume of traffic from one ELAN to another and measure the time difference between sending the frame and receiving the frame on the remote ELAN.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully configured router test topology.

Hosts: None.

Testers: Netcom SmartBits

Test Conditions

This test uses the full router test network with routers and testers. No other devices are needed. The testers will flood one ELAN with inter-ELAN traffic and measure the latency at the remote ELAN.

Process

    1. Connect the SmartBits port 1 to ELAN Test5 and port 21 to ELAN Test9.

    2. Execute SmartBits VAST Latency by Stream Test with MOL of 64 Byte frames at 10 Mbps. Run test with at least 100,000 frames.

    3. Record Latency between ELANs Test5 and Test9.

Items Under Test

Items Under Test:

    1. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch

    2. CoreBuilder Switch

    3. ELAN Services

    4. Network Management System

    5. Standards and Protocols

    6. Physical Media

    7. IP Hosts

    8. Test Equipment

Expected Results

The router should not introduce more than 2ms per hop.

Recorded Results

Offered Frames

Latency

Units

2,976,200

984.3

us / ms / s

 

 

Analysis for Test 3B

The latency measured through 2 router hops was 984.3 microseconds. Latency measurements through the 3Com ATM switches without the routers in-line resulted in a 163 microsecond delay.

 

 

Network 21 Router Test 4A - ATM Standards

Test Description and Purpose

This test is called the ATM Standards test. This test satisfies Item 13 of the Test Spec Matrix.

It is to be performed during the Pilot Test.

The purpose of this test is to observe compliance to ATM Forum LANE, UNI, and TM 4.0 standards between routers and 3Com ATM devices.

This test will observe signaling between routers and 3Com ATM CoreBuilders for LANE and TM4.0.

Test Configuration

Test Topology

Network Topology: This test will use the fully deployed, fully configured Pilot Test topology. An ATM analyzer (Sniffer) will be inserted between a router and a CoreBuilder for the test.

Hosts: No hosts are required for this test.

Testers: Sufficient traffic generation equipment to provide an actual total offered load in excess of 135 Mbps (Netcom SmartBits with ten channels of Ethernet and a Fast Ethernet Sniffer). This equipment will be connected to two SuperStack Switches for the test.

Test Conditions

This test uses the full router test network with routers and testers. An ATM Sniffer will be installed between a router and a 3Com CoreBuilder switch and used to verify conformance with ATM standards and insure that network devices will perform initialization and traffic management functions without signaling errors. Load will be applied and flow control observed.

Process

    1. Install ATM Sniffer in line in ATM link r3a3 between ADF 3 Router and ADF 3.

    2. Power off ADF 3, wait 10 seconds, power on and observe link initialization without errors.

    3. Telnet to IDF 312 and delete ELAN Test1 ports from the configuration.. Add ELAN Test1 ports back to IDF 312 and observe initialization without errors.

    4. Connect the SmartBits ports 1-10 to ELAN Test5 and ports 21-30 to ELAN Test9.

    5. Using SmartBits, apply load in excess of 135 Mbps. With the Sniffer, monitor flow control, and note any errors.

Items Under Test

Items Under Test:

    1. Routers

    2. CoreBuilder Switches

 

Items NOT Under Test and NOT Influencing Results:

    1. Test Equipment

    2. Network Management System

    3. Standards and Protocols

    4. Media

Expected Results

The network devices should be able to perform initialization and traffic management functions without signaling errors.

Recorded Results

Test

Errors Detected

Pass / Fail

Initialization of ADF 3

No

Pass (UNI 3.0 & 3.1)

Initialization of ELAN Test1 ports

No

Pass (UNI 3.0 & 3.1)

Flow control during load

No

Pass (UNI 3.0 & 3.1)

 

Analysis for Test 4A

No errors were observed on the ATM sniffer during testing with the router interfaces configured for UNI versions 3.0 and 3.1.

 

 

Network 21 Router Test 4B - IETF and Ethernet Standards

Test Description and Purpose

This test is called the IETF and Ethernet Standards test. This is a part of the test suite "Router Test 4, Standards". This test satisfies Items 13, 17, 21 and 23-26 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to verify the routers under test meet IETF and Ethernet Standards as outlined in the Router RFP and Network 21 Technical Requirements document.

This test will determine the routers' ability to conform to industry standards by using a variety of configurations and utilities to test protocols defined as requiring full support or non-hindrance. Full testing of each defined protocol is not attempted, however captures of protocol behavior with network analyzers will be analyzed to insure no anomalies indicating non-conformance occur.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed Pilot Test topology. Test configurations will be altered during the test as necessary to verify items requiring special configurations.

Hosts: At minimum two workstations or PCs with the following application support: FTP, TFTP, Ping, Trace-route, DHCP and Telnet. Additional PCs will speed testing but are not required to complete the tests.

Testers: Network Management workstation, network analyzers with ATM and Ethernet interfaces.

Test Conditions

This test uses the full router test network with routers and testers. Router configurations will be modified prior to testing to remove routing protocols.

Process

    1. Connect ATM Sniffer to link r3a3 and monitor for duration of testing.

    2. Connect Ethernet Sniffer to the Ethernet link on the Area 3 router and monitor for duration of testing.

    3. Verify primary IP scheme is correctly configured on routers.

    4. Verify SNMP is correctly configured on routers.

    5. Remove OSPF routing from both routers.

    6. Configure static route from ELAN Test1 to ELAN Test3 on both routers.

    7. Display both routers' IP route table.

    8. Ping test hosts on ELAN Test1 and ELAN Test3 in both directions.

    9. Remove static route from both routers.

    10. Display routers' IP route table.

    11. Configure RIPv2 for ELANs Test1, Test3, Test15, and admin.

    12. Display routers' IP route table.

    13. Ping test workstations on ELAN Test1 and ELAN Test3 from each other.

    14. Telnet to routers from host on ELAN Test1.

    15. Configure default route to ELAN Test7.

    16. Display routers' IP route table.

    17. Ping workstation in ELAN Test7 from workstation in ELAN Test1.

    18. Perform trace-route from workstation in ELAN Test1 to workstation in ELAN Test7.

    19. Remove default route configuration.

    20. Display routers' IP route table.

    21. Remove RIPv2 routing configuration.

    22. Display routers' IP route table.

    23. Change IP addresses as follows for the interfaces listed:

    1. Configure BGP routing for ELANs Test1, Test3, and Test15. Use the following netmasks:

    1. Display routers' IP route table.

    2. Ping test workstations on ELAN Test1 and ELAN Test3 from each other.

    3. Remove BGP configurations.

    4. Display routers' IP route table.

    5. Restore original IP addresses to ELANs Test3 and Test15.

    6. Restore OSPF primary-only configuration.

    7. Display routers' IP route table.

    8. Ping all interfaces and network devices from the Network Management Station.

    9. FTP test files between workstations on ELANs Test1 and Test9.

    10. TFTP test files between workstations on ELANs Test1 and Test9.

    11. Configure DHCP server and connect to ELAN Test9. Configure DHCP forwarding for ELAN Test1.

    12. Configure a test host on ELAN Test1 to use DHCP and reboot. Verify DHCP address obtained by test host.

    13. Remove DHCP forwarding configuration.

    14. Reboot the test host on ELAN Test1. Verify that no DHCP information is received by the test host.

    15. Restore OSPF configuration. Configure Area 2 router, interface ATM 1, subinterfaces 1-7 and Area 3 router, interface ATM 1, subinterfaces 8-14. These interfaces should be configured as backup links to each subnet.

Items Under Test

Items Under Test:

    1. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch

    2. CoreBuilder Switch

    3. ELAN Services

    4. Network Management System

    5. Standards and Protocols

    6. Physical Media

    7. IP Hosts

    8. Test Equipment

Expected Results

The routers will not display any behavior that indicates protocols under test are hindered or not supported as per the Router RFP and Technical Requirements document.

Recorded Results

Process

Test Parameter

Results

1

No observed protocol violations on ATM Sniffer during testing

Yes

2

No observed protocol violations on Ethernet analyzer during testing

Yes

3

Primary IP scheme supports VLSM

Yes

4

SNMP support is available

Yes

5

Hostname support is available (DNS implied)

Yes

6

Routers support static route configurations

Yes

7

Routers support display of routing table

Yes

8

Static routes appear in routing table

Yes

9

Ping test successful (confirms router support of static routes and forwarding ICMP and UDP packets)

Yes

11

Static route removed from routing table

Yes

12

RIP configurable on routers

Yes

13

Routing tables reflect RIP routes

Yes

13

RIP updates occurring on schedule

Yes

13

Other RIP parameters within protocol guidelines

Yes

14

Ping test successful (confirms RIP routing and packet forwarding)

Yes

15

Telnet successful (confirms telnet support)

Yes

16

Default route configurable by routers

Yes

17

Default route appears in route tables

Yes

18

Ping test successful (confirms proper operation of default route)

Yes

19

Trace-route successful

Yes

21

Default route removed from routing tables

Yes

23

RIP routes removed from routing tables

Yes

25

BGP configurable on routers

Yes

26

BGP routes appear in routing tables

Yes

26

BGP parameters within protocol specifications

Yes

27

Ping test successful (confirms support of BGP routing)

Yes

29

BGP routers removed from routing tables

Yes

31

OSPF configurable on routers

Yes

32

OSPF routes appear in routing tables

Yes

32

Other OSPF parameters within protocol specifications

Yes

33

Ping test successful (confirms direct support of ICMP and OSPF)

Yes

34

Files transferred successfully using FTP

Yes

35

Files transferred successfully using TFTP

Yes

36

DHCP configured successfully

Yes

37

Workstation receives DHCP information from DHCP server

Yes

39

Workstation does not receive DHCP information from DHCP server

Yes

 

Analysis for Test 4B

All test procedures were completed successfully.

 

 

Network 21 Routing Test 5A - Lane Resiliency and ATM Re-routing Test

Test Description and Purpose

This test is called the LANE Resiliency and ATM Re-routing Test. This is a part of the test suite "Router Test 5 -- Functionality". This test satisfies Items 5 and 12 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to Demonstrate the router can use backup LANE services and ATM re-routing capabilities to provide fault tolerant operation.

This test will transmit Pings from one ELAN to another; the primary LANE server is disconnected and another set of Pings is transmitted from the same source but to a different ELAN to be certain LANE services are necessary. A second test will transmit Pings from one ELAN to another and have the ATM link disconnected to verify that the router can use alternate ATM routes.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully configured router test network.

Hosts: Three IP hosts are needed for Pings.

Testers: No testers are necessary unless problems are encountered.

Test Conditions

The full router test network is augmented with three hosts, one each in ELANs Test9 in Area 3, and ELANs Test3 and Test6 in Area 2. Pings are used for traffic.

Process

    1. Connect one host to ELANs Test9, Test3 and Test6.

    2. Begin continuous Ping from the host in ELAN Test9 to the host in ELAN Test3. Leave this running for the duration of the test.

    3. Power off ADF 2.

    4. Ping from the test host in ELAN Test9 to the test host in ELAN Test6. Verify Ping function from ELAN Test9 to ELAN Test6.

    5. Power on ADF 2

    6. Ping from the test host in ELAN Test9 to the test host in ELAN Test6. Verify Ping function from ELAN Test9 to ELAN Test6.

    7. Disconnect link a2a3 in Area 3.

    8. Ping from the test host in ELAN Test9 to the test host in ELAN Test6. Verify Ping function from ELAN Test9 to ELAN Test6.

    9. Restore link a2a3 to operational status.

    10. Ping from the test host in ELAN Test9 to the test host in ELAN Test6. Verify Ping function from ELAN Test9 to ELAN Test6.

 

Items Under Test

Items Under Test:

    1. Standards and Protocols

    2. Router

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch

    2. CoreBuilder Switch

    3. ELAN Services

    4. Network Management System

    5. Physical Media

    6. IP Hosts

    7. Test Equipment

Expected Results

After disconnecting the primary LANE server, the Ping between ELANs Test9 and Test6 will initiate without incident. When disconnecting link a2a3, the Ping between ELANs Test9 and Test3 will continue after a possible short interruption.

Recorded Results

Test Parameter

Result

Ping Startup With Secondary LANE

Yes

Ping Continuity With Link Down

Yes

 

Analysis for Test 5A

LANE resiliency was tested by failing the primary LANE server for an ELAN Test9. Upon failure of the primary LANE server, connectivity to the ELAN Test9 host was interrupted for six to eight seconds during failover to the secondary LANE server. After the secondary LANE server was functional, sessions between test hosts functioned normally. The failure of link r2a3, resulted in an outage to the ELAN Test3 host lasting for approximately 60 seconds while E-IISP convergence took place and new control and data direct VC's were established for the ELAN Test3 LEC's and the router backbone LEC on the Area 2 router. Previous testing with 3Com switches (no routers) resulted in E-IISP convergence times of approximately 50 seconds.

 

 

Network 21 Router Test 5B - Failure Propagation Test

Test Description and Purpose

This test is called the Failure Propagation Test. This is a part of the test suite "Router Test 5 -- Functionality". This test satisfies Item 19 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to demonstrate that failures do not propagate upstream.

This test will place IP hosts in each Area to Ping neighbors. Network Devices will be disabled to show that they do not affect upstream neighbors and leave the remaining network functional.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully functional router test topology. Three Ping hosts are used to demonstrate traffic functionality.

Hosts: Four IP hosts. One on ELAN Test9 in Area 3, one on ELAN Test7 in Area 3, one on ELAN Test6 in Area 2, and one on ELAN Test7 in Area 6.

Testers: No testers are necessary unless problems are encountered.

Test Conditions

Hosts are sending Pings to their neighbors at one-second intervals. Various Devices are disabled to demonstrate the effect.

Process

    1. Connect three hosts on ELANs Test9, Test6, and Test7 in Areas 3, 2, and 6 respectively.

    2. Initiate continuous Pings from each host to their neighbor hosts.

    3. Disconnect switch IDF 221 in Area 2. Record which hosts are returning responses to the Pings. Restore switch IDF 221 to operational status.

    4. Disconnect switch BDF 22 in Area 2. Record which hosts are returning responses to the Pings. Restore switch BDF 22 to operational status.

    5. Disconnect links a2a3 and a2a6 in Area 2. Record which hosts are returning responses to the Pings.. Restore links a2a3 and a2a6 to operational status.

Items Under Test

Items Under Test:

    1. SuperStack Switch

    2. CoreBuilder Switch

    3. Router

 

Items NOT Under Test and NOT Influencing Results:

    1. ELAN Services

    2. Network Management System

    3. Standards and Protocols

    4. Physical Media

    5. IP Hosts

    6. Test Equipment

Expected Results

When devices are disconnected, they will not have an upstream affect on their neighbors.

Recorded Results

IDF 221 Disconnected:

Source

Destination

Outage Prediction

Outage?

Area 2

Area 3

Y

Yes

Area 2

Area 6/Test7

Y

Yes

Area 2

Area 6/Test8

Y

Yes

Area 3

Area 2

Y

Yes

Area 3

Area 6/Test7

N

No

Area 3

Area 6/Test8

N

No

Area 6/Test7

Area 2

Y

Yes

Area 6/Test7

Area 3

N

No

Area 6/Test7

Area 6/Test9

N

No

Area 6/Test8

Area 2

Y

Yes

Area 6/Test8

Area 3

N

No

Area 6/Test8

Area 6/Test7

N

No

 

BDF 22 Disconnected:

Source

Destination

Outage Prediction

Outage?

Area 2

Area 3

Y

Yes

Area 2

Area 6/Test7

Y

Yes

Area 2

Area 6/Test8

Y

Yes

Area 3

Area 2

Y

Yes

Area 3

Area 6/Test7

N

No

Area 3

Area 6/Test8

N

No

Area 6/Test7

Area 2

Y

Yes

Area 6/Test7

Area 3

N

No

Area 6/Test7

Area 6/Test9

N

No

Area 6/Test8

Area 2

Y

Yes

Area 6/Test8

Area 3

N

No

Area 6/Test8

Area 6/Test7

N

No

 

ADF 2 Disconnected

Source

Destination

Outage Prediction

Outage?

Area 2

Area 3

Y

Yes

Area 2

Area 6/Test7

Y

Yes

Area 2

Area 6/Test8

Y

Yes

Area 3

Area 2

Y

Yes

Area 3

Area 6/Test7

N

No

Area 3

Area 6/Test8

N

No

Area 6/Test7

Area 2

Y

Yes

Area 6/Test7

Area 3

N

No

Area 6/Test7

Area 6/Test9

N

No

Area 6/Test8

Area 2

Y

Yes

Area 6/Test8

Area 3

N

No

Area 6/Test8

Area 6/Test7

N

No

 

Analysis for Test 5B

Failure of IDF and BDF equipment resulted in loss of connectivity as predicted. No unexpected results were observed. Failure of fiber link a2a3 resulted in a temporary loss of connectivity lasting approximately 60 seconds while E-IISP convergence and LEC VC re-establishment took place. Failure of link a2a6 resulted in loss of connectivity as predicted. All connectivity was restored when the fiber links were re-connected.

 

Network 21 Router Test 5C - BOOTP/DHCP Forwarding Test

Test Description and Purpose

This test is called the BOOTP/DHCP Forwarding Test. This is a part of the test suite "Router Test 5 - Functionality". This test satisfies Item 22 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to verify that the router can forward a BOOTP/DHCP broadcast frame to a designated host.

This test will connect a Sniffer, DHCP client, and DHCP server to the network and try to boot the client.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully configured router test topology. A Sniffer, DHCP client, and DHCP server are connected to the network to test the router's forwarding capability of the request.

Hosts: One DHCP client on ELAN Test9 in Area 3 and a DHCP server on ELAN Test13 in Area 3.

Testers: A Sniffer is connected with the DHCP server on ELAN Test13.

Test Conditions

This test uses the router across ELANs Test9 and Test13 to test DHCP forwarding function of the router.

Process

    1. Connect the DHCP client to ELAN Test9 in IDF 312 and the DHCP server to ELAN Test13 in IDF 313. Attach Ethernet Sniffer to the same port as the DHCP client in ELAN Test9.

    2. Configure the Area 3 router ATM 1 interface, subinterface 4 to forward BOOTP/DHCP broadcasts to the DHCP server in ELAN Test13.

    3. Begin Sniffer capture.

    4. Start/Restart DHCP client.

    5. Verify that DHCP client obtains DHCP lease.

    6. Verify proper treatment of forwarded frame with Sniffer.

Items Under Test

Items Under Test:

    1. Standards and Protocols

    2. Router

 

Items NOT Under Test and NOT Influencing Results:

    1. SuperStack Switch

    2. CoreBuilder Switch

    3. ELAN Services

    4. Network Management System

    5. Physical Media

    6. IP Hosts

    7. Test Equipment

Expected Results

The DHCP client will properly obtain IP addressing from the DHCP server at boot-up.

Recorded Results

Test Parameter

Result

DHCP Client boots with DHCP

Yes

Sniffer shows proper function

Yes

 

Analysis for Test 5C

DHCP forwarding is supported by the Bay Networks routers.

 

 

Network 21 Router Test 5D - Port/Address Filtering and System Effect Test

Test Description and Purpose

This test is called Port/Address Filtering and System Effect Test. This is a part of the test suite "Router Test 5 - Functionality". This test satisfies Items 27 and 28 of the Test Spec Matrix.

It is to be performed during the Router Pilot.

The purpose of this test is to verify the address filtering capability of the router and show that the filter can be configured without system disruption.

This test will place hosts on the network and install a filter to disable an active pair of hosts.

Test Configuration

Test Topology

Network Topology: This test uses the fully deployed, fully configured router test topology. Three IP hosts are added for reachability tests.

Hosts: Three IP hosts on ELANs Test7, Test9, and Test12 all in Area 3.

Testers: Testers are not necessary unless problems are encountered.

Test Conditions

The full router test network is used with three hosts using Ping to verify connectivity to neighbors.

Process

    1. Connect IP hosts to ELANs Test7 on IDF 313, Test9 on IDF 312, and Test12 on IDF 311.
    2. Initiate continuous Pings from each host to their neighbor hosts.
    3. Verify connectivity to neighbors. Record IP addresses of test hosts.
    4. Install filter to disable frames between the IP address of the test host on ELAN Test7 and the IP address of the test host on ELAN Test9.
    5. Verify connectivity to neighbors.
    6. Remove filter.
    7. Install filter to disable any frames from the IP a