Broadcom 570x Ethernet Controller User's Guide

Broadcom 570x Ethernet Controller Users Guide
Broadcom 570x Ethernet Controller
Contents
Broadcom NetXtreme Gigabit Ethernet Controller
CHAPTER 1FEATURES.......5 Broadcom Advanced Server Program (BASP) Introduction.7 Limitations........7 1.Broadcom Advanced Server Program for Windows 2000....9 2.Software for Linux..........10 CHPATER 2WINDOWS 2000 DRIVER INSTALLATION....11 Installing the Adapter Software....11 Installing Broadcom Advanced Server Program......15 Uninstalling Broadcom Advanced Server Program.....16 Accessing the Adapter VLAN Configuration Interface....18 Save and Restore Configuration.20
CHAPTER 3LINUX DRIVER INSTALLATION...23 Installing Source RPM Package..23 Building Driver From TAR File..25 Kernel Source Tree Guidelines.......26 Network Installation......26 Unloading and Removing the Driver......26 Module Parameters...27 Driver Messages....32 Statistics..33 BASP Driver for Linux.......33
Braodcom 570x Ethernet Controller
BASP Limitations...34 Installing BASP......34 BASP Configuration for Red Hat Distribution....35 Installing BASP TAR Archive...35 Broadcom NICE Patches..44 BASP SNMP Agent for Linux...46 Installing the RPM Package.48 Known Problems....49 CHAPTER 5 TROUBLESHOOTING..51 Windows...51 Linux ....52 Testing Network Connectivity...53 Software Problems and Solutions...55
Broadcom NetXtreme 570x Gigabit Ethernet Controller
Chapter 1
Features

Gigabit Ethernet (IEEE 802.3-1999) Logical Link Control (IEEE 802.2) Flow Control (IEEE 802.3X) Standard Ethernet frame size (1518 bytes) MII/GMII and TBI (SERDES style) Transceiver interfaces Layer 2 Priority Encoding (802.1P) High-Speed On-Chip RISC Processors Adaptive interrupt frequency Programmable rule checking and frame classification Up to 16 classes of service (CoS) (4 if no external memory) Up to 16 Distribution Rings (4 if no external memory) GMI/MII Management Interface 16M external SSRAM address space Selectable memory protection for external (on-board) memory Statistics for SNMP MIB II, Ethernet like MIB, and Ethernet MIB (802.3z, Clause 30)
Four unique MAC unicast addresses Support for multicast addresses via 128 bits hashing hardware function Serial EEPROM Support for PXE available JTAG support PCI v2.2 32/64-Bit, 33/66 MHz Bus Interface (5700/01) PCI-X v1.0 64-bit 100 MHz Bus Interface (5701) PCI Power Management Interface (v1.1) PCI Hot Plug (IBM, Compaq, Dell, and Microsoft) ACPI and Wake-on-LAN Support 64 Bit BAR support 3.3 V/1.8 V CMOS with 5V tolerant I/Os
Broadcom Advanced Server Program (BASP) Introduction

BASP is a Broadcom intermediate software driver for Windows 2000, Linux, that provides load-balancing, fault-tolerance, and VLAN features. These features are provided by creating teams (virtual adapters) that consist of multiple NIC interfaces. A team can consist of one to eight NIC interfaces and each interface can be designated primary or standby*. All primary interfaces in a team will participate in Load-balancing operations by sending and receiving a portion of the total traffic. Standby interfaces will take over in the event that all primary interfaces have lost their links. VLANs can be added to a team to allow multiple VLANs with different VLAN IDs. A virtual adapter is created for each VLAN added. Load-balancing and fault-tolerance features will work with any third party's NIC adapters. VLANs only work with Broadcom NIC adapters.
NOTE
Standby can only be used in Smart Load-Balance mode (see below).
Limitations
Smart Load-Balance (SLB) is a protocol specific scheme and the level of support for IP, IPX, and other protocols are listed below.
Load-balancing IP IPX Other protocols Yes Yes* No
Fault-tolerance Yes Yes** Yes**
*Only outbound load-balancing for IPX (on NetWare only). **Only for Broadcom NICs. Smart Load-balance (SLB) mode works with all Ethernet switches without
configuring the switch ports to any special trunking mode. Only IP traffic will be load-balanced in both inbound and outbound directions. IPX traffic will be load-balanced in outbound direction only. Other protocol packets will be sent and received through one primary NIC only. Fault-tolerance for non-IP traffic is only supported using Broadcom NICs. The Generic Trunking mode requires the Ethernet switch to support some form of port trunking mode (e.g. Cisco's Gigabit EtherChannel or other switch vendor's link aggregation mode). This mode is protocol-independent and all traffic should be load-balanced and fault-tolerant.
NOTE
Broadcom recommends disabling the spanning tree protocol at the switch when using BASP. This will minimize the downtime due to spanning tree loop determination when failing over.
1. Broadcom Advanced Server Program for Windows 2000

The following options are supported under Windows 2000.
Failover and Load Balance
Adapter teaming for failover (heterogeneous support for released 3Com 10/100 server adapters, released Intel 10/100 server adapters, Intel Pro 1000/T server adapters, and Intel 82559 LAN on Motherboard (LOM) Smart Load Balance Generic Link Aggregation (GEC/FEC, Open trunk) Link aggregation (IEEE 802.3ad) - static implementation only
Virtual LAN (VLANs)
Up to 64 VLANs per adapter using IEEE 802.1Q-1988 tagging. VLANs only work with Broadcom NIC adapters.
2. Software for Linux
Limitations The driver has only been tested on Red Hat 7.1 and 7.2 Linux distributions. Packaging The driver has been released in two packaging formats: source RPM and compressed tar formats. Module Parameters Optional parameters for the driver can be supplied as command line arguments to the insmod command. Broadcom Advanced Server Program (BASP) BASP is a kernel module designed for the Linux 2.2 and 2.4 kernel that provides load-balancing, fault-tolerance, and VLAN features.
10
Chapter 2
Windows 2000 Driver Installation
NOTE
The Broadcom NetXtreme Gigabit Ethernet Adapter must be physically installed in your system prior to installing the driver software.
When the Windows 2000 system first boots up after installing a new hardware device, such as a Gigabit Ethernet Adapter, the system automatically detects the new hardware and prompts you to install the driver software for that device. A network device driver must be installed before the Gigabit Ethernet Adapter can be used with your Windows 2000 system.
Installing the Adapter Software

To install the adapter software for Windows 2000, do the following
NOTE
Before beginning this procedure, verify that the Windows 2000 system has been upgraded to the latest version with the latest service pack applied. 1. Start your Windows 2000 system and log in. You must have Network Administrator privileges to install the driver software. When you boot up the Windows 2000 system after installing the adapter card, a series of Found New Hardware Wizard windows are displayed.
11
2. In the Install Hardware Device Drivers window, click Search for a

suitable driver for my device (recommended), then click Next.
3.
In the Locate Driver Files window, select the CD-ROM drives or Specify a location check boxes.
4.
When prompted, insert the Broadcom CD-ROM into your systems CD-ROM drive, type the path to the driver, and select OK.
Example: e:\
12
Where "e:" is the designation of the CD-ROM drive on your system.
5.
In the Driver Files Search Results window, verify that the correct path to the driver software is shown, then click Next.
NOTE
If the Insert Disk window appears, do not insert the Windows 2000 disk:
13
Some early versions of Windows 2000 may display a spurious message requesting you to insert the Windows 2000 disk. If this occurs, leave the Broadcom CD-ROM in the drive and click OK. If asked to specify the location of the Windows 2000 CD-ROM instead, click Browse (dont use the text entry field) and locate the path to the Gigabit Ethernet drivers that was specified above. Once installation of the driver software is complete, you are ready to modify configuration parameters.
14
Installing Broadcom Advanced Server Program

To install the Broadcom Advanced Server Program Driver Properties (BASP), do the following:
NOTE
Before installing on Windows 2000 Advanced Server with Terminal Services, the command "change user /install" must be issued, otherwise an error will occur. 1. Insert the Broadcom CD into the computer's CD-ROM drive.
2. Browse the CD to this path: Windows2000\BcmServ, and

double-click on baspinst.exe. This will install the BASP software and display the following window.
15
Uninstalling Broadcom Advanced Server Program

To uninstall the Broadcom Advanced Server Program Driver Properties (BASP), do the following:
1. Select Start>Settings>Network and Dial-up Connections. The

following screen appears.
2.
When the Network and Dial-up Connections window opens, right-click on any network adapter. This displays the Local Area Connection Properties window.
16
3.
Highlight the Broadcom Advanced Server Program Driver, and select Uninstall. The following screen appears.
4.
Select Yes. The following screen appears.
5.
Select Yes to restart your computer. This completes the uninstall procedure.
17
Accessing the Adapter VLAN Configuration Interface

Use this procedure to access the adapter properties for VLAN Configuration:
1. Open the system Control Panel and double-click Network and

Dial-up Connections icon.
2.
Double-click any Local Area Connection icon and click BASP properties. The BASP driver properties window will display:
18
The BASP driver properties window lists the installed adapters and the VLANs configured for each Team. Each VLAN is identified with a unique identifier number and name that will only appear in this window. Interface components of the VLAN Configuration window are described in detail below:
VLAN List - This list displays all of the VLANs that have been configured. Control Buttons - There are two control buttons: Create VLAN, and Delete VLAN. These buttons are used for creating and deleting VLANs.
Adding a VLAN
You can define up to 64 VLANs per team. To configure a new VLAN, perform the following steps: 1. 2. 3. From the BASP window, select the team to which you want to add a VLAN. Enter a VLAN Name and VLAN ID, then click the Create VLAN button. When you are finished adding VLANs to this team, click OK. A new virtual adapter is created for each VLAN.
NOTE
To maintain optimum adapter performance, your system should have 64 MB of system memory for each eight VLANs created per adapter. When adding VLANs to a single adapter, a one-adapter team must be created.
Deleting a VLAN NOTE

Adapters that are part of a team inherit all the basic configuration properties of the team, including VLANs associated with the team. If you delete a team, any
19
VLANs configured for that team will also be deleted. To delete a configured VLAN, perform the following steps: 1. From the BASP window, select the VLAN you want to delete and click the Delete VLAN button. The selected VLAN will be deleted from the VLAN list window.
2. When you are finished deleting VLANs, click the OK button to accept
the changes.
Save and Restore Configuration

This feature is used to save the current configuration in case of a system crash. The restore feature allows the user to apply the previous configuration. Save and restore VLAN and Adapter configurations as shown below:
Save a VLAN Configuration

To save a configuration:
1. Click Save at the Broadcom Advanced Server Program Driver

Properties screen.
20
2.
At the Save Configuration screen, enter the path and filename of the configuration to be saved (e.g., C:\temp\teamconfig.bcg).
The resulting file will be a text file that can be viewed with a text editor as shown below. Note that only the team configuration information is shown.
21
Restore a VLAN Configuration

1. To restore a configuration, click Restore at the Broadcom Advanced Server Program Driver Properties screen.
2.
At the information screen, click Yes to erase the current configuration and replace it with a saved configuration. Note that if the current configuration has not been saved, it will be lost.
3.
At the Restore Configuration screen, enter the path and filename of the configuration to be restored in the "Restore the configuration file window" and click OK.
The saved configuration will be restored.
22
Chapter 3
Linux Driver Installation
This section describes the Linux driver for the Broadcom NetXtreme BCM5700 series 10/100/1000 Mbps Ethernet Network Controllers. The Linux driver is released in two packaging formats: source RPM and compressed tar formats. The file names for the two packages are bcm5700-<version>.src.rpm and bcm5700-<version>.tar.gz, respectively. Identical source files to build the driver are included in both packages. The tar file contains additional utilities such as patches and driver diskette images for network installation.
Installing Source RPM Package

1. Install the source RPM package: rpm -ivh bcm5700-<version>.src.rpm
NOTE
If installing the driver on SuSE 7.x distributions, refer to the Notes section below before continuing. 2. Change the directory to the RPM path and build the binary driver for your kernel: cd /usr/src/{redhat,OpenLinux,turbo,packages,rpm ..} rpm -bb SPECS/bcm5700.spec Note that the RPM path is different for different Linux distributions.
23
3. Install the newly built package (driver and man page):

rpm -ivh RPMS/i386/bcm5700-<version>.i386.rpm Note that the --force option is needed if installing on Red Hat 7.1, 7.2, and others that already contain an older version of the driver. The driver will be installed in the following paths:
2.2.x kernels: /lib/modules/<kernel_version>/net/bc m5700.o
2.4.x kernels: /lib/modules/<kernel_version>/kernel /drivers/net/bcm5700.o
2.4.x kernels with bcm5700 driver patched in (e.g. Red Hat 7.1, 7.2): /lib/modules/<kernel_version>/kernel /drivers/net/bcm/bcm5700.o or /lib/modules/<kernel_version>/kernel /drivers/addon/bcm5700/bcm5700.o
4.
Load the driver. insmod bcm5700
5.
To configure the network protocol and address, refer to Linux-specific documentation.
24
Building Driver From TAR File

1. Create a directory and extract the TAR files: tar xvzf bcm5700-<version>.tar.gz
NOTE
If installing the driver on SuSE 7.x distributions, refer to the Notes section below before continuing. 2. Build the driver bcm5700.o as a loadable module for the running kernel: cd src make 3. Test the driver by loading it: insmod bcm5700.o
4. Install the driver and man page: make install
NOTE
See the RPM instructions above for the location of the installed driver. 5. To configure network protocol and address, refer to Linux-specific documentations.
Notes
NOTE
If compiling the driver under SuSE's 7.x kernel and errors are reported, follow the general guidelines below to rebuild the kernel source tree.
25
Kernel Source Tree Guidelines

cd /usr/src/linux-<kernel_version>.SuSE cp /boot/vmlinuz.config .config cp /boot/vmlinuz.version.h include/linux/version.h cp /boot/vmlinuz.autoconf.h include/linux/autoconf.h make oldconfig make dep where <kernel_version> is the actual kernel version used in the SuSE distribution. Example: /usr/src/linux-2.4.4.SuSE
Network Installation
For network installations through NFS, FTP, or HTTP (using a network boot disk or PXE), a driver diskette that contains the bcm570x driver is needed for Red Hat 7.x. The driver diskette images for the most recent Red Hat versions are included. Boot drivers for other Linux versions can be compiled by modifying the Makefile and the make environment. Further information is available from Red Hat's website. To create the driver diskette, select the appropriate image file and do the following: dd if=dd.img of=/dev/fd0H1440.
Unloading and Removing the Driver

To unload the driver, use ifconfig to bring down all eth# interfaces opened by the driver, then do the following: rmmod bcm5700 If the driver was installed using rpm, do the following to remove it: rpm -e bcm5700
26
If the driver was installed using make install from the tar file, the driver bcm5700.o has to be manually deleted from the system. Refer to the section "Building Driver From TAR File" for the location of the installed driver.
Module Parameters
Optional parameters for the driver can be supplied as command line arguments to the insmod command. Typically, these parameters are set in the file /etc/modules.conf (see the man page for modules.conf). These parameters take the form <parameter>=value[,value,...] where the multiple values for the same parameter are for multiple NICs installed in the system.
NOTE
The default or other meaningful values will be used when invalid values are selected. Some combinations of parameter values may conflict and lead to failures. The driver cannot detect all such conflicting combinations. All the module parameters are listed below.
line_speed Selects the line speed of the link. This parameter is used together with full_duplex and auto_speed to select the speed and duplex operation of the link and the setting of autonegotiation. The valid values are: o
0 - Autonegotiate for highest speed supported by link partner (default) 10 - 10 Mbps 100 - 100 Mbps 1000 - 1000 Mbps
o o o
If line_speed is set to 10, 100, or 1000, the NIC will autonegotiate for the
27
selected speed (and selected duplexity) if auto_speed is set to 1. If auto_speed is set to 0, the selected speed and duplexity will be set without autonegotiation. Note that 1000 Mbps must be negotiated for copper twisted pair links.
auto_speed Enables or disables autonegotiation. The valid values are:
o o
0 - Autonegotiation disabled 1 - Autonegotiation enabled (default)
Note that this parameter is ignored and assumed 1 if line_speed is set to 0. full_duplex
Selects the duplexity of the link. This paramter is used together with line_speed to select the speed and duplexity of the link. Note that this parameter is ignored if line_speed is 0. The valid values are:
o o
0 - half duplex 1 - full duplex (default)
rx_flow_control Enables or disables receiving flow control (pause) frames. This parameter is used together with auto_flow_control. The valid values are: o o
0 - pause receive disabled (default) 1 - pause receive enabled if auto_flow_control is set to 0, or pause receive advertised if auto_flow_control is set to 1
tx_flow_control Enables or disables transmitting flow control (pause) frames. This parameter is used together with auto_flow_control. The valid values are: o o
0 - pause transmit disabled (default) 1 - pause transmit enabled if auto_flow_control is set to 0, or
28
pause transmit advertised if auto_flow_control is set to 1 auto_flow_control Enables or disables autonegotiation of flow control. This parameter is used together with rx_flow_control and tx_flow_control to determine the advertised flow control capability. The valid values are:
o o
0 - flow control autonegotiation disabled (default) 1 - flow control autonegotiation enabled with capability specified in rx_flow_control and tx_flow_control (only valid if line_speed is set to 0 or auto_speed is set to 1)
tx_checksum
Enables or disables hardware transmit TCP/UDP checksum. The valid values are:
o o
0 - checksum disabled 1 - checksum enabled (default)
rx_checksum Enables or disables hardware receive TCP/UDP checksum validation. The valid values are: o
0 - checksum disabled
o 1 - checksum enabled (default) scatter_gather Enables or disables scatter-gather and 64-bit DMA on x86. This option is only useful when running on TUX-enabled kernels or newer kernels with zero-copy TCP. The valid values are: o
0 - scatter-gather and 64-bit DMA on x86 disabled
o 1 - scatter-gather and 64-bit DMA on x86 enabled (default) tx_pkt_desc_cnt Configures the number of transmit descriptors. Default is 100. The valid range is from 1 to 600. Note that the driver may not be able to allocate the required
29
amount of memory if this parameter is set too high.
rx_std_desc_cnt
Configures the number of receive descriptors for frames up to 1528 bytes.Default is 200. The valid range is from 1 to 800. This parameter should not be set less than 80 on systems with high network traffic. Setting this parameter higher allows the NIC to buffer larger bursts of network traffic without dropping frames, especially on slower systems. Note that the driver may not be able to allocate the required amount of memory if this parameter is set too high.
rx_adaptive_coalesce
Enables or disables adaptive adjustments to the receive interrupt coalescing parameters. Enabling it allows the driver to dynamically adjust the receive coalescing parameters to achieve high throughput during heavy traffic and low latency during light traffic. The valid values are:
0 - disabled
o 1 - enabled (default) rx_coalesce_ticks Configures the number of 1 usec ticks before the NIC generates receive interrupt after receiving a frame. This parameter works in conjunction with the rx_max_coalesce_frames parameter. Interrupt will be generated when either of these thresholds is exceeded. 0 means this parameter is ignored and interrupt will be generated when the rx_max_coalesce_frames threshold is reached. The valid range is from 0 to 500, and default is 100. This parameter is not used and will be adjusted automatically if rx_adaptive_coalesce is set to 1.
rx_max_coalesce_frames
Configures the number of received frames before the NIC generates receive interrupt. The valid range is from 0 to 100, and default is 10. This parameter and rx_coalesce_ticks cannot be both 0, otherwise no receive interrupts will be generated. This parameter is not used and will be adjusted automatically if rx_adaptive_coalesce is set to 1.
tx_coalesce_ticks
30
Configures the number of 1 usec ticks before the NIC generates transmit interrupt after transmitting a frame. This parameter works in conjunction with the tx_max_coalesce_frames parameter. Interrupt will be generated when either of these thresholds is exceeded. 0 means this parameter is ignored and interrupt will be generated when the tx_max_coalesce_frames threshold is reached. The valid range is from 0 to 500, and default is 300.
tx_max_coalesce_frames
Configures the number of transmitted frames before the NIC generates transmit interrupt. The valid range is from 0 to 100, and default is 42. This parameter and tx_coalesce_ticks cannot be both 0, otherwise no transmit completion interrupt will be generated. This parameter should always be set lower than tx_pkt_desc_cnt.
stats_coalesce_ticks
Configures the number of 1 usec ticks between periodic statistics block DMAs. The valid range is from 0 to 3600000000, and default is 1000000 (1 sec.). Set to 0 to disable statistics updates. This parameter is not used and will be set to default if rx_adaptive_coalesce is set to 1.
enable_wol
Enables or disables magic packet Wake-On-LAN when the system is shutdown. Note that not all systems support Wake-On-LAN. The valid values are:
o o
0 magic packet Wake-On-LAN disabled (default) 1 magic packet Wake-On-LAN enabled
Driver Messages
The following are the most common sample messages that may be logged in the file /var/log/messages. Use dmesg -n <level> to control the level at which messages will appear on the console. Most systems are set to level 6 by default.
31
Broadcom Gigabit Ethernet Driver bcm5700 with Broadcom NIC Extension (NICE) ver. 2.2.4 (02/26/02) Driver signon eth#: Broadcom BCM5701 1000Base-T found at mem faff0000, IRQ 16, node addr 0010180402d8 eth#: Broadcom BCM5701 Integrated Copper transceiver found eth#: Scatter-gather ON, 64-bit DMA ON, Tx Checksum ON, Rx Checksum ON NIC detected bcm5700: eth# NIC Link is Up, 1000 Mbps full duplex Link up and speed indication bcm5700: eth# NIC Link is Down Link down indication
Statistics
Detailed statistics and configuration information can be viewed in the file /proc/net/nicinfo/eth#.info.
BASP Driver for Linux

BASP Overview
BASP is a kernel module designed for 2.4.x kernels that provides load-balancing, fault-tolerance, and VLAN features. These features are provided by creating teams that consist of multiple NIC interfaces. A team can consist of 1 to 8 NIC interfaces and each interface can be designated primary, or hot-standby (SLB team only). All primary NIC interfaces in a team will participate in Load-balancing operations by sending and receiving a portion of the total traffic. Hot-standby interfaces will take over in the event that all
32
primary interfaces have lost their links. VLANs can be added to a team to allow multiple VLANs with different VLAN IDs. A virtual device is created for each VLAN added. BASP supports Smart Load-balance (SLB), Generic trunking and IEEE 802.3ad Link Aggregation. In SLB and 802.3ad mode, all the NIC drivers must support Broadcom NIC Extension (NICE). In this release, several NIC drivers patched with NICE are included.
SLB mode works with all Ethernet switches without configuring the switch ports to any special trunking mode. Only IP traffic will be load-balanced in both inbound and outbound directions. Generic trunking mode does not require NICE and can work with any NIC, however, it requires the Ethernet switch to support the technology and be properly configured. This mode is protocol-independent and all traffic should be load-balanced and fault-tolerant. 802.3ad mode requires NICE drivers and Ethernet switches supporting IEEE 802.3ad Link Aggregation. This mode is protocol-independent and all traffic should be load-balanced and fault-tolerant. All the physical interfaces in the 802.3ad teams are defaulted to be LACP active. A 802.3ad team requires all the member NICs supporting NICE. All the member NICs, once in the 802.3ad team, ill be set with the same MAC address.
BASP also provides remote management through the SNMP protocol, and this package is installed separately (see "BASP SNMP Agent for Linux").
BASP Limitations
BASP supports Red Hat 7.1 and 7.2. The following installation procedures work with these distributions. BASP has also been tested on SuSE 7.2 and 7.3, Caldera 3.1, Turbo Linux 7.0, and Mandrake 8.1. Minor modification to the akefile may be required if problems are experienced when compiling BASP on other i386 Linux distributions.
33
BASP also supports Red Hat Linux 7.1 and 7.2 for IA-64. VLANs are only supported by Broadcom NetXtreme Gigabit Ethernet. As opposed to VLANs support in other platforms, e.g. Windows and Netware, VLANs are not supported by Alteon Acenic driver (acenic.c).
Installing BASP
For users of Redhat 7.1 and 7.2 (i386 and IA-64), follow instructions in "Installing BASP RPM Package" section. For users of other Linux i386 and IA-64 distribution, follow instructions in "Installing BASP TAR Archive" section.
Installing BASP RPM Package

1. To install the RPM source package, run % rpm -i basplnx-{version}.src.{arch}.rpm 2. Change directory to the RPM path and build the binary driver for the kernel % cd /usr/src/redhat % rpm -bb SPECS/basplnx.spec Note that the RPM path is different for different Linux distributions. 3. Install the newly built package % rpm -i RPMS/i386/basplnx-{version}.{arch}.rpm The driver and other required files will be automatically installed.
34
4. To load the driver % insmod basp

5. Refer to "BASP Configuration for Red Hat Distribution" to set up the teams.
Installing BASP TAR Archive

BASP for Linux is shipped in mixed forms, where the platform and kernel specific files are in source code, and the core file is in object form. Three packages are shipped in this release: two tar archives and two RPM packages. basplnx-{version}.i386.tgz is the tar archive for i386 platform, and basplnx-{version}.ia64.tgz is the tar archive for IA-64 platform. To uncompress and expand the tar archive, run % tar xvfz basplnx-{version}.{arch}.tgz The installation process involves the following steps:
1. To build kernel module, "basp.o", % make
NOTE
The Make process will automatically build the correct module for different kernel options, e.g. symbol versioning and SMP support. There is NO need to define -DMODVERSIONS in the Makefile. 2. To create device file and to copy files, % make install 3. To update the module reference, % depmod a
35
4.
To load the driver, % insmod basp
5.
Refer to "BASP Configuration and Startup for Other Linux Distribution" to set up the teams.
BASP Files
Makefile baspcfg bcmtype.h blf.c blf.h blfcore.h blfcore.o blfopt.h blfver.h nicext.h pal.c pal.h release.txt nice-2.2.16 nice-2.4.16 scripts scripts/basp scripts/baspteam scripts/baspif scripts/team-sample scripts/team-gec scripts/team-vlan basp.4 makefile precompiled configuration utility commonly use type header file BASP module entry points ioctl interface core interface precompiled core object automatically generated header file from Make version header file NICE header file platform abstraction implementation header for platform abstraction this file NICE enabled driver for 2.2 kernel NICE enabled driver for 2.4 kernel contains sample scripts init script, goes to /etc/rc.d/init.d start/stop script, goes to /etc/basp start/stop network, i/f, goes to /etc/basp sample script of SLB team with three NICs sample script of GEC team with three NICs sample script of SLB team with 2 VLANs man page
36
baspcfg.8
man page for baspcfg utility
BASP Configuration for Red Hat Distribution

NOTE
To avoid failover problems when using BASP, make sure that the spanning tree is disabled on the switch that the network adapter is connected.
NOTE
When adding 64 VLANs, the 64th VLAN must have a VLAN ID of 0 (63 VLANs are tagged and 1 VLAN is untagged).
The BASP distribution includes a utility program and several scripts for team configuration. Following steps for Red Hat Linux distributions only. Most of the steps are only required to be performed after the first time installation. Step 2 "Modify the configuration script" should be performed whenever there is any change to the team configuration. For users of other Linux distributions, follow instructions in the "baspcfg" section. The configuration process involves the following steps: 1. Copy a configuration script from the "/etc/basp/samples" directory to the "/etc/basp" directory. Note the configuration script must be prefixed with "team-". Modify the configuration script to: (a) change the team type (b) add/delete the physical network interfaces (c) add/delete the virtual network interfaces The syntax of the configuration script can be found below. Note that when configuring Teaming, at least one Primary Adapter is required.
2.
37
3.
Run netconf to configure the IP address of the virtual network interface. It is important to manually enter the interface name, i.e., sw0, when configuring the IP address. This is because the virtual network interface (sw0), is not shown in the physical interface list.
NOTE
Failure to enter the virtual interface name will result in missing "ifcfg-" script in /etc/sysconfig/network-scripts directory. This script is required to start the BASP team. 4. Manually start the team for the first time: % /etc/rc.d/init.d/basp start
NOTE
This step is only required for the first time installation. The team configuration will be automatically started on subsequent reboots. Note that if not all virtual network interfaces are configured with an IP address, the following error message will appear when starting the BASP team:
Adding Basp Team 1 (OK) Starting Basp Team 1 (Failed)

When this happens, repeat step (3) to configure IP address for all the virtual network interfaces.
NOTE
Forming multiple teams is possible by copying the sample files into "/etc/basp/team-<name>" and modifying this file as described in the sample file.
3-1 38
NOTE
To create more that one virtual interface (VLAN) for each team, refer to the respective description section in the sample files.
BASP Configuration and Startup for Other Linux Distribution

BASP Configuration (baspcfg) is a command line tool to configure the BASP teams, add/remove NICs, and add/remove virtual devices. This tool, may be used to configure your own startup script files. Please read your distribution-specific documentation for more information on your distributors startup procedures. Following is the usage of this tool: baspcfg v2.0.4 - Broadcom Advanced Server Program Configuration Utility Copyright (c) 2000-2001 Broadcom Corporation. All rights reserved.
usage: baspcfg <commands>
Commands
addteam <tid> <type> <tname> delteam <tid> addva <tid> <vlan_id> <vname> [macaddr] delva <tid> <vlan_id> bind <tid> <role> <device> unbind <tid> <device> show [tid] create a team delete a team add a virtual adapter to a team del a virtual adapter from a team bind a physical adapter to a team unbind a physical adapter from a team display team configurations
39
Where
tid type tname vlan_id vname macaddr role device An unique ID for each team, starting from 0 Team type: 0=SLB, 1=FEC/GEC, 2=802.3ad ASCII string of the team VLAN ID: from 1 to 4094, 0=untagged or no VLAN ASCII string of the virtual device MAC address (optional), e.g. 00:10:18:00:11:44 Role of the physical device: 0=primary, 1=hot-standby ASCII string of the physical device, e.g. eth0
The following sample startup script should be used to start the BASP after the first time installation and configuration, or in the subsequent reboots. #!/bin/bash # load basp module insmod basp # create new team baspcfg addteam 0 0 team-one # bind physical interfaces / two primary one backup baspcfg bind 0 0 eth0 baspcfg bind 0 0 eth1 baspcfg bind 0 1 eth2 # create the virtual interface baspcfg addva 0 0 sw0 # bind ip address to virtual interface and initialize ifconfig sw0 192.168.0.1 up
NOTE
40
Baspcfg command can only be executed in Super User mode only, configure with other login mode will yield "Permission Denied. Must be root to manage BASP" error message. When configuring Teaming, at lease one Primary Adapter is required.
BASP Startup Scripts for Red Hat distributions
basp This script is intended to be installed in /etc/rc.d/init.d directory. After copying the script, run "chkconfig --add basp". This script will be executed at runlevel 2, 3, 4 and 5. When "basp" is run, it will search the /etc/basp directory to list all the files with "team-" prefix, and then it will invoke "baspteam" script to add or delete the teams. It is normal that each "team-*" file in /etc/basp represents 1 team.
baspteam This script is called by "basp" script to add or delete a team. To install, create "/etc/basp" directory and copy this script over. To manually add a team: % baspteam team-sample add To delete a team: % baspteam team-sample del Note that "team-sample" is the configuration script.
team-sample
41
This script contains a SLB team configuration with 3 NICs: eth0, eth1 and eth2. The team name is "TeamSample". All 3 NICs are primary. One virtual interface is also created for this team and the name of the virtual interface is "sw0". "sw0" is the device that "ifconfig" should be run against to set up the IP address. VLANs are not enabled in this script. This script and "team-gec" are intended to be customized. Refer to the configuration scripts section for details. This script should be copiedto /etc/basp directory and retain the "team-" prefix.
team-gec This configuration script creates a GEC team with 3 network interfaces, eth0, eth1 and eth2. The team name is "TeamGEC". All 3 NICs are primary. One virtual interface is added to the team with the name "sw0" and VLANs are not enabled. This script and "team-sample" are intended to be customized. Refer to the configuration scripts section for details. This script should be copied to /etc/basp directory and retain the "team-" prefix.
BASP Configuration Scripts for Redhat Distributions

Both team-sample and team-gec are configuration scripts that follow the same syntax, as follows: TEAM_ID: TEAM_TYPE: this number uniquely identifies a team 0 = SLB, 1 = Generic Trunking/GEC/FEC, 2 = 802.3ad
42
TEAM_NAME: TEAM_PAx_NAME:
ascii name of the team ascii name of the physical interface x, where x can be 0 to 7 role of the physical interface x 0 = Primary, 1 = Hot-standby. This field must be 0 for Generic Trunking/GEC/FEC team. ascii name of the virtual interface x, where x can be 0 to 63 802.1Q VLAN ID of the virtual interface x. For untagged virtual interface, i.e., without VLAN enable, set it to 0. The valid VLAN ID can be 0 to 4094.
TEAM_PAx_ROLE:
TEAM_VAx_NAME:
TEAM_VAx_VLAN:
NOTE
Teaming scripts are intended for Red Hat distributions ONLY, use with other Linux distribution will causes ERROR.
Broadcom NICE Patches

Also included in this release are network device drivers patched with Broadcom NICE support. These drivers are originally taken from the Linux 2.4.16 kernel distribution. To install patched drivers:
1. Copy the Broadcom NICE header file, "nicext.h", to the

appropriateLinux kernel include directory, e.g. % cp /usr/src/nice-2.4.16/nicext.h /usr/src/linux/include/linux
43
2.
Rename the original network device driver under the Linux kernel source tree, "/usr/src/linux/drivers/net". Copy the patched drivers to the Linux kernel network driver sourcedirectory, i.e. "/usr/src/linux/drivers/net".
3.
4. Follow the kernel rebuild instructions to configure kernel support for

these drivers. % cd /usr/src/linux % make config 5. If the patched drivers are configured into the kernel, goto step (7). If the patched drivers are configured as modules, goto step (6).
6. In the case of supporting only the module version of these drivers, it

is possible to simply run the following to compile patched drivers and to install them into the proper module directory: % make modules % make modules_install 7. Rebuild the kernel to compile these patched drivers
% make clean % make dep % make

8. Either reboot the system or unload/load the patched modules. Run configuration scripts to test the patch.
Uninstalling the RPM Package

To uninstall RPM package, % rpm -e basplnx and to reboot the system,
44
% reboot
Removal of Physical Interface in Generic Trunking and 802.3ad Mode

In Generic Trunking and 802.3ad mode, all the physical and virtual interfaces belonging to a team have the same MAC address. This MAC address is the same address as that of the first physical interface bounded to the team. In the case that this first physical interface is removed dynamically from the team using "baspcfg" tool and bounded to the protocol directly, this could lead to a duplicate MAC address problem on the network. Note that if the removed physical interface does not participate in any traffic, there will not be any problem. To properly remove a physical interface, follow the steps listed below:
1. Backup the original team configuration script % cp /etc/basp/team-gec /etc/basp/backup-gec
NOTE
"team-gec" is the name of the configuration script.
NOTE
"backup-gec" is the name of the backup script. The name of the backup script must NOT be prefixed with "team-". 2. Modify the team configuration script to remove the physical interface
3. Stop the running team
45
% /etc/basp/baspif /etc/basp/backup-gec stop % /etc/basp/baspteam /etc/basp/backup-gec del 4. Restart the team

% /etc/basp/baspteam /etc/basp/team-gec add % /etc/basp/baspif /etc/basp/team-gec start
BASP SNMP Agent for Linux

This SNMP agent is designed to support the configuration and statistics information pertaining to the Broadcom BASP driver. The BASP SNMP agent is available in two packaging formats: TAR archive and RPM. Both packages include the exact same script and MIB files.
Installing the TAR Archive

To uncompress and expand the tar archive, run
% tar xvfz baspsnmp-{version}.tar

The installation process involves the following steps: 1. Copy the getBaspInfo and genBaspTraps script files into /usr/bin directory. Copy the BASP-Config-MIB.txt, BASP-Statistics-MIB.txt and Brcm-BSAPTrap-MIB.txt into the /usr/share/snmp/mibs directory.
2.
3. Locate the snmpd.conf file. It is normally located at: /etc/snmp or

/usr/lib/snmp or $HOME/.snmp and add the following lines to the snmpd.conf. pass .1.3.6.1.4.1.4413.1.2.1 /usr/bin/getBaspInfo pass .1.3.6.1.4.1.4413.1.2.2.1 /usr/bin/getBaspInfo
46
pass .1.3.6.1.4.1.4413.1.2.2.2 /usr/bin/getBaspInfo pass .1.3.6.1.4.1.4413.1.2.2.3 /usr/bin/getBaspInfo 4. Stop the snmpd daemon and restart it again. % /etc/init.d/snmpd stop % /etc/init.d/snmpd start
5. Run the genBaspTraps script to allow monitoring of the BASP trap

events: % genBaspTraps This script can be terminated by hitting Ctrl-C keys if BASP trap event monitoring is no longer needed.
6. The snmpget and snmpgetnext commands can be used to receive the BASP snmp objects such as: % snmpget localhost public BASP-Config-MIB::btTeamNumber % snmpgetnext localhost public BASP-Config-MIB::btTeamNumber
BASP SNMP objects are provided in the following text files:
o o o
BASP-Config-MIB.txt BASP-Statistics-MIB.txt Brcm-BSAPTrap-MIB.txt
Installing the RPM Package

Complete the following steps to install BASP SNMP agent from the RPM package. 1. To install the RPM package, run
47
% rpm -i baspsnmp-{version}.i386.rpm The BASP script and MIB files will be installed. The snmpd.conf configuration file will me modified to add support for the BASP SNMP agent.
2. Follow steps 4 - 6 in the "Installing the TAR Archive" section.
NOTE
The current RPM installation fails to append the additional directives needed to the snmpd.conf file to support Basp objects. Thus please follow the instruction (3) in the Install - TAR archive to modify the snmpd.conf file.
SNMP Files
genBaspTrap getBaspInfo BASP-Config-MIB.txt BASP-Statistics-MIB.txt Brcm-BSAPTrap-MIB.txt release.txt script monitoring the BASP trap events script to process SNMP get/getnext inquiries SNMP MIB file for BASP configuration objects SNMP MIB file for BASP statistics objects SNMP MIB file for BASP trap objects this file
Uninstalling the RPM package

To uninstall RPM package, run: % rpm -e baspsnmp-{version}.i386.rpm and to reboot the system, % reboot
Known Problems
1. The network configuration utility, Yast2, distributed with Suse Linux 7.x failed to save the IP address assigned to the virtual network
48
interface. Following error message will be seen when running the "basp" init script: Adding basp team-example OK Starting basp team-example FAILED To activate the virtual network interface, enter following command: % ifconfig sw0 172.16.10.10 up Consult ifconfig(8) for details in manually configuring a network interface. 2. 802.3ad team member links disconnect and reconnect continuously when connected to the HP2524 switch. This is a 3rd party issue. It is seen only when configuring an 802.3ad team with greater than 2 members on the server and connecting an HP2524 switch, with lacp enabled as passive or active. The HP switch will show an lacp channel being brought up successfully with only 2 members. All other member's links will disconnect and reconnect. This does not occur with a Cisco Catalyst 6500.
49
Chapter 5 Troubleshooting
Hardware Diagnostics
Loopback diagnostic tests are available for testing the adapter hardware under Windows. These tests provide access to the adapter's internal/external diagnostics, where packet information is transmitted across the physical link
Checking if Proper Drivers are Loaded

The following section describes how to check if the proper drivers are loaded for Windows and Linux.
Windows
The Broadcom Advanced Control Suite, Vital Sign screen allows you to view vital adapter information, network status, and network connectivity. Active adapters are listed.
50
1.
From the Vital Sign screen, select the Broadcom adapter to be tested.
2. Press Refresh to display the driver status of the selected adapter.
NOTE
Information for non-Broadcom adapters is less comprehensive than information listed for Broadcom adapters.
Linux
To verify that the bcm5700.o driver is loaded properly, run lsmod
51
If the driver is loaded, a line similar to the one below appears, where <size> is the size of the driver in bytes, and <n> is the number of adapters configured. Module BCM5700 Size <size> Used by <n>
Testing Network Connectivity

The following section describes how to test network connectivity for Windows and Linux.
Windows
Use the ping command to determine if network connectivity is working.
1. Select Run from the Windows Control Panel, this displays the Run
command window.
52
2.
Type cmd (Windows 2000 and Windows .NET) or type command (Windows 98) and click OK. Type ipconfig /all to display the command window.
3.
4.
Type ping <IP address> from the command line, then press Enter. This displays the network connectivity information.
53
Software Problems and Solutions

This section provides a list of known software problems and solutions for the operating systems below.
Windows 2000
The following table lists know problems and solutions using Windows 2000: Problem: Able to create Broadcom LAC connections icons having same name under network properties menu. Solution: In Windows 2000, each network adapter installed properly will have an icon in Network Properties window. The name of the icon is usually in the form of "Local Area Connection ##" and where ## is a number starting from 1. The names of icon can be changed by right clicking the icon and selecting rename. The name of the icon is only meaningful to the Network Properties window. Sometimes the system administrators will rename these icons to easily differentiate the network connections. When BASP is configured, BASP will create additional adapter icons and rename the icons. The
54
naming convention is to allow end users to quickly identify the adapter with corresponding team and VLAN. The names of the icons created by BASP therefore are not recommended to be changed.
Problem: Uninstalling the BASP software and user is prompted to reboot early
Solution: In Windows 2000, after configuring BASP team, the system may sometimes prompt user to reboot. This is because Plug and Play on W2k may fail to commit any change in the network protocol binding. Users can choose not to reboot and continue configuring intermediate driver without having any side effect. When user has finished all the configuration, it is required to reboot.
Problem: Cannot Solution: In Windows 2000, BASP creates additional enable VLAN network connections in "Network Connection and Dail-up" after it is disabled Window. Similar to physical network connections, these virtual connections can be disabled via the context menu. However, if these virtual connections are re-enabled, the system will report error as "connection failed!". This is known problem with Windows 2000. When this happens, reboot the system will enable the virtual connection again.
55
Problem: Able to configure Internet Connection Sharing (ICS) when there aren't any unassigned adapters.
Solution: Windows 2000 networking is shipped with the Internet Connection Sharing (ICS), which is designed to allow multiple computers accessing the Internet via a Windows 2000 system. To configure ICS, a user would need to select a network connection as "outside" connection, and another network connection as "inside" connection. Broadcom has observed that ICS allow any available network connection to be "outside" and "inside" connection, regardless of the BASP team configuration. Broadcom recommends that the user not select any network connection that is part of the BASP team to be "outside" and "inside." Solution: In configuring BASP team configuration, the actual changes of the configuration is not committed until the user click "OK" in "Network Properties" window. If a user chooses not click OK and instead goes back to the BASP team configuration window, all the previous uncommitted changes will be lost and user will need to reenter the configurations. As a workaround, the user should always click "OK" in "Network Properties" window after making changes. Solution: When a team is created, the TCP/IP properties for the adapters are unselected. The user can manually select and configure TCP/IP properties of the adapters and configure an IP address. This is a limitation of the Windows 2000 network installation paradigm, where this invalid configuration is still allowed.
Problem: Team configuration is not retained when user goes back to edit.
Problem: IP address is configurable on a member of the team.
56
Problem: When creating 64 VLANs, all virtual adapters show disconnected or one of the 64 VLANs show disabled.
Solution: The maximum VLAN configurations are 63 tagged and one untagged VLAN ID 0. If 64 tagged VLANs are created, they are disconnected. A reboot is required and only 63 tagged VLANs show links, while a 64th
Linux
The following table lists known problems and solutions using Linux: Linux-Basp Problem: When obtaining the IP for a SLB's virtual interface via DHCP, the IP-gets lost under heavy traffic. Problem: Make install fails on Turbo Linux 7.0 IA64. Solution: Always set a static IP for all the virtual interfaces in a SLB team.
Solution: The symbolic link under /lib/modules/<kernel-version>/build points to an empty directory. To successfully install BASP using `make install`, re-link /lib/modules/<kernel-version>/build to point to /usr/src/<kernel-version>/.
Linux-Core
57
Problem: Compiling the driver fails under SuSE's 7.x.
Solution: If compiling the driver under SuSE's 7.x distributions and errors are reported, follow the general guidelines below to rebuild the kernel source tree: cd /usr/src/linux-<kernel_version>.SuSE cp /boot/vmlinuz.config .config cp /boot/vmlinuz.version.h include/linux/version.h cp /boot/vmlinuz.autoconf.h include/linux/autoconf.h make oldconfig make dep where <kernel_version> is the actual kernel version used in the SuSE distribution. Example: /usr/src/linux-2.4.4.SuSE Now you will need to rebuild the bcm5700 module. You may need to `make clean` if you attempted to build the driver before the previous steps. make clean make make install
Problem: Zero copy performance is low on Red Hat 7.1.
Solution: Red Hat 7.1 loads the ipchains module by default. IPCHAINS is not compatible with Zero Copy. Remove the IPCHAINS module and disable IPCHAINS from the system run level. Example: rmmod ipchains chkconfig ipchains off
Broadcom Advanced Server Program (BASP)
58
The following table lists known BASP problems and solutions: Problem: 802.3ad team member links disconnect and reconnect continuously. Solution: This is a 3rd party issue. It is seen only when configuring an 802.3ad team with greater than 2 members on the server and connecting an HP2524 switch, with lacp enabled as passive or active. The HP switch will show an lacp channel being brought up successfully with only 2 members. All other member's links will disconnect and reconnect. This does not occur with a Cisco Catalyst 6500.
59

Broadcom 570x Ethernet Controller User's Guide

Uploaded by

Copyright:

Available Formats

Broadcom 570x Ethernet Controller User's Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Broadcom 570x Ethernet Controller User's Guide

Uploaded by

Copyright:

Available Formats

Broadcom 570x Ethernet Controller Users Guide

Broadcom 570x Ethernet Controller

Braodcom 570x Ethernet Controller

Broadcom 570x Ethernet Controller

Broadcom NetXtreme 570x Gigabit Ethernet Controller

Braodcom 570x Ethernet Controller

Broadcom 570x Ethernet Controller

Braodcom 570x Ethernet Controller

Broadcom Advanced Server Program (BASP) Introduction

Load-balancing IP IPX Other protocols Yes Yes* No

Fault-tolerance Yes Yes** Yes**

Broadcom 570x Ethernet Controller

Braodcom 570x Ethernet Controller

1. Broadcom Advanced Server Program for Windows 2000

Failover and Load Balance

Virtual LAN (VLANs)

Broadcom 570x Ethernet Controller

2. Software for Linux

Braodcom 570x Ethernet Controller

Installing the Adapter Software

Broadcom 570x Ethernet Controller

2. In the Install Hardware Device Drivers window, click Search for a

Braodcom 570x Ethernet Controller

Where "e:" is the designation of the CD-ROM drive on your system.

Broadcom 570x Ethernet Controller

Braodcom 570x Ethernet Controller

Installing Broadcom Advanced Server Program

2. Browse the CD to this path: Windows2000\BcmServ, and

Broadcom 570x Ethernet Controller

Uninstalling Broadcom Advanced Server Program

1. Select Start>Settings>Network and Dial-up Connections. The

Braodcom 570x Ethernet Controller

Select Yes. The following screen appears.

Broadcom 570x Ethernet Controller

Accessing the Adapter VLAN Configuration Interface

1. Open the system Control Panel and double-click Network and

Braodcom 570x Ethernet Controller

Deleting a VLAN NOTE

Broadcom 570x Ethernet Controller

Save and Restore Configuration

Save a VLAN Configuration

1. Click Save at the Broadcom Advanced Server Program Driver

Braodcom 570x Ethernet Controller

Broadcom 570x Ethernet Controller

Restore a VLAN Configuration

The saved configuration will be restored.

Braodcom 570x Ethernet Controller

Installing Source RPM Package

Broadcom 570x Ethernet Controller

3. Install the newly built package (driver and man page):

2.2.x kernels: /lib/modules/<kernel_version>/net/bc m5700.o

2.4.x kernels: /lib/modules/<kernel_version>/kernel /drivers/net/bcm5700.o

Load the driver. insmod bcm5700

To configure the network protocol and address, refer to Linux-specific documentation.

Braodcom 570x Ethernet Controller

Building Driver From TAR File

4. Install the driver and man page: make install

Broadcom 570x Ethernet Controller

Kernel Source Tree Guidelines

Fault-tolerance Yes Yes Yes