Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Brkarc 3465

Download as pdf or txt
Download as pdf or txt
You are on page 1of 95

Catalyst 6500 Architecture

BRKARC-3465

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1
Session Goal

To provide you with a


thorough understanding
of the Catalyst®
6500 switching architecture,
packet flow, forwarding
engine functions, and key
feature operations.

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
Agenda
Chassis and Power Supplies

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 3
Catalyst 6500 Chassis Architecture

 Modular chassis in a variety of form factors


3, 4, 6, 9, and 13-slot versions
 Enhanced (―E‖) chassis offer:
Higher system power capacity
80-Gbps per slot capacity
<200ms fabric switchover
 Classic switching bus traces/connectors
 Dual Crossbar fabric traces/connectors
6513 has single fabric traces
in slots 1-8
 Redundant power supplies
 Fan tray for system cooling
6509-V-E chassis offers
redundant fan trays and
air filtration

Note: All non-E Series chassis are End-of-Sale except


for the WS-C6513.
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 4
Catalyst 6513

Clock Clock EEPROM

SLOT 1 Single Channel


SLOT 2 Single Channel
SLOT 3 Single Channel
SLOT 4 Single Channel
SLOT 5 Single Channel
19 RU Fan Tray
SLOT 6 Single Channel
SLOT 7 Single Channel
SLOT 8 Single Channel
SLOT 9 Dual Channels
SLOT 10 Dual Channels
SLOT 11 Dual Channels
SLOT 12 Dual Channels
Note: Modules requiring dual fabric SLOT 13 Dual Channels
Channels (6816,all 67xx except
6724) and WiSM will not work in Switch Fabric Shared Bus
slots 1-8.
Power Supply Power Supply
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 5
Catalyst 6513-E

Clock Clock EEPROM

SLOT 1 Dual Channels


SLOT 2 Dual Channels

Fan TrayFan Tray (removed from rear)


SLOT 3 Dual Channels
SLOT 4 Dual Channels
SLOT 5 Dual Channels
19 RU
SLOT 6 Dual Channels
SLOT 7 Dual Channels
SLOT 8 Dual Channels
SLOT 9 Dual Channels
SLOT 10 Dual Channels
SLOT 11 Dual Channels
SLOT 12 Dual Channels
Note: Supervisor 2T required to SLOT 13 Dual Channels
enable dual channels in slots 1-8
Switch Fabric Shared Bus
Power Supply Power Supply
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 6
Power Supply Redundancy
The Catalyst 6500 Can Utilize Two Power Supplies to Work in Either
Combined or Redundant Mode

Redundant Mode Combined Mode


Catalyst 6500 Catalyst 6500

50% 50% 83% 83%

Power Supply 1 Power Supply 2 Power Supply 1 Power Supply 2


• Each power supply operates at ~50% capacity • Each power supply provides up to 83% of its capacity
• Neither supply operates at >60% or <40% capacity • The total system power available is 167% of the
capacity of a single supply
• If one fails, the second supply can power the system on
its own • If one fails, the second supply may not be able to power
the system on its own - this could result in devices or
• This is the default and recommended configuration for
linecards being shut down
the power supplies
• This is not the recommended mode for production
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 8
Agenda

Supervisor Engine and Switch


Fabric Architecture

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 9
Catalyst 6500 Supervisors
Supervisor 32: Some Facts
Supervisor 32 Quick Facts
NO Switch Fabric

Integrated Policy Feature Card 3 (PFC3B)


supporting hardware acceleration for select
features

Integrated Multilayer Switch Feature Card 2a


(MSFC2a) supporting two CPU’s for Layer 2 and
Layer 3 functionality

IPV4 and IPv6 unicast and multicast forwarding


support in hardware

All uplinks can be active in systems with redundant


Supervisors

(more information in the notes)


BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 10
Supervisor 32 Backplane Architecture

Classic BUS

L2 CAM

Layer 2 FWD Engine

FIB TCAM

FIB Table

MET
Replication QOS ACL
ASIC
Security ACL
Layer 3 Counters
FWD
SP SP Netflow TCAM
SP Engine
Flash DRAM Netflow Table
Port ASIC Netflow Stats
RP RP Adjacency
Flash DRAM RP
Adj Stats

Policy Feature Card


MSFC2A 8 x GE SFP 1 x 10 / (PFC3B)
100 /1000
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 11
Supervisor 32-10G Backplane Architecture

Classic BUS

L2 CAM

Layer 2 FWD Engine

MET
Replication FIB TCAM

ASIC FIB Table

QOS ACL

Security ACL
MUX Layer 3 Counters
FWD
SP SP Netflow TCAM
SP Engine
Flash DRAM Netflow Table
Port Port
Netflow Stats
ASIC ASIC
RP RP Adjacency
Flash DRAM RP
Adj Stats

Policy Feature Card


MSFC2A 10GE 10GE 1 x 10 / (PFC3B)
100 /1000
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 12
Catalyst 6500 Supervisors
Supervisor 720: Some Facts
Supervisor 720 Quick Facts
Integrated 720Gbps Switch Fabric

Integrated Policy Feature Card 3 (PFC3) supporting


hardware acceleration for select features

Integrated Multilayer Switch Feature Card 3


(MSFC3) supporting two CPUs for Layer 2 and
Layer 3 functionality

IPv6 unicast and multicast forwarding support in


hardware

Virtual Switching System (VSS) support with


Sup720-10G models

All uplinks can be active in systems with redundant


Supervisors

BRKARC-3465 (more information in the notes)


© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 13
Supervisor 720 Backplane Architecture

Classic BUS
Crossbar Fabric Channels
L2 CAM

Layer 2 FWD Engine

FIB TCAM
Switch Fabric
FIB Table

QOS ACL

Security ACL
MET
Fabric / Layer 3 Counters
Replication FWD
Netflow TCAM
ASIC Engine
Netflow Table
RP RP
Flash DRAM RP Port Port
Netflow Stats

SP SP ASIC ASIC Adjacency


Flash DRAM SP
Adj Stats

Policy Feature Card


MSFC3 1G 1G 1G (PFC3)

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 14
Supervisor 720-10G Backplane Architecture

Classic BUS
Crossbar Fabric Channels

FIB TCAM
L2 CAM
Fabric Switch Fabric FIB Table

ASIC QOS ACL

Security ACL
Layer
2/3 Counters
Fabric /
FWD Netflow TCAM
MET Replication Engine
Netflow Table
RP SP ASIC
Netflow Stats
RP SP Adjacency
Port Port
Flash Flash
ASIC ASIC Adj Stats
RP SP
DRAM DRAM Quad Port PHY Policy Feature Card
MSFC3 (PFC3)
10G 1G 1G 1G 10G

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 15
Catalyst 6500 Supervisors
Supervisor 2T: Some Facts
Supervisor 720 Quick Facts
Integrated 2-Tbps Switch Fabric

Integrated Policy Feature Card 4 (PFC4) supporting


hardware acceleration for select features

Integrated Multilayer Switch Feature Card 5


(MSFC5) supporting a single CPU for L2 and L3
functionality

Connectivity Management Processor (CMP) for


improved management capability

One external compact flash slot (power controlled


by IOS)

All uplinks can be active in systems with redundant


Supervisors

BRKARC-3465 (more information in the notes)


© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 16
Supervisor 2T Backplane Architecture

Classic BUS
Crossbar Fabric Channels

FIB TCAM
L2
NetFlow
Fabric Switch Fabric CAM CL1 FIB Table
CL2
TCAM TCAM
ASIC QOS ACL

Security ACL
Layer FIB
2/3 Counters
L3/4 Engine
Fabric / LIF FWD
Netflow TCAM ADJ
MET Replication Table
Engine
Netflow Table
ASIC RPF
Netflow Stats Table
CPU
Port Port Adjacency
LIF ACE
ASIC ASIC DB Adj Stats
L2 Engine Counter
DRAM Flash LIF
Quad Port PHY
Stats
L2 CAM (128K)
MSFC5 10G 10G Policy Feature Card (PFC4)
1G 1G 1G PFC4
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 17
Supervisor Chassis Requirements

Supervisor 32s Supervisor 720s Supervisor 2Ts

All E-Series All E-Series Only E-Series


Chassis
All non-E Series All non-E Series
E-Fans for E- E-Fans for E- E-Fans for E-
Series Series Series
Fan Trays
Fan2 for non-E Fan2 for non-E
Series Series

Power Supplies 2500W AC / DC or greater

3-Slot : 1 and 2
4-slot : 1 and 2
Supervisor
6-slot : 5 and 6
Slots
9-slot : 5 and 6
13-slot : 7 and 8

E-Fan cannot be used in non-E Series Chassis


Fan2 cannot be used in E-Series
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 18
Catalyst 6500 Supervisor 720
The 720Gb Switch Fabric

Switch Fabric

- Integrated 720Gbps Switch Fabric


- Provides backplane interconnects
between linecards
- Fabric Traces are distributed across
each linecard slot
- Each Fabric Trace can run at 8Gb/sec
OR 20Gb/sec

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 19
Catalyst 6500 Supervisor 2T
The 2Tbps Switch Fabric

Switch Fabric

- Integrated 2Tbps Switch Fabric


- 26 Channels to support the 6513-E
- Provides backplane interconnects
between linecards
- Fabric Traces are distributed across
each linecard slot
- Each Fabric Trace can run at 20Gb/sec
OR 40Gb/sec

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 20
Catalyst 6500 Supervisors
Switch Fabric
The Supervisor 720 and Supervisor 2T support a Switch Fabric which offers
each connected linecard a set of discrete communication paths into the
switch backplane…

Linecard Linecard
Slot #9 Slot #8

Linecard Linecard
Slot #1 Slot #7

Linecard Linecard
Slot #2 Slot #6

Linecard Linecard Supervisor


Data Flows Slot #3 Slot #4 Slot #5

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 21
Catalyst 6500 CLI Example
6509E#show platform hardware capacity fabric
Switch Fabric Resources
Bus utilization: current: 25%, peak was 75% at 19:28:31 UTC Mon Feb 2 2009
Fabric utilization: Ingress Egress
Module Chanl Speed rate peak rate peak
1 0 20G 10% 50% @13:49 06Jan09 20% 50% @13:49 06Jan09
1 1 20G 20% 50% @13:49 06Jan09 10% 50% @13:49 06Jan09
2 0 20G 0% 1% @20:30 13Jan09 0% 1% @20:46 06Jan09
2 1 20G 0% 1% @20:47 16Jan09 0% 1% @16:52 06Jan09
3 0 20G 20% 40% @13:49 06Jan09 0% 0% @13:49 06Jan09
6 0 20G 0% 1% @17:44 06Jan09 0% 1% @00:36 08Jan09
8 0 8G 0% 3% @16:33 09Feb09 50% 100% @13:49 06Jan09

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 22
Catalyst 6500 Multilayer Switch Feature Card
 MSFC Serves as Control Plane for 6500 MSFC3

 Supervisors 720 and 32 have Two CPU’s –


SP and RP
SP serves as L2 control plane
RP serves as L3 control plane

 Supervisor 2T has One CPU


Single CPU performs L2 and L3 functions

 CMP on MSFC5 provides CPU,


file system, and boot management
MSFC5

 Local Bootflash holds IOS images

 Config held in NVRAM

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 23
Catalyst 6500 Supervisor 2T
MSFC5: Connectivity Management Processor (CMP)
The Connectivity Management Processor (CMP) supports new
capabilities that will aid Network Administrators in managing the
system:

CPU Image Recovery


- TFTP boot of the system

CPU File Transfer


- Image on USB device or TFTP

Remote CPU Reset


- Hard or Soft reset

CPU Console Logging


- Record CPU console log for
troubleshooting

USB Support
- USB serial console access

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 24
Catalyst 6500 Policy Feature Card
PFC3  PFC Serves as Data Plane for 6500

 Two primary ASICs – L2 and L3

 TCAM’s used for high speed lookup into


Forwarding (FIB), ACL (Security and QoS)
and Netflow Tables

 PFC3 – 48Mpps Maximum Forwarding

 PFC4 – 60Mpps Maximum Forwarding

 Common features supported in hardware by


PFC4 PFC3 and PFC4 include:
IPv4 - IPv6 - MPLS - Multicast - Policing - Classification -
RACL - VACL - PACL - GRE - Tunneling - URPF -
Control Plane Policing - and more

 Features introduced by the PFC4 include:


Flexible NetFlow - ACL Dry Run - ACL Hitless Commit -
Cisco TrustSec – VPLS - Egress NetFlow - IPv6 uRPF -
Roles Based Access Control – 512K Multicast Routes –
Improved EtherChannel Hash – and more

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 26
Agenda

Module Architecture

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 29
Catalyst 6500 Classic Linecards
Dbus
Rbus

EoBC

Linecard
Port
ASIC

Dbus is path over which header and Rbus is path over which forwarding
data is forwarded to the supervisor for result is sent back to linecards
forwarding lookup
Ethernet Out of Band Channel
DBUS runs at 16Gbps provides control path for Supervisor
to control linecards
RBUS runs at 4Gbps
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 30
Catalyst 6500 CEF256 Linecards
Dbus 8Gb Fabric Channel to Switch Fabric
Rbus
EoBC

Fabric ASIC

Linecard
Dbus
Rbus

Replication Port Port Port Port


ASIC ASIC ASIC ASIC ASIC

CEF256 provides connection to BUS Local replication ASICfor multicast


and Switch Fabric replication and SPAN

Single Fabric Channel @ 8Gbps

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 31
Catalyst 6500 CEF720 Linecards
20Gbps Fabric 20Gbps Fabric
Channel Dbus Channel
Rbus
EoBC

Fabric and Centralized Fabric and


Replication Forwarding Card Replication

Linecard
ASIC ASIC

Port Port Port Port


ASIC ASIC ASIC ASIC

CEF720 has no local forwarding Packet header forwarded to Supervisor


over BUS for forwarding lookup
Uses CFC card as pathway to
Supervisor Data sent over fabric channel to
destination linecard
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 33
Catalyst 6500 dCEF720 Linecards
20Gbps Fabric 20Gbps Fabric
Channel Channel

Fabric and Distributed Fabric and


Replication Forwarding Card Replication

Linecard
ASIC L2 FWD ASIC
L3 FWD

Port Port Port Port


….. …..
ASIC ASIC ASIC ASIC

dCEF720 uses DFC for local DFC contains same hardware and logic
forwarding as PFC on Supervisor

Linecard has no connection to


BUS
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 34
Catalyst 6500 dCEF2T Linecards
40Gbps Fabric 40Gbps Fabric
Channel Channel

FABRIC INTERFACE

Distributed
Forwarding Card

Linecard
FIRE FIRE FIRE FIRE
ASIC ASIC L2 FWD ASIC ASIC
L3 FWD
POR POR POR POR POR POR POR POR
T T T T T T T T
ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC

CTS CTS CTS CTS CTS CTS CTS CTS


ASIC ASIC ASIC ASIC ASIC ASIC ASIC ASIC

dCEF2T uses DFC for local forwarding DFC contains same hardware and logic as PFC on Supervisor

Linecard has no connection to BUS CTS ASICs provide wire-rate encryption / decryption

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 35
Catalyst 6500 Linecards
Centralized Forwarding Cards (CFC)
The Centralized Forwarding Card (CFC) provides BUS connectivity for the
CEF720 linecards…

The CFC is
available on
CEF720 linecards
and provides the
connection to the
Classic BUS

CFC is used to
communicate with
the Supervisor
when centralized
forwarding is used

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 36
Catalyst 6500 Linecards
Distributed Forwarding Card 3 (DFC3)
The DFC3 is an option- it is used to provide local forwarding lookups for the
linecard to incrementally boost overall switch performance - if installed on a
CEF720 linecard, it takes the place of the CFC…

The DFC3 supports


forwarding rates up to
48Mpps Three different versions of
the DFC3 are supported…
The DFC3 also stores a local
copy of the forwarding DFC3A
tables, as well as Security DFC3B/DFC3BXL
and QoS ACL’s that are DFC3C/DFC3CXL
centrally defined
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 37
Catalyst 6500 Linecards
Distributed Forwarding Card 4 (DFC4)
The DFC4 is an option for CEF720 linecards - it is used to provide local forwarding
lookups for the linecard to incrementally boost overall switch performance - if installed
on a CEF720 linecard, it takes the place of the CFC…

The DFC4 supports forwarding


rates up to 60Mpps

The DFC4 also stores a local


copy of the forwarding tables, as
well as Security and QoS ACL’s
that are centrally defined

The DFC4 is located underneath


a protective cover that protects
the daughtercard from getting
damaged when the linecard is
Two different versions of the DFC
inserted or removed from a
are supported…
chassis
DFC4-A / AXL
The DFC4 WILL be field
DFC4-E / EXL
upgradable
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 38
Catalyst 6500 Linecards
DFC3/4 Interoperability with PFC3/4

 DFC3s work only with PFC3s, and DFC4s work only with PFC4s.

 When mixing DFCs and PFCs of different capabilities, the lower common
denominator is in effect:
Example 1 : A PFC3BXL on the Supervisor with a DFC3B on the module will result in the PFC3BXL
running in PFC3B mode.
Result : The larger FIB and NetFlow tables of the XL will not be used as they will need to be
programmed to match the smaller tables sizes of the non-XL.

Example 2: A PFC3C on the Supervisor with a DFC3B on the module will result in the PFC3C
running in PFC3B mode.
Result : The VSS capability of the PFC3C will be disabled when it runs in PFC3B mode since
PFC3B mode does not support VSS.

 Mixing of different PFCs in the same chassis is not supported.

 When inserting a module with a lower level DFC than the PFC on the
Supervisor, the system must be reloaded for the PFC to reprogram itself to the
lower mode.

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 39
Mixing Different Linecard Architectures
When utilizing Centralized Forwarding, the backplane will operate in one of
three modes—these modes are determined by the combination of linecards
installed in the chassis, from which module the traffic is sourced from and to
which module the traffic is destined.

Mode Description Illustration


• Between non fabric modules and between a non fabric
and a fabric enabled linecard
• Throughput – 15 Mpps (@ 64 byte frames) Data H Data H
FLOW THROUGH • Bandwidth – 16 Gbps of bandwidth shared throughou
• Data Bus frame size is variable; min of 4 cycles (64B
Data) on the DBus for every frame +1 wait cycle Bus

• Between fabric linecards when a non fabric linecard is


in the chassis. Data H D H
• Throughput – 15 Mpps (@ 64 byte frames); independent
of frame size for CEF256 and CEF720
TRUNCATED • Bandwidth – 16 G shared for classic; 8 G per CEF256;
20 G/channel CEF720 Bus
• Data Bus frame size is variable; min of 4 cycles (64
Bytes Data) on the Data Bus for every frame.

• When only ALL fabric enabled linecards in a chassis


• Throughput – 30 Mpps (@ any frame size) D H D H D H D H
• Bandwidth – 8 G CEF256; 20 G/channel CEF720
COMPACT • Data Bus frame size is constant (compact header); 2
cycles (32 B Data) on the DBus for every frame + no wait
cycle Bus
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 41
Agenda

Layer 2 Forwarding

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 42
Layer 2 Forwarding Logic

Frame received

Layer 2 Table SMAC lookup DMAC lookup Layer 2 Table

Learn New MAC? Router MAC? L3 forwarding


Yes Yes
Layer 2 Table No No

Update entry Known MAC? L2 forwarding


Yes
Layer 2 Table
No

L2 flooding
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 43
Catalyst 6500 Internals MAC Table
Layer 2 Forwarding on PFC Table Port
MAC
A 1
B 2
C 3
The PFC has an D 4
E 5
integrated CAM Table F 6
that supports 4096
rows * X pages =
MAC address space

16, 24,
or 32
PFC pages
4096
rows

MAC Table

PFC3A/B = 16 pages (64K


entries)
PFC3C = 24 pages (96K
entries)
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential
PFC4 = 32 pages (128K 44
Catalyst 6500 Internals CAM
Table
Layer 2 Forwarding on PFC MAC
A
Port
1
B 2
C 3
Frame D 4
16, 24, E 5
or 32 F 6
Pages

VLAN MAC PFC


Hash 0000.2222.7777 | 20
0000.1111.cccc | 10
4096
0000.dddd.a112 | 30
Rows
MAC Table Row 0000.bbbb.ac1c | 30

MAC Table
HIT!!!
1. Hash result identifies the starting Page and Row in MAC table
2. Lookup key (VLAN + MAC) compared to contents of indexed line on each page, sequentially
3. Destination lookup: Match returns destination interface(s), Miss results in Flood
4. Source lookup: Match updates age of matching entry, Miss installs new entry in table

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 45
Displaying the Layer 2 Table

6513E.SUP2T.SA.2#show mac address-table

Legend: * - primary entry


age - seconds since last seen; n/a - not available; S - secure entry;
R - router's gateway mac address entry; D - Duplicate mac address entry

Displaying entries from active supervisor:

vlan mac address type learn age ports


----+----+---------------+-------+-----+----------+-----------------------------
* 192 00d0.0053.bc00 dynamic Yes 5 Gi7/3
R 205 0024.c4dc.d740 static No - Router
R 20 0024.c4dc.d740 static No - Router
* 192 0014.5e31.4220 dynamic Yes 65 Gi7/3
* 60 00d0.2bfc.23f5 dynamic Yes 30 Gi5/14
* 192 00e0.1e5d.e9ff dynamic Yes 30 Gi7/3
<…>

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 46
EtherChannel Load Sharing

Combines multiple physical interfaces into ONE logical interface

EtherChannel Load Sharing Deterministic

PFC3 algorithm supports 8 results (3 bits)

PFC4 algorithm supports 256 results (8 bits)

Load Sharing is by flow and NOT per packet

EtherChannel can be configured for L2 and L3 interfaces

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 47
EtherChannel “Power-of-2” Ports
PFC3 Flow Distribution

Frame 1 2 3 4 5 6 7 8

EtherChannel Hash 3 bit result

E/Chan Link1 Link2 Link3 Link4 Link5 Link6 Link7 Link8


Bundle
2 Links 50% 50% -- -- -- -- -- --
3 Links 37.5% 37.5% 25% -- -- -- -- --
4 Links 25% 25% 25% 25% -- -- -- --
5 Links 25% 25% 25% 12.5% 12.5% -- -- --
6 Links 25% 25% 12.5% 12.5% 12.5% 12.5% -- --
7 Links 25% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% --
8 Links 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5%

Even Distribution for Flows is for those cases highlighted in RED


BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 48
EtherChannel “Power-of-2” Ports
PFC4 Flow Distribution

Frame 1 2 3 ……….. 256

EtherChannel Hash 8 bit result

E/Chan Link1 Link2 Link3 Link4 Link5 Link6 Link7 Link8


Bundle
2 Links 50% 50% -- -- -- -- -- --
3 Links 33.6% 33.2% 33.2% -- -- -- -- --
4 Links 25% 25% 25% 25% -- -- -- --
5 Links 20.4% 19.9% 19.9% 19.9% 19.9% -- -- --
6 Links 16.8% 16.8% 16.8% 16.8% 16.4% 16.4% -- --
7 Links 14.5% 14.5% 14.5% 14.5% 14% 14% 14% --
8 Links 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5% 12.5%

Even Distribution for Flows is for those cases highlighted in RED


BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 49
Agenda

IP Unicast Forwarding

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 52
Catalyst 6500 IP Unicast Forwarding
Note

This session covers IP Unicast forwarding.

There is a dedicated Breakout Session at Cisco Live for IP


Multicast Forwarding with the Catalyst 6500:

BRKARC-3322 Catalyst 6500 IP Multicast Architecture

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 53
Catalyst 6500 Interface Management

Supervisor 720 Supervisor 2T


Supervisor 2T

4K VLAN POOL 16K Bridge 128K Logical


Domains Interfaces
VLANs L3 Ports
L3 Ports
SVI Tunnels
VLAN 1…4K SVI Tunnels
CoPP Etc…
VLAN 1…4K CoPP Etc…
VLAN 1…4K

•VLANs used for both L2 bridging •Separate L2 bridging and L3 routing


and L3 routing
•Break the 4K VLAN barrier
•L3 interfaces internally consume
VLANs from the 4K VLAN pool •Allows VLAN reuse on a per port basis
•Massive scale of L3 interfaces

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 54
Catalyst 6500 PFC3/DFC3 Lookup Process

L3 Engine
Netflow TCAM 5 4 FIB TCAM &
SSRAM

Netflow Table 7 4 Security ACL


TCAM

Netflow Statistics 4 QoS ACL TCAM


8
Adjacency Statistics 6 Adjacency Table

3 8 IP Packet Parse

L2 Engine
2 IP Packet Parse
1 8 L2 MAC Table

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 55
Catalyst 6500 PFC4/DFC Lookup Process
Input Forwarding Engine Lookup
Architecturally, the PFC/DFC4 is almost the same as the PFC/DFC3

What changes is the Dual-Cycle Input (IFE) and Output (OFE) Processing

Here we perform the Input Forwarding Engine (IFE) pass...

IFE process:

Packet Header GV IF RP CL1


1.IF: Get Port and Ingress LIF QoS info

2.RP: Src FIB Lookup, Source QoS

3.CL1: Ingress ACL TCAM Lookup


PO CL2
4.CL2: Select Ingress Class and Policy

5.NF: Ingress NetFlow lookup

6.L3: Dst FIB Lookup, Dst QoS


RI PL L3 NF
7.PL: Apply Ingress Policing and Marking
L2 L3
Engine Engine
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 56
Catalyst 6500 PFC4/DFC Lookup Process
Output Forwarding Engine Lookup
Architecturally, the PFC/DFC4 is almost the same as the PFC/DFC3

What changes is the Dual-Cycle Input (IFE) and Output (OFE) Processing

Here we perform the Output Forwarding Engine (OFE) pass...

OFE process:

RBUS Result GV IF RP CL1 1.IF: Get Egress LIF QoS info

2.CL1: Egress ACL TCAM lookup

3.CL2: Select Egress Policy and Class


PO CL2 4.NF: Select NF Egress Policy and Class

5.PL: Apply Egress Policing and Marking

6.RI: Generate RBUS result


RI PL L3 NF

L2 L3
Engine Engine
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 57
Catalyst 6500 IP Unicast Forwarding
Layer 3 Forwarding on PFC
Routing Protocols receive routing updates from the
network... Control Plane (RP)
Holds routing tables in
Routing information Base
(RIB) from Static Routes
Routing Protocols and all running
OSPF, EIGRP, ISIS, BGP, etc
Routing Protocols

FIB (on PFC/DFC) Software CEF


FIB & ADJ tables are Takes RIB and builds a
used by EARL to Hardware CEF Forwarding Information
perform L3 lookups & Loads FIB into PFC Base (FIB) containing
forwarding & distributes to DFC’s IP/mask prefixes

Hardware Based CEF Process


1. FIB lookup based on destination prefix (longest-match)
2. FIB “Hit” returns Adjacency pointer
3. Adjacency contains Rewrite (next-hop) information
4. ACL, QoS & NetFlow lookups occur in parallel, and effect final result
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 58
Catalyst 6500 IP Unicast Forwarding
Layer 3 Forwarding on PFC
Located on the PFC are the “FIB” and “Adjacency Table”…
The FIB contains: 172.20.45.1
• L3 entries are arranged logically 10.1.1.100
from MOST to LEAST specific MASK (/32)
(based on /mask) … IF, MACs, MTU
• Overall FIB hardware shared by: 10.1.3.0
– IPv4 Unicast IF, MACs, MTU
10.1.2.0
– IPv4 Multicast MASK (/24) IF, MACs, MTU
– IPv6 Unicast …
– IPv6 Multicast IF, MACs, MTU
10.1.0.0
– MPLS
172.16.0.0
The Adjacency Table: MASK (/16)
• L2 “Re-Write” information and / or …
pointers for replication
0.0.0.0 Adjacency
• Hardware adjacency table also
shared among protocols MASK (/0) Table
FIB TCAM
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 59
Catalyst 6500 Internals
Layer 3 Forwarding on PFC
Assuming a lookup was performed for a packet with a destination
of 10.1.5.2 /24, then the following would occur…

1 Packet 172.20.45.1
10.1.1.100
MASK (/32)
IF, MACs, MTU
2 Key Gen …
10.1.3.0 IF, MACs, MTU
10.1.2.0
7
Load-Sharing
6 IF, MACs, MTU
3 Lookup Key MASK (/24) Hash
… IF, MACs, MTU
4
HIT! 10.1.0.0
172.16.0.0
5
MASK (/16)
… Adjacency Table
0.0.0.0
MASK (/0)

FIB TCAM
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 60
Supervisor FIB TCAM Resources
 IPv6 and IPv4 multicast require
2 entries
NON-XL PFCs XL PFCs
MPLS and IPv4 only one
IPv4, MPLS 192k 512k
 XL PFCs = 1M entries
IPv6, Multicast 32k 256k
 Non-XL PFCs = 256K entries
 By default TCAM is allocated as
seen in the table

SUP720-3BXL Example Changing default (requires Reboot!)

6509E#sh mls cef maximum-routes 6509E(config)#mls cef maximum-routes ?


FIB TCAM maximum routes : ip number of ip routes
======================= ip-multicast number of multicast routes
Current :- ipv6 number of ipv6 routes
------- mpls number of MPLS labels
IPv4 + MPLS - 512k (default)
IPv6 + IP Multicast - 256k (default)

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 62
Displaying IPv4 Forwarding Summary
6509E#show platform hardware capacity forwarding
<snip>
L3 Forwarding Resources
FIB TCAM usage: Total Used %Used
72 bits (IPv4, MPLS, EoM) 196608 28 1%
144 bits (IP mcast, IPv6) 32768 7 1%

detail: Protocol Used %Used


IPv4 28 1%
MPLS 0 0%
EoM 0 0%

IPv6 1 1%
IPv4 mcast 3 1%
IPv6 mcast 3 1%

Adjacency usage: Total Used %Used


1048576 171 1%
<snip>

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 63
Displaying Hardware IPv4 Prefix Entries
6509E#show platform hardware cef
Codes: decap - Decapsulation, + - Push Label
Index Prefix Adjacency
68 255.255.255.255/32 receive
75 10.10.1.1/32 receive
76 10.10.1.0/32 receive
77 10.10.1.255/32 receive
78 10.10.1.2/32 Gi1/1, 0030.f272.31fe
3200 224.0.0.0/24 receive
3201 10.10.1.0/24 glean
3202 10.100.0.0/24 Gi1/1, 0030.f272.31fe
3203 10.100.1.0/24 Gi1/1, 0030.f272.31fe
3204 10.100.2.0/24 Gi1/1, 0030.f272.31fe
3205 10.100.3.0/24 Gi1/1, 0030.f272.31fe
<…>

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 64
Finding the Longest-Match Prefix Entry
6509E#show platform hardware cef 171.1.1.0

Codes: decap - Decapsulation, + - Push Label


Index Prefix Adjacency

6500#show platform hardware cef lookup 171.1.1.0

Codes: decap - Decapsulation, + - Push Label


Index Prefix Adjacency
3531584 171.0.0.0/8 Vl192 ,00d0.0053.bc00

6500#show platform hardware cef ipv6 lookup FF00::

Codes: + - Push label


Index Prefix Adjacency
512 FF00::/8 glean

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 65
IPv4 CEF Load Sharing
 Up to 16* hardware load-sharing paths per prefix
 Use maximum-paths command in routing protocols
to control number of load-sharing paths 10.10.0.0/16
 IPv4 CEF load-sharing is per-IP flow via Rtr-A
via Rtr-B
 Per-packet load-balancing not supported
 Load-sharing based on Source and Destination
IP addresses by default
―Unique ID‖ in PFC3 and PFC4 prevents polarization
A B
 Configuration option supports inclusion of
L4 ports in the hash
mls ip cef load-sharing full 10.10.0.0/16
 Unique ID not included in hash in ―full‖ mode

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 66
Load-Sharing Prefixes and Paths
6509E#show platform hardware cef lookup 10.100.20.1

Codes: decap - Decapsulation, + - Push Label


Index Prefix Adjacency
3222 10.100.20.0/24 Gi1/1, 0030.f272.31fe
Gi1/2, 0008.7ca8.484c
Gi2/1, 000e.382d.0b90
Gi2/2, 000d.6550.a8ea

6500#show platform hardware cef exact-route 10.77.17.8 10.100.20.199

Interface: Gi1/1, Next Hop: 10.10.1.2, Vlan: 1019, Destination Mac: 0030.f272.31fe

6500#show platform hardware cef exact-route 10.44.91.111 10.100.20.199

Interface: Gi2/2, Next Hop: 10.40.1.2, Vlan: 1018, Destination Mac: 000d.6550.a8ea

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 67
Agenda

NetFlow

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 68
Catalyst 6500 NetFlow
Netflow is a process designed to collect information about traffic flows that
pass through the switch - Netflow collection of flow records is a hardware
process while the exporting of flow records to an external collector is a control
plane process…
Netflow
Collection
Server
Netflow

Data Flow
Exported Netflow
Record
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 69
Catalyst 6500 NetFlow
PFC3 Flow Masks
The Catalyst 6500 supports the following flow masks - these are used to
identify which pieces of information in the header will be used as input into
generating a key for flow lookups…

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 70
Catalyst 6500 NetFlow
TCAM Lookup on PFC3
Key Flow Data
Packet Key Flow Data
1 Key Flow Data
Key 6 Flow DataHIT!
2 Mask Compare Statistics
Key Flow Data
7
FlowKey
Flow Key Key Flow Data
Key Flow Data
Key Flow Data
NetFlow
HIT! Key Table Index Flow Data
3 Key 5 Flow Data
Key Result Flow Data
Hash Function
Flow Data
Mask
4
Hash Key
Hash Key Compare 128K/256K 128K/256K
entries rows

Netflow TCAM Netflow Table


Key
128 entries

BRKARC-3465 Alias CAM


© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 71
Catalyst 6500 NetFlow
NetFlow Export Process

Supervisor

Netflow
Export Netflow
Data

WS-X6748-GE-TX w\DFC4 Netflow Collector

Netflow
Data
EOBC

WS-X6908-10G-2T\2TXL
Netflow
Data

WS-X6708-10GE-3C/3CXL, WS-X6716-10x-3C/3CXL, WS-X6816-


10x-2T/2TXL and 69xx line cards have the capability to
perform direct export from the line card itself
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 73
Catalyst 6500 NetFlow
PFC4 Key Enhancements
The PFC4 Can Do Everything the PFC3 Can Do and Adds These New Capabilities:

Increased Support for NetFlow Entries


Up to 1M NetFlow entries (512K for Ingress and 512K for Egress) can now be stored in PFC4XL.

Improved NetFlow Hash


The hash efficiency is improved to 99%, allowing a greater percentage of the NetFlow table to be utilized .

Egress NetFlow
Provides support for collecting flow statistics for packets after they have had ingress processing applied to
them.

Sampled NetFlow
Allows users to to have NetFlow records created based on a sample of traffic matching the flow.

Flexible NetFlow
Supports the NetFlow V9 Record Format including new fields for IPV6 and Multicast information.

TCP Flags
TCP Flags (SYN, FIN, RST, ACK, URGENT, PUSH) are now collected as part of a flow record.

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 74
Catalyst 6500 NetFlow
PFC4 NetFlow Processing

IFE NetFlow Sampling IFE NetFlow Statistics


Accounting of packets
and Lookup admitted by input processing

Incoming Ingress Ingress L3 Ingress


Packet ACL NetFlow Lookup QoS

IFE Process

OFE Process
Outgoing Egress Egress Egress
Packet QoS NetFlow ACL

OFE NetFlow Statistics OFE NetFlow


Accounting of forwarded packet Sampling and Lookup

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 75
Catalyst 6500 NetFlow
TCAM Lookup on PFC4
DST IP SRC IP Proto DST Port SRC Port
10.1.2.11 10.1.1.10 0x6 80 33992

DataKey
Data Key Index
Index Flow Data Statistics
Data
Data Key
KeyKey Index
Index
Data
Data Key Index
Index Flow Data Statistics
DataKey
Data Key Index
Index
1 Data Key
DataKey
Key Index
Index Flow Data Statistics
Data
Data KeyKey Index
Index
Data
Data Key Index
Index Flow Data Statistics
Flow
FlowKey
Key DataKey
Key Index Compare
Data
DataKey
Key Index
Index Flow Data Statistics
Data
Data Key Index
Index Flow Data
Data KeyKey
Data Index
Index Flow Data Statistics
Data Key
DataKey
Key Index
Index 6
Data Index Flow Statistics
DataKey
HIT!
Data Key Index
Index HIT! Data 7
Data
Data Key
KeyKey Index
Index
2 Data
Data Key Index
Index Flow Data Statistics
DataKey
Data Key Index
Index 5 Update
DataKey
Data Key Index
Index Flow Data Statistics
Hash Function Compare Data Key Index Stats
all pages Data KeyKey
Data Index
Index Index to Flow Data Statistics
Data Key
DataKey
Key Index
Index
Data
DataKey
Key Index
Index NF Data Flow Data Statistics
4 Data
Data Key Index
Index Table
Data Key Index
Lookup Data Key
Key
512K
entries
3 Indexes row in Lookup Table
NetFlow Data Table NetFlow
NetFlow Lookup Table Statistics
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential Table 76
Catalyst 6500 NetFlow
PFC4 Sampled NetFlow
• Supervisor 2T supports up to 1K Hardware based M:N Samples
For every N packets, the sampler selects M packets (M<N)

• Random sampling method is supported :


Randomly sample M consecutive packets out of every N packets

Random M/N Packet Sampling: M = 2 and N = 5


1 N 1 N 1 N

To be
Exported
NetFlow Cache

Netflow
Table
Copy-based (or deep-packet sampling)
will be supported in later releases

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 77
Catalyst 6500 NetFlow
PFC4 Flexible NetFlow Configuration
Flow Export
Flow Record
Export Profile
Key Field Non-Key Field
Multiple Exporters can
be associated with a Export Profile
Key Field Non-Key Field single FNF monitor

… … …

Key Fields trigger the creation of a new Flow entry


every time their value change Flow Monitor
Non-Key Fields are data that is indexed by the Key Fields.
Flow Monitor
Key Fields are defined using the ―match‖ statement
Non-Key-Fields are defined using the ―collect‖ statement

Interfaces
Ingress Ingress Same Flow Monitor can be
or/and …. or/and associated with multiple
Interfaces.
Egress Egress
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 78
Catalyst 6500 NetFlow
MSFC5 Yielding NDE
Supervisor 2T and Direct Export capable Line Cards support a Yielding NDE
capability to provide more optimal CPU utilization when using NDE
Supervisor

Netflow
Export Netflow
Data

Netflow Collector
WS-X6748-GE-TX w\DFC4 Netflow Data Export process begins
1
Netflow
Data Yielding NDE monitors the CPU Utilization
2

EOBC If the CPU Utilization crosses the user-configured


3 threshold, then the NDE process backs off to
yield to higher priority tasks.
New Export Rate = Current Export Rate – ( (Total CPU – Threshold) * Current Export Rate)
Once the CPU Utilization crosses beneath the
4
threshold, the NDE process will resume its
normal pace
6509E(config)#flow hardware export threshold <threshold> linecard <threshold>
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 81
Flexible NetFlow Automation with EEM
Example I: Malformed Packets Detection & Reporting
TTL = 0 triggers an EEM event
Attacker sending
malformed pkts with NetFlow cache
TTL=0
srcIf SrcIPadd DstIf DstIPadd TTL *MAR 29 2010 12:29:02.604 UTC:
Fa1/0 173.1.1.2 Fa0/0 10.0.277.1 0 %HA_EM-6-LOG: my-ttl-applet: flow
record with zero TTL
Fa1/0 173.1.1.2 Fa0/0 10.0.277.1 10

Fa1/0 173.1.1.2 Fa0/0 10.0.277.1 200

syslog message generated


based on pre-configured
policies

Example II : Anomaly Flow Detection and Mitigation


NetFlow ED triggers policies to monitor flow rate.
Compromised user Typically, voice conversations are 64kbps
sending traffic with
high rate
NetFlow cache
srcIf SrcIPadd DstIf DstIPadd bytes *Feb 18 01:24:30.455: %LINK-5-
CHANGED: Interface FastEthernet
Fa1/0 173.1.1.2 Fa0/0 10.0.277.1 34346
1/0, changed state to
Fa1/0 173.1.1.2 Fa0/0 10.0.277.1 300 administratively down
Fa1/0 173.1.1.2 Fa0/0 10.0.277.1 1000 interface Fa1/0 is shut down when
the flow rate exceeds
1Mbps

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 82
Displaying NetFlow Utilization
6509E#show platform hardware capacity netflow

Netflow resources:

Netflow table size: 515032 entries total

Netflow table usage: Module/Instance Input flows Output flows

3 10% 10%

7 25% 25%

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 83
Agenda

Access Control Lists

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 84
Catalyst 6500 Access Control Lists
ACL Types and Feature Support

DA MAC SA MAC Proto SA IP DA IP


Ethertype SA Port DA Port

Access Control Lists can be described by the types of


packet or frame headers they filter on
Standard ACLs – Source IP Address
Extended ACLs – Source and Destination IP plus L4 ports
MAC-based ACLs – L2 MAC address
Role Based – Security Group Tag (Cisco TrustSec)

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 85
Catalyst 6500 Access Control Lists
ACLs in Hardware
Hardware Support
1 DFC Policy Feature Card (PFC)
Distributed Forwarding
Create the ACL or DFC Card (DFC)
traffic classification
policy using CLI or Router ACLs
Vlan ACLs
Network Management Port Based ACLs 2
PFC
System Role Based ACLs
PFC
IP Access-List extended
Internet DFC
permit ip any host 10.2.2.4
permit ip any host 10.5.2.33
Hardware- Assist
permit ip any host 10.11.0.0 Features 3
permit ip any host 10.4.0.0

Netflow
WCCP
Reflexive ACLs
Network Address
Translation
Cisco Trust Sec

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 86
Catalyst 6500 Access Control Lists
Three Forms of Security ACLs
The PFC3/PFC4 supports three forms of Security ACLs: the RACL, VACL and
PACL…

Router ACL (RACL) VLAN ACL (VACL) Port ACL (PACL)

Used to permit or Used to permit or


Used to permit or deny the movement deny the movement
deny the movement of traffic between of traffic between
of traffic between Layer 3 Layer 3
Layer 3 Subnets Subnets/VLANs or Subnets/VLANs or
within a VLAN within a VLAN

Applied as a policy Applied as a policy


Applied as an input to a VLAN - is to a Layer 2 Switch
or output policy to a inherently applied to port interface - is
Layer 3 interface both inbound and applied for inbound
outbound traffic traffic only

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 87
Catalyst 6500 Access Control Lists
ACL Order of Processing
Should a RACL, VACL and PACL all be configured at the same time, there is a
distinct order in which each form of ACL is processed…

Input RACL Output RACL

VACL VACL

Input PACL

Note that no Output PACL


exists

Destination

Source
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 88
Catalyst 6500 ACLs
PFC3 TCAM Entry Population
Protocol
xxxxxxxx 10.1.2.100 xx xxxx xxxx 1 Permit
Dest IP Source IP xxxxxxxx 10.1.68.101 xx xxxx xxxx 2 Deny
00000000 FFFFFFFF 00 0000 0000
xxxxxxxx 10.33.2.25 xx xxxx xxxx 3 Deny
Dest Port Source Port 4
5
1=―Compare‖ 6
0=―Mask‖ 7
ip access-list extended example 8
1
permit ip any host 10.1.2.100 xxxxxxxx xxxxxxxx 06 xxxx 0016 Permit
2
deny ip any host 10.1.68.101
xxxxxxxx xxxxxxxx 06 xxxx 0017 Deny
xxxxxxxx xxxxxxxx 11 xxxx 0202 3
deny ip any host 10.33.2.25 00000000 00000000 FF 0000 FFFF Deny
xxxxxxxx xxxxxxxx 06 xxxx 0080 4 Permit
permit tcp any any eq 22
5
xxxxxxxx xxxxxxxx 11 xxxx 00A1 Permit
deny tcp any any eq 23 6

deny udp any any eq 514 7

permit tcp any any eq 80 8

permitBRKARC-3465
udp any any eq © 161 Masks Values
2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 89
Catalyst 6500 ACLs
TCAM Lookup
Generate Compare 3
Lookup
xxxxxxxx
xxxxxxxx
10.1.1.10 xxxxxxxx
10.1.2.11
| 10.1.2.11 xx|xxxx
| 06 xxxx
84C80050
xxxx
| 0050 xxxxxxxx 10.1.2.100 xx xxxx xxxx
Key 1
Lookup Key 2 xxxxxxxx 10.1.68.101 xx xxxx xxxx
2
xxxxxxxx 10.33.2.25 xx xxxx xxxx
SIP=10.1.1.10 00000000 FFFFFFFF 00 0000 0000 3
DIP=10.1.2.11 4
Protocol=TCP (6)
SPORT=33992 5
DPORT=80
1 Entries 6
matching only 7
Packet
destination IP 8
ip access-list extended example xxxxxxxx xxxxxxxx 06 xxxx 0016
1
permit ip any host 10.1.2.100 xxxxxxxx xxxxxxxx 06 xxxx 0017 2
xxxxxxxx xxxxxxxx 11 xxxx 0202
deny ip any host 10.1.68.101 00000000 00000000 FF 0000 FFFF 3
HIT! xxxxxxxx xxxxxxxx 06 xxxx 0050
deny ip any host 10.33.2.25 4 Permit
xxxxxxxx xxxxxxxx 11 xxxx 00A1 45
permit tcp any any eq 22
Entries matching Result
deny tcp any any eq 23 only protocol and 6
deny udp any any eq 514
destination port 7
permit tcp any any eq 80 8
permit BRKARC-3465
udp any any eq 161
Masks Values
© 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 90
Catalyst 6500 ACLs
PFC4 Mask Utilization
permit ip 10.1.1.0 0.0.0.255 any
permit ip 10.2.1.0 0.0.0.255 any
permit ip 10.3.0.0 0.0.255.255 any

PFC3 ACL TCAM PFC4 ACL TCAM


10.1.1.0 permit Mask 0.0.0.255 10.1.1.0 permit
MASK 10.2.1.0 permit Mask 0.0.0.255 10.2.1.0 permit
- - Mask 0.0.255.255 10.3.0.0 permit
- -
0.0.0.255 - - - - -
- - - - -
- - - - -
- - - - -
10.3.0.0 permit - - -
- - -
MASK - - - - -
- - - - -
- -
- -
0.0.255.255 - - Implements 1:1 Mask to Entry
- -
- - ratio

Implements 8:1 Mask to Entry ratio Total 256K Masks, 256K Entries
Total 4K Masks, 32K Entries
Mask resource is limited Mask resource is no longer a
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved.
limited resource
Cisco Confidential 92
Catalyst 6500 SUP2T/PFC4 Enhancements
Access Control List Lookup Example

TCAM A TCAM B
BANK 0 BANK 1 BANK 2 BANK 3 Forwarding Engine
VACL (PFC4 or DFC4)
RACL
SGT
QoS

3 ACE
Counters
TCAM Controller (L2 ASIC)

2 2 X Lookup Keys 4 X Results 4 7


4 X Result Data Final Result to
ACL ACL
Packet Header Information Labels LOUs Netflow
6
8
Classification Module 1 Classification Module 2
1 5

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 93
Catalyst 6500 ACLs
PFC4 ACL Dry Run

SUP2T-E#show configuration session test status


 Make sure the ACL will fit in ====================================
the TCAM before you apply Status of last config validation:
the ACL Timestamp: 2010-02-20@17:27:06
======================================
ACLs that do not fit can cause
software forwarding and SLOT = [1] Result = Configuration will fit in TCAM
possible high CPU utilization
 Special configuration session
Create and edit ACls
Verifies if the changes will fit
within the hardware resources
 The actual changes are not
programmed into the hardware
during the configuration
session
 Configuration changes can be
verified step by step

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 95
Catalyst 6500 ACLs
PFC4 ACL Hitless Update

 Allows updates to an ACL IPv4


IPv6
without interrupting traffic
MAC
 Multiple features updated
at once
IPv4,IPv6, MAC… ACL Updates
RACL, VACL, PBR…
 Global configuration
option (default is on)
 Feature does consume
double the number of
TCAM entries

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 97
Catalyst 6500 ACLs
PFC4 ACL Hitless Update

TCAM A TCAM B

 Each ACL feature is initially BANK 0 BANK 1 BANK 2 BANK 3

programmed into two VACL-1

different spaces into the RACL-1 VACL-2


TCAM RACL-2 SGT-1 QoS-2
SGT-2 QoS
Primary space (Label -1)
Shadow space (label-2)
TCAM Controller
 While an ACL is being 4 X Results
2 X Lookup Keys
updated the PFC4 will use a
temporary label that points ACL
ACL
to the shadow TCAM space Labels
1, 2
LOUs

 Once the ACL changes


have been completed the Classification Module 1
then PFC4 will then use the
original label again

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 98
Agenda

Packet Walks

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 99
Centralized Forwarding:
Classic to Classic

L3/4 Supervisor
Engine Engine 2T D

Port Port4
Classic
L2 Engine 720Gbps ASIC ASIC
Module B
2 Switch
3 PFC4 Fabric

DBUS
RBUS

Source S
Destination D
Classic Port Port
Module A ASIC ASIC
Blue VLAN
1 Orange VLAN
S

Entire Packet
Packet Header
Result

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 100
Centralized Forwarding:
Classic to CEF720
D

Port Port
L3/4 Supervisor CEF720
ASIC ASIC CFC
Engine Engine 2T Module B

Fabric Interface/
6
L2 Engine 4 720Gbps Replication Bus
2 Switch5 20Gbps Engine Interface
3 PFC4 Fabric

DBUS
RBUS

Source S
Destination D
Classic Port Port
Module A ASIC ASIC
Blue VLAN
1 Orange VLAN
S

Entire Packet
Packet Header
Result

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 102
Centralized Forwarding with Fabric:
CEF720 to Classic

L3/4 Supervisor D
Engine Engine 2T
Port Port
Classic
720Gbps ASIC 8
ASIC
L2 Engine Module B
63 74 Switch
PFC4 Fabric

DBUS
RBUS
20Gbps

Source S
CEF720
Fabric Interface/
5 2Bus Module A Destination D
Replication Interface
Blue VLAN
Engine
Orange VLAN

Port Port CFC


Entire Packet
ASIC ASIC
1 Packet Header
S Result

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 104
Centralized Forwarding with Fabric:
CEF720 to CEF720
D

Port Port
L3/4 Supervisor CEF720
ASIC ASIC CFC
Engine Engine 2T Module B

Fabric Interface/
6
L2 Engine 720Gbps Replication Bus
3 4 Switch 20Gbps Engine Interface
PFC4 Fabric

DBUS
RBUS
20Gbps

Source S
CEF720
Fabric Interface/
5 2Bus Module A Destination D
Replication Interface
Blue VLAN
Engine
Orange VLAN

Port Port CFC


Entire Packet
ASIC ASIC
1 Packet Header
S Result
NOTE: Forwarding from CEF720 to CEF720 w/DFC3 is the same process
except CEF720 w/DFC3 does not have any Bus connections
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 106
Distributed Forwarding:
CEF720 with DFC4 to CEF720 with DFC4
D

Port Port
CEF720
ASIC ASIC DFC4
Supervisor Engine 2T L3/4 Module B
Engine w/DFC4
5
Fabric Interface/
720Gbps
PFC4 Switch 20Gbps Replication Layer 2
Fabric Engine Engine
20Gbps

Source S
CEF720
4 Destination D
Module A
Fabric Interface/ 2 Layer 2 Blue VLAN
w/DFC4
Replication 3Engine
Orange VLAN
Engine
L3/4
Engine Entire Packet
Port Port DFC4
ASIC ASIC Packet Header

1 Result
S
NOTE: Forwarding from CEF720 w/DFC3 to CEF720 is the same process
except CEF720 has a Bus connections

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 108
Summary

 The Catalyst 6500 architecture provides a robust


infrastructure upon which the system can provide hardware-
based forwarding at high speeds
 L2 and L3 switching are done via the same hardware
forwarding process, so there is no difference in performance
between the two
 Enabling features such as Netflow, QoS and ACLs can be
done without impact to forwarding performance as these
features are processed in hardware in parallel to the L2 and
L3 lookup processes
 The Catalyst 6500 architecture is designed so that unicast and
multicast can coexist within the same infrastructure, providing
a versatile platform for the networks of today and tomorrow

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 110
Conclusion

You should now have a


thorough understanding of the
Catalyst 6500 switching
architecture, packet flow, and key
forwarding engine functions…

Any Questions?

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 111
Complete Your Online
Session Evaluation

 Receive 25 Cisco Preferred Access points for each session


evaluation you complete.
 Give us your feedback and you could win fabulous prizes.
Points are calculated on a daily basis. Winners will be notified
by email after July 22nd.
 Complete your session evaluation online now (open a browser
through our wireless network to access our portal) or visit one
of the Internet stations throughout the Convention Center.
 Don’t forget to activate your Cisco Live and Networkers Virtual
account for access to all session materials, communities, and
on-demand and live activities throughout the year. Activate
your account at any internet station or visit
www.ciscolivevirtual.com.

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 112
Visit the Cisco Store for
Related Titles
http://theciscostores.com

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 113
BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 114
Thank you.

BRKARC-3465 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 115

You might also like