Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Brocade Administration & troubleshooting

•Download as PPSX, PDF•
•
P
prakashjjaya

The document provides information about Brocade SAN switches including their product lines, features, and specifications. It discusses various switch models ranging from 8-port to 384-port configurations supporting 1, 2, 4, 8, and 10Gbps speeds. Features covered include dynamic path selection, ISL trunking, extended fabric, hardware-enforced zoning, advanced performance monitoring, and FCIP tunneling. The document also reviews FOS enhancements, new 10Gbps blades, and concepts like NPIV and NPV.

1 of 54
Brocade Administration & Troubleshooting 10th Jan 2014 Jayaprakash Aridos
Brocade Storage SAN Family The Leading SAN Connectivity Solutions for Open Systems & Mainframe Environments FC Routing, iSCSI and Extension Solutions Fabric Manager SAN16B-2 (2005-B16) 8 to 16-ports 1, 2, 4Gbps FC SAN64B-2 (2005-B64) 32 to 64-ports 1, 2, 4Gbps FC, FICON 4 Gbps SAN Switch Module for IBM BladeCenter® 1, 2, 4Gbps FC SAN256B-2 (2109-M48) 16 to 384-ports 1, 2, 4, 10Gbps FC, FICON *10 Gbps on SAN256B in 3Q07 NEW SAN32B-3 (2005-B5K) 16 to 32-ports 1, 2, 4Gbps FC EFCM SAN Fabric Management Tools SAN256M (2027-256) 32 to 256-ports 1, 2, 4, 10Gbps FC, FICONSAN140M (2027-140) 4 to 140-ports 1, 2, 4, 10Gbps FC, FICON SAN18B-R (2005-R18) 256B FCR Blade (FC #3450) 256B iSCSI (FC #3460) 6/1/2015 Brocade Administration & Troubleshooting 2
Feature Description SAN Benefits 8 Gbit/sec Bandwidth Next generation SAN performance today. Double SAN bandwidth. Dynamic Path Selection Dynamically balance traffic across multiple links and trunk groups. ISL Trunking Build SAN with up to 32 Gbit/sec performance optimized trunks. Extended Fabric Extend native FC links up to 500 km. Combine with ISL Trunking up to 250 km. Enhanced Business Continuity Improved distances and performance for Metro Mirroring and Remote Backup 8 Gbit/sec Link Performance Up to 32Gbit/sec Trunk SAN Brocade Advanced Features Enhancing Performance and Availability SAN Improved Performance 8 Gbit/sec Inter-Switch Links (ISLs) 8 Gbit/sec links to next-gen devices Up to 32 Gbit/sec ISL Trunks Infrastructure Simplification Simpler SAN topology Increased SAN Availability Investment Protection 6/1/2015 Brocade Administration & Troubleshooting 3
Feature Description SAN Benefits Hardware Enforced Zoning Prevents one device from communicating to another device it is not authorized to access. Enforced at the ASIC level Isolate devices from each other in the fabric such that there is no possible interaction between devices that are not explicitly defined. Enhance security and fault isolation. Fabric Watch Monitoring and alerting of key SAN statistics such as perf, error, security Enhance Business Continuity Improve application availability Alert admin of marginal/hard errors Advanced Performance Monitoring Highly granular SAN perf. monitoring to differentiate traffic between devices Enhance performance monitoring Simplify capacity planning Enable bill-back capabilities Advanced Security Robust encryption, authentication and authorization SAN policies Enhance Business Continuity Protect SAN from hackers Reduce user errors SAN SAN Brocade Advanced Features Enhancing Fault Isolation, Management and Security SAN 6/1/2015 Brocade Administration & Troubleshooting 4
Brocade Advanced Features FCIP, FC Routing and Partitioning Feature Description SAN Benefits Virtual SAN Fabrics Partition the SAN fabric at port level to isolate mngt, admin roles, RSCNs, fabric events Allows separation of a SAN fabric into smaller fabrics that may overlap and share devices SAN LPARS Create HW logical partition at card level for independent managed directors in a single chassis Independent fabric services per partition allows true Segmentation of data, control and management traffic FCIP Tunneling Extends FC links up to 1000’s km over IP network. FC and FCIP Fast Write Enhanced Business Continuity. Improved Global Mirroring and Tape Vaulting FC Routing Route selected SAN traffic between SAN islands w/o merging fabrics and admin Improve storage resource sharing, Enables SAN consolidation, Maintains SAN security and fault isolation SAN SAN Production Test Backup Spare ports 6/1/2015 Brocade Administration & Troubleshooting 5
Feature Description SAN Benefits FICON/FC Intermix FICON and FCP protocol intermix at the port level. Auto-sensing protocol and port speeds increase flexibility and provides greater port level granularity. N-Port Virtualization (NPIV) Allows multiple Linux Logical Partitions to share a single FCP channel Use fewer FCP channels on Mainframe Better channel utilization and simpler infrastructure High Integrity Fabrics (FICON Cascading) Binds Switches to Fabric for increased Security Enhanced Business Continuity. Improved distances/performance for Metro Mirroring and Remote Backup FICON CUP FICON in-band management of Directors from the Mainframe Simplify management and monitoring of directors Single point of management for Enterprises In-band Management z9 M48 Brocade Advanced Features FICON Specific Features M48 z9 xSeries zSeries DS4000 iSeries pSeries DS6000 DS8000 M48 M48 6/1/2015 Brocade Administration & Troubleshooting 6
New Hardware from Brocade •10Gbps blade (FC #3470) • 6-ports for 10Gbps ISL connectivity using SW, LW and ELW XFP media • All Brocade directors now offer 10Gbps inter-switch links • Ideal for Long distance, high bandwidth BC/DR solutions • Support distances over 100km – iSCSI blade (FC #3460) • Supports iSCSI initiators and Fibre Channel target/initiator • 8 GigE ports and 8 FC ports (1/2/4 Gbps) • 64 iSCSI initiators per port, 512 initiators per blade • Up to 4 blades per SAN256B (2048 tested) 6/1/2015 Brocade Administration & Troubleshooting 7
FOS Enhancements from Brocade •FC Routing and FCIP: • Increased Scalability for the SAN18B-R or FC-FCIP Routing Blade for M48 • Up to 12 Layer 2 switches and 12 FC Routers in the backbone fabric • FCIP tunnels offer QoS capabilities to ensure specific bandwidth • FC and FCIP FastWrite capabilities to enhance long distance mirroring solutions •Security management: • Complete migration of Secure Fabric OS features in the base FOS code • A new Security Administrator role for separation of security and fabric administrators • Tracking of logins to see breaches in the fabric. • Fabric authentication with standards-based improvement for device-to-fabric attachment (FC-SP DH-CHAP) • IPv6 capabilities for all management interfaces – IP over FC support: • Supporting Broadcast Zoning to reduce device interruption • Targeted for the film industry • When using FC as a common backbone for for host-to-storage and host-to-host file transfer. • Reducing overall production time and improving the integrity/security of digital data transfers. •Access Gateway feature added to SAN16B-2 to allow greater scalability and simplify connectivity 6/1/2015 Brocade Administration & Troubleshooting 8
Brocade Switch Module for BladeCenter Connecting to McDATA SAN fabrics Brocade BladeCenter SAN Switch Modules 20072006 Access Gateway Interoperability Native McDATA Interoperability McDATA/QLogic FC Switch Modules End Sale Connecting BladeCenter to McDATA fabrics: •2007 – use the no-cost feature on Brocade called Access Gateway •2008 – use Access Gateway feature or the M-EOS (NI) mode Do not use the former McDATA/QLogic module •QLogic HW and firmware •Being End-of-Lifed •Not as good a solution as Brocade SAN Switch in long run 2008 6/1/2015 Brocade Administration & Troubleshooting 9
NPIV & NPV • Two technologies that seem to have come to the fore recently are NPIV (N_Port ID Virtualization) and NPV (N_Port Virtualization). NPIV • What NPIV does is allow a single physical N_Port to have multiple WWPNs, and therefore multiple N_Port_IDs, associated with it. After the normal FLOGI process, an NPIV-enabled physical N_Port can subsequently issue additional commands to register more WWPNs and receive more N_Port_IDs (one for each WWPN). The Fibre Channel switch must also support NPIV, as the F_Port on the other end of the link would “see” multiple WWPNs and multiple N_Port_IDs coming from the host and must know how to handle this behavior. • N port identifier virtualization (NPIV) provides a means to assign multiple FC IDs to a single N port. This feature allows multiple applications on the N port to use different identifiers and allows access control, zoning, and port security to be implemented at the application level. The following figure shows an example application using NPIV. 6/1/2015 Brocade Administration & Troubleshooting 10
6/1/2015 Brocade Administration & Troubleshooting 11
NPV – NPV introduces a new type of Fibre Channel port, the NP_Port. The NP_Port connects to an F_Port and acts as a proxy for other N_Ports on the NPV-enabled switch. Essentially, the NP_Port “looks” like an NPIV-enabled host to the F_Port on the other end. An NPV-enabled switch will register additional WWPNs (and receive additional N_Port_IDs) via NPIV on behalf of the N_Ports connected to it. The physical N_Ports don’t have any knowledge this is occurring and don’t need any support for it; it’s all handled by the NPV-enabled switch. • So why is this functionality useful? There is the immediate benefit of being able to scale your Fibre Channel fabric without having to add domain IDs, yes, but in what sorts of environments might this be particularly useful? Consider a blade server environment, like an HP c7000 chassis, where there are Fibre Channel switches in the back of the chassis. By using NPV on these switches, you can add them to your fabric without having to assign a domain ID to each and every one of them. Benefits of NPIV – Without NPIV, it’s not possible because the N_Port on the physical host would have only a single WWPN (and N_Port_ID). Any LUNs would have to be zoned and presented to this single WWPN. Because all VMs would be sharing the same WWPN on the one single physical N_Port, any LUNs zoned to this WWPN would be visible to all VMs on that host because all VMs are using the same physical N_Port, same WWPN, and same N_Port_ID. – With NPIV, the physical N_Port can register additional WWPNs (and N_Port_IDs). Each VM can have its own WWPN. When you build SAN zones and present LUNs using the VM-specific WWPN, then the LUNs will only be visible to that VM and not to any other VMs. 6/1/2015 Brocade Administration & Troubleshooting 12
6/1/2015 Brocade Administration & Troubleshooting 13 Ex., VMWare hosts & TOPS LPARS Ex., SANTap Module in Meritor
Traditional Datacenter Brocade Cisco McData Benefits of Brocade Access Gateway No cost feature solves real-world SAN issues Access Gateway Access Gateway Access Gateway Access Gateway Access Gateway Access Gateway Concerns with Bladed Servers: •Difficult to connect to McDATA fabrics •Limited SAN scalability •Obscure Admin responsibility (SAN admin vs Server admin) Datacenter using Access Gateway Access Gateway offers these benefits: •Connects to McDATA Fabrics •Simplifies fabric & allows greater scalability •Clear Admin responsibility SAN FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch Change Modes SW AG 6/1/2015 Brocade Administration & Troubleshooting 14
SAN16B-2 using Brocade Access Gateway No cost feature simplifies edge connectivity Typical Core-Edge Topology: •One to two core switches •3 to many edge switches •In this example: •1 SAN32B-3 core switch •3 SAN16B-2 edge switches •4 domains to manage SAN32B-3 SAN16B-2 SAN16B-2 SAN16B-2 Change 16B from Switch mode to AG mode SAN32B-3 Access Gateway Access Gateway Access Gateway Core-Edge Topology with Access Gateway: •Reduce the total number of domains •Connect to FOS, M-EOS and Cisco fabrics with NPIV •In this example: •1 SAN32B-3 core switch •3 SAN16B-2 edge Access Gateways •1 domain to manage NEW 6/1/2015 Brocade Administration & Troubleshooting 15
AG Feature Support Statement •AG is supported on the 4Gb Brocade SAN Switch Module for BladeCenter and SAN16B-2: – FOS 5.2.1b or above for SAN Switch Module for BladeCenter – FOS 5.3+ for SAN16B-2 •Access Gateway feature requires that NPIV (N-port ID virtualization) capability be enabled on the external switches: – Brocade switches running FOS 5.1 with NPIV enabled – McDATA switches running EOS 9.0 with NPIV enabled • M-EOS 9.6 offers NPIV in base code – Cisco switches running OS 3.0 with NPIV enabled 6/1/2015 Brocade Administration & Troubleshooting 16
Typical AG Deployment Scenarios •When to use AG? – Enterprise and large data centers where the fabric size is becoming a burden • Greater than 50 switches in an all b-type (Brocade) fabric • Greater than 30 switches in a fabric that includes m-type (McDATA) products – Connecting BladeCenter or 16B-2 to McDATA or Cisco SAN fabrics • Requires NPIV feature on the external switches – Customer’s SAN Admin group does not want an embedded switch in server products (i.e. BladeCenter) •When NOT to use AG? – Connecting SAN targets (such as storage) directly to switch module – Environments where customers require switch features not supported by AG • ISL Trunking • Long Distance support greater than 10km (using Extended Fabric license) Use AG Feature only when needed to help overcome an issue. Default mode for switch module should always be FC switch. 6/1/2015 Brocade Administration & Troubleshooting 17
Brocade Director Roadmap Investing Today & Tomorrow 256M Director • 256-ports 1,2,4 Gig • Hard Partitions • Virtual Fabrics • 10Gbit/sec ISL • Open Trunking 140M Director • 140-ports 1,2,4 Gig • 10Gbit/sec ISL • Open Trunking • NPIV Next Generation Core • Common Management - EFCM • Native Interoperability (M-EOS) • Partitioning • Virtual Fabrics • Features: •1, 2, 4, 8 and 10Gbps •FCP, FICON, FCIP, •FCR, iSCSI, Apps Seamless growth for all director platforms 20072006 2008 & beyond 256B Director • 384-ports 1,2,4 Gig • FCP, FICON, FCIP • FC Routing • Virtual Fabrics • NPIV SAN32B-3BROCADE AG (BladeCenter) •10Gig blade (ISL) • iSCSI Blade FOS 6.x • 8Gbps blades • FCP and FICON • Native Interoperability (M-EOS) E/OS 9.6  NPIV  Security Ench  Interoperability enhancements and validation E/OS 9.7  IPv6 • Interoperability enhancements and validation E/OS 9.7 • IPv6 • Interoperability enhancements and validation  Interoperability enhancements and validation 6/1/2015 Brocade Administration & Troubleshooting 18
Next Generation Core Platform Consolidating and Scaling Your Infrastructure Next Gen Core No rip-and-replace for Next Gen SAN. Seamless growth for all director platforms. NG Core Director connects to all 4Gb directors Edge Core 140M 256M 256B 2007: 4Gb products are just now becoming prevalent. End-to-End solutions with servers, disk and tape. 2008-2011: 4Gb directors will connect into hi-perf Core. Multiple protocols & very high bandwidth allow for greatest data center consolidation. 8Gb FC devices will start rolling out. 8Gb 8Gb 4Gb 6/1/2015 Brocade Administration & Troubleshooting 19
Networking Fundamentals • FC Topology • Fabric Scalability • Initiator/Target relationship • Switch Ports • FC definitions • ISL Concepts • Cable selection • Host Support 6/1/2015 Brocade Administration & Troubleshooting 20
Core-Edge Design the Fabric for your requirements Full mesh Cascade Mesh Configurations are appropriate when: Traffic patterns are evenly distributed Overall bandwidth consumption is low The maximum config is four to five switches Cascade Configurations are appropriate when: Traffic patterns are localized onto individual switches Core-Edge Configs are appropriate when: Fabric is likely to grow A flexible system is required because of unknown or undefined requirements Reliability is required – this type of config has been well-tested and is used in most production environments Consider the Fabric Port Count : The total number of FC ports in the Fabric, this would include ALL ports on ALL switches for A fabric, remember that you have dual fabrics, larger numbers should mean moving from cascade / mesh to core-edge 6/1/2015 Brocade Administration & Troubleshooting 21
Fabric or Network Architectures • Types of architectures are: – Single-Switch – Cascade – Mesh – Core-Edge – Director 6/1/2015 Brocade Administration & Troubleshooting 22
Cascade • Maximum hop count supported is three 6/1/2015 Brocade Administration & Troubleshooting 23
Mesh Partial Mesh 6/1/2015 Brocade Administration & Troubleshooting 24
Core-Edge 6/1/2015 Brocade Administration & Troubleshooting 25
Fabric Choices – What are they? FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch 6/1/2015 Brocade Administration & Troubleshooting 26
How many fabrics are show below? FC_Switch4 FC_Switch8 FC_Switch2 FC_Switch5 FC_Switch3 FC_Switch6 FC_Switch7 FC_Switch1 FC_Switch10FC_Switch9 6/1/2015 Brocade Administration & Troubleshooting 27
Fabric Scalability Scale fabric size by replacing existing core with a larger core Scale # Core Switches Scale # ISLs Scale performance by adding ISLs or additional core switches Scale fabric size by adding switches Scale # ports Examples of Fabric Scalability 6/1/2015 Brocade Administration & Troubleshooting 28
1 2 3 Storage Storage New Firmware Storage Both Switches have New Firmware Design the Fabric for your requirements Serviceability using a Dual Fabric Design • Firmware upgrade can be done without I/O interruption if the following Rolling Upgrade is applied – Dual path is required from server and storage • Add new switches or upgrade current switches easily 6/1/2015 Brocade Administration & Troubleshooting 29
Initiator/Target Relationship SCSI over Fibre Channel SCSI over TCP/IP (iSCSI) TCP/ IP TCP/ IP HOST (Initiator) Controller (Target) FC driver FC driver SCSI WAFL RAID iGroup SCSI driver File System Application SCSI Adapters Windows or UNIX Fibre Channel HBAs Data ONTA LUN iSCSI HBAs or Ethernet NICs Direct Attached Storage (DAS) Fabric/Network iSCSI driver Fibre Channel or Serial ATA Attach iSCSI driver 6/1/2015 Brocade Administration & Troubleshooting 30
WWNN and WWPN Examples HBA WWNN (World Wide Node Name) 20:00:00:2b:34:26:a6:54 HBA WWPN (World Wide Port Name) 21:00:00:2b:34:26:a6:54 22:00:00:2b:34:26:a6:54 6/1/2015 Brocade Administration & Troubleshooting 31
Switch Ports • E_Port - An expansion port connecting two switches to make a fabric. • F_Port - A fabric port to which an N_Port attaches. • FL_Port - A fabric loop port to which a loop attaches; needs FL card LED turned on. It is the gateway to the fabric for NL_Ports on a loop. • G_Port - A generic port that supports either E_Port or F_Port functionality. • L_Port - Node Loop port; a port supporting the Arbitrated Loop protocol. • N_Port - A fibre channel port in a fabric or point-to-point connection. 6/1/2015 Brocade Administration & Troubleshooting 32
SAN Made Easy – Auto Discovery What do I want to be when I grow up? y/n Do you want to talk loop? G_Port I’m waiting for someone to talk to me… yes no Are you a switch or a fabric point-to-point device? F_Port fabric pt-to-pt E_Port switch y/n Is something plugged into the port? n o yes U_Port FL_Port 6/1/2015 Brocade Administration & Troubleshooting 33
FC Definitions • ISL: Inter-Switch Link or a Switch-to-Switch Link; ISLs connect between two switch nodes to form E_ports. • Locality: The degree that I/O is confined to a particular switch or segment of a fabric. If two devices that need to communicate with each other are located on the same switch or segment, then these two devices are said to have high locality. If these same devices are located on different switches or segments of a fabric and these two devices need to communicate with each other, then these devices are said to have low locality. • Redundancy: When devices have two or more fabrics and multiple paths for a source to reach its destination the fabric is considered to have redundancy. This is critical so that when an initiator primary path fails, the secondary initiator path will be available so that initiator hosts can still communicate with their targets, at reduced performance. • Resiliency: The ability of a fabric to adapt to or tolerate a failure of a component. A fabric is said to have resiliency when it can tolerate 1 or more device failures from any component within the fabric, whether it is a switch, ISL, or HBA failure. • RSCN: Registered State Change Notification is the fabric mechanism that allows notifications to be sent to nodes if a change occurs within the fabric, ie. device going offline or online on a fabric port. • SCR: State Change Registrations are used by devices to register to receive RSCNs. 6/1/2015 Brocade Administration & Troubleshooting 34
FC Definitions • ISL Oversubscription Ratio: Inter-switch Link Oversubscription Ratio is the ratio of device, or data input ports that might drive I/O between switches to the number of ISLs over which the traffic could cross. • ISL Oversubscription = Number of Host Nodes: Number of ISLs, or IO=Nhn:Ni. • Fan-in ratio: The ratio of storage ports to a single host port • Fan-out ratio: The ratio of host ports to a single storage port • Buffer-to-buffer credits: The number of buffer-to-buffer credits determines the number of Fibre Channel frames that a switch can transmit on a link at one time before requiring an acknowledgement back from the receiver. Performance degradation may occur if there aren’t enough credits available to sustain communication between switches. As the distance between switches increases, additional buffer-to-buffer credits are required to maintain maximum performance. Credits are allocated from a common pool of memory on the switch ASIC. Formula to approximate # of Credits required over long distance: • Buffer Credits = ((Distance in KM) * (Data Rate) * 1000) / 2112 • Data Rate = 1.0625 Mbaud for 1 Gbit/sec Fibre Channel • Data Rate = 2.1250 Mbaud for 2 Gbit/sec Fibre Channel • Data Rate = 4.2500 Mbaud for 4 Gbit/sec Fibre Channel 6/1/2015 Brocade Administration & Troubleshooting 35
Best Practice – ISL Oversubscription • A 7:1 ISL oversubscription ratio is aligned with an industry average of 6:1 fan-out. The trend in the storage industry is that the hosts to storage ratios are increasing, as is the performance of storage devices. A 7:1 ISL oversubscription ratio should be targeted in SAN designs, with the ISL oversubscription ratio being adjusted higher or lower to meet particular performance requirements. While this ISL oversubscription ratio is conservative, it is felt that the cost of not having enough performance and having to reshuffle devices and ISLs is much greater than the cost of having a few extra spare ports that can be used to connect SAN devices at a later point in time. • Rule of thumb: The higher the ISL oversubscription ratio, the lower the performance and conversely, the lower the ISL oversubscription ratio, the higher the expected I/O performance. An ISL oversubscription ratio of 3:1 results in high performance and fewer available ports while an ISL oversubscription ratio of 15:1 results in lower potential performance and additional available ports reserved for devices. With the advent of 4Gbps ISLs, higher oversubscription ratios can exist while maintaining more than adequate bandwidth (since bandwidth is doubled per ISL) and higher device port counts for 2Gbps devices. 6/1/2015 Brocade Administration & Troubleshooting 36
FC SAN Host Support Host ClusterOS Vendor Multipath File SystemHBA Native HP PVLinks / VERITAS DMP MC ServiceGuard / VERITAS VCS Native MSCS VERITAS VCS Emulex / Qlogic HACMP QLogic QLogic QLogic QLogic QLogic QLogic Emulex QLogic VMware MSCS VirtualCenter (VMotion) Volume Mgr LVM VERITAS VxVM NSS VMware NTFS VERITAS VxFS JFS/2 Raw LVM / VERITAS VxVM JFS/ HFS Raw VERITAS VxFS ext3 / ext2 / Reiser / GFS* ext3 ext2 Reiser VMFS 2.x Raw LVM Under Test Oracle 9i, 10g, RAC Novell Clusters SANpath / MPIO VERITAS DMP / MPxIO* VERITAS VCS / Native SUN Cluster* Oracle 9i, 10g RAC / RH Cluster Suite* * (via PVR) MPIO NetApp DSM / VERITAS DSM for MPIO* MMC / VERITAS VxVM* Emulex / Native 4Gb 6/1/2015 Brocade Administration & Troubleshooting 37
Cable Distance Chart 6/1/2015 Brocade Administration & Troubleshooting 38
Zoning and Troubleshooting • Important Commands • Zoning (how to zone) • Zoning Best practices • Troubleshooting Procedure • Basic Troubleshooting 6/1/2015 Brocade Administration & Troubleshooting 39
FC switch tools – provided by switch manufacturer (Brocade) – switchshow • Displays status of the FC switch and all its ports • Show FC nodes currently logged into the switch (depends on FC zones, if any) – cfgshow • Show zones currently available on the FC switch • Shows information about the current FC configuration and which zone(s) are enabled – supportshow: Displays switch information for debugging purposes – ssshow: Displays information about the name server – nsshow: Verifies that clients are logged into the name server – fabricshow: Displays fabric membership information – configure • Changes switch configuration settings. • Switch need to be offline to run this command 6/1/2015 Brocade Administration & Troubleshooting 40
FC switch tools – provided by manufacturer (Brocade) (cont.) – alicreate, zonecreate: Create aliases and zones – cfgcreate, cfgsave, cfgenable,cfgshow: Manage zone configs – version: Displays firmware version information – portshow, portcfgshow, porterrshow, portLogDumpPort, portenable, portdisable • Manage ports – diagshow: Displays switch diagnostics – webUI: Web GUI available by browsing to the switch ip adress – nodefind, nszonemeber – To search wwns across fabric and inside zoning6/1/2015 Brocade Administration & Troubleshooting 41
Switch Zoning
Domain on Brocade Switches • Make sure that the Domain ID is set to a different value on all switches in a fabric • Example : if there are two fabrics in solution then the Domain ID on each switch in Fabric A should be set to an increasing odd number and for Fabric B set each Domain ID to an increasing even number – Fabric A – 11, 13, 15, 17, etc. – Fabric B – 10, 12, 14, 16, etc. Note: if HP-UX is involved then skip 8, this ID was used for Loop Configs • Name Server – service in fabric that provides directory services and info about ALL devices in the fabric 6/1/2015 Brocade Administration & Troubleshooting 43
Define and Implement Zoning How do I manage Zoning? • Manage zone physically or Logically • Three components to the zone information – One or more devices are placed in a zone – One or more zones are placed in a configuration – One and only one config is made the effective • Soft Zoning: Name Server assisted – Name Server restricts visibility – Always available when zoning enabled – No reduction in performance • Hard Zoning: Hardware Enforced – Available when certain rule checking criteria are met through hardware logic checking. – Provides additional security in addition to Soft zoning – Prevents illegal access from “bad” citizens. – No reduction in performance with hard-Port level zoning. – Available using port or WWN with Brocade 2 Gbit/sec 6/1/2015 Brocade Administration & Troubleshooting 44
Define and Implement Zoning Zoning Setup Guidelines • Create a detailed diagram of the fabric, showing all the switches with their ISLs • Create a “blowup” diagram of each switch in the fabric to account for devices • Account for private loop devices if they exist • There are special considerations for mixed 1 Gbit/sec and 2Gbit/sec based fabrics • For security reasons, consider disabling a port if the zoned fabric is going to contain unused ports, with nothing connected to them • Configure one zone at a time and then test it – Do not create all the zones at once; it will be troublesome to debug – After the first zone is setup in the fabric, plug in devices and then test the connections to confirm that everything is functioning properly – This process may seem a little tedious, but it will save time and money trying to debug this after creating all the zones and then plugging in the devices 6/1/2015 Brocade Administration & Troubleshooting 45
Define and Implement Zoning Implementing Zoning • Naming convention – There typically of three types of devices, server HBA, the storage port, and the tape port. – These will have an alias. • SRV – for servers • STO – for Storage • TPE – for Tape • For example, SRV_MAILPROD_SLT5 – a server, hostname “mailprod”, in PCI slot 5 – Keep names as small as possible to conserve space in zone database – Minimize duplication in alias definitions where possible – Keep zoning database as clean and accurate as possible • Fabric Name – Fabric name is the name that the fabric is generally known by. – PROD configuration is to easily identify the configuration that can be implemented and provide the most generic services. – BACKUP_XX, TEST_XX may be used 6/1/2015 Brocade Administration & Troubleshooting 46
Define and Implement Zoning 10 Zoning Rules - Brocade 1) If security is a priority, then a Hard Zone-based architecture coupled with Hardware Enforcement is recommended 2) Using aliases, though optional, should force some structure when defining your zones. 3) Add Secure Fabric OS® into the Zone Architecture if extra security is required. 4) If a SilkWorm 12000 is part of the fabric, then use it to administer zoning within the Fabric 5) If QuickLoop is required for legacy devices and the switch is running Brocade Fabric OS v4.x: – QuickLoop / QuickLoop zones cannot run on switches running Brocade Fabric OS v4.x. – QuickLoop Fabric Assist - Brocade Fabric OS v4.x cannot have a Fabric Assist host directly connected to it. 6/1/2015 Brocade Administration & Troubleshooting 47
Define and Implement Zoning 10 Zoning Rules - Brocade 6) Before implementing a zone run the Zone Analyzer and isolate any possible problems. 7) Before enabling or changing a fabric configuration, verify that no one is issuing I/O in the zone that will change. 8) Changes to zoning should be done during preventative maintenance to minimize any potential disruption. 9) After changing or enabling a zone configuration, confirm that nodes and storage are able to see and access one another. 10) LUN Masking should be used in conjunction with fabric zoning for maximum effectiveness. 6/1/2015 Brocade Administration & Troubleshooting 48
Zoning Example – Single Fabric zone1 zone2 Storage Host1 Host2 Host3 Host4 FC Fabric What is needed on the hosts systems and on which systems is it needed in this configuration? 6/1/2015 Brocade Administration & Troubleshooting 49
Simple Troublshooting Cmds • cfgactshow |grep cfg =====>to check the current zoneset config • • zoneshow *zone_name* <-- To search in zones • alishow *alias_name* <-- To search in alias • cfgactshow |grep <host_name> To search zone in active zoneset • switchshow |grep <wwn> <-- to check flogi information • nszonemember <wwn> <-- FCNS database and to check zoned' objects • portshow <interface> <-- To Show interface • portloginshow <interface> <-- To list port logins • fabricshow <-- To check switch topology 6/1/2015 Brocade Administration & Troubleshooting 50
Zoning Ex., • Step by step zone creation and activation • cfgactvshow |grep cfg - check the current activated zoneset or configuration • alicreate "S0ADCF1S2P8_HBA2_PROD", "c0:50:76:00:51:3e:00:06" <-- To create fcalias • alicreate "S0ADCF1S2P8_HBA2_LPM", "c0:50:76:00:51:3e:00:07" • • zonecreate "USCLSITPS002_S0ADCF1S2P8_HBA2_CX1571_SPA0", "S0ADCF1S2P8_HBA2_PROD; S0ADCF1S2P8_HBA2_LPM; CX1571_SPA0" <-- To create zone • zoneshow *USCLSITPS002_S0ADCF1S2P8_HBA2_CX1571_SPA0* <--- Check the zone is created on the switch • • cfgsave <-- To save configuration file • • cfgadd "Fab2_VF70_10212013", "USCLSITPS002_S0ADCF1S2P8_HBA2_CX1571_SPA0" <-- To add zone into existing configuration (zoneset) • cfgsave <-- To save latest configuration • • cfgenable "Fab2_VF70_10212013" <-- To activate configuration file (activate zoneset) 6/1/2015 Brocade Administration & Troubleshooting 51
Troubleshooting Ideas 6/1/2015 Brocade Administration & Troubleshooting 52
Q & A 6/1/2015 Brocade Administration & Troubleshooting 53
6/1/2015 Brocade Administration & Troubleshooting 54

More Related Content

Brocade Administration & troubleshooting

  • 2. Brocade Storage SAN Family The Leading SAN Connectivity Solutions for Open Systems & Mainframe Environments FC Routing, iSCSI and Extension Solutions Fabric Manager SAN16B-2 (2005-B16) 8 to 16-ports 1, 2, 4Gbps FC SAN64B-2 (2005-B64) 32 to 64-ports 1, 2, 4Gbps FC, FICON 4 Gbps SAN Switch Module for IBM BladeCenter® 1, 2, 4Gbps FC SAN256B-2 (2109-M48) 16 to 384-ports 1, 2, 4, 10Gbps FC, FICON *10 Gbps on SAN256B in 3Q07 NEW SAN32B-3 (2005-B5K) 16 to 32-ports 1, 2, 4Gbps FC EFCM SAN Fabric Management Tools SAN256M (2027-256) 32 to 256-ports 1, 2, 4, 10Gbps FC, FICONSAN140M (2027-140) 4 to 140-ports 1, 2, 4, 10Gbps FC, FICON SAN18B-R (2005-R18) 256B FCR Blade (FC #3450) 256B iSCSI (FC #3460) 6/1/2015 Brocade Administration & Troubleshooting 2
  • 3. Feature Description SAN Benefits 8 Gbit/sec Bandwidth Next generation SAN performance today. Double SAN bandwidth. Dynamic Path Selection Dynamically balance traffic across multiple links and trunk groups. ISL Trunking Build SAN with up to 32 Gbit/sec performance optimized trunks. Extended Fabric Extend native FC links up to 500 km. Combine with ISL Trunking up to 250 km. Enhanced Business Continuity Improved distances and performance for Metro Mirroring and Remote Backup 8 Gbit/sec Link Performance Up to 32Gbit/sec Trunk SAN Brocade Advanced Features Enhancing Performance and Availability SAN Improved Performance 8 Gbit/sec Inter-Switch Links (ISLs) 8 Gbit/sec links to next-gen devices Up to 32 Gbit/sec ISL Trunks Infrastructure Simplification Simpler SAN topology Increased SAN Availability Investment Protection 6/1/2015 Brocade Administration & Troubleshooting 3
  • 4. Feature Description SAN Benefits Hardware Enforced Zoning Prevents one device from communicating to another device it is not authorized to access. Enforced at the ASIC level Isolate devices from each other in the fabric such that there is no possible interaction between devices that are not explicitly defined. Enhance security and fault isolation. Fabric Watch Monitoring and alerting of key SAN statistics such as perf, error, security Enhance Business Continuity Improve application availability Alert admin of marginal/hard errors Advanced Performance Monitoring Highly granular SAN perf. monitoring to differentiate traffic between devices Enhance performance monitoring Simplify capacity planning Enable bill-back capabilities Advanced Security Robust encryption, authentication and authorization SAN policies Enhance Business Continuity Protect SAN from hackers Reduce user errors SAN SAN Brocade Advanced Features Enhancing Fault Isolation, Management and Security SAN 6/1/2015 Brocade Administration & Troubleshooting 4
  • 5. Brocade Advanced Features FCIP, FC Routing and Partitioning Feature Description SAN Benefits Virtual SAN Fabrics Partition the SAN fabric at port level to isolate mngt, admin roles, RSCNs, fabric events Allows separation of a SAN fabric into smaller fabrics that may overlap and share devices SAN LPARS Create HW logical partition at card level for independent managed directors in a single chassis Independent fabric services per partition allows true Segmentation of data, control and management traffic FCIP Tunneling Extends FC links up to 1000’s km over IP network. FC and FCIP Fast Write Enhanced Business Continuity. Improved Global Mirroring and Tape Vaulting FC Routing Route selected SAN traffic between SAN islands w/o merging fabrics and admin Improve storage resource sharing, Enables SAN consolidation, Maintains SAN security and fault isolation SAN SAN Production Test Backup Spare ports 6/1/2015 Brocade Administration & Troubleshooting 5
  • 6. Feature Description SAN Benefits FICON/FC Intermix FICON and FCP protocol intermix at the port level. Auto-sensing protocol and port speeds increase flexibility and provides greater port level granularity. N-Port Virtualization (NPIV) Allows multiple Linux Logical Partitions to share a single FCP channel Use fewer FCP channels on Mainframe Better channel utilization and simpler infrastructure High Integrity Fabrics (FICON Cascading) Binds Switches to Fabric for increased Security Enhanced Business Continuity. Improved distances/performance for Metro Mirroring and Remote Backup FICON CUP FICON in-band management of Directors from the Mainframe Simplify management and monitoring of directors Single point of management for Enterprises In-band Management z9 M48 Brocade Advanced Features FICON Specific Features M48 z9 xSeries zSeries DS4000 iSeries pSeries DS6000 DS8000 M48 M48 6/1/2015 Brocade Administration & Troubleshooting 6
  • 7. New Hardware from Brocade •10Gbps blade (FC #3470) • 6-ports for 10Gbps ISL connectivity using SW, LW and ELW XFP media • All Brocade directors now offer 10Gbps inter-switch links • Ideal for Long distance, high bandwidth BC/DR solutions • Support distances over 100km – iSCSI blade (FC #3460) • Supports iSCSI initiators and Fibre Channel target/initiator • 8 GigE ports and 8 FC ports (1/2/4 Gbps) • 64 iSCSI initiators per port, 512 initiators per blade • Up to 4 blades per SAN256B (2048 tested) 6/1/2015 Brocade Administration & Troubleshooting 7
  • 8. FOS Enhancements from Brocade •FC Routing and FCIP: • Increased Scalability for the SAN18B-R or FC-FCIP Routing Blade for M48 • Up to 12 Layer 2 switches and 12 FC Routers in the backbone fabric • FCIP tunnels offer QoS capabilities to ensure specific bandwidth • FC and FCIP FastWrite capabilities to enhance long distance mirroring solutions •Security management: • Complete migration of Secure Fabric OS features in the base FOS code • A new Security Administrator role for separation of security and fabric administrators • Tracking of logins to see breaches in the fabric. • Fabric authentication with standards-based improvement for device-to-fabric attachment (FC-SP DH-CHAP) • IPv6 capabilities for all management interfaces – IP over FC support: • Supporting Broadcast Zoning to reduce device interruption • Targeted for the film industry • When using FC as a common backbone for for host-to-storage and host-to-host file transfer. • Reducing overall production time and improving the integrity/security of digital data transfers. •Access Gateway feature added to SAN16B-2 to allow greater scalability and simplify connectivity 6/1/2015 Brocade Administration & Troubleshooting 8
  • 9. Brocade Switch Module for BladeCenter Connecting to McDATA SAN fabrics Brocade BladeCenter SAN Switch Modules 20072006 Access Gateway Interoperability Native McDATA Interoperability McDATA/QLogic FC Switch Modules End Sale Connecting BladeCenter to McDATA fabrics: •2007 – use the no-cost feature on Brocade called Access Gateway •2008 – use Access Gateway feature or the M-EOS (NI) mode Do not use the former McDATA/QLogic module •QLogic HW and firmware •Being End-of-Lifed •Not as good a solution as Brocade SAN Switch in long run 2008 6/1/2015 Brocade Administration & Troubleshooting 9
  • 10. NPIV & NPV • Two technologies that seem to have come to the fore recently are NPIV (N_Port ID Virtualization) and NPV (N_Port Virtualization). NPIV • What NPIV does is allow a single physical N_Port to have multiple WWPNs, and therefore multiple N_Port_IDs, associated with it. After the normal FLOGI process, an NPIV-enabled physical N_Port can subsequently issue additional commands to register more WWPNs and receive more N_Port_IDs (one for each WWPN). The Fibre Channel switch must also support NPIV, as the F_Port on the other end of the link would “see” multiple WWPNs and multiple N_Port_IDs coming from the host and must know how to handle this behavior. • N port identifier virtualization (NPIV) provides a means to assign multiple FC IDs to a single N port. This feature allows multiple applications on the N port to use different identifiers and allows access control, zoning, and port security to be implemented at the application level. The following figure shows an example application using NPIV. 6/1/2015 Brocade Administration & Troubleshooting 10
  • 11. 6/1/2015 Brocade Administration & Troubleshooting 11
  • 12. NPV – NPV introduces a new type of Fibre Channel port, the NP_Port. The NP_Port connects to an F_Port and acts as a proxy for other N_Ports on the NPV-enabled switch. Essentially, the NP_Port “looks” like an NPIV-enabled host to the F_Port on the other end. An NPV-enabled switch will register additional WWPNs (and receive additional N_Port_IDs) via NPIV on behalf of the N_Ports connected to it. The physical N_Ports don’t have any knowledge this is occurring and don’t need any support for it; it’s all handled by the NPV-enabled switch. • So why is this functionality useful? There is the immediate benefit of being able to scale your Fibre Channel fabric without having to add domain IDs, yes, but in what sorts of environments might this be particularly useful? Consider a blade server environment, like an HP c7000 chassis, where there are Fibre Channel switches in the back of the chassis. By using NPV on these switches, you can add them to your fabric without having to assign a domain ID to each and every one of them. Benefits of NPIV – Without NPIV, it’s not possible because the N_Port on the physical host would have only a single WWPN (and N_Port_ID). Any LUNs would have to be zoned and presented to this single WWPN. Because all VMs would be sharing the same WWPN on the one single physical N_Port, any LUNs zoned to this WWPN would be visible to all VMs on that host because all VMs are using the same physical N_Port, same WWPN, and same N_Port_ID. – With NPIV, the physical N_Port can register additional WWPNs (and N_Port_IDs). Each VM can have its own WWPN. When you build SAN zones and present LUNs using the VM-specific WWPN, then the LUNs will only be visible to that VM and not to any other VMs. 6/1/2015 Brocade Administration & Troubleshooting 12
  • 13. 6/1/2015 Brocade Administration & Troubleshooting 13 Ex., VMWare hosts & TOPS LPARS Ex., SANTap Module in Meritor
  • 14. Traditional Datacenter Brocade Cisco McData Benefits of Brocade Access Gateway No cost feature solves real-world SAN issues Access Gateway Access Gateway Access Gateway Access Gateway Access Gateway Access Gateway Concerns with Bladed Servers: •Difficult to connect to McDATA fabrics •Limited SAN scalability •Obscure Admin responsibility (SAN admin vs Server admin) Datacenter using Access Gateway Access Gateway offers these benefits: •Connects to McDATA Fabrics •Simplifies fabric & allows greater scalability •Clear Admin responsibility SAN FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch Change Modes SW AG 6/1/2015 Brocade Administration & Troubleshooting 14
  • 15. SAN16B-2 using Brocade Access Gateway No cost feature simplifies edge connectivity Typical Core-Edge Topology: •One to two core switches •3 to many edge switches •In this example: •1 SAN32B-3 core switch •3 SAN16B-2 edge switches •4 domains to manage SAN32B-3 SAN16B-2 SAN16B-2 SAN16B-2 Change 16B from Switch mode to AG mode SAN32B-3 Access Gateway Access Gateway Access Gateway Core-Edge Topology with Access Gateway: •Reduce the total number of domains •Connect to FOS, M-EOS and Cisco fabrics with NPIV •In this example: •1 SAN32B-3 core switch •3 SAN16B-2 edge Access Gateways •1 domain to manage NEW 6/1/2015 Brocade Administration & Troubleshooting 15
  • 16. AG Feature Support Statement •AG is supported on the 4Gb Brocade SAN Switch Module for BladeCenter and SAN16B-2: – FOS 5.2.1b or above for SAN Switch Module for BladeCenter – FOS 5.3+ for SAN16B-2 •Access Gateway feature requires that NPIV (N-port ID virtualization) capability be enabled on the external switches: – Brocade switches running FOS 5.1 with NPIV enabled – McDATA switches running EOS 9.0 with NPIV enabled • M-EOS 9.6 offers NPIV in base code – Cisco switches running OS 3.0 with NPIV enabled 6/1/2015 Brocade Administration & Troubleshooting 16
  • 17. Typical AG Deployment Scenarios •When to use AG? – Enterprise and large data centers where the fabric size is becoming a burden • Greater than 50 switches in an all b-type (Brocade) fabric • Greater than 30 switches in a fabric that includes m-type (McDATA) products – Connecting BladeCenter or 16B-2 to McDATA or Cisco SAN fabrics • Requires NPIV feature on the external switches – Customer’s SAN Admin group does not want an embedded switch in server products (i.e. BladeCenter) •When NOT to use AG? – Connecting SAN targets (such as storage) directly to switch module – Environments where customers require switch features not supported by AG • ISL Trunking • Long Distance support greater than 10km (using Extended Fabric license) Use AG Feature only when needed to help overcome an issue. Default mode for switch module should always be FC switch. 6/1/2015 Brocade Administration & Troubleshooting 17
  • 18. Brocade Director Roadmap Investing Today & Tomorrow 256M Director • 256-ports 1,2,4 Gig • Hard Partitions • Virtual Fabrics • 10Gbit/sec ISL • Open Trunking 140M Director • 140-ports 1,2,4 Gig • 10Gbit/sec ISL • Open Trunking • NPIV Next Generation Core • Common Management - EFCM • Native Interoperability (M-EOS) • Partitioning • Virtual Fabrics • Features: •1, 2, 4, 8 and 10Gbps •FCP, FICON, FCIP, •FCR, iSCSI, Apps Seamless growth for all director platforms 20072006 2008 & beyond 256B Director • 384-ports 1,2,4 Gig • FCP, FICON, FCIP • FC Routing • Virtual Fabrics • NPIV SAN32B-3BROCADE AG (BladeCenter) •10Gig blade (ISL) • iSCSI Blade FOS 6.x • 8Gbps blades • FCP and FICON • Native Interoperability (M-EOS) E/OS 9.6  NPIV  Security Ench  Interoperability enhancements and validation E/OS 9.7  IPv6 • Interoperability enhancements and validation E/OS 9.7 • IPv6 • Interoperability enhancements and validation  Interoperability enhancements and validation 6/1/2015 Brocade Administration & Troubleshooting 18
  • 19. Next Generation Core Platform Consolidating and Scaling Your Infrastructure Next Gen Core No rip-and-replace for Next Gen SAN. Seamless growth for all director platforms. NG Core Director connects to all 4Gb directors Edge Core 140M 256M 256B 2007: 4Gb products are just now becoming prevalent. End-to-End solutions with servers, disk and tape. 2008-2011: 4Gb directors will connect into hi-perf Core. Multiple protocols & very high bandwidth allow for greatest data center consolidation. 8Gb FC devices will start rolling out. 8Gb 8Gb 4Gb 6/1/2015 Brocade Administration & Troubleshooting 19
  • 20. Networking Fundamentals • FC Topology • Fabric Scalability • Initiator/Target relationship • Switch Ports • FC definitions • ISL Concepts • Cable selection • Host Support 6/1/2015 Brocade Administration & Troubleshooting 20
  • 21. Core-Edge Design the Fabric for your requirements Full mesh Cascade Mesh Configurations are appropriate when: Traffic patterns are evenly distributed Overall bandwidth consumption is low The maximum config is four to five switches Cascade Configurations are appropriate when: Traffic patterns are localized onto individual switches Core-Edge Configs are appropriate when: Fabric is likely to grow A flexible system is required because of unknown or undefined requirements Reliability is required – this type of config has been well-tested and is used in most production environments Consider the Fabric Port Count : The total number of FC ports in the Fabric, this would include ALL ports on ALL switches for A fabric, remember that you have dual fabrics, larger numbers should mean moving from cascade / mesh to core-edge 6/1/2015 Brocade Administration & Troubleshooting 21
  • 22. Fabric or Network Architectures • Types of architectures are: – Single-Switch – Cascade – Mesh – Core-Edge – Director 6/1/2015 Brocade Administration & Troubleshooting 22
  • 23. Cascade • Maximum hop count supported is three 6/1/2015 Brocade Administration & Troubleshooting 23
  • 24. Mesh Partial Mesh 6/1/2015 Brocade Administration & Troubleshooting 24
  • 26. Fabric Choices – What are they? FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch FC Switch 6/1/2015 Brocade Administration & Troubleshooting 26
  • 27. How many fabrics are show below? FC_Switch4 FC_Switch8 FC_Switch2 FC_Switch5 FC_Switch3 FC_Switch6 FC_Switch7 FC_Switch1 FC_Switch10FC_Switch9 6/1/2015 Brocade Administration & Troubleshooting 27
  • 28. Fabric Scalability Scale fabric size by replacing existing core with a larger core Scale # Core Switches Scale # ISLs Scale performance by adding ISLs or additional core switches Scale fabric size by adding switches Scale # ports Examples of Fabric Scalability 6/1/2015 Brocade Administration & Troubleshooting 28
  • 29. 1 2 3 Storage Storage New Firmware Storage Both Switches have New Firmware Design the Fabric for your requirements Serviceability using a Dual Fabric Design • Firmware upgrade can be done without I/O interruption if the following Rolling Upgrade is applied – Dual path is required from server and storage • Add new switches or upgrade current switches easily 6/1/2015 Brocade Administration & Troubleshooting 29
  • 30. Initiator/Target Relationship SCSI over Fibre Channel SCSI over TCP/IP (iSCSI) TCP/ IP TCP/ IP HOST (Initiator) Controller (Target) FC driver FC driver SCSI WAFL RAID iGroup SCSI driver File System Application SCSI Adapters Windows or UNIX Fibre Channel HBAs Data ONTA LUN iSCSI HBAs or Ethernet NICs Direct Attached Storage (DAS) Fabric/Network iSCSI driver Fibre Channel or Serial ATA Attach iSCSI driver 6/1/2015 Brocade Administration & Troubleshooting 30
  • 31. WWNN and WWPN Examples HBA WWNN (World Wide Node Name) 20:00:00:2b:34:26:a6:54 HBA WWPN (World Wide Port Name) 21:00:00:2b:34:26:a6:54 22:00:00:2b:34:26:a6:54 6/1/2015 Brocade Administration & Troubleshooting 31
  • 32. Switch Ports • E_Port - An expansion port connecting two switches to make a fabric. • F_Port - A fabric port to which an N_Port attaches. • FL_Port - A fabric loop port to which a loop attaches; needs FL card LED turned on. It is the gateway to the fabric for NL_Ports on a loop. • G_Port - A generic port that supports either E_Port or F_Port functionality. • L_Port - Node Loop port; a port supporting the Arbitrated Loop protocol. • N_Port - A fibre channel port in a fabric or point-to-point connection. 6/1/2015 Brocade Administration & Troubleshooting 32
  • 33. SAN Made Easy – Auto Discovery What do I want to be when I grow up? y/n Do you want to talk loop? G_Port I’m waiting for someone to talk to me… yes no Are you a switch or a fabric point-to-point device? F_Port fabric pt-to-pt E_Port switch y/n Is something plugged into the port? n o yes U_Port FL_Port 6/1/2015 Brocade Administration & Troubleshooting 33
  • 34. FC Definitions • ISL: Inter-Switch Link or a Switch-to-Switch Link; ISLs connect between two switch nodes to form E_ports. • Locality: The degree that I/O is confined to a particular switch or segment of a fabric. If two devices that need to communicate with each other are located on the same switch or segment, then these two devices are said to have high locality. If these same devices are located on different switches or segments of a fabric and these two devices need to communicate with each other, then these devices are said to have low locality. • Redundancy: When devices have two or more fabrics and multiple paths for a source to reach its destination the fabric is considered to have redundancy. This is critical so that when an initiator primary path fails, the secondary initiator path will be available so that initiator hosts can still communicate with their targets, at reduced performance. • Resiliency: The ability of a fabric to adapt to or tolerate a failure of a component. A fabric is said to have resiliency when it can tolerate 1 or more device failures from any component within the fabric, whether it is a switch, ISL, or HBA failure. • RSCN: Registered State Change Notification is the fabric mechanism that allows notifications to be sent to nodes if a change occurs within the fabric, ie. device going offline or online on a fabric port. • SCR: State Change Registrations are used by devices to register to receive RSCNs. 6/1/2015 Brocade Administration & Troubleshooting 34
  • 35. FC Definitions • ISL Oversubscription Ratio: Inter-switch Link Oversubscription Ratio is the ratio of device, or data input ports that might drive I/O between switches to the number of ISLs over which the traffic could cross. • ISL Oversubscription = Number of Host Nodes: Number of ISLs, or IO=Nhn:Ni. • Fan-in ratio: The ratio of storage ports to a single host port • Fan-out ratio: The ratio of host ports to a single storage port • Buffer-to-buffer credits: The number of buffer-to-buffer credits determines the number of Fibre Channel frames that a switch can transmit on a link at one time before requiring an acknowledgement back from the receiver. Performance degradation may occur if there aren’t enough credits available to sustain communication between switches. As the distance between switches increases, additional buffer-to-buffer credits are required to maintain maximum performance. Credits are allocated from a common pool of memory on the switch ASIC. Formula to approximate # of Credits required over long distance: • Buffer Credits = ((Distance in KM) * (Data Rate) * 1000) / 2112 • Data Rate = 1.0625 Mbaud for 1 Gbit/sec Fibre Channel • Data Rate = 2.1250 Mbaud for 2 Gbit/sec Fibre Channel • Data Rate = 4.2500 Mbaud for 4 Gbit/sec Fibre Channel 6/1/2015 Brocade Administration & Troubleshooting 35
  • 36. Best Practice – ISL Oversubscription • A 7:1 ISL oversubscription ratio is aligned with an industry average of 6:1 fan-out. The trend in the storage industry is that the hosts to storage ratios are increasing, as is the performance of storage devices. A 7:1 ISL oversubscription ratio should be targeted in SAN designs, with the ISL oversubscription ratio being adjusted higher or lower to meet particular performance requirements. While this ISL oversubscription ratio is conservative, it is felt that the cost of not having enough performance and having to reshuffle devices and ISLs is much greater than the cost of having a few extra spare ports that can be used to connect SAN devices at a later point in time. • Rule of thumb: The higher the ISL oversubscription ratio, the lower the performance and conversely, the lower the ISL oversubscription ratio, the higher the expected I/O performance. An ISL oversubscription ratio of 3:1 results in high performance and fewer available ports while an ISL oversubscription ratio of 15:1 results in lower potential performance and additional available ports reserved for devices. With the advent of 4Gbps ISLs, higher oversubscription ratios can exist while maintaining more than adequate bandwidth (since bandwidth is doubled per ISL) and higher device port counts for 2Gbps devices. 6/1/2015 Brocade Administration & Troubleshooting 36
  • 37. FC SAN Host Support Host ClusterOS Vendor Multipath File SystemHBA Native HP PVLinks / VERITAS DMP MC ServiceGuard / VERITAS VCS Native MSCS VERITAS VCS Emulex / Qlogic HACMP QLogic QLogic QLogic QLogic QLogic QLogic Emulex QLogic VMware MSCS VirtualCenter (VMotion) Volume Mgr LVM VERITAS VxVM NSS VMware NTFS VERITAS VxFS JFS/2 Raw LVM / VERITAS VxVM JFS/ HFS Raw VERITAS VxFS ext3 / ext2 / Reiser / GFS* ext3 ext2 Reiser VMFS 2.x Raw LVM Under Test Oracle 9i, 10g, RAC Novell Clusters SANpath / MPIO VERITAS DMP / MPxIO* VERITAS VCS / Native SUN Cluster* Oracle 9i, 10g RAC / RH Cluster Suite* * (via PVR) MPIO NetApp DSM / VERITAS DSM for MPIO* MMC / VERITAS VxVM* Emulex / Native 4Gb 6/1/2015 Brocade Administration & Troubleshooting 37
  • 38. Cable Distance Chart 6/1/2015 Brocade Administration & Troubleshooting 38
  • 39. Zoning and Troubleshooting • Important Commands • Zoning (how to zone) • Zoning Best practices • Troubleshooting Procedure • Basic Troubleshooting 6/1/2015 Brocade Administration & Troubleshooting 39
  • 40. FC switch tools – provided by switch manufacturer (Brocade) – switchshow • Displays status of the FC switch and all its ports • Show FC nodes currently logged into the switch (depends on FC zones, if any) – cfgshow • Show zones currently available on the FC switch • Shows information about the current FC configuration and which zone(s) are enabled – supportshow: Displays switch information for debugging purposes – ssshow: Displays information about the name server – nsshow: Verifies that clients are logged into the name server – fabricshow: Displays fabric membership information – configure • Changes switch configuration settings. • Switch need to be offline to run this command 6/1/2015 Brocade Administration & Troubleshooting 40
  • 41. FC switch tools – provided by manufacturer (Brocade) (cont.) – alicreate, zonecreate: Create aliases and zones – cfgcreate, cfgsave, cfgenable,cfgshow: Manage zone configs – version: Displays firmware version information – portshow, portcfgshow, porterrshow, portLogDumpPort, portenable, portdisable • Manage ports – diagshow: Displays switch diagnostics – webUI: Web GUI available by browsing to the switch ip adress – nodefind, nszonemeber – To search wwns across fabric and inside zoning6/1/2015 Brocade Administration & Troubleshooting 41
  • 43. Domain on Brocade Switches • Make sure that the Domain ID is set to a different value on all switches in a fabric • Example : if there are two fabrics in solution then the Domain ID on each switch in Fabric A should be set to an increasing odd number and for Fabric B set each Domain ID to an increasing even number – Fabric A – 11, 13, 15, 17, etc. – Fabric B – 10, 12, 14, 16, etc. Note: if HP-UX is involved then skip 8, this ID was used for Loop Configs • Name Server – service in fabric that provides directory services and info about ALL devices in the fabric 6/1/2015 Brocade Administration & Troubleshooting 43
  • 44. Define and Implement Zoning How do I manage Zoning? • Manage zone physically or Logically • Three components to the zone information – One or more devices are placed in a zone – One or more zones are placed in a configuration – One and only one config is made the effective • Soft Zoning: Name Server assisted – Name Server restricts visibility – Always available when zoning enabled – No reduction in performance • Hard Zoning: Hardware Enforced – Available when certain rule checking criteria are met through hardware logic checking. – Provides additional security in addition to Soft zoning – Prevents illegal access from “bad” citizens. – No reduction in performance with hard-Port level zoning. – Available using port or WWN with Brocade 2 Gbit/sec 6/1/2015 Brocade Administration & Troubleshooting 44
  • 45. Define and Implement Zoning Zoning Setup Guidelines • Create a detailed diagram of the fabric, showing all the switches with their ISLs • Create a “blowup” diagram of each switch in the fabric to account for devices • Account for private loop devices if they exist • There are special considerations for mixed 1 Gbit/sec and 2Gbit/sec based fabrics • For security reasons, consider disabling a port if the zoned fabric is going to contain unused ports, with nothing connected to them • Configure one zone at a time and then test it – Do not create all the zones at once; it will be troublesome to debug – After the first zone is setup in the fabric, plug in devices and then test the connections to confirm that everything is functioning properly – This process may seem a little tedious, but it will save time and money trying to debug this after creating all the zones and then plugging in the devices 6/1/2015 Brocade Administration & Troubleshooting 45
  • 46. Define and Implement Zoning Implementing Zoning • Naming convention – There typically of three types of devices, server HBA, the storage port, and the tape port. – These will have an alias. • SRV – for servers • STO – for Storage • TPE – for Tape • For example, SRV_MAILPROD_SLT5 – a server, hostname “mailprod”, in PCI slot 5 – Keep names as small as possible to conserve space in zone database – Minimize duplication in alias definitions where possible – Keep zoning database as clean and accurate as possible • Fabric Name – Fabric name is the name that the fabric is generally known by. – PROD configuration is to easily identify the configuration that can be implemented and provide the most generic services. – BACKUP_XX, TEST_XX may be used 6/1/2015 Brocade Administration & Troubleshooting 46
  • 47. Define and Implement Zoning 10 Zoning Rules - Brocade 1) If security is a priority, then a Hard Zone-based architecture coupled with Hardware Enforcement is recommended 2) Using aliases, though optional, should force some structure when defining your zones. 3) Add Secure Fabric OS® into the Zone Architecture if extra security is required. 4) If a SilkWorm 12000 is part of the fabric, then use it to administer zoning within the Fabric 5) If QuickLoop is required for legacy devices and the switch is running Brocade Fabric OS v4.x: – QuickLoop / QuickLoop zones cannot run on switches running Brocade Fabric OS v4.x. – QuickLoop Fabric Assist - Brocade Fabric OS v4.x cannot have a Fabric Assist host directly connected to it. 6/1/2015 Brocade Administration & Troubleshooting 47
  • 48. Define and Implement Zoning 10 Zoning Rules - Brocade 6) Before implementing a zone run the Zone Analyzer and isolate any possible problems. 7) Before enabling or changing a fabric configuration, verify that no one is issuing I/O in the zone that will change. 8) Changes to zoning should be done during preventative maintenance to minimize any potential disruption. 9) After changing or enabling a zone configuration, confirm that nodes and storage are able to see and access one another. 10) LUN Masking should be used in conjunction with fabric zoning for maximum effectiveness. 6/1/2015 Brocade Administration & Troubleshooting 48
  • 49. Zoning Example – Single Fabric zone1 zone2 Storage Host1 Host2 Host3 Host4 FC Fabric What is needed on the hosts systems and on which systems is it needed in this configuration? 6/1/2015 Brocade Administration & Troubleshooting 49
  • 50. Simple Troublshooting Cmds • cfgactshow |grep cfg =====>to check the current zoneset config • • zoneshow *zone_name* <-- To search in zones • alishow *alias_name* <-- To search in alias • cfgactshow |grep <host_name> To search zone in active zoneset • switchshow |grep <wwn> <-- to check flogi information • nszonemember <wwn> <-- FCNS database and to check zoned' objects • portshow <interface> <-- To Show interface • portloginshow <interface> <-- To list port logins • fabricshow <-- To check switch topology 6/1/2015 Brocade Administration & Troubleshooting 50
  • 51. Zoning Ex., • Step by step zone creation and activation • cfgactvshow |grep cfg - check the current activated zoneset or configuration • alicreate "S0ADCF1S2P8_HBA2_PROD", "c0:50:76:00:51:3e:00:06" <-- To create fcalias • alicreate "S0ADCF1S2P8_HBA2_LPM", "c0:50:76:00:51:3e:00:07" • • zonecreate "USCLSITPS002_S0ADCF1S2P8_HBA2_CX1571_SPA0", "S0ADCF1S2P8_HBA2_PROD; S0ADCF1S2P8_HBA2_LPM; CX1571_SPA0" <-- To create zone • zoneshow *USCLSITPS002_S0ADCF1S2P8_HBA2_CX1571_SPA0* <--- Check the zone is created on the switch • • cfgsave <-- To save configuration file • • cfgadd "Fab2_VF70_10212013", "USCLSITPS002_S0ADCF1S2P8_HBA2_CX1571_SPA0" <-- To add zone into existing configuration (zoneset) • cfgsave <-- To save latest configuration • • cfgenable "Fab2_VF70_10212013" <-- To activate configuration file (activate zoneset) 6/1/2015 Brocade Administration & Troubleshooting 51
  • 52. Troubleshooting Ideas 6/1/2015 Brocade Administration & Troubleshooting 52
  • 53. Q & A 6/1/2015 Brocade Administration & Troubleshooting 53
  • 54. 6/1/2015 Brocade Administration & Troubleshooting 54