MB - V Fas6000 Sa600 C 013

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series
For NetApp Authorized Service Engineers

Doc Rev -013
README FIRST
README FIRST
This AP has been updated to include commands for systems running "Cluster-Mode" (C-Mode) ONTAP.
The login name for C-Mode systems is "admin", not "root".
The ONTAP version and mode is listed in your dispatch!
C-Mode: Has two console command shells, clustershell and nodeshell. The default shell is clustershell.
IF clustershell, the console prompt includes a double colon (:: ). Ex(1): cluster ::> Ex(2): cluster ::storage>
To switch from clustershell to nodeshell, enter 'run local' at the ::> prompt, then the double colons (::) are
removed. To exit nodeshell, enter 'exit' or Ctrl-D.
From clustershell, nodeshell commands can be entered by prefacing the 7-Mode command with run local".
Ex: cluster::> run local sysconfig -v Note, all 7-Mode commands are not supported in C-Mode.
Known Bugs/Issues - Bug Table and Notes Below

Bug
489060 1
665675 2
Description
NDMP, Qtree-SnapMirror, Vol-SnapMirror or SnapVault processes can hang TO/GB
During POST some systems may report a transient error regarding bad DIMMs
First Fixed Release

See Note 1
See Note 2
Bug Notes:
1 The AP will cover asking the customer if they are running these processes. If so, there is a link how to disable them.
2 The AP will cover a way to address and troubleshoot this issue should it arise.
Link to Statement of Volatility by Platform is: http://support.netapp.com/info/web/ECMP1132988.html
AP doc rev is at top of page - If using hard-copy for secure site, be sure to print all the linked documents in this AP.

Page 1 of 19
SECTION OUTLINE for the SA600 and V-FAS6000 Series Motherboard (MB) Replacement
This procedure will take 90-180 minutes
I. Appliance/Motherboard Tray Visual Checks
II. Node Pre-Checks
III. Node State Check and Shutdown Procedure
IV. Capture the Current System Configuration
V. Remove the Cables, Trays - Cable Management and FC
VI. Extract the System Tray and Remove the Side Rails
VII. Move/Inspect the DIMMs
VIII. Move the onboard Ethernet SFPs and the PCI Cards
IX.
X.
XI.
XII.
XIII.
XIV.
XV.
XVI.
Reinsert the System Tray, FC and Cable Trays and cables

Set Date and Time
Verify FC Adapter Configuration
Verify Disk List
Run Diagnostics (40-60 minutes)
Boot PROM Variable Checks
Boot the Operating System - 'giveback' if applicable
Submit logs and Part Return
I. SA600 and V-FAS6000 Series: Appliance/Motherboard Tray Visual Checks

Step Action Description
1 Visually verify if you are working on correct model and READ the NOTES below.
System Serial Number
SA600 or V-FAS Model Number
Fig 1
PS-1 AC
Switch
Fig 2
PS-2 AC
Switch
PCI Slot Numbering
6u
Activity, Status (!), Power LEDs

The Activity LED will be OFF on down system
Fig 3
6 Ethernet
Ports
( e0a-e0f )
RLM Port
FC Port Tray ( 0a-0b, 0c-0d, 0e-0f, 0g-0h )
IOIOI (Console) Port
Notes:
1. This Action Plan covers FAS and V-Series Controller running
ONTAP 7-Mode or Cluster-Mode OS.
a. V-Series Steps denoted with
Fig 4
2. This procedure will take 90-180 minutes.
3. This Action Plan needs to be followed in step order.
SA600,
V-FAS6000 Series
MB Tray
4. FC port configuration, disk list and the system date are captured
prior to removing the original Controller.
5. Many parts need to be moved from the Original Controller to the
Replacement Controller Module.
6. System variables; date-time, and FC port configuration must be
verified before rebooting the system.
Go to Section II, "Node Pre-Checks" on next page.

Page 2 of 19
II. SA600 and V-FAS6000 Series: Node Pre-Checks

1 Verify the "Order Reference 8xxxxxxxxx number on the RMA packing slip is the same as the Part Request (PREQ) number listed
in your dispatch notes.
2 Adhere to anti-static precautions. (A paper ESD strap is included inside the RMA box if you don't have your own)
3 Remove the replacement Motherboard Tray from the anti-static bag and examine the housing and connector for damage.
III. SA600 and V-FAS6000 Series: Node State Check and Shutdown Procedure
1 Always capture the nodes console output to a text file, ex: NetApp-dispatch-num.txt, even if using the end-user's computer.
To review the Job Aid on how to connect to console (IOIOI) port and serial emulator options, here > Console Attach
2 Walk up visual observation checks - Identifying the correct node to be serviced, i.e.. the "target" node.
NOTE: A "controller takeover" will turn the Status ( ! ), middle LED from GREEN to AMBER on the target and partner nodes.
) LED, left most in figure, should be OFF on the target node. If it is flashing, this node is still up - or you
a) The "Activity" (
are looking at the partner node, not the target node. See STOP message.
The
(Activity) LED should be off.
No flashing on the target node.
Fig 5
S
T
O
P
3
4
5
! (Status) LED will turn from

GREEN to AMBER on the target
and partner node if a controller
"takeover" has occurred.
WARNING for HA (Active-Active) configurations:

If the failure has caused the partner to takeover, you may have been dispatched on the partner's controller serial number, not the
failed (target) controller. If the LCD panel displays the word "takeover" or "cf takeover", this is the "partner" node, not the
target node. See Explanation of LCD panel messages in the link in Step 3.
The target node needs to be halted or "waiting for giveback" for this part replacement. LCD Detail >> LCD Panel
Confirm the console response is "LOADER>" or "Waiting for giveback". If it is "Waiting for giveback" continue with step 5
below. If it is "LOADER>", go to Section IV.
Continue with section III on next page.

Page 3 of 19
III. SA600 and V-FAS6000 Series: Node State Check and Shutdown Procedure (cont.)
6 If the console response is: "Waiting for giveback..." follow steps 6a-6c.
a) At the "Waiting for giveback " prompt, Press: Ctrl-C
b) At the message: "Do you wish to halt this node rather than wait [y/n]? " Enter: y
c) After the system drops to the LOADER> prompt, continue with step 7.
Step 6): Hitting Enter
Waiting for giveback...(Press Ctrl-C to abort wait)
displays this prompt
^C
Step 6a): Press CTRL-C
This node was previously declared dead.
Pausing to check HA partner status ...
partner is operational and in takeover mode.
You must initiate a giveback or shutdown on the HA
partner in order to bring this node online.
Information on
Partner Status
The HA partner is currently operational and in takeover mode.

This node cannot continue unless you initiate a giveback on the partner.
Once this is done this node will reboot automatically.
waiting for giveback...
Do you wish to halt this node rather than wait [y/n]? y
Halting...
Uptime: 3m10s
System halting...
.....
.....
Step 6b): Enter: y
LOADER>
7
IF Cluster-Mode, continue with step 7(a) otherwise skip to Section IV on next page.
a) After the target system drops to the LOADER-A|B> prompt, login to the partner and check if the auto-giveback option is
enabled by entering the following command. You can copy-n-paste the command syntax.
Cluster-Mode
cluster::> sto fa show -node local -fields auto-giveback
IF enabled C-Mode
node
-------------Node-B
will show:
auto-giveback
------------true
b) Disable the auto-giveback option if enabled. (copy-n-paste)

Cluster-Mode
cluster::> sto fa modify -node local -auto-giveback false
8
Go to Section IV, "Capture the Current System Configuration" on next page.

Page 4 of 19
IV SA600 and V-FAS6000 Series: Capture the Current System Configuration

NOTE Confirm the "console" output is being saved to a text file. It will be needed later in this action plan.
1 The date and time is stored in the system PROM in Greenwich Mean Time, (GMT) also known as Universal Time Clock, (UTC).
At the LOADER> prompt, enter: show date Record on paper the system's GMT time and the local time to determine the
number of hours (and minutes) the local time is ahead or behind GMT.
LOADER> show date
Current date & time is: 08/30/2011
2
10:07:00
Step 1): Enter: show date
Enter: printenv This command displays (and captures) all boot environmental variables.
LOADER> printenv
Enter: printenv
An example of a "printenv" output is here > printenv-C-all.pdf

3
Follow Steps 3A-C if this a V-series Filer (V6000) or has SAN attached Tape Drives < ask customer. If not, go to step 4.
Additional Steps for V-Series Filer
A. Open up the console log.
B. Use the "Find" function to search for variables prefixed with: "fc-no" - Do not rely on a visual search.
C. Identify these variables from the printenv output in the console log:
(i) fc-non-array-adapter-list,
(ii) fc-no-mgmt-ports OR fc-no-mgmt? NOTE - Only one of these two variables may be used.
An example of "printenv" output highlighting the variables is here >>> printenv.pdf
Continue with Section IV on next page.

Page 5 of 19
IV SA600 and V-FAS6000 Series: Capture the Current System Configuration (cont.)
5 From the LOADER> prompt enter autoboot to initiate a prom bootstrap.
a) When this message appears: "Press CTRL-C for Boot Menu" , press CTRL-C (^C) to load the "Boot Menu". After about
30-40 seconds, the "Maintenance menu" will appear.
NOTE
If the original MB fails to boot to the Maintenance menu due to an error, skip to Section V.
b) For ONTAP 7.x, ONTAP 8.0.x 7-mode and ONTAP 8.1 (7-Mode, C-Mode), refer to left menu, enter 5 for "Maintenance
mode boot".
c) If asked "Continue with boot?" Answer: y
ONTAP 7.x , 8.0.x 7-Mode and ONTAP 8.1 (7,C-Mode)
ONTAP 8.0.x Cluster-Mode Only
LOADER> autoboot
Loading
X86_64/freebsd/image1/kernel:0x100000/3375736
0x538280/3221872
Step 5: Enter autoboot
.....
Copyright (C) 1992-2010 NetApp.
All rights reserved.
*******************************
Step 5a): Wait for
*
*
this message, then
* Press Ctrl-C for Boot Menu. *
hit ^C (CTRL-C)
*
*
*******************************
^CBoot Menu will be available.
LOADER> autoboot
Loading
x86_64/freebsd/image2/kernel:....0x100000/3386664
0x53b000/3222096 0x84da50/1190096
Step 5: Enter autoboot
.....
NetApp Data ONTAP 8.0.1 Cluster-Mode
Copyright (C) 1992-2010 NetApp.
All rights reserved.
*******************************
Step 5a): Wait for
*
*
this
message, then
hit ^C (CTRL-C)
*
*
*******************************
^CBoot Menu will be available.
Please choose one of the following:

How would you like to continue booting?
(1) Normal Boot.
(2) Boot without /etc/rc.
(3) Change password.
(4) Clean configuration and initialize all disks.
(5) Maintenance mode boot.
(6) Update flash from backup config.
(7) Install new software first.
(8) Reboot node.
Selection (1-8)? 5
Step 5b): Enter 5
You have selected the maintenance
boot option:
.....
.....
In a High Availablity configuration, you MUST
ensure that the partner node is (and remains) down,
Step
5c): If is
thismanually
node hasdisabled
a partner on
node
or that
takeover
thethis
message
will be displayed.
Answer: y software
to the
partner
node, because
High Availability
is
not started
or fully
enabled
in Maintenance mode.
"Continue
with boot?"
question.
FAILURE TO DO SO CAN RESULT IN YOUR FILESYSTEMS
BEING DESTROYED
NOTE: It is okay to use 'show/status' sub-commands
such as 'disk show or aggr status' in Maintenance
mode while the partner is up
Continue with boot? yes
.....
.....
*>
maintenance mode console prompt
Continue with Section IV on next page.
(normal)
Normally
(install)
Install new software first
(password [<user>]) Change user password
(setup)
Run setup first
(init)
Initialize disks and create
flexvol
(maint)
Boot into maintenance mode
(syncflash)
Update flash from backup
config
(reboot)
Reboot node
Step 5b):
Please make a selection: maint
Enter maint
.....
.....
In a High Availablity configuration, you MUST
ensure that the
partner node is (and remains) down, or that
takeover is manually
disabled on the partner node, because High
Availability
software is not started or fully enabled in
Maintenance mode.
FAILURE TO DO SO CAN RESULT IN YOUR FILESYSTEMS
BEING DESTROYED
NOTE: It is okay to use 'show/status' sub-commands
such as
'disk show or aggr status' in Maintenance mode
while the partner is up
.....
*>
maintenance mode console prompt

Page 6 of 19
IV SA600 and V-FAS6000 Series: Capture the Current System Configuration (cont.)
7 From the > *> prompt enter fcadmin config to log the configuration of the integrated FC host adapters.
a) Note the "0a-0h" Adapter ports to see if configured as a "target" adapter. If so, it will need to be configured later.
*> fcadmin config
STEP 7: Enter: fcadmin config
Example Only
Local
Adapter Type
State
Status
--------------------------------------------------0a
initiator CONFIGURED.
online
0b
online
0c
initiator CONFIGURED
online
0d
online
0e
offline
0f
offline
0g
target
CONFIGURED
offline
0h
target
CONFIGURED
offline
In this Example:
Adapters, 0g and 0h will
need to be configured as
a "target" adapter on the
replacement MB Tray.
Enter: storage show disk -p to capture Primary and Secondary path information.
Enter: storage disk show -x to capture disk models.
The "storage show disk -x" sample console output is below. Later in this procedure we will verify the same number of disks are
NOTE listed after the MB is replaced.
For example of a V-Series 'storage show disk -x', click here V-Series storage show disk -x
10 Examine the disk listing - See Text Box Step 10 below.
11 At the > *> prompt enter: halt (after prom initialization the console will display the "LOADER>" prompt)
8
9
*> storage show disk -x
Step 9: This command prints out all the attached disks.
Example Only
A typical listing may display many more disks and SAS

& FC adapters than this partial listing.
DISK SHELF BAY SERIAL

----- --------- -------------------0a.16
1
0 JFVMV5BC
0a.17
1
1 JFVLWV4C
0a.18
1
2 JFVLD57C
0a.19
1
3 JFVLXS8C
0a.20
1
4 JFVKT1GC
0a.21
1
5 JFVKSMVC
0a.22
1
6 JFVLL1LC
0a.23
1
7 JFVHXBGC
0a.24
1
8 JFVLLDVC
0a.25
1
9 JFVHWXUC
0a.26
1
10 JFVJURYC
......................
......................
0b.34
2
2 3LN6QL4F00009912B5LN
0b.35
2
3 3LN6QKH900009911MFEM
0b.36
2
4 3LN6QKH700009911ME6F
0b.37
2
5 3LN6QK8P00009912BKAK
0b.38
2
6 3LN6QKJ200009911MFL9
0b.39
2
7 3LN6QLW900009911JRLH
0b.40
2
8 3LN6QKHM00009911MBU3
0b.41
2
9 3LN6QKD000009912B6FG
0b.42
2
10 3LN6QJH100009912B640
0b.43
2
11 3LN6QJK800009912B45M
0b.44
2
12 3LN6QL5400009912B5R3
.......................
.......................
*> halt
12
VENDOR
-------NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
MODEL
---------------X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
X278_HVIPB146F15
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
NETAPP
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
X278_S15K5146F15
REV
---NA01
NA01
NA01
NA01
NA01
NA01
NA01
NA01
NA01
NA01
Step 10: Under the DISK heading,
NA01
all FC and SAS disks are listed. In
this example '0a and 0b' FC
Adapters only are shown. Often
NA02
there are more. After the MB is
NA02
replaced, confirm the same number
NA02
of disks are listed.
NA02
NA02
NA02
NA02
NA02
NA02
NA02
NA02
Step 11: Enter: halt
Go to Section V, "Remove the Cables, Trays - Cable Management and FC" on next page.

Page 7 of 19
V SA600 and V-FAS6000 Series: Remove the Cables, Trays - Cable Management and FC
1 Confirm the console response is "LOADER>". Next turn off both power supplies (Refer to Fig 2 in Section I.) on the back of the
head unit and disconnect the power cords from the power supplies.
2 Label each cable connector with its port number and PCI slot if applicable and then unplug all the cables.
NOTE If possible keep the cables in the cable clips on the cable tray to keep them in the correct position for reconnection.
3 If cable management tray is installed, remove it by gently pushing in the sides of the tray at the arrows and lifting the tray up. Ref
Fig 6a-b.
Fig 6b
Fig 6a
Cable
Management
Tray
4
5
Step 4: Pinch
both sides "in"
to remove.
Loosen the thumbscrew on each end of the Fibre Channel tray, and then swing the thumbscrew retainer bars away from the Fibre
Channel tray. Ref Fig 7 below.
Extract the Fibre Channel tray while supporting the bottom, out of the chassis and set it on an antistatic surface.
Fig 7
Thumbscrew
retainer bar
Go to Section VI, "Extract the System Tray and Remove the Side Rails" on next page.

Page 8 of 19
VI SA600 and V-FAS6000 Series: Extract the System Tray and Remove the Side Rails
1 Fig 8: Loosen the six (blue) thumbscrews that secure the Motherboard Tray to the chassis.
CAUTION The Motherboard Tray is heavy, 35-42 lbs (16-19Kgs) Use grip handles on each side when extracting from the rails.
2 Fig 9, 9a: Lower the handle - Release the slide by pressing the locking tabs on each rail and extract the Tray.
3 Fig 9: Use grip handles on each side of Tray to fully extract
4 Fig 9b: Remove the side rails from the motherboard tray by unscrewing the three screws on each rail.
Fig 8
Fig 9a
Step 2: Lower Cam Handle and pull

to extract the Tray. Press locking
tabs on the each rail to remove the
Tray from the chassis
Step 1: Loosen the 6 Blue

Thumbscrews to open up the system
Fig 9
Fig 9b
Step 4: Remove the 3 screws and remove the

left and right side rails on the original Tray
Step 3: Use grip handles on each side

of Tray to fully extract. (HEAVY)
VII. SA600 and FAS6000 Series: Move/Inspect each DIMM

NOTE Adhere to anti-static precautions. (A paper ESD strap is included inside the RMA box if you don't have your own)
1 See Fig 10 for location of the memory DIMMS.
Total Number of DIMM Slots - FAS6030 and FAS6040 have 8 DIMMs; SA600, FAS6070 and 6080 have 16 DIMMs.
Handle the DIMM by the board edges- Do not touch the gold
Fig 10
CAUTION:
contacts. Oil from the skin contaminates the connection.
2 Move the Memory DIMMs, one at a time, from the Controller
Module and install each one into the same slot in the replacement
Controller Module. See next step for proper insertion technique.
3 Align DIMM with key slot and insert the DIMM straight into the slot.
Use both thumbs on the outer edge of DIMM, Fig 11 and press
evenly but firmly on the DIMM. It should "snap" in. If not, eject it and
re-insert until it snaps in place.
Fig 11
DIMMs
DIMM Layout
FAS6030 and FAS6040 use only 8 Memory DIMMs.
SA600, FAS6070 and 6080 use 16 Memory DIMMs.
4
Go to Section VIII, "Move the onboard Ethernet SFPs and the PCI Cards" on next page.

Page 9 of 19
VIII SA600 and V-FAS6000 Series: Move the onboard Ethernet SFPs and the PCI Cards
1 Remove any SFP/GBICs, one at a time, installed in the Ethernet ports from the original Chassis and install each one into the
same Ethernet port location in the replacement Chassis.
2 Loosen the blue thumbscrew on the primary and secondary PCI retaining brackets and lift the retaining bracket up and off the
chassis frame. Ref Fig 12 below.
NOTE Label each card with It's PCI slot number before removing.
3 Move one card at a time into the new TRAY in same slot.
NOTE A RED LED may be flashing on the NVRAM card which may be installed in slot 1 or slot 2
4 Install the primary and secondary PCI adapter retaining brackets.
Loosen both blue thumbscrews to remove brackets

Primary bracket
Secondary bracket
PCI card
Fig 12
NOTE: PCI cards need to

be marked with their slot
number before they are
removed.
Insert into the same slot
on the replacement Tray
IX. SA600 and V-FAS6000 Series: Reinsert the System Tray, FC and Cable Trays and reconnect the cables
Step
1
2
3
4
Action Description
Attach the slide rails that were removed from the original motherboard tray to the sides of the replacement motherboard tray.
Reinsert the motherboard tray and raise the cam lever and tighten the 6 thumbscrews. (Reference pictures in Section VI)
Reinsert the Fibre Channel Tray and tighten the 2 thumbscrews. (Reference pictures in Section V)
Re-attach the Cable Management Tray if removed. (Reference pictures in Section V)
Do not connect the FC cables in ports 0a-0h as the "maintenance mode" boot or the Fibre Channel Diags may fail. A "step" has
STOP
been added after the FC ports are configured in Section XI to reconnect the cable(s).
5 Cables: Fully insert each cable that was removed to its proper port until it clicks in. Test by pulling on them. Especially the FC and
any ports on SAS cards if equipped!
6 Re-attach laptop to the console port and capture the display output even if the end-user is doing it.
7 Go to Section X, "Set Date and Time" on next page.

Page 10 of 19
X. SA600 and V-FAS6000 Series: Set Date and Time

Step
1
2
3
Action Description
Re-attach laptop to the console port and capture the display output even if using the end user's computer.
Reconnect the power cords and turn the power supplies on.
IMMEDIATELY after the console message "Starting AUTOBOOT press Ctrl-C to abort" is displayed, press Ctrl-C (^C) key
a couple times to abort the autoboot. See Console output example below.
Phoenix TrustedCore(tm) Server
Copyright 1985-2006 Phoenix Technologies Ltd.
.......
"." =@Deleted
lines to save space
CPU Type: Intel(R) Xeon(R) CPU
L5410
2.33GHz
STEP 3: Press "CTRL-C"
Starting AUTOBOOT press Ctrl-C to abort...
Loading x86_64/freebsd/image1/kernel:....0x100000/3386728 0x53b000/3222096
.....
LOADER>
IF you miss the window to abort the autoboot, look for this message: "Press CTRL-C for boot menu" and complete steps 4a-4b,
otherwise if at the "LOADER>" prompt, skip to step 5.
a. Immediately Press ^C (CTRL-C) to access the "Boot menu".
b. Next, drop to the "LOADER>" prompt from the Boot Menu by following the linked process > here
At the LOADER-> prompt enter: show date to display the date and time in GMT (UTC) on the new MB Tray.
LOADER-A> show date
Current date & time is: 10/14/2010 16:36:50
LOADER>
Time is displayed in in 24hr mode
The original motherboard's GMT time and local time should have been recorded in Section IV. If you don't have it, you can obtain
the GMT time from the partner node, or another NetApp appliance or any Unix Server using: "date -u". (The "-u" option displays
the time in GMT/UTC) The new motherboard's Real Time Clock (RTC) must be set within 2 minutes of the time displayed (which
is GMT time) for users to be able to re-connect to this appliance.
NOTE Detailed instructions for another method of obtaining the time in GMT and setting date and time is here>
RTC Check
7 To set the time issue: set time hh:mm:ss Set the time in GMT using 24 hour format - Do not set the time to local time.
NOTE If this maintenance period spans across the midnight hour in GMT time, the DATE will also need to be set.
8 To change the date, issue: set date mm/dd/yyyy (mm = 2-digit month, dd = 2-digit Day, yyyy = 4-digit Year)
9 If the date or time was changed, issue: show date again to verify the GMT date and time are correct.
10 Update Firmware on the Replacement MB Tray:
At the LOADER-A|B> prompt, Enter : update_flash to copy the firmware on the Compact Flash card to the motherboard's
flash PROM.
NOTE
If a message displays that states: "Current firmware version is newer than provided image", skip to step 11.
6
LOADER> update_flash
STEP 10: Enter: update_flash to
New BIOS Version: 1.9.0
New Loader Version: 1.8
update the flash PROM
Saving Primary Image to Secondary
Updating Secondary Boot Flash
Programming .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+ done. 2031616
bytes written
Updating Primary Boot Flash
Programming .+.+.+.+.+.+.+.+.+.+.+.+.+.+ done. 917504 bytes written
LOADER>
11
Go to Section XI, "Verify FC Adapter Configuration" on next page.

Page 11 of 19
XI. SA600 and V-FAS6000 Series: Verify FC Adapter Configuration

1 Boot into maintenance mode by following steps > here
(i) If the system reports the battery voltage is too low or a critical failure, do NOT proceed - Do NOT bypass the system stop.
Engage NGS for assistance. Under NO CIRCUMSTANCES bypass the system halt to "boot" the system on
S
T
O
P
a NVRAM battery voltage issue.

(ii) If the system fails to boot to the Maintenance menu, confirm the memory DIMMS and all PCI cards are properly seated.
Engage NGS for assistance.
(iii) If the system fails to boot, reporting DIMM errors or ECC DIMM errors during POST as seen in console log below, it may be
bug >> 665675
Follow step 2 to troubleshoot Bug 665675.
Copyright 1985-2006 Phoenix Technologies Ltd.
"." = Deleted lines to save space
...............
...............
28672MB RAM tested 28672MB RAM installedFixed Disk 0:
STEC
NACF1GM1U-B11
ERROR Bad DIMM found in slot: 01
ERROR Bad DIMM found in slot: 02
*******************************
*
*
Bad DIMMs errors / ECC DIMM errors
*
*
*******************************
ECC error at DIMM 8. Error(s) in bits 103-100 = 0x2. Correctable Machine Check Error at
CPU2. NB Error:
STATUS(Val,OverF,Enable,AddrV,Synd(0x82),CECC,Link(0),CPU0,CKSynd(0x1d5),ExtErr(0x8),Err
Code(Src,NTO,Read,Mem,Gen)), ADDR(0x3a5380)
ECC error at DIMM 13. Error(s) in bits 131-128 = 0x7. Correctable Machine Check Error at
CPU6. NB Error:
.............
If the system boots normally skip to step 3. For memory errors follow steps (a-e) to troubleshoot Bug 665675 .
a) Attempt to boot again from the "LOADER>" by entering: autoboot up to 2-3 times to correct DIMM errors noninvasively.
b) If the memory error issue is resolved, skip to step 3. If DIMM(s) continue reporting errors, return to "LOADER>" prompt
and continue with step c.
c) Swap each DIMM reporting errors with a known good DIMM from another memory bank on the MB. Document which DIMMs
were moved and where. Fig 13 below shows the DIMM numbering. Each bank has a set of four slots.
d) Attempt to boot again from the "LOADER>" prompt by entering: autoboot
e) If the memory error issue is resolved, continue with step 3. If DIMM(s) continue reporting errors, call NGS and report
which DIMMs are impacted and the DIMM slot moves documented.
Fig 13
Memory Bank
DIMMs
Continue with Section XI on next page.
Memory Bank
4321
DIMM Layout for SA600, V-FAS6070, 6080

Total Number of DIMM Slots: 16
Memory Bank
Memory Bank
DIMM Layout for V-FAS6030, 6040

Total Number of DIMM Slots: 8
5678
DIMMs

Page 12 of 19
XI. SA600 and V-FAS6000 Series: Verify FC Adapter Configuration (cont.)

4 Review the fcadmin config output from Section IV. If any onboard Adapter Ports (0a-0h) were configured as "target" adapters
proceed with next steps - If all were configured as "initiators", skip to step 8.
5 At the *> prompt, enter: fcadmin config to view the configuration of the FC Adapters on the Replacement Controller Module.
If adapters 0a-0h match the original configuration, skip to step 8. If one or more needs to be set, continue.
If the adapter that needs to be changed to a target, is listed as " online", it must be off-lined first before it can be changed.
NOTE
Issue: fcadmin offline <HA>
6 For each Adapter to be configured as a target enter: fcadmin config -t target <HA> Issue one command per adapter.
This example configures Adapter port 0g and 0h as target:
7 Enter: fcadmin config to confirm the changed FC Adapters are displaying as PENDING: (target) ports.
*> fcadmin config
STEP 5: Enter: fcadmin config
Local
Adapter Type
State
Status
--------------------------------------------------0a
online
0b
online
0c
online
0d
online
0e
offline
0f
offline
0g
offline
0h
offline
Example Only
STEP 6: Enter:
fcadmin config -t target <HA> for each
port to be configured as a target
*> fcadmin config -t target 0g

Wed Oct 29 15:18:17 GMT [fci.config.state:info]: Fibre channel initiator adapter
0g is in the PENDING (target) state.
A reboot is required for the new adapter configuration to take effect.
*> fcadmin config -t target 0h
Wed Oct 29 15:18:39 GMT [fci.config.state:info]: Fibre channel initiator adapter
0h is in the PENDING (target) state.
A reboot is required for the new adapter configuration to take effect.
*> fcadmin config
Local
Adapter Type
State
Status
--------------------------------------------------0a
online
0b
online
0c
online
0d
online
0e
offline
0f
offline
0g
initiator PENDING (target)
offline
0h
initiator PENDING (target)
offline
STEP 7: Enter: fcadmin config to

confirm each target port is shown as
PENDING (target).
*>
8
9
If any FC cables were disconnected from adapters '0a-0h' due to boot issue, firmly reconnect them now. Must click in.
Go to Section XII, "Verify Disk List" on next page.

Page 13 of 19
XII. SA600 and V-FAS6000 Series: Verify Disk List

STOP Follow these steps to verify the disks are discovered.
1 Enter: storage show disk -x to force some disk I/O.
a) In this listing verify the same number of disks are displayed as compared to the output captured in Sec IV.
IF disks are missing: Confirm all FC and SAS cables are fully seated and re-run: storage show disk -x
2 Enter: storage show disk -p to confirm all adapters list a PRIMARY and SECONDARY path. No? Re-check cable seating.
3
Skip to step 4 if this is not a V-series Filer (V6000). If V-Filer, perform the below Additional Steps.
Additional Steps for V-Series Filer
S
T
O
P
4
5
The "storage show disk -x" command displays the connectivity to the third party array and any (optional)
NetApp disks.
If the storage listing is missing or incomplete:
(i) Confirm all FC connections are in the correct HA port and firmly seated on the NetApp controller.
(ii) If the connections are correct and firmly seated, confirm the WWPNs were correctly entered by end-user
by reviewing the output of the "fcadmin channels" command BEFORE and AFTER the MB swap Engage NGS for assistance.
If disk output is correct, enter: halt at the maintenance mode prompt: " * >" , to exit to "LOADER>" prompt.
Go to Section XIII, "Run Diagnostics" on next page.

Page 14 of 19
XIII. SA600 and V-FAS6000 Series: Run Diagnostics (40-60 minutes depending on model and PCI Adapters)
1 Test the Replacement Tray with diagnostics by entering boot_diags at the "LOADER>" prompt.
2 In the Diagnostic Menu, enter: run mb mem agent cf-card (These diagnostics tests are basic confidence tests for the new
motherboard, memory, agent & rlm, and CompactFlash)
3 If asked "OK to run NVRAM diagnostic (yes/no)?" Answer: no
LOADER> boot_diags
Loading x86_64/diag/diag.krn:...0x200000/12606928 0xe05dd0/4664936 0x1278c38/8 Entry at
0x00202018
STEP 1: Enter: boot_diags
Starting program at 0x00202018
Copyright (c) 1992-2009 NetApp.

init mca for BSP
Diagnostic Monitor
version: 5.7
built: Fri Aug 27 16:06:30 PDT 2010
-------------------------------------all
Run all system diagnostics
mb
FAS6030 motherboard diagnostic
mem
Main memory diagnostic
nvram
NVRAM diagnostic
iscsi
iSCSI Target HBA diagnostic
agent
agent & rlm diagnostic
cf-card CompactFlash controller diagnostic
fcal
FCAL controller diagnostic
stress System wide stress diagnostic
V-FAS6040,V-FAS6070, V-FAS6080 and

SA600 are also valid models
Commands:
Config (print a list of configured PCI devices)
Default (restore all options to default settings)
Exit
(exit diagnostics)
Help
(print this commands list)
Options (print current option settings)
Version (print the diagnostic version)
Run
<diag ... diag> (run selected diagnostics)
NOTE: New RUN

Options:
Count
<number> (loop selected diagnostic(s) (number) of passes)
Command options
Loop
<yes|no> (loop selected diagnostic(s))
Status <yes|no> (print status messages)
Stop
<yes|no> (stop-on-error / keep running)
Xtnd
<yes|no> (extended tests / regular tests)
Mchk
<auto|off|on|halt> (machine check control)
STEP 2:
Cpu
<0|1> (default cpu)
Enter: run mb mem agent cf-card
Seed
<number> (random seed (0:use machine generated number))
Enter Diag, Command or Option: run mb mem agent cf-card
WARNING!
Do not run the NVRAM diagnostic immediately after a

system crash or if there is a possibility that log
data is stored. Run only on new boards, or after a
normal system shutdown, or if there is no chance of
preserving customer data.
OK to run NVRAM diagnostic (yes/no)? no
Continue with Section XIII on next page.
STEP 3: If this message is

displayed, enter no

Page 15 of 19
XIII. SA600 and V-FAS6000 Series: Run Diagnostics (cont.)

5 The test output below only includes the test suite summary line. Look to see that all these show as PASSED. If any state
FAILED, scroll back through your test output to see which test FAILED and call NGS to report the test failure. Read all the
NOTE: Text box information below.
DIAGNOTIC RESULTS CONFIRMATION CHECKS

Verify all Tests state: PASSED or SKIPPED.
FAS6030 Motherboard Diagnostic

All tests on next 2 pages should indicate PASSED or Skipped.
-----------------------------If any Test FAILED - call NGS! Note- Text Box on FC test below
.....
****** Misc. test suite ......................... PASSED
.....
****** Cache test suite ......................... PASSED
Performing comprehensive
.....
****** Comprehensive BGE
.....
.....
.....
.....
.....
Testing FCAL card on
.....
****** Comprehensive
.....
.....
.....
.....
.....
.....
.....
.....
Environmental check,
6
BGE test on e0a

test ................... PASSED
BGE test on e0b
test ................... PASSED
BGE test on e0c
test ................... PASSED
BGE test on e0d
NOTE: The BGE test prints for

all 6 onboard Ethernet ports e0a,
e0b, e0c, e0d, e0d, e0f
test ................... PASSED

BGE test on e0e
test ................... PASSED
BGE test on e0f
test ................... PASSED
channel 0a
FCAL test .................. PASSED
channel 0b
channel 0c
channel 0d
channel 0e
channel 0f
channel 0g
NOTE: The FCAL test prints

for all 8 onboard FCAL ports
0a, 0b, 0c, 0d, 0e, 0f, 0g, 0h.
If a FC test fails remove the

cable for that port if attached
and retest "mb' test only

channel 0h
subsystem: any ......... PASSED
mb test .................... PASSED
Continue with Section XIII on next page.
NOTE: That the Comprehensive

mb test "PASSED"

Page 16 of 19
XIII. SA600 and V-FAS6000 Series: Run Diagnostics (cont.)

7 In text box Step 7 below, verify all the memory was discovered: V-FAS6030/40 ~16GB, V-FAS6070/80 ~32GB
8 If all tests show PASSED or SKIPPED, enter: exit to exit the main diagnostic menu. If any tests listed as FAILED, report failure
to NGS.
Testing : 1608 MB (start=1b000000, end=7f800000)
Testing : 14336 MB (start=100000000, end=480000000)
Total Memory Size : 16376 MB
Main Memory Diagnostic
----------------------
STEP 7:
Note: PLEASE CONFIRM
V-FAS6030/40 should total ~16GB

V-FAS6070/80, SA600 should total ~32GB
Output is from a FAS6030
Performing comprehensive main memory test

.....
****** Comprehensive Memory test ................ PASSED
Agent Diagnostic
---------------.....
****** Comprehensive RLM test ................... PASSED
CompactFlash Diagnostic
-----------------------.....
****** Comprehensive CompactFlash test .......... PASSED
Pass = 1, Current date = Wednesday Aug 30 10:47:27 2011

--- Completed pass 1.
Enter Diag, Command or Option: exit

__
.....
CPU Type: AMD Opteron(tm) Processor 252
LOADER>
NOTE: That the Comprehensive

Memory test, Comprehensive RLM
test & Comprehensive
CompactFlash test "PASSED"
Note: Test Suite

Complete message
STEP 8: Enter: exit to exit the Diags. The prom will initialize
displaying many messages. After about 10-20 seconds, it will
drop to the LOADER> prompt.
Go to Section XIV, "Boot PROM Variable Checks" on next page.

Page 17 of 19
XIV. SA600 and V-FAS6000 Series: Boot PROM Variable Checks

1 IF ONTAP version is < 8.0.2 (ONTAP 8.1 and > are not affected), unset the variable bootarg.init.wipeclean . (copy-n-paste)
LOADER-A> unsetenv bootarg.init.wipeclean
2
Validate the boot env variables:
At the LOADER-A|B> prompt, enter: printenv to list "all" the boot PROM variables. Search the output to confirm the required
variables are properly set based on your system type and configuration . Follow steps A-C in Table-1 below.
1) If you accidently issued a "set-defaults", all the configuration variables are unset - You have to examine the printenv capture of
STOP
the original MB to determine which ones will need to be manually entered in step 3 for your System Type & Configuration.
Table-1
Step System Type & Configuration
A) IF HA:
Verify this variable is set with correct partner sys-id value.
B) IF C-Mode:
Verify this variable is set to true.
C) IF V-Series:
fc-non-array-adapter-list is only required if the V-Series is also
hosting NetApp multipath shelves.
One of these two variables "may" be set if the storage
switches are McData.
3
Variable name and Value to be validated

partner-sysid xxxxxxxxxx
bootarg.init.boot_clustered true
fc-non-array-adapter-list <fc-adapter-list>
fc-no-mgmt-ports <fc-adapter-list>
OR (only one of these two may be set)
fc-no-mgmt? true
If any of the variables are missing or values are not set correctly, set them now one by one by following steps a-d below.
a) Open the console log file that was previously captured from the old MB and scroll to the "printenv" output listing.
b) Use the "Find" function and search for the variable required and its value. Sample "printenv" output ishere
c) Once identified, copy them using "Ctrl-C", then at the LOADER prompt enter:
setenv <(CTRL-V)to paste the variable and value>
d) Enter: savenv to save the variable and value.
LOADER> setenv variable-name value
LOADER> savenv
PROM will accept improper variable names. Recheck they were entered properly by issuing printenv again.
STOP
Go to Section XV, "Boot the Operating System - 'giveback' if applicable" on next page.

Page 18 of 19
XV. SA600 and V-FAS6000 Series: Boot the Operating System - 'giveback' if applicable
1 At the LOADER-A|B prompt, enter: autoboot to boot ONTAP.
2 After the console stops printing messages, press the <enter> key.
a) If the system booted up to a "login>" prompt, example below, skip to Section XVI, otherwise skip to step 3.
Loading X86_64/freebsd/image1/kernel:0x100000/3375736
0x538280/3221872
.... Many typical
system startup
.....
messages removed for clarity
.....
*******************************
These are
typical Boot strap console
*******************************
messages. If the partner did not takeover
.....
OR this is a stand-alone system, you
should eventually get a "login" prompt
login:
when you
hit <enter>.
3
If the system booted up to a "Waiting for giveback>" prompt (press the <enter> key) , example below, the node was part of an
HA configuration and was taken over by its partner.
.....
"...." = Deleted
.....
lines to save space
*******************************
*******************************
.....
.....
5
6
Engage end-user and verify that they do want the Giveback to be performed at this time.
- If yes , continue with Step 5 below.
- If the customer elects to defer the giveback process skip to Section XVI.
Login into the PARTNER node (7-Mode=root , C-Mode=admin ). Engage end-user for password.
Check takeover status by entering the appropriate command shown for the specified ONTAP Mode. If "partner not ready" may
have to wait 2-4 minutes for the NVRAMs to synchronize.
7-Mode
partner(takeover)> cf status
7
8
9
NOTE 3:
If you see this message, this node is part of a
HA configuration and the partner node took
over for it.
If the "controller giveback" fails due to
partner "not ready", wait 5 minutes for the
NVMEMs to synchronize. If the giveback fails
due to "open CIFS sessions", failed disks or
for any other reason, contact NGS.
Cluster-Mode
cluster::> run local cf status
7-Mode Only: Ask the customer if there are any heavy NDMP, SnapMirror or SnapVault processes running. If yes , they
should be disabled due to bug - 489060.
The procedure to disable these processes is >> here.
If running "A-SIS" (aka, DeDup processes), it is recommended to stop all "sis" jobs on all volumes on the target and partner
node because they may hang the takeover/giveback process.
Continue with Section XV on next page.

Page 19 of 19
XV. SA600 and V-FAS6000 Series: Boot the Operating System - 'giveback' if applicable (cont.)
10 Enter the proper controller giveback command(s) based on the mode running as follows:
A giveback can fail due to: "a failed disk" - remove failed disks or for "Open CIFS sessions". If due to open CIFS sessions,
NOTE
check with customer how to close out CIFS sessions. Terminating CIFS can cause loss of data if DBs are running under CIFS.
7-Mode
partner(takeover)> cf giveback
11
Cluster-Mode
cluster::> storage failover giveback -fromnode local
Wait! 90 seconds for 7-Mode or 3 minutes for Cluster-Mode after giveback reported complete.
Check controller failover status by entering the appropriate command shown for the specified ONTAP Mode.
7-Mode
target(takeover)> cf status
Controller Failover enabled,
XYZ is up.
Look for failover enabled
12
Cluster-Mode
cluster::> storage failover show
Follow step (a)
ONTAP 8 failover show

a) IF Cluster-Mode: Confirm the "giveback" status of the storage, refer this doc >
If the "giveback" is incomplete, wait 2 minutes and re-check. If still not complete after 10 minutes, contact Support.
NOTE
Do not proceed to next step if 'incomplete or partial giveback'!
IF Cluster-Mode: Follow steps (a-b) below, otherwise skip to Section XVI.
a) From the clustershell on each node, enter the command below to list the logical interfaces that are not on their home
server and port.
Cluster-Mode
cluster::> net int show -is-home false
Example of output here> net int show
b) If any interfaces are listed as "false" in the above command, enter the command below to revert them back to their home
port. Issues? Call NGS.
Cluster-Mode
cluster::> net int revert *
Example of output here> net int show
XVI. SA600 and V-FAS6000 Series: Submit logs and Part Return
1 Request end-user to send NetApp an ASUP Message from the target node so the configuration setup can be verified by NGS. If
the target system is not UP, send ASUP from its partner. Use the corresponding command for the version of ONTAP running.
Enter your dispatch's 7-digit FSO number (begins with 5).
7-Mode
filer> options autosupport.doit 5xxxxxx
2
3
4
5
6
7
8
9
10
Cluster-Mode
cluster::> invoke * -type all -message 5xxxxxx
IF NDMP, SnapMirror or SnapVault options were disabled, enable them now. Refer to page 2 of doc > >
here
C-Mode Only: Re-enable "auto-giveback" options if they were disabled on either node. C-Mode command
here
IF a-sis (Dedup) processes were disabled, customer can re-start them now.
Ask customer if using Operations Manager? If so, can they still access the controllers. If not, see bug > > 583160
Email the console log with the NetApp Reference Number in the Subject Line to
xdl-tpm-console-logs@netapp.com
Place the defective part in the antistatic bag and seal the box.
Follow the return shipping instructions on the box to ship the part(s) back to NetApps RMA processing center. If the
shipping label is missing see process to obtain a shipping label here >
Missing Shipping Label?
Verify with customer that the system is OK and if working with NGS ask them if it is OK to be released.
Close dispatch per Rules of Engagement.

MB - V Fas6000 Sa600 C 013

Uploaded by

Copyright:

Available Formats

MB - V Fas6000 Sa600 C 013

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MB - V Fas6000 Sa600 C 013

Uploaded by

Copyright:

Available Formats

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

For NetApp Authorized Service Engineers

Known Bugs/Issues - Bug Table and Notes Below

First Fixed Release

Link to Statement of Volatility by Platform is: http://support.netapp.com/info/web/ECMP1132988.html

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

Reinsert the System Tray, FC and Cable Trays and cables

I. SA600 and V-FAS6000 Series: Appliance/Motherboard Tray Visual Checks

SA600 or V-FAS Model Number

PCI Slot Numbering

Activity, Status (!), Power LEDs

FC Port Tray ( 0a-0b, 0c-0d, 0e-0f, 0g-0h )

IOIOI (Console) Port

Go to Section II, "Node Pre-Checks" on next page.

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

II. SA600 and V-FAS6000 Series: Node Pre-Checks

! (Status) LED will turn from

WARNING for HA (Active-Active) configurations:

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

The HA partner is currently operational and in takeover mode.

Step 6b): Enter: y

b) Disable the auto-giveback option if enabled. (copy-n-paste)

Go to Section IV, "Capture the Current System Configuration" on next page.

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

IV SA600 and V-FAS6000 Series: Capture the Current System Configuration

Step 1): Enter: show date

An example of a "printenv" output is here > printenv-C-all.pdf

Continue with Section IV on next page.

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

ONTAP 8.0.x Cluster-Mode Only

Please choose one of the following:

maintenance mode console prompt

Continue with Section IV on next page.

maintenance mode console prompt

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

STEP 7: Enter: fcadmin config

*> storage show disk -x

Step 9: This command prints out all the attached disks.

A typical listing may display many more disks and SAS

DISK SHELF BAY SERIAL

Step 11: Enter: halt

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

Step 2: Lower Cam Handle and pull

Step 1: Loosen the 6 Blue

Step 4: Remove the 3 screws and remove the

Step 3: Use grip handles on each side

VII. SA600 and FAS6000 Series: Move/Inspect each DIMM

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

Loosen both blue thumbscrews to remove brackets

NOTE: PCI cards need to

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

X. SA600 and V-FAS6000 Series: Set Date and Time

Go to Section XI, "Verify FC Adapter Configuration" on next page.

Motherboard (MB)Tray Replacement for the SA600 and V-FAS6000 Series

XI. SA600 and V-FAS6000 Series: Verify FC Adapter Configuration

a NVRAM battery voltage issue.

Continue with Section XI on next page.

DIMM Layout for SA600, V-FAS6070, 6080