CNTRL 1

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2019.05.
18 14:28:14 =~=~=~=~=~=~=~=~=~=~=~=
cntrl1> date
Sat May 18 15:32:00 IST 2019
cntrl1> uptime
3:32pm up 12 mins, 0 NFS ops, 0 CIFS ops, 0 HTTP ops, 467798 FCP ops, 0 iSCSI ops
cntrl1> sysconfig -a
NetApp Release 7.3.6P4: Fri Nov 18 01:15:52 PST 2011
System ID: 0142246234 (cntrl1); partner ID: 0142246351 (cntrl2)
System Serial Number: 850000100334 (cntrl1)
System Rev: B1
System Storage Configuration: Single-Path HA
System ACP Connectivity: NA
slot 0: System Board 1665 MHz (System Board XVII A7)
Model Name: FAS2040
Part Number: 110-00133
Revision: A7
Serial Number: 8028506
BIOS version: 6.1
Processors: 2
Processor ID: 0x6ec
Microcode Version: 0x55
Memory Size: 4096 MB
NVMEM Size: 256 MB of Main Memory Used
CMOS RAM Status: OK
Controller: A
slot 0: Private BGE 10/100 Ethernet Controller
e0P MAC Address: 00:a0:98:4b:f8:0e (auto-unknown-cfg_down)
Device Type: BCM5721
memory mapped I/O base 0xd9a00000, size 0x10000
slot 0: Dual 10/100/1000 Ethernet Controller G20
e0a MAC Address: 00:a0:98:4b:f8:13 (auto-unknown-down)
e0b MAC Address: 00:a0:98:4b:f8:12 (auto-unknown-cfg_down)
e0c MAC Address: 00:a0:98:4b:f8:11 (auto-unknown-cfg_down)
e0d MAC Address: 00:a0:98:4b:f8:10 (auto-unknown-cfg_down)
Device Type: Rev 6
memory mapped I/O base 0xd9640000, size 0x20000
I/O base 0x0000cfc0, size 0x20
I/O base 0x0000cfe0, size 0x20
I/O base 0x0000dfc0, size 0x20
I/O base 0x0000dfe0, size 0x20
I/O base 0x0000ffe0, size 0x20
slot 0: Interconnect HBA: Mellanox IB MT25204
GUID: 0x212800004bf80e LID: 0x15a Remote LID: 0x1cf
Firmware rev: 1.0.800
Hardware rev: 160
Command rev: 1
prefetchable memory base 0xdb800000, size 0x800000
slot 0: Fibre Channel Target Host Adapter 0a
(Dual-channel, QLogic 2432(2462) rev. 2, 64-bit, <ONLINE>)
Firmware rev: 4.5.2
Host Port Addr: 010200
Cacheline size: 16
SRAM parity: Yes
FC Nodename: 50:0a:09:80:88:4a:81:cf (500a0980884a81cf)
FC Portname: 50:0a:09:81:98:4a:81:cf (500a0981984a81cf)
Connection: PTP, Fabric
SFP Vendor Name: AVAGO
SFP Vendor P/N: AFBR-57R6APZ
SFP Vendor Rev:
SFP Serial No.: B91112C1DA
SFP Connector: LC
SFP Capabilities: 1, 2, 4 Gbit/Sec
I/O base 0x0000ee00, size 0x100
slot 0: Fibre Channel Target Host Adapter 0b
(Dual-channel, QLogic 2432(2462) rev. 2, 64-bit, <ONLINE>)
Firmware rev: 4.5.2
Host Port Addr: 010200
Cacheline size: 16
SRAM parity: Yes
Connection: PTP, Fabric
SFP Vendor Name: AVAGO
SFP Vendor P/N: AFBR-57R6APZ
SFP Vendor Rev:
SFP Serial No.: B91112C0P7
SFP Connector: LC
SFP Capabilities: 1, 2, 4 Gbit/Sec
I/O base 0x0000ef00, size 0x100
slot 0: SAS Host Adapter 0c (LSI Logic 1068E rev. B3, SAS, <UP>)
Firmware rev: 1.31.02.00
Base WWN: 5:00a098:0015d12:48
Phy State: [0] Enabled, 3.0 Gb/s
[1] Enabled, 3.0 Gb/s
00.0 : NETAPP X287_S15K7288A15 NA00 272.0GB 520B/sect
(6SJ17F900000N1374T2F)
00.1 : NETAPP X287_HVPBP288A15 NA00 272.0GB 520B/sect (JLWUUKEC)
00.2 : NETAPP X287_HVPBP288A15 NA00 272.0GB 520B/sect (JLW98Y8C)
(6SJ0ZMTF0000N1347AP3)
(6SJ17YVK0000N137EWV3)
00.5 : NETAPP X287_S15K5288A15 NQ06 272.0GB 520B/sect
(3LM0M5F500009742RS56)
(6SJ1852J0000N136A6F4)
(6SJ185JF0000N134EYQG)
Shelf 0: ESAS Firmware rev. SAS A: 05 SAS B: 05
slot 0: SAS Host Adapter 0d (LSI Logic 1068E rev. B3, SAS, <OFFLINE (hard)>)
Firmware rev: 1.31.02.00
Base WWN: 5:00a098:0015d12:4c
Phy State: [4] Enabled, Rate unknown
[5] Enabled, Rate unknown
QSFP Vendor: not available
QSFP Part Number: not available
QSFP Type: not available
QSFP Serial Number: not available
I/O base 0x0000bf00, size 0x100
slot 0: ATA/IDE Adapter 0e (0x000001f0)
0e.0 STEC NACF1GM1U-B11 01/17/07 977MB
512B/sect (STI1MQJ111173025955)
I/O base 0x0000ffc0, size 0x8
I/O base 0x0000ffb8, size 0x4
I/O base 0x0000ffc8, size 0x8
I/O base 0x0000ffbc, size 0x4
I/O base 0x0000ffd0, size 0x10
Baseboard Management Controller:
Firmware Version: 1.3
IPMI version: 2.0
DHCP: off
BMC MAC address: 00:a0:98:4b:f8:0f
IP address: 172.16.20.41
IP mask: 255.255.255.0
Gateway IP address: 172.16.20.1
BMC ARP interval: 10 seconds
BMC has (1) user: naroot
ASUP enabled: on
ASUP mailhost: 172.16.20.55
ASUP from: ksrtcits.reports@gmail.com
ASUP recipients: karkam007@gmail.com autosupport@netapp.com
Uptime: 1 Days, 01:18:40
cntrl1> sysconfig -a# # # #d
Device HA SHELF BAY CHAN Disk Vital Product Information
---------- --------------- ----- ------------------------------
0c.00.0 0c 0 0 SA:A 6SJ17F900000N1374T2F
0c.00.1 0c 0 1 SA:A JLWUUKEC
0c.00.2 0c 0 2 SA:A JLW98Y8C
0c.00.3 0c 0 3 SA:A 6SJ0ZMTF0000N1347AP3
0c.00.4 0c 0 4 SA:A 6SJ17YVK0000N137EWV3
0c.00.5 0c 0 5 SA:A 3LM0M5F500009742RS56
0c.00.6 0c 0 6 SA:A 6SJ1852J0000N136A6F4
0c.00.7 0c 0 7 SA:A 6SJ185JF0000N134EYQG
cntrl1> sysconfig -d# # # #m
No medium changers present on system.
cntrl1> sysconfig -m# # # #c
sysconfig: There are no configuration errors.
cntrl1> sysconfig -c# # # #a# # # #A
NetApp Release 7.3.6P4: Fri Nov 18 01:15:52 PST 2011
System ID: 0142246234 (cntrl1); partner ID: 0142246351 (cntrl2)
System Serial Number: 850000100334 (cntrl1)
System Rev: B1
System Storage Configuration: Single-Path HA
System ACP Connectivity: NA
slot 0: System Board
Processors: 2
Memory Size: 4096 MB
slot 0: Private BGE 10/100 Ethernet Controller
e0P MAC Address: 00:a0:98:4b:f8:0e (auto-unknown-cfg_down)
slot 0: Dual 10/100/1000 Ethernet Controller G20
e0a MAC Address: 00:a0:98:4b:f8:13 (auto-unknown-down)
e0b MAC Address: 00:a0:98:4b:f8:12 (auto-unknown-cfg_down)
e0c MAC Address: 00:a0:98:4b:f8:11 (auto-unknown-cfg_down)
e0d MAC Address: 00:a0:98:4b:f8:10 (auto-unknown-cfg_down)
slot 0: Interconnect HBA: Mellanox IB MT25204
slot 0: Fibre Channel Target Host Adapter 0a
slot 0: Fibre Channel Target Host Adapter 0b
slot 0: SAS Host Adapter 0c
8 Disks: 2176.0GB
1 shelf with ESAS
slot 0: SAS Host Adapter 0d
slot 0: ATA/IDE Adapter 0e (0x000001f0)
0e.0 977MB
Baseboard Management Controller:
Firmware Version: 1.3
IPMI version: 2.0
DHCP: off
BMC MAC address: 00:a0:98:4b:f8:0f
IP address: 172.16.20.41
IP mask: 255.255.255.0
Gateway IP address: 172.16.20.1
BMC ARP interval: 10 seconds
BMC has (1) user: naroot
ASUP enabled: on
ASUP mailhost: 172.16.20.55
ASUP from: ksrtcits.reports@gmail.com
ASUP recipients: karkam007@gmail.com autosupport@netapp.com
Uptime: 1 Days, 01:19:34
sysconfig: There are no configuration errors.
Device HA SHELF BAY CHAN Disk Vital Product Information
---------- --------------- ----- ------------------------------
0c.00.0 0c 0 0 SA:A 6SJ17F900000N1374T2F
0c.00.1 0c 0 1 SA:A JLWUUKEC
0c.00.2 0c 0 2 SA:A JLW98Y8C
0c.00.3 0c 0 3 SA:A 6SJ0ZMTF0000N1347AP3
0c.00.4 0c 0 4 SA:A 6SJ17YVK0000N137EWV3
0c.00.5 0c 0 5 SA:A 3LM0M5F500009742RS56
0c.00.6 0c 0 6 SA:A 6SJ1852J0000N136A6F4
0c.00.7 0c 0 7 SA:A 6SJ185JF0000N134EYQG
volume aggr0(1) (1 RAID group):
group 0: 1 disk
volume aggr0 (1 RAID group):
group 0: 3 disks
Aggregate aggr0(1) (offline, raid_dp, foreign, degraded) (block checksums)
Plex /aggr0(1)/plex0 (online, normal, active, pool0)
RAID group /aggr0(1)/plex0/rg0 (double degraded)
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks)
Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
dparity 0c.00.1 0c 0 1 SA:A 0 SAS 15000 272000/557056000
274845/562884296
parity FAILED N/A 272000/557056000
data FAILED N/A 272000/557056000
Aggregate aggr0 (online, raid4, degraded) (block checksums)

Plex /aggr0/plex0 (online, normal, active, pool0)
RAID group /aggr0/plex0/rg0 (degraded)
Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
parity 0c.00.4 0c 0 4 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data 0c.00.7 0c 0 7 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data FAILED N/A 272000/557056000
data 0c.00.3 0c 0 3 SA:A 0 SAS 15000 272000/557056000
280104/573653840
Pool1 spare disks (empty)
Partner disks
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys
(MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
partner 0c.00.5 0c 0 5 SA:A 0 SAS 15000 0/0
280104/573653840
280104/573653840
274845/562884296
280104/573653840
No medium changers present on system.
No tape drives present on system.
cntrl1> Sat May 18 15:33:35 IST [cntrl1: asup.smtp.host:info]: Autosupport cannot
connect to host 172.16.20.55 (Network comm problem) for message: PERFORMANCE
SNAPSHOT
Sat May 18 15:33:35 IST [cntrl1: asup.smtp.unreach:error]: Autosupport mail was not
sent because the system cannot reach any of the mail hosts from the
autosupport.mailhost option. (PERFORMANCE SNAPSHOT)
Sat May 18 15:33:35 IST [cntrl1: emssnmp:error]: snmpd: send_trap: Failure in
sendto (Network is down)
cntrl1> sysconfig -A# # # #r

Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
274845/562884296
data FAILED N/A 272000/557056000
Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
parity 0c.00.4 0c 0 4 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data 0c.00.7 0c 0 7 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data FAILED N/A 272000/557056000
data 0c.00.3 0c 0 3 SA:A 0 SAS 15000 272000/557056000
280104/573653840
Partner disks
(MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
280104/573653840
280104/573653840
274845/562884296
280104/573653840
cntrl1> Sat May 18 15:33:35 IST last message repeated 5 times
cntrl1> sysconfig -r# # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ## # #

## # # ## # # ## # # #aggr status -r
Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
274845/562884296
data FAILED N/A 272000/557056000

Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
parity 0c.00.4 0c 0 4 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data 0c.00.7 0c 0 7 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data FAILED N/A 272000/557056000
data 0c.00.3 0c 0 3 SA:A 0 SAS 15000 272000/557056000
280104/573653840
Partner disks
(MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
280104/573653840
280104/573653840
274845/562884296
280104/573653840
cntrl1> aggr status -r# # # #u
aggr status: Unrecognized option -u
usage:
aggr status [<atv-name>] [-d | -r | -v | -l | -c | -b | -s | -f | -i | -?]
- print the status of aggregate or traditional volume <atv-name>
(or all aggregates and traditional volumes)
cntrl1> aggr status -u# # # #f
Broken disks (empty)

cntrl1> aggr status -f# # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ## # #
## # # # show # # # #_ space# # # ## # # ## # # ## # # ## # # ## # # # # # # #space
-g
Aggregate 'aggr0'
Total space WAFL reserve Snap reserve Usable space BSR NVLOG
A-SIS
796GB 79GB 0GB 717GB 0GB
9GB
Space allocated to volumes in the aggregate
Volume Allocated Used Guarantee

vol0 61GB 1GB volume
Sql_Data_Strg1 201GB 200GB none
var_mysql 398GB 396GB none
Aggregate Allocated Used Avail

Total space 660GB 598GB 46GB
Snap reserve 0GB 0GB 0GB
WAFL reserve 79GB 7GB 71GB
cntrl1> aggr show_space -g# # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ##
# # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # #vol status -v
Volume State Status Options
vol0 online raid4, flex root, diskroot, nosnap=off,
degraded nosnapdir=off, minra=off,
sis no_atime_update=off, nvfail=off,
ignore_inconsistent=off,
snapmirrored=off,
create_ucode=off,
convert_ucode=off,
maxdirsize=28835,
schedsnapname=ordinal,
fs_size_fixed=off,
compression=off,
guarantee=volume, svo_enable=off,
svo_checksum=off,
svo_allow_rman=off,
svo_reject_errors=off,
no_i2p=off,
fractional_reserve=100,
extent=off,
try_first=volume_grow,
read_realloc=off,
snapshot_clone_dependency=off
Containing aggregate: 'aggr0'
Plex /aggr0/plex0: online, normal, active

RAID group /aggr0/plex0/rg0: degraded
Snapshot autodelete settings for vol0:

state=off
commitment=try
trigger=volume
target_free_space=20%
delete_order=oldest_first
defer_delete=user_created
prefix=(not specified)
destroy_list=none
Volume autosize settings:
state=off
Sql_Data_Strg1 online raid4, flex nosnap=on, nosnapdir=off,
degraded minra=off, no_atime_update=off,
sis nvfail=off,
snapmirrored=off,
create_ucode=on,
convert_ucode=on,
maxdirsize=28835,
fs_size_fixed=off,
compression=off, guarantee=none,
svo_enable=off, svo_checksum=off,
svo_allow_rman=off,
svo_reject_errors=off, no_i2p=on,
fractional_reserve=20,
extent=off,
read_realloc=off,

Snapshot autodelete settings for Sql_Data_Strg1:

state=on
commitment=try
trigger=volume
destroy_list=none
state=on
maximum-size=259441488 KB
increment-size=10810060 KB
var_mysql online raid4, flex nosnap=on, nosnapdir=off,
degraded minra=off, no_atime_update=off,
sis nvfail=off,
snapmirrored=off,
create_ucode=on,
convert_ucode=on,
maxdirsize=28835,
fs_size_fixed=off,
compression=off, guarantee=none,
svo_enable=off, svo_checksum=off,
svo_allow_rman=off,
svo_reject_errors=off, no_i2p=on,
fractional_reserve=0, extent=off,
read_realloc=off,

Snapshot autodelete settings for var_mysql:

state=on
commitment=try
trigger=volume
destroy_list=none
state=on
maximum-size=518882968 KB
increment-size=21620120 KB
cntrl1> vol status -v# # # #s

cntrl1> vol status -s# # # #f
Broken disks (empty)

cntrl1> vol status -f# # # #r
Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
274845/562884296
data FAILED N/A 272000/557056000

Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
parity 0c.00.4 0c 0 4 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data 0c.00.7 0c 0 7 SA:A 0 SAS 15000 272000/557056000
280104/573653840
data FAILED N/A 272000/557056000
data 0c.00.3 0c 0 3 SA:A 0 SAS 15000 272000/557056000
280104/573653840
Partner disks
(MB/blks)
--------- ------ ------------- ---- ---- ---- ----- --------------
--------------
280104/573653840
280104/573653840
274845/562884296
280104/573653840
cntrl1> vol status -r# # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ## # #
## # # ## # # ## # # ## # # #environmental status all
environmental not found. Type '?' for a list of commands
cntrl1> environmental status all# # # ## # # ## # # ## # # ## # # ## # # ## # # ##
# # ## # # ## # # ## # # ## # # ## # # #
Usage: environment status |
[status] [shelf [<adapter>]] |
[status] [shelf_log] |
[status] [shelf_stats] |
[status] [shelf_power_status] |
[status] [chassis [all | list-sensors | Voltage | Temperature |
Power Supplies OutVoltage | Battery | Power Supply 1 | Power Supply 2]]
cntrl1> ifconfig -a
e0a: flags=0x2548867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
inet 172.16.20.31 netmask-or-prefix 0xffffff00 broadcast 172.16.20.255
partner e0a (not in use)
ether 00:a0:98:4b:f8:13 (auto-unknown-down) flowcontrol full
e0b: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:a0:98:4b:f8:12 (auto-unknown-cfg_down) flowcontrol full
e0c: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
e0d: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
e0P: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:a0:98:4b:f8:0e (auto-unknown-cfg_down) flowcontrol full
lo: flags=0x1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
inet 127.0.0.1 netmask-or-prefix 0xff000000 broadcast 127.0.0.1
ether 00:00:00:00:00:00 (VIA Provider)
SNAPSHOT
cntrl1> ifconfig -aSat May 18 15:37:34 IST last message repeated 5 times
e0a: flags=0x2548867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

inet 172.16.20.31 netmask-or-prefix 0xffffff00 broadcast 172.16.20.255
partner e0a (not in use)
ether 00:a0:98:4b:f8:13 (auto-unknown-down) flowcontrol full
e0b: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
e0c: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
e0d: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
e0P: flags=0x2508866<BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500
ether 00:a0:98:4b:f8:0e (auto-unknown-cfg_down) flowcontrol full
lo: flags=0x1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 8160
inet 127.0.0.1 netmask-or-prefix 0xff000000 broadcast 127.0.0.1
ether 00:00:00:00:00:00 (VIA Provider)
cntrl1> df -Ah
Aggregate total used avail capacity
aggr0 717GB 670GB 46GB 94%
aggr0/.snapshot 0TB 0TB 0TB ---%
cntrl1> df -Ah# # # ## # # ## # # #-Vh
Filesystem total used avail capacity Mounted on
/vol/vol0/ 48GB 1523MB 46GB 3% /vol/vol0/
/vol/vol0/.snapshot 12GB 146MB 11GB 1% /vol/vol0/.snapshot
/vol/Sql_Data_Strg1/ 226GB 199GB 26GB 88%
/vol/Sql_Data_Strg1/
/vol/Sql_Data_Strg1/.snapshot 0TB 0TB 0TB ---%
/vol/Sql_Data_Strg1/.snapshot
/vol/var_mysql/ 432GB 394GB 38GB 91% /vol/var_mysql/
/vol/var_mysql/.snapshot 0TB 0TB 0TB ---%
/vol/var_mysql/.snapshot
cntrl1> df -h
Filesystem total used avail capacity Mounted on
/vol/vol0/ 48GB 1523MB 46GB 3% /vol/vol0/
/vol/vol0/.snapshot 12GB 146MB 11GB 1% /vol/vol0/.snapshot
/vol/Sql_Data_Strg1/ 226GB 199GB 26GB 88%
/vol/Sql_Data_Strg1/
/vol/Sql_Data_Strg1/.snapshot 0TB 0TB 0TB ---%
/vol/Sql_Data_Strg1/.snapshot
/vol/var_mysql/ 432GB 394GB 38GB 91% /vol/var_mysql/
/vol/var_mysql/.snapshot 0TB 0TB 0TB ---%
/vol/var_mysql/.snapshot
cntrl1> rdfile /etc/messages
Sun May 12 16:00:00 IST [cntrl1/cntrl2: raid.fdr.reminder:warning]: Failed Disk
0c.00.5 Shelf 0 Bay 5 [NETAPP X287_S15K7288A15 NA00] S/N [6SJ17AW70000N1376BYQ]
is still present in the system and should be removed.
Sun May 12 21:26:03 IST [cntrl1/cntrl2: httpd.config.mime.missing:warning]:
/etc/httpd.mimetypes file is missing.
Sun May 12 21:26:04 IST [cntrl1/cntrl2: raid.rg.recons.missing:notice]: RAID group
partner:/aggr0/plex0/rg0 is missing 1 disk(s).
Sun May 12 21:26:04 IST [cntrl1/cntrl2: raid.rg.recons.cantStart:warning]: The
reconstruction cannot start in RAID group partner:/aggr0/plex0/rg0: No 520 bps disk
of required size available in spare pool, targeting any partner spare pool
Sun May 12 21:26:28 IST [cntrl1/cntrl2: nbt.nbns.registrationComplete:info]: NBT:
All CIFS name registrations have completed for the partner server.
Sun May 12 21:26:37 IST [cntrl1/cntrl2: asup.smtp.sent:notice]: Cluster
Notification mail sent: Cluster Notification from cntrl1 (REBOOT (CLUSTER
TAKEOVER)) WARNING
Sun May 12 21:27:00 IST [cntrl1/cntrl2: monitor.globalStatus.critical:CRITICAL]:
cntrl2 has taken over this node. There are not enough spare disks.
Mon May 13 12:15:17 IST [cntrl1/cntrl2: sshd_0:info]: Failed password for mctd from
172.16.26.60 port 55363ssh2
172.16.26.60 port 55363ssh2
172.16.26.60 port 55363ssh2
Mon May 13 12:19:16 IST [cntrl1/cntrl2: sshd_0:error]: error: Disconnecting: Shell
not supported on takenover partner
172.16.26.60 port 55427ssh2
172.16.26.10 port 51714ssh2
172.16.26.10 port 51714ssh2
Mon May 13 12:23:57 IST [cntrl1/cntrl2: openssh.invalid.channel.req:warning]: SSH
client (SSH-2.0-OpenSSH_5.1) from 172.16.26.10 sent unsupported channel request
(10, env).
Mon May 13 12:23:57 IST [cntrl1/cntrl2: sshd_0:error]: error: Disconnecting: Shell
Mon May 13 12:27:28 IST [cntrl1/cntrl2: HTTPPool00:warning]: HTTP Authentication
from 172.16.26.60 to realm Administration failed
Mon May 13 21:58:33 IST [cntrl1/cntrl2: sshd_0:info]: Failed password for root from
172.16.26.10 port 51738ssh2
Mon May 13 21:58:51 IST last message repeated 2 times
Tue May 14 13:23:19 IST [cntrl1/cntrl2: httpd.config.mime.missing:warning]:
Tue May 14 13:23:19 IST [cntrl1/cntrl2: raid.rg.recons.missing:notice]: RAID group
Tue May 14 13:23:19 IST [cntrl1/cntrl2: raid.rg.recons.cantStart:warning]: The
Tue May 14 13:23:43 IST [cntrl1/cntrl2: nbt.nbns.registrationComplete:info]: NBT:
Tue May 14 13:23:53 IST [cntrl1/cntrl2: asup.smtp.sent:notice]: Cluster
TAKEOVER)) WARNING
Tue May 14 13:24:00 IST [cntrl1/cntrl2: monitor.globalStatus.critical:CRITICAL]:
Fri May 17 12:28:49 IST [cntrl1/cntrl2: httpd.config.mime.missing:warning]:
Fri May 17 12:28:50 IST [cntrl1/cntrl2: raid.rg.recons.missing:notice]: RAID group
Fri May 17 12:28:50 IST [cntrl1/cntrl2: raid.rg.recons.cantStart:warning]: The
Fri May 17 12:29:02 IST [cntrl1/cntrl2: monitor.globalStatus.critical:CRITICAL]:
Fri May 17 12:29:14 IST [cntrl1/cntrl2: nbt.nbns.registrationComplete:info]: NBT:
Fri May 17 12:29:21 IST [cntrl1/cntrl2: asup.smtp.sent:notice]: Cluster
TAKEOVER)) WARNING
Fri May 17 12:32:39 IST [cntrl1/cntrl2: openssh.invalid.channel.req:warning]: SSH
client (SSH-2.0-OpenSSH_5.1) from 172.16.26.10 sent unsupported channel request
(10, env).
Fri May 17 12:32:39 IST [cntrl1/cntrl2: sshd_0:error]: error: Disconnecting: Shell
Fri May 17 12:40:19 IST [cntrl1/cntrl2: HTTPPool03:warning]: HTTP Authentication
from 172.16.26.174 using NTLM failed
Fri May 17 13:04:39 IST [cntrl1/cntrl2: sshd_1:error]: error: Disconnecting: Shell
Fri May 17 08:14:55 GMT [localhost: rc:notice]: The system was down for -449
seconds
Fri May 17 08:14:55 GMT [localhost: cf.fsm.takeoverOfPartnerDisabled:notice]:
Cluster monitor: takeover of cntrl2 disabled (interconnect error)
Fri May 17 08:14:55 GMT [localhost: cf.fsm.takeoverByPartnerDisabled:notice]:
Cluster monitor: takeover of by cntrl2 disabled (interconnect error)
Fri May 17 12:20:46 IST [cntrl1: cf.nm.nicTransitionUp:info]: replayed event:
Interconnect link 0 is UP
Fri May 17 12:20:50 IST [cntrl1: sas.initialization.failed:error]: replayed event:
Initialization failed on SAS adapter 0d.
Fri May 17 12:20:51 IST [cntrl1: nvram.battery.turned.on:info]: replayed event: The
NVRAM battery is turned ON. It is turned OFF during system shutdown.
Fri May 17 12:20:53 IST [cntrl1: disk.init.failure.spinup:error]: replayed event:
Disk 0c.00.1 has failed to spin up and cannot be used. Please replace it with a new
drive.
drive.
Fri May 17 12:20:53 IST [cntrl1: disk.init.failureBytes:error]: replayed event:
Disk 0c.00.5 failed due to failure byte setting.
Fri May 17 12:20:54 IST [cntrl1: diskown.isEnabled:info]: replayed event: software
ownership has been enabled for this system
Fri May 17 12:20:56 IST [cntrl1: cf.nm.nicReset:warning]: replayed event:
Initiating soft reset on Cluster Interconnect card 0 due to rendezvous reset
Fri May 17 12:20:57 IST [cntrl1: cf.rv.notConnected:error]: replayed event:
Connection for cfo_rv failed
Fri May 17 12:20:57 IST [cntrl1: fmmb.current.lock.disk:info]: replayed event: Disk
0c.00.4 is a local HA mailbox disk.
Fri May 17 12:20:57 IST [cntrl1: fmmb.instStat.change:info]: replayed event: normal
mailbox instance on local side.
0c.00.0 is a partner HA mailbox disk.
mailbox instance on partner side.
Fri May 17 12:20:57 IST [cntrl1: cf.fm.partner:info]: replayed event: Cluster
monitor: partner 'cntrl2'
Fri May 17 12:20:57 IST [cntrl1: cf.fm.timeMasterStatus:info]: replayed event:
Acting as cluster time master
Fri May 17 12:20:57 IST [cntrl1: shelf.config.spha:info]: replayed event: System is
using single path HA attached storage only.
Fri May 17 12:20:58 IST [cntrl1: ses.status.psWarning:warning]: replayed event:
DS12-ESAS shelf 0 on channel 0c power warning for Power supply 1: non-critical
status; DC undervoltage fault. This module is on the rear side of the shelf, at the
left.
right.
Fri May 17 12:21:03 IST [cntrl1: cf.nm.nicTransitionDown:warning]: replayed event:
Cluster Interconnect link 0 is DOWN
Fri May 17 12:21:06 IST [cntrl1: sas.link.error:error]: replayed event: Could not
recover link on SAS adapter 0d after 15 seconds. Offlining the adapter.
Fri May 17 12:21:06 IST [cntrl1: cf.disk.ResvFail:CRITICAL]: replayed event: Disk
0c.00.4 has been reserved by the cluster partner and cannot be accessed.
Fri May 17 12:21:06 IST [cntrl1: cf.disk.ResvTakeOver:CRITICAL]: replayed event:
This node will wait for cluster giveback and the disk reservations to be released.
Fri May 17 13:14:55 IST [cntrl1: ses.channel.rescanInitiated:info]: replayed event:
Initiating rescan on channel 0c.
Fri May 17 13:44:52 IST [cntrl1: coredump.spare.none:info]: replayed event: No
sparecore disk was found.
Fri May 17 13:44:53 IST [cntrl1: raid.cksum.replay.summary:info]: replayed event:
Replayed 0 checksum blocks.
Fri May 17 13:44:53 IST [cntrl1: raid.stripe.replay.summary:info]: replayed event:
Replayed 0 stripes.
Fri May 17 13:44:53 IST [cntrl1: cf.fm.launch:info]: replayed event: Launching
cluster monitor
Fri May 17 13:44:54 IST [cntrl1: cf.fm.discardNvram:notice]: replayed event:
Cluster monitor: node was previously taken over, nvram may be discarded
Fri May 17 13:44:55 IST [cntrl1: kern.syslog.msg:notice]: replayed event: The
system was down for -449 seconds
Fri May 17 13:44:55 IST [cntrl1: cf.fsm.takeoverOfPartnerDisabled:notice]: replayed
event: Cluster monitor: takeover of cntrl2 disabled (interconnect error)
Fri May 17 13:44:55 IST [cntrl1: cf.fsm.takeoverByPartnerDisabled:notice]: replayed
event: Cluster monitor: takeover of by cntrl2 disabled (interconnect error)
Fri May 17 13:44:57 IST [cntrl1: dfu.firmwareUpToDate:info]: Firmware is up-to-date
on all disk drives
Fri May 17 13:44:57 IST [cntrl1: cf.nm.nicTransitionUp:info]: Interconnect link 0
is UP
Fri May 17 13:44:57 IST [cntrl1: perf.archive.start:info]: Performance archiver
started. Sampling 23 objects and 211 counters.
Fri May 17 13:44:58 IST [cntrl1: cf.fsm.takeoverOfPartnerEnabled:notice]: Cluster
monitor: takeover of cntrl2 enabled
Fri May 17 13:44:58 IST [cntrl1: cf.fsm.takeoverByPartnerEnabled:notice]: Cluster
monitor: takeover of cntrl1 by cntrl2 enabled
Fri May 17 13:44:59 IST [cntrl1: fcp.service.startup:info]: FCP service startup
Fri May 17 13:44:59 IST [cntrl1: scsitarget.ispfct.linkUp:notice]: Link up on Fibre
Channel target adapter 0b.
Channel target adapter 0a.
Fri May 17 13:44:59 IST [cntrl1: scsitarget.ispfct.lipReset:notice]: FCP Target 0a:
LIP Reset from AL_PA 0x0 (WWPN unknown)
Fri May 17 13:44:59 IST [cntrl1: scsitarget.ispfct.lipReset:notice]: FCP Target 0b:
Fri May 17 13:45:01 IST [cntrl1: netif.linkDown:info]: Ethernet e0a: Link down,
check cable.
Fri May 17 13:45:06 IST [cntrl1: rc:ALERT]: timed: time daemon started
Fri May 17 13:45:06 IST [cntrl1: raid.rg.recons.missing:notice]: RAID group
/aggr0/plex0/rg0 is missing 1 disk(s).
Fri May 17 13:45:06 IST [cntrl1: raid.rg.recons.cantStart:warning]: The
reconstruction cannot start in RAID group /aggr0/plex0/rg0: No 520 bps disk of
required size available in spare pool, targeting any spare pool
Fri May 17 13:45:06 IST [cntrl1: mgr.boot.disk_done:info]: NetApp Release 7.3.6P4
boot complete. Last disk update written at Fri May 17 13:52:24 IST 2019
Fri May 17 13:45:06 IST [cntrl1: monitor.raid.brokenDisk:warning]: data disk in
RAID group /aggr0/plex0/rg0 is broken.
Fri May 17 13:45:06 IST [cntrl1: monitor.brokenDisk.notice:notice]: When a disk is
broken, the system shuts down automatically every 24 hours to encourage you to
replace the disk. If you reboot the system it will run for another 24 hours
before shutting down. (The 24 hour timeout may be increased by altering the
"raid.timeout" value using the "options" command.)
Fri May 17 13:45:06 IST [cntrl1: snmp_trap_main:error]: snmpd: send_trap: Failure
in sendto (Network is down)
Fri May 17 13:45:06 IST last message repeated 2 times
Fri May 17 13:45:06 IST [cntrl1: emssnmp:error]: snmpd: send_trap: Failure in
Fri May 17 13:45:07 IST [cntrl1: mgr.boot.reason_ok:notice]: System rebooted after
a giveback.
Fri May 17 13:45:07 IST [cntrl1: httpd.config.mime.missing:warning]:
Fri May 17 13:45:15 IST [cntrl1: nbt.nbns.socketError:error]: NBT: Cannot send
broadcast message on NBNS socket. Error 0x32: Network is down.
Fri May 17 13:45:17 IST [cntrl1: raid.rg.spares.low:warning]: /aggr0/plex0/rg0
Fri May 17 13:45:21 IST [cntrl1: asup.smtp.host:info]: Autosupport cannot connect
to host 172.16.20.55 (Network comm problem) for message: SHUTDOWN PENDING (degraded
mode)
Fri May 17 13:45:21 IST [cntrl1: asup.smtp.unreach:error]: Autosupport mail was not
autosupport.mailhost option. (SHUTDOWN PENDING (degraded mode))
mode)
mode)
Fri May 17 13:45:27 IST [cntrl1: nbt.nbns.registrationComplete:info]: NBT: All CIFS
name registrations have completed for the local server.
mode)
Fri May 17 13:46:00 IST [cntrl1: monitor.globalStatus.nonCritical:warning]: There
are not enough spare disks.
mode)
mode)
mode)
mode)
Fri May 17 14:00:00 IST [cntrl1: kern.uptime.filer:info]: 2:00pm up 1:39 0 NFS
ops, 0 CIFS ops, 0 HTTP ops, 2 FCP ops, 0 iSCSI ops
Fri May 17 14:00:00 IST [cntrl1: monitor.shutdown.brokenDisk.pending:warning]: data
disk in RAID group /aggr0/plex0/rg0 is broken. Halting system in 24 hours.
mode)
mode)
mode)
Fri May 17 08:43:41 GMT [localhost: rc:notice]: The system was down for 122 seconds
Fri May 17 08:43:41 GMT [cntrl1: cf.fsm.takeoverOfPartnerEnabled:notice]: Cluster
Fri May 17 08:43:41 GMT [cntrl1: cf.fsm.takeoverByPartnerEnabled:notice]: Cluster
Fri May 17 14:13:31 IST [cntrl1: sas.initialization.failed:error]: replayed event:
Fri May 17 14:13:31 IST [cntrl1: nvram.battery.turned.on:info]: replayed event: The
drive.
drive.
Fri May 17 14:13:34 IST [cntrl1: diskown.isEnabled:info]: replayed event: software
Fri May 17 14:13:37 IST [cntrl1: cf.fm.timeMasterStatus:info]: replayed event:
Fri May 17 14:13:37 IST [cntrl1: shelf.config.spha:info]: replayed event: System is
Fri May 17 14:13:37 IST [cntrl1: coredump.spare.none:info]: replayed event: No
left.
right.
Fri May 17 14:13:38 IST [cntrl1: raid.vol.replay.nvram:info]: replayed event:
Performing raid replay on volume(s)
Fri May 17 14:13:38 IST [cntrl1: raid.cksum.replay.summary:info]: replayed event:
Fri May 17 14:13:38 IST [cntrl1: raid.stripe.replay.summary:info]: replayed event:
Replayed 0 stripes.
Fri May 17 14:13:39 IST [cntrl1: cf.fm.launch:info]: replayed event: Launching
cluster monitor
Fri May 17 14:13:40 IST [cntrl1: cf.fsm.takeoverByPartnerDisabled:notice]: replayed
event: Cluster monitor: takeover of by cntrl2 disabled (partner booting)
Fri May 17 14:13:41 IST [cntrl1: kern.syslog.msg:notice]: replayed event: The
system was down for 122 seconds
Fri May 17 14:13:41 IST [cntrl1: cf.fsm.takeoverOfPartnerEnabled:notice]: replayed
event: Cluster monitor: takeover of cntrl2 enabled
Fri May 17 14:13:41 IST [cntrl1: cf.fsm.takeoverByPartnerEnabled:notice]: replayed
event: Cluster monitor: takeover of cntrl1 by cntrl2 enabled
Fri May 17 14:13:42 IST [cntrl1: dfu.firmwareUpToDate:info]: Firmware is up-to-date
on all disk drives
Fri May 17 14:13:43 IST [cntrl1: perf.archive.start:info]: Performance archiver
Fri May 17 14:13:43 IST [cntrl1: fcp.service.startup:info]: FCP service startup
Fri May 17 14:13:46 IST [cntrl1: sas.link.error:error]: Could not recover link on
SAS adapter 0d after 15 seconds. Offlining the adapter.
Fri May 17 14:13:47 IST [cntrl1: netif.linkDown:info]: Ethernet e0a: Link down,
check cable.
Fri May 17 14:13:51 IST [cntrl1: rc:info]: relog syslog Fri May 17 14:08:57 IST
[cntrl1: emssnmp:error]: snmpd: send_trap: Failure in sendto (Network is down)
Fri May 17 14:13:51 IST [cntrl1: rc:info]: relog syslog Fri May 17 14:08:57 IST
[cntrl1: emssnmp:error]: snmpd: send_trap: Failure in sendto (Network is down)
Fri May 17 14:13:51 IST [cntrl1: rc:ALERT]: timed: time daemon started
Fri May 17 14:13:52 IST [cntrl1: raid.rg.recons.missing:notice]: RAID group
Fri May 17 14:13:52 IST [cntrl1: raid.rg.recons.cantStart:warning]: The
Fri May 17 14:13:52 IST [cntrl1: mgr.boot.disk_done:info]: NetApp Release 7.3.6P4
boot complete. Last disk update written at Fri May 17 14:11:34 IST 2019
Fri May 17 14:13:52 IST [cntrl1: monitor.raid.brokenDisk:warning]: data disk in
Fri May 17 14:13:52 IST [cntrl1: monitor.brokenDisk.notice:notice]: When a disk is
Fri May 17 14:13:52 IST [cntrl1: snmp_trap_main:error]: snmpd: send_trap: Failure
mode)
Fri May 17 14:13:52 IST [cntrl1: mgr.boot.reason_ok:notice]: System rebooted after
power-on.
Fri May 17 14:13:52 IST [cntrl1: httpd.config.mime.missing:warning]:
Fri May 17 14:14:01 IST [cntrl1: raid.rg.spares.low:warning]: /aggr0/plex0/rg0
Fri May 17 14:14:02 IST [cntrl1: nbt.nbns.socketError:error]: NBT: Cannot send
Fri May 17 14:14:05 IST [cntrl1: monitor.globalStatus.nonCritical:warning]: There
mode)
mode)
Fri May 17 14:14:16 IST [cntrl1: nbt.nbns.registrationComplete:info]: NBT: All CIFS
mode)
mode)
mode)
mode)
mode)
mode)
mode)
mode)
mode)
mode)
mode)
mode)
Fri May 17 14:49:36 IST [cntrl1: asup.smtp.drop:error]: Autosupport mail
(/etc/log/autosupport/201905171345.0) was not sent for host (0). The system will
drop the message
Fri May 17 15:00:00 IST [cntrl1: kern.uptime.filer:info]: 3:00pm up 46 mins, 0
NFS ops, 0 CIFS ops, 0 HTTP ops, 908084 FCP ops, 0 iSCSI ops
Fri May 17 16:38:57 IST [cntrl1: ses.channel.rescanInitiated:info]: Initiating
rescan on channel 0c.
Fri May 17 16:44:05 IST [cntrl1: raid.assim.tree.foreign:error]: raidtree_verify:
Aggregate aggr0 is a foreign aggregate and is being taken offline. Use the 'aggr
online' command to bring it online.
Fri May 17 16:44:05 IST [cntrl1: raid.assim.tree.dupName:error]: Duplicate
aggregate names found, an instance of foreign:aggr0 is being renamed to
foreign:aggr0(1).
Fri May 17 17:04:39 IST [cntrl1: disk.init.failure.spinup:error]: Disk 0c.00.2 has
failed to spin up and cannot be used. Please replace it with a new drive.
to host 172.16.20.55 (Network comm problem) for message: REBOOT (after giveback)
autosupport.mailhost option. (REBOOT (after giveback))
drop the message
Fri May 17 22:28:36 IST [cntrl1: scsitarget.ispfct.reset:notice]: Resetting Fibre
to host 172.16.20.55 (Network comm problem) for message: SPARES_LOW
autosupport.mailhost option. (SPARES_LOW)
drop the message
Sat May 18 00:00:00 IST [cntrl1: kern.uptime.filer:info]: 12:00am up 9:47 0 NFS
Sat May 18 00:00:00 IST [cntrl1: monitor.shutdown.brokenDisk.pending:warning]: data
Sat May 18 00:00:00 IST last message repeated 2 times
Sat May 18 12:00:00 IST [cntrl1: kern.uptime.filer:info]: 12:00pm up 21:47 0 NFS
Sat May 18 12:06:33 IST [cntrl1: ses.channel.rescanInitiated:info]: Initiating
Sat May 18 12:06:51 IST [cntrl1: ses.channel.rescanInitiated:info]: Initiating
Sat May 18 13:33:50 IST [cntrl1: console_login_mgr:info]: root logged in from
console
disk in RAID group /aggr0/plex0/rg0 is broken. Halting system in 1 hour.
Sat May 18 15:00:00 IST [cntrl1: kern.uptime.filer:info]: 3:00pm up 1 day, 47
mins, 0 NFS ops, 0 CIFS ops, 0 HTTP ops, 45250133 FCP ops, 0 iSCSI ops
Sat May 18 15:00:00 IST [cntrl1: monitor.brokenDisk.notice:notice]: When a disk is
Sat May 18 15:00:00 IST [cntrl1: monitor.shutdown.brokenDisk:EMERGENCY]: data disk
in RAID group /aggr0/plex0/rg0 is broken. Halting system now.
Sat May 18 15:00:03 IST [cntrl1: ems.engine.inputSuppress:error]: Event
'nbt.nbns.socketError' suppressed 19 times since Fri May 17 14:14:03 IST 2019.
Sat May 18 15:00:03 IST [cntrl1: nbt.nbns.socketError:error]: NBT: Cannot do
broadcast send on NBNS broadcast socket. Error 0x32: Network is down.
Sat May 18 15:00:03 IST [cntrl1: kern.shutdown:notice]: System shut down because :
"BROKEN DISK".
Sat May 18 15:00:03 IST [cntrl1: fcp.service.shutdown:info]: FCP service shutdown
Sat May 18 09:50:39 GMT [localhost: rc:notice]: The system was down for 1234
seconds
Sat May 18 09:50:40 GMT [cntrl1: cf.partner.short_uptime:warning]: Partner up for 2
seconds only
Sat May 18 09:50:40 GMT [cntrl1: cf.fsm.takeoverOfPartnerDisabled:notice]: Cluster
monitor: takeover of cntrl2 disabled (interconnect error)
Sat May 18 09:50:40 GMT [cntrl1: cf.fsm.takeoverByPartnerDisabled:notice]: Cluster
monitor: takeover of cntrl1 by cntrl2 disabled (interconnect error)
Sat May 18 15:20:18 IST [cntrl1: cf.nm.nicTransitionUp:info]: replayed event:
Sat May 18 15:20:29 IST [cntrl1: sas.initialization.failed:error]: replayed event:
Sat May 18 15:20:29 IST [cntrl1: nvram.battery.turned.on:info]: replayed event: The
Sat May 18 15:20:32 IST [cntrl1: diskown.isEnabled:info]: replayed event: software
Sat May 18 15:20:34 IST [cntrl1: cf.nm.nicReset:warning]: replayed event:
Sat May 18 15:20:35 IST [cntrl1: cf.rv.notConnected:error]: replayed event:
Sat May 18 15:20:35 IST [cntrl1: fmmb.current.lock.disk:info]: replayed event: Disk
Sat May 18 15:20:35 IST [cntrl1: fmmb.instStat.change:info]: replayed event: normal
Sat May 18 15:20:35 IST [cntrl1: fmmb.instStat.change:info]: replayed event: normal
Sat May 18 15:20:35 IST [cntrl1: cf.fm.partner:info]: replayed event: Cluster
Sat May 18 15:20:35 IST [cntrl1: cf.fm.timeMasterStatus:info]: replayed event:
Sat May 18 15:20:36 IST [cntrl1: shelf.config.spha:info]: replayed event: System is
Sat May 18 15:20:36 IST [cntrl1: coredump.spare.none:info]: replayed event: No
Sat May 18 15:20:36 IST [cntrl1: raid.assim.tree.foreign:error]: replayed event:
raidtree_verify: Aggregate aggr0 is a foreign aggregate and is being taken offline.
Use the 'aggr online' command to bring it online.
Sat May 18 15:20:36 IST [cntrl1: raid.assim.tree.dupName:error]: replayed event:
Duplicate aggregate names found, an instance of foreign:aggr0 is being renamed to
foreign:aggr0(1).
Sat May 18 15:20:36 IST [cntrl1: raid.cksum.replay.summary:info]: replayed event:
Sat May 18 15:20:36 IST [cntrl1: raid.stripe.replay.summary:info]: replayed event:
Replayed 0 stripes.
Sat May 18 15:20:37 IST [cntrl1: cf.fm.launch:info]: replayed event: Launching
cluster monitor
Sat May 18 15:20:37 IST [cntrl1: cf.fm.partner:info]: replayed event: Cluster
Sat May 18 15:20:39 IST [cntrl1: cf.nm.nicReset:warning]: replayed event:
Sat May 18 15:20:39 IST [cntrl1: kern.syslog.msg:notice]: replayed event: The
system was down for 1234 seconds
Sat May 18 15:20:40 IST [cntrl1: cf.partner.short_uptime:warning]: replayed event:
Partner up for 2 seconds only
Sat May 18 15:20:40 IST [cntrl1: cf.fsm.takeoverOfPartnerDisabled:notice]: replayed
event: Cluster monitor: takeover of cntrl2 disabled (interconnect error)
Sat May 18 15:20:40 IST [cntrl1: cf.fsm.takeoverByPartnerDisabled:notice]: replayed
event: Cluster monitor: takeover of cntrl1 by cntrl2 disabled (interconnect error)
Sat May 18 15:20:40 IST [cntrl1: dfu.firmwareUpToDate:info]: Firmware is up-to-date
on all disk drives
Sat May 18 15:20:41 IST [cntrl1: perf.archive.start:info]: Performance archiver
Sat May 18 15:20:42 IST [cntrl1: cf.fsm.takeoverOfPartnerEnabled:notice]: Cluster
Sat May 18 15:20:43 IST [cntrl1: cf.fsm.takeoverByPartnerEnabled:notice]: Cluster
Sat May 18 15:20:43 IST [cntrl1: fcp.service.startup:info]: FCP service startup
Sat May 18 15:20:43 IST [cntrl1: scsitarget.ispfct.linkUp:notice]: Link up on Fibre
Sat May 18 15:20:43 IST [cntrl1: scsitarget.ispfct.linkUp:notice]: Link up on Fibre
Sat May 18 15:20:43 IST [cntrl1: scsitarget.ispfct.lipReset:notice]: FCP Target 0a:
Sat May 18 15:20:43 IST [cntrl1: scsitarget.ispfct.lipReset:notice]: FCP Target 0a:
Sat May 18 15:20:43 IST [cntrl1: scsitarget.ispfct.lipReset:notice]: FCP Target 0b:
Sat May 18 15:20:43 IST [cntrl1: scsitarget.ispfct.lipReset:notice]: FCP Target 0b:
Sat May 18 15:20:44 IST [cntrl1: sas.link.error:error]: Could not recover link on
SAS adapter 0d after 15 seconds. Offlining the adapter.
Sat May 18 15:20:45 IST [cntrl1: netif.linkDown:info]: Ethernet e0a: Link down,
check cable.
Sat May 18 15:20:49 IST [cntrl1: rc:ALERT]: timed: time daemon started
Sat May 18 15:20:50 IST [cntrl1: raid.rg.recons.missing:notice]: RAID group
Sat May 18 15:20:50 IST [cntrl1: raid.rg.recons.cantStart:warning]: The
Sat May 18 15:20:50 IST [cntrl1: mgr.boot.disk_done:info]: NetApp Release 7.3.6P4
boot complete. Last disk update written at Sat May 18 15:00:04 IST 2019
Sat May 18 15:20:50 IST [cntrl1: monitor.raid.brokenDisk:warning]: data disk in
Sat May 18 15:20:50 IST [cntrl1: monitor.brokenDisk.notice:notice]: When a disk is
Sat May 18 15:20:50 IST [cntrl1: snmp_trap_main:error]: snmpd: send_trap: Failure
Sat May 18 15:20:50 IST [cntrl1: mgr.boot.reason_ok:notice]: System rebooted.
Sat May 18 15:20:50 IST [cntrl1: asup.smtp.host:info]: Autosupport cannot connect
to host 172.16.20.55 (Network comm problem) for message: PERFORMANCE SNAPSHOT
Sat May 18 15:20:51 IST [cntrl1: httpd.config.mime.missing:warning]:
Sat May 18 15:21:00 IST [cntrl1: raid.rg.spares.low:warning]: /aggr0/plex0/rg0
Sat May 18 15:21:00 IST [cntrl1: nbt.nbns.socketError:error]: NBT: Cannot send
Sat May 18 15:21:02 IST [cntrl1: monitor.globalStatus.nonCritical:warning]: There
Sat May 18 15:21:14 IST [cntrl1: nbt.nbns.registrationComplete:info]: NBT: All CIFS
Sat May 18 15:23:58 IST [cntrl1: console_login_mgr:info]: root logged in from
console
cntrl1> rdfile /etc/messages.0
Sun May 5 01:00:04 IST [cntrl1/cntrl2: raid.rg.scrub.start:notice]:
partner:/aggr0/plex0/rg0: starting scrub
Sat May 11 17:02:14 IST [cntrl1/cntrl2: raid.config.filesystem.disk.failed:error]:
File system Disk /aggr0/plex0/rg0/0c.00.5 Shelf 0 Bay 5 [NETAPP X287_S15K7288A15
NA00] S/N [6SJ17AW70000N1376BYQ] failed.
Sat May 11 17:02:14 IST [cntrl1/cntrl2: raid.rg.recons.missing:notice]: RAID group
Sat May 11 17:02:14 IST [cntrl1/cntrl2: raid.rg.recons.cantStart:warning]: The
Sat May 11 17:02:17 IST [cntrl1/cntrl2: asup.smtp.sent:notice]: Cluster
Notification mail sent: Cluster Notification from cntrl1 (FILESYSTEM DISK FAILED)
WARNING
cntrl1> rdfile /etc/messages.0# # # #1
Sun Apr 28 01:00:04 IST [cntrl1/cntrl2: raid.rg.scrub.start:notice]:
cntrl1> lun show -v
/vol/Sql_Data_Strg1/Sql_Data_Strg1 200g (214748364800) (r/w, online,
mapped)
Serial#: SKTKO4ktcpXr
Share: none
Space Reservation: disabled
Multiprotocol Type: linux
Maps: DBServer=0 DBserver1=0
/vol/var_mysql/var_mysql 400.0g (429523992576) (r/w, online, mapped)
Serial#: SKTKO4ktdH2j
Share: none
Space Reservation: disabled
Multiprotocol Type: linux
Maps: DBServer=2 DBserver1=2
cntrl1> igroup show -v
DBServer (FCP):
OS Type: linux
Member: 21:00:00:24:ff:2e:17:78 (not logged in)
Member: 21:00:00:24:ff:2e:17:79 (not logged in)
UUID: c1b7d8b0-c365-11e1-8859-00a0984bf813
ALUA: Yes
DBserver1 (FCP):
OS Type: linux
Member: 21:00:00:24:ff:2e:17:54 (logged in on: vtic, 0a)
Member: 21:00:00:24:ff:2e:17:55 (logged in on: vtic, 0b)
UUID: da430430-c384-11e1-8733-00a0984bf813
ALUA: Yes
cntrl1> disk show -n
disk show: No disks match option -n.
cntrl1> disk show -n# # # #a
DISK OWNER POOL SERIAL NUMBER
------------ ------------- ----- -------------
0c.00.6 cntrl2 (142246351) Pool0 6SJ1852J0000N136A6F4
0c.00.0 cntrl2 (142246351) Pool0 6SJ17F900000N1374T2F
0c.00.5 cntrl2 (142246351) Pool0 3LM0M5F500009742RS56
0c.00.2 cntrl2 (142246351) Pool0 JLW98Y8C
0c.00.4 cntrl1 (142246234) Pool0 6SJ17YVK0000N137EWV3
0c.00.3 cntrl1 (142246234) Pool0 6SJ0ZMTF0000N1347AP3
0c.00.7 cntrl1 (142246234) Pool0 6SJ185JF0000N134EYQG
0c.00.1 cntrl1 (142246234) Pool0 JLWUUKEC
cntrl1> disk show -a# # # #v
DISK OWNER POOL SERIAL NUMBER
------------ ------------- ----- -------------
0c.00.6 cntrl2 (142246351) Pool0 6SJ1852J0000N136A6F4
0c.00.0 cntrl2 (142246351) Pool0 6SJ17F900000N1374T2F
0c.00.5 cntrl2 (142246351) Pool0 3LM0M5F500009742RS56
0c.00.2 cntrl2 (142246351) Pool0 JLW98Y8C
0c.00.4 cntrl1 (142246234) Pool0 6SJ17YVK0000N137EWV3
0c.00.3 cntrl1 (142246234) Pool0 6SJ0ZMTF0000N1347AP3
0c.00.7 cntrl1 (142246234) Pool0 6SJ185JF0000N134EYQG
0c.00.1 cntrl1 (142246234) Pool0 JLWUUKEC
cntrl1> vif stATUS# # # ## # # ## # # ## # # #atus
No configured vifs present
SNAPSHOT
cntrl1> snap list

Volume vol0
working...
%/used %/total date name

---------- ---------- ------------ --------
1% ( 1%) 0% ( 0%) May 18 12:00 hourly.0
2% ( 1%) 0% ( 0%) May 18 08:00 hourly.1
3% ( 1%) 0% ( 0%) May 18 00:00 nightly.0
4% ( 1%) 0% ( 0%) May 17 20:00 hourly.2
5% ( 1%) 0% ( 0%) May 17 16:00 hourly.3
7% ( 2%) 0% ( 0%) May 15 12:00 hourly.4
8% ( 1%) 0% ( 0%) May 15 08:00 hourly.5
9% ( 1%) 0% ( 0%) May 15 00:00 nightly.1
Volume Sql_Data_Strg1
working...
No snapshots exist.
Volume var_mysql
working...
No snapshots exist.
cntrl1> license
a_sis QLAYDCG
cifs not licensed
cluster QCXGUPC
cluster_remote not licensed
compression not licensed
disk_sanitization not licensed
fcp YMAJTYG
flash_cache not licensed
flex_clone not licensed
flex_scale not licensed
flexcache_nfs WUCLPWA
gateway_hitachi not licensed
http WRELSHL
iscsi IYILSHL
multistore not licensed
nearstore_option SOKNOFF
nfs not licensed
operations_manager not licensed
protection_manager not licensed
provisioning_manager not licensed
smdomino not licensed
smsql not licensed
snapdrive_unix not licensed
snapdrive_windows not licensed
snaplock not licensed
snaplock_enterprise not licensed
snapmanager_hyperv not licensed
snapmanager_oracle not licensed
snapmanager_sap not licensed
snapmanager_sharepoint not licensed
snapmanagerexchange not licensed
snapmirror not licensed
snapmirror_sync not licensed
snapmover not licensed
snaprestore not licensed
snapvalidator not licensed
sv_applications_pri not licensed
sv_exchange_pri not licensed
sv_linux_pri not licensed
sv_marketing_pri not licensed
sv_ontap_pri not licensed
sv_ontap_sec not licensed
sv_oracle_pri not licensed
sv_sharepoint_pri not licensed
sv_sql_pri not licensed
sv_unix_pri not licensed
sv_vi_pri not licensed
sv_vmware_pri not licensed
sv_windows_ofm_pri not licensed
sv_windows_pri not licensed
syncmirror_local UCINOFF
v-series not licensed
vld not licensed
cntrl1> quota report
quota: quotas are off.
cntrl1> snap reserSat May 18 15:41:51 IST last message repeated 5 times
ve
cntrl1> quota report# # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ## # #
## # # ## # # ## # # #snap reserve
Volume vol0: current snapshot reserve is 20% or 12582912 k-bytes.
Volume Sql_Data_Strg1: current snapshot reserve is 0% or 0 k-bytes.
Volume var_mysql: current snapshot reserve is 0% or 0 k-bytes.
cntrl1> rdfile /etc/exports
#Auto-generated by setup Fri Dec 9 08:11:02 GMT 2011
cntrl1> cifs shares
Name Mount Point Description
---- ----------- -----------
ETC$ /etc Remote Administration
BUILTIN\Administrators / Full Control
HOME /vol/vol0/home Default Share
everyone / Full Control
C$ / Remote Administration
BUILTIN\Administrators / Full Control
c
ntrl1> fcp show adapter -v

Slot: 0a
Description: Fibre Channel Target Adapter 0a (Dual-channel, QLogic 2
432 (2462) rev. 2)
Status: ONLINE
Host Port Address: 0x010200
Firmware Rev: 4.5.2
PCI Bus Width: 64-bit
PCI Clock Speed: 33 MHz
Cacheline Size: 16
FC Packet Size: 2048
SRAM Parity: Yes
External GBIC: No
Data Link Rate: 4 GBit
Adapter Type: Local
Fabric Established: Yes
Connection Established: PTP
Mediatype: auto
Partner Adapter: None
Standby: No
Target Port ID: 0x101
Slot: 0b
Description: Fibre Channel Target Adapter 0b (Dual-channel, QLogic 2
432 (2462) rev. 2)
Status: ONLINE
Host Port Address: 0x010200
Firmware Rev: 4.5.2
PCI Bus Width: 64-bit
PCI Clock Speed: 33 MHz
Cacheline Size: 16
FC Packet Size: 2048
SRAM Parity: Yes
External GBIC: No
Data Link Rate: 4 GBit
Adapter Type: Local
Fabric Established: Yes
Connection Established: PTP
Mediatype: auto
Partner Adapter: None
Standby: No
Target Port ID: 0x102
cntrl1> storage show fault
No faults found in storage subsystem
cntrl1> lun show-m
lun: unrecognized command "show-m"
The following commands are available; for more information
type "lun help <command>"
clone help online share
comment map resize show
config_check maxsize serial snap
create move set stats
destroy offline setup unmap
cntrl1> lun show -m
LUN path Mapped to LUN ID Protocol
-----------------------------------------------------------------------
/vol/Sql_Data_Strg1/Sql_Data_Strg1 DBServer 0 FCP
DBserver1 0 FCP
/vol/var_mysql/var_mysql DBServer 2 FCP
DBserver1 2 FCP
cntrl1> lgroup show -v
lgroup not found. Type '?' for a list of commands
cntrl1> cf status
Cluster enabled, cntrl2 is up.
cntrl1> Sat May 18 15:45:22 IST [cntrl1: asup.smtp.host:info]: Autosupport canno
t connect to host 172.16.20.55 (Network comm problem) for message: PERFORMANCE S
NAPSHOT
Sat May 18 15:45:22 IST [cntrl1: asup.smtp.unreach:error]: Autosupport mail was
not sent because the system cannot reach any of the mail hosts from the autosupp
ort.mailhost option. (PERFORMANCE SNAPSHOT)
Sat May 18 15:45:22 IST [cntrl1: emssnmp:error]: snmpd: send_trap: Failure in se
ndto (Network is down)

CNTRL 1

Uploaded by

Copyright:

Available Formats

CNTRL 1

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CNTRL 1

Uploaded by

Copyright:

Available Formats

=~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2019.05.

Aggregate aggr0 (online, raid4, degraded) (block checksums)

Pool1 spare disks (empty)

Pool0 spare disks (empty)

cntrl1> sysconfig -A# # # #r

Pool1 spare disks (empty)

Pool0 spare disks (empty)

cntrl1> sysconfig -r# # # ## # # ## # # ## # # ## # # ## # # ## # # ## # # ## # #

Aggregate aggr0 (online, raid4, degraded) (block checksums)

Pool1 spare disks (empty)

Pool0 spare disks (empty)

Broken disks (empty)

Space allocated to volumes in the aggregate

Volume Allocated Used Guarantee

Aggregate Allocated Used Avail

Plex /aggr0/plex0: online, normal, active

Snapshot autodelete settings for vol0:

Plex /aggr0/plex0: online, normal, active

Snapshot autodelete settings for Sql_Data_Strg1:

Plex /aggr0/plex0: online, normal, active

Snapshot autodelete settings for var_mysql:

Pool1 spare disks (empty)

Broken disks (empty)

Aggregate aggr0 (online, raid4, degraded) (block checksums)

Pool1 spare disks (empty)

Pool0 spare disks (empty)

e0a: flags=0x2548867<UP,BROADCAST,RUNNING,MULTICAST,TCPCKSUM> mtu 1500

cntrl1> snap list

%/used %/total date name

ntrl1> fcp show adapter -v

You might also like