MOP for Installation of HSS G8->G9 HW swap

-------------------------------------------
Based on official Installation Guide, available in NOLS:
Release 16.5
Installing and Upgrading CMS-
8200 HSS (HP)
A25001-A0006-A2558-05-76P1
Issue 05

1) Perform complete cmcli backup

cmcli backup /var/tmp/backuprepo_`hostname`.dmp

2) HSS cmcli config dump

cmcli info
Authentication Required..
Please Enter UserName: cmadmin
Please Enter Password: ********
DNs Found: 1
DNs: [Lab] <-------------------------
....
CSCF5:
-----------------------
Sending Request to CMREPO : 10.6.0.66
DU Name: CSCF5 <-------------------------
Type: CSCF
State: ACTIVE

cmcli backup /opt/cmbase/cli/br/backuprepo_test_hssG8.dmp --dn=Lab --du=CSCF5

3) Perform comissioning steps for G9

(follow installation guide steps 4 and 4.1: disk raid config; adjust boot method;
set high performance mode)
1. Power on HP Proliant BL460c Gen9 and watch console. Press F9 to choose
system
utilities
2. Choose System utilities -> System Configuration-> RBSU->Boot Options->Boot
Mode and enable Legacy BIOS mode.
3. Press F10 to save configuration
4. Select Smart Arry P244br controller.
5. Launch Smart Array Controller GUI and choose Smart Array Controller
Configure->
Arrays->Logical Driverr and choose RAID1.
6. Update RAID-1 and Strip size 1024.
4.1 Hardware RAID Configuration
The following is the procedure to configure Hardware RAID-1 and RAID 1+0 on
HP
servers:
1. Power on the blade.
2. Connect to the console of the blade.
3. Change the BIOS settings as given below:
� Select the BIOS Serial Console Port Option as COM1.
� Select the BIOS Serial Console Baud Rate as 115200.
� Select the EMS Console Option as COM1.
4. Change the Power setting parameters.
 a) Change the power profile parameter to Maximum Performance.
b) Change the power regulator to HP static high performance mode.

4) Install iLO license

OA GUI > enclosure info> device bays > select blade > iLO > Web administration
Blade's iLO GUI > Administration > Licensing
Enter the license
Press 'Install'

5)Prepare TIAMS (upload TPD, run scripts, bootnet check)

5.1) Check swrepo version
pkginfo |grep -i swrepo
root@uasdmtm101ne> pkginfo |grep -i swrepo
Applications/Engineering swrepo_bm_CT 16.0-1651510 This is the
swrepo package

5.2) Check schema existence

If not already there:
Upload *.schemas to /opt/cmbase/common/config/schema/hssd/

root@uasdmtm101ne> ls /opt/cmbase/common/config/schema/hssd/
1601410.schema IMSDL1600146.ALL.015.schema
1651400.schema IMSDL1650140.000.schema
16.5_mp3_schema_delta.schema IMSDL1650140.ALL.019.schema
989640.schema IVS-299_delta.schema
IMSDL1550631.000.schema IVS787_delta.schema
IMSDL1600146.000.schema REG-5320_delta.schema

5.3) Reinstall HSS platform and application media

cd /home/iserver/HSS_cap_upg/HSS_medium
tar -xf IMSDL1650140.000_cpio.tar

cd IMSDL1650140.000_cpio
./extract_IMSDL1650140.000.sh

cd IMSDL1650140.000
./setupImsInstallServer.sh `pwd`

cd /home/iserver/HSS_cap_upg/HSS_medium
rm -Rf IMSDL1650140.000_cpio

6) Traffic isolation

7) Power off G8
su - rtp99
stopRTP &
(wait)
halt
(wait until OA GUI enclosure display shows blade light shift from green to orange;
or, if on site, watch the led in the blade)

8) Move G8 to new slot, but without pluging it in (leave it slightly out)

9) Move G9 to the new slot

10) Reconfigure VC
unassign profile NE-enc0-HSS-1
remove profile NE-enc0-HSS-1

add profile NE-enc0-HSS-1 -NoDefaultEnetConn -NoDefaultFcConn -NoDefaultFcoeConn

-quiet NAG=Default HideUnusedFlexNICs=false BootMode=Auto SriovMode=Simplified
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=Enabled SriovType=DEFAULT
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=UseBIOS SriovType=DEFAULT
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=UseBIOS SriovType=DEFAULT
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=UseBIOS SriovType=DEFAULT
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=UseBIOS SriovType=DEFAULT
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=UseBIOS SriovType=DEFAULT
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=UseBIOS SriovType=DEFAULT
add enet-connection NE-enc0-HSS-1 Network=Unassigned PXE=UseBIOS SriovType=DEFAULT
add server-port-map NE-enc0-HSS-1:1 NE-enc0-OAM-NetA-71 Untagged=true
add server-port-map NE-enc0-HSS-1:2 NE-enc0-OAM-NetB-71 Untagged=true
add server-port-map-range NE-enc0-HSS-1:3 UplinkSet=NE-enc0-NetA-X10
vlanids=131,141
add server-port-map NE-enc0-HSS-1:3 NE-enc0-HSS_APP-NetA-61 Untagged=true
add server-port-map-range NE-enc0-HSS-1:4 UplinkSet=NE-enc0-NetB-X10
vlanids=131,141
add server-port-map NE-enc0-HSS-1:4 NE-enc0-HSS_APP-NetB-61 Untagged=true
add server-port-map-range NE-enc0-HSS-1:6 UplinkSet=NE-enc0-NetB-X10
vlanids=151,161
add server-port-map NE-enc0-HSS-1:7 NE-enc0-HSM1-NetA-101 Untagged=true
add server-port-map NE-enc0-HSS-1:8 NE-enc0-HSM2-NetB-102 Untagged=true

assign profile NE-enc0-HSS-1 enc0:6

set enet-connection NE-enc0-HSS-1 1 SpeedType=Custom Speed=1000

set enet-connection NE-enc0-HSS-1 2 SpeedType=Custom Speed=1000
set enet-connection NE-enc0-HSS-1 3 SpeedType=Custom Speed=2500
set enet-connection NE-enc0-HSS-1 4 SpeedType=Custom Speed=2500
set enet-connection NE-enc0-HSS-1 6 SpeedType=Custom Speed=1000
set enet-connection NE-enc0-HSS-1 7 SpeedType=Custom Speed=1000
set enet-connection NE-enc0-HSS-1 8 SpeedType=Custom Speed=1000

11) Power on G9
press power button or go to OA GUI > Device bays > (select bay) > Virtual Buttons >
Power on

12) Preparations after Isolation

12.1) Remove HSS config from CMREPO
cmcli --user cmadmin --passwd yt_xk39b remove <DN/DU/version>

ie
cmcli --user cmadmin --passwd yt_xk39b remove NE/HSSDU1_DUSpec/16.5

cmcli --user cmadmin --passwd yt_xk39b remove NE/HSSDU1_Common/16.5

12.2) Run preparation scripts

cd /home/iserver/linux/http/static/preparation_tools
./prepTspImage.pl -mappl <HSS app medium> -moem <TSP platform medium> -cmrepo y
-CTPD <HSS Common TPD> -TPD <HSS DUSpec TPD> -SP <target SP> -mfw <firmware>
ie:
./prepTspImage.pl -mappl
/home/iserver/HSS_cap_upg/HSS_medium/IMSDL1650140.000_cpio/IMSDL1650140.000aa.cpio
-moem /home/iserver/HSS_cap_upg/TSP_CAF/LX_TSP_CAF_INST_COMMON_1651400.tar.gz
-cmrepo y -CTPD /home/iserver/HSS_cap_upg/TPD/TPD_HSSD_Cog9NE165.xls -TPD
/home/iserver/HSS_cap_upg/TPD/TPD_HSSD_DUg9NE165.xls -SP
/home/iserver/HSS_cap_upg/SP/IMSDL1650140.SPT.005.tar.gz -mfw
/home/iserver/HSS_cap_upg/FW/HP_SPP2015.10.tar

12.3) Test preparation file availability:

cd /var/tmp
netstat -an |grep 5571 |tail -1 |sed s/:/' '/g |awk '{print $5}'
wget https://(ip):5571/(du)/boot-(du).iso --no-check-certificate
rm boot-(du).iso

to see available isos:

locate boot- |grep autoinstall

ie:
cd /var/tmp
netstat -an |grep 5571 |tail -1 |sed s/:/' '/g |awk '{print $5}'
wget https://10.162.98.88:5571/uasdmhs101ne/boot-uasdmhs101ne.iso --no-check-
certificate
rm boot-uasdmhs101ne.iso

-
cat /etc/exports
exportfs -a

12.4) Set iso and connect via console

Login to OA
set server dvd connect https://10.162.98.88:5571/uasdmhs101ne/boot-uasdmhs101ne.iso
6
connect bay 6
(cdrom mount test: dir map1/oemhp_vm1/cddr1)
vsp

note: during the image load, use java console to see all error messages.

12.5) Set boot origin and reset blade

In OA GUI
Boot Options -> One time Boot from: cdrom -> apply
Virtual Devices -> Reset

13) Install HSS on G9

Follow HSS installation in console.

14) Post install checks (check ips; config crosscheck; status check)
in tiams:
ims_getdata.py
(do TPD crosscheck)
in HSS:
netstat -an |grep 16611
netstat -an |grep 3868
csdiag.pl
su - rtp99
status1 -e
cmcli display (DN/DU) >/var/tmp/config_dump_`hostname`.txt

15) Functional tests HSS G9

16) Decide if fallback/acceptance/go live

Fallback
--------
1) Power off G9
su - rtp99
stopRTP &
(wait)
halt
(wait until OA GUI enclosure display shows blade light shift from green to orange;
or, if on site, watch the led in the blade)

2) Unplug G9 and switch it with G8 (leave G9 unplugged)

3) Plug in G8 blade

4) Remove G9 cmcli info

cmcli --user cmadmin --passwd yt_xk39b remove NE/HSSDU1_DUSpec/16.5
cmcli --user cmadmin --passwd yt_xk39b remove NE/HSSDU1_Common/16.5

5) Restore cmcli config backup for HSS G8

cmcli restore /var/tmp/backuprepo_HSSDU1_DUSpec_hssG8.dmp

ie:
root@uasdmtm101ne> cmcli restore /var/tmp/backuprepo_HSSDU1_DUSpec_hssG8.dmp
Logs are captured in /tmp/cmcli-1521815860.log
Sending Request to CMREPO : 10.162.98.90
Authentication Required..
Please Enter UserName: cmadmin
Please Enter Password: ********
Server Response --->
{"Operation":"RESTORE","Status":"SUCCESS","ErrorStr":"","Resp":"Restore Data
Successful.","Mode":"","Info":
["NE","HSSDU1_DUSpec","/tmp/NE_HSSDU1_DUSpec_Backup.tar.gz"]}

6) Power on G8
press power button or go to OA GUI > Device bays > (select bay) > Virtual Buttons >
Power on

7) Status check
netstat -an |grep 16611
netstat -an |grep 3868
netstat -an |grep "12251\|12216\|12317"
csdiag.pl
su - rtp99
status1 -e
cmcli display `grep Uniq /tspinst/scripts/aiParameter.sh |sed s/=/' '/g |awk
'{print $2}'` >/var/tmp/config_dump_`hostname`.txt

8) cmrepo configuration modification regression test

cmcli display (DN/DU) |grep SupportedInterfaceIDs
cmcli MOD <du/dn> ims/common/diameterdisp InterfaceIdentities
SupportedInterfaceIDs=<new value>
cmcli MOD <du/dn> ims/common/diameterdisp InterfaceIdentities
SupportedInterfaceIDs=<old value>

ie: (note the following test has impact)

cmcli --user cmadmin --passwd yt_xk39b info |grep Uniq
cmcli --user cmadmin --passwd yt_xk39b display `grep Uniq
/tspinst/scripts/aiParameter.sh |sed s/=/' '/g |awk '{print $2}'` |grep
SupportedInterfaceIDs
cmcli --user cmadmin --passwd yt_xk39b MOD `grep Uniq
/tspinst/scripts/aiParameter.sh |sed s/=/' '/g |awk '{print $2}'`
ims/common/diameterdisp InterfaceIdentities SupportedInterfaceIDs="SLH"
--where="SupportedInterfaceIDs='SLH,CX,ZH,SWX,S6A,SH,WX,HD'"
cmcli --user cmadmin --passwd yt_xk39b MOD `grep Uniq
/tspinst/scripts/aiParameter.sh |sed s/=/' '/g |awk '{print $2}'`
ims/common/diameterdisp InterfaceIdentities
SupportedInterfaceIDs="SLH,CX,ZH,SWX,S6A,SH,WX,HD"
--where="SupportedInterfaceIDs='SLH'"

In this case, RtpTrcErr was checked to see this cmcli change test were having an
impact on the system. It worked as expected.

9) Functional tests
10) Acceptance

-----------------------------------------------------------------------

HSS G8 installation (Pre-prod)

------------------
-Prerequisite: G9 has been accepted

1) VC server profile configuration for new slot

login to VC
show config
copy and adapt server profile from old slot

ie:

add profile NE-enc0-HSS-entreprise -NoDefaultEnetConn -NoDefaultFcConn

-NoDefaultFcoeConn -quiet NAG=Default HideUnusedFlexNICs=false BootMode=Auto
SriovMode=Simplified
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=Enabled
SriovType=Disabled
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=UseBIOS
SriovType=Disabled
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=UseBIOS
SriovType=Disabled
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=UseBIOS
SriovType=Disabled
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=UseBIOS
SriovType=Disabled
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=UseBIOS
SriovType=Disabled
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=UseBIOS
SriovType=Disabled
add enet-connection NE-enc0-HSS-entreprise Network=Unassigned PXE=UseBIOS
SriovType=Disabled
add server-port-map NE-enc0-HSS-entreprise:1 NE-enc0-OAM-NetA-71 Untagged=true
add server-port-map NE-enc0-HSS-entreprise:2 NE-enc0-OAM-NetB-71 Untagged=true
add server-port-map-range NE-enc0-HSS-entreprise:3 UplinkSet=NE-enc0-NetA-X10
vlanids=131,141
add server-port-map NE-enc0-HSS-entreprise:3 NE-enc0-HSS_APP-NetA-61 Untagged=true
add server-port-map-range NE-enc0-HSS-entreprise:4 UplinkSet=NE-enc0-NetB-X10
vlanids=131,141
add server-port-map NE-enc0-HSS-entreprise:4 NE-enc0-HSS_APP-NetB-61 Untagged=true
add server-port-map-range NE-enc0-HSS-entreprise:6 UplinkSet=NE-enc0-NetB-X10
vlanids=151,161
add server-port-map NE-enc0-HSS-entreprise:7 NE-enc0-HSM1-NetA-101 Untagged=true
add server-port-map NE-enc0-HSS-entreprise:8 NE-enc0-HSM2-NetB-102 Untagged=true

assign profile NE-enc0-HSS-entreprise enc0:8

set enet-connection NE-enc0-HSS-entreprise 1 SpeedType=Custom Speed=1000

set enet-connection NE-enc0-HSS-entreprise 2 SpeedType=Custom Speed=1000
set enet-connection NE-enc0-HSS-entreprise 3 SpeedType=Custom Speed=2500
set enet-connection NE-enc0-HSS-entreprise 4 SpeedType=Custom Speed=2500
set enet-connection NE-enc0-HSS-entreprise 6 SpeedType=Custom Speed=1000
set enet-connection NE-enc0-HSS-entreprise 7 SpeedType=Custom Speed=1000
set enet-connection NE-enc0-HSS-entreprise 8 SpeedType=Custom Speed=1000

2) OA iLO config (ie ilo ip; blade name)

OA GUI > Enclosure information > Enclosure Bay IP addressing > IPv4

3) TIAMS preparation
cd /home/iserver/linux/http/static/preparation_tools
./prepTspImage.pl -mappl <HSS app medium> -moem <TSP platform medium> -cmrepo y
-CTPD <HSS Common TPD> -TPD <HSS DUSpec TPD> -SP <target SP> -mfw <firmware>

ie:
./prepTspImage.pl -mappl
/home/iserver/HSS_cap_upg/HSS_medium/IMSDL1650140.000_cpio/IMSDL1650140.000aa.cpio
-moem /home/iserver/HSS_cap_upg/TSP_CAF/LX_TSP_CAF_INST_COMMON_1651400.tar.gz
-cmrepo y -CTPD /home/iserver/HSS_cap_upg/TPD/G8_enterprise/TPD_HSSD_CoNEent165.xls
-TPD /home/iserver/HSS_cap_upg/TPD/G8_enterprise/TPD_HSSD_DUNEent165.xls -SP
/home/iserver/HSS_cap_upg/SP/IMSDL1650140.SPT.005.tar.gz

4) Test preparation file availability:

cd /var/tmp
netstat -an |grep 5571 |tail -1 |sed s/:/' '/g |awk '{print $5}'
wget https://(ip):5571/(du)/boot-(du).iso --no-check-certificate
rm boot-(du).iso

to see available isos:

locate boot- |grep autoinstall
ie:
cd /var/tmp
netstat -an |grep 5571 |tail -1 |sed s/:/' '/g |awk '{print $5}'
wget https://10.162.98.88:5571/uasdmhs102ne/boot-uasdmhs102ne.iso --no-check-
certificate
rm boot-uasdmhs102ne.iso

-
cat /etc/exports
exportfs -a

5) Set iso and connect via console

Login to OA
set server dvd connect https://10.162.98.88:5571/uasdmhs102ne/boot-uasdmhs102ne.iso
8
connect bay 8
(cdrom mount test: dir map1/oemhp_vm1/cddr1)
vsp

note: during the image load, use java console to see all error messages.

6) Set boot origin and reset blade

In OA GUI
Boot Options -> One time Boot from: cdrom -> apply
Virtual Devices -> Reset

7) Install HSS on G9
Follow HSS installation in console.

8) Configure LDAP clients in OneNDS

8.1) Login as superuser in ADM and go to:
DSA Management > LDAP Client Management

8.2) add new client:

ie:
Name: HSS (uasdmhs102ne) at NEUI
Hostname: uasdmhs102ne
IP: 10.68.57.21

click 'Add'

8.3) Apply settings:

click 'Apply to DSs'

9) Add SOAP triggers to OneNDS

9.1) Login as superuser in ADM and go to:
Notification Manager > Registration

9.2) Select SOAP service HSS-BC-OLIMPUS

Application receiver: Create new
nsNinReceiver: HSS-BC-OLIMPUS-HOST02
*nsNreEndpointUrl: http://10.68.57.22:30300
Click 'Apply'

9.3) Select SOAP service HSS-RR-OLIMPUS

Create new
HSS-RR-OLIMPUS-HOST02
http://10.68.57.22:30300
Click 'Apply'

9.4) Click 'Update Server'

10) Post install checks (check ips; config crosscheck; status check)
in tiams:
ims_getdata.py
(do TPD crosscheck)

in HSS:
netstat -an |grep 16611
netstat -an |grep "3868\|12216"
(netstat -an |grep "12251\|12216\|12317")
csdiag.pl
su - rtp99
status1 -e
ip a
ip a |grep "inet "
netstat -rn
cmcli --user cmadmin --passwd yt_xk39b display `grep Uniq
/tspinst/scripts/aiParameter.sh |sed s/=/' '/g |awk '{print $2}'`
>/var/tmp/config_dump_`hostname`.txt

11) password change

passwd root
passwd rtp99

12) Functional tests HSS G9

ie:
tshark -i any -d sctp.port==12251,diameter -R diameter.cmd.code!=280
tshark -V -i any -d sctp.port==12251,diameter -R diameter.cmd.code!=280 |grep
Result |grep val

13) Acceptance/Go live