Troubleshooting SBTS

Single RAN System, SRAN 20C,
Issue 04
Troubleshooting SBTS
DN09249569
Issue 05C
Approval Date 2021/04/06
Nokia condential
Nokia is committed to diversity and inclusion. We are continuously reviewing our customer
documentation and consulting with standards bodies to ensure that terminology is inclusive
and aligned with the industry. Our future customer documentation will be updated
accordingly.
The information in this document applies solely to the hardware/software product

(“Product”) specied herein, and only as specied herein. Reference to “Nokia” later in this
document shall mean the respective company within Nokia Group of Companies with whom
you have entered into the Agreement (as dened below).
This document is intended for use by Nokia's customers (“You”) only, and it may not be used
except for the purposes dened in the agreement between You and Nokia (“Agreement”)
under which this document is distributed. No part of this document may be used, copied,
reproduced, modied or transmitted in any form or means without the prior written
permission of Nokia. If You have not entered into an Agreement applicable to the Product,
or if that Agreement has expired or has been terminated, You may not use this document in
any manner and You are obliged to return it to Nokia and destroy or delete any copies
thereof.
The document has been prepared to be used by professional and properly trained
personnel, and You assume full responsibility when using it. Nokia welcomes your comments
as part of the process of continuous development and improvement of the documentation.
This document and its contents are provided as a convenience to You. Any information or
statements concerning the suitability, capacity, tness for purpose or performance of the
Product are given solely on an “as is” and “as available” basis in this document, and Nokia
reserves the right to change any such information and statements without notice. Nokia has
made all reasonable efforts to ensure that the content of this document is adequate and
free of material errors and omissions, and Nokia will correct errors that You identify in this
document. Nokia's total liability for any errors in the document is strictly limited to the
correction of such error(s). Nokia does not warrant that the use of the software in the
Product will be uninterrupted or error-free.
NO WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO
ANY WARRANTY OF AVAILABILITY, ACCURACY, RELIABILITY, TITLE, NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, IS MADE IN RELATION TO THE
CONTENT OF THIS DOCUMENT. IN NO EVENT WILL NOKIA BE LIABLE FOR ANY DAMAGES,
INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT, INCIDENTAL OR
CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED TO LOSS OF PROFIT,
REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY OR DATA THAT MAY ARISE
FROM THE USE OF THIS DOCUMENT OR THE INFORMATION IN IT, EVEN IN THE CASE OF
2 © 2021 Nokia.Nokia confidential

ERRORS IN OR OMISSIONS FROM THIS DOCUMENT OR ITS CONTENT.
This document is Nokia proprietary and condential information, which may not be
distributed or disclosed to any third parties without the prior written consent of Nokia.
Nokia is a registered trademark of Nokia Corporation. Other product names mentioned in

this document may be trademarks of their respective owners.
Copyright © 2021 Nokia. Nokia condential.
Important Notice on Product Safety
This product may present safety risks due to laser, electricity, heat, and other
sources of danger.
Only trained and qualied personnel may install, operate, maintain or otherwise
handle this product and only after having carefully read the safety information
applicable to this product.
The safety information is provided in the Safety Information section in the “Legal,
Safety and Environmental Information” part of this document or documentation
set.
Nokia is continually striving to reduce the adverse environmental effects of its products and
services. We would like to encourage you as our customers and users to join us in working
towards a cleaner, safer environment. Please recycle product packaging and follow the
recommendations for power use and proper disposal of our products and their components.
If you should have questions regarding our Environmental Policy or any of the environmental
services we offer, please contact us at Nokia for any additional information.
© 2021 Nokia.Nokia confidential 3


Table of Contents
Summary of changes .................................................................................................................. 15
1 Troubleshooting workflow ......................................................................................................... 17

1.1 How to contact Nokia technical support ...................................................................... 17
1.1.1 How to open a support ticket by phone or email ........................................... 18
1.1.2 How to open a support ticket online ................................................................. 19
2 Troubleshooting tools ............................................................................................................... 23

2.1 Troubleshooting with WebEM ........................................................................................ 23
2.1.1 Alarm monitoring using WebEM ......................................................................... 23
2.1.1.1 Site view ..................................................................................................... 23
2.1.1.2 Detailed site view ...................................................................................... 25
2.1.1.3 Alarms in WebEM ....................................................................................... 29
2.1.2 Counter and KPI monitoring in WebEM ............................................................. 30
2.1.2.1 Performance Management view ............................................................. 30
2.1.2.2 Performance Management (PM) counter data ..................................... 32
2.1.2.3 KPI Dashboard ........................................................................................... 33
2.1.3 Configuration management in WebEM ............................................................. 34
2.1.3.1 Compare Objects ...................................................................................... 34
2.1.4 Diagnostic tools in WebEM .................................................................................. 35
2.1.4.1 Synchronization ........................................................................................ 35
2.1.4.2 IP Connectivity Test ................................................................................. 36
2.1.4.3 RF Diagnostic Test .................................................................................... 37
2.1.4.4 EAC Functionality Test ............................................................................. 38
2.1.4.5 Ethernet Port Mirroring ........................................................................... 39
2.1.4.6 IP Traffic Capturing .................................................................................. 40
2.1.4.7 SFP monitoring .......................................................................................... 41
2.1.4.8 Antenna Line Online Monitoring ............................................................. 42
2.1.4.9 RF Monitoring ............................................................................................ 42
2.1.4.9.1 PIM Desensitization ....................................................................... 43
2.1.4.9.2 Distance to PIM (DTP) ................................................................... 43
2.1.4.9.3 RF Scan ............................................................................................ 44
2.1.4.10 Test Models ............................................................................................. 44
2.1.4.10.1 LTE Downlink ................................................................................ 44
2.1.4.10.2 LTE Uplink ..................................................................................... 45
2.1.4.10.3 WCDMA Downlink ........................................................................ 46
2.1.4.10.4 WCDMA Uplink .............................................................................. 46
2.1.4.10.5 GSM BCCH Transmission ............................................................ 47
2.1.4.10.6 GSM Loop Test ............................................................................ 47
2.1.4.11 Terminal ................................................................................................... 48
2.1.4.12 Snapshot .................................................................................................. 48
2.1.4.13 Reset to test dedicated state .............................................................. 49

2.1.4.14 TWAMP RTT Measurements .................................................................. 49

2.1.4.15 Ethernet link OAM ................................................................................... 50
2.1.4.16 Ethernet Service OAM ............................................................................ 52
2.1.4.17 PPP Interfaces ......................................................................................... 53
2.1.4.18 IP Routing ................................................................................................ 53
2.1.4.18.1 IPv4/IPv6 Routing ........................................................................ 53
2.1.4.18.2 Routing Policies ........................................................................... 54
2.1.4.19 IP Security Associations ......................................................................... 55
2.1.4.20 PMTU Discovery ...................................................................................... 56
2.1.4.21 GSM Traffic Trace ................................................................................... 56
2.1.4.22 PDH Loopback ......................................................................................... 57
2.1.5 Reset, block and unblock in WebEM ................................................................... 57
2.1.5.1 Finding information about reset reasons in a snapshot file .............. 57
2.1.5.2 Blocking and unblocking BTS units ......................................................... 58
2.1.5.3 Resetting BTS units .................................................................................. 60
2.1.5.4 RAT reset .................................................................................................... 61
2.1.6 Transfer operations ............................................................................................. 62
2.1.6.1 WebEM file transfer operations failed ................................................... 62
2.2 Troubleshooting with NetAct ........................................................................................ 63
2.2.1 Performance monitoring in NetAct ................................................................... 64
2.2.2 Alarm monitoring in NetAct ................................................................................ 64
2.2.3 Reporting tool and default reports ................................................................... 64
2.2.4 Tracing and MDT .................................................................................................. 64
2.3 Troubleshooting with Traffica ....................................................................................... 65
2.3.1 Network monitoring using Traffica .................................................................... 65
2.3.1.1 Traffica general use case for LTE and WCDMA troubleshooting ....... 67
2.3.2 Activating the real-time KPI-monitoring with Traffica feature ..................... 68
2.3.3 Deactivating the real-time KPI-monitoring with Traffica ............................... 70
2.3.4 Recovery procedure for Traffica reports when they are not received by
Traffica tool ............................................................................................................. 70
2.4 Troubleshooting with DCAP ........................................................................................... 73
2.4.1 Troubleshooting with L3DC (Megamon) ........................................................... 74
2.4.2 Troubleshooting with L3DA (Emil) ..................................................................... 75
2.4.2.1 Triggering snapshot collection using L3DA (Emil) ................................ 76
2.4.3 Troubleshooting with GEO Interface ................................................................. 77
2.5 Troubleshooting with PTF .............................................................................................. 77
2.6 Troubleshooting with RFS .............................................................................................. 78
3 Collecting and analyzing troubleshooting data ...................................................................... 79

3.1 Remote syslog collection for continuous log storage ............................................... 79
3.1.1 Remote syslog overview ..................................................................................... 79
3.1.2 Remote syslog troubleshooting ........................................................................ 81
3.2 BTS snapshot ................................................................................................................... 82
3.2.1 Collecting BTS snapshot manually .................................................................... 83

3.2.1.1 Saving a BTS snapshot file using Web Element Manager .................... 83
3.2.2 Collecting BTS snapshot automatically ............................................................ 85
3.2.2.1 Enabling snapshot collection based on TRBLCADM-triggered events
............................................................................................................................ 85
3.2.2.1.1 Enabling snapshot collection triggered by a fault or alarm
.................................................................................................................... 85
3.2.2.1.2 Enabling snapshot collection triggered by the occurrence of
certain strings in the syslog ................................................................... 88
3.2.3 RF related logs ...................................................................................................... 90
3.3 Taking Info Model snapshots (IMS2) with WebEM ....................................................... 90
3.4 WebEM logs ...................................................................................................................... 90
3.4.1 Collecting application logs .................................................................................. 91
3.4.2 Collecting browser console logs (JAVA logs) .................................................... 91
3.4.3 Collecting browser network logs (HAR logs) ..................................................... 93
4 Fault monitoring ......................................................................................................................... 94

4.1 Fault and alarm troubleshooting outline ..................................................................... 94
4.2 Finding faults in a snapshot file .................................................................................... 94
4.3 Sleeping cell ..................................................................................................................... 96
4.3.1 Basic sleeping cell troubleshooting procedure ............................................... 96
4.3.2 Sleeping cell troubleshooting when alarms are raised ................................... 97
4.3.3 Sleeping cell troubleshooting when alarms are not raised ............................ 98
5 Radio unit monitoring and troubleshooting ........................................................................... 99

5.1 Antenna line device problems ....................................................................................... 99
5.1.1 Antenna line, VSWR, and MHA troubleshooting outline ................................. 99
5.1.2 Radio Access Server (RAS), Remote Electrical Tilt (RET) troubleshooting guide
.................................................................................................................................. 101
5.1.3 RET not detected ............................................................................................... 101
5.1.4 RET not calibrated or jammed ......................................................................... 102
5.2 RRH troubleshooting outline ....................................................................................... 103
5.3 Fault 1807 raised when bandwidth or frequency is exceeded .............................. 104
5.4 Fault 1811 raised when routing direction changed on the radio unit .................. 105
5.5 Faults 4309 and 4310: Connection unstable ........................................................... 106
5.6 Radio performing autonomous reset ........................................................................ 107
5.7 Nokia CPRI radio not detected after BTS software upgrade .................................. 107
6 RAT specific troubleshooting ................................................................................................. 109

6.1 GSM troubleshooting ................................................................................................... 109
6.1.1 GSM related logs ................................................................................................ 109
6.1.1.1 BTS logs ................................................................................................... 109
6.1.1.2 BSC logs ................................................................................................... 111
6.1.1.3 Tools ......................................................................................................... 111
6.2 WCDMA troubleshooting .............................................................................................. 112
6.2.1 WCDMA related logs .......................................................................................... 112

6.2.1.1 RNC logs ................................................................................................... 112

6.2.2 WCDMA use cases .............................................................................................. 113
6.2.2.1 KPI degradation ...................................................................................... 113
6.2.2.2 Fault 110 is raised when the WCDMA cell is blocked with the
operational state disabled .......................................................................... 114
6.3 LTE troubleshooting ..................................................................................................... 115
6.3.1 LTE related logs ................................................................................................. 115
6.3.1.1 Cell trace .................................................................................................. 117
6.3.1.1.1 Cell trace content ....................................................................... 117
6.3.1.1.2 Activating trace ........................................................................... 118
6.3.1.1.3 Viewing and analyzing cell trace data using L3DA (Emil) ....... 119
6.3.1.1.4 Tracing with NetAct .................................................................... 121
6.3.2 BTSMED ............................................................................................................... 121
6.3.3 LTE use cases ..................................................................................................... 122
6.3.3.1 Cell problems .......................................................................................... 124
6.3.3.1.1 Cell is not in on-the-air state .................................................... 124
6.3.3.2 Call processing problems ...................................................................... 125
6.3.3.2.1 Troubleshooting call processing problems ............................. 125
6.3.3.2.2 Parameters influencing call establishing and processing ..... 126
6.3.3.2.3 Data related to high RRC setup failure rate and RACH problems
.................................................................................................................. 131
6.3.3.2.4 Data related to RAB setup problems ....................................... 132
6.3.3.2.5 VoLTE/ViLTE call drops when LTE1321 is enabled in the network
.................................................................................................................. 132
6.3.3.3 BTS crash or reset .................................................................................. 134
6.3.3.3.1 BTS in a reset loop ...................................................................... 134
6.3.3.3.2 Baseband pool reset for fault recovery ................................... 135
6.3.3.3.3 Adding new frequency or cell without BTS restart ................ 137
7 Software and synchronization troubleshooting .................................................................. 139

7.1 Software problems ....................................................................................................... 139
7.1.1 Viewing a software download report .............................................................. 139
7.1.2 Reset to a failsafe partition ............................................................................. 141
7.1.2.1 Resetting to failsafe partition - case 1 ............................................... 141
7.1.2.2 Resetting to failsafe partition - case 2 ............................................... 142
7.1.3 Activating and using WebEM Remote Tool (former BTS CLI) when Web
Element Manager is unavailable ......................................................................... 143
7.1.3.1 Activating WebEM Remote Tool (former BTS CLI) when Web Element
Manager is unavailable ................................................................................. 143
7.1.3.2 Performing a software rollback using WebEM Remote Tool (former BTS
CLI) when Web Element Manager is unavailable ....................................... 146
7.1.4 Commissioning problems - RMOD link synchronization and SW update ... 148
7.2 Synchronization problems ........................................................................................... 149
7.2.1 GPS problems – synchronization source not detected ................................ 149

7.2.2 Fallback solution for phase synchronization ................................................. 150

7.3 System upgrade use cases .......................................................................................... 151
7.3.1 ALDs connected to the antenna line and/or RS485 ports not detected after
system upgrade .................................................................................................... 152
7.3.2 Fault 10 raised on one BTS after successful software upgrade ................. 153
7.3.3 Keeping legacy cell set in use after system upgrade ................................... 154
7.3.4 Fault 23: Incompatible SW version detected is raised after software fallback
.................................................................................................................................. 156
7.3.5 DSS cell degraded after system upgrade ...................................................... 156
7.3.6 Radio unit not detected and not configured after system upgrade ......... 157
8 System module troubleshooting ........................................................................................... 159

8.1 Replacing the master system module (MSM) ............................................................ 159
8.2 Replacing the extension system module (ESM) ........................................................ 159
8.3 Restoring system module after replacement ........................................................... 160
8.4 Recovering the system module with the RFS tool ................................................... 164
9 Hardware monitoring and troubleshooting .......................................................................... 171

9.1 Performing routine maintenance tasks ..................................................................... 171
9.2 Identification of faulty units ........................................................................................ 174
9.3 Troubleshooting the transmission module ............................................................... 176
9.3.1 Dead peer detected (error in SCTP) ................................................................ 176
9.3.2 Ethernet interface not working ....................................................................... 176
9.4 LED indications .............................................................................................................. 178
9.4.1 Nokia AirScale System Module LEDs ............................................................... 178
9.4.1.1 AirScale system module common plug-in unit (ASIA and ASIB) LEDs
.......................................................................................................................... 178
9.4.1.1.1 ASIA LED indicators ..................................................................... 181
9.4.1.1.2 ASIB LED indicators ..................................................................... 181
9.4.1.2 AirScale capacity extension plug-in unit (ABIA) LEDs ........................ 181
9.4.2 Flexi Multiradio 10 LEDs ................................................................................... 183
9.4.2.1 Flexi Multiradio 10 System Module (FSMF) LEDs ............................... 183
9.4.2.2 Capacity extension sub-module (FBBA) LEDs .................................... 185
9.4.2.3 Capacity extension sub-module (FBBC) LEDs .................................... 186
9.4.2.4 Flexi Multiradio 10 System Module (FSIH) LEDs ................................. 187
9.4.2.5 Capacity extension sub-module (FBIH) LEDs ..................................... 189
9.4.3 Fronthaul switch LEDs ....................................................................................... 190
9.4.3.1 MDEA mobile fronthaul switch outdoor DC LEDs .............................. 190
9.5 Optical fibers ................................................................................................................. 192
9.5.1 Maintaining optical fibers ................................................................................. 192
9.5.2 Cleaning optical fibers ...................................................................................... 194
9.5.3 Fault 3030: Failure in optical interface .......................................................... 196
9.5.4 Optical cable connection issue with FRGU, FRMF and FXEE radios ............ 198
9.5.5 CPRI-A radio issues in a Nokia network .......................................................... 200
9.6 Fault 4078 triggers temperature check for AMOB AirScale subrack ..................... 201

10 Related troubleshooting documents .................................................................................. 203
11 Glossary of terms ................................................................................................................... 205

List of Figures
Figure 1 Troubleshooting methodology ................................................................................... 17
Figure 2 Emergency contact field .............................................................................................. 19
Figure 3 Support portal Create Software Support Ticket step ............................................. 20
Figure 4 Support portal severity field ....................................................................................... 20
Figure 5 Support portal legal entity, impacted product, and entitlement fields ............... 21
Figure 6 Support portal form with subject, description, and reference number fields
........................................................................................................................................... 21
Figure 7 Site View ......................................................................................................................... 24
Figure 8 Site action options ....................................................................................................... 25
Figure 9 Detailed Site View ......................................................................................................... 26
Figure 10 System module marked as master .......................................................................... 28
Figure 11 Site action options ..................................................................................................... 28
Figure 12 Table View ................................................................................................................... 31
Figure 13 Plot View ...................................................................................................................... 31
Figure 14 KPI Dashboard ............................................................................................................ 33
Figure 15 Compare Object view in WebEM ............................................................................... 34
Figure 16 Ethernet Port Mirroring ............................................................................................. 40
Figure 17 Capture point options for IP traffic capturing ...................................................... 40
Figure 18 LTE Downlink tab view ............................................................................................... 45
Figure 19 LTE Uplink tab view .................................................................................................... 45
Figure 20 WCDMA Downlink tab view ........................................................................................ 46
Figure 21 WCDMA Uplink tab view ............................................................................................. 47
Figure 22 BTS site information .................................................................................................. 49
Figure 23 Checking critical link events ...................................................................................... 52
Figure 24 GSM Traffic Trace view .............................................................................................. 56
Figure 25 PDH Loopback ............................................................................................................ 57
Figure 26 Information about reset reasons in the BTSxxxx_1011_blackbox file ............... 58
Figure 27 Details pane for the unit selected for blocking or unblocking ............................ 59
Figure 28 Confirmation window for blocking or unblocking a unit ....................................... 60
Figure 29 Details pane for the unit selected for reset .......................................................... 61
Figure 30 Confirmation window for the reset action of a unit ............................................. 61
Figure 31 Site Status bar ............................................................................................................ 62
Figure 32 Reset confirmation .................................................................................................... 62
Figure 33 An overview of Traffica integration in the network ............................................... 66
Figure 34 Simplified tool solution architecture ....................................................................... 75
Figure 35 Selecting triggers for eNB snapshot collection ..................................................... 77
Figure 36 Customer log server .................................................................................................. 80
Figure 37 Selecting the target location .................................................................................... 84
Figure 38 Navigation Panel Timeline ......................................................................................... 90
Figure 39 Finding an alarm ID in the RawAlarmHistory.txt file .............................................. 95

Figure 40 Table of parameters ................................................................................................ 114

Figure 41 Overview of symptoms for call processing related problems ........................... 115
Figure 42 Software update failed message ........................................................................... 140
Figure 43 Software update report .......................................................................................... 141
Figure 44 Selecting the Workflow Engine... option .............................................................. 161
Figure 45 Selecting the LTE1992: Flexi eNB configuration restore option ...................... 162
Figure 46 An example of the configuration of restore operation ...................................... 163
Figure 47 Configuring SON Scheduler .................................................................................... 163
Figure 48 Target BTS software located in the RFS tool directory ...................................... 165
Figure 49 Remove operator certificate .................................................................................. 166
Figure 50 Activate non-running partition (rollback) ............................................................. 166
Figure 51 Clear BTS configuration .......................................................................................... 167
Figure 52 Result view 1 ............................................................................................................. 168
Figure 56 LMP IP plug installed ................................................................................................ 172
Figure 57 LMP port IP cap not reinstalled after maintenance ............................................ 172
Figure 58 Connector IP boot correctly installed firmly in place ......................................... 172
Figure 59 Connector IP boot incorrectly installed (IP seals not firmly in place on any edge)
......................................................................................................................................... 173
Figure 60 Connector IP boot incorrectly installed (not pushed all the way in) ................. 173
Figure 61 Cable installed without IP boot .............................................................................. 173
Figure 62 Launching Physical Interfaces configuration window ......................................... 177
Figure 63 Location of ASIA LED indicators ............................................................................ 181
Figure 64 Location of ASIB LED indicators ............................................................................ 181
Figure 65 Location of ABIA LED indicators ............................................................................ 182
Figure 66 LEDs of the Flexi Multiradio 10 System Module (FSMF) ..................................... 184
Figure 67 LEDs of the capacity extension sub-module (FBBA) ........................................... 185
Figure 68 LEDs of the capacity extension sub-module (FBBC) .......................................... 187
Figure 69 LEDs of the Flexi Multiradio 10 System Module (FSIH) ...................................... 188
Figure 70 LEDs of the capacity extension sub-module (FBIH) ............................................ 189
Figure 71 LEDs of the MDEA mobile fronthaul switch outdoor DC .................................... 190
Figure 72 Physically damaged fiber ........................................................................................ 193
Figure 73 Fiber test equipment .............................................................................................. 193
Figure 74 Fiber cleaner ............................................................................................................. 194
Figure 75 Example of connector cleaner ............................................................................... 195
Figure 76 Example of how to clean a fiber ............................................................................ 195
Figure 77 Example of how to clean a fiber ............................................................................ 196
Figure 78 A. Installing optical cable connector seal B. Collision between connector seal and
latch. .............................................................................................................................. 198
Figure 79 Problematic and non-problematic connectors ................................................... 199

Figure 80 Removing connector lock-and-release latch ....................................................... 199

List of Tables
Table 1 Report names and numeric report identifiers in a computer log ........................... 73
Table 2 RUs with product codes and versions ...................................................................... 108
Table 3 Required logs for GSM call processing ..................................................................... 110
Table 4 Tools needed for log collection ................................................................................. 112
Table 5 LTE use cases ............................................................................................................... 122
Table 6 Parameters influencing establishing a call ............................................................... 126
Table 7 Parameters influencing call processing .................................................................... 126
Table 8 Explanation of NetAct Diagnostic Info paths .......................................................... 174
Table 9 LED indicators .............................................................................................................. 179
Table 10 ABIA LED indicators ................................................................................................... 182
Table 11 LEDs of the Flexi Multiradio 10 System Module (FSMF) ...................................... 184
Table 12 LEDs of the capacity extension sub-module (FBBA) ........................................... 186
Table 13 LEDs of the capacity extension sub-module (FBBC) ........................................... 187
Table 14 LEDs of the Flexi Multiradio 10 System Module (FSIH) ........................................ 188
Table 15 LEDs of the capacity extension sub-module (FBIH) ............................................. 189
Table 16 LEDs of the MDEA mobile fronthaul switch outdoor DC ..................................... 190
Table 17 Visual identification of SFPs and fibers .................................................................. 197

Summary of changes
A list of changes between document issues. You can navigate through the respective
changed topics.
This document is based on the document SBTS Troubleshooting, release 19, Issue 01.
Changes between issues 05B (2021-01-12, SRAN 20C) and 05C (2021-03-09,
SRAN 20C)
Radio unit not detected and not configured after system upgrade
A new procedure on how to troubleshoot if radio units (RUs) are not detected or
configured after system upgrade was added.
DSS cell degraded after system upgrade
A new section on how to configure the NRCELL PRACH Frequency start

(msg1FrequencyStart) parameter if there's a dynamic spectrum sharing (DSS) cell
degradation after system upgrade was added.
Nokia CPRI radio not detected after BTS software upgrade
A procedure on how to configure the CABLINK Link speed (linkspeed) parameter

after the radio software is upgraded was added.
Changes between issues 05A (2020-11-20, SRAN 20C) and 05B (2021-01-12,
SRAN 20C)
Fault 4078 triggers temperature check for AMOB AirScale subrack
A new section on fault 4078: Cooling fan broken and 4080: Cooling fan is
over speeding updates to check the AMOB AirScale subrack temperature, when the
fan is suspected to be broken, was added.
Fault 23: Incompatible SW version detected is raised after software fallback
A new section on fault 23: Incompatible SW version detected detected during

software fallback process was added.

Keeping legacy cell set in use after system upgrade
A procedure on how to resolve the degradations caused after system upgrade, when the
Activate prohibition of CA-specific cell deployments
(actProhibitCaSpecCellDepl) parameter is set to the false default value.
Changes between issues 05 (2020-09-16, SRAN 20C) and 05A (2020-11-20,

SRAN 20C)
Fault 110 is raised when the WCDMA cell is blocked with the operational state disabled
A new section on how to avoid triggering fault 110 was added.
Fault 10 raised on one BTS after successful software upgrade
A new section on how to avoid triggering fault 10 after software upgrade was added.

1. Troubleshooting workflow
The troubleshooting methodology consists of the following procedures: problem
identification, monitoring, collecting and analyzing data and finding the fault scenario.
Introduction
The purpose of this document is to give instructions on how to proceed with common
problematic situations that can appear on the SBTS.
Methodology
Troubleshooting involves monitoring, collecting and analyzing data, finding the impacted
domain and taking appropriate actions to resolve issues. The Figure: Troubleshooting
methodology presents a general example of the procedures performed and tools used
during troubleshooting.
You can decide on the order of actions and select the tools, depending on the use case.
You can do the root cause analysis for a problem by yourself or seek Nokia technical support
for help. For example, for a site configuration related problem, follow the procedures
described in this document to find the impacted domain and initiate the resolution actions.
If it's impossible to specify the cause or the problem concerns possible defects inside the
product, see the How to contact Nokia technical support.
Figure 1: Troubleshooting methodology
1.1 How to contact Nokia technical support

Software support refers to the existing technical support, software maintenance, or
emergency support, according to the contract. The software support request creates a
support ticket categorized as customer support.

Introduction
You can create and access your support tickets through the Support portal.
The support ticket can be created in two ways:

By phone or email
Online on the Support Portal
Support ticket classification based on TL9000 severity

The support ticket flow is based on the severity of the incident.
Critical tickets are created for:

product inoperability.
a significant reduction in performance (for example, traffic or data handling capability).
any loss of emergency capabilities (for example, emergency calls or duplex failure).
safety hazard or risk of security breach.
Major tickets are created for:

reduction in product capability, but can still handle for example, simplex failure or loss of
redundancy.
any loss of administrative or maintenance visibility of the product or the diagnostic
capability.
repeated degradation of an essential component or function.
degradation of the product's ability to provide any required notification of malfunction.
Minor tickets are created for:

limited deterioration for software or feature use on system level.
issues that occur on non-production environments.
1.1.1 How to open a support ticket by phone or email

For all critical or outage incidents, call the Nokia helpdesk. These incidents cannot be
raised online.
Procedure
1 Go to the Support portal.

2 From the Emergency Contact field, select the country from the drop-down list
and click the Submit button.
Result
A list of telephone numbers and email addresses to contact Nokia Software Support is
generated.
Figure 2: Emergency contact field
1.1.2 How to open a support ticket online

The online tickets can be created for issues with major severity without outage, minor
severity, or for information request.
Procedure
1 Go to the Support portal.
2 Go to Support tab ► Support Tickets ► Create Software Support

Ticket.

Figure 3: Support portal Create Software Support Ticket step
3 Select the severity from the drop-down list.
You can only select major or minor severity for the tickets created online. Outage or
critical severity tickets can be created by phone or email.
Figure 4: Support portal severity field
4 Select one of the Legal Entity options available to you, then click the Next
button.
5 Select the impacted product, then click the Next button.
6 Select the correct entitlement, then click the Next button.

Figure 5: Support portal legal entity, impacted product, and entitlement fields
7 Fill in the Subject, Reference Number (optional field), and Customer

Description fields for your support ticket, then click the Save button.
Figure 6: Support portal form with subject, description, and reference number fields
Do not use screenshots in the description filed. They can be added as attachments
after the ticket is created.
8 (Optional) Add additional contacts to receive email notifications.

9 Click the Save button.

2. Troubleshooting tools
2.1 Troubleshooting with WebEM

The Web Element Manager (WebEM) enables you to monitor faults, alarms, counters and
KPIs, and use the diagnostic tools for the troubleshooting cases.
For saving different types of WebEM logs, see the WebEM logs section.
For general WebEM use cases, see the Single RAN Operating Documentation / Single RAN
System / WebEM User Guide document.
2.1.1 Alarm monitoring using WebEM

WebEM provides the possibility to view faults and alarms in: Site View, Detailed Site View,
and Alarm Management menu.
2.1.1.1 Site view

This view shows the configuration layout and related information.
Access to Site View view: BTS Status ► Site Runtime View ► Site View
Shortcut keys: S+V
Site View shows the hardware available on the site, as well as the configured cells.
Note:
If the cell is not in OnAir state the information about the highlighted path between the
cell and the hardware that served it may be incomplete.

Figure 7: Site View
Elements shown in Site View can be filtered by:
Hardware
Show all
RETs/RETUs
Cell Group
BTS Mediators
MHAs/LNAs
External SMODs
PWRMODs (FPFHs)
SMODs/BBMODs
Other Node
RAEs/RAEUs
BTS Controllers
Fronthaul switches
Master Node
RATs
Show all
LTE
WCDMA
GSM
Radio ports
Show all ports

Show only used ports
The Export as image icon allows the user to save the site view in PNG format.
In configurations with two FSMF system modules, the modules are displayed in two separate
cabinets, due to the physical placement of the modules. In configurations with two ASIA or
ASIAA system modules, the modules are displayed in one cabinet.
In Site View, some alarms can appear. If an alarm occurs in a particular unit or cell, it is
indicated by the alarm icon (the color of the icon represents the severity of the alarm).
In case of multiple alarms, the alarm icon on a particular unit indicates only the highest
severity, although there may be other alarms. To see the list of alarms for a unit, select it
and go to the Details panel and open a Faults tab. To see more details about an alarm,
use the icon to go to the Active Alarms tab (Alarm Management). The list of alarms
is also visible on a tooltip for that alarm.
Selecting any of the elements displays more details, such as basic information (for example
state and name), and related parameters and faults, when the Details panel is expanded.
When applicable, some items can be reset, blocked, or unblocked using the appropriate
buttons from the Details panel.
At the top of the view there are site action options, used to block or unblock the selected
RAT, reset the site, or block and unblock the site. Note that the unblock site function causes
the entire BTS to reset.
Figure 8: Site action options
CAUTION!
Blocking the radio does not turn the power off. Switch off the radio before making any
physical changes to the equipment.
Clicking the Launch ASiR Web EM button opens a new browser window with the ASiR
element manager. This option is available only when BTS is working in LTE-only mode and
the FEATCADM Activate support for ASi (actAsi) parameter value is set to true.
2.1.1.2 Detailed site view

This view shows the layout and related information.
Access to Detailed Site View view: BTS Status ► Site Runtime

View ► Detailed Site View
Detailed Site View shows the hardware available on the site and configured cells, as
well as physical and logical links between them (including used ports).
Figure 9: Detailed Site View
Connections between site elements and Antenna Line Devices (ALDs) can be filtered from
the upper menu. Note that the displayed fiber lengths might differ from the actual lengths.
Available filters:
Cell carriers:
Show cells carriers
ALDs:
Show all
Show MHAs
Show RETs
Show RAEs
ALD links:
Show all
Show configuration links
Show data links
Show DC links
Show RET/RAE unit associations
All elements can also be sorted. FHSs, RMODs and ALDs can be sorted by:

Configuration ID
Product name
HW type (only for ALD)
State
Cells can be sorted by:
Configuration ID
Band
State
In the Detailed Site View, some alarms can appear. If an alarm occurs in a particular
unit, it will be indicated by the alarm icon (the color of the icon represents the severity
of the alarm).In case of multiple alarms, the alarm icon on a particular unit indicates only the
highest severity, although there may be other alarms. To see the list of alarms for that unit,
go to the Details panel and open the Faults tab. To see even more details about the
alarm, use the icon to go to the Active Alarms tab (Alarm Menagement). A list of
alarms is also visible on a tooltip for that alarm.
Note:
The Detailed Site View tab in Web Element Manager does not reflect properly
fronthaul cable ports (OPT1,OPT2) actually used by the RUs. For example, for AEQD a
physical cable is connected to an OPT_IF2 port, but on the Detailed Site View tab
shows the cable is connected to port OPT_IF1. This visual mismatch does not cause in
additional alarms nor impacts a BTS functional operation.
The Export as image icon allows the user to save the site view in PNG format.
Selecting the cabinet allows the user to see its details such as basic information, related
parameters and faults, when the Details panel is expanded. The Show physical
cabinet button opens a pop-up view displaying the hardware available in the cabinet
(including hardware names and port names). Clicking anywhere outside the pop-up window
closes it.
In configurations with two FSMF system modules, the modules are displayed in two separate
cabinets, due to the physical placement of the modules. In configurations with two ASIA or
ASIAA system modules, the modules are displayed in one cabinet.
The master module is marked by the M icon.

Figure 10: System module marked as master
Selecting any of the modules, physical connectors, antenna connectors, cells or carriers,
allows you to see their details such as basic information (state, name, and so on), and
related parameters and faults, when the Details panel is expanded. The Show Channel
Relation button highlights links and other objects connected to a selected object. If a
channel is selected, all the hardware through which data is routed, or elements creating that
channel are highlighted. If the selected object is a unit, the button shows the relation
between all channels creating or routing data to or from it. If a cell is selected, the button
shows the channel relation for this cell.
The Diagnostics button opens the diagnostics for the selected transport unit. When
applicable, units, cells or GSM TRXs can be reset, blocked or unblocked using the appropriate
buttons from the Details panel. The icon informs about ongoing procedures for
objects. A list of procedures is on a tooltip of this icon. A blocked object is indicated by the
icon.
At the top of the view there are site action options, used to block or unblock the selected
RAT, reset the site or block and unblock the site. Note that the unblock site function causes
the entire BTS to reset.
Figure 11: Site action options
CAUTION!
Blocking the radio does not turn the DC power off. Switch off the radio before making
any physical changes to the equipment.
Other icons that might appear on objects:

- active UEs in LTE cell

- not configured hardware

- external ALD
- satellite count
In case any inband or guardband IoT cells are configured on the BTS, these are represented
in Runtime View next to the hosting FDD-LTE cell.
Clicking the Launch ASiR Web EM button opens a new browser window with the ASiR
element manager. This option is available only when the BTS is working in LTE-only mode
and the FEATCADM Activate support for ASi (actAsi) parameter value is set to
true.
2.1.1.3 Alarms in WebEM

Alarm Management allows to see details on active BTS faults as well as fault history.
Access to Active Alarms view: Alarms ► Alarm Management ► Active Alarms
Access to Alarm History view: Alarms ► Alarm Management ► Alarm History
Alarm Managementis divided into two tabs: Active Alarms showing the actual (real-
time) data about BTS faults, and Alarm History showing the historical data. Both tabs are
configured and navigated in the same way.
Note:
The alarms in the Alarm History tab are available only when history is loaded in the
Navigation Panel.
Details of a selected alarm, such as time of appearance, description, and instructions on how
to deal with the fault, are visible in the Details panel.
Alarms can be sorted using the text field and ordered by severity or impacted RAT using the
Appeared column.
The Show Source button in the Details panel navigates to the Detailed Site View
and selects the object that triggered the alarm.
The Objects tab in the Navigation Panel can be used to filter alarms by the related
managed object.
Historical information goes back to a maximum of 24 hours or the last BTS reset.
Faults can be filtered by using:

Text field to filter any alarm names, IDs, dates, objects that triggered the alarm
containing the provided text fragment
Alarm type:
Critical alarms, presented by the icon
Major alarms, presented by the icon
Minor alarms, presented by the icon
Warnings, presented by the icon
Technology:
LTE
WCDMA
GSM
General BTS alarms
Additionally, the information presented in the table can be ordered and saved by selecting a
column and saving to a CSV file by using the Save button: .
Note:
Alarms before NTP synchronization can have incorrect timestamps, which are not
updated after NTP synchronisation.
Fault Toggling History

Fault toggling is a functionality that allows the user to suppress alarms that are raised and
cleared in a short period of time, thus having no further impact on the fault analysis. This
view enables generating and saving a Fault Toggling History report.
2.1.2 Counter and KPI monitoring in WebEM

Using Web Element Manger, it is possible to view counters and KPIs using two menus:
Performance Management (PM) and KPI Dashboard. Here you can collect the PM counter
data on a 24 hour period.
2.1.2.1 Performance Management view

Access to Performance Management view: Performance ► Performance
Management ► BTS Counters/Transport Counters/Real-time Measurements

Performance Management contains the following tabs: BTS Counters (opened by default),
Transport Counters and Real-time Measurements. Performance data is gathered
and presented in real time as long as WebEM is connected to the BTS.
After selecting an object from the Navigation Panel ► Objects, the related counters
and their values are displayed in either table view or plot view. The Table/Plot View
button is used to switch between the two. The selected counters are listed in the Details
pane, each counter with a different color assigned. The detailed value is displayed once
clicked on the chart. Up to 20 items can be displayed at the same time. The data is displayed
for the time intervals specified in a Display data for drop-down list.
Figure 12: Table View
Figure 13: Plot View
The date and time of the performance measurement are used for BTS and TRS counters.

Real-time measurements are displayed in the BTS time.
In online mode (when WebEM gathers data from the BTS), the BTS Counters tab shows
historical values in table view and real-time data in chart view by default. It is possible to
show values for a selected timespan, when one is selected from the Navigation
Panel ► Timeline. For BTS and TRS counters, all data gathered up to the current
timestamp is available.
In offline mode, when data is loaded from a snapshot, the BTS Counters tab shows the
latest value and saved data in a chart view by default. It is possible to show values of real-
time and hardware monitoring for a selected timespan, when one is selected from the
Navigation Panel ► Timeline panel. For BTS and TRS counters, all the data gathered
up to the current timestamp is available.
When the InfoModel snapshot file (IMS2) is used, only real-time measurements and hardware
monitoring are available. When a snapshot is used, the Performance Management (PM) files
are also available, and data from all PM files from the snapshot is displayed.
Items in Navigation Panel can be filtered by name. Users can also manually select
counters and save a filter as an XML, by using the Export filter button in Navigation
Panel. To load a saved filter, click Import filter.
The Save Snapshot option allows to save the currently visible graph as a PNG file (only in
graph view).
BTS counters can be exported using the Export to CSV button.
2.1.2.2 Performance Management (PM) counter data

Preconditions and collection of Performance Management (PM) counter data.
Preconditions for PM counter data collection

The PM counter data can be collected either via Web Element Manager (covering a 24-hour
history) or via NetAct.
Collecting PM counter data using WebEM

The simplest way to collect PM counter data is to save an SBTS snapshot via WebEM (see:
Saving a BTS snapshot file using Web Element Manager). However, for PM counter data only,
it is faster and more efficient to save it by using the Performance ► Performance
Management ► BTS Counters option in the top menu, or the BC keyboard shortcut.

Note:
PM counter data covers a 24-hour history only. If a longer period is required, use
NetAct instead.
For more information on how to collect PM counters using NetAct, see the NetAct Customer
Documentation.
2.1.2.3 KPI Dashboard

KPI Dashboard view allows to see respective key performance indicators (KPIs) for all
objects.
Access to KPI Dashboard view: Performance ► KPI Dashboard
KPIs are displayed for objects selected in Navigation Panel ► Objects. The dashboard
can be grouped by objects or KPI name. KPIs can be filtered, by using the KPI filter, ordered
either by name or by related object, and searched by name. The list of all KPIs can be
collapsed and expanded. The Details panel displays details for all the selected KPIs.
Figure 14: KPI Dashboard

2.1.3 Configuration management in WebEM
2.1.3.1 Compare Objects

The compare object functionality allows comparison of different configurations.
The object comparison can be used to correlate between a planned configuration and an
existing one. This is useful for solving configuration-based issues.
To access this functionality, go to WebEM Configuration ► Configuration

Management ► Compare Objects.
To use the Compare Objects function, go to Configuration

Management ► Navigation Panel ► Current BTS configuration and select a
managed object. This object can be compared to another object that is selected from the
second drop-down list, and is from the same managed object path.
Note:
This function can only be used to compare similar objects from the same managed
object path, for example WNCELG-1 with WNCELG-2.
Figure 15: Compare Object view in WebEM
For more information on the compare objects functionality, see Single RAN Operating
Documentation / WebEM User Guide / Configuration tab / Configuration Management tab /
Compare Objects.

2.1.4 Diagnostic tools in WebEM

WebEM functionalities that allow you to troubleshoot the SBTS.
2.1.4.1 Synchronization
The Synchronization view has three tabs: Status, showing the actual (real-time)
details on synchronization, Tuning, allowing to configure synchronization tuning, Test
Clocks, allowing to synchronize the external RF measuring equipment.
Status
To access the Synchronization view, go to WebEM Diagnostic ► Synchronization.
Status displays the actual (real-time) details on the synchronization source and time
servers. It also shows the synchronization source detailed status, such as synchronization
hub, synchronous Ethernet, and Timing over Packet (ToP) status as well as GNSS satellite
information.
The Details panel displays the Synchronization hub and Timing over Packet
details.
Tuning
Fast tuning can be used if the BTS clock must be adjusted or the user wants to synchronize
the BTS clock quickly. To start the fast tuning, select the Fast option and click the Tune
button.
Note that if the system module operates as a synchronization slave, it is not possible to
tune the BTS clock.
Tuning the BTS clock manually is used for the following purposes:
To read the current digital-to-analog converter (DAC) word from the BTS or to change
the current DAC word on the BTS.
To browse the history information about the difference between BTS clock and external
reference clock frequency to find out the frequency stability or accuracy.
If the BTS determines that the reference signal quality during fast tuning is insufficient, fast
tuning is interrupted and the BTS continues with normal tuning.
The Get History button allows the user to view the history information about the

difference between the BTS clock and the external reference clock frequency to find out the
frequency stability or accuracy. It is also possible to save and print the history information,
using the respective buttons.
Note that there can be entries with the date 01.01.2004 in the frequency history file. This
date is shown if Network Time Protocol (NTP) time has not been available.
Test Clocks
Test Clocks allow to test the functionality of the test clock output signal in the SYNC OUT
connector of the BTS. It is also possible to enable or disable the frame clock output signal
and to select which frame clock is used (100 Hz, 50 Hz, 25 Hz, 12.5 Hz or SFN0). The clock
signal is used for synchronizing external RF measuring equipment and must be disabled
during a normal BTS operation.
The current status of the 10 MHz test clock is visible at the top of the view. Use the Enable
checkbox to enable or disable the test and select which frame clock is used with the Output
list. The Send button saves the settings.
For synchronization troubleshooting, see the Synchronization problems section.
For more WebEM troubleshooting use cases, see the Troubleshooting with WebEM section.
2.1.4.2 IP Connectivity Test

IP Connectivity Test allows to test ping connections to certain configured remote
host addresses.
To access the IP Connectivity Test view, go to WebEM Diagnostic ► IP

Connectivity Test.
To do an IP connectivity test, fill in the following settings:

Scope - choose one option from a drop-down list. The possible options are:
User-defined address (default)
All configured hosts
All configured IPv4 hosts
All configured IPv6 hosts
BFD <Y> x.x.x.x
CMP/CA server x.x.x.x
<IPRT-Y | IPRTV6-Y> routing default gateway x.x.x.x
NTP server x.x.x.x

Primary BSC x.x.x.x

Primary DNS server x.x.x.x
Primary LDAP x.x.x.x
Primary MME x.x.x.x
Primary OAM x.x.x.x
Real-time PM collection entity x.x.x.x
Remote syslog server x.x.x.x
RNC
RTT Measurement session <Y> x.x.x.x
Secondary BSC x.x.x.x
Secondary DNS server x.x.x.x
Secondary MME x.x.x.x
SGW x.x.x.x
ToPF ToP master x.x.x.x
ToPP ToP master x.x.x.x
RNC x.x.x.x
Address - only used and mandatory in case of a user-defined address scope.
Source address - choose an address from a drop-down list.
Payload Size - bytes [16...61411]
DSCP - [0...63]
Press Start to display the test results in the table below. To stop testing, press Stop.
Note that a test failure does not always mean there is no connection. Various scenarios
must be taken into consideration. Therefore, make sure to select the correct source
address.
2.1.4.3 RF Diagnostic Test

RF Diagnostic Test allows to test and verify the RF characteristics of RF modules. RF
Diagnostic Test is supported for rel. 2 or newer hardware.
To access the RF Diagnostic Test view, go to WebEM Diagnostic ► RF

Diagnostic Test.
The RF Diagnostic Test view contains two tabs: Test Setup and Test Result. RF
diagnostic is valid on LTE only.

Test Setup
To execute the test, select which cells are to be tested (all or an individual cell) and define
thresholds for the BTS:
RF Power Error High Threshold (dB)
RF Power Error Low Threshold (dB)
RTWP High Threshold (dBm)
RTWP Low Threshold (dBm)
TX Antenna VSWR High Threshold
Click Activate remote RF diagnostic test check-box.
To define thresholds for bands, click the icon and fill in the required parameters (Band
class, Total RF power high threshold, Total RF power low threshold). To
delete thresholds for bands, use the icon.
Click the Save threshold button to send the configured thresholds to the BTS.
Click the Start Test button to initiate the test. The progress and results are displayed in
the Test Result tab when available.
Test Result
The Test Result tab contains a table with general test information and a table with a
detailed result for each antenna. Test results are displayed for selected cells. The list of cells
is in the left table and can be filtered by local cell ID. Test results can be saved for one cell
(Save Cell Results) or for all cells (Save All Results). Results are saved in a text
file.
2.1.4.4 EAC Functionality Test

EAC Functionality Test allows to test the functionality of the External Faults and
Controls (EAC) for the commissioned BTS. If the software update is in progress, the test
cannot be run. The information for the test is checked from the site configuration file.
To access the Functionality Test view, go to WebEM Diagnostic ► EAC

Functionality Test.
This view contains two tabs: Faults and Controls.

Note that only commissioned EAC lines can be tested in EAC Functionality Test.
Faults are used to test external fault lines, if EAC is configured. Select the lines to be
tested in the Selected to test column and the results are updated when available. The
Save Results button allows to save the results to a TXT file.
Controls are used to test the external control lines. Change the state of the output line
and click Send to send the changes to the BTS. After the BTS has changed the state of the
external control line, check the state of the supervised external device. The operator can
update test results in the Test result column. The Save Results button allows to save
the results to a TXT file.
2.1.4.5 Ethernet Port Mirroring

Ethernet port mirroring is used to identify, monitor and troubleshoot network
abnormalities. The traffic that goes to and from a specific port number is automatically
copied and transmitted to a monitoring port.
To access the Ethernet Port Mirroring, go to WebEM Diagnostic ► Ethernet

Port Mirroring.
The port mirroring function enables the user to perform on-site mirroring (tracing, sniffing)
of Ethernet traffic. The function applies to the Ethernet traffic on a physical Ethernet
interface of the BTS as well as on an internal Ethernet interface, if a BTS-integrated, QoS-
aware switch is present.
To enable Ethernet port mirroring, mark the checkbox Enable Ethernet port
mirroring. Click Add Mirror group and set the direction, source and destination for
the mirroring. The Send button applies the changes to the configuration plan and triggers
its validation and activation. No BTS reset is required.

Figure 16: Ethernet Port Mirroring
2.1.4.6 IP Traffic Capturing

IP Traffic Capturing allows to capture the traffic to and from a local port into a file
or stream towards another port.
To access the IP Traffic Capturing view, go to WebEM Diagnostic ► IP Traffic

Capturing.
To capture traffic at transport network interface level, select one of the capture point
options. For more information, see Figure: Capture point options for IP traffic capturing .
Figure 17: Capture point options for IP traffic capturing
Selection of a capturing point depends on the intended packets to be monitored. When

IPSec is configured, if we capture trace at point A, we can capture packets before IPSec
encryption. If we capture trace at point B, then we can to capture the packets after IPSec
encryption.

The following options are also available:

Include U-plane - includes user plane traffic
Output - allows to define the data output: save the data to a file (and optionally protect
it with a password) or stream the data to another port.
Clicking on Start initiates the capture. The status and duration are displayed. The Stop
button stops the process. The BTS automatically stops the traffic capturing after 24 hours if
the user does not stop it. The captured data is saved on the BTS, and is generated as a file
only if the user clicks the Generate and Download File button.
2.1.4.7 SFP monitoring

The SFP monitoring view allows you to view details on optical fiber connectivity
between modules (master system module, radio units, and extension system module) or
backhaul links.
A Small Form-factor Pluggable (SFP) is a compact connector that consists of a housing fitted
on a host board and a pluggable transceiver. It is developed for high-speed data transfer,
including both copper-based and fiber-optic systems.
To access the SFP monitoring view, go to WebEM Diagnostic ► SFP Monitoring.
Note:
The SFP monitoring is supported on FSMF and Nokia AirScale system modules.
The SFP monitoring window shows current (real-time) information about the connection
between the RF modules and system modules of the current configuration. Select any of the
listed SFPs to display further information in the Details panel.
Note:
The optical interface diagnostic data is received from the SFP connectors.
The SFP monitoring can be used in troubleshooting or for new site installations where there
are SFP interface problems in the system module backhaul. It also provides an indication of
the fiber link quality of the system module backhaul link, and facilitates troubleshooting of
the network elements.

Note:
In case of FOSP SFP type (supplier WTD or Accelink, supplier part code RTXM228-402-
C39) the nominal transmission rate is reported to be 10G. However, the actual
nominal transmission rate is 9.8G.
For more information about SFP monitoring, see Single RAN Operating Documentation /
WebEM User Guide / Diagnostic Tab / SFP Monitoring.
2.1.4.8 Antenna Line Online Monitoring

This WebEM tab allows to monitor online antenna line characteristics.
To access the Antenna Line Online Monitoring view, go to WebEM

Diagnostic ► Antenna Line Online Monitoring.
The Antenna Line Online Monitoring view allows the operator to monitor online
antenna line characteristics such as:
The values of Voltage Standing Wave Ratio (VSWR) for the Tx-capable antenna lines
The values of Received Total Wideband Power (RWTP) for LTE and WCDMA
The values of Received Signal Strength Indicator (RSSI) for GSM
To start monitoring, select the antenna line and click the Start button. The Stop button
stops the monitoring. The obtained results can be exported to a CSV file.
For general WebEM use cases, see the Single RAN Operating Documentation / WebEM User
Guide document.
2.1.4.9 RF Monitoring
This view allows to monitor the spectrum in order to detect radio frequency interference
(RFI) and radio disturbance in the peripheral radio frequency (RF) path.
To access the RF Monitoring view, go to WebEM Diagnostic ► RF Monitoring.
Tests available in the RF Monitoring view:

PIM Desensitization

Distance to PIM (DTP)

RF Scan
2.1.4.9.1 PIM Desensitization

This tab allows to perform PIM desensitization tests.
Access to PIM Desensitization: Diagnostic ► RF Monitoring ► PIM

Desensitization
Passive Intermodulation Distortion (PIM) Desensitization is a test that verifies if PIM has any
impact on the uplink channel for a particular antenna. To perform the test it is necessary to
specify which antenna is to be tested. The following information on test configuration is
displayed:
Local cell
Frequency band
Bandwidth
Frequency/EARFCN
Power
This test can be performed for the LTE and WCDMA technologies. The BTS configuration
used for the test can be exported into ZIP file. Use the Start button to launch the test. The
test results are saved to a local drive. For detailed instructions on viewing the test results,
see WebEM User Guide / Instructions / Viewing RF monitoring results.
2.1.4.9.2 Distance to PIM (DTP)

This tab allows to perform Distance to PIM (DTP) tests.
Access to Distance to PIM (DTP): Diagnostic ► RF Monitoring ► Distance

to PIM (DTP)
Distance to Passive Intermodulation Distortion (DTP) is a test that helps localize the actual
place where Passive Intermodulation Distortion (PIM) is happening on an antenna line. To
perform the test it is necessary to specify which antenna is to be tested. It is possible to
configure additional test settings:
Velocity factor
Cable length
The following information on test configuration is displayed:

Local cell
Power
This test can be performed for LTE and WCDMA technologies. Use the Start button to
launch the test. The test results are saved to a local drive. For detailed instructions on
viewing the test results, see WebEM User Guide / Instructions / Viewing RF monitoring results.
2.1.4.9.3 RF Scan
This tab allows to perform RF scanning.
Access to RF Scan: Diagnostic ► RF Monitoring ► RF Scan
In this tab it is possible to trigger a scan to detect radio frequency (RF) interference. There
are several attributes that must be selected or configured to perform a scan:
Radio
Capture type
Antenna
Local cell
This test can be performed for the LTE and WCDMA technologies. Use the Start button to
launch the test. The test results are saved to a local drive. For detailed instructions on
viewing the test results, see WebEM User Guide / Instructions / Viewing RF monitoring results.
2.1.4.10 Test Models

The Test Models views allow running downlink and uplink test models for each cell.
To access the Test Models view, go to WebEM Diagnostic ► Test Models
The Test Models views allow running downlink and uplink test models for LTE and WCDMA
cells.
It also allows for running transmission and loop tests for GSM.
The tests are available only if the BTS is in Test Dedicated State.
2.1.4.10.1 LTE Downlink

The LTE Downlink tab from WebEM test models allows to perform LTE downlink tests.

Access to LTE Downlink: Diagnostic ► Test Models ► LTE Downlink
To perform a test, select the cells for each test, as well as the test model and the physical
cell ID. Clicking Start begins the test, and Stop stops the test. The results are displayed
for each cell and can be saved to a file. The Details panel presents the status of each
started test.
Figure 18: LTE Downlink tab view
Note:
2.1.4.10.2 LTE Uplink

The LTE Uplink tab from WebEM test models allows to perform LTE uplink tests.
Access to LTE Uplink: Diagnostic ► Test Models ► LTE Uplink
To perform a test, select cells for each test, as well as the test model, reporting time interval
and resource block offset. Clicking Start begins the test, and Stop stops the test. The
results are displayed for each cell and can be saved to a file. The Details panel presents
the status of each started test.
Figure 19: LTE Uplink tab view

Note:
2.1.4.10.3 WCDMA Downlink

The WCDMA Downlink tab from WebEM test models allows to perform WCDMA downlink
tests.
Access to WCDMA Downlink: Diagnostic ► Test Models ► WCDMA Downlink
To perform a test, select the cells for each test, as well as the test model, write the primary
scrambling code and select the number of DPCH. Clicking Start begins the test, and Stop
stops the test. The results are displayed for each cell and can be saved to a file. The
Details panel presents the status of each started test.
Figure 20: WCDMA Downlink tab view
Note:
2.1.4.10.4 WCDMA Uplink

The WCDMA Uplink tab from WebEM test models allows to perform WCDMA uplink tests.
Access to WCDMA Uplink: Diagnostic ► Test Models ► WCDMA Uplink
To perform a test, select the cells for each test and write the primary scrambling code.
Clicking Start begins the test, and Stop stops the test. The results are displayed for each
cell and can be saved to a file. The Details panel presents the status of each started test.

Figure 21: WCDMA Uplink tab view
Note:
2.1.4.10.5 GSM BCCH Transmission

The GSM BCCH Transmission tab from the WebEM test models allows to perform GSM
BCCH transmission tests.
Access to GSM BCCH Transmission view: Diagnostics ► Test Models ► GSM

BCCH Transmission
To perform a test, select the TRX number for each test, as well as the absolute radio
frequency number (ARFN). Choose the rest of test parameters. Clicking Start begins the
test, and Stop stops the test. The results are displayed for each cell and can be saved to a
file. The Details panel presents the status of each started test.
Note:
2.1.4.10.6 GSM Loop Test

The GSM Loop Test tab from WebEM test models allows to perform GSM loop tests.
Access to GSM Loop Test view: Diagnostics ► Test Models ► GSM Loop Test
To perform a test, set up the test parameters and click Start. Clicking Stop stops the test.
The results are displayed for each cell and can be saved to a file. The Details panel
presents the status of each started test.

Note:
2.1.4.11 Terminal
The Terminal view allows you to execute text commands in WebEM.
To access the Terminal view, go to WebEM Diagnostic ► Terminal.
Terminal is the text console allowing you to execute text commands. The Terminal
output is cleared on each reload or view change. It is possible to save the output to a TXT
file by using the output-to-file command. Using the help command lists all the
available commands, while using the help <command> displays help information on the
given command. Press the Tab key while typing a command to autocomplete it.
Note:
Terminal view in WebEM is not an equivalent of WebEM Remote Tool.
2.1.4.12 Snapshot
The Snapshot view is used to save the BTS snapshot.
To access the Snapshot view, go to WebEM Diagnostic ► Snapshot.
Shortcut keys (for Snapshot): S+N
Snapshot is a functionality that can be used for troubleshooting and training purposes, in
order to troubleshoot a problem with the site or to simulate a BTS site for training
purposes. The snapshot file can be saved in connected mode and it contains the current
status of elements, such as: used HW configuration, logs, alarms, HW and SW version
information. Unlike the IMS2 file, the snapshot contains only the latest information, without
the history.
The Snapshot view allows to download technical log files from the BTS. This tab contains a
Collect snapshot button and snapshot collection settings. The snapshot collection
settings allow to:
Select LTE cells.
Select snapshot coverage - all log files or the most important log files.

Select target location - local or remote.

Input filter keyword (optional).
Include a reason for a report (optional).
Statuses of downloaded files are visible in the table at the bottom of the site. In order to
cancel a technical report procedure, press the Cancel button. When the file collection is
completed, the snapshot is saved in the selected location.
For more information on taking snapshots, see Taking snapshots with WebEM.
2.1.4.13 Reset to test dedicated state

Reset to Test Dedicated State sets the site to a test dedicated state in order to
perform tests.
To access the Reset to Test Dedicated State view, go to WebEM

Diagnostic ► Reset to Test Dedicated State.
To perform tests such as LTE UL or DL, WCDMA UL or DL, the Execute procedure must be
run first. It causes the site to reset to the Test Dedicated State. While in the Test
Dedicated State, the site is treated as Blocked.
While the site is in Test Dedicated State, the icon is displayed in the BTS site
information area.
Figure 22: BTS site information
To leave the Test Dedicated State reset the BTS.
2.1.4.14 TWAMP RTT Measurements

RTT Measurements allow to measure and supervise the IP network conditions through
the mobile backhaul between the BTS and some other point.
To access the TWAMP RTT Measurements view, go to WebEM

Diagnostic ► TWAMP ► TWAMP RTT Measurements.
The RTT Measurements tab allows to measure and supervise the IP network conditions
through the mobile backhaul between the BTS and some other point. The Round Trip Time
(RTT) measurement functionality provides Two-Way Active Measurement Protocol (TWAMP)-

light measurements, as specified in the IETF RFC 5357. The purpose of the measurements is
to have an estimation of the quality and performance of the IP-based mobile backhaul for
each QoS class independently, indicated by DiffServ Code points.
The results of the measurements are available in this tab if the hardware supports the RTT
measurements and if it is configured using Commissioning Wizard or Parameter
Editor.
2.1.4.15 Ethernet link OAM

This view gives user access to link monitoring, Remote Failure Indication (RFI) and
loopback functionalities.
To access the Ethernet Link OAM view, go to WebEM Diagnostic ► Ethernet Link
OAM.
Link Monitoring
WebEM shows information on:
Latest Events
Event type
Event subtype
Event time
Event code
Source address
Destination address
Event Statistics
Event Type
Last remote interval events
Total remote events
Total remote TLVs
OAM/non-OAM Statistics
Number of recived OAM protocol data units (OAMPDUs)
Non-OAMPDUs frames transmitted
Non-OAMPDUs frames received
There are four types of link events defined in the standard for a degraded Ethernet
connection:
Errored Symbol Period Event
Errored Frame Event

Errored Frame Period Event

Errored Frame Second Summary Event
Loopback
This function is used for testing purposes in which the remote peer loops all received non-
OAM Protocol Data Units (non-OAMPDUs). If the loopback mode is activated and it does not
receive a disable command, the BTS returns to the normal working state after a timeout is
reached.
The loopback testing is used for:

Checking if the traffic is correctly sent and looped back to the local node.
Checking statistics (errored frames).
Identifying erroneous packets and packet drops in the link.
Note:
Loopback mode interrupts the regular traffic, therefore it is recommended to use it

for offline testing or only if all alternatives for online testing are exhausted.
Remote Failure Indication (RFI)

To check if there are any Critical Link Events detected, open the Details panel in
the Ethernet Link OAM window.

Figure 23: Checking critical link events
2.1.4.16 Ethernet Service OAM

This view provides information on Ethernet Continuity Check, Remote Defect Indication
(RDI), Loopback and Link Trace.
To access the Ethernet Service OAM view, go to WebEM Diagnostic ► Ethernet

Service OAM.
The Maintenance Association Endpoint (MEP) sends Ethernet Continuity Check

Messages (Eth-CCMs) to its MEP peers according to the configured period. The MEP expects
Eth-CCMs from every MEP peer. The Eth-CCM reports continuity loss, or unintended
connectivity between MEPs. WebEM returns data such as:
ID
Mac address (static)
Mac address (from remote MEP)
Out of sequence CCM
CCM reception
CCM period

Last RDI
Ethernet Link Trace retrieves information about the relationships between

neighboring MEPs or Maintenance Association Intermediate Points (MIPs). It allows the
detection of the number of S-OAM aware hops between two MEPs.
Ethernet Loopback verifies bidirectional connectivity between two peer MEPs, or

between an MEP and an MIP.
To start the Loopback or Link Trace, click the corresponding button in the Action
column.
2.1.4.17 PPP Interfaces

The PPP Interfaces view contains two tabs: Single Link View and Multi Link
View.
To access the PPP Interfaces view, go to WebEM Diagnostic ► PPP Interfaces.
The PPP Interfaces view allows to view PPP Single Link and Multi Link configuration and
status.
In case of PPP Multi Link view, details of all PPP links forming PPP multi link configuration can
be checked.
2.1.4.18 IP Routing
The IP Routing view allows to display information about static routes, routing policies and
so on.
2.1.4.18.1 IPv4/IPv6 Routing

These views provide information about the static routes and the forwarding table.
To access the IPv4 Routing view, go to WebEM Diagnostic ► IP Routing ► IPv4

Routing.
To access the IPv6 Routing view, go to WebEM Diagnostic ► IP Routing ► IPv6

Routing.
This tab provides information about static routes and forwarding tables for selected routing

tables. The data is presented using two tables: Static routes and Forwarding table.
Information provided by Static routes table:

Destination
Prefix length
Preferred source
Gateway
Preference
IP MTU
Trigger ID
Trigger state
Information provided by Forwarding table:

Type
Destination
Prefix length
Gateway
IP interface reference
Preferred source
Preference
IP MTU
Trigger ID
The Change Static Routes button redirects the user to the object class in Parameter
Editor responsible for static route configuration.
2.1.4.18.2 Routing Policies

This view provides information about the order number, reference to routing table, the
source IP address, and prefix length.
To access the Routing Policies view, go to WebEM Diagnostic ► IP

Routing ► Routing Policies.
This view provides the following information about routing policies:

Order number
Reference to routing table
Source IP address
Source IP prefix length

The Change Routing Policies button redirects the user to the object class in
Parameter Editor responsible for routing policy configuration.
Note:
The policy order number of any new or old policy cannot be changed to an existing
policy order number.
2.1.4.19 IP Security Associations

This view provides information on Internet Key Exchange (IKE), the overall status, and the
association information.
To access the IP Security Associations view, go to WebEM Diagnostic ► IP

Security Associations.
The IP Security Associations view provides information on:

Internet Key Exchange (IKE) associations
Policy order number
IKE request ID
Local tunnel endpoint
Remote tunnel endpoint
Status
Mode
Overall status of IKE associations
Association information
Policy order number
Association request ID
Local IP address
Remote IP address
Peer state
Inbound security policy index (SPI)
Outbound SPI
The displayed data can be filtered by Policy order number.

2.1.4.20 PMTU Discovery

In this view, the on-demand path maximum transmission unit (PMTU) discovery can be
performed.
To access the PMTU Discovery view, go to WebEM Diagnostic ► PMTU Discovery.
To start the test, insert the Source IP address, Destination IP address and DSCP.
In return, WebEM displays Path Maximum Transmission Unit and Local Maximum
Transmission Unit.
2.1.4.21 GSM Traffic Trace

This view provides a graphical representation of TRX timeslots.
To access the GSM Traffic Trace view, go to WebEM Diagnostic ► GSM Traffic
Trace.
The GSM Traffic Trace displays a graphical representation of the active or idle timeslots
for a GSM cell or sector at a given time. To display data for all TRXs under a specific cell or a
single TRX, choose the correct object in the Navigation Panel (Objects tab). GSM
Traffic Trace data for each displayed TRX is presented in the form of a graphical
representation of eight numbered timeslots. Each timeslot is colored according to the type
of traffic.
Data collection starts after selecting the cell or the sector or the TRX. To refresh the results
click the Refresh button. It is possible to enable Auto refresh by marking the checkbox.
Select the Auto refresh time interval from a drop-down list.
Figure 24: GSM Traffic Trace view

2.1.4.22 PDH Loopback

This tab allows to perform Plesiochronous digital hierarchy (PDH) loopback.
To access the PDH Loopback view, go to WebEM Diagnostic ► PDH Loopback.
To enable the PDH Loopback function, configure the interface and loop configuration and
then click Send. Additionally, it is possible to set the time out.
Figure 25: PDH Loopback
2.1.5 Reset, block and unblock in WebEM
2.1.5.1 Finding information about reset reasons in a

snapshot file
The following procedure is an overview of troubleshooting methodology using a snapshot
file.
Before you start
This procedure requires the 7zip and NotePad++ applications, which can be downloaded

from the Internet, free of charge.
Procedure
1 Save a BTS snapshot on the computer.
For more information, see BTS snapshot.
2 Unzip the snapshot files using 7zip.
3 Extract the BTSxxxx_1011_blackbox.xz file using 7zip.
4 Open the BTSxxxx_1011_blackbox file.
5 Find the following information:
Reset counter
Reset reason and timestamp
Active software version
Passive software version
Figure 26: Information about reset reasons in the BTSxxxx_1011_blackbox file
2.1.5.2 Blocking and unblocking BTS units

Blocking and unblocking procedures can be done using BTS Hardware menu in BTS Site
Manager or the Site View or Detailed Site View functions of Web Element

Manager.
Purpose
Many troubleshooting procedures require blocking or unblocking the whole BTS site or
specific BTS units.
This procedure is used to block or unblock:

BTS site
Radio unit
Local cells
These procedures can be performed in BTS Hardware menu in BTS Site Manager or Site
View or in Detailed Site View in Web Element Manager.
To block or unblock a BTS unit, do the following:
Procedure
1 In WebEM go to BTS Status ► Site Runtime View ► Site view or Detailed
Site View.
2 Select the unit you want to block or unblock and go to the Details pane.
3 Click the Block button.
Step example
Figure 27: Details pane for the unit selected for blocking or unblocking
Step result

Figure 28: Confirmation window for blocking or unblocking a unit
Result
The selected element is blocked or unblocked.
Note:
Blocking a unit may interrupt traffic.

Unblocking a BTS site and system module causes a reset.
2.1.5.3 Resetting BTS units

Resetting procedures can be done using the BTS Hardware menu in BTS Site Manager or
Site View or Detailed Site View functions of Web Element Manager.
Purpose
Many troubleshooting procedures require resetting the whole BTS site or specific BTS units.
This procedure is used to reset:

BTS site
Radio unit
Local cells
This procedure can be performed in BTS Hardware menu in BTS Site Manager or Site View
or in Detailed Site View in Web Element Manager.
To reset a BTS unit, do the following:
Procedure
1 In WebEM go to BTS Status ► Site Runtime View ► Site view or Detailed

Site View.
2 Select the unit to reset and go to the Details pane.
3 Click the Reset button.
Step example
Figure 29: Details pane for the unit selected for reset
Step result
Figure 30: Confirmation window for the reset action of a unit
Result
The selected element is reset.
2.1.5.4 RAT reset

Resetting procedures can be done using Web Element Manager.
Purpose
Troubleshooting procedures sometimes require only one RAT reset (LTE, WCDMA or GSM).
This can be done in Web Element Manager.

Note:
The RAT reset works only on commissioned technologies.
Procedure
1 Go to Web Element Manager.
2 In the Site Status bar, in the upper-right corner, find the icons for each
commissioned RAT.
3 Click on to reset the respective RAT.
Figure 31: Site Status bar
4 Confirm the RAT reset.
Figure 32: Reset confirmation
2.1.6 Transfer operations
2.1.6.1 WebEM file transfer operations failed

Recovery actions for the transfer failed error message received during WebEM
operations.
If the browser certificate for WebEM is not valid or is missing or there is a timeout on the

browser, the file transfer operation can generate a transfer failed error message in
WebEM.
The file transfer operations that can be performed in WebEM are:

Log in
Collect diagnostic snapshots
Save Info Model snapshot (IMS2) file
Save the SW update report
Save the SW history report
Download the SW update
Activate the network configuration file (NCF)
Install the certificate and get the certificate file
Get the certificate signing request (CSR) file for the Citizens Broadband Radio Service
(CBRS) certificate
Turn on history
Run RF sniffing
Get historical counters from Performance Management
Manage the Remote Antenna Extension (RAE)
Disconnect or reconnect
Get Plug and Play report
Download the site configuration file (SCF)
If you receive the transfer failed error message during one of the file transfer
operations, you can either:
install the browser certificates.
refresh the page or open a new browser page to launch WebEM and try the transfer
operation again.
For more information about launching WebEM and browser certificates, see the Launching
the WebEM tool when connected to the BTS (online)Operating Documentation / WebEM User
Guide / Instructions / Launching the WebEM tool when connected to the BTS (online)
document.
2.2 Troubleshooting with NetAct

NetAct enables you to monitor faults, alarms, parameters, counters and KPIs.
For general NetAct use cases, see the NetAct Operating Documentation / Troubleshooting
/ Troubleshooting NetAct document.

For cell trace related troubleshooting, see the Tracing with NetAct section.
2.2.1 Performance monitoring in NetAct

The NetAct Performance Manager+ application allows the user to view and process
counter and KPI data.
For information on available tools and functions, see the NetAct operating documentation:
Performance Management / Performance Management Overview

Performance Management / Performance Manager+ Overview
Thresholder and Profiler Overview
2.2.2 Alarm monitoring in NetAct

The NetAct Monitor application allows the user to view and process alarm information.
For information on available tools and functions, see the NetAct operating documentation:
Fault Management Overview and Operations

Fault Management Helps
2.2.3 Reporting tool and default reports

The NetAct Reporting tool provides KPI information useful in troubleshooting.
The NetAct Reporting tool supports administration and maintenance of the network. For
troubleshooting purposes, it provides valuable information about various KPI statistics.
Based on the collected KPI statistics, it is possible to check the exact time when the problem
began. It is also possible to check the decrease and increase levels in the KPIs.
The Reporting tool allows the collection of default and custom reports.
2.2.4 Tracing and MDT

The Minimization of Drive Test (MDT) is a manual method that allows the operator to gather
information about the network in a particular area. The MDT enables operators to use the
UE to collect radio measurements and associated location information that are reported

periodically according to the set report interval and report amount. The measurement is
reported with position information coming from the GPS.
There are predefined in build profiles in NetAct TraceViewer to cover the following use
cases:
detection of coverage problems in the neighbor cell
detection of coverage problems in the traced cell
monitoring the coverage quality of the traced cell
For more information on LTE MDT see the Single RAN Operating Documentation / Tracing in
LTE and SRAN document.
2.3 Troubleshooting with Traffica

Traffica supports real-time monitoring for LTE and WCDMA traffic. The real-time
information is available on both the transport and radio layers.
For general Traffica use cases, see the Single RAN Operating Documentation /
WCDMA and GSM / IPA-RNC and mcRNC / Troubleshooting RNC /
Troubleshooting IPA-RNC document.
2.3.1 Network monitoring using Traffica

An overview of the Traffica application (a real-time traffic monitoring tool) and a general
use case for LTE and WCDMA troubleshooting using Traffica.
Traffica is a real-time traffic (RTT) monitoring tool designed to monitor and analyze network
traffic. Traffica allows the operator to see how the network functions, from the network
element level down to individual subscriber information. Traffica also supports RNC Layer 3
Data Collector (L3DC) and BSC real-time traffic, adding options to monitor the WCDMA and
GSM networks.
Traffica consists of:

a Traffic News client for offline analysis of collected data
a Traffic Views client for real-time monitoring (various graph sets to display collected
data in different time periods)
A Traffica user can configure threshold-based real-time alarms that can be forwarded to the
NetAct Monitor application.

For more details on Traffica applications and functions, see the following documents:
Serve atOnce Traffica Operating Documentation/ Traffica Principles
Serve atOnce Traffica Operating Documentation / Traffica Insights Reference Guide
The Traffica application is introduced with the following features:
The SR000919: SBTS Real Time Performance Monitoring feature provides real-time
performance monitoring for the SBTS. The SBTS is connected directly to Traffica using a
real-time data interface. It contains the results of ongoing performance measurements (PM-
counter values) in real-time. These results are sent every minute.
The SR000478: LTE L3 Data Collector Northbound Interface feature offers a real-time
network monitoring solution based on the SBTS cell trace interface (introduced with the
SR000478: LTE L3 Data Collector Northbound Interface feature). The collected trace data is
sent to the Layer 3 Data Collector (L3DC) that transfers the data to Traffica for
visualization.
Figure 33: An overview of Traffica integration in the network

2.3.1.1 Traffica general use case for LTE and WCDMA

troubleshooting
This procedure allows you to analyze and troubleshoot LTE and WCDMA cases using
linked graphs, Traffic Views and News.
Purpose
For detailed procedures for the default WCDMA use cases listed below, see the Default 3G
use cases section in the Serve atOnce Traffica Operating Documentation / Traffica Insights
Reference Guide document.
Troubleshooting the E-UTRAN radio access bearer (RAB) or Radio Resource Control (RRC)
failures
RNC performance monitoring
3G cell performance monitoring
Purpose
For detailed procedures for the default LTE use cases listed below, see the Default 4G use
cases section in the Serve atOnce Traffica Operating Documentation / Traffica Insights
Reference Guide document.
Cell overshooting analysis

Troubleshooting the E-UTRAN radio access bearer (RAB) or Radio Resource Control (RRC)
failures
ERAB session drop troubleshooting
Handover failure troubleshooting
BTS or cell performance monitoring
Special event or BTS group monitoring
Crossed feeder detection
Tracking area real-time monitoring
Before you start
To visualize the cell trace data and real-time counter data in Traffica, the following
requirements must be fulfilled:
The L3 Data Collector (L3DC) is installed and configured.

Note:
In Traffica use case the L3DC deployment is as follows:

Windows users
for SBTS - MegaPlexer
for RNC/BSC - Megamon
Linux users
for SBTS - Tracer LTE with Traffica plugin
for RNC/BSC - Tracer 3G
The Traffica - L3DC interface is configured.
Procedure
1 Follow network and service performance indicators using Traffic Views.
Spot degradations (for example, RRC/RAB setup failure increase).

Detect sites with no traffic.
Define threshold alarms to automate fault detection.
2 Analyze problem severity and location using Traffic Views.
Use failure graphs to check whether the problem is limited to certain network elements
(NEs) or cells.
3 Troubleshoot the problem using linked graphs and Traffic News.
Use drill-down graphs to isolate the problem further down in topology.

Study call event reasons and call failure reasons to determine the actual fault.
2.3.2 Activating the real-time KPI-monitoring with Traffica

feature
Before you start
Activation procedure does not cause downtime, and the feature can be activated at any
time of the day.

Procedure
1 Start configuring the parameters.
Select from the available options

Go to the Radio Network Configuration page if using BTS Site Manager.
Go to the Parameter Editor/ Navigation Panel if using Web Element
Manager.
2 Activate the real-time Performance Monitoring (RT PM).
1. Expand the MRBTS object.

2. Select the LNBTS object.
3. Set the Activate RT performance monitoring (actRtPerfMonitoring)
parameter value to All_PM_Counters.
3 Create the RT PM Collection Entity (rTPmCollEntity) parameter structure.
1. Go to the MRBTS ► MNL ► MNLENT ► PMCADM object.

2. Create the RT PM Collection Entity (rTPmCollEntity) parameter structure.
Step result
The RT PM Collection Entity parameter structure is created.
4 Set the IP address of the RT PM collection entity.
On the RT PM Collection Entity structure, set the RT PM Collection Entity

IP Address (rTPmCollEntityIpAddr) parameter value.
5 Set the port number of the RT PM collection entity.
On the RT PM Collection Entity structure, set the RT PM Collection Entity

Port Number (rTPmCollEntityPortNum) parameter value.
6 Set the interval of the RT PM collection entity.
On the RT PM Collection Entity structure, set the RT PM Collection

Interval (rTPmCollInterval) parameter value to 60 s.
7 Send the parameters to the SBTS.

2.3.3 Deactivating the real-time KPI-monitoring with

Traffica
Before you start
Deactivation procedure does not cause downtime, and the feature can be deactivated at
any time of the day.
Procedure
1 Start configuring the parameters.

Go to the Radio Network Configuration page if using BTS Site Manager.
Go to the Parameter Editor/ Navigation Panel if using Web Element
Manager.
2 Remove the RT PM Collection Entity (rTPmCollEntity) parameter structure.
1. Go to the MRBTS ► MNL ► MNLENT ► PMCADM object.

2. Select the RT PM Collection Entity (rTPmCollEntity) parameter structure
and delete it.
Step result
The RT PM Collection Entity is removed.
3 Deactivate the RT performance monitoring.
1. Expand the MRBTS object.

2. Select the LNBTS object.
3. Set the Activate RT Performance Monitoring (actRtPerfMonitoring)
parameter value to false.
4 Send the parameters to the eNB.
2.3.4 Recovery procedure for Traffica reports when they

are not received by Traffica tool

Procedure
1 Make sure that the Traffica feature is enabled in the RNC.
If the RNC support for Traffica feature is not activated properly in the RNC, the Traffica
server does not receive Traffica reports from the RNC.
Traffica feature is not optional in the RNC, thus no licence key or FIFILE is needed.
However, by default RNC Support for Traffica is disabled and must be enabled
using a PRFILE parameter.
To check that the Traffica feature is enabled, open a man-machine language (MML)
session to the RNC and check the PRFILE parameter value using the following command:
ZWOI:9,130;
If the parameter value is 00FF, it means that the Traffica feature is enabled in the RNC.
For more information on Traffica feature activation, see the WCDMA RAN Operating
Documentation / RAN1150: RNC Support for Traffica, Feature
Activation Manual feature.
2 Make sure that the connection between the Traffica server(s) and the RNC is working.
2.1 Connect to the working Operation and Maintenance Unit (OMU) service
terminal and execute the command ZOS:*,*,575,,,,,6034,,,2,0;.
2.2 Check the computer log from the working OMU service terminal with the ZGD;
command.
2.3 If no Traffica servers are connected, the computer log shows the following
information (see printout below), that is, all the IP address fields are full of
zero.
In this case, check that ping works between the Traffica server and the RNC, and
that the required User Datagram Protocol (UDP) ports are open in the firewalls
between the RNC and the Traffica.
Note:
The firewalls can be configured so that ping (Internet Control Message

Protocol) is blocked but IP packets containing UDP data to predefined
ports are still allowed to pass through.
2.4 Check also that the RNC IP address is configured correctly in the Traffica

server side.
0013 0575 0001 2007-04-11 C-08 ===START PRINT SERVER INFO==
0014 0575 0001 2007-04-11 C-08 type count addr1 addr2 s
0015 0575 0001 2007-04-11 C-08 0x3101 1@0.0.0.0:0 0.0.0.0:0 0
0016 0575 0001 2007-04-11 C-08 0x3102 1@0.0.0.0:0 0.0.0.0:0 0
0017 0575 0001 2007-04-11 C-08 0x3103 1@0.0.0.0:0 0.0.0.0:0 0
0018 0575 0001 2007-04-11 C-08 0x3104 1@0.0.0.0:0 0.0.0.0:0 0
0019 0575 0001 2007-04-11 C-08 0x3105 1@0.0.0.0:0 0.0.0.0:0 0
0020 0575 0001 2007-04-11 C-08 0x3106 1@0.0.0.0:0 0.0.0.0:0 0
0021 0575 0001 2007-04-11 C-08 0x3107 1@0.0.0.0:0 0.0.0.0:0 0
0022 0575 0001 2007-04-11 C-08 0x3108 1@0.0.0.0:0 0.0.0.0:0 0
0023 0575 0001 2007-04-11 C-08 0x310A 1@0.0.0.0:0 0.0.0.0:0 0
0024 0575 0001 2007-04-11 C-08 0x310B 1@0.0.0.0:0 0.0.0.0:0 0
If the Traffica server has successfully established connection, the printout looks
like the example below, that is, some valid IP address is shown. The exact details
of what is shown depend on the subscription parameters given by the Traffica.
0029 0575 0001 2007-04-11 C-08 ===START PRINT SERVER INFO==
0030 0575 0001 2007-04-11 C-08 type count addr1 addr2 s
0031 0575 0001 2007-04-11 C-08 0x3101 1@10.8.160.99:2121 0.0.0.0:0 1
0032 0575 0001 2007-04-11 C-08 0x3102 1@10.8.160.99:2121 0.0.0.0:0 1
0033 0575 0001 2007-04-11 C-08 0x3103 1@10.8.160.99:2121 0.0.0.0:0 1
0034 0575 0001 2007-04-11 C-08 0x3104 1@10.8.160.99:2121 0.0.0.0:0 1
0035 0575 0001 2007-04-11 C-08 0x3105 1@10.8.160.99:2121 0.0.0.0:0 1
0036 0575 0001 2007-04-11 C-08 0x3106 1@10.8.160.99:2121 0.0.0.0:0 1
0037 0575 0001 2007-04-11 C-08 0x3107 1@10.8.160.99:2121 0.0.0.0:0 1
0038 0575 0001 2007-04-11 C-08 0x3108 1@10.8.160.99:2121 0.0.0.0:0 1
0039 0575 0001 2007-04-11 C-08 0x310A 1@10.8.160.99:2121 0.0.0.0:0 1
0040 0575 0001 2007-04-11 C-08 0x310B 1@10.8.160.99:2121 0.0.0.0:0 1
3 Check the subscription status of the report types.
Check that the Traffica report types are subscribed to by Traffica, open a man-machine
language (MML) session to the RNC as follows:
3.1 Connect to the working Operation and Maintenance Unit (OMU) service
terminal and execute the ZOS:*,*,575,,,,,6034,,,2,0; command.

3.2 Check the computer log from the working OMU service terminal with the ZGD;
command.
3.3 Check from the OMU computer log that the subscription status of the report
type is in subscribed state, that is, the last column is 1 like shown in 2.d
step.
If the report type is unsubscribed, see Traffica product documentation how

reports are ordered. The following table presents the report names and the
numeric report identifiers shown in the computer log:
Table 1: Report names and numeric report identifiers in a computer log
Numeric report identier Report name
0x3101 RRC/RAB
0x3102 Soft Handover failure
0x3103 External AAL2 transport resource
0x3104 Internal AAL2 transport resource
0x3105 ATM VC Traffic
0x3106 Call resource handling
0x3107 Call resource error code
0x3108 Hearbeat reply
0x310A Packet Call statistics
0x310B Packet Call failure
2.4 Troubleshooting with DCAP

The Data Collection and Analysis Platforms (DCAP) provides an enhanced
troubleshooting, data collection and analytics capabilities.
The Data Collection and Analysis Platforms (DCAP) is a troubleshooting server that contains
a unified and secure real-time data collecting, storing and sharing solution.
With DCAP, you can collect, store and share data across all technologies. The data can be

retrieved remotely from the network elements using a secure connection interface. The
network health and status can be viewed live based on collected network monitoring data.
For Windows DCAP troubleshooting, see the following documents:

DCAP Operating Documentation / Troubleshooting Server
DCAP Operating Documentation / Central Monitoring Solution
DCAP Operating Documentation / IMSI Provider
DCAP Operating Documentation / IMSIMon3G
DCAP Operating Documentation / NetAct OandM Agent
For Linux DCAP troubleshooting, see the DCAP Operating Documentation/DCAP Linux
documents.
If DCAP is not available, contact the Nokia technical support team.
2.4.1 Troubleshooting with L3DC (Megamon)

Megamon is an entity designed to collect, store and visualize data from many BTS
simultaneously.
The Layer 3 Data Collector (L3DC), also called Megamon, is part of the Data Collection and
Analysis Platforms (DCAP) tools. Megamon includes a built-in data viewer and analyzer, the
Layer 3 Data Analyzer (L3DA), also called Emil.
L3DC can be configured into a quite diverse set of roles:

as a mediation device for Traffica
as BTS Diagnostics Toolkit supporting Troubleshooting Data collection
as a Trace Collection Entity and as an analyzer / viewer for 3GPP tracing
as a data provider for 3rd party entities for geo-location analysis
as a data provider for MDT solutions
as a data provider for Care engines, providing an own API for Care purposes
as data provider for Per Call Monitoring Data (PCMD)
The PCMD information can be used for assessing individual user performance and for
assessing the performance of individual cells on call by call basis.
In case of SRAN architecture with SBTS, another DCAP tool called MegaPlexer is mainly used
for LTE RAT specific data collection.
Megamon can be used to collect and analyze GSM and WCDMA specific data from BSC and
RNC.

For more information about the DCAP Megamon tool, see the DCAP L3 Data Collector
(Megamon) Operating Documentation and the DCAP L3 Data Collector for Traffica
Operating Documentation.
2.4.2 Troubleshooting with L3DA (Emil)

Emil enables efficient data collection from a BTS, it supervises the BTS via cell tracing and
triggers BTS snapshot creation, based on the trigger events in the cell trace content.
The Layer 3 Data Analyzer (L3DA), also called BTS diagnostics toolkit or Emil, is part of Data
Collection and Analysis Platforms (DCAP) tools.
Emil is a Windows-based tool that can be run on a standalone PC offering a graphical user
interface for parsing and examining message monitoring logs. It combines all the messages
related to one call or a signaling event from the message monitoring log to one scenario.
This tool enables the operator, technical support, and a Nokia developer to collect data from
a BTS in an efficient manner. The collected data such as trace data or BTS snapshot data is
sent from the operator to Nokia. This tool enables correcting an error at first occurrence
given that all the necessary information is available for a detailed analysis.
The L3DC, also called Megamon, is an entity designed to collect, store and visualize data
from many BTS simultaneously. It includes a built-in data viewer and analyzer, the L3DA
(Emil).
Logs that can be viewed and analyzed in L3DA can be seen via offline view or online
streaming if L3DA is connected to L3DC.
Figure 34: Simplified tool solution architecture
Some cell trace messages indicate a suspected or actual malfunction. These are used as
triggers for snapshots on that specific BTS. From a set of predefined trigger events, you can
choose which should actually trigger a snapshot. When a snapshot file is available, the
operator may deliver it to Nokia for analysis.
The Emil LTE Cell trace diagnostics toolkit has the following function:
Cell trace based snapshot collection (together with Megamo or Tracer LTE).

The Emil BTS diagnostics toolkit has the following functions:

Message monitoring
Syslog streaming
TTI trace streaming
L2 trace streaming
Trigger based snapshot collection
The Emil PCMD/GEO/Traffica mode has the following function:

It handles data to a different format and to different tools.
For L3DA (Emil) troubleshooting use cases, see the following sections:
Viewing and analyzing cell trace data using L3DA (Emil)
Call processing problems
Troubleshooting call processing problems
For more information about the Emil tool, see the Single RAN Operating Documentation /
Tracing in LTE and SRAN and the DCAP Operating Documentation / DCAP L3 Analyzer
Viewer (Emil) documents.
2.4.2.1 Triggering snapshot collection using L3DA (Emil)

Procedure
1 Open the Emil tool.
2 Go to Filter ► Set snapshot filter for TCP.
3 Configure the triggering options in the opened window.

Figure 35: Selecting triggers for eNB snapshot collection
4 Activate the filter.
2.4.3 Troubleshooting with GEO Interface

The Geo Interface tool is part of the Data Collection and Analysis Platforms (DCAP) tools.
GEO Interface is one of the functional modes of Emil. This mode is activated using a specific
Emil license that is designed for this purpose.
The Emil tool provides network usage data which can be mapped in geographical locations.
This data is useful for network optimization.
For more information about the GEO Interface, see the DCAP Operating Documentation /
DCAP GEO Interface document.
2.5 Troubleshooting with PTF

The Preventative Troubleshooting Framework (PTF) tool enables automatic and intelligent
data collection from operator network based on triggering (for example, due to an event like
an alarm being raised by the system), scheduling with periodical data collection or on-
demand manner. The solution reduces the effort required to collect the symptom data and
it provides accuracy in collecting the right data at the right time. In addition, PTF can
support symptom data collection for automated and centralized Network Element health
check services. Hence, PTF solution usage enables faster resolving of issues or even identify,

expect and prevent forthcoming issues at the earliest.
For more information about PTF troubleshooting, see the PTF Operating Documentation.
2.6 Troubleshooting with RFS

The Restore Factory Settings (RFS) tool is used for on-site system module recovery
procedure that can prevent unit repairs. The RFS tool can be used when the recovery
procedures (resets, reconfiguration, SW download) are not possible or they do not recover
to a state before the fault. The restore tool enables local identification and SW reset of the
system module.
For system module recovery procedure, see Recovering the system module with the RFS
tool.
For more information about the RFS tool, see the SRAN Maintenance Documentation / TS-
SRAN-HW-0080 (System Module recovery with Restore Factory Settings) document.

3. Collecting and analyzing troubleshooting data

To efficiently perform the troubleshooting process, a set of supporting data must be
collected.
In case of call processing and signaling problems, data can be collected using one of the
following methods:
BTS snapshot
Cell trace
Additional data can be obtained by collecting:
drive test data

end-user feedback
core network indicators
problematic site location (for example, from geographic location tools such as Traffica)
For RAT specific logs, see:

GSM related logs
WCDMA related logs
LTE related logs
It is recommended to contact Nokia Services if you cannot solve the problem, despite having
all the available data.
3.1 Remote syslog collection for continuous log storage

This functionality allows you to store 50 GB of collected syslogs from five BTSs for up to
two hours.
3.1.1 Remote syslog overview

An introduction to the remote syslog functionality, server hardware requirements, and
transport implementation

Syslog and remote syslog functionality

Syslog is a standard UNIX operating system way of sending log information to a host
computer that collects it. There is limited internal storage in the SBTS and the logs can be
frequently overwritten. In addition to storing the logs in its own buffer, the SBTS sends
syslog prints to the remote syslog server.
Syslog messages from different sources within the SBTS are brought together by the
transport software (TRSW) and continuously pushed towards the remote syslog server. This
reduces the outage time, error correction time and service effort. The SBTS acts as a
syslog client to the remote syslog server.
Remote syslog server

The remote syslog server is a server owned by the customer.
Figure: Customer log server presents the types of servers that can be used for remote
syslog storage. The application used can be either Rsyslog for servers using Linux, or Rsyslog
with Windows Agent for servers using Windows. The supported format for remote syslogs is
Request for Comments (RFC) 5424. For Windows servers, the format can be either RFC 5424
or Nokia proprietary. The Transport Layer Security (TLS) is supported on the Linux PC with a
signed certificate, or it can be additionally simulated on the Windows server.
Figure 36: Customer log server
The hardware requirements for the customer log server are:

disk capacity of 300 GB or higher
at least 50 GB for storing remote syslog files
Note:
The 50 GB of storage can collect logs from five BTSs for up to two hours.
For information on activating and deactivating the remote syslog functionality, see the

LTE1408: Remote Syslog for Continuous Log Storage feature.
The Rsyslog application is a third-party, open-source package that can be installed on the
Linux server.
Note:
For more information on configuring Rsyslog, see the Rsyslog Linux System
Administration and the Rsyslog configuration pages.
The Rsyslog Windows Agent is designed to work closely with Rsyslog for Linux, to
integrate both environments. For more information, see the Rsyslog Windows Agent page.
3.1.2 Remote syslog troubleshooting

Configuration, data storage details, and troubleshooting scenarios.
Syslog data stored on the customer log server

When the remote syslog is switched between two customer log servers, the syslog
transmission is stopped. The syslog server address must be changed in the Remote
syslog configuration (remoteSyslogConfig) parameter structure, so that the SBTS
can send the syslog to the new server. The data must be downloaded from the previous
server, because there is no automatic data synchronization between the main and
secondary servers.
The syslog client can be identified by the server through the UTC time stamp, SBTS IP
address, and ID, which are included in the syslog print.
The server orders the data in the following way:
A separate file for each SBTS log

A separate file for each minute or hour of a day
The remote syslog file is rotated and zipped when its size exceeds 100 MB. The server
monitors disk space utilization reserved for syslogs every 10 minutes and performs a clean-
up as follows:
Files older than five days are removed.
If the partition usage reaches 80%, the oldest files are removed until the partition usage
is decreased to 70%.

Restarting the remote syslog

After the syslog duration ends, the Remote syslog transmission status
(remoteSyslogTransmissionStatus) parameter is automatically set to false, but the
Activate remote syslog transmission (actRemoteSyslogTransmission)
parameter remains set to true. To avoid commissioning it twice, restart the trace by
changing the Remote syslog transmit duration
(remoteSyslogTransmitDuration) parameter.
Syslog data does not reach the syslog server

Syslog data does not reach the syslog server when the SBTS does not transmit the data, or
the transmission is interrupted regularly. In this case check the following aspects:
The Remote syslog transmission status

(remoteSyslogTransmissionStatus) parameter is set to true.
The server link state is connected.
There are no CPU load alarms on the SBTS. The 6502 High memory consumption
fault can raise the 7115 BASE STATION INFORMATION, 7652 BASE STATION
NOTIFICATION or 7655 CELL NOTIFICATION alarms.
3.2 BTS snapshot

The BTS snapshot content, related features, instructions to save a snapshot and
snapshot collection use cases
The BTS snapshot functionality allows the user to save to a file certain information on the
current status of the BTS elements. This file contains only the latest information about the
BTS and not the history or the full BTS runtime data for a certain period of time.
The BTS snapshot collection can be performed as follows:

Manually, using one of the following tools:
WebEM
WebEM Remote Tool
Automatically, via
TRBLCADM - triggered events
Care Engine
Examples of data included in the BTS snapshot:

Active alarms and alarm history

Installed SW versions
BTS configuration (commissioning parameters)
Internal log files for different BTS components (including MAC TTI traces)
PM counter data for a particular BTS (collected since the last BTS reset and kept for a
maximum of 24 hours)
For snapshots related to fault handling, see the Finding faults in a snapshot file section.
3.2.1 Collecting BTS snapshot manually

Presentation of procedures used to collect manually BTS snapshots
3.2.1.1 Saving a BTS snapshot file using Web Element

Manager
Instructions on how to save a BTS snapshot file using WebEM.
Before you start
WebEM must be properly installed and connected to the BTS.
The BTS can be accessed through a browser, such as Google Chrome as follows:
When the PC is connected locally to the LMP port of the SBTS - at the URL
https://192.168.255.129
When connected remotely - at the URL: https:< mPlaneIpAddressDN>
Procedure
1 Go to WebEM Diagnostic ► Snapshot.
For more details, see the Diagnostic tool in WebEM / Snapshot section.
2 For Coverage, select either All log files or Most important log files.
The All log files snapshot coverage includes all the files, while the Most
important log files snapshot coverage includes only the most important files in
the technical log.
3 For the target location, select either Local or Remote.

The target location can be chosen based on the associated options below:
Local drive: Create a snapshot and place it on the SBTS and download a copy to the
local drive.
Server: Create a snapshot and place it on the SBTS and save a copy to an external
server.
Note:
If the target location is set to Remote, make sure that the Diagnsostic
Snapshot destination (diagSnapDestination) parameter is
configured. If the parameter value is lss, configure the parameters under the
Local symptoms server configuration (lssConfig) structure as
well. To configure these parameters go to
Configuration ► Configuration Management ► Parameter
Editor. Use the search box to find the required parameters.
Step example
Figure 37: Selecting the target location
4 (Optional) Insert Filter keyword
The Filter keyword field allows the user to filter the snapshot files to collect only
the content of interest. Using the keyword filtering the snapshot file size and the
process duration are reduced.
Filtering is done based on the file name found in the file trbl_log_list.csv.
Fill this field with a regular expression (for example .*syslog).
Note:
The filtering process using keyword field is not applicable to radio units logs,
because these logs are always entirely collected, no matter what keyword is given.

5 Click Collect snapshot.
Note:
To stop the process during snapshot collection, click Cancel.
Result
Within the created archive package there is a TXT file listing all the files included in the
package: snapshot_file_list.txt.
3.2.2 Collecting BTS snapshot automatically

Presentation of procedures used to collect automatically BTS snapshots.
For more information about automatic snapshot collection, see the Single RAN Operating
Documentation / Automatic BTS Log Collection document.
3.2.2.1 Enabling snapshot collection based on TRBLCADM-

triggered events
Detailed information about activation of snapshot collection based on events triggered
by the TRBLCADM object.
3.2.2.1.1 Enabling snapshot collection triggered by a fault

or alarm
The procedure that activates the snapshot collection based on events triggered by a fault
or alarm.
Purpose
Events like fault detection, alarm sending or fault cancellation may require activation of
snapshot collection. The following procedure describes the activation of snapshot collection
based on events triggered by a fault / alarm, using WebEM.

Note:
All the data in the screenshots included in this procedure is for demonstration
purposes only.
Procedure
1 Go to Parameter Editor.

Go to Configuration ► Configuration Management ► Parameter
Editor.
Use the PE keyboard shortcut.
Click the Parameter Editor icon in the top menu.
2 Go to the TRBLCADM object.
Object path: MRBTS ► MNL ► MNLENT
If the TRBLCADM object does not exist, create it.
3 In the Configuration Management panel, search for the Fault snapshot

triggers (faultSnapshotTriggers) structure.
Find the Fault snapshot triggers structure with one of the following options:
Filter out the entries in the Parameter Name column using the Fault snapshot
triggers string.
Filter out the entries in the Abbreviation column using the
faultSnapshotTriggers string.
4 Click Add Element.
5 Configure the Fault snapshot triggers (faultSnapshotTriggers) structure.
5.1 Set the Trigger type (triggerType) parameter value, using the table
below.
Note:
The Trigger type parameter defines the methods to detect the event
that triggers the snapshot collection on NetAct or LSS level.

The event that triggers the snapshot Option

collection
Fault detection onFaultDetection
Alarm sending onAlarmSend
Fault cancellation onFaultCancellation
5.2 Set the Fault number (faultNumber) parameter value.
The Fault number parameter defines the Id of the fault that triggers an
event-based snapshot collection.
Note:
The user can verify the Fault number parameter value by checking the
Fault ID field of a given alarm, following the path Alarms ► Alarm
Management ► Active alarms.
6 Depending on the snapshot target location:

Local - go to the last step and click the Validate Plan and then the Activate
Plan buttons.
Remote - perform the following steps.
7 Set the Diagnsostic Snapshot destination (diagSnapDestination)

parameter value.

If the Diagnsostic Snapshot destination parameter value is set to lss,
configure the Local symptoms server configuration (lssConfig).
7.1 Click the Add Element button in the row corresponding to the Local
symptoms server configuration (lssConfig) parameter.
7.2 Configure the following parameters:
Local symptoms server IP address (lssIPaddress)

Local symptoms server IP port number (lssPortNo)

8 Click the Validate Plan and then the Activate Plan buttons.
3.2.2.1.2 Enabling snapshot collection triggered by the

occurrence of certain strings in the syslog
Detailed information about activation of snapshot collection triggered by syslog
substring occurrence.
Purpose
The customer may be asked to activate the collection of snapshots, triggered by the
occurrence of certain strings in the syslog.
Note:
All the data in the screenshots included in this procedure is for demonstration
purposes only.
Procedure

Editor.
2 Go to the TRBLCADM object.
Object path: MRBTS ► MNL ► MNLENT
If the TRBLCADM object does not exist, create it.
3 In the Configuration Management panel, search for the Syslog substring

trigger (syslogSubstringTrigger) parameter.
Find the Syslog substring trigger parameter with one of the following options:
Filter out the entries in the Parameter Name column using the Syslog
substring trigger string.
Filter out the entries in the Abbreviation column using the

syslogSubstringTrigger string.
4 Fill in the Planned value field of the Syslog substring trigger

(syslogSubstringTrigger) parameter with the customer string.
Note:
The string must be at least 10 characters long.
5 Depending on the snapshot target location:

Local - go to the last step and click the Validate Plan and then the Activate
Plan buttons.
Remote - perform the following steps.
6 Configure the Diagnostic Snapshot destination (diagSnapDestination)

parameter value.

If the Diagnostic Snapshot destination parameter value is set to lss,
configure the Local symptoms server configuration (lssConfig).
6.1 Click the Add Element button in the row corresponding to the Local
symptoms server configuration (lssConfig) parameter.
6.2 Configure the following parameters:
Local symptoms server IP address (lssIPaddress)

Local symptoms server IP port number (lssPortNo)
Result
The snapshot is generated and collected when the string filled in the TRBLCADM Syslog
substring trigger (syslogSubstringTrigger) parameter occurs in the syslog.

3.2.3 RF related logs

Summary on RF related logs
The BTS snapshot file contains all the standard RF logs by default.
For more information on RF troubleshooting, see the RF Monitoring and RF Diagnostic Test
sections.
3.3 Taking Info Model snapshots (IMS2) with WebEM

Instructions on how to take IM snapshots with WebEM.
Procedure
1 Go to WebEM and select the Timeline tab from theNavigation Panel.
2 Click the Save IMS2 button.
Figure 38: Navigation Panel Timeline
3.4 WebEM logs

Detailed information about WebEM logs: application logs, console logs (JAVA) and
network logs (HAR).
In case of problems with the connection between WebEM and the SBTS, it is necessary to
collect additional logs from the browser: application logs, console logs (JAVA) and network
logs (HAR). Console JAVA logs contain errors and other debug items that WebEM saves in the
console. HAR log files are used to check HTTP requests and responses that WebEM sends to
and receives from the SBTS.
For WebEM troubleshooting use cases, see the Troubleshooting with WebEM section.
For general WebEM use cases, see the WebEM User Guide document.

3.4.1 Collecting application logs

Instructions to save application logs.
Procedure
1 Open WebEM.
2 In the top menu, go to Options.
3 Click Download application logs.
3.4.2 Collecting browser console logs (JAVA logs)

Instructions to save browser console logs.
Procedure
1 Open Google Chrome and enter Developer Tools.
2 Open Developer Tools.
Use the keyboard shortcut Ctrl+Shift+I
or
Go to Customize and Control Google Chrome ► More Tools ► Developer

Tools

3 Go to Console ► Console settings tab.
4 Check the Preserve log checkbox to ensure that WebEM does not clear the log
during test execution.
5 Reproduce the problem.
6 Select one of the collected log entries and right click on it to open the pop-up menu.

7 Select Save as... to save the collected log.
3.4.3 Collecting browser network logs (HAR logs)

Instructions to save browser network logs.
Procedure
1 Open Google Chrome and enter Developer Tools.
2 Click the Network tab.
3 Select the Preserve log checkbox to ensure that WebEM does not clear the log
during test execution.
4 Reproduce the problem.
5 Select one of the collected log entries and right click on it to open the pop-up menu.
6 Select Save as HAR with content to save the collected log.

4. Fault monitoring
Issues with software or hardware are signaled using faults, which in turn raise alarms.
Fault messages come either from hardware devices or software components. The faults are
system-specific or hardware-specific. Faults are mapped to corresponding alarms, which are
sent to the element manager and to NetAct. Faults can also trigger LED indicators.
The BTS can detect faults after it has been commissioned and it is in a configured state.
During the start-up, the BTS waits for the real-time clock from the Network Time Protocol
(NTP) server, and the fault diagnosis starts after the time has been set, or a time-out has
been detected. If the BTS cannot set the time using the NTP server, the default time (that is
1.1.2004) is used.
When starting to solve a problem, first see the instructions given in an individual alarm
description (see the fields Fault source and Instructions in the alarm description).
4.1 Fault and alarm troubleshooting outline

Instructions on how to check the BTS faults and LED states.
Procedure
1 Establish a connection with the BTS using the element manager.
2 Read the fault and alarm information.
For more information, see Alarm monitoring using WebEM.
3 Check the unit states signaled by LEDs.
For descriptions of a module's LED states, see LED indications.
4.2 Finding faults in a snapshot file

The following procedure is an overview of troubleshooting methodology using a snapshot
file.

Before you start
This procedure requires the 7zip and NotePad++ applications, which can be downloaded
from the Internet free of charge.
Procedure
1 Save a BTS snapshot on the computer.
For information on saving snapshots, see the BTS snapshot section.
2 Unzip the snapshot files using 7zip.
3 Go to the LTEBTS/BTSLogFile or TDLTEBTS/BTSLogFile folder (depending on the

BTS technology).
4 Extract the BTSXXXXX_1011_RawAlarmHistory.txt/.xz file using 7zip and then

open *_RawAlarmHistory.txt using NotePad++.
5 Locate the fault in the *_RawAlarmHistory.txt file.
Using NotePad++, perform the Find operation and locate an alarm ID, for example
2004 .
Figure 39: Finding an alarm ID in the RawAlarmHistory.txt file
6 Find the fault source for a particular alarm ID.

Step example
2019-Mar-13 11:36:22 (0010) EFaultId_NoConnectionToUnitAl
/MRBTS-1/RAT-1/RUNTIME_VIEW-1/MRBTS_R-1/EQM_R-1/APEQM_R-1/RMOD_R-1
This fault indicates that the connection with the module that belongs to the BTS
configuration has been lost.
7 Troubleshoot the issues with the identified faulty port.
4.3 Sleeping cell

Sleeping cell overview, related alarms and recovery methods.
The sleeping cell is a case of cell outage that makes mobile service unavailable for
subscribers. This cell is unable to set up traffic. A sleeping cell can be caused, for example,
by faulty configuration, faulty radio circuits, other critical hardware, memory problems, or
other resource leakage. The configuration of broadcast information can also be corrupt,
with the result that mobile devices cannot read the system information required to access
the radio cell. The UEs in the sleeping cell coverage area cannot use any services.
For manual sleeping cell recovery actions see the following procedures:
Sleeping cell troubleshooting when alarms are raised
Sleeping cell troubleshooting when alarms are not raised
For automatic sleeping cell detection and recovery, see the Single RAN Operating
Documentation / Troubleshooting / Automatic Sleeping Cell Detection and Recovery
document.
4.3.1 Basic sleeping cell troubleshooting procedure

Follow this procedure to troubleshoot the sleeping cell issue.
Symptoms
The end user cannot make a call in a given area, despite being OnAir.
KPI or counter accessibility shows no RRC attempts in a given cell, while cell availability is
100%.

Procedure
1 Check if there is no mismatch in the cell configuration.
2 Check for HW issues (especially in the antenna line and RF module).
3 Collect a snapshot.
4 Lock/Unlock the cell or Reset the WCDMA RAT if needed.
For more information, see RAT reset.
5 If any specific alarm in RawAlarmHistory (see Finding faults in a snapshot file)

appeared around the zero RRC attempts timestamp, set a fault-triggered snapshot
collection for recurring scenario.
4.3.2 Sleeping cell troubleshooting when alarms are raised

Recovery methods for sleeping cells.
Purpose
The operator is informed about the existence of the sleeping cell with the following alarms:
7114: CELL SERVICE PROBLEM
7654: CELL OPERATION DEGRADED
The recovery parameter is set by the user through the configuration management.
Procedure
1 Block/unblock the sleeping cell.
This procedure can cause an RF outage for all RF equipment associated with a cell. This
applies only to the LTE3796: Automatic Sleeping Cell Detection and Recovery feature.
2 Shut down the sleeping cell gracefully.
3 Reset the Radio Frequency Module (RFM).
This procedure can cause an RF outage for the UEs attached to this RFM. This applies
only to the LTE3796: Automatic Sleeping Cell Detection and Recovery feature.

4 Reset the system module.
This procedure causes an RF outage for all cells connected to that system module.
5 Reset the site.
This procedure causes an RF outage for all cells connected to any system module at the
site.
4.3.3 Sleeping cell troubleshooting when alarms are not

raised
Purpose
According to the element manager or NetAct, the cell is in the onAir state, but no calls can
be set up or handed over to the cell. No alarms to indicate the problem have been raised.
Procedure
1 Check if KPIs are showing traffic in the cell.
2 Check if there is no mismatch in the cell configuration.
Check the configuration of:

power levels
maximum number of RRC connected users
neighbor cells
3 Check if there are no faulty RF cabling or other critical hardware issues.
4 Check the configuration of broadcast information.
UEs might not be able to read the system information required to access the cell.
5 Collect the logs (snapshot and cell traces).

5. Radio unit monitoring and troubleshooting

This section contains use cases related to issues with radio units (RF module / RRH),
remote electrical tilt (RET), masthead amplifier (MHA) and optical interfaces (SFP
connectors and fibers).
The radio unit problems are most likely related to:
Voltage Standing Wave Ratio (VSWR) or poor Received Signal Strength Indicator (RSSI)
fiber connections
power connections
5.1 Antenna line device problems

This section contains use cases related to issues with antenna line devices, remote
electrical tilt (RET), and masthead amplifier (MHA).
5.1.1 Antenna line, VSWR, and MHA troubleshooting

outline
The following procedure shows steps that can be performed in case of various antenna
line-related problems.
Description and symptoms

There are problems with antenna line functionality, for example active alarms with MHAs
and VSWR.
Uplink and downlink power levels are not correct and do not match the related
commissioning parameters.
MHAs are not visible in HW view or Commissioning.
Solution
Procedure
1 Check that the DC power is going to the MHA.
If the MHA is not visible in the HW view or Commissioning, measure the DC voltage
from an antenna connector with a proper tool.

Missing DC voltage indicates a problem in an RF module and an existing DC voltage

problem in either the MHA, the feeder cable, or the connections between the MHA
and feeder cable.
For more information, see the detailed procedure below:
2 Check if the antenna cables (main and diversity) are correctly connected to the
correct ports.
3 Re-check antenna line connectors (7/16” connectors with torque 25 Nm) for loose
connections. Also check the conditions of the cables to see if there is any visible
damage.
4 Use proper instruments to measure the antenna line, like Site MasterTM Cable and
Antenna analyzer, to verify if connectors, feeders and antennas are installed
correctly.
5 In case of MHA problems, first check that the commissioning has been done properly.
Check that the DC power is going to the MHA.
If the MHA is not visible in the HW view or Commissioning, measure the DC voltage from
an antenna connector with the corresponding tool. Missing DC voltage indicates a
problem in the RF module and an existing DC voltage problem in either the MHA, the
feeder cable, or the connections between the MHA and the feeder cable.
6 Check the VSWR and the antenna line faults, for example, with the Site Master, to
ensure that connectors, feeders, and antennas are connected properly.
A VSWR fault indicates that there are problems in the antenna line devices. Check that
the cables, connectors, MHAs, and antennas match and are not damaged. The
overcurrent fault indicates a short circuit in the antenna line or that the power feed to
MHAs is too low.
7 Check the site in 24 hours to ensure that the problem has been solved.

5.1.2 Radio Access Server (RAS), Remote Electrical Tilt

(RET) troubleshooting guide
This procedure is designed to help troubleshoot potential problems that may occur
during installation of the following RAS models: FASA, FASB, and FASC (as a supplement
to the installation instructions in the RAS Quick Guide).
Solution
Procedure
1 Ensure that RET and RF (BiasT Port) cables are firmly connected to the RAS ports.
2 Cycle the DC power (on ► off ► on).
3 Perform a RET calibration.
3.1 If the calibration is successful, set the tilt to maximum angle, then set the tilt
to minimum angle.
3.2 Repeat the tilting actions several times.
3.3 If the RET operates without alarms and other abnormalities, continue with RAS
installation to the site.
4 If problems continue, upgrade or re-load the latest firmware, and then repeat steps
Cycle the DC power and Perform a RET calibration.
5 If problems continue, swap cables and repeat steps Cycle the DC power and
Perform a RET calibration.
6 If problems continue, return the RAS to Nokia for inspection using the appropriate
RMA or Red Tag process.
5.1.3 RET not detected

Troubleshooting procedure for the case when the RET is not detected.


After the commissioning is completed, the SBTS does not detect the RET.
Solution
Procedure
1 Ensure that the correct firmware is installed.
Check the antenna type and verify that the device is updated with the correct firmware.
2 Disable/enable AISG and DC voltages.
In Web Element Manager, from the main menu go to Configuration

► Configuration Management ► Commissioning Wizard and disable AISG
and DC voltages, then click the Send button. Once the SBTS is back up, re-enable the
AISG and DC voltages and click once again the Send button. Verify the RET unit visibility
once the SBTS is back up.
3 Check the connection between antenna line and jumper cable.
There should not be any Bias-T installed between the antenna line and the jumper
cable.
4 Check if Top Bias-Ts are installed backwards.
If Top Bias-Ts are installed backwards, they should be properly installed. The antenna
crew should go to the top of the tower and fix this.
5 Check antenna line labeling.
Follow antenna lines from an antenna to the RF module and make sure the labeling is
consistent. Swap antenna lines 1 and 2. Top Bias-T should be connected to the antenna
line 1 (according to RF module labeling).
5.1.4 RET not calibrated or jammed

Troubleshooting procedure for the case when the RET is not calibrated or it is jammed.

After the commissioning is completed, the SBTS detects the RET, but the Not calibrated

or Motor jammed errors are displayed.
Solution
Procedure
1 Unjam the RET motor by recalibrating.
Set the RET angle to zero degrees and try to calibrate the RET again.
2 Send an antenna crew up to the tower to unjam the motor.
This step requires an antenna crew to physically unjam the RET motor. If unjamming the
RET motor does not solve the problem, replace the RET actuator. If the RET is still
jammed, then the antenna may need replacing.
5.2 RRH troubleshooting outline

The following procedure is an overview of the RRH troubleshooting methodology.
Purpose
Follow these steps until the radio unit is operational.
Procedure
1 Check all active alarms in the SBTS and follow the information in the online help
window.
For more information, see Fault and alarm troubleshooting outline.
2 Check and clean cables between the system module and RF module.
3 Depending on the fault source, reset the RF module and/or system module.
4 Replace the SFP transceivers.
5 Replace the RRH.
6 Power down the entire BTS for five minutes and then restart it.

7 Check or replace the power modules.
8 Replace the chassis.
5.3 Fault 1807 raised when bandwidth or frequency is

exceeded
Fault 1807 can appear on a single cell or on the whole antenna, depending on whether the
bandwidth or frequency ranges are exceeded.
Fault 1807: Invalid frequency channel for the BTS HW is raised when the radio
channel that has been requested in the commissioning file is not supported by the detected
BTS radio or the masthead amplifier (MHA) hardware because of an incorrect carrier
bandwidth or incorrect center frequency.
When this fault is raised, the BTS capacity decreases. The cell (or the whole antenna) with
the incorrect configuration is not working.
Fault 1807: Invalid frequency channel for the BTS HW has the following
related alarms:
7653: CELL FAULTY
7112: BASE STATION CONFIGURATION PROBLEM
Fault is raised on the whole antenna

If the maximum instantaneous bandwidth is exceeded, the fault 1807: Invalid
frequency channel for the BTS HW is raised on the whole antenna.
In this case, the Alarming object in WebEM Alarm management ► Active alarms is
the BSC or RNC.
Fault is raised on one cell

For the single-band RF unit, the carrier spacing between the two most distant carriers from
the FDD-LTE, WCDMA, or GSM must not exceed the maximum total bandwidth of the RF
unit.
If the frequency exceeds the range on a cell, but the maximum total bandwidth is in range,
fault 1807: Invalid frequency channel for the BTS HW is raised on that cell.

In this case, the Alarming object in WebEM Alarm management ► Active alarms is
the affected cell.
Solution
Check the parameters of the frequencies against the BTS hardware configuration.
Check whether the connected radio supports the carrier bandwidth requested in the
commissioning file, and whether the central frequencies of the TX and RX are correct.
There are different types of variations of the hardware that can have limitations in the
supported frequency range and bandwidth.
5.4 Fault 1811 raised when routing direction changed on

the radio unit
Fault 1811 can appear when the routing direction is not configured properly. The alarm is
cleared after the recovery reset and the new radio detection, so that a correct
configuration can be performed.
The 1811: Baseband bus configuration of <x> was rejected fault is raised
when the baseband-bus-related configuration of <x> is rejected by the hardware, software,
digital signal processor (DSP), or fronthaul switch (FHS) SW. This prevents the BTS from
functioning, and depending on the case, some processing resources, cells, or the entire BTS
can be out of order.
List of reported alarms:

7650 BASE STATION FAULTY
7651 BASE STATION OPERATION DEGRADED
7652 BASE STATION NOTIFICATION
7653 CELL FAULTY
7654 CELL OPERATION DEGRADED
7115 BASE STATION INFORMATION
7116 BASE STATION SERVICE PROBLEM
7101 BASE STATION/UNIT AUTONOMOUS RESET NOTIFICATION
For the radio unit (RU) routing direction changes, the 1811: Baseband bus
configuration of <x> was rejected fault is triggered when there is a difference
between the applied and calculated routing directions reported in WebEM. A recovery reset
must be performed that triggers the new radio detection procedure.

After this, the 1811 fault is cleared in WebEM, and a correct routing direction can be
configured.
5.5 Faults 4309 and 4310: Connection unstable

This fault indicates that the RP3 connection between the system module and radio unit is
not stable at 3 Gbps or 6 Gbps.

The Connection Unstable At 6 Gbps (4309) or Connection Unstable At 6
Gbps (4310) fault is active.
The transmission to the radio at 3 Gbps or 6 Gbps is lost.
Data is not being transmitted to or from the radio at all, or, in case of fault 4310, is
transmitted at a lower speed (3 Gbps).
The cells supported by the faulty link are disabled, or their operability is degraded.
Solution
Note:
This fault can be triggered by mistake as a result of frequent power on/off cycles of
the RRH (for example, due to testing). If power resets of the RRH are frequently
required, keep the SBTS on the air for at least five minutes before the next power
reset.
Procedure
1 If fault 4038 Radio master conflict is reported at the same time, troubleshoot
it first.
2 Check if the optical transmission network connecting the system module and radio
supports the required transmission speed.
3 Ensure that the cable connecting the modules in connected firmly.
4 Replace the cable.

5 Change the radio where the unstable link is terminated.
6 If this does not solve the issue, contact Nokia Technical Support.
5.6 Radio performing autonomous reset

Troubleshooting procedure when the radio performs autonomous reset

The radio is performing autonomous reset.
Solution
This is an expected behaviour. The radio that was not detected by system module performs
autonomous recovery reset. Recovery reset is initiated after a defined time period. The time
period depends on the number of already performed consecutive recovery resets.
Troubleshooting procedure: Solution

▪ Connect the radio to the system module or power off the radio.
5.7 Nokia CPRI radio not detected after BTS software

upgrade
Instructions on how to configure the linkspeed parameter after the radio software
upgrade failed.
Purpose
When installing on site a new radio unit (RU) from the table below, if the RMOD Link speed
(linkspeed) parameter is set to Auto, and the RU runs on a factory SW or an old SW, then
the CPRI link negotiation between the system module and the RU can fail and the RU isn't
detected. To resolve this issue, configure the linkspeed parameter after the RU SW
upgrade has failed.

Table 2: RUs with product codes and versions
RU Product code Version(s)
FZHM 086984A A/X21/X11
FZQE 473196A A/X21/X11
FZHS 087556A A/X21/X11
FZHN 473262A A
FZHR 473791A A
FZHW 473836A A
FZFI 087858A A
Procedure
1 Start the WebEM application.

Editor.
3 Set the RMOD Link speed (linkspeed) parameter to Cpri6.
Object path: MRBTS ► EQM ► APEQM ► RMOD
4 Wait for the RU to be detected and the site to be on Air, and change back the
linkspeed parameter to Auto.

6. RAT specific troubleshooting
6.1 GSM troubleshooting
6.1.1 GSM related logs

The GSM logs can be collected as BTS or BSC logs.
6.1.1.1 BTS logs

Guidelines for BTS logs
The type of detected problem or defect determines the level of information and data
collected. Faults can be grouped generally and the type of fault determines what type of
logs or traces is required. The following checklist summarizes the mandatory requirements
for each fault type.

Table 3: Required logs for GSM call processing
Defect BSC Logs SBTS SCF BTS Packet BTS Log BTS Drive Test
Type Snapshot Abis Over Internal Log
PSN Log Log
WebEM n/a Applicable Applicable n/a Applicable n/a n/a
Commission Applicable Applicable Applicable Applicable Applicable Applicable n/a

ing
O&M and Applicable Applicable Applicable Applicable n/a n/a n/a

alarm
handling
Telecom call Applicable Applicable Applicable Applicable Applicable Applicable n/a

related
Remote Applicable Applicable Applicable Applicable Applicable Applicable Applicable

Transcoder
/ MGW
GPRS / Applicable Applicable Applicable Applicable Applicable Applicable Applicable

EGPRS
KPI Changes Applicable Applicable Applicable Applicable Applicable Applicable Applicable
Note:
It is very important to ensure that a fault condition exists when the traces or logs are
collected and, if different tools are used, that the clocks are synchronized (or the time
difference is noted).
Collecting BTS snapshots with WebEM

The SBTS snapshots can be collected with WebEM. For more information on how to do that,
see Saving a BTS snapshot file using Web Element Manager.
Note:
While collecting snapshots for GSM issues, LTE cells cannot be selected, otherwise
GSM DSP dump files are not collected. The snapshot must be collected as early as
possible after the issue has been noticed.

IP Traffic capturing and Wireshark logs

For a detailed procedure, see .
Note:
While collecting Wireshark logs for GSM issues from WebEM, it is not required to select
a U-plane option.
In case GSM U-plane traffic is needed, the IP traffic data capture must be collected outside
of the SBTS, in a router or a switch, due to the limitation that only user-plane header
information is captured.
6.1.1.2 BSC logs

Guidelines for BSC logs
For BSC logs, the BSCLogcol tool is recommended. It collects basic BSC logs including
cellular radio network data, active alarms with alarm history, and BSC measurement files. It is
mandatory to collect logs from the BSC with BSClogcol before any trouble report is
submitted and attach them to the ticket. Internal BSC monitoring is optional. Measurement
file collection is needed for KPI and measurement counter analysis cases, and to support any
analysis with measurement counters.
The Bsclogcol.exe User Interface (UI) provides the following symptom report
options:
Collect basic data
Collect emergency logs
Collect basic data and measurements
Collect basic data and perform database integrity checks
Collect basic data and measurements and perform database integrity checks
Collect mcBSC embedded SW info
6.1.1.3 Tools
A summary of the tools needed for GSM log collection
The following table summarizes the tools needed for the corresponding type of trace or log:

Table 4: Tools needed for log collection
Required Log BSC Logs & BSC BTS snapshot Packet Abis Over BTS Drive
Tools Measurement & BTS SCF PSN Logs Internal Test Log
les Log
WebEM n/a Applicable n/a n/a n/a
BSCLogcol tool Applicable n/a n/a n/a n/a
Wireshark n/a n/a Applicable Applicabl n/a

e
Drive Test tool n/a n/a n/a n/a Applicabl

e
6.2 WCDMA troubleshooting

For general WCDMA troubleshooting, see the Single RAN Operating Documentation
/ WCDMA and GSM / IPA-RNC and mcRNC / Troubleshooting RNC /
Troubleshooting IPA-RNC document.
6.2.1 WCDMA related logs

A short description of the WCDMA related logs
For NetAct logs collected for WCDMA, see the Reporting tool and default reports section.
6.2.1.1 RNC logs

The basic log macro can be used for RNC alarm and configuration analysis.
In some instances, it is necessary to compare alarm history or configuration settings

between RNC and BTS. In such situations, it is recommended to use the special macro TS-
RNC-SW-038, which can be found in Support Portal. The macro requires a PC that can
execute an FTP connection to the RNC.

Tip:
There are three types of logs that can be collected by macro: Emergency, Standard,
and Basic. For alarm and configuration analysis, the Basic log version is enough. The
Standard log also includes samples of monitoring messages between the BTS and the
RNC.
6.2.2 WCDMA use cases

This chapters contains troubleshooting procedures (use cases) for most common
problems.
6.2.2.1 KPI degradation

Follow this procedure to troubleshoot the KPI degradation issue.
Symptoms
After a SW upgrade or a feature activation, certain KPI or KPIs are degraded.
Procedure
1 Confirm that the SW upgrade or feature activation time matches the KPI change
timestamp.
2 Decompose the KPI to raw counters MxxxxCxxx.
For more information on counters, see Single RAN Operating Documentation >
IPARNC and mcRNC > RNC Counters > Introduction to RNC Counters
3 Check the SW release note (TSN if KPI).
Step result
The counters are changed.
4 Check for changed and new parameters in case of KPI change.
Single RAN Operating Documentation ► IPA-RNC and mcRNC ► RNW

Parameters

Figure 40: Table of parameters
5 Check the number of affected sites.
If even a single site is affected, HW inspection is necessary.
6.2.2.2 Fault 110 is raised when the WCDMA cell is blocked

with the operational state disabled
Fault 110 can be raised due to the WCDMA cell operational state disabled
Identification
Fault 110: Unit RNW configuration failure is raised when the radio network unit
configuration or reconfiguration fails.
If you block a WCDMA cell in WebEM tool while the setup procedure for the cell is still in
progress, fault 110: Unit RNW configuration failure can be raised.
Related alarms to fault 110:

7101: BASE STATION/UNIT AUTONOMOUS RESET NOTIFICATION
7116: BASE STATION SERVICE PROBLEM
How to avoid the problem

1. Go to WebEM BTS Status ► Site Runtime View ► Site View.
2. Select the WCDMA cell, go to the Details panel ► Status, and check if the
operational state is enabled.
3. If the operational state is enabled, block the WCDMA cell.

Note:
If the WCDMA cell is disabled due to another known reason, for example a critical fault,
block the cell without waiting for the operational state to be enabled.
6.3 LTE troubleshooting
6.3.1 LTE related logs

The LTE logs can be collected as cell traces or as RF-related logs.
Throughout this topic, the term BTS refers to the SBTS and its LTE part and features. That
can be accessed on the SBTS through a browser, such as Google Chrome, at the URL
https://192.168.255.129 when the PC is connected locally to the LMP port of the
SBTS or https:< mPlaneIpAddressDN> when connected remotely.
Symptoms for call processing problems

The following symptoms are related to call processing problems:
Figure 41: Overview of symptoms for call processing related problems

For more information, see Flexi Multiradio BTS LTE / Maintenance Documentation / Flexi
Multiradio BTS LTE Technical Support Notes Library / Flexi LTE BTS Problem Report
Instructions, which applies to both FDD and TD. This document includes:
A section dedicated to L3 Data collector / Analyzer
Conditions for SW release upgrade, feature activation, and hardware introductions
S1, X2, S6a and SGi interface traces
Symptoms from Uu, UE tracing and Application Server
Event-triggered symptom data collection (LTE1099), Syslog substring-triggered
automated snapshot collection (LTE2250), and log collection triggered by BTS fault
cancellation (LTE2237).
Configuration is done under the TRBLCADM object:
MRBTS ► MNL ► MNLENT ► TRBLCADM.
Per Call Measurement Data (PCMD).

Note:
Starting with SRAN 18SP, the Nokia service account toor4nsn with root privileges is
blocked by default. For backward compatibility reasons, it can be activated to cope
with exceptional cases, such as when older scripts with toor4nsn user login must be
executed.
6.3.1.1 Cell trace

Cell trace content and instructions to activate cell trace
For more information about features related to tracing functionality, see Single RAN
Operating Documentation / Tracing in LTE and SRAN document.
6.3.1.1.1 Cell trace content

Short description of the content of cell trace data
Cell trace data consists of:
S1 Interface: S1AP data

Context management
Handover signaling
UE capability info indication
E-RAB management
NAS transport
Location reporting
Error indication
X2 Interface: X2AP data
Basic X2 mobility
Error indication
Uu Interface: RRC data
RRC connection signaling
Counter check
Inter-RAT mobility
Measurements
UE capability
UL/DL information transfer

Note:
The user plane is not traced.
6.3.1.1.2 Activating trace

Instructions to activate cell trace, MDT cell trace, or subscriber trace functionality with
Web Element Manager.
Procedure

Editor.
2 Go to the LNBTS object.
Object path: MRBTS ► LNBTS.
3 Set the cell trace related parameters.
Configure the following parameters to true:

Activate cell trace (actCellTrace)
Activate cell trace with IMSI (actCellTraceWithIMSI)
Activate MDT cell trace (actMDTCellTrace)
Activate subscriber and equipment trace (actSubscriberTrace)
4 Go to the CTRLTS object.
LNBTS ► CTRLTS

Note:
If the CTRLTS object is not present then

1. Create the CTRLTS object.
2. Configure the NetAct IP address (netActIpAddr) parameter.
5 Create an MTRACE object under CTRLTS.
The CRTLTS object contains the basic trace parameters. Every active cell trace session
equals one MTRACE object.
6 Configure the MTRACE object parameters.
To view the cell trace data in a tool other than NetAct, enter the appropriate address in
MTRACE Trace collection entity IP address (tceIpAddress).
Result
The cell trace data can be viewed with the NetAct TraceViewer, L3DA, or Traffica.
Note:
Tracing should be enabled only during troubleshooting procedure time frame, as it

requires a lot of processing resources from the BTS.
6.3.1.1.3 Viewing and analyzing cell trace data using L3DA

(Emil)
Instructions how to use the L3DA application
Purpose
The following call processing problems can be analyzed using the L3DA (Emil) tool:
Call setup problems
Call drops
Handover failures
For more information about cell tracing, see the Single RAN Operating Documentation /
Tracing in LTE and SRAN document.

For information about Emil troubleshooting, see the Troubleshooting with L3DA (Emil)
section.
Before you start
To collect the cell trace data, the tracing must be activated on the BTS.
Procedure
1 Start the cell trace.
To view traces in L3DA, the L3DC IP address must be included in the cell trace
configuration. The cell trace configuration is done with Web Element Manager or the
TraceViewer application. The L3DC must be selected as the Trace Collection Entity
during the cell trace configuration (the L3DC’s IP address put as the tceIpAddress
parameter value).
2 Open the L3DA.
To manage sacks and profiles:

2.1 Go to Environment ► Change sack and profile (SCREEN).
2.2 Select the one you need and click Load.
2.3 To configure the log paths (and other preferences) go to Tools ► Options.
3 Identify problematic calls by checking the Out cause column.
Every call scenario is displayed in one row. They can be grouped by clicking on one of
the columns in the main window (move the cursor on the column to see its description).
The grouping helps to locate common root causes for failed events. Columns useful for
troubleshooting:
Out cause
Examples of abnormal call releases:
No UE Reply
No EPC Message
No UE Messages
Radiolink Failure
Re-est Reject
Failure Phase
Examples:

RAB Active
RAB Setup
RAB Access
RRC Active
RRC Access
Source S1 HO Setup
Target S1 HO Access
Missing Reply
Examples:
RachPreamble
InitialUEMessage
rrcConnectionSetup
securityModeCommand
UE Distance
HO Attempts
4 To view the scenario details, right-click on the selected row and choose Trace view
or Message view.
The Trace view shows the scenario in an L3 view, Sequence view, or Message
view.
6.3.1.1.4 Tracing with NetAct

The NetAct TraceViewer application allows the user to activate and configure tracing.
For general NetAct use cases, go to the NetAct Operation Documentation /

Troubleshooting / Troubleshooting NetAct document and see the following
sections:
TraceViewer Overview
Tracing Subscribers and Equipment
For NetAct troubleshooting use cases for SBTS, see the Troubleshooting with NetAct
section.
6.3.2 BTSMED
BTSMED related logs

Starting with SRAN 20A, BTSMED is no longer supported. Instead, Direct Integration is used.
For more information on the solution, see the BTS Direct Integration document in
the Single RAN Operating Documentation.
6.3.3 LTE use cases

This section contains troubleshooting procedures (use cases) for most common problems
in LTE.
The table below lists symptoms of the most common issues encountered in the LTE
network. The troubleshooting methodology for some of the use cases can be accessed by
clicking their respective names in the table. When creating a problem report for Nokia
Services, it is recommended to use the same names as in the table.
Table 5: LTE use cases
Use case category Symptoms
Management tool • Problems with Web Element Manager connection via remote IP
problems address or local management port.
• Issues with Web Element Manager account.
• Issues with management tools under Linux.
Software problems • Software download or activation fails when started from NetAct
or Web Element Manager.
• Software fallback fails when performed as a recovery action.
Commissioning problems • Problems with starting or nishing commissioning.

• Missing commissioning parameters or units.
Synchronization problems • Synchronization source / unit not detected.

• GPS receiver alarms are present.
State change problems • The unit or cell state is incorrect.

• Changing state of unit or cell cannot be executed.
• Accidental state change.
BTS crash or reset • BTS crashes or resets as a recovery action.

• BTS resets as a result of adding a new frequency or cell.
• BTS continuously resets after it boots up.
Call processing problem • Calls cannot be established.

• RACH storms are visible.
• High RRC setup failure rate.
• Certain types of service cannot be activated for an established
call.

Cell problems Cell is faulty or cell operation is degraded.
High call drop rate Calls are dropping.
Uplink signal level or quality Interference or signal levels are abnormal, or the IQ constellation is
problems bad.
HO problems Handover-related mobility is not working properly.
Throughput problems The DL or UL throughput is lower than expected or is uctuating.
Paging problem • Paging messages are not visible in S1/Iub/Uu interfaces.

• Paging content is corrupted or IDs are invalid.
Call load related Problems related mostly to a high call load coming, for example,
performance issues from mass events. Performance degradation can be seen from BTS
stability, alarms, or call processing KPIs.
DSP crash A single DSP has crashed.
FSP connectivity/crash The whole FSP has crashed or reset as a recovery action.
Memory consumption Memory consumption is abnormally high or low; memory is leaking.

problems
CS fallback / cell reselection The interoperability between different RAT technologies is not
or redirect problems working uently (for example, UEs keep camping in a bad quality
network).
Time based problems Different timestamps do not match the NTP time or are not
aligned internally.
ANR problems • ANR is creating an excessive number of neighbours.

• Neighbour objects are not created or their content is empty.
• Automatically created neighbours cannot succeed in an HO.
S1/Iub/X2 interface Either a link setup fails or link connection is breaking.

problems
Latency problems The measurable time events (for example, ping times, DL/UL
packet delays, signaling responses) are abnormally long.
Alarms problem • BTS has clearly visible problems, but no alarms are visible in the
Web Element Manager or NetAct (for example, unit resets are seen
from logs or the RF carrier is lost, but no alarms are visible).
• Web Element Manager or NetAct shows an abnormal number of
certain types of alarms.
• A certain type of alarms is toggling between on and off.

KPI/PM counter problems • Counters are not visible in the BTS Web Element Manager or
NetAct.
• Counters or KPIs have invalid values, for example, success rate
over 100%.
• Counters cannot be processed with official tools.
Hardware problems Issues related to faulty units, not working interfaces or recovering
services.
Radio unit problems • Radio unit connectivity problems.

• Radio unit crash.
Antenna line device Problems with antenna line devices (for example, antenna line
problems devices are not detected, VSWR alarms, RET calibration).
6.3.3.1 Cell problems
6.3.3.1.1 Cell is not in on-the-air state

Follow this procedure to troubleshoot an issue with a cell not coming on the air.
Symptoms
After commissioning, HW or SW reset, or unblock, the cells do not go on the air state.
Procedure
1 Verify that the BTS and TRS are commissioned.
2 Verify that the cell is unblocked and unlocked.
3 Verify the S1 link connectivity.
In case of any problems, an S1 link alarm is raised.
4 Verify that the site has a valid timing source.
5 Check for configuration-related issues that must be corrected by recommissioning

the site.

6 Swap the hardware unit.
If the problem is not caused by configuration issues, then it is likely a hardware issue,
which means that the unit must be swapped.
6.3.3.2 Call processing problems

This section contains use cases related to call processing problems.
6.3.3.2.1 Troubleshooting call processing problems

The following generic steps describe a typical call processing troubleshooting procedure.
Before you start
Use the following tools for call processing troubleshooting:

KPI data viewer (for example, NetAct Reporting Suite)
KPI and counter documentation
Cell trace data
Wireshark logs
BTS snapshot
Procedure
1 Check which KPIs are indicating bad performance.
2 Check the KPI formula to find out which counters are causing the KPI degradation.
The counter description (Updated field) helps to determine the signaling scenario that
affects the degradation. This information is useful to decide which interface should be
further investigated.
3 Check when the KPI degradation started.
Check if any significant activities (for example, a site reconfiguration or SW upgrade)

have been done at the time when the KPI degradation started.
4 Analyze available logs and traces to gather more details.
Based on the counter investigation, the following data can be analyzed:

Call traces: the analysis can be done using the L3DA.

Interface monitoring: Wireshark can be used.

BTS snapshot: it can be opened with the element manager.
6.3.3.2.2 Parameters influencing call establishing and

processing
In case of any call-related issues, check the following cell parameters.
During commissioning, several cell parameters can be configured in such a way that UEs may
be having problems with establishing calls (including emergency sessions), or problems with
call processing may occur. The tables below list those parameters and explain their values.
Table 6: Parameters influencing establishing a call
Parameter name Meaning
SIB Cell barred flag The cell barred flag parameter indicates if the cell
(cellBarred) status is barred or notBarred. A barred cell is a cell
that a UE is not allowed to camp on, not even for
emergency calls. If a cell on which a UE is currently camped
becomes barred, a cell reselection is triggered.
SIB Cell reserved for Cells reserved for operator use are treated as barred by
operator use regular UEs without a USIM with AC 11 or 15 inserted. UEs
(primPlmnCellres) with AC 11 or 15 USIM inserted are specic operator UEs
and thus allowed to camp on and access such cells.
Table 7: Parameters influencing call processing

MPUCCH_FDD / MPUCH_TDD Max This parameter is a threshold for the maximum number of
Number RRC (maxNumRrc) UEs in a cell that can establish an RRC connection. The
maximum value depends on the bandwidth of the cell. The
following values are allowed:
• For 1.4 MHz bandwidth: 0–40
• For 3 MHz bandwidth: 0–120
The default value is not suitable for all bandwidths. The
recommended values are:
• For 1.4 MHz bandwidth: 20
• For 3 MHz bandwidth: 80
MPUCCH_FDD / MPUCH_TDD Add This parameter is an additional margin for the maximum
number active UEs radio number of active UEs in a cell, accessing the cell via a
reason handover handover with the HO desirable for radio
(addAUeRrHo) reasons HO-cause. This additional margin is also used
for UEs accessing the cell via handover with the Reduce
load in serving cell HO-cause but only when
Composite Available Capacity (CAC) in the cell is bigger
than zero. If CAC in the cell is zero, the HO for Reduce
load in serving cell is always rejected. This
margin is added to the LNCEL_TDD / LNCEL_FDD
Maximum number of active UEs
(maxNumActUe) threshold. The following values are
allowed:

MPUCCH_TDD / MPUCCH_FDD Add This parameter is an additional margin for the maximum
number active UEs time number of active UEs in the cell, accessing the cell via
critical handover handover with the HO-cause: Time Critical HO. This
(addAUeTcHo) margin is added to the LNCEL_TDD / LNCEL_FDD
Maximum number of active UEs
(maxNumActUe) threshold. The following values are
allowed:

MPUCCH_TDD / MPUCCH_FDD Max This parameter is a threshold for the maximum number of
Number RRC Emergency UEs in a cell that can establish an RRC connection, used to
(maxNumRrcEmergency) admit RRC connections for emergency calls. In order to
prioritize emergency sessions over normal traffic,
maxNumRrcEmergency should be greater than
maxNumRrc + max (addAUeRrHo, addAUeTcHo). The
parameter is only relevant if the LNBTS Activate
emergency call via redirection
actEmerCallRedir parameter value is set to true or
the LNBTS_TDD / LNBTS_FDD Activate support of
IMS emergency sessions for Rel 9 UE
actIMSEmerSessR9 parameter value is set to true
and ignored otherwise.
The emergency call is only recognized when the UE was in
idle state before initiating the connection.
The maximum additional margin depends on the
bandwidth of the cell. The following values are allowed:

LNCEL_TDD / LNCEL_FDD This parameter is a threshold for the maximum number of

Maximum number of active active UEs in a cell. Active UEs are UEs with at least one
UEs (maxNumActUe) data radio bearer (DRB). In case CAT-M UEs are served in
the same cell (hosting cell) and have at least one DRB, the
admitted CAT-M UEs reduce the number of legacy active
UEs accordingly because they share the same resources.
The maximum margin depends on the bandwidth of the
cell. The following values are allowed:
user can congure this parameter based on different
bandwidths and BTS types. The recommended values are:
Note:
maxNumActUe + max (addAUeRrHo, addAUeTcHo)

must be less than the margin values for the used
bandwidth (see values above: 40, 120, 840, 1000, 1250,
1500).
LNCEL_FDD / LNCEL_TDD Max This parameter is a threshold for the maximum number of
number act DRB established DRBs in a cell. The maximum number of
(maxNumActDrb) established DRBs depends on the bandwidth of the cell.
The following values are allowed:

For more information about parameters, see Single RAN / Operating Documentation / Single
RAN System / SBTS Reference Data / SBTS Parameters.
When troubleshooting call problems, the state of a cell must be taken into consideration.
6.3.3.2.3 Data related to high RRC setup failure rate and

RACH problems
This is an example of a call processing troubleshooting area along with examples of useful
data.
Related KPIs
LTE_5218d Total E-UTRAN RRC Connection Setup Success Ratio
RRC Connection Setup Failure Ratio per Cause (LTE_5229c, LTE_5230c, LTE_5590a)
For KPI and counter descriptions, see Single RAN / Operating Documentation / Single RAN
System / SBTS Reference Data / Counters/Performance Indicators.
L3 Data Analyzer indicators

Out Cause: No UE Reply
Failure Phase: RRC Setup
Missing Reply: RrcConnectionSetup
Related parameters
SIB Timer T300 (T300)
MPUCCH_FDD / MPUCH_TDD Max Number RRC (maxNumRrc)
MPUCCH_TDD / MPUCCH_FDD Max Number RRC Emergency (maxNumRrcEmergency)
Note:
Parameter optimization is a complex task; therefore, it is recommended to use the

Nokia Network Planning and Optimization (NPO) Services.

6.3.3.2.4 Data related to RAB setup problems

This is an example of a call processing troubleshooting area along with examples of useful
data.
Related KPIs
LTE_5017a E-UTRAN E-RAB Setup Success Ratio
E-UTRAN E-RAB Setup Ratio per Cause (LTE_5086a, LTE_5087a, LTE_5088a, LTE_5089a)
For KPI and counter descriptions, see Single RAN / Operating Documentation / Single RAN
System / SBTS Reference Data / Counters/Performance Indicators.
L3 Data Analyzer indicators

Out Cause: RAB Setup Failure
Failure Phase: RAB Setup
Related parameters
LNCEL_FDD / LNCEL_TDD Max number act DRB (maxNumActDrb)
LNCEL_TDD / LNCEL_FDD Maximum number of active UEs (maxNumActUe)
LNCEL_TDD / LNCEL_FDD Max number QCI1 DRBs (GBRs) (maxNumQci1Drb)
Note:
Parameter optimization is a complex task; therefore, it is recommended to use the

Nokia Network Planning and Optimization (NPO) Services.
6.3.3.2.5 VoLTE/ViLTE call drops when LTE1321 is enabled

in the network
Follow this procedure to troubleshoot an issue with the VoLTE/ViLTE call drops.


The LTE1321: E-RAB modification - GBR feature introduces a check on E-RAB-Guaranteed
Bitrate (GBR) if the GBR is increased during E-RAB modification. This could lead to
VoLTE/ViLTE call drops during the voice or video conference setup phase if the BTS
maximum GBR parameters do not take GBR overhead during voice or video conference
setup into account.
Solution
In releases before FL15A/TL15A, the BTS did not check the E-RAB-Guaranteed Bitrate DL/UL
if this E-RAB parameter is changed during the E-RAB modification procedure. With the
introduction of the LTE1321: E-RAB modification – GBR feature, the BTS checks whether an
increased GBR value is still in the allowed range defined by the Maximum GBR uplink
(maxGbrUl) and Maximum GBR downlink (maxGbrDl) parameters in LNBTS QCI
translation table QCI 1 (qciTab1) and LNBTS QCI translation table QCI 2
(qciTab2) parameter tables of the LNBTS object, respectively.
If VoLTE or ViLTE calls are extended to include additional parties or if VoLTE or ViLTE
conference calls are started, the E-RAB modification procedure is used to temporarily
increase the QCI1/QCI2 GBR by a factor of two or three. As a consequence, the GBR
configured by an MME for an existing E-RAB might exceed the Maximum GBR as defined in
the BTS configuration (LNBTS object) and the call might be dropped.
In order to avoid this call drop during VoLTE or ViLTE conference setup, it is recommended
to increase the following parameters to at least three times the value used for a simple
(non-conference) VoLTE or ViLTE call in order to have sufficient margin for the GBR increase
during conference call setup
LNBTS QCI translation table QCI 1 (qciTab1) / Maximum GBR uplink
(maxGbrUl)
LNBTS QCI translation table QCI 1 (qciTab1) / Maximum GBR downlink
(maxGbrDl)
LNBTS QCI translation table QCI 2 (qciTab2) / Maximum GBR uplink
(maxGbrUl)
LNBTS QCI translation table QCI 2 (qciTab2) / Maximum GBR downlink
(maxGbrDl)

Troubleshooting procedure: Increasing the value of maximum GBR

downlink/uplink using Web Element Manager
Procedure

Editor.
Object path: MRBTS ► LNBTS
3 Under the QCI translation table QCI 1 (qciTab1) set the following
parameters to a value at least three times the value used for a simple call.
Parameters:
Maximum GBR downlink (maxGbrDl)
Maximum GBR uplink (maxGbrUl)
4 Under the QCI translation table QCI 2 (qciTab2) set the following
parameters to a value at least three times the value used for a simple call.
Parameters:
Maximum GBR downlink (maxGbrDl)
Maximum GBR uplink (maxGbrUl)
6.3.3.3 BTS crash or reset

This section contains use cases related to BTS crash or restart problems.
6.3.3.3.1 BTS in a reset loop

Follow this procedure to troubleshoot the BTS-in-a-reset-loop issue.

Symptoms
The BTS continuously resets after it boots up. It gets stuck in a continuous reset loop, and
it is not possible to gather the log files to analyze the fault.
Procedure
1 Check all cables and connectors.
Ensure the connectors are not loose and the cables are not bent.
2 Reboot the RF modules.
3 Switch the power off and on again.
4 If the problem occurred after a SW upgrade, revert to an older SW version.
5 Manually replace the site configuration file.
For more information about uncommissioning manually, see the Single RAN Operationg
Documentation/ Single RAN System/ Commissioning and Configuring SBTS/
Comissioning SBTS/ Uncommissioning/ Uncommissioning manually document.
6.3.3.3.2 Baseband pool reset for fault recovery

Baseband (BB) pool reset for ABIA hosting LTE cells.
The SBTS supports the following BB resource resets:

Site reset
BTS reset
Module (FSP) warm reset
Digital signal processor (DSP) reset
BB pooling for ABIA allows efficient usage of BB resources, permitting a larger number of
cells to be hosted by a single ABIA. A higher individual cell capacity can be achieved in an
unbalanced load situation.
When service degradation is detected on cells from a specific BB pool, the BTS can perform
an autonomous reset of the affected BB pool as part of fault recovery, instead of resetting
the entire ABIA. This reduces the system downtime and does not disable cells. In case of a
capacity module fault, the required BB pool is reset instead of the entire BB unit based on

the reset criteria.
The user is able to check in advance which cells are impacted by the BB pool reset with the
BBPOOL List of cells identifiers hosted by the baseband
pool(bbPoolCellsList) parameter.
The APEQM Allow BBPool Recovery Reset Action (allowBbPoolRecoveryReset)

parameter specifies if BB pool reset is used as a recovery action. It is set to true by default.
The BB pool reset includes system-on-chip reset and software reset of the associated user
plane (U-plane) software.
The BTS reports one of the following alarms before the BB pool autonomous reset:
7652: BASE STATION NOTIFICATION alarm, when the FMCADM Activate
Category Alarms (actCategoryAlarms) parameter is set to false
7101: BASE STATION/UNIT AUTONOMOUS RESET NOTIFICATION alarm, when the
FMCADM Activate Category Alarms (actCategoryAlarms) parameter is set to
true
The following faults can trigger the BB pool autonomous reset:

Fault 6: Unit SW download failed
Fault 16: Unit synchronization failed
Fault 69: POST test failed
Fault 1806: Communication failure
Fault 2074: SRIO retry stopped
Fault 4019: Master unit has lost connection to the slave unit
Fault 6910: SRIO HW issue detected during sRIO message receiving
Fault 6710: AIF2 HW data buffer overflow
When the alarm is raised, the corresponding fault ID and BBPOOL autonomous reset as
recovery action as fault name is reported in the alarm details.
The BTS performs autonomous BB pool reset in the following scenarios:
If the BB pool reset does not disable any cells.

If the BB pool is hosting LTE cells.
If the FMCAD Activate Force Unit Autonomous
Reset(actForceUnitAutoReset) parameter is set to true, then the BTS triggers a BB
pool reset for fault recovery, regardless of the impact on the corresponding cells. The
BTS disables the working cells only after performing a graceful shutdown of these
cells.
If the same BB pool fault has not occurred more than once in the last hour.
If the BB pool has not been autonomously reset three times in the last hour.

The autonomous recovery is not executed in the following scenarios:

If the K2A/K2B pair has a DSP fail, the whole ABIA is reset and service is lost for all
cells configured on the corresponding ABIA.
If a cell is in degraded state, the autonomous recovery is not executed.
If cells have 32 and 64 TRX TDD configuration, the autonomous recovery is not executed.
If a BB pool reset is not triggered due to the recurrence of the same fault or three
autonomous resets in the last hour, then the BTS performs a capacity module warm
reset as long as its preconditions are met.
6.3.3.3.3 Adding new frequency or cell without BTS restart

The LTE2060: Add New Frequency or Cell without Reset feature removes the need for a
site outage when changing both configuration and the parameters associated with site
expansion (for example, adding RF hardware to add cells or carriers).
Configuring new cells and, adding new RF and baseband (BB) hardware currently requires an
eNB restart on every occasion. This involves service outage of all existing cells. The
LTE2060: Add New Frequency or Cell Without Reset feature supports configuration of new
cells and addition of new BB, RF, and ALD hardware (only with a newly added RF module)
without an eNB restart in many scenarios which are relevant for typical customer upgrade
situations such as introduction of an additional frequency layer into an existing network.
Carrier aggregation (CA) between a newly added cell and existing cells can be configured
without an eNB restart. Also, the uplink coordinated multipoint transmission (UL CoMP) can
be configured without restart only between new added cell and existing cell, provided the
UlComp is already enabled and digital signal processor (DSP) pools need not be rearranged.
Within the scope of the LTE2060: Add New Frequency or Cell Without Reset feature, there
are still the following restrictions:
rearranging existing DSP pools still requires reloading the DSP SW, which requires
temporary dropping of all cells from affected FSP cards
changing the Activate flexible base band usage (actFlexBbUsage)
parameter no longer requires an eNB restart, but all cells will be temporary dropped
configuring the dual mode with CPRI RF modules requires an eNB restart
a DSP or FSP reset is required for fiber lenghts longer than 10 km
a DSP or FSP reset is required if a new cell with band 39 or band 40 is added, and there is
no other cell with such a band

Note:
Adding new cell with longer fiber (longer than 10 km) than in existing cells (shorter
than 10 km) requires locking and recalculation due to alignment of DSP timings
between the cells. If DSP timings are already aligned to long fibers, adding a new cells
with long fiber does not impact on existing cells.

7. Software and synchronization troubleshooting
7.1 Software problems

This section contains use cases related to the most common software issues and faults.
Most troubleshooting procedures related to the software faults require:

downloading software
All available software versions can be downloaded from Support Portal, at the following
path Care ► Software Supply Tool.
checking the BTS software version
To check the current software used by the BTS in WebEM, choose
Software ► Software Management ► Software version from the upper menu.
upgrading the BTS software
To upgrade the software using WebEM, choose Software ► Software
Management ► Software Update from the upper menu.
blocking, unblocking, resetting, or replacing the unit
See Basic troubleshooting steps for more information about these procedures.
7.1.1 Viewing a software download report

Follow this procedure in case of errors following a BTS software update.
Purpose
If a software update fails, users can access the software download report to identify why.
Procedure
1 When a SW update fails, click on the Save SW Update Report button.

Figure 42: Software update failed message
The software update report can also be found in the snapshot file
(BTSXXXXX_1011_SWDLreport).
Result
As a result, a TXT file is downloaded.
The figure below shows an example report. In this example, the software download was
cancelled on user request.

Figure 43: Software update report
7.1.2 Reset to a failsafe partition

Troubleshooting for cases where the module cannot boot up either from an active or a
passive partition or when problems appear after a Plug and Play procedure, resulting in
incorrect software download. The following procedures are only for AirScale System
Module and describes the reset to a failsafe partition.
Symptoms
Case 1: The module cannot boot up either from an active or a passive partition - for the
troubleshooting procedure, see Resetting to failsafe partition - case 1.
Case 2: Some problems appear following a Plug and Play procedure and the software is not
downloaded correctly - for the troubleshooting procedure, see Resetting to failsafe
partition - case 2.
7.1.2.1 Resetting to failsafe partition - case 1

Troubleshooting for cases where the module cannot boot up either from an active or a
passive partition. The following procedure is only for AirScale System Module and
describes the reset to a failsafe partition.

Procedure
1 Press the service button.
2 Wait until the unit boots up from the failsafe partition.
3 Download the software again.
4 Update both partitions prior to the commissioning steps.
Result
The unit boots up from the failsafe partition.
Recovery action
To switch back from the failsafe partition to the normal partition quick press the Reset
button or execute a power reset.
7.1.2.2 Resetting to failsafe partition - case 2

Troubleshooting for cases when problems appear after a Plug and Play procedure,
resulting in incorrect software download. The following procedure is only for AirScale
System Module and describes the reset to a failsafe partition.
Procedure
1 Press the service button.
2 Wait until the unit boots up from the failsafe partition.
3 Repeat the Plug and Play procedure.
Result
The unit boots up from the failsafe partition.
Recovery action
To switch back from the failsafe partition to the normal partition quick press the Reset

button or execute a power reset.
7.1.3 Activating and using WebEM Remote Tool (former

BTS CLI) when Web Element Manager is unavailable
Detailed information about activation of WebEM Remote Tool (former BTS CLI) when Web
Element Manager is unavailable and a use case example
WebEM Remote Tool is an application that allows the operator to interact with the BTS using
consecutive lines of text (command lines). The interface is handled by a shell or a Command
Language Interpreter, and the request can be executed remotely or locally.
Note:
For a detailed description and user guide of the WebEM Remote Tool, see the Single
RAN Operating Documentation / Single RAN System / WebEM Remote Tool document.
7.1.3.1 Activating WebEM Remote Tool (former BTS CLI)

when Web Element Manager is unavailable
Detailed information about activation of WebEM Remote Tool (former BTS CLI) when Web
Element Manager is unavailable
Purpose
This procedure presents the activation of WebEM Remote Tool (former BTS CLI) using
NetAct and is recommended when WebEM is not available.
Procedure
1 Log in to NetAct.
2 Open CM Editor.
3 Go to Actual Configuration in the left side bar and find the BTS where WebEM
Remote Tool must be activated.
Actual configuration ► PLMN-PLMN ► MRBTS
4 Go to the FEATCDM object.

Object path: CONF ► MRBTS ► MNL ► MNLENT ► FEATCDM
Step example
5 Check the Modify actual values box above the list of parameters.
6 Set the Activate CLI (actCli) parameter value to true.
Step example
7 Update the plan.

7.1 Right-click on the FEATCDM object.
7.2 Click on the Send to network option.
Step example
Result
WebEM Remote Tool is activated in the selected BTS.
Note:
The same operation can be done using WebEM.

7.1.3.2 Performing a software rollback using WebEM

Remote Tool (former BTS CLI) when Web Element
Manager is unavailable
A use case example of WebEM Remote Tool (former BTS CLI): performing a software
rollback using WebEM Remote Tool when WebEM is not available.
Purpose
This section presents how WebEM Remote Tool is prepared for use, how to edit the scripts
used to connect to the BTS, as well as an example of how to perform a software rollback
using WebEM Remote Tool.
Procedure
1 Download the latest software package of WebEM Remote Tool from Support Portal.
Note:
For detailed instructions on downloading the WebEM Remote Tool software

package from Support Portal, see the Downloading WebEM Remote Tool from
Support Portal section in the Single RAN Operating Documentation/ Single RAN
System/ Commissioning and Configuring SBTS/ WebEM Remote Tool document.
2 Extract the files from the downloaded software package archive to a chosen location.
3 Edit the script with credentials used to connect to the BTS.
Linux users
1. Find the getSfpData.sh file.
2. Edit the getSfpData.sh file as in the following example:

#!/bin/bash
ADMIN_CLI=../admin-cli.sh
USERNAME='Nemuadmin'
PASSWORD='nemuuser'
HOST='localhost'
PORT='3600'
FORMAT='human'
DATA='{"requestId":1,"requestType":"infoModel","parameters":{"name":"sfpData"}
}'
$ADMIN_CLI --bts-username=$USERNAME --bts-password=$PASSWORD --bts-host=$HOST
--bts-port=$PORT –-format=$FORMAT --data=$DATA
Windows users
1. Find the getSfpData.bat file.
2. Edit the getSfpData.bat file as in the following example:
@echo off
set ADMIN_CLI=admin-cli.bat
set USERNAME=Nemuadmin
set PASSWORD=nemuuser
set HOST=localhost
set PORT=3600
set FORMAT=human
set
DATA={\"requestId\":1,\"requestType\":\"infoModel\",\"parameters\":{\"name\":\
"sfpData\"}}
call %ADMIN_CLI% --bts-username=%USERNAME% --bts-password=%PASSWORD% --bts-
host=%HOST% --bts-port=%PORT% --format=%FORMAT% --data=%DATA%
pause
4 Open the Command prompt.
5 Go to the BTS CLI directory.
6 For Linux users only: allow the execution of the admin-cli.sh file.
6.1 chmod +x admin-cli.sh

7 For Linux users only: allow to execute node.
7.1 Go to node ► linux ► bin
7.2 chmod +x node
8 Execute the admin-cli command with a software rollback request as in the

examples below:
Linux users
admin-cli.sh --bts-username=Nemuadmin --bts-password=nemuuser -
-bts-host=192.168.255.129 --bts-port=443 --format=human --
data={\"requestId\":46,\"parameters\":{\"name\":\"swRollback\"}
}
Windows users
admin-cli.bat --bts-username=Nemuadmin --bts-password=nemuuser
--bts-host=192.168.255.129 --bts-port=443 --format=human --
data={\"requestId\":46,\"parameters\":{\"name\":\"swRollback\"}
}
Result
The software rollback is performed.
7.1.4 Commissioning problems - RMOD link

synchronization and SW update
Troubleshooting scenario for failed SW upgrade performed during radio module link
synchronization.
Problem description
When the BTS is in uncommissioned state and has a synchronization source provided, it
must synchronize the radio module (RMOD) links to that source. In order for that to happen,
the radios must be reestablished.
If the SW update is performed during RMOD link synchronization, the RMOD can reset. The
BTS raises the Fault 30: BTS SW download or activation failed, and the SW
update can fail.

Solution
Troubleshooting procedure: RMOD SW update

▪ If the SW update performed during RMOD link synchronization failed, perform the
SW update operation again.
7.2 Synchronization problems

This section contains procedures for synchronization verification.
7.2.1 GPS problems – synchronization source not detected

Troubleshooting procedure for the case when there are GPS problems and the
synchronization source is not detected
Symptoms
Element manager shows Not available status under GNSS receiver in

Synchronization source and displays the fault BTS reference clock missing.
Note:
The path to where this status is displayed in Web Element Manager is

Diagnostic ► Synchronization ► Status .
Recovery procedure
Procedure
1 Check the cable connection at the GPS antenna head.
Verify if the GPS cable was securely connected to the GPS antenna head – unplug it and
plug it back in.
2 Check the GPS cable connections at surge arrester.
Verify if the wiring at the surge arrester follows the documentation. This includes
shielding, color code, and correct pairs.

3 Check the GPS cable's connection to the system module.
In the case of an HDMI connection, ensure that the connector is properly inserted,
rearrange the cable, and inspect the HDMI socket in the SM so that there are no bent
pins.
4 If the previous steps did not solve the issue, replace the GPS antenna head.
7.2.2 Fallback solution for phase synchronization

The following procedure is a fallback solution in case an FDD-LTE BTS in a phase
synchronization mode cannot complete startup due to a missing GNSS receiver control
interface.
Purpose
The problem was visible in the field, where, as a result of an installation issue with a GNSS
receiver or a hardware malfunction, the GNSS receiver control interface (containing, among
others, time of day) was not available to the BTS. Without this information, the BTS cannot
go on the air. As a solution, the autonomous switching from a phase to a frequency mode
before a system frame number alignment was introduced.
Procedure
1 The BTS performs phase mode startup.
2 The BTS detects that the GNSS receiver does not provide a sufficient signal to be
used as a phase synchronization reference.
The following faults can be raised:

4011 (GPS receiver alarm: control interface not available)
4122 (GPS receiver alarm: no tracking satellites)
4123 (GPS receiver alarm: survey in progress)
4124 (GPS receiver alarm: no stored position)
4125 (GPS receiver alarm: position questionable)
4126 (GPS receiver alarm: EEPROM invalid)
3 The BTS waits for other phase synchronization sources or for the issue with GNSS to
be resolved, until the timer expires.
Fault 4153 (Reference clock missing in startup) is raised with additional info

that startup will be continued in a frequency mode with deactivated features.
4 If the BTS is not Sync Hub slave in RF sharing, the tuning mode is changed to a
frequency mode.
The BTS deactivates features requiring network-level phase synchronization (for

example OTDOA, eICIC, MBMS, SIB8 broadcast) and features related to inter-BTS carrier
aggregation.
5 The BTS performs frequency tuning.
Result
The BTS goes on the air.
When the GNSS becomes available, all GPS receiver alarms and fault 4153 are cleared. The
BTS raises fault 4001 (BTS reset required), informing that a manual reset is required.
There is no need to configure the BTS to a phase synchronization mode or manually enable
features dependent on the phase synchronization. Once a manual reset is done, those
features will be on the air automatically.
Note:
In case the GNSS receiver self-survey is not yet finished, the GNSS receiver does not
start to provide a 1pps/ToD signal to the BTS. In case the BTS starts up during this
phase and it is configured for phase synchronization mode, it completes the startup
using frequency synchronization and a manual restart is needed later on to change to
phase synchronization mode. The GNSS receiver self-survey takes about 30 minutes
and is usually only needed after an initial GNSS start-up or a manually triggered
deletion of the positioning data.
7.3 System upgrade use cases

Important information to be taken into account for validation during the system upgrade
process and troubleshooting aspects for the system upgrade process
Before any upgrade or migration to SRAN 20A, the validation of the LTE BTS
configuration and the correction of any errors are prerequisites. The validation can be
done by creating a plan with any parameter change and running the validation only for
the created plan. For details on these procedures, see the section Provision in the NetAct
Operating Documentation / Configuration Management Helps / CM Operations Manager

Help.
If the First endpoint distinguished name (firstEndpointDN) parameter
references the SMOD or BBMOD managed objects, and the Second endpoint
distinguished name (secondEndpointDN) parameter references the TDD RMOD
managed object, the Link speed (linkSpeed) parameter must be set to one of the
following values: Obsai6Gbps, Cpri5, Cpri6, or Cpri7.
For system upgrade use cases, see the Single RAN Operating Documentation / System
Upgrade and Migration document.
7.3.1 ALDs connected to the antenna line and/or RS485

ports not detected after system upgrade
Troubleshooting procedure for when ALDs connected to the antenna line and/or RS485
ports are not detected after the system upgrade.
Problem description
ALDs connected to the antenna line and/or RS485 ports are not detected after the system
upgrade.
Solution
The following parameters were not properly configured before system upgrade process:
RSL HDLC communication enabled (hdlcCommunicationAllowed)
RSL DC voltage enabled (dcVoltageEnabled)
ANTL HDLC communication enabled (hdlcCommunicationAllowed)
ANTL DC voltage enabled (dcVoltageEnabled)
Procedure

Editor ► .

2 Go to the RSL object.
3 Set the HDLC communication enabled (hdlcCommunicationAllowed)

parameter value to true.
4 Set the DC voltage enabled (dcVoltageEnabled) parameter value to true.
5 Go to the ANTL object.
6 Set the HDLC communication enabled (hdlcCommunicationAllowed)

parameter value to true.
7 Set the DC voltage level (dcVoltage) parameter value to LOW, HIGH, or AUTO.
7.3.2 Fault 10 raised on one BTS after successful software

upgrade
In case of successful software upgrade on the host BTS, fault 10 Failure in optical
interface with alarm 7653 Cell faulty is raised in one of the BTSs. To avoid this
situation, configure the SYNC parameters.
Identification
Before system upgrade, check the configurations with:
two or more BTSs connected with Sync Hub Direct Forward (SHDF) chain, where the SYNC
BTS synchronization mode (btsSyncMode) parameter in the host BTS is
configured to LoosePhaseAndTimeSync or FreqSync and the SYNC BTS
synchronization mode (btsSyncMode) parameters in all the pair BTSs are
configured to PhaseSync.
radio units shared between any two BTSs.
In case of successful software upgrade on the host BTS, some connections to shared radio
units can be dropped and the fault 10 Failure in optical interface with alarm
7653 Cell faulty is raised in one of the BTSs. This fault disables the cells until the clocks
are resynchronized. After this period, the alarms are cleared by themselves.

How to avoid the problem

To avoid this situation, configure the SYNC Phase error threshold for fault
recovery action (phaseErrorRecoveryThreshold) parameter in the pair BTSs.
Nokia recommends the value of 5000 ns for this parameter.
7.3.3 Keeping legacy cell set in use after system upgrade

Procedure to keep the legacy cell set in use after system upgrade, using Web Element
Manager (WebEM) for AirScale system modules
Problem description
After system upgrade, the Activate prohibition of CA-specific cell
deployments (actProhibitCaSpecCellDepl) parameter is set to the false default
value. This can cause network degradations.
For network level operations, see the NetAct Operating Documentation/Troubleshooting

NetAct document.
For BTS level operations, follow the procedure below.
Solution
Procedure
1 Start WebEM and connect to the BTS.

Editor.
Use the keyboard shortcut. Press the P key and then the E key.
Object path: MRBTS ► LNBTS
4 Set the Activate prohibition of CA-specific cell deployments

(actProhibitCaSpecCellDepl) parameter to true.
5 Set the Activate inter-eNB DL carrier aggregation

(actInterEnbDLCAggr) parameter to true.
6 If not previously configured, set the carrier aggregation parameters.
6.1 Set the A2 default profile identifier (actDLCAggr) to true.
6.2 Configure the Activate flexible carrier aggregation management

(actFlexCaMgmt) parameter.
6.3 Configure the Carrier aggregation cluster ID (caClusterId)

parameter.
6.4 Configure the Carrier aggregation cluster member ID

(caClusterMemberId) parameter.
6.5 Configure the Activate inter-site CA HARQ enhancement

(actIscaHarqEnh) parameter.
6.6 Go to the CAGENB object.
Object path: MRBTS ► LNBTS ► CAGENB
6.7 Configure the eNB list for CA cluster (caClusterEnbList) parameter.
6.8 Go to the LNCEL object.
Object path: MRBTS ► LNBTS ► LNCEL
6.9 Configure the Activate improvements to CA architecture

enhancements (actCaArchEnhImp) parameter.
6.10 Configure the Activate enhanced flow control (actFlowCtrlEnh)

parameter.
6.11 Go to the CAREL object.
Object path: MRBTS ► LNBTS ► LNCEL ► CAREL
6.12 Configure the Scheduling CA fairness control factor

(caSchedFairFact) parameter.
7 Set the Activate baseband pooling (actBbPooling) parameter to true.
8 If not previously configured, create and configure the BBPOOLALLOC object for each
LNCEL.
Object path:MRBTS ► LNBTS ► LNCEL ► BBPOOLALLOC
9 Restart the BTS.
7.3.4 Fault 23: Incompatible SW version detected is raised

after software fallback
If some hardware units aren't detected during software fallback, and they are detected
after the operation, fault 23 is raised on the newly detected units.
In case some of the hardware units aren’t detected during software fallback process (they
aren't visible on the BTS software running on the system module), the software fallback is
executed correctly only for the detected hardware units.
Fault 23: Incompatible SW version detected is raised for the hardware units
detected after software fallback as having the software version activated before software
fallback process was initiated.
For system upgrade use cases, see the SRAN Operating Documentation / Single RAN System
/ Upgrading SRAN / SRAN System Upgrade and Migration document.
7.3.5 DSS cell degraded after system upgrade

If after system upgrade a dynamic spectrum sharing (DSS) cell is degraded and fault 7519
and 6137 are raised, reconfigure the msg1FrequencyStart parameter.
When upgrading the system from SRAN 20B to SRAN 20C, a DSS cell gets degraded. This is
caused by the introduction of the LTE physical uplink shared channel (PUSCH) reservation
mechanism. This mechanism requires support of an optimised UL physical resource block
(PRB) configuration with the LTE physical random access channel (PRACH) located at the
lower edge of the spectrum. The following faults appear:
7519: EN DSS UL Cell Coordination Failure - Resource Conflict -

Non matching resource reservation configuration

6137: EN DSS UL Cell Coordination Failure - Resource Conflict -
Non matching resource reservation configuration
To clear the faults, you must adjust the NRCELL PRACH Frequency start
(msg1FrequencyStart) parameter value.
Note:
The specific value of the msg1FrequencyStart parameter depends on the size of

the LTE PUCCH. This parameter value is chosen to close the gap between the NR
PRACH and the lower part of the LTE PUCCH.
For details, see the 5G Operating Documentation/ AirScale BTS - 5G/ 5GC001586: LTE-NR
CRS Rate Matching DSS for FDD EN-DC PSCell feature.
7.3.6 Radio unit not detected and not configured after

system upgrade
If after software upgrade, a BTS presents any of the following symptoms:
The alarm 7116: BASE STATION SERVICE PROBLEM with fault 10: No
connection to unit is raised on physically equipped radio units.
Cells not on-air and WebEM detailed site view shows radio units (RUs) that are not
detected and not configured.
Note:
This procedure can also be performed even if the BTS doesn't present any of the
symptoms above, as an additional verification after software upgrade.
Solution
Procedure
1 Connect to the BTS and log in to WebEM.

2 View the detected physical cabling using BTS Status ► Site Runtime
View ► Detailed Site View.
3 Check the detected physical cabling against the corresponding configuration in

actual site configuration file (SCF): Configuration ► Configuration
Management ► Commissioning Wizard ► Navigation
Panel ► Steps ► Radio Module ► Radio Module Link Configuration.
4 Make the necessary changes so there is a match between the physical cabling and the
actual configuration intended.
Note:
To recover quickly, until the steps above are performed, roll back to the passive
software. For details, see the Single RAN Operating Documentation/ SRAN
System/ System Upgrade and Migration/ Software rollback section.

8. System module troubleshooting
8.1 Replacing the master system module (MSM)

How to replace the MSM in case of failure.
Procedure
1 Open WebEM and click Software ► Software Update.
2 Click Backup BTS Site Configuration to back up the SCF and mark the SW
version number.
3 Power off the SBTS.
4 Disconnect the MSM cabling.
5 Connect a new master system module.
6 Restart and commission the SBTS using the saved backup SCF and upload the correct
SW version.
If the system module contains FDSW (production SW), use FDSW Rescue Console to
upgrade the module SW to SBTS SW (the same SW level as on extension). For more
information, see the Single RAN Operating Documentation / Commissioning SBTS /
Upgrading FDSW using Rescue Console.
8.2 Replacing the extension system module (ESM)

How to replace or recover the extension system module in case of failure.
The inserted core has the same software version as the primary core, with no
module role defined previously.
The hardware topology is the same as before hot removal:
The new module is taken directly after the reset of the secondary core.
The hardware topology is different from before hot removal:

Alarm for the hardware mismatch is raised.

Recommission before new hardware topology is configured. For more information, see
Single RAN Operating Documentation / Commissioning SBTS / Recommissioning.
Note:
The system module hot removal is performed while the BTS is powered up and
running.
The inserted core has the same software version as the primary core serving
as secondary core module.
If the hardware topology is the same as before hot removal, then the new module is taken
directly as target secondary core.
The inserted core does not have the same software version as the primary
core or the FDSW:
The hardware topology is the same as before hot removal:
Fault 23=EFaultId_IncompatibleSwVersionAl is raised.
Update the software version to a software version of the primary core. For more
information, see Single RAN Operating Documentation / BTS Upgrade and Migration /
Upgrading software to SBTS using WebEM.
The hardware topology is different from before hot removal:
Fault 23=EFaultId_IncompatibleSwVersionAl is raised.
Update the software version to a software version of the primary core. For more
information, see Single RAN Operating Documentation / BTS Upgrade and Migration /
Upgrading software to SBTS using WebEM.
Alarm for the hardware mismatch is raised.
Recommission before new hardware topology is configured. For more information, see
Single RAN Operating Documentation / Commissioning SBTS / Recommissioning.
8.3 Restoring system module after replacement

Follow this procedure to automatically restore system module's configuration after
replacing faulty hardware.

Purpose
The LTE1992: Flexi HW Replacement - Detect and Restore feature introduces an automated
method to restore the software and configuration on the SBTS's system module that has
been installed to replace the existing, defective hardware. Instead of restoring the SBTS
from a created configuration plan, the SBTS is restored from a plan created automatically by
NetAct. The plan is created from the SBTS's actual configuration that was previously
synchronized and stored in NetAct's configuration management CM history database.
Before you start
The LTE1992: Flexi eNB Configuration Restore feature is an HW restore feature and
supports only the same HW solutions as the previous one.
The Auto Connection Dedicated Role parameter must be set to true; otherwise,
the Prepare for configuration restore operation will fail.
When an automatic restore operation mode is selected, the date and time for the actual
configuration data in the history must be set.
The automatic restore operation will use the actual configuration data in the history to
configure the SBTS. This option sets the auto-configuration polling setting in the SON
Scheduler to active.
Procedure
1 Go to the Configuration/CM Operations Manager.
2 Select the Tools menu and select the Workflow Engine option.
Step example
Figure 44: Selecting the Workflow Engine... option
3 From the Operation List drop-down list, select the LTE1992: Flexi eNB
configuration restore option.

Step example
Figure 45: Selecting the LTE1992: Flexi eNB configuration restore option
4 Go to the CM Editor and add MRBTS instances to the workflow engine.

Drag and drop the MRBTS instance to the MO(s) field in the Workflow Engine.
Copy and paste the MRBTS instance to the MO(s) field in the Workflow Engine.
5 Select the Start arrow for the "Prepare for configuration restore" operation.
6 Optional:
In the right-hand-side part of the Workflow Engine, set parameters for the operation.
You need to fill in

Name
Description
Options, that is
auto-connection identification (auto connection hardware ID, or auto connection
site ID, or GPS identification and GPS tolerance)
automatic or manual eNB configuration restore option
history date and time (in case of automatic restore).
When the manual restore operation mode is selected, date and time must be set
in the Restore configuration manually configuration window.
Step example

Figure 46: An example of the configuration of restore operation
When GPS identification is used at auto-connection, the PREBTS GPS antenna

parameters (latitude, longitude, altitude) are automatically defined using the location
parameters of the assigned Site object (mandatory). The GPS tolerance defines the
allowed deviation (meters) between the eNB-reported GPS location and the location
defined to the PREBTS GPS antenna parameters (latitude, longitude, altitude) from the
assigned Site object.
7 Optional:
If the automatic eNB restore option is selected, it is necessary to configure SON
Scheduler.
7.1 Go to Configuration ► SON

Scheduler ► Autoconfiguration ► Settings ► Polling
mechanism ► LTE polling settings
7.2 Check the Active checkbox.
Step example
Figure 47: Configuring SON Scheduler

8 Prepare an eNB for the configuration restore by clicking Start and confirm the
operation by clicking Yes.
The feedback dialog appears, displaying the status of the operation.
8.4 Recovering the system module with the RFS tool

The Restore Factory Settings (RFS) tool restores the software to the factory condition. As
a result, configuration databases are cleaned and recreated, and the target unit is started
with a given target SW release in a non-commissioned state.
Purpose
The Restore Factory Settings (RFS) tool can be used in the following situations:
Problems during installation and commissioning (for example HW exchange)

No access via Web Element Manager
SW mismatch between different kinds of modules
Failure during SW update
Commissioning is not successful or not possible
Deletion of double-target ID
Restoring vendor certificate
Restoring local account
Restoring service account
Before you start
The restore factory setting procedure is not related to any hardware configuration.
No working Web Element Manager connection is required.
RFS recovery requires a successful ping response at one of the IP addresses:
192.168.255.1
192.168.255.5
192.168.255.7
192.168.255.16
192.168.255.119
192.168.255.127
192.168.255.129
192.168.255.131
Recovery can be started when a system module is up and running or at the very same
time a system module is started.

Typically, there is no need to remove any of the power/optical cabling in the existing BTS
setup when using a recovery tool.
Procedure
1 Download the desired BTS software from Support Portal.
Support Portal
2 On the PC, navigate to the folder where the RFS tool is installed.
3 Copy the downloaded software to the BTSSW folder.
Step example
Figure 48: Target BTS software located in the RFS tool directory
4 Connect your PC to the LMP port of the system module.
The PC's network properties must match the following settings:

IP address as 192.168.255.126
Subnet mask as 255.255.254.0
5 Optional:
Save the backup commissioning file and/or snapshot of the BTS.
6 Start the RFS tool by launching the RFSTool.exe from the restore tool root
directory, and follow the information displayed on the screen.
The restoring procedure takes from 3 to 18 minutes. Do not power off the system
module during the procedure.
7 (Optional) Delete the operator certificates from RFS Certificate Management.
The RFS tool can remove the operator certificates from the active or passive partition
to prepare the target unit to start over the Plug and Play scenario. Select the preferred
option to remove the operator certificate from the prompt.

Figure 49: Remove operator certificate
8 (Optional) Activate the passive partition (SW rollback).
To activate the passive partition you must select this option from the prompt and
execute step step 9.(Optional) Clear the BTS configuration A system module restart is
required to complete de SW rollback scenario.
Figure 50: Activate non-running partition (rollback)
9 (Optional) Clear the BTS configuration.
Select the option to clear the BTS configuration. It is not removed by default.

Figure 51: Clear BTS configuration
10 Select the desired SW from the list.
Select the target SW from the list stored in \BTSSW\ and confirm to start the
procedure.
11 Close the application and check the log file.
The system module is automatically restarted for the new SW. An additional reset can
also be needed before full SW activation is completed.
After each execution, a report file in TXT format (significant to each system module
based on the serial number and current date or time) is created. The Restore Factory
Settings (RFS) tool report is stored in C:\Temp\logs and includes, on a more detailed
level, the actions and message scenarios between the tool and the target FSM unit. The
RFS report file name is specified by the serial number, product code, current date and
time (for example, L6130705629_084792A.101__20140902_143756.txt). When
the unit identification information cannot be retrieved, meaning that access to the
system module is not available or the system module cannot provide the requested
information, the report file does not specify the serial number.
Result

Figure 52: Result view 1


After a successful recovery, the system module is set to default factory settings. If the
problem persists and the BTS cannot reach operational state, it may indicate a more severe
system module issue that cannot be recovered with a recovery tool.
After a successful recovery scenario, a ping response at 192.168.255.129 may occasionally

not be resumed. This may happen when the system module RAT is changed during the
restore procedure. An additional power cycle is required to regain access to the system
module.
Note:
NOTE: This update takes place only on the system module (not on the radio).
To make an update on a 3 + 4 configuration, it is necessary to use RFStool twice, for

FSMF and for AirScale separately.

9. Hardware monitoring and troubleshooting

This section contains use cases related to the most common hardware issues and faults.
9.1 Performing routine maintenance tasks

Follow this procedure during routine site visits to ensure that the base station remains in
a good condition.
Purpose
The Nokia BTS design itself is maintenance-free and therefore has no specified regular
maintenance intervals.
Procedure
1 Check whether the maintenance straps inside the module covers are damaged. Make
sure there are no visual defects. Replace worn or damaged maintenance straps.
2 Check the fans and air circulation.
For modules with fans, keep the fans free of leaves and debris. When cleaning the fans,
first remove the power connector from the fan or disable power to the module
(depending on the variant). For all modules, keep air inlets and outlets free of
obstructions to maintain proper air circulation and prevent overheating of the BTS.
Keep the BTS environment clean of dust.
3 Check the connector seals.
Check the connector seals visually. Replace worn or broken seals.
4 Make sure that all connector IP seals, plugs and boots are connected correctly.
The LMP port or any unpopulated connector openings have IP protection.

Figure 56: LMP IP plug installed
Figure 57: LMP port IP cap not reinstalled after maintenance
IP seals are firmly fitted around all edges of the IP boot.

Figure 58: Connector IP boot correctly installed firmly in place

Figure 59: Connector IP boot incorrectly installed (IP seals not firmly in place on any edge)
Figure 60: Connector IP boot incorrectly installed (not pushed all the way in)
Ensure that cables and connectors are fitted with IP boots.

Figure 61: Cable installed without IP boot

5 Check the antenna line and transmission cable connectors.
6 Check the module fixing screws.
Check the tightness of the module fixing screws. For RRHs, also check the tightness of
the solar shield fixing screws. Tighten all loose screws and replace all damaged or
missing screws.
9.2 Identification of faulty units

This section describes how a faulty unit can be located with the NetAct system.
Finding additional fault-related information in NetAct can be useful for getting diagnostic
info. See the explanations of selected paths below:
Table 8: Explanation of NetAct Diagnostic Info paths
Paths Meaning
<path>= /SMOD-1/FSPC Baseband module (FSP)
<path>= RF connected to FBBC module

/SMOD-1/bbext1/BBMOD-1/rf2/RMOD-3 It means that radio module RMOD-3 is connected to
optical port RF2 on BBMOD-1, which is connected to port
BBEXT1 on system module SMOD-1.
<path>= /SMOD-1/rf_ext1/SFP SFP on system module

It means that the SFP is connected to optical port
RF_EXT1 on system module SMOD-1.
<path>= /SMOD-1/opt-rf1/RMOD-2/FAN Fan on RF Module (RFM)

It means that the fan on radio module RMOD-2 is
connected to optical port OPT_RF1 on system module
SMOD-1.

Paths Meaning
RET <path>= It means that the RET is connected to port ANT1 on radio
/SMOD-1/opt_rf1/RMOD-2/ant-1/ALD-1/ module RMOD-2, which is connected to optical port
RETU-2 OPT_RF1 on system module SMOD-1.
Note:
The RET physical unit is present in the SCF as RETU

and belongs to the ALD object.
Note:
A few RETs may be connected to one port in chain

conguration. There is no possibility to indicate which
RET has failed in this case.
<path>= MHA
/SMOD-1/opt_rf1/RMOD-2/ant-4/ALD-1/ It means that MHA-1 is connected to antenna port ANT4
LNA-1 on radio module RMOD-2, which is connected to optical
port OPT_RF1 on system module SMOD-1.
Note:
The MHA physical unit is present in the SCF as LNA

and belongs to the ALD object. The MHA consists of
two LNA objects.
<path>= /SMOD-1/GNSS GPS on system module

It means that the GPS receiver is on system module
SMOD-1.
<path>= /SMOD-1/syncport/GNSS External GPS

It means that the GPS is connected to the syncport on
system module SMOD-1.
<path>= /SMOD-1/BBEXT-1/BBMOD-1/ SFP on extension baseband module

rf-2/SFP It means that the SFP is connected to optical port RF2 on
BBMOD-1, which is connected to port BBEXT1 on system
module SMOD- 1.
<path>= /FT Transport module (FT)
Note:
Suffix _R describes the unit in runtime.
Example

48 unitName=FRGT path=/SMOD_R-1(Primary)/rf_ext2/RMOD_R-1
serial_no=L1140707000 additionalFaultID=48 RMOD-2
It means that RMOD_1 is connected to the RF_EXT2 port on SMOD_R-1. The fault number is
48, serial number of the faulty unit is L1140707000, and this is RMOD-2.
9.3 Troubleshooting the transmission module

This section contains use cases related to the transmission module problems.
9.3.1 Dead peer detected (error in SCTP)

The STCP (transport path) errors and red LEDs in the transmission module can be caused
by a cable cut or no cable connection to the interface or by a switched-off port at the
connected far endpoint.
Procedure
1 Check whether the cables are connected to the correct connectors.
2 Check whether the interface at the far-end interface is switched on.
3 Check whether backhaul routers and repeaters are compatible, and whether or not
they are faulty.
4 Check if the speed and duplex setting of the BTS port is consistent with the switch
port that is connected directly to the BTS.
For example, they are both Autodetect or both 1000 Mbit/s Full duplex.
5 Check whether there are active faults at the far end.
9.3.2 Ethernet interface not working

Follow this procedure in case of issues with an Ethernet interface.
Symptoms

The configured Ethernet interface is not working (for example, a Synchronization

lost message appears)
The Ethernet interface is enabled and configured but does not carry any user plane
traffic.
Procedure
1 Check that the cables are connected to the correct connectors.
Make sure the connectors are not loose and the cables are not bent.
2 Check that the Ethernet interface is enabled.
Verify that the interface is enabled in BTS Web Element Manager. Select
Configuration ► Commissioning Wizard ► Transport
Configuration ► Physical Layer Configuration. Make sure that the
interface is enabled (the In use box is selected at the interface).
Step example
Figure 62: Launching Physical Interfaces configuration window
3 Check that the IP configuration is correct.

4 Check that there are no active faults for the Ethernet interface.
Postrequisites
Note:
The FSMF is equipped with two integrated transport interfaces: EIF1 and EIF2/RF/6.
The EIF2/RF/6 interface is SW configurable for transport purposes or for the RP3-01
interface. To operate in either (transport or RP3-01), it requires an SFP. In the SBTS,
the optical interface (EIF2/RF/6) is available as transport interface EIF2 (if not used for
RP3-01 connection).
9.4 LED indications

Most hardware components have tricolor LEDs on the front panel to indicate the
operational status and fault conditions. It is recommended to read the information on the
LED indications carefully. A blinking red LED does not always require removing the
module.
9.4.1 Nokia AirScale System Module LEDs

Description of AirScale System Module LED indicators.
9.4.1.1 AirScale system module common plug-in unit (ASIA

and ASIB) LEDs
Description of AirScale system module common plug-in-unit (ASIA and ASIB) LED
indicators.
LEDs indicate operation of the plug-in units/modules as follows:

Unit/module status LED.
All SFP and RJ45 interface status LED.
FAN status LED.
SEI interface link status LED (ASIA).
All LEDs are three-color (RED/GREEN/YELLOW) and have three possible states when lit:
stable, blinking, or alternating colors.

Table 9: LED indicators

Unit LED name LED description LED status
ASIA, ASIB STATUS Operational status of the Stable GREEN: HW online or

whole system module commissioning congured
Blinking GREEN: Commissioning Internal
SW download while congured
Stable YELLOW: HW blocked or HW
detected or commissioning
uncongured
Blinking YELLOW: Commissioning
preparing or commissioning
conguring or commissioning prepared
Stable RED: HW resetting or HW failed
Blinking RED: HW degraded
FAN Operational status of FAN Stable GREEN: all FANs working

Stable YELLOW: some faulty
Stable RED: all faulty
SEI1 Link status of System Not available

Extension interface SEI1
SEI2 Link status of System Not available

Extension interface SEI2
SRIO1 Link status of SRIO SFP+ Stable GREEN: SRIO port online
interface SRIO1 Stable YELLOW: SRIO port connecting
Stable RED: SRIO port degraded
SRIO2 (ASIA only) Link status of SRIO SFP+ Off: SRIO port failed or not used
interface SRIO2
EIF1 Link status of 1/10GE Stable GREEN: EIF online

SFP/SFP+ TRS interface Stable Yellow: SFP disconnected
EIF1 Stable RED: no link detected
Stable RED: EIF failed
EIF2 Link status of 1/10GE Blinking RED: EIF degraded
SFP/SFP+ TRS interface Off: EIF port not used
EIF2
EIF3 Link status of

100/1000Base-T TRS
interface EIF3
EIF4 Link status of

100/1000Base-T TRS
interface EIF4
EIF5 Link status of

100/1000Base-T TRS
interface EIF5
LMP Link status of Local Stable GREEN: LMP port online

Management Port Stable YELLOW: LMP port offline (LTE)
Off: LMP port not used (LTE)
Off: LMP port offline or not used
(Single RAN)

9.4.1.1.1 ASIA LED indicators

Figure 63: Location of ASIA LED indicators
ASIAA LED indicators are located in the same locations.
9.4.1.1.2 ASIB LED indicators

Figure 64: Location of ASIB LED indicators
ASIBA LED indicators are located in the same locations.
9.4.1.2 AirScale capacity extension plug-in unit (ABIA) LEDs

Description of AirScale capacity extension plug-in unit (ABIA) LED indicators.

Figure 65: Location of ABIA LED indicators
LEDs indicate operation of the plug-in units/modules as follows:

Unit/module status LED.
All SFP and RJ45 interface status LED.
FAN status LED.
SEI interface link status LED (ASIA).
All LEDs are three-color (RED/GREEN/YELLOW) and have three possible states when lit:
stable, blinking, or alternating colors.
Table 10: ABIA LED indicators

Unit LED name LED description LED status
ABIA STATUS Operational status of Stable GREEN: HW online or

the ABIA plug-in unit commissioning congured
Blinking GREEN: Commissioning
Internal SW download while
congured
Stable YELLOW: HW blocked or HW
detected or commissioning
uncongured
Blinking YELLOW: commissioning
preparing or commissioning
conguring or commissioning
prepared
Stable RED: HW resetting or HW
failed
Blinking RED: HW degraded
RF1 Link status of RF SFP+ Stable GREEN: optical cable online

interface RF1 Stable YELLOW: optical cable
disconnected
RF2 Link status of RF SFP+ Stable RED: optical cable failed
interface RF2 Blinking RED: optical cable
degraded
RF3 Link status of RF SFP+ Off: optical cable not used
interface RF3
RF4 Link status of RF SFP+

interface RF4

interface RF5

interface RF6
9.4.2 Flexi Multiradio 10 LEDs

List and description of Flexi Multiradio 10 modules LED indicators
9.4.2.1 Flexi Multiradio 10 System Module (FSMF) LEDs

List and description of Flexi Multiradio 10 System Module (FSMF) LED indicators.
The graphic and table below explain LED indications on the FSMF module.

Figure 66: LEDs of the Flexi Multiradio 10 System Module (FSMF)
Table 11: LEDs of the Flexi Multiradio 10 System Module (FSMF)
LED name LED description Color
EIF2/RF/6 RF/EXT link 6 status or • Red: no connection

transport interface 2 status • Green: connection OK
• Yellow: not in use
RF/EXT1 RF link 1 status • Red: no connection

• Green: connection OK


SRIO SRIO connection status • Red: connection is faulty/no link

detected
• Red, blinking: connection is degraded
• Yellow: SFP disconnected
• Green: connection OK/link detected
• Off: port is not in use
EIF1/TRS transport interface 1 status • Red: no connection

• Red, blinking: degraded
• Off: uncommissioned site, SW
upgrade, disabled or locked connector
FAN fan status • Red: fan fault

• Green: fan OK

STATUS system module status • Red: module self-test or reset (LED

red for < 5 seconds) or major alarm or
critical alarm
• Red, blinking: minor alarm
• Yellow: stand-by or blocked
• Yellow, blinking: SW download or
conguration ongoing, module non-
operational
• Green: module operational (the cell
can be locked in the RNC)
• Green, blinking: module is loading
software or parameters or local
maintenance access when modules are
operational
• Green and Yellow, blinking: only TRS
commissioned
9.4.2.2 Capacity extension sub-module (FBBA) LEDs

List and description of Flexi Multiradio 10 capacity extension sub-module (FBBA) LED
indicators
The graphic and the table below explain LED indications on the FBBA module.
Figure 67: LEDs of the capacity extension sub-module (FBBA)

Table 12: LEDs of the capacity extension sub-module (FBBA)
SRIO SRIO1 connection status • Red: no connection

• Off: uncommissioned or SW upgrade
STATUS operational status • Red: module self-test or reset (LED

critical alarm
operational
• Green: module operational
operational
RF/EXT RF link status • Red: no connection

9.4.2.3 Capacity extension sub-module (FBBC) LEDs

List and description of Flexi Multiradio 10 capacity extension sub-module (FBBC) LED
indicators.
The graphic and table below explain LED indications on the FBBC module.

Figure 68: LEDs of the capacity extension sub-module (FBBC)
Table 13: LEDs of the capacity extension sub-module (FBBC)
RF1/ RF2/ RF3 RF link status • Red: no connection

SRIO/RF4 RF4 link status or SRIO • Red: no connection

interface status • Green: connection OK

critical alarm
operational
operational
9.4.2.4 Flexi Multiradio 10 System Module (FSIH) LEDs

List and description of Flexi Multiradio 10 System Module (FSIH) LED indicators.

The graphic and table below explain LED indications on the FSIH module.
Figure 69: LEDs of the Flexi Multiradio 10 System Module (FSIH)
Table 14: LEDs of the Flexi Multiradio 10 System Module (FSIH)
FAN fan status • Red: fan fault

• Green: fan OK
STATUS system module status • Red: module self-test or reset (LED

critical alarm
operational
operational
RF1, RF2, RF3, RF4, RF5, RF link status • Red: no connection

RF6 • Green: connection OK
RF/EXT1, RF/EXT2, • Yellow: not in use
RF/EXT3, RF/EXT4,
RF/EXT5, RF/EXT6
SRIO1, SRIO2 SRIO1-2 connection status • Red: no connection

EIF1, EIF2 transport interface status • Red: no connection


9.4.2.5 Capacity extension sub-module (FBIH) LEDs

List and description of Flexi Multiradio 10 capacity extension sub-module (FBIH) LED
indicators.
The graphic and table below explain LED indications on the FBIH module.
Figure 70: LEDs of the capacity extension sub-module (FBIH)
Table 15: LEDs of the capacity extension sub-module (FBIH)

critical alarm
operational
operational
RF/EXT1, RF/EXT2, RF1-6 link status • Red: no connection

RF/EXT3, RF/EXT4, • Green: connection OK
RF/EXT5, RF/EXT6 • Yellow: not in use
SRIO1, SRIO2 SRIO1-2 connection status • Red: no connection


9.4.3 Fronthaul switch LEDs

List and description of Fronthaul switch (FHS) LED indicators
9.4.3.1 MDEA mobile fronthaul switch outdoor DC LEDs

List and description of MDEA mobile fronthaul switch (FHS) outdoor DC LED indicators
The figures below present LED positions in the MDEA mobile fronthaul switch (FHS) outdoor
module and the table below explains LED indications on the MDEA module.
Figure 71: LEDs of the MDEA mobile fronthaul switch outdoor DC
Table 16: LEDs of the MDEA mobile fronthaul switch outdoor DC

LED name LED LED Module state Description

description indication
STATUS Operational Stable Resetting / The module is resetting.

status of the RED Switched on
whole FHS
Stable Initialization / The module has not established
YELLOW Disconnected an optical connection yet.
Alternatin Module The highlightingDuration

g RED, Highlighting parameter of module MO is
YELLOW greater than 0.
and
GREEN
Stable Module Blocked The blocked parameter of module

YELLOW MO is true.
Stable Module Faulty The parameter status of module

RED MO is faulty.
Blinking Module The parameter status of module

RED Degraded MO is degraded.
Blinking SW Downloading The SW update procedure is

YELLOW (start-up) ongoing and the state of the
module MO is disabled.
Blinking Conguration The module has established an

YELLOW optical connection and the state
of the module MO is disabled.
Blinking SW Downloading The SW update procedure is

GREEN (normal ongoing and the state of the
operation) module MO is enabled.
Stable Working The state of the module MO is

GREEN enabled or dedicated.
FAN Operational OFF Resetting / The module agent has not taken
status of FAN Switched on control over fan.
Blinking Fan Faulty or At least one critical or major

RED Degraded alarm with Fan Supervision as a
source is active.
Blinking Fan Working No critical or major alarm with Fan

GREEN Supervision as a source is active.
EIF Link status of OFF Port not used

10GE SFP+
TRS interface
EIF

LED name LED LED Module state Description

description indication
LMP Link status of OFF Port not used

Local
Management
Port
FH1 - FH12 Link status of OFF Disabled Either of the following:

RF/SM SFP+ • The SFP module is not installed.
interface FH1 • The port has not been activated
- FH12 yet (In case of CPRI it means port
state A).
Stable Disconnected The port is enabled but link is not

RED established (In case of CPRI it
means port state B or C).
Stable Connected The port is working properly

GREEN • In case of CPRI, it means port
state D, E, F or G
• in case of OBSAI:
- RX is in FRAME_SYNC state
- TX is in FRAME_TX state
9.5 Optical fibers
9.5.1 Maintaining optical fibers

Clean fiber optic components are a requirement for quality connections between fiber
optic equipment (for example, system modules and radio units).
Inspection, cleaning, and re-inspection are critical steps that must be performed before
making any fiber optic connection.
DANGER!
Always turn off any laser sources before inspecting fiber connectors, optical
components, or bulkheads. Always wear the appropriate safety glasses when required.
Ensure laser safety glasses are used and meet current regulations. They must match
the lasers used within the environment.

Fiber test equipment

The first step in testing the fibers is visual inspection. It is important to check whether the
fiber is not overbent or physically damaged. Excessive bending of a fiber cable can cause
mechanical destruction or micro/macro-bending losses.
Note:
Do not bend optical cables beyond the minimum radius of 70 mm.
An example of a physically damaged optical cable can be seen in the figure below.
Figure 72: Physically damaged fiber
More detailed testing can be performed using proper tools. The following equipment is
required to complete a satisfactory fiber testing:
Visible Light Source – used for checking end-to-end fiber continuity
Light Source and Power Meter – used for measuring the fiber insertion loss
Fiber Scope – used for inspecting fiber cleanliness
Figure 73: Fiber test equipment
The tester can choose the equipment manufacturer, as long as the equipment can carry out
the required tests successfully and accurately.

Cleaning the fiber

Cleaning the fiber optic equipment is one of the most basic and important procedures for
maintaining high-quality connections.
Any contamination in the fiber connection can cause failure of the component or failure of
the whole system. Even microscopic dust particles can cause a variety of problems for
optical connections. A particle that partially or completely blocks the core generates strong
back reflections, which can cause instability in the laser system. Dust particles trapped
between two fiber faces can scratch the glass surfaces. Even if a particle is only situated on
the cladding or the edge of the end face, it can cause an air gap or misalignment between
the fiber cores, which can significantly degrade the optical signal.
In addition to dust, other types of contamination must also be cleaned off the end face.
Such materials include:
oils (frequently from human hands)
film residues (condensed from vapors in the air)
powdery coatings (left after water or other solvents evaporate away)
Note:
With the high-powered lasers now in use for communications systems, any
contaminant can be burned into the fiber end face if it blocks the core while the laser
is turned on. This burning may damage the optical surface enough that it cannot be
cleaned.
The fibers should be cleaned using a dry-cloth cleaner, for example Cletop.
Figure 74: Fiber cleaner
9.5.2 Cleaning optical fibers

It is important to clean every fiber prior to creating a connection. Any contamination in
the fiber connection can cause failure of the whole system.

Before you start
Make sure to have spare refill cartridges for appropriate fiber cleaning in the field. Do not
reuse old or dirty cartridges.
Procedure
1 Advance the tape belt and expose a fresh film section through the slots in the top of
the Cletop self-contained connector cleaner.
Step example
Figure 75: Example of connector cleaner
2 Hold the fiber optic connector in one hand at a 90-degree to the cassette ferrule
guide.
3 Insert the ferrule tip into the first ferrule guide hole.
4 Wipe a single stroke in the direction shown on the instruction label.
Step example
Figure 76: Example of how to clean a fiber
5 Turn the fiber optic connector ¼ turn and repeat the same operation either in the
second guide hole or on a fresh film section.

Step example
Figure 77: Example of how to clean a fiber
Note:
Do not wipe the fiber optic ferrule back and forth in the guide hole and never
wipe more than once over the same film section.
9.5.3 Fault 3030: Failure in optical interface

This fault occurs when a system module has detected an abnormal operation or a failure
in an optical RP3 interface between system module and radio unit. There are several
possible fault sources, for example, fiber optical cable, SFP, or individual module.

Fault Failure in optical interface (3030) is active.
Solution
Before the detailed troubleshooting procedure, check if the fault can be solved by:
troubleshooting other alarms first
returning to the previous state, if possible, in case an alarm appeared after changes such
as:
software updates
configuration changes
commissioning
hardware swap
resetting the radio unit and system module
If none of these actions help, perform the following steps on site, until the issue is resolved:

Procedure
1 Check the cables between the system module and radio unit.
Dirty optical connectors might cause the signal level to drop excessively. Clean the cable
end, if needed. Ensure that a minimum bending radius has not been exceeded
(excessively tight bending might damage an optical fiber).
2 Make sure that no MultiMode (MM) or SingleMode (SM) optical cable is mixed in the
configuration.
Do not mix the MM and SM optical cables or SFPs in the configuration. For a quick visual
checking at the site, see the table below.
Table 17: Visual identification of SFPs and fibers
Item SM color MM color
SFP "Locking ring" blue black
Connector housing blue gray or black
Cable ID (plain ber) yellow OM2: orange; OM3: blue-

green
3 Make sure that SFP connectors are the same as suggested in the Nokia
documentation.
4 Make sure that the same type of SFP connector is used at both ends of the cable.
5 Make sure the cables are connected correctly.
6 Replace the SFP.
7 Replace the radio.
8 Replace the system module.

9.5.4 Optical cable connection issue with FRGU, FRMF and

FXEE radios
The connection between system module and radio may be unreliable if the connector
locking mechanism is released when the connector seal is installed.

Loss of optical connection between system and radio
Solution
It is possible that the shapes of the optical cable connector, the connector seal and the unit
mechanics are incompatible. As a result, the interior of the optical cable connector seal
collides with the connector's lock-and-release latch, releasing the connector locking clasps.
Figure 78: A. Installing optical cable connector seal B. Collision between connector seal and latch.
The above scenario is applicable to a specific cable connector. This problematic cable
connector is:
provided by a vendor, which is indicated on the label of the cable with the marking V6 (for
example, with marking Lot: V6 1535)
on a straight and flexible cable
Problematic and non-problematic connectors are shown in the figure below.

Figure 79: Problematic and non-problematic connectors
To solve this issue, the following workaround is proposed:
▪ Remove the connector lock-and-release latch by using pliers.
Step example
Figure 80: Removing connector lock-and-release latch
Step result
Once the lock-and-release latch of the optical connector has been removed, the
connector can no longer be accidentally unlocked by the connector seal. However, the
connector can also no longer be unlocked and removed from the SFP module by simply
pressing the lock-and-release latch – it can only be released by carefully pressing down
the two locking clasps of the connector, for example, with a small screwdriver.

9.5.5 CPRI-A radio issues in a Nokia network

There are certain permanent differences in the way CPRI-A radios act in a Nokia network
in comparison to an Alcatel network. These differences and potential issues are explained
below.
SFP swapping
After swapping the SFP connectors, the SFP monitoring tool continues to show the data of
the previous SFP instead of the new one. To update the monitoring tool, a reset of the radio
unit is required.
The affected radios are:

NG2 family (UHID, UHLA, UHMA, UHIA)
NG3 family (UHHA, UHFB), OEM1 (UNBC, UHBB)
MH Asset (UHFA, UHBA)
Optical cable swapping between SBTSs

When moving radios between BTSs, the operator must wait for 30 seconds after
disconnecting a CPRI link, before inserting the CPRI link in the target BTS. Otherwise, an
additional 30 seconds of outage will occur on the moved radio, and two minutes 30 seconds
of outage will occur on another radio in the target SBTS.
All CPRI-A radios are affected.
Optical cable swapping in the same SBTSs

When moving radios between different ports on the same SBTS, the operator must wait for
30 seconds after disconnecting a CPRI link, before inserting the CPRI links into the target
ports. Otherwise, the moved radios will go off the air and remain in an off-the-air state until
a site reset.
All CPRI-A radios are affected.
ACF file download

In case of a failed ACF file download, the operator must execute a RET calibration.
Otherwise, the RET remains uncalibrated after the ACF download.

The affected radios are UHFB and UHED.
Radio alarm rate

For the affected radios it is possible, but very unlikely, that alarm events which exceed a rate
of 10 events per 5 seconds from one radio may not be reported in the alarm event stream
to the BTS.
The affected radios are: UHLB, UHFD, UHLC, UHFF, UHCB.
Radio LEDs
The LED indications on certain radios do not follow the Nokia LED indication standards.
However, the graphical representation of LEDs in the BTS Web Element Manager follows the
Nokia standards.
The affected radios are: UHMB, UHHB, UHAA, UHED, UHEE.
RX blocking
The UHIA and UHID radios fail the RX Blocking Sensitive Requirement by around 2
dB. If the blocker differs from specified blocking level by more than +/- 2 dB, then sensitivity
meets the requirement.
The affected radios are: UHIA and UHID.
Low voltage alarm

The UHFB and UHHA radios do not report a low voltage alarm for low-voltage scenarios.
Overcurrent alarm generation

The UHED ALD Overcurrent Behavior, including radio protection, is normal, except that the
radio does not generate an Overcurrent Alarm.
9.6 Fault 4078 triggers temperature check for AMOB

AirScale subrack
Fault 4078: Cooling fan broken was updated to check the temperature if the fan is

broken, to avoid potential site visits or even unnecessary hardware replacements. Fault
4080: Cooling fan is over speeding is triggered when the fan's maximum
rotation per minute value reaches 9960.
Before SRAN 20C release, fault 4078: Cooling fan broken was raised on the AMOB
Airscale subrack, even though the temperature was in the defined threshold and the fans
were still working. This was due to incorrectly reported fan speed for the AMOB Airscale
subrack, caused by pin short hardware issues during installation.
To solve this issue, the AMOB AirScale subrack pulse width modulation (PWM) frequency that
controls the fan speed is changed from 5 kHz to 1 kHz.
To avoid potential site visits or even unnecessary hardware replacements, fault 4078 was
updated to check the AMOB AirScale subrack temperature if the fan is suspected to be
broken. In this case, fault 4078 is raised only when the fan speed is equal to 0 and the
temperature over 80 °C.
If the fans rotate full speed and produce a loud noise on site, fault 4080: Cooling fan
is over speeding is raised on the BTS. For AMOB Airscale subrack, fault 4080 is raised if
the fan's maximum rotation per minute value reaches 9960.

10. Related troubleshooting documents

List of documents that include SBTS, NetAct, DCAP and PTF troubleshooting.
Additional SRAN troubleshooting documents:

For a collection of troubleshooting features, see the SRAN Operating Documentation /
Single RAN Features / LTE and SRAN Operability Features / Monitoring and
troubleshooting features.
For SBTS performance measurements, see the Single RAN Operating Documentation /
SRAN Reference Data/Performance Measurements.
For SBTS faults and alarms, see the Single RAN Operating Documentation / SRAN
Reference Data/Alarms and Faults.
For SBTS parameters, see the Single RAN Operating Documentation/SRAN Reference
Data/Parameters.
For troubleshooting faults, alarms and sub-string based snapshot collections, see the
Single RAN Operating Documentation/Automatic BTS Log Collection document.
For the sleeping cell functionality, see the Single RAN Operating
Documentation/Automatic Sleeping Cell Detection and Recovery document.
For cell, subscriber and trace information, see the Single RAN Operating
Documentation/Tracing in LTE and SRAN document.
For RF sharing troubleshooting, see the Single RAN Operating Documentation /
Troubleshooting Classical RF Sharing document.
For commissioning and recommissioning procedures, see the Single RAN Operating
Documentation/Commissioning SBTS document.
For BTS upgrade an migration, see the Single RAN Operating Documentation/BTS
Upgrade and Migration document.
For SBTS system upgrade and migration, see the Single RAN Operating
Documentation/SRAN System Upgrade and Migration document.
Additional NetAct troubleshooting documents:

NetAct Operating Documentation/Troubleshooting NetAct
NetAct Operating Documentation/NetAct Logging References
NetAct Operating Documentation/NetAct Troubleshooting Tools
Additional DCAP troubleshooting documentation for Windows:

DCAP Operating Documentation/DCAP Central Monitoring Solution
DCAP Operating Documentation/DCAP GEO Interface
DCAP Operating Documentation/DCAP IMSI Provider

DCAP Operating Documentation/DCAP IMSIMon3G

DCAP Operating Documentation/DCAP L3 Analyser/Viewer (Emil)
DCAP Operating Documentation/DCAP L3 Data Collector (Megamon)
DCAP Operating Documentation/DCAP L3 Data Collector for Traffica
DCAP Operating Documentation/DCAP NetAct OandM Agent
DCAP Operating Documentation/DCAP Troubleshooting Server
Additional DCAP troubleshooting documentation for Linux:

DCAP Operating Documentation/Nokia Data Collection and Analytics Platforms
Troubleshooting Guide
Additional PTF troubleshooting documentation:

PTF Operating Documentation

11. Glossary of terms

List of definitions related to the troubleshooting process.
Commission A sequence of tasks (including parameter and software conguration,

tests and reporting) to ensure that the stand-alone equipment in
question has been properly installed and congured, has no faulty
modules and is ready for integration. Commissioning is performed on
new equipment or as a nal migration phase.
Drive test A manual method that allows the operator to gather information
about the network in a particular area.
Fallback Function that automatically restores and brings into use the previous
conguration of a system.
InfoModel snapshot le Used for BTS troubleshooting purposes and contains a full BTS state
(IMS2) snapshot. The IMS2 le can be loaded to WebEM to view the BTS
state and its conguration the same way as through an actual
connection to the BTS. The InfoModel snapshot contains full BTS
runtime data for a certain period of time, making it possible to
analyze BTS behavior during that period.
Migration to SRAN A sequence of tasks including software upgrade and commissioning

to convert an existing BTS into an SBTS.
Plug and Play (PnP) A hardware or software that, after being installed ("plugged in"), can
immediately be used ("played with"), as opposed to hardware or
software which requires conguration. In plug-and-play, the software
or hardware is able to recognize the introduction of a new entity and
make use of its services, or provide services to the new entity without
human intervention.
Recommission The current conguration is overwritten by a new one.
Reset the software to When the module cannot boot up either from an active or a passive
failsafe partition partition or when problems appear after a Plug and Play procedure,
resulting in incorrect software download.

Rollback Function that returns the values dening the characteristics of a site
(like IP addresses and radio network parameters related to each RAT)
to a point before the previous SW upgrade. When rollback is
performed, the previous software version is restored and the
previous conguration of the system is brought into use.
Site conguration le (SCF) A le in the XML format that contains site characteristics, for example
site ID, IP addresses, and radio network parameters related to each
RAT, RAT independent conguration parameters for management
layer, transport, and security.
Snapshot A le that can be saved in connected mode on WebEM tool, and it
contains the current status of elements, such as: used hardware
conguration, logs, alarms, HW and SW version information. Unlike the
IMS2 le, the snapshot contains only the latest information, without
the history.
SRAN conguration Building a conguration is a exible procedure that consists of only

two steps: creating a baseband (BB) conguration and an RF
conguration. The user can choose the target RF conguration, RF
cell sets and BB cell set based on the available capacity and ports for
radio connections (per RAT).
For general rules for creating the supported SBTS congurations, see
SRAN Operating Documentation / Single RAN BTS Site Solutions /
SBTS Supported Configurations.
Software update Software update can be performed to add some new features to
better support customer needs, to replace software with bugs, or to
guarantee compatibility with other element types due to their own
software upgrade.
Syslog Also known as tracelog, this is system information that is sent out
using specic channels, like consoles, les or the network. This well-
prepared data contains status information, current settings or code-
path information. The system under test is aware of the logging
activity.
System upgrade A process that involves downloading and activating a new software
release on a system, as well as converting the existing system
conguration to be compatible with the new software release.
Trace Similar to log, but the system under test is not aware of the tracing
activity. For example, using Wireshark or Emil for examining a network
interface can give all the information the application sends through
that interface, without having tracing awareness.

Uncommission The current conguration is cleared either manually or by using

WebEM tool.

Troubleshooting SBTS

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Troubleshooting SBTS

Uploaded by

Copyright:

Available Formats

Single RAN System, SRAN 20C,

The information in this document applies solely to the hardware/software product

2 © 2021 Nokia.Nokia confidential

ERRORS IN OR OMISSIONS FROM THIS DOCUMENT OR ITS CONTENT.

Nokia is a registered trademark of Nokia Corporation. Other product names mentioned in

Copyright © 2021 Nokia. Nokia condential.

Important Notice on Product Safety

© 2021 Nokia.Nokia confidential 3

4 © 2021 Nokia.Nokia confidential

Summary of changes .................................................................................................................. 15

1 Troubleshooting workflow ......................................................................................................... 17

2 Troubleshooting tools ............................................................................................................... 23

© 2021 Nokia.Nokia confidential 5

2.1.4.14 TWAMP RTT Measurements .................................................................. 49

3 Collecting and analyzing troubleshooting data ...................................................................... 79

6 © 2021 Nokia.Nokia confidential

4 Fault monitoring ......................................................................................................................... 94

5 Radio unit monitoring and troubleshooting ........................................................................... 99

6 RAT specific troubleshooting ................................................................................................. 109

© 2021 Nokia.Nokia confidential 7

6.2.1.1 RNC logs ................................................................................................... 112

7 Software and synchronization troubleshooting .................................................................. 139

8 © 2021 Nokia.Nokia confidential

7.2.2 Fallback solution for phase synchronization ................................................. 150

8 System module troubleshooting ........................................................................................... 159

9 Hardware monitoring and troubleshooting .......................................................................... 171

© 2021 Nokia.Nokia confidential 9

10 Related troubleshooting documents .................................................................................. 203

11 Glossary of terms ................................................................................................................... 205

10 © 2021 Nokia.Nokia confidential

© 2021 Nokia.Nokia confidential 11

Figure 40 Table of parameters ................................................................................................ 114

12 © 2021 Nokia.Nokia confidential

Figure 80 Removing connector lock-and-release latch ....................................................... 199

© 2021 Nokia.Nokia confidential 13

14 © 2021 Nokia.Nokia confidential

DSS cell degraded after system upgrade

A new section on how to configure the NRCELL PRACH Frequency start

Nokia CPRI radio not detected after BTS software upgrade

A procedure on how to configure the CABLINK Link speed (linkspeed) parameter

Fault 4078 triggers temperature check for AMOB AirScale subrack

Fault 23: Incompatible SW version detected is raised after software fallback

A new section on fault 23: Incompatible SW version detected detected during

© 2021 Nokia.Nokia confidential 15

Keeping legacy cell set in use after system upgrade

Changes between issues 05 (2020-09-16, SRAN 20C) and 05A (2020-11-20,

A new section on how to avoid triggering fault 110 was added.

Fault 10 raised on one BTS after successful software upgrade

16 © 2021 Nokia.Nokia confidential

Figure 1: Troubleshooting methodology

1.1 How to contact Nokia technical support

© 2021 Nokia.Nokia confidential 17

The support ticket can be created in two ways:

Support ticket classification based on TL9000 severity

Critical tickets are created for:

Major tickets are created for:

Minor tickets are created for:

1.1.1 How to open a support ticket by phone or email

18 © 2021 Nokia.Nokia confidential

Figure 2: Emergency contact field

1.1.2 How to open a support ticket online

2 Go to Support tab ► Support Tickets ► Create Software Support

© 2021 Nokia.Nokia confidential 19