WiMAX DT&CQT Test Guide

For internal use only

Product Name

Confidentiality

WiMAX RNP

For internal use only

Product Version

Total 46 pages

1.0

WiMAX DT&CQT Test Guide

(For internal use only)

Prepared by

Date

Liu Rong

Reviewed by

Date

Reviewed by

Date

Approved by

Date

2008-10-30

Huawei Technologies Co., Ltd.

All Rights Reserved

2015-12-2

Huawei Confidential

Page 1 of 46

WiMAX DT&CQT Test Guide

For internal use only

Revision History
Date

Version

Description

Reviewer

Author

2008-11-05

1.00

The initial draft is

modified according to
review comments.

Meng Xiaojing, Wang Hui,

Chen Dehai, Jiang Bo,
Yang Fan

Liu Rong

2015-12-2

Huawei Confidential

Page 2 of 46

WiMAX DT&CQT Test GuideKeywords

Drive Test (DT), Call Quality Test (CQT), single base station (BS) verification test, radio
frequency (RF) optimization test, service optimization test, acceptance test

Abstract
This document describes how to perform DTs and CQTs. The involved test items include the
Ping test, FTP test, and HTTP test. The involved scenarios are as follows:

Single BS verification test

RF optimization test

Service optimization test

Acceptance test

Benchmark test.

Acronyms and Abbreviations

Abbreviati
on

Full Spelling

WiMAX

Worldwide Interoperability for Microwave Access

CQT

Call Quality Test

DT

Driver Test

VoIP

Voice over IP

GW

Gate Way

AMC

Adaptive modulation and coding

MIMO

Multiple input multiple output

MOS

Mean Opinion Score

For internal use only

Overview

Testing is required for collecting basic data in the full process of the network planning and
optimization service. The test lifecycle begins when network planning is started and ends
when network optimization is complete. Testing is classified into the DT and the CQT by test
mode. It is also classified into the following types by test scenario:

Single BS verification test

RF optimization test

Service optimization test

Acceptance test

Benchmark test

This document describes the processes, test items, and precautions for WiMAX DTs and
CQTs with the Huawei GENEX Probe used.
This document is intended to:

Provide WiMAX test guides for test engineers.

Ensure validation of test results.

Ensure that testing provides an appropriate basis for further troubleshooting and in-depth
problem analysis.

The primary sections of this document are chapter 2"DT and CQT" and chapter 2 " Tests in
Different Scenarios".
Figure 1.1 shows the structure of this document.

2015-12-2

Huawei Confidential

Page 5 of 46

For internal use only

Figure 1.1 Document structure

Chapter 1 describes the document structure.

Chapter 2 describes test items of WiMAX DTs and CQTs that are performed with the GENEX
Probe. The following information is described:

Preparations, procedures, and scenarios for each test item

Preparations for each service test

Standardized test operations

Chapter 3 describes the following DT and CQT test scenarios:

Single BS verification test

RF optimization test

Service optimization test

Acceptance test

Benchmark test

Test requirements vary with scenarios.

Chapter 3 also describes all baseline key performance indexes (KPIs) in DTs and CQTs for
service optimization, including the following information:

Formulas and definitions

Signaling measurement points

Test conditions

Test and measurement methods

Chapter 4 summarizes the whole document.

Chapter 5 lists all templates and appendixes used in this document.
Chapter 6 lists appendixes and describes related signaling flows and evaluation methods.

2015-12-2

Huawei Confidential

Page 6 of 46

For internal use only

DT and CQT

2.1 Test Preparations

2.1.1 Selecting Test Tools
A test suite often consists of the units shown in Figure 1.1.
Figure 1.1 Test tool buildup

High performance portable personal computer (PC) with sufficient space on the hard
disk
Ensure that the PC can work stably during tests.

HASP for test software

Huawei now uses the Couei tool or the Huawei GENEX Probe for WiMAX tests. The
GENEX Probe is preferred. If the GENEX Probe cannot meet the test requirements in a
project, other appropriate software can be used.

2015-12-2

WiMAX test terminals and data lines that meet the protocol standards and that are
compatible with the test software

Huawei Confidential

Page 7 of 46

For internal use only

Various types of WiMAX test terminals that support different performance are available.
The test terminals are described in detail in specialized guides.

Scanner

Global Positioning System (GPS)

Huawei now uses GARMIN series GPS. If the output mode is set to NMEA, the test data
is counted once every two seconds. If the output mode is set to TEXT, the test data is
counted once every second. The TEXT mode, however, is supported only by the latest
GPS version. To ensure accuracy of tests, the GPS must lock at least four satellites.

Car-mount power inverter whose power ranges from 360 W to 500 W

2.1.2 Preparing the Test Environment

Check the test environment before a test to prevent the test from being impacted by incorrect
environment settings. Test environment check involves the following aspects:

Check network devices.

Ensure that the network devices such as the BTSs and transmission devices in the area to
be tested work normally without any major alarm. Ensure that the settings of all BTSs
and gateways (GWs) in the area are correct and valid.

Check capability of the test terminals.

Ensure that the functions such as power control, Automatic Modulation Control (AMC),
and Multiple Input Multiple Output (MIMO) to be tested are enabled in the system.
Ensure that the test terminals support testing of the related functions.

Check the account for testing is valid.

Ensure that the account information for testing is correct and valid. Ensure that the test
results are not impacted by authentication or roaming restriction.

For an FTP throughput test, locate the FTP server near the GW to prevent test results
from being impacted by delay, bandwidth, and other factors beyond the optimization
scope. Ensure that the transmission performance between the FTP server and the network
meets the test requirements. Prepare several FTP servers if necessary. If the FTP
throughput is found unsatisfactory in a test, replace the FTP server and then test the FTP
throughput again. Compare the test results to ensure that the poor FTP throughput is not
caused by the FTP server.

For an HTTP test, use a website for which the server works stably. Use the service
homepage of the related operator whenever possible. For example, use the Monternet
homepage for an HTTP test of China Mobile. If an operator does not provide a
homepage, you are advised to use an internationally recognized Web server.

For a Voice over IP (VoIP) test of an operator, the called number should also be provided
by the same operator. If the called number is provided by another operator, the test result
might be impacted by problems in interconnection between two operators.

2.1.3 Collecting Network Data

Collect the following data and then import the data to the test devices:

Data (such as engineering parameters and cell data) of the network to be tested

Maps (digital maps that can be imported to the DT software and related hard-copy
administrative maps)

Digital maps are used to associate the route recorded in the GPS and all information obtained
in a test with geographic locations for the test and later analysis. In this way, accuracy and
integrity of tests are ensured according to the geographic conditions such as landforms,
2015-12-2

Huawei Confidential

Page 8 of 46

For internal use only

directions of streets, and locations. To use digital maps in tests, import files with extension
tab from the Mapinfo. If no digital map is available, scan related hard-copy maps and then
make digital raster graphics by using the Mapinfo.
In a test, site engineering parameters enable the test software to show from which cell the
current call is made, among which BSs the call is located, and whether the appropriate serving
cell is used. Different DT software uses different methods to import site engineering
parameters. For specific import method, see the online help, operation guide, and import
template of the software. All DT software, however, imports the same site engineering
parameters as follows:

BS name

Cell name

BS ID

Cell longitude

Cell latitude

Antenna azimuth

Frequency point

Different DT software supports different formats of engineering parameters and maps. For
details, see operation guides of related DT software.

2.1.4 Preparing a Test Car

In a test, the test PCs, test terminals, and GPS require power supply. Both PCs and terminals
can use batteries. Batteries, however, often fail to meet requirements of long-time tests. Figure
1.1 shows the recommended car-mount power supply.
Figure 1.1 Car-mount power supply
Car cigarette lighter

12 V DC power

Car batteries
Inverter

Power socket
220 V AC power

Check the car-mount power supply system before a test:

2015-12-2

Ensure that the car-mount power supply system meets the test requirements. If the
cigarette lighter needs to be used, ensure that it can work stably. If the cigarette lighter
does not work, replace a test car.

Huawei Confidential

Page 9 of 46

For internal use only

Ensure that the car-mount power supply system (inverter) works. Note that the maximum
power supplied by an inverter is limited. An inverter often supports simultaneous power
supply for up to two PCs. If an inverter is used to power three or more devices, the
inverter reports alarms and fails to work at last. For this reason, ensure that all PCs and
test terminals are fully charged before tests. During tests, charge PCs and terminals by
turns.

Before a test, inform the driver of the approximate mileage required by the test. Ensure that sufficient
fuel is available.

2.1.5 Preparing a Test Plan

Test engineers should prepare for a test from the following aspects:

What
What is the test type (DT or CQT)? What items does the test cover?

Where
Are test route and points defined? Test routes and points depend on test scenarios. For
details, see the principles for selecting test routes and points in different scenarios.

When
Is the test schedule defined? Start time and end time are planned for each test. Contact
related personnel for the test schedule before a test.

Who
Who are appointed for the test? Obtain contact information on test engineers, support
engineers in the OMC equipment room, and technical support engineers.

How
How to perform the test? What is the required driving speed? How to deal with
emergencies in the test? How to name test files? How to fill out test information records?
For details, see DT and CQT operations.

2.2 Test Items

This section describes how to use the GENEX Probe for each test item. Test items vary with
projects.

2015-12-2

Huawei Confidential

Page 10 of 46

For internal use only

2.2.1 Performing a Ping test

Configure the test data as follows:

2015-12-2

Scenario Name: Set the name of the test task.

Repeat: Define how many times the test task is performed.

Test Item: Select the test item. Select Ping here.

IP: Set the destination IP address.

Packet Size (Byte): Set the size of data packets.

Fragment Flag: Enable or disable packet fragmentation.

Test Control Mode: Set the test control mode. Set it to Test By Count by default.

Test Count: Define how many times the ping test is performed. Set it to 50 or a smaller
value.

Test Interval (S): Set intervals for the ping test. Set the interval to 120 seconds or a small
value.

Huawei Confidential

Page 11 of 46

For internal use only

2.2.2 Testing FTP Upload

Configure the test data as follows:

2015-12-2

Scenario Name: Set the name of the test task.

Repeat: Define how many times the test task is performed.

Test Item: Select the test item. Select FTP Upload here.

Port: Select the FTP server port.

User: Set the user name.

Password: Set password of the user name.

Remote File: Set the directory for saving the uploaded file. For example, set it to
ftp://172.20.14.10/UploadTest.

Local File: Set the name of the local file to be uploaded.

Test Count: Define how many times the file is uploaded. Set it to 50 or a smaller value.

Test Interval (S): Set intervals for file upload. Set the interval to 120 seconds or a small
value.

Disconnection Mode: Break down or keep the PS connection after the target file is
uploaded.

Huawei Confidential

Page 12 of 46

For internal use only

2.2.3 Testing FTP Download

Configure the test data as follows:

2015-12-2

Scenario Name: Set the name of the test task.

Repeat: Define how many times the test task is performed.

Test Item: Select the test item. Select FTP Download here.

Port: Set the port of the target host. Set it to 21 by default.

User: Set the user name used to access the target host.

Password: Set the password of the user name that is used to access the target host.

Remote File: Set the directory where the target file is saved on the host. The directory
format is as follows: ftp://IP address/file name (file path included). For example, set the
directory to ftp://10.0.0.1/download/test.dat.

Write Local File: Set the local directory for saving the downloaded files. For example,
set it to c:\test.dat.

Test Interval (S): Set intervals for file download. Set the interval to 120 seconds or a
small value.

Test Count: Define how many times the file is downloaded. Set it to 50 or a smaller
value.

Multi Task Count: Set the number of threads in the case of multi-thread download. Set
this parameter to 9 or a smaller value.

Disconnection Mode: Break down or keep the PS connection after the target file is
downloaded.

Huawei Confidential

Page 13 of 46

For internal use only

2.2.4 Performing an HTTP Test

Configure the test data as follows:

2015-12-2

Scenario Name: Set the name of the test task.

Repeat: Define how many times the test task is performed.

Test Item: Select the test item. Select Http here.

URL: Specify the Universal Resource Locator (URL) of the website to be tested.

Port: Specify the port number.

Test Interval (S): Set intervals for the HTTP test. Set the interval to 120 seconds or a
small value.

Test Count: Define how many times the HTTP test is performed. Set it to 50 or a smaller
value.

Huawei Confidential

Page 14 of 46

For internal use only

2.2.5 Performing a Wait Test

Configure the test data as follows:

2015-12-2

Scenario Name: Set the test name.

Repeat: Define how many times the test task is performed.

Test Item: Select the test item. Select Wait here.

Wait Duration (s): Specify the test duration.

Huawei Confidential

Page 15 of 46

For internal use only

2.2.6 Performing a NetEntry Test

Configure the test data as follows:

2015-12-2

Scenario Name: Set the test name.

Repeat: Define how many times the test is performed.

Test Item: Select the test item. Select NetEntry here.

Test Interval (S): Specify the test duration.

Huawei Confidential

Page 16 of 46

For internal use only

2.2.7 Performing a NetDisconnect Test

Configure the test data as follows:

Scenario Name: Set the test name.

Repeat: Define how many times the test is performed.

Test Item: Select the test item. Select NetDisconnect here.

Test Interval (S): Specify the test duration.

The NetEntry test and the NetDisconnect test must be performed in pairs. In addition, the NetEntry test
must be performed before the NetDisconnect test.

2.3 DT Operations
2.3.1 Operations Before a DT
Determining DT Method and DT Personnel
Determine the following information according to test requirements and purposes:

2015-12-2

Huawei Confidential

Page 17 of 46

For internal use only

Appropriate test method

Test engineers and related support personnel

Test area and route

Test tools

The current test methods come from pilot deployment and small-scale commercial
deployment. Test methods for large-scale commercial deployment are under discussion. If a
customer disapproves the test method proposed by Huawei, perform a pretest by using the test
method required by the customer. Revise the test method according to the test conditions and
the customers comments. At the same time, try to persuade the customer to accept the test
method proposed by Huawei.

Checking and Preparing DT Devices

Ensure that the hardware of the DT devices is in good condition.

Ensure that the all accessories of the DT devices are in good condition.

Ensure that the DT devices are equipped with power supply and batteries.

Ensure that the software of the DT devices is properly installed and used.

Operate the DT devices before each DT to verify availability of the devices. If a DT

device does not work after power-on, check the following factors:

Check whether the DT device is properly powered on, whether all switches are on,
and whether the related indicator shows that the device is powered on.

Check whether the data cable is well connected to the DT device.

Check whether the serial port is connected to the specified position.

Check whether the GPS information is received and whether the antenna is located in
an appropriate place.

Check whether the port is properly configured in the operating system (OS).

Check whether the license is valid.

Selecting a DT Route
Comply with the following principles when selecting a DT route:

2015-12-2

For a network acceptance test, select a route where signals are strong. Keep the test away
from the uncovered areas caused by objective factors such as the network topology. If
the test route has to pass through an uncovered area, negotiate with the customer to
confirm that the related test results are omitted if the signal level is lower than a specified
value.

For a pretest before service provisioning, select a route that passes through all landforms
in the tested area, including overhead roads, tunnels, expressways, and densely populated
urban streets. For two-way roads, perform the test in both directions to avoid outstanding
problems.

For a network optimization test, determine a reasonable test solution before network
optimization. Specify fixed start and end points. Keep the test route and driving direction
the same before and after optimization.

For network performance monitoring tests, choose a new route and driving direction for
every test. Simulate different subscriber behaviors to discover network problems in time.

Communicate with the customer before selecting and determining a test route. Ensure
that the test route contains the locations that the customer is concerned about.

Huawei Confidential

Page 18 of 46

For internal use only

Determining DT Time
Determine DT time according to the specific test requirements and purposes. For example,
perform a DT for service problem diagnosis in busy hours and perform single BS verification
DT in idle hours. If a customer specifies the DT time, perform the DT at the specified time.

2.3.2 Operations During a DT

Placing the Antenna of Test Terminals
Three types of test terminals are available for WiMAX testing: Customer Premises Equipment
(CPE), PCMCIA card, and USB Dongle. Keep the antennas of test terminals away from
unnecessary block or attenuation.

If the CPE is used, fix its outdoor omni-directional antenna on the top of the car.

If a PCMCIA card is used, erect its external antenna.

If a USB Dongle is used, place it vertically.

Signal loss caused by the car body ranges from 3 dB to 5 dB. To reduce the loss, you are
advised to connect the USB Dongle with a USB extension cable and then vertically fix the
USB Dongle on the car enclosure. See Figure 1.1.
Figure 1.1 Position of a DT terminal

Do not place a test terminal at will.

Monitoring Test Status

During a test, keep an eye on working status of the test system. If the test system is
exceptional, request the driver to stop the car on the side of the road. Repair the test system,
and then review the test log file of the last section to determine the section to be re-tested.
2015-12-2

Huawei Confidential

Page 19 of 46

For internal use only

Pay attention to the driving speed during a test. The suggested maximum speed is 40 km/h.
Remind the driver to slow down if the speed exceeds the specified maximum value.

Recording Test Files

To facilitate later data analysis, name test files according to the related naming rules. The
name of a DT test file often contains the test scope, sequence number of the test file, test time,
and test mode. For example, a test file is named as Johannesburg Woodmead01FTPdownload-20081015.
To prevent irrelevant problems from impacting the test data, record emergencies such as
traffic jam and power failures of the test terminals during a test. To facilitate later data
analysis, record the major problems listed in the log files, for example, disconnection from the
network, frequent handoff (HO) failures, and low download rate.

2.3.3 Operations After a Test

After a DT, submit the test log files and the Radio Network DT Information Record to the data
analyzers according to the test requirements and purposes. If a customer has special
requirements on test data, comply with the requirements.

2.4 CQT Operations

2.4.1 Operations Before a Test
The CQT plays an important role in WiMAX testing because WiMAX applies mainly to
nomadic data subscribers. Besides a large number of DTs, CQTs are also required in network
optimization projects. CQT preparations are similar to DT preparations. Communicate with
the related customer before optimization and create a CQT test point list. The list often covers
big hotels, business centers, and VIP subscribers in the network coverage.
The list should also cover some standby test points. If some CQT test points fail in the
acceptance due to network topology or other objective factors, persuade the customer to
replace the failed test points with the standby ones.

Huawei is not responsible for CQTs beyond the network acceptance test points. Huawei network
optimization engineers can provide customers with necessary technical guide and training and define the
proximate scope of service provisioning. The specific test tasks, however, should be accomplished by
customers engineers. If a customer requests Huawei network optimization engineers to perform the
tests, the customer should pay Huawei for the tests.

2.4.2 Operations During a Test

Comply with the following principles in a CQT:

2015-12-2

Perform optimization and acceptance tests in areas with strong signals. Use highly
sensitive test terminals that work stably. Do not perform tests in high buildings because
the radio environment is often complicated there. In high buildings, the quality of signals
is not guaranteed even if the signals are strong. This problem is discovered in GSM,
Huawei Confidential

Page 20 of 46

For internal use only

CDMA, and WCDMA networks. To WiMAX networks that have fewer frequency points,
the problem must be more severe.

For a service provisioning test, use the terminals that the customer uses for service
provisioning. Take both typicality and universality into consideration when selecting test
points. To support typicality, choose traffic hot spots such as business centers and VIP
subscribers as test points. To support universality, distribute test points evenly in the
tested area. Divide high buildings into three parts (high, medium, and low) for testing. To
obtain test data that reflects subscribers' perception, perform tests where subscribers
often use services.

CQT tests generate a large number of test files. Name CQT test files according to the
related naming rules. The name of a test file should contain the test point, test mode, test
time, test direction. For example, a test file is named as Build28Woodlands-First FloorSouthWindow-FTPdownload-20081015. During a CQT, fill out the CQT Record in
which the items can be modified according to actual project conditions.

2.4.3 Operations After a Test

After a CQT, process the test data and output test documents such as XX Project Service
Provisioning Proposal according to the CQT requirements and purposes. If a customer has
special requirements on test data, comply with the requirements.

2015-12-2

Huawei Confidential

Page 21 of 46

For internal use only

Tests in Different Scenarios

3.1 Single BS Verification Test

Single BS verification is used to check whether device functions are normal. It lays a good
foundation for later RF and service optimization. Single BS verification tests verify and solve
problems relating to functions and engineering installation of network devices. Product
support engineers and engineering installation engineers should actively assist in single BS
verification tests.
Determine test items of single BS verification with operators according to actual network
conditions. Ensure that test items can help to effectively discover problems and meet schedule
requirements.
Single BS verification is used to check device functions. For this reason, ensure that the test
results are not impacted by other factors such as the radio environment and other signals in
the network. Comply with the following principles when performing a single BS verification
test:

Ensure that the DT route is in the main coverage of the tested cell and keep Line-of-sight
(LOS) transmission if possible. Ensure that signals are strong enough to avoid an
inaccurate test result due to a poor radio environment.

Select CQT points within 100 meters around the BS and keep LOS transmission. Huawei
often requires that the Carrier to Interference + Noise Ratio (CINR) should be at least 25
dB and that the Received Signal Strength Indicator (RSSI) should be at least 55 dBm.
The tested CINR and RSSI values, however, vary with terminal type. Therefore, the
specific value ranges depend on actual conditions.

For detailed test procedures and methods, see WiMAX Single BS Verification Guide.

3.2 RF Optimization Test

3.2.1 Test Procedure
As a stage of network optimization, RF optimization aims to optimize RF signals. In this way,
signal coverage is improved to ensure appropriate distribution of radio signals in later service
optimization.

2015-12-2

Huawei Confidential

Page 22 of 46

For internal use only

DTs provide data for RF optimization. RF optimization refers to a process of Benchmark test
analysis. Initial DT data is provided as inputs of RF optimization analysis. After RF
optimization, another DT is required. Compare results of the post-optimization DT with those
of the initial DT, and then output an RF Optimization Benchmark Report. Figure 1.1 shows
the test procedure.
Figure 1.1 Procedure for an RF optimization test

Note that the DT route after RF optimization might slightly differ from the initial DT route.
Check whether the DT route changes before each DT.

3.2.2 Test Route

Initial DT Route
Perform RF optimization by cluster. For rules for dividing a network into clusters, see the
WiMAX RF Optimization Guide. Specify a test route or test points in each cluster.

Comply with the following principles when selecting a DT route:

Select a route that passes through all landforms in the tested cluster, including
overhead roads, tunnels, expressways, and densely populated urban streets. For twoway roads, perform the test in both directions to avoid outstanding problems.

To guarantee basic effect of RF optimization, select a test route that covers all cells if
possible.

Pay attention to restrictions such as one-way roads and left turn forbidden signs when
selecting a test route. Consult local drivers or drive along the planned test route to
ensure that the route is acceptable before confirming the route with the customer.

For an RF optimization report to be submitted to the customer, ensure that the test
route is approved by the customer.

You can use the Mapinfo to create a test route. The procedure is as follows:

Create a layer on a digital map.

Mark the start point and the end point of the test.

Connect the start point to the end point with arrow-headed broken lines.

Comply with the following principles when selecting test points for a CQT:

2015-12-2

Select all test points approved by both parties before the RF optimization. In addition,
select other test points in hot-spot areas such as business centers, VIP subscribers, and
residential areas.

Huawei Confidential

Page 23 of 46

For internal use only

Divide high buildings into three parts (high, medium, and low) for testing. In each
part, perform tests with the antenna adjusted to different directions.

To make test data objectively reflect the radio environment, ensure that all the test
points are evenly distributed in the tested cluster.

DT Route Feedback and Optimization

When an initial test is complete in a cluster, the related test engineers should inform the RF
optimization manager of the following test data:

Test duration

Inappropriate sections in the test route (for example, one-way road and roads in poor
condition)

Optimize the test route for the next test to lower test complexity, reduce test cost, and prevent
the local cluster from being overlapped with other clusters.
Network optimization analyzers might also revise a test route if they find that the network
fails to cover all sections of the pre-defined test route in an initial test due to the network
topology or other subjective factors.

3.2.3 Test Method

An RF optimization test aims to optimize RF signals. Use a scanner and a test terminal to
continuously download data. Only downlink signals are tested in this way. To
comprehensively analyze a network, the OMC equipment rooms support is required. Obtain
the uplink measurement report from the air interface of the test terminal by tracing single-user
signaling on the GW.

3.3 Service Optimization Test

Service optimization follows RF optimization.
Service optimization aims to solve network performance problems by adjusting network
parameters and engineering parameters. In this way, service optimization makes network KPIs
meet the acceptance requirements.
Principles for selecting test routes (for DTs) and test points (for CQTs) in the service
optimization stage are similar to those in the RF optimization stage. The scope of service
optimization should be smaller than that of RF optimization but larger than that of KPI
acceptance. That is to say, service optimization is performed for the final KPI acceptance
when signal coverage is guaranteed.
Service optimization focuses on the following network performance problems:

Low network access success ratio

Low HO success ratio

Small uplink or downlink throughput

Long delay

Determine test items and methods for service optimization by referring to the test items and
test flags of final network acceptance. Adjust network parameters and engineering parameters
based on a joint analysis of service optimization test results and traffic statistics. In this way,
services are optimized.

2015-12-2

Huawei Confidential

Page 24 of 46

For internal use only

This section describes definitions and test methods of possible KPIs in network acceptance.
No reference value of any KPI is mentioned in this document.

All the test items described later in this document are baseline KPIs that come from pilot deployment
and small-scale commercial deployment. Some test methods probably are not suitable for tests of largescale commercial deployment. Test items in specific projects can be modified according to actual
conditions and the latest version of baseline KPIs. The required signal strength and quality mentioned in
this section are for reference only. Test results vary from terminal to terminal. Modify the reference
values in specific tests based on actual conditions.

3.3.1 DT Items and Methods

Accessibility
The related KPI is network access success ratio.

Definition
Network access ratio = Initial network successes / (RNG-REQ requests on the air
interface Access failures caused by the network) x 100%

Test method and conditions

The tested area is a target coverage area of the system.

The CINR is higher than 16 dB and the RSSI is higher than 75 dBm in both uplink
and downlink in the tested area.

Drive the test car at the specified speed (20 km/h to 30 km/h) along the test route.
Test network accessibility when the car moves at different speeds.

Retainability
The related KPI is data call drop rate.

Definition
Data call drop rate = Disrupted data transmission count / Data transmission setup count x
100%

Test method and conditions

Use a terminal to transfer a 5-Mbyte file when the car moves at a speed lower than 40
km/h. If the file transfer rate drops to zero and remains zero for over five minutes, a call
drop is counted. Repeat the test for over 50 times.
The CINR is higher than 16 dB and the RSSI is higher than 75 dBm in both uplink and
downlink in the tested area.

Mobility

Intra-BTS HO success ratio

Definition
Intra-BTS HO success ratio = Intra-BTS HO successes / Intra-BTS HO count x 100%

2015-12-2

Test method and conditions

Huawei Confidential

Page 25 of 46

For internal use only

1.

Choose two sectors in the same BS for testing. Choose a test route that passes through
the HO zone between the tested sectors. The CINR is higher than 16 dB and the RSSI is
higher than 70 dBm along the test route.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the card at a speed lower than 40 km/h along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for at least 50 times.

Inter-BTS HO success ratio (intra-GW)

Definition
Inter-BTS HO success ratio (intra-GW) = Inter-BTS HO successes / Inter-BTS HO
count x 100%

Test method and conditions

1.

Choose two sectors that are respectively in two BSs within the same GW for testing.
Choose a test route that passes through the HO zone between the tested sectors. The
CINR is higher than 16 dB and the RSSI is higher than 70 dBm along the test route.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the card at a speed lower than 40 km/h along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for at least 50 times.

Inter-BTS HO success ratio (inter-GW)

Definition
Inter-BTS HO success ratio (inter-GW) = Inter-BTS HO successes / Inter-BTS HO
count x 100%

Test method and conditions

1.

Choose two sectors that are respectively in two BSs in different GWs for testing. Choose
a test route that passes through the HO zone between the tested sectors. The CINR is
higher than 16 dB and the RSSI is higher than 70 dBm along the test route.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the card at a speed lower than 40 km/h along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for at least 50 times.

Intra-BTS HO duration (signaling plane)

HO Delay

2015-12-2

Definition

Huawei Confidential

Page 26 of 46

For internal use only

Interval between the time when an terminal sends an HO_IND message and the time
when the terminal receives the RNG_RSP message from the target BS
This KPI is measured at the terminal side.

Test method and conditions

1.

Choose two sectors in the same BS for testing. Choose a test route that passes through
the HO zone between the tested sectors.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the test car at the specified speed (20 km/h to 30km/h) along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for several times.

Intra-BTS HO duration (service plane)

Definition
Interval between the time when an terminal receives the last packet from the serving
BS and the time when the terminal receives the first packet from the target BS
This KPI is measured at the terminal side.

Test method and conditions

1.

Choose two sectors in the same BS for testing. Choose a test route that passes through
the HO zone between the tested sectors.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the test car at the specified speed (20 km/h to 30km/h) along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for several times.

Inter-BTS HO duration (intra-GW, signaling plane)

Definition
Interval between the time when an terminal sends an HO_IND message and the time
when the terminal receives the RNG_RSP message from the target BS
This KPI is measured at the terminal side.

1.

Choose two sectors that are respectively in two BSs within the same GW for testing.
Choose a test route that passes through the HO zone between the tested sectors.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the test car at the specified speed (20 km/h to 30km/h) along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for several times.

Inter-BTS HO duration (intra-GW, service plane)

2015-12-2

Test method and conditions

Definition

Huawei Confidential

Page 27 of 46

For internal use only

Interval between the time when an terminal receives the last packet from the serving
BS and the time when the terminal receives the first packet from the target BS
This KPI is measured at the terminal side.

Test method and conditions

1.

Choose two sectors that are respectively in two BSs within the same GW for testing.
Choose a test route that passes through the HO zone between the tested sectors.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the test car at the specified speed (20 km/h to 30km/h) along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for several times.

Inter-BTS HO duration (inter-GW, signaling plane)

Definition
Interval between the time when an terminal sends an HO_IND message and the time
when the terminal receives the RNG_RSP message from the target BS
This KPI is measured at the terminal side.

Test method and conditions

1.

Choose two sectors that are respectively in two BSs in different GWs for testing. Choose
a test route that passes through the HO zone between the tested sectors.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the test car at the specified speed (20 km/h to 30km/h) along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for several times.

Inter-BTS HO duration (inter-GW, service plane)

Definition
Interval between the time when an terminal receives the last packet from the serving
BS and the time when the terminal receives the first packet from the target BS
This KPI is measured at the terminal side.

2015-12-2

Test method and conditions

1.

Choose two sectors that are respectively in two BSs in different GWs for testing. Choose
a test route that passes through the HO zone between the tested sectors.

2.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

3.

Drive the test car at the specified speed (20 km/h to 30km/h) along the test route.

4.

Check changes in the active PN and status of Ping packets when the car moves. Record
the related data at both the BTS side and the terminal side.

5.

Check the IP address of the terminal after HO.

6.

Repeat the test for several times.

Huawei Confidential

Page 28 of 46

For internal use only

Ping Delay
The related KPI is Ping delay (no load).

Definition
Ping delay (no load) = The time when the host receives the Ping Response message (T2)
The time when the host sends the Ping Request message (T1)

Test method and conditions

1.

Choose a radial route in a sector. The route starts near the BTS and ends at the cell
border.

2.

Ensure that no other subscriber except the test terminal exists in the cell. Disable
adjacent cells.

3.

At the start point, ensure that the test terminal accesses the network and that only one
expected active PN is available. Record the IP address of the terminal.

4.

Send consecutive Ping packets to the FTP server and confirm that the results are normal.
Enable the log function.

5.

Drive the test car at the specified speed (20 km/h to 30km/h) along the test route.

6.

Monitor status of Ping packets when the car moves. Record the related data at both the
BTS side and the terminal side.

7.

If the connection breaks down during the test, re-create the connection as soon as
possible. In this case, record the location and other related data.

VoIP Service

Intra-BTS VoIP HO success ratio

Definition
Intra-BTS VoIP HO success ratio = Intra-BTS HO successes / Intra-BTS HO count x
100%

Test method and conditions

1.

Choose two sectors in the same BS for testing. Choose a test route that passes through
the HO zone between the tested sectors.

2.

Enable a downlink VoIP connection on the PC at the terminal side and set the coding
mode to G.711u.

3.

Initial a VoIP call, and then drive the test car from the serving sector to the target sector.

4.

End the VoIP call after an HO.

5.

Repeat the test for several times.

6.

Change the coding mode to G.729, and then repeat the test for several times.

Inter-BTS VoIP HO success ratio (intra-GW)

Definition
Inter-BTS VoIP HO success ratio (intra-GW) = Inter-BTS HO successes / Inter-BTS
HO count x 100%

2015-12-2

Test method and conditions

1.

Choose two sectors that are respectively in two BSs within the same GW for testing.
Choose a test route that passes through the HO zone between the tested sectors.

2.

Enable a downlink VoIP connection on the PC at the terminal side and set the coding
mode to G.711u.

3.

Initial a VoIP call, and then drive the test car from the serving sector to the target sector.

Huawei Confidential

Page 29 of 46

For internal use only

4.

End the VoIP call after an HO.

5.

Repeat the test for several times.

6.

Change the coding mode to G.729, and then repeat the test for several times.

Inter-BTS VoIP HO success ratio (inter-GW)

Definition
Inter-BTS VoIP HO success ratio (inter-GW) = Inter-BTS HO successes / Inter-BTS
HO count x 100%

Test method and conditions

1.

Choose two sectors that are respectively in two BSs in different GWs for testing. Choose
a test route that passes through the HO zone between the tested sectors.

2.

Enable a downlink VoIP connection on the PC at the terminal side and set the coding
mode to G.711u.

3.

Initial a VoIP call, and then drive the test car from the serving sector to the target sector.

4.

End the VoIP call after an HO.

5.

Repeat the test for several times.

6.

Change the coding mode to G.729, and then repeat the test for several times.

3.3.2 CQT Items and Methods

Accessibility

Network access success ratio

Definition
Network access ratio = Initial network successes / (RNG-REQ requests on the air
interface Access failures caused by the network) x 100%

Test method and conditions

Try to access the network for 100 times at the far point, midpoint, and near point.
The CINR is higher than 16 dB and the RSSI is higher than 75 dBm in both uplink
and downlink in the tested area.

Traffic stream setup success ratio

Definition
Traffic stream setup success ratio = (DSA-ACK count / DSA-REQ count) x 100%

Test method and conditions

Try to set up a traffic stream for 100 times at the far point, midpoint, and near point.
The CINR is higher than 16 dB and the RSSI is higher than 75 dBm in both uplink
and downlink in the tested area.

Retainability
The related KPI is data call drop rate.

Definition
Data call drop rate = Disrupted data transmission count / Data transmission setup count x
100%

2015-12-2

Test method and conditions

Huawei Confidential

Page 30 of 46

For internal use only

Use a terminal that is located at a fixed place to transfer a 5-Mbyte file. If the file
transfer rate drops to zero and remains zero for over five minutes, a call drop is counted.
Repeat the test for over 50 times.
The CINR is higher than 16 dB and the RSSI is higher than 75 dBm in both uplink and
downlink in the tested area.

FTP Throughput

Single-subscriber downlink throughput at near point

Definition
Maximum downlink throughput of a single subscriber in a sector

Test method and conditions

1.

Place the test terminal at the near point of the tested sector.

2.

Download files for at least five minutes and record the average throughput.

Single-subscriber uplink throughput at near point

Definition
Maximum uplink throughput of a single subscriber in a sector

Test method and conditions

3.

Place the test terminal at the near point of the tested sector.

4.

Upload files for at least five minutes and record the average throughput.

Single-sector downlink throughput at near point

Definition
Maximum downlink throughput of a sector

Test method and conditions

1.

Ensure that no subscriber except the test terminals exists in the tested sector.

2.

Place at least three test terminals at the near point of the tested sector.

3.

Download files for at least five minutes and record the average throughput of each
terminal.

Single-sector uplink throughput at near point

Definition
Maximum uplink throughput of a sector

Test method and conditions

4.

Ensure that no subscriber except the test terminals exists in the tested sector.

5.

Place at least three test terminals at the near point of the tested sector.

6.

Upload files for at least five minutes and record the average throughput of each terminal.

Maximum number of subscribers in a single sector

Definition
Maximum number of subscribers who simultaneously use services in a single sector

Test method and conditions

Disable signals from other sectors. Measure the maximum number of subscribers
who can simultaneously enable stable transmission in the tested sector.

Access Delay

2015-12-2

Traffic stream setup duration

Huawei Confidential

Page 31 of 46

For internal use only

Definition
Interval between the DSA-REQ message and the DSA-ACK message of a traffic
stream

Test method and conditions

Measure the interval between the DSA-REQ message and the DSA-ACK message of
a traffic stream at the terminal side.

Air interface connection setup duration

Definition
Interval between the time when an terminal sends an initial RNG-REQ message and
the time when the BS sends an REG-RSP message to the terminal
This KPI is measured at the terminal side.

Test method and conditions

Measure the interval between the time when an terminal sends the initial RNG-REQ
message and the time when the BS sends an REG-RSP message to the terminal.

Access delay

Definition
Interval between the RNG-RSP message and the DSA-ACK message of the initial
traffic stream

Test method and conditions

Measure the interval between the time when an terminal sends the initial RNG-REQ
message and the time when the terminal receives the DSA-ACK message. Measure
this KPI at the terminal side.

VoIP Service

VoIP call setup delay

Definition
Interval between the time when a VoIP call is initiated and the time when the first
VoIP packet is detected

Test method and conditions

Perform this test at a fixed location. Measure the interval between the time when a
VoIP call is initiated and the time when the call is set up.

VoIP call drop rate

Definition
VoIP call drop rate = Call drop count / Call count x 100%

Test method and conditions

Perform this test at a fixed location. Make a VoIP call and keep the conversation for
one minute. A call drop is counted if redialing is required during the conversation.
Repeat this test for several times.

VoIP call setup success ratio

Definition
VoIP call setup success ratio = VoIP call setup successes / VoIP call count x 100%

Test method and conditions

Perform this test at a fixed location. If a call is set up within 20 seconds after the
calling party dials, it means a VoIP call setup success. Divide the number of call setup
successes by the number of call attempts. Remarks: 1) The calling party and the

2015-12-2

Huawei Confidential

Page 32 of 46

For internal use only

called party simultaneously access the network. 2) The call is set up within 20
seconds after the calling party dials. 3) The called party is not involved in this test.

VoIP call end-to-end (E2E) delay

The time when the CSN sends a data packet (T1) The time when terminal 1 sends the
data packet (T2)

VoIP packet loss ratio

(Total number of data packets sent by terminal 1 Total number of data packets received
by terminal 2) / Total number of data packets sent by terminal 1

VoIP Mean Opinion Score (MOS)

MOS of a certain number of subscribers based on their Absolute Category Rating (ACR)
scores

VoIP one-way delay

The time when the CSN sends a data packet (T1) The time when terminal 1 sends the
data packet (T2)

VoIP jitter

Definition
Average value of the E2E delay (The time when terminal 2 receives a data packet
The time when terminal 1 sends the data packet)

Test method and conditions

1.

Connect the CPE to a PC through a network cable.

2.

Set downlink and uplink modulation schemes of the CPE to QPSK1/2 CTC.

3.

Enable the IxChariot software on both the FTP server and the PC at the terminal side.
Ensure that the Endpoint process is enabled on both PCs.

4.

Enable a downlink VoIP connection on the PC at the terminal side and set the coding
mode to G.711u.

5.

Initiate a VoIP call and then complete it. Obtain VoIP data such as VOIP MOS, one-way
delay, E2E delay, jitter, and lost data.

6.

Enable an uplink VoIP connection on the PC at the terminal side and set the coding mode
to G.711u. Repeat step 5.

7.

Enable a downlink VoIP connection on the PC at the terminal side and set the coding
mode to G.729. Repeat step 5.

8.

Enable an uplink VoIP connection on the PC at the terminal side and set the coding mode
to G.729. Repeat step 5.

Single-BS single-sector VoIP capacity

Definition
VoIP connection pairs allowed for multiple terminals in a single sector of a single BS

2015-12-2

Test method and conditions

1.

Connect three CPE terminals to a PC through network cables.

2.

Set the CPE downlink modulation scheme to 64QAM 5/6 CTC and the CPE uplink
modulation scheme to 16QAM 3/4 CTC.

3.

Enable the IxChariot software on both the FTP server and the PC at the terminal side.
Ensure that the Endpoint process is enabled on both PCs.

4.

Configure 30 bi-directional VoIP connections on the PC at the terminal side and set the
coding mode to G.711u. Note that the VoIP connections can be distributed to several
CPE terminals.

Huawei Confidential

Page 33 of 46

For internal use only

5.

Initiate a VoIP call and then complete it. Obtain VoIP data such as VOIP MOS, one-way
delay, E2E delay, jitter, and lost data.

6.

If the MOS is higher than 3.5, increase the number of VoIP connection pairs one by one
and then repeat step 5. Obtain the maximum number of VoIP connection pairs and other
VoIP KPIs.

7.

Enable 30 bi-directional VoIP connections on the PC at the terminal side and set the
coding mode to G.729. Repeat steps 5 and 6.

Streaming Service

Delay
(Maximum buffer size in the measurement period Minimum buffer size in the
measurement period) / Normalized streaming rate
DF = [VB(MAX)-VB(MIN)]/MR

Jitter

Streaming packet loss

Definition
Number of lost or out-of-sequence streaming data packets in the measurement period

Test method and conditions

1.

Connect the CPE to a PC through a network cable.

2.

Set downlink and uplink modulation schemes of the CPE to QPSK1/2 CTC.

3.

Enable the IxChariot software on both the FTP server and the PC at the terminal side.
Ensure that the Endpoint process is enabled on both PCs.

4.

Enable a downlink streaming connection on the PC at the terminal side.

5.

Initiate and complete streaming setup. Obtain the related data such as Delay Factor and
Media Lost Data.

3.4 Acceptance Test

Perform an acceptance test before delivering an optimized network.
Before a network acceptance test, negotiate with the customer about basic requirements for
the acceptance test, including the following information:

Acceptance scope (clusters)

Test specifications

Start time of the acceptance test

DT KPI measurement method

The acceptance scope, however, should be a subset of the scope of the earlier RF and service
optimization.
To guarantee standard and professional acceptance services, negotiate with customers about
types and formats of the acceptance documents to be signed and archived. In this way, the
brand of Huawei network optimization service is upgraded.

2015-12-2

Huawei Confidential

Page 34 of 46

For internal use only

3.4.1 Operations Before an Acceptance Test

Before network acceptance, check status and alarms of all BSs in the acceptance area. Ensure
that no alarm is generated in any cell in the acceptance area and that all cells are in normal
state.
Use the Local Maintenance Terminal (LMT) or the M2000 to check board alarms and
transmission alarms of each network element (NE).
To ensure satisfactory results of an acceptance test, perform at least one formal pretest prior to
the acceptance test. If the pretest discovers that the network fails to meet the test
requirements, optimize the network till all problems are solved. For troubleshooting methods,
see related guides.

3.4.2 Operations During an Acceptance Test

To guarantee timely and successful network acceptance, divide the entire project into several
clusters.
Start acceptance in a cluster once the cluster meets the basic acceptance conditions. The
acceptance includes DTs and traffic statistics measurement. Acceptance clusters are
independent of each other.
Negotiate with customers about acceptance clusters by referring to the clusters defined for RF
and parameter optimization.
Define important clusters according to actual conditions.
Discuss each cluster with customers to decide whether to perform DTs and traffic statistics
measurement, whether to load traffic, and how much traffic to be loaded. Perform CQTs for
acceptance in important clusters.

3.4.3 Operations After an Acceptance Test

Archive test data according to the pre-defined principles after each acceptance test. Request
customers to approve the related test results with signature.

3.5 Benchmark Test

Testing that involves two or more comparative tests is called benchmark test.
The benchmark test now is classified into the following types:

Tests before and after network migration or optimization

Benchmark tests between operators

Benchmark tests between technologies

Both test procedures and test items vary with test types.

3.5.1 Tests Before and After Network Migration or

Optimization
Figure 1.1 shows the benchmark test for network migration or optimization. An initial DT and
a DT after network migration or optimization constitute a benchmark test.

2015-12-2

Huawei Confidential

Page 35 of 46

For internal use only

The initial DT is performed before the DT after network migration or optimization. The
Benchmark Report needs often to show seamless migration or performance improvement.
Therefore, the initial DT mode should be similar to the DT mode after network migration or
optimization. For details, see the following description.
Figure 1.1 Procedure for the benchmark test for network migration or optimization

For a benchmark test for network migration, the following factors should be the same before
and after network migration in principle:

Test tool

Test car

Terminal location

Test route

Driving speed

Test time

Test procedure

Today, different test tools and procedures are used when WiMAX 16d networks are migrated
to WiMAX 16e networks. Comply with the following principles when performing a DT after
migration:

Use highly sensitive test software and terminals specialized for WiMAX 16e networks.

Keep the antenna gain and height the same as those in the initial test of the original
WiMAX 16d network. If a CQT with a high gain antenna is performed for the original
WiMAX 16d network, use an antenna with the same gain to perform a test in the same
location for the WiMAX 16e network.

The WiMAX 16e network often has more BSs than the original WiMAX 16d network.
Before the test, adjust the antenna direction of the test terminal to find the best coverage.

3.5.2 Benchmark Tests Between Operators

Figure 1.1 shows the procedure for a benchmark test between different operators. The
comparison between test results of different operators constitutes a benchmark test.
A benchmark test between operators differs from that for network migration or optimization
in the following aspects:

Tests of different operators are performed simultaneously.

The Benchmark Report clearly shows competitiveness comparison between different

operators.

Therefore, operators often have concern about the test result.

2015-12-2

Huawei Confidential

Page 36 of 46

For internal use only

Figure 1.1 Procedure for a benchmark test between operators

Network optimization engineers should make full preparation for benchmark tests between
operators and define test strategies. Test engineers should prepare related test tools and
understand the following test information:

Test content
Make clear test purposes: Is the test used to compare services between different
operators? Is the FTP download service or the VoIP voice service to be tested? Check
whether the prepared test tools support the test items. Report the problem in time if the
test tools cannot support the test items.

Test plan
Make clear the specific test requirements. To guarantee accuracy of test results, test
organizers often specify strict requirements on test plans, including the following

Test route

Test time

Test duration

Driving speed

Specific settings of each test item

Pretest
To verify the test tool and procedure, perform a pretest according to the initial test plan.
Pretests also provide a basis for estimation of test results.

Test plan optimization

Revise and optimize test plans according to pretest results.

3.5.3 Benchmark Tests Between Technologies

Benchmark tests between different technologies refer to simultaneous service tests of different
network modes such as the GSM, CDMA, WCDMA, and WiMAX in the same region.
The same services are tested in benchmark tests between technologies. For example, test
coverage, speech quality, and data download performance of different networks
simultaneously. For detailed test procedures, see the DT&CQT Guide for related network
technologies.

2015-12-2

Huawei Confidential

Page 37 of 46

For internal use only

Summary

This document, based on the GENEX Probe software platform, describes test conditions,
procedures, and precautions for all test tasks in each stage of WiMAX network optimization.
The following problems exist in this document because WiMAX products now are not widely
commercialized:

New versions of the DT software are continuously developed. For specific test items and
their parameter settings, see user guides of related software versions.

No test terminal that applies to all networks is available. Different test terminals are used
for different networks. Therefore, this document does not mention specific settings of
test terminals. The settings are described in related guides.

All test items and procedures mentioned in this document are summarized based on pilot
deployment or small-scale commercial deployment. Therefore, the test items and
procedures are subject to change. If a customer objects to the test items, the related field
engineers should discuss with the headquarters (HQ) before making a decision.

No mature test software or measurement KPI is available for VoIP tests. Only simple
CQT assessment is available. Therefore, both methods and tools for VoIP tests need to be
improved.

Any suggestions and comments on this document are welcomed.

2015-12-2

Huawei Confidential

Page 38 of 46

For internal use only

Attachment

5.1 CQT Record

CQT Record
Table.xls

2015-12-2

Huawei Confidential

Page 39 of 46

For internal use only

Appendix

6.1 Signaling Flows

6.1.1 Network Access Flow
Figure 1.1 shows the network access flow.

2015-12-2

Huawei Confidential

Page 40 of 46

For internal use only

Figure 1.1 Network access flow chart

6.1.2 HO Flows
Figure 1.1 shows the HO request flow.

2015-12-2

Huawei Confidential

Page 41 of 46

For internal use only

Figure 1.1 HO request flow chart

Serving
BS

MS

Serving/Target
GW

Target
BS1

Target
BS 2...x

MOB_MSHO-REQ
HO-Request

(MS ID, capabilities,Target BS list[Signal Quality])

HO-Request

(MS ID, connection params, capabilities,required BW and QoS)

HO--Response

(Result f lag, MS ID, Serv ice lev el prediction, HO optimization f lag, HO_authorization_policy _support, HO ID, HOAction time)
Make a decision

HO-Response
(MS ID, Target BS List [Serv ice lev el prediction, HO optimization f lag, HO_authorization_policy _support, HO ID, HO Action time ])

MOB_BSHO_RSP
(RecommendedBS=BS#2
serv ice lev el prediction=2)

MOB_HO_Indication
HO_IND_ty pe0b00:
serv ing BS release

Figure 1.2 shows the HO execution flow.

Figure 1.2 HO execution flow chart
MS

Serving/
Target GW

Serving BS

Anchor
Authenticator

Target BS

MOB_HO-IND
(HO_IND_type:0b00
Serving BS-release)

HO Confirm
AK Request
AK Response
HO Confirm
(AK&AK Context)

6.2 Speech Quality

Operators focus on speech quality today. As a KPI of network Quality of Service (QoS), it
directly reflects the speech quality that subscribers experience during conversations. Speech
quality tests are classified into MOS tests and Perceptual Evaluation of Speech Quality
(PESQ) tests.

6.2.1 MOS
According to ITU-T Recommendation P.830, the speech quality is classified into five levels
that correspond to five MOS scores.
MOS is a five level scale from bad (1) to excellent (5). Dozens of listeners evaluate the tested
speech in the same channel environment with scale described in . Average the scores then.
MOC scores fluctuate from test to test. To reduce the fluctuation, ensure that enough listeners

2015-12-2

Huawei Confidential

Page 42 of 46

For internal use only

take part in the test, that sufficient voice materials are used, and that the test environment is
the same from all listeners.
Table 1.1 lists the MOS scale.
Table 1.1 MOS scale
Quality of
Speech

MOS
Score

Impairment

Excellent

Imperceptible

Good

Perceptible but annoying

Fair

Slightly annoying

Poor

Annoying

Bad

Very annoying

In communication systems, voice signals are coded to save bandwidth. Bit-rate reduction
deteriorates speech quality. Therefore, the MOS score cannot be five even if network
transmission is perfect.

6.2.2 PESQ
MOS testing is time consuming and costly, though it is the most reliable evaluation mode that
applies to all coding and decoding algorithms and network performance. In addition, MOS
scores vary with listeners. Therefore, objective evaluation methods for speech quality are
developed. This section describes Perceptual Evaluation of Speech Quality (PESQ) and ITUT P.862.1.

PESQ
PESQ was developed jointly by British Telecom and KPN Research and was officially
approved as new ITU-T recommendation P.862 in 2001. PESQ compares the originals signal
with the degraded signals. The resulting quality score is approximately equal to the subjective
MOS measured using panel tests according to ITU-T P.800. PESQ is an intrusive algorithm.
PESQ supports powerful testing functions. It can be used to test both the performance of NEs
(such as Codec) and E2E voice quality. PESQ can also be used to test signal degradation
causes such as coding and decoding distortion, errors, packet losses, delay, jitter, and filtering.
PESQ is the best one of the commercialized and standardized testing algorithms in the
industry.
Figure 1.1 shows the PESQ process.

2015-12-2

Huawei Confidential

Page 43 of 46

For internal use only

Figure 1.1 PESQ process

PESQ-LQO (P.862.1)
ITU-T P.862.1 is not an independent recommendation. It aims to provide a single mapping
from raw P.862 score to the Listening Quality Objective Mean Opinion Score (LQO-MOS).
PESQ-LQO supports precise estimation of human perception of speech quality. Therefore,
PEAQ-LQO scores are closer to MOS scores. PESQ-LQO scores over 3.4 are higher than the
raw PESQ scores, while PESQ-LQO scores below 3.4 are lower than the raw PESQ-LQO
scores. To improve Quality of Experience (QoE) of subscribers, increase the percentage of
PESQ-LQO scores that are higher than 3.4.
The function of mapping from a PESQ score to a PESQ-LQO score is defined as follows
(where x is the PESQ MOS score and y is the corresponding PESQ-LQO score):

y 0.999

4.999 0.999
1 e 1.4945*x 4.6607

Figure 1.1 Mapping from PESQ (P.862) scores to PESQ-LQO (P.862.1) scores

For test software that uses the PESQ algorithm, the corresponding PESQ-LQO scores are
calculated based on related raw PESQ scores.

2015-12-2

Huawei Confidential

Page 44 of 46

For internal use only

6.2.3 ITU-T P.563 The ITU-T Standard for SingleEnded Speech Quality Assessment
ITU-T Recommendation P.563, which was approved by the ITU in May 2004, defines a
single-ended method for objective speech quality assessment. It is a non-intrusive method.
Unlike ITU-T P.862 PESQ, P.563 does not require original reference signal information in
estimating the quality of processed speech signals and thus is quite useful for monitoring the
quality of speech for in-service networks. Non-intrusive test methods apply to a wider scope.
The P.563 model, however, shows very limited performance because no reference signal is
available. This performance level is much lower than that of intrusive methods such as PESQ.
Therefore, P.563 applies only to assessment of proximate speech quality.
Figure 1.1 shows the implementation process of P.563. P.563 algorithm often applies to test
terminals.
Figure 1.1 P.563 implementation process

2015-12-2

Huawei Confidential

Page 45 of 46

For internal use only

References
1.

2015-12-2

Fu Qiang, GENEX Probe User Guide-20080826

Huawei Confidential

Page 46 of 46