QoE using a mobile application

approach: a solution from Ciqual to

qualify the quality of experience of end
users on mobile networks
 What is Ciqual and what is it for ?

• Ciqual is a solution that permits to collect and analyze the

KPIs from the end user’s device regarding data, voice,
coverage, OTT and its custom feedbacks.

• It is a mobile application approach used to qualify the

quality of experience of the customers.

22-Mar-18 9
 Benefits of Mobile Application Approach
Truly independent view of consumers within a country
• neutral and independent measuring tool

No more need of satisfaction campaigns and questionnaires from users

• This system is expensive and may be inconclusive

Ease of Deployment
• Deployed as a downloadable mobile app on the Google Play Store or iTunes, or as SDK into existing App

Industry recognised KPIs

• The tool analyzes and takes advantage of KPIs used in IOS and Android operating systems. It's not about
tool-specific KPIs
1:1 Relationship with Consumer
• If required, data could be linked to a user to enable faster issue resolution e.g. if a consumer escalates a
complaint.
22-Mar-18
 End users actors in the quality of the network

A mobile application to put the user as a A data analysis software to interpret all the
major actor in the improvement of QoS and data collected and proactively manage user
QoE. issues

Thanks to user’s data, Ciqual provides all the User data analysis via dashboards, graphs and
necessary information about voice quality, statistics, as well as maps to be able to correlate
mobile cells, the device used (hardware and and understand more effectively any network
software), data, coverage and even direct user issues.
feedback.

11
 2 ways to integrate the mobile app

Standalone version Ciqual MyConnection Integrated solution to an already existing app

The standalone app of Ciqual is available on The Ciqual system is integrated with an already
the Apple Store and Play Store. Ideal for PoC existing application through a Software Development
for hundreds or thousands of users., the app is Kit (SDK). Ideal for a wide development. The
advantage of this method is to benefit from the
ready to use.
client base that already uses the original application.

12
 Global network architecture
CIQUAL reports the quality of experience data, voice and coverage of each user, as well
as the use of mobile applications if the user authorizes it.

Database
Ciqual SDK Lightweight client software Dashboards and
using less than 1% of CPU reports accessible via
resources and around a data analysis tool
North-bound
200kb of data interface
(available Dec
2015)
Reports &
Analysis
http Active Tests http/https
Ciqual
MyConnection app
Laptop/Smartphone/Tablet Administration
Test server
Test services and mobile
network in roaming and Storage of test data, Customer management,
WiFi data analysis groups, licenses and user
permissions
 Active tests for the users

• Speed Test : upload and download link

done from a FTP server

22-Mar-18 14
 Active tests for the users

• Connection : Predefined services

availability test such as social
networks, web browsing, email client,
streaming and online video, etc.

• The software will ping the service and

depending on the response time will
assign a rating of 0 to 5

22-Mar-18 15
 Feedback system

• Feedback : allows the user to

evaluate their quality of experience
on calls, data (speed, audio or video
streaming), network coverage
(signal strength, access technology)
and to report directly any problem

22-Mar-18 16
 Archived results

• Events and tests performed by the

user are archived

22-Mar-18 17
 Tests and measurements
User-Triggered Tests Scheduled Tests Passive Observations Subjective Feedback

Speed and latency tests • DNS • Data • Emoticon-based feedback

triggered by the user • Ping • Voice (Android only) • Direct subjective feedback
via a customized
‘Star tests’ on availability • Reachability • Coverage(Android only) questionnaire.
of 6 predefined services, • FTP, HTTP download • Allows NPS questions and
launched when the app is • Triggered automatically • Measurements free-text feedback from
opened. by the client during any collected in the the end customer
data session, even background without • Correlate perceived
when running in the user intervention customer experience with
background. actual KPIs.
 Key Performance Indicator (KPI)
Data
Détails de certains KPIs importants
• Throughput
– NPS – Net Promoter Score
• Latency
• Collted by the feebacks
• RTT (reach, DNS, ping)
• Packet loss – NIC – No Internet Connection
• Duration of sessions • Data access activated on the phone but no
• Volume data data is exchanged with the network
• Other KPIs*: RSSI/RSRP/RSRQ/SINR – NVC – No Voice Connectivity
Voice* • Unsuccessful call attempt due to lack of
network connectivity
• Call attempt
• Reason for end of call (normal, drop) – PCFR – Perceived Call Failure Rate
• Call duration • Call cut due to poor connectivity or bad
• Type of call (normal, emergency) signal
• Other KPIs during the call (signal strength, interruptions, cuts) – In-Service Ratio
Coverage* • Time when the user remained covered by
the network of his operator
• In coverage
• Out of coverage
• Name of the mobile operator

*Android only 19
 New dimensions to aggregate and filter by
User
• IMEI*, IMSI* or Device ID

Location
• Latitude and longitude as reported by the device.
• Country*, network* and technology, including WiFi
• Cell ID*, LAC/RA*, APN, Wi-Fi SSID
• Local IP address
• Wi-Fi hotspot IP address
• Custom reporting layers (districts, areas…)

Device type
• Model, Make, OS release

Apps*
• Foreground app QoE
• Background app usage
*Android only
 Dimensions and filters

Service
Time Participants Localisation Device
provider
• Year • User groups • Country • GPS • Brand
• Month • IMEI/Device • Network coordinates • Model
• Week ID • Access • Cell location • OS
• Day technology (CGI) • Firmware
• Hour • APN/SSID • CPU
• Date
Privacy – what Ciqual does NOT collect
Contacts
Content of messages or
conversations
URLs, browsing history
Keystrokes, Camera
 EXAMPLE : POC WITH THE REGULATOR OF MALI

22-Mar-18 23
 Preface
• PoC report with real measurements from 9 AMRTP employees
using Ciqual’s MyConnection app.
• Data collection November 10th- 28th
• It comprises five separate sections:
– Data
– Voice
– Coverage
– Customer Feedback
– App Usage.
DATA
Data global activity by day
Data – Technology comparison
Data – Cellular technology comparison
Data – Experience of each user
Data – User map
Data – 3G Radio
Orange throughput and latency
DNS Performance per technology
Ping Performance per technology and
hostname
Reach performance
FTP Tests
VOICE
 Voice – overall performance

Most calls
were made in
Bamako
Details on Bamako
Voice - Technology
Voice – 3G Radio
Voice – device performance
SERVICE STATE
Overall service state
Details on Bamako
FEEDBACKS
Example of customer feedback
APP USAGE
OTT app usage
OTT app performance in Bamako
A major reference of Ciqual : IMDA
the regulator of Singapour
 IMDA Singapore - App based QoE Approach
• Operators in Singapore did provide standard QoS results to IMDA,
BUT IMDA were concerned that the results:
– weren’t independent
– subject to manipulation by the telco
• IMDA decided they needed objective and independent view of the
Customer Experience.
• IMconnected: IMDA decided upon the app approach:
– IMconnected App is lightweight and non-intrusive
– Easy to brand and distribute (Google Play / iTunes)
– Little interaction from the end user

22-Mar-18 61
 IMDA Singapore
IMDA use the data gathered from their
mobile application “IMConnected” to
complement their current QoS
framework.

The data collected, including user

feedback, enable IMDA to publish
independent bi-annual reports to
consumers advising of the state of the
mobile and public Wi-Fi networks across
Singapore.

Using the mobile application approach

enables timely data collection and
ensures QoE KPIs are collected in as many
locations as possible – providing full
coverage across Singapore.
22-Mar-18
 IMDA Singapore

22-Mar-18
 To conclude
• Ciqual is a mobile approach application for regulators or operators that analyzes
the QoE of the customers

• It can be used :
– to get trends of the network 24/7 in a whole country
– to focus on some users (recurrent complaints in one sector for example)
– to identify a problem on a specific brand or model of smartphone

• Easier to deploy through a mobile app to download

• The end user is getting involved to the improvement of the quality of the network

22-Mar-18 64
 Thank you for your attention
Sales : Laurent CATINOT
Telecom Division Director
Email: cat@nexatech.fr
Phone: +33 1 47 05 15 76
Mobile: +33 6 86 17 23 67

Technical : Jonathan MARTIN

Telecom Technical Support
Email: support.telecom@nexatech.fr
Phone: +33 1 47 04 16 79
Mobile: +33 7 88 80 02 47

Address : NEXA TECHNOLOGIES

16 rue Pétrarque
75116 Paris - FRANCE
22-Mar-18 65