Ngenius Business Analytics: System Administration Guide
Ngenius Business Analytics: System Administration Guide
Ngenius Business Analytics: System Administration Guide
April 2020
Use of this product is subject to the End User License Agreement available at http://www.NetScout.com/legal/terms-and-
conditions or which accompanies the product at the time of shipment or, if applicable, the legal agreement executed by and
between NetScout Systems, Inc. or one of its wholly-owned subsidiaries ("NETSCOUT") and the purchaser of this product
("Agreement").
Government Use and Notice of Restricted Rights: In U.S. government ("Government") contracts or subcontracts, Customer will
provide that the Products and Documentation, including any technical data (collectively "Materials"), sold or delivered pursuant
to this Agreement for Government use are commercial as defined in Federal Acquisition Regulation ("FAR") 2.101and any
supplement and further are provided with RESTRICTED RIGHTS. All Materials were fully developed at private expense. Use,
duplication, release, modification, transfer, or disclosure ("Use") of the Materials is restricted by the terms of this Agreement and
further restricted in accordance with FAR 52.227-14 for civilian Government agency purposes and 252.227-7015 of the Defense
Federal Acquisition Regulations Supplement ("DFARS") for military Government agency purposes, or the similar acquisition
regulations of other applicable Government organizations, as applicable and amended. The Use of Materials is restricted by the
terms of this Agreement, and, in accordance with DFARS Section 227.7202 and FAR Section 12.212, is further restricted in
accordance with the terms of NETSCOUT'S commercial End User License Agreement. All other Use is prohibited, except as
described herein.
This Product may contain third-party technology. NETSCOUT may license such third-party technology and documentation ("Third-
Party Materials") for use with the Product only. In the event the Product contains Third-Party Materials, or in the event you have
the option to use the Product in conjunction with Third-Party Materials (as identified by NETSCOUT in the Documentation
provided with this Product), then such third-party materials are provided or accessible subject to the applicable third-party terms
and conditions contained either in the "Read Me" or "About" file located in the Software or on an Application CD provided with
this Product, or in an appendix located in the documentation provided with this Product. To the extent the Product includes
Third-Party Materials licensed to NETSCOUT by third parties, those third parties are third-party beneficiaries of, and may enforce,
the applicable provisions of such third-party terms and conditions.
Open-Source Software Acknowledgement: This product may incorporate open-source components that are governed by the GNU
General Public License ("GPL") or licenses that are compatible with the GPL license ("GPL Compatible License"). In accordance
with the terms of the GNU GPL, NETSCOUT will make available a complete, machine-readable copy of the source code
components of this product covered by the GPL or applicable GPL Compatible License, if any, upon receipt of a written request.
Please identify the product and send a request to:
To the extent applicable, the following information is provided for FCC compliance of Class A devices:
This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the
FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is
operated in a commercial environment. This equipment generates, uses, and can radiate radio frequency energy and, if not
installed and used in accordance with the instruction manual, may cause harmful interference to radio communications.
Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required
to correct the interference at their own expense.
Modifications to this product not authorized by NETSCOUT could void the FCC approval and terminate your authority to
operate the product. Please also see NETSCOUT's Compliance and Safety Warnings for NetScout Hardware Products
document, which can be found in the documents accompanying the equipment, or in the event such document is not
included with the product, please see the compliance and safety warning section of the user guides and installation
manuals.
No portion of this document may be copied, photocopied, reproduced, translated, or reduced to any electronic medium or
machine form without prior consent in writing from NETSCOUT. The information in this document is subject to change without
notice and does not represent a commitment on the part of NETSCOUT.
The products and specifications, configurations, and other technical information regarding the products described or referenced
in this document are subject to change without notice and NETSCOUT reserves the right, at its sole discretion, to make changes at
any time in its technical information, specifications, service, and support programs. All statements, technical information, and
recommendations contained in this document are believed to be accurate and reliable but are presented "as is" without
warranty of any kind, express or implied. You must take full responsibility for their application of any products specified in this
document. NETSCOUT makes no implied warranties of merchantability or fitness for a purpose as a result of this document or
the information described or referenced within, and all other warranties, express or implied, are excluded.
Except where otherwise indicated, the information contained in this document represents the planned capabilities and intended
functionality offered by the product and version number identified on the front of this document. Screen images depicted in this
document are representative and intended to serve as example images only.
992-0619-08-001
T a b le o fC o n te n ts 4
Description Refer to
Explorer users with the ‘Session Analysis Drilldown’ privilege Section 3.1.1 Explorer User Privileges
can launch nGenius Session Analyzer.
Dashboards now includes Traffic Volume Monitor Section 4.4 Dashboards
dashboard.
System Check Tool now includes Platform Information and Section 4.6 System Check Tool
NTP, Timezone and firewalld checks.
Device selection now supports ISNG RAN devices. Section 4.7.1 Selecting Data Feeds
Administrators can set several Device Enrichment Section 4.7.2 Device Enrichment
properties for ISNG and ISNG RAN devices.
l Explorer
l MicroStrategy
l System Administration
Privilege Description
nBA - Allow access to Explorer Dashboard Edit You can create and delete dashboards that you
have built
nBA - Allow access to Explorer Dashboard Explore You can use the Explore option to drill down into
any data point in the Metric Viewer, Metric
Report, or Metric Grid
nBA - Allow access to Explorer Metric Builder You can access the Metric builder and create,
edit, delete, export, or import metrics
nBA - Allow access to Explorer Data Browser You can access the Data Browser and save
filters and columns in the Data Browser
nBA - Allow Explorer access to display IMSI data You can see the IMSI value displayed
nBA - Allow Explorer access to display MSISDN data You can see the MSISDN value displayed
nBA - Allow access to Explorer mapping and exception You can access the configuration tab to map
rule configuration fields and create exception reports
Session Analysis Drilldown Allows drill-down to nGenius Session Analyser.
Note: Only one user with architect privileges associated with the customer developer
user group is permitted per site.
Note: This guide is written for administrators who have the ‘nBA -Administrator’ privilege. It is
assumed that an administrator user, with the ‘nBA -Administrator’ privilege, as well as the Explorer
privileges described in the previous section, has been created in nGeniusONE/nCM.
Privilege Description
nBA - Administrator Access to Admin Tools is controlled by the ‘nBA -Administrator’ privilege. This
privilege exposes the tools available for system administration, configuration and
monitoring.
nBA Netscout In addition to the above, there is the ‘nBA Netscout Administrator’ privilege. This is
Administrator intended to be used by Netscout support personnel and allows access to additional
administration tools and configuration parameters which are not documented in this
guide. This privilege replaces the ‘nBA – System Health Administrator’ privilege from
previous releases. It is recommended that system administrators use the new ‘nBA
Administrator’ privilege instead.
For instructions on how to configure the system, refer to Section 5 System Configuration. For
details on Export Topic Configuration, refer to Section 6 Topic Configuration.
Tool Description
Home Screen – Alerts A dashboard displaying any system alerts. See Section
Summary 4.3 Alerts Summary for details.
Dashboards A series of dashboards displaying the usage and
performance of the system. See Section 4.4 Dashboards
for details.
Containers This tool is used to view the status of and occasionally
restart the docker containers which comprise the system.
See Section 4.5 Containers for details.
Configuration Used to configure system properties. Refer to Section 5
System Configuration for details.
System Check This tool generates a report on the status of the system.
See Section 4.6 System Check Tool for details.
Devices This tool is used to configure the InfiniStream devices
feeding the system. See Section 4.7 Devices for details.
https://<haproxy_host>:8443/portal
Once logged in, you should see the main Launch Pad page. Based on licenses and privileges, various
modules will be displayed. Click on ‘System Administration’ to launch Admin Tools.
The dashboard presents a color-coded status of the system, displaying the number of alerts which
occurred over the selected time range and their description.
You can also receive system alerts via email and/or have the alert forwarded to an alarm
management system. Contact NETSCOUT Support for assistance in configuring this.
See Section A System Alerts for a description of possible alerts. Please contact NETSCOUT should
any alert occur.
4.4 Dashboards
Admin Tools includes a series of dashboards that display the usage and performance of the system.
Export Summary Export application statistics showing Display can be filtered by host name.
batch processing and pipeline topic
measures.
Kafka Brokers Kafka broker statistics showing status, Display can be filtered by broker host
topic, and inbound/outbound measures. name.
Kafka Cluster Kafka cluster statistics showing cluster Display can be filtered by broker host
health, overall throughput, broker, name.
controller, and requested rates.
Kafka Topics Kafka statistics per topic, including Display can be filtered by topic.
partition count and throughput.
User Activity History of user logon activity. Explorer and Admin Tools activity only.
Traffic Volume Trends of traffic volumes, Top hosts and Key metrics for monitoring the traffic
Monitor sources, data quality and processing being processed by the system. Can be
time. used for capacity planning, ensuring
traffic is being processed evenly across
available hosts, checking for anomalies in
feed quality and ensuring traffic is being
processed in a timely manner.
4.5 Containers
This tool is used to view the status of and restart the Docker containers which comprise the system.
To access, click on the icon in Admin Tools. Docker containers are portable packages of software
which run in the Linux native virtual environment. Certain system properties require that a container
is restarted so changes can take effect. See Section 5 System Configuration for details.
Caution: Do not restart or stop a Docker container unless following instructions in this
guide. Components of the system cease operation during a Docker restart and loss of data
or user access can occur.
This tool generates a detailed report on the status of the system. To access, click on the icon in
Admin Tools.
A system check report can be run following maintenance or when a system alert occurs --provide
these additional details on the status of the system when contacting NETSCOUT Support.
To run a report, click on the icon. It usually takes a few minutes for the
report to complete. Examine the results by either expanding each category or exporting as a text file.
l The icon indicates that all checks in the category pass with no issues.
l The icon warns that some non-critical checks failed. The specific issues are highlighted
in yellow when the category is expanded.
l The icon indicates a critical failure. The specific issues are highlighted in red when the
category is expanded.
l Platform information
l System Check (NTP, Timezone, firewalld checks)
l List of hosts
l Docker container status
l Timestamp when data was last loaded, per table
l Timestamp when aggregation last occurred, per table
l Collection backlog status
l Probe polling status
l CPU, memory and disk utilization
l Count of dimensions provisioned
l Export topics status
l Database loading status
l Network status
l Database status
4.7 Devices
The Devices tool is used to configure the InfiniStream devices (probes) feeding the system. To
access, click on the icon in Admin Tools.
Notes:
To edit configuration, click the icon or in the case of editing multiple devices, click the
icon. Then choose ‘Select Data Feeds’ and configure the data sets to be
enabled on the selected device.
You can also search for a device or filter the list of devices, if required.
To edit Device properties, click the click the icon or in the case of editing multiple devices, click the
icon. Then choose ‘Device Enrichment’ and configure the properties for each
device in the popup presented. The user may supply one or more property values (not all properties
need to be filled in) and then click 'Save'. Device Enrichment properties may be cleared by editing
one or more devices and clearing the appropriate field.
Note: Device Region should not be confused with nBA deployment regions. They are independent
concepts, with the former augmenting device data in the database and the latter controlling which
nBA deployment region devices are collected within.
To edit configuration, click the icon or in the case of editing multiple devices, click the
icon. Then choose ‘Set Group’ and enter the group name which will identify the
group.
To edit configuration, click the icon or in the case of editing multiple devices, click the
icon. Then choose ‘Add to Region’ and enter the region name chosen during
installation. The selected region is shown in a column in the device list, which only appears for
regional deployments. You can also filter the list of devices by region, if required.
Note: Any instance of Admin Tools across the regions can be used to configure the devices for all
other regions.
System properties are edited using the configuration tool. To access, click on the icon in Admin
Tools.
Notes:
l Users with the with the ‘nBA Administrator’ privilege, see properties grouped into the ‘shortcut’
folders described below.
l Users with the ‘nBA Netscout Administrator’ privilege need to use the full navigation path.
5.1 Database
Data is stored in 5-minute (‘live’), hourly, daily, weekly and monthly aggregate tables. The length of
time that data is stored for depends on system sizing (disk space) and local security/data protection
guidelines. To check the current data retention settings, click on the shortcut icon.
By default, data retention for each aggregation period is defined by the following properties:
Notes:
l Weekly and monthly data retention is disabled by default, please contact NETSCOUT Support
to enable.
l Additional, more granular properties may exist to overwrite the above at tabular level. If so,
these should be exposed in this view also.
To avoid the database running out of space, you should only change the database retention periods
following a sizing review with NETSCOUT. If a change to the retention periods is approved, edit the
specified property. Save and restart the ‘dbm-ilm’ Containers. See Section 1 Containers on System
Administration Tools for details on restarting containers.
5.2 Explorer
5.2.1 Exposing/Hiding Data Sets
Depending on which data sources are being processed, you can hide unused data sets from users or
expose them if they subsequently become enabled. For example, the ‘Video’ data set is hidden by
default but needs to be exposed if the Video QoE feature is licensed.
Figure 5.1 - Example of the Video Data Set with “Hidden” Set to "True"
Note: Data set configuration is currently not persisted during an upgrade and needs to be re-
applied.
Figure 5.2 - Example of the Hidden "Application_Id" Field in the Control Plane Data Set
Note: User labels are currently not persisted during an upgrade and need to be re-applied.
Caution: You must specify the field names, not the user labels. Inspect the data sets, as
described in the previous sections to identify the right field names to use.
Note: Changes to the Global Filter take effect immediately and do not require a container restart.
Note: Global Filter configuration is currently not persisted during an upgrade and needs to be re-
applied.
Note: Changes to the session timeout property take effect immediately and do not require a
container restart.
System properties are edited using the configuration tool. To access, click on the icon in Admin
Tools.
Notes:
l Users with the with the ‘nBA Administrator’ privilege, see properties grouped into the ‘shortcut’
folders described below.
l Users with the ‘nBA Netscout Administrator’ privilege need to use the full navigation path.
To configure Kafka topics, click on the shortcut icon and edit the ‘topic_config’ property
according to the options described below.
The following tables list the topic names that must exist in the Kafka cluster for each of the default
topics configured in the ‘topic_config’ property.
6.3 Options
Templates for each supported topic type are pre-configured in the ‘topic_config’ property but are disabled by default. To enable each default topic,
simply change “enabled” to ‘true’ and, if required, customize using the options described below.
Note: Changes to the configuration of existing topics, including splitting topics, takes effect immediately and do not require a container restart. The
addition of new topics, not defined by default, require restart of the collection container (or Export Container, if it exists).
Note: It is also possible to filter export using whitelisting and blacklisting. Contact NETSCOUT Support for assistance in configuring this
option.
Caution: Be aware of conflicts with the ‘backwards compatibility’ setting when changing schema in the Avro registry.
1. Nominating a single person to create and administer the metrics in Explorer. This ensures:
l Metric definitions are correct.
l There are no duplicate metrics.
l Metrics are regularly backed up.
This user requires the ‘nBA-Allow access to Explorer Metric Builder’ privilege and MicroStrategy
privileges associated with either the ‘Admin’ user or ‘Web’ user group.
2. Creating custom metrics in a folder structure under a different ‘Organization’ from the
NETSCOUT default metric packs. This will preserve the default metrics for future reference, if
required. New folders can be created per user group, metric type or use case, depending on
what user groups agree on.
3. Exporting metrics for archiving, on a regular basis, to provide a historic record of metric
definitions and guard against accidental deletion or erroneous editing of metrics.
4. Advising all users to export their own dashboards for archiving, on a regular basis, to guard
against accidental deletion or erroneous editing (by themselves!).
5. As some optional features require NETSCOUT metric packs and dashboards, obtaining and
importing these new metric packs and dashboards from NETSCOUT. See nGenius Business
Analytics Explorer User Guide for details on how to import metrics and dashboards.
If the provisioned attributes are not populated in the data, then check that the attributes exist in the
source file and that the provisioning process has been run correctly. Refer to the nGenius Business
Analytics nBA Provisioning Guide for more information on the provisioning process.
Web application access is protected using the HTTPS Secure Sockets Layer (SSL) security protocol.
Command Line Interfaces (CLI) are protected using the Secure Shell (SSH and SFTP) security
protocols.
All applications and command line interfaces (CLI) require named users to authenticate with a
username and password in order to gain access to the system.
All web applications are protected against SQL Injection and Cross Site Scripting (XSS) attacks. Nessus
scans are run during nBA development.
All transmission links which transport information about an individual within and external to the
product are secure (encrypted and require authentication) to avoid eavesdropping. All personal data
at rest (including subscriber information held in a database or file and log files which contains
username) is secured against unauthorized access, manipulation and theft.
User access to this data can be controlled using role-based security. Privileges assigned to users via
nGeniusONE can obfuscate the IMSI and MSISDN in the Explorer application. User groups in
MicroStrategy can obfuscate the IMSI and MSISDN in the business intelligence tool. Direct database
access is protected by system usernames and passwords known only to the persons who installed
and maintain nBA. Refer to Section 3 User Management for more information.
Additionally, during provisioning, subscribers who have requested that their data is not processed,
are tagged with an ‘opt out’ flag. Any data which can identity these subscribers (IMSI, MSISDN, IMEI),
is permanently anonymized (partial SHA-256 hash) when stored in the database. This affects the
‘right to be forgotten’. Refer to the nGenius Business Analytics Provisioning Guide for more
information on the 'opt out' flag.
Always configure named users as opposed to shared, anonymous users e.g. ‘admin’. This applies
equally to users with evaluated privileges and allows auditing of system access. No user should have
‘Root’ privileges. Follow standard Linux user management practices to ensure users only authorized
users have access to file systems and SSH.
Install HA Proxy on a server in a different firewall zone to the rest of the nBA servers in order to
provide a ‘Demilitarized Zone (DMZ)’ separating clients from applications. Clients can only access the
HA Proxy server; the rest is inaccessible behind a firewall.
The Collection process can generate the following Collection Latency type alert: