Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

AddOns Released MSCloudServices

Download as pdf or txt
Download as pdf or txt
You are on page 1of 67

Splunk® Supported Add-ons Splunk

Add-on for Microsoft Cloud Services


released
Generated: 4/27/2018 8:13 am

Copyright (c) 2018 Splunk Inc. All Rights Reserved


Table of Contents
Overview...............................................................................................................1
About the Splunk Add-on for Microsoft Cloud Services..............................1
Source types for the Splunk Add-on for Microsoft Cloud Services.............3
Release notes for the Splunk Add-on for Microsoft Cloud Services...........4
Release history for the Splunk Add-on for Microsoft Cloud Service...........5

Installation..........................................................................................................18
Installation overview for the Splunk Add-on for Microsoft Cloud
Services...................................................................................................18
Hardware and software requirements for the Splunk Add-on for
Microsoft Cloud Services.........................................................................19
Install the Splunk Add-on for Microsoft Cloud Services............................20

Configuration.....................................................................................................23
Configure an Active Directory Application in Azure AD for the Splunk
Add-on for Microsoft Cloud Services........................................................23
Connect to your Microsoft Office 365 account with the Splunk Add-on
for Microsoft Cloud Services....................................................................25
Configure a certificate and private key to enable service-to-service
calls for the Splunk Add-on for Microsoft Cloud Services........................26
Configure Office 365 Management APIs inputs for the Splunk Add-on
for Microsoft Cloud Services....................................................................32
Configure a Storage Account in Microsoft Cloud Services.......................34
Connect to your Azure App Account with Splunk Add-on for Microsoft
Cloud Services.........................................................................................36
Configure Azure Audit Modular inputs for the Splunk Add-on for
Microsoft Cloud Services.........................................................................37
Configure Azure Resource Modular inputs for the Splunk Add-on for
Microsoft Cloud Services.........................................................................39
Connect to your Azure Storage account with the Splunk Add-on for
Microsoft Cloud Services.........................................................................41
Configure Azure Storage Table Modular Input for Splunk Add-on for
Microsoft Cloud Services.........................................................................43
Configure Azure Storage Blob Modular Input for Splunk Add-on for
Microsoft Cloud Services.........................................................................45
Configure Azure Virtual Machine Metrics Modular Input for Splunk
Add-on for Microsoft Cloud Services........................................................49
Troubleshoot the Splunk Add-on for Microsoft Cloud Services.................52

i
Table of Contents
Troubleshooting.................................................................................................56
Troubleshoot the Splunk Add-on for Microsoft Cloud Services.................56

Reference............................................................................................................60
Lookups for the Splunk Add-on for Microsoft Cloud Services...................60
Performance reference for the Azure storage input in the Splunk
Add-on for Microsoft Cloud Services........................................................61
APIs used in the Splunk Add-on for Microsoft Cloud Services.................63

ii
Overview

About the Splunk Add-on for Microsoft Cloud


Services
Version 2.1.0
Microsoft Office 365, Azure Active Directory, Sharepoint
Vendor Products Online, Exchange Online, Azure Storage Table, Azure
Storage Blob, Azure Audit, Azure Resource Group
Splunk platform
6.5 and later
versions
Platforms Platform independent
Visible Yes. This add-on contains views for configuration.
The Splunk Add-on for Microsoft Cloud Services allows a Splunk administrator to
pull activity logs, service status, operational messages, Azure audit, Azure
resource data and Azure Storage Table and Blob data from a variety of Microsoft
Cloud services using the Office 365 Management APIs, Azure Service
Management APIs, and Azure Storage APIs.

After the Splunk platform indexes the events, you can analyze the data using the
prebuilt panels included with the add-on. This add-on provides the inputs and
CIM-compatible knowledge to use with other Splunk apps. See Splunk Enterprise
Security, Splunk App for PCI Compliance, and Splunk IT Service Intelligence.

The following table below lists the APIs that are used in the Splunk Add-on for
Microsoft Cloud Service.

Inputs
Method Description (Link to Microsoft site)
Name
Azure Query Tables https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx
Storage Query
Table https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx
Entities
Azure List Blobs https://msdn.microsoft.com/en-us/library/azure/dd135734.aspx
Storage
Blob Get Blob https://msdn.microsoft.com/en-us/library/azure/dd179440.aspx
https://msdn.microsoft.com/en-us/library/azure/dn931934.aspx

1
Azure Azure
Audit Insights - List
the
management
events
List all virtual
machines in a
https://msdn.microsoft.com/en-us/library/azure/mt163572.aspx
Azure resource
Resource group
- Virtual Get the
Machine instance view
https://msdn.microsoft.com/en-us/library/azure/mt589031.aspx
of a virtual
machine
List public IP
Azure
addresses
Resource
within a https://msdn.microsoft.com/en-us/library/azure/mt163657.aspx
- Public IP
resource
Address
group
Azure List network
Resource interface
- Network cards within a https://msdn.microsoft.com/en-us/library/azure/mt163627.aspx
Interface resource
Card group.
List virtual
Azure
networks
Resource
within a https://msdn.microsoft.com/en-us/library/azure/mt163587.aspx
- Virtual
resource
Network
group
Azure Query Tables https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx
Virtual
Machine Query https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx
Metrics Entities
Download the Splunk Add-on for Microsoft Cloud Services from Splunkbase at
https://splunkbase.splunk.com/app/3110/.

Discuss the Splunk Add-on for Microsoft Cloud Services on Splunk Answers at
https://answers.splunk.com/app/questions/3110.html.

2
Source types for the Splunk Add-on for Microsoft
Cloud Services
The Splunk Add-on for Microsoft Cloud Services provides the index-time and
search-time knowledge for Microsoft cloud services data in the following formats.

CIM d
Source type Event type Description
mod

File and directory


Change
mso365_change_endpoint change events in
Analysis
SharePoint.
ms:o365:management Account change Change
mso365_change_account
events in SharePoint. Analysis
Authentication events
mso365_authentication in Microsoft Azure Authent
AD.
Virtual machine
mscs:resource:virtualMachine
metadata.
mscs_inventory_vm
Azure
mscs:resource:networkInterfaceCard networkInterfaceCard
metadata.
Azure
mscs:resource:publicIPAddress publicIPAddress
metadata.
Azure virtualNetwork
mscs:resource:virtualNetwork
None metadata.
None
Azure audit log
mscs:azure:audit
events.
Data relevant to
mscs:storage:table Azure Storage
Tables.
Events relevant to
virtual machine
metrics, such as CPU
mscs:vm:metrics mscs_perf_vm_cpu
usage, memory usage
and memory
provisioning data.

3
Data relevant to
mscs:storage:blob None
Azure Storage Blobs.

Release notes for the Splunk Add-on for Microsoft


Cloud Services
About this release

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services is compatible
with the following software, CIM versions, and platforms.

Splunk
platform 6.5 and later
versions
CIM 4.4 and later
Platforms Platform independent
Microsoft Office 365, Azure Active Directory, Sharepoint Online,
Vendor
Exchange Online, Azure Storage Table, Azure Storage Blob,
Products
Azure Audit, and Azure Resource Group.
New Features

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Service has the following
new features and enhancements.

• Support for Office365 Government Cloud


• Support for Azure Government Cloud
• Support for the Audit General class of Office365 events

Fixed issues

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services fixes the
following issues.

Date
Issue number Description
resolved
Microsoft Cloud Services - Table is not
2018-01-22 ADDON-16769
unique per account/region
2018-01-19 ADDON-15540 Not Receiving MSCS data

4
ADDON-15008, Wrong account number shows in Azure
2017-09-05
ADDON-11154 App account page
Unable to get information from default
ADDON-13410,
2017-08-31 metric azure tables that are using the name
ADDON-14132
convention $Metrics
Add Audit.General endpoint subscription
2017-05-03 ADDON-12428
needed
2017-03-06 ADDON-11505 Table is not unique per account/region
Known issues

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services contains the
following new known issues.

Date filed Issue number Description


2018-01-09 ADDON-16542 UI Error on Inputs Tab for Audit.General data
2017-08-15 ADDON-15540 Not Receiving MSCS data
Third-party software attributions

Version 2.1.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the
following third-party software or libraries.

• Bootstrap
• httplib2
• pythonfutures
• remote-pdb113
• select2
• SortedContainers
• Azure SDK for Python

Release history for the Splunk Add-on for Microsoft


Cloud Service
Latest release

The latest version of the Splunk Add-on for Microsoft Cloud Service is version
2.1.0. See Release notes for the Splunk Add-on for Microsoft Cloud Service for
the release notes of this latest version.

5
Version 2.0.3

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Services is compatible
with the following software, CIM versions, and platforms.

Splunk
platform 6.4 and later
versions
CIM 4.4 and later
Platforms Platform independent
Microsoft Office 365, Azure Active Directory, Sharepoint Online,
Vendor
Exchange Online, Azure Storage Table, Azure Storage Blob,
Products
Azure Audit, and Azure Resource Group.
New Features

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Service has the following
new features and enhancements.

• Enhanced stability and performance in data collection through the O365


Management APIs
• Updates to pagination handling for the O365 Management Activity APIs
• Added proxy support for Audit and Resource data inputs
• Optimized performance for the Diagnostics and websitesapplogs tables

Fixed issues

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Service fixes the following
issues.

Date
Issue number Description
resolved
Error message in internal log for O365
2017-06-09 ADDON-14908
Sharepoint
Splunk_TA_microsoft-cloudservices contains
2017-06-06 ADDON-14248 long path names which exceed Windows 260
path length limit

6
Known issues

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Services contains the
following new known issues.

Date filed Issue number Description


Microsoft Cloud Services - Table is not
2018-01-22 ADDON-16769
unique per account/region
2017-08-15 ADDON-15540 Not Receiving MSCS data
Event with same Id may be fetched
2017-07-20 ADDON-15343 several times for O365 Azure AD Audit
due to O365 management API behaviour
Fails to encrypt account when multiple
account and input are configured at the
same time through backend

Workaround:
2017-07-17 ADDON-15300 Perform one of the following: 1. Add
accounts through the Web UI
2. After adding multiple accounts by
editing the configuraton files, open
the add-on configuration page in a
browser before adding new inputs.

Possible data duplication after


2017-06-23 ADDON-15129 disable/enable O365 data input during
data collection
ADDON-15008, Wrong account number shows in Azure
2017-06-07
ADDON-11154 App account page
Proxy type Sock4/Sock5 is not
2017-05-24 ADDON-14876
supported in Resouce/Audit channel
The start_time cannot be deleted for
2017-05-11 ADDON-14748
Audit input
Error happens during upgrading TA

2017-02-06 ADDON-13476 Workaround:


Disable TA before upgrade, re-enable it
after upgrade done.
2016-11-21 ADDON-12262 Local files generated immediately after

7
install the TA
2016-10-06 ADDON-11505 Table is not unique per account/region
ADDON-11419,
ADDON-11413, same inputs name with different case
2016-09-22 ADDON-11510, have problems with check-points on
ADDON-12585, windows
ADDON-11606
Data cannot be collected if blob name
2016-09-22 ADDON-11423
contains special characters
Add-on throws "Failed to load endpoint",
"Refresh token failed", "Failed to init
ADDON-11316,
2016-09-18 ServerInfo", "Failed to send rest request"
ADDON-8280
errors during restart after initial
installation
Proxy type and DNS Resolution
2016-09-04 ADDON-11164
configuration does not work for storage
Fails to get VM meta data in classic
2016-08-22 ADDON-10984
category
Third-party software attributions

Version 2.0.3 of the Splunk Add-on for Microsoft Cloud Services incorporates the
following third-party software or libraries.

• Bootstrap
• httplib2
• pythonfutures
• remote-pdb113
• select2
• SortedContainers
• Azure SDK for Python

Version 2.0.2

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Services is compatible
with the following software, CIM versions, and platforms.

Splunk
platform 6.4 and 6.5
versions

8
CIM 4.4 or later
Platforms Platform independent
Microsoft Office 365, Azure Active Directory, Sharepoint Online,
Vendor
Exchange Online, Azure Storage Table, Azure Storage Blob,
Products
Azure Audit, and Azure Resource Group.
Fixed issues

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Service fixes the following
issues.

Publication
Issue number Description
Date
Cannot use proxy without Authentication in
2017/02/20 ADDON-12556
Storage channel.
The length of the checkpoint file name exceeds
2017/02/20 ADDON-12665
the limitation of the operating system.
Cannot parse SAS token which is not start with
2017/02/20 ADDON-12666
'?'.
Known issues

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Services contains the
following new known issues.

Date Issue number Description


Truncated Key/value pairs in Splunk Add-on for
2017/06/02 ADDON-14969
Microsoft Cloud Services.
The proxy value you configured in this add-on
cannot be used for the Azure resource and Azure
audit input channel.
2017/02/07 ADDON-13487
Workaround: Configure the proxy on the local
system for Azure resource and Azure audit input
channel.
2017/02/06 ADDON-13476 Error occurs during upgrading Splunk add-on for
Microsoft cloud service on Windows platform.

Workaround: If you want to upgrade this add-on


on Windows platform, disable the add-on first, then

9
enable it after upgrading.
For the known issues in the previous release, see release history of the Splunk
add-on for Microsoft cloud service.

Third-party software attributions

Version 2.0.2 of the Splunk Add-on for Microsoft Cloud Services incorporates the
following third-party software or libraries.

• Bootstrap
• httplib2
• pythonfutures
• remote-pdb113
• select2
• SortedContainers
• Azure SDK for Python

Version 2.0.1

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Services is compatible
with the same software, CIM versions and platforms as Version 2.0.2.

Fixed issues

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Service fixes the following
issues.

Resolved
Issue number Description
Date
Only the first 30 inputs (in the alphabet order) of
Azure Storage Table (including Virtual Machine
Metrics) can work.
2016/10/14 ADDON-10454
Only the first 30 Azure Storage Blob inputs (in
the alphabet order) can work.
Known issues

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Services contains the
following known issues.

Date Issue number Description

10
This add-on does not check the input name stanza at the
2016-10-13 ADDON-11638
frontend.
This add-on fails to configure the certificate in the latest
2016-10-12 ADDON-11609
Firefox browser.
This add-on can only get data when blob name in Microsoft
Cloud Service only contains ASCII code. It cannot get data
2016-09-24 ADDON-11423
if the blob name contains multibyte character set, such as
Latin characters, Japanese characters.
If the names of the Azure storage blob inputs under the
same account are the same except the case, such as
INPUTS and inputs, the checkpoint conflicts to each other
2016-09-20 ADDON-11419
on Windows platform.

This issue also exists in other modular inputs.


The changes in the inputs.conf won't take effect until
2016-09-20 ADDON-11409
restarting Splunk platform.
If you set the log level to ERROR for Azure Audit and Azure
2016-09-20 ADDON-11400 Blob input, there are still some INFO level logs recorded in
the log file.
The error message error_message=The range specified
is invalid for the current size of the resource exists
2016-09-19 ADDON-11349 in the log file if the blob input has been collected and
revised later to a smaller size. The error message can be
ignored.
There will be some errors, such as Failed to load
endpoint, Refresh token failed, Failed to init ServerInfo
2016-09-19 ADDON-11316 or Failed to send rest request in the log file when you
restart Splunk platform. But it does not effect data
collection.
There will be some data loss if the Splunk platform restart
or shutdown accidently.
2016-09-15 ADDON-11298
Workaround: If you need to restart Splunk platform, you
have to disable the inputs beforehand to prevent the data
loss.
You can only add the Office365 account via Splunk web,
2016-09-09 ADDON-11178
you can not add it using the configuration file.
2016-09-05 ADDON-11164 The Proxy Type and DNS Resolution settings do not work

11
for Azure Storage Table and Azure Storage Blob input.
2016-08-23 ADDON-10984 This add-on cannot get Virtual Machine (classic) metadata.
Splunk searches sometimes display duplicate events. This
2016/03/30 ADDON-8505 is a known issue with the Microsoft Office 365 Management
API.
Splunk searches sometimes display events out of order.
2016/03/30 ADDON-8504 This is a known issue with the Microsoft Office 365
Management API.
Stanza "o365_certificate_setting" in
2016/03/29 ADDON-8432 splunk_ta_ms_o365_server_ucc_system_setting.conf.spec
has incorrect default values.
Certificate status messages "* but invalid" should not
2016/03/29 ADDON-8424
appear until a longer time has passed.
If you configure an X.509 certificate and private key and
upload the keyCredentials JSON for any integration
2016/03/08 ADDON-8221 account configured in the add-on, you also need to be
uploaded it for all other accounts configured in the add-on,
or any accounts not using the certificate cannot collect data.
Management log reports rest request error during Splunk
2016/01/31 ADDON-7653 platform stop/restart immediately after a configuration
change. This error can be ignored.
Input will stop when the proxy_url exists but is invalid as a
2016/01/26 ADDON-7597 proxy. Workaround: Change your proxy URL to a valid
proxy value.
Third-party software attributions

Version 2.0.1 of the Splunk Add-on for Microsoft Cloud Services incorporates the
following third-party software or libraries.

• Bootstrap
• httplib2
• pythonfutures
• remote-pdb113
• select2
• SortedContainers
• Azure SDK for Python

12
Version 2.0.0

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services is compatible
with the same software, CIM versions and platforms as Version 2.0.1.

New features

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services has the
following new features.

Date Issue number Description


2016/09/20 ADDON-10883 Mapping to Cloud of ITSI data model.
2016/09/20 ADDON-10728 Add modular input for Azure Storage Blob data.
2016/09/20 ADDON-10727 Add modular input for Azure Storage Table data.
2016/09/20 ADDON-10129 Add modular input for Azure Audit data.
2016/09/20 ADDON-10696 Add modular input for Azure Resource data.
Add modular input for Azure Virtual Machine
2016/09/20 ADDON-10222
Metrics data.
Fixed issues

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Service fixes the following
issues.

Resolved
Issue number Description
Date
If there is space in the name of inputs or account,
2016-09-05 ADDON-11033
this add-on will fail to ingest data.
This add-on does not work if you install the
2016-07-19 ADDON-9329 add-on under /etc/apps/SPLUNK_HOME/ect/apps
folder
If the global proxy is enabled in
2016-08-30 ADDON-8735 splunk-launch.conf, the add-on cannot display
the Account or Proxy tab under Configuration.
Known issues

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services contains the
following known issues.

13
Date Issue number Description
Only the first 30 inputs (in the alphabet order) of Azure
Storage Table (including Virtual Machine Metrics) can work.

2016-09-27 ADDON-10454 Only the first 30 Azure Storage Blob inputs (in the alphabet
order) can work.
Workaround: You can reduce the number of inputs by
using wildcard or regex expression in the Blob list.
This add-on can only get data when blob name in Microsoft
Cloud Service only contains ASCII code. It cannot get data
2016-09-24 ADDON-11423
if the blob name contains multibyte character set, such as
Latin characters, Japanese characters.
If the names of the Azure storage blob inputs under the
same account are the same except the case, such as
INPUTS and inputs, the checkpoint conflicts to each other
2016-09-20 ADDON-11419
on Windows platform.

This issue also exists in other modular inputs.


The changes in the inputs.conf won't take effect until
2016-09-20 ADDON-11409
restarting Splunk platform.
If you set the log level to ERROR for Azure Audit and Azure
2016-09-20 ADDON-11400 Blob input, there are still some INFO level logs recorded in
the log file.
The error message error_message=The range specified
is invalid for the current size of the resource exists
2016-09-19 ADDON-11349 in the log file if the blob input has been collected and
revised later to a smaller size. The error message can be
ignored.
There will be some errors, such as Failed to load
endpoint, Refresh token failed, Failed to init ServerInfo
2016-09-19 ADDON-11316 or Failed to send rest request in the log file when you
restart Splunk platform. But it does not effect data
collection.
There will be some data loss if the Splunk platform restart
or shutdown accidently.
2016-09-15 ADDON-11298
Workaround: If you need to restart Splunk platform, you
have to disable the inputs beforehand to prevent the data
loss.

14
You can only add the Office365 account via Splunk web,
2016-09-09 ADDON-11178
you can not add it using the configuration file.
The Proxy Type and DNS Resolution settings do not work
2016-09-05 ADDON-11164
for Azure Storage Table and Azure Storage Blob input.
2016-08-23 ADDON-10984 This add-on cannot get Virtual Machine (classic) metadata.
Splunk searches sometimes display duplicate events. This
2016/03/30 ADDON-8505 is a known issue with the Microsoft Office 365 Management
API.
Splunk searches sometimes display events out of order.
2016/03/30 ADDON-8504 This is a known issue with the Microsoft Office 365
Management API.
Stanza "o365_certificate_setting" in
2016/03/29 ADDON-8432 splunk_ta_ms_o365_server_ucc_system_setting.conf.spec
has incorrect default values.
Certificate status messages "* but invalid" should not
2016/03/29 ADDON-8424
appear until a longer time has passed.
If you configure an X.509 certificate and private key and
upload the keyCredentials JSON for any integration
2016/03/08 ADDON-8221 account configured in the add-on, you also need to be
uploaded it for all other accounts configured in the add-on,
or any accounts not using the certificate cannot collect data.
Management log reports rest request error during Splunk
2016/01/31 ADDON-7653 platform stop/restart immediately after a configuration
change. This error can be ignored.
Input will stop when the proxy_url exists but is invalid as a
2016/01/26 ADDON-7597 proxy. Workaround: Change your proxy URL to a valid
proxy value.
Third-party software attributions

Version 2.0.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the
following third-party software or libraries.

• Bootstrap
• httplib2
• pythonfutures
• remote-pdb113
• select2
• SortedContainers
• Azure SDK for Python

15
Version 1.0.0

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services was released on
April 1, 2016. Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services is
compatible with the following software, CIM versions, and platforms.

Splunk platform
6.3.X or later
versions
CIM 4.4 or later
Platforms Platform independent
Microsoft Office 365, Azure Active Directory, Sharepoint
Vendor Products
Online, Exchange Online, and other cloud services.
New features

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services has the
following new features.

Date Issue number Description


2016/03/10 ADDON-3941 Create a new add-on for Microsoft cloud services.
Known issues

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services contains the
following known issues.

Issue
Date Description
number
Splunk searches sometimes display duplicate events. This
2016/03/30 ADDON-8505 is a known issue with the Microsoft Office 365 Management
API.
Splunk searches sometimes display events out of order.
2016/03/30 ADDON-8504 This is a known issue with the Microsoft Office 365
Management API.
Stanza "o365_certificate_setting" in
2016/03/29 ADDON-8432 splunk_ta_ms_o365_server_ucc_system_setting.conf.spec
has incorrect default values.
Certificate status messages "* but invalid" should not
2016/03/29 ADDON-8424
appear until a longer time has passed.
2016/03/15 ADDON-8280

16
Add-on throws "Failed to send rest request" errors during
restart after initial installation unless the user waits for about
one minute after installing the add-on and before restarting
the Splunk platform. Workaround: Restart the Splunk
platform a second time.
If you configure an X.509 certificate and private key and
upload the keyCredentials JSON for any integration
2016/03/08 ADDON-8221 account configured in the add-on, you also need to be
upload it for all other accounts configured in the add-on, or
any accounts not using the certificate cannot collect data.
Management log reports rest request error during Splunk
2016/01/31 ADDON-7653 platform stop/restart immediately after a configuration
change. This error can be ignored.
Input will stop when the proxy_url exists but is invalid as a
2016/01/26 ADDON-7597 proxy. Workaround: Change your proxy URL to a valid
proxy value.
Third-party software attributions

Version 1.0.0 of the Splunk Add-on for Microsoft Cloud Services incorporates the
following third-party software or libraries.

• Bootstrap
• httplib2
• pythonfutures
• remote-pdb113
• select2
• SortedContainers

17
Installation

Installation overview for the Splunk Add-on for


Microsoft Cloud Services
Complete the following steps to install and configure this add-on on your
supported platform.

1. Review the Hardware and software requirements for the Splunk Add-on
for Microsoft Cloud Services.
2. Install the Splunk Add-on for Microsoft Cloud Services.
3. If you want to collect Office 365 Management API data, perform the
following steps.
♦ Configure an Active Directory Application in Azure AD for the
Splunk Add-on for Microsoft Cloud Services.
♦ Connect to your Microsoft Office 365 account with the Splunk
Add-on for Microsoft Cloud Services.
♦ Configure a certificate and private key to enable service-to-service
calls for the Splunk Add-on for Microsoft Cloud Services.
♦ Configure inputs for the Splunk Add-on for Microsoft Cloud
Services.
4. If you want to collect Azure Resource or Azure Audit data, perform the
following steps.
♦ Configure an Active Directory Application in Azure AD for the
Splunk Add-on for Microsoft Cloud Services.
♦ Connect to your Azure App Account with Splunk Add-on for
Microsoft Cloud Services.
♦ Configure Azure Audit Modular inputs for the Splunk Add-on for
Microsoft Cloud Services or Configure Azure Resource Modular
inputs for the Splunk Add-on for Microsoft Cloud Services.
5. If you want to collect Azure Storage Table, Azure Storage Blob or Azure
Virtual Machine Metrics data, perform the followiing steps.
♦ Configure a Storage Account in Microsoft Cloud Service.
♦ Connect to your Azure Storage account with the Splunk Add-on for
Microsoft Cloud Services.
♦ Configure Azure Storage Table Modular Input for Splunk Add-on for
Microsoft Cloud Services, Configure Azure Storage Blob Modular
Input for Splunk Add-on for Microsoft Cloud Services or Configure
Azure Virtual Machine Metrics Modular Input for Splunk Add-on for
Microsoft Cloud Services .

18
Hardware and software requirements for the Splunk
Add-on for Microsoft Cloud Services
Microsoft account and related modular inputs

Splunk add-on for Microsoft Cloud Service uses three types of Microsoft account
to collect data, Office 365 account, Azure App account and Azure Storage
account.

• If you want to collect data from Office 365 Management API input, you
need to apply for Office 365 account in Microsoft Cloud Service and
connect to it in Splunk add-on for Microsoft Cloud Service.
• If you want to collect data from Azure audit and Azure resource input,
you need to apply for Azure app account first and then connect to it in the
Splunk add-on for Microsoft Cloud Service.
• If you want to collect data from Azure Storage Table input (including
Virtual machine metrics) and Azure Storage Blob input, you need to
apply for the Azure Storage account first and then connect it in the Splunk
add-on for Microsoft Cloud Services.

Microsoft account permission requirements

In order to collect data from Office365, Azure audit and Azure resource, you
need to configure an Azure Active Directory Application with read permissions.
See more on Add permissions to your Active Directory Application
In order to collect data from Azure storage table and Azure storage blob, see
configure storage account to get data.

Exchange audit logging

To collect audit logs for mailbox access from Exchange Online, you need to turn
on mailbox audit logging in Office 365. See Enable mailbox auditing in Office 365
for instructions.

Note: Exchange audit logging only logs activity by administrators and delegates
by default. If you want to log actions by mailbox owners as well, configure which
owner actions you want to include in the log.

19
Splunk platform requirements

The Splunk Add-on for Microsoft Cloud Services requires Splunk platform version
6.5 and above.

Because this add-on runs on the Splunk platform, all of the system requirements
apply for the Splunk software that you use to run this add-on.

• If you plan to run this add-on entirely in Splunk Cloud, contact Splunk
Support for guidance and assistance.
• If you manage on-premises forwarders to get data in to Splunk Cloud, see
System Requirements in the Installation Manual in the Splunk Enterprise
documentation, which includes information about forwarders.
• If you plan to run this add-on in an on-premises deployment of the Splunk
platform, see System Requirements in the Installation Manual in the
Splunk Enterprise documentation.
• If you plan to run this app in a self-managed AWS instance, there are no
additional requirements. Refer to the Virtual hardware information for
sizing considerations specific to AWS.

For information about installation locations and environments, see Install the
Splunk Add-on for Microsoft Cloud Services.

Splunk role requirements

To use this add-on's configuration UI, you need the admin role.

To use this add-on's troubleshooting dashboard, no special roles are required.

Install the Splunk Add-on for Microsoft Cloud


Services
Installation walkthrough

Refer to Installing add-ons for detailed instructions describing how to install a


Splunk add-on in the following deployment scenarios:

• single-instance Splunk Enterprise


• distributed Splunk Enterprise
• Splunk Cloud

20
Distributed deployments

Use the tables below to determine where and how to install this add-on in a
distributed deployment of Splunk Enterprise.

Where to install this add-on

This table provides a quick reference for installing this add-on to a distributed
deployment of Splunk Enterprise.

Splunk
instance Supported Required Comments
type
Install this add-on to all search heads
where Microsoft Cloud Services
knowledge management is required.
Splunk recommends that you turn
Search
Yes Yes visibility off on your search heads to
Heads
prevent data duplication errors that can
result from running inputs on your
search heads instead of (or in addition
to) on your data collection node.
Not required if you use heavy
Indexers Yes No forwarders to collect Microsoft Cloud
Services data.
Heavy This add-on only supports heavy
Yes Yes
Forwarders forwarders for data collection.
Universal forwarders are not supported
Universal for data collection, because the modular
No No
Forwarders inputs require Python and the Splunk
REST handler.
Distributed deployment feature compatibility

This table provides a quick reference for the compatibility of this add-on with
Splunk distributed deployment features.

Distributed
deployment Supported Comments
feature
Yes

21
Search Head Disable add-on visibility on search heads.
Clusters You can install this add-on on a search head
cluster for all search-time functionality, but
configure inputs on forwarders to avoid
duplicate data collection.
Before installing this add-on to a cluster, make
the following changes to the add-on package:
1. Remove the eventgen.conf files and all files
in the samples folder
2. Remove the inputs.conf file.
Before installing this add-on to a cluster, make
the following changes to the add-on package:
Indexer Clusters Yes 1. Remove the eventgen.conf files and all files
in the samples folder
2. Remove the inputs.conf file.
Supported for deploying the unconfigured
Deployment add-on only. Configure this add-on using the
No
Server add-on's configuration UI on your data
collection node(s).

22
Configuration

Configure an Active Directory Application in Azure


AD for the Splunk Add-on for Microsoft Cloud
Services
In order to gather data from the Microsoft Office 365 Management APIs and
Windows Azure Service Management APIs, you must first create an active
directory application in Azure AD. This application securely authenticates the
Splunk Add-on for Microsoft Cloud Services via the OAuth2 protocol, so that it
can access and gather the data according to the services and permission levels
that you specify.

Obtain a redirect URL for your application

Note: This is an optional procedure, you only need to perform this step if you
need to configure the Microsoft Office365 account in Splunk Add-on for Microsoft
Cloud Service

As part of the registration of your application in Azure AD, you need to supply a
redirect URL that Azure can use to authenticate the Splunk Add-on for Microsoft
Cloud Services. To determine what this URL should be:

1. Navigate to the Splunk Add-on for Microsoft Cloud Services on the Splunk
platform node that is responsible for collecting data for this source.
2. Open the Configuration tab, then click Add Account.
3. The window displays a Redirect URL for this Splunk platform instance.
Copy it to your clipboard.

If your data collection node does not have a Splunk Web UI, use http://<host
and port of your Splunk
server>/en-US/app/Splunk_TA_microsoft-cloudservices/redirect as your
redirect URL.

Create an application in Microsoft Azure AD

Follow the instructions in the Microsoft documentation to create an active


directory application: Use portal to create an Azure Active Directory application
and service principal that can access resources for either your Azure portal or
Azure Government portal.

23
When prompted, select or enter the following parameters.

Sign-on URL and App ID URI: Required for Microsoft Office 365 account.
These are irrelevant for the Splunk Add-on for Microsoft Cloud Services.
Enter any valid URIs.

Reply URL: Required for Microsoft Office 365 account.


Enter the redirect URL from the step Obtain a redirect URL for your
application.

Client ID: Required for Microsoft Office 365 and Azure App account.
Copy this value. You need this value and a valid secret key to connect to
your account from the add-on.

Key: Required for Microsoft Office 365 and Azure App account.
Copy this value to a secure location as soon as the Azure AD admin
console displays it.

X.509 certificate: Required for Microsoft Office 365 account.


Skip this section of the instructions for now. You can add this later,
following the instructions in Configure a certificate and private key to
enable service-to-service calls for the Splunk Add-on for Microsoft Cloud
Services.

Tenant ID: Required for Azure App account. Copy this value for the future use.

Add permissions to your Active Directory Applications

Application permissions to access Office 365 Management APIs

In order to get data from Office 365 management APIs, you need to add the
Office 365 Management APIs to the permissions to other applications list.
Select all the required permissions listed under both Application Permissions and
Delegated Permissions:

• Read activity reports for your organization


• Read activity data for your organization
• Read service health information for your organization

For detailed instructions, see the permissions your app requires to access the
Office 365 Management APIs on MSDN.

24
Office 365 tenant admin consent

Now that the add-on is configured with the permissions it needs to use the Office
365 Management APIs, a tenant admin must explicitly grant the add-on these
permissions in order to access their tenant's data by using the APIs. To grant
consent, the tenant admin must log in to Azure AD, using the following specially
constructed URL, where they can review your add-on's requested permissions.
This step is not required when using the APIs to access data from your own
tenant.

For detailed information, see 365 tenant admin consent on MSDN.

Application permissions to access Windows Azure Service Management


APIs

Select Access Azure Service Management as organization under Delegated


Permissions.

Grant the Active Directory Application Read Access

Note: This is an optional procedure, you only need to perform this step if you
need to configure the Azure App account in Splunk Add-on for Microsoft Cloud
Service.

After creating the Active Directory Application, login to either the Azure portal or
the Azure Government portal to grant this application the read access to
Microsoft Cloud Service (You must have a Premium P1 Active Directory level
edition or higher to perform this operation). See Use portal to create an Azure
Active Directory application and service principal that can access resources for
more information.

Connect to your Microsoft Office 365 account with


the Splunk Add-on for Microsoft Cloud Services
Set up integration between the Splunk Add-on for Microsoft Cloud Services and
your Microsoft Office 365 account so that you can ingest your Microsoft cloud
services data into the Splunk platform.

Note: You can only connect to your account using Splunk Web, configure
Microsoft Office 365 account via configuration file is not supported.

25
Prerequisite: Before you complete these steps, follow the directions in Configure
an Active Directory Application in Azure AD for the Splunk Add-on for Microsoft
Cloud Services to prepare your Microsoft account for this integration.

Connect to your account using Splunk Web

1. Clear your cache, start a new browser session, or use a different browser than
the one you use to sign in to the Azure AD admin console. This best practice
helps to avoid issues with incorrectly cached credentials that interfere with the
OAuth dance.

2. Access Splunk Web on the node of your Splunk platform installation that
collects data for this add-on.

3. Launch the add-on, then click Configuration > O365 account.

4. Click Account > Add Account.

5. Enter a friendly Name for the account.

6. Chose the account type, public or GCC High if you are using the high-security
government version.

7. Enter the Client ID that Azure AD automatically assigned to your integration


application.

8. In the Key (Client Secret) field, enter the secret key that you created for your
application in the Azure AD console.

9. Click Add.

10. The Splunk Add-on for Microsoft Cloud Services authenticates using the
client ID and secret you provided. Microsoft prompts you to log in with your
account credentials to complete the authentication.

Next step: Configure a certificate and private key to enable service-to-service


calls for the Splunk Add-on for Microsoft Cloud Services.

Configure a certificate and private key to enable


service-to-service calls for the Splunk Add-on for

26
Microsoft Cloud Services
Note: This step is only used when you need to configure Microsoft Office 365
Management APIs inputs. If you don't have to configure Microsoft Office 365
Management APIs inputs, you can skip this step.

This add-on uses OAuth to authenticate from the Splunk platform to your
Microsoft Office 365 account using an authorization token refreshed
automatically with a refresh token. This authorization token has a mandatory
expiration set by Microsoft, so the refresh token only keeps your integration
current for a limited period. To avoid having to periodically re-enter a secret key
manually, you can upload a Base64-encoded X.509 Certificate and private key to
enable service-to-service calls and use the key credentials to update the
manifest of your integration application in Azure AD.

If you are using the configuration files to configure your connection to your
Microsoft cloud services, this procedure is mandatory. If you are using Splunk
Web, this procedure is highly recommended.

If you skip this step, then when your authorization token expires, you will need to
edit your account configuration that handles your connection to Microsoft Office
365 by entering a new secret key from the Azure AD admin console.

Configure a certificate and private key

You can configure the certificate and private key in Splunk Web on your data
collection node (recommended), or in the configuration files.

Configure a certificate in Splunk Web

1. In Splunk Web on the instance responsible for data collection with this
add-on, go to the Splunk Add-on for Microsoft Cloud Services >
Configuration.
2. Click Certificate.
3. Choose one of the two options.
Option 1: Upload Using your preferred tool, generate a X.509
your own certificate certificate file and a private key with a length of at
and private key least 2048 characters and upload them on this
screen. For more information about using
self-signed certificates, see How to self-sign
certificates. Click Choose a Certificate and browse
to the certificate file (.cer) in your file system.

27
You need to decrypt the private key before you
upload it on Splunk add-on for Microsoft Cloud
Service.
Option 2: Use an Choose this option if you want to use a certificate
auto-generated that the Splunk Add-on for Microsoft Cloud
certificate Services auto-generates for you.
4. The add-on displays the keyCredentials JSON object for your certificate.
5. Copy the entire JSON object to your clipboard.

Next, see Upload the certificate credentials to your integration application in


Azure AD.

Configure a certificate using the configuration files

1. Generate a Base64-encoded X.509 certificate and put it in


$SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/certificate.cer.

Make sure the certificate is a X.509 certificate and the key length is at
least 2048 . Shorter key lengths are not accepted by Microsoft Office 365
as valid keys.
2. Create
$SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_cert
and add the following stanza.

[certificate]
private_key = <Your private key, using '\' as link breaker>
3. Next, you need to obtain the keyCredentials JSON object. Run

python
$SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/bin/splunk_ta_microsoft_office365/g
$SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/certificate.cer
4. Copy the results to the manifest_json field in
$SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_cert

Once your certificate is created using Splunk Web or using the configuration files,
it will look like the following example.

"keyCredentials": [{"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977",


"customKeyIdentifier": "3wLEeFNsctRc+jWX057nFP0+QD8=", "value":
"MIIDETCCAfkCCG2f2svvWEwAMA0GCSqGSIb3DQEBDQUAMCQxDDAKBgNVBAMMA3NvMTEUMBIGA1UECgwLU2VsZiB
"type": "AsymmetricX509Cert", "usage": "Verify"}]
For more information about using self-signed certificates, see How to self-sign
certificates.

28
Next, continue with the procedure in the next section.

Upload the certificate credentials to your integration


application in Azure AD

1. Sign in to the Azure management portal and navigate to the integration


application that you created in Connect to your Microsoft Office 365
account with the Splunk Add-on for Microsoft Cloud Services.
2. Click Manage Manifest > Download Manifest. It will look similar to the
below example.

{
"appId": "0399fdb3-c651-4360-ae33-97ed0598b5af",
"appRoles": [],
"availableToOtherTenants": false,
"displayName": "zliang-test-app",
"errorUrl": null,
"groupMembershipClaims": null,
"optionalClaims": null,
"acceptMappedClaims": null,
"homepage": "http://localhost:8000",
"identifierUris": [
"https://a830edad9050849NDA3079.onmicrosoft.com/6136b06e-df48-4776-82c0-424641
],
"keyCredentials": [],
"knownClientApplications": [],
"logoutUrl": null,
"oauth2AllowImplicitFlow": false,
"oauth2AllowUrlPathMatching": false,
"oauth2Permissions": [
{
"adminConsentDescription": "Allow the application to access
zliang-test-app on behalf of the signed-in user.",
"adminConsentDisplayName": "Access zliang-test-app",
"id": "8448c8ef-a250-481e-ba5c-d877badd3e07",
"isEnabled": true,
"type": "User",
"userConsentDescription": "Allow the application to access
zliang-test-app on your behalf.",
"userConsentDisplayName": "Access zliang-test-app",
"value": "user_impersonation"
}
],
"oauth2RequirePostResponse": false,
"objectId": "aa082da8-0f43-4a09-a364-630f4df75a62",
"passwordCredentials": [],
"publicClient": false,
"replyUrls": [
"http://localhost:8000"

29
],
"requiredResourceAccess": [
{
"resourceAppId": "00000002-0000-0000-c000-000000000000",
"resourceAccess": [
{
"id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
"type": "Scope"
}
]
}
],
"samlMetadataUrl": null
}
3. Open the manifest in a text editor.
4. Place your cursor inside the empty brackets after "keyCredentials": and
replace the keyCredentials key-value pair with the one generated in your
configured certificate. See the below example.

{
"appId": "0399fdb3-c651-4360-ae33-97ed0598b5af",
"appRoles": [],
"availableToOtherTenants": false,
"displayName": "zliang-test-app",
"errorUrl": null,
"groupMembershipClaims": null,
"optionalClaims": null,
"acceptMappedClaims": null,
"homepage": "http://localhost:8000",
"identifierUris": [
"https://a830edad9050849NDA3079.onmicrosoft.com/6136b06e-df48-4776-82c0-424641
],
"keyCredentials": [{"keyId":
"92fe4c65-9ce3-4d6d-9c76-31b511a8a977", "customKeyIdentifier":
"3wLEeFNsctRc+jWX057nFP0+QD8=", "value":
"MIIDETCCAfkCCG2f2svvWEwAMA0GCSqGSIb3DQEBDQUAMCQxDDAKBgNVBAMMA3NvMTEUMBIGA1UECgwLU
"type": "AsymmetricX509Cert", "usage": "Verify"}],
"knownClientApplications": [],
"logoutUrl": null,
"oauth2AllowImplicitFlow": false,
"oauth2AllowUrlPathMatching": false,
"oauth2Permissions": [
{
"adminConsentDescription": "Allow the application to access
zliang-test-app on behalf of the signed-in user.",
"adminConsentDisplayName": "Access zliang-test-app",
"id": "8448c8ef-a250-481e-ba5c-d877badd3e07",
"isEnabled": true,
"type": "User",
"userConsentDescription": "Allow the application to access
zliang-test-app on your behalf.",

30
"userConsentDisplayName": "Access zliang-test-app",
"value": "user_impersonation"
}
],
"oauth2RequirePostResponse": false,
"objectId": "aa082da8-0f43-4a09-a364-630f4df75a62",
"passwordCredentials": [],
"publicClient": false,
"replyUrls": [
"http://localhost:8000"
],
"requiredResourceAccess": [
{
"resourceAppId": "00000002-0000-0000-c000-000000000000",
"resourceAccess": [
{
"id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
"type": "Scope"
}
]
}
],
"samlMetadataUrl": null
}

5. Check to make sure the edited JSON is valid.


6. (Optional) If the keyCredentials array in your application's manifest is not
empty, copy the value, from your generated keyCredentials array, and
paste it inside your existing keyCredentials array in your manifest, along
with a "," in between the copied value and the existing values in order to
construct a valid JSON array. See the below example.

{"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977",
"customKeyIdentifier": "3wLEeFNsctRc+jWX057nFP0+QD8=",
"value":
"MIIDETCCAfkCCG2f2svvWEwAMA0GCSqGSIb3DQEBDQUAMCQxDDAKBgNVBAMMA3NvMTEUMBIGA1UECgwLU
"type": "AsymmetricX509Cert", "usage": "Verify"}
7. Save the file. Do not change the file name.
8. In the Azure management portal, click Manifest > Upload Manifest.
9. Upload the edited JSON file that you just saved.
10. On the Splunk platform instance responsible for data collection for this
add-on, click on Troubleshooting.

If the Certificate Status panel says anything other than "Uploaded and verified as
valid", wait a moment and refresh the page. If the certificate is still not reported
as valid, try again with a new certificate and key file.

31
Configure Office 365 Management APIs inputs for
the Splunk Add-on for Microsoft Cloud Services
Prerequisites: Before you enable inputs, complete the previous steps in the
configuration process:

• Configure an Active Directory Application in Azure AD for the Splunk


Add-on for Microsoft Cloud Services
• Connect to your Microsoft Office 365 account with the Splunk Add-on for
Microsoft Cloud Services
• Configure a certificate and private key to enable service-to-service calls
for the Splunk Add-on for Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting
data for this add-on, usually a heavy forwarder. You can configure inputs using
Splunk Web (recommended) or using the configuration files.

Note:

• If you want to collect audit logs for mailbox access from Exchange Online,
you need to turn on mailbox audit logging in Office 365, which is not
enabled by default. See Exchange audit logging.
• If you configure the Office365 input for the first time, the activity log (such
as Audit.Exchange, Audit.Sharepoint and Audit.AzureActivityDirectory) will
subscribe the data from Microsoft side. But it will take up to 12 hours for
the first content blobs to become available for that subscription in
Microsoft.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance
responsible for collecting data for this add-on, usually a heavy forwarder.

1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs.

2. Click Create New Input and then select Office 365 Management APIs.

3. Enter the Name, Account, Data and Index using information in the input
parameter table below.

4. Click Add.

32
5. Verify that data is successfully arriving by running the following search on your
search head:

sourcetype=ms:o365:management*

If you do not see any events, check the Troubleshooting tab on your data
collection node to verify that your accounts, forwarders, and inputs are all
configured successfully. See Troubleshoot the Splunk Add-on for Microsoft Cloud
Services for information about enabling this dashboard on your heavy forwarder.

Configure inputs in the configuration files

Configure your inputs using the configuration files on the Splunk platform
instance responsible for collecting data for this add-on, usually a heavy
forwarder.

1. Create
$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_m

2. Add the following stanza.

[<management_input_name>]
account = <value>
data = <value>
index = <value>
3. (Optional) If you want to change the data sources or polling intervals, edit the
data parameter. These default values represent all the data sources currently
available for collection with this add-on.

Note: CurrentStatus also includes HistoricalStatus. CurrentStatus uses the


interval defined here, but HistoricalStatus uses 86400 (24 hours), because
Microsoft generates historical status once per day. For more information, see
https://msdn.microsoft.com/EN-US/library/office/dn707386.aspx.

4. (Optional) Configure a custom index.

5. Restart your Splunk platform instance.

6. Verify that data is successfully arriving by running the following search on your
search head:

sourcetype=ms:o365:management*

33
If you do not see any events, check the Troubleshooting tab on your data
collection node to verify that your accounts, forwarders, and inputs are all
configured successfully. See Troubleshoot the Splunk Add-on for Microsoft Cloud
Services for information about enabling this dashboard on your heavy forwarder.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Corresponding
Attribute field in Splunk Description
Web
management_input_name Name A friendly name for your input.
The Microsoft Office 365 account from which you want t
account Account
data.
The Microsoft cloud services from which you want to col
through the API, with intervals for data collection for eac
service. The add-on automatically lists all services curre
available. You can remove any or click the interval value
the frequency with which the add-on polls for new data f
data Data API.
Note: CurrentStatus also includes HistoricalStatus.
CurrentStatus uses the interval defined here, but Histori
uses 86400 (24 hours), because Microsoft generates his
status once per day. For more information, see
https://msdn.microsoft.com/EN-US/library/office/dn7073
The index in which the Microsoft cloud services data sho
index Index
stored. The default is main.

Configure a Storage Account in Microsoft Cloud


Services
In order to gather data from Azure Storage Table, Azure Storage Blob and Azure
Virtual Machine Metrics, you need to create or configure a storage account in
Microsoft Azure.
Prerequisite: In order to create a storage account, you need a Microsoft Azure
account with global administrator account permissions.

34
Create and Manage Storage Account

See Create Azure storage accounts for the instructions to create and manage the
storage account.

Configure the Storage Account to get data

• Splunk Add-on for Microsoft Cloud Services provides two methods for you
to get Azure storage table and Azure virtual machine metrics data. You
can use either Access Key or Account Token (SAS: Shared access
signature) follow the steps below.
• If you want to get Azure storage blob data, besides the two methods
mentioned above, you can also use None Secret to get the data without
inputting key or token.

Get storage account access key

1. Login to your Azure portal or Azure Government portal


2. Select the storage account you want to use
3. Copy either Key1 or Key2 of Access Key under Settings.

Get storage account token (SAS)

1. Login to your Azure portal or Azure Government portal


2. Select the storage account you want to use
3. Configure Shared access signature based on the data you want to
collect, such as allowed services, allowed resource types and start
and expiry date/time under Settings.
4. Generate SAS and copy it to clipboard

Get storage blob data without key or token

1. Login to your Azure portal or Azure Government portal


2. Click All resources in the menu, then select the storage account that that
you want to use.
3. Click Overview and then select Blobs under Services.
4. Select the container you want to configure and then click Access policy.
5. Select Container for the Access type.

35
Connect to your Azure App Account with Splunk
Add-on for Microsoft Cloud Services
Connect between the Splunk Add-on for Microsoft Cloud Services and your
Azure App account so that you can ingest your Microsoft cloud services data into
the Splunk platform. You can configure this connection using Splunk Web on
your data collection node (recommended), or using the configuration files.

Prerequisite: Before you complete these steps, follow the directions in Configure
an Active Directory Application in Azure AD for the Splunk Add-on for Microsoft
Cloud Services to prepare your Microsoft account for this integration.

Connect to your account using Splunk Web

Access Splunk Web on the node of your Splunk platform installation that collects
data for this add-on.

1. Launch the add-on, then click Configuration.


2. Click Azure App Account > Add Azure App Account.
3. Enter a friendly Name for the account.
4. Enter the Client ID , Key (Client Secret) and Tenant ID using the
account parameter table below.
5. Click Add.

Connect to your account using configuration files

If you do not have access to Splunk Web on your data collection node, you can
configure the connection to your account using the configuration files.

1. Create or open
$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/mscs_azure_accounts.
2. Add the following stanza.

[<account_stanza_name>]
client_id = <value>
client_secret = <value>
tenant_id = <value>

Account Attributes

Corresponding
Attribute Description
name in Splunk Web

36
Enter a friendly name for your
account_stanza_name Name
Azure app account
Use the Clinet ID that Azure AD
client_id Client ID automatically assigned to your
integration application
client_secret Key (Client Secret) Enter the password for Client ID
Enter the Tenant ID when you
tenant_id Tenant ID Create an application in Microsoft
Azure AD

Configure Azure Audit Modular inputs for the


Splunk Add-on for Microsoft Cloud Services
Prerequisites: Before you enable inputs, complete the previous steps in the
configuration process:

• Configure an Active Directory Application in Azure AD for the Splunk


Add-on for Microsoft Cloud Services
• Connect to your Azure App Account with Splunk Add-on for Microsoft
Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting
data for this add-on, usually a heavy forwarder. You can configure inputs using
Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance
responsible for collecting data for this add-on, usually a heavy forwarder.

1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs.

2. Click Create New Input and then select Azure Audit.

3. Enter the Name, Azure Account, Subscription ID, Start Time, Interval and
Index using the information in input parameter table below.

37
Configure inputs using configuration file

Configure your inputs using the configuration files on the Splunk platform
instance responsible for collecting data for this add-on, usually a heavy
forwarder.

1. Create a file named mscs_azure_audit_inputs.conf under


$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.

2. Add the following stanza.

[<input_stanza_name>]
account = <value>
subscription_id = <value>
start_time = <value>
index = <value>
interval = <value>
3. Save and restart Splunk platform.

Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Corresponding
Attribute field in Splunk Description
Web
input_stanza_name Name A friendly name for your input.
The Azure App account from which you
account Azure Account
want to gather data.
The instance queries the management
events belong to this subscription. The
subscription_id Subscription ID
subscription ID is the one you
configured in Microsoft account
start_time Start Time The add-on starts collecting data with a
date later than this time. The format is
YYYY-DD-MMThh:mm:ssTZD and the
default is 30 days before the
configuration, e.g.
2016-07-15T09:00:00+0800 stands for
fetching data from 2016-07-15 09:00:00
in UTC+8 time zone.

38
Note: The maximum start time of Azure
Audit inputs is 90 days before the
configuration.
The number of seconds to wait before
interval Interval the Splunk platform runs the command
again. The default is 3600 seconds.
The index in which to store Azure audit
index Index
data.

Configure Azure Resource Modular inputs for the


Splunk Add-on for Microsoft Cloud Services
Prerequisites: Before you enable inputs, complete the previous steps in the
configuration process:

• Configure an Active Directory Application in Azure AD for the Splunk


Add-on for Microsoft Cloud Services
• Connect to your Azure App Account with Splunk Add-on for Microsoft
Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting
data for this add-on, usually a heavy forwarder. You can configure inputs using
Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance
responsible for collecting data for this add-on, usually a heavy forwarder.

1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs.

2. Click Create New Input and then select Azure Resource.

3. Fill out the Name, Azure App Account, Subscription ID, Resource Type,
Resource Group List, Interval and Index fields using the input parameter table
below.

4. Click Add.

39
Configure inputs using configuration file

Configure your inputs using the configuration files on the Splunk platform
instance responsible for collecting data for this add-on, usually a heavy
forwarder.

1. Create a file called mscs_azure_resource_inputs.conf under


$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.

2. Add the following stanza.

[<input_stanza_name>]
account = <value>
subscription_id = <value>
resource_type = <value>
resource_group_list = <value>
index = <value>
interval = <value>
3. Save and restart Splunk platform.

Input Parameter

Corresponding
Attributes Fields in Splunk Description
Web
input_stanza_name Name A friendly name for your input.
Azure App The Azure App account from which
account
Account you want to gather data.
The instance queries the
management events belong to this
subscription_id Subscription ID subscription. The subscription ID is
the one you configured in Microsoft
account
resource_type Resource Type You can choose from Virtual
Machine, Public IP Address,
Network Interface Card and Virtual
Network using Splunk Web, or set
resource_type to virtual_machine,
public_ip_address,
network_interface_card or
virtual_network in the configuration

40
file.
The resource group list is defined by
subscription ID and resource type. If
you leave this field blank, this add-on
Resource Group will query all resource lists under the
resource_group_list
List subscription ID and the resource type
you choose. You can add multiple
resource group list separated by
commas.
The number of seconds to wait before
the Splunk platform runs the
interval Interval
command again. The default is 3600
seconds.
The index in which the Microsoft
index Index
cloud services data should be stored.

Connect to your Azure Storage account with the


Splunk Add-on for Microsoft Cloud Services
Connect the Splunk Add-on for Microsoft Cloud Services and your Azure Storage
account so that you can ingest your Azure storage table, Azure storage bolb and
Azura virtual machine metrics data into the Splunk platform. You can configure
this connection using Splunk Web on your data collection node (recommended),
or using the configuration files.

Prerequisite: Before you complete these steps, follow the directions in Configure
a Storage Account in Microsoft Cloud Service to prepare your Microsoft account
for this integration.

Connect to your account using Splunk Web

Access Splunk Web on the node of your Splunk platform installation that collects
data for this add-on.

1. Launch the add-on, then click Configuration.

2. Click Azure Azure Storage Account and enter the corresponding fields using
the input parameter table below.

41
Note: There are three Account Secret Type that you can select to configure
Azure storage account, Access Key, Account Token and None Secret.

• If you want to collect Azure storage table Azure virtual machine metrics
data, you have to configure the account with Access Key or Account
Token type.
• If you want to collect Azure storage blob data, you can use any of three
types.

Connect to your account using configuration files

If you do not have access to Splunk Web on your data collection node, you can
configure the connection to your account using the configuration files.

1. Create or open
$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/mscs_storage_accounts.conf

2.Add the following stanza.

[<account_stanza_name>]
account_name = <value>
account_secret = <value>
account_secret_type = <value>
Input Parameters

Each attribute in the following table corresponds to a field in Splunk Web.

Corresponding
Attribute field in Splunk Description
Web
account_name Account Name The name for the storage account.
You can enter the key or token
generated when you Configure a
account_secret Account Secret
Storage Account in Microsoft Cloud
Service.
account_secret_type Access Key, If you set account_secret_type=0, it
Account Token or means the storage account use
None Secret None Secret type. You do not have
to set Account Name and Account
Secret. If you configure the inputs
using configuration file, you can leave

42
account_name and account_secret
blank.

If you set account_secret_type=1, it


means the storage account use
Access Key type. You have to enter
the key generated when you
Configure a Storage Account in
Microsoft Cloud Service
If you set account_secret_type=2, it
means the storage account use
Account Token type. You have to
enter the token generated when you
Configure a Storage Account in
Microsoft Cloud Service

Configure Azure Storage Table Modular Input for


Splunk Add-on for Microsoft Cloud Services
Prerequisites: Before you enable inputs, complete the previous steps in the
configuration process:

• Configure a Storage Account in Microsoft Cloud Service


• Connect to your Azure Storage account with the Splunk Add-on for
Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting
data for this add-on, usually a heavy forwarder. You can configure inputs using
Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance
responsible for collecting data for this add-on, usually a heavy forwarder.

1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs.

2. Click Create New Input.

3. Fill out the Name, Azure Storage Account, Table List, Start Time, Interval,
Index and Sourcetype fields using the input parameter table below.

43
Configure inputs using configuration file

1. Create a file called inputs.conf under


$SPLUNK_HOME/etc/appes/Splunk_TA_microsoft-cloudservices/local

2. Configure Azure storage table inputs with the following stanza.

[mscs_storage_table://<input_name>]
account = <value>
table_list = <value>
start_time = <value>
index = <value>
interval = <value>
sourcetype = <value>
3. Save and restart Splunk platform.

Input Parameters

Corresponding
Attributes field in Splunk Descr
Web
mscs_storage_table://<input_name> Name A friendly name for your input.
Azure Storage
account Choose a Storage Account you have con
Account
The table list under the storage account.
separated by commas. You can also use
table_list Table List
If the table name uses regex syntax, plea
name. For example: table*, :table\d+.
The add-on starts collecting data with a d
30 days before the configuration. The for
start_time Start Time
e.g. 2016-07-15T09:00:00+0800 stands
09:00:00 in UTC+8 time zone.
The number of seconds to wait before th
collection_interval Interval
again. The default is 3600 seconds.
index Index The index in which to store Azure Storag
sourcetype Sourcetype The default is mscs:storage:table.

Note: If you want to change the default s


field of the event, the behaviour is unpred
the timestamp field. To prevent the issue

44
under
SPLUNK_HOME/etc/apps/Splunk_TA_micro

Configure Azure Storage Blob Modular Input for


Splunk Add-on for Microsoft Cloud Services
Prerequisites: Before you enable inputs, complete the previous steps in the
configuration process:

• Configure a Storage Account in Microsoft Cloud Service


• Connect to your Azure Storage account with the Splunk Add-on for
Microsoft Cloud Services

Configure your inputs on the Splunk platform instance responsible for collecting
data for this add-on, usually a heavy forwarder. You can configure inputs using
Splunk Web (recommended) or using the configuration files.

Note: Since the format of the data in Azure Storage Blob channel varies
(including text and binary data), Splunk suggests you to leverage the options for
sourcetypes to make the event data more effective. See Overview of Event
Processing for details.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance
responsible for collecting data for this add-on, usually a heavy forwarder.

1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs.

2. Click Create New Input and select Azure Storage Blob.

3. Enter the Name, Storage Account, Container Name, Blob list, Interval,
Index and Sourcetype using the inputs parameters table below.

Configure inputs using Configuration File

1. Create a file called inputs.conf under


$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.

2. Configure Azure Storage Blob input with the following stanza.

45
[<mscs_storage_blob://<input_name>>]
account = <value>
container_name = <value>
blob_list = <value>
exclude_blob_list = <value>
blob_mode = <value>
decoding = <value>
index = <value>
interval = <value>
sourcetype = <value>
Inputs Parameter

Each attribute in the following table corresponds to a field in Splunk Web.

Corresponding
Attribute field in Splunk Description
Web
Enter a friendly name of
mscs_storage_blob://<input_name> Name
your inputs.
Select the storage
Azure Storage
account account name you
Account
configured.
Enter the container
name under the storage
container_name Container Name account. You can only
add one container
name for each input.
blob_list Blob List Enter the Blob name
which you want to
collect the data from.
You can add multiple
blob names separated
by commas. If you
leave this field empty,
this add-on will collect
all the blob lists under
the Container Name
you just configured.

Note: You can enter the


specific blob list name,
use wildcard or use

46
regex expression in this
field.

• If you want to
collect data from
a specific blob
list, just enter the
name of the blob
list here, such as
blob_name.
• You can use
wildcard in this
field, e.g. blob*,
this add-on will
collect data from
the blob lists of
the names
starting from
blob. And you
can also use
comma to
separate multiple
blob names, e.g.
blob, name*.
• If you want to
use regex, the
syntax is JSON
format: {"regex
syntax":3}, 3
stands for regex
file.
• If you want to
enter the blob list
which has both
wildcard and
regex, you can
enter both
separated by
commas, for
example, {"regex
syntax" :3,
blob* :2}, 2
stands for

47
wildcard list.
• If you want to
enter the blob list
using all of the
three
expressions, you
can use the
syntax like
{"regex
syntax" :3,
blob* :2, blob :1},
1 stands for
using a specific
blob list name.

Note:

• The blob name


must be at least
one character
long but cannot
be more than
1,024 characters.
• Blob names are
case-sensitive.
• Reserved URL
characters must
be properly
escaped.
• The number of
path segments
comprising the
blob name
cannot exceed
254.
exclude_blob_list Excluded Blob Optional. Enter the Blob
List name that you do not
want to collect the data
from. You can add
multiple blob names
separated by commas.
The syntax of the

48
Excluded Blob List is
the same as Blob List.
The default is append.
blob_mode NULL Do not change the
value of this field.
Specify the character
set of the file, such as
UTF-8 or UTF-32. If you
decoding Decoding leave this field blank,
this add-on will use the
default character set of
the file.
The number of seconds
to wait before the
Splunk platform runs
collection_interval Interval
the command again.
The default is 3600
seconds.
The index in which to
index Index store Azure Storage
Blob data.
The default is
sourcetype Sourcetype
mscs:storage:blob.
Note: If there is a file match the syntax both in Blob List and Exclude Blob List,
Exclude Blob List is in higher priority. For example, if there is a blob list name
blob1, and it match the syntax you set in Blob List and Exclude Blob List, this
add-on will exclude this list because Exclude Blob List is in higher priority.

Configure Azure Virtual Machine Metrics Modular


Input for Splunk Add-on for Microsoft Cloud
Services
Prerequisites: Before you enable inputs, complete the previous steps in the
configuration process:

• Configure a Storage Account in Microsoft Cloud Service


• Connect to your Azure Storage account with the Splunk Add-on for
Microsoft Cloud Services

49
Configure your inputs on the Splunk platform instance responsible for collecting
data for this add-on, usually a heavy forwarder. You can configure inputs using
Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web

Configure your inputs using Splunk Web on the Splunk platform instance
responsible for collecting data for this add-on, usually a heavy forwarder.

1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs.

2. Click Create New Input and select Azure Virtual Machine Metrics.

3. Type the Name, Storage Account, Start Time and Indexusing the input
parameters below and then click Add.

Note: You cannot configure Table List, Interval and Sourcetype using Splunk
Web.

Configure inputs using configuration file

1. Create a file called inputs.conf under


$SPLUNK_HOME/etc/appes/Splunk_TA_microsoft-cloudservices/local

2. Configure Azure virtual machine metrics inputs with the following stanza.

[<mscs_storage_table://<input_name>]
account = <value>
table_list = WADMetricsPT1M*
start_time = <value>
index = <value>
interval = 60
sourcetype = mscs:vm:metrics
3. Save and restart Splunk platform.

Input Parameters

Corresponding
Attribute field in Splunk Description
Web
mscs_storage_table://<input_name> Name A friendly name for your input.

50
Azure Storage Choose a Storage Account you
account
Account have configured.
Enter a table list name under the
storage account. You cannot
change the Table List name in
Splunk Web, which is
WADMetricsPT1M*.
table_list Table List
Note: The best practice is to
keep the default value
WASMetricsPT1M* in the table
list.
The add-on starts collecting data
with a date later than this time.
The default is 30 days before
the configuration. The format is
start_time Start Time ?YYYY-DD-MMThh:mm:ssTZD,
e.g. 2016-07-15T09:00:00+0800
stands for fetching data from
2016-07-15 09:00:00 in UTC+8
time zone.
The number of seconds to wait
before the Splunk platform runs
the command again. The default
is 60 seconds and you cannot
change it in Splunk Web. If you
want to change the interval time,
collection_interval Interval
you have to configure it using
the configuration file.

Note: If you want to use ITSI


data model, the best practice is
to set the interval to 60 seconds.
The index in which to store
index Index
Azure Storage Table data.
sourcetype Sourcetype The default is mscs:vm:metrics.
You cannot change the
sourcetype in Splunk Web. If
you want to change the
sourcetype, you have to
configure it using the

51
configuration file.

Troubleshoot the Splunk Add-on for Microsoft


Cloud Services
General troubleshooting

For helpful troubleshooting tips that you can apply to all add-ons, see
Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support
and resource links for add-ons in Splunk Add-ons.

Troubleshooting dashboard for Office365 inputs

The Splunk Add-on for Microsoft Cloud Services includes a troubleshooting


dashboard to help you identify issues with inputs, accounts, and X.509
certificates. Access this dashboard on your data collection node, usually a heavy
forwarder.

Note: The troubleshooting dashboard is only used for identifying issues of


Office365 inputs, accounts and X.509 certificates. You cannot identify the issues
for Azure related inputs and accounts using this dashboard.

To enable this dashboard on a heavy forwarder, backup your _internal data


locally by editing your local inputs.conf and outputs.conf files as shown:

In local/inputs.conf

[monitor://$SPLUNK_HOME/var/log/splunk]
index = _internal
_INDEX_AND_FORWARD_ROUTING = true
In local/outputs.conf

[indexAndForward]
index=true
selectiveIndexing=true
Access the dashboard by opening the app on your data collection node. Click on
any error states to drill down into detailed error messages and mitigation
suggestions.

52
Certificate status messages

The troubleshooting dashboard and the Certificate configuration screen display a


status message that describes the current state of your service-to-service call
configuration. The table lists the possible status messages with more information.

Status message Description


No certificate No certificate has been uploaded or auto-generated on the
configured yet Certificate tab yet.
Certificate is A certificate file and private key are uploaded on the
uploaded but not Certificate tab of the add-on, but the add-on has not yet
verified yet validated it against the manifest in Azure AD.
A certificate file and private key are uploaded on the
Uploaded and
Certificate tab of the add-on, and the manifest in Azure AD
verified as valid
contains the matching keyCredentials object.
You have uploaded a certificate file and private key on the
Certificate tab of the add-on, but either the certificate file or
key are invalid or the manifest in Azure AD does not
contain the correct keyCredentials JSON object that
matches this certificate and key.

• If you have not yet updated the manifest, follow the


Uploaded but documented instructions in Configure a certificate
invalid and private key to enable service-to-service calls for
the Splunk Add-on for Microsoft Cloud Services to
do so.
• If you updated the manifest already but still see this
message, check to make sure that your JSON
formatting is valid. If the message persists, generate
and upload a new certificate and private key and
re-update the manifest.
The certificate file has been auto-generated on the
Auto-generated but
Certificate tab of the add-on, but the add-on has not yet
not verified yet
validated it against the manifest in Azure AD.
The certificate file has been auto-generated on the
Auto-generated and
Certificate tab of the add-on, and the manifest in Azure AD
verified as valid
contains the matching keyCredentials object.
Auto-generated but You have auto-generated a certificate on the Certificate
invalid tab of the add-on, but it is either invalid or the manifest in

53
Azure AD does not contain the correct keyCredentials
JSON object that matches this auto-generated certificate.

• If you have not yet updated the manifest, follow the


documented instructions in Configure a certificate
and private key to enable service-to-service calls for
the Splunk Add-on for Microsoft Cloud Services to
do so.
• If you updated the manifest already but still see this
message, check to make sure that your JSON
formatting is valid. If the message persists, generate
a new certificate and re-update the manifest.
Certificate Verify Failed error

If you receive SSL untrusted certificate error, it means that the website is not
in the trusted list. Add the following website to the trusted domains.

Office 365:
https://manage.office.com
https://login.windows.net

Azure and Storage related:


https://management.azure.com
https://*.table.core.windows.net
https://*.blob.core.windows.net

Accessing Logs of Azure Inputs

There are four different logs for different types of inputs. The table below is the
detailed description of each log.

Log Filename Sourcetype Descriptio


Azure Aud
splunk_ta_microsoft-cloudservices_azure_audit.log mscs:azure:audit:log Log chann
related log
Azure
Resource
splunk_ta_microsoft-cloudservices_azure_resource.log mscs:azure:resource:log
channel
related log
splunk_ta_microsoft-cloudservices_storage_blob.log mscs:storage:blob:log Azure
Storage

54
Blob
channel
related log
Azure
Storage
Table
channel
related log
and
splunk_ta_microsoft-cloudservices_storage_table.log mscs:storage:table:log
Azure
Virtual
Machine
Metrics
channel
related log
Not receiving MSCS data after configuring certificate

If you are not receiving data, and your configured certificate says
"Auto-generated and verified as valid", upgrade to version 2.1 or above of the
Splunk Add-on for Microsoft Cloud Services.

55
Troubleshooting

Troubleshoot the Splunk Add-on for Microsoft


Cloud Services
General troubleshooting

For helpful troubleshooting tips that you can apply to all add-ons, see
Troubleshoot add-ons in Splunk Add-ons. For additional resources, see Support
and resource links for add-ons in Splunk Add-ons.

Troubleshooting dashboard for Office365 inputs

The Splunk Add-on for Microsoft Cloud Services includes a troubleshooting


dashboard to help you identify issues with inputs, accounts, and X.509
certificates. Access this dashboard on your data collection node, usually a heavy
forwarder.

Note: The troubleshooting dashboard is only used for identifying issues of


Office365 inputs, accounts and X.509 certificates. You cannot identify the issues
for Azure related inputs and accounts using this dashboard.

To enable this dashboard on a heavy forwarder, backup your _internal data


locally by editing your local inputs.conf and outputs.conf files as shown:

In local/inputs.conf

[monitor://$SPLUNK_HOME/var/log/splunk]
index = _internal
_INDEX_AND_FORWARD_ROUTING = true
In local/outputs.conf

[indexAndForward]
index=true
selectiveIndexing=true
Access the dashboard by opening the app on your data collection node. Click on
any error states to drill down into detailed error messages and mitigation
suggestions.

56
Certificate status messages

The troubleshooting dashboard and the Certificate configuration screen display a


status message that describes the current state of your service-to-service call
configuration. The table lists the possible status messages with more information.

Status message Description


No certificate No certificate has been uploaded or auto-generated on the
configured yet Certificate tab yet.
Certificate is A certificate file and private key are uploaded on the
uploaded but not Certificate tab of the add-on, but the add-on has not yet
verified yet validated it against the manifest in Azure AD.
A certificate file and private key are uploaded on the
Uploaded and
Certificate tab of the add-on, and the manifest in Azure AD
verified as valid
contains the matching keyCredentials object.
You have uploaded a certificate file and private key on the
Certificate tab of the add-on, but either the certificate file or
key are invalid or the manifest in Azure AD does not
contain the correct keyCredentials JSON object that
matches this certificate and key.

• If you have not yet updated the manifest, follow the


Uploaded but documented instructions in Configure a certificate
invalid and private key to enable service-to-service calls for
the Splunk Add-on for Microsoft Cloud Services to
do so.
• If you updated the manifest already but still see this
message, check to make sure that your JSON
formatting is valid. If the message persists, generate
and upload a new certificate and private key and
re-update the manifest.
The certificate file has been auto-generated on the
Auto-generated but
Certificate tab of the add-on, but the add-on has not yet
not verified yet
validated it against the manifest in Azure AD.
The certificate file has been auto-generated on the
Auto-generated and
Certificate tab of the add-on, and the manifest in Azure AD
verified as valid
contains the matching keyCredentials object.
Auto-generated but You have auto-generated a certificate on the Certificate
invalid tab of the add-on, but it is either invalid or the manifest in

57
Azure AD does not contain the correct keyCredentials
JSON object that matches this auto-generated certificate.

• If you have not yet updated the manifest, follow the


documented instructions in Configure a certificate
and private key to enable service-to-service calls for
the Splunk Add-on for Microsoft Cloud Services to
do so.
• If you updated the manifest already but still see this
message, check to make sure that your JSON
formatting is valid. If the message persists, generate
a new certificate and re-update the manifest.
Certificate Verify Failed error

If you receive SSL untrusted certificate error, it means that the website is not
in the trusted list. Add the following website to the trusted domains.

Office 365:
https://manage.office.com
https://login.windows.net

Azure and Storage related:


https://management.azure.com
https://*.table.core.windows.net
https://*.blob.core.windows.net

Accessing Logs of Azure Inputs

There are four different logs for different types of inputs. The table below is the
detailed description of each log.

Log Filename Sourcetype Descriptio


Azure Aud
splunk_ta_microsoft-cloudservices_azure_audit.log mscs:azure:audit:log Log chann
related log
Azure
Resource
splunk_ta_microsoft-cloudservices_azure_resource.log mscs:azure:resource:log
channel
related log
splunk_ta_microsoft-cloudservices_storage_blob.log mscs:storage:blob:log Azure
Storage

58
Blob
channel
related log
Azure
Storage
Table
channel
related log
and
splunk_ta_microsoft-cloudservices_storage_table.log mscs:storage:table:log
Azure
Virtual
Machine
Metrics
channel
related log
Not receiving MSCS data after configuring certificate

If you are not receiving data, and your configured certificate says
"Auto-generated and verified as valid", upgrade to version 2.1 or above of the
Splunk Add-on for Microsoft Cloud Services.

59
Reference

Lookups for the Splunk Add-on for Microsoft Cloud


Services
The Splunk Add-on for Microsoft Cloud Service has the following lookups that
map fields from Microsoft Cloud Service systems to CIM-compliant values in the
Splunk platform. The lookup files are located in
$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/lookups.

Filename Description
Maps a status field to
o365_certficate_status_lookup.csv
a friendly description.
Maps
management_api_data
o365_management_api_data_lookup.csv
field to a friendly
name.
Maps Operation and
ResultStatus to
model_type,action,
o365_model_lookup.csv
change_type, and
object_category
fields.
Maps Operation to
model_type, action,
o365_model_operation_only_lookup.csv change_type, and
object_category
fields.
Maps ResultStatus to
o365_status_lookup.csv a CIM-compliant
status value.
o365_troubleshooting_error_code_lookup.csv Maps o365_error to
Problem, Problem
Detail,Possible
Reason, and Proposal
values for the
Troubleshooting

60
dashboard.
Maps
microsoft_error_code
to o365_error,
Problem, Problem
o365_troubleshooting_microsoft_error_code_lookup.csv Detail, Possible
Reason, and
Proposalvalues for
the Troubleshooting
dashboard.
Maps vm_size to
cpu_cores,
mscs_vm_cpu_mem_storage.csv
mem_capacity and
storage_capacity.

Maps vm_id to
mscs_vm_ip.csv private_ip and
public_ip.

Maps a power_state
mscs_vm_power_state.csv field to a common
description.

Performance reference for the Azure storage input


in the Splunk Add-on for Microsoft Cloud Services
This page provides reference information about Splunk's performance testing of
the Azure storage input in Splunk Add-on for Microsoft Cloud Services. The
testing occurred with version 2.0.0, when the Azure storage input was first
introduced. Use this information to enhance the performance of your own Azure
storage data collection tasks.

Note: Many factors impact performance results, including file size, file
compression, event size, deployment architecture, and hardware. These results
represent reference information and do not represent performance in all
environments.

Testing Architecture

Splunk tested the performance of the Storage input using a single-instance


Splunk Enterprise 6.4.3 on an C4 High-CPU Double Extra Large instance to
ensure CPU, memory, storage, and network do not introduce any bottlenecks.

61
Instance specs:

Note: The EC2 in the testing environment is in the same area of Azure storage
input, the network latency is low.

Instance type C4 High-CPU Double Extra Large


Memory 15 GB
Compute Units (ECU) 31 Unit
Cores 8
Storage Type GP2(SSD)
Architecture 64-bit
Network performance High
EBS Optimized: Max Bandwidth 1000 Mbps
Test Environment

Deployment Type Role EC2 Type Count


Standalone Deployment Standalone C4 High-CPU Double Extra Large 1
Testing Result

Testing Result for Azure Storage Table inputs

The detailed test result is listed below.

• Input number stands for the number of the inputs, one input collects one
table.
• Each table contains 2,131,200 events.
• Each event is 500 Bytes.

Input Throughput Throughput Throughput


Number (MB/s) (GB/day) (Event/s)
1 3.44 290 7045
2 5.7 480 11670
4 6.84 577 14000
8 6.12 516 12533
Analysis: The maximum throughput is 6.8MB/s with four data inputs. 100%
higher than one data input. It can scale 100% with max throughput 577GB/day

62
for single instance.

Testing Result for Azure Storage Blob inputs

The detailed test result is listed below.

• Each Blob input contains 259,202 events.


• Each event is 190 Bytes.

Blob Blob Input Throughput Throughput Throughput


Number Size Number (MB/s) (GB/day) (Event/s)
20 50 M 1 3.8 MB/s 320 21136
40 50 M 2 7.7 MB/s 650 42092
80 50 M 4 12.5 MB/s 1054 72960
160 50 M 8 12.1 MB/s 1020 66500
Analysis: If we use large file size (50M in our test environment), the max
throughput is 12.5MB/s with four data inputs, about 300% higher than the single
input.

APIs used in the Splunk Add-on for Microsoft Cloud


Services
The following table below lists the APIs that are used in the Splunk Add-on for
Microsoft Cloud Services.

Inputs Name Method Description (Link to Microsoft site)


Azure Query Tables https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx
Storage
Table Query Entities https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx
Azure List Blobs https://msdn.microsoft.com/en-us/library/azure/dd135734.aspx
Storage Blob Get Blob https://msdn.microsoft.com/en-us/library/azure/dd179440.aspx
Azure Insights
- List the
Azure Audit https://msdn.microsoft.com/en-us/library/azure/dn931934.aspx
management
events
Azure List all virtual https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/
Resource - machines in a

63
Virtual resource group
Machine Get the https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/
instance view
of a virtual
machine
Azure List public IP
Resource - addresses
https://msdn.microsoft.com/en-us/library/azure/mt163657.aspx
Public IP within a
Address resource group
Azure List network
Resource - interface cards
Network within a https://msdn.microsoft.com/en-us/library/azure/mt163627.aspx
Interface resource
Card group.
Azure List virtual
Resource - networks
https://msdn.microsoft.com/en-us/library/azure/mt163587.aspx
Virtual within a
Network resource group
Azure Virtual Query Tables https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx
Machine
Metrics Query Entities https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx
Tenant-specific
Office 365 content blobs
Management https://msdn.microsoft.com/en-us/office-365/office-365-manageme
Activity DLP.All is not
supported

64

You might also like