Content (2) - 1

SOAR Advanced Implementation
Splunk SOAR Advanced Implementation

turn data into doing™ 1 Copyright © 2023 Splunk, Inc. All rights reserved 17-Jan-23
Document Usage Guidelines
• Should be used only for enrolled students
• Not meant to be a self-paced document, an instructor is needed
• Do not distribute
18 October 2021
Course Prerequisites
• Required:
– Experience with Python programming
– Administering Splunk SOAR
– Developing Splunk SOAR Playbooks
– Enterprise Splunk Data Administration
– Enterprise Splunk System Administration
– Either Using or Administering Splunk Enterprise Security

Course Outline
• Module 1: Implementing Splunk and SOAR Solutions
• Module 2: Configuring External Splunk Search
• Module 3: Integrating SOAR into Splunk
• Module 4: Accessing Splunk from SOAR
• Module 5: Custom Coding
• Module 6: Using the SOAR REST API

Module 1:
Implementing Splunk
and SOAR Solutions

Modules Objectives
• Review SOAR concepts and features
• Review documentation and other resources
• Overview of Splunk/SOAR integration
• Configure Splunk and SOAR for integration

Agenda: SOAR and Splunk
Splunk can send events to
Splunk SOAR to be evaluated and
SOAR
potentially initiate playbooks
SOAR's searchable
database can be hosted on
Splunk, enriching the search
capabilities and providing
in-depth reporting
SOAR can run searches in

Splunk to acquire insight
into events

Agenda: Lab Exercises
• Extensive lab exercises • Playbooks:
• Two servers – Custom code
– Enterprise Security – Running Splunk searches
– SOAR – Passing data between

playbooks
• Configure:
• REST API
– Remote search
– Retrievingobject information
– Splunk event forwarding
to SOAR – Django queries
– Splunk app on SOAR – Creating objects

Review: Security Data Sources
Data sources
can be
anything,
Security Data Sources • Incident especially if
• Vulnerability Splunk is
Playbooks • Threat Intel already
splunking it.
Actions
Apps
Assets
Owners

Review: Playbooks
Playbooks run
automatically as
data is ingested;
Security Data Sources • Investigate endpoint analyzing,
• Reimage endpoint prioritizing, and
Playbooks • Deploy indicators alerting users as
• Investigate phishing needed.
Actions
Apps
Assets
Owners

Review: Actions
Playbooks and
users can take
immediate
Security Data Sources • "block ip" actions to
• "disable user" investigate and
Playbooks • "geolocate ip" mitigate threats.
• "detonate file" Users can also
Actions • "get events" run actions from
• "send email" the Investigation
Apps • "file reputation" page.
• "list processes"
Assets • "snapshot vm"
Owners

Review: Apps
Apps provide
SOAR with the
code and REST
Security Data Sources • Cisco ASA API
• Microsoft AD configuration to
Playbooks • Maxmind connect with
• Cuckoo your security
Actions • ThreatGrid appliances,
• IBM Qradar servers or cloud
Apps • Splunk services to
• SMTP ingest data and
Assets • Tanium to run actions.
Owners

Review: Assets
Each asset is
one instance of
an app. Assets
Security Data Sources • perimeter_fw contains the
• primary_dc specific
Playbooks • primary_siem configuration
• exchangeserver (address,
Actions • cfo_laptop credentials,
parameters, etc)
Apps to use that app
Assets
Owners

Review: Owners
The people in
your
organization use
Security Data Sources • Own events SOAR to create
• Execute tasks playbooks,
Playbooks • Approve actions approve
• Review actions, and use
Actions the Investigation
page to monitor
Apps status.
Assets
Owners

Review: Data Sources and Containers
Container ID
Artifacts
- Original
Sources - Actions
- Playbooks
Assets with On_Poll
- Files
APIs Comments/Notes
Case Connections
Playbooks
Audit Trail
Manual Tags
Severity
SLA
TLP
Status
Label

Splunk to SOAR: Pull
Container ID
Splunk app on Artifacts

SOAR - Original
Sources - Actions
- Playbooks
Assets with On_Poll
- Files
APIs Comments/Notes
Case Connections
Playbooks
Audit Trail
Manual Tags
Severity
SLA
TLP
Status
Label

Splunk to SOAR: Push
Container ID
SOAR app on Artifacts

Splunk - Original
Sources - Actions
- Playbooks
Assets with On_Poll
- Files
APIs Comments/Notes
Case Connections
Playbooks
Audit Trail
Manual Tags
Severity
SLA
TLP
Status
Label

Review: Documentation
• All SOAR documentation can be found in either of the
following two places:
– On the SOAR server Administration menu
– On the Splunk documentation page (docs.splunk.com)

Review: SOAR Community
• my.phantom.us
• Requires registration
• Download product, playbooks
• Knowledge base articles, blogs and videos
• Slack community workspace
– splunkbase.splunk.com
– Requiresregistration
– Download Splunk and SOAR apps
• DEMO

Review: SOAR Server Architecture
External Platforms & Services Human-Machine Interfaces
data sources assets REST API user interface
external data & analytic tools security tools & action targets machine-to-machine interface analyst input / output
siem firewall reporting & metrics
threat intel platform endpoint vulnerability mgmt Investigation Page
email malware sandbox external APIs visual playbook editor
data lake reputation service message bus platform administration…
… … …
app app
process process
poll invoke action

normalize data return data
client-side framework
ingestd actiond
service service
event ingestion app execution
web server web framework

decided
service Manual
orchestration & decision making Low
Playbook Medium
Playbook
Playbook
Playbook
High
datastore
concurrent playbook severity-based

and action execution priority queuing
search & analytics
Platform Services
workflowd watchdogd
service service
LEGEND
user notifications health monitoring
External Communication IPC
SOAR Microservices

Debug Logging
• Enable logging in playbook settings to add
detailed debugging information
• Make sure DEBUG level is set for the Action and
Decide daemons in Administration >
System Health > Debugging
• Switch back to WARN before
going to production mode
soar_home/var/logs/phantom

Server Certificates
• Both Splunk and SOAR should be configured with valid SSL
certificates
– Possibleinsecure communications otherwise
– Problems with inter-server communications

Configuring SplunkWeb SSL Certificates
1. Add certificate files to Splunk server
2. Edit etc/system/local/web.conf
3. Restart Splunk: bin/splunk restart

docs.splunk.com/Documentation/Splunk/latest/Security/SecureSplunk
Webusingasignedcertificate

Configuring Splunkd SSL Certificates
1. Add certificate files to Splunk server
2. Edit etc/system/local/server.conf
3. Restart Splunk: bin/splunk restart

docs.splunk.com/Documentation/Splunk/latest/Security/Aboutsecuring
SplunktoSplunkcommunication

Configuring SOAR SSL Certificates
• Copy certificate files to SOAR server
• Rename and place in following paths:
– /opt/soar/etc/ssl/certs/httpd_cert.crt (certificate file)
– /opt/soar/etc/ssl/private/httpd_cert.key (private key)
– If different paths are used, edit /etc/nginx/conf.d/default.conf
• 3. Restart nginx service

– As SOAR user: /opt/soar/bin/phsvc restart nginx

Lab Exercise 1: Set up Servers
• Tasks:
– Examine documentation resources
– Set up SOAR server
– Set up Splunk server

Module 2:
Remote Search

Module Objectives
• Identify the benefits of externalizing search to Splunk
• Configure the SOAR instance for externalization
• Configure the Splunk instance for externalization
• Use re-index to migrate existing content to the Splunk instance

Remote Search
• An external Splunk instance can be
configured to store SOAR searchable
content
• The external Splunk instance can be used to run more complex
reports on SOAR activities and status
• The Splunk app for SOAR must be installed on the external Splunk
search head
– Requires user accounts and a HEC endpoint
– Also enables several other features

Splunk App for SOAR Roles
• splunk_app_soar
– Add to Splunk admin users who will manage the app
• splunk_app_soar_dashboards
– Add to users who need view access to the app's dashboards
• phantomsearch
– Special user account with search permission
• phantomdelete
– Special user with delete permission

Assigning Roles to Admin
• Must do this step before
proceeding with the rest of setup
• Edit the Admin role
• Add splunk_app_soar to the
inherited roles

SOAR Search User Account
• Used by SOAR server to run
searches
• Set a password and disable password
change on first login
• Remove all roles except
phantomsearch
Make sure to uncheck

Require password change…

SOAR Delete User Account
• Used from SOAR server to remove
results from searches for deleted containers
– Uses Splunk |delete command; not
true removal of data but is excluded
from all searches until archived
• Member of phantomdelete role only
• Note that in some versions of Splunk, the
admin role does not have delete_by_keyword capability
– Enable this on admin role to be able to assign phantomdelete role to
other users

Install SOAR Indexes
• In the Splunk App for SOAR, select
the Configurations tab
• In the Advanced Options section,
expand the Create indexes item, and
select Create Indexes, and Create
• The list of indexes created is
displayed

Global HEC Settings
• Settings > Data Inputs > HTTP Event Collector > Global Settings
– Enableall tokens
– Make a note of the port number
– Save

Adding a HEC Endpoint Token
• Settings > Data Inputs > HTTP Event Collector > New Token
– Setthe name
– Next

Configure Indexes
• Select all phantom_* indexes, os and splunk_soar
– Subject to Splunk license quota
– Typically, very light impact on daily indexing
• Review
• Submit
• Record token value for
use on SOAR server

Configuring the SOAR Server
• Administration > Administration Settings > Search Settings
• Select External Splunk Enterprise Instance
• Host: Splunk server host name
• Access: enter phantomsearch and
phantomdelete user credentials
• Enter port numbers
• Select Use SSL for both ports
• Enter the HEC token
• Test connectivity, and save
Migrating Old Data to Splunk
• After configuring remote search, new
events are sent to Splunk server
• Global search in SOAR is executed
on Splunk, results shown in SOAR
• Data on SOAR from before remote search setup is not
automatically migrated
• Use the REINDEX tool to migrate existing event records to Splunk
• In some cases, pre-existing search records (such as app
information) on a new SOAR instance will not be available for
search until reindexing
SOAR Data on Splunk
• Events created for activities on
objects:
– Containers
– Artifacts
– Assets
– Apps
– App runs
– Actions
– Playbooks
– Notes and comments
• Does not include case tasks, phases, workflows or docs

Searching SOAR Data
• Use standard SPL
• JSON data automatically extracted into fields
• Use fields command and table to format

Alerts for SOAR Updates
• Changes in container properties can't be easily trapped in SOAR
as triggers for custom coding
– Example: "if the owner of a container changes, run a playbook"
– This could be handled with a scheduled playbook (Timer app), but
does not have a real-time SOAR solution
• This can be handled real-time using a Splunk alert that monitors
the phantom_container index for changes
– When a change is detected, Splunk can run the Run Playbook action

More Features of the Splunk App for SOAR
• Reporting
• Automation Analytics
• Container Overview & details
• Notes & comments search
• Splunk logs from SOAR
– Requires the NGINX and AuditD TA's
• SOAR as an ITSI service Requires a Universal Forwarder
on the SOAR instance
– MonitorSOAR metrics, performance, etc.
– Supports the ITSI SOAR Content Pack
• restsoar search command

Example: Automation Analytics

|restsoar and |restsoarstream
• The restsoar generating command allows you to search SOAR
REST endpoints from Splunk
|restsoar endpoint=container/2 soar_server="soar"
– The above would search the "soar" server for all details of container
ID 2
• |restsoarstream is similar, but is a streaming command, can
be used within the body of searches
docs.splunk.com/Documentation/SOARApp/latest/User/SOARRESTAPI
commands
• Requires configuration to connect to the SOAR server

SOAR User
• On SOAR, create a new automation
user
– Used by Splunk server to connect to
the SOAR server
– Using the default automation user
is not a best practice
• Make sure:
– Allowed IP addresses include the Splunk server
– User type "Automation", and Role "Automation + Observer"
• "any" is allowed for IPs but is not a security best practice

Access Token
• Edit the new SOAR automation user
• Select and copy the contents of
the Authorization Configuration
for REST API
• Record this for use on the
Splunk server

Configuring SOAR Servers
1. In the Splunk App for SOAR,
select the
Configurations tab
2. Click Create Server
3. Paste in the Authorization Configuration
4. Optional:
– Display name for SOAR server
– Proxy server address
– Enable Adaptive Response Relay
5. Click Save
Lab Exercise 2: Remote Search
• Tasks:
– Install the Splunk app for SOAR on Splunk
– Configure Splunk-side settings
– Configure external search on SOAR
– Migrate data to Splunk
– Execute searches on Splunk for SOAR events

Module 3:
Exporting Splunk
Events to SOAR

Module Objectives
• Describe the Splunk App for SOAR Export
• Send Enterprise Security notables to SOAR
• Automatically trigger SOAR playbooks for Splunk notables

Sending Splunk Events to SOAR
• The Splunk App for SOAR Export is designed
to forward Splunk events to SOAR
• It also provides alert actions that can be used from Enterprise
Security to send notable events to SOAR, and optionally
run playbooks
• Configuration is (mostly) on the Splunk side
• First, install the app from Splunkbase
splunkbase.splunk.com/app/3411
docs.splunk.com/Documentation/SOARExport

Phantom Role
• Add the Phantom role to inheritance
for the Admin role
• Also, for any other roles that
require access to the Phantom add-on

Phantom User
• On SOAR, create a new
automation user
– Used by Splunk server to connect to
the SOAR server
– Using the default automation user
is not a best practice
• Make sure:
– Allowed IP addresses include the Splunk server
– User type and Role should be "Automation"
• "any" is allowed for IPs but is not a security best practice

Access Token
• Edit the new SOAR automation user
• Select and copy the contents of
the Authorization Configuration
for REST API
• Record this for use on the
Splunk server

Configuring SOAR Servers
1. In the Splunk App for SOAR
Export, select the
Configurations tab
2. Click Create Server
3. Paste in the Authorization Configuration
4. Optional:
– Display name for SOAR server
– Proxy server address
– Enable Adaptive Response Relay
5. Click Save
Adaptive Response Relay
• Queue adaptive responses on local Splunk
heavy forwarder for forwarding to SOAR
– Search head(s) send adaptive
response actions to forwarder
– Optional alert action naming
• Forwarder stores actions and

forwards to SOAR
– Example: send actions to SOAR when resources are most available
docs.splunk.com/Documentation/PhantomApp/latest/UserGuide/Adaptiveresponseactions

Multi-Value Field Handling
• By default, fields in Splunk events
with multiple values generate
duplicate artifacts when sent by
ES Adaptive Response or the
sendtophantom alert action
– Can be converted to lists instead
• This does not apply to event forwarding, which always converts
multi value fields to lists

Testing Connectivity
Connectivity issues are often a result of incorrectly configured SSL

certificates on the SOAR server my.phantom.us/kb/7

Syncing Playbooks
• SOAR Export app, Phantom Server Configuration
tab, Manage menu for each server
• This step configures the Splunk server with a list of
all the Playbooks available on the SOAR server
• This is a static list; repeat this whenever new
playbooks become available
• Stored in phantom.conf

Managing Workbooks
docs.splunk.com/Documentation/SOARExport/latest/UserGuide/Manageworkbooks
Manage workbooks across

multiple SOAR servers

Exporting Events to SOAR
• The SOAR Export app can execute scheduled searches to send
any types of Splunk events to a SOAR server
– Not just notable events
– Can be based on a saved search or a data model
– Interval or real-time
• The SOAR Export app will:

– Execute the search
– Map result fields to CEF
– Create containers on the SOAR server
– Add event data to artifacts in the containers

Create a Saved Search
`notable` to get full
details on notable
events multi-value fields are passed as a list or use
mvjoin() to consolidate into a string of
comma separated values
When saving, use a naming
If it is a notable event, include the event_id; Configure just the convention, like "soar-export-xxxx", to
also include a field with the text you want to fields you want to help identify these searches
use to identify the container in the analyst send to SOAR
queue (the container name property) After saving, make
sure the permissions
Also include a field (like source) for the new saved
that will be used to create the search are
container name accessible from the
SOAR Export app

Searches for SOAR Export
• Event forwarding uses saved
searches to select event data to
forward to SOAR
• The saved searches should select the
events and fields to be
forwarded
• Make the search readable
by all users in the SOAR
Export app OR global

Configuring Event Export: 1
SOAR Export App > Event Forwarding > Add New
If your search is not available to select,

make sure it has global access permission
The data model export is
similar, does not require
a saved search; select a
data model and object This is the target SOAR
to forward server configuration name
For container name, select

field with event description Label defaults to "events";
custom labels must exist on
SOAR server
Optionally set time range

Configuring Event Forwarding: 2
Use grouping to create Select CEF data

multiple containers type
based on a field's values.
Remove field
Each discrete value
from forwarding
generates 1 container,
with all matching event
fields contained in it. This
feature can be difficult to Add field to forwarding
manage, use only if
needed.
Fields in the saved Select matching CEF

search results name or enter new

Save and Preview
Allows viewing
results from a
selected time range
Use to send existing

events to SOAR now

Event Forwarding Management
• After saving, forwarding configurations are accessible in the
SOAR Export Event Forwarding page
• Enable/disable
• Delete or clone
• Use Save button at bottom after changes

Event Forwarding Configuration
Event forwarding is configured
as a saved search alert that
calls your saved search
The alert sends the search results

to the phantom_forward.py
script

Global Mappings
• SOAR Export App > Configure Global Field Mappings
• View and change any saved mappings
• Mappings are saved in phantom.conf

Custom CEF on SOAR Server
If you create new CEF values
in your mappings, define them
on the SOAR server and give
them a standard data type, so
they can be handled properly
by SOAR actions
A notable event event_id field is a

special case: make sure to define it as
a "splunk notable event id" CEF
field for SOAR to recognize it for
context-aware actions

Adaptive Response: Send to SOAR

Container Created from a Notable Event
• Fields from notable event are stored in artifacts
– Container name is copied from notable correlation search name
• Global field mappings are applied

Run Playbook Adaptive Response
• Sends notable event
• Executes the selected
playbook on the new
container

Debugging Adaptive Responses
• Use ES Audit Adaptive Response Action
Center to search for sendtophantom
action name
• Search in cim_modactions index for errors
related to adaptive response invocation
Tip: use searches like this as an

alert to be notified of errors during
event forwarding

Calling sendtophantom in a Search
Can be executed
directly from a search
page
Fields will be added to
CEF in artifact; if field
names do not match
defined CEF field
names, context is
not set
Set required
parameters; permits
custom values
This approach bypasses the field mapping built into the Multi-value fields are handled according
Phantom App and allows for more control, especially to the advanced settings for the
for custom fields like notable event IDs. connection.

sendtophantom Details
Notable events have a rule_title field
with the simple name of the correlation
search
• If source field exists, it becomes name of container

• If search_name field exists, it becomes name of artifact(s)
• Supports custom values for severity, label, etc.
• sendalert sendtophantom is called once for the entire result set
– Severity, sensitivity and label is same for all containers
sendtophantom: Non-notable Searches

Calling sendtophantom as an Alert
• To use sendtophantom in a scheduled search,
use the Edit Schedule option and add Send to
SOAR action
– Don't use |sendalert … in search
• Container name will be name of saved search
• Because notable events don't have an event
ID until after they are saved, its not a good
practice to call sendtophantom from a
correlation search
– The container on SOAR and the notable in ES won't be linked

Comparing Splunk-to-SOAR Methods
• Event Forwarding
– Usefulmapping tools but can't use nonstandard severity, status, data
types, etc.
• |sendalert sendtophantom
– Mustdo all field mapping in search but can adapt to SOAR-side
customizations
• Adaptive Response for notable events
– Limited to one event at a time, inflexible field mapping
• Splunk app on SOAR (data poll ingest)
– "pull" method on timed approach only, no CIM|CEF remapping

The Risk Notable Playbook Pack
• A collection of playbooks and workbook guides
• Provides powerful automation tools for investigation of risk
notables
• Requires Splunk ES, generating Risk notables
docs.splunk.com/Documentation/ES/latest/User/RiskScoring
docs.splunk.com/Documentation/ESSOC/latest/user/Useplaybookpack

Lab Exercise 3: Sending Events to SOAR
• Tasks:
– Install the Splunk App for SOAR Export
– Configure Splunk and SOAR for connectivity
– Use Adaptive Response actions to send notable events to SOAR
– Configure event forwarding to send events to SOAR in real time

Module 4 :
Accessing Splunk
from SOAR

Module Objectives
• Install and configure the Splunk app on SOAR
• Use the Splunk app to execute searches in Splunk indexes
• Run Splunk searches from playbooks
• Update notable events from playbooks

SOAR Splunk App
• Install and configure
on SOAR
• See documentation
• Actions
host events: search in Splunk for events related to a server
– Ge t
– Run query: execute a search and retrieve results
– Update event: change a notable event's status, urgency, or
add comments
– On poll: create containers with results from Splunk events

Configure Splunk Asset
Splunkd port
Username: used to
access the splunk
server.
Time zone is
required; typically
Don't enable certificate
UTC verification unless you know
that the splunkd (not splunk
Don't need to set any of web) port is configured with a
the data ingestion valid certificate (our lab
parameters on Asset servers do not have
Settings, but a default certificates for splunkd.)
label is required on the
Ingest Settings tab.

Searching from Playbooks
• Using the run query action in a playbook enables more
investigative tools
• Example:
– An event is ingested indicating a potential virus infection on a server
– Use run query to discover other servers that have recently
connected with the infected server
– Use this list to hunt for the virus and/or isolate the suspect systems

Plan the Search
Work out a search in Splunk Make sure you use the
first, and configure it to run as same namespace context
efficiently as possible in Splunk to test your
search as you will use when
In this example, host-001 is you connect using the
used as a sample host name for Splunk asset from SOAR or
an infected server and working set its ACL to have global
out a search to retrieve a list of access (i.e., all apps).
all other servers this one has
connected to

Calling Saved Searches
• Use the savedsearch command to call saved searches on Splunk
Saved search named mysearch with a variable $server$:
host = $server$ …
Call the mysearch saved search, and substitute www1 for $server$:
|savedsearch mysearch server="www1"
• This command could be used in the run query action in a
playbook
– Use a format block to fill in the variable value

Add the Search to the Playbook
You can compose and use any
SPL here but calling a saved Insert variable values with
search on Splunk lets you keep the CEF or result values
SPL out of the playbook. Easier
maintenance on the Splunk server.
Here you just need to put the name
of the saved search and fill in any
required variables.
Use a format block to take a CEF

value (like destinationHostName)
and insert it into the search you
planned

Calling the Run Query Action
If calling a saved search, use the
savedsearch command; otherwise use
search to run the query, or other command Fill in query from format
as needed block, or use optional
Display can extract a subset of fields to formatted parameter
return from the search, useful if it's a
saved search and you don't have access
to the search definition parse_only to test the query
without running it
attach_result will copy the results as a
JSON text file to the file vault
Fill in end_time and

start_time if needed, or can
search_mode: smart (default) returns field be part of query
extractions for fields used in query; fast
returns only fields listed in display or
|fields command; verbose returns all
available fields

Utility Bock: Merge Lists
• The list_merge custom function in
the utility block can take 2 or more
inputs and merge them into one list
– Useful when you have 2 possible CEF
values and you want to handle them
as one
• The downstream combined output is

blockname:custom_function_result.data.*.item

Results from Run Query

Query Action Result Structure
These are examples of how
these results can be passed
as inputs to downstream
blocks
run_query_1:action_result.status
run_query_1:action_result.data
run_query_1:action_result.summary.total_events

Update Event Action
Use to update notable The event ids field requires
event status, owner, one or more notable event
urgency, or to add a IDs. Ideally, notables
comment. imported from ES to SOAR
will include a field
containing the event ID. To
enable context awareness
for event ID CEF fields, add
a new custom CEF field of
type Splunk Notable Event
ID with name event_id.
then, during import from
ES, imported values with
that field name will
automatically be
recognized as notable
event IDs.

Review: I2A2 Design
• I2A2
– Inp uts :
required information
– Interactions: apps or people
– Actions: operations performed
– Artifacts: outputs or changes
• Goals
– Simpleplaybooks
– Reasonable scope
– Modular designs

Example I2A2 Design Diagram
Ye Delete
Get file Score Copy to
> 10? s from file End
reputation vault
system
N
o

Lab Exercise 4: Searching in Splunk
• Tasks:
• Design a new playbook:
– Search for peers of the destinationHostName or
destinationAddress in the container: hosts that have either been
a src or a dest of the infected server
– Update the related notable event in Splunk: change its status to in
progress, and add a comment with a link to the container in SOAR
• Begin developing the playbook

Module 5:
Custom Coding

Module Objectives
• Edit the global block
• Use custom function blocks
• Using the SOAR playbook API in code
• Working with artifacts and files

Review: Playbook Code
• Playbooks use Python 3
• The on_start() and on_finish() blocks are pre-defined when
you create a playbook
• on_start() is called first
– Use it for initialization, etc.
• on_finish() is called after all other blocks have exited
– Use it for cleanup, logging, etc.
• The global block is executed when the playbook is loaded
– Anyobjects initialized here are not released when the playbook
ends—not a good place for variables, credentials, etc.

Python Versions
• All new playbooks are coded in Python 3
• Legacy playbooks on Python 2.7 can be converted to Python 3
• Command line tools for migrating 2.7 to 3:
– playbooks_to_py3
– customfunctions_to_py3
– docs.splunk.com/Documentation/SOARonprem/5.0.1/PlaybookAPI/C
onvertFromPython2toPython3

Custom Code and the VPE
• Place all custom code in:
• A code block
• custom function
• Modifying code in other block types "locks" the block in the VPE
• Can't use the block configuration panel
• Modifying the links from this block to other blocks ignored
• Code modifications can be reverted
– Copy/pastethe custom code code first if it's needed, and paste into a
new code block

Playbook API
• The Playbook API provides a Python module to assist writing
code in playbooks
docs.splunk.com/Documentation/SOARonprem/latest/PlaybookAPI/Introduction
• Many common functions are used in coding, such as:

Function Use
phantom.debug() output to debug window
phantom.collect, collect2() get data from datapath[s]
phantom.create_container() create new event
phantom.add_artifact() add artifacts to containers

Custom Code versus VPE
• While the all-code approach might be quicker to develop for
some, it is not necessarily maintainable in the future
• Execution performance and high-volume tasks can be more
efficient via custom code versus VPE
• VPE-based playbooks take more time to design and construct,
but are more maintainable over time by a larger team of
non-programmers
• Often, you'll find a balance is needed between VPE based design
and expedited coding

Installing Additional Python Packages
• Must be done on all nodes of a cluster
• SSH onto the SOAR server as phantom user
• Run phenv pip install packageName
– Run phenv pip -V to check
• For more details:
– docs.splunk.com/Documentation/SOARonprem/latest/DevelopApps/FAQ

Editing the Global Block
1. Open a new playbook (or put your
playbook in edit mode)
2. Open the Python Playbook
Editor
3. Click on the Global Block Icon
4. Edit the direct Python code

Global Block Uses
• Use the global block to add import statements or make
other definitions
• Avoid executing code in global
– It is only executed once when the playbook is first loaded
• Use on_start() for initialization code that must run at the
beginning of each playbook execution

Example: Global Block Function def
• This example:
– Importsthe re package
– Defines a regex function
• The function can now be

used in all block code of
this playbook

Inline Code Block
• Inline code stored within the defining playbook
– Pro:easy to add quickly to a playbook
– Con: Can't be easily shared between playbooks
• Code blocks are an excellent tool to use if you need to generate a

value that will be used by later blocks and can't be created easily
with a format block
– Example: check to see if the event's artifacts have either an IP
address sourceAddress or a sourceHostName
– Output the value as "Source" to be used in a later block that can work
on either IP or hostname input

Using the Inline Code Block
Name becomes name
of function
Define input parameters

and output variables
Input parameters are local

variables
Output variables are local variables,

and will become part of the playbook
run data—available as inputs to other
blocks
All code here

Set output variables but don't use
return; automatically handled with
phantom.save_run_data()

Custom Functions
• Custom functions:
– Define completely new blocks for use in playbooks
– Written in Python and can be defined with input and output parameters
– Excellent for custom code that is used by many playbooks
• Stored in git repos like playbooks

– Repo can be shared between SOAR servers
– Functions can be exported and imported
– The community repo contains out-of-the-box utility custom functions

Custom Function Management
• Create, update, import
• Select a custom function to export or delete
• Deleting or updating a custom function affects all
dependent playbooks
• Expand a row to see inputs, outputs and dependent playbooks

Creating Custom Functions
• On Custom Function page, click +CUSTOM FUNCTION
• Add a name and description
All your code

should go here
Click to add inputs

or outputs

Custom Function Input Arguments
• List
– Pass argument as a list
– Static values passed as single element list
– Function called once for entire list
• Item
– Call function once for each element in the list
• Optional: CEF type, placeholder, help text,
additional inputs

Custom Function Output Parameters
• Data Path: output variable name
– Name of result value
– Written to object store
• Available for retrieval with phantom.collect2()

– Or in pick lists for inputs to other playbook blocks
• Example output data path assuming custom function called in a
playbook from a block named test:
test:custom_function_result.data.out1
• Other fields optional
– CEF type: for VPE input parameter selection lists
Coding Custom Functions
• Inputs defined in function header
• Outputs defined as an empty
dictionary, populate it with results
• Add imports and Python code
• Most Phantom API calls allowed
• see
docs.splunk.com/Documentation/SOARonprem/latest/Playbook/VPECustomFun
ctionBlock#Playbook_APIs_supported_from_within_a_custom_function
• Use VALIDATE to check code
• SAVE when complete
Example: Regex Custom Function
• A custom function to
extract specific
strings starting with
"TC" followed by 5 to
7 digits, with an
optional space
• Output is JSON in a
Python dictionary
Outputs are
returned to calling
playbook

Using Custom Functions in Playbooks
• Custom functions are executed by the
Utility block
– Select from list
– Fill input parameters
• Output from function can be used as input

by other blocks

Custom Function Code Versioning
• All playbooks use the latest version of a custom function's code
• When a custom function is modified, the developer can choose to
deactivate dependent playbooks to avoid in-production code from
being affected before testing

Custom Function Interface Changes
• If the inputs and/or outputs of a custom function change, the
custom function must be manually updated in all playbooks
– Playbooks always call the newest version of a custom function
• Arguments are passed by name
– Calls from unmodified playbooks pass None for new arguments
– Any arguments passed that do not exist in the new custom function
are ignored

Utility Block Setup Section
def cf_local_regex_2(action=None, success=None, container=None, results=None, handle=None,
filtered_artifacts=None, filtered_results=None, custom_function=None):
phantom.debug('cf_local_regex_2() called')
parameters = [] Keyword arguments
parameters.append({'source_string': None,'pattern': None})
Parameters collection passes
################################### input arguments to custom
## Custom Code Start function; note that keyword
Playbook developer can add
################################### arguments such as container,
custom code here (setup section)
# Write your custom code here... results, etc. NOT passed by
to add/override parameters
#################################### default
before calling custom function,
## Custom Code End debugging, etc.
###################################
# call custom function "local/regex", returns the custom_function_run_id

phantom.custom_function(custom_function='local/regex', parameters=parameters, name='cf_local_regex_2')
return

Keyword Argument Inputs
• If custom code must access
SOAR objects such as the
container, action results, etc.,
they must be passed to the
custom function as
keyword arguments
• Example: custom code that
works on the container
– Add a container input argument to the custom function
– When configuring the container input in the playbook, select keyword
arguments > container
Accessing Container Data
• The container is passed as a
{ "in_case":false,
"tenant_name":"_default_",
"sensitivity":"amber",
parameter to each function

"create_time":"2018-10-09 22:58:13.22068+00",
"owner":"admin",
"closing_owner_id":0,
– And can be passed to a custom

"id":10,
"custom_fields":{ },
"close_time":"", "open_time":"2018-10-09 22:59:01.979933+00",
function as a keyword argument

"container_type":"default",
"label":"events",
"current_phase_id":0,
• Use container['key'] in code

"due_time":"2018-10-10 10:58:01.552+00",
"version":"1",
"current_rule_run_id":213,
to access container values

"owner_id":1,
"status":"open",
"owner_name":"",
"hash":"51404772b554c81a7b22e9389c1e5b77",
"description":"A file download has been detected by network scan",
"tags":[ ],
"start_time":"2018-10-09 22:58:13.225313+00",
"closing_rule_run_id":0,
"phase_name":"", "kill_chain":"",
"artifact_update_time":"2018-10-09 22:59:01.899555+00",
"artifact_count":1,
"severity":"medium",
"asset_name":"", "name":"TEST event",
"url":"https://54.202.245.23/mission/10",
"tenant_id":0,
"source_data_identifier":"c3fe2341-1b57-4963-82d1-a4ed53d9c074",
"end_time":"","ingest_app_id":"",
"container_update_time":"2018-10-10 16:25:11.494584+00" }

Accessing Artifacts and Results
• Use the collect() or collect2() functions to get artifact or
result data
– collect2() can access filtered data
• The datapath list parameter defines the item(s) to extract
data = phantom.collect2(container=container,
datapath=['artifact:*.cef.sourceDnsDomain'])
docs.splunk.com/Documentation/SOARonprem/latest/PlaybookAPI/Dat
aAccessAPI

Iterating Query Result Data
results = phantom.collect2(container=container,
datapath=['run_query_1:action_result.data'],
action_results=results)
for row in results[0]:

phantom.debug("Peer %s is found %s times." %
(row["peer"], row["count"])
In a custom function, the
• action_results.data is a single-element list results object must be

passed as a keyword
argument to enable the
• Each row in the query result is a dictionary in custom code to access it.
action_results.data[0]
• In each row, each field is a name-value pair
Datapaths: Collecting the Right Data
• Use the collect() or collect2()
functions to access data values from
action results and artifact CEF fields
- collect2() can return filtered
results from a filter block

phantom.act() Optional Parameters
• Start_time: Time in the future when the action should be scheduled
for execution
• Callback: Function (defined in the same Playbook) to be called upon completion

of the action
• Assets: A list of assets on which the action is to be executed

Playbook Information
• get_playbook_info()
– Useful information like parent playbook ID
• get_playbook_summary()
– Details about playbook execution including app usage, actions, etc.
• DEMO: playbook failure alerting
– Email reports if parent playbook execution has errors
– Add to any playbook as last block before end

Use Case: Data Persistence
• Passing data from one playbook to another is a common use case
• Input playbook and End block outputs
– Simple, but limited to synchronous, single parent/child relationships
• Artifacts in a container (visible to UI) or the container data[]
– Useful when multiple playbooks all work on one container
• Disk storage (fast but requires management), or object storage
(excellent for simple values), custom lists
– Useful for "global" cross-playbook data storage

Save_run_data / get_run_data
• Save data in the playbook context
- Deleted when playbook ends
• Not visible through the UI
• Only JSON compliant objects,
dictionaries, lists, strings and
numbers are supported as objects
that can be saved and retrieved
• Not supported in custom function code

Save_object/get_object
• Stores and retrieves into the
SOAR database
• Key-based: repeated updates
of the same key overwrite
• auto_delete deletes data
related to a container when that
container is deleted
• Data can persist after playbook execution and can be retrieved by
other playbooks
• Not supported in custom function code
Vault API
• Get data about files that are in the
container vault
• Add new files to the vault
• Use this to get hashes, or files to
follow up
– Detonate file
– File reputation
• Deprecated API calls:

– Vault.add_attachment()
– Vault.get_file_path()
– Vault.get_file_info()
Updating the Container
• You can also update the
properties of the container itself
– Change the name
– Change the severity
– Change the SLA
• Pass JSON that corresponds

directly to the names of the container’s fields
• Optionally:
– Phantom.set_label()
Note
– Phantom.set_severity()
The API block can also do these
actions from the UI in a playbook.
– Phantom.set_sensitivity()

Container Data: Write
def write_container_data(action=None, success=None, container=None, results=None, handle=None,
filtered_artifacts=None, filtered_results=None):
phantom.debug('write_container_data() called')
input_parameter_0 = "my_data_2"
input_parameter_1 = "More Stuff"
################################################################################
## Custom Code Start
################################################################################
#input_parameter_0 is the key

#input_parameter_1 is the value
data = phantom.get_container(container['id'])['data']
data.update({input_parameter_0:input_parameter_1})
phantom.update(container, {'data':data} )
################################################################################
## Custom Code End
################################################################################
get_container_data(container=container)
return

Container Data: Read
def get_container_data(action=None, success=None, container=None, results=None, handle=None,
filtered_artifacts=None, filtered_results=None):
phantom.debug('get_container_data() called')
input_parameter_0 = "my_data_2"
get_container_data__container_data = None
################################################################################
## Custom Code Start
################################################################################
get_container_data__container_data = phantom.get_container(container['id'])['data'][input_parameter_0]
phantom.debug("read: " % get_container_data__container_data)
################################################################################
## Custom Code End
################################################################################
phantom.save_run_data(key='get_container_data:container_data',
value=json.dumps(get_container_data__container_data))
return

Working with Lists
• Utility block Add to List, Remove List
– Useful for basic operations
• Phantom App list actions
– Search,multi-column rows,
update individual rows
• Phantom API get_list(), add_list()
– Flexible, supports search, other operations
• Lists are stored as single value (BLOB) in object database
– No locking, use caution where multiple playbooks could be using a
list at the same time

Creating Containers and Artifacts
• Full API for container and artifact operations
– docs.splunk.com/Documentation/SOARonprem/5.0.1/PlaybookAPI/ContainerAPI
• phantom.create_container()
• Phantom.add_artifact()
sta, msg, cid = phantom.create_container(name="XXXXXXX", label="events")
phantom.add_artifact(container=cid, raw_data={}, cef_data={"userName":"admin"},
label="XXXX", name="XXXXXX", severity="high", artifact_type="XXXXX")
• Also: Phantom App actions for containers and artifacts

– Use JSON to define new objects

Lab Exercise 5: Custom Code
• Tasks:
– Write the results of the peer search to a custom list
– Pass the name of the custom list to a new playbook
– Use the new playbook to create new events in SOAR for any peers
that are "high" or "critical" priority

Module 6:
The SOAR REST API

Module Objectives
• Access SOAR REST documentation
• Use REST endpoints to access SOAR data
• Use REST to send data to SOAR

Basic REST Query Endpoints
• Basic endpoint URL: • Returns all objects of that type
https://servername/rest/[type]
• Default list output is paged
• Query types: – Add page=X
action_run • Add page_size=0 to return all
artifact
asset • Add /id to access a specific
app object
app_run
container
playbook_run
cluster_node
docs.splunk.com/Documentation/SOARonprem/latest/PlatformAPI
Filtering and Selecting
• By exact field value
.../rest/container?_filter_field=value For subfields, use
"_field__subfield"
– Use quotes for string values, [ … ] for list values, & for AND
• By case insensitive substring in a field value
.../rest/container?_filter_field__icontains="value"
– Notedouble underscore; use "contains" for case-sensitive, lots of

other functions, like startswith, isnull, etc.
• Select only a specific field
.../rest/container/X/name

Object Detail
• Most object types support additional detail
.../rest/container/X/artifacts
– Container contents plus artifacts related to this container

.../rest/container/X/actions
.../rest/container/X/playbook_runs
.../rest/ph_user/X/roles
.../rest/action_run/X/app_runs
– And many more
docs.splunk.com/Documentation/SOARonprem/latest/PlatformAPI/RESTQueryData#Requesting_Object_Detail

Creating and Updating Objects
• Use POST to create new objects
curl –u uid:pwd https:servername/rest/container
–d '{"name":"hello world", "label":"events"}'
– Must pass required values

– Returns new object ID
• Update an existing object

curl –u uid:pwd https:servername/rest/container/X
–d '{"status":"open"}'

Adding Artifacts to a Container
Create a new artifact with basic configuration
curl –u uid:pwd https:servername/rest/artifact
–d '{"run_automation":"false",
"container_id":"X",
"name":"xxxxxx",
"cef":{"destinationHostName":"xxxxxx"}}'

Data Ingestion via REST
• Use the REST app
• Configure an asset for each type of REST data
– Each asset becomes a new endpoint to post data
• Configure processing scripts to convert incoming data into proper
JSON format

Using REST in Playbooks
• Configure an asset for the HTTP app to
query the server
• Configure the asset's Base URL to
include …/rest

REST Input/Output
• Use format blocks in the playbook to structure
the REST URLs used in the Location parameter
• Result data from get_data is contained
in the parsed_response_body
data element

Accessing get_data Results
• A single object result in parsed_response_body (like
"container/X") is stored as a dictionary within a set of nested lists
[
[ { "name":"xxx","label":"yyy",...} ]
]
• Access the object fields with code like:

container_id = results[0][0]['id']

REST API in Custom Functions
• Use phantom.build_phantom_rest_url() to compose URLs for
REST API
• Use phantom.requests.get() to call REST API endpoints from
custom function code
• See example at
docs.splunk.com/Documentation/SOARonprem/latest/Playbook/VPECustomFu
nctionBlock#Use_the_REST_API_from_within_a_custom_function

Accessing list-based get_data Results
• Lists of objects are stored in a data list, with a count attribute:
[ [
{"count":"N",
"data": [{ object 1 },
{ object 2 },
{ object N }]
] ]
• Access the object fields with code like:

container_ids = []
for container in get_info_result[0][0]['data']:
container_ids.append(container['id'])

Lab Exercise 6: Using SOAR REST
• Tasks:
– Experiment with REST from the browser to search for specific data
– Use REST calls in a playbook to get a list of custom list names
– Delete temporary custom lists

Wrap-up Slides

Community
• Splunk Community Portal • Slack User Groups
splunk.com/en_us/community.html splk.it/slack
– Splunk Answers • Splunk Dev Google Group
answers.splunk.com groups.google.com/forum/#!forum/splunkdev
– Splunk Apps
• Splunk Docs on Twitter
splunkbase.com twitter.com/splunkdocs
– Splunk Blogs
splunk.com/blog/ • Splunk Dev on Twitter
twitter.com/splunkdev
– Splunk Live!
splunklive.splunk.com • IRC Channel
– .conf #splunk on the EFNet IRC server
conf.splunk.com

Splunk How-To Channel
• Check out the Splunk Education How-To channel on YouTube:
splk.it/How-To
• Free, short videos on a variety of Splunk topics

Support Programs
• W eb
– Documentation: dev.splunk.com and
docs.splunk.com
– Wiki: wiki.splunk.com
• Splunk Lantern
Guidance from Splunk experts
– lantern.splunk.com
• Glob al S up p or t
Support for critical issues, a dedicated resource
to manage your account – 24 x 7 x 365
– Web: splunk.com/index.php/submit_issue
– Phone: (855) SPLUNK-S or (855) 775-8657
• Enterprise Support
– Access customer support by phone and manage your
cases online 24 x 7 (depending on support contract)
Thank You


Content (2) - 1

Uploaded by

Copyright:

Available Formats

Content (2) - 1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Content (2) - 1

Uploaded by

Copyright:

Available Formats

SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

SOAR can run searches in

Splunk SOAR Advanced Implementation

– SOAR – Passing data between

– Splunk app on SOAR – Creating objects

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk app on Artifacts

Splunk SOAR Advanced Implementation

SOAR app on Artifacts

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

poll invoke action

event ingestion app execution

web server web framework

orchestration & decision making Low

concurrent playbook severity-based

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

3. Restart Splunk: bin/splunk restart

Splunk SOAR Advanced Implementation

3. Restart Splunk: bin/splunk restart

Splunk SOAR Advanced Implementation

• 3. Restart nginx service

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Make sure to uncheck

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

• Does not include case tasks, phases, workﬂows or docs

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

• restsoar search command

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation

Splunk SOAR Advanced Implementation