This document discusses Splunk for developers. It provides an overview of empowering developers with Splunk, building Splunk apps, and gaining application intelligence across the development lifecycle. Key points include instrumenting application logs for insights, integrating and extending Splunk, building unit testing and code integration, and gaining end-to-end visibility across development tools. The document also discusses resources for Splunk developers including tutorials, code samples, SDKs, and developer licenses.
4. Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Build
Unit Testing
Code
Check-in Integration
Testing Deploy
Staging
Application Development Challenges
4
5. Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Build
Unit Testing
Code
Check-in Integration
Testing Deploy
Staging
Lack of visibility across the product
development lifecycle
Pressure to increase velocity and
agility with DevOps
Limited insights into behavior and
performance from application logs
Application Development Challenges
5
6. Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Quickly trace and identify errors anywhere
in the codebase with real-time search
and monitoring
Instrument your app logs to gain
application intelligence
Break down dev tool silos with real-time
insights from machine data
GAIN END-TO-END VISIBILITY
ACROSS THE DEV TOOL CHAIN
FIND AND FIX
ISSUES FASTER
PUSH BETTER CODE
USING ANALYTICS
Splunk for Application Lifecycle Intelligence
6
7. Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Real-time dashboards show error rate
in production and impact of pushing
new builds
Developers can search and visualize
web logs, Java logs, eventlogs etc;
trace tx without complex
instrumentation
Alerts notify developers as soon as a
problem arises
Find and Fix Issues Faster
7
8. Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Gain end-to-end visibility to make
informed decisions
Analytics insights without the need for
additional analytics tools
Ask questions while exploring and
collecting data
Push Better Code Using Analytics
8
9. Grigori Melnik, Principal Product Manager – Splunk Developer Platform9 9
CI / Build
Servers
Project and Issue
Tracking
Code
Repository
QA / Testing
Tools
End-To-End Visibility Across The Dev Tool Chain
Deployment Servers /
Automation
11. Grigori Melnik, Principal Product Manager – Splunk Developer Platform11
CI / Build
Server
Code
Review
Task
Tracking
What Data Can You Splunk?
Logs – Which code has already been reviewed for this release/sprint? Who has
completed the most code reviews? What code has NOT been reviewed?
Logs/API – Who is changing files? What kinds of files are being changed? What
branches are most active? What types of activities are occurring for a branch?
Version
Control
Logs/API – How many builds completed today/this week/this month? Which
check-in kicked off this build? Which tests ran against this failed build?
Logs – Which tasks are assigned to which developers? What progress is being
made to complete assigned tasks? What tasks remain for this release/sprint?
1
12. Grigori Melnik, Principal Product Manager – Splunk Developer Platform12
Key Benefits of Application Lifecycle Intelligence
Reduced Time
to Market
Shrink the time it takes
to get code through
dev/test to market
through faster issue
identification and
resolution
Increased
Agility
With real-time visibility
into processes like code
check-ins, builds and
tests to support
DevOps practices like
continuous integration
“Our devs are now able to
find and fix issues five to ten
times faster.”
“We can monitor all the
automation and handoffs it
takes to deploy 5-10 times
a day”
Application
Insights
Instrument customer
application logs to
capture critical
business events and
user behavior
“My code isn’t ready until it’s
Splunk-ready”
1
15. Grigori Melnik, Principal Product Manager – Splunk Developer Platform15
Evolving the Splunk Platform
Collection
Indexing
Search Processing Language
Core Functions
Inputs, Apps, Other
Content
SDKs & plug-ins
Operational Intelligence Platform
Content
Core Engine
User and Developer Interfaces
Web Framework
REST API
16. Grigori Melnik, Principal Product Manager – Splunk Developer Platform16
Powerful Platform for Enterprise Developers
1
REST API
Build Splunk Apps Extend and Integrate Splunk
Simple XML
JavaScript/CSS Extensions C#
JavaScript
Python
Ruby
Java
PHP
Data Models
Search Extensibility
Modular Inputs
SDKs
KV Store
17. Grigori Melnik, Principal Product Manager – Splunk Developer Platform
Log directly to
Splunk via TCP,
UDP, HTTP
Integrate search
results with other
applications using
custom
visualizations
Create and run
searches from
other applications
The REST API and SDKs
17
VisualizeSearch Manage
Add/Delete Users
Manage Inputs
Index
18. Grigori Melnik, Principal Product Manager – Splunk Developer Platform18
The Splunk REST API
Exposes an API method for every feature in the product
– Whatever you can do in the UI – you can do through the API
– Index, Search, Visualize, Manage
API is RESTful
– Endpoints are served by splunkd
– Requests are GET, POST, and DELETE HTTP methods
– Responses are Atom XML & JSON
– Versioning as of Splunk 5.0
– Search results can be output in CSV/JSON/XML
1
19. Grigori Melnik, Principal Product Manager – Splunk Developer Platform19
SDKs Overview
19
• Stay true to the semantics of the particular language
• E.g. Keep Python “pythonic”
• E.g. C#: Fully async , PCL, support for Rx
• Provide implementation that feels natural to the developer
• E.g. Project, build, IDE (where applicable) support
• Cover REST API endpoints based on use cases of language
• Namespaces
• owner: splunk username (defaults to current user)
• app: app context (defaults to default app)
• sharing: user | app | global | system
22. Splunk Developer Guidance
Splunk Reference Apps
Complete, working real-world Splunk solutions
built together with partners (Conducive; Auth0)
̶ 2 (pseudo-) production releases
̶ entire code & test repos on GitHub
̶ under Apache 2.0
Associated Guidance
I. Start-to-Finish Journey Documentary
II. Essentials
dev.splunk.com/goto/devguide
23. 1. Started with a Questions BacklogArchitecture
– What does a typical Splunk application reference architecture look like?
– What common paradigms are applicable to Splunk app development?
– What are the typical deployment topologies? Why should I choose a specific one? What are the confounding factors
on the choice of my topology?
– How do I partition my Splunk solutions?
– What are the tradeoffs of various types of inputs?
– How do I architect my Splunk solution and deployment for a very large scale?
– How do I architect my Splunk solution for the cloud? What are specific considerations for deploying to AWS or Azure?
– What’s the landscape of Splunk extension points?
– How do I integrate data from Splunk into existing applications and systems?
– How do I plan and design a robust alerting and monitoring subsystem on top of Splunk?
– What should I consider for my sizing requirements?
– What are recommended configurations of Splunk deployment to meet my sizing requirements?
– Should I architect my solution to index my data in local data center (zone) or centrally?
– What are things we can automatically degrade so we can make sure our core experience is working?
– When something happens, how do I effectively propagate the info and react to it?
– How are other solutions on Splunk built? What were the challenges? How have they been addressed?
Packaging and Deployment
– How do I piece together various parts of a Splunk app (custom search commands, mod inputs etc.)?
– How do I package a Splunk solution with a single install that automatically rolls out all the necessary dependencies?
– How do I manage my Splunk solution versioning, backward and future compat?
– What's the best way to split up custom apps for deployment?
Development
– How should I set up my development environment to be productive with Splunk?
– What are different ways of how I develop my Splunk app ? Pros and cons of using specific SDK vs REST APIs?
Pros and cons of using SimpleXML vs Advanced XML vs Web Framework …
– How do I analyze a data source for a TA?
– What are the different ways of enriching the data in Splunk? What are their tradeoffs?
– When should I use event types and transactions for data classification?
– How do I extend Splunk to define a custom input capability?
– When should I use modular inputs vs scripted inputs vs..?
– What are streaming vs non-streaming outputs considerations?
– How do I deal with long-running scripts? Handling shutdown/restart of Splunk? Concurrency? State persistence etc.
– Why should I not use transactions?
– When should I use pivot vs tstats?
– Why should I use data models?
– When my data source touches on many data models, should I assume complete separation or heavy inheritance?
– How do I extend an existing data model?
– What does CIM offer and why should I build CIM-compliant apps?
– In the context of CIM, what are the tradeoffs of using my props.conf and transforms.conf and rewriting them on
indexing, completely discarding the vendor supplied field names? How do I reconcile the advantages of a clean
interface & normalisation, but at the cost of losing alignment with published vendor documentation, and a learning
curve for existing users?
– How do I manage my solution declarative configuration? How do I detect/troubleshoot bad config?
– How do I log and analyze data that is not event driven (certain web feeds, html parsing, image meta data)?
– Compare and contrast ad-hoc searching vs background searching
– How do I handle transient faults?
– How do I effectively manage credentials?
– What’s the effect of search head location on my app and the overall user experience?
– How do I develop an integrated mechanism to let me connect Splunk to my MOM (messaging middleware) and index
my messages?
– How do I handle the requirement that app configs must be different across different server types in a distributed
environment (e.g. apps on search heads shouldn't have inputs enabled)?
Quality/Compliance
– What quality gates should I consider? What kind of para-functional characteristics are important to consider?
– What heuristics do I use to bless/block a release?
– How do I test a data model?
– How do I prepare event generation when building/testing an app?
– What kind of perf testing should I do and how?
– How do I test UI?
– How do I security certify my solution?
– How do I design to satisfy my retention and compliance policies?
– How do I architect to design my availability requirements?
– How do I handle geographic disaster recovery / fault tolerance?
– How do I properly instrument my solution so that I know what’s happening?
Sustained Engineering
– How do I maintain/service/support Splunk apps?
– How do my customers handle updating their customized configs once new versions of my app come out?
Business
– Why should I build on Splunk?
– What kind of skill do I need my devs to have to build a Splunk solution?
– What is the community building? How are current devs creating unique experiences using Splunk – I typically want to
see some marketplace success
– Cost and pricing are very important to me as a entrepreneur developer. If I am coming in to build a tool that will be
commercialized I need to know that the cost structure of Splunk won’t cause my service to be economically
unprofitable.
What does a typical Splunk application architecture look like?
How should I set up my dev environment to be productive with
Splunk?
How do I integrate Splunk into existing systems?
How do I prepare my event generation when developing &
testing an app?
How do I package an app? deal with app versioning and updates?
24. 2. Mined business requirements with partner
3. Formulated learning objectives
4. Reconciled 2 & 3 with our designs
…
25. Data
Search language
Aggregating siloed metrics into
meaningful KPIs
Data manipulation
Data normalization
Sub-searches
Config-driven
Persistence with KV store
Macros
Viz:
Dynamic scaling
Customizing in-the box viz
controls
General search patterns
Search optimizations
Ux Prototyping
Adapting 3rd party viz library
Composite charts with interactions
Dealing with high-volume data sets
Troubleshooting perf issues
Post-process or not-post-process –
deployment implications
Automated UI testing (w.Selenium)
Setting the stage
Overall Splunk app structure
UI technology selection:
Simple XML vs SplunkJS
Modularity
Dev & test env
Dev workflow
Modularity
Data onboarding
CIM compliance
Tools
Post-processing
Integrating with 3rd party
component
Unit testing (w.Mocha)
Persisting state (per user)
Data modeling
Using lookups
Building a baseline lookup table
Windows of time/Custom time ranges
Overlaying time data
Using sub-searches to correlate data
Troubleshooting searches
Custom nav
Ux activities permeating all dev
Data mining:
Exploration
Preparation: filtering/deduping/
bucketing
Using advanced statistics functions
Threshold-based anomaly detection
Evaluating goodness /accuracy
Plus non-functional topics: App versioning
Packaging Installation
Security review
Deployment
Publishing to splunkbase
App certification
Order Flow, message queues, Garbage Collection, Java Heap
identify errors by java class, thread
alert actions - jira ticket, service now ticket, webhook
PM’s love to look at feature usage; are new features being used?
How do we allocate developer time to create/enhance features
Roll Ubisoft Video
http://www.splunk.com/en_us/resources/video.5rcTNqdDpGoBAtUNietLLxtORoC0QW7Y.html
Compliance - Developers can’t login to production systems but Splunk allows them to troubleshoot
10GB of indexing/day
Full Enterprise Features
Free trial for 6 months
Finds and traces bugs in real time so you can fix them faster
Reduces time to market through faster issue identification and resolution
Provides insights into user behavior and application usage
Works with the applications, tools and systems that dev teams use every day for full lifecycle visibility
We’re headed to the East Coast!
2 inspired Keynotes – General Session and Security Keynote + Super Sessions with Splunk Leadership in Cloud, IT Ops, Security and Business Analytics!
165+ Breakout sessions addressing all areas and levels of Operational Intelligence – IT, Business Analytics, Mobile, Cloud, IoT, Security…and MORE!
30+ hours of invaluable networking time with industry thought leaders, technologists, and other Splunk Ninjas and Champions waiting to share their business wins with you!
Join the 50%+ of Fortune 100 companies who attended .conf2015 to get hands on with Splunk. You’ll be surrounded by thousands of other like-minded individuals who are ready to share exciting and cutting edge use cases and best practices. You can also deep dive on all things Splunk products together with your favorite Splunkers.
Head back to your company with both practical and inspired new uses for Splunk, ready to unlock the unimaginable power of your data! Arrive in Orlando a Splunk user, leave Orlando a Splunk Ninja!
REGISTRATION OPENS IN MARCH 2016 – STAY TUNED FOR NEWS ON OUR BEST REGISTRATION RATES – COMING SOON!