Machine Learning and Analytics Breakout Session
•
0 likes•515 views
This document discusses operationalizing machine learning with Splunk. It begins with an overview of machine learning and the challenges of applying it to real-time data. It then provides examples of machine learning use cases in IT operations, security, and customer analytics. The document outlines the machine learning process of getting data, exploring it, fitting and validating models, predicting outcomes, and operationalizing results. It highlights machine learning capabilities in Splunk products like the ML Toolkit, UBA, and ITSI and provides next steps for audiences to learn more.
Report
Share
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (12)
Similar to Machine Learning and Analytics Breakout Session
Similar to Machine Learning and Analytics Breakout Session (20)
More from Splunk
More from Splunk (20)
Recently uploaded
Recently uploaded (20)
Machine Learning and Analytics Breakout Session
- 1. Copyright © 2016 Splunk Inc. Operationalizing Machine Learning Brian Nash Staff Architect – Machine Learning, Business Analytics, & IoT
- 2. 2 Disclaimer During the course of this presentation, we may make forward looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in the this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release.
- 3. Copyright © 2016 Splunk Inc. Why do we need ML?
- 4. Copyright © 2016 Splunk Inc. Historical Data Real-time Data Statistical Models DB, Hadoop/S3/NoSQL, Splunk Machine Learning T – a few days T + a few days Why is this so challenging using traditional methods? • DATA IS STILL IN MOTION, still in a BUSINESS PROCESS. • Enrich real-time MACHINE DATA with structured HISTORICAL DATA • Make decisions IN REAL TIME using ALL THE DATA • Combine LEADING and LAGGING INDICATORS (KPIs) Splunk Security Operations Center Network Operations Center Business Operations Center
- 5. Copyright © 2016 Splunk Inc. What is ML?
- 6. 6 ML 101: What is it? • Machine Learning (ML) is a process for generalizing from examples – Examples = example or “training” data – Generalizing = building “statistical models” to capture correlations – Process = ML is never done, you must keep validating & refitting models • Simple ML workflow: – Explore data – FIT models based on data – APPLY models in production – Keep validating models “All models are wrong, but some are useful.” - George Box
- 7. 7 3 Types of Machine Learning 1. Supervised Learning: generalizing from labeled data
- 8. 8 3 Types of Machine Learning 2. Unsupervised Learning: generalizing from unlabeled data
- 9. 9 3 Types of Machine Learning 3. Reinforcement Learning: generalizing from rewards in time Leitner System Recommender systems
- 10. Copyright © 2016 Splunk Inc. ML Use Cases
- 11. 11 IT Ops: Predictive Maintenance 1. Get resource usage data (CPU, latency, outage reports) 2. Explore data, and fit predictive models on past / real-time data 3. Apply & validate models until predictions are accurate 4. Forecast resource saturation, demand & usage 5. Surface incidents to IT Ops, who INVESTIGATES & ACTS Problem: Network outages and truck rolls cause big time & money expense Solution: Build predictive model to forecast outage scenarios, act pre-emptively & learn
- 12. 12 Security: Find Insider Threats Problem: Security breaches cause big time & money expense Solution: Build predictive model to forecast threat scenarios, act pre-emptively & learn 1. Get security data (data transfers, authentication, incidents) 2. Explore data, and fit predictive models on past / real-time data 3. Apply & validate models until predictions are accurate 4. Forecast abnormal behavior, risk scores & notable events 5. Surface incidents to Security Ops, who INVESTIGATES & ACTS
- 13. 13 Business Analytics: Predict Customer Churn Problem: Customer churn causes big time & money expense Solution: Build predictive model to forecast possible churn, act pre-emptively & learn 1. Get customer data (set-top boxes, web logs, transaction history) 2. Explore data, and fit predictive models on past / real-time data 3. Apply & validate models until predictions are accurate 4. Forecast churn rate & identify customers likely to churn 5. Surface incidents to Business Ops, who INVESTIGATES & ACTS
- 14. 14 Summary: The ML Process Problem: <Stuff in the world> causes big time & money expense Solution: Build predictive model to forecast <possible incidents>, act pre-emptively & learn 1. Get all relevant data to problem 2. Explore data, and fit predictive models on past / real-time data 3. Apply & validate models until predictions are accurate 4. Forecast KPIs & notable events associated to use case 5. Surface incidents to X Ops, who INVESTIGATES & ACTS Operationalize
- 15. Copyright © 2016 Splunk Inc. ML with Splunk
- 16. 16 Splunk User Behavior Analytics (UBA) • ~100% of breaches involve valid credentials (Mandiant Report) • Need to understand normal & anomalous behaviors for ALL users • UBA detects Advanced Cyberattacks and Malicious Insider Threats • Lots of ML under the hood: – Behavior Baselining & Modeling – Anomaly Detection (30+ models) – Advanced Threat Detection • E.g., Data Exfil Threat: – “Saw this strange login & data transfer for user mpittman at 3am in China…” – Surface threat to SOC Analysts
- 17. 17 Machine Learning in Splunk ITSI Adaptive Thresholding: • Learn baselines & dynamic thresholds • Alert & act on deviations • Manage for 1000s of KPIs & entities • Stdev/Avg, Quartile/Median, Range Anomaly Detection: • Find “hiccups” in expected patterns • Catches deviations beyond thresholds • Uses Holt-Winters algorithm
- 18. 18 ML Toolkit & Showcase • Splunk Supported framework for building ML Apps – Get it for free: http://tiny.cc/splunkmlapp • Leverages Python for Scientific Computing (PSC) add-on: – Open-source Python data science ecosystem – NumPy, SciPy, scitkit-learn, pandas, statsmodels • Showcase use cases: Predict Hard Drive Failure, Server Power Consumption, Application Usage, Customer Churn & more • Standard algorithms out of the box: – Supervised: Logistic Regression, SVM, Linear Regression, Random Forest, etc. – Unsupervised: KMeans, DBSCAN, Spectral Clustering, PCA, KernelPCA, etc. • Implement one of 300+ algorithms by editing Python scripts
- 19. Copyright © 2016 Splunk Inc. Building ML Apps
- 20. 20 Analysts Business Users 1. Get Data & Find Decision-Makers 2 IT Users ODBC SDK API DB Connect Look-Ups Ad Hoc Search Monitor and Alert Reports / Analyze Custom Dashboards GPS / Cellular Devices Networks Hadoop Servers Applications Online Shopping Carts Analysts Business Users Structured Data Sources CRM ERP HR Billing Product Finance Data Warehouse Clickstreams
- 21. 21 2. Explore Data, Build Searches & Dashboards • Start with the Exploratory Data Analysis phase – “80% of data science is sourcing, cleaning, and preparing the data” – Tip: leverage ITSI KPIs – lots of domain knowledge • For each data source, build “data diagnostic” dashboard – What’s interesting? Throw up some basic charts. – What’s relevant for this use case? – Any anomalies? Are thresholds useful? • Mix data streams & compute aggregates – Compute KPIs & statistics w/ stats, eventstats, etc. – Enrich data streams with useful structured data – stats count by X Y – where X,Y from different sources – Build new KPIs from what you find
- 22. 22 3. Fit, Apply & Validate Models • ML SPL – New grammar for doing ML in Splunk • fit – fit models based on training data – [training data] | fit LinearRegression costly_KPI from feature1 feature2 feature3 into my_model • apply – apply models on testing and production data – [testing/production data] | apply my_model • Validate Your Model (The Hard Part) – Why hard? Because statistics is hard! Also: model error ≠ real world risk. – Analyze residuals, mean-square error, goodness of fit, cross-validate, etc. – Take Splunk’s Analytics & Data Science Education course
- 23. 23 4. Predict & Act • Forecast KPIs & predict notable events – When will my system have a critical error? – In which service or process? – What’s the probable root cause? • How will people act on predictions? – Is this a Sev 1/2/3 event? Who responds? – Deliver via Notable Events or dashboard? – Human response or automated response? • How do you improve the models? – Iterate, add more data, extract more features – Keep track of true/false positives
- 24. 24 5. Operationalize Your Models • Operationalizing closes the loop of the ML Process: 1. Get data 2. Explore data & fit models 3. Apply & validate models 4. Forecast KPIs & events 5. Surface incidents to Ops team • When you deliver the outcome, keep track of the response – Human-generated response (detailed journal logs, etc) – Machine-generated response (workflow actions, etc) – External knowledge (closed tickets data, DB records, etc) • Then operationalize: feed back Ops analysis to data inputs, repeat • Lots of hard work & stats, but lots of value will come out. Operationalize
- 25. Copyright © 2016 Splunk Inc. Show me the ML!
- 26. 26 Next Steps with Splunk ML • Reach out to your Tech Team! We can help architect ML workflows. • Lots of ML commands in Core Splunk (predict, anomalydetection, stats) • ML Toolkit & Showcase – available and free, ready to use – Get it for free: http://tiny.cc/splunkmlapp • Splunk UBA: Applied ML for Security – Unsupervised learning of Users & Entities – Surfaces Anomalies & Threats • Splunk ITSI: Applied ML for ITOA use cases – Manage 1000s of KPIs & alerts – Adaptive Thresholding & Anomaly Detection • ML New Product Initiative (NPI) Program: – Connect with Product & Engineering teams - mlprogram@splunk.com
- 27. 27 SEPT 26-29, 2016 WALT DISNEY WORLD, ORLANDO SWAN AND DOLPHIN RESORTS • 5000+ IT & Business Professionals • 3 days of technical content • 165+ sessions • 80+ Customer Speakers • 35+ Apps in Splunk Apps Showcase • 75+ Technology Partners • 1:1 networking: Ask The Experts and Security Experts, Birds of a Feather and Chalk Talks • NEW hands-on labs! • Expanded show floor, Dashboards Control Room & Clinic, and MORE! The 7th Annual Splunk Worldwide Users’ Conference PLUS Splunk University • Three days: Sept 24-26, 2016 • Get Splunk Certified for FREE! • Get CPE credits for CISSP, CAP, SSCP • Save thousands on Splunk education!
Editor's Notes
- Time for ML demo! Get the ML App: http://tiny.cc/splunkmlapp Want more? Take Splunk’s Analytics & Data Science course! Course prework: http://bit.ly/splunkanalytics
- We’re headed to the East Coast! 2 inspired Keynotes – General Session and Security Keynote + Super Sessions with Splunk Leadership in Cloud, IT Ops, Security and Business Analytics! 165+ Breakout sessions addressing all areas and levels of Operational Intelligence – IT, Business Analytics, Mobile, Cloud, IoT, Security…and MORE! 30+ hours of invaluable networking time with industry thought leaders, technologists, and other Splunk Ninjas and Champions waiting to share their business wins with you! Join the 50%+ of Fortune 100 companies who attended .conf2015 to get hands on with Splunk. You’ll be surrounded by thousands of other like-minded individuals who are ready to share exciting and cutting edge use cases and best practices. You can also deep dive on all things Splunk products together with your favorite Splunkers. Head back to your company with both practical and inspired new uses for Splunk, ready to unlock the unimaginable power of your data! Arrive in Orlando a Splunk user, leave Orlando a Splunk Ninja! REGISTRATION OPENS IN MARCH 2016 – STAY TUNED FOR NEWS ON OUR BEST REGISTRATION RATES – COMING SOON!