Taking Splunk to the Next Level – Management - Advanced
•
2 likes•1,569 views
Your team is up and running with Splunk. Now you want to maximize your investment and solve additional business problems. Attend this session led by a Splunk expert on how to expand beyond the initial use case. Learn how to how to capture, document and present Splunk's data and present impactful ways to calculate ROI using concrete metrics; cost savings, time savings, efficiency gains, and competitive advantage.
Report
Share
Taking Splunk to the Next Level – Management - Advanced
- 1. Taking Splunk to the Next Level for Management Doug May Director, Global Business Value Consulting Splunk> June 11, 2015
- 3. Help Splunk document the projected and already realized business value of making machine data accessible, usable, and valuable for everyone Common Deliverables: › CFO-Ready Business Case › Value Realization Studies › Usage Maturity & Staffing Readiness › Enterprise Adoption Roadmaps › Customer and Industry Benchmarks Business Value Consulting at Splunk 3 400+ Engagements Worldwide Since 2013
- 4. Focusing on Value Takes it to the Next Level 4 Your process requires it Create and maintain visibility Replicate success across the organization Accelerate enterprise adoption Maximize business results
- 5. Splunk is a Hidden Gem 5 Way cool, dude. What business value do I get? I’m invincible!
- 6. Top Challenges to Documenting Value Lack of Splunk and Industry Benchmarks x Data Lack of Tools to Make Value Measurement Easy x Tools Not Enough Time to Assess Your Value x Time
- 7. Splunk Can Help Documenting Value All Splunk Tools Are Available to All of You ToolsTime Tools, Content and Team Will Save You Time Access to Splunk and Industry Benchmarks Data
- 8. Best Practices for Documenting & Positioning Value Taking your Splunk deployment to the next level 4 Measure and Track Your Success 1 Align with Key Business Objectives Qualify and Quantify Business Value 2 3 Incremental Steps with a Big Picture Plan
- 9. Value is in the Eye of the Beholder 1 Align with Key Business Objectives Did you know you can save 15% on your car insurance when you call Geico? Is that important to you? Maybe it’s not.
- 10. Link your project to important goals and strategies to prioritize your project REAL EXAMPLE -Aligning with Company Priorities 10 Profit Double revenues while increasing margins Productivity Design and implement to most effective and efficient business system People Attract, engage, and retain the best talent Partners Become a critical part of our customers’ growth strategies Portfolio Double servings per day and be #1 provider Planet Create advantage by fulfilling our Live Positively commitments “We also launched a productivity and reinvestment program to create $550 million to $650 million in annual savings by 2015. By freeing up resources via supply- chain optimization, improved marketing effectiveness, operational excellence and systems standardization, we can invest more in innovation, marketing and additional “feet on the street” to drive our growth.” - CEO From investor presentations, annual reports, and executive presentations
- 11. Steps to Qualify Value • Align your project with something strategic • Talk with influential and knowledgeable people • Document why something should change or be added • Describe the current challenges or barriers • Identify the “desired” state • Summarize and socialize - gain support Qualify and Quantify Business Value 2
- 12. Qualifying Value Example 12 Visibility to Environment Health & User Exp. Brute force approach providing visibility to key processes isn’t working and won’t scale Operations still lacks complete end-to-end visibility to the environment’s health, use and trends Blinds spots still exist in monitoring and data access for Operations which could help improve troubleshooting and uptime / availability Incident / Issue Notification Brute force approach to proactive monitoring isn’t working consistently and won’t scale There’s a “Waterfall effect” – small issues go without broader notification triggering other issues eventually leading to a bigger incident Users are aware of issues before Operations and call the helpdesk All the lights are “green” but still ~65% of incidents overall are reported first by the business Troubleshooting Incidents / Issues Operations troubleshooting is cumbersome and suboptimal It’s still manual across IT silos It’s difficult to find root cause of incidents quickly Performance issues are difficult to resolve Outages and impact are elongated due to manual efforts and silos Teams are distracted from their core work when they’re troubleshooting Recurring Incidents / Issues The Problem Management process isn’t working because there are many high severity incidents still without root cause determined As a result, Operations is solving the same problems again and again Opportunities exist to improve on incident avoidance since @25%+ of incidents are repeats DESIRED STATE VISION: Complete visibility to environment health & trends across full application stack for all stakeholders Proactively avoid issues before the business is impacted Reduce MTTR with rapid root cause analysis
- 13. Quantifying Value with Splunk Tools Financial Analysis Made Easy • Over 45 Value Calculators • Driven by Actual Customer Results • Complete Financial Analysis • Best Practice TCO Models Don’t Forget • Follow the Impact • Capture All the Value • Summarize and Socialize 13
- 14. Interactive Value Assessment (IVA) Highlights ThepowerofSplunkValueinasimplepackage Target your business case Calculate value seamlessly Be credible Deliver value on the spot! Choose 1 or many Groups 45 Value Calculators Automatically surface those that are relevant Built-in Industry Benchmarks and Customer Case Studies Presentation options of benefit summaries & financial analysis
- 15. Splunk IVA Demo
- 16. ExecuteAgainst a Strategy Take directional, incremental steps • Avoid being reactive – don’t drive by data source • Develop a plan to expand Splunk • Link the plan to strategic company goals • Use Splunk tools and benchmarks to document and quantify the anticipated value • Set baselines for success • Commit to measure value realized post deployment 3 Incremental Steps with a Big Picture Plan
- 17. What Your Splunk Strategy Might Look Like
- 18. Measuring & Tracking Success Helping you take it to the next level • Demonstrating success will help further the cause • Tell the story of your Splunk usage • Compare your success against Splunk customer benchmarks • Assess your usage and staffing maturity • Then bring it all together 4 Measure and Track Your Success Value Realization Usage Maturity Skills Readiness
- 19. Measure Success with Value Realization “Money follows money well spent” • Summarize BEFORE and AFTER Splunk • Capture metrics of improvement • Socialize your success
- 20. Usage Maturity Assessment – IT OPS Drive expansion through highlighting value opportunities 20 Groups % Data Indexed Log Collection Incident Investigation Root Cause Analysis Proactive Alerting Operational Dashboards Business Analytic s Capacity PlanningLevel 1 Triage Level 2 & 3 Escalation Virtualization 0% OS - Unix 25% OS - Windows 0% Storage 33% Network 100% = Splunk fully in use = Splunk partially in use = Splunk not in use
- 21. Usage Maturity Assessments – APP DEV Drive expansion through highlighting value opportunities 21 Top Apps % Indexed Evaluate and Assess Needs Develop and Release Data Collection Business Insight Test Failure Analysis Defect Investigation SAP 0% Warehouse Mgt 0% E-Commerce Website 50% Call Center 80% = Splunk fully in use = Splunk partially in use = Splunk not in use
- 22. Usage Maturity Assessments – SECURITY Drive expansion through highlighting value opportunities 22 Data Sources % Indexed Log Collection Level 1 Triage Monitoring / Alerting Investigations Incident Response Compliance Reporting Routine Log Reviews Threat Intel: (3rd Party) 70% Threat Intel: (OS Blacklist) 70% Network: (Firewall) 90% Network: (IDS/IPS) 90% Endpoint: (PCLM) 80% Access & Identity Mgt 75% = Splunk fully in use = Splunk partially in use = Splunk not in use CurrentlyhandledbyMSSP
- 23. Usage Maturity Assessments – SECURITY CONTROLS Drive expansion through highlighting value opportunities 23 Critical Control In Place? Monitor unauthorized devices or software Monitor unmanaged devices or software Monitor configuration compliance Monitor patch compliance Monitor malware defense Monitor application software security Monitor wireless access control Analyze audit logs with time-based correlation Critical Control In Place? Monitor use of ports, protocols, and services Monitor controlled use of admin privileges Monitor perimeter IDS Monitor controlled / uncontrolled access Monitor orphan, expired, miss use of accounts Monitor potential exfiltration of information Monitor secure IP restriction policies Maintain data going back months = Splunk fully in use = Splunk partially in use = Splunk not in use
- 24. A Real Customer Example - Operations Most common uses of Splunk delivering value Business Service Components % of Data Indexed Log / Data Collection Incident Investigation Root Cause Analysis Proactive Alerting Operational Dashboards Business AnalyticsLevel 1 Triage Level 2 & 3 Escalation Custom Web Apps 80% 3rd Party Web-Apps 100% Apps 75% Web Server 50% Database 100% OS 100% Network 95% = Splunk fully in use = Splunk partially in use = Splunk not in use E-Commerce Site
- 25. Splunk IT Operations Benchmarks Know what toproject and/or compare how you’re doing 25 Reduced Sev1 and Sev2 incidents by 43% Reduced MTTR by 95% and reduced escalations by 50% Improved capacity utilization and avoided $200k in infrastructure 15% to 45% reduction in system incidents 70% to 90% faster investigation of system incidents 67% to 82% reduction in financial impact from outages 5% to 20% optimization with server capacity allocation
- 26. Splunk Application Support/Dev Benchmarks Know what toproject and/or compare how you’re doing 26 15% to 45% reduction in application incidents 70% to 90% faster investigation of QA defects and incidents 10% to 50% faster time to market 10% to 50% increase in value for key projects Went from 1 release/day to 8 because of Splunk Shortened their development cycles by 30% Reduced the number of incidents leading to 9M Euro per year in revenue recaptured
- 27. Splunk Security & Compliance Benchmarks Know what toproject and/or compare how you’re doing 27 70% to 90% improvement with detection and research of events 70% to 90% faster investigation of security incidents 10% to 50% lower risks with data breaches, fraud and IP theft 70% to 90% reduction in compliance labor Reduced investigation effort by more than 75% Reduced the time to report on SAS70 compliance by 83% Reduced the number of security incidents by 80%
- 28. Map Your Progressvs. Benchmarks Estimates based on Value Realization and Usage Maturity 28 Incident Avoidance Incident/Problem Investigation 15% 45%Splunk Benchmark 70% 90%Splunk Benchmark 35% 20% 10% 0% 0% 75% 50% 25% 25% 25% Groups Infrastructure Inventory Manufacturing Payroll Collaboration
- 29. Splunk Staffing Readiness Be sure you have the staff and skills to maximize value 29 A successful and scalable deployment of Splunk relies on the orchestration of key roles and responsibilities, primarily centered around: Architecture Administration User adoption (Power User) Application development
- 30. Basic Communication Framework 30 Architect Admin Works with power users to determine which data sources should be indexed to meet each department’s needs Scales the Splunk architecture to meet business demand Power Users Department Users Adds data sources to the Splunk platform according to business needs Assist power users with the development of advanced dashboards, alerting and reporting Maintains the Splunk SW and it’s infrastructure for optimal performance 1 Power user per department Provides basic support for new and existing reports and dashboards Works with their group to identify opportunities where Splunk can provide value
- 31. Splunk Roles & Recommended Training 31 Splunk Roles Using Splunk Splunk Administration Searching and Reporting Creating Knowledge Objects Advanced Searching & Reporting Developing Apps with Splunk Developing with Splunk SDKs Architect Required Required Optional Optional Optional Optional Optional Admin Required Required Optional Optional Power User Required Required Required Optional Developer Required Optional Required Required Optional Required Optional for Splunk on-premises
- 32. Splunk Power User Status Recommendation: 1power-user pergroup 32 Splunk Power User(s) Using Splunk Splunk Administration Searching and Reporting Creating Knowledge Objects Advanced Searching & Reporting Developing Apps with Splunk Developing with Splunk SDKs • Web • Anurag D. • Security • Josh H. • Infrastructure • Mike G. = Splunk training completed= Required = Optional = Training required but not completed = Optional training not completed Responsibilities • Works with their group to identify opportunities where Splunk can provide value • Collaborates with the Splunk admin(s) to add new data sources to address their requirements • Provides basic support for new and existing reports and dashboards to their group
- 33. Map Your Roles & Highlight Training Gaps 33 Splunk Admin #name Splunk Developer #name Security Power User #name Collaboration Power User #name Database Power User #name CRM Power User #name Network Power User #name Financial Apps Power User #name Splunk Architect #name = Fully Trained = Partially Trained = Not assigned Web Power User #name Server Power User #name Your Company
- 34. Bringing it all together
- 35. Position Value in Expansion Area Taking it to the Next Level Value Opportunity: • faster detection, • faster investigation, • faster root cause analysis of application incidents • fewer developer escalation After 3 to 6 months After 3 to 6 months Document Success for Server & Network teams Document Success for App & DB teams Position Value in Expansion Area Application Development Value Opportunity: • faster test analysis, • faster investigation of pre- production bugs, • faster releases cycles Position Value in Expansion Area Security & Compliance Value Opportunity: • faster detection, faster triage, • faster investigation of security incidents Value Realized: • faster detection, • faster investigation, • faster root cause analysis of system incidents IT Operations Application Support
- 36. Success from Current Use PositiveROIachievedon~$1.7Mspendtodate Proactively monitoring a $1.5B revenue platform entirely with Splunk. Reducing manual effort and impact Avoiding revenue displacement and loss “We almost had an outage today. We saw some things in Splunk. That saved us a 1.5 hour incident and almost $300,000.” Opportunities: Get full stack of data in for additional efficiencies (network, VM, storage, DB) Web Team 42% reduction in business impact Avoiding revenue loss of $2.3M/year Value $2.5M/year | 2,445 hours/year Rapid search and investigation of security incidents. Went from reactive to proactive. Reducing manual effort, impact and risk Innovating – search to alert to IDS “If we didn’t have Splunk, I am not sure what we would have done with the April incident.” Opportunities: Apply to PCI readiness saving GRC team effort, enabling continuous compliance. 50% reduction in incident investigation Avoiding 16k+ hours/year Value $1.3M/year | 16,380 hours/year Security 20,414 Yearly Hours 50% reduction in incident investigation (when leveraged) Value $124,102/yr* | 1,589 hours/yr* Infrastructure Resolving complex issues rapidly; opportunity for even more value. Reducing manual effort and impact Realizing only partial benefits today “When there’s a problem, it’s tricky to figure out where it is. Splunk’s a helpful tool to have.” Opportunities: Get full environment data in. Use more consistently across team to capture value. $3.92M Yearly Value See detailed calculations of value, usage adoption, and staffing maturity schedules in the Appendix. Benchmarks Used for Infrastructure Calcs From a real Splunk customer
- 37. Functional Adoption Summary Comparing[customer]’s currentusageagainstthemostcommonSplunkusesdrivingvalue IT & APPLICATION OPERATIONS % Usable Data Indexed Log Collection Incident Investigation Root Cause Analysis Proactive Alerting Operational Dashboards Business Analytics Capacity PlanningLevel 1 Triage Level 2 & 3 Escalation Web Team 75% NW*, VM, DB, Storage Infrastructure 20% DB, VM, Windows, Storage = Splunk fully in use = Splunk partially being used = Splunk not being used SECURITY & COMPLIANCE % Data Indexed Log Collection Level 1 Triage Monitoring / Alerting Investigations Incident Response Compliance Reporting Routine Log Reviews Security 80% 3rd party intel, AIM MSSP Refer to adoption charts for each team in the Appendix for more details From a real Splunk customer
- 38. NOTE: VMware data not ingested. Storage visibility is limited to VM instance. Host and SAN would be beneficial. * Network data is being collected today but in a separate Splunk instance due to be joined later this year. Functional Adoption – Web Team .Com Business Service % Data Indexed Log Collection Incident Investigation Root Cause Analysis Proactive Alerting Operational Dashboards Business Analytics Capacity Plannin gLevel 1 Triage Level 2 & 3 Escalation Web/App Server 100% Database 0% Virtualization 10% OS 100% Storage 20% Network* 90% = Splunk fully in use = Splunk partially being used = Splunk not being used From a real Splunk customer
- 39. Functional Adoption – Security Controls 39 Critical Control In Place? Monitor unauthorized devices or software Monitor unmanaged devices or software Monitor configuration compliance Monitor patch compliance Monitor malware defense Monitor application software security Monitor wireless access control Analyze audit logs with time-based correlation Critical Control In Place? Monitor use of ports, protocols, and services Monitor controlled use of admin privileges Monitor perimeter IDS Monitor controlled / uncontrolled access Monitor orphan, expired, miss use of accounts Monitor potential exfiltration of information Monitor secure IP restriction policies Maintain data going back months = Splunk fully in use = Splunk partially in use = Splunk not in use Current assessment of Splunk usage at [customer] for the SANS 20 security controls. From a real Splunk customer
- 40. [customer]’s Splunk Team 40 = Fully Trained Splunk Architect #name Splunk Admin #name Splunk Developer #name Security Power User #name Collaboration Power User #name Labor Power User #name Mobile CRM Power User #name Infrastructure Power User #name GSIT Power User #name Splunk Architect #name = Partially Trained = Not assigned Splunk Admin #name Web/Mobile Power User #name Warehouse Power User #name From a real Splunk customer
- 41. In a matter of hours, Splunk lets us build dashboards to compare and correlate whatever we want— nothing else lets us do that. Real-time Operational Intelligence 41 Splunk reduced outage frequency 15%, delivering an annual ROI of $1.3M Drives capacity and maintenance window planning Delivered executive dashboards showing activations by minute, by channel, by market, by device type in hours, not weeks or months “ “ Ty Prikkhi Senior Operations Manager
- 42. Splunk Application Support/Dev Benchmarks Know what toproject and/or compare how you’re doing 42 15% to 45% reduction in application incidents 70% to 90% faster investigation of QA defects and incidents 10% to 50% faster time to market 10% to 50% increase in value for key projects Went from 1 release/day to 8 because of Splunk Shortened their development cycles by 30% Reduced the number of incidents leading to 9M Euro per year in revenue recaptured
- 43. Future Value Opportunities(1of2) AProactiveOperations approachwillreduceimpacthours Collaboration toavoid171,348employeehours/year Basic monitoring puts Collaboration at risk as it grows from ~6k to 200k+ users and becomes the portal to key apps Proactively monitor to avoid incidents and employee productivity loss (171k hrs) Speed incident investigation and resolution, reducing manual effort “We expect 20% more issues as we go from @6,000 to 200,000+ users.” Incidents reduced by 25% | Impact 67% Avoiding 34 hours/year of BII time Value $5.2M/year | 1,501 IT hours/year Collaboration Shift from reactive to proactive improving Labor stability and availability enabling maximum scheduling efficiency Proactively monitor to avoid incidents and protect Partner productivity Speed incident investigation and resolution, reducing manual effort “Last Tuesday if we got a heads up from Splunk we could have resolved it in 1 hour instead of 5.” 70% reduction in incident investigation Sev1 time reduced 96 hours/year Value $433,544/year | 5,549 hours/year Labor Scheduling Become more proactive further leveraging centralized, real-time data to avoid and reduce impact time Proactively monitor to avoid incidents and business impact Further reduce investigation effort over current, isolated log search solution “If we had a dashboard showing us the app, database, server, and network health, we could get ahead of potential issues and resolve them before impact.” 25% reduction in incidents Avoiding 12 hours/year impact time Value $1.0M/year | 828 hours/year Warehouse 19,725 Yearly Hours $7.5M Yearly Value From a real Splunk customer
- 44. Best Practices for Documenting & Positioning Value Taking your Splunk deployment to the next level 4 Measure and Track Your Success 1 Align with Key Business Objectives Qualify and Quantify Business Value 2 3 Incremental Steps with a Big Picture Plan
- 45. Ask Me or Your Account Team For… • The Interactive Value Assessment (IVA) model • Usage adoption maturity templates • Splunk staff readiness templates • Splunk common benefits and benchmarks
- 46. Copyright © 2014 Splunk Inc. Questions? Thank you! Doug May dmay@splunk.com
Editor's Notes
- Your process requires it 85% of investments over 50,000 USD require a formal business case (IDC) Create or maintain visibility to Splunk’s strategic importance Prioritize Splunk investment over other projects Facilitate continued support and resources (FTE, maintenance, etc) Ease approval of future resource requests People, infrastructure, Splunk license, professional services Supporting renewals; staff departures Eliminate any doubt of Splunk’s value to your organization Help Other Succeed in your organization If they understand what you’ve done and what value you’ve received, they can do the same thing Promote yourself or your team Show your success to help promote your people and your own accomplishments
- You all know what a great platform Splunk is. So if it’s so great, why does our team exist? Well…Users love Splunk and clearly understand the value it delivers to them operationally, but they struggle with articulating it to their senior management in business terms. This leaves executives asking what THEY get from Splunk. They understand their people love it, but can’t put dollars, euros, yuan, or yen on it easily. The Value that Splunk brings to the business is a hidden gem for most executives. When they are able to understand the business value it delivers for them, in most cases it’s priceless.
- Problem: With over 5 million subscribers and annual growth rates of 10-20%, Cricket Communications has rapidly become a leading US-based provider of “no signed contracts, no limits” mobile cellular phone services (including voice, text, broadband and data). Cricket Communications regularly handles 3,000 new subscriber requests per hour – about 50 activations per minute. In order to keep up with this tremendous demand for its trademark services, Cricket Communications automated its order processing system and workflows, eliminating manually introduced errors. However, the carrier soon discovered that all its complex applications and systems needed to work without fail in order to keep the flow of orders going. If any part or subsystem experienced a failure or degradation, the whole system would quickly come to a grinding halt. Solution: Cricket Communications deployed Splunk to quickly detect and analyze system performance issues. Using proactive triggers to send alerts from Splunk, the carrier has been able to address problems before they escalate to their event management team. The Applications Operations team calculated that with Splunk in place, they have reduced outage frequency by about 15%, translating into an annual positive revenue protection impact of $1,200,000. The team also gained new operational efficiencies using Splunk and as a result was able to reassign one Full Time Employee (an approximate savings of $100,000) to other tasks. Lastly, by loading log data into Splunk and creating relevant executive dashboards, the company was also able to start looking at business trends on activations, cancellations and other critical business metrics. Benefits: In addition to tremendous cost-savings, Cricket saw the following benefits with Splunk: ROI – Cricket reported an annual ROI savings of $1,300,000 by using Splunk Application Monitoring – Helps prevent downtime and ensure rapid account activation Vendor Management – Rapid recognition and understanding of where problems lie – with Cricket Communications’ applications or those provided by third parties Operational Intelligence – Direct visibility into business transactions and subscriber selections