4. [T]here’s going to
be more change in
the next five to ten
years than there’s
been in the last 50.”
Mary Barra, CEO, GM
Source: Nov. 2015 interview with Matthew DeBord,
Business Insider. 4
“
5. “In order to grow Citi, we first have to grow our own
perspective, skills and capabilities… Our curiosity, our
openness to learning and trying new things, our ability
to adjust and adapt quickly and our willingness to fail
fast and fail small are the essence of a culture that
innovates and exposes new value to our clients in real
time.”
Stephen Bird, CEO Citi Global Consumer Group
Source: “A Transformation Journey,” Brad Miller, 2016.
5
10. Deliver value, reliably with small batches
10
Sources: “Good Software is a Series of Little Failures,” Coté, April 2016; The Lean Startup, Eric Ries, 2011. The Lean Enterprise, Barry
O'Reilly, Jez Humble, and Joanne Molesky. See also overview of this approach at the IRS from Dec 2015. "Application Modernization,
Service By Microservice," Kurt Bittner and Randy Heffner, Forrester, Dec 2015; "Best Practices For Agile-Plus- Architecture," Randy
Heffner, Forrester February, 2015.
11. From 37% availability to $440m in back-taxes
• Only 37% of calls answered,
shrinking budgets
• From 2 year to 9 week
releases
• 2m+ users paid $440m in
taxes
11
Sources: “‘Your IRS Wait Time is 3 Hours’ - Is Lean Possible in Government?”, Emily Price, Pivotal, April 2017; “Agile
Transformation is Product Management,” podcast, Oct 2017; “Minimum Viable Taxes: Lessons learned building an MVP inside
the IRS,” slides , Andrea Schneider & Lauren Gilchrist, 2015.
Before After
12. From white-board to ~20 features a week & ~$1m/day in
savings
• Avg. 4 years for idea to
delivery
• First version in 120 days
• Break-even in 7 days
• Saving ~$1m/day in fuel,
reduced staff by 80%
• Now adding ~20 features a
week
• Created intellectual assets
that have seeded 8 more
projects
12
Sources: “The Pentagon has tried to get Silicon Valley on its side for years. Now it’s part of the air war against ISIS," Dan Lamothe, The
Washington Post, July, 2017; “DoD and USAF moving at startup speeds,” Lt Col Enrique Oti, Defense Innovation Board, July 2017.
Before After
6 people, 4-6 hours 2 people, minutes
13. 13
Source: “Large-Scale Enterprise Platform Transformation with Microservices, DevOps, and PaaS,” Vipul SavJani,
Christopher Tretina, August 2016.
14. More functionality & resiliency for mission-critical, customer
facing services means the business is
• 3 year journey
• Executive buy-in to improve
resiliency, scaling, and time-
to-market
• Targeting high-value apps,
e.g., tech roll, inbound
customer service, xfinity
screens
• Small batch, agile approach
• Includes business from the
start
• Delivers value quickly to build
trust & momentum 14Source: “Comcast Cloud Foundry Journey,” Greg Otto, Comcast, June 2013.
15. "DevOps is not about what you do, [it's about]
outcomes"
40%+ productivity/cost,
rebooted member facing app
3+ week to 3 days,
50% reduction in
incidents
A white-board to ~20
features a week, in 120
days
Sources: Gene Kim quoted in Start and Scaling DevOps in the Enterprise; Crafting your cloud-native strategy, 2017, Coté; FedEx #SpringOne 2017 talk, Dec, 2017; Air
Force story, Washington Post, July 2019; “Agile Transformation is Product Management,” Oct 2017; Mojgan Lefebvre, Liberty Mutual, June, 2017.
Delivered 3x
features
year/year
40% policy strike rate,
vs. 20% industry
average
From 37%
availability to $440m
in back-taxes
17. Cover w/ Image
One tire at a time
• Development – actually do agile
• Management – the start & end of problems
• Organization & teams – user-centric,
balanced
• Platforms – automation, day 2, & automation
• Compliance & security – removing chaos
• Scaling tactics – spreading to 100’s of teams
17
18. Agile & DevOps
Better software makes for better
business, but we think about
software wrong
18
20. Agile Adoption Still Leaves One Remaining BarrierAgile Adoption Still Leaves One Remaining Barrier
OPERATIONS
Releas
e
Test
Burndown Chart
Sprint
backlog
Product
backlog
BUSINESS DEVELOPMENT
Sprint
Release
One Day
Less than a
30 day sprint
Daily Scrum
meeting
backlog
weeks
Sprint
Review
21. DevOps: Culture, Automation, Lean, Measurement,
Sharing
Source: “2017 DevOps Report,” Puppet, DORA, et.al., July, 2017.
Be careful to look beyond just automation.
Keep the bigger focus on the entire software lifecycle.
Transformational
Leadership
Personal Recognition
Supportive Leadership
Intellectual Stimulation
Inspirational Communication
Vision
Test and Deployment
Automation
Continuous Integration
Trunk-based Development
Shifting Left on Security
Loosely Coupled Architecture
Empowered Teams
Lean Product
Management
Team Experimentation
Working in small
batches
Gathering and
Implementing Customer
Feedback
Continuous
Delivery
IT
Performance
Deployment
Plan
Organizational
Performance
Non-commercial
Performance
( – )
2016
22. Eliminate big, upfront analysis by using frequent
feedback
22
Sources: "How the US Air Force Made Its ISR Network Cheaper to Run and Easier to Upgrade," M. Wes Haga, Oct, 2017; “Air Force
Intelligence Unit Goes Agile,” Charles Babcock, Information Week, June, 2017; “Limit upfront analysis by including frequent, real-world
feedback from users,” Coté, Nov 2017.
With a more agile approach, we pick a place to start and
get to a point where you can have an intelligent
conversation… a point where the requirements are 80%
done and the application is good enough.
A [waterfall] mistake could cost $100 million, likely
ending the career of anyone associated with that
decision. A smaller mistake is less often a career-ender
and thus encourages smart and informed risk-taking.”
““
- M. Wes Haga, US Air Force
24. Cover w/ Image
Management’s own
changes• Driving strategy from the top
• Creating & championing teams
• Facilities changes, policy changes
• Policing/partnering new technologies
• Applying small batch to the org. itself
• Fighting the corporate change Eeyores
24
“In order to get people to
scale, they have to understand
how to connect the dots. They
have to see it themselves in
what they do - whether it’s
developing software, or
protecting and securing the
network, or provisioning
infrastructure - they have to
see how the work they do
every day connects back to
enabling the business to either
be productive, or generate
revenue.”
- Niki Allen, Boeing
25. A strategy for change at Boeing
25Source: “Leading the Digital Transformation,” Niki Allen, Boeing, Dec 2017.
27. Functional organizations are a poor fit
• Optimized for cost &
repeatability
• Requires coordination
overhead
• Locally optimized
• Elusive responsibly for final
outcome
27
ENTERPRISE
ARCH CSO INFRA
MIDDLEWARE
&
APPDEV
DATA
Ent Arch
Proj Mgmt
Biz An
Info Sec Srv Build
Cap Plan
Network
Ops
Mid. Eng.
Client SW Dev
Svc Govern
SW Arch
SW Dev
Data Arch
DBA
Change Control
Source: “DevOps Who Does What,” Cornelia Davis, June 2017.
28. The organization supports the agile teams
28
Ent Arch Portfolio Mgmt
Info Sec
Service Engineering
Capacity PlanningNetwork management
Ops/SREMiddleware Engineering
SW Arch
SW Dev
Client SW Dev
Service Governance
Ops
Cap Plan
SW Arch
SW Dev
Client SW Dev
CUSTOMER FACING APP TEAM
Ops
Cap Plan
Biz An
Prod MgmtData Arch
DBA
Biz An
Prod MgmtData Arch
SW Arch
SW Dev
Client SW Dev
LEGACY SERVICE TEAM
Ops
Cap Plan
Biz An
Prod MgmtData Arch
ENABLEMENT
Change Control
CUSTOMER FACING APP TEAM
PLATFORM TEAM
Source: “DevOps Who Does What,” Cornelia Davis, June 2017.
30. From coding 20% of the time coding to coding 90% of the
time
An agile methodology, proven
over 25+ years:
• Balanced teams w/all roles
needed, dedicated to the
product
• Paired programming, &
beyond
• Test-driven Development
• Short iterations
• Continuous Integration &
Continuous Delivery
30Source: “Don’t Forget People and Process in Your Digital Transformation,” Allstate case study, March, 2017.
33. “But what about my job?”
•Enterprise architects
•PMO
•QA
•Operations
•Design
•Business analysts
33
• How do they deliver
value to production?
• Slim matrixing to
enable teams
• Maybe you’re not
helping
• …perhaps join the
team!
34. 34
“If I’m doing 8 or 15 releases a week, how am I going to get
through all those CABs?”
Mark Ardito, HCSC
35. The EA asset embodies the history
of the company and all of its
learnings to date. It is how the
company competes and how it
manages its daily operations.”
A Seat at the Table, Mark Schwartz
Source: “A Transformation Journey,” Brad Miller, 2016.
35
“
36. A real cloud
platform
Release management in the
bottleneck
36
Source: “The Need For Speed: Drive Velocity And Quality With DevOps,” Robert Stroud &
Eveline Oehrlich, Forrester, Geb 2017.
37. A fully automated build pipeline & cloud platform
37Sources: “Speed Thrills: How to Harness the Power of CI/CD for Your Development Team,” Ben Kamysz &Jared Ruckle, Pivotal, Aug 2017.
(<= 5 days)
38. Standardize on a platform
38
Source: “The Upside-Down Economics of Building Your Own Platform,” Jared Ruckle and Matt Walburn, 2017. Also, “DevOps Who Does
What,” Cornelia Davis, June, 2017; “How Platforms Work,” Casey West, August, 2016.
39. Unclogging a 15 month case backlog with a new
platform
39
Before Pivotal Cloud Foundry With Pivotal Cloud Foundry
Step Task Time Step Task Time
1 Application Intake Form Submitted 10 minutes 1 Application Intake Form Submitted 10 minutes
2 Setup GitHub Repository <1 minute 2 Setup GitHub Repository <1 minute
3 Register FQDN's ~1 day 3 Register FQDN's <1 minute
4 Request SSL Cert ~1 week Request SSL Cert <1 minute
5 Request ICAM Cert ~1 week Request ICAM Cert ~1 week
6 Provisioning Production Servers ~3 weeks Provisioning Production Servers <1 minute
7
Install certs on test & stage load
balancers ~3 days
Install certs on test & stage load
balancers <1 minute
8 Secrets vault for Test/Stage/Prod ~3 days Secrets valut for Test/Stage/Prod <1 minute
9 DB password vault for Test & Stage ~ 3 days DB password vault for Test & Stage <1 minute
10 Create encrypted data bags ~3 days Create encrypted data bags <1 minute
11 Write Jenkins Job ~2 days 4 Write Jenkins Job ~2 days
12 Write Chef deployment script ~2 days 5 Test CI & deployment 2 days
13 Test CI & deployment ~2 days
Total ~60 days Total ~4 days
Source: Pivotal government customer.
40. Cover w/ Image
Platforms: “A lot of effort went
into making this effortless.”
•Canary deployment
•A/B testing
•Container orchestration
•Health checks & remediation
•Scale out/scale in
•Across multiple IaaS
•Day 2 operations
•Integrated middleware
40Image from Dan Pope, 2009.
41. Embedded
OS
(Windows & Linux)
NSX-T
CPI (15 methods)
v1
v2
v3
...
CVEs
Product Updates
Java | .NET | NodeJS
Pivotal Application
Service (PAS)
Application Code &
Frameworks
Buildpacks | Spring Boot | Spring Cloud |
Steeltoe
Elastic | Packaged Software | Spark
Pivotal Container
Service (PKS)
>cf push >kubectl run
YOU build the containerWE build the container
vSphere
Azure &
Azure StackGoogle CloudAWSOpenstack
Pivotal
Network
“3Rs”
Github
Concours
e
Concours
e
Pivotal Services
Marketplace
Pivotal and
Partner Products
Continuous
delivery
Public Cloud
Services
Customer
Managed
Services
Repair
— CVEs
Repave Rotate
— Credhub
43. “You can type anything you want in a Word
document!” Mark Ardito, HCSC
44
44. Cover w/ Image
Small Batches Reduce Risk
& Increase Controls
Five risks that small batches reduce
1. Bug swarms – less software leads to faster
fixing.
2. Useless software – don’t wait 6+ months to find
out, find out every 24 hours.
3. Stymied Innovation – daily opportunities to
learn and innovate.
4. Budget overruns – check in daily on ROI,
projections. Shut it down early and re-try if
needed.
5. Schedule elongation – if you ship every day,
guaranteed, how can you ever be late?
45
Source: "Dear Developers, Small Batch Releases Are Your Friend," Coté, Feb 2016, also in The Register. For more details, see
John Field’s “Pivotal Cloud Foundry: the Auditor’s Guide,” Dec, 2017.
"There have obviously been culture
shocks. What is more interesting
though is that the teams that tend to
have the worst culture shock are not
those typical teams that you might think
of, audit or compliance. In fact, if
you’re able to successfully
communicate to them what you’re
doing, DevOps and all of the
associated practices seem like
common sense. [Auditors] say, 'Why
weren’t we doing this before?’”
"DevOps Enterprise Adoption at E*TRADE,”
InfoQ/DOES, Jan 2016.
45. Security – reduce risk by going faster & automating, & acting
•Repair
•Repave
•Rotate
•Authentication/UAA
•RBAC
•BOSH enforcement
•Encryption, data, network
•Isolation segments
•Zero-trust model
•Trusted auditing a-plenty
•Distributed tracing &
microservices ops
46
Source: “Security & Compliance with Pivotal Cloud Foundry,” Ben Bertka, Sujit Mohanty, and Jared Ruckle, Oct, 2017. “The Three Rs of
Enterprise Security: Rotate, Repave, and Repair,” Justin Smith, April 2016; “Understanding Cloud Foundry Security,” Pivotal docs;
47. Starting: “pilot low-risk apps, and ramp-up.”
48
Sources: Home Depot meetup, Oct 2015; Humana at CF Summit 2015; EU payday loan company; Pivotal Labs on large auto company;
“Getting started,” Coté, Oct 2016; Comcast’s Christopher Tretina at SP1 2016; “Cloud-Native at Home Depot, With Tony McCulley,”
Number of AI’s equates to ~130 apps composed on ~900 services.
48. Managing the change: pace yourself
• Liberty Mutual 10 (simple) apps in 10 weeks
• Allstate 16 apps in a year
• THD ~130 apps in a year
• Auto manufacture ~115 after two years
• BUT! If you don’t start, you’ll suffer analysis paralysis
49
Sources: “Cloud-Native at Home Depot, With Tony McCulley’; “Don’t Forget People and Process in Your Digital Transformation,” The New Stack, March,
2017; Pivotal customer analysis, cases, and conferences.
52. 53
“We are uncovering better
ways of developing software
by doing it and helping others
do it.”
- The Agile Manifesto, 2001
Thanks!@cote | cote@pivotal.io