Keeping an application running at scale can be a daunting task. When do you need to add more capacity? Larger databases? Additional servers? These questions get harder as the complexity of your application grows. Microservice based architectures and cloud-based dynamic infrastructures are technologies that help you keep your application running with high availability, even during times of extreme scaling. We will discuss some of the best practices we’ve learned working with New Relic customers on how you can manage your applications running at scale, and how technologies such as microservices and dynamic infrastructure can help you with this challenge. This session is brought to you by AWS Summit San Francisco Platinum Sponsor New Relic.
Report
Share
Report
Share
1 of 102
Download to read offline
More Related Content
ENT310 Microservices? Dynamic Infrastructure? - Adventures in Keeping Your Application Running at Scale
2. Who am I?
30 years in industry
5 in New Relic
(Architect Lead, Cloud, Service Migration)
7 in Amazon Retail & AWS
(Built First AppStore, AWS Elastic Beanstalk)
Who Specialize in:
Cloud computing
Services & Microservices
Scalability, Availability
leeatchison@leeatchison
Senior Director Strategic Architecture
9. 9
Keeping Your App Running…At Scale
Availability…
…is more than
you think it is.
10. Does this sound like something you’ve heard recently…
…overheard OPs conversation...
11. The conversation…
“We were wondering how changing a
setting on
our MySQL database might impact
our performance…
12. The conversation…
“We were wondering how changing a
setting on
our MySQL database might impact
our performance…
… but we were worried
that the change may
cause our production
database to fail…”
13. The “scary” overheard conversation…
“… Since we didn’t want to
bring down production,
we decided to make the
change to our backup (replica)
database instead…
Under
Construction
… but we were worried
that the change may
cause our production
database to fail…”
14. The “scary” overheard conversation…
“… Since we didn’t want to
bring down production,
we decided to make the
change to our backup (replica, hot
standby) database instead…
… After all, it wasn’t
being used for anything
at the moment.”
Under
Construction
15. The “scary” overheard conversation…
Until, of course, the backup was
needed…
Under
Construction
X
16. The “ scary” overheard conversation…
Until, of course, the backup was
needed…
This is a true story
Under
Construction
!!!!X
X
25. Need Data at Every Level
Amazon EC2 Instance
BrowserMobile
Server (Virtual)
Hardware
Server OS
Application &
Application
Microservices
Typical Server / Amazon EC2
Instance
• Application & Application
Microservices
• Server OS
• Hardware (virtual)
26. Amazon EC2 Instance
BrowserMobile
Server (Virtual)
Hardware
Server OS
Application &
Application
Microservices
Low Level Monitoring
Amazon
CloudWatch
AWS
CONSOLE
Amazon CloudWatch
Monitors
• Amazon EC2 instance
• Virtualization
• Hardware
• [CPU / Disk / Networking]
Doesn’t Monitor:
• Server OS
• Memory / Filesystem
• Processes
• Configuration
• Application
- Latency
- Error rates
27. Amazon EC2 Instance
BrowserMobile
Server (Virtual)
Hardware
Server OS
Application &
Application
Microservices
DASHBOARDS
Infrastructure / Application Monitoring
New Relic
Application
Monitoring
New Relic
Infrastructure
Monitoring
Amazon
CloudWatch
AWS
CONSOLE
Monitors (Server):
• How O.S. is performing
• Configuration Changes
• Processes
• Hardware
Monitors (Application):
• App health
• App performance
• Microservices
Doesn’t Monitor:
• Virtualization
28. Amazon EC2 Instance
BrowserMobile
Server (Virtual)
Hardware
Server OS
Application &
Application
Microservices
Full Stack Monitoring
New Relic
Application
Monitoring
New Relic
Infrastructure
Monitoring
Amazon
CloudWatch
AWS
CONSOLE
Integrations
New Relic
Monitors
CloudWatch
monitors
DASHBOARDS
AWS / Amazon CloudWatch
• Visibility into virtualization
• CPU / Disk / Networking
• 14 AWS Services
APM
• CPU / Disk / Networking
• Memory / Filesystem
• Processes
- Infrastructure components
- Configuration inventory
• Application / Microservices:
- Latency
- Error rates
- App insights
35. Cloud as a “Better Data Center”
Resources are allocated to
uses, just like in a data
center
Provisioning process
is faster
Lifetime of components is
relatively long
Capacity planning is
still important and
still applies
36. Why use a “Better Data Center”?
Add new Capacity
(faster)
Improve Application Availability
(redundancy)
Compliance
38. Cloud as a “Dynamic Tool for Dynamic Apps”
Use Only the Resources
you need
Allocate / de-allocate
resources on the fly
Resource allocation is an
integral part of your
application architecture
39. Dynamic Cloud
Resources are: Application in charge:
Allocated Application is aware of and is controlling
traditional OPs resources
Consumed De-allocated
41. Dynamic Usage Example…
Docker Container Age
(by Minute and Hour)
1,200,000
11% underone minute
Container age (minutes)
42. Dynamic Cloud Technologies
Dynamic Cloud is about scaling and availability
Amazon EC2 Auto Scaling
Mobile / IoT Dynamic routing
Load balancing
Queues and notifications
Docker
43. Dynamic Cloud Enables Better Applications Faster
Traditional Data Center Cloud Data Center Dynamic Cloud
Good Better Best
The way you’ve done things in the past
won’t work in the future.
44. Dynamic Cloud
Server running application/
processes
Process running
a command
Function performing a
task or operation
Amazon EC2 Docker AWS Lambda
Things happen faster because of…
45. Microcomputing & AWS Lambda
• Highly dynamic
• Incredibly scalable
• No infrastructure to provision
• Massively shared infrastructure
Also known as:
• Functions as a Service (FaaS)
• Compute as a Service (CaaS)
• Serverless
47. Amazon S3
Bucket
Amazon
DynamoDB
Amazon API
Gateway
Amazon SQS
RESOURCESSOME
Amazon S3
Bucket
Amazon API
Gateway Amazon SQS
RESOURCESSOME
AWS Lambda
Script
AWS Lambda
Instances
• Takes an event from an AWS
resource (A Trigger)
• Creates an instance
to execute
AWS Lambda
48. Amazon S3
Bucket
Amazon
DynamoDB
Amazon API
Gateway
Amazon SQS
RESOURCESSOME
Amazon S3
Bucket
Amazon API
Gateway Amazon SQS
RESOURCESSOME
AWS Lambda
Script
AWS Lambda
Instances
• Takes an event from an AWS
resource (A Trigger)
• Creates an instance
to execute
• Can impact original or different
AWS Resource
AWS Lambda
49. Amazon S3
Bucket
Amazon
DynamoDB
Amazon API
Gateway
Amazon SQS
RESOURCESSOME
Amazon S3
Bucket
Amazon API
Gateway Amazon SQS
RESOURCESSOME
AWS Lambda
Script
AWS Lambda
Instances
• Takes an event from an AWS
resource (A Trigger)
• Creates an instance
to execute
• Can impact original or different
AWS Resource
• Any number of instances can
run at a time
AWS Lambda
51. Dynamic Cloud has unique monitoring requirements…
How do I track what the dynamic cloud is
doing for me (or to me)?
52. What is a Dynamic Cloud Application?
• Application & Application Microservices
Responsible for the parts you care about
• Infrastructure
• Allocation/Provisioning
• Scaling
Let cloud manage rest
Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Provisioning
Application &
Application
Microservices
Application &
Application
Microservices
BrowserMobile
53. Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Provisioning
Application &
Application
Microservices
Application &
Application
Microservices
BrowserMobile
Monitoring Dynamic Cloud Applications
AWS
CONSOLE
Amazon
CloudWatch
54. Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Provisioning
Application &
Application
Microservices
Application &
Application
Microservices
BrowserMobile
AWS InfrastructureApplication Performance
Amazon
CloudWatch
AWS
CONSOLE
New Relic
Application
Monitoring
New Relic
Infrastructure
Monitoring
DASHBOARDS
Integrations
55. Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Provisioning
Application &
Application
Microservices
Application &
Application
Microservices
BrowserMobile
Amazon
CloudWatch
AWS
CONSOLE
New Relic
Application
Monitoring
New Relic
Infrastructure
Monitoring
DASHBOARDS
New Relic
Monitors
CloudWatch &
AWS monitors
Integrations
AWS InfrastructureApplication Performance
56. Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Provisioning
Application &
Application
Microservices
Application &
Application
Microservices
BrowserMobile
How do you monitor this?
?How do you
monitor this?
57. Where did it go? It was just here!!
The thing you monitored 10 minutes ago…
...doesn’t exist anymore!?
58. Monitoring the Dynamic Cloud
Monitor the Cloud Components
themselves
Monitor the lifecycle of the Cloud
Components
Very different than monitoring traditional Data Center components
61. Changing World
Dev
Now - DYNAMIC World
Ops
• We know:
• Change is inevitable
• We must:
• Embrace and drive change
• Enabling:
• Quicker growth
• More reliable growth
62. 62
Keeping Your App Running…At Scale
Dynamic
Cloud…
...make availability
happen.
Migration…
...how do I get my app
to the cloud?
64. Ops
Use the Cloud
• Move in a controlled way
• Learn as you go
• Measure everything
Does not have to be painful…
65. Experiment
Secure the Cloud
Enable Servers, Enable SaaS
Enable Value-Added Services
Enable Unique Services
Mandate Cloud Usage
Progressions in Cloud Adoption…
The Controlled Way
Standard steps
most companies
follow
67. Enterprise IT Cloud Adoption Strategy
Experiment
Non-evasive, safe technologies
- Amazon S3
- Perhaps: Amazon CloudFront, Amazon SQS,
Amazon SES
▪ Stay away from Amazon EC2/Servers
Security: Easy as one-offs
No “Policies” implemented yet
“Just seeing what this is all about”
Progressions in Cloud
Adoption
What is this cloud thing?
69. Progressions in Cloud
Adoption
Enterprise IT Cloud Adoption Strategy
Secure the Cloud
AWS Identity & Access Management (IAM) (Credentials)
VPC (Secure network)
AWS Direct Connect (just another data center)
Cloud policies begin to be formed
All parts of the company are now involved
Critical evolution point
Can we trust the cloud?
71. Progressions in Cloud
Adoption
Enterprise IT Cloud Adoption Strategy
Enable Servers, Enable SaaS
Amazon EC2
- Basic “data center migration”
- Just another server type available…
Multiple AZs/Regions
- Part of multi-datacenter resiliency strategy
Independently: SaaS usage increases
- Non-critical or internal uses first
The cloud seems to work pretty well…
77. Progressions in Cloud
Adoption
Enterprise IT Cloud Adoption Strategy
Mandate Cloud Usage
Cloud as a data center replacement
Company is now “all in” with cloud
Netflix…
Why do we need our own data centers?
78. What is the cloud?
Can we trust the cloud?
The cloud works pretty well…
Dynamic Cloud becomes a thing…
Dynamic Cloud is deeply ingrained…
Why do we need our own data centers?
Progressions in Cloud AdoptionThe steps aren’t easy…
79. Experiment
Secure the Cloud
Enable Servers, Enable SaaS
Enable Value-Added Services
Enable Unique Services
Mandate Cloud Usage
Progressions in Cloud Adoption
Different Companies
Different Speed
Different Needs
86. Adoption Success Strategies
Understand
where your
culture is
Consciously plan
your acceptance
Drive your cultural
change to your
desired level
Monitor
your adoption
Understand
your needs
87. Monitor Your Adoption
Before Migration
Baseline application
(servers, databases,
caches, applications,
microservices)
Determine your steady
state
88. Monitor Your Adoption
During Migration
Incorporate cloud’s
internal monitoring
Continue
application
monitoring
Understand and solve all deviations from steady state…
89. The Biggest Role Monitoring Plays In Migration
Performance Post Migration
& During Optimization
Pre-migration Feasibility & Benchmarking
90. Continue Monitoring…
Infrastructure is
now out of your
control
Some cloud
specific concerns (Amazon
EC2 instance failures,
instance degradation)
Dynamic Technologies
Impact Our Applications
Understand
application
impact
Ongoing
application &
infrastructure
monitoring is
essential
Monitor Your Adoption
91. 919191919191
Fairfax Media Limited is a leading multi platform media
company in Australasia, reaching 10.6 million
Australians and 2.9 million New Zealanders.
Media/Entertainment
“Because we monitored our on-premises systems with New Relic
before we migrated them to Amazon Web Services, we were
able to identify potential issues and fix them during the
migration process.”
- Cheesun Choong
Head of Product Platforms
Results
Reduced
diagnosis time
from hours to
minutes
Migrated to AWS
with confidence
Identified
underutilized servers
to save money
92. 92
Keeping Your App Running…At Scale
Dynamic
Cloud…
...make availability
happen.
Migration…
...how do I get my app
to the cloud?
Availability…
…is more than
you think it is.
Monitor your application and infrastructure
93. Monitoring just the server
EC2 Instance
Server OS
Server (Virtual)
Hardware
Application &
Application Microservices
AWS
CONSOLE
Worked when rate of change was low…
Amazon
CloudWatch
95. Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Provisioning
Application &
Application
Microservices
Application &
Application
Microservices
BrowserMobile
Full Stack Monitoring
New Relic
Application
Monitoring
New Relic
Infrastructure
Monitoring
DASHBOARDS
• Top to bottom monitoring…
• Full stack accountability...
• Dynamic infrastructure control...
You need:
Amazon
CloudWatch
96. Digital Fan Experience for Major League Baseball
New Relic empowers our developers
to experiment and work fast without
compromising on the quality of the
MLB fan experience.
– Sean Curtis
Senior Vice President of Engineering
99. Change is speeding up
Traditional Data Center Cloud Data Center Dynamic Cloud
Dynamic Cloud enables better applications faster.
Good Better Best
The way you’ve done things in the past
won’t work in the future.
100. Server OS
Server (Virtual)
Hardware
Application &
Application
Microservices
Provisioning
Application &
Application
Microservices
Application &
Application
Microservices
BrowserMobile
Full Stack Monitoring
New Relic
Application
Monitoring
New Relic
Infrastructure
Monitoring
DASHBOARDS
Amazon
CloudWatch
101. Thank you
Lee Atchison ∙ Senior Director Strategic Architecture
New Relic
Architecting for Scale
By: Lee Atchison
Published by: O’Reilly Media
www.architectingforscale.com
leeatchison@leeatchison
102. This document and the information herein (including any information that may be incorporated by reference) is provided for informational
purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell
securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and
may not be replicated or disclosed without New Relic’s express written permission.
Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact
or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking
statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as
“believes,” “anticipates,”, “expects” or words of similar import.
Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are
subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New
Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such
forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks,
uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially
from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking
statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s
Investor Relations website at http://ir.newrelic.com or the SEC’s website at www.sec.gov.
New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no
warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
Safe Harbor