Serverless Core
Serverless Core
Serverless Core
Serverless
Copyright © 2024 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Serverless Developer Guide
Amazon's trademarks and trade dress may not be used in connection with any product or service
that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any
manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are
the property of their respective owners, who may or may not be affiliated with, connected to, or
sponsored by Amazon.
Serverless Developer Guide
Table of Contents
Why a serverless guide? .................................................................................................................. 1
Meeting you where you are now .............................................................................................................. 2
Organization of this Guide ......................................................................................................................... 3
1 - Serverless Fundamentals ................................................................................................................ 3
2 - Service Starters ................................................................................................................................. 3
3 - Serverless Experiences ..................................................................................................................... 3
Identifying scenarios ....................................................................................................................... 5
Asynchronous data processing .................................................................................................................. 5
Synchronous interactions ............................................................................................................................ 6
Streaming ....................................................................................................................................................... 7
Picking up prerequisites .................................................................................................................. 9
Serverless learning path ........................................................................................................................... 10
Amazon Web Services Account ............................................................................................................... 11
Programming Language ........................................................................................................................... 14
Development Environment ...................................................................................................................... 14
Amazon Cloud Infrastructure .................................................................................................................. 15
Regions .................................................................................................................................................... 15
Amazon Resource Name (ARN) .......................................................................................................... 16
Security Model ............................................................................................................................................ 17
Summary ...................................................................................................................................................... 19
Shifting to a serverless mindset ................................................................................................... 20
Traditional development ........................................................................................................................... 20
Serverless solutions ................................................................................................................................... 22
Summary ...................................................................................................................................................... 23
Transitioning to event-driven architecture .................................................................................. 24
How does traditional translate to event-driven? ................................................................................. 26
Decoupled event-driven architecture ..................................................................................................... 28
Summary ...................................................................................................................................................... 30
Next steps .................................................................................................................................................... 31
Focusing on core services ............................................................................................................. 32
Core serverless services ............................................................................................................................ 32
Networking & Content Delivery ........................................................................................................ 34
Front-end Web & Mobile ..................................................................................................................... 34
Application Integration ........................................................................................................................ 34
iii
Serverless Developer Guide
iv
Serverless Developer Guide
v
Serverless Developer Guide
There are over two hundred Amazon Web Services. Each service has at least one developer guide
with hundreds of pages. Services also have API references, tutorials, workshops, and dozens of
related articles and videos.
Detailed and comprehensive documentation is invaluable. But, when you consider a typical solution
is built with six or more services, the volume of information can be overwhelming.
The goal of this serverless developer guide is to give you directed learning paths for the core
services needed to build serverless solutions.
In serverless solutions, you focus on writing code that serves your customers, without managing
servers. Serverless technologies are pay-as-you-go, can scale both up and down, and are easy to
expand across geographic regions.
1
Serverless Developer Guide
This guide will highlight what you need to know right away and link to service documentation for
advanced topics and details.
For example, you'll learn that the Lambda service spins up environments to run compute functions,
but to manage function scaling or reduce startup time, we'll link you to relevant sections of the
Lambda developer guide.
1 - Serverless Fundamentals
The guide starts with prerequisites to understand and use AWS, such as account creation and an
overview of Amazon cloud infrastructure (regions, ARNs, security model). Then, it will introduce
the mindset shift you need to start your serverless journey. Next, it will dive into event-driven
architecture and concepts helpful to transition to serverless.
Along the way, this guide will introduce core services and additional articles, workshops, and
tutorials, you can choose to reinforce your learning with hands-on activities.
Expected outcomes: You will understand how services combine, and you’ll create a microservice to
retrieve data.
2 - Service Starters
Core service starters will quickly explain the value and technical fundamentals of each service. Each
starter will also mention advanced topics, so you can start with the essentials, but be aware of
capabilities to dive into when you need them.
Starters are short reads (less than 2,300 words; 10-15 min) that connect concepts and practical
hands-on use.
3 - Serverless Experiences
There are many hands-on options to learn serverless. Choosing your path can be difficult.
Additional modules provide practical examples of unit and integration testing, using
infrastructure as code to deploy resources, and how to build common architectural patterns
used in serverless solutions.
3 - Serverless Experiences 4
Serverless Developer Guide
• Asynchronous data processing– big data processing, image/video manipulation, web hooks
• Synchronous interactions – web apps, web services, microservices, web hooks
• Streaming – processing inbound data streams, from apps, IoT devices
Solutions can be built with many services, but for each of these scenarios certain services are more
frequently encountered. Use this to page to focus on which services will be essential for you to
learn.
As the volume of data grows, coming from increasingly diverse sources, organizations find they
need to move quickly to process this data to ensure they make faster, well-informed business
decisions. To process data at scale, organizations need to elastically provision resources to manage
the information they receive from mobile devices, applications, satellites, marketing and sales,
operational data stores, infrastructure, and more.
Learn how to build a scalable serverless data processing solution. Use Amazon Simple Storage
Service to trigger data processing or load machine learning (ML) models so that Lambda can
perform ML inference in real time.
Examples:
• File processing – Suppose you have a photo sharing application. People use your application
to upload photos, and the application stores these user photos in an Amazon S3 bucket. Then,
your application creates a thumbnail version of each user's photos and displays them on the
user's profile page. In this scenario, you may choose to create a Lambda function that creates a
thumbnail automatically. Amazon S3 is one of the supported AWS event sources that can publish
object-created events and invoke your Lambda function. Your Lambda function code can read the
photo object from the Amazon S3 bucket, create a thumbnail version, and then save it in another
Amazon S3 bucket.
• Image identification — In the previous photo sharing application concept, imagine you want to
provide automatic categorization of images for your users. Images will be queued for processing
by Amazon Rekognition. After analysis, faces are detected and your app can use similarity scores
to group photos by family members. Objects, scenes, activities, landmarks, and dominant colors
are detected and labels are applied to improve categorization and search.
Synchronous interactions
Microservice architecture breaks applications into loosely coupled services. Each microservice is
independent, making it easy to scale up a single service or function without needing to scale the
entire application. Individual services are loosely coupled, letting independent teams focus on a
single business process, without the need for them to understand the entire application.
Microservices also let you choose which individual components suit your business needs, giving you
the flexibility to change your selection without rewriting your entire workflow. Different teams can
use the programming languages and frameworks of their choice to work with their microservice,
and this microservice can still communicate with any other in the application through application
programming interfaces (APIs).
Examples:
• Websites – Suppose you are creating a website and you want to host the backend logic on
Lambda. You can invoke your Lambda function over HTTP using Amazon API Gateway as the
HTTP endpoint. Now, your web client can invoke the API, and then API Gateway can route the
request to Lambda.
Synchronous interactions 6
Serverless Developer Guide
• Mobile applications – Suppose you have a custom mobile application that produces events.
You can create a Lambda function to process events published by your custom application. For
example, you can configure a Lambda function to process the clicks within your custom mobile
application.
Streaming
Streaming data allows you to gather analytical insights and act upon them, but also presents a
unique set of design and architectural challenges.
Lambda and Amazon Kinesis can process real-time streaming data for application activity tracking,
transaction order processing, clickstream analysis, data cleansing, log filtering, indexing, social
media analysis, Internet of Things (IoT) device data telemetry, and metering.
Examples:
• Data and analytics – Suppose you are building an analytics application and storing raw data in
a DynamoDB table. When you write, update, or delete items in a table, DynamoDB streams can
publish item update events to a stream associated with the table. In this case, the event data
provides the item key, event name (such as insert, update, and delete), and other relevant details.
You can write a Lambda function to generate custom metrics by aggregating raw data.
• Monitoring metrics - Amazon Prime Video monitors metrics from devices worldwide to ensure
quality-of-service. The team chose Amazon Kinesis Data Streams to deliver video stream
metadata and to collect metrics. Data is sent to Amazon OpenSearch Service for application
monitoring and forensic analysis. The services aggregate, analyze, and visualize petabytes of
data, to provide real-time insights that help the team find and fix streaming issues as they
happen. (Read full article)
Streaming 7
Serverless Developer Guide
Streaming 8
Serverless Developer Guide
Before you dive in to code, there are some key concepts you need to understand:
Topics are shown in orange pill boxes. Large topics may be broken down into several sub-topics in a
blue bubble. Icons represent related services or tools. Essential topics are noted with a green check
box. Important, but not essential, items are noted with a red heart. When a high level orange topic
is marked as essential, that means all of the sub-topics are essential too.
This map is a starting point. Not everyone will agree which topics are essential or important, so
check with your team. Find out which topics are essential or important for your journey.
9
Serverless Developer Guide
• AWS Identity and Access Management : Root account, Admin account, Least-privilege
• Note: AWS Identity and Access Management (IAM) controls who is authenticated and authorized
to use resources. IAM applies to developer accounts, roles, and resources in other services.
Programming Language
Development Environment
• Region
• Availability Zones
• Management Console
• ARN - Amazon Resource Names uniquely identify AWS resources
• Note: Each region has multiple Availability Zones (AZs) with one or more datacenters.
• Tip: Pick a region close to your customers!
Security Model
If you are creating a new account, you will create a root account using an email address. The root
account has unrestricted access, similar to root accounts for an operating system. As a best practice,
you should create an administrative user too.
• Signing in to the AWS Management Console provides an excellent step by step guide to account
types and login steps.
• AWS Identity and Access Management (IAM) is the service to manage entities and resources
authorized to use services and service resources.
If you do not have an AWS account, complete the following steps to create one.
1. Open https://portal.aws.amazon.com/billing/signup.
Part of the sign-up procedure involves receiving a phone call and entering a verification code
on the phone keypad.
When you sign up for an AWS account, an AWS account root user is created. The root user
has access to all AWS services and resources in the account. As a security best practice, assign
administrative access to a user, and use only the root user to perform tasks that require root
user access.
AWS sends you a confirmation email after the sign-up process is complete. At any time, you can
view your current account activity and manage your account by going to https://aws.amazon.com/
and choosing My Account.
After you sign up for an AWS account, secure your AWS account root user, enable AWS IAM Identity
Center, and create an administrative user so that you don't use the root user for everyday tasks.
1. Sign in to the AWS Management Console as the account owner by choosing Root user and
entering your AWS account email address. On the next page, enter your password.
For help signing in by using root user, see Signing in as the root user in the AWS Sign-In User
Guide.
For instructions, see Enable a virtual MFA device for your AWS account root user (console) in
the IAM User Guide.
For instructions, see Enabling AWS IAM Identity Center in the AWS IAM Identity Center User
Guide.
For a tutorial about using the IAM Identity Center directory as your identity source, see
Configure user access with the default IAM Identity Center directory in the AWS IAM Identity
Center User Guide.
• To sign in with your IAM Identity Center user, use the sign-in URL that was sent to your email
address when you created the IAM Identity Center user.
For help signing in using an IAM Identity Center user, see Signing in to the AWS access portal in
the AWS Sign-In User Guide.
1. In IAM Identity Center, create a permission set that follows the best practice of applying least-
privilege permissions.
For instructions, see Create a permission set in the AWS IAM Identity Center User Guide.
2. Assign users to a group, and then assign single sign-on access to the group.
For instructions, see Add groups in the AWS IAM Identity Center User Guide.
Programming Language
Amazon Web Service solutions are built for software developers. We assume that you have some
experience with coding and deploying programs using one of the supported languages. This guide
will not teach you how to program, but it will at times provide code samples.
Writing functions in an interpreted language like Python or JavaScript is easier because the code
can be added directly through the Management Console web interface.
You can use one of the listed languages, or you can bring your own runtime container.
This guide will default to Python in examples and tutorials, but we’ll strive to include other
languages over time.
Development Environment
For serverless development, you will likely want to set up and use a familiar editor/IDE, such as
Visual Studio Code or PyCharm. Alternatively, you may prefer AWS Cloud9 a browser-based IDE
and terminal “in the cloud” with direct access to your AWS account.
You should definitely install the AWS CLI for command line control and automation of services. For
example, you can list your Amazon S3 buckets, update Lambda functions, or send test events to
invoke service resources. Many tutorials will show how to complete tasks with the AWS CLI.
Programming Language 14
Serverless Developer Guide
Another useful, but optional, tool is the AWS Serverless Application Model CLI, aka the “SAM CLI”.
AWS SAM templates define infrastructure services and code. You can use AWS SAM CLI to build and
deploy from these templates to the cloud. AWS SAM CLI also provides features to test and debug
locally and deploy changes to infrastructure and code.
Tip
There are other tools that our customers use, such as the AWS Cloud Development Kit
(AWS CDK) for programmatic creation of infrastructure and code, or several 3rd party IaC
options.
You will find that some services provide emulators that can run on your local laptop. These tools
can be useful for local development, but they are also limited in terms of service and API coverage.
Serverless services are better suited to their native cloud environment.
Related resources:
• Install AWS CLI - to control and manage your AWS services from the command line
• Install AWS SAM CLI - to create, deploy, test, and update your serverless code and resources from
the command line
• Note: These tools are provided by AWS Cloud9, but you should update to the latest available
versions.
AWS provides services across the globe. You only need to understand how regions, availability
zones, and data centers are related so that you can select a region. You will see the region code in
URLs and Amazon Resource Names (ARNs), unique identifiers for AWS resources.
Regions
Every solution you build that runs in the AWS cloud will be deployed to at least one region.
• Region – a physical location around the world where we cluster data centers
• Availability Zone or “AZ” - one or more discrete data centers with redundant power, networking,
and connectivity within a Region
• Data center – a physical location that contains servers, data storage drives, and network
equipment
Amazon has many regions all across the globe. Inside each region, there are one or more
Availability Zones located tens of miles apart. The distance is near enough for low latency —
the gap between requesting and receiving a response, and far enough to reduce the chance that
multiple zones are affected if a disaster happens.
Each region is identified by a code, such as “us-west-1”, “us-east-1” or “eu-west-2”. Within each
region, the multiple isolated locations known as Availability Zones or AZs are identified with the
region code followed by a letter identifier. For example, us-east-1a. AWS handles deploying to
multiple availability zones within a region for resilience.
Services are identified with regional endpoints. The general syntax of a regional endpoint is as
follows:
protocol://<service-code>.<region-code>.amazonaws.com
The region code is also used to identify AWS resources with Amazon Resource Names, also called
“ARNs”. Because AWS is deployed all over the world, ARNs function like an addressing system to
precisely locate which specific part of AWS we are referring to. ARNs have a hierarchical structure:
arn:partition:service:region:account-id:resource-id
arn:partition:service:region:account-id:resource-type/resource-id
arn:partition:service:region:account-id:resource-type:resource-id
Try to identify the service, region, and resource for the following example ARNs:
arn:aws:dynamodb:us-west-2:123456789012:table/myDynamoDBTable
arn:aws:lambda:us-east-2:123456789012:function:my-function:1
If you are interested in learning more, check out a map of Regions and Availability Zones, a view of
our data centers, and the complete list of regional service endpoints.
Security Model
Security is a top priority for AWS. Before you start building serverless solutions, you need to know
how security factors into AWS solutions.
Security Model 17
Serverless Developer Guide
A shared security model means that Amazon manages certain aspects of security, and you are
responsible for others.
• Amazon is responsible for the security “of” the cloud. This includes such thing as the physical
security of the data centers.
• You are responsible for the security “in” the cloud. For example, you are responsible for your
data, and for granting functions only the access the need to complete their work.
For developers getting started with serverless and experts alike, the responsibility of securing the
resources and functions will take effort to understand and implement.
We’ll explain more along the way, but you should at least know that AWS commitment to security
is not taken lightly. AWS services have carefully established security mechanisms for you to create
secure solutions from the start. However, you have the responsibility to learn how and properly
implement these mechanisms in your solutions.
For more details, see the Shared Responsibility Model, AWS Cloud Security, and Security
Documentation Index with links to security documents for every service.
Security Model 18
Serverless Developer Guide
Summary
• You need an Amazon Web Services account to get started.
• Python and JavaScript/Typescript are popular programming languages for serverless. You will
see these most frequently in examples, tutorials, and workshops.
• Java, C#, Go, Ruby, and PowerShell are available runtimes, but you can also bring your own.
• Set up your development environment with your preferred local IDE
• AWS data centers are organized into one or more Availability Zones located in multiple regions
across the globe
• Region codes and ARNs are used to identify and connect to specific AWS services and resources
• Responsibility for security of serverless solutions is shared between you and AWS.
Summary 19
Serverless Developer Guide
Shifting to serverless
Revised: 2023-03-13
Developers are usually familiar with traditional web applications, but new to serverless
development. You may even know how to use some Amazon services directly, such as S3 or
DynamoDB, but you are not sure what it takes to go completely serverless.
We’ll take a look at a traditional application, then we’ll explain the shift in thinking needed to
begin to understand serverless development. You won’t be able to suddenly go build serverless
apps, but how the pieces fit together should make more sense.
Traditional development
Traditional web apps generally handle synchronous requests and responses. This cycle has been the
basis of web since the beginning of the Internet. Over time, developers created and shared code
to speed up development. You have probably used or at least recognize one or more of these web
frameworks: Express, Django, Flask, Ruby on Rails, Asp.net, Play, Backbone, Angular, Spring Boot,
Vapor, etc.
Web frameworks help you build solutions faster by including common tools and features.
The following diagram represents some of the complex mix of components that are included with
frameworks. Routers send URLs to classes or functions to handle requests and return responses.
Utility classes retrieve form data, query strings, headers, and cookies. A bundled abstraction layer
stores and retrieves data in SQL or NoSQL databases. Additional components manage connections
to external services through synchronous API calls or asynchronous message queues. Extension
points exist to bolt-on even more components, such as asynchronous hooks, or single sign-on
authentication (SSO) through a 3rd party.
Traditional development 20
Serverless Developer Guide
We call these solutions traditional because request/response has been the model for web
applications for decades. We call them monolithic because everything is provided in one package.
Traditional frameworks do an awful lot, but can be awfully complex doing it.
To be fair, traditional development does have advantages. Developer workstation setup is generally
quick. You typically setup a package manager or installer such as NPM, Gradle, Maven, homebrew,
or a custom CLI to initialize a new application, then you run the bare bones app with a command.
Frameworks that bring everything can boost initial productivity, especially when starting with a
simple solution.
But, this everything in one box approach makes scaling and troubleshooting problems difficult. As
an application grows, it tends to rely on more external systems. Asynchronous integrations with
these systems are preferred because they do not block the flow. But, asynchronous requests are
difficult to invoke and troubleshoot in a traditional architecture.
For asynchronous actions, the application logic must include timeouts, retry logic, and the status.
Single errors can cascade and impact many components. Workflows become more involved just to
keep the solution running.
Increases (or decreases) in demand for a particular feature require scaling up (or down) the entire
system. This is inefficient and difficult because all of the components and state are tightly coupled
together.
Traditional development 21
Serverless Developer Guide
The architecture of traditional monolithic webapps tends to become more complex over time.
Complexity increases ramp-up time for new developers, makes tracking down the source of bugs
more challenging, and delays the delivery of new features.
Serverless solutions
To build serverless solutions, you need to shift your mindset to break up monoliths into loosely
connected services. Consider how each service will do one thing well, with as few dependencies as
possible. If that sounds like a microservice architecture, you’re right!
You may have created microservices before, but it was probably inside a traditional framework.
Imagine if your microservice existed, but without the framework. For that to happen, services need
a way to get input, communicate with other services, and send outputs or errors.
Event-driven architecture (EDA) is a modern architecture pattern built from small, decoupled
services that publish, consume, or route events. Events are messages sent between services. This
architecture makes it easier to scale, update, and independently deploy separate components of a
system.
The following diagram shows an event-driven serverless microservice. A client request is converted
by an API Gateway into an event that is sent to a Lambda compute service. A Lambda function
retrieves info from a DynamoDB data store. That data is returned in an event to API Gateway, which
sends a response to the client with all the appropriate headers, cookies, and security tokens.
Serverless solutions 22
Serverless Developer Guide
Summary
• Traditional web application frameworks bundle services into a single complex monolith.
• Serverless is built on independent services that communicate through events in an event-driven
architecture.
• To build successful serverless solutions, you need to shift your mindset to building loosely
connected services that do one thing well.
Summary 23
Serverless Developer Guide
Event-driven architecture (EDA) is the first step on the serverless learning path. Understanding how
services interact through events is essential to successful serverless development.
In this chapter, you will dive into the transition from traditional to event-driven architecture.
You'll need a data store for menus, locations, orders, status. Customers will send network requests
to a web API. Compute services will be needed process orders, create and update data, and resize
images of the dishes.
Long-running tasks will need to be handled asynchronously. Multiple services will need to be
coordinated into business specific workflows. The workflow will include messaging users, reacting
to decisions, synchronizing information. Along the way, you will need to log actions, monitor app
activity, and trace data flows to debug.
All the while, components will send and receive events, shown in the following diagram as
interconnecting lines with a constant flow of events ("E") from service to service:
24
Serverless Developer Guide
If you are new to event-driven architecture, that picture of serverless architecture can be
disorienting. Fear not! As a developer, you are probably familiar with traditional frameworks, so
let's look at the steps in a traditional request/response cycle for comparison:
25
Serverless Developer Guide
Traditional frameworks hide a lot of these steps, but the complexity is still there. You're just used to
it. As you learn more about event driven serverless architecture, this way of building solutions will
become familiar too.
The diagram represents a microservice that retrieves data from a database in an event-driven
architecture. Imagine this microservice retrieves shopping cart items for a customer order:
First, a web/mobile client makes an HTTP request to GET /cart/A1234B56 for a list items in a cart.
A component needs to accept the request and extract metadata, such as http method, path, extra
path info, query string parameters, headers, and cookies. That component will also verify the
request is from an authenticated and authorized entity. In AWS, API Gateway accepts the inbound
URL, extracts the parameters, query string, and headers and creates an event to send to other
services for processing.
Example based on event.json API Gateway proxy event for a REST API:
{
"resource": "/cart",
"path": "/cart/A1234B56",
"httpMethod": "GET",
"headers": {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9",
"accept-encoding": "gzip, deflate, br",
...
"queryStringParameters": null,
"multiValueQueryStringParameters": null,
"pathParameters": null,
"stageVariables": null,
"body": null,
"isBase64Encoded": false
}
}
Tip
A dedicated API service may at first seem like overhead, but putting this responsibility into
a separate service adds flexibility and scalability to your solution. You can learn more about
API Gateway in the Starters section.
For example: an item placed in a shopping cart, a file uploaded to a storage system, or an order
becoming ready to ship. Events can either carry the state, such as: quantity (qty), item price
(itemPx), and currency; or simply contain identifiers needed to look up related information, such as:
customerId and orderId, as shown in the following example of a NewOrderEvent:
Next, API Gateway integrates with Lambda, a compute service, to handle the new event. Lambda
function code parses the parameters in the inbound event, connects to the data store, and retrieves
the cart. The function queries the database API through an SDK library. Because the DynamoDB
database is also serverless and built to respond with low latency, there is no need for a connection
pool.
After converting currency to USD and removing unavailable items, the function sends the result as
a new event to API Gateway.
Finally, API Gateway converts the event into a response to send to the waiting client.
Event-driven applications rely on communication through events that are also observable by other
services and systems. Event producers are unaware of which (if any) consumers are listening. This
strategy makes it easier to extend and scale, without disrupting existing workflows.
For example, API Gateway can enforce rate and volume limits for requests to your API on a per-
customer basis. Or, a service could watch the inbound query parameters to create a list of popular
product searches. The database could send events when products are added to the cart, to feed to
a predictive ML algorithm for ordering supplies.
Due to the loose coupling between components of an event-driven system, your compute
functions are not even aware of these other activities. You can scale components independently.
One service can fail, without impacting other services. Events can be flexibly routed, buffered,
provide a log for audit, and lead to more agile development.
Let’s revisit the diagram with various services connected through events. We can start to see now
that it is actually not as complex as it might have appeared.
Think of it like looking down on a big city with messengers moving packages and letters between
people and businesses. The sources and destinations range from the suburbs to the city core.
Inbound requests could be managed by a dispatcher, like API Gateway, Lambda Function URLs, or
Amazon EventBridge. Or, some messages might be dropped in a box for asynchronous delivery.
This would be like events routed to queues or orchestrated in complex workflows with Step
Functions. The monitoring, tracing, and metrics services, CloudWatch and AWS X-Ray, are like
managers, watching the a stream of events to make sure packages are delivered, and if not, to
troubleshoot the problem. After drop-off at a business, some packages are transferred by in-house
delivery agents to their final destination. This situation is similar to how services that store data
and files, may stream events to compute services, which triggers ever more actions.
When you look at the city of services, it is still complex, but when you break it down, each piece
becomes easier understand. As a developer, focus on components of a serverless system as
independent processors of workloads. You do not have to learn an entire monolithic system all at
once!
Connecting services with events and event-driven architecture gives you a consistent and scalable
way to build solutions with hundreds of services.
Summary
• Serverless is built on independent services that communicate through events in an event-driven
architecture.
Summary 30
Serverless Developer Guide
• event-driven architecture (EDA) - a modern architecture pattern built from small decoupled
services that publish, consume, or route events.
• events - represent a change in state, or an update
• Decoupled microservice architecture helps you build modern, agile, and extendable applications
faster than traditional monolithic applications, and free developers from needing to learn
everything at once about existing systems.
Next steps
• See What is EDA? for advantages of a decoupled architecture.
• Learn more about the advantages of modernizing monolithic applications in the AWS
Prescriptive Guidance enabling data persistence in microservices reference document.
Next steps 31
Serverless Developer Guide
AWS has over 220 services. Simply reading through the list to introduce them can take hours!
Think of all the services as tools in a toolbox. It’s great to have access to every one of them, but
you’ll start out using some more frequently than others. In this guide, we will focus on core services
you need to build serverless solutions.
You can read high level explanations of the core services here, and an example of how they interact
within the context of an example microservice, or you can choose to skip ahead to the hands on
workshop that uses three common services to build a working microservice.
Application Integration
• AWS Step Functions - orchestration service; useful when you have workflows with more than
one state, need to branch, or run tasks in parallel. The Step Functions service acts as the state
model for your application.
• Amazon EventBridge - integration with AWS & 3rd party services through events
• Amazon Simple Queue Service - simple queue service; buffering requests
• Amazon Simple Notification Service - simple notification system, publish/subscribe topics, and
sending a limited number of SMS/email messages
• AWS IoT Core - bi-directional communication for Internet-connected devices (such as sensors,
actuators, embedded devices, wireless devices, and smart appliances) to connect to the AWS
Cloud over MQTT, HTTPS, and LoRaWAN
• Amazon Simple Email Service - simple email system, bulk email sending service
Compute
• AWS Lambda - serverless compute functions; responsible for nearly all processing in serverless
projects
• Amazon Elastic Compute Cloud - non-serverless compute alternative; useful when you need
always-on and fully customizable capabilities. EC2 is often used for initial “lift and shift”
migration to the cloud. You can continue to use EC2 while migrating portions of your workflow
to serverless patterns.
• AWS App Runner - fully managed service to deploy your containerized web applications and
APIs. App Runner will scale compute instances and network resources automatically based on
incoming traffic.
• AWS Fargate - serverless computer for clusters of containers; useful when you need custom
containers but do not want to maintain and manage the infrastructure or cluster.
Compute 35
Serverless Developer Guide
configuration files, you write code that creates infrastructure. Your IDE can validate the definition
and even provide hints through code completion.
Developer Tools
• AWS CLI - command line utility for managing AWS resources
• AWS SAM CLI - command line utility for rapidly creating, deploying, and testing AWS resources
with AWS SAM templates
• Tools & SDKs - libraries for connecting to services and resources programmatically
• Cloud9 - cloud-based integrated development environment
• X-Ray — monitoring and debug
Let’s imagine an example microservice that looks up weather by zip code and returns JSON data.
The solution starts with the client resolving the hostname through Route 53 DNS. The browser
HTTP URL GET request will be sent to API gateway. If the URL is valid, API Gateway verifies access
with Cognito, then creates an event for the request and sends it to a serverless Lambda function
for processing.
The Lambda function receives the event and a context object with additional information related to
the environment as inputs to a designated handler method. The handler method in this case, uses
an SDK to send a query to DynamoDB for weather data for the given zip code. The function may
filter and customize the data based on the location and preferences of the user, perhaps converting
degrees in Celsius to Fahrenheit.
Before returning the data, bundled into a new event, back to API Gateway, the function handler
might create additional events. It might send one to an SQS queue, where a data analytics service
Developer Tools 36
Serverless Developer Guide
could be listening. The handler function might create and send another event to an SNS queue so
that alerts for high temperature are sent to users through SMS messages.
The function finally wraps up the JSON weather data into a new event and sends it back to API
gateway. Afterward, the function continues to handle hundreds of additional requests. Request
from users slow down after 2AM, so after some time the Lambda service will tear down the
function execution environment to conserve resources. As a Customer, you will only be charged for
function usage.
Although this is a contrived example, you should start to see how these core services can be put
together to build a solution. Communication between services through events is a foundational
concept. In the next chapter, we will dive deeper into how services and resource communicate with
events and event-driven architecture.
Summary
• There are hundreds of Amazon Web Services available.
• You only need to learn a subset of core fundamental services to build solutions.
• Events and event-driven architecture are foundational concepts to building serverless solutions.
Summary 38
Serverless Developer Guide
Fundamentally, the workflow is the same. You create and deploy code, invoke and test features,
update and fix bugs, package code with assets and libraries, apply security and secrets, and release
to production. After release, you monitor results, and then repeat the cycle.
Tip
For a hands-on approach, check out the Serverless Patterns Workshop.
In the workshop, you will experience a typical serverless development lifecycle, including
creating and deploying code and infrastructure through the web-based management
console, the CLI, and with Infrastructure as Code (IaC) automation.
39
Serverless Developer Guide
40
Serverless Developer Guide
Development Workflow
• Synchronous (essential)
• Asynchronous
• AWS SAM CLI can automate updates, packaging, and local testing.
• Packaging can include containers and layers - see Lambda for more info
Secure
• Least-priviledge
• Execution Role
• Resource-based Policy (RBAC)
• Services: IAM
• Note: If you start with less restrictive development permissions, always audit and use least-
privilege for production.
Monitor
• Logs, Errors
• Services: CloudWatch, X-Ray
• Note: Dive deeper into synchronous invocation with Serverless Workshop Module 2 and IaC using
Serverless Application Model (AWS SAM).
• Services: API Gateway, Lambda, DynamoDB + AWS SAM CLI
Development workflow
Development workflow is similar regardless of programming language, tooling, or architecture.
After you pick a language, app framework, and set up your local workstation, the DevSecOps cycle
begins:
Your unique business process may vary, but developing code for features (step 2) is nearly
universal. Developers write code, save, build and test locally. They commit after everything works
Development workflow 42
Serverless Developer Guide
as planned. This mini-cycle is sometimes referred to as the inner loop. You can think of it as a
constantly spinning flywheel, with releases flying off to be deployed into production.
Security (Sec) is, or should be, part of the process at every step, as shown in the diagram:
For traditional applications, developers commonly use a local development environment to test
their code end-to-end before committing. This local environment needs to maintain parity with
staging and production environments. That can be a full-time job in itself and prone to differences
and drift.
With serverless, more resources are in the cloud. Replicating the production environment on your
workstation is more challenging. Fortunately, spinning up developer sandboxes that replicate
production infrastructure is easier with serverless.
The essential skill you need is how to create infrastructure programmatically, known as
infrastructure as code or IaC. With IaC, you can automate the creation of service resources, code,
security settings, and connections between services. Because serverless can scale down to zero,
every developer can have an environment that is an exact replica of production. A developer could
spin up a dozen such instances, if necessary. For example, perhaps you want to stress test with
various allocations of memory and CPU.
You may be wondering, “Can’t I do this with containers?” That is similar, but serverless IaC brings
the benefits of creating resources and code all the way down to the individual function level. You
can deploy new code for a specific function without disrupting existing infrastructure or other
functions.
With the web-based console, you can create and deploy services, resources, and code into the
cloud. Wizards and automated workflows simplify the process. You can get a Management
Console mobile application to monitor resources, view alarms, and access billing information. The
following screenshot shows favorited and recently visited services:
When you need to create and deploy more than a few resources, tools are essential to automate
the process. You will need to understand Infrastructure as Code and tools to automate creating and
deploying your code and infrastructure resources.
These are some of the tools used to automate creation and maintenance of your cloud
infrastructure and code:
• AWS CLI tool can control and automate AWS services through the same service API used by the
language SDKs
• AWS Serverless Application Model uses higher level templates and CLI to speed up the code
development cycle
• AWS Cloud Development Kit (AWS CDK) is a code-first approach to defining infrastructure
In 2006, there were only a few services, including S3 and EC2. Managing these services was not
complex, but as more services were added to the AWS portfolio, creating and managing services
manually became cumbersome.
In 2011, AWS released AWS CloudFormation as a service to create and configure resources. With
AWS CloudFormation, you create text templates to define infrastructure and code. The service
provisions and configures those resources. Adding resources became as easy as copy/paste!
AWS CloudFormation provisions your resources in a safe, repeatable manner, enabling you to
frequently build your infrastructure and applications without manual steps. When you change the
configuration, AWS CloudFormation determines the right operations to perform to update your
stack. AWS CloudFormation can even roll back changes.
As new services were created, AWS CloudFormation template complexity increased. Large customer
environments are defined by templates with tens of thousands of lines. While better than manual
creation, at scale the templates can become difficult to manage.
In Nov 2016, AWS introduced the Serverless Application Model or AWS SAM, an open-source
framework for defining serverless applications. AWS SAM templates use a shorthand syntax to
define functions, APIs, databases, and event source mappings with just a few lines of text (YAML)
per resource.
During deployment, AWS SAM transforms and expands the AWS SAM syntax into AWS
CloudFormation syntax. Because of this, any AWS CloudFormation syntax can be added to AWS
SAM templates. This gives AWS SAM all the power of AWS CloudFormation, but with fewer lines of
configuration.
In July 2019, AWS launched AWS Cloud Development Kit (AWS CDK) a code-first approach to
defining cloud application infrastructure. With AWS CDK, you define and provision infrastructure
using code constructs. This brings the power of programming to IaC. Imagine you need to create
hundreds of queues named according to your product categories. No problem! You could write a
loop to provision the queues.
To review:
• AWS CloudFormation is a service that uses text templates to reliably automate provisioning
infrastructure stacks.
• AWS SAM templates are a higher level of abstraction YAML that transform into AWS
CloudFormation templates, and can embed AWS CloudFormation template syntax.
• AWS Cloud Development Kit (AWS CDK) is a code-first approach to defining infrastructure.
Remember, serverless services rely on an event-driven architecture. Invoking your code to test
requires creating and sending events to drive actions. AWS SAM CLI can invoke functions locally,
either directly or inside a container.
You can run code locally during the development of a feature, using a test harness. This sets up
an inner loop where you can iterate and test quickly. The following diagram illustrates the “Inner
Loop” with a TestHarness to verify your Business Logic locally. It also shows how you can deploy
AWS SAM templates to testing and production environments with the --profile setting.
For workflows that require resources that only exist in the cloud, you need a cloud-based
environment for testing.
Since serverless services like Lambda and Amazon API Gateway are pay-per-use, developers no
longer need to share environments. Every developer can have their own sandboxed AWS developer
account. You do not need to replicate all resources in a local environment, but instead you can test
with real resources in the cloud.
To review:
• You can invoke and test code locally that connects to some cloud resources.
• For cloud native tests, developers can set up sandboxed infrastructure environments that better
mimic production.
Related resource(s):
• Learn how to write and run unit and integration tests in the Serverless Patterns Workshop. See
Module 2 - Unit Test and Module 3 - Set up tests to test Lambda services with AWS SAM and
Python.
• In the Lambda developer guide, see testing serverless functions and applications for practical
test strategies.
For scripting languages (Node.js, Python, and Ruby), you can edit your function code in the
embedded Lambda console code editor. If your code is larger than 3MB, or you need to use
libraries, or you use languages that the editor doesn't support (Java, Go, C#), then you must upload
your function code as a .zip archive.
When you add libraries, make code changes, you will need to create a new package or container
to upload to the cloud. This is a slow process that AWS SAM CLI can automate and accelerate.
AWS SAM CLI has a deploy command that will push updates to AWS, and a feature called AWS
SAM Accelerate that speeds up deployments by using service APIs instead of AWS CloudFormation
templates. AWS SAM can also watch your local environment and automatically deploy changes to
the cloud.
You need to learn about Lambda layers, another feature that speeds up update/deploy cycle.
Lambda layers package libraries and dependencies to reduce deployment archive size. Layers
promote code sharing and separation of responsibilities so that you can upload changes and iterate
faster on the business logic.
Related resource(s):
• AWS SAM Accelerate - Quickly update serverless applications in AWS Cloud during development
• Lambda Containers - A container image, compatible with the Open Container Initiative (OCI)
specification, is one way to bring all your function code and dependencies to the cloud.
• Lambda .zip archives - A .zip file archive includes your application code and its dependencies.
• Lambda Layers - A layer is a .zip file archive that can contain additional code or data. A layer can
contain libraries, a custom runtime, data, or configuration files.
Security is a shared responsibility between AWS and you. The shared responsibility model describes
this as security of the cloud and security in the cloud:
• Security of the cloud – AWS is responsible for protecting the infrastructure that runs AWS
services in the AWS Cloud. AWS also provides you with services that you can use securely. Third-
party auditors regularly test and verify the effectiveness of our security as part of the AWS
compliance programs.
• Security in the cloud – Your responsibility is determined by the AWS service that you use. You
are also responsible for other factors including the sensitivity of your data, your company’s
requirements, and applicable laws and regulations.
From a developer perspective, you will use the Identity and Access Management service (IAM) to
grant least-privilege access to users, services, and resources. For example, you will provide each
of your functions with an execution role which defines specifically what actions the function can
take. Permissions can be created and applied granularly using resource and attribute based access
control policies.
• Monitoring provides the information about a system’s state and can help you discover that there
is an issue.
• Observability uses instrumentation to provide context and insights into what is causing issues.
You can use AWS X-Ray to trace user requests through your application to identify bottlenecks and
debug problems.
Instrumenting your application involves sending trace data for events within your application,
along with metadata about each request. Many AWS services are already integrated with X-Ray,
so for any traced request to your application, you can see detailed information for calls that your
application makes to downstream AWS resources, microservices, databases, and web APIs.
In the following example, you can see the durations of various requests and updates saving state in
an app called Scorekeep:
Summary
• The development lifecycle for serverless is similar to traditional for the “inner loop” to develop
features.
• The management console gives you full manual control, but learning how to use infrastructure
as code tools will help you spin up multiple environments quickly to replicate production for
testing.
• AWS SAM, AWS CDK, and AWS CloudFormation are essential infrastructure as code tools.
• AWS SAM CLI provides tools to accelerate creation, invocation, testing, and deploy of code and
resources to the AWS cloud.
Summary 51
Serverless Developer Guide
If you skipped ahead, here are four essential concepts to catch up:
• Traditional web frameworks provide structure and components to build monolithic apps.
• Serverless solutions break monoliths into loosely coupled services that communicate through
events.
• Events represent changes in state, or an update, which can contain data.
• Event-driven architecture (EDA) is the foundation for solutions that are faster to market, extend,
and scale.
Customers tell us they want production-ready patterns, working examples, and they do not want
to mechanically copy and paste solutions.
In parallel to this guide, a group of Amazon engineers are building a series of workshops based
on architectural and design patterns that customers commonly use in real-world solutions. You
get hands-on experience with infrastructure and code that you could actually deploy as part of a
production solution.
If you've read this far, then you've identified some serverless use cases, picked up the prerequisites,
shifted your mindset to serverless, transitioned your approach to event-driven architecture, and
begun to experience serverless development.
Core service starters will quickly explain the value and technical fundamentals of each service.
Each starter will also mention advanced topics, so you can start with the essentials, but be aware
of capabilities to dive into when you need them.
Starters are short reads (less than 2,300 words; 10-15 min) that connect concepts and practical
hands-on use.
• Visit Serverless Land
Serverless Land is an Amazon site with blogs, videos, code snippets, and learning resources for
AWS Serverless.
Note: This survey is hosted by an external company. AWS handles your information as described
in the AWS Privacy Notice. AWS will own the data gathered via this survey and will not share the
information collected with survey respondents.
Living document
This guide is a living document. It will expand with key topics new developers need on their
serverless journey. Please share your suggestions to help others start building successful serverless
solutions.
Starters are short reads (less than 2,300 words; 10-15 min) that connect concepts and practical
hands-on use.
Serverless Starters
• Get started with… Identity and Access Management (IAM)
• Get started with… Lambda functions
• Get started with… API Gateway
• Get started with… DynamoDB
• Learn by doing
Interactions with AWS services and resources by developers and entities require:
• Authentication: proof that the entity requesting access is who they claim to be
• Authorization: actions that are allowed or denied
AWS provides and uses a service called Identity and Access Management (IAM) for authentication
and authorization. IAM is used to manage developer accounts and secure the interaction between
services and resources.
Warning
Security is an important, complex, and broad topic. Large organizations generally have
specific operational procedures that developers need to follow. This guide will explain only
essential concepts necessary to get started with AWS services. If in doubt, consult your IT
department or the official security documentation.
Fundamentals
With IAM, developers attach policies, JSON documents that define granular permissions, to
resources. IAM provides pre-built AWS managed policies for common access levels. You can also
define your own policies with the least-privilege level necessary to complete tasks.
Information about IAM policies may come at you fast. If it gets to be too much, put it in PARC:
At a high level, these four terms should be enough to get you started connecting serverless
resources.
Account prerequisites
But, before you start, you need an AWS account. The following sections provide the best practice
steps to create an account and an administrative user.
If you do not have an AWS account, complete the following steps to create one.
1. Open https://portal.aws.amazon.com/billing/signup.
2. Follow the online instructions.
Part of the sign-up procedure involves receiving a phone call and entering a verification code
on the phone keypad.
When you sign up for an AWS account, an AWS account root user is created. The root user
has access to all AWS services and resources in the account. As a security best practice, assign
Fundamentals 57
Serverless Developer Guide
administrative access to a user, and use only the root user to perform tasks that require root
user access.
AWS sends you a confirmation email after the sign-up process is complete. At any time, you can
view your current account activity and manage your account by going to https://aws.amazon.com/
and choosing My Account.
After you sign up for an AWS account, secure your AWS account root user, enable AWS IAM Identity
Center, and create an administrative user so that you don't use the root user for everyday tasks.
1. Sign in to the AWS Management Console as the account owner by choosing Root user and
entering your AWS account email address. On the next page, enter your password.
For help signing in by using root user, see Signing in as the root user in the AWS Sign-In User
Guide.
For instructions, see Enable a virtual MFA device for your AWS account root user (console) in
the IAM User Guide.
For instructions, see Enabling AWS IAM Identity Center in the AWS IAM Identity Center User
Guide.
For a tutorial about using the IAM Identity Center directory as your identity source, see
Configure user access with the default IAM Identity Center directory in the AWS IAM Identity
Center User Guide.
Fundamentals 58
Serverless Developer Guide
• To sign in with your IAM Identity Center user, use the sign-in URL that was sent to your email
address when you created the IAM Identity Center user.
For help signing in using an IAM Identity Center user, see Signing in to the AWS access portal in
the AWS Sign-In User Guide.
1. In IAM Identity Center, create a permission set that follows the best practice of applying least-
privilege permissions.
For instructions, see Create a permission set in the AWS IAM Identity Center User Guide.
2. Assign users to a group, and then assign single sign-on access to the group.
For instructions, see Add groups in the AWS IAM Identity Center User Guide.
A common confusion arises when signing in to AWS. Remember, for day to day activities, you
should not be signing in as the root user.
Fundamentals 59
Serverless Developer Guide
Principals
IAM implements authentication, proving who an entity claims to be, with principals, which are
entities such as IAM users, federated users from Google, Facebook, etc, IAM roles, AWS accounts,
and AWS services.
Tip
An IAM role is identical in function to an IAM user, with the important distinction that it
is not uniquely associated with one entity, but assumable by many entities. Typically, IAM
roles correspond to a job function.
A loose analogy for IAM roles are that of professional uniforms: a surgeon's scrubs, a firefighter's
hardhat, or a startup CTO's favorite hoodie. Many people can assume the role of a surgeon,
firefighter, and startup CTO, which identifies them with a certain job function.
One of the most useful things about IAM roles is they can be associated not only with human
entities, but also with AWS services. These types of roles are known as service roles. This means you
can assign an IAM role directly to a service. With an IAM role assigned to the service instance, you
can then associate specific IAM policies with the instance role, so that the service instance itself can
access other AWS services. This is extremely useful for automation.
Authorization - PARC
So far we've been talking about principals. Principals represent the authentication component. For
authorization, you will attach JSON documents called IAM policies to principals.
Principals
As mentioned, principals are the entities that are allowed or denied access.
Actions
Actions are the type of access that is allowed or denied. Actions are commonly AWS service API
calls that represent create, read, describe, list, update, and delete semantics.
Resources
Resources are the AWS resources the action will act upon.
Fundamentals 60
Serverless Developer Guide
All AWS resources are identified by an Amazon Resource Name (ARN) . Because AWS services are
deployed all over the world, ARNs function like an addressing system to precisely locate a specific
component. ARNs have hierarchical structures:
arn:partition:service:region:account-id:resource-id
arn:partition:service:region:account-id:resource-type/resource-id
arn:partition:service:region:account-id:resource-type:resource-id
Related resource(s):
• IAM identifiers provides an exhaustive list in the docs for IAM ARNs
Conditions
Other Elements
• All IAM policies have an Effect field which is set to either Allow or Deny.
• Version field defines which IAM service API version to use when evaluating the policy.
• Statement field consists of one or many JSON objects that contain the specific Action, Effect,
Resource, and Condition fields described previously
• Sid (statement ID) is an optional identifier for a policy statement; some services like Amazon
Simple Queue Service and Amazon Simple Notification Service might require this element and
have uniqueness requirements for it
Fundamentals 61
Serverless Developer Guide
Policies
When you set permissions, you attach a JSON policy to a principal. In the following example, an
AWS managed policy named AWSLambdaInvocation-DynamoDB will be attached to a role that is
related to a Lambda function:
Fundamentals 62
Serverless Developer Guide
You can also create customer policies that apply to specific actions, resources, and custom
conditions. The following policy example includes a condition that applies to an IP range:
Fundamentals 63
Serverless Developer Guide
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:RunInstances"
...
<additional actions>
],
"Resource": "*",
"Condition": {
"IpAddress": { "aws:SourceIp": "12.34.56.78/32"}
}
}
]
}
First, notice there is no Principal element in the policy. This is because attaching a policy to a
principal implicitly specifies the principal to which the policy applies. Policies can exist apart from
principals, so that common policies can be re-used for many roles, services, etc.
The Effect is Allow, so this is a policy that explicitly grants access. This is the most common type
of policy. You can write policies that explicitly Deny access as well.
The Action is an array of multiple actions that permit using EC2 to run instances and describe
instance metadata. Actions can be a single action, or multiple actions in an array like depicted here.
There is no specific ARN in the Resource field, but instead *, which is a wildcard character that
means this policy applies to all resources.
Finally, we have a Condition set that applies this policy only if the caller's IP address matches
exactly 12.34.56.78. Conditions are optional, and do not need to be specified if the policy is to be
applied unconditionally.
IAM policies can be combined, each with varying degrees of sensitivity and specificity. The net
effect of combining policies is fine-grained access control for every resource in an AWS account.
Fundamentals 64
Serverless Developer Guide
For IAM principals, requests to AWS are implicitly denied. This means that if no policies are
attached to a principal, IAM's default behavior is to deny access.
Next, if the principal does have an attached policy, and there is an explicit allow, the implicit deny
is overridden. However, an explicit deny in any policy overrides any allows. In complex situations,
there can be additional steps, but the following diagram represents this simplified model of how
IAM evaluates identity based policies:
Warning
Identity based policies do not affect the root user, so actions taken by the root user
account are implicitly allowed.
The root user is special in this regard and is the only principal that has this type of access.
Fundamentals 65
Serverless Developer Guide
Advanced topics
You can do a lot just using AWS managed policies. As you progress on your journey, you should
explore the following more advanced topics.
Resource-based policies
When you create a permissions policy to restrict access to a resource, you can choose an identity-
based policy or a resource-based policy.
Identity-based policies are attached to a user, group, or role. These policies let you specify what
that identity can do (its permissions). For example, you can attach the policy to the group named
RemoteDataMinders, stating that group members are allowed to get items from an Amazon
DynamoDB table named MyCompany.
Resource-based policies are attached to a resource. For example, you can attach resource-based
policies to Amazon S3 buckets, Amazon Simple Queue Service queues, VPC endpoints, and AWS
Key Management Service encryption keys.
With resource-based policies, you can specify who has access to the resource and what actions they
can perform on it.
Related resource(s):
With a permissions boundary, you set the maximum permissions that an identity-based policy can
grant to an IAM entity.
When you set a permissions boundary for an entity, the entity can perform only the actions
that are allowed by both its identity-based policies and its permissions boundaries. Permissions
boundaries limit the maximum permissions for the user or role.
For example, assume that the role named CoreServiceAdmin should be allowed to manage only
Amazon S3, Amazon CloudWatch, and AWS Lambda. To enforce this rule, you can set a policy to set
the permissions boundary for the CoreServiceAdmin role.
Advanced topics 66
Serverless Developer Guide
Related resource(s):
Additional resources
• Simplifying serverless permissions with AWSAWS SAM Connectors - AWS Compute blog post
by Kurt Tometich, Senior Solutions Architect, AWS, from Oct 2022 that introduces a AWS SAM
abstraction that creates minimally scoped IAM policies
• Building AWS Lambda governance and guardrails - AWS Compute blog post by Julian Wood,
Senior Solutions Architect, AWS, from Aug 2022 that highlights how Lambda, as a serverless
service, simplifies cloud security and compliance so you can concentrate on your business logic.
Next Steps
• Work through the Getting Started Resource Center 30-45 min tutorial on Setting Up Your AWS
Environment to properly set up your AWS account, secure the root user, create an IAM user, and
setup AWS CLI and (optionally) Cloud9 environment.
All projects need a compute capability to handle processing tasks. Here are some examples:
Additional resources 67
Serverless Developer Guide
In traditional applications, you write code to do these tasks. You organize that code into functions.
You put the function code inside an application framework. Whichever framework you picked will
run inside a language dependent runtime environment. Finally, that runtime environment will be
hosted on a virtual or physical server.
Setting up, configuring and maintaining the frameworks, runtime environments, and virtual or
physical infrastructure slows down your delivery of features, bug fixes, and improvements.
What is Lambda?
In Lambda, you write function code. Lambda runs the functions. That’s it. There are no servers.
What is Lambda? 68
Serverless Developer Guide
“No Server Is Easier To Manage Than No Server” - Werner Vogels, VP and CTO
The Lambda service runs instances of your function only when needed and scales automatically
from zero requests per day to thousands per second. You pay only for the compute time that’s
actually used — there is no charge when your code is not running.
Fundamentals
Serverless solutions are based on event-driven architecture, or EDA, where services send and receive
events, which represent an update or change in state. The primary activity of Lambda functions is
to process events.
Fundamentals 69
Serverless Developer Guide
Within the Lambda service, your function code is stored in a code package, deployed as a .zip or a
container image. All interaction with the code occurs through the Lambda API. There is no direct
invocation of functions from outside of the Lambda service.
What you will learn on your journey to building applications with Lambda:
Fundamentals 70
Serverless Developer Guide
The following is a text representation of the key concepts in the preceding diagram.
The Lambda learning path forks into two paths. The conceptual path focuses on the programming
model, runtime environment, and security concepts. The other path includes practical steps to
build a application while introducing development workflow activities such as how to create and
Fundamentals 71
Serverless Developer Guide
deploy functions, invoke and test, update and package, and monitor the logs and troubleshoot
errors.
Programming Model
• Event plus Context and Environment variables (ENV) are inputs to a Handler function
• ENV variables
• Runtime environment
• Management Console
• Infrastructure as Code (IaC) - AWS CloudFormation (CFN), AWS SAM (SAM), AWS Cloud
Development Kit (AWS CDK)
• Deploy .zip file archives — when you need additional libraries, or compiled languages.
• Versions - by publishing a version of your function, you can store your code and configuration as
separate stable resources
Invoke/Test
• Synchronous invocation
• Testing locally and in the cloud with the help of AWS SAM templates and AWS SAM CLI
Update / Package
Monitor
• Logs in CloudWatch
• Errors and tracing in X-Ray
• Metrics
Secure
• Execution role
Fundamentals 72
Serverless Developer Guide
Workshop - Intro to Serverless - Before diving too deep, you can choose to try out serverless in
a workshop or tutorial. Connect to a data source and create a REST API with your first Lambda
function.”
Programming Model
The Lambda service provides the same event-based programming model for all languages. The
Lambda runtime passes an invocation event and context to your Lambda function handler which
does some work and produces a resulting event:
The invocation event contains data, as a JSON packet, which varies from service to service. For
example, API gateway events include path, HTTP method, query string parameters, headers,
cookies, and more. DynamoDB events could contain updated or delete record data. S3 events
include the bucket name and object key, among other things.
The context contains information about the environment the function is running inside. Additional
contextual information can be set in familiar environment variables (ENV).
The function handler is a method in your function code that processes the inbound event. The
handler, which is a standard function in your language of choice, does some work and emits a result
event.
Fundamentals 73
Serverless Developer Guide
After the handler finishes processing the first event, the runtime sends it another, and another.
Each instance of your function could process thousands of requests.
Unlike traditional servers, Lambda functions do not run constantly. When a function is triggered by
an event, this is called an invocation. Lambda functions are limited to 15 minutes in duration, but
on average, across all AWS customers, most invocations last for less than a second.
Even the smallest Lambda-based application uses at least one event that invokes your function.
Lambda invokes your function in an execution environment, which contains a secure and isolated
runtime environment.
Fundamentals 74
Serverless Developer Guide
You can use runtimes that Lambda provides for JavaScript (Node.js), TypeScript, Python, Java,
Python, Go, C#, and PowerShell, or you can build your own custom runtime environment inside of a
container.
If you package your code as a .zip file archive, you must configure your function to use a runtime
that matches your programming language. For a container image, you include the runtime when
you build the image.
Conceptually, there are only three steps to processing events with Lambda:
1. Configure the entry point to your function, known as the handler, and deploy the function.
2. Lambda service initializes the function, then it invokes the handler with an invocation event and
context.
3. Your handler function processes the event and returns a response event.
Subsequent events will invoke the handler again, without the initialization delay. During this cycle,
the function stays in memory, so clients and variables declared outside of the handler method can
be reused.
After a period of time, Lambda will eventually tear down the runtime. This can happen for a variety
of reasons; some examples: scaling down to conserve resources, updating the function, updating
the runtime.
The function handler is the essential component of your function code. As noted previously, the
handler is the entry point, but it may not be the only function in your code. In fact, a best practice
is keeping the handler sparse and doing the actual processing in other functions in your code.
Python
Fundamentals 75
Serverless Developer Guide
Node.js
Java
@Override
public String handleRequest(Map<String,String> event, Context context)
{
LambdaLogger logger = context.getLogger();
String response = new String("200 OK");
logger.log("EVENT: " + gson.toJson(event));
return response;
}
}
C#
namespace HelloWorld;
Fundamentals 76
Serverless Developer Guide
{
public string FunctionHandler(string input, ILambdaContext context)
{
context.Logger.LogLine($"Transforming {input} to upper case");
return input.ToUpper();
}
}
Handlers in interpreted languages can be deployed directly through the web-based AWS
Management Console. Compiled languages, such as Java and C#, or functions that use external
libraries are deployed using .zip file archives or container images. Because of that additional
process, this guide will focus on Python for examples.
Regardless of language, Lambda functions will generally return a response event on successful
completion. The following program listing is an example response event to send back to API
Gateway so that it can handle a request:
{
"statusCode": 200,
"headers": {
"Content-Type": "application/json"
},
"isBase64Encoded": false,
"multiValueHeaders": {
"X-Custom-Header": ["My value", "My other value"],
},
"body": "{\n \"TotalCodeSize\": 104330022,\n \"FunctionCount\": 26\n}"
}
You might have noticed the logging statements in the preceding handler code. Where do those log
messages go?
During invocation, the Lambda runtime automatically captures function output to Amazon
CloudWatch.
In addition to logging your function's output, the runtime also logs entries when function
invocation starts and ends. This includes a report log with the request ID, billed duration,
Fundamentals 77
Serverless Developer Guide
initialization duration, and other details. If your function throws an error, the runtime returns that
error to the invoker.
To help simplify troubleshooting, the AWS Serverless Application Model CLI (AWS SAM CLI) has
a command called sam logs which will show you CloudWatch Logs generated by your Lambda
function.
For example, the following terminal command would show the live tail of logs generated by the
YourLambdaFunctionName Lambda function:
Logging and debugging go hand in hand. Traces of events are available with Amazon X-Ray for
debugging.
Securing functions
AWS Identity and Access Management (IAM) is the service used to manage access to AWS services.
Lambda is fully integrated with IAM, allowing you to control precisely what each Lambda function
can do within the AWS Cloud. There are two important things that define the scope of permissions
in Lambda functions:
• resource policy: Defines which events are authorized to invoke the function.
• execution role policy: Limits what the Lambda function is authorized to do.
Using IAM roles to describe a Lambda function’s permissions, decouples security configuration
from the code. This helps reduce the complexity of a lambda function, making it easier to maintain.
A Lambda function’s resource and execution policy should be granted the minimum required
permissions for the function to perform it’s task effectively. This is sometimes referred to as the
rule of least privilege. As you develop a Lambda function, you expand the scope of this policy to
allow access to other resources as required.
Fundamentals 78
Serverless Developer Guide
Advanced Topics
You can do a lot by just creating a function and connecting it to an event source like API Gateway
or S3 triggers.
As you progress on your journey, you should explore the following more advanced topics.
Some services can trigger Lambda functions directly, for example, when an image is added to an
S3 bucket, a Lambda can be triggered to resize it. Some services cannot invoke Lambda directly;
but you can instead use an event source mapping which is a polling mechanism that reads from an
event source and invokes a Lambda function.
You can use event source mappings to process items from a stream or queue in the following
services:
• Amazon DynamoDB
• Amazon Kinesis
• Amazon MQ
• Amazon Managed Streaming for Apache Kafka (Amazon MSK)
• Self-managed Apache Kafka
• Amazon Simple Queue Service
Related resource:
• Event source mapping official documentation, including the default behavior that batches
records together into a single payload that Lambda sends to your function.
Advanced Topics 79
Serverless Developer Guide
If you need a custom runtime that is not provided by AWS, you can create and deploy a custom
container image. AWS provides base images preloaded with a language runtime and other
components that are required to run the image on Lambda. AWS provides a Dockerfile for each of
the base images to help with building your container image.
Custom containers are one way you might experiment with lift and shift of existing code to
Lambda runtimes. If you do this, consider the architectural differences between always running
containers, versus on demand nature of Lambda functions.
Related resource:
A Lambda layer is a .zip file archive that can contain additional code or other content. A layer can
contain libraries, a custom runtime, data, or configuration files. Layers are also necessary if your
function .zip archive exceeds the size limit.
Layers provide a convenient way to package libraries and other dependencies that you can use with
your Lambda functions. Using layers reduces the size of uploaded deployment archives and makes
it faster to deploy your code. Layers also promote code sharing and separation of responsibilities so
that you can iterate faster on writing business logic.
Related resource:
Extensions
You can use Lambda extensions to augment your Lambda functions. For example, use Lambda
Extensions to integrate with your preferred monitoring, observability, security, and governance
tools.
Lambda supports internal or external extensions. An internal extension runs as part of the runtime
process. An external extension runs as an independent process in the execution environment and
continues to run after the function invocation is fully processed.
Advanced Topics 80
Serverless Developer Guide
Related resources:
• Datadog Lambda Extension - an extension that supports submitting custom metrics, traces, and
logs asynchronously while your Lambda function executes.
• Lambda Extensions - official documentation
Lambda SnapStart for Java can improve startup performance by up to 10x at no extra cost,
typically with no changes to your function code. The largest contributor to startup latency (often
referred to as cold start time) is the time that Lambda spends initializing the function, which
includes loading the function's code, starting the runtime, and initializing the function code.
With SnapStart, Lambda initializes your function when you publish a function version. Lambda
takes a Firecracker microVM snapshot of the memory and disk state of the initialized execution
environment, encrypts the snapshot, and caches it for low-latency access.
Note: You can use SnapStart only on published function versions and aliases that point to versions.
You can't use SnapStart on a function's unpublished version ($LATEST).
Related resources:
• Accelerate Your Lambda Functions with Lambda SnapStart - an AWS Compute blog article by
Jeff Barr from Nov 2022 that shows the configuration change and vast difference from roughly
six seconds init time to 142 milliseconds of restore time with SnapStart
Advanced Topics 81
Serverless Developer Guide
A function URL is a dedicated HTTP(S) endpoint for your Lambda function. You can create and
configure a function URL through the Lambda console or the Lambda API. When you create a
function URL, Lambda automatically generates a unique URL endpoint for you. Once you create a
function URL, its URL endpoint never changes. Function URL endpoints have the following format:
https://<url-id>.lambda-url.<region>.on.aws
After you configure a function URL for your function, you can invoke your function through its
HTTP(S) endpoint with a web browser, curl, Postman, or any HTTP client.
Related resources:
Additional resources
• AWS Lambda Developer Guide - extensive and complete documentation for Lambda
Next steps
Additional resources 82
Serverless Developer Guide
Desktop and mobile browsers, command line clients, and applications all make requests to your
web-based APIs. Your application API must handle these requests, scale based on incoming traffic,
ensure secure access, and be available in multiple environments.
For serverless applications, Amazon’s API Gateway acts as the entry point, also called the “front
door”, for your web-based applications.
You are likely familiar with how to use and setup backend APIs in traditional application
frameworks. In this starter, you will learn the essential role Amazon API Gateway plays in the
request/response workflow. Additionally, API Gateway can optimize response time with cache,
API Gateway 83
Serverless Developer Guide
prevent function throttling, and defend against common network and transport layer DDoS
attacks.
Amazon API Gateway is an AWS service for creating, publishing, maintaining, monitoring, and
securing REST, HTTP, and WebSocket APIs. API Gateway can transform inbound web requests into
events that are processed by Lambda functions, AWS services, and HTTP endpoints.
API Gateway can process many concurrent API calls and provides additional management
capabilities, including:
Although similar in purpose, the REST API and HTTP API configuration and management console
experiences are quite different. WebSocket APIs are included in the advanced section, as they are
less frequently used when starting out.
• REST API configuration models resources with Client, Method request, Integration Request,
Integration endpoint, Integration Response, Method Response, return to Client.
• HTTP API configuration models routes with a tree of HTTP methods and associated integration
configuration.
Important
Unless otherwise stated, REST APIs are discussed in the fundamentals.
Fundamentals
An API (Application Program Interface) is a collection of endpoints that connect to your application
logic. You may already be familiar with creating APIs in traditional frameworks. This section will
introduce how the concepts map to API Gateway.
Core Concepts
You can choose to get started with the core concepts in a workshop or tutorial, or read through the
high level concepts here.
Let’s start with a high level work flow for an API request, pictured in the following diagram.
Imagine a UI component that requests User data from the server to show a table of users:
Create an API
When you create an API Gateway REST API, there are two essential components:
Fundamentals 85
Serverless Developer Guide
• method – HTTP methods (GET, POST, PUT, PATCH, OPTIONS and DELETE) An HTTP method is
an action to be taken on a resource. API Gateway also has an ANY method, that matches any
method type.
• resource – A resource is related to your business logic, for example users, orders, or messages.
The following diagram shows the components of a URL request: API Endpoint, Stage, and
Resource.
• API Endpoint: The hostname for the API. Unless you designate a custom domain name, all APIs
created using API Gateway will have this structure.
• Stage Name: An API deployment. You can deploy different snapshots of your API to various
stages, for example: “v2", "latest", "dev", "qa".
• Resource: The piece of your business logic provided by the request.
To create an REST API resource, you specify the resource path, then add a method with an API
integration endpoint. You can then further configure the integration and test it in the console.
The following screenshot shows an example REST API integration for a GET method for the /users
resource.
After you are satisfied with your configuration, you must deploy the API to a stage so it
will become available to process requests.
Fundamentals 86
Serverless Developer Guide
During the creation of an API method, you must integrate it with an backend endpoint. A backend
endpoint is also referred to as an integration endpoint and can be one of the following:
Like an API request, an API integration has an integration request and an integration response.
The integration request encapsulates the HTTP request received by the backend. The integration
response is an HTTP response that wraps the output returned by the backend.
For advanced non-proxy integrations (REST API only), setting up an integration response involves
the following:
Fundamentals 87
Serverless Developer Guide
• Configuring how to pass the backend-returned result to a method response with a given HTTP
status code
• Configuring how to transform specified integration response parameters to preconfigured
method response parameters
• Configuring how to map the integration response body to the method response body according
to the specified body-mapping templates.
In the simpler case of proxy integrations, the preceding steps are handled automatically.
After you have deployed your REST API, you can invoke it. The following shows the standard
format of an API URL
https://{restapi_id}.execute-api.{region}.amazonaws.com/{stage_name}/
You can call an API using a web browser, through the CLI using cURL or the Postman application.
After you deploy your API, you can turn on invocation logs using CloudWatch to monitor your API
calls.
You can call a REST API before deployment for testing in two ways:
• In the API Gateway console by using the API Gateway's TestInvoke feature.
• Through the CLI using the test-invoke-method command.
Both of these methods bypass the Invoke URL and any authorization steps to allow API testing.
Alternatively, after the API is successfully deployed, you can use any command line or graphical
tool, like cURL or Postman to call your API.
Related resource(s):
Fundamentals 88
Serverless Developer Guide
• Postman home page - Learn how to invoke APIs with the Postman application
To authenticate and authorize access to your Rest APIs, you can choose from the following:
• Amazon Cognito user pools as an identity source for who access the API.
• Lambda functions to control access to APIs by using a variety of identity sources.
• Resource-based policies to allow or deny specified access from source IP addresses or VPC
endpoints.
• AWS Identity and Access Management roles, policies, and IAM tags to control access for who can
invoke certain APIs.
Related resource(s):
Advanced Topics
You can connect a microservice with a REST API and a single integration to a Lambda function. As
you progress on your journey, you should explore the following advanced topics.
Advanced Topics 89
Serverless Developer Guide
API Gateway offers two types of RESTful API products: REST APIs and HTTP APIs .
• REST APIs support more features than HTTP APIs, but pricing is higher than HTTP APIs.
• HTTP APIs consist of a collection of routes that direct incoming API request to backend resources.
Routes are a combination of an HTTP method and a resource path, such as “GET /users/
details/1234”. HTTP APIs also provide a $default route that is a catch-all for requests that
don’t match other more specific routes.
Choosing between the type of API depends on your specific use case:
• Choose a REST API if you need advanced features, such as mock integration, request validation,
a web application firewall, certificates for backend authentication, or a private API endpoint with
per-client rate limiting and usage throttling,
• Choose an HTTP API if you need minimal features, lower price, and auto-deployment.
Related resource(s):
• Choosing between REST APIs and HTTP APIs - detailed comparison between REST and HTTP APIs
Your API integration contains an integration request and integration response. You can have API
Gateway directly pass the request and response between the frontend and backend, or you can
manually set up an integration request and integration response.
• proxy integration – you let API Gateway automatically pass all data in the HTTP request/response
between the client and backend, automatically, without modification
• non-proxy integration – you set up a custom integration request and integration response, where
you can alter the data that flows between client and backend
Advanced Topics 90
Serverless Developer Guide
• Non-proxy custom integrations can transform data before it gets to your integration service
and before the output is sent to clients. For Lambda, your function code can focus on the
business task rather than parsing data in the input event. Non-proxy can be a good fit for legacy
code migrations too, because you can transform the data to match expectations of existing code.
Related resource(s):
• Tutorial: Build a Hello World REST API with Lambda proxy integration (proxy example)
• Using API Gateway as a proxy for DynamoDB (custom non-proxy integration example)
• Best practices for working with Apache Velocity Template Language (VTL) (custom non-proxy
integration example)
To reduce the number of calls made to your endpoint and improve the latency of requests to your
API, you can cache responds from your endpoint for a specified time-to-live period. API Gateway
will respond to the request by using the cache instead of your end point. This can speed up your
latency.
Related resource(s):
• Enabling API caching to enhance responsiveness - Learn how to enable caching for your REST
APIs.
You can use either REST APIs or HTTP APIs to send binary payloads such as a JPEG or gzip file. For
REST APIs, you need to take additional steps to handle binary payloads for non-proxy integrations.
Related resource(s):
• Return binary media from a Lambda proxy integration - Learn how to use a Lambda function to
return binary media. This works for both REST and HTTP APIs.
• Working with binary media types for REST APIs - Additional considerations for REST non-proxy
integrations
Advanced Topics 91
Serverless Developer Guide
If you want to handle the routing implementation outside of API Gateway, for example inside a
Lambda function, you can use a greedy path variable in the form of a proxy resource {proxy+} to
match all the child resources of a route for both HTTP and REST APIs.
Related resource(s):
• Use a proxy resource to streamline API setup – Set up a proxy resource and greedy path variable
for REST APIs.
• Working with routes for HTTP APIs – Set up a greedy path variable for HTTP APIs
WebSocket APIs
WebSocket APIs are a connection of WebSocket routes that are integrated with backend HTTP
endpoints, Lambda functions, or other AWS services. A client can send messages to a service, and
services can independently send messages to clients, making these APIs bidirectional. Because
of this, WebSocket APIs are often used as chat applications or for multiplayer games or financial
trading platforms.
Related resource(s):
• About WebSocket APIs in API Gateway - Get started with WebSocket APIs
• Tutorial: Building a serverless chat app with a WebSocket API, Lambda, and DynamoDB -
Intermediate level WebSocket API tutorial using AWS CloudFormation
OpenAPI
The OpenAPI 3 definition allows you to import and export APIs using API Gateway. OpenAPI is a
standard format that is language agnostic and vendor-neutral and is used to define and structure
REST APIs. There are many Open API extensions to support the AWS-specific authorization and API
Gateway-specific API interactions for REST APIs and HTTP APIs.
Advanced Topics 92
Serverless Developer Guide
You can use OpenAPI API definitions in AWS SAM templates for more complicated applications.
Or, you can build APIs with API Gateway and export the OpenAPI 3.0 definition to use with other
services.
Related resource(s):
Additional resources
• API Gateway Developer Guide - extensive and complete documentation for Amazon API Gateway
• REST API tutorial - REST API tutorial
• HTTP API tutorial - HTTP API tutorial
• Building Happy Little APIs | I Didn’t Know Amazon API Gateway Did That - AWS video series
introducing to API Gateway
Next Steps
Additional resources 93
Serverless Developer Guide
You are likely familiar with storing data in SQL and NoSQL databases in traditional solutions. Due
to its rapid response and low latency, Amazon DynamoDB, a NoSQL data store released in 2012 is a
frequently used data storage service for serverless solutions.
What is DynamoDB?
Amazon DynamoDB is a fully managed serverless NoSQL database service. DynamoDB stores data
in tables. Tables hold items. Items are composed of attributes. Although these components sound
similar to a traditional SQL table with rows and fields, there are also differences which will be
explained in the fundamentals section.
DynamoDB 94
Serverless Developer Guide
Data access is generally predictable and fast, in the millisecond (ms) range. If you need even faster
response time, the DynamoDB Accelerator (DAX) provides in-memory acceleration for microsecond
level access to data.
Traditional web frameworks maintain persistent network connections to SQL databases with
connection pools to avoid latency accessing data. With serverless architecture and DynamoDB,
connection pools are not necessary to rapidly connect and scale the database. Instead, you can
adjust your tables' throughput capacity, as needed.
What is DynamoDB? 95
Serverless Developer Guide
For rapid local development, modeling, and testing, AWS provides a downloadable version of
DynamoDB that you can run on your computer. The local database instance provides the same API
as the cloud-based service.
Fundamentals
In DynamoDB, tables, items, and attributes are the core components. Data items stored in tables
are identified with a primary key, which can be a simple partition hash key or a composite of a
partition key and a sort key. Although these terms may sound familiar, we will define all of them to
clarify how similar and different they are from traditional SQL database terms.
Core Components
Fundamentals 96
Serverless Developer Guide
• Table – a collection of data. For example, a table called People could store personal contact
information about friends, family, or anyone else of interest. You could also have a Cars table to
store information about vehicles that people drive.
Data in a DynamoDB is uniquely identified with a primary key, and optional secondary indexes for
query flexibility. DynamoDB tables are schemaless. Other than the primary key, you do not need to
define additional attributes when you create a table.
• Item – An item is a group of attributes that is uniquely identifiable among all of the other items.
In a People table, each item represents a person. For a Cars table, each item represents one
vehicle.
Items in DynamoDB are similar to rows, records, or tuples in other database systems. In DynamoDB,
there is no limit to the number of items you can store in a table. DynamoDB items have a size limit
of 400KB. An item collection, a group of related items that share the same partition key value, are
used to model one-to-many relationships. (1)
• Attribute –An attribute is a fundamental data element, something that does not need to be
broken down any further. For example, an item in a People table contains attributes called
PersonID, LastName, FirstName, and so on. In a Cars table, attributes could include Make,
Model, BuildYear, and RetailPrice. For a Department table, an item might have attributes such as
DepartmentID, Name, Manager, and so on. Attributes in DynamoDB are similar in many ways to
fields or columns in other database systems.
Most of attributes are scalar, which means that they can have only one value. Strings and numbers
are common examples of scalars. Attributes may be nested, up to 32 levels deep. An example could
be an Address which contains Street, City, and PostalCode.
Watch an AWS Developer Advocate explain these core concepts in this video: Tables, items, and
attributes (6 min).
As mentioned in the video, the primary key for the following table consists of both a partition key
and sort key. The sort keys “inventory::armor” and “inventory::weapons” contain double colons to
Fundamentals 97
Serverless Developer Guide
add query flexibility to get all inventory. This is not a DynamoDB requirements, just a convention
by the developer to make retrieval more flexible.
All of the data for account1234 will be stored in the same database partition to ensure retrieval of
related data is quick.
Related resources:
Reading data
DynamoDB is a non-relational NoSQL database that does not support table joins. Instead,
applications read data from one table at a time. There are four ways to read data:
• GetItem – Retrieves a single item from a table. This is the most efficient way to read a single
item because it provides direct access to the physical location of the item. (DynamoDB also
provides the BatchGetItem operation, allowing you to perform up to 100 GetItem calls in a
single operation.)
Fundamentals 98
Serverless Developer Guide
• Query – Retrieves all of the items that have a specific partition key. Within those items, you can
apply a condition to the sort key and retrieve only a subset of the data. Query provides quick,
efficient access to the partitions where the data is stored.
• Scan – Retrieves all of the items in the specified table. This operation should not be used
with large tables because it can consume large amounts of system resources. Think of it like a
“SELECT * FROM BIG_TABLE” in SQL. You should generally prefer Query over Scan.
• ExecuteStatement retrieves a single or multiple items from a table.
BatchExecuteStatement retrieves multiple items from different tables in a single operation.
Both of these operations use PartiQL, a SQL-compatible query language.
• Partition key - also called a hash key, identifies the partition where the data is stored in the
database.
• Sort key - also called a range key, represents 1:many relationships
The primary key can be a partition key, nothing more. Or, it can be a composite key which is a
combination of a partition key and sort key. When querying, you must give the partition key, and
optionally provide the sort key.
Amazon DynamoDB provides fast access to items in a table by specifying primary key values.
However, many applications might benefit from having one or more secondary (or alternate) keys
available, to allow efficient access to data with attributes other than the primary key. To address
this, you can create one or more secondary indexes on a table and issue Query or Scan requests
against these indexes.
A secondary index is a data structure that contains a subset of attributes from a table, along with
an alternate key to support Query operations. You can retrieve data from the index using a Query,
in much the same way as you use Query with a table. A table can have multiple secondary indexes,
which give your applications access to many different query patterns.
• Global secondary index — An index with a partition key and a sort key that can be different
from those on the base table. A global secondary index is considered "global" because queries
on the index can span all of the data in the base table, across all partitions. A global secondary
index is stored in its own partition space away from the base table and scales separately from the
base table.
Fundamentals 99
Serverless Developer Guide
• Local secondary index — An index that has the same partition key as the base table, but a
different sort key. A local secondary index is "local" in the sense that every partition of a local
secondary index is scoped to a base table partition that has the same partition key value.
In DynamoDB, you perform Query and Scan operations directly on the index, in the same way that
you would on a table.
Data types
DynamoDB supports many different data types for attributes within a table. They can be
categorized as follows:
• Scalar Types – A scalar type can represent exactly one value. The scalar types are number, string,
binary, Boolean, and null.
• Document Types – A document type can represent a complex structure with nested attributes,
such as you would find in a JSON document. The document types are list and map.
• Set Types – A set type can represent multiple scalar values. The set types are string set, number
set, and binary set.
Related resource:
Operations on tables
Operations are divided into Control plane, Data plane, Streams, and Transactions:
• Control plane operations let you create and manage DynamoDB tables. They also let you work
with indexes, streams, and other objects that are dependent on tables. Operations include
CreateTable, DescribeTable, ListTables, UpdateTable, DeleteTable.
• Data plane operations let you perform create, read, update, and delete (also called CRUD) actions
on data in a table. Some of the data plane operations also let you read data from a secondary
index. Operations include: ExecuteStatement, BatchExecuteStatement, PutItem, BatchWriteItem
(to create or delete data), Get Item, BatchGetItem, Query, Scan, UpdateItem, DeleteItem
• DynamoDB Streams operations let you enable or disable a stream on a table, and allow access
to the data modification records contained in a stream. Operations include: ListStreams,
DescribeStreams, GetSharedIterator, GetRecords
Fundamentals 100
Serverless Developer Guide
Note: you can also use PartiQL - a SQL-compatible query language for Amazon DynamoDB, to
perform data plane and transactional operations.
Advanced Topics
You can do a lot just creating a DynamoDB table with a primary key. As you progress on your
journey, you should explore the following more advanced topics.
NoSQL Workbench is a cross-platform visual application that provides data modeling, data
visualization, and query development features to help you design, create, query, and manage
DynamoDB tables.
• Data modeling - build new data models, or design models based on existing data models.
• Data visualization - map queries and visualize the access patterns (facets) of the application
without writing code. Every facet corresponds to a different access pattern in DynamoDB. You
can manually add data to your data model.
• Operation builder - use the operation builder to develop and test queries, and query live
datasets. You can also build and perform data plane operations, including creating projection
and condition expressions, and generating sample code in multiple languages.
You can also run a local instance of DynamoDB on your workstation. Combined with NoSQL
workbench, this can provide a fast local setup for experimentation and learning.
Related resources:
• NoSQL Workbench & Building data models with NoSQL Workbench - model and query data with
a desktop tool
• Setting up DynamodDB local (downloadable version)
DynamoDB Streams
DynamoDB Streams is an optional feature that captures data modification events. The data about
these events appear in the stream in near-real time, and in the order that the events occurred, as a
stream record.
If you enable a stream on a table, DynamoDB Streams writes a stream record whenever one of the
following events occurs:
• A new item is added to the table: the stream captures an image of the entire item, including all
of its attributes.
• An item is updated: the stream captures the "before" and "after" image of any attributes that
were modified in the item.
• An item is deleted from the table: the stream captures an image of the entire item before it was
deleted.
Each stream record also contains the name of the table, the event timestamp, and other metadata.
Stream records have a lifetime of 24 hours; after that, they are automatically removed from the
stream.
You can use DynamoDB Streams together with AWS Lambda to create an event source mapping—
a resource that invokes your Lambda function automatically whenever an event of interest appears
in a stream
For example, consider a Customers table that contains customer information for a company.
Suppose that you want to send a "welcome" email to each new customer. You could enable a
stream on that table, and then associate the stream with a Lambda function. The Lambda function
would run whenever a new stream record appears, but only process new items added to the
Customers table. For any item that has an EmailAddress attribute, the Lambda function would
invoke Amazon Simple Email Service (Amazon SES) to send an email to that address.
Related resources:
• Change data capture with Amazon DynamoDB -- stream item-level change data in near-real time
• Change data capture for DynamoDB Streams - DynamoDB Streams captures a time-ordered
sequence of item-level modifications in any DynamoDB table and stores this information in a log
for up to 24 hours.
Transactions
Amazon DynamoDB transactions simplify the developer experience of making coordinated, all-or-
nothing changes to multiple items both within and across tables. Transactions provide atomicity,
consistency, isolation, and durability (ACID) enabling you to maintain data correctness in your
applications more easily.
You can use the DynamoDB transactional read and write APIs to manage complex business
workflows that require adding, updating, or deleting multiple items as a single, all-or-nothing
operation. With the transaction write API, you can group multiple Put, Update, Delete, and
ConditionCheck actions. You can then submit the actions as a single TransactWriteItems
operation that either succeeds or fails as a unit.
Related resource:
• DynamoDB Transactions: How it works - Explains how to group actions together and submit as
all-or-nothing TransactWriteItems or TransactGetItems operations
You can run ad hoc PartiQL queries against tables. PartiQL operations provide the same availability,
latency, and performance as the other DynamoDB data plane operations.
Related resources:
In most cases, the DynamoDB response times can be measured in single-digit milliseconds. If your
use case requires a response in microseconds, is read-heavy, or has bursty workloads, DAX provides
fast response times for accessing eventually consistent data, increased throughput, and potential
operational cost savings.
Related resource:
• In-memory acceleration with DynamoDB Accelerator (DAX) - for response times in microseconds,
increased throughput, and potential operational cost savings for large read-heavy or bursty
workloads.
Additional resources
• Amazon DynamoDB Developer Guide - extensive and complete documentation for Amazon
DynamoDB
• Getting started with DynamoDB - tutorial to create a table, write/read/update and query data.
You will use the AWS CLI, with the option to run PartiQL DB queries.
• Getting Started Resource Center - Choosing an AWS database service - Choosing the right
database requires you to make a series of decisions based on your organizational needs. This
decision guide will help you ask the right questions, provide a clear path for implementation, and
help you migrate from your existing database.
• Creating a single-table design with Amazon DynamoDB - blog article by James Beswick (26
JUL 2021) showing how to model many to one and many to many relationships with indexes in
DynamoDB.
• Additional Amazon DynamoDB Resources - more links to blog posts, guides, presentations,
training, and tools
Next steps
In parallel to this guide, a group of Amazon engineers are building a series of workshops based
on architectural and design patterns that customers commonly use in real-world solutions. You
get hands-on experience with infrastructure and code that you could actually deploy as part of a
production solution.
Learn by doing
In parallel to this guide, a group of Amazon engineers are building a series of workshops based
on architectural and design patterns that customers commonly use in real-world solutions. You
get hands-on experience with infrastructure and code that you could actually deploy as part of a
production solution.
There are many hands-on options to learn serverless. Choosing your path can be difficult.
109