 Amazon CloudWatch Events

 AWS CodeCommit
 AWS Config
 AWS Internet of Things (IoT) button

3. Polling events Stream-based polling

Three services use the polling model. Two of them, DynamoDB and Kinesis Data Streams, are
stream-based. In the polling model, events put information into the stream AWS Lambda polls
the steam and if it finds records, it will deliver the payload and invoke the Lambda function. the
Lambda service itself is pulling data from the stream for processing by the Lambda function

With stream-based polling, Lambda gets a batch of records from within a shard, and
attempts to invoke the Lambda function. If there’s an error processing the batch of records,
Lambda will retry until the data expires, which can be up to seven days. The key thing to note is
that a failure in this model blocks Lambda from reading any new records from the stream until
the failed batch of records either expires or is processed successfully. This is important because
the events in each shard from the stream need to be processed in order

With streams, errors in a shard block further processing A failure in this model blocks Lambda
from reading any new records from the stream until the failed batch of records either expires or
is processed successfully. This is important because the events in each shard from the stream
need to be processed in order.

4. Polling events non-stream polling

Polling events non-stream polling

Amazon SQS, is not stream-based. This is the most recently added event source for Lambda. In
the polling model, events put information into the queue respectively. AWS Lambda polls the
queue, and if it finds records, it will deliver the payload and invoke the Lambda function. In this
model, the Lambda service itself is pulling data from the queue for processing by the Lambda
function

With queues, errors in a batch are returned to queue Lambda will keep retrying a failed message
until it is processed successfully, or the retries or retention period are exceeded. If the message
fails all retries, it will either go to the DLQ if configured, or it will be discarded.

An error doesn’t stop processing of the batch, but there may be a change to the order in which
messages are processed.

So when sequence matters, this is what you want. In contrast, with Amazon SQS, AWS Lambda
will poll a batch of records. If an invocation fails or times out, the failing message is returned to
the queue. It will be available for invocation again once the visibility timeout period expires.
Processing continues on other messages in the queue. Lambda will keep retrying the failed
message until the message is processed successfully, or the message retention period expires. If
the message expires, it will either go to the DLQ if you have one configured, or it will be
discarded. The key distinction with Amazon SQS is that an error doesn’t stop processing of the
batch, although there may be a change to the order in which messages are processed. If your
application doesn’t care about the order in which events are processed, you have a good reason
to use this method: An additional consideration is that in stream-based polling, there is no cost to
make the polling calls, but with SQS polling, standard SQS rates apply for each request.

6. Lifecycle of a Lambda function

 1. When a function is first invoked, an execution environment is launched and
bootstrapped. Once the environment is bootstrapped, your function code executes. Then,
Lambda freezes the execution environment, expecting additional invocations.
 2. If another invocation request for the function is made while the environment is in this
state, that request goes through a warm start. With a warm start, the available frozen
container is thawed and immediately begins code execution without going through the
bootstrap process.
 3. This thaw and freeze cycle continues as long as requests continue to come in
consistently. But if the environment becomes idle for too long, the execution
environment is recycled.
 4. A subsequent request starts the lifecycle over, requiring the environment to be
launched and bootstrapped. This is a cold start.

7. Different between warm start and cold start

 Warm start : With a warm start, the available frozen container is “thawed” and
immediately begins code execution without going through the bootstrap process. This
thaw and freeze cycle continues as long as requests continue to come in consistently
warm start

 Cold Start : But if the environment becomes idle for too long, the execution environment
is recycled. A subsequent request will start the lifecycle over, requiring the environment
to be launched and bootstrapped. This is a cold start.
For many application access patterns, the cold start latency is not a problem. But, if you have a
workload where a cold start is never acceptable, you can use provisioned concurrency to keep a
certain number of execution environments “warm”, even when the function hasn’t been invoked
in a while, or in cases where Lambda has to create new environments to keep up with requests.

We’ll talk more about concurrency in the Configuring your Lambda functions module, and
you’ll find a link to more information about provisioned concurrency in the section titled
Lifecycle of a Lambda function in the material that follows the video.

In 2019, AWS introduced provisioned concurrency to give you the ability to keep a number of
environments “warm” to prevent cold starts from impacting your function’s performance.

8. How to write functions to take advantage of a warm start

 1) Store and reference dependencies locally.
 2) Limit re-initialization of variables.
 3) Add code to check for and reuse existing connections.
 4) Use tmp space as transient cache.
 5) Check that background processes have completed.
 6) Minimize cold start times

Plan for real-world conditions and traffic patterns — cold starts may not have much impact on
your application and you could use provisioned concurrency to keep environments warm. But
it’s always a good idea to minimize the impact of a cold start.
Plan for real-world conditions and traffic patterns — cold starts may not have much impact on
your application and you could use provisioned concurrency to keep environments warm. But
it’s always a good idea to minimize the impact of a cold start.

Article Refer : https://aws.amazon.com/lambda/resources/workshops-and-tutorials/ (AWS

Lambda Foundations)

A useful article benchmark cold start, warm start performance between languages : https://filia-
aleks.medium.com/aws-lambda-battle-2021-performance-comparison-for-all-languages-
c1b441005fd1

9. Which types of Lambda runtime are supported by AWS?

For the latest list of supported runtimes, see AWS Lambda Runtimes in the AWS Lambda
Developer Guide.

 Node.js runtimes
 Python runtimes
 Ruby runtimes
 Java runtimes
 Go runtimes
 .NET runtimes

You can also use custom runtimes.

10. Describe the handler method in the lambda function?

The handler method is the entry point that AWS Lambda calls to start executing your Lambda
function. The handler method always takes two objects: the event object and the context object.

 DynamoDB Event

 KinesisFirehose Event
Event: The event object provides information about the event that triggered the Lambda function
(DynamoDBEvent, SQS Event, Kinesis Event, etc). This could be a pre-defined object that an
AWS service generates, or it could be a custom user-defined object in the form of a serializable
string. For example, it could be a pojo or a json stream.

Context: The context object is generated by AWS, and provides metadata about the execution.
You can use it to interact with Lambda. For example, it includes getAwsRequestId(),
getLogGroupName(), getRemainingTimeInMillis(); and the getInvokedFunctionArn(); function.

getInvokedFunctionArn : Gets the function Arn of the resource being invoked.

Note: with NodeJS the third argument is callback used for non-async purpose

https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html

11. Could you please tell me how to design a lambda

function
 Separate business logic
 Write modular functions
 Treat functions as stateless
 Only include what you need
 Use environment variables
 Avoid recursive code
  Separate your core business logic from the handler method.
This not only makes your code more portable but also allows you to target unit tests at
your code without worrying about the configuration of the function.

Service: External integration & service abstractions

Note: Handler function configuration, some specific lambda code, log something, and No
business logic here

 Write modular functions Create single purpose functions.

Follow the same principles you would apply to develop microservices.

Write modular functions

 Treat functions as stateless: No information about state should be saved within the
context of the function itself. Because your functions only exist when there is work to be
done, it’s particularly important for serverless applications to treat each function as
stateless.

DynamoDB is serverless and scales horizontally to handle your Lambda invocations. It also has
single-millisecond latency, which makes it a great choice for storing state information.

Consider Amazon ElastiCache if you have to put your Lambda function in a VPC. It may be less
expensive than DynamoDB.

Amazon S3 can be used as an inexpensive way to store state data, if throughput isn’t critical and
the type of state data you are saving won’t change rapidly.

 Only include what you need : Minimize both your deployment package’s dependencies
and its size.
This can have a significant impact on the startup time for your function. For example,
only choose the modules that you need — don’t include an entire AWS SDK.
 Reduce the time it takes Lambda to unpack deployment packages authored in Java. Put
your dependency.jar files in a separate /lib directory.
 Opt for simpler Java dependency injection (IoC) frameworks.
For example choose Dagger or Guice, over more complex ones like Spring Framework.
(https://medium.com/bigeye/dependency-injection-103-guice-vs-dagger-6ab7afa68d3d_
https://www.baeldung.com/guice-spring-dependency-injection )
  Use environment variables : Take advantage of environment variables for operational
parameters. They allow you to pass updated configuration settings without changes to the
code itself. You can also use environment variables to store sensitive information
required by the function.
https://docs.aws.amazon.com/lambda/latest/dg/env_variables.html
 Avoid recursive code : Avoid a situation where a function calls itself. This could
continue to spawn new invocations that would make you lose control of your
concurrency. You can quickly set the concurrent execution limit to zero by using the
console or command line to immediately throttle requests if you accidentally deploy
recursive code.

11. How many ways to deploy the lambda function

1. AWS Management Console
2. External tool (IDE) SAM CLI, Serverless Framwork
3. Link to S3 Bucket

12. Three core components to the configuration

 Memory
 Timeout
 Concurrency ( It’s a long story and challenge )

You are billed based on memory + duration

What are Cold Starts?

Cold starts refer to the time a lambda function needs to provision the environment to execute the
actual handler code. The basic steps of a cold start are:

1. Downloading code (e.g. from S3)

2. Start Environment
3. Initialise Code (outside of handler function)
In which cases do cold starts appear?

 The lambda function didn't run for a certain time

 Lambda needs to scale-out
 Lambda refreshes the environment

It is never 100% predictable when a cold starts to happen.

What are methods for avoiding cold starts?

 Function warmers: Call your lambda every n seconds. -> No 100% solution
 Provisioned Concurrency. See this article
 Not using lambda at all -> Use fargate or EC2

What is Provisioned Concurrency?

Provisioned Concurrency keeps your lambdas running instead of shutting them down after a
certain period. With that, no cold start times will be there but it is pretty expensive.

Which workloads are not well suited for lambda?

 Long-running processes. Often a container or many smaller lambda functions make more sense
 Simple filtering operations between AWS services -> No lambda needed

How can you use secrets in AWS Lambda?

Use the AWS Secretsmanager and access it via the API. Attach IAM permissions to the Lambda
role.

Where should you initialize static code?

Outside of the handler function. Because everything outside of the handler function won't run
again for warm lambdas.

How would you assign a certain private IP range to your lambda?

Put your lambda into a VPC in a certain subnet and this is possible for your internal network.

How do you monitor Lambda functions?

Lambda functions come with CloudWatch metrics. There is a CloudWatch Group per lambda
function with different streams. Metrics such as events, invocations, errors, latency, etc. are
automatically included. Custom metrics can be added.

Third-party tools such as Dashbird or Lumigo are very popular as well for monitoring lambdas.
What is Lambda @ Edge

Edge lambdas run in the Content-Delivery Network of AWS. They execute closer to the user's
location. They can be called from a CloudFront request. The available languages are Python and
Node.

What are the benefits compared to containers or EC2?

 You don't manage the infrastructure

 No operations for NACLs, Security Groups, and network operations
 You only pay for the time the lambda runs and not for idle times
 You focus only on the business logic

How do you log in Lambda Functions?

All standard outputs will be redirected to CloudWatch by default. It is encouraged to use a more
structured logger approach with JSON logger.

What is an idempotent lambda?

An idempotent function has the exact same behavior even if it will be executed several times.
That means somehow the function knows it was already executed or the execution of the code
won't change something.

This is often needed in an event-driven architecture since many calls in AWS are executed at
least once. For example, an SQS standard queue can execute the same message twice. For that
we need to ensure that the lambda function knows it was already executed (e.g. save the state in a
DynamoDB table) and do a certain action (e.g. exit early) if the function was already executed.

4. Explain troubleshooting AWS Lambda or other similar serverless platforms

Many issues can arise when using AWS Lambda or other similar serverless platforms. Different
issues related to deployment, invocation, execution, networking, or container image might occur
while using Lambda. You need to follow specific functions or services for troubleshooting any
issue. The following table suggests the relevant positions regarding AWS Lambda to
troubleshoot issues:

Issue type Services/Functions

Deployment AWS CloudFormation, AWS CodeDeploy, AWS CodePipeline

Invocation Use specific functions based on invoking issues

Execution Use AWS CloudWatch Logs and X-Ray

Networking Need to make changes in Virtual Private Cloud (VPC) configuration

Container Image Use specific functions based on issues

 5. What is AMI, and how to create it?

AMI stands for the Amazon Machine Image that contains the necessary information for
launching an instance. Amazon EBS snapshots, launch permissions, and block device mapping
are three main elements of AMI. Generally, we call it a template to use with the AWS service.

 You must use the same configuration to launch many instances using a single AMI. But use
different configurations to launch different instances using different AMIs.

You can use Lambda functions to create AMI. Steps for creating AMI are as follow:

 Use an existing AMI for launching an instance from a trusted AMI

 Customized it by adding components like installing software
 Save this configuration and named it “custom AMI”
 You can launch instances from this custom AMI

===============-============

External Python Libraries to AWS Lambda using Lambda Layers

If you are a Cloud Engineer or Developer who loves to code in python like myself, 7 out of 10
times, you might find yourself using AWS lambda. For Instance, you might be a Data Engineer
trying to use AWS lambda to preprocess your dataset and you need to import pandas or number
library. You will have a section of your code where you need to import the packages as
illustrated below

import pandas as pd
import numpy as np
If you run this line in your plain lambda function, you will get the response below from lambda

Unable to import module 'lambda_function': No module named 'pandas'

How do we resolve this? We need to create our own libraries as custom package, then find a way
to attach the package to lambda.

There are different ways to go about this but my favorite is using Lambda layers. According to
AWS,  "Lambda layers provide a convenient way to package libraries and other dependencies
that you can use with your Lambda functions."

We will divide this lab into 4 Sections:

1. Create a lambda function in AWS and test

2. Install your packages on your local machine and zip
3. Create lambda layer and upload packages
4. Attach layer to lambda and test

Create a lambda function in AWS and test

Login to your AWS and Navigate to lambda service. Click on "Create lambda" and fill the
details below. Name the lambda function "external-package-lambda" . Choose Python3.7 as your
runtime version.
Click Create. Wait till lambda creates. Scroll down and edit the code to look similar to what I
have below. All I added was 2 lines extra lines of code to import 2 libraries (pandas and numpy).
Try to type them in yourself and make sure your code looks exactly like this.

Go ahead and click "Deploy" to save the function. Next, click "Test" and you should see a pop
up that prompts you to name the test event. Enter Event name as "test" and leave everything else
as default. See below
Scroll down and click "Create". You should return back to main lambda page. Click "Test" again
and you should get similar response below.

Unable to import module 'lambda_function': No module named 'pandas'

Install your packages on aws cloudshell and zip

AWS Cloudshell is a newly created service in AWS where developers can programmatically
interact with AWS resources without spinning up any server. This will be perfect for our case.
We will be installing our packages from cloudshell, zipping it and then uploading to s3.

Quickly open the s3 console and create an s3 bucket and give it a unique name. This is where
you will store the zip package created. Save the name somewhere

Open aws cloudshell. It can can be found at the top right of the console. Looks like the image
below
Wait for the the terminal to load. Once it's loaded, create a directory named "packages", then
change into the directory by running the commands below one after the other

mkdir packages
cd packages

Next, create a virtual environment named venv and activate the virtual environment by running
the commands below one after the other.

python3 -m venv venv

source venv/bin/activate

Your virtual environment should be activated now. Make sure you remain in the same directory.
What you will do next is create a folder named "python" and change into the python directory by
running the commands below one after the other.

mkdir python
cd python

Its important you create a directory named python. This is mandatory when working with lambda
layers. Now that you are in python directory, install pandas and then numpy by running the
commands below one after the other.

pip install pandas -t .

pip install numpy -t .

We are using pip to install the packages specifying the current directory (python) as the target.
Wait until installation is done.

Try to list what is in the current directory by typing "ls" in terminal and it should look similar to
the below image.
Those are the packages we will be using for lambda. We will ned to zip the python folder where
they live but before we do that, let's save space and delete objects with ".dis-info" extension from
the folder. They are not needed. The best way to this is running the command below

rm -rf *dist-info

After running the above, we should be left with only the relevant packages needed.

Get out of the the python directory by going a directory backward using the command below

cd ..

Now, we will zip the python directory and give it a name called "my-first-lambda-package.zip".
Run the command below to achieve this.

zip -r my-first-lambda-package.zip python

Once done, you should have a zip file named my-first-lambda-package.zip in the current
directory.

Next, upload the packaged zip into the s3 bucket you created earlier. My command looks like the
below

aws s3 cp my-first-lambda-package.zip s3://your-s3-bucket-name/

Visit the s3 bucket you created to confirm the zip file was uploaded successfully. Make sure you
copy the object URL when you visit s3 to confirm by clicking the checkbox beside the object
and clicking Copy URL above . See below (Save the URL somewhere)
Create lambda layer and upload packages