WHITE PAPER

Microservice Architecture:
API Gateway Considerations

Sanjay Gadge, Principal Architect

Vijaya Kotwani, Senior Engineer
Table of Contents

Introduction ........................................................................................... 3

The What and Why of API Gateways .................................................... 4

Security ................................................................................................. 5
Authentication and Authorization .................................................................................. 5
Threat Protection from DDoS ....................................................................................... 6
Secure Communication ................................................................................................ 6
Deployment Considerations ......................................................................................... 6

Service Registry and Service Discovery ............................................... 7

Service Registry ........................................................................................................... 7
Service Discovery ........................................................................................................ 8

Orchestration ........................................................................................ 9

Transformation ...................................................................................... 9

Monitoring ............................................................................................. 10
Health Monitoring ......................................................................................................... 10
Traffic and Data Monitoring ........................................................................................... 10

Load Balancing and Scaling ................................................................. 11

High Availability and Failover ................................................................ 12

Governance .......................................................................................... 12

Conclusion ............................................................................................ 13

References ............................................................................................ 13
 Introduction

Introduction
In today’s extremely competitive business environment, Microservice architecture consists of a suite of independently
customers are more demanding than ever and will abandon deployable, small, modular, and compassable (composable)
a business that is too slow to respond. This has put an onus services. Each service runs a unique process and
on IT to deliver solutions that provide a holistic and uniform communicates through a well-defined, lightweight
experience to the customer, across all business channels. mechanism to serve a business goal. It aligns with the
Microservice architecture has the potential to address this business to deal with changes in agile fashion, matches
business challenge; it is all about achieving speed and safety business changes with agile response, and delivers solutions
at scale and it provides the flexibility to pick and choose in a decentralized manner. In addition to modular services,
technology for implementing a solution. This approach the API gateway and other elements are integral parts of
positions IT as a business partner rather than in a traditional microservice architecture (see figure below).
support role.

Figure 1.
In addition to modular
services, the API
gateway and other
elements are integral
parts of microservice
architecture.

Microservice Architecture: API Gateway Considerations 3

 The What and Why of API Gateways

The What and Why of

API Gateways
Basically, the API Gateway is a reverse proxy to microservices which provides flexibility to freely add or remove microservice
and acts as a single-entry point into the system. It is similar instances to adapt to the load/demand of microservices.
to a Facade pattern from object-oriented design and similar There may be an instance when different consumers of the
to the notion of an “Anti-Corruption Layer” in Domain service require particular data and/or have it in a special
Driven Design. It makes the processes of API design, format. For example:
implementation, and management considerably simpler and
more consistent. The gateway helps to address some of the • An e-commerce mobile app shows product information
key concerns, including: and availability of product on the product details
screen, but the desktop website version of the same
• How to deal with features such as security, throttling, e-commerce site shows recommendations, in addition to
caching and monitoring at one place product information and availability.
• How to avoid chatty communication between clients and
microservices • There are mobile apps where one understands XML
• How to satisfy the needs of heterogeneous clients payload and others understand JSON.
• How to route requests to backend microservices
• How to discover working microservice instances The API Gateway is the best place to address these
• How to discover when a microservice instance is not transformation requirements, which can be accomplished
running by providing application-specific APIs for the same business
feature at the gateway level. A one-size-fits-all approach
With a single-entry point into the system, it becomes would make it hard to extend functionality, as the degree of
easy (and manageable) to enforce runtime governance diversity increases.
such as common security requirements, common design
decisions (e.g. every consumer of the service should have The gateway also helps by recording data for analysis
X-Correlation-ID header), and real-time policies such as and auditing purposes, load balancing, caching, and
monitoring, auditing and measuring API usage, and throttling. static response handling. The diagram below shows how
The gateway abstracts microservices from their consumers, the gateway typically fits into the overall microservice
architecture.

Figure 2.
Demonstration of
how a gateway
typically fits into the
overall microservice
architecture

Microservice Architecture: API Gateway Considerations 4

 Security

Security
Security is an important requirement of any enterprise Every API request is authenticated at the gateway layer. On
solution. At a certain point in the architecture, the best behalf of the end user, the application client first grabs an
options available for authentication, authorization, threat access token from the authentication server by presenting
protection, message protection, etc. must be chosen. credentials. This access token is then passed along with the
API request in Authorization HTTP header. The API Gateway
Authentication and Authorization then validates the access token with the authorization
server. The JWT token, which contains the claims for the
Federated identity is the preferred solution for implementing user, is then passed to backend microservices. Backend
authentication and authorization in microservice architecture. microservices then use the information inside the JWT token
Each microservice does not necessarily need to obtain for authorization purpose. The same JWT token is passed
and store users’ credentials in order to authenticate them. along when one microservice communicates with another
Instead, microservices can use an identity management microservice.
system that is already storing a user’s identity to authenticate
the user. This approach allows the decoupling of the The flow above has the potential to be a “confused deputy
authentication and authorization functions. It also makes it problem,” as every microservice is relying on the API
easier to centralize these two functions, to avoid a situation Gateway for authentication. Ideally, every microservice should
where every service must manage a set of credentials for authenticate the token as received from its caller (gateway
every user. or microservice). There is a trade-off between security and
performance. The above mentioned architecture leans
There are three major protocols for federated identity: toward performance, as there are other mechanisms to
OpenID, SAML, and OAuth. The figure below shows the mitigate the security risk.
security architecture using OAuth2.0.

Figure 3.
Security architecture
using OAuth2.o

Microservice Architecture: API Gateway Considerations 5

 Security

The microservice architecture is part of the overall IT • Black list / White list IP addresses, if you can definitely
infrastructure for an enterprise. If the enterprise IT is cloud identify valid and invalid end users of your APIs
focused, then you should use a well-known cloud based
authorization server, like Azure Active Directory or AWS • Limiting the connections to backend microservices
IAM, which can also be integrated with on premise identity
stores, like active directory. An open-source server like • Blocking requests:
IdentityServer4 makes it possible to implement your own o that seem to target a specific API
authorization server and integrate with existing identity stores. o that have a User-Agent header, set to a value that
does not correspond to normal client traffic
o that have a referrer header, set to a value that can
Threat Protection from DDoS be associated with an attack
o that have other headers with values that can be
Most of the API Gateway provides (either integral or add-
associated with an attack
on packages) features that can handle DDoS attacks,
by regulating and controlling the traffic as it proceeds to
backend microservices. Consider configuring the following
Secure Communication
traffic regulating parameters for the API Gateway:
It is always desirable to have SSL/TLS compliant endpoints
at the API Gateway, as well as at the microservices layer, to
• Limiting the rate of requests: Maximum number
safeguard against man-in-middle attacks, and bi-directional
of requests an API can access within a given time
encryption of message data to protect against tampering.
frame, based on rate limiting approach. Some of the
approaches are Authenticated User, Request Origin,
If you are dealing with a number of certificates, then
Authenticated User, and Request Origin. For example,
managing those becomes a huge administrative burden.
you might decide that an API may be accessed only
There are solutions like letsencrypt.org, an AWS certificate
once a day, by an authenticated user from a specific
manager, which makes it possible to transparently issue or
mobile application.
revoke certificates.

• Limiting the number of connections: Maximum number

of connections that can be opened by a single client for Deployment Considerations
an API
To strengthen security further, you should host all
microservices on private subnet and whitelist the gateway
• Closing slow connections: Time span, after which a
IP at the microservices layer. If it is not possible to have
connection will be closed from a client that is writing data
microservices on private subnet, then you should validate
too infrequently, which can represent an attempt to keep
each token with the authorization server per service call,
connections open as long as possible
however, this will impact performance.

Microservice Architecture: API Gateway Considerations 6

 Service Registry and Service Discovery

Service Registry and

Service Discovery
Ease in scaling is one of the key advantage of microservice There are two models for checking the status of a service:
architecture. You will keep adding or removing microservice pull model and push model. Although some of the registries
instances to adapt to incoming traffic. In addition, your support the pull model, it is not recommended, as it puts an
teams may be adding new microservices or refactoring additional load on the registry. Moreover, it is the service’s
existing microservices into multiple (especially when moving responsibility to update the registry about its availability/
from monolith to microservices). Service instances have unavailability to serve the request.
dynamically assigned network locations.
As a critical component, the service registry needs to be
Service Registry highly available. You should plan for a cluster of registries
that uses replication to maintain consistency. Never try to
The service registry helps to keep track of these instances. It cache the network locations obtained from the registry at
is a database containing the network locations of the service the registry user (API Gateway or registry aware client). That
instances. Every service instance registers itself on start-up information eventually becomes out-of-date, causing clients
and de-registers on shutdown. The API Gateway uses this to become unable to discover service instances.
information during service discovery. The figure below shows
service registration and discovery.

Figure 4.
Service registration and
discovery

Microservice Architecture: API Gateway Considerations 7

 Service Registry and Service Discovery

Service Discovery
The number and location of service instances is dynamic. Server-side discovery is preferred for various reasons:
Consequently, your client code needs to use a more
elaborate service discovery mechanism. There are two main • It removes the discovery burden from the client so it can
service discovery patterns: client-side discovery and server- focus on business functions.
side discovery, as shown in the figure below.
• If you have multiple clients, then you need to code and
maintain the discovery code for each client.

• It reduces number of calls over the internet.

Figure 5.
There are two main
service discovery
patterns: client-side
discovery and server-
side discovery.

Microservice Architecture: API Gateway Considerations 8

 Orchestration & Transformation

Orchestration Transformation
It is often necessary to orchestrate across multiple fine- Often microservices have to deal with different clients
grained microservices to accomplish a business use case. on the front end. They have different needs, from both
As shown in the figure below, there are two options for protocol (SOAP, REST, JSON,and XML) and data
implementing the orchestration: using the API Gateway as perspective. Similarly, when you are moving from monolith to
the orchestration layer or coding orchestration in a separate microservices, backend services may understand different
microservice. protocols (SOAP, REST, AMQP etc.).

You should avoid orchestration at the gateway layer. It The API Gateway provides a place for data transformation,
violates the single responsibility principle and, in the case of where messages can be translated between backend, API,
scaling the API, you will have to scale the gateway along with and app formats and protocols. The gateway provides a
orchestrated microservices. Some of the API Gateways have central data transformation point through which all traffic is
little to no capability for orchestration. translated for:

Although it is discouraged to use orchestration at the • Requested payload transformations

gateway layer, if you still want to use it for whatever reason, • Header transformations
then there should not be any business logic involved while • Protocol transformations
orchestrating.
Develop your transformation adapters as reusable
components. Also, do not try to code all different clients’
needs in a single API. You should think of creating client-
specific APIs with pluggable transformation logic.

Figure 6.
There are two options
for implementing
orchestration: using
the API Gateway as the
orchestration layer or
coding orchestration in
a separate microservice.

Microservice Architecture: API Gateway Considerations 9

 Monitoring

Monitoring
Being a single entry point into the system, all of the traffic Traffic and Data Monitoring
in and out of the system passes through the API Gateway,
so monitoring the gateway is critical. This provides an Analyzing the traffic data will help you to take proactive
opportunity to capture the information about data flow, which measures, to ensure smooth working of the software
becomes an input for IT administration and IT policies. systems and shape up the IT policies. You should consider
monitoring the following basic metrics:
Health Monitoring
• Number of requests per API
Health monitoring is done to make sure the gateway is up • Request categorization by criteria (for example, remote-
and running. For health monitoring, it is recommended to host)
capture: • Performance statistics
• Successful and exception messages
• System status and health (CPU, Memory, Thread usage) • Blocked messages breaching gateway policies
• Network connectivity
• Security alerts You should also regularly analyze the traffic over long range
• Backups and recovery period to be able to identify predictable traffic levels and be
• Maintenance of logs ready for any surge.

Microservice Architecture: API Gateway Considerations 10

 Load Balancing and Scaling

Load Balancing and

Scaling
Traffic and data analysis helps in understanding/estimating The distributed state poses a certain restriction in terms of
the load on the system. In turn, this helps in scaling the active/active and active/passive clustering. For example, in
gateway and underlying services accordingly. The gateway case the counter and cache state is important, the system
can scale both horizontally and vertically. An API Gateway should be designed to make sure at least one API Gateway
being load balanced runs the same configuration to virtualize is active at all times. This means that for a resilient high-
the same APIs, and executes the same policies. If multiple availability system, you should employ a minimum of at least
API Gateway groups are deployed, load balancing should be two active API gateways at any given time, extra in passive
across groups. mode.

The API Gateway does not impose any special requirements The API Gateway also ensures zero downtime by
on load balancers. Loads are balanced on a number of implementing configuration deployment in a rolling fashion.
characteristics including the response time or system load. This means, while each API Gateway instance in the cluster
The execution of API Gateway policies is stateless, and or group takes a few seconds to update its configuration, it
the route through which a message takes on a particular stops serving new requests, but all existing in-flight requests
system has no bearing on its processing. Some items like are honored. Meanwhile, the rest of the cluster or group
caches and counters, which are held on a distributed cache, can still receive new requests. The load balancer ensures
are updated on a per-message basis. This helps the API that requests are pushed to the nodes that are still receiving
Gateway to operate successfully in both sticky and non- requests.
sticky modes.

Figure 7.
Load balancing and
scaling

Microservice Architecture: API Gateway Considerations 11

 High Availability and Failover & Governance

High Availability and

Failover
Being a critical component and ONLY entry point in • Monitor the network infrastructure carefully, to identify
microservice architecture, you should deploy the API gateway any issues early on. You can do this using API Gateway
in High Availability (HA) mode. API Gateway instances are Manager and API Gateway Analytics. Interfaces are also
usually deployed behind standard load balancers, which provided to standard monitoring tools, such as syslog
periodically query the state of the API Gateway. If a problem and Simple Network Management Protocol (SNMP).
occurs, the load balancer redirects traffic to the hot stand-by
instance.

Events/ alerts are configured for getting notifications in case Governance

any issue occurs. If an event or alert is triggered, the issue
can be identified using API Gateway analytics and the active
API Gateway can then be repaired. As the number of APIs keep on increasing, it is essential
to establish policies and monitoring. The policies can
API Gateway instances are stateless by nature. No broadly be categorized as design-time governance and
session data is created, and therefore there is no need to runtime governance. The policies are highly influenced by IT
replicate the session state across API Gateways. However, (business) objectives and goals.
API Gateways can maintain cached data, which can be
replicated using a peer-to-peer relationship, across a cluster Select and configure API Gateway policies that help you
of API Gateways. implement runtime governance, such as:

High Availability can be maintained using either of these Tracking the life cycle of APIs:
options: Active/Active, Active/Standby, or Active/Active
Systems. These are described as follows: • Handling routing, blocking, and processing
• Understanding the API utilization and raising the alerts/
• Active/Standby: System is turned off alarms in case usage crosses the threshold
• Active/Passive: System is operational but not containing • Traffic throttling, smoothing, and load balancing
state • Rate limiting per-API usage
• Active/Active: System is fully operational and with current • API versioning
system state • Schema versioning for input/ output request parameters

HA and Failover Guidelines Design time governance includes:

• To achieve maximum availability, API Gateway should be • Defining the standards for API definitions (example:
used in Active/ Active for each production API Gateway Swagger)
• Keyword tags used to categorize APIs
• Proper traffic analysis, to limit traffic to backend services, • Conformance to REST API design guidelines
to protect against message flooding. This is particularly
important with legacy systems that have been recently
service-enabled. Legacy systems may not have been
designed for the traffic patterns to which they are now
subjected.

Microservice Architecture: API Gateway Considerations 12

 Conclusion & References

Conclusion
The API Gateway is the essential component of
microservices architecture. It helps to: 1741 Technology Dr.
San Jose, CA 95110
• Decouple consumers of the services from backend
services +1.408.273.8900
• Implement policies in one place info@globallogic.com
www.globallogic.com
• Achieve reusability
• Monitor the entire technology platform performance 2017 GlobalLogic, Inc.
• Enable easy scaling of services All Rights Reserved

References
• Pattern: API Gateway / Backend for Front-End
(Microservices.io)
• Building Microservices: Using an API Gateway (NGINX)
• Enabling Microservice Architecture with Middleware
(WSO2 Blog)
• Welcome to IdentityServer4 (IdentityServer4)
• Oracle® Fusion Middleware Part 1. API Gateway
administration (Oracle)

Microservice Architecture: API Gateway Considerations 13