Use AWS Secrets Manager secrets in Amazon Elastic Kubernetes Service

To show secrets from Secrets Manager as files mounted in Amazon EKS pods, you can use the AWS Secrets and Configuration Provider (ASCP) for the Kubernetes Secrets Store CSI Driver. The ASCP works with Amazon Elastic Kubernetes Service (Amazon EKS) 1.17+ running an Amazon EC2 node group. AWS Fargate node groups are not supported. With the ASCP, you can store and manage your secrets in Secrets Manager and then retrieve them through your workloads running on Amazon EKS. If your secret contains multiple key/value pairs in JSON format, you can choose which ones to mount in Amazon EKS. The ASCP uses JMESPath syntax to query the key/value pairs in your secret. The ASCP also works with Parameter Store parameters.

If you use a private Amazon EKS cluster, ensure that the VPC that the cluster is in has a Secrets Manager endpoint. The Secrets Store CSI Driver uses the endpoint to make calls to Secrets Manager. For information about creating an endpoint in a VPC, see VPC endpoint.

If you use Secrets Manager automatic rotation for your secrets, you can also use the Secrets Store CSI Driver rotation reconciler feature to ensure you are retrieving the latest secret from Secrets Manager. For more information, see Auto rotation of mounted contents and synced Kubernetes Secrets.

Step 1: Set up access control

The ASCP retrieves the Amazon EKS pod identity and exchanges it for an IAM role. You set permissions in an IAM policy for that IAM role. When the ASCP assumes the IAM role, it gets access to the secrets you authorized. Other containers can't access the secrets unless you also associate them with the IAM role.

If calls from the ASCP to look up the Region and IAM role associated with the pod are throttled by Kubernetes, you can change the throttling quotas using helm install, as shown in Step 2.

Step 2: Install and configure the ASCP

The ASCP is available on GitHub in the secrets-store-csi-provider-aws repository. The repo also contains example YAML files for creating and mounting a secret.

During installation, you can configure the ASCP to use a FIPS endpoint. For a list of endpoints, see AWS Secrets Manager endpoints.

Step 3: Identify which secrets to mount

To determine which secrets the ASCP mounts in Amazon EKS as files on the filesystem, you create a SecretProviderClass YAML file. The SecretProviderClass lists the secrets to mount and the file name to mount them as. The SecretProviderClass must be in the same namespace as the Amazon EKS pod it references.

The following examples show how to use SecretProviderClass to describe the secrets you want to mount and what to name the files mounted in the Amazon EKS pod.

Example: Mount secrets by name or ARN

The following example shows a SecretProviderClass that mounts three files in Amazon EKS:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
        - objectName: "arn:aws:secretsmanager:us-east-2:111122223333:secret:MySecret2-d4e5f6"
        - objectName: "MySecret3"
          objectType: "secretsmanager"
        - objectName: "MySecret4"
          objectType: "secretsmanager"
          objectVersionLabel: "AWSCURRENT"

Example: Mount key/value pairs from a secret

The following example shows a SecretProviderClass that mounts three files in Amazon EKS:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: "arn:aws:secretsmanager:us-east-2:111122223333:secret:MySecret-a1b2c3"
        jmesPath: 
            - path: username
              objectAlias: dbusername
            - path: password
              objectAlias: dbpassword

Example: Define a failover Region for a multi-Region secret

To provide availability during connectivity outages or for disaster recovery configurations, the ASCP supports an automated failover feature to retrieve secrets from a secondary region.

The following example shows a SecretProviderClass that retrieves a secret that is replicated to multiple Regions. In this example, the ASCP tries to retrieve the secret from both us-east-1 and us-east-2. If either Region returns a 4xx error, for example for an authentication issue, the ASCP does not mount either secret. If the secret is retrieved successfully from us-east-1, then the ASCP mounts that secret value. If the secret is not retrieved successfully from us-east-1, but it is retrieved successfully from us-east-2, then the ASCP mounts that secret value.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    region: us-east-1
    failoverRegion: us-east-2
    objects: |
      - objectName: "MySecret"

Example: Choose a failover secret to mount

The following example shows a SecretProviderClass that specifies which secret to mount in case of failover. The failover secret isn't a replica. In this example, the ASCP tries to retrieve the two secrets specified by objectName. If either returns a 4xx error, for example for an authentication issue, the ASCP does not mount either secret. If the secret is retrieved successfully from us-east-1, then the ASCP mounts that secret value. If the secret is not retrieved successfully from us-east-1, but it is retrieved successfully from us-east-2, then the ASCP mounts that secret value. The mounted file in Amazon EKS is named MyMountedSecret.

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-secrets
spec:
  provider: aws
  parameters:
    region: us-east-1
    failoverRegion: us-east-2
    objects: |
      - objectName: "arn:aws:secretsmanager:us-east-1:111122223333:secret:MySecret-a1b2c3"
        objectAlias: "MyMountedSecret"
        failoverObject: 
          - objectName: "arn:aws:secretsmanager:us-east-2:111122223333:secret:MyFailoverSecret-d4e5f6"

Step 4: Mount the secrets as files in the Amazon EKS pod

The following instructions show how to mount secrets as files using example YAML files ExampleSecretProviderClass.yaml and ExampleDeployment.yaml.

Troubleshoot

You can view most errors by describing the pod deployment.