Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Chickens & Eggs: Managing secrets in AWS with Hashicorp Vault

•
•
Jeff Horwitz
Jeff Horwitz

Presented to the Philly DevOps Meetup November 29, 2016. Managing secrets is hard. It’s even harder in the cloud. At Jornaya (formerly LeadiD), we chose Hashicorp Vault to manage our secrets in AWS, and I’d like to share our experience with everyone.

1 of 53
Chickens & Eggs Managing secrets in AWS with Hashicorp Vault Jeff Horwitz Jornaya jhorwitz@jornaya.com
Applications need secrets • Where do we store the secrets? • How do we manage the secrets? • How do servers/applications obtain the secrets?
Jornaya Environment • Applications run in AWS in Autoscaling groups • Configuration management via Chef server • No PCI, HIPAA or other compliance requirements • Secrets include passwords, private keys, API keys
Vault “Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing. Vault handles leasing, key revocation, key rolling, and auditing. Through a unified API, users can access an encrypted Key/Value store and network encryption-as-a-service, or generate AWS IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH credentials, and more.” https://www.vaultproject.io/
Vault ✓ Platform independent ✓ High availability ✓ User & server authentication options ✓ Fine-grained access control ✓ Good language and tooling support ✓ Under active development
Other Options • Instance userdata • Chef encrypted databags • Chef vault • Citadel • KMS
Vault Server • Responds to client requests • Interacts with backends • storage, authentication, secret, audit • Encrypts/Decrypts secrets with master key • Master key is never stored on disk
Playing with Vault ~$ vault server --dev ==> WARNING: Dev mode is enabled! In this mode, Vault is completely in-memory and unsealed. Vault is configured to only have a single unseal key. The root token has already been authenticated with the CLI, so you can immediately begin using the Vault CLI. The only step you need to take is to set the following environment variables: export VAULT_ADDR='http://127.0.0.1:8200' The unseal key and root token are reproduced below in case you want to seal/unseal the Vault or play with authentication. Unseal Key: e95cf9d02c044c67dfd5d379d3a56cee7a0209f8cc681fca1435b6c022fcf028 Root Token: 611f6670-f871-ef2d-2110-87a54261d407
Talking to Vault • Vault provides an HTTP(S) RESTful API • JSON responses • The vault command is a user-friendly wrapper • Modules available for various languages • Config management (e.g. Chef via Ruby gem)
Reading & Writing Secrets ~$ export VAULT_ADDR=http://127.0.0.1:8200 ~$ vault write secret/phillydevops/luggage-combo value=12345 Success! Data written to: secret/phillydevops/luggage-combo ~$ vault read secret/phillydevops/luggage-combo Key Value --- ----- refresh_interval 720h0m0s value 12345
HTTP API ~$ curl -X POST -d '{"value": "12345"}' 
 -H 'X-Vault-Token: 611f6670-f871-ef2d-2110-87a54261d407'
 http://127.0.0.1:8200/v1/secret/phillydevops/luggage-combo
HTTP API ~$ curl -k -H 'X-Vault-Token: 611f6670-f871-ef2d-2110-87a54261d407' 
 http://127.0.0.1:8200/v1/secret/phillydevops/luggage-combo | jq { "request_id": "ac71da58-cc11-431b-598e-7eb3fe3e37a3", "lease_id": "", "renewable": false, "lease_duration": 2592000, "data": { "value": "12345" }, "wrap_info": null, "warnings": null, "auth": null }
Shamir’s Secret Sharing • https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing • Splits a key into n shards (unseal keys) • k shards needed to derive original key (k < n) • No one person can easily obtain the original key • No need to revoke shards if people leave
Unsealing Process
Unsealing Process • Vault server does not store the master key on disk • Given the master key via the unsealing process • Need minimum number of shards to unseal • Key is only stored in memory • Restarting will lose the key and “seal” the vault.
Unsealing Demo ~$ vault read secret/phillydevops/luggage-combo Error reading secret/phillydevops/luggage-combo: Error making API request. URL: GET http://127.0.0.1:8200/v1/secret/phillydevops/luggage-combo Code: 503. Errors: * Vault is sealed ~$ vault unseal Key (will be hidden): ******** Sealed: false Key Shares: 1 Key Threshold: 1 Unseal Progress: 0
Backends Storage backends Secret backends Authentication backends Audit backends
Storage Backends • Physical storage for secrets • Only see encrypted secrets • High availability features
Storage Backends Backend Support HA? Consul Official Yes Zookeeper Community Yes etcd Community Yes DynamoDB Community Maybe S3 Community No Swift Community No Azure Community No mysql Community No postgresql Community No inmem Official No file Official No
Storage Backends Backend Support HA? Consul Official Yes Zookeeper Community Yes etcd Community Yes DynamoDB Community Maybe S3 Community No Swift Community No Azure Community No mysql Community No postgresql Community No inmem Official No file Official No
Storage Backends Backend Support HA? Consul Official Yes Zookeeper Community Yes etcd Community Yes DynamoDB Community Maybe S3 Community No Swift Community No Azure Community No mysql Community No postgresql Community No inmem Official No file Official No
Clustering • Storage backend must support high availability • Active and standby servers • State maintained by Consul (storage backend) • Use Consul DNS or API to discover active server • Standby will redirect to active by default
Cluster Status $ vault status Sealed: false Key Shares: 5 Key Threshold: 3 Unseal Progress: 0 Version: Vault v0.6.1 Cluster Name: vault-cluster-abcd0123 Cluster ID: deadbeef-beef-beef-beef-deadbeefcafe High-Availability Enabled: true Mode: active Leader: https://vault-1.example.com:8200
Secret Backends • Static or dynamic secrets • Backend configuration • Multiple backend mountpoints
Generic Backend • Default backend • Mounted at secret • Stores and retrieves static secrets as k/v pairs • Great for: • username/password • RSA private keys • API keys
Tokens • Authentication in vault is performed via tokens • UUIDs returned by authentication backends • Tokens can expire and must be renewed • Tokens can be revoked • A single “root” token never expires
Authentication Backends • Returns a token based on credentials • Tokens may have a TTL (backend-dependent) • Options to fit different workflows and platforms
Authentication Backends • AppRole (App ID) • AWS EC2 • GitHub • LDAP • TLS Certificates • Tokens • Username & Password
LDAP Backend ~$ vault auth -method ldap username=jhorwitz Password (will be hidden): Successfully authenticated! You are now logged in. The token below is already saved in the session. You do not need to "vault auth" again with the token. token: deadbeef-cafe-beef-beef-deadbeafcafe token_duration: 28799 token_policies: [admins, default]
Lookup a Token ~$ vault token-lookup deadbeef-cafe-beef-beef-deadbeafcafe Key Value --- ----- accessor 5776eb4b-05b1-3bab-98d3-08d34040a806 creation_time 1480366497 creation_ttl 28800 display_name ldap-jhorwitz explicit_max_ttl 0 id deadbeef-cafe-beef-beef-deadbeafcafe meta map[policies:admins,default username:jhorwitz] num_uses 0 orphan true path auth/ldap/login/jhorwitz policies [admins default] renewable true ttl 27379
aws-ec2 Backend • Good for server-level authentication • Fixes the chicken & egg problem on EC2 instances • Let AWS do the dirty work for you
Instance Identity $ curl http://169.254.169.254/latest/dynamic/instance-identity/document { "privateIp" : “10.0.0.1”, "devpayProductCodes" : null, "availabilityZone" : "us-east-1a", "accountId" : "1234567890", "version" : "2010-08-31", "instanceId" : "i-deadbeef", "billingProducts" : null, "instanceType" : "t2.nano", "pendingTime" : "2016-09-09T17:30:35Z", "architecture" : "x86_64", "imageId" : "ami-cafebeef", "kernelId" : null, "ramdiskId" : null, "region" : "us-east-1" }
Identity Signature $ curl http://169.254.169.254/latest/dynamic/instance-identity/pkcs7 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAaCAJIAEggGjewog ICJwcml2YXRlSXAiIDogIjEwLjUwLjAuNjgiLAogICJkZXZwYXlQcm9kdWN0Q29kZXMiIDogbnVs bCwKICAiYXZhaWxhYmlsaXR5Wm9uZSIgOiAidXMtZWFzdC0xYSIsCiAgImFjY291bnRJZCIgOiAi Mjk4Nzg1NDUzNTkwIiwKICAidmVyc2lvbiIgOiAiMjAxMC0wOC0zMSIsCiAgImluc3RhbmNlSWQi IDogImktYzRkMDQwZjUiLAogICJiaWxsaW5nUHJvZHVjdHMiIDogbnVsbCwKICAiaW5zdGFuY2VU …
Whitelisting • Vault maintains a whitelist of instance IDs • Cannot reauthenticate using an ID in the whitelist • You can turn this off or remove ID from whitelist • Use a nonce to prevent replay attacks
Roles • Clients specify a role with authentication requests • Roles are “bound” to instance properties • IAM role or instance profile • AMI ID • Vault cross-checks with EC2 before returning token
Creating a Role $ vault write auth/aws-ec2/role/deployer 
 bound_iam_role_arn=arn:aws:iam::1234567890:instance-profile/deployer policies=deployer
 Success! Data written to: auth/aws-ec2/role/deployer
 
 

Policies • ACLs that are applied to roles, users and groups • Applied at token creation • Tokens can have multiple policies
Sample User Policy path "secret/test/*" { policy = "write" } path "secret/*" { policy = "read" } path "secret/classified/*" { capabilities = ["deny"] }
Sample Server Policy path "secret/env/prod/*" { policy = "read" } path "secret/shared/*" { policy = "read" } path "secret/ssl/private-key" { policy = "read" }
Integration • Chef authenticates instance to aws-ec2 backend • Chef stores token • local filesystem (for scripts and apps) • node run state (for Chef recipes)
Get PKCS7 Signature def instance_identity_pkcs7 uri = URI.parse('http://169.254.169.254/latest/dynamic/instance-identity/pkcs7') resp = Net::HTTP.get_response(uri) resp.body.delete("n", '') end
Generate Nonce nonce = SecureRandom.uuid
Send Login Request def aws_ec2_login(address, role, pkcs7, nonce) # find the leader, since Net::HTTP doesn't handle redirects uri = URI.parse("#{address}/v1/sys/leader") resp = Net::HTTP.get_response(uri) leader = JSON.parse(resp.body)['leader_address'] uri = URI.parse("#{leader}/v1/auth/aws-ec2/login") data = { 'role' => role, 'pkcs7' => pkcs7, 'nonce' => nonce } req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json') req.body = data.to_json resp = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http| http.request(req) end …
Write Token to File file node['leadid_vault']['client']['token_file'] do content lazy { node.run_state['token'] || node['leadid_vault']['client']['token'] } owner node['leadid_vault']['client']['uid'] group node['leadid_vault']['client']['gid'] mode '0440' sensitive true action :create_if_missing only_if { node['leadid_vault']['client']['enabled'] } end.run_action(:create)
Using the Token • Chef recipes use the vault gem • Other apps use consul-template
Configure Vault Chef gem def configure(opts = {}) require 'vault' Vault.configure do |config| config.address = opts[:address] config.ssl_ca_cert = opts[:ssl_ca_cert] config.ssl_verify = opts[:ssl_verify] if opts[:token] config.token = opts[:token] elsif opts[:token_file] config.token = File.read(opts[:token_file]) else raise 'must specify either token or token_file' end end end
Helper to read secrets def read(path) require 'vault' Vault.logical.read(path) end
Reading secrets in Chef Chef::Recipe.send(:include, VaultHelpers) if node['leadid_vault']['client']['enabled'] # Grab the Datadog API and Application Key from vault api_key = read( "#{node['leadid_base']['datadog']['secret_path']}" + "/datadog/api-key").data node.default['datadog']['api_key'] = api_key[:value] end … include_recipe 'datadog::dd-agent' include_recipe 'datadog::dd-handler'
consul-template $ consul-template -consul consul.example.com -template "/opt/jornaya/app.yml.ctmpl:/opt/jornaya/app.yml:service nginx restart"
consul-template database: host: db.example.com dbname: myapp {{with secret "secret/leadid/env/prod/db-user"}} user: {{.Data.username}} password: {{.Data.password}} {{end}}
consul-template database: host: db.example.com dbname: myapp user: donald password: WeHaveTheBestPasswords
Learnings • Don't use a self-signed certificate. There is pain. • For SSL, name your servers or use static IP addresses. • Clustering was the easiest part. Do it. • Set up your path hierarchy ahead of time. • Be prepared to RTFM. • Different backends behave differently. • Tokens do not live forever.
Questions?

More Related Content

Chickens & Eggs: Managing secrets in AWS with Hashicorp Vault

  • 1. Chickens & Eggs Managing secrets in AWS with Hashicorp Vault Jeff Horwitz Jornaya jhorwitz@jornaya.com
  • 2. Applications need secrets • Where do we store the secrets? • How do we manage the secrets? • How do servers/applications obtain the secrets?
  • 3. Jornaya Environment • Applications run in AWS in Autoscaling groups • Configuration management via Chef server • No PCI, HIPAA or other compliance requirements • Secrets include passwords, private keys, API keys
  • 4. Vault “Vault secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing. Vault handles leasing, key revocation, key rolling, and auditing. Through a unified API, users can access an encrypted Key/Value store and network encryption-as-a-service, or generate AWS IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH credentials, and more.” https://www.vaultproject.io/
  • 5. Vault ✓ Platform independent ✓ High availability ✓ User & server authentication options ✓ Fine-grained access control ✓ Good language and tooling support ✓ Under active development
  • 6. Other Options • Instance userdata • Chef encrypted databags • Chef vault • Citadel • KMS
  • 7. Vault Server • Responds to client requests • Interacts with backends • storage, authentication, secret, audit • Encrypts/Decrypts secrets with master key • Master key is never stored on disk
  • 8. Playing with Vault ~$ vault server --dev ==> WARNING: Dev mode is enabled! In this mode, Vault is completely in-memory and unsealed. Vault is configured to only have a single unseal key. The root token has already been authenticated with the CLI, so you can immediately begin using the Vault CLI. The only step you need to take is to set the following environment variables: export VAULT_ADDR='http://127.0.0.1:8200' The unseal key and root token are reproduced below in case you want to seal/unseal the Vault or play with authentication. Unseal Key: e95cf9d02c044c67dfd5d379d3a56cee7a0209f8cc681fca1435b6c022fcf028 Root Token: 611f6670-f871-ef2d-2110-87a54261d407
  • 9. Talking to Vault • Vault provides an HTTP(S) RESTful API • JSON responses • The vault command is a user-friendly wrapper • Modules available for various languages • Config management (e.g. Chef via Ruby gem)
  • 10. Reading & Writing Secrets ~$ export VAULT_ADDR=http://127.0.0.1:8200 ~$ vault write secret/phillydevops/luggage-combo value=12345 Success! Data written to: secret/phillydevops/luggage-combo ~$ vault read secret/phillydevops/luggage-combo Key Value --- ----- refresh_interval 720h0m0s value 12345
  • 11. HTTP API ~$ curl -X POST -d '{"value": "12345"}' 
 -H 'X-Vault-Token: 611f6670-f871-ef2d-2110-87a54261d407'
 http://127.0.0.1:8200/v1/secret/phillydevops/luggage-combo
  • 12. HTTP API ~$ curl -k -H 'X-Vault-Token: 611f6670-f871-ef2d-2110-87a54261d407' 
 http://127.0.0.1:8200/v1/secret/phillydevops/luggage-combo | jq { "request_id": "ac71da58-cc11-431b-598e-7eb3fe3e37a3", "lease_id": "", "renewable": false, "lease_duration": 2592000, "data": { "value": "12345" }, "wrap_info": null, "warnings": null, "auth": null }
  • 13. Shamir’s Secret Sharing • https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing • Splits a key into n shards (unseal keys) • k shards needed to derive original key (k < n) • No one person can easily obtain the original key • No need to revoke shards if people leave
  • 15. Unsealing Process • Vault server does not store the master key on disk • Given the master key via the unsealing process • Need minimum number of shards to unseal • Key is only stored in memory • Restarting will lose the key and “seal” the vault.
  • 16. Unsealing Demo ~$ vault read secret/phillydevops/luggage-combo Error reading secret/phillydevops/luggage-combo: Error making API request. URL: GET http://127.0.0.1:8200/v1/secret/phillydevops/luggage-combo Code: 503. Errors: * Vault is sealed ~$ vault unseal Key (will be hidden): ******** Sealed: false Key Shares: 1 Key Threshold: 1 Unseal Progress: 0
  • 18. Storage Backends • Physical storage for secrets • Only see encrypted secrets • High availability features
  • 19. Storage Backends Backend Support HA? Consul Official Yes Zookeeper Community Yes etcd Community Yes DynamoDB Community Maybe S3 Community No Swift Community No Azure Community No mysql Community No postgresql Community No inmem Official No file Official No
  • 20. Storage Backends Backend Support HA? Consul Official Yes Zookeeper Community Yes etcd Community Yes DynamoDB Community Maybe S3 Community No Swift Community No Azure Community No mysql Community No postgresql Community No inmem Official No file Official No
  • 21. Storage Backends Backend Support HA? Consul Official Yes Zookeeper Community Yes etcd Community Yes DynamoDB Community Maybe S3 Community No Swift Community No Azure Community No mysql Community No postgresql Community No inmem Official No file Official No
  • 22. Clustering • Storage backend must support high availability • Active and standby servers • State maintained by Consul (storage backend) • Use Consul DNS or API to discover active server • Standby will redirect to active by default
  • 23. Cluster Status $ vault status Sealed: false Key Shares: 5 Key Threshold: 3 Unseal Progress: 0 Version: Vault v0.6.1 Cluster Name: vault-cluster-abcd0123 Cluster ID: deadbeef-beef-beef-beef-deadbeefcafe High-Availability Enabled: true Mode: active Leader: https://vault-1.example.com:8200
  • 24. Secret Backends • Static or dynamic secrets • Backend configuration • Multiple backend mountpoints
  • 25. Generic Backend • Default backend • Mounted at secret • Stores and retrieves static secrets as k/v pairs • Great for: • username/password • RSA private keys • API keys
  • 26. Tokens • Authentication in vault is performed via tokens • UUIDs returned by authentication backends • Tokens can expire and must be renewed • Tokens can be revoked • A single “root” token never expires
  • 27. Authentication Backends • Returns a token based on credentials • Tokens may have a TTL (backend-dependent) • Options to fit different workflows and platforms
  • 28. Authentication Backends • AppRole (App ID) • AWS EC2 • GitHub • LDAP • TLS Certificates • Tokens • Username & Password
  • 29. LDAP Backend ~$ vault auth -method ldap username=jhorwitz Password (will be hidden): Successfully authenticated! You are now logged in. The token below is already saved in the session. You do not need to "vault auth" again with the token. token: deadbeef-cafe-beef-beef-deadbeafcafe token_duration: 28799 token_policies: [admins, default]
  • 30. Lookup a Token ~$ vault token-lookup deadbeef-cafe-beef-beef-deadbeafcafe Key Value --- ----- accessor 5776eb4b-05b1-3bab-98d3-08d34040a806 creation_time 1480366497 creation_ttl 28800 display_name ldap-jhorwitz explicit_max_ttl 0 id deadbeef-cafe-beef-beef-deadbeafcafe meta map[policies:admins,default username:jhorwitz] num_uses 0 orphan true path auth/ldap/login/jhorwitz policies [admins default] renewable true ttl 27379
  • 31. aws-ec2 Backend • Good for server-level authentication • Fixes the chicken & egg problem on EC2 instances • Let AWS do the dirty work for you
  • 32. Instance Identity $ curl http://169.254.169.254/latest/dynamic/instance-identity/document { "privateIp" : “10.0.0.1”, "devpayProductCodes" : null, "availabilityZone" : "us-east-1a", "accountId" : "1234567890", "version" : "2010-08-31", "instanceId" : "i-deadbeef", "billingProducts" : null, "instanceType" : "t2.nano", "pendingTime" : "2016-09-09T17:30:35Z", "architecture" : "x86_64", "imageId" : "ami-cafebeef", "kernelId" : null, "ramdiskId" : null, "region" : "us-east-1" }
  • 33. Identity Signature $ curl http://169.254.169.254/latest/dynamic/instance-identity/pkcs7 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAaCAJIAEggGjewog ICJwcml2YXRlSXAiIDogIjEwLjUwLjAuNjgiLAogICJkZXZwYXlQcm9kdWN0Q29kZXMiIDogbnVs bCwKICAiYXZhaWxhYmlsaXR5Wm9uZSIgOiAidXMtZWFzdC0xYSIsCiAgImFjY291bnRJZCIgOiAi Mjk4Nzg1NDUzNTkwIiwKICAidmVyc2lvbiIgOiAiMjAxMC0wOC0zMSIsCiAgImluc3RhbmNlSWQi IDogImktYzRkMDQwZjUiLAogICJiaWxsaW5nUHJvZHVjdHMiIDogbnVsbCwKICAiaW5zdGFuY2VU …
  • 34. Whitelisting • Vault maintains a whitelist of instance IDs • Cannot reauthenticate using an ID in the whitelist • You can turn this off or remove ID from whitelist • Use a nonce to prevent replay attacks
  • 35. Roles • Clients specify a role with authentication requests • Roles are “bound” to instance properties • IAM role or instance profile • AMI ID • Vault cross-checks with EC2 before returning token
  • 36. Creating a Role $ vault write auth/aws-ec2/role/deployer 
 bound_iam_role_arn=arn:aws:iam::1234567890:instance-profile/deployer policies=deployer
 Success! Data written to: auth/aws-ec2/role/deployer
 
 

  • 37. Policies • ACLs that are applied to roles, users and groups • Applied at token creation • Tokens can have multiple policies
  • 38. Sample User Policy path "secret/test/*" { policy = "write" } path "secret/*" { policy = "read" } path "secret/classified/*" { capabilities = ["deny"] }
  • 39. Sample Server Policy path "secret/env/prod/*" { policy = "read" } path "secret/shared/*" { policy = "read" } path "secret/ssl/private-key" { policy = "read" }
  • 40. Integration • Chef authenticates instance to aws-ec2 backend • Chef stores token • local filesystem (for scripts and apps) • node run state (for Chef recipes)
  • 41. Get PKCS7 Signature def instance_identity_pkcs7 uri = URI.parse('http://169.254.169.254/latest/dynamic/instance-identity/pkcs7') resp = Net::HTTP.get_response(uri) resp.body.delete("n", '') end
  • 42. Generate Nonce nonce = SecureRandom.uuid
  • 43. Send Login Request def aws_ec2_login(address, role, pkcs7, nonce) # find the leader, since Net::HTTP doesn't handle redirects uri = URI.parse("#{address}/v1/sys/leader") resp = Net::HTTP.get_response(uri) leader = JSON.parse(resp.body)['leader_address'] uri = URI.parse("#{leader}/v1/auth/aws-ec2/login") data = { 'role' => role, 'pkcs7' => pkcs7, 'nonce' => nonce } req = Net::HTTP::Post.new(uri, 'Content-Type' => 'application/json') req.body = data.to_json resp = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http| http.request(req) end …
  • 44. Write Token to File file node['leadid_vault']['client']['token_file'] do content lazy { node.run_state['token'] || node['leadid_vault']['client']['token'] } owner node['leadid_vault']['client']['uid'] group node['leadid_vault']['client']['gid'] mode '0440' sensitive true action :create_if_missing only_if { node['leadid_vault']['client']['enabled'] } end.run_action(:create)
  • 45. Using the Token • Chef recipes use the vault gem • Other apps use consul-template
  • 46. Configure Vault Chef gem def configure(opts = {}) require 'vault' Vault.configure do |config| config.address = opts[:address] config.ssl_ca_cert = opts[:ssl_ca_cert] config.ssl_verify = opts[:ssl_verify] if opts[:token] config.token = opts[:token] elsif opts[:token_file] config.token = File.read(opts[:token_file]) else raise 'must specify either token or token_file' end end end
  • 47. Helper to read secrets def read(path) require 'vault' Vault.logical.read(path) end
  • 48. Reading secrets in Chef Chef::Recipe.send(:include, VaultHelpers) if node['leadid_vault']['client']['enabled'] # Grab the Datadog API and Application Key from vault api_key = read( "#{node['leadid_base']['datadog']['secret_path']}" + "/datadog/api-key").data node.default['datadog']['api_key'] = api_key[:value] end … include_recipe 'datadog::dd-agent' include_recipe 'datadog::dd-handler'
  • 49. consul-template $ consul-template -consul consul.example.com -template "/opt/jornaya/app.yml.ctmpl:/opt/jornaya/app.yml:service nginx restart"
  • 50. consul-template database: host: db.example.com dbname: myapp {{with secret "secret/leadid/env/prod/db-user"}} user: {{.Data.username}} password: {{.Data.password}} {{end}}
  • 52. Learnings • Don't use a self-signed certificate. There is pain. • For SSL, name your servers or use static IP addresses. • Clustering was the easiest part. Do it. • Set up your path hierarchy ahead of time. • Be prepared to RTFM. • Different backends behave differently. • Tokens do not live forever.