Dynamic Secrets - DB

Vault can generate secrets on-demand for some systems. For example, when an app needs to access an Amazon S3 bucket, it asks Vault for AWS credentials. Vault will generate an AWS credential granting permissions to access the S3 bucket. In addition, Vault will automatically revoke this credential after the time-to-live (TTL) expires.

Challenge

Data protection is a top priority, and database credential rotation is a critical part of any data protection initiative. Each role has a different set of permissions granted to access the database. When a system is attacked by hackers, continuous credential rotation becomes necessary and needs to be automated.

Solution

Applications ask Vault for database credentials rather than setting them as environment variables. The administrator specifies the TTL of the database credentials to enforce its validity so that they are automatically revoked when they are no longer used.

Each app instance can get unique credentials that they don't have to share. By making those credentials short-lived, you reduce the chance that they might be compromised. If an app was compromised, the credentials used by the app can be revoked rather than changing more global sets of credentials.

Personas

The end-to-end scenario described in this tutorial involves two personas:

admin with privileged permissions to configure secrets engines
apps read the secrets from Vault

Scenario Introduction

In this tutorial, you are going to configure the PostgreSQL secrets engine, and create a read-only database role. The Vault-generated PostgreSQL credentials will only have read permission.

Enable the database secrets engine
Configure PostgreSQL secrets engine
Create a role
Request PostgreSQL credentials
Manage leases
Define a password policy
Define a username template

Prerequisites

To perform the tasks described in this tutorial, you need to have:

A Vault environment of version 1.2 or later. Refer to the Getting Started tutorial to install Vault. Make sure that your Vault server has been initialized and unsealed.
Docker

Policy requirements

Each persona requires a different set of capabilities. These are expressed in policies. If you are not familiar with policies, complete the policies tutorial.

1.The admin tasks require these capabilities.

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the database secrets engine and create roles
path "database/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage the leases
path "sys/leases/+/database/creds/readonly/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

path "sys/leases/+/database/creds/readonly" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

2. The apps tasks require these capabilities.

# Get credentials from the database secrets engine 'readonly' role.
path "database/creds/readonly" {
  capabilities = [ "read" ]
}

Start Postgres

The tutorial requires a Postgres database. Docker provides a Postgres server image that satisfies this requirement.

NOTE: This tutorial works for an existing Postgres database given appropriate credentials and connection information.

1.Pull a Postgres server image with docker.

docker pull postgres:latest

2. Create a Postgres database with a root user named root with the password rootpassword.

vaultonenode_default is the Network Name, if you dont have this network name in the system, then container may not start

docker run \
    --detach \
    --network=vaultonenode_default \
    --name learn-postgres \
    -e POSTGRES_USER=root \
    -e POSTGRES_PASSWORD=rootpassword \
    -p 5432:5432 \
    --rm \
    postgres

The database is available.

3. Verify that the postgres container is running.

docker ps -f name=learn-postgres --format "table {{.Names}}\t{{.Status}}"

4. The below command will provide you the IP Address of the Vault Server

docker inspect -f '{{.Name}} - {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $(docker ps -aq)

The credentials generated by the Vault role in the Create a role step requires a role named ro that has been granted the ability to read all tables.

With the Database Container is ready, create the role.

The credentials generated by the Vault role in the Create a role step requires a role named ro that has been granted the ability to read all tables.

5. Create a role named ro.

docker exec -i \
    learn-postgres \
    psql -U root -c "CREATE ROLE \"ro\" NOINHERIT;"

6. Grant the ability to read all tables to the role named ro.

docker exec -i \
    learn-postgres \
    psql -U root -c "GRANT SELECT ON ALL TABLES IN SCHEMA public TO \"ro\";"

The role is created and assigned the appropriate permissions.

Step1: Enable the database secrets engine

(Persona: admin)

The database secrets engine generates database credentials dynamically based on configured roles.

1.Enable the database secrets engine at the database/ path.

vault secrets enable database

The database secrets engine is enabled.

Step2: Configure PostgreSQL secrets engine

(Persona: admin)

The database secrets engine supports many databases through a plugin interface. To use a Postgres database with the secrets engine requires further configuration with the postgresql-database-plugin plugin and connection information.

NOTE: This task uses the connection information defined in the Start Postgres step.

1.Configure the database secrets engine with the connection credentials for the Postgres database.

replace IPADDRESSOFPOSTGRESS with your Postgres Server IP Address

vault write database/config/postgresql \
     plugin_name=postgresql-database-plugin \
     connection_url="postgresql://{{username}}:{{password}}@IPADDRESSOFPOSTGRES:5432/postgres?sslmode=disable" \
     allowed_roles=readonly \
     username="root" \
     password="rootpassword"

The secrets engine is configured to work with Postgres.

Step3: Create a role

(Persona: admin)

In configure Postgresql secrets engine step, you configured the PostgreSQL secrets engine with the allowed role named readonly. A role is a logical name within Vault that maps to database credentials. These credentials are expressed as SQL statements and assigned to the Vault role.

1.Define the SQL used to create credentials.

tee readonly.sql <<EOF
CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}' INHERIT;
GRANT ro TO "{{name}}";
EOF

The SQL contains the templatized fields {{name}}, {{password}}, and {{expiration}}. These values are provided by Vault when the credentials are created. This creates a new role and then grants that role the permissions defined in the Postgres role named ro. This Postgres role was created when Postgres was started.

2. Create the role named readonly that creates credentials with the readonly.sql.

vault write database/roles/readonly \
      db_name=postgresql \
      [email protected] \
      default_ttl=1h \
      max_ttl=24h

The role generates database credentials with a default TTL of 1 hour and max TTL of 24 hours.

Step4: Request PostgreSQL credentials

(Persona: apps)

The applications that require the database credentials read them from the secret engine's readonly role.

1.Read credentials from the readonly database role.

vault read database/creds/readonly

The Postgres credentials are displayed as username and password. The credentials are identified within Vault by the lease_id.

Validation

2. Connect to the Postgres database and list all database users.

docker exec -i \
    learn-postgres \
    psql -U root -c "SELECT usename, valuntil FROM pg_user;"

The output displays a table of all the database credentials generated. The credentials that were recently generated appear in this list.

Step5: Manage leases

(Persona: admin)

The credentials are managed by the lease ID and remain valid for the lease duration (TTL) or until revoked. Once revoked the credentials are no longer valid.

1.List the existing leases.

vault list sys/leases/lookup/database/creds/readonly

All valid leases for database credentials are displayed.

2. Create a variable that stores the first lease ID.

LEASE_ID=$(vault list -format=json sys/leases/lookup/database/creds/readonly | jq -r ".[0]")

3. Renew the lease for the database credential by passing its lease ID.

vault lease renew database/creds/readonly/$LEASE_ID

The TTL of the renewed lease is set to 1h.

4. Revoke the lease without waiting for its expiration.

vault lease revoke database/creds/readonly/$LEASE_ID

5. List the existing leases.

vault list sys/leases/lookup/database/creds/readonly

The lease is no longer valid and is not displayed.

6. Read new credentials from the readonly database role.

vault read database/creds/readonly

All leases associated with a path may be removed.

7. Revoke all the leases with the prefix database/creds/readonly.

vault lease revoke -prefix database/creds/readonly

The prefix flag matches all valid leases with the path prefix of database/creds/readonly.

8. List the existing leases.

vault list sys/leases/lookup/database/creds/readonly

All the leases with this path as a prefix have been revoked.

Step6: Define a password policy

To perform the steps in this section, you need Vault 1.6 or later. Refer to the password policy tutorial for more details.

The database secret engines generate passwords that adhere to a default pattern that may be overridden with a new password policy. A policy defines the rules and requirements that the password must adhere to and can provide that password directly through a new endpoint or within secrets engines.

The passwords you want to generate adhere to these requirements.

length of 20 characters
at least 1 uppercase character
at least 1 lowercase character
at least 1 number
at least 1 symbol

1.Define this password policy in a file named example_policy.hcl.

tee example_policy.hcl <<EOF
length=20

rule "charset" {
  charset = "abcdefghijklmnopqrstuvwxyz"
  min-chars = 1
}

rule "charset" {
  charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
  min-chars = 1
}

rule "charset" {
  charset = "0123456789"
  min-chars = 1
}

rule "charset" {
  charset = "!@#$%^&*"
  min-chars = 1
}
EOF

The policy is written in HashiCorp Configuration Language (HCL). The length field sets the length of the password returned to 20 characters. Each rule stanza defines a character set and the minimum number of occurrences those characters need to appear in the generated password. These rules are cumulative so each one adds more requirements on the password generated.

2. Create a Vault password policy named example with the password policy rules defined in example_policy.hcl.

vault write sys/policies/password/example policy=@example_policy.hcl

This policy can now be accessed directly to generate a password or referenced by its name example when configuring supported secrets engines.

3. Generate a password from the example password policy.

vault read sys/policies/password/example/generate

The password generated adheres to the defined requirements.

Apply the password policy

4. Configure the database secrets engine with the example password policy.

vault write database/config/postgresql \
     password_policy="example"

The same connection information is used to establish the connection with the database server. The difference is that the password_policy has been set to the example policy.

5. Read credentials from the readonly database role.

vault read database/creds/readonly

The credentials display the username and password generated. The password generated adheres to the example password policy defined in the secrets engine's configuration.

Step7: Define a username template

To perform the steps in this section, you need Vault 1.7 or later.

The database secret engines generate usernames that adhere to a default pattern. A customized username template may be provided to meet the needs of your organization.

Ensure that custom username templates include enough randomness to prevent the same username being generated multiple times.

1.Read credentials from the readonly database role.

vault read database/creds/readonly

The generated username, v-token-readonly-wGLPkpDyc6AgqBfMZTD3-1604195404, uses the default pattern expressed as a Go template, {{ printf "v-%s-%s-%s-%s" (.DisplayName | truncate 8) (.RoleName | truncate 8) (random 20) (unix_time) | truncate 63 }}.

Refer to the Username Templating documentation to learn more functions that can be applied.

2. Configure the database secrets engine with the username template.

vault write database/config/postgresql \
    username_template="myorg-{{.RoleName}}-{{unix_time}}-{{random 8}}"

This username template is prefixed with myorg-, uses the name of role, readonly, the unix timestamp in seconds, and a random sequence of 8 characters.

3. Read credentials from the readonly database role.

vault read database/creds/readonly

The username generated adheres to the template provided to the configuration.

Dynamic Secrets - DB

Challenge​

Solution​

Personas​

Scenario Introduction​

Prerequisites​

Policy requirements​

Start Postgres​

Step1: Enable the database secrets engine​

Step2: Configure PostgreSQL secrets engine​

Step3: Create a role​

Step4: Request PostgreSQL credentials​

Validation​

Step5: Manage leases​

Step6: Define a password policy​

Apply the password policy​

Step7: Define a username template​

Challenge

Solution

Personas

Scenario Introduction

Prerequisites

Policy requirements

Start Postgres

Step1: Enable the database secrets engine

Step2: Configure PostgreSQL secrets engine

Step3: Create a role

Step4: Request PostgreSQL credentials

Validation

Step5: Manage leases

Step6: Define a password policy

Apply the password policy

Step7: Define a username template