Running aws-vault in macOS with local Docker containers • Flailing Wildly

Overview

aws-vault is a tool for storing your AWS credentials in your system keychain instead of as a plain text file on-disk.

Credentials and other secrets (including your various system passwords) are stored inside your system keychain. They are encrypted, and cannot easily be stolen by a rogue script or application. By keeping your AWS credentials in your system keychain, they are available to you when you are logged in, unavailable when you are logged out, and provide an important layer of security that the standard plain text storage method does not.

It is designed to work cooperatively with the AWS Unified CLI Tools. It also provides utilities for other AWS best practices such as being able to generate session tokens, or logging into the AWS Console with your IAM credentials using a simple command.

You can learn more about the thinking behind it from the original 99 designs blog post.

Leveraging the Keychain

By default, every Mac user has a system and a login keychain that stores the bulk of your secure information (e.g., certificate authorities which enable SSL/TLS connections, website passwords or credit cards saved in your browser).

On macOS, credentials are stored in a non-login keychain in Keychain Access.app. Instead, they are stored in a new aws-vault keychain. In order to manage these credentials with the Keychain Access.app app, you’ll need to import it.

File → Import Items…
Choose aws-vault.keychain-db from the default directory.
Right click → Change Settings for Keychain “aws-vault”…
Change the value for Lock after {NUMBER} minutes of inactivity to something like 1440 minutes (1 day). Feel free to tune for security/convenience according to your tastes.

AWS Config File

After adding credentials to aws-vault (e.g., aws-vault add default), you can instruct the aws-cli to use aws-vault instead of ~/.aws/credentials.

Here is an ~/.aws/config entry for the default profile:

[default]
region=us-east-1
credential_process=aws-vault exec -j default

After all credentials are stored in aws-vault, and all ~/.aws/config entries have been updated with the credential_process line, ~/.aws/credentials should be empty.

Viewing Credentials

If you want to view the credentials for a profile, or if you want to expose them as environment variables, you can run:

aws-vault exec default -- env | grep --no-color ^AWS | sort

If you want to use them, the aws-cli will pick up environment variables before it looks for a credentials definition. So, if you want to use aws-vault with aws-cli without specifying the credential_process setting in your ~/.aws/config entry, you can do something like this:

aws-vault exec default -- aws s3 ls

Token Types

The AWS CLI (and any other tools built on AWS SDKs) will leverage the AWS_SESSION_TOKEN and AWS_SECURITY_TOKEN values before leveraging the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values.

AWS_SESSION_TOKEN and AWS_SECURITY_TOKEN tokens are more secure because they are ephemeral, and expire after a short (measured in hours) TTL. For this reason, these should generally be used instead of the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values.

But there is an exception — there are certain types of IAM-related tasks which cannot be performed using AWS_SESSION_TOKEN and AWS_SECURITY_TOKEN tokens, because they are IAM tokens themselves. In these cases, you want to fall back to the long-lived AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values. You can do this by passing the --no-session option to the aws-vault command.

aws-vault exec default --no-session -- env | grep --no-color ^AWS | sort

Here, you can see that the AWS_SESSION_TOKEN and AWS_SECURITY_TOKEN tokens are not generated, so the AWS CLI (and any other tools built on AWS SDKs) will leverage the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values instead.

Passing to (local) Docker

It is becoming more popular to provide Docker containers for running software, especially when that software has a number of (potentially-complex) dependencies. By wrapping everything up into a nice little Docker image, it makes it much simpler to build and distribute software that is meant to run locally.

With the traditional ~/.aws directory, you can mount it as read-only inside a Docker container if you want that Docker container to be able to communicate with AWS on your behalf.

docker run -ti -v $HOME/.aws:/root/.aws:ro {image_name} sh

Easy, right? Wrap it in a Makefile or a Bash alias, and call it a day.

But using aws-vault makes this a little more complicated. aws-vault runs on your local machine (not inside your Docker container), and your ~/.aws/credentials file is empty. How do we pass your credentials into a Docker container?

By exporting the environment variables and passing them to docker run.

docker run -ti \
    --env-file <(aws-vault exec default -- env | grep --no-color ^AWS_) \
    {image_name} sh

Wow! What does this do?

Exports the credentials to the environment.
Filters the environment variables by those that begin with AWS.
Run Docker, passing the AWS_* environment variables into Docker.
This particular command will start an interactive sh shell session. You can run other commands using docker run as appropriate.

Conclusion

aws-vault is a great tool for managing your credentials, helping you work with AWS-related tooling, and makes it easy to follow best-practices. If you’re interested in learning more, check out the README.