Secrets

Secrets are used to provide data that is considered sensitive like passwords, keys, and other sensitive information. Acorn provides multiple secret types, and makes it easy to create closed loop systems where information can be kept within the defined Acorn.

Using secrets in an Acornfile

Secrets must be defined in the Acornfile. The values can be generated by Acorn or provided by an existing secret in the cluster. Secrets should not be assigned values from user arguments. Secrets can be consumed in containers by referencing them by the secret://<secret_name>/<key> URI syntax.

It is possible to define the behavior when the secret is updated to a new value. The default is to redeploy the containers one at a time. If that operation is unsafe, or requires more user intervention, you can add the ?onchange=no-action to the end of the secret reference URI.

Consume secret in an environment variable

Some containers use environment variables for secret bits of data. The MariaDB container for instance uses an environment variable to set the root password.

containers: {
    db: {
        image: "mysql"
        // ...
        env: {
            "MYSQL_ROOT_PASSWORD": "secret://db-root-password/token"
        }
    }
}
// ...
secrets: {
    "db-root-password": {
        type: "token"
    }
}

The example shows the secret db-root-password being consumed as an environment variable in the db container. The secret is of type token and will automatically be generated at runtime by Acorn. The user can then use the Acorn CLI to get the credential if needed.

Consume all secret keys as environment variables

Sometimes you have multiple secrets that need to be consumed as environment variables. The example below shows how to consume all the values in a secret as environment variables named after the keys. The keys must be set exactly as you want them to show up in the environment

containers: app: {
    image: "mysql"
    env: {
        "secret://mysql-info": ""
    }
}

secrets: "mysql-info": {
    type: "opaque"
    data: {
        "MYSQL_ROOT_PASSWORD": "Foo"
    }
}

This will set the environment variable "MYSQL_ROOT_PASSWORD" to the value "Foo" in the container.

This is really beneficial if you need to consume somewhat arbitrary secret data as environment variables. Take an app that will run a 3rd party script that requires unknown environment variables. The user can bind in a secret at deploy time that has the needed info.

containers: nginx: {
    image: "nginx"
    env: "secret://php-env-vars": ""
}

secrets: "php-env-vars": {
    type: "opaque"
}

Now if the user creates a secret like:

acorn secret create php-env-vars --data PHP_ENV_VAR1=foo --data PHP_ENV_VAR2=bar

Then runs the acorn like so from the directory with the Acornfile:

acorn run -n php-app -s php-env-vars:php-env-vars .

The nginx container will have the env vars.

Consuming secrets in templates

Secrets in templates are wrapped in ${}. It is possible to pass secret configuration data to a container in a pre-rendered form. This next example shows how you can add a configuration file with sensitive data.

containers: {
    web: {
        image: "nginx"
        // ...
        files: {
            "/etc/nginx/conf.d/website.conf": "secret://website-conf/template"
        }
    }
}
// ...
secrets: {
    "proxy-auth": {
        type: "opaque"
        data: {
            "basic-auth-string": "credentialsgohere"
        }
    }
    "website-conf": {
        type: "template"
        data: {
            template: """
            server {
                listen 80;
                // ...
                location / {
                    // ...
                    proxy_set_header Authorization "Basic ${secret://proxy-auth/basic-auth-string}"
                }
            }
            """
        }
    }
}

The above example has a container that will use the website-conf secret to create a config file. Before rendering the config file, Acorn will substitute the basic-auth-string into the template. This technique makes it possible for the user to pass in the sensitive basic-auth-string at runtime by binding a pre-existing secret.

Populating a directory with files

You can populate a directory with sensitive data using a secret. The example below shows populating the ~/.ssh directory with private keys from a secret.

containers: {
    git: {
        image: "my-git"
        dirs: {
            "/home/user/.ssh": "secret://user-provided-ssh-keys"
        }
    }
}
secrets: {
    "user-provided-ssh-keys": {
        type: "opaque"
        data: {
            //Example content from bound secret.
            //id_rsa: "-----BEGIN OPENSSH PRIVATE KEY-----...."
            //id_rsa_aws: "-----BEGIN OPENSSH PRIVATE KEY-----..."
            //Do not put private keys in Acornfile
        }
    }
}

The above will populate the /home/user/.ssh directory with the content of each item in files named after the key.

# inside the git container
ls /home/user/.ssh
id_rsa id_rsa_aws

Types of secrets

Acorn makes multiple types of secrets available to the Acorn author.

Basic: Used to generate and/or store usernames and passwords.
Template: Used to store configuration files that contain sensitive information.
Token: Used to generate and/or store long secret strings.
Generated: Used to take the output of a job and pass along as a secret bit of info.
Opaque: A generic secret that can store defaults in the Acorn, or is meant to be overridden by the user to pass unknown/unstructured sensitive data.
Credential: These are a special type of secret that will prompt the user for the values at runtime. These are useful for things like API keys to popular services like OpenAI.

Basic secrets

Basic secrets are defined in the secrets block with the type "basic".

// ...
secrets: {
    "my-creds": {
        type: "basic" // required
        data: {  
            username: "" // optional
            password: "" // optional
        }
    }
}

The basic secret type is used for username / password pairs. The key names must be username and password. If one or both of the fields are defined with a non-empty string, those values will be used. If the empty string, the default value, is used Acorn will generate random values for one or both.

Template secrets

The template secret can be used to render and store multiple secret values into a single output. It has the following format:

// ...
secrets: {
    "password-file": {
        type: "template" // required
        data: {
            "password.txt": """
            password=${secret://token/token}
            """
        }
    }
    "token": {
        type: "token"
    }
}

In the above example the secret renders a template secret with one key called "password.txt", consuming the token from the secret named "token." See advanced topics for other uses for the template secret type.

Token secrets

Token secrets are useful for generating a password or secure string used for passwords when the user is already known or not required.

secrets: {
    "my-token": {
        type: "token" // required
        params: {
            length: 32 // optional
            characters: "abcdedfhifj01234567890" // optional
        }
        data: {
            token: "" // optional
        }
    }
}

The token secret type must be defined. The params allow customization of the generated token. By default tokens are 54 characters in length. By defining the length param the token can be customized to be within 0-256 characters long. The characters param allows the user to define the allowed character values within the token.

The token field in the data object is optional and needs to be left the default empty string if Acorn should generate the token. If the token is defined that value will always be used.

Generated secrets

Generated secrets allow storing sensitive data output from a job.

containers: {
    "frontend-proxy": {
        // ...
        files: {
            "/etc/htpasswd": "secret://htpasswd-file/content"
        }
        // ...
    }
}
jobs: {
    "htpasswd-create": {
        env: {
            "USER": "secret://user-creds/username"
            "PASS": "secret://user-creds/password"
        }
        entrypoint: "/bin/sh -c"
        image:      "httpd:2"
        // Output of a generated secret needs to be placed in the file /run/secrets/output.
        cmd: ["htpasswd -Bbc /run/secrets/output $USER $PASS"]
    }
}
secrets: {
    "user-creds": {
        type: "basic"
    }
    "htpasswd-file": {
        type: "generated" // required
        params: {
            job: "htpasswd-create" // required
            format: "text" // optional
        }
    }
}

In the above example, there is a basic secret that Acorn will generate a username and password for. The job will run with the basic secret data passed in as environment variables. When the job runs it will generate an htpasswd file and write the content out to the required /run/secrets/output target. The contents of the /run/secrets/output file will be placed into the secret htpasswd-file and consumed by the frontend-proxy container. The key content must be used when referencing the value.

The job parameter is always required, and is the name of the job that will generate the output. The format parameter is optional and defaults to text.

Opaque secrets

Opaque secrets have no defined structure and can have arbitrary key value pairs. These types of secrets are best used for allowing a user to input sensitive data at runtime. In some cases an unstructured secret can be used if the user will be passing data that will be used in user defined templates. Expected keys should be predefined with reasonable defaults to provide the user some context.

secrets: {
    "user-secret-data": {
        type: "opaque"
    }
}

Credential secrets

Credential secrets are similar to opaque because they can hold any data you need, but they provide helpful UX to the end user by displaying instructions and prompting the user for the values. The type must start with credential but should be extended to display something meaningful to the user like credential.openai.com/gpt4.

secrets: "openai-key": {
    type: "credential.openai.com/gpt4"
    params: {
        instructions: localData.instructions
    }
    data: {
        OPENAI_API_KEY: ""
    }
}

localData: instructions: """
# OpenAI API Key

To get this key follow these instructions: https://platform.openai.com/docs/quickstart?context=python
"""

Now when the above is run, the user will be prompted for the value of OPENAI_API_KEY and the instructions will be displayed.

acorn run -n gpt4 .
[+] Building 1.4s (5/5) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                     0.0s
 => => transferring dockerfile: 58B                                                                                                                                      0.0s
 => [internal] load .dockerignore                                                                                                                                        0.0s
 => => transferring context: 64B                                                                                                                                         0.0s
 => [internal] load build context                                                                                                                                        0.0s
 => => transferring context: 616B                                                                                                                                        0.0s
 => [1/1] COPY . /                                                                                                                                                       0.0s
 => exporting to image                                                                                                                                                   0.7s
 => => exporting layers                                                                                                                                                  0.1s
 => => exporting manifest sha256:c31bd14fd9d434240bd4f50477f168f8d0d93440845fd76a33d187b84ddbd6f7                                                                        0.0s
 => => exporting config sha256:c83fad1b39a27e78d5055a3c01cfc789abff1852a55633427a563bffd09a5ad1                                                                          0.0s
 => => pushing layers                                                                                                                                                    0.4s
 => => pushing manifest for 495962627484.dkr.ecr.us-east-2.amazonaws.com/acorn/cloudnautique-46c452ebb422/acorn:latest@sha256:c31bd14fd9d434240bd4f50477f168f8d0d934408  0.2s
gpt4

   OpenAI API Key

  To get this key follow these instructions:
  https://platform.openai.com/docs/quickstart?context=python


? OPENAI_API_KEY *****************

External secrets

External secrets are defined in the Acornfile to specify a specific secret must be present in the cluster before the Acorn can be deployed. The definition must include the field external with the value of the expected name of the secret in the cluster.

containers: app: {
    image: ubuntu
    entrypoint: ["sleep"]
    command: ["3600"]
    env: {
        USER: "secret://foo/user"
        PASS: "secret://foo/pass"
    }
}

secrets: foo:{
    external: "basic-creds"
}

The above example requires a secret named basic-creds to be present in the cluster before the Acorn can be deployed.

For readability and documentation purposes, the best practice to define the type and data fields that the external secret is expected to have. This is optional and the values will be ignored by Acorn.

containers: app: {
    image: ubuntu
    entrypoint: ["sleep"]
    command: ["3600"]
    env: {
        USER: "secret://foo/user"
        PASS: "secret://foo/pass"
    }
}

secrets: foo: {
    external: "basic-creds"
    type: "opaque"
    data: {
        user: "username"
        pass: "password"
    }
}

Looking at the above example a user knows they must create a secret named basic-creds with keys/values for user and pass before the Acorn can be deployed.

Using secrets in an Acornfile​

Consume secret in an environment variable​

Consume all secret keys as environment variables​

Consuming secrets in templates​

Populating a directory with files​

Types of secrets​

Basic secrets​

Template secrets​

Token secrets​

Generated secrets​

Opaque secrets​

Credential secrets​

External secrets​