Configuring secrets

Metaflow provides a built-in mechanism, the @secrets decorator for accessing secrets like database passwords securely in tasks. In the case of Outerbounds, the secrets are stored and managed for you. By following the instructions below, you can grant tasks access to specific secrets.

info

By default, secrets are stored in the control plance account operated by Outerbounds. If you'd like to move secrets completely into the data plane account you control, please reach out to Outerbounds support directly.

Integrations view

In your Outerbounds deployment, navigate to the Integrations view to manage your secrets. Here you can configure custom secrets as key-value pairs, or use any of the various integrations available for popular services like databases, IAM roles, and API keys.

Using secrets

After you have configured one or more secrets as described above, you can access them in your flows @secrets decorator. During task execution, the secrets are retrieved automatically and made available through environment variables. For instance, if you named a secret my-secret, it will be accessible in your flow as follows:

from metaflow import FlowSpec, step, secrets

class SecretsFlow(FlowSpec):
    @secrets(sources=["outerbounds.my-secret"])
    @step
    def start(self):
        import os
        assert os.environ.get("SECRET_KEY1") == "secret_value1"
        assert os.environ.get("SECRET_KEY2") == "secret_value2"
        self.next(self.end)

    @step
    def end(self):
        pass

if __name__ == "__main__":
    SecretsFlow()

For each specific integration, you can find the relevant code snippet in the form on the Integrations view.