Skip to main content

Fast Bakery

Fast Bakery is an efficient containerization backend within Outerbounds that packages ML/AI dependencies into reproducible images automatically. This enables rapid deployment and efficient scaling for demanding workloads.

Fast Bakery enhances the existing @pypi and @conda decorators by automatically converting specified environments into container images. You can activate Fast Bakery by running or deploying a flow with @pypi or @conda using the --environment=fast-bakery flag.

Examples​

# For local Kubernetes runs
python flow.py --environment=fast-bakery run --with kubernetes

# For Argo Workflows
python flow.py --environment=fast-bakery argo-workflows create
python flow.py argo-workflows trigger

For more details on the technical implementation, read this blog post.

Advanced Usage

Out of the box, Fast Bakery supports resolving and containerizing publicly available Python packages and can use any public image as a base image. However, you can also configure Fast Bakery to use private Python packages and private container images.

note

To run the commands below, ensure you have:

  • Installed the outerbounds CLI.
  • Configured your environment using outerbounds configure ... (found on the Setup page in the Outerbounds UI).
  • Admin privileges on the platform.

Private Container Image Registries​

Fast Bakery builds containers by first pulling a base image. If your base image is hosted in a private registry, you must grant Fast Bakery access by creating a container-registry integration via the outerbounds CLI.

Private Amazon ECR​

If you have private images in ECR and want to use them as base images, you can configure Fast Bakery to access them.

@kubernetes(image="1234565789012.dkr.ecr.us-east-2.amazonaws.com/myreg/myimage:v0.1.0")
@pypi(
    packages={
        "pandas": "2.3.3",
        "scikit-learn": "1.7.2",
    },
    python="3.12",
)
@step
def start(self):
    ...

There are two options on how to provision the permissions and configure it.

Option 1 - Provision ECR Image Pull Permissions Through Assumed Role:

  1. Create an IAM role with permissions to pull images from your target ECR. Configure it to be assumable by the Outerbounds task role. Please use the AWS IAM integration on the Outerbounds UI to correctly create/configure this IAM role.
  2. Create a container-registry integration using the IAM role ARN from step 1:
outerbounds integrations \
     container-registry create my-ecr-registry \
     --description "My Private ECR Registry" \
     --registry-domain 1234565789012.dkr.ecr.us-east-2.amazonaws.com/ \
     --target-role-arn arn:aws:iam::1234565789012:role/outerbounds-fast-bakery-ecr-readonly

Option 2 - Provision ECR Image Pull Directly to the Task Role in your Perimeter:

  1. Find the task IAM role in the Outerbounds UI by going to the Perimeters page, copy the IAM task Role ARN.
  2. Provision that the task IAM role with permissions to pull images from your private ECR.
  3. Create a container-registry integration using the IAM role ARN from step 1:
outerbounds integrations \
    container-registry create my-ecr-registry \
    --description "My Private ECR Registry" \
    --registry-domain 1234565789012.dkr.ecr.us-east-2.amazonaws.com/ \
    --use-task-role

All set! 🎉 You can use any image in this ECR instance as the base image for your flows and build it using Fast Bakery.

Private GitLab Container Registry​

Similarly, if you use GitLab for your private container images, you can configure access using username/password credentials.

@kubernetes(image="registry.gitlab.com/myorg/myprog/myrepo:v0.1.0")
@pypi(
    packages={
        "pandas": "2.3.3",
        "scikit-learn": "1.7.2",
    },
    python="3.12",
)
@step
def start(self):
    ...

Steps to configure:

  1. Generate credentials with read permissions to the GitLab container registry. These can be for a dedicated service account user, a project level token, a PAT, or any other principal.
  2. Create the integration using the outerbounds CLI:
outerbounds integrations \
        container-registry create my-gitlab-registry \
        --description "My Private GitLab Registry" \
        --registry-domain registry.gitlab.com/myorg/myprog \
        --username myuser --password mypass

All set! 🎉 You can now start to set images in your GitLab container registries as base images, and Fast Bakery will build on top of them for as long as the credentials you provided are active. To rotate the credentials, you can run:

outerbounds integrations \
        container-registry update my-gitlab-registry \
        --registry-domain registry.gitlab.com/myorg/myprog \
        --username myuser --password mynewpassword

PyPI Packages from Private Git URLs​

Beyond private container registries, Fast Bakery also supports Python packages hosted in private Git repositories. The @pypi decorator supports installing packages directly from private Git repositories:

@pypi(
    packages={
        "git+https://github.com/myorg/my-private-repo.git@main": ""
    },
    python="3.12",
)
@step
def start(self):
    ...

Steps to configure:

  1. Generate credentials with read access to your repository. Fast Bakery will use these credentials to do an equivalent of git config --global url."url-with-creds".insteadOf "url-without-creds". Hence, you can scope these credentials to either just the repository level or the organization level, depending on your security needs.
  2. Create a git-pypi-repository integration, like:
outerbounds integration \
        git-pypi-repository create my-git-repo \
        --description "My Private Git Repo" \
        --repository-url https://github.com/myorg \
        --username myusername \
        --password mypassword

All set! 🎉 Fast Bakery is now able to build Python dependencies defined as git URLs.

Note: This configuration will also work for transitive dependencies referencing other private git URLs.

Private PyPI Packages​

In addition to Git URLs, Fast Bakery supports private PyPI repositories by dynamically formulating pip index URLs with credentials for each build. Below are configuration examples for the most common private PyPI repository providers:

AWS CodeArtifact​

If you're using AWS CodeArtifact to host your private Python packages, here's how to configure Fast Bakery to access them.

To use packages from AWS CodeArtifact (e.g., account 123456789012, region us-west-2):

  1. Create an IAM role with the permissions to download packages from your two Code Artifacts repositories, and configure it to be assumable by the Outerbounds task role. Please use the AWS IAM integration on the Outerbounds UI to correctly create/configure this IAM role.
  2. Create a code-artifacts integration, like:
outerbounds integrations \
        code-artifacts create my-code-artifacts \
        --description "My Code Artifacts Integration for 123456789012 us-west-2" \
        --domain myorg \
        --domain-owner 123456789012 \
        --aws-region us-west-2 \
        --target-role arn:aws:iam::1234565789012:role/outerbounds-fast-bakery-code-artifacts-readonly
  1. Add the repositories to the private PyPI list:
outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myprivateproject1 \
        --repository-host-integration-name my-code-artifacts # integration name from step 2
outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myprivateproject2 \
        --repository-host-integration-name my-code-artifacts # integration name from step 2

All set! 🎉 Fast Bakery is now able to resolve and download Python packages in your myprivateproject1 and myprivateproject2 PyPI repositories.

GitLab Package Registry​

For organizations using GitLab's Package Registry, the configuration process is similar but uses GitLab-specific authentication.

To use packages from a GitLab project (e.g., project ID 12345678):

  1. Generate credentials with read permissions for the GitLab package registry. These can be for a dedicated service account user, a project level token, a PAT, or any other principal.
  2. Create a gitlab-artifacts integration, like:
outerbounds integrations \
        gitlab-artifacts create myproject-gitlab-package-registry \
        --description "My Private Gitlab Package Registry" \
        --gitlab-url gitlab.com \
        --project-id 12345678 \
        --username myusername \
        --password mypassword
  1. Add the repository to the list of private PyPI repositories:
outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myrepo \
        --repository-host-integration-name myproject-gitlab-package-registry # integration name from step 2

All set! 🎉 Fast Bakery is now able to resolve and download Python packages in your myrepo PyPI repository from GitLab.

Azure DevOps Artifacts​

Azure DevOps Artifacts offers flexible authentication options. You can choose between managed identity (recommended for better security) or static credentials.

There are two ways to provision access to Azure DevOps Feeds:

Option 1 - Using Task Managed Identity (Recommended):

  1. Provision the Outerbounds Task managed identity, which is created per perimeter, with access to your Feed:
    1. In Azure DevOps, go to Organization Settings → Users → Add Users. Search for each managed identity with the name format p-(perimeter-name) (e.g., p-default), and give them access to your Azure DevOps project, myproject.
    2. Once the task principals have been provisioned access to the Azure DevOps organization and projects, you will next need to provision them access to the Azure Artifacts Feed (PyPI repositories). Select the Feed you want to provision access for and go to Feed Settings. Then navigate to Permissions → Add Users/Groups and search for the task principals you would like to provision access for, and add them as Feed Reader.
  2. Create an azure-artifacts integration, like:
outerbounds integrations \
        azure-artifacts create myproject-azure-artifacts \
        --description "Azure Articacts in myproject" \
        --organization myorg \
        --project-name myproject
  1. Add the Azure DevOps Artifacts PyPI repositories to the list of private PyPI repositories:
outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myrepo1 \
        --repository-host-integration-name myproject-azure-artifacts # integration name from step 2
outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myrepo2 \
        --repository-host-integration-name myproject-azure-artifacts # integration name from step 2

All set! 🎉 Fast Bakery is now able to resolve and download Python packages in your myrepo1 and myrepo2 PyPI repositories from Azure DevOps Artifacts.

Option 2 - Using Static Credentials:

  1. Generate credentials with read access to your Azure Artifacts Feeds. This can be a PAT or any other principal with access to download packages.
  2. Create the integration with credentials:
outerbounds integrations \
        azure-artifacts create myproject-azure-artifacts \
        --description "Azure Artifacts integration for myproject" \
        --organization myorg \
        --project-name myproject \
        --username username \
        --password password
  1. Add the Azure DevOps Artifacts PyPi repositories to the list of private pypi repositories, like:
$ outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myrepo1 \
        --repository-host-integration-name myproject-azure-artifacts # integration name from step 2
$ outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myrepo2 \
        --repository-host-integration-name myproject-azure-artifacts # integration name from step 2

All set! 🎉 Fast Bakery is now able to resolve and download Python packages in your myrepo1 and myrepo2 PyPI repositories from Azure DevOps Artifacts.

Artifactory (JFrog)​

Artifactory users can configure Fast Bakery to access their private PyPI repositories using standard Artifactory credentials.

To use PyPI repositories hosted on Artifactory (e.g., mycompany.pe.jfrog.io):

  1. Generate credentials for a user with the permission to download packages from the PyPI repositories.
  2. Create an artifactory integration, like:
outerbounds integrations \
        artifactory create mycompany-artifactory-pypi-repo \
        --description "Artifactory integration for mycompany" \
        --domain mycompany.pe.jfrog.io \
        --username username \
        --password password
  1. Add the Artifactory repository to the list of private PyPI repositories:
outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myrepo1 \
        --repository-host-integration-name mycompany-artifactory-pypi-repo # integration name from step 2
outerbounds integrations \
        private-pypi-repositories add \
        --repository-name  myrepo2 \
        --repository-host-integration-name mycompany-artifactory-pypi-repo # integration name from step 2

All set! 🎉 Fast Bakery is now able to resolve and download Python packages in your myrepo1 and myrepo2 PyPI repositories from Artifactory.

Private Conda Packages​

Fast Bakery also supports Python packages from private conda channels. It does this by dynamically formulating a list of conda channel URLs with static or short-lived credentials for each build request. Currently, Artifactory is the only private conda channel supported by Fast Bakery.

Artifactory (JFrog)​

To configure Fast Bakery for private Conda channels hosted on Artifactory, follow these steps:

  1. Generate credentials for a user with the permission to download packages from the private conda channels in Artifactory.
  2. Create an artifactory integration, like:
outerbounds integrations \
        artifactory create mycompany-artifactory-conda \
        --description "Artifactory integration for mycompany" \
        --domain mycompany.pe.jfrog.io \
        --username username \
        --password password
  1. Add the Artifactory private channels to the list of private Conda channels:
outerbounds integrations \
        private-conda-channels add \
        --repository-name  mychannel1 \
        --repository-host-integration-name mycompany-artifactory-conda # integration name from step 2
outerbounds integrations \
        private-conda-channels add \
        --repository-name  mychannel2 \
        --repository-host-integration-name mycompany-artifactory-conda # integration name from step 2

All set! 🎉 Fast Bakery is now able to resolve and download Python packages in your mychannel1 and mychannel2 private Conda channels from Artifactory.

Summary​

Fast Bakery is a versatile tool that integrates with a wide range of dependency management systems and container registries. If your use case requires special accommodations like private network connectivity or is not covered here, please reach out to the Outerbounds support team for assistance.