Skip to main content

Project lifecycle

Prerequisites

This page assumes familiarity with Project Structure and CI/CD integration.

Overview​

When you deploy a project with obproject-deploy, several resources are created on the platform. Understanding what gets created — and how to inspect or remove it — is essential for managing feature branches, cleaning up after experiments, and building custom deployment pipelines.

What deploy creates​

obproject-deploy reads your obproject.toml and processes the project directory in four stages:

StageWhat it doesResources created
1. AssetsRegisters data and model assets from data/ and models/ directoriesData assets, model assets
2. FlowsDeploys each flow under flows/ to Argo WorkflowsWorkflow templates, CronWorkflows (for @schedule), Sensors (for @trigger)
3. AppsDeploys app capsules from deployments/App capsules
4. MetadataRegisters a flowproject spec describing all deployed resourcesFlowproject metadata record

Each resource is scoped to a project and branch. The branch is derived from your git ref (e.g., main becomes the production branch, feature-v2 becomes a test branch).

Branch naming​

Branch names are normalized during deployment:

  • - and / characters are replaced with _
  • The result is lowercased

For example, git branch feature/add-scoring becomes feature_add_scoring.

Workflow template IDs follow a specific format where underscores are stripped entirely:

{project}.{metaflow_branch}.{flow_name}

For a project named fraud_detection on branch main with a flow called TrainFlow:

frauddetection.prod.trainflow

For a test branch feature-v2:

frauddetection.test.feature_v2.trainflow

The prod / test.{branch} prefix is Metaflow's @project branch convention.

Controlling what gets deployed​

By default, obproject-deploy deploys all flows, apps, and assets on every branch. This can lead to unnecessary app deployments on feature branches that persist after merge. Two mechanisms let you control this.

Per-component branch filtering​

Place an obproject_deploy.toml in any deployments/<app>/ or flows/<flow>/ directory to declare which branches should deploy that component:

# deployments/my-dashboard/obproject_deploy.toml
[deploy]
branches = ["main", "release/*"]

Glob patterns are supported. When a branch doesn't match, the component is skipped with a message:

⏭️  Skipping app 'my-dashboard' (branch 'feature_foo' not in ['main', 'release/*'])

When no obproject_deploy.toml is present, the component deploys on all branches (backward compatible). On non-main branches, an info message suggests adding the file.

Your CI workflow stays simple — just call obproject-deploy — and the filtering happens automatically based on the config files checked into the repo.

Recommended setup

Add obproject_deploy.toml with branches = ["main"] to each app in deployments/. This prevents app proliferation across feature branches while still deploying flows everywhere so you can test workflow changes. Combine with a teardown job to clean up branch resources after merge.

CLI flags​

For ad-hoc control without config files, use these flags:

obproject-deploy --skip-apps      # skip all app deployments
obproject-deploy --skip-flows     # skip all flow deployments
obproject-deploy --skip-assets    # skip asset registration

Inspecting deployed resources​

Use the outerbounds flowproject CLI to inspect what's currently deployed.

List workflow templates​

outerbounds flowproject list-templates --id fraud_detection/main

This queries Argo directly for templates matching the project and branch annotations, so it reflects the actual cluster state regardless of what the metadata record says.

View metadata​

outerbounds flowproject get-metadata --id fraud_detection/main | jq .

The metadata record contains the full deployment spec: workflows, assets, apps, and their configurations as registered by the last obproject-deploy run.

Tearing down a branch​

When a feature branch is merged or abandoned, use teardown-branch to clean up all its deployed resources:

# Preview what will be deleted
outerbounds flowproject teardown-branch --id fraud_detection/feature-v2 --dry-run

# Execute the teardown
outerbounds flowproject teardown-branch --id fraud_detection/feature-v2 --yes

Teardown deletes resources in this order:

  1. Workflow templates — cascade-deletes associated CronWorkflows and Sensors
  2. Data assets
  3. Model assets
  4. Apps (capsules)
  5. Flowproject metadata

If any individual deletion fails, the command continues with remaining resources and reports errors at the end with a non-zero exit code.

Promoting assets before teardown​

Teardown deletes asset metadata (the catalog entries), not the underlying data. If a feature branch trained a model or produced a dataset you want to keep on main, you need to promote those assets before tearing down the branch.

promote_assets() reads every asset on the source branch, takes the latest instance of each, and re-registers it on the target branch with the same blob references, annotations, and tags. The underlying S3 objects are not copied — only the metadata pointer is created.

from obproject.assets import promote_assets

# Promote all assets from feature branch to main
result = promote_assets('my_project', source='feature-v2', target='main')
# result = {"promoted": [...], "errors": [...]}

# Promote with aliases carried forward
result = promote_assets('my_project', source='feature-v2', target='main',
                        with_aliases=True)

You can also promote individual assets or specific instances:

# Promote only the classifier model
promote_assets('my_project', source='feature-v2', target='main',
               asset='classifier', kinds=['models'])

# Promote a specific aliased instance
promote_assets('my_project', source='feature-v2', target='main',
               asset='classifier', instance='@validated')

Promoted instances include lineage annotations (promoted_from_branch, promoted_from_instance) so you can trace where the production asset originated. With with_aliases=True, any aliases set on the source branch (e.g., @champion, @validated) are recreated on the target branch pointing to the promoted instance.

When to promote vs. re-run

Promote when training is expensive (large models, GPU-hours) and you want the exact same weights in production. Re-run when training is cheap and you want full reproducibility guarantees from the main branch code. Most projects use a mix: promote models, re-run data pipelines.

Automating teardown in CI/CD​

Add a teardown step to your CI/CD pipeline when branches are deleted or PRs are closed. If you want to preserve assets, add a promote step before teardown:

# GitHub Actions example
on:
  pull_request:
    types: [closed]
    branches: [main]
  delete:
    branches-ignore: [main]

jobs:
  promote-and-teardown:
    if: >
      (github.event_name == 'delete') ||
      (github.event_name == 'pull_request' && github.event.action == 'closed' && github.event.pull_request.merged == true)
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - name: Configure Outerbounds
      run: |
        # ... authentication setup (see CI/CD integration) ...

    - name: Promote assets
      if: github.event_name == 'pull_request'
      run: |
        SOURCE=${{ github.head_ref }}
        TARGET=${{ github.event.pull_request.base.ref }}
        PROJECT=$(yq .project obproject.toml)
        python3 -c "
        from obproject.assets import promote_assets
        result = promote_assets('$PROJECT', source='$SOURCE', target='$TARGET',
                                with_aliases=True)
        for p in result['promoted']: print(f'Promoted {p[\"kind\"]}/{p[\"name\"]}')
        "

    - name: Teardown branch
      run: |
        BRANCH=${{ github.head_ref || github.event.ref }}
        PROJECT=$(yq .project obproject.toml)
        outerbounds flowproject teardown-branch \
          --id "$PROJECT/$BRANCH" --yes -o json

See ob-project-asset-promotion for a complete working example.

Building a custom deploy pipeline​

If you need more control than obproject-deploy provides, you can use the outerbounds flowproject commands directly. This is useful when you want to:

  • Deploy a subset of flows or assets
  • Integrate with a non-standard CI/CD system
  • Add custom validation steps between deploy stages

Registering metadata​

After deploying flows and assets through your own tooling, register the metadata so the platform knows what's deployed:

outerbounds flowproject set-metadata '{
  "project": "fraud_detection",
  "branch": "main",
  "workflows": [
    {"flow_template_id": "frauddetection.prod.trainflow"}
  ],
  "data": [
    {"id": "training_data"}
  ],
  "models": [
    {"id": "fraud_classifier"}
  ]
}'

Deleting metadata only​

If you manage resource lifecycle separately and only need to clean up the metadata record:

outerbounds flowproject delete-metadata --id fraud_detection/feature-v2 --yes

This does not touch workflow templates, assets, or apps.

See Also​