Causeway · Guides

Devcontainers and DevPod

Reproducible dev environments, shipped in the repo, identical for every engineer. The fix for 'works on my machine' that actually holds.

"Works on my machine" is the oldest bug in data engineering. It happens because dbt, Databricks Connect, Python versions, Java versions (for PySpark), and platform CLIs drift independently across laptops. Devcontainers fix it by declaring the entire environment as code and rebuilding it from that declaration on every checkout. DevPod extends devcontainers to remote compute, so the same declaration runs on your laptop, a beefy EC2 VM, or a shared devpod cluster.

By the end of this guide you will have:

1. Why devcontainers

Three failures disappear the moment a repo ships a devcontainer:

Devcontainers replace the setup README ("install Python 3.12, then install JDK 17, then set JAVA_HOME, then…") with an image that does all of it.

Note

A devcontainer is not a production deployment artifact. It is a development environment. The same repo can ship a devcontainer for engineering and a production Docker image for deployment; they are different concerns.

2. Authoring devcontainer.json

Create .devcontainer/devcontainer.json at the repo root. Start with a base image and add features:

{
  "name": "churn-analytics",
  "image": "mcr.microsoft.com/devcontainers/python:3.12",
  "features": {
    "ghcr.io/devcontainers/features/docker-in-docker:2": {},
    "ghcr.io/devcontainers/features/aws-cli:1": {},
    "ghcr.io/devcontainers/features/node:1": {},
    "ghcr.io/customink/codespaces-features/astro-cli:1": {}
  },
  "postCreateCommand": "uv sync && dbt deps && pre-commit install",
  "customizations": {
    "vscode": {
      "extensions": [
        "ms-python.python",
        "ms-python.vscode-pylance",
        "charliermarsh.ruff",
        "dbtLabsInc.dbt",
        "dorzey.vscode-sqlfluff",
        "samuelcolvin.jinjahtml",
        "redhat.vscode-yaml",
        "databricks.databricks",
        "eamodio.gitlens",
        "GitHub.vscode-pull-request-github"
      ],
      "settings": {
        "python.defaultInterpreterPath": "/workspaces/churn-analytics/.venv/bin/python",
        "editor.formatOnSave": true
      }
    }
  },
  "forwardPorts": [8080, 8793],
  "remoteUser": "vscode"
}

Key fields:

Tip

Put the heavy postCreateCommand work behind postStartCommand instead if you want fast restart times. postCreate runs on build; postStart runs on every container start.

3. Open in container

  1. Install the Dev Containers extension (ms-vscode-remote.remote-containers).
  2. Open the repo in VS Code.
  3. Click the green button in the lower-left, then Reopen in Container. Or run Dev Containers: Reopen in Container from the command palette.

VS Code builds the image, starts the container, and reconnects the editor to it. Terminals, the Git client, and extensions all run inside the container. First build takes 2–5 minutes. Subsequent opens take seconds.

Important

The repo is bind-mounted into the container at /workspaces/<repo-name>. Edits from the editor land on the host filesystem in real time. Do not run rm -rf /workspaces/... from inside the container thinking it is a disposable sandbox; it is your real checkout.

4. Post-create commands

The postCreateCommand runs one bash command. For anything non-trivial, point it at a shell script:

"postCreateCommand": "bash .devcontainer/post-create.sh"

#!/usr/bin/env bash
set -euo pipefail

uv sync --frozen
uv run dbt deps
uv run pre-commit install
uv run pre-commit install --hook-type commit-msg

echo "Environment ready."

Keep the script short and idempotent. Engineers will run it multiple times (rebuilds, feature additions). A script that fails on second run is a script that costs trust.

5. Secrets inside a devcontainer

Secrets are the devcontainer's biggest footgun. Three patterns, only one is safe:

Safe: host-to-container passthrough

Mount ~/.aws/, ~/.databricks/, and ~/.dbt/ from the host:

"mounts": [
  "source=${localEnv:HOME}/.aws,target=/home/vscode/.aws,type=bind,readonly",
  "source=${localEnv:HOME}/.databricks,target=/home/vscode/.databricks,type=bind,readonly",
  "source=${localEnv:HOME}/.dbt,target=/home/vscode/.dbt,type=bind,readonly"
]

Secrets live on the host, are shared read-only into the container, and never ship in the image.

Less safe: devcontainer env vars

"containerEnv": {
  "DATABRICKS_HOST": "${localEnv:DATABRICKS_HOST}",
  "DATABRICKS_TOKEN": "${localEnv:DATABRICKS_TOKEN}"
}

The env vars get copied from the host into the container at start. Acceptable for OAuth tokens. Not acceptable for long-lived credentials — they leak into docker inspect and any process that reads /proc/<pid>/environ.

Unsafe: baking secrets into the image

"args": { "DBT_TOKEN": "dbt_xxx..." }

Never. The secret ends up in the image layer, the container registry, and the devcontainer JSON in Git. Rotate immediately if you find this.

Warning

A devcontainer commits to the repo. Anything hard-coded in devcontainer.json is public to everyone with repo read access. Never write a literal secret there.

6. Features vs. Dockerfile

Two ways to add tools to the container:

For a custom Dockerfile, replace image with build.dockerfile:

{
  "name": "churn-analytics",
  "build": {
    "dockerfile": "Dockerfile",
    "context": ".."
  },
  "features": { ... }
}

Keep the Dockerfile minimal. If you find yourself installing Python, Java, and ten tools, lean on features instead.

7. Performance on macOS

Docker Desktop's file-sharing on macOS is famously slow. Large node_modules directories, .venv trees, and target/ folders can kill the editor. Mitigations:

"mounts": [
  "source=uv-cache-${containerWorkspaceFolderBasename},target=/home/vscode/.cache/uv,type=volume"
]

Tip

If a devcontainer feels slow, check the file-sharing first. time ls -R . on a 100-file directory should take milliseconds. If it takes seconds, your file-share is the bottleneck.

8. DevPod: running devcontainers remotely

DevPod takes the same devcontainer.json and runs it anywhere:

Install DevPod (brew install devpod, or the desktop app). Add a provider:

devpod provider add aws
devpod provider use aws

Create a workspace against the repo:

devpod up github.com/your-org/churn-analytics

DevPod provisions an EC2 VM, clones the repo, builds the devcontainer on the VM, and connects VS Code to it via SSH. The laptop becomes a thin client; the environment runs where the compute is.

When DevPod wins

When DevPod loses

9. Teardown

Devcontainers and devpods accumulate. Every project is a new container; the disk fills.

# Dev Containers extension
docker ps -a --filter "label=devcontainer.local_folder" --format "{{.ID}}\t{{.Names}}"
docker rm -f <container-id>

# DevPod
devpod list
devpod delete <workspace-name>

DevPod's auto-stop is your friend: idle workspaces shut down after a configurable inactivity period. Turn it on.

10. The paved path

A paved-path devcontainer strategy looks like:

Important

The devcontainer must reflect production — same Python version, same JDK version, same CLI versions. A devcontainer where dbt 1.9 works locally and dbt 1.10 runs in CI is a devcontainer that lies.

Troubleshooting

Container build fails with "permission denied" on mounted files

The UID inside the container does not match the UID on the host. Add "updateRemoteUserUID": true to the devcontainer JSON, or map UIDs explicitly with "containerUser": "1000".

postCreateCommand runs every time I restart

It should run only on first create. If it is running on every restart, you probably put the command in postStartCommand by mistake. Check the devcontainer JSON.

Extensions listed in the JSON do not install

The container has to reach the Marketplace (or Open VSX, depending on your fork). Check outbound connectivity from the container. Corporate proxies often block the Marketplace; you need http.proxy in user settings or a VSIX sideload strategy.

DevPod fails to provision the VM

Usually an AWS quota or an IAM-permission issue. Run devpod up --debug <repo> and read the provider-specific log. Platform team owns the provider config; file an issue.

See also