Search the docs, videos, blog, apps, teams, skills and release notes.
Open source. macOS, Windows and Linux.
At a glance
What it is
Guides expert-level docker compose patterns implementation: cloud and automation decision frameworks, production-ready patterns, and concrete templates for docker compose patterns workflows.
A skill is a written procedure a bot loads when a job calls for it. This one is a single file, SKILL.md, and the whole file is on this page.
The whole file, exactly as it downloads. Read it before you rely on it.
SKILL.md
927 lines · 5,360 words
Use this skill when:
User is designing or refactoring a docker-compose.yml file for local development, CI/CD, or staging environments
User asks how to structure multi-service applications with Docker Compose (e.g., app + database + cache + reverse proxy)
Guides expert-level docker compose patterns implementation: cloud and automation decision frameworks, production-ready patterns, and concrete templates for docker compose patterns workflows.
Nothing. It ships with Murage, which is open source. Your bots run on the AI plan you already have.
Yes. Open it in Settings → Skills to read, edit or duplicate it. Skill Guard scans it again after every edit.
Download the app, connect the AI you already pay for, and tell your Chief of Staff what needs doing. Plan on about ten minutes from install to a working team.
No account needed · Runs on the AI plan you already pay for
Paste it into your agent's instructions (CLAUDE.md, AGENTS.md or a custom GPT), or add it to a bot in Murage.
Ask Ember for the bot, then switch the skill on in its settings.
Step 1
Send your Chief of Staff this message. It proposes the bot and its role first, and creates it when you say go.
Send this to Ember
“Create a bot that uses the Docker Compose Patterns skill, and show me the plan first.”
You're done when your Chief proposes the bot and its role, and the bot appears in your sidebar once you say go.
Step 2
You're done when you see the bot’s skills and the library.
Step 3
You're done when you have read what Skill Guard found and the skill shows as on for that bot.
Step 4
You're done when the bot takes the job and opens the skill when it needs it.
Prefer to do it yourself? Choose + in the sidebar, then New Bot.Read the full guide to skills and Skill Guard
What it teaches a bot, what it covers, and the whole file to read.
What it covers
When to Use
Process
Output Format
Rules
Edge Cases
Example
User wants to manage multiple environments (dev/staging/prod) using Compose overrides or profiles
User needs guidance on secrets management, health checks, networking, volume strategies, or dependency ordering in Compose
User asks about service scaling, rolling restarts, or blue-green-style patterns achievable with Compose
User wants to optimize Compose for CI pipeline speed (layer caching, build contexts, parallel startup)
User is troubleshooting a broken Compose setup (services not starting, networking issues, volume permission errors, dependency cycles)
Do NOT use this skill when:
User needs Kubernetes orchestration patterns -- use a Kubernetes-specific skill instead (Compose is not a replacement for Kubernetes in production at scale)
User is asking about standalone docker build or docker run commands without a Compose context
User needs infrastructure-as-code for cloud resources (Terraform, Pulumi, CDK) -- Compose only manages containers, not cloud infrastructure
User is asking about Docker Swarm mode (Compose v3 YAML is reused but the deployment model is entirely different -- use a Swarm-specific skill)
User needs service mesh, mTLS, or advanced traffic management -- Compose lacks sidecar proxy support; recommend a Kubernetes + Istio/Linkerd path
User is working in a fully managed container platform like AWS ECS with Fargate -- while Compose-to-ECS adapters exist, the production patterns diverge significantly
User needs multi-region active-active deployment -- Compose has no concept of availability zones or regional failover
Before writing a single line of YAML, map the services and their relationships:
Identify all services and categorize them: application services (stateless), data services (stateful), infrastructure services (reverse proxy, queue, cache), and tooling services (migration runners, seed scripts, test runners)
Draw the dependency graph explicitly -- which services must be healthy before others start? A typical web app has: db → migrations → api → frontend. Circular dependencies are impossible; if you perceive one, a service boundary is wrong
Classify network exposure: which services need to be reachable from the host (published ports), which only need to communicate service-to-service (internal networks only), and which must be completely isolated
Identify volume needs: which services write persistent state (databases, uploaded files, logs), which need shared read-only assets (static files mounted into multiple services), and which should use ephemeral storage (caches that should reset on restart)
Count environment-specific differences: list every value that changes between dev/staging/prod -- these become variables in .env files or Compose override files, never hardcoded in the base docker-compose.yml
Select the right file organization strategy before writing configuration:
Single-file pattern (appropriate for: ≤4 services, single environment, personal projects): one docker-compose.yml with all configuration. Keep it simple; resist premature abstraction
Base + override pattern (appropriate for: 2+ environments, team projects): docker-compose.yml as the canonical service definition, docker-compose.override.yml automatically applied in dev (mounts source code, exposes debug ports), docker-compose.prod.yml applied explicitly for production-like runs. Use docker compose -f docker-compose.yml -f docker-compose.prod.yml up
Profile-based pattern (appropriate for: optional services that vary by role): use the profiles key to group services. Core services have no profile (always start). Optional services like mailhog, pgadmin, or load-testing are in named profiles and started only with --profile tooling
Include-based pattern (Compose v2.20+ / Docker Compose plugin): use include: to compose multiple separate Compose files into one logical application. Useful when sub-teams own service definitions independently. Not widely supported in older CI environments -- check the Docker Compose version available
Decide on .env file strategy:
Use .env for non-sensitive defaults (image tags, port numbers, service names)
Never commit secrets to .env -- use .env.example with placeholder values checked into version control
In CI, inject secrets via environment variables or a secrets manager, not from files
Docker Compose networking determines both security posture and service discoverability:
Default network: every docker compose up creates one default bridge network. All services join it automatically. Service names become DNS hostnames. This is sufficient for simple projects
Segmented networks (recommended for production-like environments): create explicit named networks and assign services to only the networks they need
frontend_net: reverse proxy ↔ application containers
backend_net: application containers ↔ databases, caches, queues
Only the reverse proxy is on both networks (it bridges them)
Databases must NOT be on frontend_net; they should never be reachable from the proxy layer
internal: true flag: mark backend networks as internal (no internet access) for database and cache networks. This prevents a compromised application container from making outbound calls on the database network
Published ports: only publish ports to the host when human or external-tool access is needed (e.g., port 5432 published for a developer's database GUI). Never publish database ports in staging/prod -- use docker compose exec or a bastion approach instead
IPv6: Compose supports IPv6 but requires explicit enable_ipv6: true on network definitions and an address pool. Only configure this if your application explicitly needs it
The depends_on key alone does not wait for a service to be ready -- it only waits for the container to start. Use health checks to enforce true readiness:
Define healthcheck on every data service: at minimum, databases, caches, and message brokers should declare health checks
PostgreSQL: test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"], interval: 5s, timeout: 5s, retries: 10, start_period: 30s
MySQL/MariaDB: test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
Redis: test: ["CMD", "redis-cli", "ping"], interval: 3s, timeout: 3s, retries: 5
RabbitMQ: test: ["CMD", "rabbitmq-diagnostics", "check_port_connectivity"]
Generic HTTP: test: ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"]
Wire depends_on with condition: service_healthy: this forces Compose to pause the dependent service startup until the dependency reports healthy
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthyUse condition: service_completed_successfully for one-shot services like database migration runners -- the API service should not start until migrations have finished
Set start_period to accommodate slow-starting services (JVM warmup, database initialization). Failures during start_period do not count toward retries. For PostgreSQL initializing a large schema, use start_period: 60s
restart policies: use restart: unless-stopped for long-running services in development, restart: on-failure:5 in staging to catch crash loops early without infinite restart loops
Never put credentials in docker-compose.yml directly. Apply a layered configuration strategy:
Layer 1 -- .env file (non-sensitive, environment-specific defaults): image tags (APP_IMAGE_TAG=main-abc1234), port mappings (POSTGRES_EXPOSED_PORT=5433), resource limits, feature flags
Layer 2 -- Environment variable passthrough: for sensitive values in CI, define the variable in the Compose file as ENV_VAR: (no value) -- Docker Compose will read it from the shell environment. This is how CI secrets should flow in
Layer 3 -- Docker secrets (Compose v3.1+): use the secrets: top-level key to mount files into containers at /run/secrets/<secret_name>. Applications must be written to read from files, not environment variables. This is the most secure approach available in Compose
secrets:
db_password:
file: ./secrets/db_password.txt
services:
db:
secrets:
- db_password
environment:
POSTGRES_PASSWORD_FILE: /run/secrets/db_passwordNever do: environment: DATABASE_URL: postgres://user:hardcoded_password@db/mydb -- this leaks credentials into docker inspect output, process environment dumps, and version control if accidentally committed
Config objects (Compose v3.3+): use configs: for non-sensitive configuration files (nginx configs, application config files) that should be mounted read-only into containers without being baked into the image
Compose files without resource limits allow a single misbehaving service to starve all others:
mem_limit and cpus (Compose v2 syntax, still works with Compose plugin):
mem_limit: 512m
cpus: "0.5"deploy.resources (Compose v3 syntax, preferred for compatibility with Swarm):
deploy:
resources:
limits:
cpus: "0.5"
memory: 512M
reservations:
cpus: "0.25"
memory: 256MTypical sizing guidelines for development stacks:
PostgreSQL: 512M--1G memory, 0.5--1 CPU
Redis: 128M--256M memory, 0.25 CPU
Node.js/Python API: 256M--512M memory, 0.5 CPU
Java/JVM service: 512M--2G memory, 1 CPU (set -Xmx to 80% of container limit)
Nginx/Traefik reverse proxy: 64M--128M memory, 0.1 CPU
OOM kill behavior: when a container exceeds mem_limit, it receives SIGKILL. Set mem_swappiness: 0 to disable swap for containers that must not degrade gracefully
Log limits: always configure log rotation to prevent disk exhaustion
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"Build performance directly impacts developer productivity and CI costs:
Use BuildKit: set DOCKER_BUILDKIT=1 or use docker compose build with the Compose plugin (BuildKit is enabled by default in Docker Desktop 23.0+). BuildKit enables parallel layer builds, better caching, and --mount=type=cache in Dockerfiles
Separate dev and prod Dockerfiles: Dockerfile for production (minimal, multi-stage, no dev dependencies), Dockerfile.dev for development (includes hot-reload tools, debuggers, test runners). Reference with build.dockerfile: Dockerfile.dev in the override file
Multi-stage builds in production Dockerfiles: builder stage (install all deps, compile, run tests) → runtime stage (copy only artifacts, minimal base image). This pattern reduces final image size by 60--80% for compiled languages
Layer ordering for cache efficiency: order Dockerfile instructions from least-frequently-changed to most-frequently-changed: OS packages → language runtime → dependency manifests (package.json, requirements.txt) → dependency install → application code copy
Build context optimization: always include a .dockerignore file. Exclude: node_modules/, __pycache__/, .git/, *.log, .env, test output directories. A bloated build context is one of the top causes of slow builds
cache_from in CI: specify previously built images as cache sources to avoid rebuilding unchanged layers
build:
context: .
cache_from:
- myapp:${CI_COMMIT_BRANCH:-main}Bind mounts for development hot-reload: in the dev override file, mount source code into the container so the process inside can detect file changes. Combine with inotify-based file watchers (nodemon, watchexec, air for Go). Use delegated consistency mode on macOS to improve I/O performance: - type: bind, source: ./src, target: /app/src, consistency: delegated
Production-ready Compose configurations document their own operation:
Add a # Usage comment block at the top of every Compose file with the four most common commands: start, stop, logs, exec into a specific service
Document all required environment variables in .env.example with descriptions and example values for every variable. A new developer should be able to run the stack in under 5 minutes using only the README and .env.example
Include a Makefile or justfile with named targets for common workflows: make up, make down, make logs, make migrate, make test, make shell SERVICE=api. This abstracts the verbose docker compose -f ... -f ... --env-file ... commands
Version-pin base images: use digests or specific version tags (postgres:15.4-alpine3.18), never latest. Unpinned images cause silent breakage when upstream updates land
Label all services with metadata for tooling and observability:
labels:
com.mycompany.service: "api"
com.mycompany.version: "${APP_VERSION}"
com.mycompany.team: "platform"When producing a Docker Compose configuration recommendation, structure the response as follows:
[host:80/443]
│
[reverse-proxy] ── frontend_net ──> [api:3000]
│
backend_net ──┼──> [postgres:5432]
└──> [redis:6379].env.example)# Application
APP_IMAGE_TAG=latest
APP_PORT=3000
NODE_ENV=development
# Database
POSTGRES_HOST=db
POSTGRES_PORT=5432
POSTGRES_DB=myapp_dev
POSTGRES_USER=myapp
POSTGRES_PASSWORD=changeme_in_production
# Cache
REDIS_HOST=redis
REDIS_PORT=6379
REDIS_PASSWORD=
# Reverse Proxy
NGINX_HOST_PORT=80docker-compose.yml)# docker-compose.yml
# Base service definitions -- environment-agnostic configuration.
# Run with: docker compose up
# For production: docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
version: "3.9"
# ─────────────────────────────────────────────────────────────────────────────
# Named volumes for persistent data
# ─────────────────────────────────────────────────────────────────────────────
volumes:
postgres_data:
driver: local
redis_data:
driver: local
# ─────────────────────────────────────────────────────────────────────────────
# Networks: segmented to limit blast radius of a compromised container
# ─────────────────────────────────────────────────────────────────────────────
networks:
frontend_net:
driver: bridge
backend_net:
driver: bridge
internal: true # No internet access -- data services only
# ─────────────────────────────────────────────────────────────────────────────
# Reusable configuration fragments (YAML anchors)
# ─────────────────────────────────────────────────────────────────────────────
x-logging: &default-logging
driver: json-file
options:
max-size: "10m"
max-file: "3"
x-restart-policy: &default-restart
restart: unless-stopped
services:
# ─── Reverse Proxy ──────────────────────────────────────────────────────────
nginx:
image: nginx:1.25.3-alpine
<<: *default-restart
ports:
- "${NGINX_HOST_PORT:-80}:80"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
networks:
- frontend_net
depends_on:
api:
condition: service_healthy
logging: *default-logging
deploy:
resources:
limits:
cpus: "0.2"
memory: 64M
# ─── Application API ────────────────────────────────────────────────────────
api:
image: myapp-api:${APP_IMAGE_TAG:-latest}
build:
context: .
dockerfile: Dockerfile
target: production
<<: *default-restart
environment:
NODE_ENV: production
PORT: 3000
DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
REDIS_URL: redis://:${REDIS_PASSWORD}@redis:6379
networks:
- frontend_net
- backend_net
depends_on:
db:
condition: service_healthy
redis:
condition: service_healthy
migrations:
condition: service_completed_successfully
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:3000/health || exit 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
logging: *default-logging
deploy:
resources:
limits:
cpus: "0.5"
memory: 512M
reservations:
cpus: "0.25"
memory: 256M
# ─── Database Migration Runner (one-shot service) ───────────────────────────
migrations:
image: myapp-api:${APP_IMAGE_TAG:-latest}
build:
context: .
dockerfile: Dockerfile
target: production
command: ["npm", "run", "db:migrate"]
environment:
DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
networks:
- backend_net
depends_on:
db:
condition: service_healthy
# No restart -- this is a one-shot task. It exits 0 on success, non-zero on failure.
restart: "no"
# ─── PostgreSQL Database ─────────────────────────────────────────────────────
db:
image: postgres:15.4-alpine3.18
<<: *default-restart
environment:
POSTGRES_DB: ${POSTGRES_DB:-myapp_dev}
POSTGRES_USER: ${POSTGRES_USER:-myapp}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD must be set}
volumes:
- postgres_data:/var/lib/postgresql/data
# Init scripts run only on first DB initialization
- ./db/init:/docker-entrypoint-initdb.d:ro
networks:
- backend_net
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-myapp} -d ${POSTGRES_DB:-myapp_dev}"]
interval: 5s
timeout: 5s
retries: 10
start_period: 30s
logging: *default-logging
deploy:
resources:
limits:
cpus: "1.0"
memory: 1G
reservations:
cpus: "0.5"
memory: 512M
# ─── Redis Cache ─────────────────────────────────────────────────────────────
redis:
image: redis:7.2.3-alpine
<<: *default-restart
command: >
redis-server
--requirepass ${REDIS_PASSWORD:-}
--maxmemory 256mb
--maxmemory-policy allkeys-lru
--save ""
volumes:
- redis_data:/data
networks:
- backend_net
healthcheck:
test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD:-}", "ping"]
interval: 3s
timeout: 3s
retries: 5
logging: *default-logging
deploy:
resources:
limits:
cpus: "0.25"
memory: 256Mdocker-compose.override.yml)# docker-compose.override.yml
# Applied automatically by `docker compose up` in development.
# Adds hot-reload, debug ports, and developer tooling.
services:
api:
build:
dockerfile: Dockerfile.dev # Dev image includes nodemon, dev deps
target: development
command: ["npm", "run", "dev"] # Hot-reload entrypoint
environment:
NODE_ENV: development
LOG_LEVEL: debug
volumes:
# Mount source for hot-reload; exclude node_modules to use container's copy
- ./src:/app/src:cached
- /app/node_modules # Anonymous volume prevents host overwriting container deps
ports:
- "9229:9229" # Node.js inspector/debugger port
db:
ports:
- "${POSTGRES_EXPOSED_PORT:-5432}:5432" # Expose for local DB GUI tools
redis:
ports:
- "6379:6379" # Expose for local Redis clients
# ─── Developer Tooling (profile-gated -- start with --profile tooling) ───────
pgadmin:
image: dpage/pgadmin4:7.8
profiles:
- tooling
environment:
PGADMIN_DEFAULT_EMAIL: dev@localhost
PGADMIN_DEFAULT_PASSWORD: admin
ports:
- "5050:80"
networks:
- backend_net
depends_on:
db:
condition: service_healthy
mailhog:
image: mailhog/mailhog:v1.0.1
profiles:
- tooling
ports:
- "1025:1025" # SMTP
- "8025:8025" # Web UI
networks:
- frontend_net| Pattern | Use When | Avoid When | Complexity |
|---|---|---|---|
| Single file | ≤4 services, 1 environment | Multiple environments, team projects | Low |
| Base + override | Team project, 2+ envs | Very simple local tooling | Medium |
| Profile-based | Optional services (tools, test DBs) | All services required always | Low-Medium |
| Include-based | Sub-team service ownership, monorepo | Older CI (check Compose version) | Medium-High |
| Secrets file mount | Any credential in staging/prod | Dev secrets (overkill) | Low |
| YAML anchors | Repeated config blocks (3+ services) | 1-2 repetitions (just copy) | Low |
NEVER use condition: service_started for database dependencies. This is the default behavior and it does not wait for the database to accept connections. Always define a healthcheck on the database service and use condition: service_healthy. Failure to do this causes race-condition startup failures that are intermittent and hard to debug.
NEVER hardcode credentials in docker-compose.yml. Use ${VARIABLE:?error message} syntax for required secrets -- the :? syntax causes Compose to fail immediately with a clear error if the variable is unset, rather than starting with an empty password.
ALWAYS set restart: "no" explicitly on one-shot services (migration runners, seed scripts, test runners). If omitted, Compose defaults may cause the service to restart in a loop after it exits, consuming resources and polluting logs.
NEVER mount the entire project root into a container. volumes: - .:/app mounts .git/, .env, credentials, and IDE files into the container. Mount only the directories the service needs (./src:/app/src). Use explicit anonymous volumes to protect container-specific directories like node_modules from being overwritten by host mounts.
ALWAYS specify image versions with a specific tag, never latest. latest is not a version -- it is an alias that changes when the upstream maintainer decides to update it. Unpinned images cause environments to silently diverge. Pin to a version + variant combination like postgres:15.4-alpine3.18.
NEVER put both image: and build: in the same service definition without understanding the semantics. When both are present, image: specifies the name to tag the built image with. docker compose up will not automatically pull -- it will use the local image. In CI, explicitly run docker compose build before docker compose up to ensure the image is current.
ALWAYS configure log rotation. Without logging.options.max-size and max-file, a verbose service can fill a disk within hours. Every service that might produce high log volume (any application container) must have log limits configured.
NEVER use the same Compose file for both development and production without overrides. Development needs bind-mounted source code, exposed debug ports, and verbose logging. Production needs immutable images, no exposed ports, and structured log output. Conflating these in a single file forces compromises that make both environments worse.
ALWAYS validate Compose files before committing. Run docker compose config to merge and validate the full resolved configuration (including all -f override files). This catches YAML syntax errors, undefined variables, and merge conflicts. Add this as a CI lint step.
NEVER create a data service without an explicit named volume. Anonymous volumes (no name in the volumes: top-level section) are assigned a hash ID and are not pruned by standard docker volume prune -- they accumulate silently. Named volumes are visible in docker volume ls, can be backed up by name, and survive docker compose down without data loss.
Use YAML anchors (&anchor / *anchor / <<:) for configuration fragments repeated 3 or more times, but document what the anchor contains. Common candidates: logging configuration, restart policy, environment variable sets. Do NOT anchor entire service definitions -- this makes service configs hard to read independently.
ALWAYS set memory limits on JVM-based services and configure heap size to 75--80% of the container limit. A JVM without --memory constraints will grow to consume all available host memory because it uses the host's total RAM as its sizing reference, not the container's cgroup limit. Use -XX:MaxRAMPercentage=75.0 (JDK 11+) to make heap sizing automatic and container-aware.
On Linux, container processes run as specific UIDs, but named volumes are created with root ownership by default. A service running as uid=1000 (common for Node.js or Python images) cannot write to a volume owned by uid=0.
Resolution approach:
Option A (entrypoint fix): add an entrypoint script that runs chown -R appuser:appgroup /data before launching the main process. This requires the entrypoint to briefly run as root (user: root at container start), then exec su-exec appuser <main_command>
Option B (init container): add a companion one-shot service with user: root that runs chown on the volume, then use depends_on: condition: service_completed_successfully to gate the main service
Option C (tmpfs for ephemeral needs): if the data is not persistent, use tmpfs: /data instead of a volume -- tmpfs is always writable by the container user
Avoid: changing volume permissions by making the container run as root. This removes a security boundary and is a vulnerability in shared environments.
When Docker creates bridge networks, it assigns subnets from the 172.16.0.0/12 range by default. Many corporate VPNs and office networks also use this range, causing routing conflicts where Docker traffic is sent through the VPN tunnel instead of the local bridge.
Resolution approach:
Explicitly configure network subnets outside the conflicting range:
networks:
backend_net:
driver: bridge
ipam:
config:
- subnet: 192.168.200.0/24For a team-wide issue, configure Docker's default address pool in /etc/docker/daemon.json:
{ "default-address-pools": [{"base":"10.200.0.0/16","size":24}] }Document the chosen subnets in the project README so future developers don't reassign conflicting ranges
JVM services may take 30--90 seconds to start and bind their health endpoint. The default healthcheck.start_period of 0s means the first few health check failures count against retries, potentially killing the service before it has a chance to start.
Resolution approach:
Set start_period to 110% of the measured cold-start time. For a JVM service that takes 60 seconds to start in CI: start_period: 75s
Use a lightweight health endpoint (/actuator/health for Spring, /health for most frameworks) that does not do deep dependency checks on startup -- save dependency checks for readiness probes
In development, set interval: 10s to reduce CPU overhead from repeated health checks during startup; in CI, use interval: 5s for faster feedback
Consider adding a JAVA_TOOL_OPTIONS: "-XX:TieredStopAtLevel=1" environment variable in the dev override to disable JIT compilation during development, reducing startup time by 50% at the cost of throughput
CI environments introduce constraints not present in development: no persistent layer cache between runs (unless configured), no Docker Desktop, potentially rootless Docker, and the need for deterministic teardown.
Resolution approach:
Explicitly pull base images before building: docker compose pull --ignore-pull-failures caches image layers, then docker compose build uses the cached layers
For GitHub Actions: use actions/cache with the Docker layer cache exporter: type=gha,mode=max
Always run cleanup: docker compose down -v --remove-orphans as a CI teardown step, even on failure (use if: always() in GitHub Actions). The -v flag removes anonymous volumes that would otherwise accumulate and fill the CI runner disk
Use docker compose run --rm service_name test_command to run tests in the Compose network, then exit without leaving a stopped container behind
Set COMPOSE_PROJECT_NAME to a CI-run-specific value (e.g., the PR number) when running multiple simultaneous CI jobs on the same host to prevent project-name collisions
Docker Desktop on macOS runs containers in a Linux VM. Bind mounts that cross the macOS/VM boundary are notoriously slow for I/O-intensive workloads. A Node.js project with 50,000 files in node_modules performing module resolution on every hot-reload can be 10x slower than the same workload on native Linux.
Resolution approach:
Never mount node_modules, __pycache__, .venv, or similar dependency directories from the host. Use anonymous volumes to let these directories live inside the container:
volumes:
- ./src:/app/src:cached
- /app/node_modules # This anonymous volume shields node_modules from host I/OUse VirtioFS (enabled in Docker Desktop Settings > General) instead of the legacy gRPC FUSE file sharing -- VirtioFS delivers 2--3x better I/O throughput for bind mounts on macOS
For Python projects, use consistency: cached on all read-only source mounts
As a last resort, consider using docker compose watch (Docker Compose 2.22+), which syncs only changed files from host to container rather than mounting an entire directory
Running migrations as a Compose service is powerful but introduces failure modes if not handled carefully.
Resolution approach:
The migration service must be idempotent -- running it twice must produce the same result as running it once. All major migration frameworks (Flyway, Liquibase, Alembic, Sequelize) are idempotent by default; verify your tool maintains this guarantee
If the migration fails (exits non-zero), the dependent API service must not start. With condition: service_completed_successfully, Compose enforces this correctly
During development, migrations run on every docker compose up. In staging/prod, trigger migrations as a separate deploy step before rolling out new application images
Never run migrations with the same database user as the application. Create a migrator user with DDL privileges and a appuser with only DML (SELECT, INSERT, UPDATE, DELETE). This limits the blast radius of SQL injection vulnerabilities
Developers on Apple Silicon (M1/M2/M3) Macs run an ARM64 Linux VM. Many legacy or enterprise images are only built for linux/amd64, causing significant performance degradation through QEMU emulation (commonly 5--10x slower CPU performance).
Resolution approach:
Always specify the platform explicitly for images that don't have ARM64 variants to make the performance tradeoff visible and intentional:
platform: linux/amd64 # Forces emulation on Apple Silicon -- known performance costFor in-house application images, build multi-arch images using docker buildx build --platform linux/amd64,linux/arm64 --push. This eliminates the emulation penalty
Use the DOCKER_DEFAULT_PLATFORM=linux/amd64 environment variable as a developer-machine workaround when many services lack ARM64 images, so you don't need to annotate each service
When a service is slow in development on Apple Silicon and fast on Linux CI, suspect platform emulation before optimizing application code
Input: "I'm building a SaaS web application with a React frontend, a Node.js/Express API, PostgreSQL, and Redis. I have a team of 5 developers. We need dev, staging, and prod configurations. Some developers use macOS M2, some use Linux. We've been struggling with database race conditions on startup and slow hot-reload on macOS."
Output:
This project maps to the base + override pattern with YAML anchors for shared configuration. The two specific problems reported -- database race conditions and macOS hot-reload performance -- both have known, specific solutions.
Root cause of race conditions: the previous Compose file almost certainly used depends_on without condition: service_healthy. The fix is to add a proper PostgreSQL health check and wire the API's depends_on to condition: service_healthy.
Root cause of macOS hot-reload slowness: the node_modules directory was likely included in the bind mount. Shielding it with an anonymous volume will resolve this.
myapp/
├── docker-compose.yml # Base: service definitions, networks, volumes
├── docker-compose.override.yml # Dev: auto-applied, hot-reload, exposed ports
├── docker-compose.prod.yml # Prod-like: immutable images, no debug ports
├── docker-compose.ci.yml # CI: parallel-safe, deterministic teardown
├── .env.example # Template for all required variables
├── Makefile # Developer convenience commands
├── Dockerfile # Production multi-stage build
├── Dockerfile.dev # Development build with nodemon
└── nginx/
└── nginx.conf # Reverse proxy configuration.env.example# ─── Application ───────────────────────────────────────────────────────────────
APP_IMAGE_TAG=latest
API_PORT=3000
NODE_ENV=development
# ─── Database ──────────────────────────────────────────────────────────────────
POSTGRES_DB=myapp_dev
POSTGRES_USER=myapp
# REQUIRED: set this in your local .env -- never commit a real value
POSTGRES_PASSWORD=changeme_for_local_dev
# Dev only: exposed port for DB GUI tools (TablePlus, DataGrip)
POSTGRES_EXPOSED_PORT=5433
# ─── Cache ─────────────────────────────────────────────────────────────────────
REDIS_PASSWORD=
# ─── Proxy ─────────────────────────────────────────────────────────────────────
NGINX_HOST_PORT=80docker-compose.yml (Base)# docker-compose.yml
# Base configuration shared across all environments.
# Do NOT run this file directly -- use docker compose up (applies override.yml)
# or: docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
version: "3.9"
# ─────────────────────────────────────────────────────────────────────────────
# YAML Anchors -- reused configuration fragments
# ─────────────────────────────────────────────────────────────────────────────
x-logging: &logging
driver: json-file
options:
max-size: "10m"
max-file: "3"
x-api-base: &api-base
image: myapp-api:${APP_IMAGE_TAG:-latest}
build:
context: .
dockerfile: Dockerfile
environment:
PORT: ${API_PORT:-3000}
DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
REDIS_URL: redis://:${REDIS_PASSWORD:-}@redis:6379
networks:
- frontend_net
- backend_net
logging: *logging
volumes:
postgres_data:
driver: local
redis_data:
driver: local
networks:
frontend_net:
driver: bridge
backend_net:
driver: bridge
internal: true # Databases have no internet access
services:
nginx:
image: nginx:1.25.3-alpine
restart: unless-stopped
ports:
- "${NGINX_HOST_PORT:-80}:80"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
networks:
- frontend_net
depends_on:
api:
condition: service_healthy
logging: *logging
deploy:
resources:
limits: { cpus: "0.2", memory: 64M }
api:
<<: *api-base
restart: unless-stopped
depends_on:
db:
condition: service_healthy # FIX: replaces the previous bare depends_on
redis:
condition: service_healthy
migrations:
condition: service_completed_successfully
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:${API_PORT:-3000}/health || exit 1"]
interval: 10s
timeout: 5s
retries: 5
start_period: 30s
deploy:
resources:
limits: { cpus: "0.5", memory: 512M }
reservations: { cpus: "0.25", memory: 256M }
migrations:
<<: *api-base
command: ["npm", "run", "db:migrate"]
restart: "no" # One-shot: exits 0 on success, blocks api startup on failure
depends_on:
db:
condition: service_healthy
# Migrations only need backend_net -- override the anchor's network list
networks:
- backend_net
db:
image: postgres:15.4-alpine3.18
restart: unless-stopped
environment:
POSTGRES_DB: ${POSTGRES_DB:-myapp_dev}
POSTGRES_USER: ${POSTGRES_USER:-myapp}
# :? causes immediate failure with clear error if POSTGRES_PASSWORD is unset
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in your .env file}
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- backend_net
healthcheck:
# FIX: proper readiness check that resolves the startup race condition
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-myapp} -d ${POSTGRES_DB:-myapp_dev}"]
interval: 5s
timeout: 5s
retries: 10
start_period: 30s
logging: *logging
deploy:
resources:
limits: { cpus: "1.0", memory: 1G }
reservations: { cpus: "0.5", memory: 512M }
redis:
image: redis:7.2.3-alpine
restart: unless-stopped
command: >
redis-server
--requirepass "${REDIS_PASSWORD:-}"
--maxmemory 256mb
--maxmemory-policy allkeys-lru
--save ""
volumes:
- redis_data:/data
networks:
- backend_net
healthcheck:
test: ["CMD-SHELL", "redis-cli -a '${REDIS_PASSWORD:-}' ping | grep PONG"]
interval: 3s
timeout: 3s
retries: 5
logging: *logging
deploy:
resources:
limits: { cpus: "0.25", memory: 256M }docker-compose.override.yml (Development -- auto-applied)# docker-compose.override.yml
# Applied automatically in development.
# macOS M2 hot-reload fix: node_modules shielded by anonymous volume.
services:
api:
build:
dockerfile: Dockerfile.dev
command: ["npm", "run", "dev"] # nodemon with --watch
environment:
NODE_ENV: development
LOG_LEVEL: debug
volumes:
# FIX for macOS hot-reload: mount only src/, shield node_modules
- ./src:/app/src:cached # cached consistency = better macOS I/O
- ./package.json:/app/package.json:ro
- /app/node_modules # Anonymous volume -- container's node_modules, not host's
ports:
- "9229:9229" # Node.js debugger (Chrome DevTools / VS Code)
db:
ports:
# Expose DB to host so developers can connect with TablePlus, DataGrip, etc.
- "${POSTGRES_EXPOSED_PORT:-5433}:5432"
redis:
ports:
- "6379:6379"
# ─── Optional tooling -- start with: docker compose --profile tooling up ───
pgadmin:
image: dpage/pgadmin4:7.8
profiles: [tooling]
environment:
PGADMIN_DEFAULT_EMAIL: dev@localhost
PGADMIN_DEFAULT_PASSWORD: admin
ports:
- "5050:80"
networks:
- backend_net
depends_on:
db:
condition: service_healthyMakefile# Makefile -- developer convenience commands
# Usage: make up | make down | make logs | make migrate | make shell SERVICE=api
.PHONY: up down logs restart migrate shell test lint validate
# Start full dev stack
up:
docker compose up --build
# Start silently in background
up-d:
docker compose up --build -d
# Stop and remove containers (keep volumes)
down:
docker compose down --remove-orphans
# Stop and wipe all data volumes (full reset)
reset:
docker compose down -v --remove-orphans
# Tail logs (all services, or: make logs SERVICE=api)
logs:
docker compose logs -f $(SERVICE)
# Run database migrations manually
migrate:
docker compose run --rm migrations
# Open a shell in a running service
shell:
docker compose exec $(or $(SERVICE),api) sh
# Run tests inside the container
test:
docker compose run --rm api npm test
# Validate the merged Compose configuration (catches YAML errors and unset vars)
validate:
docker compose config --quiet && echo "Compose config is valid"
# Lint the Compose files
lint:
docker compose -f docker-compose.yml -f docker-compose.prod.yml config --quietAfter implementing these configurations, verify the following:
| Check | Command | Expected |
|---|---|---|
| Startup race condition resolved | docker compose up 3 times | API starts only after DB is healthy every time |
| macOS hot-reload works | Edit a .js file in src/ | Service reloads in <2 seconds |
| Volumes are named | docker volume ls | myapp_postgres_data, myapp_redis_data visible |
| No credential leakage | docker inspect myapp-api | No raw passwords in Env array |
| Log limits active | docker inspect myapp-api | grep LogConfig | max-size: 10m present |
| Backend network is internal | docker network inspect myapp_backend_net | "Internal": true |
| Config validates | make validate | No errors, no warnings |
SKILL.md
927 lines · 5,360 words
The text of SKILL.md is shown as it downloads, under the Apache-2.0 license, by Ferrox Labs. The license is the one in the file's own frontmatter.
Your new bot
Docker Compose Patterns bot
Waiting in your sidebar once you say go.
No account needed · Runs on the AI plan you already pay for