Production deployment workflows and CI/CD best practices.
When to activate
- Set up CI/CD Pipelines
- Dockerize the application
- Plan deployment strategies (blue-green deployment, canary deployment, rolling updates)
- Implement Health Checks and Readiness Probes
- Prepare for production release
- Configure environment-specific settings
Deployment Strategies
Rolling Deployment - Default
Gradually replace instances — during a rolling update, the old and new versions run simultaneously.
实例 1: v1 → v2 (首先更新)
实例 2: v1 (仍在运行 v1)
实例 3: v1 (仍在运行 v1)
实例 1: v2
实例 2: v1 → v2 (其次更新)
实例 3: v1
实例 1: v2
实例 2: v2
实例 3: v1 → v2 (最后更新)
Pros: Zero downtime, gradual rollout Cons: Two versions run simultaneously — changes must be backward compatible Use cases: Standard deployments, backward-compatible changes
Blue-Green Deployment
Run two identical environments. Switch traffic atomically.
蓝色环境 (v1) ← 流量接入
绿色环境 (v2) 空闲，运行新版本
# 验证后：
蓝色环境 (v1) 空闲 (变为备用)
绿色环境 (v2) ← 流量接入
Pros: Instant rollback (switch back to the blue environment), clean cutover Cons: Requires 2x infrastructure resources during deployment Use cases: Critical services, zero tolerance for issues
Canary Deployment
First route a small percentage of traffic to the new version.
v1: 95% 流量
v2: 5% 流量 (金丝雀)
# 如果指标良好：
v1: 50% 流量
v2: 50% 流量
# 最终：
v2: 100% 流量
Pros: Detect issues with real traffic before a full rollout Cons: Requires traffic splitting infrastructure and monitoring Use cases: High-traffic services, high-risk changes, Feature Flags
Docker
Multi-stage Dockerfile (Node.js)
# 阶段 1: 安装依赖
FROM node:22-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --production=false
# 阶段 2: 构建
FROM node:22-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npm run build
RUN npm prune --production
# 阶段 3: 生产镜像
FROM node:22-alpine AS runner
WORKDIR /app
RUN addgroup -g 1001 -S appgroup && adduser -S appuser -u 1001
USER appuser
COPY --from=builder --chown=appuser:appgroup /app/node_modules ./node_modules
COPY --from=builder --chown=appuser:appgroup /app/dist ./dist
COPY --from=builder --chown=appuser:appgroup /app/package.json ./
ENV NODE_ENV=production
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1
CMD ["node", "dist/server.js"]
Multi-stage Dockerfile (Go)
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags="-s -w" -o /server ./cmd/server
FROM alpine:3.19 AS runner
RUN apk --no-cache add ca-certificates
RUN adduser -D -u 1001 appuser
USER appuser
COPY --from=builder /server /server
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s CMD wget -qO- http://localhost:8080/health || exit 1
CMD ["/server"]
Multi-stage Dockerfile (Python/Django)
FROM python:3.12-slim AS builder
WORKDIR /app
RUN pip install --no-cache-dir uv
COPY requirements.txt .
RUN uv pip install --system --no-cache -r requirements.txt
FROM python:3.12-slim AS runner
WORKDIR /app
RUN useradd -r -u 1001 appuser
USER appuser
COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=builder /usr/local/bin /usr/local/bin
COPY . .
ENV PYTHONUNBUFFERED=1
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health/')" || exit 1
CMD ["gunicorn", "config.wsgi:application", "--bind", "0.0.0.0:8000", "--workers", "4"]
Docker Best Practices
# 推荐做法 (GOOD practices)
- 使用具体的版本标签 (node:22-alpine, 而非 node:latest)
- 使用多阶段构建以最小化镜像大小
- 以非 root 用户运行
- 首先复制依赖文件（利用层缓存）
- 使用 .dockerignore 排除 node_modules, .git, tests
- 添加 HEALTHCHECK 指令
- 在 docker-compose 或 k8s 中设置资源限制
# 错误做法 (BAD practices)
- 以 root 用户运行
- 使用 :latest 标签
- 在一个 COPY 层中复制整个仓库
- 在生产镜像中安装开发依赖
- 在镜像中存储机密信息（应使用环境变量或机密管理器）
CI/CD Pipeline
GitHub Actions (standard pipeline)
name: CI/CD
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- run: npm ci
- run: npm run lint
- run: npm run typecheck
- run: npm test -- --coverage
- uses: actions/upload-artifact@v4
if: always()
with:
name: coverage
path: coverage/
build:
needs: test
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
steps:
- uses: actions/checkout@v4
- uses: docker/setup-buildx-action@v3
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/build-push-action@v5
with:
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
deploy:
needs: build
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main'
environment: production
steps:
- name: Deploy to production
run: |
# 平台特定的部署命令
# Railway: railway up
# Vercel: vercel --prod
# K8s: kubectl set image deployment/app app=ghcr.io/${{ github.repository }}:${{ github.sha }}
echo "Deploying ${{ github.sha }}"
Pipeline Stages
PR 开启:
代码扫描 (lint) → 类型检查 (typecheck) → 单元测试 → 集成测试 → 预览部署
合并到 main:
代码扫描 (lint) → 类型检查 (typecheck) → 单元测试 → 集成测试 → 构建镜像 → 部署到预发环境 → 冒烟测试 → 部署到生产环境
Health Checks
Health check endpoint
// 简单健康检查
app.get("/health", (req, res) => {
res.status(200).json({ status: "ok" });
});
// 详细健康检查 (用于内部监控)
app.get("/health/detailed", async (req, res) => {
const checks = {
database: await checkDatabase(),
redis: await checkRedis(),
externalApi: await checkExternalApi(),
};
const allHealthy = Object.values(checks).every(c => c.status === "ok");
res.status(allHealthy ? 200 : 503).json({
status: allHealthy ? "ok" : "degraded",
timestamp: new Date().toISOString(),
version: process.env.APP_VERSION || "unknown",
uptime: process.uptime(),
checks,
});
});
async function checkDatabase(): Promise<HealthCheck> {
try {
await db.query("SELECT 1");
return { status: "ok", latency_ms: 2 };
} catch (err) {
return { status: "error", message: "Database unreachable" };
}
}
Kubernetes Probes
livenessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 10
periodSeconds: 30
failureThreshold: 3
readinessProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 2
startupProbe:
httpGet:
path: /health
port: 3000
initialDelaySeconds: 0
periodSeconds: 5
failureThreshold: 30 # 30 * 5s = 150s 最大启动时间
Environment Configuration
Cloud-native application (Twelve-Factor App) patterns
# 所有配置通过环境变量传递 —— 严禁写死在代码中
DATABASE_URL=postgres://user:pass@host:5432/db
REDIS_URL=redis://host:6379/0
API_KEY=${API_KEY} # 由机密管理器注入
LOG_LEVEL=info
PORT=3000
# 环境特定行为
NODE_ENV=production # 或 staging, development
APP_ENV=production # 显式应用环境
Configuration validation
import { z } from "zod";
const envSchema = z.object({
NODE_ENV: z.enum(["development", "staging", "production"]),
PORT: z.coerce.number().default(3000),
DATABASE_URL: z.string().url(),
REDIS_URL: z.string().url(),
JWT_SECRET: z.string().min(32),
LOG_LEVEL: z.enum(["debug", "info", "warn", "error"]).default("info"),
});
// 启动时验证 —— 如果配置错误则立即崩溃 (fail fast)
export const env = envSchema.parse(process.env);
Rollback Strategy
Instant rollback
# Docker/Kubernetes: 指向之前的镜像
kubectl rollout undo deployment/app
# Vercel: 提升之前的部署版本
vercel rollback
# Railway: 重新部署之前的提交
railway up --commit <previous-sha>
# 数据库: 回滚迁移 (如果可逆)
npx prisma migrate resolve --rolled-back <migration-name>
Rollback checklist
- Previous images/artifacts are available and tagged
- Database migrations are backward compatible (no breaking changes)
- Feature flags can disable new features without deploying
- Monitoring alerts are configured for spikes in error rates
- Rollback has been tested in staging before production release
Production Readiness Checklist
Before any production deployment:
Application
- All tests pass (unit tests, integration tests, E2E)
- No hardcoded secrets in code or configuration files
- Error handling covers all edge cases
- Logs are structured (JSON) and contain no personally identifiable information (PII)
- Health check endpoint returns meaningful status
Infrastructure
- Docker image builds are reproducible (pinned versions)
- Environment variables are documented and validated at startup
- Resource limits are set (CPU, memory)
- Horizontal scaling is configured (min/max instances)
- SSL/TLS is enabled for all endpoints
Monitoring
- Application metrics are exported (request rate, latency, error rate)
- Alerts are configured for error rate > threshold
- Log aggregation is set up (structured logs, searchable)
- Availability (Uptime) monitoring is in place for health check endpoints
Security
- CVE vulnerability scanning is performed on dependencies
- CORS is configured for allowed origins
- Rate limiting is enabled for public endpoints
- Authentication and Authorization are verified
- Security response headers are set (CSP, HSTS, X-Frame-Options)
Operations
- The rollback plan is documented and tested
- Database migrations have been tested against production-scale data
- Runbook for common failure scenarios
- On-call rotation and escalation paths are defined