shipping-and-launch

Release and launch

Overview

Launch with confidence. The goal is not just to deploy, but to deploy safely: with monitoring in place, a rollback plan ready, and a clear idea of what success looks like. Every launch should be reversible, observable, and incremental.

When to use

Deploying a feature to production for the first time
Releasing major changes to users
Migrating data or infrastructure
Opening a beta or early access program
Any risky deployment (that is, all deployments)

Pre-release checklist

Code quality

All tests pass (unit, integration, e2e)
Build succeeds with no warnings
Lint and type checks pass
Code has been reviewed and approved
No TODO comments that should be resolved before launch
No console.log debug statements
Error handling covers expected failure modes

Security

No secrets in code or version control
Dependency audit for the ecosystem (npm audit、pip-audit、cargo audit etc.) has no critical or high vulnerabilities
All user-accessible endpoints have input validation
Authentication and authorization checks are in place
Security headers are configured (CSP, HSTS, etc.)
Rate limiting is set on authentication endpoints
CORS is configured for specific origins (not wildcards)

Performance

Core Web Vitals are within the "Good" thresholds
No N+1 queries in critical paths
Images are optimized (compressed, responsive sizes, lazy loading)
Bundle size is within budget
Database queries have appropriate indexes
Caching is configured for static assets and repeated queries

Accessibility

All interactive elements support keyboard navigation
Screen readers can convey page content and structure
Color contrast meets WCAG 2.1 AA (4.5:1 for text)
Focus management is correct for modals and dynamic content
Error messages are descriptive and associated with form fields
No accessibility warnings in axe-core or Lighthouse

Infrastructure

Production environment variables are set
Database migrations are applied (or ready to apply)
DNS and SSL are configured
CDN is configured for static assets
Logging and error reporting are configured
Health check endpoint exists and responds

Documentation

README has been updated with any new setup requirements
API docs are up to date
Any architectural decisions have an ADR
Changelog has been updated
User-facing documentation has been updated (if applicable)

Feature flag strategy

Release via feature flags to decouple deployment and release:

// Feature flag check
const flags = await getFeatureFlags(userId);

if (flags.taskSharing) {
  // New feature: task sharing
  return <TaskSharingPanel task={task} />;
}

// Default: existing behavior
return null;

Feature flag lifecycle:

1. DEPLOY with flag OFF     → Code is in production but inactive
2. ENABLE for team/beta     → Internal testing in production environment
3. GRADUAL ROLLOUT          → 5% → 25% → 50% → 100% of users
4. MONITOR at each stage    → Watch error rates, performance, user feedback
5. CLEAN UP                 → Remove flag and dead code path after full rollout

Rules:

Each feature flag has an owner and an expiration date
Clean up flags within 2 weeks after full rollout
Do not nest feature flags (creates exponential combinations)
Test both flag states (on and off) in CI

Phased rollout

Rollout sequence

1. DEPLOY to staging
   └── Full test suite in staging environment
   └── Manual smoke test of critical flows

2. DEPLOY to production (feature flag OFF)
   └── Verify deployment succeeded (health check)
   └── Check error monitoring (no new errors)

3. ENABLE for team (flag ON for internal users)
   └── Team uses the feature in production
   └── 24-hour monitoring window

4. CANARY rollout (flag ON for 5% of users)
   └── Monitor error rates, latency, user behavior
   └── Compare metrics: canary vs. baseline
   └── 24-48 hour monitoring window
   └── Advance only if all thresholds pass (see table below)

5. GRADUAL increase (25% -> 50% -> 100%)
   └── Same monitoring at each step
   └── Ability to roll back to previous percentage at any point

6. FULL rollout (flag ON for all users)
   └── Monitor for 1 week
   └── Clean up feature flag

Rollout decision thresholds

Use these thresholds to determine whether each stage should proceed, pause for investigation, or roll back:

Metric	Proceed (green)	Pause and investigate (yellow)	Roll back (red)
Error rate	Within 10% of baseline	10-100% above baseline	>2x baseline
P95 latency	Within 20% of baseline	20-50% above baseline	>50% above baseline
Client JS errors	No new error types	New errors occur in <0.1% of sessions	New errors occur in >0.1% of sessions
Business metrics	Neutral or positive	Drop <5% (possibly noise)	Drop >5%

When to roll back

Roll back immediately if any of the following occur:

Error rate increases to more than 2x baseline
P95 latency increases by more than 50%
Surge in user-reported issues
Data integrity issue detected
Security vulnerability discovered

Monitoring and observability

What to monitor

Application metrics:
├── Error rate (total and by endpoint)
├── Response time (p50, p95, p99)
├── Request volume
├── Active users
└── Key business metrics (conversion, engagement)

Infrastructure metrics:
├── CPU and memory utilization
├── Database connection pool usage
├── Disk space
├── Network latency
└── Queue depth (if applicable)

Client metrics:
├── Core Web Vitals (LCP, INP, CLS)
├── JavaScript errors
├── API error rates from client perspective
└── Page load time

Error Reporting

// Set up error boundary with reporting
class ErrorBoundary extends React.Component {
  componentDidCatch(error: Error, info: React.ErrorInfo) {
    // Report to error tracking service
    reportError(error, {
      componentStack: info.componentStack,
      userId: getCurrentUser()?.id,
      page: window.location.pathname,
    });
  }

  render() {
    if (this.state.hasError) {
      return <ErrorFallback onRetry={() => this.setState({ hasError: false })} />;
    }
    return this.props.children;
  }
}

// Server-side error reporting
app.use((err: Error, req: Request, res: Response, next: NextFunction) => {
  reportError(err, {
    method: req.method,
    url: req.url,
    userId: req.user?.id,
  });

  // Don't expose internals to users
  res.status(500).json({
    error: { code: 'INTERNAL_ERROR', message: 'Something went wrong' },
  });
});

Post-launch verification

Within the first hour after launch:

1. Check health endpoint returns 200
2. Check error monitoring dashboard (no new error types)
3. Check latency dashboard (no regression)
4. Test the critical user flow manually
5. Verify logs are flowing and readable
6. Confirm rollback mechanism works (dry run if possible)

Error Budget release gate

The service's error budget (the percentage of requests or time that the SLO allows to fail) determines whether it is safe to release now. Treat it as an objective gate, not a bargaining chip:

Budget remaining > 20%  →  Ship normally; monitor closely
Budget remaining 0–20%  →  Slow rollouts only; no high-risk changes
Budget exhausted        →  Freeze feature work; focus entirely on reliability
Budget resets           →  Resume normal pace; bake in the fix that recovered it

A high burn rate during canary (consuming budget faster than the baseline pace) is thePausesignal in the rollout decision thresholds table above: handle it the same as a rise in error rate.

Rollback strategy

Every deployment needs a rollback plan before it happens:

## Rollback Plan for [Feature/Release]

### Trigger Conditions
- Error rate > 2x baseline
- P95 latency > [X]ms
- User reports of [specific issue]

### Rollback Steps
1. Disable feature flag (if applicable)
   OR
1. Deploy previous version: `git revert <commit> && git push`
2. Verify rollback: health check, error monitoring
3. Communicate: notify team of rollback

### Database Considerations
- Migration [X] has a rollback: `npx prisma migrate rollback`
- Data inserted by new feature: [preserved / cleaned up]

### Time to Rollback
- Feature flag: < 1 minute
- Redeploy previous version: < 5 minutes
- Database rollback: < 15 minutes

Common rationalization excuses

Rationalization excuse	Reality
"It works in staging, so it will work in production"	Production has different data, traffic patterns, and edge cases. Monitor after deployment.
"This doesn't need feature flags"	Every feature benefits from a kill switch. Even "simple" changes can break things.
"Monitoring is extra overhead"	Without monitoring, you'll find out about problems from user complaints instead of a dashboard.
"Add monitoring later"	Add it before launch. You can't debug what you can't see.
"Rolling back is admitting failure"	Rolling back is responsible engineering practice. Shipping a bad feature is the failure.
"The error rate looks normal, keep shipping"	Look at the burn rate, not just the current error rate. Consuming budget faster than the baseline is a pause signal, even if each individual threshold is green.

Danger signals

Deploying without a rollback plan
No monitoring or error reporting in production
Big-bang releases (releasing everything at once, with no staging)
Feature flags with no expiration time or owner
No one monitoring during the first hour after deployment
Production configuration done from memory instead of from code
"It's Friday afternoon, let's ship it"
Error budget is exhausted, but feature development continues as usual

Validation

Before deployment:

Pre-release checklist is complete (all sections are green)
Feature flag is configured (if applicable)
Rollback plan is documented
Monitoring dashboards are set up
Team has been notified of the deployment

After deployment:

Health check returns 200
Error rate is normal
Latency is normal
Critical user flows are available
Logs are flowing normally
Rollback has been tested, or it has been verified that it can be executed at any time

For each deployed service:

Error budget policy is in place: clearly define what actions to take when the budget drops below 20% and when it is exhausted

About this skill