DEV Community

Cover image for How to Deploy Applications on AWS Without Downtime
OutworkTech
OutworkTech

Posted on

How to Deploy Applications on AWS Without Downtime

Every team reaches the same inflection point.

Early on, deployments happen late at night. SSH into the server, pull the latest code, restart the process, hope nothing breaks. Users are few enough that a 2-minute maintenance window goes unnoticed.

Then the product grows. Users are in different time zones. Revenue runs 24/7. That 2-minute window is now a 2-minute outage — and someone will notice.

Zero-downtime deployment on AWS is not a single technique. It's a set of strategies, each solving a different risk profile. This post explains all three, how to implement them on AWS specifically, and — critically — how to handle the part most guides skip: database migrations.


Why Downtime Happens in the First Place

Before picking a strategy, understand the root cause.

Downtime during deployments happens because of one of three things:

1. The old version is killed before the new one is ready.
You stop the running container, start the new one, and there's a gap where nothing is serving traffic.

2. The new version starts but isn't healthy yet.
The container is running but the app is still initializing — database connections warming up, caches loading, health checks not yet passing. Traffic hits it anyway.

3. A database migration breaks the running version.
You run a migration that removes a column or renames a field. The old application code, still running while the new one deploys, tries to read that column and throws an error.

Every zero-downtime strategy is solving one or more of these three problems. Keep that in mind as we go through each approach.


The Three Strategies

Strategy 1: Rolling Deployment

Rolling deployment replaces old instances with new ones gradually. At no point do you have zero instances running — the load balancer only routes traffic to healthy instances.

AWS supports zero downtime via ECS rolling deployments with minimum_healthy_percent=100%, CodeDeploy blue-green deployments, Application Load Balancer target group switching, and Kubernetes rolling updates on EKS.

How ECS rolling deployment works:

Four systems work on the deployment simultaneously: the ECS scheduler enforcing minimum and maximum task counts, the ECS agent provisioning the task and starting containers, the load balancer deciding whether the new target should receive traffic, and the old container finishing requests before ECS kills it.

Here's the ECS service configuration that makes rolling deployments safe:

aws ecs create-service \
  --cluster production \
  --service-name api-service \
  --task-definition api:latest \
  --desired-count 4 \
  --deployment-configuration \
    minimumHealthyPercent=100,maximumPercent=200 \
  --health-check-grace-period-seconds 120
Enter fullscreen mode Exit fullscreen mode

Two settings matter most here:

minimumHealthyPercent=100 — ECS will never drop below 100% of your desired task count during a deployment. It adds new tasks first, waits for them to pass health checks, then removes old ones. No gap in coverage.

maximumPercent=200 — allows ECS to temporarily run double the tasks during the transition. At 4 desired tasks, you might briefly have 8 running — 4 old, 4 new. Once the new ones are healthy, the old 4 drain and terminate.

health-check-grace-period-seconds=120 — gives the application 2 minutes to initialize before the load balancer starts checking its health. Without this, ECS might terminate a perfectly good container that's still warming up.

When rolling deployment is the right choice:

  • Standard feature releases with no breaking changes
  • Services running 3+ instances (rolling needs room to maneuver)
  • Teams that want simplicity without extra infrastructure cost

When it's not:

  • Breaking schema changes where old and new code can't coexist
  • High-risk releases where you want instant rollback without redeployment

Strategy 2: Blue-Green Deployment

Blue-green maintains two identical environments — blue (current) and green (new). You deploy to green, test it with real traffic, and switch the load balancer. Rollback is a traffic switch, not a redeployment.

AWS Elastic Beanstalk enables zero-downtime releases by maintaining two identical environments and performing a DNS CNAME swap. This strategy allows for a 30-second rollback without redeploying code or rebuilding containers.

On ECS with CodeDeploy, the implementation looks like this:

# appspec.yml — CodeDeploy configuration for ECS blue-green
version: 0.0
Resources:
  - TargetService:
      Type: AWS::ECS::Service
      Properties:
        TaskDefinition: <TASK_DEFINITION>
        LoadBalancerInfo:
          ContainerName: "api"
          ContainerPort: 8000
        PlatformVersion: "LATEST"

Hooks:
  - BeforeAllowTraffic: "arn:aws:lambda:us-east-1:123:function:validate-green"
  - AfterAllowTraffic: "arn:aws:lambda:us-east-1:123:function:smoke-test-green"
Enter fullscreen mode Exit fullscreen mode

The ALB target group switch in Terraform:

# Two target groups — one for blue, one for green
resource "aws_lb_target_group" "blue" {
  name     = "api-blue"
  port     = 8000
  protocol = "HTTP"
  vpc_id   = var.vpc_id

  health_check {
    path                = "/health"
    healthy_threshold   = 2
    unhealthy_threshold = 3
    timeout             = 5
    interval            = 10
  }
}

resource "aws_lb_target_group" "green" {
  name     = "api-green"
  port     = 8000
  protocol = "HTTP"
  vpc_id   = var.vpc_id

  health_check {
    path                = "/health"
    healthy_threshold   = 2
    unhealthy_threshold = 3
    timeout             = 5
    interval            = 10
  }
}

# ALB listener — points to blue by default
resource "aws_lb_listener_rule" "main" {
  listener_arn = aws_lb_listener.https.arn

  action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.blue.arn
    # CodeDeploy shifts this to green during deployment
  }

  condition {
    path_pattern {
      values = ["/*"]
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

The deployment flow:
Green environment provisioned with new version
Health checks run against green (no live traffic yet)
Lambda hook validates green is behaving correctly
ALB shifts traffic: Blue → Green
Monitor error rates and latency for 10 minutes
If healthy: terminate blue environment
If unhealthy: shift traffic back to blue (30-second rollback)

The cost tradeoff:

Engineers must account for the infrastructure cost, which typically runs between $50 and $100 per month for dual environments.

For most SaaS products, that cost is trivially justified by the risk reduction. For cost-sensitive teams, keep the green environment stopped and only spin it up during the deployment window.

When blue-green is the right choice:

  • High-stakes releases (billing changes, auth changes, major refactors)
  • When you need instant rollback without redeployment
  • Regulated industries with hard uptime requirements

Strategy 3: Canary Deployment

Canary releases route a small percentage of real production traffic to the new version — 5%, then 25%, then 100% — with monitoring between each step.

Canary deployment routes a small percentage of real production traffic to the new version before gradually increasing to 100%. Best for high-traffic applications where you want to validate the new version against real user behaviour before full rollout. Real-world validation with limited blast radius — only 5% of users experience any issues.

AWS ALB weighted target groups make this straightforward:

import boto3

client = boto3.client('elbv2')

def shift_canary_traffic(
    listener_arn: str,
    blue_tg_arn: str,
    green_tg_arn: str,
    green_weight: int  # 0-100
):
    """
    Gradually shift traffic from blue to green.
    Call with green_weight=5, then 25, then 50, then 100.
    """
    blue_weight = 100 - green_weight

    client.modify_listener(
        ListenerArn=listener_arn,
        DefaultActions=[{
            'Type': 'forward',
            'ForwardConfig': {
                'TargetGroups': [
                    {
                        'TargetGroupArn': blue_tg_arn,
                        'Weight': blue_weight
                    },
                    {
                        'TargetGroupArn': green_tg_arn,
                        'Weight': green_weight
                    }
                ],
                'StickinessDuration': 300
            }
        }]
    )

    print(f"Traffic split: Blue {blue_weight}% / Green {green_weight}%")

# Canary rollout sequence
def automated_canary_rollout(listener_arn, blue_tg, green_tg):
    steps = [5, 25, 50, 100]

    for weight in steps:
        shift_canary_traffic(listener_arn, blue_tg, green_tg, weight)
        print(f"Shifted {weight}% to green. Monitoring for 5 minutes...")

        # Check error rate before proceeding
        if not health_check_passes(green_tg, threshold_error_rate=0.02):
            print("Error rate exceeded threshold. Rolling back.")
            shift_canary_traffic(listener_arn, blue_tg, green_tg, 0)
            return False

        time.sleep(300)  # 5 minutes between steps

    print("Canary rollout complete. 100% on green.")
    return True
Enter fullscreen mode Exit fullscreen mode

When canary is the right choice:

  • New features where you want real user validation before full rollout
  • High-traffic products where even a 5% blast radius is still thousands of users
  • Teams with strong observability who can catch issues in the metrics before they escalate

When it's not:

  • Small teams with low traffic (5% of 100 users is 5 users — not enough signal)
  • Breaking changes that can't have two versions running simultaneously

The Part Nobody Talks About: Database Migrations

All three strategies above handle the application layer cleanly. They all fail silently on the same thing: destructive database migrations.

During a rolling or canary deployment, both the old version and the new version are running simultaneously. If your migration removes a column the old version is still reading, you get errors. If it renames a table the old version is querying, you get errors.

The solution is the expand-contract pattern — the only safe way to make database schema changes with zero downtime.

Use the expand-contract pattern: add new columns and tables alongside old ones, deploy application that uses both, migrate data, then remove old columns in a later deployment. Never rename or drop columns in the same deployment that changes application code.

Here's what this looks like in practice:

The wrong way (causes downtime):

-- Migration and code change in one deployment
-- Old code breaks immediately when this runs
ALTER TABLE users RENAME COLUMN username TO display_name;
Enter fullscreen mode Exit fullscreen mode

The right way (three deployments, zero downtime):

Deployment 1 — Expand
Add the new column. Old code ignores it. New code writes to both.

-- Safe: adding a nullable column never breaks existing code
ALTER TABLE users ADD COLUMN display_name VARCHAR(255);
Enter fullscreen mode Exit fullscreen mode
# Application code — writes to both columns during transition
def update_user_profile(user_id: str, name: str):
    db.execute("""
        UPDATE users
        SET username = %s,
            display_name = %s  -- Write to new column too
        WHERE id = %s
    """, (name, name, user_id))

Deployment 2  Migrate
Backfill existing data. Code now reads from new column, falls back to old.

Enter fullscreen mode Exit fullscreen mode


python

Backfill script — run as a one-off job, not in migration

def backfill_display_names():
users = db.query("SELECT id, username FROM users WHERE display_name IS NULL")
for user in users:
db.execute(
"UPDATE users SET display_name = %s WHERE id = %s",
(user['username'], user['id'])
)

Application reads new column with fallback

def get_display_name(user: dict) -> str:
return user.get('display_name') or user.get('username')

Deployment 3 — Contract
Old column no longer needed. Safe to remove.

-- Only safe after 100% traffic is on the new code
ALTER TABLE users DROP COLUMN username;
Enter fullscreen mode Exit fullscreen mode

Three deployments instead of one. Each deployment is safe to roll back independently. No user sees an error.

Use CREATE INDEX CONCURRENTLY to avoid table locks when adding indexes to large tables.

-- This locks the table — never do on production under load
CREATE INDEX idx_users_email ON users(email);

-- This doesn't lock — always use CONCURRENTLY on production
CREATE INDEX CONCURRENTLY idx_users_email ON users(email);
Enter fullscreen mode Exit fullscreen mode

Health Checks: The Deployment Gate Nobody Gets Right

Every zero-downtime strategy relies on health checks to decide when a new instance is ready to receive traffic. A bad health check design breaks the whole system.

The common mistake — health check that always passes:

# This is useless as a deployment gate
@app.get("/health")
def health():
    return {"status": "ok"}  # Always returns 200
Enter fullscreen mode Exit fullscreen mode

A health check that actually gates deployment:

@app.get("/health")
async def health():
    checks = {}

    # Check database connectivity
    try:
        await db.execute("SELECT 1")
        checks['database'] = 'healthy'
    except Exception as e:
        checks['database'] = f'unhealthy: {str(e)}'

    # Check cache connectivity
    try:
        await redis.ping()
        checks['cache'] = 'healthy'
    except Exception as e:
        checks['cache'] = f'unhealthy: {str(e)}'

    # Check critical external dependency
    try:
        response = await httpx.get(
            f"{settings.PAYMENT_SERVICE_URL}/ping",
            timeout=2.0
        )
        checks['payment_service'] = 'healthy' if response.status_code == 200 else 'degraded'
    except Exception:
        checks['payment_service'] = 'unhealthy'

    # Return 503 if any critical dependency is down
    critical_checks = ['database', 'cache']
    is_healthy = all(
        checks.get(c) == 'healthy'
        for c in critical_checks
    )

    status_code = 200 if is_healthy else 503
    return JSONResponse(
        status_code=status_code,
        content={"status": "healthy" if is_healthy else "unhealthy", "checks": checks}
    )
Enter fullscreen mode Exit fullscreen mode

When this health check returns 503, ECS, the ALB, and CodeDeploy all treat the instance as unhealthy and won't route traffic to it. The deployment pauses automatically — which is exactly what you want when a new container can't reach the database.


Graceful Shutdown: The Silent Killer

Even with the right deployment strategy and health checks, you'll still see occasional errors if your containers don't shut down gracefully.

When ECS decides to terminate a container, it sends SIGTERM. The container has a window (default 30 seconds) to finish in-flight requests before ECS sends SIGKILL. If your application ignores SIGTERM, active requests get killed mid-flight.

import signal
import asyncio
from contextlib import asynccontextmanager
from fastapi import FastAPI

# Track active requests
active_requests = 0
shutdown_event = asyncio.Event()

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup
    yield
    # Shutdown — wait for in-flight requests
    shutdown_event.set()
    if active_requests > 0:
        print(f"Waiting for {active_requests} active requests to complete...")
        timeout = 25  # Leave 5 seconds buffer before SIGKILL
        while active_requests > 0 and timeout > 0:
            await asyncio.sleep(1)
            timeout -= 1

app = FastAPI(lifespan=lifespan)

def handle_sigterm(signum, frame):
    print("SIGTERM received — initiating graceful shutdown")
    loop = asyncio.get_event_loop()
    loop.create_task(shutdown_event.wait())

signal.signal(signal.SIGTERM, handle_sigterm)

@app.middleware("http")
async def track_requests(request, call_next):
    global active_requests
    active_requests += 1
    try:
        response = await call_next(request)
        return response
    finally:
        active_requests -= 1
Enter fullscreen mode Exit fullscreen mode

Also configure the ALB deregistration delay to give containers time to drain:

resource "aws_lb_target_group" "api" {
  name     = "api-production"
  port     = 8000
  protocol = "HTTP"
  vpc_id   = var.vpc_id

  # Give containers 30 seconds to finish in-flight requests
  # before the ALB stops sending them traffic
  deregistration_delay = 30

  health_check {
    path              = "/health"
    healthy_threshold = 2
    interval          = 10
  }
}
Enter fullscreen mode Exit fullscreen mode

Automatic Rollback on Failure

Manual rollback is fine when you're watching the deployment. Automatic rollback is what saves you at 3 AM.

ECS deployment circuit breakers handle this natively:

aws ecs update-service \
  --cluster production \
  --service api-service \
  --deployment-configuration '{
    "minimumHealthyPercent": 100,
    "maximumPercent": 200,
    "deploymentCircuitBreaker": {
      "enable": true,
      "rollback": true
    }
  }'
Enter fullscreen mode Exit fullscreen mode

With rollback: true, ECS automatically reverts to the previous task definition if the new deployment fails health checks. No human intervention needed.

For CodeDeploy blue-green, you can set CloudWatch alarm-based automatic rollback:

# In your CodeDeploy deployment group configuration
autoRollbackConfiguration:
  enabled: true
  events:
    - DEPLOYMENT_FAILURE
    - DEPLOYMENT_STOP_ON_ALARM
alarmConfiguration:
  enabled: true
  alarms:
    - name: api-error-rate-high     # Rolls back if 5xx rate spikes
    - name: api-latency-p99-high    # Rolls back if latency spikes
Enter fullscreen mode Exit fullscreen mode

Choosing the Right Strategy

Is this a high-risk release? (billing, auth, major refactor)

├── YES → Blue-Green
│ Instant rollback, full environment isolation

└── NO → Is your traffic high enough to validate at 5%?

├── YES → Canary
│ Real user validation, gradual exposure

└── NO → Rolling Deployment
Simple, cheap, sufficient for most releases

Most startups should start with rolling deployments and graduate to blue-green for high-stakes releases. Canary deployments make sense once you have enough traffic that 5% of it provides meaningful signal.


Zero-Downtime Deployment Checklist

Before any production deployment:

  • [ ] Health check returns 503 on dependency failure, not 200
  • [ ] ALB deregistration delay set to ≥ 30 seconds
  • [ ] Application handles SIGTERM gracefully
  • [ ] Database migration follows expand-contract pattern
  • [ ] Deployment circuit breaker enabled with automatic rollback
  • [ ] CloudWatch alarms configured on error rate and latency
  • [ ] Previous image tagged and available for emergency rollback
  • [ ] CREATE INDEX CONCURRENTLY used for all index additions

The Actual Point

Zero-downtime deployment is not a feature of AWS. It's a discipline applied through AWS.

The tools are all there — ECS rolling updates, CodeDeploy blue-green, ALB weighted routing, CloudWatch alarms. But they only protect you if your application handles shutdown gracefully, if your health checks actually check health, and if your database migrations are designed to run safely alongside the version they're replacing.

Get those three things right and downtime during deployments becomes optional — something that can happen but doesn't have to.


This post is part of OutworkTech's backend engineering series. Related reading: CI/CD Pipelines Explained for Growing Startups and How to Handle 1M+ Users Without Breaking Your System.

OutworkTech builds and scales backend systems, APIs, and SaaS infrastructure for companies that need engineering depth without the overhead. If your deployment process needs to be production-grade — let's talk.

Top comments (0)