AWS Observability in 2026: The Complete Stack — CloudWatch, X-Ray, OpenTelemetry, and Application Signals

You can't fix what you can't see. As systems grow more distributed — microservices, serverless functions, event-driven flows — the gap between "deployed" and "observable" becomes the difference between resolving incidents in minutes versus hours.

AWS's observability stack has evolved significantly. X-Ray SDK entered maintenance mode in February 2026, replaced by OpenTelemetry. Application Signals introduced SLO-based monitoring. CloudWatch absorbed capabilities that previously required third-party tools.

This post covers the complete AWS observability architecture for modern applications — what to collect, where to send it, and how to build alerting that actually works.

The Three Pillars of Observability

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   METRICS   │    │    LOGS     │    │   TRACES    │
│             │    │             │    │             │
│ What's      │    │ What        │    │ Where in    │
│ happening?  │    │ happened?   │    │ the path?   │
│             │    │             │    │             │
│ CPU, memory │    │ Error msgs  │    │ Request     │
│ request rate│    │ debug info  │    │ flow across │
│ error count │    │ audit trail │    │ services    │
│ latency p99 │    │             │    │             │
└─────────────┘    └─────────────┘    └─────────────┘
      │                   │                   │
      └───────────────────┼───────────────────┘
                          │
                    ┌─────────────┐
                    │ CORRELATION │
                    │ Trace ID →  │
                    │ connects    │
                    │ all three   │
                    └─────────────┘

Metrics tell you WHAT is wrong (error rate spiked).
Logs tell you WHY it's wrong (NullPointerException at line 42).
Traces tell you WHERE in the request path it went wrong (payment-service → database timeout).

All three need a common correlation key (trace ID) to be useful together.

The AWS Observability Stack

┌─────────────────────────────────────────────────────────────────┐
│  COLLECTION (Instrumentation)                                    │
│  ADOT (OpenTelemetry) | CloudWatch Agent | SDKs                  │
├─────────────────────────────────────────────────────────────────┤
│  PROCESSING & STORAGE                                            │
│  CloudWatch (Metrics + Logs) | X-Ray (Traces) | S3 (Archive)    │
├─────────────────────────────────────────────────────────────────┤
│  ANALYSIS                                                        │
│  Log Insights | Metrics Insights | Trace Analytics | ServiceLens │
├─────────────────────────────────────────────────────────────────┤
│  SLO & ALERTING                                                  │
│  Application Signals | CloudWatch Alarms | Composite Alarms      │
├─────────────────────────────────────────────────────────────────┤
│  VISUALIZATION                                                   │
│  CloudWatch Dashboards | ServiceLens Map | Managed Grafana        │
└─────────────────────────────────────────────────────────────────┘

Metrics: CloudWatch Metrics + EMF

Default Metrics (Free)

AWS services automatically emit metrics to CloudWatch:

• EC2: CPU, network, disk, status checks
• ECS/Fargate: CPU, memory utilization per task
• Lambda: invocations, duration, errors, throttles, cold starts
• ALB: request count, latency, HTTP 4xx/5xx, healthy host count
• RDS: connections, IOPS, replication lag, free storage

Custom Metrics

For application-specific metrics, use Embedded Metric Format (EMF):

{
  "_aws": {
    "Timestamp": 1692286800000,
    "CloudWatchMetrics": [{
      "Namespace": "MyApp/Orders",
      "Dimensions": [["Service", "Environment"]],
      "Metrics": [
        {"Name": "OrdersProcessed", "Unit": "Count"},
        {"Name": "ProcessingTime", "Unit": "Milliseconds"}
      ]
    }]
  },
  "Service": "order-service",
  "Environment": "production",
  "OrdersProcessed": 47,
  "ProcessingTime": 230
}

Why EMF over PutMetricData: EMF lets you log structured JSON that CloudWatch automatically extracts as metrics AND preserves as log entries — one write, two outputs. No PutMetricData API calls (cheaper, lower latency).

Container Insights

For ECS and EKS, Container Insights provides:

• Per-container CPU and memory
• Per-pod/task network I/O
• Cluster-level resource utilization
• Kubernetes-aware dimensions (namespace, deployment, pod)

Enable with: ADOT collector as DaemonSet (EKS) or sidecar (ECS).

Logs: CloudWatch Logs + Log Insights

Structured Logging (Essential)

Unstructured logs are unsearchable at scale. Always log structured JSON:

{
  "timestamp": "2026-08-13T10:30:00Z",
  "level": "ERROR",
  "service": "payment-service",
  "traceId": "1-abc123-def456",
  "requestId": "req-789",
  "message": "Payment failed",
  "error": "TimeoutException",
  "customerId": "cust-123",
  "amount": 99.99,
  "duration_ms": 5002
}

Log Insights Queries

CloudWatch Log Insights provides SQL-like querying across log groups:

# Find slowest requests in the last hour
fields @timestamp, service, duration_ms, requestId
| filter duration_ms > 1000
| sort duration_ms desc
| limit 20

# Error rate per service
filter level = "ERROR"
| stats count(*) as errors by service
| sort errors desc

# Trace specific request across services
filter traceId = "1-abc123-def456"
| sort @timestamp asc

Log Architecture Patterns

Retention and Cost

• Set retention policies per log group (don't default to "never expire")
• Production: 30-90 days in CloudWatch, archive to S3 Glacier after
• Dev/staging: 7-14 days (no archival)
• Infrequent Access class: 50% cheaper for logs you rarely query

Traces: X-Ray + OpenTelemetry (ADOT)

The Migration: X-Ray SDK → OpenTelemetry

Important: X-Ray SDK entered maintenance mode in February 2026. AWS now recommends OpenTelemetry for all new instrumentation.

ADOT Architecture

┌────────────┐     ┌──────────────┐     ┌─────────────┐
│Application │────→│ ADOT Collector│────→│  X-Ray      │
│(OTel SDK)  │     │ (sidecar or  │     │  CloudWatch │
│            │     │  DaemonSet)  │     │  Prometheus │
└────────────┘     └──────────────┘     └─────────────┘

ADOT Collector receives traces/metrics via OTLP protocol and exports to one or more backends. You can send to X-Ray AND Grafana Tempo simultaneously.

Auto-Instrumentation (Zero Code Changes)

For Java, Python, Node.js, and .NET — ADOT auto-instrumentation captures:

• HTTP requests (incoming and outgoing)
• Database queries (SQL, DynamoDB, Redis)
• AWS SDK calls (S3, SQS, SNS, Lambda invocations)
• gRPC calls

EKS: Deploy ADOT auto-instrumentation as a Kubernetes operator — injects instrumentation into pods automatically.

Lambda: Enable Lambda X-Ray Active Tracing — one toggle, zero code.

Trace Anatomy

Trace: 1-abc123-def456 (entire request lifecycle)
│
├── Span: API Gateway (12ms)
│     └── Span: Lambda: order-handler (450ms)
│           ├── Span: DynamoDB: GetItem (23ms)
│           ├── Span: HTTP: payment-service (380ms)
│           │     └── Span: RDS: INSERT (45ms)  ← SLOW?
│           └── Span: SQS: SendMessage (15ms)

Each span shows: service name, duration, status, metadata. Find the bottleneck instantly.

Application Signals: SLO-Based Monitoring

Application Signals (GA 2025) is CloudWatch's answer to "monitor what matters to users" — automatically tracks SLIs (Service Level Indicators) and lets you define SLOs (Service Level Objectives).

What It Auto-Discovers

Without any configuration, Application Signals detects:

• Services and their dependencies (service map)
• Call volume between services
• Latency (p50, p90, p99)
• Error rate and fault rate
• Availability

Defining SLOs

SLO: "Payment Service Availability"
├── SLI: Success rate (HTTP 2xx / total requests)
├── Target: 99.9% over 30-day rolling window
├── Error budget: 0.1% (43 minutes/month of allowed errors)
└── Alert: When burn rate exceeds 10x normal → page on-call

Why SLOs > Threshold Alarms

Alerting That Works

Alarm Strategy

Composite Alarms

Reduce noise by combining related alarms:

CompositeAlarm: "Payment Service Degraded"
├── AND: Error rate > 5%
├── AND: Latency p99 > 2000ms
└── AND: NOT in maintenance window

→ Only fires when BOTH error rate AND latency are degraded
→ Eliminates false positives from single-metric spikes

Alert Routing

CloudWatch Alarm → SNS Topic → Multiple targets:
├── PagerDuty/Opsgenie (critical: pages on-call)
├── Slack channel (warning: notification only)
├── Lambda (auto-remediation for known issues)
└── ITSM (ServiceNow incident creation)

Dashboards: What to Show

Per-Service Dashboard (Auto-Generated with Application Signals)

• Request rate (rpm)
• Error rate (%)
• Latency (p50, p90, p99)
• Dependency health (downstream services)
• Recent deployments (correlate changes with metrics)

Platform Dashboard (SRE/Platform Team)

• SLO status across all services (green/yellow/red)
• Error budget remaining per service
• Top 5 highest-latency services
• Recent alerts and resolution time
• Deployment frequency and failure rate

Cost Dashboard (FinOps)

• Compute spend trend (CloudWatch billing metric)
• Data transfer costs by service
• Lambda invocations/cost correlation
• Over-provisioned resources (Compute Optimizer findings)

Observability for Different Architectures

Serverless (Lambda + API Gateway + DynamoDB)

Containers (ECS/EKS + ALB + RDS)

Hybrid (Containers + Lambda + Step Functions)

Cost Optimization for Observability

Observability itself can be expensive. Control costs:

Summary

AWS observability in 2026 centers on three shifts:

1. X-Ray SDK → OpenTelemetry (ADOT) — industry-standard instrumentation, multi-backend export, auto-instrumentation for zero code changes
2. Threshold alarms → SLO-based alerting — Application Signals measures what users experience, not just infrastructure health
3. Separate tools → unified correlation — trace IDs connect metrics, logs, and traces so you move from "something's wrong" to "here's the failing span" in seconds

The architecture: Instrument with ADOT (OTel), store in CloudWatch + X-Ray, analyze with Log Insights + Trace Analytics, alert on SLOs via Application Signals, visualize in CloudWatch Dashboards or Managed Grafana.

Start with auto-instrumentation and structured logs. Add custom metrics and SLOs as your services mature. Don't over-instrument day one — observability should grow with your system's complexity.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS observability, infrastructure automation, and cloud architecture. Connect on LinkedIn.