Bound ServiceAccount Tokens: 9 Tips to Kill Static Ones

The non-expiring kubernetes.io/service-account-token Secret is the most over-scoped credential in most clusters. It never expires, it authenticates from anywhere on the internet, and nothing rotates it. Kubernetes has spent five releases retiring it: v1.24 stopped auto-minting it, and v1.30 promoted the cleaner that invalidates the leftovers to stable. The replacement is a time-bound, audience-scoped JWT from the TokenRequest API, tied to a Pod's lifetime, and it behaves differently enough that an integration working last quarter can start throwing 401s after an upgrade. These tips are for platform and security engineers who govern non-human identity on Kubernetes and want to retire static tokens without an outage. This is the concrete mechanism the SPIFFE-and-OAuth identity essay argued for, and the follow-through to non-human identity governance in the field. The same instinct drove dropping static keys in GitHub Actions with OIDC: every credential you cannot delete is one you have to defend.

The tips

1. Inventory every legacy token Secret before the cleaner finds them for you. The LegacyServiceAccountTokenCleanUp controller went GA in v1.30. It labels any auto-generated token Secret unused for a year with kubernetes.io/legacy-token-invalid-since, and the token stops being accepted the moment that label lands. Find these on your schedule, not as a surprise 401 in production. List every remaining secret-based token cluster-wide:

   kubectl get secrets -A --field-selector type=kubernetes.io/service-account-token

1. Read the last-used label to separate live tokens from fossils. The control plane stamps each legacy token with kubernetes.io/legacy-token-last-used, a date at day granularity. That label is your triage: a Secret used this week is a live integration you have to cut over, and one untouched for months is a fossil you can delete now.

   kubectl get secret <name> -o jsonpath='{.metadata.labels.kubernetes\.io/legacy-token-last-used}'

If a Secret already carries legacy-token-invalid-since and you need it working again immediately, delete that one label to reactivate it, then migrate it properly. The cleaner will relabel it otherwise.

1. Issue tokens on demand with kubectl create token, never by reading a Secret. For any human or script that needs to act as a ServiceAccount, mint a short-lived token through the TokenRequest API rather than pulling a static one with kubectl get secret ... -o jsonpath. It respects RBAC, carries a real expiry, and leaves no long-lived artifact on disk or in etcd.

   kubectl create token deploy-bot -n ci --duration=30m

1. Know the 10-minute floor and the 1-hour default. The TokenRequest API refuses to issue a token shorter than 10 minutes: ask for --duration=5m and the server clamps or rejects it. Specify nothing and the default lifetime is one hour. Do not design a 90-second credential, because the platform will not give you one. For genuinely ephemeral use, 10 minutes is the floor you build against.

2. Bind the token to the object that should kill it. --bound-object-kind accepts Pod, Secret, and Node. A token bound to a Pod becomes invalid the instant that Pod is deleted, even if its clock has not run out, so a token leaked from a since-terminated job is already dead. That is the property static Secrets never had.
   

   kubectl create token app -n prod \
     --bound-object-kind=Pod --bound-object-name=app-7d9f --duration=15m

1. In Pods, mount a projected token and set expirationSeconds deliberately. Pods get their token through a projected volume, not a Secret mount. The kubelet rotates it automatically once the token passes 80% of its TTL or 24 hours, whichever comes first. Set expirationSeconds to match how long a single request path actually needs, not "forever":

   volumes:
   - name: token
     projected:
       sources:
       - serviceAccountToken:
           path: token
           audience: vault
           expirationSeconds: 3600

1. The most common post-migration 401 is an audience mismatch, not a permissions problem. Bound tokens carry an aud claim, and the API server rejects any token whose audience is not in its --api-audiences. A token minted with --audience=https://vault.internal is refused by the API server itself, because it was never addressed to it. Before you touch RBAC, decode the token and check where it is actually pointed:

   kubectl create token app -n prod | cut -d. -f2 | base64 -d 2>/dev/null | jq .aud

1. Understand the node-audience gate before wiring cloud or Vault audiences. By default a kubelet may only request tokens for audiences already referenced by Pods on that node. If your CSI driver or sidecar asks for a fresh audience the node has never mounted, the request fails until you grant it through an RBAC rule carrying the request-serviceaccounts-token-audience verb. Teams adding a new external audience mid-cluster-life hit this wall and misread it as an app bug. It is an authorization gap on the node, not your code.

2. Make consumers re-read the token file, and do not trust a recreated ServiceAccount's old token. Two production traps hide in the details here. First, because the kubelet rotates the projected file in place, a client that reads the token once at startup authenticates fine for an hour and then starts failing, so reload from the mount path on every request. Second, bound tokens pin the ServiceAccount's UID, so deleting and recreating a ServiceAccount with the same name invalidates every token already issued against the old UID. Istio and other mesh sidecars surface exactly this as a does not match claim error. Watch kubernetes/kubernetes#138689 too: a transient TokenRequest or NodeAuthorizer failure during a control-plane restart can leave a projected token file stale, so make token-refresh failures an alert rather than a silent log line.

Wrap-up

Run the --field-selector type=kubernetes.io/service-account-token inventory today, sort by the legacy-token-last-used label, and cut the live ones over to TokenRequest before the v1.30 cleaner turns "unused for a year" into an unscheduled 401. Bound tokens are the version of a credential you do not have to defend, since expiry, audience, and Pod lifetime do the containment for you. Move identity on the same clock as the rest of your controls, including a default-deny egress policy, so a leaked token cannot phone home even in the minutes before it dies.

Sources

• https://kubernetes.io/docs/concepts/security/service-accounts/
• https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md
• https://github.com/kubernetes/enhancements/issues/2799
• https://www.kubernetes.io/docs/reference/kubectl/generated/kubectl_create/kubectl_create_token/
• https://github.com/kubernetes/kubernetes/issues/138689

Originally published at indragustiprasetya.com