Stateless services and sessions
In one line
A stateless service keeps no user data in its own memory between requests, so any instance can serve any request, and instances can be added, killed or replaced freely; the state lives in a shared store or in a signed token.
What it is
A service is stateless when everything it needs to handle a request comes from the request itself (headers, token, body) or from shared stores (database, Redis, S3). It may cache data in memory for speed, but losing that cache must not change any answer. A stateful service keeps something in local memory or disk that other instances cannot see: a login session, an upload in progress, an open WebSocket, a game room.
Sessions are the classic example. After a user logs in, the server must remember who they are. There are three common ways: sticky sessions (the load balancer always sends the user to the same instance, which keeps the session in memory), a central session store (a session id cookie points to a record in Redis or a database), and self-contained tokens (a signed JWT carries the user id and expiry, so any instance can verify it without a lookup).
Stateless does not mean the system has no state. It means state is pushed down into a few components built to hold it (databases, caches, queues), and the many compute instances above them become interchangeable. Some things are stateful by nature, such as WebSocket connections; for those you keep the per-connection state small and keep a registry of which server holds which connection.
Why interviewers ask
Horizontal scaling, autoscaling, rolling deploys, spot instances and surviving instance failure all assume that any instance can be killed at any moment without losing user data. Interviewers check whether your design quietly depends on in-memory state, whether you can compare sticky sessions, a session store and JWTs, and whether you know the revocation problem with tokens. Saying 'the API tier is stateless, so we scale it behind a load balancer' is expected in almost every design round; being able to defend it is what earns credit.
How it works
- 1Put all instances behind a load balancer that can send any request to any healthy instance (round robin or least connections).
- 2Move session data out of process memory: either into Redis (session id in an HttpOnly, Secure cookie; Redis key session:<id> with a TTL such as 30 minutes, refreshed on activity) or into a signed token the client sends with every request.
- 3If you use JWTs, keep access tokens short-lived (5 to 15 minutes) and pair them with a refresh token stored server-side, so you can revoke access by revoking the refresh token. For immediate revocation, keep a small deny-list of token ids in Redis until they expire.
- 4Move other per-user state out too: uploads go straight to S3 with pre-signed URLs or multipart upload ids stored in the database; shopping carts go to a database or Redis; rate-limit counters go to Redis.
- 5Treat local disk as scratch space only. Container file systems vanish on restart; anything that must survive goes to object storage or a database.
- 6For genuinely stateful connections (WebSockets, SSE, game sessions), keep a connection registry (user_id to gateway instance) in Redis and route messages through pub-sub, so a gateway restart only forces those clients to reconnect.
- 7Verify by chaos: kill random instances under load and during deploys. If users get logged out, lose carts or see errors beyond retries, some state is still local.
Worked example · Moving a monolith's in-memory sessions to Redis before a festive sale
- A shopping app runs 6 app servers with sessions in memory and sticky sessions on the load balancer. Each server holds about 50,000 active sessions.
- During a deploy, each restarted server logs out its 50,000 users and empties their carts. A crash does the same. Autoscaling to 30 servers for the sale does not help existing users, because sticky sessions keep them on the old, hot servers.
- The team moves sessions to Redis: a 32-byte random session id in the cookie; Redis stores about 1 KB per session with a 30-minute sliding TTL. 300,000 active sessions x 1 KB = 300 MB, small for one Redis primary with a replica.
- Each request does one Redis GET, about 0.3 to 1 ms inside the same zone. At 20,000 requests per second that is 20,000 Redis reads per second, well within one Redis node's capacity (often 100,000+ simple operations per second).
- Stickiness is turned off. New instances take traffic immediately, deploys log nobody out, and a crashed instance only fails its in-flight requests, which clients retry.
Picture it
A stateless service is like a bank with a shared ledger: you can go to any counter, and the clerk looks up your account in the central system. A stateful one is like a shop where only one particular shopkeeper remembers what you owe; if he is on leave, nobody can help you.
Key patterns
Session store
When: Web apps with server-side sessions, admin tools, anything needing instant logout
How: Random session id in an HttpOnly, Secure, SameSite cookie; session data in Redis with a sliding TTL; delete the key to log out everywhere.
Short-lived token plus refresh token
When: Mobile apps and APIs across many services
How: Signed access token (JWT, 5-15 min) verified locally by every service; long-lived refresh token stored and revocable on the auth server; rotate refresh tokens on use.
Externalise everything that must survive
When: Uploads, carts, workflows, counters
How: Object storage for files, database or Redis for carts and counters, a durable workflow engine for long processes. The instance becomes disposable.
Connection registry for stateful edges
When: WebSockets, SSE, game servers
How: Map user_id to gateway instance in Redis with a TTL heartbeat; send messages through pub-sub to the right gateway; clients reconnect with backoff and resume from a last-seen id.
Real-world use
The Twelve-Factor App
Heroku's widely cited guidelines say processes should be stateless and share nothing; any data that must persist goes to a backing service such as a database. Most container platforms assume this model.
Kubernetes Deployments
Rolling updates, autoscaling and node draining all kill and replace pods freely. Stateful systems need StatefulSets, stable identities and persistent volumes, which is why teams keep API tiers stateless.
AWS Application Load Balancer stickiness
ALB can pin a client to a target using a load-balancer-generated cookie (AWSALB) or an application cookie. It is useful for legacy apps but uneven load and lost sessions on target failure are the known costs.
Spring Session and Express session stores
Common frameworks provide drop-in session stores backed by Redis or a database, so moving sessions out of memory is usually a configuration change rather than a rewrite.
Chat and real-time gateways
Products like Slack and Discord keep millions of long-lived WebSocket connections on gateway servers. These are stateful by nature, so the business logic behind them is kept stateless and the gateways hold only connection state that can be rebuilt on reconnect.
In a project
Your team's ticket booking API keeps seat-selection holds in a local in-memory map for 10 minutes. It works on one server, but after scaling to 8 servers, users see 'hold expired' errors and sometimes two users get the same seat during a big movie release.
- Identify every piece of in-process state: seat holds, sessions, rate-limit counters and an in-memory list of recently viewed shows.
- Move seat holds to Redis with SET seat:<show>:<seat> <user> NX EX 600, so the hold is atomic and visible to every instance, and expires on its own.
- Move sessions to Redis-backed sessions; keep the recently viewed list as a client-side value since losing it is harmless.
- Remove sticky sessions from the load balancer and switch to least-outstanding-requests routing.
- Add a test that kills two of eight instances during a load test of 5,000 bookings per minute and checks that no seat is double-held and no user is logged out.
Impact: Double holds disappear because the hold is a single atomic record, the API autoscales from 8 to 40 instances for big releases without user-visible effects, and deploys can happen during the day.
Trade-offs
Use it when
- Any API or web tier that must scale horizontally or autoscale
- You deploy often and want zero-downtime rolling updates
- You run on spot or preemptible instances that can vanish with short notice
- Many services need to authenticate the same user (tokens verified locally)
Avoid it when
- The workload needs data locality for speed, such as a game server holding a match in memory; keep it stateful but make it recoverable
- Large, long-lived JWTs that must be revocable instantly; use server-side sessions or a deny-list instead
- Putting big blobs (whole carts, permissions lists) in cookies; cookies are limited to about 4 KB and are sent on every request
- A shared session store that becomes a single point of failure; replicate it and handle its outage explicitly
Common mistakes
- Calling a service stateless while it keeps uploads, carts or locks in local memory or on local disk.
- Relying on sticky sessions for correctness, so a crash or deploy logs users out and new instances get no traffic.
- Issuing JWTs valid for 30 days with no way to revoke them after a password change or a stolen phone.
- Storing sensitive data inside a JWT payload; it is only signed and base64-encoded, so anyone can read it.
- Making the session store a single Redis node with no replica, so its failure logs everyone out.
- Forgetting that WebSocket servers are stateful and need graceful draining and client reconnect logic.
Interview questions
Quick revision
- Stateless = any instance can serve any request; losing an instance loses no user data.
- State goes down into databases, caches, queues and object storage.
- Sessions: sticky (fragile), store in Redis (simple, revocable), signed token (no lookup, hard to revoke).
- JWT: short-lived access (5-15 min) + revocable refresh token.
- JWT payload is readable; never put secrets in it.
- Cookies: HttpOnly, Secure, SameSite; about 4 KB limit.
- WebSocket gateways are stateful: registry in Redis, pub-sub routing, graceful drain.
- Test it: kill instances under load and during deploys.
Check yourself
1. Which change makes an API tier stateless?
2. What is the main weakness of long-lived JWT access tokens?
3. Why are WebSocket gateways considered stateful?
4. An app stores the shopping cart in a cookie. What problem appears as carts grow?
See it in a full design
These walkthroughs use this lesson.
Saved on this device. Create a free account to keep it on every device.
Practise this with Sam
A real mock interview, out loud, with an honest scorecard. Free to start.