Scaling WebSockets Past One Box

WebSockets scale differently from HTTP, and the difference ambushes teams at the worst moment — the marketing launch, the live event, the Monday every customer opens the dashboard at 9:00. HTTP requests are confetti: brief, stateless, spread across any server. A WebSocket is a tenancy: one connection, one server, held for hours, consuming a file descriptor and a slice of memory the whole time. Scaling confetti and scaling tenants are different sports. Here's the playbook for the second one.

Problem 1: the message that lands on the wrong server

The founding puzzle of multi-node WebSockets: Alice's socket lives on node A, Bob's on node B, and your app server publishes "new message in room 7" — to whom? The answer is a backplane: every socket node subscribes to a shared pub/sub bus, publishes go to the bus, and each node forwards to whichever of its connections care.

App server ──publish──▶ Redis pub/sub ──▶ Node A ──▶ Alice's socket
   (or broker)              │
                            └─────────▶ Node B ──▶ Bob's socket

Redis pub/sub is the standard first backplane — fire-and-forget is exactly right here, because a socket node that missed a message has clients that will reconcile anyway (more below). Reverb, Soketi and Socket.IO all support precisely this topology out of the box; at very large scale the bus itself shards or graduates to a broker, but you'll know when — Redis pub/sub comfortably moves hundreds of thousands of messages a second before then.

Problem 2: sticky sessions — need them or not?

Less than people think. The WebSocket upgrade handshake must complete against one node, and after that the TCP connection naturally stays there — no stickiness config required for the socket itself. Where stickiness sneaks back in: fallback transports (long-polling in Socket.IO needs consecutive requests hitting the same node) and any node-local state you were tempted to keep. The cleaner posture: make socket nodes stateless — connection registry and channel membership derivable from the backplane or a shared store — so any node can die and its clients simply reconnect elsewhere. Statelessness also makes deploys civilized: drain a node (stop accepting upgrades, let clients migrate on their own reconnect logic), then kill it — the same graceful-shutdown philosophy as pod termination.

Problem 3: the reconnect stampede

Here's the one that takes down launches. A node dies (or deploys, or a load balancer hiccups) and 50,000 clients reconnect simultaneously — each performing a TLS handshake, an auth call to your app, and a channel subscription. Your socket tier survives; your auth endpoint dies, which kills reconnects, which triggers retries, which… you get it. The defenses, all mandatory at scale:

• Client-side jittered exponential backoff. Reconnect after 1s + random, then 2s + random, capped at ~30s. The jitter is the active ingredient — without it you've scheduled synchronized retry waves.
• Cheap, cacheable auth. Signed tokens with a few minutes' validity let reconnects re-subscribe without hitting the database; your /broadcasting/auth path should be the fastest endpoint you own, and it deserves its own rate-limit bucket so a stampede degrades gracefully instead of toppling the app.
• Rolling drains, never mass restarts. Deploy socket nodes one at a time with connection draining; a full-fleet restart is a self-inflicted stampede.

Problem 4: what the socket missed

I said it in the Pusher post and it triples in importance at scale: delivery over a WebSocket is best-effort. Between disconnect and reconnect, events happened; pub/sub backplanes don't replay. Design the recovery explicitly, by data type: state-shaped data (dashboards, statuses) refetches current truth on reconnect — simple and bulletproof; stream-shaped data (chat, feeds) needs sequence numbers per channel and a "give me everything since seq N" endpoint, with the client tracking its high-water mark. Choose per feature; the dashboard doesn't need the chat machinery.

Capacity: the numbers that matter

Per-node ceilings to test BEFORE launch day:
- File descriptors (ulimit -n)      default 1024 will end your evening early
- Memory per connection             ~tens of KB each; 100k conns ≈ real GBs
- CPU at target fan-out             1 msg → 50k recipients is where CPU lives
- LB idle timeout vs ping interval  balancer killing "idle" sockets at 60s
                                    masquerades as a mystery client bug

Load-test with a tool that opens connections (k6 or artillery in WebSocket mode), not just messages — idle-connection ceilings and fan-out CPU are separate cliffs. Then instrument the four numbers per node — connections, messages/sec, memory, reconnect rate — on the same dashboards as everything else. A reconnect-rate spike is your earliest, clearest signal that something upstream just blinked.

None of this is exotic anymore — a Reverb or Soketi fleet with a Redis backplane, jittered clients and reconcile-on-reconnect comfortably serves six-figure concurrency on unremarkable hardware. The teams that struggle didn't lack technology; they scaled tenants like confetti. Now you know which sport you're playing.

Launch coming and the socket tier untested? A week of load-testing beats a launch-day incident by any accounting — let's schedule the fire drill.