In the previous article, we explored traditional polling.
At first glance, polling seemed like a reasonable solution.
A browser simply asked the server:
"Anything new?"
If nothing changed:
{
"updates": []
}
The browser waited a few seconds and asked again.
Simple.
Predictable.
Easy to implement.
For small systems, it worked perfectly.
But then users changed.
And suddenly, polling became a business problem.
The Incident Nobody Talks About
Imagine it's 2008.
You're part of an engineering team building a rapidly growing social platform.
Your CEO walks into the room and says:
"Users are complaining that messages feel slow."
The database is healthy.
The servers are healthy.
The network is healthy.
Yet users are unhappy.
Why?
Because technology was measuring milliseconds.
Users were measuring feelings.
To them:
Message Sent
│
▼
Nothing Happens
│
▼
3 Seconds Later...
│
▼
Message Appears
The application felt broken.
Not because it was slow.
Because it wasn't instant.
And expectations were changing.
The Product Team's Suggestion
The product manager proposes a simple solution.
"Let's poll more often."
Instead of every 10 seconds:
Poll Every 1 Second
Everyone nods.
Problem solved.
Or so they think.
Three weeks later, infrastructure costs spike.
Monitoring dashboards light up.
Operations teams start asking questions.
The Postmortem
After investigation, engineers discover something surprising.
The system isn't struggling because users are sending messages.
The system is struggling because users are asking if there are messages.
Imagine:
200,000 Users
Each user polls every second.
200,000 Requests / Second
Now imagine only 2% of those requests actually contain new information.
That means:
196,000 Requests
exist solely to hear:
"No updates."
Every second.
All day.
Every day.
The servers are spending most of their time answering questions nobody needed to ask.
Looking at the Problem Like an Architect
A junior developer might see:
"Too many requests."
A system architect sees:
"The communication model is wrong."
The real issue wasn't server performance.
The issue was this:
Client
│
▼
Keeps Asking
Even when the server already knows nothing has changed.
The architecture was forcing unnecessary conversations.
A Different Idea
One engineer proposes something unusual.
Instead of:
Client:
Anything new?
Server:
No.
Client:
Anything new?
Server:
No.
What if the conversation became:
Client:
Tell me when something changes.
Server:
Okay. I'll wait.
And that's exactly what Long Polling became.
Long Polling Explained Like a Recruiter
Imagine you're hiring for a company.
Traditional polling looks like this:
Recruiter:
Any new applicants?
System:
No.
Recruiter:
Any new applicants?
System:
No.
Recruiter:
Any new applicants?
System:
No.
Long Polling changes the workflow.
Recruiter:
Call me when somebody applies.
System:
Understood.
Hours later:
New Applicant Submitted
│
▼
Phone Rings
The recruiter isn't constantly checking.
The system notifies them only when something important happens.
That is Long Polling.
What Actually Happens Under the Hood
Let's say a user opens a chat application.
The browser sends:
GET /messages
Normally the server would immediately respond.
With Long Polling:
Request Arrives
│
▼
Server Checks For Updates
│
▼
No Updates Found
│
▼
Keep Connection Open
Now the server waits.
Maybe 5 seconds.
Maybe 20 seconds.
Maybe 60 seconds.
Suddenly:
New Message Arrives
The server immediately responds.
{
"message": "Hello!"
}
The browser receives the message.
Then instantly creates another waiting request.
Request
│
Wait
│
Response
│
Reconnect
│
Wait Again
To the user:
It feels instantaneous.
To the infrastructure:
It's still HTTP.
Just used differently.
Why Business Leaders Loved It
From a buyer's perspective, Long Polling created something valuable:
Faster Customer Support
Without Long Polling:
Customer Sends Message
│
▼
Agent Sees It 10 Seconds Later
With Long Polling:
Customer Sends Message
│
▼
Agent Sees It Almost Immediately
Response times improve.
Customer satisfaction improves.
Support metrics improve.
Revenue often improves.
Nobody buying the software cares about Long Polling.
They care that customers stop complaining.
Why Developers Loved It
Because it solved a real problem without requiring a new protocol.
Developers already had:
HTTP
Load Balancers
Reverse Proxies
Application Servers
No major infrastructure changes were needed.
Most teams could implement Long Polling with existing technology.
That made adoption relatively easy.
Why Operations Teams Hated It
Because Long Polling quietly moved pressure somewhere else.
Traditional polling creates:
Many Requests
Short Connections
Long Polling creates:
Fewer Requests
Long-Lived Connections
At first this sounds better.
Until your system reaches:
500,000 Active Users
Now you have:
500,000 Open Connections
waiting simultaneously.
The CPU usage might decrease.
But memory usage increases.
Connection tracking increases.
Load balancer complexity increases.
Timeout management becomes critical.
The bottleneck simply moved.
The Scaling Wall
Around this point, many companies hit a new challenge.
Imagine a global chat application.
1 Million Connected Users
Every user has:
One Waiting Request
Now the infrastructure must remember:
Who is connected
Which request belongs to whom
How long they've been waiting
When to timeout
When to reconnect
The system starts behaving less like a website.
And more like a real-time communication platform.
That distinction becomes important.
Because HTTP was never designed for this.
The Architect's Lesson
Long Polling teaches one of the most important lessons in software architecture.
The first solution that works is rarely the final solution.
The evolution looked like this:
Polling
│
▼
Too Much Waste
│
▼
Long Polling
│
▼
Too Many Open Connections
│
▼
Search For Something Better
Long Polling wasn't the destination.
It was the bridge.
A brilliant compromise between:
User expectations
Existing browser capabilities
Available infrastructure
Engineering constraints
For nearly a decade, some of the largest systems on the internet relied on that compromise.
The Question That Changed Everything
Eventually engineers started asking a different question.
Not:
"How can the client ask more efficiently?"
But:
"Why is the client asking at all?"
What if the connection stayed open permanently?
What if the server could speak first?
What if communication flowed both directions?
That question led to one of the biggest shifts in modern web architecture:
WebSockets.
And that's where our story goes next.
Top comments (2)
Really nice narrative arc. Two things worth adding before the WebSockets jump, since they come up constantly in interviews:
1) SSE is the missing middle. Between long polling and WebSockets sits Server-Sent Events: one long-lived HTTP connection the server streams events down (EventSource), with built-in auto-reconnect and a Last-Event-ID for resuming. For the one-directional case you're describing - server tells client "new message" - SSE is usually the right tool: plain HTTP, so proxies/auth/load balancers all just work, no protocol upgrade. Reach for WebSockets only when you genuinely need the client pushing continuously too (live cursors, games, collab editing). A lot of "we need WebSockets" is really SSE.
2) The reconnect gap is long polling's silent bug. Between the server responding and the client firing its next request, there's a window where an event can be published with nobody listening - and it's lost. The fix is a cursor: the client sends "everything after event N," and the server replays anything buffered during the gap. Without a since-token, long polling drops messages under bursty load, which is exactly when it matters.
And one ops note: long-lived requests die on default proxy/LB idle timeouts (~60s on nginx/ELB), so hold the request slightly under that, close cleanly, and have the client treat a timeout as a normal reconnect, not an error. Bonus gotcha - when a broadcast fires, every waiter responds and reconnects at once (thundering herd); jitter the reconnect and fan out via Redis pub/sub so one event wakes all waiters instead of each hitting the DB.
Great points.
I intentionally kept the article focused on the evolution from Polling → Long Polling and the architectural trade-offs that drove that transition.
You're absolutely right that in production systems the story gets more interesting:
I think these topics deserve their own deeper postmortem-style analysis rather than being squeezed into the Long Polling article.
The goal of this series is to follow the same journey many engineering teams went through: Polling → Long Polling → "Why are we still doing request-response?" → SSE/WebSockets.
Thanks for highlighting those production realities.