A study that tracked 657,607 links across the web just hit the front page of Hacker News with 123 points and 27 comments. The findings are sobering: the old web is disappearing, and the rate of decay is accelerating. This isn't just a technical curiosity — it has implications for SEO, knowledge preservation, and the long-term viability of the internet as a reference source.
What the Study Found
The researchers at 0.mk crawled and tracked over 657,000 links to understand the phenomenon of "link rot" — when URLs that once worked stop returning the content they originally pointed to. The results paint a picture of a web that's losing its memory at an alarming rate.
Key findings include:
- A significant percentage of links older than 5 years are dead or redirected
- News articles and academic papers are among the worst affected
- Government websites, despite mandates for digital preservation, show significant link rot
- The rate of decay appears to be accelerating as platforms shut down, restructure, or change their URL schemes
Why Links Die
Link rot happens for several reasons:
Platform shutdowns. When a website, service, or platform shuts down, every link pointing to it dies. Google+ killed millions of links. Vine's shutdown orphaned years of content. When forums and blogs close, their contributions to the web's collective knowledge vanish.
URL restructures. Sites redesign and reorganize, breaking old URLs. CMS migrations, domain changes, and platform switches all contribute. Even when sites implement redirects, these are often temporary and eventually break.
Content deletion. Articles get unpublished, posts get deleted, APIs get deprecated. The content that was there yesterday might not be there tomorrow — and nothing flags the loss.
Paywalls and logins. Content that was freely accessible gets moved behind paywalls. The URL still "works" but the content is no longer freely available, making it effectively dead for most reference purposes.
The Implications
For SEO and content marketing: If you're building content that references external sources, link rot is silently degrading your content quality. Broken links hurt user experience and may impact search rankings. Regular link auditing is no longer optional.
For journalism and research: News articles and academic papers frequently cite URLs as sources. When those links rot, the citations become unverifiable. This undermines the integrity of the web as a reference system.
For the web as a knowledge commons: The internet was supposed to be a permanent record. Instead, it's becoming a temporary one. The "old web" — the early 2000s blogosphere, the first generation of web apps, the original content that built today's internet culture — is actively disappearing.
What Can Be Done
Several approaches exist to combat link rot:
The Wayback Machine (archive.org) remains the most important defense. When a link dies, the Wayback Machine often has a cached copy. The Internet Archive's work is critical, but they can't archive everything — and they rely on being crawled before content disappears.
Perma.cc and similar services create permanent links to cited sources. Academic and legal communities have adopted these, but they're not widely used in general web content.
DOI (Digital Object Identifier) systems provide persistent identifiers for academic content. But DOIs only work for academic papers, not general web content.
Content-addressed storage (IPFS, Arweave) offers a fundamentally different approach: instead of linking to a location, you link to content by its hash. The content can be stored anywhere and retrieved by anyone. If one copy disappears, others can serve it. This is the most technically robust solution but has the lowest adoption.
Regular link audits are the practical, immediate solution. Tools like Screaming Frog, Ahrefs, and Semrush can scan your site for broken links. For smaller sites, free tools like Broken Link Checker (WordPress plugin) or simple scripts that curl every external link work fine.
What This Means for Content Creators
If you write articles, build websites, or create any kind of web content, link rot affects you in two ways:
Your external links are breaking. Every link you've ever published to another website is slowly degrading. If you've been writing for years, a significant portion of your external references are now dead. This reduces the quality and trustworthiness of your content.
Your content is being linked to — and those links might break. If you ever reorganize your site, change your CMS, or let your domain lapse, every inbound link to your content breaks. The work you've done to build SEO authority and web presence can vanish in a single infrastructure change.
Practical steps:
- Audit your links regularly. At minimum once a year, scan your content for broken links and fix or replace them.
- Archive important references. Before citing a URL, save it to the Wayback Machine (just visit web.archive.org/save/URL). This ensures a copy exists even if the original dies.
- Use permalinks where available. Many sites offer stable link formats. Use those instead of temporary or session-based URLs.
- Implement proper redirects. If you reorganize your site, implement 301 redirects from old URLs to new ones. Keep these redirects indefinitely.
- Preserve your own content. If you ever shut down a site or blog, don't just let the domain lapse. Redirect to an archive, export your content, or set up a static placeholder.
The Bigger Picture
The 657,607-link study is a reminder that the web is not a permanent medium. Unlike print, which degrades slowly and predictably, digital content can vanish instantly and without warning. A platform shutdown can delete millions of pages overnight. A CMS migration can break every URL on a site. A domain lapse can make years of content unreachable.
We've built a civilization's worth of knowledge on a medium that's fundamentally ephemeral. The old web is disappearing, and we're not doing enough to preserve it. Studies like this one are important not just for the data they provide, but for the awareness they raise.
The internet's greatest promise was permanence — the idea that once you publish something, it's there forever. The reality is proving very different. Understanding the scale and speed of link rot is the first step toward building a web that actually keeps its promises.
Top comments (0)