RSS lost to social networks. What insights does this history provide for blockchain?

Original author: Sinclair

Original title: 《The Rise and Demise of RSS》

Original compilation: Yao Changlin

The older generation of internet users should feel a strong resonance when reading this article, and the new generation working on blockchain will also see many familiar phenomena. The player with dual identities, Huo Ju, feels even more complex; he recommended this article like this:

"By 2019, looking back at the rise and fall of RSS allows us to see its gains and losses more clearly. This article tells the complete history from the rise of RSS in 1999 to the closure of Google Reader in 2013, when content reading and distribution were replaced by social networks.

The reason for recommending this article is that this history has many similarities with blockchain. Compared to today's social networks, RSS was a more distributed, more equitable, and more open system, but it was ultimately defeated. One important reason is that centralized products are easier to design, provide a better experience, and are easier for users to understand without needing to learn complex concepts, offering better ways for operating companies to make money.

Today, the comparison between blockchain and centralized products is just the same. This time, can decentralized products find a different path?"

This article is a bit long and contains many obscure and old technical terms in English, but it's okay if you don't understand the technical details. Additionally, there are many English names mentioned; we have replaced a few important names with Chinese, and they will be highlighted in red the first time they appear, so remembering these people will help you not get lost.

Veteran internet users should be quite familiar with RSS. In fact, RSS has two different definitions: Really Simple Syndication and Rich Site Summary, but essentially it is a program-friendly way of subscribing to information. Today, there are still applications and websites using RSS technology, but for most users, RSS has become a vague concept.

Looking back at the history of RSS, there are two stories worth telling: The first story is about a great vision for the future of the internet that never materialized; the second story is how the collaboration to develop a universal standard became the most controversial split in the history of open-source software.

In the late 1990s, during the dreamy decade between the IPO of Netscape and the dot-com bubble, no one knew where the internet was headed, but everyone realized that the internet would be more imaginative than it is now. There was speculation that the internet would be revolutionized by "aggregated networks." Originally, the internet was a point-to-point transmission, delivering information unidirectionally from the server to the user, but the new model could break the original architecture and repackage and distribute information across the web in a "channel" format.

At that time, there was an influential newsletter among investors called Releas 1.0, in which Werbach predicted: "Aggregated networks will evolve into the core model of the internet ecosystem, allowing companies and individuals to control their online identities while enjoying the benefits of the vast network."

RSS's future was once so bright; what happened later?

Werbach asked readers to imagine a scenario: a fencing enthusiast has two options when purchasing a sword: one is to log onto an e-commerce website or run to a physical store, and the other is to visit the fencing website they log onto daily and click on the ads on the right sidebar to make a purchase. Similar to how programs from major TV stations can be broadcast on local small TV stations, this would attract more viewers. Aggregated networks could provide users with information through intermediary websites, making it easier for users to control their information interactions with the internet.

RSS was one of the most promising standards to achieve this aggregated future. Werbach considered RSS to be "a model of lightweight aggregation protocols." Another article from the same period stated that RSS was the first protocol to realize the potential of XML (eXtensible Markup Language). RSS could help readers and content aggregators customize channels from the websites they wanted.

However, 20 years later, with the rise of social networks and the closure of Google Reader, RSS is only used in podcasts, tech podcasts, and some news feeds, becoming a slowly dying technology. Indeed, many people still rely on RSS readers, stubbornly adding RSS to blogs as a form of nostalgia. These persistences have turned into a protest against centralized networks, against the few companies that control the entire web, and against a network that is completely different from Werbach's imagination.

RSS's future was once so bright; what happened later? Was the collapse of RSS inevitable? Was it really caused by infighting over standards?

Stuck in the Mud

RSS was invented twice, which means it lacks a recognized inventor and falls into endless disputes. But it also shows that RSS was a timely good idea.

In 1998, Netscape was looking for the next user increment. Its flagship product, the "Netscape Browser," once held an 80% market share, but that was quickly eroded by Microsoft's IE browser. So Netscape needed a new project. In May, Netscape gathered a team to work on a new project called "Project 60," developing an RSS news tool based on Ben Hammersley's Atom project. Two months later, Netscape launched a new portal product called "My Netscape," competing with portals like Yahoo, MSN, and Excite.

In March of the following year, Netscape added a new feature called "My Netscape Network" to the My Netscape portal. Users could customize their My Netscape pages, adding channel features to subscribe to the latest headlines from other websites. As long as a website published a "specified format" file, users could click "Add Channel" to subscribe to their favorite websites on their My Netscape homepage. A module containing the website's headlines would then appear on the user's My Netscape page.

This "specified format" file is the RSS file. However, in the declaration of My Netscape Network, Netscape defined RSS as "RDF Site Summary." In fact, this definition is not precise enough because RDF (Resource Description Framework) is a syntax for describing specific properties of designated resources.

In fact, in 1999, W3C also intended to draft the RDF standard. Although RSS was theoretically supposed to be based on RDF, the RSS reference document provided by Netscape did not use any RDF tags at all. In the Netscape RSS specification document, its author Dan Libby mentioned that in the MNN version, Netscape deliberately limited the complexity of RSS. This specification was versioned as 0.90, indicating that subsequent versions would be more consistent with W3C standards.

The initial RSS standard was developed by Libby and Netscape colleagues Eckart Walther and Ramanathan Guha. Guha mentioned in an email that most of the ideas developed by Guha and Walther came from Libby's early thoughts; after AOL acquired Netscape, the two left, and Libby took on the main updating work. Guha made many contributions to the development of RDF, and he and Walther intended to apply RDF to RSS. Later, Libby mentioned in an email that the RDF version of RSS was ultimately cut down for two reasons: first, time constraints, and second, RDF was too complex for ordinary users.

While Netscape was caught up in the "portal war" striving for traffic, "web blogging" quietly became popular. The CEO of UserLand Software, Winer, invented the earliest content management system, allowing ordinary people without technical knowledge to set up their own blogs. Winer's blog, Scripting News, is one of the oldest blogs on the internet. More than a year before Netscape launched My Netscape Network, on December 15, 1998, Winer announced that his blog product would support both XML and HTML formats.

Winer's XML format was called "Scripting News Format." Some believed it was similar to Microsoft's Channel Definition Format, but there is no written evidence. Like Netscape's RSS, Winer defined the structure of the text to facilitate reading and calling by other programs. When Netscape released RSS version 0.90, UserLand announced support for both formats. However, Winer felt that Netscape's version was "regrettably incomplete" and "lacked the core parts most needed by authors and readers." It could only reference a series of links, while the "Scripting News Format" could contain more content: more paragraphs, each of which could have many links.

In June 1999, two months after Netscape released My Netscape Network, Winer also released a new version of the "Scripting News Format"—ScriptingNews 2.0b1. Winer accelerated the promotion of his standard, and users did not recognize the significant flaws in RSS 0.90. Winer's version added some new elements that could be compatible with RSS. But there was a huge difference between the two standards: Winer's so-called "fat" format could contain entire articles, not just some links.

In July, Netscape was about to release RSS 0.91 and faced a major issue regarding updating the text specification. RSS no longer represented "RDF Site Summary," but rather "Rich Site Summary." All RDF elements were removed. Many script news tags were also merged. In this text specification, Libby explained:

RDF references have been removed from the standard. The RSS containing RDF was intended to provide a metadata format for website summaries. The removal was based on two important considerations. First, data providers need to provide an aggregation data format, not a metadata format; RDF files must be very precise to meet the standard. But this would make reading difficult and hard to manually develop RDF files. Second, there are no tools available to generate RDF files. Based on these two points, we decided to adopt a standard XML approach.

Winer was very satisfied with RSS 0.91, claiming it was "better than I imagined," and regarded it as a replacement for the ScriptingNews 2.0b1 format. For a time, everyone believed that RSS would soon have a unified official version.

The Great Split

A year later, everyone discovered that RSS 0.91 had many shortcomings. Many things people wanted to do with RSS could not be accomplished with version 0.91, which had many unnecessary restrictions, such as a maximum of 15 links per channel.

At this time, RSS technology had been widely adopted. Netscape had lost interest in RSS 0.91, while most other companies were using Winer's standard, such as O'Reilly Net's RSS aggregator Meerkat and the news aggregation site Moreover.com. Representatives of various stakeholders communicated via email about how to improve the 0.91 standard, but no consensus was reached.

Disagreements over Namespaces Reflect Disagreements over the Nature of RSS

A mailing list called Syndication recorded all emails discussing the RSS standard, which is still accessible today. It remains a valuable historical record of how these profound disagreements ultimately tore apart the entire RSS community.

One side of the split was represented by Winer. He wanted to improve RSS but iterated in a relatively conservative manner. In June 2000, Winer released version 0.91, which was not much different from Netscape's version. Winer stated in his blog that since Netscape no longer maintained updates, 0.91 served as a starting point to demonstrate how RSS could be used in practical scenarios.

At the same time, he believed that the simple and easy-to-use RSS was already successful enough; the complex new features mentioned in the Syndication emails would not provide any value for content distribution. He especially opposed the addition of namespaces and refused to restore the deleted RDF format (namespaces allow programmers to customize subformats of RSS, meaning that new features require unanimous agreement from everyone. However, namespaces would also make reading and writing RSS more difficult). In the Syndication mailing list, Winer mentioned that these changes were very important and could trigger a split in the standard:

We are still thinking about how to advance the development of RSS. I certainly want to use content similar to ICE in RSS2, and publishing and subscription features are also a high priority, but the premise is simplicity. I also want more room for expansion, but I will absolutely not use "namespaces," "schemas," or repeat the old path of RDF. I understand that there may still be people who need to retain these features, so there may be a new forked version. I have many ideas for the forked version and will share them with everyone when the time is right.

The opposing side mainly consisted of three individuals: Rael Dornfest from O'Reilly, Ian Davis, CEO of a startup in the search field called Calab, and the 14-year-old Aaron Swartz. Swartz is the co-founder of Reddit, a well-known hacktivist. In an email from Davis to me, he mentioned that in 2000, Swartz's father often accompanied him to technical conferences.

These three individuals believed that RSS needed namespace functionality to meet the diverse needs of users. In several other emails, Davis suggested building a module based on namespaces to make RSS more extensible without becoming too complex. The camp supporting namespaces argued that RSS would soon be used not only for synchronizing blogs but also for many other use cases. Under the premise of not increasing complexity, namespaces were the only solution.

The controversy over namespaces was merely a surface issue; the core of the dispute was "What should RSS be used for?" Winer originally created his standard to synchronize his blog. In contrast, Netscape released the RSS standard to build mini-sites within its portal. Some believed that Netscape's original intention should be respected. In an email to Syndication, Davis mentioned that RSS was initially intended to build "mini site maps," but many new demands had arisen over the past year, and RSS should now expand to support more types of information, not just simple news headlines.

This expanded Netscape's vision for RSS. Libby mentioned in an email to me that the debate over the development of RSS centered on "building a global semantic web" vs. "making it easier for people to publish their works."

Winer referenced a completely different logic in his reply to Davis's email: Scripting News was the first RSS network, and its purpose was entirely different from Netscape's. The community's divergence over the inventors of RSS and their goals made a split inevitable.

The version split occurred after Dornfest announced the proposal for the RSS 1.0 specification and formed the RSS-DEV working group (which included Davis, Swartz, and others, but not Winer). In this 1.0 version, RSS was redefined as "RDF Site Summary," and RDF elements were added back in. Considering Winer's historical contributions to the dissemination of RSS, version 1.0 did not remove Winer's name. However, version 1.0 also stated that RSS would not develop along the path Winer had planned. Simply adding some elements to RSS without considering extensibility would cause RSS to lose many application scenarios. Version 1.0 was also based on an XML namespace-defined module system.

The RSS-DEV working group unilaterally named itself "RSS 1.0," which made Winer furious. In another email, he mentioned that he had a significant body of work that was stolen, likely referring to O'Reilly and the RSS-DEV working group.

Other members of the mailing list also believed that the RSS-DEV working group should not use the name RSS without obtaining community consent. However, the working group insisted on using it. Dan Brickley, a member of the working group, argued that RSS 1.0 was based on the earliest vision of RSS, traceable back to MCR (the predecessor of RDF) and CDF, among others. He also believed that RDF was originally part of RSS, and that version 1.0 contributed to RSS far more than Winer did, thus deserving the name RSS.

The RSS-DEV working group released the final version in December. Almost simultaneously, Winer released his upgrade to RSS 0.91—RSS 0.92, which quickly adopted several changes by various podcasts. Thus, RSS officially split.

If the RSS-DEV working group had seriously invited Winer to join, this split might have been avoided. Winer was clearly important, and the working group acknowledged him as the main author of Syndication, contributing significantly to the popularization of RSS. However, Davis mentioned in an email that Winer wanted to control RSS and turn it into private property, so he was unwilling to work with them. Winer declined the working group's invitation. O'Reilly's CEO, Tim O'Reilly, explained at a UserLand discussion in September 2000:

Everyone gathered to discuss the evolution of RSS, and Winer was present. When the opinions of those present turned to a direction he did not support, Winer withdrew, claiming that O'Reilly wanted to replace him through discussion, even though O'Reilly's Dornfest was just one of a dozen authors, and Dornfest had experienced the entire development history of RSS.

Winer formally replied to Tim O'Reilly:

Two weeks before the meeting, I had just met with Dale, and he did not mention RSS 1.0 at all. Before the release, I spoke with Dornfest on the phone on Friday, and there was no news. The first time I heard about RSS 1.0 was in the official announcement.

Let me ask an impolite question: if "RSS 1.0" was conspired without any voting or discussion, without bringing the committee together to make decisions, what would you do?

UserLand has done a lot of work to develop and popularize RSS. Now you kick us out and take this name. This is very excessive. If I want to continue developing, I have to use a new name. Tim, can you tell me why this happened and how it happened?

I did not find any historical emails discussing the use of the name RSS 1.0. Winer stated in his email that he did not attempt to control RSS but merely wanted to use it in his product.

Many developers grew tired of the community's endless debates and decided to develop a new version. In 2013, another fork occurred. Developers created a new version called Atom—removing RDF but embedding XML namespaces. Atom was submitted as the final version to the Internet Engineering Task Force (the organization responsible for developing and promoting internet standards).

After that, there were three different RSS versions on the market: Winer's 0.92 (updated to RSS 2.0 in 2002 and renamed "Really Simple Syndication"), the RSS 1.0 from the RSS-DEV working group, and Atom. Today, RSS 2.0 and Atom are still active.

Decline

Different RSS standards indeed hindered the promotion of RSS, but they did not prevent RSS from becoming popular in the 2000s. In 2004, The New York Times began using RSS to provide headlines, starting to popularize RSS and its usage among ordinary users. Later, Google Reader, which had millions of users, was launched in 2005. By 2013, RSS had become popular enough that The New York Times even declared in Swartz's obituary that RSS was "ubiquitous." Before one-third of the Earth's population registered on Facebook, RSS was the only link to internet news for many people.

The New York Times published Swartz's obituary in January 2013. At this point, RSS had reached a turning point, gradually becoming an unfamiliar product to many. Google Reader was shut down in July 2013, with the surface reason being a continuous decline in user numbers over the years. Many online commentators declared RSS dead. However, even before Google Reader's closure, the number of users using RSS had already been decreasing. In May 2009, Steve Gillmor wrote on TechCrunch: "It's time to completely shut down RSS and switch to Twitter. RSS couldn't get any worse." He pointed out that Twitter was a better information subscription tool because, in addition to providing articles, it could also offer different people's perspectives.

Today, RSS is not dead, but it is far less popular than before. Many people have tried to explain the current state of RSS, and the most convincing explanation comes from Gillmor's 2009 assertion: social networks have replaced RSS, providing users with updated information and generating profits for the companies operating social networks. Just as Google shut down Google Reader to promote Google+. Because Google could indeed profit from Google+, while Google Reader could not be monetized. In 2013, Marco Arment, the founder of Instapaper, stated on a podcast:

The closure of Google Reader seems coincidental, but it is actually the latest casualty in the war between Facebook and Google on the internet. It appears that Google Reader still had a large number of users, but in reality, it conflicted with Google+'s strategy: Google needed people to use Google+ to read and share, in order to compete with Facebook for users' time, advertising data, advertising revenue, growth, and more.

This shows that both users and tech companies believe that social networks are more efficient than RSS.

Another interesting theory about the demise of RSS comes from The New York Times, which had always wanted to recommend RSS to users. In 2016, it complained that RSS was not user-friendly enough for ordinary users and was too geeky to use. In 2004, before the RSS icon was updated, The New York Times had been using an orange box linking to the RSS data source, which, when clicked, led to a full-screen page filled with XML links, intimidating ordinary users. This brilliant tweet pointed out the essence of RSS's demise:

Ordinary users do not find RSS easy to use because RSS is not designed for ordinary users and has too many technical barriers. Once there are better products, users will abandon RSS.

If RSS had iterated efficiently, it might have become more user-friendly; perhaps RSS could have connected people subscribing to the same channels to share their thoughts; or perhaps browser compatibility could have improved, leading to a better user experience. However, while members of the RSS community were striving to create consensus, large companies like Facebook were rapidly upgrading their products and continuously breaking conventions. While the community was still unifying opinions, efforts to improve the product were wasted on redundant work.

Davis told me that if the community could compromise and quickly reach a consensus, then Atom would not exist, and the time spent arguing could have been used to improve the product. So, when we ask ourselves why RSS declined, the first answer is that social networks replaced RSS; but if we dig deeper, why could social networks replace RSS? The answer may be that the difficulties faced by RSS developers were greater than developing Facebook. As Dornfest mentioned in his letter to the committee: "The political issues now are far more serious than continuous development iterations."

Thus, we remain submerged in information silos. Even so, the aggregated network that Werbach predicted in 1999 has been realized, just not in the originally envisioned way. After all, "The Onion" is disseminated through aggregated networks like Facebook and Twitter, and so is "Seinfeld."

I consulted Werbach for his opinion, and he agreed with my viewpoint. He believes that RSS is a failed technology because it did not integrate the blogging world, the content world, or different resources. However, the revolutionary nature of social networks lies in their ability to aggregate different content and resources, which is also the original vision of RSS and aggregated networks.

Unfortunately, the aggregation of information on the modern web exists only on a very few websites, meaning that no one can manage their online information reach as Werbach imagined. One reason is that RSS did not provide tech companies with the opportunity to control access and sell ads, so tech companies would not support RSS.

A more straightforward reason is that a unified standard from a centralized company is much simpler than a standard that requires community voting to reach consensus. Consensus is difficult to establish and time-consuming, and without a unified standard, developers will propose multiple competing standards. If this article offers any lessons, it is this: To achieve a better and more open web, the world needs better cooperation.