How the Web can finally Achieve Missing Features Caused by a “Worse is Better” Strategy

In the 1960s, Ted Nelson envisioned a universal hypertext system called Xanadu that was much more sophisticated than today’s web.  Xanadu was carefully designed to achieve the entirety of Nelson’s vision in an elegant way.  Xanadu was what Richard P. Gabriel would call The Right Thing — a project that doesn’t compromise its design in the areas of Simplicity, Correctness, Consistency, or Completeness.  However, as Gabriel observed in The Rise of “Worse is Better“, “The Right Thing” often loses the competition for adoptiono to a design that prioritizes simplicity over consistency and completeness.

In 1990, when Gabriel wrote “The Rise of Worse is Better“, he was a Lisp programmer engaging in some self-criticism about the failure of Lisp and the MIT philosophy to gain market adoption verses C++ and Unix, but his argument could apply equally to Xanadu’s relationship with the web.

In his follow-up articlee, “Is Worse Really Better?“, Gabriel argued:

Worse-is-better programs are like viruses that spread quickly and are soon pervasive. And over time, if they are successful, they will be improved. The argument is an evolutionary one a system that establishes a large territory and molds its environment before rivals show up has the advantage over those rivals.

The web’s dramatic growth certainly meets Gabriel’s standard for viral spread. The question remains as to whether the compromises the web made will ever actually be improved upon to the point that it approaches the “Right Thing”:

Once the virus has spread, there will be pressure to improve it, possibly by increasing its functionality closer to 90%, but users have already been conditioned to accept worse than the right thing. Therefore, the worse-is-better software first will gain acceptance, second will condition its users to expect less, and third will be improved to a point that is almost the right thing.

Nelson’s design for Xanadu prioritized designing the correct back-end system with the goal of enabling others to develop the desired front-end functionality.  By contrast, as Jeff Atwood describes it, the web developed with the absolute minimal architecture necessary:

the most fundamental building block of the web, the hyperlink, barely works at all. Hyperlinks are fraught with peril and pitfalls even under the best of conditions. The current state of hyperlinking is almost literally the stupidest thing we could build that works.

What’s still missing from the web

The web is still missing 4 main features proposed by Xanadu.  It is not possible to graft these features onto the web using the uncompromising principles of “The Right Thing”, however it is now possible to design an application for web authors that adds the first two missing features —

1. two-way links and
2. contextual citations.

Advances in cryptocurrencies make some approximation of the third feature — micropayments — foreseeable in the medium-term, while the fourth feature — transclusion — is likely only fully possible after a completely new back-end is implemented that does not use html’s embedded markup.  However, a “quick and dirty” implementation that mimics transclusion and parallel quotation is possible in the mid-term.

Bridging towards our Potential

My CiteIt.net Contextual Citations are a partial implementation of parallel documents that can bridge far enough towards Nelson’s vision of parallel documents for the public to see contextual citations in action.  My hope is that once the public is able to see what is possible, they will begin to demand a more full implementation of parallel documents and entrepreneurial programmers will be inspired to develop such designs.

4 missing features

Features that are possible now are listed below in green.  Orange is used to designate mid-term features and red is used for long-term goals:

1. Two-way Links/Fix Link rot
2. Contextual Citations/Parallel Documents
3. Micropayments
4. True transclusion

The Importance of Business Strategy

While the individual features are technologically feasible, it will take more than just technology for hypertext to realize its potential.  It is of equal importance that a business strategy be found to generate the funding to develop and maintain the technologies in an application that is adopted and by a sizable market of users.

 

One such strategy could combine two existing projects into a single app with the following two features:

1. Link rot : Wayback machine:
2. Two-way Links /Contextual Citations : CiteIt.net
   

Showcase Project: Wikipedia

1) Internet Archive

To gain customer interest for an app, a showcase project could be used to demonstrate how such a system could work on a large enough scale.

The Internet Archive currently partners with Wikipedia to fix linkrot using its Wayback machine. The Internet Archive also has plans to implement Contextual-like Citations for Wikipedia sources.

2) CiteIt.net

Contextual Citations, as implemented by CiteIt.net in this document, are a “bridging”-technology for bridging from tranditional Html links to fully parallel links as envisioned by Ted Nelson. Although CiteIt.net’s Contextual Citations do not display both sources in parallel, they, along with something like Google’s Text Fragments allow the author to specify the exact quote, even if the document contains multiple instances of the quote.  With CiteIt, Website owners can choose whether to display the context using a popup window or  a contextual div above or below the quote.  If sufficient adoption is achieved, browser performance and display could be optimized through a plugin or native functionality.

In the immediate term, CiteIt.net offers a WordPress plugin for the old classic pre-Gutenberg editor.  This app and webservice could be extended and integrated into The Internet Archive’s Wikipedia project.  This would require a fair bit of work to make the features production ready, but when these two features are achieved, they could serve as an attractive showcase that advertises two of hypertext’s missing features.

Automatic Partnership

Bringing these features to market on one’s own would be a challenge, so it may be useful to partner with a company like Automattic, which makes WordPress‘s Jetpack and Anti-spam app Akismet.

The benefit to Automatic and Jetpack of adding these features is that there are likely a significant number of WordPress authors who would appreciate the these features:

1. First, the app would have the ability to save an archived copy of every page authors link to.  An app that would submit these links to the Wayback machine would enable authors to create permanent links to their sources, even if the page has been taken down.
2. 

   Google Text Fragments

   The second part of the app would feature CiteIt.net’s Contextual Citations, which would make it easy for authors to create contextual citations that mimic the transclusion that Nelson advocates.  Present-day hyperlinked quotations only link to the page a quote is coming from, but tools like Google’s Text Fragments make it possible to specify the exact instance of the quote where a quote occurs multiple times. 1

Link Database

Another benefit of implementing Contextual Citations is that it would allow all citations to be voluntarily collected in a database that could be made widely available to developers and researchers.  This would allow a partial realization of one of Xanadu’s goals of having 2-way links that provide the reader a way to query to a “page’s” incoming links, not just view the outgoing links, as is customary now.   It currently takes the resources of enormous companies like Google and Microsoft to spider the web to determine a page’s incoming links.  A public link database would give smaller groups of programmers an opportunity to innovate with web link depiction and analysis and a could encourage authors to use semantic markup with their quotations.  Once the groundwork is laid, I can imagine many programmers creating their own ways of visualizing the links between pages in parallel.

KISS: Limited Scope

Gabriel hyperbolicly called his approach “Worse is Better” but another term for it could be KISS — Keep It Simple Stupid.  The strength of KISS philosophy is that it does a better job of avoiding the scope creep that sometimes afflicts “The Right Thing”.  Limiting the initial scope of this Hypertext Improvement Proposal to only the first two features has the potential to provide authors and readers with tangible benefits while avoiding getting bogged down in additional complexity.

Parallel Documents

Once the first two phases have been completed (phase 1) and there is an established base of documents, work could be done on phase 2 — to upgrade contextual citations to an implementation of parallel documents.  This could involve building out a javascript framework to load the linked sites within the browser window or it could involve adding support for parallel documents to the browser.  Browsers like Brave and Firefox may have an incentive to innovate where others are content to let others be first in this area.

Nelson provided a proof of concept demo (slow load) of what he envisions Xanadu could look like in in a web browser.  Once existing document quotations have been marked up, programmers could experiment to develop their own interfaces.

Here is a very basic screenshot of what contextual citations could look like if they simply combined

• CiteIt, with
• Browser-based popup, with
• Google’s Text Fragments

Converting the contextual popup to a native browser window would enable readers to scroll through the entire cited document.  A line connecting the citing quote to the cited document could be attempted later, as Nelson intended.

Future Browser Innovation

There are hundreds of Linux Window Managers.  It is my believe that browser innovation will happen once work is done to build out the authoring infrastructure and authors have a way to display context through basic popups.

Conclusion

Richard P. Gabriel predicted that minimalistic viral software like the web would outcompete “the right thing”.  Though the web hasn’t matured the way Gabriel predicted, the potential for something approaching Ted Nelson’s vision is still there with the right technology and business strategy.

The approach I’ve outline starts with the strategy of:

• limiting scope so that visible progress is achieved quickly
• starting with two existing projects:
  1. The Wayback Machine
  2. CiteIt.net
• building a showcase with Wikipedia to demonstrates what authors can achieve if they use the Automattic app.
• partnering with an organization like Automattic and/or Substack that has an existing user base to build Editor support for these features, along with a database of marked-up quotes that can serve as a base for future innovation

 

I’m open to other approaches.  Send me an email if you have ideas.