distributed roots

  • by

A lot of my conversations with friends these days have been about my choice to leave Facebook and other centralized platforms. The short answer is, those companies are not the Internet that I want to build.

My career in software development started at a now-forgotten company called Static Online, which straddled the space between games and video streaming. I joined in ’99, the company peaked in late 2000, and closed in 2001 – typical for a dot.com startup in those days. What we did there has shaped how I see the internet, and I’m grateful to the folks I got to work with. If I have any regrets, it’s that we didn’t publish our work, and that it took me another 6 years to connect with the F/OSS community.

You see, we were working on building a distributed, dynamically-routed, peer-to-peer network capable of live video streaming from any user (even someone on dial-up) to as many users as wanted to watch. This wasn’t possible with any other technology at the time, and while it resembles the architecture of several current projects, such as ZeroNet, the tech isn’t widely available even today.

With Static focused on video streaming in an era when most people were either on dial-up or slow DSL connections, we had created a protocol to:
– divide a large file (eg, a video stream) into chunks, identified by unique hashes;
– represent the relationship between chunks;
– dynamically identify the nearest (fastest) server, relative to a given client;
– upload the chunks in parallel to one or more servers, which in turn streamed to several of their neighbors, and so on;
– exponentially amplifying the bandwidth potential of a single upload stream.

If this sounds familiar to you, it should. This approach is remarkably similar to the BitTorrent protocol that was published a few years later. We also implemented a file sharing tool within our client/browser, but this wasn’t our core business so that feature just collected dust.

I was a basically a kid at the time that we built all this, and when the funding at Static ran out all I wanted to do was go study tai chi and meditate by a stream for a while. (No joke – I did that in the summer of 2001)

Fast forward to 2019 and we are seeing a massive resurgence of peer-to-peer innovation. At the same time, with all the privacy concerns around Facebook and the recent ban of adult content on Tumblr and related sites, more and more users are moving away from centralized platforms, where content moderation can be done by AI’s, and whose ad-click buttons track our movements all over the web.

Instead, folks are opting for decentralized / peer-to-peer services once again — like Mastodon and PeerTube.

Right now, I feel a little like that kid again, dreaming of a globally decentralized network where users build communities by co-hosting the infrastructure they share, of an internet where we are all unburdened with online ads that spy on us.

It’s good to look back 20 years and feel like I’m still on the right track.