BitTorrent: Old Technology in a New Box

The myth of BitTorrent goes something like this: Bram Cohen, hacker extraordinaire, realized circa 2001 that it would be more efficient to break files up into pieces on different servers and to download those pieces separately. This would distribute the load across multiple servers, providing a more robust architecture for accessing the file. The trouble is, the practice was common well before BitTorrent came on the scene. Cohen simply wrote another implementation of a technology that had already become commonplace in the P2P community. The first implementation I know of was Justin Chapweske´s work on SwarmCast in 2000. As I remember it, Justin´s creativity pointed the way for us all.

Heck, we even released swarm downloading in LimeWire long before BitTorrent ever made a public release, as I first announced here. I wrote almost none of the downloading code, but my old LimeWire buddies Chris Rohrs and Sumeet Thadani have more of a claim to having “invented” swarm downloading than Bram Cohen. LimeWire´s also an open source project, and we were working on the swarming implementation as early as January of 2001, as you can see from the CVS logs. Cohen didn´t even start working on it at all until May of 2001. What´s more, it never occurred to us at LimeWire to think of it as a new idea because, well, it wasn´t.

Why do I care? It´s just that it keeps coming up, most recently in the O´Reilly e-mail from a couple of days ago seeking ETech 2007 participants, where they describe “BitTorrent’s use of sufficiently advanced resource locators and fragmented files” as the type of new innovation they´re looking for. I was a history major in college (in addition to computer science), so these things matter to me. Cohen himself perpetuates the myth, most blatantly on the BitTorrent web site where it says: “While it wasn’t clear it could be done, Bram wanted to enable effective swarming distribution – – transferring massive files from server to client with the efficiency of peer-to-peer — reliably, quickly and efficiently.” The fact is, it was clear it could be done because people like Justin and us over at LimeWire had already done it!

The Wired article on Cohen from January 2005 takes the cake, though. The article says “Cohen realized that chopping up a file and handing out the pieces to several uploaders would really speed things up.” Again, he “realized” it because he saw that others were already doing it. They go on to describe how traditional file sharing networks are “slow because they suffer from supply bottlenecks. Even if many users on the network have the same file, swapping is restricted to one uploader and downloader at a time.” It´s all just blatantly wrong.

Now, don´t get me wrong. I love BitTorrent. I think BitTorrent is amazing and a perfect example of the kind of enabling technology that makes us all more free. It offers the clearest hope for a future of media distribution beyond the inadequate cable and network broadcast model we see today. It´s just that BitTorrent´s innovation was far less sexy. BitTorrent worked because it did less, not because it had any amazing new ideas. BitTorrent took what many p2p applications were already doing and scrapped most of it. BitTorrent scrapped search. It didn´t bother with a fully connected network. It didn´t worry about file management. It just took the downloading component and packaged it up nicely. Cohen realized that the downloading technology alone was all people wanted or needed in many cases, and that the tricky distributed search part was often unnecessary. Hats off — it has really changed the way many of us think about technology.

That said, BitTorrent was old technology in a new package. The innovation was in the packaging.


9 Responses to BitTorrent: Old Technology in a New Box

  1. GV says:

    this is way over my head but I am enjoying a stroll through your mental meanderings.

  2. Ken says:

    Don’t forget edonkey! What really makes bittorrent shine is its transparent flexibility: choose your client, choose your search engines, choose your trackers, and choose what part of the “network” you want to be involved in uploading/downloading on.

  3. adamfisk says:

    Good point on the eDonkey front. Jed McCaleb’s actually a pretty good friend, a good soccer player, and a great programmer. I agree the strength of BitTorrent is it’s transparency, although I think it could go further. There are major, major issues with it that I’ll write about soon.

    1) It doesn’t implement HTTP, choosing to break compatibility with the entire rest of the Internet.
    2) It doesn’t traverse firewalls. Any p2p app worth its salt can traverse firewalls these days. Not BitTorrent.
    3) Two trackers can be distributing the same file, but they never know of each others’ sources.
    4) Making the server problem way too hard — all you need is a hash embedded in URL, and there should be 0 required server-side changes to implement multi-source downloading.

    Those are just the ones that come to mind right now, but I’m sure I can think of more. Oh yeah, it doesn’t publish resources in any standard way.

  4. David Harrison says:

    Bram Cohen’s major contribution was NOT swarming. It was the first system to address the lack of motivation for users to contribute resources to the swarm. The solution turned out to not be that difficult: repeated rate-based tit-for-tat with forgiveness. Or more simply put:

    Give and ye shall receive.

  5. adamfisk says:

    Yeah, I don’t buy it. Just running your application as a service solves the problem far better. In Bram Cohen’s tit-for-tat, you always have the same number of uploaders as you have downloaders. You’re immediately limiting yourself to the speed of the uploaders, a bad idea given slower upstream speeds than downstream. Woops.

    More importantly, though, tit-for-tat solves a problem no one has. Running the software as a service immediately makes a system perform far better than any tit-for-tat approach because you always have more uploaders than downloader, i.e. excess capacity to play with.


  6. Ant Bryan says:

    You might be interested in Metalink. It’s used by download managers (HTTP/FTP) and hybrid HTTP/FTP/P2P clients.

  7. nesha gavrilo says:

    and what about soulseek p2p software. it’s still delicious after so many years.

  8. adamfisk says:

    Soulseek is a spectacular tool for finding music. I like the niche they’ve carved out in terms of becoming a haven for more esoteric music that:

    1) Attracts true music aficionados
    2) Doesn’t make much money anyway, so the music industry doesn’t really care

    That said, I’m much more interested in general media distribution systems — music in particular doesn’t interest me all that much. That’s not to knock SoulSeek, but I’m personally much more interested in using peer-to-peer as an architecture for distributing all types of content, ideally non-copyrighted.

  9. licspoile says:

    Think about it… I want to improve my normal control Good joke 🙂 What makes a bowling alley so quiet? You can hear a pin drop.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: