This One’s for the People

OK, so I’ve received some feedback that my last blog entry was, well, maybe not so interesting for the average member of my exploding fan base. I don’t want to use the word “boring”, but it turns out the vast majority of people in my life have not chosen to specialize in the next generation of peer-to-peer network architectures. They need more, and I’m here to give it.

So, this one’s a little something for the people. A little something to really cap off the summer of 2006. It’s all about a little discovery I made the other day that could, well, change everything.

Have you ever joyfully begun to sip from a nice hot cup of coffee in the morning, peacefully anticipating how that cup of coffee will launch you into the day, only to discover that no matter what you do, the coffee drips a little bit under the lid every time you take a sip? I’ve had this experience probably 20 times. With each damn sip, a little more coffee drips out and destroys the entire good morning coffee feeling you’ve paid good money for. Every time this happens, I find myself infuriated at the incompetence of the coffee cup designers that they could have designed such a pathetic excuse for a coffee cup. Who’s with me?

Well, the other day the cause of the coffee cup carnage came to me in a rare moment of clarity. This all happens because the seam of the coffee cup — the line running along the cup from top to bottom where the cardboard is glued together — is algined with or near the coffee cup hole you’re sipping from. Just a slight twist of the lid will solve the entire problem. That’s right. It’s that simple. As a result of this disasterous misalignment of lid and cup, little devilish drops of coffee make their way over the lip of the cup and under the plastic lid — through that tiny little extra space created from the cup opening aligning with the seam! It changes everything!

Now, some of you may say “Adam, you’re an idiot. I’ve known that since my second cup of coffee.” To you I say, read another friggin’ blog. These are the types of insights you’ll find here. If you’re just soooo smart that you’re unable to appreciate them, well, then, you’re not welcome.

Actually, this blog will continue to be primarily devoted to exceptionally dry, jargon-filled rants about peer-to-peer networks and how we can architect them for the maximum benefit of us all. Like I said, though, this one’s for the people. A little something to lift us all up and to just make our lives a little bit better, all with less coffee spilled on our shirts. Spread the word.

Towards an Open Infrastructure

Writers such as Jon Udell, Tim O’Reilly, and Yochai Benkler have alluded to the rise of the “open infrastructure” where open source and likely peer-to-peer projects provide the network primitives that can be combined and built upon to create services competing with everything from Akamai’s content delivery to Amazon S3’s data storage.

OK, I’m on board. Love it. Stepping back for a moment, though, this is really a new description of an idea that’s been around for some time under different guises. Sun’s JXTA project, for example, has for years attempted to deliver precisely the type of network primitives that would characterize such an open infrastructure, even providing for the type of “service delivery network” Udell envisions. JXTA is a pain to set up, though, and the project is simply not as focused or coordinated as it could be. It also has not had a killer use case to put it over the top. I say this as someone who has participated actively in the JXTA community and as a genuine fan of the project.

The Globus project took this step into an open infrastructure years ago as well, emerging from the world of grid computing. Globus allows developers to use the grid for everything from data backup to distributed processing. New terminology can nudge old technologies over the hump into the mainstream consciousness, however, as we’ve seen in the last several years with AJAX. Could Globus be to “open infrastructure” what LightStreamer is to “AJAX”? It’s possible.

OK, so let’s forge ahead. What are the key components of an open infrastructure? In many ways, an open infrastructure resembles an open, networked operating system. Much like an operating system, the open infrastructure would provide access to CPU, disk space, memory, and network resources. The p2p/grid computing world offers all of these. To compete with the Googles and the Yahoos of the world, what Udell calls the “galactic clusters”, an open infrastructure system needs to leverage p2p’s resource pooling. Projects like Globus emerged from the academic world, from the desire of researchers with access to supercomputers to share their processing power. Projects like SETI@home have demonstrated how much further this idea can go if we bring edge resources into the fold, creating the world’s largest supercomputer from networking hundreds of thousands of much less powerful computers. SETI@home or even the more generalized BOINC don’t quite meet the open infrastructure demand, however, as they demonstrate the specific use case of distributed processing with a central, coordinating node. To utilize all edge resources, we need a more generalized system that does not rely on this centralized coordination and that can fulfill any task. The problem then becomes the heterogeneous nature of Internet hosts, particularly the fact that most nodes are behind Network Address Translators (NAT)s or firewalls. NATs cut off a node’s resources, preventing it from contributing to the pool. That’s where Session Initiation Protocol (SIP) steps in. With resounding success in VoIP, SIP and with the associated STUN and ICE protocols provide a robust and generalized way for two or more users to connect on the Internet regardless of their specific NAT or firewall configuration.

So, an open infrastructure needs p2p to be most effective, and P2P needs SIP. Together, the possibility emerges of a generalized system where the collective bandwidth, CPU, memory, and disk space resources of every internetworked computer on the planet can be dynamically combined to perform arbitrary tasks. Other protocols and specifications such as XMPP, RDF, and SPARQL would also likely play vital roles in such a system, as would distributed hash tables, but I’ll get more into that later.

As Udell points out, the open infrastructure would closely parallel the pooling of knowledge resources we see in Wikipedia or the collaborative filtering of Slashdot. In this case, though, we’re collectively sharing the resources of the computers themselves. The computers are doing the collaborating.

Hello Blog People

So I’ve finally broken down and started a blog. I’m doing this out of desperation. I’ve simply accumulated too many Writely rants and quasi-journal entries addressed to the public that it’s a little embarrassing not to lay it all out there. I also come across daily developments in the tech world that I feel so strongly about that I’m constantly stopping my work to jot down my opinion, only for no one but me to ever see it! In many cases this is the desire to correct others in the interest of furthering general understanding, as I experienced most recently with one of Cringely’s entries where he’s a bit off in his understanding of p2p, but that will have to wait for a future entry.

To give you a little background, I received my undergraduate degree from Brown University in U.S. History and Computer Science. My first job straight out of college was working as a programmer at LimeWire. In fact, I was hired to work for a company called “LimeObjects”. By the time I actually started 2 weeks later, though, Mark Gorton and others had decided to fold LimeObjects and to turn it into LimeWire. I will talk in more detail about my 4 years at LimeWire in future entries. For now, though, I’ll just say that people like Chris Rohrs, Greg Bildson, Susheel Daswani, Robert Soule, Sumeet Thadani, Sam Berlin, and Anurag Singla, and I had a heck of a lot of fun from the early days of Gnutella through turning Gnutella into the mature family of protocols it is today. We also worked with a tremendous and talented group of people from around the world who participated in the Gnutella community through the Gnutella Developer’s Forum (GDF). Bringing LimeWire from nothing to a profitable company, and open-sourcing it along the way, was an invaluable and a fascinating experience I will always cherish.

I decided to leave LimeWire in February of 2004. There were many factors leading to my decision, as I will elucidate in upcoming entries. They all relate to technology and to my interest in interoperability between peer-to-peer and the larger Internet. My ideas were too radical a departure from LimeWire to make them a part of the same project, however, and it made the most sense to strike off on my own and to start a separate company. That company is now Last Bamboo LLC, founded with 4 friends/partners, and now very close to the public release of our first application, “LittleShoot”. I can’t say more about LittleShoot yet, but stay tuned. It’s cool.

For now, this blog will comment on related happenings on the Internet. I’ll also throw out little tidbits from the history of LimeWire and Gnutella here and there as appropriate. I hope everyone enjoys it, and please don’t hesitate to tell me when you think I’m nuts. As Yochai Benkler and many others have pointed out, the Internet’s enabling of collaborative filtering of ideas is essential to maximizing human creative resources. Please engage.