Elusiva, Virtualization, Google Base, LittleShoot Updates

April 10, 2007

Yes, I’m still alive. Barely. Actually, that’s not true. I’m alive and well, but I’ve been buried chin deep in LittleShoot code for the last several months and have been severely neglecting my blog.

First, I want to encourage everyone to check out my good buddy Igor Shmukler’s launch of his office virtualization company, Elusiva. Igor is of the breed of Russian-born programmers I’m sure many of you have encountered who have truly intimidating systems (as in operating systems) knowledge. Igor was soldering circuit boards and hacking Windows drivers at the age of 11 in Russia while I was trading baseball cards in Gill, MA. By the way, anyone want 50 Roger Clemens Topps rookies? Hmnn…maybe not. Seriously, though, Igor knows his stuff. If you have a need for office virtualization tools, I’d highly recommend checking it out.

In other news, LittleShoot hacking continues behind the scenes. The endless details have kept us from launching just yet, but we’re as excited as ever. For any open source folks out there, there are some really exciting modules to either use or contribute to. The multi-source downloading code, for example, is completely reusable as a separate jar, as it just about every other part of LittleShoot. If you feel like hacking on Ajax, the Hibernate database layer, or the SIP, TURN, ICE, or STUN implementations, they’re all carefully parceled out for your coding pleasure. They’re just not released quite yet.

LittleShoot has taken many twists and turns. Back in September I was ecstatic to start playing with Google Base, essentially offloading all of our searching and file publishing to Google instead of our servers. I love many of the concepts behind Google Base, like the embracing of open standards such as the Atom Publishing Protocol (APP), Amazon’s OpenSearch, and the simple REST API. In practice, though, Google Base was a disaster for us. The performance was just too inconsistent, and the documentation is contradictory. In some places they say updates to your data are instantly searchable from within your private account while in elsewhere they claim it can take up to 24 hours. In practice, they’re usually instantly searchable, but can take up to 24 hours every once in awhile. Good luck getting a straight answer from the Google Base folks. If you’re considering Google Base for anything that’s mission-critical other than creating a new venue for advertising your eBay sale items, proceed with caution!

We decided Google Base didn’t perform well enough for future LittleShoot users, and I implemented a centralized searching solution using Hibernate, MySQL, etc. I also hired Leo Kim to set up our server cluster over at Cari.net, and he did a great job implementing a scalable architecture that will keep LittleShoot humming along as the user base grows. Leo actually just got hired by my good old buddies over at LimeWire. I couldn’t quite match their salary offer just yet (ahh, like not even close). Someday, Mark, someday!

Let’s see, what else? Oh, I feel extremely fortunate to have retained Wendy Seltzer as our legal counsel, particularly for making sure LittleShoot adheres to the DMCA. Wendy is just awesome. She’s so passionate. She’s also going one on one with the NFL, and she’s winning. Check it out. What a great example of what you can do with a little knowledge. Its one thing to have someone who knows the law, but it’s something completely different to work with someone who is also truly passionate about the future of free speech and digital media.

Oh, one last thing. Bloomberg news recently published a feature on Mark Gorton, my former boss at LimeWire and a friend. Beyond coming up the idea for LimeWire and forming our original team, Mark is also passionate about making New York more bicycle friendly and about bringing free, open source software to non-profits and governments that often don’t have the knowledge or resources to take advantage of technology. Mark’s an amazing example of how you can live the life you want if you just give it a little effort.

That’s it for now. I’ll write more often as the LittleShoot launch draws near.

Advertisements

MySpace Zapr Link Tool, Bandwidth Hell, and NAT Traversal

September 12, 2006

I just read Mark Cuban’s blog for the first time in awhile, and I like his fast and loose style, so don’t be surprised if my posts get a little less formal.

Moment’s after catching up with Mark, Mick from Zapr blogged about the new MySpace Zapr link tool. I quickly gave it a spin. At first, it blew me away. The link you ultimately used to download some of Henning Schulzrinne’s fascinating lecture slides on my machine looked like this:

http://72.3.247.245:81/GETF?(null)&(null)&adamfisk&HORTON-LAPTO&2f615f21d986d501

I looked at that and scratched my head. I even shot off a quick e-mail to my good buddy Jerry Charumilind to figure out what I was missing here. I assumed the 72.3.247.245 was the IP address of the Starbucks I’m sitting in here in Sheriden Square, New York City, and that they had somehow figured out how to publicly address my machine using my locally-running Zapr instance to open up some ports. UPnP? No, that just wouldn’t work in most cases. Too brittle. Were they doing an ICE-style direct transfer, the way I would? Not possible without Zapr installed on both ends.

Then I turned to my trusty nslookup and discovered “72.3.247.245” is, in fact, one of their servers. I suspect they use the raw IP address to make it look like they’re doing something fancy, but, in fact, they’re just relaying traffic. Suddenly the world made sense again!

Don’t get me wrong, it’s still a nifty service and a nice implementation. It’s getting towards the seamless integration of the “dark web” I’d like to see — the integration of edge resources, in this case content. If they were using open protocols, I’d give them a thumbs up. Unfortunately, we can add them to the long list of services ignoring interoperability. If they are using standards, they’re certainly not advertising it. Aside from that, the main problem I see is their bandwidth costs if the service gets popular. Yikes! They’re not caching it, as your file immediately becomes unavailable if you go offline. This means not only will a user’s machine get hammered if something is popular, but so will Zapr’s. The user’s machine would just never hold up (think of the most popular YouTube videos hosted on your machine at home), and the Zapr servers would have a heck of a time too.

How do you get around this? Just like Skype does, just like Gizmo does, and just like LittleShoot does. Require users on both ends to have the software installed and pierce the NATs and firewalls to connect the computers directly. That solves the problem of killing the central server. What about the user’s poor machine? Keep track of all the locations for the file and load balance it across the Internet. How you do that is another question I’ll leave for another day (hint: not with BitTorrent).


Chris Holmes and Architectures of Participation

August 30, 2006

My good friend Chris Holmes’s recent Tech Talk to Google is now available on Google video. Chris’s work touches on a lot of things, but you can think of it as helping to implement an open standards and open source-based infrastructure for things like Google Maps and Google Earth. You should check out his thoughts.

I get all excited when Chris talks about open standards as a cornerstone of democracy. With the web changing rapidly, we all need to remember this lesson. The web itself was based on the simple open architecture of HTTP and HTML. Analogous standards exist for geographic data. Chris’s work focuses on expanding the web platform to also support geographic data, much as my work focuses on expanding the web platform to support P2P.

I’ll write more about “architectures of participation” in the future. While “Web 2.0” is a much catchier name, I think “architectures of participation” clears up a lot of the confusion surrounding these issues. I also think it digs deeper. A lot of the Web 2.0 thinking focuses on collaboration on the level of individual web sites. I have no problem with that, and I just love collaborative projects like Wikipedia. There’s a distinct lack of discussion about how architectures of participation at the standards layer enables all of this, though, I think because more people understand web sites than the standards driving them.

Wikipedia would, of course, never exist if we didn’t have HTTP and HTML. HTTP and HTML are really quite simple protocols, but look what they’ve enabled! Imagine what could happen if we really started growing the protocol layer of the web, integrating things like geographic standards and SIP onto standard web projects. What could collaborative projects do atop a more powerful infrastructure? I’m not sure, but it’s a question we should be taking a harder look at.