P2P in Flash 10 Beta — the Questions Facing a YouTube, Skype, and BitTorrent Killer

May 21, 2008

As I’ve reported, the inclusion of P2P in Flash 10 Beta represents a fundamental disruption of the Internet platform. As with all disruptions, however, this one will progress in fits and starts. Flash 10’s details limit the full power of its P2P features. While features like VoIP will be fully enabled, it will take some ingenuity to turn Flash 10 into a more generalized P2P platform. Here are the issues:

1) Flash Media Server (FMS)

You’ll need Flash Media Server (FMS) to take advantage of Flash P2P. At $995 for the “Streaming Server” and $4,500 for the “Interactive Server”, FMS is beyond the reach of most developers working on their own projects, severely limiting Flash P2P’s disruptive potential. In an ideal world, the new P2P protocols would be openly specified, allowing open source developers to write their own implementations. As it stands now, a single company controls a potentially vital part of the Internet infrastructure, and encryption will likely thwart the initial reverse engineering efforts of open source groups like Red5.

2) No Flash Player in the Background

As David Barrett (formerly of Akamai/Red Swoosh) has emphasized on the Pho List, Flash Player only runs when it’s loaded in your browser. As soon as you navigate to another page, Flash can no longer act as a P2P server. P2P programs like Red Swoosh, BitTorrent, and LittleShoot don’t have this limitation, and it means Flash can’t save web sites as much bandwidth as those full-blown applications can. This limits but does not eliminate Flash’s threat to CDNs. Sure, you could get around this using AIR, but that creates another major barrier to adoption.

3) Usability

While Flash 10 has the ability to save files to your computer and to load them from your computer (essential for P2P), it pops up a dialog box each time that happens. While this is an important security measure, it cripples Flash 10’s ability to mimic BitTorrent because you’d have dialogs popping up all the time to make sure you as a user had authorized any uploads of any part of a file.

4) Limited APIs

While all the required technology is there in the Real Time Media Flow Protocol (RTMFP), ActionScript’s API limits some of the P2P potential of Flash 10. P2P downloading breaks up files into smaller chunks so you can get them from multiple other computers. Flash 10 can only save complete files to your computer — you can’t save in small chunks. As a result, you’d have to use ActionScript very creatively to achieve BitTorrent or LittleShoot-like distribution or to significantly lower bandwidth bills for sites serving videos. It might be possible, but you’d have to work some magic.

So, that’s the deal. There’s still a lot more documentation coming our way from Adobe, so there are undoubtedly useful nuggets yet to be discovered.

Even given all these limitations, however, the key point to remember is the Internet has a new, immensely powerful protocol in its arsenal: Matthew Kaufman and Michael Thornburgh’s Real Time Media Flow Protocol (RTMFP). While Flash might use it primarily for direct streaming between two computers now (think VoIP), it introduces the potential for so much more.

Keep your helmet on.


Cringely, Skype, Open Infrastructure

August 16, 2006

I have seemingly plunged myself into a running debate with Robert X. Cringely about the finer points of p2p telephony and NAT traversal. I would first like to acknowledge the remarkable breadth of Cringely´s technical knowledge. An early Apple employee and a longtime savvy observer of technology, Cringely somehow has a strong grasp of the highly specialized technology underlying VoIP. His range is startling.

That said, Cringely continues to stumble over the finer technical details. I only bring this up because the fundamental problems with Skype lie in those details, as I´ll explain. They are what make Skype a “closed garden” and a detriment to the “open infrastructure” I´ve advocated. Cringely puts forth an explanation of Skype´s NAT traversal, asserting that Skype uses STUN, TURN, and ICE servers to do the heavy lifting. There are several problems with this assertion. First, there´s no such thing as an “ICE server”. ICE is a client-side protocol that makes use of STUN and TURN “candidate” endpoints to establish the best possible connection between two peers. Second, and most importantly, Skype doesn´t implement any single one of these protocols regardless. While Cringely likely understands this, his post makes no reference to this key distinction.

For the uninitiated, STUN, TURN, and ICE allow clients to traverse NATs. They are all IETF drafts that continue to change frequently, and they are typically used alongside SIP to power VoIP. This is true for almost every VoIP provider, except for Skype. Skype unfortunately chose to implement a proprietary version of each one, breaking interoperability with other VoIP providers. This makes Skype much like your cell phone in the U.S., where you typically cannot switch cell phone providers while keeping the same phone. If you switch from Sprint to Verizon, for example, you have the joy of putting your $200 phone in a box in your closet or going through the hassle of selling it on eBay. Skype has given us a similar gift. You could never use Skype software with Vonage or Gizmo, for example. If Skype used SIP, TURN, STUN, and ICE, you theoretically could.

Skype´s proprietary protocol is also an issue on web pages. On eBay, you can now press a hyperlink to call users over Skype. This link starts with “skype:” much as typical links start with “http:”. Links on web pages to initiate a phone call will become increasingly common. You could easily imagine links on MySpace pages for calling other users, for example, and some savvy MySpace users likely already have them. The problem is that every other VoIP provider uses SIP, which long ago standardized its own interoperable URIs that start with “sip:”. So, because Skype chose to implement proprietary versions of everything, you will likely have to choose between two links when making a call, one of the form “skype:afisk” and another of the form “sip:afisk@lastbamboo.org”.

Imagine if web servers made a similar choice circa 1994. In this world, instead of every link starting with “http:”, some would start with “http:” and others would start with “mywretchedprotocol:”. All browsers would have to support both. What a nightmare! We have Skype to thank for starting us along that path with VoIP.

The implications of this issue go further. While SIP certainly has its problems (why did they ever include UDP?), the carefully designed interworking of the SIP family of protocols is a thing of beauty. SIP does not depend on STUN or TURN or ICE, for example, just as STUN does not depend on TURN or SIP or ICE, etc. This allows each protocol to evolve independently and allows different developers to implement different parts of the protocol family. One open source project can simply write a STUN server, for example, while another could write a SIP server, and another a TURN client. In the end, the user gets better software because engineers can break apart the problem and focus on implementing one piece well. And they´re all documented in Internet drafts that anyone can read. Skype´s use of proprietary protocols butchers this system.

Because of its careful engineering, SIP can also carry any type of traffic over any type of transport. You can use SIP to transfer RSS feeds using HTTP over TCP as LittleShoot does, for example. Or you can just make a phone call. SIP is simply the signaling protocol used to establish the connection. Skype doesn´t have anywhere near this flexibility.

Skype´s decision to use a closed protocol has security implications as well. When calls are routed through supernodes, for example, there´s a built-in “man-in-the-middle” that can monitor all traffic. Skype encrypts calls, but do they use both server and client authentication to prevent the man-in-the-middle from launching a replay attack? If they don´t, then it´s theoretically possible for an attacker to become a supernode to listen to all of your calls. As a closed protocol, Skype isn´t open to public scrutiny in the security community that could otherwise identify and fix such vulnerabilities. There could be people implementing this exploit to monitor and decrypt your Skype calls right now. While one independent security audit claims Skype does implement both client and server authentication, this is one person evaluating their architecture as opposed to the throngs of security experts who would be eager to identify holes if the system were open. We just don´t know.

These issues all point to the importance of an open infrastructure and to the power of SIP as a bedrock of the next generation of Internet applications. As people like Vint Cerf have noted, SIP may be to the next ten years what HTTP was to the last ten, unless Skype gets in the way and everything degenerates into a battle of ugly proprietary implementations of the same thing.

I choose to believe that good engineering wins in the end. Protocols like SIP, HTTP, and XMPP will enable a new generation of far more powerful applications capable of seamlessly circumventing NATs and pooling resources to put the maximum possible power in the hands of the user.

Cringely Doesn’t Understand P2P

August 2, 2006

Mark Stephens, a.k.a. Robert X. Cringely, the guy who seems to get everything, just doesn’t seem to understand peer-to-peer. His recent post “The Skype is Falling” misses the mark several times. As Yochai Benkler has noted so clearly, p2p is about computers collaborating to share resources, much as Wikipedia is about humans collaborating to share knowledge. On Wikipedia, different people have different levels of knowledge on different topics. That’s what makes it work so well! Specialists in different areas on Wikipedia don’t have to worry about the things they don’t know about — they just contribute what they can. Together, they combine the best of human knowledge for everyone’s maximum benefit.

Peer-to-peer is at its most powerful when it does precisely the same thing — when each peer contributes as much as it’s capable of contributing. Just like humans, computers come in many shapes and sizes. Some have 200 GB hard drives and a dial-up modem. Others are cell phones with no hard drive and little memory, but with bandwidth to spare. Well-architected p2p networks take this into account, harvesting whatever resources are available at any time for the maximum benefit of the network as a whole.

Computers without firewalls and not behind NATs are one of the most precious resources on p2p networks because they supply services others can’t. Because anyone can connect to them, they serve as the glue holding any p2p network together. Without these nodes, most p2p networks simply would not function. Perhaps most importantly, they facilitate connections between NATted/firewalled nodes, allowing any node on a network to theoretically connect to any other.

This is where Cringely just doesn’t get it. He goes into detail describing how Skype uses “servers” to facilitate NAT traversal between peers. He points to the use of servers as somehow making Skype not p2p. The fact is, Skype uses distributed non-firewalled peers as “servers” to allow other peers to connect. This is ahh, well, precisely like every other p2p network on the planet. This architecture could not be more peer-to-peer. In fact, this is p2p at its best – the network is harvesting all available peer resources dynamically.

Cringely claims that “a lot of Skype connections aren’t p2p at all” because of this server interaction and that these servers need to have a “surplus of bandwidth to handle the conversation relay.” This is where his thesis really starts to unravel. He apparently does not understand that the servers are simply used as a signaling protocol to relay contact information between two peers. The call itself typically happens directly between the two computers using UDP NAT hole punching, a practice that VoIP really brought into mainstream use but that is also used in various games and in most p2p networks. There are certainly cases where the hole punching fails, such as with trying to connect 2 peers both behind symmetric NATs, but the call is typically direct. If it weren’t, call quality would often be horrible.

This is a key difference because the vast majority of the bandwidth requirements for calls are for the call itself. The headers exchanged for establishing the call are negligible. I would hazard a guess that 99% of the packets transferred in VoIP calls around the globe are voice packets, not headers. These packets never touch the server unless UDP hole punching fails. It’s an open question as to what Skype does when UDP hole punching fails, but I’d actually be surprised if they were even using “supernodes” in this case, likely instead routing their calls through their own servers. I just don’t think supernodes would be able to provide enough bandwidth for high enough call quality.

Far from the sort of faux p2p Cringely describes, Skype’s use of non-firewalled nodes to negotiate voice sessions between two or more peers is p2p at its finest. Now, don’t get me wrong, I have plenty of problems with Skype’s use of a proprietary protocol instead of SIP, and I’d prefer them to be open source, but this part of Cringely’s analysis just doesn’t make any sense.

Hello Blog People

July 20, 2006

So I’ve finally broken down and started a blog. I’m doing this out of desperation. I’ve simply accumulated too many Writely rants and quasi-journal entries addressed to the public that it’s a little embarrassing not to lay it all out there. I also come across daily developments in the tech world that I feel so strongly about that I’m constantly stopping my work to jot down my opinion, only for no one but me to ever see it! In many cases this is the desire to correct others in the interest of furthering general understanding, as I experienced most recently with one of Cringely’s entries where he’s a bit off in his understanding of p2p, but that will have to wait for a future entry.

To give you a little background, I received my undergraduate degree from Brown University in U.S. History and Computer Science. My first job straight out of college was working as a programmer at LimeWire. In fact, I was hired to work for a company called “LimeObjects”. By the time I actually started 2 weeks later, though, Mark Gorton and others had decided to fold LimeObjects and to turn it into LimeWire. I will talk in more detail about my 4 years at LimeWire in future entries. For now, though, I’ll just say that people like Chris Rohrs, Greg Bildson, Susheel Daswani, Robert Soule, Sumeet Thadani, Sam Berlin, and Anurag Singla, and I had a heck of a lot of fun from the early days of Gnutella through turning Gnutella into the mature family of protocols it is today. We also worked with a tremendous and talented group of people from around the world who participated in the Gnutella community through the Gnutella Developer’s Forum (GDF). Bringing LimeWire from nothing to a profitable company, and open-sourcing it along the way, was an invaluable and a fascinating experience I will always cherish.

I decided to leave LimeWire in February of 2004. There were many factors leading to my decision, as I will elucidate in upcoming entries. They all relate to technology and to my interest in interoperability between peer-to-peer and the larger Internet. My ideas were too radical a departure from LimeWire to make them a part of the same project, however, and it made the most sense to strike off on my own and to start a separate company. That company is now Last Bamboo LLC, founded with 4 friends/partners, and now very close to the public release of our first application, “LittleShoot”. I can’t say more about LittleShoot yet, but stay tuned. It’s cool.

For now, this blog will comment on related happenings on the Internet. I’ll also throw out little tidbits from the history of LimeWire and Gnutella here and there as appropriate. I hope everyone enjoys it, and please don’t hesitate to tell me when you think I’m nuts. As Yochai Benkler and many others have pointed out, the Internet’s enabling of collaborative filtering of ideas is essential to maximizing human creative resources. Please engage.