Amazon just started: http://developer.amazonwebservices.com/connect/entry!default.jspa?categoryID=152&externalID=1697&fromSearchPage=true

I’ll soon be running up against any limitations of GAE’s ACID compliance with issuing temporary keys for LittleShoot in emulated “sessions.” I’ll be writing up a post on porting LittleShoot to GAE that will go into more detail on all of this. I’ve only written a few python scripts before, so it’s been fun plunging deeper into Python.

It looks like I’ll be using a combination of GAE and AWS for the foreseeable future.

The big attraction of GAE for me is certainly the database. Simple DB has only the D in ACID (Atomicity, Consistency, Isolation, and Durability). You can’t do transactions and what is worse there is no guarantee that you can read back what you just wrote since it may propagate. Frankly I just don’t see how you could use this for an application of any complexity and the case studies I have seen show you end up using a real database and have to deal with all the usual scaling issues. GAE has ACID to a degree (indexes suffer from a second commit phase delay) but at this is something you can work with.

On the other hand GAE is missing lots of stuff in its little sandbox and one may well have to end up using a combination of AWS and GAE. Images and media are better on S3 and any sort of communication functions that can’t be done on GAE but that can be wrapped in HTTP can run on AWS virtual servers.

So at least for us it’s not a huge issue, and the benefits outweigh the costs. The scalability and reliability of GAE is the ultimate cost saver, really. We need to set up our web server and associated database for scaling. With AWS, we’d likely eventually run HA proxy load balancing between a bunch of nodes all talking to Simple DB. While this is doable, just banging on Simple DB to get it to do what we want is significantly more work than tweaking things for Big Table. With GAE, we don’t have to do the load balancing stuff at all. So the time savings are the ultimate factor for me. We still have to worry about scaling our SIP, STUN, and TURN servers, but that’s a problem we have in any scenario.

I think this gets at an overarching theme, though, that everyone’s architecture is different and it’s important to understand the different options available. Just keeping up with what’s possible is a challenge these days with the pace of change. The key is to figure out the best combination of fast, cheap, and scalable for your application based on what you know you can implement.

I’m just concerned by the traffic between your GAE servers and your AWS servers. This additional inbound traffic in respect to AWS and outbound traffic with repect to GAE will generate additional charges which wouldn’t exist if you were locating all your servers at AWS.

GAE is a lot cheaper indeed but given the additional charges, I guess the overall gain is a bit less than expected.

What do you think ?

Regards,

Cédric

This brings up another factor that’s somewhat unique to the LittleShoot architecture. As a P2P application, LittleShoot’s centralized *web server* components are very simple. That makes porting our web server code to Python a pretty straightforward project.

I’d say that’s probably a good rule of thumb. If you either 1) have a simple web server or 2) are starting from scratch, GAE makes a ton of sense for the web server side.

So, you’re completely correct we’re sticking with AWS for a lot of stuff on the non-web server side. Many sites won’t have such a complex architecture and will be more HTTP based, however.

Best,
Adam

Your feedback on AWS and GAE is very valuable; I’m considering myself AWS and GAE for my own project and am wondering what’s possible with GAE. You mention rewriting all your code in python so as to run on GAE, but according to the architecture page on LittleShoot.com (great name btw), you seem to leverage low level protocols such as UDP, STUN, TURN and SIP which as far as I know don’t run over HTTP. Is it possible to run those on GAE or will you have to stick with AWS for a part of your architecture ?

Thanx again for your feedback.

Regards,

Cédric