Where Google App Engine Spanks Amazon’s Web Services: S3, EC2, Simple DB, SQS

First off, I loooove Amazon Web Services (AWS), and we make heavy use of S3, EC2, Simple DB, and Elastic IPs for LittleShoot. We run everything on Amazon, but that’s about to change.  I’ve been in the App Engine beta for about a month, and despite all of the astonishing ways AWS makes building web sites easy, Google App Engine makes it far easier. Here’s why:

1) Google App Engine Has Better Scalability  

Google migrates your application across its infrastructure automatically as needed. With EC2, you have to manually detect machine load and bring instances up or down accordingly. You need to set up load balancing and clustering. While Amazon gives you far more control, it’s also far more work.  With Google App Engine, it’s all done for you.

2) Google App Engine Has a Better Database

Google App Engine’s Big Table blows Amazon’s Simple DB out of the water, and that’s coming from a big fan of Simple DB and Amazon CTO Werner Vogels. Simple DB thankfully yanks developers out of the relational database mindset and automatically replicates data across machines for scalability. You do have to learn a completely new query syntax, however, and, as this blog has noted, sorting is not officially supported. Simple DB is also still in beta.  

With App Engine, you’re using the same database, Big Table, Google engineers use to power some of the busiest sites on the Internet. Billions of queries have been hammering out kinks in Big Table for years. You know it will scale.  What’s more, App Engine’s “GQL” gives developers a familiar SQL-like syntax, lowering the learning curve compared to Simple DB.  Big Table also supports sorting.  Perhaps most significantly, Simple DB costs far more. While Google’s final pricing announcement later this year may change, today’s announcement didn’t mention any difference in price for data stored in the database versus anywhere else. On Simple DB, that data costs $1.50 per GB-month. On App Engine, it appears to cost $0.15 – $0.18 per GB-month. Wow.

3) Google App Engine is Cheaper 

Beyond the database, App Engine gives you the first 5 million or so page views per month for free.  That’s a lot of page views. It doesn’t put you up with the Internet’s top dogs, of course, but at 5 million page views you should be making cash. App Engine is free precisely when you’re building your company and keeping costs low is the most important. If you go beyond that 5 million, Google’s I/O event today will reveal newly announced prices that are remarkably similar to Amazon’s current offerings. They both price everything per GB or CPU-hour, and the numbers are barely distinguishable. That first 5 million page views and the apparent huge disparity in database storage pricing are by far the biggest differentiators, both dramatically tipping the scales in favor of Google.

Conclusion

For building scalable web applications quickly, App Engine beats AWS by a surprisingly wide margin.  Note, however, this refers specifically to web applications. For anything custom, you need Amazon. Because App Engine only supports Python, you also need Amazon for running any non-Python code. While this is a significant difference, many good developers are facile with multiple languages and can move rapidly between them.  Amazon’s flexibility makes it win out for many applications, but not for the most common application there is: web sites.  App Engine is more of a “domain-specific cloud” for web applications, but it’s shockingly good at what it does.  

Oh yeah, and it’s cheap.

10 Responses to Where Google App Engine Spanks Amazon’s Web Services: S3, EC2, Simple DB, SQS

  1. […] Where Google App Engine Spanks Amazon’s Web Services: S3, EC2, Simple DB, SQS First off, I loooove Amazon Web Services (AWS), and we make heavy use of S3, EC2, Simple DB, and Elastic IPs for […] […]

  2. cedric says:

    Hi Adam,

    Your feedback on AWS and GAE is very valuable; I’m considering myself AWS and GAE for my own project and am wondering what’s possible with GAE. You mention rewriting all your code in python so as to run on GAE, but according to the architecture page on LittleShoot.com (great name btw), you seem to leverage low level protocols such as UDP, STUN, TURN and SIP which as far as I know don’t run over HTTP. Is it possible to run those on GAE or will you have to stick with AWS for a part of your architecture ?

    Thanx again for your feedback.

    Regards,

    Cédric

  3. adamfisk says:

    Hi Cedric- Astute observation. I didn’t want to get into that in the article because I didn’t want to make it too focused on LittleShoot and our architecture. You’re completely correct that we use some low-level protocols like STUN, TURN, and SIP that you cannot run on GAE. We run all of that on Amazon and will continue to indefinitely. Those are also all open source implementations we’ve written in Java, and porting those in particular to Python would take a long time.

    This brings up another factor that’s somewhat unique to the LittleShoot architecture. As a P2P application, LittleShoot’s centralized *web server* components are very simple. That makes porting our web server code to Python a pretty straightforward project.

    I’d say that’s probably a good rule of thumb. If you either 1) have a simple web server or 2) are starting from scratch, GAE makes a ton of sense for the web server side.

    So, you’re completely correct we’re sticking with AWS for a lot of stuff on the non-web server side. Many sites won’t have such a complex architecture and will be more HTTP based, however.

    Best,
    Adam

  4. cedric says:

    That makes sens indeed. Porting everything to python would be a lot of work, which might not be worth it. You’re leveraging each grid provider for what it’s good at, very nice.

    I’m just concerned by the traffic between your GAE servers and your AWS servers. This additional inbound traffic in respect to AWS and outbound traffic with repect to GAE will generate additional charges which wouldn’t exist if you were locating all your servers at AWS.

    GAE is a lot cheaper indeed but given the additional charges, I guess the overall gain is a bit less than expected.

    What do you think ?

    Regards,

    Cédric

  5. adamfisk says:

    Hi Cedric- Yeah, the extra bandwidth costs for going from AWS to GAE and vice versa are certainly a concern. The extra latency is also somewhat of an issue. In our case, that traffic will initially just be AWS (our SIP servers) notifying GAE when peers come on and offline. So those are small messages in the scheme of things and a tiny fraction of our traffic — much less common than searches, for example.

    So at least for us it’s not a huge issue, and the benefits outweigh the costs. The scalability and reliability of GAE is the ultimate cost saver, really. We need to set up our web server and associated database for scaling. With AWS, we’d likely eventually run HA proxy load balancing between a bunch of nodes all talking to Simple DB. While this is doable, just banging on Simple DB to get it to do what we want is significantly more work than tweaking things for Big Table. With GAE, we don’t have to do the load balancing stuff at all. So the time savings are the ultimate factor for me. We still have to worry about scaling our SIP, STUN, and TURN servers, but that’s a problem we have in any scenario.

    I think this gets at an overarching theme, though, that everyone’s architecture is different and it’s important to understand the different options available. Just keeping up with what’s possible is a challenge these days with the pace of change. The key is to figure out the best combination of fast, cheap, and scalable for your application based on what you know you can implement.

  6. […] Where Google App Engine Spanks Amazon’s Web Services: S3, EC2, Simple DB, SQS […]

  7. magixman says:

    This review is spot on although the cost factor remains to be seen in the real world – still I think it is probably going to work out cheaper on GAE.

    The big attraction of GAE for me is certainly the database. Simple DB has only the D in ACID (Atomicity, Consistency, Isolation, and Durability). You can’t do transactions and what is worse there is no guarantee that you can read back what you just wrote since it may propagate. Frankly I just don’t see how you could use this for an application of any complexity and the case studies I have seen show you end up using a real database and have to deal with all the usual scaling issues. GAE has ACID to a degree (indexes suffer from a second commit phase delay) but at this is something you can work with.

    On the other hand GAE is missing lots of stuff in its little sandbox and one may well have to end up using a combination of AWS and GAE. Images and media are better on S3 and any sort of communication functions that can’t be done on GAE but that can be wrapped in HTTP can run on AWS virtual servers.

  8. adamfisk says:

    Hey what’s up Sam? Long time no see. I’m actually playing with App Engine stuff now and working off Guido’s excellent example code. Never used Django before, but it’s just so damn straightforward. Really nice to work with once you get up to speed (took me a sec for sure). The testing framework is the most recently impressive — way lower barriers to getting tests up and running on MVC code than the equivalents in Java land, although Java still certainly has its own advantages.

    I’ll soon be running up against any limitations of GAE’s ACID compliance with issuing temporary keys for LittleShoot in emulated “sessions.” I’ll be writing up a post on porting LittleShoot to GAE that will go into more detail on all of this. I’ve only written a few python scripts before, so it’s been fun plunging deeper into Python.

    It looks like I’ll be using a combination of GAE and AWS for the foreseeable future.

  9. It’s awesome in support of me to have a site, which is valuable for
    my knowledge. thanks admin

Leave a comment