Amazon Web Services: The De Facto Cloud API?

Read a couple of posts last week that coalesced some thoughts I’d been having into this one.  First was the fascinating rumor about a Google EC2 clone.  Hat tip to High Scalability Blog for putting me on to this one.  The second was James Urquhart’s musings about the desirability of the Amazon API’s as a standard.

James and most of his commenters are worried that we might standardize on something that doesn’t have all the bells and whistles.  Guess what guys, standards NEVER have all the bells and whistles.  By the time people get done arguing about them and they get enough momentum and use to be more than just standards in name, the world has moved on.  So what?  If we waited for standards to be perfect, we’d would have none.  The point of the standards is that they’re good enough to help reduce friction in areas where innovation at every turn is no longer desirable.

Are we there yet with the Cloud?

I think so.  I’m not saying there is no innovation left to do–there is tons of it coming, and it’ll all be good.  But, there is value at this stage in having a standard too.  There has been a lot of success demonstrated with the Amazon Cloud from companies large and small.  I would be curious to hear whether Amazon thinks their API should be a standard (and whether they’re prepared to give up some control in exchange), or whether they think it still has too much growing up to do.

I look at it like this:

First, the wonderful thing about standards is there are so many to choose from.  Just because we annoint Amazon as one such standard does not mean there can never be others that are completely different.  It does not mean the standard can never have a superset or optional features to cover many of the cases Urquhart raises.  And remember, there is still that period of making it into a standard during which it can be molded a bit before it settles in.

Second, speaking of supersets, I think it is important to think about Cloud Standards as a layered model.  I would definitely not put all of Amazon’s offerings into the standing.  Perhaps just EC2, EBS, and S3.  Note that S3 is spreading even more rapidly than EC2 as a de facto standard.  These three seem pretty benign, reasonably abstract (meaning they don’t expose too much ugly detail about what goes on underneath), and reasonably proven.

So what do we get for it?

If the standard works and matters, we get a lot more vendors supporting the standard.  That’s a good thing, and something I have to believe every Amazon Web Services client would very much like to see happen.  The second thing we get is that some, but not all, of the innovators will quit trying to reinvent the wheel that is the Amazon layer (IaaS if you must use an ‘aaS name for it) and they will move on to other layers.  Depending on how much more innovation you think that layer needs, this is a good thing.  For those that think it is a bad thing, there is still time for someone with a better mousetrap to put it out there and show us.  But what if making it a standard causes a ton of innovation around how to solve some of the problems like I/O bandwidth behind the scenes without affecting the API?  Wouldn’t that be an excellent thing?  Heck, I’d love to be able to add an “11″ to the AWS I/O speed control, who wouldn’t?

But time is a wasting.  For whatever reasons, it seems like the other players are very late to the party relative to Amazon.  If they wait too long, Amazon gets a de facto standard before the market has much leverage to pry loose some control.  This is arguably what happened around the IBM PC, DOS, and Windows.  BTW, you read that like its a bad thing, but it wasn’t.  We accomplished a heck of a lot by not having to continue innovating over the bus, the BIOS, and so on.  It just could’ve been a lot better if there hadn’t been so much control vested in Microsoft and IBM.  As I write this, I wonder a little bit if players like Google really aren’t taking too long.  Maybe AppEngine is the exact image of what they think the Cloud should look like, and the problem is the world just didn’t bite the way they have with AWS.  Google could be saying, “Okay, we hear you, so we’ll do it your way now.”  There has to be some reason they’d endorse S3 and potentially EC2.

If Google’s ready to go there, why not the rest of us?

(Cross-posted @ SmoothSpan Blog)