Lambda kicks in, a serverless world made of messages

Serverless vs Traditional Cost Graph

There were so many possible angles of attack in covering the Amazon Web Services re:Invent 2016 conference that I seemingly never got to the point of delivery. RedMonk covered the event with a new recap format, borrowed from fivethirtyeight media, which you can see over here. Fintan has written a couple of posts – one on Elastic GPUs, FPGAs, Deep Learning & Compute and another on Hybrid, Greengrass and the Future of On Premises Hardware.

Greengrass is intriguing, because it illustrates AWS Lambda, a serverless or functions-as-a-service model as an on ramp for hybrid cloud/onprem computing. As with pretty much any new AWS service, adoption is disappointing seemingly right until the point it’s a crushing success. In the couple of months of last year it still seemed that AWS Lambda was struggling to find a place. By the time serverlessconf in May in Williamsburg NY had created a center of gravity to talk about serverless it was apparent things were changing.

And by re:Invent in December I basically didn’t talk to anyone that wasn’t using AWS Lambda to augment their other infrastructure. Many of them were frustrated that they hadn’t really had a chance to properly understand Lambda.  We met super smart switched on people from startups, and they were like: “wait, AWS has that already?”, or “oh, we know AWS now has that feature, but we just haven’t re-architected to take advantage of it yet.”

Event-driven computing, with ephemeral managed functions, is a great model for efficient cloud computing. I have written before about how serverless is as much an economic phenomenon as a technical one. Please check out this great Trek10 post for more on that (I totally stole the lovely hand drawn chart above from them). Then check out this post by Fintan on serverless for DevOps.

One obvious issue with Functions as a service is manageability. If you split everything into functions that creates a significant management overhead. Joe Emison is a smart guy and he talks about this a lot. As with micro-services, where every system failure is like a game of Clue, Lambda function approaches create new overheads.  Call it the Serverless Premium.

As I said in our roundup at the time:

“I definitely see X-Ray (visibility) being used alongside Lambda Step Functions (new technology for managing Lambda proliferation, with a state machine and visual tooling for service flows).”

By GCP Next a couple of months back I was intrigued by “breaking the glass“, how to maintain visibility and manageability of serverless functions and PaaS. It could be time to resurface message oriented middleware concepts as a way to think about managing apps based on independent functions and microservices. Guaranteed delivery, fan outs, publish and subscribe etc.

Fintan got to the heart of it in terms of AWS:

“then there is step functions with retries, very powerful.”

Step functions with retries, a world made of messages and active points. Think of Kafka, for example, a distributed commit log, used by Salesforce to bridge the gap between and Heroku services, and now offered as a managed service on Heroku. Kafka isn’t a message bus, but boundaries are fluid. Message buses will definitely be part of the new world. Note Pivotal is getting serious about RabbitMQ, a 10 year old project that deserves some love.

Serverless is going to be huge, but I suspect we’ll see AWS delivering some MoM style functionality to help manage functions and triggers, above and beyond the monitoring and API Gateway functionality so far. Especially as we move into multicloud serverless apps messaging is going to prove its worth. 2 phase commits aren’t even a dirty word any more.

Service Mesh approaches are related, with technologies such as linkerd fast gaining traction. Good timing for to release linkerd 1.0 this week.

The distinctions between categories keep getting more fluid, and serverless is only accelerating the fragmentation. It will be very interesting to see what themes emerge from serverlessconf in Austin this week.


AWS, Salesforce and Pivotal are clients.

(Read this and other great posts  @ RedMonk)

LinkedIn Twitter
James, aka @Monkchips is co-founder of RedMonk, the open source analyst firm, which specialises in developer advocacy and analytics.