AWS re:Invent 2018 - What happened in Vegas this time?!

To maximise my ‘goshwow’ time at AWS re:Invent, I decided to get a head start on writing my conference blog this year. I thought I'd start by getting some predictions from the trainers about what we at QA thought they might announce. I offered a prize of a randomly-selected, probably-mis-sized vendor t-shirt to whoever got one correct.

What we at QA predicted would be announced (in no particular order)

1. A Machine Learning certification
2. Auto-scaling for EC2 Fleet
3. A "We've solved the speed of light problem" announcement. (It’s going to happen I tell you!)
4. Making their internal Java implementation publicly-available
5. Ruby runtime for AWS Lambda
6. Confirming that the Moon is, in fact, a giant S3 bucket
7. Two new storage classes for S3: TAPE ("because customers keep asking for it") and CARVED_IN_STONE (minimum retention period 100 years)
8. A MacOS AMI which will run on the new a6.large instances
9. Amazon Joe - a programmable coffee machine and its API
10. re:Invent Europe
11. AWS PersonalLink - an interface to the human brain, with a (literally) embedded chip, which will allow us to upload knowledge without having to study it
12. ML-based Auto Scaling
13. AWS Stark - an AWS on-premise solution
14. Inter-region data transfer over inter-region VPC Peering connections to get standard intra-region pricing ... or in fact global availability of inter-region VPC Peering (apart from China and GovCloud, natch)

How did we do?

Some of our predictions actually came true before the conference, but I've left them in to make our guesses look more impressive...

1. ML Exam - this was announced the week before re:Invent and I sat it on the Monday. 

1. ASG for Fleet - again, announced the week before re:Invent (scale instances across purchase options).
2. Speed of light - sadly not announced. Well, this year at least...
3. Java - announced the week before re:Invent (Amazon Corretto)
4. Ruby on Lambda - that's a t-shirt for... um... me. Not surprising at all, but you could tell who the Ruby developers in the room were from the applause! Werner kind of spoiled it a bit straight afterwards though, by saying "or in fact, any language!" He mentioned C++, Erlang and COBOL. Yes, you did read that correctly. But I think it may have been a joke; C++ and Rust are published as reference implementations
5. The moon is S3 - I got a little bit excited when I thought I saw a picture of the moon on a slide, but it turned out to just be Werner's head.
6. S3 Tape - That's a partial for Justin - maybe a sock rather than a t-shirt. Deep Archive - coming soon, mega-cheap storage class at 0.001c per GB per month. No I don't want to spend any time thinking about how they can do it that cheap...
7. MacOS - Another partial for Justin. Perhaps he'll get the other sock. They did announce the a1 family (and the r5a and the m5a), but it's "a" for ARM rather than "a" for Apple. We can run cheap ARM-based workloads in the cloud, running on Annapurna Graviton processors.
8. Coffee - This was my favourite prediction, sadly no dice.
9. re:Invent Europe - Nope! However, they did start putting up "save the date" notices for re:Inforce, a security conference running in Boston in June, which was announced on the 25th.
10. Brain interface - Not this year...
11. ML auto scaling - Announced the week before re:Invent (Predictive Auto Scaling). However, I'm counting this as a partially-correct prediction, because I suspect that under the bonnet, this feature is using Amazon Forecast (preview), which was announced by Andy Jassy during his keynote. Along with, of course, Textract (preview) and Personalize (I can't find an announcement...), a bunch of pre-trained / fully-managed ML models for forecasting time-series data, doing OCR++ on steroids and recommendations / personalization. In addition to the ML Marketplace for SageMaker, where people can sell and buy custom models and algorithms, SageMaker RL, a manged reinforcement learning service which I've described as "metaphysics for mortals". So now we can do supervised, unsupervised and reinforcement learning in one place. If, that's assuming, we can be bothered, rather than using one of Amazon's pre-trained models (or indeed renting one from someone else). And just because there weren't enough ML announcements, AWS Inferentia, a custom ML inference chip and Elastic Inference, attachable inference processors so you can boost your model serving times. I also have to mention Amazon Comprehend Customisation, which was announced just before the conference but I seem to have missed it...
12. AWS on-prem - This was meant to be a joke!!!!! It kind of started with the announcement of Firecracker, an open-source micro-VM environment (5MB memory overhead) which is behind things like Lambda and Fargate, which effectively means you can run micro-compute on-demand on-premise (and on ARM and elsewhere in future). So that was cool. Then Amazon RDS on VMWare (public preview) was announced, which I think is self-explanatory. But then, and honestly unless I fell asleep at some point, this wasn't even a keynote announcement, up pops Amazon Outposts, enabled by the same technology that's powering RDS on VMWare (just like i3.metal comes from this very cordial relationship between AWS and VMWare). I was definitely not expecting that!
13. Cheaper global peering - Close one! Transit Gateway and Global Accelerator, for managing complex VPC peering topologies and for getting your traffic into AWS's global private network more quickly. I'm not sure if this even counts as a partial because, well, you pay extra for them, not less... No, I've decided I'm awarding a partial to Andy, because they've definitely launched a couple of features that expose more of their global network implementation to us.

Anything Else Good?

Well, there were a lot of mostly fairly obvious announcements, which have all been covered in far more detail already by much more committed bloggers. I'm typing this bit at McCarran airport waiting for my flight back home, so this list is from memory and I'm probably missing some really important ones, but clearly this is the stuff that's sticking out for me. You can find them all here.

A Bunch of storage stuff

• Infrequent Access for EFS
• Intelligent tiering for S3. Presumably driven by ML behind the scenes; automatically lifecycle un-used objects into cheaper S3 storage classes (and back out again)
• SFTP as a service on top of S3. Very useful from a migration point of view
• Datasync, a custom protocol for transferring data from your on-prem into S3 and EFS. Requires a local agent
• EFS for Windows seemed to push a lot of people's buttons. Except of course it's not called EFS for Windows, it's called FSx and there's a high-performance variant called FSx for Lustre

A bunch of management tools

• Security Hub - what it sounds like
• A well-architected tool - to help ensure that you are.
• Cloud Map - managed service discovery as a service. I have some ideas for how to play with that already.

Some developer-centric announcements

Werner's keynote was heavy on the developer announcements.

• Lots more IDE plug-ins for Serverless development.
• Lambda Layers - re-usable Lambda components and
• Nested Applications - mashing up apps launched from the Serverless Application Repository. Werner's keynote was heavy on the developer announcements.
• Loads more service integrations for Step Functions, so I don't have to write a Lambda to get Step Functions to send a message to SQS or SNS for example

Databases

• DynamoDB on Demand - auto-provisioning for DynamoDB - stop guessing capacity!
• Transaction support for DynamoDB
• DMS support for DynamoDB - migrate your Cassandra database to Dynamo (others will come soon I suspect)
• Global Aurora - was actually announced last year, but it looks like it's now available.
• New databases! TimeStream, designed specifically for time-series data; in the olden days, we'd recommend using DynamoDB for this, but then accessing the data could be challenging, so having a database designed from the ground up to support time series is going to be great.
• QLDB - Quantum Ledger Database - it's, er... it's a quantum ledger database. Think internal blockchain for your single source of truth. It's an append-only, immutable database that is cryptographically verifiable. Of course they also announced Managed Blockchain for more public networks.

What's trending in the world of swag this year?

Well as you know, socks became the order of the day in 2017. Bizarrely I've received more feedback about that comment than anything else I've ever blogged...

Still lots of socks. Prize draws for Star Wars Lego sets (you know, like the £650 Millenium Falcon that we all really want but can't really justify buying for ourselves) were a popular method of scan-baiting, as were Amazon Shows and DeepLenses.

But the real trend this year and I think it's a good one, was for badges - there were lots of official and limited edition "pins" given out for attending various sessions and events, and lots of the exhibitors had obviously been informed, because many of them were also giving out exclusive pins. Looking at a delegates' lanyard became the de facto method for working out their interests / specialisms / certifications. Yes I do have a lot of certs...

What about extra-curricular activities?

re:Play was bigger than ever, with even more crazy / fun activities, but was also right up at the end of the Strip at the Festival Grounds because the usual venue, right by The Venetian, is now a building site (presumably becoming yet another resort hotel). Being Amazon, we had dozens of transportation options but I decided to walk. It wasn't the 26 minutes mentioned on the field guide...

Skrillex (Werner's favourite DJ) made a return as the headliner. According to Werner, he actually asked to come back for the first time since 2014. I'm sure he was very good, if you like that sort of thing.

I tried the Ballcano out. It was a combination of a ballpond and an assault course. I won't be doing it again, it was an hour-long queue and it would be an understatement to say that I didn't really enjoy it; I'll leave it to the youngsters in future.

Pub Crawl night was pretty cool - my first stop was at Madame Tussauds where they had a karaoke! I opened the show by performing my now-world-famous rendition of Copacabana to Beiber, MJ and Katy Perry. I clearly really impressed every other attendee as no-one else sang after me... (apart from the karaoke host guy...)

The Certification Appreciation soirée at Brooklyn Bowl was ... interesting. I think there was a better turn out from guests this year, possibly because it wasn't the same night as re:Play this time, but there seemed to be fewer AWSers this year. A whole bunch of the Curriculum Development team were in attendance and I finally got meet a trainer that I've had a number of communications with over the last few years, but I felt like last year there were more. That could just be because last year, a whole bunch of European trainers that I knew quite well attended and this year there were none. That's a completely subjective observation, I don't want my friends at AWS to see this as a criticism and I could be completely wrong. There's a first time for everything, right?

Come on, Dan! What about the ‘goshwow’?

OK, well, my first year at re:Invent was 2016, The Year They Drive A Truck Onto The Stage. Last year, there was nothing quite as crazy. Don't get me wrong, SageMaker and Fargate (and Aurora Serverless and Multi-Region Multi-Master and, and, and ... were all pretty cool, but not driving-a-truck-onto-the-stage cool.

So how about Amazon Ground Station? Because, let's face it, if you're going to put a satellite into orbit, you're going to need to build a ground station (or network of same) and that's going to cost you, so how about you focus your funds on the satellite (and delivery vehicle) and just rent a ground station when you need one. It makes perfect sense. It's absolutely bat poop screaming crazy! First prediction for next year? Satellite as a Service, powered by Blue Origin. You heard it here first...

Or, thinking about ML and IoT, what about an easier way to build and test robots? Maybe integrate it with SageMaker for the ML side. Massive industrial applications / implications of course, and you know, let's get rid of the undifferentiated heavy lifting. Let's call it RoboMaker for want of a better name. Or, and this is ridiculously goshwow, you may remember that last year Amazon launched DeepLens, an affordable "toy" machine vision device to encourage developers to start playing with Machine Learning (because SageMaker makes it easier, but we still have a massive skills gap). So this year they launched SageMaker RL, to democratise reinforcement learning but we need people to "play with it". And they launched RoboMaker. So how do we encourage people to "learn" how to "learn" machines and robots?

How about we sell you a robotic race car? We'll make it pretty cheap, heavily discount it during the conference and actually give a shedload away free during the conference [no I didn't get one and no I'm not bitter]. Let's call it DeepRacer and to really encourage people to play with it and work on the robotic RL models, we'll add an incentive by announcing the DeepRacer Championship Cup. Every AWS Summit next year will have a speedway to which you can bring your models. The winner from each Summit gets to take their model to re:Invent 2019, which presumably means at least a free conference pass, right? Massive incentive. And a heck of a CV addition for whomever wins the grand trophy...

Is that enough ‘goshwow’ for you?

I think AWS continually delight, surprise, innovate and excite their customers and I think if nothing else, this particular exercise has proved that whatever you think is coming next, you're probably wrong. Which reminds me, final memory, the Certification Lounge had a "testimonials" booth, where you could record a video of yourself answering one of five questions. I chose "what is the benefit of certification" because that's a question I answer nearly every day. One of the other options was "what do you see happening in the future". As we've seen, between us the sixteen Amazon Accredited Instructors at QA miserably failed to predict anything, so let's stop trying.

Just go with it, and embrace it, and re:Invent all the things. Again.