It’s that time of the year again! AWS celebrated it’s sixth annual conference (called re:Invent) in Las Vegas, Nevada. The conference gathered more than 45.000 people this year. re:Invent 2017 is the place to be if you want to discover what is new on AWS. Not everybody has the time or the resources to go for a week to the conference, so we, as faithful AWS Partners, go there and bring you information on the new features and services presented there.
These are the most recent announcements (especially those that caught my attention) from re:Invent 2017.
- Our usual update on EC2 instances
- I want metal, baby!
- No new families?
- Spot gets better
- Time Sync
- Private Link update
- New Amazon MQ
- It’s containers’ time!
- Machine Learning
- The news from Werner Vogels, CTO @ Amazon
- Cloud is changing
1. Our usual update on EC2 instances
One of the fundamental base layers of Cloud Computing are the virtual machines on which all (even AWS) services run. Being a base layer doesn’t mean it’s static or less important: it has to be cared for and modernized. AWS announced an evolution to the “C” (Compute Optimized”) family a month or so ago, and now it announces it updates to the “M” (General Purpose) family with its 5th generation: the M5 family. As a rule of thumb, I always say that every generation of EC2 is cheaper than the previous one. Let’s see if this statement holds true for the new instance refresh:
Let’s take C4s’ with more or less the same characteristics:
As you see: for the same CPU, you get a bit more memory at 85% the price!
Now for the new M5 instances:
In M5, you get more CPU for the same amount of memory and less price.
So, as you see, it’s true. You get more “bang for your buck” when using new instance generations. I recommend that you upgrade when you can.
But that is just an incremental improvement to the instance families (a new generation).
2. I want metal, baby!
A surprising new addition was an entirely new family! We tend to think that Cloud is all about Virtual Machines, but that seems to have changed with this announcement: Bare Metal Instances. The first family of bare metal is the “i3.metal” family (equivalent to the i3 family). The special characteristic of these instances is that they provide direct access to hardware.
These are the first in a series of EC2 instances that allow the operating system to run directly on the underlying hardware while still providing access to all of the benefits of the Cloud. Bare Metal instances can take advantage of Elastic Load Balancing, Auto Scaling, Amazon CloudWatch, Auto Recovery an so forth. They are a first class citizens in EC2! Even being “bare metal”, they have access to EBS! And, as expected, they are able to access the full suite of AWS database, IoT, mobile, analytics, artificial intelligence and security services.
3. No new families?
In addition to that, AWS announced the H1 family, designed for applications that require low cost, high disk throughput and high sequential disk I/O access to very large datasets (they are designed for MapReduce, and other BigData workloads.
4. Spot gets better
First of all, AWS announced that they have simplified Spot pricing. The spot prices will now vary more slowly than before (where there could be radical changes in spot price from one moment to the other). This makes us more confident that our instances will not be taken away abruptly due to a big price spike. Let’s see how this is panning out:
Since the announcement (Nov 28), spot prices have stabilized for two instance types that I picked at random.
Moreover, AWS has announced Hibernation for Spot Instances. Before, Spot Instances would be stopped or terminated. Now, Amazon EC2 Spot can hibernate Amazon EBS-backed instances in the event of an interruption. This is great for applications that aren’t specially prepared for abrupt termination (note that you still have a two-minute notice before the Spot Instance is suspended, stopped or terminated). The application won’t notice anything, for it, it’s just as if you had closed and then opened your laptop.
5. Time Sync
An announcement that was quite silent but is relevant: the new Amazon Time Sync Service. This service provides a highly accurate and reliable time reference that is natively accessible from Amazon EC2 instances. Two important notes: you have to substitute ntpd for chrony (see https://chrony.tuxfamily.org/comparison.html) to use the service, and it’s free!
6. PrivateLink update
Also interesting is the update on AWS PrivateLink. This service is the newest generation of VPC Endpoints and it has been designed to provide access to AWS services in a highly available and scalable manner while keeping all the traffic within the AWS network. Now AWS extends the PrivateLink model, allowing users to set up and use VPC Endpoints to access their own services and those made available by others. This will avoid unnecessary VPC peering or VPC sharing.
CloudFormation is my favourite service in AWS and I consider it to be a core service (letting you automate cloud in a simple manner).
In 2018, CloudFormation will get Configuration drift detection capability in all AWS commercial regions. This will help CloudFormation detect when changes to resources managed by CloudFormation are made out of band (via the EC2 console, for example). This is a situation that is not wanted (since CloudFormation can do unexpected things, and even fail when updating a resource that has been updated out of band).
8. New Amazon MQ
Message brokers are used by companies for their business-critical applications. Messaging is key for them because it empowers their business. To avoid complications with third-party suppliers, AWS has launched Amazon MQ, a managed message broker service for Apache ActiveMQ. Easy to configure and easy to use.
Think of RDS for Amazon MQ. It takes care of the administration and maintenance of Active MQ, a popular open-source message broker that is fast and full of features. With this new service, we will have direct access to the ActiveMQ console, APIs and protocols for messaging. Also, Amazon MQ includes the use of AWS Identity and Access Management (IAM).
9. It’s containers’ time!
Containers are a powerful way for developers to develop, package, and deploy their applications. At AWS, they know that nowadays containers are the latest trend, seeing a +400% growth since 2016. Now AWS is providing us with two new services: Amazon Elastic Container Service for Kubernetes (EKS) and AWS Fargate.
With EKS, developers can use Kubernetes on AWS without maintaining Kubernetes clusters. Also, applications running on Amazon EKS are fully compatible with applications running on any standard Kubernetes environment, which means that we will be able to migrate Kubernetes applications to Amazon EKS with zero code changes. EKS will provision and run the Kubernetes control cluster for you. The compute power will come from instances that you provision and manage (like in it’s counterpart Elastic Container Service).
AWS Fargate allows you to run your containers on AWS without an underlying instance (well really AWS will be running the instances for you). With AWS Fargate, containers become a fundamental compute primitive, so users don’t have to manage the underlying instances.
With the latest releases from AWS, the database panorama gets interesting. Now Aurora gets new features, DynamoDB goes multi-region and Amazon Neptune is born.
The new AWS Aurora Multi-Master feature will allow customers to distribute writes for their databases in multiple different AZs, being able to scale out both reads and writes (until now you could scale out only reads)
An announcement that really shocked me was Aurora Serverless. It will provide an Aurora database in a serverless fashion. ‘What is a serverless database?’, you may ask. Basically, Aurora will give you a database endpoint and scale it for you, even “be shutting it down”. Now you will really pay for what you use, enabling cost savings and new patterns for some more than common scenarios (think of daily imports and processes, devel environments, etc).
DynamoDB is not forgotten, getting features to make it more manageable. “In the olden days” you would have to manually set up architectures that backed up DynamoDB tables. Now, with its new backup and restore feature, customers will be able to set up on-demand backups, point in time restore for short-term retention and backup hundreds of them instantly.
Another neat feature is DynamoDB Global Tables. we will be able to distribute our DynamoDB tables across multiple regions in an active-active fashion (every region accepting writes). This will make globally distributed, multi-region architectures easier.
And, last, but not least, Amazon Neptune. A new service that provides a graph database to complete the database space on AWS. RDS provides relational databases, DynamoDB provides Key/Value and Document store. A graph database was missing in AWS. These databases follow a different paradigm for storing and representing information (they do so in the form of graphs). To learn more about graph databases, I recommend reading https://neo4j.com/graph-databases-book-b/, which discusses another graph database (Neo4j), but discusses the uses and data modelling for these types of database.
S3 has got new powers. With S3 Select, a powerful new S3 capability to pull out only the object data you need using standard SQL expressions, you can improve data access performance by up to 400%. Now you can select only parts of your data needed from CSV and JSON files (even gzip compressed) directly with S3 Amazon Glacier also gets the new Select capability. Now we can run queries directly on data stored in Glacier, retrieving only what we need and turning Glacier into a part of our data lake.
12. Machine Learning
Machine learning is becoming increasingly important each day. That’s why among all what is new on AWS, some new services caught everyone’s attention.
First of all, the new Amazon SageMakers’ proposal is to make Machine Learning easier. With this new service, you will be able to easily build, train and deploy machine learning models. Next, Amazon Rekognition gets video powers. This is a deep learning powered video analysis service that tracks people, detects activities and recognises objects, celebrities and inappropriate content.
Both Amazon SageMaker and Amazon Rekognition are going to be key while using the latest AWS device: Amazon DeepLens. The world’s first Deep Learning enabled developer kit.
There are also some sophisticated new services for Machine Learning on AWS. First, there is the new Amazon Transcribe, offering automatic speech recognition. It will transcribe even phone quality audio. After you have the text derived from a phone conversation, you may not be able to understand it. Don’t you worry, though. AWS also launched Amazon Translate, a new service that automatically translates text between languages. And if you don’t have time to read it all, there is also the new Amazon Comprehend, a natural language processing (NLP) service that uses machine learning to find insights and relationships in text.
Internet of Things has been one of the top topics when talking about what is new on AWS. Amazon Web Services has launched new services to make IoT easier.
From AWS IoT 1-click, a service to create an AWS Lambda trigger in one-click, to AWS IoT Device Defender, a service to define and enforce security policies for fleets of devices. IoT gets easier with new services that simplify IoT devices management, security and analytics.
14. The news from Werner Vogels, CTO @ Amazon
He started going back to the past to talk about 21st-century architectures (controllable, resilient, adaptive, data-driven). Of course, a lot has changed since 2012. According to Vogels, the key technology drivers of today are data, IoT, GPU-based computing for Machine Learning and Deep Learning. Which clearly go along with the latest AWS services announced the day before.
Anyway, he also revealed some new services on AWS. First, Alexa for Business, that opens AWS intelligent assistant for work. You’ll be able to say “Alexa! Start a meeting!” and she will wire you to the Amazon Chime meeting you had scheduled. You’ll be able to add your own capabilities to Alexa for Business.
There was also some good news for developers: AWS Cloud9, a Cloud IDE for writing, running and debugging code, in the cloud. Something quite neat is its ability to facilitate pair programming (when two people are editing the same file, the changes get updated in real time). It has support for Lambda functions, and maybe the best news about Cloud9 is that it is FREE.
Finally, a new we all were expecting: new features for AWS Lambda. Now Lambda gets more powerful, with API Gateway VPC integration, Concurrency controls, the max memory is upped to 3GB, .Net Core 2.0 and Go support. Also, the new AWS Serverless Application Repository allows users to share Lambdas.
15. Cloud is changing
This year’s re:Invent has made me reflect on the Cloud evolution over the years. Years ago, I would explain Cloud like you had a box Legos. A limited amount of bricks, and that with time, new bricks would appear.
Little by little, over the years, new bricks have been offered, and I’ve had to stop using the “limited amount of bricks”, but still would be using the Lego analogy.
Now it’s time to change the analogy altogether. I see Cloud as a hardware store. There are lots of tools to choose from, and each one of them has specialized use cases with overlapping functionality. There are various types of screwdrivers in a hardware store that adapt to different use cases (a fixed screwdriver is the best cost wise if you want to screw something from time to time. If you want to screw hundreds of screws per day you may need a power screwdriver). The hardware store gives a full offering of different tools for everybody’s needs, and sometimes, you need help to choose the right tools for you. We can help you with that part 😉
TAGS: Amazon DeepLens, Amazon MQ, Amazon SageMaker, Aurora, aws, AWS Fargate, Bare metal instances, cloudformation, Containers, deep learning, IoT, Kubernetes, machine learning, privatelink, reinvent, reinvent2017, serverless, spot, TimeSync, Werner Vogels