Every year at the end of November, all Cloud eyes are on one event: AWS’ re:Invent. The Cloud leader celebrates its annual conference since 2002 in Las Vegas, a mandatory event for anyone in the Cloud industry. Luckily for me, I was able to attend AWS re:Invent 2018 as a representative from CAPSiDE on our first year as AWS Premier Consulting Partner.
If you’ve never heard of this event, there is one word to describe it – big. This year, AWS hosted over 50,000 attendees and delivered over 3,500 sessions focused on their ever expanding Cloud platform and its latest releases and updates. Back from a crazy and Cloud-filled week, I shared the news with our team of Cloud engineers at our quarterly all-hands ‘Wednesday Break’.
With over 80 announcements at the event (60+ the week before!) – we stormed through each and every one and, together, we decided which are the new AWS features that, in our experience, are going to benefit most both customers and engineers. So here are our TOP 5 AWS re:Invent 2018 highlights!
Highlights @ AWS re:Invent 2018
1. AWS Outposts (preview)
One of the previews at AWS re:Invent 2018, AWS Outposts will bring AWS to existing data centres. With this new service, you will be able to enjoy a consistent, seamless experience across on-premises and the Cloud. This will be a great opportunity to run on-premises applications with the exact same APIs, consoles, features, hardware and tools you already use on AWS.
AWS Outposts will come in two variants:
- VMware Cloud on AWS Outposts, which allows you to use the same VMware control plane and APIs used to run your infrastructure.
- AWS native variant of AWS Outposts, which allows you to use the same exact APIs and control plane used to run in the AWS cloud, but on-premises.
For the time being, there is not much public information available about AWS Outposts, but what we know is that it will be widely available by mid-2019.
2. Dynamo DB: Transactions & On-demand
Another highlight of AWS re:Invent 2018 was the introduction of Dynamo DB Transactions and Dynamo DB On-demand. These features are now available globally in all commercial regions.
DynamoDB Transactions provide developers atomicity, consistency, isolation, and durability (ACID) across one or more tables within a single AWS account and region. Transactions bring the scale, performance, and enterprise benefits of DynamoDB to a broader set of workloads. Each transaction can include up to 10 items or 4MB of data. These are enabled for all single-region DynamoDB tables and disabled by default for global tables. Even when enabled remote regions, they are still eventually consistent.
With DynamoDB On-Demand, you can serve thousands of requests per second without capacity planning. It offers a simple pay-per-request pricing for read and write requests so that you only pay for what you use. For tables using on-demand mode, DynamoDB instantly accommodates customers’ workloads as they ramp up or down to any previously observed traffic level. If the level of traffic hits a new peak, DynamoDB adapts rapidly to accommodate the workload.
In the DynamoDB console, you can choose the on-demand read/write capacity mode when creating a new table, or change it later in the Capacity tab. While AWS did release auto-scaling of DynamoDB back in 2017, this new pricing model brings even more flexibility.
3. AWS Transit Gateway / Shared VPC
Finally, AWS announced at AWS re:Invent 2018 an AWS native hub-spoke connectivity solution! The new AWS Transit Gateway, shared by subnet, allows you to build a network topology to connect your existing VPCs, data centres, remote offices, and remote gateways to a managed Transit Gateway. This update will give you full control over network routing and security, even if your VPCs, Active Directories, shared services, and other resources span multiple AWS accounts. AWS Transit Gateway supports up to 1.25Gbps per VPN connection.
With AWS Transit Gateway, you can simplify your overall network architecture, reduce operational overhead, and gain the ability to centrally manage crucial aspects of your external connectivity, including security. You can also consolidate your existing edge connectivity and route it through a single ingress/egress point. Something you should consider is that Transit Gateway doesn’t support Aurora Serverless, CloudHSM, EMR, Glue or NLB.
The service commercial model is a pay-per-use per Transit Gateway ‘attachment’ by hours ($0.06 per hour in Ireland) + data transfer ($0.02 per GB in Ireland). An ‘attachment’ being each VPC subnet and VPN connection etc. Also, AWS Transit Gateway has its own route table with BGP support, as well as equal-cost multi-path routing an (ECMP) of VPNs for higher throughput should you need it – though note multiple VPNs will mean multiple ‘attachments’ and so related charges.
Overall this will greatly simplify connectivity and routing topologies, particularly for large multi-VPC/multi-account configurations.
4. Amazon CloudWatch Logs Insights
The new Amazon CloudWatch Logs Insights are designed to work at a Cloud scale, with no setup or maintenance required. They plow through massive logs in seconds and gives you fast, interactive queries and visualisations. Moreover, they can handle any log format, and auto-discovers fields from JSON logs.
Amazon CloudWatch Logs Insights include a sophisticated ad-hoc query language, with commands to fetch desired event fields, filter based on conditions, calculate aggregate statistics… You can also use regular expressions to extract data from an event field and visualize query results using line and stacked area charts.
Related to this AWS also announced CloudWatch Application Insights for .NET and SQL Server – though this is still in preview, so not too much information on this last topic yet.
5. Amazon EKS Managed Cluster Updated
Containers are everywhere and, among them, Kubernetes is the king. Amazon EKS now supports Managed Cluster Updates as well as Kubernetes’ newest version 1.11.5 for all new and existing clusters. Therefore, removing the need to create new clusters or migrate applications to a new cluster. This actually was announced just before re:Invent, but fixes such a big operational pain we couldn’t not mention it.
Now you can apply new versions of Kubernetes to a cluster either by calling the update-cluster-version API, or by using the “Update Cluster Version” button in the Amazon EKS console. You can get details on the status of in-flight updates by calling the describe-updates API and see in-flight updates by calling the list-updates API. This is a small but important new feature, particularly for those of us in operations, however at the time of writing at least, managed cluster updates don’t include Kubernetes Nodes yet, so still some work to be done by AWS on that one!
Extra: Time-series with Amazon Forecast
AWS is strongly betting on Machine Learning (ML), as it will be a key feature in future IT developments. There were a huge number of announcements in this area alone. For example, ML can be very useful at predicting time-series data by analysing historical data. At AWS re:Invent 2018, AWS announced the new Amazon Forecast solutions, now in preview.
Amazon Forecast is a a fully managed deep learning service for time-series forecasting, allowing users to build and operate scalable, highly accurate forecasting technology into an easy-to-use and fully-managed service. Amazon Forecast solutions can generate predictions on time-series data to estimate operational, IoT sensor or metrics, resource requirements and demand. For example, weekly/monthly sales, daily inventory levels, or hourly website traffic. These solutions can be applied to sectors like retail, supply chain, and server capacity, among others. Of course it works seamlessly with their new Amazon Timestream time-series database, also announced at AWS re:Invent 2018!
One more thing…
As usual, there’s always more with AWS – so many things I haven’t been able to cover and even several key announcements since the event! Including…
New AWS Region in Stockholm
Hey Europe, there’s a new AWS Region in the house! Stockholm has become the 5th region in Europe with eu-north-1, providing AWS users a total of 15 Availability Zones around the continent. The region also offers low-latency connections to other cities and AWS regions in the area. Moreover, AWS CloudFront edge locations are already operational in three more djacent cities: Copenhagen, Helsinki and Oslo.
Amazon RDS Minor Upgrade notifications
From now on, Amazon RDS will include enhancements to the Auto Minor Version Upgrade feature for Amazon RDS database instances. You will get a notification every time a new minor version upgrade is available for your database instance or cluster. The events will be pushed to AWS Events to CloudWatch/SNS or viewable via DescribePendingMaintenanceActions. Another key one for those of us focused on operations.
AWS Ground Station (preview)
If there was a price for the most unexpected announcement of AWS re:Invent 2018, for me anyway, it would be AWS Ground Station. With this service, AWS users will be able to rent access to ground stations near to a global network of regions and stream data directly from your satellites (because who doesn’t own a satellite these days!) to Amazon EC2 via custom ENI to take the byte stream. AWS Ground Station offers faster data downloads and immediate processing, reducing analysis times for whatever your use case may be: natural disasters, accurate weather prediction, business trends assessment… The project will start off with 2 ground stations and 10 more will be up for rent throughout 2019.