re:Invent 2023 Well-Architected Highlights

Now that the dust has settled and my jet lag has just about gone away, let’s take some time to reflect on the 2023 event, the announcements, and what they mean for Ubertas Consulting and our customers.

What a week!

By my count, there were almost 190 new launches, whether new services or new features for existing services.

Outside the excitement of new announcements, there were lots of opportunities for networking and exploring the expo floor. I attended a variety of community mixers where I made some great contacts around AWS User Groups, meeting many people willing to give up their time to help me learn how to grow my Ipswich group.

A real highlight was meeting two individuals who ran an AWS User Group in Uzbekistan. Hearing about their efforts was truly inspiring, and to top it all off, they gave me a gift that they had brought to re:Invent from home; it now sits on my bookcase as a great reminder of the power and importance of the AWS community.

On the expo floor, as well as speaking to sponsors and learning what problems they seek to solve, there was time to explore some of the more interactive exhibits. For example, some AWS employees had built SmartBar, a robotic bartender that mixes cocktails (and mocktails, when I visited at 11am), all driven by AWS technologies. Pretty cool!

The week ended with my Friday morning session (video available here) on how to evolve from migration to modernisation using the concept of modernisation pathways. The feedback we received was great, and the experience of speaking at re:Invent is something I won’t forget in a hurry.

Announcement highlights

So, 190 or so new announcements. That’s a lot to digest. Fortunately, I’ve been through all of them, picked out the highlights relevant to Ubertas Consulting’s areas of expertise, and aligned these to the six Well-Architected Framework pillars for ease, starting with cost optimisation.

You’ll note that there are no Generative AI announcements in my highlights. GenAI was the overwhelming theme of the week, and it deserves its own dedicated content. Keep your eyes peeled for this in the first quarter of 2024; we’ll explore how Ubertas Consulting perceives Generative AI impacting the world of migration and modernisation and our plans for utilising it.

Cost optimisation

This topic came to the forefront of everyone’s minds on Thursday morning with Werner’s keynote and the introduction of his set of 7 ‘laws’ under the umbrella of being ‘The Frugal Architect‘. This set out to reinforce the relationship between engineering decisions and cost, defining easy-to-understand ways of thinking to control cost on AWS.

Whilst ‘The Frugal Architect’ is helpful, it is theoretical. AWS also announced several new services and features on the topic of cost optimisation. Here are my highlights.

Cost Optimization Hub

The first is Cost Optimization Hub. Available in the AWS Management Console, this aggregates recommendations from over 15 different sources to give you a cross-organisational view of how you could optimise cost.

CloudWatch: new Infrequent Access storage class

Ingesting logs into CloudWatch can become very expensive. A new storage class for logs in CloudWatch has been introduced called Infrequent Access, offering a more cost-effective solution to ingesting logs that are rarely accessed.

To my mind, infrequently accessed logs are best stored in Amazon S3, but this provides an alternative for use cases where the additional log-specific features of CloudWatch are required.

Backing up to Amazon EBS Snapshots Archive

We (should) all know that a comprehensive backup strategy is crucial to running a Well-Architected workload on AWS.

Managed services such as AWS Backup make this easier and will likely achieve everything you need. But with comprehensive backups often come large data storage costs, so using different storage classes for less frequently accessed backup data is important.

At re:Invent, it was announced that AWS Backup now supports backing up to Amazon EBS Snapshots Archive, a cheaper storage tier for those older snapshots unlikely to require regular retrieval.

Security

AWS consistently preaches that security is job zero, and rightly so. Operating in the cloud offers significant opportunities for innovation and growth; however, with great power comes great responsibility.

The breadth and depth of services available on AWS means that it’s not always possible to know how to protect against all attack vectors. This is where the AWS-managed security services come into play.

A comprehensive suite of security tools exists, including Security Hub, GuardDuty, Inspector, and Macie. At re:Invent, several improvements to these offerings were announced.

Security Hub

There were three announcements relating to Security Hub:

1. The ability to centrally configure standards and controls for existing and new accounts and regions. A real time saver!

2. Security Hub findings are now enriched with more data to help with the triage of and response to security issues. The new data includes resource tags, the de-facto way of adding custom metadata to resources.

3. Improvements to Security Hub dashboards. Included in this are custom dashboards. These can now be created, allowing you to cater the information presented to the persona consuming them.

GuardDuty

It wasn’t just Security Hub that saw improvements. GuardDuty also had its time to shine.

Previously, GuardDuty supported the monitoring of containers running on EKS. At re:Invent, it was announced that runtime monitoring support in the container space had been extended to include ECS and Fargate workloads.

Non-containerised workloads aren’t missing out; EC2 runtime monitoring was also announced as being in preview.

IAM Access Analyzer

Outside of the typical AWS-managed security services, IAM Access Analyzer also received some attention.

Keeping IAM principals’ permissions to least privilege is a crucial part of the security model in the cloud, and it has now become a little easier.

IAM Access Advisor helps you identify actions carried out by a user or role to help build up new guardrails and policies. Now, Access Analyzer can identify unused permissions. Watch out for the pricing on this one; it’s charged per principal and can quickly rack up.

Access Analyzer also now supports custom policy checks with automated reasoning. It’s a great way to identify if new policies are more permissive than the existing ones; something that would be ideal to integrate into a CI/CD pipeline.

Reliability

We’ve touched on a reliability-related announcement in the cost-optimisation section, with support for EBS Snapshots Archive in AWS Backup.

But backups are only as good as the recovery process that uses them, and regular testing of this process is essential. Operations teams need to practice responding to failure events to keep to recovery point and recovery time objectives (RPO and RTO, respectively).

AWS Backup now supports restore testing

Manual testing of backup procedures can be time-consuming and is often forgotten about or de-prioritised. It was announced that AWS Backup now supports restore testing, allowing automated restores of backups to be performed. Restored resources can be validated before being torn down to keep costs minimal, all automatically.

Operational excellence

You can gather all sorts of data about an application running on AWS, but it’s often distributed across different dashboards and sections of the console. From an operational perspective, this can make it challenging to truly understand the overall health of your application.

myApplications

At re:Invent, AWS introduced myApplications, a new feature that aims to provide a single pane of glass across all facets of an application. Resources get grouped into the concept of an application using resource tags, which then make use of the filtering capabilities of other services to show you data specific to your applications. For example, the new enriched findings in Security Hub or cost allocation tags in Billing and Cost Management.

myApplications builds upon existing functionality provided by Resource Groups. You can read about the full capabilities of this new feature in the announcement blog post.

Performance efficiency

Within the Well-Architected framework, the performance efficiency pillar is described as “guidance that helps customers use cloud resources efficiently to meet their business requirements and maintain that efficiency as demand changes and technologies evolve”. With that in mind, I’d like to highlight two new releases that help customers do just that.

ElastiCache Serverless

The first is the introduction of a new option for running a cache on AWS, ElastiCache Serverless. This is designed to take both the cognitive and operation overhead of managing a scalable cache away from customers, letting them focus on the business value derived from using the cache.

It supports Redis and Memcached engines, delivering exceptional benefits for those using Memcached in particular, given the often complex challenges that must be solved when scaling and high availability are required.

There is a minimum storage charge when using this service, so in my eyes, it doesn’t accomplish the true ‘scale-to-zero’ characteristic of a serverless offering; however, when compared to OpenSearch Serverless, another offering with a similar pricing model, it is significantly lower and unlikely to be prohibitive to use.

Amazon Aurora Limitless Database

The second is the announcement of Amazon Aurora Limitless Database (currently only available in limited preview).

It was previously possible to use Aurora at massive scale, but only for read access; write transactions were bottlenecked by the performance of the single writer instance. This changes with Limitless Database. Write transaction throughput and performance can now be scaled beyond the constraints of a single writer database instance.

The way in which this is implemented is pretty clever; I’d recommend reading the announcement blog post for all the details.

Sustainability

There was a surprising lack of sustainability-focused announcements at re:Invent. The only one I could find relating to the sustainability pillar in some way, is the introduction of new Graviton4 and Trainium2 chips designed by AWS. The push for customers to use custom AWS silicon continues, backed by evidence showing greater energy efficiency and price performance.

‘Cost as a close proxy for sustainability’

We’ve talked about the introduction of Werner Vogel’s ‘The Frugal Architect’ laws as part of the cost optimisation section of this article. In his keynote, when discussing cost, he used a phrase I wanted to highlight to conclude the sustainability highlights.

Werner said, “Cost is also a close proxy for sustainability”. Due to the cloud billing model (pay-per-use), the amount you spend is a reasonable estimate of the amount of resource you use. It’s a nice way to think about (roughly) measuring the environmental impact of your workloads without diving into the complexities of carbon emissions.

Sustainability is a crucial topic for us here at Ubertas Consulting. Our thoughts and actions are outlined here.

Rounding up

It’s been an incredible experience being a part of this year’s re:Invent. However, I’m appreciating a rest after all the excitement! I hope our coverage of the event has been valuable to you.

With so many big announcements across a range of themes, our focus now shifts to getting to know and use these new features and services thoroughly, putting them to good use in 2024.

Personally, I’m thrilled that these new releases address some of the pain points I’ve experienced. From a business perspective, we’re excited to bring these new features and services to our customers and help them continue to innovate and build on AWS.

If there’s a specific announcement (even if I’ve not highlighted it here) that you’d like to chat about, feel free to message me on LinkedIn or book a free consultation.

Alex Kearns
Principal Solutions Architect, Ubertas Consulting

LinkedIn