Travtus Looks to the Future With Amazon SageMaker

AWS SageMaker Machine Learning Project

About Travtus

Travtus is a London-based Artificial Intelligence Research & Development company.

With their long-standing experience in Real Estate Operations and with the latest research in Deep Learning and Conversational Interface design, they bring automation to Real Estate and are the creators of a leading Artificial Intelligence teammate Adam, who enables Real Estate companies to provide exceptional customer service.

Adam helps to bring efficiency and automation across all areas of Property Management, by engaging with customers in the way they prefer. From email and text, to web chat and apps, Adam helps customers access the services they need 24 x 7 x 365.

The Challenge

Travtus’ product Adam is an AI teammate for real estate. It uses machine learning to automate multifamily property management, reviewing resident communications to gather intelligence on the property.

Preparing and managing training data, MLP Ops and Production model monitoring are time consuming activities and Travtus were keen to explore Amazon SageMaker to determine its strengths and capabilities, and the activity and cost of migrating to this service.

As part of the program of works, several pain points in the current state architecture were identified. These areas of concern served as the basis for the design and scope of a proof-of-concept project.

The Solution

Amazon SageMaker features make it easier for developers to automate and scale all steps of the end-to-end machine learning workflow. This is enabled through powerful capabilities such as:

Faster data preparation
A purpose-built repository for prepared data
Workflow automation
Greater transparency of training data (to mitigate bias and explain predictions)
Distributed training capabilities to train large models up to two times faster than would otherwise be possible with today’s machine learning processors
Model monitoring on edge devices.

With this in mind a proof-of-concept (PoC) project was agreed with the goal of demonstrating the value of SageMaker to Travtus and how it could help solve key problems for the business.

The scope of the project was a pre-trained PyTorch text classification model with SageMaker pipelines. The objective was to demonstrate how to execute model training and development workflow using Amazon SageMaker. The method employed was to use a toy problem of finetuning a pre-trained Pytorch text classification model on a data set provided by Travtus. The trained model would be hosted as a serverless service, which was built into Lambda execution functions in Service Accounts. This showed how SageMaker pipelines brings CI/CD best practices into Machine Learning. This feature was demonstrated through a SageMaker Studio Notebook for ease of use. The second part of the project was to alter the pipeline to a pre-trained model – again, trained on a Travtus toy data set. This trained model was hosted as a real-time endpoint, which was then built into Lambda execution functions.

Travtus were keen to explore SageMaker to understand its feasibility, and quantify costs, time and complexity benefits that can be leveraged from migrating to the service. Based on the challenges defined at the start of the project, the following outcomes were delivered:

Executed a model training and deployment workflow using SageMaker
Created a PyTorch pipeline to fine-tune a pre-trained RoBERTa model on custom data
Implemented CI/CD best practices into machine learning by utilising SageMaker pipelines
Utilised advanced SageMaker capabilities
Provided technical documentation and knowledge share

The Benefits

During the proof-of concept project we were able to solve all the agreed problem areas, and delivered the immediate benefits:

Data science teams able to apply CI/CD changes to run machine learning models
Model and data versioning no longer rely on time-consuming manual processes and the risk of introducing human error
SageMaker brings a range of process-related improvements that can benefit machine learning teams:
- New model training processes
- Robust lineage tracking and model reproducibility
- Improved production model monitoring
Automated Model Retrains to perform new model training runs at scheduled intervals in time or once a certain threshold of new labeled data has been reached
Using Amazon EventBridge Scheduler with SageMaker Pipelines
SageMaker provides a scalable and flexible platform that allows Travtus to build high-performing models while still being cost-effective
SageMaker Model Monitor allows users to monitor the deployed real-time endpoints.
Overall, SageMaker can help teams streamline their machine learning workflows, improve model accuracy and reliability, and better manage their production models

Amazon SageMaker delivered the following capabilities

S3 Data Capture

Model Monitor provides a data capture configuration to the real-time endpoint, which can input data to the hosted model and prediction outputs. The data is stored on S3, in JSON Lines format.

Data Drift

Model Monitor allows data scientists to analyse and monitor the data from the hosted model. Monitoring jobs can be scheduled to run custom scripts on the captured data. These jobs can be used to detect data drift in the environment where the model is hosted.

“Our business has an increasingly complex ML stack – we want to evaluate SageMaker as a way to give us more control & save time, with the aim to improve production model monitoring & development speed. This exercise has accelerated our learning as we define our MLOps architecture for the future."

Andrew Day,Co-Founder at Travtus

Why Ubertas Consulting?

Ubertas Consulting is a Cloud consultancy specialising in Amazon Web Services.

As an Advanced Partner, AWS Channel Reseller (Solution Provider Program) and Well-Architected Framework Program Partner, our mission is to assist companies drive innovation and build new capabilities through embracing Cloud-native technologies and modernising with Amazon Web Services (AWS).

Our consultancy services span the breadth of AWS technologies and range from advisory through to full implementation as we assist customers in applying the foundations of Well-Architected to migrate and operate their mission-critical workloads in the Cloud.

Contact us for a free consultation

"*" indicates required fields