Real-Time Personalization APIs

Machine learning recommenders are an effective tool to enhance the user experience in client applications through personalized product and content recommendations. However, putting recommendations to work in client applications and keeping these recommenders updated in real-time with in-app user behavior often requires building and maintaining complicated middleware components between your client applications and recommenders. This project provides the components necessary for deploying low-latency real-time APIs that sit between your client applications and recommenders. Key features include flexible/smart response caching, decorating recommender responses with the item metadata needed to render them in your applications, seamless A/B testing across your recommenders, API authentication, automatic user context, user event collection and fan out, and more.

Although this project is tightly integrated with Amazon Personalize, an AI service from AWS that allows you to easily create state-of-the-art ML recommenders based on your data, it also supports experimental backend integrations with personalization providers built with Amazon SageMaker, AWS Lambda, or HTTP endpoints. Key features include:

• Highly configurable multi-layer caching that reduces both end-to-end latency and the number of calls made to API endpoints and origin recommenders
• Multiple opinionted API deployment and authentication approaches including OAuth2, API keys, and no authentication
• Item metadata needed to render recommendations in client applications is automatically included in API responses (i.e., response decoration)
• Fan-out of incremental/streamed events to Amazon Personalize event trackers, Amazon Kinesis Data Streams, and Amazon Kinesis Data Firehose
• Automatically derive context from intrinsic data in each request to enhance the relevancy of recommendations, enrich incremental events, and resolve dynamic filter values
• Seamlessly perform A/B tests across different recommender implementations
• Dynamic and flexible configuration that makes it easy to safely deploy, monitor, and rollback configuration changes
• Automatic generation of application configuration given existing Amazon Personalize resources (optional)
• Automatic generation of OpenAPI/Swagger specification based on application configuration
• Deployment of Swagger web interface that allows for easy inspection and testing of API endpoints (optional)

Architecture

This solution provides opinionated architectures for several common deployment scenarios. The following diagram illustrates the components deployed for an API architecture that utilizes OAuth2 for authentication, CloudFront for regional shared caches, and Amazon API Gateway for API management. This is just one of the supported configurations. The API origin function is the heart of this solution.

See the installation instructions below for details on additional deployment scenarios with alternative authenication and caching approaches.

Caching

Depending on the deployment configuration selected, caching is automatically enabled at multiple points/layers in the request path.

• Amazon CloudFront is deployed as a CDN (content delivery network) in front of the APIs to provide a distributed shared cache and to reduce overall network latency.
• Control the maximum age that responses are cached at the individual recommender and request levels.
• Control whether responses can be cached in shared (CloudFront or API Gateway) and/or private (browser/device) caches.
• Configuration inheritance: set a single cache policy that applies to all recommenders with the option to override cache policies at the namespace and recommender levels.
• Automatic support for HTTP cache validation through ETag response headers and If-None-Match request header inspection.
• The result is reduced latency and fewer calls to your origin recommenders.

See the Caching documentation for details.

Inference item metadata

Machine learning recommenders like Amazon Personalize often provide only a list of recommended item IDs in their inference responses. However, client applications typically need rich metadata for each item (e.g., name, description, price, category, genre, image URL, media URL, detail URL, etc) to be able to render recommended items in their UIs. Amazon Personalize does support returning item metadata with responses for campaigns and recommenders. However, this capability is limited to returning up to 10 columns per item and up to a maximum of 50 items per response (as opposed to 500 items without item metadata). There is also an additional cost for enabling item metadata in Amazon Personalize responses. If these limits are not enough to suppport your use case, the Personalization APIs project provides an item metadata sidecar implementation that stores item metadata either in a local database right where recommendations are retrieved in the API layer (for small and modest sized item catalogs) or in Amazon DynamoDB (for very large item catalogs). Both options provide very low-latency retrieval of item metadata that is automatically injected into recommender responses before returning them to your applications. This capability allows you to focus more on displaying recommendations in your client applications rather than building undifferentiating middleware to merge item catalog information into your recommendation responses.

See the Inference item Metadata documentation for details.

Fan-out incremental/streamed events

The Personalization API's /events endpoint provides a distribution point to send events to any combination of Amazon Personalize event trackers, Amazon Kinesis Data Streams streams, and Amazon Kinesis Data Firehose delivery streams. Or you can have the API send events just to a Kinesis data stream and then configure your own consumers downstream from Kinesis. This allows you to persist clickstream data in datastores such as S3 (data lake), OpenSearch, or Redshift and then build dashboards, perform deeper analysis, and more.

See the API entry points and Configuration documentation for details.

Automatic context

The current context of the end-user--their current device, current location, seasonality, and so on--can be extremely valuable signal to recommenders like Amazon Personalize. Contextual attributes allow recommendations to be personalized not only to the user's interest but also that are relevant to a moment in time. The Personalization APIs solution is able to derive user context based on intrinsic data available in the request and seamlessly pass that context to recommenders at inference and when new events are ingested. This relieves you from having to build logic into your client and server applications to derive and transmit contextual attributes.

See the Automatic context and Configuration documentation for details.

A/B testing

Deploying a recommender system is rarely a one-and-done effort. It's an iterative process. Comparing the effectiveness of one recommender against another and how they impact business objectives requires online testing. The most common online testing strategy is A/B testing where separate groups of end-users are presented different variations of recommendations and their behavior is measured to determine which variation produces the best results.

The Personalization APIs solution is integrated with Amazon CloudWatch Evidently, a new capability of CloudWatch that allows you to manage, deploy, and monitor A/B tests in your applications, which enables you to easily run your own A/B tests across different recommender implementations.

See the Experimentation documentation for details.

Configuration

AWS AppConfig is used to manage the configuration for this solution. With AppConfig you can easily create, manage, and deploy API configuration changes as well as set deployment strategies to control roll out, bake time, and rollback changes.

At deployment time, you can have the project automatically generate a configuration based the recommenders, campaigns, and event trackers in one or more Amazon Personalize dataset groups in your AWS environment. This is a great way to save a lot of time setting up the foundation of your configuration. You can then take the generated base configuration and customize it further to suit your needs.

See the Configuration documentation for details.

Installing and configuring the solution

IMPORTANT NOTE: Deploying this solution in your AWS account will create and consume AWS resources, which will cost money. Therefore, if after installing this solution you choose not to use it as part of your recommender API strategy, be sure to follow the Uninstall instructions below to avoid ongoing charges and to clean up all data.

There are two options for installing this solution. The easiest and most convenient installation option is using CloudFormation directly. If your application and recommenders are hosted in one of the AWS regions listed in Option 1 below, click the "Launch Stack" button for the appropriate region. Otherwise, to install the solution in another AWS region, use Option 2.

Option 1: Install using CloudFormation (single-click, easiest)

To support easy single-click deployments, this solution has been packaged and staged in the following regions. If your recommenders, such as Amazon Personalize resources, are hosted in one of these regions, it is recommended to use the appropriate "Launch Stack" button below. If your recommenders are in a different region, use Option 2.

Region name	Region code	Launch
US East (N. Virginia)	us-east-1
US East (Ohio)	us-east-2
US West (Oregon)	us-west-2
Europe (Ireland)	eu-west-1
Asia Pacific (Sydney)	ap-southeast-2

Option 2: Install using Serverless Application Model (manual installation)

To manually install this solution or to install this solution into a region not listed under Option 1 above, perform the following steps to install using the AWS Serverless Application Model (SAM) tool.

Clone the solution respository

git clone git@github.com:aws-samples/personalization-apis.git

Install the solution using SAM

To install using the SAM CLI, you first need the following tools locally installed.

• SAM CLI - Install the SAM CLI
• Python 3 installed
• Docker - Install Docker community edition

Then ensure you are logged in to public.ecr.aws in Docker so SAM can download the Docker build images by running the following command in your shell.

```bash aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin