Sync product updates in DDB with dependent services

[james-jory]

DDB is our source of truth for product and category data and the Products service is the interface for accessing and updating this data across the application. There are other service integrations that also work off product data including Elasticsearch for product search and Personalize for product recommendations.

The DDB products and category tables are seeded with data from the products.yaml and categories.yaml files at deployment time. For Elasticsearch, there is deployment time option to automatically index products from products.yaml or the user can choose to go step through the Search workshop. For Personalize, there is also a deployment time option to train models based on product, user, and (synthetically generated) interaction data.

This currently works well since product information is essentially read-only in the application. However, with the addition of product add, update, and delete operations, the limitations of this approach start to become problems. For example, when a new product is added, it should be automatically indexed in ES and added to the Items dataset in Personalize for cold start recommendations.

This could be solved by adding logic to the Products service to make calls to the Search and Recommendations service when products are added, updated, or deleted but this starts to more tightly couple these services. An alternative approach that is more scalable, flexible, and decoupled is to use DDB streams and Lambda functions to keep dependent services up to date with changes to products.

For Personalize, this would unlock adding some key use-cases to the project including filtering out of stock items and recommendations for cold/new items.

[BastLeblanc]

I'm highly in favor of DDB streams and Lambda functions.

We should decouple components as much as possible (it's a Retail Demo, but also a way to demonstrate architecture best practices)