The following diagram shows the overall architecture for Amundsen.
The frontend service serves as web UI portal for users interaction. It is Flask-based web app which representation layer is built with React with Redux, Bootstrap, Webpack, and Babel.
The search service leverages Elasticsearch's search functionality and provides a RESTful API to serve search requests from the frontend service. Currently only table resources are indexed and searchable. The search index is built with the elasticsearch publisher.
The metadata service currently uses a Neo4j proxy to interact with Neo4j graph db and serves frontend service's metadata.
The metadata is represented as a graph model:
The above diagram shows how metadata is modeled in Amundsen.
Amundsen provides a data ingestion library for building the metadata. At Lyft, we build the metadata once a day
using an Airflow DAG(example).