Node microservices demo

This is a small demo of fine grained services and their security properties. Code is written in JavaScript/Node.js. Similar demos with Python and R are planned.

Wait why? A brief detour into policy...

TBS has been pushing hard to modernize IT practices. It's Enterprise Architecture Framework explains:

Application architecture practices must evolve significantly for the successful implementation of the GC Enterprise Ecosystem Target Architecture. Transitioning from legacy systems based on monolithic architectures to architectures that oriented around business services and based on re‑useable components implementing business capabilities, is a major shift.

This mandatory policy is aggressively modern and directs departments to

• "design systems as highly modular and loosely coupled services"
• "use distributed architectures"
• "support zero-downtime deployments"
• "expose services, including existing ones, through APIs"
• "ensure automated testing occurs"
• "design for cloud mobility"

The shift to highly available and evolvable distributed systems is strategically important to support modern service delivery, but it's easy to overlook that building and supporting such systems is very different from what the government is used to. Distributed systems are complex and hard.

What's missing to help departments adapt to this "major shift" are working examples of how to build this type of architecture. That is the aim of this project.

This demo

A distributed system is made of many small (micro even!) parts (services) that collaborate to deliver some funtionality, deployed across a series of machines. The basic idea behind this demo is to show how this way of building and deploying applications allows for more granular security decisions and higher levels of observability, as can been seen when we visualize traffic flowing through these services with Kiali.

Visualizations like the above are a powerful way to make security visible: you can see that the system has distinct components (with their own security settings), and communications between them is encrypted with mTLS.

This is a zero trust microcosm, and systems built like this have security properties and compliance implications that security teams need to be learning about.

As is common with microservices projects, this repository is organized in the monorepo style keeping services in it's own folder (code that changes together stays together).

All code embeds opinions, and this repo and the code within are no exception. There are other options worth exploring, but the patterns and technologies here come from balancing a lot of tradeoffs specific to service delivery in the Government of Canada.

The code and readme files make special note of the security properties that emerge from this style of architecture.

The API Service

Since APIs are mandatory, the API forms a core part of this service. Living in the API folder, this is a GraphQL API that reads from/writes to tables in the database, and produces only JSON as an output. It's narrow scope allows for tightly scoped database permissions, and makes security profiling of "normal" behaviour tractable.

See the API documentation for more details.

The Migrations service

This lives in the migrations folder and exists to move the database schema from one known state to the next. The requirement to "support zero-downtime deployments" means teams will need to look carefully at how to do this with their choosen database and web framework (and possibly choose ones that make this easy to do).

The powerful permissions needed to alter a database schema (ie create/drop tables) are required only once at startup, and it's the job of the migration service to create/verify the schema needed and then exit. This design leverages the plumbing provided by modern container orchestrators like ECS or Kubernetes to achieve "least privilege".

See the Migrations documentation for more details.

The UI service

Found in the ui folder, this service queries the API and is focused on safely encoding the data received into accessible HTML using React.

See the UI documentation for more details.

Running it

Distributed architectures require an orchestrator, and Kubernetes is a vendor neutral open source stadard. You can run a local version of it using Minikube. Minikube requires a lot of resources... so throw everything you can at it.

For reference, this is what was used to develop this.

$ minikube config view
- cpus: 16
- memory: 40G

You'll also need Kustomize and istioctl installed. Once you're all set run the following commands:

minikube start
make credentials
make demo

Then launch the Kiali dashboard:

istioctl dashboard kiali

Refresh the application several times to see traffic flow on the dashboard. The url can be found with:

minikube service istio-ingressgateway -n istio-system --url