FAQ

Design

Why Python?

• It's a very widely used and known language
• It is very easy to prototype with because of the dynamic
• Flexibility: is a dynamically type language, which makes writing small scripts and prototyping very fast and convenient but it provides optional type checking we consider essential for a big system or support library
• Very good support for data science / machine learning

Why an actor and pub/sub architecture?

We start with some assumptions:

• We want something that is event driven
• We want it to be future proof for distribution
• We want to use Python

The first assumption implies that calculations have to work concurrently, otherwise obviously one calculation waiting for an event would block everything.

So the cheapest thing we can get for concurrency is tasks (AKA fibers, green threads).

If we want communication between those tasks we need to share data somehow. Using channels might on a first glimpse look like some overhead, but that comes with a clear design advantage, because suddenly you can break down the design of the whole system in modules that communicate with each other and we can draw modules with arrows between them. This is a more modern, scalable and flexible way to design a system rather object oriented modeling, UML stuff etc. It is basically an embedded micro-services architecture.

So that leaves us with Python, because that's it what makes us think about possible performance problems here. We gained some experience using channels for inter-task communication with Rust and those are damn fast. So there should be no problem here.

At the end, if and only if we are impacting performance by the design choices we take, then there is plenty of low hanging fruit to improve that situation, e.g.:

• Use alternatives to the (default) CPhython interpreter, e.g. cython or pypy
• Rewrite critical code paths in a system language (like Rust) and expose python bindings