Home

What is Scattersphere?

Scattersphere is a lightweight job coordination API designed to run a simple DAG of Tasks. It has been inspired by many projects on GitHub: Orchestra, Chronos, Cats IO, Monix Task, and others - even Mesos and Spark.

• It is designed to be self-contained, extensible, and fast.
• It is Open Source, licensed under the Apache License 2.0.
• It's written in Scala. I like Scala.

Why Scattersphere?

Scattersphere came about in the early 2010s while working at an Internet start-up that ran processing software on an Intel Modular Server cluster. A mini-framework came from that project, allowing us to run scripts and tasks from the command line, but with very little control on monitoring tasks or debugging. It was more about "fire and forget" with some simple error handling.

As time went on, the project lay dormant.

But there was a pattern seen at almost every job: a need to run tasks in succession, but with the ability to debug and easily manage the code. Spark was overkill, as these were simple tasks: Java Runnables.

So, as the demand for this type of framework continues to grow and evolve, the Scattersphere project has finally come about as a full-fledged project.

The name came from the idea that jobs and tasks were "scattered" all over the servers, and possibly all over the globe. A globe is a spherical shape.

Scattersphere. A portmonteau.

The name just stuck.

QUICK START

Where do we go from here?

A good place to start is in the logical order of how a job is created and executed in Scattersphere.

First, you create a Task. Then, you add that Task to a Job. Once your Job has been defined, you pass it to a JobExecutor, queue() it, then run() it.