Kyrix - democratizing details-on-demand data visualizations

Kyrix facilitates the creation of data visualizations with details-on-demand interactions (e.g. pan and zoom, see the demo gallery¹ above). In visualizations of such, the underlying dataset is often large. To deal with large data, Kyrix is focused on optimizing two goals: 1) usable declarative API library for visualization developers and 2) 500ms response time to user interactions, which is required to enable interactive browsing.

1. This gallery GIF is itself a Kyrix application.

Installation instructions for the impatient

1. Install Docker and Docker Compose (must be v1.17.0 or later). For Mac users, installing Docker Desktop will install Docker Compose automatically. Ubuntu 18.04 users can also simply install with sudo apt update; sudo apt install -y docker.io docker-compose;.
2. Run sudo ./run-kyrix.sh --nba --build in the root directory. You might need to make run-kyrix.sh executable, i.e. sudo chmod +x run-kyrix.sh.
3. Wait a couple minutes, then point your browser at <ip address>:8000 - remember that if you are using a cloud instance you may (probably) need to open your cloud provider's firewall for this port. If that sounds scary, you can create an SSH tunnel from your PC (e.g. Mac) using ssh -N <server ipaddr> -L 8000:<same ipaddr>:8000 to forward your laptop's port 8000 to the server via SSH tunneling.

note that you'll need to wait for a message saying Backend server started... like this:

Serving /project
New project definition coming...
There is diff that requires recomputing indexes. Shutting down server and recomputing...
Precomputing...
Done precomputing!
Completed recomputing indexes. Server restarting...
Backend server started...
*** done! Kyrix ready at: http://<host>:8000/

Getting started by reading a tutorial, and more documentations.

Setup-free big data visualizations

Kyrix is fully dockerized, with a live PostgreSQL database running right after docker startup. Front-end vis libraries often assume data fits in browser memory, and thus scale to only small data. By working with a containerized database, Kyrix scales to much larger data while being free of the hassle of installing and maintaining databases.

Docker works by creating fully isolated virtual machines ("containers") on your computer, making it much easier to achieve correct installations every time (learn more). See docker config details.

Declarative authoring

Kyrix offers two declarative grammars for authoring complex details-on-demand visualizations. Low-level Kyrix grammar is verbose but expressive. Kyrix-S grammar is a high-level and concise grammar designed for zoomable scatterplot visualizations. For example, Kyrix-S turns

{
    data: {  
        db: "nba",  
        query: “SELECT * FROM games"  
    },  
    layout: {  
        x: {  
            field: "home_score",  
            extent: [69, 149]  
        },  
        y: {  
            field: "away_score",  
            extent: [69, 148]  
        },  
        z: {  
            field: "agg_rank",  
            order: "asc"  
        }  
    },  
    marks: {  
        cluster: {  
            mode: "circle"
        },  
        hover: {  
            rankList: {  
                mode: "tabular",  
                fields: ["home_team", "away_team", "home_score", "away_score"],  
                topk: 3  
            },  
            boundary: "convexhull"  
         }  
    },  
    config: {  
        axis: true  
    }  
};

into

Embedding into your favourite web app

Kyrix offers APIs to integrate a kyrix application into a web application. There's no limit on what you can imagine: complex coordinated views, programmatic pan/zoom control, cross filtering...

Telling us about your Kyrix app

We'd like to learn more about your application requirements to make Kyrix better. If you have built a Kyrix app, send us a screenshot on Gitter!

Contributing to Kyrix

We welcome contributions! Please let us know about your change beforehand if it is big. Also, see contributing guidelines.

Kyrix is maintained by a group of researchers at MIT's data system group. The system is not possible without the contribution from these wonderful developers:

Wenbo Tao	Adam Sah	Xinli Hou	Xiaoyu Liu	Yedi Wang	Peter Griggs
Maxime Schoemans	Erica Zhou	Abhishek Bassan	Scarlett Zhang	Amy Zhang	Jim Peraino

Disclaimer

The current v1.0.0-beta release contains research-quality code, and is intended for demo purposes. The code should not be used for large-scale secure serving. Currently, v1.0.0-beta only supports data with up to 10~100 millions of data records. We plan to release a distributed version of Kyrix in the future.

Citing Kyrix

@inproceedings{tao2019kyrix,
  title={Kyrix: Interactive pan/zoom visualizations at scale},
  author={Tao, Wenbo and Liu, Xiaoyu and Wang, Yedi and Battle, Leilani and Demiralp, {\c{C}}a{\u{g}}atay and Chang, Remco and Stonebraker, Michael},
  booktitle={Computer Graphics Forum},
  volume={38},
  number={3},
  pages={529--540},
  year={2019},
  organization={Wiley Online Library}
}