Roadmap/Progress Update?

[beegmon]

I have been playing around with EventQL for a couple of months now and so far it has been perfect for my use case!

However, I am concerned by the number of open issues/pull requests sitting out there for months (sometimes over a year) without any activity or responses. It also looks like the last commit was early May as well.

I understand things can move slowly in software, especially with a project such as this. I am certainly not complaining, but before I go and build further on top of EventQL I was wondering if there is a roadmap/timeline in place for 0.5, 0.6, and beyond. Not saying this project is dead at all, but is certainly feels like more of a ghost town around here than I would like for a project that I wish to use/become pretty dependent upon.

Any update on a roadmap or progress toward 0.6 (what is noted in the docs as the possible "stable" production version) would be great so I can better determine the longer term viability of using EventQL in my project.

Thanks!

[kmatt]

Additionally http://eventql.io/ currently resolves to https://hermes.thinleen.fr/ with a 403, and may have been so since September.

[eilifm]

Any update on the status of development?

[asmuth]

First of all, sorry for taking so long to update on the project status. It's something I've had on my list for a while, but I never got around to finishing up my half-written blog post.

As is probably obvious from the repo, the company behind EventQL didn't work out. Between my late realization that it would not be possible to handle both sales and development for a product as complex as EventQL with the limited amount of resources we had, general fundraising troubles, a change in my family situation, ClickHouse releasing their product weeks before us and the opportunity to join another very interesting project, I decided it did not (commercially) make sense to continue working full-time on EventQL.

The state of the code at that point (and ever since) is what I would call a "minimum viable product". On one hand, pretty much all the things promised in the documentation and feature list do actually work and were used in a production environment for some time. On the other hand, the largest cluster we ever ran was ~20 servers and around 150TB of data. And even that required a bunch of work by me to keep it running stably, fixing things up here and there as they broke. So, all in all not exactly huge by the goals we set out for ourselves.

Since we shut off our servers in 2017 I am not aware of anyone that is running a significant production deployment and I guess sime bitrot has started to set in. So I think the status of the project today can be described as a "research project" at best. I will make sure to update the readme accordingly.

Now, I hope all of that didn't sound too negative. EventQL is definitely not vaporware -- it is a working analytical database as promised on the box, albeit very immature, and we did have a few really nice customer dashboards running on it, as well as customers using the SQL interface directly. After reaching that goal and after all the time, sweat and love that went into the project, it was always clear to me that I would eventually return to working on it. And in the past months I finally found that I had some spare cycles for open source again, so I've been thinking about the best way to salvage...

There are parts of EventQL that I still really like and that, already today, work well enough for a beta-level project. I reckon these can be stabilized, maintained and extended as a volunteer-run project. So I think what we should do is to strip the codebase down so that only the nice(st) parts are left, re-license it to a permissive license and then see where we go from there.

Concretely, here is my current (tentative) plan of next steps:

• Remove the distributed (cluster-wide) automatic partitioning. This feature was always a bit of a pain and in retrospect I think that it wasn't a great design; during loading of data and periods of heavy writes, the recursive partition splits would lead to crazy amounts of network traffic between nodes, always saturating the network links we had.

  Testing and tuning this feature was one of the trickiest and most annoying parts of developing EventQL and definitely requires a serious test cluster. Also, automatic partitioning is in the first place only useful for very large clusters with frequent addition and removal of nodes and I feel that the "very large cluster" target audience wants something that's more mature than EventQL, or at least, is maintained by a larger group of people, anyway.

  After cluster-wide automatic partitioning is removed, you will of course still be able to run a replicated set of EventQL nodes in a "multi-master" configuration that will automatically sync itself. And you will still get "exactly once" (deduplicated) inserts like you do now. Only when your data set is too large to keep a full copy of it on every EventQL node, you would have to explicitly configure how the data is to be sharded instead of relying on the current automated "partition subdivision" mechanism.

• Once the cluster-wide partitioning is gone, we can remove a lot of hairy metadata coordination code and the Zookeeper dependency, greatly improving the experience for "casual" users I think.

• There are a number of nice admin and query web interfaces that we built for the commercial offering. Maybe we can also include those in the open source distribution now.

• Probably the most important change is that what code is left after the steps above would then be re-licensed to Apache 2.0.

I hope to get around to start acting on the larger plan in the next weeks, but I'll try to do the readme
and website updates as quickly as possible; so hopefully soon there'll be some activity here again. All the best!

[KrishnaPG]

If the "storage" and "query engine" parts can be separated out, there is scope for the "query engine" part to grow as there is huge gap in that area: storage agnostic distributed query engines.

Current distributed query engines, such as Drill, Presto are primarily Java based and even though they are storage agnostic creating a storage adapter for them requires java tight integration, not so friendly for distributed systems developers (C/C++, NodeJs where most of the embedded + web development happens).

How easy it is to separate out the "query engine" part to make it work on any data input?

[derN3rd]

Hey @asmuth, will it be possible to get one of the admin and query web interfaces? We plan to implement eventql for a large experiment at our company and I just though it would be cool, to have a easy panel where my colleagues could try out some queries.

Thanks in advance

[brucemen711]

eventql is pretty good, is any progress on this project ?

[intel352]

It looks like this project is pretty much dead. The creator hasn't responded on this thread since 2019, the eventql website mentions 0.4.3 release but download page is for 0.4.0-rc0. The historical downloads page is non-functional. Build status in github shows build is broken.

Real shame, was hoping this was a viable option for our new projects...

[derN3rd]

Real shame, was hoping this was a viable option for our new projects...

Maybe ClickHouse is something for you. We are using it for a couple of months now and are in love with it