What is missing? How to help?

[italomaia]

Hello. It would be nice some documentation explaining what is still missing for this project or how to help. The project's idea is quite nice.

[jirutka]

+1, this library is amazing, I’d like to see it moving forward. @rtsisyk, could you please respond?

[rtsisyk]

Hi guys,

Sorry for the silence for the long time. The primary problem is that I've completely lost my interest in LuaJIT optimizations. I spent several years working with LuaJIT in Tarantool. LuaJIT 2.x was the integral part of our project from the early beginning. I’m definitely sure that we have created the best database and application server for the LuaJIT.

Now I have to say that I don’t buy into LuaJIT anymore. My favorite joke about LuaJIT is that it can be either Lua or JIT, not both at the same time. JIT has never worked out of the box. You need to rewrite all your nice Lua code to deal with NYI, Lua/C functions, suddenly aborted JIT traces and so on. This process is always complicated, time-consuming and unpredictable. The resulting code is unreadable, obfuscated and hard to maintain. Moreover, you have a chance to end with JIT-friendly code which is surprisingly slower that the original interpreted version.

I realized that in terms of human resources it is cheaper rewrite all my performance-critical code using compiled language, like C/C++/Rust/Go/Swift rather than get stable JIT traces. In other words, JIT optimizations in LuaJIT simply doesn’t pay off in my business.

Anyway, LuaJIT is the best interpreter on the market. Just believe me. Mike Pall has done the very great job. Tracing JIT is the cutting edge technology and LuaJIT and luafun in particular are still very useful for the some cases. JIT just requires some more efforts I can't afford.

My offer is pretty simple. I'd like to grant the full permissions to somebody who has a vision how to move this project forward and can propose a new roadmap for LuaFun. LuaFun is probably the most popular Lua and LuaJIT module on GitHub. A lot of people have interest in this project. I just can't spent enough time on LuaJIT-related stuff anymore. I want LuaFun to become more community-driven rather than my personal one-man project. I'm on GitHub every day, let's have a talk.

[rtsisyk]

The first step is done - I moved this repository to https://github.com/luafun/luafun/.

[lukego]

Brutal honesty @rtsisyk :-)

[rtsisyk]

Brutal honesty @rtsisyk :-)

Sad, but true :( I'm tired of all this JIT magic. For example, a week good I had to disable JIT in one of Tarantool test cases because LuaJIT degraded from 5-10 ms to (!) 500 ms without any changes in the code itself!

Since Mike has resigned, all we need to join efforts to maintain LuaJIT. Do you plan to fork LuaJIT for Snabb? Tarantool already uses a fork.

[lukego]

I am excited about tracing JIT. I want to master it, document it, and build tools to make it more transparent. I am working on this very actively at the moment as part of a broader performance benchmarking/optimization effort in Snabb.

I understand your frustration though. On the one hand JIT behavior is not well documented, not widely understood, not easy to understand with the tools available. On the other hand you cannot use it effectively without understanding it. I can see this would be especially frustrating for people who are working with PUC Lua rather than targeting LuaJIT exclusively (as we do.)

Having said that, I think LuaJIT complexity is on the same level as other technologies that we are all using successfully like CPUs, GPUs, kernels, databases, etc. Just needs to be better understood by more people.

Snabb also has a LuaJIT fork. This has only minimal differences. I am interested in maintaining a branch that has extended profiling and low-level programming (e.g. intrinsics/dynasm) support. I think it will be great for all of us maintaining LuaJIT branches to sync up and establish suitable merging relationships to propagate changes. See also LuaJIT/LuaJIT#219.