Add support for RISC-V

[petar-jovanovic]

All RISC-V related work should be listed here.

[petar-jovanovic]

Initial support for interpreter for march=rv32g mabi=ilp32d was at one point available, but it is not available anymore.

[MikePall]

First, I have some general questions:

• What do you intend to contribute, when you're done? A pure interpreter port will not be accepted -- you need to eventually submit the JIT-compiler support, too.
• Which RISC-V ISAs, extensions and ABIs do you intend to support? List all combinations.
• Which operating systems do you intend to support?
• Who are the contributor(s) and their affiliations? I can guess, but I'd like it to be spelled out.
• Who is sponsoring this port? The sponsor may remain anonymous in public, but then I still need to know (mail me privately).
• Is the sponsor prepared to sponsor the initial review and integration into the LuaJIT default code base by me?
• Is the sponsor prepared to sponsor the inevitable initial bug fixes and the extra effort for continued maintenance that a new architecture entails?

Another thing to consider: the currently submitted RV32G target does not correspond to any practical combination of available hardware and OS at this point. Embedded CPUs are RV32I (without FPU), whereas Debian/Ubuntu target RV64G only. And
then there's the question of MMUs ...

About the code submitted so far:

• The DynASM RISC-V port has undefined variables, writes to globals, unused remnants of the MIPS port, etc. And, most importantly, integer registers are named x0-x31 in RISC-V.
• The RISC-V disassembler doesn't adhere to the standard Lua coding style or the whitespace rules of the code base.
• The literal translation from vm_misp.dasc needs more refinement. E.g. the majority of slt[u] instructions can be (carefully) folded into the following branch. Register save/restore is definitely broken: FP registers are 64 bit. There are probably more issues in there -- I've only taken a cursory look. This file needs serious review.
• The RISC-V ABI is not the same as the MIPS ABI. Copy & paste of code in lj_ccall.c without word-by-word checking against the ABI document is not a good idea.

[hippi777]

hi there! :)

maybe this can be helpful:
https://github.com/michaeljclark/riscv-disassembler

good luck with it! :)

[petar-jovanovic]

Hi Mike,
These are all good and valid questions and concerns.
I should have said this when I created the issue, but it is not late now anyway. The work done so far has been done without sponsors, but inspired by the interest shown from some potential sponsors. At the same time, some folks (such as attendees at this year’s FOSDEM conference where LuaJIT on RiscV was one of the topics) asked us to share what we got so far, as some of them had interest to work on it too. It seemed OK to us to share the patches, so someone else can join the effort or continue to work on it on its own (for the people we talked to, an initial project that needs to be improved seemed much desireable starting point than starting from the scratch).
The code is not ready for a review (I would have emailed you about this directly otherwise), this is work in progress (current patches written by Djordje Kovacevic and Milos Poletanovic, both engineers at Syrmia), we just shared a w-i-p version that without no doubt needs further fixing and tuning.
What we hoped to reach by creating this issue and sharing the initial code support, is to gather all the people interested in LuaJIT on RISC-V and have something concrete to continue with once there is feedback from everyone.
To sum up, it is not ready for your review and I do agree with you that a new port will require stable and dedicated sponsor/sponsorship to maintain it.

[apivovarov]

libluajit is needed for sysbench! Please, port it to RISC-V.

[petar-jovanovic]

libluajit is needed for sysbench! Please, port it to RISC-V.

@apivovarov Further work on LuaJIT for Risc-V requires sponsorship, so until that issue is sorted out, future progress will be sporadic unfortunatelly.

[archanox]

The code is not ready for a review (I would have emailed you about this directly otherwise), this is work in progress (current patches written by Djordje Kovacevic and Milos Poletanovic, both engineers at Syrmia), we just shared a w-i-p version that without no doubt needs further fixing and tuning.

May I suggest marking this as a draft PR?

[petar-jovanovic]

May I suggest marking this as a draft PR?

No, this is not a draft PR. It was w-i-p that has not been continued due to the lack of interest/sponsorship.

[lazyparser]

May I suggest marking this as a draft PR?

No, this is not a draft PR. It was w-i-p that has not been continued due to the lack of interest/sponsorship.

Hi, I'm Wei Wu, the director for PLCT Lab. Porting LuaJIT to RV64GC is listed on our roadmap[1].

One of the long term goals of PLCT Lab is bringing RISC-V to all important open source projects as tier-1 platform.

I'd like to make either sponsorship or contribution on the porting project. I've searched mailing list[2] and the sponsorship page[3], and found that the discussion of sponsorship were not active. Is there any way that we can support/collaborate[4]?

[1] https://plctlab.github.io/PLCT-Roadmap-2021.html

[2] https://www.freelists.org/post/luajit/Want-to-sponsor

[3] http://luajit.org/sponsors.html

[4] Either sponsorship or community contribution is good to me. I want to avoid duplicate works.

[petar-jovanovic]

Hi Wei Wu, pleased to meet you. I will send you a longer response to your email, but short version is that if there is a sponsorship, there are people who can help. So far, I have seen only a general interest for RISC-V which does not make things happen.
Alternatively, you can try to port the LuaJIT on your own and contribute it. At the moment, I am not aware anyone is doing it, so it will not be a duplicate work. Also, make sure you have answers to the questions Mike has raised earlier.

[MikePall]

@lazyparser I should mention that contributing a new architecture port to LuaJIT is absolutely a non-trivial undertaking. One needs not only very detailed knowledge about the architecture in question, but also about Lua, C, compilers and the LuaJIT codebase.

Based on previous occurences, someone with money at hand, but only a marginal understanding of any single of these requirements will not successfully complete this task. Also, I'm very, very, very stringent about the quality of the code that I accept.

[lazyparser]

Hi @MikePall

@lazyparser I should mention that contributing a new architecture port to LuaJIT is absolutely a non-trivial undertaking. One needs not only very detailed knowledge about the architecture in question, but also about Lua, C, compilers and the LuaJIT codebase.

I totally agree. Implementing or porting a production level JIT compiler is a serious project, especially LuaJIT. Although the PLCT lab has ported several compilers/VMs to RISC-V, we are very cautious about starting the project for porting LuaJIT, for the internal complexity of LuaJIT.

Based on previous occurences, someone with money at hand, but only a marginal understanding of any single of these requirements will not successfully complete this task. Also, I'm very, very, very stringent about the quality of the code that I accept.

I really respect and appreciate the strict criteria in the LuaJIT project, which makes the project successful with a high quality codebase. This is also one of the reasons that I'm looking for experienced developers who has good understanding of the LuaJIT codebase. (the other reason is trying to avoid duplication work in open source)

After reading again your comments, I'm afraid that my previous comments may conveyed information that I did not mean to . I have never thought of spending money to get things done. (English is not my native language. Please bear with my translated English.)

Let me try to express my idea/purpose in different words: I was a compiler engineer and co-founded PLCT Lab, which is an engineering team working on compilers, simulators and language virtual machines. The big goal is to have a RISC-V laptop by the end of 2022, with Debian/Ubuntu or Fedora running on it. One of the critical paths is porting language VMs with JIT compilers. The PLCT Lab is working on JavaScript VM (V8: upstreamed; Spidermonkey: started), Java VM (OpenJDK RV32GC: WIP), etc. I humbly understand how hard of porting & optimizing the virtual machines or JIT compilers. All the important VMs and compilers are expected to be ported to RISC-V platform, and I am working really hard on it. There are 20+ engineers in PLCT Lab working on V8, OpenJDK, Spidermonkey, and others. As mentioned earlier, LuaJIT is an important project, depended by lots of softwares that are used by RISC-V users. I am very happy to find that there were already a few previous effort for porting LuaJIT to RISC-V. I want to accelerate the porting process, while keeping the code quality at the high level.

[lazyparser]

Hi Wei Wu, pleased to meet you. I will send you a longer response to your email, but short version is that if there is a sponsorship, there are people who can help. So far, I have seen only a general interest for RISC-V which does not make things happen.
Alternatively, you can try to port the LuaJIT on your own and contribute it. At the moment, I am not aware anyone is doing it, so it will not be a duplicate work. Also, make sure you have answers to the questions Mike has raised earlier.

Hi @petar-jovanovic,

Thanks very much for this. I'll reply the questions in Mike's previous comment carefully.

[kirana1996]

Hi @lazyparser,
We want lujit support for riscv64. Is there any update on porting or can you help us to port for riscv64.

[lazyparser]

Hi @lazyparser, We want lujit support for riscv64. Is there any update on porting or can you help us to port for riscv64.

Still in the planning stage and still on the roadmap of the PLCT Lab. We may start to port LuaJIT after the spidermonkey porting get done.

[fedya]

any news here?

[lazyparser]

any news here?

The PLCT Lab had forked the LuaJIT repo[1] and started to porting to RISC-V this month.

Currently no workable version available. The best guess is that interpreter part will be done in 8-12 weeks, and the JIT part may need more 12-18 weeks.

Based on the requirements in LuaJIT community, the PLCT Lab is going to upstream the ports after all the works (interpreter, JIT, testsuite) get done.

[1] https://github.com/plctlab/LuaJIT

[infiWang]

I'm currently working with PLCT Lab under @lazyparser on initial bring-up work on the RISC-V platform. My current progress can be found here.

At the time of speaking, we have DynASM RV32G/RV64G support functioning, plus some preliminary work on the interpreter backend. Here's a screenshot of a DynASM-based BF JIT compiler vs. a naive BF interpreter on a HiFive Unmatched executing the famous "mandelbrot" program.

BF Comparison

I'm looking forward to a functional interpreter in ~8 weeks, following which we'll continue working on the JIT section.

[infiWang]

I'm looking forward to a functional interpreter in ~8 weeks, following which we'll continue working on the JIT section.

It's been a little over nine weeks since the last progress update. The interpreter appears to be mostly functioning as of right now (301/304 tests, LuaJIT/LuaJIT-test-cleanup, without 5.2 support), with minor exceptions like DynASM reporting immediate overflow during building and occasional segfaults. Therefore, cleanups and further optimization might be wanted still.

I'm now concentrating on the FFI library and the SSA IR assembler. FFI without callback support should be functional soon, whereas JIT is anticipated to have a working prototype early next year.

[davidlt]

@infiWang are there any news about your progress?

[infiWang]

Are there any news about your progress?

The interpreter is generally in good shape, plus FFI w/o callback should function now. Please try it out and let me know how it goes!

JIT is mainly still in the works. My patch is experiencing problems with constant (re)materialization during register allocation. I'm afraid trace patching/dispatching is also flawed in some way.

Update: FFI issue has been fixed by riscv(interp): reallocate TMP registers to comply with ABI . It's safe to say that we have finished the interpreter and interpreter FFI porting now.

[infiWang]

JIT seems to be working correctly now. Debug information and potential UB's sanitization should be the only things left.

[dumblob]

@infiWang first of all that is awesome news, thanks a lot for all this hard work! I worked on compilers in my past and know how hard it is.

I think it will need some heavy testing (I did not see such comprehensive tests in your repo but I suppose this will be published later - esp. some Lua-heavy existing SW which has its own large test suite would be interesting to test with LuaJIT).

I have just randomly looked around your repo and it seems the license headers in files are not updated.

I think before distributions can start shipping this gem, this should be prepared to not cause any confusion or license issues.

Kudos once again!