Alternative I/O backend using io_uring on Linux

[JohannesLichtenberger]

in a package named org.sirix.io.iouring for instance and implement a Reader, Writer and Storage analogous to the other packages in org.sirix.io.

[JohannesLichtenberger]

Maybe using https://github.com/ikorennoy/jasyncfio until Java's standard I/O supports io_uring the additional backend might be really helpful :-)

[sourabhkashyap95]

Hey @JohannesLichtenberger, planning to work on this issue.
Do we have any other lib implemented on io_uring? Also, like to make small changes related to a typo, but I think I don't have access to push changes.

[JohannesLichtenberger]

I have recently started an issue over here how to proceed as it's not entirely clear to me: ikorennoy/jasyncfio#66

First of all I think we should switch to Java 19 and use virtual threads.

[JohannesLichtenberger]

@sourabhkashyap95 I've added a blocking version which has to be adapted (and even the blocking version has at least one bug and is currently not working): https://github.com/sirixdb/sirix/tree/master/bundles/sirix-core/src/main/java/org/sirix/io/iouring

In order to debug the tests you can simply change the STORAGE_TYPE in ResourceConfiguration to StorageType.IO_URING

[sourabhkashyap95]

@JohannesLichtenberger , will go through the thread once and will check with you in case of any doubts.

[JohannesLichtenberger]

@sourabhkashyap95 maybe you can debug reading/writing of page fragments (the variable length of a page -- for instance the UberPage currently is 27, but when read it's a bigger integer). I thought about byte order issues, as currently it's written in big endian, but read in little median byte order on my machine. But even if we're not using the native byte order for the ByteBuffers the bug persists.

[sourabhkashyap95]

@JohannesLichtenberger Currently I am trying to build the project but its failing, I think with this message /usr/local/bin/docker-compose: line 1: Not: command not found
It seems like docker-compose issue, currently, I am on Apple M1 pro chip, osx-x86_64 architecture.

[JohannesLichtenberger]

You don't need Docker Compose. You can fork the repo, clone and open it in your IDE as a gradle project. Then you might make the aforementioned change and set breakpoints in the IOUringReader and IOUringWriter for instance when writing and reading the UberPage.

[JohannesLichtenberger]

@sourabhkashyap95 hope it helps

[sourabhkashyap95]

@JohannesLichtenberger trying to run the tests changing the storage type but getting can't load native library as i have apple M1 machine.

[JohannesLichtenberger]

Oh, I guess you'd need a recent Linux derivate. I've filed an issue with the library because of offset based writes: ikorennoy/jasyncfio#68 I think there's a bug in the library

[JohannesLichtenberger]

On my new laptop with a PCIe 4.0 SSD sadly the io_uring layer for some reason is also consistently slower than the epoll based FileChannel implementation. I'd thought that even if it's not the most ideal way with more or less mostly synchronous I/O it should be faster in theory.

[JohannesLichtenberger]

We should check if we can tune our current implementation, but in all my latency checks it was slower. I think in part it might be caused by the library and/or false parameters used. The event loop in the implementation seems to have some overhead.