Language interoperability

[kesavkolla]

Is there any plan to provide other programming language bindings for velox? I am looking for rust bindings so that I can use velox in rust.

[pedroerp]

@kgpai has started looking into Python bindings and there were discussions in the past with the Intel team about generalizing the Java bindings from Gluten, but we haven't discussed about Rust yet (though it would be nice to support).

Is it something you would be interested in contributing?

[kesavkolla]

@pedroerp If there is some architectural guidence on how to go about providing language bindings then I can do rust bindings. I think currently the language bindings are happening outside the velox and I don't know how are they implementing. If there is some structure from velox team then other language bindings can flurish.

[kgpai]

@kesavkolla Once we have the python bindings out ; you can probably use that as a reference for rust . We are still working on the api's and trying to determine the public / private interface , so its hard to provide guidance on bindings just yet.

[kesavkolla]

@kgpai I understand that. In general we should be using something similar to Arrow C interface. This will give seemless interactions of data between programming language to native C.

[kesavkolla]

Is it possible to develop WASI layer for velox? That way we can integrate velox into any programming language. Today we have to work with C/C++ in native programming language integrations. If there is a WASI compatible layer then people can use in any programming language. I hope runtimes like wasmer or wasmtime can work with very low overhead.

[kesavkolla]

Is there any movement on this front? Is there any easy path that developers can use velox from other programming languages?

[kgpai]

Hi Kesav, Firstly, Apologies for the delayed response. Atm I am not aware of an plans to add WASI layer for Velox. You are welcome to contribute this by creating an issue and detailing the design etc. We have started with python bindings for Velox and have created PyVelox with rudimentary bindings (atm only expression eval and simple flat vectors) though we plan to add a lot more to this in the immediate short term.
So to answer your question atm only Python is officially supported , also the support is relatively rudimentary, but we are working on it and will add lot more stuff (encodings, complex vectors, arrow/velox conversion, operators etc..) .
Again we would welcome any contribution to better support this across other languages.

[philippemnoel]

Hi there! I’m interested in using Velox in Rust as well and will be looking on implementing bindings. Would love to work on them alongside the core Velox team and contribute them directly to Velox if that would be the wish of the team.

Is this something others would be willing to work on alongside myself and my team? Any tips other than inspire ourselves from PyVelox?

[philippemnoel]

Hi @kgpai!

Thank you, that sounds great. We can get started and show you our work as we go. I think the submodule+separate repo approach makes sense for now, if we do use Velox we'd be happy to maintain the bindings, I suspect. I would need to confirm on that, though, and it would depend on how heavy a usage we do. Also note that we probably wouldn't make bindings for everything, just what we need, as a start. Maybe that's where others can jump in as needed for their use cases.

Re: Slack channel. I messaged velox@meta.com as per the README but never heard back. Would you be willing to invite me to the Slack, please? philippemnoel@gmail.com is my email. I'll be glad to attend the OS meetings.

We're considering bundling Velox into a Postgres extension built in Rust. This is where the interest is for bindings :)

Lmk!

[pedroerp]

Slack invite sent

[philippemnoel]

Thank you!

[ozgrakkurt]

Hey!! was there any progress on this? I would like to help also

[pedroerp]

I'm not aware of any progress on this direction, but would indeed be nice to provide Rust bindings.