Split out dependencies from wpilib-sys

[auscompgeek]

wpilib-sys is rather large. The majority of it is just libraries built for athena, most of which the crate doesn't even provide bindings for. The dependencies won't change mid-season, and the same license doesn't apply to all the binaries.

It'd be nice to eventually have bindings for cscore. However, it depends on wpiutil. It'd be rather silly to end up having to download multiple copies of wpiutil.

I've started by creating an frc-netcomm-sys crate here: https://gitlab.com/auscompgeek/frc-netcomm-sys

[Lytigas]

I'm not sure I agree that 1.76 MB is large (if binary size is what you mean) for something downloaded once per project and then cached. Besides, the linker should remove dead symbols when compiling the final binary.

It is probably true that wpilib-sys ships too many shared dependencies on its own. I started out just trying to expose HAL calls, so that mean't writing bindings for them and shipping the necessary libraries to make things work.

I don't think I understand the interplay of these libraries as well as you, but let me lay out what I think your solution is so we're on the same page: split each library or group of libraries that we currently ship (FRC_NetworkCommunication, NiFpga(Lv), niriodevenum, ...), create a crate that links that library, for the others to depend on and possibly expose rust bindings to that library.

I'm on board with the idea.

For most libraries, it seems like the only code that ought to be calling them is the HAL, which we can already ship in binary form. Creating bindings to them seems to be without purpose, unless we want to re-implement the HAL in Rust. What library would use frc-netcomm-sys directly? WPIUtil seems to be the only exception.

Who owns and/or is responsible for publishing all of these libraries to crates.io? How do we ensure that we never run into versioning issues and that old code is not broken by a new release? I like the system we have for getting libraries right now--they're fetched at build time from wherever the wpilib team hosts them. To users, the exact version is a hidden implementation detail. Releases are guaranteed to be lock-step and compatible, because there is only one release coming from one CI pipeline. Everything is automated. I'm somewhat worried about the complexity increase if we try to split out too many dependencies and don't group them together well enough.

This reminds me, we should probably change

wpilib-sys = {path = "../wpilib-sys", version = "0.4.0"}

to

wpilib-sys = {path = "../wpilib-sys", version = "= 0.4.0"}

[auscompgeek]

I'm not sure I agree that 1.76 MB is large (if binary size is what you mean)

wpilib-sys-0.4.0 is a bit over 5 MiB, making it larger than encoding_rs. Add that up over the number of releases you'll inevitably publish, crates.io and mirrors aren't very happy.

By comparison, syn and regex are about 1 MiB each, and those are already larger than average.

I don't think I understand the interplay of these libraries as well as you, but let me lay out what I think your solution is so we're on the same page: split each library or group of libraries that we currently ship (FRC_NetworkCommunication, NiFpga(Lv), niriodevenum, ...), create a crate that links that library, for the others to depend on and possibly expose rust bindings to that library.

Yup, that's the general idea. I think it'll be fine if we group all the ChipObject libraries together like WPILib do.

Creating bindings to them seems to be without purpose, unless we want to re-implement the HAL in Rust. What library would use frc-netcomm-sys directly?

Perhaps. I should probably stop procrastinating.

Maybe someone could convince a vendor to write their library code in Rust. The CTRE Phoenix CCI library doesn't use the WPILib HAL (they call into NetComm directly).

Who owns and/or is responsible for publishing all of these libraries to crates.io?

Totally open question I suppose. Avoiding a bus factor of 1 here is probably a good idea though.

How do we ensure that we never run into versioning issues and that old code is not broken by a new release?

As I understand, the NI libraries probably won't change past October. wpiutil wouldn't have any breaking changes once the season hits either.

[auscompgeek]

I've gone and created a repo for frc-chipobject-sys as well. That might end up with other dependencies itself - I've also created an ni-visa-sys repo, and I've noticed @connorworley has created a repo for ni-fpga (which will need to gain a proper -sys crate).

[connorworley]

I've actually been building using the custom cross images I set up, so I don't need a sys package for my use case. However, having one would be nice to keep things aligned with the rest of the project.

…

On Fri, Dec 6, 2019, 20:32 David Vo ***@***.***> wrote: I've gone and created a repo for frc-chipobject-sys as well. That might end up with other dependencies itself - I've also created an ni-visa-sys repo, and I've noticed @connorworley <https://github.com/connorworley> has created a repo for ni-fpga (which will need to gain a proper -sys crate). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#66?email_source=notifications&email_token=AALRH2W6CCA6TBPEUIRYFCTQXMRNXA5CNFSM4HYQMRA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEGF5SNY#issuecomment-562813239>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AALRH2TILGCBBBWTLRHU42LQXMRNXANCNFSM4HYQMRAQ> .

[auscompgeek]

You can't have multiple crates link to the same library, so we're going to need a -sys package either way.