Access external library

[acul009]

Is your feature request related to a problem? Please describe.
This might already be possible, but I didn't find out how.
I already have a rust crate which exposes a library. I would like to use this library in flutter.

I can call the library functions if I just write some wrappers in the api folder, but I would love to tell FRB to scan my already existing library.
Is that somehow possible?

While I could try to write wrappers, I'd need wrappers for every struct and function in the library, which would be quite a lot.

// /src/api/simple.rs
use svalin::client::{Client, FirstConnect};

// even with this wrapper function, the return type if from my library, which doesn't work all that well :(
// This also does not work, as the generated rust functions are missing the use statement, triggering a "not found" error
pub async fn first_connect(address: String) -> Result<FirstConnect> {
    Client::first_connect(address).await
}

Describe the solution you'd like
Some way to embed my already existing library with this project. Maybe just setting a symlink from the /api dir to the could work?

Describe alternatives you've considered
I don't think I have any, as I need both headless variants of my program as well as a gui version

Additional context
Damn you, after experimenting a bit with this project I don't want to go back to any other gui framework for rust.

Everything else feels so clunky and flutter_rust_bridge is just way to easy and fun :P

[acul009]

So after a bit of tinkering, I found out that you actually seem to support this in a way:

https://cjycode.com/flutter_rust_bridge/guides/types/translatable/external/diff-crate

I do still have issues with enums, which might be resolved by this. I will update this post once I figure out more.

[acul009]

So I've experimented some more and I think I fugured out a bit more about how you handle those external types.
It looks like external types are always just for data conversion and do not provide their methods and functions.
If no type helpers are provided, you fall back to just using a reference pointer.

My problem is, that I'm interested in these functions, which seems to leave me only one choice at the moment:
Refactor everything into a single crate and provide the types in a subdirectory.

From what I can see, I have 2 issues at the moment:

1. I need to get input from multiple crates. In my case all my important crates are in a single workspace. If they weren't I still think it might be possible to read the rs files from the cargo cache.

2. I have to be able to read the main directory with the .lib file. In my case I can circumvent this by putting all code besides the main and lib files in a module and specifying the path like this: rust_input: ../svalin/src/*/**/*.rs

Just so you don't get the wrong impression:
This project is really awesome and I don't demand you implement any of this for me :)
I'm mostly just writing this, so someone else in my position can find some detailed info on why this doesn't work.

Nothing is worse than a forum entry with "nvm, I fixed it" at the end

[fzyzcjy]

Happy to hear that flutter+rust+flutter_rust_bridge is good for you!

Yes, the current way may be using the https://cjycode.com/flutter_rust_bridge/guides/types/translatable/external/diff-crate feature.

However, I am also considering scanning 3rd party crates (e.g. last time someone also raised scenarios like yours when there are 3rd party crate that is controllable).
There may be some small issues to be resolved before implementing. For example:

• What about name conflicts between a 3rd party type and a 1st party type? (It is hard to know whether a name is from 1st or 3rd party due to Rust's super flexible use. Maybe we just throw error and let users manually put ignore markers on them?)
• What if 3rd party crate want to add frb markers like #[frb(sync)] but do not want to introduce direct dependency to frb? (Maybe instead allow users to write /// #[frb(sync)]. Or, since this is user-controllable 3rd party crate, just let users depend on frb.)
• What if users cannot control the 3rd party crate? (Then maybe use the mirror feature today)

do not provide their methods and functions

This can be done via https://cjycode.com/flutter_rust_bridge/guides/miscellaneous/methods#methods-in-external-crates (a feature added recently)

[normalllll]

Sorry, I can't use it according to https://cjycode.com/flutter_rust_bridge/guides/miscellaneous/methods#methods-in-external-crates

using #[frb(external)] Still got an opaque type

my external crate:

impl MyClient {
    pub async fn get_posts(&self, tags: Vec<String>, limit: usize, page: usize) -> Result<Vec<crate::model::Post>, Box<dyn std::error::Error>> {

    }

    pub async fn download_to_file(&self, url: String, file_path: String, progress_callback: impl Fn(usize, usize) + 'static) -> Result<(), Box<dyn std::error::Error>> {

    }

    pub async fn download_to_memory(&self, url: String, progress_callback: impl Fn(usize, usize) + 'static) -> Result<Vec<u8>, Box<dyn std::error::Error>> {

    }
}

frb:

struct MyClient{

}

#[frb(external)]
impl MyClient{
    pub fn new() -> Self{}
    pub async fn get_posts(&self, tags: Vec<String>, limit: usize, page: usize) -> Result<Vec<Post>, Box<dyn std::error::Error>>{ }
    pub async fn download_to_file(&self, url: String, file_path: String, progress_callback: impl Fn(usize, usize)->DartFnFuture<()> + 'static) -> Result<(), Box<dyn std::error::Error>>{}
    pub async fn download_to_memory(&self, url: String, progress_callback: impl Fn(usize, usize) ->DartFnFuture<()>+ 'static) -> Result<Vec<u8>, Box<dyn std::error::Error>>{}
}

got dart:

@sealed
class Post extends RustOpaque {
  Post.dcoDecode(List<dynamic> wire) : super.dcoDecode(wire, _kStaticData);

  Post.sseDecode(int ptr, int externalSizeOnNative)
      : super.sseDecode(ptr, externalSizeOnNative, _kStaticData);

  static final _kStaticData = RustArcStaticData(
    rustArcIncrementStrongCount:
        RustLib.instance.api.rust_arc_increment_strong_count_Post,
    rustArcDecrementStrongCount:
        RustLib.instance.api.rust_arc_decrement_strong_count_Post,
    rustArcDecrementStrongCountPtr:
        RustLib.instance.api.rust_arc_decrement_strong_count_PostPtr,
  );
}

[normalllll]

I want to automatically mirror the model of the external crate

[fzyzcjy]

Hi, external does not automatically give non-opaque types. Currently (before the proposed feature is implemented) the mirroring needs to be manually specified via #[frb(mirror)].

[acul009]

In my personal opinion that's actually fine for most uses, as you're mostly working with "methods" most of the time anyway.

For my use case I'll try using a compination of #[frb(mirror)] and #[frb(external)].

It would be really cool to enable reading the information automatically though. Maybe something like this?

// this allows accessing the public fields of the struct,
// so the default implementation if the type was defined inside api
frb_external_mirror!(external_crate::submodule::MyStruct)

// this allows using the methods and function associated with a type
// A lot of types don't have public fields, which means this would allow to use most libraries already.
frb_external_opaque!(external_crate::submodule::MyApiHandler)

That would require scanning the external crate, which sounds kind of difficult or at least like it's a pain to implement.

What about name conflicts between a 3rd party type and a 1st party type? (It is hard to know whether a name is from 1st or 3rd party due to Rust's super flexible use. Maybe we just throw error and let users manually put ignore markers on them?)

I think throwing an error asking to write the whole path (e.g.: extcrate::submodule::Type) would be fine in this scenario. You can always improve the detection at a later time without breaking compatibility.

What if 3rd party crate want to add frb markers like #[frb(sync)] but do not want to introduce direct dependency to frb? (Maybe instead allow users to write /// #[frb(sync)]. Or, since this is user-controllable 3rd party crate, just let users depend on frb.)

I'm not sure that is even neccesary. Usually the programmer using the bridge would have to decide I think.
Personally I'd prefer to add sync and mirror markers for an external crate manually in my api folder.

What if users cannot control the 3rd party crate? (Then maybe use the mirror feature today)

I think all cases should be treated as if the person using frb doesn't have access to the crate. Conceptually a crate implies that it's usable with other code too, not just within a project, otherwise you'd just use submodules, right?

I'll try working with the #[frb(external)] for now, as that is pretty much what I need. I'd love to have these generated automagically though ;)
That would mean you could plug in pretty much any crate without too much work.

[acul009]

so I couldn't test if the generated code works yet, but I have no doubts it will.

@normalllll
The original example can be found here:
https://github.com/fzyzcjy/flutter_rust_bridge/blob/master/frb_example/pure_dart/rust/src/api/external_impl.rs

Here's my example for anyone else looking for some code which demonstrates using an external library:

use anyhow::Result;
use flutter_rust_bridge::frb;

// if you want to use external methods you need a *pub* use statement!!!
pub use svalin::client::{Client, FirstConnect, Init, Login};

// Once you have aquired your type with the pub use statement, you can mirror the function signatures.
#[frb(external)]
impl Client {
    pub fn get_profiles() -> Result<Vec<String>> {} // as by the docs, you don't need a function body here
    pub async fn first_connect(address: String) -> Result<FirstConnect> {}
}

// If you want to forward the internal structure instead, you can use mirror to tell frb the signature of your type
// Be sure to prefix your original struct name with an underscore to avoid a name conflict.
// you don't need this if you just want to access functions. this is only required if you want to read the fields of a struct
#[frb(non_opaque, mirror(FirstConnect))]
pub enum _FirstConnect {
    Init(Init),
    Login(Login),
}

// once again you just mirror the function signature here
#[frb(external)]
impl Init {
    pub async fn init(&self) -> Result<()> {}
}

#[frb(external)]
impl Login {
    pub async fn login(&self) -> Result<()> {}
}

Edit: fixed missing non_opaque on enum - required when part of the enum is opaque but the enum itself shouldn't be

[fzyzcjy]

Looks reasonable!

[...] I think all cases should be treated as if the person using frb doesn't have access to the crate.

Based on discussions above, here are more brainstorms:

• Maybe we can make users never touch 3rd party crate, while still allowing for arbitrary configurations (e.g. add #[frb(sync)], by creating syntaxes inside the main crate. For example,

frb! {
use external_crate::submodule::MyStruct;
#[frb(whatever)] struct MyStruct {} // Use `{}`, i.e. no need to really manually specify the fields - all done automatically
#[frb(whatever)] pub fn this_is_external_func() {}
}

[acul009]

Maybe we can make users never touch 3rd party crate, while still allowing for arbitrary configurations (e.g. add #[frb(sync)], by creating syntaxes inside the main crate. For example,

That's exactly what I meant. With this functionality it would be a breeze to use any rust crate in flutter.
You'd open up the possibillity of using quite a lot of libraries only available in cli tools to be directly used in a GUI.

[fzyzcjy]

Totally agree. Maybe we can firstly try our best to derive everything automatically, and only when something does not work (e.g. very weird types), skip it and require users to manually do something (e.g. make a thin wrapper using a few lines of code)

[acul009]

I'm pretty new to rust so more complex code is stilll difficult for me to work with, but let me know if I can help in any way.

[fzyzcjy]

Possibly related: #1911

[acul009]

That sounds pretty much like what I'm doing right now.

Basically you'd need a resolver which finds out which crate a type is from and then looks up where to find the relevant code in the Cargo.toml. That could be a local directory, git repo or something from crates.io.

That sounds possible, but also like a lot of work, so maybe implementing this in small steps would be better.

It would already be really cool if mirror(Type) could look up the definition by itself. That still requires a hell lot of work, but it's probably more doable.

[vhdirk]

For my use-case, having opaque types is good enough. And mirroring the types that do not have to be opaque, while somewhat cumbersome, is fine, too.
That being said, I wouldn't mind annotating the external library since it is under my control. That's how I do it with pyo3 bindings: for pyo3 it isn't necessary to modify the external crate, though it saves a ton of duplications if you do.

[fzyzcjy]

For future readers: Related - #1980

[fzyzcjy]

Part 0: #2000
Part 1: #2007