Support React Native

[hans00]

Make it support React Native.

And I made a example for it.
https://github.com/hans00/react-native-transformers-example

On Android need some library patch.

TODO:

• Check models are works fine
• Research more efficiently image processing

[xenova]

Woah nice! There are a lot of people who are interested in doing stuff like this! Looking forward to reviewing this when you're ready! cc @pcuenca

[jhen0409]

I'm investigating a performance issue with Whisper tiny.en, and it looks like the performance is not as expected in the example.

I quickly add log for onnxruntime-react-native/lib/backend.ts, this is the result:

# Encode result
 LOG  ONNX session run finished 767 ms

# Decode result
 LOG  ONNX session run finished 4518 ms
# ... 4 runs ...
 LOG  ONNX session run finished 4856 ms
 LOG  Time: 40016
 LOG  Result: {"text": " (buzzing)"}

Click to expand

const t0 = performance.now()
// this.#inferenceSession === NativeModules.Onnxruntime
const results: Binding.ReturnType = await this.#inferenceSession.run(this.#key, input, outputNames, options);
const output = this.decodeReturnType(results);
console.log('ONNX session run finished', performance.now() - t0)

Also see logs of native part (added log to OnnxruntimeModule.java), it is significantly less than the time taken by JS part:

# Encode result
09:47:59.203 ONNXRUNTIME_RN run() is finished, time: 273 ms

# Decode result
09:48:00.280 ONNXRUNTIME_RN run() is finished, time: 339 ms
# ... 4 runs ...
09:48:23.807 ONNXRUNTIME_RN run() is finished, time: 541 ms

I think this problem may come from blocking caused by native module passing too large data. I think request/help onnxruntime for migrate to JSI module may solve this problem.

EDIT: It seems the logger have some bugs leads me to think that the problem is from the native bridge. I add timeout await to the decodeReturnType call and found the issue is from this function.

[pax-k]

@xenova are there any other blockers? I'm using @hans00 's PR in production with Expo + RN and works fine (at least for text embeddings with MiniML/Supabase models).

Also I'm already porting https://github.com/xenova/whisper-web to RN by building on top of this PR.

If you have a @next release channel, that would be great, so we can test this further in production without potentially breaking the main release channel.

Otherwise, please merge it and we'll open PRs to fix any potential bugs.

[Sly777]

@xenova is there any timeline for checking and merging this?