TfLiteGpuDelegate Init: BATCH_MATMUL: Node 333 is already a consumer of the value 334

[harrylal]

Issue type

Support

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

2.16

Custom code

Yes

Mobile device

Android 10

GPU model and memory

Mali - G72*3

Tflite Model

tflite model

Current behavior?

Expected Behavior: The model should initialize and run on the GPU as it does on the CPU.
Actual Behavior: When attempting to run the model on the GPU using the TensorFlow Lite GPU delegate, the initialization fails.

Standalone code to reproduce the issue

adb install -r -d -g android_arm_benchmark_model.apk
adb push TwinLiteNetLane.tflite /data/local/tmp/
adb shell am start -S   -n org.tensorflow.lite.benchmark/.BenchmarkModelActivity   --es args '"--graph=/data/local/tmp/TwinLiteNetLane.tflite --use_gpu=true"'

Relevant log output

04-29 08:55:31.116   502   502 I ADB_SERVICES: service_to_fd shell,v2,TERM=xterm-256color,raw:am start -S -n org.tensorflow.lite.benchmark/.BenchmarkModelActivity --es args "--graph=/data/local/tmp/TwinLiteNetLane.tflite --use_gpu=true"
04-29 08:55:32.058  2766  2766 I tflite_BenchmarkModelActivity: Running TensorFlow Lite benchmark with args: --graph=/data/local/tmp/TwinLiteNetLane.tflite --use_gpu=true
04-29 08:55:32.062  2766  2766 I tflite  : Log parameter values verbosely: [0]
04-29 08:55:32.062  2766  2766 I tflite  : Graph: [/data/local/tmp/TwinLiteNetLane.tflite]
04-29 08:55:32.063  2766  2766 I tflite  : Use gpu: [1]
04-29 08:55:32.064  2766  2766 I tflite  : Loaded model /data/local/tmp/TwinLiteNetLane.tflite
04-29 08:55:32.064  2766  2766 I tflite  : Initialized TensorFlow Lite runtime.
04-29 08:55:32.065  2766  2766 I tflite  : Created TensorFlow Lite delegate for GPU.
04-29 08:55:32.065  2766  2766 I tflite  : GPU delegate created.
04-29 08:55:32.072  2766  2766 I tflite  : Replacing 395 out of 395 node(s) with delegate (TfLiteGpuDelegateV2) node, yielding 1 partitions for the whole graph.
04-29 08:55:32.077  2766  2766 E tflite  : TfLiteGpuDelegate Init: BATCH_MATMUL: Node 333 is already a consumer of the value 334
04-29 08:55:32.077  2766  2766 I tflite  : Created 0 GPU delegate kernels.
04-29 08:55:32.077  2766  2766 E tflite  : TfLiteGpuDelegate Prepare: delegate is not initialized
04-29 08:55:32.077  2766  2766 E tflite  : Node number 395 (TfLiteGpuDelegateV2) failed to prepare.
04-29 08:55:32.078  2766  2766 E tflite  : Restored original execution plan after delegate application failure.
04-29 08:55:32.079  2766  2766 E tflite  : Failed to apply GPU delegate.

[sushreebarsa]

@harrylal Could you try for disabling specific GPU delegate optimizations related to Batch Matrix Multiplication, if applicable?
Thank you!

[harrylal]

@sushreebarsa Thank you for the response.

Exporting the model on a different enviornement (ultralytics docker) resolved this issue.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No