Not all operations are supported by the Edge TPU

[ItsMeTheBee]

Hey there!

I´m using tensorflow gpu version 2.3.0 and my custom model (anchors and image size are the same, i only modified the number of classes and used relu layers instead of leaky).

First of all I needed to add this line converter.experimental_new_converter = False because my tensorflow version is higher than 2.2 (just in case this is interesting for you).
After that I was able to run edgetpu_compiler -s quant.tflite but the output tells me that some operations will be run on the CPU

Edge TPU Compiler version 14.1.317412892

Model compiled successfully in 329 ms.

Input model: quant.tflite
Input size: 8.38MiB
Output model: quant_edgetpu.tflite
Output size: 8.40MiB
On-chip memory used for caching model parameters: 1.84MiB
On-chip memory remaining for caching model parameters: 256.75KiB
Off-chip memory used for streaming uncached model parameters: 5.64MiB
Number of Edge TPU subgraphs: 1
Total number of operations: 25
Operation log: quant_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 19
Number of operations that will run on CPU: 6

Operator                       Count      Status

RESIZE_NEAREST_NEIGHBOR        1          Operation version not supported
MAX_POOL_2D                    6          Mapped to Edge TPU
CONCATENATION                  1          More than one subgraph is not supported
QUANTIZE                       1          More than one subgraph is not supported
QUANTIZE                       2          Mapped to Edge TPU
QUANTIZE                       1          Operation is otherwise supported, but not mapped due to some unspecified limitation
CONV_2D                        2          More than one subgraph is not supported
CONV_2D                        11         Mapped to Edge TPU

Do you have any idea how to fix this?

[alexanderswerdlow]

I have the exact same problem with the same number/type of unsupported ops. Also on tf 2.3.0 (gpu).

[alexanderswerdlow]

Update: Tensorflow 2.1.0 seems to work for me. Didn't have to change anything about the EdgeTPU compiler, just the TF version while running keras_to_tflite_quant.py

[raulcastar]

I have experienced the same issue, finally a was able to fully compile a yolov3-tiny custom model with the latest tensorflow version (v2.3). The main problem here is that the edge tpu compiler can't convert the RESIZE_NEAREST_NEIGHBOR operation (that comes from the Upsampling2D layer in the keras transformation) and it creates the rest of failed compiled operations due to subgraphs.

What the edge tpu compiler really can't convert is the half_pixel_centers = True argument of the resize neighbor operation in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/image_ops_impl.py#L1627 so the solution is to set it to false, I guess the same problem appears when trying to compile the bilinear operation. The workaround that I did was to hardcode half_pixel_centers = False in the python/site-packages/tensorflow/python/ops/gen_image_ops.py that the system generates when compiling tensorflow, but I have yet to do more research on the problem.

[hamhochoi]

I have experienced the same issue, finally a was able to fully compile a yolov3-tiny custom model with the latest tensorflow version (v2.3). The main problem here is that the edge tpu compiler can't convert the RESIZE_NEAREST_NEIGHBOR operation (that comes from the Upsampling2D layer in the keras transformation) and it creates the rest of failed compiled operations due to subgraphs.

What the edge tpu compiler really can't convert is the half_pixel_centers = True argument of the resize neighbor operation in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/image_ops_impl.py#L1627 so the solution is to set it to false, I guess the same problem appears when trying to compile the bilinear operation. The workaround that I did was to hardcode half_pixel_centers = False in the python/site-packages/tensorflow/python/ops/gen_image_ops.py that the system generates when compiling tensorflow, but I have yet to do more research on the problem.

It solved the problem (Y)!

[jeonds1127]

I have experienced the same issue, finally a was able to fully compile a yolov3-tiny custom model with the latest tensorflow version (v2.3). The main problem here is that the edge tpu compiler can't convert the RESIZE_NEAREST_NEIGHBOR operation (that comes from the Upsampling2D layer in the keras transformation) and it creates the rest of failed compiled operations due to subgraphs.

What the edge tpu compiler really can't convert is the half_pixel_centers = True argument of the resize neighbor operation in https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/image_ops_impl.py#L1627 so the solution is to set it to false, I guess the same problem appears when trying to compile the bilinear operation. The workaround that I did was to hardcode half_pixel_centers = False in the python/site-packages/tensorflow/python/ops/gen_image_ops.py that the system generates when compiling tensorflow, but I have yet to do more research on the problem.

Operator Count Status

MAX_POOL_2D 6 Mapped to Edge TPU
RESIZE_NEAREST_NEIGHBOR 1 Mapped to Edge TPU
QUANTIZE 4 Mapped to Edge TPU
CONCATENATION 1 Mapped to Edge TPU
CONV_2D 13 Mapped to Edge TPU

It solved the problem (Y)! Thank You :)