Replies: 1 comment
-
CoreML does not seem to have good support for models with dynamic input shapes. Also, it may lack some necessary operators, so they fall back to CPU. Usually, if some heavy operators are placed at different devices, the performance decrease a lot for frequent memory copying, and even poorer than pure CPU, because diffusion models contain loops where many operators are executed for multiple times. Maybe you can check out the official documentation of ONNXRuntime for more details about its CoreML EP. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I'm interested in running these models in CoreML on Apple silicon. I've attempted to convert the Onnx models unsuccessfully and the onnx runtime runs these models on CPU (rather than using CoreML).
I have seen that some other complex models have had successful conversions (https://github.com/apple/ml-stable-diffusion).
I imagine the level of effort to convert these may be too much for an opensource project, I'm interested to discuss this.
Beta Was this translation helpful? Give feedback.
All reactions