Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃悰 [Bug] An assertion error when upgrading to enqueueV3 interface #2811

Closed
zewenli98 opened this issue May 3, 2024 · 0 comments 路 Fixed by #2832
Closed

馃悰 [Bug] An assertion error when upgrading to enqueueV3 interface #2811

zewenli98 opened this issue May 3, 2024 · 0 comments 路 Fixed by #2832
Labels
bug Something isn't working

Comments

@zewenli98
Copy link
Collaborator

Bug Description

https://github.com/pytorch/TensorRT/blob/main/py/torch_tensorrt/dynamo/runtime/_PythonTorchTensorRTModule.py#L63-L65

This should be:

assert (
    self.engine.num_io_tensors == len(self.input_names) + len(self.output_names)
)

because num_io_tensors is not inflated by the number of optimization profiles, for enqueueV3 interface. This is different from the enqueueV2 interface:

assert engine.num_bindings == num_io_tensors * engine.num_optimization_profiles
@zewenli98 zewenli98 added the bug Something isn't working label May 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant