Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ONNX] dynamo_export always uses external ONNX weights if the argument is ExportedProgram, even for small models #126479

Open
borisfom opened this issue May 16, 2024 · 3 comments
Open

[ONNX] dynamo_export always uses external ONNX weights if the argument is ExportedProgram, even for small models #126479

borisfom opened this issue May 16, 2024 · 3 comments
Assignees
@justinchuby
Labels
module: onnx Related to torch.onnx onnx-triaged triaged by ONNX team triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@borisfom
Copy link
Contributor

馃悰 Describe the bug

dynamo_export always uses external ONNX weights if the argument is ExportedProgram.
dynamo_export never uses external ONNX weights if the argument is Module - I filed separate issue for that.
This behavior is inconsistent with what onnx.export() had and forces users to calculate memory requirements themselves and branch on that.

Versions

Pytorch nightly 05/14/24
@borisfom borisfom mentioned this issue May 16, 2024
Open
@borisfom
Copy link
Contributor Author

This may be acceptable behavior if #126269 were fixed. Then user could always call dynamo_export(exported_program.module()) which would be the same behavior as onnx.export in terms of external weights.

@titaiwangms
Copy link
Collaborator

cc @justinchuby @shubhambhokare1 @xadupre
We should discuss about this.

@titaiwangms titaiwangms added module: onnx Related to torch.onnx onnx-triaged triaged by ONNX team labels May 17, 2024
@titaiwangms titaiwangms changed the title [dynamo] dynamo_export always uses external ONNX weights if the argument is ExportedProgram, even for small models [ONNX] dynamo_export always uses external ONNX weights if the argument is ExportedProgram, even for small models May 17, 2024
@titaiwangms titaiwangms mentioned this issue May 17, 2024
Open
@justinchuby justinchuby self-assigned this May 17, 2024
@titaiwangms titaiwangms mentioned this issue May 17, 2024
Open
@titaiwangms
Copy link
Collaborator

titaiwangms commented May 17, 2024 •
edited

cc @justinchuby @shubhambhokare1 @xadupre We should discuss about this.

We can leverage on "unlifting" in ExportedProgram.

pytorch/torch/export/exported_program.py

Line 498 in 3f28906

module = _unlift_exported_program_lifted_states(self)

@mikaylagawarecki mikaylagawarecki added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@justinchuby justinchuby

Labels
module: onnx Related to torch.onnx onnx-triaged triaged by ONNX team triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@justinchuby @borisfom @titaiwangms @mikaylagawarecki