Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[export] Add customization API for unflattener to control restored module types. #125245

Open
zhxchen17 opened this issue Apr 30, 2024 · 0 comments
Assignees

Comments

@zhxchen17
Copy link
Contributor

zhxchen17 commented Apr 30, 2024

馃殌 The feature, motivation and pitch

Today when we call unflattener, the restored module will look like the following:

(UnflattenedModule):
   (InterpreterModule)
      (InterpreterModule)
      (InterpreterModule)
      (InterpreterModule)

This prevents some eager transformations from functioning e.g. pipeline + tensor parallelism or quantization. Ideally it should be possible for users to restore the original submodule types like the following:

(UnflattenedModule):
   (ModuleList)
      (Linear)
      (Linear)
      (Linear)

We had an internal discussion recorded in this doc: https://docs.google.com/document/d/1CAKtSiPWMyjOwaEbTmoT1yM2vF2ZucxxbVeFYhhdy8A/edit

This issue is filed for tracking purpose.

Alternatives

No response

Additional context

No response

cc @avikchaudhuri @gmagogsfm @tugsbayasgalan @angelayi @suo @ydwu4

@zhxchen17 zhxchen17 self-assigned this Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant