Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forward input dims in Reshape ShapeHelper #2828

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

tungld
Copy link
Collaborator

@tungld tungld commented May 15, 2024

In this reshape op in a quantized bert model, the first two dimensions of the output are the same as the ones in the input, so we would like to forward IndexExprs of these dimensions from the input to the output so that DimAnalysis works. But it failed because of using IndexExpr.selectOrSelf.

%184 = onnx.Constant dense<[0, 0, 12, 64]> : tensor<4xi64>
%494 = "onnx.Reshape"(%491, %184) {allowzero = 0 : si64, onnx_node_name = "/bert/encoder/layer.0/attention/self/Reshape"} : (tensor<?x?x768xf32>, tensor<4xi64>) -> tensor<?x?x12x64xf32>

I think the better solution would be updating selectOrSelf, but I don't really understand it and just make a change in the Reshape's ShapeHelper as a workaround.

Signed-off-by: Tung D. Le <tung@jp.ibm.com>
@tungld
Copy link
Collaborator Author

tungld commented May 15, 2024

@AlexandreEichenberger if you know how to modify selectOrSelf to fix this issue, please let me know or please feel free to create a PR for that. Thanks!

Signed-off-by: Tung D. Le <tung@jp.ibm.com>
outputDims[i] = outputDims[i].selectOrSelf(
outputDims[i] == -1, numOfElements.floorDiv(numOfElementsFromShape));
dimShape == -1, numOfElements.floorDiv(numOfElementsFromShape));
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed this to make the compare expression literal.

@tungld
Copy link
Collaborator Author

tungld commented May 15, 2024

It looks like I found the way to change selectOrSelf but it only works if the condition is literal at compile time.

@AlexandreEichenberger
Copy link
Collaborator

@tungld is there still a case to look into?

@tungld
Copy link
Collaborator Author

tungld commented Jun 5, 2024

@tungld is there still a case to look into?

I changed selectOrSelf to handle the case where the comparison is literal. It works for the quantized bert model. I hope it's enough for most cases.

@tungld
Copy link
Collaborator Author

tungld commented Jun 5, 2024

@AlexandreEichenberger It's ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants