Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Percentage FLOPS or Multiply Adds - inspired by #199 #200

Open
mert-kurttutan opened this issue Nov 28, 2022 · 4 comments
Open

Percentage FLOPS or Multiply Adds - inspired by #199 #200

mert-kurttutan opened this issue Nov 28, 2022 · 4 comments

Comments

@mert-kurttutan
Copy link
Contributor

mert-kurttutan commented Nov 28, 2022

Option for column representing Percentage FLOPS or Multiply Adds. (similar to #199)

I think this option would be really useful to see which part of model should be optimized if necessary. It also useful to have an idea about the scaling of the model as make your model bigger and bigger.

At first sight, it seems that this could be implemented in a way similar to that of #199.

If this seems reasonable enough, I can come up with PR.

@TylerYep
Copy link
Owner

This seems like a good idea, PRs appreciated :)

I would prefer some cleaner way of providing the percentage form of all of these measurements though. If we keep going we'll have 2x the options for all columns (num_params, percent_params), (num_macs, percent_macs), etc.

But if we can't think of anything better, adding a few extra params would be fine.

@mert-kurttutan
Copy link
Contributor Author

mert-kurttutan commented Nov 29, 2022

I agree that there should be prettier API to deal with these.
I think there are several ways to do it:

  1. As you suggested, just add new settings params for each option
  2. (Maybe more uniform) Add percentage settings for all params that return percentage for all of them
  3. Use option 2) and add other options for individual percentage settings. For instance, say I only want percentage for all info except for Multiply Add. I would turn on all_percentage settings and turn off percentage for Multiply Add, which will overwrite the setting by all_precentage

Just brainstorming, some of them may seem unnecessarily complex:)
After having more clear idea about this API, I can start with some PR.

@mert-kurttutan
Copy link
Contributor Author

Sorry, maybe it was not clear. Do you any feedback on the list of formats I suggested? I will move on based on a particular choice of format.

@TylerYep
Copy link
Owner

TylerYep commented Dec 1, 2022

I guess 1) is okay for now. From the options, I think anything else will make the API more confusing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants