Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ADD] PyTorch embedding layer to reduce memory footprint #451

Open
wants to merge 22 commits into
base: reg_cocktails
Choose a base branch
from
Open

[ADD] PyTorch embedding layer to reduce memory footprint #451

wants to merge 22 commits into from

Conversation

ravinkohli
Copy link
Contributor

@ravinkohli ravinkohli commented Jul 16, 2022 •
edited

Updates embedding layers to pytorch embedding.

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Note that a Pull Request should only contain one of refactoring, new features or documentation changes.
Please separate these changes and send us individual PRs for each.
For more information on how to create a good pull request, please refer to The anatomy of a perfect pull request.

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

Description

Motivation and Context

How has this been tested?

@ravinkohli ravinkohli marked this pull request as draft July 16, 2022 15:36
ravinkohli
ravinkohli commented Jul 18, 2022
View reviewed changes
autoPyTorch/api/base_task.py Outdated
@@ -249,6 +249,7 @@ def __init__(

self.input_validator: Optional[BaseInputValidator] = None

# if search_space_updates is not None else get_search_updates(categorical_indicator)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# if search_space_updates is not None else get_search_updates(categorical_indicator)

@ravinkohli ravinkohli marked this pull request as ready for review July 21, 2022 08:32
@ravinkohli ravinkohli changed the title Reg cocktails pytorch embedding [ADD] PyTorch embedding layer to reduce memory footprint Aug 9, 2022
@ravinkohli ravinkohli mentioned this pull request Aug 9, 2022
Closed
1 task
@ravinkohli ravinkohli added this to To Do in Regularisation Cocktails via automation Aug 16, 2022
@ravinkohli ravinkohli added the enhancement New feature or request label Aug 16, 2022
@ravinkohli ravinkohli moved this from To Do to In Progress in Regularisation Cocktails Aug 16, 2022
@ravinkohli ravinkohli force-pushed the reg_cocktails-pytorch_embedding branch from 3761b53 to d4717fb Compare August 16, 2022 15:22
@ravinkohli ravinkohli linked an issue Aug 16, 2022 that may be closed by this pull request
Replace Embedding to use nn.Embedding from pytorch #428
Open
ravinkohli and others added 17 commits October 25, 2022 17:44
@ravinkohli
update eta for experiments
0bd7d5f
@ravinkohli
add check if True is in value range
76dae54
@ravinkohli
Reg cocktails common paper modifications 2 (#417)
d26c611 
* remove remaining differences

* Reg cocktails common paper modifications 5 (#418)

* add hasttr

* fix run summary
@ravinkohli
have working embedding from pytroch
bc60e31
@ravinkohli
divide columns to encode and embed based on threshold
13fad76
@ravinkohli
cleanup unwanted changes
cf4fd98
@ravinkohli
use shape after preprocessing in base network backbone
af41dd7
@ravinkohli
remove redundant call to load datamanager
9706875
@ravinkohli
add init file for column splitting
def144c
@ravinkohli
fix tests
926a757
@ravinkohli
fix precommit and add test changes
7567d26
@ravinkohli @nabenabe0928
[ADD] Calculate memory of dataset after one hot encoding (pytorch emb…
09fdc0d 
…edding) (#437)

* add updates for apt1.0+reg_cocktails

* debug loggers for checking data and network memory usage

* add support for pandas, test for data passing, remove debug loggers

* remove unwanted changes

* :

* Adjust formula to account for embedding columns

* Apply suggestions from code review

Co-authored-by: nabenabe0928 <47781922+nabenabe0928@users.noreply.github.com>

* remove unwanted additions

* Update autoPyTorch/pipeline/components/preprocessing/tabular_preprocessing/TabularColumnTransformer.py

Co-authored-by: nabenabe0928 <47781922+nabenabe0928@users.noreply.github.com>
@ravinkohli
suggestions from review
3aef02e
@ravinkohli
add preprocessed_dtype to determine double or float
8e3dbef
@ravinkohli
test fix in progress
52427bc
@ravinkohli
TODO: fix errors after rebase
90512ee
@ravinkohli
Reg cocktails apt1.0+reg cocktails pytorch embedding reduced (#454)
895b904 
* reduce number of hyperparameters for pytorch embedding

* remove todos for the preprocessing PR, and apply suggestion from code review

* remove unwanted exclude in test
@ravinkohli ravinkohli force-pushed the reg_cocktails-pytorch_embedding branch from cbd4d7e to 1be80d5 Compare October 25, 2022 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nabenabe0928 nabenabe0928 Awaiting requested review from nabenabe0928

@theodorju theodorju Awaiting requested review from theodorju

Assignees
No one assigned
Labels
enhancement New feature or request
Projects
Regularisation Cocktails
In Progress
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Replace Embedding to use nn.Embedding from pytorch
1 participant
@ravinkohli