Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid setting transforms in the buffer #150

Open
lballes opened this issue Feb 24, 2023 · 1 comment
Open

Avoid setting transforms in the buffer #150

lballes opened this issue Feb 24, 2023 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@lballes
Copy link
Contributor

lballes commented Feb 24, 2023

Currently, the DataBuffer receives transform and target_transform and applies them when accessing an item. When deserializing the Learner state, this requires resetting the transforms (see DataBuffer.set_transforms) and generally bloats the code of the buffer.

I would propose to remove the transforms form the buffer and instead generalize the _TransformedDataset such that it can be applied to DataBuffers as well. This way, transforms are handled in a unified way by wrapping all relevant datasets in _TransformedDataset in the Learner.

To make _TransformedDataset more general, instead of passing transform and target_transform, we could pass a single transform that operates on the entire data point. When wrapping a buffer in _TransformedDataset, we just need to make sure that the metadata are "ignored" (e.g., transform=lambda (data_point, metadata): (my_transform(data_point), metadata)).

@lballes lballes added the discussion Discussion about new solutions or potentially bigger changes label Feb 24, 2023
@lballes lballes added this to the Renate 0.2 milestone Feb 24, 2023
@lballes lballes self-assigned this Mar 17, 2023
@lballes lballes removed this from the Renate 0.2 milestone Mar 31, 2023
@lballes
Copy link
Contributor Author

lballes commented Apr 26, 2023

Any concerns on this? Otherwise I would add this to the next milestone and take care of it.

@wistuba wistuba added this to the Renate 0.3 milestone Apr 26, 2023
@lballes lballes added enhancement New feature or request and removed discussion Discussion about new solutions or potentially bigger changes labels Apr 27, 2023
@lballes lballes removed this from the Renate 0.3 milestone May 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants