Generalising FakeData

[ctr26]

Hi all,

I've added a convenience module for creating a fake image folder in torch vision. I recently needed to create a mock folder and this can be a little painful without a FakeImage folder like this

Thanks

Craig

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8344

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 3099a07 with merge base d868be9 ():

NEW FAILURES - The following jobs have failed:

• Lint / python-source-and-configs / linux-job (gh)
  RuntimeError: Command docker exec -t 16884bfc8caa891c4f603ba43b782ad31c3f569d86d735ba46003cf533bd1b6a /exec failed with exit code 1
• Lint / python-types / linux-job (gh)
  RuntimeError: Command docker exec -t b9387e68930a8d4e025343515dc58bdc0801c7aa03e6d3c335839d9518e9fb36 /exec failed with exit code 1
• Tests / unittests-windows (3.12, windows.4xlarge, cpu) / windows-job (gh)
  Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

Hi @ctr26!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

[NicolasHug]

Thanks for the PR @ctr26 , but what is the benefit of FakeImageFolder over the existing FakeData class?

[ctr26]

Thanks for the PR @ctr26 , but what is the benefit of FakeImageFolder over the existing FakeData class?

Thank you for your question regarding the benefit of FakeImageFolder over the existing FakeData class.

The key advantage lies in its compatibility with testing environments that utilise ImageFolder derived datasets (most of them). In my specific use case, it was challenging for me to perform mock data tests on my MLOps pipeline, which relied on ImageFolder rather than Dataset. The existing FakeData class, although useful, does not support the kwargs expected by ImageFolder, leading to errors when used in this context and generally isn't a drop replacement.

The alternative being hacky solutions like downloading stock datasets like celeba as a mock, which seems excessive.

Thanks

Craig

[NicolasHug]

Thanks for the details. IIUC you just need the signature of FakeData to be compatible with that of VisionDataset in order to avoid special-casing the call to FakeData?

Instead of creating a new class, would it be enough for your use-case to add *args, **kwargs to the signature of FakeData (both of which would be ignored/unsued)?

[ctr26]

That would an easier solution yes. My only concern there was that it would have unforeseen downstream effects. I've pushed the changes

[facebook-github-bot]

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

[NicolasHug]

Thanks for the feedback @ctr26 . I'm still not sure I completely understand why FakeData and VisionDataset need to have the same signature. But since this seems to be a very specific use-case (and not necessarily a broadly useful one), I might recommend for you to simply inherit from FakeData and accept **kwargs in your own subclass, like so:

class MyFakeData(FakeData):
    def __init__(self, ..., *kwargs):
        super().__init__(...)