-
Notifications
You must be signed in to change notification settings - Fork 97
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Vision dataset downloading #2910
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:50
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:50
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:50
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
rdondera-microsoft
had a problem deploying
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Failure
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 19:52
— with
GitHub Actions
Inactive
Test Results for assets-test250 tests 249 ✅ 4h 37m 2s ⏱️ Results for commit b788660. ♻️ This comment has been updated with latest results. |
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:05
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:05
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:05
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
had a problem deploying
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Failure
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Inactive
rdondera-microsoft
had a problem deploying
to
Testing
May 15, 2024 22:07
— with
GitHub Actions
Failure
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 00:33
— with
GitHub Actions
Inactive
rdondera-microsoft
had a problem deploying
to
Testing
May 21, 2024 00:33
— with
GitHub Actions
Failure
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 00:33
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
had a problem deploying
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Failure
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
had a problem deploying
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Failure
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 01:17
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
rdondera-microsoft
had a problem deploying
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Failure
rdondera-microsoft
temporarily deployed
to
Testing
May 21, 2024 04:57
— with
GitHub Actions
Inactive
iamrk04
approved these changes
May 21, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Support downloading vision + multimodal datasets in the dataset downloader component.
Implementation
It is not possible to directly save vision HF datasets to JSONL (as is done for NLP) because binary image data is not compatible with this setup. The solution was to keep the JSONL format but use a URL field that refers to image data saved to the default datastore. For each HF dataset, a specific adapter class extracts a label and a PIL image for a dataset instance and common code uploads the image, passes through the fields etc.
Extensions
New vision/multimodal datasets from HF can be added by adding new dataset adapters.
Tests
Ran the benchmarking pipeline e2e in the cloud: https://ml.azure.com/experiments/id/9f7237b4-3232-49d3-ae02-e07326242835?wsid=/subscriptions/dbd697c3-ef40-488f-83e6-5ad4dfb78f9b/resourcegroups/rdondera/providers/Microsoft.MachineLearningServices/workspaces/benchmarking&tid=72f988bf-86f1-41af-91ab-2d7cd011db47 runs "resisc45", "food101", "patch_camelyon", "gtsrb".