Added support for audio/video support in hub.ingest

[aadityasinha-dotcom]

🚀 🚀 Pull Request

Checklist:

• My code follows the style guidelines of this project and the Contributing document
• I have commented my code, particularly in hard-to-understand areas
• I have kept the coverage-rate up
• I have performed a self-review of my own code and resolved any problems
• I have checked to ensure there aren't any other open Pull Requests for the same change
• I have described and made corresponding changes to the relevant documentation
• New and existing unit tests pass locally with my changes

Changes

Resolves #1556

[aadityasinha-dotcom]

@FayazRahman can you review this?

[istranic]

Hi @aadityasinha-dotcom Apologies for taking so long to respond to your PR. Are you still available to continue this work?

[aadityasinha-dotcom]

@istranic yes!!

[istranic]

@aadityasinha-dotcom ok great. I think next steps are:

• Resolve the conflics
• Add tests for video
• Migrate all test to more realistic datasets (multiple data points per class, etc.) that should live in this repo
  -See ingest_coco or ingest_yolo tests for inspiration. Added support for ingestion yolo and tweaked git_ignore to allow yolo #2099

[istranic]

Also @aadityasinha-dotcom is there a way to make this work will all file types that are supported in deeplake, not just audio+video?

[istranic]

Also @aadityasinha-dotcom Can you rename the function to ingest_classification, and implement parallel uploading using deeplake.compute. An example for similar datasets is here - You can also see more complex implementation in ingest_yolo and ingest_coco

I'm asking this because we're focusing a lot more on speed, reliability, and scale, so everything needs to work with bigger workloads.

[istranic]

Hey @aadityasinha-dotcom How's the progress on this PR. This is becoming quite high-priority internally, so just want to confirm it's moving along. If not, the current progress on this PR is really helpful, and our team can cake over and add some of the modifications I requested above.

[aadityasinha-dotcom]

Hey @istranic, sorry been busy last week. I have worked on adding the tests and yes, it will be helpful if someone can do modifications.
If you got any task for me, I am ready to work on it.

[istranic]

@aadityasinha-dotcom Got it, thx for the update.

[aadityasinha-dotcom]

@istranic anything else left in this PR?

[istranic]

Hey @istranic, sorry been busy last week. I have worked on adding the tests and yes, it will be helpful if someone can do modifications. If you got any task for me, I am ready to work on it.

@ProgerDav Can you pls look at this PR, since it's related to work you've been doing.

How come full tests aren't running.

[aadityasinha-dotcom]

In line 183 it says memory_path is not defined

[aadityasinha-dotcom]

Maybe because of the undeclared variable