Downloading 'bridge' Dataset using tfds-nightly

[Wolfwjs]

Hello,

I am encountering an issue while trying to download the bridge dataset using TensorFlow Datasets (TFDS). I am currently using tfds-nightly version 4.9.3.dev202311230044.

Issue Description

When I attempt to load the 'bridge' dataset with the following code:

import tensorflow_datasets as tfds
tfds.load('bridge', data_dir="mypath/“)

I receive an error indicating that the dataset cannot be downloaded.

In addition to the issue described above, I am considering the possibility of manually downloading the dataset from the following URL:
https://rail.eecs.berkeley.edu/datasets/bridge_release/data/tfds/bridge_dataset/1.0.0/

Before I proceed with this approach, I have a few questions:

1. Is manually downloading the dataset from this URL a recommended solution to the problem I am experiencing?

2. Will manually downloaded data integrate seamlessly with the TensorFlow Datasets API, or are there additional steps required to ensure compatibility?

Any guidance or suggestions regarding manual downloading would be greatly appreciated.

Thank you!

[kpertsch]

Thanks for flagging this -- I was able to reproduce the error in our dataset Colab, not only for Bridge but for all datasets. I suspect this is an issue with the current tfds nighly. I tried with some non-robot datasets and those still seem to work -- @quanvuong could you check with the TFDS folks what's up?

In the meantime: for downloading the datasets you can use the "manual download" approach:
You can get the raw bucket path using the command from our colab:

Then you can download the dataset with gsutil -m cp -r dataset2path(dataset_name) <your_target_path>
-- sorry for the inconvenience!

[quanvuong]

Raised the issue to tfds team here tensorflow/datasets#5170

[Wolfwjs]

Hi @kpertsch , thank you for your suggestion. I would like to confirm, if I use the command gsutil -m cp -r gs://gresearch/robotics/bridge/0.1.0 /mypath/, does it download the latest bridge v2 data?

[kpertsch]

Yes, this will download the Bridge v2 dataset that's part of OXE -- it is slightly differently formatted than the officially released Bridge dataset & lacks the additional camera streams. The official Bridge data is linked on the Bridge website and can be downloaded in TFDS format here: https://rail.eecs.berkeley.edu/datasets/bridge_release/data/tfds/bridge_dataset/1.0.0/

[Wolfwjs]

Perfect, the Bridge v2 dataset is exactly what I need. Additionally, could you please explain what is meant by "lacks the additional camera streams"? Does this refer to a lack of image or video data?

[kpertsch]

Basically the dataset in OXE only has a single camera stream per demonstration (because RT-X was only trained with single camera input), but the official Bridge dataset from the website has up to 3 additional camera streams on some trajectories. We will update the Bridge data version in OXE with a future update to fully match the officially released one!

[Wolfwjs]

Thank you for your patient response! I look forward to your updates, and for now, I will start with a single camera stream.