Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gs' not implemented #5324

Open
zelongjiang123 opened this issue Mar 14, 2024 · 2 comments
Open

gs' not implemented #5324

zelongjiang123 opened this issue Mar 14, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@zelongjiang123
Copy link

zelongjiang123 commented Mar 14, 2024

/!\ PLEASE INCLUDE THE FULL STACKTRACE AND CODE SNIPPET

Short description
Description of the bug.

Environment information

  • Operating System: windows
  • Python version: 3.11.8
  • tensorflow-datasets version: 4.9.4
  • tensorflow version: 2.15.0

Reproduction instructions

git clone https://github.com/tensorflow/datasets.git
cd datasets
pip install -e .
python -c "import tensorflow_datasets as tfds; tfds = tfds.load('berkeley_autolab_ur5')"

Link to logs

Traceback (most recent call last):
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\utils\py_utils.py", line 420, in try_reraise
    yield
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 222, in builder
    return cls(**builder_kwargs)  # pytype: disable=not-instantiable
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 288, in decorator
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_builder.py", line 1362, in __init__
    super().__init__(**kwargs)
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 288, in decorator
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_builder.py", line 285, in __init__
    self.info.initialize_from_bucket()
    ^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 168, in __call__
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_builder.py", line 474, in info
    info = self._info()
           ^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\robotics\dataset_importer_builder.py", line 82, in _info
    features = self.get_ds_builder().info.features
               ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\robotics\dataset_importer_builder.py", line 150, in get_ds_builder    
    ds_builder = tfds.builder_from_directory(ds_location)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\read_only_builder.py", line 149, in builder_from_directory       
    return ReadOnlyBuilder(builder_dir=builder_dir)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 288, in decorator
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\read_only_builder.py", line 65, in __init__
    info_proto = dataset_info.read_proto_from_builder_dir(builder_dir)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_info.py", line 1059, in read_proto_from_builder_dir      
    return read_from_json(info_path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\dataset_info.py", line 1035, in read_from_json
    json_str = epath.Path(path).read_text()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\etils\epath\abstract_path.py", line 157, in read_text
    return f.read()
           ^^^^^^^^
  File "C:\Users\Zelong\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\tensorflow\python\lib\io\file_io.py", line 116, in read
    self._preread_check()
  File "C:\Users\Zelong\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\tensorflow\python\lib\io\file_io.py", line 77, in _preread_check
    self._read_buf = _pywrap_file_io.BufferedInputStream(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tensorflow.python.framework.errors_impl.UnimplementedError: File system scheme 'gs' not implemented (file: 'gs://gresearch/robotics/berkeley_autolab_ur5/0.1.0/dataset_info.json')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 168, in __call__
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 643, in load
    dbuilder = _fetch_builder(
               ^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 498, in _fetch_builder
    return builder(name, data_dir=data_dir, try_gcs=try_gcs, **builder_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2288.0_x64__qbz5n2kfra8p0\Lib\contextlib.py", line 81, in inner   
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\logging\__init__.py", line 168, in __call__
    return function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\load.py", line 219, in builder
    with py_utils.try_reraise(
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2288.0_x64__qbz5n2kfra8p0\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\utils\py_utils.py", line 422, in try_reraise
    reraise(e, *args, **kwargs)
  File "C:\Users\Zelong\Desktop\Berkeley data\datasets\tensorflow_datasets\core\utils\py_utils.py", line 389, in reraise
    raise exception from e
RuntimeError: UnimplementedError: Failed to construct dataset "berkeley_autolab_ur5", builder_kwargs "{'data_dir': None}": File system scheme 'gs' not implemented (file: 'gs://gresearch/robotics/berkeley_autolab_ur5/0.1.0/dataset_info.json')

Expected behavior
It should find and download the dataset

@zelongjiang123 zelongjiang123 added the bug Something isn't working label Mar 14, 2024
@fylux
Copy link
Collaborator

fylux commented Mar 18, 2024

Hi, some questions:

  1. Is the command that you posted what triggers the error? Or you have somewhere a command trying to build the dataset "berkeley_autolab_ur5" ?
git clone https://github.com/tensorflow/datasets.git
cd datasets
pip install -e .
python -c "import tensorflow as tf; print('TensorFlow version:', tf.__version__)"
  1. Can you paste the output of pip install -e . ? It could be that you are missing some dependency.

@zelongjiang123
Copy link
Author

Hi, some questions:

  1. Is the command that you posted what triggers the error? Or you have somewhere a command trying to build the dataset "berkeley_autolab_ur5" ?
git clone https://github.com/tensorflow/datasets.git
cd datasets
pip install -e .
python -c "import tensorflow as tf; print('TensorFlow version:', tf.__version__)"
  1. Can you paste the output of pip install -e . ? It could be that you are missing some dependency.

for q1: yes. the command is

python -c "import tensorflow_datasets as tfds; tfds = tfds.load('berkeley_autolab_ur5')"

q2: output is

PS C:\Users\Zelong\Desktop\Berkeley data\datasets> pip install -e .
Obtaining file:///C:/Users/Zelong/Desktop/Berkeley%20data/datasets
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: absl-py in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (1.4.0)
Requirement already satisfied: click in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (8.1.7)
Requirement already satisfied: dm-tree in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (0.1.8)
Requirement already satisfied: etils>=1.6.0 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from etils[enp,epath,epy,etree]>=1.6.0->tensorflow-datasets==4.9.4+nightly) (1.7.0)
Requirement already satisfied: immutabledict in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (4.2.0)
Requirement already satisfied: numpy in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (1.26.4)
Requirement already satisfied: promise in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (2.3)
Requirement already satisfied: protobuf>=3.20 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (3.20.3)
Requirement already satisfied: psutil in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (5.9.8)
Requirement already satisfied: pyarrow in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (15.0.0)
Requirement already satisfied: requests>=2.19.0 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (2.31.0)
Requirement already satisfied: tensorflow-metadata in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (1.14.0)
Requirement already satisfied: termcolor in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (2.4.0)
Requirement already satisfied: toml in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (0.10.2)
Requirement already satisfied: tqdm in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (4.66.2)
Requirement already satisfied: wrapt in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-datasets==4.9.4+nightly) (1.14.1)
Requirement already satisfied: fsspec in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from etils[enp,epath,epy,etree]>=1.6.0->tensorflow-datasets==4.9.4+nightly) (2024.2.0)
Requirement already satisfied: importlib_resources in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from etils[enp,epath,epy,etree]>=1.6.0->tensorflow-datasets==4.9.4+nightly) (6.1.2)
Requirement already satisfied: typing_extensions in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from etils[enp,epath,epy,etree]>=1.6.0->tensorflow-datasets==4.9.4+nightly) (4.10.0)
Requirement already satisfied: zipp in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from etils[enp,epath,epy,etree]>=1.6.0->tensorflow-datasets==4.9.4+nightly) (3.17.0)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from requests>=2.19.0->tensorflow-datasets==4.9.4+nightly) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from requests>=2.19.0->tensorflow-datasets==4.9.4+nightly) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from requests>=2.19.0->tensorflow-datasets==4.9.4+nightly) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from requests>=2.19.0->tensorflow-datasets==4.9.4+nightly) (2024.2.2)
Requirement already satisfied: colorama in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from click->tensorflow-datasets==4.9.4+nightly) (0.4.6)
Requirement already satisfied: six in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from promise->tensorflow-datasets==4.9.4+nightly) (1.16.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in c:\users\zelong\appdata\local\packages\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\localcache\local-packages\python311\site-packages (from tensorflow-metadata->tensorflow-datasets==4.9.4+nightly) (1.62.0)
Building wheels for collected packages: tensorflow-datasets
  Building editable for tensorflow-datasets (pyproject.toml) ... done
  Created wheel for tensorflow-datasets: filename=tensorflow_datasets-4.9.4+nightly-0.editable-py3-none-any.whl size=9609 sha256=779ea94611e63c8c81ea76742f6a7c89a005c2b977260b5bc34307c149a67f81
  Stored in directory: C:\Users\Zelong\AppData\Local\Temp\pip-ephem-wheel-cache-w1kcf9us\wheels\1e\d5\9e\7330eb962857f089b89be7c5eb1f5c911033052a0a1cbfff31
Successfully built tensorflow-datasets
Installing collected packages: tensorflow-datasets
  Attempting uninstall: tensorflow-datasets
    Found existing installation: tensorflow-datasets 4.9.4
    Uninstalling tensorflow-datasets-4.9.4:
      Successfully uninstalled tensorflow-datasets-4.9.4
  WARNING: The script tfds.exe is installed in 'C:\Users\Zelong\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed tensorflow-datasets-4.9.4+nightly

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants