Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Read gin file from google cloud bucket path #175

Open
OhadRubin opened this issue Nov 10, 2022 · 3 comments
Open

Feature request: Read gin file from google cloud bucket path #175

OhadRubin opened this issue Nov 10, 2022 · 3 comments

Comments

@OhadRubin
Copy link

Hey,
I've seen libraries like T5X that are able to read model configs from cloud buckets.
As a product from google I kind of expected this feature to be implemented here and was surprised to find it wasn't.
Implementation plan:

  1. save gin file from remote path into a temporary file
  2. use that as the gin file
@marcospiau
Copy link

Hi @OhadRubin, after some time searching for a solution, I found this code:

config.register_file_reader(tf.io.gfile.GFile, tf.io.gfile.exists)

Using it I was able to read gin files from a GCS bucket. I think that frameworks like T5 and T5x are able to to it because they also import this code.

Best,
Marcos

@OhadRubin
Copy link
Author

I actually found this at some point too, this should really be added to the documentation!

@marcospiau
Copy link

Yes, it really should!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants