Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable S3 Anonymous Mode #2274

Open
jaychia opened this issue Apr 29, 2024 · 3 comments
Open

Enable S3 Anonymous Mode #2274

jaychia opened this issue Apr 29, 2024 · 3 comments

Comments

@jaychia
Copy link

jaychia commented Apr 29, 2024

This would allow for access to publicly available datasets without valid S3 credentials: handy for public demos hosted on Google Colab!

I did not see anything available here: https://lancedb.github.io/lance/read_and_write.html#s3-configuration

@jaychia
Copy link
Author

jaychia commented Apr 29, 2024

Note also that the object_store crate might already support this functionality but under a different key name: https://docs.rs/object_store/latest/object_store/aws/enum.AmazonS3ConfigKey.html#variant.SkipSignature

@jaychia
Copy link
Author

jaychia commented Apr 30, 2024

Actually, just verified that the skip_signature: true flag works! We should add this to the lance documentation though.

@westonpace
Copy link
Contributor

@jaychia do you want to make a quick PR (actually, 2 PRs probably)? I think https://lancedb.github.io/lance/read_and_write.html#s3-configuration would be the right spot for Lance and https://lancedb.github.io/lancedb/guides/storage/#aws-s3 would be the right spot for it for lancedb. In the Lance docs we should just add it to the list of available options. In the LanceDb docs it might be good to have a small section / paragraph titled "publicly available buckets" that explains that the skip signature flag is required.

The source for those docs is https://github.com/lancedb/lance/blob/main/docs/read_and_write.rst and https://github.com/lancedb/lancedb/blob/main/docs/src/guides/storage.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants