Skip to content

Notes about implementing Snowflake for custom drivers and tools.

License

Notifications You must be signed in to change notification settings

joshuataylor/snowflake-notes

Repository files navigation

Snowflake Notes

This repository aims to document and share insights into how Snowflake works under the hood, hopefully helping de-mystify some of Snowflake's undocumented internals.

Initially, there is a focus on documenting "internal" API Endpoints that drivers and Snowsight uses, to help share undocumented information/nuances that isn't publicly accessible.

These notes aim to be your companion when working with these API endpoints, with the intention of unlocking use cases such as:

  • Creating a driver for an unsupported language
  • Accessing Snowsights APIs to get Worksheet/Dashboard information
  • Using internal APIs for query information/query profiler information and utilise features not yet accessible via SQL.

I hope more awesome tools are created as a result of these, and a more collaborative discussion can be had.

About the Author

👋, I'm Josh Taylor, I am a Senior Data Engineer at Sendle, having worked with the Data Team using Snowflake since 2020, transitioning from being a Software Engineer since 2008.

I have really enjoyed digging into Snowflakes internals, such as creating drivers for Elixir/Rust, understanding how things work "under the hood" has been quite fun.

I hope you find these notes useful, and they help with any research you are doing as you explore Snowflakes internals, and hope you share any notes, comments and other insights you have found when dealing with Snowflakes "nuances".

Note

If you have any questions, feel free to email me (link on my GitHub profile) or start a discussion.

  • If you wish to contribute, submit feedback, etc - please create an issue or a Pull Request.

Important

Endpoints that drivers use are pretty stable, but might change at anytime.

  • If using an internal API (ie something from the Classic Console/Snowsight), Snowflake might also not be too happy if you start doing bulk requests to certain endpoints. These responses might also change at anytime. So play nicely.

Tip

If you are just wanting to do basic queries etc with Snowflake, try using the official drivers as you'll have official support from Snowflake.

Notes - Table of Contents

Outlines how to inspect requests to the Snowflake API, and how to setup a Reverse Proxy to inspect traffic to Snowsight.

Provides documentation for how connectors authenticate to Snowflake, and perform queries, monitor queries, etc. Has cURL examples and example requests/responses.

Snowsight

Covers how to authenticate to Snowsight, and perform queries to retrieve worksheets. Also see an example Python 3.x implementation - snowsight_basic.py.

Other Resources

There have been some great posts/resources from the community which cover the internals and other tips when using Snowflake.

If you have come across resources relating to Snowflake internals, please share!

Tip

The select.dev blog is fantastic, some of my favourites:

Snowflake Tools

Snowflake has open-sourced various tools which might be of interest:

We are aware of changes to the Snowsight UI that will eventually cause breaking changes to snowsightextensions. In short, snowsightextensions is reaching end of life.

License

This repository is licensed under the Apache 2.0 license.

This repository is NOT affiliated with Snowflake, nor endorsed by Snowflake in any way, shape, or form. This repository does not intend to infringe on any trademarks or copyrights that belong to Snowflake, it's intended to aid curiosity

These notes, opinions, thoughts, and other ramblings are my own (unless contributed by others, thanks!), and have been created in my personal time - with the intention of helping others learn and create interesting tools - they do not reflect the opinions of current/past employers nor Snowflake.

Where possible, I have tried to follow the Terms of Service and the Community Terms of Service. Snowflake Code of Business Conduct and Ethics and Partner Content Guidelines have been referenced where relevant as well.

This repository references the following projects:

About

Notes about implementing Snowflake for custom drivers and tools.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages