Snowflake Spend dbt Package

This is a dbt package for understanding the cost your Snowflake Data Warehouse is accruing.

To get started with this package, you will need to have access to the appropriate databases.

To grant appropriate roles for these tables the following command was run:

GRANT IMPORTED PRIVILEGES ON DATABASE snowflake TO ROLE <role>;

Learn more about the appropriate permissions.

dbt has great docs on package management. We are working to get this package on the dbt hub site. In the mean time, you can install it using the git package syntax, which the GitLab data team uses in our packages.yml file

packages:
  - git: https://gitlab.com/gitlab-data/snowflake_spend.git
    revision: v1.2.0

You will need to update your dbt_project.yml to enable this package. You can see how the GitLab data team has this configured.

snowflake_spend:
  enabled: true

You will need a csv called snowflake_contract_rates.csv which has two columns: effective date and rate. The effective date is the day the new contracted rate started and it should be in YYYY-MM-DD format. The rate is the per credit price for the given time period. You can see how the data team configures their csv file. You will need to run dbt seed for the csv to be loaded as a table and for the model to run succesfully.

These models are documented and tested. If you'd like to see what these look like live, you can see them in the GitLab Data Team's public dbt docs.

Maintainers of this projects are @m_walker and @jjstark. Reviewers are @mpeychet_.

We include sample Sisense (formerly Periscope) dashboards in the /analytics folder.

This dbt package is made available by the GitLab Data Team under an MIT License.

Troubleshooting

Error: Found duplicate project dbt_utils. This occurs when a dependency has the same project name as some other dependency.

You are most likely referencing dbt-utils using the git/revision syntax. Use the dbt Hub package syntax instead in your packages.yml, eg:

packages:

  # Avoid this
  - git: "https://github.com/fishtown-analytics/dbt-utils.git"
    revision: 0.2.1

  # Use this instead
  - package: fishtown-analytics/dbt_utils
    version: 0.2.1

How this Package Gets Released -- For Maintainers Only

In order to cut a new release of this package:

1. Create a new tag at the commit that is to be released. Incrementing either the major version, minor version, or bug version of the previously released tag. The new tag name should follow the pattern v#.#.#. Push the tag to origin/master, if the tag is created locally.

2. With a GitLab API-enabled private token create a new release with a command similar to this:

   curl --header 'Content-Type: application/json' --header "PRIVATE-TOKEN: <your-private-token>" \
   --data '{ "name": "dbt: snowflake_spend #.#.#", "tag_name": "v#.#.#", "description": "Initial tagged release"}' \
   --request POST https://gitlab.com/api/v4/projects/12955687/releases
   

3. Update the release notes by going to your tag and click on the "edit release notes" pencil. The release notes should follow the same general form as the notes for v1.1.0 and include a bulleted summary of merged MRs and a thank you to contributors.

4. Create a blog post like this with all of the details of the release.