Feature: GitHub Integration

[vaishnavigvs]

This is a discussion thread for feature feedback in terms of user experience, enhancements or anything that you expected/want to see in this feature. Please feel free to reach out to us here and discuss within the community too!

[seanaguinaga]

I turned this feature on the moment I saw it available. It’s working flawlessly.

This has removed so much stress from my deploys. And was super easy to configure.

I know your docs recommend squashing, but since this is automatic, do you think it’s fine that it doesn’t squash migrations?

I’ve moved from dev -> staging -> prod already and it was the easiest thing I’ve done with hasura so far!

I’d be interested to know what happens when I revert a commit?

[vaishnavigvs]

Hey Sean! So glad to see you try out dev to prod all the way!

It is okay if the migrations aren't squashed too as they get applied sequentially. We recommend squashing as it becomes easier to keep track, when the number of migrations become very high.

As for reverting a commit, we rather recommend applying the verified relevant down migrations using the CLI, commit and redeploying that, as it isn't trivial to revert migrations on a db automatically.

[KayakinKoder]

This is what we've been waiting for :) There are only two issues/thoughts on our mind currently:

1.) It sounds like you're working on a cloud version of the docker image, so that we can develop cloud features locally. Once that's ready, I think we'll have all of the pieces needed for a great local -> production CI/CD flow

2.) One thing that's still unclear, and we're just in the early stages of things, is how to handle the initial setup and migration. Meaning, we have an existing database, and we set up Hasura locally/create the initial migration files locally. Then we push to say master branch, which triggers our Hasura cloud project...how can we be 100% sure that this isn't going to make any changes to our production database. The steps here https://github.com/hasura/github-integration-starter "Moving from Hasura Cloud to local dev", noting the skip-execution flag are clear; but if we set up our project locally first and push to cloud, the steps aren't entirely clear how our initial push will "skip execution"/not affect our production database

[rikinsk]

This depends on the state of your production db at the time of initializing migrations. If your production db is in sync with the local db at the time of initializing migrations, you will need to mark the init migration as applied on the cloud project as well. in this case you should run hasura migrate apply --skip-execution --endpoint <cloud-endpoint> as well so that cloud is aware that it doesnt need to run this init migration on the production db. On the other hand if your production db is blank, then you just need to run migrations on it to bring it upto speed with your local db. Hope this clarifies the process.

[ledburyb]

I'd like to use this feature, however currently we're generating a single metadata JSON file with some custom Python code and applying it using a replace_metadata API call. Is there any way to make this workflow work and still use this for our CI/CD?

[rikinsk]

This is something we don't support as of now. But it is a valid use case and we will add support for this as well

[ledburyb]

Is there a reason you apply the metadata and then run the migrations? What if I have a new feature for which I'm adding a database table and tracking it with Hasura at the same time? Wouldn't applying the metadata then fail?

[rikinsk]

@ledburyb this is due to the introduction of multiple databases in Hasura v2.0. As the list of databases to be connected to is now present in the metadata, it needs to be applied first so that Hasura knows to which databases the migrations need to be applied to.

When you have tables in your metadata which do not currently exist, those tables get marked as inconsistent after the initial metadata apply. After the migrations are run, this is followed by a metadata reload which would then mark the tables as consistent again.

Hope this helps clarify the sequence of steps.

[hansy]

@rikinsk This approach makes sense, but it doesn't seem to be working for me. The "Apply Metadata" step routinely fails for me with the Inconsistent object: no such table/view exists in source error whenever I introduce a new table.

I have to manually apply the migrations first (via hasura migrate apply), then apply/reload metadata.

[lughino]

The idea is very good.
However, I cannot see a real use case for this feature as-is.
It's impossible to work with a fixed "staging" branch to attach the automated git integration and benefit from the CI.
In almost any git workflow, like git-flow or trunk-based development, you create a feature branch that exists only for the sake of the feature. It has nothing to do with the environment available.
Ideally, I should be able to create any feature branch and deploy to the various environment, according to the promotion steps I have in my workflow.
E.g. (removing all the necessary steps not relevant for the example):
Push to a new development branch => auto-deploy to staging => run product reviews, etc. => accept PR and merge to main => auto-deploy to canary => manual deploy to prod

This is obviously just a simple example, the ways and flavors are limitless, but this integration adds so many limitations that it's not possible to use it in a real-world scenario.

From what I see, an official Github action that gives the power to launch all the commands to achieve the CI/CD in any custom way would be much more useful and flexible.

[gaboesquivel]

For some reason it's the github integration deployment is trying to run all migrations

[seanholahan]

@gaboesquivel did you ever find a solution for this?

[seanholahan]

this works

hasura migrate apply --admin-secret “admin secret” --skip-execution --version “version”—endpoint “endpoint” --database-name default

[RoryO]

Github deploy overwrites SSL configuration upon deploy.

When working with hasura locally, there are no SSL options, which is fine because dealing with SSL locally is massively painful

However, upon deploying, because the committed metadata does not have SSL configuration the SSL configuration is lost and the deploy fails, also breaking the connection to the DB

You can reconnect to the DB by re-configuring the SSL settings pointing to the correct env vars, but as of now there's no way to use this feature when the production connection has SSL configuration.

[RoryO]

I figured out a work around. If you export the metadata from the cloud project, you'll get a diff in the database.yml which contains the SSL configuration

      ssl_configuration:
        sslcert:
          from_env: SSL_CERT
        sslkey:
          from_env: SSL_KEY
        sslmode: verify-ca
        sslpassword: null
        sslrootcert:
          from_env: SSL_ROOT_CERT

If you apply the exported metadata locally with the ssl configuration, the local hasura instance applies and ignores the configuration.

[seanaguinaga]

Context Issue

context deadline exceeded (Client.Timeout or context cancellation while reading body)

What does this mean?

[lucas-janon]

Hi, I'm getting the following error:

Error finding the hasura project with metadata and migrations at the given repo/dir/branch. Please make sure that the given repo is accessible and the directory is a valid Hasura project.

Fetching metadata & migrations (prev. step) worked, and the "View Metadata" link works too (sends me to the /metadata folder of my Hasura project).

My setup is a monorepo and I specified the folder containing the project in the "directory" config.

[meetzaveri]

hey @lucas-janon , we would encourage you to create a support ticket for this via hasura cloud.

[shutupwritedev]

Sometimes a deployment fails and I need to push up a new commit to compensate. (Happens to all of us I'm sure. ) However, it takes upwards of 15 minutes for Hasura Cloud to be aware of the late Github commit. This is quite a long time. Is there a way to reduce this time or force an update?

[mike-4040]

I am using Hasura as a GraphQL router, without a database connection.
No problem setting this up in UI, either Cloud or local. All works fine.
But when I am trying to deploy changes via GitHub integrations deployment consistently fails with

github-integration failed to complete because error parsing migrate status: 0 connected sources found on hasura

I am guessing, that the integration assumes that Hasura doesn't work without data sources and treats metadata/databases/databases.yaml with only empty [] in it as an error.

1. Am I correct?
2. Any workaround?