Add Buildkite OIDC support

[sj26]

Add support for Buildkite OIDC tokens.

This doesn't yet add tests, but should be functional. I'm testing end-to-end using a local development copy of Buildkite.

I can see some good end-to-end tests, eg. test/system/oidc_test.rb, but unsure if you'd like those per issuer, or some other issuer-specific testing. Please let me know what coverage you would like 🙏

[codecov]

Codecov Report

Merging #4159 (37add97) into master (66c6183) will decrease coverage by 0.03%.
The diff coverage is 60.00%.

@@            Coverage Diff             @@
##           master    #4159      +/-   ##
==========================================
- Coverage   98.73%   98.70%   -0.03%     
==========================================
  Files         302      302              
  Lines        6852     6857       +5     
==========================================
+ Hits         6765     6768       +3     
- Misses         87       89       +2     

Files	Coverage Δ
app/models/oidc/provider.rb	93.93% <60.00%> (-6.07%)	⬇️

[sj26]

I managed to exercise this end-to-end using a pipeline like:

steps:
- key: publish
  env:
    GEM_HOST: http://localhost:3000
    TOKEN: ...
    AUDIENCE: http://localhost:3000
  command: |
    set -euo pipefail
    
    echo "--- Fetch a gem"
    gem fetch rails
    
    echo "--- Request OIDC token"
    export BUILDKITE_OIDC_TOKEN="$$(buildkite-agent oidc request-token --audience "$${AUDIENCE}" --lifetime 60)"
    
    echo "--- Exchange for Rubygems token"
    export GEM_HOST_API_KEY="$$(ruby -r net/http -r json <<RUBY
      gem_host = ENV.fetch("GEM_HOST", "https://rubygems.org")
      token = ENV.fetch("TOKEN")
      uri = URI.parse("#{gem_host}/api/v1/oidc/api_key_roles/#{token}/assume_role")
      oidc_token = ENV.fetch("BUILDKITE_OIDC_TOKEN")
      data = JSON.dump({"jwt" => oidc_token})
      headers = {"content-type" => "application/json"}
      response = Net::HTTP.post(uri, data, headers)
      response.value
      value = JSON.parse(response.body)
      print value.fetch("rubygems_api_key")
    RUBY
    )"
    
    echo "--- Push gem"
    gem push --host "$${GEM_HOST}" *.gem

and the seeds in this PR, but with agent.buildkite.localhost which is my local copy of Buildkite:

[sj26]

I wrapped that big ruby command up into a Buildkite plugin:

https://github.com/sj26/rubygems-oidc-buildkite-plugin

which means the pipeline can now look like this -- although I fudged it to download an existing gem and use my local environments in the screenshot example:

steps:
- key: gem-publish
  plugins:
  - sj26/rubygems-oidc:
      token: rg_oidc_akr_...
  command: |
    puts "--- Build gem"
    gem build *.gem
    
    echo "--- Push gem"
    gem push *.gem

[sj26]

I took a look at the UI support specific to GitHub Actions, and it looks like it's just the instructions here (this is local dev, these tokens aren't sensitive):

I don't think adding those is a blocker to landing this higher level support for the issuer.

There does also seem to be support for showing OIDC id tokens, but I couldn't find anything github specific in there, and it renders a Buildkite OIDC token fine:

[segiddins]

was able to add the provider to https://oidc-api-token.rubygems.org/profile/oidc/providers/2 with no code changes needed

[sj26]

@segiddins wonderful!

What sort of testing and coverage would you like here? How can I help land support into rubygems.org? 🙏

[sj26]

Hi folks!

I'd love to land this into rubygems.org, and start using it for our gem publishing flows for things like rspec-buildkite and buildkite-test_collector.

How can I help?

[sj26]

Would it be helpful to add gem push instructions into the app, like for GitHub? I can wrap up the instructions I posted earlier:

#4159 (comment)

[segiddins]

https://rubygems.org/profile/oidc/providers/2

[sj26]

Amazing! Looks like it's all mapped well.

How can I try that out? It doesn't look like I can make roles for OIDC to assume on rubygems.org at the moment. If it's a feature flag, could I get it on for sj26 and buildkite, so we can test?

[segiddins]

You should be able to from https://rubygems.org/profile/oidc/api_key_roles

[sj26]

Oh nice! I just couldn't find the link. I'll give it a go and report back.

[simi]

Oh nice! I just couldn't find the link. I'll give it a go and report back.

Any news @sj26?