Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Getting mirror clone links returns 401 #829

Open
ugrave opened this issue Mar 25, 2024 · 2 comments · May be fixed by #835
Open

Getting mirror clone links returns 401 #829

ugrave opened this issue Mar 25, 2024 · 2 comments · May be fixed by #835
Labels
bug

Comments

@ugrave
Copy link

ugrave commented Mar 25, 2024 •
edited

Jenkins and plugins versions report

Environment 
  Jenkins: 2.440.1
  OS: Linux - 5.10.176-157.645.amzn2.x86_64
  Java: 17.0.10 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
  ---
  Office-365-Connector:4.21.0
  active-directory:2.33
  analysis-model-api:12.1.0
  ansicolor:1.0.4
  antisamy-markup-formatter:162.v0e6ec0fcfcf6
  apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
  asm-api:9.6-3.v2e1fa_b_338cd7
  authentication-tokens:1.53.v1c90fd9191a_b_
  authorize-project:1.7.1
  aws-credentials:218.v1b_e9466ec5da_
  aws-java-sdk:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-api-gateway:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-autoscaling:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-cloudformation:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-cloudfront:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-codebuild:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-codedeploy:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-ec2:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-ecr:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-ecs:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-efs:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-elasticbeanstalk:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-elasticloadbalancingv2:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-iam:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-kinesis:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-lambda:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-logs:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-minimal:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-organizations:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-secretsmanager:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-sns:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-sqs:1.12.671-445.ve02f9b_558f2e
  aws-java-sdk-ssm:1.12.671-445.ve02f9b_558f2e
  aws-secrets-manager-credentials-provider:1.213.vca_3f37306fed
  aws-secrets-manager-secret-source:1.72.v61781b_35c542
  badge:1.9.1
  bootstrap5-api:5.3.3-1
  bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9
  branch-api:2.1152.v6f101e97dd77
  build-symlink:1.1
  build-timeout:1.31
  caffeine-api:3.1.8-133.v17b_1ff2e0599
  checks-api:2.0.2
  cloudbees-bitbucket-branch-source:880.vcf4056c5a_71f
  cloudbees-disk-usage-simple:182.v62ca_0c992a_f3
  cloudbees-folder:6.858.v898218f3609d
  command-launcher:107.v773860566e2e
  commons-lang3-api:3.13.0-62.v7d18e55f51e2
  commons-text-api:1.11.0-95.v22a_d30ee5d36
  config-file-provider:968.ve1ca_eb_913f8c
  configuration-as-code:1775.v810dc950b_514
  credentials:1337.v60b_d7b_c7b_c9f
  credentials-binding:657.v2b_19db_7d6e6d
  custom-build-properties:2.90.v4c63458e3ec8
  custom-tools-plugin:0.8
  customizable-header:50.v04b_6c01e5341
  dark-theme:439.vdef09f81f85e
  data-tables-api:2.0.2-1
  display-url-api:2.200.vb_9327d658781
  docker-commons:439.va_3cb_0a_6a_fb_29
  docker-workflow:572.v950f58993843
  durable-task:550.v0930093c4b_a_6
  ec2-fleet:2.6.0
  echarts-api:5.5.0-1
  email-ext:2.100
  extended-choice-parameter:381.v360a_25ea_017c
  font-awesome-api:6.5.1-3
  forensics-api:2.4.0
  git:5.2.1
  git-client:4.7.0
  git-parameter:0.9.19
  git-server:114.v068a_c7cc2574
  groovy:453.vcdb_a_c5c99890
  groovy-postbuild:228.vcdb_cf7265066
  gson-api:2.10.1-15.v0d99f670e0a_7
  h2-api:11.1.4.199-12.v9f4244395f7a_
  handy-uri-templates-2-api:2.1.8-30.v7e777411b_148
  htmlpublisher:1.33
  http_request:1.18
  instance-identity:185.v303dc7c645f9
  ionicons-api:56.v1b_1c8c49374e
  jackson2-api:2.17.0-379.v02de8ec9f64c
  jakarta-activation-api:2.1.3-1
  jakarta-mail-api:2.1.3-1
  javax-activation-api:1.2.0-6
  javax-mail-api:1.6.2-9
  jaxb:2.3.9-1
  jdk-tool:73.vddf737284550
  jersey2-api:2.41-133.va_03323b_a_1396
  jira:3.13
  job-dsl:1.84
  joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
  jquery3-api:3.7.1-2
  json-api:20240303-41.v94e11e6de726
  json-path-api:2.9.0-58.v62e3e85b_a_655
  junit:1259.v65ffcef24a_88
  mailer:463.vedf8358e006b_
  matrix-auth:3.2
  matrix-project:822.824.v14451b_c0fd42
  mattermost:3.1.3
  metrics:4.2.18-442.v02e107157925
  mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_
  mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_
  monitoring:1.95.0
  okhttp-api:4.11.0-172.vda_da_1feeb_c6e
  pipeline-aws:1.43
  pipeline-build-step:505.v5f0844d8d126
  pipeline-graph-analysis:216.vfd8b_ece330ca_
  pipeline-graph-view:232.vc7ca_8d934725
  pipeline-groovy-lib:704.vc58b_8890a_384
  pipeline-input-step:491.vb_07d21da_1a_fb_
  pipeline-maven:1322.v9ef317a_3e0a_9
  pipeline-milestone-step:111.v449306f708b_7
  pipeline-model-api:2.2184.v0b_358b_953e69
  pipeline-model-definition:2.2184.v0b_358b_953e69
  pipeline-model-extensions:2.2184.v0b_358b_953e69
  pipeline-rest-api:2.34
  pipeline-stage-step:305.ve96d0205c1c6
  pipeline-stage-tags-metadata:2.2184.v0b_358b_953e69
  pipeline-stage-view:2.33
  pipeline-utility-steps:2.16.0
  plain-credentials:179.vc5cb_98f6db_38
  plugin-util-api:4.1.0
  prism-api:1.29.0-13
  prometheus:2.2.3
  rebuild:320.v5a_0933a_e7d61
  resource-disposer:0.23
  role-strategy:689.v731678c3e0eb_
  saferestart:0.7
  scm-api:689.v237b_6d3a_ef7f
  script-security:1326.vdb_c154de8669
  snakeyaml-api:2.2-111.vc6598e30cc65
  sonar:2.17.2
  ssh-agent:333.v878b_53c89511
  ssh-credentials:326.v7fcb_a_ef6194b_
  ssh-slaves:2.948.vb_8050d697fec
  sshd:3.322.v159e91f6a_550
  stashNotifier:1.439.v202358346a_7d
  structs:337.v1b_04ea_4df7c8
  theme-manager:215.vc1ff18d67920
  timestamper:1.26
  token-macro:400.v35420b_922dcb_
  trilead-api:2.142.v748523a_76693
  variant:60.v7290fc0eb_b_cd
  warnings-ng:11.2.2
  workflow-aggregator:596.v8c21c963d92d
  workflow-api:1291.v51fd2a_625da_7
  workflow-basic-steps:1042.ve7b_140c4a_e0c
  workflow-cps:3880.vb_ef4b_5cfd270
  workflow-cps-global-lib:609.vd95673f149b_b
  workflow-durable-task-step:1331.vc8c2fed35334
  workflow-job:1400.v7fd111b_ec82f
  workflow-multibranch:773.vc4fe1378f1d5
  workflow-scm-step:427.v4ca_6512e7df1
  workflow-step-api:657.v03b_e8115821b_
  workflow-support:881.v7663695646cf
  ws-cleanup:0.45
  xvnc:1.24

Bitbucket Version: v8.9.5

What Operating System are you using (both controller, and any agents involved in the problem)?

Amazon Linux for controller and agents: see Enviroment

Reproduction steps

  1. Setup multibranch pipeline with ssh checkout of mirror. Using configured admin accesstoken with repo scope.
  2. Jenkins does not use mirror for checkout. Instead it using the primary server

Expected Results

Jenkins should use the configured mirror

Actual Results

Mirror is not used at all. Fallback to primary server is always used for checkout. In Jenkins log the following is shown:

Could not determine mirror clone links of xxx on https://xxx for org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject@41d28fe3[Project/XXX] falling back to primary server
com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: 401: .
HttpResponseProxy{HTTP/1.1 401  [Set-Cookie: BITBUCKETSESSIONID=XXX,
X-AUSERNAME: access-token-user%2F2%2FXXXX, 
X-ASESSIONID: XXXX,
WWW-Authenticate: OAuth realm="https%3A%2F%2Fmirror-url", ...] org.apache.http.client.entity.DecompressingEntity@6fb61eb7}
	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getRequest(BitbucketServerAPIClient.java:987)
	at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getMirroredRepository(BitbucketServerAPIClient.java:499)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.getCloneLinksFromMirror(BitbucketSCMSource.java:1278)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.initMirrorCloneLinks(BitbucketSCMSource.java:1244)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.initCloneLinks(BitbucketSCMSource.java:1238)
	at com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource.build(BitbucketSCMSource.java:1027)
...

Anything else?

The url which is used in BitbucketServerAPIClient.getMirroredRepository contains already a jwt in the query parameter:
https://MIRROR_URL/rest/mirroring/latest/upstreamServers/XXX/repos/XXX?jwt=TOKEN.
If i used this url for ex with curl https://MIRROR_URL/rest/mirroring/latest/upstreamServers/XXX/repos/XXX?jwt=TOKEN the request is successful.

Are you interested in contributing a fix?

No response
@ugrave ugrave added the bug label Mar 25, 2024
@ugrave
Copy link
Author

ugrave commented Mar 26, 2024

After more testing and debugging i found out the the problematic code is here:

bitbucket-branch-source-plugin/src/main/java/com/cloudbees/jenkins/plugins/bitbucket/server/client/BitbucketServerAPIClient.java

Lines 956 to 962 in 4733e8c

private String getRequest(HttpGet httpget) throws IOException, InterruptedException {
if (authenticator != null) {
authenticator.configureRequest(httpget);
}
try(CloseableHttpClient client = getHttpClient(httpget);

If getRequest is called from getMirroredRepository with the self link which contains already the token as query parameter, the request is failing with 401 if the authenticator (in my case its an instance of BitbucketAccessTokenAuthenticator which adds the Bearer Authorization header) is used. If i remove the header the request is sucessfull. With the header the request is failing with 401.

Also note that the url contains the url of the mirrored bitbucket instance and not the url of the primary bitbucket instance. The token itself is configured for the primary instance.

@ugrave
Copy link
Author

ugrave commented Mar 28, 2024

I checked the api documentation of the endpoint which returns the url of the mirror.
(https://developer.atlassian.com/server/bitbucket/rest/v803/api-group-mirroring/#api-mirroring-latest-repos-repoid-mirrors-get)
It says the url contains already the authorization link to the mirror. The additional authenticator conbfiguration is not needed.

@ugrave ugrave linked a pull request Mar 28, 2024 that will close this issue
Open
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Skip authentication for mirror self urls ugrave/bitbucket-branch-source-plugin
1 participant
@ugrave