Exception: Query 'repo_commit_list' failed to run by returning code of 403: 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'

[SemenMartynov]

Describe the bug
I find it difficult to describe what caused the bug. Apparently, some requests are executed too often, and it is necessary to add a pause between them.

To Reproduce
Just check this log
https://github.com/SemenMartynov/SemenMartynov/actions/runs/4421561355/jobs/7772242992

 Traceback (most recent call last):
  File "/waka-readme-stats/main.py", line 221, in <module>
    run(main())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 208, in main
    stats = await get_stats()
            ^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 156, in get_stats
    yearly_data, commit_data = await calculate_commit_data(repositories)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/yearly_commit_calculator.py", line 37, in calculate_commit_data
    await update_data_with_commit_stats(repo, yearly_data, date_data)
  File "/waka-readme-stats/yearly_commit_calculator.py", line 63, in update_data_with_commit_stats
    commit_data = await DM.get_remote_graphql("repo_commit_list", owner=owner, name=repo_details["name"], branch=branch["name"], id=GHM.USER.node_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 293, in get_remote_graphql
    res = await DownloadManager._fetch_graphql_paginated(query, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 267, in _fetch_graphql_paginated
    initial_query_response = await DownloadManager._fetch_graphql_query(query, **kwargs, pagination="first: 100")
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 231, in _fetch_graphql_query
    raise Exception(f"Query '{query}' failed to run by returning code of {res.status_code}: {res.json()}")
Exception: Query 'repo_commit_list' failed to run by returning code of 403: {'documentation_url': 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits', 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'}
sys:1: RuntimeWarning: coroutine 'AsyncClient.get' was never awaited
##[debug]Docker Action run completed with exit code 1

Expected behavior
To make everything work as it used to.)

Github repository link
If applicable, please share the Github repository link.

[aravindvnair99]

@SemenMartynov it looks like your workflow has been failing for more than month for various reasons. However, the most recent executions are related to the error you're pointing out.

Could you confirm if your GitHub token is unique and not being used elsewhere? Could you regenerate the token and keep it unique to your repository for this workflow?

[ssoad]

I have same issue

Traceback (most recent call last):
  File "/waka-readme-stats/main.py", line 221, in <module>
    run(main())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 208, in main
    stats = await get_stats()
            ^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 156, in get_stats
    yearly_data, commit_data = await calculate_commit_data(repositories)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/yearly_commit_calculator.py", line 37, in calculate_commit_data
    await update_data_with_commit_stats(repo, yearly_data, date_data)
  File "/waka-readme-stats/yearly_commit_calculator.py", line 63, in update_data_with_commit_stats
    commit_data = await DM.get_remote_graphql("repo_commit_list", owner=owner, name=repo_details["name"], branch=branch["name"], id=GHM.USER.node_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line [29](https://github.com/ssoad/ssoad/actions/runs/4432355991/jobs/7780126238#step:3:30)3, in get_remote_graphql
    res = await DownloadManager._fetch_graphql_paginated(query, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 271, in _fetch_graphql_paginated
    query_response = await DownloadManager._fetch_graphql_query(query, **kwargs, pagination=pagination)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 2[31](https://github.com/ssoad/ssoad/actions/runs/4432355991/jobs/7780126238#step:3:32), in _fetch_graphql_query
    raise Exception(f"Query '{query}' failed to run by returning code of {res.status_code}: {res.json()}")
Exception: Query 'repo_commit_list' failed to run by returning code of [40](https://github.com/ssoad/ssoad/actions/runs/4432355991/jobs/7780126238#step:3:41)3: {'documentation_url': 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits', 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'}
sys:1: RuntimeWarning: coroutine 'AsyncClient.get' was never awaited

What's the problem ? @anmol098

[SemenMartynov]

The key was definitely unique and was not used anywhere else. But I re-generated it anyway, and in a strange way it turned out to be a solution.
https://github.com/SemenMartynov/SemenMartynov/actions/runs/4432343645/jobs/7780279301

@ssoad please try to do the same.

[aravindvnair99]

Thanks for confirming @SemenMartynov!

I'll keep this open for @ssoad also to confirm.

[ssoad]

@SemenMartynov Thanks. It works @aravindvnair99 Now you can close this issue

[aravindvnair99]

Thanks for confirming @ssoad!

[SemenMartynov]

@aravindvnair99 failed again =(
https://github.com/SemenMartynov/SemenMartynov/actions/runs/4458107908/jobs/7829634589

[pseusys]

Actually before the action waited for 0.3 seconds between the requests. Apparently if we return this wait, the execution time will skyrocket.
But we can try if there's no other option.

[anmol098]

We need to add a interim wait for between the subsequent api calls. Execution time I think should not affect anyone Its like a cron and it should take its own time to complete.
For debug run we can probably remove wait time

[SemenMartynov]

Today I see another mistake.
I don't know if this is the result of trying to fix the original problem, or if it's some new problem.
https://github.com/SemenMartynov/SemenMartynov/actions/runs/4463566749/jobs/7838909571

[SemenMartynov]

Looks good, thx!
https://github.com/SemenMartynov/SemenMartynov/actions/runs/4485188763/jobs/7891968341

[SemenMartynov]

Hi guys,

I'm sorry, but I'm getting this error from github again.
Perhaps it is worth increasing the pause time, or making it based on Fibonacci numbers or something else...
https://github.com/SemenMartynov/SemenMartynov/actions/runs/4642416372/jobs/8216297418

[theCapypara]

Hi! Just wanting to add to this, that I also run into this rate limit for months now:
https://github.com/theCapypara/theCapypara/actions/runs/5056669031/jobs/9074422155
There probably needs to be some kind of backoff implemented.

[mrepol742]

I also have the same problem. Getting the new apikey from wakatime fixed the issue..

[mikebronner]

I'm receiving this as well, getting new API key and Github Token did not help.

Run anmol098/waka-readme-stats@master
/usr/bin/docker run --name wakareadmestatswakareadmestatsmaster_a59d0b --label 16c5e9 --workdir /github/workspace --rm -e "INPUT_WAKATIME_API_KEY" -e "INPUT_GH_TOKEN" -e "INPUT_SHOW_PROJECTS" -e "INPUT_SECTION_NAME" -e "INPUT_PULL_BRANCH_NAME" -e "INPUT_PUSH_BRANCH_NAME" -e "INPUT_SHOW_OS" -e "INPUT_SHOW_EDITORS" -e "INPUT_SHOW_TIMEZONE" -e "INPUT_SHOW_COMMIT" -e "INPUT_SHOW_LANGUAGE" -e "INPUT_SHOW_LINES_OF_CODE" -e "INPUT_SHOW_LANGUAGE_PER_REPO" -e "INPUT_SHOW_LOC_CHART" -e "INPUT_SHOW_DAYS_OF_WEEK" -e "INPUT_SHOW_PROFILE_VIEWS" -e "INPUT_SHOW_SHORT_INFO" -e "INPUT_SHOW_UPDATED_DATE" -e "INPUT_SHOW_TOTAL_CODE_TIME" -e "INPUT_COMMIT_BY_ME" -e "INPUT_COMMIT_MESSAGE" -e "INPUT_COMMIT_USERNAME" -e "INPUT_COMMIT_EMAIL" -e "INPUT_COMMIT_SINGLE" -e "INPUT_LOCALE" -e "INPUT_UPDATED_DATE_FORMAT" -e "INPUT_IGNORED_REPOS" -e "INPUT_SYMBOL_VERSION" -e "INPUT_DEBUG_LOGGING" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/mikebronner/mikebronner":"/github/workspace" wakareadmestats/waka-readme-stats:master
Traceback (most recent call last):
  File "/waka-readme-stats/main.py", line 230, in <module>
    run(main())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 217, in main
    stats = await get_stats()
            ^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 162, in get_stats
    yearly_data, commit_data = await calculate_commit_data(repositories)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/yearly_commit_calculator.py", line 37, in calculate_commit_data
    await update_data_with_commit_stats(repo, yearly_data, date_data)
  File "/waka-readme-stats/yearly_commit_calculator.py", line 63, in update_data_with_commit_stats
    commit_data = await DM.get_remote_graphql("repo_commit_list", owner=owner, name=repo_details["name"], branch=branch["name"], id=GHM.USER.node_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 297, in get_remote_graphql
    res = await DownloadManager._fetch_graphql_paginated(query, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 277, in _fetch_graphql_paginated
    query_response = await DownloadManager._fetch_graphql_query(query, **kwargs, pagination=pagination)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 235, in _fetch_graphql_query
    return await DownloadManager._fetch_graphql_query(query, retries_count - 1, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 237, in _fetch_graphql_query
    raise Exception(f"Query '{query}' failed to run by returning code of {res.status_code}: {res.json()}")
Exception: Query 'repo_commit_list' failed to run by returning code of 403: {'documentation_url': 'https://docs.github.com/free-pro-team@latest/rest/overview/rate-limits-for-the-rest-api#about-secondary-rate-limits', 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again. If you reach out to GitHub Support for help, please include the request ID 1D00:32CE1E:5DE5724:93DC5E8:66084C06.'}
sys:1: RuntimeWarning: coroutine 'AsyncClient.get' was never awaited