422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs for coveralls==4.0.0

[abravalheri]

Describe the Bug
The last builds of validate-pyproject started failing when running coveralls --submit with the following error:

Error running coveralls: Could not submit coverage: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs
Traceback (most recent call last):
  File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/coveralls/api.py", line 298, in submit_report
    response.raise_for_status()
  File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/coveralls/cli.py", line 88, in main
    coverallz.submit_report(report_file.read())
  File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/coveralls/api.py", line 301, in submit_report
    raise CoverallsException(
coveralls.exception.CoverallsException: Could not submit coverage: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs

Things to notice: the build runs on Cirrus CI, so I am using the configuration as described in https://coveralls-python.readthedocs.io/en/latest/usage/configuration.html#other-ci-systems (and it has been working so far until recently).

To Reproduce
This is a script that tries to reproduce locally what we verify in https://cirrus-ci.com/task/6209238735781888:

> docker run --rm -it python:3.12-bookworm /bin/bash

git clone https://github.com/abravalheri/validate-pyproject /tmp/validate-pyproject
cd /tmp/validate-pyproject
python3 -m venv .venv
.venv/bin/python -VV
# Python 3.12.3 (main, Apr 24 2024, 11:17:35) [GCC 12.2.0]

.venv/bin/python -c 'import os, sys; print(os.name, sys.platform, getattr(sys, "abiflags", None))'
# posix linux

.venv/bin/python -m pip install -U pip
.venv/bin/python -m pip -VV
# pip 24.0 from /tmp/validate-pyproject/.venv/lib/python3.12/site-packages/pip (python 3.12)

.venv/bin/python -m pip install -U tox 'coveralls==4.0.0' coverage
.venv/bin/python -m pip list
# Package            Version
# ------------------ --------
# cachetools         5.3.3
# certifi            2024.2.2
# chardet            5.2.0
# charset-normalizer 3.3.2
# colorama           0.4.6
# coverage           7.4.4
# coveralls          4.0.0
# distlib            0.3.8
# docopt             0.6.2
# filelock           3.14.0
# idna               3.7
# packaging          24.0
# pip                24.0
# platformdirs       4.2.1
# pluggy             1.5.0
# pyproject-api      1.6.1
# requests           2.31.0
# tox                4.15.0
# urllib3            2.2.1
# virtualenv         20.26.1
.venv/bin/tox -- -n 5 --randomly-seed=42 -rfEx --durations 10 --color yes

export COVERALLS_REPO_TOKEN=<redacted>
export CI_NAME=local-test
export CI_BRANCH=pre-commit-ci-update-config
export CI_PULL_REQUEST=168
export CI_BUILD_NUMBER=1
export CI_BUILD_URL=https://127.0.0.1:8888/build/${CI_BUILD_NUMBER}

.venv/bin/coverage xml -o coverage.xml
.venv/bin/coveralls --verbose --submit coverage.xml
# Missing .coveralls.yml file. Using only env variables.
# Error running coveralls: Could not submit coverage: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs
# Traceback (most recent call last):
#   File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/coveralls/api.py", line 298, in submit_report
#     response.raise_for_status()
#   File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/requests/models.py", line 1021, in raise_for_status
#     raise HTTPError(http_error_msg, response=self)
# requests.exceptions.HTTPError: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs
#
# The above exception was the direct cause of the following exception:
#
# Traceback (most recent call last):
#   File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/coveralls/cli.py", line 88, in main
#     coverallz.submit_report(report_file.read())
#   File "/tmp/validate-pyproject/.venv/lib/python3.12/site-packages/coveralls/api.py", line 301, in submit_report
#     raise CoverallsException(
# coveralls.exception.CoverallsException: Could not submit coverage: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs

Expected Behavior
There should be no HTTP/API errors when submitting coverage reports.

Package Versions

coverage           7.4.4
coveralls          4.0.0

Pytest plugins:

cov-5.0.0, xdist-3.6.1, randomly-3.15.0

Packages inside of the tox environment:

.tox/default/bin/python -m pip list
Package            Version
------------------ -------------------------
coverage           7.5.1
execnet            2.1.1
fastjsonschema     2.19.1
iniconfig          2.0.0
markdown-it-py     3.0.0
mdurl              0.1.2
packaging          24.0
pip                24.0
pluggy             1.5.0
pytest             8.2.0
pytest-cov         5.0.0
pytest-randomly    3.15.0
pytest-xdist       3.6.1
PyYAML             6.0.1
repo_review        0.11.0
setuptools         69.5.1
trove-classifiers  2024.4.10

Trace Logs
Provide coverage logs w/ DEBUG logging enabled (run coverage debug ...)

I am running pytest-cov instead of coverage.

[abravalheri]

Seems to work on coveralls<4, so maybe a regression?

(See comparison of builds: https://cirrus-ci.com/build/5998598171656192 fails for coveralls==4, https://cirrus-ci.com/build/5512918874193920 succeeds for coveralls<4, code diff: https://github.com/abravalheri/validate-pyproject/pull/169/files)

[TheKevJames]

Thank you for the detailed bug report! Very much appreciated.

I'm thinking this relates to us removing the "retry on 422" code (#427), which had previously been added to help specifically with Github Actions, but also ran for other services with some hackish ID incrementing logic.

Basically, for your use-case, it would have caught the 422, then manually re-set the service_job_id to a new random value and tried again.

• Could you please verify CI_JOB_ID (and the various other methods of setting this, such as COVERALLS_SERVICE_JOB_ID) is unset in your use-case?
• Could you set CI_JOB_ID explicitly to any value and see if that resolves it?

It seems like perhaps my docs are incorrect and this value is not optional, but we may have never noticed based on the retry logic.

[abravalheri]

Thank you very much @TheKevJames for having a look on this.

Could you please verify CI_JOB_ID (and the various other methods of setting this, such as COVERALLS_SERVICE_JOB_ID) is unset in your use-case?

I believe that CI_JOB_ID is left unset. I don't set it in the CI file, and Cirrus CI also doesn't set it automatically.

Could you set CI_JOB_ID explicitly to any value and see if that resolves it?

There we go: https://github.com/abravalheri/validate-pyproject/pull/171/files ¹. Still failing though: https://cirrus-ci.com/task/5660226890760192 with 422 Client Error: Unprocessable Entity for url.

---

This seems to be what the coveralls docs have to say about CI_JOB_ID:

CI_JOB_ID (optional)
# For parallel builds, the number assigned to each job comprising the build. When missing, Coveralls will assign an incrementing integer (1, 2, 3 ...). This value should not change between multiple runs of the build.

So I guess their API should also consider the job id optional?

Footnotes

1. I did set the env var to the value of CIRRUS_TASK_ID - described as "Unique task ID". This is what it looks like: https://api.cirrus-ci.com/v1/task/5048508087009280/logs/debug_coveralls.log. however I don't think I can make it a number with the requirement that does not change between multiple runs of the same build. ↩