Use Coveralls Github Action to send data to coveralls again

[AdamWill]

This was lost when Travis stopped working, so our coveralls data is frozen in 2021. This should get it working again using https://github.com/marketplace/actions/coveralls-github-action if I got it right.

[AdamWill]

hum, we don't really want to do it seven times per PR. I'll have to tweak this a bit.

[AdamWill]

whew, well that syntax is surprisingly finicky - the quotes have to be single, YAML thing I guess? - but it works now. we only upload coverage data from the 3.11 route. we should bump that to be the current stable version periodically.

[AdamWill]

The change to run the workflow on push as well as pull request I think is necessary for the badge coverage number to be updated (it seems to only be updated when coveralls calculates coverage of the master branch, not when it calculates coverage for a PR).

I'm trying to get the thing where coveralls posts a comment on the PR to work, but it doesn't seem to be yet. I need to check a setting on coveralls.io but I don't have access to the project settings there, something to do with permissions...trying to figure it out...

[waldyrious]

I'm trying to get the thing where coveralls posts a comment on the PR to work, but it doesn't seem to be yet. I need to check a setting on coveralls.io but I don't have access to the project settings there, something to do with permissions...trying to figure it out...

@AdamWill I got a "Third-party application approval request for mwclient" email from GitHub asking to grant Coveralls access to the repository, and have accepted it. Hopefully that unblocks the issue :)

In any case, I've made you an owner in the mwclient GitHub organization, so I suppose you should now have the same access levels as I do. In particular, you should be able to access the application access configurations at https://github.com/organizations/mwclient/settings/oauth_application_policy, and should probably be able to access https://coveralls.io/github/mwclient/mwclient/settings as well. Please confirm!

[AdamWill]

Indeed I can, but I've set all the things that I can find right, and coveralls still isn't commenting. I've sent a mail to coveralls support, maybe they can help. :D

[AdamWill]

So Coveralls support got back with a very detailed reply:

"Apologies for the delay.

It sounds like you're all set up to receive PR Comments, so, as I suspected, I see the issue has a different cause, which is that Coveralls can't find the base build for your PR build, which it needs to derive a coverage change calculation, which is the content of any PR Comment it would send. (We don't do a great job of letting you know we couldn't send one, right now. We are working on that.)

The clue that this is the case, is that your PR Build reads "FIRST BUILD" in the Build Info table at the top of its Build Page. This sometimes happens when users are getting set up: They've added all the Coveralls configuration details to their PR, but they haven't merged those changes into their default branch yet. Since the base build of a PR build will almost always be a push build on your default branch, there is no base build for that new PR yet.

That's usually fixed by merging the PR changes into your default branch, at which point, we start getting coverage reports for your push builds on master, and, so, your next PR build(s) have base builds and can receive PR Comments.

I'll leave it there since I'm almost certain that the changes related to switching from Travis-to-GitHub-Actions have created this situation for you, but if for some reason, you take those next steps and still don't receive PR Comments, please let me know and we'll take it from there."

so, tl;dr, it sounds like if we go ahead and merge this, we should then get comments as expected on all other PRs. Are other maintainers OK with that? If nobody yells, I'll go ahead and merge this in a few days and we'll see what happens.

[AdamWill]

If I'm reading that right, possibly we'll need to merge this and then do one more push to master somehow before comments start working. Not totally sure if the merge of this PR would give us the initial push build on master that coveralls needs.