Add changelog script

[keunes]

No description provided.

[ByteHamster]

To be honest, the text replacements inside some files feels a bit risky to me. For example, a commit might reference another commit in its message (Reverts y). The script then does weird stuff.

Instead, how about using python? There you can easily access different attributes of the results instead of having to concatenate the strings earlier using jq. Something like this:

import requests
import time

TOKEN = ""
REPO = "AntennaPod/AntennaPod"
BASE = "2.7.1"
HEAD = "master"

outputFile = open('PRs.txt', 'w')
commits = requests.get("https://api.github.com/repos/" + REPO + "/compare/" + BASE + "..." + HEAD, headers={'Authorization': 'token ' + TOKEN}).json()
numCommits = len(commits["commits"])
for i in range(numCommits):
    print("Commit " + str(i) + " of " + str(numCommits))
    sha = commits["commits"][i]["sha"]
    commit = commits["commits"][i]["commit"]
    if "Merge pull request #" in commit["message"]:
        continue
    time.sleep(1) # Avoid rate limit
    try:
        prs = requests.get("https://api.github.com/search/issues?q=repo:" + REPO + "+type:pr+is:merged+" + sha, headers={'Authorization': 'token ' + TOKEN}).json()
        firstPr = prs["items"][0]
        outputFile.write(firstPr["pull_request"]["merged_at"] + " " + firstPr["html_url"] + " " + firstPr["title"] + " " + firstPr["user"]["login"] + "\n")
    except requests.exceptions.RequestException as e:
        outputFile.write("Orphan: " + commit["message"] + " " + commit["author"]["login"] + "\n")
outputFile.close()

The script doesn't have the same functionality as your bash script yet, but I think it is quite a bit easier to read (fewer wild sed replacements) and more "stable" to different commit messages

[keunes]

To be honest, the text replacements inside some files feels a bit risky to me

Is there any security risk? Or you mean more like potentially flaky'?

For example, a commit might reference another commit in its message (Reverts y). The script then does weird stuff.

Where might this happen? The only 'replacing' happens in the line where I add 'orphan' in the commit list. But that relies on the line starting with "commit-sha" so I don't see how Reverts sha in the commit message might interfere.

Instead, how about using python?

Most happy to drop this if someone can provide same functionality 🙂 I struggled making this but kinda managed thanks to some basic understanding, but I know 0 about Python :-)

[ByteHamster]

but I know 0 about Python

Me neither, I copied this code together from examples. But I still think it is easier to understand what the code actually does than the bash text replacements. There one really needs to think about what ends up in which file and how each file is structured. With python, there is one single outputFile.write(...) statement that actually prints the file.

Most happy to drop this if someone can provide same functionality

Does this work for you?

#!/usr/bin/env python3
import requests
import time

REPO = "AntennaPod/AntennaPod"

print("Hello, welcome to the AntennaPod PR list generator!")
print("First, please enter your GitHub API token.")
print("If you don't have one yet, create it at https://github.com/settings/tokens")
TOKEN = input('Token: ')
print("Grand, thank you! (" + TOKEN + " is noted)")

print()
print("Now, what do you want to compare?")
print("Please enter a release code or branch")
print("[default: latest GitHub release]")
BASE = input('Base: ')
if BASE == "":
    release = requests.get("https://api.github.com/repos/" + REPO + "/releases/latest", headers={'Authorization': 'token ' + TOKEN}).json()
    BASE=release["tag_name"]
    print("Okido, latest release (" + BASE + ") it is!")
else:
    print("Noted")

print()
print("Then, what should be our endpoint?")
print("[default: 'master']")
HEAD = input('Head: ')
if HEAD == "":
    print("Righty, master it is!")
    HEAD="master"
else:
    print("Roger that.")

print()
prsSeen = set()
outputFile = open('PRs.csv', 'w')
outputFile.write("PR Merge date,PR URL,PR Title,PR Author,type,functionaliy group\n")
commits = requests.get("https://api.github.com/repos/" + REPO + "/compare/" + BASE + "..." + HEAD, headers={'Authorization': 'token ' + TOKEN}).json()
numCommits = len(commits["commits"])
for i in range(numCommits):
    print("Commit " + str(i) + " of " + str(numCommits))
    sha = commits["commits"][i]["sha"]
    commit = commits["commits"][i]["commit"]
    if "Merge pull request #" in commit["message"] or "Merge branch" in commit["message"]:
        print("  Is merge")
        continue
    time.sleep(1) # Avoid rate limit
    try:
        prs = requests.get("https://api.github.com/search/issues?q=repo:" + REPO + "+type:pr+is:merged+" + sha, headers={'Authorization': 'token ' + TOKEN}).json()
        if len(prs["items"]) == 0:
            print("  No search results: " + commit["message"].splitlines()[0])
            raise Exception('Results')
        firstPr = prs["items"][0]
        if firstPr["number"] in prsSeen:
            print("  Already seen: " + firstPr["title"])
            continue
        outputFile.write(firstPr["pull_request"]["merged_at"] + "," + firstPr["html_url"] + "," + firstPr["title"] + "," + firstPr["user"]["login"] + "\n")
        print("  " + firstPr["title"])
        prsSeen.add(firstPr["number"])
    except Exception as e:
        print("  Orphan: " + commit["message"].splitlines()[0])
        outputFile.write("Orphan,Orphan," + commit["message"].splitlines()[0] + "," + commit["author"].get("login", "Unknown author") + "\n")
outputFile.close()

[keunes]

Thanks a lot @ByteHamster. I must admit that the Python script is slightly easier to read also. I took your code and made some changes following some issues I encountered running it.

I just wonder why we print cli output both as 'orphan' (no search results) and as 'exception'? If I'm not mistaken they virtually always overlap. If they might not 100% overlap, maybe we should not raise an exception in case of an orphan commit?

Also, any particular reason we only look at the 'first PR' for each commit? (I can't think of a reason to do otherwise, but I'm curious.)

[ByteHamster]

If they might not 100% overlap, maybe we should not raise an exception in case of an orphan commit?

Ah, I would instead remove the text output. Raising the exception ensures that the orphan is written to the output file in any case.

Also, any particular reason we only look at the 'first PR' for each commit?

Ah, hmm. In your script, one commit was able to add multiple PRs to the output. Is this what we want?

[keunes]

Raising the exception ensures that the orphan is written to the output file in any case.

Maybe we can simplify further by not raising an 'exception' and and doing this instead: if(orphan) elif(already seen) else(normal)?

Or should we use an exception?

In your script, one commit was able to add multiple PRs to the output. Is this what we want?

Dunno. I find it hard to think of any situation in which this might occur. Like - could a commit end up as the only commit in PR 1 while also being a part of another PR? Or a commit that was part of a PR for version 2.6 (would be ignored) and of 3.0 (we'd want to include it)? Somehow feels quite unlogical for the same code change to land in a release twice.

[ByteHamster]

Maybe we can simplify further by not raising an 'exception' and and doing this instead: if(orphan) elif(already seen) else(normal)?

Sure, if that works. I'm not experienced with Python and just assumed that it will crash the whole app when the request fails. But if that can be checked directly, it would be great as well :)

Like - could a commit end up as the only commit in PR 1 while also being a part of another PR?

This is possible but I don't think we ever had this for AntennaPod. I think it is save to ignore.

Or a commit that was part of a PR for version 2.6 (would be ignored) and of 3.0 (we'd want to include it)?

Then we probably have two different commits because they refer to different target branches

[keunes]

Ok, thanks. Save to check only one PR per commit then, as is. I'll see with ChatGPT and test if it works with the if elif else statements 🙂

[ByteHamster]

When a PR gets squash-merged (like e9ba45e, for example), the SHA changes. Then the script does not find the PR. If a commit title contains a PR number, the script could look up that PR specifically instead of using the search function. That should reduce the number of orphans.

[ByteHamster]

Ready for merging?

[keunes]

I think so yes.