Add local Changelog generation based on PR titles and labels #4219

vigneshsankariyer1234567890 · 2024-03-31T11:15:43Z

Description

Currently, we use Github's Changelog generator: https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes

However, this is not the best option since it increases reliance on Github actions themselves. Furthermore, there is a need for local changelog generation in order to simplify the release process.

In the initial issue, attempts were made to look for a tool that could generate the Changelogs directly. Some tools include git-chglog, which actually generate the logs based on the commits themselves. However, these tools are not useful because FerretDB does not follow Changelog commit message standards at the moment, which leads to non-standard commit messages. This also means that it is difficult to use a tool that groups commit messages to hence get the list of PRs.

Instead, I opted for a custom approach that utilised the existing changelog template found in FerretDB (under .github/release.yml). Instead of using commits directly, the idea is to get the merged PRs from the Github API endpoint after the latest tag (which indicates a release). With the definition provided in the release.yml, we can emulate the Changelog generation action locally by subsequently grouping the PRs by the labels defined in the template, and lastly rendering as markdown to stdout.

Here is a demo of how it works:

Screen.Recording.2024-04-03.at.1.15.46.PM.mov

To test it, generate binaries for tools as per instructions in the CONTRIBUTING.md, and launch task changelog directly from root folder, with an input MILESTONE_TITLE according to the milestone that you wish to generate the Changelog for.

Closes #2682 .

Readiness checklist

I added/updated unit tests (and they pass).
I added/updated integration/compatibility tests (and they pass).
I added/updated comments and checked rendering.
I made spot refactorings.
I updated user documentation.
I ran task all, and it passed.
I ensured that PR title is good enough for the changelog.
(for maintainers only) I set Reviewers (@FerretDB/core), Milestone (Next), Labels, Project and project's Sprint fields.
I marked all done items in this checklist.

CLAassistant · 2024-03-31T11:15:49Z

All committers have signed the CLA.

introduce mod at changelog

tools/go.work

AlekSi · 2024-04-01T07:20:07Z

tools/generatechangelog/go.mod

Please move that into tools/go.mod

moved into tools/go.mod

First of all:

FerretDB/CONTRIBUTING.md

Lines 359 to 363 in c299c53

4. In the pull request review conversations,

please either leave a new comment or resolve (close) the conversation,

which ensures other people can read all comments.

But do not do that simultaneously.

Conversations should typically be resolved by the conversation starter, not the PR author.

Second, tools/go.mod wasn't updated.
Third, CI fails: https://github.com/FerretDB/FerretDB/actions/runs/8506940135/job/23298489403?pr=4219. Please run formatting, linting, etc locally as described in CONTRIBUTING.md

Apologies regarding comment resolution.

I have updated tools/go.mod to include the dependency for yaml.v3 (and amended formatting issues)

tools/generatechangelog/generatechangelog.go

tools/generatechangelog/generatechangelog_test.go

codecov · 2024-04-01T10:23:02Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.21%. Comparing base (f95b5d2) to head (9a393ff).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4219      +/-   ##
==========================================
- Coverage   75.35%   67.21%   -8.14%     
==========================================
  Files         323      316       -7     
  Lines       22407    22051     -356     
==========================================
- Hits        16884    14821    -2063     
- Misses       4303     6070    +1767     
+ Partials     1220     1160      -60

see 127 files with indirect coverage changes

Flag	Coverage Δ
filter-false	`?`
filter-true	`67.21% <ø> (-1.50%)`	⬇️
hana-1	`?`
integration	`67.21% <ø> (-1.58%)`	⬇️
mongodb-1	`5.11% <ø> (-0.01%)`	⬇️
mysql-1	`?`
mysql-2	`?`
mysql-3	`?`
postgresql-1	`46.38% <ø> (-0.09%)`	⬇️
postgresql-2	`49.52% <ø> (-0.10%)`	⬇️
postgresql-3	`49.72% <ø> (-8.45%)`	⬇️
sqlite-1	`45.52% <ø> (-0.16%)`	⬇️
sqlite-2	`48.61% <ø> (-0.18%)`	⬇️
sqlite-3	`48.87% <ø> (-0.11%)`	⬇️
unit	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

AlekSi · 2024-04-01T10:50:41Z

@vigneshsankariyer1234567890 I will do a re-review shortly; please see our contribution documentation in the meantime

AlekSi

#4219 (comment)

AlekSi · 2024-04-02T04:09:46Z

tools/go.mod

@@ -20,6 +20,8 @@ require (
 	mvdan.cc/gofumpt v0.6.0
 )

+require gopkg.in/yaml.v3 v3.0.1


Please move it to the group above

Added to group as requested, and run task lint to verify formatting.

AlekSi · 2024-04-02T04:10:15Z

tools/tools.go

@@ -25,11 +25,14 @@ import (
 	_ "golang.org/x/tools/cmd/stringer"
 	_ "golang.org/x/vuln/cmd/govulncheck"
 	_ "mvdan.cc/gofumpt"
+
+	_ "github.com/FerretDB/FerretDB/tools/generatechangelog"


It should not be there – note that checkdocs, checkswitch, checkcomments are all not imported

Removed verified binary generation still works.

vigneshsankariyer1234567890 · 2024-04-02T04:31:26Z

@AlekSi regarding tests for the introduced tool, I noticed that tools/Taskfile.yml has a script for testing checkcomments, checkdocs, checkswitch etc. Should I also be adding the test for generatechangelog within that?

AlekSi · 2024-04-02T04:32:06Z

Yes, absolutely. Please follow existing things and style as close as possible

vigneshsankariyer1234567890 · 2024-04-02T05:37:45Z

@AlekSi requested your review, and ensured lint and formatting passes. At the moment, running task all fails due to internal test failures, but I'm checking if it is an issue with my device.

AlekSi · 2024-04-02T15:48:08Z

tools/generatechangelog/changelog_template.tmpl

+- {{ .Title }} by @{{ .User.Login }} in {{ .URL }}
+{{- end }}
+{{- "\n" }}
+{{- end }}


Let's add a missing line break there

added line break

AlekSi · 2024-04-02T15:48:42Z

tools/generatechangelog/generatechangelog.go

+const (
+	OrgOwner = "FerretDB"
+	Repo     = "FerretDB"
+)


I think we could remove those constants

removed constants, and used directly in github call

But they are still there?..

AlekSi · 2024-04-02T15:50:03Z

tools/generatechangelog/generatechangelog.go

+	return &commit.Author.Date.Time, nil
+}
+
+func FetchPRs(ctx context.Context, client *github.Client, owner, repo, sinceDate string) ([]PRItem, error) {


We could remove owner and repo

removed params for owner and repo

AlekSi · 2024-04-02T15:54:48Z

tools/generatechangelog/generatechangelog.go

+	query := fmt.Sprintf("repo:%s/%s is:pr is:merged merged:>=%s", owner, repo, sinceDate)
+	opts := &github.SearchOptions{Sort: "created", Order: "desc", ListOptions: github.ListOptions{PerPage: 100}}
+
+	issues, _, err := client.Search.Issues(ctx, query, opts)


That would not work correctly if two versions were being developed and released in parallel. For example, v1.21.1 and v1.22.0.

Let's simplify all that: make the tool accept the Milestone title, get it, get a list of all PRs in it

Understood. Just to be clear, this would be the flow:

Get the milestone based on title

Search for PRs which are closed and merged based on the milestone number

Format according to release template

Re 2 – not “search”. Search is a different and more complicated API. We just need to get a list of PRs within the milestone: https://github.com/FerretDB/FerretDB/milestone/63?closed=1

Understood. I used the "Issues" API instead, which has a ListByRepo method which allows to pass the milestone number and filter by closed. Subsequently, I identified Issues as Pull Requests by using the Issue::IsPullRequest method. However, I'm not sure about the distinction between Closed and Merged Pull requests, so I will take a further look at the API to clarify this.

AlekSi · 2024-04-02T15:55:51Z

tools/generatechangelog/generatechangelog.go

+	res := []PRItem{}
+
+	for _, issue := range issues.Issues {
+		res = append(res, func(i *github.Issue) PRItem {


I'm not sure why we use anonymous inline function with struct and JSON tags, but I think we should not

Removed the JSON tags, they were there originally for verification

I still don't understand why we use anonymous inline structs. I think we should not.

tools/generatechangelog/generatechangelog.go

AlekSi · 2024-04-02T15:58:16Z

tools/generatechangelog/generatechangelog_test.go

+func TestGroupPRsByCategories(t *testing.T) {
+	prItems := []PRItem{
+		{
+			URL:    "http://example.com/pr1",


I don't think there is a need to test each individual piece individually. We could use a real (old) milestone and real (old) PRs

I've added an integration test that does each individual part and checks based on the output, which has real PRs.

vigneshsankariyer1234567890 · 2024-04-03T05:14:54Z

@AlekSi requested for your re-review. As requested, I have changed it such that task changelog takes in a variable "MILESTONE_TITLE", and can be used like this: task changelog MILESTONE_TITLE="your_milestone_here". I have also amended it to use the Issues API directly, and added an integration test that checks if the output matches the expected PRs for an old version (v0.9.1).

mergify · 2024-04-16T14:18:38Z

@vigneshsankariyer1234567890 this pull request has merge conflicts.

…gelog

AlekSi

There are multiple CI failures that mask linter failures. Please run them locally.

AlekSi · 2024-05-06T12:25:20Z

tools/generatechangelog/generatechangelog.go

+const (
+	OrgOwner = "FerretDB"
+	Repo     = "FerretDB"
+)


But they are still there?..

AlekSi · 2024-05-06T12:31:45Z

tools/generatechangelog/generatechangelog.go

+}
+
+func NewGitHubClient() *github.Client {
+	return github.NewClient(nil).WithAuthToken(os.Getenv("GITHUB_TOKEN"))


We should use https://github.com/FerretDB/gh there

AlekSi · 2024-05-06T12:33:28Z

tools/generatechangelog/generatechangelog.go

+	res := []PRItem{}
+
+	for _, issue := range issues.Issues {
+		res = append(res, func(i *github.Issue) PRItem {


I still don't understand why we use anonymous inline structs. I think we should not.

mergify · 2024-05-07T12:06:48Z

@vigneshsankariyer1234567890 this pull request has merge conflicts.

add changelog package in tools

7a9f648

vigneshsankariyer1234567890 requested a review from AlekSi as a code owner March 31, 2024 11:15

mergify bot assigned vigneshsankariyer1234567890 Mar 31, 2024

vigneshsankariyer1234567890 added 3 commits March 31, 2024 19:27

add comments

0a83e9e

remove go mod and sum changes at root and tools

b6098ac

introduce mod at changelog

add comments

da4d258

AlekSi requested changes Apr 1, 2024

View reviewed changes

vigneshsankariyer1234567890 added 3 commits April 1, 2024 15:37

remove workspace files

627385b

remove mod files within generatechangelog

4745dcb

complete refactoring

d4fd140

vigneshsankariyer1234567890 requested a review from AlekSi April 1, 2024 10:17

remove go mod changes from previous commit

287c450

AlekSi reviewed Apr 1, 2024

View reviewed changes

vigneshsankariyer1234567890 and others added 2 commits April 2, 2024 12:04

update tools/go.mod

5f0c301

Merge branch 'main' into generate-local-changelog

a3c02c6

AlekSi reviewed Apr 2, 2024

View reviewed changes

vigneshsankariyer1234567890 added 2 commits April 2, 2024 12:20

refactor go.mod and remove generatechangelog import

0d615c9

add main comments to support documentation

d485f50

vigneshsankariyer1234567890 added 2 commits April 2, 2024 12:34

add generatechangelog test to be run in test runner

a4ecaec

lint go.mod

9a0c7c6

vigneshsankariyer1234567890 requested a review from AlekSi April 2, 2024 05:30

fix niggling format issues on taskfile

2fd6f71

AlekSi reviewed Apr 2, 2024

View reviewed changes

change to use milestones as input

34da6ac

vigneshsankariyer1234567890 added 3 commits April 3, 2024 12:48

add integration tests

3b9c9e0

add line break for tmpl

70ac52c

lint generatechangelog

e61ed8d

vigneshsankariyer1234567890 requested a review from AlekSi April 3, 2024 05:15

AlekSi added the community Issues and PRs assigned to community members label Apr 15, 2024

mergify bot added the conflict PRs that have merge conflicts label Apr 16, 2024

Merge remote-tracking branch 'upstream/main' into generate-local-chan…

7797c2c

…gelog

mergify bot removed the conflict PRs that have merge conflicts label Apr 17, 2024

AlekSi added 2 commits May 1, 2024 13:24

Merge branch 'main' into generate-local-changelog

22ff43b

Merge branch 'main' into generate-local-changelog

9a393ff

AlekSi requested changes May 6, 2024

View reviewed changes

mergify bot added the conflict PRs that have merge conflicts label May 7, 2024

	4. In the pull request review conversations,
	please either leave a new comment or resolve (close) the conversation,
	which ensures other people can read all comments.
	But do not do that simultaneously.
	Conversations should typically be resolved by the conversation starter, not the PR author.

Add local Changelog generation based on PR titles and labels #4219

Are you sure you want to change the base?

Add local Changelog generation based on PR titles and labels #4219

Conversation

vigneshsankariyer1234567890 commented Mar 31, 2024 • edited

Description

Readiness checklist

CLAassistant commented Mar 31, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Apr 1, 2024 • edited

Codecov Report

AlekSi commented Apr 1, 2024

AlekSi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vigneshsankariyer1234567890 commented Apr 2, 2024

AlekSi commented Apr 2, 2024

vigneshsankariyer1234567890 commented Apr 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vigneshsankariyer1234567890 commented Apr 3, 2024

mergify bot commented Apr 16, 2024

AlekSi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mergify bot commented May 7, 2024

vigneshsankariyer1234567890 commented Mar 31, 2024 •

edited

CLAassistant commented Mar 31, 2024 •

edited

codecov bot commented Apr 1, 2024 •

edited