Remove hardcoded versions from release cronjobs #814

ninsbl · 2022-10-06T20:26:24Z

Fixes #760

wenzeslaus

No comment on the use of GitHub API, but if you are going with this, the branch-version code needs to be unified across the scripts (as if there would be one script with parameters).

wenzeslaus · 2022-10-07T03:24:38Z

utils/cronjobs_osgeo_lxd/cron_grass7_relbranch_build_binaries.sh

-GMAJOR=7
-GMINOR=8
-GPATCH=7 # required by grass-addons-index.sh
+BRANCH=`curl https://api.github.com/repos/osgeo/grass/branches | grep releasebranch_7 | grep '"name":' | cut -f4 -d'"' | sort -V | tail -n 1`


The major version is hidden in this expression, so releasebranch_7 needs to be some branch prefix variable or something.

Right. Another option, to get rid of hardcoded versions completely is to fetch the latest major version -1 to run cron jobs for one version older than current.

That said, maybe also files should be renamed e.g.:
cron_grass7_relbranch_build_binaries.sh to cron_grass_legacy_relbranch_build_binaries.sh
and
cron_grass8_relbranch_build_binaries.sh to cron_grass_latest_relbranch_build_binaries.sh

I probably should also unify how versions are fetched across the different scripts...

In case of renaming of sh-files, changes should be handed down to https://github.com/OSGeo/grass-addons/blob/grass8/utils/cronjobs_osgeo_lxd/README.md and https://github.com/OSGeo/grass-addons/blob/grass8/utils/cronjobs_osgeo_lxd/cron_job_list_grass

Renaming has been done in #814. So, in essence, only https://github.com/OSGeo/grass-addons/blob/grass8/utils/cronjobs_osgeo_lxd/cron_job_list_grass still contains version numbers (log file storage into the right server directory).

wenzeslaus · 2022-10-08T03:24:44Z

utils/cronjobs_osgeo_lxd/README.md

-  * grass7-stable: `cron_grass7_relbranch_src_snapshot.sh`
-  * grass8-stable: `cron_grass8_relbranch_src_snapshot.sh`
-  * grass8-devel: `cron_grass8_main_src_snapshot.sh`
+  * grass-legacy: `cron_grass_legacy_relbranch_src_snapshot.sh`


...while at it, do you want to use whole words for release branch? The branch names say releasebranch, although they are unfortunately not separated words, they are at least whole words.

So, you mean like this?

grass-legacy: cron_grass_legacy_src_snapshot.sh

etc.?

My understanding was (implemented in ac22e46 to fb3fdbd), like renaming from e.g. cron_grass7_relbranch_src_snapshot.sh to cron_grass_legacy_releasebranch_src_snapshot.sh

As for the actual changes in the code: Fetching the version from github (https://raw.githubusercontent.com/osgeo/grass/main/include/VERSION) is tested locally. When reviewing changes, please note that the version is picked up from github at the time the script runs. So, depending on the order in which things are done during release, that may be correct (or not). I don`t know the details of the release procedure, but I am happy to adjust the code if needed.

Also special attention should be put on the patch version. Here only numbers are taken (e.g. dev is removed). From the hardcoded example, that was my understanding how the patch version was defined...

Also I noticed that there is https://grass.osgeo.org/grass-stable/manuals/addons/, but no https://grass.osgeo.org/grass-legacy/manuals/addons/, yet https://grass.osgeo.org/grass7/manuals/addons/. So if we want to support a legacy version, that may be good to add on the webserver...

but no https://grass.osgeo.org/grass-legacy/manuals/addons/

I have now activated the grass-legacy/ redirect.

Good! The README.md is adjusted accordingly. However, please note that the version numbers remain in the target log-files in the cron-table (cron_job_list_grass). Ideas how to remove those are welcome...

neteler · 2022-10-23T12:09:12Z

~~While at it, could you please relax the timing schedule in the cronjob list? They are too close:~~

(... taken care of in #828)

ninsbl · 2022-10-23T14:03:45Z

Hmm... Whoile trying to merge other changes I messed up the PR again.
I ran commit -im to finish a merge and I guess that included a bunch of other unwanted commits / files...

Will cleanup and re-submit a tidy PR once #826 is merged...

neteler · 2022-12-20T17:39:06Z

Will cleanup and re-submit a tidy PR once #826 is merged...

(PR #826 has been merged)

utils/cronjobs_osgeo_lxd/cron_grass_legacy_releasebranch_build_binaries.sh

wenzeslaus · 2023-04-17T17:53:46Z

utils/cronjobs_osgeo_lxd/cron_grass_legacy_releasebranch_build_binaries.sh

+CURRENT_MAJOR=`echo "$CURRENT_VERSION" | sed -n '1{p;q}'`
+GMAJOR=`expr $CURRENT_MAJOR - 1`
+
+BRANCH=`curl https://api.github.com/repos/osgeo/grass/branches | grep releasebranch_$GMAJOR | grep '"name":' | cut -f4 -d'"' | sort -V | tail -n 1`


This will always result in the same thing as CURRENT_BRANCH since GMAJOR is taken from CURRENT_BRANCH, no?

wenzeslaus · 2023-04-17T17:57:11Z

utils/cronjobs_osgeo_lxd/cron_grass_legacy_releasebranch_build_binaries.sh

+MAIN_VERSION=`curl https://raw.githubusercontent.com/osgeo/grass/$BRANCH/include/VERSION`
+
+GMINOR=`echo "$MAIN_VERSION" | sed -n '2{p;q}'`
+GPATCH=`echo "$MAIN_VERSION" | sed -n '3{p;q}' | sed 's/[^0-9]*//g'`


I don't understand, minor is taken from the file while major is taken from the branch and used for the branch again. Why not getting the right branch and then the version numbers from there?

Is this all just to keep the two scripts in sync? I think the "current" thing is really relevant for the legacy, but here where the code can be simpler. Maybe I just needs to code comments to clarify.

…_binaries.sh Co-authored-by: Vaclav Petras <wenzeslaus@gmail.com>

neteler · 2023-05-18T19:50:03Z

This PR would need to be updated, following the new scheme (version numbers as of yesterday):

label	meaning
legacy	legacy stable version, no longer recommended for use
old current	current stable version, widely used
new current	upcoming stable version, for early adopters
preview	development version, for developers and new feature enthusiasts

@ninsbl

After merging of OSGeo#896 and cronjob deployment on grass.osgeo.org various issues came up which are addressed in this PR. Important: - fix incorrect branch in `cron_grass_new_current_build_binaries.sh` - generate programmer's manual only in `cron_grass_new_current_build_binaries.sh` Improvements: - fail early rather than useless continuation (`set -e`) - compile with 2 cores (`make -j2`) - fix copying of `INSTALL` and `REQUIREMENTS.html` to server download directory as being renamed to `INSTALL.md` and `REQUIREMENTS.md` - also copy `CITATION.cff` to server download directory Misc: - no longer generate gettext `.pot` file for transifex since we migrated to https://weblate.osgeo.org/projects/grass-gis/ This PR potentially also addresses OSGeo#814 (can @ninsbl please verify what's yet missing)?

After merging of #896 and cronjob deployment on grass.osgeo.org various issues came up which are addressed in this PR. Important: - fix incorrect branch in `cron_grass_new_current_build_binaries.sh` - generate programmer's manual only in `cron_grass_new_current_build_binaries.sh` Improvements: - fail early rather than useless continuation (`set -e`) - compile with 2 cores (`make -j2`) - fix copying of `INSTALL` and `REQUIREMENTS.html` to server download directory as being renamed to `INSTALL.md` and `REQUIREMENTS.md` - also copy `CITATION.cff` to server download directory Misc: - no longer generate gettext `.pot` file for transifex since we migrated to https://weblate.osgeo.org/projects/grass-gis/ (this PR potentially also addresses #814)

After merging of OSGeo#896 and cronjob deployment on grass.osgeo.org various issues came up which are addressed in this PR. Important: - fix incorrect branch in `cron_grass_new_current_build_binaries.sh` - generate programmer's manual only in `cron_grass_new_current_build_binaries.sh` Improvements: - fail early rather than useless continuation (`set -e`) - compile with 2 cores (`make -j2`) - fix copying of `INSTALL` and `REQUIREMENTS.html` to server download directory as being renamed to `INSTALL.md` and `REQUIREMENTS.md` - also copy `CITATION.cff` to server download directory Misc: - no longer generate gettext `.pot` file for transifex since we migrated to https://weblate.osgeo.org/projects/grass-gis/ (this PR potentially also addresses OSGeo#814)

ninsbl added the bug Something isn't working label Oct 6, 2022

ninsbl requested a review from neteler October 6, 2022 20:26

ninsbl removed the bug Something isn't working label Oct 6, 2022

wenzeslaus reviewed Oct 7, 2022

View reviewed changes

wenzeslaus changed the title ~~remove hardcoded versions from release cronjobs~~ Remove hardcoded versions from release cronjobs Oct 7, 2022

wenzeslaus reviewed Oct 8, 2022

View reviewed changes

ninsbl added 21 commits April 16, 2023 22:40

versionless cronjobs

7c5f6c8

unify and remove hardcoded major

3285253

versionless names

601049b

versionless names

14d26d6

rename to versionless

5b62141

rename to versionless

841a7f3

rename

f6984cc

full branchname

aea462e

rename cronjobs files

a4cd3aa

rename cronjobs files

7903d54

rename

634a909

rename

159719f

rename

917965e

rename

298cc95

rename

0548c78

rename

1d917db

rename

23540d0

rename

70e1198

rename

b8529b5

replace major 7 with legacy

3624dad

versionless cronjobs

f20d06b

ninsbl added 3 commits April 16, 2023 23:04

unify and remove hardcoded major

4de73f9

versionless names

f5ebfee

try versionless log

e094f3d

ninsbl force-pushed the versionless_cronjobs branch from 91be659 to e094f3d Compare April 16, 2023 21:18

wenzeslaus reviewed Apr 17, 2023

View reviewed changes

ninsbl and others added 2 commits April 18, 2023 15:21

Update utils/cronjobs_osgeo_lxd/cron_grass_legacy_releasebranch_build…

df03226

…_binaries.sh Co-authored-by: Vaclav Petras <wenzeslaus@gmail.com>

Update utils/cronjobs_osgeo_lxd/cron_grass_legacy_releasebranch_build…

3d5041f

…_binaries.sh Co-authored-by: Vaclav Petras <wenzeslaus@gmail.com>

ninsbl mentioned this pull request May 18, 2023

cronjobs: new filename scheme and G84 added #896

Merged

Merge branch 'grass8' into versionless_cronjobs

f358413

neteler mentioned this pull request May 29, 2023

cronjobs: various fixes and cleanup #903

Merged

Merge branch 'grass8' into versionless_cronjobs

2de533a

ninsbl self-assigned this Jan 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove hardcoded versions from release cronjobs #814

Remove hardcoded versions from release cronjobs #814

ninsbl commented Oct 6, 2022

wenzeslaus left a comment

wenzeslaus Oct 7, 2022

ninsbl Oct 7, 2022

ninsbl Oct 7, 2022

neteler May 29, 2023

wenzeslaus Oct 8, 2022

neteler Oct 9, 2022

ninsbl Oct 9, 2022

neteler Oct 10, 2022

ninsbl Oct 10, 2022

neteler commented Oct 23, 2022 •

edited

ninsbl commented Oct 23, 2022

neteler commented Dec 20, 2022

wenzeslaus Apr 17, 2023

wenzeslaus Apr 17, 2023

wenzeslaus Apr 17, 2023

neteler commented May 18, 2023 •

edited

Remove hardcoded versions from release cronjobs #814

Are you sure you want to change the base?

Remove hardcoded versions from release cronjobs #814

Conversation

ninsbl commented Oct 6, 2022

wenzeslaus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neteler commented Oct 23, 2022 • edited

ninsbl commented Oct 23, 2022

neteler commented Dec 20, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neteler commented May 18, 2023 • edited

neteler commented Oct 23, 2022 •

edited

neteler commented May 18, 2023 •

edited