{data}[gfbf/2023a] polars v0.19.19 #19770

lexming · 2024-02-01T11:09:35Z

(created using eb --new-pr)

This PR is for v0.19.19 instead of the more recent v0.20.x because the latter need nightly features in Rust that are not yet available in Rust v1.75.

Depends on:

change tar command used in get_source_tarball_from_git to get reproducible tarballs easybuild-framework#4248
update Cargo easyblock to extract crates into a vendor subdir and overwrite git repo URLs with local paths easybuild-easyblocks#3118
{data}[gfbf/2023a] DeltaLake v0.15.1 and fix Arrow v14.0.1 to install pyarrow as an extension #19758
Fix stability of reproducible tarballs from git repos easybuild-framework#4517

lexming · 2024-02-01T14:47:57Z

Test report by @lexming
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
node301.hydra.os - Linux Rocky Linux 8.8, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/lexming/473d85488bbc985ed5832b88f4fb1a09 for a full test report.

lexming · 2024-02-01T15:44:28Z

Test report by @lexming
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node301.hydra.os - Linux Rocky Linux 8.8, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/lexming/0b32ad1d8a6765776ad556c86727af19 for a full test report.

…asyconfigs into 20240201120931_new_pr_polars01919

lexming · 2024-02-09T22:39:44Z

@boegelbot: please test @ generoso

boegelbot · 2024-02-09T22:40:08Z

@lexming: Request for testing this PR well received on login1

PR test command 'EB_PR=19770 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_19770 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 12869

Test results coming soon (I hope)...

- notification for comment with ID 1936697091 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-02-09T22:41:16Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/59898416841730ad7eb426a3aa9b4c8d for a full test report.

boegel · 2024-02-14T12:57:52Z

@lexming failing checksum for jsonpath_lib-0.3.0.tar.gz?

lexming · 2024-02-14T22:07:37Z

@boegel Indeed, jsonpath_lib has to be pulled from a git repo, so this PR needs a solution to easybuilders/easybuild-framework#4248

…ball

akesandgren · 2024-04-03T05:51:40Z

Superseeded by #20275

lexming · 2024-04-04T23:33:29Z

@akesandgren I would still like to get this merged, we are using this version in our clusters

lexming · 2024-04-04T23:41:06Z

@boegelbot: please test @ generoso
EB_ARGS="--force-download"

boegelbot · 2024-04-04T23:45:09Z

@lexming: Request for testing this PR well received on login1

PR test command 'EB_PR=19770 EB_ARGS="--force-download" EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_19770 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 13252

Test results coming soon (I hope)...

- notification for comment with ID 2038453176 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-04-04T23:46:55Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/8eb3b4138bf705ec023fc70a23b516d1 for a full test report.

lexming · 2024-04-05T00:25:32Z

the bot cannot use the 5.0.x branch, so we wait

boegel · 2024-04-15T17:51:28Z

@boegelbot please test @ jsc-zen3
EB_ARGS="--force-download"
EB_BRANCH=5.0.x

boegelbot · 2024-04-15T18:00:21Z

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=19770 EB_ARGS="--force-download" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_19770 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 3953

Test results coming soon (I hope)...

- notification for comment with ID 2057490882 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-04-15T18:02:12Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/b2ee8b7eb8ba434b1114a113c5b3ce7a for a full test report.

lexming · 2024-04-25T15:16:08Z

I cannot reproduce the failures with the checksum of jsonpath_lib-0.3.0.tar.gz. I did an installation of polars v0.19.19 on a new system of ours and EB 5.0.x generated a tarball with checksum f91c11e6f2dd81ddfdde101e4d65e6cebc4cf9a4c18f9e4fbb4400df2fdccabc as expected.

I'll see if can inspect the tarball in the bot systems.

lexming · 2024-04-25T15:16:50Z

@boegelbot please test @ generoso
EB_ARGS="--force-download"
EB_BRANCH=5.0.x

boegelbot · 2024-04-25T15:20:09Z

@lexming: Request for testing this PR well received on login1

PR test command 'EB_PR=19770 EB_ARGS="--force-download" EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_19770 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 13360

Test results coming soon (I hope)...

- notification for comment with ID 2077547550 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-04-25T15:42:04Z

Test report by @boegelbot
FAILED
Build succeeded for 1 out of 2 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/614775b35dd96e8dd22410b7e3010bba for a full test report.

lexming · 2024-04-26T00:06:05Z

@boegel found the issue with all the failed tests for this PR, fix in easybuilders/easybuild-framework#4517

lexming · 2024-04-26T17:21:08Z

@boegelbot please test @ generoso
EB_ARGS="--force-download"
EB_BRANCH=5.0.x

boegelbot · 2024-04-26T17:25:09Z

@lexming: Request for testing this PR well received on login1

PR test command 'EB_PR=19770 EB_ARGS="--force-download" EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_19770 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 13365

Test results coming soon (I hope)...

- notification for comment with ID 2079788168 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-04-26T17:27:02Z

Test report by @boegelbot
FAILED
Build succeeded for 0 out of 1 (1 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/a503b144d3e74d758ed266e7d383b839 for a full test report.

lexming · 2024-04-26T17:35:01Z

The 5.0.x used by generoso is outdated. The generated tarball has wrong permissions and timestamps.
@boegel does the bot pull the branch from the remote repo, or just does a local checkout?

update: actually EB_BRANCH seems to be totally ignored. From the failed test report:

EasyBuild info
easybuild-framework version: 4.9.2.dev0-ra9ef0d22b9b428ad902585f1e750a7b8a5d924cd
easybuild-easyblocks version: 4.9.2.dev0-rb6a05c994f2eac6e6fe494d1e3be0fe75df0f1bc

update 2: support for branches seems to be better on jsc-zen3, trying that one...

lexming · 2024-04-26T17:49:51Z

@boegelbot please test @ jsc-zen3
EB_ARGS="--force-download"
EB_BRANCH=5.0.x

boegelbot · 2024-04-26T17:50:13Z

@lexming: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=19770 EB_ARGS="--force-download" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_19770 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 4037

Test results coming soon (I hope)...

- notification for comment with ID 2079826826 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-04-26T18:24:54Z

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/c4a2ffbbd07b7cfab365f509a1a0d497 for a full test report.

lexming · 2024-04-29T12:33:43Z

Test report by @lexming
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node601.hydra.os - Linux Rocky Linux 8.9, x86_64, AMD EPYC 9384X 32-Core Processor (x86_64_v4), Python 3.6.8
See https://gist.github.com/lexming/644fba85b10f1a135530f8ec4f1b1353 for a full test report.

lexming · 2024-04-30T07:07:58Z

@boegelbot please test @ jsc-zen3
EB_ARGS="--force-download"
EB_BRANCH=5.0.x

boegelbot · 2024-04-30T07:10:14Z

@lexming: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ "5.0.x" != 'develop' ]]; then EB_BRANCH="5.0.x" ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/"5.0.x" source init_env_easybuild_develop.sh; fi; EB_PR=19770 EB_ARGS="--force-download" EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH="5.0.x" /opt/software/slurm/bin/sbatch --job-name test_PR_19770 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 4043

Test results coming soon (I hope)...

- notification for comment with ID 2084534051 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-04-30T07:45:14Z

Test report by @boegelbot
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/90de7b0b24a15aaf99174f9b09f6f077 for a full test report.

lexming · 2024-04-30T07:55:33Z

Test report by @lexming
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
node600.hydra.os - Linux Rocky Linux 8.9, x86_64, AMD EPYC 9384X 32-Core Processor (x86_64_v4), Python 3.6.8
See https://gist.github.com/lexming/1a2bb3a773a93932f1cf33e2ab7fe11e for a full test report.

lexming · 2024-04-30T08:01:32Z

@boegel all tests are good with most recent 5.0.x, this is ready

jfgrimm

lgtm

adding easyconfigs: polars-0.19.19-gfbf-2023a.eb

406c226

lexming added the update label Feb 1, 2024

This comment was marked as outdated.

Sign in to view

lexming mentioned this pull request Feb 1, 2024

singleM vscentrum/vsc-software-stack#235

Open

27 tasks

lexming mentioned this pull request Feb 1, 2024

update Cargo easyblock to extract crates into a vendor subdir and overwrite git repo URLs with local paths easybuilders/easybuild-easyblocks#3118

Merged

Merge branch 'develop' of https://github.com/easybuilders/easybuild-e…

538b1e0

…asyconfigs into 20240201120931_new_pr_polars01919

update checksum of jsonpath_lib-0.3.0.tar.gz using a reproducible tar…

9cecb2f

…ball

lexming changed the base branch from develop to 5.0.x March 6, 2024 08:14

lexming added the EasyBuild-5.0 EasyBuild 5.0 label Mar 6, 2024

lexming added this to the 5.0 milestone Mar 6, 2024

akesandgren mentioned this pull request Apr 3, 2024

{data}[gfbf/2023a] polars v0.20.2 #20275

Merged

1 task

easybuilders deleted a comment from boegelbot Apr 4, 2024

lexming mentioned this pull request Apr 26, 2024

Fix stability of reproducible tarballs from git repos easybuilders/easybuild-framework#4517

Merged

lexming added 2 commits April 26, 2024 19:09

update checksum of jsonpath_lib-0.3.0.tar.gz to be reproducible

9232f81

Merge branch '5.0.x' into 20240201120931_new_pr_polars01919

3c579a9

lexming force-pushed the 20240201120931_new_pr_polars01919 branch from c78afea to 3c579a9 Compare April 26, 2024 17:20

remove default PythonPackage settings from polars v0.19.19

178d731

jfgrimm approved these changes May 8, 2024

View reviewed changes

jfgrimm merged commit 817b61f into easybuilders:5.0.x May 8, 2024
7 checks passed

lexming deleted the 20240201120931_new_pr_polars01919 branch May 8, 2024 15:53

{data}[gfbf/2023a] polars v0.19.19 #19770

{data}[gfbf/2023a] polars v0.19.19 #19770

Conversation

lexming commented Feb 1, 2024 • edited

This comment was marked as outdated.

lexming commented Feb 1, 2024

lexming commented Feb 1, 2024

lexming commented Feb 9, 2024

boegelbot commented Feb 9, 2024

boegelbot commented Feb 9, 2024

boegel commented Feb 14, 2024

lexming commented Feb 14, 2024

akesandgren commented Apr 3, 2024

lexming commented Apr 4, 2024

lexming commented Apr 4, 2024

boegelbot commented Apr 4, 2024

boegelbot commented Apr 4, 2024

lexming commented Apr 5, 2024

boegel commented Apr 15, 2024

boegelbot commented Apr 15, 2024

boegelbot commented Apr 15, 2024

lexming commented Apr 25, 2024

lexming commented Apr 25, 2024

boegelbot commented Apr 25, 2024

boegelbot commented Apr 25, 2024

lexming commented Apr 26, 2024

lexming commented Apr 26, 2024

boegelbot commented Apr 26, 2024

boegelbot commented Apr 26, 2024

lexming commented Apr 26, 2024 • edited

lexming commented Apr 26, 2024

boegelbot commented Apr 26, 2024

boegelbot commented Apr 26, 2024

lexming commented Apr 29, 2024

lexming commented Apr 30, 2024

boegelbot commented Apr 30, 2024

boegelbot commented Apr 30, 2024

lexming commented Apr 30, 2024

lexming commented Apr 30, 2024

jfgrimm left a comment

Choose a reason for hiding this comment

lexming commented Feb 1, 2024 •

edited

lexming commented Apr 26, 2024 •

edited