/
RELEASE_NOTES
2901 lines (2579 loc) · 184 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
This file contains a description of the major changes to the easybuild-framework EasyBuild package.
For more detailed information, please see the git log.
These release notes can also be consulted at https://easybuild.readthedocs.io/en/latest/Release_notes.html.
v4.9.1 (5 April 2024)
---------------------
update/bugfix release
- various enhancements, including:
- make `is_rpath_wrapper` faster by only checking file contents if file is not located in subdirectory of RPATH wrapper subdirectory (#4406)
- add terse support to `--missing-modules` (#4407)
- adapt version pattern for EnvironmentModules to allow using development version (#4416)
- use `--all` option with EnvironmentModules v4.6+ to get available hidden modules (#4417)
- add support for appending to path environment variables via `modextrapaths_append` + add corresponding `allow_append_abs_path` (#4436)
- improve output produced by `--check-github` (#4437)
- add script for updating local git repos with `develop` branch (#4438)
- show error when multiple PR options are passed (#4440)
- improve `findPythonDeps` script to recognize non-canonical package names (#4445)
- add support for `--from-commit` and `--include-easyblocks-from-commit` (#4468)
- improve logging & handling of (empty) `--optarch` values (#4481)
- add `--short` option to `findUpdatedEcs` script (#4488)
- add generic GCC and Clang compiler flags for RISC-V (#4489)
- various bug fixes, including:
- clean up log file of `EasyBlock` instance in `check_sha256_checksums` (#4452)
- fix description of `backup-modules` configuration option (#4456)
- replace `'` with `"` for `printf` in CI workflow for running test suite to have bash replace a variable (#4461)
- use `cp -dR` instead of `cp -a` for shell script "extraction" (#4465)
- fix link to documentation in `close_pr` message (#4466)
- fix `test_github_merge_pr` by using more recent easyconfigs PR (#4470)
- add workaround for 404 error when installing packages in CI workflow for testing Apptainer integration (#4472)
- other changes:
- clean up & speed up environment checks (#4409)
- use more performant and concise dict construction by using dict comprehensions (#4410)
- remove superflous string formatting (#4411)
- clean up uses of `getattr` and `hasattr` (#4412)
- update copyright lines to 2024 (#4494)
v4.9.0 (30 December 2023)
-------------------------
feature release
- various enhancements, including:
- allow tweaking of easyconfigs from different toolchains (#3669)
- add support for `--list-software --output-format=json` (#4152)
- add `--module-cache-suffix` configuration setting to allow multiple (Lmod) caches (#4403)
- various bug fixes, including:
- deduplicate warnings & errors found in logs and add initial newline + tab in output (#4361)
- fix support for Environment Modules as modules tool to pass unit tests with v4.2+ (#4369)
- adapt module function check for Environment Modules v4+ (#4371)
- only install GitHub token when testing with Lmod 8.x + Python 3.6 or 3.9 (#4375)
- use `-qopenmp` instead of `-fiopenmp` for OpenMP in Intel compilers (#4377)
- fix `LIBBLAS_MT` for FlexiBLAS, ensure `-lpthread` is included (#4379)
- relax major version match regex in `find_related_easyconfigs` using for `--review-pr` (#4385)
- eliminate duplicate multideps from generated module files (#4386)
- resolve templated values in extension names in `_make_extension_list` (#4392)
- use source toolchain version when passing only `--try-toolchain` (#4395)
- fix writing spider cache for Lmod >= 8.7.12 (#4402)
- fix `--inject-checksums` when extension specifies patch file in tuple format (#4405)
- fix `LooseVersion` when running with Python 2.7 (#4408)
- use more recent easyblocks PR in `test_github_merge_pr` (#4414)
- other changes:
- extend test that checks build environment to recent `foss/2023a` toolchain (#4391)
v4.8.2 (29 October 2023)
------------------------
update/bugfix release
- various enhancements, including:
- add support for `%(sysroot)s` template value (#4359)
- add `dependency_names` method to `EasyConfig` class to get set of names of (direct) dependencies (#4360)
- various bug fixes, including:
- add CI workflow to run unit tests with Python 2 (again) (#4333)
- fix typo in help message for `--silence-hook-trigger` (#4343)
- include major version (`*majver`) templates in auto-generated documentation (#4347)
- reset `tempfile.tempdir` to `None` to avoid that tmpdir path gets progressively deeper with each easystack item (#4350)
- fix `findPythonDeps.py` script when called with an (absolute or relative) path to an easyconfig instead of a filename (#4365)
- fix broken test for `reasons_for_closing`, which fails because commit status of easyconfigs PR is no longer available (#4366)
- other changes:
- reduce number of CI jobs by testing for Lua and Tcl module syntax in a single CI job (#4192)
v4.8.1 (11 September 2023)
--------------------------
update/bugfix release
- various enhancements, including:
- add end-to-end test for running EasyBuild in different Linux distros using containers (#3968)
- suggest default title in `--review-pr` (#4287)
- add `build_and_install_loop` hooks to run before and after the install loop for individual easyconfigs (#4304)
- implement support for `cancel_hook` and `fail_hook` (#4315, #4325)
- add postiter hook to the list of steps so the corresponding hook can be used (#4316)
- add `run_shell_cmd` hook (#4323)
- add `build_info_msg` easyconfig parameter to print message during installation of an easyconfig (#4324)
- add `--silence-hook-trigger` configuration option to supress printing of debug message every time a hook is triggered (#4329)
- add support for using fine grained Github tokens (#4332)
- add definitions for ifbf and iofbf toolchain (#4337)
- add support for submodule filtering and specifying extra Git configuration in `git_config` (#4338, #4339)
- various bug fixes, including:
- improve error when checksum dict has no entry for a file (#4150)
- avoid error being logged when `checksums.json` is not found (#4261)
- don't fail in `mkdir` if path gets created while processing it (#4300, #4328)
- ignore request for external module (meta)data when no modules tool is active (#4308)
- use sys.executable to obtain path to `python` command in tests, rather than assuming that `python` command is available in `$PATH` (#4309)
- fix `test_add_and_remove_module_path` by replacing string comparison of paths by checking whether they point to the same path (since symlinks may cause trouble) (#4312)
- enhance `Toolchain.get_flag` to handle lists (#4319)
- only add extensions in module file if there are extensions (#4331)
v4.8.0 (7 July 2023)
--------------------
feature release
- various enhancements, including:
- use version suffixes in PR title instead of Python versions when using --new-pr (#4253 + #4286)
- add script to find updated ECs for installed module (#4271)
- various bug fixes, including:
- fix regex for extracting glibc version from output of '`ldd --version`' in Gentoo Linux (#4290)
- allow versionsuffix to be set to None in `det_full_ec_version` (#4292)
- other changes:
- stop running tests with Python 2.7 since it is no longer supported in GitHub Action (#4267)
- replace imp.load_source by leveraging importlib.util.module_from_spec when using Python 3 (#4280)
v4.7.2 (27 May 2023)
--------------------
update/bugfix release
- various enhancements, including:
- extend (experimental) support for generating container images with Apptainer (#3975)
- add default download timeout (#4226)
- make the RPATH section check with `readelf -d` in sanity check optional (#4249)
- various bug fixes, including:
- add `make_extension_string` and `_make_extension_list` methods to `EasyBlock` class, so easyblocks can customize them (#3697)
- fix typo in condition guarding installation of GitHub token in CI workflow to run unit tests + fix broken tests related to GitHub integration features (#4216)
- fix --check-github as gist url now includes username (#4231)
- switch to Ubuntu 22.04 in GitHub Actions workflow to run container tests (#4238)
- use "/usr/bin/env python" in test_run_cmd_script (#4241)
- other changes:
- print potential errors after running a command only once (#4065)
- change github_* template descriptions to be somewhat clearer (#4251)
- remove workflow to test bootstrap script (#4255)
v4.7.1 (March 20th 2023)
------------------------
update/bugfix release
- various enhancements, including:
- add option to make sanity_check_paths arch dependent (#3845)
- add support for %(start_dir)s easyconfig template (#4073)
- add Ubuntu friendly package naming scheme EasyBuildDebFriendlyPNS (#4115)
- allow to directly import constants from easybuild.framework.easyconfig.constants (#4144)
- accept single source as dict and single checksum in check_checksums_for (#4180)
- add pre/post extension hook that is triggered before/after individual extension installations (#4193)
- enforce absolute paths as start dir of extensions (#4196)
- print warning when non-existing `start_dir` is specified for extension (#4201)
- add support to silence deprecation warning for easyconfigs and toolchains (#4207)
- skip directories when fixing shebangs (#4209)
- add support for 'modunloadmsg' easyconfig parameter (#4223)
- various bug fixes, including:
- fix use of `locate` in `check_os_dependency` (#4166)
- add VERSION to easybuild.easyblocks namespace in test sandbox (#4171)
- also apply filter to asyncprocess test subsuite (#4172)
- don't use deprecated SafeConfigParser when running with Python 3.x (#4173)
- update CI workflows to run 'apt-get update' if installation of packages via 'apt-get install' failed, likely due to stale apt cache (#4176)
- fixes for findPythonDeps script (#4179)
- check type of versionsuffix value in det_full_ec_version, and raise useful error message if it's not a string (#4184)
- avoid GC3Pie deprecation warning (#4185)
- correctly restore sys.path in tests (#4186)
- suppress output of --skip-test-step test (#4187)
- fix flaky test_search_file due to accidental matches for `.*C++` search pattern (#4190)
- silence distutils deprecation warnings (#4204)
- improve handling of start_dir and add tests for cases where either ext_dir or initial start_dir or both are unset or None (#4206)
- restore initial environment before processing each easystack item (#4213)
- make test_help_long work on small terminals (#4218)
- other changes:
- use better test assertions by replacing use of assertFalse/assertTrue (#4170, #4205)
- remove Travis CI configuration file (#4174)
- filter out deprecation warnings for platform.dist in get_os_version and platform.linux_distribution in get_os_name (#4175)
- only give read permissions in GitHub Actions workflows (#4182)
- fix website/docs links in README (#4199)
- only print "default: " for configuration option of strlist type if default is not empty (#4220)
v4.7.0 (January 9th 2023)
-------------------------
feature release
- print deprecation warning with running EasyBuild with Python 2 (#4136)
- various enhancements, including:
- add support for checksums specified in external `checksums.json` file (#3749)
- vendor distutils.version.LooseVersion as easybuild.tools.LooseVersion (since distutils is deprecated in Python 3.10) (#3794, #4156)
- take into account custom configuration options specified in easystack file (#4057)
- add support for using --output-format=md (MarkDown) (#4117, #4155)
- add support for --filter-rpath-sanity-libs to skip RPATH sanity check for designated libraries (#4119)
- add sanity_check_load_module method to EasyBlock to provide control over when module is loaded during sanity check step (#4125)
- add eb_bash_completion_local.bash script to setup.py (#4127)
- check whether nvidia-smi/rocm-smi command is available before trying to run it in `get_gpu_info` (#4131)
- add gfbf as subtoolchain of foss (#4143)
- add support for postinstallmsgs (#4145)
- make iimkl toolchain aware of intel-compilers (#4146)
- add definition for nvofbf toolchain (#4157, #4163)
- various bug fixes, including:
- also use EasyConfig instances cache in process_easyconfig when build_specs is empty dict (#4107)
- fix build options that should have empty list as default value (#4108)
- catch easyconfig parsing failure so we can generate and post a test report (#4109)
- add explicit return when no checksums match those specified in a tuple (#4112)
- tweak findPythonDeps.py script to use canonical package names (#4118)
- add `-fdefault-double-8` to `r8` toolchain compiler option for GCC (to be consistent with Intel) (#4121)
- always start with empty list for banned/requires libraries to avoid that corresponding build option is updated in-place (#4137)
- fix container tests by using EPEL archive URL for downloading Singularity RPM (#4138)
- use -march=x86-64 -mtune=generic instead of -xSSE2 when using Intel oneAPI compilers (#4147)
- partially skip test_det_copy_ec_specs if no GitHub token is available (#4149)
- other changes:
- drop support for easystack files using 'software' top-level key (#4057)
- also run unit tests with Python 3.11 + add Python 3.11 to classifiers in setup.py (#4092, #4141)
- use new EasyBuild logo in README (#4123)
- automatically cancel Github Action workflow runs for outdated commits (#4132)
- auto-enable use of oneAPI C/C++ compilers for intel-compilers >= 2022.2.0 (#4135)
- trim CI test configurations to avoid hitting GitHub rate limits (#4148)
- various tweaks to docstrings and help messages to fix problems with auto-generated documentation in MarkDown format (#4129, #4130, #4154, #4160, #4162, #4168)
- update copyright lines for 2023 (#4161)
v4.6.2 (October 21st 2022)
--------------------------
update/bugfix release
- various enhancements, including:
- add support for easystack file that contains easyconfig filenames + implement parsing of configuration options (#4021)
- skip over unset $EB_PYTHON/$EB_INSTALLPYTHON in eb wrappeer script (#4080)
- add GITHUB_RELEASE and GITHUB_LOWER_RELEASE templates (#4084)
- add `%(cuda_cc_cmake)s` template (#4087)
- various bug fixes, including:
- make check_sha256_checksums verify all checksums if they're specified as a dict value (#4076)
- replace use of symlink with copied files in `alt_location` tests to fix failing EasyBuild installation on BeeGFS (#4083)
- fix trying to generate RPATH wrappers for Clang (#4088)
- make sure that GitPython version is a proper version before checking minimal required version (#4090, #4091)
- first look for patch in alt_location when it is specified (#4093)
- other changes:
- make scripts executable (#4081)
- make --inject-checksums inject dictionary value for checksums which maps filename to SHA256 checksum (#4085)
- update to v3 of actions/checkout and actions/setup-python in CI workflows (#4089)
- use SYSTEM template constant in dependencies instead of True in framework tests (#4094)
v4.6.1 (September 12th 2022)
----------------------------
update/bugfix release
- various enhancements, including:
- add script to find dependencies of Python packages (#3839)
- add 'ai' default module class (#4053)
- various bug fixes, including:
- fix code style issues reported by recent flake8 linter (#4049)
- stick to autopep8 < 1.7.0 with Python 2.7 (#4055)
- ensure we call EasyBlock.patch_step for postinstallpatches (#4063)
- fix leaked handles in set_columns, complete_cmd, run_cmd_qa, det_terminal_size functions + tests (#4066)
- fix `quote_str` when string with both ' and " ends with a double quote (#4068)
- fix type-checking of patches to allow dict values + correctly handle patches specified as dict values in --new-pr (#4070)
- relax toolchain test by accepting both -march=native (x86_64) and -mcpu=native (aarch64) (#4071)
- other changes:
- run python in the same process as `eb` wrapper script by using `exec` (#4048)
- add get_linked_libs_raw function, and use it from both check_linked_shared_libs and sanity_check_rpath (#4051)
- update CI workflows (except container tests) to use Ubuntu 20.04, since Ubuntu 18.04 is deprecated (#4064)
- use SYSTEM constant for dependency that uses system toolchain in dumped easyconfig (#4069)
v4.6.0 (July 8th 2022)
----------------------
feature release
- various enhancements, including:
- allow searching for sources/patches in alternative location by specifying 'alt_location' in source/patch spec (#3994)
- show URLs used for download attempts in trace output (#4026)
- add support for setting environment variables via 'pushenv' with modextravars (#4030)
- add support for OneAPI compilers using toolchain option 'oneapi' (#4031, #4032, #4039)
- make check_linked_shared_libs more robust by taking into account that 'ldd' may fail (#4033)
- fall back to sequential extension install if parallel install is not implemented (#4034)
- add support for using template values in name/version of extensions (#4036)
- various bug fixes, including:
- make sure that ARCH constant has 'aarch64' (rather than 'arm64') as value on macOS ARM (#4018)
- tweak 'eb' wrapper script to correctly handle errors when python command being considered fails to run (#4019)
- tweak is_patch_for function to make it more robust (#4028)
- other changes:
- update Lmod used to run tests to version 8.7.6 (#4027, #4030)
- tweak apply_patch to not create .orig files (by default) when applying patch files (#4038)
v4.5.5 (June 8th 2022)
----------------------
update/bugfix release
- various enhancements, including:
- add toolchain definitions for nvompi (NVHPC + OpenMPI) (#3969), nvpsmpi (NVHPC + ParaStationMPI) (#3970), gfbf (GCC, FlexiBLAS, FFTW) (#4006)
- add support for FFTW.MPI toolchain component ($FFT*DIR variables) (#4012)
- add support for customizing EasyBuild command used in jobs via --job-eb-cmd (#4016)
- various bug fixes, including:
- fix copying of easyconfig file with --copy-ec without --rebuild if module is already installed (#3993)
- ignore deprecation warnings regarding cryptography in Python 3.6 + Blowfish with Python 3.10 in test suite output (#3999)
- fix typo in debug log message in easyblock.py (#4000)
- fix printing of list of attempted download URLs when url-encoded characters are used in URL (#4005)
- set $FFT(W)_LIB_DIR to imkl-FFTW's lib path in build environment if usempi toolchain option is enabled (#4011)
- correctly identify Apple Silicon M1 as Arm 64-bit by also considering arm64 next to aarch64 (#4014)
- fix 'eb --show-system-info' on Apple M1 system (#4015)
- other changes:
- change 'eb' command to import easybuild.framework to check if EasyBuild framework is available (#3995, #3998)
v4.5.4 (March 31st 2022)
------------------------
update/bugfix release
- various enhancements, including:
- warn about potentially missing patches in --new-pr (#3759, #3966)
- add support for 'clone_into' field in git_config source spec to specify different name for top-level directory (#3949)
- add bash completion for easyconfigs from local dir but not robot search path (#3953)
- add a 'sync pr' message when the PR has a mergeable state but is showing a failed status for the test suite on the last commit (#3967)
- add gmpit toolchain definition (GCC + MPItrampoline) (#3971)
- use 'zypper search -i' to check whether specified OS dependency is installed on openSUSE + make sure that rpm is considered for checking OS dependencies on RHEL8 (#3973)
- add support for post-install patches (#3974)
- add support for 'download_instructions' easyconfig parameter key to specify some download or installation steps for user in case of complicated way of obtaining needed files (#3976, #3981)
- also try collecting AMD GPU info (via rocm-smi) for --show-system-info (#3978, #3982)
- various bug fixes, including:
- ensure --review-pr can find dependencies included in PR (#3979)
- run 'apt-get update' in GitHub Actions workflow for container tests to update container package list before installing anything (#3985)
- other changes:
- enable code linting check for all supported Python versions (#3725)
- update copyright lines for 2022 (#3986)
v4.5.3 (February 11th 2022)
---------------------------
update/bugfix release
- various enhancements, including:
- also check for git in --check-eb-deps (#3954)
- add end2end test for 'eb --containerize' (#3958)
- various bug fixes, including:
- take into account that patch files can also be zipped when checking filename extension for patches (#3936)
- initialize BACKUP_PKG_URL with empty string in install_eb_dep.sh script (#3939)
- fix get_os_name and get_os_version to avoid reporting UNKNOWN in output of eb --show-system-info (#3942)
- consistently use actions/setup-python@v2 in CI workflows (#3944)
- switch to using pip3 for installing EasyBuild in container recipes generated by EasyBuild (#3945)
- specify easybuild.io as EasyBuild homepage in setup.py (#3947)
- avoid crash in get_os_version on modern SLES-based OSs (#3955)
- other changes:
- indicate compatibility with Python 3.10 in setup.py (#3940)
v4.5.2 (January 24th 2022)
--------------------------
update/bugfix release
- various enhancements, including:
- automatically prepend value for env-for-shebang configuration setting with sysroot (#3919)
- enhance update_build_option to return original value + add support for update_build_options (#3923)
- fix compatibility with Python 3.10 (#3926, #3933)
- also take /etc/os-release into account in get_os_name and get_os_version (#3930)
- enhance get_cpu_architecture and get_cpu_family to be aware of RISC-V (#3931)
- various bug fixes, including:
- relax pattern checks in test_toy_exts_parallel (#3921)
- use sources.easybuild.io as backup URL for downloading Environment Modules sources to run test suite (#3928)
- make intelfftw toolchain component aware of GCCcore (#3929)
- other changes:
- deprecate use of (actual) patch files that don't have a filename ending with .patch (#3920)
- remove version restriction for keyring in requirements.txt to test with latest version (#3925)
- simplify requirements.txt by removing Python 2.6 support (#3927)
- only run GitHub tests when testing with Lua module syntax, to avoid hitting GitHub rate limit when running tests (#3938)
v4.5.1 (December 13th 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- also dump environment to reprod directory (#3374)
- determine which extensions can be skipped in parallel (if --parallel-extensions-install is enabled) (#3890)
- fall back to sequential installation for extensions with unknown dependencies when using --parallel-extensions-install (#3906)
- allow oversubscription in sanity check for OpenMPI-based toolchains (#3909)
- various bug fixes, including:
- don't try to ensure absolute path for path part of repositorypath (#3893, #3899)
- fix typo in EULA agreement error message (#3894)
- only remove lock if it was created in the same EasyBuild session (not if it existed already) (#3889)
- introduce EasyBlock.post_init method to correctly define builddir variable when build-in-installdir mode is enabled in easyconfig or easyblock (#3900)
- also show download progress bar when using --inject-checksums (#3905)
- pick up custom extract_cmd specified for extension (#3907)
- make test_run_cmd_async more robust against fluke failures (#3908)
v4.5.0 (October 29th 2021)
--------------------------
feature release
- various enhancements, including:
- add --review-pr-max and --review-pr-filter options to limit easyconfigs used by --review-pr + retain order of easyconfigs being compared with (#3754)
- use Rich (if available) to show progress bars when installing easyconfigs (#3823, #3826, #3833, #3835, #3844, #3864, #3867, #3882)
- see also https://docs.easybuild.io/en/latest/Progress_bars.html
- add support for checking required/optional EasyBuild dependencies via 'eb --check-eb-deps' (#3829)
- add support for collecting GPU info (via nvidia-smi), and include it in --show-system-info and test report (#3851)
- added support for --insecure-download configuration option (#3859)
- make intelfftw toolchain component aware of imkl-FFTW module (#3861)
- make sure the contrib/hooks tree is included in the source tarball for easybuild-framework (#3862)
- add check_async_cmd function to facilitate checking on asynchronously running commands (#3865, #3881)
- add initial/experimental support for installing extensions in parallel (#3667, #3878)
- see also https://docs.easybuild.io/en/latest/Installing_extensions_in_parallel.html
- filter out duplicate paths added to module files, and print warning when they occur (#3770, #3874)
- various bug fixes, including:
- ensure that path configuration options have absolute path values (#3832)
- fix broken test by taking into account changed error raised by Python 3.9.7 when copying directory via shutil.copyfile (#3840)
- ensure newer location of CUDA stubs is taken into account by RPATH filter (#3850)
- replace 'which' by 'command -v' in 'eb' wrapper script to avoid dependency on 'which' command (#3852)
- refactor EasyBlock to decouple collecting of information on extension source/patch files from downloading them (#3860)
- in this context, the EasyBlock.fetch_extension_sources method was deprecated, and replaced by EasyBlock.collect_exts_file_info
- fix library paths to add to $LDFLAGS for intel-compilers toolchain component (#3866)
- remove '--depth 1' from git clone when 'commit' specified (#3871, #3872)
- make sure correct include directory is used for FlexiBLAS toolchain component (#3875)
- explictly disable rebase when pulling develop branch to create a merge commit, since not specifying how to reconcile divergent branches is an error with Git 2.33.1 and newer (#3879)
- tweak test_http_header_fields_urlpat to download from sources.easybuild.io rather than https://ftp.gnu.org (#3883)
- other changes:
- change copy_file function to raise an error when trying to copy non-existing file (#3836, #3855, #3877)
- only print the hook messages if EasyBuild is running in debug mode (#3843)
- deprecate old toolchain versions (pre-2019a common toolchains) (#3876, #3884)
- see also https://docs.easybuild.io/en/latest/Deprecated-easyconfigs.html#deprecated-toolchains
v4.4.2 (September 7th 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- add per-extension timing in output produced by eb command (#3734)
- add definition for new toolchain nvpsmpic (NVHPC + ParaStationMPI + CUDA) (#3736)
- include list of missing libraries in warning about missing FFTW libraries in imkl toolchain component (#3776)
- check for recursive symlinks by default before copying a folder (#3784)
- add --filter-ecs configuration option to filter out easyconfigs from set of easyconfigs to install (#3796)
- check type of source_tmpl value for extensions, ensure it's a string value (not a list) (#3799)
- also define $BLAS_SHARED_LIBS & co in build environment (analogous to $BLAS_STATIC_LIBS) (#3800)
- report use of --ignore-test-failure in success message in output (#3806)
- add get_cuda_cc_template_value method to EasyConfig class (#3807)
- add support for fix_bash_shebang_for (#3808)
- pick up $MODULES_CMD to facilitate using Environment Modules 4.x as modules tool (#3816)
- use more sensible branch name for creating easyblocks PR with --new-pr (#3817)
- various bug fixes, including:
- remove Python 2.6 from list of supported Python versions in setup.py (#3767)
- don't add directory that doesn't include any files to $PATH or $LD_LIBRARY_PATH (#3769)
- make logdir writable also when --stop/--fetch is used and --read-only-installdir is enabled (#3771)
- fix forgotten renaming of 'l' to 'char' __init__.py that is created for included Python modules (#3773)
- fix verify_imports by deleting all imported modules before re-importing them one by one (#3780)
- fix ignore_test_failure not set for Extension instances (#3782)
- update iompi toolchain to intel-compiler subtoolchain for oneAPI versions (>= iompi 2020.12) (#3785)
- don't parse patch files as easyconfigs when searching for where patch file is used (#3786)
- make sure git clone with a tag argument actually downloads a tag (#3795)
- fix CI by excluding GC3Pie 2.6.7 (which is broken with Python 2) and improve error reporting for option parsing (#3798)
- correctly resolve templates for patches in extensions when uploading to GitHub (#3805)
- add --easystack to ignored options when submitting job (#3813)
- other changes:
- speed up tests by caching checked paths in set_tmpdir + less test cases for test_compiler_dependent_optarch (#3802)
- speed up set_parallel method in EasyBlock class (#3812)
v4.4.1 (July 6th 2021)
----------------------
update/bugfix release
- various enhancements, including:
- enhance detection of patch files supplied to 'eb' command with better error messages (#3709)
- add per-step timing information (#3716)
- add module-write hook (#3728)
- add ignore-test-failure configuration option to ignore failing test step (#3732)
- add toolchain definition for nvompic (NVHPC + OpenMPI) (#3735)
- warn about generic milestone in --review-pr and --merge-pr (#3751)
- various bug fixes, including:
- don't override COMPILER_MODULE_NAME, inherited from Ffmpi, in Fujitsu toolchain definition (#3721)
- avoid overwritting pr_nr in post_pr_test_report for reports with --include-easyblocks-from-pr (#3724, #3726)
- fix crash in get_config_dict when copying modules that were imported in easyconfig file (like 'import os') (#3729)
- parse C standard flags in CFLAGS for Fujitsu compiler (#3731)
- fix error message for --use-ccache (#3733)
- error out when passing a list to run_cmd* to avoid running wrong command unintended, unless shell=True is used (#3737)
- minor fixes to output of test reports when using multiple PRs (#3741)
- fix location for modules installed with intel-compilers toolchain in HierarchicalMNS by always checking toolchain compiler name against template map (#3745)
- avoid checking msg attribute of GitCommandError (#3756)
- consider sources provided as dict in EasyBlock.check_checksums_for (#3758)
- don't make changes to software installation directory when using --sanity-check-only (#3761)
- honor specified easyblock for extensions (#3762)
- other changes:
- make sure that tests requiring a github token have 'github' in the test name so that they can be easily filtered (#3730)
- deprecate EasyBuild bootstrap script (#3742)
- use temporary download folder in test_http_header_fields_urlpat (#3755)
v4.4.0 (June 2nd 2021)
----------------------
feature release
- various enhancements, including:
- enhance apply_regex_substitutions to allow specifying action to take in case there are no matches (#3440)
- performance improvements for easyconfig parsing and eb startup (#3555)
- add support for downloading easyconfigs from multiple PRs with --from-pr (#3605, #3707, #3708)
- add support for prepending custom library paths in RPATH section via --rpath-override-dirs (#3650)
- allow amending easyconfig parameters which are not the default (#3651)
- update HierarchicalMNS for Intel OneAPI compilers (#3653)
- add support for --sanity-check-only (#3655)
- add support for running commands asynchronously via run_cmd + complete_cmd functions (#3656)
- add support for using oneAPI versions of 'intel' toolchain components (#3665)
- add toolchain definition for gofbf (foss with FlexiBLAS rather than OpenBLAS) (#3666)
- add support for using ARCH constant in easyconfig files (#3670)
- honor keyboard interrupt in 'eb' command (#3674)
- add toolchain definition for Fujitsu toolchain (#3677, #3704, #3712, #3713, #3714, #3717)
- extend sanity check step to check whether specific libraries are not linked into installed binaries/libraries (#3678)
- via banned-linked-shared-libs and required-linked-shared-libs EasyBuild configuration options
- via banned_linked_shared_libs and required_linked_shared_libs methods in toolchain support
- via banned_linked_shared_libs and required_linked_shared_libs methods in easyblock
- via banned_linked_shared_libs and required_linked_shared_libs easyconfig parameters
- add locate_solib function to locate Linux shared libraries (#3682)
- add system agnostic function to locate shared libraries (#3683)
- add update_build_option function to update specific build options after initializing the EasyBuild configuration (#3684)
- allow opting out of recursively unloaded of modules via recursive_module_unload easyconfig parameter (#3689)
- check for correct version values when parsing easystack file (#3693)
- run post-install commands specified for a specific extension (#3696)
- add support for --skip-extensions (#3702)
- include PR title in output produced by --merge-pr (#3706)
- various bug fixes, including:
- avoid metadata greedy behaviour when probing for external module metadata (mostly relevant for integration with Cray Programming Environment) (#3559)
- catch problems early on if --github-user is not specified for --new-pr & co (#3644)
- re-enable write permissions when installing with read-only-installdir (#3649)
- also run sanity check for extensions when using --module-only (#3655)
- improve logging when failing to load class from exts_classmap (#3657)
- fix use of --module-only on existing installations without write permissions (#3659)
- correct help text for subdir-user-modules (#3660)
- avoid picking up easyblocks outside of sandbox in framework tests (#3680)
- use unload/load in ModuleGeneratorLua.swap_module, since 'swap' is not supported by Lmod (#3685)
- update HierarchicalMNS to also return 'Toolchain/<name>/<version>' as $MODULEPATH extension for cpe* Cray toolchains (#3686)
- make EasyConfigParser.get_config_dict return a copy rather than a reference (#3692)
- make sure that $TAPE is unset when using piped tar (#3698)
- fix extending message for changed files in new_pr_from_branch (#3699)
- enhance sched_getaffinity function to avoid early crash when counting available cores on systems with more than 1024 cores (#3701)
- correctly strip extension from filename in extract_cmd and back_up_file functions (#3705)
- other changes:
- deprecate adding a non-existing path to $MODULEPATH (#3637)
- bump cryptography requirement from 3.2.1 to 3.3.2 (#3643, #3648)
- test bootstrap script in separate workflow, and limit test configurations a bit (#3646)
- update setup.py to indicate compatibility with Python 3.8 and 3.9 (#3647)
- replace log_error parameter of which() by on_error (#3661, #3664)
- don't skip sanity check for --module-only --rebuild (#3645)
- move disable_templating function into the EasyConfig class (#3668)
- pin GitPython version for Python<3.6, don't test bootstrap script against Python 3.8/3.9 (#3675)
- tweak foss toolchain definition to switch from OpenBLAS to FlexiBLAS in foss/2021a (#3679)
- suggest missing SSH key when not able to read from remote repository in --check-github (#3681)
- drop support for Python 2.6 (#3715)
v4.3.4 (April 9th 2021)
-----------------------
update/bugfix release
- various enhancements, including:
- add support for filtering dependencies by using False as version (#3506)
- add create_unused_dir function to create a directory which does not yet exist (#3551)
- avoid running expensive 'module use' and 'module unuse' commands when using Lmod as modules tool, update $MODULEPATH directly instead (#3557, #3633)
- create CUDA cache (for JIT compiled PTX code) in build dir instead of $HOME (#3569)
- add "Citing" section to module files (#3596)
- add support for using fallback 'arch=*' key in dependency version specified as arch->version mapping (#3600)
- also check for pending change requests and mergeable_state in check_pr_eligible_to_merge (#3604)
- ignore undismissed 'changes requested' review if there is an 'approved' review by the same user (#3607, #3608)
- sort output of 'eb --search' in natural order (respecting numbers) (#3609)
- enhance 'eb' command to ensure that easybuild.main can be imported before settling on python* command to use (#3610)
- add --env-for-shebang configuration option to define the env command to use for shebangs (#3613)
- add templates for architecture independent Python wheels (#3618)
- mention easyblocks PR in gist when uploading test report for it + fix clean_gists.py script (#3622)
- also accept regular expression value for --accept-eula-for (#3630)
- update validate_github_token function to accept GitHub token in new format (#3632)
- various bug fixes, including:
- fix $BLAS_LIB_MT for OpenBLAS, ensure -lpthread is included (#3584)
- use '--opt=val' for passing settings from config file to option parser to avoid error for values starting with '-' or '--' (#3594)
- avoid raised exception when getting output from interactive command in run_cmd_qa (#3599)
- add option to write file from file-like object and use in download_file (#3614)
- make sure that path to eb is always found by tests (#3617)
- other changes:
- add pick_default_branch function to clean up duplicate code in tools/github.py (#3592)
- refactor the CI configuration to use inclusion instead of exclusion (#3616)
- use develop branch when testing push access in --check-github (#3629)
- deprecate --accept-eula, rename to --accept-eula-for (#3630)
v4.3.3 (February 23rd 2021)
---------------------------
update/bugfix release
- various enhancements, including:
- advise PR labels in --review-pr and add support for --add-pr-labels (#3177)
- add support for using customized HTTP headers in download_file (#3472, #3583)
- also take toolchain dependencies into account when defining template values (#3541, #3560)
- add support for --accept-eula configuration option + 'accept_eula' easyconfig parameter (#3535, #3536, #3546)
- detect 'SYSTEM' toolchain as special case in easystack files (#3543)
- enhance extract_cmd function to use 'cp -a' for shell scripts (.sh) (#3545)
- allow use of alternate envvar(s) to $HOME for user modules (#3558)
- use https://sources.easybuild.io as fallback source URL (#3572, #3576)
- add toolchain definition for iibff toolchain (#3574)
- add %(cuda_cc_space_sep)s and %(cuda_cc_semicolon_sep)s templates (#3578)
- add support for intel-compiler toolchain (>= 2021.x versions, oneAPI) (#3581, #3582)
- various bug fixes, including:
- add --init and --recursive options to 'git submodule update' command that is used when creating source tarball for specific commit (#3537)
- filter out duplicate paths in RPATH wrapper script (#3538)
- don't clean up imported modules after verifying imports of included Python modules (#3544)
- avoid no-op changes to $LD_* environment variables in ModulesTool (#3553)
- fix UTF-8 encoding errors when running EasyBuild with Python 3.0.x-3.6.x (#3565)
- create lib64 symlink as a relative symlink (#3566)
- don't reuse variable name in the loop to fix adding extra compiler flags via toolchainopts (#3571)
- symlink 'lib' to 'lib64' if it doesn't exist (#3580)
- include %(mpi_cmd_prefix)s and %(cuda_*)s templates in output of --avail-easyconfig-templates (#3586)
- other changes:
- rename EasyBlock._skip_step to EasyBlock.skip_step, to make it part of the public API (#3561)
- make symlinking of posix_c.so to posix.so in test suite configuration conditional (#3570)
- use 'main' rather than 'master' branch in GitHub integration functionality (#3589)
v4.3.2 (December 10th 2020)
---------------------------
update/bugfix release
- add (experimental) support for specifying easyconfig files via an "easystack" file (#3479, #3511, #3515, #3517, #3520, #3521)
- see also https://easybuild.readthedocs.io/en/latest/Easystack-files.html
- add definition for new 'gobff' toolchain using BLIS and LibFLAME (#3505)
- various enhancements, including:
- add support for toolchain options like 'extra_cxxflags' to specify extra compiler options (#2193)
- fix combination of --copy-ec and --from-pr (#3482)
- enhance copy_files function: support single file target, error on empty input list, support verbose mode (#3483)
- cache result of fetch_files_from_pr function (mainly to speed up framework test suite) (#3484)
- add locate_files function to filetools module (#3485)
- add support for %(module_name)s template value (#3497)
- clarify input format for --cuda-compute-capabilities in 'eb --help' output (#3509)
- add support for skiping unit tests (test step) via --skip-test-step (#3524)
- various bug fixes, including:
- also ignore vsc.* imports coming from from pkg_resources/__init__.py (setuptools) in fake vsc namespace (#3491)
- don't pass username in github_api_get_request when no GitHub token is available (#3494)
- also inject -rpath options for all entries in $LIBRARY_PATH in RPATH wrappers (#3495)
- avoid TypeError being raised by list_toolchains (#3499)
- check if PR is already merged in --merge-pr (#3502)
- graciously handle wrong PR id in fetch_pr_data (#3503)
- fix regression in apply_regex_substitutions: also accept list of paths to patch (#3507)
- update installation procedure for EasyBuild in generated Singularity container recipes (#3510)
- fix GitHub Actions workflow for test suite: run outside of repo checkout + also test bootstrap script with Python 3.9 (#3518)
- bump cryptography from 2.9.2 to 3.2 for Python 2 in requirements.txt (#3519)
- fix 'eb --help=rst' when running with Python 3 (#3525)
- other changes:
- exclude test configurations with Lmod 7 and Python 3, except for Python 3.6 (#3496)
- significantly speed up parsing of easyconfig files by only extracting comments from an easyconfig file when they're actually needed (#3498)
- don't include file/ldd/readelf commands run during RPATH sanity check in --trace output (#3508)
v4.3.1 (October 29th 2020)
--------------------------
update/bugfix release
- various enhancements, including:
- further GCC toolchain fixes for aarch64 (#3433)
- take into account --include-easyblocks-from-pr when uploading test reports (#3446)
- add path to pkg-config files in sysroot to $PKG_CONFIG_PATH when --sysroot is specified (#3451)
- add support for NVHPC compiler + toolchain (based on PGI) (#3454)
- check for _VERSION and _PREFIX Cray environment variables with both software and module name (#3460)
- allow including easyblocks from multiple PRs (#3480, #3481)
- various bug fixes, including:
- avoid UnicodeDecodeError in apply_regex_substitutions when patching files that include non-UTF-8 characters (#3450)
- avoid appending lib stubs pattern to RPATH filter over and over again (#3452)
- fix missing string template on error for incorrect extension 'sources' value (#3461)
- fix compatibility with Python 3.9 by renaming fancy root logger (#3465)
- also remove empty checksums list specified in easyconfig file when using --inject-checksums (#3466)
- avoid confusing error log message when determining impi version while trying to define value for %(mpi_cmd_prefix)s template (#3474)
- unset $LD_LIBRARY_PATH when checking for OS dependencies with 'rpm' & co (#3477)
- don't change directory in download_repo function in tools.github (#3486)
- take source_urls, checksums, patches into account when extension source is specified via 'sources' (#3487)
- other changes:
- consider $EB_INSTALLPYTHON in 'eb' command to specify 'python' command to use for running EasyBuild (#3428)
- use only the sub folder name for createSubmoduleDeps script (#3464)
v4.3.0 (September 13th 2020)
----------------------------
feature release
- various enhancements, including:
- add script to create 'sources' entries for git submodules (#3369, #3436)
- add templates for CUDA compute capabilities (#3382)
- '%(cuda_compute_capabilities)s', '%(cuda_sm_comma_sep)s', '%(cuda_sm_space_sep)s'
- add EasyBuild configuration option --generate-devel-module (#3388)
- to allow disabling generating of "devel" modules via --disable-generate-devel-module
- set up a minimal build environment when using system compiler (#3399)
- by default, $CC is set to 'gcc' and $CXX is set to 'g++' when using 'system' toolchain
- minimal build environment can be customized via --minimal-build-env configuration option
- add --sysroot configuration option to specify alternative location of system root (#3419)
- this is useful when installing software in a Gentoo Prefix environment (for example)
- various bug fixes, including:
- allow including easyblocks from multiple locations, by combining --include-easyblocks and --include-easyblocks-from-pr (#3311)
- also escape backslashes in quote_py_str() (#3386)
- use one argument 'module swap' statements in Tcl modulefiles (required by Modules 4.2.3+) (#3397)
- fix copying of (non-existing) file with apply_patch (#3400)
- create symlink from lib64 to lib subdir in installation directories to avoid that GCC prefers /lib64 system directories (#3401)
- fix default value for 'lib64_fallback_sanity_check' build option (#3402)
- correctly determine commit status in --merge-pr (#3406)
- stop installing ccache wrapper for Fortran compiler (#3409)
- fix issues with applying PR patch in --from-pr (#3414)
- make RPATH wrapper script more robust by using 'python -E -s -S' to run rpath_args.py (#3422)
- don't inject -Wl,-rpath options when '-x c++-header' compiler option is used (#3424)
- fix lack of -mno-recip on aarch64 by tweaking GCC options used when 'precise' toolchain option is enabled (#3425)
- make sure self.start_dir is set in ExtensionEasyBlock (#3426, #3435)
- exclude local variables from typo check in easyconfig files (#3427)
- other changes:
- stick to cryptography 2.9.2 when using Python 2.7 to avoid broken test suite in CI (#3392)
- automatically enable --ignore-locks with --fetch (#3404)
- switch to status badge based on tests run in GitHub Actions CI in README (#3415)
- make flake8 code style checks pass (#3416, #3417)
- limit Travis to only test with Python 2.6 + Lomod 7 (#3418)
- ignore deprecation warning raised for cryptography when using Python 3.5 produced by output of test suite (#3423)
v4.2.2 (July 8th 2020)
----------------------
update/bugfix release
- various enhancements, including:
- add support for using 'sources' and 'git_config' for extensions in 'exts_list' (#3294)
- add support for software minver template (like %(pyminver)s, etc.) (#3344, #3345)
- add support for updating dictionary or tuple easyconfig parameters with self.cfg.update (#3356)
- various bug fixes, including:
- fix crash in --avail-easyconfig-constants when using --output-format=rst + ensure sorted output (#3341)
- always take into account builddependencies when generating template values, also when we're not iterating over builddependencies (#3346)
- fix running command as 'easybuild' user in generated Singularity definition file (#3347)
- allow ignoring versionsuffix in --try-update-deps (#3350, #3353)
- retain order of paths when generating prepend_path statements for module file (don't sort them alphabetically) (#3367)
- also put easyblocks used by extensions in 'reprod' directory (#3375)
- also copy template values in EasyConfig.copy method to ensure they are defined when installing extensions (#3377)
- skip lines that start with 'module-version' when determining whether a module exists in ModulesTool.exist (#3379)
v4.2.1 (May 20th 2020)
----------------------
update/bugfix release
- various enhancements, including:
- also mention CPU architecture (x86_64, POWER) in comment for test reports (#3281)
- add support for enhancing existing sanity check in easyconfigs, as opposed to overwriting paths/commands (#3288)
- clean up locks when EasyBuild session is cancelled with a signal like SIGTERM (#3291, #3321)
- add 'find_glob_pattern' function to filetools module (#3297)
- add constants for common OS dependencies (OS_PKG_IBVERBS_DEV, OS_PKG_OPENSSL_DEV, ...) (#3309, #3334)
- flesh out get_mpi_cmd_template function from Mpi.mpi_cmd_for method (#3312)
- add variable 'moddependpaths' to specify extra $MODULEPATH entry to consider for loading dependency modules (#3324)
- allow copying of tweaked easyconfigs when using --try-* with --copy-ec (#3332)
- various bug fixes, including:
- make ModulesTool.exist more robust w.r.t. module wrappers, aliases, defaults, etc. (#3216, #3337)
- clean up rst output of --list-toolchains (#3246)
- cast CPU arch name provided by archspec to a regular string (#3286)
- get pr_title and pr_descr built_options in new_pr_from_branch instead of new_pr (and commit_msg in both) (#3298)
- make pypi_source_urls more robust by using HTMLParser rather than xml.etree.ElementTree (#3303, #3329)
- fix broken test for --include-easyblocks-from-pr (#3304)
- don't use distutils.dir_util in copy_dir (#3310)
- print trace message for sanity check command before running it (#3316)
- fix problems with processing of easyconfigs using a Cray* toolchain when there are no actual external modules (#3319)
- make test_find_eb_script more robust in case $EB_SCRIPT_PATH is already set (#3320)
- fix several small problems wit --try-update-deps (experimental feature) (#3325, #3326, #3330)
- add --disable-job in 'eb' command used in jobs, to prevent infinite job cycle (#3328)
- avoid empty entries in $LD_LIBRARY_PATH and other path-like environment variables (#3333)
- other changes:
- fix code style issues in easybuild.tools + add flake8 linting test (#3282)
- introduce contextmanager for disabling templating and reduce resolving errors (#3287)
- add 'change_into_dir' named argument to 'extract_file' + print deprecation warning if it's not specified (#3292)
- improve install_eb_dep.sh script to install EasyBuild dependencies in CI environment (#3314)
v4.2.0 (April 14th 2020)
------------------------
feature release
- add support for --try-update-deps (experimental feature), to upgrade dependencies based on available easyconfigs (#2599)
- adding locking to prevent two parallel builds of the same installation directory (#3009)
- for more information, see https://easybuild.readthedocs.io/en/latest/Locks.html
- significantly speed up -D/--dry-run by avoiding useless 'module show' (#3203)
- add support for creating an index & using it when searching for easyconfigs (#3210)
- for more information, see https://easybuild.readthedocs.io/en/latest/Easyconfigs_index.html
- additional GitHub integration features:
- add support for targeting easyblocks and framework repositories in --new-pr (#1876, #3189)
- add support for --include-easyblocks-from-pr (#3206)
- for more information, https://easybuild.readthedocs.io/en/latest/Integration_with_GitHub.html
- various other enhancements, including:
- add a contrib/hooks dir with some examples of hooks used (#2777)
- also mention working directory + input passed via stdin (if any) in trace output of run_cmd (#3168)
- probe external modules for missing metadata that is not provided via extermal module metadata file (#3174)
- also update $CMAKE_PREFIX_PATH and $CMAKE_LIBRARY_PATH in generated module file (#3176)
- optionally call log.warning in print_warning (#3195)
- add an option to git_config to retain the .git directory (#3197)
- allow use of SYSTEM as --try-toolchain option (#3213)
- mention CPU arch name in comment for uploaded test report, if it's known by archspec (#3227)
- make --merge-pr take into account --pr-target-branch (#3236)
- make --check-contrib print a warning when None is used for checksums (#3244)
- update install-EasyBuild-develop.sh script and create script for 2020a merge sprint (#3245)
- add template for mpi_cmd_prefix (#3264)
- update copy_dir to include option to merge directories (#3270)
- support template name for CUDA version (#3274)
- various bug fixes, including:
- use correct module for errors_found_in_log (#3119)
- fix EasyConfig.update code to handle both strings and lists as input (#3170)
- fix removing temporary branch on --check-github (#3182)
- fix shebang even if first line doesn't start with '#!' (#3183)
- make boostrap script work with Python 3 (#3186)
- read patch files as bytestring to avoid UnicodeDecodeError for patches that include funky characters (#3191)
- fix online check in --check-github: try repeatedly and with different URLs to cater for HTTP issues (#3194)
- don't crash when GitPython is not installed when using Python 3 (#3198)
- fix os_name_map for RHEL8 (#3201)
- don't add shebang to binary files (#3208)
- use checkout@v2 in GitHub Actions to fix broken re-triggered tests (#3219)
- don't filter out None values in to_checksums, leave them in place (#3225)
- fix defining of $MPI_INC_* and $MPI_LIB_* environment variables for impi version 2019 and later (#3229)
- use correct target account/repo when creating test report & posting comment in PR (#3234)
- reorganize EasyBlock.extensions_step to ensure correct filtering of extensions (#3235)
- also support %(installdir)s and %(builddir)s templates for extensions (#3237)
- unset $GITHUB_TOKEN in Travis after installing token, to avoid failing test_from_pr_token_log (#3252)
- fix reporting when skipping extensions (#3254)
- avoid API rate limit errors on online check in --check-github (#3257)
- show easyconfig filenames for parallel build (#3258)
- make EasyConfig.dump aware of toolchain hierarchy, to avoid hardcoded subtoolchains in dependencies easyconfig parameters (#3261)
- fix use of --copy-ec with a single argument, assume copy to current working directory (#3262)
- fix posting of comment in PR with --upload-test-report (#3272)
- take into account dependencies marked as external modules when composing template values like %(pyver)s (#3273)
- other changes:
- increase timeout & use api.github.com for connectivity check in check_github (#3192)
- restore flake8 default ignores (#3193)
- enable tracking of test suite coverage using coveralls (#3204)
- make tests use easybuilders/testrepository rather than hpcugent/testrepository after it was moved (#3238)
- improve raised error in remove_dir and deprecate rmtree2 (#3228)
v4.1.2 (March 16th 2020)
------------------------
bugfix release
- fix gitdb dependency on Python 2.6 in test configuration (#3212)
- fix broken test for --review-pr by using different PR to test with (#3226)
- censor authorization part of headers before logging ReST API request (#3248)
- see also https://github.com/easybuilders/easybuild-framework/security/advisories/GHSA-2wx6-wc87-rmjm
v4.1.1 (January 16th 2020)
--------------------------
update/bugfix release
- various enhancements, including:
- add check_log_for_errors function (in easybuild.tools.run) to detect and handle multiple errors (#3118)
- implement support for 'eb --show-ec' to show contents of specified easyconfig file (#3132)
- also update $XDG_DATA_DIR (share/) and $GI_TYPELIB_PATH environment variables (lib*/girepository-*) in generated module files (#3133)
- add support for --copy-ec to copy easyconfig file to specified location (#3142)
- mention --disable-* option in --help output for boolean options enabled by default (#3151)
- add --cuda-compute-capabilities configuration option (#3161)
- various bug fixes, including:
- ignore imports from vsc namespace made from pkgutil.py (#3120)
- only actually change permissions using os.chmod in adjust_permissions if the current permissions are not correct already (#3125)
- use shutil.copyfile to just copy file contents if target path exists and is owned by someone else (#3127)
- fix or avoid warnings that commonly arise in build log (#3129)
- disable buffering in asyncprocess.Popen using bufsize=0, to fix run_cmd_qa missing output (#3130)
- update pip & install wheel package in generated Singularity container recipes (#3136)
- avoid crash in modify_env & unset unset_env_vars when using (older versions) of Python 3.5 & 3.6 by using list(...) (#3140)
- take into account that lib64 could be a symlink to lib (or vice versa) in get_software_libdir function (#3141)
- only parse docstring if it exists in gen_easyblock_doc_section_rst function (#3144)
- only add useful entries for $CPATH, $(LD_)LIBRARY_PATH and $PATH (non-empty directories) (#3145, #3152)
- fix --list-software=detailed when using Python 3 by leveraging sort_looseversions function from py2vs3 module (#3146)
- ensure subdirectories in software install directory have correct search (exec) permission (#3147)
- take into account that a checksum value may be a tuple of valid checksum in EasyBlock.check_checksums (#3153)
- other changes:
- bump to Lmod 8.2.9 in GitHub CI config (#3115)
- update copyright statements for 2020 (#3149)
- make Hound CI code style checker ignore "Black would make changes" produced by flake8-black (#3162)
v4.1.0 (December 4th 2019)
--------------------------
update/bugfix release
- various enhancements, including:
- performance improvements:
- skip validation when copying EasyConfig object for extension (#3071)
- correctly specify that ActiveMNS & co are singleton classes when using Python 3 (#3073)
- don't call out to prohibitively expensive getRootLoggerName in getLogger, just use 'root' instead (#3074)
- fix inconsistent module path usage that leads to repeated reloading in HMNS (#3099)
- add support for specifying different dependency version based on processor architecture (#3047)
- support use of glob patterns for paths to files with external modules metadata (#3075)
- take into account that external modules may not be visible directly (due to module hierarchy) (#3083)
- add support for including 'extensions' statement in Lua modules with Lmod 8.2.8+ (#3085, #3107, #3110)
- add support for --sync-pr-with-develop (#3087)
- add support for --new-branch-github, --new-pr-from-branch, --sync-branch-with-develop, --update-branch-github (#3103)
- fix typo in docstring for new_branch_github (#3106)
- various bug fixes, including:
- correctly handle iccifortcuda toolchain with standalone iccifort in det_toolchain_compilers (#3055)
- init git repo with git.repo.clone() instead of copy_dir() (#3062)
- fix regular expression so depends-on statements are recognized correctly in Tcl module files (#3065)
- update GitPython to latest version that supports Python 2.6 in requirements.txt to fix broken test_new_update_pr (#3066)
- imply --disable-pre-create-installdir with --inject-checksums (#3069)
- handle patches in extensions more like normal patches (#3067)
- take into account that platform.linux_distribution and platform.dist was removed in Python 3.8 (#3078)
- always include mandatory easyconfig parameters in dumped easyconfig (#3081)
- hide backup module file when using Lmod 6.x (fixes #9302) (#3089)
- add better error message when mandatory key is missing from a dictionary easyconfig parameter (#3092)
- also create symlinks for default modules in class module folders (#3093)
- fix semantics of --set-default-module: only set default for specified easyconfigs, not for the ones that are installed as dependencies via --robot (#3094)
- fix various issues in extracting comments from original easyconfig file and including them again in dumped easyconfig (#3095)
- don't use %(version)s template in exts_default_options in dumped easyconfig (#3096)
- fix generated module statements in case only a single version is listed in multi_deps (#3097)
- fix broken test_show_system_info on macOS due to 'Python' binary (#3105)
- take into account that dependency version could be a dict rather than a string value in template_constant_dict (#3111)
- other changes:
- deprecate running EasyBuild with Python 2.6 via new check_python_version() function (#3076)
- deprecate support for using Lmod 6.x (#3077)
- trim set of test configurations in Travis CI (#3086)
- flesh out env_vars_external_module from Toolchain._simulated_load_dependency_module (#3088)
v4.0.1 (October 15th 2019)
--------------------------
update/bugfix release
- various enhancements, including:
- add 'parallel' to list of config templates (#3036)
- add GitHub Actions workflow to run easybuild-framework test suite (#3039)
- add 'retest' as a reason to --close-pr, to close/re-open PRs to trigger re-test in Travis (#3040)
- define $EB_SCRIPT_PATH in 'eb' wrapper script, and consider it before location of 'eb' determined via $PATH in get_paths_for function (#3046)
- add support for --remove-ghost-install-dirs configuration option, and warn about (potential) ghost install dirs by default when --force/--rebuild is used (#3050)
- various bug fixes, including:
- update bootstrap script to support installing EasyBuild v4.0 (#3017)
- fix broken test_download_repo due to archiving of easyconfigs (#3019, #3023)
- avoid that --inject-checksums introduces list of patches for extensions as a single long line (#3025, #3034)
- enhance regex in fix_shebang method to fix more Python/Perl shebangs + avoid patching binary files (#3029)
- delete test gist that is created by --check-github (#3031)
- disable templates when defining easyconfig parameters in EasyConfig.set_keys() (#3037)
- avoid setting GC3Pie's max_in_flight to None if --job-max-jobs is not specified (#3038)
- fix use of obtain_file method for extensions (#3042)
- error out if some GC3Pie job failed (#3044)
v4.0.0 (September 20th 2019)
----------------------------
feature release (incl. backwards-incompatible changes)
A concise overview of major changes in EasyBuild v4.0 is available at https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-of-changes.html .
- fixed compatibility with Python 3.5+ (#2708, #2713, #2714, #2719, #2721, #2723, #2729, #2743, #2744, #2751, #2756, #2759, #2761, #2762, #2765, #2766, #2767, #2768, #2774, #2775, #2778, #2780, #2785, #2787, #2789, #2791, #2792, #2794, #2800, #2801, #2805, #2806, #2895, #2932, #2982, #2992, #3007, #3011)
- supported Python versions: 2.6, 2.7, 3.5, 3.6, 3.7
- some functionality from the Python standard library should be imported from the new easybuild.tools.py2vs3 package
- see https://easybuild.readthedocs.io/en/latest/Python-2-3-compatibility.html & https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-relocated-functions-constants.html for more information
- ingested relevant code from vsc-base & vsc-install (#2708, #2713, #2714, #2763, #2790, #2993)
- vsc-base & vsc-install are no longer required as dependencies
- the functionality that was provided by these packages is now (mostly) available from the easybuild.base package
- any import statements from the 'vsc' namespace will result in an error
- see also https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-of-changes.html#ingested-functionality-from-vsc-base-and-vsc-install and https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-relocated-functions-constants.html
- setuptools is longer required (neither for installing or using EasyBuild) (#2836, #2837, #2984, #2986, #2988)
- see also https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-of-changes.html#no-more-required-python-packages
- this required moving some classes and constants, see https://easybuild.readthedocs.io/en/latest/EasyBuild4-overview-relocated-functions-constants.html
- the 'dummy' toolchain is deprecated and replaced by the 'system' toolchain (#2877, #3001)
- if 'dummy' is still used as a toolchain in easyconfig files, a warning will be printed
- use "toolchain = SYSTEM" instead
- for more information, see https://easybuild.readthedocs.io/en/latest/System_toolchain.html
- a warning is printed when local variables in easyconfig file don't follow the recommended naming scheme (#2938, #2968)