Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package OpenBLAS and use OpenBLAS in scipy #3331

Merged
merged 113 commits into from Apr 12, 2023
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
113 commits
Select commit Hold shift + click to select a range
50fe021
Add OpenBLAS package
lesteve Dec 9, 2022
380b1c7
Remove some stuff
lesteve Dec 9, 2022
c807d85
Switch to OpenBLAS develop branch
lesteve Dec 9, 2022
c57d320
Build openblas in core build
lesteve Dec 9, 2022
f026e63
iter
lesteve Dec 10, 2022
fc771f1
iter
lesteve Dec 10, 2022
c7dce40
Add build for openblas only to quick fail
lesteve Dec 10, 2022
78e8eeb
Copy all .h from openblas in package/.libs/include
lesteve Dec 10, 2022
e8141b8
Attempt at using openblas in scipy
lesteve Dec 10, 2022
125634d
May actually need this
lesteve Dec 10, 2022
b945cca
Revert "Add build for openblas only to quick fail"
lesteve Dec 10, 2022
228f50f
iter
lesteve Dec 10, 2022
7057189
Build scipy too in core + numpy build
lesteve Dec 10, 2022
83a66a5
iter
lesteve Dec 10, 2022
27d38dc
iter
lesteve Dec 10, 2022
1e5743c
Attempt to replace void by int returns
lesteve Dec 12, 2022
ed2a969
Use LDFLAGS
lesteve Dec 12, 2022
285d34e
Remove -std=c++14 flag
lesteve Dec 13, 2022
48ba869
trigger CI
lesteve Dec 13, 2022
27a63ab
wip
lesteve Jan 2, 2023
3267085
Trigger CI
lesteve Jan 23, 2023
d73c122
Merge branch 'main' of https://github.com/pyodide/pyodide into use-op…
lesteve Jan 23, 2023
6ded2d5
Trigger CI
lesteve Jan 23, 2023
02c7f6f
Revert unwanted change?
lesteve Jan 23, 2023
cdf7b3b
wip
lesteve Jan 31, 2023
5960c81
Merge branch 'main' of https://github.com/pyodide/pyodide into use-op…
lesteve Jan 31, 2023
2480b40
Trigger CI
lesteve Jan 31, 2023
f3f01f3
wip
lesteve Feb 6, 2023
9973316
fix
lesteve Feb 6, 2023
35ec411
Merge remote-tracking branch 'upstream/main' into use-openblas
rth Mar 10, 2023
a85f789
wip
lesteve Mar 7, 2023
41fa6d6
Remove scipy_slu_config.h patch
hoodmane Mar 10, 2023
1f04fa9
Try again with scipy_slu_config.h patch
hoodmane Mar 10, 2023
d34a738
Rediff make-int-returns patch
hoodmane Mar 10, 2023
e95b32f
Fix patch again...
hoodmane Mar 10, 2023
579cf40
Remove patch that removes zcopy decl
hoodmane Mar 10, 2023
568ab35
Remove some more header removal parts from the patch
hoodmane Mar 10, 2023
1f4cba0
Fix scipy patch again
hoodmane Mar 10, 2023
7e57cf8
Update ilu_cdrop_row void to int patch
hoodmane Mar 10, 2023
f974023
Use sed to replace some more void return values with ints
hoodmane Mar 10, 2023
2877657
Apply replacement to both .c and .h files
hoodmane Mar 10, 2023
55d370a
More sed for void to int
hoodmane Mar 10, 2023
731e5b6
Add patch for https://github.com/scipy/scipy/pull/18124
hoodmane Mar 10, 2023
561e3db
Add -Wno-parentheses to cflags
hoodmane Mar 10, 2023
61393e8
Add another sed
hoodmane Mar 10, 2023
f5eb7db
More sed
hoodmane Mar 10, 2023
4c3bf9e
More sed
hoodmane Mar 10, 2023
3cd21d7
MORE INT!
hoodmane Mar 10, 2023
2fd7b71
Fix typo
hoodmane Mar 10, 2023
be005b3
Fix declaration of sasum in blas_subroutine wrappers.c
hoodmane Mar 10, 2023
7e912d5
Patch more bad return types in blas_subroutine_wrappers.c
hoodmane Mar 10, 2023
00d2622
Apply doublereal ==> float replacement on all f2c output
hoodmane Mar 10, 2023
5c11873
Don't apply transform to GMRESREVCOM
hoodmane Mar 10, 2023
9d7cbe4
More fixes
hoodmane Mar 10, 2023
a474a9f
Add comment
hoodmane Mar 10, 2023
1e1e4c2
Fix return type of clanhs too screw you f2c
hoodmane Mar 10, 2023
d865c60
Merge branch 'main' of https://github.com/pyodide/pyodide into use-op…
lesteve Mar 31, 2023
966c40b
Use f2c -R to avoid generating doublereal return types instead of float
lesteve Mar 31, 2023
25f8f69
Avoid c_abs symbol conflict
lesteve Mar 31, 2023
eaa1356
Update OpenBLAS to 0.3.22 release
lesteve Mar 31, 2023
9fe97d2
LAPACKE not needed
lesteve Mar 31, 2023
6d0391c
scipy compiles
lesteve Apr 1, 2023
334151c
scipy loads in browser
lesteve Apr 1, 2023
cc68e64
Hacky reuse of CLAPACK to include f2c symbols into libopenblas.so
lesteve Apr 2, 2023
65c3a38
Remove RIMME f2clapack.sh attempt
lesteve Apr 2, 2023
c8dc2d9
Use openblas in suitesparse
lesteve Apr 2, 2023
d418127
Build full scipy
lesteve Apr 2, 2023
648f204
Fix signature mismatch in scipy.stats
lesteve Apr 2, 2023
8e51195
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 2, 2023
feabda1
Remove some unwanted real -> double changes
lesteve Apr 2, 2023
59becc0
Fix issue in arpack (but complext tests fail)
lesteve Apr 3, 2023
5a874a0
gensim work-around
lesteve Apr 3, 2023
9680caf
Remove some stuff from scipy patch without being 100% sure ...
lesteve Apr 3, 2023
fbb27fe
Debug zgetrf issue through Cython scipy linalg bindings
lesteve Apr 5, 2023
b90d7bc
Revert "Debug zgetrf issue through Cython scipy linalg bindings"
lesteve Apr 5, 2023
7bbd289
Update OpenBLAS to 0.3.23 which fixes 0 determinant for complex matrices
lesteve Apr 5, 2023
38d6c55
Update pyodide-build/pyodide_build/_f2c_fixes.py
lesteve Apr 6, 2023
f5bded8
Update packages/CLAPACK/meta.yaml
lesteve Apr 6, 2023
06d7e10
Add comment about f2c -R
lesteve Apr 6, 2023
19548f2
Remove unneeded patches
lesteve Apr 6, 2023
bcf8306
Removed commented out code
lesteve Apr 6, 2023
cd7879e
Remove ilaenv and xerbla f2py wrapper
lesteve Apr 6, 2023
1687559
Rename CLAPACk -> libf2c and remove mentions of CLAPACK where appropr…
lesteve Apr 6, 2023
c387cd8
Reduce scipy subpackages by perl replacement
lesteve Apr 6, 2023
ef87b37
Add tip to build less scipy sub-packages and reduce iteration time
lesteve Apr 6, 2023
63ffba0
Update libf2c patches
lesteve Apr 6, 2023
5d88956
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 6, 2023
9fbe4e4
[ci skip] Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
lesteve Apr 6, 2023
c3a4e06
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Apr 6, 2023
23c4df5
Reinstate prettierignore + tweaks
lesteve Apr 6, 2023
6a2d944
Fix libf2c meta.yaml
lesteve Apr 6, 2023
7db7ff6
Revert unwanted change to make.inc
lesteve Apr 6, 2023
b83ad8b
Change version name ...
lesteve Apr 6, 2023
bc1f360
Update scipy patches
lesteve Apr 6, 2023
610025a
Add scipy Boost patch
lesteve Apr 6, 2023
05e22f9
Remove sed replacement since it is done in a patch
lesteve Apr 6, 2023
eb49920
Use gensim patch rather than cryptic sed replacement
lesteve Apr 6, 2023
0e26683
Remove unneeded flag for OpenBLAS
lesteve Apr 6, 2023
3a4f32f
Minor tweak to scipy tip
lesteve Apr 6, 2023
246ca91
OpenBLAS headers not needed for scipy compilation
lesteve Apr 6, 2023
3b27a89
Add comment about patch included upstream
lesteve Apr 7, 2023
45b8727
Add comment about OpenBLAS flag
lesteve Apr 7, 2023
94c50b0
Add comment about libf2c
lesteve Apr 7, 2023
734be58
Remove unneeded flag
lesteve Apr 7, 2023
7ae7f08
Path xerbla_array rather than regex replacement
lesteve Apr 7, 2023
82f6703
Keep c_abs and z_abs in OpenBLAS
lesteve Apr 7, 2023
608a64d
Fix c_abs signature
lesteve Apr 7, 2023
10f6268
Revert unwanted change
lesteve Apr 7, 2023
6bf026c
Add missing file
lesteve Apr 8, 2023
2d3effe
Trigger CI
lesteve Apr 11, 2023
4b198ff
libf2c is a static library now
lesteve Apr 11, 2023
119038f
Merge branch 'main' of https://github.com/pyodide/pyodide into use-op…
lesteve Apr 11, 2023
f0753d6
Add changelog entry
lesteve Apr 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 10 additions & 1 deletion .github/workflows/main.yml
Expand Up @@ -128,13 +128,22 @@ jobs:
make -C cpython
ccache -s

- name: build openblas
shell: bash -l {0}
run: |
# This is necessary to use the ccache from emsdk
source pyodide_env.sh
ccache -z
PYODIDE_PACKAGES="openblas" make
ccache -s

- name: build Pyodide core + numpy
shell: bash -l {0}
run: |
# This is necessary to use the ccache from emsdk
source pyodide_env.sh
ccache -z
PYODIDE_PACKAGES="core,numpy" make
PYODIDE_PACKAGES="core,numpy,openblas" make
ccache -s

- name: check-size
Expand Down
24 changes: 24 additions & 0 deletions packages/openblas/meta.yaml
@@ -0,0 +1,24 @@
package:
name: openblas
version: de4d5646eb1ac6a0e663a4c9bc797dad8226aedb

source:
sha256: 148a959b61ffa4d77e810ceb3114765501d70c982cbb302a00954b81bc918e46
url: https://github.com/xianyi/OpenBLAS/archive/de4d5646eb1ac6a0e663a4c9bc797dad8226aedb.zip
patches:
- patches/0001-Add-Wno-implicit-function-declaration-flag.patch

build:
type: shared_library
script: |
# seems like .zip does not maintain executable flags, need to reset these
chmod u+x c_check
chmod u+x f_check
chmod u+x exports/gensymbol
make CC=emcc HOSTCC=gcc TARGET=RISCV64_GENERIC NOFORTRAN=1 USE_THREAD=0
mkdir -p dist
cp libopenblas.so dist/
mkdir -p ${WASM_LIBRARY_DIR}/{lib,include}
# TODO do I need include as in CLAPACK?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have guessed yes, but you can also probably not include it and see if there are any related include errors as done now.

# cp -r INCLUDE/* ${WASM_LIBRARY_DIR}/include
cp dist/libopenblas.so ${WASM_LIBRARY_DIR}/lib
@@ -0,0 +1,25 @@
From 09fd1aa0aa6a98e1cebaa6e34fca1e424dab8f48 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Lo=C3=AFc=20Est=C3=A8ve?= <loic.esteve@ymail.com>
Date: Fri, 9 Dec 2022 16:40:13 +0100
Subject: [PATCH] Add -Wno-implicit-function-declaration flag

---
Makefile.rule | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile.rule b/Makefile.rule
index 5f787a9c..6890046a 100644
--- a/Makefile.rule
+++ b/Makefile.rule
@@ -228,7 +228,7 @@ NO_AFFINITY = 1
# Common Optimization Flag;
# The default -O2 is enough.
# Flags for POWER8 are defined in Makefile.power. Don't modify COMMON_OPT
-# COMMON_OPT = -O2
+COMMON_OPT = -O2 -Wno-implicit-function-declaration

# gfortran option for LAPACK to improve thread-safety
# It is enabled by default in Makefile.system for gfortran
--
2.34.1