PyPI wheels M1 Pro

[embarrassedMath]

As the title implies, I am attempting to install pyopencl via PyPI on Mac ARM64.
While the installation appears to be successful, I encounter crashes when either when compiling OpenCL source containing an error or when passing a non buffer argument to a kernel -- such as a float.
In both cases, python crashes with the error zsh: trace trap python3 ...; and the intersection of both stacks starts with:

0   libunwind.dylib               	       0x193d9eb4c _Unwind_GetIP + 244
1   libc++abi.dylib               	       0x187568f40 __gxx_personality_v0 + 296
2   libunwind.1.0.dylib           	       0x10427c770 unwind_phase2 + 388
3   libunwind.1.0.dylib           	       0x10427c9a8 _Unwind_Resume + 100

What I am doing:

python -m venv .venv
source .venv/bin/activate
pip install --no-cache pyopencl

When pip installs pyopencl no wheels matching my system where found, thus pip compiles and builds these wheels from source.
I suspect that my issue lives here, but I am not familiar enough to troubleshoot this further.

How I am testing:
I've made the following scripts to test against.
pyopencl_test_argument.py

import pyopencl as cl
import pyopencl.array as cl_array
import numpy as np


source = """
kernel void sum(global const float *a,
                global const float *b, 
                global float *c ) {
    int gid;

    gid = get_global_id(0);
    
    c[gid] = a[gid] + b[gid];
}

kernel void sum_constant(global const float *a,
                 const float x,
                 global float *c ) {
    int gid;
    
    gid = get_global_id(0);

    c[gid] = a[gid] + x;
}
"""


def main():
	ctx = cl.create_some_context()
	queue = cl.CommandQueue(ctx)
	prg = cl.Program(ctx, source).build()
	
	a = np.random.rand(50000).astype(np.float32)
	b = np.random.rand(50000).astype(np.float32)
	x = 5

	a_dev = cl_array.to_device(queue, a)
	b_dev = cl_array.to_device(queue, b)
	dest_dev = cl_array.empty_like(a_dev)

	prg.sum(queue, a.shape, None, a_dev.data, b_dev.data, dest_dev.data)

	assert np.all( dest_dev.get() == (a + b)  )

	prg.sum_constant(queue, a.shape, None, 
  	       a_dev.data, 
		   np.float32(x), 
      	   dest_dev.data )

	assert np.all( dest_dev.get() == (a + x)  )


if __name__ == "__main__":
	main()

pyopencl_test_build_error.py

import pyopencl as cl
import pyopencl.array as cl_array
import numpy as np


source_without_error = """
kernel void sum(global const float *a,
                global const float *b, 
                global float *c ) {
    int gid;

    gid = get_global_id(0);
    
    c[gid] = a[gid] + b[gid];
}
"""


source_with_error = """
kernel void sum(global const float *a,
                global const float *b, 
                global float *c ) {
    int gid

    gid = get_global_id(0);
    
    c[gid] = a[gid] + b[gid];
}
"""


def test_sum(ctx, queue, source):
	
	prg = cl.Program(ctx, source).build()
	
	a = np.random.rand(50000).astype(np.float32)
	b = np.random.rand(50000).astype(np.float32)

	a_dev = cl_array.to_device(queue, a)
	b_dev = cl_array.to_device(queue, b)
	dest_dev = cl_array.empty_like(a_dev)

	prg.sum(queue, a.shape, None, a_dev.data, b_dev.data, dest_dev.data)

	assert np.all( dest_dev.get() == (a + b)  )


def main():	
	ctx = cl.create_some_context()
	queue = cl.CommandQueue(ctx)
	
	test_sum( ctx, queue, source_without_error )

	try:
		test_sum( ctx, queue, source_with_error )
	except cl.RuntimeError as e:
		...



if __name__ == "__main__":
	main()

The crash reports stack trace for these are:
python pyopencl_test_argument.py

0   libunwind.dylib               	       0x193d9eb4c _Unwind_GetIP + 244
1   libc++abi.dylib               	       0x187568f40 __gxx_personality_v0 + 296
2   libunwind.1.0.dylib           	       0x105fd4770 unwind_phase2 + 388
3   libunwind.1.0.dylib           	       0x105fd49a8 _Unwind_Resume + 100
4   _cl.cpython-311-darwin.so     	       0x11c8bbb84 void pybind11::detail::initimpl::constructor<pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>::execute<pybind11::class_<pyopencl::kernel>, 0>(pybind11::class_<pyopencl::kernel>&)::'lambda'(pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)::operator()(pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const + 280
5   _cl.cpython-311-darwin.so     	       0x11c8bb940 void pybind11::cpp_function::initialize<void pybind11::detail::initimpl::constructor<pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>::execute<pybind11::class_<pyopencl::kernel>, 0>(pybind11::class_<pyopencl::kernel>&)::'lambda'(pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&), void, pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::detail::is_new_style_constructor>(pybind11::class_<pyopencl::kernel>&&, void (*)(pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::detail::is_new_style_constructor const&)::'lambda'(pybind11::detail::function_call&)::operator()(pybind11::detail::function_call&) const + 172
6   _cl.cpython-311-darwin.so     	       0x11c8bb888 void pybind11::cpp_function::initialize<void pybind11::detail::initimpl::constructor<pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&>::execute<pybind11::class_<pyopencl::kernel>, 0>(pybind11::class_<pyopencl::kernel>&)::'lambda'(pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&), void, pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::detail::is_new_style_constructor>(pybind11::class_<pyopencl::kernel>&&, void (*)(pybind11::detail::value_and_holder&, pyopencl::program const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::detail::is_new_style_constructor const&)::'lambda'(pybind11::detail::function_call&)::__invoke(pybind11::detail::function_call&) + 24
7   _cl.cpython-311-darwin.so     	       0x11c850690 pybind11::cpp_function::dispatcher(_object*, _object*, _object*) + 3644
8   Python                        	       0x104fe6b44 cfunction_call + 60
9   Python                        	       0x104f9cca4 _PyObject_Call + 124
10  Python                        	       0x104f9be8c _PyObject_MakeTpCall + 128
11  Python                        	       0x105079e6c _PyEval_EvalFrameDefault + 42120
12  Python                        	       0x10507e554 _PyEval_Vector + 116
13  Python                        	       0x104f9c138 _PyObject_FastCallDictTstate + 96
14  Python                        	       0x1050060e8 slot_tp_init + 188
15  Python                        	       0x104ffe598 type_call + 136
16  _cl.cpython-311-darwin.so     	       0x11c84c2a4 pybind11_meta_call + 40
17  Python                        	       0x104f9be8c _PyObject_MakeTpCall + 128
18  Python                        	       0x105079e6c _PyEval_EvalFrameDefault + 42120
19  Python                        	       0x10507e554 _PyEval_Vector + 116
20  Python                        	       0x104f9f320 method_vectorcall + 308
21  Python                        	       0x10500a334 call_attribute + 144
22  Python                        	       0x105005250 slot_tp_getattr_hook + 412
23  Python                        	       0x104fecc08 _PyObject_GetMethod + 348
24  Python                        	       0x1050794c8 _PyEval_EvalFrameDefault + 39652
25  Python                        	       0x10506eda0 PyEval_EvalCode + 168
26  Python                        	       0x1050c57a0 run_eval_code_obj + 84
27  Python                        	       0x1050c5704 run_mod + 112
28  Python                        	       0x1050c5544 pyrun_file + 148
29  Python                        	       0x1050c4f94 _PyRun_SimpleFileObject + 268
30  Python                        	       0x1050c492c _PyRun_AnyFileObject + 216
31  Python                        	       0x1050e150c pymain_run_file_obj + 220
32  Python                        	       0x1050e0e4c pymain_run_file + 72
33  Python                        	       0x1050e072c Py_RunMain + 704
34  Python                        	       0x1050e1868 Py_BytesMain + 40
35  dyld                          	       0x187235058 start + 2224

python pyopencl_test_build_error.py

0   libunwind.dylib               	       0x193d9eb4c _Unwind_GetIP + 244
1   libc++abi.dylib               	       0x187568f40 __gxx_personality_v0 + 296
2   libunwind.1.0.dylib           	       0x102720770 unwind_phase2 + 388
3   libunwind.1.0.dylib           	       0x1027209a8 _Unwind_Resume + 100
4   _cl.cpython-311-darwin.so     	       0x102f22c08 pyopencl::program::build(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object) + 1088
5   _cl.cpython-311-darwin.so     	       0x102f4145c void pybind11::detail::argument_loader<pyopencl::program*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object>::call_impl<void, pybind11::cpp_function::cpp_function<void, pyopencl::program, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg_v, pybind11::arg_v>(void (pyopencl::program::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg_v const&, pybind11::arg_v const&)::'lambda'(pyopencl::program*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object)&, 0ul, 1ul, 2ul, pybind11::detail::void_type>(pyopencl::program&&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul>, pybind11::detail::void_type&&) && + 132
6   _cl.cpython-311-darwin.so     	       0x102f41204 void pybind11::cpp_function::initialize<pybind11::cpp_function::cpp_function<void, pyopencl::program, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg_v, pybind11::arg_v>(void (pyopencl::program::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg_v const&, pybind11::arg_v const&)::'lambda'(pyopencl::program*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object), void, pyopencl::program*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg_v, pybind11::arg_v>(void&&, pyopencl::program (*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg_v const&, pybind11::arg_v const&)::'lambda'(pybind11::detail::function_call&)::operator()(pybind11::detail::function_call&) const + 100
7   _cl.cpython-311-darwin.so     	       0x102f41194 void pybind11::cpp_function::initialize<pybind11::cpp_function::cpp_function<void, pyopencl::program, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg_v, pybind11::arg_v>(void (pyopencl::program::*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg_v const&, pybind11::arg_v const&)::'lambda'(pyopencl::program*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object), void, pyopencl::program*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::arg_v, pybind11::arg_v>(void&&, pyopencl::program (*)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, pybind11::object), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::arg_v const&, pybind11::arg_v const&)::'lambda'(pybind11::detail::function_call&)::__invoke(pybind11::detail::function_call&) + 24
8   _cl.cpython-311-darwin.so     	       0x102ed8690 pybind11::cpp_function::dispatcher(_object*, _object*, _object*) + 3644
9   Python                        	       0x101886b44 cfunction_call + 60
10  Python                        	       0x10183be8c _PyObject_MakeTpCall + 128
11  Python                        	       0x101919e6c _PyEval_EvalFrameDefault + 42120
12  Python                        	       0x10190eda0 PyEval_EvalCode + 168
13  Python                        	       0x1019657a0 run_eval_code_obj + 84
14  Python                        	       0x101965704 run_mod + 112
15  Python                        	       0x101965544 pyrun_file + 148
16  Python                        	       0x101964f94 _PyRun_SimpleFileObject + 268
17  Python                        	       0x10196492c _PyRun_AnyFileObject + 216
18  Python                        	       0x10198150c pymain_run_file_obj + 220
19  Python                        	       0x101980e4c pymain_run_file + 72
20  Python                        	       0x10198072c Py_RunMain + 704
21  Python                        	       0x101981868 Py_BytesMain + 40
22  dyld                          	       0x187235058 start + 2224

As an aside, I am able to install and use pyopencl through Conda. Though, we do not use Conda for our project.

[inducer]

A guess I've got is that it's trying to raise an error, but something about raising the exception goes awry, and that's how you get a crash. Are you using the same compilers to build pyopencl as what was used to build your Python interpreter? Are you including the right version of the C++ standard library?

[inducer]

Upon a closer look, I'm actually not sure why Apple Silicon wheels are not being built. cibw claims it can do it. #704 contains some upgrades, maybe that'll help? If so, I'll do a release.

[inducer]

Looks like you need to explicitly request them. I (think I) have added the appropriate snippet to the cibw configuration.

[inducer]

Looks like that worked. https://github.com/inducer/pyopencl/releases/tag/v2023.1.4 should have those built once it ends up on the package index.

[embarrassedMath]

This is certainly a compiler/linker issue. And further, more likely than not an issue which I made for myself. I've since removed and reinstalled any compilers and linkers, namely Homebrew's llvm. This seems to have resolved my issue.
But, I think that your guess was correct. Pyopencl._cl was attempting to raise an exception or error, libunwind was given the job to unwind this, but the linking of these two got borked; resulting in the hard crash.

Having access to the Apple Silicon wheels will be appreciated. I'll be sure to test those soon.