Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE #380

Open
vshuraeff opened this issue Oct 12, 2022 · 8 comments
Open

pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE #380

vshuraeff opened this issue Oct 12, 2022 · 8 comments

Comments

@vshuraeff
Copy link

tried to run under python 3.9.15, installed requirements-full.txt, tried pyopencl versions 2021.1.1; 2021.2.2; but 2022.2.3 is only version which installed without errors.
so I trying to run openCL test and get this:

> btcrecover git:(master) PYOPENCL_COMPILER_OUTPUT=1 python -m btcrecover.test.test_seeds -v OpenCL_Tests 
test_BIP39_BTC_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... ERROR

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform

Stderr:
/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py:270: CompilerWarning: Built kernel retrieved from cache. Original from-source build had warnings:
Build on <pyopencl.Device 'AMD Radeon Pro Vega 48 Compute Engine' on 'Apple' at 0x1021d00> succeeded, but said:

<program source>:59:19: warning: no previous prototype for function 'SWAP'
    unsigned long SWAP (const unsigned long val)
                  ^
<program source>:130:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:131:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:132:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:133:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:394:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_global, __global, __global, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:395:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_private, __private, __private, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:396:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_glbl_to_priv, __global, __private, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:397:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_priv_to_glbl, __private, __global, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:458:27: warning: comparison of integers of different signs: 'int' and 'unsigned long'
        for (int j = 0; j < end; j++){
                        ~ ^ ~~~
<program source>:488:23: warning: comparison of integers of different signs: 'int' and 'unsigned long'
    for (int j = 0; j < m_len_word; j++){
                    ~ ^ ~~~~~~~~~~
<program source>:630:20: warning: initializing '__global unsigned long *' with an expression of type 'unsigned long const __global[4]' discards qualifiers
    __global word *pwdBuffer = pwdbuffer_arg[0].buffer;
                   ^           ~~~~~~~~~~~~~~~~~~~~~~~

  warn(text, CompilerWarning)
test_BIP39_Eth_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... ERROR

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform
test_Electrum_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... ERROR

Stdout:
Using the 'en' wordlist.
OpenCL: Auto Selecting Best Platform

======================================================================
ERROR: test_BIP39_BTC_OpenCL_Brute (__main__.TestRecoveryFromAddress)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 115, in skip_or_test
    test_func(self)
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 1191, in test_BIP39_BTC_OpenCL_Brute
    self.assertEqual(btcrseed.WalletBIP39._return_verified_password_or_false_opencl(wallet,
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/btcrseed.py", line 1000, in _return_verified_password_or_false_opencl
    clResult = self.opencl_algo.cl_pbkdf2(self.opencl_context_pbkdf2_sha512[i], cleaned_mnemonic_ids_list,
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 705, in cl_pbkdf2
    result = concat(self.opencl_ctx.run(bufStructs, func, iter(passwordlist), salt))
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in concat
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in <listcomp>
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 263, in run
    func(self, pwdim, pass_g, salt_g, result_g)
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 702, in func
    prg.pbkdf2(s.queue, pwdim, None, pass_g, salt_g, result_g,
  File "/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py", line 895, in kernel_call
    return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
  File "<pyopencl invoker for 'pbkdf2'>", line 8, in enqueue_knl_pbkdf2
pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform

Stderr:
/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py:270: CompilerWarning: Built kernel retrieved from cache. Original from-source build had warnings:
Build on <pyopencl.Device 'AMD Radeon Pro Vega 48 Compute Engine' on 'Apple' at 0x1021d00> succeeded, but said:

<program source>:59:19: warning: no previous prototype for function 'SWAP'
    unsigned long SWAP (const unsigned long val)
                  ^
<program source>:130:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:131:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl, "")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:132:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__private, printFromWord_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:133:5: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
    def_printFromWord(__global, printFromWord_glbl_n, "\n")
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:116:27: note: expanded from macro 'def_printFromWord'
        for (int j = 0; j < len_bytes; j++){    \
                        ~ ^ ~~~~~~~~~
<program source>:394:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_global, __global, __global, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:395:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_private, __private, __private, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:396:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_glbl_to_priv, __global, __private, md_pad__global, printFromLong_glbl_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:397:1: warning: comparison of integers of different signs: 'int' and 'const unsigned int'
def_hash(hash_priv_to_glbl, __private, __global, md_pad__private, printFromLong_n)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<program source>:330:35: note: expanded from macro 'def_hash'
    for (int block_i = 0; block_i < nBlocks; block_i++)     \
                          ~~~~~~~ ^ ~~~~~~~
<program source>:458:27: warning: comparison of integers of different signs: 'int' and 'unsigned long'
        for (int j = 0; j < end; j++){
                        ~ ^ ~~~
<program source>:488:23: warning: comparison of integers of different signs: 'int' and 'unsigned long'
    for (int j = 0; j < m_len_word; j++){
                    ~ ^ ~~~~~~~~~~
<program source>:630:20: warning: initializing '__global unsigned long *' with an expression of type 'unsigned long const __global[4]' discards qualifiers
    __global word *pwdBuffer = pwdbuffer_arg[0].buffer;
                   ^           ~~~~~~~~~~~~~~~~~~~~~~~

  warn(text, CompilerWarning)

======================================================================
ERROR: test_BIP39_Eth_OpenCL_Brute (__main__.TestRecoveryFromAddress)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 115, in skip_or_test
    test_func(self)
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 1234, in test_BIP39_Eth_OpenCL_Brute
    self.assertEqual(btcrseed.WalletEthereum._return_verified_password_or_false_opencl(wallet,
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/btcrseed.py", line 1000, in _return_verified_password_or_false_opencl
    clResult = self.opencl_algo.cl_pbkdf2(self.opencl_context_pbkdf2_sha512[i], cleaned_mnemonic_ids_list,
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 705, in cl_pbkdf2
    result = concat(self.opencl_ctx.run(bufStructs, func, iter(passwordlist), salt))
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in concat
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in <listcomp>
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 263, in run
    func(self, pwdim, pass_g, salt_g, result_g)
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 702, in func
    prg.pbkdf2(s.queue, pwdim, None, pass_g, salt_g, result_g,
  File "/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py", line 895, in kernel_call
    return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
  File "<pyopencl invoker for 'pbkdf2'>", line 8, in enqueue_knl_pbkdf2
pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE

Stdout:
Using the 'en' wordlist.
Assuming a 12 word mnemonic. (This can be overridden with --mnemonic-length)
OpenCL: Auto Selecting Best Platform

======================================================================
ERROR: test_Electrum_OpenCL_Brute (__main__.TestRecoveryFromAddress)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 115, in skip_or_test
    test_func(self)
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/test/test_seeds.py", line 1279, in test_Electrum_OpenCL_Brute
    self.assertEqual(btcrseed.WalletElectrum2._return_verified_password_or_false_opencl(wallet,
  File "/Users/vshuraeff/work/crypto/btcrecover/btcrecover/btcrseed.py", line 1000, in _return_verified_password_or_false_opencl
    clResult = self.opencl_algo.cl_pbkdf2(self.opencl_context_pbkdf2_sha512[i], cleaned_mnemonic_ids_list,
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 705, in cl_pbkdf2
    result = concat(self.opencl_ctx.run(bufStructs, func, iter(passwordlist), salt))
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in concat
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 524, in <listcomp>
    return [obj for lval in ll for obj in lval]
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 263, in run
    func(self, pwdim, pass_g, salt_g, result_g)
  File "/Users/vshuraeff/work/crypto/btcrecover/lib/opencl_brute/opencl.py", line 702, in func
    prg.pbkdf2(s.queue, pwdim, None, pass_g, salt_g, result_g,
  File "/Users/vshuraeff/.asdf/installs/python/3.9.15/lib/python3.9/site-packages/pyopencl/__init__.py", line 895, in kernel_call
    return self._enqueue(self, queue, global_size, local_size, *args, **kwargs)
  File "<pyopencl invoker for 'pbkdf2'>", line 8, in enqueue_knl_pbkdf2
pyopencl._cl.LogicError: clEnqueueNDRangeKernel failed: INVALID_PROGRAM_EXECUTABLE

Stdout:
Using the 'en' wordlist.
OpenCL: Auto Selecting Best Platform

----------------------------------------------------------------------
Ran 3 tests in 0.235s

FAILED (errors=3)
@vshuraeff
Copy link
Author

downgrading pyopencl to 2022.2.2 (pip3 install pyopencl==2022.2.2) gets
test_BIP39_BTC_OpenCL_Brute (__main__.TestRecoveryFromAddress) ... [1] 39487 segmentation fault PYOPENCL_COMPILER_OUTPUT=1 python -m btcrecover.test.test_seeds -v

@vshuraeff
Copy link
Author

downgrading to pyopencl==2022.1.4
gives the same segmentation fault :( where I need to check, what is going on?

@vshuraeff
Copy link
Author

well, egmentation fault gets when shell is not restarted, after restart it gives same INVALID_PROGRAM_EXECUTABLE on almost any versions of pyopencl in 2021 and 2022 series.

very sad :(

@vshuraeff
Copy link
Author

downgrading python to 3.8.15 gives the same result on mac with vega

@3rdIteration
Copy link
Owner

I haven't tested pyopencl functionality on any Mac platform. Do you get the same behavior in Linux?

@vshuraeff
Copy link
Author

no, I does not trying to test it under Linux, but as I see, only downgrading python to 3.8 gets working btcrecover to bruteforce, but still cant run any tests

@3rdIteration
Copy link
Owner

Just use CPU on Mac

@vshuraeff
Copy link
Author

I can run it on Mac GPU (Vega 48) with about 900 P/s, but I cant run tests, as I describe above.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vshuraeff @3rdIteration