fcrypt / gpgme / crypto: test case fails #2341

markus2330 · 2019-01-26T15:33:54Z

Fcrypt test cases failed on master:

122/131 MemCheck  #93: testmod_fcrypt ...................***Failed   29.99 sec

OK closing connection

Terminating gpg-agent returned with status “256”FCRYPT       TESTS

==================


/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:161: error in test_gpg: failed to install the GPG test key

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:189: error in test_file_crypto_operations: kdb set failed

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:190: error in test_file_crypto_operations: file content did not change during encryption

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:193: error in test_file_crypto_operations: kdb get (pregetstorage) failed

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:197: error in test_file_crypto_operations: kdb get (postgetstorage) failed

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:198: error in test_file_crypto_operations: postgetstorage did not encrypt the file again

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:233: error in test_file_signature_operations: kdb set failed

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:234: error in test_file_signature_operations: file content did not change during encryption

/home/jenkins/workspace/libelektra_master-Q2SIBK3KE2NBEMJ4WVGJXAXCSCB77DUBUULVLZDKHQEV3WNDXBMA/src/plugins/fcrypt/testmod_fcrypt.c:237: error in test_file_signature_operations: kdb get failed


fcrypt Results: 39 Tests done — 9 errors.

https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/master/361/pipeline

@petermax2 any idea?

The text was updated successfully, but these errors were encountered:

markus2330 · 2019-01-26T15:51:29Z

Seems like gpgme is also affected:

 62/137 MemCheck #103: testmod_gpgme ....................***Failed   44.22 sec

GPGME        TESTS

==================

[repetitions cut]


GPGME        TESTS

==================


OK closing connection

Terminating gpg-agent returned with status “256”GPGME        TESTS

==================


/home/jenkins/workspace/libelektra_PR-1950-LKQXQDFYJCAZVRKFRKER3H4ZYYVI5QHMP5O7HG5TCW7WRVITCO5A/src/plugins/gpgme/testmod_gpgme.c:227: error in test_encryption_decryption: kdb get failed

/home/jenkins/workspace/libelektra_PR-1950-LKQXQDFYJCAZVRKFRKER3H4ZYYVI5QHMP5O7HG5TCW7WRVITCO5A/src/plugins/gpgme/testmod_gpgme.c:243: error in test_encryption_decryption: key type not restored to string during decryption

/home/jenkins/workspace/libelektra_PR-1950-LKQXQDFYJCAZVRKFRKER3H4ZYYVI5QHMP5O7HG5TCW7WRVITCO5A/src/plugins/gpgme/testmod_gpgme.c:244: error in test_encryption_decryption: key content not restored during decryption

/home/jenkins/workspace/libelektra_PR-1950-LKQXQDFYJCAZVRKFRKER3H4ZYYVI5QHMP5O7HG5TCW7WRVITCO5A/src/plugins/gpgme/testmod_gpgme.c:254: error in test_encryption_decryption: key content not restored during decryption


gpgme Results: 45 Tests done — 4 errors.

https://build.libelektra.org/jenkins/blue/organizations/jenkins/libelektra/detail/PR-1950/34/pipeline

petermax2 · 2019-01-26T19:32:28Z

@petermax2 any idea?

to be honest, no. The return codes are not documentes, but they could mean:

./src/gpg-error.h.in:968:#define GPGRT_LOG_RUN_DETACHED 256
./src/gpg-error.h.in:1175:#define ARGPARSE_FLAG_STOP_SEEN 256  /* Set to true if a "--" has been seen. */

I could not find the meaning of 256 by skimming through the code.

Also I can not reproduce the problem locally, as usual. 😞

markus2330 · 2019-01-26T19:41:01Z

Thank you for taking a look!

Reproducing will be rather difficult, it seems to only occur sporadically and only on the master branch (@sanssecours or did you see it also within a PR?).

Who prints

OK closing connection

and

Terminating gpg-agent returned with status “256”

?

mpranj · 2019-01-26T19:44:47Z

I saw this sporadically while working on a PR. It vanished after rebuilding.

I think I also saw it on my private jenkins under extremely high load. Is it at all possible that something times out here?

petermax2 · 2019-01-26T19:46:25Z

Is it at all possible that something times out here?

Maybe! Depending on the implementation of libassuan.

petermax2 · 2019-09-14T15:41:06Z

Status update: I still have no idea how to overcome this issue. Any ideas are welcome!

markus2330 · 2019-09-15T04:47:04Z

Thank you for the update!

I think the way forward is to report it to the GPG people. We already know everything relevant, like the exit code. But only they can (if anyone) interpret how this can occur.

If they say: ohh, yes, this error sometimes occurs, we should consider that our plugins run gpg again if this error happens.

markus2330 · 2019-09-15T09:19:20Z

Btw. I am not sure if there are still occurrences of this problem. I did not see them recently.

sanssecours · 2019-09-15T10:58:17Z

Btw. I am not sure if there are still occurrences of this problem. I did not see them recently.

We disabled testmod_fcrypt on the Jenkins build server.

markus2330 · 2019-09-15T11:23:30Z

Thank you for the reminder. And on the other build systems the problem never occurred?

sanssecours · 2019-09-15T11:37:09Z

And on the other build systems the problem never occurred?

It might have, but according to the description of issue #2439 testmod_fcrypt only fails regularly on the Jenkins build server.

markus2330 · 2019-09-15T18:07:35Z

If someone notices a fail, please post it here.

petermax2 · 2020-04-13T19:29:25Z

We also observed the exact same error caused by testmod_crypto on Debian Unstable during the last PR (#3395 ).

petermax2 · 2020-04-13T19:53:59Z

libelektra/src/plugins/crypto/gpgagent_teardown.h

Line 15 in 2e6a579

int status = system ("gpg-connect-agent --quiet KILLAGENT /bye");

I will rewrite this section to use execv in order to gain a better understanding of what's happening on the build server.

Try to kill the gpg-agent if the shutdown of the agent should fail. This is an attempt to resolve ElektraInitiative#2341 .

petermax2 · 2020-04-23T14:26:05Z

Should the error occur again, please feel free to re-open this issue.

This should resolve possible memory leak triggers introduced with ElektraInitiative#3404 (original issue: ElektraInitiative#2341 ).

petermax2 · 2020-04-25T18:17:14Z

The error occurred again, I'm still researching.

to use fork () and execv () instead of system (). See ElektraInitiative#2341 for full discussion.

petermax2 · 2020-04-30T11:29:53Z

Please ping me if the error occurs again.

mpranj · 2020-04-30T21:17:05Z

Happened with gpgme again. (I don't know if we should open separate issues for every plugin or just keep this meta-issue?)

 80/137 MemCheck  #87: testmod_gpgme ....................***Failed   43.49 sec

GPGME        TESTS

==================

[... mpranj truncated a little bit here ...]

GPGME        TESTS

==================



/home/jenkins/workspace/libelektra_PR-3419/src/plugins/gpgme/testmod_gpgme.c:227: error in test_encryption_decryption: kdb get failed

/home/jenkins/workspace/libelektra_PR-3419/src/plugins/gpgme/testmod_gpgme.c:243: error in test_encryption_decryption: key type not restored to string during decryption

/home/jenkins/workspace/libelektra_PR-3419/src/plugins/gpgme/testmod_gpgme.c:244: error in test_encryption_decryption: key content not restored during decryption

/home/jenkins/workspace/libelektra_PR-3419/src/plugins/gpgme/testmod_gpgme.c:254: error in test_encryption_decryption: key content not restored during decryption



gpgme Results: 45 Tests done — 4 errors.

petermax2 · 2020-04-30T21:44:47Z

I don't know if we should open separate issues for every plugin or just keep this meta-issue?

I think it's the same (or very similar) gpg-related error. I don't think it is necessary to open another issue.

petermax2 · 2020-05-01T14:26:32Z

gpgme (or gpgme in combination with valgrind) seems to cause problems if the output of stdout and stderr is being redirected.

GPGME        TESTS
==================

==155492== Warning: invalid file descriptor 1012 in syscall close()
==155492== Warning: invalid file descriptor 1013 in syscall close()
==155492== Warning: invalid file descriptor 1014 in syscall close()
==155492== Warning: invalid file descriptor 1015 in syscall close()
==155492==    Use --log-fd=<number> to select an alternative log fd.
==155492== Warning: invalid file descriptor 1016 in syscall close()
==155492== Warning: invalid file descriptor 1017 in syscall close()
==155492== Warning: invalid file descriptor 1018 in syscall close()
==155491== 
==155491== HEAP SUMMARY:
==155491==     in use at exit: 25,235 bytes in 176 blocks
==155491==   total heap usage: 802 allocs, 626 frees, 119,751 bytes allocated
==155491== 
==155491== LEAK SUMMARY:
==155491==    definitely lost: 0 bytes in 0 blocks
==155491==    indirectly lost: 0 bytes in 0 blocks
==155491==      possibly lost: 0 bytes in 0 blocks
==155491==    still reachable: 23,146 bytes in 163 blocks
==155491==         suppressed: 2,089 bytes in 13 blocks
==155491== Reachable blocks (those to which a pointer was found) are not shown.
==155491== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==155491== 
==155491== For counts of detected and suppressed errors, rerun with: -v
==155491== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
GPGME        TESTS
==================


gpgme Results: 45 Tests done — 0 errors.
==155462== 
==155462== HEAP SUMMARY:
==155462==     in use at exit: 4,833 bytes in 39 blocks
==155462==   total heap usage: 955 allocs, 916 frees, 698,713 bytes allocated
==155462== 
==155462== LEAK SUMMARY:
==155462==    definitely lost: 0 bytes in 0 blocks
==155462==    indirectly lost: 0 bytes in 0 blocks
==155462==      possibly lost: 0 bytes in 0 blocks
==155462==    still reachable: 3,241 bytes in 35 blocks
==155462==         suppressed: 1,592 bytes in 4 blocks
==155462== Reachable blocks (those to which a pointer was found) are not shown.
==155462== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==155462== 
==155462== For counts of detected and suppressed errors, rerun with: -v
==155462== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

I guess this would explain why the test output is written to the log files multiple times.

GPGME        TESTS
==================

[truncated output]

GPGME        TESTS
==================

GPGME        TESTS
==================


gpgme Results: 45 Tests done — 0 errors.

This contribution is related to ElektraInitiative#2341 .

markus2330 · 2020-10-12T11:43:08Z

Is this issue fixed now?

mpranj · 2020-10-12T12:04:43Z

@petermax2 rewrote parts of the crypto tests and moved them to separate CI jobs (debian-buster-cryptoplugins and debian-stretch-cryptoplugins).

I have not seen these tests fail since those improvements. I think it can be safely closed.

petermax2 · 2020-10-12T12:32:31Z

Glad to hear, that it's working now. Thank you for the feedback!

markus2330 assigned petermax2 Jan 26, 2019

sanssecours mentioned this issue Feb 25, 2019

Build Jobs: Parts of Test Suite Fail Regularly #2439

Closed

markus2330 added the testing errors in tests without implication on real usage label Apr 12, 2020

petermax2 mentioned this issue Apr 13, 2020

3391 crypto cleanup #3395

Merged

16 tasks

petermax2 added a commit to petermax2/libelektra that referenced this issue Apr 14, 2020

crypto: add shut-down logic in testmod_crypto

0787796

Try to kill the gpg-agent if the shutdown of the agent should fail. This is an attempt to resolve ElektraInitiative#2341 .

petermax2 mentioned this issue Apr 14, 2020

crypto: add shut-down logic in testmod_crypto #3404

Merged

16 tasks

petermax2 closed this as completed in #3404 Apr 23, 2020

petermax2 mentioned this issue Apr 25, 2020

3246 fix kdb exec path #3409

Merged

16 tasks

petermax2 added a commit to petermax2/libelektra that referenced this issue Apr 25, 2020

tests: prodive plugin function for shutting down gpg-agent

0dfec9d

This should resolve possible memory leak triggers introduced with ElektraInitiative#3404 (original issue: ElektraInitiative#2341 ).

petermax2 reopened this Apr 25, 2020

petermax2 added a commit to petermax2/libelektra that referenced this issue Apr 25, 2020

tests: rewrite gpg-agent shutdown code

55dc1cd

to use fork () and execv () instead of system (). See ElektraInitiative#2341 for full discussion.

petermax2 mentioned this issue Apr 25, 2020

tests: rewrite gpg-agent shutdown code #3410

Merged

16 tasks

petermax2 added a commit to petermax2/libelektra that referenced this issue Apr 25, 2020

tests: rewrite gpg-agent shutdown code

e7dfe3c

to use fork () and execv () instead of system (). See ElektraInitiative#2341 for full discussion.

petermax2 added a commit to petermax2/libelektra that referenced this issue Apr 28, 2020

tests: rewrite gpg-agent shutdown code

63fe280

to use fork () and execv () instead of system (). See ElektraInitiative#2341 for full discussion.

petermax2 added a commit to petermax2/libelektra that referenced this issue Apr 30, 2020

tests: rewrite gpg-agent shutdown code

64fb4ac

to use fork () and execv () instead of system (). See ElektraInitiative#2341 for full discussion.

petermax2 changed the title ~~fcrypt: test case fails~~ fcrypt / gpgme / crypto: test case fails Apr 30, 2020

petermax2 mentioned this issue May 1, 2020

tests: add libgpgme to valgrind suppressions #3425

Merged

16 tasks

petermax2 added a commit to petermax2/libelektra that referenced this issue May 1, 2020

gpgme: prevent double-initialization by unit test

96a8c21

This contribution is related to ElektraInitiative#2341 .

petermax2 added a commit to petermax2/libelektra that referenced this issue May 3, 2020

gpgme: prevent double-initialization by unit test

ee96aa7

This contribution is related to ElektraInitiative#2341 .

petermax2 added a commit to petermax2/libelektra that referenced this issue May 3, 2020

gpgme: prevent double-initialization by unit test

9e59846

This contribution is related to ElektraInitiative#2341 .

mpranj closed this as completed Oct 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fcrypt / gpgme / crypto: test case fails #2341

fcrypt / gpgme / crypto: test case fails #2341

markus2330 commented Jan 26, 2019

markus2330 commented Jan 26, 2019

petermax2 commented Jan 26, 2019

markus2330 commented Jan 26, 2019

mpranj commented Jan 26, 2019

petermax2 commented Jan 26, 2019

petermax2 commented Sep 14, 2019

markus2330 commented Sep 15, 2019

markus2330 commented Sep 15, 2019

sanssecours commented Sep 15, 2019

markus2330 commented Sep 15, 2019

sanssecours commented Sep 15, 2019

markus2330 commented Sep 15, 2019

petermax2 commented Apr 13, 2020

petermax2 commented Apr 13, 2020

petermax2 commented Apr 23, 2020

petermax2 commented Apr 25, 2020

petermax2 commented Apr 30, 2020

mpranj commented Apr 30, 2020

petermax2 commented Apr 30, 2020

petermax2 commented May 1, 2020

markus2330 commented Oct 12, 2020

mpranj commented Oct 12, 2020

petermax2 commented Oct 12, 2020

fcrypt / gpgme / crypto: test case fails #2341

fcrypt / gpgme / crypto: test case fails #2341

Comments

markus2330 commented Jan 26, 2019

markus2330 commented Jan 26, 2019

petermax2 commented Jan 26, 2019

markus2330 commented Jan 26, 2019

mpranj commented Jan 26, 2019

petermax2 commented Jan 26, 2019

petermax2 commented Sep 14, 2019

markus2330 commented Sep 15, 2019

markus2330 commented Sep 15, 2019

sanssecours commented Sep 15, 2019

markus2330 commented Sep 15, 2019

sanssecours commented Sep 15, 2019

markus2330 commented Sep 15, 2019

petermax2 commented Apr 13, 2020

petermax2 commented Apr 13, 2020

petermax2 commented Apr 23, 2020

petermax2 commented Apr 25, 2020

petermax2 commented Apr 30, 2020

mpranj commented Apr 30, 2020

petermax2 commented Apr 30, 2020

petermax2 commented May 1, 2020

markus2330 commented Oct 12, 2020

mpranj commented Oct 12, 2020

petermax2 commented Oct 12, 2020