Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenSC 0.24.0 PKCS#11 + Firefox + TLS client-side auth via PIV = connection timeout #2987

Open
SCjona opened this issue Jan 16, 2024 · 34 comments

Comments

@SCjona
Copy link

SCjona commented Jan 16, 2024

System & OpenSC Information

Component: opensc-pkcs11.so
Smartcard(s): 2x Yubico YubiKey OTP+FIDO+CCID 00 00 (YubiKey 5). Also happens with just one card connected.
Operating System: Arch Linux
Package: https://archlinux.org/packages/extra/x86_64/opensc/
Version number: 0.24.0 (package is compiled from Github releases)
Also happened to other coworkers on MacOS with brew and its OpenSC package.

Problem Description & Steps to reproduce

This bug only occurs in Firefox. Chromium works fine.

The following is copied from Mozilla Bugtracker: https://bugzilla.mozilla.org/show_bug.cgi?id=1872998

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:121.0) Gecko/20100101 Firefox/121.0

Steps to reproduce:

I'm using a YubiKey to authenticate myself to a web server which requires client-side TLS certificates for authentication. This worked fine with OpenSC version 0.23.0 (on MacOS and Linux (Windows untested)), but yields a timeout error on OpenSC version 0.24.0.

You can work around this issue by going to Settings -> Security Devices (under Certificates) -> Select the Smartcard under the OpenSC module -> Log In.
This will prompt the pin and after completing the login everything works as expected.

Actual results:

Opening an website which is protected by client-side TLS certificate authentication yields an network timeout error after 5 to 10 seconds. No PIN/Password prompt for the smart card is shown.

Expected results:

A PIN/Password prompt for the smartcard should be shown. After entering the correct PIN the page should load.

Logs

https://gist.github.com/SCjona/0c3c963afbcd17a18178c0d8f06d53a8

@SCjona
Copy link
Author

SCjona commented Jan 16, 2024

I found the culprit using git bisect: 1e625a3

I can get a working build using:

  1. git checkout 0.24.0
  2. git revert 1e625a3824c6a639e4c1590fe20899ade93242d0
  3. make

@dengert
Copy link
Member

dengert commented Jan 16, 2024

Your certificate appears to be self signed. It could be that FireFox 121 has changed how it trusts a self signed certificate.

Google for: firefox 121 self signed certificate

OpenSC can cache certificates under ~/.cache/opensc does it have any files?

The debug log ends unexpectedly at:
P:931480; T:0x123639550109376 18:15:15.251 [opensc-pkcs11] pkcs11-session.c:303:C_GetSessionInfo: C_GetSessionInfo(0x7072eddd5660) = CKR_OK

OpenSC debug does not print some parts dealing with the PIN unless the debug level >= 8.

Could you get a opensc-spy trace to see what PKCS11 calls are going on. See:
https://github.com/OpenSC/OpenSC/wiki/Using-OpenSC#pkcs-11-spy

@dengert
Copy link
Member

dengert commented Jan 16, 2024

But 1e625a3 is adding that the key is on the token, which in past it ignored. You may still have an issue with self signed certificate. Best to tell Mozilla of what you found.

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

Your certificate appears to be self signed. It could be that FireFox 121 has changed how it trusts a self signed certificate.

Do you mean self signed server certificate or client certificate? The server has an SSL certificate signed by Lets Encrypt and uses HSTS. Accessing the server without OpenSC works without SSL errors. My Yubikey has two PIV certificates, one self-signed for SSH and one signed by a custom (self signed) CA which is used for client-side TLS auth. Importing the custom CA into the trusted CA list of firefox does not make a difference.


OpenSC can cache certificates under ~/.cache/opensc does it have any files?

No the directory does not exist.


The debug log ends unexpectedly at:
P:931480; T:0x123639550109376 18:15:15.251 [opensc-pkcs11] pkcs11-session.c:303:C_GetSessionInfo: C_GetSessionInfo(0x7072eddd5660) = CKR_OK
OpenSC debug does not print some parts dealing with the PIN unless the debug level >= 8.

The same happens with debug level 9 (link below). I use the following procedure to create the logs:

  1. Close firefox
  2. Turn on debug config in /etc/opensc.conf
  3. Open firefox with the problematic website
  4. Wait for the page to run into the timeout error
  5. Close firefox
  6. Disable debug config in /etc/opensc.conf

Full debug level 9 log with only the Yubikey that has the client-side auth:
https://gist.github.com/SCjona/3e5a9ec5f8bae39031f3ac147e45c43b


Could you get a opensc-spy trace to see what PKCS11 calls are going on. See:
https://github.com/OpenSC/OpenSC/wiki/Using-OpenSC#pkcs-11-spy

  1. Removed OpenSC PKCS11 module from firefox and closed it.
  2. Started firefox with PKCS11SPY=/usr/lib/opensc-pkcs11.so PKCS11SPY_OUTPUT=logfile firefox in my shell.
  3. Upon loading the spy module in firefox I the following error:
Thread 1 "firefox" received signal SIGSEGV, Segmentation fault.
0x00007fffc4252a0b in spy_interface_function_list (pInterface=0x0) at pkcs11-spy.c:1578
Downloading source file /usr/src/debug/opensc/opensc-0.24.0/src/pkcs11/pkcs11-spy.c
1578            if (strcmp(pInterface->pInterfaceName, "PKCS 11") != 0) {                                                                                                                                                                   
(gdb) bt
#0  0x00007fffc4252a0b in spy_interface_function_list (pInterface=0x0) at pkcs11-spy.c:1578
#1  0x00007fffc4257cc7 in C_GetInterface (pInterfaceName=pInterfaceName@entry=0x7fffe975c6c3 "PKCS 11", pVersion=pVersion@entry=0x0, ppInterface=ppInterface@entry=0x7fffffff2e40, flags=flags@entry=1) at pkcs11-spy.c:1661
#2  0x00007fffe96b0634 in secmod_LoadPKCS11Module (mod=0x7fffb552e620, oldModule=0x0) at ../../lib/pk11wrap/pk11load.c:528
#3  0x00007fffe96c61cd in SECMOD_AddModule (newModule=0x7fffb552e620) at ../../lib/pk11wrap/pk11util.c:544
#4  SECMOD_AddNewModuleEx (moduleName=<optimized out>, dllPath=<optimized out>, defaultMechanismFlags=0, cipherEnableFlags=0, modparms=0x0, nssparms=0x0) at ../../lib/pk11wrap/pk11util.c:704

env var PKCS11SPY points to a valid file
/usr/lib/opensc-pkcs11.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=c3ecdbfc7034f1d137ba1ae2be71cf5efe4ebf19, stripped

logfile is created, but empty

@Jakuje
Copy link
Member

Jakuje commented Jan 17, 2024

The 1e625a3 adds a appropriate profile object flags. If this makes the TLS authentication non-working, it means that the certificate is not publicly readable, but opensc claims otherwise.

The spy log would be helpful. Can you rebuild the OpenSC with 0bc425e to fix the pkcs11 spy crash?

@Jakuje
Copy link
Member

Jakuje commented Jan 17, 2024

Can you provide a listing of the objects on your token with pkcs11-tool -O --login and pkcs11-tool -O (without login)?

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

The spy log would be helpful. Can you rebuild the OpenSC with 0bc425e to fix the pkcs11 spy crash?

Note: I'm using OpenSC latest master (commit c354501) custom build (build flags) for both pkcs11 module and spy module for this one

env vars:
PKCS11SPY=/home/user/tmp/OpenSC/src/pkcs11/.libs/opensc-pkcs11.so PKCS11SPY_OUTPUT=/home/user/opensc-spy.log

This still yields the same crash as mentioned above:

Thread 1 "firefox" received signal SIGSEGV, Segmentation fault.
0x00007fffc3153c04 in spy_interface_function_list (pInterface=0x0) at pkcs11-spy.c:1578
1578            if (strcmp(pInterface->pInterfaceName, "PKCS 11") != 0) {
(gdb) bt
#0  0x00007fffc3153c04 in spy_interface_function_list (pInterface=0x0) at pkcs11-spy.c:1578
#1  0x00007fffc31585fe in C_GetInterface (pInterfaceName=pInterfaceName@entry=0x7fffe975c6c3 "PKCS 11", pVersion=pVersion@entry=0x0, ppInterface=ppInterface@entry=0x7fffffff1ea0, flags=flags@entry=1) at pkcs11-spy.c:1676
#2  0x00007fffe96b0634 in secmod_LoadPKCS11Module (mod=0x7fffb6132820, oldModule=0x0) at ../../lib/pk11wrap/pk11load.c:528
#3  0x00007fffe96c61cd in SECMOD_AddModule (newModule=0x7fffb6132820) at ../../lib/pk11wrap/pk11util.c:544
#4  SECMOD_AddNewModuleEx (moduleName=<optimized out>, dllPath=<optimized out>, defaultMechanismFlags=0, cipherEnableFlags=0, modparms=0x0, nssparms=0x0) at ../../lib/pk11wrap/pk11util.c:704

Doing git checkout 0.24.0 and git cherry-pick 0bc425e02f1addc54e3cc00c1881b4d4ed30451a yields the same result.

The correct binary is being used:

(gdb) info source
Current source file is pkcs11-spy.c
Compilation directory is /home/user/tmp/OpenSC/src/pkcs11
Located in /home/user/tmp/OpenSC/src/pkcs11/pkcs11-spy.c
Contains 2034 lines.
Source language is c.
Producer is GNU C17 13.2.1 20230801 -mtune=generic -march=x86-64 -g -O2 -fPIC.
Compiled with DWARF 5 debugging format.
Does not include preprocessor macro info.

Can you provide a listing of the objects on your token with pkcs11-tool -O --login and pkcs11-tool -O (without login)?

pkcs11-tool -O --login
Using slot 0 with a present token (0x0)
Logging in to "YUBIKEY NAME".
Please enter User PIN: 
Private Key Object; RSA 
  label:      PIV AUTH key
  ID:         01
  Usage:      decrypt, sign, signRecover, unwrap
  Access:     sensitive, always sensitive, never extractable, local
Public Key Object; RSA 2048 bits
  label:      PIV AUTH pubkey
  ID:         01
  Usage:      encrypt, verify, verifyRecover, wrap
  Access:     none
Certificate Object; type = X.509 cert
  label:      Certificate for PIV Authentication
  subject:    DN: CN=CENSORED (SSH)
  serial:     CENSORED
  ID:         01
Certificate Object; type = X.509 cert
  label:      Certificate for Card Authentication
  subject:    DN: CN=CENSORED (CLIENT-SIDE TLS)
  serial:     CENSORED
  ID:         04
Public Key Object; EC  EC_POINT 384 bits
  EC_POINT:   CENSORED
  EC_PARAMS:  CENSORED
  label:      CARD AUTH pubkey
  ID:         04
  Usage:      verify
  Access:     none
Data object 1822415312
  label:          'Cardholder Fingerprints'
  application:    'Cardholder Fingerprints'
  app_id:         2.16.840.1.101.3.7.2.96.16
  flags:           private
Data object 1822415408
  label:          'Printed Information'
  application:    'Printed Information'
  app_id:         2.16.840.1.101.3.7.2.48.1
  flags:           private
Data object 1822415504
  label:          'Cardholder Facial Image'
  application:    'Cardholder Facial Image'
  app_id:         2.16.840.1.101.3.7.2.96.48
  flags:           private
Profile object 1822420624
  profile_id:          CKP_PUBLIC_CERTIFICATES_TOKEN (4)
Private Key Object; EC
  label:      CARD AUTH key
  ID:         04
  Usage:      sign
  Access:     sensitive, always sensitive, never extractable, local
Data object 1822414928
  label:          'Card Capability Container'
  application:    'Card Capability Container'
  app_id:         2.16.840.1.101.3.7.1.219.0
  flags:          <empty>
Data object 1822415024
  label:          'Card Holder Unique Identifier'
  application:    'Card Holder Unique Identifier'
  app_id:         2.16.840.1.101.3.7.2.48.0
  flags:          <empty>
Data object 1822415120
  label:          'Unsigned Card Holder Unique Identifier'
  application:    'Unsigned Card Holder Unique Identifier'
  app_id:         2.16.840.1.101.3.7.2.48.2
  flags:          <empty>
Data object 1822415216
  label:          'X.509 Certificate for PIV Authentication'
  application:    'X.509 Certificate for PIV Authentication'
  app_id:         2.16.840.1.101.3.7.2.1.1
  flags:          <empty>
Data object 1822415600
  label:          'X.509 Certificate for Digital Signature'
  application:    'X.509 Certificate for Digital Signature'
  app_id:         2.16.840.1.101.3.7.2.1.0
  flags:          <empty>
Data object 1822415696
  label:          'X.509 Certificate for Key Management'
  application:    'X.509 Certificate for Key Management'
  app_id:         2.16.840.1.101.3.7.2.1.2
  flags:          <empty>
Data object 1822415792
  label:          'X.509 Certificate for Card Authentication'
  application:    'X.509 Certificate for Card Authentication'
  app_id:         2.16.840.1.101.3.7.2.5.0
  flags:          <empty>
Data object 1822415888
  label:          'Security Object'
  application:    'Security Object'
  app_id:         2.16.840.1.101.3.7.2.144.0
  flags:          <empty>
Data object 1822415984
  label:          'Discovery Object'
  application:    'Discovery Object'
  app_id:         2.16.840.1.101.3.7.2.96.80
  flags:          <empty>
Data object 1822416176
  label:          'Biometric Information Templates Group Template'
  application:    'Biometric Information Templates Group Template'
  app_id:         2.16.840.1.101.3.7.2.16.22
  flags:          <empty>
Data object 1822416272
  label:          'Secure Messaging Certificate Signer'
  application:    'Secure Messaging Certificate Signer'
  app_id:         2.16.840.1.101.3.7.2.16.23
  flags:          <empty>
pkcs11-tool -O
Using slot 0 with a present token (0x0)
Public Key Object; RSA 2048 bits
  label:      PIV AUTH pubkey
  ID:         01
  Usage:      encrypt, verify, verifyRecover, wrap
  Access:     none
Certificate Object; type = X.509 cert
  label:      Certificate for PIV Authentication
  subject:    DN: CN=CENSORED(SSH)
  serial:     CENSORED
  ID:         01
Certificate Object; type = X.509 cert
  label:      Certificate for Card Authentication
  subject:    DN: CN=CENSORED (CLIENT-SIDE TLS AUTH)
  serial:     CENSORED
  ID:         04
Public Key Object; EC  EC_POINT 384 bits
  EC_POINT:   CENSORED
  EC_PARAMS:  CENSORED
  label:      CARD AUTH pubkey
  ID:         04
  Usage:      verify
  Access:     none
Profile object 3883351616
  profile_id:          CKP_PUBLIC_CERTIFICATES_TOKEN (4)
Private Key Object; EC
  label:      CARD AUTH key
  ID:         04
  Usage:      sign
  Access:     sensitive, always sensitive, never extractable, local
Data object 3883345920
  label:          'Card Capability Container'
  application:    'Card Capability Container'
  app_id:         2.16.840.1.101.3.7.1.219.0
  flags:          <empty>
Data object 3883346016
  label:          'Card Holder Unique Identifier'
  application:    'Card Holder Unique Identifier'
  app_id:         2.16.840.1.101.3.7.2.48.0
  flags:          <empty>
Data object 3883346112
  label:          'Unsigned Card Holder Unique Identifier'
  application:    'Unsigned Card Holder Unique Identifier'
  app_id:         2.16.840.1.101.3.7.2.48.2
  flags:          <empty>
Data object 3883346208
  label:          'X.509 Certificate for PIV Authentication'
  application:    'X.509 Certificate for PIV Authentication'
  app_id:         2.16.840.1.101.3.7.2.1.1
  flags:          <empty>
Data object 3883346592
  label:          'X.509 Certificate for Digital Signature'
  application:    'X.509 Certificate for Digital Signature'
  app_id:         2.16.840.1.101.3.7.2.1.0
  flags:          <empty>
Data object 3883346688
  label:          'X.509 Certificate for Key Management'
  application:    'X.509 Certificate for Key Management'
  app_id:         2.16.840.1.101.3.7.2.1.2
  flags:          <empty>
Data object 3883346784
  label:          'X.509 Certificate for Card Authentication'
  application:    'X.509 Certificate for Card Authentication'
  app_id:         2.16.840.1.101.3.7.2.5.0
  flags:          <empty>
Data object 3883346880
  label:          'Security Object'
  application:    'Security Object'
  app_id:         2.16.840.1.101.3.7.2.144.0
  flags:          <empty>
Data object 3883346976
  label:          'Discovery Object'
  application:    'Discovery Object'
  app_id:         2.16.840.1.101.3.7.2.96.80
  flags:          <empty>
Data object 3883347168
  label:          'Biometric Information Templates Group Template'
  application:    'Biometric Information Templates Group Template'
  app_id:         2.16.840.1.101.3.7.2.16.22
  flags:          <empty>
Data object 3883347264
  label:          'Secure Messaging Certificate Signer'
  application:    'Secure Messaging Certificate Signer'
  app_id:         2.16.840.1.101.3.7.2.16.23
  flags:          <empty>

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

I changed the fprintf to printf in the spy module, this is the output:

*************** OpenSC PKCS#11 spy *****************
Loaded: "/home/user/tmp/OpenSC/src/pkcs11/.libs/opensc-pkcs11.so"

0: C_GetInterface
P:726150; T:0x140737352631552 2024-01-17 13:55:00.696
[in] pInterfaceName [in] pVersion = NULL
[in] flags = CKF_INTERFACE_FORK_SAFE

Thread 1 "firefox" received signal SIGSEGV, Segmentation fault.
0x00007fffc48e9bf4 in spy_interface_function_list (pInterface=0x0) at pkcs11-spy.c:1579
1579            if (strcmp(pInterface->pInterfaceName, "PKCS 11") != 0) {
(gdb) 

@dengert
Copy link
Member

dengert commented Jan 17, 2024

Good to see @Jakuje is looking at this too.

https://www.mozilla.org/en-US/firefox/releases/
says 121.0 was released December 19, 2023, with 121.1 January 9, 2024 So it is very new. On your OS can you backport to a previous version?

You said:

Also happened to other coworkers on MacOS with brew and its OpenSC
What versions are they using?

To me it looks like the problem is in Firefox 121, not OpenSC. That may also b

Your card has only 2 certificate and keys.

Note that the "Certificate for Card Authentication" id 04 with key ref 9E is not meant for a user key and cert, as it is designed so the card can authenticate itself and it does NOT require a login to do that.
NIST sp800-73-4 and earlier versions: specify:

3.1.4 X.509 Certificate for Card Authentication
FIPS 201 specifies the mandatory asymmetric Card Authentication key (CAK) as a private key that
may be used to support physical access applications. The read access control rule of the
corresponding X.509 Certificate for Card Authentication is “Always,”

What you may have been seeing with previous versions of FireFox is it does a C_Login to verify the PIN before doing anything. But FireFox 121.0 is not doing that and then getting confused or has some other issue.

If reverting 1e625a3 fixes the problem,
IMHO that points the finger at Firefox, as we now respond with TRUE.

Unfortunately, I do my linux testing using Ubuntu 22.04, where FireFox was recently is updated to 121.0 but Ubuntu is installing it as a SNAP package, and they have not figured out how to get SNAP packaged to work with smartcards!
https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/1967632
Their firefox-esr (without SNAP) is at 115.7.

@dengert
Copy link
Member

dengert commented Jan 17, 2024

pkcs11-spy.c:1579 if (strcmp(pInterface->pInterfaceName, "PKCS 11") != 0) {

needs to check if pInterface != NULL and if pInterface->pInterfaceName != NULL.

@Jakuje
Copy link
Member

Jakuje commented Jan 17, 2024

The objects looks reasonable. I do not see any private certificates and the profile objects matches this expectation. You can get also PKCS#11 trace from the NSS to see what it does with the pkcs11 modules:

https://www-archive.mozilla.org/projects/security/pki/nss/tech-notes/tn2

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

On your OS can you backport to a previous version

I just tried Firefox 120.0.1, same result. I can try older versions till I get dynamic linking issues (after which I'd have to compile the thing and that will take a while). Anything particular you want me to try? https://archive.archlinux.org/packages/f/firefox/

Also happened to other coworkers on MacOS with brew and its OpenSC
What versions are they using?

Same OpenSC and Firefox as me, stopped working for them after installing OpenSC 0.24.0, they also have the same Yubikey config.

Note that the "Certificate for Card Authentication" id 04 with key ref 9E is not meant for a user key and cert

Interesting that you mention that, I really just needed another Authentication card slot as the first one was used by SSH (not sure if I can mix that stuff, we're not using PKI for SSH, just plain old authorized_keys).
I wonder if Firefox may just be acting up because its in the wrong slot.

Unfortunately, I do my linux testing using Ubuntu 22.04, where FireFox was recently is updated to 121.0 but Ubuntu is installing it as a SNAP package, and they have not figured out how to get SNAP packaged to work with smartcards!
https://bugs.launchpad.net/ubuntu/+source/firefox/+bug/1967632
Their firefox-esr (without SNAP) is at 115.7.

if you want to install the newest versions as non-snap: https://ubuntuhandbook.org/index.php/2022/04/install-firefox-deb-ubuntu-22-04/

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

You can get also PKCS#11 trace from the NSS to see what it does with the pkcs11 modules: https://www-archive.mozilla.org/projects/security/pki/nss/tech-notes/tn2

NSPR_LOG_MODULES=nss_mod_log:4 NSPR_LOG_FILE=/home/user/nspr.log NSS_DEBUG_PKCS11_MODULE="OpenSC" firefox

nspr.log: https://gist.github.com/SCjona/250d1073cad2384b3a93266332a3fb79

@dengert
Copy link
Member

dengert commented Jan 17, 2024

Here is a way to install Firefiox nightly builds without SNAP

https://blog.nightly.mozilla.org/2023/10/30/introducing-mozillas-firefox-nightly-deb-packages-for-debian-based-linux-distributions/

It installed 123.a1 in my Ubuntu 22.04. Will see if I can get at least spy to work.

@Jakuje
Copy link
Member

Jakuje commented Jan 17, 2024

You can get also PKCS#11 trace from the NSS to see what it does with the pkcs11 modules: https://www-archive.mozilla.org/projects/security/pki/nss/tech-notes/tn2

NSPR_LOG_MODULES=nss_mod_log:4 NSPR_LOG_FILE=/home/user/nspr.log NSS_DEBUG_PKCS11_MODULE="OpenSC" firefox

nspr.log: https://gist.github.com/SCjona/250d1073cad2384b3a93266332a3fb79

The log shows the C_Sign() worked with the key ID 0x04 in this case, but there was no attempt to Login. The session is still CKS_RO_PUBLIC_SESSION (this is probably not what you want).

The slot with key 4 is in some way weird that it allows operations without authentication. Not sure if Firefox/NSS handles this well. See the [1], which describes this behavior (no need for the pin).

I would certainly propose to try a different slot, probably one from the range of 82-95, as they should not have any weird behavior such as this one or user constent in the 9c: Digital Signature slot.

[1] https://developers.yubico.com/PIV/Introduction/Certificate_slots.html

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

I created another cert and put it in the digital signature slot (9c). Using this cert I get prompts for entering the PIN and can successfully load the page. Not really usable as a workaround though as it prompts for a pin each time it creates a new SSL connection. I guess using the Key Management Slot (9d) would be a better choice due to the PIN "caching" ability (like the Authentication slot (9a) has) as described by the Yubikey docs. So I guess Firefox does not handle the Card Authentication (9e) slot correctly.

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

Moved the certificate to the key management slot (9d). This restores the previous working behavior and resolves the issue.
I'll update the Mozilla Bugtracker issue.

@Jakuje
Copy link
Member

Jakuje commented Jan 17, 2024

I am not sure if this will be considered by Firefox though as I would consider this more like a configuration issue -- you probably did not want your key to be able to provide signatures without login and it worked mostly accidentally because firefox logged in into everything before correct introduction of the profile objects in 0.24.0.

@dengert
Copy link
Member

dengert commented Jan 17, 2024

@Jakuje in regards to the segfault I made the following hack:

diff --git a/src/pkcs11/pkcs11-spy.c b/src/pkcs11/pkcs11-spy.c
index 00575444e..53b2113d6 100644
--- a/src/pkcs11/pkcs11-spy.c
+++ b/src/pkcs11/pkcs11-spy.c
@@ -1575,7 +1575,7 @@ spy_interface_function_list(CK_INTERFACE_PTR pInterface)
        CK_VERSION *version;
 
        /* Do not touch unknown interfaces. We can not do anything with these */
-       if (strcmp(pInterface->pInterfaceName, "PKCS 11") != 0) {
+       if (pInterface->pInterfaceName > 0xffffff && strcmp(pInterface->pInterfaceName, "PKCS 11") != 0) {
                return;
        }

because gdb was showing pInterface->pInterfaceName = 0x20 but can not track down where this came from FireFox or from pkcs11-global.c

Here is the start of a spy trace. Firfox 123a1 starts a lot of threads.

*************** OpenSC PKCS#11 spy *****************
Loaded: "/opt/ossl-3.1.2/lib/pkcs11/opensc-pkcs11.so"

0: C_GetInterface
P:40476; T:0x140737352648576 2024-01-17 11:22:12.009
[in] pInterfaceName 00007ffff4b5fd96 / 7
    00000000  50 4B 43 53 20 31 31                             PKCS 11     
[in] pVersion = NULL 
[in] flags = CKF_INTERFACE_FORK_SAFE
Returned:  7 CKR_ARGUMENTS_BAD

1: C_GetInterface
P:40476; T:0x140737352648576 2024-01-17 11:23:56.974
[in] pInterfaceName 00007ffff4b5fd96 / 7
    00000000  50 4B 43 53 20 31 31                             PKCS 11     
[in] pVersion = NULL 
[in] flags = 
Returned:  0 CKR_OK

2: C_GetInterface
P:40476; T:0x140737352648576 2024-01-17 11:24:44.485
[in] pInterfaceName 00007ffff4b61a9e / 25
    00000000  56 65 6E 64 6F 72 20 4E 53 53 20 46 49 50 53 20  Vendor NSS FIPS
    00000010  49 6E 74 65 72 66 61 63 65                       Interface
[in] pVersion = NULL
[in] flags =
Returned:  7 CKR_ARGUMENTS_BAD

3: C_Initialize
P:40476; T:0x140737352648576 2024-01-17 11:24:51.238
[in] pInitArgs = 0x7ffff4bfa5a8
     flags: 3
       CKF_LIBRARY_CANT_CREATE_OS_THREADS
       CKF_OS_LOCKING_OK
Returned:  0 CKR_OK

4: C_GetInfo
P:40476; T:0x140737352648576 2024-01-17 11:24:51.489
[out] pInfo:
      cryptokiVersion:         3.0
      manufacturerID:         'OpenSC Project                  '
      flags:                   0
      libraryDescription:     'OpenSC smartcard framework      '
      libraryVersion:          0.24
Returned:  0 CKR_OK











@dengert
Copy link
Member

dengert commented Jan 17, 2024

@SCjona I agree with @Jakuje that using the cert/key without a pin is a good idea.

Years ago I added code to card-piv.c to allow the use of the 30 retires key management keys/certs to be used based on the certificate keyUsage. i.e. not just for encryption. But only if the FASC-N in the CHUID started with 9999. (i.e. not gov issued) The FASC-N uses some 5-bit encoding. And you have to write a Discovery object to the card saying how many of the retired keys are present.

I could try and find the instructions if you are interested.

@dengert
Copy link
Member

dengert commented Jan 17, 2024

Sorry should have said "without a pin is a bad idea."

@dengert
Copy link
Member

dengert commented Jan 17, 2024

Most likely the reason Firefox worked before 1e625a3 has to do with:
Firefox looks for profiles with CKA_TOKEN True. and would not find any.
After 1e625a3 it does:

13: C_FindObjectsInit
P:40476; T:0x140737352648576 2024-01-17 11:24:51.499
[in] hSession = 0x7fffdfbbc4a0
[in] pTemplate[2]:
    CKA_TOKEN             True
    CKA_CLASS             CKO_PROFILE
Returned:  0 CKR_OK

14: C_FindObjects
P:40476; T:0x140737352648576 2024-01-17 11:24:51.501
[in] hSession = 0x7fffdfbbc4a0
[in] ulMaxObjectCount = 0xa
[out] ulObjectCount = 0x1
Object 0x7fffd68cb8d0 matches
Returned:  0 CKR_OK

15: C_FindObjectsFinal
P:40476; T:0x140737352648576 2024-01-17 11:24:51.501
[in] hSession = 0x7fffdfbbc4a0
Returned:  0 CKR_OK

16: C_CloseSession
P:40476; T:0x140737352648576 2024-01-17 11:24:51.501
[in] hSession = 0x7fffdfbbc4a0
Returned:  0 CKR_OK

17: C_GetAttributeValue
P:40476; T:0x140737352648576 2024-01-17 11:24:51.501
[in] hSession = 0x7fffdfbbc2e0
[in] hObject = 0x7fffd68cb8d0
[in] pTemplate[1]:
    CKA_PROFILE_ID        00007fffffffaa90 / 8
[out] pTemplate[1]:
    CKA_PROFILE_ID        CKP_PUBLIC_CERTIFICATES_TOKEN
Returned:  0 CKR_OK

And because it found CKP_PUBLIC_CERTIFICATES_TOKEN it now know it did not require a PIN to find the certificates,
so does prompted for a PIN. But why it can not use the certificate/key is still not clear.

@SCjona what version is your Yubikey?
it could be possible that it is not following the NIST PIV specs and really does require a PIN.
Did you uses any Yubikey tool to set the PIN required bit on the on the card?
The card-piv.c assumed the Yubikey is following the NIST specs.

@dengert
Copy link
Member

dengert commented Jan 17, 2024

Going back to original debug log, at line 7494 is the C_Sign request ending at line 7618, with the command sent and response received between lines 7583 and 7598 Why the log ends at line 7800 is still not clear.

So Firefox requested a C_Sign, and OpenSC returned a signature. So other then the segfault in spy, it looks like OpenSC is working.

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

what version is your Yubikey?

YubiKey 5 NFC - Firmware 5.2.7

it could be possible that it is not following the NIST PIV specs and really does require a PIN.

Chromium doesn't need a PIN, so I think you can assume it follows NIST PIV specs

Did you uses any Yubikey tool to set the PIN required bit on the on the card?

don't think so

using the cert/key without a pin is a bad idea.

we're not using the key for auth directly, it just authenticates you to a reverse proxy, which grants access to internal services, which are protected by another login page (because the software doesn't support PKI based auth).

so for me there are just two risk scenarios here:

  1. user yubikey gets stolen - you still need a valid login with random generated passwords (they are saved in the users password manager) and you have to do so before someone puts you on the CRL of the reverse proxy
  2. user system is infected with malware - at this point even having a PIN won't help you a whole lot as the malware can just intercept it

maybe i forgot a scenario here, but with just those two i don't see a benefit in having to enter a PIN. I just thought PIN was a requirement at all times.

@SCjona
Copy link
Author

SCjona commented Jan 17, 2024

@dengert

because gdb was showing pInterface->pInterfaceName = 0x20 but can not track down where this came from FireFox or from pkcs11-global.c

I'd assume pInterface is just NULL and offsetof(pInterface, pInterfaceName) is 0x20, so you really just need to check if pInterface is not NULL

@dengert
Copy link
Member

dengert commented Jan 18, 2024

@SCjona This might be your best bet;
Changes were made in 2016 to pkcs15-piv.c to allow the 20 "retired" key slots to be used based in certificate keyUsage.

#905

This requires a PIV History object to be written to the token if used via OpenSC which is also listed here:
And see also #847 (comment)

#847 was the original complaint.

https://support.yubico.com/hc/en-us/articles/4585159896220-Troubleshooting-Retired-PIV-Slots-Unavailable-When-Accessing-via-PKCS11

Since then you can find more info - Google for C10114C20100FE00

dengert added a commit to dengert/OpenSC that referenced this issue Jan 18, 2024
The po->C_GetInterface is passed the callers ppInterface where
*ppInterface may not be valid.

if the po->C_GetInterface may not update the *ppInterface and return
an error. In this case  spy_interface_function_list should not be called,
as it assumes the *ppInterface has been modified.

Found debugging FireFox version 121 where FireFox passes a ppInterface
where *ppInterface is not a valid pointer, causing a segfault in
spy_interface_function_list.

FireFox calls C_GetInterface twice with flags = CKF_INTERFACE_FORK_SAFE
twice then on third time requests with flag = 0  where po->GetInterface
can support and it updates the *ppInterface  with valid data.

See OpenSC#2987

 On branch pkcs11-spy-segfault
 Changes to be committed:
	modified:   pkcs11-spy.c
Jakuje pushed a commit that referenced this issue Jan 23, 2024
The po->C_GetInterface is passed the callers ppInterface where
*ppInterface may not be valid.

if the po->C_GetInterface may not update the *ppInterface and return
an error. In this case  spy_interface_function_list should not be called,
as it assumes the *ppInterface has been modified.

Found debugging FireFox version 121 where FireFox passes a ppInterface
where *ppInterface is not a valid pointer, causing a segfault in
spy_interface_function_list.

FireFox calls C_GetInterface twice with flags = CKF_INTERFACE_FORK_SAFE
twice then on third time requests with flag = 0  where po->GetInterface
can support and it updates the *ppInterface  with valid data.

See #2987

 On branch pkcs11-spy-segfault
 Changes to be committed:
	modified:   pkcs11-spy.c
@frankmorgner
Copy link
Member

What's the status of this issue? It seems that your hopping from one problem to another. Ideally, you would open one issue for one problem so that others can spot the solution(s) without spending hours of reading through this issue.

@dengert
Copy link
Member

dengert commented Jan 24, 2024

It is not clear yet where the real problem is: OS, FireFox or OpenSC.

The circumvention for @SCjona is to not use the PIV 9E key which is intended for the card to authenticate to physical devices like a door lock without using a PIN. One way to do this is outlined in #2987 (comment) to use one of the "retired" key slots on Yubico.

The Spy issue was found trying to debug FireFox and committed to OpenSC master yesterday.

@SCjona if you could provide a spy trace using OpenSC master build that would help locate the problem.

@SCjona
Copy link
Author

SCjona commented Jan 24, 2024

@dengert
Copy link
Member

dengert commented Jan 24, 2024

Thanks for quick response. In https://gist.github.com/SCjona/eb9d15e4e7ebe798b2d660f92a9ed4eb

In lines 424- 564 it found the certificate which you CENSORED.
form 700 to 827 it found the CK_ID=04 from certificate and public key which looks correct for PIV
Certificate for card authentication.
From 829 - 900 it fond the private key with EC key with curve: 06 05 2B 81 04 00 22 which is OBJECT IDENTIFIER 1.3.132.0.34 secp384r1.
From 912 if opens a PKCS11 session,

in line 929:

84: C_Sign
P:670611; T:0x126052609844928 2024-01-24 15:15:38.614
[in] hSession = 0x72a4b203b200
[in] pData[ulDataLen] 000072a4e9085488 / 48
    00000000  E6 3B 4F FD 06 DA 7A 14 38 09 7A 0C 4D 3E B4 C1  .;O...z.8.z.M>..
    00000010  57 91 3C 82 B9 8F 29 E7 C2 56 58 18 2D CE FC F1  W.<...)..VX.-...
    00000020  9B E3 AD 8D 78 74 5E 31 41 EF B3 7B 89 A7 09 C9  ....xt^1A..{....
[out] pSignature[*pulSignatureLen] 000072a4b21dfee0 / 96
    00000000  8A 27 28 64 1B E3 FB 60 F0 C9 4B A3 BE 48 45 75  .'(d...`..K..HEu
    00000010  61 98 13 DF 3E 57 8B 7C C3 0E 87 38 68 1A 05 21  a...>W.|...8h..!
    00000020  2E 1B 0F 67 DD 67 4B 9C 8F FF C4 6B D1 2A B8 E4  ...g.gK....k.*..
    00000030  CE BA 32 69 9A DD 15 74 07 8C BC 5E B8 4C C9 DC  ..2i...t...^.L..
    00000040  A4 C7 61 E8 9E 4B 84 E5 43 69 22 19 11 F6 33 93  ..a..K..Ci"...3.
    00000050  72 93 58 34 F9 3D 8E D6 0A CD 80 D3 B8 C2 44 BF  r.X4.=........D.
Returned:  0 CKR_OK

It signed something without requiring entey a PIN as the key with CK_ID=04 does NOT require a PIN as expected.

In 955 it closes the session.

So in my option, OpenSC is working as expected.

These probable do not apply but somethings to look at:

  • Has you certificated changed recently. It could have been cached somewhere: (FireFox, OpenSC under ~/.cache/opensc or server and the cached certificate's public key does not match private key on the card. (Does not look like OpenSC cache, as original log shows certificate being red from card.)

  • Firefox or server is doing more checks of the certificate for example expiration date? keyUsage?

  • Does new Firefox need CA or intermediate CA certificates? Has CA or intermediate certificate expired?

@dengert
Copy link
Member

dengert commented Jan 24, 2024

Correction: It signed something without requiring entry of PIN as the key with CK_ID=04 does NOT require a PIN as expected.

@SCjona
Copy link
Author

SCjona commented Jan 25, 2024

Has you certificated changed recently

I freshly created it, but I don't think it's a cache issue.

Firefox or server is doing more checks of the certificate for example expiration date? keyUsage?

certainly, it should check at least expiration, probably also keyUsage. it's nginx with this config

server
{
  // .... snip ...
  ssl_client_certificate ca.crt;
  ssl_verify_client optional;
  ssl_verify_depth 2;
  // ... snip ....
  location /
  {
    if ($ssl_client_verify != SUCCESS)
    {
      return 403;
    }
    // ...snip...
  }
}

Does new Firefox need CA or intermediate CA certificates? Has CA or intermediate certificate expired?

all certificates and the certificate chain is valid. as explained before client side certs are signed with a custom CA, server side cert is lets encrypt.


I also did a spy trace with with the same setup with me logging into the card manually first (via security devices in firefox settings), which makes everything work fine. the spy trace is 33MB though. i can send you that one via email if you want.

frankmorgner pushed a commit that referenced this issue Jan 25, 2024
The po->C_GetInterface is passed the callers ppInterface where
*ppInterface may not be valid.

if the po->C_GetInterface may not update the *ppInterface and return
an error. In this case  spy_interface_function_list should not be called,
as it assumes the *ppInterface has been modified.

Found debugging FireFox version 121 where FireFox passes a ppInterface
where *ppInterface is not a valid pointer, causing a segfault in
spy_interface_function_list.

FireFox calls C_GetInterface twice with flags = CKF_INTERFACE_FORK_SAFE
twice then on third time requests with flag = 0  where po->GetInterface
can support and it updates the *ppInterface  with valid data.

See #2987

 On branch pkcs11-spy-segfault
 Changes to be committed:
	modified:   pkcs11-spy.c
@dengert
Copy link
Member

dengert commented Jan 25, 2024

Sed the trace to deengert at gmail.com

@dengert
Copy link
Member

dengert commented Jan 25, 2024

The spy trace 33 MB exceeds the gmail max email size or 25GB. (Google for: gmail max email size )
I bet most of it is polling requests, just sent the first 2GB or you see something interesting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants