Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Support new keyboxd gpg pubring format #54

Open
shumvgolove opened this issue Mar 13, 2024 · 15 comments
Open

[Feature]: Support new keyboxd gpg pubring format #54

shumvgolove opened this issue Mar 13, 2024 · 15 comments

Comments

@shumvgolove
Copy link

Hey 👋

New gpg installations doesn't use pubring.kbx format since 2.4.1, and instead use keyboxd (aka sqlite db) with the $HOME/.gnupg/public-keys.d/pubring.db path.

Would be really awesome if yubikey-touch-detector gained support for keyboxd format, since newer installations are stuck without gpg detection 🥲

Thanks!

@maximbaz
Copy link
Owner

Many thanks for letting me know!

I'll try to get to this soon, but if you are eager to try something, I suspect and hope that simply adjusting the path here to the new file will do the trick.

gpgPubringPath := path.Join(gpgme.GetDirInfo("homedir"), "pubring.kbx")

And if this works, we can update that part of the code to initialize gpgPubringPath with the new path (if it exists on the system) or with the old path as the fallback.

@shumvgolove
Copy link
Author

Yeah, unfortunately this doesn't work :^(

Firstly, I've tried just to symlink the pubring.db to pubring.kbx with:

ln -s ~/.gnupg/public-keys.d/pubring.db ~/.gnupg/pubring.kbx

And executing the yubikey-touch-detector gives:

❯ yubikey-touch-detector -v
DEBU[2024-03-13T19:31:11Z] Starting YubiKey touch detector              
DEBU[2024-03-13T19:31:11Z] HMAC watcher on '/dev' is successfully established 
DEBU[2024-03-13T19:31:11Z] U2F watcher on '/dev' is successfully established 
DEBU[2024-03-13T19:31:11Z] GPG watcher is successfully established      
DEBU[2024-03-13T19:31:11Z] SSH watcher is successfully established 

However, signing/encrypting test string with:

echo test | gpg -se -u "$KEYID" -r "$KEYID" | gpg -

sadly doesn't output anything :(

With your suggestion, I've adjusted main.go like so:

diff --git a/main.go b/main.go
index 3b2d0c4..d697169 100644
--- a/main.go
+++ b/main.go
@@ -78,7 +78,7 @@ func main() {
        } else if ctx.SetProtocol(gpgme.ProtocolAssuan) != nil {
                log.Debugf("Cannot initialize Assuan IPC: %v. Disabling GPG and SSH watchers.", err)
        } else {
-               gpgPubringPath := path.Join(gpgme.GetDirInfo("homedir"), "pubring.kbx")
+               gpgPubringPath := path.Join(gpgme.GetDirInfo("homedir"), "public-keys.d/pubring.db")
                if _, err := os.Stat(gpgPubringPath); err == nil {
 
                        requestGPGCheck := make(chan bool)

And recompiled binary with go build. Executing modified yubikey-touch-detector and above gpg test also doesn't register any gpg touch events. I guess there's something different with new format: gpg docs mention keyboxd daemon that indeed running in my current user session:

❯ ps | grep keybox
 5845  sh         │       0.0 0.0 00:00:00 │ keyboxd --homedir /home/sh/.gnupg --daemon

@shumvgolove
Copy link
Author

I've tried to kill all gpg related processes with:

pkill -9 gpg-agent; pkill -9 scdaemon; pkill -9 keyboxd

Then, I've executed yubikey-touch-detector and tested with:

echo test | gpg -se -u "$KEYID" -r "$KEYID" | gpg -

And finally events are being printed:

DEBU[2024-03-13T19:45:42Z] Starting YubiKey touch detector              
DEBU[2024-03-13T19:45:42Z] U2F watcher on '/dev' is successfully established 
DEBU[2024-03-13T19:45:42Z] HMAC watcher on '/dev' is successfully established 
DEBU[2024-03-13T19:45:42Z] SSH watcher is successfully established      
DEBU[2024-03-13T19:45:42Z] GPG watcher is successfully established      
DEBU[2024-03-13T19:45:45Z] [notifiers/debug] GPG_1                      
DEBU[2024-03-13T19:45:49Z] AssuanSend/status: PROGRESS, learncard k 0 0 
DEBU[2024-03-13T19:45:49Z] AssuanSend/status: PROGRESS, learncard k 0 0 
DEBU[2024-03-13T19:45:49Z] AssuanSend/status: PROGRESS, learncard k 0 0 
DEBU[2024-03-13T19:45:49Z] [notifiers/debug] GPG_0 

But that worked only once. After that, encrypting/signing test string doesn't produce any event in yubikey-touch-dector. Weird 🤔

@maximbaz
Copy link
Owner

I see... Thanks for your experiments!

While I don't have the setup yet to reproduce this, I can speculate based on your findings.

I think you would appreciate to know these details first: the way gpg detection works does not actually "depend" on the format of pubring.kbx file, we don't even open that file! Simplifying slightly, we have a computationally-expensive way to "ask" gpg whether it's waiting on a touch, and in order to not to burn your battery we only want to ask gpg when we think there might be an ongoing gpg operation. And I noticed back in the day, that whenever any gpg operation was executed, gpg itself would open pubring.kbx for its own purposes. So all yubikey-touch-detector does, is it listens on "open" event on that file, and if that file was opened, it is assumed that now is a good time to ask gpg whether it in fact waits for a touch or not.

Based on your latest log, it looks like gpg was optimized not to open the new file every single time, but probably once on the first usage. So that is probably why the detection worked once, and never again, because yubikey-touch-detector doesn't have this signal anymore.

We'll need to think how to handle this 🤔 It was clear from the beginning that this whole logic is based on a dirty hack, and that one day gpg might change how it internally works with regards to opening keyring file. Today is that day, it seems 😁

cc @mochaaP you might find this thread curious, since you contributed recently to improving gpg detection.

@mochaaP
Copy link
Contributor

mochaaP commented Mar 14, 2024

A few questions:

  • Could anyone post a strace log of GPG daemon?
  • Is the SQLite connection persisted?
  • What's the journal mode & locking mode on the SQLite db?

If the lock could be detected in our process, we could use that
If that didn't work, another option might be hooking into the GPG daemon, but that will be platform specific (and also not very safe)

@Zempashi
Copy link

Here is a gpg-agent trace:

pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=109284}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=107341}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=113532}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=110907}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
#### make access to gpg
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=97653}, {sigmask=[], sigsetsize=8}) = 1 (in [3], left {tv_sec=3, tv_nsec=722223909})
accept(3, {sa_family=AF_UNIX}, [110 => 2]) = 7
rt_sigprocmask(SIG_BLOCK, ~[], [HUP INT USR1 USR2 TERM], 8) = 0
clone3({flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, child_tid=0x7fb00ea00990, parent_tid=0x7fb00ea00990, exit_signal=0, stack=0x7fb00e200000, stack_size=0x7fff80, tls=0x7fb00ea006c0} => {parent_tid=[12037]}, 88) = 12037
rt_sigprocmask(SIG_SETMASK, [HUP INT USR1 USR2 TERM], NULL, 8) = 0
futex(0x7fb00fc7e060, FUTEX_WAKE_PRIVATE, 1) = 1
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=3, tv_nsec=721822197}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=114795}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
rt_sigprocmask(SIG_BLOCK, ~[], [HUP INT USR1 USR2 TERM], 8) = 0
clone3({flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, child_tid=0x7fb00d600990, parent_tid=0x7fb00d600990, exit_signal=0, stack=0x7fb00ce00000, stack_size=0x7fff80, tls=0x7fb00d6006c0} => {parent_tid=[12039]}, 88) = 12039
rt_sigprocmask(SIG_SETMASK, [HUP INT USR1 USR2 TERM], NULL, 8) = 0
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
futex(0x7fb00fc7e060, FUTEX_WAKE_PRIVATE, 1) = 1
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=339646}, {sigmask=[], sigsetsize=8}) = 1 (in [3], left {tv_sec=4, tv_nsec=299400})
accept(3, {sa_family=AF_UNIX}, [110 => 2]) = 10
rt_sigprocmask(SIG_BLOCK, ~[], [HUP INT USR1 USR2 TERM], 8) = 0
clone3({flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, child_tid=0x7fb00e000990, parent_tid=0x7fb00e000990, exit_signal=0, stack=0x7fb00d800000, stack_size=0x7fff80, tls=0x7fb00e0006c0} => {parent_tid=[12040]}, 88) = 12040
rt_sigprocmask(SIG_SETMASK, [HUP INT USR1 USR2 TERM], NULL, 8) = 0
futex(0x7fb00fc7e060, FUTEX_WAKE_PRIVATE, 1) = 1
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=3, tv_nsec=999706340}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=111468}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
newfstatat(AT_FDCWD, "/home/user/.gnupg", {st_mode=S_IFDIR|0700, st_size=372, ...}, 0) = 0
pselect6(7, [3 4 5 6], NULL, NULL, {tv_sec=4, tv_nsec=108423}, {sigmask=[], sigsetsize=8} <detached ...>

here is a strace from keyboxd:

pselect6(6, [3 4 5], NULL, NULL, {tv_sec=1, tv_nsec=156218382}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=4, tv_nsec=361}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=4, tv_nsec=280}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=4, tv_nsec=390}, {sigmask=[], sigsetsize=8}) = 1 (in [3], left {tv_sec=1, tv_nsec=548978755})
accept(3, {sa_family=AF_UNIX}, [110 => 2]) = 6
rt_sigprocmask(SIG_BLOCK, ~[], [HUP INT USR1 USR2 TERM], 8) = 0
clone3({flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, child_tid=0x7f629be00990, parent_tid=0x7f629be00990, exit_signal=0, stack=0x7f629b600000, stack_size=0x7fff80, tls=0x7f629be006c0} => {parent_tid=[12456]}, 88) = 12456
rt_sigprocmask(SIG_SETMASK, [HUP INT USR1 USR2 TERM], NULL, 8) = 0
futex(0x7f62a2689060, FUTEX_WAKE_PRIVATE, 1) = 1
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=1, tv_nsec=548581642}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=4, tv_nsec=531}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=4, tv_nsec=301}, {sigmask=[], sigsetsize=8}) = 1 (in [3], left {tv_sec=0, tv_nsec=191218874})
accept(3, {sa_family=AF_UNIX}, [110 => 2]) = 8
rt_sigprocmask(SIG_BLOCK, ~[], [HUP INT USR1 USR2 TERM], 8) = 0
clone3({flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, child_tid=0x7f62a1c00990, parent_tid=0x7f62a1c00990, exit_signal=0, stack=0x7f62a1400000, stack_size=0x7fff80, tls=0x7f62a1c006c0} => {parent_tid=[12460]}, 88) = 12460
rt_sigprocmask(SIG_SETMASK, [HUP INT USR1 USR2 TERM], NULL, 8) = 0
futex(0x7f62a2689060, FUTEX_WAKE_PRIVATE, 1) = 1
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=0, tv_nsec=190621570}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=4, tv_nsec=420}, {sigmask=[], sigsetsize=8}) = 0 (Timeout)
pselect6(6, [3 4 5], NULL, NULL, {tv_sec=4, tv_nsec=501}, {sigmask=[], sigsetsize=8} <detached ...

Here is a test about db lock:

❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db
~
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db                                                                                                                         
~ ## Make access to gpg
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db                                                                                                                         
Runtime error near line 1: database is locked (5) 
~
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db
Runtime error near line 1: database is locked (5) 
~
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db
Runtime error near line 1: database is locked (5) 
~
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db
Runtime error near line 1: database is locked (5) 
~
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db
Runtime error near line 1: database is locked (5) 
~
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db
Runtime error near line 1: database is locked (5) 
~ ## touching yubikey
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db
~                          
❯ echo 'BEGIN EXCLUSIVE; COMMIT;' | sqlite3  ~/.gnupg/public-keys.d/pubring.db

@mochaaP
Copy link
Contributor

mochaaP commented Mar 30, 2024

seems great, will try to implement this after I finish my work on some Win32 drivers 😅

@mochaaP
Copy link
Contributor

mochaaP commented Apr 2, 2024

done that. will look at it later this weekend

@shumvgolove
Copy link
Author

Sorry for the delay!

I guess it's a little bit late to include debug logs since @Zempashi provided them, but just in case, I've got similar results with pubring.db locking.

Thanks for working on it @mochaaP and congrats with win drivers 🙂

@mochaaP
Copy link
Contributor

mochaaP commented Apr 3, 2024

seems this is quite trivial: we just need to poll fcntl(2) to detect the lock. or we could link to sqlite3, but that might be overkill just for this task.

@maximbaz
Copy link
Owner

maximbaz commented Apr 3, 2024

By linking to sqlite3, would we be able to be "notified" about the lock, instead of constantly polling?

@mochaaP
Copy link
Contributor

mochaaP commented Apr 4, 2024

to my knowledge, no. the underlying syscall is the same after all.

@mochaaP
Copy link
Contributor

mochaaP commented Apr 4, 2024

But it would be easier to find the offset to lock instead of reading the database ourselves

@maximbaz
Copy link
Owner

maximbaz commented Apr 4, 2024

I'd really love us to first explore some other ways which can be event-based, instead of poll-based... For example, although inotify's open event does not trigger every time anymore, perhaps read or access events do. If yes, it would be great to know how often they get triggered when yubikey does not wait for a touch. If we have an event, that doesn't have too much of false positives, then we can additionally filter those out either by using the lock check, or straight with the existing CheckGPGOnRequest, depending on the amount of false-positives... 🤔

It looks like if we simply back up our .gnupg folder and remove it, it will be recreated in the new format, and so it should be easier to test this for us, and yet have a fallback 😁

@mochaaP
Copy link
Contributor

mochaaP commented Apr 4, 2024

I already poked around using inotify, unfortunately, with no avail. 😿

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants