SSH Domain auth with certificates but without ssh-agent?

[tomtastic]

What Operating System(s) are you running on?

Windows

Which Wayland compositor or X11 Window manager(s) are you using?

No response

WezTerm version

wezterm 20240429-102818-0184e88e

Ask your question!

Is it possible to connect to a remote SSH domain multiplex using SSH certificates (via libssh), but without requiring them to be available in an ssh-agent ?

eg. In the same way that a simple ssh connection wouldn't require the ssh-agent?

$ ssh -v target_server
debug1: Will attempt key: C:\\Users\\user/.ssh/id_rsa
debug1: Will attempt key: C:\\Users\\user/.ssh/id_ecdsa ECDSA SHA256:GPcZoAOWDF3mEc9ZNe2qDsa71ottMJPJyNsx13vB+Z4
debug1: Will attempt key: C:\\Users\\user/.ssh/id_ecdsa ECDSA-CERT SHA256:GPcZoAOWDF3mEc9ZNe2qDsa71ottMJPJyNsx13vB+Z4
debug1: Offering public key: C:\\Users\\user/.ssh/id_ecdsa ECDSA SHA256:GPcZoAOWDF3mEc9ZNe2qDsa71ottMJPJyNsx13vB+Z4
debug1: Authentications that can continue: publickey
debug1: Offering public key: C:\\Users\\user/.ssh/id_ecdsa ECDSA-CERT SHA256:GPcZoAOWDF3mEc9ZNe2qDsa71ottMJPJyNsx13vB+Z4
debug1: Server accepts key: C:\\Users\\user/.ssh/id_ecdsa ECDSA-CERT SHA256:GPcZoAOWDF3mEc9ZNe2qDsa71ottMJPJyNsx13vB+Z4
Authenticated to target_server ([10.0.0.1]:22) using "publickey".

When I try in WezTerm, I get:

> wezterm connect DD     # <-- the mux domain for target_server
14:28:02.573  ERROR  mux::connui >
Failed: Error: unhandled auth case; methods=PUBLIC_KEY, status={PUBLIC_KEY: Denied}

14:28:02.595  ERROR  wezterm_gui > Error: unhandled auth case; methods=PUBLIC_KEY, status={PUBLIC_KEY: Denied}; terminating
14:28:02.595  ERROR  mux::connui > while running ConnectionUI loop: recv_timeout: channel is empty and disconnected

[wez]

Likely needs support from libssh. I'd suggest using wezterm ssh -v HOSTNAME when troubleshooting/debugging this sort of issue, as it gives the most useful diagnostics from libssh.

[tomtastic]

> dir .ssh
  Directory of C:\Users\tmatoo\.ssh

08/05/2024  15:56    <DIR>          .
08/05/2024  15:56    <DIR>          ..
08/05/2024  14:53             1,578 config
08/05/2024  14:30               634 id_ecdsa
13/05/2024  07:22             1,338 id_ecdsa-cert.pub
08/05/2024  14:30               233 id_ecdsa.pub

> wezterm ssh -v target_server
07:22:50.983  INFO   mux::ssh > Using ssh config: {
    "hostname": "target_server",
    "identityfile": "C:\\Users\\user/.ssh/id_dsa C:\\Users\\user/.ssh/id_ecdsa C:\\Users\\user/.ssh/id_ed25519 C:\\Users\\user/.ssh/id_rsa",
    "port": "22",
    "user": "user",
    "userknownhostsfile": "C:\\Users\\user/.ssh/known_hosts C:\\Users\\user/.ssh/known_hosts2",
    "wezterm_ssh_backend": "libssh",
    "wezterm_ssh_verbose": "true",
}
07:22:50.989  INFO   libssh::ssh_config_parse_file > Reading configuration data from C:\Users\user/.ssh/config
07:22:51.083  INFO   libssh::ssh_config_parse_line > Unsupported option: ServerAliveInterval, line: 17
07:22:51.120  INFO   libssh::ssh_config_parse_line > Unsupported option: ServerAliveCountMax, line: 18
07:22:51.122  INFO   libssh::ssh_config_parse_line > Unsupported option: ForwardAgent, line: 36
07:22:51.123  INFO   libssh::ssh_config_parse_line > Unsupported option: AddKeysToAgent, line: 37
07:22:51.231  INFO   wezterm_ssh::sessioninner     > resolved target_server:22 -> 10.0.0.1:22
07:22:51.249  INFO   libssh::ssh_connect           > libssh 0.8.90 (c) 2003-2023 Aris Adamantiadis, Andreas Schneider and libssh contributors. Distributed under the LGPL, please refer to COPYING file for information about your rights, using threading threads_winlock
07:22:51.274  INFO   libssh::ssh_connect           > Socket connecting, now waiting for the callbacks to work
07:22:51.276  INFO   libssh::ssh_connect           > Actual timeout : 10000
07:22:51.306  INFO   libssh::ssh_socket_pollcallback > Received POLLOUT in connecting state
07:22:51.307  INFO   libssh::socket_callback_connected > Socket connection callback: 1 (0)
07:22:51.308  INFO   libssh::ssh_client_connection_callback > session_state=2
07:22:51.309  INFO   libssh::ssh_socket_unbuffered_write    > Enabling POLLOUT for socket
07:22:51.312  INFO   libssh::callback_receive_banner        > Received banner: SSH-2.0-OpenSSH_7.4p1-RHEL7-7.4p1-22 mdy1.0
07:22:51.314  INFO   libssh::ssh_client_connection_callback > session_state=3
07:22:51.316  INFO   libssh::ssh_client_connection_callback > SSH server banner: SSH-2.0-OpenSSH_7.4p1-RHEL7-7.4p1-22 mdy1.0
07:22:51.349  INFO   libssh::ssh_analyze_banner             > Analyzing banner: SSH-2.0-OpenSSH_7.4p1-RHEL7-7.4p1-22 mdy1.0
07:22:51.355  INFO   libssh::ssh_analyze_banner             > We are talking to an OpenSSH server version: 7.4 (70400)
07:22:51.365  INFO   libssh::ssh_client_select_hostkeys     > Order of wanted host keys: "ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256"
07:22:51.397  INFO   libssh::ssh_known_hosts_read_entries   > Failed to open the known_hosts file '/etc/ssh/ssh_known_hosts': No such file or directory
07:22:51.439  INFO   libssh::ssh_client_select_hostkeys     > Algorithms found in known_hosts files: "rsa-sha2-512,rsa-sha2-256,ssh-rsa"
07:22:51.444  INFO   libssh::ssh_client_select_hostkeys     > Changing host key method to "rsa-sha2-512,rsa-sha2-256,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com"
07:22:51.446  INFO   libssh::packet_send2                   > packet: wrote [type=20, len=908, padding_size=11, comp=896, payload=896]
07:22:51.449  INFO   libssh::ssh_send_kex                   > SSH_MSG_KEXINIT sent
07:22:51.454  INFO   libssh::ssh_socket_unbuffered_write    > Enabling POLLOUT for socket
07:22:51.498  INFO   libssh::ssh_packet_socket_callback     > packet: read type 20 [len=1276,padding=5,comp=1270,payload=1270]
07:22:51.502  INFO   libssh::ssh_packet_process             > Dispatching handler for packet type 20
07:22:51.504  INFO   libssh::ssh_client_connection_callback > session_state=5
07:22:51.507  INFO   libssh::ssh_kex_select_methods         > Negotiated curve25519-sha256,rsa-sha2-512,chacha20-poly1305@openssh.com,chacha20-poly1305@openssh.com,aead-poly1305,aead-poly1305,none,none,,
07:22:51.510  INFO   libssh::packet_send2                   > packet: wrote [type=30, len=44, padding_size=6, comp=37, payload=37]
07:22:51.513  INFO   libssh::ssh_socket_unbuffered_write    > Enabling POLLOUT for socket
07:22:51.532  INFO   libssh::ssh_packet_socket_callback     > packet: read type 31 [len=612,padding=11,comp=600,payload=600]
07:22:51.534  INFO   libssh::ssh_packet_process             > Dispatching handler for packet type 31
07:22:51.539  INFO   libssh::ssh_socket_unbuffered_write    > Enabling POLLOUT for socket
07:22:51.542  INFO   libssh::packet_send2                   > packet: wrote [type=21, len=12, padding_size=10, comp=1, payload=1]
07:22:51.665  INFO   libssh::crypt_set_algorithms2          > Set output algorithm to chacha20-poly1305@openssh.com
07:22:51.668  INFO   libssh::crypt_set_algorithms2          > Set HMAC output algorithm to aead-poly1305
07:22:51.670  INFO   libssh::crypt_set_algorithms2          > Set input algorithm to chacha20-poly1305@openssh.com
07:22:51.674  INFO   libssh::crypt_set_algorithms2          > Set HMAC input algorithm to aead-poly1305
07:22:51.676  INFO   libssh::ssh_init_rekey_state           > Set rekey after 134217728 blocks
07:22:51.678  INFO   libssh::ssh_init_rekey_state           > Set rekey after 134217728 blocks
07:22:51.680  INFO   libssh::ssh_packet_client_curve25519_reply > SSH_MSG_NEWKEYS sent
07:22:51.686  INFO   libssh::ssh_packet_socket_callback         > Processing 100 bytes left in socket buffer
07:22:51.690  INFO   libssh::ssh_packet_socket_callback         > packet: read type 21 [len=12,padding=10,comp=1,payload=1]
07:22:51.692  INFO   libssh::ssh_packet_process                 > Dispatching handler for packet type 21
07:22:51.695  INFO   libssh::ssh_packet_newkeys                 > Received SSH_MSG_NEWKEYS
07:22:51.703  INFO   libssh::ssh_packet_newkeys                 > Signature verified and valid
07:22:51.706  INFO   libssh::ssh_client_connection_callback     > session_state=6
07:22:51.708  INFO   libssh::ssh_packet_socket_callback         > Processing 84 bytes left in socket buffer
07:22:51.723  INFO   libssh::ssh_packet_socket_callback         > packet: read type 7 [len=64,padding=10,comp=53,payload=53]
07:22:51.734  INFO   libssh::ssh_packet_process                 > Dispatching handler for packet type 7
07:22:51.737  INFO   libssh::ssh_packet_ext_info                > Received SSH_MSG_EXT_INFO
07:22:51.739  INFO   libssh::ssh_packet_ext_info                > Follows 1 extensions
07:22:51.741  INFO   libssh::ssh_packet_ext_info                > Extension: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
07:22:51.744  INFO   libssh::ssh_connect                        > current state : 7
07:22:51.753  INFO   libssh::packet_send2                       > packet: wrote [type=5, len=24, padding_size=6, comp=17, payload=17]
07:22:51.756  INFO   libssh::ssh_service_request                > Sent SSH_MSG_SERVICE_REQUEST (service ssh-userauth)
07:22:51.793  INFO   libssh::ssh_socket_unbuffered_write        > Enabling POLLOUT for socket
07:22:51.808  INFO   libssh::ssh_packet_socket_callback         > packet: read type 6 [len=24,padding=6,comp=17,payload=17]
07:22:51.814  INFO   libssh::ssh_packet_process                 > Dispatching handler for packet type 6
07:22:51.817  INFO   libssh::ssh_packet_service_accept          > Received SSH_MSG_SERVICE_ACCEPT
07:22:51.818  INFO   libssh::ssh_socket_unbuffered_write        > Enabling POLLOUT for socket
07:22:51.883  INFO   libssh::packet_send2                       > packet: wrote [type=50, len=48, padding_size=10, comp=37, payload=37]
07:22:51.899  INFO   libssh::ssh_packet_socket_callback         > packet: read type 51 [len=24,padding=8,comp=15,payload=15]
07:22:51.901  INFO   libssh::ssh_packet_process                 > Dispatching handler for packet type 51
07:22:51.905  INFO   libssh::ssh_packet_userauth_failure        > Access denied for 'none'. Authentication that can continue: publickey
07:22:51.907  INFO   libssh::ssh_packet_userauth_failure        > Access denied for 'none'. Authentication that can continue: publickey
07:22:51.909  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_rsa
07:22:51.911  INFO   libssh::ssh_pki_import_pubkey_file         > Error opening C:\Users\user/.ssh/id_rsa.pub: No such file or directory
07:22:51.913  INFO   libssh::ssh_pki_import_privkey_file        > Error opening C:\Users\user/.ssh/id_rsa: No such file or directory
07:22:51.915  INFO   libssh::ssh_userauth_publickey_auto        > Private key C:\Users\user/.ssh/id_rsa doesn't exist.
07:22:51.917  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_ed25519
07:22:51.919  INFO   libssh::ssh_pki_import_pubkey_file         > Error opening C:\Users\user/.ssh/id_ed25519.pub: No such file or directory
07:22:51.921  INFO   libssh::ssh_pki_import_privkey_file        > Error opening C:\Users\user/.ssh/id_ed25519: No such file or directory
07:22:51.923  INFO   libssh::ssh_userauth_publickey_auto        > Private key C:\Users\user/.ssh/id_ed25519 doesn't exist.
07:22:51.925  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_ecdsa
07:22:51.927  INFO   libssh::ssh_key_algorithm_allowed          > Checking ecdsa-sha2-nistp384 with list <ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256>
07:22:51.929  INFO   libssh::ssh_socket_unbuffered_write        > Enabling POLLOUT for socket
07:22:51.931  INFO   libssh::packet_send2                       > packet: wrote [type=50, len=216, padding_size=9, comp=206, payload=206]
07:22:51.947  INFO   libssh::ssh_packet_socket_callback         > packet: read type 51 [len=24,padding=8,comp=15,payload=15]
07:22:51.950  INFO   libssh::ssh_packet_process                 > Dispatching handler for packet type 51
07:22:51.982  INFO   libssh::ssh_packet_userauth_failure        > Access denied for 'publickey'. Authentication that can continue: publickey
07:22:51.984  INFO   libssh::ssh_packet_userauth_failure        > Access denied for 'publickey'. Authentication that can continue: publickey
07:22:51.986  INFO   libssh::ssh_userauth_publickey_auto        > Public key for C:\Users\user/.ssh/id_ecdsa refused by server
07:22:51.988  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_dsa
07:22:51.990  INFO   libssh::ssh_pki_import_pubkey_file         > Error opening C:\Users\user/.ssh/id_dsa.pub: No such file or directory
07:22:51.991  INFO   libssh::ssh_pki_import_privkey_file        > Error opening C:\Users\user/.ssh/id_dsa: No such file or directory
07:22:51.993  INFO   libssh::ssh_userauth_publickey_auto        > Private key C:\Users\user/.ssh/id_dsa doesn't exist.
07:22:51.995  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_ed25519
07:22:51.996  INFO   libssh::ssh_pki_import_pubkey_file         > Error opening C:\Users\user/.ssh/id_ed25519.pub: No such file or directory
07:22:51.998  INFO   libssh::ssh_pki_import_privkey_file        > Error opening C:\Users\user/.ssh/id_ed25519: No such file or directory
07:22:51.999  INFO   libssh::ssh_userauth_publickey_auto        > Private key C:\Users\user/.ssh/id_ed25519 doesn't exist.
07:22:52.001  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_ecdsa
07:22:52.003  INFO   libssh::ssh_key_algorithm_allowed          > Checking ecdsa-sha2-nistp384 with list <ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256>
07:22:52.005  INFO   libssh::ssh_socket_unbuffered_write        > Enabling POLLOUT for socket
07:22:52.007  INFO   libssh::packet_send2                       > packet: wrote [type=50, len=216, padding_size=9, comp=206, payload=206]
07:22:52.025  INFO   libssh::ssh_packet_socket_callback         > packet: read type 51 [len=24,padding=8,comp=15,payload=15]
07:22:52.026  INFO   libssh::ssh_packet_process                 > Dispatching handler for packet type 51
07:22:52.029  INFO   libssh::ssh_packet_userauth_failure        > Access denied for 'publickey'. Authentication that can continue: publickey
07:22:52.030  INFO   libssh::ssh_packet_userauth_failure        > Access denied for 'publickey'. Authentication that can continue: publickey
07:22:52.032  INFO   libssh::ssh_userauth_publickey_auto        > Public key for C:\Users\user/.ssh/id_ecdsa refused by server
07:22:52.042  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_rsa
07:22:52.120  INFO   libssh::ssh_pki_import_pubkey_file         > Error opening C:\Users\user/.ssh/id_rsa.pub: No such file or directory
07:22:52.122  INFO   libssh::ssh_pki_import_privkey_file        > Error opening C:\Users\user/.ssh/id_rsa: No such file or directory
07:22:52.124  INFO   libssh::ssh_userauth_publickey_auto        > Private key C:\Users\user/.ssh/id_rsa doesn't exist.
07:22:52.126  INFO   libssh::ssh_userauth_publickey_auto        > Trying to authenticate with C:\Users\user/.ssh/id_dsa
07:22:52.128  INFO   libssh::ssh_pki_import_pubkey_file         > Error opening C:\Users\user/.ssh/id_dsa.pub: No such file or directory
07:22:52.129  INFO   libssh::ssh_pki_import_privkey_file        > Error opening C:\Users\user/.ssh/id_dsa: No such file or directory
07:22:52.131  INFO   libssh::ssh_userauth_publickey_auto        > Private key C:\Users\user/.ssh/id_dsa doesn't exist.
07:22:52.133  INFO   libssh::ssh_userauth_publickey_auto        > Tried every public key, none matched

Using libssh-rs to connect to user@target_server:22
SSH-2.0-OpenSSH_7.4p1-RHEL7-7.4p1-22 mdy1.0

⚠️  Process RemoteSshDomain didn't exit cleanly
Exited with code 1.
This message is shown because exit_behavior="CloseOnCleanExit"Error: unhandled auth case; methods=PUBLIC_KEY, status={PUBLIC_KEY: Denied}

[wez]

libssh doesn't currently support this. https://gitlab.com/libssh/libssh-mirror/-/merge_requests/415 added support for it, but it is not currently in a released version of libssh.

[tomtastic]

Thanks Wez, I hope it gets to an actual release version as we're heavy users of SSH certs :)