Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[16.0] fs_storage: support SSH private keys authentication #331

Open
wants to merge 4 commits into
base: 16.0
Choose a base branch
from

Conversation

sebalix
Copy link
Contributor

@sebalix sebalix commented Jan 29, 2024

SSH connections can now be done with private keys by setting the pkey+ passphrase options. Coupled with the eval_options_from_env this allows to set these ones from the environment, e.g:

{"host": "sftp.example.net", "username": "odoo", "pkey": "$SSH_KEY", "passphrase": "$SSH_PASSPHRASE", "port": 22}

Copy link
Contributor

@simahawk simahawk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG. Could we have some test cov?

@sebalix sebalix force-pushed the 16-fs_storage-add-ssh-private-keys-support branch from c2fe279 to 4b47b01 Compare January 29, 2024 10:38
@sebalix
Copy link
Contributor Author

sebalix commented Jan 29, 2024

@simahawk what's the best approach here to test? Current module doesn't depend on paramiko, if this one is installed then fsspec can use it. To test the feature I would need to add paramiko as tests dependencies, is it OK?

Or should we put tests in a dedicated test_fs_storage_ssh module?

SSH connections can now be done with private keys by setting the `pkey`
+ `passphrase` options.
Coupled with the `eval_options_from_env` this allows to set these ones
from the environment, e.g:

`{"host": "sftp.example.net", "username": "sftp", "pkey": "$SSH_KEY", "passphrase": "$SSH_PASSPHRASE", "port": 22}`
@sebalix sebalix force-pushed the 16-fs_storage-add-ssh-private-keys-support branch from 4b47b01 to c0b3315 Compare January 29, 2024 14:28
@sebalix
Copy link
Contributor Author

sebalix commented Jan 29, 2024

I added an extra parameter pkey_type to force the key type (RSA, DSS, OPENSSH...).
It happens that some versions of ssh-keygen generates OPENSSH keys even with the -t rsa parameter if -m PEM is not set.
https://serverfault.com/questions/939909/ssh-keygen-does-not-create-rsa-private-key

Still need to test it a bit more.

EDIT: this doesn't work neither. Better to generate the key in a proper manner with ssh-keygen -t rsa -m PEM [...], and everything works as expected. I reverted my last changes.

@sebalix sebalix force-pushed the 16-fs_storage-add-ssh-private-keys-support branch from 11f29f1 to 6f33885 Compare January 29, 2024 16:41
@simahawk
Copy link
Contributor

@simahawk what's the best approach here to test? Current module doesn't depend on paramiko, if this one is installed then fsspec can use it. To test the feature I would need to add paramiko as tests dependencies, is it OK?

Or should we put tests in a dedicated test_fs_storage_ssh module?

In fact, we are adding too many SFTP related feature to the base module... IMO we need a separated module for sftp: fs_storage_sftp.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants