Request: Certificate validation to use OS trusted root certificates first

[MallocArray]

Description

The updated notes about what types of certificates are expected for the ca_path parameter was helpful in getting me going with them.
I do still have concerns about the overall burden needed to manage these certificates, when other Ansible modules can validate certs without needing to specify a path to specific files.

Ideally, if the host that is running Ansible already trusts the root CA, then it seems like a certificate path shouldn't be necessary. Only if it wasn't already trusted, then it would make sense to need to specify a ca_path for a certain cert bundle.

In my case I have a mix of devices that have certificates issues by DigiCert, an external CA, as well as our internal CA. Between our AWX servers, development boxes, and Execution Environments, we have worked to make sure our internal CAs are add as trusted roots where we will be using Ansible playbooks.

With needing to specify the exact location of a certificate, we have to ensure all of these certs are in a central location that our Ansible related playbooks can reach.
We have a ./files/certificates folder in our git repository, but as this runs either on our development servers, or inside of AWX, or inside of an Execution Environment, the full path can vary widely, so we have to use a relative path to these certificates.
The problem we are running into now is the ca_path parameter is relative to the playbook location and not the working directory where you launch ansible-playbook from.

In our case, we some playbooks in the following folder structure:
./Dell
./Dell/networking
./Dell/idrac

If we are running a playbook that lives in ./Dell/networking, we would run this with something like ansible-playbook ./Dell/networking/manage_vlan.yml ... and in order for the ca_path to work, we could have to use
ca_path: ../../files/certificates/DigiCert_root_bundle.pem
This would go back 2 folders to the root of the project, and then go into the files/certificates folder to find the .pem needed.
Since I want to make our configs using variables as much as I can, I store this location as a group_var for the hosts so I have a single location to manage it.

The next problem comes when I want to run a playbook that isn't in a subfolder of the Dell folder, but directly in this folder, such as ./Dell/ome_config.yml
Now when it uses the variable for the ca_path it still goes up 2 folders, which is now too far back and can't find /files/certificate path anymore. One solution would be to hard code this path, but as we have playbooks running on multiple platforms, the full path changes so that won't work.

The final issue, is that some of our system have a different intermediate CA from Digicert and some use the internal CA, so a single bundle from Digicert doesn't work for all hosts and the ones using our internal CA need completely different certs.

All of these are trusted by our hosts, but needing to specify a path is causing significant issues and either needs a sprawl of new variables or hard coded paths, or the easiest solution is just to turn off validate_certs

I've looked at other Ansible modules we are using and some are able to have a validate_certs parameter without needing a cert path:
https://docs.ansible.com/ansible/latest/collections/community/general/ldap_entry_module.html
https://docs.ansible.com/ansible/latest/collections/awx/awx/user_module.html
While some do require a ca_path parameter:
https://docs.ansible.com/ansible/latest/collections/ansible/builtin/uri_module.html

Questions:

• I'm not familiar with Python, but is there any way to get the Dell modules to at least try the host's trusted root CA list first and then only need the ca_path if it isn't in that list?
• With validate_certs: false in the 5.0.x collection today, does it still use SSL and just doesn't validate it is trusted, or it is completely unsecure in plain text?

New or existing component

5.x release that requires a ca_path to a specific certificate chain for validate_certs parameter instead of using OS's default list of trusted roots, just like a browser will trust all of the trusted root certs without prompting

Community Note

• Please vote on this issue by adding a 👍 reaction
  to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions,
  they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[sachin-apa]

@MallocArray Thanks For Sharing the Scenarios.

You are right OS trusted certificates can be used to validate certificates.

we do have a plan for adding a dependent library certifi which helps in identifying the right path of the OS trusted certificates for python and read from the same.
This path contains curated list of Trusted certificates

We will retain the precedence to the env variable if set, if not we would read the ca certificate from the path provided by the certifi library.
If user has a mix of self signed certificates and public certificates, self signed certificates can be appended to the existing bundle provided by the certifi path python -m certifi , or the if only self signed certificate is used then they can directly set the environment variable.

Do please share your input if any.

[MallocArray]

certifi is interesting, but looks like it is its own copy of trusted certs and not directly connected to the OS current list of trusted certs. I already have a step where I copy our personal trusted roots to the ca-certificates or trust anchor folder and run the required commands to have the OS update the list. With this proposed solution of using certifi, I would also need to append it's cacert.pem with all of my trusted roots as well. This is doable, but not really using the OS's existing trusted list but making yet another copy, although that copy should be in a consistent path.

In your proposed case where the ENV var is used first and then the ca path from certifi, would it also support a tertiary option of the ca_path: parameter if I did have a single specific file I wanted to use, or does ca_path go away entirely? I'm thinking of wanting certifi to be in addition to the ability to specify a specific cert path as part of an Ansible parameter and not forcing only an ENV var for that use case. Want to keep it feeling like a native Ansible module as much as possible, but ca_path should be an option field and if not filled out, then use the ENV var or certifi provided file.

[sachin-apa]

@MallocArray

OS list of certificate path varies and depends on different OS and hence we cannot decide on path, in this case I believe it is better to set the env variable with a path.

After a patch ca_path parameter is made optional and the priority is from playbook param ca_path and then the env variable.

Enhancement will be to check certifi path if ca_path is not passed as a parameter and env variables is not set. So yes ca_path will continue to be optional parameter from playbook and the precedence will be as follows.

1. ca_path from playbook
2. env variables
3. certifi lib path

@anupamaloke Do please add if I have missed anything.

[MallocArray]

I agree with this plan. Certifi should be able to take care of many cases, especially if we add custom certs to the file certifi uses, but can also add a static path with ca_path if needed.