Support "lazy loading" for SSL Certificates

[brianehlert]

NGINX Plus read certificates from disk, on-demand, and certificates can be replaced without needing to rewrite configuration or reload NGINX.
This is known a lazy loading. And is optimal for large scale and environments with high volume of change.
Adds general value to NGINX Plus Ingress Controller, and general secret updates. (Secret updates should not trigger a reload)

https://www.nginx.com/blog/nginx-plus-r18-released/#dynamic-certificate-loading_disk
The value is that certificates can be updated dynamically without incurring the tax of reload.

This is not moving certificate management to the key/value store which has the potential of increased and unexpected memory use and impacts resulting in OOM for the pods.

[lucacome]

Should we open a separate discussion for in-memory certificates? I think it would be the most beneficial for use especially from a security point of view.

[brianehlert]

If we want to do in-memory, that should be a separate discussion.
The concern I have is that is that the memory consumption grows pretty steadily with additional certificates that need to be stored.

If we begin with lazy loading as a new default, we can expand to in-memory as a conscious choice. Mainly so the person deploying is conscious of the memory use impacts and not surprised by an OOMKill cascade across their NIC pods.

[brianehlert]

Released with 3.4: #4788