curl-originated error messages not informative enough

[pitrou]

Describe the bug

While trying to debug a TLS certificate issue, I found out that the AWS SDK returned poorly informative error messages, such as:

OSError: When listing buckets: AWS Error NETWORK_CONNECTION during ListBuckets operation: curlCode: 60, SSL peer certificate or SSH remote key was not OK

I then tried using the curl command line directly and, to my surprise, the error message was much more informative:

curl: (60) SSL: no alternative certificate subject name matches target host name '127.0.0.1'
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

I inquired on the curl issue tracker and it turns out that the AWS SDK should use the CURLOPT_ERRORBUFFER option so as to get the detailed error message.

Expected Behavior

The AWS SDK should give out the detailed error made available by libcurl, such as "no alternative certificate subject name matches target host name: ...".

Current Behavior

The AWS SDK gives out an unhelpful error message, as returned by libcurl when CURLOPT_ERRORBUFFER is not set.

Reproduction Steps

Sorry, I have nothing self-contained unless you think of installing PyArrow as "self-contained". However, this is straightforward enough that it probably doesn't need reproduction?

In any case, see apache/arrow#37001 (comment) for an example.

Possible Solution

The AWS SDK could reserve a dedicated error buffer for each libcurl connection and set the CURLOPT_ERRORBUFFER with it.

Additional Information/Context

No response

AWS CPP SDK version used

1.11.267, but current git main seems affected

Compiler and Version used

gcc 12.3.0

Operating System and version

Ubuntu 22.04

[jmklix]

We could make a PR similar to this one to give better error messages for libCurl: #2676