Fix SNI tests for LibreSSL #2041

lgv5 · 2023-02-25T22:15:57Z

Since beginning of 2022, LibreSSL dropped support for IP addresses in SNI. This is aligned to RFC 6066, section 3:

Literal IPv4 and IPv6 addresses are not permitted in "HostName".

In order to deal with it, make servers listen on and clients connect to "localhost" instead of "127.0.0.1", adjust "server.crt" to use "localhost" as Common Name instead of "127.0.0.1", and default Mojo::IOLoop::Client address to "localhost" instead of "127.0.0.1".

Summary

Check commit message.

Motivation

I use OpenBSD, which ships with LibreSSL.

References

Didn't find any.

Grinnz · 2023-02-25T22:21:32Z

I'm not sure what the right solution is but I'll note that attempting to connect to localhost breaks more often.

jberger · 2023-02-25T22:28:21Z

I'm not sure what the right solution is but I'll note that attempting to connect to localhost breaks more often.

We might first check that either localhost resolves or check /etc/hosts for localhost, otherwise skip the test.

kraih · 2023-02-25T22:46:32Z

And don't forget to squash your commits.

Grinnz · 2023-02-25T22:49:54Z

I'm not sure what the right solution is but I'll note that attempting to connect to localhost breaks more often.

We might first check that either localhost resolves or check /etc/hosts for localhost, otherwise skip the test.

I'm referring here to the change in Mojo::IOLoop::Client, which will affect all existing code.

lgv5 · 2023-02-25T23:07:54Z

And don't forget to squash your commits.

Sure thing. I didn't check enough PRs to see if it was common to force-push mid-review, and I prefer not to do so. Will squash once / if this ready to be merged.

I'm not sure what the right solution is but I'll note that attempting to connect to localhost breaks more often.

Well, to be honest, it was done a bit out of laziness. I can do a second pass thru the tests replacing all the instances of unset address with address => "localhost".

kraih · 2023-02-26T02:27:55Z

If i remember correctly, last time we used localhost instead of 127.0.0.1, we got a lot of failing test results from cpantesters. I wonder if the situation has changed.

lgv5 · 2023-02-27T09:15:48Z

I rolled back the change for Mojo::IOLoop::Client default address and fixed the tests accordingly. For my own surprise, the only affected test was t/mojo/user_agent_tls.t. The full test suite passes "in my machine(tm)":

$ env TEST_ALL=Yes make test
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/*/*.t
t/mojo/asset.t ............................. ok
t/mojo/base.t .............................. ok
t/mojo/bytestream.t ........................ ok
t/mojo/cache.t ............................. ok
t/mojo/cgi.t ............................... ok
t/mojo/collection.t ........................ ok
t/mojo/content.t ........................... ok
t/mojo/cookie.t ............................ ok
t/mojo/cookiejar.t ......................... ok
t/mojo/daemon.t ............................ ok
t/mojo/daemon_ipv6_tls.t ................... ok
t/mojo/date.t .............................. ok
t/mojo/dom.t ............................... ok
t/mojo/dynamic_methods.t ................... ok
t/mojo/eventemitter.t ...................... ok
t/mojo/exception.t ......................... ok
t/mojo/file.t .............................. ok
t/mojo/headers.t ........................... ok
t/mojo/home.t .............................. ok
t/mojo/hypnotoad.t ......................... ok
t/mojo/ioloop.t ............................ ok
t/mojo/ioloop_ipv6.t ....................... ok
t/mojo/ioloop_tls.t ........................ ok
t/mojo/json.t .............................. ok
t/mojo/json_pointer.t ...................... ok
t/mojo/json_xs.t ........................... ok
t/mojo/loader.t ............................ ok
t/mojo/log.t ............................... ok
t/mojo/morbo.t ............................. ok
t/mojo/parameters.t ........................ ok
t/mojo/path.t .............................. ok
t/mojo/prefork.t ........................... ok
t/mojo/promise.t ........................... ok
t/mojo/promise_async_await.t ............... ok
t/mojo/proxy.t ............................. ok
t/mojo/psgi.t .............................. ok
t/mojo/reactor_detect.t .................... ok
t/mojo/reactor_ev.t ........................ ok
t/mojo/reactor_poll.t ...................... ok
t/mojo/request.t ........................... ok
t/mojo/request_cgi.t ....................... ok
t/mojo/response.t .......................... ok
t/mojo/roles.t ............................. ok
t/mojo/signatures.t ........................ ok
t/mojo/subprocess.t ........................ ok
t/mojo/subprocess_ev.t ..................... ok
t/mojo/template.t .......................... ok
t/mojo/tls.t ............................... ok
t/mojo/transactor.t ........................ ok
t/mojo/url.t ............................... ok
t/mojo/user_agent.t ........................ ok
t/mojo/user_agent_online.t ................. ok
t/mojo/user_agent_socks.t .................. ok
t/mojo/user_agent_tls.t .................... ok
t/mojo/user_agent_unix.t ................... ok
t/mojo/util.t .............................. ok
t/mojo/websocket.t ......................... ok
t/mojo/websocket_frames.t .................. ok
t/mojo/websocket_proxy.t ................... ok
t/mojo/websocket_proxy_tls.t ............... ok
t/mojolicious/app.t ........................ ok
t/mojolicious/charset_lite_app.t ........... ok
t/mojolicious/command.t .................... ok
t/mojolicious/commands.t ................... ok
t/mojolicious/dispatch.t ................... ok
t/mojolicious/dispatcher_lite_app.t ........ ok
t/mojolicious/embedded_app.t ............... ok
t/mojolicious/embedded_lite_app.t .......... ok
t/mojolicious/exception_lite_app.t ......... ok
t/mojolicious/external_app.t ............... ok
t/mojolicious/external_lite_app.t .......... ok
t/mojolicious/group_lite_app.t ............. ok
t/mojolicious/json_config_lite_app.t ....... ok
t/mojolicious/json_config_mode_lite_app.t .. ok
t/mojolicious/layouted_lite_app.t .......... ok
t/mojolicious/lite_app.t ................... ok
t/mojolicious/log_lite_app.t ............... ok
t/mojolicious/longpolling_lite_app.t ....... ok
t/mojolicious/multipath_lite_app.t ......... ok
t/mojolicious/ojo.t ........................ ok
t/mojolicious/ojo_signatures.t ............. ok
t/mojolicious/pattern.t .................... ok
t/mojolicious/production_app.t ............. ok
t/mojolicious/proxy_app.t .................. ok
t/mojolicious/rebased_lite_app.t ........... ok
t/mojolicious/renderer.t ................... ok
t/mojolicious/restful_lite_app.t ........... ok
t/mojolicious/routes.t ..................... ok
t/mojolicious/signatures_lite_app.t ........ ok
t/mojolicious/static_lite_app.t ............ ok
t/mojolicious/tag_helper_lite_app.t ........ ok
t/mojolicious/testing_app.t ................ ok
t/mojolicious/tls_lite_app.t ............... ok
t/mojolicious/twinkle_lite_app.t ........... ok
t/mojolicious/types.t ...................... ok
t/mojolicious/upload_lite_app.t ............ ok
t/mojolicious/upload_stream_lite_app.t ..... ok
t/mojolicious/validation_lite_app.t ........ ok
t/mojolicious/websocket_lite_app.t ......... ok
t/mojolicious/yaml_config_lite_app.t ....... ok
t/pod.t .................................... ok
t/pod_coverage.t ........................... ok
t/test/mojo.t .............................. ok
All tests successful.
Files=103, Tests=4779, 144 wallclock secs ( 2.20 usr  0.54 sys + 85.48 cusr 19.76 csys = 107.98 CPU)
Result: PASS

Grinnz · 2023-02-27T09:17:46Z

That resolves my primary concern, installation tests failing is much less of a problem than potential application behavior changes.

Grinnz

It may break tests in other instances, but as you describe, there's no other way to fix it for LibreSSL. Perltidy failure looks unrelated

jberger · 2023-03-08T19:01:36Z

Test failure for me:

t/mojo/websocket_proxy_tls.t ............... 1/? 
#   Failed test 'right content'
#   at t/mojo/websocket_proxy_tls.t line 126.
#          got: ''
#     expected: 'https://localhost:56835/proxy'

#   Failed test 'right content'
#   at t/mojo/websocket_proxy_tls.t line 139.
#          got: ''
#     expected: 'https://localhost:56835/proxy'

#   Failed test 'connection was kept alive'
#   at t/mojo/websocket_proxy_tls.t line 140.
Mojo::Reactor::Poll: I/O watcher failed: Can't locate object method "send" via package "Mojo::Transaction::HTTP" at t/mojo/websocket_proxy_tls.t line 151.

and the test hung

jberger · 2023-03-08T19:07:13Z

More info:

% openssl version
LibreSSL 3.3.6

% perl -Ilib script/mojo version 
CORE
  Perl        (v5.34.0, darwin)
  Mojolicious (9.32, Waffle)

OPTIONAL
  Cpanel::JSON::XS 4.09+   (4.29)
  EV 4.32+                 (n/a)
  IO::Socket::Socks 0.64+  (n/a)
  IO::Socket::SSL 2.009+   (2.072)
  Net::DNS::Native 0.15+   (n/a)
  Role::Tiny 2.000001+     (n/a)
  Future::AsyncAwait 0.52+ (0.59)

Thanks for testing a development release, you are awesome!

M1 Max Macbook Pro running Ventura, and I should note, my tests don't fail when running the tests in the main branch.

lgv5 · 2023-03-08T19:38:01Z

Under OpenBSD, I'm testing with

oolong$ openssl version
LibreSSL 3.7.1
oolong$ perl -Ilib script/mojo version 
CORE
  Perl        (v5.36.0, openbsd)
  Mojolicious (9.32, Waffle)

OPTIONAL
  Cpanel::JSON::XS 4.09+   (4.35)
  EV 4.32+                 (4.33)
  IO::Socket::Socks 0.64+  (0.74)
  IO::Socket::SSL 2.009+   (2.081)
  Net::DNS::Native 0.15+   (n/a)
  Role::Tiny 2.000001+     (2.002004)
  Future::AsyncAwait 0.52+ (0.64)

Thanks for testing a development release, you are awesome!

This all started when I was trying to update the Mojo port in OpenBSD, which currently sits at 8.22. In the process, I started porting some deps (Future, Future::AsyncAwait, Test::Refcount and XS::Parse::Sublike), so getting my same test setup in OpenBSD should prove a bit difficult. I would see if I can manage to test without EV or IO::Socket::Socks as they seems the best suspects.

I also have a M1 lying around and will try the test suite in there.

lgv5 · 2023-03-08T19:53:20Z

$ env TEST_ALL=1 prove t/mojo/websocket_proxy_tls.t
t/mojo/websocket_proxy_tls.t .. ok
All tests successful.
Files=1, Tests=29,  1 wallclock secs ( 0.06 usr  0.02 sys +  0.73 cusr  0.17 csys =  0.98 CPU)
Result: PASS
$ openssl version
LibreSSL 3.7.1
$ perl -Ilib script/mojo version
CORE
  Perl        (v5.36.0, openbsd)
  Mojolicious (9.32, Waffle)

OPTIONAL
  Cpanel::JSON::XS 4.09+   (4.35)
  EV 4.32+                 (n/a)
  IO::Socket::Socks 0.64+  (n/a)
  IO::Socket::SSL 2.009+   (2.081)
  Net::DNS::Native 0.15+   (n/a)
  Role::Tiny 2.000001+     (n/a)
  Future::AsyncAwait 0.52+ (0.64)

Thanks for testing a development release, you are awesome!

Except for versions, that's the same combination that failed for you. I'll try tomorrow with the Mac.

lgv5 · 2023-03-09T18:47:23Z

Managed to test this on MacOS, althought on an older version of Perl than yours, @jberger . Faced the same issue; fixed it sprinkling more localhost instead of 127.0.0.1 in the proxy configuration: I only touched clients and servers originally, but no proxies. Now the issue is that t/mojo/daemon_ipv6_tls.t fails too, arguably by the same reason that t/mojo/websocket_proxy_tls.t. It seems that MacOS has some issues with localhost and solving it to the expected 127.0.0.1, at least according to several internet folks having such issues between 2012-2016.

lgv5 · 2023-03-09T19:51:46Z

So, after a bit more of experimenting, I think I nailed down the issue: I believe that what's happening is that the servers that are bound to localhost are listening only on IPv6 localhost aka ::1, while the proxies are trying to connect with 127.0.0.1. I arrived to this conclusion in the following way: in one terminal run nc -kl localhost 12345, in another run echo hi | nc -v localhost 12345, echo hi | nc -v ::1 12345 and echo hi | nc -v 127.0.0.1 12345. The last one gets a connection refused error. Assuming that is what is happening on Mojo's side too, I too believe this could be fixed on Mojo's side, by binding to all the addresses returned by a hostname instead of just the first one.

The last commit makes t/mojo/websocket_proxy_tls.t pass in MacOS, but most of the other TLS tests fail, because of a ciphers mismatch (Mojo expects AES256-GCM, LibreSSL is offering CHACHA20-POLY1305). FWIW, I get exactly the same successes and failures both with and without any patches in MacOS.

OpenBSD is still happy with the latest commit:

All tests successful.
Files=103, Tests=4779, 149 wallclock secs ( 2.29 usr  0.70 sys + 86.10 cusr 19.75 csys = 108.84 CPU)
Result: PASS

lgv5 · 2023-03-10T20:59:05Z

Squashed the commits and rebased against main now that the perltidy action should be fixed.

lgv5 · 2023-03-20T21:07:38Z

Rebased against latest main. Can this be reviewed again? @jberger do you still run into issues in Mac with it?

lgv5 · 2023-05-11T02:51:28Z

Ping.

Rebased against latest main. @jberger did you run into issues on Mac? Didn't try this rebased version there.

jberger · 2023-05-16T14:08:23Z

re-testing:

mojo % TEST_ALL=1 prove -lr t    
t/mojo/asset.t ............................. ok    
t/mojo/base.t .............................. ok   
t/mojo/bytestream.t ........................ ok    
t/mojo/cache.t ............................. ok   
t/mojo/cgi.t ............................... ok    
t/mojo/collection.t ........................ ok    
t/mojo/content.t ........................... ok   
t/mojo/cookie.t ............................ ok    
t/mojo/cookiejar.t ......................... ok    
t/mojo/daemon.t ............................ ok    
t/mojo/daemon_ipv6_tls.t ................... 1/? 
    #   Failed test 'right status'
    #   at t/mojo/daemon_ipv6_tls.t line 54.
    #          got: undef
    #     expected: '200'

    #   Failed test 'right content'
    #   at t/mojo/daemon_ipv6_tls.t line 55.
    #          got: ''
    #     expected: 'works!'

    #   Failed test 'no error'
    #   at t/mojo/daemon_ipv6_tls.t line 56.

    #   Failed test 'right status'
    #   at t/mojo/daemon_ipv6_tls.t line 58.
    #          got: undef
    #     expected: '200'

    #   Failed test 'right content'
    #   at t/mojo/daemon_ipv6_tls.t line 59.
    #          got: ''
    #     expected: 'works!'

    #   Failed test 'no error'
    #   at t/mojo/daemon_ipv6_tls.t line 60.

    #   Failed test 'right error'
    #   at t/mojo/daemon_ipv6_tls.t line 62.
    #                   'SSL connect attempt failed error:0A000126:SSL routines::unexpected eof while reading
    # '
    #     doesn't match '(?^u:name)'
    # Looks like you failed 7 tests of 7.

#   Failed test 'IPv6, TLS, SNI and a proxy'
#   at t/mojo/daemon_ipv6_tls.t line 63.
# Looks like you failed 1 test of 3.
t/mojo/daemon_ipv6_tls.t ................... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/3 subtests 
t/mojo/date.t .............................. ok    
t/mojo/dom.t ............................... ok     
t/mojo/dynamic_methods.t ................... ok   
t/mojo/eventemitter.t ...................... ok    
t/mojo/exception.t ......................... ok    
t/mojo/file.t .............................. ok    
t/mojo/headers.t ........................... ok    
t/mojo/home.t .............................. ok   
t/mojo/hypnotoad.t ......................... ok   
t/mojo/ioloop.t ............................ ok    
t/mojo/ioloop_ipv6.t ....................... ok   
t/mojo/ioloop_tls.t ........................ 1/? Mojo::Reactor::Poll: I/O watcher failed: Can't call method "write" on an undefined value at t/mojo/ioloop_tls.t line 99.

and at that point the test hung

jberger · 2023-05-16T14:19:06Z

But actually now I'm getting the same test failures both in the PR branch and main

lgv5 · 2023-12-30T17:07:13Z

Hi! I rebased this patch and added a new commit that deals with TLSv1.0 and TLSv1.1 being disabled in LibreSSL since 3.8.1. Let me know if I should keep this split in two commits or if I should squash them together. All the tests pass. Without the second commit and with LibreSSL 3.8.1 or newer, the last 2 tests from t/mojo/user_agent_tls.t fail because of the version= parameter.

Since beginning of 2022, LibreSSL dropped support for IP addresses in SNI. This is aligned to RFC 6066, section 3: > Literal IPv4 and IPv6 addresses are not permitted in "HostName". In order to deal with it, clients connect to "localhost" instead of "127.0.0.1" and adjust "server.crt" to use "localhost" as Common Name instead of "127.0.0.1". Adjust TLS version for LibreSSL deprecation of TLSv1.1 and older.

kraih · 2024-02-09T13:14:34Z

We probably should fix all TLS tests to work with github actions again first.

lgv5 force-pushed the main branch from 22705da to 4689b47 Compare March 2, 2023 21:06

kraih requested review from a team, marcusramberg, jhthorsen and christopherraa March 8, 2023 18:48

Grinnz previously approved these changes Mar 8, 2023

View reviewed changes

lgv5 dismissed Grinnz’s stale review via dcd4fd5 March 9, 2023 19:33

lgv5 force-pushed the main branch from dcd4fd5 to c30f33a Compare March 10, 2023 20:57

lgv5 force-pushed the main branch from c30f33a to ba3d977 Compare March 20, 2023 21:04

lgv5 force-pushed the main branch from ba3d977 to 36e1551 Compare May 11, 2023 02:47

lgv5 force-pushed the main branch from 36e1551 to 6f31727 Compare December 30, 2023 16:56

lgv5 force-pushed the main branch from 6f31727 to 044e4d5 Compare December 30, 2023 23:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix SNI tests for LibreSSL #2041

Fix SNI tests for LibreSSL #2041

lgv5 commented Feb 25, 2023

Grinnz commented Feb 25, 2023

jberger commented Feb 25, 2023

kraih commented Feb 25, 2023

Grinnz commented Feb 25, 2023

lgv5 commented Feb 25, 2023

kraih commented Feb 26, 2023

lgv5 commented Feb 27, 2023

Grinnz commented Feb 27, 2023

Grinnz left a comment

jberger commented Mar 8, 2023

jberger commented Mar 8, 2023 •

edited

lgv5 commented Mar 8, 2023

lgv5 commented Mar 8, 2023

lgv5 commented Mar 9, 2023

lgv5 commented Mar 9, 2023 •

edited

lgv5 commented Mar 10, 2023

lgv5 commented Mar 20, 2023

lgv5 commented May 11, 2023

jberger commented May 16, 2023

jberger commented May 16, 2023

lgv5 commented Dec 30, 2023

kraih commented Feb 9, 2024

Fix SNI tests for LibreSSL #2041

Are you sure you want to change the base?

Fix SNI tests for LibreSSL #2041

Conversation

lgv5 commented Feb 25, 2023

Summary

Motivation

References

Grinnz commented Feb 25, 2023

jberger commented Feb 25, 2023

kraih commented Feb 25, 2023

Grinnz commented Feb 25, 2023

lgv5 commented Feb 25, 2023

kraih commented Feb 26, 2023

lgv5 commented Feb 27, 2023

Grinnz commented Feb 27, 2023

Grinnz left a comment

Choose a reason for hiding this comment

jberger commented Mar 8, 2023

jberger commented Mar 8, 2023 • edited

lgv5 commented Mar 8, 2023

lgv5 commented Mar 8, 2023

lgv5 commented Mar 9, 2023

lgv5 commented Mar 9, 2023 • edited

lgv5 commented Mar 10, 2023

lgv5 commented Mar 20, 2023

lgv5 commented May 11, 2023

jberger commented May 16, 2023

jberger commented May 16, 2023

lgv5 commented Dec 30, 2023

kraih commented Feb 9, 2024

jberger commented Mar 8, 2023 •

edited

lgv5 commented Mar 9, 2023 •

edited