Releases: libp2p/go-libp2p
v0.32.0
🔦 Highlights
WebRTC Direct
This release adds support for WebRTC Direct. WebRTC Direct allows browser nodes to connect to go-libp2p directly, without any configuration (e.g. TLS certificates) needed on the go-libp2p side. This is useful for browser nodes that aren’t able to use Webtransport. Note that WebRTC Direct cannot be used to connect a browser node to a go-libp2p node behind a NAT / firewall. This requires using WebRTC, which is currently being worked on #2009.
Happy Eyeballs for TCP
In the last couple of releases we’ve shipped Smart Dialing and Black hole Detection. This release continues our effort to improve the dial prioritisation logic to avoid spurious dials with the introduction of Happy Eyeballs for TCP addresses. As recommended by RFC 8305, when dialing a peer’s TCP addresses, we now dial the IPv6 address first and only dial the peer’s IPv4 address if we have not established the IPv6 TCP connection within 250ms.
Misc
host.NewStreamnow waits for a hole punched connection to be available rather than returningnetwork.ErrTransientConnimmediately.- go-libp2p now works on riscv64 architectures.
Changelog
- chore: update examples to v0.31 by @p-shahi in #2544
- libp2phttp: don't initialise ServeMux if not nil by @sukunrt in #2548
- basichost: handle the SetProtocol error in NewStream by @marten-seemann in #2555
- swarm: don't dial unspecified addresses by @sukunrt in #2560
- examples: stop using deprecated peer.ID.Pretty by @Icarus9913 in #2563
- upgrader: drop support for multistream simultaneous open by @marten-seemann in #2557
- WebRTC Direct transport implementation by @marten-seemann in #2337
- mock: use go.uber.org/mock by @sukunrt in #2540
- core/peer: remove deprecated Encode function by @marten-seemann in #2566
- core/peer: remove deprecated ID.Pretty method by @marten-seemann in #2565
- websocket: don't resolve /dnsaddr addresses by @marten-seemann in #2571
- swarm: fix DialPeer behaviour for transient connections by @sukunrt in #2547
- webrtc: fail Write early if deadline has exceeded before the call by @sukunrt in #2578
- webrtc: put buffer back to pool by @sukunrt in #2574
- webrtc: fix deadlock on connection close by @sukunrt in #2580
- update gomock to v0.3.0 by @marten-seemann in #2581
- config: warn if connmgr limits conflict with rcmgr by @piersy in #2527
- quicreuse: remove QUIC metrics tracer by @marten-seemann in #2582
- interop: fix redis env var by @sukunrt in #2585
- libp2phttp: don't strip
/suffix when mounting handler by @sukunrt in #2552 - Fix typos by @vuittont60 in #2600
- swarm: add loopback to low timeout filter by @dennis-tra in #2595
- webrtc: correctly report incoming packet address on muxed connection by @sukunrt in #2586
- Spell Check: Fix typos by @tkzktk in #2604
- swarm: wait for transient connections to upgrade for NewStream by @sukunrt in #2542
- Fix missing deprecation tag by @librick in #2605
- tcp: fix build on riscv64 by @marten-seemann in #2590
- webrtc: fix race in TestMuxedConnection by @sukunrt in #2607
- swarm: use happy eyeballs ranking for TCP dials by @sukunrt in #2573
- circuitv2: don't check ASN for private addrs by @sukunrt in #2611
- swarm: cleanup stream handler goroutine by @sukunrt in #2610
- swarm: use typed atomics by @marten-seemann in #2612
- test/basichost: fix flaky test due to rcmgr by @marten-seemann in #2613
- update quic-go to v0.39.1, set a static resumption token generator key by @marten-seemann in #2572
- ci: migrate to renamed interop test action by @thomaseizinger in #2617
- swarm: fix recursive resolving of DNS multiaddrs by @marten-seemann in #2564
- webrtc: fix race in TestRemoveConnByUfrag by @sukunrt in #2620
- identify: don't filter dns addresses based on remote addr type by @sukunrt in #2553
- chore: update dependencies for v0.32 release by @sukunrt in #2621
- chore: update js-libp2p examples repo by @sukunrt in #2624
New Contributors
- @Icarus9913 made their first contribution in #2563
- @piersy made their first contribution in #2527
- @vuittont60 made their first contribution in #2600
- @tkzktk made their first contribution in #2604
- @librick made their first contribution in #2605
Full Changelog: v0.31.0...v0.32.0
Contributors
Assets 2
v0.31.0
🔦 Highlights
libp2p+HTTP
We're introducing a new experimental API in go-libp2p, enabling developers to utilize libp2p with the well-known semantics of HTTP. This isn't a special flavor of HTTP; it's standard HTTP, but enhanced with libp2p. Developers can now benefit from HTTP intermediaries such as CDN caching and layer 7 load balancing. This allows developers to create HTTP applications that operate over NATs and seamlessly tap into libp2p's diverse transport options to boost connectivity. In addition, the HTTP transport now joins the roster of supported transports in libp2p.
The new API is under libp2phttp.Host in the "github.com/libp2p/go-libp2p/p2p/http" package. This is like the stream transport host (host.Host in "github.com/libp2p/go-libp2p/core/host"), but exposes HTTP request/response rather than a stream API. Refer to the godoc for details on the API and examples.
QUIC Generic Segmentation Offload
This release updates quic-go to v0.38.x, which enables GSO (Generic Segmentation Offload) in the send path, drastically increasing the packet send rate. Without GSO, quic-go had to use a single sendmsg syscall for every UDP datagram sent. GSO allows us to pass one giant (up to 64k) datagram to the sendmsg syscall, and have the kernel chop it into MTU sized (~1300 bytes) datagrams before sending them out on the wire. For more details on syscall optimizations, CloudFlare published an excellent blog post about this a while ago. GSO is currently only available on Linux (and with kernels >4.18).
Changelog
- chore: be more descriptive about where public dashboards come from by @p-shahi in #2508
- metrics: update dashboard names from libp2p to go-libp2p by @sukunrt in #2512
- examples: update go-libp2p to v0.30.0 by @marten-seemann in #2507
- quic: don't claim to be able to dial draft-29 in CanDial by @marten-seemann in #2520
- update quic-go to v0.38.1 by @marten-seemann in #2506
- feat: add ErrPeerIDMismatch error type to replace ad-hoc errors by @aschmahmann in #2451
- swarm: return errors on filtered addresses when dialing by @sukunrt in #2461
- swarm: fix Unwrap for DialError, implement Unwrap for TransportError by @sukunrt in #2437
- swarm: return a more meaningful error when dialing QUIC draft-29 by @marten-seemann in #2524
- ci: limit interop tests runs to core folders by @p-shahi in #2513
- identify: disable racy test when running with race detector by @marten-seemann in #2526
- host: disable black hole detection on autonat dialer by @sukunrt in #2529
- move libp2p/go-libp2p-gostream to p2p/net/gostream by @marten-seemann in #2535
- feat: Implement HTTP spec by @MarcoPolo in #2438
- p2p/http: cache json wellknown mappings in the .well-known handler by @Jorropo in #2537
- swarm: track dial cancellation reason by @sukunrt in #2532
- reuseport: use DialContext instead of Dial in test to fail quickly by @sukunrt in #2541
- dashboards: improve naming for black hole panel by @sukunrt in #2539
Full Changelog: v0.30.0...v0.31.0
Contributors
Assets 2
v0.30.0
This is the first release that is compatible with Go 1.21. It drops compatibility with (the now outdated) Go 1.19.
Deprecation of the database-backed peerstore
The database-backed peerstore, pstoreds, is now deprecated. It will be removed in a future release of go-libp2p.
The main reason for a database-backed peerstore was that it easily allowed persisting peers across reboots of a go-libp2p node. However, this comes with the problem that entries for these peers will never be pruned. It also means significantly higher latencies compared to the in-memory peerstore. A better way of persisting (a subset of) peers for bootstrapping purposes is to regularly query the in-memory peerstore and to explicitly persist those entries.
See #2329 for more motivation and discussion.
Removal of mplex
The ecosystem is in the process of removing support for one of our stream multiplexers, mplex. The only supported stream multiplexer now is yamux. Kubo (a go-libp2p user) removed support for mplex recently. See libp2p/specs#553 for more details.
Note that due to go-libp2p's modular design, it's still possible to use mplex (or any other custom multiplexer) using the libp2p.Muxer configuration option.
Removal of QUIC draft-29
QUIC draft-29 was a somewhat widely deployed draft version of QUIC before publication of RFC 9000. The multiaddr codepoint used /quic (RFC 9000 QUIC uses /quic-v1). go-libp2p has been using /quic-v1 for a long time (since v0.24.0), and is now dropping support for draft-29.
Full Changelog
- github.com/libp2p/go-libp2p:
- update dependencies ahead of the v0.30 release (#2504) (libp2p/go-libp2p#2504)
- ci: copy new Unified CI templates and bump go.mod to Go 1.20 (#2471) (libp2p/go-libp2p#2471)
- transport tests: add deadline tests (#2286) (libp2p/go-libp2p#2286)
- chore: remove unused and outdated package-list.json (#2499) (libp2p/go-libp2p#2499)
- muxer: remove support for mplex (#2498) (libp2p/go-libp2p#2498)
- transport tests: refactor workers in TestMoreStreamsThanOurLimits (#2472) (libp2p/go-libp2p#2472)
- use standard library sha256 implementation for Go 1.21 (#2309) (libp2p/go-libp2p#2309)
- quic: update quic-go to v0.37.5 (#2497) (libp2p/go-libp2p#2497)
- cleanup: add continue in case of failure in the (*BasicHost).Addrs certhash loop (#2492) (libp2p/go-libp2p#2492)
- tests: add a CertHashes testcase in TestInferWebtransportAddrsFromQuic (#2495) (libp2p/go-libp2p#2495)
- basichost: use byte representation of WebTransport multiaddr as map key (#2494) (libp2p/go-libp2p#2494)
- webtransport: check for UDP multiaddr component in address matcher (#2491) (libp2p/go-libp2p#2491)
- swarm: remove unnecessary reqno for pending request tracking (#2460) (libp2p/go-libp2p#2460)
- quic: drop support for QUIC draft-29 (#2487) (libp2p/go-libp2p#2487)
- metrics: add links to public dashboards (#2486) (libp2p/go-libp2p#2486)
- swarm: remove leftover TODO (#2474) (libp2p/go-libp2p#2474)
- peerstore: deprecate the database-backed peerstore (#2475) (libp2p/go-libp2p#2475)
- identify: fix sorting of observed addresses (#2476) (libp2p/go-libp2p#2476)
- update go-multiaddr to v0.11.0 (#2467) (libp2p/go-libp2p#2467)
- chore: update golang-lru to v2.0.4, fixing semver violation (#2448) (libp2p/go-libp2p#2448)
- swarm: don't open new streams over transient connections (#2450) (libp2p/go-libp2p#2450)
- update quic-go to v0.36.3, fixing RSA key size vulnerability (#2455) (libp2p/go-libp2p#2455)
- core/crypto: restrict RSA keys to <= 8192 bits (#2454) (libp2p/go-libp2p#2454)
- chore: add notable project requirement (#2453) (libp2p/go-libp2p#2453)
- examples: update go-libp2p to v0.29.0 (#2432) (libp2p/go-libp2p#2432)
- examples: fix description of command line flags for pubsub (#2400) (libp2p/go-libp2p#2400)
- basichost: remove invalid comment (#2435) (libp2p/go-libp2p#2435)
- github.com/libp2p/go-reuseport (v0.3.0 -> v0.4.0):
- release v0.4.0 (#111) (libp2p/go-reuseport#111)
- use SO_REUSEPORT_LB on FreeBSD (#106) (libp2p/go-reuseport#106)
- chore: bump go.mod to Go 1.20 and run go fix (#109) (libp2p/go-reuseport#109)
- github.com/multiformats/go-multiaddr (v0.10.1 -> v0.11.0):
- release v0.11.0 (#214) (multiformats/go-multiaddr#214)
- update sha256-simd to v1.0.1 (#212) (multiformats/go-multiaddr#212)
- update golang.org/x/exp slice comparison to match standard library version (#210) (multiformats/go-multiaddr#210)
Contributors
| Contributor | Commits | Lines ± | Files Changed |
|---|---|---|---|
| Marco Munizaga | 5 | +405/-561 | 36 |
| Marten Seemann | 12 | +302/-477 | 92 |
| Sukun | 4 | +125/-52 | 6 |
| Jorropo | 5 | +83/-28 | 13 |
| Prem Chaitanya Prathi | 1 | +28/-1 | 2 |
| GitHub | 2 | +6/-19 | 2 |
| Prithvi Shahi | 1 | +10/-12 | 1 |
| Christian Stewart | 1 | +7/-2 | 2 |
| cce | 1 | +4/-4 | 3 |
| downIoads | 1 | +2/-2 | 1 |
New Contributors
- @downIoads made their first contribution in #2400
Full Changelog: v0.29.0...v0.30.0
Contributors
Assets 2
v0.29.2
v0.28.3
v0.27.9
v0.27.8
This patch release contains backports of:
- updating the qtls dependencies (qtls is quic-go's fork of crypto/tls). The new versions now contain a backport of the Go standard library fix included in the Go 1.20.7 / 1.19.12 release for quic-go's crypto/tls fork: golang/go@2350afd
- core/crypto: restrict RSA keys to <= 8192 bits: #2454. The analogous vulnerability in go-libp2p's crypto package.
- swarm: don't open new streams over transient connections: #2450. This fixes a regression introduced in v0.26.0.
Note that in order to be protected against the DoS attack making use of large RSA keys, it's necessary to update to this patch release AND to use the updated Go compiler (1.20.7 or 1.19.12, respectively).
Full Changelog: v0.27.7...v0.27.8
v0.29.1
This patch release contains backports of:
- update quic-go to v0.36.3, fixing RSA key size vulnerability: #2455. This in turn is a backport of the Go standard library fix included in the Go 1.20.7 / 1.19.12 release for quic-go's crypto/tls fork: golang/go@2350afd
- core/crypto: restrict RSA keys to <= 8192 bits: #2454. The analogous vulnerability in go-libp2p's crypto package.
- swarm: don't open new streams over transient connections: #2450. This fixes a regression introduced in v0.26.0.
Note that in order to be protected against the DoS attack making use of large RSA keys, it's necessary to update to this patch release AND to use the updated Go compiler (1.20.7 or 1.19.12, respectively).
Full Changelog: v0.29.0...v0.29.1
v0.28.2
This patch release contains backports of:
- updating the qtls dependencies (qtls is quic-go's fork of crypto/tls). The new versions now contain a backport of the Go standard library fix included in the Go 1.20.7 / 1.19.12 release for quic-go's crypto/tls fork: golang/go@2350afd
- core/crypto: restrict RSA keys to <= 8192 bits: #2454. The analogous vulnerability in go-libp2p's crypto package.
- swarm: don't open new streams over transient connections: #2450. This fixes a regression introduced in v0.26.0.
Note that in order to be protected against the DoS attack making use of large RSA keys, it's necessary to update to this patch release AND to use the updated Go compiler (1.20.7 or 1.19.12, respectively).
Full Changelog: v0.28.1...v0.28.2
v0.29.0
⚠️ Warning
Be warned if you run go get -u github.com/libp2p/go-libp2p the Go toolchain will incorrectly update github.com/quic-go/qtls-go1-20 to an incompatible version of v0.3.0 (at v0.* everything may change according to semver). You may have to downgrade github.com/quic-go/qtls-go1-20 to v0.2.2 manually.
It's recommended to do go get github.com/libp2p/go-libp2p instead.
🔦 Highlights
Smart Dialing
In our last release, we shipped Smart Dialing. To reiterate, it’s a clever way to reduce the number of spurious dials. Instead of dialing all addresses in parallel (which is what we did before v0.28), we now carefully rank the addresses and dial them one by one.
However, there were two areas where the logic we introduced could lead to suboptimal results:
- There are some networks that block UDP. This means that we won’t be able to dial any QUIC or WebTransport connections at all. This is problematic since our smart dialing logic dials a QUIC address first before dialing a TCP address, which would lead to a regression for these users.
- Similarly, not all ISPs support IPv6 yet. This is problematic as well, since we prefer IPv6 addresses over IPv4.
This is why smart dialing was disabled by default in v0.28.
For this release, we implemented a logic we call Black Hole Detection in this release. We now detect if UDP or IPv6 is completely blocked (blackholed) by observing the outcomes of dial attempts. When a black hole is detected, we skip QUIC / WebTransport and IPv6 addresses, respectively. This significantly reduces the number of unsuccessful dials for users in these environments. To monitor the status of black hole detection, we added two Grafana dashboards to our swarm dashboard, showing the percentage successful / failed dials on UDP and IPv6, and if that leads to our blackhole logic kicking in.
This now allows us to use RFC8305 Happy Eyballs for QUIC: When we dial a peer that has a QUIC IPv6 and IPv4 address, we dial the IPv6 address first, and only start dialing the IPv4 address if we haven’t heard back from the peer within 250ms. Only if we don’t hear back within another 250ms, we start dialing on TCP.
In a future release, we will enable a similar logic for TCP IPv6 and IPv4 addresses, however, this will require us to refactor our code a bit (see #2394 for details).
Preliminary measurements on the IPFS network show that:
- In ~90% of the cases, we end up with a connection on the first address we dial.
- Canceled connection attempts are reduced by more than 60% (compared to the v0.27 release).
Smart Dialing is now enabled by default. We don’t expect it to cause any performance regression, but if you find any problems, please open an issue. It can be disabled via a constructor option to libp2p.New:
libp2p.SwarmOpts(swarm.WithDialRanker(swarm.NoDelayDialRanker))Metrics
- It's now possible to spin up a local Grafana dashboards - with all libp2p dashboards imported and ready to go - by just running
docker-compose upin thedashboardsdirectory: #2383. - Resource Manager metrics are now enabled by default: #2370.
Changelog
- github.com/libp2p/go-libp2p:
- webtransport: reject listening on a multiaddr with a certhash (#2426) (libp2p/go-libp2p#2426)
- swarm: deprecate libp2p.DialRanker option (#2430) (libp2p/go-libp2p#2430)
- chore: update deps ahead of the v0.29 release (#2427) (libp2p/go-libp2p#2427)
- quic: Update to quic-go v0.36.2 (#2424) (libp2p/go-libp2p#2424)
- autonat: fix typo in WithSchedule option comment (#2425) (libp2p/go-libp2p#2425)
- identify: filter nat64 well-known prefix ipv6 addresses (#2392) (libp2p/go-libp2p#2392)
- update go-multiaddr to v0.10.1, use Unique function from there (#2407) (libp2p/go-libp2p#2407)
- swarm: enable smart dialing by default (#2420) (libp2p/go-libp2p#2420)
- transport integration tests: make TestMoreStreamsThanOurLimits less flaky (#2410) (libp2p/go-libp2p#2410)
- holepunch: skip racy TestDirectDialWorks (#2419) (libp2p/go-libp2p#2419)
- swarm: change relay dial delay to 500ms (#2421) (libp2p/go-libp2p#2421)
- identify: disable racy TestLargeIdentifyMessage with race detector (#2401) (libp2p/go-libp2p#2401)
- swarm: make black hole detection configurable (#2403) (libp2p/go-libp2p#2403)
- net/mock: support ConnectionGater in MockNet (#2297) (libp2p/go-libp2p#2297)
- docs: Add a Github workflow for checking dead links (#2406) (libp2p/go-libp2p#2406)
- rcmgr: enable metrics by default (#2389) (#2409) (libp2p/go-libp2p#2409)
- chore: remove outdated info in README and link to libp2p-implementers slack (#2405) (libp2p/go-libp2p#2405)
- metrics: deduplicate code in examples (#2404) (libp2p/go-libp2p#2404)
- transport tests: remove mplex tests (#2402) (libp2p/go-libp2p#2402)
- swarm: implement Happy Eyeballs ranking (#2365) (libp2p/go-libp2p#2365)
- docs: fix some comments (#2391) (libp2p/go-libp2p#2391)
- metrics: provide separate docker-compose files for OSX and Linux (#2397) (libp2p/go-libp2p#2397)
- chore: Update .github/workflows/stale.yml [skip ci]
- identify: use zero-alloc slice sorting function (#2396) (libp2p/go-libp2p#2396)
- rcmgr: move StatsTraceReporter to rcmgr package (#2388) (libp2p/go-libp2p#2388)
- swarm: implement blackhole detection (#2320) (libp2p/go-libp2p#2320)
- basichost / blankhost: wrap errors (#2331) (libp2p/go-libp2p#2331)
- network: don't allocate in DedupAddrs (#2395) (libp2p/go-libp2p#2395)
- rcmgr: test snapshot defaults and that we keep consistent defaults (#2315) (libp2p/go-libp2p#2315)
- rcmgr: register prometheus metrics with the libp2p registerer (#2370) (libp2p/go-libp2p#2370)
- metrics: make it possible to spin up Grafana using docker-compose (#2383) (libp2p/go-libp2p#2383)
- identify: set stream deadlines for Identify and Identify Push streams (#2382) (libp2p/go-libp2p#2382)
- fix: in the swarm move Connectedness emit after releasing conns (#2373) (libp2p/go-libp2p#2373)
- metrics: add example for metrics and dashboard (#2232) (libp2p/go-libp2p#2232)
- dashboards: finish metrics effort (#2362) (libp2p/go-libp2p#2362)
- transport tests: many streams and lots of data (#2296) (libp2p/go-libp2p#2296)
- webtransport: close the challenge stream after the Noise handshake (#2305) (libp2p/go-libp2p#2305)
- test: document why InstantTimer is required (#2351) (libp2p/go-libp2p#2351)
- examples: update go-libp2p to v0.28.0 (#2366) (libp2p/go-libp2p#2366)
- rcmgr: fix link to dashboards in README (#2363) (libp2p/go-libp2p#2363)
- chore: Update .github/workflows/stale.yml [skip ci]
- docs: fix some comments errors (#2356) (libp2p/go-libp2p#2356)
- github.com/libp2p/go-yamux/v4 (v4.0.0 -> v4.0.1):
- Release v4.0.1 (libp2p/go-yamux#106)
- fix: sendWindowUpdate respects deadlines (#105) (libp2p/go-yamux#105)
- chore: Update .github/workflows/stale.yml [skip ci]
- github.com/multiformats/go-multiaddr (v0.9.0 -> v0.10.1):
- release v0.10.1 (#206) (multiformats/go-multiaddr#206)
- fix nat64 well-known prefix check (#205) (multiformats/go-multiaddr#205)
- release v0.10.0 (#204) (multiformats/go-multiaddr#204)
- add a Unique function (#203) (multiformats/go-multiaddr#203)
- manet: add function to test if address is NAT64 IPv4 converted IPv6 address (#202) (multiformats/go-multiaddr#202)
- github.com/multiformats/go-multihash (v0.2.2 -> v0.2.3):
- chore: release v0.2.3
- perf: outline...
