Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Publish self-announces as pkarr signed packets through the derper #2052

Open
wants to merge 66 commits into
base: main
Choose a base branch
from

Conversation

Frando
Copy link
Member

@Frando Frando commented Mar 1, 2024

Description

This build on top of #2045.

  • Adds a new frame PublishPkarr to the relay protocol
  • MagicEndpoint will send a pkarr SignedPacket to its home relay whenever it changes home relays
  • The relay server will publish this packet to a pkarr relay, if configured
  • Our relay servers would publish to our our instance of https://github.com/n0-computer/iroh-dns-server/, which would make the records available over DNS, which will give global node resolution with feat: node discovery via DNS #2045
  • The iroh node gets minimal new configurability: Users can either enable or disable the self-announces

based on #2045

Notes & open questions

Change checklist

  • Self-review.
  • Documentation updates if relevant.
  • Tests if relevant.

@Frando Frando force-pushed the feat/dns-derper-pkarr-publish branch 2 times, most recently from f5fe772 to d7e66d8 Compare March 7, 2024 23:37
@Frando Frando force-pushed the feat/dns-derper-pkarr-publish branch from d4791b5 to 67629cb Compare March 25, 2024 16:22
@Frando Frando force-pushed the feat/dns-derper-pkarr-publish branch from 2b977f2 to dd95550 Compare March 25, 2024 21:44
@Frando Frando force-pushed the feat/dns-derper-pkarr-publish branch from dd95550 to 6b48860 Compare March 26, 2024 11:23
@Frando Frando force-pushed the feat/dns-derper-pkarr-publish branch from 6b48860 to 59b2fd4 Compare March 26, 2024 12:02
@Frando Frando force-pushed the feat/dns-derper-pkarr-publish branch from 59b2fd4 to 8950c66 Compare March 26, 2024 12:03
Co-authored-by: Kasey <kasey@n0.computer>
@ramfox
Copy link
Contributor

ramfox commented Mar 28, 2024

Did some dogfooding!

used a local iroh-relay to test w/ the following config:

# iroh-relay-config.toml
pkarr_relay = "https://testdns.iroh.link/pkarr"
addr = "[::]:3340"
enable_relay = true
enable_stun = true
stun_port = 3478
hostname = "my.relay.network"

Works with two local nodes dialing by node_id!

Can also confirm that if the relay server is not "pkarr_relay" enabled, the normal relay interactions still work, there is no problem when sending a PkarrPacket from the node to the relay server.

@Frando Frando force-pushed the feat/dns-derper-pkarr-publish branch from 8950c66 to d9dc571 Compare April 2, 2024 11:27
@dignifiedquire dignifiedquire added this to the v0.14.0 milestone Apr 2, 2024
@rklaehn
Copy link
Contributor

rklaehn commented Apr 3, 2024

Took a quick look. I like that this is for generic pkarr packets and not just for the specific use case of node discovery.

Still on the fence whether this should be included with the derper relay or not. But since these are just fully self contained pkarr packets with no additional instructions I think having it as an option is nice. So currently I am slightly in favour.

How exactly would we modify this if we were to change the relay protocol to be more QUIC native? I guess that is a question for @flub when he comes back. I don't really know how that would look like and if it would be horrible or just a bit weird.

@ramfox ramfox modified the milestones: v0.14.0, v0.15.0 Apr 10, 2024
github-merge-queue bot pushed a commit that referenced this pull request Apr 15, 2024
## Description

This enables global node discovery over DNS, i.e. dialing nodes by just
their node id.

Current setup is as follows:

* When dialing a node only by its NodeId, the new `DnsDiscovery` service
is invoked. It will lookup a TXT record at (by default)
`_iroh_node.b32encodednodeid.testdns.iroh.link` over regular DNS or
DNS-over-http. Right now the Cloudflare DNS servers are configured. At
`testdns.iroh.link` we run a custom [DNS
server](https://github.com/n0-computer/iroh-dns-server/tree/main)
* Nodes publish their Derp address to this DNS server through Pkarr
signed packets. This is an intermediate step, we decided that the
publishing by default should not happen by the nodes directly but
mediated through the Derp servers. Work for the latter happens in #2052

This PR thus allows for the following:
```sh
# terminal/computer 1
$ iroh console --start
Iroh is running
Node ID: qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua
$ blob add ./myfile
...
Blob: o5uanh5s2zwn2sucy47puqidsfx2advxos7kajq3ajwitcwobhba
...

# terminal/computer 2
iroh console --start
blob get o5uanh5s2zwn2sucy47puqidsfx2advxos7kajq3ajwitcwobhba --node qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua
```


<!-- A summary of what this pull request achieves and a rough list of
changes. -->

## Notes & open questions

* Misses node configuration in the CLI for the node origin domain (right
now hardcoded to `testdns.iroh.link`). How do we want to expose this -
CLI flag? Or in the config file? I'd say the latter.

* Offload publishing to the Derpers - see #2052 

* Right now the records published via pkarr have a TTL of 30s - the
iroh-dns-server will use that TTL as-is when serving the records over
DNS. both can/should change?

* We can also *very* easily allow to lookup nodes not only by NodeId,
but by any domain name. In the `iroh-dns` crate I included an example
`resolve` that does just that. By setting a `CNAME` record you can even
use any domain and simply point to the record hosted at the
`testdns.iroh.link` server.
So if, on your custom domain, you added a record like this
```
_iroh_node.frando.n0.computer CNAME _iroh_node.qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua.iroh.link.
```
You can use this with the example to resolve to the node id and derp
addresses:
```
cargo run --example resolve -- domain frando.n0.computer
```

<!-- Any notes, remarks or open questions you have to make about the PR.
-->

## Change checklist

- [x] Self-review.
- [x] Documentation updates if relevant.
- [x] Tests if relevant.

Closes #1248

---------

Co-authored-by: Kasey <kasey@n0.computer>
Co-authored-by: Asmir Avdicevic <asmir.avdicevic64@gmail.com>
Co-authored-by: Ruediger Klaehn <rklaehn@protonmail.com>
Base automatically changed from feat/dns to main April 19, 2024 12:50
ppodolsky pushed a commit to izihawa/iroh that referenced this pull request Apr 20, 2024
## Description

This enables global node discovery over DNS, i.e. dialing nodes by just
their node id.

Current setup is as follows:

* When dialing a node only by its NodeId, the new `DnsDiscovery` service
is invoked. It will lookup a TXT record at (by default)
`_iroh_node.b32encodednodeid.testdns.iroh.link` over regular DNS or
DNS-over-http. Right now the Cloudflare DNS servers are configured. At
`testdns.iroh.link` we run a custom [DNS
server](https://github.com/n0-computer/iroh-dns-server/tree/main)
* Nodes publish their Derp address to this DNS server through Pkarr
signed packets. This is an intermediate step, we decided that the
publishing by default should not happen by the nodes directly but
mediated through the Derp servers. Work for the latter happens in n0-computer#2052

This PR thus allows for the following:
```sh
# terminal/computer 1
$ iroh console --start
Iroh is running
Node ID: qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua
$ blob add ./myfile
...
Blob: o5uanh5s2zwn2sucy47puqidsfx2advxos7kajq3ajwitcwobhba
...

# terminal/computer 2
iroh console --start
blob get o5uanh5s2zwn2sucy47puqidsfx2advxos7kajq3ajwitcwobhba --node qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua
```


<!-- A summary of what this pull request achieves and a rough list of
changes. -->

## Notes & open questions

* Misses node configuration in the CLI for the node origin domain (right
now hardcoded to `testdns.iroh.link`). How do we want to expose this -
CLI flag? Or in the config file? I'd say the latter.

* Offload publishing to the Derpers - see n0-computer#2052 

* Right now the records published via pkarr have a TTL of 30s - the
iroh-dns-server will use that TTL as-is when serving the records over
DNS. both can/should change?

* We can also *very* easily allow to lookup nodes not only by NodeId,
but by any domain name. In the `iroh-dns` crate I included an example
`resolve` that does just that. By setting a `CNAME` record you can even
use any domain and simply point to the record hosted at the
`testdns.iroh.link` server.
So if, on your custom domain, you added a record like this
```
_iroh_node.frando.n0.computer CNAME _iroh_node.qp2znfedwdij4llc5noizwfemfgba7bzxozvr4bp7hfsdmwqbpua.iroh.link.
```
You can use this with the example to resolve to the node id and derp
addresses:
```
cargo run --example resolve -- domain frando.n0.computer
```

<!-- Any notes, remarks or open questions you have to make about the PR.
-->

## Change checklist

- [x] Self-review.
- [x] Documentation updates if relevant.
- [x] Tests if relevant.

Closes n0-computer#1248

---------

Co-authored-by: Kasey <kasey@n0.computer>
Co-authored-by: Asmir Avdicevic <asmir.avdicevic64@gmail.com>
Co-authored-by: Ruediger Klaehn <rklaehn@protonmail.com>
@dignifiedquire dignifiedquire removed this from the v0.15.0 milestone Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 📋 Backlog
Development

Successfully merging this pull request may close these issues.

None yet

4 participants