Specify pathing into a CID vs to a CID

[RangerMauve]

Right now when you have a structure like {foo: Link} Pathing into it with /foo could either mean "get the value at foo (the link)" or "get the data pointed at by foo".

I think we should standardize around using /foo to mean "get the link" and /foo/ to mean "traverse into the link".

This applies to IPLD Patch, to IPLD URLs, and to traversal in general.

One question this raises is what to do when foo isn't a Link and someone tries to do /foo/.
Another question is how this would look when translated into path segments. Would it result in a path segment that's an empty string?

cc @rvagg on thoughts. Might be useful to formalize somewhere and make sure the JS and Go (and Rust) implementations are aligned.

[BigLep]

2022-11-22 triage conversation:

1. agreed this is a good way to solve this but it has implications for how we do pathing historically.
2. find a place to put this into ipld/ipld doc or spec and ping folks like lidel on impliciations
3. we could special-case unixfs pathing vs. ipld pathing

[BigLep]

2022-11-29 triage conversation:

1. ensure that https://github.com/ipld/ipld/blob/master/docs/data-model/pathing.md is updated with a link to any spec.
2. @RangerMauve will write up a doc on what a spec would look like

[rvagg]

@alanshaw raised a different problem with paths and links that we should roll up into a pathing spec:

I have a fun DAG linking question. In the case that a block is a CID, should I have any expectations of being able to path through it? If so, what are they? For example, if I have node => CID node => node:

$ echo '{ "bar": "Hello World!" }' | ipfs dag put
bafyreihlhgo2srya7ihdyc5ae76rki47irecf4thjfjhsmrjyeurlyhbua

$ echo '{ "/": "bafyreihlhgo2srya7ihdyc5ae76rki47irecf4thjfjhsmrjyeurlyhbua" }' | ipfs dag put
bafyreiexxrljy6el4v7phukkux3ukhzuiuajsilzd63576oysy3vhll3ia

$ echo '{ "foo": { "/": "bafyreiexxrljy6el4v7phukkux3ukhzuiuajsilzd63576oysy3vhll3ia" } }' | ipfs dag put
bafyreidb7fynlo5246wst4tyl3wucfoskujuuq4523srgc73c6xlvnah2i

How do I path to "Hello World!" from "bafyreidb7fynlo5246wst4tyl3wucfoskujuuq4523srgc73c6xlvnah2i"?

$ ipfs dag get bafyreidb7fynlo5246wst4tyl3wucfoskujuuq4523srgc73c6xlvnah2i/foo
{"/":"bafyreihlhgo2srya7ihdyc5ae76rki47irecf4thjfjhsmrjyeurlyhbua"}

$ ipfs dag get bafyreidb7fynlo5246wst4tyl3wucfoskujuuq4523srgc73c6xlvnah2i/foo/bar
Error: func called on wrong kind: "LookupBySegment" called on a link node (kind: link), but only makes sense on map or list

$ ipfs dag get bafyreidb7fynlo5246wst4tyl3wucfoskujuuq4523srgc73c6xlvnah2i/foo//bar
Error: func called on wrong kind: "LookupBySegment" called on a link node (kind: link), but only makes sense on map or list

---

I think this is a case of not having a path "spec" meaning we have edge cases that are neither specified nor implemented for. We also have behaviours different between JS and Go around these edges (and likely Rust too soon). So a spec should clear it up.

Backward compatibility will be a challenge though, but maybe we solve that by saying there is "unixfs pathing" and "ipld pathing" which have slightly different semantics.

[RangerMauve]

I still kinda like how bafyreidb7fynlo5246wst4tyl3wucfoskujuuq4523srgc73c6xlvnah2i/foo//bar looks. We could add a thing where if you're looking up by segment on a Link, and the name is empty, resolve the link to whatever it's pointing at.

[rvagg]

@aschmahmann wondering if you have any input on this? We're considering working this into a spec change but you're likely to have opinions that would be useful input.

[aschmahmann]

Thanks @rvagg for the tag. My main 2c here is that this seems clever, but that while clever can be good it can also end up as a source of confusion for people.

For the sake of comparison, given that for IPLD paths we need some "escaping" to deal with things like ADLs/lenses, (byte) ranges, unfriendly characters, etc. what's the advantage of having bafyfoo/bar/ being follow-the-link and bafyfoo/bar being give-me-the-link rather than something louder like bafyfoo/bar being follow-the-link and bafyfoo;link=bar?

To some extent this is stylistic would we prefer elegant but perhaps confusing vs a more brute approach? Perhaps with the right error messages and UX in tooling elegant is ok, but sometimes the brute way is the way to go.

[aschmahmann]

I'm not sure what to think about blocks that are only links. I could see defining // as ok here, even though it'll probably end up as a source of some confusion. However, it'd probably be helpful to understand the usage patterns behind such blocks so we know how important following them is (e.g. you could also just allow bafyfoo to autoresolve through all of the CID-only-blocks if you wanted) vs how niche they are.

The only uses I can think of off the top of my head are:

1. I don't like that you used SHA2-512 because that's too many bytes, so I'll use SHA2-256 on top of that CID to shrink it
2. There's some existing scheme out there that uses say Blake2-256(Blake2-256(Data)) and I'd rather define a custom IPLD codec (e.g. the single-blake2-256-hash codec) than a custom multihash (e.g. a double-blake2-256 multihash) for working with the data.

[BigLep]

2023-01-24 maintainer conversation:
Ideal action here would be specs added to ipld/ipld around pathing. No one is able to take this on currently. The best proxy we'll get is the pathing discussion in ipfs/specs#293
There isn't more action outside of ipfs/specs#293 that we plan to take for the foreseeable future.