Extend IPLD schema to support inline types

[Gozala]

At the moment IPLD only supports inline definitions for

• maps
• lists
• links

ipld/specs/schemas/schema-schema.ipldsch.json

Lines 575 to 590 in d8bac57

	"InlineDefn": {
	"union": {
	"members": [
	"TypeDefnMap",
	"TypeDefnList",
	"TypeDefnLink"
	],
	"representation": {
	"keyed": {
	"map": "TypeDefnMap",
	"list": "TypeDefnList",
	"link": "TypeDefnLink"
	}
	}
	}
	},

Which means you can not inline

• variants a.k.a unions and enums
• records a.k.a structs

Which seems like a good idea when schemas are authored by hand (because it encourages clear names and reuse), yet pretty unfortunate limitation when schemas are generated by programs.

Here is our concrete use case web3-storage/ucanto#107 we have UCAN + IPLD base RPC system, something along the lines of GraphQL. We would like a service to be able to describe it's protocol via IPLD schema, however there's no good way to name individual records and variants forcing use to either:

1. Use some other schema (or perhaps an extension of IPLD schema).
2. Enforce manual naming, which would both degrade library API and is prone to collisions (as host language won't help you there)
3. Generate names from hashes (I actually like this approach, but it imposes asynchrony in JS which is not great)

Proposal

1. Lift this limitation and allow inlining unions and structs in IPLD schema.
2. Allow CID based referencing of types

That way we could simply generate schema with inline types defs and optionally optimize it by lifting inline definitions and referring to them via CIDs. That way type names simply become petnames and only for the convenience

[Gozala]

Digging bit more into this, I see the note on why this limitation had been put in place

ipld/specs/schemas/schema-schema.ipldsch

Lines 352 to 356 in d8bac57

	## UnionMemberInlineDefn is a very similar purpose to InlineDefn,
	## but found specifically within UnionMember.
	## It only allows describing a link type (and not maps nor lists, as InlineDefn does),
	## which is a constraint applied to union membership largely to make sure
	## if there are errors in processing unions, we can make legible messages about it!

ipld/specs/schemas/schema-schema.ipldsch

Lines 560 to 562 in d8bac57

	## TypeNameOrInlineDefn also isn't used to describe members in Unions -- this is a choice
	## aimed to limit syntactical complexity (both at type definition authoring
	## time, as well as for the sake of error messaging during typechecking).

ipld/specs/schemas/schema-schema.ipldsch

Lines 579 to 580 in d8bac57

	## InlineDefn should look *exactly the same* as the top-level type declarations...
	## it's just that within an InlineDefn, we're restricted to a subset of the members.

It looks like my intuition about rational for this limitation was correct and it is mostly in place to make things (like schemas and validation errors) more readable. However I would argue that we could generate good error messages by specifying full path to the cause (in fact we do that already in the ucanto) which can be a lot more informative than messaging a (type), especially if those are generated.

That is to say I think IPLD schema could support inline types even if ipldsch syntax does not.

[RangerMauve]

Could you post some examples of your "dream DSL/DMT" for what this would look like in practice? Maybe also what the error messages could look like VS what is there right now?