Add macro destructuring

[Technologicat]

Considering things like unpythonic.syntax - especially the Turing tarpit that is letdoutil - it would be nice to be able to take an AST node, conveniently detect whether it is a macro invocation, and then look at (and possibly edit) its tree and args.

The principle of data abstraction says this is clearly a job for the macro expander, which defines the actual AST layouts that correspond to macro invocations.

For expr and name macros this is easy. The problem is block and decorator macros. The basic case is easy:

with block1[arg0, ...]:
    with block2[brg0, ...]:
        ...

Looking at the outer with, obviously args = [arg0, ...] and tree is the inner with statement (as a single-item list, because a block macro always receives a list of nodes). If they are invoked in the same with:

with block1[arg0, ...], block2[brg0, ...]:
    ...

it is still easy to say that for block1, we have args = [arg0, ...]. But what is tree?

The macro block2 hasn't been invoked yet, so what will happen when this expands, is that the expander will edit the AST in-place so that we'll have:

with block2[brg0, ...]:
    ...

which is the actual tree passed to the macro block1. Obviously, we shouldn't edit the AST if we're just inspecting; but if possible, we should allow the user to edit the AST, applying changes to the original.

One possibility is to make a separate full-module preprocessing pass to convert several macro invocations in the same with to nested with statements, which would unify the handling, allowing us to say the syntaxes are strictly equivalent. It'll cost some performance, and we have to be very careful, in case the context manager list mixes macros and run-time context managers (which, as of 3.0.0, is perfectly legal, by design).

Decorator macros need something similar. There it's always the same decorated node that gets passed in, even if several decorator macros are used. They may also be mixed with run-time decorators in the same decorator_list. One possibility here is to make a shallow copy of the decorated node, and modify its decorator list to pop the innermost macro, like the expander would do when the node expands. This way, inspecting the result again destructures the next decorator macro invocation, and so on. But this solution makes it difficult to map any edits done on the copy to the original AST.

So this needs some thought, hence the ∞ milestone for now.

[Technologicat]

As of unpythonic v0.15.0, there is an experiment with a very rudimentary macro invocation destructuring system in unpythonic.syntax.nameutil. This functionality was needed by some of the macros, and being experimental for now, the logical place for it was within unpythonic itself.

We should probably extend that and move it into mcpyrate.