Keep track of what the properties were when a snapshot was taken

[rincebrain]

Describe the feature would like to see added to OpenZFS

It's sometimes useful to know, for example, that recordsize was 1M at a particular point in the past, so you know that data older than XYZ probably doesn't have large records yet, or if you changed the checksum from fletcher4 to sha256, then again to skein or blake3, when you might need to look for older files that you might want to rewrite for (dedup|nopwrite|not bottlenecking scrub on your crappy CPU that no longer has SHA-NI) purposes.

Right now, I don't think ZFS knows this directly - you might get some of it in zpool history if it hasn't run over, and this isn't on the other side of a send|recv, but I don't think any object keeping the state of properties hangs around, since it's more or less just a verbatim copy of "then".

(This would also be useful if we made it correctly persist indirect properties - e.g. it would be useful to know that the recordsize for new objects effectively changed, even if the parent or grandparent or ... was where it changed and was inherited from. So it's "slightly" more work than just "also persist that object", if we want it to reflect reality well.)

How will this feature improve OpenZFS?

I've got a bunch of datasets that, over the years, I've changed settings on, and would eventually like to go rewrite the instances of the old settings since I decided the space/speed/etc tradeoffs differed. (Consider if you wanted to reduce the overhead of redundant_metadata after that was introduced.) But right now, I would need to go use some unupstreamed changes I have to zdb that let me count the number of objects in different states on the pool to figure out whether a given dataset has any "old" setting things in it, and that's really cumbersome, not to mention unworkable for people who aren't comfortable reaching into ZFS for fun.

Additional context

I don't think this introduces any new security implications? I mean, if it preserved keylocation and that was a security hole to know, then maybe, but if knowing that path is a security hole, we already have bigger issues...