Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Saving a struct with an anonymous function type in a type parameter is highly non future-proof #401

Open
marius311 opened this issue Jun 18, 2022 · 1 comment

Comments

@marius311
Copy link
Contributor

marius311 commented Jun 18, 2022

I've got a struct which has an anonymous function as a field, with the type in the struct type parameters. I don't need to serialize the function, I can reconstruct it at load-time, but since its anonymous it might have a new "name" upon reconstructing if the order of anonymous functions defined in the running session / containing module ever changes. This seems to cause JLD2 to error since the type parameters may not be exactly the same when reconstructing. Something like this:

using JLD2

# define struct and custom serialization
struct FunWrap{F}
    f :: F
    function FunWrap()
        f = () -> 1
        new{typeof(f)}(f)
    end
end
JLD2.writeas(::Type{<:FunWrap}) = Tuple{}
JLD2.wconvert(::Type{Tuple{}}, f::FunWrap) = ()
JLD2.rconvert(::Type{<:FunWrap}, ::Tuple{}) = FunWrap()


f = FunWrap()
@save "test.jld2" f
# redefine to create a new name for the anonymous function. this
# simulates making some (even completely unrelated) change to the
# contianing module
struct FunWrap{F} 
    f :: F
    function FunWrap()
        f = () -> 1
        new{typeof(f)}(f)
    end
end
@load "test.jld2" f
# ERROR: TypeError: in typeassert, expected FunWrap{var"#7#8"}, got a value of type FunWrap{var"#11#12"}

Is there any way to get this to work? Doesn't seem like the custom serialization helps here, and neither would typemap.

I should mention the real use-case is saving an Interpolations.jl object, which uses an anonymous function. Its not hard to make a custom serialization which just saves the data and tries to recreate the Interpolation object at load-time, but loading can then fail due to the issue above. Thanks for any help.

@JonasIsensee
Copy link
Collaborator

Thank you for this detailed report.

This is indeed a rather difficult question.
Implementation and (future-proof) representation within files are one thing.
However, even at a conceptual level I'm not sure there is a meaningful solution.

I'm happy to discuss, though.
See e.g. #316 #338 #377 as proof that I care about this and that I've struggled ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants