Is there serialization coming for data structures imported using "ingredients"?

[MontyTHall]

Looks like importing is a bit of a hassle and I'm pointed to instruction to create an "ingredients" function to import code. The problem is that I use this library to generate data. I simply am unable to serialize or deserialize. Are there plans to rectify this? I've tried Michael Heatherly's PlutoSerialize but for some reason it too doesn't like serializing data structures that are loaded via "ingredients".

Sample pluto code:

SOMELIB = ingredients("blah.jl")
someData = SOMELIB.someFunction("directory to process")

import FileIO
save("serialize_file.jld2", Dict("CACHE"=>someData))
load("serialize_file.jdl2", "CACHE") <- no dice constantly complains. In fact, it's just faster regenerating the data.

Or is there another way to serialize "someData" that was loaded by "ingredient" or has "ingredients been replaced"?

[disberd]

Have you looked at using the typemap kwarg to the load function?
https://juliaio.github.io/JLD2.jl/dev/advanced/#Upgrading-old-structures-on-load

I stumbled upon this problem (though not with ingredients) and using typemap solved the problem for me

[MontyTHall]

Excellent! That page almost works. Do you know how to type map parameterized types? I do the following in pluto.

UC = ingredients("blah.jl")  ( contains Data{M} )

version 1 - fail - parametric
JLD2.rconvert(::Type{UC.Data{M}, nt::NamedTuple) where M = Data{M}(nt.....)
typeMap = Dict(
   "Main.UC.Data" => JLD2.Upgrade(UC.Data)
)

version 2 - fail - specific
JLD2.rconvert(::Type{UC.Data{UC.SomeOtherType}, nt::NamedTuple) = Data{UC.SomeOtherType}(nt.....)
typeMap = Dict(
   "Main.UC.Data{Main.UC.SomeOtherType}" => JLD2.Upgrade(UC.Data{UC.SomeOtherType})
)

[disberd]

I actually did not use the rconvert option but directly the jldopen one. I Think I linked the wrong sub-heading in the first comment.
I used code more similar to the one here https://juliaio.github.io/JLD2.jl/dev/advanced/#Explicit-Type-Remapping

So in your case I get you could try something like

UC = ingredients("blah.jl")  ( contains Data{M} )
f = jldopen("example.jld2","r"; typemap=Dict(
    "Main.UC.Data" => UC.Data,
    "Main.UC.SomeOtherType" => UC.SomeOtherType,
))
f["variable"] # This would hopefully work

[MontyTHall]

Thank you so much!!!! That did it! Apparently I made it harder than I needed! I wait 30 minutes every time my notebook opens up. Now can serialize - that should speed things up!