Pandas series with object type get converted to strings #457
Labels
bug
extensions
issues affecting numpy/pandas/etc
has-MRE
Has a minimal reproducible example for debugging
This error is a slightly different take on #407 and #358.
My issue is that if a dataframe has a column of mixed type (e.g.
[4, 'foo']
), then it will be converted to strings on unpickling (e.g.['4', 'foo']
):I know there isn't an easy fix for this, but I'm hopeful that there's some fix, since (at least in this case) the actual dataframe data looks a lot like JSON format already! In particular, one suggestion I have is that currently just a single
dtype
is stored per column:My suggestion that I believe would solve this particular bug would be that if the
dtype
isobject
, to store a list (one for each element) instead. For example, in this case:Personally, I also think storing
values
as a dict of lists (rather than as a string) would be more robust and easier to read/interpret. I feelpd.DataFrame.to_dict()
is already pretty close to what would be required!The text was updated successfully, but these errors were encountered: