I'd like to encode/decode with make_refs=False with real encoding/decoding

[MicheleMannino]

Hello,

I'd like to encode/decode data classes with make_refs=False and NOT using repr(), but real encoding/decoding:

from dataclasses import dataclass
from dataclasses import field
from typing import Any

@dataclass
class Test:
    devices: list = field(default_factory=list)

@dataclass
class AppResponse:
    model: Any
    actions: Any

d = [1, 2, 3]
response = AppResponse(model=Test(devices=d), actions=[Test(devices=Test(d))])

jsonpickle.encode(response, make_refs=False)

'{"actions": [{"devices": {"devices": [1, 2, 3], "py/object": "__main__.Test"}, "py/object": "__main__.Test"}], "model": {"devices": "[1, 2, 3]", "py/object": "__main__.Test"}, "py/object": "__main__.AppResponse"}'

As you can see, the second occurrence of "devices" is a string, "[1, 2, 3]", and not a list because of the fact that make_refs=False relies on the repr() function. I'd like to have all the occurrences to be encoded the same way, like lists, without data loss. I've tried using max_depth, but without success.

[gabyx]

Is that really not a Bug?
I think cycle detetion is wrong here?? There is not cycle involved here. But repr gets called.
I have the same problem with

import jsonpickle

class A(object):
    def __init__(self, **kwargs):
 
        for k, v in kwargs.items():
            self.__dict__[k] = v
            
            
t = A(relFilePath="asd")
c= {"1":t, "2":t}
s = jsonpickle.encode(c, make_refs=False)
cc= jsonpickle.decode(s)

print(s)
print(cc)

{"1": {"py/object": "__main__.A", "relFilePath": "asd"}, "2": "<__main__.A object at 0x000001658C064CF8>"}
{'1': <__main__.A object at 0x000001658C064DA0>, '2': '<__main__.A object at 0x000001658C064CF8>'} ///<<<< String here

[davvid]

@gabyx it works just fine if you pass keys=True to encode(..) and decode(..).

If you could kindly provide a documentation patch to help make this more obvious it would be much appreciated.

True, the cycle detection is noticing that the list is the same object and short-circuiting itself. Having so many booleans does make things complicated, but perhaps the make_refs=False code path shouldn't consider list as a cycle candidate.

That's actually a very hard question. I present you with this puzzle:

x = []
x.append(x)

What should cycle detection do? Is there a reasonable encoding?

[gabyx]

Shouldnt be cycle detection doing a graph traversal (assuming it should always be a DAG, with several roots) and really detect if there is a cycle:

x = 3
c = [x]
d = [x]
b = [c, d]
d.append(b)
a = [c, b]

So in this example you visit a first -> then its childs where you visit a[0] first where you descent and fully serialize the data, then you visit a[1] where you end up in a visit path a[1]->b[1]->d[1] which directly triggers the cycle detection because d[1] := a[1] := b is already in the path from root
Is that implemented like that?

I updated the example above -> keys= isn't the problem here... I can have simple string keys and it does not work... and uses repr

[MicheleMannino]

In my above example, it seems to be that there are not any circular references...

[MicheleMannino]

Using keys=True and make_refs=False and keys=True

>>> jsonpickle.encode(response, keys=True, make_refs=False)
'{"actions": [{"devices": {"devices": [1, 2, 3], "py/object": "__main__.Test"}, "py/object": "__main__.Test"}], "model": {"devices": "[1, 2, 3]", "py/object": "__main__.Test"}, "py/object": "__main__.AppResponse"}'

>>> jsonpickle.encode(response, keys=True)
'{"actions": [{"devices": {"devices": [1, 2, 3], "py/object": "__main__.Test"}, "py/object": "__main__.Test"}], "model": {"devices": {"py/id": 4}, "py/object": "__main__.Test"}, "py/object": "__main__.AppResponse"}'

I'd like to have "devices": [1, 2, 3] and not "devices": {"py/id": 4}