SerializationAndJSON.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6148fc1d",
   "metadata": {},
   "source": [
    "# Serialization \n",
    "\n",
    "### Topics\n",
    "\n",
    "- serializing objects\n",
    "- deserializing objects\n",
    "- pickle\n",
    "- JSON"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1035668",
   "metadata": {},
   "source": [
    "## Serializing objects\n",
    "\n",
    "- objects created in memory are not persistent\n",
    "    - they hold a lot of important information/states of objects as the program is being executed\n",
    "- to make an object persistent, we need to create a series of bytes that represent the state of the object, and write those bytes to a file\n",
    "    - need to encode objects (**serializing**)\n",
    "    - also need to decode objects from series of bytes (**deserializing**)\n",
    "- web services often use a service described as **RESTful**\n",
    "    - REST - REpresentational State Transfer\n",
    "- the server and client will exchange representation of objects using RESTful services\n",
    "- several ways to serialize and objects\n",
    "- Pickle, JSON, XML, etc."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e002f55b",
   "metadata": {},
   "source": [
    "## Serializing objects using pickle\n",
    "\n",
    "- Python `pickle` module is an object-oriented way to store object state directly in a special storage format\n",
    "- essentially converts an object's state (and all the state of all the objects it holds as attributes) into a series of bytes\n",
    "    - these bytes then can be transported or stored however we see fit\n",
    "- pickle file are for temporary persistence\n",
    "    - Pickled file using Python 3.7 may not be unpickled by newer version of Python or vice versa\n",
    "- NOTE - pickle module is not secure\n",
    "    - only unpickle data you trust!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "ff2be1bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "6cbbeb66",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on module pickle:\n",
      "\n",
      "NAME\n",
      "    pickle - Create portable serialized representations of Python objects.\n",
      "\n",
      "MODULE REFERENCE\n",
      "    https://docs.python.org/3.10/library/pickle.html\n",
      "    \n",
      "    The following documentation is automatically generated from the Python\n",
      "    source files.  It may be incomplete, incorrect or include features that\n",
      "    are considered implementation detail and may vary between Python\n",
      "    implementations.  When in doubt, consult the module reference at the\n",
      "    location listed above.\n",
      "\n",
      "DESCRIPTION\n",
      "    See module copyreg for a mechanism for registering custom picklers.\n",
      "    See module pickletools source for extensive comments.\n",
      "    \n",
      "    Classes:\n",
      "    \n",
      "        Pickler\n",
      "        Unpickler\n",
      "    \n",
      "    Functions:\n",
      "    \n",
      "        dump(object, file)\n",
      "        dumps(object) -> string\n",
      "        load(file) -> object\n",
      "        loads(bytes) -> object\n",
      "    \n",
      "    Misc variables:\n",
      "    \n",
      "        __version__\n",
      "        format_version\n",
      "        compatible_formats\n",
      "\n",
      "CLASSES\n",
      "    builtins.Exception(builtins.BaseException)\n",
      "        _pickle.PickleError\n",
      "            _pickle.PicklingError\n",
      "            _pickle.UnpicklingError\n",
      "    builtins.object\n",
      "        _pickle.Pickler\n",
      "        _pickle.Unpickler\n",
      "        PickleBuffer\n",
      "    \n",
      "    class PickleBuffer(builtins.object)\n",
      "     |  Wrapper for potentially out-of-band buffers\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  raw(self, /)\n",
      "     |      Return a memoryview of the raw memory underlying this buffer.\n",
      "     |      Will raise BufferError is the buffer isn't contiguous.\n",
      "     |  \n",
      "     |  release(self, /)\n",
      "     |      Release the underlying buffer exposed by the PickleBuffer object.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods defined here:\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "    \n",
      "    class PickleError(builtins.Exception)\n",
      "     |  Method resolution order:\n",
      "     |      PickleError\n",
      "     |      builtins.Exception\n",
      "     |      builtins.BaseException\n",
      "     |      builtins.object\n",
      "     |  \n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.Exception:\n",
      "     |  \n",
      "     |  __init__(self, /, *args, **kwargs)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods inherited from builtins.Exception:\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __delattr__(self, name, /)\n",
      "     |      Implement delattr(self, name).\n",
      "     |  \n",
      "     |  __getattribute__(self, name, /)\n",
      "     |      Return getattr(self, name).\n",
      "     |  \n",
      "     |  __reduce__(...)\n",
      "     |      Helper for pickle.\n",
      "     |  \n",
      "     |  __repr__(self, /)\n",
      "     |      Return repr(self).\n",
      "     |  \n",
      "     |  __setattr__(self, name, value, /)\n",
      "     |      Implement setattr(self, name, value).\n",
      "     |  \n",
      "     |  __setstate__(...)\n",
      "     |  \n",
      "     |  __str__(self, /)\n",
      "     |      Return str(self).\n",
      "     |  \n",
      "     |  with_traceback(...)\n",
      "     |      Exception.with_traceback(tb) --\n",
      "     |      set self.__traceback__ to tb and return self.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __cause__\n",
      "     |      exception cause\n",
      "     |  \n",
      "     |  __context__\n",
      "     |      exception context\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |  \n",
      "     |  __suppress_context__\n",
      "     |  \n",
      "     |  __traceback__\n",
      "     |  \n",
      "     |  args\n",
      "    \n",
      "    class Pickler(builtins.object)\n",
      "     |  Pickler(file, protocol=None, fix_imports=True, buffer_callback=None)\n",
      "     |  \n",
      "     |  This takes a binary file for writing a pickle data stream.\n",
      "     |  \n",
      "     |  The optional *protocol* argument tells the pickler to use the given\n",
      "     |  protocol; supported protocols are 0, 1, 2, 3, 4 and 5.  The default\n",
      "     |  protocol is 4. It was introduced in Python 3.4, and is incompatible\n",
      "     |  with previous versions.\n",
      "     |  \n",
      "     |  Specifying a negative protocol version selects the highest protocol\n",
      "     |  version supported.  The higher the protocol used, the more recent the\n",
      "     |  version of Python needed to read the pickle produced.\n",
      "     |  \n",
      "     |  The *file* argument must have a write() method that accepts a single\n",
      "     |  bytes argument. It can thus be a file object opened for binary\n",
      "     |  writing, an io.BytesIO instance, or any other custom object that meets\n",
      "     |  this interface.\n",
      "     |  \n",
      "     |  If *fix_imports* is True and protocol is less than 3, pickle will try\n",
      "     |  to map the new Python 3 names to the old module names used in Python\n",
      "     |  2, so that the pickle data stream is readable with Python 2.\n",
      "     |  \n",
      "     |  If *buffer_callback* is None (the default), buffer views are\n",
      "     |  serialized into *file* as part of the pickle stream.\n",
      "     |  \n",
      "     |  If *buffer_callback* is not None, then it can be called any number\n",
      "     |  of times with a buffer view.  If the callback returns a false value\n",
      "     |  (such as None), the given buffer is out-of-band; otherwise the\n",
      "     |  buffer is serialized in-band, i.e. inside the pickle stream.\n",
      "     |  \n",
      "     |  It is an error if *buffer_callback* is not None and *protocol*\n",
      "     |  is None or smaller than 5.\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  __init__(self, /, *args, **kwargs)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  __sizeof__(self, /)\n",
      "     |      Returns size in memory, in bytes.\n",
      "     |  \n",
      "     |  clear_memo(self, /)\n",
      "     |      Clears the pickler's \"memo\".\n",
      "     |      \n",
      "     |      The memo is the data structure that remembers which objects the\n",
      "     |      pickler has already seen, so that shared or recursive objects are\n",
      "     |      pickled by reference and not by value.  This method is useful when\n",
      "     |      re-using picklers.\n",
      "     |  \n",
      "     |  dump(self, obj, /)\n",
      "     |      Write a pickled representation of the given object to the open file.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods defined here:\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  bin\n",
      "     |  \n",
      "     |  dispatch_table\n",
      "     |  \n",
      "     |  fast\n",
      "     |  \n",
      "     |  memo\n",
      "     |  \n",
      "     |  persistent_id\n",
      "    \n",
      "    class PicklingError(PickleError)\n",
      "     |  Method resolution order:\n",
      "     |      PicklingError\n",
      "     |      PickleError\n",
      "     |      builtins.Exception\n",
      "     |      builtins.BaseException\n",
      "     |      builtins.object\n",
      "     |  \n",
      "     |  Data descriptors inherited from PickleError:\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.Exception:\n",
      "     |  \n",
      "     |  __init__(self, /, *args, **kwargs)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods inherited from builtins.Exception:\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __delattr__(self, name, /)\n",
      "     |      Implement delattr(self, name).\n",
      "     |  \n",
      "     |  __getattribute__(self, name, /)\n",
      "     |      Return getattr(self, name).\n",
      "     |  \n",
      "     |  __reduce__(...)\n",
      "     |      Helper for pickle.\n",
      "     |  \n",
      "     |  __repr__(self, /)\n",
      "     |      Return repr(self).\n",
      "     |  \n",
      "     |  __setattr__(self, name, value, /)\n",
      "     |      Implement setattr(self, name, value).\n",
      "     |  \n",
      "     |  __setstate__(...)\n",
      "     |  \n",
      "     |  __str__(self, /)\n",
      "     |      Return str(self).\n",
      "     |  \n",
      "     |  with_traceback(...)\n",
      "     |      Exception.with_traceback(tb) --\n",
      "     |      set self.__traceback__ to tb and return self.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __cause__\n",
      "     |      exception cause\n",
      "     |  \n",
      "     |  __context__\n",
      "     |      exception context\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |  \n",
      "     |  __suppress_context__\n",
      "     |  \n",
      "     |  __traceback__\n",
      "     |  \n",
      "     |  args\n",
      "    \n",
      "    class Unpickler(builtins.object)\n",
      "     |  Unpickler(file, *, fix_imports=True, encoding='ASCII', errors='strict', buffers=())\n",
      "     |  \n",
      "     |  This takes a binary file for reading a pickle data stream.\n",
      "     |  \n",
      "     |  The protocol version of the pickle is detected automatically, so no\n",
      "     |  protocol argument is needed.  Bytes past the pickled object's\n",
      "     |  representation are ignored.\n",
      "     |  \n",
      "     |  The argument *file* must have two methods, a read() method that takes\n",
      "     |  an integer argument, and a readline() method that requires no\n",
      "     |  arguments.  Both methods should return bytes.  Thus *file* can be a\n",
      "     |  binary file object opened for reading, an io.BytesIO object, or any\n",
      "     |  other custom object that meets this interface.\n",
      "     |  \n",
      "     |  Optional keyword arguments are *fix_imports*, *encoding* and *errors*,\n",
      "     |  which are used to control compatibility support for pickle stream\n",
      "     |  generated by Python 2.  If *fix_imports* is True, pickle will try to\n",
      "     |  map the old Python 2 names to the new names used in Python 3.  The\n",
      "     |  *encoding* and *errors* tell pickle how to decode 8-bit string\n",
      "     |  instances pickled by Python 2; these default to 'ASCII' and 'strict',\n",
      "     |  respectively.  The *encoding* can be 'bytes' to read these 8-bit\n",
      "     |  string instances as bytes objects.\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  __init__(self, /, *args, **kwargs)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  __sizeof__(self, /)\n",
      "     |      Returns size in memory, in bytes.\n",
      "     |  \n",
      "     |  find_class(self, module_name, global_name, /)\n",
      "     |      Return an object from a specified module.\n",
      "     |      \n",
      "     |      If necessary, the module will be imported. Subclasses may override\n",
      "     |      this method (e.g. to restrict unpickling of arbitrary classes and\n",
      "     |      functions).\n",
      "     |      \n",
      "     |      This method is called whenever a class or a function object is\n",
      "     |      needed.  Both arguments passed are str objects.\n",
      "     |  \n",
      "     |  load(self, /)\n",
      "     |      Load a pickle.\n",
      "     |      \n",
      "     |      Read a pickled object representation from the open file object given\n",
      "     |      in the constructor, and return the reconstituted object hierarchy\n",
      "     |      specified therein.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods defined here:\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  memo\n",
      "     |  \n",
      "     |  persistent_load\n",
      "    \n",
      "    class UnpicklingError(PickleError)\n",
      "     |  Method resolution order:\n",
      "     |      UnpicklingError\n",
      "     |      PickleError\n",
      "     |      builtins.Exception\n",
      "     |      builtins.BaseException\n",
      "     |      builtins.object\n",
      "     |  \n",
      "     |  Data descriptors inherited from PickleError:\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.Exception:\n",
      "     |  \n",
      "     |  __init__(self, /, *args, **kwargs)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods inherited from builtins.Exception:\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __delattr__(self, name, /)\n",
      "     |      Implement delattr(self, name).\n",
      "     |  \n",
      "     |  __getattribute__(self, name, /)\n",
      "     |      Return getattr(self, name).\n",
      "     |  \n",
      "     |  __reduce__(...)\n",
      "     |      Helper for pickle.\n",
      "     |  \n",
      "     |  __repr__(self, /)\n",
      "     |      Return repr(self).\n",
      "     |  \n",
      "     |  __setattr__(self, name, value, /)\n",
      "     |      Implement setattr(self, name, value).\n",
      "     |  \n",
      "     |  __setstate__(...)\n",
      "     |  \n",
      "     |  __str__(self, /)\n",
      "     |      Return str(self).\n",
      "     |  \n",
      "     |  with_traceback(...)\n",
      "     |      Exception.with_traceback(tb) --\n",
      "     |      set self.__traceback__ to tb and return self.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __cause__\n",
      "     |      exception cause\n",
      "     |  \n",
      "     |  __context__\n",
      "     |      exception context\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |  \n",
      "     |  __suppress_context__\n",
      "     |  \n",
      "     |  __traceback__\n",
      "     |  \n",
      "     |  args\n",
      "\n",
      "FUNCTIONS\n",
      "    dump(obj, file, protocol=None, *, fix_imports=True, buffer_callback=None)\n",
      "        Write a pickled representation of obj to the open file object file.\n",
      "        \n",
      "        This is equivalent to ``Pickler(file, protocol).dump(obj)``, but may\n",
      "        be more efficient.\n",
      "        \n",
      "        The optional *protocol* argument tells the pickler to use the given\n",
      "        protocol; supported protocols are 0, 1, 2, 3, 4 and 5.  The default\n",
      "        protocol is 4. It was introduced in Python 3.4, and is incompatible\n",
      "        with previous versions.\n",
      "        \n",
      "        Specifying a negative protocol version selects the highest protocol\n",
      "        version supported.  The higher the protocol used, the more recent the\n",
      "        version of Python needed to read the pickle produced.\n",
      "        \n",
      "        The *file* argument must have a write() method that accepts a single\n",
      "        bytes argument.  It can thus be a file object opened for binary\n",
      "        writing, an io.BytesIO instance, or any other custom object that meets\n",
      "        this interface.\n",
      "        \n",
      "        If *fix_imports* is True and protocol is less than 3, pickle will try\n",
      "        to map the new Python 3 names to the old module names used in Python\n",
      "        2, so that the pickle data stream is readable with Python 2.\n",
      "        \n",
      "        If *buffer_callback* is None (the default), buffer views are serialized\n",
      "        into *file* as part of the pickle stream.  It is an error if\n",
      "        *buffer_callback* is not None and *protocol* is None or smaller than 5.\n",
      "    \n",
      "    dumps(obj, protocol=None, *, fix_imports=True, buffer_callback=None)\n",
      "        Return the pickled representation of the object as a bytes object.\n",
      "        \n",
      "        The optional *protocol* argument tells the pickler to use the given\n",
      "        protocol; supported protocols are 0, 1, 2, 3, 4 and 5.  The default\n",
      "        protocol is 4. It was introduced in Python 3.4, and is incompatible\n",
      "        with previous versions.\n",
      "        \n",
      "        Specifying a negative protocol version selects the highest protocol\n",
      "        version supported.  The higher the protocol used, the more recent the\n",
      "        version of Python needed to read the pickle produced.\n",
      "        \n",
      "        If *fix_imports* is True and *protocol* is less than 3, pickle will\n",
      "        try to map the new Python 3 names to the old module names used in\n",
      "        Python 2, so that the pickle data stream is readable with Python 2.\n",
      "        \n",
      "        If *buffer_callback* is None (the default), buffer views are serialized\n",
      "        into *file* as part of the pickle stream.  It is an error if\n",
      "        *buffer_callback* is not None and *protocol* is None or smaller than 5.\n",
      "    \n",
      "    load(file, *, fix_imports=True, encoding='ASCII', errors='strict', buffers=())\n",
      "        Read and return an object from the pickle data stored in a file.\n",
      "        \n",
      "        This is equivalent to ``Unpickler(file).load()``, but may be more\n",
      "        efficient.\n",
      "        \n",
      "        The protocol version of the pickle is detected automatically, so no\n",
      "        protocol argument is needed.  Bytes past the pickled object's\n",
      "        representation are ignored.\n",
      "        \n",
      "        The argument *file* must have two methods, a read() method that takes\n",
      "        an integer argument, and a readline() method that requires no\n",
      "        arguments.  Both methods should return bytes.  Thus *file* can be a\n",
      "        binary file object opened for reading, an io.BytesIO object, or any\n",
      "        other custom object that meets this interface.\n",
      "        \n",
      "        Optional keyword arguments are *fix_imports*, *encoding* and *errors*,\n",
      "        which are used to control compatibility support for pickle stream\n",
      "        generated by Python 2.  If *fix_imports* is True, pickle will try to\n",
      "        map the old Python 2 names to the new names used in Python 3.  The\n",
      "        *encoding* and *errors* tell pickle how to decode 8-bit string\n",
      "        instances pickled by Python 2; these default to 'ASCII' and 'strict',\n",
      "        respectively.  The *encoding* can be 'bytes' to read these 8-bit\n",
      "        string instances as bytes objects.\n",
      "    \n",
      "    loads(data, /, *, fix_imports=True, encoding='ASCII', errors='strict', buffers=())\n",
      "        Read and return an object from the given pickle data.\n",
      "        \n",
      "        The protocol version of the pickle is detected automatically, so no\n",
      "        protocol argument is needed.  Bytes past the pickled object's\n",
      "        representation are ignored.\n",
      "        \n",
      "        Optional keyword arguments are *fix_imports*, *encoding* and *errors*,\n",
      "        which are used to control compatibility support for pickle stream\n",
      "        generated by Python 2.  If *fix_imports* is True, pickle will try to\n",
      "        map the old Python 2 names to the new names used in Python 3.  The\n",
      "        *encoding* and *errors* tell pickle how to decode 8-bit string\n",
      "        instances pickled by Python 2; these default to 'ASCII' and 'strict',\n",
      "        respectively.  The *encoding* can be 'bytes' to read these 8-bit\n",
      "        string instances as bytes objects.\n",
      "\n",
      "DATA\n",
      "    ADDITEMS = b'\\x90'\n",
      "    APPEND = b'a'\n",
      "    APPENDS = b'e'\n",
      "    BINBYTES = b'B'\n",
      "    BINBYTES8 = b'\\x8e'\n",
      "    BINFLOAT = b'G'\n",
      "    BINGET = b'h'\n",
      "    BININT = b'J'\n",
      "    BININT1 = b'K'\n",
      "    BININT2 = b'M'\n",
      "    BINPERSID = b'Q'\n",
      "    BINPUT = b'q'\n",
      "    BINSTRING = b'T'\n",
      "    BINUNICODE = b'X'\n",
      "    BINUNICODE8 = b'\\x8d'\n",
      "    BUILD = b'b'\n",
      "    BYTEARRAY8 = b'\\x96'\n",
      "    DEFAULT_PROTOCOL = 4\n",
      "    DICT = b'd'\n",
      "    DUP = b'2'\n",
      "    EMPTY_DICT = b'}'\n",
      "    EMPTY_LIST = b']'\n",
      "    EMPTY_SET = b'\\x8f'\n",
      "    EMPTY_TUPLE = b')'\n",
      "    EXT1 = b'\\x82'\n",
      "    EXT2 = b'\\x83'\n",
      "    EXT4 = b'\\x84'\n",
      "    FALSE = b'I00\\n'\n",
      "    FLOAT = b'F'\n",
      "    FRAME = b'\\x95'\n",
      "    FROZENSET = b'\\x91'\n",
      "    GET = b'g'\n",
      "    GLOBAL = b'c'\n",
      "    HIGHEST_PROTOCOL = 5\n",
      "    INST = b'i'\n",
      "    INT = b'I'\n",
      "    LIST = b'l'\n",
      "    LONG = b'L'\n",
      "    LONG1 = b'\\x8a'\n",
      "    LONG4 = b'\\x8b'\n",
      "    LONG_BINGET = b'j'\n",
      "    LONG_BINPUT = b'r'\n",
      "    MARK = b'('\n",
      "    MEMOIZE = b'\\x94'\n",
      "    NEWFALSE = b'\\x89'\n",
      "    NEWOBJ = b'\\x81'\n",
      "    NEWOBJ_EX = b'\\x92'\n",
      "    NEWTRUE = b'\\x88'\n",
      "    NEXT_BUFFER = b'\\x97'\n",
      "    NONE = b'N'\n",
      "    OBJ = b'o'\n",
      "    PERSID = b'P'\n",
      "    POP = b'0'\n",
      "    POP_MARK = b'1'\n",
      "    PROTO = b'\\x80'\n",
      "    PUT = b'p'\n",
      "    READONLY_BUFFER = b'\\x98'\n",
      "    REDUCE = b'R'\n",
      "    SETITEM = b's'\n",
      "    SETITEMS = b'u'\n",
      "    SHORT_BINBYTES = b'C'\n",
      "    SHORT_BINSTRING = b'U'\n",
      "    SHORT_BINUNICODE = b'\\x8c'\n",
      "    STACK_GLOBAL = b'\\x93'\n",
      "    STOP = b'.'\n",
      "    STRING = b'S'\n",
      "    TRUE = b'I01\\n'\n",
      "    TUPLE = b't'\n",
      "    TUPLE1 = b'\\x85'\n",
      "    TUPLE2 = b'\\x86'\n",
      "    TUPLE3 = b'\\x87'\n",
      "    UNICODE = b'V'\n",
      "    __all__ = ['PickleError', 'PicklingError', 'UnpicklingError', 'Pickler...\n",
      "\n",
      "FILE\n",
      "    /opt/anaconda3/envs/py/lib/python3.10/pickle.py\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(pickle)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "1da7d203",
   "metadata": {},
   "outputs": [],
   "source": [
    "some_data = ['a list', 'containing', 5, 'items', {\"including\": (\"str\", \"int\", \"dict\", \"list\", \"tuple\")}]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9f33d1ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a list',\n",
       " 'containing',\n",
       " 5,\n",
       " 'items',\n",
       " {'including': ('str', 'int', 'dict', 'list', 'tuple')}]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "some_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "129920c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's pickle/serialize the object list object\n",
    "with open(\"pickled_list.pkl\", 'wb') as file:\n",
    "    pickle.dump(some_data, file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d1d68389",
   "metadata": {},
   "outputs": [],
   "source": [
    "# lets deserialize/load the object from file\n",
    "with open(\"pickled_list.pkl\", 'rb') as file:\n",
    "    loaded_list = pickle.load(file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "638c92e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(loaded_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "60b360d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a list',\n",
       " 'containing',\n",
       " 5,\n",
       " 'items',\n",
       " {'including': ('str', 'int', 'dict', 'list', 'tuple')}]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ad111db7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "some_data == loaded_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "be798858",
   "metadata": {},
   "outputs": [],
   "source": [
    "loaded_list.append([1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "307bc277",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a list',\n",
       " 'containing',\n",
       " 5,\n",
       " 'items',\n",
       " {'including': ('str', 'int', 'dict', 'list', 'tuple')},\n",
       " [1, 2, 3]]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loaded_list"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "871b1f1e",
   "metadata": {},
   "source": [
    "## Serializing objects using JSON\n",
    "\n",
    "- there are many text-based format to exchange data\n",
    "- XML - Extensive Markup Language (XML)\n",
    "- YAML - Yet Another Markup Language\n",
    "- CSV - Comma-Separated Value\n",
    "- most of these techniques have obscure features that can be exploited from security point of view\n",
    "    - allow arbitrary commands to be executed on the host machine\n",
    "\n",
    "### JSON - JavaScript Object Notation\n",
    "\n",
    "- human-readable text-based format for exchanging data\n",
    "- one of the most popular tecnique used by RESTful API services\n",
    "- JSON as the name says is more popular in JavaScript language to transfer data to a browswer from the web server\n",
    "- JSON though popular is not as robust as the **pickle** module\n",
    "- it can serialize only basic data: \n",
    "    - integers, floats, strings and simple containers such as list and dictionaries\n",
    "- generally, `json` module's functions try to serialize the object's state using the value of the object's `__dict__` attribute\n",
    "- json provides **dump** and **dumps**, **load** and **loads** functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ccbc2fab",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "acaf1b92",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on package json:\n",
      "\n",
      "NAME\n",
      "    json\n",
      "\n",
      "MODULE REFERENCE\n",
      "    https://docs.python.org/3.10/library/json.html\n",
      "    \n",
      "    The following documentation is automatically generated from the Python\n",
      "    source files.  It may be incomplete, incorrect or include features that\n",
      "    are considered implementation detail and may vary between Python\n",
      "    implementations.  When in doubt, consult the module reference at the\n",
      "    location listed above.\n",
      "\n",
      "DESCRIPTION\n",
      "    JSON (JavaScript Object Notation) <https://json.org> is a subset of\n",
      "    JavaScript syntax (ECMA-262 3rd edition) used as a lightweight data\n",
      "    interchange format.\n",
      "    \n",
      "    :mod:`json` exposes an API familiar to users of the standard library\n",
      "    :mod:`marshal` and :mod:`pickle` modules.  It is derived from a\n",
      "    version of the externally maintained simplejson library.\n",
      "    \n",
      "    Encoding basic Python object hierarchies::\n",
      "    \n",
      "        >>> import json\n",
      "        >>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])\n",
      "        '[\"foo\", {\"bar\": [\"baz\", null, 1.0, 2]}]'\n",
      "        >>> print(json.dumps(\"\\\"foo\\bar\"))\n",
      "        \"\\\"foo\\bar\"\n",
      "        >>> print(json.dumps('\\u1234'))\n",
      "        \"\\u1234\"\n",
      "        >>> print(json.dumps('\\\\'))\n",
      "        \"\\\\\"\n",
      "        >>> print(json.dumps({\"c\": 0, \"b\": 0, \"a\": 0}, sort_keys=True))\n",
      "        {\"a\": 0, \"b\": 0, \"c\": 0}\n",
      "        >>> from io import StringIO\n",
      "        >>> io = StringIO()\n",
      "        >>> json.dump(['streaming API'], io)\n",
      "        >>> io.getvalue()\n",
      "        '[\"streaming API\"]'\n",
      "    \n",
      "    Compact encoding::\n",
      "    \n",
      "        >>> import json\n",
      "        >>> mydict = {'4': 5, '6': 7}\n",
      "        >>> json.dumps([1,2,3,mydict], separators=(',', ':'))\n",
      "        '[1,2,3,{\"4\":5,\"6\":7}]'\n",
      "    \n",
      "    Pretty printing::\n",
      "    \n",
      "        >>> import json\n",
      "        >>> print(json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4))\n",
      "        {\n",
      "            \"4\": 5,\n",
      "            \"6\": 7\n",
      "        }\n",
      "    \n",
      "    Decoding JSON::\n",
      "    \n",
      "        >>> import json\n",
      "        >>> obj = ['foo', {'bar': ['baz', None, 1.0, 2]}]\n",
      "        >>> json.loads('[\"foo\", {\"bar\":[\"baz\", null, 1.0, 2]}]') == obj\n",
      "        True\n",
      "        >>> json.loads('\"\\\\\"foo\\\\bar\"') == '\"foo\\x08ar'\n",
      "        True\n",
      "        >>> from io import StringIO\n",
      "        >>> io = StringIO('[\"streaming API\"]')\n",
      "        >>> json.load(io)[0] == 'streaming API'\n",
      "        True\n",
      "    \n",
      "    Specializing JSON object decoding::\n",
      "    \n",
      "        >>> import json\n",
      "        >>> def as_complex(dct):\n",
      "        ...     if '__complex__' in dct:\n",
      "        ...         return complex(dct['real'], dct['imag'])\n",
      "        ...     return dct\n",
      "        ...\n",
      "        >>> json.loads('{\"__complex__\": true, \"real\": 1, \"imag\": 2}',\n",
      "        ...     object_hook=as_complex)\n",
      "        (1+2j)\n",
      "        >>> from decimal import Decimal\n",
      "        >>> json.loads('1.1', parse_float=Decimal) == Decimal('1.1')\n",
      "        True\n",
      "    \n",
      "    Specializing JSON object encoding::\n",
      "    \n",
      "        >>> import json\n",
      "        >>> def encode_complex(obj):\n",
      "        ...     if isinstance(obj, complex):\n",
      "        ...         return [obj.real, obj.imag]\n",
      "        ...     raise TypeError(f'Object of type {obj.__class__.__name__} '\n",
      "        ...                     f'is not JSON serializable')\n",
      "        ...\n",
      "        >>> json.dumps(2 + 1j, default=encode_complex)\n",
      "        '[2.0, 1.0]'\n",
      "        >>> json.JSONEncoder(default=encode_complex).encode(2 + 1j)\n",
      "        '[2.0, 1.0]'\n",
      "        >>> ''.join(json.JSONEncoder(default=encode_complex).iterencode(2 + 1j))\n",
      "        '[2.0, 1.0]'\n",
      "    \n",
      "    \n",
      "    Using json.tool from the shell to validate and pretty-print::\n",
      "    \n",
      "        $ echo '{\"json\":\"obj\"}' | python -m json.tool\n",
      "        {\n",
      "            \"json\": \"obj\"\n",
      "        }\n",
      "        $ echo '{ 1.2:3.4}' | python -m json.tool\n",
      "        Expecting property name enclosed in double quotes: line 1 column 3 (char 2)\n",
      "\n",
      "PACKAGE CONTENTS\n",
      "    decoder\n",
      "    encoder\n",
      "    scanner\n",
      "    tool\n",
      "\n",
      "CLASSES\n",
      "    builtins.ValueError(builtins.Exception)\n",
      "        json.decoder.JSONDecodeError\n",
      "    builtins.object\n",
      "        json.decoder.JSONDecoder\n",
      "        json.encoder.JSONEncoder\n",
      "    \n",
      "    class JSONDecodeError(builtins.ValueError)\n",
      "     |  JSONDecodeError(msg, doc, pos)\n",
      "     |  \n",
      "     |  Subclass of ValueError with the following additional properties:\n",
      "     |  \n",
      "     |  msg: The unformatted error message\n",
      "     |  doc: The JSON document being parsed\n",
      "     |  pos: The start index of doc where parsing failed\n",
      "     |  lineno: The line corresponding to pos\n",
      "     |  colno: The column corresponding to pos\n",
      "     |  \n",
      "     |  Method resolution order:\n",
      "     |      JSONDecodeError\n",
      "     |      builtins.ValueError\n",
      "     |      builtins.Exception\n",
      "     |      builtins.BaseException\n",
      "     |      builtins.object\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  __init__(self, msg, doc, pos)\n",
      "     |      Initialize self.  See help(type(self)) for accurate signature.\n",
      "     |  \n",
      "     |  __reduce__(self)\n",
      "     |      Helper for pickle.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Static methods inherited from builtins.ValueError:\n",
      "     |  \n",
      "     |  __new__(*args, **kwargs) from builtins.type\n",
      "     |      Create and return a new object.  See help(type) for accurate signature.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Methods inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __delattr__(self, name, /)\n",
      "     |      Implement delattr(self, name).\n",
      "     |  \n",
      "     |  __getattribute__(self, name, /)\n",
      "     |      Return getattr(self, name).\n",
      "     |  \n",
      "     |  __repr__(self, /)\n",
      "     |      Return repr(self).\n",
      "     |  \n",
      "     |  __setattr__(self, name, value, /)\n",
      "     |      Implement setattr(self, name, value).\n",
      "     |  \n",
      "     |  __setstate__(...)\n",
      "     |  \n",
      "     |  __str__(self, /)\n",
      "     |      Return str(self).\n",
      "     |  \n",
      "     |  with_traceback(...)\n",
      "     |      Exception.with_traceback(tb) --\n",
      "     |      set self.__traceback__ to tb and return self.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors inherited from builtins.BaseException:\n",
      "     |  \n",
      "     |  __cause__\n",
      "     |      exception cause\n",
      "     |  \n",
      "     |  __context__\n",
      "     |      exception context\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |  \n",
      "     |  __suppress_context__\n",
      "     |  \n",
      "     |  __traceback__\n",
      "     |  \n",
      "     |  args\n",
      "    \n",
      "    class JSONDecoder(builtins.object)\n",
      "     |  JSONDecoder(*, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)\n",
      "     |  \n",
      "     |  Simple JSON <https://json.org> decoder\n",
      "     |  \n",
      "     |  Performs the following translations in decoding by default:\n",
      "     |  \n",
      "     |  +---------------+-------------------+\n",
      "     |  | JSON          | Python            |\n",
      "     |  +===============+===================+\n",
      "     |  | object        | dict              |\n",
      "     |  +---------------+-------------------+\n",
      "     |  | array         | list              |\n",
      "     |  +---------------+-------------------+\n",
      "     |  | string        | str               |\n",
      "     |  +---------------+-------------------+\n",
      "     |  | number (int)  | int               |\n",
      "     |  +---------------+-------------------+\n",
      "     |  | number (real) | float             |\n",
      "     |  +---------------+-------------------+\n",
      "     |  | true          | True              |\n",
      "     |  +---------------+-------------------+\n",
      "     |  | false         | False             |\n",
      "     |  +---------------+-------------------+\n",
      "     |  | null          | None              |\n",
      "     |  +---------------+-------------------+\n",
      "     |  \n",
      "     |  It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as\n",
      "     |  their corresponding ``float`` values, which is outside the JSON spec.\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  __init__(self, *, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, strict=True, object_pairs_hook=None)\n",
      "     |      ``object_hook``, if specified, will be called with the result\n",
      "     |      of every JSON object decoded and its return value will be used in\n",
      "     |      place of the given ``dict``.  This can be used to provide custom\n",
      "     |      deserializations (e.g. to support JSON-RPC class hinting).\n",
      "     |      \n",
      "     |      ``object_pairs_hook``, if specified will be called with the result of\n",
      "     |      every JSON object decoded with an ordered list of pairs.  The return\n",
      "     |      value of ``object_pairs_hook`` will be used instead of the ``dict``.\n",
      "     |      This feature can be used to implement custom decoders.\n",
      "     |      If ``object_hook`` is also defined, the ``object_pairs_hook`` takes\n",
      "     |      priority.\n",
      "     |      \n",
      "     |      ``parse_float``, if specified, will be called with the string\n",
      "     |      of every JSON float to be decoded. By default this is equivalent to\n",
      "     |      float(num_str). This can be used to use another datatype or parser\n",
      "     |      for JSON floats (e.g. decimal.Decimal).\n",
      "     |      \n",
      "     |      ``parse_int``, if specified, will be called with the string\n",
      "     |      of every JSON int to be decoded. By default this is equivalent to\n",
      "     |      int(num_str). This can be used to use another datatype or parser\n",
      "     |      for JSON integers (e.g. float).\n",
      "     |      \n",
      "     |      ``parse_constant``, if specified, will be called with one of the\n",
      "     |      following strings: -Infinity, Infinity, NaN.\n",
      "     |      This can be used to raise an exception if invalid JSON numbers\n",
      "     |      are encountered.\n",
      "     |      \n",
      "     |      If ``strict`` is false (true is the default), then control\n",
      "     |      characters will be allowed inside strings.  Control characters in\n",
      "     |      this context are those with character codes in the 0-31 range,\n",
      "     |      including ``'\\t'`` (tab), ``'\\n'``, ``'\\r'`` and ``'\\0'``.\n",
      "     |  \n",
      "     |  decode(self, s, _w=<built-in method match of re.Pattern object at 0x7fb7bc4735e0>)\n",
      "     |      Return the Python representation of ``s`` (a ``str`` instance\n",
      "     |      containing a JSON document).\n",
      "     |  \n",
      "     |  raw_decode(self, s, idx=0)\n",
      "     |      Decode a JSON document from ``s`` (a ``str`` beginning with\n",
      "     |      a JSON document) and return a 2-tuple of the Python\n",
      "     |      representation and the index in ``s`` where the document ended.\n",
      "     |      \n",
      "     |      This can be used to decode a JSON document from a string that may\n",
      "     |      have extraneous data at the end.\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |      dictionary for instance variables (if defined)\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "    \n",
      "    class JSONEncoder(builtins.object)\n",
      "     |  JSONEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)\n",
      "     |  \n",
      "     |  Extensible JSON <https://json.org> encoder for Python data structures.\n",
      "     |  \n",
      "     |  Supports the following objects and types by default:\n",
      "     |  \n",
      "     |  +-------------------+---------------+\n",
      "     |  | Python            | JSON          |\n",
      "     |  +===================+===============+\n",
      "     |  | dict              | object        |\n",
      "     |  +-------------------+---------------+\n",
      "     |  | list, tuple       | array         |\n",
      "     |  +-------------------+---------------+\n",
      "     |  | str               | string        |\n",
      "     |  +-------------------+---------------+\n",
      "     |  | int, float        | number        |\n",
      "     |  +-------------------+---------------+\n",
      "     |  | True              | true          |\n",
      "     |  +-------------------+---------------+\n",
      "     |  | False             | false         |\n",
      "     |  +-------------------+---------------+\n",
      "     |  | None              | null          |\n",
      "     |  +-------------------+---------------+\n",
      "     |  \n",
      "     |  To extend this to recognize other objects, subclass and implement a\n",
      "     |  ``.default()`` method with another method that returns a serializable\n",
      "     |  object for ``o`` if possible, otherwise it should call the superclass\n",
      "     |  implementation (to raise ``TypeError``).\n",
      "     |  \n",
      "     |  Methods defined here:\n",
      "     |  \n",
      "     |  __init__(self, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)\n",
      "     |      Constructor for JSONEncoder, with sensible defaults.\n",
      "     |      \n",
      "     |      If skipkeys is false, then it is a TypeError to attempt\n",
      "     |      encoding of keys that are not str, int, float or None.  If\n",
      "     |      skipkeys is True, such items are simply skipped.\n",
      "     |      \n",
      "     |      If ensure_ascii is true, the output is guaranteed to be str\n",
      "     |      objects with all incoming non-ASCII characters escaped.  If\n",
      "     |      ensure_ascii is false, the output can contain non-ASCII characters.\n",
      "     |      \n",
      "     |      If check_circular is true, then lists, dicts, and custom encoded\n",
      "     |      objects will be checked for circular references during encoding to\n",
      "     |      prevent an infinite recursion (which would cause an RecursionError).\n",
      "     |      Otherwise, no such check takes place.\n",
      "     |      \n",
      "     |      If allow_nan is true, then NaN, Infinity, and -Infinity will be\n",
      "     |      encoded as such.  This behavior is not JSON specification compliant,\n",
      "     |      but is consistent with most JavaScript based encoders and decoders.\n",
      "     |      Otherwise, it will be a ValueError to encode such floats.\n",
      "     |      \n",
      "     |      If sort_keys is true, then the output of dictionaries will be\n",
      "     |      sorted by key; this is useful for regression tests to ensure\n",
      "     |      that JSON serializations can be compared on a day-to-day basis.\n",
      "     |      \n",
      "     |      If indent is a non-negative integer, then JSON array\n",
      "     |      elements and object members will be pretty-printed with that\n",
      "     |      indent level.  An indent level of 0 will only insert newlines.\n",
      "     |      None is the most compact representation.\n",
      "     |      \n",
      "     |      If specified, separators should be an (item_separator, key_separator)\n",
      "     |      tuple.  The default is (', ', ': ') if *indent* is ``None`` and\n",
      "     |      (',', ': ') otherwise.  To get the most compact JSON representation,\n",
      "     |      you should specify (',', ':') to eliminate whitespace.\n",
      "     |      \n",
      "     |      If specified, default is a function that gets called for objects\n",
      "     |      that can't otherwise be serialized.  It should return a JSON encodable\n",
      "     |      version of the object or raise a ``TypeError``.\n",
      "     |  \n",
      "     |  default(self, o)\n",
      "     |      Implement this method in a subclass such that it returns\n",
      "     |      a serializable object for ``o``, or calls the base implementation\n",
      "     |      (to raise a ``TypeError``).\n",
      "     |      \n",
      "     |      For example, to support arbitrary iterators, you could\n",
      "     |      implement default like this::\n",
      "     |      \n",
      "     |          def default(self, o):\n",
      "     |              try:\n",
      "     |                  iterable = iter(o)\n",
      "     |              except TypeError:\n",
      "     |                  pass\n",
      "     |              else:\n",
      "     |                  return list(iterable)\n",
      "     |              # Let the base class default method raise the TypeError\n",
      "     |              return JSONEncoder.default(self, o)\n",
      "     |  \n",
      "     |  encode(self, o)\n",
      "     |      Return a JSON string representation of a Python data structure.\n",
      "     |      \n",
      "     |      >>> from json.encoder import JSONEncoder\n",
      "     |      >>> JSONEncoder().encode({\"foo\": [\"bar\", \"baz\"]})\n",
      "     |      '{\"foo\": [\"bar\", \"baz\"]}'\n",
      "     |  \n",
      "     |  iterencode(self, o, _one_shot=False)\n",
      "     |      Encode the given object and yield each string\n",
      "     |      representation as available.\n",
      "     |      \n",
      "     |      For example::\n",
      "     |      \n",
      "     |          for chunk in JSONEncoder().iterencode(bigobject):\n",
      "     |              mysocket.write(chunk)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data descriptors defined here:\n",
      "     |  \n",
      "     |  __dict__\n",
      "     |      dictionary for instance variables (if defined)\n",
      "     |  \n",
      "     |  __weakref__\n",
      "     |      list of weak references to the object (if defined)\n",
      "     |  \n",
      "     |  ----------------------------------------------------------------------\n",
      "     |  Data and other attributes defined here:\n",
      "     |  \n",
      "     |  item_separator = ', '\n",
      "     |  \n",
      "     |  key_separator = ': '\n",
      "\n",
      "FUNCTIONS\n",
      "    dump(obj, fp, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw)\n",
      "        Serialize ``obj`` as a JSON formatted stream to ``fp`` (a\n",
      "        ``.write()``-supporting file-like object).\n",
      "        \n",
      "        If ``skipkeys`` is true then ``dict`` keys that are not basic types\n",
      "        (``str``, ``int``, ``float``, ``bool``, ``None``) will be skipped\n",
      "        instead of raising a ``TypeError``.\n",
      "        \n",
      "        If ``ensure_ascii`` is false, then the strings written to ``fp`` can\n",
      "        contain non-ASCII characters if they appear in strings contained in\n",
      "        ``obj``. Otherwise, all such characters are escaped in JSON strings.\n",
      "        \n",
      "        If ``check_circular`` is false, then the circular reference check\n",
      "        for container types will be skipped and a circular reference will\n",
      "        result in an ``RecursionError`` (or worse).\n",
      "        \n",
      "        If ``allow_nan`` is false, then it will be a ``ValueError`` to\n",
      "        serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``)\n",
      "        in strict compliance of the JSON specification, instead of using the\n",
      "        JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).\n",
      "        \n",
      "        If ``indent`` is a non-negative integer, then JSON array elements and\n",
      "        object members will be pretty-printed with that indent level. An indent\n",
      "        level of 0 will only insert newlines. ``None`` is the most compact\n",
      "        representation.\n",
      "        \n",
      "        If specified, ``separators`` should be an ``(item_separator, key_separator)``\n",
      "        tuple.  The default is ``(', ', ': ')`` if *indent* is ``None`` and\n",
      "        ``(',', ': ')`` otherwise.  To get the most compact JSON representation,\n",
      "        you should specify ``(',', ':')`` to eliminate whitespace.\n",
      "        \n",
      "        ``default(obj)`` is a function that should return a serializable version\n",
      "        of obj or raise TypeError. The default simply raises TypeError.\n",
      "        \n",
      "        If *sort_keys* is true (default: ``False``), then the output of\n",
      "        dictionaries will be sorted by key.\n",
      "        \n",
      "        To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the\n",
      "        ``.default()`` method to serialize additional types), specify it with\n",
      "        the ``cls`` kwarg; otherwise ``JSONEncoder`` is used.\n",
      "    \n",
      "    dumps(obj, *, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, cls=None, indent=None, separators=None, default=None, sort_keys=False, **kw)\n",
      "        Serialize ``obj`` to a JSON formatted ``str``.\n",
      "        \n",
      "        If ``skipkeys`` is true then ``dict`` keys that are not basic types\n",
      "        (``str``, ``int``, ``float``, ``bool``, ``None``) will be skipped\n",
      "        instead of raising a ``TypeError``.\n",
      "        \n",
      "        If ``ensure_ascii`` is false, then the return value can contain non-ASCII\n",
      "        characters if they appear in strings contained in ``obj``. Otherwise, all\n",
      "        such characters are escaped in JSON strings.\n",
      "        \n",
      "        If ``check_circular`` is false, then the circular reference check\n",
      "        for container types will be skipped and a circular reference will\n",
      "        result in an ``RecursionError`` (or worse).\n",
      "        \n",
      "        If ``allow_nan`` is false, then it will be a ``ValueError`` to\n",
      "        serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``) in\n",
      "        strict compliance of the JSON specification, instead of using the\n",
      "        JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).\n",
      "        \n",
      "        If ``indent`` is a non-negative integer, then JSON array elements and\n",
      "        object members will be pretty-printed with that indent level. An indent\n",
      "        level of 0 will only insert newlines. ``None`` is the most compact\n",
      "        representation.\n",
      "        \n",
      "        If specified, ``separators`` should be an ``(item_separator, key_separator)``\n",
      "        tuple.  The default is ``(', ', ': ')`` if *indent* is ``None`` and\n",
      "        ``(',', ': ')`` otherwise.  To get the most compact JSON representation,\n",
      "        you should specify ``(',', ':')`` to eliminate whitespace.\n",
      "        \n",
      "        ``default(obj)`` is a function that should return a serializable version\n",
      "        of obj or raise TypeError. The default simply raises TypeError.\n",
      "        \n",
      "        If *sort_keys* is true (default: ``False``), then the output of\n",
      "        dictionaries will be sorted by key.\n",
      "        \n",
      "        To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the\n",
      "        ``.default()`` method to serialize additional types), specify it with\n",
      "        the ``cls`` kwarg; otherwise ``JSONEncoder`` is used.\n",
      "    \n",
      "    load(fp, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)\n",
      "        Deserialize ``fp`` (a ``.read()``-supporting file-like object containing\n",
      "        a JSON document) to a Python object.\n",
      "        \n",
      "        ``object_hook`` is an optional function that will be called with the\n",
      "        result of any object literal decode (a ``dict``). The return value of\n",
      "        ``object_hook`` will be used instead of the ``dict``. This feature\n",
      "        can be used to implement custom decoders (e.g. JSON-RPC class hinting).\n",
      "        \n",
      "        ``object_pairs_hook`` is an optional function that will be called with the\n",
      "        result of any object literal decoded with an ordered list of pairs.  The\n",
      "        return value of ``object_pairs_hook`` will be used instead of the ``dict``.\n",
      "        This feature can be used to implement custom decoders.  If ``object_hook``\n",
      "        is also defined, the ``object_pairs_hook`` takes priority.\n",
      "        \n",
      "        To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``\n",
      "        kwarg; otherwise ``JSONDecoder`` is used.\n",
      "    \n",
      "    loads(s, *, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)\n",
      "        Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` instance\n",
      "        containing a JSON document) to a Python object.\n",
      "        \n",
      "        ``object_hook`` is an optional function that will be called with the\n",
      "        result of any object literal decode (a ``dict``). The return value of\n",
      "        ``object_hook`` will be used instead of the ``dict``. This feature\n",
      "        can be used to implement custom decoders (e.g. JSON-RPC class hinting).\n",
      "        \n",
      "        ``object_pairs_hook`` is an optional function that will be called with the\n",
      "        result of any object literal decoded with an ordered list of pairs.  The\n",
      "        return value of ``object_pairs_hook`` will be used instead of the ``dict``.\n",
      "        This feature can be used to implement custom decoders.  If ``object_hook``\n",
      "        is also defined, the ``object_pairs_hook`` takes priority.\n",
      "        \n",
      "        ``parse_float``, if specified, will be called with the string\n",
      "        of every JSON float to be decoded. By default this is equivalent to\n",
      "        float(num_str). This can be used to use another datatype or parser\n",
      "        for JSON floats (e.g. decimal.Decimal).\n",
      "        \n",
      "        ``parse_int``, if specified, will be called with the string\n",
      "        of every JSON int to be decoded. By default this is equivalent to\n",
      "        int(num_str). This can be used to use another datatype or parser\n",
      "        for JSON integers (e.g. float).\n",
      "        \n",
      "        ``parse_constant``, if specified, will be called with one of the\n",
      "        following strings: -Infinity, Infinity, NaN.\n",
      "        This can be used to raise an exception if invalid JSON numbers\n",
      "        are encountered.\n",
      "        \n",
      "        To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``\n",
      "        kwarg; otherwise ``JSONDecoder`` is used.\n",
      "\n",
      "DATA\n",
      "    __all__ = ['dump', 'dumps', 'load', 'loads', 'JSONDecoder', 'JSONDecod...\n",
      "\n",
      "VERSION\n",
      "    2.0.9\n",
      "\n",
      "AUTHOR\n",
      "    Bob Ippolito <bob@redivi.com>\n",
      "\n",
      "FILE\n",
      "    /opt/anaconda3/envs/py/lib/python3.10/json/__init__.py\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(json)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "bd67991c",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Contact:\n",
    "    def __init__(self, first, last):\n",
    "        self.first = first\n",
    "        self.last = last\n",
    "        \n",
    "    @property\n",
    "    def full_name(self):\n",
    "        return f'{self.first} {self.last}'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "82eb6334",
   "metadata": {},
   "outputs": [],
   "source": [
    "c = Contact(\"John\", \"Smith\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "94a82295",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'John Smith'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c.full_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "7fe5fd4b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\"first\": \"John\", \"last\": \"Smith\"}'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.dumps(c.__dict__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "64d7a93a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# better approach\n",
    "from typing import Any\n",
    "import json\n",
    "\n",
    "class ContactEncoder(json.JSONEncoder):\n",
    "    def default(self, obj: Any) -> Any:\n",
    "        if isinstance(obj, Contact):\n",
    "            return {\n",
    "                \"__class__\": \"Contact\",\n",
    "                \"first\": obj.first,\n",
    "                \"last\": obj.last,\n",
    "                \"full_name\": obj.full_name,\n",
    "            }\n",
    "        return super().default(obj)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "02b39dd6",
   "metadata": {},
   "outputs": [],
   "source": [
    "c = Contact(\"John\", \"Smith\")\n",
    "text = json.dumps(c, cls=ContactEncoder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "7cef9a94",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\"__class__\": \"Contact\", \"first\": \"John\", \"last\": \"Smith\", \"full_name\": \"John Smith\"}'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "5e5db026",
   "metadata": {},
   "outputs": [],
   "source": [
    "def decode_contact(json_object: Any) -> Any:\n",
    "    if json_object.get(\"__class__\") == \"Contact\":\n",
    "        return Contact(json_object[\"first\"], json_object[\"last\"])\n",
    "    else:\n",
    "        return json_object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "ff310759",
   "metadata": {},
   "outputs": [],
   "source": [
    "c2 = json.loads(text, object_hook=decode_contact)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "c62d0320",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'John Smith'"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c2.full_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "9567d312",
   "metadata": {},
   "outputs": [],
   "source": [
    "some_text = ('{\"__class__\": \"Contact\", \"first\": \"Milli\", \"last\": \"Dale\", '\n",
    "            '\"full_name\": \"Milli Dale\"}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "bdafc26a",
   "metadata": {},
   "outputs": [],
   "source": [
    "c3 = json.loads(some_text, object_hook=decode_contact)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "013cbdc2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Milli Dale'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "c3.full_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "33358b5f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on Contact in module __main__ object:\n",
      "\n",
      "class Contact(builtins.object)\n",
      " |  Contact(first, last)\n",
      " |  \n",
      " |  Methods defined here:\n",
      " |  \n",
      " |  __init__(self, first, last)\n",
      " |      Initialize self.  See help(type(self)) for accurate signature.\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Readonly properties defined here:\n",
      " |  \n",
      " |  full_name\n",
      " |  \n",
      " |  ----------------------------------------------------------------------\n",
      " |  Data descriptors defined here:\n",
      " |  \n",
      " |  __dict__\n",
      " |      dictionary for instance variables (if defined)\n",
      " |  \n",
      " |  __weakref__\n",
      " |      list of weak references to the object (if defined)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(c3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb7ec999",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}