Serializer failing on schemas with a composite type, i.e. Money.Ecto.Composite.Type

[frahugo]

Reporting an issue with the way the serialization is performed in version 1.0.0.

Our application uses ex_money_sql for a money composite type in some of our Ecto schema. For instance:

defmodule Account do
  use Ecto.Schema

  schema "accounts" do
    field :balance, Money.Ecto.Composite.Type
  end
end

According to ex_money_sql documentation, if we were to use the money field in an embedded schema, we would use another type:

defmodule Account do
  use Ecto.Schema

  embedded_schema do
    field :balance, Money.Ecto.Map.Type
  end
end

We are now getting this error when we use PaperTrail.insert with an Account changeset, i.e.

params
|> Account.create()
|> PaperTrail.insert()

** (Protocol.UndefinedError) protocol Jason.Encoder not implemented for {"CAD", Decimal.new("1500")} of type Tuple, Jason.Encoder protocol must always be explicitly implemented. This protocol is implemented for the following type(s): Any, Atom, BitString, Cldr.LanguageTag, Date, DateTime, Decimal, Ecto.Association.NotLoaded, Ecto.Schema.Metadata, Float, ...

As we can see, Jason tries to encode a tuple that was meant to be sent to Postgresql (composite type).

This is caused by the use of Ecto.embedded_dump/2 in PaperTrail.Serializer.

According to the Ecto documentation, the embedded_dump function should be given an embedded schema:

Dumps the given struct defined by an embedded schema.
This converts the given embedded schema to a map to be serialized with the given format.

As a workaround, we had to implement a hack:

defimpl Jason.Encoder, for: Tuple do
  def encode({<<currency::binary-size(3)>>, %Decimal{} = amount}, _opts) do
    %{
      amount: amount,
      currency: currency
    }
    |> Jason.encode!()
  end

  def encode(tuple, _opts) do
    raise "Unsupported Tuple serialization (#{inspect(tuple)})"
  end
end

So is there another way to generate the json data to be stored in Version?

[frahugo]

@kipcole9 (Kip, just CCing you so you are aware of this issue)

[kipcole9]

Thanks for the ping @frahugo. This is a bit tricky since it's not possible to round-trip json encoding of a schema. But thats ok in this case since papertrail's use case doesn't include decoding it.

Possible resolution

I have pushed a commit that will detect if Money.Ecto.Composite.Type.dump/3 is being called by Ecto.embedded_dump/2. If it is, it returns a map instead of the usual tuple.

Please try the papertrail branch by configuring:

{:ex_money_sql, github: "kipcole9/money_sql", branch: "papertrail"}

In mix.exs.

It's a bit hacky, but it's all public APIs. There will only be an issue if Ecto somehow changes its public API - and thats not very likely.

Please open an issue on the money_sql repo if you see any problems. If it tests cleanly then I'll publish a new version.

Note this is not a general purpose solution to the problem. Not all Postgres types can be serialised to JSON (without a custom encoder) and therefore it should be expected that other issues arise from time-to-time.

Summary

The approach you are taking in your snippet is the same as what the Postgres ROW_TO_JSON does so I think its the right approach, given that there is no JSON serialisation format for a tuple.

The tricky bit, as you point out, is that Ecto.embedded_dump/2 may not be the right tool for the dumping a schema (as apposed to dumping an embedded schema). There is no guarantee at all that all database types can be serialised to JSON since database types can have quite arbitrary shapes if you use CREATE TYPE. Composite types, range types, and possibly enumerated types won't have a JSON encoding.

One other constraint is that the Money.Ecto.Composite.Type is configured with embed_as: :dump. This is because Ecto.embedded_load/2 will cast the data if :embed_as is configured for the type as :self. Since we can't tell if the cast/2 call is coming from user input (and is therefore locale aware) or from serialisation (like JSON, where the data is not locale aware), we can't use :self.

[frahugo]

@kipcole9 Your proposed solution works great. If you generate a new version of ex_money, we'll use it ;-)

Thanks a lot

[kipcole9]

@frahugo I have published ex_money_sql version 1.11.0 with the following changelog entry:

Enhancements

• When dumping a Money.Ecto.Composite.Type, detect if dump/3 is being called by Ecto.Type.embedded_dump/2. If it is, then return a map that can be serialized to JSON. If it isn't (most cases) then return the tuple to be serialized to Postgres. See papertrail issue.

Note this enhancement is a workaround for papertrail - it does not fix the underlying issue which is that not all database types can be serialised to JSON. I will send a documentation PR to the papertrail repo.

[kipcole9]

See pull request 231.