Error when importing entries with Decimal metadata

[kpine]

I have a CSV importer that adds Decimal metadata to entries, e.g. Decimal(114) which renders to foo: 114. When I attempt to import a file with these transactions, an error is raised. The transactions are accepted by beancount when using the importer directly (Numbers (Decimal) are a valid metadata type), but fails when importing via fava.

Exception on /beancount/api/add_entries [PUT]
Traceback (most recent call last):
  File "/app/lib/python3.11/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/fava/json_api.py", line 152, in _wrapper
    res = func(*validator(data))
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/fava/json_api.py", line 349, in put_add_entries
    g.ledger.file.insert_entries(entries)
  File "/app/lib/python3.11/site-packages/fava/core/file.py", line 171, in insert_entries
    fava_options.insert_entry = insert_entry(
                                ^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/fava/core/file.py", line 328, in insert_entry
    content = _format_entry(entry, currency_column, indent)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/fava/core/file.py", line 360, in _format_entry
    string = align(format_entry(entry, prefix=" " * indent), currency_column)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/beancount/parser/printer.py", line 369, in format_entry
    return EntryPrinter(dcontext, render_weights, prefix=prefix)(entry)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/lib/python3.11/site-packages/beancount/parser/printer.py", line 123, in __call__
    method(obj, oss)
  File "/app/lib/python3.11/site-packages/beancount/parser/printer.py", line 182, in Transaction
    self.write_metadata(entry.meta, oss)
  File "/app/lib/python3.11/site-packages/beancount/parser/printer.py", line 159, in write_metadata
    raise ValueError("Unexpected value: '{!r}'".format(value))
ValueError: Unexpected value: '114'

If I manually add a transaction through the UI, with the same metadata, it looks like fava converts it to a string, as the ledger file contains foo: "114" instead. I suppose numbers that come through the API are not converted any valid type.

I can update my importer to use a string, which will solve it, but could fava either convert to string or Decimal in this case?

[yagebu]

Yes, like with #980, Fava does not really support all kinds of metadata like Decimals or Balances to be round-tripped through the import interface. A PR fixing this would be welcome :)

[chen-chao]

I also hit this bug. Then I found the type mismatch between fava and beancount. The basic flow of fava importer is like:

1. Fava backend calls importer to extract entries.
2. The entries are serialized as JSON and sent back to the front end.
3. User modifies the entries in the frontend page.
4. User sends the entries as JSON back to the Fava backend.

So the original data type given by the importer is lost during the JSON serialization and deserialization.

Usually it's not an issue as beancount tries best at handling all kinds of data. But in this Decimal metadata case, the Decimal is serialized into JSON and finally deserialized as int or float, and beancount has a strict type check. Hence the error.