calling Tree.fold raises strange error "exception Invalid_argument("error expected JSON text (JSON value)")"

[kentookura]

The following code from https://github.com/kentookura/ocaml-forester/blob/main/test/store.ml

open Lwt.Syntax
open Core

module Store = Irmin_git_unix.FS.KV(Rep.Tree)
module Info = Irmin_unix.Info(Store.Info)
module Sync = Irmin.Sync.Make(Store)
module Tree = Store.Tree

let config = Irmin_git.config ~bare:true "/tmp/irmin"

let main_branch config =
  let* repo = Store.Repo.v config in
  Store.main repo


let fold ~init ~f tree=
  Tree.fold ~order:`Sorted ~contents:(fun k _v acc -> if k = [] then Lwt.return acc else f k acc) tree init

let main () =
  let* t = main_branch config in
  let* tree = Store.tree t in
  let+ _ = fold ~init:[] ~f:(fun k acc -> Lwt.return (k::acc)) tree in 
  ()

let () = Lwt_main.run (main ())

raises the error

Fatal error: exception Invalid_argument("error expected JSON text (JSON value)")
Raised at Stdlib.invalid_arg in file "stdlib.ml", line 30, characters 20-45
Called from Irmin_git__Content_addressable.Make.find.(fun) in file "src/irmin-git/content_addressable.ml", line 46, characters 25-37
Called from Irmin__Tree.Make.Contents.value_of_key in file "src/irmin/tree.ml", line 329, characters 19-61
Called from Irmin__Tree.Make.Contents.fold in file "src/irmin/tree.ml", line 392, characters 19-36
Called from Irmin__Tree.Make.Node.fold.step.apply in file "src/irmin/tree.ml", line 1436, characters 14-73
Called from Lwt.Sequential_composition.bind.create_result_promise_and_callback_if_deferred.callback in file "src/core/lwt.ml", line 1844, characters 16-19
Re-raised at Lwt.Miscellaneous.poll in file "src/core/lwt.ml", line 3123, characters 20-29
Called from Lwt_main.run.run_loop in file "src/unix/lwt_main.ml", line 27, characters 10-20
Called from Lwt_main.run in file "src/unix/lwt_main.ml", line 106, characters 8-13
Re-raised at Lwt_main.run in file "src/unix/lwt_main.ml", line 112, characters 4-13
Called from Dune__exe__Store in file "test/store.ml", line 25, characters 9-31

I am quite baffled by this, is this a bug?

The perhaps confusingly named Rep.Tree is a custom content type defined in this file

The end goal is to write a function analogous to https://github.com/kentookura/ocaml-forester/blob/main/bin/forester/Process.ml which takes a git remote as an argument.

[art-w]

(My apologies for the late reply, I was on holidays!)

This is a bit hard to reproduce on my end, do you have instructions for how /tmp/irmin was created?

The error seems to be coming from your Rep.Tree encoding/decoding functions rather than from Irmin itself (but Irmin or Repr could do a better job at explaining the issue). Tree.fold needs to decode the values and it seems that the decoding function can't read what was encoded on disk. The Jsonm error is because Repr chooses this as the default encoding for all the record/variant that you defined... but that should be fine, however your Asai range definition looks suspicious! :)

Could you try recreating your /tmp/irmin repo (which is likely wrongly encoded?) and rerunning your Tree.fold test with the following definitions for Asai ranges in lib/core/Rep.ml?

(* dune
(library
 (name Core)
 (preprocess (pps ppx_deriving.show ppx_irmin))
                                    ^^^^^^^^^ add this
 ...  *)

(** Reps of Asai types *)

type string_source = Asai.Range.string_source = {
  title : string option;
  content : string;
}
[@@deriving irmin]

type source = [ `File of string | `String of string_source ] [@@deriving irmin]

type position = Asai.Range.position = {
  source : source;
  offset : int;
  start_of_line : int;
  line_num : int;
}
[@@deriving irmin]

type view = [ `Range of position * position | `End_of_file of position ]
[@@deriving irmin]

let range : Range.t ty =
  Repr.map view_t
    (function
      | `Range range -> Asai.Range.make range
      | `End_of_file pos -> Asai.Range.eof pos)
    Asai.Range.view

(This might not the full fix as I'm not exactly sure what went wrong! Happy to take a deeper look if I can reproduce your /tmp/irmin :) )

[kentookura]

No worries :)

The test succeeds after I deleted the git repository. I suppose the error arose because the repo persisted while I was messing around with the representations, so an encoding error makes sense.

Thanks for your tips re. the asai types!