Using cats-parse with Scala string interpolation

[michel-steuwer]

How would I best use cats-parse with Scalas string interpolation feature, where the input I am parsing is not just a plain string but also arbitrary values that are interpolated in it?

So how would I best write a parser for something like:

json"{ name: $name, id: $id }"

from https://docs.scala-lang.org/overviews/core/string-interpolation.html#advanced-usage

[johnynek]

If I were parsing this, I would do something like:

// parse expressions like names, `{1 + 1}` etc...
val pexpr: Parser[Expr] = ...

sealed trait Element
case class ExprEl(toExpr: Expr) extends Element
case class StringEl(toStr: String) extends Element

val pstr: Parser[List[Element]] = {
  import cats.parse.{Parser => P}
  val notDollar = P.not(P.charIn('\\' :: '"' :: Nil)).with1 *> ((P.char('\\') *> P.anyChar) | P.anyChar)
  
  val strEl: P[StringEl] = notDollar.repAs[String].map(StringEl(_))  

  val expr: P[ExprEl] = P.char('$') *> pexpr.map(ExprEl(_)) 
  
  P.string("json") *> (strEl | expr).rep <* P.char('"')
}

Something like that...

Does that make sense (warning, I didn't compile this, I just wrote it in the comment).

[michel-steuwer]

Sorry, I think I didn't make my question clear enough.
I don't want to parse json"{ name: $name, id: $id }" directly.
I want to use cats-parse to implement a parser to be used for an implementation of the Scala string interpolation feature.

At https://docs.scala-lang.org/overviews/core/string-interpolation.html#advanced-usage
they give a simple example implementation for a string interpolation json parser:

implicit class JsonHelper(val sc: StringContext) extends AnyVal {
  def json(args: Any*): JSONObject = {
    val strings = sc.parts.iterator
    val expressions = args.iterator
    var buf = new StringBuilder(strings.next())
    while(strings.hasNext) {
      buf.append(expressions.next())
      buf.append(strings.next())
    }
    parseJson(buf)
  }
}

The challenge here is that we don't have a single input string to parse, but two iterators: one for the string snippets, and one for the elements that separate these snippets.
So for

json"{ name: $name, id: $id }"

The sc.parts.iterator from above will return: "{ name: " and then ", id: " and finally " }" and the args.iterator will return the values of a variable name and id that are in the scala scope (as values of type Any).

How do I use cats-parse (or any other Scala combinator library) with this string / Any value iterator interfact?

[johnynek]

I really don't know how this library might relate to what you are proposing.

I mean, you might want to parse the result of that string? so, are you talking about how to implement parseJson? I'd probably say it is better to use a library like circe or jawn for that.

[ritschwumm]

@johnynek well, all this is about parsing a (slightly fancy) json string into a data structure, isn't it?

@michel-steuwer i think the "slightly fancy" will be a problem here:

what you get from StringContext is more structured than a plain string, and you'd want to make use of this additional structure.
you could for example convert the template into a Seq[Either[Char,Parameter]] and run a parser on this - but i don't think cat-parse supports parsing any other sequence of tokens than a String.

one way to attack that problem is concatenating the template literal strings, interspersed with some private unicode markers.
then you can have an almost normal json parser that just additionally expects these markers in certain places and returns the value associated to them in the template parameters when they occur.

[johnynek]

Yeah. We only support String as an input (since it is by far the most common case and it we can optimize it considerably).

If you want to see an example parser for a structured type, see the JSON parser here:

https://github.com/typelevel/cats-parse/blob/main/bench/src/main/scala/cats/parse/bench/self.scala

Lastly, what I would do here is to just convert the whole interpolated argument to a string and then run the parser on that string.