Reported errors are just string messages

[julienrf]

The built-in Validated type only report errors as String messages, making errors hard to process by downstream tools.

We could enrich the error model with an optional abstract description of the error:

sealed trait Validated[+A]
case class Valid[A](value: A) extends Validated[A]
case class Invalid(errors: Seq[Error]) extends Validated[Nothing]

/**
  * @param message Human-readable message
  * @param model   Machine-readable error model
  */
case class Error(message: String, model: Option[Error.Model])
object Error {
  def fromMessage(message: String): Error = Error(message, None)
  /**
    * @param location A path that identifies the location of the error
    * @param key      A unique identifier for the error (e.g. `max`, `length`, `missing`, etc.)
    * @param args     Additional arguments for the error
    */
  case class Model(location: String, key: String, args: Seq[Any])
}

The proposed error model above is poorly typed. We could have something more precise (at the risk of breaking the ABI more often…).

[bmeesters]

Since the 1.0.0 tag is removed, is this pushed for the long-term, since it would break the API? Is there a possibility to make the Validated generic in its error type (i.e. like Cats Validated)? Maybe with a type class so you can at least extract a message.

[julienrf]

I’m still not sure how we want to model errors instead of just strings. Changing this would most likely break the backward binary compatibility (but maybe not…), so it would have to go in endpoints-algebra 2.x.

Do you think it’s urgent to have it in 1.x?

[bmeesters]

For us it would not be urgent, just more convenient. Maybe it is possible to create an experimental algebra post 1.0.0 that does not provide any backwards compatibility guarantees until 2.0.0? I just wonder if we can provide something like that without a lot of duplication with the current algebras.

[julienrf]

I would be interested in knowing more about your use case, maybe that will help us to find the solution 😄

[bmeesters]

Well, not any particular use case right now to be honest. Generally though I like to rely on strongly typed programming instead of stringly typed programming. But there is not big incentive to get something in now.

[harpocrates]

I have a particular use case, although probably no time to work on this in the coming days (maybe next week). I want to be able to represent errors in the API like Twitter or Github do - with some error code, messages, maybe some extra information too. In other words, I want custom errors to have lots of potentially nested structure.

AFAICT, the methods in Errors force ClientError and ServerError to be isomorphic to Invalid:

  /** Convert the ''endpoints'' internal client error type into the [[ClientErrors]] type
    * @group operations */
  def invalidToClientErrors(invalid: Invalid): ClientErrors

  /** Convert the [[ClientErrors]] type into the ''endpoints'' internal client error type
    * @group operations */
  def clientErrorsToInvalid(clientErrors: ClientErrors): Invalid

  /** Convert the ''endpoints'' internal server error type into the [[ServerError]] type
    * @group operations */
  def throwableToServerError(throwable: Throwable): ServerError

  /** Convert the [[ServerError]] type into the ''endpoints'' internal server error type
    * @group operations */
  def serverErrorToThrowable(serverError: ServerError): Throwable

Since Invalid is itself just Seq[String], even with endpoints custom errors I end up needing to convert my ADT that represents an error to/from some stringly-typed representation.

I'm not sure what should be changed: Invalid or Errors, but it seems oddly restrictive that my "custom" errors have to be convertible to/from a list of strings.

[julienrf]

I think there is everything we need to produce such responses. You can see an example of error responses that are probably similar to what you want to do: https://github.com/julienrf/endpoints-problemdetailserrors/ (and some explanation here)

The motivation for having the operation invalidToClientErrors(invalid: Invalid): ClientErrors is to be able to model errors produced by endpoints (e.g., when parsing query parameters or entities) into your ClientErrors type. This operation allows server interpreters to work with custom error models without having to be modified.

Conversely, the motivation for clientErrorsToInvalid(clientErrors: ClientErrors): Invalid is for client interpreters to be able to manipulate your custom ClientErrors type (by transforming it into Invalid). So, this transformation won’t affect the way your HTTP responses will be produced by server interpreters.

We could get rid of these operations, but then supporting custom models of errors would be a little bit more complicated: users would also have to customize their client and server interpreters to define what to do with these errors. We’d need to experiment with such a design first.

I still think there is everything you need in the current design, though.

[harpocrates]

Oh I think I see what you mean now. The functions are not to squeeze a ClientError into an Invalid, but rather to project an Invalid into a ClientError for a server and re-create just an Invalid for the client.

This makes sense - my use case is covered. Thanks for the explanation!

[bmeesters]

These links are very helpful. I might have been a bit too early with the stringly typed comment (sorry about that). I have currently used endpoints mostly with the happy path and default errors.

[bmeesters]

I am currently looking into this again a bit more. We are using the ujson parsers which could throw an exception if used directly:

sealed trait ParsingFailedException extends Exception

case class ParseException(clue: String, index: Int, line: Int, col: Int)
  extends Exception(clue + " at index " + index) with ParsingFailedException

case class IncompleteParseException(msg: String, cause: Throwable)
  extends Exception(msg, cause) with ParsingFailedException

However the error is caught and mapped to the message: Invalid JSON document text, which doesn't include anything hinting where the error might be.

Can be somehow retrieve if something goes wrong in parsing the JSON using the error of the underlying library instead of an Invalid?

edit: Could we consider using an abstract error type that is made concrete in the interpreters to just be the error model of the underlying library? That way we would never loose any information.

[julienrf]

Can be somehow retrieve if something goes wrong in parsing the JSON using the error of the underlying library instead of an Invalid?

edit: Could we consider using an abstract error type that is made concrete in the interpreters to just be the error model of the underlying library? That way we would never loose any information.

We’d need to experiment with such solutions. The idea of using an abstract error type could work. Like we do in endpoints.algebra.Errors.