RichText output in REST API responses

[sixers]

REST API currently outputs a Slate-compatible representation for rich text fields.

Example output:

{
    "description":
    [
        {
            "children":
            [
                {
                    "text": "asdasd"
                }
            ],
            "type": "h1"
        },
        {
            "children":
            [
                {
                    "text": "asd",
                    "bold": true
                }
            ],
            "type": "p"
        },
        {
            "children":
            [
                {
                    "text": "asd",
                    "italic": true
                }
            ],
            "type": "p"
        }
    ]
}

It also expects the input to be formatted as Slate-compatible JSON. It doesn't complain when it's not, but the Admin dashboard crashes if what's stored is not a correct slate representation. I haven't checked Local and GraphQL APIs, but I assume they work the same way.

Is there a specific reason to use this representation in API? I understand that Payload needs to keep this representation internally, but using it also for public APIs makes them much less convenient, and more difficult to integrate with other systems, which usually output HTML.

In my case specifically, I can't easily import data from WooCommerce.
It would be great if rich text fields could simply accept HTML and convert it to Slate representation internally. Same with output.

The documentation mentions how to generate HTML from Slate representation, could you also provide some guide for the reversed conversion?

[jmikrut]

Hey @sixers — great questions!

In the long run, having content formatted like this is a huge win. In our own projects, this means we get to map different types of elements and "marks" to different React components, which would be harder to do if the string was HTML.

Another reason why we store RichText data as JSON is that it makes your content useful in places outside of straight HTML—like native apps and other end use cases where HTML is irrelevant.

If you wanted to store HTML in your database, but still have a rich text editor-style UX in the admin panel, you could just use a field with type: 'textarea', and then bring in a custom admin field component using a third-party HTML editor like TinyMCE or similar.

We have considered supporting an HTML editor in the future, but we really think that storing rich text as JSON is the better solution over the long run, even if it introduces a bit more complexity.

We have not yet needed to take HTML and convert it into Slate-compatible JSON, although there are examples in Slate's documentation that could be helpful to you. Take a look at this:

https://github.com/ianstormtaylor/slate/blob/main/site/examples/paste-html.tsx#L115

This example shows how to do it when a user "pastes" HTML into the RichText editor, although you could use the same principles.

On another note, we will be improving how the rich text field handles improperly formatted data in the future. It would be ideal if only the field in question errored, while the rest of the admin panel still functioning properly.

What do you think?

[sixers]

@jmikrut Thanks! That makes a lot of sense.

Still, I think it would be nice to be able to support a "simple" use case, where you don't care about the internals of the rich text field, and just want to be able to display it easily (or import it from some other system).

The only thing I care about is external representation, so Slate is perfectly fine as editor as long as the API can accept HTML string as input and produce HTML output.

Perhaps RichText field could support serializer / deserializer options which could be used to configure external representation of the Rich Text field. By default, it could be an identity function (slate <=> slate), but could be changed to use a custom (de)serialization. Something like this:

const Products: CollectionConfig = {
  slug: "products",
  admin: {
    useAsTitle: "name",
  },
  fields: [
    {
      name: "description",
      label: "Description",
      type: "richText",
      externalRepresentation: {
        serializer: (content) => convertSlateToHTML(content),
        deserializer: (content) => convertHTMLToSlate(content),
      }
      required: true,
    }
  ]
}

So what's kept in Mongo is still a Slate JSON, Slate editor is still used in the admin panel, the only thing that changes is the public API representation of the field.

WDYT?

[Ontopic]

Nice idea! Can help with keeping the layout / styles within the admin and frontend in sync when adding custom elements as well better.

[DanRibbens]

I can understand the appeal of having a simple HTML field. Though another idea would be to allow the requester to have the option of how to format richText as a query param, could that work? The normal response would default to the slate structure, but any frontends that just want HTML could specificy that in the request. That would be more versatile in cases where you required differing formats across different apps.
You'd still have to have an import that can "serialize" content from HTML to slatejs, but that could also be something we make a package to do just that.
Thoughts?

[Ontopic]

It's hard to explain to writers why things look different in the CMS than on the site. The contenteditable-html-WYSIWYG-magic is kind of what they got used to over the years. Not something I want to get back to - at all - yet I need to give them some more "visual cues".

For this reason I eated my "own" Slate field, with https://slate-plugins.udecode.io/ as the editor. It still uses the Payload (Slate) document structure, so I store it as a Payload richText field and communicate with the Payload hooks (getting all the perks of the regular RichtText field). Works quite nicely. I just thought @sixers suggestion of replacing the serializer (so you can kinda use the same one as in the frontend) could be a nice clean solution to get closer to a WYSIWYG feel without hacking over all the layers like I kinda did 🙃

[jmikrut]

Chiming in here - this may all be possible to do for sure. I was initially going to suggest that this could all be possible at the moment with hooks:

• beforeChange : take HTML, create Slate JSON
• afterRead: take JSON, serialize HTML

But the catch there is that we need some way to identify that we need to deserialize HTML on the admin panel, specifically. Because if you use an afterRead hook, then the admin panel will be fed HTML, and the panel itself needs to be more clever than that. The admin panel would always need to receive Slate JSON for the Slate editor to work.

It's possible that the admin panel could do some magic in an afterRead hook that recognizes that the req is coming from the Payload admin panel, and thus does not serialize HTML in those cases...

This might be fully possible with hooks and could even be packaged up nicely as a Payload plugin.

Thoughts?