Upgrade to latest remark 13

[vladar]

Major bump of all remark plugins is coming

Remark has had a significant major upgrade recently and changed the underlying parser. The ecosystem seems to have almost caught up since then. We also upgraded Gatsby remark plugins.

The following plugins are now fully compatible with remark@13:

• gatsby-transformer-remark@4.0.0
• gatsby-remark-autolink-headers@4.0.0
• gatsby-remark-code-repls@5.0.0
• gatsby-remark-copy-linked-files@4.0.0
• gatsby-remark-embed-snippet@6.0.0
• gatsby-remark-graphviz@3.0.0
• gatsby-remark-images-contentful@4.0.0
• gatsby-remark-images@5.0.0
• gatsby-remark-katex@5.0.0
• gatsby-remark-prismjs@5.0.0
• gatsby-remark-responsive-iframe@4.0.0
• gatsby-remark-smartypants@4.0.0

Original PR: #29678

Incompatible plugins

Unfortunately gatsby-remark-custom-blocks is not compatible with remark13 yet because the underlying remark-custom-blocks package is not updated yet: zestedesavoir/zmarkdown#416

One alternative is to use remark-directive. We don't provide a plugin for it yet, so if you're up to it - you can publish a new gatsby plugin for remark-directive support! Or use to-gatsby-remark-plugin to convert remark-directive to gatsby plugin.

Still, remark-directive has a different syntax than remark-custom-blocks, so it will require manual migration of your markdown.

Migration

We tried to make the upgrade effortless. But there could still be subtle differences in output and edge cases.

Most changes affect how lists are rendered. The new HTML output is more consistent in general. But in some cases, the new HTML structure may become inconsistent with your CSS selectors. So if you don't have screenshot-based tests for your projects we'd recommend checking the most important pages manually or by diffing the HTML of your builds.

The first build:

gatsby clean && gatsby build --no-uglify && cp -r public public-first-build

Then run the second build and diff:

gatsby build --no-uglify && diff -u -r public-first-build public

Known output changes

Make sure to check remark changelog for the full list of changes. Below we list some actual diffs we've seen while testing the new version.

1. List items separated by a newline are wrapped with paragraph

Take this markdown as an example:

- Foo.

- Bar.

- Baz

HTML before:

<ul>
  <li><strong>Foo.</strong></li>
  <li>Bar.</li>
  <li>Baz.</li>
</ul>

HTML after:

<ul>
  <li><p><strong>Foo.</strong></p></li>
  <li><p>Bar.</p></li>
  <li><p>Baz.</p></li>
</ul>

But if you have a CSS rule like this: li > strong { color: red } - it will break.

2. Nested lists are rendered without a wrapping paragraph (for tight lists)

- Foo.
- Bar:
  - Bar1
  - Bar2

Rendered as:

<ul>
  <li>Foo.</li>
  <li>
-  <p>Bar:</p>
+  Bar:
    <ul>
      <li>Bar1</li>
      <li>Bar2</li>
    </ul>
  </li>
</ul>

3. All list items are rendered consistently

If at least one list item contains a new line - every item in this list will be wrapped in a paragraph. The list is considered tight only if every list item occupies a single line. In this case, there will be no wrapping paragraphs.

In the previous version, you could get mixed markup in list items (e.g. one item wrapped in a paragraph, but not the other).

4. White-spaces are trimmed

-<p>foo. </p>
+<p>foo.</p>

5. <img alt="" /> attribute is always added with remark-images

- <img src="/foo/bar.gif"></a>
+ <img src="/foo/bar.gif" alt=""></a>

6. Emails in the text are replaced with links

See remarkjs/remark#475

-<strong>please email foo@example.com</strong>
+<strong>please email <a href="mailto:foo@example.com">foo@example.com</a></strong>

If you notice other changes like this, please let us know.

[KyleAMathews]

Tried them out on my blog with no apparent problems! KyleAMathews/blog#44 & https://blog88227-remarkalpha.gtsb.io/

[jpvajda]

Here is my PR for these changes, the site built locally and all tests pass ✅ newrelic/docs-website#1608

[vladar]

The issue with remark upgrade is that it may render slightly differently with the new version. So unless you have screenshot-based tests those changes may be unnoticed.

[jpvajda]

@vladar Ok, I'll check the site visually to look for any weirdness. Curious, what should I be looking for exactly?

[vladar]

If only I knew - then we wouldn't need to test this 😂 I've updated the description with one known example of how things can break.

[jpvajda]

@vladar I poked around and didn't see any anomalies, the preview link is here: feel free to take a tour: https://pr-1608.d1vx4w5vgyla4s.amplifyapp.com

[nikoladev]

I've made a bug report for the old MDX plugin and the issues it has with "complex" math formulas and what I did to work around it. I also made a repo with a minimal reproducible example for it.

At the moment I don't have time to test out if the new changes fix the bugs. Could someone clone my repo and try it out?

[nikoladev]

Update: as mentioned by @vladar in this thread, gatsby-plugin-mdx will only be updated once MDX V2 is released. Since that issue I linked to relies on gatsby-plugin-mdx, it is out of scope for this discussion.

[kimbaudi]

@nikoladev - I tried your complex math formula on a gatsby site that I am working on and it seems to render properly.

the complex math formula:

$$
\begin{aligned}
  \bar{x} &= \dfrac{0+1+2+ \ldots +8}{9} = 3.56 \\
  \bar{y} &= \dfrac{23.5+25+ \ldots +37.5}{9}=30
\end{aligned}
$$

screenshot of rendered complex formula (in dark mode):

However, I am using the latest npm packages (except for remark-math which needs to be on v3 until gatsby-plugin-mdx uses remark 13. Also, I am using rehype-katex instead of remark-html-katex (i believe rehype-katex is recommended over remark-html-katex. I didn't clone your repo, but just copy/pasted the complex math formula from your repo into my gatsby site to see there was a problem. Apparently, it seems to render correctly.

So in your gatsby-config.js, change:

    {
      resolve: 'gatsby-plugin-mdx',
      options: {
        defaultLayouts: {
          default: require.resolve(path.resolve(__dirname, 'src', 'templates', 'Page.jsx')),
        },
        remarkPlugins: [
          require('remark-math'),
          require('remark-html-katex'),
        ],
      },
    },

to:

    {
      resolve: 'gatsby-plugin-mdx',
      options: {
        defaultLayouts: {
          default: require.resolve(path.resolve(__dirname, 'src', 'templates', 'Page.jsx')),
        },
        remarkPlugins: [require('remark-math')],
        rehypePlugins: [require('rehype-katex')],
      },
    },

And make sure to install rehype-katex (and remove remark-html-katex)

[nikoladev]

I'm repeating my comment from the other issue here, so that it's also documented in this discussion. But please continue the discussion in that issue so that we don't clutter up this discussion.

@kimbaudi Thanks for looking into it :)

I tried your fix, and it doesn't seem to work. You can try it out in this branch to see if I did something wrong.

Are you sure you removed the 3 backticks around the broken examples in index.mdx when testing out your changes? You can see what I mean by looking at the changes to index.mdx in this diff.

[nikoladev]

@vladar Shouldn't the remark-*, unist-*, and mdast-* dependencies in gatsby-plugin-mdx also be updated as part of this change?

[vladar]

Unfortunately, only MDX V2 (beta) supports remark13 - mdx-js/mdx#1440 (comment)

The current version of MDX still needs remark12. We will bump MDX plugin as soon as V2 is published (or if they decide to support remark13 in V1)

[kimbaudi]

Looking forward to when MDX v2 is published with support for remark 13.

I'm currently using gatsby-plugin-mdx with remark-math and rehype-katex:

    {
      resolve: `gatsby-plugin-mdx`,
      options: {
        remarkPlugins: [require(`remark-math`)],
        rehypePlugins: [require(`rehype-katex`)],
      },
    },

Since gatsby-plugin-mdx is using MDX v1 and remark 10, I'm unable to upgrade remark-math from v3.0.1 to v4.0.0 until gatsby-plugin-mdx uses the latest remark 13

gatsby/packages/gatsby-plugin-mdx/package.json

Line 48 in 4336d04

"remark": "^10.0.1",

[kimbaudi]

I created a bug report a few weeks back after upgrading to gatsby-remark-katex v5.0.0: #30865

adding here since it is related

[eerison]

Hey guys

why remark-custom-blocks (dependency of gatsby-remark-custom-blocks) is a blocker to support the last version of gatsby-transformer-remark?

as I can see remark is not a dependency of this package: https://github.com/zestedesavoir/zmarkdown/blob/zmarkdown%4011.2.2/packages/remark-custom-blocks/package.json#L31-L33

or am I missing something here?

Edit: or should we use remark-directive ?