Support sub-routes

[blittle]

When transitioning between pages in Hydrogen, an RSC request is sent to the server. Depending on what data requests are necessary, the transition can be noticeably slow while waiting for the RSC response. If Hydrogen supported sub-routes, on each RSC request, we should be able to determine what requests are necessary to re-run.

Consider a simplified PDP page:

When this page is directly loaded (or hard refreshed), data needs to be fetched for the header as well as the product information. If the user clicks the link for "Freestyle Collection", currently the RSC request will re-fetch everything for a new page render. We can 1) cache those request results and 2) try to optimize the RSC response to the browser. But it would be ideal if the RSC endpoint can be smart and know that the header did not change at all. This would be possible if the product details are in a sub-route, and as long as the destination route is a sibling sub-route.

Questions

1. The server is stateless between requests. Everything is "refetched" to potentially setup parent contexts above the sub-route. The client could send up state for each RSC request, but will that be sufficient to render all sub-routes?
2. How do we render a sub-route on the server independent of the default App tree?
3. How do we inject into the client sub-route RSC response?
   a. We would need to make sure the RSC response supports both nested and base route changes
4. An alternative to sub-routes might be providing an "app shell" outside the router that doesn't get re-rendered on each RSC request. So if you want global data for the app that is fetched once, put it there. This might be a good approach because we can put restrictions on what data can be provided (use server context under the hood) and we serialize that data back and forth.

[jplhomer]

@wizardlyhel came up with a brilliant idea for sub-routes, partial refetching in RSC, and an option for continuing to use the new Remixed React Router (WIP). Here's my attempt at writing it down!

tl;dr - lean into Remix's pattern of "layout routes" to create multiple server component entrypoints for a Hydrogen app

Here's how this could work in Hydrogen:

Components

src
├── App.server.jsx # equivalent to Remix's `root`
└── routes
    ├── product.server.jsx
    ├── collection.server.jsx

Pretty much the same as today, except App.server is a layout which is persisted in the client (more on that later), and each individual route is rendered as a separate tree based on the incoming RSC request.

If devs have different "areas" of a site, they could even provide other layouts:

src
├── App.server.jsx # equivalent to Remix's `root`
└── routes
    ├── product.server.jsx
    ├── collection.server.jsx
    ├── __featured.server.jsx
    ├── __featured/thing.server.jsx

Where __featured is a new layout component, etc.

To make this work, we convert App.server (and any other layout components) to render the equivalent of RR's <Outlet> component:

export default function App() {
  // Layout stuff here. Fetch data if you want!
  return (
    <Layout>
      <Outlet />
    </Layout>
  );
}

Next, as part of our dev/build process, we generate a manifest which contains a map of all the file routes and their relationships, including a parentId for each component (leading up to the root App.server):

You can inspect this by visiting https://remix.run/docs/en/v1 and typing __remixManifest into your console

The client

Today, our entry-client looks a lot like this:

function ClientApp() {
  // makes a fetch request to the RSC endpoint
  const response = useServerResponse(serverState);

  return (
    <Suspense>
      {response.readRoot()}
    </Suspense>
  );
}

With this new proposal, the client would be much more aware of the app structure than it is today. We provide our entry-client with the manifest generated in the last step, and construct a virtual route tree based on the current pathname:

function ClientApp() {
  // This is generated programmatically. This is only a visual representation 
  // of what is inferred based on parsing the manifest.
  return (
    <>
      <Suspense>
        <Content route="App.server">
          <Suspense>
            <Content route="product.server">
          </Suspense>
        </Content>
      </Suspense>
    </>
  );
}

function Content({route}) {
  // makes a fetch request to the RSC endpoint
  const response = useServerResponse({...serverState, route});

  return (
    <Suspense>
      {response.readRoot()}
    </Suspense>
  );
}

Note: this is a bit contrived, because we couldn't simply pass children to layout components due to the way of loading them as an RSC component stream. Instead, we'd leverage something like Context to pass the child values to the corresponding <Outlet> component.

Speaking of which: this is where (Remixed) React Router comes in! RR is perfectly set up to be used on the client:

• Native <Outlet> support
• You can use lazy-loaded components with Suspense boundaries
• You can use fancy loader, transition and actions as needed

This effectively accomplishes what this issue aims for: a true sub-route experience, where the outer layouts stay rendered while the contents inside transitions as the RSC request changes.

The RSC server

Since we're passing route as a param from the client, our entry-server can reference the correct server component (based on import glob) and render it 🎉

The SSR server

SSR emulates what entry-client does by resolving the correct routes and building the virtual route tree. It creates multiple RSC readable streams and pipes them into an SSR HTML output.

Potential issues:

• Server context: What happens if you render server context in a layout component and expect to be able to use it in a server component downstream? It isn't rendered in the app tree — is that a deal breaker?
• Re-rendering layout components: It's possible you're using request data to render a specific layout. This approach falls short, because the layout isn't re-rendered with each request. This is how Remix behaves today. Is that OK?
• No more component-based routes: We have to fully lean into file-based routing, or custom routes defined as an array. This is because we rely so much on parsing these routes ahead of time and turning them into a manifest. I'm OK with this tradeoff, and it's the same one that Remix makes.

[blittle]

Regarding the issues:

1. Server context: I think that this constraint makes sense. It's similar to remix as well. A nested route cannot access data from the parent route through useLoaderData(). I spoke to them about this, and it's necessary so that they can run the loaders in parallel without dependencies (no waterfall). I think we can make the similar tradeoff and educate on it.
2. Re-rendering layout components: This is a central feature of a nested router, and I think is also as expected.
3. No more component-based routes: This is the biggest downside, which might not be that bad. The main usecase we had for component routes was A/B testing and i18n. But I'm willing to bet that we can still do those things with the file-based approach. It might be a bit more complicated, but still doable. Maybe let's think through it though?

[blittle]

What would be the things that determine what is a layout component and what isn't? Something in the name? Something the file exports?

[frandiox]

Would it work without the underscores?

src
├── App.server.jsx
└── routes
    ├── product.server.jsx
    ├── collection.server.jsx
    ├── featured.server.jsx
    ├── featured/index.server.jsx
    ├── featured/thing.server.jsx

Where featured.server.jsx renders <Outlet/> and everything in the featured/* directory will be rendered as a child route.

[blittle]

I bet we might be able to make that work @frandiox, but I think there's still some value in being explicit, especially when there get to be many routes. Is this also an area where being opinionated where convention over configuration is okay?

[wizardlyhel]

I am less worry about sharing data between layout components. I think we can make that an invisible stitching with

• Cache API or InMemory Cache - These cache is independent from the React app to begin with so we can still take advantage of the benefits it has
• Route path - Instead of relying on a single RSC request, we can (maybe) associate multiple RSC requests to share the data that is collectively requested in a short span of time

[colindunn109]

tl;dr - lean into Remix's pattern of "layout routes" to create multiple server component entrypoints for a Hydrogen app

You can use lazy-loaded components with Suspense boundaries

Chiming in to say this sounds AWESOME. We've done some looking into Remix as a option and loved some of what they were doing. Being able to incrementally adopt they're routing where it makes sense would be a huge win. 🚀

[benjaminsehl]

• You can use lazy-loaded components with Suspense boundaries
• You can use fancy loader, transition and actions as needed

Wouldn't infinite scrolling of product lists be a great application of this? Would love to make that pagination super easy style super easy and performant. So many UX considerations to make it good -- URL param updating, scroll position restoration, memory management with virtualized lists, intersection observer, skeleton components, etc.

[benjaminsehl]

@wizardlyhel is now working in this in #1858