Generate Examples -- Continued

[Spiteless]

This is following on from the work of #940

This is currently a draft PR

• Last updated 2024-08

Latest updates

• .yml files are successfully loaded via getStaticProps and passed to TestCaseComponentPage
• Updated to Shiki 1.4 from Shiki 0.x
• Brought in CSS from the codepen file linked in Add the generate-examples library #940 (link)

Screenshot

Current issues

• The during step that shows selection for deletion isn't being hit
  • Haven't updated the code to capture a pendingDeletion selection yet (loadFixtures.ts)
• Color scheme is very bad, will need to pick something better eventually

Older changes

Summary of older changes

• forked main
• Cherry picked the 3 commits from Add the generate-examples library #940

4e0b81cc SimeonC         All working with tests                                                           
3d2e85ff pre-commit-ci.. [pre-commit.ci] auto fixes from pre-commit.com hooks                             
47ca6352 SimeonC         wip: Delete workspace.json for cherry-pick - git cherry pick 070c870 - Delete .. 

• Moved cursorless-rx/ ... /generate-exmples folder to packages/generate-examples to conform with monorepo
• Updated loadFixture directory to point correctly to cursorless-vscode-e2e recorded folder
• Resolved an error in packages/generate-examples/src/lib/generateHtml.ts @ SelectionParser.parse that made pnpm build fail
  • This might not be the right solution, but it the code it working again
• Populated results from loadFixture with original data -- as of yet unused
• Ran relevant pnpm install/setup commands

Checklist

• I have added tests
• I have updated the docs and cheatsheet
• I have not broken the cheatsheet

[SimeonC]

Thanks for picking this up! If you need any help figuring out what I was doing let me know. I can answer questions but completely have no time to code outside of work commitments 😞

[Spiteless]

All good, happy to see you're well :)

I'm definitely curious about the decisions you made to choose these libraries and what your vision was for how you wanted this feature to end up.

Recently I was looking for more opportunities for to learn what Cursorless can do and wanted a series of gifs that could demo its capabilities. We've already got all these test files so that seemed like a good place to start, and it turns out you already are plenty of the way there!

[SimeonC]

I think we just had the same desire, while I was learning Cursorless at the start I just wanted something like a "library" of examples of what each command could do. My overall thinking was just a "search command/scope/etc, see examples" type app, but never got past the "just make the HTML work" stage.

I don't think there was anything really "deep" in my decision-making, shiki lib had the ability to give me the objects/arrays that I could manipulate like a "middleware" to add all the highlighting/tokens before rendering the HTML (bonus that it had all the different styling options too). That was really the only decision I made, the rest is all just hard work figuring out how to make overlapping ranges highlight (no multi-ranges is the easy part), hence the tests cause I broke my own code so many times trying to fix it, I suddenly had a better appreciation for TDD 🤣

[Spiteless]

I think one of my main questions is where this will end up residing. Eg is it gonna be available locally via something like the cheat sheet or is it only online via the docs.

That might be a better question if Andreas or Pokey, but I think it affects how I would explore building this.

My first inclination was to see if anything could be reused from the Docusaurus space:

• Use a build system to turn the test YAML files into MDX
• Create a custom component to render the MDX with all the highlight, ranges, hats info in the grey matter area

If it's going to reside in the cheat sheet then there's no established components, theme etc to start from and something like shiki seems like the right choice.

I honestly have no idea though.

[SimeonC]

Sorry can't help you there, definitely a question for Pokey or Andreas as I didn't really figure out with them where was best.

[AndreasArvidsson]

With the caveat that I haven't really read through the previous pull request we definitely want code examples in the online documentation. We might link from the cheat sheet to the online docs, but I don't think we should actually cram all the documentation into the cheat sheet itself.

[pokey]

A package is the way to go, but I'd recommend starting from a clean checkout of main and then proceeding as follows:

1. Run the steps in the docs for creating a new package (your package is a library, so don't add private in step 4)
2. Lift just the typescript files from Add the generate-examples library #940
3. Take only whatever else you need from Add the generate-examples library #940 to actually make things work. There's a lot of nx cruft in that PR that we don't need anymore

I'm not sure generate-examples is the name I'd go with. I think the primary export of this package should be a react component whose input is a TestCaseFixture. So I think I'd maybe call the package test-case-component? @AndreasArvidsson any thoughts on name?

Fwiw you might want to look at the cheatsheet package, as that's an example of a package whose primary export is a react component

For local testing / experimentation, I'd just add a dummy page to our cursorless-org package so you can use our next.js local dev setup. That's what we do for our cheatsheet (https://github.com/cursorless-dev/cursorless/blob/main/packages/cursorless-org/src/pages/cheatsheet.tsx; actually deployed at https://www.cursorless.org/cheatsheet fwiw) You can just use a dummy fixture for now. See README at https://github.com/cursorless-dev/cursorless/tree/main/packages/cursorless-org for info on how to run local dev server

Once you have a working react component, we can look into getting it into docusaurus using mdx

Lmk if any of the above doesn't make sense, and feel free to drop into another meet-up if you get stuck!

[Spiteless]

Made some more progress

Screenshot

There's supposed to be a during step that isn't being hit. Will investigate more later.

[pokey]

cool! shall I take a look, or did you want to hack some more before I look? Or easier to give me a walkthrough at a meet-up?

[Spiteless]

I think a walkthrough at a meetup would make more sense. A lot closer but not ready yet.

[Spiteless]

I think that there might be some uncommitted code from awhile back to handle the decoartions and decorations.pendingDelete state. @SimeonC , if this rings a bell please let me know.

I'm reading through the code looking for the location where the delete a line information comes in. it should be on the during step which is contained in loadFixture

    const during = data.decorations
      ? await safeGenerateHtml(
          "decorations",
          {
            ...data.initialState,
            decorations: data.decorations.map(
              ({
                name,
                type,
                start,
                end,
              }: {
                name: string;
                type: string;
                start: SelectionAnchor;
                end: SelectionAnchor;
              }) => ({
                name,
                type,
                anchor: start,
                active: end,
              }),
            ),
          },

But the *.yml files don't have a decorations property.

In the code from then codepen, there is a highlighting feature

Screenshot

But the code from pr/940 doesn't generate the same highlighting feature when running pnpm build

The *.yml files have an ide property now, which is I think where the deletion information is being stored, so I'm currently figuring out how the code applies classes to lines and trying to apply the correct classes that should be generated from the ide object.

[Spiteless]

ide:
  flashes:
    - style: pendingDelete
      range: {type: line, start: 1, end: 4}

This is the ide section where some flashing styling comes in.

[pokey]

run

cursorless/packages/cursorless-engine/src/scripts/transformRecordedTests/transformations/upgrade.ts

Line 4 in bccc6d8

export function upgrade(fixture: TestCaseFixtureLegacy): TestCaseFixture {

on the fixture

[pokey]

remove loaded and make data have type TestCaseFixture

[Spiteless]

Errors

Error when loading upgrade

import { upgrade } from "@cursorless/cursorless-engine"

...

const data = (
    await Promise.all(
      [...dataActions, ...dataDecorations].map((val) => {
        const upgraded = upgrade(val)
        return loadFixture(upgraded);
      }),
    )
  ).filter((val) => val !== undefined);

[SimeonC]

Looking at my last commit message has "Needs to update to match new fixture structure" 😅

It's likely to be wrong now, it used to be the selections array for the "highlights" so you can adjust the tests for those. decorations was the definition for the hat over the word letter.
I was lazy and they were the last tests I wrote so it's the first tests in the spec file 🤣 https://github.com/cursorless-dev/cursorless/pull/2272/files#diff-6936a06423cb56d7239bd0a17172236d9132242a38320441f4c11d81e3a296bbR17

[Spiteless]

Got sick, been down for the count and unable to make cursorless meets for the last weeks. Back at it soon ™️