Formalise Certified Nodes process

[joepavitt]

With our certified nodes program, we have a collection of widgets that customers can trust.

With Marian leaving though, we don't seem to ahve a formally documented process by which new nodes can be added to this list. As such, we should have, documented likely in the Handbook, a clearly defined process by which a set of nodes can be considered "certified"

[ZJvandeWeg]

@joepavitt @knolleary

I think there's a few things here to fix:

1. There's two repos right now, one that generates the verification data, one that publishes the catalogue.json
2. The scanning tool used no longer works -- npm audit might need to be swapped in
3. The bash script generating the output consumes a list of nodes, validates them, and only the ones that passed inspection make it onto the list for next time. The list thus becomes shorter and shorter.

[ZJvandeWeg]

@joepavitt Who started this work?

[joepavitt]

Now? Or for the nodes already done?

For the latter, my understanding was it was all Marian?

[ZJvandeWeg]

Sorry, I meant now. I figured it should've been assigned to someone and I wasn't sure who. The original work I'm aware of :)

[joepavitt]

Sorry, yeah, the "Planning" repo says "Started" as it was started in the past, but the "Development" board entry I've just added shows it's "todo" for someone to take in the immediate future

[ZJvandeWeg]

@joepavitt Just to shortcut the development and for me to hack in my evening I've created https://github.com/FlowFuse/certified-nr-nodes/pull/5/files. I don't think it's production ready, but might shave a few hours of another engineers time. Hope it helps.

It's not my intention to complete this, as my output is unpredictable in delivery time at best.

[knolleary]

To start with, I'm writing down what we have in place today to make sense of how it ties together.

certified-nr-nodes repo

• The certified-nr-nodes repo contains the master list of nodes.
• It includes an audit-modules workflow that runs in a GH action. The action pip installs njsscan and npm installs @sandworm/audit which are then run against each module in the list generating some output. It also rewrites the modules.json file to only include modules that 'passed'.
• As part of the audit script, it appears to do a POST to a FFC hosted NR instance - certified-nodes-prod. I cannot find an instance by that name.
• The GH audit action has to be run manually.

ff-catalogue repo

• The catalogue repo is used to generate the NR Catalogue json used by instances. The repo includes a list.txt file that lists what modules it should include from the public catalog - plus has @flowfuse/node-red-dashboard-2-user-addon hardcoded in from our private registry.
• The gh build action generates the catalogue and pushes the results back to the gh-pages branch - where it gets published under the public url we share.
• There is an audit workflow in that repo that looks to do much the same as the certified-nr-nodes repo - but has never been run.

I'll talk to @hardillb next week as he helped setup the ff-catalogue side of things and may have some context on how the certified node audit was intended to be managed between the two.

---

We need:

1. a single list of certified nodes that is only modified via PR on our part
2. an action that runs regularly against the nodes to verify them.
   • if the verification finds any issues, we need a way to flag that up for manual review
3. the catalogue needs to use the same list - which suggests we do all of this in one repo

I'll review the sandwork/njsscan tools that were previously used. ZJ mentions they no longer work - but not clear how so. npm audit is certainly a good tool these days to monitor for these nodes. The other tools dig deeper into the code and will bring additional value assuming they are made to work properly.