Monorepo?

[gmaclennan]

Does it make sense to switch this to a monorepo with related modules? In particular mapeo-schema, which is very closely tied to this? @EvanHahn maybe you have some experience with monorepo setups? My impression is that they are a lot of work to get all the tooling working.

[EvanHahn]

tl;dr: I am generally pro-monorepo. Happy to try something. npm workspaces look nice.

I have two experiences consolidating multiple repos: braintree-web, a work team project, and Helmet, a personal open source project. Both of these experiences were positive. It was a little difficult to get working but the benefits far outweighed the costs, in my opinion.

I think there are several benefits to monorepos, but the biggest is improved development velocity. It speeds things up in a few ways:

• Spend less time organizing. For example, this issue is actually three issues: the "main" one, and two smaller ones. Maybe this could be a single issue with subtasks.
• Spend less time releasing. I mentioned two previous projects that used to be multi-repo. In both cases, doing a release was tricky; I had to mentally walk the dependency graph, making sure to release things in the right order. And if you make a mistake...
• Reduce duplication. For example, I opened a PR to update nanobench types in this repo and did the same thing in multi-core-indexer. In a monorepo, this could have been a single change. And prior to my change, both repos had code that did the same thing.
• Easier standardization. For example, multi-core-indexer uses tap, mapeo-core-next uses brittle, and CoMapeo-mobile uses jest. A monorepo would make it easier to standardize. Monorepos would still let us have these differences if we wanted to (or didn't have time to migrate).
• Faster navigation. It's usually small, but there's a cost to jumping between repositories. For example, if you're not sure which component does something, git grep works in a monorepo setting and doesn't in a multi-repo setting. I recently wanted to figure out which of our projects depend on multi-core-indexer; that took me ~a minute of GitHub searching compared to ~a second with git grep.

These issues could totally have been solved another way. For example, we could reduce duplication with more dependencies. But I think monorepos are a more elegant solution to these problems.

It may not work for us for some reason—every project is different—but I'm glad we're considering it. I'm also unsure which projects would be folded into a monorepo and which would be left out (if any). Should CoMapeo-mobile be part of the monorepo, for example?

I don't know the best monorepo tool for 2024, but I'd evaluate npm workspaces and Lerna. npm workspaces seem pretty easy, at a glance.

If this is something we're considering, I'm happy to do some proper investigation and write up a proposal.

[achou11]

Agree that a monorepo would probably be to our benefit in this case (I don't have a ton experience with them personally).

@EvanHahn thanks for the details! I'd be very interested in you taking the lead and proposing something. Personally would hope that npm workspaces work well enough for us, but again, no prior experience with it.

Off the top of my head, the modules that should be part of a potential monorepo would be:

• mapeo-sqlite-indexer
• multi-core-indexer
• mapeo-schema
• mapeo/mock-data
• digidem/types

I'd be hesitant to include application repos like comapeo-mobile, but that's just a gut feeling.

[achou11]

also just adding that https://nx.dev/ is the successor project to lerna, if that helps in any case.