[RFC] Package ecosystem reorganization

[JozefFlakus]

Overview

The current official package/modules ecosystem consist of:

• @marblejs/core
• @marblejs/messaging
• @marblejs/websockets
• @marblejs/testing
• @marblejs/middleware-logger
• @marblejs/middleware-io
• @marblejs/middleware-joi
• @marblejs/middleware-jwt
• @marblejs/middleware-body
• @marblejs/middleware-cors
• @marblejs/middleware-multipart

Problem statement

1. Main framework monorepo contains deprecated packages (eg. @marblejs/middleware-joi) that are no maintained anymore - their versions are automatically bumped-up from release to release. The Joi validation middleware was the first community driven package that was used for HTTP request validation. Due to the difficulties in proper type inference, it was deprecated with the introduction of version v2.0, when io-ts-based middleware was released.
2. Community doesn't have a dedicated environment/space where they can propose new non-core modules and easily contribute to the ecosystem, eg. like fp-ts-contrib does.
3. @marblejs/core API layer is too big. It contains a core/common abstractions that are used across other modules, like: messaging, testing, websockets, middlewares mixing core functionalities with HTTP protocol context. With an introduction of messaging module HTTP protocol is not the only one available transport layer that Marble.js apps can be built on.

Proposed solution

1. New, separate monorepo marblejs/contrib (?) for community driven packages that are out of scope core modules. First candidates for migration: @marblejs/middleware-joi and maybe @marblejs/middleware-jwt.
2. @marblejs/core package should be split into two parts: reduced API layer for @marblejs/core and brand new @marblejs/http package. All existing HTTP related API would be moved to the new scope, where the core package will include all basic and common interfaces and API's that are used across all dependent modules: testing, websockets, messaging and new http.
3. The new package layer reorganization can be introduced with the release of version v4.0

Things to consider

• Some middlewares can be baked into main modules, eg. cors, multipart, body or logger can be scoped and baked into the future @marblejs/http module. The only problem that I foresee is the maintenance experience/cost - it will be hard to maintain so many things in one place. Maybe separation into dedicated packages is still valid and good path. 🤔

Migration path

• It is not so drastic braking change
  • joi-based middleware due to it's deprecation should not be used so widely,
  • the change mostly affects imports:

Instead of:

import { r, HttpStatus, useContext, combineRoutes } from '@marblejs/core';

we will have:

import { useContext } from '@marblejs/core';
import { r, HttpStatus, combineRoutes } from '@marblejs/http';

[matthewpflueger]

Just discovered this library and love it! I would really like to see the core module slimmed down and not depend on any Node.js modules especially. My use case is I would like to share Event schemas between code that runs in the browser and the websocket server. Thankfully the Event schema Marble.js relies on is super simple so hand rolling is fine but would love to use the built in utilities Marble has defined for event creation, etc.

[DawidYerginyan]

I just found out this RFC and really like the extraction of the core module to be really the "core" of marblejs which is not "contaminated" by any infrastructure/protocol stuff like HTTP or other messaging protocols. Database-related things and other stuff like that (If ever) can be then built on top of the framework core. Good design decision 👍🏻 💪🏻 . This way you potentially could use only the @marblejs/core package to support other protocols in case you need them while they're not really supported by the framework yet. This opens many possibilities of even maybe using Marble outside of network-based apps if desired 😄 .

[JozefFlakus]

Update:

I've just created a dedicated repository github.com/marblejs/contrib and moved following packages from the main repo:

• middleware-joi
• middleware-jwt

The plan is to manage them independently (different, not synchronized semantic versioning, independent from @marblejs/* main packages). When released migrated packages will be available under @marblejs-contrib scope, eg @marblejs-contrib/middleware-joi, etc. Then the old ones will be removed from the main repo and marked as deprecated.

[JozefFlakus]

Update:

Related pull request for the introduction of @marblejs/http module: #337