Proposal: Move to native modules

[matthewp]

tldr; Make native web modules the primary way we show usage of CanJS and DoneJS.

Problem

Using Steal for development has the following problems:

• Slowness: Loading any large app can take a significant amount of time for each page load (10+ seconds often).
• Version compatibility: Steal follows the Node module resolution, which is not built for client-side loading. As such it will nest versions in subfolders and Steal correctly loads according to semver.
• Poor integration with popular tooling: Many of today's most popular tools is not built to be run in the browser and either can't or suffers from significant slowness; this includes TypeScript, Sass, and others.

Solution

Switching all of canjs to use native module imports. For example, where a module might currently look like this:

var canReflect = require("can-reflect");
var callbacks = require("can-view-callbacks");

Change it to:

import canReflect from "../node_modules/can-reflect/can-reflect.js";
import callbacks from "../node_modules/can-view-callbacks/can-view-callbacks.js";

For end users we would recommend that people install and use can. They can avoid these nested imports by creating a can.js file in their own projects. For example in src/can.js it would be:

export * from "../node_modules/can/can.js";

And then in their components, say components/home.js they would do:

import { Component } from "../can.js";

Templating

Since the web doesn't support loading non-JS files, we can't support loading of .stache files like we do with steal. However we have already begun showing templates being defined in JavaScript. I think we just continue to do this.

For syntax highlighting, there are editor plugins that help with this. For example this one allows you to get HTML highlighting with a comment:

import { Component } from "../can.js";

Component.extend({
  tag: "home-page",
  
  view: /* html */ `
    <div>This will be syntax highlighted.</div>
  `
});

Styling

As with templates, you cannot import a CSS file in native JavaScript modules. We should still continue to suggest using the modlet pattern. To bundle CSS files there are a few possibilities:

• Simple concatenation works most of the time.

• Using Sass or less or whatever will handler bundling.

• If we move to using shadow DOM then inlining styles in a component's template will be required anyways:

  import { Component } from "../can.js";
  
  Component.extend({
   tag: "home-page",
   
   view: `
     <style>
       .root {
         display: inline-block;
       }
     </style>
     
     <div class="root"></div>
   `
  });

• Some other solution. For example we could create a "loader" for styles that works like this:

  import includeStyles from "../node_modules/steal-styles/steal-styles.js";
  
  includeStyles`
    body {
      background-color: green;
    }
  `; // This just creates a a `<style>` tag and injects into the head. Maybe in production this gets extracted out...

I think we should just pick one of these for the next DoneJS version and incorporate it into the tooling.

Server-side rendering

Node.js doesn't support native modules and likely won't for quite a while (another year at least). Because of this we will need to transpile to CommonJS.

To make this work we can incorporate commonjs exporting (which steal-tools can already do) into donejs develop. For example:

steal-tools export --cjs

Will export the project to dist/cjs/ folder. done-ssr can be changed to check this folder in order to load code before falling back to its existing steal loader. In this way, it will not be a breaking change for done-ssr.

Production

We should continue to support steal and steal-tools for production use, where bundling is still usually required.

We should explore bundling to native modules so that the loader (whether the steal or slim loader) can be excluded.

Roadmap

This transition cannot occur until some proper tool is developed to make it possible:

• Create a CommonJS -> ES module transiler in stealjs/transpile.
  • Update steal-tools export to support ES transpilation.
• Create a codemod that can transpile an existing project written in CommonJS.
  • Include in the codemod the ability to modify the test.html file to use native module loading instead (<script type="module">).
• Submit PRs in all repos. Use a conventional branch such as es-module so that it can be tested in canjs/canjs.
• Make this the default in a CanJS major release.
• Make any changes needed to support bundling this in steal (I'm not sure what if anything this would include). Likely very little.
• Support donejs project dist/cjs/ convention in done-ssr.
• Update DoneJS to use ES modules fully (generators, guides, etc).

[frank-dspeed]

@matthewp i have worked exactly on this since 1,5 years :) i am happy that you join this effort

[frank-dspeed]

we should drop steal complet and replace it with rollup i know you love steal but rollup already produces the needed SystemJS Packages and also iife and umd we should make that tool chain work as best and easy as possible via rollup-plugin-canjs to allow a easy migration we have only a few cases that are not handled already but can easy be handled even importing stache files and css works similar

on the long run we should replace stache files and component files with nativ mjs files
for a good migration path that is well backward compatible we should produce js files that are modules and polyfill only the parts like.

#1153 is related

• custom components
• dynamic import (or modules) this is easy done with SystemJS.import

at present a good feature detection that handels only the cases that are needed for canjs is missing but can be done relativ easy with more effort

[justinbmeyer]

Does this form of addressing preclude modules installed anywhere other than the top-level node_modules folder?

To be developed individually, projects would need to have their code moved (or some other solution):

// the following wouldn't work from /can-ajax.js
import canReflect from "../node_modules/can-reflect/can-reflect.js";

Poor integration with popular tooling

Version compatibility

How does native modules solve these problems?

[matthewp]

An alternative I've been thinking about is to embrace using unpkg directly which side-steps the version problem with using locally installed node modules. Maybe a steal-tools export build that produces:

import canReflect from "//unpkg.com/can-reflect@^1.0.0/can-reflect.js";
import callbacks from "//unpkg.com/can-view-callbacks@^4.0.0/can-view-callbacks.js";

instead of:

import canReflect from "../node_modules/can-reflect/can-reflect.js";
import callbacks from "../node_modules/can-view-callbacks/can-view-callbacks.js";

The version numbers inserted into the URL would be taken from the package.json.

---

As a compliment to the above I'd want a service worker for development usage that caches URLs coming from unpkg so you could still develop on an airplane and refreshes should be fast.