How do we handle dynamic specifiers (without dynamic import)?

[targos]

ESM cannot load a module "synchronously" with a dynamic specifier. Until we have top-level await, I think we need a supported way to do it. Two use cases:

let env;
if (process.env.NODE_ENV === 'production') {
  env = require('./prod.js');
} else {
  env = require('./dev.js');
}

let canvas, hasCanvas;
try {
  canvas = require('canvas');
  hasCanvas = true;
} catch (e) {
  hasCanvas = false;
}

I can think of three solutions:

1. Explain in the documentation that one has to write the dynamic code in a commonJS module and import it.
2. Add import.meta.require
3. Add a builtin module that provides the require function (npm proposal)

[devsnek]

i would be up for unscoped require from the module module or import.meta.require, but definitely not scoped require from an imported module

[hybrist]

I'm -1 on the builtin module. This seems to go directly against the spirit of TC39 (as I understand it). import.meta is the official way to get module-scoped values and import.meta.require could be one of them.

[MylesBorins]

I'm a huge +1 on import.meta.require

[mcollina]

+1 to import.meta.require.

[devsnek]

I can open a pr for that in a few hours unless someone else wants to take it

[giltayar]

-1 on import.meta.require for this use case (if we decide that this is the way to go for CJS interoperability, then that's OK, but IMHO it's not OK for this use case).

My reasoning is this: ESM is async, and this is not going to change, AFAICT. Developers working with ESM will have to live with that asynchronicity through various means. Having a way to workaround that asynchronicity will mean that when the module they import will move from CJS to ESM, they will have to rewrite the code to deal with that.

Yes, because top-level await is not yet a thing in JS, they have to wrap their dynamic import in an async IIFE, but that's the way it is currently with ESM. Once developers move to ESM land, they need to start thinking ESM-think and not import their CJS-think into it.

So, yes, -1 on adding import.meta.require for this use case. We should not add complexity to an already complex subject (just because we can).

[MylesBorins]

@giltayar there are all sorts of sync apis that exist in node. To me the "async" nature of ESM has to do with the initial module graph, there is 0 guarantee that the execution with make it through the entire graph without hitting a sync api. That being said I recognize that I am conflating "dynamic imports" + "cjs interop"... although it is hard to separate the two when cjs can be used for dynamic moduless.

edit: keep in mind that require does not work for esm

[giltayar]

@MylesBorins - I am not against import.meta.require as a solution for CJS interop, if that is what is decided! I am against getting this feature entering NodeJS through a backdoor as a solution to a use case that for me is not that important.

As for async/sync - it's not the sync-ness of the api that bothers me, but rather it's the fact that we're importing a module in a sync way in an ESM module that should allow only async importing, and which when that imported CJS module will be migrated to ESM will anyway need to be rewritten as an async import (because, as you said, require does not work for ESM).

[MylesBorins]

@giltayar I do think it is worth thinking about the initial graph and subsequent graphs as different beasts. While they may share a cache, when you dynamically import a module you are essentially creating a new graph.

[giltayar]

@MylesBorins - interesting point of view (initial and subsequent graphs)!.

To continue that way of thinking, and if I understand where you're coming from, then from the ESM point of view you are creating a new graph even if you are synchronously importing a CJS module. Synchronicity does not help you with "extending" the existing graph, because, by decree, ESM's initial graph must be statically analyzable. And dynamic import, whether sync or async, is not.

So we're not gaining anything with synchronous import.meta.require, except for a comfortable and known way of importing modules. At least for this use case!

[weswigham]

If I, as a user, can write a global require function implementation in commonjs that is in the global scope (and I'm fairly certain I could, by inspecting the stack of an error I make to determine the dirname and filename of the calling file) that is then usable in a module, I don't see why node shouldn't offer that functionality in it's core, the "spirit of the tc39" not withstanding. I dislike both import.meta and a module that brings in require because both require varying degrees of refactoring to migrate to modules... It's very much not meeting users where they are (that is to say, using a seemingly global synchronous require for most imports).

[TimothyGu]

@weswigham A global require approach has far reaching drawbacks. What should it do, e.g., when a vm.Script created without a file name invokes it? eval? Would that break CJS expectations that require() be always per-module? We have not put that option into consideration, for some very good reasons.

[weswigham]

@TimothyGu I'm not sure that the technical considerations for the vm module should override the ergonomic considerations of migration.

[weswigham]

Especially, since, as I said, I could write a package that does this today, and offer a better experience with an import "shim-global-require"; or something. I don't think allowing a shim package to offer a better migration experience than node proper is correct. And by owning it in core, you can make sure it behaves correctly with respect to loaders and require hooks.

[MylesBorins]

I've kicked off #10 to discuss interop... this issue should likely stick to the initial thread of dynamic imports, although I recognize they are very very intertwined