Requiring code with 'inline imports'

"inline-imports" is a Babel transform that turns imports into lazily loaded requires. The require call is deferred until the imported identifier is referenced. This allows you to write idiomatic code without the performance costs of loading code up-front (I/O, parsing, and executing).

Why?

A require is cheap. But thousands of requires are expensive. The I/O cost of resolving a module and reading the file, plus the cost of parsing and executing an "average" file is 0.5ms-2ms.

Nuclide has many non-overlapping features, which means that typical users will never run certain parts of the code. Writing code that conditionally loads modules to accommodate for this is really unidiomatic. Also, a lot of code that is not needed at startup crowds out code that is.

inline-imports solves this problem by only doing the costly require when the imported identifier is referenced.

How?

Transform

import statements are converted into memoized lazy require calls, and references are adjusted accordingly. As an oversimplified example:

// Before:
import bigModule from 'big-module';

module.exports = function(val) {
  return bigModule(val);
};

// After:
var bigModule;

function bigModule_() {
  return bigModule = require('big-module');
}

function doExpensiveWork(val) {
  return (bigModule || bigModule_())(val);
};

The actual implementation preserves all of the behaviors of import, including living bindings and allowing circular references. For examples of the actual output, see https://github.com/facebook/nuclide/tree/master/pkg/nuclide-node-transpiler/spec/fixtures/inline-imports.

Transpile

The transform is applied during development, to tests, and to builds (internal and OSS).

• Transpile boundaries:
  • Atom client (UP and integration tests),
  • apm test
  • web-views (debugger)
  • tasks (path-search)
  • server
  • npm test
  • builds
• Transpile "wrappers":
  • Atom's built-in
    • Atom client, apm test
  • nuclide-node-transpiler (a.k.a "require hook")
    • web-views, tasks, server, npm test
  • builds

Now, there's an abstraction called NodeTranspiler that the wrappers use to ensure consistent transpiling. The only exception is apm test - we need a custom runner for that (coming soon!). Atom's transpiler is being monkey-patched when UP starts in development (see atom-babel-compiler-patcher.js).

• NodeTranspiler currently applies the exact transforms that Atom does, except:
  • Source maps are turned off.
  • Stage 0 transforms are disabled as an optimization. (comprehensions, do-expressions and function-bind).
  • es3.memberExpressionLiterals is turned off (noisy. obj['default'] vs obj.default).
  • Two custom transforms on-top: (1) Remove 'use babe'; so final builds don't get mistakenly re-transpiled by Atom. (2) inline-imports.

Gotchas and opting out

The trade-off with inline-imports is that load order is no longer easily determinable. This means you must write code that doesn't depend on load order side-effects.

To ensure that a module loads in a particular order, use require. Only imports are transformed.

Anti-patterns

There are certain prevailing code patterns that deopt inline-imports. To understand them, you must first understand how Atom loads packages:

// This file is only required if the package is enabled.

export function activate() {
  // If this package is enabled, this function will be called during the
  // load cycle.
  // 
  // This function is called before any service provider/consumer functions.
}

export function provideServiceFoo(service) {
  // If this package is enabled, regardless of whether or not there are
  // consumers for "service Foo", this function will be called during the
  // load cycle.
}

export function consumerServiceBar(service) {
  // If this package is enabled, and if there is a provider of "service Bar",
  // then this function is called during the load cycle.
  return new Disposable(() => {
    // The disposable returned by `consumerServiceBar` is called when the
    // provider of "service Bar" is deactivated.
  });
}

atom.commands, atom.contextMenus, atom.tooltips, etc.

These are typically registered during load. Don't use an import reference as a callback. Wrap it in a function instead:

  // Don't do this
  import doAThing from './do-a-thing';

  export function activate(state: ?Object) {
    atom.commands.add(
      'atom-workspace',
      'do-a-thing:do',
      doAThing
    );
  }

  // Do this instead:
  import doAThing from './do-a-thing';

  export function activate(state: ?Object) {
    atom.commands.add(
      'atom-workspace',
      'do-a-thing:do',
      e => { doAThing(e); }
    );
  }

By wrapping the doAThing reference in a function, the code path that loads do-a-thing.js is deferred until the command 'do-a-thing:do' command is triggered. The transformation, essentially, results in:

  var doAThing;

  function doAThing_() {
    return doAThing = require('do-a-thing');
  }

  export function activate(state: ?Object) {
    atom.commands.add(
      'atom-workspace',
      'do-a-thing:do',
      e => { 
        (doAThing || doAThing_())(e); 
      }
    );
  }

Destructing references in the module body

This is an anti-pattern, since the destructure will force the module to load:

  // Don't do this:
  import {atomEventDebounce} from '../../nuclide-atom-helpers';
  const {onWorkspaceDidStopChangingActivePaneItem} = atomEventDebounce;

  // Or this:
  import ServiceFramework from '../../nuclide-server/lib/serviceframework';
  const newServices = ServiceFramework.loadServicesConfig();

By referencing atomEventDebounce to destructure onWorkspaceDidStopChangingActivePaneItem, you forced the loading of nuclide-atom-helpers.

Soon some of the grab bag modules like "commons" will be split up to avoid wanting to do this.

export default

Stick to export default, only used named exports when necessary. export default encourages single responsibility modules. Exporting more than one thing from a module means that a reference to any one of those exports will force the entire to load.

Service providers

// lib/main.js
import type {HyperclickProvider} from '../../hyperclick';

// Use "./HyperclickProviderHelpers.js" instead of "./HyperclickProvider.js"
// because:
//    1. it's not really the full provider, it just has the suggestion
//    function,
//    2. you can import the type as "HyperclickProvider" instead of
//    "HyperclickProviderType".
import HyperclickProviderHelpers from './HyperclickProviderHelpers';

export function getHyperclickProvider(): HyperclickProvider {
  return {
    providerName: 'feature-name',
    priority: 5,
    wordRegExp: /[^\s]+/g,
    getSuggestionForWord(textEditor, text, range) {
      // Use a callback here so that loading "./HyperclickProvider.js" is
      // deferred until hyperclick is actually used.
      return HyperclickProviderHelpers.getSuggestionForWord(textEditor, text, range);
    },
  };
}

// lib/HyperclickProviderHelpers.js
export default class HyperclickProviderHelpers {
  // Use a class with a static methods, so
  //    1. You can use the decorator syntax sugar,
  //    2. Avoid creating instances of "HyperclickProviderHelpers", this way
  //    you don't have to cache the instance ahead of time, thus forcing you
  //    to load it. 
  @trackTiming('feature-name:get-suggestion')
  static async getSuggestionForWord(
    textEditor: atom$TextEditor,
    text: string,
    range: atom$Range
  ): Promise<?HyperclickSuggestion> {
    // ...
  }
}