Refactor core JupyterLab APIs to make it easier to reuse components

[ellisonbg]

@yuvipanda has been doing some fantastic work to use different JupyterLab components outside of the main lab application. Initially this work was done through improving the notebook example, but there is a broader set of usage cases where it would be really nice to have a set of APIs that make it easy to reuse core JupyterLab components in different contexts. Also relevant for widgets, dashboarding, etc.

This is a meta-issue to discuss the broader API changes needed to make this easy. I will let others fill in the technical details, as they have been working more closely on the related PR's.

@yuvipanda @SylvainCorlay @afshin @ian-r-rose @jasongrout (and others...).

[yuvipanda]

#5845 is most relavent PR.

My end goal of sorts is to have a 'simplest-notebook' thing that has just the notebook interface, working widgets, maybe extensions that only do things to the Notebook itself, and nothing else.

[saulshanabrook]

Here is a list of all the public things on JupyterLab:

One possibility is to move some of this into separate extensions:

    /**
     * The document registry instance used by the application.
     */
    readonly docRegistry: DocumentRegistry;
    /**
     * The command linker used by the application.
     */
    readonly commandLinker: CommandLinker;
    /**
     * The service manager used by the application.
     */
    readonly serviceManager: ServiceManager;
    /**
     * A list of all errors encountered when registering plugins.
     */
    readonly registerPluginErrors: Array<Error>;
    /**
     * A method invoked on a document `'contextmenu'` event.
     *
     * #### Notes
     * The default implementation of this method opens the application
     * `contextMenu` at the current mouse position.
     *
     * If the application context menu has no matching content *or* if
     * the shift key is pressed, the default browser context menu will
     * be opened instead.
     *
     * A subclass may reimplement this method as needed.
     */
    protected evtContextMenu(event: MouseEvent): void;
    /**
     * Whether the application is dirty.
     */
    readonly isDirty: boolean;
    /**
     * Whether the application is busy.
     */
    readonly isBusy: boolean;
    /**
     * Returns a signal for when application changes its busy status.
     */
    readonly busySignal: ISignal<JupyterLab, boolean>;
    /**
     * Returns a signal for when application changes its dirty status.
     */
    readonly dirtySignal: ISignal<JupyterLab, boolean>;
    /**
     * The information about the application.
     */
    readonly info: JupyterLab.IInfo;
    /**
     * Promise that resolves when state is first restored, returning layout description.
     *
     * #### Notes
     * This is just a reference to `shell.restored`.
     */
    readonly restored: Promise<ApplicationShell.ILayout>;
    /**
     * Walks up the DOM hierarchy of the target of the active `contextmenu`
     * event, testing the nodes for a user-supplied funcion. This can
     * be used to find a node on which to operate, given a context menu click.
     *
     * @param test - a function that takes an `HTMLElement` and returns a
     *   boolean for whether it is the element the requester is seeking.
     *
     * @returns an HTMLElement or undefined, if none is found.
     */
    contextMenuFirst(test: (node: HTMLElement) => boolean): HTMLElement | undefined;
    /**
     * Set the application state to dirty.
     *
     * @returns A disposable used to clear the dirty state for the caller.
     */
    setDirty(): IDisposable;
    /**
     * Set the application state to busy.
     *
     * @returns A disposable used to clear the busy state for the caller.
     */
    setBusy(): IDisposable;
    /**
     * Register plugins from a plugin module.
     *
     * @param mod - The plugin module to register.
     */
    registerPluginModule(mod: JupyterLab.IPluginModule): void;
    /**
     * Register the plugins from multiple plugin modules.
     *
     * @param mods - The plugin modules to register.
     */
    registerPluginModules(mods: JupyterLab.IPluginModule[]): void;

[saulshanabrook]

And here is the list on ApplicationShell (the widget on the app) which you can also access from JupyterLab:

    /**
     * A signal emitted when main area's active focus changes.
     */
    readonly activeChanged: ISignal<this, ApplicationShell.IChangedArgs>;
    /**
     * The active widget in the shell's main area.
     */
    readonly activeWidget: Widget | null;
    /**
     * A signal emitted when main area's current focus changes.
     */
    readonly currentChanged: ISignal<this, ApplicationShell.IChangedArgs>;
    /**
     * The current widget in the shell's main area.
     */
    readonly currentWidget: Widget | null;
    /**
     * A signal emitted when the main area's layout is modified.
     */
    readonly layoutModified: ISignal<this, void>;
    /**
     * Whether the left area is collapsed.
     */
    readonly leftCollapsed: boolean;
    /**
     * Whether the left area is collapsed.
     */
    readonly rightCollapsed: boolean;
    /**
     * Whether JupyterLab is in presentation mode with the `jp-mod-presentationMode` CSS class.
     */
    /**
    * Enable/disable presentation mode (`jp-mod-presentationMode` CSS class) with a boolean.
    */
    presentationMode: boolean;
    /**
     * The main dock area's user interface mode.
     */
    mode: DockPanel.Mode;
    /**
     * Promise that resolves when state is first restored, returning layout
     * description.
     */
    readonly restored: Promise<ApplicationShell.ILayout>;
    /**
     * Activate a widget in its area.
     */
    activateById(id: string): void;
    activateNextTab(): void;
    activatePreviousTab(): void;
    /**
     * Add a widget to the left content area.
     *
     * #### Notes
     * Widgets must have a unique `id` property, which will be used as the DOM id.
     */
    addToLeftArea(widget: Widget, options?: ApplicationShell.ISideAreaOptions): void;
    /**
     * Add a widget to the main content area.
     *
     * #### Notes
     * Widgets must have a unique `id` property, which will be used as the DOM id.
     * All widgets added to the main area should be disposed after removal
     * (disposal before removal will remove the widget automatically).
     *
     * In the options, `ref` defaults to `null`, `mode` defaults to `'tab-after'`,
     * and `activate` defaults to `true`.
     */
    addToMainArea(widget: Widget, options?: ApplicationShell.IMainAreaOptions): void;
    /**
     * Add a widget to the right content area.
     *
     * #### Notes
     * Widgets must have a unique `id` property, which will be used as the DOM id.
     */
    addToRightArea(widget: Widget, options?: ApplicationShell.ISideAreaOptions): void;
    /**
     * Add a widget to the top content area.
     *
     * #### Notes
     * Widgets must have a unique `id` property, which will be used as the DOM id.
     */
    addToTopArea(widget: Widget, options?: ApplicationShell.ISideAreaOptions): void;
    /**
     * Add a widget to the bottom content area.
     *
     * #### Notes
     * Widgets must have a unique `id` property, which will be used as the DOM id.
     */
    addToBottomArea(widget: Widget, options?: ApplicationShell.ISideAreaOptions): void;
    /**
     * Collapse the left area.
     */
    collapseLeft(): void;
    /**
     * Collapse the right area.
     */
    collapseRight(): void;
    /**
     * Expand the left area.
     *
     * #### Notes
     * This will open the most recently used tab,
     * or the first tab if there is no most recently used.
     */
    expandLeft(): void;
    /**
     * Expand the right area.
     *
     * #### Notes
     * This will open the most recently used tab,
     * or the first tab if there is no most recently used.
     */
    expandRight(): void;
    /**
     * Close all widgets in the main area.
     */
    closeAll(): void;
    /**
     * True if the given area is empty.
     */
    isEmpty(area: ApplicationShell.Area): boolean;
    /**
     * Restore the layout state for the application shell.
     */
    restoreLayout(layout: ApplicationShell.ILayout): void;
    /**
     * Save the dehydrated state of the application shell.
     */
    saveLayout(): ApplicationShell.ILayout;
    /**
     * Returns the widgets for an application area.
     */
    widgets(area: ApplicationShell.Area): IIterator<Widget>;

[saulshanabrook]

Note that the first argument to activate has to be the application, which has to have a shell property that point to the base widget.

[jasongrout]

Currently we do have plugins that don't depend on the JupyterLab app, like the rendermime plugins. Perhaps that's a useful pattern to explore - are there other types of simplified plugins that we can support? For example, ipywidgets could probably be a "document plugin", that just gets notified when a new 'notebook' document widget is created, along with the widget and context for that document.

Then we build out a few stubs for supporting things like rendermime plugins, theme plugins, document plugins, etc. outside of JLab.

[quigleyj-mavenomics]

We've written a standalone notebook viewer using JupyterLab packages with some success- the biggest pain points so far have been in mocking required services for things that aren't needed for the viewer and determining a concrete list of what services are required.

Aside from that, it'd also be nice if we could target IWidgetExtensions (or their replacements) to documents-with-kernels, instead of a particular type of document widget. But I think that may be a separate concern.

[bollwyvl]

This is a great discussion. A high-value output of this would be a single file (even icons) that could optionally be inlined into html, allowing for archive-grade artifacts. Having a DOMContentsManager would allow raw ipynb, csv, etc to be directly embedded without going through the json size bloat. Packaged separately as an nbconvert module, this could be the next-generation nbviewer. I did this in a hacky way with https://deathbeds.github.io/jyve , but it required mocking tons of things. While extensions would remain a 🐻, it would at least be tractable to have builds keyed to the hashes of the the extensions on question... on nbviewer, only a few "spins" would be practical, e.g. widgets, plotly, bokeh, mpl, etc.

[SylvainCorlay]

My end goal of sorts is to have a 'simplest-notebook' thing that has just the notebook interface, working widgets, maybe extensions that only do things to the Notebook itself, and nothing else.

Do you plan on keeping components such as the menu bar and side tabbars? (I am asking because of the sidecar widget)

[jasongrout]

@afshin, @yuvipanda - is this resolved now with #5845?

[afshin]

@jasongrout Yes.

[ellisonbg]

@quigleyj-mavenomics where is that standalone renderer? Have you looked at refactoring it based on #5845 ?

[quigleyj-mavenomics]

@ellisonbg It's not open source yet but we (Mavenomics) should have something to announce soon-ish. It's part of a wider dashboarding suite for JupyterLab that we're codenaming "Jovian" (though that name might change). Here's a preview of what the standalone viewer looks like:

Internally, we're waiting for other bits of our framework to settle (including kicking out the layout engine to a separate package- would that be of use to #1640 (comment)?), but I've done some experiments to asses risk. The JupyterFrontEnd changes will go a long way towards cleaning it up- We won't need to setup the various services and registries manually, and it can injest JupyterLab extensions directly (which I'm most excited about!)

[jasongrout]

That looks great! Are you or colleagues planning on being at the Dashboarding Jupyter Community Workshop in Paris? If you can come, contact @SylvainCorlay or @pbugnion. See https://blog.jupyter.org/jupyter-community-workshop-dashboarding-with-project-jupyter-b0e421bdf164

[jasongrout]

@quigleyj-mavenomics - just curious, do you have ipywidgets support in that dashboard viewer?

[ellisonbg]

@quigleyj-mavenomics ooohhh, thanks for sharing. Different parts of this may very well be related to the dashboards effort. Excited to see where this goes and have a look at the standalone renderer when it is released. When that happens could you open a new issue pointing the community to it (to be easy to find).

[as an aside, the name "jovian" is already being used in the jupyter community and I think it will confuse people to have a project with that name–totally understand its usage as a private codename though ;-)]

[quigleyj-mavenomics]

@jasongrout Yep! We have support for anything that the RenderMime registry can display (though we can do a whole lot more than just that!). IPyWidgets work using that mechanism, and we have experimental support for hooking them into our dashboard data binding (which is where the real power is). Here's an IntSlider bound to a variable called myval:

[jasongrout]

as an aside, the name "jovian" is already being used in the jupyter community

Actually, it's "Jovyan", with a y of course :). https://github.com/jovyan/