@matthewwithanm of Facebook's Nuclide team helped us improve our React game by avoiding the use of deprecated string refs and avoiding the use of component state for data that is unrelated to rendering. Thanks Matthew!
We merged #46 last week, completing our switch to a client/server architecture. JavaScript in Xray's user interface now communicates with the Rust core over a domain socket rather than via a native V8 extension, which dramatically simplifies our build process. We connect to the server over a domain socket, which unfortunately means that Xray doesn't work on Windows for now due to the unavailability of domain sockets in the OS. If anyone is interested in adding support for named pipes on Windows to xray_server, we'd gladly collaborate on a pull request. If you've tried to build Xray and ran into trouble, now would be a good time to try again on non-Windows platforms after carefully reading our build instructions.
We've adjusted our roadmap a bit to prioritize collaborative editing rather than focusing on producing WebAssembly-based editor build. A browser-compatible editor is still part of our long term plan and we're designing the system with that requirement in mind, but since we want all of Xray's features to support remote collaboration, it makes sense to get it into the architecture early.
Xray is currently hard-coded to open a single buffer containing the dev build of React, which isn't very useful. To fix that, we're adding a file finder that can quickly filter all files in the project that match a given subsequence query.
To obtain good search performance, we're maintaining an in-memory replica of the names of all the files and directories in the project which we can brute-force scan on a background thread whenever the query changes. We represent this data as a simple tree which reflects the hierarchy of the file system. To ensure that we can respond to user input within our 50ms deadline for coarse-grained interactions, we really want to be able to run queries before we finish reading all of the entries from the file system. To enable that, we're designing our in-memory file tree to support concurrent reads and writes.
We spent a decent amount of time exploring different approaches that could enable this, and ultimately we decided to protect the entry vector for each directory with a fine-grained read/write lock. When @as-cii first suggested this approach, I was worried that it would consume too much memory, but I then discovered the parking_lot crate, whose RwLock implementation only consumes a word of memory per instance.
The basic logic of searching will be in xray_core and is modeled as a Future to give us flexibility in how we schedule it. For xray_server, which runs as a standalone binary and has full threading capabilities, we can simply spawn the search on a thread pool. Until WebAssembly adds threading support, we can implement some kind of background scheduler that uses requestIdleCallback to break the work up into smaller chunks before yielding the thread.
Rust futures are based on a polling model, where the executor repeatedly calls poll on the future to drive it to completion. To support granular yielding in a single threaded environment, we really need to execute the minimal amount of work each time poll is called on our fs::Search future. To enable that, we maintain a stack within the future that tracks our current position within the tree. The stack keeps an Arc (atomic reference-counted) pointer to the entries of each directory, along with the current index into that list of entries. Since concurrent writers could insert entries that might invalidate these indices, we treat directory entries as clone-on-write if we detect they are referenced by more than one Arc pointer, via the Arc::make_mut method. Most of the time, writes should be able to freely mutate a directory's vector of entries, but if that write might interfere with an ongoing search, we clone the vector to avoid invalidating any active indices.
The work is still in progress, but we're hoping this design will enable a highly user responsive experience for file finding even in the presence of extremely large source directories. We'll report on our findings in the next update.
We're optimistic that we can finish up a basic (but fast) file finding UX some time next week. After that, I think it's time to tackle key bindings. Atom's key binding implementation is insanely complex and jumps through some ridiculous hoops to support a long tail of different locales and features like overlapping multi-stroke bindings, binding to key-up events, etc. Eventually, we want Xray to support all of these features as well, but in the short term, we want to keep the implementation as simple as possible. We're going to start by targeting single-stroke bindings and avoid any gymnastics to workaround browser limitations in various international locales. We'll revisit these concerns after getting some more traction in other areas of the system.
Our strategy with Atom was to "embrace the web", which led us to associate key bindings with CSS selectors. This was a neat idea and served Atom reasonably well, since CSS selectors are a powerful tool for describing a specific context in the DOM. However, in the end I don't think the power was worth the complexity of full-blown CSS selectors. Their flexibility makes it extremely difficult to build a user interface for configuring bindings, and the complex rules for evaluating selector specificity can lead to a frustrating experience.
With Xray, I want a system for making key bindings context-sensitive that is flexible enough to support most reasonable use cases, but not so flexible that it becomes hard to reason about. My thoughts are still evolving on this, but I'm thinking about representing the context in which we interpret a key binding as a set of simple tokens called an "action context". A custom component can be used to refine this context for a subset of the view hierarchy by adding or removing tokens.
Let's use an example to explain how the system would work. This is going to be a bit contrived, but it's not totally unrealistic. Imagine you wanted to write a spell-checking extension that allowed the user to display a list of suggestions next to a misspelled word that could be navigated from the keyboard. It might look something like this:
class SpellingSuggestions extends React.Component {
render () {
<ActionContext
add={["SpellingSuggestions", "VerticalNav"]}
remove={["Insert"]}>
<Action type="NavUp"/>
<Action type="NavDown"/>
<Action type="Confirm"/>
<div>...</div>
</ActionContext>
}
}In the example above, we declare a refinement to the action context via an ActionContext JSX tag at the root of the component, adding the SpellingSuggestions and VerticalNav tokens and removing Insert. We then declare three actions that this component handles via Action tags: NavUp and NavDown, and Confirm.
Normally in the editor, the up and down arrows would be bound to the MoveCursorUp and MoveCursorDown actions, which move the cursor. But when your menu is displaying, you want the arrow keys to select the next or previous item in the list instead. To enable that, the up and down arrow keys could be bound to NavUp and NavDown within the VerticalNav context. The left and right arrow keys would continue to move the cursor, and potentially dismiss the menu if you moved out of the misspelled word.
If you didn't like the menu hijacking your cursor movement, you could unbind the arrow keys in the VerticalNav context, or maybe leave the arrow keys bound but preserve the Emacs-style ctrl-p and ctrl-n bindings for cursor movement.
Users might also bind j and k to NavUp and NavDown in any context that is not Insert. The text editor would introduce Insert to the action context because it inserts text, but the spelling suggestions menu could temporarily override that by removing Insert from the context. So could a Vim extension in command mode.
This system is still pretty complex, but its semantics are much simpler than CSS selectors, and it seems like it could cover compositional scenarios like the one described above rather well. We could easily provide some kind of global registry of action context tokens that gives them a human-readable name and description, then use that in a user interface that makes it convenient for users to customize their bindings in specific contexts without opening a JSON file.