Support saving and restoring of terminal state

[vincentwoo]

Hi everyone,

With xterm.js maturing, I think a great feature to expand the range of applications xterm.js could be useful for would be the formalization of a headless mode. What do I mean? Well basically, for the ability to:

1. Run a client without a renderer.
2. Communicate to a client with a renderer the state of a headless client and vice versa.

Right now, the only way to reliably "set" the state of a client is to replay the set of all commands that produced that editor's state (at least via the public API). However, you could easily imagine wanting to do things like:

• scrubbing back and forth through a terminal's history
• running a terminal in the cloud that you intermittently connect to via the client
• (my use case) having multiple users have literally the same terminal with the ability to come and go

A lot of this is basically almost already possible if you just serialize term.rows and assorted state flags, but the adoption of a formal API would help a lot. It may also help improve the testability and robustness of the core library, too!

Thoughts?
Vincent

[Tyriar]

@vincentwoo I can see this being pretty useful, do you have an idea of what such an API would look like?

[vincentwoo]

I think you would likely want another constructor like term.Headless, and a new method on both headless and regular terminals like getState() that can then be fed into reset.

[Tyriar]

@vincentwoo would some way of serializing state on the server and sending that over the wire to the client also work? Recently I made it so you don't need to open xterm on an element to get it to retain the buffer, so that might be enough?

[vincentwoo]

Yeah, serialization is basically what I mean by "getState", as long as what's produced can be consumed by the client.

Regarding just using the regular xterm without an element, that might work. I can give it a shot on node, later.

[Tyriar]

I think #266 means it's essentially running as headless before Terminal.open is called.

[vincentwoo]

Cool. Any thoughts on how exporting state might work?

[Tyriar]

Depends what needs to be retained, just the buffer and cursor?

[vincentwoo]

I'm not sure myself, this is why this was a request. I don't know what the xterm state machine looks like internally, nor exactly what you mean by "buffer" and "cursor".

[parisk]

This is a great proposal @vincentwoo, thanks! I think the best part of it is the serialization of the state (which is the main part of this discussion as well).

IMO a good start towards this is introducing a private property (e.g. named state), where we will store the "data" of the terminal like:

• terminal geometry
• terminal mode
• buffer (this is the text printed to the terminal)
• cursor position
• cursor style

This property should be used internally to "set" the state of the terminal at each time (and be consulted by the terminal's internals to see what's happening when needed), but it should be read-only by the outside world (e.g. term.getState()).

How does this sound?

[vincentwoo]

Sounds good to me! I'm not an expert in knowing exactly every state variable that uniquely identifies the state of xterm.js, but I'm happy to kick tires. I think trying to come up with the list and testing it is also a great way of finding oddball behavior in the library, too.

[Tyriar]

Some more things we probably want to serialize:

• All the options
• The alt buffer

@parisk I was thinking a getState or serialize function would perform the serialization at that point, we could put everything into some json object and return the json pretty easily. All the state you're mentioning may not be owned by the Terminal object, for example the buffer lives in CircularList. It would make more sense to tell the CircularList to serialize itself on a call to the function. Something like this:

Terminal.prototype.getState() {
  const state = {};
  // this.serializableComponents is an ISerializable[] which ensures they
  // have componentKey/getState()
  this.serializableComponents.forEach(c => {
    state[c.componentKey] = c.getState();
  });
  // TODO: Add anything owned by Terminal to state
  // ...
  return state;
}

// Restore using something like this which would do the reverse
Terminal.prototype.restoreState(state: Object): void;

// A static method that returns a Terminal might be better:
Terminal.restoreTerminal(state: Object): Terminal;

[parisk]

That's an interesting approach 👍 . I am opening up a branch to give this a go.

[ebertmi]

Any updates on this one? I am also having a similar use case.

[Tyriar]

Copying and pasting the proposal to fix this issue from #2213 (comment) below

---

I had a chat with @jerch about this and we see these problems with the current approach of including it into core:

• We would need to expose API to save/restore state but we definitely don't want to commit to a particular format so this would probably break every few versions
• It adds a bunch of bloat to the core when so few users will actually use it

Given this, we think the best way to approach this feature is to use an addon (example xterm-addon-serialize) that serializes state using the new buffer API (Terminal.buffer.*). The serialize method could look like this:

/**
 * Serializes terminal rows into a string that can be written back to the terminal
 * to restore the state. The cursor will also be positioned to the correct cell.
 * When restoring a terminal it is best to do before `Terminal.open` is called
 * to avoid wasting CPU cycles rendering incomplete frames.
 * @param rows The number of rows to serialize, starting from the bottom of the
 * terminal. This defaults to the number of rows in the viewport.
 */
serialize(rows?: number): string;

If you're unfamiliar with terminals the resulting string would look something like this "\x1b[Hfoo\n\rbar\x1b[2;4H" for a simple row (move cursor to row 1, column 1, print "foo", go to next line and print "bar" then position the cursor to row 2 and column 4).

This has a bunch of benefits:

• Consumers who don't use it don't need to load the code
• Consumers who do use it can dynamically load the code
• It uses public API that is declared as stable and serializes into a standard format that could be used on any terminal
• It leverages all of @jerch's awesome work speeding up the parser
• It defaults to just the viewport which would be lightning fast to both serialize and restore
• It doesn't complicate/bloat the core architecture

Note that this would not be able to support color or other styles until we expose attributes in the API but that's something we want to do and could improve upon it at a later time.

[Tyriar]

Removed headless mode from title as xterm.js kind of already supports that by working fine without calling open.

[Tyriar]

We now have the serialize addon as well as xterm-headless which covers this. Any issues with them should be reported as new issues.