Feature: Versioning

[thantos]

Use Cases

Please comment with any missing use cases or on the usefulness of each use case.

• Safely make non-deterministic updates to existing workflows
• Patch workflows of running executions in development or prod to fix errors (bad inputs, logic errors, etc) without having to create a new execution.
• Create test versions of workflows (??, see Feature: Versioning #207 (comment))
• Allow callers to call a specific workflow version (??, Feature: Versioning #207 (comment))

Problem Statement 1 - Non-Deterministic Workflow Updates

An execution's workflow must stay deterministic throughout it's lifespan. This means that updates to a workflow may cause past executions to fail if the workflow was updated in a non-deterministic way.

Example of a Non-Deterministic Change

// original
workflow(() => {
    await makeCall(); // seq 0
});

// update
workflow(async () => {
    myEvent.publishEvents({ value: "workflowStarted" }); // seq 0
    await makeCall(); // seq 1 - determinism error
});

The update introduced a new sequential call to the workflow. Any execution that was waiting on makeCall to return before the update would fail.

Example of a Deterministic Change

// original
workflow(() => {
    await makeCall(); // seq 0
});

// update
workflow(async () => {
    await makeCall("someInput"); // seq 0
});

The second example is not a problem, if makeCall has been started, the result has not been impacted and the old input was used. If makeCall had yet to be started, the new input would be used now. This is due to the exactly once semantics of any call (activity, time, etc).

Problem Statement 2 - Patching Executions

One of the values of a workflow is the exactly once semantics, once a workflow has performed a task, it will never do so again.

Semantically this represents a task with side effects. For example, charging a credit card and then depositing in another location.

workflow(async ({ to: string; from: string; amount: number; }) => {
    await chargeCard(from, amount);
    await depositAccount(to, amount, "DEBIT");
})

Lets say we have run 100 real purchases through, but they have all failed at depositAccount. The root cause is that the "DEBIT" string was incorrect and the depositAccount activity was expecting "DEBIT_ACC" as a value.

Ideally we could just update the workflow and restart each of the failed executions. chargeCard will not run again, but all of the failed executions should now succeed.

workflow(async ({ to: string; from: string; amount: number; }) => {
    await chargeCard(from, amount);
    await depositAccount(to, amount, "DEBIT_ACC"); // <- right string
})

High Level Versioning Strategies

• Instance - when a non-deterministic change must be made, create a new workflow with a new name and update any callers to use it.
• Implicit - when a workflow changes, create a new version and pin the running executions to the version they started with.
• Explicit - developer takes some action to declare a new version (new object, update a string/number, etc. If this action isn't taken, the base workflow is updated.

Instance

When a non-deterministic change must be made, create a new workflow with a new name and update any callers to use it.

The need to update callers is both good and bad. It allows for an explicit cutover, if for instance the input or output contract has changed, but in cases where the caller should be unaware, it couples the change to both systems.

Implicit

This is how AWS step functions works. Each change to a template forks the step function internally. Versions are not a concept exposed to users, but existing workflows continue to run on the old template until completion.

Explicit

This could look something like a lambda version or it could look more like an alias where the author maintains both code bases. In the case of lambda versions, the version cannot be updated once created (snapshot/lock) but the LATEST version can be patches until it is created into a version.

[thantos]

Create test versions of workflows

👍 or 👎 - is it important to create test/pre-releases workflows in a single service stage?

This use case came from Lambda version's documents that list adding beta versions as a reason to use them.

[thantos]

Allow callers to call a specific workflow version

👍 or 👎 - should workflow callers be able to provide a version to call?

My initial thought is, no, the service can provide versioned entrypoints (apis), but to a caller, the service behavior should be singular.

[sam-goodwin]

Seems like the following capabilities would all be useful:

• Update code of a running workflow (the current behavior)
• Freeze a workflow version by pinning to a lambda version/alias. Updates to previous versions could still be possible but opted in to.
• Explicitly provide a version number that is accessible within the code to branch off of.