Skip to content

Commit 2fc5367

Browse files
Update readme, dumpWorkflow and cli for code generation (#11)
Co-authored-by: David Zeng <davzen@microsoft.com>
1 parent 45289a0 commit 2fc5367

File tree

10 files changed

+170
-42
lines changed

10 files changed

+170
-42
lines changed

README.md

Lines changed: 121 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
# React and promise based task flow library
2-
Now a lot of features are task flow based. You could break features into a seriese of tasks with order and dependency. For example, to start task C, you have to wait for task A / B and then take their outputs for further process. To express and reuse task flow easily and visually, it could help improve development efficiency. We would refer task flow as Workflow in code.
2+
Now a lot of features are task flow based. You could break features into a seriese of tasks with order and dependency. For example, to start task C, you have to wait for task A / B and then take their outputs for further process. To express and reuse task flow easily and visually, it could help improve development efficiency. In this repo, task flow and workflow are the same.
33

44
## Example
5+
### Simple taskflow expression
56
Take the task flow below, there are 4 params for input. The task flow would add all 4 params with 3 add nodes and then double the sum with double node. Then the double node's result would be set as task flow's output.
67

78
![SampleTaskFlow](./md/SampleTaskFlow.png)
@@ -17,9 +18,21 @@ The task flow above could be expressed with react jsx style below,
1718
<OutputNodeComponent name="res" dep="double" />
1819
</WorkflowComponent>
1920
```
20-
As you could tell from the code above, there is a container tag *WorkflowComponent*. Inside the container, there is first a *InputNodeComponent* node with *params* which is a array of name of the input parameters. *NodeComponent* add1 would take num1 and num2 defined within *InputNodeComponent* node to compute the add result with addFunc. After two node add1 and add2 finish work, add3 would take their outputs to run addFunc again with result passed to double node. Finally the result of double node computed by doubleFunc, would be set as task flow's output with alias res.
21+
In the jsx code above, there is a container with tag *WorkflowComponent*. Inside the container, there is first a *InputNodeComponent* node with *params* which is a array of name of the input parameters. *NodeComponent* add1 would take num1 and num2 defined within *InputNodeComponent* node to compute the add result with addFunc. After two node add1 and add2 finish work, add3 would take their outputs to run addFunc again with result passed to double node. Finally the result of double node computed by doubleFunc, would be set as task flow's output with alias res. The input parameter and output of a node
22+
are all refferenced by name.
2123

22-
If you want to define a re-usable task flow, then you could define a function with props to wrap the workflow.
24+
The jsx code expression above is simple, clear and visual. You don't have to write code with long and chainning expressions.
25+
26+
For each *NodeComponent*, you need to specify a function that could genrate *WorkflowExecutionNode* as below,
27+
```typescript
28+
export interface WorkflowExecutionNode {
29+
run: (...params: any[]) => any;
30+
cancel?: () => void;
31+
}
32+
```
33+
34+
### Re-usable taskflow expression
35+
If you want to define a re-usable task flow, then you could define a function with props to wrap the workflow. For example,
2336
```typescript
2437
function ComputationWorkflow(props: WorkflowInputProps) {
2538
return (<WorkflowComponent {...props}>
@@ -42,8 +55,32 @@ And in a new task flow, reuse the task flow by passsing the params. Then chain t
4255
</WorkflowComponent>
4356

4457
```
45-
Once you have defined jsx task flow, you could use buildJsxWorkflow to generate the task flow data structure. And with createWorkflowExecutor, then you could run workflow with the executor. Please reference **example** and **test** folder for more.
58+
### Taskflow conversion
59+
The example above is about taskflow expression. Once you have defined jsx taskflow, you could use buildJsxWorkflow to generate the task flows with ndoes and its depdencies. For example,
60+
```typescript
61+
const jsxWorkflow = <WorkflowComponent>
62+
<InputNodeComponent params={["num1", "num2", "num3", "num4"]} />
63+
<ComputationWorkflow name="comp" params={["num1", "num2", "num3", "num4"]} />
64+
<OutputNodeComponent name="res" dep="comp.res" />
65+
</WorkflowComponent>
66+
const workflow = buildJsxWorkflow(jsxWorkflow)
67+
```
68+
### Taskflow execution
69+
*createWorkflowExecutor* is for create task flow executor by passing the task flow structure. Call the run method, then wait for the outputs of promise and reference the result by alias . For example,
70+
```typescript
71+
const jsxWorkflow = <WorkflowComponent>
72+
<InputNodeComponent params={["num1", "num2", "num3", "num4"]} />
73+
<ComputationWorkflow name="comp" params={["num1", "num2", "num3", "num4"]} />
74+
<OutputNodeComponent name="res" dep="comp.res" />
75+
</WorkflowComponent>
76+
const workflow = buildJsxWorkflow(jsxWorkflow)
77+
const executor = createWorkflowExecutor(workflow)
78+
executor.run(1, 2, 3, 4).then((res) => {
79+
console.log(res["res"])
80+
})
81+
```
4682

83+
Please reference *example* and *test* folder for more.
4784
## Development Setup
4885
Please install vscode as IDE
4986
```ini
@@ -61,22 +98,93 @@ npm run lint
6198
```
6299
To debug test case, set sourceMap to be true in tsconfig.json, set configuration to be Jest Current File, open test file and run Start Debugging from vscode menu.
63100

64-
## Notes
65-
Currently the task flow expression only supports **react functional component**. And it would be better to add test for build and run the task flow.
101+
## Usage
102+
### Api
103+
****
104+
buildJsxWorkflow(expression: React.ReactElement, addNodeName: boolean = false)
105+
****
106+
Convert the jsx task flow expression to Workflow with node id to name mapping as optional.
107+
108+
**Parameters**:
109+
- **expression** - the jsx expression of the task flow
110+
- **addNodeName** - false by default, if true it would generate id to node name mapping.
111+
112+
**Return**
66113

67-
For task that needs to wait for user's input like clicking button, please keep the resolve function. When user clicks the button, you run the resolve function with input needed.
114+
The task flow instance
68115

69-
For UI specify task flow, it is recommended to define something like bridge to keep state and callback functions to update UI.
116+
****
117+
createWorkflowExecutor(wf: Workflow)
118+
****
119+
Create task flow executor with task flow instance.
70120

71-
Although the library has dependency on **react**, however we don't need to actually need it at run time. There is an interface **dumpWorkflow** which would dump the workflow into code. Then you could create the workflow without react. For example, in **./test/DumpWorkflow.test.tsx**
121+
**Parameters**
122+
123+
- **wf** - the task flow instance, not the task flow jsx expression
124+
125+
**Return**
126+
127+
The task flow executor instance
128+
129+
****
130+
dumpWorkflow(wf: Workflow)
131+
****
132+
133+
**Parameters**
134+
- **wf** - the task flow instance
135+
136+
**Return**
137+
138+
The code that could generate taks flow instance
139+
140+
141+
### Interfaces
142+
Task flow
143+
```typescript
144+
export interface Workflow {
145+
inputs: number[];
146+
zeroDepNodes: number[];
147+
nodes: WorkflowNode[];
148+
outputs: Record<number, string>;
149+
binding: Record<number, number[]>;
150+
nodeNames?: Record<number, string>;
151+
}
152+
```
153+
154+
Task flow executor
155+
```typescript
156+
export interface WorkflowExecutor {
157+
cancel() : void;
158+
run(...params: any[]) : Promise<any>;
159+
setTimeout(timeout: number) : void;
160+
reset() : void;
161+
state() : ExecutionStatus;
162+
inst(inst: boolean) : void;
163+
workflow() : Workflow;
164+
stats(): NodeExecutionStatus[];
165+
}
72166
```
73-
{inputs:[0,1,2],zeroDepNodes:[],nodes:[{id:0,deps:[],gen:unitNodeGenerator},{id:1,deps:[],gen:unitNodeGenerator},{id:2,deps:[],gen:unitNodeGenerator},{id:3,deps:[1],gen:double},{id:4,deps:[0,3],gen:add},{id:5,deps:[2],gen:double},{id:6,deps:[4,5],gen:add},],outputs:{6:"res",},binding:{0:[4],1:[3],2:[5],3:[4],4:[6],5:[6],},}
167+
### Components
168+
Below are the basice components. You could create custom task flow components based on theme. Please note that only function components are supported.
169+
``` typescript
170+
declare function NodeComponent(props: NodeProps): any;
171+
declare function WorkflowComponent(props: WorkflowProps): any;
172+
declare function InputNodeComponent(props: InputNodeProps): any;
173+
declare function OutputNodeComponent(props: OutputNodeProps): any;
74174
```
75-
Here you need to import **unitNodeGenerator** from the library, and other node generator functions.
76175
77-
Also, if you are using typescript, there is cli which could generate the code by specifying the file path and exported workflow instance. Please check packages of **example/client/**,
176+
## About react
177+
The library uses *react* to express task flow. It would add extra bundle size. as 1) *react* is included, 2) the jsx taskflow expression would actually be converted to code to create react elements.
178+
179+
If you don't want to depends on *react* and minize the bundle size. You could 1) dump the workflow instance code with *dumpWorkflow* method at runtime or test code. And use the generated code to generate task flow instance, 2) use cli and package target to print the genrated code for pure typescript project. Please reference the workflow target in **./example/client**.
78180
```
79181
"workflow": "taskflow-react-cli -w ./src/controller/FreWorkflow.tsx -n workflowDef"
80182
```
81183
82-
-w specifies the file path and -n specfies the name of workflow instance exported.
184+
-w specifies the file path and -n specfies the name of workflow instance exported. Here you need to import unitNodeGenerator from the library, and other node generator functions.
185+
186+
187+
## UI application
188+
The best scenario for task flow is pure data flow. For task that needs to wait for user's input like clicking button, please keep the resolve function. When user clicks the button, you run the resolve function with input needed.
189+
190+
For UI specify task flow, it is recommended to define something like bridge to keep state and callback functions to update UI. Please check **./example/client** for reference.

bin/cli.js

Lines changed: 1 addition & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

cli/Cli.ts

Lines changed: 20 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,21 @@
11
import {program} from "commander"
22
import {resolve} from "path"
3-
import {rmSync, writeFileSync} from "fs"
3+
import {mkdirSync, rmSync, writeFileSync} from "fs"
44
import {exec} from "child_process"
55

66
program.option('-w, --workflow <type>', 'relative path to workflow file')
77
program.option('-n, --name <type>', 'workflow name to compile in the file')
88
program.parse(process.argv)
99
const options = program.opts()
10+
const wfBuildGenFolder = '_workflow_gen_'
11+
const wfBuildCompileFolder = '_workflow_compile_'
1012
const wfBuildTsConfigFileName = 'tsconfig.workflow_build.json'
1113
const wfBuildTsConfigContent =
1214
`{
1315
"compilerOptions": {
1416
"target": "es5",
1517
"module": "CommonJS",
16-
"outDir": "_workflow_compile_",
18+
"outDir": "../${wfBuildCompileFolder}",
1719
"lib": ["ESNext", "dom"],
1820
"allowJs": false,
1921
"sourceMap": true,
@@ -30,36 +32,37 @@ const wfBuildTsConfigContent =
3032
}
3133
`
3234

33-
const dumpWorkflowScriptName = "dump_workflow_script.ts"
34-
35+
const dumpWorkflowScriptModuleName = "dump_workflow_script"
3536
if (options.workflow && options.name) {
36-
rmSync(resolve(wfBuildTsConfigFileName), {force: true, recursive: true})
37-
rmSync(resolve(dumpWorkflowScriptName), {force: true, recursive: true})
38-
rmSync(resolve("./_workflow_compile_"), {force: true, recursive: true})
37+
rmSync(resolve(`./${wfBuildGenFolder}`), {force: true, recursive: true})
38+
rmSync(resolve(`./${wfBuildCompileFolder}`), {force: true, recursive: true})
3939

40+
mkdirSync(resolve(`./${wfBuildGenFolder}`))
4041
let workflowTrimPath = options.workflow.substring(0, options.workflow.lastIndexOf('.'))
41-
let dumpWorkflowScript = `
42+
let dumpWorkflowScript = `
4243
import { buildJsxWorkflow } from "taskflow-react"
4344
import { dumpWorkflow } from "taskflow-react"
44-
import {${options.name}} from "${workflowTrimPath}"
45+
import {${options.name}} from "../${workflowTrimPath}"
4546
const workflow = buildJsxWorkflow(${options.name})
47+
const workflowWithName = buildJsxWorkflow(${options.name}, true)
4648
export const dumpWorkflowText = dumpWorkflow(workflow)
49+
export const dumpWorkflowwTextWithName = dumpWorkflow(workflowWithName)
4750
`
48-
writeFileSync(resolve(`./${wfBuildTsConfigFileName}`), wfBuildTsConfigContent)
49-
writeFileSync(resolve(`./${dumpWorkflowScriptName}`), dumpWorkflowScript)
50-
exec(`tsc --project ${wfBuildTsConfigFileName}`, (error, stdout, stderr) => {
51+
writeFileSync(resolve(`./${wfBuildGenFolder}/${wfBuildTsConfigFileName}`), wfBuildTsConfigContent)
52+
writeFileSync(resolve(`./${wfBuildGenFolder}/${dumpWorkflowScriptModuleName}.ts`), dumpWorkflowScript)
53+
exec(`tsc --project ./${wfBuildGenFolder}/${wfBuildTsConfigFileName}`, (error, stdout, stderr) => {
5154
if (error) {
5255
console.error(`exec error: ${error}`);
5356
console.log(`stdout: ${stdout}`);
5457
console.error(`stderr: ${stderr}`);
5558
return;
5659
}
5760

58-
const result = require("../../../_workflow_compile_/dump_workflow_script")
59-
console.log("\x1b[32m", result["dumpWorkflowText"])
61+
const result = require(`../../../${wfBuildCompileFolder}/${wfBuildGenFolder}/${dumpWorkflowScriptModuleName}`)
62+
console.log("\x1b[32m", "export const " + options.name + "Gen = " + result["dumpWorkflowText"] + "\n\n")
63+
console.log("\x1b[32m", "export const " + options.name + "GenWithName = " + result["dumpWorkflowwTextWithName"] + "\n\n")
6064
console.log("\x1b[37m")
61-
rmSync(resolve(wfBuildTsConfigFileName), {force: true, recursive: true})
62-
rmSync(resolve(dumpWorkflowScriptName), {force: true, recursive: true})
63-
rmSync(resolve("./_workflow_compile_"), {force: true, recursive: true})
65+
rmSync(resolve(`./${wfBuildGenFolder}`), {force: true, recursive: true})
66+
rmSync(resolve(`./${wfBuildCompileFolder}`), {force: true, recursive: true})
6467
});
6568
}

0 commit comments

Comments
 (0)