Introduce builtin function govm and makechan #330

Bai-Yingjie · 2021-03-29T14:33:33Z

Hi,

I've added 2 built-in functions: go and makechan, to run multiple VMs in a similar way that native Golang's go and channel do.

go(fn, arg1, arg2, ...) starts an independent concurrent goroutine which runs fn(arg1, arg2, ...)

If fn is CompiledFunction, the current running VM will be cloned to create a new VM in which the CompiledFunction will be running.
The fn can also be any object that has Call() method, such as BuiltinFunction, in which case no cloned VM will be created.
Returns a goroutineVM object that has wait, result, abort methods.

The goroutineVM will not exit unless:

All its descendant goroutineVMs exit
It calls abort()
Its goroutineVM object abort() is called on behalf of its parent VM
The latter 2 cases will trigger aborting procedure of all the descendant
goroutineVMs, which will further result in README #1 above.

var := 0

f1 := func(a,b) { var = 10; return a+b }
f2 := func(a,b,c) { var = 11; return a+b+c }

gvm1 := go(f1,1,2)
gvm2 := go(f2,1,2,5)

fmt.println(gvm1.result()) // 3
fmt.println(gvm2.result()) // 8
fmt.println(var) // 10 or 11

wait() waits for the goroutineVM to complete up to timeout seconds and
returns true if the goroutineVM exited(successfully or not) within the
timeout. It waits forever if the optional timeout not specified,
or timeout < 0.
abort() triggers the termination process of the goroutineVM and all
its descendant VMs.
result() waits the goroutineVM to complete, returns Error object if
any runtime error occurred during the execution, otherwise returns the
result value of fn(arg1, arg2, ...)

sharedReqChan := makechan(128)

client = func(name, interval, timeout) {
	print := func(s) {
		fmt.println(name, s)
	}
	print("started")

	repChan := makechan(1)
	msg := {chan:repChan}

	msg.data = "hello"
	sharedReqChan.send(msg)
	print(repChan.recv())

	for i := 0; i * interval < timeout; i++ {
		msg.data = i
		sharedReqChan.send(msg)
		print(repChan.recv())
		times.sleep(interval*times.second)
	}
}

server = func(name) {
	print := func(s) {
		fmt.println(name, s)
	}
	print("started")

	for {
		req := sharedReqChan.recv()
		if req.data == "hello" {
			req.chan.send("world")
		} else {
			req.chan.send(req.data+100)
		}
	}
}

clients := func() {
	for i :=0; i < 5; i++ {
		go(client, format("client %d: ", i), 1, 4)
	}
}

servers := func() {
	for i :=0; i < 2; i++ {
		go(server, format("server %d: ", i))
	}
}

// After 4 seconds, all clients should have exited normally
gclts := go(clients)
// If servers exit earlier than clients, then clients may be
// blocked forever waiting for the reply chan, because servers
// were aborted with the req fetched from sharedReqChan before
// sending back the reply.
// In such case, do below to abort() the clients manually
//go(func(){times.sleep(6*times.second); gclts.abort()})

// Servers are infinite loop, abort() them after 5 seconds
gsrvs := go(servers)
if ok := gsrvs.wait(5); !ok {
	gsrvs.abort()
}

// Main VM waits here until all the child "go" finish

// If somehow the main VM is stuck, that is because there is
// at least one child VM that has not exited as expected, we
// can do abort() to force exit.
abort()

More docs:
https://github.com/godevsig/tengo/blob/dev/docs/builtins.md#go

How do you think about it? I am new here, but I love tengo so much, any comments/advices are welcome~

Builtin go(fn, arg1, arg2, ...) starts a goroutine which run fn(arg1, arg2, ...) in a new VM cloned from the current running VM, and returns a job object that has wait, result, abort methods. Builtin makechan(size) accepts an optional size parameter, makes a channel to send/receive object, and returns a chan object that has send, recv, close methods. To be able to access *VM instance that it is running in for go() and makechan(), now VM will always pass VMObj to and only to BuiltinFunction as arg[0].

A goroutineVM is a VM cloned from the current VM in a shared way that the cloned VM(the child) can access globals and constants of the parent VM, but with its own stack and frames. The cloned VM will be running with a specified compiled function as the entrypoint in a new goroutine created by the parent VM.

In golang spec, "go" statement has no return value, which is not the semantic here: we want "govm" returns a goroutineVM object which can be used to wait, abort, or get result of the goroutineVM.

fix typo

d5 · 2021-03-29T20:42:17Z

Hello! Thanks for opening a PR. I really like the idea of adding some concurrency constructs to Tengo. I think this is a good start. But, there's one thing that I have to mention before going into more details. It seems that this change can significantly slow down the running performance of the regular non-concurrent code. (E.g. The Fibonacci benchmark is ~10% slower.) Also, I personally prefer to keep Tengo as minimal and simple as possible. Concurrency can significantly increase the maintenance costs overall, which we cannot afford at the moment.

Bai-Yingjie · 2021-03-30T14:50:47Z

Hi d5,
Thanks for your reply! I will look into the performance drop, right now I have no clue...
The added feature should only affect the running of the cloned VM but even in that case the costs would be

VM creation and wrapper entries in v.frames[0] and v.stack[0]
in the case parser.OpCall of (*VM).run(), a type assertion is added to check if the BuiltinFunction needs vmObj, but this is in Object.Call path, not CompiledFunction call path...

I will dig deeper of the Fibonacci test to understand why...

VM will only pass its vmObj to builtin function that has needvmObj equals true, so that normal builtin functions do not need to notice the change.

Bai-Yingjie · 2021-04-01T14:35:01Z

I have run the Fibonacci bench in cmd/bench several times manually, but the result varies from each run, from master branch I got:

$ ./bench                                 
-------------------------------------     
fibonacci(35)                             
-------------------------------------     
Result:  9227465                          
Go:      78.809522ms                      
Parser:  220.84µs                         
Compile: 347.96µs                         
VM:      8.494114298s  // this number in several runs: 7.840718186s, 10.21890762s, 7.81819645s, 9.664891104s, 9.262624859s 
-------------------------------------     
fibonacci(35) (tail-call #1)              
-------------------------------------     
Result:  9227465                          
Go:      129.355829ms                     
Parser:  64.265µs                         
Compile: 268.308µs                        
VM:      8.971564671s // this number in several runs: 7.64307332s, 9.831562385s, 9.898320392s, 9.163809704s, 7.817868341s
-------------------------------------     
fibonacci(35) (tail-call #2)              
-------------------------------------     
Result:  9227465                          
Go:      586ns                            
Parser:  56.161µs                         
Compile: 175.593µs                        
VM:      58.555µs  // this number in several runs: 109.931µs, 31.754µs, 41.383µs, 83.342µs, 42.716µs

And with this feature branch, on the same X86 PC, I got:

$ ./bench_govm
-------------------------------------
fibonacci(35)
-------------------------------------
Result:  9227465
Go:      119.630189ms
Parser:  60.796µs
Compile: 380.181µs
VM:      9.183400119s // this number in several runs: 7.647262488s, 8.710117103s, 8.138667236s, 7.610809091s, 8.748246639s
-------------------------------------
fibonacci(35) (tail-call #1)
-------------------------------------
Result:  9227465
Go:      134.411886ms
Parser:  133.77µs
Compile: 365.933µs
VM:      9.972241885s // this number in several runs: 7.123116778s, 9.14368598s, 7.47340407s, 6.671002294s, 11.595864038s
-------------------------------------
fibonacci(35) (tail-call #2)
-------------------------------------
Result:  9227465
Go:      740ns
Parser:  69.846µs
Compile: 495.052µs
VM:      45.137µs // this number in several runs: 113.465µs, 82.669µs, 44.878µs, 45.22µs, 46.951µs

I know it is just a manual run, but from the result, it looks to me more about the result variation of the same binary in different runs...

I will try to add more concurrency test to verify this goroutineVM feature.

The goroutineVM will not exit unless: 1. All its descendant VMs exit 2. It calls abort() 3. Its goroutineVM object abort() is called on behalf of its parent VM The latter 2 cases will trigger aborting procedure of all the descendant VMs, which will further result in d5#1 above.

geseq · 2021-04-02T05:34:22Z

As far as I can tell the fib benchmark itself shouldn’t be affected as there’s nothing in the execution path within the vm that has changed. I ran it a few times and it’s mostly inconclusive.

geseq · 2021-04-02T05:38:40Z

While I agree that this significant maintenance cost, I think adding concurrency constructs might be worth it.

Take below snipet as an example: $ cat cmd/tengo/test.tengo test1 := func() { v := 1 test2 := func() { len(1,1,1,1) v++ } test2() } test1() ---- before the fix: $ ./tengo test.tengo Runtime Error: wrong number of arguments in call to 'builtin-function:len' at - at test.tengo:7:2 at test.tengo:10:1 ---- after the fix: $ ./tengo test.tengo Runtime Error: wrong number of arguments in call to 'builtin-function:len' at test.tengo:4:3 at test.tengo:7:2 at test.tengo:10:1

with this commit: 1. builtinGovm() now accepts native golang function, but unlike compiled functions which run in a cloned VM, native golang functions do not need VM, they just run in a goroutine. 2. Parent VMs now have records of errors of all its descendent VMs. A VM's error is a combination of the error of itself and all errors of the descendent VMs, in another word, parent VM's error includes all errors of the child VMs. 3. ErrVMAborted is no longer treated as runtime error.

now that "govm" does not always run function in VM, so "go" is better. update docs to reflect the usage changes.

Bai-Yingjie · 2021-04-15T15:23:33Z

Thanks @geseq
Regarding the complexity of adding concurrency VM, fully agree that more concurrency constructs like mutex are needed, which brings more complexity... but worth it.
I will continue to test this go VM feature in my dev branch, to make it more stable.

And now abort() on the same VM twice will not panic.

On a panic happens in a VM: * the panic value along with both its script level and native level callstack are recorded as run time error, its parent VM should examine this error. * the panic will not cause whole program exits, but it does trigger the abort chain, so that the VM in which the panic happens and all its descendent VMs will terminate. So panic only causes the descendent group of VMs to exit. Other VM groups are not affected.

Export VMObj to std.fmt, so that fmtPrintln can get vm.Out to output to a io.Writer per VM.

and nolonger hide ErrVMAborted

jvegaseg · 2021-05-26T10:48:08Z

What about this PR. I think it could afford my needs: #335

and nil child VM to gc after it is done

This reduce memory foot print significantly when there are large number of small VMs running.

geseq · 2021-06-01T20:15:31Z

@jvegaseg I think @Bai-Yingjie is still working on improving the stability of this. We’ll re-evaluate once it’s ready.

Bai-Yingjie added 2 commits March 27, 2021 12:42

Bai-Yingjie changed the title ~~Introduce builtin function go and makechan~~ Introduce builtin function govm and makechan Mar 29, 2021

Bai-Yingjie added 3 commits March 29, 2021 15:24

Rename builtin go to govm

d67644e

In golang spec, "go" statement has no return value, which is not the semantic here: we want "govm" returns a goroutineVM object which can be used to wait, abort, or get result of the goroutineVM.

Add docs for builtin "govm" and "makechan"

e4ca97e

fix typo

Use unexported vmObj

ee08782

Use needvmObj flag in BuiltinFunction

9e5fbbc

VM will only pass its vmObj to builtin function that has needvmObj equals true, so that normal builtin functions do not need to notice the change.

Bai-Yingjie added 3 commits April 14, 2021 16:46

rename builtin govm back to go

c81780b

now that "govm" does not always run function in VM, so "go" is better. update docs to reflect the usage changes.

Bai-Yingjie added 6 commits April 15, 2021 16:36

Wrap main VM's error to make TestVMErrorUnwrap test pass

cf3fef9

And now abort() on the same VM twice will not panic.

update docs for builtin go

4462ad9

Make VM's input/output redirectable

496574d

Export VMObj to std.fmt, so that fmtPrintln can get vm.Out to output to a io.Writer per VM.

Add Args to VM so that each VM can have its own os.Args

b6d2aba

make times.sleep abortable

a14d22e

and nolonger hide ErrVMAborted

Bai-Yingjie added 4 commits May 27, 2021 17:54

only treat ErrVMAborted in root VM as error

367afb1

fix times.sleep goroutine leak on abort case

b979ae1

check aborting status before starting child VM

3a929e2

and nil child VM to gc after it is done

Start from small stack and frame space and grow on demand

4cfaeeb

This reduce memory foot print significantly when there are large number of small VMs running.

misiek08 mentioned this pull request Feb 21, 2022

Run CompiledFunction in VM #372

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce builtin function govm and makechan #330

Introduce builtin function govm and makechan #330

Bai-Yingjie commented Mar 29, 2021 •

edited

d5 commented Mar 29, 2021

Bai-Yingjie commented Mar 30, 2021

Bai-Yingjie commented Apr 1, 2021

geseq commented Apr 2, 2021

geseq commented Apr 2, 2021

Bai-Yingjie commented Apr 15, 2021

jvegaseg commented May 26, 2021

geseq commented Jun 1, 2021

Introduce builtin function govm and makechan #330

Are you sure you want to change the base?

Introduce builtin function govm and makechan #330

Conversation

Bai-Yingjie commented Mar 29, 2021 • edited

d5 commented Mar 29, 2021

Bai-Yingjie commented Mar 30, 2021

Bai-Yingjie commented Apr 1, 2021

geseq commented Apr 2, 2021

geseq commented Apr 2, 2021

Bai-Yingjie commented Apr 15, 2021

jvegaseg commented May 26, 2021

geseq commented Jun 1, 2021

Bai-Yingjie commented Mar 29, 2021 •

edited