Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add blocking submission of tasks until worker becomes available #57

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

AntonioMorales97
Copy link

By creating a blocking submission alternative in worker pool the size of task queue can be regulated/controlled. This could be useful when you want to limit the size of task queue, e.g. to the number of workers.

This PR includes small changes similar to SubmitWait where the only difference is to close the "done" channel before executing the actual task from the worker, like so:

if waitForWorker {
    close(doneChan)
    task()
} else {
    task()
    close(doneChan)
}

@gammazero
Copy link
Owner

One of the reasons this was not implemented was because it can be provided by submitting a task that closes a channel when the task is run. This can also be done with a generic wrapper that allows any function to be submitted and return when a worker starts executing a task. Same can be said for SubmitWait but that was included because it is such a frequent use case.

func waitWorker(wp *workerpool.WorkerPool, f func()) {
	started := make(chan struct{})
	wp.Submit(func() {
		close(started)
		f()
	})
	<-started
}

More examples: https://go.dev/play/p/ZLKC4JRcZRa

The purpose of the workerpool is to be non-blocking and queue tasks when there are not enough workers. I would prefer not to include code that encourages using this as a blocking workerpool since this is not what this implementation is for. If the purpose of this PR is to not submit more tasks until worker(s) are available, then perhaps better to start some number of goroutines that all read from a channel, i.e. a blocking workerpool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants