Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve processor stopped state management #449

Merged
merged 6 commits into from May 16, 2024

Conversation

jalaziz
Copy link
Contributor

@jalaziz jalaziz commented Apr 6, 2024

  • Introduce a new Stopped state for processors. Since a processor cannot be restarted after it has been run, this state better indicates that the processor has reached its final state.
  • Expose a new Done() method on the processor to allow waiting on a processor to complete. This avoids the need to apply additional "done" channels on top of the processor when processors in go routines.

Copy link
Contributor

@frairon frairon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution @jalaziz, sorry for the delay!

processor_test.go Show resolved Hide resolved
@frairon
Copy link
Contributor

frairon commented Apr 28, 2024

😄 the deadlock is removed, but the test reports a data-race.

* Introduce a new `Stopped` state for processors. Since a processor
  cannot be restarted after it has been run, this state better indicates
  that the processor has reached its final state.
* Expose a new `Done()` method on the processor to allow waiting on a
  processor to complete. This avoids the need to apply additional "done"
  channels on top of the processor when processors in go routines.
@jalaziz
Copy link
Contributor Author

jalaziz commented Apr 29, 2024

@frairon So actually, there is a data race.

The data race happens because the channel will be closed before the error is returned from Run.

I could fix the test by introducing an extra done channel for the text. However, another way to fix this is to have the done channel return the error that Run returns. This is kind of nice as you can handle the error asynchronously.

What do you think?

@frairon
Copy link
Contributor

frairon commented May 3, 2024

True that, there is a data race, but the data race is in the test only. If we add a synchronizing channel or a mutex it's fixed. But of course, that's not optimal.

On the other hand, returning the error through the done-channel also doesn't work, because only the first consumer might get the error. As soon as multiple clients wait for the processor it becomes inconsistent. We could create a new errors-channel each time the method is called and copy the error, which is even uglier.

So what if we instead provide an Error() method on the processor that returns the last error? Clients could call when either Run() returned or the channel from Done() is closed, if they call it earlier - they'll get a race-error in their tests. Or we protect it with a mutex to be sure.

What do you think about that?

@jalaziz
Copy link
Contributor Author

jalaziz commented May 7, 2024

@frairon I have implement what you suggested. Let me know what you think.

I'm a bit torn on whether all the tests should be updated to use Done() and Error(). Happy to update the tests if you think they should.

@frairon
Copy link
Contributor

frairon commented May 7, 2024

@jalaziz thanks for this, really! Since we have the Done()-method now, I think it does make sense to use it in the tests. I mean why do the dance with the defer close(done) everywhere if you don't have to. But I'd understand if you don't want to change all the tests. But if you do - be my guest :).

@jalaziz
Copy link
Contributor Author

jalaziz commented May 7, 2024

@frairon Updated the tests, but I added two tests to make sure the tests still covered the returned error.

frairon
frairon previously approved these changes May 9, 2024
@frairon
Copy link
Contributor

frairon commented May 9, 2024

Looks good, thanks! @norbertklawikowski what do you think?

@norbertklawikowski
Copy link
Contributor

LGTM, just a tiny comment :)

Copy link
Contributor

@norbertklawikowski norbertklawikowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks a lot!

@frairon frairon merged commit 90c88b0 into lovoo:master May 16, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants