Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a way to distinguish the master from workers? #38

Open
tatatodd opened this issue Jan 15, 2020 · 3 comments
Open

Is there a way to distinguish the master from workers? #38

tatatodd opened this issue Jan 15, 2020 · 3 comments

Comments

@tatatodd
Copy link

I have a use-case where I need to run logic on the master before I kick off the bigslice session, which then goes and runs the workers. The specific use-case is that I have a single binary that must capture and dump a state file (in this case Kafka offsets) before the bigslice session, for correctness.

Is there a simple way to distinguish the master from the workers? I've looked through the various bigslice packages, and haven't found anything. Ideally I'd have something like this:

package main

func main() {
    if bigslice.IsMaster() {
        // Do master-only stuff, before running the bigslice session
    }

    fn := bigslice.Func(...)
    exec.Must(...)
}
@tatatodd
Copy link
Author

FYI @mariusae @cosnicolaou

@tatatodd
Copy link
Author

FYI I've found a hacky workaround, by checking for the BIGMACHINE_MODE envvar, which is set to machine on workers started by bigmachine, and is empty on the master (driver).

This happens to work on my setup, but it's obviously a bit brittle; e.g. there's no reason the bigslice master couldn't also have been spawned by bigmachine. So it'd be nice to have a bigslice-centric robust way to determine whether something is a driver (e.g. perhaps the master registers a name somewhere, and the workers get an envvar with that name in it). But my workaround alleviates my short-term need.

@jcharum
Copy link
Contributor

jcharum commented Jan 18, 2020

Checking BIGMACHINE_MODE is how bigmachine determines its behavior internally, so it's guaranteed to work on your setup (at least as currently implemented).

e.g. there's no reason the bigslice master couldn't also have been spawned by bigmachine

Do you mean by starting a bigslice session within an RPC? If so, you'd probably want/need to fork (with BIGMACHINE_MODE unset), as workers run the same binary as the master. Their behavior diverges at the call to start bigmachine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants