Support for ephemeral processes

[rupurt]

Howdy 👋,

Love the project. Do you have any plans to support ephemeral processes for things like batch or cron jobs?

[F1bonacc1]

Hi,

Glad you like it :)
Can you elaborate a little bit more about the batch use case?

For cron jobs will something like this work for you:

processes:
  cleaner:
    command: "find ./ -type f -mtime +7 -name '*.bak' -execdir rm -- '{}' \\;"
    working_dir: "/somedir"
    availability:
      restart: "always"
      backoff_seconds: 604800 # every 7 days

Or you are looking for something that follows closer the cron syntax?

[thenonameguy]

That is why the 'PC_REPLICA_NUM' env var is passed to the process, right?

[F1bonacc1]

Yes

[rupurt]

That would be great. If I could dynamically bind to a port it should cover all cases.

[thenonameguy]

AFAIK Kafka consumers don't need to bind a port as they don't serve data themselves.

If you have this need you can always wrap your server exec command with a shell script that does a simple addition using a base port + the replica int env var above.

[rupurt]

Yeah they don't. I meant in the context of other ephemeral processes like a group of web servers

[F1bonacc1]

Hey @rupurt, @thenonameguy,

This became a significant refactoring effort and took much longer than expected.
I am a little bit concerned about releasing it as GA at this point, simply because of all the changes in the underlying logic.
Are you up to checking an RC, if it meets your expectations, if anything is missing or doesn't work as expected?

[thenonameguy]

I think it is better, also the name is more telling.
For consistency maybe also interpolate in the process name env var?

[F1bonacc1]

Sure, I will change it to PC_REPLICA_NUM .
The process name is calculated on the fly based on the number of replicas. I wanted to keep API BW compatibility + unique process name + didn't want to affect the existing process names for users that don't use the scaling/replicas feature.

[F1bonacc1]

@thenonameguy, @rupurt

If everything else works as expected. I will make this an official release.

[rupurt]

Hey @F1bonacc1,

Sorry, I haven't had a chance to test it out yet. Back from vacation so will give it a try this week.

[thenonameguy]

Everything else worked for me very well @F1bonacc1 , spent around an hour playing with it, LGTM.

[dzmitry-lahoda]

I need cron job.

I do not want to code it into my app and run it all the time.

I could have run some cron app script all the time, and it will launch my app. So that would not be so easy to see in PC TUI.

In systemd I have timers

https://akashrajpurohit.com/blog/systemd-timers-vs-cron-jobs/

And I have most degen way - bash with sleep loop for simple case. Will do this.

So systemd like timers feels interesting.

[dzmitry-lahoda]

So cron jobs vs replica sets - seems different things.

Replica sets are complicated. In K8S they just work because of full isolation of containers (none in PC). What rules will be for isolation of processes in PC scaled up?

Another issues, K8S has RS because it also have autoscalers, so actually final features is metric based autoscaler. It pretty fills overkill and will never be target of PC. Right?

Another point, may you just cp yaml block in muliline editor and make clones and restart. I assume PC is not tool to run server side processes for long (serviced/k8s are). So workaround is easy - make yaml.

[dzmitry-lahoda]

So why make YAML is good for scale but not cron ? Visibility in UI. If code bash cron wrapper cannot see in TUI nice view of CRON invocations. So simplicity and speed of rebuild/restart to is what is killer feature of PC, CRON integration for sure adds to that agenda.

[dzmitry-lahoda]

For periodic(not cron) can use simple wrap comand like watch --differences --no-title --interval=3 --exec foobar.

As for scale, I guess before scale, it can be found out that in K8S first goes hot reload of config, so it may be that PC to think of how and why hot reload before scale.

K8S has controllers to define dependencies of reload. So what would be rule to reload when upscale or downscale happens?

[F1bonacc1]

As a temporary workaround for cron jobs you can consider something like:

processes:
  job:
    command: echo Hi
    availability:
      restart: "always"
      backoff_seconds: 604800 # will run once a week