invalid argument when dir has environment variable substitution

[culhatsker]

Running this simple DAG

env:
  - WDIR: somedir
steps:
  - name: step1
    dir: ${WDIR}
    command: bash
    script: |
      mkdir -p ${WDIR}
      cd ${WDIR}
      pwd

Would yield this result:

2024/04/25 16:43:02 frontend is running at "/tmp/@dagu-test2-00990e9229c3ef5fec5dd68a0ae43e0c.sock"
2024/04/25 16:43:02 start running: step1
2024/04/25 16:43:02 step1 failed
2024/04/25 16:43:02 schedule finished.
2024/04/25 16:43:02 
Summary ->
+--------------------------------------+-------+---------------------+---------------------+--------+--------+------------------+
| REQUESTID                            | NAME  | STARTED AT          | FINISHED AT         | STATUS | PARAMS | ERROR            |
+--------------------------------------+-------+---------------------+---------------------+--------+--------+------------------+
| f340d5a8-3f04-494c-9fe7-5d61d96cbe42 | test2 | 2024-04-25 16:43:02 | 2024-04-25 16:43:02 | failed |        | invalid argument |
+--------------------------------------+-------+---------------------+---------------------+--------+--------+------------------+
Details ->
+---+-------+---------------------+---------------------+--------+---------+------------------+
| # | STEP  | STARTED AT          | FINISHED AT         | STATUS | COMMAND | ERROR            |
+---+-------+---------------------+---------------------+--------+---------+------------------+
| 1 | step1 | 2024-04-25 16:43:02 | 2024-04-25 16:43:02 | failed | bash    | invalid argument |
+---+-------+---------------------+---------------------+--------+---------+------------------+

Running dagu 1.12.11 on Ubuntu 22.04

[culhatsker]

Sorry, while making a minimal reproduction I made a mistake: ${WDIR} does not exist at the moment of running step1 but it depends on it.

Here's another example when the problem occurs:

env:
  - WDIR: ./somedir
steps:
  - name: create dir
    command: bash
    script: |
      rm -rf ${WDIR}
      mkdir -p ${WDIR}
  - name: interact with dir
    dir: ${WDIR}
    command: bash
    script: |
      pwd

But everything works as expected if you change this to:

env:
  - WDIR: ./somedir
steps:
  - name: create dir
    command: bash
    script: |
      rm -rf ${WDIR}
      mkdir -p ${WDIR}
  - name: interact with dir
    command: bash
    script: |
      cd ${WDIR}
      pwd

notice: second step doesn't have dir: ${WDIR} option but has cd ${WDIR} in script

[culhatsker]

Okay, I realized while trying to debug goroutines (of what I have no experience), that my second example is also bad, because I did not set up the dependency that second step has on the first step.
There's still a bug though, but it feels much less important now as I understand my mistake.

[yohamta]

I think that the example needs an adjustment:

env:
  - WDIR: ./somedir
steps:
  - name: create dir
    command: bash
    script: |
      rm -rf ${WDIR}
      mkdir -p ${WDIR}
  - name: interact with dir
    command: bash
    script: |
      cd ${WDIR}
      pwd
+    depends:
+      - create dir

I'd like to close this issue, please reopen if there's more issues.