Any feedback on this?

I also just ran into this and was also surprised

Unfortunately, this is a breaking change. This means we'd have to wait for the next major version release to do this by default, and that can take quite some time to happen.

In the meantime, we can consider having an opt-in setting for the behavior, so those who want to anticipate that can just set a flag.

In my opinion this is a bug. If I set a variable in the command I expect it to run with this configuration. Otherwise the stuff I see in the Taskfile is not what is happening when running the task. This is really confusing for the user.

If you are not willing to change the behavior because of backward compatibility concerns I would prefer to set the opt-in option in the Taskfile, otherwise when I forget it, commands are not working.

Thanks for your feedback!

I kindly agree with @zbindenren considering it a bug.

I also agree that this may be a breaking chance in respect of current behavior even though it was never documented.

I would argue that my pull request is not relevant anymore in respect of the direction you are suggesting.
Shall I close it?

Nevertheless I would consider documenting (and also adding tests) the current behavior as useful.
When creating the PR changing the behavior it did not break any test.

Shall I put some effort into that?

@jaedle I'm sure that some people like and use the current behavior. So, if you're willing to work on it, I think we should have a setting on the Taskfile to opt-in the new behavior. Something like:

disable_os_env_precedence: true

(Consider this name WIP. Naming things is hard. Suggestions are welcome. 🙂)

/cc @pd93. I think you had some ideas on how to deal with breaking changes and flags like this. This may be an opportunity to start discussing it.

@andreynering yes, I've actually started writing up a proposal for breaking changes, but didn't have much time to work on it last week. I'll see if I can get it sorted in the next day or two and I'll link it back to here when I've posted it.

Very quick thoughts:

I'm not a huge fan of adding experimental flags to the Taskfile itself. I think it can muddy the schema a lot - especially if the behaviour may change or be removed entirely in a future major release. However, if we're proposing a permanent change (i.e. users will be able to make this choice going forwards into new major versions), then I don't consider this to be a breaking change at all (as long as the default behaviour stays the same). We can always have a discussion about changing the default behaviour at a later date.

Since this is a hotly debated topic, it might be nice to put this behind an experimental flag before making it a permanent feature of the Taskfile so that users can try it out. That way, we're not making more breaking changes in the future if it's not implemented to everyone's liking. This kind of flag would be enabled with a TASK_X_ENV_PRECEDENCE envvar. This allows users to try out the behaviour with one-off commands (via flag) or by setting up their environment (envvar).

As I said, I'll post something with much more detail soon.

Thanks for your feedback! This is really appreciated.

As we are not sure about the direction up l would feel free to document the current behaviour in tests + docs, if you are fine with it.

@andreynering @jaedle @zbindenren Apologies for the delay, I've been travelling for work a lot lately, but I had some time today to write up the breaking changes proposal. Comments/thoughts are very welcome

The proposal lgtm @pd93 . Nice work. Maybe some additional thoughts:

• how long is a flag experimental? Maybe we should do it like kubernetes does. 3-6 months and then it is dropped or added.
• how do we introduce breaking changes to the taskfile yaml itself. Maybe define when and how and under what circumstances to relaease a v4 etc...

Maybe some additional thoughts:

@zbindenren replied to your comments in the proposal discussion to keep everything together.

I would consider this a bug than a breaking change, honestly ...

One of the primary reasons I am moving all the Makefile and scripts to Taskfile is that I can control which .env files I am loading, and hopefully ignore whatever the programmer decided to do for the personal preferences:

version: '3'

dotenv:
  # organization level overwrites .sht.env is reserved
  - '{{.HOME}}/.sht.env'

Kind of related to this, #1384

I couldn't find documentation about the order of loading these env variables ... reading the code something told me I was expecting something that isn't possible.
Would be nice to, at the very least, start with documenting the existing order (regardless if it is "broken" or not).

I also find this as a bug. Expected behavior would be to overwrite environment variables, not respect them. So for example if I want to define a global .dotenv file where I redefine the PATH, why can't I do that?

I just ran into this issue because I want to override the KUBECONFIG env var on a task, here's a snippet of what I would like to be able to do. However since KUBECONFIG is exported in my shell it won't get overridden. I'll have to discover another way to handle this in the meantime.

version: "3"
tasks:
  sync:
    desc: Sync ExternalSecret resources
    summary: task {{.TASK}} [cluster=main] [ns=default]? [secret=plex]?
    cmd: |
      {{if eq .secret ""}}
        kubectl get externalsecret.external-secrets.io --all-namespaces --no-headers -A | awk '{print $1, $2}' \
          | xargs --max-procs=4 -l bash -c 'kubectl -n $0 annotate externalsecret.external-secrets.io $1 force-sync=$(date +%s) --overwrite'
      {{else}}
        kubectl -n {{.ns}} annotate externalsecret.external-secrets.io {{.secret}} force-sync=$(date +%s) --overwrite
      {{end}}
    env:
      KUBECONFIG: "{{.ROOT_DIR}}/kubernetes/{{.cluster}}/kubeconfig"
    vars:
      ns: '{{.ns | default "default"}}'
      secret: '{{ .secret | default ""}}'
    preconditions:
      - kubectl -n {{.ns}} get externalsecret {{.secret}}

@onedr0p the silly workaround for now is to use the default template function on a var and then assign it to the env like this:

# yaml-language-server: $schema=https://taskfile.dev/schema.json
version: '3'

tasks:
    
    default:
      vars:
        KUBECONFIG: '{{ default "/your/default/kubeconfig/path/here" .KUBECONFIG }}'
      env:
        KUBECONFIG: '{{ .KUBECONFIG }}'
      cmds:
        - echo {{.KUBECONFIG}}

this will let you run

23-11-28 20:35:08|~/Desktop|main|+110-95|[!?]|15ms|󰈺 ❯ task
task: [default] echo /your/default/kubeconfig/path/here
/your/default/kubeconfig/path/here

23-11-28 20:36:31|~/Desktop|main|+110-95|[!?]|13ms|󰈺 ❯ KUBECONFIG=ASDF task
task: [default] echo ASDF
ASDF

Then it is definitely a bug and we should fix it.

I am also trying to set how and where KUBECONFIG is aassigned and, following @clintmod suggestion, found some odd behaviour. When including a Taskfile, it doesn't seem like the vars can be interpolated under the includes section. Here is my example:

Taskfile.yml

# https://taskfile.dev

version: '3'

vars:
  KUBECONFIG: '/path/to/test1/kubeconfig.yaml'
  KUBECONFIG_TEST4: '/path/to/test4/kubeconfig.yaml'

includes:
  include-test1:
    taskfile: https://gist.githubusercontent.com/yasn77/85a22ca5fe3fde62247a8a10e8e72310/raw/Taskfile-kubeconfig-test.yml
    vars:
      KUBECONFIG: '{{.KUBECONFIG}}'
  include-test2:
    taskfile: https://gist.githubusercontent.com/yasn77/85a22ca5fe3fde62247a8a10e8e72310/raw/Taskfile-kubeconfig-test.yml
    vars:
      KUBECONFIG: '/path/to/test2/kubeconfig.yaml'

tasks:

  default:
    desc: "default"
    cmds:
      - task: include-test1:get-kubeconfig
      - task: include-test2:get-kubeconfig
      - task: include-test1:get-kubeconfig
        vars:
          KUBECONFIG: '/path/to/test3/kubeconfig.yaml'
      - task: include-test1:get-kubeconfig
        vars:
          KUBECONFIG: '{{.KUBECONFIG_TEST4}}'

Included Gist

# https://taskfile.dev

version: '3'

vars:
  KUBECONFIG: "/path/to/incuded/kubeconfig.yaml"

tasks:
  get-kubeconfig:
    desc: "Show Kubeconfig location"
    env:
      KUBECONFIG: "{{.KUBECONFIG}}"
    cmds:
      - echo "KUBECONFIG Location - $KUBECONFIG" 

Expected Output

➜ task -s
KUBECONFIG Location - /path/to/test1/kubeconfig.yaml
KUBECONFIG Location - /path/to/test2/kubeconfig.yaml
KUBECONFIG Location - /path/to/test3/kubeconfig.yaml
KUBECONFIG Location - /path/to/test4/kubeconfig.yaml

Actual Output

➜ task -s
KUBECONFIG Location - /path/to/incuded/kubeconfig.yaml
KUBECONFIG Location - /path/to/test2/kubeconfig.yaml
KUBECONFIG Location - /path/to/test3/kubeconfig.yaml
KUBECONFIG Location - /path/to/test4/kubeconfig.yaml

Test 1 is not correctly interpolating the variable 😞

If it's deemed needed, I am happy to open a new issue about this

The problem is because of the way variables are implemented in go-task. They're backwards in how they function in a shell and in other build systems. In other build systems they're immutable and designed to be overriden from the top down (first in wins).

In go-task it is bottom up.

Take the above coupled with the fact that you have the concept of vars vs env vars and you get this unexpected co-mingled behavior for something that you only interact with from the command line and do not expect there to be a separate concept between env vars and vars.

Having variables that are mutable, as they're implemnted today will be the source of endless bugs.

Immutability (like in make and ant) is much more predictable and easier to reason about.

I very often organize automation in task files but if some variable is already defined on the OS level there is no way to override it in any of the env sections in taskfile. the last resort is to export/override vars in cmd but this then needs to be copied/pasted to all commands that may leverage that overridden env var which may be ugly to do so and lead to errors.

see my command describing it briefly: #993 (comment)

I like GoTask but this issue is coming back to me over and over again. It would be nice to address mine and other people's problems someday.

FWIW, this conversation reminds me of Environment variables precedence in Docker Compose

I'm not sure if this falls to the same source issue but it seems there is an issue with dotenv precedence as well:

.env.test

FOO=default

Taskfile-test.yml

version: '3'
dotenv:
  - .env.test
tasks:
  default:
    cmds:
      - echo "{{.FOO}}"
      - echo "$FOO"

Output

❯ task -t Taskfile-test.yml
task: [default] echo "default"
default
task: [default] echo "$FOO"
default
❯ FOO=not-default task -t Taskfile-test.yml
task: [default] echo "default"
default
task: [default] echo "$FOO"
not-default

If this is an unrelated issue i'd be happy to open a new one