Fabric does not pass the environment to a local shell like Invoke does

[justinas]

Consider this task definition, saved as both fabfile.py and tasks.py:

from invoke import task

@task
def make(c):
    c.run('env')

fab make outputs:

PWD=/tmp
SHLVL=1
_=/usr/bin/env

inv make outputs:

LC_ALL=en_US.UTF-8
NVM_DIR=/home/justinas/.nvm
LC_MEASUREMENT=en_US.UTF-8
LC_PAPER=en_US.UTF-8
LC_MONETARY=en_US.UTF-8
<...> (lots of stuff in my environment)

I initially found this when trying to use fpm which checks the PATH variable to check that the executables it requires to function exist:

from invoke import task

@task
def make(c):
    c.run('fpm -s dir -t rpm -C dist --name somename .')

This works with invoke, but not with fab.

justinas@js:/tmp$ fab make
{:timestamp=>"2018-05-11T13:11:26.218874+0300", :message=>"Need executable 'rpmbuild' to convert dir to rpm", :level=>:error}
justinas@js:/tmp$ inv make
{:timestamp=>"2018-05-11T13:11:29.967762+0300", :message=>"Created package", :path=>"somename-1.0-1.x86_64.rpm"}

[justinas]

I discovered the description of this behavior. However, it makes no sense when running the command locally

[ploxiln]

This is typical for running commands on remote systems:

$ env | wc -l
      41
$ ssh testdeploy01.ec2.st-av.net env | wc -l
      14
$ ssh testdeploy01.ec2.st-av.net grep Env /etc/ssh/sshd_config
AcceptEnv LANG LC_*

 AcceptEnv
         Specifies what environment variables sent by the client will be copied into
         the session's environ(7).  See SendEnv in ssh_config(5) for how to configure
         the client.  The TERM environment variable is always sent whenever the
         client requests a pseudo-terminal as it is required by the protocol.  Vari-
         ables are specified by name, which may contain the wildcard characters `*'
         and `?'.  Multiple environment variables may be separated by whitespace or
         spread across multiple AcceptEnv directives.  Be warned that some environ-
         ment variables could be used to bypass restricted user environments.  For
         this reason, care should be taken in the use of this directive.  The default
         is not to accept any environment variables.

One could try to pass through all environment variables by collecting the whole local environment and setting them on the other side, but there are quite a number which would need to be ignored because they would conflict with the appropriate value for the remote system, like PATH, GOPATH (if you use the Go language), TMPDIR, HOME, SSH_AUTH_SOCK, ...

[justinas]

Note that I am talking solely about local tasks. I have even discovered a TODO comment that confirms that passing environment should be on by default for local commands.

[ploxiln]

ah, I see, makes sense

[timonweb]

@justinas I ended up using run from invoke for local tasks, I import it like this:

from invoke import run as local

@task
def test(c):
    local('ls')

[bitprophet]

This is a facet of #1752 - the intent is for local to behave like Invoke and preserve the env, and for run to behave properly like SSH and discard the env. However, right now the configurations driving this behavior are not properly split up - it's effectively a bug. We'll be fixing it soon.

[justinas]

Thanks for the response!

[mtorromeo]

I just bumped into the same issue and I just wanted to note that it's not just the local environment that gets discarded.

Maybe you are already aware of this bug also.

To clarify, this didn't work for me:

@task
def test(c):
    c.run('echo $ENV', env={'ENV': 'production'}) # no output

For the time being I found this workaround (not very clean) that I can use:

@task
def test(c):
    with c.prefix('ENV=production'):
        c.run('echo $ENV') # prints "production"

[bitprophet]

Yes, aware of that, thanks!

[bitprophet]

You can say run('...', preserve_env=True) to work around that for now, IIRC.

[mtorromeo]

preserve_env doesn't seem to exist. Maybe you were referring to replace_env=False?
That doesn't trigger an error but the environment is still being discarded.

PS: I need to set the environment for remote tasks, not local.

[bitprophet]

Yea I meant replace_env and it's weird that that doesn't work. I'll take a look when I get to splitting up the config between local/remote (#1752)

[aviddiviner]

I just ran into this as well. I was doing c.run('make build') and none of my ENV pulled through (specifically GOPATH which caused the build to fail). Adding the replace_env=False fixed it.

Edit: I see according to the docs:

run.replace_env: True, instead of False, so that remote commands run with a ‘clean’, empty environment instead of inheriting a copy of the current process’ environment.

It was just a bit confusing to me since I was trying to run a local command.

[dbsxdbsx]

The issue is still there when running local command. @bitprophet