Add option to ignore stdin data

[zapman449]

http: error: Request body (from stdin or a file) and request data (key=value) cannot be mixed.

This error is rather cryptic. I'm writing a shell script to interact with a rest API, and I'm doing this:

http --output subnet --pretty format --form post "${URLHOST}subnets" "cidr=${cidr}" "name=${name}" "description=${name}" "availabilityZoneId=${az}" "networkId=${VPCid}" "$ACCESSKEY" "$SECRETKEY" "$ACCOUNTID" "_providerId=aws" "_regionId=${REGION}"

Which produces (when preceded by 'echo' a command that works perfectly well. However, this version gives the error in the subject.

Am I being stupid, or is this something else?

FYI: The result of the above is as followed (sanitized slightly)

http --output subnet --pretty format --form post https://test.grid.domain.com/subnets cidr=10.124.33.128/25 name=Load_balancers description=Load_balancers availabilityZoneId=us-east-1c networkId=vpc-31433e7e x-gridauth-accesskey:DOYOUNEEDTOKNOW x-gridauth-secretaccesskey:WHATISWITHALLTHEQUESTIONS x-gridauth-accountid:11223344556677 _providerId=aws _regionId=us-east-1

[jkbrzt]

This error happens when HTTPie's STDIN is redirected and data is also specified in the arguments at the same time:

$ echo 'data' | http POST example.org more=data   # This is invalid

[zapman449]

I'm not redirecting STDIN at all. Here's a bit more context in the script:

cat keyfile  | while read name az route cidr subnetid rest ; do
    http --output subnet --pretty format --form post \
    "${URLHOST}subnets" \
    "cidr=${cidr}" \
    "name=${name}" \
    "description=${name}"\
    "availabilityZoneId=${az}" \
    "networkId=${VPCid}"\
    "$ACCESSKEY" \
    "$SECRETKEY"\
    "$ACCOUNTID" \
    "_providerId=aws" "_regionId=${REGION}"

(and then I parse the 'subnet' file, and go on to do other things)

What I MIGHT be doing is intermixing form-fields with http-headers. However, I couldn't get that to work right either (some combination of pushing the form data through STDIN, and headers as parameters... or as headers and form data through STDIN, or whatever).

[jkbrzt]

I see, the problem is that in the loop context HTTPie inherits its STDIN (which is redirected).

You should be able to work it around by changing HTTPie's STDIN back to the terminal input (< /dev/tty):

cat file | while read line; do 
    http POST example.org  a="$line" < /dev/tty
done

[zapman449]

Wow. That works. I had NO idea that the inner statements of a shell loop inherit the stdin of the surrounding loop.

Thank you for the help... I'm not sure I would have found that on my own.

(Now I'm curious why (I'm assuming all SH derived shells) made that design choice... )

[jkbrzt]

I think there should be an option to disable the reading of STDIN that would work everywhere, so I'm keeping this open and tagging it as a feature.

[CrazyPython]

This is very confusing, and goes against httpie being able to work anywhere. Using such a command inside a bash loop is a very common application. Httpie should not complain when key/value is given and it receives stdin: instead letting key/value dominate.

[sigmavirus24]

Httpie should not complain when key/value is given and it receives stdin: instead letting key/value dominate.

That maybe is intuitive to you, but to many others the exact opposite is far more intuitive. Because tooling should not try to guess, it is far better to raise an error and provide you an escape hatch if you feel you know better, which this does.

[jenstroeger]

I ran into this issue when using httpie inside of a function passed to GNU parallel:

function call_api {
    local FOO=`http --json --auth $TOKEN: post $HOST/api/foo name="Ţẽṧẗ" < /dev/tty | python3 -c "import json,sys;obj=json.load(sys.stdin);print(obj['result']['id']);"`
    […]
    http --download --auth $TOKEN: GET $HOST/api/bla > /dev/tty
}

export -f call_api
export HOST=…
export TOKEN=…
parallel --env HOST --env TOKEN ::: call_api call_api call_api call_api call_api

Thanks to the above comment the parallel function invocation works now. Leaving this comment here for future reference 😊

[sgoll]

For future reference, there now is a CLI argument --ignore-stdin that can be used to explicitly tell HTTPie that request data (key=value) should be preferred over request body (from stdin).

http --ignore-stdin http://… lorem=ipsum dolor=sit

Cf. https://httpie.org/docs#redirected-input

[rcorre]

Encountered this while using http in a for loop. It worked fine locally, but failed in gitlab. --ignore-stdin resolved it.

[NightMachinery]

I tried redirecting stdin to null </dev/null, but it made httpie behave weird. Why's that? I mean, shouldn't there be a way to make a command ignore stdin from the shell itself?

[jkbrzt]

@NightMachinary you want to use the --ignore-stdin option. That’s how you tell HTTPie to not read STDIN.

$ echo 'this STDIN data is ignored' | http  --ignore-stdin POST httpbin.org/post hello=world

[NightMachinery]

@jakubroztocil I know that, I am saying if http did not have --ignore-stdin, how could we eliminate stdin going to http from the shell itself?

[jkbrzt]

@NightMachinary you cannot eliminate STDIN (and the other standard streams), but you can close it:

$ echo ignored data | http httpbin.org/anything 0<&-

https://superuser.com/questions/813472/how-do-i-close-stdin-in-a-shell-script

[jkbrzt]

@NightMachinary and redirecting /dev/null is a bit different:

$ http httpbin.org/anything < /dev/null

This still invokes HTTPie with an open & redirected STDIN, so HTTPie will read it and end up with an empty string, which it will then happily use as the request body data (effectively defaulting to POST), unless you tell it not to with --ignore-stdin.

So it’s basically equivalent to this:

$ echo -n | http httpbin.org/anything