<? try-input redirection

[faho]

Description

This tries to open the given file to use as stdin, and if it fails, for any reason, it uses /dev/null instead.

This is useful in cases where we would otherwise do either of these:

test -r /path/to/file
and read -l foo < /path/to/file
# becomes 
read -l foo <? /path/to/file

cat /path/to/file 2>/dev/null | string match foo
# becomes
string match foo <? foo

This both makes it nicer and shorter, and helps with TOCTTOU - what if the file is removed/changed after the check?

The reason for reading /dev/null instead of a closed fd is that a closed fd will often cause an error.

In case opening /dev/null fails, it still skips the command. That's really a last resort for when the operating system has turned out to be a platypus and not a unix.

Fixes #4865

The main issue here, as discussed in 4865, is that this is arguably inconsistent with >? redirection - where that errors out, <? does not error out. Unfortunately the only solution I can come up with is to make < noclobber and make <? clobber, which also doesn't really seem right and would be an awkward change.

<? also currently cannot be combined with file descriptors - <?&3 errors out. This could be lifted in future ("use this file descriptor if it is open"), tbh I'm fine with this at the moment because we don't have a way to globally open file descriptors anyway so you could really only use stdin, and that's pretty much always open.

TODOs:

• Changes to fish usage are reflected in user documentation/manpages.
• Tests have been added for regressions fixed
• User-visible changes noted in CHANGELOG.rst

Note: The second commit is for illustration, I'm not going to merge it because it's not really necessary and annoying for the people who want new scripts on old fish (which we don't "support" as such, but try not to gratuitously break).

[mqudsi]

I like this in principle but I don't think we should just add this without taking some time to revisit fish scripting overall and put together our ideas in one place and see what sticks and what doesn't. This is a fairly big change (in fish philosophy, not code) and there are "symbol soup" concerns that we have thus far avoided (and even lobotomized out).

The addition of various builtins and subcommands was fine; they're confined to the command itself, they're easy to look up, no one has to use them, and they're fairly self-documenting. In my two cents, changes to the language should be evaluated a lot more carefully.

I think this should be part of a careful, methodical, and holistic review of fish scripting and not something we should just adopt on an ad-hoc basis.

[faho]

The big thing here is that we have redirections as syntax - there is no with-stdin /path/to/file string match -q ... or redirect --maybe /path/to/file or string match -f /path/to/file -q .... The former would be pretty verbose and a big change to how redirections work, the latter would require an argument to every single builtin and would not do anything for commands.

So there is no way to attempt a redirection that can be caught or handled without printing an error (you couldn't really redirect the error anyway).

The alternatives are pretty verbose, subtly wrong (test -r foo && string match < foo always leaves room for foo to change in-between) or require an external command (cat foo 2>/dev/null | string match).

That's why I think that this addition is necessary, and why there is no good alternative using a builtin. We also already use the exact symbol with redirections <? and >?, and ? is pretty intuitive as a fallibility indicator. (yes, the antisymmetry is a shame)

---

In general I'm a fan of rethinking fish script, and I'm not a fan of adding symbols.

Fish's original design was extremely ad-hoc and sometimes incoherent. That's why it had ^ stderr redirection and % expansion, and why it just added & backgrounding instead of a background foo decorator.

But here, where we already have the symbols, I think reusing one as a modifier makes sense.

[krobelus]

require an external command (cat foo 2>/dev/null | string match).

That sounds like a good solution, it's simple and obvious. What's wrong with the external command?

I haven't formed a strong opinion on <? yet but one thing to note is that Bash's $(<path/to/file) can be used to achieve similar goals and we might wanna consider that as well (though that one doesn't support streaming).

If I had to decide now I'd say it's probably fine to add because it's hardly ambiguous.
The fact that unlike bash we refuse to run jobs like cat < nosuchfile | echo foo means the want might be justified.
That being said, this seems like a feature mostly for scripts; if it turns out to be useful in interactive scenarios that would be exciting to me

[faho]

That sounds like a good solution, it's simple and obvious. What's wrong with the external command?

It requires calling an external process, it's pretty verbose.

We could also get by without < - after all it's just cat | foo, so why do we have <?

I haven't formed a strong opinion on <? yet but one thing to note is that Bash's $(<path/to/file) can be used to achieve similar goals

That's a command substitution. It is absolutely not the same thing.

It also can not be used for this, because it will print an error if the file does not exist, and the error is unredirectable.

we might wanna consider that as well

That one is quite literally just a shortcut, and since it syntactically means you need to allow redirections without commands I do not believe that change is a good idea.

It saves three characters at the cost of making the syntax more complex and making errors less detectable - now a missing command is no longer an error if there's a redirection.

(in zsh $(<foo) will end up doing $(cat <foo) - unless you changed $NULLCMD, because of course it's configurable. the bash docs explain that it is "faster" to omit the cat, presumably because bash opens it itself)

[krobelus]

ok. I'm more likely to use cat in scripts but this seems fine.
For race conditions I don't think it's a real fix, only a workaround that prevents some specific issues. A real solution would something like flock

[faho]

For race conditions I don't think it's a real fix, only a workaround that prevents some specific issues

I mean in general unix-style shellscripting has unavoidable TOCTTOUs. It is essentially impossible to avoid.

But here, it is a fix for this specific one - once the file is open it's open.

---

Longer term, I would like to have a thing to open a file at a file descriptor. I would not want to reuse exec for it like posixy shells do, it would look more like

redirect -l file --write /path/to/file # redirect sets $file to the fd, it is closed once $file goes out of scope
foo >$file

redirect --stdout /path/to/file # anything going to stdout now goes to /path/to/file

[faho]

Merged as 2024313.

(like I said, the second commit was for illustration and was not merged)