Quoting on Windows with PowerShell?

[lf-]

I have had a bunch of frustrating experiences searching for string literals in software with ripgrep in PowerShell on Windows, which I initially thought was a ripgrep issue, but it appears to instead be a severe difference between how my shell works and how I expect it to work.

I wrote a C++ program that prints its argv verbatim. I include a few tests below:

PS C:\Users\Jade\dev\ctest> ./test.exe 'a' '2' "3"
ARG 0:: C:\Users\Jade\dev\ctest\test.exe
ARG 1:: a
ARG 2:: 2
ARG 3:: 3
PS C:\Users\Jade\dev\ctest> ./test.exe 'a' '2' '"3"'
ARG 0:: C:\Users\Jade\dev\ctest\test.exe
ARG 1:: a
ARG 2:: 2
ARG 3:: 3
PS C:\Users\Jade\dev\ctest> ./test.exe 'a' '2' ''"3"''
ARG 0:: C:\Users\Jade\dev\ctest\test.exe
ARG 1:: a
ARG 2:: 2
ARG 3:: 3
PS C:\Users\Jade\dev\ctest> $s = '"a"'
PS C:\Users\Jade\dev\ctest> $s
"a"
PS C:\Users\Jade\dev\ctest> ./test.exe 'a' '2' $s
ARG 0:: C:\Users\Jade\dev\ctest\test.exe
ARG 1:: a
ARG 2:: 2
ARG 3:: a
PS C:\Users\Jade\dev\ctest> ./test "`"""a""`""
ARG 0:: C:\Users\Jade\dev\ctest\test.exe
ARG 1:: a
PS C:\Users\Jade\dev\ctest> ./test.exe "$s"
ARG 0:: C:\Users\Jade\dev\ctest\test.exe
ARG 1:: a
PS C:\Users\Jade\dev\ctest> ./test.exe """"wtf""""
ARG 0:: C:\Users\Jade\dev\ctest\test.exe
ARG 1::  wtf

I even thought I was being hit by some incredibly strange libc misfeature so I wrote a rust program which passes a parameter with quotes in it .... and got this:

PS C:\Users\Jade\dev\ctest\what> cargo r
   Compiling what v0.1.0 (C:\Users\Jade\dev\ctest\what)
    Finished dev [unoptimized + debuginfo] target(s) in 0.40s
     Running `target\debug\what.exe`
ARG 0:: ../test.exe
ARG 1:: a
ARG 2:: """b
ARG 3:: "c"

    Command::new("../test.exe")
        .arg("a")
        .arg("\"\"\"b")
        .arg("\"c\"")
        .spawn()
        .unwrap();

I am at my wits' end. How can I search something with double quotes in it on Windows with PowerShell 5? It just seems to eat all double quotes going to external programs, unconditionally. Strangely enough, PowerShell 7 (the open source one) passes a quadruple quoted string (wHat) as single quoted to the callee program. All of the tests above are PowerShell 5.

[BurntSushi]

I personally have no idea. I don't use PowerShell. Someone else will need to field this one. But indeed, ripgrep isn't doing anything funny with its arguments. Have you tried using \"?

@r-darwish maybe you know the answer to this?

[r-darwish]

Apparently, escaping the quotes with \ seems to do the trick in both Powershell 5 and 7. Cmd seems to be the oddest one as it converts the double quotes into single quotes

Powershell has some complex argument conversion magic because it tries to achieve a delicate balance between being a language with actual types but still be useful as a shell language without forcing you to quote every string or escape any hyphen between numbers so it won't be considered as the minus operator. I'm not sure what's CMD's excuse, though.

What I don't understand is why the \ did the trick and not `, which is the actual escape character in powershell, but I'm sure someone in StackOverflow will be happy to provide an answer. I will update this comment if I get an answer about that.

Edit: This answer is the best that I found.

[wistlo]

That is a great answer, but it's from 2011 and does not consider Powershell 7. I am finding simple Perl one-liners that have worked thousands of times for years in PS 5 and older are broken in PS 7. REally simple things, like stripping out empty lines

PS> perl '-0777 -i.p.bak -lpe ""s/\R+/\n/g"" ' $filename

Yes, I could do it in Powershell and wait 5 minutes, or get Perl to do it in 2 seconds. That's the unexaggerated performance difference between Strawberry Perl (the Windows Perl package) and native Powershell text maniuplation on a 300K line csv file.

EDIT: I discovered the double-quotes were corrupting this one-liner in Powershell 7. As there are no "air gaps" in the string, I dropped the double quotes and it worked:

PS> perl '-0777 -i.p.bak -lpe s/\R+/\n/g ' $filename

To see something as fundamental as parsing quote marks change behavior in a new release does shake my confidence in Powershell.

[BillHally]

I am finding simple Perl one-liners that have worked thousands of times for years in PS 5 and older are broken in PS 7.

PowerShell recently made a breaking change to the passing of arguments to native commands.