[perl #121283] system() on Win32 tries to launch bogus paths+bad cmd parsing

ideas from IRC
--------------------------
[16:14] <@xdg> bulk88, I'd suggest avoiding the quoted single quote, too
[16:14] <@xdg> s/wasn't/not/ and see what happnes
[16:15] <Mithaldu> xdg: haha, right :D
[16:16] <@bulk88> "system(1, $^X, '-e', 'sleep 10; print qq[whatever\n]');" that works
[16:16] <@bulk88> should perl system() escape all "s it finds in list args mode?
[16:16] <Mithaldu> bulk88: basically, you can't have "s anywhere
[16:16] <Mithaldu> don't rely on that
[16:16] <@xdg> https://gist.github.com/dagolden/9238751
[16:17] <+dipsy> [ gist:9238751 ] 
[16:17] <Mithaldu> that quoting is incomplete and unreliable for all but the most simple cases
[16:17] <@xdg> bulk88, see my gist
[16:17] <@xdg> that worked for me
[16:17] <@bulk88> "Does exactly the same thing as exec LIST , except that a fork is done first and the parent process waits for the child process to exit. Note that argument processing varies depending on the number of arguments. If there is more than one argument in LIST, or if LIST is an array with more than one value, starts the program given by the first element of the list with arguments given by the rest of the list. If there is only one scalar argument, the argument is checked for shell metacharacters, and if there are any, the entire argument is passed to the system's command shell for parsing (this is /bin/sh -c on Unix platforms, but varies on other platforms). If there are no shell metacharacters in the argument, it is split into words and passed directly to execvp , which is more efficient."
[16:18] <Mithaldu> bulk88: what's your achieve here?
[16:18] <@xdg> bulk88, as I said, that's wrong.  And even perlport doesn't clarify sufficiently
[16:18] <@bulk88> so its not a bug, just undefined behavior?
[16:19] <Mithaldu> bulk88: it does SOME quoting, it's just really shitty and incomplete quoting
[16:19] <Mithaldu> don't bother with it
[16:19] <@xdg> My opinion is that it's probably a bug, but no one understands windows shell quoting in Perl well enough to diagnose, fix or document properly. :-(
[16:20] <Mithaldu> well, there is a proper quoting algorithm, it's just in perl
[16:20] <@bulk88> there is Perl_do_exec3 in doio.c but that is the correct logic, but for nix shells
[16:20] <Mithaldu> https://metacpan.org/source/BINGOS/ExtUtils-MakeMaker-6.90/lib/ExtUtils/MM_Win32.pm#L497
[16:20] <+dipsy> [ lib/ExtUtils/MM_Win32.pm - metacpan.org ] 
[16:21] <Mithaldu> that is the exact logic you mean
[16:21] <Mithaldu> and i don't think anyone wants to implement that in c
[16:21] <@xdg> I'll rephrase.  I'm sure what perl does for system on windows is buggy, but no one seems to be expert enough and motivated enough to fix it
[16:21] <@xdg> bulk88, I think win32/win32.c has the Windows equivalents
[16:21] <@xdg> I've walked through it ages ago but am very rusty
[16:22] <@xdg> there's has_shell_metacharacters and such
[16:22] <@bulk88> the system/exec/backticks code is horrible in win32.c, I stepped it recently, and gave up since IDK what is correct and wrong behavior in it
[16:22] <@bulk88> see https://rt.perl.org/Ticket/Display.html?id=121283
[16:22] <@xdg> exactly.  motivation and expertise are not in the same person
[16:23] <+dipsy> [ Bug #121283 for perl5: system() on Win32 tries to launch bogus paths+bad cmd parsing ] 
[16:23] <Mithaldu> bulk88: as i said, the MM_Win32.pm code linked above is correct, and the system() code is definitely incorrect
[16:23] <@xdg> filing tickets like that is a good way to make sure it doesn't get lost
.................cut.......
[16:23] <@xdg> Mithaldu, I'm not convinced that MM_Win32 would necessarily work with system(1,...) either
[16:23] <Mithaldu> xdg: how so?
[16:25] <Mithaldu> oh right, it's only quoted if there's a single argument, which is useless, since the MM_Win32 code can only quote singular strings in a command invocation
[16:28] <@bulk88> so perl does not escape metacharacters in list mode system() on any OS?
[16:29] <@TonyC> bulk88: it doesn't need to on POSIX systems
[16:29] <@bulk88> and doesn't put quotes around yoru list mode args
[16:30] <@bulk88> because shell is guarenteeded never to be called in list mode system on nix?
[16:30] <Mithaldu> bulk88: "If there is only one scalar argument, the argument is checked for shell metacharacters, and if there are any, the entire argument is passed to the system's command shell for parsing"
[16:30] <Mithaldu> other than that there is no mention of meta characters
[16:30] <Mithaldu> so in list mode it doesn't do anything on any os other than call it
[16:31] <Mithaldu> i think the page for system() should have a red <blink> warning to not use it on windows if you've anything but \w characters
[16:32] <@TonyC> it should be fixed, but tuits
[16:32] <@xdg> except on windows, when list form gets smashed back into a single command line string
[16:32] <@xdg> create_command_line in win32.c
[16:32] <Mithaldu> ah, that's not documented
[16:32] <@bulk88> so the problem is on nix char * argv[] is passed unmodified through the OS to the child proc, on Win32, char * argv[] is flattened into 1 string by concating, then split in the child?
[16:33] <Mithaldu> concat: yes
[16:33] <@xdg> bulk88, IIRC, CreateProcess hands the command string to the child process and the child process is responsible for parsing
[16:33] <Mithaldu> split: not sure what you mean, but after the concat the string is simply sent on by cmd
[16:33] <@xdg> But windows migth wash execution through the shell
[16:33] <Mithaldu> xdg: correct
[16:33] <Mithaldu> and 2 parsing algorithms exist
[16:34] <@bulk88> I was refering to the CreateProcess API, which takes 1 string *, not argv
[16:36] <Mithaldu> oooh, that's what you mean with "split in the child"
[16:36] <Mithaldu> yeah, that's it
[16:36] <Mithaldu> the child process is responsible for making sense of the string
[16:36] <Mithaldu> most will enlist the help of one of 2 parsing functions provided by C libs
[16:37] <@bulk88> http://msdn.microsoft.com/en-us/library/windows/desktop/17w5ykft%28v=vs.85%29.aspx http://msdn.microsoft.com/en-us/library/windows/desktop/bb776391%28v=vs.85%29.aspx
[16:37] <+dipsy> urgh. long url. Try http://tinyurl.com/8kzrkdf
[16:37] <+dipsy> [ Parsing C++ Command-Line Arguments (C++) ] [ CommandLineToArgvW function (Windows) ] 
[16:37] <Mithaldu> that's one c library
[16:37] <Mithaldu> there's another one with a different function
[16:40] <@bulk88> would anyone agree this "windows flattens all cmd line args" behavior breaks perl's nix compatbility in that aspect? since the child process will never recover the correct arg count in the parent process when parsing the string?
[16:40] <@bulk88> would auto quoting "fix" this incompatibility with perl on nix?
[16:41] <Mithaldu> bulk88: in the general case no, since a child might ignore the quoting
[16:41] <Mithaldu> that question is more easily answered if you specify which child you're talking about
[16:41] <@xdg> I wouldn't say "breaks perl's 'nix compatibility".  I would say "another 'nix incompatibility on windows"
[16:41] <@xdg> since there are already plenty
[16:42] <@xdg> It all *should* be better documented in perlport
[16:42] <@bulk88> the child is a MS CRT cmd line app
[16:42] <@bulk88> such as perl
[16:42] <Mithaldu> in that case proper quoting can fix it
[16:42] <@bulk88> not a cygwin proc
---------------------------


-- 
bulk88 ~ bulk88 at hotmail.com

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=121283

• [perl #121283] system() on Win32 tries to launch bogus paths+bad cmd parsing by bulk88 via RT
• Re: [perl #121283] system() on Win32 tries to launch bogus paths+badcmd parsing by Smylers
• Re: [perl #121283] system() on Win32 tries to launch bogus paths+badcmd parsing by Jan Dubois