bpo-35537: Document posix_spawn() change in subprocess

[vstinner]

Document that subprocess.Popen no longer raise an exception on error
like missing program on very specific platforms when using
os.posix_spawn() is used.

https://bugs.python.org/issue35537

[eamanu]

What about if we say: Now, Popen can use ... ```` or just remove the now``` .

Just a suggestion

[eamanu]

Can you mention some example of that platforms?

[vstinner]

I prefer to not mention WSL and QEMU User Emulation since it's a bug and these bugs can be fixed whenever. I prefer to not mention them to be more future-proof (not have to update the doc).

[gpshead]

I'd mention the platforms anyways with a "at the time of this writing, buggy platforms we are aware of include Windows Subsystem for Linux (version XYZ) and QEMU (version XYZ)" ... If there are bugreports filed against them, those can be linked to from the doc as well.

Otherwise it such text sets off alarm bells for anyone reading it, they'll have the same question: am I impacted? do I use any "very specific platforms"? and come seeking more information.

when we become aware of upstream bugs being fixed we can update the text to say "versions prior to ABC" or similar.

[vstinner]

@izbyshev: Would you mind to review this change? Are you fine with documentating the limitation rather trying to fixing it?

[izbyshev]

@vstinner I hope that problems with QEMU, WSL and any other Linux environments where vfork is emulated via fork will be avoided if bpo-35823 is accepted. If that happens, there will be no known platform that both (a) has the problem you describe and (b) is eligible for posix_spawn as currently implemented in subprocess. Therefore, this change will effectively be the statement that we're OK with using posix_spawn in the future on platforms that suffer from such problems. Or, to put it another way, that we're willing to take the risk if we can't reliably check whether such issues exist when we decide to use posix_spawn. Is that what you intend to do?

Personally, I'd wait at least until we're done with bpo-35823. If it's merged, I'd prefer not to merge a part about "very specific platforms" and wait until we encounter another such case. If it's rejected, I agree that we should document this issue. Also, we could add a parameter to subprocess.Popen() to allow users to force traditional fork()/exec() path to workaround such issues.

[vstinner]

Personally, I'd wait at least until we're done with bpo-35823.

Ok, let's do that ;-)

[vstinner]

@izbyshev: More generally, I like how things are going one. Slowly, we listed more and more bugs and corner cases, and we had to experiment posix_spawn() and dig into the code (well, in practice that part mostly done by you @izbyshev :-D) to be able to get this list. I'm fine with dropping posix_spawn() from subprocess if we can successfully use vfork in _posixsubprocess. I'm open to different options, as soon as subprocess runs faster :-)

[vstinner]

Four months later, I see no activity on the idea of modiying _posixsubprocess to use vfork(), so I updated this change and I propose to merge it.

@eamanu and @gpshead: Would you mind to review my update PR?