| |

Subscribe / Log in / New account

Windows NT synchronization primitives for Linux

Posted Feb 21, 2024 12:24 UTC (Wed) by farnz (subscriber, #17727)
In reply to: Windows NT synchronization primitives for Linux by mb
Parent article: Windows NT synchronization primitives for Linux

The justification for merging is that it's a significant performance improvement for applications running under Wine. And the justification for not putting the full functionality into futex is that the wait-for-all operation is only needed under Wine, and even then, only when you have applications using edges of the Windows API. Most of the time, pulse and wait-for-any are all that you need.

Windows NT synchronization primitives for Linux

Posted Feb 21, 2024 12:37 UTC (Wed) by mb (subscriber, #50428) [Link] (18 responses)

So what's wrong with dkms? Why does this single-purpose driver have to be in mainline?

Windows NT synchronization primitives for Linux

Posted Feb 21, 2024 13:01 UTC (Wed) by farnz (subscriber, #17727) [Link] (1 responses)

The kernel maintainers want everything in mainline, not in DKMS. DKMS is meant for backports from a later mainline kernel, or for cases where legal issues prevent something being merged into mainline (e.g. licensing conditions).

You'd have to ask Greg K-H and others why they don't want a stable API or ABI for modules so that things can stay outside mainline forever.

Windows NT synchronization primitives for Linux

Posted Feb 21, 2024 13:13 UTC (Wed) by johill (subscriber, #25196) [Link]

Perhaps hyperbolic, but taken to the extreme that argument could also mean no drivers, filesystems, etc. really need to be in the kernel since you could always compile extra modules out of tree, put them into the initramfs, and be done with it.

But yeah, that's not how Linux works? There might be whole architectures with fewer users than this feature would have ...

It's also tremendously impractical with modules signing, having to have compilers everywhere, etc.

Windows NT synchronization primitives for Linux

Posted Feb 21, 2024 14:55 UTC (Wed) by pizza (subscriber, #46) [Link]

> So what's wrong with dkms? Why does this single-purpose driver have to be in mainline?

Because there's a non-trivial amount of users that would see a substantial improvement? As I mentioned earlier in this thread, all sorts of insanity is merged into Linux every cycle that only yields a low-single-percentage improvement on narrow (and more often than not, proprietary/internal) use cases; this would seem to be a no-brainer by those same general principles.

Windows NT synchronization primitives for Linux

Posted Feb 21, 2024 17:54 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (14 responses)

> So what's wrong with dkms? Why does this single-purpose driver have to be in mainline?

Why not move out drivers for all of the one-off devices out of the kernel, then? DKMS significantly complicates the OS updates and secure boot.

Windows NT synchronization primitives for Linux

Posted Feb 22, 2024 9:59 UTC (Thu) by farnz (subscriber, #17727) [Link] (13 responses)

And just looking at the kernel, there's a huge number of single-purpose drivers that most users are never going to need - 100G Ethernet, Infiniband, FireWire, UIO, parallel SCSI, PCMCIA, amateur radio, SLIP, FPGA drivers, SGI system drivers, analogue TV tuners, multipath block I/O, old PC-style gameports, Industrial I/O (iio), obscure HID devices, CXL, ATM, PATA and probably more. The kernel's full of stuff that most people don't need - either because it's legacy (analogue TV tuners, ATM, PATA), or special case (IIO, UIO, amateur radio), or because it's too expensive for most of us (100G Ethernet, Infiniband, FPGA drivers).

Windows NT synchronization primitives for Linux

Posted Feb 22, 2024 19:04 UTC (Thu) by mb (subscriber, #50428) [Link] (12 responses)

That's what-about-ism. ntsync has nothing to do with analogue TV tuners and all the other stuff you mentioned.

But thanks to all for explaining the technical details and the background behind ntsync.

Windows NT synchronization primitives for Linux

Posted Feb 22, 2024 21:35 UTC (Thu) by farnz (subscriber, #17727) [Link] (11 responses)

But all of those other stuff are "single use drivers" for things that I'm never likely to use; I have more chance of benefiting from ntsync than I do from all of those other things.

Windows NT synchronization primitives for Linux

Posted Feb 22, 2024 22:09 UTC (Thu) by mb (subscriber, #50428) [Link] (10 responses)

Exactly. What about all those other unrelated things over here?

As I said: Thanks for explaining some technical details. But now you keep on saying things that are off topic.

Windows NT synchronization primitives for Linux

Posted Feb 22, 2024 22:25 UTC (Thu) by farnz (subscriber, #17727) [Link] (9 responses)

Because you said "So what's wrong with dkms? Why does this single-purpose driver have to be in mainline?" - but that applies to every last one of those things that are also single-purpose drivers in mainline.

I'd like you to explain what criteria makes a single-purpose driver sensible to include in mainline (rather than being things that should be in dkms), such that all the single-purpose drivers in mainline that I'm never likely to use meet that criteria, but this doesn't.

Windows NT synchronization primitives for Linux

Posted Feb 23, 2024 6:26 UTC (Fri) by mb (subscriber, #50428) [Link] (8 responses)

Yes, but this is not what I meant.

With "single-purpose" I did _not_ mean that a driver would only drive a single type of hardware. That is _obvious_. That's the case for most drivers.
And I also was not referring to the number of users of this driver.
There are even drivers in the mainline where probably only a single instance of the hardware exists. Does it make sense? No, it does't. But that's a completely different thing. Doesn't have anything to do with how we discuss ntsync.

I was referring to the number of _applications_ that would use the driver. I'm sorry that I didn't make that clearer to begin with. That was my fault.
ntsync only serves a single _application_.

It is like the DCO driver for OpenVPN. Which is a DKMS. And I don't see anything wrong with that. Except for maybe that it breaks module signing.

And yes, I do know that we have more features in the kernel that only serve a single application. You don't need to bring that up. ;-)

Windows NT synchronization primitives for Linux

Posted Feb 23, 2024 10:57 UTC (Fri) by farnz (subscriber, #17727) [Link] (2 responses)

The OpenVPN DCO driver is out of tree not because it has a single use, but because the kernel has a guideline of "don't break userspace applications", and the OpenVPN DCO driver currently wants the freedom to make changes that will deliberately break userspace applications. This is a very good reason to not enter mainline - you don't want to be under the strictures that Linus applies.

Windows NT synchronization primitives for Linux

Posted Mar 6, 2024 19:18 UTC (Wed) by florianfainelli (subscriber, #61952) [Link] (1 responses)

And yet this driver is now attempting to enter mainline: https://lore.kernel.org/netdev/20240106215740.14770-1-ant...

The technical reasons for attempting to upstream should be simple and clear: it facilitates the distribution of your module and it gives you some amount of maintenance "for free".

Windows NT synchronization primitives for Linux

Posted Mar 7, 2024 11:36 UTC (Thu) by farnz (subscriber, #17727) [Link]

Yep. Now that they're confident that they don't want to deliberately break user-space, they're going for mainline, just like everything else, because there's good reasons to be in mainline. It's just that when you know you want to break mainline's rules (unstable userspace interfaces like OpenVPN DCO, wrong licence like NVidia's proprietary driver), you need to stay out until you're ready to keep to mainline's rules.

Windows NT synchronization primitives for Linux

Posted Feb 23, 2024 11:24 UTC (Fri) by timon (guest, #152974) [Link] (4 responses)

But why then do you think that the ntsync driver only serves a single application? Do you count anything running under Wine as the same application?

From what I understand from the article and its comments, there might be numerous (legacy?) applications that use those NT features and that one may want to run on a Linux kernel with Wine, or there might be not a single application ever using this NT feature with Wine.

That Wine wants to be able to offer the Windows APIs as complete, conformant and performant as possible seems like a laudable goal and a win for FOSS -- and I don't see why one would want to hinder their efforts by relegating their work to out-of-tree modules via DKMS, as long as there are no legal or severe technical hurdles.

Windows NT synchronization primitives for Linux

Posted Feb 23, 2024 17:47 UTC (Fri) by mb (subscriber, #50428) [Link] (3 responses)

Please see the root of this thread.
This is going way off topic to what I originally commented on.

The original assumption was that there were *zero* use cases. (which was *not* claimed by me)
Which has been clarified as being wrong.

The corrected assumption is that there are proprietary apps hidden somewhere that wine does not want to break.
Whether that warrants a kernel driver or not is a completely different question. IMO it doesn't, but feel free to have a different opinion. I'm Ok with that.

Windows NT synchronization primitives for Linux

Posted Feb 23, 2024 18:35 UTC (Fri) by farnz (subscriber, #17727) [Link] (2 responses)

Note that the thing that warrants a kernel driver is the wait-for-any functionality of the call; the challenge is that if the kernel driver only implements wait-for-any, and neither implements a way to "return" those waits to userspace early nor a wait-for-all option, then Wine has a regression and won't use this driver for wait-for-any, even though it works for that purpose.

The exact implementation of this is up to the people doing the work; do you put a wait-for-all mode in the kernel, even though it's rarely needed, or do you come up with a way for Wine to reclaim all the wait-for-anys sitting in the kernel and push all wait-for-anys (including the reclaimed ones) through the slow path, not using the kernel driver?

Both work, since we only have evidence that there are applications that do not use wait-for-all, but would benefit from a faster wait-for-any than can be implemented in userspace alone.

Windows NT synchronization primitives for Linux

Posted Feb 29, 2024 18:55 UTC (Thu) by raven667 (subscriber, #5198) [Link] (1 responses)

> the challenge is that if the kernel driver only implements wait-for-any, and neither implements a way to "return" those waits to userspace early nor a wait-for-all option, then Wine has a regression and won't use this driver for wait-for-any, even though it works for that purpose

I'm catching up but isn't the way that MS solves these kinds of compat issues in the Windows world is tedius baked-in lists of specific applications which need the differential behavior? A WINE config/cli flag that indicates when emulated wait-for-all is needed which uses the existing slow implementation with a default which uses the fast kernel implementation, that aborts with a sensible error if the app uses features which aren't implemented, so the user/admin can restart the app with the right compat option saved. Maybe an option to upload those compat lists somewhere so the config could be distributed, if the existence of the app isn't sensitive data.

I know having things work correctly automatically is more awesome but sometimes just doing the dumb brute force thing is more effective and efficient than working through all the details, coordination and judgement to automate something. Technical debt isn't always bad as long as you are picky when you create it.

Windows NT synchronization primitives for Linux

Posted Feb 29, 2024 19:20 UTC (Thu) by farnz (subscriber, #17727) [Link]

That would work, too, but then needs the people proposing a Linux-only accelerator for wait-for-any behaviour to create the compat options + list, so that people can use it.

This is not an unsoveable problem; it's "just" that Wine isn't happy with a gain in performance for some applications at the expense of others now failing to work at all, and people will have to do the work so that this becomes a gain in performance for some applications (those that only use wait-for-any) but not others.