Announcing Neptune OS v0.4, now running Linux kernel drivers in userspace on seL4
Dear list, I am announcing release v0.4 of Neptune OS [0], a general purpose Windows NT-like operating system built on top of seL4. The main feature of this release is a new driver subsystem that allows unmodified (or at most minimally modified) Linux kernel drivers to run as native userspace processes on seL4. By 'native' I mean that these are just regular, unprivileged user processes: there is no virtual machine or any kind of virtualization technology involved, at all. These drivers communicate with the rest of the system via the standard NT driver APIs and appear to the OS as regular, native NT drivers without the need of any special handling. Therefore, I believe with some effort our approach can be adapted to work on other NT-like OSes, including ReactOS and Windows itself, and potentially (with some greater effort) to other seL4-based OSes as well. Before we get into the technical details, here [1,2] are two videos demonstrating Linux kernel GPU (AMDGPU, i915, virtio_gpu) and Ethernet (e1000e) drivers running as native user processes on Neptune OS. We show that these GPU drivers can successfully POST the GPU cards and perform mode setting and framebuffer mapping, and that we can have a basic networking stack capable of packet TX/RX via the Ethernet driver. I hope these examples demonstrate the validity of our approach as the drivers involved here are massive, highly nontrivial beasts (especially in the case of AMDGPU) that drive complex, modern hardware, and their correct function, even for seemingly simple things like mode setting (which is actually not so simple in modern graphics cards), depends on the correct behavior of memory management, timers, workqueues, IRQs, RCU, and many other Linux kernel subsystems. With the exception of i915 (which needs some patching due to its use of the wbinvd instruction), the drivers involved are otherwise completely unmodified. Now some technical details. One of the defining features of the NT architecture is its support for multiple userspace programming interfaces, including Win32 and POSIX, via the so-called Environment Subsystems. These environment subsystems sit on top of the NT Executive and essentially "translate" the APIs spoken by, say a Win32 application, to the native NT API exposed by the Executive. Since Neptune OS has moved the NT Executive to userspace on seL4 and every NT driver now runs as unprivileged user processes, the idea of a userspace Linux driver subsystem becomes a natural extension to these environment subsystems. To the rest of the NT OS, the Linux driver subsystem is simply a regular NT driver (sitting under drivers/lib/lnxdrv in the source tree) that speaks the standard NT driver API (exposed by the Executive) and participates in the usual IO request packet flow. It somehow "magically" translates the NT driver API into the Linux kernel API so that a Linux device driver, say amdgpu or i915, responds to IO requests just like any other native NT driver does. Of course, it would be completely impractical to actually implement a compatibility layer that translates the NT driver API to the Linux kernel API, as the latter is not only highly complex, but also very much unstable. The solution to this problem is inspired by the approaches taken by UML (usermode Linux) and the lkl (Linux-kernel-as-a-library) project. The idea is to add a new architecture port to the Linux kernel (under arch/ntos in the Linux source tree). This architecture port defines how the Linux kernel requests hardware resources, such as memory and IRQ, by calling the host NT OS (eg. request_irq will eventually be routed to IoConnectInterrupt, an NT driver API call). Additionally, a new config switch CONFIG_KERNEL is added to the main arch-independent part of the Linux kernel, which gates the schedulers and other core kernel components behind it. When built without CONFIG_KERNEL, the resulting kernel binary is stripped of what makes Linux an independent OS kernel and becomes essentially a runtime library for kernel driver modules. When loaded by the aforementioned NT driver (lnxdrv.sys), this Linux kernel binary is supplied with an "import table" which the Linux side can call, and returns an "export table" for the NT side to call. The import table defines the interface with which the Linux side can request resources from the NT side (eg. allocating memory or scheduling an IO work item), and the export table defines the services that the Linux side provides to the NT side (eg. responding to a READ IO request). Some thunking magic needs to happen for the PE and ELF images to be able to call each other, but it all turns out to work pretty well (this is similar to the PE+ELF split used in modern WINE). Communication between the NT side and the Linux side is done entirely via these import and export tables. This way we shield the NT side from the complex, volatile Linux kernel API and maintain a clean separation between the two sides. Additionally, this design allows us the flexibility to potentially extend our work to other non-NT OSes, including other seL4-based OSes: another OS just needs to implement the same interface in these import/export tables to reuse our work. One further benefit is that we are part of the Linux kernel source tree, so updating to a newer Linux kernel is just a git rebase away (as I have done during the development cycle), as the changes to the arch-independent kernel is very minimal (I'm not sure if upstream Linux is ever going to accept this into the mainline tree, but one can hope). Anyway, if you have read this far, I hope you do find this work interesting. I believe with the introduction of the Linux driver subsystem, there would be a real chance that Neptune OS could become something useful beyond a hobby operating system. I am therefore actively looking for funding to continue the work on Neptune OS, so if you are interested, please do let me know. [0] https://github.com/cl91/NeptuneOS [1] https://youtu.be/BJIrUZIGgBc [2] https://youtu.be/YTdqeGk54to --- Dr. Chang Liu, PhD. github.com/cl91/NeptuneOS
participants (2)
-
Dr. Chang Liu, PhD. -
Julia Vassiliki