[seL4] Re: new RFC: seL4 Device Driver Framework

On 10/17/22 17:54, Peter Chubb wrote:

...

...

...

...

...

"William" == William ML Leslie writes:

William> On Mon, 17 Oct 2022 at 11:37, Demi Marie Obenour William> wrote:

...

...

Will it be possible for clients to pre-register buffers with the multiplexer, and for the multiplexer to in turn register them with the driver? That would allow for devices to DMA directly to client buffers while still having the IOMMU restricting what the driver can do.

William> It seems that's what the framework should aim for - as in, William> don't do anything that precludes mapping pages directly for William> access by the device, where the hardware supports it.

One of the things that needs to be evaluated is the cost of setting up and tearing down IOMMU mappings. For small transfers, copying the data may be cheaper.

Setting up and tearing down IOMMU mappings is very expensive. For zero-copy to be worthwhile, the total amount of data transferred per mapping must be large. This leads to an API in which buffers are pre-registered before performing I/O and stay mapped in the IOMMU until the client explicitly requests that they be unmapped. This also means that, if the client does not trust the device, it will typically need to perform a copy to avoid TOCTOU attacks, except in the case where the data is just being passed to another component without being looked at or processed. Of course, only parts that are being accessed need to be copied, so this is still an advantage. For instance, a network stack might only need to copy the packet headers, since it typically does not need to inspect the packet body. In short, zero-copy can be a useful performance win in some cases, but high-level APIs should generally avoid it unless the device is known to be trusted by the caller (not necessarily by the entire system!). With an untrusted device, there is a high risk of TOCTOU attacks, and preventing them generally requires making a copy anyway. The main cases I can think of where zero copy is a win with an untrusted device are: 1. A large amount of data is coming in, but most of it does not need to be processed at all. Packet capture is a classic example of this: one often only needs to read the headers to determine that a packet should be discarded, and can avoid bringing large parts the packet body into cache. 2. The client will be passing the data to a component that does not trust it, without any processing of its own. The other component will likely need to make its own copy of the data, so the client’s copy would be useless.

In any case, for things like network receive, the multiplexor is going to have to do a copy, to preserve inter-component privacy. You can't share a common DMA area for receive buffers if you want to prevent Component A seeing component B's traffic.

Some NICs might support hardware flow steering, which could avoid the copy. That implies trusting the hardware, though. -- Sincerely, Demi Marie Obenour (she/her/hers)