Hi Corey,
I'm back from leave, but Adrian mostly answered the question. I will reiterate the most important points:
* Currently the PIT is only used to calibrate the tsc if the processor isn't known to support the platform info MSR to read the frequency from. This code is currently only written to work on our local x86 machines, we have an issue for cleaning this code up.
* The TSC freq we read is passed in bootinfo to the initial task, so user space can use this for its calculations.
* I tried using the HPET, but found on our platforms it took hundreds of cycles to reprogram, whereas using tsc deadline mode is far faster. I have not tried the PM timer and am not aware of the perf cost.
Whether the API should be in ticks or microseconds is an interesting question. I initially wrote the API to use ticks and converted it to microseconds based on feedback. However on some arm platforms it's the initial task that calibrates the clock, not seL4, so it may actually make sense to use ticks. It would also remove some calculations from the kernel and make the verification effort (which is starting this year) a bit easier. I will raise a discussion internally, I can't recall if there were any other strong reasons for avoiding ticks in the interface.
Cheers and feel free to hit me up with more questions! I'm to see the RT kernel used out in the wild.
Anna.
-----Original Message-----
From: Corey Richardson [mailto:corey@octayn.net]
Sent: Tuesday, 3 January 2017 1:41 PM
To: Danis, Adrian (Data61, Kensington NSW)
Hi Corey,
I will attempt to provide you with what answers I can, although I'm not the expert in this so Anna may say that all of this is wrong later
One important thing to consider for Real Time is not just the accuracy of the timer, but the Worst Case Execution Time of the kernel and any padding that needs to be given to timing estimates. The reason that the TSC deadline timer is used in the RT kernel is because it is both extremely cheap to read and, of all the timers, the cheapest to reprogram. This means that the kernel takes less time to perform timer ticks, and allows for a less pessimistic WCET analysis. Any potential error in timing, provided it is bounded, can also be added ass pessimism to the WCET. I have not done the maths to work out what gives less pessimism, the error of the TSC or the cost of a more accurate timer, but that is the pertinent question. This is why when we read the TSC in the kernel we just do 'rdtsc' and make no effort to serialize, the 'error' in doing so is much less than the cost of serializing and so results in less pessimism.
It is for similar reasons that the RT kernel goes away from tick based timing to tickless scheduling, although now precision does come into play. Clearly you now have a trade off between tick rate and deadline precision. Taking 'unnecessary' timer interrupts when operating at higher degrees of precision will again increase the pessimism of any scheduling analysis over what it would be with tickless scheduling. Nothing stops you from setting up a timer and counting ticks at user level though.
Sure, tickless is great, +1 to tickless. The biggest thing I'm concerned with is accumulated error in the tsc value. I guess that's not really relevant at the timescales that these measurements are used for.
I mentioned on your github PR but will repeat here for the mailing list. The PIT is only used to measure the frequency of the TSC if we cannot pull the frequency directly out of the platform info MSR.
Adrian
On Sun 25-Dec-2016 12:21 PM, Corey Richardson wrote:
Hi Anna,
While analyzing some time related APIs, I've noticed that the x86 rt kernel assumes that the tsc frequency is constant and well represented by the measurement made at init by comparing against the PIT. It also exposes only microseconds and uses the internal calibration to convert that to/from ticks. I'm concerned about this behavior. The tsc can have up to 200ppm error if uncalibrated and not temperature controlled, from what I'm able to determine. I'd rather have all the public APIs take ticks and allow user space to determine how their timing requirements best correspond to ticks. This came up when wanting to measure how much running time a process/thread has used. Ideally, it's kept as a running count of ticks. In the current API, one can only observe microseconds consumed, converted by a potentially lossy ticksToUs (lossy when the rate isn't exactly known or tick->us conversion isn't an exact integer multiplication). It'd be more useful for me to have tick count directly.
I'm also somewhat concerned with using the PIT to estimate the TSC rate, as I can't find anything about expected error of the PIT, and any slop there will influence all future kernel behaviors until reboot. One potential alternative is the ACPI PM timer or the HPET. Those run at higher frequencies, which would help reduce the error bounds on the rate estimation.
Let me know what you think! If it's useful at all, I've written up some stuff about the work that inspired these thoughts: https://gitlab.com/robigalia/meta/blob/master/blog/_drafts/2016-12-23 -what-time-is-it.adoc
Merry Christmas, -- cmr http://octayn.net/
_______________________________________________ Devel mailing list Devel@sel4.systems https://sel4.systems/lists/listinfo/devel
-- cmr http://octayn.net/ +16038524272