[time-nuts] Re: PPS latency? User vs kernel mode

[Trent Piepho]

On Mon, Dec 13, 2021 at 8:03 AM David Taylor via time-nuts
<time-nuts at lists.febo.com> wrote:
>
> On 13/12/2021 04:17, Adam Space wrote:
> >> What do you mean by "kernel mode"?
> > I am referring to this guide
> > <http://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html>  that another user here
> > recommended to me a bit ago. I am not too sure myself since I am relatively
> > new to this.
>
> That document is now historic background.
>
> It refers to the early "kernel mode" when (AIUI) part of the OS rather than NTP
> did the fine clock adjustment.  I recall that early Raspberry PI OS kernels did
> not include all the functions required for the most accurate timekeeping, and
> that a kernel recompile was required.

For kernel mode, there are really two things I can think of that might
be referred to.  This is stated well in the ntp.org FAQ:

5.2.4.2. [...] two mechanisms: Capturing an external event with high
accuracy, and applying that event to the clock model.

In Linux, we have pps-gpio for the first of those.  There are also
other pps clients, e.g. serial port CD line, but the gpio is the
common one.  This timestamps the PPS pulse in the kernel, which avoids
a number of significant sources of error.

Linux gained a RFC2783 kernel PPS system in 2009 and the pps-gpio
client was added in 2011.  Prior to this, kernel pps timestamping
required patching the kernel to add separately maintained
implementations.

But even with pps-gpio in the kernel in 2011, configuring it was
difficult.  The dynamic device tree overlay support used on modern RPi
distros didn't exist.  pps-gpio didn't even support device tree based
configuration.  Configuration was done by writing C code in the kernel
to create platform_data that configured various things, pps driver,
pinmux, gpio controller, etc.  This was trivial for embedded software
developers and kernel hackers, but put it completely out of the reach
of many.

Prior to kernel pps support, the common way to get a PPS was done in
userspace.  The PPS was connected to the serial port CD line and
userspace code would wait for a DCD interrupt via the TIOCMIWAIT tty
ioctl.  This has significantly more error than the above kernel
timestamping method.  But one did not need any sort of kernel
configuration to use it and Linux has supported UART DCD interrupts
and TIOCMIWAIT for a very long time (older than the history in git,
likely in the early 90s.)

Now we can consider the 2nd part, applying that event to the clock model.

In the beginning, we had the adjtime() call.  This adds an offset to
the clock, which the kernel applies gradually over a period of time.
This offset must be repeatedly calculated as the clock's phase error
drifts from zero and then reprogrammed via control loop.  I.e., the
PLL or FLL is implemented in userspace (ntpd).

Then we gained adjtimex() (also ntp_adjtime() and clock_adjtime())
from David Mills starting in 1993 and leading up to RFC 5905 in 2010.
This allows the PLL/FLL to be implemented in the kernel.  But it's
still controlled from ntpd.  ntpd does the actual network protocol
and/or gets PPS timestamps and steers the PLL via adjtimex calls.
Here we have ntpd opens /dev/pps0, receives timestamps, and calculates
values to steer a kernel PLL via adjtimex calls.

And finally we can have the kernel act on the PPS timestamps itself.
But NTP network traffic might be a bridge too far.  Mills describes
this as a "hardpps()".  This was added to Linux in 2011 by Alexander
Gordeev, based on Mills' work, but with a different implementation.
This is a "kernel consumer" of PPS timestamps, which can be used to
steer the PLL entirely inside kernel code.  It's enabled with the
kernel config option NTP_PPS.  One can easily see if this is present
in the kernel via the usual methods.  It is often NOT enabled because
it doesn't work with tickless kernels.  I do now know if current ntpd
supports this or what it takes to use it with ntpd.  I have used it,
but I wrote my own code from scratch to integrate it with a GPS module
to perform time sync without installing either ntpd nor chrony.