[time-nuts] Re: NIST NTP servers way off for anyone else?

[Magnus Danielson]

Hi,

On 2021-12-14 02:26, Adam Space wrote:
> I'm not sure if anyone else uses the NIST's NTP servers, but I've noticed
> that the offsets I'm getting from Gaithersburg servers seem to be
> really far off, like 40-50 ms off. This is pretty odd since they usually
> have a 2 - 3 ms accuracy at worst.
>
> It is interesting to think about what is going on here. NIST has a secondary
> time scale
> <https://www.nist.gov/pml/time-and-frequency-division/time-services/utcnist-time-scale/secondary-utcnist-time-scales-and>
> at Gaithersburg, maintained by a couple of caesium clocks that are
> typically kept within 20ns of UTC(NIST), i.e. their primary time scale in
> Boulder. They also host their remote time transfer calibration service and
> their Internet Time Service (i.e. NTP servers) out of Gaithersburg.
>
> It seems highly unlikely that their time scale there is that far off. One
> thing that immediately comes to mind is asymmetric network delays causing
> this. I do think this has to be the reason for the large discrepancy, but
> even so, it is an impressive feat of asymmetric path delays. The maximum
> error in offset from a client to server due to asymmetric network delays is
> one half of the delay. (This corresponds to one path being instantaneous
> and the other path taking the entire delay time). When I query their
> servers, I am getting about a 45ms offset, and a delay of around 100ms.
> This would mean the maximum error due to asymmetric path delays is around
> 50ms--and less even if we're being realistic (one of the delays can't
> literally be zero). Basically, for the offset error to be accounted for
> primarily by asymmetric network delays, the delays would have to be *very*
> asymmetric.

For the asymmetry to be 45 ms, the difference between forward and 
backward path would need to be 90 ms, since the time error will be the 
difference in delay divided by 2. The round trip time is instead the sum 
of the two delays.

Now, as you observe this between two clocks with a time, difference, the 
time-difference add onto the TE, but does not show up on the RTT.

So, 90 ms difference would fit, but delays would be 95 ms and 5 ms +/- 5 
ms, since we can trust the RTT to be unbiased. Now we come to what is 
physical possible, and 5 ms is 1000 km fiber delay. You can calculate 
yourself from your location the minimum distance and thus delay. In 
practice fiber is pulled not as straight as one would wish. I use at 
least square root of 2 as multiplier, but many agree that this is still 
optimistic and it can be far worse.

What can cause such delay in a network? In IP/MPLS, the routing 
typically does not care about forward and backward direction being the 
same. Rather, they trim it to shed the load, i.e. Traffic Engineering. 
That means that for two pair of nodes in that network, it can be sent 
over a shorter path in one direction and longer in the other. In 
addition, buffer fill levels can be high on a path, meaning that you end 
up in the end of a buffer for each router hop due to traffic load. Delay 
is a means to throttle TCP down in rate. Random Early Discard (RED) is 
meant to spread that evenly between streams to cause throttle earlier 
than dropping packets due to full buffers, but it still means dropping 
packets. That affects UDP traffic too. MPLS-TE then tries to work on 
that on a secondary level.

With that, depending on your actual distance, which I do not know, it 
becomes fuzzy if the network or servers have asymmetry. If you have 
enough distance, then some of the time error can not be allocated to 
network asymmetry as the short path needs to be higher. This then needs 
to be allocated over to clock errors.

All this is a result of having three unknowns and two measures, you 
cannot fully resolve that equation system. It needs aiding. Having the 
right time on one end does not help if one attempts to know the time 
error of the other end.

It would help if you could add observations from other locations near 
Gaithersburg, network wise.

> Is anyone else experiencing the same thing?

Which makes this question very relevant. Measuring with less of the 
biases and noise of the network may provide clearer answer on the 
Gaithersburg servers.

Cheers,
Magnus