[time-nuts] Raspberry Pi 4B Ethernet delay

[Fio Cattaneo (.US)]

Yes, your observations on data rates are correct, I was mostly doing back
of an envelope quick calcs. Speed of light constancy is medium-dependent
of course, I meant "constant" regardless of data rate, sorry for being
confusing.  In the case of twisted copper pairs, at very minimum you need
to consider the twisting rate, plus many other factors which I am not
particularly expert, like RF delays. There are published numbers on this.

>>> So, by far, the network software stack imposes the largest overhead.
> Agree.  But the software delays are roughly predictable.  (Cache misses
and scheduling are the big ones I can think of right now.)

Yes, most software and hardware delays are fairly predictable, even
interrupt delay on PCI or PCI-E bus is well defined and fairly predictable
on a lightly loaded machine.
Most uncertainty on the software stack is broadly falls in 3 categories,
of which you already mention the first two (A) L1/L2/L3 cache misses. An
L3 cache miss in particular is very heavy as it involves having to go out
of the CPU physical package into the memory bus and the memory DIMMs (B)
OS scheduler, especially when the critical path involves user code as
opposed to kernel code. Kernel code is also affected, for instance most
network processing is done by deferred threads, which are roughly akin to
user processes (C) Interrupt masking. Most modern kernels only do minimal
processing of hardware interrupts and defer most of the work to deferred
threads as mentioned in (c), but still, if there are multiple pending
interrupts and all of your cores are busy, then interrupt handling will be
delayed. There are other sources of delay such as System Management
interrupts which are particularly bad because not even the OS kernel is
aware of those, they completely handled by the BIOS, or things like cache
pollution between processors. There are several mitigation techniques,
none of them foolproof.

> Anybody know anything about the USB/Ethernet stack vs the non-USB
Ethernet stack?

None that I know of....... Unfortunately for us time-nutters this is
something that is largely ignored in the computer science trade as USB is
not considered to be performance critical.
To extend this, I was actually surprised to realize that even interrupt
latency of serial ports and parallel ports is very poorly studied, again
because it is not performance critical. I just bought a signal function
generator and I'm about to upgrade to a digital oscilloscope so I can
actually get some hard numbers on this on modern PCI-E based RS232 and
parallel cards.


-----Original Message-----
From: time-nuts <time-nuts-bounces at lists.febo.com> On Behalf Of Hal Murray
Sent: Friday, 8 November, 2019 15:41
To: Discussion of precise time and frequency measurement
<time-nuts at lists.febo.com>
Cc: hmurray at megapathdsl.net
Subject: Re: [time-nuts] Raspberry Pi 4B Ethernet delay


fio at cattaneo.us said:
> Suppose the average NTP packet is 100 bytes, and assume standard 8-10
> encoding no with extra overhead, so 1 byte = 10 bits:

The 10/100/1000 on Ethernet speed refers to net data rate.  The actual bit
rate on the wire is faster to compensate for things like 8B/10B encoding.
For example, 100 megabit Ethernet is actually 125 on the wire if you look
at it with a scope.

But if you are working on the back of an envelope, 8 == 10.


> The speed of light is course always the same...

Well, no.  Light is slower in fibers and cable.[1]  My rule of thumb is
that the conversion factor is the same as miles to kilometers.  (Crappy
cable is significantly slower than good cable.)


> So, by far, the network software stack imposes the largest overhead.

Agree.  But the software delays are roughly predictable.  (Cache misses
and scheduling are the big ones I can think of right now.)

Anybody know anything about the USB/Ethernet stack vs the non-USB Ethernet
stack?

------

1) The high speed traders reactivated a microwave link from New York to
Chicago to save a couple of ms.
  https://www.ft.com/content/2bf37898-b775-11e2-841e-00144feabdc0



-- 
These are my opinions.  I hate spam.




_______________________________________________
time-nuts mailing list -- time-nuts at lists.febo.com
To unsubscribe, go to
http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com
and follow the instructions there.