[time-nuts] Brooks Shera

[Javier Serrano]

On Mon, Mar 25, 2013 at 2:56 PM, Tom Van Baak <tvb at leapsecond.com> wrote:

> I realize there are many cases where clock domain considerations are important. But why does it matter in a device that is simply doing long-term 1PPS statistical sampling?
>
> Could one of you clock domain specialists actually spell out the GPSDO problem for the rest of us, nanosecond-by-nanosecond?


It's hard to discuss this generally, so let's take an example design.
I don't know Brooks Shera's design at all, so I don't know how
applicable the following example is to it. In any case it is an
illustration of the type of clock domain problems one can have while
designing a GPSDO.

Imagine you want to know how many ticks of a 10 MHz source (such as a
VCXO) there are between two rising edges of an externally provided
PPS. By externally provided I mean it is not derived from the 10 MHz
VCXO. Most people today would do it with an FPGA.

One way of doing it wrong is the following:

- Have a free-running counter clocked off the VCXO, counting +1 at
each tick. If this is a 32-bit counter it contains 32 flip-flops (FFs)
that you take the output from. These outputs also go to a bunch of
gates which feed the D inputs of those FFs. These gates implement the
"+1". The VCXO output is hooked to the clk inputs of all FFs, so at
every rising edge of the VCXO signal, a new value appears at the Q
outputs of the FFs. This value is the old value plus 1. Sorry if I am
explaining this at a too-easy level. The signals out of the FFs take
some time to go through the gates, but the FPGA Place&Route tool knows
(because you tell it so) that if the worst case combinational delay
between any Q output of an FF and any D input of any FF is well below
100 ns, all is fine. This is the beauty of synchronous design. You
tell the P&R tool what your clock period is and it makes sure all
signals at D inputs are stable by the time the next rising edge of the
clock comes.

- Now you say "OK, I am going to freeze the value of that counter at
every rising edge of PPS and store it, so a simple subtraction
afterwards will tell me the period I am measuring". What happens if
you tap the Q outputs of those 32 FFs and send these signals to the 32
D inputs of another bunch of FFs which are clocked by the PPS? The PPS
rising edge can come anywhere withing the 100 ns period of your clock.
Because of the different latencies for each line inside the FPGA,
there are chances that some bits in the 32 bit word (as seen by the D
inputs of the PPS-clocked FFs at the time of a PPS rising edge) have
changed after a VCXO rising edge, while others still haven't. This can
give you completely bogus values at the Q outputs of these FFs. You
might think that you can do a software sanity check and fix the bogus
values, but this is impossible. You cannot know which bits are wrong
just by looking at this time-stamp. In addition to this problem, there
is also the issue of metastability in FFs. If the rising edge of PPS
as seen by an FF in its clk input is very close in time to a
transition (rising or falling) in the signal hooked to its D input,
the Q output can stay in a metastable state (neither '0' nor '1') for
a long time. This is more or less important depending on what you are
doing with these outputs further down in your design. It's generally
very bad.

So how do you do it properly?

You bring the PPS into the VCXO clock domain. You can do this by
feeding the PPS signal to a chain of three FFs clocked by the VCXO:

- FF1 has its D input hooked to the PPS signal.
- FF2 has its D input hooked to the Q output of FF1.
- FF3 has its D input hooked to the Q output of FF2.

The output of FF1 can be metastable from time to time, with a small
but not negligible probability. The output of FF2 is for all practical
purposes safe. For it to be metastable, you would need a
doubly-unlikely event: that FF1's output has gone metastable in the
previous clk tick and that the D input of FF2 is seeing an unsafe (far
from '0' or '1', near the transition region) voltage at the time of
the current tick's rising edge.

Now if you tap the outputs of FF2 and FF3 and hook them to gates which
implement (Q(FF2) and not Q(FF3)), the output of this combinational
block is a nice 1-tick-long pulse which is '1' after detecting a
rising edge of PPS. This pulse is in the VCXO clock domain, so you can
safely use it as an ENABLE of the bank of 32 FFs you use to get a time
stamp. This bank now gets clocked by the VCXO signal, so all FFs in
the design are clocked by the same clock, and the P&R tool can do its
job properly.

I hope I understood your question well and this answers it. Cheers!

Javier