[time-nuts] What is "accuracy"? (newbie timenut, hi folks!)

[Magnus Danielson]

Hi Belinda,

On 05/06/2016 03:34 AM, BJ wrote:
> Hi Time Nuts,
>
> I'm fairly new to the fascinating world of time and frequency, so I
> apologise profusely in advance for my blatant ignorance.

We have all been there, so don't worry about it!
Your questions is actually really relevant and you've done your homework.

> When I ask "what is accuracy" (in relation to oscillators), I am not asking
> for the textbook definition - I have already done extensive reading on
> accuracy, stability and precision and I think I understand the basics fairly
> well - although, after you read the rest of this, you may well (rightly)
> think  I am deluding myself. It doesn't help matters when some textbooks,
> papers and web articles use the words precision, accuracy and uncertainty
> interchangeably. (Incidentally, examples of my light reading include the
> 'Vig tutorial' on oscillators, HP's Science of Timekeeping Application note,
> various NIST documents including the tutorial introduction on frequency
> standards and clocks, Michael Lombardi's chapter on Time and Frequency in
> the Mechatronics Handbook and many other documents including PTTI and other
> conference proceedings). Anyway, you can safely assume I understand the
> difference between accuracy and precision in the confused musings that
> follow below.

That's not a bad set of readings, so you're doing good there.
BTW. I ended up eating dinner with John Vig this sunday as fellow 
time-nut Francis Grosz was meeting up with us both.

For the terms themselves, the real reference you will find at BIPM where 
the "International Vocabulary of metrology" VIM (JCGM 200) and 
"Evaluation of measurement data - Guide to the expression of uncertainy 
in measurement" GUM (JCGM 100) is the real definitions, but at times 
people is sloppy about it.

On the other hand, VIM and GUM is sloppy in that it assumes white noise, 
where as we know we have non-white noise and have had to invent new 
statistical tools such as Allan Deviation and Modified Allan Deviation, 
kind of ironic but that's where it is.

David Allan and I had a discussion about it the other day and agree that 
something needs to be done about it. It's an interesting topic in its 
own right, so let's come back to it later.

Being at the International Frequency Control Seminars (IFCS) several 
presentations start with illustrations of accuracy and precision, some 
even with nice cartoons, just to make the point again and again... so 
that nobody confuses the two.

> What I am trying to understand is, what does it REALLY mean when the
> manufacturer's specs for a frequency standard or 'clock' claim a certain
> accuracy. For ease and argument's sake let us assume that the accuracy is
> given as 100 ppm or 1e-4 ....
>
> As per the textbook approach, I know I can therefore expect my 'clock' to
> have an error of up to 86400x1e-4= 8.64 s per day.

That would be accuracy, as it is about how good the average aim is.

> But does that mean that, say, after one day I can be certain that my clock
> will be fast/slow by no more than 8.64 seconds or could it potentially be
> greater than that? In other words, is the accuracy a hard limit or is it a
> statistical quantity (so that there is a high probability that my clock will
> function this way, but that there is still a very small chance (say in the
> 3sigma range) that the error may be greater so that the clock may be
> fast/slow by, say, 10 seconds)? Is it something inherent, due to the nature
> of the type of oscillator (e.g. a characteristic of the crystal or atom,
> etc.) or does it vary so that it needs to be measured, and if so, how is
> that measurement made to produce the accuracy figure? Are environmental
> conditions taken into account when making these measurements (I am assuming
> so)? In other words, how is the accuracy of a clock determined?

By tradition and also put into standard, as far as I recall it, the 
given accuracy is a 3-sigma value, that is 99,7% of all devices is 
within this accuracy limit.

To the degree environmentals is taken into it can... vary. Some 
sales-men take a more opportunistic approach to it, but there is 
standards for what goes where. I'm not the master of the standards, but 
I'm sure fellow time-nuts Bernd Neubig can enlighten us, as he was 
awarded for his 40 years of work, and in particular his many 
contributions to the standards.

> Note that I am conscious of the fact that I am being somewhat ambiguous with
> the definitions myself. It is my understanding that the accuracy (as given
> in an oscillator's specs) relates to frequency - i.e. how close the
> (measured?) frequency of the oscillator is to its nominal frequency - rather
> than time i.e. how well the clock keeps time in comparison to an official
> UTC source.... but I am assuming it is fair to say they are two sides of the
> same coin.

Indeed. The precision given is relating to it's frequency value. Then, 
as you measure it for some time you integrate it up to phase and time, 
but that will also integrate environmentals which is a separate 
systematic effect.

> Does accuracy also take stability into account (since, clearly, if an
> oscillator experiences drift, that will affect the accuracy - or does it?)
> or do these two 'performance indicators' need to be considered
> independently?

Frequency-offset and drift is relatively easy properties to establish 
and compensate, it's systematics. The stability is a random variations, 
the noise processes that goes on top of that. If you look into the VIM 
and GUM you discover the Type A and Type B types of precision, and it is 
exactly about this where you have random noise vs. systematic effects.

> I am guessing that the accuracy value is provided as general indicator of
> oscillator performance (i.e. the accuracy does REALLY just mean one can
> expect an error of up to, or close to?, a certain amount) and that stability
> (as indicated by the ADEV) is probably more significant/relevant.

They are quite distinct properties, and very relevant as separate 
properties.

If you take a SEC/Stratum 3 oscillator, it should be within +/- 4.6 ppm,
including the linear drift for say 15 years. It then have stability as 
indicated by ADEV, or actually by the TDEV limit curve. The actual specs 
is in ITU-T Recommendation G.813.

> It is also entirely possible I am asking all the wrong questions. As you can
> see, confusion reigns. I am hoping things will become clearer to me as I
> start playing around with hardware (fingers and toes crossed on that one).

You ask very relevant questions. You are on a good path to learn more, 
and you have lots of fun in front of you! :)

> In the meantime, if anyone could provide some clarity on this topic or set
> my crooked thinking straight, my gratitude will be bountiful.

That is what we are happy to help with here, so don't worry about it.
Keep the questions going and remember that there is at least 10 others 
silently reading this list that will learn from the discussion.

Cheers,
Magnus