[time-nuts] Re: ADEV on data with gaps

[Magnus Danielson]

Dear Jim.

The best current strategy to calculate ADEV with datagaps was presented 
by David Howe and colleagues recently in 2021 and 2022.

One of the published articles is this:

https://www.academia.edu/79180802/Time_series_Imputation_Algorithm

I don't think we where able to include it into the IEEE 1139 and 1193, 
but I will be happy to be proven wrong. If not, we will for sure include 
it in next revision.

Stable32 does not have that algorithm implemented, Stable32 rather treat 
gaps by not calculating the squared value if any of the phase samples is 
missing. Demetrios asked about this some time back on the list. I think 
Timelab does the same in best case.

The imputation algorithm uses the same thinking as the TOTALDEV, and 
consider the noise of each datasample, as independent when you have 
second differentiation, and with independence you can move them around 
as long as you do not make data re-appear twice. The approach chosen is 
to reflect samples into the sequence in reverse order. In TOTALDEV you 
mirror them out around the window you analyze.

The imputated time sequence can then be processed by ADEV and friends as 
usual. It has been checked for robustness and have much better 
robustness to loss of data than previous methods. See article.

This work created quite a buzz when it came, and for a good reason as it 
is an usually strong step forward that addresses actual concerns for the 
type of opertional context you describe, where continuous monitoring is 
not quite possible.

I would also like to raise your awareness of the article by Prof. 
François Vernotte et al "Uncertainties of drift coefficients and 
extraplolation errors: Application to clock error prediction" in 
Metrologia Feb 2001. This is relevant to understand how uncertainty of 
phase due to various random noises behave. The gap from measuring to 
when they appear again follow these rules. They align up well with 
simulations I have done. I was able to contribute this article into the 
IEEE 1139 and 1193 standards, and it was done to provide support for 
GALILEO.

Please let me know if I can be of further assitance.

Cheers,
Magnus

On 2023-08-04 18:15, Lux, Jim via time-nuts wrote:
> I've got a series of clock measurements where the actual measurement 
> occasionally has problems (e.g. it's measured against GPS, and there 
> are GPS partial outages).
>
> What's a strategy for calculating ADEV (and related parameters) where 
> there are data in the sequence that are excised? For example, does 
> Timelab or Stable32 have a way to have timestamps for the 
> frequency/phase measurements that recognizes gaps?
>
> At a first glance, it seems that the calculation for each tau will 
> have a different number of measurements contributing.  Say I have 1000 
> seconds of data, once a second, and there's a gap of 100 seconds 
> somewhere in the middle.  For the 1 second tau, I've got 900 
> measurements.  for a 100 second tau, I'll have 800 measurements 
> (because for 200, one or the other end point is missing), etc.
>
> Some plots attached showing some data.  In this case, I'm looking at a 
> post calculated "clock bias" between GPS and the clock - that is, 
> there's a process that estimates the clock offset (in meters). A first 
> difference gets rate, and clock has some relatively fixed bias (that 
> changes with time), and random variations on top of that.  However, 
> the measurement depends on the SNR of the GPS signals (lower plot on 
> biasdrift, it's the measurement uncertainty estimate)
>
> At some point, of course, the ADEV measurement is limited by the GPS 
> measurement uncertainty - if it's 1 ns, then ADEV is never going to 
> measure less than 1E-9 over 1 second.
>
>
> _______________________________________________
> time-nuts mailing list -- time-nuts at lists.febo.com
> To unsubscribe send an email to time-nuts-leave at lists.febo.com