[time-nuts] HP 5370B

[Mike S]

At 08:50 PM 5/10/2008, John Miles wrote...

>Accuracy ratings for digital displays are commonly specified as a 
>percentage
>+/- 1 LSD.  So given tolerances of +/- 1 dB +/- 1 LSD for the analyzer 
>and
>+/- 1 dB for the DUT, 2.01 dB is technically within spec.

No. The stated spec was peak to peak, so tolerances at both points must 
be accounted for. The stated conditions were:

spec: "2 dB p-p"
analyzer accuracy: "around 1 dB" (I assume exactly 1 dB absolute = +/- 
1 = 2 dB p-p, since nothing further was stated)

A measurement of "2.01 dB p-p" (which was also stated) could be 
measured from a DUT with 0.01 dB p-p, or from one with 4.01 dB p-p (max 
error at both peak and trough).

In the former case, an actual peak of 0.01 might be measured as 1.01 
and an actual trough of 0 as -1.00, for a measured p-p of 2.01. In the 
latter, an actual peak of 2.01 might be measured as 1.01 and an actual 
trough of -2 might be measured as -1.00, again for a measured p-p of 
2.01.

Such a measurement cannot be used to claim compliance with the spec 
(given the stated measurement accuracy).

If significant digits were to be factored in, then the example should 
have included a spec tolerance. Since the figures 1.99, 2 and 2.01 were 
mentioned, it's reasonable to assume the "2" is an exact number lying 
between the other two. Even if the spec is only to a single significant 
digit, a measurement of 2.01 still doesn't meet the spec, since it 
guarantees compliance no better than 4.01, as stated above.