[volt-nuts] Basic question concerning voltage references

[Poul-Henning Kamp]

--------
In message <CAJgz9AYgQQjOmOK57yBiohF6Qh0DX6sEH-M_RxaBWQC54uDt7Q at mail.gmail.com>, Russ Ramirez writes:

>My question is this. If I measured a 10 volt reference known to be good to
>8 1/2 digits, but with the LSD being > 0, assuming a transfer standard with
>traceability and documented uncertainties etc, would a 6 1/2 digit
>voltmeter read 10.00000 volts, or round up to the limits of its AD
>converter resolution; say X uV over 10.00000 volts?

Somewhere in your instrument there is an A/D converter of some kind.

Even if it is perfect, you can never know if the last bit from the A/D
represents a voltage which is slightly higher or slightly lower than
the number it reads.

This is why specifications for digital instruments always includes "± 1 digit"

In theory, if you have *truly* perfect electronics, it could be "± 0.5 digit",
because in that case you might as well use the same ±1 as everybody else
and double the size of the range.

The A/D in a 6½ digit instrument by definition must have at least
22 bits (2^21 = 2097152 + sign bit) and if we scale things so the
least significant bit is 5 microvolts, we have a full scale voltage
of 2^21 * 5e-6V = 10.48576V.

QED:  Uncertainty on your 6½ digit meter on a 10V signal will always
be at least ± 5 microvolts.

If you do the same math for a true 8½ digit DVM, you need a 29 bit
A/D converter (2^28 = 268435456 + sign) which means you can use 40
nV as stepsize yielding 10.73V range, and thus in teory get ± 40nV
measurements.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.