Is there a general rule of thumb for how accurate a gauge should be in compared to the parts being checked? In my chemistry class, I remember making standards where the instrument needed 1 more significant figure in compared to the standard solution we were creating. Is it the same rule for gauges?

  • $\begingroup$ Remember that accuracy and sensitivity are not the same thing - for accuracy, you need calibration. I wouldn't trust the last decimal point on a set of digital calipers for something 'mission critical'. The bigger question here, is what's the acceptable tolerance on the 'parts being checked'? $\endgroup$ Sep 19, 2018 at 20:39
  • $\begingroup$ @JonathanRSwift that depends on the calipers,: my son looked at the calipers I bought (for 10 bucks) and said he could do better with a ruler :) - his set of calipers he uses for work are calibrated to 3 dp but cost SIGNIFICANTLY more... $\endgroup$
    – Solar Mike
    Sep 20, 2018 at 5:54
  • $\begingroup$ I still wouldn't trust the last decimal point, if it were actually critical that you hit say 7.00mm+/-0mm. A screen that says that could be anywhere from 6.995 to 7.0049. Whether that matters or not, is up to OP to decide... $\endgroup$ Sep 20, 2018 at 7:32
  • $\begingroup$ Please see this question and answer here: engineering.stackexchange.com/a/22340/2559 $\endgroup$
    – GisMofx
    Sep 20, 2018 at 10:49

1 Answer 1


A measuring device will have as two main contributions to its accuracy and precision, a calibration uncertainty and a scale uncertainty.

Consider a ruler. It expands and contracts as it is heated and cooled. Consider a beam that is being measured by that ruler. The beam does not expand and contract at the same extent as the ruler. The measurements at two different temperatures will give two different lengths. This is the calibration uncertainty. It causes readings to be inaccurate.

Consider a ruler marked in millimeters and one marked in inches. The former has a scale (device) uncertainty of $\pm 0.5$ mm. The latter has a device uncertainty of $\pm 0.5$ in $\approx 13$ mm. Device uncertainties affect the precision of the measurements, not the accuracy.

The best approach to obtain a rule of thumb is to use relative uncertainties rather than absolute.

Consider that you will measure a beam that is EXACTLY 1 m long regardless of temperature. You measure on a cold day and on a hot day using a ruler that expands and contracts by ... say ... 100 microns. Your relative calibration uncertainty is $100/10^6 = \pm 100$~ppm. Using the millimeter ruler, your relative device uncertainty is $0.5/10^3 = \pm0.05$%. Using the inch ruler, your relative device uncertainty is $13/10^3 = \pm 1.3$%.

As a rule of thumb, I routinely teach (in a colloquial manner) that better than $\pm 0.5$% is national labs standards, better than $\pm 1$% is calibration standards, better than $\pm 5$% is highly regarded, better than $\pm10$% is routinely reproducible, better than $\pm15$% is engineering seat of the pants, and anything higher than $\pm20$% is gossip.

Are these rules codified in some resource? Yes, in my hand-written or print out notes for some of my classes. Otherwise, no. The requirements for measurement tolerance (overall uncertainty) may be codified in a document that defines how the component is to be used. At that point, your job is to find a measuring device (gauge) that has no more than that overall uncertainty, presumably being better.

In summary, it is NOT about the "extra" number of significant digits. It is about the extent that you accept that you will have an uncertainty and will need to define/control it.

  • $\begingroup$ This also brings up a side-point about analogue vs digital measurements - with the ruler, the scale may be sensitive to only $\pm 0.5 \text{mm}$, but you have the advantage when reading it to be able to see which end of that 0.5mm it's at, i.e. is it closest to 7.0, or 7.5? A digital measuring device may offer less insight in the real world, even with an identical level of sensitivity. $\endgroup$ Sep 20, 2018 at 7:41
  • $\begingroup$ What you "see" is irrelevant in the expression of the uncertainty. The valid report of a measured value of something that is with the given range of 7 in length will always be $7.0 \pm 0.5$ with any device that measures only to that degree of precision. The word "sensitivity" seems to be a colloquial for precision; I have never seen it in used in literature standards. $\endgroup$ Sep 20, 2018 at 13:41
  • $\begingroup$ It's irrelevant in a statistical analysis/report, sure, but I was merely highlighting that in some situations, when offered an analogue or a digital scale, with the same sensitivity, it may be preferable to use the analogue scale, if you don't have access to a tool with greater sensitivity. $\endgroup$ Sep 20, 2018 at 13:53
  • $\begingroup$ Sensitivity means how small the gradations on a scale are. A micrometer is more sensitive than a ruler. Precision means how closely grouped the measurements are each time, i.e. it's applied to the measurements, not the measuring tool. A broken micrometer with a dodgey ratchet spring that triggers when it's not touching the part may produce measurements that are extremely imprecise, where the ruler is more reliable here... $\endgroup$ Sep 20, 2018 at 13:55
  • $\begingroup$ @JonathanRSwift Let's see what is said in the GUM. Sensitivity is a partial derivative the defines how one factor changes due to another with all else constant. What you call sensitivity is defined as precision. It is NOT dependent on time. What you call precision is better called frequency of measurement. A micrometer is more precise than a millimeter ruler, not more "sensitive". I agree with the use of reliability and take issue with (incorrectly) calling a broken micrometer imprecise rather than just unreliable. $\endgroup$ Sep 21, 2018 at 0:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.