Can redundancy be used instead of periodic calibration?

Question

A lot of production equipment requires periodic calibration as part the documentation that the equipment is under control. The companies selling the equipment knows this and sometimes lowers the price of the equipment and offer an expensive service agreement (better to sell a long time service than a product once).

But wouldn't it be valid to use redundancy and drop calibration - what are the chances that both drift or breaks at the same time (AND that the process comes out of control). If one goes out then it just time to order a calibration..

An example could be monitoring systems like temperature loggers. Having two loggers (perhaps from different manufactures) and just drop calibrating.

Do anyone have some arguments why this would NOT be such a good idea?

Update

Thank you very much for the big interest in the topic!

I changed the header to use the term periodic calibration instead of just calibration - in order to be more clear. I would also like to elaborate here a bit more on what I mean by periodic calibration and proven state of control.

In EU GMP guidelines for process equipment the following is stated:

5.3 Calibration

5.30 Control, weighing, measuring, monitoring and test equipment that is critical for assuring the quality of intermediates or APIs should be calibrated according to written procedures and an established schedule.

5.31 Equipment calibrations should be performed using standards traceable to certified standards, if existing.

5.32 Records of these calibrations should be maintained.

5.33 The current calibration status of critical equipment should be known and verifiable.

5.34 Instruments that do not meet calibration criteria should not be used.

5.35 Deviations from approved standards of calibration on critical instruments should be investigated to determine if these could have had an impact on the quality of the intermediate(s) or API(s) manufactured using this equipment since the last successful calibration.

EU Guidelines to GMP vol. 4 part II

This industry commonly uses two calibrations to prove/document that the equipment has been under control the entire production period (the period between before/start and after/end calibrations).

My point is that it seems unlikely that both devices measures wrongly AND that the process is out of control (unless the root cause affects both the measurements and condition). The idea of redundancy instead of calibration is derived from the principle of separating the control and monitoring system - since a drift would not be caught and the process could then go out of control/specs.

Answer 1

There are a few basic issues. The premise of this idea is that the errors of each measurement device will balance out, so with enough devices, the average value will stay roughly accurate. If not, at least you would be able to notice the difference between the two devices and know when to re-calibrate. For some types of measurements, this is roughly true, but for others it is not.

The first big problem is that some types of measuring devices will always tend to drift in the same direction. For example, if your device compares a force against a spring, that spring will get weaker over time as it is fatigued, never stronger, so your measured values will all drift in the same direction. Here, averaging the results from multiple devices wouldn't help, and you wouldn't even notice the error as the springs will decay at roughly the same rate.

The other fundamental problem is that even for a measurement where errors were bilateral and random, it would take a lot of devices to be statistically confident that your average was good. Just using two or three devices - all errors could well be in the same direction, so your average value would still be wrong. You would need a lot of measurement devices to get any precision in your average.

In addition to these two basic principles, there are a bunch of practical reasons to use calibrated measurement devices when values matter. One is that handling the specimen to measure it a number of times would take more time, machinery, and/or handling which increases costs and risk of damage. Another is that for most work governed by a standard, units will be defined with respect to an absolute value (usually a physical constant, but sometimes an artifact) and if your measurements don't relate to that value, you can't be sure that others will agree with them. For some critical work, a traceable lineage to a governmentally established constant is required behind each measurement.

For all of these reasons, it is almost always better to use one instrument that is periodically calibrated instead of a bunch of instruments with unknown accuracy and trying to get a statistical average.

---

Based on your edit I think I understand that you don't want to avoid calibration altogether, just to reduce the frequency and reduce the risk that an instrument drifting out of spec (but within it's calibration period) is going to lead to a failure of your final product. This is a good point - there is a risk analysis to be done of process failures based on the frequency of instrument failures, the frequency of control process failures, and the cost of a defective product. Based on those factors, you could add two instruments, maybe only one of which is in its designated calibration cycle, with the second just acting as a trouble monitor. If the two instruments disagree, you won't be certain which is right, but you'll at least know that something is going wrong and to check calibration. There's nothing wrong with this idea, and it will probably help detect some sudden failures, although the point that common mode failures won't be detected still stands. On the other hand, it's probably not a big economic impact to keep both instruments within their calibration period, (ie. don't reduce the calibration interval) and then you could have more confidence in your results.

Answer 2

The topic is – as easily seen from length and number of answers – not easy. This also means it does highly depend on every specific case to answer this topic.

Part I:
If we define that the process-parameter ( $p$ ) hast to be monitored. The measurement ( $m$ ) of the process parameter always contains an uncertainty ( $u$ ). So lets state the following model:

$m = p + u$

The purpose of every calibration is to specify the uncertainty. Strictly spoken the uncertainty can only be specified for the calibration which was already done. However, a very reasonable assumption is, that the uncertainty will not change dramatically. But, it will change over time. So the model will look like this:

$m = p + u(t)$

Your proposed redundancy has different effects on the uncertainty, depending on the way redundancy is achieved. One way could be to measure the same process parameter with one additional sensor of the same type (1). Another way would be to either use a different sensor (2) or even observe a different process parameter with a different sensor (3).

Just using the same sensor to observe the same parameter will most likely have a similar degradation of the calibration quality. Because the cause of the degradation is not changed, so both sensor signals have possibly a higher uncertainty over time.

The two other options have a potential to come up with an uncertainty-model which would not make the calibration un-necessary but might increase re-calibration periods.

Part II:
In any case there is a significant difference between error and uncertainty. If it would be possible to know the error of every measurement one could easily compensate for that.
But the only thing a calibration can provide is the uncertainty of a measurement. Or put it the other way around a calibration will provide you with an estimate how certain one can be that in (usually) 95% of all measurements the returned value will be in the specified interval around the true value. The following figure shows the upper and lower bounds of the uncertainty assuming a very simple degradation model.

Assuming you want to keep your process parameter at a level of "1" and your uncertainty has an initial value of +/-10% (which is huge but nicer to illustrate). The reason you have to control parameter in the first place is because you need to prove that your process (or product) has a specific quality. As it can be easily be seen without re-calibration (bounds m+ and m-, dotted) after some imaginary time the uncertainty has steadily increased. By re-calibrating in the middle of the interval you reduce the uncertainty (m+r and m-r, solid).

From your description I assume that you need to report or save the calibration before and after every period together with the measurement-logs of the process to prove your quality of the whole production cycle.

It is absolutely understandable to reduce the effort and associated costs of recalibration. The only way to do it is do gather more knowledge of this specific process.
Right now the brute-force approach is to calibrate two times and assume the uncertainty in between those calibrations was within in the two calibrations.
If it would be possible to obtain a better model for the uncertainty degradation the number of calibrations could be reduced. However, in order to have a better model it is necessary to combine a larger number of calibrations. One possible way might be to use all the calibrations and recalibration available. It might be possible to develop a better understanding of the uncertainty and thereby develop a better model which would extend the calibration intervals.

For example given the sensors would degrade in the manner depicted in the figure above, a solution could be to change the control parameters of the production system control over time. But the way to do this or if this is even possible highly depends on your process which we don't know yet.

Finally, even though it is counterintuitive having redundant sensors does not necessarily decrease uncertainty.
By adding an additional sensor of the same type to the system you basically double the number of measurements keeping the same uncertainty. The only way a redundant sensor reduces uncertainty is when the first sensor measurement was not representative of the system and did not only have uncertainty from the sensor but additional fluctuations from the system itself. Think of a measurement like shooting at a target. Depending on your skills you will have a certain probability of hitting the bulls eye. That means by shooting more often you only increase of hitting the bulls eye but you never decrease the spread of your shots. Every shot has the same probability of hitting the bulls eye. Likewise measuring more often increases the chance of measuring the "right" value, but this is not of interest here. It is of interest how certain you can be that every measured value is within a certain "interval around" the right value.

Summing it up:

1. Adding sensors to a measurement does not necessarily decrease the uncertainty of the combined measurement.

2. Having two sensors of the same type will not change the degradation rate of the combined measurement value compared to a single sensor.

3. If the possibility is big that one sensor would return erroneous values during the production run, than having redundant sensors makes a lot of sense. But, this has nothing to do with the uncertainty of the values. In this case the post-calibration would reveal that the process was not properly monitored due to sensor failure so there is not way to assure the quality of the production run at all. In case of sensor failure the other sensor would have kicked in and provided (at best) the same quality as if one sensor would have survived the whole production run.

4. The only way to reduce the number of calibrations is to collect all calibrations from the past and try to build a sensor-degradation-model from that which might show that the current number of calibrations is unnecessarily high.

5. By utilising different sensor types or measurement of process parameters it might be possible to come up with a uncertainty model which degrades slower and is more robust with respect to sensor failure.

Answer 3

Having two measuring devices usually a bad idea for avoiding recalibration. If the devices give different results, you don't know which is right until you recalibrate, or do some fault-finding. Of course two different readings tell you something is wrong with the measurement system, which is more information than just one wrong reading.

If you have three or more devices, it is often sensible to use "majority voting" to decide that one device is giving bad data and should be ignored.

As other answers have said, you also need to consider "common-mode" causes of errors which affect every device the same way - not only "uncontrollable" things like environmental conditions, but also issues like using a common electrical power supply, which might go out of specification without any indication and affect the output of all the devices.

Answer 4

It's always a trade-off. Usually between economics and accuracy.

If you have two temperature loggers, and one drops out of calibration, but you don't know which one, you now have much larger uncertainties on your temperature observations. And you've got twice the bill for recalibration.

The nice thing about calibrated kit, is that this, in most cases, gives you a pretty reasonable idea of the distribution of errors on your observations, and that makes it much easier to account for them. Whereas having N bits of uncalibrated kit give you a lot of uncertainty about what the actual distribution of errors is, which makes it much harder to manage them and account for them. So you've got a system where you've paid extra for redundancy, but have not compensated for the reduced accuracy.

If you have to calibrate each time one of your recorders disagrees with another, you could end up doing more than twice the recalibrations you do currently.

And if the two recorders drift off calibration, but in sync with each other, then you'll miss a critical recalibration. And understanding the distribution of such common-mode failures is really difficult; so once again, you won't have a known error distribution that you could measure and manage; instead you'll have an unknown error distribution, and that will increase your costs.

Answer 5

More independent measurements with the same accuracy only change the probability profile of the error, not the worst case error.

A good example is rolling two dice. You can consider each die as providing a average value of 3.5±2.5. Any one roll of any one die can be anywhere in that range with equal probability. However, the average of two dice has a triangular profile. Six different possible combinations result in 3.5, whereas only one combination each results in 1 or 6.

Rolling more dice narrows the probability of the average around 3.5, but the probability of 1 or 6 never goes to 0. The worst case is still the same as before. No amount of extra dice can change the worst case.

Since most of the time for production measurements, we care about the worst case error, multiple measurements doesn't help.

And, note that the better average case of the multiple dice only works because each measurement (each die) is independent of the others. The same measurement process, even performed independently by different instruments, may have some common sources of error. For example, temperature, atmospheric pressure, age, etc, can all affect different instruments that measure the same thing in the same way.

It's rare that you get truly independent measurements, and then you only get a better average answer, not a better worst case anyway.