1
$\begingroup$

I am running a simulation of fluid flow and then doing an actual physical experiment to confirm the results of the simulation.

My simulated results show a value of 0.529 m/s and my experiment show a result of 0.532 m/s. So the values differ by less than 1%. (* It really doesn't matter what the number refer to *)

I studied accuracy and precision in my Physics classes. But, I can't figure out how to "prove" to myself, using math or rational logic 8-), that a less than 1% difference is basically "accurate" or "the same"..

$\endgroup$
3
  • $\begingroup$ So, how are you taking the physical measurements? what are they? how accurate is each reading? how repeatable are the measurements or settings on the equipment between runs? can each run be "identical"? Get the answers and then do the calculations with the low and high values for each reading. See what that tells you. $\endgroup$
    – Solar Mike
    Apr 9 at 17:23
  • $\begingroup$ The accuracy of measurement is quite subjective, depends on its application, and the impact of the inaccuracy. For instance, if we build a 10 m long bridge, the 1% error in surveying will lead to 10cm misalignment, which is not good but acceptable. Now, for a 100m long bridge, the result will be unacceptable. In this case, the end result counts. $\endgroup$
    – r13
    Apr 9 at 19:28
  • 1
    $\begingroup$ With fluid flow simulation, you start worrying if the simulation is 50% different from the experiment, not 1%. With a 1% discrepancy, either you got lucky or else you tweaked the model to match the experimental data! $\endgroup$
    – alephzero
    Apr 10 at 12:06
3
$\begingroup$

Generally you would work out the precision of your measuring equipment. That'll give you a sensible +/- value either side of your measurement. You may find that your simulated result falls within this band, in which case your simulation agrees.

Also look into 'uncertainty' in measurement - this will provide you with a percentage representing how confident you are that your measurement is 'good' in the first place - you'll need to take multiple measurements to follow this route though. The National Physical Laboratory has a good introductory guide here.

$\endgroup$
1
$\begingroup$

We do not seek to prove whether two values are the same. We seek to determine whether two values differ from each other to a defined level of confidence.

Your simulation has a limit in its precision. This may be set by something as simple as the number of significant digits in a numerical constant that you use during the calculation. For example, setting the speed of light in a simulation as $3.0$ m/s gives it a precise to $\pm 0.05 \approx 1.7$%. This is equivalent to a device uncertainty of the simulation. Simulations that run multiple times, for example using random values as input, may give a scattering of values from which you are to determine the average. Any calculations to obtain an average should also obtain a standard uncertainty. The device precision $\Delta_D$ and the standard uncertainty $S$ are combined using a simple rule of quadrature (adding variances) to give a total uncertainty $U_T^2 = \Delta_D^2 + S^2$ on the value.

Your one-time measurement has a precision that is set by the device used to measure it. This is the tick marks on the device. Let's presume that your measurement device is perfectly accurate (perfectly calibrated). If you do more than one measurement to obtain an average, you also have a standard uncertainty. The device and standard uncertainty are combined again using a rule of quadrature.

To proceed with your inquiry, you should first report the two values with their calculated precisions. You can then apply confidence tests to prove (determine) whether the two values differ from each other to a given bounds.

$\endgroup$
0
$\begingroup$

A while ago, there was study done. They had a similar problem like yours. Namely, there was a bar, simply supported at the ends and it was in a flow with a pressure.

enter image description here

They gave the problem with the same parameters to about 15-20 expert researchers and professionals in the field, with the state-of-the-art hardware and software at the time. Arguably that was in the last millennium (so not state of the art anymore), however this problem is not complex by any measure.

The task at hand was to predict the position of the center of the beam after 7 milli seconds.

The result was the following:

enter image description here

As you can see the results after the initial peak vary... a lot. And that was by people that were considered experts with many years of experience.

In general, given that nowadays any designer without any real knowledge of statics, mechanics of materials, numerical methods, etc, can run simulations and produce nice looking images of loaded structures, generally when I see results from junior engineers that are:

  • in the same order of magnitude I am content
  • within 20% I am happy
  • within 10-5 % its cause for celebration
  • between 0 and 5% it's cause for alarm (something went to well and I need to double check the results).

So, if you validated your simulation results against the experiment and your are within 1%, then you should be really proud of yourself.

level of accuracy that is required.

Apart from the above (long) note of caution, there are some comments regarding te level of accuracy. The level of accuracy that is required can greatly differ and its depended on the field you are working on. 4 If you are working on everyday engineering problems, then you can get away easily by 20%.

However, if you are doing research work then the accuracy can be much more stringent. Take for example the recent Muon g-2 experiment that it's in the news right now:

  • $0.0011659\color{red}{2089}$ is the measured value and
  • $0.0011659\color{red}{1800}$ the prediction of the Standard Model

The red is the difference. Approximately $2.5\cdot 10^{-4}=0.00025 \%$.

$\endgroup$
2
  • $\begingroup$ And, to be clear, that error in the Muon g-2 experiment is considered HUGE. Which is why it's in the news: this is way outside the margin of error and is therefore indicative of "new physics" (though not to the 5-sigma confidence level used in physics, only 4.2-sigma). $\endgroup$
    – Wasabi
    Apr 9 at 19:48
  • $\begingroup$ Thanks for pointing out, that in this context the 0.00025% is considered huge deviation of error. That was exactly what I was trying contrast. $\endgroup$
    – NMech
    Apr 9 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.