How can the vendor be approved/disapproved based on validation test data?

Question

The power requirement for a sub system is 5V +/- 10% @ 1A of current. A RFQ was issued and a prospect vendor was selected. A first article of inspection (FAIR) has been submitted which includes 15 power supply samples. The samples have been tested with and without a 5 ohm resistor load using two separate calibrated DMM’s . Without the resistor the output voltage is 5V with little variation. Under a 5 ohm load the numbers are not too great. Below are the measurements (V):

4.56, 4.71, 4.95, 5.12, 4.61, 4.83, 4.65, 4.69, 4.79, 4.81, 4.67, 4.74, 4.63, 4.74, 4.81

Should the vendor be approved as a supplier?

Answer 1

TL;DR: It depends, but probably not

You might argue that "all 15 samples are within spec, so the supplier should be approved". Not so fast: depending on the parameters of your full production run, the 15 samples may or may not be statistically significant.

---

There are numerous calculators online to do these calculations; I used this one. A good resource for the actual formulas is online at NIST's Engineering Statistics Handbook or in any undergraduate Statistics textbook.

From the spec, we know that our confidence interval is 10% (two-sided). This means that any parts within 10% of nominal are acceptable.

Next, we need to determine the confidence level. This is usually 95% but sometimes 99%, and represents "how sure" we can be of the result.

The final piece of information we can give if it's known is the population size. In this scenario, this is the total number of parts to be ordered from the vendor over the lifetime of the product/process. Most calculators allow this to be left blank and, if missing, assume a large value, because the effect of the population size decreases as it grows in relation to the sample size.

Assuming a 95% confidence level and our 10% confidence interval, with population left blank, we need a sample size of 96 parts in order to have a statistically significant result. Increasing to a confidence level of 99% requires a sample size of 166 parts.

So, for industry-standard confidence levels, we cannot conclude that the vendor be approved based on the initial sample of 15 parts alone.

Wait a minute—the entire FAIR run was within spec! Well yes, but what's to say the next 15 won't be out of spec? We don't know—that's why we have statistics! :-)

---

Well, under what circumstances is our sample significant?

Just for illustration, I entered population values until I found a statistically-significant result at $n=15$: for 95% confidence level, $n=15$ samples would be significant only if the population (total production run) is 18 parts! For a confidence level of 99%, the situation is even worse: significance only if the production run is 16 parts!

---

Other Notes:

The calculations above assume that the process follows a normal distribution and that the sample is representative of the population. In practice, both of these assumptions may be inaccurate.

Answer 2

The average and standard deviation from the above sample set are:

• Average ($\mu$): 4.754V
• Standard Deviation ($\sigma$): 0.142368
• Mean: 4.74
• USL: 5.5
• LSL: 4.5
• C$_{p}$ : 1.1706
• C$_{pl}$: 0.5947
• C$_{pu}$: 1.7466
• C$_{pk}$: 0.5947

Equations

$\text{Pr}(\mu - \sigma \leq x \leq \mu + \sigma) \approx 0.6827$

$\text{Pr}(\mu - 2\sigma \leq x \leq \mu + 2\sigma) \approx 0.9545$

$\text{Pr}(\mu - 3\sigma \leq x \leq \mu + 3\sigma) \approx 0.9973$

$C_{p} = \frac{(USL - LSL)}{6\sigma}$

$C_{pu} = \frac{USL-\mu}{3*\sigma}$

$C_{pl} = \frac{\mu-LSL}{3*\sigma}$

$C_{pk} = min(C_{pu},C_{pl})$

Based on the above data, it appears that the process is between 1 sigma and 2 sigma capable. Approximately about 10% of the product could fall outside of the requirements.

Also a sample set of 30 is considered a generally acceptable sample size. The sample set in this problem is 15 which is on the low side.

---

Suggestions:

• Check with the vendor if they sorted the sample parts
• The power supply is not delivering the required 1A of current at full load
• The current carrying conductor might not be capable to delivering the current at full load
• The power supply might have some minor design issues too
• Also check the test system too

The distribution appears to be a normal bell curve. A supplier Quality engineering might be able help address this issue.

Answer 3

I agree with What Trevor said in a comment. You specified 5V ±10% (4.5-5.5 V) and that's what you got. I don't see the problem here. All the samples you got are within spec. There may be other reasons to not use this vendor, but samples failing to meet specs is not one of them.

However, first article samples don't tell you that the vendor is OK, they only tell you whether the vendor is definitely not OK. In the real world, you don't just qualify a vendor and then blindly buy from them after that. It is a ongoing relationship. The samples were all within spec, but then again, unless the vendor is really stupid, of course they were. They can afford to manually check and hand-pick 15 items to send you.

Especially since one out of 15 was quite close to the edge, you should have the vendor explain to you how their design or calibration procedure guarantees all the units will be in spec, how you know they will stay in spec over the expected lifetime, etc. In short, it's still your job to have some confidence that these sample are representative and that you know there is a system in place you believe in to deliver future product in spec.

If you are really worried about this, then set up your own incoming inspection. You could possibly do this with a random sampling from every lot received, or every unit, depending on how critical this is to you.

Of course if you're really this worried, then your original specs are probably too loose. Allowing the bare power supply to be ±10% sounds unnecessarily high in the first place. That's not pushing any limits, so tighter requirements like ±5% shouldn't cost much more, if any. Some electronic parts are specified for 5V ±10%, so you can't have the bare supply eat up the entire error budget. The voltage ulitimately delivered where it is used will have a wider tolerance than whatever it is at the supply.