What can we do?

This issue contains a process development tool that is difficult to grasp and use, but it is so important that I feel I must explain it as simply as I can. The tool is so powerful that most industries have embedded it in their business methods. A few companies - Motorola, DuPont, General Electric, Boeing and Honeywell - have found this tool to be a great success enabler. We live in high tech times where some products have failure rates of less than one in a billion or even a trillion. This vital technique is one any venturer can adopt-after some practice.

Consider an invention assembled by a handyman from available modules in a novel container. Battery driven, it must operate unattended for long periods and meet government regulations. It's a real improvement. The crunch comes with ensuring such performance in production. Factors involved are materials supplied by others, design of the process and product to be manufactured and execution needed to make it all happen. Hunches and anecdotes cannot suffice. I will illustrate this tool using a consistency problem commonly encountered in all three factors above.

Our work begins with supplier selection and creating guarantees. Seven manufacturers make the modules at a range of prices that may or may not reflect their performance levels and reliabilities. Two of these make sensors; three make smart batteries while two make the mechanical item. The smart batteries differ in size, shape and mode of operation. There are 12 configurations, but since the method is general, I can illustrate how the tool works by selecting a battery supplier and guarantee.

To take the market, we must certify that our product will meet new highs in performance. The fundamental question is "What can we certify, at what level of performance, and at what cost?" The following trip is not for the faint of heart. But I will keep it simple.

Our first step is to obtain quotes from each battery supplier along with their performance guarantees. The quotes indicate that improved performance comes at a price. After comparing this information with what we know about the market, we tentatively conclude that a battery life of at least 12 months is needed to take the market at a price premium of 10%. All three venders assert they can meet the 12-month guarantee.

Now assume we assemble 36 devices for testing under field conditions. We will use these to test 12 batteries from each supplier, A, B, and C. We should be able to detect differences in battery performance (average life) and reliability (consistency). Battery-life results are given in tables 1 and 2.

Table 1 Battery test results and summary

Type	S/N	Mos.	Type	S/N	Mos.	Type	S/N	Mos.
A	1	13	A	5	13	A	9	13
A	2	13	A	6	14	A	10	11
A	3	14	A	7	13	A	11	13
A	4	12	A	8	12	A	12	12
B	1	14	B	5	3	B	9	14
B	2	12	B	6	15	B	10	12
B	3	15	B	7	11	B	11	17
B	4	10	B	8	12	B	12	18
C	1	14	C	5	15	C	9	15
C	2	15	C	6	18	C	10	13
C	3	17	C	7	14	C	11	15
C	4	14	C	8	13	C	12	13

Table 2 Performance summary

Supplier	A	B	C
Avg. Months	12.8	12.8	14.7
Range*	11-14	3-18	13-18

*Min-max (Consistency)

All three battery types averaged above our target life. However, batteries from suppliers A & B failed to exceed our goal essentially half of the time while those from C always exceeded 12 months. Supplier B produced a grossly defective part, lasting only three months. Supplier C is preferred. Our hunch is that failures will be rare and that we can guarantee our product for 12 months. But how rare is "rare"? Do we proceed with our hunch or do we determine how safe we are? Let's revisit at what we know.

From our research on the product, we know that current battery consistency (or range = max - min) is (9 - 7) = 2. This compares with ranges of 3, 15 and 5 for suppliers A, B & C. Back when battery life was four months, the range was only 3.5 to 4.5-pretty narrow and consistent. Maybe scatter increases with performance. Is 12 months as safe as we thought, given the small number of batteries tested? How can we know?

This is where the tool alluded to enters. When throwing darts, our darts land on or near target most of the time. Our skill can be characterized by two quantities: our average hit distance from bull's eye and the size of our "pattern". Mathematicians have determined that most measurable events have averages and "patterns" similar to what we find throwing darts. And they have figured out how to calculate a quantity known as the Standard Deviation, SD, descriptive of our pattern. SD is universally true although it may take different forms. To keep things simple, I will use the dart-throwing pattern and apply that to the question of battery life.

SD is illustrated by the bell curve shown in Figure 1. In the central region, its curvature is downward. On each wing, the bell curve levels out such that the curvature turns upward. The point at which the curvature changes from down to up is the SD. Pause here and estimate SD by the rule above. A second method: draw a line parallel to the base 40% down from the peak-the line will intersect the bell curve at the SD points. Figure 1 was scanned in from a well-known text on quality control. Did the author really understand SD? Even experts can err-demand proof, always. The original was schematic, right?


Figure 1 showing % of events occurring per SD away from the average (central peak).

The SD gives us a shortcut way to estimate failure rates from test data. For example, 16% of the time, individual batteries test more than one standard deviation below the lot average. 2% of the time, they test more than two Standard Deviations below the average. These percentages are shown in Figure 1.

By drawing bell curves for our data, the points of curvature change can be estimated. To illustrate, I did this for our data and drew arrows in Figure 2 for each supplier that show the SD locations. My estimate of the SD for supplier A is 0.9, that for B is about 3.9, and for C it is 1.6. Your estimates might vary slightly from mine, but that would not change our conclusions.



Figure 2 SD points for battery suppliers

To determine if a guarantee of 12 is realistic we first calculate where our 16% failure level is for supplier C. Those batteries averaged 14.7 months with a SD of 1.6 months. The 16% failure level (about one in six) is thus 14.7 - 1.6 = 13.1 months. At two SD below the average, 14.7 - 3.2 = 11.5 months, we can expect a 2% failure rate (one in 50). So our guarantee of 12 will have a failure rate of maybe 5%. A guarantee of 10 months seems very safe and is still 25% better than the current average. If that is enough to take the market at prices reflecting the added cost, we should proceed with a 10-month guarantee. But if a highly consistent (say not more than one failure in 500) guarantee for 12 months is emphatically needed to move the market our way, we can ask supplier C to reduce variability below SD=0.9 or improve average performance or both. We can also ask the sensor suppliers to reduce battery drain.

I have only considered battery life. Analyzing the other sources of product variation may impact our guarantee in like manner. This is one reason why moderately complex innovations are so hard to sell. Qualifying market-worthy product can be expensive.

Without considering SD, we might well have set our guarantee at 12 and wondered why one in 20 of our prized inventions suffered battery failure to the detriment of our market and bank account. Most developments are less complex. But if you can grasp how to use your new tool, you can avoid innumerable commercial problems. Understanding data is the single most critical issue in process and product development.

If you can look at a bell curve and "know" by that inspection where to set your guarantee, you can do away with hunches and join the elite. Modern competition demands nothing less.

Congratulations, if you are still with me, this one is tough.

Next issue I will explore the important discipline of forecasting.

For further reading I suggest Statistical Methods for Quality Improvement, Thomas Ryan. Wiley, ISBN 0-471-84337-7

© Copyright 2000, 2009 by Harry Rosenberg. All rights reserved.

Back to Top | Menu