Draft comparison of electronic reliability prediction methodologies
1 like4,675 views
The document assesses the effectiveness of various electronic reliability prediction methodologies using field data from British and Danish manufacturers. It highlights discrepancies between predicted and actual failure rates across several models, emphasizing the sensitivity of predictions to physical parameters. The findings suggest that caution should be exercised when selecting prediction techniques, as models show significant variation in their reliability estimates.
![A Comparison of Electronic Reliability Prediction Methodologies
J.A.Jones and J.A.Hayes
International Electronics Reliability Institute
Department of Electronic and Electrical Engineering
Loughborough University of Technology
Leicestershire, LE11 3TU, United Kingdom
Tel: 01509-222897 Fax: 01509-222854
1. SUMMARY AND CONCLUSIONS
One of the most controversial techniques used at present in the field of reliability is the use of
reliability prediction techniques based on component failure data for the estimation of system
failure rates. The International Electronics Reliability Institute (IERI) at Loughborough
University are in an unique position. Over a number of years a large amount of reliability
information has been collected from leading British and Danish electronic manufacturing
companies. This data is of such high quality that IERI are able to perform the comparison
exercise with a number of boards of different types.
A number of boards were selected from the IERI field failure database and their reliability
was predicted and compared with the actual observed performance in the field. The prediction
techniques were based on the MIL-217E, HRD4, Siemens (SN29500), CNET, and Bellcore
(TR-TSY-000332) models. For each method the associated published failure rates were used.
Hence parts count analyses were performed on a number of boards from the database and
these were compared with the failure rate observed in the field. The prediction values were
seen to differ greatly from the observed field behaviour and from each other. Further analysis
showed that each method of prediction was sensitive to widely different physical parameters.
This suggests that predictions obtained by the different models can't be compared and that
great care should be taken when selecting a prediction technique since the physical
parameters of the system will affect the prediction obtained in possibly unforeseen ways.
2. INTRODUCTION
Reliability prediction often effects major decisions in system design. It is based on the
assumption that systems fail as a result of failures of component parts, and those parts fail
partly as a result of exposure to application stress[1]. This means that by some consideration
of the structure of such a piece of equipment and by further consideration of its usage it is
possible to obtain an estimate of the systems reliability in that particular application.
There are many reasons why this task may be necessary. These include feasibility evaluation
where the compatibility of a design concept is weighed against the design reliability
requirements for acceptance, and design comparison where different parts of a system can be
compared and any necessary trade off such as cost, reliability, weight etc. can be made.
Further uses are for the identification of potential reliability problems and as a reliability input
into other tasks such as maintainability analysis, testability evaluations and FMECA. [2]
1](https://image.slidesharecdn.com/draftcomparisonofelectronicreliabilitypredictionmethodologies-120226115557-phpapp01/85/Draft-comparison-of-electronic-reliability-prediction-methodologies-1-320.jpg)
![Electronic failure prediction methodology (EFPM) is normally carried out in two stages. The
first stage is known as parts count analysis and requires comparatively little information about
the system. This method is generally used early in the design phase to obtain a preliminary
estimate of the system reliability. The second stage is known as parts stress analysis and
involves detailed knowledge about the system but can, in principle, provide a more realistic
estimate of the reliability. The parts stress method tends to be used towards the end of the
design cycle when actual circuit parameters have been established.
Rightly or wrongly reliability prediction methods are widely accepted throughout the
electronics industry. These methods are often used as a yardstick for the comparison of
different equipment. However, many manufacturers have commented that the models can be
wildly inaccurate when compared with the performance in the field, particularly in the case of
the observed failure rates of modern microelectronic devices, and their use can lead to
increased costs and complexity while deluding engineers into following a flawed set of
perceptions and leaving truly effective reliability improvement measures unrecognised[1].
In general the telecommunication industry models offer improved accuracy when compared
to other models due primarily to the quantity and quality of in house data on mature
equipment but tend to be less accurate for newer designs and for similar systems produced by
other industries. The main advantage of the MIL-217 approach is its wide acceptance
throughout the defence industry. This enables it to be used as a general yardstick which
allows comparison between different equipment. However it has many shortcomings witch
have been the subject of much discussion on recent years.
IERI at Loughborough University are in an unique position. A collaborative exercise
involving a number of British and Danish electronic systems manufacturers has resulted in
the establishment of a database containing high quality data at the system, board and
component level. This has enabled IERI to carry out predictions on a large number of boards
and to compare these with observed performance in the field. The initial investigation
involved parts count analysis only and It should be stated that some of the predictions
described in this paper were performed using older versions of some of the handbooks that
have more recently been updated. This means that although the principles of the prediction
remain the same the actual figures obtained from these now obsolete handbooks may differ
from those obtained from the new, updated versions.
3. THE THEORY OF RELIABILITY PREDICTION
A electronic system can be considered to be a network of components all interconnected to
one another in various complex ways. This real life model is however unsuitable for
reliability analysis since it is far too complex. In order to study the reliability of systems a
number of assumptions need to be made.
1) Any component failure causes a system failure.
This is normally modelled as a series configuration (or chain structure). This is the simplest
of the many models available and as such is the most widely used for reliability modelling of
systems. A series configuration of n items will have a reliability function defined by (1)
2](https://image.slidesharecdn.com/draftcomparisonofelectronicreliabilitypredictionmethodologies-120226115557-phpapp01/85/Draft-comparison-of-electronic-reliability-prediction-methodologies-2-320.jpg)
![R(t) = P( x 1 , x 2 , ... , X n ) = P( x 1)P( x 2 | x 1)P( x 3 | x 1 x 2) ... P( x n | x 1 x 2 ... x n-1) (1)
where P(x1) etc. Is the probability that item X1 will fail and P(x2| x1) etc. is the conditional
probability that X2 will fail given X1 has failed.
2) The components that make up the system must be independent, This means that a failure by
a single component must not affect other components in the system.
If the n items x1,x2, ... ,xn are independent, then
n
R(t) = P( x 1)P( x 2)... P( x n ) = p( x )
i=1
i (2)
This suggests that assuming that no component failure affects any other then the reliability of
a system made up of these components can simply be calculated by multiplying the
probability of failure for each component in the system together. However in a general system
this can be difficult since each component’s probability of failure could be a complex
function. If however simple functions are used then it is possible to proceed further.
3) The component failure behaviour must be governed by a constant-hazard model
This last assumptions means that if the component model is e- i t then equation (2) becomes
n
n (- i t )
R(t) =
i=1
e- i t = e i=1
(3)
This equation is the not only the most commonly used and the most elementary system
reliability formula it is also the most commonly abused. It should be remembered that in order
for this equation to be valid then the condition given in clause 3 above must be met. There is
ample evidence to suggest that this is not in fact the case [4] but data handbook suppliers and
users still assume that this is true. Recently methodologies have been developed [5] that will
allow prediction of system reliability but do not make this assumption.
4. OVERVIEW OF PREDICTION METHODOLOGIES
In most of the available prediction methodology handbooks the equation for the reliability of
a system given in equation (3) is modified by the addition of different multipliers, called
factors. These factors relate to parameters which can effect the overall reliability of a
system such as environment, stress level, temperature etc. In general the equation for system
failure rate , given the failure rate of the constituent components, will be
3](https://image.slidesharecdn.com/draftcomparisonofelectronicreliabilitypredictionmethodologies-120226115557-phpapp01/85/Draft-comparison-of-electronic-reliability-prediction-methodologies-3-320.jpg)
![n m
SS = E . Gi F .Ni (4)
i=1 k 1 k
where Gi is generic failure rate for the i'th device, Fk is the stress factor multiplier for the
k'th stress type for the i'th device, N i is the quantity of i'th device type, and E is the
environment factor for the system.
Each model used in this paper uses an equation similar to (4) and is described briefly below.
4.1 BELLCORE
This method is defined in [6] and was developed by Bell communications research for use by
the electronics industry so that they would be aware of Bellcore’s' view of the requirements of
reliability prediction procedures for electronic equipment’s. The data presented is based upon
field data, laboratory tests, MIL-HDBK-217E, device manufacturers data, unit suppliers data
or engineering analysis. The factors used in this model take into account the variations in
equipment operating environment, quality and device application conditions such as device
temperature and electrical stress level.
4.2 CNET
This method is defined in [7] and was developed by French National Centre Of
Telecommunications. The data used comes from the analysis of breakdowns of the
equipment used by the military and civil Administration, and other equipment and component
manufacturers. This handbook forms a common base intended to make reliability predictions
uniform in France. The factors used in this model take into account the variations in
equipment operating environment, quality and device temperature. It should be noted that the
method used in this paper is the 'simplified method' so called because it comes from a British
Telecom translation of the important parts of the actual CNET handbook.
4.3 HRD4
This method is defined in [8] and was developed by British Telecom Materials and
Components Centre for use by designers and users of electronic equipment so that there exists
a common basis exists for system reliability prediction. The generic failure rates given in the
handbook are estimates of the upper 60% confidence levels, based upon, wherever possible,
data collected from the in service performance of the equipment installed in the UK inland
telecommunications network. Where such data is not available for particular components,
alternative sources or estimated values have been used and the status of the source indicated
by a letter code. The factors used in this model take into account the variations in
equipment operating environment, quality and device temperature.
4.4 MIL-217
Mil-Hdbk-217[9] was developed by the US Department of Defence with the assistance of the
military departments, federal agencies, and industry for use by the electronic manufacturers
supplying to the military. The handbook describes two methods, namely parts count and parts
stress which are used to predict the reliability of electronic components, systems, or
subsystems in different stages of the design. The failure rates given in the handbook have in
4](https://image.slidesharecdn.com/draftcomparisonofelectronicreliabilitypredictionmethodologies-120226115557-phpapp01/85/Draft-comparison-of-electronic-reliability-prediction-methodologies-4-320.jpg)
![the main derived from test bed and accelerated life studies. The factors used in this model
take into account the variations in equipment operating environment, and device quality.
4.5 SIEMENS
This method is defined in [10] and was developed by Siemens AG for the use of Siemens and
Siemens associates as a uniform basis for reliability prediction. The standard presented in the
document is based on failure rates under specified conditions. The failure rates given were
determined from application and testing experience taking external sources (e.g. Mil-Hdbk-
217) into consideration. Components are categorised into many different groups each of
which has a slightly different reliability model. The factors used in this model take into
account the variations in device operating temperature and electrical stress.
5. ANALYSIS OF PREDICTION METHODOLOGIES
Six different board designs were selected from the IERI database. These designs were chosen
such that they contained a wide range of component types and were from several different
applications. Table 1 describes the environments and applications of the boards.
Table 1:Brief description of the circuit boards
BOARD ENVIRONMENT APPLICATION
1 Ground Mobile Radio System
2 Ground Mobile Radio System
3 Ground Benign Telephone Exchange
4 Ground Benign Telephone Exchange
5 Ground Mobile Command System
6 Ground Mobile Command System
The analysis of two of the boards will be given in detail to show the methodology and to
illustrate how the models can be used for parts provisioning purposes.
5.1 CIRCUIT BOARD ONE
The first board design chosen from the IERI database contained 338 components with six
different component types. The board came from a system used in a high quality radio
application. Table 2 contains the predicted failure rate for each component type used on the
board. Each failure rate is given in FITs and is the failure contribution for each component
type on the board taking into consideration environment, component numbers etc.
Table 2:Contribution to failure rate by each component type for circuit board one.
DEVICE BELLCORE CNET HRD4 MIL-217 SIEMENS
Ceramic Multilayer Capacitor 210 45 3 4 35
pn-Junction Diode 125 300 300 9 25
Bipolar Digital IC 936 390 168 50 60
Metal Oxide Resistor 1590 596 51 52 795
Discrete Bipolar Transistor 7812 20937 4000 1225 1250
Tantalum Electrolytic Capacitor 1350 3780 40 594 180
Total 12023 26048 4562 1934 2345
5](https://image.slidesharecdn.com/draftcomparisonofelectronicreliabilitypredictionmethodologies-120226115557-phpapp01/85/Draft-comparison-of-electronic-reliability-prediction-methodologies-5-320.jpg)
![stress, while others are more sensitive to the discrete factors used to model environment
and quality.
Table 7: Most sensitive parameter in each prediction model
Prediction Model Greatest sensitivity
Bellcore Electrical Stress
CNET(Simplified) Quality
HRD4 Quality
MIL-217E Environment, Quality
Siemens Temperature
It is not surprising that direct comparisons of the models result in wide variations. Although
the models are based upon the same criteria there is disagreement about the effects the
different parameters have on the failure rate.
Also by carefully examining the models, it was observed that although the quality levels are
clearly defined within each procedure, it is extremely difficult to find a quality level
description that is compatible across all models. In addition every organisation has developed
their model according to the experience they have obtained in the field and have tailored the
model to meet their specific needs.
It is inadvisable to compare the models' prediction with field performance unless the systems
used are manufactured according to the guidelines and procedures that are specified by the
model designers. Under such circumstances the system manufacturers would argue that their
models are suitable for reliability prediction.
Care must also be taken when comparing the predicted reliability with that observed in the
field as prediction models assume a constant hazard rate. It has been observed however that
early life failures do occur in the field even after system burn-in [4]
Even if the above considerations are taken into account, there is no guarantee that the field
reliability will be the same as that predicted. This is due to the underlying reason that models
are generally simple empirical approximations. Indeed it is postulated that because of this,
their use should actively be discouraged. Moreover, they do not take into account many of
the other critical factors such as vibrations, mechanical shock, etc.
14](https://image.slidesharecdn.com/draftcomparisonofelectronicreliabilitypredictionmethodologies-120226115557-phpapp01/85/Draft-comparison-of-electronic-reliability-prediction-methodologies-14-320.jpg)
Draft comparison of electronic reliability prediction methodologies
- 1. A Comparison of Electronic Reliability Prediction Methodologies J.A.Jones and J.A.Hayes International Electronics Reliability Institute Department of Electronic and Electrical Engineering Loughborough University of Technology Leicestershire, LE11 3TU, United Kingdom Tel: 01509-222897 Fax: 01509-222854 1. SUMMARY AND CONCLUSIONS One of the most controversial techniques used at present in the field of reliability is the use of reliability prediction techniques based on component failure data for the estimation of system failure rates. The International Electronics Reliability Institute (IERI) at Loughborough University are in an unique position. Over a number of years a large amount of reliability information has been collected from leading British and Danish electronic manufacturing companies. This data is of such high quality that IERI are able to perform the comparison exercise with a number of boards of different types. A number of boards were selected from the IERI field failure database and their reliability was predicted and compared with the actual observed performance in the field. The prediction techniques were based on the MIL-217E, HRD4, Siemens (SN29500), CNET, and Bellcore (TR-TSY-000332) models. For each method the associated published failure rates were used. Hence parts count analyses were performed on a number of boards from the database and these were compared with the failure rate observed in the field. The prediction values were seen to differ greatly from the observed field behaviour and from each other. Further analysis showed that each method of prediction was sensitive to widely different physical parameters. This suggests that predictions obtained by the different models can't be compared and that great care should be taken when selecting a prediction technique since the physical parameters of the system will affect the prediction obtained in possibly unforeseen ways. 2. INTRODUCTION Reliability prediction often effects major decisions in system design. It is based on the assumption that systems fail as a result of failures of component parts, and those parts fail partly as a result of exposure to application stress[1]. This means that by some consideration of the structure of such a piece of equipment and by further consideration of its usage it is possible to obtain an estimate of the systems reliability in that particular application. There are many reasons why this task may be necessary. These include feasibility evaluation where the compatibility of a design concept is weighed against the design reliability requirements for acceptance, and design comparison where different parts of a system can be compared and any necessary trade off such as cost, reliability, weight etc. can be made. Further uses are for the identification of potential reliability problems and as a reliability input into other tasks such as maintainability analysis, testability evaluations and FMECA. [2] 1
- 2. Electronic failure prediction methodology (EFPM) is normally carried out in two stages. The first stage is known as parts count analysis and requires comparatively little information about the system. This method is generally used early in the design phase to obtain a preliminary estimate of the system reliability. The second stage is known as parts stress analysis and involves detailed knowledge about the system but can, in principle, provide a more realistic estimate of the reliability. The parts stress method tends to be used towards the end of the design cycle when actual circuit parameters have been established. Rightly or wrongly reliability prediction methods are widely accepted throughout the electronics industry. These methods are often used as a yardstick for the comparison of different equipment. However, many manufacturers have commented that the models can be wildly inaccurate when compared with the performance in the field, particularly in the case of the observed failure rates of modern microelectronic devices, and their use can lead to increased costs and complexity while deluding engineers into following a flawed set of perceptions and leaving truly effective reliability improvement measures unrecognised[1]. In general the telecommunication industry models offer improved accuracy when compared to other models due primarily to the quantity and quality of in house data on mature equipment but tend to be less accurate for newer designs and for similar systems produced by other industries. The main advantage of the MIL-217 approach is its wide acceptance throughout the defence industry. This enables it to be used as a general yardstick which allows comparison between different equipment. However it has many shortcomings witch have been the subject of much discussion on recent years. IERI at Loughborough University are in an unique position. A collaborative exercise involving a number of British and Danish electronic systems manufacturers has resulted in the establishment of a database containing high quality data at the system, board and component level. This has enabled IERI to carry out predictions on a large number of boards and to compare these with observed performance in the field. The initial investigation involved parts count analysis only and It should be stated that some of the predictions described in this paper were performed using older versions of some of the handbooks that have more recently been updated. This means that although the principles of the prediction remain the same the actual figures obtained from these now obsolete handbooks may differ from those obtained from the new, updated versions. 3. THE THEORY OF RELIABILITY PREDICTION A electronic system can be considered to be a network of components all interconnected to one another in various complex ways. This real life model is however unsuitable for reliability analysis since it is far too complex. In order to study the reliability of systems a number of assumptions need to be made. 1) Any component failure causes a system failure. This is normally modelled as a series configuration (or chain structure). This is the simplest of the many models available and as such is the most widely used for reliability modelling of systems. A series configuration of n items will have a reliability function defined by (1) 2
- 3. R(t) = P( x 1 , x 2 , ... , X n ) = P( x 1)P( x 2 | x 1)P( x 3 | x 1 x 2) ... P( x n | x 1 x 2 ... x n-1) (1) where P(x1) etc. Is the probability that item X1 will fail and P(x2| x1) etc. is the conditional probability that X2 will fail given X1 has failed. 2) The components that make up the system must be independent, This means that a failure by a single component must not affect other components in the system. If the n items x1,x2, ... ,xn are independent, then n R(t) = P( x 1)P( x 2)... P( x n ) = p( x ) i=1 i (2) This suggests that assuming that no component failure affects any other then the reliability of a system made up of these components can simply be calculated by multiplying the probability of failure for each component in the system together. However in a general system this can be difficult since each component’s probability of failure could be a complex function. If however simple functions are used then it is possible to proceed further. 3) The component failure behaviour must be governed by a constant-hazard model This last assumptions means that if the component model is e- i t then equation (2) becomes n n (- i t ) R(t) = i=1 e- i t = e i=1 (3) This equation is the not only the most commonly used and the most elementary system reliability formula it is also the most commonly abused. It should be remembered that in order for this equation to be valid then the condition given in clause 3 above must be met. There is ample evidence to suggest that this is not in fact the case [4] but data handbook suppliers and users still assume that this is true. Recently methodologies have been developed [5] that will allow prediction of system reliability but do not make this assumption. 4. OVERVIEW OF PREDICTION METHODOLOGIES In most of the available prediction methodology handbooks the equation for the reliability of a system given in equation (3) is modified by the addition of different multipliers, called factors. These factors relate to parameters which can effect the overall reliability of a system such as environment, stress level, temperature etc. In general the equation for system failure rate , given the failure rate of the constituent components, will be 3
- 4. n m SS = E . Gi F .Ni (4) i=1 k 1 k where Gi is generic failure rate for the i'th device, Fk is the stress factor multiplier for the k'th stress type for the i'th device, N i is the quantity of i'th device type, and E is the environment factor for the system. Each model used in this paper uses an equation similar to (4) and is described briefly below. 4.1 BELLCORE This method is defined in [6] and was developed by Bell communications research for use by the electronics industry so that they would be aware of Bellcore’s' view of the requirements of reliability prediction procedures for electronic equipment’s. The data presented is based upon field data, laboratory tests, MIL-HDBK-217E, device manufacturers data, unit suppliers data or engineering analysis. The factors used in this model take into account the variations in equipment operating environment, quality and device application conditions such as device temperature and electrical stress level. 4.2 CNET This method is defined in [7] and was developed by French National Centre Of Telecommunications. The data used comes from the analysis of breakdowns of the equipment used by the military and civil Administration, and other equipment and component manufacturers. This handbook forms a common base intended to make reliability predictions uniform in France. The factors used in this model take into account the variations in equipment operating environment, quality and device temperature. It should be noted that the method used in this paper is the 'simplified method' so called because it comes from a British Telecom translation of the important parts of the actual CNET handbook. 4.3 HRD4 This method is defined in [8] and was developed by British Telecom Materials and Components Centre for use by designers and users of electronic equipment so that there exists a common basis exists for system reliability prediction. The generic failure rates given in the handbook are estimates of the upper 60% confidence levels, based upon, wherever possible, data collected from the in service performance of the equipment installed in the UK inland telecommunications network. Where such data is not available for particular components, alternative sources or estimated values have been used and the status of the source indicated by a letter code. The factors used in this model take into account the variations in equipment operating environment, quality and device temperature. 4.4 MIL-217 Mil-Hdbk-217[9] was developed by the US Department of Defence with the assistance of the military departments, federal agencies, and industry for use by the electronic manufacturers supplying to the military. The handbook describes two methods, namely parts count and parts stress which are used to predict the reliability of electronic components, systems, or subsystems in different stages of the design. The failure rates given in the handbook have in 4
- 5. the main derived from test bed and accelerated life studies. The factors used in this model take into account the variations in equipment operating environment, and device quality. 4.5 SIEMENS This method is defined in [10] and was developed by Siemens AG for the use of Siemens and Siemens associates as a uniform basis for reliability prediction. The standard presented in the document is based on failure rates under specified conditions. The failure rates given were determined from application and testing experience taking external sources (e.g. Mil-Hdbk- 217) into consideration. Components are categorised into many different groups each of which has a slightly different reliability model. The factors used in this model take into account the variations in device operating temperature and electrical stress. 5. ANALYSIS OF PREDICTION METHODOLOGIES Six different board designs were selected from the IERI database. These designs were chosen such that they contained a wide range of component types and were from several different applications. Table 1 describes the environments and applications of the boards. Table 1:Brief description of the circuit boards BOARD ENVIRONMENT APPLICATION 1 Ground Mobile Radio System 2 Ground Mobile Radio System 3 Ground Benign Telephone Exchange 4 Ground Benign Telephone Exchange 5 Ground Mobile Command System 6 Ground Mobile Command System The analysis of two of the boards will be given in detail to show the methodology and to illustrate how the models can be used for parts provisioning purposes. 5.1 CIRCUIT BOARD ONE The first board design chosen from the IERI database contained 338 components with six different component types. The board came from a system used in a high quality radio application. Table 2 contains the predicted failure rate for each component type used on the board. Each failure rate is given in FITs and is the failure contribution for each component type on the board taking into consideration environment, component numbers etc. Table 2:Contribution to failure rate by each component type for circuit board one. DEVICE BELLCORE CNET HRD4 MIL-217 SIEMENS Ceramic Multilayer Capacitor 210 45 3 4 35 pn-Junction Diode 125 300 300 9 25 Bipolar Digital IC 936 390 168 50 60 Metal Oxide Resistor 1590 596 51 52 795 Discrete Bipolar Transistor 7812 20937 4000 1225 1250 Tantalum Electrolytic Capacitor 1350 3780 40 594 180 Total 12023 26048 4562 1934 2345 5
- 6. As can be seen from the totals row in Table 2 the reliability predicted for this board differs widely between the different models. The actual field behaviour of this board is summarised in Table 3 which shows the actual numbers of operating hours observed, the total number of failure in this time and the calculated failure rate with the upper and lower 95% 2 confidence limits. Table 3:Actual field behaviour for circuit board one. Number of failures 19 Total number of operating hours 4444696 FIT rate 4274 Upper 95% 2 confidence limit 6400 Lower 95% 2 confidence limit 2572 Deviation from observed value 25,000 21,774 20,000 15,000 10,000 7,749 5,000 288 0 -1,929 -2,340 -5,000 Bellcore CNET HRD4 MIL-217 Siemens Prediction Methodology Figure 1:Deviation of predicted reliability of board one from observed value using different models The differences from the various predicted values to the actual field figures are summarised in Figure 1. It can be seen that some of the models gave predictions that were optimistic (MIL-217 and Siemens) whereas others give pessimistic predictions. Table 4 shows the percentage contribution to failure of each component on the circuit board. It is found that for each prediction model that the most likely cause of failure was the bipolar transistor which has the largest percentage contribution to failure in each case. Table 4:Largest percentage contributions to failure for circuit board one. PREDICTION LARGEST NEXT LARGEST CONTRIBUTION MODEL CONTRIBUTION Bellcore Bipolar Transistor (65%) Metal Oxide Resistor (13%) CNET Bipolar Transistor (80%) Tantalum Electrolytic Capacitor (14%) HRD4 Bipolar Transistor (87%) pn-Junction Diode (6.5%) MIL-217 Bipolar Transistor (63%) Tantalum Electrolytic Capacitor (30%) Siemens Bipolar Transistor (53%) Metal Oxide Resistor (33%) 6
- 7. Analysis of the field information shows that the bipolar transistor was the main cause of failure of this circuit board in the field, and so spare provisioning using any of the available models would have proved accurate. 5.2 CIRCUIT BOARD TWO The second board chosen from the IERI database was slightly more complex board containing 149 components with eighteen different component types. The board comes from a system used in a telecommunication application. Table 5 contains the predicted failure rate in FITs for each component type used on the board. Table 5:Contribution to failure rate by each component type for circuit board two. DEVICE BELLCORE CNET HRD4 MIL-217 SIEMENS Transformer 3 9 7 6 5 Coil activated relay 770 605 440 1430 88 Aluminium electrolytic capacitor 210 22 120 16 120 Polyester capacitor 17 4 6 1 14 pn-Junction diode 230 149 345 152 230 Zener diode 16 63 87 94 350 LED 9 15 280 65 0 Bipolar digital IC (11-100 gates) 59 413 7 3 20 Bipolar linear IC (1-10 transistors) 14 57 13 3 500 Bipolar linear IC (11-100 transistors) 42 80 13 3 150 MOS digital IC (1-10 gates) 83 903 27 3 40 Rectangular connector 7 1 50 8 22 Varistor 6 0 10 0 10 Carbon film resistor 182 27 10 11 26 Wire-wound resistor 127 42 6 1 30 Metal film resistor 7 25 0.5 0.5 2 Rocker switch 5 44 30 1 20 Bipolar transistor 25 19 16 38 20 TOTAL 1812 2478 1467.5 1835.5 1647 As can be seen from the totals row the reliability predicted for this board also differs widely between the different methods. The actual field behaviour of this board is summarised in Table 6 and the difference between the predicted values and the observed field value is shown in Figure 2 Table 6:Actual field behaviour for circuit board two. Number of failures 5 Total number of operating hours 8.5x106 FIT rate 587 Upper 95% 2 confidence limit 1202 Lower 95% 2 confidence limit 190.6 7
- 8. Deviation from observed value 2,500 2,000 1,891 1,500 1,225 1,248 1,060 1,000 880 500 0 Bellcore CNET HRD4 MIL-217 Siemens Prediction Methodology Figure 2:Deviation of predicted reliability from observed values for different models Notice in this case all predictions are pessimistic. The percentage contribution to failure of each component on the circuit board is shown in Table 7. Table 7:Largest percentage contributions to failure for circuit board two. PREDICTION LARGEST CONTRIBUTION NEXT LARGEST MODEL CONTRIBUTION Bellcore Coil Activated Relay (42%) pn-Junction Diode (12%) CNET MOS Digital IC with 1-10 gates (36%) Coil Activated Relay (24%) HRD4 Coil Activated Relay (30%) pn-Junction Diode (23%) MIL-217 Coil Activated Relay (77%) pn-Junction Diode (8%) SIEMENS Bipolar Linear IC with 1-10 transistors (30%) Zener Diode (21%) Field observation showed all the failures on this board to be caused by coil activated relays and bipolar transistors with the relay causing one more failure than the transistor. If parts provisioning had been done according to two of the models, then the incorrect part would have been identified. The reliability of the rest of the boards used in this study were calculated using all the aforementioned models. Figure 3 shows the percentage deviation from the observed field failure rate for the six board designs selected from the IERI-CORD database. The predictions not only differ widely between the various models but they also differ greatly from the observed field failure rate. The models are not always consistent in the deviation from this observed field value, they can be optimistic in some cases while pessimistic in others. his suggests that there are some underlying factors that are causing divergence of the models. 8
- 9. Board type Board one Board two Board three Board four Board five Board six -200 -100 0 100 200 300 400 500 600 % deviation from field value BellCore CNET HRD4 MIL-217E Siemens Figure 3:Percentage deviation from the observed field value for six boards In order to investigate this apparent inconsistency in the various models it is necessary to look at each model carefully and analyse the way in which the predicted failure rate is influenced by the various parameters that influence system performance. This technique is known as sensitivity analysis and it is useful when examining a model for major dependencies. If the various prediction models are sensitive to similar parameters then the inconsistency shown above must come from the nature of the underlying data. If the models are sensitive to different parameters then this suggests that different models of failure were used when the models were derived. Hence this makes it impossible to use base failure rates from one model in another and also makes it imperative to only use in house data derived in the same manner as the chosen model when extending the coverage of the model to such things as custom components. This makes it much more difficult for companies performing predictions to use data gathered in house. 6. SENSITIVITY ANALYSIS OF PREDICTION METHODOLOGIES This is done by varying temperature, quality, stress, and environment in turn while keeping the others at typical or nominal values. The results are presented graphically and show percentage variation of predicted failure rate from the nominal value while a single parameter is varied within the limits that the models allow. The largest spread shows the parameter that has the greatest effect on the model's prediction. Care should be taken in some cases where the effect is accentuated by a highly non-linear variation in one of the parameters' factors. This is particularly true when the parameters are discrete, as in the case of environment, where selection of a particular environment can cause a large change in the associated factor. The degree of non-linearity can be demonstrated using graphs of normalised stress versus deviation where normalised stress is defined as the ratio between values of the range of available factors and the nominal factor value for the parameter under investigation. 9
- 10. 6.1 BELLCORE MODEL The percentage deviation in the board failure rate from the nominal with respect to different stress levels using the Bellcore methodology is shown in Figure 4. 200 Percentage deviation from nominal 150 90% 100 65ºC Low 50 0 GM 30ºC High -50 10% GB -100 Temperature Environment Electrical Stress Quality Model parameter Figure 4:Sensitivity of predicted value for the Bellcore methodology As can be seen Figure 4 the allowed variation in the electrical stress makes the largest difference to the calculated failure rate. 200 Percentage Deviation from nominal 150 100 50 0 Temperature (50) Electrical Stress (100) 0 0.5 1 1.5 2 Normalised Stress Figure 5:Variation in predicted value for the Bellcore methodology with changes in temperature and electrical stress. The non-linear nature of the effect of varying electrical stress and temperature are shown in Figure 5. This shows that the BELLCORE model is based upon an Arrhenius style acceleration formula which is reflected in this non-linearity. 6.2 CNET MODEL Figure 6 shows the percentage deviation in the board failure rates with respect to different stress levels using the CNET(simplified) methodology. As can be seen from Figure 6, the range in quality factor has the largest influence on the calculated failure rates. 10
- 11. 1,000 Percentage deviation from nominal Low 800 600 400 200 ML 70ºC 0 40ºC High SL (200) Temperature Environment Electrical Stress Quality Model parameter Figure 6:Sensitivity of predicted value for the CNET methodology The quality factor used in this model is based upon a set of discrete quality bands over which the factor is defined. This means that the quality factor is a stepwise non-linear function of the device quality. 6.3 HRD4 MODEL The percentage deviation in the board failure rates with respect to different stress levels using the HRD4 methodology is shown in Figure 7. 150 Percentage deviation from nominal 100 150ºC Low 50 0 GM <70ºC (50) High (100) GB Temperature Environment Electrical Stress Quality Model parameter Figure 7:Sensitivity of predicted value for the HRD4 methodology As can be seen from Figure 7, it is the allowed range of quality factors that makes the largest difference to the calculated failure rates. The quality factor used in this model is based upon a set of discrete quality bands over which the factor is defined. Again this means that the quality factor is a stepwise non-linear function of the device quality. 6.4 MIL-217 MODEL Figure 8 shows the percentage deviation in the board failure rates with respect to different stress levels using the MIL-217E methodology. The figure shows that it is the allowed 11
- 12. variation in environment factors that causes the largest difference in the calculated failure rates. 5000 Percentage deviation from nominal CL 4000 3000 2000 1000 Low 0 GB High -1000 Temperature Environment Electrical Stress Quality Model parameter Figure 8: Sensitivity of predicted value for the MIL-217 methodology However this effect is enhanced because of the large difference caused by the cannon launch (CL) environment as is apparent from Figure 9.. If the CL environment is omitted then the quality would become the predominant factor. 5000 Percentage deviation from nominal 4000 3000 2000 1000 0 -1000 GB GF MP NS NH ARW AIT AIA AUC AUB AUF MFF USL CL GMS GM NSB NU NUU AIC AIB AIF AUT AUA SF MFA ML Environment Figure 9:The effect of MIL-217 Environments on the predicted value The quality factor used in this model is based upon a set of discrete quality bands over which the factor is defined. This means that the quality factor is a stepwise non-linear function of the device quality. 6.5 SIEMENS MODEL The percentage deviation in the board failure rates with respect to different stress levels using the Siemens methodology is shown in Figure 10. As can be seen from the figure it is the range of temperature factor that causes the largest difference in the calculated failure rates. 12
- 13. The non-linear nature of the electrical stress and temperature effects are shown in Figure 11 This 5000 Percentage deviation from nominal 4000 130ºC 3000 2000 1000 100% 0 40ºC 10% -1000 Temperature Environment Electrical Stress Quality Model parameter Figure 10:Sensitivity of predicted value for the Siemens methodology means that the Siemens model is based upon an Arrhenius style acceleration formula which is reflected by this non-linearity. 5000 Percentage deviation from nominal 4000 3000 2000 1000 Electrical Stress 0 Temperature -1000 0 0.5 1 1.5 2 2.5 3 3.5 Normalised Stress Figure 11:Variation in predicted value for the Siemens methodology as temperature and electrical stress are varied. 7. CONCLUSIONS The results are summarised in Table 7.It is evident that some of the models are more sensitive to a factor that varies according to an Arrhenius model, such as temperature and electrical 13
- 14. stress, while others are more sensitive to the discrete factors used to model environment and quality. Table 7: Most sensitive parameter in each prediction model Prediction Model Greatest sensitivity Bellcore Electrical Stress CNET(Simplified) Quality HRD4 Quality MIL-217E Environment, Quality Siemens Temperature It is not surprising that direct comparisons of the models result in wide variations. Although the models are based upon the same criteria there is disagreement about the effects the different parameters have on the failure rate. Also by carefully examining the models, it was observed that although the quality levels are clearly defined within each procedure, it is extremely difficult to find a quality level description that is compatible across all models. In addition every organisation has developed their model according to the experience they have obtained in the field and have tailored the model to meet their specific needs. It is inadvisable to compare the models' prediction with field performance unless the systems used are manufactured according to the guidelines and procedures that are specified by the model designers. Under such circumstances the system manufacturers would argue that their models are suitable for reliability prediction. Care must also be taken when comparing the predicted reliability with that observed in the field as prediction models assume a constant hazard rate. It has been observed however that early life failures do occur in the field even after system burn-in [4] Even if the above considerations are taken into account, there is no guarantee that the field reliability will be the same as that predicted. This is due to the underlying reason that models are generally simple empirical approximations. Indeed it is postulated that because of this, their use should actively be discouraged. Moreover, they do not take into account many of the other critical factors such as vibrations, mechanical shock, etc. 14
- 15. 8. REFERENCES 1. C. T. Leonard and M. Pecht, "How failure prediction methodology affect electronic equipment design", Quality and Reliability Engineering International, Vol. 6, 1990.pp 243-249, 2. S. F. Morris, "Use and application of MIL-HDBK-217", Solid State Technology 1990. August, pp 65-69,. 3. D. S. Campbell and J. A. Hayes, "The organisation of a study of the field failure of electronic components", Quality and Reliability Engineering International, Vol. 3, , 1987, pp 251-258. 4. D. S. Campbell, J. A. Hayes, J. A. Jones & A. P. Schwarzenberger, "Reliability Behaviour of Electronic Components As a Function of Time", Quality and Reliability Engineering International, Vol. 8, , 1992, pp 161- 166. 5 J Møltoft, "Reliability Engineering Based on Field Information - The Way Ahead", Quality and Reliability Engineering International, Vol. 10, 1994, pp 399-409. 6. Bellcore Technical Reference TR-TSY-000332, "Reliability Prediction Procedure for Electronic Equipment", Issue 2, July 1988. 7. Centre National D'Etudes des Telecommunications, "Recueil de Donnees de Fiabilite du CNET",1983 edition. 8. British Telecom, "Handbook of Reliability Data for Components used in Telecommunications Systems", Issue 4, January 1987. 9. US Mil-Hdbk-217,"Reliability Prediction of Electronic Equipment", Version E, October 1986. 10. Siemens AG, SN29500, "Reliability and Quality Specifications Failure Rates of Components", Siemens Technical Liaison and Standardisation 1986. 15
- 16. AUTHORS Mr Jeff. A. Jones Jeff Jones has been a research fellow at Loughborough University since it the reliability work began in 1984 . He has a first degree in electronic engineering and physics and has since obtained a masters degree by research (MPhil) entitled "The Implementation of a Field Reliability Database". He is currently the principal British expert and deputy convenor on IEC/TC56 WG 7 Component Reliability and alternate British expert on CECC SC/56x, Dependability. He is an active member within the UK dependability community He is a chartered physicist, a chartered engineer and a member of the IEEE. Mr J.A.Jones , IERI, Department of Electronic and Electrical Engineering, Loughborough University, Loughborough, Leics., LE11 3TU, United Kingdom Tel: +44 1509 222897 Email : [email protected] Mr Joe. A. Hayes Mr Joe Hayes is a lecturer within the department of Electronic and Electrical Engineering at Loughborough University. He has a degree in physics from Newcastle University and has a masters degree in semiconductor physics. He has been involved with the reliability work at Loughborough since it started and has played a major role in the development of the International Electronics Reliability Institute.. He is a chartered physicist and a member of the institute of physics Mr J.A.Hayes, Department of Electronic and Electrical Engineering, Loughborough University, Loughborough, Leics., LE11 3TU, United Kingdom Tel: +44 1509 2222851 Email : [email protected] 16