Classroom DOE

5-1
Exercises 2
Catapult Experiment
Number of
Factors

2k-p
Number of Size of
Levels Fraction
1
Catapult Reference Drawing

This sketch
shows the
Cup location of pin
setting “1” for
Hook Pin
the various
factors.
Any markings on
Stop Angle the catapult to
the contrary are
Pull Back Angle to be ignored.

2
The Challenge

Use the Strategy for Experimentation to complete the following:

Conduct a 5 factor, 2 level half fractional factorial design with 3

replicates, 5 center points, and two blocks (operator) to optimize the
catapult settings to hit a target within +/- 5”

Factors: A: Pull back angle 160 and 180

B: Pin 2 and 4
C: Hook 1 and 3
D: Stop Angle 2 and 4
E: Cup position 4 and 6
Maximum number of resources 65 launches prior to validation
Perform the experiment and obtain the prediction equations for both:
•Mean (maximum distance)
•Variation (minimum variation)
3
The Analysis

• Use log(s) or log(s2) for variation analysis

• For each of the optimum settings, maximum mean and minimum
variation (note: they may be the same):
– Complete 9 shots at each target (predicted) value for
confirmation
– Compute ybar, s, Z, and DPMO using the predicted value as
the target and using the +/- 5” tolerance
– Be ready to report your results
• Some comments
– Have you done MSA?
– Are the operators trained the same?
– Have you held other factors (e.g. ball placement) constant?

4
The Solution

Conduct a 5 factor, 2 level half fractional factorial design with 3

replicates, 5 center points, and two blocks (operator) to optimize the
catapult settings to hit a target within +/- 5”

Factor Low setting High setting

A 160 180
B 2 4
C 1 3
D 2 4
E 4 6

5
25-1 Design

Stat  DOE  Factorial  Create Factorial Designs

6
Display The Available Designs

• Select Display Available Designs to view the designs available and

their resolution

• A ½ fraction with 16 runs meets our design criteria

7
25-1 Design: Setup

• Select ½ fraction which will meet

our 16 experimental run criteria

8
25-1 Design: Setup

• Enter ‘Name’ for each

factor (optional)
• Select ‘Type’ (Numeric or Text)
• Enter the ‘Low’ and ‘High’
settings for each factor

9
Graph The Result:

• Graphical DOE analysis consists of a number of visual displays of

the effects
• We will start with the most common, the effects plots
• In MINITAB, Stat  DOE  Factorial  Factorial Plots

Select the type

of factorial plot Setup the
desired interaction plot
similarly Let’s interpret each one….
10
Main Effects Result:

Main Effects Plot (data means) for Distance

Pull back angle Pin Hook
Point Type
Corner
120 Center

110

100
Mean of Distance

90

80 This point represents

160 170
Stop Angle
180 2 3
Cup position
4 1 2
the average
3
of all runs
with Factor Pin at the
120
high level (+1)
110

100

90

80
From
2 this
3 plot,
4 all4 main5 effects
6 appear to have the the
same magnitude (steeper slope).
11
Interaction Effects Result:

Interaction Plot (data means) for Distance

2 3 4 1 2 3 2 3 4 4 5 6
160
Pull
back
120 angle Point Type
Pull back angle 160 Corner
80 170 Center
160 180 Corner
Pin Point Type
2 Corner
120 3 Center
Pin
4 Corner
80
160
Hook Point Type
1 Corner
120 2 Center
Hook
3 Corner
• Non-parallel lines 80
160
indicate an Stop
Angle Point Type
120
interaction effect Stop Angle
2
3
Corner
Center
80 4 Corner

• The closer the lines

are to being parallel, Of the 2-way Cup position
the less likely an interactions, all are the
interaction exists same 12
 Build model

• Stat  DOE  Factorial 

Analyze Factorial Design
• Select ‘Distance’ as
the response

• Since the experiment is

unreplicated, we will look at
both the Normal Probability
Plot and the Pareto Chart of
Effects first
13
Build model

• Select ‘Terms’
• Since this is a Resolution V
design, terms higher than 2nd
order are aliased with main
effects and 2-way interactions

• Thus, we need only to include

terms up through order two (2)
in the model.

14
Analyze A 25-1 Factorial Design:

• Select Pareto and Normal Effect Plot

to evaluate term effects
• Enter 0.05 for alpha (to give each
factor a chance to be part of the
model

15
25-1 Factorial Design Output:
Sales Example

The coefficients (in coded units) for the

constant, factors and interactions which
are used to build a prediction model

16
 Pareto Chart Of Effects

• Any factor or interaction that crosses the red line, is a significant factor
(Alpha @ 0.05) or p < 0.05
• We use 0.05 as this is an approximate method and we wish to err on the
conservative side
• Eliminate highest order terms first
Pareto Chart of the Standardized Effects
(response is Distance, Alpha = .05)

2.02
If a term is not B
F actor N ame
A P ull back angle
significant, we A B
C
P in
H ook
C
may remove it D
D
E
S top A ngle
C up position

E
and re-run CE

the model
Term

AD
AC
BE
BC
AE
CD
BD
DE

0 5 10 15 20 25 30 35
Standardized Effect 17
Normal Probability Plot For The Effects:

Normal Probability Plot of the Standardized Effects

(response is Distance, Alpha = .05)

99
Effect Type
Not Significant
95 B Significant

F actor N ame
90 A A P ull back angle
C B P in
80
C H ook
D
70 E D S top A ngle
E C up position
Percent

60 CE
AC
50 BE
40
30
From both the Pareto Chart and
20
the Normal Probability Plot, it
10 BC
appears that A,B, C, D,E and CE
5 AD AC, BE are significant

1
0 10 20 30 40
Standardized Effect

18
Check Residual by Plots
Looks fine..
Residual Plots for Distance
Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
99 10

90 5

Residual
Percent

0
50
-5
10
-10
1
-10 -5 0 5 10 50 100 150 200
Residual Fitted Value

Histogram of the Residuals Residuals Versus the Order of the Data

16 10

5
12
Frequency

Residual

0
8
-5
4
-10
0
-10 -5 0 5 1 5 10 15 20 25 30 35 40 45 50 55
Residual Observation Order
19
 Should We Reduce The Model ?

• Generally, our analysis of a Fractional Factorial design results in stating

that a subset of the factors are carried to a subsequent experiment
• Our conclusions thus far tend to indicate that A,B, C, D,E and CE AC,
BE are significant
• At this point, removing terms from the model can be done, however, we
need to be clear on our motivations
• The Pareto Chart and Normal Probability Plot tend to indicate a drop-
off in effect size after the E term
• If we were to begin removing (pooling) the smallest effects to create an
error term, a number of the smaller effects become significant
• Since we’re doing screening with this type of experiment, we’ve kept
the default alpha = 0.05 to minimize the chance of screening out an
important effect

20
Evaluate Pareto Chart To Reduce Model:
Sales Example

Pareto Chart of the Standardized Effects

(response is Distance, Alpha = .05)

2.02
F actor N ame
B A P ull back angle
A B P in
C H ook
C D S top A ngle
D E C up position

E
CE
Term

AD
AC
BE
BC
AE
CD
BD Start
Startby
byremoving
removingone oneinsignificant
insignificant
DE (higher
(higherorder,
order,ififpossible)
possible)term
term
0 5 10 15 20 25 30 35
Standardized Effect
21
 25-1 Factorial Model: Sales Example

Stat DOE  Factorial  Analyze Factorial Design

Select the insignificant

term ‘DE’, then select the
‘<‘ sign to remove that
insignificant interaction
from the selected terms
section
22
Evaluate Pareto Chart To Reduce Model:
Sales Example

Pareto Chart of the Standardized Effects

(response is Distance, Alpha = .05)

2.02
F actor N ame
B A P ull back angle
A B P in
C H ook
C D S top A ngle
D E C up position

E
CE
Term

AD
AC
BE
BC
AE
CD
BD Start
Startby
byremoving
removingone oneinsignificant
insignificant
DE (higher
(higherorder,
order,ififpossible)
possible)term
term
0 5 10 15 20 25 30 35
Standardized Effect
23
 Evaluate Pareto Chart To Reduce Model:
Sales Example

Pareto Chart of the Standardized Effects

(response is Distance, Alpha = .05)

2.01
F actor N ame
B A P ull back angle
B P in
A C H ook
D S top A ngle
C E C up position

D
E
Term

CE
AD
AC
BE
Reduced
ReducedModel
Modelwith
withall
allsignificant
significant
BC
terms
termsand
andinteractions
interactions
0 5 10 15 20 25 30 35
Standardized Effect
24
Final Model Results: Sales Example

• The final reduced model has

been obtained
• The p-values
(except for the main factor ‘C’)
are < 0.05
• R2(adj) is excellent
• We will now check residuals

Curvature is significant; linear modeling is

normally not applicable to this process in the
inference space. We could not see the
curvature until we added Center Points –
Normally we would not proceed with the
current model. And try fitting Response
Surface Models, in our case study we still
keep using the linear model.
25
.
Check Residuals

Residual Plots for Distance

Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
99 10

90 5

Residual
Percent

0
50
-5
10
-10
1
-10 -5 0 5 10 50 100 150 200
Residual Fitted Value

Histogram of the Residuals Residuals Versus the Order of the Data

16 10

5
12
Frequency

Residual

0
8

4
-5
• Residuals appear to be randomly
-10
0
dispersed about the center
-10 -5 0 5 1 5 10 15 20 25 30 35 40 45 50 55
Residual Observation Order
• Residuals appear to be relatively
normally distributed
• No transformation is necessary
no Log’s
• The final model! 26
 Create The Prediction Equation For
The Final Model (coded)
Create the prediction equation using the coefficient column
•you ignore the Block coefficients
and center points create an
equation from the coefficients of
the constant and the terms that
are found in the final reduced
model

Prediction Equation for the coded Final Model:

Distance=103.938+20.604*Pull back angle+22.479*Pin+14.187*Hook+13.646*Stop
Angle+10.354*Cup position+1.854*Pull back angle*Hook-2.354*Pull back angle*Stop
Angle-1.438*Pin*Hook+1.813*Pin*Cup position+2.354*Hook*Cup position
27
 Create The Prediction Equation For
The Final Model (uncoded)
Create the prediction equation using the coefficient column

•you ignore the Block coefficients

and cemter points create an
equation from the coefficients of
the constant and the terms that
are found in the final reduced
model

Prediction Equation for the uncoded Final Model:

Distance=-449.771-2.390*Pull back angle+16.292*Pin-24.792*Hook+53.667*Stop
Angle+0.208*Cup position+0.185*Pull back angle*Hook-0.235417*Pull back
angle*Stop Angle-1.438*Pin*Hook+1.813*Pin*Cup position+2.354*Hook*Cup
position 28
 Create Sort for summarized statistics of mean and
Standard deviation
Perform the experiment and obtain the prediction equations for both:
Mean (maximum distance) ,Variation (minimum variation)
- Find by sorting and using summarized statistics of the replicated runs
Data Sort

29
Create Index for summarized statistics of mean and
Standard deviation
- Delete Center points for summarized statistics of mean and Standard deviation
calculation

Data Subset Worksheet

'CenterPt'=1 And
'Blocks'=1

30
 Calculate # Runs:

(25-1 =) 16 + 5 Center points + (25-1 * ½ (Half fractional Design =) 8 corner points = 29 )

* 2 Blocks = 58 runs
As we have 8 corner points we have a replication every 8 sorted rows
Blocks Pull back angle
Pin Hook Stop AngleCup position
Distance
1 180 2 1 2 4 63 1
1 160 4 1 2 4 64 2
1 180 2 3 2 6 116 3
1 160 4 3 2 6 113 4
1 180 2 1 4 6 96 5
1 160 4 1 4 6 113 6
1 180 2 3 4 4 114 7
1 160 4 3 4 4 109 8
1 180 2 1 2 4 53 1
1 160 4 1 2 4 64 2
1 180 2 3 2 6 112 3
1 160 4 3 2 6 109 4
1 180 2 1 4 6 98 5
1 160 4 1 4 6 115 6
1 180 2 3 4 4 112 7
1 160 4 3 4 4 112 8
1 180 2 1 2 4 58 1
1 160 4 1 2 4 62 2
1 180 2 3 2 6 113 3
1 160 4 3 2 6 107 4
1 180 2 1 4 6 100 5
1 160 4 1 4 6 112 6
1 180 2 3 4 4 112 7
1 160 4 3 4 4 112 8
1 170 3 2 3 5 122 1
1 170 3 2 3 5 121 2
1 170 3 2 3 5 120 3
1 170 3 2 3 5 119 4

31
Create Index for summarized statistics of mean and
Standard deviation

- Create Index for summarized statistics of mean and Standard deviation

calculation -> Use Subset Worksheet
•Calc  Make Patterned data  Simple set of numbers

32
Create summarized statistics of mean and standard
deviation

- Create summarized statistics of mean and Standard deviation calculation

•Stat  Basic Statistics  Store Descriptive Statistics

33
Find Mean (maximum distance) and Variation
(minimum variation) for Equation evaluation

Max Mean = 113.667 Index Row 3

Min Std Var. 1.15470 Index Row 7

34
Find Predictor Variables for Max mean and Min
Std Dev by Index Row

Max mean Min Std Dev

Index 3 Index 7
Pull back angle 180 180
Pin 2 2
Hook 3 3
Stop Angle 2 4
Cup position 6 4

35
 Predicting Mean and Std Dev Using DOE Equation

Prediction Equation (coded) for the Final Model:

Distance=-449.771-2.390*Pull back angle+16.292*Pin-24.792*Hook+53.667*Stop Angle+0.208*Cup
position+0.185*Pull back angle*Hook-0.235417*Pull back angle*Stop Angle-1.438*Pin*Hook+1.813*Pin*Cup
position+2.354*Hook*Cup position

From summarized statistics

Max mean Min Std Dev
Index 3 Index 7
Pull back angle 180 180
Pin 2 2
Hook 3 3
Stop Angle 2 4
Cup position 6 4

36
 Predicting Mean and Std Dev Using DOE Equation
manually

Substitute the values into the prediction equation:

Prediction Equation for the max mean:

Distance=-449.771-2.390*180+16.292*2-24.792*3+53.667*2+0.208*6+
0.185*180*3-0.235417*180*2-1.438*2*3+1.813*2*6+2.354*3*6

Distance = 119.145

Prediction Equation for the min Std Dev:

Distance=-449.771-2.390*180+16.292*2-24.792*3+53.667*4+0.208*4+
0.185*180*3-0.235417*180*4-1.438*2*3+1.813*2*4+2.354*3*4

Distance = 119.936

37
 Creating Confidence Intervals And Prediction
Intervals Using MINITAB

Stat  DOE  Factorial Design  Analyze Factorial Design

• Enter the optimal settings for the individual
factors in uncoded values in order (180 2 3 2
6) for mean (max) and (180 2 3 2 6) for std dev
(min).

• Make sure ‘Terms’ has the appropriate

terms for the final model
• Select ‘Prediction’

38
 Creating Confidence Intervals And Prediction
Intervals Using MINITAB

Prediction for the max mean with CI :

Prediction for the min Std Dev with CI :

39
 Validate Model

Complete 9 shots at each target (predicted) value for confirmation

Compute ybar, s, Z, and DPMO using the predicted value as the target and
using the +/- 5” tolerance

predicted value mean (max) =119.145

-using the +/- 5” tolerance = min 114.145 max 124.145

predicted value Std Dev (max) =119.936

-using the +/- 5” tolerance = min 114.936 max 124.936

9 Shots are mad by normal

random distribute in this
example, die to lack of
measures
40
 Validate Model

Six Sigma  Process report

Report 1: Executive Summary

ybar, s, Z, and DPMO

Process Performance Process Demographics
LS L USL

D ate:
Reported by :
P roject:
D epartment:
P rocess:
C haracteristic:
U nits:
U pper S pec: 124.145
Low er S pec: 114.145
116 120 124 128
N ominal:
O pportunity :
A ctual (LT) Report 2: Process Capability for Verify Distance

I and MR Chart Capability Indices

1000000
UCL=131.98
Process Benchmarks 130 ST LT
100000
A ctual (LT) P otential (S T) _
M ean 119.145 121.355
10000
X=121.35 S tDev * 3.670
120
1000 Z.U S L * 0.76
S igma 0.68 2.18
(Z.Bench) Z.LS L * 1.96
100
110 LCL=110.73 Z.Bench 2.18 0.68
10 1 2 3 4 5 6 7 8 9 Z.S hift 1.50 1.50
1 DPMO 248316.7 14636.3 UCL=13.06 P .U S L * 0.223568
P .LS L * 0.0247489
1 2 3 4 5 6 7 8 9 10
P .Total 0.0146363 0.248317
Yield 98.54 75.17
5 __
MR=4.00 DPMO 14636.3 248316.7
Cp * *
0 LCL=0 C pk * *
1 2 3 4 5 6 7 8 9 C C pk * *
Observation
Pp * 0.45
P pk * 0.25
Potential (ST) Capability Actual (LT) Capability

P rocess Tolerance P rocess Tolerance

110.343 132.366

114.145 124.145 114.145 124.145

Data S ource:
S pecifications S pecifications Time S pan:
Data Trace:

41