Design of Experiments - Part 1

Tutorial 9
 Terminology
• Factors: Independent Variable.
• Levels: Groups of values or classifications for each factor.
• Treatment: A factor at specified level or specific combination of factor levels in an experiment.
• Response: Dependent Variable.
• Main Effect: The change in response caused by the change in the level of each factor independently. It
refers to the primary factors in the study.
• Interaction: The change in response caused by the change in the level of multiple factors simultaneously.
• Null hypothesis (H0): There’s no effect in the population.
• Alternative hypothesis (Ha or H1): There’s an effect in the population.
• ANOVA (Analysis of Variance): Statistical method used to determine if there are any statistically
significant differences between the means of three or more independent groups.
• Total Sum of Squares: Total variation in response that can be attributed to various factors.
• Degrees of Freedom: Number of values or information that are free to vary.
• Mean Square of Factors: Represents the variation between the sample means.
• Mean Square Error: Represents the variation within the samples.
• F-Value: The ratio of the mean square between groups to the mean square within groups.
(> F Critical Value, statistically significant and null hypothesis is rejected)
• P-Value: The probability that the observed difference between the means of two or more groups is due to
chance. (< α = 0.05, statistically significant and null hypothesis is rejected)
• Residuals: Represent the difference between each individual observation and the mean of the group that the
observation came from.
• R2: Represents the percentage of variation in a response variable that is explained by its relationship with
one or more predictor variables.
Guidelines for Designing Experiments
 Main Effect and Interaction
• Can we predict the 4th cell mean from
the means of other 3 cell means? If
yes, then the effect of one is not
affected by the other, and thus no
interaction.

• Compare the mean differences in any

individual row (or column) with the
mean differences in other rows or
columns. If the size and the direction
of the differences in one row (or
column) are the same as the
corresponding differences in other
rows (or columns), then no
interaction.
 The Statistical Model for ANOVA
The observations from a two-factor factorial experiment may be
described by the model:

• where μ is the overall mean effect,

• τi is the effect of the ith level of factorA,

• βj is the effect of the jth level of factor B,

• (τβ)ij is the effect of the interaction between Aand B, and

εijk is an NID(0, σ2) random error component.
ANOVA Rules
 Hardness Test Example
Determine whether 4 different tips produce different (mean) hardness readings on a Rockwell
hardness tester. The treatment factor is the design of the tip for the machine that determines
the hardness of metal. The tip is one component of the testing machine.
One-Way ANOVA Calculations
Tip Hardness Values yi
1 9.3, 9.4, 9.6, 10 38.3
2 9.4, 9.3, 9.8, 9.9 38.4
3 9.2, 9.4, 9.5, 9.7 37.8
4 9.7, 9.6, 10, 10.2 39.5
yj 154 = y

= 1483.54 (sum of squared observations or hardness values in this case) – ((1542) / (4 x 4)) = 1.29

Factor A (tip) = ((38.32) + (38.42) + (37.82) + (39.52) / 4) – ((1542) / (4 x 4)) = 0.385

= 1.29 − 0.385 = 0.905

 One-Way ANOVA Calculations
Degrees of Freedom Mean Squares (MS)

Tip DF: a − 1 = 4 − 1 = 3 = 0.385 / 3 ​= 0.12833

Error DF: Total DF – Tip DF = 15 – 3 = 12

= 0.905 / 12 ​= 0.07542
Total DF = an - 1 = 4(4) – 1 = 15

F-Value

= 0.12833 / 0.754 ​= 1.7 < 3.49,

Null hypothesis (H0) is accepted
 One-Way ANOVA Results
Assigning the tips to a random metal specimen for each test and
analyzing the data using one-way ANOVA, we observe that the
variation due to tip is 0.385. However, F-Value is 1.7, indicating that
there is no significant difference among these tips in this case.

Moreover, the R2 value is 29.84%, which means that the tip

contributes only to 29.84% of the total variation in the hardness
values. Further tests and consideration of additional factors are
required to ensure the accuracy of the results.
 Steel Alloy Panels Example
A vehicle manufacturer wishes to test the ability of three types of steel-alloy panels to resist
corrosion when three different paint types are applied. Three panels with differing steel-alloy
composition are coated with three types of paint. The following coded data represent the ability
of the painted panels to resist weathering.

Use a two-way ANOVA procedure to determine whether any difference in the ability of the
panels to resist corrosion may be assigned to either the type of paint or the steel-alloy
composition of the panels.
 Two-Way ANOVA Calculations
Paint Type Steel Alloy 1 Steel Alloy 2 Steel Alloy 3 yi
1 40 51 56 147

2 54 55 50 159

3 47 56 50 153

yj 141 162 156 459 = y

= 23607 (sum of squared observations or resistance to corrosion in this case) – ((4592) / (3 x 3 x 1))
= 198

Factor A (Paint Type ) = ((1472) + (1592) + (1532) / 3(1)) – ((4592) / (3 x 3 x 1)) = 24

Factor B (Steel Alloy) = ((1412) + (1602) + (1582) / 3(1)) – ((1432) / (3 x 3 x 1)) = 78

= 198 − 78 – 24 = 96
 Two-Way ANOVA Calculations
Degrees of Freedom Mean Squares (MS)

Paint Type DF: a − 1 = 3 − 1 = 2 = 24 / 2 ​= 12

Steel Alloy DF: b – 1 = 3 – 1 = 2 = 78 / 2 ​= 39

Error DF: Total DF – (Paint Type DF + Steel
= 96 / 4 ​= 24
Alloy DF) = 8 – (2+2) = 4
Total DF: abn – 1 = 3(3)(1) – 1 = 8

F-Value
= 12 / 24 ​= 0.5 < 6.94,
Null hypothesis (H0) is accepted
= 39 / 24 ​= 1.625 < 6.94,
Null hypothesis (H0) is accepted
 Factorial Experiments
• When there are several factors of interest in an experiment, a factorial design should be used.

• In such designs, factors are varied together.

• Specifically, by a factorial experiment, we mean that in each complete trial or replicate of the
experiment, all possible combinations of the levels of the factors are investigated.

• Thus, if there are two factors Aand B with a levels of factor Aand b levels of factor B, then each
replicate contains all ab possible combinations.
 Roadholding Example
A motor company wishes to check the influences of tyre type and shock absorber settings on the
roadholding of one of its cars. Two types of tyre are selected from the tyre manufacturer who normally
provides tyres for the company’s new vehicles. A shock absorber with three possible settings is
chosen from a range of shock absorbers deemed to be suitable for the car. An experiment is
conducted by conducting roadholding tests using each tyre type and shock absorber setting. The
(coded) data resulting from the experiment are given below:
Factorial Design ANOVA Calculations
Tyre B1 B2 B3 yi
A1 5, 6,8 19 8, 5, 3 16 6, 9, 12 27 62

A2 9, 7, 7 23 10, 9, 8 27 12, 10, 9 31 81

yj 42 43 58 143 = y

= 1233 (sum of squared observations or roadholding values in this case) – ((1432) / (2 x 3 x 3)) = 96.944

Factor A (Tyre) = ((622) + (812) / 3(3)) – ((1432) / (2 x 3 x 3)) = 20.056

Factor B (Shock Absorber) = ((422) + (432) + (582) / 3(3)) – ((1432) / (2 x 3 x 3)) = 26.778

Interaction = ((192) + (162) + (272) + (232) + (272) + (312) / 3) – ((1432) / (2 x 3 x 3)) –

20.056 – 26.778 = 5.444
= 96.944 − 20.056 – 26.778 – 5.444 = 44.666
 Factorial Design ANOVA Calculations
Degrees of Freedom Mean Squares (MS)

= 20.056 / 1 ​= 20.056
Tyre DF: a − 1 = 2 − 1 = 1

Shock Absorber DF = b – 1 = 3 – 1 = 2 = 26.778 / 2 ​= 13.389

Interaction DF = (a-1)(b-1) = 2(1) = 2 = 5.444 / (2)(1) ​= 2.722

Error DF: Total DF – (Tyre DF + Shock Absorber DF = 44.666 / 12 ​= 3.722

+ Interaction DF) = 17 – 5 = 12
Total DF = abn – 1 = 2(3)(3) – 1 = 17
F-Value
= 20.056 / 3.722 ​= 5.39 > 4.75,
Null hypothesis (H0) is rejected
= 13.389 / 3.722 ​= 3.6 < 3.89,
Null hypothesis (H0) is accepted
= 2.722 / 3.722 ​= 0.731 < 3.89,
Null hypothesis (H0) is accepted
 Minitab: One-Way ANOVA

1. Click on Stat 2. Select the factor C2 Tip. Choose

and ANOVA, C3 Hardness to be under
then click on responses. Make sure that the
One-Way. response data are in one column
for all factor levels. Then click Ok.
You could choose the graphs and
values to be stored (the same as
what has been done in factorial
design).
Obs Tip Hardness
1 1 9.3
2 1 9.4
3 1 9.6
Data Used: 4 1 10
5 2 9.4
6 2 9.3
7 2 9.8
8 2 9.9
9 3 9.2
10 3 9.4
11 3 9.5
12 3 9.7
13 4 9.7
14 4 9.6
15 4 10
16 4 10.2
Resulting One-Way ANOVA
 Minitab: Two-Way ANOVA

1. Click on Stat 2. Select the factors C2 Tip and C4

and ANOVA, Specimen. Choose C3 Hardness
then click on to be under responses. Then click
Fit General Ok. You could choose the graphs
Linear Model. and values to be stored (the same
as what has been done in factorial
design).

Obs Tip Hardness Specimen

1 1 9.3 1
2 1 9.4 2
3 1 9.6 3
4 1 10 4
Data Used: 5 2 9.4 1
6 2 9.3 2
7 2 9.8 3
8 2 9.9 4
9 3 9.2 1
10 3 9.4 2
11 3 9.5 3
12 3 9.7 4
13 4 9.7 1
14 4 9.6 2
15 4 10 3
16 4 10.2 4
Resulting Two-Way ANOVA
 Minitab: Factorial Design
1. Click on Stat 2. Select the Factors C1
and DOE, then Application Method and C2 Primer
click on Factorial Type. Choose the General Full
and Define Factorial Design and click Ok.
Custom
Factorial Design.
Application Primer Type Adhesion
Method Force
Dipping 1 4.0
Dipping 1 4.5
Dipping 1 4.3
3. Click on Stat and Spraying 1 5.4
Data Used:
DOE, then click Spraying 1 4.9
Factorial and Spraying 1 5.6
Dipping 2 5.6
Analyze Factorial
Dipping 2 4.9
Design. Select C3 Dipping 2 5.4
Adhesion Force Spraying 2 5.8
under responses. Spraying 2 6.1
Under Storage, click Spraying 2 6.3
Dipping 3 3.8
Residuals then click Dipping 3 3.7
Ok twice. Also, you Dipping 3 4.0
could choose the Spraying 3 5.5
graphs required by Spraying 3 5.0
Spraying 3 5.0
selecting Graphs.
Resulting Factorial Design
 Minitab: Residuals versus Application Method
1. Click on 2. Click on With Groups under
Graph, then One Y, then click Ok.
click on
Individual
Value Plot.

3. Select C8 RESI1 Resulting Graph:

(Residuals) under
the graph variables
and C1 Application
Method under
categorical
variables and click
Ok. You could also
click Scale and
Reference Lines to
show reference
lines at Y = 0.
 Minitab: Main Effects and Interaction Plots

1. Click on Stat 2. Select the factors C1

and DOE, then Application Method and C2 Primer
click on Factorial Type. Choose C3 Adhesion Force
and Factorial to be the response and Only
Plots. model terms beside terms to
display. Then click on Graphs.
Application Primer Type Adhesion
Method Force
Dipping 1 4.0
Dipping 1 4.5
Dipping 1 4.3
3. Tick beside Data Used: Spraying 1 5.4
Main effects Spraying 1 4.9
Spraying 1 5.6
plot, Interaction Dipping 2 5.6
plot ,and Display Dipping 2 4.9
Dipping 2 5.4
full matrix. Then Spraying 2 5.8
click Ok twice. Spraying 2 6.1
Spraying 2 6.3
Dipping 3 3.8
Dipping 3 3.7
Dipping 3 4.0
Spraying 3 5.5
Spraying 3 5.0
Spraying 3 5.0
 Minitab: Main Effects and Interaction Plots – Another Way
1. Click on Stat 2. For Main Effects Plot, select the
and ANOVA, factors C1 Application Method
then click on and C2 Primer Type. Choose C3
Main Effects Adhesion Force to be under
Plot. The same responses. Then click Ok.
for Interaction
Plot. Application Primer Type Adhesion
Method Force
Dipping 1 4.0
Dipping 1 4.5
Dipping 1 4.3
3. For Interaction Spraying 1 5.4
Data Used:
Plot, select the Spraying 1 4.9
factors C1 Spraying 1 5.6
Dipping 2 5.6
Application Method Dipping 2 4.9
and C2 Primer Type. Dipping 2 5.4
Choose C3 Adhesion Spraying 2 5.8
Force to be under Spraying 2 6.1
Spraying 2 6.3
responses and tick
Dipping 3 3.8
Display full Dipping 3 3.7
interaction plot Dipping 3 4.0
matrix. Then click Spraying 3 5.5
Ok. Spraying 3 5.0
Spraying 3 5.0
Resulting Main Effects and Interaction Plots