Measurement Systems Analysis

Table Of Contents
Measurement Systems Analysis............................................................................................................................................ 1
Overview........................................................................................................................................................................... 1
Gage Run Chart ............................................................................................................................................................... 3
Gage Linearity and Bias Study ......................................................................................................................................... 6
Gage R&R Study (Crossed) ............................................................................................................................................. 8
Gage R&R Study (Nested) ............................................................................................................................................. 17
Attribute Agreement Analysis ......................................................................................................................................... 21
Attribute Gage Study (Analytic Method) ......................................................................................................................... 27
Index .................................................................................................................................................................................... 31

2003 Minitab Inc.

Measurement Systems Analysis

Overview
Measurement Systems Analysis Overview
Minitab offers several commands to help you determine how much of your process variation arises from variation in your
measurement system.

Gage Run Chart, Gage R&R (Crossed), Gage R&R (Nested), and Attribute Gage Study (Analytic Method) examine
measurement system precision.

Gage Linearity and Bias Study examines gage linearity and accuracy.

Attribute Agreement Analysis helps assess the agreement of subjective ratings or classifications given by multiple
appraisers for both nominal and ordinal data.

Note

Attribute Agreement Analysis was previously called Attribute Gage R&R Study in Minitab Release 13. Attribute
Agreement Analysis, a technique to assess appraisers' agreement, is different from Attribute Gage Study
(Analytic Method), a method to examine the bias and repeatability of an attribute measurement system.

Any time you measure the results of a process you will see variation. This variation comes from two sources: differences
between parts made by the process and imperfections in taking measurements. Thus, measuring the same part
repeatedly does not result in identical measurements.
Statistical Process Control (SPC) is concerned with identifying sources of part-to-part variation, and reducing that variation
as much as possible to get a more consistent product. But before you conduct any SPC analyses, you may want to check
that the variation you observe is not mostly due to errors in your measurement system.

Measurement system error

Measurement system errors can be classified into two categories: accuracy and precision.

Accuracy describes the difference between the measurement and the part's actual value.

Precision describes the variation you see when you measure the same part repeatedly with the same device.

Within any measurement system, you can have one or both of these problems. For example, you can have a device that
measures parts precisely (little variation in the measurements) but not accurately. You can also have a device that is
accurate (the average of the measurements is very close to the accurate value), but not precise, that is, the
measurements have large variance. You can also have a device that is neither accurate nor precise.

Accuracy
The accuracy of a measurement system is usually broken into three components:

Linearity a measure of how the size of the part affects the accuracy of the measurement system. It is the difference
in the observed accuracy values through the expected range of measurements.

Bias a measure of the bias in the measurement system. It is the difference between the observed average
measurement and a master value.

Stability a measure of how accurately the system performs over time. It is the total variation obtained with a
particular device, on the same part, when measuring a single characteristic over time.

To examine your measurement system's accuracy, see Gage Linearity and Bias Study.

Precision
Precision, or measurement variation, can be broken down into two components:

Repeatability the variation due to the measuring device. It is the variation observed when the same operator
measures the same part repeatedly with the same device.

Reproducibility the variation due to the measurement system. It is the variation observed when different operators
measure the same parts using the same device.

2003 Minitab Inc.

Measurement Systems Analysis

Choosing a command
To look at your measurement system's precision, see Gage R&R Study.
To look at a plot of all of the measurements by operator/part combination, and thus visualize the repeatability and
reproducibility components of the measurement variation, see Gage Run Chart.
To look at both the bias and repeatability in an attribute measurement system, see Attribute Gage Study (Analytic
Method).

Measurement Systems Analysis

Stat > Quality Tools > Gage Study
Gage Run Chart
Gage Linearity and Bias Study
Gage R&R Study (Nested)
Gage R&R Study (Crossed)
Attribute Gage Study (Analytic Method)
Stat > Quality Tools
Attribute Agreement Analysis

Examples of Measurement Systems Analysis

Minitab Help offers examples of the following measurement systems analyses:
Gage Run Chart
Gage Linearity and Bias Study
Gage R&R Study (Crossed) ANOVA Method
Gage R&R Study (Crossed) X-bar and R method
Gage R&R Study (Nested)
Attribute Gage Study (Analytic Method)
Attribute Agreement Analysis

Number of Distinct Categories Statement

Minitab calculates the number in this statement by dividing the standard deviation for Parts by the standard deviation for
Gage, then multiplying by 1.41 and truncating this value. This number represents the number of non-overlapping
confidence intervals that will span the range of product variation. You can also think of it as the number of groups within
your process data that your measurement system can discern.
Imagine you measured 10 different parts, and Minitab reported that your measurement system could discern 4 distinct
categories. This means that some of those 10 parts are not different enough to be discerned as being different by your
measurement system. If you want to distinguish a higher number of distinct categories, you need a more precise gage.
The Automobile Industry Action Group (AIAG) [1] suggests that when the number of categories is less than 2, the
measurement system is of no value for controlling the process, since one part cannot be distinguished from another.
When the number of categories is 2, the data can be divided into two groups, say high and low. When the number of
categories is 3, the data can be divided into 3 groups, say low, middle and high. A value of 5 or more denotes an
acceptable measurement system.

References Measurement Systems

[1]

Automotive Industry Action Group (AIAG) (2002). Measurement Systems Analysis Reference Manual, 3rd edition.
Chrysler, Ford, General Motors Supplier Quality Requirements Task Force.

[2]

A. Duncan (1986). Quality Control and Industrial Statistics, 5th edition, Irwin.

[3]

N.L. Desu and S. Kotz (1990). Sample Size Methodology, Academic Press.

[4]

J.L. Fleiss (1981). Statistical Methods for Rates and Proportions, 2nd edition, John Wiley & Sons.

[5]

D. Futrell (1995). "When Quality is a Matter of Taste, Use Reliability Indexes," Quality Progress, 28 (5), 81-86.

2003 Minitab Inc.

Measurement Systems Analysis

[6]

N. Johnson and S. Kotz (1969). Discrete Distributions, John Wiley & Sons.

[7]

R.J.A. Little and D. B. Rubin (2002). Statistical Analysis With Missing Data, 2nd edition, John Wiley & Sons.

[8]

D.C. Montgomery and G.C. Runger (1993-4). "Gauge Capability and Designed Experiments. Part I: Basic Methods,"
Quality Engineering, 6 (1), 115135.

[9]

D.C. Montgomery and G.C. Runger (1993-4). "Gauge Capability Analysis and Designed Experiments. Part II:
Experimental Design Models and Variance Component Estimation," Quality Engineering, 6 (2), 289305.

[10] S.R. Searle, G. Casella, and C.E. McCulloch (1992). Variance Components, John Wiley & Sons.
[11] S. Siegel and N.J. Castellan (1988). Nonparametric Statistics for the Behavioral Sciences, 2nd edition, McGraw-Hill.

Gage Run Chart

Gage Run Chart
Stat > Quality Tools > Gage Study > Gage Run Chart
A gage run chart is a plot of all of your observations by operator and part number. A horizontal reference line is drawn at
the mean, which can be calculated from the data, or a value you enter from prior knowledge of the process. You can use
this chart to quickly assess differences in measurements between different operators and parts. A stable process would
give you a random horizontal scattering of points; an operator or part effect would give you some kind of pattern in the
plot.

Dialog box items

Part numbers: Enter the column containing part names or numbers.
Operators: Enter the column containing operator names or numbers.
Measurement data: Enter the column containing the observed measurements.
Trial numbers: Enter the column containing trial numbers (optional).
Historical mu: Enter your own value for the horizontal reference line. By default, the reference line is the mean of all
observations (optional).
<Gage Info>
<Options>

Data Gage Run Chart

Worksheet Structure
Structure your data so each row contains the part name or number, operator (optional), and the observed measurement.
Parts and operators can be text or numbers.
PartNum

Operator

Measure

Daryl

1.48

Daryl

1.43

Daryl

1.83

Daryl

1.83

Daryl

1.53

Daryl

1.38

Beth

1.78

Beth

1.33

...

...

...

To make a gage run chart

1

Choose Stat > Quality Tools > Gage Study > Gage Run Chart.

In Part numbers, enter the column of part names or numbers.

In Operators, enter the column of operator names or numbers.

In Measurement data, enter the column of measurements.

If you like, use one or more of the available dialog box options, then click OK.

2003 Minitab Inc.

Measurement Systems Analysis

Gage Info
Stat > Quality Tools > Gage Study > choose Gage Run Chart, Gage Linearity and Bias Study, Gage R&R Study
(Crossed), Gage R&R Study (Nested), or Attribute Gage Study (Analytic Method) > Gage Info
Use these fields to fill in the label on the Session window and graphical output. Each field corresponds to a line on the
label. If you do not use these fields, the label will be printed with blank lines.

Dialog box items

Gage name: Type the name of the gage.
Date of study: Type the date.
Reported by: Type the name of the person reporting the study information.
Gage Tolerance: Type the gage tolerance.
Miscellaneous: Type any miscellaneous notes.

Gage Run Chart Options

Stat > Quality Tools > Gage Study > Gage Run Chart > Options
Lets you enter a new title to replace the default title.

Dialog box items

Title: Enter a new title to replace the default title on the graphical output.

Two Examples of Gage Run Chart

In this example, you draw a gage run chart with two data sets: one in which measurement system variation contributes
little to the overall observed variation (GAGEAIAG.MTW), and one in which measurement system variation contributes a
lot to the overall observed variation (GAGE2.MTW). For comparison, see the same data sets analyzed by the Gage R&R
Study ANOVA Method or the Gage R&R Study Xbar and R method.
For the GAGEAIAG data, ten parts were selected that represent the expected range of the process variation. Three
operators measured the ten parts, two times per part, in a random order. For the GAGE2 data, three parts were selected
that represent the expected range of the process variation. Three operators measured the three parts, three times per
part, in a random order.

Step 1: Gage Run Chart with GAGEAIAG data

1

Open the worksheet GAGEAIAG.MTW.

Choose Stat > Quality Tools > Gage Study > Gage Run Chart.

In Part numbers, enter Part.

In Operators, enter Operator.

In Measurement data, enter Response. Click OK.

Step 2: Gage Run Chart with GAGE2 data

1

Open the file GAGE2.MTW.

Choose Stat > Quality Tools > Gage Study > Gage Run Chart.

In Part numbers, enter Part.

In Operators, enter Operator.

In Measurement data, enter Response. Click OK.

2003 Minitab Inc.

Measurement Systems Analysis

Graph window output
Gage Run Chart with GAGEAIAG data

Gage Run Chart with GAGE2 data

Interpreting the results

Gage Run Chart Example GAGEAIAG.MTW
For each part, you can compare both the variation between measurements made by each operator, and differences in
measurements between operators.
You can also look at the measurements in relationship to the horizontal reference line. By default the reference line is the
mean of all observations.
Most of the variation is due to differences between parts. Some smaller patterns also appear. For example, Operator 2's
second measurement is consistently (seven times out of ten) smaller than the first measurement. Operator 2's
measurements are consistently (eight times out of ten) smaller than Operator 1's measurements.

2003 Minitab Inc.

Measurement Systems Analysis

Gage Run Chart Example GAGE2.MTW
For each part, you can compare both the variation between measurements made by each operator, and differences in
measurements between operators.
You can also look at the measurements in relationship to the horizontal reference line. By default the reference line is the
mean of all observations.
The dominant factor here is repeatability large differences in measurements when the same operator measures the
same part. Oscillations might suggest the operators are "adjusting" how they measure between measurements.

Gage Linearity and Bias Study

Gage Linearity and Bias Study
Stat > Quality Tools > Gage Study > Gage Linearity and Bias Study
Gage linearity tells you how accurate your measurements are through the expected range of the measurements. It
answers the question, "Does my gage have the same accuracy for all sizes of objects being measured?"
Gage bias examines the difference between the observed average measurement and a reference or master value. It
answers the question, "How biased is my gage when compared to a master value?"

Dialog box items

Part numbers: Enter the column containing the part name or number.
Reference values: Enter the column containing the reference values.
Measurement data: Enter the column containing the observed measurements.
Process Variation (optional): Enter the process standard deviation. You can get the process standard deviation from the
Total Variation row of the 6 SD column of the output from the Gage R&R Study ANOVA Method. This is the number
that is usually associated with process variation. If you do not know the value for the process variation, you can enter the
process tolerance instead.
<Gage Info>
<Options>

Data Gage Linearity and Bias Study

Worksheet Structure
Structure your data so each row contains a part, reference value, and the observed measurement on that part (the
response). Parts can be text or numbers.
PartNum

Reference Response

2.7

2.5

2.4

2.5

2.7

2.3

2.5

2.5

2.4

2.4

2.6

2.4

5.1

3.9

...

...

...

To do a gage linearity and bias study

1

Choose Stat > Quality Tools > Gage Study > Gage Linearity and Bias Study.

2003 Minitab Inc.

Measurement Systems Analysis

2

In Part numbers, enter the column of part names or numbers.

In Master Measurements, enter the column of master measurements.

In Measurement data, enter the column of observed measurements.

In Process Variation, enter a value. You can get this value from the Gage R&R Study ANOVA method: it is the
value in the Total Variation row of the 6 SD column. This number is usually associated with process variation. If you
do not know the process variation value, enter the process tolerance instead.

If you like, use any dialog box options, then click OK.

Gage Info
Stat > Quality Tools > Gage Study > choose Gage Run Chart, Gage Linearity and Bias Study, Gage R&R Study
(Crossed), Gage R&R Study (Nested), or Attribute Gage Study (Analytic Method) > Gage Info
Use these fields to fill in the label on the Session window and graphical output. Each field corresponds to a line on the
label. If you do not use these fields, the label will be printed with blank lines.

Dialog box items

Gage name: Type the name of the gage.
Date of study: Type the date.
Reported by: Type the name of the person reporting the study information.
Gage Tolerance: Type the gage tolerance.
Miscellaneous: Type any miscellaneous notes.

Gage Linearity and Bias Options

Stat > Quality Tools > Gage Study > Gage Linearity and Bias Study > Options
Use to specify the method to estimate the repeatability standard deviation or to enter a new title to replace the default title.

Dialog box items

Methods of estimating repeatability standard deviation
Sample range: Choose to use the sample range to estimate the repeatability standard deviation.
Sample standard deviation: Choose to use the sample standard deviation to estimate the repeatability standard
deviation.
Title: Enter a new title to replace the default title on the graphical output.

Example of Gage Linearity and Bias Study

A plant foreman chose five parts that represented the expected range of the measurements. Each part was measured by
layout inspection to determine its reference (master) value. Then, one operator randomly measured each part twelve
times. You obtained the process variation (14.1941) from a Gage R&R study using the ANOVA method. Minitab displays
the process variation in the Session window (Total Variation row of the Study Var (6 * SD) column).
The data set used in this example has been reprinted with permission from the Measurement Systems Analysis
Reference Manual (Chrysler, Ford, General Motors Supplier Quality Requirements Task Force).
1

Open the worksheet GAGELIN.MTW.

Choose Stat > Quality Tools > Gage Study > Gage Linearity and Bias Study.

In Part numbers, enter Part.

In Reference values, enter Master.

In Measurement data, enter Response.

In Process Variation, type 14.1941. Click OK.

2003 Minitab Inc.

Measurement Systems Analysis

Graph window output

Interpreting the results

The %Linearity (absolute value of the slope * 100) is 13.2, which means that the gage linearity accounts for 13% of the
overall process variation.
The %Bias for the reference average is 0.4, which means that the gage bias accounts for less than 0.4% of the overall
process variation.

Gage R&R Study (Crossed)

Gage R&R Overview
Gage repeatability and reproducibility studies determine how much of your observed process variation is due to
measurement system variation. Minitab allows you to perform either crossed or nested Gage R&R studies.

Use Gage R&R Study (Crossed) when each part is measured multiple times by each operator.

Use Gage R&R Study (Nested) when each part is measured by only one operator, such as in destructive testing. In
destructive testing, the measured characteristic is different after the measurement process than it was at the
beginning. Crash testing is an example of destructive testing.

Minitab provides two methods for assessing repeatability and reproducibility: X-bar and R, and ANOVA. The X-bar and R
method breaks down the overall variation into three categories: part-to-part, repeatability, and reproducibility. The ANOVA
method goes one step further and breaks down reproducibility into its operator, and operator-by-part, components.
The ANOVA method is more accurate than the X-bar and R method, in part, because it considers the operator by part
interaction [8, 9]. Gage R&R Study (Crossed) allows you to choose between the X-bar and R method and the ANOVA
method. Gage R&R Study (Nested) uses the ANOVA method only.
If you need to use destructive testing, you must be able to assume that all parts within a single batch are identical enough
to claim that they are the same part. If you are unable to make that assumption then part-to-part variation within a batch
will mask the measurement system variation.
If you can make that assumption, then choosing between a crossed or nested Gage R&R Study for destructive testing
depends on how your measurement process is set up. If all operators measure parts from each batch, then use Gage
R&R Study (Crossed). If each batch is only measured by a single operator, then you must use Gage R&R Study (Nested).
In fact, whenever operators measure unique parts, you have a nested design.

2003 Minitab Inc.

Measurement Systems Analysis

If you have attribute data, you can use Attribute Gage Study (Analytic Method) to analyze the bias and repeatability of
your process.

Gage R&R Study (Crossed)

Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed)
Gage repeatability and reproducibility studies determine how much of your observed process variation is due to
measurement system variation. Minitab provides two methods for Gage R&R Crossed: X-bar and R, or ANOVA. The Xbar and R method breaks down the overall variation into three categories: part-to-part, repeatability, and reproducibility.
The ANOVA method goes one step further and breaks down reproducibility into its operator, and operator by part,
components.

The ANOVA method is more accurate than the X-bar and R method, in part, because it accounts for the Operator by Part
interaction [8, 9].

Dialog box items

Part numbers: Enter the column containing the part names or numbers.
Operators: Enter the column containing operator names or numbers (optional).
Measurement data: Enter the column containing the observed measurements.
Method of Analysis
ANOVA: Choose to use the ANOVA method.
Xbar and R: Choose to use the X-bar and R method.
<Gage Info>
<Options>

Data Gage R&R Study (Crossed)

Structure your data so that each row contains the part name or number, operator (optional), and the observed
measurement. Parts and operators can be text or numbers.
PartNum

Operator

Measure

Daryl

1.48

Daryl

1.43

Daryl

1.83

Daryl

1.83

Daryl

1.53

Daryl

1.38

Beth

1.78

Beth

1.33

...

...

...

2003 Minitab Inc.

Measurement Systems Analysis

The Gage R&R studies require balanced designs (equal numbers of observations per cell) and replicates. You can
estimate any missing observations with the methods described in [7].

To do a gage R&R study (crossed)

1

Choose Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed).

In Part numbers, enter the column of part names or numbers.

In Measurement data, enter the column of measurements.

Under Method of Analysis, choose ANOVA (the default) or Xbar and R.

If you like, use any dialog box options, then click OK.

Data Sets Used in Examples

The same two data sets are used in the Gage R&R Study and the Gage Run Chart examples:

GAGE2.MTW, in which measurement system variation has a large effect on the overall observed variation

GAGEAIAG.MTW, in which measurement system variation has a small effect on the overall observed variation

You can compare the output for the two data sets, as well as compare results from the various analyses.
The table below lists the other measurement systems analysis examples.
Example

Data set

Gage Linearity Study

GAGELIN.MTW

Attribute Gage Study (Analytic

Method)

AUTOGAGE.MTW

Attribute Agreement Analysis

ESSAY.MTW

GAGEAIAG.MTW and GAGELIN.MTW are reprinted with permission from the Measurement Systems Analysis Reference
Manual (Chrysler, Ford, General Motors Supplier Quality Requirements Task Force).

Gage R&R Graph Window Output

The Gage R&R study produces six graphs:

Components of Variation is a visualization of the final table in the Session window output, displaying bars for: Total
Gage R&R, Repeatability, Reproducibility (but not Operator and Operator by Part), and Part-to-Part variation.

R Chart by Operator displays the variation in the measurements made by each operator, allowing you to compare
operators to each other. The R chart is displayed when the number of replicates is less than 9. Otherwise, a S chart is
displayed.

X-bar Chart by Operator displays the measurements in relation to the overall mean for each operator, allowing you to
compare operators to each other, and to the mean.

By Part displays the main effect for Part, so you can compare the mean measurement for each part. If you have many
replicates, boxplots are displayed on the By Part graph.

By Operator displays the main effect for Operator, so you can compare mean measurement for each operator. If you
have many replicates, boxplots are displayed on the By Operator graph.

Operator by Part Interaction displays the Operator by Part effect, so you can see how the relationship between
Operator and Part changes depending on the operator.

Missing Data
Gage R&R studies require balanced designs (equal numbers of observations per cell) and replicates to calculate the
variance components. If you have missing data, you may want to estimate the missing observations. See [7] for methods
of estimating. Once the missing observations have been estimated, the data can be analyzed as a balanced design.

Gage Info
Stat > Quality Tools > Gage Study > choose Gage Run Chart, Gage Linearity and Bias Study, Gage R&R Study
(Crossed), Gage R&R Study (Nested), or Attribute Gage Study (Analytic Method) > Gage Info
Use these fields to fill in the label on the Session window and graphical output. Each field corresponds to a line on the
label. If you do not use these fields, the label will be printed with blank lines.

10

2003 Minitab Inc.

Measurement Systems Analysis

Dialog box items
Gage name: Type the name of the gage.
Date of study: Type the date.
Reported by: Type the name of the person reporting the study information.
Gage Tolerance: Type the gage tolerance.
Miscellaneous: Type any miscellaneous notes.

Gage R&R Study (Crossed) Options

Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed) > Options
Allows you to control the Session and Graph window output.

Dialog box items

Study variation: Enter a multiplier to use in the StudyVar column of the Session window output. The default multiplier is
6, which is the number of standard deviations needed to capture 99.73% of your process measurements.
Process tolerance: Enter a known tolerance range. A %Tolerance column will be displayed in the bottom table of the
Session window output and in the components of variation graph. This column shows the percentage of the process
tolerance taken up by each variance component.
Historical standard deviation: Enter a known historical standard deviation. A %Process column will be displayed in the
bottom table of the Session window output and in the components of variation graph. This column shows the percentage
of process standard deviation taken up by each variance component.
Alpha to remove interaction term: Enter the alpha value used to remove the interaction term from the model. This
option is only available with the ANOVA method.
Do not display percent contribution: Check to suppress display of percent contribution.
Do not display percent study variation: Check to suppress display of percent study variation.
Draw figures on separate graphs, one figure per graph: Check to display each graph separately. By default, Gage
R&R Study produces a layout of 6 graphs in one window.
Title: Enter a new title to replace the default title on the graphical output.

Example of Gage R&R Study (ANOVA method)

In this example, we do a gage R&R study on two data sets: one in which measurement system variation contributes little
to the overall observed variation (GAGEAIAG.MTW), and one in which measurement system variation contributes a lot to
the overall observed variation (GAGE2.MTW). For comparison, we analyze the data using both the ANOVA method
(below) and the Xbar and R method. You can also look at the same data plotted on a Gage Run Chart.
For the GAGEAIAG data set, ten parts were selected that represent the expected range of the process variation. Three
operators measured the ten parts, two times per part, in a random order. For the GAGE2 data, three parts were selected
that represent the expected range of the process variation. Three operators measured the three parts, three times per
part, in a random order

Step 1: Use the ANOVA method with GAGEAIAG data

1

Open the file GAGEAIAG.MTW.

Choose Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed).

In Part numbers, enter Part.

In Operators, enter Operator.

In Measurement data, enter Response.

Under Method of Analysis, choose ANOVA.

Click OK.

Step 2: Use the ANOVA method with GAGE2 data

1

Open the file GAGE2.MTW.

Choose Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed).

In Part numbers, enter Part.

In Operators, enter Operator.

In Measurement data, enter Response.

2003 Minitab Inc.

11

Measurement Systems Analysis

6

Under Method of Analysis, choose ANOVA.

Click OK.

Session window output

ANOVA method with GAGEAIAG data
Gage R&R Study - ANOVA Method
Two-Way ANOVA Table With Interaction

Source
Part
Operator
Part * Operator
Repeatability
Total

DF
9
2
18
30
59

SS
2.05871
0.04800
0.10367
0.03875
2.24913

MS
0.228745
0.024000
0.005759
0.001292

F
39.7178
4.1672
4.4588

P
0.000
0.033
0.000

Gage R&R

%Contribution
(of VarComp)
10.67
3.10
7.56
2.19
5.37
89.33
100.00

Source
Total Gage R&R
Repeatability
Reproducibility
Operator
Operator*Part
Part-To-Part
Total Variation

VarComp
0.0044375
0.0012917
0.0031458
0.0009120
0.0022338
0.0371644
0.0416019

Source
Total Gage R&R
Repeatability
Reproducibility
Operator
Operator*Part
Part-To-Part
Total Variation

StdDev (SD)
0.066615
0.035940
0.056088
0.030200
0.047263
0.192781
0.203965

Study Var
(6 * SD)
0.39969
0.21564
0.33653
0.18120
0.28358
1.15668
1.22379

%Study Var
(%SV)
32.66
17.62
27.50
14.81
23.17
94.52
100.00

Number of Distinct Categories = 4

Gage R&R for Response

ANOVA method with GAGE2 data

Gage R&R Study - ANOVA Method
Two-Way ANOVA Table With Interaction

Source
Part
Operator
Part * Operator
Repeatability
Total

DF
2
2
4
18
26

SS
38990
529
26830
133873
200222

MS
19495.2
264.3
6707.4
7437.4

F
2.90650
0.03940
0.90185

P
0.166
0.962
0.484

Two-Way ANOVA Table Without Interaction

Source

12

DF

SS

MS

2003 Minitab Inc.

Measurement Systems Analysis

Part
Operator
Repeatability
Total

2
2
22
26

38990
529
160703
200222

19495.2
264.3
7304.7

2.66887
0.03618

0.092
0.965

Gage R&R

%Contribution
(of VarComp)
84.36
84.36
0.00
0.00
15.64
100.00

Source
Total Gage R&R
Repeatability
Reproducibility
Operator
Part-To-Part
Total Variation

VarComp
7304.67
7304.67
0.00
0.00
1354.50
8659.17

Source
Total Gage R&R
Repeatability
Reproducibility
Operator
Part-To-Part
Total Variation

StdDev (SD)
85.4673
85.4673
0.0000
0.0000
36.8036
93.0547

Study Var
(6 * SD)
512.804
512.804
0.000
0.000
220.821
558.328

%Study Var
(%SV)
91.85
91.85
0.00
0.00
39.55
100.00

Number of Distinct Categories = 1

Gage R&R for Response

Graph window output

ANOVA method with GAGEAIAG data

2003 Minitab Inc.

13

Measurement Systems Analysis

ANOVA method with GAGE2 data

Interpreting the results

Session Window Output - GAGEAIAG.MTW

Look at the p-value for the Operator*Part interaction in the ANOVA Table. When the p-value for Operator by Part is <
0.25, Minitab fits the full model (shown in the ANOVA Table With Operator * Part Interaction (p = 0.00)). In this case,
the ANOVA method will be more accurate than the X-bar and R method, which does not account for this interaction.

Look at the %Contribution column in the Gage R&R Table. The percent contribution from Part-To-Part (89.33) is larger
than that of Total Gage R&R (10.67). This tells you that much of the variation is due to differences between parts.
However, according to AIAG, a Total Gage R&R %Contribution greater than 10% is poor.

For this data, the number of distinct categories is four. According to the AIAG, you need at least five distinct categories
to have an adequate measuring system. See Number of distinct categories statement.

Graph Window Output GAGEAIAG.MTW

In the Components of Variation graph (located in the upper left corner), the percent contribution from Part-To-Part is
larger than that of Total Gage R&R, telling you that much of the variation is due to differences between parts;
however, more than 10% of the variation is due to the measurement system.

In the By Part graph (located in upper right corner), there are large differences between parts, as shown by the nonlevel line.

In the By Operator graph (located in the middle of the right column), there are small differences between operators, as
shown by the nearly level line.

In the Xbar Chart by Operator (located in lower left corner), most of the points in the X-bar and R chart are outside the
control limits, indicating variation is mainly due to differences between parts.

The OperatorInteraction graph is a visualization of the p-value for OperPart 0.00 in this case indicating a
significant interaction between each Part and Operator.

Session Window Output GAGE2.MTW

Look at the p-value for the OperatorPart interaction in the ANOVA Table. When the p-value for Operator by Part is >
0.25, Minitab fits the model without the interaction and uses the reduced model to define the Gage R&R statistics.

Look at the %Contribution column in the Gage R&R Table. The percent contribution from Total Gage R&R (84.36) is
larger than that of Part-To-Part (15.64). Thus, most of the variation arises from the measuring system; very little is due
to differences between parts.

A 1 tells you the measurement system is poor; it cannot distinguish differences between parts.

Graph Window Output GAGE2.MTW

In the Components of Variation graph (located in the upper left corner), the percent contribution from Total Gage R&R
is larger than that of Part-to-Part, telling you that most of the variation is due to the measurement system primarily
repeatability; little is due to differences between parts.

In the By Part graph (located in upper right corner), there is little difference between parts, as shown by the nearly
level line.

14

2003 Minitab Inc.

Measurement Systems Analysis

In the Xbar Chart by Operator (located in lower left corner), most of the points in the X-bar and R chart are inside the
control limits, indicating the observed variation is mainly due to the measurement system.

In the By Operator graph (located in the middle of the right column), there are no differences between operators, as
shown by the level line.

The OperatorInteraction graph is a visualization of the p-value for OperPart 0.484 in this case indicating the
differences between each operator/part combination are insignificant compared to the total amount of variation.

Example of a Gage R&R Study (Crossed) (X-bar and R method)

In this example, we do a gage R&R study on two data sets: one in which measurement system variation contributes little
to the overall observed variation (GAGEAIAG.MTW), and one in which measurement system variation contributes a lot to
the overall observed variation (GAGE2.MTW). For comparison, we analyze the data using both the X-bar and R method
(below) and the ANOVA method. You can also look at the same data plotted on a Gage Run Chart.
For the GAGEAIAG data set, ten parts were selected that represent the expected range of the process variation. Three
operators measured the ten parts, two times per part, in a random order. For the GAGE2 data, three parts were selected
that represent the expected range of the process variation. Three operators measured the three parts, three times per
part, in a random order.

Step 1: Use the X-bar and R method with GAGEAIAG data

1

Open the file GAGEAIAG.MTW.

Choose Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed).

In Part numbers, enter Part.

In Operators, enter Operator.

In Measurement data, enter Response.

Under Method of Analysis, choose Xbar and R.

Click OK.

Step 2: Use the X-bar and R method with GAGE2 data

1

Open the file GAGE2.MTW.

Choose Stat > Quality Tools > Gage Study > Gage R&R Study (Crossed).

In Part numbers, enter Part.

In Operators, enter Operator.

In Measurement data, enter Response.

Under Method of Analysis, choose Xbar and R.

Click OK.

Session window output

X-bar and R method with GAGEAIAG data
Gage R&R Study - XBar/R Method

Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

VarComp
0.0020839
0.0011549
0.0009291
0.0308271
0.0329111

Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

StdDev (SD)
0.045650
0.033983
0.030481
0.175577
0.181414

2003 Minitab Inc.

%Contribution
(of VarComp)
6.33
3.51
2.82
93.67
100.00

Study Var
(6 * SD)
0.27390
0.20390
0.18288
1.05346
1.08848

%Study Var
(%SV)
25.16
18.73
16.80
96.78
100.00

15

Measurement Systems Analysis

Number of Distinct Categories = 5

Gage R&R for Response

Number of Distinct Categories = 5

X-bar and R method with GAGE2 data

Gage R&R Study - XBar/R Method
%Contribution
(of VarComp)
78.11
78.11
0.00
21.89
100.00

Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

VarComp
7229.94
7229.94
0.00
2026.05
9255.99

Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

StdDev (SD)
85.0291
85.0291
0.0000
45.0116
96.2081

Study Var
(6 * SD)
510.174
510.174
0.000
270.070
577.248

%Study Var
(%SV)
88.38
88.38
0.00
46.79
100.00

Number of Distinct Categories = 1

Gage R&R for Response

Graph window output

X-bar and R method with GAGEAIAG data

16

2003 Minitab Inc.

Measurement Systems Analysis

X-bar and R method with GAGE2 data

Interpreting the results

Session Window Output GAGEAIAG.MTW
Look at the %Contribution column in the Gage R%R Table. The measurement system variation (Total Gage R&R) is
much smaller than what was found for the same data with the ANOVA method. That is because the X-bar and R method
does not account for the Operator by Part effect, which was very large for this data set. Here you get misleading estimates
of the percentage of variation due to the measurement system.
According to AIAG, when the number of distinct categories is 5 it represents an adequate measuring system. However, as
explained above, you would be better off using the ANOVA method for this data. See Number of distinct categories
statement.

Graph Window Output GAGEAIAG.MTW

In the Components of Variation graph, a low percentage of variation (6.33%) is due to the measurement system (Gage
R&R), and a high percentage (93.67%) is due to differences between parts.
In the X-bar Chart by Operator graph, although the X-bar and R method does not account for the Operator by Part
interaction, this plot shows you that the interaction is significant. Here, the X-bar and R method grossly overestimates the
capability of the gage. You may want to use the ANOVA method, which accounts for the Operator by Part interaction.
Most of the points in the X-bar Chart are outside the control limits when the variation is mainly due to part-to-part
differences.

Session Window Output GAGE2.MTW

Look at the %Contribution column in the Gage R&R Table. A large percentage (78.11%) of the variation in the data is due
to the measuring system (Gage R&R); little is due to differences between parts (21.89%).
A 1 in Number of distinct categories tells you the measurement system is poor; it cannot distinguish differences between
parts.

Graph Window Output GAGE2.MTW

In the Components of Variation graph, a high percentage of variation (78.11%) is due to the measurement system (Gage
R&R) primarily repeatability, and the low percentage (21.89%) is due to differences between parts.
Most of the points in the X-bar chart will be within the control limits when the observed variation is mainly due to the
measurement system.

Gage R&R Study (Nested)

Gage R&R
Gage repeatability and reproducibility studies determine how much of your observed process variation is due to
measurement system variation. Minitab allows you to perform either crossed or nested Gage R&R studies.

Use Gage R&R Study (Crossed) when each part is measured multiple times by each operator.

2003 Minitab Inc.

17

Measurement Systems Analysis

Use Gage R&R Study (Nested) when each part is measured by only one operator, such as in destructive testing. In
destructive testing, the measured characteristic is different after the measurement process than it was at the
beginning. Crash testing is an example of destructive testing.

Minitab provides two methods for assessing repeatability and reproducibility: X-bar and R, and ANOVA. The X-bar and R
method breaks down the overall variation into three categories: part-to-part, repeatability, and reproducibility. The ANOVA
method goes one step further and breaks down reproducibility into its operator, and operator-by-part, components.
The ANOVA method is more accurate than the X-bar and R method, in part, because it considers the operator by part
interaction [8, 9]. Gage R&R Study (Crossed) allows you to choose between the X-bar and R method and the ANOVA
method. Gage R&R Study (Nested) uses the ANOVA method only.
If you need to use destructive testing, you must be able to assume that all parts within a single batch are identical enough
to claim that they are the same part. If you are unable to make that assumption then part-to-part variation within a batch
will mask the measurement system variation.
If you can make that assumption, then choosing between a crossed or nested Gage R&R Study for destructive testing
depends on how your measurement process is set up. If all operators measure parts from each batch, then use Gage
R&R Study (Crossed). If each batch is only measured by a single operator, then you must use Gage R&R Study (Nested).
In fact, whenever operators measure unique parts, you have a nested design.
If you have attribute data, you can use Attribute Gage Study (Analytic Method) to analyze the bias and repeatability of
your process.

Gage R&R Study (Nested)

Stat > Quality Tools > Gage Study > Gage R&R Study (Nested)
Gage repeatability and reproducibility studies determine how much of your observed process variation is due to
measurement system variation. Use Gage R&R Study (Nested) when each part is measured by only one operator, such
as in destructive testing. In destructive testing, the measured characteristic is different after the measurement process
than it was at the beginning. Crash testing is an example of destructive testing.
Gage R&R Study (Nested) uses the ANOVA method for assessing repeatability and reproducibility. The ANOVA method
goes one step further and breaks down reproducibility into its operator, and operator-by-part, components.

If you need to use destructive testing, you must be able to assume that all parts within a single batch are identical enough
to claim that they are the same part. If you are unable to make that assumption then part-to-part variation within a batch
will mask the measurement system variation.
If you can make that assumption, then choosing between a crossed or nested Gage R&R Study for destructive testing
depends on how your measurement process is set up. If all operators measure parts from each batch, then use Gage
R&R Study (Crossed). If each batch is only measured by a single operator, then you must use Gage R&R Study (Nested).
In fact, whenever operators measure unique parts, you have a nested design.

Dialog box items

Part or batch numbers: Enter the column containing the part or batch names or numbers.
Operators: Enter the column containing operator names or numbers (optional).
Measurement data: Enter the column containing the observed measurements.
<Gage Info>
<Options>

18

2003 Minitab Inc.

Measurement Systems Analysis

Data Gage R&R Study (Nested)

Structure your data so that each row contains the part name or number, operator, and the observed measurement. Parts
and operators can be text or numbers. Part is nested within operator, because each operator measures unique parts.
Note If you use destructive testing, you must be able to assume that all parts within a single batch are identical enough
to claim that they are the same part.
In the example below, PartNum1 for Daryl is truly a different part from PartNum1 for Beth.
PartNum

Operator

Measure

PartNum

Operator

Daryl

1.48

Daryl

Measure
1.48

Daryl

1.43

Daryl

1.43

Daryl

1.83

Daryl

1.83

Daryl

1.83

Daryl

1.83

Daryl

1.53

Daryl

1.53

Daryl

1.52

Daryl

1.52

Beth

1.38

Beth

1.38

Beth

1.78

Beth

1.78

Beth

1.33

Beth

1.33

...

...

...

...

...

...

The Gage R&R studies require balanced designs (equal numbers of observations per cell) and replicates. You can
estimate any missing observations with the methods described in [7].

To do a gage R&R study (nested)

1

Choose Stat > Quality Tools > Gage Study > Gage R&R Study (Nested).

In Part or batch numbers, enter the column of part or batch names or numbers.

In Operators, enter the column of operator names or numbers.

In Measurement data, enter the column of measurements.

If you like, use any of the available dialog box options, then click OK.

Gage Info
Stat > Quality Tools > Gage Study > choose Gage Run Chart, Gage Linearity and Bias Study, Gage R&R Study
(Crossed), Gage R&R Study (Nested), or Attribute Gage Study (Analytic Method) > Gage Info
Use these fields to fill in the label on the Session window and graphical output. Each field corresponds to a line on the
label. If you do not use these fields, the label will be printed with blank lines.

Dialog box items

Gage name: Type the name of the gage.
Date of study: Type the date.
Reported by: Type the name of the person reporting the study information.
Gage Tolerance: Type the gage tolerance.
Miscellaneous: Type any miscellaneous notes.

Gage R&R Study (Nested) Options

Stat > Quality Tools > Gage Study > Gage R&R Study (Nested) > Options
Allows you to control the Session and Graph window output.

Dialog box items

Study variation: Enter a multiplier to use in the StudyVar column of the Session window output. The default multiplier is
6, which is the number of standard deviations needed to capture 99.73% of your process measurements.
Process tolerance: Enter a known tolerance range. A %Tolerance column will be displayed in the bottom table of the
Session window output and in the components of variation graph. This column shows the percentage of the process
tolerance taken up by each variance component.

2003 Minitab Inc.

19

Measurement Systems Analysis

Historical standard deviation: Enter a known historical standard deviation. A %Process column will be displayed in the
bottom table of the Session window output and in the components of variation graph. This column shows the percentage
of process standard deviation taken up by each variance component.
Do not display percent contribution: Check to suppress display of percent contribution.
Do not display percent study variation: Check to suppress display of percent study variation.
Draw figures on separate graphs, one figure per graph: Check to display each graph separately. By default, Gage
R&R Study produces a layout of 6 graphs in one window.
Title: Enter a new title to replace the default title on the graphical output.

Example of gage R&R study (nested)

In this example, three operators each measured five different parts twice, for a total of 30 measurements. Each part is
unique to operator; no two operators measured the same part. You decide to conduct a gage R&R study (nested) to
determine how much of your observed process variation is due to measurement system variation.
1

Open the worksheet GAGENEST.MTW.

Choose Stat > Quality Tools > Gage Study > Gage R&R Study (Nested).

In Part or batch numbers, enter Part.

In Operators, enter Operator.

In Measurement data, enter Response.

Click OK.

Session window output

Gage R&R Study - Nested ANOVA
Gage R&R (Nested) for Response

Source
Operator
Part (Operator)
Repeatability
Total

DF
2
12
15
29

SS
0.0142
22.0552
19.3400
41.4094

MS
0.00708
1.83794
1.28933

F
0.00385
1.42549

P
0.996
0.255

Gage R&R

%Contribution
(of VarComp)
82.46
82.46
0.00
17.54
100.00

Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

VarComp
1.28933
1.28933
0.00000
0.27430
1.56364

Source
Total Gage R&R
Repeatability
Reproducibility
Part-To-Part
Total Variation

StdDev (SD)
1.13549
1.13549
0.00000
0.52374
1.25045

Study Var
(6 * SD)
6.81293
6.81293
0.00000
3.14243
7.50273

%Study Var
(%SV)
90.81
90.81
0.00
41.88
100.00

Number of Distinct Categories = 1

Gage R&R (Nested) for Response

20

2003 Minitab Inc.

Measurement Systems Analysis

Graph window output
Gage R&R (Nested) for Response

Interpreting the results

Look at the %Contribution columns for Total Gage R&R and Part-to-Part. The percent contribution for differences between
parts (Part-To-Part = 17.54) is much smaller than the percentage contribution for measurement system variation (Total
Gage R&R = 82.46). This indicates that most of the variation is due to measurement system error; very little is due to
differences between part.
A 1 in number of distinct categories tells you that the measurement system is not able to distinguish between parts. See
Number of distinct categories statement.
Look at the Components of Variation Graph located in upper left corner. Most of the variation is due to measurement
system error (Gage R&R), while a low percentage of variation is due to differences between parts.
Look at the X-bar Chart located in the lower left corner. Most of the points in the X-bar chart are inside the control limits
when the variation is mostly due to measurement system error.

Attribute Agreement Analysis

Attribute Agreement Analysis
Stat > Quality Tools > Attribute Agreement Analysis
Use attribute agreement analysis to assess the agreement of nominal or ordinal ratings given by multiple appraisers. The
measurements are subjective ratings by people rather than direct physical measurements. Examples include:

Automobile performance ratings

Classification of fabric quality as "good" or "bad"

Ratings of wine color, aroma, and taste on a 1 to 10 scale

In these situations, quality characteristics are difficult to define and assess. To obtain meaningful classifications, more
than one appraiser should classify the response measure. If the appraisers agree, the possibility exists that the ratings are
accurate. If the appraisers disagree, rating usefulness is limited.
Note

Attribute Agreement Analysis was previously called Attribute Gage R&R Study in Minitab Release 13. Attribute
Agreement Analysis, a technique to assess appraisers' agreement, is different from Attribute Gage Study
(Analytic Method), a method to examine the bias and repeatability of an attribute measurement system.

Dialog box items

Data are arranged as
Attribute column: Choose to enter the column containing the responses.
Samples: Enter the column containing the sample or part number.

2003 Minitab Inc.

21

Measurement Systems Analysis

Appraisers: Enter the column containing the appraiser name or number.
Multiple columns (Enter trials for each appraiser together): Choose to enter the columns containing the responses.
Keep the trials for each appraiser in adjoining columns.
Number of appraisers: Type the number of appraisers.
Number of trials: Type the number of trials.
Appraiser names (optional): Type the appraisers' names or enter a column of names.
Known standard/attribute (optional): Enter a column containing the attribute or known standard for each sample. The
column can contain either numeric or text attributes, but the data type needs to match the response type.
Categories of the attribute data are ordered: Check to specify that the data have more than two levels and are ordinal.
<Information>
<Options>
<Graphs>
<Results>

Data Attribute Agreement Analysis

Data can be text or numeric. The assigned ratings can be nominal or ordinal:

Nominal data are categorical variables that have two or more levels with no natural ordering. For example, the levels
in a food tasting study may include crunchy, mushy, and crispy.

Ordinal data are categorical variables that have three or more levels with a natural ordering, such as strongly disagree,
disagree, neutral, agree, and strongly agree.

Data can be stacked in an attribute column or unstacked in multiple columns. For attribute column data, stack all of the
responses in one column and set up grouping indicator columns for the appraiser and part number. The grouping
indicators define each sample. Grouping indicators and responses can be text or numeric. For multiple column data,
responses for each appraiser and each trial are in separate columns.
Here is the same data set structured both ways:

22

2003 Minitab Inc.

Measurement Systems Analysis

Attribute column data

Sample Appraiser
1

Multiple columns data

Response

Sample

Appraiser A

Appraiser A

Appraiser B

Trial 1

Trial 2

Trial 1

Trial 2

Good

Good

Bad

Good

Good

Good

Good

Good

Bad

Good

Bad

Bad

Good

Good

Good

Good

Bad

Bad

Good

Bad

Good

Appraiser B

Good

Bad

Good

Good

Good

Good

Good

Bad

Good

Bad

Bad

Good

Good

Good

Good

Bad

Bad

Good

Bad

Note

When your data are in multiple columns, you must enter the trials for each operator together, as shown above.
The order of the trials must be the same for each operator.

To do an attribute agreement analysis

1

Choose Stat > Quality Tools > Attribute Agreement Analysis.

Do one of the following:

If your data are stacked in one column, enter the data column in Attribute column. In Samples, enter the column
of sample indicators. In Appraisers, enter the column of appraiser names or numbers.
If your data are unstacked and in multiple columns, enter the data columns in Multiple columns. Keep the trials for
each appraiser together. In Number of appraisers, type the number of appraisers. In Number of trials, type the
number of trials.

If your data have more than two levels and are ordinal, check Categories of the attribute data are ordered.

If you like, use any dialog box options, then click OK.

Attribute Agreement Analysis Information

Stat > Quality Tools > Attribute Agreement Analysis > Information
Use these fields to fill in the label on the Session window and graphical output. Each field corresponds to a line on the
label. If you do not use these fields, Minitab will display the label with blank lines.

2003 Minitab Inc.

23

Measurement Systems Analysis

Dialog box items
Date of study: Enter the date the study was conducted.
Reported by: Enter the name of the person reporting the study information.
Product name: Enter the name of the product.
Miscellaneous: Enter any miscellaneous notes.

Attribute Agreement Analysis Options

Stat > Quality Tools > Attribute Agreement Analysis > Options
Use to specify a value for the confidence level.

Dialog box items

Calculate Cohen's kappa if appropriate: Check to calculate Cohen's kappa when appropriate. Minitab will calculate
Cohen's kappa when two appraisers rate a single trial or when each appraiser rates two trials.
Confidence level: Enter the confidence level of the interval estimation of the percentages of the assessment agreement
within appraisers and between each appraiser and standard. The default is set at 95.

Attribute Agreement Analysis Graphs

Stat > Quality Tools > Attribute Agreement Analysis > Graphs
Use to choose which graphs to display.

Dialog box items

Plot the percentage of agreed responses among
Trials per appraiser (no graph for single trial): Check to display a graph of the matched proportions for each
appraiser. This graph is displayed when the number of trials for each appraiser is more than one.
Trials per appraiser and known standard/attribute: Check to display a graph of the matched proportions between
the ratings of each appraiser and the attribute. This graph is displayed only when the attribute is known and provided
for each sample.
Display figures on one graph: Check to display both graphs on one page.

Attribute Agreement Analysis Results

Stat > Quality Tools > Attribute Agreement Analysis > Results
Use to control the display of Session window output, including Fleiss' kappa, Cohen's kappa, Kendall's coefficient of
concordance, and Kendall's correlation coefficient.

Dialog box items

Control the Display of Results
Display nothing: Choose to suppress all Session window output. Minitab will still display graphs.
Percentages of assessment agreement within and between appraisers: Choose to display the percentages of
assessment agreement within and between appraisers.
In addition, kappa and Kendall's (ordinal data) coefficient: Choose to display the output described above, plus
kappa coefficient tables and Kendall's coefficient (ordinal data only) tables.

Example of Attribute Agreement Analysis

An educational testing company is training five new appraisers for the written portion of the twelfth-grade standardized
essay test. The appraisers' ability to rate essays consistent with standards needs to be assessed. Each appraiser rated
fifteen essays on a five-point scale (-2, -1, 0, 1, 2).
1

Open the worksheet ESSAY.MTW.

Choose Stat > Quality Tools > Attribute Agreement Analysis.

In Attribute column, enter Rating.

In Samples, enter Sample.

In Appraisers, enter Appraiser.

In Known standard/attribute, enter Attribute.

Check Categories of the attribute data are ordered and click OK.

24

2003 Minitab Inc.

Measurement Systems Analysis

Session window output
Attribute Agreement Analysis for Rating
Each Appraiser vs Standard

Assessment Agreement
Appraiser
Duncan
Hayes
Holmes
Montgomery
Simpson

# Inspected
15
15
15
15
15

# Matched
8
13
15
15
14

Percent
53.33
86.67
100.00
100.00
93.33

95 % CI
(26.59, 78.73)
(59.54, 98.34)
(81.90, 100.00)
(81.90, 100.00)
(68.05, 99.83)

# Matched: Appraiser's assessment across trials agrees with the known standard.

Fleiss' Kappa Statistics

Appraiser
Duncan

Hayes

Holmes

Montgomery

Simpson

Response
-2
-1
0
1
2
Overall
-2
-1
0
1
2
Overall
-2
-1
0
1
2
Overall
-2
-1
0
1
2
Overall
-2
-1
0
1
2
Overall

Kappa
0.58333
0.16667
0.44099
0.44099
0.42308
0.41176
0.62963
0.81366
1.00000
0.76000
0.81366
0.82955
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
1.00000
0.81366
0.81366
1.00000
0.91597

SE Kappa
0.258199
0.258199
0.258199
0.258199
0.258199
0.130924
0.258199
0.258199
0.258199
0.258199
0.258199
0.134164
0.258199
0.258199
0.258199
0.258199
0.258199
0.131305
0.258199
0.258199
0.258199
0.258199
0.258199
0.131305
0.258199
0.258199
0.258199
0.258199
0.258199
0.130924

Z
2.25924
0.64550
1.70796
1.70796
1.63857
3.14508
2.43855
3.15131
3.87298
2.94347
3.15131
6.18307
3.87298
3.87298
3.87298
3.87298
3.87298
7.61584
3.87298
3.87298
3.87298
3.87298
3.87298
7.61584
3.87298
3.87298
3.15131
3.15131
3.87298
6.99619

P(vs > 0)
0.0119
0.2593
0.0438
0.0438
0.0507
0.0008
0.0074
0.0008
0.0001
0.0016
0.0008
0.0000
0.0001
0.0001
0.0001
0.0001
0.0001
0.0000
0.0001
0.0001
0.0001
0.0001
0.0001
0.0000
0.0001
0.0001
0.0008
0.0008
0.0001
0.0000

Kendall's Correlation Coefficient

Appraiser
Duncan
Hayes
Holmes
Montgomery
Simpson

Coef
0.89779
0.96014
1.00000
1.00000
0.93258

SE Coef
0.192450
0.192450
0.192450
0.192450
0.192450

Z
4.61554
4.93955
5.14667
5.14667
4.79636

P
0.0000
0.0000
0.0000
0.0000
0.0000

Between Appraisers

Assessment Agreement

2003 Minitab Inc.

25

Measurement Systems Analysis

# Inspected
15

# Matched
6

Percent
40.00

95 % CI
(16.34, 67.71)

# Matched: All appraisers' assessments agree with each other.

Fleiss' Kappa Statistics

Response
-2
-1
0
1
2
Overall

Kappa
0.680398
0.602754
0.707602
0.642479
0.736534
0.672965

SE Kappa
0.0816497
0.0816497
0.0816497
0.0816497
0.0816497
0.0412331

Z
8.3331
7.3822
8.6663
7.8687
9.0207
16.3210

P(vs > 0)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000

Kendall's Coefficient of Concordance

Coef
0.966317

Chi - Sq
67.6422

DF
14

P
0.0000

All Appraisers vs Standard

Assessment Agreement
# Inspected
15

# Matched
6

Percent
40.00

95 % CI
(16.34, 67.71)

# Matched: All appraisers' assessments agree with the known standard.

Fleiss' Kappa Statistics

Response
-2
-1
0
1
2
Overall

Kappa
0.842593
0.796066
0.850932
0.802932
0.847348
0.831455

SE Kappa
0.115470
0.115470
0.115470
0.115470
0.115470
0.058911

Z
7.2971
6.8941
7.3693
6.9536
7.3383
14.1136

P(vs > 0)
0.0000
0.0000
0.0000
0.0000
0.0000
0.0000

Kendall's Correlation Coefficient

Coef
0.958102

26

SE Coef
0.0860663

Z
11.1100

P
0.0000

2003 Minitab Inc.

Measurement Systems Analysis

Graph window output

Interpreting the results

Minitab displays three assessment agreement tables: Each Appraiser vs Standard, Between Appraisers, and All
Appraisers vs Standard. Kappa and Kendall's statistics are included in each of these tables. Overall, these statistics
suggest good agreement. The overall Kendall coefficient between appraisers is 0.966317 (p=0.0000). Kendall's coefficient
for all appraisers vs the standard is 0.958102 (p=0.0000). However, a closer look at the Each Appraiser vs Standard table
indicates that Duncan's and Hayes' assessments match poorly with the standard. Holmes and Montgomery, however,
matched 15 of 15 assessments, for a percentage match of 100%. Note that each percentage matched is associated with
a confidence interval.
The Appraiser vs Standard graph provides a graphical display of the Each Appraiser vs Standard assessment agreement
table. You can easily compare the assessment agreements for the five appraisers.
Based on this data, you conclude that Duncan, Hayes, and Simpson need additional training.
Note

The Within Appraiser table is not displayed because each appraiser conducted a single trial.

Attribute Gage Study (Analytic Method)

Attribute Gage Study (Analytic Method) verview
Attribute gage studies assess the amount of bias and repeatability of a gage when the response is a binary attribute
variable. Unlike a variable gage, an attribute gage does not give information on the quality of a part but only indicates
whether a part is accepted or rejected based on a set of specified standards.
Attribute gage study results depend heavily on the way you select the parts and the number of runs performed on each
part. You should select the parts at nearly equidistant intervals and know the reference value for each selected part. The
number of required runs performed on each part depends on the method you use to test the bias of the gage. Minitab
offers two methods to test whether the bias is zero: AIAG and regression. If you use AIAG (default), you must have
exactly 20 trials per part. The minimum number of trials necessary to use the regression method is 15. However, 20 or
more trials are recommended. See Data Attribute Gage Study (Analytic Method) for more information on the data
requirements.
Minitab calculates the bias and repeatability of the gage using the method described in [1]. Because no actual
measurements from the gage are available to estimate bias and repeatability, Minitab calculated bias and repeatability by
fitting the normal distribution curve using the calculated probabilities of acceptance and the known reference values for all
parts.
Note

Use Attribute Agreement Analysis when you assess the agreement of ratings given by multiple appraisers.

2003 Minitab Inc.

27

Measurement Systems Analysis

Attribute Gage Study (Analytic Method)

Stat > Quality Tools > Gage Study > Attribute Gage Study (Analytic Method)
Attribute gage studies calculate the amount of bias and repeatability of a measurement system when the response is a
binary attribute variable. To obtain good estimates of bias and repeatability, you should strictly follow the rules in [1] to
select parts with known reference values.
Note

Attribute Gage Study (Analytic Method), a method to examine the precision of an attribute measurement
system, is different from Attribute Agreement Analysis, a technique to assess appraisers' agreement. Attribute
Agreement Analysis was previously called Attribute Gage R&R Study in Minitab Release 13.

Dialog box items

Part numbers: Enter the column containing the part names or numbers.
Reference values: Enter the column containing the numeric reference values. You must set up your reference value
column differently for summarized and raw data. See Data Attribute Gage Study (Analytic Method) for more information.
Binary attributes in
Summarized counts: Choose to enter a column containing the summarized counts.
Number of trials: Enter the number of trials.
Attribute label: Enter the attribute label (default is Acceptance).
Raw data: Choose to enter a column containing the raw data.
Attribute for calculation: Enter the attribute for estimating probabilities, which must match one of the two raw data
values.
Tolerance limit for calculation
Lower limit: Enter the lower tolerance limit. You must specify either a lower or upper tolerance limit
Upper limit: Enter the upper tolerance limit. You must specify either a lower or upper tolerance limit
Display the other limit: Enter another limit to display on the Gage Performance Curve (optional).
<Gage Info>
<Options>

Data Attribute Gage Study (Analytic Method)

You must have at least eight parts to conduct an attribute gage study. The smallest part must have zero acceptances, and
the largest part must have the maximum number of possible acceptances. For the AIAG method for testing bias, exactly
six parts must be greater than zero number of acceptances and less than 20--the maximum number of acceptances
allowed in the AIAG situation. For the regression method, you may have more than six parts between the two extreme
reference values.
If you specify the lower tolerance limit, the part with the lowest reference value must have 0 acceptances and the part with
the highest reference must have the maximum number of possible acceptances. For the lower limit, as the reference
values increase, the number of acceptances increases. If you specify the upper tolerance limit, the part with the lowest
reference value must have the maximum number of possible acceptances and the part with the highest reference value
must have 0 acceptances. For the upper limit, as the reference values increase, the number of acceptances decrease.
You can enter the number of trials as a constant or a column of data. When the number of trials is not identical for all
parts, you must enter a column containing the number of trials for each part.
The number of trials for each part must be greater than or equal to 15. If you use the AIAG method (default) to test the
bias of the gage, you must have exactly 20 trials per part.
Minitab accepts either summarized or raw data for attribute gage studies.
Summarized Data
Part
Number

Part Number
Reference

Acceptances

Reference

Response

1.35

Reject

1.35

Reject

1.35

Reject

1.35

1.40

1.45

1.35

Reject

13

...

...

...

1.70

Accept

28

Raw Data

1.50

2003 Minitab Inc.

Measurement Systems Analysis

1.55

15

1.60

18

1.65

19

1.70

20

Structure summarized data so that each row

contains the part name or number, reference value,
and summarized count. Parts can be text or
numbers. Reference values and counts must be
numeric

1.70

Accept

1.70

Accept

1.70

Accept

Structure raw data so that each row contains the part

name or number, reference value, and binary
response. Parts and responses can be text or
numbers. Reference values must be numeric.

To do an attribute gage study (Analytic method)

1

Choose Stat > Quality Tools > Gage Study > Attribute Gage Study (Analytic Method).

In Part numbers, enter the column of part numbers.

In Reference values, enter the column of reference values.

Do one of the following:

When the binary responses are summarized counts, enter the column of counts in Summarized Counts. In
Number of trials, enter the number of trials. In Attribute label, type the attribute label (the default is Acceptance).
When the binary responses are raw data, enter the column of raw data in Raw data. In Attribute for calculation,
type the attribute for calculation, which must match one of the two distinct responses.

In Lower limit, type the lower tolerance limit. In Upper limit, type the upper tolerance limit.

If you like, use any dialog box options, then click OK.

Gage Info
Stat > Quality Tools > Gage Study > choose Gage Run Chart, Gage Linearity and Bias Study, Gage R&R Study
(Crossed), Gage R&R Study (Nested), or Attribute Gage Study (Analytic Method) > Gage Info
Use these fields to fill in the label on the Session window and graphical output. Each field corresponds to a line on the
label. If you do not use these fields, the label will be printed with blank lines.

Dialog box items

Gage name: Type the name of the gage.
Date of study: Type the date.
Reported by: Type the name of the person reporting the study information.
Gage Tolerance: Type the gage tolerance.
Miscellaneous: Type any miscellaneous notes.

Attribute Gage Study (Analytic Method) Options

Stat > Quality Tools > Gage Study > Attribute Gage Study (Analytic Method) > Options
Allows you to determine a method to test the bias of the gage and enter a new title to replace the default title.

Dialog box items

Test bias = 0 using
AIAG Method: Choose to use the AIAG Method.
Regression Method: Choose to use the Regression Method.
Title: Enter a new title to replace the default title on the graphical output.

Example of Attribute Gage Study (Analytic Method)

An automobile manufacturer wants to measure the bias and repeatability of an automated attribute measurement system.
The system has a lower tolerance of 0.020 and an upper tolerance of 0.020. The manufacturer ran 10 parts, which had
reference values at intervals of 0.005 from 0.050 to 0.005, through the gage 20 times.
1

Open the worksheet AUTOGAGE.MTW.

2003 Minitab Inc.

29

Measurement Systems Analysis

2

Choose Stat > Quality Tools > Gage Study > Attribute Gage Study (Analytic Method).

In Part numbers, enter Part number.

In Reference values, enter Reference.

Click Summarized counts and enter Acceptances. In Number of trials, type 20.

Click Lower limit and type -0.020. Click OK.

Graph window output

Interpreting the results

The bias in the attribute gage system is 0.0097955, and the adjusted repeatability is 0.0458060. The test for bias indicates
that bias is significantly different from 0 (t = 6.70123, df=19, p = 0.00), suggesting that bias is present in the attribute
measurement system.

30

2003 Minitab Inc.

Index
A

Gage Run Chart (Stat menu) .................................... 3

Attribute Agreement Analysis...................................... 21

Attribute Agreement Analysis (Stat menu) .............. 21

Kappa ......................................................................... 24

Attribute Gage Study (Analytic Method) ..................... 28

Cohen's kappa ........................................................ 24

Attribute Gage Study (Analytic Method) (Stat menu)

............................................................................. 28

Fleiss' kappa ........................................................... 24

B
Bias ............................................................................... 6
C
Cohen's kappa ............................................................ 24
D
Destructive testing ...................................................... 18
F

Kendall's coefficient of concordance .......................... 24

Kendall's correlation coefficient .................................. 24
L
Linearity ........................................................................ 6
Gage ......................................................................... 6
M
Measurement systems analysis ................................... 1
Attribute Agreement Analysis.................................. 21

Fleiss' kappa ............................................................... 24

Attribute Gage Study (Analytic Method) ................. 28

Gage Linearity and Bias Study ................................. 6

Gage Linearity and Bias Study ..................................... 6

Gage R&R Study (Crossed)...................................... 9

Gage Linearity and Bias Study (Stat menu) .............. 6

Gage R&R Study (Nested)...................................... 18

Gage R&R Study (Crossed).......................................... 9

Gage R&R Study (Crossed) (Stat menu) .................. 9
Gage R&R Study (Nested).......................................... 18
Gage R&R Study (Nested) (Stat menu) .................. 18

Gage Run Chart........................................................ 3

N
Number of distinct categories ....................................... 2

Gage Run Chart............................................................ 3

2003 Minitab Inc.

31