FACTOR ANALYSIS

Factor analysis is a powerful statistical technique used by data professionals in

various business domains to uncover underlying data structures or latent variables.
Factor analysis is sometimes confused with Fact Analysis of Information Risk, or
FAIR, but they are not the same – factor analysis encompasses statistical methods
for data reduction and identifying underlying patterns in various fields, while FAIR is
a specific framework and methodology used for analysing and quantifying
information security and cybersecurity risks.

This article examines factor analysis and its role in business, explores its definitions,
various types, and provides real-world examples to illustrate its applications and
benefits. With a clear understanding of what factor analysis is and how it works,
you’ll be well-equipped to leverage this essential data analysis tool in making
connections in your data for strategic decision-making.

The Importance of Factor Analysis

In data science, factor analysis enables the identification of extra-dimensionality and

hidden patterns in data and can be used to simplify data to select relevant variables
for analysis.

Finding Hidden Patterns and Identifying Extra-Dimensionality

A primary purpose of factor analysis is dataset dimensionality reduction. This is

accomplished by identifying latent variables, known as factors, that explain the
common variance in a set of observed variables.

In essence, it helps data professionals sift through a large amount of data and
extract the key dimensions that underlie the complexity. Factor analysis also allows
data professionals to uncover hidden patterns or relationships within data, revealing
the underlying structure that might not be apparent when looking at individual
variables in isolation.

Simplifying Data and Selecting Variables

Factor analysis simplifies data interpretation. Instead of dealing with a multitude of

variables, researchers can work with a smaller set of factors that capture the
essential information. This simplification aids in creating more concise models and
facilitates clearer communication of research findings.

Data professionals working with large datasets must routinely select a subset of
variables most relevant or representative of the phenomenon under analysis or
investigation. Factor analysis helps in this process by identifying the key variables
that contribute to the factors, which can be used for further analysis.

How Does Factor Analysis Work?

Factor analysis is based on the idea that the observed variables in a dataset can be
represented as linear combinations of a smaller number of unobserved, underlying
factors. These factors are not directly measurable but are inferred from the patterns
of correlations or covariances among the observed variables. Factor analysis
typically consists of several fundamental steps.

1. Data Collection

The first step in factor analysis involves collecting data on a set of variables. These
variables should be related in some way, and it’s assumed that they are influenced
by a smaller number of underlying factors.

2. Covariance/Correlation Matrix

The next step is to compute the correlation matrix (if working with standardized
variables) or covariance matrix (if working with non-standardized variables). These
matrices help quantify the relationships between all pairs of variables, providing a
basis for subsequent factor analysis steps.

Covariance Matrix

A covariance matrix is a mathematical construct that plays a critical role in statistics

and multivariate analysis, particularly in the fields of linear algebra and probability
theory. It provides a concise representation of the relationships between pairs of
variables within a dataset.

Specifically, a covariance matrix is a square matrix in which each entry represents

the covariance between two corresponding variables. Covariance measures how two
variables change together; a positive covariance indicates that they tend to increase
or decrease together, while a negative covariance suggests they move in opposite
directions.

A covariance matrix is symmetric, meaning that the covariance between variable X

and variable Y is the same as the covariance between Y and X. Additionally, the
diagonal entries of the matrix represent the variances of individual variables, as the
covariance of a variable with itself is its variance.

Correlation Matrix

A correlation matrix is a statistical tool used to quantify and represent the

relationships between pairs of variables in a dataset. Unlike the covariance matrix,
which measures the co-variability of variables, a correlation matrix standardizes this
measure to a range between -1 and 1, providing a dimensionless value that indicates
the strength and direction of the linear relationship between variables.

A correlation of 1 indicates a perfect positive linear relationship, while -1 signifies a

perfect negative linear relationship. A correlation of 0 suggests no linear relationship.
The diagonal of the correlation matrix always contains ones because each variable is
perfectly correlated with itself.

Correlation matrices are particularly valuable for identifying and understanding the
degree of association between variables, helping to reveal patterns and
dependencies that might not be immediately apparent in raw data.

3. Factor Extraction

Factor extraction involves identifying the underlying factors that explain the common
variance in the dataset. Various methods are used for factor extraction, including
principal component analysis (PCA) and maximum likelihood estimation (MLE).
These methods seek to identify the linear combinations of variables that capture the
most variance in the data.

PCA

PCA is a dimensionality reduction and data transformation technique used in

statistics, machine learning, and data analysis. Its primary goal is to simplify
complex, high-dimensional data while preserving as much relevant information as
possible.
PCA accomplishes this by identifying and extracting a set of orthogonal axes, known
as principal components, that capture the maximum variance in the data. These
principal components are linear combinations of the original variables and are
ordered in terms of the amount of variance they explain, with the first component
explaining the most variance, the second component explaining the second most,
and so on. By projecting the data onto these principal components, you can reduce
the dimensionality of the data while minimizing information loss.

As a powerful way to condense and simplify data, PCA is an invaluable tool for
improving data interpretation and modelling efficiency, and is widely used for various
purposes, including data visualization, noise reduction, and feature selection. It is
particularly valuable in exploratory data analysis, where it helps researchers uncover
underlying patterns and structures in high-dimensional datasets. In addition to
dimensionality reduction, PCA can also aid in removing multicollinearity among
variables, which is beneficial in regression analysis.

MLE

MLE is a fundamental statistical method used to estimate the parameters of a

statistical model. The core premise behind MLE is to find the parameter values that
maximize the likelihood function, which measures how well the model explains the
observed data. In other words, MLE seeks to identify the parameter values that
make the observed data the most probable under the assumed statistical model.

To perform MLE, one typically starts with a probability distribution or statistical model
that relates the parameters to the observed data. The likelihood function is then
constructed based on this model, and it quantifies the probability of observing the
given data for different parameter values. MLE involves finding the values of the
parameters that maximize this likelihood function.

In practice, this is often achieved through numerical optimization techniques, such as

gradient descent or the Newton-Raphson method. MLE is highly regarded for its
desirable properties, such as asymptotic efficiency and consistency, making it a
widely used and respected method for parameter estimation in statistical modelling
and data analysis.

4. Factor Rotation
Once factors are extracted, they are often rotated to achieve a simpler, more
interpretable factor structure. Rotation methods like Varimax and Promax aim to
make the factors more orthogonal or uncorrelated, which enhances their
interpretability.

Varimax rotation.

Varimax rotation. Source: [Link]

5. Factor Loadings

Factor loadings represent the strength and direction of the relationship between each
variable and the underlying factors. These loadings indicate how much each variable
contributes to a given factor and are used to interpret and label the factors.

6. Interpretation

The final step of factor analysis involves interpreting the factors and assigning
meaning to them. Data professionals examine the factor loadings and consider the
variables that are most strongly associated with each factor. This interpretation is a
critical aspect of factor analysis, as it helps in understanding the latent structure of
the data.
Types of Factor Analysis

Factor analysis comes in several variations, depending on the assumptions and

constraints applied to the analysis. The two primary types are exploratory factor
analysis (EFA) and confirmatory factor analysis (CFA).

Exploratory Factor Analysis (EFA)

EFA is used to explore and uncover the underlying structure of data. It is an open-
ended approach that does not impose any specific structure on the factors. Instead,
it allows the factors to emerge from the data. EFA is often used in the early stages of
research when there is little prior knowledge about the relationships between
variables.

Path diagram of an orthogonal two-factor EFA solution.

Path diagram of an orthogonal two-factor Exploratory Factor Analysis (EFA) solution.

Source: [Link]

Confirmatory Factor Analysis (CFA)

CFA, on the other hand, is a more hypothesis-driven approach—it starts with a
predefined factor structure. The data is then tested to see if it fits that structure. This
type of analysis is used when there is a theoretical model or prior knowledge about
the expected relationships between variables and factors.

Benefits of Factor Analysis

Factor analysis offers several advantages to data professionals working a wide

range of business/enterprise settings:

Data reduction and enhanced interpretability. By reducing the dimensionality of data,

you can more easily analyse and interpret complex datasets. This results in
enhanced data interpretation and explainability – by identifying latent factors, factor
analysis provides a more meaningful interpretation of the relationships among
variables, making it easier to understand complex phenomena.

Multivariable selection and analysis. Factor analysis aids in variable selection by

identifying the most important variables that contribute to the factors. This is
especially valuable when working with large datasets. Crucially, factor analysis is a
form of multivariate analysis, which is essential in use cases that require examining
relationships between multiple variables simultaneously.

Factor Analysis Examples

Organizations can use factor analysis in a wide range of applications to identify

underlying factors or latent variables that explain patterns in data.

Market Research

Market researchers often use factor analysis to identify the key factors that influence
consumer preferences. For example, a survey may collect data on various product
attributes like price, brand reputation, quality, and customer service. Factor analysis
can help determine which factors have the most significant impact on consumers’
product choices. By identifying underlying factors, businesses can tailor their product
development and marketing strategies to meet consumer needs more effectively.

Financial Risk Analysis

Factor analysis is commonly used in finance to analyse and manage financial risk.
By examining various economic indicators, asset returns, and market conditions,
factor analysis helps investors and portfolio managers understand how different
factors contribute to the overall risk and return of an investment portfolio.

Customer Segmentation

Businesses often use factor analysis to identify customer segments based on their
purchasing behavior, preferences, and demographic information. By analysing these
factors, companies can create better targeted marketing strategies and product
offerings.

Employee Engagement

Factor analysis can be used to identify the underlying factors that contribute to
employee engagement and job satisfaction. This information helps businesses
improve workplace conditions and increase employee retention.

Brand Perception

Companies may employ factor analysis to understand how customers perceive their
brand. By analysing factors like brand image, trust, and quality, businesses can
make informed decisions to strengthen their brand and reputation.

Product Quality Controls

In manufacturing, factor analysis can help identify the key factors affecting product
quality. This analysis can lead to process improvements and quality control
measures, ultimately reducing defects and enhancing customer satisfaction.

These examples are just a handful of use cases that demonstrate how factor
analysis can be applied in business. As a versatile statistical tool, it can be adapted
to various data-driven decision-making processes for helping organizations gain
deeper insights and make informed choices.

Factor Analysis vs. FAIR

Factor analysis is different from Fact Analysis of Information Risk, or FAIR. Factor
analysis encompasses statistical methods for data reduction and identifying
underlying patterns in various fields, while FAIR is a specific framework and
methodology used for analysing and quantifying information security and
cybersecurity risks. Unlike traditional factor analysis, which deals strictly with data
patterns, FAIR focuses specifically on information and cyber risk factors to help
organizations prioritize and manage their cybersecurity efforts effectively.

Bottom Line

With factor analysis in their cachet of tools, data professionals and business
researchers have a powerful and battle-tested statistical technique for simplifying
data, identifying latent structures, and understanding complex relationships among
variables. Through these discoveries, organizations can better explain observed
relationships among a set of variables by reducing complex data into a more
manageable form, making it easier to understand, interpret, and draw meaningful
conclusions.
FACTOR ANALYSIS GUIDE WITH AN EXAMPLE
What is Factor Analysis?

Factor analysis uses the correlation structure amongst observed variables to model
a smaller number of unobserved, latent variables known as factors. Researchers use
this statistical method when subject-area knowledge suggests that latent factors
cause observable variables to covary. Use factor analysis to identify the hidden
variables.

Analysts often refer to the observed variables as indicators because they literally
indicate information about the factor. Factor analysis treats these indicators as linear
combinations of the factors in the analysis plus an error. The procedure assesses
how much of the variance each factor explains within the indicators. The idea is that
the latent factors create commonalities in some of the observed variables.

For example, socioeconomic status (SES) is a factor you can’t measure directly.
However, you can assess occupation, income, and education levels. These variables
all relate to socioeconomic status. People with a particular socioeconomic status
tend to have similar values for the observable variables. If the factor (SES) has a
strong relationship with these indicators, then it accounts for a large portion of the
variance in the indicators.

The illustration below demonstrates how the four hidden factors in blue drive the
measurable values in the yellow indicator tags.

Factor analysis illustration.

Researchers frequently use factor analysis in psychology, sociology, marketing, and

machine learning.
Let us dig deeper into the goals of factor analysis, critical methodology choices, and
an example. This guide provides practical advice for performing factor analysis.

Analysis Goals

Factor analysis simplifies a complex dataset by taking a larger number of observed

variables and reducing them to a smaller set of unobserved factors. Anytime you
simplify something, you are trading off exactness with ease of understanding. Ideally,
you obtain a result where the simplification helps you better understand the
underlying reality of the subject area. However, this process involves several
methodological and interpretative judgment calls. Indeed, while the analysis
identifies factors, it is up to the researchers to name them! Consequently, analysts
debate factor analysis results more often than other statistical analyses.

While all factor analysis aims to find latent factors, researchers use it for two primary
goals. They either want to explore and discover the structure within a dataset or
confirm the validity of existing hypotheses and measurement instruments.

Exploratory Factor Analysis (EFA)

Researchers use exploratory factor analysis (EFA) when they do not already have a
good understanding of the factors present in a dataset. In this scenario, they use
factor analysis to find the factors within a dataset containing many variables. Use this
approach before forming hypotheses about the patterns in your dataset. In
exploratory factor analysis, researchers are likely to use statistical output and graphs
to help determine the number of factors to extract.
Exploratory factor analysis is most effective when multiple variables are related to
each factor. During EFA, the researchers must decide how to conduct the analysis
(e.g., number of factors, extraction method, and rotation) because there are no
hypotheses or assessment instruments to guide them. Use the methodology that
makes sense for your research.

For example, researchers can use EFA to create a scale, a set of questions
measuring one factor. Exploratory factor analysis can find the survey items that load
on certain constructs.

Confirmatory Factor Analysis (CFA)

Confirmatory factor analysis (CFA) is a more rigid process than EFA. Using this
method, the researchers seek to confirm existing hypotheses developed by
themselves or others. This process aims to confirm previous ideas, research, and
measurement and assessment instruments. Consequently, the nature of what they
want to verify will impose constraints on the analysis.

Before the factor analysis, the researchers must state their methodology including
extraction method, number of factors, and type of rotation. They base these
decisions on the nature of what they’re confirming. Afterwards, the researchers will
determine whether the model’s goodness-of-fit and pattern of factor loadings match
those predicted by the theory or assessment instruments.

In this vein, confirmatory factor analysis can help assess construct validity. The
underlying constructs are the latent factors, while the items in the assessment
instrument are the indicators. Similarly, it can also evaluate the validity of
measurement systems. Does the tool measure the construct it claims to measure?

For example, researchers might want to confirm factors underlying the items in a
personality inventory. Matching the inventory and its theories will impose
methodological choices on the researchers, such as the number of factors.

We’ll get to an example factor analysis in short order, but first, let’s cover some key
concepts and methodology choices you will need to know for the example.
Factors

In this context, factors are broader concepts or constructs that researchers cannot
measure directly. These deeper factors drive other observable variables.
Consequently, researchers infer the properties of unobserved factors by measuring
variables that correlate with the factor. In this manner, factor analysis lets
researchers identify factors they cannot evaluate directly.

Psychologists frequently use factor analysis because many of their factors are
inherently unobservable because they exist inside the human brain.

For example, depression is a condition inside the mind that researchers can’t directly
observe. However, they can ask questions and make observations about different
behaviors and attitudes. Depression is an invisible driver that affects many outcomes
we can measure. Consequently, people with depression will tend to have more
similar responses to those outcomes than those who are not depressed.

For similar reasons, factor analysis in psychology often identifies and evaluates other
mental characteristics, such as intelligence, perseverance, and self-esteem. The
researchers can see how a set of measurements load on these factors and others.

Method of Factor Extraction

The first methodology choice for factor analysis is the mathematical approach for
extracting the factors from your dataset. The most common choices are maximum
likelihood (ML), principal axis factoring (PAF), and principal components analysis
(PCA).

You should use either ML or PAF most of the time.

Use ML when your data follow a normal distribution. In addition to extracting factor
loadings, it also can perform hypothesis tests, construct confidence intervals, and
calculate goodness-of-fit statistics.

Use PAF when your data violates multivariate normality. PAF doesn’t assume that
your data follow any distribution, so you could use it when they are normally
distributed. However, this method can’t provide all the statistical measures as ML.

PCA is the default method for factor analysis in some statistical software packages,
but it is not a factor extraction method. It is a data reduction technique to find
components. There are technical differences, but in a nutshell, factor analysis aims
to reveal latent factors while PCA is only for data reduction. While calculating the
components, PCA does not assess the underlying commonalities that unobserved
factors cause.

PCA gained popularity because it was a faster algorithm during a time of slower,
more expensive computers. If you’re using PCA for factor analysis, do some
research to be sure it’s the correct method for your study. Learn more about PCA in,
Principal Component Analysis Guide and Example.

There are other methods of factor extraction, but the factor analysis literature has not
strongly shown that any of them are better than maximum likelihood or principal axis
factoring.

Number of Factors to Extract

You need to specify the number of factors to extract from your data except when
using principal component components. The method for determining that number
depends on whether you are performing exploratory or confirmatory factor analysis.

Exploratory Factor Analysis

In EFA, researchers must specify the number of factors to retain. The maximum
number of factors you can extract equals the number of variables in your dataset.
However, you typically want to reduce the number of factors as much as possible
while maximizing the total amount of variance the factors explain.

That is the notion of a parsimonious model in statistics. When adding factors, there
are diminishing returns. At some point, you’ll find that an additional factor does not
substantially increase the explained variance. That’s when adding factors needlessly
complicates the model. Go with the simplest model that explains most of the
variance.

Fortunately, a simple statistical tool known as a scree plot helps you manage this
trade-off.

Use your statistical software to produce a scree plot. Then look for the bend in the
data where the curve flattens. The number of points before the bend is often the
correct number of factors to extract.
The scree plot below relates to the factor analysis example later in this post. The
graph displays the Eigenvalues by the number of factors. Eigenvalues relate to the
amount of explained variance.

Scree plot that helps us decide the number of factors to extract.

The scree plot shows the bend in the curve occurring at factor 6. Consequently, we
need to extract five factors. Those five explain most of the variance. Additional
factors do not explain much more.

Some analysts and software use Eigenvalues > 1 to retain a factor. However,
simulation studies have found that this tends to extract too many factors and that the
scree plot method is better (Costello & Osborne, 2005).

Of course, as you explore your data and evaluate the results, you can use theory
and subject-area knowledge to adjust the number of factors. The factors and their
interpretations must fit the context of your study.

Confirmatory Factor Analysis

In CFA, researchers specify the number of factors to retain using existing theory or
measurement instruments before performing the analysis. For example, if a
measurement instrument purports to assess three constructs, then the factor
analysis should extract three factors and see if the results match theory.

Factor Loadings

In factor analysis, the loadings describe the relationships between the factors and
the observed variables. By evaluating the factor loadings, you can understand the
strength of the relationship between each variable and the factor. Additionally, you
can identify the observed variables corresponding to a specific factor.

Interpret loadings like correlation coefficients. Values range from -1 to +1. The sign
indicates the direction of the relations (positive or negative), while the absolute value
indicates the strength. Stronger relationships have factor loadings closer to -1 and
+1. Weaker relationships are close to zero.

Stronger relationships in the factor analysis context indicate that the factors explain
much of the variance in the observed variables.

Factor Rotations

In factor analysis, the initial set of loadings is only one of an infinite number of
possible solutions that describe the data equally. Unfortunately, the initial answer is
frequently difficult to interpret because each factor can contain middling loadings for
many indicators. That makes it hard to label them. You want to say that particular
variables correlate strongly with a factor while most others do not correlate at all. A
sharp contrast between high and low loadings makes that easier.

Rotating the factors addresses this problem by maximizing and minimizing the entire
set of factor loadings. The goal is to produce a limited number of high loadings and
many low loadings for each factor.

This combination lets you identify the relatively few indicators that strongly correlate
with a factor and the larger number of variables that do not correlate with it. You can
more easily determine what relates to a factor and what does not. This condition is
what statisticians mean by simplifying factor analysis results and making them easier
to interpret.

Graphical illustration

Let me show you how factor rotations work graphically using scatterplots.
Factor analysis starts by calculating the pattern of factor loadings. However, it picks
an arbitrary set of axes by which to report them. Rotating the axes while leaving the
data points unaltered keeps the original model and data pattern in place while
producing more interpretable results.

To make this graphable in two dimensions, we will use two factors represented by
the X and Y axes. On the scatterplot below, the six data points represent the
observed variables, and the X and Y coordinates indicate their loadings for the two
factors. Ideally, the dots fall right on an axis because that shows a high loading for
that factor and a zero loading for the other.

Scatterplot of the initial factor loadings.

For the initial factor analysis solution on the scatterplot, the points contain a mixture
of both X and Y coordinates and aren’t close to a factor’s axis. That makes the
results difficult to interpret because the variables have middling loads on all the
factors. Visually, they are not clumped near axes, making it difficult to assign the
variables to one.

Rotating the axes around the scatterplot increases or decreases the X and Y values
while retaining the original pattern of data points. At the blue rotation on the graph
below, you maximize one factor loading while minimizing the other for all data points.
The result is that the loads are high on one indicator but low on the other.

Scatterplot of rotated loadings in a factor analysis.

On the graph, all data points cluster close to one of the two factors on the blue
rotated axes, making it easy to associate the observed variables with one factor.

Types of Rotations

Throughout these rotations, you work with the same data points and factor analysis
model. The model fits the data for the rotated loadings equally as well as the initial
loadings, but they’re easier to interpret. You’re using a different coordinate system to
gain a different perspective of the same pattern of points.

There are two fundamental types of rotation in factor analysis, oblique and
orthogonal.

Oblique rotations allow correlation amongst the factors, while orthogonal rotations
assume, they are entirely uncorrelated.
Graphically, orthogonal rotations enforce a 90° separation between axes, as shown
in the example above, where the rotated axes form right angles.

Oblique rotations are not required to have axes forming right angles, as shown below
for a different dataset.

Oblique rotation for a factor analysis.

Notice how the freedom for each axis to take any orientation allows them to fit the
data more closely than when enforcing the 90° constraint. Consequently, oblique
rotations can produce simpler structures than orthogonal rotations in some cases.
However, these results can contain correlated factors.

Oblique Rotations Orthogonal Rotations

Promax Varimax

Oblimin Equimax

Direct Quartimin Quartimax

In practice, oblique rotations produce similar results as orthogonal rotations when the
factors are uncorrelated in the real world. However, if you impose an orthogonal
rotation on genuinely correlated factors, it can adversely affect the results. Despite
the benefits of oblique rotations, analysts tend to use orthogonal rotations more
frequently, which might be a mistake in some cases.

When choosing a rotation method in factor analysis, be sure it matches your

underlying assumptions and subject-area knowledge about whether the factors are
correlated.

Factor Analysis Example

Imagine that we are human resources researchers who want to understand the
underlying factors for job candidates. We measured 12 variables and perform factor
analysis to identify the latent factors.

The first step is to determine the number of factors to extract. Earlier in this post, I
displayed the scree plot, which indicated we should extract five factors. If necessary,
we can perform the analysis with a different number of factors later.

For the factor analysis, we will assume normality and use Maximum Likelihood to
extract the factors. I would prefer to use an oblique rotation, but my software only
has orthogonal rotations. So, we will use Varimax. Let’s perform the analysis!

Interpreting the Results

Statistical output for the factor analysis example.

In the bottom right of the output, we see that the five factors account for 81.8% of the
variance. The %Var row along the bottom shows how much of the variance each
explains. The five factors are roughly equal, explaining between 13.5% to 19% of the
variance. Learn about Variance.

The Communality column displays the proportion of the variance the five factors
explain for each variable. Values closer to 1 are better. The five factors explain the
most variance for Resume (0.989) and the least for Appearance (0.643).

In the factor analysis output, the circled loadings show which variables have high
loadings for each factor. As shown in the table below, we can assign labels
encompassing the properties of the highly loading variables for each factor.

Factor Label High Loading Variables

1 Relevant Background Academic record, Potential, Experience

2 Personal Characteristics Confidence, Likeability, Appearance

3 General Work Skills Organization, Communication

4 Writing Skills Letter, Resume

5 Overall Fit Company Fit, Job Fit

In summary, these five factors explain a large proportion of the variance, and we can
devise reasonable labels for each. These five latent factors drive the values of the 12
variables we measured.

WHAT IS THE KAISER-MEYER-OLKIN (KMO) TEST?

The Kaiser-Meyer-Olkin (KMO) Test is a measure of how suited your data is for
Factor Analysis. The test measures sampling adequacy for each variable in the
model and for the complete model. The statistic is a measure of the proportion of
variance among variables that might be common variance. The lower the proportion,
the more suited your data is to Factor Analysis.

KMO returns values between 0 and 1. A rule of thumb for interpreting the statistic:

KMO values between 0.8 and 1 indicate the sampling is adequate.

KMO values less than 0.6 indicate the sampling is not adequate and that remedial
action should be taken. Some authors put this value at 0.5, so use your own
judgment for values between 0.5 and 0.6.

KMO Values close to zero means that there are large partial correlations compared
to the sum of correlations. In other words, there are widespread correlations which
are a large problem for factor analysis.

For reference, Kaiser put the following values on the results:

0.00 to 0.49 unacceptable.

0.50 to 0.59 miserable.

0.60 to 0.69 mediocre.

0.70 to 0.79 middling.

0.80 to 0.89 meritorious.

0.90 to 1.00 marvellous.

Running the Kaiser-Meyer-Olkin (KMO) Test

The formula for the KMO test is:

where:

R = [rij] is the correlation matrix,

U = [uij] is the partial covariance matrix,

Σ = summation notation (“add up”).

In SPSS: Run Factor Analysis (Analyse>Dimension Reduction>Factor) and check

the box for” KMO and Bartlett’s test of sphericity.” If you want the MSA (measure of
sampling adequacy) for individual variables, check the “anti-image” box. An anti-
image box will show with the MSAs listed in the diagonals.

The test can also be run by specifying KMO in the Factor Analysis command. The
KMO statistic is found in the “KMO and Bartlett’s Test” table of the Factor output.

In R: use the command KMO(r), where r is the correlation matrix you want to
analyze. Find more details about the command in R on the Personality-Project
website.

In Stata, use the post estimation command estat kmo.

Kaiser-Meyer-Olkin (Kaiser, 1974) is a statistical test used in factor analysis to

determine if the data is suitable for factor analysis. KMO measures the sampling
adequacy of each observed variables in the model as well as the complete model.
KMO is calculated based on the correlation between the variables. It ranges from 0
to 1 with values closer to 1 suggesting the variables are correlated and the data is
well-suited for factor analysis, otherwise the variables are uncorrelated and there
may not be a common factor influencing them.

The following criteria are used for evaluating KMO:

Above 0.90 Marvellous

0.80 to 0.90 Meritorious

0.7 to 0.80 Average

0.60 to 0.70 Mediocre

0.50 to 0.60 Terrible

Below 0.50 Unacceptable

References

Kaiser, Henry F. 1974. “An Index of Factorial Simplicity.” Psychometrika 39 (1): 31–
36.

KAISER MEYER OLKIN: KMO: TEST: ASSESSING SAMPLE

ADEQUACY: THE KMO TEST IN FACTOR ANALYSIS
1. Introduction to Factor Analysis and the Importance of Sample Adequacy

Factor analysis is a statistical method used to describe variability among observed,

correlated variables in terms of a potentially lower number of unobserved variables,
called factors. It helps researchers to investigate concepts that are not easily
measured directly by condensing a large number of variables into a few interpretable
underlying factors. The effectiveness of factor analysis, however, is heavily
dependent on the adequacy of the sample size. adequate sample size ensures that
the factor structure can be reliably discovered and that the factors are stable and
replicable across different samples.

Sample adequacy is crucial because it determines the feasibility and reliability of the
factor analysis. If the sample size is too small, the factor solution may not represent
the underlying structure accurately. Conversely, an excessively large sample might
lead to the detection of trivial factors. The Kaiser-Meyer-Olkin (KMO) measure of
sampling adequacy is a statistic that indicates the proportion of variance among
variables that might be common variance. The higher the KMO, the more suitable
the data for factor analysis.

Here are some in-depth insights into the importance of sample adequacy in factor
analysis:

1. The Role of the KMO Test: The KMO test evaluates the suitability of data for
factor analysis. It compares the magnitudes of the observed correlation coefficients
to the magnitudes of the partial correlation coefficients. A KMO value closer to 1
suggests that a factor analysis may be useful with your data, while a value closer to
0 suggests the opposite.

2. Determining Sample size: There is no one-size-fits-all answer for the 'adequate'

sample size in factor analysis. However, a commonly cited rule of thumb is a
minimum ratio of 5 respondents per variable. More complex models with more
variables may require a larger sample size.

3. Bartlett’s Test of Sphericity: This test complements the KMO test by checking the
hypothesis that the correlation matrix is an identity matrix, which would indicate that
the variables are unrelated and therefore unsuitable for structure detection.

4. Partial Correlation: In assessing sample adequacy, it's also important to consider

the partial correlation between variables. If the partial correlations are low, it
indicates that the variables share something in common, which can be extracted as
a factor.

5. Factorability of the Correlation Matrix: Before performing factor analysis, it's

essential to assess the factorability of the correlation matrix. Measures like the KMO
and Bartlett’s test help in determining this.

To illustrate these points, let's consider an example. Suppose a researcher is

investigating the concept of 'customer satisfaction' and has collected data on 20
variables related to service quality, product quality, price, and customer support. To
determine if these 20 variables can be reduced to fewer factors, the researcher
would perform a KMO test. If the KMO value is high (e.g., above 0.8), it suggests
that the variables are sufficiently interrelated to proceed with factor analysis. The
researcher can then use factor analysis to identify underlying dimensions of
customer satisfaction, which might reveal factors such as 'Service Excellence',
'Product Value', and 'Support Quality'.

Understanding the importance of sample adequacy through measures like the KMO
test is fundamental in factor analysis. It ensures that the factors derived are
meaningful and representative of the data, providing valuable insights into the
underlying constructs being studied.
2. Understanding the Kaiser-Meyer-Olkin (KMO) Test

The Kaiser-Meyer-Olkin (KMO) Test is a measure of how suited your data is for
Factor Analysis. The test looks at the patterns of correlations among the variables
and compares the magnitude of the observed correlation coefficients to the
magnitude of the partial correlation coefficients. A high value indicates that the sum
of the partial correlations is relatively low, meaning that the shared variance among
variables can be attributed to underlying factors.

Insights from Different Perspectives:

1. Statistical Perspective: From a statistical standpoint, the KMO Test is a way to

quantify the degree of interrelationships among variables. A KMO value closer to 1
suggests that a factor analysis may be useful with your data. If the value is less than
0.5, the results of the factor analysis are likely to be poor.

2. Practical Application: In practice, researchers might use the KMO Test to decide
whether to proceed with factor analysis. For example, if a psychologist has designed
a new questionnaire, they might use the KMO Test to check if the responses to
different questions are sufficiently interrelated to justify using factor analysis to
identify underlying constructs.

3. Educational Viewpoint: Educators might explain the KMO Test as a preliminary

check before diving into the complex world of factor analysis. It's a way to ensure
that the data will yield meaningful results and that the factors extracted will represent
something real and significant.

In-Depth Information:

- Partial Correlation: The KMO Test examines the partial correlation between
variables. If the partial correlation is low, it means that the variables share something
in common, which is what factor analysis seeks to identify.

- Adequacy Values: The KMO values range from 0 to 1. A value of 0.6 is considered
the minimum for a satisfactory factor analysis, with higher values indicating more
suitable data.

- Bartlett's Test of Sphericity: This test often accompanies the KMO Test. It checks
whether the correlation matrix is an identity matrix, which would indicate that the
variables are unrelated and therefore unsuitable for factor analysis.

Examples to Highlight Ideas:

- Example of High KMO Value: Imagine a set of questions on a survey about health
behaviors. If most people who exercise also eat healthily, and those who don't
exercise tend to eat poorly, the KMO Test might return a value close to 1, indicating
these behaviors are suitable for factor analysis.

- Example of Low KMO Value: Conversely, if a survey asks about health behaviors
and political views, the KMO Test might return a low value, suggesting that these
sets of variables do not share common underlying factors and should not be
analyzed together in a factor analysis.

The KMO Test is a vital statistic for researchers conducting factor analysis, as it
informs them about the appropriateness of their data for this type of complex
statistical procedure. It serves as a guardrail, ensuring that the analysis conducted
will be meaningful and the factors identified will be reliable.

3. A Mathematical Overview

The Kaiser-Meyer-Olkin (KMO) Test is a measure of how suited your data is for
Factor Analysis. The test estimates the proportion of variance among all the
observed variable. A higher KMO value indicates that a factor analysis may be useful
with your data. If the value is less than 0.50, the results of the factor analysis
probably won't be very useful.

1. Calculation of the KMO Statistic:

The KMO statistic is a measure of the proportion of variance among variables that
might be common variance. The formula is:

$$ KMO = \frac{\sum_{i \neq j}^{} r_{ij}^2}{\sum_{i \neq j}^{} r_{ij}^2 + \sum_{i \neq
j}^{} u_{ij}^2} $$

Where \( r_{ij} \) is the correlation coefficient between variables \( i \) and \( j \), and \(
u_{ij} \) is the partial correlation coefficient between \( i \) and \( j \).

2. Interpretation of the KMO Values:

- 0.00 to 0.49: Unacceptable.

- 0.50 to 0.59: Miserable.

- 0.60 to 0.69: Mediocre.

- 0.70 to 0.79: Fair.

- 0.80 to 0.89: Great.

- 0.90 to 1.00: Superb.

3. The Anti-Image Correlation Matrix:

This matrix is the negative of the partial correlations, after partialling out all other
variables. It helps in identifying partial correlations that are small, indicating that the
variables share something in common, which is suitable for factor analysis.

4. Example of KMO Test Application:

Imagine a study examining the factors influencing eating habits. The researcher
collects data on snacking frequency, preference for sweet or savory, and time of day
when snacking is most likely. The KMO test will help determine if these variables can
be grouped into underlying factors, such as 'sweet tooth' or 'evening munchies'.

The KMO test is a critical step in the pre-analysis phase of factor analysis, ensuring
that the dataset is suitable for structure detection. It provides a clear indication of the
adequacy of sample size and the appropriateness of factor analysis in the research
study. Understanding the mechanics behind the KMO test allows researchers to
make informed decisions about their data and the analytical strategies they employ.

4. When and Why to Use the KMO Test in Research?

The Kaiser-Meyer-Olkin (KMO) Test plays a pivotal role in research, particularly

when dealing with factor analysis. This measure of sampling adequacy helps
researchers determine the appropriateness of factor analysis for their data.
Essentially, it evaluates whether the partial correlations among variables are small,
which would indicate that the variables share something in common, making factor
analysis suitable. The KMO Test is a statistic that ranges from 0 to 1, with values
closer to 1 suggesting that factor analysis is appropriate. Values below 0.5, however,
are considered unacceptable.

When to Use the KMO Test:

1. Pre-Analysis Stage: Before diving into factor analysis, the KMO Test should be
employed to assess if the dataset is suitable for such a complex procedure.

2. Sparse Data: In cases where the dataset is sparse or has many zero entries, the
KMO Test can help determine if enough non-zero entries exist for a meaningful
analysis.

3. Research Design: When designing a study, the KMO Test can guide the sample
size and variable selection to ensure robust factor analysis results.

Why to Use the KMO Test:

1. Data Suitability: It provides a quick check to see if the data will yield reliable
factors.

2. Resource Efficiency: Using the KMO Test can save time and resources by
preventing futile factor analysis on unsuitable data.

3. Statistical Rigor: It adds a layer of statistical rigor to the research, ensuring that
the factor analysis is grounded in adequate data.

Examples Highlighting the Use of the KMO Test:

- In a study examining psychological scales, the KMO Test might reveal that the
items on a questionnaire are sufficiently interrelated for factor analysis, suggesting
that the scale measures a coherent construct.

- Conversely, in market research, a low KMO value could indicate that customer
preferences are too diverse to be captured by a few underlying factors, thus guiding
the researcher to alternative methods of analysis.

The KMO Test is an indispensable tool in the researcher's toolkit. It serves as a

gatekeeper, ensuring that factor analysis is performed only when the data is primed
to reveal its underlying structure, thereby safeguarding the integrity and validity of
the research findings.

5. What Do the Numbers Mean?

Interpreting the results of a Kaiser-Meyer-Olkin (KMO) Test is a critical step in factor

analysis, as it informs the researcher about the adequacy of the sample data for
conducting this statistical procedure. The KMO Test measures the suitability of data
for factor analysis by comparing the magnitude of the observed correlation
coefficients to the magnitude of the partial correlation coefficients. A higher KMO
value indicates that the sum of partial correlation coefficients is relatively low,
meaning that the shared variance among variables can be attributed to underlying
factors. Conversely, a lower KMO value suggests that the shared variance is more
likely due to the noise and thus, factor analysis may not be appropriate.

From different perspectives, the KMO Test results can be interpreted as follows:

1. Statistical Perspective: The KMO value ranges from 0 to 1. A value closer to 1

indicates that factor analysis is likely to be reliable. Values between 0.5 to 0.7 are
considered mediocre, values between 0.7 to 0.8 are good, values between 0.8 to 0.9
are great, and values above 0.9 are superb. For example, a KMO value of 0.85
suggests that the majority of the variance in the original variables can be explained
by the underlying factors, which is ideal for factor analysis.

2. Practical Perspective: In practice, a KMO value below 0.5 often leads researchers
to either collect more data, reconsider the set of variables included in the analysis, or
explore alternative statistical methods. For instance, if a study on consumer behavior
yields a KMO value of 0.45, the researcher might need to reassess the survey
questions to ensure they are capturing the constructs effectively.

3. Interdisciplinary Perspective: Different fields may have varying thresholds for

acceptable KMO values. In psychology, where complex constructs are common, a
slightly lower KMO might still be acceptable, while in fields like finance or medicine,
higher thresholds are often expected due to the need for precise and conclusive
results.

4. Methodological Perspective: The KMO Test also helps in determining the number
of factors to extract. If the KMO value is high, it supports the extraction of multiple
factors. Conversely, a low KMO value might suggest that a single factor or just a few
factors should be extracted.

5. Educational Perspective: When teaching factor analysis, the KMO Test is used to
illustrate the importance of sample adequacy. Educators often use simulated
datasets to show how varying KMO values affect the factor analysis outcome.

To highlight the importance of the KMO value with an example, consider a

psychological study aiming to identify factors that influence academic success. If the
KMO test yields a value of 0.6, this might indicate that some variables do not
strongly relate to any factor, or that there is not enough data to clearly define the
factors. The researcher might then decide to conduct further studies to refine the
variables or increase the sample size.

In summary, the KMO Test serves as a gatekeeper in factor analysis, ensuring that
the data is primed for revealing the underlying structure. Interpreting its results
requires a nuanced understanding of both the statistical principles at play and the
practical implications for the research at hand. The KMO value is not just a number;
it's a guidepost for the analytical journey ahead.

6. Step-by-Step Guide to Performing the KMO Test

Guide to performing

The Kaiser-Meyer-Olkin (KMO) Test is a measure of how suited your data is for
Factor Analysis. The test looks at the patterns of correlations among the variables
and compares the magnitude of the observed correlation coefficients to the
magnitude of the partial correlation coefficients. A high value indicates that the factor
model underlying the variables is well-defined, while a low value suggests that factor
analysis may not be appropriate.

Insights from Different Perspectives:

- Statistical Perspective: Statisticians value the KMO test as it helps in determining
the adequacy of each variable and the complete model. It is a way to quantify the
degree of interrelationships among the variables.

- Practical Application: For researchers, the KMO test is a preliminary check that can
save time and resources by indicating whether a dataset is likely to yield meaningful
factors.

- Educational Viewpoint: Educators might use the KMO test to teach about the
importance of understanding the structure of data before proceeding with complex
analyses.

step-by-Step guide:

1. Collect Data: Ensure that you have collected data on all the variables you wish to
analyse. The data should be continuous and preferably normally distributed.

2. Input Data: Enter your data into a statistical software package that can perform
factor analysis and the KMO test.

3. Compute Correlation Matrix: The software will compute a correlation matrix which
is the foundation for the KMO test.

4. Calculate Anti-Image correlation matrix: This matrix is essential as it contains

partial correlations. The KMO test uses this to check sampling adequacy.

5. Determine KMO Values: The software will provide you with KMO values for
individual variables (KMOi) and the overall model (KMOt). Values closer to 1 indicate
adequacy.

6. Interpret Results: A KMOt value of 0.6 or above is considered acceptable. Values

between 0.5 and 0.6 are mediocre, below 0.5 are unacceptable.

Example to Highlight an Idea:

Imagine you're researching education techniques and have data on student

satisfaction, grades, and attendance. You want to know if these can be reduced to
underlying factors like 'engagement' or 'academic rigor'. The KMO test will tell you if
your data is suitable for such an analysis. If the KMO value is high, you can proceed
with confidence that factor analysis is appropriate for your data set.
The KMO test is a valuable tool in the pre-analysis phase of factor analysis,
providing a quantitative measure to assess the suitability of data for structure
detection. It guides researchers in making informed decisions about their data
analysis strategies.

7. Common Misconceptions and Pitfalls in Using the KMO Test

Misconceptions and Pitfalls

When it comes to assessing the adequacy of a sample for factor analysis, the
Kaiser-Meyer-Olkin (KMO) Test is a widely recognized measure. However, despite
its popularity, there are several common misconceptions and pitfalls associated with
its use that can lead to misinterpretation of results or inappropriate application of the
test. Understanding these pitfalls is crucial for researchers to ensure that their factor
analysis is both valid and reliable. The KMO Test, which yields values between 0
and 1, is often misinterpreted as a definitive indicator of sample adequacy, but in
reality, it should be used as a guideline rather than a strict rule.

Misconceptions and Pitfalls:

1. Assuming a High KMO Guarantees Factorability:

While a high KMO value (above 0.8) suggests that a dataset is likely suitable for
factor analysis, it does not guarantee that the identified factors will be meaningful or
interpretable. For example, a dataset with complex, multi-dimensional constructs
may yield a high KMO but still produce factors that are difficult to interpret.

2. Overlooking the Importance of Individual Item-Level Measures of Sampling

Adequacy (MSA):

The overall KMO test is an average of the MSA for each variable. Researchers often
overlook the individual MSA values, which can indicate if specific variables are not
suitable for factor analysis. A variable with an MSA value below 0.5 might need to be
excluded, even if the overall KMO is acceptable.

3. Ignoring the impact of Sample size:

The KMO value can be influenced by the sample size. Larger samples can inflate the
KMO value, giving a false impression of adequacy. Conversely, small sample sizes
can lead to lower KMO values, potentially discouraging researchers from proceeding
with factor analysis when it might still be appropriate.

4. Misinterpreting KMO Values in the 'Mediocre' Range:

KMO values between 0.5 and 0.7 are often labelled as 'mediocre,' leading some
researchers to abandon their analysis. However, these values do not necessarily
preclude factor analysis; they simply suggest that the results should be interpreted
with caution.

5. Neglecting the Role of Correlations Among Variables:

The KMO test assesses the proportion of variance among variables that might be
common variance. A dataset with low correlations between variables can result in a
low KMO, indicating that factor analysis may not be suitable. However, this does not
consider the possibility of latent structures that could emerge with a more nuanced
approach.

6. Confusing KMO with Other Adequacy Tests:

The KMO test is sometimes confused with Bartlett's Test of Sphericity, which
assesses whether the correlation matrix is an identity matrix, implying that variables
are unrelated. While both tests are used in the preliminary stages of factor analysis,
they serve different purposes and should not be used interchangeably.
Examples to Highlight Ideas:

- Example of Misinterpreting 'Mediocre' KMO Values:

Consider a psychological study measuring various aspects of well-being. The KMO

value for the dataset is 0.6. While this is in the 'mediocre' range, the researcher
decides to proceed with factor analysis, resulting in a meaningful factor structure that
aligns with theoretical expectations.

- Example of Overlooking Individual MSA Values:

In a survey measuring customer satisfaction, one item related to 'speed of service'

has an MSA value of 0.4, while the overall KMO is 0.82. Ignoring the low MSA value
and including this item in the analysis could lead to a distorted factor structure.

By being aware of these misconceptions and pitfalls, researchers can more

effectively use the KMO Test as part of their factor analysis, ensuring that their
findings are robust and their interpretations are sound. It's important to approach the
KMO test with a critical eye and consider it within the broader context of the research
design and objectives.

8. The KMO Test in Action

The Kaiser-Meyer-Olkin (KMO) Test is a measure of how suited your data is for
Factor Analysis. The test estimates the proportion of variance among all the
observed variable. A higher KMO value indicates that a factor analysis may be useful
with your data. If the value is less than 0.50, the results of the factor analysis
probably won't be very useful.

Case studies provide a practical lens through which we can examine the efficacy of
the KMO Test in real-world scenarios. These studies not only illustrate the test's
utility in determining sample adequacy but also highlight the diversity of situations in
which the KMO Test can be applied.

1. Educational Research: In a study examining the factors influencing academic

success among high school students, researchers utilized the KMO Test to ensure
their variables—such as study habits, family support, and socio-economic status—
were suitable for factor analysis. The KMO value of 0.82 indicated a strong
correlation pattern, justifying the subsequent extraction of factors.
2. Marketing Analysis: A consumer behavior survey collected data on customer
preferences and buying habits. The KMO Test yielded a value of 0.65, suggesting a
moderate amount of common variance. This led to a successful factor analysis,
revealing underlying patterns in consumer choices.

3. Psychology: When exploring the dimensions of mental well-being, psychologists

applied the KMO Test to their questionnaire data. The result was a KMO value of
0.78, which supported the factor analysis that uncovered factors related to stress,
happiness, and social relationships.

4. Healthcare Studies: In a healthcare setting, researchers investigating the factors

affecting patient satisfaction used the KMO Test. A KMO value of 0.60 was obtained,
indicating an acceptable level of common variance and leading to a factor analysis
that identified key areas for improvement in patient care.

Through these examples, we see the KMO Test's versatility across different fields,
proving its worth as a preliminary step in the factor analysis process. It serves as a
critical checkpoint, ensuring that the data is primed for revealing the underlying
structure that factor analysis seeks to uncover.

9. Ensuring Robustness in Factor Analysis

Factor Analysis
While the Kaiser-Meyer-Olkin (KMO) test is a pivotal measure for assessing the
suitability of data for factor analysis, it's crucial to recognize that it is just the
beginning of ensuring robustness in your factor analysis. The KMO test evaluates
sampling adequacy, which can prevent you from proceeding with a factor analysis on
datasets that are unlikely to yield meaningful results. However, passing the KMO test
doesn't guarantee that your factor analysis will be free from issues. It's akin to
passing the first checkpoint in a quality control process; there are several more
stages to clear before you can be confident in the robustness of your findings.

Ensuring robustness in factor analysis involves a multi-faceted approach that

considers the intricacies of the dataset and the factor analysis method employed.
Here are some key considerations:

1. Communality Estimates: After the KMO test, it's essential to examine the initial
communality estimates. These estimates reflect the amount of variance in each
variable that is accounted for by the factor solution. Low communalities may indicate
that the variable does not fit well with the factor model.

2. Factor Extraction Methods: There are several methods for extracting factors, such
as principal Component analysis (PCA) and Maximum Likelihood (ML). Each method
has its assumptions and is suitable for different types of data. It's important to
choose a method that aligns with your data characteristics and research goals.

3. Number of Factors: Determining the number of factors to retain can be subjective.

Criteria like the Kaiser criterion, scree plot, and parallel analysis can provide
guidance, but they should be used in conjunction with theoretical considerations and
not in isolation.

4. Rotation Methods: Once factors are extracted; rotation can help in achieving a
simpler and more interpretable structure. Varimax rotation is commonly used for
orthogonal rotation, while Promax is an option for oblique rotation. The choice
depends on whether you expect factors to be correlated or not.

5. Factor Scores: If you plan to use factor scores in further analyses, it's important to
consider how they are computed. Regression, Bartlett, and Anderson-Rubin are
some methods for calculating factor scores, each with its own implications for
subsequent analyses.
6. Replicability: A robust factor analysis should yield similar results across different
samples or subsets of data. Cross-validation can help in assessing the stability and
generalizability of the factor solution.

7. Sensitivity Analysis: Conducting sensitivity analyses by varying the assumptions

and parameters of your factor analysis can provide insights into the stability of your
results.

For example, let's consider a psychological study aiming to identify factors of

resilience among adolescents. The KMO test yields a satisfactory result, indicating
that factor analysis is appropriate. However, upon extracting factors using PCA, the
researchers find that the two-factor solution suggested by the scree plot doesn't align
well with their theoretical framework, which posits three dimensions of resilience.
After re-evaluating their approach and employing ML extraction with Promax rotation,
they identify three factors that correspond to their theoretical expectations and
demonstrate good internal consistency and stability across different samples.

While the KMO test is a valuable tool in the preliminary assessment of data for factor
analysis, it's just one piece of the puzzle. A comprehensive approach that considers
all aspects of factor analysis, from extraction to interpretation, is necessary to ensure
the robustness and reliability of the results. This holistic view, combined with a
critical assessment of both statistical outputs and theoretical underpinnings, will lead
to more meaningful and actionable insights from your factor analysis.