This PDF is available at http://nap.nationalacademies.

org/27993

AI Applications for Automatic Pavement

Condition Evaluation (2024)

DETAILS
84 pages | 8.5 x 11 | PAPERBACK
ISBN 978-0-309-72767-9 | DOI 10.17226/27993

CONTRIBUTORS
Linda M. Pierce, Sarah E. Lopez, Jose R. Medina, Vivek Jha; National Cooperative
Highway Research Program; Transportation Research Board; National Academies of
BUY THIS BOOK Sciences, Engineering, and Medicine

FIND RELATED TITLES SUGGESTED CITATION

National Academies of Sciences, Engineering, and Medicine. 2024. AI
Applications for Automatic Pavement Condition Evaluation. Washington, DC: The
National Academies Press. https://doi.org/10.17226/27993.

Visit the National Academies Press at nap.edu and login or register to get:
– Access to free PDF downloads of thousands of publications
– 10% off the price of print publications
– Email or social media notifications of new titles related to your interests
– Special offers and discounts

All downloadable National Academies titles are free to be used for personal and/or non-commercial
academic use. Users may also freely post links to our titles on this website; non-commercial academic
users are encouraged to link to the version on this website rather than distribute a downloaded PDF
to ensure that all users are accessing the latest authoritative version of the work. All other uses require
written permission. (Request Permission)

This PDF is protected by copyright and owned by the National Academy of Sciences; unless otherwise
indicated, the National Academy of Sciences retains copyright to all materials in this PDF with all rights
reserved.
AI Applications for Automatic Pavement Condition Evaluation

NCHRP
Synthesis 636
National
Cooperative
Highway
A SYNTHESIS OF HIGHWAY PRACTICE Research Program

AI Applications for Automatic

Pavement Condition Evaluation

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

TRANSPORTATION RESEARCH BOARD 2024 EXECUTIVE COMMITTEE*

OFFICERS
Chair: Carol A. Lewis, Professor, Transportation Studies, Texas Southern University, Houston
Vice Chair: Leslie S. Richards, General Manager, Southeastern Pennsylvania Transportation Authority (SEPTA), Philadelphia
Executive Director: Victoria Sheehan, Transportation Research Board, Washington, DC

MEMBERS
Michael F. Ableson, CEO, Arrival Automotive–North America, Detroit, MI
James F. Albaugh, President and CEO, The Boeing Company (retired), Scottsdale, AZ
Carlos M. Braceras, Executive Director, Utah Department of Transportation, Salt Lake City
Douglas C. Ceva, Vice President, Customer Lead Solutions, Prologis, Inc., Jupiter, FL
Nancy Daubenberger, Commissioner of Transportation, Minnesota Department of Transportation, St. Paul
Marie Therese Dominguez, Commissioner, New York State Department of Transportation, Albany
Garrett Eucalitto, Commissioner, Connecticut Department of Transportation, Newington
Chris T. Hendrickson, Hamerschlag University Professor of Engineering Emeritus, Carnegie Mellon University, Pittsburgh, PA
Randell Iwasaki, President and CEO, Iwasaki Consulting Services, Walnut Creek, CA
Ashby Johnson, Executive Director, Capital Area Metropolitan Planning Organization (CAMPO), Austin, TX
Joel M. Jundt, Secretary of Transportation, South Dakota Department of Transportation, Pierre
Hani S. Mahmassani, W.A. Patterson Distinguished Chair in Transportation; Director, Transportation Center, Northwestern University,
Evanston, IL
Scott C. Marler, Director, Iowa Department of Transportation, Ames
Ricardo Martinez, Adjunct Professor of Emergency Medicine, Emory University School of Medicine, Decatur, GA
Michael R. McClellan, Vice President, Strategic Planning, Norfolk Southern Corporation, Atlanta, GA
Russell McMurry, Commissioner, Georgia Department of Transportation, Atlanta
Craig E. Philip, Research Professor and Director, VECTOR, Department of Civil and Environmental Engineering, Vanderbilt University,
Nashville, TN
Steward T.A. Pickett, Distinguished Senior Scientist, Cary Institute of Ecosystem Studies, Millbrook, NY
Susan A. Shaheen, Professor and Co-director, Transportation Sustainability Research Center, University of California, Berkeley
Marc Williams, Executive Director, Texas Department of Transportation, Austin

EX OFFICIO MEMBERS
Michael R. Berube, Deputy Assistant Secretary for Sustainable Transportation, U.S. Department of Energy, Washington, DC
Shailen Bhatt, Administrator, Federal Highway Administration, U.S. Department of Transportation, Washington, DC
Amit Bose, Administrator, Federal Railroad Administration, Washington, DC
Tristan Brown, Deputy Administrator, Pipeline and Hazardous Materials Safety Administration, U.S. Department of Transportation,
Washington, DC
Steven Cliff, Executive Officer, California Air Resources Board, Sacramento
Rand Ghayad, Senior Vice President, Association of American Railroads, Washington, DC
LeRoy Gishi, Chief, Division of Transportation, Bureau of Indian Affairs, U.S. Department of the Interior, Germantown, MD
William H. Graham, Jr. (Major General, U.S. Army), Deputy Commanding General for Civil and Emergency Operations, U.S. Army Corps
of Engineers, Washington, DC
Robert C. Hampshire, Deputy Assistant Secretary for Research and Technology, U.S. Department of Transportation, Washington, DC
Zahra “Niloo” Parvinashtiani, Engineer, Mobility Consultant Solutions, Iteris Inc., Fairfax, VA, and Chair, TRB Young Members
Coordinating Council
Sophie Shulman, Deputy Administrator, National Highway Traffic Safety Administration, Washington, DC
Karl Simon, Director, Transportation and Climate Division, U.S. Environmental Protection Agency, Washington, DC
Paul P. Skoutelas, President and CEO, American Public Transportation Association, Washington, DC
Polly Trottenberg, Deputy Secretary of Transportation, U.S. Department of Transportation, Washington, DC
Jim Tymon, Executive Director, American Association of State Highway and Transportation Officials, Washington, DC
Veronica Vanterpool, Acting Administrator, Federal Transit Administration, Washington, DC
Michael Whitaker, Administrator, Federal Aviation Administration, U.S. Department of Transportation, Washington, DC
Vinn White, Deputy Administrator, Federal Motor Carrier Safety Administration, Washington, DC

* Membership as of August 2024.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

N AT I O N A L C O O P E R AT I V E H I G H W AY R E S E A R C H P R O G R A M

NCHRP SYNTHESIS 636

AI Applications for Automatic
Pavement Condition Evaluation
A Synthesis of Highway Practice

Linda M. Pierce
NCE
Spokane, WA

Sarah E. Lopez
NCE
Reno, NV

Jose R. Medina
NCE
Phoenix, AZ

Vivek Jha
NCE
Fountain Valley, CA

Subscriber Categories
Data and Information Technology • Design • Pavements

Research sponsored by the American Association of State Highway and Transportation Officials
in cooperation with the Federal Highway Administration

2024

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

NATIONAL COOPERATIVE HIGHWAY NCHRP SYNTHESIS 636

RESEARCH PROGRAM
Systematic, well-designed, and implementable research is the most Project 20-05, Topic 54-14
effective way to solve many problems facing state departments of ISSN 0547-5570
transportation (DOTs) administrators and engineers. Often, highway ISBN 978-0-309-72767-9
problems are of local or regional interest and can best be studied by Library of Congress Control Number 2024946997
state DOTs individually or in cooperation with their state universities
© 2024 by the National Academy of Sciences. National Academies of
and others. However, the accelerating growth of highway transporta-
Sciences, Engineering, and Medicine and the graphical logo are trade-
tion results in increasingly complex problems of wide interest to high-
marks of the National Academy of Sciences. All rights reserved.
way authorities. These problems are best studied through a coordinated
program of cooperative research.
Recognizing this need, the leadership of the American Association
of State Highway and Transportation Officials (AASHTO) in 1962 ini- COPYRIGHT INFORMATION
tiated an objective national highway research program using modern Authors herein are responsible for the authenticity of their materials and for obtaining
scientific techniques—the National Cooperative Highway Research written permissions from publishers or persons who own the copyright to any previously
Program (NCHRP). NCHRP is supported on a continuing basis by published or copyrighted material used herein.
funds from participating member states of AASHTO and receives the Cooperative Research Programs (CRP) grants permission to reproduce material in this
full cooperation and support of the Federal Highway Administration publication for classroom and not-for-profit purposes. Permission is given with the
(FHWA), United States Department of Transportation, under Agree- understanding that none of the material will be used to imply TRB, AASHTO, APTA, FAA,
FHWA, FTA, GHSA, or NHTSA endorsement of a particular product, method, or practice.
ment No. 693JJ31950003.
It is expected that those reproducing the material in this document for educational and
The Transportation Research Board (TRB) of the National Academies not-for-profit uses will give appropriate acknowledgment of the source of any reprinted or
of Sciences, Engineering, and Medicine was requested by AASHTO to reproduced material. For other uses of the material, request permission from CRP.
administer the research program because of TRB’s recognized objectivity
and understanding of modern research practices. TRB is uniquely suited Cover photo credit: A digital image of surface distress captured by a dashboard-mounted
smartphone. Image courtesy of RoadBotics by Michelin.
for this purpose for many reasons: TRB maintains an extensive com-
mittee structure from which authorities on any highway transportation
subject may be drawn; TRB possesses avenues of communications and
cooperation with federal, state, and local governmental agencies, univer- NOTICE
sities, and industry; TRB’s relationship to the National Academies is an The report was reviewed by the technical panel and accepted for publication according to
insurance of objectivity; and TRB maintains a full-time staff of special- procedures established and overseen by the Transportation Research Board and approved
ists in highway transportation matters to bring the findings of research by the National Academies of Sciences, Engineering, and Medicine.
directly to those in a position to use them. The opinions and conclusions expressed or implied in this report are those of the
The program is developed on the basis of research needs iden- researchers who performed the research and are not necessarily those of the Transportation
tified by chief administrators and other staff of the highway and Research Board; the National Academies of Sciences, Engineering, and Medicine; the
FHWA; or the program sponsors.
transportation departments, by committees of AASHTO, and by
the FHWA. Topics of the highest merit are selected by the AASHTO The Transportation Research Board does not develop, issue, or publish standards or spec-
Special Committee on Research and Innovation (R&I), and each year ifications. The Transportation Research Board manages applied research projects which
provide the scientific foundation that may be used by Transportation Research Board
R&I’s recommendations are proposed to the AASHTO Board of Direc- sponsors, industry associations, or other organizations as the basis for revised practices,
tors and the National Academies. Research projects to address these procedures, or specifications.
topics are defined by NCHRP, and qualified research agencies are
The Transportation Research Board; the National Academies of Sciences, Engineering, and
selected from submitted proposals. Administration and surveillance of Medicine; and the sponsors of the National Cooperative Highway Research Program do not
research contracts are the responsibilities of the National Academies endorse products or manufacturers. Trade or manufacturers’ names or logos appear herein
and TRB. solely because they are considered essential to the object of the report.
The needs for highway research are many, and NCHRP can make
significant contributions to solving highway transportation problems
of mutual concern to many responsible groups. The program, however,
is intended to complement, rather than to substitute for or duplicate,
other highway research programs.

Published reports of the

NATIONAL COOPERATIVE HIGHWAY RESEARCH PROGRAM

are available from

National Academies Press

500 Fifth Street, NW, Keck 360
Washington, DC 20001

(800) 624-6242

and can be ordered through the Internet by going to

https://nap.nationalacademies.org
Printed in the United States of America

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

The National Academy of Sciences was established in 1863 by an Act of Congress, signed by
President Lincoln, as a private, nongovernmental institution to advise the nation on issues
related to science and technology. Members are elected by their peers for outstanding
contributions to research. Dr. Marcia McNutt is president.

The National Academy of Engineering was established in 1964 under the charter of the
National Academy of Sciences to bring the practices of engineering to advising the nation.
Members are elected by their peers for extraordinary contributions to engineering. Dr. John L.
Anderson is president.

The National Academy of Medicine (formerly the Institute of Medicine) was established in
1970 under the charter of the National Academy of Sciences to advise the nation on medical and
health issues. Members are elected by their peers for distinguished contributions to medicine
and health. Dr. Victor J. Dzau is president.

The three Academies work together as the National Academies of Sciences, Engineering,
and Medicine to provide independent, objective analysis and advice to the nation and conduct
other activities to solve complex problems and inform public policy decisions. The National
Academies also encourage education and research, recognize outstanding contributions to
knowledge, and increase public understanding in matters of science, engineering, and medicine.

Learn more about the National Academies of Sciences, Engineering, and Medicine at
www.nationalacademies.org.

The Transportation Research Board is one of seven major program divisions of the
National Academies of Sciences, Engineering, and Medicine. The mission of the Transportation
Research Board is to mobilize expertise, experience, and knowledge to anticipate and solve
complex transportation-related challenges. The Board’s varied activities annually engage about
8,500 engineers, scientists, and other transportation researchers and practitioners from the
public and private sectors and academia, all of whom contribute their expertise in the public
interest. The program is supported by state departments of transportation, federal agencies
including the component administrations of the U.S. Department of Transportation, and other
organizations and individuals interested in the development of transportation.

Learn more about the Transportation Research Board at www.TRB.org.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

COOPERATIVE RESEARCH PROGRAMS

CRP STAFF FOR NCHRP SYNTHESIS 636

Monique R. Evans, Director, Cooperative Research Programs
Waseem Dekelbab, Deputy Director, Cooperative Research Programs, and Manager, National Cooperative
Highway Research Program
Edward Harrigan, Senior Program Officer
Daniel J. Magnolia, Senior Program Assistant
Natalie Barnes, Director of Publications
Heather DiAngelis, Associate Director of Publications
Dominique Williams, Editor

NCHRP PROJECT 20-05 PANEL

Joyce N. Taylor, Maine Department of Transportation, Augusta, ME (Chair)
Anita K. Bush, Nevada Department of Transportation, Carson City, NV
Joseph D. Crabtree, Kentucky Transportation Center, Lexington, KY
Mostafa Jamshidi, Nebraska Department of Transportation, Lincoln, NE
Jessie X. Jones, Arkansas Department of Transportation, Little Rock, AR
Raymond J. Khoury, Virginia Department of Transportation, Richmond, VA
Brenda Moore, North Carolina Department of Transportation (retired), Cary, NC
Jesus Alberto Sandoval-Gil, Arizona Department of Transportation, Phoenix, AZ
Cynthia J. Smith, Mississippi Department of Transportation, Jackson, MS
Jean M. Wallace, Minnesota Department of Transportation, St. Paul, MN
Mary Huie, FHWA Liaison
Jim McDonnell, AASHTO Liaison

TOPIC 54-14 PANEL

Edgardo D. Block, Connecticut Department of Transportation, Newington, CT
Seonghwan Jacob Cho, Indiana Department of Transportation, West Lafayette, IN
Jenny Li, Texas Department of Transportation, Austin, TX
Charlie Pan, Nevada Department of Transportation, Carson City, NV
Christian A. Sabillon-Orellana, University of Texas, Austin, Austin, TX
Maryam Sakhaeifar, FHWA Liaison
Brian Roberts, TRB Liaison

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

ABOUT THE NCHRP SYNTHESIS PROGRAM

Highway administrators, engineers, and researchers often face problems for which information
already exists, either in documented form or as undocumented experience and practice. This infor-
mation may be fragmented, scattered, and unevaluated. As a consequence, full knowledge of what has
been learned about a problem may not be brought to bear on its solution. Costly research findings
may go unused, valuable experience may be overlooked, and due consideration may not be given to
recommended practices for solving or alleviating the problem.
There is information on nearly every subject of concern to highway administrators and engineers.
Much of it derives from research or from the work of practitioners faced with problems in their day-
to-day work. To provide a systematic means for assembling and evalu­ating such useful information
and to make it available to the entire highway community, the American Association of State High-
way and Transportation Officials—through the mechanism of the National Cooperative Highway
Research Program—authorized the Transportation Research Board to undertake a continuing study.
This study, NCHRP Project 20-05, “Synthesis of Information Related to Highway Practices,” searches
out and synthesizes useful knowledge from all available sources and prepares concise, documented
reports on specific topics. Reports from this endeavor constitute an NCHRP report series, Synthesis
of Highway Practice.
This synthesis series reports on current knowledge and practice, in a compact format, without the
detailed directions usually found in handbooks or design manuals. Each report in the series provides
a compendium of the best knowledge available on those measures found to be the most successful
in resolving specific problems.

FOREWORD
By Edward Harrigan
Staff Officer
Transportation Research Board

3D laser-based pavement imaging systems have been widely adopted by state departments of trans-
portation (DOTs) in the last decade for automated pavement condition survey (APCS) assessment;
2D imaging technologies and smartphones are also used to perform pavement condition evaluations,
especially for local transportation agencies. Collected pavement images are then used to semi- or
fully automatically extract pavement distresses through various methods. Among these methods,
models based on artificial intelligence (AI) with machine learning and deep learning (ML/DL) have
gained much attention for pavement distress identification in the last several years. However, most
AI models either are not yet fully integrated with how state DOTs use the pavement distress data or
have not been sufficiently developed to employ quality 3D pavement image data.
The collected distresses, such as cracking, faulting, flushing, and raveling, are key indicators of
triggering pavement maintenance and rehabilitation activities. Without a clear understanding of state
DOTs’ ultimate use of this distress data, AI model development efforts for distress detection and/or
classification, which include AI model formulation, distress annotation, training, and performance
evaluation, could be misguided and fail to reach their full potential. For example, the AI-based models
for automated crack detection using the classification of image blocks with cracking distress may not
be able to output accurate cracking length and width information. Therefore, the outcome produced
by the model may not meet the state DOT’s need for project-level applications, such as planning crack
sealing projects.
Alternatively, the performance of supervised learning AI models for automated pavement distress
extraction relies heavily on several factors, including the quality of the pavement image data used,
data size and diversity, the annotation quality (labeled ground truth distresses), the model formula-
tion, model training, and so forth. However, the performance evaluation method used for many
developed models is not always clear, especially for the diversity of the data used for that evaluation

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

and its established ground truth. This ambiguity makes comparing the performance of different models
challenging and unreliable.
The objective of this synthesis is to document current state DOT practices for both automated
pavement distress identification and AI (ML/DL) technologies for pavement condition evaluation.
Information for this study was gathered through a literature review, a survey of state DOTs, and
follow-up interviews with selected DOTs. However, the relatively recent development and implemen-
tation of 3D technology and the use of AI for APCS analysis resulted in difficulties identifying any
state DOT that can to provide details and specifics for case example development. Therefore, in lieu
of case examples, the report provides a general summary of efforts made for AI model development
and training.
Linda M. Pierce, Sarah E. Lopez, Jose R. Medina, and Vivek Jha of NCE collected and synthesized
the information and wrote the report. The members of the topic panel are acknowledged on page iv.
This synthesis is an immediately useful document that records the practices that were acceptable
within the limitations of the knowledge available at the time of its preparation. As progress in research
and practice continues, new knowledge will be added to that now at hand.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

CONTENTS

1 Summary
3 Chapter 1 Introduction
3 Background
3 Objective
3 Scope and Approach
4 Report Organization

5 Chapter 2 Literature Review

6 Automated Cracking Protocols
15 AI and ML/DL
31 Summary

32 Chapter 3 State of the Practice

32 APCS
33 AI Technology
36 Data and Image Requirements
37 Quantifying Distress
39 Use of APCS Results
41 AI Challenges and Benefits
42 Summary

45 Chapter 4 Case Examples

47 Chapter 5 Conclusions
47 Overall Findings
48 Suggestions for Future Research

49 References
53 Appendix A Agency Survey Questionnaire
59 Appendix B Agency Survey Responses

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

SUMMARY

AI Applications for Automatic

Pavement Condition Evaluation

Departments of transportation (DOTs) use pavement condition surveys to assess current

pavement conditions and predict future pavement conditions to (1) determine the type and
timing of maintenance and rehabilitation treatments; (2) define future budget needs; and
(3) develop, or refine, pavement performance models. Over the last two decades, pavement
condition assessment has transitioned from manual to semiautomated to fully automated
pavement condition surveys (APCSs). Due to large volumes of data and images and advances
in computing and processing, the utilization of artificial intelligence (AI) technology for
distress identification has increased in recent years. While AI technologies continue to evolve,
they have demonstrated efficiency and accuracy (when trained and validated) in identifying
and quantifying pavement distress (e.g., cracking, rutting, and potholing).
The objective of this synthesis is to document state DOT automated pavement distress
identification and the use of AI [i.e., machine learning/deep learning (ML/DL)] technologies
to evaluate pavement conditions.
This synthesis is based on the combined results of a literature review of AI technology for
an APCS, a survey of state DOTs, and follow-up questions with DOTs that indicated having
experience using AI to analyze an APCS.
Forty-three (43) of the 52 surveyed DOTs (83%) responded to the survey. Of the
43 responding DOTs, 38 (88%) indicated using an APCS to determine pavement condition.
APCS data and image collection requirements included:
• Distress type and severity (38 of 43 DOTs or 88%),
• A linear referencing system (37 of 43 DOTs or 86%),
• Distress extent (35 of 43 DOTs or 81%),
• Reporting interval (31 of 43 DOTs or 72%),
• Data repeatability (30 of 43 DOTs or 70%),
• Precision and accuracy (29 of 43 DOTs or 67%),
• Compatibility with the existing pavement management system (24 of 43 DOTs or 56%),
• Spatial resolution (19 of 43 DOTs or 44%),
• Sampling rate (15 of 43 DOTs or 35%), and
• Productivity (11 of 43 DOTs or 26%).
The predominant distress types identified for asphalt pavements included transverse and
longitudinal cracking (7 of 43 DOTs or 16%) and alligator cracking (5 of 43 DOTs or 12%);
the predominant distress types identified for concrete pavements included transverse (6 of
43 DOTs or 14%) and longitudinal cracking (5 of 43 DOTs or 12%).
Eighteen (18) of 43 DOTs (42%) indicated AI technology was not used to analyze the
APCS results, 12 of 43 DOTs (28%) indicated being unsure, and 8 of 43 DOTs (19%)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

2   AI Applications for Automatic Pavement Condition Evaluation

indicated AI technology was used to analyze the APCS results. AI models used for distress
detection included:
• Random forest (RF) (2 of 43 DOTs or 5%),
• ML (2 of 43 DOTs or 5%),
• Neural networks (1 of 43 DOTs or 2%),
• Pattern recognition (1 of 43 DOTs or 2%), and
• Unsure which AI models were used (16 of 43 DOTs or 37%).
AI model selection (or development), training, and validation are needed to improve and
validate the analysis results. Most agencies (15 of 43 DOTs or 35%) were unsure how the AI
results were developed, trained, or evaluated. Conversely, 6 of 43 DOTs (14%) noted using
accuracy, precision, and repeatability criteria; 5 of 43 DOTs (12%) compared the AI results
to manual survey results; 3 of 43 DOTs (7%) each compared random or predefined refer-
ence sections to the AI results; and 2 of 43 DOTs (5%) compared the AI results to traditional
APCS results.
APCS results are predominantly used for Highway Performance Monitoring System (HPMS)
(38 of 43 DOTs or 88%) and MAP-21 reporting (33 of 43 DOTs or 77%), pavement condition
assessment and performance modeling (36 of 43 DOTs or 84% for both) and prevalent distress
type detection (22 of 43 DOTs or 51%). Regarding agency decision-making efforts, the APCS
results are used to verify performance models (32 of 43 DOTs or 74%), establish performance
targets (31 of 43 DOTs or 72%), target performance goals (31 of 43 DOTs or 72%), select
treatment types (31 of 43 DOTs or 72%), and plan multiyear budgets (31 of 43 DOTs or 72%).
Agencies also noted several benefits of using AI technology, including objectivity (14 of
43 DOTs or 33%), accuracy and increased productivity (13 of 43 DOTs or 30% for both),
and cost savings (7 of 43 DOTs or 16%). Conversely, agencies noted challenges of using AI
technology, included accuracy, precision, and repeatability (6 of 43 DOTs or 14%); com-
parison to manual surveys (3 of 43 DOTs or 7%); use of random and predefined reference
sections (3 of 43 DOTs or 7% for both); and comparison to traditional APCS results (2 of
43 DOTs or 5%).
Several future research needs were identified to expand the use and understanding of AI
technology to analyze APCS results. Future research needs include:
• Developing a standard image format (e.g., .jpeg, .png, private extension) and distress library
(e.g., pavement type, distress types, severity levels);
• Developing a publicly available image data set for AI model training;
• Providing guidance for AI model selection (or development), recalibration, training, and
validation; and
• Developing a methodology for using AI technologies in the quality control and accep-
tance process.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

CHAPTER 1

Introduction

Background
3D laser-based pavement imaging systems have been widely adopted by departments of trans-
portation (DOTs) to conduct automated pavement condition assessments. 2D imaging technolo-
gies and smartphones are also being used to conduct pavement condition evaluations, especially
by local transportation agencies. Collected pavement images are used to identify pavement dis-
tresses using various semi- and fully automated methods. Among these methods, models based
on Artificial Intelligence (AI) with machine learning/deep learning (ML/DL) have gained atten-
tion in the last several years.
The collected functional and structural distresses are key indicators of triggering pavement
maintenance and rehabilitation activities. If developers do not clearly understand DOTs’ use of
distress data, AI model development efforts for distress detection and/or classification, including
AI model formulation, distress annotation, training, and performance evaluation, can be mis-
guided and fail to reach their full potential. For example, automated distress detection methods
based on the use of image blocks to classify distress may not be able to output accurate informa-
tion on distress extent and severity. Therefore, the outcome generated by the model may not meet
an agency’s needs for project-level applications, such as planning maintenance and rehabilitation
activities.
The performance of supervised-learning AI models for automated pavement distress extrac-
tion relies heavily on several factors, including data collection, formatting, quality (i.e., resolution),
pavement image size, annotation quality (labeled ground truth distresses), and model formulation
and training. However, the performance evaluation method used for developed models is not
always clear, especially as it pertains to the diversity of the data used for evaluation and ground
truth. The lack of consistency in clarity makes the performance comparison of different AI models
challenging.

Objective
The objective of this synthesis is to document state DOT automated pavement distress identi-
fication and the use of AI (e.g., ML/DL) technologies for pavement condition evaluation.

Scope and Approach

This study focused on DOT automated pavement condition survey (APCS) practices and their
use of AI to assess pavement surface distress. Information collected included:
• Requirements for automated pavement distress identification,
• Applications of pavement distress condition information,

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

4   AI Applications for Automatic Pavement Condition Evaluation

• Types of agency decision-making supported by pavement condition data, and

• Uses of AI (e.g., ML/DL) technologies for pavement distress detection and classification.

Methods for collecting the desired information included a literature review, a DOT question-
naire, and follow-up questions as needed. The literature search results were used to develop a
survey of DOT practices, which was distributed to each state DOT, the District of Columbia
DOT (District DOT), and the Puerto Rico Highway and Transportation Authority.
To obtain more detailed information related to practice, follow-up questions were sent to
agencies that indicated using AI technologies for automated pavement condition distress iden-
tification. The follow-up questions investigated data collection requirements, AI processing
efforts, and the ability to assess historical data.
The information obtained from the literature review, the survey of practice, and the follow-up
questions provided the basis for this synthesis.

Report Organization
This synthesis is organized into the following chapters:
1. Chapter One—Introduction. This chapter provides background information and synthesis
objectives and summarizes the scope, approach, and organization of the document.
2. Chapter Two—Literature Review. This chapter summarizes the findings of the literature
review. Relevant topics covered in the literature review include automated cracking measure-
ment protocols, AI data acquisition, models, factors that affect models, and data acceptance.
3. Chapter Three—State of the Practice. This chapter summarizes the DOT survey results and
includes topics related to DOT practices on APCSs, the use of AI technology, data and image
requirements, distress quantification, the use of survey results, and the benefits and chal-
lenges of using AI technologies.
4. Chapter Four—Case Examples. This chapter summarizes two examples, one illustrating
the manual review of images for the quality control of AI results and the other illustrating the
evaluation of an AI algorithm. The relatively recent developments and implementation of AI
and associated 3D technology for APCS analysis precluded the development of DOT-specific
case examples providing details and specifics at present.
5. Chapter Five—Conclusions. The synthesis concludes with a summary of key findings and
suggested areas for future research and outreach to further utilize AI technologies.
6. Appendices—Appendix A presents the DOT questionnaire, and Appendix B summarizes the
DOTs’ responses to each survey question.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

CHAPTER 2

Literature Review

Road networks are important assets that need to be maintained at a reasonable degree of service
through routine maintenance and rehabilitation treatments to address the progression of deterio-
ration over time. Widespread use of pavement management systems, combined with pavement
condition surveys, allow agencies to assess the current road network surface condition, project
short- and long-term pavement performance, identify appropriate maintenance and rehabilitation
treatments and timing, and determine budgetary needs. Current pavement condition assessment
practices have transitioned from manual to APCSs using a variety of technologies. Advances in
computing, data collection, and image processing and detection technology have made it possible
to collect and analyze pavement surface conditions more easily than in the past.

McGhee (2004) and Pierce and Weitzel (2019) documented DOTs’ use of APCS techniques.
In terms of evaluating pavement conditions, AI technologies continue to evolve and have proven
to be a valuable tool in research and practical applications (Xu and Zhang 2022). AI can use data
from a variety of sources, including pictures, videos, and sensors, to assess pavement surface
condition. Typically, APCSs involve the use of 3D laser-based pavement imaging systems. More
recent developments include analyzing 2D images from digital cameras or smartphones and
processing with models based on ML/DL. AI technologies have a high degree of accuracy and
efficiency in the detection and classification of pavement surface distresses, including cracking,
rutting, and potholing (Sholevar et al. 2022). Pavement condition surveys that use AI technol-
ogy can offer unbiased and reliable assessment, minimizing the subjectivity and data collection
variability usually associated with manual pavement condition surveys. This chapter summarizes
current APCS protocols and using AI to evaluate APCSs.

Historically, pavement conditions have been assessed using two primary methods: manual
(e.g., walking or windshield) or semi- or fully automated surveys. In a walking survey, cracking
and other distresses are measured by a certified rater based on the DOT’s distress protocols.
Walking survey methods are time-consuming but can serve as a ground truth due to the preci-
sion of the data being collected. Windshield surveys are performed while driving at slow speed,
with the certified rater estimating the extent and severity of distress. Windshield surveys provide
less precision than walking surveys but can cover larger networks in less time. Semi- and fully
automated pavement condition surveys can utilize similar data collection equipment; however,
semiautomated methods include the use of a certified rater visually observing and noting the
presence of distress (i.e., type, severity, and extent), while fully automated methods rely on soft-
ware (e.g., AI) to identify distress (i.e., type, severity, and extent).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

6   AI Applications for Automatic Pavement Condition Evaluation

Automated Cracking Protocols

Several cracking and distress data collection protocols exist. The most relevant include:
• AASHTO R 86 (AASHTO 2022), Standard Practice for Collecting Images of Pavement Surface
for Distress Detection;
• AASHTO R 85 (AASHTO 2018), Standard Practice for Quantifying Cracks in Asphalt Pave-
ment Surfaces from Collected Pavement Images Utilizing Automated Methods;
• ASTM D6433 (ASTM 2023), Standard Practice for Roads and Parking Lots Pavement Condi-
tion Index Surveys;
• Distress Identification Manual for the Long-Term Pavement Performance Program (LTPP
Distress Manual) (Miller and Bellinger 2014); and
• Surface Condition Assessment for the National Network of Roads (SCANNER) [United
Kingdom (UK) Department for Transport 2021].
It is important to note that AASHTO R 85 (AASHTO 2018) and SCANNER protocols are
inconsistent with ASTM D6433 (ASTM 2023) and the LTPP Distress Manual regarding crack-
ing definitions, categories, and severity. The difference lies in the way the automated and manual
surveys classify the cracking. AASHTO R 85 (AASHTO 2018) and SCANNER cracking protocols
specify fewer cracking categories, while ASTM D6433 (ASTM 2023) and the LTPP Distress Manual
were developed based on a walking survey to collect more cracking types for different types of
surfaces. The following section provides a brief discussion of each protocol (Wang et al. 2020).

AASHTO R 86
AASHTO R 86 (AASHTO 2022) describes automated methods used to collect images to
determine the presence of pavement surface distress. The standard does not specify the type of
equipment but rather the following minimum criteria to ensure consistent data collection:
• The agency determines the lane and direction of travel for image collection.
• Images are at least 13 ft wide and no longer than 325 ft. Preferably, the image is 14 ft wide to
include at least 12 in. of the shoulder to capture distress at the pavement edge.
• The image quality is capable of identifying:
– 33% of all cracks with a width less than 0.12 in.,
– 60% of all cracks with a width greater than or equal to 0.12 in. and less than 0.2 in., and
– 85% of all cracks with a width greater than or equal to 0.2 in.
Assessment is based on 10 100-ft samples with at least five such cracks per sample. The crack width
must be within 20% or 0.04 in., whichever is greater, of the actual crack width at a confidence level
of 85%.
• Pavement sections with no distress have less than 10 ft of false cracking (e.g., cracking detected
but not present) over a 540 ft2 area. Assessment is based on 10 100-ft samples.
• Images may be captured using visible or infrared video, dimensional map, or any combination
of technologies capable of meeting the standard criteria.
• The agency should confirm that the equipment can meet the criteria and includes checking
the location accuracy, system stability and environmental impacts, validation, and quality
control and assurance.

AASHTO R 85
AASHTO R 85 (AASHTO 2018) describes automated methods used to measure asphalt pave-
ment surface cracking at the network level. Cracking can be measured with any equipment meet-
ing the definitions stated by the standard (Table 1).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   7

Table 1.   Cracking definitions per AASHTO R 85 (AASHTO 2018).

Crack Type Definition

Longitudinal Crack at least 12 in. long and oriented ± 20° relative to the lane centerline

Pattern Interconnected cracks or a group of cracks in each area

Transverse Crack at least 12 in. long and oriented 70° to 110° relative to the lane centerline

Other All cracks not characterized as longitudinal, transverse, and pattern cracks

Sample units of 52.8 ft long or less are used for the analysis. The standard divides the lane into
five cracking zones (Figure 1).
Cracking zones are used to identify the location of longitudinal and pattern cracks. The extent
of pattern cracking is measured by adding the length in feet or by the area influenced by the
pattern cracks, and the severity is the average width detected in the sample unit. For longitudinal
cracking and other cracking, the extent is reported by the sum of all crack lengths, and the severity
is the average crack width. The extent and severity for pattern, longitudinal, and other cracks are
reported for each cracking zone. The extent of transverse cracks is the sum of length of all trans-
verse cracks within the sample unit, and the severity is the average width of the cracks. Cracks
located in Zones 2 and 4 are assumed to be load related. Non-distressed sections are defined as
containing less than 10 ft of false negative cracking (e.g., cracking is present but not detected)
over a pavement area of 540 ft2 with a minimum of 10 100-ft long samples. The minimum standard
for accurate crack detection includes:
• At least 85% of the cracks 0.2 in. or wider are mapped,
• At least 33% of cracks less than 0.12 in. and 60% of cracks from 0.12 – 0.2 in. wide are mapped, and
• At least 85% of crack lengths are recorded.

AASHTO R 85 (AASHTO 2018) also recommends establishing a ground truth by identifying

a set of pavement images containing different crack types, extents, and severities and comparing
the results to a 100% manual survey of the same pavement sections. Usually, this comparison
is presented in a tabular form and includes crack type, severity, and the percentage of matching
values between the two methods. Successful ground truth testing is defined by a 95% or better
correlation between the image and the manual survey results.

Varies 12 ft (typical) Varies

CL

3.3 ft 2.4 ft 3.3 ft

Inside Outside
WP WP

Lane Edge Zone 1 Zone 2 Zone 3 Zone 4 Zone 5 Lane Edge

Adjacent Lane Pavement Surface Shoulder or

Survey Lane Adjacent Lane
Figure 1.   Cracking zones.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

8   AI Applications for Automatic Pavement Condition Evaluation

ASTM D6433
ASTM D6433 (ASTM 2023) describes measuring pavement distress and severity to deter-
mine the pavement condition index (PCI). PCI is a composite measure of pavement condition
and ranges from zero (very poor) to 100 (excellent). The ASTM D6433 (ASTM 2023) method
recommends subdividing the pavement into samples, typically 20 contiguous concrete slabs or
2,500 ft2 of asphalt pavement. The PCI calculation is based on the visual measurement of pave-
ment distress type, severity, and extent for each inspection sample. Distress types and severity
levels for asphalt- and concrete-surfaced pavements are summarized in Tables 2 and 3 (Note: The
extent of all distress types is number of slabs).

Table 2.   ASTM D6433 (ASTM 2023) distress types for asphalt-surfaced
pavements.

Distress Type Low Severity Medium Severity High Severity

Alligator cracking Fine, parallel, hairline, Formed pattern, may be Well-defined pieces,
(ft2) no spalling lightly spalled spalled at edges
Visible a few days of the Visible several weeks Extensive, considerably
Bleeding (ft2)
year, non-sticky per year, sticky sticky
Bumps and sags (ft) Noticeable vibration, no Significant vibration, Excessive vibration,
and corrugation (ft2) speed reduction some speed reduction must reduce speed

Depression (ft2) > 0.5 – ≤ 1 in. depth > 1 – ≤ 2 in. depth > 2 in. depth
Edge cracking (ft)
Some breakup and Considerable breakup or
(see block cracking No raveling
raveling raveling
for severity level)
No-seal 0.38 – ≤ 3 in.,
Medium or high severity
no-seal ≤ 3 in. with
Joint reflection No-seal ≤ 0.38 in. or any with secondary cracks,
secondary cracks, or
cracking (ft) filled crack no-seal > 3 in., or any
sealed with secondary
with ~4 in. distress
cracks
Lane/shoulder
> 1 – ≤ 2in. > 2 – ≤ 4in. > 4 in.
drop (ft)
No-seal with medium or
Block (ft2), and
No-seal > 0.38 – ≤ 3 in., high severity random
longitudinal and No-seal < 0.38 in. or any
no-seal ≤ 3 in. or filled cracks, no-seal > 3 in.,
transverse cracking filled crack
with light, random cracks or any with ~ 4 in.
(ft)
severely broken area
Good condition, low or Moderate distress, Badly deteriorated or
Patching (ft2)
better ride quality medium ride quality high severity ride quality

Polishing (ft2) Measure extent of affected area

0.5 – 1 in. deep and 18 –
1 – 2 in. deep and 18 –
0.5 – 1 in. deep and 4 – 30 in. wide, 1 – 2 in.
30 in. wide or > 2 in.
Potholes (count) 18 in. wide or 1 – 2 in. deep and 8 – 18 in.
deep and 18 – 30 in.
deep and 4 – 8 in. wide. wide, or > 2 in. deep and
wide
4 – 18 in. wide
Railroad crossing Noticeable vibration, no Significant vibration, Excessive vibration,
(ft2) speed reduction some speed reduction speed must be reduced
Aggregate loss > 20 per
Raveling (ft2) Not applicable Very rough, pitted
square yard

Rut depth (ft2) Mean > 0.25 – ≤ 0.5 in. Mean > 0.5 – ≤ 1 in Mean > 1 in.

Noticeable vibration, no Significant vibration, Excessive vibration,

Shoving (ft)
speed reduction some speed reduction speed must be reduced
Slippage cracking
Width ≤ 0.38 in. > 0.38 width ≤ 1.5 in. Width > 1.5 in.
(ft2)
Noticeable vibration, no Significant vibration, Excessive vibration,
Swell (ft2)
speed reduction some speed reduction speed must be reduced
Loss of fines, fading Coarse aggregate Coarse aggregate
Weathering (ft2)
asphalt color exposure ≤ 0.25 in. exposure > 0.25 in.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   9

Table 3.   ASTM D6433 (ASTM 2023) distress types for concrete-surfaced
pavements.

Distress Type Low Severity Medium Severity High Severity

Noticeable vibration, Significant vibration, Excessive vibration,
Blowup
no speed reduction some speed reduction speed reduced
No-seal > 0.5 – ≤ 2in.
No-seal > 2 in., or any
≤ 0.5 in., any sealed or ≤ 2 in. and fault ≤
Corner break crack and fault > 0.38
crack, no faulting 0.38 in., or fault ≤ 0.38
in.
in.
Low severity cracks Low severity with > 8
with 4 – 8 pieces or pieces, medium Medium severity with >
Divided slab medium severity severity with 6 – 8 8 pieces, or high
cracks with 4 – 5 pieces, or high severity severity with > 6 pieces
pieces with 4 – 5 pieces
≤ 15% of slab, most > 15% of slab, most
Durability “D” crack ≤ 15% of slab
pieces loose or missing pieces missing

Faulting > 0.13 – ≤ 0.38in. > 0.38 – ≤ 0.75 in. > 0.75 in.

Few locations of Openings, pumping, or > 10% joint sealant

Joint seal damage
sealant debonding oxidized sealant debonded or missing

Lane/shoulder drop > 1 – ≤ 2in. > 2 – ≤ 4in. > 4 in.

No-seal ≤ 0.5 in., any No-seal > 0.5 – ≤ 2 in.,
sealed crack with no-seal ≤ 2 in. and fault No-seal > 2 in. or any
Linear crack
sealant in good < 0.38 in., sealed and fault > 0.38 in.
condition fault < 0.38 in.
Patching No distress Moderately distressed Badly distressed

Polished aggregate and

Count number of affected slabs
pumping

Popouts Measure > 1 per 3 ft2

Low severity with > 5
Low severity with 2 – 5
pieces, medium Medium severity with >
pieces, or medium
Punchout severity with 4 – 5 5 pieces, or high
severity with 2 – 3
pieces, or high severity severity with > 4 pieces
pieces
with 2 – 3 pieces
Excessive vibration,
Noticeable vibration, Significant vibration,
Railroad crossing speed must be
no speed reduction some speed reduction
reduced
Good condition, minor
Scaling Scaling ≤ 15% of slab Scaling > 15% of slab
presence of scaling

Shrinkage crack Indicate presence only

Depth ≤ 1 in. and > 5 x Depth > 1 – ≤ 2 in. and
Depth > 2 in. and > 12
Spalling, corner break 5 in., or 1 – 2 in. and 5 > 12 x 12 in., or > 2 in.
x 12 in.
x 5 – 12 x 12 in. and 5 x 5 – 12 x 12 in.
Loose pieces any width
Any tight pieces, loose
and > 1.5 ft long or
pieces < 4 in. wide and
missing pieces < 4 in. Missing pieces > 4 in.
Spalling, joint < 1.5 ft long, or missing
wide and > 1.5 ft long wide and > 1.5 ft long
pieces < 4 in. wide and
or > 4 in. and < 1.5 ft
< 1.5 ft long
long

The accuracy of the pavement survey is based on identifying the pavement distress type (within
95%) and measuring linear distress (e.g., reflection cracking, edge cracking) within 10% and area
(e.g., alligator cracking, polished aggregate) within 20%. PCI categories (e.g., good, fair, poor) are
typically set by the agency and are dependent on the agency’s interpretation of acceptable levels
of service.
The PCI calculation weighs the severity of each distress type as a function of impact on main-
tenance and rehabilitation requirements. For example, potholes have a higher PCI deduct value

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

10   AI Applications for Automatic Pavement Condition Evaluation

a. PCI = 90 (minimal to no distress). b. PCI = 75 (block, longitudinal, and c. PCI = 40 (alligator cracking).
transverse cracking).
Source: NCE, used with permission.

Figure 2.   PCI category examples.

than alligator cracking, which has a higher deduct value than edge cracking. Examples of overall
pavement condition are shown in Figure 2, and an example of PCI definitions is provided in Table 4.

LTPP Distress Manual

The LTPP Distress Manual was originally developed in 1987 (and revised over the follow-
ing decades) to provide a consistent and uniform methodology for identifying and quantifying
pavement distress. The LTPP Distress Manual follows similar principles as ASTM D6433 (ASTM
2023); however, a composite measure is not determined, and pavement distress is quantified
according to pavement type instead (e.g., asphalt, jointed portland cement concrete, and con-
tinuously reinforced concrete), distress type, severity, and extent (Tables 5–7).

SCANNER
The SCANNER survey was first offered as a traffic-speed pavement condition survey for the
main road network in the UK. In 2003 and 2004, the SCANNER method was adopted by local
transportation authorities to determine pavement performance on “A” class roads in England.
Today, the SCANNER survey is mandatory and used for all road classifications in England (UK
Department for Transport 2021).

Table 4.   Example of PCI score definition [U.S. Department of the Air Force
(USAF) 2004].

PCI Category Description

100 – 86 Good Minor or no distress, routine maintenance recommended

85 – 71 Satisfactory Low severity distress, routine maintenance recommended

Low and medium severity distress, routine to major maintenance

70 – 56 Fair
recommended
Low to high severity distress, routine maintenance to reconstruction
55 – 41 Poor
recommended depending on functional or structural distress types
Mostly medium and high severity distress, challenges with applying
40 – 26 Very poor
maintenance treatments, needing more extensive repair
Mostly high severity distress, causing functional operation problems,
25 – 11 Serious
needing immediate repair

10 – 0 Failed Safe operation of vehicles no longer possible, requires reconstruction

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   11

Table 5.   LTPP distress types for asphalt-surfaced pavement (Miller and
Bellinger 2014).

Distress Type Low Severity Medium Severity High Severity

Bleeding (ft2),raveling
Measure extent of affected area
(ft2), and polishing (ft2)
> 0.75 in. wide or any
Block (ft2), joint reflection ≤ 0.25 in. wide or > 0.25 – ≤ 0.75 in. wide
crack ≤ 0.75 in. with
(ft), longitudinal (ft) and sealed crack, sealant or any crack ≤ 0.75 in.
moderate to high
transverse cracking (ft) in good condition with low random cracks
random cracks
Some breakup and Considerable breakup
No breakup or material
Edge crack (ft) material loss < 10% of and material loss
loss
length > 10% of length
Few connecting Interconnected, some Complete pattern,
Fatigue crack (ft2) cracks, no spalling or spalling, no pumping, spalled, may be sealed
pumping, non-sealed may be sealed and pumping
Rut depth (in.) and
Direct measure preferred
lane/shoulder drop (in.)
Low severity distress, Moderate severity High severity distress,
Patching (ft2) no pumping, rut ≤ 0.25 distress, no pumping, rut > 0.5 in., pumping
in. rut > 0.25 – ≤ 0.5 in. or patching
Potholes (ft2) ≤ 1 in. deep > 1 – ≤ 2 in. deep > 2 in. deep

Similar to systems in the United States, the SCANNER survey uses laser-based equipment to
measure the shape and texture of the pavement. Downward-facing digital cameras are mounted
on a vehicle to collect images (typically 10.5-ft width) to quantify cracking, and a forward-facing
video camera captures the driver’s perspective. The vehicles used for the data collection are certified
on an annual basis, and the surveys are conducted by independent contractors (UK Department
for Transport 2021).

Table 6.   LTPP distress types for jointed concrete pavement (Miller and
Bellinger 2014).

Distress Type Low Severity Medium Severity High Severity

Spall ≤ 10% length, no Low severity spall Spall moderate to high
faulting, material loss, > 10% length, or fault ≤ > 10% length, fault >
Corner break (count)
or patching, and 0.5 in., and broken ≤ 2 0.5 in., or broken > 2
broken ≤ 2 pieces pieces pieces
Tight, no loose or Well-defined pattern, Developed pattern,
Durability “D” crack (ft 2) missing pieces, no some small loose or significant loose or
patching missing pieces missing pieces
Lane/shoulder drop or
Direct measure preferred
separation, faulting (in.)
< 0.12 in. wide, no ≥ 0.12 – < 0.25 in. wide, ≥ 0.5 in. wide, or
Longitudinal crack (ft)
spalling, faulting or spalled < 3 in., or fault spalled ≥ 3 in., or fault
and transverse crack (ft)
sealed ≤ 0.5 in. ≥ 0.5 in.
Longitudinal joint seal
Number of sealed joints and length of sealed joints with joint seal damage
damage (ft)

Map crack/scaling (ft2) Number of occurrences and square feet of affected area
Low severity distress, Moderate severity High severity distress,
Patching (ft2) no material loss or distress, fault < 0.12 fault ≥ 0.12 in.,
pumping, no faulting in., no pumping patched, pumping
Polishing (ft2) Indicate potential reduction in surface friction

≤ 3 in. wide, no > 3 – ≤ 6 in. wide, loss > 6 in. wide, spalled >
Spalling (ft)
patching, or spalls of material 2 pieces, or patched
Transverse joint seal
≤ 10% of joint > 10 – ≤ 50% of joint > 50% of joint
damage (count)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

12   AI Applications for Automatic Pavement Condition Evaluation

Table 7.   LTPP distress types for continuously reinforced concrete pavement
(Miller and Bellinger 2014).

Distress Type Low Severity Medium Severity High Severity

Blowups (count) Count number of occurrences

Tight, no loose or Well-defined pattern, Developed pattern,
Durability crack (ft2) missing pieces, no some small loose or significant loose or
patching missing pieces missing pieces
Lane/shoulder drop or
Direct measure preferred
separation (in.)
< 0.12 in. wide, no ≥ 0.12 – < 0.25 in. wide, ≥ 0.5 in. wide, or
Longitudinal crack (ft) spalling, faulting or spalled < 3 in., or fault spalled ≥ 3 in., or fault
sealed ≤ 0.5 in. ≥ 0.5 in.
Longitudinal joint seal
Not applicable
damage (ft)

Map crack/scaling (ft2) Not applicable

High severity distress,
Low severity distress, Moderate severity
fault ≥ 0.12 in.,
Patching (ft2) no material loss or distress, fault < 0.12
patched, may have
pumping, no faulting in., no pumping
pumping
Polishing (ft2) Not applicable, indicate potential reduction in surface friction
Crack spalling < 3 in. Crack spalling ≥ 6 in.
Crack spalling ≥ 3 –
or fault < 0.12 in., no or punched down ≥
Punchouts (count) < 6 in., or fault ≥ 0.12 –
material loss or 0.25 in., or loose, or
< 0.25 in.
patching broken > 2 pieces
≥ 6 in., material loss,
< 3 in., no material loss ≥ 3 – < 6 in., material
Spalling (ft) broken > 2 pieces, or
or patching loss
patched
Spalling ≤ 10% of Spalling > 10 – ≤ 50% Spalling > 50% of
Transverse crack (ft)
length of length length
Transverse joint No spalling or fault Spalling < 3 in. within ± Spalling ≥ 3 in. within ±
deterioration (count) ± 2 ft of joint 2 ft of joint 2 ft of joint

The SCANNER survey collects the longitudinal profile, transverse profile, edge condition,
and texture of the road and the presence and extent of cracking (Table 8).
Cracking is quantified using a crack map and includes crack length, offset position, angle, and
type (i.e., joint or crack). Total cracking is estimated by overlaying a crack map with a grid and
adding the areas of the grid squares containing cracks. Each pavement cell is 8 in. by 8 in., and the
size of the grid map is 20 cells long (approximately 13 ft) and 16 cells (approximately 11 ft) wide
(Figure 3) (UK Department for Transport 2021).
Transverse cracking is defined as a crack covering an area of at least two cells in the longitu-
dinal direction and the entire width of the lane (Figure 4). In Figure 4, 13 cells within a 1.3 ft
length in the longitudinal direction and across the full survey width were identified to contain a
crack. Within the sample area, 32 cells, or 41%, contain a crack. Since more than 20% of the total
cells within this area have been identified as containing a crack, the cells are defined as a trans-
verse crack.
The intensity of wheel path cracking is measured by first examining the position of the crack
to determine whether it falls within the wheel path area. The wheel path area is defined as an area
approximately 2 ft to both the right and left of the survey centerline with a wheel path width of 2.6 ft
(Figure 5). A wheel path crack occurs when 80% of the detected crack is within the wheel path
area (UK Department for Transport 2021).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   13

Table 8.   SCANNER data collection.

Distress or Condition Type Definition

Based on transverse profile measurements, averaged over 4 in.

Rut depth (in.)
intervals.

Longitudinal profile Based on longitudinal profile measurements.

Texture (in.) Macrotexture measured at 0.04 in. intervals and averaged over 33 ft.
Processing downward-facing images using automatic crack detection
Cracking software, describes transverse and longitudinal location, length, and
angle (relative to direction of traffic) of the crack.
Measured from transverse profile at pavement edge, reported on 33 ft
Edge deterioration
intervals.

a. Detected cracking. b. Processed cracking.

Figure 3.   Cracking grid map for crack intensity measurements (adapted from UK Department for
Transport 2021).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

14   AI Applications for Automatic Pavement Condition Evaluation

a. Detected cracking. b. Processed cracking (> 20% purple crosshatch over 1.3 ft
length in longitudinal direction).

Figure 4.   Cracking grid map for transverse cracks (redrawn from UK Department for
Transport 2021).

a. Detected cracking (solid black lines). b. Processed cracking (purple crosshatch).

Figure 5.   Cracking grid map for wheel path cracks (redrawn from UK Department for Transport
2021).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   15

AI and ML/DL
AI can be defined as a series of robust computational algorithms and statistical models capable
of adapting and learning from data patterns. AI technology has been increasingly incorporated
into pavement engineering research and practice and related fields because of the continuous
advancement in automatic data acquisition devices, computer vision techniques, and ML/DL
algorithms (Majidifard et al. 2020a, Hou et al. 2021, Guerrieri and Parla 2022, Sholevar et al.
2022, Xu and Zhang 2022).
AI is used with pavement condition surveys to automatically identify and classify different
types of distresses. The use of AI has notable advantages, including data collection automation,
accurate models when trained using large databases, and real-time monitoring. A significant
challenge in using AI for pavement condition assessment is data availability (i.e., getting a rep-
resentative and sufficient number of data set images that have been labeled with pavement dis-
tresses). The AI models also need to be trained using a wide range of pavement distresses (type
and severity) to accurately classify the proper condition. Lastly, it is important to understand the
AI model architecture to interpret and trust the results.
AI models and techniques will be discussed in the following sections; however, a brief descrip-
tion of the process for pavement distress detection is as follows:
• Collect a representative data set of images from a road network,
• Label the images with the proper distress types and severities,
• Conduct pre-image processing to remove any noise in the images,
• Build an AI model for image analysis that can extract the features of the labeled images (i.e.,
an AI model will learn those features corresponding to pavement distresses), and
• Apply the AI model to the data set (typically, 70%) for training purposes, and use the remainder
of the data set (typically, 30%) for model validation.

Data Acquisition
Digital image processing technologies have been applied widely to pavement condition assess-
ment. For example, digital cameras have been used to detect cracks, and laser scanning has been
used to determine rut depth and the International Roughness Index (IRI). Image technologies have
also been employed for jointed plain concrete pavement, joint faulting, pavement texture, and pot-
hole identification. More recently, cameras and light detecting and ranging (LiDAR) systems in
unmanned aerial vehicles have been used to identify pavement distresses and roughness (Du et al.
2021). Figure 6 illustrates currently used data acquisition methods, each of which are described
briefly in the following sections.

Vision-Based
Extensive effort has been dedicated to processing images from digital cameras mounted to
vehicles to detect cracking, permanent deformation, and surface texture (Du et al. 2021). Con-
ventional image processing and traditional ML/DL models are typically used to perform image
analyses (Sholevar et al. 2022). Figure 7 illustrates the use of digital images to identify surface
distress from a dashboard-mounted smartphone.

3D Imaging
Laser-based technologies, line projection, and ground penetrating radar are technologies
employed to capture 3D images. 3D laser scanning data has become the predominant approach
for automatic pavement condition data collection due to its ability to gather depth information
and its robustness to lighting conditions relative to that of 2D images (Wang et al. 2017; Zhang
et al. 2018). Most 3D data acquisition systems use triangulation (i.e., laser line profiling) and

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

16   AI Applications for Automatic Pavement Condition Evaluation

Automated
Methods

Vision-Based 3D Imaging Vibration-Based

Digital Cameras Laser-Based Accelerometers

Unmanned
Stereovision Smartphones
Aerial Vehicle

Ground
Smartphones Penetrating
Radar

Figure 6.   Automated data acquisition

techniques.

structured illumination (i.e., patterns of light used to define an object’s geometric shape and
depth), projecting a laser line onto the object being measured. A digital area scanning camera is
placed at a defined distance and angle from the light source. The camera is equipped with either a
charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor.
The camera captures pictures in structured illumination, and the image is analyzed by examin-
ing the deformations of the laser lines on the object. This examination allows the depth (z-axis)
and the horizontal position (x-axis) of each point to be determined (Figure 8). To obtain the
y-axis position, the 3D system is typically paired with an encoder (Tsai and Li 2012).

Vibration-Based
Inertial profiler systems include an accelerometer to measure the vertical vibration of the
vehicle, sensors to measure the relative movement between the road surface and the vehicle, and
a distance-measuring instrument to record the distance traveled along the roadway (Figure 9).
Most recent research has used DL/ML to analyze vibration measurements from accelerometers
to quantify pavement roughness (Liu et al. 2019; Aboah and Adu-Gyamfi 2020). The vibrations
collected by static accelerometers or smartphones can be used to measure roughness and to detect
other distresses, such as potholes (Eriksson et al. 2008; Medina et al. 2020). Other researchers

Source: RoadBotics by Michelin, used with permission.

Figure 7.   Surface distress detection using a

dashboard-mounted smartphone.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review 17

a. Data collection equipment. b. Processed image.

Figure 8. Example of 3D imaging system (Pierce and Weitzel 2019).

have investigated the use of vibration sensors installed in the pavement and use ML techniques to
detect pavement deterioration (Takanashi et al. 2020; Gao et al. 2021).
Most smartphones have cameras that can capture high resolution images from pavements to
detect pavement distresses, such as cracking and potholes (Huyan et al. 2020; Chitale et al. 2020).
They also have embedded sensors, such as accelerometers and GPS sensors. In the past decade,
accelerometers embedded in smartphones have gained popularity as an alternative low-cost tool
to measure car vibrations (Islam et al. 2014; Bridgelall 2015; Bridgelall and Tolliver 2018; Medina
et al. 2020). Figure 10 shows a smartphone collecting vibration data.

AI Models
ML/DL techniques have shown to be successful for many types of problems involving com-
puter vision applications (Guerrieri and Parla 2022), including identifying pavement distresses
and other civil engineering applications. Figure 11 illustrates the image analysis methods used to
detect pavement distresses (Sholevar et al. 2022). The following sections provide a brief descrip-
tion of each image analysis method.

Figure 9. Example of a high-speed inertial profiler

(FHWA n.d.).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

18   AI Applications for Automatic Pavement Condition Evaluation

Source: Medina et al. 2020, used with permission.

Figure 10.   Example of a smartphone collecting

vibration data.

DL
In civil engineering and intelligent transportation systems, DL approaches have emerged as
the most frequently used computational strategy, producing comparable results between pave-
ment condition surveys and field conditions (Zhu et al. 2021; Ranyal et al. 2022). The capacity of
DL to learn large amounts of data is one of its advantages. Another advantage, particularly over
conventional ML or other image processing methods, is that the data-driven, end-to-end learn-
ing process is more accurate, more accessible, and faster (Sholevar et al. 2022). Deep convolu-
tional neural networks (CNNs) have assumed the utmost relevance in performing vision-based
tasks as a result of the quick development of DL approaches. Classification, object detection,
and picture segmentation are the main pattern recognition tasks carried out by vision-based DL
algorithms (Ranyal et al. 2022).

Traditional Machine
Deep Learning Image Processing
Learning

Classification Support Vector Machine Edge Detection

Decision Tree and

Object Detection Threshold
Random Forest

Semantic Segmentation Boosting Manual Filters

Artificial Neural
Network

Figure 11.   Image analysis methods.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   19

Classification
Classification methodology divides an image into blocks and assigns each block a target class
by using the sliding window technique to analyze images of any size. CNNs with a simple design
are used in image classification models to extract features or patterns from pictures (Sholevar
et al. 2022). The classification detection method is helpful when making decisions about whether
a picture contains the desired object, an anomaly, or nothing. Localization, which establishes the
location of the identified object, is a notable classification function (Ranyal et al. 2022). Figure 12
illustrates an example of DL classification and how the image is broken down into smaller blocks
to classify the crack.

Object Detection
Object detection methodology combines classification and localization to identify the items
in an image and determine their locations. Bounding boxes are used to demonstrate the detec-
tion and position of an object when applying categorization to different objects (Ranyal et al.
2022). Object detection architectures like You Only Look Once (YOLO), Regions with CNN,
and Single-Shot Detector (SSD) are widely used for pavement distress detection because of their
speed, simplicity, and capacity to search within a large image for any number of predefined
distresses. These benefits make object detection the most practical method for identifying the
location and severity of pavement distresses in large, complicated images (Sholevar et al. 2022).
Figure 13 shows an example of road damage detection using object detection. As seen in the
figure, the model detects longitudinal cracking (blue box); however, even in trained data sets, it
is difficult to differentiate between cracks and shadows usually caused by light posts or overhead
power lines (red box).

Semantic Segmentation
By dividing an image into sections, semantic segmentation extracts potentially significant
portions of the image for further processing using classification and object detection depending
on its shape and borders. By highlighting the foreground features, semantic segmentation makes

Source: Cha et al. 2017, used with permission.

Figure 12.   Example of DL classification.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

20   AI Applications for Automatic Pavement Condition Evaluation

Source: Ale et al. 2018, used with permission.

Figure 13.   Example of object detection.

it easier to assess the objects. Semantic segmentation, as opposed to classification and object
detection, shows pixel by pixel features of an item, as seen in Figure 14 (Ranyal et al. 2022). Fully
convolutional networks and encoder-decoders are the two most common architectures for seg-
mentation models (Sholevar et al. 2022).
A summary of advantages and limitations of the DL method is summarized in Table 9.
Table 10 provides a list of studies that have used DL in pavement distress detection.

a. Original image. b. Pixel by pixel.

Source: Zhang et al. 2019, used with permission.

Figure 14.   Example of crack segmentation and pixel by pixel

feature extraction.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Table 9.   DL Method advantages and limitations (Ranyal et al. 2022; Sholevar
et al. 2022).

Method Advantages Limitations Accuracy

Performs better than Requires large data set for

conventional ML methods. training.
Classification High-speed for training and Determine presence of distress 90-97%
testing phases. but not location, severity, and
extent.
High accuracy in distress Requires large data set for
detection. training.
Provides classification and Complex architecture.
Object localization and allows for Physical characteristics of
70-99%
Detection mapping. distresses are a challenge when
Use of LiDAR and laser and using 2D images.
point clouds and allows for
distress characterization.
Pixel by pixel classification. Requires large data set for
Allows for a more in-depth training.
image analysis. Low speed in training and
Measures severities and extent. testing.
Semantic
Requires post-processing 70-97%
Segmentation
algorithms to extract exact,
smooth shapes from pixelated
outlines.
Results prone to noise.

Table 10.   DL research studies (Sholevar et al. 2022).

Distress Types Identified

Reference
None Bumps Bleeding Cracking Patching Potholes Rutting

Nguyen et al. 2018

Shah and Deshmukh

2019

Eisenbach et al. 2017

Fan et al. 2019

Nie and Wang 2018

Lei et al. 2020

Commandre et al.
2017

Anand et al. 2018

Maeda et al. 2018

Song et al. 2020

Zhang et al. 2017

Chen et al. 2020

Konig et al. 2019

Yang et al. 2020

1
deformation.
2
included crack sealing
3
study also identified utility covers
4
fatigue and transverse cracking
Note: ü indicates distress type identified in the research study

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

22   AI Applications for Automatic Pavement Condition Evaluation

Traditional ML
Traditional ML models analyze patterns to classify and quantify pavement condition informa-
tion. Typical ML technologies include (a) support vector machine (SVM) method, (b) decision tree
(DT) and RF, and (c) bootstrap aggregation.

SVM Method
The SVM method is frequently used for solving regression and classification problems. The
SVM method depends on extracting high-quality features from the image (Cao et al. 2020). The
model features (e.g., pavement distress) must be very well defined to ensure model reliability.
In addition to pavement distress detection, SVM methods have been used for cancer detection,
financial forecasting, picture classification, digital handwriting recognition, and facial expression
classification (Hoang et al. 2018; Xu and Zhang 2022). SVM methods use a hyperplane in the
input variable space to split the positive and negative training samples. Typically for pavement
crack detection, the image is preprocessed or filtered to a smooth texture to enhance the exist-
ing defect or cracks. The image is subdivided into blocks, with each block producing a feature or
support vector. Finally, the SVM method is used to detect and classify cracks, potholes, and other
defects using the information in the support vector structure (Daniel and Preeja 2014).

DT and RF
DT algorithms are also commonly used for regression and classification. DTs have a hierarchical
structure consisting of a root node (or first splitting node), branches to connect to internal nodes
or decision/attribute nodes, and leaf nodes, which represent all possible outcomes within the data
set. This process is applied to all data sets iteratively until all or most of the data points are classified.
RFs are algorithms that combine the output from DTs to achieve a single result. DT and RF models
have also been developed for pavement distress detection (Cubero-Fernandez et al. 2017; Hoang
and Nguyen 2019). The output of each DT is evaluated to detect a feature classification (i.e., crack
detection), with the majority or average results reported as the final output. Figure 15 shows an
example of the RF crack definition process from the original image to the final output.

Bootstrap Aggregation
Bootstrap aggregation, or bagging, is an ML technique typically used in DT techniques to
improve the stability and accuracy of the AI models. The bootstrap aggregation technique ran-
domly creates multiple subsets to train a separate model, then combines (or aggregates) the
predictions of each separate model into a final prediction model. Ahmad et al. (2023) developed
an ML architecture using bootstrapping to improve the accuracy of pavement crack detection
and segmentation from different data sets.

Boosting
Boosting refers to a generic and efficient approach combining imprecise and inaccurate classi-
fiers to produce an accurate prediction rule. In other words, boosting is founded on the idea that
discovering several imprecise rules of thumb or classifiers is sometimes far simpler than discover­
ing a single, extremely precise prediction rule (Schapire 2003). To achieve the desired outcome,
a training set of data is scanned repeatedly, letting the learning algorithm concentrate on the cases
incorrectly identified in previously repeated scans (the “hardest cases”). Judgments made using a
weighted mixture of the unreliable classifiers can provide results with higher accuracy than indi-
vidual weak classifiers (Schapire and Freund 2013). Traditional crack detection models may con-
tain a high number of false detections, and the Adaptive Boosting (AdaBoost) model is frequently
used to improve model accuracy (Xu et al. 2022). AdaBoost model operation is examined in depth
using a range of techniques by paying close attention to the risk of overfitting to the training set.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   23

Original image Ground truth Identify dark pixels

Apply thresholds to
detect crack pixels Crack region detection

Final output
Source: Li et al. 2015, used with permission.

Figure 15.   Example of RF crack definition.

The evaluation of “confidence” measures for the weak classifiers can improve the model and dis-
crimination rules even after the training set is correctly categorized (Cord and Chambon 2012;
Schapire and Freund 2013). Figure 16 compares the output for a boosting method to the original
and ground truth images from different data sets. Ground truth images are manually processed to
isolate desired features (e.g., cracks) from the original image and create a label with these features.

Artificial Neural Networks

Artificial neural networks (ANNs) are information processing architectures made up of sev-
eral basic processing elements, or “neurons,” interconnected in a parallel manner. Each neuron
has the ability to share its outputs, if any, to and accept weighted inputs from many other neurons.
Throughout the weighted interconnections, information is represented in a distributed manner.
A collection of patterns is repeatedly given to the ANN during a training session, and the ANN
learns the class to which each of the input patterns belong. Figure 17 provides an example of crack
detection used by an ANN model. Advantages and limitations of the traditional ML method are
summarized in Table 11.
Table 12 provides a summary of relevant efforts for pavement distress detection using tradi-
tional ML models.

Image Processing
Traditional image processing refers to filtering noise captured by digital cameras or imaging
sensors. In pavement condition assessment, images can be affected by shadows, rain, stains, and
other factors. Usually, cracking in an image shows a thin, irregular shape along with dark lines
bounded by noise. Typical image processing methods include the following items described in
the subsequent section:

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

24   AI Applications for Automatic Pavement Condition Evaluation

CRACK500

Cracktree200

CFD

AEL

GAPs384

a. Original image. b. Ground truth. c. Boosting.

Note: CRACK500, Cracktree200, Crack Forest Dataset (CFD), AFL, and GAPs384 are data sets.
Source: Yang et al. 2019, used with permission.

Figure 16.   Example of boosting.

Edge Detection
Edge detection methods are able to clearly identify the crack outline. A differential function is
used to determine edge information based on a crack’s grayscale change (Hou et al. 2021). Several
edge detection methods exist (referred to as operators), including Sobel, Prewitt, Roberts, and
Canny (Cao et al. 2020).

Thresholding
Thresholding-based methods have been widely used for crack detection since the 1980s due
to their simplicity and effectiveness (Oliveira and Correia 2009, Li et al. 2015, Zhou et al. 2016, Cao
et al. 2020). These methods determine whether a pixel belongs to the target area or whether the

a. Original image b. ANN.

Source: Nguyen et al. 2018, used with permission.

Figure 17.   Example of ANN model for crack definition.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   25

Table 11.   Advantages and limitations of traditional ML methods

(Sholevar et al. 2022, Xu and Zhang 2022).

Method Advantages Limitations Accuracy

Less dependent on high volume of Limited generalization.

SVM 78 – 98%
learning data.
Less dependent on high volume of Results can be biased or
learning data. model overfit.
DT and RF High efficiency, feasibility, 80 – 98%
accuracy, and shorter processing
time.
Less dependent on high volume of Sensitive to outliers, and
Boosting learning data. overemphasizing outliers can 85 – 98%
lead to model overfitting.
Less dependent on high volume of Nondetailed evaluation.
learning data. Does not detect distress
Ability to extract image features or location, severity, or extent.
ANNs 92 – 98%
patterns at different image scales. Difficulty detecting global
High speed at training and testing context due to the “black box”
phases. nature of the algorithm.

background characteristic attributes meet the set threshold values. In this way, a gray image can be
transformed into a binary image. The key to this method is identifying an acceptable threshold value.

Region Growing
Another method for detecting pavement cracks is the recognition technique, which is based
on region growing. This method was proposed and adopted in the Texas Department of Trans-
portation’s APCS in the mid-2000s. The region growing method can be used to identify the edge
distribution of crack defects and draw the crack contour but cannot provide a description of the
information contained within the crack’s internal pixels. Through the use of image characteristics

Table 12.   Traditional ML research studies for pavement condition assessment.

Diagonal Longitudinal Transverse

Reference Cracking Potholes Rutting
Cracking Cracking Cracking

H. Li et al. 2019 

Ai et al. 2018 

B. Li et al. 2019 
Ahmadi et al.
2021
   

Cao et al. 2021 

Pan et al. 2018  

Oliveira and
Correia 2013

Moussa and
Hussain 2011


Lin and Liu 2010 

Pan et al. 2017  

Note:  indicates distress type identified in the research study

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

26   AI Applications for Automatic Pavement Condition Evaluation

a. Verified crack seeds. b. Crack clusters. c. Verified crack clusters-long.

Source: Huang and Xu 2006, used with permission.

Figure 18.   Example of recognition technique based on region growing method.

(e.g., border cell contrast and pixel brightness), the original image is transformed into a grid
of cells (8 × 8 pixels), and each cell is categorized as either a crack (i.e., seed) or non-crack cell
(Figure 18). Seed clusters are then formed from the crack seeds and crack features are extracted
(Huang and Xu 2006; Zhou et al. 2016). Gathering similar pixels to form a region is the funda-
mental concept behind the region-growing algorithm. The accuracy of image segmentation is
greatly affected by the seed selection (Cao et al. 2020).

Manual Filters
Manual filters are applied to isolate the desired features from an image. Typically, the image
is converted into binary code before the threshold filters are applied. Figure 19 illustrates an
example of manually isolating a crack from an image. Manual filters are time-consuming and,
therefore, are not practical for network level APCSs.

Factors Affecting AI Models

It is important to consider the impact of data assumptions prior to using AI algorithms for
pavement distress evaluation. The accessibility of the data used to train and test the AI algorithm
has a substantial impact on its performance. Since data are the basis of all AI-oriented models,

a. Original image. b. Processed image.

Source: Medina et al. 2020, used with permission.

Figure 19.   Example images of manual threshold segmentation.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   27

the absence of valid data can seriously weaken the method’s validity. For example, if the data used
for training any model fail to provide a ground truth, any results obtained from the model will
not be valid (Xu et al. 2022). Furthermore, accurate pavement distress detection exponentially
increases model accuracy with more training images (Ranyal et al. 2022). The factors affecting
AI models include data quality and diversity, data preprocessing, model architecture and train-
ing, and overfitting.

Data Quality and Diversity

Data quality for model training has a significant impact on model robustness. Image quality
can also significantly affect any proposed model, and the presence of image noise can affect the
detection of pavement distress or the classification model (Majidifard et al. 2020b, Xu et al. 2022).
It is important to gather a diverse and comprehensive data set of images covering various scenarios
and variations (i.e., different types of distresses with different illumination). The data set should
include labeled images to train the model effectively.
The image size required for AI mode training can vary depending on the specific task and
the architecture of the model being used. However, some general considerations should be kept
in mind:
• The image size should be compatible with the architecture of the AI model;
• The larger the image, the more computational effort will be required;
• The image size should be chosen based on the features to be extracted (e.g., minimum size
required to detect cracking or other distresses); and
• Many of the data sets used to train AI models come from private sources, making it difficult
to compare models. Approximately 78% of the AI distress models are trained on private data
sets; only 22% are trained on public data sets.
Table 13 provides a description of the type of processing task, the device used for data collec-
tion, the number of images, and the image size for some of the open-source data sets.

Data Preprocessing
Data preprocessing refers to the action of modifying data before they are used to improve
their performance. The input variables should be carefully chosen and optimized for the AI
algorithm. Preprocessing the image and data set is a typical step in AI models for distress detec-
tion applications and may include removing unrelated features or objects, resizing the inputs,
and augmenting the data to improve the initial data set and avoid model overfitting (Wang and
Hu 2017, Ghasemi et al. 2018, Chun et al. 2021, Hsieh and Tsai 2021, Sholevar et al. 2022). Other
preprocessing techniques include removing image noise, resizing, cropping, gray scaling, and
normalization (Sholevar et al. 2022). In some cases, noise can be detected. In Figure 20, noise is
very difficult to detect because the crack and noise patterns are very similar.

Model Architecture and Training

It is important to choose the correct AI architecture for the image processing task. For pave-
ment distress detection, a model capable of extracting image features and spatial relationships
is very important. For training purposes, AI models can be broadly classified into supervised,
unsupervised, and reinforced learning (Praveena and Jaiganesh 2017). Supervised and unsuper-
vised training are commonly used methods for crack and pavement distress analysis.
The goal of supervised training techniques is to identify the link between input (i.e., indepen-
dent variable) and target attributes (i.e., dependent variable). A model serves as a representation
of the identified relationship. Developed models describe and explain hidden events within a
data set and may be used to predict an attribute’s target value, given the values or labels of its

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

28   AI Applications for Automatic Pavement Condition Evaluation

Table 13.   Public data sets (Arya et al. 2021, Ranyal et al. 2022, Sholevar et al.
2022, Xu et al. 2022, Zhu et al. 2022).

No. of
Data set Task Type Device Distress Resolution
Images

CDF Segmentation iPhone 5 Cracking 118 480x320

Professional 991x462,
Aigle-RN Segmentation Cracking 38
camera 311x462

Crack500 Segmentation LG-H345 Cracking 500 2000x1500

German Asphalt
Professional Several
Pavement Distress Object detection 1,969 1920x1080
camera Distresses
(GAPs v1)
Professional
GaMM Segmentation Cracking 42 1920x480
camera

Cracktree200 Segmentation N/A Cracking 206 800x600

CrackIT Segmentation Optical Cracking 84 1536x2048

EdmCrack6001 Segmentation GoPro 7 Cracking 600 1920x1080

Cracking,
GAPs v2 Object detection N/A pothole, patch, 2,468 1920x1080
open joint
Pavement image Google Several
N/A 7,237 N/A
data set application distresses
Several
Street-view images N/A Smartphones 9,053 N/A
distresses
Road Damage Object detection, Several 600x600 and
Smartphones 26,336
Dataset classification distresses 720x720

Deep Crack Segmentation N/A Crack 537 544x384

Unmanned
Unmanned Asphalt Several
Object detection aerial 3,151 5632x3584
Pavement Distress distresses
vehicles
1
EdmCrack600 is a data set from roads in Edmonton, Canada.
N/A= Not Available

a. Original image. b. Image noise (top) and crack (bottom).

Source: Zhang et al. 2019, used with permission.

Figure 20.   Example of image noise.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   29

input attributes (Cao et al. 2020). Common supervised models include SVM, ANN, and RF
(Chou et al. 1994, Li et al. 2009, Nguyen et al. 2009, Moussa and Hussain 2011, Del Rio-Barral
et al. 2022).
Unsupervised training, also referred to as data mining, occurs when the data do not contain
labels. This means data samples have no recognized output results, and the model needs to learn
relationships between samples and then classify them. The advantage of using unsupervised train-
ing is the elimination of human influence and subjectivity in creating labels (Cao et al. 2020).
Unlike supervised and unsupervised learning, reinforced learning (RL) is a unique AI tech-
nology. While the goal of extrapolating or generalizing the information gained from a training
set of labeled examples in supervised learning is to have the system to respond appropriately to
new input and the goal of unsupervised learning is to determine the structure of the unlabeled
data, the goal of RL is to maximize the overall cumulative reward. RL involves learning by trial
and error based on the feedback acquired from the interaction between agents and the environ-
ment, as opposed to learning from examples of appropriate conduct (Xu et al. 2022).

Overfitting
In AI, overfitting refers to the algorithm behavior that occurs when the model accurately
predicts the data from the trained data set but cannot accurately generalize to a new data set.
Figure 21 shows an example of overfitting. The model was trained and validated using images
from a different data set. When the model was used with new images, the model erroneously
detects a moderate severity patch bound by a blue line and cracks/open joints bound by red lines.
A common technique to address overfitting is referred to as gradient boosting (GB). GB com-
bines the predictions from many weak classifiers to create a single, strong classifier. By building
a separate tree each time, GB can get around the primary flaw of DT models (overfitting) by
calibrating mistakes introduced by earlier trees and adjusting the weights of the observations.
Another way to minimize overfitting is to train a model with randomly selected images (70% of
the data set) and use approximately 15% of the images for validation and the other 15% to evalu-
ate the final AI model fit. Typically, the validation data set is used to calibrate or tune the AI
model, providing an unbiased assessment; however, the increased use of the validation data set
can result in a more biased AI model.

Source: Medina et al. 2020, used with permission.

Figure 21.   Example of overfitting.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

30   AI Applications for Automatic Pavement Condition Evaluation

Data Acceptance
Ground Truth
Ground truth is a commonly used term that refers to the correct or true value of a specific
observation or problem. Cracks and other distresses are digitized manually using ground truth
images or field measurements (Figure 22). The Hausdorff scoring method is commonly used to
compare the AI models to the ground truth results. The Hausdorff method provides a score that
ranges from 0 to 100, with 100 representing perfect distress segmentation (Oliveira and Correia
2009, Tsai et al. 2012, and Zhang et al. 2018).

Model Validation
Typical evaluation metrics for ML/DL models include precision, recall, and accuracy. Preci-
sion is the ratio of the correct detected results to all the detected results. Recall is the ratio of the
correct detected results to all the ground truth results. The F1 score is commonly used to deter-
mine model accuracy and is the harmonic mean of precision and recall. The F1 score is typically
preferred over other statistical metrics due to its ability to balance precision and recall. Table 14
summarizes typical model evaluation metrics.
Model validation of detected distresses is one of the primary tasks for evaluating ML/DL
models. In addition to the performance metrics described above, other methods used to vali-
date models include cross-validation, the holdout method, k-fold cross-validation, and leave-
one-out cross-validation.
In ML, cross-validation refers to the process in which models are trained on subsets of the input
data and then evaluated on the complementary subset data. The purpose of cross-validation is to
detect data overfitting, identify any bias, and provide an understanding of whether the model will
generalize with other independent data sets. The difference between cross-validation and k-fold
cross-validation is the k-value refers to the number of groups that will be used to train and test
the ML/DL models. The holdout method differs from the cross-validation method in the way the
training and testing data are split. In the holdout method, the data set is split into the training and
testing set only once; in the cross-validation process, the data set is split randomly and in multiple
folds or data sets. The leave-one-out cross-validation method is typically used when the data set
is small or when the precision is more important than the computational cost since this method
includes more training-test iterations than other methods. This method considers each observa-
tion as the validation set and the rest of the observations as the training data set. The process
repeats the validation process for each observation, resulting in a large computational cost.

a. Original image. b. Ground truth.

Source: Yang et al. 2019, used with permission.

Figure 22.   Example of manually setting ground truth.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Literature Review   31

Table 14.   AI model performance measures (Eisenbach et al. 2017).

Measure Definition1

True positive rate = =

+

True negative rate =

+

Precision =
+
1
Balanced error rate = 1− ( + )
2
+
Accuracy =
+ + +

∙ − ∙
=
Matthews correlation coefficient √
=( + )( + )( + )( + )

G-mean =√ ∙

Area under the Receiver Operator

= ∫
Characteristic (ROC) curve
∙
F1 score 1 = 2∙
+

G-measure =√ ∙

Break-even point = =

Area under the precision recall curve =∫

1
tp = number of true positives, fn = number of false negatives, fp = number of false positives, and
tn = number of true negatives.

Summary
This section provided a literature review of the current automated pavement condition assess-
ment protocols, such as AASHTO R 85 (AASHTO 2018), ASTM D6433 (ASTM 2023), LTPP
Distress Manual, and SCANNER. It also included an overview on the use of AI technology for
APCS. APCS analysis using AI methods includes:
• Data acquisition using vision-based (e.g., digital cameras), 3D imaging, or vibration-based (e.g.,
accelerometers) systems;
• Modeling using DL (e.g., classification, object detection, and semantic segmentation), ML
(e.g., SVM, decision tree, RF, boosting, and ANN), and image processing e.g., (edge detection,
threshold, and manual filters) methods; and
• Data acceptance based on ground truth testing (e.g., manual versus AI results, image review
versus AI results) and model validation based on precision, recall, and the F1 score.
The literature review also identified several factors affecting AI models. These factors included
data quality and diversity (e.g., image quality, image noise, different distress, different illumina-
tion levels), data preprocessing (e.g., removing features, resizing images, and data augmentation),
model architecture and training (e.g., capable of extracting image features, spatial relationships),
and overfitting (e.g., model accurately predicts from the trained data set but inaccurately on a
new data set).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

CHAPTER 3

State of the Practice

A questionnaire was developed to determine DOT practices regarding the use and applica-
tion of AI technologies with an APCS. The questionnaire was provided to the 50 state DOTs,
the District DOT, and the Puerto Rico Highway and Transportation Authority. Agencies were
questioned about the use, benefits, and challenges of AI technology; APCS data collection; and
the use of pavement condition survey results.
The intended recipients of the survey questionnaire were the pavement management (or com-
parable position) staff or persons responsible for the DOT pavement condition data collection
and analysis. The detailed questionnaire is provided in Appendix A, and the DOTs’ responses
are summarized in Appendix B.
Forty-three (43) of the 52 surveyed DOTs (83%) responded to the questionnaire. The number
of responses received to specific questions can be less than 43. In the text and table and figure
captions, the term n denotes the response sample size; the term n = x in the figure captions refers
to the total number (x) of responses to a specific survey question.

APCS
Agencies were asked whether they currently conduct an APCS to quantify pavement surface
distress (excluding inertial profile measurements). Thirty-eight (38) of 43 DOTs (88%) responded
in the affirmative (Figure 23a). DOTs that indicated not using an APCS (5 of 43 DOTs, 12%) were
excluded from responding to the remainder of the questionnaire.
The 38 DOTs that conducted an APCS were asked whether other technologies (e.g., smart-
phones, more frequent assessment of safety-related distress) were used to complement the APCS.
Thirty-one (31) of 38 DOTs (82%) indicated “no,” 2 of 38 DOTs (5%) indicated “not sure,” and
5 of 38 DOTs (13%) indicated “yes” (Figure 23b). The additional technologies noted by the DOTs
included:
• Falling weight deflectometer and friction testing (Iowa);
• Laser Crack Measurement System (LCMS), LiDAR, Sideway-force Coefficient Routine Inves-
tigation Machine (SCRIM), and a visual survey (Kentucky);
• Roadway and right-of-way imagery and mobile LiDAR (Montana);
• A pavement condition rating (PCR) method to support pavement management system processes
(Ohio); and
• Visual rating on approximately 6% of the network for audit sections (Texas).

Agencies were asked to indicate who collected and analyzed the APCS (survey question #6).
Twenty-two (22) of 38 respondents (58%) indicated using a vendor for data collection and analysis,
12 of 38 respondents (32%) indicated using DOT personnel and DOT-owned and operated

32

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

State of the Practice   33

Not sure, 2

Question 1: Does your agency Question 2: Does your agency use

conduct an APCS? (n=43) other technology to complement the
APCS? (n=38)

Figure 23.   Agency use of APCSs.

Question 6: Is the APCS conducted using DOT equipment and staff or

contracted through a vendor?

Figure 24.   APCS data collection and analysis (n 5 38).

equipment, and 4 of 38 DOTs (11%) indicated using a combination of DOT and vendor equipment
and personnel to complete the APCS (Figure 24). In addition, all 38 DOTs indicated using one of the
same five vendors (survey question #7). Of these vendors, two are used by 29 of the 38 DOTs (76%).

AI Technology
As noted in Chapter 2, the use of AI technology with the APCS has gained popularity over
recent years. Agencies were asked a series of questions related to the current and future use of
AI along with the use of new and upcoming technologies. Eight (8) of 38 DOTs (21%) indicated
currently using AI technology to process the APCS, 12 of 38 DOTs (32%) indicated being unsure
whether AI was used (i.e., either the analysis is conducted using vendor software or the vendor
conducts an APCS), and 18 of 38 DOTs (47%) indicated AI was not used to process the APCS
(Figure 25b) (survey question #4). When asked whether they would consider using AI technology
in the future, 18 of 30 DOTs (60%) that already use AI indicated “yes,” and 12 of 30 DOTs (40%)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

34   AI Applications for Automatic Pavement Condition Evaluation

Question 3: Do you think the new Question 4: Does your agency (or Question 5: Would your agency
technology (e.g., autonomous vendor) use AI technology to consider using AI in the future to
vehicle, crowdsource) may be used process the APCS (n=38)? collect and process the automated
for future AI-based automated pavement condition data (n=30)?
pavement condition data collection
(n=38)?

Figure 25.   Use of AI.

indicated being unsure (Figure 25c) (survey question #5). Regarding the use of new technologies
(e.g., autonomous vehicles, crowdsource), 15 of 38 DOTs (39%) indicated they may be used for
future AI-based APCS, 18 of 38 DOTs (47%) indicated being unsure, and 5 of 38 DOTs (13%)
indicated “no” (Figure 25a) (survey question #3).
As described in Chapter 2, a variety of models and tools are available with AI technology.
Twenty-four (24) of 38 DOTs (63%) responding to the overall survey answered survey ques-
tion #16. Sixteen (16) of 24 DOTs (67%) indicated they were unsure which models were used
(Figure 26). Two (2) of 24 DOTs (8%) indicated using RF and ML, 1 of 24 DOTs (4%) indicated
using neural network, and 1 of 24 DOTs (4%) indicated using pattern recognition. No DOT
indicated using deep learning. Two (2) of 24 DOTs (8%) answered “other.” Texas noted that

Not Sure 16

Random Forest 2

Machine Learning 2

Neural Network 1

Pattern Recognition 1

Deep Learning 0

No. of Agencies
Question 16: What AI technologies, tools, and models does your agency
currently use for pavement condition evaluation (select all that apply)?

Figure 26.   Currently used AI technology, tools, and

models (n 5 24).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

State of the Practice   35

Not Sure 15

Accuracy, Precision, and Repeatability 6

Compare to Manual Surveys 5

Random Reference Sections 3

Predefined Reference Sections 3

Compare to Traditional APCS 2

Google Earth Images for AI Training 0

No. of Agencies
Question 17: How does your agency conduct AI-technique development, training,
and evaluation (e.g., ground truth testing) (select all that apply)?

Figure 27.   Methods for AI technique development, training,

and evaluation (n 5 27).

research is being conducted to evaluate AI models, and Utah is evaluating ML and DL models
to use with the APCS.
Question 18: Is your agency
When asked how agencies conducted AI technique development, training, and evaluation AI process in accordance with
(Figure 27) (survey question #17), AASHTO R 85 (AASHTO
2018)?
• 15 of 27 DOTs (56%) indicated being unsure;
Figure 28.   AI process
• 6 of 27 DOTs (22%) indicated requiring the evaluation of accuracy, precision, and repeatability; in compliance with
• 5 of 27 DOTs (19%) compared the AI results to a manual survey; AASHTO R 85
• 3 of 27 DOTs (11%) compared the AI results to random or predefined reference sections; (AASHTO 2018)
• 2 of 27 DOTs (7%) compared the AI results to the results from the traditional APCS; and (n 5 27 responses).
• No DOT used Google Earth images for AI training.
Florida indicated using manual image classification, and Tennessee indicated providing the
vendor with the distress library to train the AI models. (Note: The values of n do not add up to
27 because agencies were allowed to select multiple responses for this question.)
As described in Chapter 2, protocols and methods exist for crack detection using an APCS.
Respondents were asked to indicate whether their DOT required the AI process to be in accor-
dance with AASHTO R 85 (AASHTO 2018) (survey question #18). Twenty-one (21) of 27 DOTs
(78%) indicated they were unsure, 2 of 27 DOTs (7%) indicated the AI process was in accordance
with AASHTO R 85 (AASHTO 2018), and 2 of 27 DOTs (7%) indicated the AI process was
not in accordance with AASHTO R 85 (AASHTO 2018) (Figure 28). Alabama and Michigan Question 19: Can your
(2 of 27 DOTs, 7%) indicated the AI process was mostly in accordance with AASHTO R 85 agency’s AI process be used
(AASHTO 2018) (e.g., DOT requires the use of a 3 ft wheel path width). on historical records (i.e.,
archived videos or images)?
Agencies were asked whether the AI process can be used on historical records (survey ques- Figure 29.   Use of
tion #19). Seventeen (17) of 25 DOTs (68%) indicated being unsure, 4 of 25 DOTs (16%) indicated AI process with
“no,” and 3 of 25 DOTs (12%) indicated “yes” (Figure 29). One (1) of 25 DOTs (4%) indicated the historical records
AI process was applicable for historical records if the data were collected using an LCMS. (n 5 25 responses).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

36   AI Applications for Automatic Pavement Condition Evaluation

Data and Image Requirements

Agencies were asked to respond to multiple choice questions related to the APCS data and
image requirements (survey question #8). Figure 30 provides a summary of responses to APCS
requirements. DOTs indicated requiring the APCS to include:

• an assessment of distress type (38 of 38 DOTs, 100%), severity (38 of 38 DOTs, 100%), and
extent (35 of 38 DOTs, 92%);
• utilization of a linear referencing system (37 of 38 DOTs, 97%); and
• requirements for a specific reporting interval (31 of 38 DOTs, 82%), data repeatability (30 of
38 DOTs, 79%), precision and accuracy (29 of 38 DOTs, 76%), and several other requirements
(shown in Figure 30).

Delaware and Oregon also indicated the APCS is conducted in accordance with the data dic-
tionary provided to the vendor and with DOT distress protocols, respectively. (Note: The values
of n in Figure 30 do not add up to 38 because agencies were allowed to select multiple responses
for this question.)

Agencies were asked to indicate whether any additional data and image requirements were
needed for AI training and post processing (Figure 31) (survey questions #9 and #10). A majority
of agencies (29 of 36 DOTs, 81%) were unsure whether any additional data format requirements
were needed. Six (6) of 36 DOTs (17%) indicated no additional data format requirements were
needed for the AI data analysis. Similarly, 28 of 37 DOTs (76%) were unsure whether any addi-
tional image quality requirements were needed, while 5 of 37 DOTs (14%) indicated no additional
requirements were needed for the AI images analysis.

Florida noted the data collection and analysis were conducted in accordance with the DOT’s
data libraries and required images (i.e., in .jpg format) to quantify raveling. Regarding image
quality, Alaska indicated at least a 2mm resolution based on FAA research, although it was not
entirely sure, and Colorado noted it needed images with similar resolution images. For example,
the images collected from previous years may not have as high a resolution as the images collected

Question 8: Requirements for automated pavement distress identification include

(select all that apply).

Figure 30.   APCS requirements (n 5 38 responses).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

State of the Practice   37

Question 9: What data format requirements are needed for AI training/post-processing? (n=36)
Question 10: What image quality requirements are needed to conduct AI training/post-processing
(n=37)?

Figure 31.   Additional data and image quality requirements for AI

training/post-processing.

with current technology. Florida noted an image resolution of 4mm × 4mm pixels, and Maryland
indicated the image size and resolution as important characteristics (e.g., free of artifacts and
shadows).

Quantifying Distress
Pavement condition is quantified through the collection of pavement distress type, extent,
and severity according to each DOT’s condition definitions. Figures 32 and 33 provide a summary
of responses related to the use of AI technologies to identify distress on asphalt- and concrete-
surfaced pavements, respectively (survey questions #13 and #14). Eight (8) of 38 DOTs (21%)
indicated using AI to assess asphalt pavement condition. Quantified distress types include:
• transverse and longitudinal cracking (7 of 8 DOTs, 88%),
• alligator cracking (5 of 8 DOTs, 63%),
• block cracking and delaminations/potholes (4 of 8 DOTs, 50%),
• edge cracking (2 of 8 DOTs, 25%), and
• raveling (1 of 8 DOTs, 13%).
No DOT indicated using AI technology to quantify bleeding or weathering.
Eleven (11) of 27 DOTs (41%) indicated being unsure whether AI technologies were used to
identify (a) transverse and longitudinal cracking (5 of 11 DOTs, 45%), (b) alligator and block
cracking (4 of 11 DOTs, 36%), (c) patching and edge cracking (3 of 11 DOTs, 27%), (d) raveling
(2 of 11 DOTs, 18%), and (e) delaminations/potholes and bleeding (1 of 11 DOTs, 9%). No DOT
indicated identifying weathering. (Note: The values of n in Figure 32 do not add up to 27 because
agencies were allowed to select multiple responses for this question.)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

38   AI Applications for Automatic Pavement Condition Evaluation

Transverse Cracking 7 5
Longitudinal Cracking 7 5
Alligator Cracking 5 4
Block Cracking 4 4
Delaminations/Potholes Use AI
4 1
Not Sure
Patching 3 3
Edge Cracking 2 3
Raveling 1 2
Bleeding 1
Weathering 0

No. of Agencies
Question 13: What asphalt-surfaced pavement condition types does your
agency evaluate using AI technology (select all that apply)?

Figure 32.   Asphalt-surfaced distress types

(n 5 27 responses).

In addition, state DOTs provided the following comments:

• Alaska quantifies cracking as wheel path and non-wheel path cracking rather than alligator
cracking;
• South Carolina classifies all wheel path cracking as fatigue cracking;
• Utah has conducted research using AI technology to quantify cracking, patching, and pot­
holing; and
• Washington State uses the vendor crack rating system to conduct quality control of the manual
“visual” rating.

Pavement Type Not Used 9

Transverse Cracking 6 4
Longitudinal Cracking 5 4
Spalling 4 1
Multi-cracked Slab 3 Use AI
Corner Cracking 3 Not Sure
Patching 2 2
Punchout 2
Joint Seal Damage 2
Map Cracking 1
Scaling 0
Polished Aggregate 0
Pumping 0
Blowups 0
No. of Agencies
Question 14: What concrete-surfaced pavement condition types does your
agency evaluate using AI technology (select all that apply)?

Figure 33.   Concrete-surfaced distress types

(n 5 23 responses).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

State of the Practice   39

For concrete pavement distress, seven DOTs indicated using AI technology to quantify:

• transverse cracking (6 of 7 DOTs, 86%),

• longitudinal cracking (5 of 7 DOTs, 71%),
• spalling (4 of 7 DOTs, 57%),
• multi-cracked slabs and corner cracking (3 of 7 DOTs, 43%),
• patching and punchouts (2 of 7 DOTs, 29%), and
• map cracking (1 of 7 DOTs, 14%).

No DOT indicated using AI technology to identify blowups, joint seal damage, polished
aggregate, pumping, or scaling.

Nine (9) agencies indicated that concrete pavements were not used. The seven DOTs that
indicated being unsure whether AI technologies were used reported quantifying:

• transverse and longitudinal cracking (4 of 7 DOTs, 57%),

• patching and joint seal damage (2 of 7 DOTs, 29%), and
• spalling (1 of 7 DOTs, 14%).

In addition, DOTs that indicated being unsure whether AI technology was used also indicated
blowups, corner cracking, map cracking, multi-cracked slabs, polished aggregate, pumping, and
scaling were not included in the APCS.

The APCS provides a number of additional capabilities for roadside asset data collection and
analysis. Specifically, agencies were asked to indicate whether additional roadway features (e.g.,
slope embankment, excess vegetation growth, markings, signs) were assessed as part of the APCS
(survey question #15). Six (6) of 8 agencies (75%) reported that roadside assets (e.g., markings,
signs) were collected. Georgia noted it was in the process of evaluating AI technology to assess
roadside assets and excess vegetation growth; Utah indicated it was in the process of assessing
roadside assets. (Note: The values of n in Figure 33 do not add up to 23 because agencies were
allowed to select multiple responses for this question.)

Use of APCS Results

Specific to the pavement management activities, agencies were asked to indicate how the
APCS results were used (survey question #11) (Figure 34). Thirty-eight (38) of 38 DOTs indi-
cated the APCS results were used for the HPMS reporting requirements. In addition, 36 of
38 (95%) DOTs indicated using the APCS results to determine a pavement condition rating
or index and for pavement performance modeling. Other usage of the APCS results included
reporting for the Moving Ahead for Progress in the 21st Century Act (MAP-21) (33 of 38 DOTs,
87%), detecting individual and prevalent distress types (29 and 22 of 38 DOTs, 76% and 58%,
respectively), and road safety assessment (11 of 38 DOTs, 29%). Alabama indicated that the
APCS results are used to screen potential preservation treatment types (e.g., microsurfacing
may require two passes when rutting is greater than 0.25 in.). Oregon uses the results to deter-
mine state pavement condition performance measures (i.e., percent of miles rated “fair” or
better). Texas utilizes the results in support of developing pavement management plans. (Note:
The values of n in Figure 34 do not add up to 38 because agencies were allowed to select multiple
responses for this question.)

Pavement condition data are routinely used to support agency decisions related to bud-
geting, multi-year planning, and establishing performance targets. Agencies were asked to
identify decision-making activities based, in part, on the results of the APCS (survey question #12)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

40   AI Applications for Automatic Pavement Condition Evaluation

HPMS Reporting 38

Pavement Condition Rating or Index 36

Pavement Performance Modeling 36

MAP-21 Reporting 33

Detecting Individual Distress Type 29

Detecting Prevalent Distress Type 22

Road Safety Assessment 11

No. of Agencies
Question 11: Pavement surface condition data are used for (select all that apply).

Figure 34.   Use of APCS results (n 5 38 responses).

(Figure 35). (Note: The values of n in Figure 35 do not add up to 38 because agencies were
allowed to select multiple responses for this question.) Agencies indicated APCS results were
used to:
• verify performance models (32 of 38 DOTs, 84%),
• establish performance targets and goals (31 of 38 DOTs, 82%),
• determine treatment selection (31 of 38 DOTs, 82%),
• conduct multi-year planning (31 of 38 DOTs, 82%),
• conduct budgeting (27 of 38 DOTs, 71%),

Question 12: Pavement condition data is used to support the following agency
decision-making activities (select all that apply).

Figure 35.   APCS results in support of agency decision-

making activities (n 5 38 responses).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

State of the Practice   41

• establish performance specifications and measures (14 of 38 DOTs, 37%),

• trigger safety-related repairs (10 of 38 DOTs, 26%), and
• approximate bid quantities (9 of 38 DOTs, 24%).

Additional DOT uses included the following:

• Alabama: in support of transportation asset management;
• Maryland: in support of transportation asset management and system preservation;
• Ohio: Federal reporting; and
• South Carolina: contract performance (monitored by another division within the DOT).

AI Challenges and Benefits

As with any technology, the evaluation and implementation of AI involve challenges. Agencies
were asked to provide challenges (Figure 36) (survey question #20) and benefits (Figure 37) (survey
question #21) of using AI. Noted challenges included:
• limited DOT knowledge (14 of 25 DOTs, 56%),
• ground truth testing (10 of 25 DOTs, 40%),
• trusting the AI results (9 of 25 DOTs, 36%),
• computer computation capabilities (5 of 25 DOTs, 20%), and
• unsure (2 of 25 DOTs, 8%).
In addition, Delaware indicated relying on the expertise of the vendor. Washington State noted
challenges with distinguishing multiple crack types within a given area and mistakenly quantify-
ing paving joints and chipped paint or shoulder rumble strips as cracking.
Noted benefits included:
• objectivity with the AI process (14 of 25 DOTs, 56%),
• improved accuracy and increased productivity (13 of 25 DOTs, 52%), and
• cost savings (7 of 25 DOTs, 28%).

Limited Agency Knowledge 14

Ground Truth Testing 10

Trusting Results 9

AI Training 6

Computer Computation Capabilities 5

Not Sure 2

No. of Agencies
Question 20: What challenges does your agency have with the current AI process
for APCSs (select all that apply)?

Figure 36.   AI process challenges (n 5 25 responses).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

42   AI Applications for Automatic Pavement Condition Evaluation

Objectivity 14

Accuracy 13

Increased Productivity 13

Cost 7

Not Sure 1

No. of Agencies
Question 21: What are the agency benefits of using AI for processing the
APCS (select all that apply)?

Figure 37.   AI process benefits (n 5 25 responses).

However, one responding DOT was unsure about the benefits of the AI process. (Note: The values
of n in Figure 37 do not add up to 25 because agencies were allowed to select multiple responses
for this question.)

Summary
An online questionnaire was prepared and distributed to 52 agencies (i.e., the 50 state DOTs, the
District DOT, and the Puerto Rico Highway and Transportation Authority). In total, 43 agencies
(83%) responded to the survey. Agencies were asked to respond to questions related to the APCS,
AI technology, data and image requirements, distress types evaluated, use of APCS results, and the
challenges and benefits of using AI. Key findings from the DOT questionnaire included:
• Thirty-eight (38) of 43 DOTs (88%) reported using an APCS to quantify pavement conditions.
Of these,
– 5 of 43 DOTs (12%) indicated using other condition assessment activities in support of
the APCS, including falling weight deflectometer (FWD) testing, friction testing, LCMS,
LiDAR, SCRIM, PCR, and visually rating a portion of the network for auditing the APCS;
– 22 of 38 DOTs (58%) indicated a vendor conducted the APCS;
– 12 of 38 DOTs (32%) indicated conducting the APCS using DOT-purchased equipment
and staff; and
– 4 of 38 DOTs (11%) indicated using a combination of DOT and vendor equipment and staff.
• Eight (8) of 38 DOTs (21%) reported AI technology was used to analyze the APCS, 18 of 38
DOTs indicated it was not used, and 12 of 38 DOTs (32%) were unsure about its use. Of the
30 agencies that indicated AI technology was not used or were unsure whether it was used:
– 18 of 30 DOTs (60%) noted they would consider its use in the future, and
– 12 of 30 DOTs (40%) indicated they were unsure about its future use.
• New technologies (e.g., smartphone applications) are emerging for pavement condition data
assessment. Agencies were asked whether these technologies are used to complement the APCS:
– 15 of 38 DOTs (39%) indicated “yes,”
– 5 of 38 DOTs (13%) indicated “no,” and
– 18 of 38 DOTs (47%) indicated they were unsure.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

State of the Practice   43

• APCS analysis using AI technology requires a model to change from an image to distress
detection. Agencies were asked to identify which models were used for the APCS analysis:
– 16 of 24 DOTs (67%) indicated they were unsure,
– 2 of 24 DOTs (8%) indicated using RF or ML, and
– 1 of 24 DOTs (4%) indicated using neural networks or pattern recognition.
• The development, training, and evaluation of AI technology requires several activities. Agencies
were asked to indicate the activities used to evaluate the AI results:
– 15 of 27 DOTs (56%) indicated being unsure how the AI results were developed, trained,
or evaluated;
– 6 of 27 DOTs (22%) indicated using accuracy, precision, and repeatability criteria;
– 5 of 27 DOTs (19%) indicated comparing the AI results to manual survey results;
– 3 of 27 DOTs (11%) indicated using random or predefined reference sections for com-
parisons; and
– 2 of 27 DOTs (7%) indicated comparing the AI results to traditional APCS results.
• Regarding distress protocols:
– 2 of 27 DOTs (7%) indicated the AI process complied with AASHTO R 85 (AASHTO 2018),
– 2 of 27 DOTs (7%) indicated the process was very similar to AASHTO R 85 (AASHTO
2018),
– 2 of 27 DOTs (7%) indicated the AI process did not comply with AASHTO R 85 (AASHTO
2018), and
– 21 of 27 DOTs (78%) indicated being unsure.
• Agencies have been conducting APCSs for decades, but AI technology is a more recent devel-
opment. Three (3) of 25 DOTs (12%) indicated the AI process could be used on historical
records, 3 of 25 DOTs (12%) indicated it could not be used, and 16 of 25 DOTs (64%) were
unsure.
• To obtain quality results, several requirements for APCS data and images must be met. Thirty-
eight (38) of 38 DOTs (100%) required distress type and severity as part of the APCS. Additional
requirements included:
– a linear referencing system (37 of 38 DOTs, 97%),
– distress extent (35 of 38 DOTs, 92%),
– reporting interval (31 of 38 DOTs, 82%),
– data repeatability (30 of 38 DOTs, 79%),
– precision and accuracy (29 of 38 DOTs, 76%),
– compatibility with the existing pavement management system (24 of 38 DOTs, 63%),
– spatial resolution (19 of 38 DOTs, 50%),
– sampling rate (15 of 38 DOTs, 39%), and
– productivity (11 of 38 DOTs, 29%).
• Eight (8) of 38 DOTs (21%) indicated AI technology was used to identify asphalt surface dis-
tresses, and 7 of 38 DOTs (18%) indicated using it to identify concrete surface distresses. For
asphalt pavements, the predominant distress types identified included transverse and longitu-
dinal cracking (7 of 8 DOTs, 88%) and alligator cracking (5 of 8 DOTs, 63%). For concrete pave-
ments, predominant distress types included transverse (6 of 7 DOTs, 86%) and longitudinal
cracking (5 of 7 DOTs, 71%).
• The results of the APCS are predominantly used for HPMS (38 of 38 DOTs, 100%) and MAP-21
reporting (33 of 38 DOTs, 87%), pavement condition assessment and performance modeling
(36 of 38 DOTs, 95%, each) and detecting prevalent distress type (22 of 38 DOTs, 58%).
• APCS results are also used to support DOT decisions. Most agencies indicated using the APCS
results to:
– verify performance models (32 of 38 DOTs, 84%),
– establish performance targets (31 of 38 DOTs, 82%),
– target performance goals (31 of 38 DOTs, 82%),

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

44   AI Applications for Automatic Pavement Condition Evaluation

– select treatment types (31 of 38 DOTs, 82%), and

– plan multiyear budgets (31 of 38 DOTs, 82%).
• AI technology for APCS analysis has been used for years and has generated several benefits.
Benefits noted by the DOTs included:
– objectivity (14 of 25 DOTs, 56%),
– accuracy and increased productivity (13 of 25 DOTs, 52%, each), and
– cost savings (7 of 25 DOTs, 28%).
• Challenges noted by the DOTs regarding the use of AI technology for APCS analysis included:
– accuracy, precision, and repeatability (6 of 24 DOTs, 25%);
– comparison to manual surveys (3 of 24 DOTs, 13%);
– use of random and predefined reference sections (3 of 24 DOTs, 13%, each); and
– comparison to traditional APCS (2 of 24 DOTs, 8%).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

CHAPTER 4

Case Examples

The development of case examples describing individual DOT practices is an integral part
of the synthesis effort. However, the relatively recent development and implementation of 3D
technology and the use of AI for APCS analysis resulted in difficulties identifying any DOT able
to provide details and specifics for case example development. Therefore, this chapter provides a
general summary of efforts made for AI model development and training.
One method for quantifying distress is to evaluate the APCS results using ASTM D6433
(ASTM 2023) and/or AASHTO R 85 (AASHTO 2018). Typically, the collected data, preferably
using AASHTO R 86 (AASHTO 2022), are divided into two data sets, one for training (15 to
20% of the entire data set) and one for control (80 to 85% of the entire data set). The training
data set is used to fine-tune the algorithm to ensure different distresses (or distress indices) are
correctly identified for the specific pavement network. Because the crack identification can vary
significantly between different pavement distress and conditions, the training data set should
include distress types and conditions (i.e., good, fair, poor, and very poor) representative of the
pavement network being surveyed. As part of the training stage, it is also recommended to con-
duct a manual quality control pavement condition survey to ensure the algorithms are correctly
and accurately identifying and measuring the distresses (or indices). Typically, the following
three methods are used to conduct the manual data quality control (QC):
• Method 1: The most detailed, time-consuming, and costly approach includes experienced
technicians or engineers conducting manual detailed distress data collection (i.e., ground
truth testing). Ideally, this method includes collecting detailed pavement distress data on
15 to 20% of the pavement network.
• Method 2: A more cost-effective approach, and more typically used, is to conduct a detailed
desktop pavement distress survey using the images collected during the APCS. As part of this
method, experienced technicians or engineers utilize special software to view, identify, and
quantify distress type, severity, and extent on 15 to 20% of the pavement network.
• Method 3: In this method, limited manual distress data collection is carried out, followed by
an extensive desktop pavement condition survey (combination of Method 1 and 2). If the
budget allows, this approach is the most preferred because the in-field survey provides ground
truth information, which may be difficult to capture through images and/or a crack map alone.
Based on a comparison of the manual QC and the output from the general algorithm, the
general algorithm is modified to ensure a high percentage (> 90%) of the output from the algo-
rithm matches the manual data QC. Once this match is achieved, the algorithm is tailored to the
network data and can be applied to the control data set. To ensure the quality of the final output,
another round of QC testing using Method 2 is recommended for evaluating the control data set.
Figure 38 illustrates the overall process for developing and calibrating an algorithm for distress
prediction at the network level.

45

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

46   AI Applications for Automatic Pavement Condition Evaluation

Figure 38.   Overall process for developing and calibrating an asphalt pavement distress algorithm.

After multiple iterations over different networks, the algorithm is usually modified enough that
it tends to predict the distresses with a high level of accuracy. If this is true, the training data set
can be further reduced to less than 10% of the total data set and divided into two data sets, one for
calibration (5%) and one for spot checks (5%). The calibration data set is used to test and/or cali-
brate the modified algorithm to ensure accurate predictions. Once the algorithm matches a higher
percentage (> 90%) of the calibration data, it should then be checked against the check data set.
If the algorithms are still correctly predicting a higher percentage (> 90%) of the check data
set, then it can be applied to the remainder of the data set. Spot checks and QC utilizing Method 2
should be performed to ensure the final output reflects the ground truth testing results.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

CHAPTER 5

Conclusions

The progression of pavement condition surveys from manual to fully automated methods
has provided opportunities to use AI technologies for a more efficient and accurate detection of
pavement surface distresses (e.g., cracking, rutting, potholing). As with conventional pavement
condition surveys, AI-based analysis requires model development (or selection), training, and
validation.

The objective of this synthesis is to document state DOT practices of automated pavement
distress identification and AI (i.e., ML/DL) technologies for pavement condition evaluation.

The synthesis is based on the results of a literature review and a survey of state DOTs, the
District DOT, and the Puerto Rico Highway and Transportation Authority. As discussed in
Chapter 4, no case examples could be developed due to a lack of DOTs with the necessary experi-
ence with AI technology.

Overall Findings
Several AI modeling techniques are currently available and used for APCS analysis; these
models include DL (e.g., classification, object detection, and semantic segmentation), ML (e.g.,
SVM, DT, RF, boosting, and ANN), and image processing (e.g., edge detection, threshold, and
manual filters) methods. While AI technologies improve the efficiency and accuracy of APCS
analysis, acceptance of AI results requires ground truth testing and model validation.

In total, 43 of the 52 surveyed DOTs (83%) responded to the survey, which asked questions about
APCS, AI technology, data and image requirements, distress types evaluated, the use of APCS
results, and the challenges and benefits of using AI. The majority of DOTs (38 of 43, 88%) indi-
cated using an APCS to quantify pavement conditions. However, only 8 of 38 DOTs (21%) indicated
that AI technology was used to analyze the APCS. Predominant distress types evaluated using AI
technology included transverse and longitudinal cracking (7 of 8 DOTs, 88%, for both) and alligator
cracking (5 of 8 DOTs, 63%) for asphalt pavements and transverse and longitudinal cracking (6 and
5 of 7 DOTs, 86% and 71%, respectively) for concrete pavements.

AI technology provided several benefits, including objectivity (14 of 25 DOTs, 56%), accuracy
and increased productivity (13 of 25 DOTs, 52%), and cost savings (7 of 25 DOTs, 28%). Con-
versely, some of the challenges DOTs faced with AI technology included accuracy, precision, and
repeatability (6 of 24 DOTs, 25%); comparison to manual surveys (3 of 24 DOTs, 13%); use of
random and predefined reference sections (3 of 24 DOTs, 13%); and comparison to traditional
APCS (2 of 24 DOTs, 8%).

47

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

48   AI Applications for Automatic Pavement Condition Evaluation

Suggestions for Future Research

Suggested future research for improving the use of AI technology in analyzing APCS results
includes the following:
• Images for AI Technology. Having a standard image format, data set, and distress library
would significantly benefit agencies in developing, calibrating, training, and validating AI
models. Develop a standard image format (e.g., .jpeg, .png, or a private extension) and a
publicly available distress data set (including, pavement type, distress type, and severity level)
to assess APCS results using AI technology.
• Tool for Distress Library. Develop a software tool, using traditional computer vision tech-
niques, to assist agencies in building the AI distress library.
• Guidance for Using AI Models. Prepare a guidance document to assist agencies in using and
implementing AI technology in an APCS. The guidance document is intended to provide
detailed information related to AI technology (including ML/DL); selection, development,
or modification of existing AI models; model training; calibration and recalibration; and
approaches for validation, quality control, and acceptance.
• AI Analysis of Historical Data. Agencies have been conducting APCS for decades; however,
technology and data collection protocols have changed over this same time. Develop a meth-
odology to use AI models to interpret APCS results from older technology and compare data
collection and analysis protocols.
• APCS Data Collection Framework. APCS can generate large volumes of data, especially for
large roadway networks. In addition, storage and management of numerous years of data can
be challenging. Develop a framework for data collection, storage, and management.
• AI Training Course. AI technology is being used by APCS vendors; however, how AI is used
for distress detection is not clearly understood. Develop a training course to provide informa-
tion and educate participants on how AI technology is used to identify pavement distress.
• Update Current Distress Protocols. With the adoption of AI technologies, the current distress
protocols may require revisions. As part of this research, review current distress protocols,
identify gaps, and propose revisions to address the use of AI for distress detection.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

References

AASHTO. 2018. Standard Practice for Quantifying Cracks in Asphalt Pavement Surfaces from Collected Pavement
Images Utilizing Automated Methods. AASHTO R 85-18. AASHTO, Washington, DC.
AASHTO. 2022. Standard Practice for Collecting Images of Pavement Surfaces for Distress Detection. AASHTO
R 86-18. AASHTO, Washington, DC.
Aboah, A., and Y. Adu-Gyamfi. 2020. “Smartphone-based Pavement Roughness Estimation using Deep Learn-
ing with Entity Embedding.” Advances in Data Science and Adaptive Analysis. Volume 12, No. 03n04. World
Scientific.
Ahmad, T., V. Gharehbaghi, J. Li, C. Bennett, and R. Lequesne. 2023. “Crack Segmentation in the Wild Using Convo-
lutional Neural Networks and Bootstrapping.” Earthquake Engineering and Resilience, Volume 2(3), pp. 348–363.
Ahmadi, A., S. Khalesi, and A. Golroo. 2021. “An Integrated Machine Learning Model for Automatic Road Crack
Detection and Classification in Urban Areas.” International Journal Pavement Engineering, Volume 23(10),
3536–3552.
Ai, D., G. Jiang, L. Siew Kei, and C. Li. 2018. “Automatic Pixel-Level Pavement Crack Detection Using Informa-
tion of Multi-Scale Neighborhoods.” IEEE Access, Volume 6, pp. 24452–24463.
Ale, L., N. Zhang, and L. Li. 2018. Road Damage Detection using RetinaNet. 2018 IEEE International Conference
on Big Data (Big Data). IEEE. New York, NY.
Anand, S., S. Gupta, V. Darbari, and S. Kohli. 2018. “Crack-Pot: Autonomous Road Crack and Pothole Detection.”
Digital Image Computing: Techniques and Applications (DICTA), IEEE, pp. 1-6.
Arya, D., H. Maeda, S.K. Ghosh, D. Toshniwal, and Y. Sekimoto. 2021. “An Annotated Image Dataset for Auto-
matic Road Damage Detection using Deep Learning.” Data in Brief. Volume 36. Elsevier.
ASTM. 2023. Standard Practice for Roads and Parking Lots Pavement Condition Index Surveys. ASTM D6433.
ASTM International, West Conshohocken, PA.
Bridgelall, R. 2015. Pavement Performance Evaluation using Connected Vehicles. Doctoral dissertation. North
Dakota State University, Fargo, ND.
Bridgelall, R., and D. Tolliver. 2018. “Accuracy Enhancement of Roadway Anomaly Localization using Connected
Vehicles.” International Journal of Pavement Engineering, Volume 19, Issue 1. Taylor & Francis Online.
Cao, W., Q. Liu, and Z. He. 2020. “Review of Pavement Defect Detection Methods.” IEEE Access, Volume 8.
Cao, M.T., K.T. Chang, N.M. Nguyen, V.D. Tran, X.L. Tran, and N.D. Hoang. 2021. “Image Processing-Based
Automatic Detection of Asphalt Pavement Rutting using a Novel Metaheuristic Optimized Machine Learn-
ing Approach.” Soft Computing, Volume 25, pp. 2839–12855.
Cha, Y.J., W. Choi, and O. Büyüköztürk. 2017. “Deep Learning-based Crack Damage Detection using Convo-
lutional Neural Networks.” Computer-Aided Civil and Infrastructure Engineering, Volume 32, Issue 5. Wiley
Online.
Chen, T., Z. Cai, X. Zhao, C. Chen, X. Liang, T. Zou, and P. Wang. 2020. “Pavement Crack Detection and Recogni-
tion using the Architecture of segNet.” Journal of Industrial Information Integration, Volume 18.
Chitale, P.A., K.Y. Kekre, H.R. Shenai, R. Karani, and J.P. Gala. 2020. “Pothole Detection and Dimension Estima-
tion System using Deep Learning (Yolo) and Image Processing.” Proceedings, 35th International Conference
on Image and Vision Computing. IEEE.
Chou, J., W.A. O’Neill, and H.D. Cheng. 1994. “Pavement Distress Classification using Neural Networks.” Pro-
ceedings, International Conference on Systems, Man, and Cybernetics, Volume 1. IEEE.
Chun, P.J., T. Yamane, and Y. Tsuzuki. 2021. “Automatic Detection of Cracks in Asphalt Pavement using Deep
Learning to Overcome Weaknesses in Images and GIS Visualization.” Volume 11, Issue 3. Applied Sciences.
Commandre, B., D. En-Nejjary, L. Pibre, M. Chaumont, C. Delenne, and N. Chahinian. 2017. “Manhole Cover
Localization in Aerial Images with a Deep Learning Approach.” International Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences. Volume XLII-1/W1, pp. 6–9.

49

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

50   AI Applications for Automatic Pavement Condition Evaluation

Cord, A., and S. Chambon. 2012. “Automatic Road Defect Detection by Textural Pattern Recognition Based on
AdaBoost.” Computer-Aided Civil and Infrastructure Engineering. Volume 27, Issue 4.
Cubero-Fernandez, A., F.J. Rodriguez-Lozano, R. Villatoro, J. Olivares, and J.M. Palomares. 2017. “Efficient Pave-
ment Crack Detection and Classification.” Journal on Image and Video Processing. EURASIP.
Daniel, A., and V. Preeja. 2014. “Automatic Road Distress Detection and Analysis.” International Journal of
Computer Applications. Volume 101, Issue 10.
Del Rio-Barral, P., M. Soilan, S.M. González-Collazo, and P. Arias. 2022. “Pavement Crack Detection and Cluster­
ing via Region-growing Algorithm from 3D MLS Point Clouds.” Remote Sensing. Volume 14, Issue 22.
Du, Z., J. Yuan, F. Xiao, and C. Hettiarachchi. 2021. “Application of Image Technology on Pavement Distress
Detection: A Review.” Measurement. Volume 184. Elsevier.
Eisenbach, M., R. Stricker, D. Seichter, K. Amende, K. Debes, M. Sesselmann, D. Ebersbach, U. Stoeckert, and H.M.
Gross. 2017. “How to Get Pavement Distress Detection Ready for Deep Learning? A Systematic Approach.”
International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, pp. 2039–2047. IEEE.
Eriksson, J., L. Girod, B. Hull, R. Newton, S. Madden, and H. Balakrishnan. 2008. “The Pothole Patrol: Using a
Mobile Sensor Network for Road Surface Monitoring.” Proceedings, 6th International Conference on Mobile
Systems, Applications, and Services. Association for Computing Machinery, New York. NY.
Fan, R., M.J. Bocus, Y. Zhu, J. Jiao, L. Wang, F. Ma, S. Cheng, and M. Liu. 2019. “Road Crack Detection using
Deep Convolutional Neural Network and Adaptive Thresholding. IEEE Intelligent Vehicles Symposium (IV),
IEEE, pp. 474–479.
FHWA. 2015. The Long-Term Pavement Performance Program. FHWA-HRT-15-049. Washington, DC.
FHWA. n.d. InfoTechnology, https://infotechnology.fhwa.dot.gov/inertial-profiler-road-pavement/, accessed
October 23, 2023.
Gao, J., C.H. Ho, I. Wiese, D. Zhang, and M. Gerosa. 2021. “Application of Machine Learning Based Technology
in Pavement Condition Assessment and Prediction.” Paper No. TRBAM-21-03949. Transportation Research
Board, Washington, DC.
Ghasemi, P., M. Aslani, D.K. Rollins, R. Williams, and V.R. Schaefer. 2018. “Modeling Rutting Susceptibility of
Asphalt Pavement using Principal Component Pseudo Inputs in Regression and Neural Networks.” Inter­
national Journal of Pavement Research and Technology. Elsevier.
Guerrieri, M., and G. Parla. 2022. “Flexible and Stone Pavements Distress Detection and Measurement by Deep
Learning and Low-cost Detection Devices.” Journal of Engineering Failure Analysis. Volume 141. European
Structural Integrity Society.
Hoang, N.D., and Q.L. Nguyen. 2019. “A Novel Method for Asphalt Pavement Crack Classification Based on
Image Processing and Machine Learning.” Journal of Engineering with Computers. Volume 35.
Hoang, N.D., Q.L. Nguyen, and D. Tien Bui. 2018. “Image Processing–based Classification of Asphalt Pavement
Cracks using Support Vector Machine Optimized by Artificial Bee Colony.” Journal of Computing in Civil
Engineering. Volume 32, Issue 5. ASCE.
Hou, Y., Q. Li, C. Zhang, G. Lu, Z. Ye, Y. Chen, L. Wang, and D. Cao. 2021. “The State-of-the-art Review on
Applications of Intrusive Sensing, Image Processing Techniques, and Machine Learning Methods in Pavement
Monitoring and Analysis.” Engineering. Volume 7, Issue 6. Elsevier.
Hsieh, Y.A., and Y. Tsai. 2021. “Automated Asphalt Pavement Raveling Detection and Classification using Convo-
lutional Neural Network and Macrotexture Analysis.” Transportation Research Record 2675. Journal of the
Transportation Research Board, Washington, DC.
Huang, Y., and B. Xu. 2006. “Automatic Inspection of Pavement Cracking Distress.” Journal of Electronic Imaging.
Volume 15, Issue 1. SPIE and the Society for Imaging Science and Technology.
Huyan, J., W. Li, S. Tighe, Z. Xu, and J. Zhai. 2020. “CrackU-net: A Novel Deep Convolutional Neural Network
for Pixelwise Pavement Crack Detection.” Journal of Structural Control and Health Monitoring. Volume 27,
Issue 8. Wiley Hindawi.
Islam, S., W.G. Buttlar, R.G. Aldunate, and W.R. Vavrik. 2014. “Measurement of Pavement Roughness using
Android-based Smartphone Application.” Transportation Research Record 2457. Journal of the Transportation
Research Board, Washington, DC.
Konig, J., M.D. Jenkins, P. Barrie, M. Mannion, and G. Morison. 2019. “A Convolutional Neural Network for
Pavement Surface Crack Segmentation using Residual Connections and Attention Gating.” Proceedings -
International Conference on Image Processing, pp. 1460–1464.
Lei, X., C. Liu, L. Li, and G. Wang. 2020. “Automated Pavement Distress Detection and Deterioration Analysis
Using Street View Map.” IEEE Access. Volume 8, pp. 76163–76172.
Li, B., K.C.P. Wang, A. Zhang, E. Yang, and G. Wang. 2018. “Automatic Classification of Pavement Crack using
Deep Convolutional Neural Network.” International Journal of Pavement Engineering, 21(4), 457–463.
Li, B., K.C.P. Wang, A. Zhang, Y. Fei, and G. Sollazzo. 2019. “Automatic Segmentation and Enhancement of
Pavement Cracks Based on 3D Pavement Images.” Journal Advanced Transportation, Volume 2019, Article
ID 1813763.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

References  51

Li, H., D. Song, Y. Liu, and B. Li. 2019. “Automatic Pavement Crack Detection by Multi-Scale Image Fusion.”
IEEE Transactions on Intelligent Transportation Systems, Volume 20, No. 6, pp. 2025-2036.
Li, N., X. Hou, X. Yang, and Y. Dong. 2009. “Automation Recognition of Pavement Surface Distress Based on Sup-
port Vector Machine.” Second International Conference on Intelligent Networks and Intelligent Systems. IEEE.
Li, P., W. Chao, L. Shuangmiao, and F. Baocai. 2015. “Research on Crack Detection Method of Airport Runway
Based on Twice-threshold Segmentation.” Fifth International Conference on Instrumentation and Measure-
ment, Computer, Communication and Control. IEEE.
Lin, J., and Y. Liu. 2010. “Potholes Detection based on SVM in the Pavement Distress Image.” Proceedings,
9th International Symposium on Distributed Computing and Applications to Business, Engineering and Science,
Hong Kong, China. pp. 544–547.
Liu, Q., L. Sun, A. Kornhauser, J. Sun, and N. Sangwa. 2019. “Road Roughness Acquisition and Classification using
Improved Restricted Boltzmann Machine Deep Learning Algorithm.” Sensor Review. Volume 39, Issue 6.
Maeda, H., Y. Sekimoto, T. Seto, T. Kashiyama, and H. Omata. 2018. “Road Damage Detection and Classification
using Deep Neural Networks with Smartphone Images.” Computer-Aided Civil and Infrastructure Engineering.
Volume 33, pp. 1127-1141.
Majidifard, H., Y. Adu-Gyamfi, and W.G. Buttlar. 2020a. “Deep Machine Learning Approach to Develop a New
Asphalt Pavement Condition Index.” Construction and Building Materials. Volume 247. Elsevier.
Majidifard, H., P. Jin, Y. Adu-Gyamfi, and W.G. Buttlar. 2020b. “Pavement Image Datasets: A New Benchmark
Dataset to Classify and Densify Pavement Distresses.” Transportation Research Record 2674. Journal of the
Transportation Research Board, Washington, DC.
McGhee, K. 2004. NCHRP Synthesis 334: Automated Pavement Distress Collection Techniques. National Cooperative
Highway Research Program, Transportation Research Board, Washington DC.
Medina, J.R., R. Salim, B.S. Underwood, and K. Kaloush. 2020. “Experimental Study for Crowdsourced Ride
Quality Index Estimation using Smartphones.” Journal of Transportation Engineering, Part B: Pavements,
Volume 146, Issue 4. ASCE.
Miller, J. S., and W. Y. Bellinger. 2014. Distress Identification Manual for the Long-Term Pavement Performance
Program (Fifth Revised Edition). Report No. FHWA-HRT-13-092. FHWA, McLean, VA.
Moussa, G., and K. Hussain. 2011. “A New Technique for Automatic Detection and Parameters Estimation of
Pavement Crack.” 4th International Multi-Conference on Engineering Technology Innovation. Volume 2011.
IMETI.
Nguyen, T.S., M. Avila, and S. Begot. 2009. “Automatic Detection and Classification of Defect on Road Pavement
using Anisotropy Measure.” 17th European Signal Processing Conference. IEEE.
Nguyen, N.T.H., T.H. Le, S. Perry, and T.T. Nguyen. 2018. Pavement Crack Detection using Convolutional Neural
Network. Proceedings, 9th International Symposium on Information and Communication Technology.
Nie, M., and K. Wang. 2018. “Pavement Distress Detection Based on Transfer Learning.” 5th International Con-
ference on Systems and Informatics, pp. 435–439.
Oliveira, H., and P.L. Correia. 2009. “Automatic Road Crack Segmentation using Entropy and Image Dynamic
Thresholding.” 17th European Signal Processing Conference. IEEE.
Oliveira, H., and P.L. Correia. 2012. Automatic Road Crack Detection and Characterization. IEEE Transactions
on Intelligent Transportation Systems, Volume 14(1), pp. 155–168.
Pan, Y., X. Zhang, J. Tian, X. Jin, L. Luo, and K. Yang. 2017. “Mapping Asphalt Pavement Aging and Condition
using Multiple Endmember Spectral Mixture Analysis in Beijing, China.” Journal of Applied Remote Sensing,
Volume 11, No. 1.
Pan, Y., X. Zhang, G. Cervone, and L. Yang. 2018. “Detection of Asphalt Pavement Potholes and Cracks Based
on the Unmanned Aerial Vehicle Multispectral Imagery.” IEEE Journal of Selected Topics in Applied Earth
Observations and Remote Sensing, Volume 11, No. 10.
Pierce, L.M., and N.D. Weitzel. 2019. NCHRP Synthesis 531: Automated Pavement Condition Surveys. National
Cooperative Highway Research Program, Transportation Research Board, Washington DC.
Praveena, M., and V. Jaiganesh. 2017. “A Literature Review on Supervised Machine Learning Algorithms and
Boosting Process.” International Journal of Computer Applications. Volume 169, Issue 8.
Ranyal, E., A. Sadhu, and K. Jain. 2022. “Road Condition Monitoring using Smart Sensing and Artificial Intel-
ligence: A Review.” Sensors. Volume 22, Issue 8.
Schapire, R.E. 2003. “The Boosting Approach to Machine Learning: An Overview.” Nonlinear Estimation and
Classification. Springer, New York, NY.
Schapire, R.E., and Y. Freund. 2013. “Boosting: Foundations and Algorithms.” Kybernetes. Volume 42, Issue 1.
Shah, S., and C. Deshmukh. 2019. “Pothole and Bump Detection using Convolution Neural Networks.” Trans-
portation Electrification Conference (India). IEEE.
Sholevar, N., A. Golroo, and S.R. Esfahani. 2022. “Machine Learning Techniques for Pavement Condition Evalu-
ation.” Automation in Construction. Volume 136.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

52   AI Applications for Automatic Pavement Condition Evaluation

Song, W., G. Jia, H. Zhu, D. Jia, and L. Gao. 2020. “Automated Pavement Crack Damage Detection using Deep
Multiscale Convolutional Features.” Journal of Advanced Transportation.
Takanashi, M., Y. Ishii, S.I. Sato, N. Sano, and K. Sanda. 2020. “Road-Deterioration Detection using Road Vibra-
tion Data with Machine-learning Approach.” International Conference on Prognostics and Health Manage-
ment. IEEE.
Tsai, Y.C.J., and F. Li. 2012. “Critical Assessment of Detecting Asphalt Pavement Cracks under Different Lighting
and Low Intensity Contrast Conditions using Emerging 3D Laser Technology.” Journal of Transportation
Engineering. Volume 138, Issue 5.
UK Department for Transport. 2021. Technical Note: Road Condition and Maintenance Data. Network Condition
& Geography Statistics Branch, London, England.
USAF. 2004. Engineering Technical Letter (ETL) 04-9: Pavement Engineering Assessment (EA) Standards. Tyndall
Air Force Base, FL.
Wang, K.C., J.Q. Li, and G. Yang. 2020. NCHRP Web-Only Document 288: Standard Definitions for Common
Types of Pavement Cracking. National Cooperative Highway Research Program, Transportation Research
Board, Washington, DC.
Wang, S., S. Qiu, W. Wang, D. Xiao, and K.C. Wang. 2017. “Cracking Classification using Minimum Rectangular
Cover–based Support Vector Machine.” Journal of Computing in Civil Engineering, Volume 31, Issue 5.
Wang, X., and Z. Hu. 2017. “Grid-based Pavement Crack Analysis using Deep Learning.” 4th International
Conference on Transportation Information and Safety. IEEE.
Xu, Y., and Z. Zhang. 2022. “Review of Applications of Artificial Intelligence Algorithms in Pavement Manage-
ment.” Journal of Transportation Engineering, Part B: Pavements. Volume 148, Issue 3.
Yang, F., L. Zhang, S. Yu, D. Prokhorov, X. Mei, and H. Ling. 2019. “Feature Pyramid and Hierarchical Boosting
Network for Pavement Crack Detection.” Transactions on Intelligent Transportation Systems. Volume 21,
Issue 4. IEEE.
Yang, F., L. Zhang, S. Yu, D. Prokhorov, X. Mei, and H. Ling. 2020. “Feature Pyramid and Hierarchical Boosting
Network for Pavement Crack Detection.” IEEE Transactions on Intelligent Transportation Systems. Volume 21,
pp. 1525–1535.
Zhang, A., K.C.P. Wang, B. Li, E. Yang, X. Dai, Y. Peng, Y. Fei, Y. Liu, J.Q. Li, C. Chen. 2017. “Automated Pixel-
Level Pavement Crack Detection on 3D Asphalt Surfaces using a Deep-Learning Network,” Computer-Aided
Civil and Infrastructure Engineering, Volume 32(10), pp. 805–819, https://doi.org/10.1111/MICE.12297
Zhang, A., K.C.P. Wang, Y. Fei, Y. Liu, C. Chen, G. Yang, J.Q. Li, E. Yang, and S. Qiu. 2019. “Automated Pixel-level
Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network.” Computer-Aided
Civil and Infrastructure Engineering. Issue 34(3).
Zhang, D., Q. Zou, H. Lin, X. Xu, L. He, R. Gui, and Q. Li. 2018. “Automatic Pavement Defect Detection using
3D Laser Profiling Technology.” Automation in Construction. Volume 96.
Zhou, Y., F. Wang, N. Meghanathan, and Y. Huang. 2016. “Seed-based Approach for Automated Crack Detection
from Pavement Images.” Transportation Research Record 2589. Journal of the Transportation Research Board,
Washington, DC.
Zhu, J., J. Zhong, T. Ma, X. Huang, W. Zhang, and Y. Zhou. 2022. “Pavement Distress Detection using Convo-
lutional Neural Networks with Images Captured via UAV.” Automation in Construction. Volume 133.
Zhu, W., J. Wu, T. Fu, J. Wang, J. Zhang, and Q. Shangguan. 2021. “Dynamic Prediction of Traffic Incident Duration
on Urban Expressways: A Deep Learning Approach Based on LSTM and MLP.” Journal of Intelligent and Con-
nected Vehicles. Volume 4(2).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

APPENDIX A

Agency Survey Questionnaire

NCHRP TOPIC 54-14

ARTIFICIAL INTELLIGENCE APPLICATIONS FOR AUTOMATIC
PAVEMENT CONDITION EVALUATION

QUESTIONNAIRE

The following includes the proposed questions included in the agency questionnaire. The proposed

questions included yes/no responses or asked the user to select from a specific list of responses and

provided space for users to include comments as needed.

Dear Agency Pavement Management Representative,

The Transportation Research Board (TRB), through the National Cooperative Highway Research

Program (NCHRP), under the sponsorship of the American Association of State Highway and

Transportation Officials (AASHTO), and in cooperation with the Federal Highway Administration

(FHWA) is preparing a synthesis report on artificial intelligence applications for automatic pavement

condition evaluation.

The purpose of this questionnaire is to identify and summarize the procedures and practices used by

state DOTs related to the use of artificial intelligence technology with a fully automated pavement

condition survey. The results of the questionnaire will be incorporated into a synthesis of DOT practice,

with the intent of helping agencies evaluate and improve their current practices.

This questionnaire is being sent to personnel responsible for pavement management at all state DOTs.

If you are not the appropriate person at your agency to complete this questionnaire, please forward this

request to the correct person. A PDF of the questionnaire is attached so you may preview all of the

questions.

53

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

54   AI Applications for Automatic Pavement Condition Evaluation

Please complete and submit this questionnaire by July 21, 2023. We estimate that it should take no

more than 30 minutes to complete. If you have any questions or problems with operation or access to the

questionnaire, please contact our principal investigator Dr. Linda Pierce.

QUESTIONNAIRE TIPS
If you are unable to complete the questionnaire, you can return to the questionnaire at any time by re-

entering through the questionnaire link as long as you access the questionnaire through the same

computer. Re-entering the questionnaire will return you to the last completed question.

Questionnaire navigation is conducted by selecting the “prev” (previous) or “next” button at the bottom

of each page.

Thank you for your time and expertise in completing this important questionnaire.

ACRONYMS
AASHTO – American Association of State Highway and Transportation Officials
AI – artificial intelligence
DOT – Department of Transportation
GIS – Geographic information system
HPMS – Highway Performance Monitoring System
IRI – International Roughness Index
LLM – large language models
MAP-21 – Moving Ahead for Progress in the 21st Century Act
PMS – pavement management system

DEFINITIONS
Agency – for data collection and analysis of the automated pavement condition survey, the use of

“agency” implies agency data collection and analysis, vendor data collection and analysis, or in

combination.

Artificial intelligence – computer-based methodologies for identifying pavement distress types and

distress severity and extent.

Automated pavement condition survey – fully-automated methods (i.e., minimal to no user interaction)

for detecting surface distress, excludes the collection and analysis of inertial profile data (e.g., IRI,

faulting, rutting).

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Questionnaire   55

QUESTIONS
Name_______________________________________________
Organization _________________________________________
E-mail Address _______________________________________
Phone Number _______________________________________

GENERAL
1. Does your agency conduct an automated pavement condition survey to quantify pavement surface
distress (excludes inertial profile measurements)?
 Yes  No
2. Does your agency use AI technology to process the automated pavement condition survey?
 Yes  No
 Not sure
3. Does your agency use other technology to complement the automated pavement condition survey
(e.g., smartphones to track potholes, more frequent assessment of safety-related distress)?
 Yes  No
 Not sure
4. Do you think the new technology (e.g., autonomous vehicle, crowdsource) may be used for future
AI-based automated pavement condition data collection?
 Yes  No
 Not sure
5. Would your agency consider using AI in the future to collect and process the automated pavement
condition data?
 Yes  No
 Not sure

AUTOMATED PAVEMENT CONDITION SURVEYS

6. Is the automated pavement condition survey conducted using agency equipment and staff or
contracted through a vendor?
 Conducted by agency  Conducted by vendor
 Conducted by agency and vendor  Other (please specify)
7. Please indicate the vendor or equipment provider your agency uses for the automated pavement
condition survey (the vendor or equipment provider name will not be provided in the synthesis
document).
 Vendor A  Vendor B
 Vendor C  Vendor D
 Vendor E  Other (please specify)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

56   AI Applications for Automatic Pavement Condition Evaluation

8. Requirements for automated pavement distress identification include (select all that apply):
 Distress type  Distress severity
 Distress extent  Precision and accuracy
 Data repeatability  Sampling rate
 Linear referencing system  Spatial resolution
 Reporting interval  Productivity (miles/day)
 Compatibility with existing PMS  Other (please specify)
9. What data format requirements are needed for AI training/post-processing?
 No additional requirements  Not sure
 Other (please describe)
10. What image quality requirements are needed to conduct AI training/post-processing?
 No additional requirements  Not sure
 Other (please describe)

AUTOMATED PAVEMENT CONDITION SURVEY RESULTS

11. Pavement surface condition data are used for (select all that apply):
 Pavement performance modeling  Road safety assessment
 Pavement condition rating or index  Detecting prevalent distress type
 Detecting individual distress types  HPMS reporting
 MAP-21 reporting  Other (please specify)
12. Pavement condition data is used to support the following agency decision-making activities (select
all that apply):
 Budgeting  Multi-year budget planning (network)
 Approximate bid quantities (project)  Trigger safety-related repairs
 Treatment selection  Targeted performance goals
 Establish performance targets  Verify performance models
 Contract performance specifications and
 Other (please specify)
measures

ARTIFICIAL INTELLIGENCE
13. What asphalt-surfaced pavement condition types does your agency evaluate using AI technology
(select all that apply)?
 Alligator cracking  Bleeding
 Block cracking  Delamination/potholes
 Edge cracking  Longitudinal cracking
 Patching  Raveling
 Potholing  Transverse cracking
 Weathering  Not sure
 Other (please specify)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Questionnaire   57

14. What concrete-surfaced pavement condition types does your agency evaluate using AI technology
(select all that apply)?
 Not applicable (pavement type not used)  Blowups
 Corner cracking  Joint seal damage
 Longitudinal cracking  Map cracking
 Patching  Polished aggregate
 Pumping  Punchout
 Scaling  Spalling
 Transverse cracking  Not sure
 Other (please specify)
15. What other roadway features does your agency assess using AI technology (select all that apply)?
 Not applicable  Right-of-way (e.g., slope, embankment)
 Excess vegetation growth  Roadside assets (e.g., markings, signs)
 Other (please specify)
16. What AI technologies, tools, and models does your agency currently use for pavement condition
evaluation (select all that apply)?
 Machine learning  Pattern recognition (e.g., data mining)
 Neural network  Deep learning
 Random forest  Not sure
 Other (please specify)
17. How does your agency conduct AI-technique development, training, and evaluation (e.g., ground
truth testing) (select all that apply)?
 Pre-defined reference sections  Random reference sections
 Compare to traditional automated pavement
 Compare to manual surveys
condition survey
 Accuracy, precision, and repeatability  Google Earth images for AI training
 Not sure  Other (please specify)
18. Is your agency’s AI process in accordance with AASHTO R 85-18?
 Yes  No
 Not sure  Other (please specify)
19. Can your agency’s AI process be used on historical records (i.e., archived videos or images)?
 Yes  No
 Not sure  Other (please specify)
20. What challenges does your agency have with the current AI process for automated pavement
condition surveys?
 Limited agency knowledge  Computer computation capabilities
 AI training  Ground truth testing
 Trusting results  Other (please specify)

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

58   AI Applications for Automatic Pavement Condition Evaluation

21. What are the agency benefits of using AI for processing the automated pavement condition survey?
 Increased productivity  Objectivity (consistent assessment)
 Accuracy (once trained)  Cost
 Other (please specify)

IN CLOSING
22. Do you have any other suggestions or comments related to using AI with automated pavement
condition surveys?
 Yes (please specify)  No
23. Are you willing to participate in a follow-up interview (via email) in the event additional
information or clarification of your responses are needed?
 Yes  No
24. The synthesis will also include case examples highlighting agency practices related to AI
technology and automated pavement condition surveys. Agencies will be provided the opportunity
to review the case example write-up for accuracy. Would your agency be interested in participating
in a case example?
 Yes  No
25. If available, please include additional documentation related to AI and automated pavement
condition surveys.
 Yes (can provide a file)  Yes (can provide a link)
 No

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

APPENDIX B

Agency Survey Responses

Responding DOTs (43):

• Alabama DOT • New Hampshire DOT

• Alaska DOT and Public Facilities • New Jersey DOT
• Arizona DOT • New Mexico DOT
• Colorado DOT • New York State DOT
• Connecticut DOT • North Carolina DOT
• Delaware DOT • North Dakota DOT
• Florida DOT • Ohio DOT
• Georgia DOT • Oklahoma DOT
• Idaho Transportation Department • Oregon DOT
• Illinois DOT • Rhode Island DOT
• Indiana DOT • South Carolina DOT
• Iowa DOT • South Dakota DOT
• Kentucky Transportation Cabinet • Tennessee DOT
• Maine DOT • Texas DOT
• Maryland DOT • Utah DOT
• Michigan DOT • Vermont DOT
• Minnesota DOT • Virginia DOT
• Mississippi DOT • Washington State DOT
• Missouri DOT • West Virginia Division of Highways
• Montana DOT • Wisconsin DOT
• Nebraska DOT • Wyoming DOT
• Nevada DOT

1. Does your agency conduct an automated pavement condition survey to quantify pavement surface
distress (excludes inertial profile measurements)?

Response Agencies Count

Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Minnesota, Mississippi, Missouri, Montana, Nebraska, New Hampshire,
Yes 38
New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma,
Oregon, Rhode Island, South Carolina, Tennessee, Texas, Utah,
Vermont, Virginia, Washington, West Virginia, and Wyoming
No Maine, Nevada, New Jersey, South Dakota, and Wisconsin 5
No. DOTs 43

59

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

60   AI Applications for Automatic Pavement Condition Evaluation

2. Does your agency use other technology to complement the automated pavement condition survey
(e.g., smartphones to track potholes, more frequent assessment of safety-related distress)?

Response Agencies Count

Iowa: Falling weight deflectometer and friction testing.
Kentucky: LCMS, light detection and ranging (LiDAR), Sideway-force
Coefficient Routine Investigation Machine (SCRIM), and visual survey.
Yes 5
Montana: Right-of-way imagery and mobile LiDAR.
Ohio: PCR.
Texas: Visual rating for the audit sections (about 6% of network).
Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Idaho, Illinois, Indiana, Maryland, Minnesota, Mississippi, Missouri,
Nebraska, New Hampshire, New Mexico, New York, North Carolina,
No 31
North Dakota, Oklahoma, Oregon, Rhode Island, South Carolina,
Tennessee, Utah, Vermont, Virginia, Washington, West Virginia, and
Wyoming
Not sure Georgia and Michigan 2
No. DOTs 38

3. Do you think the new technology (e.g., autonomous vehicle, crowdsource) may be used for future AI-
based automated pavement condition data collection?

Response Agencies Count

Arizona, Connecticut, Delaware, Iowa, Maryland, Michigan, Mississippi,
New Hampshire, New York, North Carolina, North Dakota, Ohio,
Not sure 18
Oklahoma, Rhode Island, South Carolina, Vermont, West Virginia, and
Wyoming
Alaska, Florida, Georgia, Idaho, Illinois, Indiana, Kentucky, Minnesota,
Yes Montana, Nebraska, New Mexico, Tennessee, Texas, Virginia, and 15
Washington
No Alabama, Colorado, Missouri, Oregon, and Utah 5
No. DOTs 38

4. Does your agency (or vendor) use AI technology to process the automated pavement condition
survey?

Response Agencies Count

Alaska, Connecticut, Florida, Indiana, Kentucky, Maryland, Minnesota,
No Mississippi, Montana, Nebraska, New Mexico, North Carolina, Ohio, 18
Rhode Island, Virginia, Washington, West Virginia, and Wyoming
Arizona, Colorado, Delaware, Idaho, Illinois, Michigan, Missouri,
Not sure 12
Oregon, South Carolina, Texas, Utah, and Vermont
Alabama, Georgia, Iowa, New Hampshire, New York, North Dakota,
Yes 8
Oklahoma, and Tennessee
No. DOTs 38

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Responses   61

5. Would your agency consider using AI in the future to collect and process the automated pavement
condition data?

Response Agencies Count

Alaska, Connecticut, Delaware, Florida, Idaho, Illinois, Indiana,
Yes Kentucky, Maryland, Michigan, Minnesota, Montana, North Carolina, 18
Oregon, Texas, Utah, Vermont, and Virginia
Arizona, Colorado, Mississippi, Missouri, Nebraska, New Mexico, Ohio,
Not sure 12
Rhode Island, South Carolina, Washington, West Virginia, and Wyoming
No. DOTs 30

6. Is the automated pavement condition survey conducted using agency equipment and staff or
contracted through a vendor?

Response Agencies Count

Alabama, Alaska, Arizona, Colorado, Delaware, Indiana, Michigan,
Mississippi, New Hampshire, New Mexico, New York, North Carolina,
Vendor 22
Oklahoma, Oregon, Rhode Island, Tennessee, Texas, Utah, Vermont,
Virginia, West Virginia, and Wyoming
Connecticut, Florida, Idaho, Kentucky, Maryland, Minnesota, Missouri,
Agency 12
Montana, Nebraska, North Dakota, Ohio, and Washington
Agency and
Georgia, Illinois, Iowa, and South Carolina 4
vendor
No. DOTs 38

7. Please indicate the vendor or equipment provider your agency uses for the automated pavement
condition survey.

Response Agencies Count

Alabama, Georgia, Idaho, Indiana, Iowa, Michigan, Minnesota,
Vendor A Montana, Nebraska, New Hampshire, North Dakota, Ohio, Oregon, 17
Rhode Island, Texas, Utah, and Washington
Alaska, Colorado, Connecticut, Maryland, Missouri, New Mexico, New
Vendor B York, North Carolina, South Carolina, Vermont, Virginia, and West 12
Virginia
Vendor C Arizona, Delaware, Florida, and Wyoming 4
Vendor D Illinois, Kentucky, Oklahoma, and Tennessee 4
Vendor E Mississippi 1
No. DOTs 38

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

62   AI Applications for Automatic Pavement Condition Evaluation

8. Requirements for automated pavement distress identification include (select all that apply):

Response Agencies Count

Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Minnesota, Mississippi, Missouri, Montana, Nebraska, New Hampshire,
Distress type 38
New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma,
Oregon, Rhode Island, South Carolina, Tennessee, Texas, Utah,
Vermont, Virginia, Washington, West Virginia, and Wyoming
Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Distress Minnesota, Mississippi, Missouri, Montana, Nebraska, New Hampshire,
38
severity New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma,
Oregon, Rhode Island, South Carolina, Tennessee, Texas, Utah,
Vermont, Virginia, Washington, West Virginia, and Wyoming
Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Linear
Minnesota, Mississippi, Missouri, Montana, Nebraska, New Hampshire,
referencing 37
New Mexico, New York, North Carolina, Ohio, Oklahoma, Oregon,
system
Rhode Island, South Carolina, Tennessee, Texas, Utah, Vermont,
Virginia, Washington, West Virginia, and Wyoming
Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Distress Mississippi, Missouri, Montana, New Hampshire, New Mexico, New
35
extent York, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Rhode
Island, South Carolina, Tennessee, Texas, Utah, Vermont, Virginia,
Washington, and Wyoming
Alabama, Alaska, Arizona, Connecticut, Delaware, Florida, Georgia,
Idaho, Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota,
Reporting
Mississippi, Missouri, Montana, New Hampshire, New Mexico, New 31
interval
York, North Carolina, Ohio, Oregon, South Carolina, Tennessee, Texas,
Utah, Vermont, Virginia, Washington, and Wyoming
Alabama, Alaska, Arizona, Colorado, Delaware, Florida, Georgia, Idaho,
Illinois, Indiana, Kentucky, Maryland, Michigan, Minnesota, Mississippi,
Data
Missouri, Montana, Nebraska, New Mexico, New York, North Carolina, 30
repeatability
North Dakota, Ohio, Oklahoma, South Carolina, Utah, Vermont,
Virginia, Washington, and Wyoming
Alabama, Alaska, Arizona, Delaware, Florida, Georgia, Idaho, Illinois,
Indiana, Iowa, Kentucky, Maryland, Michigan, Mississippi, Missouri,
Precision and
Montana, Nebraska, New Mexico, New York, North Carolina, North 29
accuracy
Dakota, Ohio, Rhode Island, Texas, Utah, Vermont, Virginia,
Washington, and Wyoming
Alabama, Alaska, Arizona, Connecticut, Delaware, Georgia, Idaho,
Compatibility Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota, Mississippi,
with existing Missouri, New Mexico, New York, North Carolina, Oregon, Rhode 26
PMS Island, South Carolina, Texas, Vermont, Virginia, Washington, and
Wyoming

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Responses   63

Response Agencies Count

Alabama, Alaska, Delaware, Florida, Georgia, Idaho, Indiana, Iowa,
Spatial
Kentucky, Maryland, Michigan, Mississippi, Missouri, Montana, New 19
resolution
Mexico, North Carolina, Ohio, Washington, and Wyoming
Alabama, Alaska, Delaware, Idaho, Indiana, Iowa, Kentucky, Maryland,
Sampling rate 14
Missouri, Montana, New Mexico, New York, North Carolina, and Ohio
Productivity Idaho, Kentucky, Maryland, Michigan, Minnesota, Mississippi, Missouri,
11
(miles/day) Nebraska, New Mexico, North Carolina, and Wyoming
Additional Delaware: Based on agency data dictionary.
2
Comments: Oregon: Based on agency protocols.
No. DOTs 38

9. What data format requirements are needed for AI training/post-processing?

Response Agencies Count

Alaska, Arizona, Colorado, Connecticut, Delaware, Georgia, Idaho,
Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota,
Not sure Missouri, Nebraska, New Mexico, New York, North Carolina, Ohio, 29
Oklahoma, Oregon, South Carolina, Texas, Utah, Vermont,
Washington, West Virginia, and Wyoming
No additional Alabama, Montana, New Hampshire, North Dakota, Tennessee, and
6
requirements Virginia
Other Florida: Pavemetrics libraries for most, range jpgs for raveling 1
No. DOTs 36

10. What image quality requirements are needed to conduct AI training/post-processing?

Response Agencies Count

Alabama, Arizona, Connecticut, Delaware, Georgia, Idaho, Illinois,
Indiana, Iowa, Kentucky, Michigan, Minnesota, Missouri, Nebraska,
Not sure New Mexico, New York, North Carolina, Ohio, Oklahoma, Oregon, 28
Rhode Island, South Carolina, Texas, Utah, Vermont, Washington,
West Virginia, and Wyoming
No additional
Montana, New Hampshire, North Dakota, Tennessee, and Virginia 5
requirements
Alaska: likely at least 2mm resolution based on Federal Aviation
Association research.
Colorado: Need similar resolution images. Older images are not as
Other 4
high resolution as new images.
Florida: 4mm x 4mm pixel resolution for range image
Maryland: Image size and resolution. Free of artifacts and Shadows
No. DOTs 37

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

64   AI Applications for Automatic Pavement Condition Evaluation

11. Pavement surface condition data are used for (select all that apply):

Response Agencies Count

Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Minnesota, Mississippi, Missouri, Montana, Nebraska, New
HPMS
Hampshire, New Mexico, New York, North Carolina, North Dakota, 38
reporting
Ohio, Oklahoma, Oregon, Rhode Island, South Carolina, Tennessee,
Texas, Utah, Vermont, Virginia, Washington, West Virginia, and
Wyoming
Alaska, Arizona, Colorado, Connecticut, Delaware, Florida, Georgia,
Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Pavement
Minnesota, Mississippi, Missouri, Montana, Nebraska, New
performance 36
Hampshire, New Mexico, New York, North Carolina, North Dakota,
modeling
Oklahoma, Oregon, Rhode Island, South Carolina, Tennessee, Texas,
Utah, Vermont, Virginia, Washington, West Virginia, and Wyoming
Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Pavement
Minnesota, Mississippi, Missouri, Montana, New Hampshire, New
condition 36
Mexico, New York, North Carolina, North Dakota, Oklahoma, Oregon,
rating or index
Rhode Island, South Carolina, Tennessee, Texas, Utah, Vermont,
Virginia, Washington, West Virginia, and Wyoming
Alabama, Alaska, Connecticut, Delaware, Florida, Georgia, Idaho,
Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota,
MAP-21 Mississippi, Missouri, Montana, New Hampshire, New Mexico, New
33
reporting York, North Carolina, North Dakota, Ohio, Oregon, Rhode Island,
South Carolina, Tennessee, Texas, Utah, Vermont, Virginia,
Washington, and West Virginia
Alabama, Alaska, Colorado, Connecticut, Delaware, Florida, Idaho,
Detecting Illinois, Iowa, Kentucky, Maryland, Michigan, Minnesota, Mississippi,
individual Missouri, Montana, New Mexico, New York, North Dakota, Oklahoma, 29
distress types Oregon, Rhode Island, South Carolina, Tennessee, Texas, Utah,
Virginia, Washington, and Wyoming
Alabama, Colorado, Delaware, Florida, Idaho, Illinois, Kentucky,
Detecting
Maryland, Michigan, Minnesota, Mississippi, Missouri, Montana,
prevalent 22
Nebraska, New Mexico, New York, North Dakota, Tennessee, Texas,
distress type
Utah, Virginia, and Wyoming
Road safety Georgia, Illinois, Kentucky, Maryland, Michigan, Minnesota, Missouri,
11
assessment Texas, Utah, Washington, and Wyoming
Alabama: Screening potential preservation treatments (e.g.,
microsurfacing might require a second pass in the wheel paths if
Other rutting is > 0.25-in.). 3
Oregon: State KPM pavement condition performance measure.
Texas: Support the development of pavement management plans.
No. DOTs 38

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Responses   65

12. Pavement condition data is used to support the following agency decision-making activities (select all
that apply):

Response Agencies Count

Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Verify Minnesota, Mississippi, Missouri, Montana, Nebraska, New
performance Hampshire, New Mexico, New York, North Carolina, North Dakota, 38
models Ohio, Oklahoma, Oregon, Rhode Island, South Carolina, Tennessee,
Texas, Utah, Vermont, Virginia, Washington, West Virginia, and
Wyoming
Alabama, Arizona, Connecticut, Delaware, Florida, Georgia, Idaho,
Multi-year
Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota,
budget
Mississippi, Missouri, Montana, Nebraska, New Hampshire, New York, 31
planning
North Carolina, North Dakota, Oregon, South Carolina, Tennessee,
(network)
Texas, Utah, Vermont, Virginia, Washington, and Wyoming
Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Georgia,
Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
Treatment
Minnesota, Mississippi, Montana, New Mexico, New York, North 31
selection
Carolina, North Dakota, Oregon, Rhode Island, Tennessee, Texas,
Utah, Vermont, Virginia, Washington, West Virginia, and Wyoming
Arizona, Colorado, Connecticut, Delaware, Florida, Georgia, Idaho,
Targeted Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota,
performance Mississippi, Missouri, Montana, Nebraska, New Hampshire, New 31
goals Mexico, New York, North Carolina, Oklahoma, Oregon, South
Carolina, Texas, Utah, Vermont, Virginia, Washington, and Wyoming
Alabama, Alaska, Connecticut, Delaware, Georgia, Idaho, Illinois,
Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota, Mississippi,
Establish
Missouri, Montana, Nebraska, New Mexico, New York, North
performance 31
Carolina, Oklahoma, Oregon, Rhode Island, South Carolina,
targets
Tennessee, Texas, Utah, Vermont, Virginia, Washington, and
Wyoming
Alaska, Connecticut, Delaware, Florida, Georgia, Idaho, Illinois,
Indiana, Iowa, Kentucky, Maryland, Michigan, Minnesota, Missouri,
Budgeting Montana, New York, North Carolina, North Dakota, Oregon, South 27
Carolina, Tennessee, Texas, Utah, Vermont, Virginia, Washington, and
Wyoming
Contract
Idaho, Kentucky, Maryland, Michigan, Minnesota, Mississippi,
performance
Missouri, New Mexico, New York, North Carolina, Texas, Utah, 14
specifications
Vermont, and Washington
and measures
Trigger safety- Illinois, Iowa, Kentucky, Maryland, Minnesota, Nebraska, New
10
related repairs Mexico, Texas, Vermont, and Washington
Approximate
Kentucky, Maryland, Montana, New Mexico, New York, North
bid quantities 9
Carolina, Utah, Washington, and Wyoming
(project)
Other Alabama: Used in transportation asset management plan scenarios. 4

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

66   AI Applications for Automatic Pavement Condition Evaluation

Response Agencies Count

Maryland: Asset management and system preservation.
Ohio: Data used for federal reporting purposes.
South Carolina: Contract performance.
No. DOTs 38

13. What asphalt-surfaced pavement condition types does your agency evaluate using AI technology
(select all that apply)?
Agencies Who Are Unsure if
Response Agencies Who Use AI Count Count
AI Is Used
Alabama, Iowa, New
Arizona, Delaware, Idaho,
Longitudinal Hampshire, New York, North
7 South Carolina, and 5
cracking Dakota, Oklahoma, and
Vermont
Tennessee
Alabama, Iowa, New
Arizona, Delaware, Idaho,
Transverse Hampshire, New York, North
7 South Carolina, and 5
cracking Dakota, Oklahoma, and
Vermont
Tennessee
Iowa, New York, North
Alligator Arizona, Delaware, Idaho,
Dakota, Oklahoma, and 5 4
cracking and Vermont
Tennessee
New York, North Dakota, Arizona, Delaware, Idaho,
Block cracking 4 4
Oklahoma, and Tennessee and Vermont
Delamination / New York, North Dakota,
4 Idaho 1
potholes Oklahoma, and Tennessee
Alabama, Oklahoma, and Delaware, Idaho, and South
Patching 3 3
Tennessee Carolina
Arizona, Delaware, and
Edge cracking Iowa and New Hampshire 2 3
Idaho
Raveling Oklahoma 1 Arizona and South Carolina 2
Bleeding None 0 Delaware 1
Weathering None 0 None 0
Alabama: Wheel path and non-wheel path cracking rather than
alligator cracking.
South Carolina: Wheel path cracking distress varieties all grouped as
fatigue cracking.
Other 4
Utah: AI usage (excluding vendor efforts) has been research related and
focused on cracking, potholing, and patching.
Washington: Use the automated crack rating system from vendor to
perform a quality acceptance check against agency visual rating.
No. DOTs 14

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Responses   67

14. What concrete-surfaced pavement condition types does your agency evaluate using AI technology
(select all that apply)?
Agencies Who Are Unsure if
Response Agencies Who Use AI Count Count
AI Is Used
Alabama, Iowa, New York,
Transverse Connecticut, Delaware,
North Dakota, Oklahoma, and 6 4
cracking Idaho, and Utah
Tennessee
Iowa, New York, North
Longitudinal Connecticut, Delaware,
Dakota, Oklahoma, and 5 4
cracking Idaho, and Utah
Tennessee
New York, North Dakota,
Spalling 4 Utah 1
Oklahoma, and Tennessee
Corner Alabama, North Dakota, and
3 0 0
cracking Oklahoma
Multi-cracked Iowa, North Dakota, and
3 0 0
slabs Oklahoma
Patching Oklahoma and Tennessee 2 Delaware and Utah 2
Punchout Alabama and Oklahoma 2 None 0
Map cracking New York 1 None 0
Joint seal
None 0 Delaware and Idaho 2
damage
Blowups None 0 None 0
Polished
None 0 None 0
aggregate
Pumping None 0 None 0
Scaling None 0 None 0
Delaware: alkali-silica reactivity.
Other Washington: Use the automated crack rating system from vendor to 2
perform a quality acceptance check against agency visual rating.
No. DOTs 11

15. What other roadway features does your agency assess using AI technology (select all that apply)?

Response Agencies Count

Excess vegetation growth Arizona, Idaho, Iowa, New York, Vermont, and Wyoming 6
Right-of-way (e.g., slope,
None 0
embankment)
Roadside assets (e.g.,
None 0
markings, signs)
Georgia and Utah: evaluating AI technology for roadside
Other 2
assets and/or excess vegetative growth.
No. DOTs 8

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

68   AI Applications for Automatic Pavement Condition Evaluation

16. What AI technologies, tools, and models does your agency currently use for pavement condition
evaluation (select all that apply)?

Response Agencies Count

Arizona, Georgia, Idaho, Iowa, Michigan, Missouri, Nebraska, New
Not sure Hampshire, New Mexico, New York, North Carolina, North Dakota, 16
Oklahoma, Oregon, South Carolina, and West Virginia
Random forest Florida and Tennessee 2
Machine learning Alabama and Vermont 2
Neural network Montana 1
Pattern recognition
Delaware 1
(e.g., data mining)
Deep learning 0
Texas: Conducting research to examine the various AI models.
Traditional statistical analysis is currently used.
Other 2
Utah: Research project has mostly evaluated machine learning and
deep learning.
No. DOTs 24

17. How does your agency conduct AI-technique development, training, and evaluation (e.g., ground
truth testing) (select all that apply)?

Response Agencies Count

Georgia, Idaho, Illinois, Iowa, Michigan, Missouri, Nebraska,
Not sure New Mexico, North Carolina, Oregon, South Carolina, Texas, 15
Vermont, West Virginia, and Wyoming
Accuracy, precision,
Arizona, Delaware, Montana, New York, Oklahoma, and Utah 6
and repeatability
Compare to manual Alabama, Montana, New Hampshire, North Dakota, and
5
surveys Washington
Pre-defined reference
Alabama, Montana, and New York 3
sections
Random reference
Alabama, Montana, and Oklahoma 3
sections
Compare to traditional
automated pavement Montana and Utah 2
condition survey
Florida: manual image classification.
Other Tennessee: Distress library was used to train the models. We 2
do not have a ground truth test section.
No. DOTs 27

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Responses   69

18. Is your agency’s AI process in accordance with AASHTO R 85-18?

Response Agencies Count

Yes New Hampshire and Oregon 2
No Texas and Wyoming 2
Arizona, Delaware, Florida, Georgia, Idaho, Illinois, Iowa, Missouri,
Montana, Nebraska, New Mexico, New York, North Carolina, North
Not sure 20
Dakota, Oklahoma, South Carolina, Tennessee, Utah, Vermont, and
West Virginia
Alabama: very close to R85; however, wheel paths are 3 ft wide.
Other Michigan: Vendor required to follow the basis of R85 with 2
modifications.
No. DOTs 26

19. Can your agency’s AI process be used on historical records (i.e., archived videos or images)?

Response Agencies Count

Yes Montana, North Dakota, and Utah 3
No Idaho, New York, Oregon, and Washington 4
Alabama, Arizona, Delaware, Georgia, Illinois, Iowa, Michigan, Missouri,
Not sure Nebraska, New Hampshire, New Mexico, North Carolina, Oklahoma, 17
South Carolina, Tennessee, Vermont, and West Virginia
Other Florida: only if previous surveys used an LCMS to collect the data. 1
No. DOTs 25

20. What challenges does your agency have with the current AI process for automated pavement
condition surveys (select all that apply)?

Response Agencies Count

Alaska, Florida, Georgia, Idaho, Iowa, Maryland, Michigan,
Limited agency
Montana, New Mexico, North Carolina, South Carolina, Tennessee, 14
knowledge
Utah, and Vermont
Alabama, Arizona, Florida, Idaho, Maryland, Michigan, New
Ground truth testing 10
Hampshire, Tennessee, Texas, and Vermont
Alabama, Alaska, Florida, Maryland, Montana, New Hampshire,
Trusting results 9
Tennessee, Utah, and Vermont
AI training Florida, Idaho, Maryland, Oklahoma, Tennessee, and Washington 6
Computer
computation Florida, Idaho, Missouri, Montana, and Washington 5
capabilities
Not sure Oregon and Wyoming 2
Alaska: AI is not currently used and has a limited understanding of
Other how it could be implemented. 5
Delaware: rely on vendor expertise.

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

70   AI Applications for Automatic Pavement Condition Evaluation

Georgia: 2023 was the first year AI was used.

Michigan: AI used by vendor; uncertain of utilization and
challenges.
Washington: current AI algorithm is having difficulties
distinguishing between multiple crack types in each area as
well as asphalt longitudinal cracking at lane edge, and shoulder
paint or rumble strips (quantifies as cracks).
No. DOTs 24

21. What are the agency benefits of using AI for processing the automated pavement condition survey
(select all that apply)?

Response Agencies Count

Alabama, Delaware, Georgia, Idaho, Illinois, Iowa, Michigan, Montana,
Objectivity
New Hampshire, New York, North Dakota, Tennessee, Vermont, and 14
(consistency)
Washington
Accuracy (once Alabama, Arizona, Idaho, Illinois, Maryland, Montana, New York, North
13
trained) Dakota, Oklahoma, South Carolina, Tennessee, Texas, and Vermont
Arizona, Delaware, Georgia, Idaho, Maryland, Michigan, Montana, New
Increased
Hampshire, New York, North Dakota, Oklahoma, Tennessee, and 13
productivity
Washington
Idaho, New Hampshire, New Mexico, New York, North Dakota,
Cost 7
Tennessee, and Utah
Not sure North Carolina, Oregon, and Wyoming 3
Other Florida: increased resolution 1
No. DOTs 25

22. Do you have any other suggestions or comments related to using AI with automated pavement
condition surveys?

Response Agencies Count

Alaska: Prior to implementation, need to see results of extensive ground
truthing of the processed data.
Indiana: More education surrounding advancements made and
functionality of current AI systems.
Missouri: To survey roadways, you need measurements, measurements
provide a definite answer, and AI is not needed for interpretations.
Montana: Leverage AI technologies for QA/QC processes. This could be
change-detection technology while conducting the surveys and QC
Yes processes to review the data. 7
Tennessee: There is a need to establish a standard distress library
which can be used for training AI models for distress classification.
Texas: Industry standard images (i.e., reporting format) would make it
easier for AI learning and results comparison.
Utah: AI has shown promise for cracking and asset detection but unable
to collect all the required data. Unless paired with other data
collection efforts, it would be insufficient to replace current automated
data collection methods.
Alabama, Arizona, Colorado, Connecticut, Delaware, Florida, Georgia,
No 31
Idaho, Illinois, Iowa, Kentucky, Maryland, Michigan, Minnesota,

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Agency Survey Responses   71

Mississippi, Nebraska, New Hampshire, New Mexico, New York, North

Carolina, North Dakota, Ohio, Oklahoma, Oregon, Rhode Island, South
Carolina, Vermont, Virginia, Washington, West Virginia, and Wyoming
No. DOTs 38

23. Are you willing to participate in a follow-up interview (via email) in the event additional information
or clarification of your responses are needed?

Response Agencies Count

Delaware, Idaho, Iowa, Kentucky, Montana, North Dakota, Tennessee,
Yes 8
and Washington
Alabama, Alaska, Arizona, Colorado, Connecticut, Florida, Georgia,
Illinois, Indiana, Maryland, Michigan, Mississippi, Missouri, Nebraska,
No New Hampshire, New Mexico, New York, North Carolina, Ohio, 28
Oklahoma, Oregon, Rhode Island, Texas, Utah, Vermont, Virginia, West
Virginia, and Wyoming
No. DOTs 36

24. The synthesis will also include case examples highlighting agency practices related to AI technology
and automated pavement condition surveys. Agencies will be provided the opportunity to review the
case example write-up for accuracy. Would your agency be interested in participating in a case
example?

Response Agencies Count

Delaware, Idaho, Iowa, Kentucky, Montana, North Dakota, Tennessee,
Yes 8
and Washington
Alabama, Alaska, Arizona, Colorado, Connecticut, Florida, Georgia,
Illinois, Indiana, Maryland, Michigan, Mississippi, Missouri, Nebraska,
No New Hampshire, New Mexico, New York, North Carolina, Ohio, 28
Oklahoma, Oregon, Rhode Island, Texas, Utah, Vermont, Virginia, West
Virginia, and Wyoming
No. DOTs 36

25. If available, please include additional documentation related to AI and automated pavement condition
surveys.

Response Agencies Count

Provided a file Not applicable 0

Provided a link Not applicable 0
Alabama, Alaska, Arizona, Colorado, Connecticut, Delaware, Florida,
Georgia, Idaho, Illinois, Indiana, Iowa, Kentucky, Maryland, Michigan,
No additional
Mississippi, Montana, Nebraska, New Hampshire, New Mexico, New
information 36
York, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Rhode
available
Island, South Carolina, Tennessee, Texas, Utah, Vermont, Virginia,
Washington, West Virginia, and Wyoming
No. DOTs 36

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Abbreviations and acronyms used without definitions in TRB publications:

A4A Airlines for America
AAAE American Association of Airport Executives
AASHO American Association of State Highway Officials
AASHTO American Association of State Highway and Transportation Officials
ACI–NA Airports Council International–North America
ACRP Airport Cooperative Research Program
ADA Americans with Disabilities Act
APTA American Public Transportation Association
ASCE American Society of Civil Engineers
ASME American Society of Mechanical Engineers
ASTM American Society for Testing and Materials
ATA American Trucking Associations
CTAA Community Transportation Association of America
CTBSSP Commercial Truck and Bus Safety Synthesis Program
DHS Department of Homeland Security
DOE Department of Energy
EPA Environmental Protection Agency
FAA Federal Aviation Administration
FAST Fixing America’s Surface Transportation Act (2015)
FHWA Federal Highway Administration
FMCSA Federal Motor Carrier Safety Administration
FRA Federal Railroad Administration
FTA Federal Transit Administration
GHSA Governors Highway Safety Association
HMCRP Hazardous Materials Cooperative Research Program
IEEE Institute of Electrical and Electronics Engineers
ISTEA Intermodal Surface Transportation Efficiency Act of 1991
ITE Institute of Transportation Engineers
MAP-21 Moving Ahead for Progress in the 21st Century Act (2012)
NASA National Aeronautics and Space Administration
NASAO National Association of State Aviation Officials
NCFRP National Cooperative Freight Research Program
NCHRP National Cooperative Highway Research Program
NHTSA National Highway Traffic Safety Administration
NTSB National Transportation Safety Board
PHMSA Pipeline and Hazardous Materials Safety Administration
RITA Research and Innovative Technology Administration
SAE Society of Automotive Engineers
SAFETEA-LU Safe, Accountable, Flexible, Efficient Transportation Equity Act:
A Legacy for Users (2005)
TCRP Transit Cooperative Research Program
TEA-21 Transportation Equity Act for the 21st Century (1998)
TRB Transportation Research Board
TSA Transportation Security Administration
U.S. DOT United States Department of Transportation

Copyright National Academy of Sciences. All rights reserved.

AI Applications for Automatic Pavement Condition Evaluation

Transportation Research Board

500 Fifth Street, NW
Washington, DC 20001

ISBN 978-0-309-72767-9
90000

9 780309 727679

Copyright National Academy of Sciences. All rights reserved.