2020 VEHICLE DISPLAYS AND

INTERFACES SYMPOSIUM

DIGEST OF
TECHNICAL PAPERS
OCTOBER 14–15, 2020

CONFERENCE SPONSORS
 2020 VEHICLE DISPLAYS AND
INTERFACES SYMPOSIUM

Digest of Technical Papers

Abstracting is permitted with credit to the source. Libraries are permitted to photocopy
beyond the limits of the U.S. copyright law for private use of patrons those articles in this
volume that carry a code at the bottom of the first page, provided per-copy fee indicated
in the code is paid through the Copyright Clearance Center, 21 Congress Street, Salem,
MA 01970. Instructors are permitted to photocopy isolated articles for noncommercial
classroom use without fee. For other copying, reprint, or republication permission, write
to SID Headquarters, 1475 S. Bascom Avenue, Suite 114, Campbell, CA 95008. All
rights reserved. Copyright © 2020 Society for Information Display.
 Table of ConTenTs
K1: Keynote Address: Future Auto Cockpit Human Experiences ............................................................ 1
John Schneider, Director – Electrical, Software, Compute, and AI Technologies,
Ford Motor Company, Detroit, MI, US

session 1: automotive Market

1.1: Invited Address: Automotive Display Market and User Interfaces Overview .................................. 3
Kyle Davis, IHS Markit, Southfield, MI, US

1.2: Invited Address: Automotive Touch Screen, Touchless Control, Micro and miniLED and
Smart Windows Market Forecast .................................................................................................... 5
Jennifer Colegrove, Touch Display Research, Inc., Santa Clara, CA, US

1.3: Invited Address: Display Industry Markets and Technologies – What Auto Display
Professionals Need to Know About the Rest of the Industry.......................................................... 7
Bob O’Brien, Display Supply Chain Consultants, Ann Arbor, MI, US

session 2: Display and HMI systems

2.1: Reflection Properties of AR Coated Flat and AG Glass Surfaces ...................................................... 9
Dave McLean, MAC Thin Films, Inc., Santa Rosa, CA, US

2.2: IoT Intelligent Display Technology .................................................................................................... 15

Liang Zhou, Ling‐ling Zhang, Jiu‐bin Zhou, Jin‐e Liu, Feng Qinin,
Shanghai Tianma Microelectronics Co., Ltd., Shanghai, China

2.3: Display Module with Integrated Driver of Multi‐Screen .................................................................... 17

Liang Zhou, Lu Yao, Ling‐ling Zhang, Jiu‐bin Zhou, Wan‐chun Du, Jin‐e Liu, Feng Qin,
Tianma Micro‐Electronics Group, Shanghai, China

2.4: High Precision Optical Bonding for Free‐Form and Curved Displays .............................................. 19
Eugen Bilcai, Henkel KGaA, Madison Heights, MI, US

2.5: Digitized Styling and Safety with Automotive Exterior Displays ...................................................... 21
Johnathan Weiser, Richard Nguyen, Kimberly Peiler, OSRAM Opto Semiconductors, Novi, MI, US
Ulrich Kizak, OSRAM Opto Semiconductors, Regensburg, Germany

2.6: A Novel Approach for High Quality SNR in Sensing Applications.................................................... 25

Gerald Morrison, SigmaSense, Austin, TX, US

session 3: Head‐Up Displays

3.1: Diffusive Microlens Array for Head‐Up Display Applications .......................................................... 41
Naoki Hanashima, Mitsuo Arima, Yutaka Nakazawa, Dexerials Corporation, Tagajo, Miyagi,
Japan
Kazuyuki Shibuya, Dexerials Corporation, Tome, Miyagi, Japan
Jingting Wu, Dexerials America Corporation; San Jose, California, US

i
3.2: Human Perception Studies of Head‐Up Display Ghosting.................................................................. 47
Steve Pankratz, William Diepholz, John Vanderlofske, 3M Company, St. Paul, MN, US

3.3: Computational Holographic Displays for 3D AR HUD Using Free‐Form Optics.............................. 53

Hakan Urey, CY Vision, San Jose, CA, US

3.4: Holographic Optical Elements and Projector Design Considerations for

Automotive Windshield Displays .................................................................................................... 55
Sam Martin, Jason Thompson, Texas Instruments, Inc., Dallas Texas, US
Ian Redmond, CERES Holographics, Ltd., St. Andrews, Scotland, UK

Tutorial

Holography and Its Automotive Applications...................................................................................... 63

Kai‐Han Chang, General Motors Global R&D, Pontiac, MI, US

Keynote address
K2: Keynote Address: Voice of the Consumer | Technology and Mobility Clarity Today
and Tomorrow.................................................................................................................................. 65
Kristin Kolodge, Executive Director of Human‐Machine Interface (HMI) and
Driver Interaction at J.D. Power

session 4: Display Metrology

4.1: Understanding and Achieving Reproducible Sparkle Measurements for an
Automotive Specification ................................................................................................................ 67
Ingo Rotscholl, Julia Brinkmann, Udo Krüger, TechnoTeam Bildverarbeitung GmbH,
Ilmenau, Germany
Jens Rasmussen, Elektrobit Automotive GmbH, Ulm, Germany
Christoph Rickers, Volkswagen AG, Wolfsburg, Germany

4.2: Measuring MicroLEDs for Color Non‐Uniformity Correction .......................................................... 73

Mike Naldrett, Matt Scholz, Bret Stonebridge, Austin Piehl, Anne Corning,
Shaina Warner, Radiant Vision Systems LLC, Redmond, Washington, US

session 5: new Display solutions

5.1: Supervising (Automotive) Displays to Safeguard Camera Monitor Systems .................................... 85
B. Axmann, F. Langner, Mercedes‐Benz AG, Stuttgart, Germany
K. Blankenbach, M. Vogelmann, Pforzheim University, Pforzheim, Germany
M. Conrad, samoconsult GmbH, Berlin, Germany
J. Bauer, Karlsruhe University, Karlsruhe, Germany

5.2: Customized Local Dimming Algorithm and BLU for Automotive Application towards
Low Power Consumption and High Visual Quality........................................................................ 93
Maxim Schmidt, Ramazan Ayasli, Chihao Xu, Saarland University, Saarbrücken, Germany

5.3: Automotive Smart Surfaces: Conformable HDR Displays and Smart Windows
to Activate Almost Any Surface ...................................................................................................... 97
J. Huggins, FlexEnable, Ltd., Cambridge, UK

ii
5.4: End‐of‐Line (EOL) Testing of Recent OEM Display Quality Standards............................................ 101
Silke R. Kirchner, Benjamin Käsdorf, Cameron R. Hughes, Nadine Götte,
Instrument Systems Optische Messtechnik GmbH, Munich, Germany

5.5: Automotive Dual Cell microZone™ LCD Development.................................................................... 103

Paul Weindorf, Qais Sharif, Elijah Auger, David Whitton, Brian Hayden,
Visteon Corporation, Van Buren Twp., MI, US

5.6: A Low‐Power Transflective TFT‐LCD Based on IGZO TFT ............................................................ 107

Tenggang Lou, Lei Wang, Xiangjian Kong, Jine Liu, Feng Qin,
Tianma Micro‐Electronics Group, Shanghai, China

5.7: A MicroLED Device with 0mm Border .............................................................................................. 111

TengGang Lou, Tianma Micro‐Electronics Group, Shanghai, China

5.8: Enabling Features of VueReal MicroLED Technology for Automotive Applications ........................ 113
Rexa Chaji, VueReal Inc, Waterloo, Ontario, Canada

5.9: New Challenges and Testing Solutions for Flexible Vehicle Displays & Interfaces .......................... 115
Eisuke Tsuyuzaki, Bayflex Solutions, Alameda, CA, US

5.10: New Material Solutions for Automotive Displays. Interfaces, and Applications................................ 127
Volker Plehn, SABIC, Wixom, MI, US

5.11: An Alternative to OLED with Full‐Array Local Dimming in Automotive Displays.......................... 129
Logan Cummins, Texas Instruments, Dallas, TX, US

Panel DIsCUssIon:
Display and Interfaces for autonomous Drive Including aDas

iii
iv
 SID VEHICLE DISPLAYS AND INTERFACES 2020
27th ANNUAL SYMPOSIUM & EXPO
PROGRAM
October 14–15, 2020

WEDNESDAY, OCTOBER 14, 2020

All technical content will be published at 8:00 AM EDT

EXHIBITS OPEN
Welcoming Remarks
Silviu Pala, Automotive Display, Southfield, MI, US
KEYNOTE ADDRESS
Future Auto Cockpit Human Experiences
John Schneider, Director – Electrical, Software, Compute, and AI Technologies,
Ford Motor Company, Detroit, MI, US

SESSION 1: AUTOMOTIVE MARKET

Co‐Chairs: Silviu Pala, Automotive Display, Southfield, MI, US
Michael Boyd, Yazaki North America, Canton, MI, US

1.1 INVITED ADDRESS:

Automotive Display Market and User Interfaces Overview
Kyle Davis, IHS Markit, Southfield, MI, US

1.2 INVITED ADDRESS:

Automotive Touch Screen, Touchless Control, Micro and miniLED and
Smart Windows Market Forecast
Jennifer Colegrove, Touch Display Research, Inc., Santa Clara, CA, US

1.3 INVITED ADDRESS:

Display Industry Markets and Technologies – What Auto Display
Professionals Need to Know About the Rest of the Industry
Bob O’Brien, Display Supply Chain Consultants, Ann Arbor, MI, US

SESSION 2: DISPLAY AND HMI SYSTEMS

Co‐Chairs: Bruce Banter, Tech‐D‐P Inc., Northville, MI, US
Drew Harbach, Peterbilt Motors Denton, TX, US

2.1 Reflection Properties of AR Coated Flat and AG Glass Surfaces

Dave McLean, MAC Thin Films, Inc., Santa Rosa, CA, US

v
 2.2 IoT Intelligent Display Technology
Liang Zhou, Ling‐ling Zhang, Jiu‐bin Zhou, Jin‐e Liu, Feng Qinin,
Shanghai Tianma Microelectronics Co., Ltd., Shanghai, China

2.3 Display Module with Integrated Driver of Multi‐Screen

Liang Zhou, Lu Yao, Ling‐ling Zhang, Jiu‐bin Zhou, Wan‐chun Du, Jin‐e Liu,
Feng Qin, Tianma Micro‐Electronics Group, Shanghai, China

2.4 High Precision Optical Bonding for Free‐Form and Curved Displays
Eugen Bilcai, Henkel KGaA, Madison Heights, MI, US

2.5 Digitized Styling and Safety with Automotive Exterior Displays

Johnathan Weiser, Richard Nguyen, Kimberly Peiler, OSRAM Opto
Semiconductors, Novi, MI, US
Ulrich Kizak, OSRAM Opto Semiconductors, Regensburg, Germany

2.6 A Novel Approach for High Quality SNR in Sensing Applications

Gerald Morrison, SigmaSense, Austin, TX, US

SESSION 3: Head‐Up Displays

Co‐Chairs: Ross Maunders, FCA US LLC, Auburn Hills, MI, US
Dan Cashen, Continental Automotive Group, Auburn Hills, MI, US

3.1 Diffusive Microlens Array for Head‐Up Display Applications

Naoki Hanashima, Mitsuo Arima, Yutaka Nakazawa, Dexerials Corporation,
Tagajo, Miyagi, Japan
Kazuyuki Shibuya, Dexerials Corporation, Tome, Miyagi, Japan
Jingting Wu, Dexerials America Corporation; San Jose, California, US

3.2 Human Perception Studies of Head‐Up Display Ghosting

Steve Pankratz, William Diepholz, John Vanderlofske, 3M Company, St. Paul,
MN, US

3.3 Computational Holographic Displays for 3D AR HUD Using

Free‐Form Optics
Hakan Urey, CY Vision, San Jose, CA, US

3.4 Holographic Optical Elements and Projector Design Considerations for

Automotive Windshield Displays
Sam Martin, Jason Thompson, Texas Instruments, Inc., Dallas Texas, US
Ian Redmond, CERES Holographics, Ltd., St. Andrews, Scotland, UK

vi
TUTORIAL on Holography and Its Automotive Applications
Kai‐Han Chang, General Motors Global R&D, Pontiac, MI, US

EXHIBITOR PRESENTATIONS SESSION

Co‐Chairs: Bruce Banter, Tech‐D‐P Inc., Northville, MI, US
Eric Miciuda, Continental Corporation, Auburn Hills, MI, US

Presentations will be available from the following Exhibitors:

‐ AGC Plasma Technology Solutions ‐ Lumineq
‐ Dexerials Corporation ‐ MAC Thin Films, Inc.
‐ ELDIM ‐ Optics Blazers AG
‐ IHS Market ‐ Optronic Laboratories, Inc.
‐ Incom ‐ Panasonic
‐ Instrument Systems Optische ‐ Radiant Vision Systems
Messtechnik GmbHIwatani ‐ SigmaSense
Corporation of America ‐ Tanvavs
‐ Keiwa ‐ TouchNetix Limited
‐ Konica Minolta Sensing Americas ‐ TT Vision USA
‐ Kyocera Display ‐ Westboro Photonics

vii
THURSDAY, OCTOBER 15, 2020
All technical content will be published at 8:00 AM EDT

EXHIBITS OPEN
Local SID Chapter Awards
Bob O’Brien, Display Supply Chain Consultants, Ann Arbor, MI, US
Silviu Pala, Automotive Display, Southfield, MI, US

KEYNOTE ADDRESS
Voice of the Consumer | Technology and Mobility Clarity Today and Tomorrow
Kristin Kolodge, Executive Director of Human‐Machine Interface (HMI) and
Driver Interaction at J.D. Power

SESSION 4: DISPLAY METROLOGY

Co-Chairs: Kimberly Peiler, OSRAM Opto Semiconductors, Inc., Novi, MI, US
Vyacheslav Birman, Continental Corporation, Auburn Hills, MI, US

4.1 Understanding and Achieving Reproducible Sparkle Measurements

for an Automotive Specification
Ingo Rotscholl, Julia Brinkmann, Udo Krüger, TechnoTeam Bildverarbeitung
GmbH, Ilmenau, Germany
Jens Rasmussen, Elektrobit Automotive GmbH, Ulm, Germany
Christoph Rickers, Volkswagen AG, Wolfsburg, Germany

4.2 Measuring MicroLEDs for Color Non‐Uniformity Correction

Mike Naldrett, Matt Scholz, Bret Stonebridge, Austin Piehl, Anne Corning,
Shaina Warner, Radiant Vision Systems LLC, Redmond, Washington, US

SESSION 5: NEW DISPLAY SOLUTIONS

Co‐Chairs: David Lambert, Panasonic, Farmington Hills, MI, US
Jerzy Kanicki, University of Michigan, Ann Arbor, MI, US

5.1 Supervising (Automotive) Displays to Safeguard Camera Monitor Systems

B. Axmann, F. Langner, Mercedes‐Benz AG, Stuttgart, Germany
K. Blankenbach, M. Vogelmann, Pforzheim University, Pforzheim, Germany
M. Conrad, samoconsult GmbH, Berlin, Germany
J. Bauer, Karlsruhe University, Karlsruhe, Germany

5.2 Customized Local Dimming Algorithm and BLU for

Automotive Application towards Low Power Consumption and
High Visual Quality
Maxim Schmidt, Ramazan Ayasli, Chihao Xu, Saarland University,
Saarbrücken, Germany

viii
 5.3 Automotive Smart Surfaces: Conformable HDR Displays and
Smart Windows to Activate Almost Any Surface
J. Huggins, FlexEnable, Ltd., Cambridge, UK

5.4 The Functional Safety Designs of Vehicle Display Driver ICs

Cheng‐Chih Deno Hsu, Himax Technologies, Hsinchu City, Taiwan

5.5 Automotive Dual Cell microZone™ LCD Development

Paul Weindorf, Qais Sharif, Elijah Auger, David Whitton, Brian Hayden,
Visteon Corporation, Van Buren Twp., MI, US

5.6 A Low‐Power Transflective TFT‐LCD Based on IGZO TFT

Tenggang Lou, Lei Wang, Xiangjian Kong, Jine Liu, Feng Qin,
Tianma Micro‐Electronics Group, Shanghai, China

5.7 A MicroLED Device with 0mm Border

TengGang Lou, Tianma Micro‐Electronics Group, Shanghai, China

5.8 Enabling Features of VueReal MicroLED Technology for

Automotive Applications
Rexa Chaji, VueReal Inc, Waterloo, Ontario, Canada

5.9 New Challenges and Testing Solutions for Flexible

Vehicle Displays & Interfaces
Eisuke Tsuyuzaki, Bayflex Solutions, Alameda, CA, US

5.10 New Material Solutions for Automotive Displays. Interfaces, and

Applications
Volker Plehn, SABIC, Wixom, MI, US

5.11 An Alternative to OLED with Full‐Array Local Dimming in

Automotive Displays
Logan Cummins, Texas Instruments, Dallas, TX, US

PANEL DISCUSSION
Moderator: Bob O’Brien, Display Supply Chain Consultants, Ann Arbor, MI, US
Display and Interfaces for Autonomous Drive Including ADAS
Participants: Members of government, academia, industry

Live Q&A with Panelists and Moderator at 11AM EDT

ix
x
 Keynote Address

KEYNOTE ADDRESS
Future Auto Cockpit Human Experiences
John Schneider, Director – Electrical, Software, Compute, and AI
Technologies, Ford Motor Company, Detroit, MI, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

1
2
 Presentation 1.1

INVITED ADDRESS
Automotive Display Market and User Interfaces Overview
Kyle Davis, IHS Markit, Southfield, MI, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

3
4
 Presentation 1.2

INVITED ADDRESS
Automotive Touch Screen, Touchless Control,
Micro and miniLED and Smart Windows Market Forecast
Jennifer Colegrove, Touch Display Research, Inc.,
Santa Clara, CA, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

5
6
 Presentation 1.3

INVITED ADDRESS
Display Industry Markets and Technologies –
What Auto Display Professionals Need to Know
About the Rest of the Industry
Bob O’Brien, Display Supply Chain Consultants,
Ann Arbor, MI, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

7
8
 Reflection Properties of AR Coated Flat and AG Glass Surfaces
David McLean, MAC Thin Films, Inc.

Information displays are common in modern vehicles and their use, size and form factors continue to
increase. Display visibility in sunlight remains an issue even with the use of anti‐glare glass, which
breaks up the specular reflection, but does not reduce the overall intensity of reflected light. Aircraft
cockpit displays require both anti‐reflection (AR) and anti‐glare (AG) functionality to provide optimum
viewability. The same design considerations apply to the automotive use case. The use of touchscreen
panels requires an effective oleophobic (anti‐fingerprint) treatment to maintain appearance and for
ease of cleaning.

The CIE 1976 L*a*b* color space is commonly used to numerically describe color. Color is expressed as
three numerical values, L* for the lightness and a* and b* for the green–red and blue–yellow color
components.

The difference between two colors is useful for the purposes of

evaluating color match or perceived differences in color. This can be
expressed mathematically as ΔC (or ΔE if lightness is included). It is
generally accepted that the threshold for a noticeable color change is
about deltaC ~ 2.2.

Figure 1: Graphical Representation of CIE 1976

L*a*b* color in 3‐dimensional space

The reflected color and color shift when changing viewing angle are critical considerations when
designing a thin film anti‐reflection coating for avionic or automotive displays. This reflected color is

9
most noticeable most when the display is off. The position of the sun with respect to the observer and
associated viewing angle changes with vehicle position and the time of day, among other factors. It is
critical to maintain a consistent reflected color over a wide range of angles to ensure pleasing display
aesthetics and meet the demands of interior designers.

An optimized AR coating with integrated oleophobic (anti‐fingerprint) properties coated onto AG glass
surfaces provides the optimal solution for reducing glare from touch panel displays. The design
approach is to create a broadband visible anti‐reflection (AR) coating with a nominal blue‐green
reflected color which maintains low color saturation across a wide range of incident angles. The
theoretical reflected color for this coating design is shown in Table 1. ΔC is kept below 2.2 between the
most extreme viewing angles.

Table 1: Nominal Coating Design Color Values

Incident Angle Y a* b* ΔC
10 0.21 ‐0.85 ‐2.89 ref
20 0.20 ‐0.75 ‐1.83 1.06
30 0.26 ‐0.91 ‐0.17 1.67
40 0.54 ‐1.45 1.54 1.79
50 1.38 ‐1.36 1.64 0.13
60 3.84 ‐0.25 0.19 1.83

The coating design proposed in Table 1 was produced by MAC Thin Films, Inc. The measured properties
were found to be in good agreement with theory as shown in Table 2.

Table 2: Actual Measured Values

Incident Angle Y a* b* ΔC
10 deg 0.23 ‐1.70 ‐3.84 ref
20 deg 0.17 ‐0.79 ‐2.99 1.25
30 deg 0.16 ‐0.05 ‐1.53 1.64
40 deg 0.30 ‐0.23 0.27 1.81
50 deg 0.86 ‐0.83 0.50 0.64
60 deg 2.87 ‐0.12 0.08 0.82

10
Thin film AR coatings are comprised of multiple layers of high and low refractive index materials with
each individual layer typically < 150 nm. Critical control of the thickness of each layer is required to
achieve the designed visual functionality. Fingerprints on the
coating can change the AR behavior. A successful design
approach requires an anti‐fingerprint treatment to reduce the
transfer of skin oils to the display and allow for them to be
easily cleaned from the surface.

The measurements of the reflected light scattered from the AR

coating on AG glass were made according to the setup shown
in Figure 2. The nominal angle of incidence is represented by
theta (θ). The collection aperture is moved at “+” and “‐”
angles from primary incident angle to collect the scattered
light. A glass wedge and n=1.52 matching oil were used to
eliminate the second surface reflection and simulates the case
of an optically bonded display. Figure 2: Measurement configuration
The degree of scattered light increases as the gloss value of the
AG glass decreases. This is indicated by a reduced peak at 0
deg scatter angle, which represents the specular component of
the reflected light, as shown in Figure 3. A corresponding
increase in the proportion of the reflected light at larger
scatter angles was observed.

The ratio of diffuse (Rde) to total (Rdi = specular + diffuse)

reflectance is a convenient way to describe the degree of light
scattering of an optical surface. Figure 4 summarizes the
behavior of various uncoated AG glasses as well the color
neutral AR coating on 70 gloss AG glass. For uncoated AG
glass, the proportion of scattered light increases with
Figure 3: Scatter behavior by gloss value
increasing incident angle in a linear manner. The addition of
the AR coating significantly reduces the specular component, thereby increasing the ratio of scattered to
total reflected light. The visual effect is a significant reduction and glare and an improvement in sunlight
readability.

11
 Rde / Rdi Trajectory
0.80

0.75

0.70
35 Gloss
0.65 70 Gloss
Rde / Rdi

0.60 70 Gloss ‐ Coated

110 Gloss
0.55
Linear (35 Gloss)
0.50 Linear (70 Gloss)
0.45 Linear (110 Gloss)

0.40
20 30 40 50 60 70
Incident Angle

Figure 4: Scatter characteristics of AG glasses by incident angle

The measured reflected color values for the AR coated samples produced for this study are shown in
Figure 5 below. The results obtained are in good agreement with the design target. The use of AG glass
reduces the visible color change by scattering light at larger and smaller angles to the primary incident
angle, which has the effect of blending the reflected color.

Figure 5: Reflected Color Measurements of Coated Samples

12
Photographic samples of the reflected color were made for the purpose of making a visual comparison
and are shown in Figure 6. The uncoated glass has a neutral color but a strong reflection with Y = 4.2%
and L* = 24.4. This can make the display hard to read in sunlight. A “Standard AR” coating greatly
reduces the magnitude of the reflected light with Y < 0.3% and L* < 2.7, an order of magnitude reduction
in apparent intensity, as indicated by a comparison of L* values. However, this coating demonstrates a
noticeable red‐shift in the reflected color when the viewing angle changes from 10 degrees to 45
degrees. This color shift is typical of common AR coatings in use today.

Figure 6: Reflected Color Measurements of Coated Samples

The optimized AR coating on standard flat glass maintains a blue‐green reflected color over the range of
viewing angles while still providing a drastic reduction in the magnitude of the reflected light.
Combining this coating with an AG glass reduces the color saturation at all viewing angles.

A comprehensive AR coating design, including an oleophobic optical layer, has been developed and
manufactured. This coating provides minimal color saturation and consistent color over a wide range of
viewing angles. The addition of this AR coating to AG glass significantly reduces reflectance and
increases the ratio of diffuse to total reflected light. Consequently, it drastically reduces glare from the
sun. Like in avionic displays, automotive display viewability is optimized by the addition of an AR coated
AG cover glass with effective anti‐fingerprint treatment.

13
14
 IOT intelligent display technology
Liang Zhou, Ling‐ling Zhang, Jiu‐bin Zhou, Jin‐e Liu and Feng Qin
Shanghai Tianma Microelectronics Co., Ltd.

Abstract: 2. Architecture of IOT display module

This paper introduces a display technology suitable Figure 1 shows the proposed IOT display architecture.
for IOT smart home. The panel of the display The DDI drives the screen with source and gate circuit,
technology product integrates the COG display driver and MCU IC drives the DDI with display interface such
IC and MCU IC. The MCU contains an 8051 or ARM as RGB interface. Besides, the MCU IC communicates
core and FPC includes FLASH module and Clock with the devices on the FPC, between them
Module. The I2C or SPI instruction interface replaces instructions are transmitted through I2C or SPI
the traditional display data interface. MCU controls interface.
other devices of intelligent terminal products
The devices on the FPC include the communication
through I/O interface to realize the integration and
module such as WIFI, Bluetooth and Zigbee , the input
intelligentization of terminal products. The display
devices such as sensor、touch controller、microphone
technology uses I2C or SPI interfaces to interact with
and camera, the output device such as speaker, DAC
the communication module instructions and is
and switch. The communication module exchanges
compatible with all kinds of communication
information with cloud and internet net.
protocols, such as Wifi, Bluetooth, Zigbee, etc. The
technology can be applied to IOT intelligent display
terminal products with different demands.
Keywords：IOT; smart home; MCU; Intelligent display

1. Introduction
Usually a simple display system consists of a display
module and a driving controller. The DDI(display
driver IC) is bonded on the edge of display panel,
called COG (chip on glass); the MCU IC is bonded on
the driving board. A significant fraction of cost is
associated with the external ICs for small to mid‐size
Figure1. IOT display architecture
display modules.
Figure 2 shows the working principle of IOT intelligent
In this paper, we describe the MCU IC is packaged in display module. Input devices or communication
COG, bonded on the edge of display panel with DDI. modules give instructions to MCU IC, for example, you
The technology increases the cost‐competitiveness of can touch the TP KEY or TP panel, or you can operate
display system, and realizes the integration and with APP. The instructions from touch sensor include
intelligentization of products. The technology can be switching UI, displaying time and controlling smart
applied to IOT intelligent display terminal products home. There is no need for main board to transmit UI
with different demands. images to MCU, only instructions are needed, and
then MCU reads UI images from flash on the FPC

15
according to these instructions. UI images are cost of the system. Besides, the precision of COG
beforehand stored in the flash. bonding is much higher than COB SMT process, more
pins are allowed in smaller areas. This will improve
the integration of the system. Furthermore, MCU and
DDI can be combined as one chip, if the process is
feasible.

Secondly, the display interface bus is shorter than

traditional system, because MCU IC bonded on panel
is closer to DDI. The display signal does not need to
pass through PCB and FPC, so it is not susceptible to
Figure2. The working principle of IOT intelligent display module interference. The product will have better reliability.

Figure 3 shows the internal diagram of MCU. Image Finally, the development cycle of the terminal system
data is cached in the display ram through RAM will be greatly shortened because the complex display
controller, graphics controller builds in geometry processing is finished by MCU IC of display module
drawing function, support the functions of drawing instead of host main board. The development of host
dot, line, curve, ellipse, triangle and rectangle. main board just focuses on the power control of
Hardware graphics accelerator is also embedded to terminal product and simple instruction set with MCU.
realize picture rotation, flipping, mirroring and picture MCU will realize the control of equipment through
in picture, transparent display. This architecture relays according to responding instructions. For
reduces greatly the loading of software of MCU. example, adjust the air conditioner switch,
temperature and air volume through your cell phone
This display module is very useful for Household APP or central console screen. Besides, for terminal
Electric Appliances, Industrial control, electronics product, the product supply chain will be simplified
instruments, Medical equipment, Human interface, without external MCU IC. It not only shortens the
Testing equipment, Lift indicators and so on. development cycle, but also makes the later
maintenance and upgrade easier.

4. Conclusion
The IOT display module uses COG package MCU IC on
panel instead of COB IC on board to realize the
integration of terminal products. The display
technology uses I2C or SPI interfaces to interact with
the peripheral including communication module
Figure3. Internal diagram of MCU instructions and is compatible with all kinds of
communication protocols, such as Wifi, Bluetooth,
3. Performance of IOT display module Zigbee, etc to realize the intelligentization of terminal
First, this MCU IC belongs to SOC and its size is much products. The technology can be applied to IOT
smaller than discrete COB devices. And it is bonded intelligent display terminal products with different
on panel, PCB areas can be saved. This will reduce the demands.

16
 Display module with integrated driver of multi-screen
Liang Zhou, Lu Yao, Ling-ling Zhang, Jiu-bin Zhou, Wan-chun Du, Jin-e Liu, Feng Qin
TIANMA MICRO-ELECTRONICS GROUP, Shanghai, China

Abstract 2. Architecture and principle of integrated driver

This article introduces a way to integrate the image of multi‐screen
segmentation processing function on each display As shown in Figure 1, each display module's
module without the driver boards of multi‐screen. interface plate is set with the ID (for example, using
For example, the integrated processing chip which a dial switch) and the host terminal passes
is integrated on the FPC or PCB of each display configuration parameters (row, column address,
module can set the row and column addresses etc.) for modules with different IDs through serial
according to the corresponding module ID. port, and the FPGA intercepts them according to its
According to the addresses, the chip can intercept own address. Taking its own pixel data from the
own display data from the source data bus, and data bus, FPGA sends it to RAM for caching, and
send the data to cache RAM. Then reducing then reduce the PCLK frequency to the normal
frequency, it can refresh the corresponding display refresh rate for display. The FPGA collects all the
by matching frame rate. As the number of splicing pixel data on the bus (the sampling clock is based
unit screens increasing, this way can save expensive on the signal source) and which pixel data will be
costs by removing the driver boards of multi‐screen. stored in RAM depends on the ID setting of the
plate.
Author Keywords
multi‐screen; Image segmentation; Driver of multi‐screen

1. Introduction
At present, for the splicing screen in the market,
usually each splicing screen will come with a
splicing drive board, where there is a splicing drive
board control chip and image processing chip. With
the increasing number of splicing unit screens,
splicing driver plates cost a lot. This paper
introduces a scheme to integrate processing chip
Figure 1. Architecture of integrated driver of multi‐screen
(such as FPGA) into FPC or PCB of display module.
According to the ID setting of each module, the
responding display data is intercepted on the data For example, there are nine screens in total, and
bus by the chip and after processing the cached each unit screen has its own interface plate, using
data is sent to its own screen. the dial switch or other ways to set the ID number
(such as 0001, 0010,...1001 in order respectively).
The FPGA on each cell screen will read the ID
number on the responding interface plate to
confirm the pixel data it needs. For example ID

17
number 0110 is read at 6th block, which is in row 2 4. Conclusion
and column 3, the FPGA displays lines 641‐1280 This paper introduces a way to integrate the image
and columns 2561‐3840 of the source image (e.g., segmentation processing function on each display
resolution 3840*1920) on the bus. The pixel data module without the driver boards of multi‐screen.
(1280*640) is first stored in RAM and then read As the number of splicing unit screens increasing,
from RAM to drive display module in this way can save expensive costs by removing the
1280*640@60Hz. The host side can transmit driver boards of multi‐screen.
configuration parameters such as unit screen
resolution and frame frequency to FPGA through
serial port or other low‐speed interface.

3. Performance
This method can save the expensive cost of splicing
drive plates. The ID number is adjustable to make
the replacement of the unit screen more flexible.
FPGA self‐addressing according to parameters
ensures simpler settings on host side.

18
 Presentation 2.4

High Precision Optical Bonding for Free-Form

and Curved Displays
Eugen Bilcai, Henkel KGaA, Madison Heights, MI, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

19
20
 Digitized Styling and Safety with Automotive Exterior Displays
Johnathan Weiser*, Richard Nguyen*, Kimberly Peiler*, Ulrich Kizak**
*OSRAM Opto Semiconductors, Novi, MI, USA 48377
**OSRAM Opto Semiconductors, Regensburg, Germany 93055
[email protected]

Abstract

Modern display technologies allow for increasingly

sophisticated animated digitized elements to be
implemented on automotive exteriors. These displays
can allow for both unique signature lighting elements
to a vehicle and can bring increased awareness to
other road occupants to the intentions of the driver. In
the present paper, we discuss future concepts of
automotive exterior displays that allow for increased Figure 2. OSIRE E4633i LEDs from OSRAM in a daisy-
safety and styling throughout the vehicle such as in the chained architecture with integrated driver IC and pre-
taillights and exterior paneling. Future regulation calibrated LED data
may be shaped by increased interest in dynamic
Multi-die packages are also available with built in
lighting aesthetics and displays on vehicle surfaces
drivers connected to new communication protocols,
including the use of monochromatic and multi-color
such as ISELED, which allow for individual
amorphous displays.
addressing of over four thousand mixed Red, Green,
Keywords and Blue (RGB) nodes from a single communication
Automotive Exterior Lighting; Signaling Displays; interface. These methods increase the capability to
Dynamic Lighting; Sequential Turn; LED Matrix; provide unique, dynamic, and impactful lighting
Digitization; Pedestrian Communication effects to the exterior of vehicles.

Introduction Dynamic Exterior Styling

Advancements in LED technology have allowed for Unique styling plays a major part in automotive
smaller, brighter, and more robust parts which has design. As such, lighting effects have become more
expanded their flexibility throughout vehicles. prominent recently in exterior elements of the vehicle.
Additionally, new driver modules have expanded the Dynamic LED lighting is an excellent way to
capability to support large numbers of individually introduce these effects into a vehicle’s design
addressable nodes in more space constrained language. Sequences for welcome lighting can greet
environments. soon-to-be occupants while accentuating features of
the vehicle.

Figure 1. OSRAM TOPLED E1608 LED

Small packaged LEDs with a size such as 1.6 x 0.8 x
0.6mm allow for narrow pixel pitch monochromatic
displays with 2mm pitch. Due to the high brightness
of using individual LEDs for each pixel, high contrast,
and hidden illumination effects behind materials can
be achieved as well. Figure 3a. Concept door with LED display for messaging

21
 These collisions also tended to involve driver
distraction. Car manufacturers are increasingly tasked
with designing taillights that are aesthetically pleasing
while also aiding to improve driver safety allowing
stop, turn, tail, and hazard to be recognized as quickly
as possible.
Figure 3b. Concept door with customized greeting Digital rear combination lamps (RCLs) provide a wide
amount of flexibility in developing patterns of various
Utilizing a matrix of LEDs as a display can add further
size, position, resolution, and animation rate. In a
personalization in welcome features, Figure 3b,
OSRAM sponsored Lighting Research Center study,
improving quality for services such as ride share and
participants rated on-axis patterns to be easier to
delivery by introducing clearly visible, unique
perceive, and higher resolution patterns to be smoother
identifiers to the exterior of the vehicle. LED displays
visually [2]. Participants also reacted to larger patterns
also introduce use cases such as the capability for
more quickly [3]. Because of the flexibility of digital
advertising, due to the high resolutions they provide to
displays, they can be placed in stylistic locations,
show text and logos. LEDs are also brighter and more
while still placing indicator patterns in visually
robust than alternative display technologies, making
relevant locations. For example, a display could wrap
them more suitable for exterior environments,
from the rear of a vehicle onto the side panel to include
especially with regards to daylight readability and use
both legal functions, and stylistic elements in a single
behind tinted lenses. Figures 3a and 3b depict a RGB
unit. Furthermore, the flexibility of dynamic LED
LED array utilizing a 1mm pitch, exhibiting over
arrays developed as a platform allow the core design
3000cd/m2 enabling direct sunlight readability.
to be implemented into multiple vehicles across a
lineup with unique styling for each vehicle that can be
easily augmented via software.

According to a research study by UMTRI [4] on

flashing brake light systems, flashing brake systems
and flashing hazard systems reduced drivers brake
response times and crash frequency in each scenario as
compared to traditional brake indicators. Shape
changes, colors, and moving effects or animations are
more examples of dynamic signaling that could impact
brake response times.

Taillights can be designed such that dynamic effects

are possible. There have been instances of animated
Figure 4. Rinspeed Metrosnap concept shown at CES 2020 signaling on vehicles as early as 1968 on the Ford
Mustang Shelby GT500. Other examples include
Figure 4 shows a concept electric vehicle with a LED vehicles, such as the 2016 Audi Q3. Both these
matrix display on the front and rear for both pedestrian examples implemented sequential turn, which was
messaging and front or rear functions. It also includes implemented in North America through the use of a
LED backlit stripes on the sides to indicate the lock legal turn indicator zone plus redundant additional
status of doors to pedestrians who approach the zones dynamically sequenced to indicate turning.
vehicle. This is one such method that LED arrays can There also have been instances of taillights that change
be utilized on a vehicle exterior. shape from tail function to brake light such as the
European model of the 2014 Volkswagen Passat. The
Safety and Visibility ‘Click-Clack’ effect caused the taillight to visually
The National Highway Traffic Safety Administration shift from a horizontal taillight to a vertical brake light.
(NHTSA) has conducted a study[1] finding that a Volkswagen claims “this signal change underlines the
significant percentage of all car accidents are from recognizability of the brake, and thus increases the
rear-end collisions. Within those collisions, a majority road safety.” [5] to expand on this, some additional
were due to the preceding car decelerating to a stop. features that can be put into taillights are animated

22
braking signals, reversing signal, and displaying conveying information to drivers and pedestrians.
symbols to warn of driving conditions (heavy rain, ice, Concepts and demonstrators are essential for
snow, fog, dust, traffic). An emergency signal or SOS supporting these studies.
feature could also be added, as well as symbols that
represent an autonomous vehicle. This can all be made Digital RCL Concept
possible into one taillight product if an exterior display The function of a RCL is to have a single light module
is utilized. that performs multiple tasks. These tasks include tail
function, stop, turn, and a reversing light. Each of
these functions have federal regulations and
requirements that are stated in Federal Motor Vehicle
Safety Standards (FMVSS) 108 [6]. Manufacturers
must pass specific photometry, luminous lit area,
distribution of intensity, color and lamp position
requirements. Stop and turn have the most stringent
requirements. These functions require 5,000 mm 2 of
luminous lens area and must satisfy minimum
intensity values at different test points. Intensities are
measured at various test points at angles off-center
Figure 6. Concept hybrid digital RCL with animation horizontally and vertically. The highest minimum
intensity is defined as on-axis to the lamp.
It is increasingly important for pedestrians to see and
comprehend what vehicles are doing, especially in According to a human spatial resolution study [7], fine
dense urban environments. With the rise of pedestrian pitch is relatable to the viewing distance from the
distractions, as well as quiet EV vehicles reducing display. In the study, subjects were tasked to view
auditory cues, noticeable reverse lights, animated displays with LEDs placed at different pixel pitches,
signals and messaging on exterior displays along the then asked to find the distance from the display that
side or rear of the vehicle may aid pedestrian has the most homogenous image. Results found that at
awareness. Figure 3a depicts a door retrofitted with a 3mm pixel pitch, a distance of 5.2 meters from the
LED display which is indicating to pedestrians that the display was needed in order to see the optimal
vehicle is driving in an automated mode. This can also homogenous image. smaller pitches would decrease
be used to indicate other intentions such as a vehicle this distance.
reversing, or doors being opened. Figure 6 depicts a
We introduce here a digital RCL concept targeting the
RCL with a LED display to show animated stop and
requirements above. This concept uses a fine pitched
turn functions, as an additional feature to traditional
matrix of LEDs to create a mono-chromatic display
dedicated indicators, which are intended to more
that can show static and dynamic images using
clearly indicate the driver’s intentions to other road
commercially available automotive grade
users.
components. We realized a slim 1:5 aspect ratio 2mm
Manufacturers and consumers are awaiting new pitch display to fulfil luminous lit area requirements of
regulations from global committees that will allow for the FMVSS regulations for a stop lamp at 5,000mm2
making these concepts a reality. Development of with a homogenous image at a distance less than 5m.
requirements to support new signal lighting
technologies is on the agenda of main global standards
and regulation bodies such as GTB, and SAE.

Academia is carrying out independent research studies

to evaluate novel lighting technologies and their
applicability to exterior lighting. Industry proposals
supported by academic findings are then presented to
regulators for their consideration. This is a vital step
toward seeing new functionalities for exterior lighting
and displays on vehicles. They study the safety Figure 7. RCL concept lamp and matrix array pcb
benefits of certain technologies, as it pertains to

23
The concept consists of multiple scalable modules Each graphical pattern should be verified to meet local
designed in a 2mm pitch 24 x 24 LED array driven by legal requirements prior to implementation. This paper
3 multiplexed 48-channel drivers for independent is only intended to discuss feasibility from a lighting
LED control. Figure 8 shows a block diagram of the technology standpoint. Figures 9 show potential
architecture of this concept. exemplary graphics that could be implemented with a
digital RCL.

Conclusion
Arrayed LEDs allow for a multitude of new designs
and applications to the exterior of a vehicle from
dramatic welcome effects, to innovative ways to
display safety critical information. The digital RCL
concept shows one exemplary way to implement this
solution including achieving this, both meeting size
and brightness requirements allowing for dynamic
new RCL designs. We look forward to seeing what
new designs can become a reality through the use of
these technologies.

References
Figure 8. Example matrix display control scheme
1. S. Lee, E. Llaneras, S. Klauer, J. Sudweeks,
We selected an OSRAM TOPLED E1608 KR “Analyses of Rear-End Crashes and Near-Crashes in
DELMS1.22 for our LED due to its small size and the 100-Car Naturalistic Driving Study to Support
suitability for exterior use. The average intensity of Rear-Signaling Countermeasure Development”,
this part from the SH brightness group at 20mA and Virginia Tech Transportation Institute, NHTSA
25C ambient is 225mcd. Available brightness and (2007).
LED selection may change over time based on
2. K. Peiler, J. Weiser, J. Bullough, N. Skinner, M.
technological advancement. We target 80cd minimum
Rea, “Responses to Dynamic Peripheral
from the lamp for the single cavity H-V requirement.
Communication for Automotive Interior Lighting”,
The concept utilizes a 4:1 multiplexing, M, with a total SID Vehicle and Interfaces (2019).
of 2,880 LEDs, N. We assume a thermal loss of 30%,
3. K. Peiler, J. Weiser, J. Bullough, N. Skinner, M.
𝜂t, with an assumption of LED junction temperature,
Rea, “Dynamic Peripheral Communication for
Tj, of 60°C and an optical loss of 15%, 𝜂o, from lenses.
Advanced Automotive Applications”, SID Digest
The resulting [JJ1]96.4cd, I, meets our target intensity
(2019).
requirements.
4. G. Li, W. Wang, S. Li, B. Cheng, P. Green,
(𝐼𝑣 ) ∗ 𝑁
( ) ∗ 𝜂𝑡 ∗ 𝜂𝑜 = 𝐼 “Effectiveness of Flashing Brake and Hazard
𝑀
Systems in Avoiding Rear-End Crashes”, Advances
in Mechanical Engineering (2014).
Equation 1. Estimation of lamp intensity
5. Volkswagen, “Signal lights – Switching signature
tail light”, https://www.volkswagen-
newsroom.com/en/evolution-of-light-4261/signal-
lights-switching-signature-tail-light-4271 (2018).

6. United States Department of Transportation,

“Lamp, Reflective Devices, and Associated
Equipment”, Federal Motor Vehicle Safety Standard
No. 108 (2015).

7. S. Merl, “Physiological investigation of human

Figure 9. exemplary digital RCL graphics resolution on LED pixel grid displays”, OSRAM
(2017).

24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
 Diffusive Microlens Array for Head-Up Display Applications
Naoki Hanashima1, Kazuyuki Shibuya2, Mitsuo Arima1, Yutaka Nakazawa1, Jingting Wu3
1
Dexerials Corporation; Tagajo, Miyagi, Japan
2
Dexerials Corporation; Tome, Miyagi, Japan
3
Dexerials America Corporation; San Jose, California, USA
[email protected]; [email protected]; [email protected];
[email protected]; [email protected]

Abstract: The goal of this paper is to present solutions widely used in microoptics such as sensors, light
for optimizing image quality in various picture generating modulators, and optical interconnects, etc. [1].
units (PGU) for automotive Head-Up Displays (HUDs).
HUD units in current production usually have LED-based
light sources, while laser-based light sources, such as
DMD or MEMS, are on the rise. Since wider field-of-view
and better image quality of HUD systems are the design
goals of Tier 1 suppliers and original equipment
manufacturers (OEMs), diffusive optical components are
needed to balance brightness and uniformity of the Cover (Glare trap)
projected information. Conventional frosted glass and
coated diffuser films typically have Gaussian distribution
profiles, which leads to sub-optimal brightness uniformity
and poor confinement of the LED backlights. Microlens Free‐shape
array (MLA) optical components can be engineered to concave mirror
control shape and uniformity profile of the transmitted Cold mirror

light. A diffusive MLA (DMLA), with top hat distribution, TFT‐LCD

is compared to conventional diffusers. Dexerials explores Lens
novel optical design technology to enhance light LED
efficiency and improve wider field of view (FOV) and
Diffuser
image quality of HUDs.
Figure 1: Typical HUD optical path; PGU type is
Keywords: Microlens array, Diffuser, Projector, UV TFT-LCD.
resin, UV Imprint, HUD, PGU, TFT-LCD, DMD, MEMS

Dexerials fabricates a diffusive MLA (DMLA) that is

1. Introduction
tailored for HUD systems. A detailed comparison of
Following the design trend and targets of both Tier 1 conventional diffusers and DMLA is shown in Table 1.
suppliers and OEMs, automotive HUDs have emerged as
Like diffusive layer of randomly dispersed microbeads in
an additional source of information that enhances the
the coated diffuser film, random microstructure of frosted
driver’s experience. Currently, the primary PGU
glass limits control of light distribution. The frosted glass
technology is based on thin-film-transistor liquid-crystal
and coated film are useful for achieving a uniform
display (TFT-LCD) (Figure 1). This paper investigates
brightness. DMLA, on the other hand, is advantageous for
performance of various diffusers in PGUs.
controlling light distribution and can be engineered to
As shown in Figures 1 and 2, a diffuser component in the have a top-hat profile.
PGU is placed between the LED board and TFT substrate
Dexerials’ DMLA, compared to conventional diffusers,
to homogenize incoming light from the LED board, and
can provide greater uniformity of projected image,
transmit the uniform light to the TFT-LCD, which are
improve light efficiency, reduce power requirements of
important requirements for HUD applications.
PGU, and contribute to heatsink downsizing. DMLA is
A conventional diffuser is an optical component with a also easier to scale up in the UV imprinting
Gaussian, or normal, distribution. Microlens array is a manufacturing process.
class of optical component that is engineered to control
the shape and uniformity profile of light passing through.
In addition, it can focus and collimate light. MLA is

41
 Table 1: Comparison of Gaussian coated diffuser, Gaussian frosted glass diffuser, and diffusive MLA.
Diffuser type Structure Feature

Coated Diffuser Microbeads -Lower transmittance by internal scattering

(Film) dispersed resin -Limited control of microbeads diameter and light distribution

Frosted Diffuser Random corrugated -Limited control of wet etched structure and light distribution
(Glass) microstructure -High reliability by being made fully of inorganic material

Diffusive -High transmittance by surface diffusion

Microlens Array Designed microlens array -High controllability of designed microstructure and light distribution
(glass, film) -Acceptable cost by UV-imprint process

Not used for display Minimized loss 3. Design

A novel design approach was used to create the surface
profile of the DMLA component. Considerations include
concavity of the single lenses, packing density of the lens
group, individual microlens positioning and aperiodicity
Enhanced according to designed statistical distribution. The size of
brightness
and
each microlens was also investigated and optimized to
Gaussian diffuser uniformity Top‐hat diffuser minimize unwanted granular and textured image.
Radius of lens curvature and pitch between adjacent lens,
Figure 2: Gaussian diffuser and top-hat diffuser light referred to as lens size in Figure 3a, are major design
distribution from LED light sources, showing light parameters for a non-diffusive MLA. While both convex
confinement property. and concave lens arrays (Figure 3b) are applicable for
diffusive applications, the arrangement of each lens is
more critical for distribution control. Packing density of
100% (defined as the lens area divided by the non-lens
2. Benefits of DMLA for HUD applications spacing area) is necessary in order to avoid transmitting
After thorough HUD system-level evaluations, it was pass-through rays. Suppression of diffraction is also
concluded that top-hat diffusers outperform Gaussian optimized to prevent unwanted colorization and
diffusers and could lead to power saving. unevenness in brightness.
The following lens parameters were optimized to ensure
As visualized in Figure 2, using a Gaussian diffuser in the
desired light distribution profile, such as flatness around
backlight unit of PGU could result in non-uniformity
zero angle and steepness at the cutoff angle (Figure 3c)
around individual LEDs in the backlight unit. This is due
[2, 3].
to the brightness being the greatest at the center of the
LED while the surround area would be dimmer. The light  Lateral and vertical height
map and distribution can be evened out with multiple  Lateral and vertical arrangement
LEDs in the backlight unit. Although HUD image
uniformity can be achieved with conventional diffusers,  Radius of curvature
the luminance required is greater due to light scattering
from TFT-LCD substrate.
Unlike conventional diffusers, Dexerials’ DMLA uses Pitch Radius
surface scattering mechanism and doesn’t have light loss
because of internal scattering. Therefore, the DMLA has
higher overall transmittance while providing adequate
diffusivity. An advantage of using a better confined top- Height
hat diffuser is meeting the image brightness and
uniformity specifications with lower leakage of PGU Figure 3a. Major lens parameters.
light.

42
 Concave MLA Convex MLA

Figure 3b. Concave and convex MLA and its

diffusive operation.

1.0 Flatness
Normalized intensity

0.5
Cutoff
FWHM angle

Angle

Figure 3c. Light distribution profile and specific

parameters from LED light source.

Dexerials’ DMLA is comprised of identical or unidentical

MLA unit cells tiled side-by-side so that each lens across
the boundary of the cell is connected continuously, as
Figure 3e. Texture-like outlook with different unit cell
illustrated in Figure 3d. Designing smaller individual lens
size and same lens size.
sizes while maintaining a larger unit cell size is better for
minimizing undesired patterns. Tiled DMLA with
different unit cell size is presented in Figure 3e. With a
lens size around 30um small and large unit cell sizes,
0.8mm x 0.8mm and 4mm x 4mm, respectively, the latter
exhibit no granular or texture-like features.
4. Fabrication for Automotive Industry
4.1 Inorganic (Glass) DMLA:
DMLA is fabricated on glass wafer by conventional
photolithography process as shown in Figure 4a. Positive
photoresist (PR) is coated on borosilicate glass substrate,
and patterned by UV exposure with gray scale
photomask, followed by a developing step and Reactive
Ion Etching (RIE). A concave MLA pattern is then
formed on the glass surface. To improve transmittance,
both sides of the DMLA are treated with broadband anti-
reflection (AR) coating (Figure 4b).
Figure 3d. Tiling of MLA. Fabricated DMLA is then evaluated using a beam profiler
at wavelength of 450nm and spectrophotometer. The top-
hat light distribution profile in circular shape indicating
isotropic diffusion is shown in Figure 4c. Full-Width at
Half-Maximum (FWHM) angle of 5 degrees and

43
transmittance of 98% with AR coating are obtained. No
apparent noisy diffraction is observed. This is an outcome
attributed to the optimized aperiodic lens design. The
DMLA with rectangular light distribution has the same
fabrication process while using a different MLA pattern.
Typical optical properties of Dexerials’ inorganic (glass)
DMLA are specified in Table 2. As a result of the top-hat
distribution, improved efficiency is verified in various
laser projectors in contrast to the frosted diffusers.

Glass
Substrate

Photoresist
coating

UV exposure
Photomask

Developing Figure 4c. Light distribution property of circular (top)

and rectangular (bottom) DMLA from laser light
source.
RIE

Table 2. Optical properties of glass DMLA

Evaluation (Inorganic diffuser)
Property Circular type Rectangular type
Figure 4a. Fabrication process flow of glass DMLA Diffusion angle 2~9 degree 3~9degree
(Inorganic diffuser). (FWHM)
Accuracy < ±0.7 degree < ±0.7 degree
Transmittance >96% >96%
Size (max) 100mm x 100mm 50mm x 50mm
AR coating Substrate Quartz glass, Quartz glass,
Borosilicate glass Borosilicate glass

Glass
4.2 Organic DMLA, UV imprinted:
AR coating
Organic DMLA can be fabricated on various film
Figure 4b. Cross section of glass DMLA (Inorganic substrates for backlight unit application in advanced
diffuser). LCD/TFT-type HUD (Figure 1), where top hat
distribution can improve light efficiency (Figure 2).
As indicated in Figure 5, the first fabrication step of film
MLA is dispensing acrylic UV-curable resin on PET
substrate. A glass master mold with concave lens
curvature is imprinted onto the uncured resin, thereby
forming a convex pattern on the resin. The UV light is
used in the next processing step to instantly cure the resin.
The polymer itself is engineered to ensure imprinting
capability, mechanical hardness, and a good adhesion to

44
the plastic substrate while maintaining its optical 14
properties in various automotive reliability tests. Optical
12

FWHM angle (deg)

properties of fabricated film DMLA and the result of high
temperature (105 degree C) storage test is shown in
10
Figures 6a and 6b, respectively. With stable optical
values, the final projected image benefits from decreased 8
MURA.
6

Film
4
substrate Glass master mold 0 100 200 300 400 500 600
Time (hours)
UV curable resin
Resin 100
dispense

Tranmittance* (%)
95
UV cure PET

90
Demold
85
Evaluation
80
0 100 200 300 400 500 600
Figure 5: Process flow of film MLA. Time (hours)

Figure 6b: High temperature reliability of film DMLA.

1.2
1 5. Conclusion
Intensity(a.u)

0.8 In conclusion, Dexerials’ DMLA features top hat light

0.6 distribution. Dexerials’ inorganic (glass) and organic
(UV-imprint) DMLA, which both exhibit good light
0.4 distribution confinement and uniformity within FWHM
0.2 angle, are expected to produce better light efficiency in
0 HUD and other projection displays than conventional
-10 -8 -6 -4 -2 0 2 4 6 8 10 diffusers.
Angle(deg)

100 Glossary of Terms

Total transmittance (%)

MLA: microlens array

95
DMLA: diffusive microlens array
90 FWHM: full-width at half maximum
HUD: head-up display
85
PGU: picture generating unit
80 TFT-LCD: thin film transistor liquid crystal display
400 450 500 550 600 650 700
DMD: digital micromirror device
Wavelength(nm)

Diffusive angle (FWHM) 10.2 degree

Total transmittance (Tt) 89.4%

Figure 6a: Optical properties of film DMLA.

45
References 3. N. Hanashima. et al., “Diffuser plate, display device,
1. Dan Daly, “Microlens Arrays”, CRC Press,2000, projection device,” WIPO, <https://patents.google.
ISBN 9780748408931 com/patent/WO2017010257A1/en>. Access July
2. M, Arima., et al., “Optical body, diffuser plate, 2020.
display device, projection device,” Patent website.
<http://www.freepatentsonline.com/y2019/0369297.
html > Access July 2020.

46
 Human Perception Studies of Head Up Display Ghosting

Steve Pankratz Ph.D., William Diepholz, John Vanderlofske Ph.D.

3M Company, St. Paul, MN, USA

Contact author email: [email protected]

luminance (depending on glass tinting), DA is typically

Abstract sufficient to characterize HUD image legibility.
To achieve high-quality, legible images in automotive Head Up
Displays (HUDs), ghost images must be mitigated. To date, this However, if other approaches to ghost mitigation—such as a
has been accomplished with a wedge-shaped PVB interlayer in reduction in ghost luminance—are used, then DA alone is not
the windshield (WS) that aligns the outer and inner surface adequate for specifying the impact of ghosting on image quality.
reflections. The metric to determine the degree of ghosting has 3M™ Windshield Combiner Film is a P-pol combiner film
been disparity angle (DA, the angular distance between the designed to reflect P-polarized light; when used with a P-pol
image and ghost). A new ghosting mitigation strategy uses P- HUD, this film enables a central image and two very weak ghost
pol combiner films, which rely not on geometric compensation images with relative ghost luminance (RGL) values often < 5%
but on ghost luminance reduction. To characterize this (Figure 1c). [7] In this regime, image legibility still depends on
mitigation strategy, another metric—relative ghost luminance disparity angle but is affected more by relative ghost luminance.
(RGL)—is required. An initial ghost perception survey shows
the relative impact of DA and RGL on perceived image quality.
The data suggest that DA alone fails to adequately characterize
image quality in a WS employing a P-pol combiner film; RGL
must also be considered. Further, for most real-world scenarios,
P-pol combiner film-type ghosting is not noticeable or is deemed
acceptable by most viewers.

Figure 1: Comparison of geometric (wedge) and

Keywords luminance (P-pol combiner film) ghost compensation.
Head Up Displays (HUDs); windshield; ghost images;
Perfect ghost compensation can be achieved with a
perception; disparity angle; relative ghost luminance; ambient
wedge WS for one viewer location, virtual image
contrast ratio.
distance (VID) and ray angle (e.g. center of image), as
in (a). Compensation degrades for variations in these
1. Introduction parameters (b). A P-pol combiner WS (c) produces
Head Up Displays (HUDs) provide safety are becoming more two very weak ghosts with non-zero disparity angle.
common in automobiles and their adoption across the
automotive market is expected to increase.[1][2][3] Advances in To date, perception studies of HUD ghosting and its impact on
optical systems modeling, projector and imager design, and image legibility have not included RGL.[8] Therefore a ghosting
insert-molding technology have enabled newer HUDs to perception study was carried out to determine the relative impact
produce bright, high-quality images that can offer augmented of DA (defined as the angular displacement between the image
reality (AR) functionality.[4] One of the key optical components and its ghost), RGL (defined as the ratio of ghost to image
in current HUD systems has been the windshield (WS) itself; it luminance) and Ambient Contrast Ratio (ACR, the ratio of
must be designed to effectively mitigate the ghost images that image luminance to background luminance). A high ACR value
occur when light is reflected off both its interior (S4) and corresponds to low light driving conditions—such as at night or
exterior (S1) surface. The standard approach is to incorporate a in a tunnel—while a low ACR occurs when the background is
PVB interlayer with a linearly or nonlinearly varying thickness bright.
across the WS in the vertical direction.[5][6] This small wedge
angle allows the S1 and S4 reflections of the HUD image to The goal of this study was to determine the relationship between
overlap and appear as a single image (Figure 1a). This these variables and HUD image legibility and perceived quality.
technique works well within a small range of viewer positions
and HUD virtual image distances (VIDs). However, if a driver’s
2. Experimental Setup
position is above or below the optimized viewer position, the
While it would be possible to create a physical system to vary
compensation becomes imperfect (Figure 1b). Additionally,
key system parameters—such as WS rake angle, HUD
wedge angle variation due to PVB tolerances and glazing
lookdown angle and VID, and glass thickness and tinting—it
processes can further degrade ghost image mitigation.
would be cumbersome and time consuming to move between all
the parameter combinations needed to map out the ghosting
Ghost disparity angle (DA) or ghost displacement is the current
perception in the 3-variable space outlined above. Therefore, an
metric for determining the degree of ghosting and for specifying
all-digital approach was used, in which simulated HUD images
allowable limits. For a wedge-system WS, with a ghost image
with ghosts were displayed on a 75”, 4K television positioned 3
luminance that can be 40%-60% of the primary image
meters from the viewer (Figure 2).

47
Software was created to simulate ghosting scenarios by images were created with RGL values between 40% and 60%,
superimposing a base HUD image, its ghost(s) with the specified simulating wedge WS scenarios, while double-ghost images
RGL and DA, and a uniform, grayscale ambient background, with RGL values between 2% and 20% were created to simulate
summing up the luminance values for each pixel to create the P-pol combiner film WS ghosting. These parameters were
composite image at the specified ACR. A calibration curve for designed to cover most typical HUD system geometries and
the television was obtained by measuring its luminance with a conditions.
Radiant Imaging Prometric imaging photometer for a series of
images with grayscale values between 0 and 255. This function Forty survey participants viewed over 100 images spanning this
was used by the software to output the grayscale values needed parameter range and rated the images on legibility (Table 1).
to yield the correct luminance for each image pixel. The
software also ensured that the image was never saturated (peak Table 1. Participants rated the images according to the
sum luminance for any pixel in the image never resulted in a following statement: “The image is: 1) Easy to read
required grayscale value higher than 255). and 2) Pleasing to look at.”

Strongly Strongly
Disagree Neutral Agree
Disagree Agree
Image Image OK
Image Image
badly but Image
noticeably still
degraded, somewhat perfect
degraded good
annoying impacted

The ratings were recorded as values from 1 to 5 (1 = perfect

image, 5 = badly degraded image). Zero-ghost images and
duplicate images were included to determine the degree of intra-
participant variability and an average zero-ghosting baseline
rating, and the image presentation sequence was randomized to
eliminate any order effects. The participants were non-experts
in HUD or display image quality and included people in the <30,
30-50 and >50 age groups. They were given no prior coaching
on how to rate the images; rather, the first five images were
Figure 2: Software creation of simulated HUD image selected to span the range from perfect to badly degraded to
with ghost(s) allow the viewer to quickly get a sense of the full range.

The font size for the survey was at the small end of what might 3. Results and Analysis
be seen in a real HUD, with a vertical angular extent of 0.194° The averaged no-ghost, baseline ratings were 1.65, 1.58 and 1.4
and line width of ~0.025°; this was chosen because smaller or for ACR values of 2, 4, and 10, indicating that some participants
thinner image features generally produce the most visible rated the images as less than perfect even when there was no
ghosting. Figure 3 shows two example images with different ghosting. (This is likely due to differences in visual acuity
ghosting parameters that approximates what would be seen in among participants, just as would be the case in the general
the study. Note that the left image has one ghost image while population.) The duplicate images, each seen twice in the
the right-hand image has two ghost images displaced on survey, had average rating differences of <0.2, which on a rating
opposite sides of the main image, corresponding to wedge WS scale from 1 to 5 indicates very good repeatability.
and P-pol combiner film WS ghosting geometries, respectively.
The rating for each image, defined according to its 3-parameter
values (DA, RGL, ACR), was averaged across the 40 study
participants, and the results for these averaged image ratings are
shown in Figure 4.

A two-way ANOVA on disparity angle and ghost luminance, in

both the wedge parameter space and P-pol combiner parameter
space, returns p-values < 0.002 for each variable and the
interaction term. This shows a statistically significant difference
Figure 3: Two example images with ghosts, shown in the mean participant responses between varying levels of
with the three parameter values. disparity angle and ghost luminance. Additionally, the
significance in the interaction term indicates participants’
For this study, the ghost disparity angle was varied from 0°–
0.1° and ACR values of 2, 4, and 10 were used. Single-ghost

 or PVB thicknesses, glass tinting and other factors, the two

For the sake of simplicity, the two ghost images always had
symmetric DA and RGL values. In reality, due to differing glass ghosts will have differing DA and RGL values.

48
Figure 4: Average image quality ratings for the 44 participants for ACR values of 2, 4, and 10, plotted vs ghost DA (x-
axis) and RGL (y-axis). The top row shows ratings for single-ghost images (as from a wedge-system mitigation) with
RGLs from 40% to 60%; the bottom row shows ratings for symmetric double-ghost images (as from P-pol combiner
mitigation) with RGLs from 2% to 20%.

responses are dependent on a combination of the disparity angle

and ghost luminance.**
4 also shows the effect of ACR, which has very little impact on
As can be seen in the upper row of single-ghost plots, simulating high RGL, wedge-type ghosting; high RGL ghosts are bright
ghosts created by a wedge-system mitigation with RGL values enough that their effect is similar regardless of ACR. However,
in the 40%-60% range, the ghosting impact on image quality is for low RGL, P-pol combiner-type ghosts, the ACR value does
quite insensitive to the RGL, indicating that once the relative affect ghost visibility; lower ACR conditions (ACR=2) produced
ghost luminance reaches a value of around 40%, increasing it a larger region of higher image quality ratings than did the
further causes little additional image quality degradation. These higher ACR conditions.
results support the current use of DA alone as a sufficient metric
for measuring ghosting in a wedge WS HUD system. These data To compare ghosting in wedge and P-pol combiner film WS
also show that tinting the glass, while it will lower the RGL to systems using these survey results, it was necessary to determine
some degree, does not significantly reduce the adverse impact of the DA and RGL parameter spaces that correspond to real-world
the ghost image on the HUD image quality. automotive HUD system geometries. This was achieved
through the development of mathematical tools to calculate the
In contrast, looking at the lower row of plots for the double- ghost DA and RGL for a system geometry defined by the WS
ghost, P-pol combiner system image ratings, for RGL values rake angle, glass thicknesses and absorptions, HUD lookdown
below about 20%, the image quality becomes increasingly angle, VID, vertical FOV and S/P polarization mixture, and
sensitive to the RGL value. For lower RGL values, ghosts with relative positions of driver, WS and HUD output mirror.
relatively higher disparity angles (which would seriously
degrade the image quality in wedge systems) are much less For a wedge system, the ideal wedge angle was calculated for a
visible and are rated by participants as good or acceptable. This 5mm thick WS; this angle perfectly compensates for the ghost
is an important finding, indicating that new metrics and image at the central eye position and central HUD image ray.
standards need to be developed for low brightness ghosts. Figure (The solid red line in Figure 5 shows the ideal reflection point

**
Taking a closer look at the p-values for the coefficients in the term has a p-value of 0. This indicates that it is very likely the
ordinary least squares [OLS] linear model for the wedge, the participants’ responses to the P-pol combiner parameter space
ghost luminance term has a p-value of 0.053 while the other are more impacted by the interaction of the disparity angle and
terms have a p-value of 0. This suggests that in the wedge ghost luminance rather than the effect of each variable
parameter space the ghost luminance does not have a significant independently.
impact on the response by itself. However, when looking at the
coefficients of the P-pol combiner OLS linear model, the
independent disparity angle and ghost luminance terms have p-
values of 0.296 and 0.061 respectively, while the interaction

49
for a 27° WS rake angle and 3° lookdown angle). Then the ghost
DA values for the top and bottom of a 4° vertical FOV HUD
image (solid green lines around central red dotted line in Figure
5) were calculated for driver eye positions up to 100mm above
and below the ideal position.

Figure 6: Survey image quality rating results with

highlighted regions of DA and RGL corresponding to
model system geometry for both a) wedge-system
ghosting b) P-pol combiner system ghosting

4. Conclusions
As shown by this initial study of human perception of HUD
Figure 5. Geometry of model system to calculate ghosting, for low-luminance ghosting typical of a P-pol
wedge and P-pol combiner film DA and RGL. (Rake combiner film WS, the disparity angle metric alone is not
and lookdown angles were varied; numbers on plot sufficient for characterizing ghosting and its impact on HUD
are shown for example.) image quality. In this regime, the ghost RGL also plays a
critical role and must be considered when defining acceptable
Variations in wedge PVB angle and/or resulting WS wedge levels of ghosting. Further, the ghosting parameters calculated
angle due to PVB angle tolerances or glazing process were not for a P-pol combiner film WS across a range of physical HUD
included in this analysis but could potentially increase the worst- geometries were shown to yield favorable image quality ratings
case DA values by an additional 0.01-0.02°. In this model and to be less sensitive to variations in DA than a wedge system.
system, for a VID of 2.5 meters, ghost DA values anywhere
from 0° to 0.05° were possible when considering all driver Future studies will build upon these initial results, which were
heights. This region on the survey ratings plot for ACR=2 is based on simulated HUD images on a static grayscale
shown in Figure 6a, and as can be seen, portions of this region background. Potential refinements include specifying unequal
received participant ratings of 3 and any increase in DA (due to, DA and RGL values for the front and back ghosts for P-pol
for example, wedge angle tolerance variations) would create combiner ghosting, using image or video ambient backgrounds
ghosting that would be rated as noticeably degrading the image. rather than uniform grayscale, probing the effects of transitory
viewing of the image (as occurs in actual HUD viewing), and
The mathematical model above was also used to calculate DA the effect of image color and image/background color
and RGL values for a P-pol combiner film WS in the same differences. Further, the metric of driver fatigue to evaluate the
system geometry. A sample set of real-world geometries, impact of ghosting may also be considered alongside image
designed to probe the full range of resulting DA and RGL quality ratings.
values, were used to calculate the points shown on the plot in
Figure 6b. It is notable that the region on the plot corresponding 5. References
to calculated ghosting produced by a P-pol combiner film WS in
the model system received average participant ratings under 3, [1.] Normile, Brian. Which cars have head-up displays for 2020?
indicating good or acceptable image quality. In other words, for [Internet]. 2020 May 14. Available from:
most real-world scenarios, P-pol combiner film-type ghosting is https://www.cars.com/articles/which-cars-have-head-up-
not noticeable or is deemed acceptable by most viewers. displays-for-2020-421615/.
Further, this type of low RGL ghosting is less sensitive to [2.] Jarrett DN. Cockpit engineering, Farnham, United
increases in ghost DA. As seen in Figures 6a and 6b, for high Kingdom: Ashgate Pub.; 2005. p. 189. ISBN 0-7546-1751-3.
RGL wedge-system ghosting, the ratings increase quickly with ISBN 9780754617518. [Retrieved 2012 Jul 14].
DA, while for the weak ghosts produced by a P-pol combiner
system the image degrades much more slowly with DA. This
means that any WS variations from glass imperfections or [3.] Harrison AK. Head-up displays for automotive applications.
glazing process that increase the DA, either locally or globally, Ann Arbor: The University of Michigan Transportation
may more significantly increase detrimental ghosting effects for Research Institute; 1994. Report No. UMTRI-94-10.
a wedge WS than for a P-pol combiner film WS.***
[4.] Gitlin, Jonathan M. Augmented reality heads-up displays for
cars are finally a real thing [Internet]. Ars Technica. 2020 July

***
In addition, even if the P-pol combiner film WS has some film and the S1 and S4 surfaces is only approximately half the
variation in slope due to the glazing process, for a given slope thickness of the WS.
change the ghost DA variation will only be half as large as for a
wedge WS, because the distance between the P-pol combiner

50
10. Available from
https://arstechnica.com/cars/2020/07/augmented-reality-heads- [7.] VanDerlofske J, Pankratz S, Johnson M, Franey E. New
up-displays-for-cars-are-finally-a-real-thing/. optical films for next generation AR head up displays (HUDs).
2019 SID Display Week Symposium, Digest of Technical
[5.] Seder, T, Cashen, D, Ferris, L, Robb, E. Minimizing HUD Papers. 2019.
ghost images from glare trap lens. 2016 SID Vehicle Displays &
Interfaces Symposium. 2016 Sep. [8.] Zong Qin, Fang-Cheng Lin, Yi-Pai Huang, Han-Ping D.
Shieh. Maximal acceptable ghost images for designing a legible
[6.] Marcus, M. Simultaneous head-up display windshield windshield-type vehicle head-up display. IEEE Photonics
wedge-angle and layer-thickness measurements. SPIE News, Journal 9. Dec 2017 (6).
2016 Jul.

51
52
 Presentation 3.3

Computational Holographic Displays for 3D AR HUD

Using Free-Form Optics
Hakan Urey, CY Vision, San Jose, CA, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

53
54
 Holographic Optical Elements and Projector Design Considerations
for Automotive Windshield Displays
Sam Martin Jason Thompson Ian Redmond
DLP Products DLP Products Ceres Holographics Ltd
Texas Instruments Texas Instruments [email protected]
Incorporated Incorporated
[email protected] [email protected]

Abstract: Holographic Optical Elements (HOEs) can be used in a range of automotive applications including head-
up displays, windshield clusters, and transparent window displays. When paired with TI DLP® Automotive
projection technology, HOEs offer a number of advantages including smaller package sizes, large bright displays,
and system design flexibility through unique manipulation of light. HOEs can now be produced in a pixel-by-pixel
fashion by a digital technique giving great versatility of function. Recorded in novel photopolymer materials, these
elements have high efficiency when used with LED/laser sources and simultaneously high transparency to ambient
light. This paper will provide a market overview of HOE automotive applications, an update on the key aspects of
this technology and performance achieved to date, and cover projector design considerations when illuminating
holographic films. Projectors designed for HOE applications have additional requirements on spectral emission to
guarantee sufficient image brightness to the viewer. A detailed analysis of spectral efficiency and color point tuning
of a DLP light engine will be provided as an example.
Keywords: transparent holographic display, automotive, cluster, DLP, LED, laser, HOE
Introduction 3. Greater than 10 000 cd/m2
4. Less than 1 liter of mechanical volume
With the increase of sensors and advanced driver
5. LED illumination source
assistance systems (ADAS) capabilities, both passenger
6. No windshield wedge required
vehicles and commercial vehicles need to display this
information to the driver in a simple and effective way. Additionally, this system can demonstrate an image
Many vehicles have adapted traditional head-up display effectively on a window with a steep rake angle and is
(HUD) to show this information to the driver while viewable while using polarized sunglasses, which is a
allowing the driver to keep their eyes on the road. challenge for HUDs based on TFT-LCD technology.
However, the large mechanical volume and windshield
Ceres HOE technology
reflection angle requirements of traditional HUDs can
cause fundamental challenges with vehicle integration. Ceres develops HOE technologies primarily for
For example, some car models have very small automotive applications, based on “volume” holograms
instrument panels making it impossible to integrate a (as opposed to thin, or surface relief elements).
HUD. Additionally, the steep windshield rake angle of “Volume” refers to the fact that the hologram is a 3-
many commercial and off-highway vehicles makes it dimensional structure of refractive index variation,
impractical to use traditional HUDs, since the angle of formed by an optically recorded exposure of an
the HUD optical axis must be between 50 and interference pattern in a photo-polymerizable material.
60 degrees relative to the windshield for the driver to As a volume structure, it has highly selective Bragg
view the reflected image. diffraction properties in angle and wavelength, essential
to windshield applications.
New technologies based on volume holographic optical
elements combined with automotive grade projection Volume holograms have key properties essential for
display systems can solve these integration challenges windshield displays:
and enable a transparent display (TD) on the
windshield. Ceres Holographics and Texas Instruments  Due to the high angle and wavelength
collaborated to design, build, and validate a prototype selectivity, they are simultaneously highly
that demonstrates a large, bright transparent window efficient at diffracting light from a projector
display exceeding the traditional specs of a HUD, while with the appropriate spectrum and incident
minimizing total mechanical volume in the instrument angle, and yet appear very transparent as they
panel. remove very little of the broadband ambient
spectrum in transmission.
1. Display size is 220 mm x 110 mm
2. Image distance ~1 m (i.e. windshield)

55
  They are in the form of a thin flexible film of Reference beam
photopolymer (15-25 m) on a polymer (plane)
substrate (e.g. 60 m) which can be laminated
within the (curved) windshield sandwich.
 Optical quality can be excellent. Specifically,
Ceres uses Covestro’s Bayfol™ material which
exhibits negligible haze, and negligible residual
absorption (in the 1% range), suitable for HOE pixel
exposure in
windshield applications.
photopolymer
 The optical function is independent of the final
film (or windshield) shape. The film may have SLM Fourier Transform Lens
the function of a curved mirror or lens which Figure 1: Pixel-wise HOE printing
would physically have a much larger depth (i.e.
sag). Or it may have the function of a tilted Each subhologram is formed by the exposure of the
optic, independent of the tilt of the HOE film. It interference pattern between the simple plane reference
may have these functions independent of the beam and the complex object beam, formed by the
final film shape (e.g. in a windshield). Fourier transform of the field on a spatial light
 An engineered diffuser function can be realized modulator (SLM), i.e. a microdisplay. Both the
for the transparent display function, so light can reference beam angle and the SLM RGB greyscale
be efficiently redirected to a uniformly pattern are programmed for each subhologram exposure
illuminated specified eyebox rectangle (or by a master control dataset.
indeed any other distribution). The subholograms are typically 0.25x0.25 mm square,
The Bayfol™ material is also suitable for mass- and each is recorded with a simultaneous exposure of
production, requiring only simple optical exposure red, green and blue laser light designed to match the
processing, exhibiting very low polymerization final illumination source. For example, exposure
shrinkage and very good long-term stability. wavelengths of 633, 532 and 473 nm naturally results in
peak diffraction efficiencies (with 6-8 nm FWHM) at
For typical automotive HOE applications, HOE 627/527/468 nm at the same geometry, after allowing
bandwidths are in the 4-8 nm FWHM range (15-25 m for an intrinsic typical ~1% polymerization shrinkage.
films) at the design angle, or 2-4 degrees at the design However, due to the digital nature of the printer control,
monochromatic wavelength. compensations can easily be made in the control data to
Pixel-wise HOE recording adjust for different target wavelengths, such as specific
peak LED wavelengths.
In traditional hologram recording, the hologram is
recorded in a single exposure in the interference field Note that, in the case of a diffuser/transparent display,
between two mutually coherent beams. This requires the display resolution is not limited by the subhologram
the wavefronts of these beams to be created by large size (0.25x0.25 mm). This size only defines the
format typically custom optics on the scale of the final quantization of the angular properties of the film, so if a
hologram. finer resolution were imaged onto this HOE, it would
still effectively resolve it from the viewer’s perspective.
Ceres however produces large HOEs by a step and
repeat optical exposure of small subholograms, each Note also that this printer produces “master” holograms
defined digitally, allowing great flexibility of design which may take many hours to produce (typically
without change of hardware, and resulting in high 10 subholograms/second), but in production these
repeatability and uniformity. holograms are optically mass-copied at a much higher
rates on the order of 2000 to 5000x faster, consistent
For this, a master hologram “printer” has been with commercially viable windshield production rates.
developed, and optimized for the specific photopolymer
materials. This printer is digitally controlled, and the Transparent display HOEs
recording beams are settable for each subhologram. TD HOEs are essentially precision engineered
Specifically, the RGB exposure values, reference beam diffractive diffusers. The function is illustrated in
angle in 2-dimensions and the entire object beam field Figure 2. The projector is shown emitting two rays; red
may be set exposure by exposure. A diagram of the towards the center of the HOE and black towards a
HOE printer is shown in Figure 1. corner. The HOE subhologram function for the red ray
is programmed to expect a ray at that incidence angle

56
and wavelength, and to diffract it (by a diffuser Projector design
function) into the uniformly illuminated rectangle of the
HOE technology is not constrained by the symmetrical
eyebox at a specific location.
specular reflection laws of traditional glass surfaces.
Similarly, the subhologram where the black ray hits the For example, it is possible to create a system that emits
HOE is designed to be optimum for this incidence angle light normal to the glass surface when illuminated from
and to diffract the light to the eyebox rectangle. Both a steep angle, or vice-versa. The following diagram
conditions are different from those of the red shows two example layouts that can be realized using a
subhologram and are digitally programmed in the DLP projector and Ceres HOE technology:
(optical) recording process.
30° possible
cone

90°
cone
Figure 2: Transparent display HOE

Note that the image originates from the HOE (diffuser)

plane, and the projector must project a focused image Figure 3: Configuration A (left) and B (right) showing
onto this plane, like a traditional projection screen. two different design architectures for transparent
displays based on Ceres HOE and DLP projector
Illumination
technology.
Since the TD HOE operates as a diffuser, the apparent
Note that the accessible angles for the HOE has
image location is the plane of the HOE. Therefore,
implications for projector distance and achievable
dispersion due to diffraction does not affect image
eyebox size. Configuration A may be more applicable
resolution. All light diffracted by the HOE appears to
to an off-highway vehicle with a near-vertical
come from the HOE plane. If the illumination
windshield, while configuration B is well suited for
wavelength does not match the HOE, it will simply
standard consumer vehicles. Bear in mind that in this
diffract less efficiently resulting in a shift in the color
latter case the windshield will be highly raked and the
point of the diffracted light.
projector will be in the dashboard. In a collaborative
The best result occurs when the HOE is designed to R&D program, Ceres Holographics and Texas
expect illumination at the actual projector wavelength. Instruments designed and built a transparent display
In terms of optical efficiency, the best result is when the prototype using configuration B architecture:
projector wavelength matches the Bragg diffraction
efficiency peak of the HOE. From this point of view, a
laser illuminated projector would offer the best system
efficiency (lumens/watt). And importantly, the resulting
thermal dissipation would be much reduced, a crucial
system consideration.
However, consideration should also be given to the
associated spectral tolerances. Laser diodes (LDs) have
a natural manufacturing peak wavelength spread which
is wider than the HOE bandwidth. Laser diode specified
peak wavelength spread can easily be ±10 nm, while
the HOE has a FWHM of <10 nm. Some lasers will
poorly match the HOE, leading to low efficiency,
brightness and a poor color point. LEDs on the other
hand have relatively wide bandwidths compared to the Figure 4: Holographic Windshield Cluster
HOE, and so are inherently more tolerant of variation in demonstrator built by Ceres Holographics and Texas
their peak wavelength. Choice of LDs versus LEDs Instruments
thus remains a topic of investigation.

57
Luminance contrast ratio The final equation is formed by substituting the
approximation for solid angle defined above.
To guarantee image visibility, the optical designer must
ensure the projector emits enough light. As with 𝐿𝐴𝑆
traditional HUDs, this light is spread out over area and Φ ∗ cos 𝜃
𝑟
solid angle in the eyebox, meaning the size of the image
For example, the TI Holographic Windshield Cluster
and eyebox influence optical flux requirements from
demonstrator has an image size of 220x110 mm
the projector. Luminance contrast ratio (LCR) describes
(0.0242 m2), an eyebox size of 140x80 mm
how visible an image is relative to the background.
(0.0112 m2), eyebox distance of 1.2 m, and diffraction
Light from transparent displays is additive to
angle of 48 degrees. To achieve the luminance target of
background light, so LCR is given by the following
10 000 cd/m2, the required diffracted luminous flux is:
equation:
𝐿𝐴𝑆
𝐿 𝐿 Φ ∗ cos 𝜃
𝐿𝐶𝑅 𝑟
𝐿 10 000 0.0242 0.0112
Φ ∗ cos 48
Exact LCR requirements are a heavily debated topic. A 1.2
1995 study of human factors aspects of automotive Φ 1.3 lm
HUD technology recommends a minimum of 1.5:1 in In other words, the luminous flux diffracted by the
bright ambient conditions [1] but it is often desirable to HOE must be 1.3 lm in order to achieve an image
have higher LCR to increase legibility. In this example luminance of 10 000 cd/m2. This is valid for the system
we will target LCR = 2 to guarantee the information is parameters defined above, but will be different for
visible to the driver. Forward-looking scene luminance different image size, eyebox size, eyebox distance, and
can exceed 10 000 cd/m2 in bright sunlight [1], so HOE diffraction angle.
transparent displays must be able to produce at least
10 000 cd/m2 image luminance to ensure LCR > 2. The The efficiency of the HOE will determine how much
following sections describe a calculation procedure to luminous flux the projector must emit in order to
estimate projector requirements for a set of system achieve this diffracted flux target.
parameters. This method is for initial estimation only HOE efficiency
and should be followed by a comprehensive optical
design to validate performance. HOEs are spectrally selective, so it is important to
consider optical efficiency as a function of wavelength.
Estimating image brightness 3-wavelength multiplexed RGB HOEs made by Ceres
Working backwards from the luminance target of Holographics have a spectral efficiency curve that can
10 000 cd/m2 in the eyebox, we can estimate the be approximated by 3 gaussians with peak wavelengths
efficiency of each optical component in order to around 460, 520, and 620 nm, and FWHM of 6, 7, and
determine required optical flux from the projector. The 8 nm respectively.
variables that determine image luminance are image
area, eyebox solid angle, HOE efficiency, and projector
luminous flux. The eyebox is small relative to its
distance from the HOE, so its solid angle can be
approximated based on its area:
𝑆
Ω ∗ cos 𝜃
𝑟
where Ω is solid angle, 𝑆 is eyebox area, 𝑟 is eyebox
distance, and 𝜃 is the angle with respect to normal at
which the light is diffracted from the HOE.
Figure 5: HOE spectral efficiency
Using optical throughput concepts, we can convert the
required luminance (10 000 cd/m2) to diffracted Exact peak wavelengths of each color channel can be
luminous flux given the following equation: tuned in the fabrication process within a certain range.
Φ 𝐿𝐴Ω Multiplying the projector spectral emission curves by
these spectral efficiency curves will determine optical
where Φ is diffracted luminous flux, 𝐿 is luminance,
efficiency of the HOE for a given projector. As such, it
𝐴 is image area, and Ω is solid angle.
is important for the projector spectral emission to align
with the HOE spectral efficiency in order to guarantee

58
highest system efficiency. projector spectral emission is Figure 7 shows the measured spectral emission of this
determined by the light sources used, so projectors projector at various LED power levels compared with
designed for HOE applications may have different the HOE spectral efficiency bands.
requirements than those designed for other applications.
Projector light source considerations
LED light sources offer robust, efficient performance
and are common in both consumer and automotive DLP
projector systems. It is common to find projection
systems on the market today that contain a phosphor
converted green LED, which uses a more efficient blue
LED under a green phosphor to produce green light, as
opposed to a true green LED which emits green light
directly. While these converted green LEDs are more
efficient (higher lumens/watt), the wide spectral Figure 7: Projector spectral emission for various LED
emission of the phosphor results in low system forward currents. LED peak wavelength and spectral
efficiency when coupled with the HOE narrow spectral shape may change with drive current, temperature, and
efficiency bandwidth. For example, Osram produces lifetime of the device.
both converted and true green LEDs in their Q8 product
line. The converted green LED, LE CG Q8WP, has a The product of the HOE spectral efficiency and the
max bin luminous flux of 800 lm and spectral emission LED spectral emission determines the optical
FWHM of 100 nm [2]. The true green LED, LE T efficiency. Here the LEDs are fairly well-matched to
Q8WP, has a max bin luminous flux of 450 lm and the HOE and result in a total optical efficiency of 2.6%.
spectral emission FWHM of 33 nm [3]. However, even
though the converted green LED produces nearly 2x the
Osram Q8 LED HOE efficiency
total luminous flux, the true green LED results in
roughly 40% more energy throughput when multiplied Blue 2.5%
by the HOE spectral efficiency curve. True Green 2.0%
Red 3.5%
Total 2.6%
Table 1: HOE efficiency for Osram Q8 LEDs used in
the DLP5530PROJTGQ1EVM.

Therefore, to achieve the target eyebox luminous flux

of 1.3 lm, we would need 50 lm emitted by the
projector. However, light incident on the HOE must be
polarized to avoid a double-image artifact, resulting in
roughly 50% loss from unpolarized LEDs. Therefore,
Figure 6: Spectral emission of Osram LE T Q8WP true the total luminous flux required from the projector in
green with 450 lm total flux, LE CG Q8WP converted this example is 100 lm.
green with 800 lm total flux, and HOE efficiency curve.
Even though the converted green LED produces nearly Eyebox luminance 10 000 cd/m2
2x luminous flux, the energy within the HOE efficiency Eyebox luminous flux 1.3 lm
curve is roughly 40% greater for the true green LED.
Luminous flux incident on HOE 50 lm

For this reason, the TI Holographic Windshield Cluster Luminous flux emitted by projector 100 lm
(unpolarized)
demonstrator uses the DLP5530PROJTGQ1EVM [4] as
its projector, which has been designed with the Osram Table 2: Summary of key optical values in system
LE T Q8WP true green LED to maximize efficiency efficiency breakdown used to estimate projector
with the Ceres HOE. Blue and red LEDs like the Osram requirements
LE B Q8WP and LE A Q8WP are direct emitting
diodes (no phosphor conversion) and produce a narrow Light source spectral shift
bandwidth that is well suited for use with HOEs. It can be seen from the measurements in Figure 7 that
the peak wavelengths of the green and red LEDs are
shifting with respect to output flux. LED peak

59
wavelength variation can result in increased or For this reason, it is important to incorporate the
decreased efficiency for that color channel if the LED spectral efficiency of the HOE in the calibration process
spectral curve moves closer to or away from its and calibrate the projector for an offset color point to
corresponding HOE spectral efficiency curve. LED account for the color shift of the HOE. This will ensure
spectrum variation can occur from natural operational light in the eyebox is properly white point calibrated.
variation (drive current, junction temperature, etc.) or
By optimizing the white point of the entire system,
minor variations in the manufacturing process resulting
including the HOE, we can determine the relative flux
in different LED bins. It is important to understand
from the projector to achieve the target white point in
potential shifts of peak wavelength of each LED across
the eyebox. Figure 9 shows projector spectral emission
the operating temperature range and lifetime of the
calibrated for an offset color point to compensate for
device. Changing efficiency of one color channel
the chromatic shift of the HOE, resulting in the target
relative to the others will result in a color shift of the
D65 white point in the eyebox.
diffracted light and must be accounted for in the white
point calibration process.
White point calibration
Traditional LED projection systems are designed to
project red, green, and blue light onto a scattering
surface, such as a projection screen. It is necessary to
tune the relative flux of each primary light source such
that their combined output appears white at the
maximum digital pixel value for all primaries. This
process is known as white point calibration. In the case
of a white (spectrally neutral) projection screen, the
light from each primary color is reflected equally. Figure 9: Pre-correcting projector color point
Therefore, calibrating the light emitted by the projector calibration results in well-calibrated D65 white light in
ensures that light reflected by the screen maintains the the eyebox.
same white point calibration. However, in the case of
display systems using HOE technology this is not Laser light sources
necessarily true. The HOE is not spectrally neutral, and
the relative transmission of each primary color is a While LEDs are a proven and recommended solution
function of the overlap between source spectral today, laser sources offer promising system-level
emission and HOE spectral efficiency. Applying a benefits. At the projector level, laser sources can enable
white point calibration to the projector will not higher F/#, improved contrast, smaller package volume,
necessarily guarantee that light diffracted by the HOE and lower cost optics [5]. However, lasers paired with a
toward the eyebox will maintain that white point. For HOE can offer significantly higher system efficiency
example, calibrating the DLP5530PROJTGQ1EVM for than achievable with LEDs. Additionally, lasers emit
a D65 white point results in a red shift when that light is polarized light which naturally eliminates the double
diffracted by the HOE, as shown in Figure 8. image reflection and mitigates the need for a polarizer,
which is necessary for LEDs.
It can be seen in Figure 7 that the LED spectral
emission is wider than the HOE efficiency bandwidth,
meaning some of the energy for each LED is lost,
resulting in reduced system efficiency. The spectral
bandwidth of laser sources is much narrower than that
of LEDs and easily fits within the HOE spectral
efficiency bands, resulting in a higher percentage of the
light from each color being diffracted towards the
viewer.
Figure 8: HOE technology can induce a color shift in
light from the projector making it necessary to calibrate
the system to an offset color point. The spectral flux
plot (left) shows normalized spectra before (projector)
and after (eyebox) HOE diffraction.

60
 References
[1] K. Gish and L. Staplin, "Human Factors Aspects
of Using Head Up Displays in Automobiles:
A Review of the Literature," Office of Crash
Avoidance Research, National Highway Traffic
Safety Administration, Washington, 1995.

[2] OSRAM Opto Semiconductors, "OSRAM

OSTAR® Projection Compact, LE CG Q8WP,
" 04 06 2020. [Online]. Available:
Figure 10: Spectral emission of ideal laser sources. The https://www.osram.com/ecat/OSRAM%20OSTAR
narrow laser bandwidth easily fits within the HOE %C2%AE%20Projection%20Compact%20LE%20
spectral efficiency bands. CG%20Q8WP/com/en/class_pim_web_catalog_10
3489/global/prd_pim_device_2191201/.
HOE efficiency Osram Q8 LEDs Ideal laser
[3] OSRAM Opto Semiconductors, "OSRAM
Blue 2.5% 12% OSTAR® Projection Compact, LE T Q8WP,
Green 2.0% 12% " 04 06 2020. [Online]. Available:
Red 3.5% 12% https://www.osram.com/ecat/OSRAM%20OSTAR
Total 2.6% 12% %C2%AE%20Projection%20Compact%20LE%20
Table 3: Comparison of HOE efficiency for LED and T%20Q8WP/com/en/class_pim_web_catalog_1034
laser sources. Differences in efficiency are due to 89/global/prd_pim_device_2191202/.
different overlap between LED spectral emission and
HOE spectral efficiency band. [4] Texas Instruments Incorporated, "DLP5530-Q1
Interior projector evaluation module with true
The efficiency benefits are obvious, but there is green LED," August 2020. [Online]. Available:
potential for speckle artifacts to be more visible when https://www.ti.com/tool/DLP5530PROJTGQ1EV
using laser illumination. Additionally, there may be M.
added complexity in the system to manage thermal [5] M. Firth, A. Norris, D. Segler and J. Thompson,
limits for lasers in the automotive environment which is "Next Generation Augmented Reality Head-Up
from -40 C to 85 C ambient. This is an ongoing Display with DLP Technology," in SID Vehicle
research topic for both Ceres Holographics and Texas Displays and Interfaces Symposium Digest of
Instruments. Technical Papers and Presentation Files, Livonia,
Conclusion 2017.
In summary, through the combination of a Ceres [6] K. A. Smet, "Tutorial: The LuxPy Python Toolbox
Holographics HOE and a DLP5530 based projector, a for Lighting and Color Science," LEUKOS, vol. 16,
large, bright window display can be implemented in a no. 3, pp. 179-201, 2020.
small package size that is compatible with virtually any
size passenger vehicle and can also effectively work
with commercial or off-highway vehicle with low rake
angles. This solution has the opportunity to enable
many vehicles to implement transparent window
displays to effectively communicate important driver
assistance information.
In future studies, the use of RGB lasers, as well as
techniques to increase HOE efficiencies may increase
efficiency of this system, dramatically lowering the
overall power level or offering increased image and
eyebox sizes. However, more evaluation is needed to
effectively address the temperature management of the
lasers and mitigate the speckle that is inherent to direct
lasers.

61
62
 Tutorial

Holography and Its Automotive Applications

Kai-Han Chang, General Motors Global R&D, Pontiac, MI, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

63
64
 Keynote Address

KEYNOTE ADDRESS
Voice of the Consumer | Technology and Mobility Clarity
Today and Tomorrow
Kristin Kolodge, Executive Director of Human-Machine Interface
(HMI) and Driver Interaction at J.D. Power

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

65
66
 Understanding and Achieving Reproducible Sparkle Measurements for an
Automotive Specification
Ingo Rotscholl*, Jens Rasmussen**, Christoph Rickers***, Julia Brinkmann*, Udo Krüger*
*TechnoTeam Bildverarbeitung GmbH, Werner-von-Siemens Str.5, 98693, Ilmenau, Germany
**Elektrobit Automotive GmbH, Lise-Meitner-Straße 10, 89081, Ulm, Germany
***Volkswagen AG, 38436, Wolfsburg, Germany
[email protected]

Abstract: The reproducible quantification of display Another important practical requirement is that each method
sparkle with Imaging Luminance Measurement Devices is works without removing the AGL such that the same method
essential for testing and conformity assessment of high to quantify sparkle can be used during each development
quality automotive displays. This study systematically step of the display. Furthermore, it is advantageous if
researches relevant setup and system influences of a Fourier existing measurement procedures, setups and equipment as
Space based sparkle evaluation by analyzing more than known from the BlackMURA [10,11] specification can be
10000 luminance images. These include 13 different Anti- used. The main aim of this study is to systematically research
Glare-Layers and two displays with different PPI in more all relevant setup influences and system specifications
than 70 system setups. The concluded procedure serves as during an ILMD (Imaging Luminance Measurement
basis to define a new automotive specification for Device) based sparkle evaluation in order to define a sparkle
reproducible sparkle measurements and may do so for other measurement procedure, which allows not only reproducible
applications that need to quantify sparkle in a reproducible sparkle measurements but ideally uses already existing
way. BlackMURA compliant procedures and equipment.
In the next section the fundamentals of sparkle
Keywords: sparkle, photometry, display measurement, measurements are described. The main focus is the pixel
display metrology, display characterization, uniformity, separation method Fourier filter approach. After that we will
anti-glare layer; AGL; imaging luminance measurement focus on aspects which enhance the reproducibility in the
device; ILMD; 2D-LMD; suggested approach and which will also be covered in the
upcoming specification.
Introduction
Alongside the quantification of the low frequency Fundamentals of Sparkle Measurements
uniformity as for instance BlackMURA [1], the Figure 1 visualizes the main problem of a sparkle measure-
quantification of high frequency uniformity as Anti-Glare- ment, which is the separation of the high frequency sparkle
Layer (AGL) caused display sparkle, is an important quality phenomenon from the periodic luminance modulations of
criterion for modern automotive displays. Thus, the the pixel matrix. The AGL in Figure 1 is only applied to the
quantification of sparkle has been the focus of several center of the luminance image. The sparkle can be seen but
studies [1-9]. However, in many cases these studies focus there is a large contribution from the periodic pixel pattern.
only on the relative comparison of several glass samples, its Becker summarized four different concepts to perform this
correlation to human perception and repeatability. The separation [1,2]: Defocussing [4,5], Undersampling [6,7],
reproducibility of absolute sparkle levels within different Spatial filtering [1,2] and Frequency Filtering [8].
settings, remains a large challenge [2].
While the first two approaches try to avoid sparkle by
adjusting the capturing condition, the last two methods rely
on image post processing. As both the Undersampling
approach as well as the Spatial filtering require unsuitable or
at least untypical sampling ratios (camera pixels per display
pixels) also called reproduction scale (RPS) for
BlackMURA, we focus our analysis on the remaining two
approaches.
The idea of the defocusing method is to first focus on the
periodic pattern and then defocus the periodic pixel pattern
while trying to keep the sparkle as sharp as possible.
However, this requires a very constraint setup to ensure that
Figure 1: Luminance image with AGL in the center the focusing conditions are identical. Nevertheless
region reproducibility issues arise because the initial focus position
reproducibility is limited.

67
 Table 1: Measurement Setups (*different camera)

Camera Lens RPS AGL

1,7 2,1;
no glass, L1, L2, M2, M6,
25mm 2,45; 3,0;
H1, H3
3,6
1MP
1,7 2,1;
no glass, L1, L2, M2, M6,
50mm 2,45; 3,0;
H1, H3
3,6
Figure 2: Frequency filtered center region of Fig. 1 1,7 2,1;
no glass, L1, L2, M6, H1,
In contrast to the defocusing, the filter in the frequency 25mm 2,45; 3,0;
H3
3,6
domain requires a sharp image of the pixel layer and the 5MP
sparkle as in Figure 1. After that a Fourier transformation is 1,7 2,1;
no glass, L1, L2, M2, M6,
applied. Based on an analysis of the frequency amplitudes, 50mm 2,45; 3,0;
H1, H3
3,6
the frequencies, which originate from the periodic pixel
pattern are detected and suppressed by a tailored band 1,7 2,1;
no glass, L1, L2, M2, M6,
rejection filter. Finally the filtered image is back- 16mm 2,45; 3,0;
H1, H3
3,6
transformed to the spatial domain. Figure 2 shows the back-
transformed center region of Figure 1 after applying the 1,7 2,1;
no glass, L1, L2, M2, M6,
frequency filter. As this approach is very flexible regarding 12MP 25mm 2,45; 3,0;
H1, H3
3,6
the setup conditions, we concentrated on this method.
1,7 2,1; no glass, L1, L2, M1, M2,
Validation: In order to validate the Fourier based approach, 50mm 2,45; 3,0; M3, M4, M5, M6, H1, H2,
we used an automotive display with 224 PPI and measured 3,6 H3, H4
the sparkle value of 12 different AGLs. Besides the AGL 1,7 2,1;
nothing was changed between the measurements no glass, L1, L2, M1, M3,
5MP* 25mm 2,45; 3,0;
M4, M5, H3, H4
(orientation, distance, focus, lens and camera). The results 3,6
were compared to an expert rating, which was not known
Initial Setup Experiment: In order to test the
before the validation experiment. The expert rating covered
reproducibility, of different setups, we performed the same
three different ratings (L-low, M-medium, H-High). The
validation experiment again using several different
numbers show the relative ranking within the groups. No
BlackMURA compliant camera/lens combinations and
glass, L1 and L2 achieved the same visual ranking. Figure 3
sampling ratios, which are shown in Table 1. Figure 2
shows the achieved results. The good correlation validates
(bottom) shows the resulting sparkle values for a RPS of
this pixel separation method.
2.45. Besides roughly comparable sparkle values, no
16%
specific tendency can be seen, which means that the
12% reproducibility is poor. This can also be seen by the high
Sparkle

8% coefficients of variation as well as the standard deviations of

4% the sparkle between the different setups.
0%
Reproducibility enhancement
no glass
L1
L2
M1
M2
M3
M4
M5
M6
H1
H2
H3
H4

In order to optimize the reproducibility, the LMK Position

system (Figure 4) was used. The system allows high speed
16% 16mm 12 MP 25mm 1MP and high reproducibility alignment and the systematic
12% 25mm 5MP 25mm 12MP change of solely selected setup parameters such as the
50mm 1MP 50mm 5MP
Sparkle

50mm 12MP
geometrical alignment or the focus position for sensitivity
8% studies.
4%
0%
no glass L1 L2 M2 M6 H1 H3

CV in % 25.3 8.7 5.8 8.4 8.7 9.7 8.4

Std in % 0.49 0.18 0.12 0.27 0.45 0.84 0.89

Figure 3: Measured sparkle ranking over visual

ranking: Validation of Fourier filter method at RPS = 3.6 Figure 4: Setup and alignment system used to
(Top) Initial setup experiment at RPS = 2.45 (Bottom) perform the measurements / sensitivity studies

68
 there is still a slight lateral shift within a range of a few
millimeter. It originates from AGL properties such as
structure, thickness and the interaction with the display. This
effect is additionally superimposed to the focus
reproducibility.
In order to overcome the fundamental focus reproducibility
limitation we performed a sparkle focus scan for the
complete measurement series of Table 1. However, instead
Figure 5: Sparkle/Focus sensitivity: Left: Sparkle of of the sparkle value at the reference position, we selected the
one AGL in different setups Right: Sparkle of maximum value as the representative sparkle value. Figure
different AGLs in one setup 5 shows the resulting values as well as the reproducibility. It
can be seen that especially the reproducibility of medium
16% 16mm 12MP 25mm 5MP* and high sparkle values improves. This is caused by the
12% 25mm 1MP 25mm 5MP effect that an undetermined blur affects strong sparkle much
Sparkle

25mm 12MP 50mm 1MP more than low sparkle values. We assume that earlier studies
8% 50mm 5MP 50mm 12MP
may have been affected by this effect [9].
4%
Local Median Evaluation: Other impact factors besides the
0% focus position are for instance tiny imperfections such as
no glass L1 L2 M2 M6 H1 H3 scratches, defect pixels, dust and fingerprints or artifacts
from the Fourier filter. These tiny imperfections may change
CV in % 18.9 10.1 8.6 3.3 4.4 5.3 4.3 the sparkle value significantly because they may produce
high outliers. An example is provided in Figure 7. The image
Std in % 0.38 0.23 0.19 0.11 0.24 0.50 0.49
at the top shows a Fourier filtered image with tiny
Figure 6: Reproducibility setup experiment with
imperfections. This image has been divided into small local
optimized focus at RPS = 2.45
areas. A local sparkle contrast was calculated in each area.
Optimized Focus: To research the focus sensitivity of the This is shown in the image at the bottom. We call this result
measured sparkle, an initial setup according to Figure 4 was local sparkle matrix. There are a few high outliers in the
used. The focus was adjusted to the pixel layer of the display. sparkle values. In order to select a statistically robust
Then a lateral camera movement was performed to slightly representative value from the local sparkle matrix that is not
change the relative distance between the camera and the affected by outliers, we select the median of all local values.
display, which also changed the relative position of the
Figure 8 shows the results for the measurement series of
focus. Each time a sparkle value was obtained.
Table 1 using not only an optimized focus position but also
Figure 5 (Left) shows the experiment for the different setups a local median based evaluation. Especially the low
of Table 1 (AGL M6, RPS = 3.6).Three things can be sparkling values improve by applying the local median. This
observed: can be explained by the effect that tiny imperfections such
 There is a high focus sensitivity as dust affect low sparkling values more than high sparkling
 There is a “random” lateral shift values. Note that the absolute values always become smaller.
 There is a similar maximum value.
The high sensitivity is caused by blurred sparkle and was
expected. The lateral shift can be explained by the
reproducibility of the focus position relative to the pixel
layer. Each camera/lens system has a specific depth of focus.
This is a region, in which all rays that originate from one
point are imaged onto the same sensor pixel so that a sharp
image is produced regardless of the exact focus position.
This effect and the general performance of the focus
procedure limit the focus reproducibility. Thus, the sparkle
values at the initial focus position are limited by the focus
reproducibility, while the maximum describes the position
where the sparkle is blurred the least.
Figure 5 (Right) shows the normalized sparkle focus
sensitivity of the different AGL in one setup (50 mm 1MP,
RPS = 2.1). The initial focus position was not changed. Figure 7: Local evaluation: filtered image with tiny
Thus, the focus reproducibility can be neglected. However, imperfections (Top), Local sparkle matrix (Bottom)

69
 16% Measurement Field Angle: Further, it has been found that
16mm_12MP 25mm 5MP*
25mm_1MP 25mm,5MP
the measurement field angle also affects the absolute sparkle
12%
value. Especially oblique angles have led to different results,
Sparkle

25mm,12MP 50mm_1MP
8% 50mm,5MP 50mm,12MP which can be seen directly in the exemplarily sparkle matrix
4% in Figure 10. Although the effect itself can be interesting, it
will affect the reproducibility between different camera/lens
0%
combinations, because they might cover different
no glass L1 L2 M2 M6 H1 H3 measurement field angles. Thus, a limitation of the
maximum field angle during evaluation is recommended to
CV in % 4.1 4.1 4.0 4.6 2.8 5.2. 4.2
enhance reproducibility. In our case this limit would be ± 7°.
Std in % 0.06 0.08 0.08 0.14 0.15 0.49 0.47

Figure 8: Reproducibility setup experiment with

optimized focus and local evaluation at RPS = 2.45
Boundary Conditions: Other setup conditions that have
been researched are the reproduction scale and the
measurement field angle. They were researched by
performing all measurement series of Table 1. The different
measurement distances each cover an additional
reproduction scale and measurement field angles. Due to the
distance/focus scan, this measurement series covers more
than 10,000 individual luminance images to research the
sensitivities.
Reproduction Scale: The RPS is the ratio between camera
and display pixels. It has a large influence on the absolute
sparkle value. Figure 9 shows an exemplarily distance focus
scan for a glass/camera/lens combinations at different
measurement distances. Two things can be noticed. First, a 10,0%
higher RPS leads to higher sparkle values. Second, a higher ϑMax=26° ϑMax=14°
8,0%
RPS leads to a higher focus sensitivity. This is again the ϑMax=7° ϑMax=3°
6,0%
effect of the depth of focus, which is smaller for shorter
measurement distances, which correspond to higher RPS 4,0%
values. Thus, smaller RPS values of Table 1 as for instance 2,0%
1.7 or 2.45 lead to more reproducible measurement results if 16mm 12MP RPS=2.1
0,0%
the focus scan is performed. no glass L1 L2 M2 M6 H1 H3
It should be further noted that for a comparison of absolute
Figure 10: Sparkle/Field angle sensitivity: Top:
sparkle values between different displays covering different
Evaluation of sparkle within different field angle
pixel pitches, the pixel pitch independent magnification (in
regions within the sparkle matrix. Bottom: Effect for
camera pixels per mm) of the measurement setup should be
different glasses for a short focal length lens setup
used.

20%
25mm 1MP 50mm 1MP
15% 25mm 5 MP 50mm 5MP
Sparkle

10% 16mm 12MP 25mm 12MP

50mm 12MP
5%
0%
No Glass L1 L2 M4 M1 M3 M5 H3 H4

CV in % 2.8 4.8 2.4 2.8 2.6 2.7. 1.7 2.7 1.8

Std in % 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.3 0.3
Figure 10: Reproducibility setup experiment with
Figure 9: Sparkle/Focus sensitivity for AGL M6 and optimized focus and local evaluation at RPS 2.45
different RPS (measurement distances) with second test display (183 PPI)

70
Validation pixel pitches, resolution and lenses to measure sparkle of
In order to validate our procedure, we used a second display automotive displays in a reproducible way.
with another PPI and repeated all measurements with several
AGLs. We performed the geometrical alignment with LMK References
Position, a distance focus scan and a median based
evaluation at approximately constant reproduction scales 1. Becker M. E., Sparkle measurement revisited: A
and limited the evaluated measurement field angle. The closerlook at the details, J. Soc Inf. Disp. 23 (10),
results are shown in Figure 11. The sparkle values do not (2015)
only correlate to the perceived sparkle but are reproducible 2. Becker M. E., 8-1: Standardization of Sparkle
among a large variety of BlackMURA compliant setups. Measurement: A Solid Basis, SID Symposium
Digest of Technical Papers, 49, 1, (2018).
Conclusion 3. Gollier, J et. al, 24.4: Display Sparkle Measurement
In order to enhance the reproducibility of sparkle and Human Response. SID Symposium Digest of
measurements, TechnoTeam, Volkswagen and Elektrobit Technical
systematically researched the influences of different Papers, 44, (2013).
measurement setups in more than 40 different measurement
4. Ferreras Paz V., Kohlenbecker S. and Persidis E., 8‐
setups per display, covering a large variety of BlackMURA 2: Sparkle Characterization of Anti‐glare Layers on
compliant measurement conditions and camera/lens Displays with a Grey Value Histogram Analysis,
combinations. SID Symposium Digest of Technical Papers, 49, 1
We found that the reproducibility of absolute sparkle values (2018)
can be limited by the depth of focus. The depth of focus 5. Scholz M., Standard System Specifications for
should be large enough to provide a sharp image of the Repeatable Sparkle Measurement in AG Displays,
sparkle while still enabling focusing on the “pixel layer”. It electronic display conference, (2018)
should also be small enough to maximize the focus position
reproducibility. However, it is nearly impossible to find a 6. Kurashige M. et. al, VHF1-3 Estimation of
compromise for the large variety of existing BlackMURA Equivalent Conditions for Display Sparkle
compliant setups and AGL/display combinations. Therefore, Measurement, Proc. International Display Workshop
the distance focus scan was introduced to find the correct (2019)
focus position. It should be noted that depending on the 7. Hayashi M., Simplified Method to Quantify
focus sensitivity of the RPS (Figure 8), the scan is only Sparkling of Antiglare. Display without Image
necessary if either the setup or the DUT type change. This is Processing and Its. Application, Proc. International
important to save tag time during end-of-line inspection. Display Workshop (2017)
Another important reported improvement was achieved by 8. Chiang Y.-H., Hsu T.-W.,Lin S.-C., Cheng-Hsien.
replacing the global sparkle value evaluation by a local L., Su J.-J., Novel Sparkling Quantification Method
median based sparkle evaluation. This procedure minimizes on TFT LCD, Proc. International Display Workshop
the effect of tiny imperfections such as dust, scratches or (2014)
defect pixels as well as artifacts from the Fourier filter and 9. Isshiki M., Inouye A., Tamada M., Kobayashi Y, 78‐
thus optimizes the reproducibility of low sparkling AGL. 3: The Optimized Condition for Display Sparkle
Further a measurement specification should specify the Contrast Measurement of Anti‐Glare Cover Glass
reproduction scale or the magnification and consider a based on the Solid Understandings, SID Symposium
boundary for the maximum of the measurement field angle. Digest of Technical Papers, 50, 1, (2019)
The achieved reproducibility in combination with the high 10. German Automotive OEM Work Group Displays,
flexibility regarding applicable already existing camera “Display Specifications for Automotive Application
lens/combinations allows the use of existing BlackMURA V5.1” (2018)
compliant setups. Finally, the derived procedures, 11. German Automotive OEM Work Group Displays,
algorithms and concepts are used to define a new automotive Uniformity Measurement Standard for Displays
specification for reproducible sparkle measurements at V1.3, (2018)
Volkswagen, which will be distributed in 2020/2021 and
which will allow a large variety of setups regarding camera

71
72
 Measuring MicroLEDs for Color Non-Uniformity Correction

Mike Naldrett, Matt Scholz, Bret Stonebridge, Austin Piehl, Anne Corning, Shaina Warner
Radiant Vision Systems, LLC
18640 NE 67th Court, Redmond, WA 98052 USA
[email protected]

Abstract: MicroLEDs continue to demonstrate their withstand weather swings from bitter cold in winters to
advantages for the display market. They offer many benefits scorching heat in summers.”3
over other technologies including higher brightness, faster Like other emerging display integrations that use flexible
response, and lower power consumption. These benefits are plastic LCD-TFT backplanes or OLED, microLED displays
driving significant investment in the technology and pushing can be used for flexible and curved panels that enable more
market forecast estimates up to 330 million units by 2025.1 display integration options within the vehicle interior,
However, several challenges remain to achieve efficient, increasing their appeal to automotive designers.
high-volume production—including ensuring high visual However, a significant challenge to successful microLED
quality. As individual emitters with their own performance, display development, production, and commercialization is
microLEDs exhibit luminance and color variation, unlike achieving a consistent, uniform appearance. As individual
traditional LCD displays that use uniform backlights. These emissive elements, microLEDs are driven independently
variations require that each microLED be measured and and can exhibit a high degree of variability in luminance
adjusted individually in order to achieve visual uniformity (Figure 1) and color (Figure 2), unlike traditional LCD
across the display. The chosen measurement and correction displays that use a uniform backlight.
system must be capable of providing very low takt times to
correct the high quantity of emitters in a single display, to
support efficient high-volume production processes.
This paper will discuss how these requirements can be
satisfied using imaging colorimeters, applying unique
correction methods and image processing techniques. The
benefits of various color measurement methods, imaging
system resolutions, and correction methods will be
demonstrated.

Keywords: microLED; color uniformity, color accuracy;

automated visual inspection; automotive displays; head-up
displays; HUDs; display measurement; subpixel
measurement; subpixel metrology, display correction;
demura; imaging photometry; imaging colorimetry; optical Figure 1. This close-up image of a microLED panel
metrology; pixel defect; curved displays; flexible displays shows wide variation in luminance of different pixels
across the display.
Introduction Variability is compounded because each microLED is a
MicroLED technology is ushering in new generation of monochromatic subpixel whose output is combined with
illuminated components, particularly within the automotive other subpixels to produce the overall brightness and color
segment where displays and lighting are rapidly of a single display pixel. This variability at the subpixel- and
diversifying. Commercialization of microLED displays in pixel-level manifests as non-uniformity across the display,
automobiles is anticipated within two to three years.2 The resulting in low yield of acceptable displays, rejection of
high brightness and contrast, wide color gamut, and high expensive components, or costly rework (Figure 3).
pixel density of microLEDs offer visibility and legibility in
the various ambient-light conditions of an automobile Measurement of microLED subpixels is necessary to
(particularly in bright daylight). These qualities make them quantify, evaluate, and potentially correct display output.
especially attractive for head-up display (HUD) However, microLEDs are challenging to measure accurately
applications—the superior brightness of microLEDs yields (due to subpixel size, density, and quantity per display) and
“the contrast necessary to clearly see an image against a equally challenging to correct—especially at the high
bright sun-lit background (including sun shining right volumes needed to support commercial production
through the windshield) and wide operation temperatures to demands.

73
New solutions to address microLED display quality Matching Human Visual Perception of Color
challenges combine traditional automated visual inspection In 1931, the Commission Internationale de L’éclairage
systems with innovative measurement methods. These (CIE) defined a standard for scientifically quantifying the
solutions have already been proven for identification of physical properties of colors as perceived by a human
defects like non-uniformity, with quantitative pass-fail observer, enabling accurate mathematical representation
results, fast cycle times, and reduced operational costs and reproduction of those colors (Figure 4). A color gamut
necessary for mass production and commercialization. is the set of all colors available (producible) by a display (a
For emissive displays, new measurement methods that can subset of the entire CIE color space).
detect and quantify the output of individual pixel and One reason microLED technology is attractive to display
subpixel emissive elements are enabling display uniformity manufacturers is because it can produce a wider color gamut,
correction. It is now possible to measure and correct the enhancing the customer’s visual experience. Human viewers
luminance and chromaticity output of each pixel, thereby can discern subtle variations in image brightness and color
producing displays with uniform appearance. This produced by a microLED display and experience it as higher
process—referred to as pixel uniformity correction, or quality device. To provide this enhanced viewing
“demura”—relies on the accuracy of subpixel-level experience, quality inspection of microLED displays
luminance and color measurement to calculate accurate requires a metrology system that can discern color variation
correction coefficients for each microLED. with equivalent sensitivity to the human eye. This
discernment is quantified based on mathematical formula of
the CIE standard to provide chromaticity coordinates within
the CIE color space.4

Figure 2. Illustration of an emissive display with color nonuniformity (far left), with magnification of areas across the
display (display pixels) A, B, and C that vary in color (center left); illustration of the example spectral data for A, B,
and C (right).

Figure 3. Input signals for target color gamut of DCP-P3 (D65) (row 1), compared to microLED display output (row
2), where dominant vertical non-uniformity pattern and block-wise stamp marks are clearly seen.5 (Images ©
Samsung Research / Samsung Electronics)

74
 to quantify differences in luminance and chromaticity
values, especially at the pixel level.
As shown in this paper, a color-calibrated, high-resolution
imaging colorimeter provides both the accuracy and speed
to meet production needs, and thus offers an effective
solution for pixel-level measurement of microLEDs in
production. An effective solution to meet commercial
manufacturing demands would include:
• Imaging colorimeter. The advantages of imaging
photometer and colorimeter systems include
efficiency—the ability to detect all meaningful
variations across displays in a single image,
accomplishing multiple measurements at once:
luminance, chromaticity, uniformity, contrast, pixel
defects, etc. Another advantage is scope—the ability to
capture the entire field of view (FOV) in a single image,
just as the device is viewed by a user. An imaging
photometer measures luminance, while chromaticity
measurements require an imaging colorimeter.
• High resolution. A microLED measurement system
must have high-resolution imaging capabilities. High-
Figure 4. A graphical representation of the CIE 1931
resolution imaging provides the precision needed to
color space, showing all colors visible to the human
distinguish and isolate each pixel and subpixel for
eye. The black numbers give the visible spectrum in
measurement, and the efficiency to capture values for
wavelengths, while perceived colors are represented
every pixel across increasingly high-resolution displays
as coordinate points (x and y) within these limits.
in a single image.
Considerations for Measurement • Low noise. Along with high resolution, low-noise
Because they are susceptible to color variation, microLED imaging capability is also needed. Image noise (which
emitters must be individually measured and corrected to can include read noise, shot noise, or electronic noise),
achieve displays with the highest visual quality possible. At interferes with the clarity of an image. No matter how
the same time, the chosen measurement and correction high the resolution of an imaging system (the number of
system must be capable of providing very low takt times to megapixels (MP) of its sensor), if the system captures
correct the high quantity of emitters in a single display at significant noise (yielding low signal-to-noise ratio, or
once, to support efficient high-volume production processes. SNR), then its effective resolution may be much lower.
Different display metrology systems offer benefits and
• Calibration. To perform accurate color measurement
drawbacks when applied to the various challenges of
microLED color measurement. For example, a according to CIE standards, a metrology device must be
spectroradiometric system can achieve a high degree of carefully calibrated. A common method uses reference
color accuracy, but its slow takt times make it an inefficient data captured by a spectrometer to calibrate the
solution to measure the millions of pixels that make up an response of an imaging colorimeter. Alternatively,
entire microLED display. Enhanced Color Calibration™ (ECC) is an algorithm-
Additionally, spectroradiometric systems typically have a based calibration method shown to have high accuracy
spot size that is too large to provide the most accurate without requiring a spectrometer device (see the
measurement of individual microLED emitters. A typical Enhanced Color Calibration section below).
microLED is <100 micrometers (µm) square, with <50 µm • Test & Analysis Tools. Image processing software
quite common, and some as small as 3 µm.6 Thus a enables manufacturers to optimize and run tests on a
measurement system capable of measuring structures with a
captured image. Ideally, an analysis package for display
diameter of just 0.075 mm would be insufficient to
differentiate and measure the characteristics of individual metrology would include tools to detect and quantify
microLED pixels smaller than 75 µm. luminance, chromaticity, uniformity, contrast, pixel and
line defects, display mura, and other qualities.
To meet production takt time requirements, some automated
visual inspection systems are designed to provide high-speed
measurement—for example, machine vision cameras.
However, these systems do not have the accuracy required

75
Importance of Color Measurement Accuracy Color Measurement Accuracy Study: Tristimulus
Replicating the human eye’s response to light and color System
using standard CIE functions is vital for measuring color This study looked at the color measurement accuracy of a
accurately. CIE-matched tristimulus (XYZ) color filters on CIE tristimulus filter imaging colorimeter system using ECC
a rotating filter wheel enable color measurement according compared to a reference meter (a spectroradiometer). LEDs
to standard CIE functions and chromaticity values. Light of different colors were measured by the reference meter and
entering the metrology device is passed through the by each color measurement system under test. First,
measurement systems were calibrated to a base output for
respective filters and then captured by a sensor. The filters
each LED (using ECC). Then, the systems measured each
adjust the incoming light, blocking certain wavelengths
LED as its output was varied (LED variability was
(such as UV that are invisible to the human eye) so that the introduced by supplying different current levels). The
sensors capture an image that is as close as possible to what accuracy of the color measurement system was defined by
the human eye sees. its ability to match reference meter measurements as
Tristimulus Systems. A recent study by Jensen, Piehl, and chromatic distance of the LED output increased from the
Renner (2020)7 demonstrates the high degree of accuracy of system calibration point.
a tristimulus system in matching human color perception. Results. The tabular measurement data shown in Table 1
This is discussed in more detail in the section titled Color and plotted in Figures 6 and 7 demonstrate that a tristimulus
Measurement Accuracy Study: Tristimulus System. system provides accurate luminance and chromaticity values
across introduced LED source variation. The tristimulus
Enhanced Color Calibration™. Some imaging systems system accurately measures nearly the entire range of
require a spectrometer device to supply calibration data to variation exhibited by each LED test source (at each
ensure measurement accuracy. Enhanced Color Calibration supplied current) as indicated in Table 1, Parts A and B.
(ECC) from Radiant Vision Systems provides the highest
level of color measurement accuracy via advanced In all but a few measurements, the Δ dominant wavelength
calibration algorithms. The ECC method creates a 12- of the tristimulus system measurements compared to the
element correction calibration matrix in order to maximize reference was less than 1 nm, indicating a high degree of
the ability of the color measurement system to tolerate accuracy for a tristimulus imaging colorimeter with ECC.
variability (provide accurate color measurement over a large These results demonstrate that a tristimulus system is
area of the CIE color space) from a calibrated color value suitable to accurately measure colored LEDs with a high
(calibration point; see Figure 5). degree of accuracy in both luminance and chromaticity—
even as sources vary widely from the calibration point. In
this study, the introduced variability of most of the LED test
sources exceeded the expected variability of microLEDs. A
tristimulus system is recommended for ensuring efficient
and accurate display correction where source variation is
high, or tolerance for variation is limited.
Results also demonstrate the robustness of a tristimulus
system for accurately measuring across white LEDs (Table
1, Part C). These results indicate that a tristimulus imaging
colorimeter can be used for wafer-level inspection, to
address pick-and-place applications for microLED displays,
and for general binning operations for both colored LEDs
and white LEDs (Figures 6 and 7).

Figure 5. Based on the spectral response of the

calibrated tristimulus system, it is expected to continue to
provide accuracy even as source spectral data deviates
further from the calibration point. This chart illustrates an
expected accuracy limit (area within the white dotted line
circle) for a tristimulus imaging colorimeter with values A,
B, and C plotted from Figure 2.

76
 Tristimulus System - Chromatic Accuracy Range - Colored LEDs
0.0070
Deep Red
Red
System Measuremetnt Accuracy (Δxy)

0.0060 Red-Orange
Amber
0.0050
Green
Cyan
0.0040
Blue

0.0030
Royal Blue

0.0020

0.0010

0.0000
0.0000 0.0050 0.0100 0.0150 0.0200 0.0250 0.0300 0.0350 0.0400 0.0450 0.0500 0.0550 0.0600 0.0650 0.0700 0.0750 0.0800 0.0850 0.0900 0.0950 0.1000 0.1050 0.1100 0.1150 0.1200
True Color Distance from calibration Point (Δxy)

Figure 6. Tristimulus system (ProMetric® I29 (29 MP) Imaging Colorimeter) accuracy range for color variation across
colored and white LED test sources.

Tristimulus System - Dominant Wavelength - Colored LEDs

5
System Measuremetnt Accuracy (ΔDom. λ (nm))

3 Deep Red

2 Red

1 Red-Orange

Amber
0
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12
Green
-1
Cyan
-2
Blue
-3
Royal Blue
-4

-5
True shift in source dominant wavelength relative to calibration point (ΔDom. λ (nm))

Figure 7. Plot showing true LED source variation at each current level (as measured by the reference
spectroradiometer) based on ± nm change in dominant wavelength from the calibration point (x-axis). This axis gives
the range of variation observed for each LED. The tristimulus system (ProMetric® I29) measurement accuracy is
shown as a ± nm difference from the reference measurement at each current level for each LED source (y-axis).

77
 Table 1, Part A. Test results for a 4-filter tristimulus imaging colorimeter system (ProMetric I29). The tan row of
cells indicates the calibration condition for each source. Values measured by the tristimulus system are reported
in columns under the Camera header. Values measured by the reference meter (measured at the same time and
within the same conditions as the tristimulus system) are reported in columns under the Reference header. The
error between dominant wavelength values measured by the tristimulus system and the reference meter are
reported under the Results header in columns ΔDom. Wv (nm), ΔLv (%), Δx, Δy, and Δxy.

Tristimulus System vs. Reference Measurement Agreement (xy)

LED Camera Reference Result

xy from
Peak Dom. Lv x y Dom. Wv Lv x y Cal. Dom. Lv x y xy
Wv Wv (nm) Point Wv (%)
(nm) (nm) (nm)

658 643.0 3.7 0.7208 0.2785 642.3 3.7 0.7193 0.2792 0.0007 0.7 -0.3 -0.0015 0.0007 0.0017
659 643.1 19.0 0.7212 0.2784 642.2 19.1 0.7198 0.2792 0.0006 0.9 -0.5 -0.0014 0.0008 0.0016
660 642.8 36.9 0.7211 0.2786 642.2 37.0 0.7197 0.2792 0.0006 0.6 -0.5 -0.0014 0.0006 0.0015
662 642.4 68.2 0.7205 0.279 642.1 68.5 0.7198 0.2793 0.0005 0.3 -0.4 -0.0007 0.0003 0.0008

DEEP RED 664 641.9 93.8 0.7199 0.2794 641.9 94.2 0.7194 0.2795 0.0004 0.0 -0.4 -0.0005 0.0001 0.0005
666 641.6 113.6 0.7192 0.2797 641.5 114.3 0.7192 0.2798 0.0005 0.1 -0.6 0.0000 0.0001 0.0001
668 641.1 127.5 0.7182 0.2802 641.0 128.1 0.719 0.2802 0.0008 0.2 -0.5 0.0008 0.0000 0.0008
670 640.8 134.7 0.7169 0.2806 640.5 135.3 0.7187 0.2806 0.0013 0.3 -0.5 0.0018 0.0000 0.0018
672 640.8 134.7 0.7162 0.2807 640.5 135.7 0.718 0.2807 0.0019 0.3 -0.7 0.0018 0.0000 0.0018
632 623.7 10.4 0.6983 0.3015 622.7 10.4 0.6961 0.3032 0.0055 0.9 0.0 -0.0022 0.0017 0.0028
632 624.2 55.8 0.6987 0.3006 622.9 56.0 0.6967 0.3029 0.0049 1.4 -0.4 -0.0020 0.0023 0.0030
633 624.0 110.7 0.6989 0.301 623.2 110.9 0.6972 0.3023 0.0041 0.7 -0.2 -0.0017 0.0013 0.0021
635 624.4 209.8 0.6995 0.3003 623.8 210.3 0.6983 0.3013 0.0026 0.6 -0.2 -0.0012 0.0010 0.0016

RED 636 624.7 291.7 0.6999 0.2998 624.5 292.5 0.6993 0.3002 0.0011 0.2 -0.2 -0.0006 0.0004 0.0007
638 624.9 353.0 0.7003 0.2995 625.2 353.6 0.7003 0.2991 0.0004 -0.3 -0.2 0.0000 -0.0004 0.0004
640 625.0 387.7 0.7005 0.2993 625.8 387.6 0.7014 0.2981 0.0019 -0.8 0.0 0.0009 -0.0012 0.0015
643 625.0 391.2 0.7006 0.2993 626.5 388.3 0.7025 0.297 0.0034 -1.5 0.7 0.0019 -0.0023 0.0030
645 625.1 371.3 0.7008 0.2991 626.8 370.7 0.703 0.2965 0.0041 -1.7 0.2 0.0022 -0.0026 0.0034
621 616.4 10.8 0.6832 0.3162 614.7 11.0 0.6788 0.3205 0.0155 1.7 -1.0 -0.0044 0.0043 0.0062
622 615.5 57.8 0.6809 0.3186 615.3 57.8 0.6805 0.3191 0.0133 0.2 -0.1 -0.0004 0.0005 0.0006
623 616.4 110.8 0.6832 0.3163 615.9 111.1 0.682 0.3175 0.0111 0.5 -0.3 -0.0012 0.0012 0.0017
625 617.4 196.9 0.6855 0.314 617.2 197.3 0.6849 0.3145 0.0069 0.2 -0.2 -0.0006 0.0005 0.0008

RED- 627 618.4 252.3 0.6877 0.3118 618.4 252.5 0.6877 0.3118 0.0030 0.0 -0.1 0.0000 0.0000 0.0000
ORANGE 630 619.4 273.4 0.6899 0.3096 619.6 274.1 0.6904 0.3091 0.0008 -0.2 -0.2 0.0005 -0.0005 0.0007
633 620.0 266.7 0.6912 0.3084 620.9 264.8 0.693 0.3065 0.0045 -1.0 0.7 0.0018 -0.0019 0.0026
636 620.7 231.8 0.6926 0.307 622.3 228.8 0.6954 0.304 0.0080 -1.6 1.3 0.0028 -0.0030 0.0041
638 621.2 204.7 0.6936 0.306 623.0 202.9 0.6967 0.3028 0.0097 -1.7 0.9 0.0031 -0.0032 0.0045
594 591.3 9.2 0.584 0.4171 591.4 9.3 0.5831 0.4159 0.0430 -0.1 -1.5 -0.0009 -0.0012 0.0015
594 592.2 50.1 0.5873 0.4118 592.1 51.4 0.5867 0.4123 0.0379 0.1 -2.5 -0.0006 0.0005 0.0008
595 593.1 93.3 0.5924 0.4063 592.8 95.6 0.5911 0.4081 0.0318 0.3 -2.4 -0.0013 0.0018 0.0022
597 594.2 152.6 0.5986 0.4004 594.4 154.1 0.5996 0.3994 0.0196 -0.2 -0.9 0.0010 -0.0010 0.0014

AMBER 599 596.3 183.1 0.609 0.3901 596.1 178.2 0.608 0.391 0.0077 0.2 2.7 -0.0010 0.0009 0.0013
602 597.7 178.2 0.6163 0.3833 597.7 178.4 0.6164 0.3829 0.0039 -0.1 -0.1 0.0001 -0.0004 0.0004
604 599.9 163.2 0.6253 0.373 599.6 161.3 0.6248 0.3745 0.0158 0.3 1.2 -0.0005 0.0015 0.0016
607 601.8 138.7 0.6346 0.3645 601.3 139.3 0.6327 0.3666 0.0270 0.5 -0.4 -0.0019 0.0021 0.0028
608 602.8 128.8 0.6388 0.3604 602.2 128.4 0.6364 0.363 0.0321 0.6 0.4 -0.0024 0.0026 0.0035

78
 Table 1, Part B.
Tristimulus System vs. Reference Measurement Agreement (xy), cont.
LED Camera Reference Result

xy
Peak Dom. Wv Lv x y Dom. Wv Lv x y from Dom. Lv (%) x y xy
Wv (nm) (nm) Cal. Wv
(nm) Point (nm)

534 541.7 39.2 0.247 0.7209 541.6 42.7 0.2479 0.7171 0.0557 0.1 -0.3 0.0004 -0.0033 0.0033
530 537.8 195.2 0.2233 0.7313 537.7 203.3 0.2227 0.7294 0.0335 0.1 -0.4 0.0000 -0.0023 0.0023
528 535.6 358.3 0.2113 0.7334 535.5 370.1 0.2105 0.7324 0.0239 0.1 -0.3 0.0000 -0.0018 0.0018
526 533.2 619.9 0.1994 0.7307 533.1 638.0 0.1985 0.7308 0.0140 0.0 -0.4 0.0001 -0.0010 0.0010
GREEN 525 531.7 821.9 0.194 0.7248 531.7 845.6 0.1931 0.7256 0.0067 0.0 -0.3 0.0003 -0.0006 0.0007
524 530.8 981.6 0.1922 0.7172 530.8 1003.8 0.1917 0.7180 0.0013 0.0 -0.1 0.0002 -0.0002 0.0003
524 530.2 1104.7 0.1923 0.7092 530.3 1132.0 0.1917 0.7103 0.0092 -0.1 0.0 0.0005 -0.0003 0.0006
525 530.0 1201.7 0.1942 0.7008 530.0 1228.2 0.1934 0.7019 0.0180 0.0 0.3 0.0004 -0.0005 0.0006
525 529.9 1238.7 0.1958 0.6963 529.9 1263.3 0.1951 0.6971 0.0228 0.0 0.4 0.0003 -0.0006 0.0007
510 513.4 37.1 0.0929 0.7005 513.4 38.0 0.0935 0.6991 0.1101 0.0 -0.9 0.0008 -0.0016 0.0018
508 510.4 150.0 0.0839 0.6657 510.5 153.0 0.0839 0.6657 0.0763 0.0 -0.7 0.0005 -0.0002 0.0005
506 508.8 255.4 0.0808 0.6425 508.8 260.3 0.0809 0.6425 0.0535 0.0 -0.6 0.0006 0.0000 0.0006
505 507.1 418.2 0.0804 0.6154 507.2 425.2 0.0804 0.6155 0.0269 -0.1 -0.4 0.0006 0.0004 0.0007
CYAN 505 506.3 544.8 0.0825 0.5997 506.3 554.2 0.0822 0.5995 0.0110 -0.1 -0.3 0.0005 0.0003 0.0006
505 505.9 649.4 0.0858 0.5898 505.9 659.5 0.0851 0.5892 0.0010 0.0 -0.1 0.0002 -0.0002 0.0003
505 505.8 735.7 0.0898 0.5835 505.7 747.7 0.0888 0.5827 0.0076 0.0 -0.2 -0.0001 -0.0004 0.0004
506 505.8 811.1 0.094 0.5794 505.7 823.1 0.0928 0.5784 0.0135 0.1 -0.1 -0.0004 -0.0007 0.0008
506 505.9 843.0 0.0967 0.5784 505.8 856.1 0.0952 0.5771 0.0160 0.1 -0.2 -0.0007 -0.0009 0.0011
473 475.5 12.6 0.116 0.0992 475.4 12.5 0.1163 0.0987 0.0110 0.1 1.2 0.0003 -0.0006 0.0007
471 473.9 56.6 0.1207 0.088 473.8 56.6 0.1208 0.0871 0.0028 0.1 0.6 0.0000 -0.0006 0.0006
470 473.1 102.6 0.123 0.0833 473.1 103.1 0.1230 0.0825 0.0065 0.0 0.1 -0.0001 -0.0003 0.0003
470 472.8 185.2 0.1242 0.082 472.8 186.1 0.1244 0.0809 0.0076 0.0 -0.4 -0.0002 0.0000 0.0002
BLUE 471 473.2 264.9 0.1237 0.0851 473.2 266.5 0.1239 0.0841 0.0044 0.0 -0.6 -0.0001 0.0001 0.0001
472 474.0 346.4 0.1223 0.0906 474.0 347.9 0.1226 0.0895 0.0012 0.0 -0.3 -0.0001 0.0001 0.0001
473 474.9 427.3 0.1204 0.0976 474.9 429.8 0.1208 0.0965 0.0085 0.0 -0.3 0.0001 0.0001 0.0001
474 475.8 504.3 0.1186 0.1051 475.7 507.6 0.1190 0.1041 0.0159 0.0 0.0 0.0002 -0.0001 0.0002
474 476.2 539.7 0.1177 0.1089 476.2 543.2 0.1181 0.1080 0.0199 0.0 0.0 0.0002 -0.0001 0.0002
451 454.1 4.1 0.1533 0.0238 454.2 4.2 0.1531 0.0243 0.0006 -0.1 -1.4 -0.0001 0.0003 0.0003
451 453.8 21.6 0.1537 0.0234 453.8 21.8 0.1538 0.0234 0.0013 0.0 -0.6 0.0000 0.0001 0.0001
450 453.6 42.2 0.1539 0.0233 453.6 42.5 0.1540 0.0232 0.0016 0.0 -0.5 0.0000 0.0000 0.0000
450 453.7 81.2 0.1538 0.0235 453.7 81.7 0.1539 0.0234 0.0014 0.0 -0.6 0.0000 0.0000 0.0000
ROYAL 451 454.1 118.4 0.1534 0.0241 454.1 119.1 0.1536 0.0239 0.0007 0.0 -0.6 0.0000 0.0000 0.0000
BLUE
451 454.6 154.2 0.153 0.0248 454.6 155.1 0.1531 0.0247 0.0002 -0.1 -0.3 -0.0001 0.0000 0.0001
452 455.2 187.9 0.1523 0.0258 455.2 189.1 0.1525 0.0255 0.0014 0.0 -0.3 0.0000 0.0000 0.0000
452 455.9 220.0 0.1516 0.0269 455.9 221.5 0.1517 0.0267 0.0027 0.0 -0.1 0.0000 0.0000 0.0000
453 456.4 235.3 0.1511 0.0276 456.4 237.1 0.1512 0.0275 0.0035 0.0 -0.1 0.0000 0.0000 0.0000

79
 Table 1, Part C.
Tristimulus System vs. Reference Measurement Agreement (xy), cont.
Camera Reference Result

xy from
Lv x y Lv x y Cal. Lv (%) x y xy
Point

29.4 0.4618 0.411 29.2 0.4601 0.4116 0.0670 0.5 -0.0017 0.0006 0.0018
149.3 0.4615 0.4135 148.5 0.46 0.4139 0.0676 0.6 -0.0015 0.0004 0.0016
287.2 0.4603 0.414 285.7 0.4587 0.4145 0.0666 0.5 -0.0016 0.0005 0.0017
534.0 0.4572 0.4136 530.9 0.4558 0.414 0.0637 0.6 -0.0014 0.0004 0.0015

WHITE 747.6 0.4542 0.4125 743.0 0.4529 0.4128 0.0606 0.6 -0.0013 0.0003 0.0013
(2700K) 930.9 0.4512 0.4111 924.3 0.4499 0.4114 0.0573 0.7 -0.0013 0.0003 0.0013
1078.6 0.4478 0.4097 1071.0 0.4469 0.4099 0.0540 0.7 -0.0009 0.0002 0.0009
1191.7 0.4446 0.4082 1182.0 0.4437 0.4084 0.0504 0.8 -0.0009 0.0002 0.0009
1230.4 0.4425 0.4074 1220.1 0.4419 0.4074 0.0484 0.8 -0.0006 0.0000 0.0006
1253.6 0.4409 0.4067 1243.1 0.4404 0.4066 0.0467 0.8 -0.0005 -0.0001 0.0005
33.4 0.4115 0.4042 33.1 0.4103 0.4046 0.0195 1.0 -0.0012 0.0004 0.0013
162.6 0.4096 0.4024 161.1 0.4084 0.4026 0.0168 0.9 -0.0012 0.0002 0.0012
307.7 0.4076 0.4003 304.9 0.4063 0.4006 0.0139 0.9 -0.0013 0.0003 0.0013
562.2 0.4038 0.3968 556.9 0.4027 0.397 0.0088 0.9 -0.0011 0.0002 0.0011

WHITE 777.1 0.4003 0.3935 769.6 0.3993 0.3936 0.0040 1.0 -0.0010 0.0001 0.0010
(3500K) 955.1 0.3971 0.3903 945.1 0.396 0.3905 0.0006 1.1 -0.0011 0.0002 0.0011
1091.3 0.3934 0.3875 1079.8 0.3927 0.3875 0.0050 1.1 -0.0007 0.0000 0.0007
1185.4 0.3895 0.3845 1172.2 0.389 0.3844 0.0098 1.1 -0.0005 -0.0001 0.0005
1213.8 0.3872 0.3829 1200.1 0.3869 0.3827 0.0125 1.1 -0.0003 -0.0002 0.0004
1228.7 0.3852 0.3816 1214.7 0.3851 0.3814 0.0148 1.2 -0.0001 -0.0002 0.0002
37.9 0.3362 0.3565 37.3 0.3341 0.3564 0.0712 1.7 -0.0021 -0.0001 0.0021
187.1 0.3349 0.3532 184.2 0.3325 0.353 0.0743 1.6 -0.0024 -0.0002 0.0024
357.8 0.3335 0.3506 352.4 0.331 0.3503 0.0770 1.5 -0.0025 -0.0003 0.0025
664.4 0.3306 0.3468 654.3 0.3281 0.3464 0.0815 1.5 -0.0025 -0.0004 0.0025

WHITE 930.6 0.3276 0.3436 916.2 0.3252 0.343 0.0858 1.6 -0.0024 -0.0006 0.0025
(5700K) 1158.6 0.3246 0.3404 1139.4 0.3222 0.3399 0.0900 1.7 -0.0024 -0.0005 0.0025
1341.0 0.3208 0.3372 1319.1 0.3187 0.3365 0.0949 1.7 -0.0021 -0.0007 0.0022
1476.7 0.3165 0.3336 1451.7 0.3147 0.3329 0.1002 1.7 -0.0018 -0.0007 0.0019
1523.3 0.3139 0.3316 1497.4 0.3122 0.3308 0.1034 1.7 -0.0017 -0.0008 0.0019
1551.7 0.3117 0.3299 1525.0 0.3101 0.329 0.1062 1.7 -0.0016 -0.0009 0.0018

80
Importance of Resolution advantage of an imaging colorimeter is its ability to capture
As noted previously, microLED size ranges from less than a large area in a single image to detect and measure non-
100 µm to as small as 3 µm—about 1/10th the width of a uniformity quickly and accurately, just as a user would view
human hair.7 Measurement accuracy within the small area of a display.
an individual microLED depends on high imaging Beyond the visual perception of a user, however, high-
resolution. To increase yields through display correction, it resolution imaging systems can continue to provide pixel-
is essential to be able to isolate and measure each individual and subpixel-level measurements at the panel level to enable
microLED emitter with precision so defects and non- correction. To assess color gamut, luminance uniformity, or
uniformity specific to a given microLED can be corrected. color uniformity at this stage, manufacturers need accurate
A high-resolution imaging system optimizes the number of data at each display pixel’s coordinate position, which can
photo-sensing elements (sensor pixels) applied across each then be downloaded into a coefficient calculator to
microLED and offers sufficient resolution to ensure all determine and apply factors for display uniformity
microLEDs in the display can be measured at once to correction.
complete correction processes within adequate takt times.
A microLED panel is composed of millions of pixels in chip
form that are typically grown on 4- to 8-inch wafers. Each
pixel contains some combination of red, green, and/or blue
subpixels. To fabricate a display, each individual microLED
chip (pixel) must be transferred to a substrate or backplane
(panel) that holds the array of units in place. Measurement is
typically performed at two stages of microLED production:
at the wafer level and at the panel level.
General visual performance standards in the display industry
allow for less than 10 dead pixels per display, thus epitaxial
yield must be very high. Each microLED on the wafer must
be measured to determine uniformity, verify individual
distribution of dies, and measure luminance across red,
green, blue, and occasionally white microLEDs. Once
wafers have been deposited onto a backplane, manufacturers
then need to verify overall uniformity of luminance and
color distribution across the entire panel.
The first step in ensuring microLED display quality is
inspection and measurement at the LED, chip, and wafer
stage to reduce the possibility of dead pixels and ensure Figure 8. Example of subpixel measurement using a
luminance and wavelength (chromaticity) uniformity. ProMetric I29 Imaging Colorimeter (main image),
Wafer-Level Measurement. For inspection of microLED magnified by a Radiant Microscope Lens showing 50
wafers, manufacturers must assess performance at the µm distance at 10X zoom (inset image).
individual subpixel (microLED) level. A high-resolution,
low-noise imaging colorimeter with a standard lens or Correcting Emissive Displays to Improve Yield
microscope lens option can be used for this process. As display size scales, yields decline drastically, and the cost
of each component is much higher. At a certain point, it
A microscope lens provides objective measurement with, for becomes viable for manufacturers to perform correction
example, 5X or 10X zoom (5 to 10 times the effective (electronic compensation, or calibration) to improve display
resolution of the imaging system applied over an area of the image quality. The concept is simple: by modifying the
device), allowing detailed measurement of each individual inputs to individual subpixels of an emissive display,
emissive element (Figure 8). Used with a high-resolution previously identified dim pixels can be adjusted to a uniform
imaging system, a microscope lens enables every display brightness level resulting in improved luminance uniformity
pixel to be captured over multiple sensor pixels for increased and correct color across the display.
measurement precision. This type of system is effective for
evaluation of display subpixels and characterization of Display pixel uniformity correction requires, first, having in-
individual microLEDs. display electronics that can control brightness of the
individual subpixels and make adjustments based on the
Panel-Level Measurement. Once individual microLED calculated correction factors for each subpixel. Second, a
chips are transferred onto a backplane, an imaging measurement system is required to accurately quantify
colorimeter with standard lens can be used to measure individual subpixel brightness and color, and compute
luminance and color uniformity across an entire panel. The specific correction factors for each of them. This method was

81
originally developed to calibrate LED video screens (e.g., requiring new approaches to continue to achieve accurate
outdoor arena displays), and has been adapted for today’s measurement (Step 1) and correction (Steps 2 and 3) results.
small, high-resolution emissive displays (OLED and Accurate pixel-level measurement relies on a measurement
microLED) using a correction technique known as “demura” system’s ability to sufficiently isolate each pixel and
or pixel uniformity correction. precisely quantify its output value. As described, imaging
Demura. The demura method employs three distinct steps: resolution determines the number of photo-sensing elements
(sensor pixels) available to cover each individual pixel.
1. Measure each subpixel in the display to calculate
Applying more sensor pixels per display pixel increases the
luminance values at each pixel coordinate location
granularity of data acquired by the imaging system for
using a high-resolution imaging colorimeter. Accurate
accurate pixel registration and pixel measurement. As
measurement values for each subpixel are essential.
overall display resolution increases, an imaging system’s
Test images are displayed on-screen to target subpixels
ability to apply sufficient sensor pixels per display pixel—
of each color set. These images enable measurements
while continuing to capture measurements for all display
and correction factors to be computed for each set. For
pixels in a single image to ensure efficiency—is reduced.
example, a green test image can be shown to illuminate
all green subpixels. An imaging colorimeter measures Radiant has developed two methods that significantly
and records the output of each individual green improve an imaging system’s ability to isolate and measure
subpixel. This is repeated for all the primary colors and, subpixels of increasingly high-resolution displays for
usually, white. correction: a “spaced pixel” method and a “fractional pixel”
method; both are described below.
2. Load measurement data from each pixel’s coordinate
position into a coefficient calculator. Correction factors Spaced Pixel Method. The spaced pixel measurement
are calculated to normalize luminance and chromaticity method (US Patent 9135851) improves the effective
discrepancies between pixels in the display using test resolution of a measurement by applying the measurement
analysis software. system’s total image sensor resolution across only a subset
of display pixels at a time. The method employs a series of
3. Apply correction factors to the signals of each subpixel
dot-matrix test patterns shown on the display screen as part
at each pixel location using an external control IC
of a measurement sequence. Each pattern illuminates a
(integrated circuit) system.
subset of the display’s subpixels, while the rest are turned
off. An imaging colorimeter measures the output (luminance
MicroLED Correction Challenges: Registration
or chromaticity) of the “on” pixels for each pattern. A
and Measurement
subsequent test image adjusts the matrix to turn off the first
The demura method has been proven effective for ensuring
set of pixels and turn on the next set of pixels for
the visual quality of millions of displays in mass production measurement. This process is repeated until all pixels in the
worldwide. However, new microLEDs offer the potential to display are measured.
increase display resolution and pixel pitch exponentially,

Figure 9. During spaced pixel measurement, a series of dot-matrix patterns illuminates sets of pixels until values
have been acquired for all pixels in the display.

82
This process increases the effective resolution of the
measurement at each display pixel, ensuring the isolation of
each pixel’s output, and thus the accuracy of measurement
calculations across displays of any arbitrary resolution.
Images from the spaced pixel measurement method are
combined into a single, synthetic image for analysis, which
compares values at each of the pixel’s x,y coordinate
locations to determine uniformity. The software calculates
the necessary correction coefficient for each display pixel
and applies the correction at each pixel’s coordinate location
to adjust values until the display is uniform.
The spaced pixel method reduces the requirement for
measurement resolution of an image-based system to
increase measurement accuracy. However, because this
Figure 10. Illustration of a traditional whole pixel
method requires multiple images, takt times are also measurement method versus the fractional pixel method. In
increased. The fractional pixel method, by comparison, the traditional method (left), display pixels are measured
improves measurement accuracy without increasing takt using 100% of the data from sensor pixels whose area is
times, for example, only a single image is required to more than 50% inside the ROI, and 0% of the data from
measure typical smartphone displays. Thus, the fractional sensor pixels whose area is less than 50% inside the ROI.
pixel method (explained in the next section) offers Using the fractional pixel method (right), display pixels are
advantages when shorter takt times are required, as in many measured using a percentage of data based on the
production-level test and correction applications. percentage of sensor pixel area inside the ROI.

Fractional Pixel Method. The fractional pixel method can The accuracy of the fractional pixel method was
be optionally combined with the spaced pixel method and demonstrated in a study published by Pedeville, Rouse, and
uses fractional image sensor pixels to further improve the Kreysar (2020).8 Figure 11 plots single-image measurement
precision of display pixel registration and measurement. As data from this study, comparing the pixel-level measurement
described, this method addresses measurement scenarios accuracy of fractional pixel measurements, whole pixel
where imaging sensor resolution per display pixel is limited, measurements, and extremely high-resolution reference
enabling measurement systems with standard resolutions to measurements. The fractional pixel data adheres closely to
continue to accurately measure and correct today’s high- the reference data, whereas the whole pixel measurements
resolution displays, even in a single-image measurement. diverge from the reference data at multiple points.
First, the fractional pixel method optimizes pixel Figure 12 shows the before-and-after result of an actual
registration. Pixel registration is a method of dynamically demura application using a 43MP imaging system
locating and setting a region of interest (ROI) around each employing both spaced and fractional pixel methods to
pixel in the measurement image. In traditional methods, ROI correct a microLED microdisplay panel.
are aligned to the imaging system’s sensor pixel array.
However, as display resolutions continue to increase relative Conclusions
to measurement system resolutions, it is more likely that the Use of microLED displays is growing in a highly
center of a display pixel will not be aligned with the center competitive automotive marketplace. Developers are racing
of a sensor pixel, reducing the ability of the ROI to precisely to find production solutions that deliver cost-effective
cover and isolate each display pixel. This misalignment can performance that meets industry standards while exceeding
result in measurement error. customer expectations.
Second, the fractional pixel method optimizes pixel Defects, variations in color or brightness, and other
measurement. The fractional pixel method calculates pixel irregularities can quickly deflate buyer satisfaction, hurt
values based on the fractional area of each sensor pixel brand reputation, and erode market share. If these issues
contained within the ROI (see Figure 10, right image). This cannot be addressed and corrected at the component level,
improves the precision of measured values over traditional low yields and high production costs will impede the
“whole pixel” methods that factor values from the whole viability of microLED display technologies for automotive
area of sensor pixels contained partially withing the ROI (see commercialization.
Figure 10, left image). The fractional pixel method ensures Image-based color measurement systems provide an
the accuracy of pixel-level measurements for emissive efficient quality control solution to support quality and
displays of much higher resolution than was previously efficiency benchmarks for microLED production. These
possible using a single-image capture (e.g., when a systems rely on their color filter method, calibrations, and
measurement system’s sensor resolution is applied to subpixel measurement capabilities to ensure accurate data is
measure an entire display at once, to increase takt time). captured at the pixel and subpixel level, enabling correction

83
that safeguards manufacturing resources. Studies of recent Using imaging colorimeters and novel correction methods,
display metrology systems and methods demonstrate the automotive manufacturers and suppliers can realize a
effectiveness of tristimulus imaging colorimeters in the 29- solution for production efficiency in microLED
43MP resolution range, combined with sophisticated manufacturing, ensuring quality, reducing waste, and
algorithms for calibration, measurement, and correction, to enabling cost-effective design of higher-performance
solve microLED display color uniformity challenges and displays in a range of types, shapes, and curvatures.
support the viability of microLED technology for the
automotive industry.

Figure 11. Normalized luminance (Lv) measured by whole and fractional pixel measurement methods (achieving 3.2
x 3.2 sensor pixels per display pixel) and reference luminance (achieving 30 x 30 sensor pixels per display pixel) for
the same row of pixels on a display.

MicroLED display before demura MicroLED display after demura

Figure 12. MicroLED microdisplay panel shown before (left) and after (right) demura correction. The panel shown is
0.7” with full HD, 1920 x 1080, LED size/pitch of 2 µm / 8 µm. Measurement and correction performed using a
ProMetric Y29 imager with microscope objective lens and TrueTest™ Software.
References 5. Kim, K., Lim, T., Kim, C., Park, S., Park, C., et al.,
1. Yole Développement. MicroLED Displays, February “High precision color uniformity based on 4D
2017. Retrieved from: transformation for micro-LED.” Proceedings of SPIE
http://www.yole.fr/MicroLEDDisplays_Market.aspx Vol. 11302, February 2020
2. Han, S., Hwang, A, “Automotive micro LED display 6. Lapedus, M., “MicroLEDs: The Next Revolution in
commercialization expected in 2-3 years,” DigiTimes, Displays?” Semiconductor Engineering, May 2019.
April 2020. Retrieved from Retrieved from
https://www.digitimes.com/news/a20200423PD210.ht https://semiengineering.com/microleds-the-next-
ml) revolution-in-displays/
3. MicroLED: Total Market for Automotive HUD 2018- 7. Jensen, J., Piehl, A., and Renner, W., “Evaluating
2024, July 2018. N-tech Research. (Retrieved from tristimulus and Bayer pattern matching system
https://www.ntechresearch.com/infographic/automotiv accuracy for color measurement based on CIE color-
e-heads-up-display/) matching functions,” Presented at the 34th annual
4. Carter, E., Schanda, J., Hirschler, R., Jost, S., Luo, M., electronic displays conference (edC), January 2020.
Melgosa, M., … Wold, J., CIE 015:2018 Colorimetry, 8. Pedeville, G., Rouse, J., and Kreysar, D., Society for
4th Edition, 2018. doi: 10.25039/tr.015.2018 Information Display (SID) Display Week 2020 Digest.
Book 2, August 2020.

84
 Supervising (Automotive) Displays for Safeguard Camera Monitor Systems
B. Axmann1, F. Langner1, K. Blankenbach2, M. Vogelmann2, M. Conrad3, J. Bauer4
1Mercedes-Benz AG, Stuttgart, Germany; 2Pforzheim University, Display Lab, Pforzheim, Germany;
3samoconsult GmbH, Berlin, Germany; 4Karlsruhe University, Karlsruhe, Germany

ABSTRACT: Modern cars are equipped with

Camera Monitor Systems (CMS) such as back-up
camera systems or mirror replacement systems.
To fulfill the demanding safety requirements for such
systems, information that is transmitted and
processed within CMS should be monitored and
supervised. In today’s CMS however, only the digital
data is being supervised.
This paper introduces new methods for the optical
safeguarding of displays using cameras. These
methods enable end-to-end or “light-to-light” fault
detection within CMS.
Keywords: camera monitor system (CMS);
automotive display; functional safety; safety
mechanism; ISO 26262
Figure 1. In-vehicle CMS and wireless video
1 INTRODUCTION transmission to a remote operator.
Modern cars and trucks are equipped with rear-view
camera systems. Side-view mirror replacement Figure 2 sketches a typical automotive CMS: An
systems are at the dawn of mass production. Such input component, usually a camera captures an
features are termed camera monitor systems (CMS, optical signal from the vehicle’s surrounding
see [1] for an overview). CMS provide obvious environment and transforms it into digital video
benefits in terms of safety. A typical rear-view (back- data. The digital video data is being routed through
up) CMS (Figure 1) consists of a camera (transmitter) the vehicle and might be processed and/or modified
that is connected via a high-speed video data by an electronic control unit such as a head unit.
interface or link (HSVL) to a head unit (processing Finally the processed / modified video stream is
unit), which enhances and modifies the image data transformed into an optical image that is shown to
(e.g. overlaying the image with augmented the driver. The digital video data might also be
information such as trajectories) and sends it to an transmitted to a remote operator workstation where
in-vehicle (car) display. it is also being transformed into an optical image
and displayed to the operator.
If an autonomous (robot) car without a steering wheel
fails, it will stop and become an obstacle. In such a To ensure the safety of such a CMS, the optical
situation, a remote operator (Figure 1, top right) can image shown to the driver or a remote operator
log in to the car system and perform remote control should match the optical signal captured by the
(like a drone pilot). As a prerequisite for remote camera. Any information overlaid to the digital video
control, the operator needs to see the images data should be correctly displayed as well.
captured by the cameras of the vehicle as well as Traditional methods to check the correct functioning
other vehicle status information (e.g. vehicle speed, of a CMS, such as video watermarking [2],
activated tell tales, etc.). safeguard the digital data only (“data-to-data”
For brevity, in Figure 1 these two use cases were protection, see Figure 2, left). The transformation of
superimposed onto one vehicle, such that camera the optical signal into digital information in the
data and other relevant information can be displayed camera and the transformation from digital
on an in-vehicle display as well as on a workstation information into an optical image within the display
of a remote-operator. As in-vehicle CMS and remote are usually left out.
operator systems shall be designed as safe as Our goal is to extend the coverage of the
possible, we investigated and prototyped new safeguarding mechanisms to enable a full “light-to-
approaches to safeguard such systems. In this light” protection (see Figure 2, right). A key element
paper, we report about how display components of is the ASIL Prepared Video Safety System
such systems could be safeguarded. (APVSS), which was developed and evaluated

85
successfully in our project. The main tasks of APVSS This enables the application of safety mechanisms
are to compare meta data (obtained by feature that facilitate the verification of the content
extraction of the image) from both camera and output displayed. These mechanisms can not only be used
device (digital front-end of a display) and to judge on to verify the content of a remote display, but could
the optical reproduction quality of the display by also for in-vehicle displays as well. However the
measurement data. This paper focusses on the latter may result in high integration effort.
display component, i.e. how to supervise displays
including the transformation from digital information 2 FUNCTIONAL SAFETY FRAMEWORK
into an optical image. Safety is one of the key issues in the development
of road vehicles [1]. The international standard ISO
26262 [4] provides guidance to achieve functional
safety, i.e. to mitigate risks resulting from failures of
electric / electronic in-vehicle systems (E/E
systems).
Engineering for functional safety includes
• The systematic identification and analysis of
potential failure modes of the E/E system
under consideration as well as
• The design and implementation of safety
mechanisms, i.e. technical solutions to detect,
mitigate, or tolerate faults or control or avoid
failures [4].
In the following, we utilize the generic model of a
video data transmission and processing system
(see [5] for more details) depicted in Figure 3 as a
conceptual framework for discussing functional
safety aspects of automotive CMSs. This model
abstracts the physical components of a specific
system into logical components of type input (I),
modifier (M), output (O), and transmission channel
(T). A real system might comprise zero, one or
multiple instances of each type. Some components
may be located outside of the actual vehicle, e.g.
the monitor of a remote operator.

Figure 2. Block diagram of today´s (left, magenta)

and our new concept (right, green) for safety.
This paper is arranged in two main paragraphs: (§2)
functional safety framework and (§3) display
supervision using a camera. §2 outlines the
conceptual framework to analyze and improve the
functional safety of CMS. Here, we aim at providing
a set of generic methods to safeguard automotive
CMS such that a particular CMS under development
with defined safety requirements can pick and
choose from this set.
§3 illustrates the application of this safety framework
by using a video system that involves a remote
operator as an example. First, we describe the
overall set-up
• To acquire the optical output of the display using
a dedicated camera and
• To correlate the camera data with the RGB input Figure 3. Video Transmission and Processing
data fed into the display. Model. Flashes mark sources for failures.

86
In case of a back-up camera system, the digital video coverage. Therefore, SM1 needs to be implemented
data consisting of pixel data and meta information is as well. Should the detection of FM3 be required as
being generated by the rear view camera as a stream well, none of the analyzed failure mechanism would
of images (Im), transmitted via a wired transmission be suitable. In such a case, an additional (usually
link (HSVL), and displayed on the central information system-specific) safety mechanism needs to be
display as a stream of images (Im''). The devised.
transmission channel contains an additional cockpit
domain controller that modifies the images by
optimizing contrast and brightness or by
superimposing the images with additional
information, such as HMI information.
Faults in the system’s components or an incorrect
transmission may lead to unintended modification of
images such that the displayed video stream Im’’
critically deviates from the original video stream Im.
2.1 Systematic Evaluation of Available Safety
Mechanisms Figure 4. Evaluation of Safety Mechanisms.
As a prerequisite to handle these faults and/or the 2.2 Generic Safety Architecture for CMS
resulting failures, the authors systematically To standardize the implementation of the safety
identified and analyzed potential failure modes of mechanisms selected this way, the authors
each component type [cf. 5]. For output components propose the generic safety architecture depicted in
realized by displays, 30+ failure modes (FMs) were Figure 5. As per this architecture, the conceptual
identified and analyzed. These failure modes include model (see Figure 3) is being extended by an
e.g. image corruption, image distortion, frozen additional component type, the so-called ASIL
image, delayed image, erroneous zoom factor, prepared video safety system (APVSS).
erroneous image orientation / color / contrast /
brightness, image artifacts, erroneous
augmentation / marking and modification / loss of
essential information.
Then, safety mechanisms (SMs) utilized or proposed
to detect these failure modes were collected. Some
of these safety mechanisms, could be locally
implemented in the display (e.g., current / voltage
monitoring of display panel components with low
temporal resolution), where others need to be
distributed across multiple components (e.g., video
watermarking).
In order to evaluate the fault coverage of safety
mechanisms and to inform the selection of suitable
safety mechanisms for a given CMS, we created a
matrix with the FMs as rows and the SMs as columns Figure 5. Video Transmission and Processing -
(Figure 4). The capability of a safety mechanism SMx
Safety Architecture
to detect failure mode FMy is indicated by a
checkmark in the corresponding cell. If known, the The local safety mechanisms of the I, M and O
diagnostic coverage, i.e. the percentage of the components of the system collect status and / or
detected faults, is being captured as well. fault information and transmit them to a global
Such an analysis can be used to devise suitable and APVSS component. This global APVSS controller
efficient safety mechanisms for a video system under could be a separate component or be integrated
development: If for example failure modes FM1 and into one of the other components.
FMm need to be detected with medium diagnostic In a first stage, the APVSS realizes a health
coverage (i.e., ≥ 90%) each, a combination of SM2 monitoring for each component that implements the
and SM1 could be implemented. SM2 alone would not detection safety mechanisms for this component.
suffice as it would be capable to detect FMm with the This local health monitoring could be distributed
required high diagnostic coverage, but not FM1. FM1 between local APVSS portions and the global
could be detected by SM2 as well, but only with low APVSS component. The component health
(i.e., ≥ 60%) and thus insufficient diagnostic monitoring generates a component health status.

87
A second stage combines the component health In theory, the same method, i.e., monitoring of the
statuses of all components into an overall health optical emission of the display using a camera,
status of the system. If critical faults or failures are could be used for in-vehicle displays as well. But
detected, this second stage commands counter here it forces some design and technical challenges
reactions, e.g. the display of an default / error image (e.g., mounting a camera in front of central
etc. or lastly the shutdown the output component to information display, interference with touch control
give clear understanding for the user. and in particular ambient light).
The prototype system was tested using MATLAB [6]
3 DISPLAY SUPERVISION USING A CAMERA and programmed in PHYTHON with the use of
This paragraph illustrates the application of the safety OPENCV image processing library.
framework presented above in order to supervise an
in-car video camera system that transmits video data
to the display / monitor of a remote operator. This
monitor can be installed, e.g., in the headquarters of
a robot car operator. In case of a breakdown or an
autonomous driving mode failure, such a remote
operator can log into the car, take over control and
eventually drive the car remotely. It is obvious that in
the latter case, significant degradation of the video
image or faulty vehicle data (for example speed)
cannot be tolerated.
3.1 Prototype Approach and Set-up
There are various approaches for optical supervision
(see [3]) but most methods require modifications to
or customization of the display. Supervising a display
or monitor by a camera allows the use of an
“unmodified” mass production monitor which is
significantly cheaper than a modified display.
The supervision of displays of a remote operator
would be an example of a low-volume use case. In
such a scenario, the operator’s displays will likely
consist of standard consumer electronics (CE) PC
monitors (Figure 6 bottom). The block diagram at the
top of Figure 6 visualizes the adaptation of the safety
architecture presented in §2 to the camera-based Figure 6. Safety architecture (top; see §2 for
supervision of a monitor. details) and prototype setup (bottom) to
This approach is relatively easy to implement. It evaluate camera-based supervision of a
requires just a simple mount of a tiny camera e.g. on remote operator´s monitor.
top of the monitor (Figure 6 bottom). This camera,
which could be a professional or CE grade device as 3.2 Pre-Processing
well, captures the actual image reproduced on the When grabbing images from a display by a camera,
operator’s monitor. it is obvious that unwanted effects are very likely to
occur. Examples are geometric distortions due to
The task of the APVSS (see Figure 2) for the remote the camera position (cf. prototype setup in Figure
operator use case is to compare the RGB data of the 6), mismatching luminance output and camera
image sent to the monitor with the RGB image data input characteristics and reflections of ambient
captured by the supervising camera. light. These and other effects have to be handled
This approach allows to use high-resolution optical by so-called pre-processing before the RGB input
safeguarding by computer vision algorithms as data of the digital display and the data acquired by
described in e.g. [6] or toolboxes. The camera image the supervising camera can be assessed and
data is being analyzed and the results are transmitted compared.
to the “health” monitoring component of the remote Figure 7 (top) illustrates the image pre-processing
APVSS (see Figure 2). Depending on the required steps. First, the image has to be checked for
safety level, the APVSS can be implemented as a periodic Moiré patterns (interference of display and
software solution on the operator’s PC or in an camera “grid”). This can be reduced via software by
optional external hardware component. a Fourier transform and band filtering or by varying

88
the focal length of the lens and/or the distance So the fundamental requirements are fulfilled:
between the monitor and the camera, or by slightly Reasonable linear RGB gray scale camera data
defocusing. which is mostly linear to the display output. If
As the add-on camera is placed above or beneath the computational time or acquisition speed is not
display, the image is geometrically distorted. The critical, both images can be analyzed separately
perspective correction (warping) can be achieved and combined to one image with higher virtual gray
using the four corner points and the display’s outline level resolution, for example 1/64 s and 1/512 s
as reference. For calibration purposes, a grid can be raises the resolution by 3 bit (i.e. by a factor of 8).
used during initialization of the remote operator’s The resulting “virtual” image is used for further
monitor for compensation of geometric distortions of analysis.
the camera lens such as barrel or pincushion.
The last pre-processing step is to achieve a linear
relationship between display output (luminance) and
camera RGB gray level data. This can be done once
at initial set-up of the system or permanently using
gray level control boxes (see Figure 100 bottom
right). If the display’s use case does not allow
visualization of such control boxes, pre-defined gray
level areas in the GUI can be used. To be able to
adapt the transfer function to changing ambient light
conditions, the system uses black (RGB = 0) areas
as reference and perpetually determines the gray
level reproduction. Figure 8. Gradation curves for various exposure
times used to expand dynamic range.
An example of the enhanced dynamic range is
given in Figure 9: A typical night scene of a rear-
view camera is recorded at different exposure
times. The luminance of PC monitors of remote
operators at office conditions (e.g., 500 lx) is in the
range of 200 cd/m². If such a camera-based optical
display supervision would be installed in a car it has
to deal with a large dimming range from 10 cd/m² to
1000 cd/m². The left image has a shorter exposure
time then the right one. The blue and yellow dotted
boxes mark the significant differences: A shorter
exposure time avoids over-exposure (saturation)
Figure 7. Pre-processing of the raw camera data
for bight content (high gray levels) and vice versa.
and subsequent monitoring methods
The combination results in a better discrimination of
As most displays and cameras have a gray scale objects with low foreground-background contrast.
resolution of 8 bits, it is obvious that just grabbing This can be easily seen for the dashed boxes of
images from the display results in low quality of the Figure 9 for bright (blue, exposure time 1/512 s) and
supervision. For professional systems we have to dark (yellow, exposure time 1/64 s) content.
respect the dynamic range of the displays and the
camera. One approach is to use a high dynamic
range camera or to acquire two subsequent images
with different exposure times for standard cameras.
Figure 8 visualizes gradation curves for different ex-
posure times. Longer exposed images present higher
resolution at lower gray scale range and vice versa. Figure 9. Visibility of bright and dark objects using
The goal is to get two linear curves of different sensi- short (left) and long (right) exposure time.
tivity to capture and resolve low and high luminance
content. As an example (cf. Figure 8), the cyan line 3.3 Content-Dependent Safety Mechanisms to
(exposure time 1/64 s) is linear up to 0.35 of the Verify Displayed Content
relative maximum gray level; saturation is reached at A core requirement of camera-based display
0.5. The red curve (exposure time 1/512 s) is mostly supervision is to detect a “modification or loss of
linear from 0.2 to 1.0 without reaching saturation. essential information” with high probability to

89
prevent e.g. critical situations for remote car control. line is achieved using Hough transformation
As the safety of data transmission is secured by (plotted as green line speed is also possible when
state-of-the-art methods, we have to deal here with reflections are present (Figure 11 right). The
modifications or degradations of the optical / visual detected position or speed can be compared with
reproduction by the display / monitor. They can be the expected position or the value for the vehicle
caused e.g. by gamma distortions which mainly speed (could be provided as part of the meta data).
affect video images but can as well reduce the If a significant deviation is detected by the APVSS,
visibility of vehicle status information. a warning is shown. In case of the remote operator
As the Graphical User Interface (GUI) of such remote use case, additional failures (like zero speed but
operator monitors should be ergonomic, the different moving video image due to a defective
types of information on the screen are displayed at speedometer defect) would be detected by the
fixed positions / in fixed areas. This can be used to human.
divide the captured images into Regions of Interest
(ROI) and applying dedicated and optimized
algorithms to analyze the corresponding content.
In-car display GUIs can be modified at some extent
by the driver. However, the ROI approach is still
applicable here, but requires additional information
about the current display layout.
To evaluate and optimize safety mechanisms to
verify ROIs based on their specific content, we used
the test GUI shown in Figure 10. It consists of 4 Figure 11. Acquisition of the displayed speed by
regions of interest. The first three are typically for extracting the speedometer needle angle.
automotive use, and the last one is used for live Despite some drawbacks, text messages can be
calibration and supervision purposes: used for safety-related error messages additionally
• A digital speedometer (top left), to tell tales for remote operators. To verify their
• Text messages and tell tales (bottom left) correctness, we applied standard optical character
recognition (OCR) to compare the captured (actual)
• A rear-view camera image (top right) and text with the transmitted (target) content. Here, the
• Gray level and colored reference boxes (bottom quality and performance of the OCR algorithm
below video image). highly depends on the pre-processing of the image.
Figure 12 shows an example sequence to properly
pre-process the image in order to extract character
information by OCR. After color removal the
contrast reproduction is inversed. A subsequent
bilateral filter provides uniform blur without
smoothing edges to reduce image artifacts. Faster
median filtering is not expedient due to smooth
edges. In the last step before OCR, the contrast-
and brightness-ratio is adjusted to gain maximum
Figure 10. Test image with multiple ROIs. contrast ratio for best OCR performance. This effort
As stated before, we assume for a remote operator has to be performed of the camera resolution in in
fixed locations of content. So we can apply dedicated the range of the display resolution for the example
algorithms for each ROI and optimize them in terms provided in Figure 10. If the camera resolution
of specific safety mechanisms. The goal is to verify and/or the character size are higher, standard OCR
that essential information is being properly displayed provides reasonable results.
in these ROIs, minor effort was made for optimizing
the speed of the algorithms for prototype evaluation.
To verify whether the vehicle speed is displayed
correctly, the angular position of the red
speedometer needle is being detected. After color
filtering and binary morphological operations, the
‘canny edge detection algorithm’ is applied to detect
the speedometer needle’s outline. Then, a
parametric description of the speedometer needle Figure 12. Stepwise processing of the text ROI to
prepare for best optical character recognition.

90
Tell tales (see Figure 10 bottom left) such as head 4 SUMMARY
light status are analyzed by “traditional” methods, Safe and unaltered reproduction of camera content
which have however some limitations. is essential for modern automotive camera monitor
The video content is supervised by feature systems (CMS) applications.
comparison of the camera image and the digital data In a first step, we developed a generic functional
at the display’s interface. 100 or more segments (see safety framework for possible failures with respect
Figure 13 for an example) are correlated for e.g. gray to their levels of criticality and detection methods.
scale and color, both in terms of mean value and Those methods were proved for the most relevant
histogram. Reflections of ambient light in typical cases for the camera-based supervision of the
office environments have a certain impact in monitor of remote operators and basically as well
correlation of the RGB grey level data from display for in-car camera supervision. However the latter
and camera. This is compensated (as mentioned was not in focus of this work.
above) by using black areas of the screen. The This high resolution optical safeguard method is
captured “luminance” (from grey level calibration) is beyond today’s state-of-the-art. We evaluated our
then subtracted from the extracted values. Figure 8 prototype set-up for supervising the speed, text, tell
is used to explain this procedure: If the gray level of tales, and video reproduction of an in-car camera.
an acquired black area is higher than the measured The latter one is the most safety-relevant topic and
one under dark room conditions (data are stored in challenging in correlation of actual and target data.
the system), this value is subtracted from the
captured image data. The method is part of the overall APV (ASIL
prepared video) framework to facilitate the
functional safety engineering of CMSs. Key pillars
of this framework are a systematic evaluation and
selection of safety mechanisms to detect
malfunctions of CMSs and a generic safety
architecture to structure the implementation of the
selected safety mechanisms.
REFERENCES
[1] A. Terzis (Ed.): Handbook of Camera Monitor
Figure 13. Segmentation of the video image. Systems - The Automotive Mirror-Replace-
ment Technology based on ISO 16505.
Figure 14 demonstrates that the segmentation is Springer 2016.
capable of detecting gamma reproduction distortions.
They could potentially result in loss of essential [2] K. Witt, J. Bauer: A Robust Method for Frozen
details such as potholes or pedestrians with bright Frame Detection in Safety Relevant Video
clothes in front of bright background. Streams Based on Digital Watermarking.
Electronic Display Conference, Nuremberg,
Germany, 2017.
[3] B. Axmann, F. Langner, K. Blankenbach, et al.:
Advanced methods for safe visualization on
automotive displays. J Soc Inf Display. 2020;
28: 483– 498. https://doi.org/10.1002/jsid.909
Figure 14. Even minor gamma distortions (right) [4] ISO 26262:2018, Road Vehicles -- Functional
are detected as “faulty reproduction”. Safety. International Standard, 2018.
Camera-based monitoring provides a reliable and [5] J. Bauer, K. Blankenbach, M. Conrad, et al.:
precise way to capture the optical display output. Neue Ansätze und Methoden für die
Those data are then correlated to the intended image Fehlermodellierung und -behandlung bei
base on interface data. We evaluated the automobilen Videodatenübertragungs-
performance to verify speed, text, tell tales (icons) strecken. 16th Workshop on Automotive
and video content as shown in Figures 11-13 towards Software Engineering @ SE19, Stuttgart,
“light-to-light” supervision (see Figure 2) with good Germany, 2019. http://ceur-ws.org/Vol-
results. This is beyond today’s state-of-the-art safety 2308/ase2019paper01.pdf
methods (supervising only digital data, “data-to- [6] C. Solomon, T. Breckon: Fundamentals of
data”). However there is more to be done towards an Digital Image Processing: A Practical
implementation in certified safety relevant systems Approach with Examples in Matlab. Wiley-
such as monitors of remote operators. Blackwell, 2010.

91
92
 Customized Local Dimming Algorithm and BLU for Automotive Application
towards Low Power Consumption and High Visual Quality
Maxim Schmidt, Ramazan Ayasli, Chihao Xu
Institute of Microelectronics
Saarland University
66123 Saarbrücken, Germany
[email protected]

Abstract: This paper presents a customized local dimming specific automotive requirements and contents, which is
algorithm for automotive application and introduce the based on the well proven SSC (Sorted-Sector-Covering)
crucial parameters of a well-designed BLU with proper algorithm with the unique feature of proper consideration of
LSF. Additionally, features for enhancing and preserving LED crosstalk [3]. As explained before, the number of LEDs
quality will be described which mitigate and overcome is increasing and generates new challenges for the real-time
issues like Halo and failures like LED outage and provide a processing, in addition to a higher memory demand. To
highly valuable LCD module capable to compete against ensure safety operation and real-time processing, no
other display technologies. The validation by using framebuffer will be used by implementing the algorithm
automotive HMI proves high power saving rate and high straight on the pixel pipeline with low hardware cost. Due to
visual quality, while the algorithm cost in IC is low. the quality requirements, several features are introduced in
the later section. Therefore, an amendment of the previously
Keywords: Direct-Lit; Local Dimming; BLU design; published SSC local dimming algorithm is desirable for
Black Level; Boosting; Power Saving; automotive HMI automotive applications.
Amended Local Dimming Algorithm
Motivation and Challenges for Automotive Local For a light spread function (LSF) with high locality, the
Dimming luminance behind a certain pixel is mainly contributed by
The visual quality requirement on automotive displays is
just a few adjacent LEDs 𝐿𝑐 as introduced in [4]. The
getting higher than ever before. To be mentioned are an constraint within the optimization is then modified to
excellent black level and a very high luminance, while at the 𝐿
same time the power consumption has to be reduced in order
to meet low carbon footprint regulation and to extend the min {∑ x(𝑙) : ∑ A𝑚 (𝑖, 𝑗) ∙ x(𝑚) ≥ c(𝑖, 𝑗)}
𝑙=1 𝑚
mileage of electrical cars. LCDs with direct-lit BLUs are
being considered as feasible due to the capability of high with 𝑚 ∈ ℳ𝑖,𝑗
contrast and low power consumption enabled by local where ℳ𝑖,𝑗 is the set with these 𝐿𝑐 LEDs defined for each
dimming (BLU example in Figure 1 with 319 LEDs). Such condensation pixel 𝑐(𝑖, 𝑗). To limit the memory required,
a system can be seen as a competitor against OLED displays, LSF data are stored and processed with a novel model for
which provide a perfect black level, but suffers from low the sake of accuracy and efficiency. In addition, just a
lifetime and aging artifacts like image sticking. selected set of the main contributing LEDs is considered
during the LED calculation, so that the algorithmic
complexity remains modest. Beside this customization, the
light contributions of the adjacent LEDs (crosstalk) are
considered and yield power saving results very close to the
mathematical optimum.

Light-Spread-Function (LSF)
Figure 1. BLU prototype with 319 LEDs [1]. Local dimming performance depends on the algorithm as
Another specific issue is that automotive HMIs have high well as on the BLU. The higher the LED number, the better
contrast and circular contents. Due to this specific content, the result, but at the same time, the higher the cost. However,
the number of LEDs in a BLU shall be high, consequently the BLU design or more specifically, the LSF has strong
the complexity of the local dimming algorithm will rise. influence on the performance [5, 6]. The impact of various
However, it allows a thinner BLU design. shapes on the local dimming performance was analyzed in
[7]. It was shown, that the Full-Width-Half-Maximum
A very simple algorithm, e.g. by just considering multiple
(FWHM) to LED-pitch ratio (𝑟50 ) influences the uniformity
representative values to derive a LED signal as e.g.
and power saving rates substantially.
described in [2] results in a not optimal solution and
sometimes degradation of visual quality. Therefore, the next Figure 2 shows 3 different direct-lit BLUs with their all-on
section will describe a local dimming algorithm considering backlights and their cross-sections. The 2 upper LSFs are

93
radial whereas the third shows a more square-like pattern
which promises a higher uniformity at the borders. The
resulting light contributions (influences) of the LED located
in the center are depicted in Figure 3 for four BLUs.

Figure 4. Locality impact on performance.

Halo Suppression
To mitigate the Halo artifact [8], the algorithm contains now
a two-dimensional spatial filter for LEDs, which can
suppress sudden jump in backlight. For an Edge-Lit BLU, a
Figure 2. Different BLUs (measured with ELDIM one-dimensional filter was introduced and demonstrated in
UMaster). [9]. The new introduced method allows a neighbor 𝐿𝐸𝐷𝑥+1
of the 𝐿𝐸𝐷𝑥 to differ by a filter factor 𝑓𝑥 as shown in the
The maximum value of a curve represents a degree of following formula
locality. The purple line results from a fourth LSF (not
1
shown in Figure 2), which is rather global and yields to a low ⋅ 𝐿𝐸𝐷𝑥+1 < 𝐿𝐸𝐷𝑥 < 𝑓𝑥 ⋅ 𝐿𝐸𝐷𝑥+1
power saving. A significant difference results from the 𝑓𝑥
yellow line. Here, the LSF is square-like and results in a very For the y-direction the constraint is defined in the same way.
fast descend at the transition to the neighboring cavities. In the optimization process, this constraint must be
Such an LSF matches to the amended local dimming as considered to ensure an optimal power saving under
introduced before. fulfillment of this constraint. An example is depicted in
Figure 5. The upper part shows two initial LED values (8
Bit) and after the application of the constraints above. The
lower part shows a cross-section through the resulting
backlight. For lower filter factors, the Halo artifact is
mitigated while the power saving is decreased.

Figure 3. Light contribution of different LSFs.

On the other hand, the risk of introducing artifacts is

increased at the transition between two neighboring LEDs,
if the two LED values strongly differ from each other. Figure
4 shows the impact of an increasing LSF locality. Clearly the
high locality yields a high efficiency but it is more prone to
Halo, less stable and not robust against variation in Figure 5. Function of spatial LED value filtering.
production as investigated in [7]. Therefore, the shape of the
LSF affects the performance and the cost of the system. In Boosting
case of an optimal LSF, fewer LEDs are needed. The capability of boosting LEDs to achieve a more local
Furthermore, it is worth mentioning that 6V LEDs shall be
backlight is provided by the LED driver and lower the
used in order to increase the power efficiency of the LED negative effect of a too wide LSF. The LED current
driver. amplitude has to be increased for the realization by a factor
𝑃𝐵, while the target luminance within the calculation as well
Quality Preserving Features as the final PWM duty-cycle are reduced by 1/𝑃𝐵 to match
In addition to the consideration of the LED crosstalk which the original luminance. This results in a more local solution
yields to optimum power saving, the demand on high visual
of the optimization process so that fewer LEDs are needed
quality remains, especially for the premium OEMs. For this to produce a backlight required by the image given. The
reason, the algorithm considers several automotive related, bright highlights are illuminated, while the black level in the
quality enhancing and preserving features which are surrounding dark areas are enhanced. Overall, the static
introduced in the following.
contrast is increased. In addition, the average PWM duty
94 cycle is inherently limited by 1/𝑃𝐵 by algorithm.
Improving Black Level at Corners interface like SPI. Finally, our local dimming IP allows
The black level at the corners of the display has a significant compensation of several LED outages at the costs of slightly
impact for the perception and thus the visual quality. It is increased power consumption, while the quality degradation
obvious, that smooth transition from black display corners to is compensated to a large extent.
the surrounding dashboard will increase the overall value of
the display and interior design. So, special processing Hardware Design Overview
methods for the corners are adequate. one is the reduction of To overcome the emerging challenges for a direct-lit BLU
the luminance objective with a function 𝑟(𝑑, 𝐴𝑃𝐿), which for automotive applications, the HW implementation differs
depends on the distance 𝑑 to the corner and their APL. substantially from our previous design for Edge-Lit [9] and
Figure 6 demonstrates the function 𝑟 (left) and the impact on includes quality persevering features, introduced in previous
the condenser content (right). Among others, these methods section as well as a novel LSF model to reduce the hardware
result in a significantly improved perception of the display. cost and fulfill the timing related constraints. The overview
of the hardware design is depicted in Figure 8 as a dataflow
block diagram.
The influence of the LSF is modelled and stored in a ROM,
which is accessed by the addressing based on the position of
the LED and the condenser position, calculated in a
pipelined architecture. The blue color marks the read-only
ROMs while the red highlighted parts describe the writable
Figure 6. Effect of condenser manipulation (right registers and SRAM. The logic parts are depicted with a
part). green background. The parts differing from the state-of-the-
The mentioned methods can be combined to achieve an art are particularly the new LSF model to properly consider
excellent performance. Figure 7 shows a measurement with the crosstalk as well as the integration of the quality
the default parameter set and the combination of the methods preserving features. This makes the design to a customized
described above. Finally, these methods result in a solution for automotive displays.
significant improvement of the black level in the corners as
shown in the lower cross section which leverage the display LED Optimizer LSF Model

quality in combination with the interior design to a higher RGB from

Gamma ROM
Pixelorder
Model ROM
Pixel
Source ROM Compensation
level and may give the display a visible differentiation
feature. RGB to
TCON
Corner Model address Degamma
LED calc
Reducer calculation ROM

To LED
Condenser Sample Point Driver
LED Register SPI Interface
SRAM SRAM

Condenser

Boosting LED-Factors Defective LEDs

Quality Features

FPGA

Figure 8. Hardware modules overview with colored

marking for read-only and read-write memory as well
as logic.

System Performance
To overcome the emerging challenges for a direct-lit BLU
for automotive applications, a local dimming processor for
various BLUs, which is placed in the pixel pipeline without
Figure 7. Backlight default vs. applied methods for any framebuffer, was implemented on a Xilinx Kintex325T
black level improvement. FPGA. With the absence of any frame buffer, the SRAM
size is low. It substantially differs from TV local dimming
Safety Mechanisms algorithms and includes particular quality persevering
features required by automotive application. In addition, the
To ensure a safe operation and quality, the safety critical
real-time requirements are fulfilled for the safety critical
outage of few LEDs can be compensated by the algorithm,
usage in an automotive application. For a robust and efficient
which assure an adapted optimal solution by considering
BLU, a 𝑟50 of ≥ 1.5 and a maximal influence of ≈ 30 % are
LED failures. This is done by insertion of an additional
proposed which can suppress and/or mitigate artifacts and
constraint to consider this outage. Then the local dimming
delivers on the other hand excellent local dimming results in
process considers the failure in the LED optimization as well
terms of visual quality and power saving. The resulting cost
in the pixel compensation step accordingly. Typically, the
for an IC implementation e.g. on TCON would lie in few
LED failure is communicated by the LED driver via a serial
95 cents range.
A prototype with 240 LEDs and the algorithm presented in Conclusions
this paper has been validated. The power saving rate for In this paper a local dimming system comprising algorithm
typical automotive HMI images is about 40 %, while the and a direct-lit BLU is presented. The performance or more
overall display quality is significantly increased by the specifically, power saving and visual quality, do depend on
preserving features. Figure 9 shows luminance the algorithm and BLU design. The algorithm presented
measurements captured with an Eldim UMaster colorimeter. considers automotive HMI’s and interior design. Quality
The images show the luminance of the final modulated enhancing/preserving methods are included. Safety
images with backlight and TFT transmission. The upper mechanism is inserted to cover failures like outage of few
both are scaled in the full luminance range. The low LEDs. Optimum LSF for BLU is proposed which allows a
luminance (0-2 nits) is magnified in the lower images uniform backlight, robust production and good local
proving that the black level is significantly improved and dimming results. It is proposed that the BLU design as well
results in a power saving rate of 40.4%. Such a device may as the customization of the algorithm shall be jointly
be perceived as a high value display. developed. Then such a LC display module can produce
excellent image quality at high power efficiency, while the
high and tough automotive requirement in terms of high
brightness and high lifetime can be met. It will increase the
value of displays and make displays to a differentiation
feature in cars.
References
[1] C. Xu, M. Schmidt, T. Lahr und M. Weber, „Dynamic
Backlights for Automotive LCDs,“ Information
Display, p. 14, July/Aug 2018.
[2] W. Zhang, Min.Chen, W. Niu und D. Huang, „LED
Control signal extraction by using Multiple
Representative Values,“ IDW, pp. 1519-1522, 2009.
[3] C. Xu, A. Karrenbauer and M. Albrecht, "Method,
Figure 9. Luminance measurements with dimming. system and apparatus for power saving backlight".
In this paper a further quality preserving feature was USA Patent US8421741, 11 September 2009.
introduced, namely the handling of the outages of LED [4] M. Schmidt, M. Grüning, D. Schäfer and C. Xu,
devices. Figure 10 shows a luminance measurement with the "Efficient Modeling of LED Crosstalk of a Matrix
outage of one LED (red circle). On the left, the outage is Backlight Unit," Proceedings of IDW'17, pp. 1457-
ignored, while on the right, specific constraints during the 1460, December 2017.
LED calculation are set to compensate the negative impact [5] M. Schmidt, M. Grüning, J. Ritter, A. Hudak and C.
of the loss of a light source. The improvement of the quality Xu, "Impact of high-resolution matrix backlight on
is clearly visible in the underneath cross-section (along the local-dimming performance and its characterization,"
black dotted lines). The resulting luminance with the enabled Journal of Information Display, 2019.
preserving feature (magenta) demonstrates a much brighter
luminance around the failed LED (row 0 – 250). In this case [6] Y. Cheng, Y. Lu, C. Tien and H. Shieh, "Design and
the luminance reaches > 85% of the target value and Evaluation of Light Spread Function for Area-
substantially compensates the deficit caused by the failed Adaptive LCD System," Journal of Display
LED. In case of several failed LEDs, the effect of the safety Technology, vol. 5, no. 2, pp. 66-71, 2009.
mechanism will be even stronger. Therefore, this is a crucial [7] M. Schmidt, J. Ritter und C. Xu, „Optimizing LSF
feature to safeguard the display operation and achieve a high Shape for Robust and Uniform Backlighting of
automotive safety integrity level. Automotive Displays with Direct-Lit Local-
Dimming,“ Proceedings of IDW, pp. 1554-1557,
November 2019.
[8] H. Ichioka, K. Otoi, K. Fujiwara, K. Hashimoto, H.
Murakami and T. Yamamoto, "Proposal of Evaluation
Method for Local-Dimming Backlights," SID
DIGEST, pp. 750-753, 2010.
[9] M. Grüning, M. Schmidt, D. Schäfer und C. Xu,
„Challenges and Methods for Local Dimming of
Long-Edge LCD TVs,“ IDW, pp. 1339-1341, 2016.

Figure 10. Breakdown of one LED. Left: no quality

preserving. Right: proper handling.
96
 Automotive smart surfaces: Conformable HDR displays
and smart windows to activate almost any surface

J. Huggins
FlexEnable Ltd.
34 Cambridge Science Park, Cambridge CB4 0FX, UK.
[email protected]

Abstract FlexEnable has developed a process for OLCD

Increased vehicle automation, connectivity and manufacture that uses organic TFTs replacing
safety require more and larger displays which conventional a-Si. The process steps for these organic
conform to the curved surfaces of the car. A new OLCD, including the array, CFA and cell assembly, are
glass-free display technology - OLCD - meets these performed at low temperatures (< 100°C), and can be
requirements by using the traditional LCD implemented on existing a:Si TFT LCD production line
architecture which is proven for automotive-grade equipment sets.
reliability and brightness, with an organic TFT
backplane built on an ultra-thin plastic substrate. As
well as allowing flexibility, the ultra-thin substrate
brings performance advantages to dual cell HDR
displays with contrast of around 1,000,000:1, whilst
retaining the conformability required for automotive.
Beyond OLCD, flexible liquid crystal cells allow other
surfaces in the car to be activated, including colour-
neutral, rapidly switchable smart window films that
can be biaxially conformed to the glazing.

Keywords: LCD; OLCD; OLED; Flexible Display; Dual Figure 1. An S-shaped, touch-enabled centre unit
Cell, Smart Windows consisting of twin 12.1” OLCDs. Image source:
Novares
Introduction
As car makers look ahead to a future of increasing The low processing temperatures enable the use of TAC
automation, so the focus in automotive moves from the (triacetate cellulose) film to be used as the substrate, which
exterior to the interior of the vehicle. This change demands possesses critical optical benefits required for LCD,
greater functionality with an increased density of displays, including low haze and zero birefringence. TAC is already
yet there are few or no flat surfaces on which to integrate commonly available as the substrate used in automotive
flat displays without imposing undesirable deisgn polarizers and offers film thicknesses 10x less than glass
constraints. used for large area displays. Throughout the overall OLCD
manufacturing process, many other standard materials and
Flexible displays offer a solution, with conformability processes that are already qualified for automotive glass-
enabling better integration into the curved surfaces in the based LCDs are used.
car interior. Curved displays can be located into tight
interior spaces without disrupting interior ergonomics. One
flexible display technology that is being increasingly
adopted in small area consumer electronics is flexible
OLED. However, this has limitations in automotive
applications with very high manufacturing costs at large
area, and a fundamental trade-off in brightness-lifetime.

Organic LCDs (OLCD) technology is designed to cost

effectively meet the area scalability and reliability-
brightness requirements for automotive by utilizing the
established technology of LCD while enabling Figure 2. Schematic OLCD cross section using TAC
conformability of flexible substrate. substrate in place of glass

97
Figure 2 shows a schematic cross section of a typical Nevertheless, the dual cell approach using conventional
OLCD single cell structure. The OTFT array is built glass LCD does introduce some trade-offs; both mechanical
directly onto a 40μm TAC film substrate which is itself (thickness, weight, lack of flexibility) and optical (moiré,
mounted onto a glass carrier during fabrication. The stack wide angle performance). All of these trade-offs are a
consists of a top-gate OTFT structure connected to a pixel direct result of the large inter-cell separation caused by the
electrode for a conventional IPS pixel structure. The cell thickness of the display glass. These optical defects can be
shown in figure 2 has a thickness of <100μm and OLCD reduced using optical compensation film diffusion layers
modules incorporating this cell have been demonstrated between the two cells. However the addition of
with a bend radius down to R10. compensation means that true pixel-level dimming is not
possible and an additional polarizer is needed. The
Applications utilizing conformable displays have been additional polarizers and compensation films reduce overall
demonstrated in both curved centre console units and transmission requiring increased backlight brightness.
curved digital side mirrors1.
The ultra-thin structure of OLCD provides a breakthrough
OLCD displays and new smart window technology offer approach to dual-cell with the opportunity to reduce the
the possibility to activate almost any surface in the car inter-cell spacing to less than the pixel pitch. By utilising
interior, providing new design aesthetics for existing use the TAC substrate at ~40um thickness and a conventional
cases (center console), as well as many new use cases polarizer film inter-cell spacing in the order of 300μm is
(invisible A-Pillar). The concept in Figure 3. shows the possible resulting in true pixel level dimming for dual cell.
range of potential applications using OLCD displays. All of this can be achieved while maintaining the flexibility
and conformability of OLCD with the whole dual cell stack
thickness within the range of a single sheet of FPD glass.
By eliminating the need for further optical compensation,
OLCD offers the added benefit of improved transmission
which reduces the BLU brightness requirements relative to
glass dual-cell displays.

Figure 3. FlexEnable concept demonstrating

applications for conformable OLCD and smart
windows

Ultra-high Contrast HDR via Dual Cell OLCD

Dual cell LCDs are in production today for TVs and
specialist monitors, and combine a standard LCD display
with a second ‘light modulating’ cell2 that allows the
backlight to be locally dimmed. Such displays incorporate
4 glass substrsates (2 per cell) instead of 2, and the
additional modulation of the backlight illumination
provides a performance improvement over conventional
LCD resulting in an increase of CR from >1000:1 to Figure 4: (a) Conventional Glass Dual Cell display
>1,000,000:1. Display black levels similar to OLED TVs compared to (b) FlexEnable Dual Cell using TAC
are achievable, with contrast values within VESA Display substrate – as used in test
HDR 1000 specifications3. Very high contrast displays are
strongly desired for automotive (as well as TVs, Monitors),
where light leakage reduction particularly for night driving
allows displays to be hidden until lit.

98
FlexEnable have demonstrated the capability of dual cell and alignment layers on the frontplane and backplane, with
OLCD using two monochrome IPS OLCD cells with a liquid crystal layer with the cell height defined by a
monochrome pixels at a pitch of 195μm. A stack using spacer unit.
off-the-shelf polarizers was assembled with an inter-cell
separation of approximately 300μm (this can be reduced
significantly further via design optimisations). A schematic
diagram is shown in Figure 4. Optical measurement of this
stack comparing both single and dual cells with pixels set
to open and closed demonstrate this OLCD approach is
capable of contrast of 266,000:1. Figure 7. Schematic of an LC Cell fabricated using
TAC substrate

This structure can be manufactured using the same supply

chain as OLCD using existing FPD manufacturing lines
enabling large area, low cost, high volume manufacturing.

Conclusions
OLCD demonstrates the conformability, reliability and low
cost capabilities required to deliver the increasing demand
for large area flexible displays in automotive applications,
off. OLCD also offers a unique route to HDR LCD for
automotive that retains the flexibility requirement, as a
result of the 10X reduction in inter-cell separation.
Figure 5. Optical micrograph and measured contrast
values for FlexEnable Dual cell test structures. The LC cell technology on TAC substrate developed for
OLCD has additional applications in automotive including
LC Cells for Automotive Smart Windows switchable smart windows where the biaxial forming of the
Smart windows offer increased privacy, the opportunity to cell is ideally suited to automotive glazing.
change the environment within the car and improve energy
efficiency with less energy required for climate control due References
to reduction in light transmission. Liquid crystal cells offer
the optical performance required (fast switching, 1. https://www.flexenable.com/newsroom/novares-
colourless, and low haze) and can be built using existing integrates-flexenables-conformable-olcds-into-its-
automotive-qualified materials from the LCD supply chain, new-demo-car-nova-car-2/
but the lack of biaxial conformability means glass LC cells 2. https://news.panasonic.com/global/press/data/201
cannot be subsequently laminated to automotive glazing. 6/11/en161128-4/en161128-4.html
FlexEnables’s LC cell approach is well suited to biaxial
curvature, because the TAC substrate employed instead of 3. https://displayhdr.org/performance-criteria-cts1-1
glass is a thermoplastic capable of repeat forming after
assembly of the LC cell. 4. Cain P, Harding J, Reeves W, Wheeler M. Ultra-
high contrast OLCD: Thin and light dual cell LCDs
Figure 7 shows the schematic cross section of the LC Cell on TAC film. Proc. Of IDW’19, pp287-290; 2019
stack. The LC cell structure consists of a the same TAC
substrate as the OLCD platform with patterned electrodes

Figure 6: Concept of a smart window using Flexible liquid crystal cells. Image source: FlexEnable

99
100
 Presentation 5.4

The Functional Safety Designs of Vehicle Display Driver ICs

Cheng‐Chih Deno Hsu, Himax Technologies, Hsinchu City, Taiwan

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

101
102
 Automotive Dual Cell microZone™LCD Development
Paul Weindorf, Qais Sharif, Elijah Auger, David Whitton, Brian Hayden
Visteon Corporation, Van Buren Twp., MI, USA

Abstract LCD displays can easily operate at >1200 cd/m2 since the LCD is
Consumer experience with high performing mobile device simply a light valve to a backlight that can be thermally managed.
displays sets high expectations for the automotive market in Although TFT LCD technology has been successfully utilized in
contrast, color performance and brightness. OLED based the automotive industry, the black background light leakage as
displays have not been able to successfully penetrate the depicted in Figure 1 has encouraged the industry for a better
automotive display market primarily due to image burn at high looking black background that does not have the problems
luminance operation. The microZone™ LCD is an alternative associated with black mura [5].
display configuration that can meet the desirable properties of
OLED displays while providing high luminance at high
temperature operation without image burn in.
Author Keywords
OLED; dual; cell; high dynamic range; HDR; wide color gamut;
high contrast; high brightness
1. Introduction
Since the introduction of OLED display technology, the
automotive industry has desired the development of automotive
grade OLEDs. OLEDs have the desirable property of high
contrast ratios on the order of 1M:1 and therefore exhibit a
perfectly black background in comparison to typical thin-film
transistor (TFT) LCDs with contrast ratios around 1500:1 which
exhibit a certain amount of visible black leakage that is visible
under night time viewing conditions. OLEDs also exhibit highly
saturated colors for a great viewing experience. However since
OLEDs are an emissive display technology, image burn-in
artifacts will always occur since pixels that are illuminated for
long periods of time at elevated temperatures will experience a
natural degradation in luminance and color [1,2]. Unlike other Figure 1. Typical LCD Light Leakage Background
disposable commercial display products, the automotive display
must be able to display the same image for extended times and not In order to address the desire for higher contrast ratios with black
exhibit a permanent burned-in image when an alternative display backgrounds, LCD suppliers have proposed the use of local
presentation is selected. In addition, the automotive display must dimming zone based backlights. In a local dimming backlight as
continue to operate satisfactorily for the life of the vehicle (>5000 shown in Figure 2, the LEDs are not turned on in areas where a
hours) in order to avoid warranty returns and poor customer true black background is desired. The number of backlight LED
perception. Elaborate countermeasures continue to be developed zones may be on the order of 96-394 for packaged LEDs and
to address the image burn-in problem [3]. These countermeasures extend to the idea of mini-LED zones on the order of millimeter
can be either at the pixel level or at the macro level with the use of zones.
luminance consumption preservation methods [4]. The pixel level
compensation techniques generally reduce the luminance of all the
pixels to the lowest burned in pixel level, but color compensation
has yet to be realized. OEMs are reluctantly relaxing
specifications for image burn-in, operational temperature, and
luminance requirements in order to allow the use of OLED
displays which have high tooling and recurring costs compared to
TFT LCD technology.
In comparison, TFT LCDs have been successfully utilized by the
millions in automotive applications because a LCD is based on the
light valve principle where the selected pixels either transmit light
or block light from a uniform backlight. As the LEDs in the
backlight degrade over time, all of the LCD pixels decline
uniformly, but differential image burn-in does not occur for “on”
pixels compared to “off” pixels. Inorganic backlight LEDs are
much more resistant to degradation under high temperature
operation compared to organic LEDs and therefore LCDs can
maintain a higher luminance operation under high temperature
conditions. Currently OLEDs are having difficulties obtaining
600-800 cd/m2 for extended periods of operation whereas TFT Figure 2. Local Dimming Backlight

103
Although there has been some adoption of the local dimming power due to the transmission of the two cells and the inter-cell
backlight in the automotive industry, the problems of driving a diffuser. In order to make an automotive viable dual cell display,
large matrix of LEDs with luminance control has discouraged several efficiency improvements were made:
wide adoption of this technology. In addition, local dimming
backlights have generally been thicker and the image experiences  Modification of the inter-cell diffuser
rather large grayish halos due to the size of the LED zones as  Use of a proprietary light control method
shown in Figure 3.  Backlight efficiency improvements

As discussed in reference [7], a diffuser is generally utilized to

eliminate the Moiré effect due to the periodic structure the two
LCDs. The diffuser is generally a polarization dependent
scattering film (PDSF) that scatters light only in one polarization
axis. However the diffuser film has a transmission decrease to
70% and therefore deleteriously decreases the system efficiency.
Figure 3. Local Dimming Zone Halo Depiction The µZLCD™ approach modifies the diffuser configuration by
using a proprietary technique to eliminate Moiré which improves
the system efficiency. Although the Moiré effect is eliminated
The complexity of driving all of the LEDs has led to exotic and
with this proprietary method, the diffuser function of also
costly electronic signage based multiplexing techniques together
smoothing the edges of the mono-cell image needs to be
with a specialized FPGA or ASIC [6] for the zone control as a
considered so that the observer does not focus on the mono-cell
function of the dynamic video input.
image. The µZLCD™ system solves this problem by using the
Due to these afore mentioned deficiencies associated with OLEDs gray scale function of the display to smooth the edges as shown in
and local dimming LCD approaches, an alternative dual cell Figure 5 which requires the use of an FPGA or similar device and
approach is proposed. a specialized image processing algorithm.
2. Dual Cell microZone™LCD
A highly efficient dual cell technology called microZone™LCD
(µZLCD™) has been developed. The µZLCD™ display as shown
in Figure 4 is based on luminance control at a micro-meter scale
by using a monochrome TFT for the rear backlight light valve
control. Since TFTs have a highly efficient and mature control
method, the small µZones may be easily driven and include gray
scale control.

Figure 5. Edge Smoothing Example of Mono-Cell

The monochrome gray scale levels may also be utilized to

increase the dynamic range of the system. In particular, the
monochrome gray scale level is dynamically controlled as a
function of the color display gray shade level which effectively
eliminates the color cell leakage component as shown in Figure 6,
thus providing high dynamic range (HDR) performance without
image artifacts.

Figure 4. Dual Cell microZone™LCD

Although the concept of using two cells has been purported in the
industry for some time, the approach is generally not considered Figure 6. Before and After Algorithm Comparison for Lower
viable for automotive applications due to the low transmission and White Gray Shades
attending high backlight power associated with the use of two
LCD cells. Typically, dual cell approaches require up to 2.5x the

104
3. Color Gamut Improvement Measurements utilizing a Radiant Vision Systems ProMetric I16
In order to better compete with highly color saturated OLED imaging colorimeter reveals excellent iso-contrast performance
displays, the color performance was improved for the µZLCD™ per Figure 9 while Figure 10 shows more numerical results in the
to obtain 96% NTSC compliance. The color performance was x and y axes. In the horizontal viewing directions up to 60° (limit
improved by: of colorimeter), excellent contrast ratios greater than 10,000:1 are
maintained.
 Using improved TFT color filters
 Using LEDs with improved green phosphor performance
As is known in the LCD industry, the major problem with
improving the color performance is associated with the green-blue
overlap of the color filter spectral transmission curves. Figure 7
shows what can be obtained by optimizing the color filter and
LED selection. Figure 8 shows that the µZLCD™ is fully
compliant to the Digital Cinema Initiatives DCI-P3 color gamut.

Figure 9. Iso-Contrast Performance

Figure 7. NTSC Color Performance

Figure 10. Horizontal and Vertical Contrast Ratio Performance

One of the benefits of using dual cell technology is the visual
improvement in black mura performance which has historically
been a challenge in automotive TFT LCD applications. Figure 11
shows that back area scan uniformity performance of the dual cell
display.

Figure 8. DCI-P3 Color Performance

It is expected that improvements in the color filter and light

emitter technologies will improve the color performance beyond
what OLEDs are capable of.
Figure 11. Area Scan Black Mura Performance [5]

105
Although the measured uniformity performance is only 59% per In summary, the following attributes have been obtained:
the GOEM specification [5], the black luminance range of 0.001
 Ultra High Contrast Ratio – 100,000:1
to 0.017 cd/m2 is near the bottom of the mesopic range and is
impossible to see under day time lighting conditions. Under  Low Power with High Color Gamut – Proprietary optical
nighttime conditions with graphical component image luminances stack and backlight configuration provides an 850 cd/m2 at
in the 10 cd/m2 range, the black non-uniformity is also not visible. about 9.7 Watts of LED power for a DCI P3 compliant
The black non-uniformity is only visible under dark room NTSC performance of 96%.
conditions with an all-black screen with no graphical image.  High Brightness – Higher luminance values from 850 to
4. LCD Configuration Power 1300 cd/m2with no image burn-in
Since power dissipation is the critical parameter for the  Moiré – None
µZLCD™, it is important to compare the power dissipation for
 Black Mura Performance – None observable
the various color configurations to a commonly used 85% NTSC
typical 12.3 inch display. Table 1 outlines the power comparison  High Dynamic Range Performance
depending on the desired NTSC level and shows that at the 85%
NTSC level there is only a modest increase in power from 7.2W
for a typical single cell approach to 8.7W for the µZLCD™ dual 6. References
cell approach. Various combinations of color filter types and [1] P. Weindorf, Automotive OLED Life Prediction
LED types may be utilized to obtain different levels of NTSC Method, Detroit SID 2005 Proceedings.
performance.
[2] P. Weindorf, D. Andres, J. Hatfield, Automotive
Table 1. Power Comparison Summary OLED Life Test and Prediction, Detroit SID 2013
Size Configuration cd/m2 Color NTSC Power Proceedings
Filter [3] Fan R, Zhang X, Tu Z. Influence of ambient
Type temperature on OLED lifetime and uniformity based
on modified equivalent lifetime detection. J Soc Inf
12.3” Single Cell 850 Old 85% 7.2W
Display. 2019;27:597-607.
12.3” µZLCD™ 850 Old 85% 8.7W https://doi.org/10.1002/jsid.788
12.3” µZLCD™ 850 New 96% 9.3W [4] P. Weindorf, Automotive OLED Luminance
12.3” µZLCD™ 850 New 110% 12.9W Consumption Control Methods, SID Symposium
May 22–27 2016, Paper 36.1, DOI 10.1002/sdtp.
10706
5. microZone™LCD Summary
The development of a highly efficient dual cell µZLCD™ will [5] ©German Automotive OEM Work Group Displays,
enable the next generation of automotive displays. OLED-like BlackMURA_Display_V115.doc/14.06.2011
performance without an image burn-in may be realized. It is [6] Himax HX8880 Data Sheet, TFT LCD Timing
important to note that the µZLCD™ is not limited in luminance controller with LVDS/Display Port input and
whereas OLEDs are limited and exhibit greater image burn-in as
LVDS/mini-LVDS output
the luminance is increased.
Figure 12 shows the working of the first prototype. [7] Haiwei Chen et al., High Dynamic Range LCD with
Pixel-level Local Dimming, SID 2017 Digest ISSN
0097-996X/17/4702-0890

Figure 12. Initial microZone™LCD Prototype

106
 A Low-power Transflective TFT-LCD Based On IGZO TFT
Tenggang Lou, Lei Wang, Xiangjian Kong, Jine Liu, Feng Qin
TIANMA MICRO-ELECTRONICS GROUP, Shanghai, China

Abstract We use bottom gate IGZO TFT to make the transflective TFT-
We have developed a transflective TFT-LCD device based on LCD. In order to enlarge the reflective area, com electrode or
IGZO TFT(thin film transistor) using GIP and Demux driving pixel electrode was placed right above the IGZO TFT. This can
circuit. The average flicker is -62.35dB when in reflective mode deteriorate the TFT performance especially the reliability
driving in 1Hz while -55dB in transmissive mode driving in performance. So the film forming process is improved to
60Hz. The flicker and the visual effect still performance well minimize the impact imposed by the electrode above the TFT.
after RA(reliability analysis) test. The flicker after RA test We use some methods to improve the GIP performance
almost unchanged comparing with the LCD without RA test. The especially the reliability performance. The source loading was
average contrast ratio is 24.9 in reflective mode while 33.4 in well designed to suitable for the demux driving condition. Also
transmissive mode. The power consumption average reduced by the materials used in LCD cell is carefully choosed.
40% in 1Hz reflective mode contrasting to 60Hz in transmissive
mode while in black, white, red, green, blue, grey, flicker,
crosstalk, checkpixel image. We proved the power consumption
can be reduced by even more if IC is well designed. The
transflective LCD device supports 256 grey.
Author Keywords
Low power LCD; low frequency driving; reflective LCD;
transmissive LCD; Transflective LCD; Gate in panel (GIP);
Demux; IGZO;

1. Introduction
Low power consumption LCD devices have longer usage time.
This attribute is meaningful to those devices with battery
powered. Such as sports watch, electronic tag and so on. To
achieve the low power consumption target, many methods can
be used. On one side, use low resistivity metal materials or
increase the metal thickness. On the other side, use low Figure 1. Schematic drawing of panel design with GIP and
dielectric coefficient insulation layers or thick organic coating to demux circuit
decrease the parasitic capacity. Also we can use special thin film
transistor to reduce the TFT’s parasitic capacity or use high 3. Performances of the transflective TFT-LCD
mobility semiconductor materials to resize the TFT dimension.
device
Reduce the driving frequency is also a good method to lower the
power consumption. Driving LCD with low frequency has many
difficulties. The one is human eyes may be more sensitivities to
brightness variations in low frequency which can cause flicker
problems. The other is imaging sticking may deterioration. Also
TFT reliability can be worsen lead to reliability problems.
IGZO for its insensitivity to temperature, adequate mobility and
low leakage current[1] is suitable for low frequency driving
devices. We fabricate a transflective TFT-LCD device based on
IGZO TFT. It can driving in 1Hz to 60Hz alternatively and has
256 gray scale image.
2. Basic information of the LCD device
Figure 1 shows a schematic of the device. With a round active
area(AA). GIP circuit is designed on the left and right of AA.
The demux circuit is placed on the bottom of AA. The resolution
is 240RGB*240 and the AA size is 1.2 inch. The demux is 1:6
designed which is 1 IC output pad corresponding with 6 sources
in AA.
Using GIP and demux we can reduce the border of LCD and Figure 2. Flicker value in reflective mode and in
driving in low frequency can reduce the power consumption, so transmissive mode
its meaningful to verify whether the GIP and demux circuit
performance well in low frequency, high temperature and high Figure 2 shows 8 samples was tested to see the flicker
humidity condition. performance. The flicker is tested in flicker image. We use DMS
to check the flicker in reflective mode and use CA310 to test the

107
flicker in transmissive mode. The driving frequency is 1Hz in For human eyes are sensitivity to brightness variation in low
reflective mode while 60Hz in transmissive mode. In reflective frequency, also the TFT can be worsen because of longer
mode the backlight is power off and the backlight is power on in voltage bias applied on the TFT in 1Hz driving mode. So we
transmissive mode. The flicker is smaller than -60dB in tested the 1Hz reflective mode samples after RA test to see if it
reflective mode and smaller than -50dB in transmissve. is deteriation. Table 2 shows the flicker after RA condition
According to the tested value of flicker we can see the “70°C, 240H, operation, 1Hz” and “60°C/90RH, 240H,
transflective TFT-LCD performance well both in 1Hz mode and operation, 1Hz”. We tested 8 samples separately for each
in 60Hz mode. condition. We can see that the LCD performance well even after
240 hours RA test for the flicker is below than -47dB after 70°C
Table 1. Reliability performance of the transflective TFT- RA condition and 60°C/90RH RA condition. The flicker is
LCD with GIP and demux circuit almost the same as the LCD without RA test. This indicates the
RA(reliability analysis) test Sample
Result
Driving transflective LCD device we fabricated based on IGZO is
Conditions quantity frequency reliable enough to long term use.
80°C, 240H,storage 4 Pass / Table 3 shows the contrast ratio value of the device in reflective
-40°C, 240H,storage 4 Pass / mode and the transmissive mode. The value is acquired by DMS
in reflective mode and SR-3A in transmissive mode. The
70°C, 240H,operation 8 Pass 60Hz
contrast ratio is beyond 23 in reflective mode while 29 in
-20°C, 240H,operation 8 Pass 60Hz transmissive mode. The contrast ratio is enough for human read
60°C 90%RH, 240H,operation 8 Pass 60Hz information on the LCD under sunshine or in the dark
environment.
70°C, 240H,operaion 8 Pass 1Hz
-20°C, 240H,operation 8 Pass 1Hz Table 4. Imaging sticking performance after 1, 2, 4, 8, 24
60°C 90%RH, 240H,operation 8 Pass 1Hz hours
release
-40°C 30min~80°C 30min 100cycle 4 Pass / time 1hour 2hour 4hour 8hour 24hour
30min

Table 1 shows the reliability analysis results. The reflective

Sample Level Level Level Level Level Level
mode and the transmissive mode were tested respectively. The 21 0 0 0 0 2.5 0
total tested time is 240 hours for each RA test condition. In
reflective mode the driving frequency is 1Hz and the backlight is
power off. In transmissive mode the driving frequency is 60Hz Sample Level Level Level Level Level Level
and the backlight is power on. In storage RA test condition and 22 2.5 2.5 2.5 2.5 2.5 0
the thermal shock RA test condition, the LCD is power off. The
RA test results show that after 240H RA test, the LCD remains
performance well.
Table 2. Flicker value after 70°C and 60°C/90RH 240
hours RA test in 1Hz driving frequency
flicker after RA test(dB)

70°C,240H, 60°C /90RH,240H,

RA test conditon
operation,1Hz operation,1Hz

sample 1&2 -59.27 -47.65

sample 3&4 -47.8 -55.15

sample 5&6 -51.2 -53.14

sample 7&8 -61.89 -52.18 (a) (b)

sample 9&10 -58.74 -53.41

sample 11&12 -47.09 -56.55

Sample1 13&14 -51.42 -51.09

sample 15&16 -62.34 -49.79

Table 3. Contrast ratio test

Contrast ratio

Sample Sample Sample Sample

test condition
17 18 19 20

reflective mode 1Hz 26.65 23.27 26.47 23.11

(c) (d)

transmissive mode 60Hz 29.96 36.68 32.52 34.63

Figure 3. Screen refresh image problem

108
Table 4 shows the imaging sticking performance in reflective The absolute power consumption in white image is 2mW in
mode. We tested 2 samples and check the imaging sticking 51Hz mode while 0.26mW in 1Hz mode. Figure 4 shows the
performance immediately after 1, 2, 4, 8, 24 hours. Release Demo that is pin to pin designed based on IGZO TFT.
30min in the table 4 means we closed the LCD power 30 Figure 5 shows the image of the Transflective TFT-LCD with
minutes. The imaging sticking pattern is chess pattern. Level 0 GIP and demux circuit. For the case of reduce the cost of demo,
means no imaging sticking can be seen in any viewing angle. the CF(color filter) mask is not matching well with the Array
Level 1 means no image sticking under vertical viewing angle. mask. So you can see yellow border on the left and right side of
Level 2 means slightly image sticking under vertical viewing the LCD where there is no BM(black matric) and PS(photo
angle. Level 3 means LCD has image sticking problem and the spacer). But it has no influence with the conclusion of the
image sticking boundary is unsharpness. Level 2.5 means the evaluation of our transflective TFT-LCD.
imaging sticking performance is between level 2 and level 3. We
checked the imaging sticking after 1 hour, 2 hour, 4 hour, 8 hour
and 24 hour. Although in the worst case is the imaging sticking
level can reaches level 2.5, but after release 30min the imaging
sticking can’t be seen by human eyes in any viewing angle.
Figure 3 shows the screen refresh problem which often occurs in
low frequency driving condition such as driving frequency is
1Hz, because the brightness of the positive frame is different
from the negative frame for the cause of charge ratio, the hold
ratio or the voltage is different form positive frame to negative
frame. And in 1Hz frame rate the human is more sensitivity to
brightness, so the screen refresh problem occurs. Figure 3(a)
shows the positive frame. Figure 3(b) shows the negative frame
is refreshed 1/3 image. Figure 3(c) shows the negative frame is
refreshed 2/3 image. Figure 3(d) shows the negative frame
image. If there are screen refresh problem or not is one of the
key factor to evaluate the LCD device while driving in low
frequency.
By means of improve the film process, enlarge the capacity and
choose the right cell materials, we finally fabricated the
transflective TFT-LCD device with no screen refresh problem in
all grey images from 0 grey to 255 grey even after RA test. Figure 4. Image of the Demo with pin to pin design based
on IGZO TFT
Table 5. Power consumption in various image in 1Hz,
5Hz, 60Hz

Frequency Black White R/G/B Grey Flicker Crosstalk

\Image (mW) (mW) (mW) (mW) (mW) (mW)

1Hz 34.28 33.66 34.64 33.86 34.19 33.85

5Hz 35.51 35.24 36.17 35.73 35.69 35.55

60Hz 54.34 56.18 56.16 60.17 57.37 58.91

Table 5 shows the power consumption of the demo with GIP

and demux circuit. The power consumption in 1Hz driving
frequency is reduced by 36.9%~42.5% in contrast to 60Hz. The
power consumption in 5Hz driving frequency is reduced by
34.7%~39.7% in contrast to 60Hz. Although the absolute power
consumption is high, we due this to the IC is not well designed
for our panel. Such as the IC’s minimum supported resolution is
720P, but our LCD’s resolution is 240P. We insert dummy black
image to let the LCD shows 240P image. Meanwhile the IC is
not well designed for low frequency driving condition, we
assume that if IC is well designed for low frequency, the power
Figure 5. Image of the Demo with GIP and demux circuit
consumption in 1Hz can be reduced 80% in contrast to 60Hz.
based on IGZO TFT (transmissive mode)
In order to confirm our guess, we fabricated a pin to pin
designed transflective LCD based on IGZO TFT that is without
GIP and demux. Pin to pin design means the gates and sources 4. Conclusion
in LCD were directly connected one by one to the IC. The IC is We fabricated a transflective TFT-LCD based on IGZO TFT.
well designed for low frequency driving. We tested the average GIP and Demux were integrated in the LCD. The device can
power consumption reduced by 87% in 1Hz in contrast to 51Hz. operate in 1Hz or 60Hz alternatively. The tested results prohibit

109
that the device can provide a good quality both in low frequency
and in high frequency.
5. References
[1] T. Tanabe, K. Kusunoki, Y. Sekine, K. Furutani,
T. Murakawa, T. Nishi, Y. Hirakata, H. Godo, J. Koyama,
S. Yamazaki, Proc. AM-FPD ’11 Digest, 109-112(2011).

110
 Presentation 5.7

A MicroLED Device with 0mm Border

TengGang Lou, Tianma Micro-Electronics Group, Shanghai, China

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

111
112
 Presentation 5.8

Enabling Features of VueReal MicroLED Technology

for Automotive Applications
Rexa Chaji, VueReal Inc, Waterloo, Ontario, Canada

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

113
114
New challenges and testing solutions for flexible vehicle displays & interfaces

Title: New Flexible Display Challenges, New Reliability Solutions

Presenter: Eisuke Tsuyuzaki
Company: Bayflex Solutions
Topic: Reliability

The rapid transition of new vehicle drivetrains has enabled automobile manufacturers to reimagine safety, comfort and entertainment in mobility. More recently, we have already begun to see lightweight
flexible devices incorporated into mainstream models such as LED lighting, flexible sensors and heated fabrics for seating. In addition, we are now beginning to see large, rollable flexible displays from concept
to realization by OLED based technologies.
From a product design and testing perspective, the challenge is that these new form factors often utilize curved or fully flexible systems that integrate a mixture of different devices, processing techniques,
and packaging and assembly approaches. Legacy testing is susceptible to both failure and performance variations under operating conditions, a new testing regime is needed.
Specifically, product designers must integrate and ensure continuous performance for new materials, curved screens, new metal ink compounds, ultra‐thin and flexible connectors, cables, bendable antennas,
and other electronic components. Testing of rigid devices only dealt with initial evaluation and conditions under which failure might occur due to breakage.
As these new flexible devices can undergo performance changes under different physical conditions (rolling, flexing, warping) when exposed to different environmental conditions (temperature, humidity),
and over periods of time and repetitive motion, testing approaches are needed to characterize performance under real‐world conditions.
Bayflex Solutions, working with equipment partners, has developed several new approaches. First, Bayflex Solutions has integrated a family of interchangeable mechanical endurance testing solutions for
both ambient and hostile conditions from Yuasa Systems that simulate the mechanical‐physical motions for flexing of components and cables, reliably and repetitively based upon extensive experience
in smartphone, tablets and laptop testing.
Second, Bayflex Solutions has integrated resistance and temperature measurements during endurance testing through electrical data capture, with a real‐time collaboration and analytics platform.
Furthermore, as discussed at SID2020, a combined mechano‐optical platform with careful camera position enables for example, real‐time edge strain analysis for early multi‐layer coating delamination
detection. This approach enables our customers to observe, monitor, analyze, and even predict component failure.
The combination of mechanical testing equipment, data collection, and analytics and reliability data provides unique insights that can be applied to component research, product design, and volume
manufacturing and deployment for new innovative vehicle display device systems & interfaces.

BAYFLEX SOLUTIONS
New challenges & testing
solutions for flexible displays
& interfaces

115
About us

• Experts in Mechanical systems manufacturing crankshafts since 1947

• Factory automation for transmission, engine blocks, differential gear assembly since 1995
• Initial large endurance systems for Harnesses and Cables & industrial applications
• Medium size endurance systems for semi-rigid & curved LCD components since 2009
• Awarded “Flexi” for mechanical testing in hostile environment
• Successfully meeting industry transition to flexible, rollable, wearable OLED form factors
incl. Consumer Electronics Display ecosystems for smartphones, tablets, wearables, laptops

Problem

• Previously failure analysis was performed on Universal tester units

• Flexible components & products require repeatable real-world mechanical tests
• Use of counterweights leads to imperfect mechanical test results
• Printed traces, films, conductive materials, connectors, display modules
• Front windscreens, Entertainment systems (inc. OLEDs), Human sensors, LED lighting,
Soft electronics, antennas, fine cables etc.

116
Scalable Modular Solutions
• Developed modular desktop endurance systems since 2009
including many tension-free mechanical jigs designed for basic components

• System options for electronic / optical data analysis & harsh environments

1. Mechanical Jig (over 120 variations)

3. + Data 4. + Systems
2. Drive unit

+ +
5

Examples

Twisting Rolling Bending

117
Reliability challenges

• Standards completed for semi-rigid form factors

• Recent activity for more flexible materials and form factors
• Integration of hostile conditions
• Data capture and Optical inspection systems
• Delamination
• Adjustment of neutral layers
• Scratch detection

Transition from FS (folding) to CS (flexing)

• “Butterfly” U-shape • “Clamshell” motion to
motion to simulate simulate flexing with a
gentle warping with a double hinge movement
push rod movement
• Requirement by new
• Requirement by materials &
manufacturers to manufacturers to
understand motion for develop flexible
rigid samples substrates

118
Transition from FS (folding) to CS (flexing)

CS family extension – CS‐M

• Tension-free ™ “Clamshell” type flexing

• Linear or planar objects

• Larger holding plates for bigger

samples

• Can be flexed to 0.5 mm radius

• Cartridge for easy examination during

testing
• New faster DR medium/small motor

• Can use in ambient & hostile conditions

• Can use with ECP electronic data &

system control package
10

119
 CS - integration in hostile environments
• Integrate mechanical jig into existing
Environmental chamber tests (motor
drive is connected thru chamber’s
service ports)
• Operating range : Controlled
Temperature (-40° to +90° C)
and Humidity (30% to 95% RH)
• Simple Tandem operation
(chamber and jig work
independently) or
Custom designed Single
programmable automatic
operation available

11

ECP – system control & data capture package

• PC based system control software for
integrated data capture and analysis

• Includes dedicated Resistance Meter,

dedicated expansion board & cables

• Enables programmed control of endurance

testing machines by measuring resistance of
the test sample continuously

• Can use in ambient & hostile conditions

• Remote collaboration platform extension

in development for 2021

12

120
CS-CAM development
Requirement to detect and predict deformation
stress exerted at specific location of test sample

• Smooth and steady motor helpful to develop stable

platform for continuous image capture
• Optimize (size & usability) of optical camera system
at precise motor movements
• Develop software analytics to determine key
metrics

13

Optical image capture

40mm 6mm 1.5mm

14

121
Surface strain relationship

15

Visual inspection

16

122
CS “Teardrop” Cartridge
Enables to easily adjust CS Holding Plates to
simulate various size & shape of hinge movement The CS-Teardrop shape is U-shape folding without
very similar to an actual full vertical support
tear becomes a teardrop shape
• Optional accessory to CS standard size jig with
new medium/small DR drive unit

• Precise adjustment to the bottom supporting element

of the U-shape flexing test

• Alternatively can cost effectively adjust neutral

layer on standard CS jig The CS-Teardrop shape Parameters for adjusting
during flexing CS-Teardrop shape

17

CS- TD Cartridge

18

123
New design concept

PICO Multi-environment chamber

Prototype desktop multi-environment for
endurance testing machines

• Designed for integrated CS flexing jig with

DR medium/small motor
• New G-shock & vibration mechanism via motor drive unit
• Anticipated release mid 2021

19

New design preview

Next-Gen Modular Systems

Prototype desktop mechanical jigs

• Designed for Compound Components

• Self-custom 2-axis mechanical jigs
• Dedicated rotary or linear motion drive units

20

124
 Integrated in North America / Europe by

www.bayflextechnologies.com

21

125
126
 Presentation 5.10

New Material Solutions for Automotive Displays,

Interfaces, and Applications
Volker Plehn, SABIC, Wixom, MI, US

PAPER UNAVAILABLE

PRESENTATION SLIDES SHOULD BE DISTRIBUTED

AFTER THE CONFERENCE

127
128
 An alternative to OLED with full-array local dimming in automotive displays
Logan Cummins
Automotive Systems Engineering
Texas Instruments
Dallas, Texas, United States 75243
[email protected]

Abstract: The automotive display market has historically and peak brightness concerns – have delayed the
lagged behind the consumer display industry in contrast implementation of OLED displays in automotive systems.
ratio, black levels, resolution, curvature, and form-factors
Lighting LCDs in automotive applications
due to the automotive specifications and environmental
conditions. If you look to the personal electronics market, Automotive displays have traditionally used globally
you might assume that emissive displays, such as organic dimmed edge-lit backlight architectures to illuminate
light emitting diodes (OLEDs) or micro-LEDs are the best through the liquid-crystal and color-filter layers in the thin-
way to achieve ideal automotive displays. However, the field transistor (TFT) LCD panel to generate colored
practicality of OLED displays in automotive has been pixels. The liquid crystals allow light to go through or
continuously delayed by numerous design and fabrication block light from passing to the color filter creating each
challenges, such as lifetime, cost, and peak brightness subpixel. LCD panels with global backlight architectures
concerns. Globally dimmed, edge-lit backlight create light everywhere, regardless of whether the subpixel
architectures are most common in automotive today, but is on or off, and rely solely on the liquid crystals to block
lack the desired contrast ratio and black levels. light. The LCD panel’s intrinsic ability to block light will
determine the contrast ratio and black levels of the display
So, how can carmakers meet modern display expectations?
as shown in Figure 1.
Full-array locally dimmed backlight architecture shows
potential to improve contrast ratio of LCDs near OLED
levels while also providing power savings over traditional
backlight methods.
This paper will explain the benefits of local dimming
backlight architecture, as well as how to consider and
solve the implementation challenges.

Keywords: Local Dimming; Globally dimmed; Edge-Lit;

Backlight; HMI; Infotainment; Automotive Displays;
OLEDs; LEDs; LCD; Automotive Center Displays (CID);
Contrast Ratio; Brightness

Introduction:
Even though the automotive display market has typically
trailed behind the consumer display industry, automakers
must somehow differentiate their infotainment human-
machine interface (HMI) displays while catching up to the
technological advances now common in smart phone,
tablet and television displays.
LCDs now pervade many aspects of modern life and are
becoming more prevalent in vehicles, replacing analog and
hybrid gauge clusters and becoming standard in the center
information display and passenger entertainment areas of
the vehicle. However, these displays lack the image quality
and contrast ratio that consumers experience with their
personal electronics.

The next alternative you might assume would be emissive

displays, such as organic LEDs (OLEDs) or micro LEDs
to achieve the ideal automotive display. But numerous
design and fabrication challenges – including lifetime, cost Figure 1: The layers within an LCD panel1

129
OLEDs in automotive applications
OLEDs and micro LEDs are emissive-based displays, a
single pixel is formed by three RGB sub-pixel LEDs. In
contrast to TFT LCD panels, emissive displays only
generate light where pixels are needed. OLED displays
have a greater contrast ratio – as much as 1 million-to-1 –
compared to 2,000-to-1 in normal TFT LCD displays.
They also have lower peak-illuminance capabilities, which
are important in automotive displays in order to overcome
bright ambient light conditions. The lower contrast ratios
and black levels of automotive displays can cause
unpleasant night-time viewing, when black cluster and
gauge backgrounds and menus produce a gray-hue effect Figure 2: Switching LEDs individually to achieve
from the LCD light leakage. a better display
While many have expected OLED technology to become
A local-dimming backlight technology can help you
more widespread in automotive applications for years, it
achieve greater contrast ratios, maintain high peak
has yet to proliferate due to lifetime, peak brightness and
illuminance, and remain within automotive environmental
cost concerns.
and cost limits.
The case for local dimming
The benefits of using a full-array local dimming
A local-dimming backlight technology is a direct-lit architecture include:
architecture where the LEDs are directly behind the LCD
panel as shown in Figure 2. Each LED or zone of LEDs  The mitigation of light leakage by dimming the
can dim individually to illuminate only those pixels of the backlight zones where the pixels have darker content.
display that are needed by dynamically adapting to the  Improved contrast ratios (up to several hundred
image content on the display. thousand-to-one) depending on the number of zones,
peak brightness and the native contrast ratio of the
display.
 Lower power consumption compared to globally
dimmed backlights, since the LEDs are not lit unless
needed.

Table 1 compares the benefits and considerations for

automotive display options.

Table 1: A side-by-side comparison of display options

130
Design factors for implementing local dimming In addition to optical layers, such as diffusers and
polarizers, increasing the air gap between the LEDs and
System cost and performance of a local dimming system panel glass will better distribute the light evenly through
are directly related to the number of LEDs and zones. If the panel. The number of dimming zones is directly
there are too few zones, the resulting zone size will be too proportional to the amount of halo artifact created by the
large, and a halo effect will occur where light bleeds into system, as more zones will better match the display’s
pixels that need to be fully dark for the best contrast. Other pixels and reduce the unwanted illumination of dark pixels.
causes of the halo effect include a light spread function of
the particular zone, zone overlap, spatial filters in the Components of an automotive display local
dimming algorithm, and the native contrast ratio of the dimming system
LCD panel.
An automotive local dimming backlight module and LCD
To achieve the native contrast ratios required for display system have similar but slightly modified
undetectable halo-effect levels comparable to OLEDs, one components when compared to a traditional globally
study2 concluded that an LCD with a 5,000-to-1 contrast dimmed backlight system. The major components include
ratio requires 200 local dimming zones, while a 2,000-to-1 a timing controller (TCON), LED drivers and LED
contrast ratio requires over 3,000 local dimming zones. backlight unit, as shown in Figure 5.

Figures 3 and 4 demonstrate the light leakage in globally

dimmed and locally dimmed displays, respectively. Figure
4 shows the higher black levels realized by local dimming
and the halo artifact introduces immediately surrounding
the white circle. Figure 3 shows a traditional edge-lit LCD
with no halo effect, but with a lower contrast ratio due to
the light leakage of the LCD.

Figure 5: The components of a local dimming

system for automotive displays

The TCON will convert a video input, such as Open Low-

Voltage Differential Signaling (LVDS) interface, LVDS or
Red-Green-Blue, into control signals for the source and
row drivers in the LCD panel. In a local dimming system,
the TCON is specialized to include the internal processing
and histogram calculations for the individual zone
dimming, with a Serial Peripheral Interface (SPI) output to
control the LED drivers.

Traditional edge-lit backlight units contain 20 to 80 LEDs

Figures 3 and 4 compare the backlight and along the edge in conjunction with light guides, diffusers
resulting light leakage of globally dimmed and and polarizers. The locally dimmed LED backlight unit
locally dimmed displays will contain anywhere from 96 to as many as 1,000 LEDs
or more uniformly dispersed in a grid, directly behind the
Careful consideration needs to be taken during the LCD to be illuminated. The LEDs are individually, or
definition of the local dimming system parameters to sometimes grouped together as two or four LEDs in series
ensure that local dimming performance outweighs the or in parallel, controlled by a single low-side channel from
added system cost and artifacts introduced. Thickness of the LED driver.
the backlight module, halo effect, thermals and system cost
are all trade-offs to consider. Instead of the single four-to-six-channel low-side LED
driver used in edge-lit architectures, a local dimming
The number of dimming zones and LEDs per zone are the architecture uses multiple 16- to 48-channel LED drivers
main priorities when designing a local dimming system. to achieve the higher zone counts. Multiple LED drivers
This combination defines the pitch of the LED array, can have the control signals daisy-chained together to
which impacts the backlight module’s overall thickness to provide an easily controlled and scalable approach based
achieve homogenous light distribution across the display. on the number of zones needed.

131
Table 2 compares OLED, local dimming and traditional The key takeaway for designing a local dimming system
edge-lit globally dimmed backlight implementations. for an automotive display is to choose the right number of
LEDs spread across the right number of zones. To see
Table 2: A comparison of display lighting options more technical information and a demonstration of the
Parameter OLED / Full array Edge-lit, local dimming backlight architecture, learn more here: out,
micro- locally globally Automotive 144-Zone Local Dimming Backlight
LED dimmed dimmed Reference Design.
Number of 100 to
zones
Millions 4 to 6 References
1,000s
LEDs 1. Rasmussen, Torben “OLED vs. LCD,” FlatpanelsHD,
1 1 to >4 20 to 40
per zone https://www.flatpanelshd.com/focus.php?subaction=s
Current per howfull&id=1474618766 , 23 September 2016.
N/A 10-50 mA 100-200 mA
string 2. Guanjun Tan, Yuge Huang, Ming-Chun Li, Seok-Lyul
Lee, and Shin-Tson Wu, "High dynamic range liquid
Full-array local dimming in automotive displays could crystal displays with a mini-LED backlight," Opt.
bridge the gap in performance displays in the consumer Express 26, 16572–16584 (2018)
market. Local dimming improves the contrast ratio of
traditional edge-lit LCD displays that underperform due to
their low native contrast ratio and light leakage.

Conclusion

Automotive displays are subject to much more

environmental variations than their consumer-grade
counterparts: whether it’s day, night, hot, cold, or even
whether the car bounces up and down as it travels over a
rough road. Vehicle displays have strict temperature
operating ranges; electromagnetic emission restrictions;
and immunity, vibration and lifetime standards. Many of
the technology advances in consumer displays fail to
overcome these strict environmental requirements, but
automotive OEMs must still provide similar performance
that users expect from their personal electronics. OLED
and globally dimmed displays are either too expensive or
do not provide the performance that customers want. Local
dimming can provide a balanced trade-off between price
and performance while meeting the strict environmental
constraints of an automotive system.

132