Digital Image:

Representation and
Acquisition
Jayanta Mukhopadhyay
Dept. of CSE,
IIT Kharagpur
How images are represented
in a computer?

Width: 256
Height: 384

A 2D Array of integers
Color images

R
Image File

Header 100 101 88 Stream of pixels EOF

width height # Comp. # bytes/pixel

Standard File Formats:

TIFF, BMP, GIF, PGM, PPM, JPEG, DICOM, ….
Image formation in optical
camera

P à p : Projection
E(P) à I(p) : Stimulation

O
p
 Perspective
Rules of projection projection

Image points formed by intersection of the ray from a point P

passing through the center of projection O with the image
plane.
Center of
projection

O
p

Image plane
Other imaging principle: Xray
imaging

Parallel
Sensors sensitive to x-Ray EM bands
projection
Image points formed by intersection of parallel rays
with the image plane.
Other imaging principle:
Ultra-Sound Imaging
Tx Rx
Scan radially / translating Tx-Rx
along a direction.
What is an image?
l Impression of the physical world.
l Spatial distribution of a measurable quantity, encoding the
geometry and material properties of objects.
Radiometric factors of image
formation
Spectral response of a sensor
l Three factors of image formation:

Spectral power distribution Surface reflectance spectrum

o Objects present in the scene.
o Spectral Energy of Light Sources.
o Spectral Sensitivity of sensors.
o For color images three different types of sensors used
o More number of components for multispectral /
hyperspectral imaging systems in remote sensing 10
applications
Visual Information Processing

Retina

Optic Nerve

Optic Chiasma
Optic Tract
n Visual information enters the Lateral Geniculate
primary visual cortex via the Nucleus (LGN)
lateral geniculate nucleus
n smaller pathways also exist Visual Cortex
n a pathway to the superior

colliculus (SC).

Adapted from slides by Steve Seitz. taitravenab.blogspot.com

 The Eye: A Camera!
Iris - colored
annulus with
radial muscles

Pupil - the hole Retina -photoreceptor cells

(aperture), size (rods and cones) acting like
controlled by the the film or array of sensors
iris. of a camera.

Lens - changes shape by

using ciliary muscles for
focusing on objects of
interest.
Adapted from slides by Steve Seitz.
 Rods, Cones and
Synapse

Rods: Low illumination scene

Cone: Color perception
Trichromacy theory
Color opponency
G-R
B – (G+R)
Color constancy

https://commons.wikimedia.org/wiki/File:Rods_Cones_Synapse.sv
g
 Cells in retina
Visual input

Photo receptors

Horizontal cells

Excitatory
Bipolar cells direct
pathway
Off / On Center
Surround receptive Ganglion cells inhibitory indirect
field pathway Mexican hat
Lateral geniculate nucleus Optic nerve receptive field
(LGN)
 Density of rods and cones

Rods and cones are non-uniformly distributed on the retina

• Rods responsible for intensity, cones responsible for color
• Fovea - Small region (1 or 2°) at the center of the visual field
containing the highest density of cones (and no rods).
• Less visual acuity in the periphery—many rods wired to the same
neuron Slide by Steve Seitz
 Rod / Cone sensitivity
The candela is 1 lambert (L) =
the luminous candela per
intensity, in a square centimetre
(0.3183 cd/cm²) or
given direction,
cd m−2
of a source that
emits
monochromatic
radiation of
frequency
540×1012 hertz
and that has a
radiant
intensity in that
direction of
As cone vision is not activated in low
1⁄
683 watt per
illumination, we are unable to read.
steradian. Adapted from slides by A. Efros
 Physiology of Color Vision

Three kinds of cones:

RELATIVE ABSORBANCE (%) 440 530 560 nm.

100
S M L

50

400 450 500 550 600 650

WAVELENGTH (nm.)

• Ratio of L to M to S cones: approx. 10:5:1

• Almost no S cones in the center of the fovea
© Stephen E. Palmer, 2002
Color perception
M L
Power
S

Wavelength
• Entire spectrum (of reflected energy from an
object or energy of an illuminant) represented by
3 numbers.
• Even different spectra may have same
representation and thus indistinguishable.
• such spectra called metamers.
Adapted from slides by Steve Seitz.
What is Color?
l A psychological property of our visual experiences
when we look at objects and lights.
l Not a physical property of those objects or lights.
l The result of interaction between physical light in the
environment and our visual system.

o Can be represented by 3
numbers.
o Trichromatic Theory
o Color rendition as a
Mixture of three
primary colors
19
 Grassman’s Laws
l Color matching appears to be linear.
l If two test lights can be matched with the same set of
weights, then they match each other:
l If A = u1P1 + u2P2 + u3P3 and B = u1P1 + u2P2 + u3P3. Then A = B.
l If we mix two test lights, then mixing the matches will
match the result:
l If A = u1P1 + u2P2 + u3P3 and B = v1P1 + v2P2 + v3P3.
Then A+B = (u1+v1) P1 + (u2+v2) P2 + (u3+v3) P3.
l If we scale the test light, then the matches get scaled
by the same amount:
l If A = u1P1 + u2P2 + u3P3, then kA = (ku1) P1 + (ku2) P2 + (ku3) P3.
Adapted from W. Freeman
 mixing two lights
produces
colors that lie
Linear color spaces along a straight
line in color
space.
l Defined by a choice of three primaries
l The coordinates of a color are given by the weights of the
primaries used to match it.
l Matching functions: weights required to match single-
wavelength light sources.

mixing three lights produces

colors that lie within the triangle
they define in color space.
Adapted from W. Freeman
Standardizing color experience
l To understand which spectra produce the
same color sensation under similar viewing
conditions.
l Color matching experiments.

Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995

Color matching experiment 1

Source: W. Freeman
Color matching experiment 1

p1 p2 p3
Source: W. Freeman
Color matching experiment
1

p1 p2 p3
Source: W. Freeman
Color matching
experiment 1

The primary color

amounts needed
for a match

p1 p2 p3
Source: W. Freeman
Color matching experiment 2

Source: W. Freeman
Color matching experiment 2

p1 p2 p3
Source: W. Freeman
Color matching experiment 2

p1 p2 p3
Source: W. Freeman
Color matching experiment 2
The primary color
We say a amounts needed
“negative” for a match:
amount of p2
was needed to
make the
match, because p1 p2 p3
we added it to
the test color’s
side.

p1 p2 p3 p1 p2 p3

Source: W. Freeman
 Using color matching functions
to predict the matches for a
new spectral signal
A monochromatic light of li
wavelength will be matched by the
amounts c1 (li ), c2 (li ), c3 (li )
of each primary.
And any spectral signal can be thought of as a linear
combination of very many monochromatic lights, with the
linear coefficient given by the spectral power at each
wavelength.
æ t (l1 ) ö
" ç ÷
t =ç ! ÷
ç t (l ) ÷
è N ø Source: W. Freeman
 Using color matching functions
to predict the primary match to a
new spectral signal
Store the color matching functions in the
rows of the matrix, C
æ c1 (l1 ) ! c1 (lN ) ö
ç ÷
C = ç c2 (l1 ) ! c2 (lN ) ÷ æ t (l ) ö
ç c (l ) ! c (l ) ÷ Let the new spectral " ç 1 ÷
è 3 1 3 N ø signal be described by t = ç ! ÷
ç t (l ) ÷
the vector t.
Then the amounts of each ! ! è N ø

primary needed to match t are: e = Ct

The components e1, e2, e3 describe the color of t. If you have
some other spectral signal, s, and s matches t perceptually,
then e1, e2, e3, will also match s (by Grassman’s Laws)
Source: W. Freeman
Additive and subtractive colors

Adapted from Gonzales and Woods

 Linear color
spaces: RGB

l Primaries are monochromatic lights

(for monitors, they correspond to the
three types of phosphors).
l Subtractive matching required for
some wavelengths.
RGB matching functions
 RGB model

l Additive model.
l An image consists of 3 bands,
one for each primary color.
l Appropriate for image
displays.

Adapted from Gonzales and Woods

 Inks: Cyan=White-Red,
Magenta=White-Green,
CMY model Yellow=White-Blue.

l Cyan-Magenta-Yellow is a
subtractive model which is
good to model absorption of
colors. é C ù é1ù é R ù
l Appropriate for paper ê M ú = ê1ú - êG ú
printing. ê ú êú ê ú
êë Y úû êë1úû êë B úû

Adapted from Octavia Camps, Penn State

 CIE chromaticity model
l The Commission Internationale de
l’Eclairage (estd. 1931) defined 3 standard
primaries: X, Y, Z that can be added to form
all visible colors.
l Y was chosen so that its color matching
function matches the sum of the 3 human
cone responses.
é X ù é0.6067 0.1736 0.2001ù é R ù é R ù é 1.9107 - 0.5326 - 0.2883ù é X ù
êY ú = ê0.2988 0.5868 0.1143ú êG ú êG ú = ê- 0.9843 1.9984 - 0.0283ú ê Y ú
ê ú ê úê ú ê ú ê úê ú
êë Z úû êë0.0000 0.0661 1.1149 úû êë B úû êë B úû êë 0.0583 - 0.1185 0.8986 úû êë Z úû

Adapted from Octavia Camps, Penn State

 CIE XYZ: Linear
color space

l Primaries are imaginary, but

matching functions are
everywhere positive
l 2D visualization: draw (x,y),
where
x = X/(X+Y+Z)
y = Y/(X+Y+Z)

Matching functions

http://en.wikipedia.org/wiki/CIE_1931_color_space
 CIE chromaticity model
l x, y, z normalize X, Y, Z
such that
x + y + z = 1.
l Actually only x and y are
needed because
z = 1 - x - y.
l Pure colors are at the
curved boundary.
l White is (1/3, 1/3, 1/3).
Adapted from Octavia Camps, Penn State
Spectral locus of monochromatic
lights and the heated black-bodies

Computer Vision - A Modern Approach

Color Slides by D.A. Forsyth
 Uniform color spaces
l Differences in x,y
coordinates do not
reflect perceptual
color differences.
l CIE u’v’ is a
projective transform
of x,y to make the
ellipses more
uniform.
McAdam ellipses:
Just noticeable
differences in color
Adapted from Linda Shapiro, U of Washington
 CIE Lab (L*a*b)
model
l One luminance channel (L*)
and two color channels (a* and
b*).
l In this model, the color
differences which we perceive
correspond to Euclidean
distances in CIE Lab.
l The a axis extends from green (-
a) to red (+a) and the b axis from
blue (-b) to yellow (+b). The
brightness (L) increases from Xn, Yn and Zn are the
the bottom to the top of the 3D reference white in XYZ space.
model.
http://en.wikipedia.org/wiki/Lab_color_space
Adapted from Linda Shapiro, U of Washington
YIQ model

éY ù é0.299 0.587 0.114 ù é R ù

ê I ú = ê0.596 - 0.275 - 0.321ú êG ú
ê ú ê úê ú
êëQ úû êë0.212 - 0.532 0.311 úû êë B úû

l Have better compression properties.

l Luminance Y is encoded using more bits than
chrominance values I and Q (humans are
more sensitive to Y than I and Q).
l Luminance used by black/white TVs.
l All 3 values used by color TVs. 43
Adapted from Octavia Camps, Penn State
 YCbCr space

l Have better compression properties. Used in

image and video compression schemes.
l Y represents the luminance, and Cb and Cr are
chrominance parts.
l Not a linear transformation, affine.
l Cb and Cr translated to bring them within the
range of 0 to 240 assuming ranges of R, G and B
44
are 0 to 255. Adapted from Octavia Camps, Penn State
Nonlinear color spaces: HSV

l Perceptually meaningful dimensions:

Hue, Saturation, Value (Intensity)

Adapted from Linda Shapiro, U of Washington

 HSV model
l HSV: Hue, saturation, value are non-linear functions
of RGB.
l Hue relations are naturally expressed in a circle.

( R+G+B)
I=
3
min( R, G, B)
S = 1-
I
ì 1 / 2[( R - G )+( R - B)] üï
-1 ï
H = cos í ý if B<G
[ ]
ïî ( R - G ) +( R - B)(G - B) ïþ
2

ì 1 / 2[( R - G )+( R - B)] üï

-1 ï 46
H = 360 - cos í ý if B>G
Adapted from Octavia Camps, Penn State [ ]
ïî ( R - G ) +( R - B)(G - B) ïþ
2
 HSV model

l Uniform: equal (small)

steps give the same
perceived color changes.
l Hue is encoded as an angle
(0 to 2p).
l Saturation is the distance to
the vertical axis (0 to 1).
l Intensity is the height along
the vertical axis (0 to 1).
Adapted from Gonzales and Woods
Opponent Color Processing

l The color opponent process: A theory proposed on

perception of color by processing signals from
cones and rods in an antagonistic manner.
l Overlapping spectral zone of three types of cones
(L for long, M for medium and S for short).
l The visual system considered to record
differences between the responses of cones, rather
than each type of cone's individual response.
l People don’t perceive reddish-greens, or bluish-yellows.
 Opponent Color Processing
l The opponent process theory accounts for
mechanisms that receive and process
information from cones.

By Googolplexbyte - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=25539422

 Opponent Color Processing

l Three opponent channels:

Red vs. Green, (G-R)
Blue vs. Yellow, (B-Y) or (B-(R+G)) and
Black vs. White, (Luminance: e.g. (R+G+B)/3).

Y-Cb’-Cr’ follows opponent color space representation.

Summary
l Images are formed by projection.
l through sensing reflected energy from the respective surface point.
l spectral sensitivity of sensor determines the color of the component.
l Human perception of color follows trichromacy law.
l Three types of cones in our retina with spectral responses in
different zones of optical band
l Color is captured in the RGB color space.
l Not suitable for direct interpretation of color components such as
Hue and Saturation.
l Not all colors are representable in positive half of RGB color space.
l CIE Chromaticity Chart provides color matching
functions.
l expresses in 2-D normalized x-y chromaticity space.
l Various other color spaces used for processing.
Thank You

52