Session #7:
SIFT
Instructors, Computer Vision
Credits: Slides adopted from Shree K Nayar’s lectures
Motivation: Automatic panoramas
Credit: Matt
Why extract features?
• Motivation: panorama stitching
– We have two images – how do we combine them?
Why extract features?
• Motivation: panorama stitching
– We have two images – how do we combine them?
Step 1: extract features Step 2: match features
Why extract features?
• Motivation: panorama stitching
– We have two images – how do we combine them?
Step 1: extract Step 2: match Step 3: align
features features images
Application: Visual SLAM
• (aka Simultaneous Localization and Mapping)
Image matching
by Diva Sian
by swashford
Harder case
by Diva Sian by scgbt
Harder still?
Answer below (look for tiny colored squares…)
NASA Mars Rover images
with SIFT feature matches
Feature matching for object search
Feature Matching
Object matching
Invariant local features
Find features that are invariant to transformations
– geometric invariance: translation, rotation, scale
– photometric invariance: brightness, exposure, …
Feature Descriptors
Advantages of local features
Locality
– features are local, so robust to occlusion and clutter
Quantity
– hundreds or thousands in a single image
Distinctiveness:
– can differentiate a large database of objects
Efficiency
– real-time performance achievable
More motivation…
Feature points are used for:
– Image alignment (e.g., mosaics)
– 3D reconstruction
– Motion tracking (e.g. for AR)
– Object recognition
– Image retrieval
– Robot/car navigation
– … other
Approach
1. Feature detection: find it
2. Feature descriptor: represent it
3. Feature matching: match it
Feature tracking: track it, when motion
Local features: main components
Detection: Identify the interest
points
Description: Extract vector
feature descriptor surrounding
each interest point
Matching: Determine
correspondence between
descriptors in two views
Credit: Kristen Grauman
SIFT - Key point Extraction
● Stands for scale invariant feature transform
● Patented by university of British Columbia
● Similar to the one used in primate visual system (human, ape, monkey, etc.)
● Transforms image data into scale invariant coordinates
David Lowe, UBC
D. Lowe. Distinctive image features from scale-invariant key points., International Journal of Computer Vision 2004.
Goal
● Extracting distinctive invariant features
○ Correctly matched against a large database of features from many images
● Invariance to image scale and rotation
● Robustness to
○ Affine distortion,
○ Change in 3D viewpoint,
○ Addition of noise,
○ Change in illumination.
Raw image matching
σ normalised
Variation in Scale σ
Variation in Scale σ
Gaussian Blurring
Space Scale
construction
Credits:
[Link]
Difference of Gaussian
[Link]
Difference of Gaussians visualisation
[Link]
Keypoint localization
Key point descriptor
Keypoint descriptor
SIFT descriptor
Scale Invariation
Rotation Invariation
