Scribd
Upload
56 views

Visual Odometry (Vo) : Csc2541, Feb 9, 2016 Presented by Patrick Mcgarey

The document discusses visual odometry (VO), which estimates the motion of a camera in real time using sequential images. It describes different types of VO, including monocular VO using a single camera, and stereo VO using two cameras to solve the scale problem. The document provides an overview of the VO pipeline and algorithms used, such as feature extraction and matching, essential matrix computation, and solving for camera motion. It also discusses applications and challenges of VO, including localization, mapping, and robustness to lighting and lack of features.

Uploaded by

herusyahputra
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
56 views

Visual Odometry (Vo) : Csc2541, Feb 9, 2016 Presented by Patrick Mcgarey

The document discusses visual odometry (VO), which estimates the motion of a camera in real time using sequential images. It describes different types of VO, including monocular VO using a single camera, and stereo VO using two cameras to solve the scale problem. The document provides an overview of the VO pipeline and algorithms used, such as feature extraction and matching, essential matrix computation, and solving for camera motion. It also discusses applications and challenges of VO, including localization, mapping, and robustness to lighting and lack of features.

Uploaded by

herusyahputra
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 36

Visual Odometry (VO)

CSC2541, Feb 9th, 2016


Presented by Patrick McGarey
Primer on Odometry

• What is Odometry?
• The Greeks invented it… “Route Measure”
• Estimating change in position over time

• What is Visual Odometry?


• Estimating the motion of a camera in real time
using sequential images (i.e., egomotion)
• The idea was first introduced for planetary rovers
operating on Mars – Moravec 1980
Pathfinder landing, 1997

2
Primer on Visual Odometry

• Camera Types
• Passive
UEye Camera
• Monocular
Point Grey Stereo Cam
• Stereo
• Omnidirectional
• Active
• Lidar Bubl Omnicam

• Time-of-flight
Velodyne Lidar
• RGB-Depth
• Hybrid
• Uses multiple sensors
Mesa TOF Cam
Kinect RGB-D

3
Primer on Visual Odometry

• Monocular Visual Odometry


• A single camera = angle sensor
• Motion scale is unobservable
(it must be synthesized)
• Best used in hybrid methods

• Stereo Visual Odometry Perspective Camera Model

• Solves the scale problem Epipolar Constraint

• Feature depth between images


• Degenerates to the monocular case
if only distant features are used

Images from Scaramuzza and Fraundorfer, 2011 4


Primer on Visual Odometry

• Monocular Visual Odometry


• A single camera = angle sensor VO Problem Illustration

• Motion scale is unobservable


(it must be synthesized)
• Best used in hybrid methods

• Stereo Visual Odometry


• Solves the scale problem
• Feature depth between images
• Degenerates to the monocular case
if only distant features are used Image from Scaramuzza and Fraundorfer, 2011

5
Primer on Visual Odometry

• Monocular Visual Odometry VO Pipeline


• A single camera = angle sensor
• Motion scale is unobservable
(it must be synthesized)
• Best used in hybrid methods

• Stereo Visual Odometry


• Solves the scale problem
• Feature depth between images
• Degenerates to the monocular case
Image from Scaramuzza and Fraundorfer, 2011
if only distant features are used

6
Primer on Visual Odometry

Image from Scaramuzza and Fraundorfer, 2011

• Essential Matrix is computed from features correspondences using epipolar constraint


• The matrix has an unknown scale factor (problem with Monocular)
• Commonly solved with the Nister Five-Point Algorithm [2003] (solves using SVD)

7
Primer on Visual Odometry

Image from Scaramuzza and Fraundorfer, 2011

8
Use Cases for Visual Odometry
• Motion estimation for vehicles Where am I ?

• Driver assistance
• Autonomy

• Point-Cloud Mapping
• VO to estimate the motion of a lidar during Image from NASA/JPL

the collection of a single scan


• Reduce rolling shutter effect
Where are my features at ?

• Challenges
• Robustness to lighting conditions
• Lack of features / non-overlapping images
• Without loop closure the estimate still drifts
Image from mirror.co.uk
9
Paper Review

Visual-Lidar Odometry and Mapping:


Low-drift, Robust, and Fast (Robotics Institute, CMU)
Paper by: Ji Zhang and Sanjiv Singh, ICRA 2015
Presented by Patrick McGarey
General Problems

• Localization and mapping in degenerate environments is challenging


• e.g., lack of discrete/unique features, or poor lighting

Images from Zhang and Singh, 2015

11
Problems with VO & LO

• VO requires moderate-good lighting


conditions

• VO fails when features are limited


• Lidar Odometry (LO) is limited
because of motion distortion Image from swinburne.edu.my

• LO inherently involves many variables rolling shutter effect


(the world is represented by points)

• LO depends on scan matching, which


fails in degenerate environments

Image from IEEE Spectrum

12
Motivation

• Estimate 6-DOF motion of a


camera/lidar rig
Hokuyu Lidar
• Collect spatially accurate 3D point
cloud of the environment

• Exploit complementary strengths


and weaknesses of a monocular
camera and lidar
Camera

• Be robust to (1) aggressive motion


Image from Zhang and Singh, 2015
and (2) intermittent feature dropouts

13
Preview

• Develop a real-time slam method that is robust to lighting changes, has


low drift and has reports pose updates at a high frequency

Video from Zhang and Singh, 2015

14
Method V-LOAM
• Use VO at high frame-rate (60Hz) to estimate motion
• Use LO at low frequency (1Hz) to refine motion estimate
• Uses course point cloud from last iteration to provide depth to features
• Use improved VO motion to undistort point cloud
• Merge current point cloud into the global map

Images from Zhang and Singh, 2015

15
Method – Visual Odometry (VO)

• Course lidar data is used to add depth to high-rate images


• Camera motion is approximated as linear for the short distances between images
• 3 Types of features are generated, those with…
• no depth, depth from lidar, and depth from triangulation (i.e., SfM)
• Solve equation with 6 unknowns using least squares
• Output high-frequency transform

Image from Zhang and Singh, 2015


16
Method – Lidar Odometry (LO)
Steps
• 1) Sweep-to-Sweep: match 2 consecutive point clouds using ICP*
• Removes some distortion, *(ICP = Iterative Closest Point algorithm)
• 2) Sweep-to-Map: to the map add undistorted point cloud
• Along with the cloud, provide low-rate sensor poses
• 3) Using input from VO, output high-rate transforms and update map
• Removes more distortion

Images from Zhang and Singh, 2015


17
Method – Overall Pipeline VO+LO

Images from Zhang and Singh, 2015

18
Results – Wide Angle vs. Fisheye Lenses
• Confirm Low Drift
• Fisheye drifts faster than wide angle for lack of features
• Incorporating LO results in equal performance

Images from Zhang and Singh, 2015


19
Results – Fast (aggressive) vs. Slow Motion
Slow Fast

* Wide angle didn’t


work in the stairs due
to a lack of features,
so in this case Fisheye
is the best lens choice

Images from Zhang and Singh, 2015


20
Results – Light vs. Dark
• Robustness to feature dropouts
• Turn on and off the lights (for 2 seconds)

Images from Zhang and Singh, 2015


21
Results – KITTI Benchmark

Images from Zhang and Singh, 2015


22
Updated Results

Video from Zhang and Singh, 2015

23
Paper Review

Stereoscan: Dense 3D
Reconstruction in Real-Time
Paper by: Geiger et al., IVS 2011
Presented by Patrick McGarey
Problems

• The accuracy of real-time stereo VO is a function of image resolution,


meaning resolution is typically reduced to achieve faster performance
• Current stereo VO pipelines are computationally expensive, implying
that mobile and embedded systems must be fully dedicated to the task

Image from Geiger et al., 2011


25
Method

• Perform 3D reconstruction in
real-time by leveraging
• Sparse features for efficient stereo
matching
• Multi-view reconstruction (recast 3D
points from last frame into scene)

H.S. in Carbonite, Star Wars


Image from Geiger et al., 2011

26
Method: Pipeline

• 1) Feature Matching: only a subset of features detected are used to


match in reduced search windows. (other CV tricks are employed)
• 2) Egomotion Estimation: minimize projection errors using EKF
• 3) Stereo Matching: uses ELAS method
• Efficient Large Scale Stereo Mapping, Geiger et al., 2010
• 4) 3D Reconstruction: cast prior 3D points into current frame and take
the mean pose of the combined 3D point and a new point on the image
(they do this to create consistent point clouds from large amounts of data)

Images from Geiger et al., 2011


27
Results
Computational Time Odometry Result
Kitt et al. required inverting large matrices that grew linearly with features

Image from Geiger et al., 2011


28
Paper Review

Real-Time Stereo Visual Odometry for


Autonomous Ground Vehicles
Paper by: Howard, IROS 2008
Presented by Patrick McGarey
Problems

• Inertial sensors are prone to drift


• Wheel odometry is unreliable in
‘off-road’ terrain

• Historical Context
• At the time (2007-2008), Stereo VO was still a
newer topic of investigation

• Inlier detection was very slow Image from NASA/JPL


• Stereo navigation had recently been used on the
MER Rovers

• The Curiosity rover was about to be launched

30
Method

• Construct a VO pipeline that


• 1) Does not make assumptions of camera motion (no priors)
• 2) Works on dense disparity images in real time (prior methods were slow)

DARPA LAGR robot, 2008

31
Method: Pipeline

• 1) Inputs: rectified, pre-filtered images


• 2) Feature Detection: detect corner features, assign 3D vals from disparity
• Used Harris corners and FAST features
• 3) Score Matrix: using sum of absolute differences generate feature scores
• Low score indicates match
• 4) Match Features: use local minima from score to generate features
• Improves on state-of-the-art computation : from cubic to squared complexity
• 5) Find Inliers: inforce rigid world constraint to reject unlikely features
• 6) Estimate Motion: minimize the reprojection error and output motion

32
Results (VO vs. Wheel Odometry)

Image from Howard, 2008


33
Supplemental Slide

34
Primer on Visual Odometry

Image from Scaramuzza and Fraundorfer, 2011

35
References

• Scaramuzza, Davide, and Friedrich Fraundorfer. "Visual odometry


[tutorial]." Robotics & Automation Magazine, IEEE 18.4 (2011): 80-92.
• Zhang, Ji, and Sanjiv Singh. "Visual-lidar odometry and mapping: Low-
drift, robust, and fast." Robotics and Automation (ICRA), 2015 IEEE
International Conference on. IEEE, 2015.
• Geiger, Andreas, Julius Ziegler, and Christoph Stiller. "Stereoscan: Dense
3d reconstruction in real-time." Intelligent Vehicles Symposium (IV), 2011
IEEE. IEEE, 2011.
• Howard, Andrew. "Real-time stereo visual odometry for autonomous
ground vehicles." Intelligent Robots and Systems, 2008. IROS 2008.
IEEE/RSJ International Conference on. IEEE, 2008.

36

You might also like