Copvright © IFAC Automatic Control in SPACE ROBOTICS AND

Aerospace . Tsukuba. Japan . 1989 MANIPULATORS

AUTONOMOUS NAVIGATION AND CONTROL

OF A MARS ROVER
D. P. Miller, D. J. Atkinson, B. H. Wilcox and A. H. Mishkin
J et Propulsioll Laburatoll'. Califomia IlIStitute of Tecll1lolog)" M.S. 301-440,
4800 Oak GruFf Dri,'e, Pasadella, CA 91109, USA

Abstract 2. Semiaulonomous Navigalion Scenario

A Mars Rover will n~~d to ~ able to navigate :ItIlOnomous· In the semiautonomous navigation (SAN) approach we are
cu~ntly investigating,. local paths are planned autonomously
I~' kilomct~rs at n time. This paper outlines the sensing. percep-
tIon. pla~OIng . a.nd execution monitoring systems that are cur- uSing s~nsor data obtained by the vehicle; however local path
rently ~lng deslgn~d for the rover. The sensing system is OOsed planning i.s guide.d by global routes planned less frequently by
~uman ~lngs uSing a topographic mllp, which is obtaincd from
~mu~d 'il-r~o vision. The in~erpretation of the imnges use a reg-
Istration of the depth map with a global height map provided by Images produced by a vehicle orbiting Mars. The orbiter could
an orbiting spacecraft. Snfe. Iow energy paths are th~n plnnned !JC • precursor !l'ission which would map a large area of Mars
In advaoce, or It could ~ part of the same mission and map
through t~ map. and expectations of what the rover's articulll-
tion sen!lOfS should sense are generated. These expectations are areas only as they are needed.
then used to ensure that the planned path is correctly ~ing exe- The sequence of operations in the ponion of SAN
cut~d . inv?lving Eanh is as follows. As command~d ('om Eanh, the
KeY~'ords: sterro vision, navigation, path planning.
orblter takes a stereo pair of pictures (by taking the two pictures
mobile robots. at di~ferent poinls in the orbit) of an l!Ca to be traversed (if this'
area IS .not already mapped). These p,ctures might have a
resolutton of about one meter, although poorer resolution could
I. Introduction be used. The pictures are sent to Earth, where they are used by
In the next decade, a mission to the planet Mars is planned a h~man operator (~rhaps with computer assistance) to
that will involve a robot moving acros~ the ~rfllCe of the planet desIgnate an approxImate path for the vehicle to follow
and coUecting samples. The roboI will have to travel dozens of designed tn avoid large obstacles, dangerous areas, and 'dead.
kilometers through rough terrain. It will have to be able to find end paths. This path and a topogntphic map for the surrounding
area are sent from Earth to the rover. This process repeats as
its way to sites of scientific intrtest and to the return ne~, perhaps oflCe for each traverse between site5 where
vehicle. The Rover will have to ~ able navig:lte expenments are to be done, or perhaps once per day or so on
through .its ~ocal ~nvironment aut~sly, due to long traverses.
~und tnp hght-tlme delays. This Implies that the ve-
The sequc"..ce of operations in t~ ponion of SAN taking
hIcle must ~ able to sense its environment. plan a
place on Mars IS u follows. The rover views the local scene
course through that environment, and react appr:opri-
and, by ",!Cans d~scussed below, computes a local topographic
ately to unexpe<;ted ~ituations as they appear. All this map. ThIs map IS matched to the local ponion of t~ global
must be done while guiding the vehicle towards the map sent from Earth, as constrainro by knowledge of the
goals that have bttn given to it from its human opera-
rove~'s CUITC~t. posit.ion from ot~r navigation devices or
t~ on the Earth. Our research concentrates on the
prevIOus positIons. In order to determin<." the accurate mver
sensing. perception, planning and ex~cution monitor-
position and to register the loc.,1 map to the global map. The
ing that must be carried out by the rover to ~nsure that local map (from the rover's sensors) and t~ 1',10001 map (fron1
a safe and cfrlCient path is found, and tbatthat path is
t~e Eanh) a~ t~n com.bi.~ to form a revised map that has
traversed correctly. hIgh resolullon 10 t~ VIClntty of the rover. This map is then
. The ~verall approach we are taking tries to mini- provid~d to t~ path planning ~yslem . A new path then is
ml7.e the interaction between the planning system and c~mputC'd, revl~lng the approxlmale path sent from Eanh. since
the propenies of any particular sensor. This is done in with the local hIgh resolution map small obstacles can ~ seen
two ways. First, the perception system builds up a which might have !JCen mis~ in the low-resolution pictures
world model from the sensors' outputs. Only this used on F..:Inh. USlOg the reVIsed path. the rover then drives
world model, including the uncenainties of th~ data ahead a shon distaoce (perhaps ten meters), and the process
is given to the path planning system. Once a suitabl~ repeats.
path has been detennined then sensory commands
must be generated that will help the rover to monitor
its progress through t~ traversaJ. To maintain the 3. Sensing and Perceplion
separation ~tween plnnning and specific ~n50ry The rover's sensing and perception system must be able to
hardware, this p~ss is divided inlo two st~ps: ex- sense the ~uITounding terrain, conven the sensed data into a
pect.atlOll generat~ and sensor scheduling. The gen- representat~on o~ Ihe local environment, and correlate this local
erauon of expectauons c.-'" ~ KComplished using a repre~ntat~,?n with the global map. thereby determining the
model of the vehicle and the terrain. The sensor rover s positIon; the local a',ld global maps must be integrated.
scheduling planner can then assign lhe relevanl sen-
and ~l~ and roughnes~ estimates determined for neamy
sors to monitor the rover's state. teITaln. 10 order to provide the appropriate world model for use
Seclion 2 of this paper fkscribes the semi ·autono- by the planning system [Wilcox88J. Figure I details the
mous navigation scenario. Section 3 presents the s~n­ elements of the sensing and perception system. lbe major
sor and perception systems of the rover. Section 4 de- operations of this system are briefly described below.
scri~s the planning systems that arc used to accom-
plish path planning and execution monitoring.
Architectures for both systems are given. 3.1 Stl'rt'O Correlation
Imp!ementalion a~ tesling lest~s arc presented in We have implemented a two camera stereo correlation ap-
secuon 5. and seCllon 6 presents some conclusions proach [Gennery 77,801. which operates on a pair of images
and results we have achieved so far.

III
112 D. P. Mille r fI al.

_-
w_

............

CWftWI ~ ..
pOll 1,10.

L-__~__~______~~~~
-....
a."... ...

I ....

::,~~~.~_~~~~4-------~
__ ~~
OIotlalTCft'1IM
001*-
«:/tIAI7TaEI!I

figure I. Smsin~ on,", Jl'"rcrption .y<l"'"

from the left and right camera., on the rovtr. Area comlation is to be thrown out. Features in the local database which are too
u5td. with same-size windows into the two images bdng small to show up in the global database a~ remined. The
moved in one-pixel increments ~Iative to each other. A~a process for rejecting points is similar to that in the ground sur·
comlation wu chosen, ratlltr than edge correlation, btcause a face finder approach previously described in IGennery 77, SOl.
Mars rover (as well a., the cumnt ellpenmental rover) will The OlltJlut or
the terrain matcher is: I) the position of the
operate in a natural terrain environment. Scenes in n.uural rover in the global coolIJinate frame, together with a goodness
environments tend to contain signirlCant textu~. but few well- of fit measure Cor the determined location, and 2) an edited
defined edges. local height map with apparently erroneous points removed.
lbe ~sult of stereo correlation is a disp:uity map. which
sptCirlts the difference in position between images of pixels 3.3 Slo~ IInd Rotlghnm; Determination
rep~nting the same point in space. Trian~ulation is then
used to build a range map from the disparity map. E.,ch point Traversability can be determined by analyzing the database
in the range m.'p has an associated covariance I1Io,trill to dctennine the slope and roughnes5 of the ground at each
representing the uncertainty in il5location. Finally, a . hori7.ontal position. This can be done hy Iocalleast·~uare fits
coordinate transfomlo'\tion is applied to the range map to put JI of planar or other surfaces and analysis of the IC5iduals.
into a coordinate frame with the same orientation as the global The slope and roughness determination module accepts as
database coordinate frame. The coordinate transformation is input the cumnt rover position, the corresponding local height
based on the known ll('ading of the cameras on the rover at the map. the global database. and any ellisting local height mnps
time im.'ges were captured. g~nerated previously from rover data collected at earlier
stopping points aJotlg the route. The rover position provides
the ofCsets necessary to properly position the local database in
3.1 Turain Maldling the global datalxl.~. Only a rectangular subset of the global
The terrain m:llclltr JTIeI1es the elevation map database in the vicinity of the rover location is ~ad in for
generated by stereo ',"Telation with the elevation map in tilt integration with the local database. A Gau5sian function is
datahase by a process of comlation and averacing, which also defined around each (equally spaced) output point, and a least
produces 'the best estimate of vehicle position as that which
prodlices the best correlation. HoWever, this computation is
~ua~s fit is performed or
alllacal database points that fall
close enough to the output point, with weights derived from the
more complicated than ordinary correlation btcause of the Gau~silln function and the covarianee matrices of the input
following: the points are not equally spaced, the~ may ~>e point~. A plane is fit in th~e dimensions to the data; the z
significant uncertainties in their horizontal positions, and there value of the plane at the (lI. y) coordinates of the output point
may be occasion:!l mistakes in the 5Iereo data. btcome5 the output height. The scatter of the points used for
The terrai~ matl:lltr performs a variAble- resolution search, the .... ast squares fit is used to determine the roughness me1l5ure.
beginning at the lowest re.wlution to produce a roul!h match,
then refining the match as the resolution is increased. The The output of the slope and roughness computation is a
uncertainty estim.1les associated with each point in the height height map of equally spaced points. Associated with e.,ch
111lI~ an: incorporated into the representations used for point is the following data: the slope at the location, a 3 x 3
matching. In the matching process. point' in the local database covariance matrill indicating the uncertainty in the height and
th.,t disagree significantly with the global database a~ slope, a roughness value, and an uncertainty rrltasure associated
eliminated. Note that a feature in the local database must be with the roughness value. All of this information is provided to
large enough to appear in the lower resolution global database the planning system.
 AUtOIlOlllOU S Na\'igatioll and COlltro l 11 3

cenainty is used to combine information from multiple sources

and represent features of the terrain in the quad-tree.
The path planner is then passed the quad-tree. Path planning
is donc using a hierarchical message-passing algorithm that tra-
verses through the high resolution area of the quad-tree in the
general di.ectioh of the goaIIMiller87). Most of the time the
high resolution area does not contain the goal (the goal is hun-
dreds or thousands of meters away and the high resolution area
is only tens of meters long). For this reason the path planning
can only be in the general direction of the ,oal. Additionally,
the path ~hould be in a direction that, according to the low reso-
lution data, will probably not lead to a blocked are .. that would
require backtracking. A variety of heuristics are used, in addi-
tion to the hierarchical representation, in order to assure that the
computation remains tractable. This is done by having t~ path
evaluation function incorporale.a crude simul:llion of the rover.
Path messages will not he sent through a ponion of space that is
untraversable. Similarly, messages will be held back that accrue
a high tr:lversal cost. The cost function is dominated by power
consumption and safety considerations (i.e., the probability of
the space being traversable, and the probability of the rover get-
ting trapped if it is not).
So the path planner plots a low cost path between the
rover' s current position and the edge of the high resolution area
of the quad-tree that lies in the general direction of the goal.
Plan Wilh The path ~ontains more than just directional information. As a
Expectations. rath. and result of Ihe simulation process used in path planning, the path
renues
also has annotations about expected power draw, and expected
stability of the rover for each segment of the path. Additionally,
expected rover orienl".ion, tilt, and acceleration may be calcu-
Figure 2. Planning systems laled from the path infonnation.
Once a path has been' computed, the navigation system per-
4. Path Planning & Execution Monitoring forms a terr~in/vehicle interaction simulation of the rover tra-
~ne ~over mus.t be able to autonomously pl~n a path and versing the path. This simulation is perfonned in order to locate
moOltor ItS exeCUllon of that path so that it may take corrective sensing landmarks for execution monitoring. The simulation
action as needed. These needs can be met by using a set of three also serves to verify the ttavcrsability and safety of the selected
integrated planning systems, each operating at a different level path.
of detail. lne first is a spatial path planner which plans gross
rover movements through the terrain. The second is a resource 4.2 Resource Scheduling
planner; this system allocates and schedules the rover's resourc- The resource planner uses all the information in the path
es such as power, pointable sen~ors . and the rover' 5 exact posi- plan to allocate the rover's resources. This planner uses models
tion. The final planner analyzes the path and activities produced of the rover's sensors to decide what sensor will need to be
by the first two systems and then plans out contingencies, i.e., used to verify the rover's successful traverse of each seg.nent of
reactions the rover will take if cenain sensor alarms fire, indi- the path. In panicular, the planner needs to know the sensor's
cating that the rover has deviated from the original plan. The range, visibility requirements, processing time, power draw,
functional modules of such a planning system are outlined in setup time, and pointing accuracy to calculate when It panicular
the figure . Details of the tree major planning modules used to sen~or i~ 10 be used. Additionally, the planner examines the
accomplish these t.~sks are given below. path from the p.,th planner to make sure that there are adequate
landmarks visihle to verify thf' rover's successful travefSC. This
4.1 Path l'lanning latter is not usually a problem. In rough terrain there are usually
Much work on path plannintl has been done for mobile ro- enough landmarks. simply by vinue of the terrain being rough .
bots. However, with few exceptions (e.g., (Linden87» all of If there are no landmarks, then the terrain is u~ually smooth and
that work has been done for vehicles designed to travel in in- safe, so an exact following of the plan is both simple. and un-
door environments (for examples, see (Brooksll2, Chatila8S, necessary (if the rover is in a fiat open desen, then it does not
C'hubbll7, CrowleyllS, Elfes87, McDennott84, Nguyen84, maller if it strays a couple meters here and there). If absolutely
SIackll7, and Thorpe84J).ln those situations a piece of space is necessary, the resource planner can request. specirlC waypoint
either tr:lvets.~ble or not. For a planetary rover, traversing rough to he added to the path; this info is passed back to the path plan-
naturnltcrrain, such a black and white analysis is not usually ner which replans that section of the path.
possible. A path plan must take into account what kind of ter- 1lle resource planner uses a directed beam-search through
rain the rover is currently on, what the transitions are like to the state/time search space (Miller86). This is in effect a directed
adjoinihg types of terrain, the size of the rocks, ~... hether they multiple worlds discrete event simulation. TIle best performing
are expected to be loose or solid, what the battery level of th~ world is pre(erentially selected until a simulation with a suc-
rover is charged at, and what the possibilities are of getting into cessful outcome is found. The schedule followed in that simula-
a tr:lPped situation if something tllrns out to be harder to tra- tion is used as the resource plan for the rover.
verse than expected. Additionally the route planner must model
non-geometric hazards that cannot be directly detected by the 4.3 ~xcculion Monitoring
sensing and perception system, but can only be inferred by the
The expectations generated by the path planner for rover
geologic context, and/or the behavior of the rover as it makes position ~nd orientation are combined with the resource alloca-
its traverse. tions to derive a set of sensor expectations. These consist of a
As described in the previous section, the rover first takes a period of time/space where a specific sensor should be reading
stereo image of the surrounding termin The resulting depth within a cenain range. A readi"g outside that range at that time
mal' is integrated into an Earth provided orbiter-view map of would indicate the the traverse is not proceeding as planned
the general area. 11lc: orbiter data can be first assembled on the [Munson73J . A sensor can either be. physical sensor on the ve-
Earth, lInd then, as the rover needs the information, the data can hicle, or a vinual sensor - a function of several of the sensors'
be sent to the rover a few kilomctcrs at a tinlt". 1lle combined data.
map of orbiter and stereo datl fives the rovcr a high resolution
There are too many possible vinu~1 <ensors for the rover's
map of its immediate surroundings with lower resolution in thc cOlllputation system to successfully monitor them all.
outlying MClS. This information is encoded statistically into I
Therefore, one job of the execution monitor planner is to select
quad-tree representation . A Bayesian model of the sensor un-
114 D. P. Miller et al.
which ale the important sensors for the panicular context that Ac:knowted,emcnt~: The research described in thi~ paper was
the rover is in. This is a.form of pmlictive monitorinl carried out by .the Jet Propulsion Laboratory - California
(Doyle89J. Institute of TechnololY under I contract with the National
The final planner on the rover uses the cxpect:llioos and the Aeronautics and Space Administration.
path pl:!n to derive a set of reflexes that the rover should havc at
the re:ldy for each selmcnt of the traverse. Different continlen-
cy reOexcs would be lrillcred by different sets of sensor
Rererences
alallm. The sensor alanns occur whenever the sensor reads out- (8rooks82J Brooks, R. A., Solving the find path problem by •
side the eT.(lKled nnlc. gond representation of free space, in Procudings of
In IIlO5I cases, the renexes milht just be to h:\ve the rover MAl 82, AAAI, pp. 381·386. 1982.
halt, take a new panorama, and repJan the remainder of the tra- (Chatila85J Chltil.. R., Position referencing and consistent
vcne. Some situations may demand more complex renexes. For world modelinl for mobile robots, in Procudinlls of III~
example, if the front wheels suddenly SllIn spinninl free , and Imcrnnrionnl Confercnce on Robotic.r and Automation.
the suspension encOOet's indicate that tbose wheels have sud- IEEE. pp. 138-145. 1985.
denly dropped, then the front of the rover has probahly broIcen (Chubb37J Chubb, D.W., An introduction and analysis of I
throulh the surface. If this was not expected, then the rover strailht line path algorithm for use in binary domains,
should stop and backup immediately, 10 avoid having the entire in the Procudings of Workshop on Spalial
vehicle getlin, mired. In other sitllntion~, (e.g .• if the senllOf' R~presentation and Multi -Sensor Fluion. SI. Charle~
system was initially un;lble to tell if an area of the surface was lIIinois, pp. 220-229, October 1987.
dllst covered or bedrock, but thoulht it was probahly dust-cov- (Crowley85J Crowley, J.L "Dynamic world modeling for an
eml) if the rover suddenly encounters less surface dralthen intelligent mobile robot using rotating ultra-sonic
had heen expectt'd. the rover might be able to modify its model ranging device." Procudings of the /985 Imunatiollal
of the sllrface on the Oy, and push through the new power pro- Conf~r~lIu 011 Robotics and Alllomatioll, pp 128-135,
jections into the rest of the plan (perhaps cancelinl a rccharlinl IEEE, SI. Louis, MO. (March 1985).
stop). (DoyIe89) Doyle, R., S.Sellers and D.Atkinson, Enhancing
Extemnllandll1arks are also used to monitor the rover's Aerospace Systems Autonomy Through Predictive
progress alonl its traverse. These landmarks, and their assoCiat- Monitoring, 27tll Aerospau Sciences Mutillg, AIAA ,
ed ~nsor I:lsks, can be of several forms including trackinla paper.89-0107, Reno. NV, January 1989.
panicular boulder in the distance, or chcckinlthe time and (Elfes87J Elfes, A., "Sonar ba~cd real world mapping and
odometer readinl when a bump of a panicular size should be navigation", I££E JOllrnal of Robotics er. Automatioll,
encollntered. Panicularly tight monitoring will be scheduled on pp249-265, RA-3. '3,1987.
those pans of the path that might approach any of the vehicle's
s.,fety m.,rgins. Under these conditions (e.g., passing between (Gennery80J Gcnnery, D.R., "Modelling the Environment of an
two narrowly spaced boulders) the execution monitoring sys- Exploring Vehicle by Means of Stereo Vision," AIM-
tem will also select and load the appropriate reaction software 339 (Computer Science Depl. Repon STAN-CS-80-
(e.I ., if both left and right proximity sensors turn on, perform 81'15), Stanford University, (Ph.D. dissenation) (1980).
an emergency Slop and sligh, back-up - to ensure that the (Gennery17J Gennery, D.B., • A Stereo Vision System for an
rover does not become wedged ~tween the two boulders). This Autonomous Vehicle," Fifth International Joint
way, in an emergency, the rover will never have 10 SlOp and Conference on Anificiallntelligence, MIT, 576-582
think before making an appropriate response. (1917).

S. Implementation and Testinl (Lindcn87J Linden, T. &: Glicksman, J., "Contingency

planninl for an autonomous land vehicle," in
The navigation system de$cribed in the previous sections is Procudillll oflh~ IOthlJCAl,AAAl, lJCAl, Milano,
being implemented and tested on two testbed robots at the Jet Italy, pp 1047-1054, 19117.
Propulsion Laboratory. For ease of prototypinl! and testing,
pan.~ of the navigation system are beinl implemented on a
IMcDennott84) McDermott, D. V., Davis, E., Planninl routes
modified Cybermation K2A industrial mobile robot. The K2A through unccnain tenitory, Artificial illt~lIige"ce, v22,
is basically an indoor robot, but is capable of surmollntinl pp. 107-156, 1984.
small obsl:lcles. ll1is allows us to test the algorithms in terrain (Miller86) Miller, D.P. ·Scheduling robot sensors for
that has finer grndations in quality than simply traversable or multisensory tasb." ProcudillllS of the 1986 Robo!.f
non-traversable. ",UI Confer~lIc/!,. SME, Long Beach, CA. (September
The second testbed i$ a custom designed six-wheeled fully 1986).
anicula~d outdoor roho! that should have most of the mobility (Miller871 Miller, D .• P .• Slack, M., G., Efficient Navigation
characteri~tic~ of a real Mars Rover. It is capable of surmount- ll1rough Dynamic Domains, in the PrDCuc/illllS of
ing obsl:lcle~ (using significant power) that are approximately a Worbhop all Spatial R~pr~s~lItalio" and Multi-S~lISOr
full wheel diameter (one meter) in ~ize (MiJler891. The two ve- Ftuioll, SI. Charles Illinois, October 1987.
hicles are software compatible with one another. (Miller89J Miller,D.P., A.Mishkin. K.Lamben. D.Bickler, and
D.8ernard, Autonomous Navillltion and Mobility for.
6 Concluslon~ and Results Planetary Rover, 271h A~rospace Sci~IIC~.r M~~tin~,
AIAA, paper '89-0859, Reno, NV, January 1989.
Due to safety, high level mission loals, and resource condi-
tions it is necessary that the rover perform some stratelic plan- (Munson72J Munson, J.,H., Robot Planning, Execution and
ning. Interaction between a Mars Rover and the Eanh-based Monitoring in an Uncenain Environment, in the
mi5sion control is very costly. In addition to the lilht-time Proc~~dinlls af '''~ 11I1I'TMtia/IQ1 Joilll Conf~r~lIce on
delay, there are also limited communication opportunities, and Artijicialllllelligellce, Stanford California, pp. 338-349,
long deci~ion-m.'kinltimes. It is therefore a very imponant September 1972.
goal to ensure that the rover has two-way contact with the Eanh (Nguyen84J Nl!uyen, V., T/~ Find-Path Prob/~m in the Plane,
only when necess.,ry. An onboard semiautcHlOIllOUS naviption MIT AI Technical Repon 760, 19114.
system is a necessary ingredient for achievinl that goal. To en- (Slack87J Slack, M. G., Miller D. P., Path Planning Through
sure that the plan does not leave the rover stranded in awkward Time and Space in Dynamic Domains, in Procudillg of
spot, or that the plan be executed faultily, resource allocation is Ih~ IOlh IJCAl, AMI,IJCAl, Milano,ll:lly, pp
also needed. However, due to the possibility of a time critical, 1067 -I 070, 1987.
life thrutening situation, the rover will also need reactive abili- (Thorpe841 Thorpe, C . E., Path Relaxation: Path Planning for a
ties. Since the requirements on these abilities change with the Mobile Robot, in Proc~~dillgs of MAl, AAAI, pp.
rover's situation, they should be dynamically installed in the 318-321,1984.
rover as pnn of the planning process to keep them consistent (WikoxS7) Wllcox, W.H., D.B. Gennery, A.H. Mishkin, 8 .e.
with the task and environment context in which the rover finds Cooper, T.B. Lawton, N.K. Lay, and S.P. Katzmann, • A
it~lf. To meet the power and reliability requirements of an ac-
Vision System for. Mars Rover," Proc. SPIE Mobile
tual Mars mission, a robust and passive sensory system is need- Robots 11, vol. 852 (5-6 Nov 1987).
cd. The architectures outlined a"elYe allow long range planning,
yet incorporate real-time performance.