2010 International Conference on Emerging Security Technologies

Supporting Search and Rescue Operations with

UAVs
Sonia Waharte and Niki Trigoni
University of Oxford
Computing Laboratory
Oxford, United Kingdom
e-mail:{sonia.waharte,niki.trigoni}@comlab.ox.ac.uk

Abstract— Search and rescue operations can greatly benefit aftermath of Hurricane Katrina, two UAVs were used to survey
from the use of autonomous UAVs to survey the environment the damaged area in the search for trapped survivors [1]. The
and collect evidence about the position of a missing person. growing recognition of the potential of using UAVs for search
To minimize the time to find the victim, some fundamental
parameters need to be accounted for in the design of the search and rescue applications is supported by an increasing number
algorithms: 1) quality of sensory data collected by the UAVs; 2) of works in the areas of image recognition for victim detection,
UAVs energy limitations; 3) environmental hazards (e.g. winds, path planning and task allocation [2]–[4].
trees); 4) level of information exchange/coordination between In this paper, our focus is the analysis of the performance of
UAVs. different search techniques when the time to find the victim
In this paper, we discuss how these parameters can affect
the search task and present some of the research avenues we (also referred to as target) is the optimization criterion. In
have been exploring. We then study the performance of different particular, we compare the benefit of sharing data between
search algorithms when the time to find the victim is the UAVs with different search techniques based on greedy heuris-
optimization criterion. tics, potential fields, and partially observable Markov decision
process.
I. I NTRODUCTION
The remainder of the paper is organized as follows. We first
Search and rescue operations are often characterized by a describe our hardware platforms in Sec. II. We emphasize
similar set of constraints: time is critical and any delay can some research challenges of search and rescue applications
result in dramatic consequences – potentially human losses; with UAVs in Sec. III. We then present the performance
operational environments are unfriendly, e.g. disaster scenes, evaluation of different search strategies in Sec. IV and Sec. V.
forests, etc. Using Unmanned Aerial Vehicles (UAVs) can We conclude this paper in Sec. VI.
provide a critical support to search and rescue operations.
UAVs are agile, fast, can exhibit autonomous behaviours and II. UAV QUADROTOR
hence perform operations hard to execute by human operators, The hardware platforms we are using are off-the-shelf,
at low operating costs. In a typical scenario, UAVs will be electrically-powered quadrotors. We are currently using several
deployed in an area of interest, perform sensory operations to models from Ascending Technologies (Fig. 2). These vehicles
collect evidence of the presence of a victim, and report their have the advantage of being easy to prepare, control and fly.
collected information to a remote ground station or rescue The payload they can carry is in the order of a few hundred
team (Fig. 1). UAVs have already demonstrated their benefit grammes. With the current battery capacity available, the flight
time of the UAVs is restricted to a few tens of minutes. Each
of these platforms has been redesigned to accommodate a 1.6
GHz Intel Atom board with 1GB of RAM and 16GB SSD
that can provide the computation abilities.
III. R ESEARCH C HALLENGES
To optimize the tasks of the UAVs during the search
operation, several factors need to be accounted for in the
design of the search strategy:
• Quality of sensory data: In search and rescue operations,
to properly assess the quality/trustworthiness of the
Fig. 1. Rescue scenario with UAVs: the UAVs fly over an area the victim
is believed to be located, gather information on its potential location, and information reported to the rescue team is of paramount
transmit it back to a remotely located rescue team. importance. If a UAV is surveying an area, a victim
should not be missed, i.e. the probability of false negative
in search and rescue operations by helping responders to focus should remain low; at the same time, the probability
their search efforts while avoiding hazards. In 2006, in the of false positive, i.e. the probability that a victim is

978-0-7695-4175-4/10 $26.00 © 2010 IEEE 142

DOI 10.1109/EST.2010.31
Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on November 05,2024 at 00:16:48 UTC from IEEE Xplore. Restrictions apply.
 (a) Hummingbird (b) Pelican (c) Falcon

Fig. 2. UAVs in use in the SUAAVE (Sensing Unmanned Autonomous Aerial VEhicles) project.

considered as detected when it is actually not there, and cost of connection (necessity to travel to a given
should remain low to avoid sending a ground rescue rendezvous point). Since it is also critical to regularly
team to a place of no interest. The challenge is then to inform the ground station of the evolution of the
establish accurate models of the quality of the sensory mission, it is important to evaluate the benefit to
data obtained from the UAVs. maintain the network fully or partially connected.
Other questions that require further investigation are:
• Energy limitations: One of the critical aspects of the How frequently should the connections be estab-
UAV platforms we are using is their limited flight time. lished between the UAVs? How frequently should
In preliminary experiments, we have established that the the UAVs report to the ground station?
current consumption when the UAV is flying and when
IV. S EARCH A LGORITHMS
it is hovering is similar. One way to save energy is then
to have the UAV remain idle (rotors off). Depending on In this work, we consider that UAVs are equipped with
the tasks to accomplish and on the capabilities of the downward-pointing cameras to detect victims on the ground.
UAVs, several search strategies can be envisionned such By changing altitudes, UAVs can change the size of their
as alternating periods of flights and periods of rest, or observation areas. The higher they fly, the bigger their obser-
sending only a subset of the UAVs to accomplish certain vation area. But as the UAV flies higher up, the level of detail
tasks whilst the rest remains idle (recharging batteries decreases. Using these observations, we have shown that with
for instance). a single UAV, changing altitudes is a valid control strategy that
can speed up the search process [5], [6]. A question we are
• Environmental hazards. During its flight, a UAV should investigating in this work is the impact of altitude on the search
be capable of avoiding environmental hazards (trees, strategy. When several UAVs are deployed, the complexity of
building, etc.) as well as avoiding collisions with other the problem increases. In an initial work [7], we investigated a
UAVs. It is also necessary to account for the fact that simple fusion algorithm in which all observations made by all
depending on the position of the UAV, some areas on UAVs are exchanged when the UAVs come into communica-
the ground might be occluded. tion range and are locally maintained. We use this mechanism
in this work when communication between UAVs occur. We
• Information sharing: Two aspects of the information also consider differences in quality of sensory data, as well
sharing process need to be considered: data fusion and as we account for the presence of obstacles during the search
network connectivity. operations. We implicitly account for energy constraints by
looking at search strategies that minimize the time to find the
– Data fusion: When multiple UAVs are deployed, the victim.
sensory data they collect can be shared and fused Search algorithms for search and rescue operations should
to generate a complete picture of the environment – be able to cope with the uncertainties of real-world deploy-
which can in turn guide the search process. This task ment. Hence real-time approaches are more appropriate and
is all the more challenging as any solution that will can be divided into three main categories:
be proposed needs to account for limitations in terms 1) Greedy heuristics
of processing, memory storage, energy consumption, 2) Potential-based heuristics
network availability and so on. 3) Partially Observable Markov Decision Process
– Network Connectivity: During deployment, commu- (POMDP) based heuristics
nications can occur between UAVs and between
UAV-Ground Station. The connections can happen To maintain the information on the probability of the victim
opportunistically or can be scheduled. A trade-off location, each UAV maintains a grid-based probabilistic map
needs to be made between reward of establishing a (belief map) composed of cells that represent the discretization
connection (resulting in exchanges of information) of the search space. Each cell contains the probability that the

143

Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on November 05,2024 at 00:16:48 UTC from IEEE Xplore. Restrictions apply.
target is present in it. These maps are commonly used for ap- • Rand: Pick s� at random (the version of Rand where
plications such as surface mapping, exploration or navigation UAVs can communicate is referred to as M S Rand).
(e.g. [8]). • Cov: Pick s that provides the highest coverage.
�

• MaxSum: Pick s for which the sum of the belief of the

�
We adopt a Bayesian approach to keep track of the target
state probability density function. This approach is sensible cells that the UAV would cover by moving at this location
in our context, where non-Gaussian sensor measurements are is maximum. The goal is to maximize the information
considered. The search problem starts with the assumption of a gain.
• MaxMin: Pick s that covers a cell for which the minimum
�
prior probability distribution function that describes the initial
belief of the target location. This can be a Gaussian distribution belief is the highest.
• MaxMax: Pick s that covers a cell that has the highest
�
or a coarse estimate of the target location depending on
environmental features such as rivers or roads (Fig. 3). If no belief.
• MinNeigh: Pick s for which the difference between the
�
prior information is known, we assume a uniform distribution.
After each observation, the probability distribution function of maximum and minimum belief is minimum in order to
the target state is recomputed. We assume that observations are increase our information on the target presence for a
independent and we consider cells inter-dependence in a single greater number of cells at once.
stationary target scenario. We also study variants of the greedy heuristics (referred to
as 1-look ahead) in which the UAV bases its control decision
not on the value of the belief of the victim presence on the
neighboring cell but on the overall gain of moving into that
cell. The optimization criteria are defined as before and the
heuristics we consider are:
• LA MaxSum: Pick s for which the sum of the belief
�

of the cells that the UAV would cover by moving at

this location is maximum. The goal is to maximize the
information gain.
• LA MaxMin: Pick s that covers a cell for which the
�
(a) Search area (b) Belief map
minimum belief is the highest.
Fig. 3. Example of scenario in which the victim is believed to be along a • LA MaxMax: Pick s that covers a cell that has the
�
road (in red) and not in the river (in blue). The search area is mapped into a
grid in which the cells in which the victim is located with high probability highest belief.
• LA MinNeigh: Pick s for which the difference between
are shaded in red and the ones with low probability are shaded in blue. �

the maximum and minimum belief is minimum.

A. Greedy heuristics B. Potential-based Algorithm

The first search strategies we implemented are based on a Potential field methods are based on the following simple
greedy choice of the cell towards which a UAV should fly concept: goals are associated with attractive potentials and
based on a predefined criterion. The general structure of the obstacles are associated with repulsive potentials. At each
heuristics we implemented is described in Alg. 1. The UAV point in the free configuration space (set of configurations that
moves to the neighboring cell s� that has the highest belief (to avoid collisions with obstacles), the potential represents the
increase our confidence of the presence of a victim after the sum of the attractive potentials and repulsive potentials. More
sensing operation). formally, let us consider a UAV at position q. Let Uobs (q)
be the potential created by an obstacle at the UAV location q,
and Ugoal (q) be the potential created by the goal. The potential
Start=s’ function U at q represents the sum of the potentials as shown
Initialize BeliefMap in (1).
while true do U (q) = Uobs (q) + Ugoal (q) (1)
s=s’
GetObservation(s) The force exerted at q is obtained by computing the negative
if target detected then of the gradient of the potential U at q. For example, if
Report Base Station we consider a UAV navigating in a 2-D world with fixed
end orientation, (q = (x, y) in �2 ), we obtain (2):
UpdateBeliefMap(s)  ∂U 
s’=NextMove(s) −−→ ∂x
end F (q) = −∇U (q) =   (2)
∂U
Algorithm 1: General structure of greedy heuristics ∂y

In the design of the attractive and repulsive potential fields,

Different tie-breaking strategies can be considered: one should consider the following factors:

144

Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on November 05,2024 at 00:16:48 UTC from IEEE Xplore. Restrictions apply.
 • Attractive potentials should increase as the UAV moves 48;:

away from the goal. When the UAV reaches the goal, no 4 89:
23!1'-14#5#&1=
forces from the goal should act upon it.
• Repulsive potentials should increase as the UAV comes <

closer to an obstacle. But repulsive potentials should not

48<:
9 ;
affect the UAV motion when it is far from the obstacle.
Practically, in our search and rescue scenario, we assigned
to each cell a weight that depends on its nature: obstacles
23!1'-14#5#&16
and already-visited locations. The weight of each cell

7
that is visited is increased by Kv after each visit. A
potential is computed for each grid cell according to Alg. 2.
Initialization
For each cell that is an obstacle, assign maxWeight
s = start location .)*/(01
%&'(# !"#$%&'(#
%)*+#,-"*(
while true do
assign weight w(s) = min(w(s) + Kv , maxW eight)
for each cell do Fig. 4. Scenario for the implementation of Hierarchical POMDP with 2
compute distance dv to visited cells UAVs flying at 2 different altitudes.
U (cell) = w(cell)/d2v
end
s’=neighboring cell with minimum potential • γ: discount factor.
move UAV to location s’ The goal is to find a set of actions that maximizes
� the expected
if target in s’ then sum of rewards over an infinite horizon E[ t γ t R(st , at )].
Report Base Station
end In our implementation, we considered that each UAV is able
s = s’ to compute its policy based on information locally maintained.
end
Time constraints have been enforced due to the size of the
Algorithm 2: Potential-based search algorithm problem and the number of states that could potentially lead to
hours of computation Since UAVs can fly at different altitudes,
C. Partially Observable Markov Decision Process we can differentiate the tasks of the UAVs as a function of their
Partially Observable Markov Decision Processes (POMDPs) altitude (Fig. 4). This can be modelled using a POMDP with
are a generalisation of a Markov Decision Process (MDP) to different observations models and set of actions for each set
situations where the system can be modelled by a MDP, but of UAVs (referred to as Hierarchical POMDP).
the underlying states are unobservable. Instead, the state dis- For simplicity, we consider that the UAVs can fly at only two
tribution must be inferred based on a model of the world and distinct altitudes.
local observations where, typically, sensors provide partial and We evaluate three strategies:
noisy information about the world. The POMDP framework 1) POMDP: UAVs fly at the same altitude but do not
is general enough to model a variety of real-world sequential exchange any information.
decision processes. An exact solution to a POMDP yields the 2) MS POMDP: UAVs fly at the same altitude and ex-
optimal action for each possible belief over the world states. change information when in communication range.
The optimal action maximizes (or minimizes) the expected 3) Hierarchical POMDP (HPOMDP): UAVs fly at different
reward (or cost) of the agent over a time horizon. The sequence altitudes and exchange information when in communi-
of optimal actions is known as the optimal policy and is chosen cation range.
so as to optimise the expected reward (or cost).
V. S IMULATIONS
Formally, a POMDP is represented by the following n-tuple:
{S, A, O, b0 , T.Ω, R, γ}, with: We consider 3D terrains discretized into cells of unit length.
In our first evaluations, we only consider trees as obstacles that
• S: finite set of discrete states,
we model in 3D space by cylinders (Fig. 5). Each cylinder is
• A: set of actions,
further discretized into an integer number of grid cells. In our
• O: set of observations,
model, we account for the fact that, depending on its position
• b0 : initial belief states distribution,
relative to the obstacles, a UAV may not be able to observe a
• T (s, a, s ): probability of transition from s to s when
� �
certain number of cells on the ground (shadowing effect).
taking action a,
We consider a sensing model accounting for false positive
• Ω(o, s , a): probability of observing o from state s after
� �
and false negative:
taking action a,
• Probability(sensing target at height h—target)=1 − βh
• R(s, a): reward when executing action a in state s,

145

Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on November 05,2024 at 00:16:48 UTC from IEEE Xplore. Restrictions apply.
 !"#$%#&"'()*+%,%&'&()*+&,&-&./00+&1&#&.2/30.&

 
 "
 
  





  

$ #
 

!


Fig. 6. Test configuration: top view of the test area with 2 UAVs at the
Fig. 5. Example of the effect of the presence of obstacles on the sensing bottom left corner, 5 trees and 1 goal (victim) centrally located.
range: a tree (cylinder) blocks a UAV field of view, and makes some areas
on the ground not observable.

TABLE I 200sec. A UAV periodically broadcasts its position and other

O BSERVATION MODEL relevant information (such as belief maps). Decision on the
next step to go towards to is taken when the UAV arrives
Altitude (h) αh βh close to the center of a cell. All UAVs fly at the same speed
5m 0.243599 0.000000 regardless of their altitude.
10m 0.028369 0.000000 To solve POMDPs, we used the ZMDP software [9].
15m 0.026099 0.046211 The topology we simulated is a 10x10 grid, with 5 trees
20m 0.001110 0.046745 randomly positioned. 2 UAVs start their search at nearby
position. One target is located in the middle of the test area
(Fig. 6).
• Probability(not sensing target at height h—target)=βh We assume that the mission is successfully completed when
• Probability(not sensing target at height h—no target)=1− the target has been positively identified. The result of the
αh simulations is depicted in Fig. 7.
• Probability(sensing target at height h—no target)=αh We can see that the approach based on POMDP and look-
ahead strategies performs the best compared to the greedy
αh (0 ≤ αh ≤ 1) and βh (0 ≤ βh ≤ 1) represent the
heuristics and the potential-based approaches. Greedy heuris-
false alarm and missed detection probabilities and vary as
tics perform in general poorly as the control decision of the
a function of the height h of the helicopter. From our field
UAVs is based only on local information which can lead
experiments [5], we obtained the values of the observation
them to stay longer in areas that have already been explored
model summarized in Table I.
instead of going to remote areas that have not been explored
The sensing areas on the ground cover a set of M (h)
yet. Potential-based approaches also generally perform poorly
cells, where h is the altitude of the UAV considered. The
due to the known problem of local minima that these search
dimension of the sensing area increases with the height of the
techniques suffer from. Additional heuristics are required to
UAVs. UAVs at high altitude have a greater sensing coverage
help the UAVs escape from these local minima but this
than UAVs at low altitude but they also have lower sensing
unavoidably impacts the time to locate the target.
resolution (smaller detection probability).
To first assess the performance of search algorithms before This simple simulation scenario demonstrates that trying to
trials with real hardware, we used a Matlab/Simulink-based estimate the best search paths by sharing information from
simulator for real-time control systems. This allows to accu- sensory operations and by exploiting the acquired or prior
rately simulate distributed systems with realistic radio trans- knowledge of the possible victim location can help speed up
mission models. The characteristics of the wireless channel the search task. But this result has a computational cost that
are: can not be neglected. With greedy heuristics, the computa-
tional cost is minimal as the control decision is based on the
• Transmit power = -30 dBm
value of the belief of the victim position in direct neighboring
• Receiver threshold = -48 dBm
locations. With potential-based searches, the computational
• Path loss coefficient = 3.5
cost is linear with the number of cells. Solving a POMDP in an
Please note that the wireless settings have been adjusted to exact fashion is an intractable problem [10]. To find a solution
scale with the test areas. The grid cells are 1x1x1 cube meters for a POMDP requires to consider each possible action and
and the maximum wireless transmission range is 2.27m. We each possible observation, which implies that the cost of
considered squared coverage areas that correspond to a 3x3 a solution grows exponentially with the desired planning
grid for the UAVs evolving at level 2, and 4x4 grid for the horizon. Rather than advocating for the use of POMDP or
UAVs evolving at level 1. The maximum simulation time is look-ahead strategies for search tasks, this simple case scenario

146

Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on November 05,2024 at 00:16:48 UTC from IEEE Xplore. Restrictions apply.
 Time to find the victim [in s] 70

60

50

40

30

20

10

0
Rand MS_Rand Cov MaxSum MaxMin MaxMax MinNeigh LA_MaxSum LA_MaxMin LA_MaxMax LA_MinNeigh Pot MS_Pot POMDP MS_POMDP HPOMDP

Greedy Potential POMDP

Search Strategy

Fig. 7. Simulation results showing the time to find the victim (completion time) with different search techniques.

[7] S. Waharte, N.Trigoni, and S. J. Julier, “Coordinated search with a

illustrates the potential of POMDP when the number of states swarm of uavs,” in SECON, 2009.
remains limited, and the importance of exploiting as much as [8] A. Elfes, “Using occupancy grids for mobile robot perception and
possible the information acquired by one or multiple UAVs to navigation,” Computer, vol. 22, no. 6, pp. 46–57, 1989.
[9] ZMDP, “http : //www.cs.cmu.edu/ trey/zmdp/.”
predict more accurately a victim location. [10] O. Madani, S. Hanks, and A. Condon, “On the undecidability of
probabilistic planning and infinite-horizon partially observable markov
VI. C ONCLUSION decision problems,” in AAAI ’99, 1999, pp. 541–548.
Unmanned aerial vehicles can provide a critical support for
search and rescue operations. However in order to achieve
their full potential, it is necessary to properly account for all
the parameters that can affect the flight of the UAVs such as
quality of sensory operations (that can depend on the position
of the UAVs for instance), energy constraints, environmental
hazards or data sharing constraints between UAVs and rescue
teams.
In this paper, we studied different search strategies based
on greedy heuristics, potential-based algorithms and partially
observable Markov decision process to design the control
strategy of several UAVs. The evaluation criterion was the time
taken by the UAVs to find the victim. Our preliminary results
show the importance of exploiting as much as possible the in-
formation obtained during the search operations and illustrate
the potential of POMDP in a simple scenario. However this
comes at a high computational cost that needs to be accounted
for.
In future work, we intend to investigate more complex sce-
narios and to account for energy and connectivity constraints.
We also intend to study how search algorithms based on
POMDP scale when the number of states increases.
R EFERENCES
[1] NSF - Hurricane Katrina, “http : //www.nsf.gov/news.”
[2] P. Doherty and P. Rudol, “A uav search and rescue scenario with human
body detection and geolocalization,” in In 20th Joint Conference on
Artificial Intelligence (AI07), 2007.
[3] M. Goodrich, B. Morse, D. Gerhardt, J. Cooper, M. Quigley, J. Adams,
and C. Humphrey, “Supporting wilderness search and rescue using a
camera-equipped mini uav: Research articles,” J. Field Robot., vol. 25,
no. 1-2, pp. 89–110, 2008.
[4] B. Morse, C. Engh, and M. Goodrich, “Uav video coverage quality maps
and prioritized indexing for wilderness search and rescue,” in HRI. New
York, NY, USA: ACM, 2010, pp. 227–234.
[5] A. Symington, S. Waharte, S. J. Julier, and N. Trigoni, “Probabilistic
target detection by camera-equipped uavs,” in ICRA, 2010.
[6] N. T. S. Waharte, A. Symington, “Probabilistic search with agile uavs,”
in ICRA, 2010.

147

Authorized licensed use limited to: UNIVERSITY OF BRISTOL. Downloaded on November 05,2024 at 00:16:48 UTC from IEEE Xplore. Restrictions apply.