IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 1

A Survey of Research on
Cloud Robotics and Automation
Ben Kehoe Student Member, IEEE, Sachin Patil Member, IEEE,
Pieter Abbeel Senior Member, IEEE, Ken Goldberg Fellow, IEEE

Abstract—The Cloud infrastructure and its extensive set of

Internet-accessible resources has potential to provide significant
benefits to robots and automation systems. We consider robots
and automation systems that rely on data or code from a network
to support their operation, i.e., where not all sensing, computa-
tion, and memory is integrated into a standalone system. This
survey is organized around four potential benefits of the Cloud:
1) Big Data: access to libraries of images, maps, trajectories,
and descriptive data, 2) Cloud Computing: access to parallel
grid computing on demand for statistical analysis, learning, and
motion planning, 3) Collective Robot Learning: robots sharing
trajectories, control policies, and outcomes, and 4) Human
Computation: use of crowdsourcing to tap human skills for
analyzing images and video, classification, learning, and error
recovery. The Cloud can also improve robots and automation
systems by providing access to a) datasets, publications, models, Fig. 1. The Cloud has potential to enable a new generation of robots and
benchmarks, and simulation tools, b) open competitions for automation systems to use wireless networking, Big Data, Cloud Computing,
designs and systems, and c) open-source software. This survey statistical machine learning, open-source, and other shared resources to
includes over 150 references on results and open challenges. improve performance in a wide variety of applications including assembly,
A website with new developments and updates is available at: inspection, driving, warehouse logistics, caregiving, package delivery, house-
http://goldberg.berkeley.edu/cloud-robotics/ keeping, and surgery.

Note to Practitioners—Most robots and automation systems

still operate independently using onboard computation, memory, or hardware updates. The Cloud also provides economies of
and programming. Emerging advances and the increasing avail-
scale and facilitates sharing of data across applications and
ability of networking in the “Cloud” suggests new approaches
where processing is performed remotely with access to dynamic users [122].
global datasets to support a range of functions. This paper Cloud Robot and Automation systems can be broadly
surveys research to date. defined as follows: Any robot or automation system that
Index Terms—Cloud Automation, Cloud Robotics, Big Data, relies on either data or code from a network to support
Cloud Computing, Open Source, Crowdsourcing its operation, i.e., where not all sensing, computation, and
memory is integrated into a single standalone system. This
definition is intended to include future systems and many
I. I NTRODUCTION
existing systems that involve networked teleoperation or net-
As illustrated in Fig. 1, the Cloud has potential to enhance worked groups of mobile robots such as UAVs [100], [116]
a broad range of robots and automation systems. The National or warehouse robots [44], [96] as well as advanced assembly
Institute of Standards and Technology (NIST) defines the lines, processing plants, and home automation systems, and
Cloud as “a model for enabling ubiquitous, convenient, on- systems with computation performed by humans [132], [155].
demand network access to a shared pool of configurable re- Due to network latency, variable quality of service, and
sources (e.g., servers, storage, networks, applications, and ser- downtime, Cloud Robot and Automation systems often include
vices) that can be rapidly provisioned and released with mini- some capacity for local processing for low-latency responses
mal management effort or service provider interaction” [115]. and during periods where network access is unavailable or
An example is the online word processing capabilities offered unreliable. This is not a binary definition; there are degrees to
by Google Docs. One can send Microsoft Word documents which any system will fit under this definition.
over the Internet, but Google Docs differs in that the document The Google self-driving car exemplifies the idea. It in-
and software does not reside locally. the data and code is dexes maps and images collected and updated by satellite,
stored in the Cloud using remote server farms with shared Streetview, and crowdsourcing from the Cloud to facilitate
processors and memory. This is helpful because one does accurate localization. Another example is the Kiva Systems
not have to worry about maintenance, outages, and software pallet robot for warehouse logistics. These robots communi-
cate wirelessly with a local central server to coordinate routing
Manuscript received Tuesday 25th November, 2014.
Authors are with the University of California, Berkeley, CA, USA, (e-mail: and share updates on detected changes in the environment.
{benk, sachinpatil, pabbeel, goldberg}@berkeley.edu). In 2010, James Kuffner coined the term “Cloud Robotics”
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 2

and described a number of potential benefits [98]. An article

in IEEE Spectrum quickly followed [70] and Steve Cousins
summarized the concept as “No robot is an island.” The next
section considers the history of this important idea.
This survey is organized around four potential benefits from
the Cloud: 1) Big Data: access to remote libraries of images,
maps, trajectories, and object data, 2) Cloud Computing: ac-
cess to parallel grid computing on demand for statistical analy-
sis, learning, and motion planning, 3) Collective Robot Learn-
ing: robots sharing trajectories, control policies, and outcomes,
and 4) Human computation: using crowdsourcing access to
remote human expertise for analyzing images, classification,
learning, and error recovery. This survey also cites examples
where the Cloud can enhance robotics and automation systems
by facilitating access to a) datasets, publications, models,
benchmarks, and simulation tools, b) open competitions for
designs and systems, and c) open-source software. Fig. 2. The RoboEarth systems architecture designed to allow robots to share
data and learn from each other [22], [156]. (Image reproduced with permission
from authors).
II. A B RIEF H ISTORY
researchers including the organizers of this Special Issue of the
The value of networking to connect machines in manufac- IEEE Transactions on Automation Science and Engineering.
turing automation systems was recognized over 30 years ago. Cloud Robotics and Automation is related to several other
In the 1980’s, General Motors developed the Manufacturing new initiatives. The “Internet of Things” [32], a term also
Automation Protocol (MAP) [80]. A diverse set of incom- introduced in 2010, describes how RFID and inexpensive
patible proprietary protocols were offered by vendors until a processors could be incorporated into a vast array of robots
shift began in the early 1990’s when the World Wide Web and physical objects from inventory items to household ap-
popularized the HTTP over IP protocols [121]. pliances [110] to allow them to communicate and share
In 1994, the first industrial robot was connected to the Web information.
with an intuitive graphical user interface that allowed visitors The term “Industry 4.0,” introduced in Germany in 2011,
to teleoperate the robot via any internet browser [63]. In the predicts a fourth industrial revolution that will use networking
mid and late 1990’s, researchers developed a series of web to follow the first (mechanization of production using water
interfaces to robots and devices to explore issues such as user and steam power), the second (mass production with electric
interfaces and robustness [64], [65] that initiated the subfield power), and the third (use of electronics to automate produc-
of “Networked Robotics” [66], [112]. tion) industrial revolutions [11].
In 1997, work by Inaba et al. on “remote brained robots” In 2012, General Electric introduced the term “Industrial
described the advantages of remote computing for robot con- Internet”, to describe new efforts where industrial equipment
trol [79]. such as wind turbines, jet engines, and MRI machines connect
In May 2001, the IEEE Robotics and Automation So- over networks to share data and processing for industries
ciety established the Technical Committee on Networked including energy, transportation, and healthcare [55], [93]. For
Robots [10] which organized a number of workshops. Chapters example, GE is using sensor readings from aircraft engines to
of the Springer Handbook on Robotics were focused on optimize fuel consumption under a myriad of conditions [59].
Networked Tele-robots (where robots are operated remotely by The power of Big Data and Cloud Computing are extensively
humans using global networks) and Networked Robots (where being used by many industries [25], [113], and are being
robots communicate with each other using local networks) harnessed to optimize water usage for irrigation [52].
respectively [99], [143]. Many related projects are emerging. In August 2014,
In 2009, the RoboEarth project was announced. It en- Ashutosh Saxena announced his goal to build a “RoboBrain:
visioned “a World Wide Web for robots: a giant network a large-scale computational system that learns from publicly
and database repository where robots can share information available Internet resources, computer simulations, and real-
and learn from each other about their behavior and environ- life robot trials.”
ment” [22], [156] as illustrated in Fig. 2. Under a major
European Union grant, the RoboEarch research team devel- III. B IG DATA
oped a series of system architectures for service robotics [30], The Cloud can provide robots and automation systems
[53], developing Cloud networking [74], [87], and computing with access to vast resources of data that are not possible
resources [78] to generate 3D models of environments, speech to maintain in onboard memory. “Big Data” describes “data
recognition, and face recognition [148]. that exceeds the processing capacity of conventional database
As noted in the previous section, James Kuffner introduced systems” [54] including images, video, maps, real-time net-
the term “Cloud Robotics” in 2010. This broader term sup- work and financial transactions [102], and vast networks of
planted earlier terminology and has been adopted by many sensors [159].
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 3

Fig. 4. Google’s object recognition system combines an enormous dataset of

images and textual labels with machine learning to facilitate object recognition
in the Cloud [9], [98]. (Image reproduced with permission).

Cloud

Google Google
Fig. 3. Data can be collected from many sources as shown in this schematic Object Recognition Cloud Storage
architecture for the Mobile Millennium, a Cloud-based transportation system Engine Object Label
Grasp
that combines streaming data from taxis, maps, and road-based sensors [77]. Execution
Mobile Millennium uses the Big Data and Collective Robot Learning aspects Results
Candidate
Image 3D CAD Grasps
of Cloud Robotics and Automation. (Image reproduced with permission from Model
Robots
authors).
Select Feasible
Pose Grasp with
Camera 3D Sensor
Point Cloud Estimation Highest Success
A recent U.S. National Academy of Engineering Report Probability

summarizes many research opportunities and challenges cre-

ated by Big Data [41] and other challenges are summarized Fig. 5. System Architecture for Cloud-based object recognition for grasping.
in [29], [165]. For example, sampling algorithms can provide The robot captures an image of an object and sends via the network to the
Google object recognition server. The server processes the image and returns
reasonable approximations to queries on large datasets to keep data for a set of candidate objects, each with pre-computed grasping options.
running times manageable [38], but these approximations can The robot compares the returned CAD models with the detected point cloud to
be seriously affected by “dirty data” [158]. refine identification and to perform pose estimation, and selects an appropriate
grasp. After the grasp is executed, data on the outcome is used to update
Hunter et al. [77] presents algorithms for a Cloud-based models in the Cloud for future reference [89]. This project uses the Big Data,
transportation system called Mobile Millennium, which uses Cloud Computing, and Collective Robot Learning aspects of Cloud Robotics
the GPS in cellular phones to gather traffic information, and Automation. (Image reproduced with permission).
process it, and distribute it and also to collect and share data
about noise levels and air quality (see Fig. 3).
Large datasets can facilitate machine learning, as has been Fig. 5.
demonstrated in the context of computer vision. Large-scale The RoboEarth project stores data related to objects and
image datasets such as ImageNet [50], PASCAL visual object maps for applications ranging from object recognition to mo-
classes dataset [56], and others [142], [151] have been used bile navigation to grasping and manipulation (see Fig. 2) [156].
for object and scene recognition. By leveraging Trimble’s The Columbia Grasp dataset [67], the MIT KIT object
SketchUp 3D warehouse, Lai et al. reduced the need for dataset [88], and the Willow Garage Household Objects
manually labeled training data [101]. Using community photo Database [40] are available online and have been used to
collections, Gammeter et al. created an augmented reality evaluate different aspects of grasping algorithms, including
application with processing in the Cloud [57]. Combining grasp stability [45], [46], robust grasping [162], and scene
internet images with querying a local human operator, Hidago- understanding [128]. Dalibard et al. attach “manuals” of
Pena et al. provided a more robust object learning tech- manipulation tasks to objects [43].
nique [72]. Deep learning is a technique using many-layered One research challenge is defining cross-platform formats
neural networks that can take advantage of Big Data [49], for representing data. While sensor data such as images and
and has been used for computer vision [97], [140] and grasp- point clouds have a small number of widely-used formats, even
ing [104]. relatively simple data such as trajectories have no common
Grasping is a persistent challenge in robotics: determin- standards yet but research is ongoing [129], [147], [149].
ing the optimal way to grasp a newly encountered object. Another challenge is working with sparse representations
Cloud resources can facilitate incremental learning of grasp for efficient transmission of data, e.g., algorithms for sparse
strategies [40], [120] by matching sensor data against 3D motion planning for robotic and automation systems [51],
CAD models in an online database. Examples of sensor data [105].
include 2D image features [75], 3D features [68], and 3D point Large datasets collected from distributed sources are often
clouds [39], as well as demonstrations [94]. “dirty” with erroneous, duplicated, or corrupted data [6],
Google Goggles [9], a free image recognition service for [158], such as 3D position data collected during robot cali-
mobile devices (see Fig. 4), has been incorporated into a bration [111]. New approaches are required that are robust to
Cloud-based system for robot grasping [89] as illustrated in dirty data.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 4

Fig. 6. A Cloud-based approach to geometric shape uncertainty for grasping.

(Top) Uncertainty in object pose and shape. (Bottom) Computed push grasps.
Kehoe et al. use sampling over uncertainty distributions to find a lower bound
on the probability of success for grasps [90]–[92].

IV. C LOUD C OMPUTING

Massively-parallel computation on demand is now widely
available [29] from commercial sources such as Amazon’s
Elastic Compute Cloud [1], [2], Google’s Compute Engine [8],
and Microsoft’s Azure [12]. These systems provide access Fig. 7. A Cloud framework for robot navigation using cooperative tracking
to tens of thousands of remote processors for short-term and mapping (C2 TAM). Riazuelo et al. demonstrate computer intensive
computing tasks [105], [106]. These services were originally bundle adjustment for navigation using simultaneous localization and map-
ping (SLAM) performed in the Cloud [134]–[136]. (Image reproduced with
used primarily by web application developers but have increas- permission).
ingly been used in scientific and technical high performance
computing (HPC) applications [20], [86], [114], [152].
Uncertainty in sensing, models, and control is a central
issue in robotics and automation [62]. Such uncertainty can be
modeled as perturbations in position, orientation, shape, and
control. Cloud Computing is ideal for sample-based Monte-
Carlo analysis. For example, parallel Cloud Computing can be
used to compute the outcomes of the cross-product of many
possible perturbations in object and environment pose, shape,
and robot response to sensors and commands [154]. This idea
is being explored in medicine [157] and particle physics [141]. Fig. 8. Distributed sampling-based motion planning. A roadmap of trees
Cloud-based sampling can be used to compute robust grasps for motion planning in high-dimensional spaces. Plaku et al. show that their
in the presence of shape uncertainty [90]–[92] (see Fig. 6). planner can “easily solve high-dimensional problems that exhaust resources
available to single machines” [126]. (Image reproduced with permission).
This grasp planning algorithm accepts as input a nominal
polygonal outline with Gaussian uncertainty around each
vertex and the center of mass and uses parallel-sampling to
[137], and mapping [119], [136] (see Fig. 7. Image processing
compute a grasp quality metric based on a lower bound on
in the Cloud has been used for assistive technology for the
the probability of achieving force closure.
visually impaired [36] and for senior citizens [58].
Cloud Computing has potential to speed up many
computationally-intensive robotics and automation systems Bekris et al. [33] propose an architecture for efficiently
applications such as robot navigation by performing SLAM in planning the motion of new robot manipulators designed for
the Cloud [134], [135] as illustrated in Fig. 7 and next-view flexible manufacturing floors in which the computation is split
planning for object recognition [125]. Cloud-based formation between the robot and the Cloud.
control of ground robots has also been demonstrated [153]. It is important to acknowledge that the Cloud is prone
For optimal sampling-based motion planning methods such to varying network latency and quality of service. Some
as RRT*, Cloud Computing is useful to generate the graphs; applications are not time sensitive, such as decluttering a
it is also important to recognize that these graphs can grow room or pre-computing grasp strategies or offline optimization
rapidly so algorithms for graph reduction are needed to facil- of machine scheduling, but many applications have real-time
itate data transfer as illustrated in Fig. 8. demands [83] and this is an active area of research [26], [27],
The Cloud also facilitates video and image analysis [123], [95], [109].
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 5

giving insight on their perspective of their current state” [21].

The RoboEarth and RoboBrain databases in Section III are
designed to be updated with new information from connected
robots. The RoboBrain project “learns from publicly available
Internet resources, computer simulations, and real-life robot
trials.” [16]
KIVA Systems [44], [96] uses hundreds of mobile platforms
to move pallets in warehouses using a local network to
coordinate motion and update tracking data.

VI. H UMAN C OMPUTATION : C ROWDSOURCING AND C ALL

C ENTERS
Human skill, experience, and intuition is being tapped to
solve a number of problems such as image labeling for
Fig. 9. Schematic architecture of CloudThink. Wilhem et al. developed an computer vision [40], [87], [98], [155], learning associa-
open-standard for self-reporting sensing devices such as sensors mounted
in automobiles. Cloud-enabled storage of sensor network data can enable
tions between object labels and locations [138], and gather-
collaborative sharing of data for traffic routing and other applications [163]. ing data [48]. Amazon’s Mechanical Turk is pioneering on-
CloudThink uses the Collective Robot Learning aspect of Cloud Robotics and demand “crowdsourcing” with a marketplace where tasks that
Automation. (Image reproduced with permission from authors).
exceed the capabilities of computers can be performed by
V. C OLLECTIVE ROBOT L EARNING human workers. In contrast to automated telephone reservation
systems, consider a future scenario where errors and excep-
The Cloud facilitates sharing of data for robot learning by tions are detected by robots and automation systems which
collecting data from many instances of physical trials and then contact humans at remote call centers for guidance.
environments. For example robots and automation systems Research projects are exploring how this can be used for
can share initial and desired conditions, associated control path planning [73], [82], to determine depth layers, image nor-
policies and trajectories, and importantly: data on the resulting mals, and symmetry from images [61], and to refine image seg-
performance and outcomes. mentation [84]. Researchers are working to understand pricing
models [145] and apply crowdsourcing to grasping [144] (see
Fig. 12). Knowledge-based solutions are being explored for
industrial automation as well [146].
Networked robotics has a long history of allowing robots to
be remotely teleoperated by humans over the web [63], and
the expanded resources of the Cloud enables new research into
remote human operation [103], [144], [164] (see Fig. 11).

VII. O PEN -S OURCE AND O PEN -ACCESS

Fig. 10. (Left) Schematic architecture of the Lightning path planning frame-
work. Berenson et al. show a system that is able to learn from experience from The Cloud supports the evolution of Cloud Robotics and
pre-computed motion plans, which could be stored in the Cloud. The planner Automation by facilitating human access to a) datasets, publi-
attempts to find a brand-new plan as well as find an existing plan for a problem
similar to the current one. Whichever finishes first is chosen [35]. Lightning cations, models, benchmarks, and simulation tools, b) open
uses the Big Data, Cloud Computing, and Collective Robot Learning aspects competitions for designs and systems, and c) open-source
of Cloud Robotics and Automation. (Image reproduced with permission from software.
authors).
The success of open source software [42], [124] is now
The “Lightning” framework (see Fig. 10), proposes a frame- widely accepted in the robotics and automation community. A
work for Collective Robot Learning by indexing trajectories
from many robots over many tasks and using Cloud Comput-
ing for parallel planning and trajectory adjustment [35].
Such systems can also be expanded to global networks to
facilitate shared path planning, including traffic routing as
shown in Fig. 9.
For grasping [37], grasp stability of finger contacts can be
learned from previous grasps on an object [45]. Sharing data
through Collective Robot Learning can also improve the ca-
pabilities of robots with limited computational resources [69].
The MyRobots project [13] from RobotShop proposes a Fig. 11. Tiered human assistance using Cloud-based resources for teleop-
eration. Leeper et al. developed an interface for operators to control grasp
“social network” for robots: “In the same way humans benefit execution using a set of different strategies. The results indicate humans are
from socializing, collaborating and sharing, robots can benefit able to select better and more robust grasp strategies [103], [164]. (Image
from those interactions too by sharing their sensor information reproduced with permission).
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 6

Fig. 12. Crowdsourcing object identification to facilitate robot grasping. Sorokin et al. developed a Cloud robot system that incorporates Amazon’s Mechanical
Turk to obtain semantic information about the world and subjective judgments [144]. This work uses the Human Computation aspect of Cloud Robotics and
Automation. (Image reproduced with permission from authors).

Fig. 13. The DARPA Robotics Challenge (DRC) used CloudSim, an open- Fig. 14. Lollibot, designed by Tom Tilley of Thailand, won the Grand
source Cloud-based simulation platform for testing the performance of the Prize in the $10 Educational Robot Design Challenge organized by the
Atlas humanoid robot (shown) on a variety of disaster response tasks [5], African Robotics Network. This design can be built from surplus parts for
[7]. The Cloud permits running interactive, real-time simulation tasks in US $8.96. [150]. (Image reproduced with permission).
parallel for purposes such as predicting and evaluating performance, validating
design decisions, optimizing designs, and training users. This competition also a widely-used open source microcontroller platform with many
resulted in enabling sharing of robotics research efforts. (Image reproduced different sensors and actuators available, and has been used in
with permission).
many robotics projects. The Raven [71] is an open-architecture
laparoscopic surgery robot developed as a research platform
an order of magnitude less expensive than commercial surgical
primary example is ROS, the Robot Operating System, which robots [23]. Recent advances in 3D printing (also known as
provides libraries and tools to help software developers create additive manufacturing) are poised to have a major impact on
robot applications [14], [18], [131]. ROS has also been ported many fields, including development of open source hardware
to Android devices [19]. ROS has become a standard akin designs [60], [81], [108].
to Linux and is now used by almost all robot developers in The Cloud facilitates open challenges and design competi-
research and many in industry, with the ROS Industrial project tions that can draw on a diverse and geographically distributed
created to support these users [17]. population of innovators.
Additionally, many simulation libraries for robotics are The DARPA Robotics Challenge (DRC) is “a competition
now open source, which allows students and researchers to of robot systems and software teams vying to develop robots
rapidly set up and adapt new systems and share the resulting capable of assisting humans in responding to natural and
software. There are many open source simulation libraries, man-made disasters”, supported by NIST and the Southwest
including Bullet [4], a physics simulator originally used for Robotics Institute (SwRI) [24]. The DRC simulator is provided
video games, OpenRAVE [15] and Gazebo [7], simulation to all contestants through CloudSim, an open-source Cloud-
environments geared specifically towards robotics, OOPSMP, based simulation platform for testing the performance of the
a motion-planning library [127], and GraspIt!, a grasping Atlas humanoid robot (shown in Fig. 13) on a variety of
simulator [117]. The open source nature of these libraries disaster response tasks [5], [7]. The Cloud permits running
allows them to be modified to suit applications and they were interactive, real-time simulation tasks in parallel for purposes
not originally designed for. such as predicting and evaluating performance, validating
Another exciting trend is in open source hardware, where design decisions, optimizing designs, and training users [28].
CAD models and the technical details of construction of de- Another example of an open competition is the “Ultra-
vices are made freely available [47]. The Arduino project [3] is Affordable Educational Robot Challenge” organized by the
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 7

African Robotics Network with support from the IEEE including application frameworks and database access, while
Robotics and Automation Society in the summer of 2012. restricting the choice of programming languages, system ar-
It attracted 28 designs from around the world including the chitectures, and database models that can be used. Software as
Grand Prize winning design shown in Fig. 14 where a modified a Service (SaaS), the highest level of structure, is exemplified
surplus Sony game controller uses the vibration motors to drive by the difference between Google Docs, a Cloud-based word
wheels and lollipops as inertial counterweights for contact processor, and Microsoft Word, which must be downloaded
sensing by the thumb switches. This robot can be built from and installed locally.
surplus parts for US $8.96 [150]. The RoboEarth project includes a Cloud Computing plat-
form called Rapyuta [118], which is a Platform as a Service
VIII. C HALLENGES AND F UTURE D IRECTIONS (PaaS) framework for moving computation off of robots and
into the Cloud. It also connects to the RoboEarth knowledge
Using the Cloud for robotics and automation systems intro- repository, integrating the Big Data aspect. We believe that
duces many new challenges. The connectivity inherent in the this PaaS approach can be extended to use the Software
Cloud raises a range of privacy and security concerns [133], as a Service (SaaS) paradigm, which offers many potential
[139]. These concerns include data generated by Cloud- advantages for robots and automation systems [85]. With SaaS,
connected robots and sensors, especially as they may include an interface allows data to be sent to a server that processes
images or video or data from private homes or corporate it and returns outputs, which relieves users of the burden
trade secrets [130], [161]. Cloud Robotics and Automation of maintaining data and software and hardware and allows
also introduces the potential of robots and systems to be companies to control proprietary software.
attacked remotely: a hacker could take over a robot and use
We call this approach Robotics and Automation as a Service
it to disrupt functionality or cause damage. For instance,
(RAaaS). To illustrate the concept, consider two scenarios for
researchers at University of Texas at Austin demonstrated that
a graduate student setting up a robot workcell. The workcell
it is possible to hack into and remotely control UAV drones
contains a 7-DoF Fanuc industrial arm with parallel-jaw grip-
via inexpensive GPS spoofing systems in an evaluation study
per and a Microsoft Kinect RGBD sensor. The purpose of the
for the Department of Homeland Security (DHS) and the
workcell is to pick up and inspect parts as they come down
Federal Aviation Administration (FAA) [76]. These concerns
an assembly line, requiring object recognition and localization,
raise new regulatory, accountability and legal issues related
grasp planning, and motion planning.
to safety, control, and transparency [107], [130]. The “We
Robot” conference is an annual forum for ethics and policy In Scenario 1 (today with ROS), the software runs locally.
research [160]. ROS (Robot Operating System), the well-known open-source
On the technical front, new algorithms and methods are library of robotics software [131], provides access to over
needed to cope with time-varying network latency and Quality 2000 open-source ROS packages. Currently however, ROS is
of Service. Faster data connections, both wired internet con- only supported on the Ubuntu Linux operating system. While
nections and wireless standards such as LTE [31], are reducing Ubuntu is popular, the computers available to the graduate
latency, but algorithms must be designed to degrade gracefully student run OSX. Many stable ROS packages are provided
when the Cloud resources are very slow, noisy, or unavailable as packages, which simplifies installation, but some software
[34]. For example, “anytime” load balancing algorithms for is only available as a source distribution, which requires
speech recognition on smart phones send the speech signal to the download and installation of dependencies. The graduate
the Cloud for analysis and simultaneously process it internally student must set up a new machine with Ubuntu and resolve all
and then use the best results available after a reasonable delay. library dependencies, including those that conflict with other
Similar algorithms will be needed for robotics and automation packages.
systems [35]. In contrast, Scenario 2 (in the future with RAaaS), the
New algorithms are also needed that scale to the size of analysis and planning software runs in the Cloud. The graduate
Big Data, which often contain dirty data that requires new student visits a website to input the robot, sensor, and gripper
approaches to clean or sample effectively [6], [158]. When the models. She then selects her desired object recognition and
Cloud is used for parallel-processing, it is vital that algorithms localization, motion planning, and grasping algorithms, and
oversample to take into account that some remote processors uses a graphical interface to connect these algorithms into a
may fail or experience long delays in returning results. When pipeline. Her robot begins sending up data in the form of
human computation is used, algorithms are needed to filter point clouds from the Kinect. The robot receives and executes
unreliable input and balance the costs of human intervention motion plans and grasps, reporting back outcomes to the
with the cost of robot failure. Cloud-based pipeline, which are combined with feedback from
Moving robotics and automation algorithms into the Cloud other robots to improve the Cloud-based software parameters
requires frameworks that facilitate this transition. The Cloud over time. We are excited about the potential of such a
provides three possible levels at which a framework could system and actively working with others on developing its
be implemented [115]. The lowest level is Infrastructure as components.
a Service (IaaS), where bare operating systems are provided This survey is based on research available in August 2014.
on (possibly virtualized) machines in the Cloud. The second A repository for new developments and updates is available
level, Platform as a Service (PaaS), provides more structure, at: http://goldberg.berkeley.edu/cloud-robotics/
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 8

ACKNOWLEDGMENTS [28] C. Aguero, N. Koenig, I. Chen, H. Boyer, S. Peters, J. Hsu, B. Gerkey,

S. Paepcke, J. Rivero, J. Manzo, E. Krotkov, and G. Pratt, “Inside
This research was supported in part by funding from Google the Virtual Robotics Challenge: Simulating Real-time Robotic Disaster
and Cisco and by the U.S. National Science Foundation under Response,” IEEE Transactions on Automation Science and Engineering
(T-ASE): Special Issue on Cloud Robotics and Automation, vol. 12,
Award IIS-1227536: Multilateral Manipulation by Human- no. 2, p. To appear, Apr. 2015.
Robot Collaborative Systems. [29] M. Armbrust, I. Stoica, M. Zaharia, A. Fox, R. Griffith, A. D. Joseph,
R. Katz, A. Konwinski, G. Lee, D. Patterson, and A. Rabkin, “A View
The authors thank Carlos Agero, Alper Aydemir, Alexandre of Cloud Computing,” Communications of the ACM, vol. 53, no. 4,
Bayen, George Bekey, Kostas Bekris, Dmitry Berenson, Gary p. 50, Apr. 2010.
Bradski, Matei Ciocarlie, Javier Civera, Steve Cousins, David [30] R. Arumugam, V. Enti, L. Bingbing, W. Xiaojun, K. Baskaran, F. Kong,
A. Kumar, K. Meng, and G. Kit, “DAvinCi: A Cloud Comput-
Culler, Raffaello D’Andrea, Steven Gentner, Brian Gerkey, ing Framework for Service Robots,” in International Conference on
Erico Guizzo, Timothy Hunter, M. Ani Hsieh, Volkan Isler, Robotics and Automation (ICRA), 2010, pp. 3084–3089.
Randy Katz, James Kuffner, Vijay Kumar, Jacek Malec, Matt [31] D. Astély, E. Dahlman, A. Furuskär, Y. Jading, M. Lindström, and
S. Parkvall, “LTE: The Evolution of Mobile Broadband,” Comm. Mag.,
Mason, David Portugal, Ben Recht, Luis Riazuelo, Javier vol. 47, no. 4, pp. 44–51, 2009.
Salmern-Garca, Sanjay Sarma, Ashutosh Saxena, Dezhen [32] L. Atzori, A. Iera, and G. Morabito, “The Internet of Things: A
Song, Carl Sutter, Richard Voyles, Alex Waibel, Lujia Wang, Survey,” Computer Networks, vol. 54, no. 15, pp. 2787–2805, Oct.
2010.
and Jeff Wiegley for ongoing insights and advice on this topic. [33] K. Bekris, R. Shome, A. Krontiris, and A. Dobson, “Cloud Automation:
Precomputing Roadmaps for Flexible Manipulation,” IEEE Robotics
& Automation Magazine: Special Issue on Emerging Advances and
R EFERENCES Applications in Automation, p. Under Review, 2014.
[34] W. Beksi and N. Papanikolopoulos, “Point cloud culling for robot
[1] “Amazon Elastic Cloud (EC2).” [Online]. Available: http://aws. vision tasks under communication constraints,” in International Con-
amazon.com/ec2/ ference on Intelligent Robots and Systems (IROS), 2014.
[2] “Amazon Web Services.” [Online]. Available: http://aws.amazon.com [35] D. Berenson, P. Abbeel, and K. Goldberg, “A Robot Path Planning
[3] “Arduino.” [Online]. Available: http://www.arduino.cc Framework that Learns from Experience,” in International Conference
[4] “Bullet Physics Library.” [Online]. Available: http://bulletphysics.org on Robotics and Automation (ICRA), May 2012, pp. 3671–3678.
[5] “CloudSim.” [Online]. Available: http://gazebosim.org/wiki/CloudSim/ [36] B. Bhargava, P. Angin, and L. Duan, “A Mobile-Cloud Pedestrian
[6] “For Big-Data Scientists, Janitor Work Is Key Hurdle to Insights.” Crossing Guide for the Blind,” in International Conference on Ad-
[Online]. Available: http://www.nytimes.com/2014/08/18/technology/ vances in Computing & Communication, 2011.
for-big-data-scientists-hurdle-to-insights-is-janitor-work.html [37] J. Bohg, A. Morales, T. Asfour, and D. Kragic, “Data-Driven Grasp
Synthesis—A Survey,” IEEE Transactions on Robotics (T-RO), no. 99,
[7] “Gazebo.” [Online]. Available: http://gazebosim.org/
pp. 1–21, 2013.
[8] “Google Compute Engine.” [Online]. Available: https://cloud.google. [38] V. Chandrasekaran and M. I. Jordan, “Computational and statistical
com/products/compute-engine tradeoffs via convex relaxation,” Proceedings of the National Academy
[9] “Google Goggles.” [Online]. Available: http://www.google.com/ of Sciences of the United States of America, vol. 110, no. 13, pp.
mobile/goggles/ E1181–90, 2013.
[10] “IEEE Networked Robots Technical Committee.” [Online]. Available: [39] M. Ciocarlie, K. Hsiao, E. G. Jones, S. Chitta, R. Rusu, and I. Sucan,
http://www-users.cs.umn.edu/∼isler/tc/ “Towards Reliable Grasping and Manipulation in Household Environ-
[11] “Industry 4.0.” [Online]. Available: http://www.bmbf.de/en/19955.php ments,” in International Symposium on Experimental Robotics (ISER),
[12] “Microsoft Azure.” [Online]. Available: http://www.windowsazure.com 2010.
[13] “MyRobots.com.” [Online]. Available: http://myrobots.com [40] M. Ciocarlie, C. Pantofaru, K. Hsiao, G. Bradski, P. Brook, and
[14] “Open Source Robotics Foundation.” [Online]. Available: http: E. Dreyfuss, “A Side of Data With My Robot,” IEEE Robotics &
//www.osrfoundation.org Automation Magazine, vol. 18, no. 2, pp. 44–57, June 2011.
[15] “OpenRAVE.” [Online]. Available: http://openrave.org/ [41] Committee on the Analysis of Massive Data; Committee on Applied
[16] “RoboBrain.” [Online]. Available: http://robobrain.me and Theoretical Statistics; Board on Mathematical Sciences and Their
[17] “ROS-Industrial.” [Online]. Available: http://rosindustrial.org Applications; Division on Engineering and Physical Sciences; National
[18] “ROS (Robot Operating System).” [Online]. Available: http://ros.org Research Council, Frontiers in Massive Data Analysis. The National
[19] “rosjava, an implementation of ROS in pure Java with Android Academies Press, 2013.
support.” [Online]. Available: http://cloudrobotics.com [42] L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb, “Social Coding in
GitHub: Transparency and Collaboration in an Open Software Repos-
[20] “TOP500.” [Online]. Available: http://www.top500.org/list/2012/06/
itory,” in ACM 2012 Conference on Computer Supported Cooperative
100
Work, 2012, p. 1277.
[21] “What is MyRobots?” [Online]. Available: http://myrobots.com/wiki/
[43] S. Dalibard, A. Nakhaei, F. Lamiraux, and J.-P. Laumond, “Manip-
About
ulation of Documented Objects by a Walking Humanoid Robot,” in
[22] “What is RoboEarth?” [Online]. Available: http://www.roboearth.org/ IEEE-RAS International Conference on Humanoid Robots, Dec. 2010,
what-is-roboearth pp. 518–523.
[23] “An Open-source Robo-surgeon,” The Economist, 2012. [Online]. [44] R. D’Andrea, “Guest editorial: A Revolution in the Warehouse: A
Available: http://www.economist.com/node/21548489 Retrospective on KIVA systems and the Grand Challenges Ahead,”
[24] “DARPA Selects Southwest Research Institute to Support DARPA IEEE Transactions on Automation Science and Engineering (T-ASE),
Robotics Challenge,” Market Watch, 2013. vol. 9, no. 4, pp. 638–639, 2012.
[25] Accenture Inc, “A New Era for Energy Companies: [45] H. Dang and P. K. Allen, “Learning Grasp Stability,” in International
Cloud Computing Changes the Game.” [Online]. Conference on Robotics and Automation (ICRA), May 2012, pp. 2392–
Available: http://www.accenture.com/SiteCollectionDocuments/PDF/ 2397.
Accenture-New-Era-Energy-Companies-Cloud-Computing-changes-Game.[46] H. Dang, J. Weisz, and P. K. Allen, “Blind Grasping: Stable Robotic
pdf Grasping using Tactile Feedback and Hand Kinematics,” in Interna-
[26] B. Addad, S. Amari, and J.-J. Lesage, “Analytic Calculus of Response tional Conference on Robotics and Automation (ICRA), May 2011, pp.
Time in Networked Automation Systems,” IEEE Transactions on 5917–5922.
Automation Science and Engineering (T-ASE), vol. 7, no. 4, pp. 858– [47] S. Davidson, “Open-source Hardware,” IEEE Design and Test of
869, 2010. Computers, vol. 21, no. 5, pp. 456–456, Sept. 2004.
[27] ——, “Client-Server Networked Automation Systems Reactivity: De- [48] G. de Croon, P. Gerke, and I. Sprinkhuizen-Kuyper, “Crowdsourcing
terministic and Probabilistic Analysis,” IEEE Transactions on Automa- as a methodology to obtain large and varied robotic data sets,” in
tion Science and Engineering (T-ASE), vol. 8, no. 3, pp. 540–548, International Conference on Intelligent Robots and Systems (IROS),
2011. 2014.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 9

[49] J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, [74] G. Hu, W. Tay, and Y. Wen, “Cloud Robotics: Architecture, Challenges
M. Ranzato, A. Senior, P. Tucker, K. Yang, Q. V. Le, and A. Y. and Applications,” IEEE Network, vol. 26, no. 3, pp. 21–28, 2012.
Ng, “Large Scale Distributed Deep Networks,” in Advances in Neural [75] K. Huebner, K. Welke, M. Przybylski, N. Vahrenkamp, T. Asfour, and
Information Processing Systems 25, F. Pereira, C. Burges, L. Bottou, D. Kragic, “Grasping Known Objects with Humanoid Robots: A Box-
and K. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 1223– Based Approach,” in International Conference on Advanced Robotics,
1231. 2009.
[50] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: [76] Humphreys, Todd, “Cockrell School Researchers Demonstrate First
A Large-Scale Hierarchical Image Database,” in IEEE Conference on Successful “Spoofing” of UAVs.” [Online]. Available: http://www.
Computer Vision and Pattern Recognition, 2009, pp. 248–255. engr.utexas.edu/features/humphreysspoofing
[51] A. Dobson, A. Krontiris, and K. E. Bekris, “Sparse Roadmap Span- [77] T. Hunter, T. Moldovan, M. Zaharia, S. Merzgui, J. Ma, M. J. Franklin,
ners,” in Algorithmic Foundations of Robotics X, 2013, pp. 279–296. P. Abbeel, and A. M. Bayen, “Scaling the Mobile Millennium System
[52] Droplet, “Cloud Powered Water Sprinkler System.” [Online]. Available: in the Cloud,” in 2nd ACM Symposium on Cloud Computing (SOCC),
http://smartdroplet.com/ 2011, p. 28.
[53] Z. Du, W. Yang, Y. Chen, X. Sun, X. Wang, and C. Xu, “Design [78] D. Hunziker, M. Gajamohan, M. Waibel, and R. D’Andrea, “Rapyuta:
of a Robot Cloud Center,” in International Symp. on Autonomous The roboearth cloud engine,” in International Conference on Robotics
Decentralized Systems, Mar. 2011, pp. 269–275. and Automation (ICRA). IEEE, 2013.
[54] E. Dumbill, “Defining Big Data,” Forbes Data Driven Blog, 2014. [79] M. Inaba, “Remote-brained robots,” in International Joint Conference
[55] P. C. Evans and M. Annunziata, “Industrial Internet: Pushing the on Artificial Intelligence, 1997, pp. 1593–1606.
Boundaries of Minds and Machines,” General Electric, Tech. Rep., [80] J. D. Irwin, The Industrial Electronics Handbook. CRC Press, 1997.
2012. [81] P. F. Jacobs and D. T. Reid, Rapid Prototyping & Manufacturing: Fun-
[56] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zis- damentals of Stereolithography. Society of Manufacturing Engineers,
serman, “The PASCAL Visual Object Classes (VOC) Challenge,” 1992.
International Journal of Computer Vision, vol. 88, no. 2, pp. 303–338, [82] A. Jain, D. Das, and A. Saxena, “PlanIt: A Crowdsourcing Approach
2010. for Learning to Plan Paths from Large Scale Preference Feedback,”
[57] S. Gammeter, A. Gassmann, L. Bossard, T. Quack, and L. Van Gool, Tech. Rep., 2014.
“Server-side Object Recognition and Client-side Object Tracking for [83] N. K. Jangid, “Real Time Cloud Computing,” in Data Management &
Mobile Augmented Reality,” in IEEE Conference on Computer Vision Security, 2011.
and Pattern Recognition, June 2010, pp. 1–8. [84] M. Johnson-Roberson, J. Bohg, G. Skantze, J. Gustafson, R. Carlson,
[58] J. Garcı́a, “Using Cloud Computing as a HPC Platform for Embedded B. Rasolzadeh, and D. Kragic, “Enhanced Visual Scene Understanding
Systems,” 2011. through Human-Robot Dialog,” in International Conference on Intel-
[59] General Electric, “The case for an industrial big data platform: Laying ligent Robots and Systems (IROS), Sept. 2011, pp. 3342–3348.
the groundwork for the new industrial age,” 2014. [Online]. Available: [85] S. Jordan, T. Haidegger, L. Kovacs, I. Felde, and I. Rudas, “The rising
https://gesoftware.com/Industrial Big Data Platform.pdf prospects of cloud robotic applications,” in International Conference
[60] N. Gershenfeld, Fab: The Coming Revolution on Your Desktop–from on Computational Cybernetics (ICCC), July 2013, pp. 327–332.
Personal Computers to Personal Fabrication. Basic Books, 2007. [86] G. Juve, E. Deelman, G. B. Berriman, B. P. Berman, and P. Maechling,
[61] Y. Gingold, A. Shamir, and D. Cohen-Or, “Micro Perceptual Human “An Evaluation of the Cost and Performance of Scientific Workflows
Computation for Visual Tasks,” ACM Trans. on Graphics, vol. 31, no. 5, on Amazon EC2,” Journal of Grid Computing, vol. 10, no. 1, pp. 5–21,
pp. 1–12, Aug. 2012. Mar. 2012.
[62] J. Glover, D. Rus, and N. Roy, “Probabilistic Models of Object [87] K. Kamei, S. Nishio, N. Hagita, and M. Sato, “Cloud Networked
Geometry for Grasp Planning,” in Robotics: Science and Systems (RSS), Robotics,” IEEE Network, vol. 26, no. 3, pp. 28–34, 2012.
2008. [88] A. Kasper, Z. Xue, and R. Dillmann, “The KIT object models database:
[63] K. Goldberg, “Beyond the Web: Excavating the Real World via An object model database for object recognition, localization and
Mosaic,” in Second International World Wide Web Conference, 1994, manipulation in service robotics,” International Journal of Robotics
pp. 1–12. Research (IJRR), vol. 31, no. 8, pp. 927–934, May 2012.
[64] K. Goldberg and B. Chen, “Collaborative control of robot motion: [89] B. Kehoe, A. Matsukawa, S. Candido, J. Kuffner, and K. Goldberg,
Robustness to error,” in Intelligent Robots and Systems, 2001. Pro- “Cloud-Based Robot Grasping with the Google Object Recognition
ceedings. 2001 IEEE/RSJ International Conference on, vol. 2. IEEE, Engine,” in International Conference on Robotics and Automation
2001, pp. 655–660. (ICRA), 2013.
[65] K. Goldberg, M. Mascha, S. Gentner, N. Rothenberg, C. Sutter, and [90] B. Kehoe, D. Berenson, and K. Goldberg, “Estimating Part Tolerance
J. Wiegley, “Desktop Teleoperation via the World Wide Web,” in Bounds Based on Adaptive Cloud-Based Grasp Planning with Slip,” in
International Conference on Robotics and Automation (ICRA), 1995. IEEE International Conference on Automation Science and Engineer-
[66] K. Goldberg and R. Siegwart, Eds., Beyond Webcams: An Introduction ing (CASE), 2012.
to Online Robots. MIT Press, 2002. [91] ——, “Toward Cloud-based Grasping with Uncertainty in Shape:
[67] C. Goldfeder, M. Ciocarlie, and P. Allen, “The Columbia Grasp Estimating Lower Bounds on Achieving Force Closure with Zero-slip
Database,” in International Conference on Robotics and Automation Push Grasps,” in International Conference on Robotics and Automation
(ICRA), May 2009, pp. 1710–1716. (ICRA), May 2012.
[68] C. Goldfeder and P. K. Allen, “Data-Driven Grasping,” Autonomous [92] B. Kehoe, D. Warrier, S. Patil, and K. Goldberg, “Cloud-Based Grasp
Robots, vol. 31, no. 1, pp. 1–20, Apr. 2011. Planning for Toleranced Parts Using Parallelized Monte Carlo Sam-
[69] B. Gouveia, D. Portugal, D. Silva, and L. Marques, “Computation pling,” IEEE Transactions on Automation Science and Engineering (T-
Sharing in Distributed Robotic Systems: a Case Study on SLAM,” ASE): Special Issue on Cloud Robotics and Automation, vol. 12, no. 2,
IEEE Transactions on Automation Science and Engineering (T-ASE): p. To appear, Apr. 2015.
Special Issue on Cloud Robotics and Automation, vol. 12, no. 2, p. To [93] J. Kelly, “The industrial internet and big data an-
appear, Apr. 2015. alytics: Opportunities and challenges,” 2013. [Online].
[70] E. Guizzo, “Cloud Robotics: Connected to the Cloud, Robots Get Available: http://wikibon.org/wiki/v/The Industrial Internet and Big
Smarter,” IEEE Spectrum Automaton Blog, 2011. Data Analytics%3A Opportunities and Challenges
[71] B. Hannaford, J. Rosen, D. Friedman, H. King, P. Roan, L. Cheng, [94] D. Kent and S. Chernova, “Construction of an object manipulation
D. Glozman, J. Ma, S. Kosari, and L. White, “Raven II: Open Platform database from grasp demonstrations,” in International Conference on
for Surgical Robotics Research,” IEEE Trans. Biomedical Engineering, Intelligent Robots and Systems (IROS), 2014.
pp. 954–959, 2012. [95] W.-J. Kim, K. Ji, and A. Ambike, “Real-time operating environmentfor
[72] E. Hidago-Pena, L. F. Marin-Urias, F. Montes-Gonzalez, A. Marin- networked control systems,” IEEE Transactions on Automation Science
Hernandez, and H. V. Rios-Figueroa, “Learning from the Web: Recog- and Engineering (T-ASE), vol. 3, no. 3, pp. 287–296, 2006.
nition method based on object appearance from Internet images,” [96] KIVA, “KIVA Systems.” [Online]. Available: www.kivasystems.com
in ACM/IEEE International Conference on Human-Robot Interaction. [97] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification
IEEE, 2013, pp. 139–140. with Deep Convolutional Neural Networks,” in Advances in Neural
[73] J. Higuera, A. Xu, F. Shkurti, and G. Dudek, “Socially-Driven Collec- Information Processing Systems 25, F. Pereira, C. Burges, L. Bottou,
tive Path Planning for Robot Missions,” Conference on Computer and and K. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 1097–
Robot Vision, pp. 417–424, May 2012. 1105.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 10

[98] J. Kuffner, “Cloud-Enabled Robots,” in IEEE-RAS International Con- [122] National Science Foundation, “Enabling a new future for cloud
ference on Humanoid Robots, 2010. computing.” [Online]. Available: http://nsf.gov/news/news summ.jsp?
[99] V. Kumar, R. Daniela, and G. S. Sukhatme, “Networked Robotics,” cntn id=132377
in Springer Handbook of Robotics, B. Siciliano and O. Khatib, Eds. [123] D. Nister and H. Stewenius, “Scalable Recognition with a Vocabulary
Springer Berlin Heidelberg, 2004. Tree,” in IEEE Conference on Computer Vision and Pattern Recogni-
[100] V. Kumar and N. Michael, “Opportunities and Challenges with Au- tion (CVPR), vol. 2, 2006, pp. 2161–2168.
tonomous Micro Aerial Vehicles,” International Journal of Robotics [124] D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman,
Research (IJRR), vol. 31, no. 11, pp. 1279–1291, 2012. L. Youseff, and D. Zagorodnov, “The Eucalyptus Open-Source Cloud-
[101] K. Lai and D. Fox, “Object Recognition in 3D Point Clouds Using Computing System,” in IEEE/ACM International Symp. on Cluster
Web Data and Domain Adaptation,” International Journal of Robotics Computing and the Grid, 2009, pp. 124–131.
Research (IJRR), vol. 29, no. 8, pp. 1019–1037, May 2010. [125] G. Oliveira and V. Isler, “View Planning For Cloud-Based Active
[102] Q. Le, M. Ranzato, R. Monga, M. Devin, K. Chen, G. Corrado, J. Dean, Object Recognition,” Department of Computer Science, University of
and A. Ng, “Building high-level features using large scale unsupervised Minnesota, Tech. Rep., 2013.
learning,” in International Conference in Machine Learning, 2012. [126] E. Plaku, K. E. Bekris, B. Y. Chen, A. M. Ladd, and L. E. Kavraki,
[103] A. E. Leeper, K. Hsiao, M. Ciocarlie, L. Takayama, and D. Gossow, “Sampling-based Roadmap of Trees for Parallel Motion Planning,”
“Strategies for Human-in-the-loop Robotic Grasping,” in ACM/IEEE IEEE Transactions on Robotics (T-RO), vol. 21, no. 4, pp. 597–608,
International Conference on Human-Robot Interaction, 2012, pp. 1–8. 2005.
[104] I. Lenz, H. Lee, and A. Saxena, “Deep learning for detecting robotic [127] E. Plaku, K. E. Bekris, and L. E. Kavraki, “OOPS for Motion Plan-
grasps,” International Journal of Robotics Research (IJRR), p. To ning: An Online, Open-source, Programming System,” in International
appear, 2014. Conference on Robotics and Automation (ICRA), Apr. 2007, pp. 3711–
[105] Z. Li, L. O’Brien, H. Zhang, and R. Cai, “On the Conceptualization 3716.
of Performance Evaluation of IaaS Services,” IEEE Transactions on [128] M. Popovic, G. Kootstra, J. A. Jorgensen, D. Kragic, and N. Kruger,
Services Computing, vol. X, no. X, pp. 1–1, 2014. “Grasping Unknown Objects using an Early Cognitive Vision System
[106] Z. Li, H. Zhang, L. O’Brien, R. Cai, and S. Flint, “On evaluating for General Scene Understanding,” in International Conference on
commercial Cloud services: A systematic review,” Journal of Systems Intelligent Robots and Systems (IROS), Sept. 2011, pp. 987–994.
and Software, vol. 86, no. 9, pp. 2371–2393, 2013. [129] E. Prestes, J. L. Carbonera, S. Rama Fiorini, V. a. M. Jorge, M. Abel,
[107] P. Lin, K. Abney, and G. A. Bekey, Robot Ethics: The Ethical and R. Madhavan, A. Locoro, P. Goncalves, M. E. Barreto, M. Habib,
Social Implications of Robotics. The MIT Press, 2011. A. Chibani, S. Gérard, Y. Amirat, and C. Schlenoff, “Towards a
[108] H. Lipson and M. Kurman, Fabricated: The New World of 3D Printing. core ontology for robotics and automation,” Robotics and Autonomous
Wiley, 2013. Systems, vol. 61, no. 11, pp. 1193–1204, 2013.
[109] B. Liu, Y. Chen, E. Blasch, and K. Pham, “A Holistic Cloud-Enabled [130] A. A. Proia, D. Simshaw, and K. Hauser, “Consumer Cloud Robotics
Robotics System for Real-Time Video Tracking Application,” in Future and the Fair Information Practice Principles: Recognizing the Chal-
Information Technology, 2014, pp. 455–468. lenges and Opportunities Ahead,” Minnesota Journal of Law, Science
[110] C.-H. Lu and L.-C. Fu, “Robust Location-Aware Activity Recognition & Technology, p. to appear, 2014.
Using Wireless Sensor Network in an Attentive Home,” IEEE Trans-
[131] M. Quigley, B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs,
actions on Automation Science and Engineering (T-ASE), vol. 6, no. 4,
E. Berger, R. Wheeler, and A. Ng, “ROS: an open-source Robot
pp. 598–609, 2009.
Operating System,” in ICRA Workshop on Open Source Software, 2009.
[111] J. Mahler, S. Krishnan, M. Laskey, S. Sen, A. Murali, B. Kehoe,
[132] A. J. Quinn and B. B. Bederson, “Human Computation: A Survey
S. Patil, J. Wang, M. Franklin, P. Abbeel, and K. Goldberg, “Learning
and Taxonomy of a Growing Field,” in Proceedings of the SIGCHI
Accurate Kinematic Control of Cable-Driven Surgical Robots Using
Conference on Human Factors in Computing Systems, ser. CHI ’11,
Data Cleaning and Gaussian Process Regression,” in IEEE Interna-
2011, pp. 1403–1412.
tional Conference on Automation Science and Engineering (CASE),
[133] K. Ren, C. Wang, Q. Wang, et al., “Security Challenges for the Public
2014.
Cloud,” IEEE Internet Computing, vol. 16, no. 1, pp. 69–73, 2012.
[112] G. McKee, “What is Networked Robotics?” Informatics in Control
Automation and Robotics, vol. 15, pp. 35–45, 2008. [134] L. Riazuelo, J. Civera, and J. Montiel, “C2TAM: A Cloud Framework
[113] McKinsey Inc, “Big Data and the Opportunities it for Cooperative Tracking and Mapping,” Robotics and Autonomous
Creates for Semiconductor Players.” [Online]. Available: Systems, vol. 62, no. 4, pp. 401–413, 2013.
http://www.mckinsey.com/∼/media/McKinsey/dotcom/client service/ [135] L. Riazuelo, M. Tenorth, D. D. Marco, M. Salas, L. Mosenlechner,
Semiconductors/Issue\Autumn\2012/PDFs/Big data and the L. Kunze, M. Beetz, J. D. Tardos, L. Montano, and J. Montiel,
opportunities it creates for semiconductor players.ashx “RoboEarth Web-Enabled and Knowledge-Based Active Perception,”
[114] P. Mehrotra, J. Djomehri, S. Heistand, R. Hood, H. Jin, A. Lazanoff, in IROS Workshop on AI-based Robotics, 2013.
S. Saini, and R. Biswas, “Performance evaluation of Amazon EC2 [136] L. Riazuelo, M. Tenorth, D. Marco, M. Salas, D. Galvez-Lopez,
for NASA HPC applications,” in 3rd Workshop on Scientific Cloud L. Mosenlechner, L. Kunze, M. Beetz, J. Tardos, L. Montano,
Computing Date - ScienceCloud ’12, 2012, pp. 41–50. and J. Montiel, “RoboEarth Semnatic Mapping: A Cloud Enabled
[115] P. Mell and T. Grance, “The NIST Definition of Cloud Computing,” Knowledge-Based Approach,” IEEE Transactions on Automation Sci-
National Institute of Standards and Technology, 2009. ence and Engineering (T-ASE): Special Issue on Cloud Robotics and
[116] N. Michael, D. Mellinger, Q. Lindsey, and V. Kumar, “The GRASP Automation, vol. 12, no. 2, p. To appear, Apr. 2015.
Multiple Micro UAV Testbed,” IEEE Robotics & Automation Maga- [137] J. Salmeron-Garcia, F. Diaz-del Rio, P. Inigo-Blasco, and D. Cagigas,
zine, vol. 17, no. 3, pp. 56–65, 2010. “A Trade-off Analysis of a Cloud-based Robot Navigation Assistant
[117] A. Miller and P. Allen, “GraspIt! A Versatile Simulator for Robotic using Stereo Image Processing,” IEEE Transactions on Automation
Grasping,” IEEE Robotics & Automation Magazine, vol. 11, no. 4, pp. Science and Engineering (T-ASE): Special Issue on Cloud Robotics
110–122, Dec. 2004. and Automation, vol. 12, no. 2, p. To appear, Apr. 2015.
[118] G. Mohanarajah, D. Hunziker, M. Waibel, and R. D’Andrea, “Rapyuta: [138] M. Samadi, T. Kollar, and M. Veloso, “Using the Web to Interactively
A Cloud Robotics Platform,” IEEE Transactions on Automation Science Learn to Find Objects.” in AAAI Conference on Artificial Intelligence,
and Engineering (T-ASE), pp. 1–13, July 2014. 2012, pp. 2074–2080.
[119] G. Mohanarajah, V. Usenko, M. Singh, M. Waibel, and R. D’Andrea, [139] Schmitt, Charles, “Security and Privacy in the Era of Big Data.”
“Cloud-based Collaborative 3D Mapping in Real-Time with Low-Cost [Online]. Available: http://www.renci.org/wp-content/uploads/2014/02/
Robots,” IEEE Transactions on Automation Science and Engineering 0313WhitePaper-iRODS.pdf
(T-ASE): Special Issue on Cloud Robotics and Automation, vol. 12, [140] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun,
no. 2, p. To appear, Apr. 2015. “OverFeat: Integrated Recognition, Localization and Detection using
[120] M. Moussa and M. Kamel, “An Experimental Approach to Robotic Convolutional Networks,” CoRR, 2013.
Grasping using a Connectionist Architecture and Generic Grasping [141] M. Sevior, T. Fifield, and N. Katayama, “Belle Monte-Carlo Production
Functions,” IEEE Trans. on Systems, Man and Cybernetics, Part C, on the Amazon EC2 Cloud,” Journal of Physics: Conference Series,
vol. 28, no. 2, pp. 239–253, May 1998. vol. 219, no. 1, Apr. 2010.
[121] M. Narita, S. Okabe, Y. Kato, Y. Murakwa, K. Okabayashi, and [142] N. Snavely, S. M. Seitz, and R. Szeliski, “Photo tourism: exploring
S. Kanda, “Reliable cloud-based robot services,” in Conference of the photo collections in 3D,” ACM transactions on graphics (TOG), vol. 25,
IEEE Industrial Electronics Society. IEEE, 2013, pp. 8317–8322. no. 3, pp. 835–846, 2006.
IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING 11

[143] D. Song, K. Goldberg, and N. Y. Chong, “Networked Telerobots,” [165] Q. Zhang, L. Cheng, and R. Boutaba, “Cloud Computing: State-of-
in Springer Handbook of Robotics, B. Siciliano and O. Khatib, Eds. the-art and Research Challenges,” Journal of Internet Services and
Springer Berlin Heidelberg, 2004. Applications, vol. 1, no. 1, pp. 7–18, 2010.
[144] A. Sorokin, D. Berenson, S. S. Srinivasa, and M. Hebert, “People
Helping Robots Helping People: Crowdsourcing for Grasping Novel
Objects,” International Conference on Intelligent Robots and Systems
(IROS), pp. 2117–2122, Oct. 2010.
[145] A. Sorokin and D. Forsyth, “Utility Data Annotation with Amazon
Mechanical Turk,” in IEEE Conference on Computer Vision and Pattern Ben Kehoe received his B.A. in Physics and Mathe-
Recognition (CVPR) Workshops, June 2008, pp. 1–8. matics from Hamline University in 2006. He is now
[146] M. Stenmark, J. Malec, K. Nilsson, and A. Robertsson, “On Distributed a Ph.D. student in the ME department at Univer-
Knowledge Bases for Small-Batch Assembly,” IEEE Transactions on sity of California, Berkeley. His research interests
Automation Science and Engineering (T-ASE): Special Issue on Cloud include cloud robotics, medical robotics, controls,
Robotics and Automation, vol. 12, no. 2, p. To appear, Apr. 2015. and grasping.
[147] M. Tenorth and M. Beetz, “KnowRob: A Knowledge Processing
Infrastructure for Cognition-Enabled Robots,” International Journal of
Robotics Research (IJRR), vol. 32, no. 5, pp. 566–590, 2013.
[148] M. Tenorth, A. Clifford Perzylo, R. Lafrenz, and M. Beetz, “The
RoboEarth language: Representing and exchanging knowledge about
actions, objects, and environments,” in International Conference on
Robotics and Automation (ICRA), May 2012, pp. 1284–1289.
[149] M. Tenorth, A. C. Perzylo, R. Lafrenz, and M. Beetz, “Representation
and Exchange of Knowledge about Actions, Objects, and Environments
in the Roboearth Framework,” IEEE Transactions on Automation Sachin Patil received his B.Tech degree in Com-
Science and Engineering (T-ASE), vol. 10, no. 3, pp. 643–651, 2013. puter Science and Engineering from the Indian Insti-
[150] The African Robotics Network (AFRON), ““Ten Dollar Robot” tute of Technology, Bombay, in 2006 and his Ph.D.
Design Challenge Winners.” [Online]. Available: http://robotics-africa. degree in Computer Science from the University
org/design challenge.html of North Carolina at Chapel Hill, NC in 2012.
[151] A. Torralba, R. Fergus, and W. T. Freeman, “80 Million Tiny Images: He is now a postdoctoral researcher in the EECS
A Large Data Set for Nonparametric Object and Scene Recognition,” department at University of California, Berkeley. His
IEEE Transactions on Pattern Analysis and Machine Intelligence, research interests include motion planning, cloud
vol. 30, no. 11, pp. 1958–1970, 2008. robotics, and medical robotics.
[152] R. Tudoran, A. Costan, G. Antoniu, and L. Bougé, “A Performance
Evaluation of Azure and Nimbus Clouds for Scientific Applications,”
in International Workshop on Cloud Computing Platforms - CloudCP
’12. ACM Press, 2012, pp. 1–6.
[153] L. Turnbull and B. Samanta, “Cloud robotics: Formation control of a
multi robot system utilizing cloud infrastructure,” in Proceedings of
IEEE Southeastcon. IEEE, 2013, pp. 1–4. Pieter Abbeel received a BS/MS in Electrical En-
[154] J. van den Berg, P. Abbeel, and K. Goldberg, “LQG-MP: Optimized gineering from KU Leuven (Belgium) and received
path planning for robots with motion uncertainty and imperfect state in- his Ph.D. degree in Computer Science from Stanford
formation,” International Journal of Robotics Research (IJRR), vol. 30, University in 2008. He joined the faculty at UC
no. 7, pp. 895–913, June 2011. Berkeley in Fall 2008, with an appointment in the
[155] L. von Ahn, “Human Computation,” Ph.D. dissertation, Carnegie Department of Electrical Engineering and Computer
Mellon University, 2009. Sciences. His current research focuses on robotics
[156] M. Waibel, M. Beetz, J. Civera, R. D’Andrea, J. Elfring, D. Gálvez- and machine learning with a particular focus on
López, K. Häussermann, R. Janssen, J. Montiel, A. Perzylo, challenges in personal robotics, surgical robotics
B. Schieß le, M. Tenorth, O. Zweigle, and R. De Molengraft, and connectomics. He has won numerous awards,
“RoboEarth,” IEEE Robotics & Automation Magazine, vol. 18, no. 2, including best paper awards at ICML and ICRA,
pp. 69–82, June 2011. the Sloan Fellowship, the Air Force Office of Scientific Research Young
[157] H. Wang, Y. Ma, G. Pratx, and L. Xing, “Toward Real-time Monte Investigator Program (AFOSR-YIP) award, the Office of Naval Research
Carlo Simulation using a Commercial Cloud Computing Infrastruc- Young Investigator Program (ONR-YIP) award, the Darpa Young Faculty
ture.” Physics in Medicine and Biology, vol. 56, no. 17, pp. N175–81, Award (Darpa-YFA), the Okawa Foundation award, the TR35, the IEEE
Sept. 2011. Robotics and Automation Society (RAS) Early Career Award, and the Dick
[158] J. Wang, S. Krishnan, M. J. Franklin, K. Goldberg, T. Kraska, and Volz Best U.S. Ph.D. Thesis in Robotics and Automation Award.
T. Milo, “A Sample-and-Clean Framework for Fast and Accurate Query
Processing on Dirty Data,” in ACM SIGMOD International Conference
on Management of Data, 2014.
[159] L. Wang, M. Liu, and M. Q.-H. Meng, “Real-time Multi-sensor
Data Retrieval for Cloud Robotic Systems,” IEEE Transactions on
Automation Science and Engineering (T-ASE): Special Issue on Cloud Ken Goldberg is Professor of Industrial Engineering
Robotics and Automation, vol. 12, no. 2, p. To appear, Apr. 2015. and Operations Research at UC Berkeley, with ap-
[160] We Robot, “We Robot Conference.” [Online]. Available: http: pointments in Electrical Engineering, Computer Sci-
//robots.law.miami.edu/ ence, Art Practice, and the School of Information. He
[161] R. H. Weber, “Internet of Things–New Security and Privacy Chal- was appointed Editor-in-Chief of the IEEE Transac-
lenges,” Computer Law & Security Review, vol. 26, no. 1, pp. 23–30, tions on Automation Science and Engineering (T-
2010. ASE) in 2011 and served two terms (2006–2009) as
[162] J. Weisz and P. K. Allen, “Pose error robust grasping from contact Vice-President of Technical Activities for the IEEE
wrench space metrics,” in International Conference on Robotics and Robotics and Automation Society. Goldberg earned
Automation (ICRA). IEEE, May 2012, pp. 557–562. his PhD in Computer Science from Carnegie Mellon
[163] E. Wilhelm, J. Siegel, S. Mayer, J. Paefgen, M. Tiefenbeck, M. Bicker, University in 1990. Goldberg is Founding Co-Chair
S. Ho, R. Dantu, and S. Sarma, “CloudThink: An Open Standard of the IEEE Technical Committee on Networked Robots and Founding
for Projecting Objects into the Cloud,” 2013. [Online]. Available: Chair of the (T-ASE) Advisory Board. Goldberg has published over 200
http://cloud-think.com/ refereed papers and awarded eight US patents, the NSF Presidential Faculty
[164] Willow Garage, “Personal Service Robotics with Tiered Human-in-the- Fellowship (1995), the Joseph Engelberger Award (2000), the IEEE Major
Loop Assistance,” NSF SBIR Award 1256643 (PI:Matei Ciocarlie), Educational Innovation Award (2001) and in 2005 was named IEEE Fellow.
2013.