Artificial Life and Real Robots

Rodney A. Brooks
MIT Artificial Intelligence Laboratory
545 Technology Square
Cambridge, MA 02139, USA
[email protected]
"Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life," Francisco J. Varela and Paul
Bourgine, eds., MIT Press, Cambridge, MA, 1992, pp. 3—10.

Abstract scheme, built within the framework of the modules,

controls which behavior-producing module has
control of which part or the robot at any given time.
The first part of this paper explores the general
The programs are layered in their construction, but
issues in using Artificial Life techniques to program
are non-hierarchical in their control flow, with lower
actual mobile robots. In particular it explores the
levels taking care of more primitive activities and
difficulties inherent in transferring programs evolved
higher levels taking care of more sophisticated ones
in a simulated environment to run on an actual robot.
(see [Brooks 86] and [Brooks 90b] for an introduction
It examines the dual evolution of organism
and review, and [Brooks 91c] for a survey of the
morphology and nervous systems in biology. It
field).
proposes techniques to capture some of the search
space pruning that dual evolution offers in the
This work draws its inspirations from
domain of robot programming. It explores the
neurobiology ethology, psychophysics, and
relationship between robot morphology and program
sociology. The approach grew out of dissatisfactions
structure, and techniques for capturing regularities
with traditional robotics and Artificial Intelligence,
across this mapping.
which seemed unable to deliver real-time performance
in a dynamic world. The key idea of the new approach
The second part of the paper is much more
is to advance both robotics and AI by considering the
specific. It proposes techniques which could allow
realistic explorations concerning the evolution of problems of building an autonomous agent that
programs to control physically embodied mobile physically is an autonomous mobile robot, and that
carries out some useful tasks in an environment
robots. In particular we introduce a new abstraction
which has not been specially structured or engineered
for behavior-based robot programming which is
for it.
specially tailored to be used with genetic
programming techniques. To compete with hand
coding techniques it will be necessary to While robots built on these principles have been
automatically evolve programs that are one to two demonstrated learning calibration information [Viola
90], behavior coordination [Maes and Brooks 90], and
orders of magnitude more complex than those
representations of the world [Mataric 90], progress in
previously reported in any domain. Considerable
learning new behaviors has proven more difficult.
extensions to previously reported approaches to
genetic programming are necessary in order to achieve Today, we are constrained to programming each new
behavior by hand.
this goal.
Work in Artificial Life has developed techniques
1 Introduction for evolving programs for controlling situated, but
unembodied (i.e., simulated), robots (e.g., [Langton
In recent years a new approach to Artificial 87], [Langton et al 90]). At some level one of the
Intelligence has developed which is based on building goals of Artificial Life is to move out of the digital
behavior-based programs to control situated and medium into that of embodied systems. Is there a
embodied robots in unstructured dynamically match between AL and AI? This paper explores the
changing environments [Brooks 91c]. Rather than prospects for using Artificial Life techniques to
modularize perception, world modeling, planning, and evolve programs to control physically embodied
execution, the new approach builds intelligent control mobile robots, so that we no longer have to do it all
systems where many individual modules directly by hand.
generate some part of the behavior of the robot. In
the purest form of this model each module There have been no reports to date of programs
incorporates its own perceptual, modeling, and evolved for embodied robots. There has been work on
planning requirements. An arbitration or mediation
learning new behaviors using reinforcement learning, programs only run in simulation rather than on a
e.g., [Kaelbling 90] and [Mahadevan and Connell 90] physically embodied robot, but the results are
used Q-1earning ([Watkins 89]). The major drawback nevertheless significant enough to warrant further
is the large number of runtime trials, many more exploration of this technique.
than needed by real animals, and the need for carefully
"shaping" the learning by splitting up the tasks into Before these techniques can be adopted and modified
little pieces that the robot learns sequentially. It for programming physically embodied mobile robots
seems that real animals have innate built-in structures there are a number of problems which must be
that facilitate learning particular constrained classes of addressed:
behaviors. The vast numbers of trials necessary are
spread over the generations, and runtime learning has • Most likely the evolution of robot programs must
a more constrained space in which it must search. be carried out on simulated robots—unfortunately
there is a vast difference (which is not appreciated
Recently [Langton 91] suggested using genetic by people who have not used real robots) between
programming for behavior-based embodied robots to simulated robots and physical robots and their
overcome these limitations. dynamics of interaction with the environment.

2 Genetic Programming • The structure of the search space of possible

programs is very dependent on the representation
used for programs and the primitives available to be
One way to solve the programming problem might
incorporated. Careful design is necessary.
be to use Artificial Life techniques to evolve
behavior-based programs.
• Natural evolution co-evolved the structure of the
physical entities and their neural controllers in a
Previously many workers have used genetic
way which arguably cut down the size of its search
algorithms to program software agents, typically
space. Can some equivalent tricks be played when
running in cellular worlds. [Collins and Jefferson 90]
evolving programs for robots?
is a good example. It demonstrates the evolution of
both neural networks and finite state machines
through a genetic algorithm running on a bit string 3 Simulations of Physical Robots
representation.
The number of trials needed to test individuals
More conventional computer programs have also precludes using physical robots for testing the bulk
been processed with genetic algorithms, such as the of the control programs produced for them by genetic
pioneering work of [Friedberg 58], [Friedberg et al means. The obvious choice is to use simulated robots
59], and more recently that of [Ray 90]. Robot and then run the successful programs on the physical
programs, and in particular behavior-based robot robots.
programs, are much more complex than any
programs that have been reported in the literature to Previously we have been very careful to avoid
have been so evolved. A reasonable comparison using simulations ([Brooks 90b, 91b, 91c]) for two
might be in terms of the memory taken to represent fundamental reasons.
the programs. By this measure behavior-based robot
• Without regular validation on real robots there is a
programs axe three orders of magnitude larger than
great danger that much effort will go into solving
those mutated competitively by genetic techniques.
problems that simply do not come up in the real
world with a physical robot (or robots).
Recently, however, [Koza 90] has shown a number
of stimulating results by applying genetic algorithms
• There is a real danger (in fact, a near certainty) that
directly to lisp-like programs rather than to more
programs which work well on simulated robots
traditional bit strings [Holland 75]. He has been very will completely fail on real robots because of the
successful in a number of domains with this differences in real world sensing and actuation—it
technique, rekindling earlier interest in the idea of is very hard to simulate the actual dynamics of the
mutating lisp program structures directly [Lenat 77]. real world.
[Koza 91] shows an example of synthesizing the base
behaviors of [Mataric 90]'s behavior-based robot 3.1 Artifactual Problems
programs. He makes a number of simplifying
assumptions, and reduces the search space
There has been a tendency to use cellular
significantly by carefully selecting the primitives by
representations of space for simulating robots in
hand after examining Mataric's source code. His
Artificial Life. (e.g., [Langton et al 90]) and
Artificial Intelligence (e.g., [Pollack and Ringuette
90] and [Wang and Beni 90]). • The data delivered by sensors are not direct
descriptions of the world as objects and their
These representations are good for conducting relationships.
computational experiments, and help uncover many
fundamental issues. Unfortunately they do not shed • Commands to actuators have very uncertain effects.
light on all the problems which will be encountered
when using physically embodied robots. For the A particular sensor, under ideal experimental
physical robot perspective, cellular worlds have three conditions, may have a particular resolution. Suppose
problems. First, there is no notion of the uncertainty the sensor is a sonar. Then to measure its resolution
that the real world presents-see the subsection below an experiment will be set up where a return signal
for more discussion of this point. Second, there is a from the test article is sensed, and the resolution will
tendency to not only postulate sensors which return be compared against measurements of actual distance.
perfect information (e.g., the cell ahead contains The experiment might be done for a number of
food—no real perception system can do such a thing), different surface types. But when that sensor is
but there is a real danger of confusing the global installed on a mobile robot, situated in a cluttered,
world view and the robot's view of the world. Third, dynamically changing world, the return signals that
the dynamics actually tend to be more brittle than in reach the sensor may come from many possible
the real world where noise and stochastic processes sources. The object nearest the sensor may not be
smooth things out quite a bit. made of one of the tested materials. It may be at such
an angle that the sonar pulse acts as though it were a
The dynamics mentioned in the last point often mirror, and so the sonar sees a secondary reflection.
leads to problems which do not occur in the real The secondary lobes of the sonar might detect
world. A good example is being concerned with how something in a cluttered situation where there was no
two robots resolve a conflict when their paths must such interference in the clean experimental situation.
cross a single cell simultaneously. This and other All sensors have comparable sets of problems
equally artifactual problems are the main concern of a associated with them. They simply do not return
number of papers in this area. clean accurate readings. At best they deliver a fuzzy
approximation to what they are apparently
But the same intellectual problem of worrying measuring, and often they return something
about simulated problems which do not actually completely different.
appear in the real world is more general than just for
cellular simulations. A number of papers have Even given these difficulties, sensor readings are
appeared that are concerned with path planning for not the same as a description of the world. Sensors
mobile robots in nongrid worlds, i.e., in two measure certain quantities or indirect aspects of the
dimensional Euclidean space. Some of these papers world. They do separate objects from the background.
expend much effort on solving the problem of the They do not identify objects. They do not give pose
paths of two robots crossing each other and introduce information about objects. They do not separate out
elaborate protocols to avoid deadlock. But real robots static objects, moving objects, and effects due to self
would never reach the state of perfect deadlock which motion. They do not operate in a stable coordinate
are postulated in these papers. They would never run system independent of the uncertain motion of the
down their respective corridors perfectly and arrive at robot. They do not integrate other sensory modalities
identical times. Simple reactive strategies would into a single consistent picture of the world. A robot
suffice to break any possible deadlock, just as random operating in the real world needs a complex
variations in ethernet controllers break deadlocks on perceptual system. As much as 50% of the human
rebroadcast. brain seems to be devoted to perception. Off the shelf
perceptual systems are not available, however. All of
Thus, while simulated worlds are in many ways the problems listed above are active areas of research
simpler than the real world, they are paradoxically by perception researchers. And, as argued elsewhere
sometimes harder to operate within. [Brooks 91a, 91b], it may be impossible to treat
perception as a black box with a clean interface to the
3.2 Real Worlds rest of intelligence.

For a physically embodied robot in the real world Just as sensors do not deliver simple descriptions
there are a number of key points to understand. of the world, high level action commands need many
layers of refinement before they become appropriately
• Sensors deliver very uncertain values even in a sta- orchestrated motor currents. In any case, the desired
ble world. action and the achieved action may differ widely
depending on the intricate details of the situation at additional sensors and actuators can be made available
hand. On flat smooth floors, odometry errors soon so that higher level behaviors can be evolved. The
accumulate to the point that a a robot needs to particular fitness function used to control the
recalibrate its position to some external reference. evolutionary search can be varied over time to
Besides a large unsystematic component, odometry emphasize the use of new capabilities and the need to
may also have systematic aspects, e.g., depending on develop higher level behaviors. There might also be
the relative nap of the carpet, on which the robot is some advantage to biasing the genetic operators
operating—this has lead some researchers to try to towards operating more on the newer behaviors than
sense the nap! The situation is much more difficult to on the early. (Although it certainly makes sense for
model, of course, when the robot is in contact with the crossover operator to take pieces of old behavior
an obstacle. eg., an orienting behavior, based on a sensor activated
early in the evolutionary search would be a good
Sensing, and action are intimately tied together in a prototype for an orienting behavior using a new
physical robot. They both rely on, and at the same sensor.)
time generate, the dynamics of the interaction of the
robot with the world. Simple state space models of In our hand written programs we capture symmetry
the world do not suffice in the internal control and repeated structure by the use of macros. For
program of a physically embodied mobile robot. example, on the six-legged robot Genghis ([Brooks
89]) there is a macro version of the leg control
4 Morphological Development behaviors which gets instantiated six times, once for
each leg. This suggests two ideas to again reduce the
genetic search space.
We now turn to the nature of the search space in
which the genetic programming techniques must
• The language that is subject to genetic
work.
programming should include a macro capability. As
the search learns how to use these effectively it will
In nature, evolution experiments with both the
greatly accelerate the production of good programs.
morphology of the individual and its neural circuitry
cotemporaneously and through the same genetic
• There must be some way to reference the
mechanism. There are two important consequences of
symmetries and regularities of the morphological
this, both of which reduce the space which evolution
structure of the robot, in order to invoke the macros
must search.
correctly. This morphological descriptor will be a
constant for each particular class of robot. The
• The control program is evolved incrementally.
constant could be evolved, but it would be much
Evolution is restricted initially to a small search
quicker to have that as an available constant
space. The size of the space grows over many
somewhere in the original pool of ancestral
generations, but by then there is a good partial
programs.
solution already found which forms the basis for
searching the newer parts of the space.
5 Evolving Behavior-Based Pro-
• Symmetric or repeated structures naturally have grams
symmetric or repeated neural circuits installedthey
do not need to be individually evolved (e.g., a [Koza 91] reports on a genetic programming
single encoding specifies the wiring of both a right implementation in simulation of the navigation
and a left leg, and likewise each added segment behaviors of [Mataric 90]'s robot Toto. The
containing a leg pair in a mutant Drosophila comes programming language he uses is a carefully chosen
with neural circuitry). subset of Lisp. While adequate for the task Koza
reports, based on our experience with physical robots,
Our approach to behavior-based programming of it is perhaps not sufficient for the more general case.
robots has always been to build layers incrementally
[Brooks 86]. For genetic programming we suggest [Langton 91] has suggested the idea of genetically
that the robot (both simulated and physical) should programming a physical robot using the Behavior
initially be operated with only some of its sensors, Language (BL) defined in [Brooks 90a].
and perhaps only some of its actuators. Programs can Unfortunately, there are many drawbacks to using BL
be evolved for this simplified robot, much as our directly.
hand written programs ([Brooks 90b]) start out with
layers which only use some of the capabilities of the
[Lenat and Brown 84] analyze the the apparent
robot. Once fundamental behaviors are present,
success of the discovery system AM [Lenat 77]. They
point out the crucial way in which the syntax of the embedding predicates in conditional special forms
Lisp programs being mutated mirrored the semantics (implemented as macros) is a good one.
of the world of simple mathematics concepts being
explored. We can likewise expect performance of a Koza hides critical constants in these special forms
crossover based genetic programming system to also. [Ray 90] points out the difficulty in having too
critically depend on the syntax of the language used many constants around in the genetic pool, and
and the way in which crossover mutates a program's instead evolves simple constructive program
semantics. segments to build them. Primitives to make this easy
would be a good idea in GEN also.
In this section we try to identify the choices made
by Koza which led to his success, and compare them 5.2 Variables and Lexicality
to what is available in BL. We propose a higher level
language, called GEN, which can be compiled into Real BL programs use many named state variables.
BL as the target language for genetically Sometimes these are simply used as semaphores,
programming physical robots. sometimes they are used as counters, sometimes they
are used for calibration offsets, and sometimes they
5.1 Primitives for Genetic are used to store sensor readings, or processed sensor
Programming of Robots readings, for later comparison.

As with Lisp and BL, lexical contours should be

[Koza 90] uses a representation for Lisp programs able to be introduced at any point in a program (both
which is crucial to the easy application of crossover. languages use LET to do this). Variable names as
He treats an S-expression as a tree rooted with the such can not be used conveniently if crossover is to
first elements, and with one branch for each of the occur, as the produced programs will not be lexically
subsequent elements. E.g., the expression (* 1 2 3) is meaningful very often. Instead some sort of indexing
thought of as a tree with four nodes, a depth of two scheme is needed. A global index for a behavior is
and a branching factor of three at the root node. Koza not a good idea, because then it will be hard to use
calls each thing which can occur at a non-terminal crossover to duplicate little pieces of code with local
node a function, but that terminology differs from variables. Instead we propose using a single form
modern Lisp terminology. In fact his operators are (variable-ref), and specific operators to move up and
often special forms or macros. down the lexical contour tree, and to rotate around the
set of variables introduced at a particular contour
For instance, he might use IF-SENSOR as an (often there will be just a single variable).
operator which accesses some hidden state (or
perceptual information) and depending on its value 5.3 Depth and Breadth
either evaluate the first argument or the second
argument-thus it is not a function in the usual sense In Koza's re-implementation of Toto's navigation
of being applied to all its evaluated arguments. Such behaviors the best solution found has 157 nodes of
embedded conditionals seem to be the only which 65 are terminals, and the tree has depth 12.
conditional forms used by Koza. This is for a very Figure 1 shows the statistics for Mataric's original
good reason. Pure Lisp can be thought of as a BL program for the same task. Overall it has 942
type-free language. But, in fact some values, such as nodes, and 500 terminals, and coincidentally a
the test argument to a conditional, are treated as maximum depth of 12.
booleans. Although everything is trivially coercible
to a boolean, including a conditional requiring a The difference can be accounted for by three
boolean test would greatly increase the search space components:
and drastically lower the density of semantically
useful program trees. 1. There are a number of housekeeping operations,
reporting, and debugging features included in the
In [Koza 91] careful analysis of [Mataric 90]'s original BL program. These can be left out of
original code was done in order to pick just the right GEN programs also.
set of conditionals with access to hidden state which
is to be tested. 2. The techniques used by Koza to compress the trees
are not present in the BL programs. GEN will be
Without the hindsight from a successful able to get some of this benefit, but not all, as it
implementation, GEN must allow more general needs to be more general than the minimal set of
testing than Koza allows. But the heuristic of primitives used by Koza.
3. The code produced by Koza has only been tested in Careful attention must be paid to this issue to
simulation, and relies a little on the simplicity of provide appropriate mapping primitives so that the
the the simulated emvironment. GEN programs for evolving programs can pull out the right boolean
embodied robots may need to be slightly more bits. One simple heuristic for the robot designer is to
complex than this, make all such mappings consistent where possible on
a particular robot design, thus maximizing the
It seems reasonable to assume that a general benefits for crossover of program fragments in
purpose GEN language program for this task might adopting new sensors.
come out a factor of two to three bigger than Koza's
program. Evolving a program of that size certainly 6 Reconnecting to Reality
seems within reach-it is not a drastic step up from
Koza's results. Eventually the programs developed by genetic
programming are to be run on physically embodied
robots. There are a number of concerns in connecting
the programs back to this reality.

6.1 Calibration

It is our experience that supposedly identical

physical robot components are not identical. We have
experienced this with very different sensing and
actuator responses from supposedly identical legs on
a six legged robot. We have also experienced this
with 20 small wheeled robots (R-1's).

On reflecting on our past practices, we realized that

when programming just a single robot system we had
Figure 1: A statistical summary of a BL program to drive the
robot TOTO in its wall following. It includes various debugging experimented by hand and built appropriate constants
behaviors and display code. The columns are P, number of into our programs to handle the responses of our
processes, N, total number of nodes, T, total number of terminals,
D, maximum depth, B, maximum branching factor, AD, average sensors and actuators. When we had the same code
depth of all nodes, AB, average breadth at non-terminal nodes, running on multiple copies of a robot, or robot
and the name chosen by the programmer as the behavior name.
component, we found this approach inadequate. We
have found it necessary therefore to build in adaptive
elements into the run-time structures of our programs
For comparison, figure 2 shows the statistics for a
which change thresholds and timeout intervals. This
walking program for the six-legged robot Attila.
is not yet a formalized process, but we need to find a
Again there are a number of debugging aids and tools
set of primitives that allow such adaptive elements to
included in this program, along with some low level
be constructed easily.
substrate processes (which need not be evolved).
The implication of this for genetic programming
5.4 Mappings of physical robots is clear. There must be a set of
primitives available to the system so that adaptive
Earlier it was pointed out that there needs to be elements can be constructed. Further, the simulation
some way to relate the evolved program to the must have enough variance in it that these adaptive
morphological regularities in the structure of the elements are essential to successfully run the
robot. There is an additional related problem which if programs on the simulated robots, so that the
not handled well will generate a search space with adaptive elements will be there when it comes time
only a low density of useful program fragments. to validate on physical robots.

Typically on real robots a ring of 8, say, bump 6.2 Adapting the Simulation
sensors, or infrared proximity sensors, will be
accessible as a single byte of 8 one bit values. The
When the programs which have been evolved on
mapping between arrangement of those bits and the
the simulated robots are tried on embodied robots
morphology of the sensors is usually quite arbitrary,
there will be two types of information available.
and indeed there may be no natural semantically valid
arrangement of the bits.
1. How well the programs work on embodied robots.
2. How different the performance of the programs is robot. This will be a critical data point in evaluating
on the simulated robots and the embodied robots. the utility of the approach.

The first type of information can be used to inform The Behavior Language compiler has been modified
the genetic programming system, as would any so that other programs can call it, rather than having
simulated test of the generated programs. to go through the previous interface optimized for
people using it to program physical robots. It already
The second type of information should be used to had a backend which produced a byte-code program for
tune up the simulation to better match reality. One which a Common Lisp program provides an
would expect that this would be done by hand. interpreter. Thus, BL programs can be automatically
However there is a more tantalizing possibility. -compiled and run on a simulation machine. In
Perhaps this information could be used to co-evolve addition, a higher language known as GEN has been
the simulater using genetic programming partially built which compiles into BL programs.
techniques-especially if the simulator were written in GEN is based on the ideas explored in the previous
the same sort of BL as the robots were programmed sections.
(our current simulator is partially written in BL). The
idea would be to run the evolved robot programs on 8 Conclusion
the embodied robots, then with those programs fixed,
evolve the simulator until it better matched what
The key ideas presented in this paper are:
happened in reality. There are deep issues in finding
measures to compare the performance of the
• Use genetic programming techniques to build
simulator to reality, since the robots won't be going
behavior-based programs which can run real robots
through exactly the same sequence of operations
(this idea is due to [Langton 91]).
Eventually, perhaps, this may be a viable approach.
• Evolution and runtime adaptation are two separate
7 Progress issues.

At the time of writing no complete experiments • All robots will need runtime adaptation elements.
have been carried out using the ideas in this paper.
• Evolution of control structure needs to run in
We have built a simulator for multiple R-2 robots1 parallel with evolution of morphology—in robots
It is not grid-based, but instead the coordinates of a this can be simulated by progressively enabling
robot can be arbitrary floating point numbers within more sensors and actuators as layers of behaviors
the workspace. R-2 robots have a two wheeled evolve for those already operational.
differential drive with passive castors for stability.
The simulator handles arbitrary independent velocities • Regularity (e.g., symmetry or repeated structures)
on the two wheels. There is a simple physics in morphological structure should be mirrored in
associated with motion of a robot when it has regularity in the control structure, and thus needs to
collided with an obstacle (which are all modeled as be representable in the control language.
immovable cylinders). The sensors currently modeled
are a ring of eight bump sensors, a ring of eight • Special care must be taken in design of the control
infrared proximity sensors, and three forward looking language to minimize the depth and breadth of
beacon sensors. We expect to add more sensor useful program trees.
models. No explicit uncertainty is built into the
sensor or actuator models. However, the BL program • To make crossover useful in a control language with
which runs the robots refers to the real time clock of variables, a new method for representing variable
the computer on which the simulation is run. Noise references was introduced.
is therefore introduced into the system by the load on
the computer, and by the interjection of the Lisp • There are real methodological dangers in using
garbage collector. simulations as a testing medium in which to
evolve programs which are intended eventually to
The complete simulation is about 500 lines of run on physical robots-great care must be taken to
combined Common Lisp and BL code. We have no develop a sufficient validation regime.
hypothesis at this point about how well programs
developed on the simulator will transfer to the real There are many possible approaches to using
Artificial Life techniques for programming physical
robots We have chosen one particular approach here,
1
Manufactured by IS Robotics.
but many of the points of concern will be common [Kaelbling 90] “Learning in Embedded Systems”, Leslie
with other approaches. Pack Kaelbling, Ph.D. Thesis, Stanford.

[Koza 90] “Evolution and Co-Evolution of Computer

Acknowledgements Programs to Control Independently-Acting Agents”,
John R. Koza, Proc. First Int. Conf. on Simulation o f
Maja Mataric, Ian Horswill and Anita Flynn Adaptive Behavior, Paris, MIT Press, Cambridge, MA,
provided helpful comments on drafts of this paper. 1990, 366-375.

Support for this research was provided in part by [Koza 91] “Evolving Emergent Wall Following Robotic
the University Research Initiative under Office of Behavior Using the Genetic Programming Paradigm”,
Naval Research contract N00014-86-K-0685, in part John R. Koza, ECAL, Paris, Dec.
by the Advanced Research Projects Agency under
Office of Naval Research contract [Langton 87] “Proceedings of Artificial Life”,
Christopher G. Langton (ed), Addison-Wesley, appeared
N00014-85-K-0124, in part by the Hughes Artificial
1989.
Intelligence Center, and in part by Mazda
Corporation. [Langton 91] Personal communication, Christopher G.
Langton, September.
References
[Langton et al 90] “Proceedings of Artificial Life, II”,
[Brooks 86] “A Robust Layered Control System for a Christopher G. Langton, Charles Taylor, J. Doyne
Mobile Robot”, Rodney A. Brooks, IEEE Journal of Farmer, and Steen Rasmussen (eds), Addison-Wesley,
Robotics and Automation RA-2, April, 14-23. appeared 1991.

[Brooks 89] “Robot that Walks: Emergent Behavior [Lenat 77] “The Ubiquity of Discovery”, Douglas B.
from a Carefully Evolved Network”, Rodney A. Brooks, Lenat, IJCAI-77, Cambridge, MA, Aug., 1093-1105.
Neural Computation, 1:2, Summer, 253-262.
[Lenat and Brown 84] “Why AM and EURISKO Appear t o
[Brooks 90a] “The Behavior Language; User's Guide”, Work”, Douglas B. Lenat and John Seely Brown,
Rodney A. Brooks, MIT A.I. Lab Memo 1227, April. Artificial Intelligence 23,269-294.

[Brooks 90b] “Elephants Don't Play Chess”, Rodney A. [Maes 90] “Designing Autonomous Agents: Theory and
Brooks, in [Maes 90], 1990, 3-15. Practice from Biology to Engineering and Back”, Pattie
Maes (ed), MIT Press, Cambridge, MA, 1990.
[Brooks 91a] “Intelligence without Representation”,
Rodney A. Brooks, Artificial Intelligence 47, Jan., [Maes and Brooks 90] “Learning to Coordinate
Behaviors”, Pattie Maes and Rodney A. Brooks,
139-159.
AAAI-90, Boston, MA, 1990, 796-802.
[Brooks 91b] “Intelligence without Reason”, Rodney A.
Brooks, IJCAI-91, Sydney, Australia, Aug., 569-595, [Mahadevan and Connell 90] “Automatic Programming
of Behavior-based Robots using Reinforcement
Learning”, Sridhar Mahadevan and Jonathan Connell,
[Brooks 91c] “New Approaches to Robotics”, Rodney A.
IBM T.J. Watson Research Report, Dec.
Brooks, Science 253, Sep., 1227-1232.
[Mataric 90] “A Distributed Model for Mobile Robot
[Collins and Jefferson 90] “AntFarm: Towards Simulated
Environment-Learning and Navigation”, Maja J .
Evolution”, Robert J. Collins and David R. Jefferson, i n
Mataric, MIT A.I. Lab Technical Report 1228, May.
[Langton et al 90], 579-601.
[Pollack and Ringuette 90] “Introducing the Tileworld:
[Friedberg 58] “A Learning Machine, Part I”, R. M.
Experimentally Evaluating Agent Architectures”, Martha
Friedberg, IBM Journal of Research and Development 2,
E. Pollack and Marc Ringuette, AAAI-90, Boston, MA,
2-13.
August, 183-189.
[Friedberg et al 59] “A Learning Machine, Part II”, R. M.
[Ray 90] “An Approach to the Synthesis of Life”,
Friedberg, B. Dunham, and J. H. North, IBM Journal o f
Thomas S. Ray, in [Langton et al 90], 371-408.
Research and Development 3, 282-287.
[Viola 90] “Adaptive Gaze Control”, Paul A. Viola, MIT
[Holland 75] “Adaptation in Natural and Artificial
SM Thesis, 1990.
Systems”, John H. Holland, University of Michigan
Press, Ann Arbor, MI.
[Wang and Beni 90] “Distributed Computing Problems
in Cellular Robotic Systems”, Jing Wang and Gerardo
Beni, IEEE/RSJ International Workshop on Intelligent
Robots and Systems, Ikabara, Japan, 819-826.

[Watkins 89] “Learning from Delayed Rewards”,

Christopher Watkins, Ph.D. Thesis, King's College,
Cambridge.

Figure 2: A statistical summary of a walking program for the

robot Attila. The columns have the same labels as those in figure
1. Notice that some behaviors are duplicated six times-once for
each leg of the robot.