Master in Artificial Intelligence
and Deep Learning
Artificial Intelligence
and Deep Learning
History of AI
Prof. Ignacio Olmeda
AI LAB
PRECURSORS OF ARTIFICIAL INTELLIGENCE:
PRE-SCIENCE, LITERATURE AND CINEMA
• The idea of building a “intelligent machines” is very old: we find
references to such machines in Myths, Philosophy, Literature,
Films and others.
• In the Iliad (Homer, circa S. VIII BCE) we find the Kourai Khryseai
or Golden Maidens, who were woman-shaped, gold automatons
created by Hephaestus to attend his palace, and also the
Keledones at the temple of Apollo at Delphoi.
• In the Metamorphoses (Ovid, 8 CE) Pygmalian sculpts an ivory
statue which comes to life under the name of Galatea.
3
• Philosophers such as Aristotle (384-322 BCE) in The Politics,
dreamt about a world here machines could perform some tasks:
“If every instrument could accomplish its own work, obeying or
anticipating the will of others, like the statues of Daedalus, or the
tripods of Hephaestus, which, says the poet, “of their own accord
entered the assembly of the Gods;”if, in like manner, the shuttle
would weave and the plectrum touch the lyre without a hand to
guide them, chief workmen would not want servants, nor masters
slaves”.
• So, form the point of view of Aristotle, the human condition is
linked to what machines can do or cannot do.
4
• In old India, the “bhuta vahana yanta,” or “spirit movement
machines” in Pali and Sanskrit, guarded the relics of the Budha
in Pataliputta (now Patna) under the rule of King Ajatasatru (492-
460 BCE)
• In China King Lan Ling, who lived during the Northern Qi
Dynasty (550-577 A.D.) invented a robot that could dance and
even before, Lie Zi from the 3rd Century BCE described a singing
and dancing robot made of wood and leather that performed for
King Mu of Zhou.
5
• In the Middle Ages we find artifacts that can be ssen as far ancestors
of latter “calculation machines” and which simulated some sort of
“intelligence” or “automation”.
• One paramount example are the ideas presented in the Ars Magna of
Ramon Lull (1235-1316) where he proposed some sort of “automated
reasoning” and laid ideas for the creation of “machines” or “artifacts”
capable of validating or refuting some arguments.
6
• The ubiquitous polymath Leonardo da Vinci (1452-1519), also
experimented with primitive robots: Leonardo’s “Knight” was
able to sit, move arms or open his jaw “autonomously”.
7
• One of the very few and primitive “robots” that still work is the
Mechanical Monk, a creation attributed to Juanelo Turriano, a
16th-century Spanish clockmaker, who built the robot
comissioned by Spanish king Felipe II to glorify the miraculous
recovery of his son from severe illness.
Mechanical Monk, 1562
8
• In the Jewish folklore we find the Golem a magically animated
antropomorphic entity created from clay or mud and which
follows the orders of his creator.
Rabbi Judah Loew y Golem (גלם,), S. XVI
9
• By 1642 Schickart was able to construct one to add and substract
numbers of up to six digits and in 1642 Pascal created the first of
a long series of calculating machines,(now called the pascalina)
motivated by helping his father who had to do tedious
calculations by hand.
• Leibnitz, also relevant for his investigations on calculus and logic,
wich had a tremendous impact on future AI and other sciences,
built the Step Reckoner in 1674, that was able to add, substract
and multiply.
Pascalina, 1642
10
• In the Leviathan (1651) Hobbes he suggest the creation of an
“artificial animal”.
• In the proper context Hobbes reffers to the State but settles an
interesting comparison between the “human” and the
“machine”:
“NATURE, the art whereby God hath made and governs the world,
is by the ‘art,’ of man, as in many other things, so in this also
imitated, that it can make an artificial animal. For seeing life is but a
motion of limbs, the beginning whereof is in some principal part
within; why may we not say, that all ‘automata’ (engines that move
themselves by springs and wheels as doth a watch) have an artificial
life? For what is the ‘heart’ but a ‘spring’; and the ‘nerves’ but so
many ‘strings’; and the ‘joints’ but so many ‘wheels,’ giving motion
to the whole body, such as was intended by the artificer? ‘Art’ goes
yet further, imitating that rational and most excellent work of nature,
‘man.’”
11
• In 1655 in De Corpore Politico he also refered to computation in
the following terms:
“By reasoning I understand computation and to compute is to
collect the sum of many things added together at the same time, or
to know the remainder when one thing has been taken from
another. To reason therefore is to add or to substract.”
12
• The 18th Century was particularly fruitful in the creation of
mechanical machines and was also the first opportunities to see
scams and fakes.
• One example was “the Turk” (or the Mechanical Turk, which
inspired the name to AWS) built by von Kempelen in 1769.
• It was supposed to a completely autonomous machine capable
of playing chess the truth is that it hid a professional player of a
short stature so, at the end, it was ot a real automata but a hoax.
13
• Goethe, in his Sorcerer’ s Aprentice (1797) imagined a broom
capable of following its creator’s instructions to clean.
• Perhaps, the most well known example of an automata (not an
intelligent one, though) is the “mechanical duck” of Vaucanson
of 1738 who was able to move its wings, quack, drink and digest
grain.
14
• We all may remember Frankenstein: The Modern Prometheus
the gothic novel of Mary Shelley, written in 1818 which presents
us an artificial creature (biologically made, though) that comes to
life as an experiment with unexpected consequences.
15
• That same year (1818) Babbage conceptually created the
Difference Engine, which was able to perform calculations using
finite differences. Notice that all these artifacts were just able to
perform “one shot” calculations, that is they were not
programmable.
• In fact, the first known programmable machine was the Analytical
Engine also created by Babbage around 1834. Ada Lovelace, the
daughter of Lord Byron is acknowledged to be the first
programmer in history (a role that has beed disputed recently)
for producing programs for the Analytical Engine.
16
• In 1863 Samuel Butler, a futurist writer, published the paper “Darwin
among the Machines,” in which he raises the possibility that
machines would, by the means of evolution, overtake humans and
make us their slaves.
• In the third quarter of the 18th Century, pychologists Wundt and
James attempted to understand how brain could work and build the
mental processes, laying ground for the new science of Psychology.
Latter, Freud (1856-1939) took a step further postulating internal
components of the brain such as the id, ego and superego and
creating a method of introspection (the psychoanalysis).
17
• The world “robot” was first used by Karel Čapek, in his play
Rossum’s Universal Robots or R.U.R. (1920), “robot” in czech
means “forced labor”.
18
• Novelists like Isaac Asimov put the basis of many ideas of
Artificial Intelligence.
• In Asimov’s book I Robot (1950) he introduced the influential
Three Laws of Robotics:
• First Law: A robot may not injure a human being, or, through
inaction, allow a human being to come to harm.
• Second Law: A robot must obey the orders given it by
human beings except where such orders would conflict with
the First Law.
• Third Law: A robot must protect its own existence as long as
such protection does not conflict with the First or Second
Law.
• Later he added the Zeroth Law: “ A robot may not harm
humanity, or, by inaction, allow humanity to come to harm.”
19
• Cinema is probably the main reference for many when trying to
find expressions of intelliget systems.
• In ×1927 Fritz Lang directs the film Metropolis, based on a novel
by his wife Thea von Harbou, in the film the robot Hel (an
“evilish” version of Maria, the heroin of the movie) threatening
humanity.
2001 A SPACE ODDYSSEY, 1968
20
• Few years later (1934) in the film Der Herr der Welt (Master of the
World), Henry Piel presented the dilemma of whether AI
technology will be useful to relieve humans of dangerous,
unhealthy and uncreative jobs or will transform into a weapon of
war.
2001 A SPACE ODDYSSEY, 1968
21
• In 1939 Westinghouse presented Elektro the Moto Man who was
a moving robot capable of ansewring with pre-recorded
messages.
2001 A SPACE ODDYSSEY, 1968
22
• We all remember references like Hall 9000 in Kubricks’s 2001: A
Space Oddissey (1968).
23
• Blade Runner (1982) showed us a more human side of AI and
raised the question of consciousness
2001 A SPACE ODDYSSEY, 1968
24
• While the archi-famous Terminator (1984) offered a quite different
version of human-like machines.
2001 A SPACE ODDYSSEY, 1968
25
• Many other examples are well known: The Matrix (1999), A.I.
(2001) or Ex Machina (2015)
2001 A SPACE ODDYSSEY, 1968
26
THE ADVENT OF A SCIENCE
• As we have seen prominent scientists as well as artists, thinkers,
philosophers etc. laid the foundation fo Artificial Intelligence.
• Even though limits of Science are very fuzzy most of historians
locate at the begining of the Twentieth Century the new sciences
that, in a coherent way, analyzed the human brain and the
processes it performs such as perception and cognition.
• Spanish Santiago Ramon y Cajal and Italian Camilo Golgi
obtained the Nobel Prize in Medicine, in 1906, for their study for
the structure of the nervous system laying the foundations of a
whole set of new sciences and technologies.
28
• Ramon y Cajal and Golgi showed that the “processing unit” that
explained that the cognitive capabilities of the brain were based
on a relatively simple structure, the neuron, consisting on a
soma, an axon and multiple dendrites.
• Electric signals were received by neuronts that transformed the
weighted inputs and fired if some threshold was achieved
continuing the process of transmitting electrical signals to other
neurons.
29
• Structurally, they showed that neurons send pulses through their
axons wich are transmitted to other neurons across the synapses.
• The pulses are aggregated in the soma on the receiving neuron
and, isf some threshold is exceeded, a new pulse is transmitted
to other neurons.
• This conceptually simple structure is, still, the most successful
paradigm in AI and it is called Artificial Neural Networks or Deep
Learning.
30
• In 1937 Claude Shanon published his MIT thesis “A Symbolic
Analysis of Relay and Switching Circuits” where he proposed that
it was possible to use arrangements of relays to solve problems
enuncated in Boolean logic.
• In the same year Alan Turing published the paper “On
Computable Numbers, with an Application to the
Entscheidungsproblem.” where he showed that no algorithm
could determine whether an arbitrary mathematical statement
was true or false.
• Besides the importance of the proof on undecidability of some
decision problems, the importance for computing, and for AI is
taht he also proposed an all-purpose computing machine.
31
• Some years later, in 1943, McCulloch and Pitts demonstrated that
the neuron was in essence a logical unit and that neurons that
produced an output of “0” or “1”, arranged in networks, were
capable of performin any computable function.
• In 1949 Hebb affirmed that ”when an axon of cell A is near
enough to excite B and repeatedly of persistently takes part in
firing it, some growth process or metabolic change takes place in
one or both cells such that A’s efficiency, as one of the cells firing
B, is increased”.
• This observation is of paramount importance since it laid the
foundations of the understanding of learning (in fact, that
principle is now known as hebbian learning). Hebb also swowed
that neurons tend to fire together in cell assemblies arguing that
thinking was a complex process of sequential activation of such
assemblies.
32
• In 1948 at the Hixon Symposium of Cerebral Mechanics
psychologist Karl Lashley (1890-1958) laid the foundations of
cognitive science.
• Cognitive Science is intimately related to AI, since its purpose is
to elucidate mental processses using concepts such as goals,
tasks and strategies.
• Essentially Lashley argued that the brain is not a static structure
that simply respodns to stimula but a hierarchical structure that
evolves on time (that is, it has plasticity).
• In 1950 two undergraduate students at Harvard, Marvin Minsky
and Dean Edmonds, built the first neural network computer.
33
• In a following conference, in 1955, Allen Newell delivered a
conference on machines that could play chess, this paper
sketched some important ideas such as incorporating the
notions of goals, aspiration levels for terminating search,
satisfaction of targets or multidimensional evaluation functions
that are used nowadays.
• Around the same years, psychologist Skinner (1904-1990) tried to
extend these ideas focussing more on scientific methods
analyzing behavior (how we react to specific stimula) more than
by analyzing “the mind“ by itself.
• Other paramount figure is linguist Noam Chomsky. He argued
that humans are endowed, by nature, with an universal grammar
that accounts for our ability to learn and use language so that
lingustic abilities are innate, not learnt.
34
• Even though other meetings were also held e.g. The mentioned
Hixon Symposium almost everybody accepts that the Darmouth
Summer Project of 1956 maked the “birth” of a new Science:
Artificial Intelligence.
• The “Darmouth Conference” was the first one focussed on:
“…to proceed with the basis of the conjeture that every aspect of
learning or any other feature of intelligence can in principle be so
precisely described that a machine can be made to simulate it. An
attepmt will be made to find out how to make machines use
language, form abstraction and concepts, solve kind of problems
now reserved for humans, and improve themselves ”
35
• The Conference was attended by Trenchard More, Arthur
Samuel, Ray Solomonoff, Oliver Selfridge, John McCarthy,
Marvin Minsky, Claude Shannon, Nathaniel Rochester, Allen
Newell and Herbert Simon.
• These ten researchers are now recognized as the “fathers” of AI.
DARMOUTH COLLEGE, 1956
36
• In the conference, the interests of researchers were quite
different anticipating the future “conflict” between symbolic and
subsymbolic approaches: e.g. while Rochester was foccused on
neuron-like computing devices Newel and Simon interest was in
the symbol-processing approach while Minsky was moving from
the neural network area to the symbolic one.
• One of the big issues at t conference was the presentation by
Newell and Simon of the Logic Theorist (LT), a working computer
program which was able to solve theorems in symblic logic which
provided evidence that processing symbol structures was
essential for inteligence.
• Besides the LT the Conference did not present any major
breakthroughs but it allowed to create a network of reaserchers
mostly from MIT, CMU, Stanford and IBM) that would dominate
the area for the ext two decades.
37
• 1956 was a particularly important year for the area of AI, that
same year Miller presented his famous paper “The Magical
number Seven” showing the limits of human cognition.
• Essentially Miller showed that the memory capacity of humans
was limited to seven pieces of information (plus minus two),
which was one of the first studies in the study of human
cognition.
• Newell and Simon’s General Problem Solver, was designed to
imitate human problem-solving protocols. The program
considered subgoals, possible actions and oredering showing
similar way in which humans approached similar problems, so
GPS is considered as the first program to think in a “humanlike”
manner.
38
• In 1958 John McCarthy invented LISP () the first programming
language specifically designed for AI and which became the
paradigmatical language in AI for the next decades.
• Marvin Minsky begun to analyze simplifications of the real world
to understand how intelligece works, he imagined a number of
“microworlds” in which certain task should be performed.
• One well know example is the SHRDLU world in which tasks such
as “find the highest block and put in a box” were performed.
39
• The final fifties and mid sixties saw the explosion of
connectionism: McCulloch and Pitss, Vinograd and Cowan or
Widrow and Hoff (among others) presented the first basic
neurons (ADALINE from Adaptive Linear Unit) as well as the first
networks of the (MADALINE, Multiple ADALINE).
• They also presented the first formal demonstrations that basic
networks (known as Perceptrons) were capable of mapping
arbitrary inputs to arbitrary outputs showing that, to the extend
that any funcion between inputs and outputs exists, it can be
learnt.
40
• By those years, we also see the irruption of other bio-inspired
techniques that alowed machines to “evolve” in a similar fashion
as the living organisms.
• Fierdberg (1958) and later Fogel (1966) demonstrated that
performing a series of small mutations to a machine-code
program allows to can generate programs with good
performance for some particular task, laying the foundations of
Genetic Algorithms and Genetic Programming.
41
• In the following years the understanding of the basics of learning
and the structure of processing elements as well as the success in
solving some interesting problems made expectations to reach a
level on confidence on this new science provbably too high:
• “…. there are now in the world machines that think, that learn
and that create. Moreover, their ability to do these things is
going to increase rapidly until—in a visible future—the range of
problems they can handle will be coextensive with the range to
which the human mind has been applied” (Herbert Simon, 1958).
• This overconfidence was due to the misconception that simple
examples cold be escalated to more complex ones but the real
world demonstrated that the horizon was farther away..
42
• One of the issues that researchers missed at that point was
context: machines were good in manipulating conceots in a
syntactic manner but they were also uncapable of understanding
meanings.
• A paramount example is the famus translation of the sentence
from Englsh to Russian : “the spirit is willing but the meat is
weak” that was re-translated to English as “the vodka is good
but the meat is rotten”.
• Another issue had to do with the complexity of the real world,
microworlds could be useful experiments but certainly solutions
do not necessarily scale to solve problems of the real world.
• An important point that was learnt is that even though machines
could be able to find a solution they do not necessarily will be
able to find (they do not need to be constructive, in a
mathematical sense).
43
• Finally, the formal tools that were available for demonsrating the
solvability of problems and the construyion of algorithms were
also weak.
• The publication of Perceptrons, by Minsky and Papert in 1969,
where they showed that the simple XOR function can not be
implemented by a single layer perceptron had a demoulishing
effect: “if neural nets can not solve this simple problem, how
could they be useful for more complicated tasks?”.
• By the end of the sixties it was clear that, though useful,
experience from microworlds could not easily scale to the real
world, more domain-specific knowledge was needed. 44
• In 1969 Ed Feigenbaum with other colleagues developed the
DENDRAL system for analysing molecular structures.
• It was that it was the first knowledge-intensive system and its
expertise was built using large numbers of special- rules
• The Dendral project scalated to a whole program with the aim
of building expert systems for several aplications, one of most
successful was the MYCIN system, created by Shortliffe, which
was capable of diagnosing blood infections.
• MYCIN consisted on about 450 rules, one important aspect of
this system is that the rules had to be built by interviewing
experts in the area.
• Another important innovation was the introduction of certainty
factors for the rules trying to mimick the way that doctors employ
in their diagnosis.
45
46
47
• The 70s and 80s AI was mostly dominated by Expert Systems and
the first commercial AI based systems were implemented.
• The first one of them was the R1 System developed by Digital
Equipment Corporation and was used to configure computer
systems.
• For the first time in history AI-based systems demonstrated their
usefullness outside of the academic context and companies like
DuPont began to experience significative cost reduction in the
implementation of processes using Expert Systems.
• Even though money flowed into AI in the eighties companies
began to experience the limits of Expert Systems which showed
unable to solve important problems such as natural language
processing.
• After the unfulfilled promise of AI systems to deliver general
purpose solutions and AI winter came and investment and
interest shrinked. 48
• By the mid eighties, neural networks emerged again as an
alternative to expert systemas and other approaches such as
logicism.
• One paramount moment happend with the publication in 1986
of the book Parallel Distributed Processing by Rumelhart and
MacClelland where the ubiquitous backpropagation algorithm
was presented.
• In the last part of the eighties and the mid nineties strong
advances were made in both understanding the cognitive
capabilities of the brain as well as in the formalization of the
properties of artificial neural networks.
• An explosive number of architectures were proposed in those
dates: Hopfield Networks, Self Organizing Maps, Fuzzy Cognitive
Maps, Bayesian Networks, etc.
49
• The first decade of this century saw another implosion of
conectionist models and, consequently, of AI: difficult problems
such as Natural Language Processing or Image Recognition still
proved to be a difficult task to these architectures and a new
“winter” happened.
• In contrast, the advances of the last decade can be catalogued
as impressive: in very few years many hard problems were solved
using new architectures, algorithms and ideas.
• This fact, together with the advances in computing power and
the availability of data led us to the actual situation were AI
systems, once again, allow us to dream of a bright future.
50
PRESENT AND FUTURE
• As we have seen, AI has many interconnections with other
Sciences, e.g.:
• Philosophy: Which formal rules may be used to draw
conclusions, How knowledge arises? Which is the nature of
Consciousness?
• Economics: How to design systems so that incentives are in
line with our desires? What are the consequences of
choosing a particular payoff? How to coordinate artificial
systems and humans in the most profitable way?
• Neuroscience: How the brain processes information? Which
are the structures that encode and support knowledge?
52
• Psychology: How humans think? Which are the internal
objectives that explain some particular behaviour?
• Computer Science: How processing machines
(computers) should be designed? How algorithms can
be designed and evaluated?
• Mathematics and Statistics: Which are the formal
foundations of learning? How we modelize randomness
and incomplete information?
• Linguistics: which is the relationship between language
and mind? How language is structured?
• These interconnections of AI with other sciences amplify the
opportunities for AI.
• AI has become a crucible of new ideas and applications.
53
• AI it is now an Icon of modern culture but also, it has
consolidated as a serious discipline:
• The pace of development has been so fast in the last few years
that it is almost impossible to envision the future.
54
• One conclusion for the last few years is that Machine Learning
has over taken knowledge-based systems
55
• There have been shifts in the interest for different concepts,
moving from direct conceptualization to learning.
56
• Among Machine Learning Methods, Neural Networks are the
clear winners.
57
• Among Machine Learning some techniques are more trending
than others:
58
• As mentioned, it is extremelly difficult to forecast which will be the
top areas in AI in the future, still we can enunciate some “hot”
topics that may lead the progress in the near future, among them:
• Human Computer Interaction: understanding of the the effects
of humans and machines that now share a common space.
• Convergence with disruptive technologies: such as Blockchain,
IoT, ubiquitous computing,…
• Sustainable AI: less demanding algorithms in terms of
computation (energy and time) and data
• Democratization of AI: understanding of all the aspects of AI
and accesibility of citizens to AI systems
• Improvements in algorithmics and, particularly, in architectures
(e.g. Quantum Computing and specific purpose chips: IPU).
59
