AI Chapter 1
AI Chapter 1
Tadesse Kebede
[email protected]
Course Detail
⚫ Course code: COSC3112
⚫ Credit hours: 3 ECTS: 5
⚫ Contact hrs: 2
⚫ Lab hrs: 3
⚫ Tutorial hrs: 2
⚫ Prerequisite:CoSc2092-Data Structures and Algorithms,
STAT2015- Probability and Statistics
⚫ Course category: compulsory
⚫ Year: III Semester: II
2
Course Description
⚫ The purpose of this course is to give students an
understanding of Artificial Intelligence methodologies,
techniques, tools and results.
⚫ Students will use python programming language to
demonstrate laboratory exercises.
⚫ Students will learn the theoretical and conceptual components
of this discipline and firm up their understanding by using AI
and Expert System tools in laboratory sessions, projects and
home assignments.
3
Course objectives
At the end of this course the students will be able to:
⚫ Understand reasoning, knowledge representation and learning
techniques of artificial intelligence
⚫ Evaluate the strengths and weaknesses of these techniques and
their applicability to different tasks
⚫ Assess the role of AI in gaining insight into intelligence and
perception
⚫ Know classical examples of artificial intelligence
⚫ Know characteristics of programs that can be considered
"intelligent"
⚫ Understand the use of heuristics in search problems and
games
4
Course objectives
⚫ Know a variety of ways to represent and retrieve knowledge
and information
⚫ Know the fundamentals of artificial intelligence programming
techniques in a modern programming language
⚫ Consider ideas and issues associated with social technical, and
ethical uses of machines that involve artificial intelligence
⚫ Introduce students for powerful learning algorithms and their
applications.
⚫ Letting students to develop simple AI powered applications
either in robotics, NLP or games.
5
Course Outline
Topic Contents
Introduction to Objectives/Goals of AI, Types of AI(General and Specific AI), Approaches to AI –
AI(3 hrs) making computer: Think like a human ( Thinking humanly),Act like a human
(Acting humanly), Think rationally (Thinking rationally) & Act rationally (Acting
rationally). The Foundations of AI, Bits of History and the State of the Art,
Proposing and evaluating Application of AI
Intelligent Agents Foundation of Agents, Agents and Environments, Acting of Intelligent Agents
(4 hrs) (Rationality), Structure of Intelligent Agents: Agent Types, Simple reflex agent,
Model-based reflex agent, Goal-based agent, Utility-based agent. Multi agent
systems, Learning agent
Searching and Solving Problems by Searching and planning, Constraint Satisfaction Problem,
Planning (6 hrs) Problem Solving Agents, Problem spaces and search, Knowledge and rationality,
Heuristic search strategies, Search and optimization (gradient descent),
Adversarial search, Planning and scheduling, Avoiding Repeated States,
Dynamic game theory
Knowledge Logic and Inference, Logical Agents, Propositional Logic, Predicate (First-Order)
Representation Logic, Inference in First-Order Logic, Knowledge Representation, Knowledge
and Reasoning (8 Reasoning, Bayesian reasoning, Probabilistic reasoning, Temporal reasoning,
hrs) Knowledge-based Systems, Case study: Medical diagnosis
6
Course Outline…
Machine Learning Knowledge in Learning, Learning Probabilistic Models, Supervised learning: Linear
Basics (3 hrs) classification models, Probabilistic models., Unsupervised learning: Clustering
models, Reinforcement learning, Deep Learning: Neural networks and back-
propagation, Convolution neural networks, Recurrent neural networks and LSTMs
Natural Language Intro to Natural Language Processing, Machine learning Application in NLP,
Processing (NLP) Natural language interaction, Computer vision and Image processing, Case study:
Basics (3 hrs) Sentiment Analysis, speech recognition, Chatbot
Ethical and Legal Privacy, Bias, AI and the future of work, Appropriate uses of AI
Considerations in
AI (2 hrs)
7
Assessment methods
⚫This course is assessed by written exams, lab
assignments, reports and presentations of
assignments.
⚫Assessment Breakdown:
⚫ Assignment/quizzes 10 %
⚫ Mid semester examination 20%
⚫ Project ` 20%
⚫ Final examination 50%
8
Reading Material
⚫ Text Books:
⚫ Russell, S. and P. Norvig (1995) Artificial Intelligence: A
Modern Approach Prentice-Hall
⚫ References:
⚫ Luger, G. (2002) Artificial Intelligence, 4th ed. Addison-
Wesley.
⚫ Bratko, Ivan (1990) PROLOG Programming for Artificial
Intelligence, 2nd ed. Addison-Wesley, 1990
⚫ Winston, P.H. (1992) Artificial Intelligence Addison-Wesley.
Ginsberg, M.L. (1993) Essentials of Artificial Intelligence.
Morgan Kaufman.
9
Topic 1: Introduction to AI
Introduction to Artificial
Intelligence
Overview of AI
Objectives/Goals of AI
Types of AI(General and Specific AI)
Approaches to AI – making computer:
Think like a human ( Thinking humanly)
Act like a human (Acting humanly)
Think rationally (Thinking rationally)
Act rationally (Acting rationally)
The Foundations of AI
Bits of History and the State of the Art
Proposing and evaluating Application of AI
11
Overview of AI
⚫ We call ourselves Homo sapiens—man the wise—because our
intelligence is so important to us.
⚫ For thousands of years, we have tried to understand how we
think; that is, how a mere handful of matter can perceive,
understand, predict, and manipulate a world far larger and
more complicated than itself.
⚫ The field of artificial intelligence, or AI, goes further still: it
attempts not just to understand but also to build intelligent
entities.
⚫ AI is one of the newest fields in science and engineering.
Work started in earnest soon after World War II, and the
name itself was coined in 1956.
12
Overview of AI
⚫ AI currently encompasses a huge variety of subfields, ranging
from the general (learning and perception) to the specific,
such as playing chess, proving mathematical theorems,
writing poetry, driving a car on a crowded street, and
diagnosing diseases.
⚫ AI is relevant to any intellectual task; it is truly a universal
field.
13
Intelligence
⚫ Intelligence is the computational part of the ability to achieve
goals in the world.
⚫ Varying kinds and degrees of intelligence occur in people,
many animals and some machines.
⚫ Intelligence is the capability of observing, learning,
remembering, reasoning and acting.
⚫ AI attempts to develop intelligent agents.
Characteristics of Intelligent system
⚫ Use vast amount of knowledge
⚫ Learn from experience and adopt to changing environment
⚫ Interact with human using language and speech
⚫ Respond in real time
14 ⚫ Tolerate error and ambiguity in communication
Artificial Intelligence
⚫ It is the science and engineering of making intelligent
machines, especially intelligent computer programs.
⚫ It is related to the similar task of using computers to
understand human intelligence, but AI does not have to
confine itself to methods that are biologically observable.
⚫ Isn't there a solid definition of intelligence that doesn't
depend on relating it to human intelligence?
⚫ Not yet. The problem is that we cannot yet characterize in
general what kinds of computational procedures we want to call
intelligent. We understand some of the mechanisms of
intelligence and not others.
15
Artificial Intelligence
⚫ The concern of AI is to develop computer based system that
behave like human and emulate the reasoning power of
humans in order to do tasks that require human intelligence.
Which task requires intelligence?
⚫ Complex arithmetic operations
⚫ For instance, Solving 220 * 350?
⚫ Mundane tasks: all tasks routinely done by all of us in our day-
to-day activities:
⚫ Example, Natural language understanding; face recognition;
planning; robotics, …
⚫ Expert tasks: which require specialists knowledge
⚫ Example, Medical diagnosis; computer maintenance; financial
16 planning
Artificial Intelligence
⚫ Artificial intelligence, or AI, is the field that studies the
synthesis and analysis of computational agents that act
intelligently. Consider each part of this definition.
⚫ An agent is something that acts in an environment; it does
something.
⚫ Agents include worms, dogs, thermostats, airplanes, robots,
humans, companies, and countries.
⚫ An agent is judged solely by how it acts. Agents that have the
same effect in the world are equally good.
17
Artificial Intelligence
⚫ Intelligence is a matter of degree. The aspects that go into an
agent acting intelligently include:
⚫ what it does is appropriate for its circumstances, its goals,
and its perceptual and computational limitations
⚫ it takes into account the short-term and long-term
consequences of its actions, including the effects on
society and the environment
⚫ it learns from experience
⚫ it is flexible to changing environments and changing goals.
18
Computational Agent
⚫ A computational agent is an agent whose decisions about its
actions can be explained in terms of computation.
⚫ That is, the decision can be broken down into primitive
operations that can be implemented in a physical device.
⚫ This computation can take many forms. In humans, this
computation is carried out in “wetware”; in computers it is
carried out in “hardware.”
⚫ Although there are some agents that are arguably not
computational, such as the wind and rain eroding a landscape,
it is an open question whether all intelligent agents are
computational.
19
Computational Agent
⚫ All agents are limited. No agent is omniscient (all knowing) or
omnipotent (can do anything).
⚫ Agents can only observe everything in very specialized and
constrained domains.
⚫ Agents have finite memory.
⚫ Agents in the real world do not have unlimited time to act.
20
Objectives/Goals of AI
⚫ The central scientific goal of AI is to understand the principles
that make intelligent behavior possible in natural or artificial
systems. This is done by
⚫ the analysis of natural and artificial agents
⚫ formulating and testing hypotheses about what it takes to construct
intelligent agents
⚫ designing, building, and experimenting with computational systems that
perform tasks commonly viewed as requiring intelligence.
22
Types of AI(General and
Specific AI)
⚫ AI can generally be categorized into two types: narrow (Special-
purpose or weak) AI and general (or strong) AI.
⚫ General-purpose AI like the robots of science fiction is incredibly
hard
⚫ Human brain appears to have lots of special and general functions,
integrated in some amazing way that we really do not understand at
all (yet)
⚫ The strong AI view holds that it is possible to create intelligent
machines that can really reason and solve problems.
⚫ Such machines are considered to be conscious and self-aware, can
independently think about problems and work out optimal solutions
to problems, have their own system of values and world views, and
have all the same instincts as living things, such as survival and
security needs.
23
⚫ It can be regarded as a new civilization in a certain sense.
Types of AI(General and
Specific AI)
⚫ Special-purpose AI is more doable (nontrivial)
⚫ E.g., chess/poker playing programs, logistics planning,
automated translation, voice recognition, web search, data
mining, medical diagnosis, keeping a car on the road, … …
……
⚫ The weak AI view holds that intelligent machines cannot
really reason and solve problems.
⚫ These machines only look intelligent, but do not have real
intelligence or self-awareness.
24
Strong AI vs Weak AI
⚫ Strong AI argues that it is possible that one day a computer will be
invented which can be called a mind in its fullest sense.
⚫ Strong AI aims to create an agent that can replicate humans
intelligence completely; i.e., it can think, reason, imagine, etc., & do all
the things that are currently associated with the human brain.
⚫ Weak AI, on the other hand, argue that computers can only
appear to think & are not actually conscious in the same way as
human brains are.
⚫ The weak AI position holds that AI should try to develop systems
which have facets of intelligence, but the objective is not to build a
completely sentient/conscious entity.
⚫ Weak AI researchers see their contribution as systems like expert
systems used for medical diagnosis, which use "intelligent" models,
26
naturally occur(??).
Strong AI vs Weak AI
⚫ Now we are in the weak AI phase.
⚫ Weak AI, in contrast, is simply the view that intelligent behavior can be modeled and used by computers
to solve complex problems. This point of view argues that just because a computer behaves intelligently
does not prove that it is actually intelligent in the way that a human is.
29
Approaches to AI – making computer:
30
Thinking humanly: The
Cognitive Modeling
⚫ Reasons like humans do
⚫ Programs that behave like humans
31
Acting humanly: The Turing
Test
Can machines act like human do? Can machines behave
intelligently?
⚫ Turing Test: Operational test for intelligent behavior
⚫ do experiments on the ability to achieve human-level
performance,
⚫ Acting like humans requires AI programs to interact with
people
⚫ Suggested major components of AI include: knowledge,
reasoning, language understanding, learning
32
Thinking Rationally: The Laws
of Thought
⚫ A system is rational if it thinks/does the right thing through
correct reasoning.
⚫ Aristotle: provided the correct arguments/thought structures
that always gave correct conclusions given correct premises.
⚫ Belay is a man; all men are mortal; therefore Belay is
mortal
⚫ These Laws of thought governed the operation of the
mind and initiated the field of Logic.
33
Acting rationally: The rational
agent
⚫ Doing the right thing so as to achieve one’s goal, given one’s
beliefs.
⚫ AI is the study and construction of rational agents (an agent
that perceives and acts)
⚫ Rational action requires the ability to represent knowledge
and reason with it so as to reach good decision.
⚫ Learning for better understanding of how the world works
34
The Foundations of AI
⚫ AI is Interdisciplinary
⚫ AI is generally associated with Computer Science, but it
has many important links with other fields such as:
⚫ Mathematics,
⚫ Biology,
⚫ Economics,
⚫ Cognition,
⚫ Psychology,
⚫ Philosophy and
⚫ Neuroscience.
35
Philosophy
⚫ Can formal rules be used to draw valid conclusions?
⚫ How does the mind arise from a physical brain?
⚫ Where does knowledge come from?
⚫ How does knowledge lead to action?
⚫ Descartes was a proponent of dualism. He held that there is a part
of the human mind (or soul or spirit) that is outside of nature,
exempt from physical laws. Animals, on the other hand, did not
possess this dual quality; they could be treated as machines.
⚫ An alternative to dualism is materialism, which holds that the
brain’s operation according to the laws of physics constitutes the
mind. Free will is simply the way that the perception of available
choices appears to the choosing entity.
36
Philosophy
⚫ The empiricism is characterized by a dictum of John Locke
(1632–1704): “Nothing is in the understanding, which was not first
in the senses.”
⚫ The logical positivism holds that all knowledge can be
characterized by logical theories connected, ultimately, to
observation sentences that correspond to sensory inputs; thus
logical positivism combines rationalism and empiricism.
⚫ The confirmation theory attempted to analyze the acquisition of
knowledge from experience by quantifying the degree of belief
that should be assigned to logical sentences based on their
connection to observations that confirm or disconfirm them.
37
Philosophy
⚫ Utilitarianism: that rational decision making based on
maximizing utility should apply to all spheres of human activity,
including public policy decisions made on behalf of many
individuals. Utilitarianism is a specific kind of consequentialism:
the idea that what is right and wrong is determined by the
expected outcomes of an action.
⚫ In contrast, a theory of rule-based or deontological ethics, in
which “doing the right thing” is determined not by outcomes but
by universal social laws that govern allowable actions, such as
“don’t lie” or “don’t kill.” Many modern AI systems adopt exactly
this approach.
38
Mathematics
⚫ What are the formal rules to draw valid conclusions?
⚫ What can be computed?
⚫ How do we reason with uncertain information?
⚫ The theory of probability can be seen as generalizing logic to
situations with uncertain information—a consideration of great
importance for AI. The formalization of probability, combined with
the availability of data, led to the emergence of statistics as a field.
⚫ The history of computation is as old as the history of numbers, but
the first nontrivial algorithm is thought to be Euclid’s algorithm for
computing greatest common divisors. The incompleteness theorem
showed that in any formal theory as strong as Peano arithmetic (the
elementary theory of natural numbers), there are necessarily true
statements that have no proof within the theory.
39
Mathematics
⚫ Alan Turing tried to characterize exactly which functions are
computable —capable of being computed by an effective procedure.
For example, no machine can tell in general whether a given program
will return an answer on a given input or run forever.
⚫ Although computability is important to an understanding of
computation, the notion of tractability has had an even greater
impact on AI. A problem is called intractable if the time required to
solve instances of the problem grows exponentially with the size of
the instances.
⚫ The theory of NP-completeness provides a basis for analyzing the
tractability of problems: any problem class to which the class of NP-
complete problems can be reduced is likely to be intractable.
40
Economics
⚫ How should we make decisions in accordance with our preferences?
⚫ How should we do this when others may not go along?
⚫ How should we do this when the payoff may be far in the future?
⚫ Decision theory, which combines probability theory with utility
theory, provides a formal and complete framework for individual
decisions (economic or otherwise) made under uncertainty—that is, in
cases where probabilistic descriptions appropriately capture the
decision maker’s environment.
⚫ Models based on satisficing—making decisions that are “good
enough,” rather than laboriously calculating an optimal decision—
gave a better description of actual human behavior.
41
Neuroscience
⚫ How do brains process information?
⚫ Neuroscience is the study of the nervous system, particularly the
brain. A collection of simple cells can lead to thought, action, and
consciousness.
⚫ Even with a computer of virtually unlimited capacity, we still require
further conceptual breakthroughs in our understanding of
intelligence.
42
Psychology
⚫ How do humans and animals think and act?
⚫ Wundt insisted on carefully controlled experiments in which his
workers would perform a perceptual or associative task while
introspecting on their thought processes. The behaviorism
movement rejected any theory involving mental processes on the
grounds that introspection could not provide reliable evidence.
⚫ Cognitive psychology views the brain as an information-processing
device. Three key steps of a knowledge-based agent: (1) the stimulus
must be translated into an internal representation, (2) the
representation is manipulated by cognitive processes to derive new
internal representations, and (3) these are in turn retranslated back
into action.
⚫ Intelligence augmentation states that computers should augment
43 human abilities rather than automate away human tasks.
Computer engineering
⚫ How can we build an efficient computer?
44
Control theory and cybernetics
⚫ How can artifacts operate under their own control?
45
Linguistics
⚫ How does language relate to thought?
⚫ Modern linguistics and AI, then, were “born” at about the same time,
and grew up together, intersecting in a hybrid field called
computational linguistics or natural language processing.
46
THE HISTORY OF ARTIFICIAL
INTELLIGENCE
The inception of artificial intelligence (1943-1956)
⚫ The first work that is now generally recognized as AI was done by
Warren McCulloch and Walter Pitts (1943).
⚫ They proposed a model of artificial neurons. Year 1949: Donald Hebb
demonstrated an updating rule for modifying the connection strength
between neurons. His rule is now called Hebbian learning.
Early enthusiasm, great expectations (1952-1969)
⚫ The intellectual establishment of the 1950s, by and large, preferred to
believe that “a machine can never do X”. John McCarthy referred to
this period as the “Look, Ma, no hands!” era.
A does of reality (1966-1973)
⚫ In almost all cases these early systems failed on more difficult
problems. The illusion of unlimited computational power was not
47 confined to problem-solving programs.
THE HISTORY OF ARTIFICIAL
INTELLIGENCE
Expert systems (1969-1986)
⚫ The picture of problem solving that had arisen during the first decade
of AI research was of a general-purpose search mechanism trying to
string together elementary reasoning steps to find complete solutions.
Such approaches have been called weak methods. The alternative to
weak methods is to use more powerful, domain-specific knowledge
that allows larger reasoning steps and can more easily handle typically
occurring cases in narrow areas of expertise.
The return of neural networks (1986-present)
⚫ In the mid-1980s at least four different groups reinvented the back-
propagation learning algorithm first developed in the early 1960s.
48
THE HISTORY OF ARTIFICIAL
INTELLIGENCE
Probabilistic reasoning and machine learning (1987-present)
⚫ In the 1980s, approaches using hidden Markov models (HMMs) came
to dominate the area. Pearl’s development of Bayesian networks
yielded a rigorous and efficient formalism for representing uncertain
knowledge as well as practical algorithms for probabilistic reasoning.
Big data (2001-present)
⚫ Remarkable advances in computing power and the creation of the
World Wide Web have facilitated the creation of very large data
sets—a phenomenon sometimes known as big data.
Deep learning (2011-present)
⚫ The term deep learning refers to machine learning using multiple layers of
simple, adjustable computing elements. Experiments were carried out with such
networks as far back as the 1970s, and in the form of convolutional neural
networks they found some success in handwritten digit recognition in the
49
1990s.
The State of the Art
⚫ ROBOTIC VEHICLES
⚫ LEGGED LOCOMOTION
⚫ AUTONOMOUS PLANNING AND SCHEDULING
⚫ MACHINE TRANSLATION
⚫ SPEECH RECOGNITION
⚫ RECOMMENDATIONS
⚫ GAME PLAYING
⚫ IMAGE UNDERSTANDING
⚫ MEDICINE
⚫ CLIMATE SCIENCE
50
Applications of AI
Solving problems that required thinking by humans:
⚫ Playing games (chess, checker, cards, ...)
⚫ Great advances: the AI software Deep Blue beats human expert
Kasparov.
⚫ Proving theorems (mathematical theorems, laws of physics, …)
⚫ Classification of text (Politics, Economic, Social, Sports, etc,)
⚫ Information filtering and summarization of text
⚫ Writing story and poems; solving puzzles
⚫ Giving advice in Medical diagnosis, Equipment repair, Computer
configuration, Financial planning,
51
How to make computers act
like humans?
The following sub-fields have emerged
⚫ Natural Language processing (enable computers to communicate in
human language, English, Amharic, ..)
⚫ Knowledge representation (schemes to store information, both facts
and inferences,…)
⚫ Automated reasoning (use stored information to answer questions
and to draw new conclusions)
⚫ Machine learning (adapt to new circumstances and accumulate
knowledge)
⚫ Computer vision (recognize objects based on patterns in the same
way as the human visual system does)
⚫ Robotics (produce mechanical device capable of controlled motion
with the ability to move, see, hear, and accordingly take actions in the
52 world, possibly responding to new perceptions)
AI vs. HI?
The following sub-fields have emerged
⚫ Artificial Intelligence (or AI) is the field that explores to
develop a system that think in the same sense as humans do.
⚫ Remember computer based chess program (Deep Blue) that beats
human expert (Gary Kasparov). What do you understand from
this?
⚫ Does AI equal human intelligence?
⚫ Is it possible to create a computer system called mind?
⚫ What is our concern in designing an Intelligent agent?
⚫ Is it to replace human beings or to support and give leverage
to them so that the humans can engage themselves in expert
works?
53
Can we build hardware as
complex as the brain?
⚫ How complicated is our brain?
⚫ a neuron, or nerve cell, is the basic information processing unit in our
brain
⚫ estimated to be on the order of 10 pow 11 neurons in a human brain
⚫ many more synapses (10pow14) connecting these neurons
⚫ cycle time: 10 pow-3 seconds (1 millisecond)
⚫ How complex can we make computers?
⚫ 106 or more transistors per CPU
⚫ supercomputer: hundreds of CPUs, 10pow9 bits of RAM
⚫ cycle times: order of 10pow- 8 seconds
⚫ Conclusion
⚫ POSSIBLY: in the future we may have computers with as many basic
processing elements as our brain, but with far fewer interconnections
(wires or synapses) than the brain and much faster updates than the brain
⚫ but building hardware is very different from making a computer behave
54
like a brain!
Can Computers Talk?
⚫ This is known as “speech synthesis”
⚫ translate text to phonetic form
⚫ e.g., “fictitious” -> fik-tish-es
⚫ use pronunciation rules to map phonemes to actual sound
⚫ e.g., “tish” -> sequence of basic audio sounds
⚫ Difficulties
⚫ sounds made by this “lookup” approach sound unnatural
⚫ sounds are not independent
⚫ e.g., “act” and “action”
⚫ modern systems (e.g., at AT&T) can handle this pretty well
⚫ a harder problem is emphasis, emotion, etc
⚫ humans understand what they are saying
⚫ machines don’t: so they sound unnatural
⚫ Conclusion: NO, for complete sentences, but YES for individual
words
55
Can Computers Recognize
Speech?
⚫ Speech Recognition:
⚫ mapping sounds from a microphone into a list of words.
⚫ Hard problem: noise, more than one person talking,
occlusion, speech variability,..
⚫ Even if we recognize each word, we may not understand
its meaning.
⚫ Recognizing single words from a small vocabulary
⚫ systems can do this with high accuracy (order of 99%)
⚫ e.g., directory inquiries
⚫ limited vocabulary (area codes, city names)
⚫ computer tries to recognize you first, if unsuccessful hands you over to a human
operator
⚫ saves millions of dollars a year for the phone companies
56
Recognizing human speech
(ctd.)
⚫ Recognizing normal speech is much more difficult
⚫ speech is continuous: where are the boundaries between words?
⚫ e.g., “John’s car has a flat tire”
⚫ large vocabularies
⚫ can be many tens of thousands of possible words
⚫ we can use context to help figure out what someone said
⚫ try telling a waiter in a restaurant:
“I would like some cream and sugar in my coffee”
62
Risks and Benefits of AI
⚫ LETHAL AUTONOMOUS WEAPONS
⚫ SURVEILLANCE AND PERSUASION
⚫ BIASED DECISION MAKING
⚫ IMPACT ON EMPLOYMENT
⚫ SAFETY-CRITICAL APPLICATIONS
⚫ CYBERSECURITY
63
The End of Topic 1