AI UNIT - 5

Natural Language Processing (NLP):

 Introduc on:

 Defini on: The field of computer science concerned with the interac on between computers and
human (natural) languages.

 Significance: Facilitates communica on, learning, and scien fic inquiry through language-based
interac ons.

 Reasons for NLP:

 Communica on with Humans:

 Enables convenient speech-based interac ons between humans and computers.

 Natural language is more user-friendly than formal languages like predicate calculus.

 Learning:

 Provides access to vast amounts of knowledge encoded in natural language.

 Resources like Wikipedia contain extensive factual informa on not readily available in formal
logic.

 Advancing Scien fic Understanding:

 U lizes AI techniques in conjunc on with linguis cs, psychology, and neuroscience to study
languages and language use.

 Contributes to insights into human cogni on and communica on processes.

 Tasks in NLP:

 Mathema cal Models for Language:

 Examines various models such as probabilis c models, neural networks, and symbolic
approaches.

 Achievable Tasks:

 Understanding and genera ng human language.

 Transla on between languages.

 Sen ment analysis and opinion mining.

 Informa on extrac on and text summariza on.

 Ques on answering and dialogue systems.

Language Models:
1. Formal vs. Natural Languages:

 Formal languages (e.g., first-order logic) are precisely defined with syntax and seman cs.

 Natural languages (e.g., English, Chinese) are ambiguous and vague, varying in interpreta on.

2. Language Model Defini on:

  Language models assign probability distribu ons to strings, indica ng likelihood in a given language.

 They aid in predic ng word sequences, sugges ng comple ons, and performing various NLP tasks.

3. Bag-of-Words Model:

 Describes word genera on as drawing from bags of words for each category.

 Although simplis c, it's useful for tasks like classifica on (e.g., Naive Bayes) but disregards word order.

4. N-gram Models:

 Depicts word probability based on context (preceding words), with varying lengths (unigram, bigram,
trigram).

 Effec ve for classifica on tasks like sen ment analysis, author a ribu on, and language iden fica on.

5. Smoothing in N-gram Models:

 Addresses low-frequency word combina ons with techniques like Laplace smoothing and backoff models.

 Aims to reduce variance and improve model performance.

6. Word Representa ons:

 Structured models like word embeddings capture seman c and syntac c rela onships between words.

 Offer be er generaliza on than atomic models like n-grams.

7. Part-of-Speech (POS) Tagging:

 Assigns gramma cal categories to words, aiding in various NLP tasks.

 Models like HMM and logis c regression are used, with features encoding word proper es and context.

8. Comparing Language Models:

 N-gram models vary in fluency and novelty of generated text, with more complex models outperforming
simplis c ones.

 Deep learning models like GPT-2 and CTRL produce fluent, diverse text but lack coherence in genera ng a
coherent thesis.

Grammar and Syntac c Structure

Subtopic 1: Introduc on to Grammar

 Defini on of Backus-Naur Form (BNF)

 Purpose of a grammar: defining tree structure for allowable phrases

 Difference between formal language (like first-order logic) and natural languages

 Importance of hierarchical structure in natural language

 Example: "Stocks rallied on Monday" demonstra ng hierarchical structure

Subtopic 2: Syntac c Categories and Phrase Structure

 Explana on of syntac c categories (noun phrase, verb phrase, etc.)

 Role of syntac c categories in constraining probable words within a sentence

 Importance of phrase structure in determining sentence seman cs

Subtopic 3: Probabilis c Context-Free Grammar (PCFG)

  Introduc on to PCFG as a language model based on hierarchical syntac c structure

 Defini on of PCFG and its probabilis c nature

 Explana on of "context-free" aspect: rules can be used in any context

 Example of a PCFG grammar rule and its interpreta on

Subtopic 4: Overgenera on and Undergenera on

 Defini on and explana on of overgenera on (genera ng ungramma cal sentences) and undergenera on
(rejec ng gramma cal sentences)

 Acknowledgment of the limita ons of the simple grammar model

 Men on of future improvements in learning a be er grammar

Subtopic 5: The Lexicon of a Language

 Defini on and purpose of the lexicon (list of allowable words)

 Dis nc on between open classes (e.g., nouns, verbs) and closed classes (e.g., pronouns, preposi ons)

 Challenges of lis ng all words in open classes due to their vastness and constant evolu on

 Example of historical changes in closed class words (e.g., "thee" and "thou")

Subtopic 6: Lexicon Example and Summa on

 Presenta on of the lexicon for the discussed language

 Clarifica on of categories such as open class and closed class

 Men on of the sum of probabili es for each category totaling 1

This breakdown organizes the informa on into clear topics and subtopics, making it easier to understand the
content.

Topic: Parsing
Subtopic 1: Introduc on to Parsing

 Defini on: analyzing a string of words to uncover its phrase structure according to grammar rules

 Analogies: search for a valid parse tree, either top-down or bo om-up

 Challenges: inefficiency of pure top-down or bo om-up parsing strategies due to repea ng efforts

Subtopic 2: Parsing Strategies

 Top-down and bo om-up parsing strategies

 Limita ons: difficul es in dis nguishing between different sentence types (e.g., commands vs. ques ons)

 Solu on: dynamic programming to store analyzed substrings to avoid reanalysis (e.g., chart parsing)

Subtopic 3: CYK Algorithm

 Overview of the CYK algorithm for parsing

 Requirement: grammar rules in Chomsky Normal Form (CNF)

 Transforma on: any context-free grammar can be automa cally transformed into CNF

 Efficiency: space and me complexity of the CYK algorithm

Subtopic 4: Beam Search Parsing

 Introduc on to beam search for parsing

 Goal: find the most probable parse with a limited search space

 Trade-off: not guaranteed to find the highest probability parse, but operates efficiently

 Example: beam search parser with probabilis c context-free grammar (PCFG)

Subtopic 5: Determinis c Parsing

 Descrip on of determinis c parsing

 Example: shi -reduce parsing, making decisions to shi or reduce cons tuents based on grammar rules

 Influence of parsing style on machine learning-based grammar induc on

Subtopic 6: Dependency Parsing

 Introduc on to dependency grammar as an alterna ve syntac c approach

 Dis nc on from phrase structure grammar

 Popularity: Universal Dependencies project and its impact on research

Subtopic 7: Learning a Parser from Examples

 Challenge of manual grammar construc on

 Approach: supervised learning using input/output pairs of sentences and parse trees

 Example: Penn Treebank as a source of annotated data for grammar learning

 Automa c grammar recovery: reducing complexity and improving generaliza on using data-oriented parsing

Subtopic 8: Unsupervised and Semisupervised Parsing

 Overview of unsupervised parsing using algorithms like the inside-outside algorithm

 Curriculum learning approach: star ng with simple sentences and gradually increasing complexity

 Semisupervised parsing: combining a small number of annotated trees with a larger corpus of unparsed
sentences

 U liza on of par al bracke ng in semisupervised parsing, leveraging annota ons like HTML tags

This breakdown organizes the informa on into clear topics and subtopics, making it easier to understand the
content.

Topic: Augmented Grammars

Subtopic 1: Introduc on to Subcategories

 Explana on of how not all words can appear in every context with equal probability

 Introduc on of subcategories to represent finer-grained dis nc ons in grammar

 Example: dis nc on between subjec ve and objec ve case for pronouns

Subtopic 2: Augmented Grammar

 Concept of augmented grammar with structured representa ons for nonterminals

 Example: representa on of noun phrase "I" as NP(Sbj, 1S, Speaker) for subjec ve case, first person singular

 Addressing preferences in parsing through augmented grammar

Subtopic 3: Lexicalized PCFG

 Introduc on to lexicalized probabilis c context-free grammar (PCFG)

 Use of phrase heads for assigning probabili es based on proper es of words in phrases

 Example: lexicalized rules for verbs and nouns to capture word proper es in parsing

Subtopic 4: Handling Agreement

 Addressing agreement in grammar rules (e.g., subject-verb agreement, case agreement)

 Augmenta on of categories with addi onal variables (e.g., subjec ve case, person, number)

 Example: grammar rule ensuring agreement between subject and verb

Subtopic 5: Seman c Interpreta on

 Introduc on to adding seman cs to grammar rules

 Example: seman c interpreta on of arithme c expressions using composi onal seman cs

 Representa on of logical terms for English seman cs using lambda calculus

 Example: grammar rules augmented with seman c interpreta ons for English sentences

Subtopic 6: Learning Seman c Grammars

 Challenges in learning seman c grammars from syntac c trees without seman c representa ons

 Example systems that learn seman c grammars from ques on-answer pairs

 U liza on of large databases of ques on-answer pairs for training parsers

 Benefits of internal logical forms for efficient parsing without exhaus ve search space

This breakdown organizes the informa on into clear topics and subtopics, making it easier to understand the
content.

Robo cs
In which agents are endowed with sensors and physical effectors with which to move about and make mischief in the
real world.

Subtopic 1: Components of Robots

 Descrip on of effectors (legs, wheels, grippers) used by robots to manipulate the physical world

 Explana on of how effectors cause changes in the robot's state, environment, and people around it

 Introduc on to sensors (cameras, radars, gyroscopes) enabling robots to perceive their environment

Subtopic 2: Maximizing Expected U lity

 Defini on of maximizing expected u lity for robots: choosing ac ons to assert physical forces that lead to
desired outcomes

 Emphasis on robots aiming to accomplish tasks in the physical world by ac ng on their environment

Subtopic 3: Challenges in Robo cs

 Explana on of environments as par ally observable and stochas c, with unpredictable human interac ons

 Descrip on of con nuous state and ac on spaces in robo c modeling, including high-dimensional spaces for
complex robots
  Constraints of robo c learning in real-world environments due to me and safety concerns

Subtopic 4: Sim-to-Real Problem

 Introduc on to the sim-to-real problem: transferring learning from simulated environments to real-world
robots

 Importance of prior knowledge about the robot, environment, and tasks for quick and safe learning in
prac cal robo c systems

Subtopic 5: Integra on with AI Concepts

 Discussion of how robo cs integrates concepts from AI, including probabilis c state es ma on, percep on,
planning, unsupervised learning, reinforcement learning, and game theory

 Recogni on of robo cs as both a challenging applica on and a fron er for advancing AI techniques,
par cularly in con nuous domains

This breakdown organizes the informa on into clear topics and subtopics, making it easier to understand the
content.

Topic: Robot Hardware

Subtopic 1: Types of Robots

 Introduc on to anthropomorphic robots, manipulators, mobile robots, and legged robots

 Descrip on of specific examples such as industrial robo c arms and wheelchair-mounted assis ve arms

 Explana on of various locomo on methods including wheels, legs, rotors, and wings

Subtopic 2: Sensing the World

 Differen a on between passive and ac ve sensors, with examples like cameras and sonar

 Overview of range finders, including sonar and stereo vision, and their applica ons

 Introduc on to ac ve op cal range finders such as me-of-flight cameras and scanning lidars

 Descrip on of radar and tac le sensors for range sensing, and their uses in robo cs

Subtopic 3: Loca on Sensing

 Explana on of loca on sensors and their reliance on range sensing for determining loca on

 Discussion of GPS as a common outdoor localiza on solu on and its limita ons indoors and underwater

 Overview of alterna ve indoor and underwater localiza on methods including beacons and sonar beacons

Subtopic 4: Propriocep ve Sensors

 Defini on of propriocep ve sensors and their role in informing the robot of its own mo on

 Descrip on of sha decoders for tracking joint posi ons, odometry for measuring distance traveled, and
iner al sensors like gyroscopes

 Explana on of force and torque sensors for measuring interac on forces with objects in the environment

Subtopic 5: Producing Mo on

 Introduc on to actuators as mechanisms ini a ng effector mo on, including electric, hydraulic, and
pneuma c actuators

 Descrip on of joints connec ng rigid bodies, including revolute and prisma c joints
  Overview of grippers used for interac ng with objects, from basic parallel jaw grippers to complex humanoid
hands

This breakdown organizes the informa on into clear topics and subtopics, making it easier to understand the content

Topic: Problem Formula on in Robo cs

Subtopic 1: Computa onal Framework

 Overview of computa onal frameworks for robo cs problems, including MDPs, POMDPs, and game theory

 Recogni on of the nondeterminis c, par ally observable, and mul agent nature of robo cs problems

Subtopic 2: Formula on of Reward Func on

 Discussion of the robot's reward func on, o en ed to serving human goals or desires

 Recogni on of the challenge in deciphering or approxima ng the user's true reward func on

Subtopic 3: Ac on, State, and Observa on Spaces

 Descrip on of the general structure where observa ons are raw sensor data, ac ons are electric currents to
motors, and state represents decision-making informa on

 Acknowledgment of the gap between low-level percepts and high-level plans, and the need for simplifica on
through decoupling

Subtopic 4: Hierarchy in Problem Solving

 Introduc on to the three-level hierarchy in robo cs problem-solving: task planning, mo on planning, and
control

 Explana on of how task planning determines high-level ac ons, mo on planning finds paths to achieve
subgoals, and control executes planned mo ons

Subtopic 5: Integra on of Separate Components

 Discussion of preference learning for es ma ng user objec ves and people predic on for forecas ng others'
ac ons

 Recogni on of the trade-off between reducing complexity and giving up opportuni es for synergy between
problem components

 Recogni on of the push for reintegra ng separate areas to achieve be er overall performance and progress
in robo cs

This breakdown provides a concise overview of the key aspects of problem formula on in robo cs, organized into
clear topics and subtopics.

Topic: Robo c Percep on

Subtopic 1: Overview

 Percep on in robo cs involves mapping sensor measurements into internal representa ons of the
environment.

 Challenges include dealing with noisy sensors, par al observability, unpredictability, and dynamic
environments.

Subtopic 2: Proper es of Good Representa ons

  Good representa ons for robots should contain sufficient informa on for decision-making, be structured for
efficient upda ng, and correspond to natural state variables in the physical world.

Subtopic 3: Computa onal Models

 Various computa onal models like Kalman filters, HMMs, and dynamic Bayes nets are used to represent
transi on and sensor models in par ally observable environments.

 Robots' past ac ons are included as observed variables in these models.

Subtopic 4: Localiza on and Mapping

 Localiza on involves determining the robot's posi on, o en using techniques like Monte Carlo localiza on
(MCL) or the extended Kalman filter (EKF).

 Simultaneous Localiza on and Mapping (SLAM) addresses scenarios where a robot simultaneously
determines its loca on and builds a map of its environment.

Subtopic 5: Other Types of Percep on

 Robots perceive various quan es like temperature, odors, and sound, o en using probabilis c techniques.

 Reac ve agents, not explicitly reasoning about probability distribu ons, are also employed in some cases.

Subtopic 6: Machine Learning in Percep on

 Machine learning plays a vital role in robot percep on, including unsupervised learning for dimensionality
reduc on and adap ve percep on techniques.

 Self-supervised methods enable robots to collect their own training data and adapt to changes in sensor
measurements.

This breakdown encapsulates the key aspects of robo c percep on, organized into clear topics and subtopics for
concise understanding.

mo on planning for robots:

1. Separa on of Mo on and Control: Mo on planning involves two main components: finding a feasible path
and execu ng the trajectory to follow that path.

2. Configura on Space: A representa on of all possible configura ons of the robot, considering its dimensions
and degrees of freedom. The configura on space helps simplify mo on planning calcula ons.

3. Mo on Planning Methods:

 Visibility Graphs: Suitable for two-dimensional configura on spaces with polygonal obstacles. They
provide guaranteed shortest-path solu ons.

 Voronoi Diagrams: Provide paths away from obstacles, ensuring safety. Useful for scenarios where
the robot's body is considered larger than its actual dimensions.

 Cell Decomposi on: Discre zes configura on space into cells, making path planning a discrete graph
search problem.

 Randomized Mo on Planning: U lizes random sampling to construct a feasible path, followed by

op miza on.

 Probabilis c Roadmaps (PRMs): Construct a graph of feasible paths based on random sampling,
useful for mul -query planning scenarios.

 Rapidly Exploring Random Trees (RRTs): Incrementally builds trees from start and goal configura ons,
connec ng them to find a feasible path.
 4. Trajectory Op miza on: Focuses on op mizing a cost func on over paths, considering factors like collision
avoidance and path efficiency. Gradient descent methods can be used for op miza on.

5. Post-processing Techniques: O en applied to improve the quality of solu ons, such as shortcu ng for RRTs
to remove unnecessary ver ces from the path.

Overall, mo on planning involves a balance between finding feasible paths, op mizing for efficiency, and ensuring
collision-free naviga on in complex environments. Various techniques and algorithms are employed based on the
specific requirements and constraints of the robot and its workspace.

Trajectory Tracking Control in Robo cs

 Introduc on to Control Theory:

 Open-loop control: Turning a mathema cal path descrip on into real-world ac ons.

 Closed-loop control: Ensuring the robot stays on track.

 Dynamics Model and Control:

 Conver ng configura ons to torques for open-loop tracking.

 Dynamics model: Describing the effects of torques on configura on.

 Inverse dynamics: Determining torque based on desired accelera on.

 Re ming:

 Transforming the path into a trajectory for tracking.

 Approximate methods for re ming using maximum velocity and accelera on.

 Control Laws:

 Propor onal (P) controller: Provides force propor onal to error.

 Propor onal-Deriva ve (PD) controller: Adds a deriva ve term to dampen oscilla ons.

 Propor onal-Integral-Deriva ve (PID) controller: Includes an integral term to correct long-term

errors.

 Computed Torque Control:

 Integra ng feedforward and feedback components to compensate for model inaccuracies.

 Adjus ng gains based on the configura on of the system.

 Op mal Control:

 Formula ng trajectory op miza on with dynamics.

 Minimizing cumula ve cost subject to constraints.

 Techniques like mul ple shoo ng and direct colloca on for trajectory op miza on.

 Linear Quadra c Regulator (LQR):

 Op mizing over an infinite horizon.

 Quadra c cost and linear dynamics.

 U lizing the Ricca equa on to find the op mal policy.

 Itera ve LQR (ILQR):

  Itera vely refining the solu on by compu ng linear approxima ons of dynamics and quadra c
approxima ons of the cost.

 Variants of LQR for trajectory tracking.

By understanding and implemen ng these control techniques, robots can efficiently track trajectories and navigate
complex environments with precision.

Planning Uncertain Movements in Robo cs

 Introduc on to Uncertainty:

 Arises from par al observability and stochas c effects of ac ons.

 Errors from approxima on algorithms like par cle filtering.

 Adapta on of Determinis c Algorithms:

 Dealing with con nuous state space by discre za on.

 Choosing the most likely state from probability distribu on for faster computa on.

 Policies for Uncertainty:

 Transi on from determinis c plans to policies.

 Online replanning: Recompu ng plans based on updated beliefs.

 Informa on Gathering Ac ons:

 Separa ng es ma on from control.

 Moving beyond determinis c MDPs to POMDPs for be er capturing uncertainty.

 Guarded Movements:

 Explicitly defining informa on gathering ac ons.

 Each movement consists of mo on command and termina on condi on.

 Example: Coastal naviga on heuris c and incen vizing ac ons for informa on gain.

 Coastal Naviga on:

 Changing cost func on to incen vize ac ons leading to informa on.

 Incorpora ng expected informa on gain into the cost func on.

 Enables the robot to invent its own informa on gathering ac ons.

By incorpora ng these techniques, robots can navigate uncertain environments more effec vely and adapt to
changes in real- me, improving their overall performance and flexibility.

Reinforcement Learning in Robo cs

 Introduc on to RL in Robo cs:

 Challenge of con nuous state and ac on spaces.

 Representa on using func on approxima on, e.g., deep neural networks.

 Challenges in Real-world Robo cs:

 Safety concerns and slower progress in real-world se ngs.

 Need to reduce sample complexity.

  Exploi ng Models:

 Model-based RL for leveraging knowledge of world dynamics.

 Local linear models for mastering dynamic tasks like juggling.

 Sim-to-real transfer for transferring policies from simula on to real-world.

 Domain randomiza on for training policies robust to varying parameters.

 Hybrid Approaches:

 Integra ng ideas from model-based and model-free algorithms.

 Itera ve improvement of policy through model fi ng and planning.

 Safe Explora on:

 Learning slowly but safely to avoid dangerous states.

 Use of approximate models with associated uncertainty to guide explora on.

 Exploi ng Other Informa on:

 Use of higher-level mo on primi ves for faster learning.

 Metalearning and transfer learning to reuse informa on from previous tasks.

 Leveraging human ac ons to guide the robot's learning process.

By addressing these challenges and leveraging various techniques, reinforcement learning can be effec vely applied
to robo cs, enabling robots to learn and adapt in real-world environments more efficiently.

Humans and Robots

 Introduc on:

 Robo cs o en involves interac on with humans.

 Two main challenges: coordina on and op mizing for human preferences.

 Coordina on:

 Modeling coordina on as a game between robot and human.

 Challenges include incomplete informa on, con nuous state and ac on spaces, and human
subop mali es.

 Predic ng human ac ons using noisily op mal assump on.

 Spli ng predic on from ac on for easier handling of interac on.

 Learning to Do What Humans Want:

 Learning cost func ons from human input.

 Preference learning: inferring cost func ons from human demonstra ons.

 Behavioral cloning: learning desired policies directly from human demonstra ons.

 Challenges in generaliza on to new states and addressing errors through itera ve learning.

 Teaching Interfaces and the Correspondence Problem:

 Kinesthe c teaching: human physically guides robot's ac ons.

 Challenges of mapping human ac ons onto robot ac ons.

  Alterna ves like keyframe demonstra ons and visual programming interfaces.

By addressing these challenges, robots can effec vely interact and collaborate with humans in various tasks,
improving overall performance and user sa sfac on.

Alterna ve Robo c Frameworks

 Introduc on:

 Delibera ve vs. reac ve approaches in robo cs.

 Reac ve controllers offer simpler policies without explicit environmental modeling.

 Reac ve Controllers:

 Reac ve agents operate based on immediate sensor feedback.

 Example: Hexapod robot using simple rules for leg movement without explicit terrain modeling.

 Control rules are o en based on reflex ac ons rather than high-level planning.

 Subsump on Architectures:

 Framework for assembling reac ve controllers from finite state machines (FSMs).

 Augmented FSMs (AFSMs) include sensor feedback and internal mers.

 Composi on of complex behaviors through hierarchical FSMs.

 Challenges and Limita ons:

 Sensor-driven approach may fail with unreliable or complex sensor data.

 Lack of delibera on limits adaptability to changing goals or environments.

 Scalability challenges for handling complex tasks with numerous condi ons.

 Conclusion:

 Robo cs employs a variety of approaches, including delibera ve, reac ve, or hybrid.

 Ongoing research and engineering efforts aim to address challenges and improve effec veness in
real-world applica ons.