Introduction to Cognitive

Science
Brief overview, syllabus
Vishnu Sreekumar
[Link]@[Link]
Outline
• Big questions in cognitive science
• What is cognitive science
• A brief history of cognitive science
• Course syllabus
Course Syllabus and Logistics
Random mini-quizzes
• Attendance is mandatory (will follow institute policy).
• Attention checks through mini-quizzes that will be conducted at a
random point in class for extra credit of up to 5% (which can be the
difference between an A and A-).
Chat GPT
• All term papers: mandatory section on the output of Chat GPT or
equivalent. (~2500 words)
• Prompts + links to the output
• Your critique of the output and a comparison with what you yourself
wrote.
• Next class: a demo of how exactly to do this.
Optional
• Project and term paper based on your own project.
• Example from last year: Do LLMs have theory of mind? Giving LLMs
false-belief tests.
Johannes Stoetter – fine art body painter
Everything is not as it seems
• Top-down processing
• Our experiences, memories, etc
influence what we perceive
• Illustrated through many visual
and auditory illusions
• Bistable perception?
• Helmholtz (1860s)
Color Constancy
Some big questions in cognitive science
• How are we able to rapidly process and interpret external
information?
• How do all our sensations unify into a seamless unified experience of
the world? – “The binding problem”
• How do we form and retrieve memories?
• How do we make decisions in a noisy uncertain world?
Cognitive Science: Interdisciplinary study of
the mind with a focus on computation

As opposed to neuroscience which focuses on the neural apparatus that implements these computations

As opposed to psychology which focuses on the behavior, feelings, and thought processes of individuals and
groups of people, and also addresses neuropsych issues, mood disorders, personality, social psych, etc

Despite the small differences in focus, there is significant overlap in topics and approaches amongst cognitive
science, psychology, and neuroscience.
How are we able to rapidly process and interpret external
information?
• Neurons: the brain’s communicators.
• 100 billion neurons.
• Lined up side to side: Kashmir to Kanyakumari 4-5 times.
• Many neurons forge tens of thousands of connections with other
neurons → a staggering 160 trillion connections in the brain!
• The world is complex, the brain even more so. A lot of pieces have to
come together to explain how we are able to do this.
A lot of pieces have to come together to explain how we are able to rapidly process and interpret information!

attention

Multisensory
perception Memory
and replay,
regulation preplay!
(pulvinar)

Predictive
brain Mental
(Bayesian simulations
brain)
 The predictive brain
“The core task of all brains … is to regulate the organism’s internal
milieu … by anticipating needs and preparing to satisfy them before
they arise.”

-Sterling & Laughlin (2015), Principles of Neural Design (p. xvi)

Do these theories have any real-world value?
• Yes – both from a basic science perspective and in terms of
application (e.g. treating mental/psychological disorders)

• [Link]
Top-down effects?
• [Link]
However,…
An Unsolved Problem in Perception

• Perception of an apple consists of many different

components: size, shape, color, smell, weight,
associations, etc., etc., etc.
• How are these different components and senses integrated into a
single perception?
• This is called the binding problem
Other significant practical applications
• Brain-Computer Interfaces (BCIs)
• [Link] (a short 2 min video on the latest
breakthrough in BCIs)
• [Link] (a longer 7 min video with
explanation)

• How many different fields could you identify that contributed to this
breakthrough?
Computational neuroscience
Signal Processing (ECE)!
Machine learning
Natural language processing models for predictive text
Computational linguistics
Curiosity-driven research
• Public speaking. Do pauses, uhms and ahs have any effect on how
people remember information?
INTERDISCIPLINARY approach
• Division of labour
• Psychology – cognitive psychology, developmental psychology …
• Linguistics – syntax, semantics, phonology …
• Neuroscience – brain structures, localization …
• Computer science – AI, computer models …
• Philosophy – theoretical foundations …
Why study cognitive science?
• Understand the diverse approaches to understanding how the human
mind works
• Appreciate that different fields (AI, Philosophy, Neuroscience,
Psychology, Physics, etc) that contribute to this endeavor
• Understand the potential applications of cognitive science in society
• It’s fun to think about thinking! → cognitive science
A brief history of cognitive science
• The roots date back far in history but the real genesis of the
interdisciplinary cog sci field lies in the 1950s (Bechtel, Abrahamsen,
Graham, 2001: reading uploaded on Moodle)
• Symposium of information theory (MIT) – Sept 11, 1956
• Alan Newell, Herbert Simon (computer scientists), Noam Chomsky
(linguist), and George Miller (psychologist) presented work that each
took a cognitive turn
• Miller (1979) wrote about how he left the symposium with a
conviction about how experimental psychology, linguistics, and the
computer simulation of cognitive processes were all part of a larger
whole.
Developments in Psychology
• A huge influence of behaviorism early on in psychology: B.F. Skinner
even opposed positing internal processes and focused on only what
was observable: behavior changes with stimulus (and
reinforcement/reward structure) changes.
• However, some people thought differently. Tolman for example even
posited that rats navigate environments by forming cognitive maps.
• Behaviorism and S-R models → Cognitivism and S-O-R models →
computational view of the mind → connectionism → dynamical
systems approaches.
• Combined with advances in brain recording techniques (more in
lecture #4)
Behaviorism
• Thorndike and the law of effect
• [Link]
WyQJo&ab_channel=GeertStienissen
• No “Aha” moment. Trial and error learning.
• Behavior changes because of its consequences – “The law of effect”
The Law of Effect

Time to find lever

decreased with # of trials
(gradual learning)
Cognitive Models of Learning
• Behaviorism: S-R models
(Stimulus-Response)
• Skinner rejected cognitive
models because thoughts
are unobservable (radical
behaviorism)
• Where does thinking come
into play?
The Thinker (French: Le Penseur) is a bronze sculpture by Auguste Rodin, usually placed
on a stone pedestal. The work shows a nude male figure sitting on a rock with his chin
resting on one hand as though deep in thought, and is often used as an image to
represent philosophy.
S-O-R Models

• S-O-R Models: Stimulus-

Organism-Response

• The organism’s
interpretation is just as
important as the stimulus
in determining behavior
Latent Learning

• Rats that weren’t

reinforced developed
spatial representations
of the maze anyway
• Cognitive maps are a
challenge to radical
behaviorism
Latent Learning in a Maze
Cognitivism →The computer model of the mind
• The mind is like a computer.
• Information Processing View of the mind
• A distinctive feature of cognitive science.

Slide from Prof Bapi Raju

What is the computer model?
• The mind is an information processing system.
• Information processing is best explained by
computations and symbols.
• Information processing in the computer = programs
operating on symbols.
• Information processing in the brain = neural computations
involving mental representations.

Slide from Prof Bapi Raju

 A typical computer
• Inside a computer, we have :
• Symbols
e.g. HTML color codes

symbols are objects to which meaning can be assigned.

• Programs
e.g. Str_replace( “I have a cat” , “c” , “h” )

programs are procedures for manipulating symbols.

Slide from Prof Bapi Raju

Mental representations
• Mental representations Boring!
are symbols in the brain
that have meaning or
encode information.
• Thinking P ~ Activating a
mental representation
that means P.

Slide from Prof Bapi Raju

Information processing in the mind
• Perception
• acquiring real-time information about the surrounding environment.
• Language use
• making use of information about syntax, semantics and phonology.
• Reasoning
• combining different sources of information, deriving new information, testing consistency of
information, etc.
• Action
• making use of information in action planning and guidance.
• Memory
• storing and retrieving information

Slide from Prof Bapi Raju

Marr’s Three Levels of Analysis
How would you even study this complex
system?
Marr (1982)
“Trying to understand perception by studying only neurons is like trying to understand bird
flight by studying only feathers; it just cannot be done. To understand bird flight, you need
to understand aerodynamics, only then can one make sense of the structure of feathers
and the shape of wings. Similarly, you can’t reach an understanding of why neurons in the
visual system behave the way they do, just by studying their anatomy and physiology.”

“The nature of the computations that underlie perception depends more upon the
computational problems that have to be solved than upon the particular hardware in
which their solutions are implemented”

What the heck does this mean?

PLEASE READ Marr (1982) – uploaded on Moodle!

Three Levels of Description
(David Marr: 1945-1980)
• A complete understanding of a computational system has to involve
three levels:
I. Computation: What is the input? What is the output? What is the
information that needs to be extracted and in order to solve what
problem? Is the output readily inferred or extracted from the input?
(usually no) – what additional assumptions must one make to make
the problem tractable?
II. Algorithm/representation (software): which computational
procedures are used
III. Implementation (hardware): how the computations are implemented

Inspired by Nancy Kanwisher’s lecture on Marr’s levels of analysis

 Example (applied to social behavior): Waiting in line
for office hours

[Link]
 Source of

Example: color constancy illumination

𝐼(λ)
𝐿(𝜆)

𝑅(𝜆)

L(λ) = R(λ) × 𝐼(λ)

Goal: Determine color of a square (R or reflectance).

Information we have: The light coming to our eye from the object (L)
However, L also depends on the illuminant I! Usually we don’t have direct access to I. So this is an
ill-posed or underdetermined problem: Determine R given L. Like asking let XY = 54, determine X.
• Many problems in cognitive science are similarly ill-posed.
• We have to bring in additional assumptions to make these problems
tractable.
• E.g. to solve the color constancy problem (i.e., how the brain achieves it),
we must first analyze what sorts of constraints we must impose on spectral
reflectances and illuminants. This is how theory is developed in cognitive
science.
• Such an analysis of the problem to be solved, the information we have, the
additional information we must assume to make a solution possible, etc
constitute the computational level of Marr’s 3 levels of analysis.
Another illustration from color vision
• Watch from [Link] [Link]
can-we-study-human-mind-and-brain-marr%E2%80%99s-
level%E2%80%99s-analysis
 Example: word learning
Gavagai!

Philosopher
Quine:
• Ears?
• Fur?
• Eyes?
• Rabbit?
• Detached
rabbit parts?

The problem of “reference”

Another ill-posed problem
Additional assumptions: where people look, social cognition, etc
II. Algorithm/representation
• How does the system do what it does?
• Can we write the “code” to do this task?
• What assumptions, computations, and representations?
• How would we find out?

• In color vision, how we would find out is via psychophysics!

Inspired by Nancy Kanwisher’s lecture on Marr’s levels of analysis

Watch here: [Link]
III. Hardware
• How is the computation physically realized in neurons and brains and
even computer models?
• How would we find out?
• via brain recordings (EEG, iEEG, single unit, fMRI, MEG, etc), computer
simulations, etc.

Inspired by Nancy Kanwisher’s lecture on Marr’s levels of analysis

Watch here: [Link]
Marr’s Three-Level Approach
• Computation
• Specifies the
problem

• Algorithm
• Specifies the way
the problem is
solved
• Implementation
• Specifies the
medium or
physical substrate
in which the
problem-solving
procedure is
executed
Slide from Prof Bapi Raju
 Summary of Marr’s levels of analysis

We need many levels of analysis to understand what minds do and what they are made up of!
Methods in Cognitive Science
• Thinking/theorizing
• Writing code
• Analyzing natural stimuli
• Physics, ecology
• Psychophysics (RT, accuracy, etc)
• Individual differences
• Neuropsych patients
• fMRI, ECoG, EEG, MEG, single units, etc
• And many more…
• Not always a one to one relationship with Marr’s levels of analysis, but they are
very useful as a guide when analyzing a given problem in cognitive science.
Summary
• Cognitive science as an inter-disciplinary science of mind and
intelligence.
• The computational approach : using computations and
representations to explain mental processes.

Slide from Prof Bapi Raju

Beyond mental representations
• Embodied cognition (sometimes a radical version of it: Anthony Chemero)
– you are not just your brain, pay attention to the body and the
environment (as opposed to Descartes’ Dualism: the mind was physically
separate from the body).
• Lakoff and Nunez: The mind arises from the nature of our brains, bodies, and bodily
experiences…
• Has only really been studied empirically in the last few decades
• Dynamical Systems (e.g. dynamical field theory explanations for A not B
tasks) – rejects the information processing view of the mind, eschews the
need for mental representations, seeks to understand cognitive systems as
dynamical systems that change over time using nonlinear dynamical
systems theory.
• Will be covered in a later lecture
Work at CogSci Lab
 Cognitive Science Research Focus
◼Faculty: Bapi Raju, Priyanka Srivastava, Kavita Vemuri,
Vinoo Alluri, Vishnu Sreekumar, Bhaktee Dongaonkar

◼Understanding Cognition
◼Spatial Cognition (Navigational abilities of visually impaired
vs. sighted people, 360 degree vision)
◼Sensory Systems (Visual and Auditory)
◼Functional and Structural Analysis of the brain networks
involved in functions such as Language, Memory, Emotion,
Empathy, etc.
◼Neurodynamics of memory and learning
◼Stress and memory
 Cognitive Science Research Focus
• Simulation Systems
• Large-scale models of the brain
• Connectomics: Resting state network analysis using fMRI,
MEG, EEG
• Intelligent data analysis methods using Structural and
Functional Neuroimaging data
• Structure-Function relationship in health and disease using
Computational Modeling
 Cognitive Science Research Focus
• Assistive Systems
• For Dyslexia and Dementia
• Human-Machine Interface (Gesture Recognizers, Game
Interfaces, Vehicle Control Systems, Virtual Reality, etc.)
• Neuro-Rehabilitation
• Time Perception
• Behavioral, EEG, Patient Studies, Computational Models
• Sense of Number
• Behavioral, Computational modeling
• Motor Sequence Learning
• Hand-motor / Oculomotor using Behavioral, fMRI, Computational
Dr Bapi Raju
modeling on Normal, Hearing-handicapped, etc.
• Action Representation and Learning
• Large Scale Models of Brain
Sight Without Light Enhancing Vision

Dr Priyanka Srivastava
Creative Thinking
Neurorehabilitation through games

Dr Kavita Vemuri
Predicting Depression
from Music Listening
Habits Dr Vinoo Alluri
Individual traits
modulate brain
connectivity Dr Vinoo Alluri
Decoding brain
states at rest in
Cocaine users Dr Vinoo Alluri
 [Link]
Dr Vishnu Sreekumar
3. Neural dynamics (iEEG/EEG/fMRI)

1. Extended reality studies of human 2. Language and Memory

memory