17/08/2021

Why do we need to understand users?

• Interacting with technology is cognitive

• Need to take into account cognitive processes involved and

cognitive limitations of users

• Provides knowledge about what users can and cannot be

expected to do

• Identifies and explains the nature and causes of problems

users encounter
Chapter 3
• Supply theories, modelling tools, guidance and methods that
COGNITIVE ASPECTS can lead to the design of better interactive products

www.id-book.com 3

Overview Cognitive processes

• What is cognition?
• Attention
• What are users good and bad at? • Perception
• Memory
• Describe how cognition has been
• Learning
applied to interaction design
• Reading, speaking and listening
• Explain what are Mental Models • Problem-solving, planning, reasoning and decision-
making
• Cover relevant theories of cognition

www.id-book.com 2 www.id-book.com 4

1
 17/08/2021

Activity: Find the price for a double room at the Quality Inn
Attention in Pennsylvania a

• Selecting things to concentrate on at a point in time from the mass

of stimuli around us

• Allows us to focus on information that is relevant to what we are

doing

• Involves audio and/or visual senses

• Focussed and divided attention enables us to be selective in terms

of the mass of competing stimuli but limits our ability to keep track
of all events

• Information at the interface should be structured to capture users’

attention, e.g. use perceptual boundaries (windows), colour,
reverse video, sound and flashing lights

www.id-book.com 5 www.id-book.com 7

Activity: Find the price of a double room at the Holiday Inn

in Columbia Activity
• Tullis (1987) found that the two screens produced quite
different results
– 1st screen - took an average of 5.5 seconds to search
– 2nd screen - took 3.2 seconds to search

• Why, since both displays have the same density of

information (31%)?

• Spacing
– In the 1st screen the information is bunched up together, making it
hard to search
– In the 2nd screen the characters are grouped into vertical
categories of information making it easier

www.id-book.com 6 www.id-book.com 8

2
 17/08/2021

Multitasking and attention Perception

• Is it possible to perform multiple tasks without one or • How information is acquired from the world and
more of them being detrimentally affected? transformed into experiences

• Obvious implication is to design representations that

• Ophir et al (2009) compared heavy vs light multi-taskers
are readily perceivable, e.g.
– heavy were more prone to being distracted than those who
infrequently multitask – Text should be legible
– heavy multi-taskers are easily distracted and find it difficult to – Icons should be easy to distinguish and read
filter irrelevant information

www.id-book.com 9 www.id-book.com 11

Is color contrast good? Find Italian

Design implications for attention
• Make information salient when it needs attending to

• Use techniques that make things stand out like color,

ordering, spacing, underlining, sequencing and
animation

• Avoid cluttering the interface with too much information

• Search engines and form fill-ins that have simple and

clean interfaces are easier to use
www.id-book.com 10 www.id-book.com 12

3
 17/08/2021

Are borders and white space

Which is easiest to read and why?
better? Find french

What is the time? What is the time?

What is the time? What is the time?

What is the time?

www.id-book.com 13 www.id-book.com 15

Activity Design implications

• Weller (2004) found people took less time to
– Icons should enable users to readily distinguish their meaning
locate items for information that was grouped
– using a border (2nd screen) compared with using color contrast – Bordering and spacing are effective visual ways of grouping
(1st screen) information

• Some argue that too much white space on web – Sounds should be audible and distinguishable
pages is detrimental to search – Speech output should enable users to distinguish between the
– Makes it hard to find information set of spoken words

• Do you agree? – Text should be legible and distinguishable from the background

– Tactile feedback should allow users to recognize and

distinguish different meanings

www.id-book.com 14 www.id-book.com 16

4
 17/08/2021

Memory Context is important

• Involves first encoding and then retrieving knowledge. • Context affects the extent to which information
can be subsequently retrieved
• We don’t remember everything - involves filtering and
processing what is attended to • Sometimes it can be difficult for people to recall
• Context is important in affecting our memory (i.e. where, when) information that was encoded in a different
context:
• We recognize things much better than being able to recall things
– “You are on a train and someone comes up to you and says
hello. You don’t recognize him for a few moments but then
• we remember less about objects we have photographed than realize it is one of your neighbors. You are only used to seeing
when we observe them with the naked eye (Henkel, 2014) your neighbor in the hallway of your apartment block and seeing
ahim out of context makes him difficult to recognize initially”

www.id-book.com 17 www.id-book.com 19

Processing in memory Activity

• Encoding is first stage of memory
• Try to remember the dates of your grandparents’ birthday
– determines which information is attended to in the environment
and how it is interpreted • Try to remember the cover of the last two DVDs you
bought or rented
• The more attention paid to something… • Which was easiest? Why?
• The more it is processed in terms of thinking about it and • People are very good at remembering visual cues about
comparing it with other knowledge… things
– e.g. the color of items, the location of objects and marks on an object
• The more likely it is to be remembered
• They find it more difficult to learn and remember arbitrary
– e.g. when learning about HCI, it is much better to reflect upon it, material
carry out exercises, have discussions with others about it, and
– e.g. birthdays and phone numbers
write notes than just passively read a book, listen to a lecture or
watch a video about it

www.id-book.com 18 www.id-book.com 20

5
 17/08/2021

Recognition versus recall What some designers get up to…

• Command-based interfaces require users to • Present only 7 options on a menu
recall from memory a name from a possible set
of 100s • Display only 7 icons on a tool bar

• GUIs providMP3 players visually-based options • Have no more than 7 bullets in a list
that users need only browse through until they
recognize one • Place only 7 items on a pull down menu
• Place only 7 tabs on the top of a website page
• Web browsers, etc., provide lists of visited – But this is wrong? Why?
URLs, song titles etc., that support recognition
memory

www.id-book.com 21 www.id-book.com 23

The problem with the classic ‘72’ Why?

• Inappropriate application of the theory
• George Miller’s (1956) theory of how much
information people can remember • People can scan lists of bullets, tabs, menu
items for the one they want
• People’s immediate memory capacity is very
limited • They don’t have to recall them from memory
having only briefly heard or seen them
• Many designers think this is useful finding for • Sometimes a small number of items is good
interaction design
• But depends on task and available screen
• But… estate

www.id-book.com 22 www.id-book.com 24

6
 17/08/2021

Is Apple’s Spotlight search tool any

Digital content management
good?
• Is a growing problem for many users
– vast numbers of documents, images, music files, video clips,
emails, attachments, bookmarks, etc.,
– where and how to save them all, then remembering what they
were called and where to find them again
– naming most common means of encoding them
– but can be difficult to remember, especially when have 1000s
and 1000s
– How might such a process be facilitated taking into account
people’s memory abilities?

www.id-book.com 25 www.id-book.com 27

Digital content management Digital Forgetting

• When might you wish to forget something
• Memory involves 2 processes that is online?
– recall-directed and recognition-based scanning
– When you break up with a partner
• File management systems should be – Emotionally painful to be reminded of them
designed to optimize both kinds of memory through shared photos, social media, etc.,
processes
– e.g. Search box and history list • Sas and Whittaker (2013) suggest new ways
• Help users encode files in richer ways of harvesting and deleting digital content
– Provide them with ways of saving files using colour, flagging, – e.g. making photos of ex into an abstract collage
image, flexible text, time stamping, etc.
– helps with closure

www.id-book.com 26 www.id-book.com 28

7
 17/08/2021

Memory aids Design implications

• SenseCam developed by Microsoft Research • Don’t overload users’ memories with
Labs (now Autographer) complicated procedures for carrying out tasks

• a wearable device that intermittently takes • Design interfaces that promote recognition
photos without any user intervention while worn rather than recall

• digital images taken are stored and revisited • Provide users with various ways of encoding
using special software information to help them remember
– e.g. categories, color, flagging, time stamping
• Has been found to improve people’s memory,
suffering from Alzheimers

www.id-book.com 29 www.id-book.com 31

SenseCam Learning
• How to learn to use a computer-based
application
• Using a computer-based application or
YouTube video to understand a given
topic
• People find it hard to learn by following
instructions in a manual
• prefer to learn by doing

www.id-book.com 30 www.id-book.com 32

8
 17/08/2021

Cognitive prosthetic devices Reading, speaking, and listening

• We rely more and more on the internet and
smartphones to look things up • The ease with which people can read, listen, or
• Cognitive resource cf. extended mind speak differs
– Many prefer listening to reading
• Expecting to have internet access reduces the
need and extent to which we remember – Reading can be quicker than speaking or listening

• Also enhances our memory for knowing where – Listening requires less cognitive effort than reading or
to find it online (Sparrow et al,2011) speaking
– Dyslexics have difficulties understanding and
• What are implications for designing technologies recognizing written words
to support how people will learn, and what they
learn? www.id-book.com 33 www.id-book.com 35

Design implications Applications

• Design interfaces that encourage • Speech-recognition systems allow users to interact with
them by asking questions
exploration
– e.g. Google Voice, Siri
• Design interfaces that constrain and guide
• Speech-output systems use artificially generated speech
learners
– e.g. written-text-to-speech systems for the blind
• Dynamically linking concepts and • Natural-language systems enable users to type in
representations can facilitate the learning questions and give text-based responses
of complex material – e.g. Ask search engine

www.id-book.com 34 www.id-book.com 36

9
 17/08/2021

Design implications Design implications

• Speech-based menus and instructions • Provide additional information/functions for
should be short users who wish to understand more about how
to carry out an activity more effectively
• Accentuate the intonation of artificially
generated speech voices
• Use simple computational aids to support rapid
– they are harder to understand than human decision-making and planning for users on the
voices move
• Provide opportunities for making text large
on a screen
www.id-book.com 37 www.id-book.com 39

Problem-solving, planning, reasoning and Dilemma

decision-making
• The app mentality developing in the psyche
• All involves reflective cognition
of the younger generation is making it worse
for them to make their own decisions
– e.g. thinking about what to do, what the options because they are becoming risk averse
are, and the consequences (Gardner and Davis, 2013)
• Often involves conscious processes, discussion with
others (or oneself), and the use of artefacts
• Relying on a multitude of apps means that
– e.g. maps, books, pen and paper they are becoming increasingly more anxious
• May involve working through different scenarios and about making decisions by themselves
deciding which is best option
• Do you agree? Can you think of an example?

www.id-book.com 38 www.id-book.com 40

10
 17/08/2021

Mental models Everyday reasoning and mental models

• Users develop an understanding of a system through (a) You arrive home on a cold winter’s night to a cold house.
learning about and using it How do you get the house to warm up as quickly as
possible? Set the thermostat to be at its highest or to the
• Knowledge is sometimes described as a mental model: desired temperature?
– How to use the system (what to do next)
(b) You arrive home starving hungry. You look in the fridge
– What to do with unfamiliar systems or unexpected situations (how and find all that is left is an uncooked pizza. You have an
the system works) electric oven. Do you warm it up to 375 degrees first and
then put it in (as specified by the instructions) or turn the
• People make inferences using mental models of how to oven up higher to try to warm it up quicker?
carry out tasks

www.id-book.com 41 www.id-book.com 43

Heating up a room or oven that is

Mental models thermostat-controlled
• Many people have erroneous mental models (Kempton,
• Craik (1943) described mental models as:
1996)
– internal constructions of some aspect of the
external world enabling predictions to be made • Why?
• Involves unconscious and conscious – General valve theory, where ‘more is more’ principle is
processes generalised to different settings (e.g. gas pedal, gas cooker, tap,
radio volume)
– images and analogies are activated – 12

• Deep versus shallow models – Thermostats based on model of on-off switch model
– e.g. how to drive a car and how it works

www.id-book.com 42 www.id-book.com 44

11
 17/08/2021

Heating up a room or oven that is How did you fare?

thermostat-controlled
• Your mental model
– How accurate?
• Same is often true for understanding how – How similar?
interactive devices and computers work: – How shallow?

– poor, often incomplete, easily confusable, based on • Payne (1991) did a similar study and found that
inappropriate analogies and superstition (Norman, people frequently resort to analogies to explain how
1983) they work
– e.g. elevators and pedestrian crossings - lot of people • People’s accounts greatly varied and were often ad
hit the button at least twice
hoc
– Why? Think it will make the lights change faster or
ensure the elevator arrives!
www.id-book.com 45 www.id-book.com 47

Exercise: ATMs Gulfs of execution and evaluation

• Write down how an ATM works
• The ‘gulfs’ explicate the gaps that exist between
– How much money are you allowed to take out?
the user and the interface
– What denominations?

– If you went to another machine and tried the same what would happen? • The gulf of execution
– the distance from the user to the physical system
– What information is on the strip on your card? How is this used?

– What happens if you enter the wrong number? • The gulf of evaluation
– Why are there pauses between the steps of a transaction? What – the distance from the physical system to the user
happens if you try to type during them?

– Why does the card stay inside the machine? • Bridging the gulfs can reduce cognitive effort
required to perform tasks
– Do you count the money? Why?
Norman, 1986; Hutchins et al, 1986

www.id-book.com 46 www.id-book.com 48

12
 17/08/2021

Model Human processor (Card et

Bridging the gulfs al, 1983)
• Models the information processes of a user
interacting with a computer

• Predicts which cognitive processes are

involved when a user interacts with a
computer

• Enables calculations to be made of how long

a user will take to carry out a task

www.id-book.com 49 www.id-book.com 51

The human processor model

Information processing
• Conceptualizes human performance in
metaphorical terms of information
processing stages

www.id-book.com 50 www.id-book.com 52

13
 17/08/2021

How it differs from information

Limitations
processing
• Based on modelling mental activities that
happen exclusively inside the head

• Do not adequately account for how people

interact with computers and other devices
in real world

www.id-book.com 53 www.id-book.com 55

Distributed cognition
• Concerned with the nature of cognitive
phenomena across individuals, artefacts, and
internal and external representations (Hutchins,
1995)
• Describes these in terms of propagation across
representational state

• Information is transformed through different

media (computers, displays, paper, heads)

www.id-book.com 54 www.id-book.com 56

14
 17/08/2021

Externalizing to reduce memory

What’s involved
load
• The distributed problem-solving that takes place • Diaries, reminders, calendars, notes, shopping lists, to-do lists
– written to remind us of what to do
• The role of verbal and non-verbal behavior
• Post-its, piles, marked emails
• The various coordinating mechanisms that are
– where placed indicates priority of what to do
used (e.g. rules, procedures)
• External representations:
• The communication that takes place as the – Remind us that we need to do something (e.g. to buy something for
collaborative activity progresses mother’s day)
– Remind us of what to do (e.g. buy a card)
• How knowledge is shared and accessed
– Remind us when to do something (e.g. send a card by a certain date)

www.id-book.com 57 www.id-book.com 59

External cognition Computational offloading

• Concerned with explaining how we interact with
external representations (e.g. maps, notes, • When a tool is used in conjunction with an external
diagrams) representation to carry out a computation (e.g. pen
and paper)
• What are the cognitive benefits and what
processes involved • Try doing the two sums below (a) in your head, (b) on
a piece of paper and c) with a calculator.
• How they extend our cognition
– 234 x 456 =??
– CCXXXIIII x CCCCXXXXXVI = ???
• What computer-based representations can we
develop to help even more? • Which is easiest and why? Both are identical sums

www.id-book.com 58 www.id-book.com 60

15
 17/08/2021

Summary
Annotation and cognitive tracing
• Cognition involves several processes including attention,
• Annotation involves modifying existing memory, perception and learning
representations through making marks
• The way an interface is designed can greatly affect how
– e.g. crossing off, ticking, underlining well users can perceive, attend, learn and remember
how to do their tasks
• Cognitive tracing involves externally • Theoretical frameworks, such as mental models and
manipulating items into different orders or external cognition, provide ways of understanding how
structures and why people interact with products

– e.g. playing Scrabble, playing cards • This can lead to thinking about how to design better
products

www.id-book.com 61 www.id-book.com 63

Design implication
• Provide external representations at the
interface that reduce memory load and
facilitate computational offloading
– e.g. Information visualizations have been
designed to allow people to make sense and rapid
decisions about masses of data

www.id-book.com 62

16