City Research Online

City, University of London Institutional Repository

Citation: Alonso, E., Sahota, P. & Mondragon, E. (2014). Computational Models of

Classical Conditioning: A Qualitative Evaluation and Comparison. Paper presented at the
6th International Conference on Agents and Artificial Intelligence (ICAART 2014), 06-03-
2014 - 08-03-2014, Angers, France.

This is the published version of the paper.

This version of the publication may differ from the final published version.

Permanent repository link: [Link]

Link to published version:

Copyright: City Research Online aims to make research outputs of City,

University of London available to a wider audience. Copyright and Moral Rights
remain with the author(s) and/or copyright holders. URLs from City Research
Online may be freely distributed and linked to.

Reuse: Copies of full items can be used for personal research or study,
educational, or not-for-profit purposes without prior permission or charge.
Provided that the authors, title and full bibliographic details are credited, a
hyperlink and/or URL is given for the original metadata page and the content is
not changed in any way.
City Research Online: [Link] publications@[Link]
 Computational Models of Classical Conditioning
A Qualitative Evaluation and Comparison

Eduardo Alonso1,2, Pavandeep Sahota1 and Esther Mondragón2

1
Department of Computer Science, City University London, London EC1V 0HB, U.K.
2
Centre for Computational and Animal Research Centre, St. Albans AL1 1RQ, U.K.

Keywords: Computational Models, Classical Conditioning, Quantitative and Qualitative Evaluation, Comparison.

Abstract: Classical conditioning is a fundamental paradigm in the study of learning and thus in understanding
cognitive processes and behaviour, for which we need comprehensive and accurate models. This paper aims
at evaluating and comparing a collection of influential computational models of classical conditioning by
analysing the models themselves and against one another qualitatively. The results will clarify the state of
the art in the area and help develop a standard model of classical conditioning.

1 INTRODUCTION phenomena. In such context, it is widely

acknowledged that computational modelling plays a
In natural environments, there is a constant need for fundamental part (e.g., Dayan and Abbot, 2001;
organisms to accommodate their behaviour to Schmajuk, 1997; 2010a).
dynamic surroundings. Learning to predict the There are two main motivations for using
regularities in such sensory rich conditions is the key computational models: on the one hand, be it in the
for adaptive behaviour and decision-making. form of a specific programming language or as a
Predictive learning studies have mostly been formal model, implementations require
conducted within the context of classical unambiguous definitions that make the underlying
conditioning –which is based on the principle that psychological models more precise. On the other
repeated pairings of two events will allow an hand, algorithms allow us to execute calculations
individual to predict the occurrence of one of them rapidly and, most importantly, accurately. The
upon presentation of the other, as consequence of the outputs of a simulation feedback the psychological
formation of a link between them (see Mackintosh, models –thus becoming an essential part of the cycle
1994; Pearce and Bouton, 2001; Hall, 2002). This of theory formation and refinement. Automation is
simple idea is at the basis of many associative critical, particularly when models are described in
learning phenomena and has proved to be relevant to non-linear equations that can only be solved
human learning both theoretically (judgment of numerically as it is the case of recent models of
causality and categorization, e.g., (Shanks, 1995)) conditioning (Vogel et al., 2004; Schmajuk, 2010b;
and practically, as the core of a good number of Alonso and Mondragón, 2011). In particular,
clinical models (Haselgrove and Hogarth, 2011; (Schmajuk and Alonso, 2012) brought together as a
Schachtman and Reilly, 2011). special issue on computational models of classical
The last 50 years has seen the progressive conditoining a collection of papers that represent the
refinement of our understanding of the mechanisms leading edge of the field. Henceforth we are
of classical conditioning and this has resulted in the referring to the papers in the issue by acronysms of
development of several influential theories that are the models themselves or the by the initials of the
able to explain with considerable precision a wide authors if none was given, that is, we are coining
variety of experimental findings, and to make non- them GP, LCT, SLGK, PHK+, TD, MKM/APECS,
intuitive predictions that have been confirmed. This AMAN and SOCR, respectively. Notwithstanding
success has spurred the development of increasingly the relative merits of each model, as a theoretical
sophisticated models that encompass more complex corpus (Schmajuk and Alonso, 2012) showed that
there is no unanimity on what the basic principles

544 Alonso E., Sahota P. and Mondragón E..

Computational Models of Classical Conditioning - A Qualitative Evaluation and Comparison.
DOI: 10.5220/0004903105440547
In Proceedings of the 6th International Conference on Agents and Artificial Intelligence (ICAART-2014), pages 544-547
ISBN: 978-989-758-015-4
Copyright c 2014 SCITEPRESS (Science and Technology Publications, Lda.)
 ComputationalModelsofClassicalConditioning-AQualitativeEvaluationandComparison

and mechanisms of classical conditioning are or on models that seem more promising in the search for a
standard procedures to investigate them. Although standard model of classical conditioning.
there is agreement, or at least some convergence, The evalution consists of two phases: a
that learning is driven by the minimization of preliminary analysis of the software used in each
prediction error (but see Witnauer et al. above for a case. Additionally, we are also considering how
different view), the models considered differ intuitive the underlying psychological assumptions
substantially on the nature of stimulus representation of each model are, and other factors such as how
(configural vs. elemental), the role of attention in the many domains of phenomena each model crosses,
formation of associations, and about how temporal that is, their generality, and whether they account for
properties affect conditioning. critical phenomena (for instance, latent inhibition or
In order to build more comprehensive theories of spontaneous recovery). Before proceeding, it should
classical conditioning it is thus critical that we carry be noted that by a “computational model” we mean
out an exhaustive analysis of such models, that is, an implementation of a (pre-existing) psychological
that we evaluate them and compare them against one model, that is, we don’t consider computational
another. Crucially, three requirements for models as formal models that act as psychological
contributors to the special issue were set (Alonso models by proxy. Also, we do not enter into the
and Schmajuk, 2012): (1) models should be tested philosophical debate about the different levels at
against a list of phenomena for which there was a which psychological phenomena can be interpreted
consensus about their reliability; (2) model and about the relationship between the so-called
parameters should be fixed across simulations; and computational level and other levels, algorithmic or
(3) authors should make available the simulations physical (see, (Alonso and Mondragón, 2012) for a
they used to test their models. In short, the models review on the uses, abuses and misuses of the
and their simulations should be replicable. concept “computational” in psychology).
The list of phenomena was compiled by
domains, as follows: acquisition phenomena (6 2.1 Software
phenomena), extinction (3), generalization (3),
discriminations (17), inhibitory conditioning (6), It is beyond the purpose of this paper to carry out
combination of separately trained CSs (3), stimulus validation and verification tests on the simulators in
competition/potentiation in training (11), CS/US wich the computational models in (Schmajuk and
preexposure effects (11), transfer (4), recovery (8), Alonso, 2012) were run. We are not checking the
higher-order conditioning (5), and temporal replicability of the results reported either. Instead,
properties (9). Phenomena were characterised as we are summarizing, Table 1, which programming
“General”, meaning that results had been language was used in each case, whether it was
demonstrated in a wide variety of documented (including a user’s guide), and whether
procedures/organisms, or “Some Data” otherwise. the code was made available.
Regardless of the advances reported, (Schmajuk
and Alonso, 2012) demontrated that models in the Table 1: Software.
area are still partial (no model covers all the
Model Language Document Code Guide
phenomena under investigation), incomplete (there SLGK C Y Y Y
are phenomena unaccounted for) and to some extent AMAN MATLAB Y Y Y
inconsistent (different models make contradictory GP MATLAB Y Y Y
predictions). (Schmajuk and Alonso, 2012) PKH+ Visual Y Y Y
represents the vanguard in computational models of Basic
classical conditioning and, at the same time, TD MATLAB N Y N
provides us with the appropriate tools to evaluate LCT MATLAB N Y N
and compare them. MKM/AMEC MATLAB N N N
SOCR MATLAB N N N

It is up to the reader to decide whether, given the

2 EVALUATION resources made available to them by the authors, the
results reported are trustworthy. We are only
The over-reaching goal of this position paper is to commenting on the programming language used and
diagnose the state of the art in computational on the software development characteristics that
modelling of classical conditioning, explain underlies all simulators. Regarding the former,
divergences and convergences, and identify those MATLAB was the preferred choice. From the point

545
ICAART2014-InternationalConferenceonAgentsandArtificialIntelligence

of view of a programmer, MATLAB is relatively problematic, since best fits leave us with snapshots
easy to learn and to use (at least, for simple of the model’s performance that are difficult to piece
applications). Speed-wise MATLAB is rather together into a comprehensive, global understanding
similar to alternatives like C, no matter whether they of the model. In addition, quantitative analysis based
compile or interpret. One of MATLAB’s on goodness-to-fit criteria can result in selecting
disadvantages is that it is not a fully bodied overly complex models that generalize poorly.
programming language, and the user is not able to Finally, comparing models is even more difficult
create modular programs and reusable code with it. with local quantitative methods. On these grounds
In addition, MATLAB is proprietary software we will prioritize global qualitative analysis over
and a proprietary language. MATLAB works only local quantitative analysis.
with MathWork’s MATLAB software – meaning (Wills and Pothos, 2012a; 2012b) have
that if you have created programs in MATLAB, you convincingly argued that relative adequacy, defined
will generally only be able to use those programs in in terms of the number and proportion of
MATLAB, and would need to do extensive porting irreversible, ordinal successes, might be a useful
to move to a different platform. MATLAB is not a metrics for model evaluation and comparison.
platform-independent language. Central to their approach is the concept of
More generally, most simulators are not irreversible success, that is, success in the absence
professionally developed, failing to address the of arbitrarily variable free parameters. In addition,
following issues: parameters should be determined at the level of the
 Inputting data is cumbersome. domain of phenomena that the model is intended to
address, not at the level of individual experiments.
 The system must be run afresh each time the
This seemingly uncontroversial proposal, that a
input parameters are changed.
model that accommodates more successes is, other
 Outputs cannot be directly exported and
things being equal, a better model, contrasts sharply
manipulated in widespread data processors such
with current practice in classical conditioning
as, for example, excel.
research, which is to examine in depth the results of
 Interfaces and visualization of data are poor. a single or a handful of experiments, rather than to
 Simulators are not portable across platforms. seek breadth. Moreover, some researchers insist that
 Simulators cannot be scaled up to accommodate model parameters should be derived independently
new parameters and/or models. on each occasion. These practices make the
Although classical conditioning software has been evaluation and comparison of computational models
recently described in the literature (Schultheis et al., of classiscal conditioning harder. To circumvent the
2008a; 2008b; Thorwart et al., 2009; Alonso et al., difficulties posed by using arbitrary free parameters,
2012; Mondragón et al., 2013a; 2013b), it is still the (Schmajuk and Alonso, 2012) required the authors
case that most psychologists in the area view to use fixed parameters across all simulations
simulations as mere tools rather than as an integral (notice, however, that we didn’t penalize the number
part of experimental methodology. Software is of parameters à la BIC). However, the fact that most
developed, implemented and documented in an ad models were tested against small datasets remains an
hoc manner, raising serious concerns about its issue. The results in terms of numer of parameters
reliability, usability and scalability. and number of phenomena replicated are shown in
Table 2. We are not disputing that the models in
2.2 Qualitative Analysis (Schmajuk and Alonso, 2012) may account for more
results than those explictely reported. However we
The very essence of a model refers to the choices can only evaluate the models in the light of the
scientists make –choices that reflect what they evidence provided.
consider relevant– and thus evaluating a model Of course, the meaning of these results is
requires careful consideration of many factors, both debatable. Nevertheless, it gives researchers in the
technical and formal (Baum, 1983). However, in area a guide of the predictive power of the models.
assessing and selecting models (and in identifying In terms of the number of phenomena replicated, it
which features a good model should show) it is seems that SLGK is the most comprehensible model.
critical that we use measurable criteria (see (Shiffrin On the other hand, LCT uses only one parameter –
et al., 2008) for a recent survey). Typically, the which makes us wonder about its real value. It is
behaviour of a model is considered locally, that is, at preferable to endorse models whose verbal
its best fitting parameter values. This approach is description allows some understanding of the

546
 ComputationalModelsofClassicalConditioning-AQualitativeEvaluationandComparison

Table 2: Qualitative analysis results. Mondragón, E., Alonso, E., Fernandez, A., & Gray, J.
(2013a), A Rescorla and Wagner simulator with
Model Number of Number of context compounds, Computer Methods and Programs
parameters phenomena in Biomedicine. DOI: 10.1016/[Link].2013.01.016.
replicated Mondragón, E., Gray, J., & Alonso, E. (2013b), A
SLGK 11 82 Complete Serial Compound Temporal Difference
GP 7 39 Simulator for Compound stimuli, Configural cues and
AMAN 16 38 Context representation, Neuroinformatics. DOI:
SOCR 5 38 10.1007/s12021-012-9172-z.
TD 11 10 Pearce, J. M., & Bouton, M. E. (2001), Theories of
LCT 1 16 associative learning in animals. Annual Review of
PHK+ 5 5 Psychology, 52, 111-139.
MKM/APECS Unclear Not fixed Schachtman, T. R., & Reilly, S. (2011), Associative
Learning and Conditioning Theory: Human and Non-
model’s processes in psychological terms. This Human Applications. Oxford, UK: Oxford University
property, that Willis and Pothos call penetrability is Press.
important, particularly in cases where computational Schmajuk, N. A. (1997), Animal Learning and Cognition:
models are taken as psychological models by proxy A Neural Network Approach. Cambridge, UK:
rather than as formal expressions of psychological Cambridge University Press.
Schmajuk, N. A. (2010a), Mechanisms in Classical
models (see, Alonso and Mondragón, 2012).
Conditioning: A Computational Approach.
Cambridge, UK: Cambridge University Press.
Schmajuk, N. A. (2010b)(Ed.), Computational Models of
REFERENCES Conditioning. Cambridge, UK: Cambridge University
Press.
Schmajuk, N. A., & Alonso, E. (Eds.)(2012).
Alonso, E., & Mondragón, E. (Eds.)(2011),
Computational Neuroscience for Advancing Artificial Computational Models of Classical Conditioning,
Learning & Behavior, 40(3).
Intelligence: Models, Methods and Applications,
Hershey, PA: IGI Global. Schultheis, H., Thorwart, A., & Lachnit., H. (2008a),
HMS: A MATLAB simulator of the Harris model of
Alonso, E., & Mondragón, E. (2012), Uses, Abuses and
Misuses of Computational Models in Classical associative learning, Behavior Research Methods, 40,
Conditioning. In N. Rußwinkel, U. Drewitz & H. van 442-449.
Rijn (Eds.), Proceedings 11th International Schultheis, H., Thorwart, A., & Lachnit, H. (2008b),
Conference on Cognitive Modeling (ICCM-12), pp. Rapid-REM: A MATLAB simulator of the replaced
96-100. Berlin, Germany: Universitaetsverlag der TU elements model, Behavior Research Methods, 40, 435-
Berlin. 441.
Alonso, E., Mondragón, E., & Fernandez, A. (2012), A Shanks, D.R. (1995), The Psychology of Associative
Java simulator of Rescorla and Wagner's prediction Learning. Cambridge, UK: Cambridge University
error model and configural cue extensions, Computer Press.
Methods and Programs in Biomedicine, 108, 346-355. Shiffrin, R. M., Lee, M. D., Kim, W., & Wagenmakers,
Alonso, E., & Schmajuk, N. (2012), Computational E.-J. (2008), A survey of model evaluation approaches
Models of Classical Conditioning guest editors’ with a tutorial on hierarchical Bayesian methods.
introduction, Learning & Behavior, 40(3), 231-240. Cognitive Science, 32, 1248–1284.
Baum, W. M. (1983), Matching, Statistics, and Common Thorwart, A., Schultheis, H., König, S., & Lachnit, H.
Sense, Journal of the Experimental Analysis of (2009), ALTSim: A MATLAB simulator for current
Behavior, 39, 499- 501. associative learning theories, Behavior Research
Methods, 41(1), 29-34.
Dayan, P., & Abbott, L. F. (2001). Theoretical
Neuroscience: Computational and Mathematical Vogel, E. H., Castro, M. E., & Saavedra, M. A. (2004),
Quantitative models of Pavlovian conditioning, Brain
Modeling of Neural Systems. Cambridge, MA: MIT
Press. Research Bulletin, 63, 173-202.
Wills, A.J. & Pothos, E.M. (2012a), On the adequacy of
Hall, G. (2002), Associative structures in Pavlovian and
instrumental conditioning. In H. Pashler, S. Yantis, D. current empirical evaluations of formal models of
categorization, Psychological Bulletin, 138, 102-125.
Medin, R. Gallistel an J. Wixted (Eds.), Stevens'
Handbook of Experimental Psychology, Volume 3, pp Wills, A. J. & Pothos, E. M. (2012b) On the adequacy of
Bayesian evaluations of categorization models: Reply
1-45. Hoboken, NJ: John Wiley and Sons.
Haselgrove, M., & Hogarth, L. (2011), Clinical to Vanpaemel & Lee (2012), Psychological Bulletin,
138, 1259-1261.
Applications of Learning Theory. London, UK:
Psychology Press.
Mackintosh, N. J. (1994)(Ed.), Animal Learning and
Cognition, San Diego, CA: Academic Press.

547