The Evidential Value and Pitfalls of Extant Language Development Research Methods
The Evidential Value and Pitfalls of Extant Language Development Research Methods
The Evidential Value and Pitfalls of Extant Language Development Research Methods
Author Note
Abstract
Several research methods have been developed with an aim to uncover the underlying cognitive and
neural mechanisms behind child language acquisition. While these methods continue to generate
promising empirical evidence, language researchers need to carefully examine the evidential value
and limitations of findings that were derived from these methods before building on them. Similarly,
language practitioners (e.g., speech-language pathologists/therapists, teachers) need to be aware that
while these methods may form or resemble the foundations of language assessment and intervention
practices (Seiger-Gardner & Almodovar, 2017; Deevy, 2017), no single paradigm can
comprehensively account for the multifaceted nature of “typical” and “atypical” language
development that we see in real-world settings, especially in multilingual contexts. If such is the case,
how can we utilise the evidence that these methods generate in practical and clinical settings? To
answer this question, this paper reviews and evaluates select language acquisition research methods
based on the following criteria: (1) ecological validity of stimuli, (2) sensitivity of cognitive-linguistic
measure, (3) suitability of comparison groups, (4) procedural suitability, (5) precision of response
classification, (6) communicative sense (Ambridge & Rowland, 2013), and (7) practical implications
(e.g., evidential value for bi/multilingual populations and in practitioner settings).
The Evidential Value and Pitfalls of Extant Language Development Research Methods
Several research methods have been developed with an aim to uncover the underlying cognitive
and neural mechanisms behind child language acquisition. While these methods continue to generate
promising empirical evidence, language researchers need to carefully examine the evidential value and
limitations of findings that were derived from these methods before building on them. Similarly,
language practitioners (e.g., speech-language pathologists/therapists, teachers) need to be aware that
while these methods may form or resemble the foundations of language assessment and intervention
practices (Seiger-Gardner & Almodovar, 2017; Deevy, 2017), no single paradigm can comprehensively
account for the multifaceted nature of “typical” and “atypical” language development that we see in
real-world settings, especially in multilingual contexts. If such is the case, how can we utilise the
evidence that these methods generate in practical and clinical settings? To answer this question, this
paper reviews and evaluates select language acquisition research methods based on the following
criteria: (1) ecological validity of stimuli, (2) sensitivity of cognitive-linguistic measure, (3) suitability
of comparison groups, (4) procedural suitability, (5) precision of response classification, (6)
communicative sense (Ambridge & Rowland, 2013), and (7) practical implications (e.g., evidential
value for bi/multilingual populations and in practitioner settings).
As with any human cognitive ability that has been subject to scientific investigation, observable
behaviours (Ambridge & Rowland, 2013) provide the widest window for measurement. For child
language, spoken language production is perhaps the most conventionally observable modality that can
facilitate straightforward paradigms for data collection and analysis. One of such paradigms is
naturalistic/spontaneous language sampling. Spontaneous language sampling can be done by
recording, transcribing, and analysing children’s naturalistic speech during typical family/caregiver
interactions (Marzan, 2013; Ambridge & Rowland, 2013; Seiger-Gardner & Almodovar, 2017). This
method holds stronger ecological validity, communicative sense, and procedural suitability compared
to experimental and neurophysiological methods due to the fact that it affords real-world data with the
least interference from the researcher/s and their equipment. With respect to sensitivity of its linguistic
measures and evidential value, spontaneous language sampling has been successfully used to profile
the phonological skills (e.g., phonetic repertoire), lexical growth (e.g., type-token ratio to measure
lexical diversity), morphosyntactic abilities (e.g., argument structure, mean length of utterance indices,
grammatical variety), and pragmatic skills (e.g., interchange type analysis) of, for example, typically-
developing bilingual Filipino children (see Marzan, 2013) as well as of monolingual children with
language disorders (see Roberts et al., 1998; Owen & Leonard, 2002).
developmental language norms that can be used to guide assessment and intervention practices for both
multilingual and monolingual populations in various learning settings (e.g., home, therapy services,
schools). For instance, a longitudinal naturalistic language sampling study (Marzan, 2013) revealed that
typically developing Filipino children (ages 1:2-5:0) equally use Tagalog and English verb stems (e.g.,
in Tagalog-English structure, “mag-wawash lang ako ng hands, okay?”, that is equivalent to the English
structure, “I will just wash my hands, okay?”) but predominantly prefer to use English nouns (e.g.,
baby, ball, fish, book, girl, dog, tree, shoe) in daily conversations. Marzan (2013) also reported that
these bilingual children use more pronouns (e.g., Tagalog: ito, ko, iyan, ako, mo; English: I, it) than
nominals when producing contingent nonverbal predicate + topic syntactic structures (e.g., Tagalog
structure, “akin iyan”, that is equivalent to the English structure, “that is mine”). Moreover, Marzan
(2013) reported that this Filipino cohort exhibited higher language productivity when they share their
experiences compared to their productivity with other pragmatic functions such as when regulating
(e.g., directing action, obtaining an object). These findings can then better inform practitioners in
selecting appropriately normed language learning and communicative targets for Tagalog-English
bilingual children.
may precede, follow, or simultaneously appear with the target pictures (Seiger-Gardner & Almodovar,
2017). The effects of the interfering stimuli to children’s naming are then measured by comparing
response latencies during related and unrelated conditions (Seiger-Gardner & Almodovar, 2017).
Consistent findings from studies (e.g., Brooks & MacWhinney, 2000; Jerger et al., 2002;
Jescheniak et al., 2006) that used PWI paradigms have revealed that typically developing children
exhibit early facilitative and interactive processing (e.g., semantic-phonological mediation priming) of
both phonological and semantic information prior to naming. This finding has then been used to test
whether phonological or semantic deficits cause word-finding difficulties in children with
Developmental Language Disorder (DLD). Results from later PWI studies (e.g., Seiger-Gardner &
Schwartz, 2008; Brooks et al., 2014) aimed to answer this and concluded that while children with DLD
also exhibited phonologically facilitated lexical access, the priming effect occurred later as they
processed phonological information longer than typically developing controls did. In addition, children
with DLD exhibited persisting semantic interference instead of priming effects (Seiger-Gardner &
Schwartz, 2008), which suggests that semantic deficits (e.g., weak word association) potentially cause
word-finding difficulties (Brooks et al., 2014).
As described earlier, PWI is one experimental paradigm that can offer sensitive measures and
precise response classifications for understanding lexical access mechanisms behind word production.
PWI tasks can also ensure ecological validity if the linguistic stimuli are carefully designed based on
norming data that are applicable to the intended participants and comparison groups. However, the
procedures involved in PWI tasks compromise communicative sense as rapid naming is rarely expected
from children in their daily conversational environments. With respect to practical implications, most
PWI studies have thus far only accounted for lexical access mechanisms in monolingual populations
and, therefore, can only directly inform focused intervention for monolingual clinical populations.
While practitioners who work with multilingual children can certainly infer possible translation of these
PWI findings in their contexts, there is still a need to improve the research evidence base for
multilingual populations.
If PWI paradigms have been used to investigate lexical access mechanisms for word
production, syntactic priming paradigms have been used to investigate how children learn to use
novel or complex grammatical constructions (e.g., double object dative) and whether there are lexical
or syntactic priming mechanisms that facilitate this process (Ambridge & Rowland, 2013). In a syntactic
priming paradigm, children are auditorily primed with a specific construction (e.g., The teacher gave
the child a sticker.) to eventually elicit a similarly structured target utterance (e.g., Ana read Ben a
book.) that has no lexical overlap with the primed utterance (Pickering & Ferreira, 2008). Some studies
(e.g., Akhtar, 1999; Matthews et al., 2007) have also modified this paradigm to test whether typically
developing children can learn novel word order constructions (e.g., weird word order). These syntactic
LANGUAGE DEVELOPMENT RESEARCH 6
priming studies have revealed that whereas 2-year-old children gradually learn to use novel word orders
if they hear low frequency verbs, the controls who hear high frequency verbs as well as the older
children (ages 3:0-4:0) in the comparison groups tend to correct the ungrammatical primes (Akhtar,
1999; Matthews et al., 2007). These findings support Tomasello’s (1992) theory that individual verbs
are linked to children’s initial word order knowledge. Additionally, other studies (e.g., Leonard et al.,
2000; Leonard et al., 2002) reported that syntactic priming helps children with DLD improve their
sentence-level productions (e.g., formulating sentences with auxiliary verbs is and are).
Despite the informative findings that syntactic priming paradigms have generated, the
sensitivity of this paradigm’s cognitive-linguistic measures is not strong enough to confirm if there are
lexically-dependent priming effects that facilitate syntactic productions or if adult-like knowledge of
primed formulation is actually learnt from successful priming in the absence of semantic overlap
(Ambridge & Rowland, 2013). Nonetheless, this paradigm can still be useful if the priming stimuli,
procedures, and targets involved are ecologically valid (i.e., based on norming data). Precision of
findings can also be achieved if appropriate response classifications and comparison groups are
controlled for. Communicative sense of syntactic priming tasks can still be relatively preserved as long
as they resemble real-world language teaching interactions. More importantly, existing evidence of the
facilitative syntactic priming effects in children with DLD presents promising clinical utility for syntax-
focused language interventions (Leonard, 2011). However, extant syntactic priming studies, like most
PWI studies, have only accounted for monolingual populations. While the evidence base under this
paradigm has yet to improve for multilingual populations, practitioners can actively consolidate the
most applicable and available evidence when implementing syntactic priming interventions for
multilingual children.
(e.g., Tagalog: Humihila ang baka ng baboy, English: The cow is pulling a pig) compared to patient
voice (e.g., Tagalog: Hinihila ang baka ng baboy, English: The/A pig is pulling the cow). In clinical
populations, self-paced listening tasks have also been used to produce evidence indicating that English-
speaking children with DLD (ages 5:0-9:0) have difficulty processing both active and passive voice
sentences, which were reportedly attributed to working memory limitations instead of grammatical
deficits (Marinis & Saddy, 2013; Deevy, 2017).
Further interest in language processing along with the growing availability of child-friendly
and fully-automated eye-tracking equipment (Ambridge & Rowland, 2013; Deevy, 2017) have
advanced visual world paradigms (e.g., Trueswell et al., 1999) for child language research. In a visual
world paradigm, children’s ability to use context in interpreting ambiguous sentences is measured by
video-recording their eye movements while being presented scenes along with spoken stimuli (Deevy,
2017; Trueswell et al., 1999). These automated video-recordings generate the raw data to be analysed
on the basis of proportion of fixations to specific elements in the scenes as a function of spoken stimulus
timing (Deevy, 2017; Trueswell et al., 1999). Relevant to clinical populations, this paradigm has been
modified to investigate the role of language ability in sentence context-processing, as evidenced by
looking patterns (Deevy, 2017). For example, an eye-tracking study conducted by Brock et al. (2008)
suggested that adolescents with Autism Spectrum Disorder (ASD), similar to language-matched peers
and non-autistic individuals that were previously studied under similar paradigms, are indeed able to
integrate context-dependent phonological and semantic information during online sentence processing
and that the degree of language impairment appears to interfere with their sensitivity to sentence
context. Hence, Brock et al.’s (2008) findings contradict previous accounts (e.g., Happé & Frith, 2006)
of weak central coherence in individuals with ASD.
Related to Brock et al.’s (2008) investigation, Kelly et al.’s (2013) visual world paradigm study
aimed to investigate the variability of looking patterns in children with language disorders. In this study,
they compared 8- to 14-year-old autistic children with (ALI) and without (ANL) language impairment,
children with DLD, and typically developing children. Pro-saccade, anti-saccade, and search distractor
(i.e., requiring sustained fixation to a target amid the presence of distracting stimuli) tasks were used to
measure reflexive and volitional eye-movement control (Kelly et al., 2013). Their results revealed that
groups with language impairment (i.e., DLD and ALI groups), and not those with ASD only (i.e., ANL
group), have difficulty in volitionally suppressing reflexive eye-movement towards nontargets. This
finding is consistent with Brock et al.’s (2008) conclusion that language impairment is associated with
context sensitivity. Thus, these studies have trailblazed future research directions towards uncovering
potential causal links (e.g., verbal mediation and executive function) between language and oculomotor
control during context processing (Deevy, 2017).
LANGUAGE DEVELOPMENT RESEARCH 8
N400 has been established as an index of semantic processing in children (Deevy, 2017). To
demonstrate the sensitivity of this measure, an EEG study conducted by Rämä et al. (2013) revealed
that typically developing 2-year-old children, or even younger ones with higher word productivity, may
have taxonomically organised words in their mental lexicon. Consistent with this finding, Torkildsen et
al.’s (2008) study provided further evidence that 20-month-old children’s vocabulary size appear to
affect their receptive fast mapping efficiency (i.e., novel world-learning ability) as those with lower
word productivity were observed to take a longer time when processing new words. Relevant to atypical
populations, EEG studies have also revealed a delayed N400 effect that was interpreted to indicate
semantic integration deficits specific to verbal domains in children with DLD (Cummings & Čeponienė,
2010) and ASD (McCleery et al., 2010). These reported implications of a delayed N400 effect have
then been interpreted to support Kail and Leonard’s (1986) storage deficit theory of language
impairment. Therefore, these EEG findings provide converging evidence, albeit still open for further
investigations, that has promising practical and clinical utility for designing focused semantic
interventions to facilitate better vocabulary development in children.
Whereas N400 measures semantic processing, eLAN/P600 biphasic pattern (i.e., eLAN
followed by P600) has been used to measure syntactic processing (Deevy, 2017). As an exemplar,
Hahne et al. (2004) reported that the eLAN/P600 biphasic pattern effect was observed in typically
developing 7- to 13-year-old children and in adult controls when asked to perform grammaticality
judgment tasks (e.g., detecting syntactic violations). However, the eLAN effect was not observed in
their 6-year-old participants, suggesting that automatic syntactic processes may not be fully acquired
until age 7 but gradually develop toward adult-like processing (Hahne et al., 2004). With respect to
atypical populations, the absence of the eLAN effect in children with DLD (ages 10:0-21:0) during
syntactic judgment tasks and the presence of the N400 effect during semantic judgment tasks have been
LANGUAGE DEVELOPMENT RESEARCH 9
interpreted as evidence of a DLD subtype specific to syntactic deficits with preserved semantic
processing (Fonteneau & van der Lely, 2008). In another study done by Epstein et al. (2013), the LAN
index, this time used as a measure of working memory load during sentence processing, was found to
attenuate in school-aged (ages 8:5-12:4) children with DLD when processing object Wh- questions (e.g.,
Who did the bear follow?) relative to subject Wh- questions (e.g., Who followed the tiger?). This
attenuating and less distinctive processing has then led the authors to conclude that children with DLD
might also have less efficient working memory when processing object Wh- questions (Epstein et al.,
2013).
Conclusion
References
Akhtar, N. (1999). Acquiring basic word order: Evidence for data-driven learning of syntactic
structure. Journal of child language, 26(2), 339-356.
Ambridge, B., & Rowland, C. F. (2013). Experimental methods in studying child language
acquisition. Wiley Interdisciplinary Reviews: Cognitive Science, 4(2), 149-168.
Brock, J., Norbury, C., Einav, S., & Nation, K. (2008). Do individuals with autism process words in
context? Evidence from language-mediated eye-movements. Cognition, 108(3), 896-904.
Brooks, P. J., & MacWhinney, B. (2000). Phonological priming in children's picture naming. Journal
of Child Language, 27(2), 335-366.
Brooks, P. J., Seiger-Gardner, L., & Sailor, K. (2014). Contrasting effects of associates and coordinates
in children with and without language impairment: A picture–word interference study. Applied
Psycholinguistics, 35(3), 515-545.
Cummings, A., & Čeponienė, R. (2010). Verbal and nonverbal semantic processing in children with
developmental language impairment. Neuropsychologia, 48(1), 77-85.
Epstein, B., Hestvik, A., Shafer, V. L., & Schwartz, R. G. (2013). ERPs reveal atypical processing of
subject versus object Wh‐questions in children with specific language
impairment. International Journal of Language & Communication Disorders, 48(4), 351-365.
Fonteneau, E., & van der Lely, H. K. (2008). Electrical brain responses in language-impaired children
reveal grammar-specific deficits. PloS one, 3(3).
Garcia, R., Roeser, J., & Höhle, B. (2019). Thematic role assignment in the L1 acquisition of Tagalog:
Use of word order and morphosyntactic markers. Language Acquisition, 26(3), 235-261.
Hahne, A., Eckstein, K., & Friederici, A. D. (2004). Brain signatures of syntactic and semantic
processes during children's language development. Journal of Cognitive Neuroscience, 16(7),
1302-1318.
Happé, F., & Frith, U. (2006). The weak coherence account: detail-focused cognitive style in autism
spectrum disorders. Journal of autism and developmental disorders, 36(1), 5-25.
Jerger, S., Martin, R. C., & Damian, M. F. (2002). Semantic and phonological influences on picture
naming by children and teenagers. Journal of Memory and Language, 47(2), 229-249.
LANGUAGE DEVELOPMENT RESEARCH 11
Jescheniak, J. D., Hahne, A., Hoffmann, S., & Wagner, V. (2006). Phonological activation of category
coordinates during speech planning is observable in children but not in adults: Evidence for
cascaded processing. Journal of Experimental Psychology: Learning, Memory, and
Cognition, 32(2), 373.
Kail, R., & Leonard, L. B. (1986). Word-finding abilities in language-impaired children. ASHA
monographs, (25), 1.
Kelly, D. J., Walker, R., & Norbury, C. F. (2013). Deficits in volitional oculomotor control align with
language status in autism spectrum disorders. Developmental science, 16(1), 56-66.
Leonard, L. B., Dromi, E., Adam, G., & Zadunaisky-Ehrlich, S. (2000). Tense and finiteness in the
speech of children with specific language impairment acquiring Hebrew. International Journal
of Language & Communication Disorders, 35(3), 319-335.
Leonard, L. B., Miller, C. A., Deevy, P., Rauf, L., Gerber, E., & Charest, M. (2002). Production
operations and the use of nonfinite verbs by children with specific language
impairment. Journal of Speech, Language, and Hearing Research.
Marinis, T., & Saddy, D. (2013). Parsing the passive: Comparing children with specific language
impairment to sequential bilingual children. Language Acquisition, 20(2), 155-179.
Marzan, J. C. (2013). Spoken language patterns of selected Filipino toddlers and preschool children
(Unpublished doctoral dissertation). University of the Philippines.
Matthews, D., Lieven, E., Theakston, A., & Tomasello, M. (2007). French children's use and correction
of weird word orders: A constructivist account. Journal of Child Language, 34(2), 381-409.
McCleery, J. P., Ceponienè, R., Burner, K. M., Townsend, J., Kinnear, M., & Schreibman, L. (2010).
Neural correlates of verbal and nonverbal semantic integration in children with autism spectrum
disorders. Journal of Child Psychology and Psychiatry, 51(3), 277-286.
Owen, A. J., & Leonard, L. B. (2002). Lexical diversity in the spontaneous speech of children with
specific language impairment. Journal of Speech, Language, and Hearing Research.
Pickering, M. J., & Ferreira, V. S. (2008). Structural priming: A critical review. Psychological
bulletin, 134(3), 427.
Rämä, P., Sirri, L., & Serres, J. (2013). Development of lexical–semantic language system: N400
priming effect for spoken words in 18-and 24-month old children. Brain and language, 125(1),
1-10.
LANGUAGE DEVELOPMENT RESEARCH 12
Roberts, J., Rescorla, L., Giroux, J., & Stevens, L. (1998). Phonological skills of children with specific
expressive language impairment (SLI-E) outcome at age 3. Journal of Speech, Language, and
Hearing Research, 41(2), 374-384.
Seiger-Gardner, L., & Almodovar, D. (2017). Language production approaches to child language
disorders. In R. G. Schwartz (Ed.), Handbook of child language disorders (pp. 501-528).
Psychology Press.
Seiger‐Gardner, L., & Schwartz, R. G. (2008). Lexical access in children with and without specific
language impairment: a cross‐modal picture–word interference study. International Journal of
Language & Communication Disorders, 43(5), 528-551.
Tomasello, M. (1992). First verbs: A case study of early grammatical development. Cambridge
University Press.
Torkildsen, J. V. K., Svangstu, J. M., Hansen, H. F., Smith, L., Simonsen, H. G., Moen, I., & Lindgren,
M. (2008). Productive vocabulary size predicts event-related potential correlates of fast
mapping in 20-month-olds. Journal of Cognitive Neuroscience, 20(7), 1266-1282.
Trueswell, J. C., Sekerina, I., Hill, N. M., & Logrip, M. L. (1999). The kindergarten-path effect:
Studying on-line sentence processing in young children. Cognition, 73(2), 89-134.