{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2022,4,2]],"date-time":"2022-04-02T05:39:40Z","timestamp":1648877980511},"reference-count":25,"publisher":"MIT Press - Journals","issue":"2","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2010,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Audiovisual speech perception provides an opportunity to investigate the mechanisms underlying multimodal processing. By using nonspeech stimuli, it is possible to investigate the degree to which audiovisual processing is specific to the speech domain. It has been shown in a match-to-sample design that matching across modalities is more difficult in the nonspeech domain as compared to the speech domain. We constructed a biophysically realistic neural network model simulating this experimental evidence. We propose that a stronger connection between modalities in speech underlies the behavioral difference between the speech and the nonspeech domain. This could be the result of more extensive experience with speech stimuli. Because the match-to-sample paradigm does not allow us to draw conclusions concerning the integration of auditory and visual information, we also simulated two further conditions based on the same paradigm, which tested the integration of auditory and visual information within a single stimulus. New experimental data for these two conditions support the simulation results and suggest that audiovisual integration of discordant stimuli is stronger in speech than in nonspeech stimuli. According to the simulations, the connection strength between auditory and visual information, on the one hand, determines how well auditory information can be assigned to visual information, and on the other hand, it influences the magnitude of multimodal integration.<\/jats:p>","DOI":"10.1162\/jocn.2009.21202","type":"journal-article","created":{"date-parts":[[2009,3,20]],"date-time":"2009-03-20T18:20:03Z","timestamp":1237573203000},"page":"240-247","update-policy":"https:\/\/2.zoppoz.workers.dev:443\/http\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":2,"title":["Audiovisual Matching in Speech and Nonspeech Sounds: A Neurodynamical Model"],"prefix":"10.1162","volume":"22","author":[{"given":"Marco","family":"Loh","sequence":"first","affiliation":[{"name":"1Universitat Pompeu Fabra, Barcelona, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gabriele","family":"Schmid","sequence":"additional","affiliation":[{"name":"2City Hospital Bogenhausen, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gustavo","family":"Deco","sequence":"additional","affiliation":[{"name":"1Universitat Pompeu Fabra, Barcelona, Spain"},{"name":"3Instituci\u00f3 Catalana de Recerca i Estudis Avan\u00e7ats (ICREA), Barcelona, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wolfram","family":"Ziegler","sequence":"additional","affiliation":[{"name":"2City Hospital Bogenhausen, Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2010,2,1]]},"reference":[{"key":"2021072900370391800_R1","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/S0028-3932(99)00074-3","article-title":"The functional anatomy of visual\u2013tactile integration in man: A study using positron emission tomography.","volume":"38","author":"Banati","year":"2000","journal-title":"Neuropsychologia"},{"key":"2021072900370391800_R2","doi-asserted-by":"crossref","first-page":"1190","DOI":"10.1038\/nn1333","article-title":"Unraveling multisensory integration: Patchy organization within human STS multisensory cortex.","volume":"7","author":"Beauchamp","year":"2004","journal-title":"Nature Neuroscience"},{"key":"2021072900370391800_R3","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1093\/cercor\/10.5.512","article-title":"Human temporal lobe activation by speech and nonspeech sounds.","volume":"10","author":"Binder","year":"2000","journal-title":"Cerebral Cortex"},{"key":"2021072900370391800_R4","doi-asserted-by":"crossref","first-page":"1196","DOI":"10.1093\/cercor\/bhg100","article-title":"Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex.","volume":"13","author":"Brody","year":"2003","journal-title":"Cerebral Cortex"},{"key":"2021072900370391800_R5","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1023\/A:1011204814320","article-title":"Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition.","volume":"11","author":"Brunel","year":"2001","journal-title":"Journal of Computational Neuroscience"},{"key":"2021072900370391800_R6","doi-asserted-by":"crossref","first-page":"7679","DOI":"10.1523\/JNEUROSCI.5640-07.2008","article-title":"The neuronal basis of attention: Rate versus synchronization modulation.","volume":"28","author":"Buehlman","year":"2008","journal-title":"Journal of Neuroscience"},{"key":"2021072900370391800_R7","doi-asserted-by":"crossref","first-page":"2213","DOI":"10.1097\/00001756-200312020-00016","article-title":"Neural processes underlying perceptual enhancement by visual speech gestures.","volume":"14","author":"Callan","year":"2003","journal-title":"NeuroReport"},{"key":"2021072900370391800_R8","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1162\/089892903321107828","article-title":"Reading speech from still and moving faces: The neural substrates of visible speech.","volume":"15","author":"Calvert","year":"2003","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2021072900370391800_R9","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1016\/S0960-9822(00)00513-3","article-title":"Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex.","volume":"10","author":"Calvert","year":"2000","journal-title":"Current Biology"},{"key":"2021072900370391800_R10","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/978-3-662-13015-5_8","article-title":"Seeing brains reading speech: A review and speculations.","volume-title":"Speechreading by humans and machines","author":"Campbell","year":"1996"},{"key":"2021072900370391800_R11","doi-asserted-by":"crossref","first-page":"20073","DOI":"10.1073\/pnas.0709794104","article-title":"Perceptual detection as a dynamical bistability phenomenon: A neurocomputational correlate of sensation.","volume":"104","author":"Deco","year":"2007","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2021072900370391800_R12","doi-asserted-by":"crossref","first-page":"2374","DOI":"10.1046\/j.1460-9568.2003.02956.x","article-title":"Attention and working memory: A dynamical model of neuronal activity in the prefrontal cortex.","volume":"18","author":"Deco","year":"2003","journal-title":"European Journal of Neuroscience"},{"key":"2021072900370391800_R13","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1111\/j.1460-9568.2006.04940.x","article-title":"Decision-making and Weber's law: A neurophysiological model.","volume":"24","author":"Deco","year":"2006","journal-title":"European Journal of Neuroscience"},{"key":"2021072900370391800_R14","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1044\/jshr.1202.423","article-title":"Interaction of audition and vision in the recognition of oral speech stimuli.","volume":"12","author":"Erber","year":"1969","journal-title":"Journal of Speech and Hearing Research"},{"key":"2021072900370391800_R15","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1176\/ajp.153.3.321","article-title":"What are the functional consequences of neurocognitive deficits in schizophrenia?","volume":"153","author":"Green","year":"1996","journal-title":"American Journal of Psychiatry"},{"key":"2021072900370391800_R16","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1523\/JNEUROSCI.18-03-01072.1998","article-title":"Cross-modal transfer of information between the tactile and the visual representations in the human brain: A positron emission tomographic study.","volume":"18","author":"Hadjikhani","year":"1998","journal-title":"Journal of Neuroscience"},{"key":"2021072900370391800_R17","doi-asserted-by":"crossref","first-page":"1376","DOI":"10.1016\/j.neuroimage.2005.10.042","article-title":"Gamma-band activity over early sensory areas predicts detection of changes in audiovisual speech stimuli.","volume":"30","author":"Kaiser","year":"2006","journal-title":"Neuroimage"},{"key":"2021072900370391800_R18","doi-asserted-by":"crossref","first-page":"2927","DOI":"10.1111\/j.1460-9568.2005.04505.x","article-title":"Cognitive flexibility and decision making in a model of conditional visuomotor associations.","volume":"22","author":"Loh","year":"2005","journal-title":"European Journal of Neuroscience"},{"key":"2021072900370391800_R19","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1126\/science.1104171","article-title":"Flexible control of mutual inhibition: A neural model of two-interval discrimination.","volume":"307","author":"Machens","year":"2005","journal-title":"Science"},{"key":"2021072900370391800_R20","doi-asserted-by":"crossref","first-page":"746","DOI":"10.1038\/264746a0","article-title":"Hearing lips and seeing voices.","volume":"264","author":"McGurk","year":"1976","journal-title":"Nature"},{"key":"2021072900370391800_R21","doi-asserted-by":"crossref","first-page":"546","DOI":"10.1016\/j.neuropsychologia.2005.07.002","article-title":"Audio-visual matching of speech and non-speech oral gestures in patients with aphasia and apraxia of speech.","volume":"44","author":"Schmid","year":"2006","journal-title":"Neuropsychologia"},{"key":"2021072900370391800_R22","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/j.1749-6632.2001.tb05712.x","article-title":"Consciousness and the binding problem.","volume":"929","author":"Singer","year":"2001","journal-title":"Annals of the New York Academy of Sciences"},{"key":"2021072900370391800_R23","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1121\/1.1907309","article-title":"Use of visual information for phonetic perception.","volume":"26","author":"Sumby","year":"1954","journal-title":"Journal of the Acoustical Society of America"},{"key":"2021072900370391800_R24","first-page":"3","article-title":"Some preliminaries to a comprehensive account of audio-visual speech perception.","volume-title":"Hearing by eye: The psychology of lipreading","author":"Summerfield","year":"1987"},{"key":"2021072900370391800_R25","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1159\/000259969","article-title":"Use of visual information for phonetic perception.","volume":"36","author":"Summerfield","year":"1979","journal-title":"Phonetica"}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/http\/direct.mit.edu\/jocn\/article-pdf\/22\/2\/240\/1938507\/jocn.2009.21202.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/2.zoppoz.workers.dev:443\/http\/direct.mit.edu\/jocn\/article-pdf\/22\/2\/240\/1938507\/jocn.2009.21202.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,29]],"date-time":"2021-07-29T00:38:40Z","timestamp":1627519120000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/direct.mit.edu\/jocn\/article\/22\/2\/240\/4794\/Audiovisual-Matching-in-Speech-and-Nonspeech"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,2,1]]},"references-count":25,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2010,2,1]]},"published-print":{"date-parts":[[2010,2,1]]}},"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1162\/jocn.2009.21202","relation":{},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2010,2]]},"published":{"date-parts":[[2010,2,1]]}}}