{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T13:01:11Z","timestamp":1772542871617,"version":"3.50.1"},"reference-count":44,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2023,2,20]],"date-time":"2023-02-20T00:00:00Z","timestamp":1676851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/2.zoppoz.workers.dev:443\/http\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Topics in Cognitive Science"],"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>With the rise of artificial intelligence (AI) and the desire to ensure that such machines work well with humans, it is essential for AI systems to actively model their human teammates, a capability referred to as Machine Theory of Mind (MToM). In this paper, we introduce the inner loop of human\u2013machine teaming expressed as communication with MToM capability. We present three different approaches to MToM: (1) constructing models of human inference with well\u2010validated psychological theories and empirical measurements; (2) modeling human as a copy of the AI; and (3) incorporating well\u2010documented domain knowledge about human behavior into the above two approaches. We offer a formal language for machine communication and MToM, where each term has a clear mechanistic interpretation. We exemplify the overarching formalism and the specific approaches in two concrete example scenarios. Related work that demonstrates these approaches is highlighted along the way. The formalism, examples, and empirical support provide a holistic picture of the inner loop of human\u2013machine teaming as a foundational building block of collective human\u2013machine intelligence.<\/jats:p>","DOI":"10.1111\/tops.12642","type":"journal-article","created":{"date-parts":[[2023,2,20]],"date-time":"2023-02-20T17:50:22Z","timestamp":1676915422000},"page":"248-267","update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["The Inner Loop of Collective Human\u2013Machine Intelligence"],"prefix":"10.1111","volume":"17","author":[{"given":"Scott Cheng\u2010Hsin","family":"Yang","sequence":"first","affiliation":[{"name":"Department of Mathematics and Computer Science Rutgers University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tomas","family":"Folke","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science Rutgers University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Patrick","family":"Shafto","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science Rutgers University"},{"name":"School of Mathematics Institute for Advanced Studies"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2023,2,20]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.292.22.2771"},{"key":"e_1_2_8_3_1","unstructured":"Abel D.(2022).A theory of abstraction in reinforcement learning.arXiv preprint arXiv:2203.00397."},{"key":"e_1_2_8_4_1","unstructured":"Bachem O. Lucic M. &Krause A.(2017).Practical coreset constructions for machine learning.arXiv preprint arXiv:1703.06476."},{"key":"e_1_2_8_5_1","doi-asserted-by":"crossref","unstructured":"Bokadia H. Yang S. C.\u2010H. Li Z. Folke T. &Shafto P.(2022).Evaluating perceptual and semantic interpretability of saliency methods: A case study of melanoma.Applied AI Letters 3 e77.","DOI":"10.1002\/ail2.77"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2006.11.004"},{"key":"e_1_2_8_7_1","unstructured":"Carroll M. Shah R. Ho M. K. Griffiths T. Seshia S. Abbeel P. &Dragan A.(2019).On the utility of learning about humans for human\u2013AI coordination.Advances in Neural Information Processing Systems 32 5174\u20135185."},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1002\/wcs.79"},{"key":"e_1_2_8_9_1","unstructured":"Chen Y.\u2010N. Celikyilmaz A. &Hakkani\u2010Tur D.(2018).Deep learning for dialogue systems. InProceedings of the 27th International Conference on Computational Linguistics: Tutorial Abstracts(pp. 25\u201331)."},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.actpsy.2006.09.007"},{"key":"e_1_2_8_11_1","doi-asserted-by":"crossref","unstructured":"Demir M. McNeese N. J. Cooke N. J. Ball J. T. Myers C. &Frieman M.(2015).Synthetic teammate communication and coordination with humans. InProceedings of the Human Factors and Ergonomics Society Annual Meeting(volume 59 pp. 951\u2013955). Los Angeles CA: SAGE Publications.","DOI":"10.1177\/1541931215591275"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2020.3002417"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/1555343415572631"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1218633"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2005.08.041"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v40i2.2850"},{"key":"e_1_2_8_17_1","doi-asserted-by":"crossref","unstructured":"Gurney N. &Pynadath D. V.(2022).Robots with theory of mind for humans: A survey. In2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO\u2010MAN)(pp. 993\u20131000). IEEE.","DOI":"10.1109\/RO-MAN53752.2022.9900662"},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-022-04743-9"},{"key":"e_1_2_8_19_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-control-042920-015547"},{"key":"e_1_2_8_20_1","unstructured":"Ho M. K. Littman M. MacGlashan J. Cushman F. &Austerweil J. L.(2016).Showing versus doing: Teaching by demonstration.Advances in Neural Information Processing Systems 29 3035\u20133043."},{"key":"e_1_2_8_21_1","unstructured":"Huang L. Freeman J. Cooke N. Colonna\u2010Romano J. Wood M. D. Buchanan V. &Caufman S. J.(2022).Exercises for artificial social intelligence in Minecraft search and rescue for teams.OSF doi:10.17605\/osf.io\/jwyvf."},{"key":"e_1_2_8_22_1","doi-asserted-by":"crossref","unstructured":"Iman M. Rasheed K. &Arabnia H. R.(2022).A review of deep transfer learning and recent advancements.arXiv preprint arXiv:2201.09679.","DOI":"10.3390\/technologies11020040"},{"key":"e_1_2_8_23_1","unstructured":"Jong N. K. Hester T. &Stone P.(2008). The utility of temporal abstraction in reinforcement learning. InAAMAS(pp. 299\u2013306). Citeseer."},{"key":"e_1_2_8_24_1","unstructured":"Koh P. W. &Liang P.(2017).Understanding black\u2010box predictions via influence functions. InInternational Conference on Machine Learning(pp. 1885\u20131894). PMLR."},{"key":"e_1_2_8_25_1","first-page":"5","article-title":"Towards a unified theory of state abstraction for MDPS","volume":"4","author":"Li L.","year":"2006","journal-title":"ISAIM"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41551-018-0304-0"},{"key":"e_1_2_8_27_1","volume-title":"Information theory, inference and learning algorithms","author":"MacKay D. J.","year":"2003"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1111\/tops.12553"},{"key":"e_1_2_8_29_1","doi-asserted-by":"crossref","unstructured":"Nowak T. Nowicki M. R. \u0106wian K. &Skrzypczy\u0144ski P.(2019).How to improve object detection in a driver assistance system applying explainable deep learning. In2019 IEEE Intelligent Vehicles Symposium (IV)(pp. 226\u2013231). IEEE.","DOI":"10.1109\/IVS.2019.8814134"},{"key":"e_1_2_8_30_1","unstructured":"Paleja R. Ghuy M. RanawakaArachchige N. Jensen R. &Gombolay M.(2021). The utility of explainable AI in ad hoc human\u2013machine teaming. In Ranzato M. Beygelzimer A. Dauphin Y. Liang P. &Vaughan J. W.(Eds.) Advances in neural information processing systems(volume 34 pp. 610\u2013623). Curran Associates Inc."},{"key":"e_1_2_8_31_1","unstructured":"Rabinowitz N. Perbet F. Song F. Zhang C. Eslami S. A. &Botvinick M.(2018).Machine theory of mind. InInternational Conference on Machine Learning(pp. 4218\u20134227). PMLR."},{"key":"e_1_2_8_32_1","unstructured":"Raileanu R. Denton E. Szlam A. &Fergus R.(2018).Modeling others using oneself in multi\u2010agent reinforcement learning. InInternational Conference on Machine Learning(pp. 4257\u20134266). PMLR."},{"key":"e_1_2_8_33_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.3629243"},{"key":"e_1_2_8_34_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0010-0277(97)00048-6"},{"issue":"1","key":"e_1_2_8_36_1","first-page":"10","article-title":"A developed system for melanoma diagnosis","volume":"3","author":"Smaoui N.","year":"2013","journal-title":"International Journal of Computer Vision and Signal Processing"},{"key":"e_1_2_8_37_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.84.4.327"},{"key":"e_1_2_8_38_1","first-page":"17582","article-title":"A mathematical theory of cooperative communication","volume":"33","author":"Wang P.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_8_39_1","unstructured":"Yang S. C.\u2010H. Folke T. &Shafto P.(2022).A psychological theory of explainability.arXiv preprint arXiv:2205.08452."},{"key":"e_1_2_8_40_1","unstructured":"Yang S. C.\u2010H. &Shafto P.(2017).Explainable artificial intelligence via Bayesian teaching. InNIPS 2017 Workshop on Teaching Machines Robots and Humans(volume 2)."},{"key":"e_1_2_8_41_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-03033-0"},{"key":"e_1_2_8_42_1","unstructured":"Yang S. C.\u2010H. Yu Y. Givchi A. Wang P. Vong W. K. &Shafto P. et\u00a0al. (2018).Optimal cooperative inference. InInternational Conference on Artificial Intelligence and Statistics(pp. 376\u2013385). PMLR."},{"key":"e_1_2_8_43_1","unstructured":"Zheng B. Verma S. Zhou J. Tsang I. &Chen F.(2021).Imitation learning: Progress taxonomies and opportunities.arXiv preprint arXiv:2106.12177."},{"key":"e_1_2_8_44_1","unstructured":"Zhu H. Neubig G. &Bisk Y.(2021).Few\u2010shot language coordination by modeling theory of mind. InInternational Conference on Machine Learning(pp. 12901\u201312911). PMLR."},{"key":"e_1_2_8_45_1","unstructured":"Zhu X. Singla A. Zilles S. &Rafferty A. N.(2018).An overview of machine teaching.arXiv preprint arXiv:1801.05927."}],"container-title":["Topics in Cognitive Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/tops.12642","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/tops.12642","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/tops.12642","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T12:02:49Z","timestamp":1747828969000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/onlinelibrary.wiley.com\/doi\/10.1111\/tops.12642"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,20]]},"references-count":44,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,4]]}},"alternative-id":["10.1111\/tops.12642"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.1111\/tops.12642","archive":["Portico"],"relation":{},"ISSN":["1756-8757","1756-8765"],"issn-type":[{"value":"1756-8757","type":"print"},{"value":"1756-8765","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,20]]},"assertion":[{"value":"2022-06-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}