{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T03:43:39Z","timestamp":1771904619841,"version":"3.50.1"},"reference-count":69,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2021,4,15]],"date-time":"2021-04-15T00:00:00Z","timestamp":1618444800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Conceptual knowledge about objects is essential for humans, as well as for animals, to interact with their environment. On this basis, the objects can be understood as tools, a selection process can be implemented and their usage can be planned in order to achieve a specific goal. The conceptual knowledge, in this case, is primarily concerned about the physical properties and functional properties observed in the objects. Similarly tool-use applications in robotics require such conceptual knowledge about objects for<jats:italic>substitute selection<\/jats:italic>among other purposes. State-of-the-art methods employ a top-down approach where hand-crafted symbolic knowledge, which is defined from a human perspective, is grounded into sensory data afterwards. However, due to different sensing and acting capabilities of robots, a robot's conceptual understanding of objects (e.g., light\/heavy) will vary and therefore should be generated from the robot's perspective entirely, which entails robot-centric conceptual knowledge about objects. A similar bottom-up argument has been put forth in cognitive science that humans and animals alike develop conceptual understanding of objects based on their own perceptual experiences with objects. With this goal in mind, we propose an extensible property estimation framework which consists of estimations methods to obtain the quantitative measurements of physical properties (rigidity, weight, etc.) and functional properties (containment, support, etc.) from household objects. This property estimation forms the basis for our second contribution: Generation of robot-centric conceptual knowledge. Our approach employs unsupervised clustering methods to transform numerical property data into symbols, and Bivariate Joint Frequency Distributions and Sample Proportion to generate conceptual knowledge about objects using the robot-centric symbols. A preliminary implementation of the proposed framework is employed to acquire a dataset comprising six<jats:italic>physical<\/jats:italic>and four<jats:italic>functional<\/jats:italic>properties of 110 household objects. This Robot-Centric dataSet (RoCS) is used to evaluate the framework regarding the property estimation methods and the semantics of the considered properties within the dataset. Furthermore, the dataset includes the derived robot-centric conceptual knowledge using the proposed framework. The application of the conceptual knowledge about objects is then evaluated by examining its usefulness in a tool substitution scenario.<\/jats:p>","DOI":"10.3389\/frobt.2021.476084","type":"journal-article","created":{"date-parts":[[2021,4,15]],"date-time":"2021-04-15T19:00:04Z","timestamp":1618513204000},"update-policy":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["From Multi-Modal Property Dataset to Robot-Centric Conceptual Knowledge About Household Objects"],"prefix":"10.3389","volume":"8","author":[{"given":"Madhura","family":"Thosar","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian A.","family":"Mueller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Georg","family":"J\u00e4ger","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Johannes","family":"Schleiss","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Narender","family":"Pulugu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ravi","family":"Mallikarjun Chennaboina","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sai Vivek","family":"Rao Jeevangekar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andreas","family":"Birk","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Max","family":"Pfingsthorn","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sebastian","family":"Zug","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2021,4,15]]},"reference":[{"key":"B1","first-page":"4923","article-title":"\u201cLearning how a tool affords by simulating 3D models from the web,\u201d","volume-title":"IEEE International Conference on Intelligent Robots and Systems","author":"Abelha","year":"2017"},{"key":"B2","doi-asserted-by":"crossref","DOI":"10.1109\/ICRA.2016.7487400","article-title":"\u201cA model-based approach to finding substitute tools in 3D vision data,\u201d","volume-title":"Proceedings - IEEE International Conference on Robotics and Automation","author":"Abelha","year":"2016"},{"key":"B3","first-page":"6479","article-title":"\u201cUsing structural bootstrapping for object substitution in robotic executions of human-like manipulation tasks,\u201d","volume-title":"IEEE International Conference on Intelligent Robots and Systems","author":"Agostini","year":"2015"},{"key":"B4","article-title":"\u201cChallenges in finding ways to get the job done,\u201d","volume-title":"Planning and Robotics (PlanRob) Workshop at 24th International Conference on Automated Planning and Scheduling","author":"Awaad","year":"2014"},{"key":"B5","doi-asserted-by":"crossref","DOI":"10.1201\/9781420024203","volume-title":"Cognition and Tool Use","author":"Baber","year":"2003"},{"key":"B6","article-title":"TOOLNET: using commonsense generalization for predicting tool use for robot plan synthesis","volume-title":"arXiv preprint","author":"Bansal","year":"2020"},{"key":"B7","doi-asserted-by":"publisher","first-page":"20120408","DOI":"10.1098\/rstb.2012.0408","article-title":"Tool use as adaptation. Philos. Trans. R. Soc. Lond","volume":"368","author":"Biro","year":"2013","journal-title":"B. Biol. Sci"},{"key":"B8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ICRA.2011.5980575","article-title":"\u201cKuka youbot - a mobile manipulator for research and education,\u201d","volume-title":"2011 IEEE International Conference on Robotics and Automation","author":"Bischoff","year":"2011"},{"key":"B9","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1017\/CBO9780511894800.003","article-title":"\u201cChap. 2: Ecology and cognition of tool use in chimpanzees,\u201d","volume-title":"Tool Use in Animals: Cognition and Ecology","author":"Boesch","year":"2013"},{"key":"B10","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1089\/big.2016.0038","article-title":"Leveraging large-scale semantic networks for adaptive robot task learning and execution","volume":"4","author":"Boteanu","year":"2016","journal-title":"Big Data"},{"key":"B11","first-page":"1","article-title":"\u201cA relational approach to tool-use learning in robots,\u201d","volume-title":"Inductive Logic Programming - 22nd International Conference","author":"Brown","year":"2012"},{"key":"B12","first-page":"1","article-title":"\u201cA relational approach to tool-use learning in robots,\u201d","author":"Brown","year":"2013","journal-title":"Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)"},{"key":"B13","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1016\/S0921-8890(03)00021-6","article-title":"An introduction to the anchoring problem","volume":"43","author":"Coradeschi","year":"2003","journal-title":"Robot. Auton. Syst."},{"key":"B14","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3233\/AIS-2009-0040","article-title":"Grounding commonsense knowledge in intelligent systems","volume":"1","author":"Daoutis","year":"2009","journal-title":"J. Ambient Intell. Smart Environ."},{"key":"B15","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1017\/CBO9780511894800.006","article-title":"\u201cChap. 4: Insight, imagination and invention: tool understanding in a non-tool-using corvid,\u201d","volume-title":"Tool Use in Animals: Cognition and Ecology","author":"Emery","year":"2013"},{"key":"B16","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1016\/S0093-934X(03)00355-9","article-title":"Embodied meaning in a neural theory of language","volume":"89","author":"Feldman","year":"2004","journal-title":"Brain Lang."},{"key":"B17","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/7287.001.0001","volume-title":"WordNet: An Electronic Lexical Database","author":"Fellbaum","year":"1998"},{"key":"B18","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"B19","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1080\/02643290442000310","article-title":"The brain's concepts: the role of the sensory-motor system in conceptual knowledge","volume":"22","author":"Gallese","year":"2005","journal-title":"Cogn. Neuropsychol."},{"key":"B20","unstructured":"\u201cCognitive semantics and image schemas with embodied forces,\u201d5776 G\u00e4rdenforsP. KroisJ. M. RosengrenM. StedeleA. WesterkampD. John Benjamins Publishing CompanyEmbodiment in Cognition and Culture2007"},{"key":"B21","doi-asserted-by":"publisher","first-page":"2280","DOI":"10.1016\/j.patcog.2014.01.005","article-title":"Automatic generation and detection of highly reliable fiducial markers under occlusion","volume":"47","author":"Garrido-Jurado","year":"2014","journal-title":"Pattern Recogn."},{"key":"B22","first-page":"89","article-title":"\u201c Chap. 5: Why is tool use rare in animals?,\u201d","volume-title":"Tool Use in Animals: Cognition and Ecology","author":"Gavin","year":"2013"},{"key":"B23","first-page":"127","article-title":"\u201cChap. 8: The theory of affordances,\u201d","volume-title":"The Ecological Approach to Visual Perception","author":"Gibson","year":"1986"},{"key":"B24","first-page":"605","article-title":"\u201cCommon sense data acquisition for indoor mobile robots,\u201d","volume-title":"Proceedings of the Nineteenth National Conference on Artificial Intelligence, Sixteenth Conference on Innovative Applications of Artificial Intelligence","author":"Gupta","year":"2004"},{"key":"B25","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1016\/0167-2789(90)90087-6","article-title":"The symbol grounding problem","volume":"42","author":"Harnad","year":"1990","journal-title":"Physica D"},{"key":"B26","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1080\/01449290310001592587","article-title":"Cognitive, physical, sensory, and functional affordances in interaction design","volume":"22","author":"Hartson","year":"2003","journal-title":"Behav. Inform. Technol."},{"key":"B27","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1016\/j.conb.2009.05.003","article-title":"Functional understanding facilitates learning about tools in human children","volume":"19","author":"Hernik","year":"2009","journal-title":"Curr. Opin. Neurobiol."},{"key":"B28","doi-asserted-by":"publisher","first-page":"9444","DOI":"10.1073\/pnas.96.16.9444","article-title":"\u201cWhat\u201d and \u201chow\u201d: evidence for the dissociation of object knowledge and mechanical problem-solving skills in the human brain","volume":"96","author":"Hodges","year":"1999","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"B29","doi-asserted-by":"publisher","first-page":"1850001","DOI":"10.1142\/S0219843618500019","article-title":"Active tactile transfer learning for object discrimination in an unstructured environment using multimodal robotic skin","volume":"15","author":"Kaboli","year":"2017","journal-title":"Int. J. Humanoid Robot."},{"key":"B30","doi-asserted-by":"crossref","DOI":"10.1109\/WACV.2018.00217","article-title":"\u201cRGBD camera based material recognition via surface roughness estimation,\u201d","volume-title":"Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018","author":"Kim","year":"2018"},{"key":"B31","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-54927-9","volume-title":"Robot Operating System (ros): The Complete Reference","author":"Koub\u00e2a","year":"2017"},{"key":"B32","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1142\/S0219843609001772","article-title":"Erratum: \u201cBirth of the object: detection of objectness and extraction of object shape through object\u2013action complexes\u201d","volume":"6","author":"Kraft","year":"2009","journal-title":"Int. J. Humanoid Robot."},{"key":"B33","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1007\/978-3-540-74788-8_10","article-title":"\u201cAn image-schematic account of spatial categories,\u201d","volume-title":"Spatial Information Theory","author":"Kuhn","year":"2007"},{"key":"B34","doi-asserted-by":"crossref","first-page":"3548","DOI":"10.1109\/IROS.2010.5649547","article-title":"\u201cORO, a knowledge management platform for cognitive architectures in robotics,\u201d","volume-title":"IEEE\/RSJ 2010 International Conference on Intelligent Robots and Systems, IROS 2010 - Conference Proceedings","author":"Lemaignan","year":"2010"},{"key":"B35","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1145\/219717.219745","article-title":"Cyc: a large-scale investment in knowledge infrastructure","volume":"38","author":"Lenat","year":"1995","journal-title":"Commun. ACM"},{"key":"B36","first-page":"547","article-title":"\u201cTeaching robots the use of human tools from demonstration with non-dexterous end-effectors,\u201d","volume-title":"IEEE-RAS International Conference on Humanoid Robots","author":"Li","year":"2015"},{"key":"B37","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1109\/TSMCA.2010.2076404","article-title":"Ontology-based unified robot knowledge for service robots in indoor environments","volume":"41","author":"Lim","year":"2011","journal-title":"IEEE Trans. Syst. Man and Cybern. Part A Syst. Hum."},{"key":"B38","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1023\/B:BTTJ.0000047600.45421.6d","article-title":"ConceptNet \u2013 A practical commonsense reasoning tool-kit","volume":"22","author":"Liu","year":"2004","journal-title":"BT Technol. J."},{"key":"B39","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1109\/TIT.1982.1056489","article-title":"Least squares quantization in pcm","volume":"28","author":"Lloyd","year":"1982","journal-title":"IEEE Trans. Inform. Theor."},{"key":"B40","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1016\/j.cognition.2009.09.002","article-title":"The linguistic and embodied nature of conceptual processing","volume":"114","author":"Louwerse","year":"2010","journal-title":"Cognition"},{"key":"B41","first-page":"164","article-title":"\u201cAffordance estimation for vision-based object replacement on a humanoid robot,\u201d","volume-title":"47th International Symposium on Robotics","author":"Mustafa","year":"2016"},{"key":"B42","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1016\/j.neubiorev.2017.04.014","article-title":"What is an affordance? 40 years later","volume":"77","author":"Osiurak","year":"2017","journal-title":"Neurosci. Biobehav. Rev."},{"key":"B43","first-page":"1024","article-title":"\u201cWordNet :: similarity - measuring the relatedness of concepts measures of relatedness,\u201d","volume-title":"Nineteenth National Conference on Artificial Intelligence (AAAI-04)","author":"Pedersen","year":"2004"},{"key":"B44","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1007\/s11370-017-0216-y","article-title":"A light non-monotonic knowledge-base for service robots","volume":"10","author":"Pineda","year":"2017","journal-title":"Intell. Serv. Robot."},{"key":"B45","first-page":"25","article-title":"\u201cChap. 2: Concept-formation,\u201d","volume-title":"Introduction to Objectivist Epistemology","author":"Rand","year":"1990"},{"key":"B46","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511894800","volume-title":"Tool Use in Animals: Cognition and Ecology","author":"Sanz","year":"2013"},{"key":"B47","article-title":"RoboBrain: large-scale knowledge engine for robots","volume-title":"arXiv preprint","author":"Saxena","year":"2014"},{"key":"B48","article-title":"Tool substitution with shape and material reasoning using dual neural networks","volume-title":"arXiv preprint","author":"Shrivatsav","year":"2019"},{"key":"B49","doi-asserted-by":"crossref","DOI":"10.1109\/ROBOT.2009.5152802","article-title":"Interactive learning of the acoustic properties of household objects","volume-title":"IEEE International Conference on Robotics and Automation","author":"Sinapov","year":"2009"},{"key":"B50","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1109\/TOH.2016.2521378","article-title":"Single-grasp object classification and feature extraction with simple robot hands and tactile sensors","volume":"9","author":"Spiers","year":"2016","journal-title":"IEEE Trans. Haptics"},{"key":"B51","first-page":"1","author":"Stoytchev","year":"2007","journal-title":"Robot tool behavior: a developmental approach to autonomous tool use"},{"key":"B52","doi-asserted-by":"crossref","DOI":"10.1109\/HUMANOIDS.2014.7041448","article-title":"\u201cAdaptive tool-use strategies for anthropomorphic service robots,\u201d","volume-title":"14th IEEE-RAS International Conference on Humanoid Robots (Humanoids)","author":"St\u00fcckler","year":"2014"},{"key":"B53","first-page":"429","article-title":"\u201cOntology-based multi-layered robot knowledge framework (OMRKF) for robot intelligence,\u201d","volume-title":"IEEE International Conference on Intelligent Robots and Systems","author":"Suh","year":"2007"},{"key":"B54","doi-asserted-by":"publisher","first-page":"6","DOI":"10.31269\/triplec.v3i2.19","article-title":"\u201cOn the subject of objects: four views on object perception and tool use,\u201d","volume":"3","author":"Susi","year":"2005","journal-title":"Cogn. Commun. Cooperat."},{"key":"B55","first-page":"16","article-title":"A comprehensive characterization of the asus xtion pro depth sensor. Jr","volume":"1","author":"Swoboda","year":"2014","journal-title":"J. Sci. Technol."},{"key":"B56","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-11179-7_46","article-title":"\u201cTool \u2013 body assimilation model based on body babbling and neuro-dynamical system,\u201d","volume-title":"International Conference on Artificial Neural Networks","author":"Takahashi","year":"2014"},{"key":"B57","doi-asserted-by":"crossref","DOI":"10.1109\/DEVLRN.2007.4354057","article-title":"\u201cHaptic discrimination of material properties by a robotic hand,\u201d","volume-title":"2007 IEEE 6th International Conference on Development and Learning, ICDL","author":"Takamuku","year":"2007"},{"key":"B58","first-page":"4261","article-title":"\u201cKNOWROB- Knowledge processing for autonomous personal robots,\u201d","volume-title":"IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Tenorth","year":"2009"},{"key":"B59","article-title":"\u201cWhat stands-in for a missing tool?: A prototypical grounded knowledge-based approach to tool substitution,\u201d","volume-title":"11th International Workshop on Cognitive Robotics in 16th International Conference on Principles of Knowledge Representation and Reasoning","author":"Thosar","year":""},{"key":"B60","doi-asserted-by":"crossref","DOI":"10.1145\/3341105.3374017","article-title":"\u201cSubstitute selection for a missing tool using robot-centric conceptual knowledge of objects,\u201d","volume-title":"Knowledge Representation and Reasoning Track in 35th ACM\/SIGAPP Symposium On Applied Computing","author":"Thosar","year":"2020"},{"key":"B61","article-title":"\u201cA review of knowledge bases for service robots in household environments,\u201d","volume-title":"6th International Workshop on Artificial Intelligence and Cognition","author":"Thosar","year":""},{"key":"B62","first-page":"130","article-title":"\u201cExploring affordances and tool use on the iCub,\u201d","volume-title":"IEEE-RAS International Conference on Humanoid Robots","author":"Tikhanoff","year":"2015"},{"key":"B63","first-page":"6231","article-title":"\u201cDifferentiable physics and stable modes for tool-use and manipulation planning - Extended abstract,\u201d","volume-title":"IJCAI International Joint Conference on Artificial Intelligence","author":"Toussaint","year":"2019"},{"key":"B64","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1017\/S0140525X11001452","article-title":"The cognitive bases of human tool use","volume":"35","author":"Vaesen","year":"2012","journal-title":"Behav. Brain Sci."},{"key":"B65","doi-asserted-by":"publisher","first-page":"121","DOI":"10.4000\/tc.556","article-title":"Object manipulation, tool use, and the social context in human and non-human primates","author":"Vauclair","year":"1994","journal-title":"Techniq. Cult."},{"key":"B66","doi-asserted-by":"publisher","first-page":"1230","DOI":"10.11591\/ijece.v8i2.pp1230-1237","article-title":"Tool use learning for a real robot","volume":"8","author":"Wicaksono","year":"2018","journal-title":"Int. J. Electr. Comput. Eng."},{"key":"B67","article-title":"\u201cLearning physical object properties from unlabeled videos,\u201d","volume-title":"British Machine Vision Conference","author":"Wu","year":"2016"},{"key":"B68","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2019.XV.001","article-title":"Improvisation through physical understanding: using novel objects as tools with visual foresight","volume-title":"arXiv preprint","author":"Xie","year":"2019"},{"key":"B69","first-page":"408","article-title":"\u201cReasoning about object affordance in a knowledge based representation,\u201d","volume-title":"European Conference on Computer Vision","author":"Zhu","year":"2014"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.frontiersin.org\/articles\/10.3389\/frobt.2021.476084\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,24]],"date-time":"2022-12-24T11:52:12Z","timestamp":1671882732000},"score":1,"resource":{"primary":{"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/www.frontiersin.org\/articles\/10.3389\/frobt.2021.476084\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,15]]},"references-count":69,"alternative-id":["10.3389\/frobt.2021.476084"],"URL":"https:\/\/2.zoppoz.workers.dev:443\/https\/doi.org\/10.3389\/frobt.2021.476084","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,15]]},"article-number":"476084"}}