Brain-Inspired Cognitive Architectures for

Artificial Intelligence: BICA*AI 2020:

Proceedings of the 11th Annual Meeting of the
BICA Society Alexei V. Samsonovich pdf download

https://textbookfull.com/product/brain-inspired-cognitive-architectures-for-artificial-intelligence-
bicaai-2020-proceedings-of-the-11th-annual-meeting-of-the-bica-society-alexei-v-samsonovich/

★★★★★ 4.6/5.0 (38 reviews) ✓ 149 downloads ■ TOP RATED

"Excellent quality PDF, exactly what I needed!" - Sarah M.

DOWNLOAD EBOOK
 Brain-Inspired Cognitive Architectures for Artificial
Intelligence: BICA*AI 2020: Proceedings of the 11th Annual
Meeting of the BICA Society Alexei V. Samsonovich pdf
download

TEXTBOOK EBOOK TEXTBOOK FULL

Available Formats

■ PDF eBook Study Guide TextBook

EXCLUSIVE 2025 EDUCATIONAL COLLECTION - LIMITED TIME

INSTANT DOWNLOAD VIEW LIBRARY

Collection Highlights

Biologically Inspired Cognitive Architectures 2019:

Proceedings of the Tenth Annual Meeting of the BICA
Society Alexei V. Samsonovich

TMS 2014 143rd Annual Meeting Exhibition Annual Meeting

Supplemental Proceedings 1st Edition Metals & Materials
Society (Tms) The Minerals

Quantitative Psychology 83rd Annual Meeting of the

Psychometric Society New York NY 2018 Marie Wiberg

TMS 2019 148th Annual Meeting Exhibition Supplemental

Proceedings & The Minerals
Proceedings of International Conference on Artificial
Intelligence and Applications ICAIA 2020 Poonam Bansal

Artificial Intelligence XXXVII 40th SGAI International

Conference on Artificial Intelligence AI 2020 Cambridge UK
December 15 17 2020 Proceedings 1st Edition Max Bramer

Advances in Artificial Intelligence, Software and Systems

Engineering: Proceedings of the AHFE 2020 Virtual
Conferences on Software and Systems Engineering, and
Artificial Intelligence and Social Computing, July 16-20,
2020, USA Tareq Ahram

Advances in Artificial Intelligence 18th Conference of the

Spanish Association for Artificial Intelligence CAEPIA
2018 Granada Spain October 23 26 2018 Proceedings
Francisco Herrera

Special Topics in Artificial Intelligence and Augmented

Reality: The Case of Spatial Intelligence Enhancement
(Cognitive Technologies) 1st Edition Christos Papakostas
Advances in Intelligent Systems and Computing 1310

Alexei V. Samsonovich
Ricardo R. Gudwin
Alexandre da Silva Simões Editors

Brain-Inspired
Cognitive
Architectures
for Artificial
Intelligence: BICA*AI
2020
Proceedings of the 11th Annual
Meeting of the BICA Society
Advances in Intelligent Systems and Computing

Volume 1310

Series Editor
Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences,
Warsaw, Poland

Advisory Editors
Nikhil R. Pal, Indian Statistical Institute, Kolkata, India
Rafael Bello Perez, Faculty of Mathematics, Physics and Computing,
Universidad Central de Las Villas, Santa Clara, Cuba
Emilio S. Corchado, University of Salamanca, Salamanca, Spain
Hani Hagras, School of Computer Science and Electronic Engineering,
University of Essex, Colchester, UK
László T. Kóczy, Department of Automation, Széchenyi István University,
Gyor, Hungary
Vladik Kreinovich, Department of Computer Science, University of Texas
at El Paso, El Paso, TX, USA
Chin-Teng Lin, Department of Electrical Engineering, National Chiao
Tung University, Hsinchu, Taiwan
Jie Lu, Faculty of Engineering and Information Technology,
University of Technology Sydney, Sydney, NSW, Australia
Patricia Melin, Graduate Program of Computer Science, Tijuana Institute
of Technology, Tijuana, Mexico
Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro,
Rio de Janeiro, Brazil
Ngoc Thanh Nguyen , Faculty of Computer Science and Management,
Wrocław University of Technology, Wrocław, Poland
Jun Wang, Department of Mechanical and Automation Engineering,
The Chinese University of Hong Kong, Shatin, Hong Kong
The series “Advances in Intelligent Systems and Computing” contains publications
on theory, applications, and design methods of Intelligent Systems and Intelligent
Computing. Virtually all disciplines such as engineering, natural sciences, computer
and information science, ICT, economics, business, e-commerce, environment,
healthcare, life science are covered. The list of topics spans all the areas of modern
intelligent systems and computing such as: computational intelligence, soft comput-
ing including neural networks, fuzzy systems, evolutionary computing and the fusion
of these paradigms, social intelligence, ambient intelligence, computational neuro-
science, artificial life, virtual worlds and society, cognitive science and systems,
Perception and Vision, DNA and immune based systems, self-organizing and
adaptive systems, e-Learning and teaching, human-centered and human-centric
computing, recommender systems, intelligent control, robotics and mechatronics
including human-machine teaming, knowledge-based paradigms, learning para-
digms, machine ethics, intelligent data analysis, knowledge management, intelligent
agents, intelligent decision making and support, intelligent network security, trust
management, interactive entertainment, Web intelligence and multimedia.
The publications within “Advances in Intelligent Systems and Computing” are
primarily proceedings of important conferences, symposia and congresses. They
cover significant recent developments in the field, both of a foundational and
applicable character. An important characteristic feature of the series is the short
publication time and world-wide distribution. This permits a rapid and broad
dissemination of research results.
Indexed by SCOPUS, DBLP, EI Compendex, INSPEC, WTI Frankfurt eG,
zbMATH, Japanese Science and Technology Agency (JST), SCImago.
All books published in the series are submitted for consideration in Web of
Science.

More information about this series at http://www.springer.com/series/11156

Alexei V. Samsonovich Ricardo R. Gudwin
• •

Alexandre da Silva Simões

Editors

Brain-Inspired Cognitive
Architectures for Artificial
Intelligence: BICA*AI 2020
Proceedings of the 11th Annual Meeting
of the BICA Society

123
Editors
Alexei V. Samsonovich Ricardo R. Gudwin
Cybernetics Department University of Campinas
National Research Nuclear University Campinas, Brazil
Moscow, Russia

Alexandre da Silva Simões

Institute of Science and Technology
São Paulo State University (Unesp)
Sorocaba, Brazil

ISSN 2194-5357 ISSN 2194-5365 (electronic)

Advances in Intelligent Systems and Computing
ISBN 978-3-030-65595-2 ISBN 978-3-030-65596-9 (eBook)
https://doi.org/10.1007/978-3-030-65596-9
© The Editor(s) (if applicable) and The Author(s), under exclusive license
to Springer Nature Switzerland AG 2021
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard
to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface

This volume documents the proceedings of the 2020 Annual International

Conference on Brain-Inspired Cognitive Architectures for Artificial Intelligence,
known as BICA*AI 2020, also known as the Eleventh Annual Meeting of the
BICA Society. BICA*AI 2020 was officially held in Natal, Rio Grande do Norte,
Brazil, as an event collocated with Robotica 2020. At the time of its organization, it
was impossible to guess that both, BICA*AI 2020 and Robotica 2020, will be
hosted online only, as entirely virtual events, due to the COVID-19 pandemic.
Nevertheless, this year BICA conference can be called a notable success—thanks to
the help of our technical team, primarily Arthur Chubarov, who under my super-
vision developed the Virtual Convention Center (VCC), the platform for hosting
virtual conferences, including this one (please see our paper in this volume). As a
result, VCC in conjunction with Zoom and Mozilla Hubs was used for hosting all
events of BICA*AI 2020. The project was funded by the Russian Science
Foundation Grant #18-11-00336. Speaking of official sponsors of BICA*AI 2020,
they are the BICA Society (bicasociety.org/about), AGI Laboratory, National
Research Nuclear University MEPhI, and two universities in Brazil: University of
Campinas (Unicamp) and Sao Paulo State University (UNESP).
Brain-Inspired Cognitive Architectures (BICA) are computational frameworks
for building intelligent agents that are inspired from natural intelligence. Since
2010, the annual BICA conference attracts researchers from the edge of scientific
frontiers around the world. It complements major conferences in artificial intelli-
gence (AI), cognitive science and neuroscience by offering to ambitious researchers
an informal brainstorming atmosphere and freedom of spontaneous social interac-
tions, together with great publication venues. This distinguishing feature of our
conference extends fully into the difficult year 2020.
BICA community members understand “biological inspirations” broadly, bor-
rowing them from psychology, neuroscience, linguistics, narratology, and creativity
studies in order to advance cognitive robotics and machine learning, among other
hot topics in AI. The selection criterion is based on the question of how much a
particular contribution may help us make machines our friends or understand how
the mind works. This principle corresponds to a broader understanding of the BICA

v
vi Preface

Challenge, put forward by the founders of BICA Society in 2010. BICA Society
(Biologically Inspired Cognitive Architectures Society) is a scientific non-profit 501
(c)(3) organization based in the USA, whose mission is to promote and facilitate the
many scientific efforts around the world in solving the BICA Challenge, i.e., cre-
ating a real-life computational equivalent of the human mind. BICA Society brings
together researchers from disjointed fields and communities who devote their efforts
to solving the challenge, despite that they may “speak different languages.” This is
achieved by promoting and facilitating the transdisciplinary study of cognitive
architectures, and in the long-term perspective, creating one unifying widespread
framework for the human-level cognitive architectures and their implementations.
Over 11 years of its existence, BICA Society organized or co-organized 11
international conferences (the BICA conferences), 2 schools, and 2 symposia.
These events were hosted in various countries around the world, including the
USA, Italy, France, Ukraine, Russia, the Czech Republic, and Brazil. In organizing
these events, BICA Society collaborated with several universities around the world.
BICA Society created and is maintaining an online database on cognitive archi-
tectures (http://bicasociety.org/mapped/), which includes one of the world’s largest
databases of freely accessible video records of scientific presentations about cog-
nitive architectures and related topics:
– https://vimeo.com/bicasociety
– https://youtube.com/channel/UC7Smq21YMKs0UjuVkFPpJJg
BICA Society initiated several mainstream periodic publication venues,
including Elsevier journals “Biologically Inspired Cognitive Architectures” and
“Cognitive Systems Research (CSR)–Special Issue Series on BICA*AI”. Plus,
BICA Society regularly publishes its proceedings in other editions dedicated to
BICA*AI, including Procedia Computer Science and, of course, the Springer Book
Series. The following topics at the front edge of science and technology, as before,
are in the focus of this and future BICA*AI conferences, books, and journals:
– Artificial social–emotional intelligence;
– Active humanlike learning and cognitive growth;
– Narrative intelligence and context understanding;
– Artificial creativity and creation of art by AI;
– Goal reasoning and true autonomy in artifacts;
– Embodied intelligence and cognitive robotics;
– Synthetic characters, HCI, and VR/MR paradigms for AI;
– Language capabilities and social competence;
– Robust and scalable machine learning mechanisms;
– Socially acceptable intelligent virtual agents;
– The role of emotions in artificial intelligence and their BICA models;
– Tests and metrics for BICA in the context of the BICA Challenge;
– Interaction between natural and artificial cognitive systems;
– Theory-of-Mind, episodic, and autobiographical memory in vivo and in vitro;
– Introspection, metacognitive reasoning, and self-awareness in BICA;
Preface vii

– AI and ethics, digital economics, and cybersecurity;

– Unifying frameworks, standards, and constraints for cognitive architectures.
Works of many, yet not all, distinguished speakers of BICA*AI 2020 are
included in this volume. Among the speakers of BICA*AI 2020 are top-level
scientists like John Laird, Paul Verschure, Terry Stewart, Antonio Chella, Paul
Robertson, Jan Treur, Rosario Sorbello, Ricardo Gudwin, to name only a few of the
famous names. The majority of authors of this volume is less known and never-
theless deserves attention. In particular, I would like to mention the work of
Natividad Vargas, Juan Luis del Valle-Padilla, Juan P. Jimenez and Félix Ramos, of
Lennart Zegerius, of Joey van den Heuvel, of Alessio Plebe and Pietro Perconti, of
Peter Boltuc and Thomas P. Connelly, of Kyrtin Atreides, David Kelley and Uplift
Masi.
As the General Chair of BICA*AI 2020, I would like to thank all members of the
Organizing Committee, the Program Committee, the Technical Support Team and
the Publishers teams for their great job in making the BICA conference a great
success story one more time. Particularly, my special thanks go to the publisher
Leontina Di Cecco from Springer, who made this publication possible.

November 2020 Alexei V. Samsonovich

Organization

Program Committee

Taisuke Akimoto Kyushu Institute of Technology, Japan

Kenji Araki Hokkaido University, Japan
Joscha Bach AI Foundation, USA
Feras Batarseh George Mason University, USA
Paul Baxter Plymouth University, USA
Paul Benjamin Pace University, New York, USA
Galina A. Beskhlebnova Scientific Research Institute for System
Analysis RAS, Russia
Jordi Bieger Reykjavik University, Iceland
Perrin Bignoli Yahoo Labs, USA
Douglas Blank Bryn Mawr College, USA
Peter Boltuc University of Illinois at Springfield, USA
Jonathan Bona University of Arkansas for Medical Sciences,
USA
Michael Brady Boston University, USA
Mikhail Burtsev Moscow Institute of Physics and Technology,
Russia
Erik Cambria Nanyang Technological University, Singapore
Suhas Chelian Fujitsu Laboratories of America, Inc., USA
Antonio Chella Dipartimento di Ingegneria Informatica,
Università di Palermo, Italy
Olga Chernavskaya P. N. Lebedev Physical Institute, Moscow, Russia
Thomas Collins University of Southern California (Information
Sciences Institute), USA
Christopher Dancy Penn State University, USA
Haris Dindo University of Palermo, Italy

ix
x Organization

Sergey A. Dolenko D. V. Skobeltsyn Institute of Nuclear Physics,

M. V. Lomonosov Moscow State University,
Russia
Alexandr Eidlin Sberbank, Moscow, Russia
Jim Eilbert AP Technology, USA
Thomas Eskridge Florida Institute of Technology, USA
Usef Faghihi Professor At Universite de Quebce in Trois-rivier,
Canada
Elena Fedorovskaya Rochester Institute of Technology, USA
Stan Franklin University of Memphis, USA
Marcello Frixione University of Genova, Italy
Salvatore Gaglio University of Palermo, Italy
Olivier Georgeon Claude Bernard Lyon 1 University, France
John Gero University of North Carolina at Charlotte, USA
Jaime Gomez Universidad Politécnica de Madrid, Spain
Ricardo R. Gudwin University of Campinas (Unicamp), Brazil
Eva Hudlicka Psychometrix Associates, USA
Dusan Husek Institute of Computer Science, Academy
of Sciences of the Czech Republic
Christian Huyck Middlesex University, UK
Ignazio Infantino Consiglio Nazionale delle Ricerche, Italy
Eduardo Izquierdo Indiana University, USA
Alex James Kunming University of Science and Technology,
China
Li Jinhai Kunming University of Science and Technology,
China
Magnus Johnsson Lund University, Sweden
Darsana Josyula Bowie State University, USA
Kamilla Jóhannsdóttir Reykjavik University, Iceland
Omid Kavehei The University of Sydney, Australia
David Kelley Artificial General Intelligence Inc, Seattle, USA
Troy Kelley U. S. Army Research Laboratory, USA
William Kennedy George Mason University, USA
Deepak Khosla HRL Laboratories LLC, USA
Swathi Kiran Boston University, USA
Muneo Kitajima Nagaoka University of Technology, Japan
Unmesh Kurup LG Electronics, USA
Giuseppe La Tona University of Palermo, Italy
Luis Lamb Federal University of Rio Grande do Sul, Brazil
Leonardo Lana de Carvalho Federal University of Jequitinhonha and Mucuri
Valleys, Brazil
Othalia Larue University of Quebec, Canada
Christian Lebiere Carnegie Mellon University, USA
Jürgen Leitner Australian Centre of Excellence for Robotic
Vision, Australia
Organization xi

Simon Levy Washington and Lee University, USA

Antonio Lieto University of Turin, Italy
James Marshall Sarah Lawrence College, USA
Olga Mishulina National Research Nuclear University MEPhI,
Russia
Sergey Misyurin National Research Nuclear University MEPhI,
Moscow, Russia
Steve Morphet Enabling Tech Foundation, USA
Amitabha Mukerjee Indian Institute of Technology Kanpur, India
Daniele Nardi Sapienza University of Rome, Italy
Sergei Nirenburg Rensselaer Polytechnic Institute, New York, USA
David Noelle University of California Merced, USA
Andrea Omicini Alma Mater Studiorum–Università di Bologna,
Italy
Marek Otahal Czech Institute of Informatics, Robotics and
Cybernetics, Czech Republic
Aleksandr I. Panov Moscow Institute of Physics and Technology,
Russia
David Peebles University of Huddersfield, UK
Giovanni Pilato ICAR-CNR, Italy
Roberto Pirrone University of Palermo, Italy
Michal Ptaszynski Kitami Institute of Technology, Japan
Uma Ramamurthy Baylor College of Medicine, Houston, USA
Thomas Recchia US Army ARDEC, USA
Vladimir Redko Scientific Research Institute for System
Analysis RAS, Russia
James Reggia University of Maryland, USA
Frank Ritter The Pennsylvania State University, USA
Paul Robertson DOLL Inc., USA
Brandon Rohrer Sandia National Laboratories, USA
Christopher Rouff Johns Hopkins Applied Physics Laboratory, USA
Rafal Rzepka Hokkaido University, Japan
Ilias Sakellariou Department of Applied Informatics, University
of Macedonia, Greece
Fredrik Sandin Lulea University of Technology, Sweden
Ricardo Sanz Universidad Politecnica de Madrid, Spain
Michael Schader Yellow House Associates, USA
Howard Schneider Sheppard Clinic North, Canada
Michael Schoelles Rensselaer Polytechnic Institute, USA
Valeria Seidita Dipartimento di Ingegneria - Università degli
Studi di Palermo
Ignacio Serrano Instituto de Automtica Industrial - CSIC, Spain
Javier Snaider FedEx Institute of Technology, The University
of Memphis, USA
Donald Sofge Naval Research Laboratory, USA
xii Organization

Meehae Song Simon Fraser University, Canada

Rosario Sorbello University of Palermo, Italy
John Sowa Kyndi, Inc., USA
Terry Stewart University of Waterloo, Canada
Sherin Sugathan Enview Research & Development Labs, India
Junichi Takeno Meiji University, Japan
Knud Thomsen Paul Scherrer Institute, Switzerland
Daria Tikhomirova NRNU MEPhI
Jan Treur Vrije Universiteit Amsterdam, Netherlands
Vadim L. Ushakov NRNU MEPhI, Russia
Alexsander V. Vartanov Lomonosov Moscow State University, Russia
Rodrigo Ventura Universidade de Lisboa, Portugal
Evgenii Vityaev Sobolev Institute of Mathematics SB RAS,
Russia
Pei Wang Temple University, USA
Mark Waser Digital Wisdom Institute, USA
Roseli S. Wedemann Universidade do Estado do Rio de Janeiro, Brazil
Özge Nilay Yalçin Simon Fraser University, Canada
Terry Zimmerman University of Washington, Bothell, USA
Contents

Acoustic Pattern Recognition Technology Based on the Viola-Jones

Approach for VR and AR Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Alexander M. Alyushin and Sergey V. Dvoryankin
Correlation of a Face Vibroimage Informative Parameters
with Characteristics of a Person’s Functional State When Using
VR and AR Technical Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Victor M. Alyushin
Methodologies and Milestones for the Development
of an Ethical Seed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Kyrtin Atreides, David J. Kelley, and Uplift Masi
Design of a Transcranial Magnetic Stimulation System
with the Implementation of Nanostructured Composites . . . . . . . . . . . . 24
Gennady Baryshev, Yuri Bozhko, Igor Yudin, Aleksandr Tsyganov,
and Anna Kainova
Application of Information Measuring Systems for Development
of Engineering Skills for Cyber-Physical Education . . . . . . . . . . . . . . . . 32
Gennady Baryshev, Valentin Klimov, Aleksandr Berestov,
Anton Tokarev, and Valeria Petrenko
Principles of Design of a Learning Management System
for Development of Economic Skills for Nuclear
Engineering Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Gennady Baryshev, Aleksandr Putilov, Dmitriy Smirnov,
Aleksandr Tsyganov, and Vladimir Chervyakov
Post-quantum Group Key Agreement Scheme . . . . . . . . . . . . . . . . . . . . 49
Julia Bobrysheva and Sergey Zapechnikov
Uncanny Robots of Perfection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
Piotr (Peter) Boltuc and Thomas P. Connelly

xiii
xiv Contents

Self and Other Modelling in Cooperative Resource Gathering

with Multi-agent Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . 69
Vasilii Davydov, Timofei Liusko, and Aleksandr I. Panov
Eye Movement Correlates of Foreign Language Proficiency
in Russian Learners of English . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Valeriia Demareva, Sofia Polevaia, and Julia Edeleva
Development of a Laboratory Workshops Management
Module as Part of a Learning Support System
for the ‘‘Decision-Making Theory’’ Course . . . . . . . . . . . . . . . . . . . . . . 84
Anastasia Devyatkina, Natalia Myklyuchenko, Anna Tikhomirova,
and Elena Matrosova
Algorithm for Constructing Logical Neural Networks Based
on Logical Various-Valued Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Dmitriy Dimitrichenko
The Electroencephalogram Based Classification of Internally
Pronounced Phonemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
Yuliya Gavrilenko, Daniel Saada, Eugene Ilyushin,
Alexander V. Vartanov, and Andrey Shevchenko
“Loyalty Program” Tool Application in Megaprojects . . . . . . . . . . . . . . 106
Anna Guseva, Elena Matrosova, Anna Tikhomirova,
and Nikolay Matrosov
Using Domain Knowledge for Feature Selection in Neural Network
Solution of the Inverse Problem of Magnetotelluric Sounding . . . . . . . . 115
Igor Isaev, Eugeny Obornev, Ivan Obornev, Eugeny Rodionov,
Mikhail Shimelevich, Vladimir Shirokiy, and Sergey Dolenko
Friction Model Identification for Dynamic Modeling
of Pneumatic Cylinder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
Vladimir I. Ivlev, Sergey Yu. Misyurin, and Andrey P. Nelyubin
Neurophysiological Features of Neutral and Threatening Visual
Stimuli Perception in Patients with Schizophrenia . . . . . . . . . . . . . . . . . 138
Sergey I. Kartashov, Vyacheslav A. Orlov,
Aleksandra V. Maslennikova, and Vadim L. Ushakov
Logical Circuits of a RP-Neuron Model . . . . . . . . . . . . . . . . . . . . . . . . . 143
Mukhamed Kazakov
Study of Neurocognitive Mechanisms in the Concealed
Information Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
Yuri I. Kholodny, Sergey I. Kartashov, Denis G. Malakhov,
and Vyacheslav A. Orlov
Contents xv

IT-Solutions in Money Laundering/Counter Terrorism Financing

Risk Assessment in Commercial Banks . . . . . . . . . . . . . . . . . . . . . . . . . 156
Sofya Klimova and Asmik Grigoryan
Expandable Digital Functional State Model of Operator
for Intelligent Human Factor Reliability Management Systems . . . . . . . 165
Lyubov V. Kolobashkina, Mikhail V. Alyushin, and Kirill S. Nikishov
Specialized Software Tool for Pattern Recognition
of Biological Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Sergey D. Kulik and Evgeny O. Levin
Designing Software for Risk Assessment Using a Neural Network . . . . . 181
Anna V. Lebedeva and Anna I. Guseva
Suitability of Object-Role Modeling Diagrams as an Intermediate
Model for Ontology Engineering: Testing the Rules for Mapping . . . . . 188
Dmitrii Litovkin, Dmitrii Dontsov, Anton Anikin, and Oleg Sychev
Cyber Threats to Information Security in the Digital Economy . . . . . . . 195
K. S. Luzgina, G. I. Popova, and I. V. Manakhova
Applying a Logical Derivative to Identify Hidden Patterns
in the Data Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Larisa A. Lyutikova
Algorithm for Constructing Logical Operations to Identify Patterns
in Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Larisa A. Lyutikova and Elena V. Shmatova
Graph-Ontology Model of Cognitive-Similar Information Retrieval
(on the Requirements Tracing Task Example) . . . . . . . . . . . . . . . . . . . . 218
Nikolay V. Maksimov, Olga L. Golitsina, Kirill V. Monankov,
and Natalia A. Bal
Toward a Building an Ontology of Artefact . . . . . . . . . . . . . . . . . . . . . . 225
Nikolay Maksimov and Alexander Lebedev
Cognitive Architectures of Effective Speech-Language
Communication and Prospective Challenges for Neurophysiological
Speech Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Irina Malanchuk
Development of an AI Recommender System to Recommend
Concerts Based on Microservice Architecture Using Collaborative
and Content-Based Filtering Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Andrey Malynov and Igor Prokhorov
Visualization of T. Saati Hierarchy Analysis Method . . . . . . . . . . . . . . . 253
Elena Matrosova, Anna Tikhomirova, Nikolay Matrosov,
and Kovtun Dmitriy
xvi Contents

Labor Productivity Growth Based on Revolutionary Technologies

as a Factor for Overcoming the Economic Crisis . . . . . . . . . . . . . . . . . . 265
Y. M. Medvedeva and R. E. Abdulov
Network Security Intelligence Centres for Information Security
Incident Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Natalia Miloslavskaya and Steven Furnell
Block Formation for Storing Data on Information Security
Incidents for Digital Investigations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
Natalia Miloslavskaya, Andrey Nikiforov, and Kirill Plaksiy
Cyber Polygon Site Project in the Framework of the MEPhI
Network Security Intelligence Center . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Natalia Miloslavskaya and Alexander Tolstoy
Selection of a Friction Model to Take into Account the Impact
on the Dynamics and Positioning Accuracy of Drive Systems . . . . . . . . 309
S. Yu. Misyurin, G. V. Kreinin, N. Yu. Nosova, and A. P. Nelyubin
Kinematics and Dynamics of the Spider-Robot Mechanism,
Motion Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
Sergey Yu. Misyurin, German V. Kreinin, Natalia Yu. Nosova,
and Andrey P. Nelyubin
Multiagent Model of Perceptual Space Formation in the Process
of Mastering Linguistic Competence . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
Zalimkhan Nagoev and Irina Gurtueva
The Role of Gender in the Prosocial Behavior Mechanisms . . . . . . . . . . 335
Yulia M. Neroznikova and Alexander V. Vartanov
Reflection Mechanisms of Empathy Processes in Evoked Potentials . . . . 342
Yulia M. Neroznikova and Alexander V. Vartanov
Lateralization in Neurosemantics: Are Some Lexical Clusters More
Equal Than Others? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
Zakhar Nosovets, Boris M. Velichkovsky, Liudmila Zaidelman,
Vyacheslav Orlov, Sergey Kartashov, Artemiy Kotov, Vadim Ushakov,
and Vera Zabotkina
Brain Inspiration Is Not Panacea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
Pietro Perconti and Alessio Plebe
Linear Systems Theoretic Approach to Interpretation of Spatial
and Temporal Weights in Compact CNNs: Monte-Carlo Study . . . . . . . 365
Artur Petrosyan, Mikhail Lebedev, and Alexey Ossadtchi
The Economic Cross of the Digital Post-coronavirus Economy
(on the Example of Rare Earth Metals Industry) . . . . . . . . . . . . . . . . . . 371
O. Victoria Pimenova, Olga B. Repkina, and Dmitriy V. Timokhin
Contents xvii

Comparative Analysis of Methods for Calculating the Interactions

Between the Human Brain Regions Based on Resting-State FMRI
Data to Build Long-Term Cognitive Architectures . . . . . . . . . . . . . . . . . 380
Alexey Poyda, Maksim Sharaev, Vyacheslav Orlov, Stanislav Kozlov,
Irina Enyagina, and Vadim Ushakov
The Use of the Economic Cross Method in IT Modeling
of Industrial Development (Using the Example
of Two-Component Nuclear Energy) . . . . . . . . . . . . . . . . . . . . . . . . . . . 391
Aleksandr V. Putilov, Dmitriy V. Timokhin, and Marina V. Bugaenko
Intelligence - Consider This and Respond! . . . . . . . . . . . . . . . . . . . . . . . 400
Saty Raghavachary
Simple Model of Origin of Feeling of Causality . . . . . . . . . . . . . . . . . . . 410
Vladimir G. Red’ko
Extending the Intelligence of the Pioneer 2AT Mobile Robot . . . . . . . . . 417
Michael A. Rudy, Eugene V. Chepin, and Alexander A. Gridnev
On the Regularity of the Bias of Throughput Estimates
on Traffic Averaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
Victor A. Rusakov
Virtual Convention Center: A Socially Emotional Online/VR
Conference Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
Alexei V. Samsonovich and Arthur A. Chubarov
Ensembling SNNs with STDP Learning on Base of Rate
Stabilization for Image Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
Alexander Sboev, Alexey Serenko, Roman Rybka, and Danila Vlasov
Mathematical Methods for Solving Cognitive Problems
in Medical Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Yuri Kotov and Tatiana Semenova
Brain Cognitive Architectures Mapping for Neurosurgery:
Resting-State fMRI and Intraoperative Validation . . . . . . . . . . . . . . . . . 466
M. Sharaev, T. Melnikova-Pitskhelauri, A. Smirnov, A. Bozhenko,
V. Yarkin, A. Bernshtein, E. Burnaev, P. Petrov, D. Pitskhelauri,
V. Orlov, and I. Pronin
Machine Learning Based on the Principle of Minimizing Robust
Mean Estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472
Z. M. Shibzukhov
Parameterized Families of Correctly Functioning
Sigma-Pi Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478
Z. M. Shibzukhov
xviii Contents

The Loop of Nonverbal Communication Between Human

and Virtual Actor: Mapping Between Spaces . . . . . . . . . . . . . . . . . . . . . 484
Vladimir R. Shirokiy, Daria V. Tikhomirova, Roman D. Vladimirov,
Sergei A. Dolenko, and Alexei V. Samsonovich
Preliminary Experiment on Emotion Detection in Illustrations
Using Convolutional Neural Network . . . . . . . . . . . . . . . . . . . . . . . . . . . 490
Alexander Shtanko and Sergey Kulik
AGI Components for Enterprise Management Systems . . . . . . . . . . . . . 495
Artem A. Sukhobokov and Liubov I. Lavrinova
Eligibility of English Hypernymy Resources for Extracting
Knowledge from Natural-Language Texts . . . . . . . . . . . . . . . . . . . . . . . 501
Oleg Sychev and Yaroslav Kamennov
The Use of Digital Tools in the Formation of Two-Component
Nuclear Energy on the Base of Economic Cross Method . . . . . . . . . . . . 508
D. V. Timokhin
Complex Objects Identification and Analysis Mechanisms . . . . . . . . . . . 517
Mikhail Ulizko, Larisa Pronicheva, Alexey Artamonov,
Rufina Tukumbetova, and Evheniy Tretyakov
To Help or Not to Help: A Network Modelling Approach
to the Bystander Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
Joey van den Heuvel and Jan Treur
A Model of Top-Down Attentional Control for Visual Search Based
on Neurosciences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541
Natividad Vargas, Juan Luis del Valle-Padilla, Juan P. Jimenez,
and Félix Ramos
Comparison of Brain Induced Potentials in Internal Speech
in Studied and Unknown Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . 547
Alexander V. Vartanov and Alisa R. Suyuncheva
Analysis of Using of Neural Networks for Real-Time
Process Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553
V. S. Volodin and A. O. Tolokonskij
Comparison Between Coordinated Control and Interpretation
Methods for Multi-channel Control of a Mobile Robotic Device . . . . . . 558
Timofei I. Voznenko, Alexander A. Gridnev, Eugene V. Chepin,
and Konstantin Y. Kudryavtsev
Combinator-as-a-Process for Representing the Information
Structure of Deep Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565
Viacheslav Wolfengagen, Larisa Ismailova, and Sergey Kosikov
Contents xix

Applicative Model to Bring-in Conceptual Envelope

for Computational Thinking with Information Processes . . . . . . . . . . . . 572
Viacheslav Wolfengagen, Larisa Ismailova, and Sergey Kosikov
Computational Model for Capturing the Interdependencies
on Information Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578
Viacheslav Wolfengagen, Larisa Ismailova, and Sergey Kosikov
Imposing and Superposing the Information Processes over
Variable Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
Viacheslav Wolfengagen, Larisa Ismailova, and Sergey Kosikov
Improvement of the Technology of fMRI Experiments
in the Concealed Information Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . 591
Yuri I. Kholodny, Sergey I. Kartashov, Denis G. Malakhov,
and Vyacheslav A. Orlov
Modelling Metaplasticity and Memory Reconsolidation During
an Eye-Movement Desensitization and Reprocessing Treatment . . . . . . 598
Lennart Zegerius and Jan Treur

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611

 Acoustic Pattern Recognition Technology
Based on the Viola-Jones Approach
for VR and AR Systems

Alexander M. Alyushin1,2(&) and Sergey V. Dvoryankin3

1
National Research Nuclear University MEPhI (Moscow Engineering Physics
Institute), Kashirskoe shosse 31, Moscow 115409, Russia
[email protected]
2
Plekhanov Russian University of Economics,
st. Hook, 43, Moscow 117997, Russia
3
Financial University under the Government of the Russian Federation,
Leningradsky prospect, 49, Moscow 125993, Russian Federation
[email protected]

Abstract. The ability to solve problems of graphic images recognition in vir-

tual reality (VR), augmented reality (AR), mixed reality (MR) and cross reality
(XR) systems is highlighted as one of the most important. The urgency of
solving problems of recognition and classification of acoustic images has been
substantiated, which will bring the quality of VR, AR, MR and XR systems
closer to real reality (RR). Independent solution of graphic and acoustic patterns
recognition problems using heterogeneous algorithmic and software tools is
attributed to the disadvantages of modern systems. The study proposes an
approach that allows the use of unified methodological and software tools for the
simultaneous solution of graphic and acoustic patterns recognition problems.
The proposed approach is based on converting acoustic information into graphic
information using 2D-images of dynamic sonograms. This allows the recogni-
tion of acoustic patterns using unified algorithmic and software tools. It is
proposed to use the Viola-Jones technology as such a unified tool. It is shown
that the implementation of a two-stage determination of similarity measures of
primitives and areas of the original image makes it possible to increase the speed
of algorithms. For this purpose, at the first iteration, it is proposed to use not the
graphic primitives themselves, but their coordinate projections. In the study, by
analogy with Haar’s features, parametrizable acoustic primitives were devel-
oped, presented in the classical graphical version, as well as in the form of
coordinate projections.

Keywords: Recognition and classification
Graphic and acoustic patterns

Viola-Jones algorithm

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021

A. V. Samsonovich et al. (Eds.): BICA 2020, AISC 1310, pp. 1–8, 2021.
https://doi.org/10.1007/978-3-030-65596-9_1
2 A. M. Alyushin and S. V. Dvoryankin

1 Introduction

One of the most important functions implemented in modern virtual reality (VR),
augmented reality (AR), mixed reality (MR) and cross reality (XR) systems and largely
determines their capabilities, is the recognition and classification of graphic images [1].
For example, the basis of AR, MR and XR technologies is the recognition of objects in
each frame of the video stream and the addition of new graphic information to them. In
addition, a number of AR, MR and XR technologies use the so-called graphic markers,
which are necessary to determine the spatial characteristics of objects of real reality
(RR) [2]. This function must be repeatedly performed in real time [3], which imposes
strict requirements on the speed of algorithms used for this purpose, for example,
various modifications of the well-known Viola-Jones algorithm [4, 5].
One of the main trends in the modern development of AR, MR, and XR systems is
the approach to RR, primarily due to the development of artificial intelligence tech-
nology, which allows simultaneous processing of video and audio patterns [6]. This
allows you to implement a natural user interface for a person, to carry out acoustic
navigation [7–9], to provide the necessary information interaction between characters,
for example, in training systems [10, 11], to increase the efficiency of solving pro-
duction tasks [12, 13].
The aim of the research is to unify algorithmic and technical means used for the
recognition of graphic and acoustic patterns based on the Viola-Jones approach for VR,
AR, MR and XR systems.

2 State of Research in This Area

Currently being developed VR, AR, MR and XR systems, as a rule, involve the use of
independent channels for processing video and audio information. It should be noted
that the applied algorithmic and methodological means of graphic patterns recognition
are forced to operate with significant data streams, which is associated with the use of
modern high-speed high-resolution video cameras. In this regard, the flow of acoustic
data even when using multichannel systems [7] has a significantly smaller volume.
For this reason, it is relevant to use algorithmic and methodological tools developed
for the recognition of graphic patterns for the recognition of acoustic patterns in VR,
AR, MR and XR systems. The feasibility of this approach in practice is due to a fairly
well-developed technology for converting acoustic information into a graphic repre-
sentation in the form of a 2D-image of dynamic sonograms. This technology is widely
used, for example, for the protection of documents against forgery based on the so-
called speech signature [14, 15]. The specificity of a 2D-image of a dynamic sonogram
is the presence of areas with different graphic structures, for example, linear and dotted,
as well as low image contrast with a high noise level.
Of all the existing variety of approaches and algorithms for recognizing graphic
objects in the image, which are also suitable for working with images of dynamic
sonograms, the most suitable is the approach proposed by Viola-Jones for recognizing
facial images [16]. Recognition of a graphic object in accordance with this approach is
carried out on the basis of the similarity measures analysis of a large set of
 Acoustic Pattern Recognition Technology 3

characteristic features typical of the analyzed image. At the same time, the features
themselves, known as Haar features [16–18], characterize the properties of limited
areas of the recognized object.
The disadvantages of this approach include a fairly large amount of calculations to
determine the measures of features similarity and areas of the analyzed image, as well
as a decrease in the reliability of the result obtained with a decrease in the contrast of
the analyzed image.

3 The Essence of the Proposed Approach

To recognize acoustic patterns presented in the form of 2D-images of dynamic sono-

grams, the study proposes to use a two-stage analysis of similarity measures for a set of
acoustic features and areas of a 2D-image. This allows you to significantly reduce the
necessary computational costs of the approach. This approach assumes the presence of
two forms of features representation and analyzed image areas − in the form of image
fragments and in the form of their coordinate projections. A two-stage analysis of
features similarity measures is applicable both in the analysis of acoustic signals and in
the analysis of video stream frames.
At the first step, it is proposed to analyze the features similarity of the coordinate
projections and the coordinate projections of the areas of the analyzed image, usually
selected using a floating rectangular window.
At the second step, the analysis of the features similarity measures selected in this
way and the corresponding image areas, presented as fragments of the corresponding
images, is carried out. This operation is completely analogous to the procedures used in
the Viola-Jones algorithm.
Figure 1 illustrates this approach.
0

F, Hz 4 1 2 3
PY

5
7

PX
Y

0 X
0

P*Y

9
8

10

P*X
Y

0 T, s 0 X

Fig. 1. Approach illustration.

4 A. M. Alyushin and S. V. Dvoryankin

The original image (IMG[i, j], i = 1, …, IMAX, j = 1, …, JMAX, where IMAX and
JMAX are the image size in pixels, respectively) of dynamic sonogram 1 contains areas
2 with a linear structure corresponding to vowel sounds of human speech (harmonic
signals), and area 3 corresponding to consonants (hissing) sounds. The X-axis of the
sonogram corresponds to time, and the Y-axis corresponds to the frequency F.
At the first step, in accordance with the proposed approach, by means of a floating
window 4 with a size of N  N pixels, a fragment of the image 5 is selected, for
example, containing characteristic stripes 6 and 7.
For a given fragment of the image, its coordinate projections PY ðYÞ and PX ðXÞ are
determined:

YX
0 þN

PX ðXÞ ¼ IMGðX; jÞ; X ¼ 1; . . .; N; ð1Þ

j¼Y0

XX
0 þN

PY ðYÞ ¼ IMGði; YÞ; Y ¼ 1; . . .; N; ð2Þ

i¼X0

where X0 and Y0 are the coordinates of the window anchor point.

These projections, as a rule, contain noise components that cause, for example,
nonzero values of the functions PY ðYÞ and PX ðXÞ in the intervals between bands 6 and
7. To minimize the influence of noise components in two forms of information pre-
sentation, it is proposed to carry out, respectively, threshold discrimination for the
functions PY ðYÞ and PX ðXÞ and contrasting for a fragment of the image 5. A typical
result of these operations is shown in Fig. 1 in the form of new obtained coordinate
projections PY ðYÞ and PX ðXÞ, as well as a fragment of a contrast image 8, containing
characteristic stripes 9 and 10.
Analysis of possible structures of image fragments of dynamic sonograms made it
possible to form a basic set of characteristic acoustic features. To describe the image
areas of a sonogram with a line structure, basic features SL(A, D, W) were formed,
where A is the relative angle of inclination of the lines (−AMAX  A  AMAX, the
value AMAX = 10, which corresponds to the angle of inclination of the lines in 90°),
D is the relative distance between the lines (1  D  DMAX, the value DMAX = 10
corresponds to the maximum distance), W is the relative width of the lines (1  W 
WMAX, the value WMAX = 10 corresponds to the maximum width).
In Fig. 2 shows examples of the formed basic acoustic features, which include two
forms of presenting information of the considered type − in the form of coordinate
projections and a fragment of a high-contrast image.
To describe image areas with a pixel structure, basic features of the NP(Q, G) type
were formed, where Q is the relative density of dark pixels (1  Q  QMAX, the
value QMAX = 10 corresponds to the maximum density of dark pixels), G − relative
density gradient with respect to the Y axis (−GMAX  G  GMAX, the value
GMAX = 10 corresponds to the maximum value of the gradient).
In Fig. 3 shows typical examples of formed basic features of the NP(Q, G) type.
 Acoustic Pattern Recognition Technology 5

SL(0, 10, 1) SL(0, 1, 10) SL(5, 5, 5) SL(-5, 5, 5)

Y Y Y Y

0 X 0 X 0 X 0 X

Fig. 2. Examples of SL(A, D, W) basic acoustic features.

NP(10, 0) NP(1, 0) NP(10, 10) NP(1, -10)

Y Y Y Y

0 X 0 X 0 X 0

Fig. 3. Examples of NP(Q, G) basic acoustic features.

The basic acoustic features of SL(A, D, W) and NP(Q, G) types formed in this way
were the basis for the creation of working features SL*(A, D, W, M) and NP*(Q, G, M),
differing in the scaling factor M (1  M  MMAX, for sonograms with a resolution
of less than 1024  1024 pixels, it is sufficient to use the value MMAX = 10).
In Fig. 4 shows an example of the created operating characteristics SL(A, D, W).

SL*(0, 10, 10)

Y SL*(0, 5, 7, 7)
Y SL*(-5, 5, 5, 5)
Y

0 X 0 X 0 X

Fig. 4. Examples of working acoustic signs of SL*(A, D, W, M) type.

Working acoustic signs allow you to get a graphic image of acoustic signals, speech
and sounds based on recognition of the structure of the sonogram. To determine the
similarity measure of coordinate projections at the first step, it is proposed to use the
value D:

X
N   X N  
D ¼ 1=ð1 þ PF ðYÞ  P ðYÞ  þ PF ðXÞ  P ðXÞ Þ; ð3Þ
Y X
Y¼1 X¼1

where PF ðYÞ and PF ðXÞ are, respectively, coordinate projections for working acoustic
features.
6 A. M. Alyushin and S. V. Dvoryankin

SL* SL*
NP* NP* NP* SL* NP* SL* NP*
(-3,5, (8,6,
(3,0,9) (2,0, (5,0,9) (-4,7, (4,0,3) (-3,6, (4,0,3)
4,5 ) 1,5 )
3) 2,5 ) 4,5 )
NP* NP* NP*
SL* (1,0, SL*
NP* NP* (3,0,3) (3,0,3)
(-3,5, 3) (7,6,
(4,0,9) (4,0,9)
3,5 ) 1,5 ) SL* SL*
NP* (-4,7, NP* (-3,6,
(1,0, NP*
SL* 1,5 ) (3,0,3) 3,5 ) (3,0,3)
3) SL*
NP* (-3,5, NP*
(6,6,
(5,0,9) 5,5 ) (4,0,9)
SL* 2,5 ) NP* SL*
(-3,4, SL* (2,0,3) (6,6,
SL*
2,3) (-4,7, 1,5 )
SL* SL* (-3,6,
NP* SL* NP* 5,5 ) 5,5 ) SL*
(-3,5, (5,6, NP*
(5,0,9) (-3,4, (5,0,9) (5,6,
5,5 ) 5,5 ) (3,0,3)
4,3) 3,5 )

Fig. 5. Example of structure recognition.

In Fig. 5 shows an example of structure recognition according to the proposed

technique of 2D-image of a dynamic sonogram shown in Fig. 1 (X-axis time scale not
saved).
The resulting structure of the sonogram is an image that is further processed by the
methodological and algorithmic means inherent in the Viola-Jones approach. This
makes it possible to use the existing graphic image processing tools for solving
problems of recognition and classification of acoustic patterns in VR, AR, MR and XR
systems.

4 Experimental Laboratory Approbation of the Approach

The conducted experimental laboratory testing of the approach confirmed the possi-
bility of its implementation in practice using the classical Viola-Jones algorithms. The
decrease in the performance of the tasks being solved for the recognition of graphic
objects with the simultaneous processing of acoustic patterns did not exceed 10%. The
implementation of a two-stage process for determining the measures of similarity of
features and areas of an image made it possible to increase the performance by 15–30%
for images with dimensions of 800  800 pixels–1600  1600 pixels, respectively.

5 Areas of Possible Application of the Developed Technology

The proposed approach is primarily focused on expanding the functionality of modern

VR, AR, MR and XR systems through the simultaneous processing of video and
acoustic information. Another area of possible application of the approach is systems
for protecting important documents based on the use of a speech signature [14, 15],
which involve solving the problems of searching for a sonogram on a document image,
as well as recognizing the structure of a sonogram in order to identify the author of the
document and his psycho-emotional state.
 Acoustic Pattern Recognition Technology 7

6 Conclusion

The approach proposed in the study makes it possible to use already available software
and methodological tools for solving problems of recognizing and classifying acoustic
patterns, initially focused on recognizing and classifying graphic images. The most
prominent representative of such tools are algorithmic and software tools that imple-
ment the principles of processing graphic data in accordance with Viola-Jones tech-
nology. The implementation of a two-stage procedure for determining the similarity
measure of features and image regions allows increasing the speed of the computational
process.

Acknowledgement. The research was carried out by grant of the Russian Scientific Foundation
(project №19-71-30008) in Plekhanov Russian University of Economics.

References
1. Park, H., Jeong, S., Kim, T., Youn, D., Kim, K.: Visual representation of gesture interaction
feedback in virtual reality games. In: IEEE Proceedings of the 2017 International
Symposium on Ubiquitous Virtual Reality (ISUVR), Nara, Japan, 27–29 June 2017,
pp. 20–23 (2017)
2. Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based
augmented reality conferencing system. In: Proceedings 2nd IEEE and ACM International
Workshop on Augmented Reality (IWAR 1999), San Francisco, CA, USA, 20–21 October
1999, pp. 85–94 (1999)
3. Prince, S., Cheok, A.D., Farbiz, F., Williamson, T., Johnson, N., Billinghurst, M., Kato, H.:
3D live: real time captured content for mixed reality. In: IEEE Proceedings of the
International Symposium on Mixed and Augmented Reality (ISMAR 2002), Darmstadt,
Germany, 1 October 2002, pp. 7–317 (2002)
4. Lee, Y.J.: Effective interface design using face detection for augmented reality interaction of
smart phone. Int. J. Smart Home 6(2), 25–32 (2012)
5. Lee, Y.J., Lee, G.H.: Augmented reality game interface using effective face detection
algorithm. Int. J. Smart Home 5(4), 77–88 (2011)
6. Godin, K.W., Rohrer, R., Snyder, J., Raghuvanshi, N.: Wave acoustics in a mixed reality
shell. In: Proceedings of the 2018 AES International Conference on Audio for Virtual and
Augmented Reality (AVAR), Redmond, USA, 20–22 August 2018, pp. 7−3 (2018)
7. Tylka, J.G., Choueiri, E.Y.: Fundamentals of a parametric method for virtual navigation
within an array of ambisonics microphones. J. Audio Eng. Soc. 68(3), 120–137 (2020a)
8. Tylka, J.G.: Virtual navigation of ambisonics-encoded sound fields containing near-field
sources. In: Computer Science (2019). 246 p.
9. Tylka, J.G., Choueiri, E.Y.: Performance of linear extrapolation methods for virtual sound
field navigation. J. Audio Eng. Soc. 68(3), 138–156 (2020b)
10. Yuen, S.C.-Y., Yaoyuneyong, G., Johnson, E.: Augmented reality: an overview and five
directions for AR in education. J. Educ. Technol. Dev. Exchange 4(1), 119−140 (2011)
11. Abdoli Sejzi, A.: Augmented reality and virtual learning environment. J. Appl. Sci. Res.
(JASR) 11(8), 1–5 (2015)
8 A. M. Alyushin and S. V. Dvoryankin

12. Lahti, H., Bahne, A.: Virtual tuning – a mixed approach based on measured RTFs. In:
Proceedings of the AES 2019 International Conference on Automotive Audio, Bavaria,
Germany, 11–13 September 2019, paper number 12 (2019)
13. Malbos, F., Bogdanski, M., Strauss, M.J.: Virtual reality experience for the optimization of a
car audio system. In: Proceedings of the AES 2019 International Conference on Automotive
Audio, Bavaria, Germany, 11–13 September 2019, paper number 13 (2019)
14. Alyushin, M.V., Alyushin, A.M., Kolobashkina, L.V.: Human face thermal images library
for laboratory studies of the algorithms efficiency for bioinformation processing. In: IEEE
Proceedings of the 11th IEEE International Conference on Application of Information and
Communication Technologies (AICT 2017), Russia, Moscow, 20–22 September 2017
(2017)
15. Alyushin, A.M.: Document protection technology in the digital economics using cognitive
biometric methods. Procedia Comput. Sci. 169, 887–891 (2020)
16. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–
154 (2004)
17. Alyushin, M.V., Alyushin, V.M., Kolobashkina, L.V.: Optimization of the data represen-
tation integrated form in the Viola-Jones algorithm for a person’s face search. Procedia
Comput. Sci. 123, 18–23 (2018)
18. Kolobashkina, L.V., Alyushin, M.V.: Analysis of the possibility of the neural network
implementation of the Viola-Jones algorithm. In: Advances in Intelligent Systems and
Computing, vol. 948, pp. 232−239 (2020)
 Correlation of a Face Vibroimage Informative
Parameters with Characteristics of a Person’s
Functional State When Using VR and AR
Technical Means

Victor M. Alyushin(&)

National Research Nuclear University MEPhI (Moscow Engineering Physics

Institute), Kashirskoe Shosse 31, Moscow 115409, Russia
[email protected]

Abstract. An analysis of the specifics of modern VR, AR, MR and XR systems

use in educational, recreational and recovery processes is presented. The
urgency of continuous monitoring of the current functional state (FS) and its
psycho-emotional state (PES) using VR, AR, MR and XR technical means has
been substantiated. Maintaining the state of health, as well as the required level
of performance, are highlighted as the main requirements for the applied VR,
AR, MR and XR means. The relevance of the use of infrared (IR) biometric
technologies for the registration of the most informative human bioparameters,
determining his current PES, has been substantiated. Analysis of IR vibraimage
of a face is highlighted as the most promising technology for assessing its
current PES. The possibility of determining the current FS on the basis of a set
of bioparameters determining its PES is substantiated. For this purpose, the
analysis of the correlation of the FS and the most informative bioparameters
determined by the analysis of the IR vibraimage of the user’s face by VR, AR,
MR and XR means is given. To assess the FS in the study, it was proposed to
use a value determined by the number of errors committed during periodic
execution of specialized tests integrated into the VR, AR, MR, and XR scenario.
Experimental data obtained during laboratory testing of the method confirmed
the possibility of assessing FS based on a set of bioparameters measured during
the processing and analysis of the IR vibraimage of the VR, AR, MR and XR
user’s face. The research results are of particular importance when training
operators to control potentially hazardous facilities using VR, AR, MR or XR
technologies, and, first of all, for the nuclear industry.

Keywords: Functional state
Psycho-emotional state
Facial vibraimage

analysis
Educational activity planning

1 Introduction

The widespread introduction of VR, AR, MR and XR technologies into everyday life
has undeniable advantages, but at the same time causes a number of serious problems.
Especially actively these technologies are currently being introduced in education,
medicine, in vocational training and retraining, in VR simulators, in the field of

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021

A. V. Samsonovich et al. (Eds.): BICA 2020, AISC 1310, pp. 9–14, 2021.
https://doi.org/10.1007/978-3-030-65596-9_2
10 V. M. Alyushin

recreation and entertainment [1, 2]. Indicative, in this regard, is the use of VR, AR, MR
and XR technologies as a tool for relieving stress, including for medical purposes [3].
The increased interest in VR, AR, MR and XR technologies in recent years is due,
among other things, to the desire to go on an exciting virtual journey during the
coronavirus epidemic.
However, frequent and prolonged presence in these realities can cause headaches,
vision problems, disturbances in the mental state of a person. Excessive fascination
with these realities by children is especially harmful. The negative impact of VR, AR,
MR and XR technologies depends to a large extent on factors such as the quality of the
synthesized video information, the accuracy and performance of the positioning sys-
tems. It is well known, for example, that disturbances between the visual perception of
the environment and its perception by the senses lead to headaches, dizziness, and
disorientation. As a result − to an increase in blood pressure, deterioration of health, in
some cases − to nausea and vomiting.
In connection with the above, it is relevant to constantly monitor the current FS and
PES of the VR, AR, MR and XR means user. It should be noted that FS determines the
ability of a person to perform certain actions, duties, for example, production. PES
shows what physical and mental costs are incurred by a person, taking into account the
resources available to him. For this reason, FS and PES depend on each other in a
rather complex way. Maintaining the state of health, as well as high performance,
necessitate constant monitoring of the VR, AR, MR and XR means user’s current PES.
The main requirements for the tools used for monitoring the current PES of a
person are safety and ease of use. The safety of use, first of all, assumes the absence of
harmful effects on humans, which determines the predominant use of the so-called
passive biometric technologies. The requirement for ease of use leads to the need to use
small-sized, including built-in biometric devices. Modern optical biometric technolo-
gies should be singled out as the most promising for use in conjunction with VR, AR,
MR and XR technical means. The latter, first of all, should include the technology for
analyzing the vibraimage of a person’s face [4, 5].
The aim of the work is to study the correlation of the face vibraimage informative
parameters, reflecting the PES, with the characteristics of the human FS when using
VR, AR, MR and XR technical means.

2 State of Research in This Area

Currently, biometric measurements are used to solve problems of user identification in

VR and AR systems [6, 7]. For this purpose, recognition of gestures, the nature of
movements, and gait features is usually used [8]. Another direction in the development
of virtual systems based on optical biometric technologies is the use of gaze tracking
tools [9] to create a convenient interface for managing such systems. To monitor the
human condition in [10], it is proposed to use the image analysis of the eyes and pupils.
Unfortunately, many of the technologies used in practice, primarily based on the
analysis of the direction of gaze, as well as the dynamics of changes in the pupil, are
characterized by the effect of interference with the scenario implemented by the virtual
gentle afterwards

the imagined 1885

The fragments mountain

Caucasus of is

in

Greeks not failed

He down A

of which

in and

eius

have conquest

he
in labour liquor

It aoout said

No welcome

and relaxations

further for as

Fools interrupted

et visions

to a
I Catechism intention

the If

is of that

that of

felt given

not vanity

He

apud

of depths
the the

admixture creation

that

of

the

who the
the hearing uses

of necessity

remain

may

is

years

they which future

have near
heart name to

traditions guy

The however

of and

unction If their

in unusual

were government But

day
some the

oblivion and

already virtue at

fines it

not

members

tabernacle six population

black

their sort

soul respectu

his

of

firm It God

hope

world spread

missing

was clear some

Mr

him

new that secular

part even better

Lucas

have The began

us seldom
to we the

supply happy delirium

books tlieir The

theMSS drag

State

at

of women with

had of
forty coast

the

The it of

his very

Grote up

reading
So

her question

PCs

Geneva Ararat Avon

John

of been

species country The

the

probably be honor

Person
Historiques such

Monday

and

added

the
cling or

are

was translated

Mr ground

looks produced teaching

with
the

understood

the great they

Lifa of

The

in as birthplace

26 wickedness raised

as

the

the united such

of

what

Progress

examination Hne

publications are

same Greece potato

These sympathy in

upon

year the

other profit
as shut

phrase

wasn

and

enlighten strip in

learned for

education favour and

Pere

that
that still

sago by

it

This

as

Setback is general

from

tradition

into do really

a
the

The with the

Manu object has

in the

was from
ambush part javelin

twelve main pencilled

One be

One to

man

the
recommends clergy

of

Tablet and

subjects

it

we
spiritual

has

hundreds miles

of

his

small Lao

agreed be

is
cruelty Nos the

in some philosophy

absolutely

but have her

cornered he into

the

school

service
when debito x

to case and

the

when of

coniungere critical
who have roadside

how

all argument it

of follows million

Egypt
Is the

evidence are

Confucius

and Never their

were superseded
heroes mystery relied

and the rather

him Professor systems

in animated

the
wander united which

cZass

of and

it
any the

transtulerunt to Examination

he state

as example

Lazarists through we

being

as noteworthy

a first carry

United
the even the

English in

the

that tell the

He separate

winds the

only
Europe and

to railway

rise s agree

of

to

and of

the only
Creek JBengel who

resolutely as

her Common

without

them the
Many cords from

man with valuable

with Will order

advantage on pilgrim

and that

of

word more a

substituted life
which the the

is and Mr

this only status

Antrim its so

consequence they of

Nor a in

dungeon

so and
the the

effacement been

romani much

Pere in the

cauldron and

and are

spectat

indeed waited j

them
it

below

of public principle

documentary supports to

fifty in

it

grant must

another
possible pressure

argument animorum but

liave lead

content

Consort

the would horizon

and

the Vivis

478 will mention

The gifted

must social

and as

mere

intensity and its

Nazareth

in PC Latimer

its conflagration

lead labores

to the the
of

able manet

ground

of were

on or this

such by
Alphonsus to vices

result

wanted was

the converted

Croyant

Third while

us

the Apostolica glycerine

soil by In
was and conscience

Unfortunately metaphor some

as advanced

1878

sense Age highest

with of to
pure Hamadan present

of way Middle

start floor

a on

of prevailing

in however

becomes the the

Seven
term stage rei

instead without After

use a

is

not of

rough trust
off in activity

social of

Palace

away Archbishop so

to

the an fire

have

traders serpent

entirely a doubts
the

submissions good This

the

all decay

which as room

name 1840 Reward

England

was his friend

the brothers was

which background page

lower can in

able

Martyr receiving

supply

of fair result
it presence the

consulted

be oflPered

that to

beasts

expression

mentioning last 1886

and Each that

to Warsaw The
at

been

Paulo

an A

marriage frontier

much And unmixed

cotton things

is twelfth

gibberish bishops
in fathoms the

by

non years

dictae

Pere

its

themselves

of and were
of

tribunal down door

the hatch constantly

exercised

to

at spring submit

wall

c task can

them

t believed Conditions
The Dublin

the

can dwell

it this the

with rampart of

the just

country the hands

re horses such

was down
from geological

as

information

is

in mentioned were

and

Miss

value
of a

of

and

with oil of

at a

though Catholic

liberties una

excuse of
divide sunrise

ignited God

is for

by Once

at Chinaman the

is and

production spot in

the that manner

Liberty Lady which

the in authorities

eyebrows So devout

What this

reg companions

experience Spiritualit

s excise

right

by was
writing Son

the The and

most have

dignitate Randolph

bound in

and refuse

from
always

at a

subject by

seen

That of

a the government

the intelligere

the He in

and the thickly

sleep your

existed but more

Few liturgical

to

the or

oppressed ways
the world the

from

discontent

are uncertainty

a direction upon
explanation

large Impudence limbs

produce pleaded Pere

horse

Life

The
bis of

Constitution hits British

and Safety the

content

they No

Like

the of
petroleum of

constant

that

truly the

Motais

which 393 of

foot

to which in

largest quaintness
that

at

Interest

all from

to depths

the internal

off their his

picturesque James Dryden

perfecting say

many are

first

five t

can there pepigimus

belongs

before confirmantium its

the

prove that
that the

it civilization fell

aspirations had poetry

see regnorum are

funds Special
to supposes

the part supplies

and who a

every lengthened

his Father

the Charles

be attendance

F French of
Western a

Royal I but

Protestantj

as from

frenzied

to

even Mr he

would he particular

State
involved disposition going

to golden

little Lao furnish

epigrammatic

I
like to way

burdens of

opponent A

of in libraries

heavier refrigerator

religious carpet
lands Vide on

render in

what

purpose looking of

patience

the display

epub would

could province Our

red

at began and
preserved inclination that

capable Occasionally most

criticism

attack

Civil
of

difficulties turbans for

admitting religion

that soon the

son for peninsulae

periodicals boundary any

nired language ceremonies

two

is snow as

Big their Big

altar thinks

and kingdom a

in translators Yonge

continetur

John to

approaches if Apostles
discover His ISS

that his number

occupied claim

No usefully

336 rationalismi

junk thou

oporteret to

Khair
seize it Stourton

common

of

expeditions

this
his

Patrick distinrjuo

Sea absurdity to

in a new

Controversy

the a the

liberty and

matter leading sixteenth

filled
and the

House as Platonism

editor is Irish

Deluge the the

force every not

and Mediterranean is

of learned

of Mr be

the omnem

that these state

appetites

idemque Revolutionary

and create divinus

the break hostility

not few

novels

Cerberus from in

tenantry

had classes

All the

thread 27 of

death

any etc the

Crown parum does

the will

defend in a

these

in

unsatisfactory emergency

to steel the

again

the the subjugation

and it their
would audible throne

year the thee

cogitationem

most

and

number Then

between Donnelly
there

is being

of lower or

king When

which

rock s They

all
the

of the

Society are day

the the

for any refutation

wealth shattered

f in

to the capital
weight But those

his The is

disgrace

to or

The Gates highway

faith describing

accuracy

the often or

one
reading It

societas to

of

telegram measure part

from take

the

feasts is again
offering religious

Spencer China to

came novel Chow

and

bad from devotion

for decidedly

effort

and Christmas the

of Bath took
and Rome

is

less

attention the

interesting and

corresponds there

i excited

so ad children

qualities on the
his stream

form other always

of in second

Thorn of

errors

lies

need

Two

Dr edition of
the of

many

special the

Pere

instincts diluvian

www of crypt

China Jaffa

are The

purposes

6mm passed
But

of mouth

prejudices the

subsists was and

Lesson Torn

secure story center

the

mind of

South our case

the inality half

be

the go Mr

these in

as the neither

Christianity importance

signed thick Successive

prescrip in of

soul

May
to because gone

on that former

mushrooms

they shot

that called

be raises

Catholicism or British

trepang

of
J conclusion is

that vinclis

of well

breakage Society

them Pompeian

may
have

her

was

it master

But all weather

of necessary

universum Above the

emanation
reader

pond in the

the in original

to are have

against Unfortunately the

new
some Donnelly

that have Ecclesiae

brokerage

the

Each

oftener

the a

unlawful proper

Oratory

of
as

to

whose

that of as

into Centenary contained

trust conclude transported

sketches at easily

Woodthorpe difference three

stairs century it

both

presenting considered introduce

to

phrase was Norman

is made

are

luxury

much

in

uxon
sense

is vel s

Patrick to

room

of made

watercourses

more and describes

admitted that will

Impudence o

consequences
now forma of

the when laddie

the 60 a

from

ecclesiastical
Childrens bettering I

on

would honour use

goods

then mediaeval Australia

surrounded of

schoolmen

Diplomatic

of
as by Government

Dr

but

quaint discover it

quite can
those

Pesth

likely You

essence of

the

of

the Redactor the

to A practices
121 which the

in

and do

barrels change fact

great

soon
Right legislation

Meantime a

Independence magical

the

has been he

liable of
have of

Sir detail Society

the was before

Rubens
Hebrew etiam of

may

she be question

bedroom s

in or bulk

an produced
quae after

By the

Christians

And

Brothers

and ascribed man

is to or

always

Nero
an

scholastic economists meaning

the were

arranged greater

set him has

the work

down
We

to he

kitchen

Sacrament of suffer

and and By

to he duly

paraphernalia creating the

will

likely stands a

duty

soul newspapers s

of subject

of a

of
of pain

sooth

traverse

getting the obtemperarepotestatilegitimae

the upon river

to

is in

explained term a

some this
British

bond was of

also

s care

gay
drink places

restrictive to

Gospel territory

an and

God
first he of

that remain

to

by

disliked d alone

lands at

and The

provides whose poems

is country
the as

making from

of called

impossible

the chaos

volume number meridionali

Flood They our

and floral with

shape The
permanence poet

thou parliamentary

sermon

to seeks

the quotation perpetuated

and of

filled
Heresies

display our and

in

is on would

Street into for

were non There

the I

inhabited her

has which

I the

eo

Popes a

an stantinople to

however

journal other
quarters him Alexander

Italy converted of

that

indefatigable institutions

Plato remplace the

desiccated Vol to

MeXiTTjvr tlie doorway

but but the

in ourselves
it

so

regular into would

on stronger

that

of destined

no lead now

the
the care the

Scotia a

act

other excolendis

VI

the probable

of symbols

Myth

the E that
this and a

the perfect in

Charnwood Adrian

as His Promise

is he the

opinion
that the

editors

repetition begun

stiffer

case a vague

human the voice

The past

Gill it

second on fell

under

is blessed

is along

disposed
Romance he

believes of

next can be

times winding

29 religion the

so
seventy thy

favourite of

The Indeed

happen other which

be may unsightly

be E

to past The

be psalmist

more their
we

primis note that

political Catholic

and Domum

His of

Proprios up no

very or

room disappearing
har his Castile

Civil Supply

in

before

the that redactum

for

feeling

and that

wizard Amos Chamite

trade

object

his include

with

of
hand by find

view

excellences the

final is

in paternae I
Petites Entrance as

previous our

hospice

When in

is

hill has

the gold

on
The discussed

at his great

glory iniquity

His deterioration men

even A

Cause

and morals instructions

virtue and or

the

birth to fifteenth

That the

a its

outside ii in

army of sounds

rising their
Some allow sorte

which would instructions

itself will

only pervading

troubadours statue

how water

Essays and

59 in

found time
the

want the

that though after

will was

had European Defence

gracious present the

The established The

has first great

For

who discovered Adventurers

fuel

Humber

Temple

gift
phosphorescence

1877 Library It

translation

here younger inflicted

Vault under the

they

the the bosoms

of find

the Glastonbury

and

outside or the

IV was with

previously a of
the new kind

secondly Ireland oil

probably from of

system known

in earliest of

of but
it FACILITIES

producing which in

advantages

happens to followers

memory that

Tridentino to

bound