Journal of Applied Information Science

10 (1) 2022, 50-54

Interactive Museum
I. S. Sarath1, Jelin Treesa Joseph2, Josemi Jose3, Noel Gijo4* and Raisa Varghese5
1
Department of Computer Science and Engineering, Christ College of Engineering, Thrissur, Kerala, India.
Email: sarathithikkatt@[Link]
2
Department of Computer Science and Engineering, Christ College of Engineering, Thrissur, Kerala, India.
Email: jelintreesajoseph@[Link]
3
Department of Computer Science and Engineering, Christ College of Engineering, Thrissur, Kerala, India.
Email: josemijose2000@[Link]
4
Department of Computer Science and Engineering, Christ College of Engineering, Thrissur, Kerala, India.
Email: noelgijou@[Link]
5
Department of Computer Science and Engineering, Christ College of Engineering, Thrissur, Kerala, India.
Email: raisavarghese@[Link]
*Corresponding Author

Abstract: Museums are where we see many things of II. LIterature Survey
artistic, cultural, historical, traditional, and objects of
scientific interest. It is a great source of knowledge. It is
not only a source of knowledge but also helps us familiarize A. A Mobile Augmented Reality System for Exhibition
our history, culture, civilization, religion, art, and the Hall Based on Vuforia [1]
architecture of our country. Usually, a normal museum
experience is limited to the sights behind barriers put by AR is an enhanced reality with virtual content. Using mobile
the museum authorities and the visitors don’t have an terminals, they enhance the user experience of AR. It is
opportunity to actually know them. Hence our solution
accomplished by the implementation of three technologies:
an Interactive Museum achieved through the help of
Augmented reality and a little gamification which lets the ● Target recognition technology
visitors feel the cultural depth of every artifact and other ● Tracking register technology
collection displayed in the museum. The AR camera will ● Highly efficiency technology (for rendering)
provide with details of the objects it recognizes including its
origin and historical, religious or cultural importance. As a Target recognition is divided into two: Machine Learning,
part of aforementioned project, we did a literature survey which uses general characteristics, and Image Matching, which
and the conclusions we arrived after the survey is given in uses info and image features in the database and identifies tar-
this paper. gets. Based on Computer vision utilize technology to calculate
Keywords: AR, Augmented reality, Computer vision, current location into that of the observer two methods are used.
Gamification, MAR, Mobile augmented reality. ● The marker-based method which recognizes specific
marker through edge detection and template matching
I. IntroductIon andcalculates the spatial transformation matrix.
● Natural feature-based method acquires general features
Augmented Reality is an emerging field which piqued the that match with the real scene and calculate the spatial
interest of everyone and has been growing steadily, from the face transformation matrix of the virtual object.
filters used in almost every social media platforms, games like
They got higher efficiency using third party software to
Pokemon Go, customer services provided by companies which
improve rendering efficiency. Vuforia utilizes computer vision
let their customers experience their product like the furniture
to provide some data to users like multimedia, 3D models, etc.
company IKEA proving user experience of how their product
will fit in their home, Footwear companies like Adidas and Gucci A mobile screen is a hub that connects the real and virtual world
letting their customers see how their footwear will match the and due to other certain features like - Vuforia is more effective
fashion of customer with AR camera, Toyota’s recent vehicle compared to other AR SDK providers. The system structure
demo and Lenskart AR Camera which let the customer try on the used by the current team in this paper consists of 3 level. Data
frames before you buy them all are a demo of how Augmented level includes basic information library, target information
Reality is carving its name in the technological sector. library, and augmented information library. The service level

Article can be accessed online at [Link]

realize intercommunication between the application level and The result they arrived at after a user survey was that although
data level, this level contains services like target recognition, view touch showed better accuracy and efficiency, it was
augmented information, and rendering. Application-level not attractive enough. Though gesture-based interaction
retrieves input from the user, provides outputto the user, handles appeared to be more attractive, it ended up being not accurate
video input to the service level, etc. enough.
The creation process utilized target recognition based on image
matching and tracking register based on marker. The team used C. Augmented Reality and its Effects on Our Life [3]
the Display board as their AR target and augmented on it and
as a part of increasing the efficiency by extracting the feature • Negative Impacts of AR: Even though AR helps us in
model of the display board (which is the target) image and various applications it can also harm our life. Overuse of
created a feature model library. The conclusion they reached AR applications will inversely affect our physical health
was that AR needs a large quantity of augmented content as well as mental health. It will lead to eye diseases. AR-
therefore it requires high production cost. based navigation systems while traveling will distract and
inaccurate information will lead to great risk.
• Augmented Reality and Virtual Reality: AR is an
B. Freeze View Touch and Finger Gesture-based
enhanced version of the physical world. AR and VR are
Interaction Methods for Handheld Augmented inverse reflections to each other. Similarities include that
Reality Interfaces [2] a similar type of technology is used in both AR & VR.
The user has control over the virtual world, the user can
In the design provided in the paper they discussed the frozen interact deeply with the environment. Both can change
view touch method for interaction in an AR application which acceptably in the medical field (robotics surgery). While
when the freeze button is pressed, the AR tracking feature is they differ by the purpose of technology used as AR
disabled. The option to press the freeze button only becomes magnifies reality by adding virtual objects while VR
available when the program recognizes a target. After the target shapes the real image purely based on the computer.
gets recognized and the frame is frozen, the camera is no longer Another notable difference is that AR can be generally
needed to be pointed at the target and manipulations could used with mobile devices while VR uses HMDS.
be made upon the object displayed in the place of the target. • Threats to AR: One of the main threats to AR is the privacy
Examples of manipulations are translation, rotation, and scaling concern. Since AR combines both Real and virtual
which are made possible with help of a menu. The user can worlds, as long as the AR are being used, it also captures
only have one button selected at a time. After the manipulations a slice of the real world during the process. This raises
of the object are done, the freeze button is again pressed to concerns for many people whether this footage might
unfreeze the object, in such a way that the manipulated object get into the wrong hands. Nowadays, since everyone is
is mapped to the place of the previous existing object. They already spending more time in the virtual world rather
accomplished this by combining the detection and description than the real world, there is a chance that people might
parts with optical flow tracking and created a full-featured get fed up looking at screens all the time, and the social
robust AR tracking system. awkwardness faced by a majority of people while
The finger gesture interaction uses gestures instead of a menu using AR equipment may also be considered as a
with options. Since they use gesture interaction, the input point threat to the acceptance and progress of AR. The
information is detected from the tip of their fingers. inaccuracy in computer vision may lure away potential
consumers.
Pressing the ‘Hand’ button enables natural gesture interaction
mode. Once the hand is detected, the most prominent fingertip
will be identified by a white circle and that finger controls D. Virtual Reality (VR) and Augmented Reality (AR)
translation and rotation modes. The next finger will be identified Technologies for Tourism and Hospitality Industry [4]
by a red circle and will control the scaling. The appearance of
a big circle indicates that the input gestures will be ignored and AR is made to work efficiently by implementing some
vice versa. The same manipulations of the previous method are technical features like computer vision, object recognition,
implemented using fingertip tracking. Scrolling or pinching GPS, accelerometers, etc.
the fingers in mid-air provides the natural input. A skin color
detector that works in HSV color space is used to separate the The different technologies used for AR are:
hand from the background. The curvature-based contour point ● Marker Based: QR/2D code visual markers are recognized
sampling and elliptical fitting methods are used for identifying through a camera to give a resultant output. Certain 3D or
the fingertips. 2D objects are placed on or near the marker by taking
reference to its position on the screen.
52 Journal of Applied Information Science Volume 10 Issue 1 June 2022

● Marker Less Based: This method utilizes inbuilt miniature ● The technology can make translation simple and effective
versions of GPS, digital compass, and an accelerometer to which is a big problem for travelers facing language
provide data by considering the location and orientation barriers.
of the device. ● This technology can apply in navigation maps and
● Projection Based: Alterations in artificial light projected augment the direction with some additional elements that
onto real-world objects caused by human interactions are ensure safe and easy traveling.
considered as input and a desired or pre-planned output is ● AR technology enhances guest satisfaction through a
visualized in the virtual world. process of effective marketing, scanning images in bill-
● Superimposition Based: A real-life object is partially or boards, etc.
fully replaced by an augmented object in this method. ● The advanced AR can be used for staff training in hotels
Since an actual object has to be replaced by an augmented and other fields and it can also create blueprints allowing
object from every angle, computer vision and image investors to look through the final product.
recognition plays an important role in this method.
Location data is considered as one of the main data in
E. Implementation of Augmented Reality System for
augmented and virtual reality because the movements of the
user are reflected in his augmented point of view. In VR, the Smartphone Advertisements [5]
following concepts play an important role:
This paper describes implementing an augmented reality
● 6 Degrees of freedom for the detection of movement of a
system for smartphone application service to the provider’s
rigid body in 3D space.
diversified information on advertisements to users. This system
● Orientation, which is based on the combination of yaw, aims for two way communication between user and application.
roll, and pitch of a 3D object. It uses position information or information from sensors to
● Coordinates of the objects. provide more accurate information. The conventional mobile
applications have limitations such as a margin of error to offer
Each tracking system produces a signal detected by a sensor.
the information.
The AR/VR unit processes this signal and forwards it to both
CPU and GPU. When there are more than two contents to be displayed in
the same position as a smartphone screen with a limited size,
With the advancements in this technology, the application areas
overlapping can block users from obtaining proper information.
are rapidly being explored. The immediate areas for hospitality
Also, it does not take into account a user’s possibility of
and tourism are as follows:
actual movement so that if a neighboring content is displayed
● As AR devices create almost realistic and easy on an obstacle that a user can’t pass by, it is difficult to decide
navigation travelers can have a detailed look and feel of the user’s actual moving path. Furthermore, since it can
the place to visit. Thus AR/VR technology can be used as recognize only positions, changing the information on the
an effective tool for planning. real world leads to updating the information on augmented
● VR can be implemented in various theme parks to provide reality. As a result, it does not guarantee the reliability of the
a virtual based environment. Thus, this technology can be information
used as an effective entertainment tool.
In AR applications that use positioning systems such as
● A VR model can be an efficient means of communication
iNeedCoffee, Wikitude, and Ovjet, etc, But since they used
of large amounts of information as it brings out the user’s
GPS for positioning, they have a decreased accuracy. Also, if
perceptual abilities. Thus it can educate people via games
errors exist in the magnetic compass, that result in inaccuracy.
and other interactive sessions.
Most of the markers less AR apps that integrate advertisements
● The new AR/VR tourism experience can be added to don’t go further than just to trigger the user’s interests.
existing applications considering visitor’s requirements
by simply modeling and animating, so virtual attractions Both the unity package and Vuforia SDK are used to implement
can be made at effective cost. this kind of a marker less AR experience. For the databases
● Exploring the restaurants and hotels and interactive present, MySQL is used. PHP and XML maintain the data flow
dining experience can be brought with help of this technology. between the AR app and the database.
A virtual tour of restaurants and hotel rooms is possible The implementation of a user’s direct booking in a
through this. Also, they can book the rooms by virtually store is given. Which uses a leaflet or outdoor signboard
surveying the room before visiting them. Also, users can of the store to augment the 3-D model. They visualized
virtually order food from restaurants. In restaurants, the AR multiple objects in the same image target using different
images of the menu may be projected in a dining table that coordinates. When a 3-D object from the leaflet is recognized
allows guests to select items of their choice. it only augments that particular object. 3-D representation of
 Interactive Museum 53

seating arrangement is augmented with live data from the Displays. The VST-HMD that the state, Keller, and their
database. team created was having a very sophisticated design.
Constant use of these devices produces fatigue which
negatively affects the efficiency provided by these
F. Augmented Reality and Gamification in Heritage devices.
Museums [6]
External noises are those which are caused by the crowd at
the place of implementation of augmented reality.
To implement an effective AR application, it is important
to identify the feedback factor which will depict the user’s Applying the basic elements of gaming in any other field is
satisfactory level. Here artifacts are the primary medium and called gamification. The main outcome of combining gaming
augmented reality is the second. The curator creates targeted elements with augmented reality is to educate people about
content and the AR developer will transmit the message and a certain topic in a very interesting manner. Here, ‘Horus’
visitors will decode it. This paper describes 3 ways of getting is such a game that makes people aware of the history and
feedback from a visitor. culture of ancient Egypt. The game is designed to be played
● The first way follows by surveying, verbal or non-verbal in a particular hall of the museum which contains certain
reactions and facial expressions and interviews along tags. These tags are the key elements that help to implement
withthe visit. The visitor is aware of this process and can augmented reality. The game starts with a 2D video which
contribute to it. shows a background story and then the gameplay starts with
the player getting to be the character ‘Horus’, whose aim
● The second way is through system statistics. The system
is to demolish the evil villain ‘god Seth’. The player scores by
will analyze the time that visitor consumes when they run
shooting the flying scary monsters which are seen in the game.
AR.
A simple and practical game is implemented here which can
● The third method is receiving feedback from Social media be played by all ages. Two reasons that limit the gameplay here
websites. are the available space of the hall and the duration of stay
The communication model starts with the sender. The person who at the museum. When high scores are earned, the player gets
is responsible to prepare the content that needs to be delivered. to post it to social media during the intervals between
The person might be an archaeologist or a curator. This person the levels. These achievements create a sense of satisfaction
carefully delivers the information and interpretations which between the players. Finally, the players get to understand the
could be textual visual or auditory. A part of the sender’s role is whole concept portrayed, in an exciting way.
given to the AR developer /designer whose role is to choose the
most suitable devices which do not disturb or burden visitors. III. concLuSIon
There are two types of noises that may arise are internal and
external noises. Internal noise may arise in AR due to 3 reasons: AR adds the element of the potential required to make the
surroundings of reality more informative and contentful.
● Tracking Unlike VR, AR enhances the current reality rather than make
Sensor-based tracking is very sensitive to noise as it may up a wholly virtual environment. Due to the present day’s
be disturbed by the ambient magnetic field. Vision-based advancements, mobile augmented reality is more convenient
tracking that is feature-based would be problematic if the and affordable to the vast majority of people hence it has great
marker is occluded. That which is Model-based may be a potential. The types of tracking and recognition technology are
problem if the tracking model is lacking texture. Hybrid mainly classified into two types: marker-based and marker less
tracking is a combination of several sensing technologies based. In mobile augmented reality where the terminal used is
including vision-based tracking. a smartphone, there are mainly two interaction methods: The
● Interaction freeze View touch method and the Finger gesture method.
Regarding tangible AR, it is difficult to determine the The rendering of the augmented view is made more efficient
state of the computerized data that is associated with by using third-party applications like Vuforia, AR Core. The
the physical tool. Besides, using a marker less tracking occlusion of augmented elements is one of the reasons for
technology without the clarity of texture, the system will decreased efficiency in renderings. Since rendering and tracking
fail. Collaborative AR could cause noise if the people efficiency is directly related to image recognition and image
roam around more while using AR. Hybrid AR could be recognition is directly related to image quality, better quality
complex if the user is in a situation that needs to deal frames contribute to better functioning of the system. Games
with all the functions together. can be implemented using augmented reality by utilizing simple
markers. The possible gameplay is more immersive compared
● Display Technique to the conventional gameplay due to the combined participation
It consists of see-through HMD projection-based display of the audience. Internal and external noises may be present
and handheld devices. See-through Head-Mounted
54 Journal of Applied Information Science Volume 10 Issue 1 June 2022

in AR. External noises are those caused by the crowd where [3] R. Aggarwal, and A. Singhal, “Augmented reality and its
AR is implemented. Internal noises are caused in scenarios like effects on our life,” 2019 9th International Conference
tracking, interaction, and usage of display techniques. The 3 on Cloud Computing, Data Science & Engineering
levels in the AR system structure are data level, service level, (Confluence), 2019, pp. 510-515.
and application level. The main threats faced by augmented [4] A. Nayyar, B. Mahapatra, D. N. Le, and G. Suseendran,
reality are privacy concerns and fatigue caused by long use. “Virtual Reality (VR) & Augmented Reality (AR)
technologies for tourism and hospitality industry,”
International Journal of Engineering & Technology,
referenceS
vol. 7, no. 2.21, pp. 156-160, 2018.
[1] F. Peng, and J. Zhai “A mobile augmented reality [5] Y.-G. Kim, and W.-J. Kim, “Implementation
system for exhibition hall based on Vuforia,” 2017 of augmented reality system for smartphone
2nd International Conference on Image, Vision and advertisements,” International Journal of Multimedia
Computing (ICIVC), Chengdu, China, 2-4 Jun. 2017. and Ubiquitous Engineering, vol. 9, no. 2, pp. 385-392,
2014.
[2] H. Bai, G. A. Lee, and M. Billinghurst, “Freeze view
[6] R. Hammady, M. Ma, and N. Temple, “Augmented
touch and finger gesture-based interaction methods for
reality and gamification in heritage museums,” Joint
handheld augmented reality interfaces,” Proceedings of International Conference on Serious Games, 2016.
the 27th Conference on Image and Vision Computing
New Zealand (IVCNZ’12), Dunedin, New Zealand, 26-
28 Nov. 2012.