SHS Web of Conferences 29, 02010 (2016) DOI: 10.

1051/ shsconf/20162902010
EEIA-2016

Augmented Reality and ARToolkit for Android: the First Steps

Liliya Demidova1, 2,*
1
Ryazan State Radio Engineering University, 390005, Ryazan, Russia
2
Moscow Technological Institute, 199334, Moscow, Russia

Abstract. The paper describes the aspects of the augmented reality application development on the base of
ARToolkit for Android. The steps consistently used for the augmented reality application development have
been considered. The simplest Java-based examples of augmented reality have been discussed .

1 Introduction 2 ARToolkit for Android

Augmented reality (AR) is a live, direct or indirect, ARToolkit is the most popular freely available AR
view of a physical, real-world environment whose software development kit. ARToolkit implements the
elements are augmented by computer-generated sensory logic necessary for AR, including [5]:
input such as sound, video, graphics or GPS data [1]. recognizing markers in the video frames;
AR is related to a more general concept called mediated determining the geometry of the scene (i.e., the
reality, in which a view of reality is modified by a position and angle of the marker);
computer. AR technology works by strengthening ones projecting the 3D model into the scene;
current perception of reality, while virtual reality creating the composite video frame (the view plus the
replaces the real world with a simulated one. On the base 3D model in registration with the marker).
of AR technology the information about the surrounding ARToolkit works with ordinary PCs and laptops
real world of the user becomes interactive. Artificial equipped with inexpensive web cams.
information about the environment and its objects can be AR applications on the base of ARToolkit for
imposed on the real world. The virtual objects of AR Android can be used for the smart phones. These devices
display information that the user cannot directly detect are of interest because they are rapidly becoming
with his own senses. The information conveyed by these ubiquitous, and therefore have the potential to transform
virtual objects helps a user perform real-world tasks in science, engineering, and technical education at
the best way. relatively low cost [5, 6].
In 1997, R. Azuma published a survey [2] that AR applications are written in special 3D programs
defined the field, described many problems, and that allow the developer to connect animation or
summarized the developments up to AR. Since then, the contextual digital information in the computer program
field of AR has grown rapidly. Nowdays we can speak to an AR "marker" [7] in the real world. When a
about almost 50 years of research and development in computing device's AR application or browser plug-in
the field of AR [3]. receives digital information from a known marker, it
R. Azuma defines AR as systems that have the begins to execute the marker's code and layer the correct
following three characteristics [2]: image or images [8].
combines real and virtual; AR applications for smart phones typically include
interactive in real time; global positioning system (GPS) to pinpoint the user's
registered in 3D. location and its compass to detect device orientation.
Registration refers to the accurate alignment of real Sophisticated AR programs used by the military for
and virtual objects. Without accurate registration, the training may include machine vision, object recognition
illusion that the virtual objects exist in the real and gesture recognition technologies [8].
environment is severely compromised. Registration is a Markers are the squares that ARToolKit recognises
difficult problem and a topic of continuing research [4]. and tracks in a video stream [9]. That is, markers are the
To enable rapid development of AR application, the physical patterns that we must created or printed out.
software development kits (SDK) have emerged. Some ARToolKit comes with PDF files for some pre-made
of the well known AR SDKs are offered by ARToolKit, markers, e.g. the Hiro marker, which we must print out
Blippar, Catchoom CraftAR, Layar, Vuforia, etc. and affix to card or board (so that they stay flat).
Markers are the optical inputs to ARToolKit.
Markers have the following constraints [9]:

*
Corresponding author: [email protected]
The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution
License 4.0 (http://creativecommons.org/licenses/by/4.0/).
SHS Web of Conferences 29, 02010 (2016) DOI: 10.1051/ shsconf/20162902010
EEIA-2016

they must be square; directory for them (Figure 3). As a result, the project
they must have a continuous border (generally either window should be appeared (Figure 4). Here we can see
full black or pure white) and they must sit on a the project structure (Figure 5).
background of contrasting colour (generally the opposite
of the border colour); by default, the border thickness is
25% of the length of an edge of the marker;
the area inside the border, which we refer to as the
marker image, must not be rotationally symmetric
(specifically, it must not have rotational symmetry of an
even order); the area inside the border can be black and
white, or coloured (and ARToolKit provides a means to
track with greater accuracy when the marker image is
coloured).
ARToolKit includes a number of fully-working
examples on Android that demonstrate basic and
advanced techniques [10, 11]. We can copy and build on
the examples to create our own application.
The examples are divided into 3 sets. Fig. 1. Android Studio start window
1. All Java-based: examples where all user-
developed code is in Java, and is based on the provided
ARBaseLib classes.
2. Mixed Java and native C/C++ using Android
NDK: examples where user-developed code is split
between the Java environment and native C/C++
environment. Code can use the provided ARBaseLib
java classes while also addressing ARToolKit in C/C++
via the libARWrapper C/C++ API.
3. AR and rendering code in native C/C++ using
Android NDK: examples where most of the user-
developed code is native C/C++. These examples offer
the greatest power to the AR developer and direct access
to the full native ARToolKit API.
The simplest examples are all Java-based. These Fig. 2. ARSimple examples selection
examples are as follows:
ARSimple: an example that extends the ARActivity
class in ARBaseLib to create a simple augmented reality
application;
ARSimpleInteraction: an example that adds simple
interaction to ARSimple.

3 Android Examples
To work with Android Examples we must install:
JDK version 7u79
(http://www.oracle.com/technetwork/java/javase/downlo
ads/index.html);
Android Studio with SDK and NDK
(http://developer.android.com/sdk/index.html); Fig. 3. The destination directorys selection
ARToolkit5-bin-5.3.1-Android
(http://artoolkit.org/download-artoolkit-sdk).
If we want to be able to run the AR application from
Android Studios built-in emulator, it is necassary to be
sure that virtualization is enabled in BIOS settings. Also,
it is possible to use any external emulators, for example,
Nox APP Player (http://en.bignox.com/#p2). Nowdays
this emulator is the stablest and powerful Android
emulator.
To start with ARSimple example we must open
Fig. 4. The project window
Android Studio and select Import project (Eclipse ADT,
Gradle, etc. (Figure 1). Then it is necassary to choose
the source files to be imported (Figure 2) and specify the

2
 SHS Web of Conferences 29, 02010 (2016) DOI: 10.1051/ shsconf/20162902010
EEIA-2016

Later, we can create our own marker, using the

template file Blank pattern.png (Figure 6) from the
same directory. Then, we must train it [9].
To test APK we can use Nox APP Player emulator
(Figure 7). It is necessary to install ARSimple APK and
run it. As soon as the Hiro marker appears in front of the
webcam (Figure 8) the colorful cube should be placed on
it (Figure 9). Now we can rotate the marker. The colorful
cube will rotate too (Figure 10). To stop APK we can
press Esc.
Fig. 5. The project structure
The files ARSimple.java, ARSimpleApplication.java,
SimpleRenderer.java are of the main interest in the
folder aRsimple. These files contain Java code of AR
application. Also, we can see here the aRBaseLib folder
and the Gradle Scripts folder.
ARBaseLib is an Android library. Android
applications can reference this library, and Android
Studio will take care of including the necessary files
when the AR application (APK Android Package) is Fig. 7. Nox APP Player
built and deployed. This allows reusable components to
be placed in the library and used in many different
examples and applications.
Gradle is an open source build automation system
that builds upon the concepts of Apache Ant and Apache
Maven and introduces a Groovy-based domain-specific
language (DSL) instead of the XML form used by
Apache Maven of declaring the project configuration.
Gradle uses a directed acyclic graph to determine the
order in which tasks can be run.
It is necessary say, that Android application package
(APK) is the package file format used by the Android Fig. 8. The Hiro marker in front of the webcam
operating system for distribution and installation of
mobile applications and middleware.
To make APK it necessary to carry out the following
commands:
Build > Make Project;
Build > Build APK.
In case of success the files such as aRSimple-debug-
unaligned.apk and aRSimple-debug.apk in the directory
C:\Users\MyDirectory\AndroidStudioProjects\ARSimple
\aRSimple\build\outputs\apk will be created.
The aligned APK is optimized for RAM usage so it Fig. 9. The colorful cube on the marker
will consumes less RAM in the devices. Therefore
aRSimple-debug.apk should be used to test and run APK.
Later (when the testing of any APK was carried out
successfully), it is expedient to create the file aRSimple-
release.apk. This file can be used for distribution.
Then we must print out the standard marker Hiro.pdf
from the directory :\Program Files\ARToolkit5-bin-
5.3.1-Android\doc\patterns (Figure 6).

Fig. 10. The colorful cube rotates with the marker

The more interesting file within the ARSimple
The Hirro marker The template marker project is SimpleRenderer.java class. Here we can see
Fig. 6. The markers how the marker definition is loaded
markerID = ARToolKit.getInstance().
addMarker("single;Data/patt.hiro;80");

3
SHS Web of Conferences 29, 02010 (2016) DOI: 10.1051/ shsconf/20162902010
EEIA-2016

Vibrator vib =
and how the cube is placed on the image (Vibrator)getSystemService(VIBRATOR_SERVICE);
vib.vibrate(100);

gl.glLoadMatrixf(ARToolKit.getInstance().
queryMarkerTransformation(markerID, 0);
We can change the installations mentioned above and
and drawn test the new APK. Also, we can replace the cube by
other figure and so on.
cube.draw(gl);

Also, we can see the cube definition 4 Conclusions

private Cube cube = new Cube(40.0f, 0.0f, 0.0f,
20.0f); The first steps of AR application development on the
base of the simplest examples have been discussed.
The first instruction defines the single marker These examples can form the basis during the creation of
patt.hiro from the directory the complex AR applications. These examples can be
:\Users\MyDirectory\Android used as the basis during the creation of the complex AR
StudioProjects\ARSimple\aRSimple\scr\main\assets\Data applications, for example, for the educational purposes.
with the edge size of 80 mm. In particular, they can be applied for visualization of the
The method queryMarkerTransformation of the work principles of the swarm intelligence algorithms
second instruction allows translating and rotating the (step by step) under decision of many important
cube. optimization problems [12, 13].
The fourth instruction defines the edge size of the
cube (40 mm) and the coordinates of it center concerning
the marker center (0, 0, 20). In this case the cube is References
raised over the marker on 20 mm.
In the simplest case we can change the marker name 1. https://en.wikipedia.org/wiki/Augmented_reality
in the first instruction and the cube parameters in the 2. R. T. Azuma, Teleoperators and Virtual
fourth instruction. Then, it is necessary, firstly, to carry Environments, 6(4), 355-385 (1997)
out such commands as Make Project and Build APK. 3. M. Billinghurs, A. Clark, G. Lee, Foundations and
and, secondly, to update the APK in the emulator and Trends in Human-Computer Interaction, 8(2-3), 73-
run it. 272 (2015)
ARInteractionSimple example adds simple 4. R. Azuma, Y. Baillo, R. Behringer, S. Feiner,
interaction to ARSimple. In this case the APK S. Julier, B. MacIntyre, IEEE Computer Graphics
reproduces the spinning cube on the marker. The and Applications, 21(6), 34-47 (2001)
spinning is toggled when the user taps the screen: 5. A. Craig, R.E. McGrath, A. Gutierrez, Technical
public void click() { Note: Augmented Reality Software Kits for Smart
spinning = !spinning; Phones (2011)
} 6. G. Milsap, E. Bourland, Advanced Rendering for
Augmented Reality on Mobile Devices (2011)
The following instructions
private Cube cube = new Cube(40.0f, 0.0f, 0.0f, 7. J. Kohler, A. Pagani, D. Stricker, Detection and
20.0f); Identification Techniques for Markers Used in
private float angle = 0.0f; Computer Vision, Visualization of Large and
private boolean spinning = false;
Unstructured Data Sets, IRTG Workshop, pp. 36-44
define the cube parameters, the initial turn angle of the (2010)
cube and the initial value of the rotation indicator 8. http://whatis.techtarget.com/definition/augmented-
correspondingly. reality-AR
The marker is defined as in ARSimple example. 9. https://www.artoolworks.com/support/library/Creati
The instruction ng_and_training_new_ARToolKit_markers
10. http://artoolkit.org/documentation/doku.php?id=4_A
gl.glRotatef(angle, 0.0f, 0.0f, 1.0f); ndroid:android_examples
11. https://www.artoolworks.com/support/library/ARTo
defines the turn with the predefined angle around the olKit_for_Android_examples#ARNative
vertical axis Z counterclockwise. The last three 12. L. Demidova, Yu. Sokolova, Modification of
parameters define the axis of the turn cube. Particle Swarm Algorithm for the Problem of the
Then, the cube is drawn SVM Classifier Development, 2015 International
cube.draw(gl);
Conference Stability and Control Processes in
Memory of V.I. Zubov (SCP), 623627 (2015)
The following instruction 13. L. Demidova, E. Nikulchev, Yu. Sokolova,
angle += 5.0f;
International Journal of Advanced Computer
defines increase in the turn angle by 5 degrees. Science and Applications, 7(2), 16-24 (2016)
When the screen is tapped, the APK is informed and
the sound of click is distributed in the emulator (the
phone vibrates correspondingly):