Context-Aware 3D Object Streaming for Mobile

Games
Hesam Rahimi, Ali Asghar Nazari Shirehjini, Shervin Shirmohammadi
Distributed and Collaborative Virtual Environments Research Laboratory (DISCOVER Lab)
School of Information Technology and Engineering, University of Ottawa
800 King Edward Ave. Ottawa, Ontario, K1N 6N5, Canada
{hrahimi | ali.nazari | shervin}@discover.uottawa.ca

Abstract— In this demo we showcase a method for the selective Our work, which is complementary to the above and can be
streaming of 3D objects in online mobile games. The proposed used in conjunction with them, is based on the idea of context-
method ranks each object by its importance for the current aware prioritized game streaming. As explained in details in
activity performed by the player, and transmits the most our previous paper [3], our method uses an importance factor
important objects first. Our goal is to improve the real-time defined for every object for a given game activity. We will set
response and interactivity of networked VEs and games through the priorities of the objects based on their importance factors
an efficient context-aware 3D object selection and prioritization and stream the most important objects to the player first. In
scheme, before streaming those objects over the network. other words, taking into account the current activity undertaken
by a player, different objects will be sorted based on their
Keywords- 3D streaming, context-awareness, virtual
environments, multiplayer games, network games, mobile games
priority for that activity, and then they will be streamed to the
player dynamically while the player is playing the game. This
I. INTRODUCTION is described in details next.
In dynamic virtual environments like online 3D games, we III. DESIGN AND IMPLEMENTATION
might need to load some 3D models, textures, and materials (all
called assets) at runtime. Sometimes after releasing our In our approach, the idea is to determine a small subset of
product, we might want to update the 3D models without the environment in each frame and to stream only that subset.
rebuilding the whole project. There are some scenarios where We use the game context under which the game is played, and
this is the case and loading 3D models dynamically at runtime we find which objects are more relevant and important for the
is needed. In these cases, usually some kind of assets that are current activity undertaken by the player. After finding the
not included inside the game executable must be added objects that are needed to fulfill the gaming tasks, we stream
dynamically. One example of such situations is where we want those objects first. By doing so, we increase the interactivity of
to have extra contents like additional weapons, environments, the game and we assure that the player will have the most
characters or even fully new levels in our game. Therefore, important objects first, in his/her gameplay experience. As we
dynamically downloading and instantiating the objects is previously explained in [3], the importance of an object is
needed. determined using a pre-defined table designed and created by
game designers. In that table, all possible activities as well as
As delivering the content of the Virtual Environments and all 3D objects are shown, and the importance of an object is
games is still challenging for mobile devices, given their defined for different activities.
limitations of network bandwidth, processing power, memory
size, and display size, we introduce a new context-aware For our proof-of-concept game, we have used the Unity 3
mechanism for progressive delivery of 3D models. By doing game engine [4] which supports rendering, lighting, audio,
so, we can address the challenge of limited bandwidth and physics, terrains, networking, and programming. This engine
resources that mobile devices are faced with. also supports iOS and Android operating systems for mobile
devices. Our roof-of-concept game is based on a sample project
II. RELATED WORK that comes with Unity 3 by default, named Bootcamp. By
default, bootcamp is nothing more than an environment where
In large virtual environments where only a specific region the player can navigate and shoot. We used this as the base of
of the environment might be visited by the user, geometry our framework and we added dynamic streaming support (from
transmission [1] is among the main methods used to reduce a web-server), context-awareness, as well as multiplayer
bandwidth usage and computing power of the receiving clients. capability. In order to make bootcamp multiplayer, we used the
Other works such as [2] address this issue to some extent by Photon Socket Server [5] adding Massively Multiplayer Online
focusing on prioritization of objects in a given scene based on Gaming (MMOG) experience to our proof-of-concept game.
object scope and viewer scope. The idea here is to transmit
only the objects whose scope has some overlap with the Our proof-of-concept game could be played either in a local
player’s viewing scope. While this improves the performance, area network (LAN) environment or via the Internet. 3D
priority is simply defined as whether or not an object’s scope is objects and assets are stored in a web-server in a 3D database.
inside the viewer’s scope; i.e., priority is defined as distance.

978-1-4577-1934-9/11/$26.00 ©2011 IEEE

 Figure 1: Different snap-shots of our proof-of-concept game: Left) the game uses our approach; Right) the game uses the
distance-based approach

Therefore, object streaming will be done via the Internet. 3D In this demo, our goal is to show the workshop attendees
objects are streamed dynamically on-demand. The game’s the difference between our streaming approach and the
goal is for a player to shoot as many weapon warehouses in the distance-based approach. As such, we run our proof-of-concept
environment faster than other players. As the warehouses are game in three different methods at the same time, in a
shared among all other players, when a player shoots a multiplayer environment. These three different methods are as
warehouse first, the score of that warehouse will go to that follow:
player. The game is finished when all the warehouses are hit.
1. The game is running in a PC, which has a bigger
IV. PROOF-OF-CONCEPT display than mobile devices, in its full version,
with all objects streamed to the player.
In this section we talk about what will be actually shown
during the demo, and how it will prove our contribution. 2. The game is running in a mobile device while the
distance-based approach is used for streaming 3D
Figure 1 shows two snapshots of our game, for the same objects.
frame using different methods. The left one shows the game’s
rendering using our method, and the right one shows the 3. The game is running in another mobile device
game’s rendering using distance-based approach. In both cases, while our context-aware method is used for
the activity undertaken by the player in that specific frame is streaming 3D objects.
“shooting”. In these pictures, streamed objects are highlighted
The first version of the game is used as a reference for the
in RED for better visual identification. When the game uses our
workshop attendees to see the game in its full version, and to
method for its object streaming during “shooting”, only the
make comparisons to the mobile devices easier. As mobile
most important objects for shooting and aiming are streamed,
devices still have limited battery, processing power, memory,
in this case weapon warehouses. But in the distance-based
and display size compared to their PC counterparts, we shown
approach, no matter what the current activity is, only the
the full version of the game running on a PC, and we also show
objects which are in certain distance from the player are
the game running on mobile devices with limited resources.
streamed.
Comparing the two mobile clients described before, each using
Therefore, when the game uses our approach, irrelevant a different method as their streaming technique, we can see that
objects for a given activity will be skipped in streaming. In this the client who uses our context-aware approach will give the
case, irrelevant objects for shooting will be obstacles such as player a higher quality of experience with better interaction and
fences, barrels, etc. But when the game uses the distance-based higher score in the gameplay experience, compared to the other
approach, not only important objects that are far from the user mobile client.
will not be streamed to the player, but also the bandwidth has
In the next section, we have described our demo setup.
been used in a non-efficient way by streaming irrelevant close
objects. Furthermore, using our technique, we have increased V. DEMO SETUP
the overall experience and game quality on the mobile device
for the player. Our demo environment consists of one PC as a server and
two mobile devices (Android-based) as clients. All of these
devices are Wi-Fi capable. We also have a wireless access
point (WAP) which is connected to the Internet offering VI. CONCLUSIONS AND FUTURE WORK
Internet connection for all devices. Figure 2 shows our demo In this demo, we tried to improve the real-timeliness of
setup environment. mobile networked games by developing an efficient context-
aware 3D objects selection and prioritization scheme before
streaming. We implemented a proof-of-concept game to
evaluate our proposed technique and showed that the player
who uses our streaming technique for 3D streaming has better
interaction with the game and receives the most important
objects first.
In our testing and analysis, we have found that depending
on the object arrangements and their positions in the scene,
depending on the number of total activities we have in the
game, depending on the number of objects in the scene, and
depending on if we have used all the available activities or not,
we might have some scenarios where the distance-based
approach might also have similar results to our context-aware
approach. Our algorithm will be better when the game is
bigger, in terms of total number of objects and activities.
For our future work, we will do further evaluations to
confirm that using our approach for object streaming is more
efficient in terms of resources, while it also increases gameplay
experience and quality. For this, we need to capture the total
amount of data transmitted from the server to the client, in
order to compare the difference in the amount of data
transferred with each streaming technique. Also, we will do
more tests and analysis with regards to the MMOG aspect of
Figure2: Demo Setup Environment our framework.
We put all 3D objects and assets to be streamed in a web REFERENCES
server. The Photon Socket Server runs in a PC which has the [1] E. Teler, D. Lischinski. “Streaming of Complex 3D Scenes for Remote
role of a server in our environment. All other clients are Walkthroughs”, Procs. Eurographics, 17-25. 2001.
connected to the server using its IP address and a specific port [2] F. Li, R. Lau, D. Kilis, L. Li, "Game-on-demand: An Online Game
number. Using a free version of Photon, we create multiple Engine Based on Geometry Streaming," ACM Transactions on
“game groups” consisting of up to twenty players per group. Multimedia Computing, Communications and Applications, accepted
2010.
Players who are in a same game group can then play together in
the same level and have interaction with each other. Once a [3] H. Rahimi, A.A.N. Shirehjini, S. Shirmohammadi, "Context-Aware
Prioritized Game Streaming", Proc. Workshop on Interactive Ambient
client wants to connect to the server, not only the server’s IP Intelligence Multimedia Environments, in Proc. IEEE International
address and port number is needed, but also his/her group name Conference on Multimedia & Expo, Barcelona, Spain, July 11-15, 2011
needs to be set. [4] http://unity3d.com/
[5] http://www.exitgames.com/Photon