Development of HMD-based 360 VR Content of Korean Heritage
Development of HMD-based 360 VR Content of Korean Heritage
Indian Journal of Science and Technology, Vol 9(24), DOI: 10.17485/ijst/2016/v9i24/96153, June 2016 ISSN (Online) : 0974-5645
Abstract
Background/Objectives: The purpose of this was to present production and detailed workflow of 360° VR contents
systematically for some world cultural heritages of Korea. For methodology, VR content criteria and conditions were
presented based on theoretical backgrounds of VR. Methods/Statistical Analysis: For the scope, production techniques and
scenarios were derived for 360° imaging contents or heritage-related contents among VR platform contents provided by
Oculus Rift, the most prominent platform for VR contents. On that basis, heritage VR content development plan and results
were systemized. Findings: Regarding the results of this study, the study presented whole processes including VR content
planning, scenario configuration, equipment preparation, shooting, latter part works using stitching, special effects (VR-
VFX), unity and unreal engine for Seosan Maae Samjon Bulsang (Rock-Carved Buddha Triad) through 360° shooting, and
derived the following results. 1. Intelligent 360° VR contents are presented as the direction for future development, enabling
simulation in heritage space through application of LBS (Location Based Service) and motion recognition technology
(Kinect), etc. 2. Heritage 360° VR contents are developing into mixed contents integrating 360° images, interactive and
mobile contents. 3. 360° VR images which have excellent compatibility can find wide range of applications even to different
types of devices and 360° web. 4. HMD-based Korea heritage 360° VR contents are evolving into experiential information
formative space. Improvements/Applications: HMD-based heritage 360° VR contents are developing into simulation,
mixed realization, and informatization types and evolving into experiential information formative space, and therefore
require systematic and detailed research.
2 Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology
Jung-Yeob Han
the systematic classification of the reality and virtuality, Table 1. Configuration conditions of the HMD19.
Paul Milgram and Fumio Kishino explain it as Virtuality Item Conditions
Continuum (VC) concept connecting the reality and vir-
High UHD4K (4128x3096) or QHD (2560×1440)
tuality with four environment such as Real Environment Resolution or higher display that users can feel the
(RE), Augmented Reality (AR), Augmented Virtuality Display vividness
(AV) and Virtual Environment (VE)14. Wide viewing The users need to secure more than 90°-
Public recognition of the virtual reality was dealt angle (FOV) 110° viewing angle similar to the human
with conceptually in Matrix, Total Recall, and Ghost in body
the Shell. And in the field of novel it was also recognized Stereoscopic Stereoscopic image realization technology
as ‘cyberspace’ in the William Gibson’s science-fiction 3D that makes to feel the sense of three-
novel, Neuromancer. Modern experience equipment for dimension (depths) by presenting a pair
of binocular parallax 2D images to each of
virtual reality was an entertainment device to experience
the eye using the difference of viewpoint
three-dimensional images, surrounding sounds and smell between both eyes
with Sensorama developed by Morton Heilgin in 196216.
Positional An omnidirectional position tracking
More advanced concept of the first HMD (Head Mounted Tracking technology detecting the location and
Display) was ‘A Head-Mounted Three Dimensional movement of users
Display’ developed in 1968 by Professor Ivan Sutherland Head tracking A technology for recognizing the direction
based on “The Ultimate Display” research in 196517. Since and speed of users’ moving heads that
then, many types of HMDs have been produced, and requires recognizing with above 0.02 sec.
finally in 2012 Oculus Rift DK1 was developed to show motion reaction rate
limitless possibilities and popularity in virtual reality 4G Internet In order for Latency, the time taking from
experience. transmission a PC to a virtual augmented reality device,
speed to be 20ms or less, above 4G (2M-150Mbps
transmission rate) data transmission rate is
2.3 HMD Properties required.
HMD is the most important device in the experience of
the 360° VR. Each step of principle, configuration, devel- In 2016, the current status of HMD development is
opment status, and the types and specifications of HMD a total of 20 including 9 commercially available devices,
are as follows. First, for the principle of the HMD, three- 9 published developer devices, and 13 on-going devices.
dimension is formed through a VR device for each scene The category of specific device names is presented in
viewed by the left eye and by the right eye based on the Table 2.
focus of users. On this, optical lens are attached to secure
90°-110° viewing angle similar to the human body. Table 2. 2016 status of HMD development
Item Manufacturer & Product name Quantity
Durovis Dive, VRTX One,
Commercial Google CardBoard, DODOCase,
version Homido, Samsung Gear VR & 9
released Gear VR2, Cal Zeiss, VR One,
Figure 2. Visual principles of HMD, HMD viewing angle18. Archos and Mojing Storm
Oculus Rift DK1 & DK2,
The HMD consists of head tracking to sense presence ANTVR Kit, Moggles, VRELIA,
Developer
and simulation and to prevent motion sickness, stereo- Avengant Glyph, Sony HMZ- 9
version
scopic 3D, FOV, positional tracking, high-definition T3W, Razer OSVR and LG
display (4K), frequency rate above 60Hz, GPU capable 360VR
of processing more than 60 frames per second and 2.3 Oculus Rift CV1,
million pixels, and 4G Internet speed. Configuration con- Light&Shadows NEO, Vrizzmo,
Development
Fove, Altergaze, Panoglass, 13
ditions of the HMD are presented in Table 1. announcement
XGVR, Viewbox, AirVR, Pinc
and Apple
Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology 3
Development of HMD-based 360° VR Content of Korean Heritage
The type of HMD is classified largely into four of the spread of VR contents to the public. But because a
mobile removable-type, PC-type, console type, and sensor required for the VR experience is missing, there is
assembled type. 1. For a mobile removable-type, Samsung a drawback to experience quality degradation. The char-
Galaxy Gear VR is representative. Its display resolution acteristics of HMD Types are presented in Table 3.
is a QHD (2560×1440) showing excellent realism, and
providing excellent portability, usability, and scalability 2.4 360° VR Recording Equipment and
through wireless connection. However, disadvantages are Production Programs
requiring continued charging of the battery and a slower
In the 360° VR content production, 360° VR record-
motion reaction rate by wireless than wired connection.
ing equipment and production programs are important.
2. For PC-type, Oculus Rift DK2 is representative. With
First, the 360° VR recording equipment are 7 GoPro Hero
excellent resolution and tracking performance users can
4 Black Cameras, 1 GoPro remote control, a holder (rig)
enjoy a variety of contents via the VR platform with no
capable of holding 7 cameras, a battery charger necessary
battery problems, above all. But, the disadvantage is that
for outdoor shooting, a tripod required for shooting, a
it is not suitable for an active experience, as it is designed
camera bag for shock absorption, a high-speed SD card, a
to be used in sitting condition by connecting a cable to the
SD card reader, slate and etc. The category of specific 360°
PC. 3. For console type, Sony HMZ-T3W is representa-
VR recording equipment is presented in Table 4.
tive. It is going to be used in conjunction with a variety of
The program consists of a stitching program, a spe-
Sony’s upcoming console devices such as PS4 and PSVita,
cial effects (VFX) program and a VR content production
and a variety of game contents already released is good
engine. 1. The stitching program is the process of pro-
point. But the disadvantage is limited activity, as it is used
duction of 360° panorama video from 7 videos recorded
by connecting a cable to the console device as well. 4. As
consisting of a video editing program that matches the
the assembled type consists of a cardboard box, two fish-
sync, a stitching program that makes one 360° video from
eye lens and a magnet, it is cheap and has contributed to
Product name Samsung Gear VR Oculus Rift CV1 Sony HMZ-T3W Google Card Board
Characteristics Resolution: Resolution: 2160 x 1200 Resolution: 1920×1080 Resolution: unmeasurable
2560×1440 Frame: 90Hz Frame: 80Hz Frame: unmeasurable
Frame: 60Hz VR FOV: 100° VR FOV: 94° VR FOV: unmeasurable
VR FOV: 96°
Image
4 Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology
Jung-Yeob Han
7 videos and a program for correcting the 360° video cre- by selecting an option following the movement of the
ated. 2. The 360° special effects program consists of 360° view. However, they are limited for selecting the method
VR-VFX program to input texts and three-dimensional of an individual content and providing additional infor-
images into the completed 360° video. 3. Lastly, VR pref- mation. 360° VR Video + UI optional type are presented
erences and UI configuration, control configuration, and in Table 6.
building are set into a VR device using unity 3d or Unreal
Engine in order for users to experience the 360° video 3.3 Case Analysis of 360° VR Video +
with HMD. Portable Type
With 360° VR Video + Portable type, extreme sports or
3. Case Analysis of 360° VR car racers can be experienced at the time of the move-
Contents ment. During the experience the view is treated from the
first-person protagonist point experiencing a long take
In the case analysis, a sheet was comprised to analyze the video during movement. When seeing the contents from
three content types of 360° VR Video, 360° VR Video + the information aspect, presence of actual movement is
UI optional type, and 360° VR Video + Portable type in experienced like travels or sports in which tension and
the co-platform provided by Oculus Rift and gear VR in immersion are heightened, as the movement of users’
order for qualitative analysis about the general character- view can be moved with the flow of main character’s
istics and advantages and disadvantages. traffic. However, if shaking is large or a scene changes
frequently, it may cause simulation sickness due to the
3.1 Case Analysis of 360° VR Videos cognitive dissonance. 360° VR Video + Portable type are
presented in Table 7.
360° VR videos are made in several sequences with a long
take shooting mode. During the experience, the view is
mostly treated from the first-person camera point with a 4. Production of 360° VR Content
fixed location to show extremely long shot videos. When of heritage
seeing the contents in the information aspect, the themes
are mostly difficult to experience for the public, as they
deal with a special concert or a place. However, they have 4.1 360° VR Contents Development Plan
the advantage that users can experience them without 360° VR content planning for Buddha Triad Carved on
being conscious of the surrounding, but as most of them Rock Surface is proposed in the Table 8 with a step-by-
are made in a documentary format, they are limited for step processes of VR content data research, scenario
selecting the method of an individual content and pro- configuration, 360° VR imaging and stitching, 360°
viding additional information. Case Analysis of 360° VR VR-VFX, source file production, engine-based VR envi-
Videos are presented in Table 5. ronment configuration, UI controller, VR device building
and publishing.
3.2 Case Study of 360° VR Video + UI
Optional Type 4.2 360° VR Shooting
360° VR Video + UI optional type consists of a single • 360° VR shooting order according to the scenario
theme, but various episodes can be experienced option- is proposed in the Table 9 with a step-by-step
ally. For shooting mode, VR recording technique that work flow of checking and setting recording
consists of dozens of sequences is used by selecting a equipment, shooting location selection, ready to
single theme. During the experience, the view is mostly shoot, checking the camera remote control after
treated from the first-person observer point with a fixed shooting, data verification and moving files.
location watching a long take video in extreme long shot
or medium shot. When seeing the contents from the 4.3 360° VR Stitching and VR-VFX
information aspect, most of them are theme-centered • Stitching is a process of making one 360° video
contents such as adventures, museum visits, or interior from the 7 images shot. 1. A step of matching
space experience. Necessary information can be provided the sync of the 7 images shot by using an image
Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology 5
Development of HMD-based 360° VR Content of Korean Heritage
6 Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology
Jung-Yeob Han
4 VR device building
In front of the Buddha Triad Carved on Rock
5
Surface
5 Prototype completion and publishing
editing program. 2. The process of stitching the 4.4 360° VR Post Production
sync matched 7 images to one 360° video using
• The post production of VR content consists of
a program. A step-by-step work flow is proposed
VR pipeline configuration, VR preferences, UI
in the Table 10.
configuration and control manager creation, VR
• VR special effects (VR-VFX) is a process of image
correction and special effects to the stitched 360° building and heritage experience with HMD,
VR video, and its step-by-step work flow is pro- and its step-by-step work flow is proposed in the
posed in the Table 10. Table 11.
Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology 7
Development of HMD-based 360° VR Content of Korean Heritage
5 6
8 Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology
Jung-Yeob Han
4. Global ICT companies. Available from: http://www.etoday. 12. Hillis K. A geography of the eye the technologies of virtual
co.kr/news/section/newsview.php?idxno=1292989 reality in rob shields. Cultures of Internet Virtual Spaces,
5. Consumer virtual reality hardware and content rev- Real Histories Living Bodies London. Sage; 1996. p. 70–98.
enue to reach $21.8 billion by 2020. Available from: 13. Steuer J. Defining virtual reality: Dimensions determining
https://www.tractica.com/newsroom/press-releases/ telepresence. Journal of Communication. 1992; 42(4):73–
consumer-virtual-reality-hardware-and-content-revenue- 93.
to-reach-21-8-billion-by-2020 14. Milgram P, Keshion F. A taxonomy of mixed reality visual
6. Baudrillard J. Simulacres et simulation. Editions Galilee display. IEICE Transactions on Information and Systems
(French) and University of Michigan Press; 1963. E77-D. 1994; 1321–9 .
7. Kim JH, Kim YM, Kim MG, Song EJ. Kidult contents devel- 15. The integration between 4D simulation, mobile technol-
opment using mobile augmented reality. Indian Journal of ogy and augmented reality for automating construction
Science and Technology. 2015; 8(9):518–25. progress monitoring and communication. Available from:
8. Never dictionary. Available from: http://dic.naver.com/ https://www.linkedin.com/pulse/integration-between-
search.nhn?dicQuery=%ED%97%A4%EB%A6%AC%ED 4d-simulation-mobile-technology-reality-elgohari
%8B%B0%EC%A7%80&query=%ED%97%A4%EB%A6% 16. Heilig M. Sensorama. US Patent #3,050,870.1962.
AC%ED%8B%B0%EC%A7%80&target=dic&ie=utf8&qu 17. Sutherland IE. The ultimate display. Proc IFIP 65 2. 1968;
ery_utf=&isOnlyViewEE= 506–8, 582–3.
9. Deacon H. Type of integrated approach and the role of local 18. Will the Oculus Rift make you sick? Available from: http://
communities for the protection of intangible heritage. 2012 doc-ok.org/?tag=field-of-view
The UNESCO-Accredited NGOs of the Intangible Cultural 19. Optimized virtual reality says Oculus. Available from:
Heritage (ICH) Internationnal Symposium, Korea Cultural http://www.inven.co.kr/webzine/news/?news=120388
Heritage Foundation; 2012. p. 22. 20. Oculus experiences. Available from: https://www2.oculus.
10. World heritage sites. Available from: http://www.unesco. com/experiences/rift/
or.kr/heritage/wh/ 21. Oculus experiences. Available from: https://www2.oculus.
11. Virtural reality. Available from: https://ko.wikipedia.org/ com/experiences/gear-vr/#psea
wiki/%EA%B0%80%EC%83%81%ED%98%84%EC%8 22. Oculus experiences. Available from: https://www2.oculus.
B%A4 com/experiences/gear-vr/806687776079309/
Vol 9 (24) | June 2016 | www.indjst.org Indian Journal of Science and Technology 9