Second Workshop on Digital Media and its Application in Museum & Heritage

Virtual Presentation and Animation of Qingming Festival by The Riverside

Zhigeng Pan, Gengdai Liu and Zhi Li

State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China
Ningbo Institute of Technology, Zhejiang University, Ningbo, China
{zgpan,liugengdai}@cad.zju.edu.cn, [email protected]

Abstract There are some examples of physical reconstruction

of this painting in China, such as Qingming Gardens in
In this paper, we present a novel method to display Kaifeng and Movie Town in Hengdian. They are both
a famous ancient Chinese painting for the purpose of large parks made after the fashion of the scene in the
cultural heritage preservation. The framework of painting.
multi-projector display wall based digital storytelling But we use a distinct method to display this painting.
system is introduced. The aim of this system is to Firstly, a multi-projector display wall system is created
exhibit the ancient painting on a large display wall for displaying it. And then the painting, Qingming
dynamically. Before being projected, the painting has Festival by The Riverside is repainted by artists and
to be repainted with very high resolution and then is projected on a large display wall. The music
segmented into many smaller tiles, so that it can be and commentary is played synchronously as the video
displayed on the large flat display wall. Additionally, goes on so that he flourishing scene of the Song
in order to make this still painting attractive and dynasty’s capital appears before audience vividly.
living, the technique of digital storytelling is used. So a It is necessary to make the objects on this still
script called water transportation story is drafted to painting moving, so as to make this famous painting
tell a story about the prosperous city. We extract the more lively. Therefore, we use IBR (Image Based
movable objects from the original painting and redraw Rendering) and computer animation technology to
them with three dimensional figures, and then the create a 2D picture with moving 3D moving objects on
computer animation technology is used to animate it and display the animation on a large screen. In
them. addition, we introduce the concept of digital
storytelling into our system. A script is composed to
1. Introduction tell a story about the city, which consists of 11
scenarios.
In this paper, a multi-projector display wall based In the next section, the related works about digital
digital storytelling system is presented. This system cultural heritage which is similar to us are introduced.
involves displaying a famous ancient Chinese painting And in the third section, we described our method for
on a large display wall dynamically [1]. This famous virtual presentation and animation of the large painting.
ancient painting, Qingming Festival by The Riverside In the third part of this paper, we give some final
that was painted in Song dynasty about 1000 years ago results of our method. Finally, the conclusion and
is a rare realistic artwork and thought to be a world- future work is discussed.
class cultural relic. It is about 500 centimeters long and
25 centimeters wide. There are more than 500 human 2. Related Works
figures, 60 animals, 100 trees, 20 wooden boats, 30
buildings and pavilions and about 20 vehicles in this The technology of digital museum is an effective
painting. It gives audience a bird’s eye view of the method to preserve those natural and cultural heritages.
flourishing scene of Bianliang city, the capital of Song More and more specialists in heritage preservation and
dynasty. It was completed from a panoramic view and computer science are engaged in this field. Hu [2]
detailed scenes along a river that stretched for dozens made a summary of the application of virtual reality
of miles were covered. At present, the original version and digital media to digital heritage preservation.
of this painting is kept in the National Palace Museum As for the current research situation of digital
in Beijing, China. cultural heritage preservation, there are four main

0-7695-3065-6/07 $25.00 © 2007 IEEE 102

DOI 10.1109/DMAMH.2007.75
aspects focused by researchers all over the world. The where the story began. As a result, a long story with 11
first one is the digitization, storage and display of scenarios was plotted. After the story was done, we
cultural relics. The second aspect involves virtual could create the animations. First of all, the TIP
museums on the Internet. Kwon [3] presented the key technique [7] was used to enable the viewpoint to
techniques of digital museum on the internet, and a move, so the audience could feel like walking into the
general scheme of the modeling, management, storage painting and touring all over in it. The three steps
and transmission of large quantity of data of digital discussed above are implemented offline.
museum. The third aspect is concerning how to create After the offline work was done, we got a video
the avatar-based multi-user immersive virtual museum. containing the story. When the video is playing, the
Usaka [4] presented the prototype system of an every frame is segmented and projected on the large
immersive virtual museum. The terminal users can display wall.
enter the virtual museum as avatars via Internet to visit
all the exhibits in it and can communicate with other
users. The forth aspect is the application of large
display projection in the real museums. Ding [5]
presented a large screen projection system using 3
projectors to display an ancient street in Hangzhou,
China. Our work can fall into this category. But in our
work, the most important distinction is that the display Figure 2. Procedure of our method
wall is much larger and much more projectors are
employed. 3.2. Projection system
IBR (Image Based Rendering) is a kind of new
technique to render 3D virtual scenes from 2D images. The display wall in the museum is 6 meters high
It has been applied to many projects. Hirose [6] and 12 meters wide. We have to design a multi-
introduce many IBR applications to digital museum. projection screens based projection system in terms of
Youichi [7] present a novel method to tour into the the size of the wall. Therefore a PC-based multi-
picture by making animation from a single picture channel VR projection system is employed in order to
using spidery mesh. In our system, we use Youichi’s project a high-resolution storytelling video on so large
method to create animation by moving the viewpoint. a display wall. Considering the area of the wall, the
distance between the wall and the projectors and
3. Methods for Virtual Presentation and brightness those projectors can give off, we decided to
Animation use 15 projectors, 16 high-performance PCs and a
clustering multi-channel switch.
3.1. Outline Those 15 projectors compose a 5*3 projectors
matrix, and accordingly the whole screen is split into
Figure 1 shows a part of the original painting, which 15 rectangular tiles, each of which is covered by the
is very long. There are many interesting story hiding image from one projector. C/S architecture can be used
behind the scene of the painting. Therefore, we can in this projection system. The video is played and
compose a story consists of many scenarios from this every frame is segmented into 15 tiles in the PC server.
single painting. Then each segment is transmitted to the corresponding
client PC through switch synchronously. Each client
connected with a projector is responsible for rendering
one tile and projecting this tile to the proper position
on the large wall. The architecture of projection system
used in our system is shown in Figure 3.
Figure 1. Qingming Festerval by The Riverside

Figure 2 shows the procedure of our method. In the

first step, we invited some professional artists to
redraw the ancient painting above with very high
resolution. The new version of the painting is more
realistic, but keeping the style and fashion of the
original painting. In the second step, we selected 11
different parts of this painting to be the main places

103
 3.3. Creation of Animation

To make the objects in the painting movable, we

extracted them from the 2D background and animate
them. If the objects were solid ones, we just translated
and rotated them in terms of requirement. If the objects
are characters, including humans and animals, we use
skeleton-based character animation to drive them.
Figure 4 shows the skeleton (green) and skinned mesh
(purple) of a working people.

Figure 3. The framework of projection system

In order to guarantee that the video on the large wall

is consistent and these 15 tiles are seamlessly stitched,
geometric calibration must be used. But the manual
calibration is boring, especially for so many projectors,
because the position and orientation of every projector
should be adjusted. So we decided to use software
geometric calibration to stitch all the tiles seamlessly
and automatically. Multiple mapping matrices are used Figure 4. Skeleton-based character animation
for tiled display during software geometric calibration
as Zhou described [8]. Multiple matrices are computed To move the viewpoint when the animation goes on,
for each projector to map each pixel on the projected we use TIP techniques to create spidery meshes firstly.
screen to the frame buffer. The method goes as follow: Because the painting was repainted with panoramic
firstly, feature points for each projector are displayed view and the perspective relationship is kept, the
alternatively. Secondly, feature points for each spidery meshes can be generated easily.
projector are taken photos. Then, the calibration Additionally, in order to give audience the feeling
algorithm starts to recognize the feature points and of historical grandeur, the hue of the image is dim and
calculate the mapping matrices and the effective yellowish. And in order to combine 3D objects with
projection regions. Multiple mapping matrices method 2D background harmoniously, textures, lights and
means that one matrix on each calibration point must shadows are created and edited elaborately.
be calculated and then the corresponding point in the
frame buffer can be found. Then all other pixels will be 3.4. Storytelling Design
mapped to the frame buffer like texture mapping. This
method guarantees that there’s no overlap region and In order to make this static painting attractive and
gaps between two adjacent projectors. The charming, the technology of digital storytelling is used.
configuration of this system has to be high, especially So a script called Qingming Story is composed
the memories and network cards of PC server and elaborately by us to show the magnificent scene along
throughput and bandwidth of the multi-channel switch the river. This story is composed of many scenarios.
due to the large quantity of data. For example, in the first scenario called The Dock,
In our system, every frame of the video must be many porters carried goods onto the boats by the
segmented into 15 tiles and all the data of these tiles riverside, and then boatmen ferried the goods along the
have to be transmitted through the switch, which put river. There are some other locomotive objects in the
heavy burden on the PC server and the multi-channel river or by the riverside, such as carriages, mill wheels,
switch. So we have to reduce the resolution of the pedestrians and other workers as shown in Figure 5.
video. However, because the distance between the wall
and the audiences is far enough, the question caused by
lower resolution can be ignored and the final result is
quite acceptable.

104
 architecture of the projection system. We can change it
to meet the problems caused by simple C/S
architecture.

References
[1] Z.Li, G.D.Liu, Z.G.Pan. Story of Ancient Water
Figure 5. Turning wheel and people working with it Transportation: Design of A Virtual Presentation System.
The 4th Workshop on Digital Museum and Digital Heritage,
Kaifeng, 2006.
Many other scenarios then follow, such as Listening
to Music in The Pavilion and The Shadow of Pagoda, [2] W.X. Hu, Z.G. Pan, etc. Digitized World Heritage
etc. Each scenario tells a historic story happened in the Protection Methods. Journal of System Simulation. Vol.15.
city along the river. No.3.2003, pp.315-318.

4. Implementation Results [3] Y.M.Kwon, I.J.Kim, etc. Virtual Heritage System:

Modeling, Database & Presentation. In Proc. of VSMM’01,
IEEE Press, 2001.
The final result of one of the scenarios is shown in
Figure 6. In this figure, a snapshot of the large display [4] T.Usaka, K.Sakamura. A Design and Evaluation of the
wall is captured. Multi-User Virtual Environment Server System for the
Digital Museum. In Proceedings of the 13th TRON Project
International Symposium, IEEE Press, 1996, pp60-69.

[5] J.H.Ding, Y.G.Wang, Z.G.Pan. Hefang Street

Walkthrough Based on Large - Scale Projector System.
Journal of Hangzhou Dianzi University, Vol.25.No.1.2005,
pp.56-59.

[6] M.Hirose. Image-based virtual world generation. IEEE

Multimedia. IEEE Press, 1997,4 (1) , pp.27-33.

[7] H.Youichi, A.Kenichi, A.Kiyoshi. Tour into the picture:

Using a spidery mesh interface to make animation from a
single image, Proceedings of the 24th Annual Conference on
Figure 6. Final result Computer Graphics and Interactive Techniques, New York,
1997,pp.225-232.
5. Conclusions
[8] R. Zhou, H. Lin, J.Y. Shi. Geometric Calibration on Tiled
In this paper, we introduce a multi-projector large Display Using Multiple Matrices. In Proceeding of Pacific
Graphics ,2005, pp.97-99.
wall based digital storytelling system applied to
cultural heritage exhibition. In this system, 15
projectors, 16 high-performance PCs and a multi-
channel switch are used for projecting a famous
ancient Chinese painting on a very large display wall.
We use software calibration method for all the
projectors in stead of manual calibration because the
former can lead better result. We repaint the original
painting and put some locomotive 3D objects on it to
make the scene more lively and magnificent. To
discover the story hidden behind the famous painting, a
script composed of 11 scenarios is plotted.
The system will continue to be implemented in the
future. At present, our system can only play animation
video and there is no interactive capability. In the real
implementation, we intend to make this system
interactive. Another point could be improved lies in the

105