An Optimization Scheme of Visual Intercom Call Monitoring

Huili Dai1,a, Shulong Chen2,b

1
Department of Computer Science,Minnan Science and Technology Institute Fujian Normal University
Quanzhou 362300,China
2
Fujian Aurine Technology Co.,Ltd,Fuzhou 350015, China
a
[email protected], [email protected]
*Huili Dai

Keywords: Visual intercom, Terminating call, Optimization scheme

Abstract. This paper designs an optimization scheme of visual intercom in which the video
monitoring and identification of the visitors will be triggered by the visiting call through the phone at
the entrance of community or unit. This paper also proposes a solution to improve the problems of
unsmooth video transaction in traditional system. The optimization scheme has been applied to several
projects and has shown remarkable value and broad market prospect.

Introduction
Visual intercom plays a positive role in community security. When the visitor call in, the phone
indoor will ring to remind the host and triggers the monitoring and identification of visitor. This
system will enhance the security and reduce the bothers for the host. In addition, the host can have
video call with visitor by answering the phone. Market research has shown that live visual intercom
software installed in Android system maintains a momentum of rapid growth in the market.
With powerful functions of Android smart phone, it is self-evident that we can save cost and time
of research by applying SIP-based audio and visual communication through the Internet through
mature framework of Linphone. The traditional way to connect the monitoring of visitors with visitors’
phone call via phone at the entrance of unit or community is supported by OpenCore, the core
technology of android multi-media framework. By implementing the Media API Android provides for
upstairs development, say Window Media Player, to receive or to play audio or video delivered by
stream media and to achieve RTSP monitoring.
In daily operation, however, the writer has found that traditional solution has two problems: first,
medias of Linphone are operate on Mediastreamer2, this may lead to connector calling error of bottom
system if the module of monitoring is realized by Mediaplayer; Second, as Mediaplayer combines
RTSP and monitoring video, the display and stop of video are relied on the base of MediaPlayer.
When the hosts decide to answer the phone, the system has to stop the RTSP before connecting to the
SIP talk. Since the process of stop and connection will cause inevitable delay, the transaction from
RTSP to SIP will be unsmooth[1].
To solve the problems mentioned above, this paper hereby proposes an optimization scheme of a
large general visual intercom. The writer introduces the third-party open-source library Live555 and
provides Live555 stream media connector for API. Here it receives data delivered by stream media
server and inserts Live555 library into Linphone. As a result, RTSP and SIP share the same library to
receive data and the stop in transaction is avoided. Users just need to renew the data receiver and the
cut of transition time can bring about favorable user experience. In addition, it has proved to be
reusable expansible in source code management[1].
____________________________________
978-1-4799-5455-1 /14/$31.00 ©2014 IEEE

85
Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on September 04,2024 at 10:41:53 UTC from IEEE Xplore. Restrictions apply.
Overall System Design
(1)System Design Ideas
Live555 inserted into Mediastreamer2 library as a static library while Mediastreamer2 in
Linphone is packed as library file in the library of Android system. JNI calling is then employed to
serve application framework. That is to separate display/stop video process of RTSP from request
process of RTSP. In this way RTSP and SIP share a same library. And the system uses the video
processing functions of Mediastreamer2 and obtains a new Filter for RTSP mutual processing.

Fig. 1 Distribution of its hierarchical structure

(2)Framework Introduction
1) Linphone Framework [2]
Since Linphone is module designed, terminating call is a process After module SAL has been
informed by SIP event message, is to be processed after is a circulating event that . This is an
important part of this paper. Then we will discuss the realization of Linphone terminating call in the
following section.
In the first stage, the system will switch SAL module to call receive function and inform the host
of call in request and create a calling object for it. Meanwhile, the system will renew bottom calling
state into Linphone Call Incoming Received and inform the Linphone manager of call state. Then the
manager will inform the main service module according to the distribution of information. Once the
main service module receives that information(judge the status as incoming received), it will trigger
the UI reminder of incoming call. At the same time, the audio module of SAL will display the ring to
remind the host. After that the system will send 180 SIP command package to inform call originator
that the call request has been delivered. In most cases, the length of code and types of facilities are
dominated by coding rules. The length of code will be helpful for the host to identify the call is made
from the phone at the entrance of unit or community and to trigger the incoming call monitoring.

86
Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on September 04,2024 at 10:41:53 UTC from IEEE Xplore. Restrictions apply.
 Fig. 2 Calls by call processing sequence diagram

2) RTSP Monitoring Framework

The reason why the writer chose Live555 to realize RTSP monitoring is that after years of
development, Live555 stream media library has acquired stable functions and delicate designs and has
been developed into a cross-platform module which supports various video coding formats. Moreover,
it can be easily extended to support other formats[3].

The Specific Design and Implementation of Optimization Scheme

The optimization scheme is realized by the modification of Live555, the pack of connector and
the conformation of Live555 into Meadiastreamer2 as a Filter module. Mediastreamer2 is a small yet
powerful stream engine specific for video call. It can process the sending and receiving of multimedia
stream like the capture, coding, decoding and rendering of audio and video in Linphone. In this
process, media works as Filter or plug-in. Different plug-in combinations can achieve different
functions. In RTP Filter, for instance, audio decoding Filteris to display audio through the Internet
while File Filter audio decoding Filter is to display local audio files. Combinations like this will help to
acquire more functions. In addition, each Filter just needs to focus on the date received and to process
and distribution. They do not need to care about the origin and destination of the data. This will be
helpful for the daily administration and repair of module. Suppose the format of video stream is H264,
then the major modification need to be conducted is:
(1) the original FileSink received data file storage mode, sent directly to an external decoder
module MSFilter;
(2) adding a state in the original notification interface protocol stack in order to make the call ends
understand the current protocol stack in which the state and the results of interface calls;
(3) Package of live555 interface rtsp_init, rtsp_connect interface, easy to understand and upper
calls;
(4) modify the task scheduling doEventLoop function call, not in the form of call blocking, call
the function by an external start the thread, the main thread does not block calls.This completes the
Live555 modifications and interface package.By now the Live555 modification and the pack of
connector are completed.

87
Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on September 04,2024 at 10:41:53 UTC from IEEE Xplore. Restrictions apply.
 void BasicTaskScheduler0::doEventLoop(char* watchVariable) {
while (1) {
if (watchVariable != NULL && *watchVariable != 0) break;
SingleStep();
}
}
Added a Live555 Filter structure encapsulates the Live555 MediaStreamer2, the structure of the
statement Live555 Filter, live555_init main calling function rtsp_init (live555_callback,
live555_data_callback, NULL, f, 1), wherein the state live555_callback callback,
live555_data_callback callback data call ms_queue_put (f-> outputs [0], blk) in live555_data_callback
function in the received data transmission to the next data queue Filter.
MSFilterDesc ms_live555_player_desc={
.id=MS_LIVE555_ID,
.name="MSLive555Player",
.text=N_("rtsp client"),
.category=MS_FILTER_OTHER,
.ninputs=0,
.noutputs=1,
.init=live555_init,
.process=live555_process,
.uninit=live555_uninit,
.methods=live555_methods
};
(1) filter to create and initialize the connection, the flow of the specified data, call
initlive555mediastreams2 () Specifically completed, the internal implementation details are as follows:

Fig. 3 Live555 streaming media control and data flow diagram

①Initialization and set ortp library can support rtp of payload values, including support for audio
and video payload;
②MeidaStream Filter core module initialization, each filter module is added to the system list;
③Initialization LIVE555 VideoStream video object structure, a combination of the following
functional modules use a Filter, ms_filter_new first instantiated according to their ID function modules,
MS_LIVE555_ID for Live555 module, MS_H264_DEC_ID for H264 decoding module,
MS_ANDROID_DISPLAY_ID for ANDROID display module, use ms_connection_helper_link after

88
Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on September 04,2024 at 10:41:53 UTC from IEEE Xplore. Restrictions apply.
initialization or ms_filter_link link their data link, which is the data output Live555 pointing H264
decoding ms_queue received data queue. Function code is as follows:
(2) When the link is good data link, start Live555MediaStreams, call video_stream_live555_open
(args-> video, & param) function will start a ticker task scheduling, task scheduling the main function
can be simply considered in turn creates a thread scheduling filter module the process execution
functions. Driven by the implementation of each module process flow data, to achieve a combination
of functional modules. Data output received RTP data queue ms_queue point H264 decoder, the
function is executed when the RTP ms_queue_put (f-> outputs [0], blk) function, which will complete
the BLK data output process, in fact, the data is added to the data queue H264 decoder ms_queue in [4].
Perform BLK = ms_queue_get (f-> inputs [0]), H264 decoder to retrieve data from the queue for
processing and transmission of the next level, H264 decoder DisplayWindow outputs data to a display
module, thus receiving the streaming media data began to appear on the video area of the user
interface.
(3) Stop Live555MediaStreams, in fact, call the function video_stream_live555_close (args->
video) closed video object.

Optimization Effect Test

After the Android smart phone based receiver software has been developed, the writer
investigated the performance of automatic talkback video monitoring software on mobile phone
through community service platform (including RTSP server and SIP server). From the data shown in
Table 1 we can see the optimized visual intercom is stable and welcome. Incoming call from the
entrance phone will trigger the ring and RTSP monitoring. The visitor’s image will be shown on the
screen of mobile phone. The host will enter a audio and video talk interface by answering the phone
without frame skipping, flash screen and blank screen. Figure 4 demonstrates the screen shots of
traditional solution while Figure 5 displays the screen shots of optimized video call among 200 trails.

Fig. 4 A black screen or splash screen phenomenon screenshot traditional solutions

89
Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on September 04,2024 at 10:41:53 UTC from IEEE Xplore. Restrictions apply.
 Fig. 5 The actual test renderings optimization program

Table 1 Video intercom client software testing which is based on Android smart phone system
Item Test Splash screen the number Black screen Are Instant
of occurrences appears Video Switch?
Traditional solutions 300times 11 (times) 17 (times) No
Optimization program 300times 0 (times) 1(times) Yes

Summary
This paper proposes a practical optimization scheme for visual intercom of community. This
server has been implemented in several enterprise projects and has been proved to be stable, reliable.
The outstanding advantages enable it to solve the potential problem of traditional system unsmooth
transaction well. Therefore, it is of practical value and broad market prospect. Hopefully it will be
helpful for developing better modes and solutions in future researches.

References
[1] Huili Dai,The design and implementation of smart home client system based on
Android,Fuzhou:Fujian Normal University,2013.
[2] Jinlong Zhao, Android-based audio playback system design and implementation, Xi'an University
of Electronic Science and Technology, Xi'an, 2012, pp. 20-33.
[3] Changling Liu,Jie Bao, Ningguo Wang, Based on Live555 network video surveillance system
design and implementation, Modern telecommunications technology, Beijing, 2012, pp. 38-42.
[4] Fengfeng Chen,Application Development RTSP streaming media transmission system, Nanjing
University of Posts and Telecommunications,Nanjing, 2013,pp. 26-30.

90
Authorized licensed use limited to: Sri Sai Ram Engineering College. Downloaded on September 04,2024 at 10:41:53 UTC from IEEE Xplore. Restrictions apply.