Experimental Performance Evaluation

of WebRTC Video Services over Mobile Networks

Mohamed Moulay Vincenzo Mancuso
IMDEA Networks Institute, Madrid, Spain IMDEA Networks Institute, Madrid, Spain
and University Carlos III of Madrid, Spain
[email protected] [email protected]

Abstract—With the rapid growing market for smartphones, basic properties in controlled environments. For instance,
and user’s confidence for immediate access to high-quality the authors of [6] used a cloud-engineered automatic testing
multimedia content, the delivery of video over wireless networks tool for WebRTC, although they have not tested the service
has become a big challenge. WebRTC is a new and evolving
standard that has been developed specifically to meet this offered by mobile operators and core networks. Similarly, the
demand and enable a high-quality experience for mobile users authors of [7] have experimentally tested one-to-many commu-
of real time communication services. However, little systematic nications over WebRTC (namely “simulcast”), although their
experimental studies have been carried out so far to assess the experiments are limited to a gigabit LAN environment.
service experienced by users in a realistic mobile setting. In
In contrast, for our work, we leverage on a large-scale
this work, we describe measurements collected from a WebRTC
implementation operated from real mobile nodes within the pan- on-line measurement platform and focus on users connecting
European MONROE platform. Using data from application and through MBB networks only. Specifically, we use the above
network, in different wireless environments, we experimentally mentioned MONROE platform, which has been designed and
investigate and compare the performance of WebRTC for static is currently operated in the frame of a European project aimed
and mobile cellular users of several networks across Europe.
at providing multi-homed, independent, large-scale monitoring
Index Terms—WebRTC; MBB; Experiments.
and evaluation of performance for mobile broadband networks
in heterogeneous environments. Acquiring access to this plat-
I. I NTRODUCTION
form allows for the deployment of vast measurement setups
Web Real-Time Communication (WebRTC) proposes to to collect data from operational MBB networks in various
easily integrate video services in web browsers, based on local European countries. Differently from other approaches based
tools [1]. In turn, such tools are based on well-known web on operator-driven quality-assessment campaigns [8], [9], or
technologies, able to simply integrate audio, video, and data on traditional drive-by tests [10], MONROE offers an open
transfer operations of the real-time communication protocol platform for repeatable and traceable experiments. Besides, it
(RTC) into a normal webpage. The WebRTC project1 was offers open access to collected data, which refer to multiple
first introduced by Google as an open source project, and then operators, and includes device-level metadata, which is the
other software developers and telecom vendors joined, which key to use and possibly filter results without raising user’s
has led to integration of WebRTC into commercial browsers privacy concerns. Therefore, this platform offers much richer
like Chrome, Opera and Firefox [2], [3]. data than what can be offered by crowdsourcing initiatives
Since offering video services and multimedia channels is like, e.g., Netalyzer [11] and Haystack [12].
a killer application for Mobile broadband (MBB) networks, In this paper, we use the MONROE platform to investi-
WebRTC and similar projects impose stringent quality re- gate session-related performance statistics linked to the use
quirements on such networks, that are nowadays evolving of an off-the-shelf, WebRTC-based streaming application.
from 4G to 5G under the pressure of a steadily increasing This application enables streaming videos in real time with
number of mobile users [4]. Therefore, there is now a strong high quality using web browsers that support WebRTC (e.g.,
need for objective information about MBB performance and Chrome, Firefox) [1]. When using Google Chrome, data from
reliability to support video and multimedia mobile services. both the sending and receiving parties in a WebRTC-based
Thus, various initiatives have arisen, among which the US telemeeting, can be gathered via the WebRTC internals page,2
FCC’s Measuring Broadband America initiative [4] and MON- thus making it possible to get a complete overview of the
ROE [5], to monitor and assess MBB performance. stream in the mobile nodes. Such session-related statistics
We focus on WebRTC performance figures in mobile envi- may help to identify root causes and track the origins of
ronments, for which so far little experimental work exists. In performance issues in video streaming, so to understand how
fact, other works on assessing WebRTC performance figures these technical factors impact Quality of Service (QoS) offered
are currently sprouting, but they have so far only investigated by the network and Quality of Experience (QoE) enjoied by

1 http://www.webrtc.org/ 2 chrome://webrtc-internals/

535
 • GetUserMedia: it provides with agile access to user
media such as microphones, cameras and display.
• RTCDataChannel: it authorizes data transfer through
peer-to-peer channels.
• RTCPeerConnection: it is responsible for setting up a
direct connections between two WebRTC applications,
which allows data channels and media streams to be
carried.
This API represents the base of any WebRTC application.
Besides, it is possible to add plugins to exploit WebRTC or to
empower it with, e.g., encryption and security features [15].

B. Protocols and Communication Services

Web browsers have to support several communication pro-
tocols and communication services to enable WebRTC. As
Fig. 1: WebRTC peer-to-peer communication. shown in Fig. 1, this includes mechanisms such as signaling,
session establishment, transport and security.
the users. Indeed, gathering such insights is crucial and may
steer the development of real-time communication schemes For transport, WebRTC commonly uses UDP with DTLS
and intelligent optimization strategies. Our results show that security, which is a TLS extension for unreliable datagram
current European MBB networks provide static users with transport. However, WebRTC gives the option to use TCP with
enough resources and QoS to suitably make use of WebRTC, TLS as well. In addition, it uses the Stream Control Transport
whereas mobility dramatically worsens network performance, Protocol (SCTP) and the Secure Real-Time Transport (SRTP)
often resulting in unacceptably low levels of QoE. to control media channels and handle encryption keys. SCTP is
In the reminder of the paper, we start by providing back- capable of multiplexing multiple application data streams and
ground on WebRTC in Section II. We present the MONROE provides reliable delivery of UDP datagrams. Hence, a peer-
platform in Section III and our measurement setup in Sec- to-peer secured media path can be established by leveraging
tion IV. Section V focuses on experimental results. Eventually, SCTP and SRTP.
Section VI summarizes the findings of the paper. For what concerns the session setup and the negotiation
of media features and connection parameters, WebRTC uses
II. W EB RTC SIP or XMPP with parameters conveyed through the Session
A. Real-time communications to and from browsers Description Protocol (SDP). In addition, WebRTC uses the
The initial idea behind the development of a mechanism for JavaScript Session Establishment Protocol (JSEP) as session
web-based real-time communication like WebRTC or other setup mechanism, which endorses secured signaling, and en-
proposals discussed in standardization fora was mainly to crypted data transport over either DTLS/UDP or TLS/TCP. It
provide a tool to empower web browsers and make multime- also uses signaling servers that help initiate peer-to-peer mul-
dia communications easier than before [1]. Specifically, the timedia capabilities handshaking and establish connections.
WebRTC approach is based on well-known web technologies WebRTC also includes mechanisms for firewall and NAT
like HTML and JavaScript, which are able to simply integrate traversal. Specifically, it inherits from the VoIP world the
RTC into a webpage. Interactive Connectivity Establishment (ICE) framework. ICE
Supported by Google first, and then by other web browser helps to use TURN/STUN, which in turn eases WebRTC peers
developers (Firefox, Opera) and telecom vendors (Ericsson, to test and collect preferred “candidates of communication”
Cisco, Alcatel-Lucent) [2], WebRTC integrates video stream- options, i.e., it sorts the known transport addresses of a media
ing capabilities into web browsers without the need of in- destination to which is possible to attempt connection. Indeed,
stalling plugins or third-party software. Its standardization— carefully selected communication candidates are the key to
led by W3C and IETF—helps separating application and com- maximize the chance of success.
munication level duties of video services [13]. In particular The most recent specifications on WebRTC have been re-
W3C and IETF are jointly concentrated on defining JavaScript leased in November 2017 with the W3C EditorDraft “WebRTC
Application Programming Interfaces (APIs) and a peer-to- 1.0: Real-time Communication Between Browsers”.3
peer communication mechanisms between browsers, to allow
direct and possibly server-less video communication [14]. III. T HE MONROE M EASUREMENT P LATFORM
The designed API for WebRTC is capable of managing a Since we use MONROE for our WebRTC measurements,
browser based client-side RTC with host-to-browser con- here we present a short overview on the measurement plat-
nection, browser-to-browser connection management, encod- form. The main idea behind MONROE is to deploy hundreds
ing/decoding, NAT traversal and media streaming [14].
The main API highlights are as follow: 3 https://www.w3.org/TR/webrtc/

536
 are managed by a central scheduling system that allows to
deploy and control experiments over multiple nodes at the
same time. Users access the scheduler through a user interface
and public APIs. An AngularJS-based web-portal is available
for such purpose, although users can also access the scheduler
via a Command Line tool (CLI). In any case, platform users
need to obtain a user certificate first, which is the same as
per FIRE platforms and experimental federations that align
with the Fed4FIRE initiative of the European Commission
Fig. 2: A MONROE node pair with three LTE Cat6 modems and for a distributed, heterogeneous and large-scale experimental
one WiFi adapter. network platform.8 Indeed, as a consequence, concerning the
)*#+%011#**%'$2%-13#2456$/% 7'$'/#8#$(%'$2%7'6$(#$'$1#% scheduling APIs, the MONROE scheduling is compatible with
other Fed4FIRE compatible interfaces.
Node

Core Containers for !"#$%&'' 9.$('6$#+%:.+%%

Components Default Experiments ()!)*%&%+,' )*#+%;<"#+68#$(*% Maintenance and Management: This block is a subsystem
used by the maintenance team to administer and maintain
MONROE nodes with ease. It relies on a DB with a friendly
Back-End

7'6$(#$'$1#% Measurement 8=5'$#% D#8"%

A%!"#+'@.$*%
&B%
DBs Responders +#".*6(.+C% +#".*6(.+C% interface to check node connectivity status and stats.
Node: Measurement nodes are dual-apu2 machines with
a Linux Debian based operating system. They include core
MONROE 8=5'$#% )*#+,*% components (e.g., for handling routing and connectivity, for
Visualization >6*4'56?'@.$% -(.+'/#%
monitoring the software behavior, etc.) plus a set of Linux
!"#$%
&'('%
Docker9 containers in which experiments are trigged in iso-
User’s end lation. Moreover, some containers run constantly in the back-
Fig. 3: MONROE platform components and information flow. ground to collect results from “default” experiments, e.g., to
collect statistics on bandwidth throughput, delay, and gather
of measurement points (namely, MONROE nodes) across Eu- metadata such as node GPS coordinates. The main idea
rope to support multiple research oriented experiments and to behind using containers is that it allows for handy control
collect, analyze, visualize and share the measurements. Nodes of multiple complex components. Besides, it enables nippy
are distributed in different places such as in labs, research reconfiguration, which can help to implement other containers
institutes, universities, buses, trucks, and trains across Spain, to the need of other experiments.
Italy, Norway, and Sweden. Thus, MONROE nodes operate as Back-end: MONROE incorporates several repositories in
MBB static or mobile users, with their off-the-shelf or custom the back-end, for maintenance and operation of the platform,
applications. Fig. 2 shows the hardware of a MONROE node, for collecting metadata and results and temporarily store user
which consists of a dual mini-motherboard system, using apu2 data, and also to integrate and import mPlane data analytics
boards.4 One of the two boards, namely the one in the “head” computed on the traffic observed by the MONROE nodes.10
node, mounts two Sierra Wireless MC7455 miniPCI express In particular, default experiment results and metadata are col-
modems,5 while the other board (in the “tail” node) has one lected in a non-relational database using Apache Cassandra,11
MC7455 modem and a WiFi card. The modems are 4G devices which follows an experiment-oriented design.
supporting the most recent LTE category an industry-grade Visualization and open data: Metadata and default ex-
modem could provide at the time of node design (CAT6). More periment results can be freely visualized in near-real time.12
details on the node hardware and design can be found in [5]. Specifically, MONROE offers a user interface that has been
The platform is open and meant to provide experiments as designed to provide a graphical representation of node de-
a service, so that all hardware specifications, open software, ployment, their status, as well result collection of selected
user manuals and experiment templates are made available experiments. Measurement data will be openly released.
on github,6 while scientific publications and data reports are
As shown in Fig. 3, users can fetch the data generated
accessible and searchable from the MONROE webpage.7
by their experiments from a temporary repository in the
A. MONROE Components MONROE back-end. This is made available through the very
The MONROE architecture is illustrated in Fig. 3 and same user interface that allows to schedule experiments.
consists of several blocks, as described in what follows.
User Access and Scheduling: It is not possible to login and 8 http://www.fed4fire.eu/
run live experiments from a node. Instead, all measurements 9 http://www.docker.com
10 mPlane (http://www.ict-mplane.eu) is a measurement platform designed
4 PC Engines, apu2 platform: https://www.pcengines.ch/apu2.htm for fixed networks in the frame of a project funded by the European
5 MC7455 miniPCI express (USB 3.0) modem: https://www.sierrawireless. Commission, and which has been extended by MONROE to account for the
com/products-and-solutions/embedded-solutions/products/mc7455/ analysis of traffic at mobile nodes.
6 https://github.com/MONROE-PROJECT 11 http://cassandra.apache.org
7 https://www.monroe-project.eu/resources/papers/ 12 http://visual.monroe-system.eu/

537
 Design Certification Experimentation TABLE I: MONROE nodes throughput comparison
Node ID Location Download [Mb/s] Upload [Mb/s]
2/4"567"'/0&,.$&"*' 2/4"567"'
!"#$%&'"()"*$+"&,' 423 Telia SE 0.178-10.38 0.165-1.58
$&'.',"#8&%'&05"' "()"*$+"&,'
428 Telia SE 0.53-13.97 0.25-2.4
-*".,"'/0&,.$&"*'10*' !")703+"&,'.&5' 429 Telenor SE 0.78-12.178 0.22-1.74
9"#,':'!";6%'
"()"*$+"&,' "("/680&' 501 Telenor SE 36.43-68.12 14.93-22.32
591 Vodafone ES 24.58-72.64 17.36-23.14
2,0*"'/0&,.$&"*'$&' <;,.$&'=<>?<@'
?",*$"B.7'01'*"#67,#' 596 Vodafone IT 32.78-65.41 14.21-25.10
*")0#$,0*3' /"*8A/.80&'

Fig. 4: Experimentation workflow using the MONROE platform. Internet and the access network of our lab, which uses a multi-
gigabit connection. Hence, the bottleneck of the stream is in
Remarkably, the measurement structure is not only capable the cellular connectivity experienced by the MONROE node.
of monitoring and analyzing the network behavior in real- The MONROE platform provided us with many static and
time by scheduling user-defined Docker containers, but also mobile nodes. Specifically, for the experiments reported in this
to cache measurement and metadata for offline analysis. paper, we have used the nodes listed in Table I. The table
B. MONROE Experiment Life-cycle includes some limited yet important information about the
location of the node. Indeed, we only report operator name and
The workflow of MONROE experiments in depicted in
country, whereas more detailed pieces of information about
Fig. 4. Initially, experimenters have to specify what kind of
the locations are omitted since this study does not serve as a
measurements they want to do and design them in Docker
thorough survey of operator’s performance comparison. The
containers, that includes various software types. This is the
examples of results reported here are only meant to highlight
design phase depicted in the figure, which includes creating
how the potentials of WebRTC have not been fully unleashed
and storing a Docker container in a github repository.
by the sub-optimal MBB service available in many places
Secondly, containers have to be certified before going live.
in Europe as of today. However, as we will see later, MBB
This is the central block in the figure. Specifically, containers
operators already offer WebRTC-ready connectivity for static
have to be tested and, to this purpose, they can be run in special
users, while mobility is still a big issue.
nodes devoted to debugging, to which users can connect
The WebRTC throughput offered by MONROE nodes—i.e.,
directly and tune the experiments to respect safety and stability
the one offered by the MBB operator used at the node—
rules. In any case, a container has to be eventually certified
differs a lot across space and operators, as well as across
by the MONROE maintenance team.
time, as shown in the third and fourth columns of Table I.
Finally, as shown in the rightmost part of the figure, the
Such throughput values have been collected with capacity
users deploy their containers into normal nodes through a user
experiment run a few minutes before and after running the
interface or the CLI to the scheduler that runs in the backend.
WebRTC experiments.
The scheduler offers both Fed4FIRE-compatible and REST
To generate the figures reported later, we have used the
APIs, and connects to nodes to sense their availability and
data collected by Google Chrome on peer-to-peer connections,
health conditions before making and enforcing any scheduling
which include WebRTC streams. The resulting logs contain,
decision. The user interface, in addition to help to select types,
per each individual stream, the timing and headers of packets
number of nodes, and suitable time-slots, also helps users to
received as well as the timing of various internal events such
collect passive and active results from multiple nodes. Indeed,
as received frames, losses, bitrate, delay and jitter.
after successfully finishing the experiments, experimenters
can collect their data directly from the user interface along V. R ESULTS AND O BSERVATIONS
with metadata, e.g., GPS, throughput, and other pieces of In this section we report statistics on WebRTC performance
background information on the nodes used in the experiment. attributes, as observed by means of Chrome internals at
IV. M EASUREMENT S ETUP the destination of the WebRTC streams. We run WebRTC
The measurement setup we have used consists in a Docker streamers in static nodes across Italy, Spain and Sweden,
container with a WebRTC streamer and an IP tunnel handler and on mobile nodes in Sweden. We only report a small yet
that makes available a multimedia file over HTTPS. The indicative subset of the results we have collected. For each
container then includes a light implementation of WebRTC experiment, we plot the following statistics: (i) BitRate, (ii)
and we have made it publicly available for experimenters frames per second (FPS) rendered at the receiver, (iii) packet
willing to design their own tests.13 When the experiment delay and (iv) jitter. We compare static and mobile cases.
is scheduled on a set of machines, each of them makes a Static case. In the first scenario that we consider, the
link available for connecting and watching the multimedia streamer and the client have high quality connectivity (nodes
file using a Chrome browser acting as WebRTC client. We 501, 591, and 596 in Table I). The selected MONROE nodes
connect the WebRTC client from a computer in our lab to a were connected to various 4G operators and, at the other end of
MONROE node using such link. Therefore, the stream goes the connection, we use a computer connected to a gigabit LAN
through the cellular uplink of the MONROE node, crosses the directly connected to a multi-gigabit metropolitan network.
In most of the test cases, the media stream turned out to
13 docker.monroe-system.eu/deployed/monrtc be smooth, the final user having a rather good experience

538
 !*$#") !("""""
89:;<9=>!>? 46789!:6
89:;<9=>!@6 4676;<,!:6
!)$#") 6>A>=9.!?> !'"""""

!($#"
+,-./-0!12345

)*+,-+.!/0123
!&"""""
!'$#"
!%"""""
!&$#"
!$"""""
!%$#"

!#$#" !#"""""

!" !"
!" !(" !#"" !#(" !%"" !%(" !&"" !&(" !" !'" !#"" !#'" !$"" !$'" !%""
6,70!145 4*5.!/23

(a) Video bitrate. (a) Video bitrate.

!'& !#"
<=>?*=@A!A1 8:;<=!1:
!'% <=>?*=@A!B8 !'& 8:;:>?@!1:
*+,-./0.+1.2345!6*017

*+,-./0.+1.2345!6*017
8ACA@=D!1A
!'$ !'%
!'$
!'#
!'#
!'"
!'"
!&
!&
!%
!%
!$ !$
!# !#
!" !"
!" !(" !'"" !'(" !#"" !#(" !)"" !)(" !" !(" !'"" !'(" !#"" !#(" !)""
89-.!:/; 89-.!6/7

(b) Video frame rate. (b) Video frame rate.

!'&# !(""
56*7869:!:; 24567!84
56*7869:!<3 24549:;!84
!'%# 3:=:96>!;: !'""
!'$#
!&""
*+,-.!/012

)*+,-!./01
!'"#

!'## !%""

!&#
!$""
!%#
!#""
!$#

!"# !"
!# !(# !'## !'(# !"## !"(# !)## !)(# !" !'" !#"" !#'" !$"" !$'" !%""
340+!/12 23/*!.01

(c) Packet delay. (c) Packet delay.

!'"" !(""
67-897:;!;< 6789:!;7
67-897:;!=5 6787<=>!;7
!&#" 5;>;:7?!<; !'""
()**+,!-+./0!1234

)*++,-!.,/01!2345

!&""
!&""
!%#"

!%"" !%""

!$#"
!$""
!$""
!#""
!#"

!" !"
!" !#" !$"" !$#" !%"" !%#" !&"" !&#" !" !'" !#"" !#'" !$"" !$'"
5)2+!134 6*3,!245

(d) Jitter delay. (d) Jitter delay.

Fig. 5: WebRTC performance experienced by static nodes under Fig. 6: WebRTC performance figures observed for a bus in a
different operators in different countries. medium-size city in Sweden.

with typical bitrates of a few Mb/s, frame rates often above rate was very variable, and delay and jitter were annoyingly
ten frames per second and latency in the ballpark of 100 high in all locations and for all operators. We have observed
ms, which is below the threshold of human perception. Jitter many other cases like this in our measurements (and not only
was normally below 100 ms. Fig. 5 illustrates an example in Sweden, of course), which leads to the conclusion that
of performance figures over time for three simultaneous con- current MBB networks are not ready to fully support WebRTC
nections using MONROE nodes in three countries. Of course (and alike multimedia services) on the move.
each connection shows different performance figures, and in To further understand the behavior of WebRTC on MBB
particular the Spanish sample shows the worst behavior in networks, Fig. 7 shows 3D plots of bitrate and delay ex-
terms of rate and delay/jitter, although we need to mention perienced by a MONROE node on a bus, for one of the
that the selected MONROE node in Spain was using an Italian operators in Sweden, as a function of the geographical position
SIM card operating in roaming. In any case, the observed of the node (also shown in the topmost subfigure using Google
performance is acceptable. Earth). The information about coordinates is provided by the
Mobile case. In this scenario we select MONROE nodes in MONROE node itself, within a metadata stream generated
buses in Sweden (nodes 423, 428, and 429 in Table I), while during the experiments. From the figure it is clear that some
the other end of the connection is in our lab, as in the other areas crossed by the bus were very poorly served, so that
case. Fig. 6 shows that the user experience was not as smooth delay are constantly high and bitrates change smoothly over
as in the first case since the bitrate was significantly degraded. the trajectory, indicating that coverage (signal strength) is far
Although WebRTC was able to keep the stream going, frame from optimal in that area.

539
 platform. To this aim, we have designed an open tool, running
in a Docker container, for generating WebRTC sessions in
mobile nodes by using standard WebRTC APIs. As such,
the work presented a complete and novel methodology for
the performance evaluation of web services using operational
MBB networks. As an initial result, we have observed that
mobility is still an important challenge for WebRTC, since
MBB operators do not yet provide with full quality coverage
for users on the move. Our approach can be used in the future
for a broader and continuous assessment of WebRTC.
(a) Bus route.
ACKNOWLEDGMENTS
4.56750!18923 Work funded by the EU H2020 research and innovation
!%D(*% programme under grant agreement No. 644399 (MONROE).
!,*** !"D(*%
!)"** !,D(*%
The work of V. Mancuso was supported by the Ramon y Cajal
!)***
grant (ref: RYC-2014-16285) from the Spanish Ministry of
-./0!123

!+"** !)D(*%
!+***
!("**
!+D(*% Economy and Competitiveness.
!(D(*%
!(***
!"** !* R EFERENCES
!*
[1] A. Zeidan, A. Lehmann, and U. Trick, “WebRTC enabled multimedia,”
!"#$"
!"#$"" in in proceedings of World Telecommunications Congress 2014, Jun.
!"#$%
!"#$%" 2014.
!()$*' !"#$&
!()$(
!()$(+!()$(, !"#$&" [2] A. Johnston, J. Yoakum, and K. Singh, “Taking on WebRTC in an
!()$(% !"#$'
:>AB.5;<0!C725
!()$('
!()$+ !"#$'" :75.5;<0!=>?5@ enterprise,” IEEE Communications Magazine, vol. 51, no. 4, pp. 48–
!()$++
!()$+,
54, April 2013.
(b) Bitrate. [3] S. Loreto and S. P. Romano, “How Far Are We from WebRTC-1.0? An
Update on Standards and a Look at What’s Next,” IEEE Communications
Magazine, vol. 55, no. 7, pp. 200–207, 2017.
40567!1/23
[4] FCC, “2013 Measuring Broadband America February Report,” FCC’s
!#**
!'**
Office of Engineering and Technology and Consumer and Governmental
!,***
!)"**
!&** Affairs Bureau, Tech. Rep., 2013.
!%**
!)*** !"**
[5] O. Alay, A. Lutu, M. Peón-Quirós, V. Mancuso, T. Hirsch, K. Evensen,
-./0!123

!+"** !,** A. Hansen, S. Alfredsson, J. Karlsson, A. Brunström, A. Safari Kha-

!+*** !)**
!("** !+**
touni, M. Mellia, and M. Ajmone Marsan, “Experience: An Open
!(*** !(** Platform for Experimentation with Commercial Mobile Broadband Net-
!"** !*
!*
works,” in Proc. of ACM Mobicom., 2017.
[6] B. Garcia, F. Gortazar, L. Lopez-Fernandez, M. Gallego, and M. Paris,
!"#$"
!"#$""
“WebRTC Testing: Challenges and Practical Solutions,” IEEE Commu-
!"#$%
!"#$%"
nications Standards Magazine, vol. 1, no. 2, pp. 36–42, 2017.
!()$*'
!()$(
!()$(+!()$(,
!"#$&
!"#$&"
[7] B. Grozev, G. Politis, E. Ivov, T. Noel, and V. Singh, “Experimental
!()$(%
!()$('
!()$+
!"#$'
!"#$'" 869.9:;0!<=>9? Evaluation of Simulcast for WebRTC,” IEEE Communications Standards
[email protected]:;0!B629 !()$++!()$+,
Magazine, vol. 1, no. 2, pp. 52–59, 2017.
(c) Packet delay. [8] M. Z. Shafiq, L. Ji, A. X. Liu, J. Pang, S. Venkataraman, and J. Wang, “A
First Look at Cellular Network Performance during Crowded Events,”
Fig. 7: WebRTC video performance measured at destination, on a in Proc. of SIGMETRICS, 2013.
public bus on service in a medium-size city in Sweden. [9] J. Huang, F. Qian, Y. Guo, Y. Zhou, Q. Xu, Z.-M. Mao, S. Sen, and
O. Spatscheck, “An In-depth Study of LTE: Effect of Network Protocol
From the above results, it is clear that peer-to-peer commu- and Application Behavior on Performance,” in Proc. of SIGCOMM,
2013.
nication over the Internet is still not “ideal” and whether one [10] Tektronix, “Reduce Drive Test Costs and Increase Effectiveness of 3G
gets good QoE or not for the whole communication period Network Optimization,” Tektronix Comm., Tech. Rep., 2009.
completely depends on how good the network connection is, [11] C. Kreibich, N. Weaver, B. Nechaev, and V. Paxson, “Netalyzr: Illu-
minating the edge network,” in Proc. of the 10th ACM SIGCOMM
and how good it remains across the whole time of the com- conference on Internet measurement, 2010, pp. 246–259.
munication session. In the ideal WebRTC scenario, endpoints [12] N. Vallina-Rodriguez, “Illuminating the Third Party Mobile Ecosystem
are web browsers running on reasonably powerful laptops with with the Lumen Privacy Monitor,” in FTC PrivacyCon 2017, January
2017.
strong WiFi or wired network connections, communicating on [13] A. Amirante, T. Castaldi, L. Miniero, and S. P. Romano, “On the seam-
top of a reasonably consistent network. This should work well. less interaction between WebRTC browsers and SIP-based conferencing
However, if the devices are mobile and have a non-consistent systems,” IEEE Communications Magazine, vol. 51, no. 4, pp. 42–47,
April 2013.
and often not good wireless connection, then the quality of the [14] E. Bertin, S. Cubaud, S. Tuffin, N. Crespi, and V. Beltran, “WebRTC,
communication is likely to be below any acceptable threshold, the day after: What’s next for conversational services?” in 2013 17th
as observed in the mobile case described above. International Conference on Intelligence in Next Generation Networks
(ICIN), Oct 2013, pp. 46–52.
VI. C ONCLUSIONS [15] A. Amirante, T. Castaldi, A. Gouaillard, L. Miniero, S. G. Murillo,
and S. P. Romano, “Bringing privacy to the Janus WebRTC server:
In this work, we have evaluated the performance of We- The PERC way,” in 2017 Principles, Systems and Applications of IP
bRTC for static and mobile users by leveraging the MONROE Telecommunications (IPTComm), Sept 2017, pp. 1–8.

540