This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 1

Mobile Edge Computing Resources Optimization:

a Geo-clustering Approach
Mathieu Bouet, Vania Conan

Abstract—Mobile Edge Computing (MEC) is an emerging

technology that aims at pushing applications and content close
to the users (e.g., at base stations, access points, aggregation
networks) to reduce latency, improve quality of experience, and
ensure highly efficient network operation and service delivery.
It principally relies on virtualization-enabled MEC servers with
limited capacity at the edge of the network. One key issue is to
dimension such systems in terms of server size, server number,
and server operation area to meet MEC goals. In this paper, we
formulate this problem as a mixed integer linear program. We
then propose a graph-based algorithm that, taking into account
a maximum MEC server capacity, provides a partition of MEC
clusters, which consolidates as many communications as possible
at the edge. We use a dataset of mobile communications to
extensively evaluate them with real world spatio-temporal human
dynamics. In addition to quantifying macroscopic MEC benefits,
the evaluation shows that our algorithm provides MEC area
partitions that largely offload the core, thus pushing the load
at the edge (e.g., with 10 small MEC servers between 55% and
64% of the traffic stay at the edge), and that are well balanced
through time.
Index Terms—Mobile edge computing, Multi-access edge com- Fig. 1. MEC deployment: tasks and applications (e.g., video chat analytics,
puting, fog computing, network virtualization, dimensioning, video chat customization) are mainly offloaded onto MEC servers at the edge
clustering. of the network to reduce latency and offload the core network. The area is thus
partitioned into MEC clusters, each cluster being served by a MEC server.
Note that it can be an n-level architecture.
I. I NTRODUCTION
OBILE Edge Computing (MEC - also know as Multi-
M access Edge Computing [1], and similar to fog com-
puting [2]) has emerged as a key enabling technology for
efficiency of a MEC system heavily depends on such aspects
as the distribution of communications and workloads in time
realizing the IoT and 5G visions. It aims at reducing la- and space. Its cost depends on server density, capacity, and
tency and ensuring efficient network operation and service interconnection. Imbalanced MEC clusters that handle highly
delivery and pushing content and services close to the users. different traffic volumes would lead to an inefficient use of
Numerous MEC applications are already envisioned and inves- resources and to unequal Quality of Experience (QoE). Put in
tigated, for example: chat/video analytics, video acceleration, terms of MEC clustering, the key question is thus: how to have
augmented/virtual reality, location-based services, connected an efficient partition of MEC areas? From this, a placement
vehicles, and IoT gateways [3]. of MEC servers can be derived.
In a MEC deployment MEC servers are positioned in the The problem is aggravated by the well documented fact that
infrastructure close to the edge of the network (see Fig. 1): they mobile traffic is very dependent on time and locality. Indeed
are small-scale datacenters with low to moderate resources mobile communications are spatially distributed according to
collocated with the base stations, access points and/or placed the population density and activity, which vary in time. For
in the access/aggregation network. They leverage virtualization instance, the mobile traffic in the business areas differ from
to support MEC applications run as virtual machines, con- the mobile traffic in the transport, residential and entertainment
tainers, microservices etc. [4]. The purpose of MEC servers areas [7]–[9]. As it was shown by Qazi et al. [10], in a
is to host as many applications as possible at the edge to MEC perspective, such variations have a direct impact on
improve latency and alleviate congestion in the core. MEC the load of the potential MEC servers. In addition, it was
thus performs application and network offloading from the shown by Tastevin et al. [9] that mobile communications in an
core data center on to the edge [5], [6]. urban environment have a high spatial locality - they tend to
The central decision in a MEC system design is to decide follow a power law, which motivates a local consolidation of
which users, applications and share of traffic should be handled applications at MEC servers. Such properties will be amplified
by the MEC servers. To address this key issue we name with the realization of the IoT and 5G visions [3].
MEC cluster the area, and by extension the base stations and In this paper, we formally describe the MEC geo-clustering
the users in the area, served by a MEC server. Indeed the problem and provides a Mixed Integer Linear Programming

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 2

(MILP) formulation. However, as the large-scale dimension who showed that the number and the locations of MEC servers
of MEC systems and mobile communications makes classic have a direct impact on the QoE (imbalance loads and high
analytical and simulation-based approaches almost inapplica- latencies) and on the operational cost. However, they did
ble, we then investigate a graph-based method. We propose not address the placement problem. They proposed an NFV-
an algorithm that, based on the spatial distribution of the based orchestration for MEC. Note that the server placement
communications, finds a MEC partition that favors application problem is significantly different from the conventional base
instantiation at the edge instead of at the core. The resulting station site selection problem since, although both problems
clusters correspond to MEC areas. Our algorithm takes into are constrained by the deployment budget, placing edge sites is
account the maximum server capacity, that we express as coupled with the computational resource provisioning. Ceselli
the maximum number of served communications per unit of et al. [15] have proposed a mixed integer linear programming
time, but that can be easily expressed in terms of resources formulation of the joint problem of base stations allocation to
(CPU, storage...) or application instances. We evaluate the MEC servers and routing to reduce infrastructure cost. Our
MILP and the clustering algorithm using a dataset of mobile proposal mainly differs on three important aspects. First, they
communications in a city provided by a mobile operator. We assume the locations of the LTE 4G base stations are known.
first show that the clustering takes into account the spatial Their analytical formulation does not scale properly. Most of
distribution of the communications and enables to largely all, the clusters they obtained are not geo-consistent, meaning
offload the core. In addition, the algorithm provides results that that the base stations associated to a MEC server can be
are close to the MILP results on small-scale problem instances. completely scattered in space.
Then, we evaluate the clustering algorithm on larger problem Control plane design. Recent proposals have addressed
sizes and outline the benefits of MEC even for very small MEC control plane design. They investigate how the current cen-
server sizes. The obtained MEC clusters have well balanced tralized LTE core architecture, where most of the traffic
loads and enable to keep a large portion of the traffic at the converge [16], can be decomposed and split to alleviate con-
edge. Finally, we evaluate the MEC partition over a week of gestion and reduce latency [17]. Software-Defined Networking
communications and show that it largely supports temporal (SDN) is used to redirect peering traffic in-between the base
dynamics. There is almost no server saturation, i.e., traffic stations and the Evolved Packet Core (EPC), thus offloading
offloaded to the core, while the loads remain balanced. the core and improving latency [5]. SDN is also combined
In summary, this paper makes the following contributions: with NFV (Network Functions Virtualization) to propose a
1) We formulate the MEC clustering problem and provide backwards-compatible orchestration architecture where virtual
a MILP formulation (Sec. III). EPC functions are chained with SDN and instantiated in MEC
2) We design a MEC clustering algorithm (Sec. IV) that servers to efficiently use resources [10].
consolidates as many communications as possible at the System approaches. While NFV has gained momentum,
edge. recent proposals have focused on shortening network func-
3) We use a real-world dataset of spatially and temporally tions instantiation and reducing their system footprint with
distributed mobile communications (Sec. V-A). approaches based on unikernels [4]. In particular, it has
4) We evaluate our proposal and show that, despite the been shown that an inexpensive commodity server is able to
spatialtemporal dynamics of the traffic, our algorithm concurrently run up to 10,000 specialized virtual machines,
provides well-balanced MEC areas that are close to instantiate a VM in as little as 10 milliseconds, and migrate
optimal on small problem instances (Sec. V-B) and it in under 100 milliseconds [18]. This technology is very
serve a large part of the communications on real-world promising in an MEC context where an application could be
problem instances (Sec. V-C and Sec. V-D). instantiated on the fly at MEC servers for a user or a group
We discuss related work in Sec. II and conclude in Sec. VI. of users and shutdown once the communication is ended.
In [11] we presented the geo-clustering algorithm and a first Progress in this direction complements our deployment work
evaluation. In this paper, we introduce a mixed integer linear as it would make it easier to instantiate locally applications at
programming formulation of the problem and a formalization MEC servers.
of the algorithm. We also evaluate both the MILP and the Application/task and content offloading. Application of-
algorithm through extensive and detailed simulations. floading, both from the device to the edge and from the core
to the edge, has been extensively studied. It notably includes
II. R ELATED W ORK task decomposition and packaging [19], assignment, and mi-
In the past few years, in parallel notably to the ETSI MEC gration [6], server scheduling and selection, content caching
ISG initiative [1] and to the OpenFog Consortium [2], MEC and pre-fetching [20]. Some of the proposals are similar to
has emerged as a new promising research area. Very recently, those addressed in Mobile Cloud Computing (MCC), which
first surveys have been published to present comprehensive addresses distributed clouds [21], [22].
panoramas of the use cases, architectures and challenges [12]–
[14]. We present in this section challenges and related work
III. MEC RESOURCES CLUSTERING
that are linked with the problem of MEC resources dimen-
sioning. In this section, we formulate the MEC resource geo-
Dimensioning and MEC server placement. The MEC clustering problem that we address and we present the cor-
server placement problem was illustrated by Qazi et al. [10] responding mathematical optimization model.

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 3

A. Problem formulation TABLE I

P ROBLEM DATA AND DECISION VARIABLES .
From a network system standpoint we consider a MEC
deployment as presented in Fig. 1. All users belong to a Input (problem data)
MEC cluster, a geographic area whose traffic can be han- N Number of cells after area discretization
G Set of N cells
dled by a MEC server, that is a small-scale datacenter with N eigh(i) The set of the cells that are spatially neighbors of the cell i
low to moderate compute and storage resources. All user C Set of clusters (clustered cells)
communications and applications, for instance ephemeral per- M Maximum cluster capacity
ti, j Amount of traffic or communications per unit of time from
communication unikernel-based video analytics applications, the cell i and to the cell j
are either handled by the local MEC server (e.g., the blue plain x i , yi Integer variable, define the x and y coordinates of the cell i
line in Fig. 1) or by a highly capacitated core data center (e.g., in a grid
Output (decision variables)
the black dotted line in Fig. 1), which can be farther in terms
ai, c Binary variable, equal to 1 iff the cell i belongs to the cluster
of latency. c
We argue that one of the key design issues in a MEC system bi, j, c Binary variable, equal to 1 iff the cell i and the cell j are in
is to efficiently dimension MEC areas (or clusters). Such a the same cluster c
fi,srj, c, d st
Float variable in [0, 1], define the fraction of flow between
MEC geo-partitioning must have the following properties: c
the cell i and the j for a cluster c when the cell sr c
1) MEC servers, as any compute, storage and network and the cell dst are the source and destination of the flow
node, have a maximum capacity (e.g., in terms of CPU, respectively
storage resources or application hosting capabilities) that
we considered as known a priori in our problem.
2) MEC server loads should be balanced both spatially We introduce two sets of binary variables. The first one
and temporally to improve user experience and system corresponds to cell-cluster attachment variables: ai,c take value
expenditures. 1 if the cell i ∈ G is in cluster c ∈ C. The second one is a set
3) The traffic between the MEC servers and the core should of intermediary variables: bi, j,c take value 1 if the cell i ∈ G
be minimized, in particular by consolidating applications and the cell j ∈ G are in the same cluster c ∈ C. Finally,
at the MEC server level, such that the global latency is since we want clusters that are geo-consistent, meaning that all
reduced. their cells are connected, we introduce a set of float variables,
This problem turns out to be both theoretically and com- noted F, for the commodity flow formulation that will ensure
putationally hard. It generalizes the graph cut based image connectivity: f i,srj,c
c,dst
∈ [0, 1] define the fraction of flow
segmentation problem with connectivity constraints, which is between i ∈ G and j ∈ G for cluster c ∈ C when src ∈ G and
NP-hard [23], and introduces capacitated components. dst ∈ G are the source and destination of the flow respectively.
In the following, we formulate the mathematical optimiza- Objective function.
tion formulation of the MEC geo-clustering problem and intro- XXX
duce alternative connectedness constraints that are restrictive Maximize t i, j bi, j,c (1)
but allow reducing the number of constraints. i ∈G j ∈G c ∈C
Note that the following problem formulation focuses on
edge-to-edge communications to be consolidated at the MEC Constraints.
servers. However, it could be easily extended to address edge- Cluster attachment unicity:
to-core communications. X
ai,c = 1, ∀i ∈ G (2)
c ∈C
B. Model
Input (problem data). Intermediate variable: the cells i and j belong to the same
We assume that the considered area has been discretized in cluster c
N cells. Let note G the set of the N cells. For example, if the 0 ≤ ai,c + a j,c − 2 bi, j,c ≤ 1,
discretization has been done according to a grid of length n, (3)
∀i ∈ G, j ∈ G, c ∈ C
then G = {0, ..., n2 −1}. For a clearer constraint formulation, let
note N eigh(i) the cells that are spatially neighbors of the cell Maximum cluster capacity (intra cluster communications):
i. In a grid, a cell that is not on a boarder has 8 neighbor cells. XX
t i, j corresponds to the amount of traffic or communications per t i, j bi, j,c ≤ M, ∀c ∈ C (4)
unit of time from the cell i ∈ G to the cell j ∈ G. The goal is i ∈G j ∈G
to cluster the area cells. We note C the set of clusters. C is a
partition of the set G. By default, C is equal to G, which means Commodity flow constraints to ensure that a cluster c is
that the discretized parts are all in a unique cluster. The aim connected (geo-consistent):
being to cluster cells, a solution of the program might result in (i) For a cluster c, a flow between i and j exists if and only
a number of empty clusters. A cluster has a maximum capacity, if i and j are in the same cluster c:
in terms of traffic or communications per unit of time that can
be processed at its MEC server, noted M. f i,srj,c
c,dst
≤ bi, j,c,
(5)
Output (decision variables). ∀src ∈ G, dst ∈ G, i ∈ G, j ∈ G, c ∈ C

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 4

Fig. 2. Visualization of the steps of our graph-based algorithm. The area where are distributed the MEC communications is discretized into nodes which form
MEC clusters. Each pass is made of two phases: one where the pair of neighbor nodes (i.e., clusters) that interact the most, while respecting the maximum
cluster capacity, are selected; one where the two selected nodes (i.e., clusters) are merged to build a new/updated graph with an increased self-loop weight
meaning that more communications or traffic are in the same cluster. The passes are repeated iteratively until no pair of neighbor nodes (i.e., clusters) can be
merged because of the maximum MEC server capacity (self-loop weights). The result corresponds to a spatial partition of MEC clusters.

(ii) Flow conservation on transit cells: have one of their neighbor cells that are connected in the same
X
i, j
X
i, j cluster.
f k,l,c − f l,k,c = 0, when i = src, j = dst Note that these connectedness constraints are more restric-
l ∈N eigh(k) l ∈N eigh(k)
tive than the commodity flow constraints since they impose
∀i ∈ G, j ∈ G, k ∈ G \ {i, j}, c ∈ C that the path between two cells is strictly inside the rectangular
(6) they define on a grid.
(iii) Flow conservation at source and destination cells:
X
sr c,dst
X
sr c,dst IV. G RAPH - BASED G EO - CLUSTERING A LGORITHM
f i,l,c − f l,i,c =
l ∈N eigh(i) l ∈N eigh( j) In this section, we present our graph-based algorithm for
 bi, j,c

 if i = src (7) MEC area-geo-clustering. We first explain how it works and
 −bi, j,c if j = dst, i , j then present its formal description.
 Given a maximum MEC server capacity, the algorithm finds
∀src ∈ G, dst ∈ G, i ∈ G, j ∈ G, c ∈ C MEC clusters (also referred to as MEC areas) which tend to
Alternative connectedness constraints. maximize the traffic handled inside the clusters (i.e. at the edge
The number of constraints explodes very rapidly with the by the MEC servers) and thus reduce the traffic that goes up
number of cells generated by the discretization of the space. to the core data center.
We thus define alternative connectedness constraints that can It is divided in two phases that are repeated iteratively.
be substituted for the flow commodity formulation to reduce Assume that we start with two graphs that have the same
the number of constraints when the space discretization struc- set of nodes (see Fig. 2). These nodes correspond to the
ture is a grid and thus manage larger problem instances. discretization of the area where the MEC communications
On a grid, the cells i and j, whose coordinates are (x i, yi ) demands are distributed into clusters. We note it C. The first
and (x j , y j ) respectively, are connected within the cluster c if graph G a = (C, Ea ) represents the adjacencies of the nodes
it exists at least |x i − x j | + |yi − y j | + 1 cells of the same cluster on the area. For instance, in a square grid, a node (a grid
in the rectangle they form on the grid: cell) has up to 8 adjacent nodes (grid cells). The second
graph Gint = (C, Eint ) represents the interactions (i.e., the
(|x i − x j | + |yi − y j | + 1) bi, j,c ≤ communications or the traffic) between the nodes. The weight
X wi, j ∈ R of the edge ei, j ∈ Eint represents the amount of
bi, j,c if i , j
(8) interaction (e.g., the number of communications or traffic)
k ∈[min(xi ,x j ),max(xi , x j )]
l ∈[min(yi ,y j ),max(yi ,y j )] between node i and node j. Note that a node i can have
∀i ∈ G, j ∈ G, c ∈ C interaction with itself, leading to a self-loop ei,i - this is
actually desired as it corresponds to communications that
These constraints can be seen as working recursively. Two are inside the corresponding MEC cluster. So, in this initial
cells are connected in a cluster if and only if at least they each partition there are as many MEC clusters as there are nodes.

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 5

Algorithm 1 Geo-clustering of MEC resources two nodes are clustered at each iteration. Third, it considers
Require: Graph of cluster adjacencies G a (C, Ea ): undirected a maximum clique/cluster capacity, that corresponds to the
graph where C is the set of clusters weight of the self-loops. Note that this threshold we introduce
Graph of cluster interactions Gint (C, Eint ): undirected is adequate to our partitioning problem. However, it could be
edge-weighted graph where Ea ⊆ Eint and ei, j ∈ removed from our algorithm, making it a similar yet different
Eint , i, j ∈ C has a weight wi, j ∈ R hierarchical clustering algorithm than community detection
Maximum cluster capacity: M, meaning that wi, j ≤ M algorithms. Finally, we purposely present a simple description
Ensure: G a (C, Ea ) and Gint (C, Eint ) of the algorithm, but heuristics may be introduced to improve
1: repeat its performance or introduce variants (e.g., order the edges in
2: Select the two adjacent clusters that have the highest the first phase, consider more complex interactions such as
interaction weight: group communications, perform a local search on the final
i, j ∈ C such that: result, consider different maximum cluster capacities etc.).
max wi, j The time complexity of the algorithm described as above is
{ei, j ∈E a } | wi, i +wi, j +w j, j ≤M
3: Merge j with i in G a (C, Ea ) {Update C and Ea } O(N .|Eint |), where |Eint | is the number of interaction edges
4: Merge j with i in Gint (C, Eint ): wi,i ← wi,i + wi, j + w j, j and N the number of vertices (i.e., the initial cardinality of the
{Update C and Eint } cluster set C) issued from the area discretization. Indeed, the
5: until No adjacent clusters can be merged: ∀ei, j ∈ Ea, wi,i + first phase basically consists in going through all the |Eint |
wi, j + w j, j ≥ M. edges of Gint and finding the non self-loop edge with the
maximum weight. The second phase consists in adding and
removing (i.e., clustering) nodes in graphs (G a and Gint ). With
adjacency lists, you simply need to iterate over the edge list
First, we consider all the edges in Eint and we select the of the nodes to be clustered and update all those nodes. The
edge ei, j that has the highest weight such that: i) node i and algorithm stops when no pair of nodes can be merged anymore,
node j are neighbors in the area (i.e., ∃ ei,a j ∈ Ea ), ii) the which means maximum N − 1 passes are done. The number of
amount of interaction between node i and node j is equal or edges |Eint | depends on the nature of the graph. For example, it
lower than the maximum cluster capacity M (i.e., wi,i + wi, j + averages k.(N −1)/2 in an Erdös-Rényi graph [25], where k >
w j,i + w j, j ≤ M). The selected edge corresponds to the best 1 is the mean vertex degree. The complexity of our algorithm
interaction reduction at this stage. would thus be O(N 2 ). Note that it is a pessimistic upper bound
Secondly, we cluster node i and node j, updating the graphs since at each pass both the number of edges and vertices, and
G a and Gint with a new node that represents their clustering. thus the number of operations, decrease.
To do so, the neighbors of the new node in G A and Gint are
the former neighbors of node i and node j. The weights of the V. E VALUATION AND ANALYSIS
links between the new node and its neighbors in Gint are given
by the sum of the weight of the links between the former node In this section, we evaluate our MEC clustering algorithm
i and its neighbors and the former node j and its neighbors. with a real dataset of mobile communications. We first com-
Finally, the weight of the new self-loop corresponds to the sum pare it to the model on small problem instances (i.e., coarse-
of the two former self-loops plus the weights between node i grained area discretization) considering different day types
and node j. Once this second phase (clustering) is completed, and different periods of the day. Then, we evaluate it on
it is then possible to re-apply the first phase (selection) of the large problem instances (i.e., fine-grained area discretization).
algorithm to the resulting graphs and to iterate (see Fig. 2). Finally, we analyze its results through time.
By construction, the number of nodes (clusters) decreases
at each pass. The passes are iterated until there are no more A. Dataset
changes meaning that a local minimum of MEC cluster inter- To evaluate our algorithm and show the benefits of the MEC
action is attained. The final result of our algorithm corresponds approach compared to a classic centralized architecture, we
to a partition of the area into MEC clusters/areas whose use the dataset published as Open Data by Telecom Italia in
load (self-loop weights) is inferior but close to the maximum 2014 [26].
MEC server capacity M. We present a formal description in This dataset contains geo-referenced Call Detail Records
Algorithm 1. Note that our algorithm can be used in an n-level (CDRs) over the city of Milan from November 1st, 2013 to
MEC architecture, n designating the number of aggregation January 1st, 2014 [27]. During this period, every time a mobile
levels inbetween the base stations and the core (we consider user engaged a telecommunication interaction with another
only one level here: MEC servers), by applying it at each level. mobile user in the region, a CDR was created containing
The algorithm is reminiscent of the community detection the date time of the call and the geographical locations
algorithms in complex networks (e.g., Louvain algorithm [24]) of the mobile users (derived from the location of the base
in the way that it iteratively clusters nodes to increase a stations they used). The dataset was created combining all
modularity (in our case function of the weight of the self- this anonymous information, with a temporal aggregation of
loops). However, it differs on several important points. First, time slots of ten minutes and a spatial aggregation of square
it takes into account spatial properties between the nodes via grid calls of 235x235 meters (a grid of 100x100 cells to cover
the graph G a , which constrains the clustering. Second, only the city of Milan). The number of records in the dataset Si0 (t)

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 6

23.3km 100 23.3km

90
80 8498
5397
70
18.8km 18.8km
60
50 8479
8500
40
14.1km 14.1km 8338
30 4270
8489 8498
20
9.4km 9.4km

10 8496

4.7km 4.7km
8500

0.0km 0 0.0km
0.0km 4.7km 9.4km 14.1km 18.8km 23.3km 0.0km 4.7km 9.4km 14.1km 18.8km 23.3km

Fig. 3. Normalized mobile communication intensity in the city of Milan Fig. 4. Clustering result obtained with our algorithm for a maximum cluster
(5pm-6pm, 11/04/2013). The communications are concentrated in the city capacity of 5% of the total communications, i.e., 8,500 communications (5pm-
center. 6pm, 11/04/2013). The numbers in the clusters correspond to their load.

in a grid square i at time t follows the rule: Si0 (t) = [Link] (t)
75
where k is a constant defined by Telecom Italia. It aims at

Intra cluster traffic (%)

hiding the true number of calls, this information being business
70
confidential. Since Telecom Italia only possesses the data of
its own customers, the computed interactions are only between 65
them. This means that (at most) 34% of population’s data was
collected, due to Telecom Italia’s market share [27]. Around 60 Monday, 7am
1,3 million people live in the city of Milan. Monday, 5pm
To evaluate our algorithm, we generate from this 55 Sunday, 7am
anonymized dataset batches of communications aggregated Sunday, 5pm
by hour and spatially distributed over the city of Milan. 5 10 15 20 25
We first consider a batch of 170,000 communications, which Max. MEC cluster capacity (% of the total comm.)
corresponds to roughly 13% of Milan’s population, to study
(a) Proportion of intra MEC cluster traffic.
MEC resources partitioning (see Sec. V-B and Sec. V-C).
Then, we use hourly batches for a whole week. The volume of
the batches is proportional to what was measured by Telecom 10 Monday, 7am
Italia, considering that at the peak hour of the week there is Monday, 5pm
170,000 communications (see Sec. V-D). Sunday, 7am
8 Sunday, 5pm
# of clusters

Fig. 3 shows the normalized mobile communication inten-

sity distributed in the city of Milan between 5pm and 6pm
during a working day (Monday). We can clearly distinguish 6
the city center, which gathers most of the mobile calls. Fig. 4,
further discussed in this section, illustrates a clustering result. 4

B. MEC resources partitioning: comparing our algorithm with 5 10 15 20 25

the MILP Max. MEC cluster capacity (% of the total comm.)
We evaluate the MILP with the alternative connectedness (b) Number of MEC clusters.
constraints. Since it is computationally hard, we use a coarse- Fig. 5. MILP results: core offloading and number of clusters (Monday
grained area discretization (a 6 x 6 grid) to have small problem 11/04/2013 and Sunday 11/10/2013).
instances. We implemented the MILP in Python 3.5 using IBM
ILOG CPLEX 12.7 [28]. Our experiments ran on an Intel Core
i7-2620M CPU @ 2.70GHz x 4 workstation with 7.8 GiB of communication demands are spatially distributed in time and
RAM and Ubuntu 14.04 64-bit. space according to human activity (residential, business, enter-
Core offloading. tainment, transport etc.), we consider a working day (Monday
We first analyze the benefits of the MEC approach with 11/04/2013) and a weekend day (Sunday 11/10/2103) at two
respect to core offloading, that is we look at which portion periods: beginning of the activities (7am to 8am) and commu-
of the communications is directly served at the edge. As the nication peak (5pm to 6pm).

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 7

Monday, 7am 70
5 Monday, 5pm

Intra cluster traffic (%)

Optimality gap (%)

4
60
3
50 Monday, 7am
2
Monday, 5pm
1 Sunday, 7am
40 Sunday, 5pm
0
5 10 15 20 25 5 10 15 20 25
Max. MEC cluster capacity (% of the total comm.) Max. MEC cluster capacity (% of the total comm.)
(a) Monday 11/04/2013. (a) Proportion of intra MEC cluster traffic.

5 35
Sunday, 7am Monday, 7am
Sunday, 5pm 30 Monday, 5pm
4
Optimality gap (%)

25 Sunday, 7am

# of clusters
Sunday, 5pm
3 20
2 15
10
1
5
0
5 10 15 20 25 5 10 15 20 25
Max. MEC cluster capacity (% of the total comm.) Max. MEC cluster capacity (% of the total comm.)
(b) Sunday 11/10/2013. (b) Number of MEC clusters.
Fig. 6. Optimality gap (intra MEC cluster traffic ratio) for two different day Fig. 7. Geo-clustering algorithm results: core offloading and number of
types. clusters (Monday 11/04/2013 and Sunday 11/10/2013).

Fig. 5a shows the the amount of communications directly same evaluation setup. Fig. 6a and 6b show the optimality gap,
handled at the MEC servers in function of the maximum which is the percentage of difference between the intra cluster
MEC server capacity (expressed in percentage of the total traffic ratio provided by the MILP and our geo-clustering
communications to serve). 170,000 communications, derived algorithm, for a business day and a weekend day respectively.
from the dataset, were considered at each experiment; each There is no observable impact of the day and daytime on
point corresponds to 20 experiments. We can observe that an the efficiency of the algorithm. However, when the maximum
important part of the communications does not have to go up MEC cluster capacity is small, the results of the algorithm are
to the core. For example, with a maximum cluster capacity of close to optimal. The gap tends to increase with the maximum
10%, which represents 17,000 communications, between 55% MEC cluster capacity as there are fewer clusters.
and 64% of the traffic is directly absorbed by the MEC servers. The difference in the number of clusters was on average
We can also observe that the gain varies according to the day +0.3. The mean execution time of the MILP was around two
and the time of the day. The lower gains are at the peak hour hours, while the algorithm was executed in approximately 0.26
of the working day, while the upper gains are at the beginning seconds.
of the weekend day. These observations can be explained
by i) the spatial locality of the mobile communications and C. MEC resources partitioning: large instances
ii) the difference of human activities (mainly business and We then analyze MEC resources partitioning on large prob-
transportation on Monday at 5pm and residential on Sunday lem instances to have higher resolution and full MEC server
at 7am). capacity range. In the rest of this section, the experiments were
Fig. 5b presents the corresponding number of MEC servers. conducted using the geo-clustering algorithm with a 33 x 33
There is no major difference. Naturally, as the maximum discretization grid.
cluster capacity increases the number of clusters diminishes Geo-clustering.
to serve traffic at the edge. Note that with a spatial uniform We first illustrate the result of the algorithm. Fig. 4
distribution of the communications, we would have for in- presents the results of the geo-clustering algorithm on Monday
stance 10 servers, instead of 6-8, for a maximum capacity of 11/04/2013 between 5pm and 6pm with a maximum cluster
10%. capacity of 5% of the total communications, that represents
We then compare our algorithm with the MILP under the maximum 8,500 communications. The algorithm started with

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 8

180

Stacked # of comm. (x 1000)

160 MEC servers
35 Max. MEC cluster capacity 140 Offloaded to the core
Core
120
(# of intra comm. x 1000)

30 100
80
MEC cluster loads

25 60
40
20 20
0
. . ov. . . ov. .
15 Nov Nov N Nov Nov N Nov
04 05 06 07 08 09 10
Time
10
(a) Partition done at 5pm-6pm on Monday 11/04/2013 with a maximum cluster
5 capacity of 10% of the total communications.
0
180

Stacked # of comm. (x 1000)

2 3 4 5 6 7 8 9 10 12 14 16 18 2022.525 MEC servers
160
Max. MEC cluster capacity (% of the total comm.) 140 Offloaded to the core
Core
120
100
Fig. 8. MEC cluster loads (5pm-6pm, 11/04/2013). 80
60
40
20
0
. . ov. . . ov. .
Nov Nov N Nov Nov N Nov
1,089 clusters (33 x 33) and took, without any code optimiza- 04 05 06 07 08 09 10
Time
tion, 18.73 seconds to provide this result. The grey atomic
squares correspond to grid cells with no or very low traffic. (b) Partition done at 5pm-6pm on Monday 11/04/2013 with a maximum cluster
capacity of 5% of the total communications.
However, they could not be ultimately merged to a neighbor
cluster because it would have increased their load above the
180
Stacked # of comm. (x 1000)

maximum threshold. We thus consider that if their intra-cluster 160 MEC servers
Offloaded to the core
traffic is lower than 0.01% of the maximum capacity, they do 140 Core
120
not form a cluster and their communications are directly served 100
in the core. We can see that the area of the clusters that cover 80
60
the city center is smaller than the ones that serve low-density 40
20
regions, the density of communications being higher there (see 0
. . ov. . . ov. .
Fig. 3). Moreover, as seen in Fig. 8, most of the loads are close Nov Nov N Nov Nov N Nov
04 05 06 07 08 09 10
to the maximum server capacity. The MEC areas are thus well Time
balanced. (c) Partition done at 7am-8am on Monday 11/04/2013 with a maximum cluster
capacity of 5% of the total communications.
Core offloading.
Fig. 9. MEC servers and core traffic distributions over a week for different
Fig. 7a shows the benefits of the MEC approach with respect partitions.
to core offloading. 170,000 communications, derived from
the dataset, were considered at each experiment; each point
corresponds to 20 experiments. The observations confirm that
D. Through time
an important part of the communications can be handled at the
edge, at the MEC servers, instead of going up to the core and Core offloading.
that the ratio of traffic depends on the day type and daytime. We finally evaluate the performance of our algorithm
The core offloading remains important even for very small through time. To this aim, we first use the mobile data on a
cluster capacities. full week. We consider that at the peak hour of this period
Fig. 7b presents the corresponding number of MEC clusters (Thursday, 5pm, 11/08/2013), there are 170,000 communi-
and hence MEC servers. We can see that naturally the number cations per hour. It represents more or less the volume of
of clusters increases rapidly as the maximum cluster capacity communications in the city of Milan for the market share
diminishes. These results suggest that a trade-off has to be of Telecom Italia in 2013. We retrieved from the dataset the
found between the ratio of traffic handled at the edge and the proportion of communications hour by hour and their spatial
number of MEC servers to deploy. distribution. We then considered three partitions obtained on
Monday 11/04/2013 at different hours (7am-8am and 5pm-
Server load balancing. 6pm) of the day and with different maximum cluster capacity
Fig. 8 shows the loads of the clusters at the peak hour on (5% and 10% of the total communications at this period of
Monday. We can observe that the partition, and hence the load, the day).
is well balanced. Indeed, most of the clusters have a load In Fig. 9a, we can observe that around 53% of the com-
close to the maximum cluster capacity. Moreover, in all cases munications are directly handled by the MEC servers during
except two (18% and 20%), the median values almost match the working days. This share increases up to 61% during
the maximum values. We had the same observations for the the week-end. Obviously, if we consider a maximum cluster
other periods we evaluated. capacity of 5% (Fig. 9b), the global load distribution through

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 9

9 1.0 demand) and latency. We also aim at combining our approach

8 0.9 with online application offloading and migration to support
7 0.8 micro-mobility.
6 0.7
0.6
Cluster ID

5 R EFERENCES
0.5
4 0.4
[1] European Telecommunications Standards Institute (ETSI), Mobile-Edge
Computing (MEC); Service Scenarios (GS MEC-IEG 004), November
3 0.3 2015.
2 0.2 [2] OpenFog Consortium Architecture Working Group, OpenFog Architec-
1 0.1 ture Overview (OPFWP001.0216), February 2016.
[3] Y. C. Hu, M. Patel, D. Sabella, and et al., Mobile Edge Computing
0 A key technology towards 5G (ETSI White Paper No. 11), European
0 4 8 12 16 20 23 Telecommunications Standards Institute (ETSI), September 2015.
Time (hour) [4] P. Ventre, C. Pisa, S. Salsano, and et al., “Performance evaluation and
tuning of virtual infrastructure managers for (micro) virtual network
functions,” in Proceedings of IEEE NFV-SDN Conference, 2016.
Fig. 10. Normalized MEC cluster loads over a day (11/04/2013) with a [5] R. Saunders, J. Cho, A. Banerjee, and et al., “P2P offloading in mobile
partition done at 5pm and a maximum cluster capacity of 5% of the total networks using SDN,” in Proceedings of ACM Symposium on SDN
communications. Research (SOSR), 2016.
[6] M. Jia, W. Liang, Z. Xu, and M. Huang, “Cloudlet load balancing in
wireless metropolitan area networks,” in Proceedings of IEEE INFO-
time remains the same, but the traffic share of the MEC COM. IEEE, 2016, pp. 1–9.
[7] H. Wang, F. Xu, Y. Li, P. Zhang, and D. Jin, “Understanding mobile
servers drops to approximately 45%. In both cases, we can traffic patterns of large scale cellular towers in urban environment,” in
notice that the traffic offloaded to the core, that corresponds Proceedings of ACM Internet Measurement Conference (IMC), 2015.
to the local traffic that could not be handled by MEC servers [8] D. Naboulsi, M. Fiore, S. Ribot, and R. Stanica, “Large-scale mobile
traffic analysis: a survey,” IEEE Communications Surveys and Tutorials,
because they were saturated, is very small (maximum 3.1% 2015.
on 11/05/2013 at 12am). Finally, we can notice that, if we [9] N. Tastevin and M. Bouet, “Characterizing and modeling the distance
consider a partition done in the morning (Fig. 9c) instead of of mobile calls: A metropolitan case study,” in Proceedings of IEEE
International Symposium on Personal, Indoor, and Mobile Radio Com-
the peak hour (Fig. 9b), the share of the MEC servers slightly munications (PIMRC), Sept 2016.
decreases, while they remain almost unsaturated. [10] Z. A. Qazi, P. Krishna, V. Sekar, and et al., “KLEIN: A Minimally
Server load balancing. Disruptive Design for an Elastic Cellular Core,” in Proceedings of ACM
Symposium on SDN Research (SOSR), 2016.
At last, Fig. 10 presents the distribution of the MEC server [11] M. Bouet and V. Conan, “Geo-partitioning of MEC resources,” in
loads. The partition corresponds to the one shown on Fig. 4. Proceedings of the ACM SIGCOMM Workshop on Mobile Edge Com-
It was done with the communications that occurred between munications (MECOMM’2017), 2017, pp. 43–48.
[12] T. Taleb, K. Samdanis, B. Mada, H. Flinck, S. Dutta, and D. Sabella, “On
5pm and 6pm. We can distinguish human activities, that is multi-access edge computing: A survey of the emerging 5g network edge
low activity until 8am and after 9pm and two peaks around cloud architecture and orchestration,” IEEE Communications Surveys
10 am and 5pm. At each hour, the load is homogeneously and Tutorials, vol. 19, no. 3, pp. 1657–1681, 2017.
[13] P. Mach and Z. Becvar, “Mobile edge computing: A survey on architec-
distributed on the servers, which means that the MEC areas ture and computation offloading,” IEEE Communications Surveys and
are well dimensioned and that all users experience good QoE. Tutorials, vol. 19, no. 3, pp. 1628–1656, 2017.
[14] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey
on mobile edge computing: The communication perspective,” IEEE
VI. C ONCLUSION AND PERSPECTIVES Communications Surveys and Tutorials, vol. PP, no. 19, 2017.
[15] A. Ceselli, M. Premoli, and S. Secci, “Mobile edge cloud network design
MEC is a key technology to support the ever-increasing optimization,” IEEE/ACM Transactions on Networking, no. 99, pp. 1–14,
growth of communication capability demands and realize the 2017.
IoT and 5G visions. As operators are transforming their net- [16] Q. Xu, F. Qian, J. Huang, A. Gerber, and et al., “Cellular data net-
work infrastructure characterization and implication on mobile content
work architectures and are looking for deploying computation placement,” in Proceedings of ACM SIGMETRICS, 2011.
resources close to the users to improve QoE, it is necessary [17] C. Chang, K. Alexandris, N. Nikaein, K. Katsalis, and T. Spyropoulos,
to adequately dimension MEC systems. In this paper, we “MEC architectural implications for LTE/LTE-A networks,” in Proceed-
ings of the Workshop on Mobility in the Evolving Internet Architecture
formulated this problem as a mixed integer linear program (MobiArch), 2016, pp. 13–18.
and presented a graph-based algorithm that enable finding [18] F. Manco, J. Martins, K. Yasukata, and et al., “The case for the superfluid
a partition of MEC areas that consolidates traffic at the cloud,” in Proceedings of USENIX HotCloud Workshop, 2015.
[19] M. Satyanarayanan, P. Bahl, R. Caceres, and N. Davies, “The case for
edge, in MEC servers. We evaluated them using a real world VM]-based cloudlets in mobile computing,” IEEE Pervasive Computing,
dataset from a mobile operator. The evaluation results, beyond vol. 8, no. 4, pp. 14–23, Oct. 2009.
quantifying the benefits of the MEC approach, show that [20] D. Liu, B. Chen, C. Yang, and A. F. Molisch, “Caching at the
wireless edge: design aspects, challenges, and future directions,” IEEE
the core can be largely offloaded. They also show that the Communications Magazine, vol. 54, no. 9, pp. 22–28, 2016.
algorithm provides MEC areas that are well balanced in terms [21] L. Tong, Y. Li, and W. Gao, “A hierarchical edge cloud architecture for
of load and close to optimal. Finally, we ran simulations over mobile computing,” in Proceedings of IEEE International Conference
on Computer Communications (INFOCOM), 2016, pp. 1–9.
one week of communications and observed that there is almost [22] H. Tan, Z. Han, X. Li, and F. Lau, “Online job dispatching and schedul-
no saturation of the MEC servers, while the traffic on the ing in edge-clouds,” in Proceedings of IEEE International Conference
core is largely reduced. In future work, we expect to explore on Computer Communications (INFOCOM), 2017, pp. 1–9.
[23] S. Vicente, V. Kolmogorov, and C. Rother, “Graph cut based image seg-
several aspects such as group communications, energy saving mentation with connectivity priors,” in Proceedings of IEEE Conference
(in particular with respect to the temporal distribution of the on Computer Vision and Pattern Recognition, 2008, pp. 1–8.

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2018.2816263, IEEE
Transactions on Network and Service Management
IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT 10

[24] V. Blondel, J. Guillaume, R. Lambiotte, and E. Mech, “Fast unfolding

of communities in large networks,” J. Stat. Mech, 2008.
[25] P. Erdos and A. Rényi, “On the evolution of random graphs,” Bull. Inst.
Internat. Statist, vol. 38, no. 4, pp. 343–347, 1961.
[26] “Open Big Data,” [Link] 2014,
accessed: 2017-03-23.
[27] G. Barlacchi, M. De Nadai, R. Larcher, and et al., “A multi-source
dataset of urban life in the city of Milan and the Province of Trentino,”
Scientific Data, vol. 2, no. 150055, 2015.
[28] “IBM ILOG CPLEX Optimizer,” [Link]
[Link]/software/integration/optimization/cplex-optimizer/, 2016.

Mathieu Bouet is a research expert in networking

and communications at Thales, France. He received
a Ph.D degree in computer science and an Ha-
bilitation degree (HDR) from Sorbonne University
(previously UPMC - Paris VI) in 2009 and 2017 re-
spectively. He currently manages research activities
on network softwarization in the networking labo-
ratory of the advanced studies department at Thales
in Gennevilliers. His research interests are mainly:
network virtualization and network optimization.

Vania Conan is a senior research expert in net-

working and communications at Thales, France. He
received an Engineering degree (1990) and PhD in
computer science (1996) from Mines ParisTech, and
an Habilitation degree from Sorbonne University,
Paris (2012). He is presently head of the networking
laboratory in the Advanced Studies department at
Thales in Gennevilliers. In the past years he has
been conducting research in the fields of software
defined communications and wireless networking.
He has published over 100 international conference
and journal papers and holds 10 patents in networking technologies. His
current research topics include mobile network protocols and virtualized
network design.

1932-4537 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See [Link] for more information.