MACHINE LEARNING IN

WIRELESS
COMMUNICATION
PRESENTED BY:
AMBROSIA PATEL(R31)
GNANESHWAR (R11)
BHASHKAR (R07)
 MACHINE LEARNING
Machine Learning is a subset of Artificial
Intelligence that enables systems to learn
from data and improve performance.
ML does not require explicit programming.
Used for pattern recognition, classification,
and automation.
Includes supervised, unsupervised, and
reinforcement learning.
Enhances decision-making in dynamic
environments by recognizing patterns and
adapting accordingly.
WHY ML IN WIRELESS COMMUNICATION?
Growing complexity of wireless networks requires
intelligent solutions.
Wireless networks are becoming highly complex due
to the increasing number of connected devices,
diverse applications, and dynamic environments.
Operators see traditional human machine interaction
as slow and error-prone.
Traditional methods are expensive because it
requires highly skilled engineers to handle these
issues and requirements.
WHY ML IN WIRELESS COMMUNICATION?
Benefits:
🔋
Optimizing energy consumption : Reduces power usage by
dynamically allocating resources.
Enhancing spectral efficiency 📡 : Optimizes frequency
utilization for better performance.
⏳
Reducing latency : Ensures low-latency communication for
real-time applications.
Automating network management 🤖 : Reduces human
intervention by dynamically adjusting network parameters.
Predicting failures 🔍 : Identifies potential network issues
before they occur.
📡
EVOLUTION OF WIRELESS NETWORKS
1G: Analog Voice – Basic voice communication.
2G: Digital Voice & SMS – Introduction of texting services.
3G: Mobile Internet – Enhanced data capabilities.
4G: High-speed Internet – Support for HD video and
streaming.
5G: Ultra-fast, low latency – Supports massive IoT and
real-time applications.
Beyond 5G: AI-driven, adaptive, and self-organizing
networks for greater efficiency
ROLE OF ML IN NETWORK EVOLUTION
Enables automation in 5G & beyond: Reduces manual interventions
by self-adjusting network parameters.
Improves QoS & security: Ensures optimal user experience and
protection against cyber threats.
Optimizes resource allocation: Dynamically distributes bandwidth
and computing power.
Supports dynamic spectrum access: Allocates spectrum efficiently
based on demand.
Enhances network reliability and adaptability: Ensures stable and
adaptive connections.
 MACHINE LEARNING
The learning system of a machine learning algorithm has three main
parts:
Decision Process: The algorithm makes predictions based on
input data.
Error Function: It evaluates prediction accuracy by comparing
with known data.
Optimization Process: Weights are adjusted iteratively to
minimize errors until accuracy reaches a set threshold.
MACHINE LEARNING METHODS
There are mainly 3 types of machine learning
Supervised ML
Unsupervised ML
Reinforcement ML
 SUPERVISED MACHINE LEARNING
Supervised Machine Learning in wireless communication is a
technique where a model is trained using labeled data (data
with known inputs and outputs) to predict network behavior or
classify wireless signals.

The model learns from past network data (like signal strength,
traffic patterns, or errors) and makes decisions to improve
network performance or solve communication problems.
 SUPERVISED MACHINE LEARNING
The supervised learning problem formulation will have a dataset of instances x (often
called training set, examples, samples, or objects), and its corresponding label y .

An ML algorithm a_θ(e.g., decision tree, linear model, neural network, etc.) will find a
function that maps those instances to label

To measure the quality of performance of the predictor, the algorithm uses a loss
function as follows

The main goal of supervised learning is to find the parameters θ^′, that minimizes the
loss function :

arg⁡min⁡θ L(y, a_θ(x))

 TERM MEANING

Training Set (x) The example we give to the model.

Labels (y) Correct answers .

The machine learning method we

ML Algorithm (a_θ ) use(e.g. decision tree, linear model, neural network).

A formula that measures how wrong

Loss Function (L) the model’s prediction is.

Numbers the model adjusts during

Parameter (θ^′) training to make better predictions.
 SUPERVISED MACHINE LEARNING
Use of Machine Learning in LTE Small Cells and Self-Organizing Networks
(SON) in 5G Networks

•By using smaller cells, mobile networks can support more users without
congestion, optimize power consumption, and provide seamless connectivity.

•This structure is particularly useful in modern communication systems like

4G and 5G, where high-speed data transfer and low latency are crucial.

•To handle high traffic demand and provide faster data rates, operators are
deploying LTE small cells (small base stations) in dense urban areas like
cities, malls etc.
 SUPERVISED MACHINE LEARNING
•But there’s a problem regarding these small cells:

1. These small cells cause unpredictable interference with each

other because they are close together.

2. Manual adjustments are difficult and expensive.

This is the reason why Self Organising Networks (SON) are

introduced in 5G.
 SUPERVISED MACHINE LEARNING
Self-Organizing Network (SON)

•A SON is an intelligent network management system that can

automatically configure, optimize, and heal the network without
human intervention.

•How Does Machine Learning Help in SON?

Bagging Tree ensemble Machine Learning (ML) is used to help SON
systems to make better decisions.
 SUPERVISED MACHINE LEARNING
1. Data Collection
2. Feature Selection / Extraction
3. Labeling the Data (Supervised Learning)
4. Train Bagging Tree Model
5. Predict Performance for New Configurations
UNSUPERVISED MACHINE LEARNING
Unsupervised Machine Learning is a type of
machine learning where the model learns from
unlabeled data without any explicit supervision.

Unlike supervised learning, it does not have

predefined categories or labels. Instead, it finds
hidden structures, clusters, or patterns in the data.
 UNSUPERVISED MACHINE LEARNING
Characteristics of Unsupervised Learning

No Labeled Data
Finds Hidden Structures
Self-Organizing Models
Used for Pattern Recognition
UNSUPERVISED MACHINE LEARNING
UNSUPERVISED MACHINE LEARNING
The diagram shows an initial set of unlabeled data points, representing different
user equipment (UE) devices scattered along two feature dimensions.

Each point represents a user, and the two axes (Feature 1 and Feature 2) denote
measurable characteristics of the network, but no labels are provided to identify
any specific group or category.

The unsupervised learning algorithm (e.g., K-means clustering) has been applied
to group the data points into clusters based on their similarity in feature values.

The data points are now assigned different colors, representing distinct clusters.
Dashed lines separate the clusters, indicating the decision boundaries formed by
the algorithm.
 UNSUPERVISED MACHINE LEARNING
Unsupervised Learning-Based Selection Using K-means Clustering for
VANETs (Vehicular Ad-Hoc Networks with eNB Support)

•VANTEs is a special case where the eNB (base station) is involved in managing
vehicular communication to improve data delivery in moving vehicle
environments.
•Basic Idea in including ML in VANET
•The base station (eNB) wants to send a message to many vehicles.
• Instead of broadcasting to all vehicles directly, it sends data to a few selected
relay vehicles, and those relay vehicles forward the message to nearby
vehicles.
•This is called Multicast via D2D Communication (Device-to-Device).
 UNSUPERVISED MACHINE LEARNING
Problem is How to select the Relay Vehicle?

The eNB does not know in advance which vehicles are

best relays.
There are no labeled "good relay vehicles".
K-means clustering can group nearby vehicles
together based on location and signal conditions.
The center of each cluster becomes a "good
representative," likely a good relay vehicle
K-MEANS ALGORITHM
K-MEANS ALGORITHM
Step 1- Initialization – Random Selection of Relay Vehicles.
Step 2- Cluster Formation – Assigning Vehicles to Nearest
Relay.
Step 3- Calculating Cluster Centers – Averaging Vehicle
Locations.
Step 4- Relay Update – Selecting Vehicles Closest to
Cluster Centers.
Step 5- Iterative Refinement – Repeating Cluster
Assignment and Relay Update.
Step 6- Relay Selection Finalization.
Step 7- Multicast Transmission Using Selected Relays.
 REINFORCEMENT LEARNING
Reinforcement Learning is a type of Machine Learning where an
agent learns by interacting with an environment and receives
feedback in the form of rewards or penalties.
Unlike supervised learning, where models learn from labeled data,
RL learns from experience through trial and error.
RL can be expressed with Markov Decision Process (MDP).
A Markov Decision Process (MDP) is a mathematical framework
used in Reinforcement Learning (RL) to model decision-making
problems in environments where outcomes are partly random
and partly under the control of an agent.
 REINFORCEMENT LEARNING

Agent interacts with the environment through a set of actions.

The RL agent takes actions in the environment, that causes a
change in the environment’s current state generating a new
state and receives a reward based on the results.
The agent receives a positive reward for good actions and a
negative reward for bad actions, which helps the agent
evaluate the performed action in a given state and learn from
experiences.
 REINFORCEMENT LEARNING
1. Transition Probability

We define the probability of the transition to state s ′ with reward r

from taking action ‘a’ in state ‘s’ at time ‘t’, for all s ′ ∈ S, s ∈ S, r ∈ R, a
∈ A(s), as:
The probability that at time ‘t’, the agent moves to state s′ and
receives reward ‘r’, given that it was in state ‘s’ and took action ‘a’ at
the previous time step.
The agent does not know this probability distribution in advance; it
learns it through interactions with the environment.
The next state s’ doesn’t depend on past states or actions.
 REINFORCEMENT LEARNING
2. Reward Function (R):
The reward is feedback to the agent after taking an action.
It tells the agent how good or bad an action is.
the agent learns to maximize its total reward over time.
Two types of reward:
- Action-based Reward: R(a)
- State-Action-based Reward: R(s,a)
For example:
- High data rate → Positive reward.
- Dropped connection → Negative reward.
 REINFORCEMENT LEARNING
3. Objective in RL – Maximizing Discounted Reward:
The goal of an RL agent is to maximize the expected summation of the
discounted rewards, which drives the agent to take selected actions.
In real-world scenarios not all rewards are equally important- some
rewards come immediately, while others come in future.
To handle this, RL uses discounted rewards, where future rewards are
weighted less than immediate rewards.
The discounted reward G_t​is expressed as:

Emphasizing the current action’s immediate reward and reducing the

impact of future actions, rewards help the expected summation of
discounted rewards to converge.
 REINFORCEMENT LEARNING
4. Policy (π) – Agent’s Strategy:
A policy (π) in Reinforcement Learning defines the behavior of the agent.
It is the agent’s control strategy which tells the agent what action(a) to
take when it is in a particular state(s).
π(a∣s)= Pr(a=a∣s=s)
Formally, policy π(a∣s) is the probability of taking action ‘a’ when in state
‘s’.
Once the agent takes an action, the agent uses a value function to
evaluate the action.
The agent either uses:
1) a state-value function to estimate how good for the agent to be in state
2) an action-value function to measure how good it is for the agent to
perform an action ‘a’ in a given state s.
 REINFORCEMENT LEARNING
The action-value function is defined in terms of the expected
summation of the discounted rewards and represents the target
Q-value:

Finding the optimal Q-value requires selecting the best actions

and maximize the expected summation of discounted rewards
under the optimal policy π.
Each policy π gives a different Q-value.
The optimal Q-value Q∗(s,a) is the maximum possible reward we
can achieve from state ‘s’ by taking action ‘a’.
 REINFORCEMENT LEARNING
•The optimal Q-value Q∗(s, a) must satisfy the Bellman optimality equation:

•The equation relates the optimal Q-value of a state-action pair to future rewards.

Reward for taking action a in state s.

R_(t+1)
(Immediate Reward)

Discount factor (reduces

y
the value of future rewards).

Maximum expected discounted return that can be

achieved for any possible next state-action pairs
(Discounted Future Reward)
 REINFORCEMENT LEARNING
This optimal Q-value Q∗(s, a) is used to train the network.
The Q-value Q(s,a) predicted by the network is subtracted from
the optimal Q-value Q∗(s,a) estimated using the Bellman
equation and backpropagated through the network
The loss function is defined as follows:
 REINFORCEMENT LEARNING
After calculating the loss function in Deep Q-Learning, the network’s
weights are updated using backpropagation and gradient descent.
Backpropagation- It is an algorithm to find out how much each part
of a neural network was contributing in error and adjust it to improve.
Gradient descent- It is an optimization technique that adjusts the
network’s weights in the direction that minimizes the loss function.
This adjustment minimizes the error between the predicted Q-value
and the target Q-value.
Over time, the network improves its Q-value predictions, helping the
agent make better action decisions to maximize future rewards.
 REINFORCEMENT LEARNING
Multi-Agent Reinforcement Learning (MARL) is Implemented in Multi-UAV
Networks
•In multi-UAV networks, Multi-Agent Reinforcement Learning (MARL) is a
technique that allows multiple UAVs (drones) to learn from their
environment and from each other to make better decisions in a dynamic
and uncertain environment.

ALGORITHM-
Step 1- UAVs as Agents
•Each UAV acts as an independent agent that makes its own decisions.
However, the actions of one UAV affect others, because they share the same
environment
 REINFORCEMENT LEARNING
Step 2- Observation of the Environment
Each UAV observes its environment, which can include:
•Positions of other UAVs.
•Locations of users needing network coverage.
•Signal strength and interference levels.
•Battery level and energy consumption.
•This information represents the state ‘s’ of the environment from the UAV’s
perspective.
 REINFORCEMENT LEARNING
Step 3- Decision-Making (Actions)
Each UAV chooses an action ‘a’ based on its current observation:
Move to a different position.
Change its transmission power.
Switch frequency channels.
Schedule data transmission differently.
The UAVs make these decisions simultaneously, meaning each UAV’s
action can affect the performance of others.
 REINFORCEMENT LEARNING
Step 4- Reward Signal
After taking actions, each UAV receives a reward based on the
impact of its action on network performance.
The reward can be based on:
Higher throughput → Positive reward.
Good coverage → Positive reward.
Interference with other UAVs → Negative reward.
High energy consumption → Negative reward.
The reward is a signal that tells the UAV how good or bad its action
was.
 REINFORCEMENT LEARNING
Step 5- Environment State Changes
Once UAVs take actions, the environment changes:
-UAVs move to new positions.
-Signal coverage changes.
-Interference levels may increase or decrease.
-Users may move.
-Wind or other disturbances can occur.
The UAVs observe this new state and receive their reward.
 REINFORCEMENT LEARNING
Step 6- Learning Through Reinforcement Learning
Each UAV updates its policy (strategy) based on:
-The action it took.
-The reward it received.
-The new state it entered.
Using MRL algorithms like:
- Q-learning.
- Deep Q-Networks (DQN).
- Multi-Agent Deep Deterministic Policy Gradient (MADDPG).
The UAVs learn which actions lead to better rewards over time.
 ML APPLICATIONS IN 5G AND B5G
Adaptive Modulation and Coding (AMC)
AMC can adjust the modulation order and/or coding
rate to adapt to the fading channel variation.
Traditional AMC methods are either inaccurate or
complex.
ML-based AMC can improve accuracy and efficiency.
Problem: Large size of lookup tables
Supervised learning and reinforcement learning are
widely applied to implement ML based AMC.
 ML APPLICATIONS IN 5G AND B5G
Channel Estimation and Equalization
It is the major technology to mitigate the inter-symbol interference
(ISI) and nonlinear distortions in wireless communication.
Adaptive equalizer based on multilayer perceptron (MLP) was
proposed.
Predicts channel variations to enhance communication reliability.
Reduces signal distortion for better performance.
Improves signal-to-noise ratio (SNR) and ensures clear and
strong signals.
 ML APPLICATIONS IN 5G AND B5G
Network Energy Saving
Millions of the base stations are being constructed, Such
rapid growth brings the issues of high energy consumption.
ML based energy saving predicts load accurately and
switch off the cell in time, which achieve the better
performance on energy saving.
The switch-off duration of cells is significantly improved,
leading to better energy efficiency without negatively
impacting system performance.
 ML APPLICATIONS IN 5G AND B5G
Network Energy Saving
ML APPLICATIONS IN 5G AND B5G
Trajectory Prediction
A large number of small cells in 5G will lead to more
frequent handover.
This will cause high latency and throughput reduction.
ML algorithms are widely considered to improve the
performance of trajectory prediction.
LSTM(Long Short-Term Memory) is commonly used to solve
problems such as trajectory prediction
 ML APPLICATIONS IN 5G AND B5G
Trajectory Prediction

There are some more applications such as Load Prediction and

Beamforming etc...
 CONCLUSION
Machine Learning is transforming wireless communication by
enhancing automation, optimizing resources, and improving network
performance.
Techniques like supervised, unsupervised, and reinforcement learning
help solve challenges in 5G and multi-UAV networks.
5G and Beyond 5G (B5G) networks heavily rely on ML for adaptive
modulation, energy savings, trajectory prediction, and self-organizing
networks (SON)
ML ensures reliable, adaptive, and intelligent wireless communication
systems, paving the way for future advancements in next-generation
networks.
THANK YOU