Threats and vulnerabilities in Federated Learning

Prerequisites -  Collaborative Learning – Federated Learning, Google Cloud Platform – Understanding Federated Learning on Cloud

In this article, we will learn review what is federated learning and its advantages over conventional machine learning algorithms. In the later part let's try to understand the threats and vulnerabilities in federated learning architecture in simple terms. 

Federated Learning and its advantages:

Let's take an example to understand how FL architecture works, say there is a network of IoT devices that send data to a centralized server that uses the data to train a model and make predictions. What if this data that is transmitted over the network is confidential and could be used to manipulate some important decisions, this is where FL architecture could be handy. As we are using IoT devices we can integrate them with little more intelligence to train the model by themselves, but there could be a problem if it's just a standalone device it may not be exposed to wide distributions of data to train the model, so what we can do is that once the devices train a basic model the model parameters are sent to the server for aggregation and this aggregated model is sent back to all the devices for making better predictions, even if the model parameters are leaked there are very little chances of inferring something from that parameters. In this way, we are not compromising data privacy and we are reducing the cost of transmitting bulky data over the network.

The formal definition of FL - "Federated learning is a machine learning technique that trains an algorithm across multiple decentralized edge devices or servers holding local data samples, without exchanging them.".  The basic idea of FL architecture that we saw in the example is not a foolproof method and the architecture itself has some lapses and vulnerabilities presenting us with some challenging security issues to deal with. Let's see some threat models and poisoning attacks in brief and in simple terms, understanding these models and attacks could help us design a foolproof privacy-preserving FL protocol.

Threat Models:

• Insider attack Vs Outsider attack: An insider attack refers to an attack made by an insider ie. the server or any of its clients, whereas an outsider attack is launched by an outsider like for example a malicious intent hacker eavesdropping over the communication channel between the server and the clients. Insider attacks are generally more damaging and adverse than outsider attacks because they have more control over the FL architecture. Some common types of insider attacks are as follows:
  • Single attack: a single malicious FL client causes the model to miss classify with high probability.
  • Byzantine attack: this attack is similar to that of the single attack but here the client behaves in an arbitrary fashion making it difficult to find out if the model that is sent is genuine.
  •  Sybil attack: here the attacker simulates multiple counterfeit FL clients and supplies corrupted parameters and mounts more powerful attacks.
•  Semi-honest attack Vs Malicious attack: In a semi-honest setting, the attacker is called semi-honest because he follows the FL protocol but tries to access the restricted states (such as the model parameters) of an honest client and they also stay passive but not contributing to the architecture. Whereas in case of a malicious attack the attacker arbitrarily deviates from the FL protocol and tries to access, modify, manipulate the honest client's local training data.
• Training Phase Vs Inference Phase: Attacks in the training phase tend to influence and corrupt the FL model, they try to poison the data and compromise the integrity of the training dataset and they also try to poison the model to disrupt the learning process. In case of inference phase attacks, they do not corrupt the model or data, instead, they cause the model to produce wrong outputs and collect the model characteristics thereby compromising privacy.

Poisoning Attacks:

Poisoning attacks depend on the attacker's objective, they may launch a random attack or attack a specific target. In random attacks, they aim to reduce the FL model's accuracy and in case of targeted attacks, they aim at influencing the model to output wrong labels ie. the labels intended by the attacker and they generally have a specific goal to achieve. Poisoning attacks can happen in two ways, data poisoning (during local data collection) and model poisoning (during the model training process). 

• Data Poisoning: here the attacker corrupts/changes labels of the data and may also try to modify the individual features or small parts of the training data. This attack is generally carried out by the FL clients/participants and the impact depends on the extent to which the FL participants engage.
•  Model Poisoning: here the attacker aims to poison the local model updates before sending them to the server or the attacker may also try to insert a backdoor to the global model for corrupting it. Model poisoning has more impact compared to data poisoning since tweaking the model completely changes its characteristics thereby misclassifying the data. These attacks have more impact when the attacker tries to escape from getting detected by using an alternating minimization strategy to alternately optimize for the training loss.