site/en/tutorials/generative/adversarial_fgsm.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XGCdmDAKpLuf"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors.\n",
        "\n",
        "Licensed under the Apache License, Version 2.0 (the \"License\");"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "GF4d1XplpLGF"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W1L3zJP6pPGD"
      },
      "source": [
        "# Adversarial example using FGSM\n",
        "\n",
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/generative/adversarial_fgsm\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/generative/adversarial_fgsm.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/generative/adversarial_fgsm.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/generative/adversarial_fgsm.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8dn1-g8BpPDx"
      },
      "source": [
        "This tutorial creates an *adversarial example* using the Fast Gradient Signed Method (FGSM) attack as described in [Explaining and Harnessing Adversarial Examples](https://arxiv.org/abs/1412.6572) by Goodfellow *et al*. This was one of the first and most popular attacks to fool a neural network. \n",
        "\n",
        "## What is an adversarial example?\n",
        "\n",
        "Adversarial examples are specialised inputs created with the purpose of confusing a neural network, resulting in the misclassification of a given input. These notorious inputs are indistinguishable to the human eye, but cause the network to fail to identify the contents of the image. There are several types of such attacks, however, here the focus is on the fast gradient sign method attack, which is a *white box* attack whose goal is to ensure misclassification. A white box attack is where the attacker has complete access to the model being attacked. One of the most famous examples of an adversarial image shown below is taken from the aforementioned paper.\n",
        "\n",
        "![Adversarial Example](images/adversarial_example.png)\n",
        "\n",
        "Here, starting with the image of a panda, the attacker adds small perturbations (distortions) to the original image, which results in the model labelling this image as a gibbon, with high confidence. The process of adding these perturbations is explained below.\n",
        "\n",
        "## Fast gradient sign method\n",
        "The fast gradient sign method works by using the gradients of the neural network to create an adversarial example. For an input image, the method uses the gradients of the loss with respect to the input image to create a new image that maximises the loss. This new image is called the adversarial image. This can be summarised using the following expression:\n",
        "$$adv\\_x = x + \\epsilon*\\text{sign}(\\nabla_xJ(\\theta, x, y))$$\n",
        "\n",
        "where \n",
        "\n",
        "*   adv_x : Adversarial image.\n",
        "*   x : Original input image.\n",
        "*   y : Original input label.\n",
        "*   $\\epsilon$ : Multiplier to ensure the perturbations are small.\n",
        "*   $\\theta$ : Model parameters.\n",
        "*   $J$ : Loss.\n",
        "\n",
        "An intriguing property here, is the fact that the gradients are taken with respect to the input image. This is done because the objective is to create an image that maximises the loss. A method to accomplish this is to find how much each pixel in the image contributes to the loss value, and add a perturbation accordingly. This works pretty fast because it is easy to find how each input pixel contributes to the loss by using the chain rule and finding the required gradients. Hence, the gradients are taken with respect to the image. In addition, since the model is no longer being trained (thus the gradient is not taken with respect to the trainable variables, i.e., the model parameters), and so the model parameters remain constant. The only goal is to fool an already trained model.\n",
        "\n",
        "So let's try and fool a pretrained model. In this tutorial, the model is [MobileNetV2](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/keras/applications/MobileNetV2) model, pretrained on [ImageNet](http://www.image-net.org/)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vag2WYR6yTOC"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "import matplotlib as mpl\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "mpl.rcParams['figure.figsize'] = (8, 8)\n",
        "mpl.rcParams['axes.grid'] = False"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wiTHY8dqxzx7"
      },
      "source": [
        "Let's load the pretrained MobileNetV2 model and the ImageNet class names."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nqhk2vYx6Ag0"
      },
      "outputs": [],
      "source": [
        "pretrained_model = tf.keras.applications.MobileNetV2(include_top=True,\n",
        "                                                     weights='imagenet')\n",
        "pretrained_model.trainable = False\n",
        "\n",
        "# ImageNet labels\n",
        "decode_predictions = tf.keras.applications.mobilenet_v2.decode_predictions"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f2cLrJH0zpfC"
      },
      "outputs": [],
      "source": [
        "# Helper function to preprocess the image so that it can be inputted in MobileNetV2\n",
        "def preprocess(image):\n",
        "  image = tf.cast(image, tf.float32)\n",
        "  image = tf.image.resize(image, (224, 224))\n",
        "  image = tf.keras.applications.mobilenet_v2.preprocess_input(image)\n",
        "  image = image[None, ...]\n",
        "  return image\n",
        "\n",
        "# Helper function to extract labels from probability vector\n",
        "def get_imagenet_label(probs):\n",
        "  return decode_predictions(probs, top=1)[0][0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iEZaMVFgSUA-"
      },
      "source": [
        "## Original image\n",
        "Let's use a sample image of a [Labrador Retriever](https://commons.wikimedia.org/wiki/File:YellowLabradorLooking_new.jpg) by Mirko [CC-BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/) from Wikimedia Common and create adversarial examples from it. The first step is to preprocess it so that it can be fed as an input to the MobileNetV2 model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "wpYrQ4OQSYWk"
      },
      "outputs": [],
      "source": [
        "image_path = tf.keras.utils.get_file('YellowLabradorLooking_new.jpg', 'https://storage.googleapis.com/download.tensorflow.org/example_images/YellowLabradorLooking_new.jpg')\n",
        "image_raw = tf.io.read_file(image_path)\n",
        "image = tf.image.decode_image(image_raw)\n",
        "\n",
        "image = preprocess(image)\n",
        "image_probs = pretrained_model.predict(image)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mvPlta_uSbuI"
      },
      "source": [
        "Let's have a look at the image."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "99Jc-SNoSZot"
      },
      "outputs": [],
      "source": [
        "plt.figure()\n",
        "plt.imshow(image[0] * 0.5 + 0.5)  # To change [-1, 1] to [0,1]\n",
        "_, image_class, class_confidence = get_imagenet_label(image_probs)\n",
        "plt.title('{} : {:.2f}% Confidence'.format(image_class, class_confidence*100))\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kElVTbF690CF"
      },
      "source": [
        "## Create the adversarial image\n",
        "\n",
        "### Implementing fast gradient sign method\n",
        "The first step is to create perturbations which will be used to distort the original image resulting in an adversarial image. As mentioned, for this task, the gradients are taken with respect to the image."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FhZxlOnuBCVr"
      },
      "outputs": [],
      "source": [
        "loss_object = tf.keras.losses.CategoricalCrossentropy()\n",
        "\n",
        "def create_adversarial_pattern(input_image, input_label):\n",
        "  with tf.GradientTape() as tape:\n",
        "    tape.watch(input_image)\n",
        "    prediction = pretrained_model(input_image)\n",
        "    loss = loss_object(input_label, prediction)\n",
        "\n",
        "  # Get the gradients of the loss w.r.t to the input image.\n",
        "  gradient = tape.gradient(loss, input_image)\n",
        "  # Get the sign of the gradients to create the perturbation\n",
        "  signed_grad = tf.sign(gradient)\n",
        "  return signed_grad"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RbuftX0eSlDQ"
      },
      "source": [
        "The resulting perturbations can also be visualised."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rVjnb6M7Smv4"
      },
      "outputs": [],
      "source": [
        "# Get the input label of the image.\n",
        "labrador_retriever_index = 208\n",
        "label = tf.one_hot(labrador_retriever_index, image_probs.shape[-1])\n",
        "label = tf.reshape(label, (1, image_probs.shape[-1]))\n",
        "\n",
        "perturbations = create_adversarial_pattern(image, label)\n",
        "plt.imshow(perturbations[0] * 0.5 + 0.5);  # To change [-1, 1] to [0,1]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DKKSFHjwCyQH"
      },
      "source": [
        "Let's try this out for different values of epsilon and observe the resultant image. You'll notice that as the value of epsilon is increased, it becomes easier to fool the network. However, this comes as a trade-off which results in the perturbations becoming more identifiable."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "dBtG0Kl5SspV"
      },
      "outputs": [],
      "source": [
        "def display_images(image, description):\n",
        "  _, label, confidence = get_imagenet_label(pretrained_model.predict(image))\n",
        "  plt.figure()\n",
        "  plt.imshow(image[0]*0.5+0.5)\n",
        "  plt.title('{} \\n {} : {:.2f}% Confidence'.format(description,\n",
        "                                                   label, confidence*100))\n",
        "  plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3DA8g-Zp69J4"
      },
      "outputs": [],
      "source": [
        "epsilons = [0, 0.01, 0.1, 0.15]\n",
        "descriptions = [('Epsilon = {:0.3f}'.format(eps) if eps else 'Input')\n",
        "                for eps in epsilons]\n",
        "\n",
        "for i, eps in enumerate(epsilons):\n",
        "  adv_x = image + eps*perturbations\n",
        "  adv_x = tf.clip_by_value(adv_x, -1, 1)\n",
        "  display_images(adv_x, descriptions[i])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fxt5VfnXHQT6"
      },
      "source": [
        "## Next steps\n",
        "\n",
        "Now that you know about adversarial attacks, try this out on different datasets and different architectures. You may also create and train your own model, and then attempt to fool it using the same method. You can also try and see how the confidence in predictions vary as you change epsilon.\n",
        "\n",
        "Though powerful, the attack shown in this tutorial was just the start of research into adversarial attacks, and there have been multiple papers creating more powerful attacks since then. In addition to adversarial attacks, research has also led to the creation of defenses, which aims at creating robust machine learning models. You may review this [survey paper](https://arxiv.org/abs/1810.00069) for a comprehensive list of adversarial attacks and defences.\n",
        "\n",
        "For many more implementations of adversarial attacks and defenses, you may want to see the adversarial example library [CleverHans](https://github.com/tensorflow/cleverhans)."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "adversarial_fgsm.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}