examples/colab/Wiki_toxicity_fairness.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xrsx7fMbWs8u"
      },
      "source": [
        "##### Copyright 2018 The TensorFlow Constrained Optimization Authors. All Rights Reserved.\n",
        "\n",
        "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n",
        "\n",
        "\u003e http://www.apache.org/licenses/LICENSE-2.0\n",
        "\n",
        "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bqdaOVRxWs8v"
      },
      "source": [
        "## Wiki Talk Comments Toxicity Prediction\n",
        "\n",
        "In this example, we consider the task of predicting whether a discussion comment posted on a Wiki talk page contains toxic content (i.e. contains content that is “rude, disrespectful or unreasonable”). We use a public \u003ca href=\"https://figshare.com/articles/Wikipedia_Talk_Labels_Toxicity/4563973\"\u003edataset\u003c/a\u003e released by the \u003ca href=\"https://conversationai.github.io/\"\u003eConversation AI\u003c/a\u003e project, which contains over 100k comments from the English Wikipedia that are annotated by crowd workers.\n",
        "\n",
        "One of the challenges with this dataset is that a very small proportion of the comments in the dataset cover sensitive topics such as sexuality or religion. So, training a neural network model on this dataset by optimizing a simple cross-entropy objective often leads to disparate performance on the smaller sensitive topics. We shall see that by imposing constraints during training, we can train a *fairer* model that enjoys equitable performance across the different topic groups. We shall use the TFCO library to optimize for two different fairness goals during training.\n",
        "\n",
        "Let's first install and import the relevant libraries."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nd_Y6CTnWs8w"
      },
      "outputs": [],
      "source": [
        "import io\n",
        "import os\n",
        "import shutil\n",
        "import sys\n",
        "import tempfile\n",
        "import zipfile\n",
        "from urllib import request\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import time\n",
        "\n",
        "import tensorflow as tf\n",
        "import tensorflow.keras as keras\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.preprocessing import sequence\n",
        "from tensorflow.keras.preprocessing import text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "R1ZL8BcBgN2t"
      },
      "outputs": [],
      "source": [
        "!pip install git+https://github.com/google-research/tensorflow_constrained_optimization\n",
        "import tensorflow_constrained_optimization as tfco"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DNAbLqqh1bpU"
      },
      "outputs": [],
      "source": [
        "if not tf.executing_eagerly():\n",
        "  tf.enable_eager_execution()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YUJyWaAwWs83"
      },
      "source": [
        "## Hyper-parameters\n",
        "\n",
        "Let's fix some hyper-parameters needed for the data preprocessing and model training."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "1aXlwlqTWs84"
      },
      "outputs": [],
      "source": [
        "hparams = {\n",
        "    \"batch_size\": 128,\n",
        "    \"cnn_filter_sizes\": [128, 128, 128],\n",
        "    \"cnn_kernel_sizes\": [5, 5, 5],\n",
        "    \"cnn_pooling_sizes\": [5, 5, 40],\n",
        "    \"embedding_dim\": 100,\n",
        "    \"embedding_trainable\": False,\n",
        "    \"learning_rate\": 0.005,\n",
        "    \"max_num_words\": 10000,\n",
        "    \"max_sequence_length\": 250\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dkSm-RIaWs86"
      },
      "source": [
        "## Load and pre-process dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DIe2JRDeWs87"
      },
      "source": [
        "We next download the dataset and preprocess it. The train, test and validations sets are provided as separate CSV files."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "id": "rcd2CV7pWs88",
        "outputId": "9aa7e6f0-ebaf-49e5-d41e-49d8c0cdbbd2"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\u003cdiv\u003e\n",
              "\u003cstyle scoped\u003e\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "\u003c/style\u003e\n",
              "\u003ctable border=\"1\" class=\"dataframe\"\u003e\n",
              "  \u003cthead\u003e\n",
              "    \u003ctr style=\"text-align: right;\"\u003e\n",
              "      \u003cth\u003e\u003c/th\u003e\n",
              "      \u003cth\u003erev_id\u003c/th\u003e\n",
              "      \u003cth\u003etoxicity\u003c/th\u003e\n",
              "      \u003cth\u003ecomment\u003c/th\u003e\n",
              "      \u003cth\u003eyear\u003c/th\u003e\n",
              "      \u003cth\u003elogged_in\u003c/th\u003e\n",
              "      \u003cth\u003ens\u003c/th\u003e\n",
              "      \u003cth\u003esample\u003c/th\u003e\n",
              "      \u003cth\u003esplit\u003c/th\u003e\n",
              "      \u003cth\u003eis_toxic\u003c/th\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/thead\u003e\n",
              "  \u003ctbody\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e0\u003c/th\u003e\n",
              "      \u003ctd\u003e2232.0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.1\u003c/td\u003e\n",
              "      \u003ctd\u003eThis: :One can make an analogy in mathematical...\u003c/td\u003e\n",
              "      \u003ctd\u003e2002\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003earticle\u003c/td\u003e\n",
              "      \u003ctd\u003erandom\u003c/td\u003e\n",
              "      \u003ctd\u003etrain\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e1\u003c/th\u003e\n",
              "      \u003ctd\u003e4216.0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.0\u003c/td\u003e\n",
              "      \u003ctd\u003e`  :Clarification for you  (and Zundark's righ...\u003c/td\u003e\n",
              "      \u003ctd\u003e2002\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003euser\u003c/td\u003e\n",
              "      \u003ctd\u003erandom\u003c/td\u003e\n",
              "      \u003ctd\u003etrain\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e2\u003c/th\u003e\n",
              "      \u003ctd\u003e26547.0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.0\u003c/td\u003e\n",
              "      \u003ctd\u003e`This is such a fun entry.   Devotchka  I once...\u003c/td\u003e\n",
              "      \u003ctd\u003e2002\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003earticle\u003c/td\u003e\n",
              "      \u003ctd\u003erandom\u003c/td\u003e\n",
              "      \u003ctd\u003etrain\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e3\u003c/th\u003e\n",
              "      \u003ctd\u003e37330.0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.3\u003c/td\u003e\n",
              "      \u003ctd\u003e`   I fixed the link; I also removed ``homeopa...\u003c/td\u003e\n",
              "      \u003ctd\u003e2002\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003earticle\u003c/td\u003e\n",
              "      \u003ctd\u003erandom\u003c/td\u003e\n",
              "      \u003ctd\u003etrain\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e4\u003c/th\u003e\n",
              "      \u003ctd\u003e37346.0\u003c/td\u003e\n",
              "      \u003ctd\u003e0.1\u003c/td\u003e\n",
              "      \u003ctd\u003e`If they are ``indisputable`` then why does th...\u003c/td\u003e\n",
              "      \u003ctd\u003e2002\u003c/td\u003e\n",
              "      \u003ctd\u003eTrue\u003c/td\u003e\n",
              "      \u003ctd\u003earticle\u003c/td\u003e\n",
              "      \u003ctd\u003erandom\u003c/td\u003e\n",
              "      \u003ctd\u003etrain\u003c/td\u003e\n",
              "      \u003ctd\u003eFalse\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/tbody\u003e\n",
              "\u003c/table\u003e\n",
              "\u003c/div\u003e"
            ],
            "text/plain": [
              "    rev_id  toxicity  ...  split  is_toxic\n",
              "0   2232.0       0.1  ...  train     False\n",
              "1   4216.0       0.0  ...  train     False\n",
              "2  26547.0       0.0  ...  train     False\n",
              "3  37330.0       0.3  ...  train     False\n",
              "4  37346.0       0.1  ...  train     False\n",
              "\n",
              "[5 rows x 9 columns]"
            ]
          },
          "execution_count": 5,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "toxicity_data_url = (\"https://raw.githubusercontent.com/conversationai/\"\n",
        "                     \"unintended-ml-bias-analysis/master/data/\")\n",
        "\n",
        "data_train = pd.read_csv(toxicity_data_url + \"wiki_train.csv\")\n",
        "data_test = pd.read_csv(toxicity_data_url + \"wiki_test.csv\")\n",
        "data_vali = pd.read_csv(toxicity_data_url + \"wiki_dev.csv\")\n",
        "\n",
        "data_train.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ojo617RIWs8_"
      },
      "source": [
        "The `comment` column contains the discussion comments and `is_toxic` column indicates whether or not a comment is annotated as toxic.\n",
        "\n",
        "In the following, we (i) separate out the labels, (ii) tokenize the text comments, and (iii) identify comments that contain sensitive topic terms.\n",
        "\n",
        "First, we separate the labels from the train, test and validation sets. The labels are all binary (0 or 1)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mxo7ny90Ws9A"
      },
      "outputs": [],
      "source": [
        "labels_train = data_train[\"is_toxic\"].values.reshape(-1, 1) * 1.0\n",
        "labels_test = data_test[\"is_toxic\"].values.reshape(-1, 1) * 1.0\n",
        "labels_vali = data_vali[\"is_toxic\"].values.reshape(-1, 1) * 1.0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "alrWi6jUWs9C"
      },
      "source": [
        "Next, we tokenize the textual comments using the `Tokenizer` provided by `Keras`. We use the training set comments alone to build a vocabulary of tokens, and use them to convert all the comments into a (padded) sequence of tokens of the same length."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yvOTBsrHWs9D"
      },
      "outputs": [],
      "source": [
        "tokenizer = text.Tokenizer(num_words=hparams[\"max_num_words\"])\n",
        "tokenizer.fit_on_texts(data_train[\"comment\"])\n",
        "\n",
        "def prep_text(texts, tokenizer, max_sequence_length):\n",
        "    # Turns text into into padded sequences.\n",
        "    text_sequences = tokenizer.texts_to_sequences(texts)\n",
        "    return sequence.pad_sequences(text_sequences, maxlen=max_sequence_length)\n",
        "\n",
        "text_train = prep_text(data_train[\"comment\"], tokenizer, hparams[\"max_sequence_length\"])\n",
        "text_test = prep_text(data_test[\"comment\"], tokenizer, hparams[\"max_sequence_length\"])\n",
        "text_vali = prep_text(data_vali[\"comment\"], tokenizer, hparams[\"max_sequence_length\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cn5zbgp-Ws9F"
      },
      "source": [
        "Finally, we identify comments belonging to certain sensitive topic groups. We consider a subset of the \u003ca href=\"https://github.com/conversationai/unintended-ml-bias-analysis/blob/master/unintended_ml_bias/bias_madlibs_data/adjectives_people.txt\"\u003eidentity terms\u003c/a\u003e provided with the dataset and group them into\n",
        "four broad topic groups: *sexuality*, *gender identity*, *religion* and *race*."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EnFfV2gEWs9G"
      },
      "outputs": [],
      "source": [
        "terms = {\n",
        "    'sexuality': ['gay', 'lesbian', 'bisexual', 'homosexual', 'straight', 'heterosexual'],\n",
        "    'gender identity': ['trans', 'transgender', 'cis', 'nonbinary'],\n",
        "    'religion': ['christian', 'muslim', 'jewish', 'buddhist', 'catholic', 'protestant', 'sikh', 'taoist'],\n",
        "    'race': ['african', 'african american', 'black', 'white', 'european', 'hispanic', 'latino', 'latina',\n",
        "             'latinx', 'mexican', 'canadian', 'american', 'asian', 'indian', 'middle eastern', 'chinese',\n",
        "             'japanese']}\n",
        "\n",
        "group_names = list(terms.keys())\n",
        "num_groups = len(group_names)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ooI3F5M4Ws9I"
      },
      "source": [
        "We then create separate group membership matrices for the train, test and validation sets, where the rows correspond to comments, the columns correspond to the four sensitive groups, and each entry is a boolean indicating whether the comment contains a term from the topic group."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zO7PyNckWs9J"
      },
      "outputs": [],
      "source": [
        "def get_groups(text):\n",
        "    # Returns a boolean NumPy array of shape (n, k), where n is the number of comments,\n",
        "    # and k is the number of groups. Each entry (i, j) indicates if the i-th comment\n",
        "    # contains a term from the j-th group.\n",
        "    groups = np.zeros((text.shape[0], num_groups))\n",
        "    for ii in range(num_groups):\n",
        "        groups[:, ii] = text.str.contains('|'.join(terms[group_names[ii]]), case=False)\n",
        "    return groups\n",
        "\n",
        "groups_train = get_groups(data_train[\"comment\"])\n",
        "groups_test = get_groups(data_test[\"comment\"])\n",
        "groups_vali = get_groups(data_vali[\"comment\"])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GFAI6AB9Ws9L"
      },
      "source": [
        "As shown below, all four topic groups constitute only a small fractions of the overall dataset, and have varying proportions of toxic comments."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 190
        },
        "id": "8Ug4u_P9Ws9M",
        "outputId": "9013a0a9-5b85-47a0-89c8-56504a32e945"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overall label proportion = 9.7%\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\u003cdiv\u003e\n",
              "\u003cstyle scoped\u003e\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "\u003c/style\u003e\n",
              "\u003ctable border=\"1\" class=\"dataframe\"\u003e\n",
              "  \u003cthead\u003e\n",
              "    \u003ctr style=\"text-align: right;\"\u003e\n",
              "      \u003cth\u003e\u003c/th\u003e\n",
              "      \u003cth\u003eTopic group\u003c/th\u003e\n",
              "      \u003cth\u003eGroup proportion\u003c/th\u003e\n",
              "      \u003cth\u003eLabel proportion\u003c/th\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/thead\u003e\n",
              "  \u003ctbody\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e0\u003c/th\u003e\n",
              "      \u003ctd\u003esexuality\u003c/td\u003e\n",
              "      \u003ctd\u003e1.30%\u003c/td\u003e\n",
              "      \u003ctd\u003e37.0%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e1\u003c/th\u003e\n",
              "      \u003ctd\u003egender identity\u003c/td\u003e\n",
              "      \u003ctd\u003e5.34%\u003c/td\u003e\n",
              "      \u003ctd\u003e7.7%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e2\u003c/th\u003e\n",
              "      \u003ctd\u003ereligion\u003c/td\u003e\n",
              "      \u003ctd\u003e1.87%\u003c/td\u003e\n",
              "      \u003ctd\u003e8.6%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e3\u003c/th\u003e\n",
              "      \u003ctd\u003erace\u003c/td\u003e\n",
              "      \u003ctd\u003e4.82%\u003c/td\u003e\n",
              "      \u003ctd\u003e9.2%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/tbody\u003e\n",
              "\u003c/table\u003e\n",
              "\u003c/div\u003e"
            ],
            "text/plain": [
              "       Topic group Group proportion Label proportion\n",
              "0        sexuality            1.30%            37.0%\n",
              "1  gender identity            5.34%             7.7%\n",
              "2         religion            1.87%             8.6%\n",
              "3             race            4.82%             9.2%"
            ]
          },
          "execution_count": 10,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "print(\"Overall label proportion = %.1f%%\" % (labels_train.mean() * 100))\n",
        "\n",
        "group_stats = []\n",
        "for ii in range(num_groups):\n",
        "    group_proportion = groups_train[:, ii].mean()\n",
        "    group_pos_proportion = labels_train[groups_train[:, ii] == 1].mean()\n",
        "    group_stats.append([group_names[ii],\n",
        "                        \"%.2f%%\" % (group_proportion * 100),\n",
        "                        \"%.1f%%\" % (group_pos_proportion * 100)])\n",
        "group_stats = pd.DataFrame(group_stats,\n",
        "                           columns=[\"Topic group\", \"Group proportion\", \"Label proportion\"])\n",
        "group_stats"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aG5ZKKrVWs9O"
      },
      "source": [
        "In particular, only 1.3% of the dataset contains comments related to sexuality. Among them, 37% of the comments have been annotated as being toxic. Note that this is significantly larger than the overall proportion of comments annotated as toxic."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5DkJpKaLWs9P"
      },
      "source": [
        "## Build CNN toxicity prediction model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "niJ4KIJgWs9Q"
      },
      "source": [
        "Having prepared the dataset, we now build a `Keras` model for prediction toxicity. The model we use is a convolutional neural network (CNN) with the same architecture used by the Conversation AI project for their debiasing analysis. We adapt \u003ca href=\"https://github.com/conversationai/unintended-ml-bias-analysis/blob/master/unintended_ml_bias/model_tool.py\"\u003ecode\u003c/a\u003e provided by them to construct the model layers.\n",
        "\n",
        "The model uses an embedding layer to convert the text tokens to fixed-length vectors. This layer converts the input text sequence into a sequence of vectors, and passes them through several layers of convolution and pooling operations, followed by a final fully-connected layer.\n",
        "\n",
        "We make use of pre-trained GloVe word vector emebeddings, which we download below. This may take a few minutes to complete!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yevbBL2oWs9Q"
      },
      "outputs": [],
      "source": [
        "zip_file_url = \"http://nlp.stanford.edu/data/glove.6B.zip\"\n",
        "zip_file = request.urlopen(zip_file_url)\n",
        "archive = zipfile.ZipFile(io.BytesIO(zip_file.read()))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a11-YWDnWs9S"
      },
      "source": [
        "We use the the downloaded GloVe embeddings to create an embedding matrix, where the rows contain the word embeddings for the tokens in the `Tokenizer`'s vocabulary. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bBS74MMYWs9T"
      },
      "outputs": [],
      "source": [
        "embeddings_index = {}\n",
        "glove_file = \"glove.6B.100d.txt\"\n",
        "\n",
        "with archive.open(glove_file) as f:\n",
        "    for line in f:\n",
        "        values = line.split()\n",
        "        word = values[0].decode(\"utf-8\")\n",
        "        coefs = np.asarray(values[1:], dtype=\"float32\")\n",
        "        embeddings_index[word] = coefs\n",
        "\n",
        "embedding_matrix = np.zeros((len(tokenizer.word_index) + 1, hparams[\"embedding_dim\"]))\n",
        "num_words_in_embedding = 0\n",
        "for word, i in tokenizer.word_index.items():\n",
        "    embedding_vector = embeddings_index.get(word)\n",
        "    if embedding_vector is not None:\n",
        "        num_words_in_embedding += 1\n",
        "        embedding_matrix[i] = embedding_vector"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t9NVp-_eWs9V"
      },
      "source": [
        "We are now ready to specify the `Keras` layers. We write a function to create a new model, which we will invoke whenever we wish to train a new model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_f_DhA6OWs9W"
      },
      "outputs": [],
      "source": [
        "def create_model():\n",
        "    model = keras.Sequential()\n",
        "\n",
        "    # Embedding layer.\n",
        "    embedding_layer = layers.Embedding(\n",
        "        embedding_matrix.shape[0], embedding_matrix.shape[1], weights=[embedding_matrix],\n",
        "        input_length=hparams[\"max_sequence_length\"], trainable=hparams['embedding_trainable'])\n",
        "    model.add(embedding_layer)\n",
        "\n",
        "    # Convolution layers.\n",
        "    for filter_size, kernel_size, pool_size in zip(\n",
        "        hparams['cnn_filter_sizes'], hparams['cnn_kernel_sizes'], hparams['cnn_pooling_sizes']):\n",
        "\n",
        "        conv_layer = layers.Conv1D(filter_size, kernel_size, activation='relu', padding='same')\n",
        "        model.add(conv_layer)\n",
        "\n",
        "        pooled_layer = layers.MaxPooling1D(pool_size, padding='same')\n",
        "        model.add(pooled_layer)\n",
        "\n",
        "    # Add a flatten layer, a fully-connected layer and an output layer.\n",
        "    model.add(layers.Flatten())\n",
        "    model.add(layers.Dense(128, activation='relu'))\n",
        "    model.add(layers.Dense(1))\n",
        "\n",
        "    return model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X-_fKjDtWs9Y"
      },
      "source": [
        "## Evaluation and plotting functions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k009haGaWs9Z"
      },
      "source": [
        "Before proceeding to training the model, we will write functions to evaluate the overall error rate, the overall false negative rate and the overall false positive rate for the given labels and predictions. We also include functions to evaluate the false positive and false negative rates for individual groups."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JF8-ENxOWs9Z"
      },
      "outputs": [],
      "source": [
        "def error_rate(labels, predictions):\n",
        "    # Returns error rate for given labels and predictions.\n",
        "    # Recall that the labels are binary (0 or 1).\n",
        "    signed_labels = (labels * 2) - 1\n",
        "    return np.mean(signed_labels * predictions \u003c= 0.0)\n",
        "\n",
        "\n",
        "def false_negative_rate(labels, predictions):\n",
        "    # Returns false negative rate for given labels and predictions.\n",
        "    if np.sum(labels \u003e 0) == 0:  # Any positives?\n",
        "        return 0.0\n",
        "    else:\n",
        "        return np.mean(predictions[labels \u003e 0] \u003c= 0)\n",
        "\n",
        "\n",
        "def false_positive_rate(labels, predictions):\n",
        "    # Returns false positive rate for given labels and predictions.\n",
        "    if np.sum(labels \u003c= 0) == 0:  # Any negatives?\n",
        "        return 0.0\n",
        "    else:\n",
        "        return np.mean(predictions[labels \u003c= 0] \u003e 0)\n",
        "\n",
        "\n",
        "def group_false_negative_rates(labels, predictions, groups):\n",
        "    # Returns list of per-group false negative rates for given labels,\n",
        "    # predictions and group membership matrix.\n",
        "    fnrs = []\n",
        "    for ii in range(groups.shape[1]):\n",
        "        labels_ii = labels[groups[:, ii] == 1]\n",
        "        if np.sum(labels_ii \u003e 0) \u003e 0:  # Any positives?\n",
        "            predictions_ii = predictions[groups[:, ii] == 1]\n",
        "            fnr_ii = np.mean(predictions_ii[labels_ii \u003e 0] \u003c= 0)\n",
        "        else:\n",
        "            fnr_ii = 0.0\n",
        "        fnrs.append(fnr_ii)\n",
        "    return fnrs\n",
        "\n",
        "\n",
        "def group_false_positive_rates(labels, predictions, groups):\n",
        "    # Returns list of per-group false positive rates for given labels,\n",
        "    # predictions and group membership matrix.\n",
        "    fprs = []\n",
        "    for ii in range(groups.shape[1]):\n",
        "        labels_ii = labels[groups[:, ii] == 1]\n",
        "        if np.sum(labels_ii \u003c= 0) \u003e 0:  # Any negatives?\n",
        "            predictions_ii = predictions[groups[:, ii] == 1]\n",
        "            fpr_ii = np.mean(predictions_ii[labels_ii \u003c= 0] \u003e 0)\n",
        "        else:\n",
        "            fpr_ii = 0.0\n",
        "        fprs.append(fpr_ii)\n",
        "    return fprs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lgScXhMUWs9c"
      },
      "source": [
        "We also write a function to generate a bar plot of the above metrics for given labels, predictions and group information. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8xIRl8f4Ws9d"
      },
      "outputs": [],
      "source": [
        "# Set plot fonts.\n",
        "plt.rcParams['font.sans-serif'] = \"Liberation Sans\"\n",
        "plt.rcParams['font.family'] = \"sans-serif\"\n",
        "plt.rcParams['font.size'] = 13\n",
        "\n",
        "# Color-blind friendly color palette from https://gist.github.com/thriveth/8560036\n",
        "cb_palette = [\"#377eb8\", \"#ff7f00\", \"#4daf4a\", \"#f781bf\", \"#a65628\", \"#984ea3\",\n",
        "              \"#e41a1c\", \"#dede00\", \"#999999\"]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "W3Sp7mpsWs9f"
      },
      "outputs": [],
      "source": [
        "def plot_metrics(labels, predictions, groups, group_names, title):\n",
        "    # Plot overall FPR, overall FNR, per-group FPRs, per-group FNRs for given labels,\n",
        "    # predictions, group membership matrix, group names, and plot title.\n",
        "\n",
        "    # Evaluate all the above metrics.\n",
        "    overall_error = error_rate(labels, predictions)\n",
        "    overall_fpr = false_positive_rate(labels, predictions)\n",
        "    overall_fnr = false_negative_rate(labels, predictions)\n",
        "    group_fprs = group_false_positive_rates(labels, predictions, groups)\n",
        "    group_fnrs = group_false_negative_rates(labels, predictions, groups)\n",
        "\n",
        "    # Plot bar plot overall and per-group FPRs and FNRs.\n",
        "    fig, ax = plt.subplots(1, 2, figsize=(8, 3))\n",
        "\n",
        "    fpr_percentages = [x * 100 for x in [overall_fpr] + group_fprs]\n",
        "    fnr_percentages = [x * 100 for x in [overall_fnr] + group_fnrs]\n",
        "\n",
        "    ax[0].barh(-np.arange(len(group_names) + 1), fpr_percentages, color=cb_palette)\n",
        "    ax[0].set_yticks(-np.arange(len(group_names) + 1))\n",
        "    ax[0].set_yticklabels([\"overall\"] + group_names)\n",
        "    ax[0].set_xlabel(\"False Positive Rate %\")\n",
        "    ax[0].set_xlim([0, max(10, max(fpr_percentages) + 1)])\n",
        "\n",
        "    ax[1].barh(-np.arange(len(group_names) + 1), fnr_percentages, color=cb_palette)\n",
        "    ax[1].set_yticks(-np.arange(len(group_names) + 1))\n",
        "    ax[1].set_yticklabels([\"overall\"] + group_names)\n",
        "    ax[1].set_xlabel(\"False Negative Rate %\")\n",
        "    ax[1].set_xlim([0, max(fnr_percentages) + 1])\n",
        "\n",
        "    fig.tight_layout()\n",
        "\n",
        "    # Specify accuracy rate in the plot title.\n",
        "    fig.suptitle(title + \": Accuracy = %.2f%%\" % ((1 - overall_error) * 100),\n",
        "                 x=0.55, y=1.1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8aWNc4CdWs9h"
      },
      "source": [
        "## Train unconstrained model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DuSA8qL7Ws9i"
      },
      "source": [
        "For the first model we train, we optimize a simple cross-entropy loss *without* any constraints.\n",
        "\n",
        "**Note**: The following code cell may take several minutes to run."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        },
        "id": "C9O4h6dOWs9r",
        "outputId": "cc34a425-ba55-43c1-df7d-e041ab5a1ff5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Train on 95692 samples\n",
            "Epoch 1/2\n",
            "95692/95692 [==============================] - 226s 2ms/sample - loss: 0.1551\n",
            "Epoch 2/2\n",
            "95692/95692 [==============================] - 224s 2ms/sample - loss: 0.1199\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "\u003ctensorflow.python.keras.callbacks.History at 0x7f94aaa94748\u003e"
            ]
          },
          "execution_count": 35,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Set random seed for reproducible results.\n",
        "np.random.seed(121212)\n",
        "# You may need to use \"tf.set_random_seed(212121)\", instead of the below, in\n",
        "# earlier versions of TensorFlow.\n",
        "tf.random.set_seed(212121)\n",
        "\n",
        "# Optimizer and loss.\n",
        "optimizer = tf.keras.optimizers.Adam(learning_rate=hparams[\"learning_rate\"])\n",
        "loss = lambda y_true, y_pred: tf.keras.losses.binary_crossentropy(\n",
        "    y_true, y_pred, from_logits=True)\n",
        "\n",
        "# Create, compile and fit model.\n",
        "model_unconstrained = create_model()\n",
        "model_unconstrained.compile(optimizer=optimizer, loss=loss)\n",
        "model_unconstrained.fit(\n",
        "    x=text_train, y=labels_train, batch_size=hparams[\"batch_size\"], epochs=2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p7AvIdktWs9t"
      },
      "source": [
        "Having trained the unconstrained model, we plot various evaluation metrics for the model on the test set."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 254
        },
        "id": "NnxMvQreMdwv",
        "outputId": "ab6b6e75-ab6b-4285-c73d-d0ac7fb3849f"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAADtCAYAAAClO5+KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3debxd093H8c83AxKJGKJmgpAYihIt\nVaSiqqi5JdV6ojW0VFsPrXoerXRAdTKUR2lVSM1qqhhaJZ2JGzUnEUNMNUUIIZLg9/yx1ml2Ts65\n99wk9567b77v12u/7tl7rb3X2vvss/bvrL3O3YoIzMzMzMqoR7MrYGZmZraoHMiYmZlZaTmQMTMz\ns9JyIGNmZmal5UDGzMzMSsuBjJmZmZWWAxnrkiSNl3RDnbR9JYWkQZ1bq65B0jRJZ3VCOa9LGt1G\nPULSh+ukH5nTxyxmPdr1fksanvNvtQhlSdLTef0N2rv+0k5Sf0kXSpou6W1Jt0vasJA+Kh/b6un1\nVrY5vs46IWlUId83JT0laY6kByR9spDWU9LF+Zx+QNJmNcr5u6Rjl+DhsE7iQMask0g6q7XAoB22\nBb6zBLazJLwNfL5O2iE5vUyGA6sBj1N/v6y+K4A9gUOBrfOy2yUtW5Vvjapp41a2uX+N/J8C5gF/\nB5B0AnAq8BNgU+A24MZCwDIS2BnYEfgzcHaxAEmHAAOA89u1t9YlOJAx6zzbLYmNRMQrEfHmktjW\nEvBn4GBJvYoLJa0LfAz4W1Nqtei+APyRdEE+pMl1KZXc87In8M2IuCUiJgMHA2sBnynmjYgXq6aX\n6203ImZU5wdOBC6IiKk529eBMRHxfxHxREScCNwPHJfTPwH8NiIeAs4BdpbUO9d7eeAM4BsR8e4S\nOhzWiRzIWOlJGiPpNkn7SZoi6a3cTbxJIU8/SedJelnSTEnjJA0upG8k6feS3pA0O6+/UzvL+ISk\neyS9mbuwb5e0aU6bBnwEOEVS5GXjJV0g6ZJc5iZ5+TGSJkuaJ+kFSb+RtGKhnP/cWspd9e9I2jTX\n521Jj0nat+oYHS3pwZz+VP4GW0wfJulfuVv+YUm7Nnj4/wisCOxetfxzpAvJC1XlrCbpckkzcr3v\nl7RfVZ4TJb2Yj/HvgJWr0ntK+m4+RrMlTZJ0aL0KFm5nrFgvT87XBzgQuBK4DNhYNW6bSfqGpCfy\nsbxX0m6FtLrnmWrc8pK0YvEWiaTReb+Ozut/OS8fkd/f2fnc+kP17RFJh+T3bnb++7m8/ApJE2vs\nxzhJN7d2TNpp0/z375UFEfFanv/4kipE0t7ANsD383x/YO1iudnNhXLXAJ7Jr58GegED8/xJwL0R\ncceSqqN1Lgcy1l1sDBxG+uY3HFiTBbuPfw2MIHVTfwQI4FZJvfM3sj8BvfO62wBPAbdJWr+RMiSt\nBNwA/BXYEtgBeJ3UvS3S7aA3gJ+RGtWKTwDTgSHAk5L2AM4FfgpsABxA6hL/SSv73iPX4zu57KnA\nxZL65rodDvwCGANsQboA/EDS0Tl9WeBGYCYwDDgc+B9guVbKrHgNuJ3Uk1F0CHB1cUE+DuPyvu6V\n6/JH4FpJH815Pg38CDgT2Ar4PQvfRvse6Rv5j/I2fp33d886dbyKdMxntrEv+wECro+IKcC9VN1e\nyoHFD0nH54PArcBNkobmLHXPszbKLhpA6tnYBrhC0iqki/KDwGbAR4G5wPX5mCJpd9L7e16u13nA\nWEm7ABcDW0saUtiPFYBdgUtrVUDSI5Jm1ZlurVPvSq9cda/Gq6RzeUn5X+BXEfFKA+UOktSD9L6+\nn5dX/ip/vr8KHL8E62edLSI8eepyEzAeuKFO2r6kC8SgPD8GmAMMLOQ5DZiRX68OvAccUEhfn/St\newPSRfh9YM1C+jLADGB0g2Vsneu0bSF9BeDDQI88/3ple4V9fAXoWVjWH9i0an9/BDxVmJ8GnJVf\nj8rl7l1I3y0v2yLPTwXGVm3zbODx/Hr3nH+zQnplf0YX16vaxrRc/iHAbGBAXr5lXnf9fNzG5OU7\n5uXbVW3n4UKeK0nfjovp51Te7/y+zAJ+UJXnRuCO/Hp4zr9VO8+5W4FLCvNfA14CehWWPQr8omq9\ni0hjNto6zxaqF6k3K4BReX50nt+ykKc3KfhbvrCs8p6tl+dvAX5fVa8fk87tHsCzwPcLaZ8jBaHL\n1TkW6wGD60xr1Vln8+pzsfD+tlSdrz8GJpPO/xuBwQ2+RzuRxsasW7V8OvDzqmXn5bL6AWOB/8nL\n1yUFgr2B60if45VyPZ4H7qwcV0/lmNwjY93F0xExvTD/CqlxAvgQqTG/r5IYEU9FxCER8STpov1M\nRPy7kD6X9A14ywbLeJTUdX21pP+WtHlEvBEREyLifep7MCLeK8zPAkZIuk/Sq5JmAf9dKKeelqp6\nAawkaQDp4nNXVf7xwIaS+gGbkC4OjxbS72fhb7j13Ei6YByY5w8BJkTEU1X5tiZd6Fuqlrcw/zhv\nAjxQI71iKLA8tfen3b9SqpC0Oql37ApJvZTG/FxLuq21W87TJ9fvvuK6EfGliLiVts+zRr1HOvcq\n25hHCoRuyrfcZgHX5+TKebFNjXp9KyLG5vPvElLwUnEAcE1EvFOrAhHxdEQ8Xmd6vs46DwP/AE6X\ntIGkZSV9n9RzOS9nmw38m9Rb8lnSubI68Nfcq9mWY4BxEfFM1fILgSPyLbie+fbT/jltHmks12dz\nL+V/kcZu7UjqNTuN1MvzDilYvo/Uc2ol4UDGuqr3SN3BtfTMf4sX2upfxxQf615pIOsNkO1Puu1T\n7c2c1mYZ+YLwMdI3428CD+WxDiPqlFkso+hEUm/JdcAupIvzL9vYRnXdKvVSof7nFW8PkHoJIF1E\n+gHvRERxf94nXXTaFBGzSLc+vpBvdYwk3c6p1h94OxYeUFk8zv1Y+DjPKrxeIf+9uWp/TgdWlrRM\nI3Wu4RDSeXUr6cI3j/TtvBfzby+1dR61ld6oWcX3It92Gwe8TOqN3Ip0+6+67NbKvZgUuG6XL+a7\nU+e20mI6mPR+PUH6TK1JGjg9HSAiroqItSLijIh4MCL+AOwDfAA4qLUN5/f2U6RjUe37wO9Ityrn\nAieQbgHOiog5pB6Z10jH6OukwOVs4MR8/g4HrsxB42V53kqiV9tZzJriFVL3di1rkS40r9RJr7Ut\nSI399BrpM0njEqoNaEcZRMSzwDGSvkq6pfQD0rfodSJiRoOb+SzpltoPKwtU9YugdqqMCzmZ1HNS\n7RlSwLKcJFUuoLnMvu0o50pSD8Z+pPfn6hp5ZgJ9JfWqCmYGFOo5u0a5xfemku8wYKEBrDTei1Tt\nC8wfY1K0C2mAdj/SuRPU7x1r6zwrBpgV/Rqo24F5e5+v9N5J2roqT7F3cOGCI56Q9BfS+K41gBcj\nou4vyiQ9Qv3P318j4lN1ynkW+IikNUhBxJuSbmHhXrbiOi9KmkHqDWnNzqSAd6ExOvmLxChJx5Fu\nBb4i6Vuk21rkYObjkgaSApqvkAKuSkBfDATfJN3ys5Jwj4x1VXcAwwqDKIH/XGAPB/6UG6dGPEy6\nwH20sJ3VJP0tXxAmAutIWqeQ3of0zfe+6o3VImmwpL0AIrkH+Bbporx+qysvaBkKF8Fcj8og1HaL\n9DPtKcDaxdsDpPE6M/MttCmk8QLFWzM7Mr/nqxG3kC4ApwH/iIjnauSZmLf5n5+h5x6c7Zh/nKeQ\nBkYXDS+8npLLWb1qf2YDr7RxG68mSR8k3dr6dUS0FCfgAtIXvv3zsZpE4TzK618t6Uu0fZ5Vev2K\nAUfNfyZYZRng9apbkJXbRJXz4v4a9TpH0smFRReTbreMpO3emD1I50Otqbo3qFLeMpI+J2lwRLyQ\ng5hVSe/fTTnP8ZKOr1pvHdIviNq6/bYT8Hytc0vSbpJ2iojXchAjUtB2UzFfvjU8ADgF+Fqh52sm\nsEp+vRop2LGyaPYgHU+eak2kxnsC6cL1aVIwsBPzL5gfLOQdA9xftf430un9n/nLgedIvyYaQuqG\nfoL0y5w+pF8pjSc11JsD15Au9ms0UgbwSVIv0VdyXYeQLhyvAH1ynmmkf9S1ZS5zPFUDmvM6L5Au\ncFuRAroxpMHIw0gBxzQWHuy7YmEbW+Vlw/P8EaQL/VHAhqRbYPcCt+b05UnB05/yvu8A/AV4iwYG\n+xbmL8nlHlv13owpzN8NPES66A4h/ZpqLnlwK+nWRJBuDQzOdX+SBQd3n0oaY3FQPta75ffy/Jw+\nnMKg2nysVwdUZz9+Shq3US99HPCH/PrIXN/Pk3oQ/pc0CHyzBs6zyi3Mm4CNSL09d+btjcrrjyYF\nLcXy/yu//wfnY3I28Ju8j8eQenU+mfMcn+t1OCmo2q2wneVJn525wAYd9Ll9iDROZkvSL8ruIo1p\nqaQfQ/qcHJv35aM5//PMHyy+H2kgcP+qbV9J6g2qVe4vSLfediWd4+fk92HFGnnPA35TtexM0vm/\ncS7n6ma3gZ7acd41uwKePNWbcgP9E1KQUbmVdBWwSVW+MbQdyKxA+nXJq6RvX7cCQwrpg/MFaxZp\njMadwNbtLONw4BFS0PAq6WfJ2xTSj8kXkhnAOtQOZNbKDepbpP8uO5J0K2Aq6SI4iHYGMnnZV4DH\n8nGcTuppKK6zE6lHYW7eh11Jt51Gt/L+TGPBQGZ30timNaqO25jC/AdIYyZeIwUA9wC7Vm33B7mO\nb5EGtR7IgoFMD+C7pP8HMo8UhJwBLJvTh7NgILPQMSqU1TOvf14r+/lFUlBQCWq/ncueTepJ+lQ7\nzrMDSYHZbFJQtwVpkOmonD6ahQOZ3sCvSIH1y6TxQD3ytt8BDs75Dsvv8TukgduH1tiXa4G/dOBn\ndkPSeV85zy9k4YDkaOZ/Tl7L58PahfSa71fe7rg65fbJx2g66fN7G7BxjXwfzPVarWr5yqQvSbNI\nn8u127vvnpo3Kb+JZmbWjeVxPtOAr0bElU2ujtkS40DGzKwby79SWod0C20t0v86eq/1tczKw4N9\nzcy6t/1IY1f6k/5Zn4MY61bcI2NmZmal5R4ZMzMzKy0HMmZmZlZaDmTMzMystBzImJmZWWk5kDEz\nM7PSciBjZmZmpeVAxszMzErLgYyZmZmVlgMZMzMzKy0HMmZmZlZaDmTMzMystBzImJmZWWk5kDEz\nM7PSciBjZmZmpeVAxszMzErLgYyZmZmVlgMZMzMzKy0HMmZmZlZaDmTMzMystHo1uwK2oIEDB8ag\nQYOaXQ2zNk2cOHF6RKza7HrYonN7Y2XRWnvjQKaLGTRoEC0tLc2uhlmbJD3d7DrY4nF7Y2XRWnvj\nW0tmZmZWWg5kzMzMrLQcyJiZmVlpOZAxMzOz0nIgY2ZmZqXlXy11MZP//QbbnXJ7h5dz9/c+2eFl\nmFnX1lntjS0+t9n1uUfGzMzMSsuBjJmZmZWWAxkzMzMrLQcyZmZmVloOZMzMzKy0HMiYmZlZaTmQ\nMTMzs9JyINOBJI2RdG71azOzJc3tjS2tHMiYmZlZaXX7QEbSbpImSpos6WFJh0u6XNLPq/K1SDpC\nUg9J35E0RdJTkv4kacOcZ7ikJyT9PG+vp6TNJf1Z0qOSpuU0NWdvzayZ3N6Ydb5uHchIWgO4Djg+\nIoYC+wBnAlOAAysNgKTBwGbA1cBxwEhg+4hYH/gzcHFhs2sBD0TE0Ih4D7gIuDUiNgU+AnwJ2Kud\n9TwyN2wt896eueg7bGZN4/bGrDm6dSADfBKYFBHjASLiCeBWoDfQB9gu5zsYuCkiZgKfAf4vImbk\ntDOBj0laM88vC1xVKGMH4Kd5+y8BjwAbtaeSEXFhRAyLiGG9+w5o3x6aWVfh9sasCbp7ILM68ErV\nsleBVUiNw2fysoOAS/PrlYCTclfuZOBeYHreFsBbEfFOYXv7AuMlTc35t6L7H1czW5jbG7Mm6O5P\nv34B+EDVslWBB4E/AldKuijnqTwC9jng6oi4oHpjkoZXzW9A6h7+VETcnpdNWJI7YGal4fbGrAm6\neyR/OzBE0o4AkoYAuwM3RsQ/gXnAycAVEfFuXuca4HBJK+R1tpV0SZ0Bdf2BAFpy3gOAQUC/jtsl\nM+ui3N6YNUG37pGJiBcl7Q+cKWl5UkNyREQ8kLNcDnwH2Kaw2q9I3br3SApgFvDtiIjqtiUiHpD0\na+BBSa8CvwVOAU6XNLUj983Muha3N2bNoYhodh2soN+aG8fmR/yiw8u5+3uf7PAyrHuTNDEihjW7\nHrboOqu9scW3tLfZrbU33f3WkpmZmXVjDmTMzMystBzImJmZWWk5kDEzM7PSciBjZmZmpdWtf35d\nRkPXXGGpH51uZp3D7Y11B+6RMTMzs9JyIGNmZmal5UDGzMzMSsuBjJmZmZWWAxkzMzMrLf9qqav5\n90QYXevBt2btNNrPUbM2uL1ZdP58dRnukTEzM7PSciBjZmZmpeVAxszMzErLgYyZmZmVlgMZMzMz\nKy0HMmZmZlZaDmTMzMystBzItJOkUZIezq/3k3RTIe3Y5tXMzLojtzlmrXMgsxgi4vqI2BtA0geA\n05pcJTPrxtzmmC2sNIGMpJ6SzpM0VdIkSfdIGiapj6SzJD0m6WlJ10laNa9zmaRfFrYxRtKY/Hqa\npAMLaecW0laQdHku5ylJv5c0sEadRkl6WFJfoAVYXtJkSV+XdH9V3h9JurIjjo2ZLXluc8zKoTSB\nDPBJ4BPAphGxCXAqcCDwY2ALYCtgfWAG8LO8zleBPSVtL+njwHDgaw2U9b/AisBmwMb59Un1MkfE\n28ChwNsRMRS4BBgiaetCtoOAS2utL+lISS2SWl55u4HamVln6JZtjtsb627K9Kyll4A1gP+SdHNE\n3ATcJOlF4PD8wUbSmcBESaMi4jVJRwAXAL2BwyLijQbK+jawTES8D7wv6a/A5o1WNCJel3QD8EXg\nPknbAX2A2+vkvxC4EGDYmvIDPMy6hm7Z5ri9se6mND0yETER+CxwAPCkpAmSdgBWAs7N3auTgeuA\nWcAqeb3bSPs5JyLuarC4LYBrc9fxZOBI2n+sxgCfk7Qs6ZvRZRHxXju3YWZN4jbHrBxKE8gARMSt\nEfEpUoNxG3AF8BxwVEQMzdOQiBgYEa8ASDqK9M3qrfxNqeI9oGdhfqXC6xuB+0ldykOB3yxCdf8I\nvAXsCXyGOreVzKzrcptj1vWVJpCR9EVJZ0vqERGzgbtz0jXAVyUtk/PtLemM/HoD4PvAEcDhwOmS\n1s/rPQtsnfMNAkYUihsA3BcR70rahNQw9GujinOBZSQtD5C7iMeS7p2/GhEPLPLOm1mnc5tjVg6l\nCWSA64HVgamSHgG+B3yB1GhMAx6UNAk4HrhcUg9SV+sZEfFkREwCzgHG5LRTgANyN+4ZwGWFsr4J\n/ELSo8CJwJeBLSX9opX6PQA8Cjwvafe8bAwwCH8zMisjtzlmJaAIj/XqKJL6kbqhh0TES42sM2xN\nRcuRHVsvW0qM7tjPtqSJETGsQwuxdmlvm+P2ZjF08OfLFtRae1OmHpky+h7wu0aDGDOzxeQ2x5Y6\nDmQ6gKTNJD0FbAqc0Oz6mFn35jbHlmZl+j8ypRERj5D+UZaZWYdzm2NLM/fImJmZWWk5kDEzM7PS\n8q2lrmbNbWB0S7NrYWZLA7c31g24R8bMzMxKy4GMmZmZlZYDGTMzMystBzJmZmZWWg5kzMzMrLT8\nq6Uu5vHXp7L3DXt2yLZv2ndch2zXzMqpI9sbs2oddQ1yj4yZmZmVlgMZMzMzKy0HMmZmZlZaDmTM\nzMystBzImJmZWWk5kDEzM7PSciBjZmZmpdUlAhlJwyXNWkLbmixpszpp4yWdsATK+JKkvvn16ZJO\nzq9XkfS5xd2+mXUstzlm3Ue3+4d4ETG0I7cvqSdwJnAj8HZEnFRI3gX4HHB5R9bBzLoOtzlmzdVm\nj4ykYyW9KGmSpG9JelTSgTltC0l3SXpM0pPFbx75m8i3Jd0i6QlJD0jaJKcNkHSdpGclTQA+WlXm\n/jn/45IekvTJQlpI+mb+FrTAeoX0Yfn1gbnsKZJ+Wdzf/E3m0lz3ZyRdJKlPThud036Z9/tZSQfk\nVe8B+gP/lLSHpDGSzpU0HDgfGJ7rfqWks6rqdrekL7d1zM2WZm5z3OaYtUergYykoaRvAp+IiE2A\nPsAGOa0v8Afg6ojYGNgBOE7SLoVNfB74fERsCDwGfCsvPwFYJW9rR+DjhTK3BcYCh0XEYOBY4BpJ\nAwrb3SAihkbEP1qp+wDgYuDoiBgCXMeCjdeYvP+bAoOBNYHiN539gF/l/T4D+GlefmD+u31E3FLJ\nHBHjgXOB8RGxZd7+IZKWyfUZBGwJXFWjrkdKapHUMveNufV2yazbc5vT8W2O2xvrbtrqkfk48K+I\neCjP/5j5t6N2AnpFxPkAEfECqXvz4ML6N0bEjPx6IrBufj0CuCoi5kXEHNIHsOIA4NaIuC9vdzww\nBSg+EOSGBvbtI8DsiLg9b+cPwFMAkpYH9gB+FBHvRsRcUoNQrPuDETExv24p1L1RfwTmAHvn+YOB\ncRHxWnXGiLgwIoZFxLBlVlimncWYdStuc5IOa3Pc3lh309YYmZWA6ZWZiJgtaXohrb+kyYX8ywF3\nF+aLH6B3gZ759SrAjELaK1Vljqjabv+8TsWrbdS7VhnFclYgBXHXS3ovL+uZ61+v7u0aGB0R70ka\nC3wRuBY4CPhue7ZhthRymzO/7m5zzBrQViDzBukDCICkZYGBefY54NVFHOj2GlDstl298Po54LaI\nGLkI222tjGI5LwHzgL0iYspiltOaMcDDknYA1gJu7cCyzLoDtzmLZwxuc2wp01bE/zdgW0kb5/lv\nkj6MABOAeZJGAkjqLelMSSMaKPfPwGcl9coN1aGFtOuA3SUNydtdVdJVktZocJ8qJgArSvp43s7u\n5K7aiHg/l3OcJOX0IyUd18B2KzeVV66TtmJlm7nBuhf4LXBFRLzbzn0wW9q4zVmY2xyzVrQayETE\n/cAPgT9J+hep2/SllBRzgE8DR0maAjwCLEtqiNpyBvA26f7xP4DiALZHgCOAqyRNAsYDf833wxuW\n75MfDlyU63cg6R5yxTFAP2ByTt+H1NC05QXgDmCipMOr0m4DNgJekLRSXjYGGARc2p76my2N3ObU\n5DbHrBWKiNYzSIpCJkkzgX0j4q6Orlx3kL8t/jz/qqBNKw4eEDv99GMdUpeb9h3XIdu1pZOkiREx\nrAO26zZnMbSnzenI9sas2uJcg1prb9r6+fUA4DVJ2+f5fQAB9y9ybZYi+X9EfI/5P6M0s1a4zVk8\nbnNsadTqYN+ImCnpCODifA/2HeCgWj8htgVJGgV8H/hdRIxtcnXMSsFtzqJzm2NLqzYfURAR1wDX\ndEJdupWIGMOC/6vCzBrgNmfRuM2xpVWXeGikmZmZ2aJwIGNmZmal1e2efl12g1fcyL8uMrNO4fbG\nugP3yJiZmVlpOZAxMzOz0nIgY2ZmZqXlQMbMzMxKy4GMmZmZlZZ/tdTFxAtv8s73xy/WNpb77vAl\nUhcz696WRHtj1hHacx1zj4yZmZmVlgMZMzMzKy0HMmZmZlZaDmTMzMystBzImJmZWWk5kDEzM7PS\nciBjZmZmpbVUBzKSRku6Ob/+qqTzG1hnP0k3dXztzKw7cXtj1jH8D/GyiDi3wXzXA9d3cHXMrBtz\ne2O25HS7HhlJgyS9J+l4SVMkrSfpUkmPSXpG0kWS+tRYr/htaQVJN0h6UdI9kr4saVZOGyXp4cJ6\nx0t6VNJkSXdL+mhePlzSk5KOkXRf3tbpnXUczKzjub0xa75uF8hkPYDeETEEODfPbwoMBtYETmpj\n/ZOAVYB1gd2Aw2plkrQPcBywa0QMBc4CrpfUN2dZC+gREVsDuwLflrReje0cKalFUssrb81s356a\nWbO5vTFrou4ayED6gC8P7AH8KCLejYi5pIbm4DbW3QW4MiLmRsRM4KI6+fYDroiIfwNExJV5+Yfz\n317AhTntYWAOqbFaQERcGBHDImLYqssPaHwPzayrcHtj1iTdeYzMq8AKpGDteknv5eU9geXaWHcl\nYHph/pk6+VYHHq1aNgNYDXgJmB0Rcwpp7+byzax7cXtj1iTdOZCB9OGeB+wVEVPasd4bpEapYq06\n+V4APlCZkSRgYF7enXu7zGxhbm/MmqBbn/wR8T5wHXBc/tBX7g8f18aqfwMOltRT0grAF+vkuw4Y\nKWm1PP954B3g3sWvvZmVidsbs+bo7j0yAMcAZwOTc9vyOHB0G+ucClwGPJmny4AtqjNFxO8lDQbu\nlNQTeAXYJyJm57LMbOni9saskykiml2HLkmSIh8cSfsDZ0bEQr8AWNK2WWtI/P2oCxZrG8t9d/iS\nqYxZKyRNjIhhza5Hd1Dm9sasI1Rfx1prb7r1raVFJekwoEVSn/zNZyTwlyZXy8y6Ibc3ZovHgUxt\nlwEtpF8ITCbdgjuhqTUys+7K7Y3ZYlgaxsi0W/7/D0c1ux5m1v25vTFbPO6RMTMzs9JyIGNmZmal\n5VtLXYzW6O9fHZlZp3B7Y92Be2TMzMystBzImJmZWWk5kDEzM7PSciBjZmZmpeVAxszMzErLv1rq\nYmY8+QhXHLRJp5Q18qpJnVKOmXVNndneWPl11WuGe2TMzMystBzImJmZWWk5kDEzM7PSciBjZmZm\npeVAxszMzErLgYyZmZmVlgMZMzMzKy0HMmZmZlZaDmTMzMystBzIVJE0SNJ7ko6XNEXSupLGSZok\naZqksZKWy3l7Svq5pCclPSbpN5KWzWlbSLorL39S0gnN3TMz64rc5pgtHgcytfUAekfEEOAnwLMR\nsQmwGbA9cETO93VgGDAU2BRYGzhJUl/gD8DVEbExsANwnKRdahUm6UhJLZJa3pzzbkful5l1TZ3W\n5ri9se7Gz1qq7/r8dyT5OEXEW5ImABvltAOAsRExF0DSPsC7wAigV0Scn9d7QdLlwMHAndUFRcSF\nwIUAG6zcJzpsj8ysK+uUNsftjXU3DmTqezX/3Rk4WdK6wHvAGsDYnDYQeL2yQkTMBpC0EtBf0uTC\n9pYD7u7oSptZabnNMVsEDmRakbtrbwa+AVwUEe9LurqQ5SVg1UL+AUBf4Dng1YgY2pn1NbNyc5tj\n1n4eI9O63qRGoiU3KDsA25nHdDgAABAZSURBVAH9cvrvgFGS+kjqCVwKHA1MAOZJGgkgqbekMyWN\n6PxdMLMScZtj1k7ukWlFRMyUdAowTtIM4HbgWOBSSY8CPwfWAh4DZgP/AE6LiDmSPg2cI2k0IOAO\n4G9N2A0zKwm3OWbtpwiP9epKNli5T5z6iUGdUtbIqyZ1SjnWPUmaGBHDml0PW3Sd2d5Y+TXzmtFa\ne+NbS2ZmZlZaDmTMzMystBzImJmZWWk5kDEzM7PSciBjZmZmpeWfX3cxK2+wGSOvaml2NcxsKeD2\nxroD98iYmZlZaTmQMTMzs9JyIGNmZmal5UDGzMzMSsuBjJmZmZWWAxkzMzMrLT80souR9CYwpdn1\naMBAYHqzK9Eg17VjDImI/s2uhC26ErU3zVSmz2SzdMYxWi8iVq2V4P8j0/VMKcMThSW1lKGe4Lp2\nFEn+ByTlV4r2ppnK9JlslmYfI99aMjMzs9JyIGNmZmal5UCm67mw2RVoUFnqCa5rRylTXa02v4dt\n8zFqW1OPkQf7mpmZWWm5R8bMzMxKy4GMmZmZlZYDmU4maVtJd0t6XNIkSYfWyXdoTn9c0j2Stm1C\nXUfksidLmirpuBp5Rkl6O+epTLd0cj0HSYqqOkyWNLBG3m9Jeiwf1zslbdjJdf1YjXpOl3RRVb6m\nHVdJR0p6S9IJhWUDJd0o6Yl8LvxMUs32o9Fz3JrH79HC6rV37Tn3lxaSVpT0vKQxeb65xygiPHXS\nBCwLPAccnOcHA68DH6zKt0VevlGePwh4FlimE+u6OjALGJHnNwTeBLavyjcKGN/k4zooncpt5tsL\neAb4QJ4/EZjQ5LovB0wGtu0KxxU4D7gamAicUFh+LXA+IGB5oAU4usb6DZ3jnpo3+T2qeUzqtneN\nnvtL0wRcAjwJjMnzTT1GS3VU2QQjACLiyvz3cWAcMLIq3+eBcRExNee7inSCDO+0msJ7wBci4k+5\nDk8Aj5KCrLI6FBgbES/n+V8AH5K0cRPrdDJwV0Tc28Q6FF0REZ8lNeIASOoP7Av8PJK3gAtI52m1\nRs9xax6/Rwur1959mMbP/aWCpL1Iwe9v83x72ocO4UCmcw0FplYtewzYrEa+x6qWTa2Rr8NExCsR\ncX1lPt+C2Rz4e43s60i6VdIUSXdI+nBn1bNI0lhJD0tqkfSFGlkWOK4R8Tbpm2mnHdciSasBXwa+\nVydLpx/XiPhbjcUb5b9PFJbVOm+h8XPcmsfvUZVW2rt/5UWNnPvdnqSVgLOBw4D38+L2tA8dwoFM\n51oemF21bHZevij5OoWktYHfAz+OiIerkh8HbiTdCtkEuBm4JZ/wnWUW8BvgZxGxOfAN4AJJO1Xl\n61LHFfgm8NuIeLFGWlc4rhXLA3Mj4v3CsnrHrasdY1uY36NWFNs7IGj83F8anA2cFxHFL9rtaR86\nhAOZzjUL6FO1bPm8fFHydThJWwP/BC6JiIV6DiLibxHx3xHxUkS8HxFnAe8CO3RWHSNiekR8KSLu\nr9QJuAnYuyprVzquPUm3ui6tld4VjmvBLGDZqsF79Y5blznGVpffozpqtHftOfe7NUmfBjYAzqpK\navoxciDTuR4BqsdjbAI8WCPfkMqMJJG6g6vzdaj8ob4F+EZEnFEnz7qSVq9eDMzr6PoV6rCypMFV\ni3vUqEP1ce0PrAU81LE1rGlnYE5E3FcrsSsc14LHSGMIise41nkLjZ/j1jx+j2qo096159zv7g4i\nBTJPSppG6vk+kNQb3txj1OzRz0vTBPQGngYOy/Nbkn4tsHFVvk0p/IoAOIJ0/7FXJ9Z1uVzmAW3k\n+wlwJ9A3zx8GvACs0Il13RN4ifSYd0j3tt8EPlKVbzfgeWDtPH8q8OcmnQvfAW7vqscVGM+Cv1q6\nHBhDCqZWBB6unMdV6zV0jntq3uT3qOYxqdveNXruL20TMJr5v1pq6jHyIwo6maStgP8DVgXeAUZH\nxO8knQ68FRE/zPlGkn7RsgzpAnZ0LDw+pSPrOZI0Kr16UOCVpJ9vvhURP5S0HOm+6QjSrY8Xgf+O\nOj0NHVjfrwNfybPvAD+KiCslXUr6VdDFOd9xpAG2PUj7dmREPNeZdc31uABYNiJGFZb95xxoxnHN\nt7seybPrkrqGZwDXk8YL/BrYivTt60rglIgISfuRGq2983ZqnuMdVW9rP79HC2qjvTubOud+p1ay\ni5E0GhgUEaPy2L2mHSMHMmZmZlZaHiNjZmZmpeVAxszMzErLgYyZmZmVlgMZMzMzKy0HMmZmZlZa\nDmTMzMystBzILCUkTZP0vKTJVdMRDa574BKsy3BJ7xfqMFXSI43UpY3tfljSxML8lyT1za9Pl3Ty\n4ta9qrzIx6ayH1PyAyv3a3D9XpK+0nbOmuuem4/bnyStWFi+sqSHJK2yKNs1WxxdsJ0JSd+okdah\n/3ek+rOdP6e7LcHtV7ehkyU9Juk2SR9scBsbStpjEcpeMT/Edqqk/6tK21HSH9u7zcXVq7MLtKb6\nekRc2+xKZG9HxNDKjKShwD8kPRURdyzKBiNiArBN3l5P4EzSgxffjoiTlkCdazkwIloqM5J2AO6Q\ntGUs+GC1Wj4EHAOc354CJX2I9N+fNwa+S/qvv2fm5LOA0yLi1fZs02wJ6krtzDPAdyRdFxHPdGK5\nC3y2I2JEB5RR3YaK9CDam4H1Glh//5zvlnaWexjw14jYVdKdkj4UEf/K/8TzHNJjCzqVe2QMAEn7\nS7q/0ENyaJ18R0qaJOnRPB1aSPtyTntC0j2Stmm0/IiYDIwDPpW3NVjS7bmXY5KkMyUtm9P2zb0O\nk3L6N/Py4ZIqDyq7B+gP/FPSHpLG5F6M3SS9nAOdSr2/Kunv+fXOkibkbzdTJH2hPccxIv4OvAxs\nkbe3tqRxua7TJI2VtJykjUgPttwoH/OBktaTdGMu+2lJP9aCD2Kr2AS4J//XzAmk53AhaXdg5Yi4\noj11NussTWhnniL9B+PzWqlT3c987mF4QtKTkn4r6SJJ5+a09ny2p0k6UNJpkq6rKv9aSacuwr4t\nILcH1wDrShqYt7dT3rdHJT0l6Vt5+SGk/xx/iKRb2joOVTYB7s6v7yW3P6RHFlwWEU80WuclptnP\na/DUORMwjdR7UCttADAH2C7P7056OOHKxXWBfqR/Pz0oL1+P9IHtDRzAgs87OpTUiPSoUd5wYFaN\n5VcBp5Ge13Ef8N28fHmgBTg+z78MDM+vVwJuAAYWtwsMAgIYmOfHAOcCPUn/7n9Eody/AUcBawNv\nAXvk5UNJz2waXOe4BTCsMC/Sg9XeZv7znK4CflnYj8eBY/P8KODhwroPAj/Kr1cA/gV8sUa5BwE/\nzq/3IH3r60d6+OVWwBXAn4CDm33eeVq6pi7YzownPVJlCvCZQlrkv3U/87mteI70eBiAjwOzgXPz\nfEOf7ap92yy3D/3y8v55fsgi7NusqmXLkHpkH8rzIj3e5uA8v2k+rpVn+I0p7EvDbV9ubyr5fpzb\now+RgpttSV9Ibwe27Kzzzj0yS5eztfC96xERMRMYEBGVKPtO0m3HQVXrzwFeA74iaWhEPB0Re0fE\nPOAzwNiIeBogIi4lNUjbNVIxSduTLsrXkBqurYCf5229RXrC6qdz9heA/1J6XszMiNg3IqY3Uk5E\nVJ4D8tlc7jrA1qRGaS9gUkTckvNOBm6r5K3j2sqxJDUERwG7xvznN40EvlbYjwnARjW2MwT4IHBq\nJG+Qnl1ycI28LcCOknqRHoQ5gRQA/ZLUWN4O7Af8TFK/Ng+K2ZLVpdqZiJhDer7a2ZIGVCW39pnf\nGFgLuCSn3UX6clHR6Ge7WJdHSE/UrrRl+5ACjymLsG99C8f3cVL7swIpQKxEausBV+f5R0mBUq06\ntqftmwCMyO3PzsADwIWkY/wz4Kuk9uis1o7FkuQxMkuXmveu8+2LEyR9BugDvJ+TFgh0I2KepB2B\nk4C/Snob+EFE/JrUM7KLpL0Kq7wHfKBOXfrmi3+lnBeAz0a617od8E5EzCrkfxVYLb/eE/gf0r3g\n3pLOiYhTGzkA2W+BWyQdTfqgjouI15UefDa0UC9I37SmtbKt/4yRkXQFMCci/lFI3xk4WdK6pOOx\nBjC2xnZWIvXw3CupsmwZUu/RAiLiCUk3kXpg7gOeBb4IHEu63z02It6Q9DTpW9iEVupvtqR1pXam\nss27JN1G6kE4qpDU2md+JVI79FYhrTjOptHPdrXfkoKWK0i9GZcW6tKeffvPGBml8SmTgL9ExPOF\nPF8EvpQDuPdJDwmt1YHRnrbvcuBiUvszhtSTdEdE3C9pnYh4StLzwEfq1HuJcyBjkLowjwZ2yCdh\nX1J0v5CImAQcqnS13RO4XtJ4UhfsfdH4oNoFBqpVeQFYTlL/iHgzL1s1Lyf3dhwNHK00uPZmSf9k\nfsPYqohokTSD1BAdBPwwJz1H+na0fYP7UO1EYJKkX0XE3/NxvBn4BnBRRLwv6eo661Z6cLaKiHca\n2IfTgdNzA3YPqfs4VBj7Q+oaf3cR98VsSWtGO1N0AvCopEsLy+p+5iVtDiwradncqwOph2Z6Oz/b\n1a4ARktai3S7alShLou0bxHxjtJYwfMk3ZC/mA0n/Qhg+4i4P+/Ty3U20XDbl4/F5/L2hpJ6fD6c\nkytBUqe2Pb61ZJDuXb8IPJ0vhCcCc0ndmv8haUtJf8gBRpAuoHNIPQnXACMlrZHzbiDpGkl9FqE+\nz5B6Gr6et7UC8CXgOkkfkPRPSavnvP8CXq+xjbn578p1yrgMOJLUrX1rXnYb6VvJx3K5/ST9Rg3+\nnDHSryJ+SmpMepLu6fcFWnJDtwOpm7hyXOcC/SX1iohnST0nx+Wye0g6WdLINoo9BfhdbvghjbPZ\nTukn2esDUxupu1knaGo7ExEzgONJt0EqWvvMPwZMZ/5FezhpLAi047Ndox7P5336Kakno/ILw8Vq\nQ3Mv2GTmfzEbALxB6qlB0rGkXt5iHVdq4DjUlHvYfkUaF1T58vWSpPWBj5Laos7RWYNxPDV3ovVB\neKuQBsVNI41C3510oX+BNEJ9GmnshYBTSYPaHs3TVwrbOQZ4hPTBeYB0q6hWecOpMdi3Ks+GpA/X\npDydBvTKaV8jDd6blP9+r3q7ua5/JA1YO5zCwLacvgGpB+ecqnJ3JgUUk/P0fWoMtst5Fxjsm5f1\nJd3q+Vqe/y7wb+Bh0v3jfYCZwLdIA+yeIjWWm5PuZ/8+79NjpG9uK7VyjD5ECuR6F5atRWokn6rU\nwZOnzpq6YDszvsby28hDSPJ83c88qTfo8VzW+aTbQr/Iae35bC9wXEi3fALYv6pui9WGksYWzs1t\nwzKkH0I8m9uJQ0nt6KuknqA9cn2ntHUc6tThWODCqmW75HWnAjt21nmnXLiZmZlVkaTIF8o8Nu1f\nEXFKk6tlBb61ZGZmVoOku0i3wMiDencG/tLUStlC3CNjZmZWg6QtgAtIt8UgDew9o4lVshocyJiZ\nmVlp+daSmZmZlZYDGTMzMystBzJmZmZWWg5kzMzMrLQcyJiZmVlp/T+U78qEf0AWFgAAAABJRU5E\nrkJggg==\n",
            "text/plain": [
              "\u003cFigure size 576x216 with 2 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "scores_unconstrained_test = model_unconstrained.predict(text_test)\n",
        "plot_metrics(labels_test, scores_unconstrained_test, groups_test, group_names,\n",
        "             \"Unconstrained Model\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fhTCSc9pWs9v"
      },
      "source": [
        "While the overall false positive rate is less than 2%, the false positive rate on the sexuality-related comments is significantly higher. This is because the sexuality group is very small in size, and has a disproportionately higher fraction of comments annotated as toxic. Hence, training a model without constraints results in the model believing that sexuality-related terms are a strong indicator of toxicity."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KmxyAo9hWs9w"
      },
      "source": [
        "## Train with constraints on false positive rates"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l3dYUchIWs9w"
      },
      "source": [
        "To avoid large differences in false positive rates across different groups, we\n",
        "next train a model by constraining the false positive rates for each group to be within a desired limit. Specifically, we optimize the error rate of the model subject to the *per-group false positive rates being lesser or equal to 2%*.\n",
        "\n",
        "Performing minibatch training with per-group constraints can however be challenging for this dataset, as the groups we wish to constraint are all small in size, and its likely that the individual minibatches contains very few examples from each group. Hence the gradients we compute during training will be noisy, and result in the model converging very slowly.\n",
        "\n",
        "To mitigate this problem, we recommend using two streams of minibatches, with the first stream formed as before from the entire training set, and the second stream formed solely from the sensitive group examples. We will compute the objective using minibatches from the first stream and the per-group constraints using minibatches from the second stream. Because the batches from the second stream are likely to contain larger number of examples from each group, we expect our updates to be less noisy. This strategy is similar to the one described in the \u003ca href=\"https://github.com/google-research/tensorflow_constrained_optimization/tree/master/examples/jupyter/Minibatch_training.ipynb\"\u003eMinibatch_training.ipynb\u003c/a\u003e notebook.\n",
        "\n",
        "We create separate features, labels and groups tensors to hold the minibatches from the two streams."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vMuuTOEOWs9x"
      },
      "outputs": [],
      "source": [
        "# Set random seed.\n",
        "np.random.seed(121212)\n",
        "# You may need to use \"tf.set_random_seed(212121)\", instead of the below, in\n",
        "# earlier versions of TensorFlow.\n",
        "tf.random.set_seed(212121)\n",
        "\n",
        "# Features tensors.\n",
        "batch_shape = (hparams[\"batch_size\"], hparams['max_sequence_length'])\n",
        "features_tensor = tf.Variable(np.zeros(batch_shape, dtype='int32'), name='x')\n",
        "features_tensor_sen = tf.Variable(np.zeros(batch_shape, dtype='int32'), name='x_sen')\n",
        "\n",
        "# Labels tensors.\n",
        "batch_shape = (hparams[\"batch_size\"], 1)\n",
        "labels_tensor = tf.Variable(np.zeros(batch_shape, dtype='float32'), name='labels')\n",
        "labels_tensor_sen = tf.Variable(np.zeros(batch_shape, dtype='float32'), name='labels_sen')\n",
        "\n",
        "# Groups tensors.\n",
        "batch_shape = (hparams[\"batch_size\"], num_groups)\n",
        "groups_tensor_sen = tf.Variable(np.zeros(batch_shape, dtype='float32'), name='groups_sen')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-wh26V7nWs9z"
      },
      "source": [
        "We instantiate a new model, and compute predictions for minibatches from the two streams."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kawyrkQIWs9z"
      },
      "outputs": [],
      "source": [
        "# Create model, and separate prediction functions for the two streams.\n",
        "# For the predictions, we use a nullary function returning a Tensor to support eager mode.\n",
        "model_constrained = create_model()\n",
        "\n",
        "def predictions():\n",
        "  return model_constrained(features_tensor)\n",
        "\n",
        "def predictions_sen():\n",
        "  return model_constrained(features_tensor_sen)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UG9t7dw1Ws91"
      },
      "source": [
        "We then set up a constrained optimization problem with the error rate as the objective and with constraints on the per-group false positive rate."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "EhKAMGSJWs93"
      },
      "outputs": [],
      "source": [
        "epsilon = 0.02  # Desired false-positive rate threshold.\n",
        "\n",
        "# Set up separate contexts for the two minibatch streams.\n",
        "context = tfco.rate_context(predictions, lambda: labels_tensor)\n",
        "context_sen = tfco.rate_context(predictions_sen, lambda: labels_tensor_sen)\n",
        "\n",
        "# Compute the objective using the first stream.\n",
        "objective = tfco.error_rate(context)\n",
        "\n",
        "# Compute the constraints using the second stream.\n",
        "# For each group, subset the examples belonging to that group from the second\n",
        "# stream, and add a constraint on the group's false positive rate.\n",
        "constraints = []\n",
        "for ii in range(num_groups):\n",
        "  context_sen_subset = context_sen.subset(\n",
        "      lambda kk=ii: groups_tensor_sen[:, kk] \u003e 0)\n",
        "  # We pass the group index ii as a default argument to the the function, so\n",
        "  # that the function retains a local copy of this variable, and is unaffected\n",
        "  # by changes to the variable outside the function.\n",
        "  constraints.append(tfco.false_positive_rate(context_sen_subset) \u003c= epsilon)\n",
        "\n",
        "# Create a rate minimization problem.\n",
        "problem = tfco.RateMinimizationProblem(objective, constraints)\n",
        "\n",
        "# Set up a constrained optimizer.\n",
        "optimizer = tfco.ProxyLagrangianOptimizerV2(\n",
        "    optimizer=tf.keras.optimizers.Adam(learning_rate=hparams[\"learning_rate\"]),\n",
        "    num_constraints=problem.num_constraints)\n",
        "\n",
        "# List of variables to optimize include the model weights,\n",
        "# and the trainable variables from the rate minimization problem and\n",
        "# the constrained optimizer.\n",
        "var_list = (\n",
        "    model_constrained.trainable_weights + list(problem.trainable_variables) +\n",
        "    optimizer.trainable_variables())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CoFWd8wMWs94"
      },
      "source": [
        "We are ready to train the model. We maintain a separate counter for the two minibatch streams. Every time we perform a gradient update, we will have to copy the minibatch contents from the first stream to the tensors `features_tensor` and `labels_tensor`, and the minibatch contents from the second stream to the tensors `features_tensor_sen`, `labels_tensor_sen` and `groups_tensor_sen`.\n",
        "\n",
        "Also note since we are solving a constrained optimization problem, the final iterate from training  may not always be the best solution, and an intermediate iterate may do a better job of optimizing objective while satisfying the constraints. So, we periodically snapshot the model parameters and evaluate its performance on the validation set. At the end of training, we choose the best iterate from the recorded snapshots using a heuristic provided the TF constrained optimization library.\n",
        "\n",
        "**Note**: The following code cell may take several minutes to run."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "id": "mHiCT2Qet1n9",
        "outputId": "3faa9c91-507f-4900-9454-32e8a3c366e1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Iteration 747 / 747: Elapsed time = 1093 secs, Error = 0.047, Violation = -0.001 "
          ]
        }
      ],
      "source": [
        "# Create temporary directory to record model snapshots.\n",
        "temp_directory = tempfile.mktemp()\n",
        "os.mkdir(temp_directory)\n",
        "\n",
        "# Indices of sensitive group members.\n",
        "protected_group_indices = np.nonzero(groups_train.sum(axis=1))[0]\n",
        "\n",
        "num_examples = text_train.shape[0]\n",
        "num_examples_sen = protected_group_indices.shape[0]\n",
        "batch_size = hparams[\"batch_size\"]\n",
        "\n",
        "# Number of steps needed for one epoch over the training sample.\n",
        "num_steps = int(num_examples / batch_size)\n",
        "\n",
        "# Number of steps to skip before check-pointing the current model.\n",
        "skip_steps = int(num_steps / 10)\n",
        "\n",
        "# List of recorded objectives and constrained violations.\n",
        "objectives_list = []\n",
        "violations_list = []\n",
        "\n",
        "start_time = time.time()\n",
        "\n",
        "# Loop over minibatches.\n",
        "for batch_index in range(num_steps):\n",
        "    # Indices for current minibatch in the first stream.\n",
        "    batch_indices = np.arange(batch_index * batch_size, (batch_index + 1) * batch_size)\n",
        "    batch_indices = [ind % num_examples for ind in batch_indices]\n",
        "\n",
        "    # Indices for current minibatch in the second stream.\n",
        "    batch_indices_sen = np.arange(batch_index * batch_size, (batch_index + 1) * batch_size)\n",
        "    batch_indices_sen = [protected_group_indices[ind % num_examples_sen]\n",
        "                         for ind in batch_indices_sen]\n",
        "\n",
        "    # Assign features, labels, groups from the minibatches to the respective tensors.\n",
        "    features_tensor.assign(text_train[batch_indices, :])\n",
        "    labels_tensor.assign(labels_train[batch_indices])\n",
        "\n",
        "    features_tensor_sen.assign(text_train[batch_indices_sen, :])\n",
        "    labels_tensor_sen.assign(labels_train[batch_indices_sen])\n",
        "    groups_tensor_sen.assign(groups_train[batch_indices_sen, :])\n",
        "\n",
        "    # Gradient update.\n",
        "    optimizer.minimize(problem, var_list=var_list)\n",
        "\n",
        "    # Once in every skip_steps iterations, snapshot model parameters\n",
        "    # and evaluate objective and constraint violations on validation set.\n",
        "    if batch_index % skip_steps == 0:\n",
        "        # Evaluate model on validation set.\n",
        "        scores_vali = model_constrained.predict(text_vali)\n",
        "\n",
        "        error = error_rate(labels_vali, scores_vali)  # Error rate\n",
        "        group_fprs = group_false_positive_rates(\n",
        "            labels_vali, scores_vali, groups_vali)\n",
        "        violations = [z - epsilon for z in group_fprs]  # FPR constraint violations\n",
        "\n",
        "        objectives_list.append(error)\n",
        "        violations_list.append(violations)\n",
        "\n",
        "        # Save model weights to temporary directory.\n",
        "        model_constrained.save_weights(\n",
        "            temp_directory + \"constrained_\" +\n",
        "            str(int(batch_index / skip_steps)) + \".h5\")\n",
        "\n",
        "    # Display most recently recorded objective and max violation in constraints.\n",
        "    elapsed_time = time.time() - start_time\n",
        "    sys.stdout.write(\n",
        "        \"\\rIteration %d / %d: Elapsed time = %d secs, Error = %.3f, Violation = %.3f \" %\n",
        "        (batch_index + 1, num_steps, elapsed_time, objectives_list[-1],\n",
        "         max(violations_list[-1])))\n",
        "\n",
        "# Select the best model from the recorded iterates using TFCO's find best\n",
        "# candidates heuristic.\n",
        "best_index = tfco.find_best_candidate_index(\n",
        "    np.array(objectives_list), np.array(violations_list), rank_objectives=False)\n",
        "\n",
        "# Load model weights for the best iterate from the snapshots saved previously.\n",
        "model_constrained.load_weights(\n",
        "    temp_directory + \"constrained_\" + str(best_index) + \".h5\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DdJfplDpWs97"
      },
      "source": [
        "Having trained the constrained model, we plot various evaluation metrics for the model on the test set."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 254
        },
        "id": "h1jRbseosqxQ",
        "outputId": "9fe38e95-32ff-45c0-ee3d-46641b12aa97"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAADtCAYAAAClO5+KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd7wcVf3/8debEEgCIfSqEGpCkRoQ\nRCSKBRHpLaIYQIogCoKF71cB/SlYKYoi+EVC70Wk2ghShHBDhxRaaNKRQBKSUD6/P85ZMtnsvXdv\n3Tt738/HYx53Z86ZM2fmzp757Jkzu4oIzMzMzMpooUZXwMzMzKyzHMiYmZlZaTmQMTMzs9JyIGNm\nZmal5UDGzMzMSsuBjJmZmZWWAxkrLUkDJX1L0kRJMyS9IekeSYdJKuW5LWmspJC0ZA9v50hJrX73\nQqEeN7SSvpCkZ3Oe4V2sy/2SxnUg/3hJ13RyWwfmOp/XmfX7O0k7SLpP0hxJz0g6uip9Wj6+1dOR\n7ZS7sKRTc96xreRZTtIttc45SStJ+pekNyVdL2nxGuu+JmnjTu249WmlbOzNJA0EbgT+FzgL2Bj4\nBHAl8EvgCknqxfosJen9bijqUmAlYHo3lNVVs4DPSFq+Rtq2wNK9XJ/usB/wMLCrpCGNrkyZSPoo\ncC1wJ+n99j3gREkHVmX9NekcLk5ntVHucsDfgM+3kWdToAVYuZUs/w94DdgCWBKoDpxOBC6PiPtb\n24aV18KNroBZJx0NfBzYLCIeKSx/UNJjwBXAZ4C/9lJ9Pgp0OXCKiLeBt7tenW7xCimYGQOcVpX2\nZeA24HO9XanOkrQasA3wSVIQvAtwUUMrVS7fAJ4AvhHpm1QnSdoI+D5wdiHfjIh4sQPl7gu8QTqn\nnmslzw+AP5KC0KtrpH8G2D8iJks6B/gK8BP4IAjaDRjRgTpZibhHxsrqcOCCqiAGgIi4Elg9Ij4I\nYiR9W9LjkuZK+o+kUyQtmtOG5+7qz0q6JHdP/0fSjwrrD5b0O0nPS5ot6UlJx+a0saQLI7mcEySN\nzq93k/S0pItz+uqSrsq3wWbl2yo7FbYz362l3FV/rKTjJb2Y17tE0hKFdYbnMl/Mt9hukrR2IX0h\nSSdL+q+k6ZL+CAyq8zhfT7oofCAft91zGlVpX5b0SL718IqkcyQtVUhfM98CmC3pCUljapSxkaSb\n862A6ZIulbRCaxXMx+jUOvblK8ATEXEr8GfShbO6rDUlXZeP4wuSzijepsjnyD2S3s7n01GFtAVu\neeXbJdMK8yHpG5ImSHo+Lxsk6bS8vbn53PqfqnKWl3Rh/h++KukiSStIGpnL/GJV/s3z8lF1HJd6\nrQfcGfN/Hfx1wFqSPtyFcq8hBRoz28hzdET8pI30lYBn8uungRULaacBP46IV7tQR+vLIsKTp1JN\nwGpAAHvUmf+bwGzSbYU1gV2BV4E/5PThubz7gK/mPD/Oyz6W85wIPEv6RL8q6UI+nXQxHAz8MOdf\nEVgcGJ3n7yB1dy+fy5lA6pr/CLA6qUt8DrBGTh+b11syz08DJgM/A9Yh9SK8C/xPTh8EPJ7L3RLY\niNQLNQ0YnPMcAcwFDgDWJvVm/Se9/Vs9ZmNzGZvl+qxbSNud9Al6RE4bnpfvBLwPHAWsBXwaeAq4\nqbBuC+lT9ea5rteTbgmMy+kr5Pkbc/qWwL3A3YBynvHANYUylwOG1nEeTAGOy693AN6p/F/ysiGk\nHoer8v/nY/nYnpvTN8zH8bh8HPcknVf716pXXnYqMK0wH8BU4EvAh/KyX5DOx9Gkc2uPfE7sm9MF\n/Jt0Lm2Wj8vdwK05/U7g4qrt/gx4pJXjsC8wo41pm1bWewD4Y9WydfM+bVs4X0/o5Pt6yVzW2Dby\n7FI85wrL32Hee2g7YGp+PQZ4FFi40e2Wp56bGl4BT546OuWLWwBb1Jn/CXLQUlh2DOkWzmDmBTIn\nF9IXJV2Uv5nnrwWuryrjI8BK+fWRFAID5gUy36paZziwQmF+UM731Tw/lgUDmXuryrgTuCq/3jfn\nX6uQvgLpgvvlPH8XaXxAsYxrqSOQya+nAicW0q4GxhWO2/C8/B8Ugpa8bI9KHmBkfv2FQvpywHvM\nC2T+l3QxXaKQZ5O83sfz/HiqAoY6zoGP5jIqF7uFgZcq/9+8bK9cl2JwsyNpfMdCwO+Bh6rK/Q7w\n7dbqRe1A5uqqPMuRehCLy+4Czsmvt8jrbVZI3wI4FxgGHETqzVi8kD4V+H4rx2IoKdBsbRrcynpX\n1DgX98x127Fwvl4H/D0f3ymkQHqhOv5HXQlknmXeh4798v9iSF7+2XwO3U26dTUOWKQj54+nvj35\n1pKVUaVru93zV9IwYA3ShaGohRRErFO1LG0gYg7wFlC5LXIjsIOkCyTtJGloRDwUES+0U4V7q+YH\nA6fm201vkT6JU9hOLS1V868U8m8GPBsRjxfq/hIwiTQgE9Kn5gfaKbMtlwL7KlmK1JtxaY18m1L7\nOEPqRVg3v/6gLhHxCulWQMVmpIvlm4U895F6v7ryxMl+pF6NZyRVxgZexvy3lzYDnouIlwvbvi4i\nDo6I9yt1KxYaEb+MiJM7WJfqc+Id4HBJU/KttBmkHqvi/ziADwaqRsSEiPhqRExn3v9iVwBJHyH1\nKl5Qa+MR8VZEPN7G1NoYrbOATSQdJWlRSRsA3y3sA6RzczDwc9L4qXOAk0njaHrSrcB+khYhBff/\nzNu8N9It5j8C55MCtbWBg3u4PtaLPNjXyujZ/HdNFrxwVhua/75ZtfytQnrlCaFZVXmCPIA3Is6Q\nNB04lHTr4V1J5wJHttHwF7dDHtcyHnie1JBOI/UAPNbOPrRaL2AJYJV88SsaBDyUXy9eo4zq/G25\nmDTYclvSRWAG6RP3KlX5htL2ca7Uua26LAF8rMb+DGH+cQ91yxe3vYFlmHfBLaavExFTSYHDW9Xp\nBe2l16u6jEtIgcs3Sbc35wIXVm13VkS8V6uwiHhT0pWkC/j5pPEmt0REawNnOyUi/prHBJ0E/Io0\nJuVbpPFGr+Y8m1etdr+kVUn7dmJ31qfKCaTg5SDSrctrSe+1zSQNJQWDn4mI2ZKuIPWYnt6D9bFe\n5EDGSici/iPpSeCLzN/gf0BpAO5fmHdhHVaVpTJf92POEXERcFHulfgS6THv/1L/p83RwPKkBvXB\nXM9ajzZ3xHRSd/l2NdIqwcDbpECgqPp4tCoiHpX0EOk2wjqk21rvaMGn26fXKLd4nBfNr4cAr7dS\nl+mkWwBfrVGVN+qtc5Ud8jY+wYJPhF1C6pU5jvl7umppL70YYFYsXitjRe4x/BzpFuSFheVDgUpv\n3yvAYpIGRsQCgVh2DvDXfG7uRnoEurVt7guc2Ua1Ph8Rt9VKiIhTJZ1JevT+BWAUaczW5DbKewT4\nuqRBETG7jXydFhGPKz2VtlxEvJyDlT9ExBOFgchvFf726Pc0We/yrSUrq98Ce0r6eHWCpO2BPwGf\nzLcoHgO2rsq2FWlcwdT2NpSf+tm10iBGxH8j4nekQbXrd6DOi+S/xacnvlTZTAfKKbqH1FPxZvH2\nADAAqNwimUL6xF80uoPbuZj0PR+foPZtJYCJ1D7OlYHUU/KyD+oiaQ2g+MTLPaSetmer9mcgaRBw\nZ+wHjI+I2yKipTiRxn3sm/PdD6ycL4iV+m2v9CVsi+T0rYoFKz1RVvlk/yYLBjpbtFO3gfnvB+eE\npE1It+Eq50TlltLHCnk2lnS7pMr3qownBbTfJ91KvbKNbV5Luk3X2lTztqOkDSXtFRFvR8Tz+Xbb\nXsA/I2KmpHUknZ97YIo2AV7oqSCmIpKXJY0mHavKU06VDyvL5L8rkD6AWLNo9CAdT546M5Eu1NeS\nLh7fJvUUrEe6Zz8TOL2Q9+ukp0AOIjXye5Easl/k9OGki+0uVdt4g/wEBml8xa2kQaMfJn3Kfw04\nJqePzWXsnLcxOs9vXChvDdKtpF+Tnlg6KO/D86TxDMtQe7DvqVX1uoZ0YYY0HuFx0kDbLfI2vkO6\nhVIZ/Ph90u2Kr5JuDR1LuuhFG8d3LPMPUl091+slYEDVcRue5z9PGiB9LCkY2Z50G/CynC7S2J0H\nSV39m5CCwZdZ8Kmli0njakaQer5mFbYznjqfWiL1HMwBDm0lffO8Dx8jBRVP52O5LmlQ+aPAhTnv\n+vm4/jrv+66kXq+DcvrxOX2nfJx/SQreiscxSLcji3V4Crglr/NpUjB3fV531ZznznzcNiY9PfUv\n4I6qcirbP7eH3nO7k87fQ/P+H5j/L5XzbAjpfL0jH8+1SLee3iG/T3K+fwDHVv2PViS9hyOvsyKw\nYiHPinkam/NsnueXrtEuPADsV7X8PtK4nfXzOXhYo9swT914bja6Ap48dXbKjdbhpE+QM0i3K24H\n9q6R9yjgSdIF/RnS49XVF+S2AplVgMtJn5xnk4KHEwplLE0axDmX9KTKaKoCmZzvcFLg8hbpU/PS\npPEnb5Pu2Vca6roCmTy/GmncznTSRbsF2KmQPhD4Q06fDvwfcBgdCGTysruAMwrzleM2vLBsX9LF\nfy4p6DkdGFJIXz+XMyf/P75EuiiPK+TZiPRNrzPzcfkX+YmlnD6e+QOZBY5RIe3rpIvvCm3s69PA\n7/PrEaTgaibwImmAa/EJqh1JF8rZpKfhihfoYfn/8CbpdtBPSEHdtEKeWoHMNqTbL7PysdmU9AVv\nb5CfkiJ9T8oVuezXyN8AXVXOBrn8T/Xge+7b+XjPJvUU7VCVvmau50v5HJhEfiKvtf9X/n9Granq\nuNWaxleVfVg+hqpavgkpEHyT9OV9fmqpiabK9zKYmVmJKX1B41eA9cMNu/UjHuxrZlZiklYkjV36\nIem7gxzEWL/iQMbMrNxuIz3eflxEXNXoypj1Nt9aMjMzs9Ly49dmZmZWWg5kzMzMrLQcyJiZmVlp\nOZAxMzOz0nIgY2ZmZqXlQMbMzMxKy4GMmZmZlZYDGTMzMystBzJmZmZWWg5kzMzMrLQcyJiZmVlp\nOZAxMzOz0nIgY2ZmZqXlQMbMzMxKy4GMmZmZlZYDGTMzMystBzJmZmZWWg5kzMzMrLQcyJiZmVlp\nLdzoCtj8ll122Rg+fHijq2HWrokTJ74aEcs1uh7WeW5vrCzaam8cyPQxw4cPp6WlpdHVMGuXpKcb\nXQfrGrc3VhZttTe+tWRmZmal5UDGzMzMSsuBjJmZmZWWAxkzMzMrLQcyZmZmVlp+aqmPmfyfN9ny\n+Ju7rby7fvS5bivLzJpLd7c31jf0t3bfPTJmZmZWWg5kzMzMrLQcyJiZmVlpOZAxMzOz0nIgY2Zm\nZqXlQMbMzMxKy4GMmZmZlZYDmR4kaZyk06tfm5l1N7c31l85kDEzM7PSavpARtJnJU2UNFnSw5K+\nJukiSSdX5WuRdJCkhST9UNIUSU9J+oekNXOe0ZKekHRyLm+ApA0k3SrpUUnTcpoas7dm1khub8x6\nX1MHMpJWAq4Cjo6IkcDOwCnAFGCPSgMgaS1gfeAy4ChgDLBVRKwO3AqcUyh2FeCBiBgZEe8BZwM3\nRsR6wEeBA4EdO1jPg3PD1vLOrOmd32Ezaxi3N2aN0dSBDPA5YFJEjAeIiCeAG4GBwGBgy5xvH+Da\niJgO7An8PiJez2mnAB+XtHKeXxS4tLCNrYFf5fJfAh4B1u5IJSPirIgYFRGjBg4Z1rE9NLO+wu2N\nWQM0eyCzIvBK1bLXgGVIjcOeednewHn59VLAsbkrdzJwD/BqLgtgZkTMLpS3CzBe0mM5/8Y0/3E1\nswW5vTFrgGb/9esXgOWrli0HPAj8DbhE0tk5T+UnYJ8DLouIM6sLkzS6an4NUvfw5yPi5rxsQnfu\ngJmVhtsbswZo9kj+ZmCEpG0AJI0Atgf+HBH/Bt4BfgBcHBHv5nUuB74maYm8zuaSzm1lQN1QIICW\nnHd3YDiweM/tkpn1UW5vzBqgqXtkIuJFSbsBp0hajNSQHBQRD+QsFwE/BDYrrPZHUrfu3ZICmAF8\nPyKium2JiAck/R/woKTXgAuA44GTJD3Wk/tmZn2L2xuzxlBENLoOVrD4yuvEBgf9ttvKu+tHn+u2\nssyKJE2MiFGNrod1Xne3N9Y3NGO731Z70+y3lszMzKyJOZAxMzOz0nIgY2ZmZqXlQMbMzMxKy4GM\nmZmZlVZTP35dRiNXXqIpR5ybWd/j9saagXtkzMzMrLQcyJiZmVlpOZAxMzOz0nIgY2ZmZqXlQMbM\nzMxKy08t9TX/mQgn1Prh2z7uBP9ml1nplLW9scbpg229e2TMzMystBzImJmZWWk5kDEzM7PSciBj\nZmZmpeVAxszMzErLgYyZmZmVlgMZMzMzKy0HMh0kaaykh/PrXSVdW0g7onE1M7Nm5DbHrG0OZLog\nIq6OiJ0AJC0PnNjgKplZE3ObY7ag0gQykgZI+p2kxyRNknS3pFGSBks6VdJUSU9LukrScnmdCyX9\noVDGOEnj8utpkvYopJ1eSFtC0kV5O09J+oukZWvUaaykhyUNAVqAxSRNlvQtSfdX5f2ZpEt64tiY\nWfdzm2NWDqUJZIDPAZ8B1ouIdYGfAnsAvwA2BDYGVgdeB36d1/kG8AVJW0n6JDAa+GYd2/pfYElg\nfWCd/PrY1jJHxCxgP2BWRIwEzgVGSNq0kG1v4Lxa60s6WFKLpJZXZtVROzPrDU3Z5ri9sWZTpt9a\neglYCfiqpOsi4lrgWkkvAl/Lb2wknQJMlDQ2Iv4r6SDgTGAgsH9EvFnHtr4PLBIR7wPvS7oN2KDe\nikbEG5KuAQ4A7pW0JTAYuLmV/GcBZwGMWll974cszPqnpmxz3N5YsylNj0xETAT2AnYHnpQ0QdLW\nwFLA6bl7dTJwFTADWCavdxNpP+dExC11bm5D4IrcdTwZOJiOH6txwJckLUr6ZHRhRLzXwTLMrEHc\n5piVQ2kCGYCIuDEiPk9qMG4CLgaeAw6JiJF5GhERy0bEKwCSDiF9spqZPylVvAcMKMwvVXj9Z+B+\nUpfySOBPnaju34CZwBeAPWnltpKZ9V1uc8z6vtIEMpIOkHSapIUi4m3grpx0OfANSYvkfDtJ+nl+\nvQbwY+Ag4GvASZJWz+s9C2ya8w0Htitsbhhwb0S8K2ldUsOweDtVnAssImkxgNxFfD7p3vlrEfFA\np3fezHqd2xyzcihNIANcDawIPCbpEeBHwFdIjcY04EFJk4CjgYskLUTqav15RDwZEZOA3wDjctrx\nwO65G/fnwIWFbX0H+K2kR4HvAYcCG0n6bRv1ewB4FHhe0vZ52ThgOP5kZFZGbnPMSkARHuvVUyQt\nTuqGHhERL9WzzqiVFS0H92y9esQJPo/6G0kTI2JUo+th83S0zSlte2ON06C2vq32pkw9MmX0I+DK\neoMYM7Mucptj/Y4DmR4gaX1JTwHrAcc0uj5m1tzc5lh/VqbvkSmNiHiE9EVZZmY9zm2O9WfukTEz\nM7PSciBjZmZmpeVbS33NypvBCS2NroWZ9Qdub6wJuEfGzMzMSsuBjJmZmZWWAxkzMzMrLQcyZmZm\nVloOZMzMzKy0/NRSH/P4G4+x0zVf6PT61+5yfTfWxsyaWVfbG7OeVO/1zD0yZmZmVloOZMzMzKy0\nHMiYmZlZaTmQMTMzs9JyIGNmZmal5UDGzMzMSsuBjJmZmZVWnwhkJI2WNKObyposaf1W0sZLOqYb\ntnGgpCH59UmSfpBfLyPpS10t38x6ltscs+bRdF+IFxEje7J8SQOAU4A/A7Mi4thC8qeALwEX9WQd\nzKzvcJtj1ljt9shIOkLSi5ImSfqupEcl7ZHTNpR0i6Spkp4sfvLIn0S+L+kGSU9IekDSujltmKSr\nJD0raQLwsapt7pbzPy7pIUmfK6SFpO/kT0HzrVdIH5Vf75G3PUXSH4r7mz/JnJfr/oyksyUNzmkn\n5LQ/5P1+VtLuedW7gaHAvyXtIGmcpNMljQbOAEbnul8i6dSqut0l6dD2jrlZf+Y2x22OWUe0GchI\nGkn6JPCZiFgXGAyskdOGAH8FLouIdYCtgaMkfapQxJeBL0fEmsBU4Lt5+THAMrmsbYBPFra5OXA+\nsH9ErAUcAVwuaVih3DUiYmRE3NlG3YcB5wCHRcQI4Crmb7zG5f1fD1gLWBkoftLZFfhj3u+fA7/K\ny/fIf7eKiBsqmSNiPHA6MD4iNsrl7ytpkVyf4cBGwKU16nqwpBZJLXPfnNvaLpk1Pbc5Pd/muL2x\nZtNej8wngfsi4qE8/wvm3Y76BLBwRJwBEBEvkLo39yms/+eIeD2/ngisml9vB1waEe9ExBzSG7Bi\nd+DGiLg3lzsemAIUfxDkmjr27aPA2xFxcy7nr8BTAJIWA3YAfhYR70bEXFKDUKz7gxExMb9uKdS9\nXn8D5gA75fl9gOsj4r/VGSPirIgYFRGjFllikQ5uxqypuM1JeqzNcXtjzaa9MTJLAa9WZiLibUmv\nFtKGSppcyD8IuKswX3wDvQsMyK+XAV4vpL1Stc3tqsodmtepeK2detfaRnE7S5CCuKslvZeXDcj1\nb63uHRoYHRHvSTofOAC4AtgbOK4jZZj1Q25z5tXdbY5ZHdoLZN4kvQEBkLQosGyefQ54rZMD3f4L\nFLttVyy8fg64KSLGdKLctrZR3M5LwDvAjhExpYvbacs44GFJWwOrADf24LbMmoHbnK4Zh9sc62fa\ni/hvBzaXtE6e/w7pzQgwAXhH0hgASQMlnSJpuzq2eyuwl6SFc0O1XyHtKmB7SSNyuctJulTSSnXu\nU8UEYElJn8zlbE/uqo2I9/N2jpKknH6wpKPqKLdyU3npVtKWrJSZG6x7gAuAiyPi3Q7ug1l/4zZn\nQW5zzNrQZiATEfcDPwH+Iek+UrfpSykp5gBfBA6RNAV4BFiU1BC15+fALNL94zuB4gC2R4CDgEsl\nTQLGA7fl++F1y/fJvwacneu3B+kecsXhwOLA5Jy+M6mhac8LwN+BiZK+VpV2E7A28IKkpfKyccBw\n4LyO1N+sP3KbU5PbHLM2KCLaziApCpkkTQd2iYhberpyzSB/Wjw5P1XQriXXGhaf+NXHO729a3e5\nvtPrmnWEpIkRMaoHynWb0wUdaXO62t6Y9aTi9ayt9qa9x6+HAf+VtFWe3xkQcH/3VbV55e+I+BHz\nHqM0sza4zekatznWH7U52Dcipks6CDgn34OdDexd6xFim5+kscCPgSsj4vwGV8esFNzmdJ7bHOuv\n2v2Jgoi4HLi8F+rSVCJiHPN/V4WZ1cFtTue4zbH+qk/8aKSZmZlZZziQMTMzs9Jqul+/Lru1llzb\nTx6ZWa9we2PNwD0yZmZmVloOZMzMzKy0HMiYmZlZaTmQMTMzs9JyIGNmZmal5aeW+ph44S1m/3h8\nl8oYdNzobqmLmTW37mhvzDqjO69T7pExMzOz0nIgY2ZmZqXlQMbMzMxKy4GMmZmZlZYDGTMzMyst\nBzJmZmZWWg5kzMzMrLT6dSAj6QRJ1+XX35B0Rh3r7Crp2p6vnZk1E7c3Zj3DX4iXRcTpdea7Gri6\nh6tjZk3M7Y1Z92m6HhlJwyW9J+loSVMkrSbpPElTJT0j6WxJg2usV/y0tISkayS9KOluSYdKmpHT\nxkp6uLDe0ZIelTRZ0l2SPpaXj5b0pKTDJd2byzqpt46DmfU8tzdmjdd0gUy2EDAwIkYAp+f59YC1\ngJWBY9tZ/1hgGWBV4LPA/rUySdoZOAr4dESMBE4FrpY0JGdZBVgoIjYFPg18X9JqNco5WFKLpJZX\nZk7v2J6aWaO5vTFroGYNZCC9wRcDdgB+FhHvRsRcUkOzTzvrfgq4JCLmRsR04OxW8u0KXBwR/wGI\niEvy8i3y34WBs3Law8AcUmM1n4g4KyJGRcSo5RYbVv8emllf4fbGrEGaeYzMa8ASpGDtaknv5eUD\ngEHtrLsU8Gph/plW8q0IPFq17HVgBeAl4O2ImFNIezdv38yai9sbswZp5kAG0pv7HWDHiJjSgfXe\nJDVKFau0ku8FYPnKjCQBy+blzdzbZWYLcntj1gBNffJHxPvAVcBR+U1fuT98VDur3g7sI2mApCWA\nA1rJdxUwRtIKef7LwGzgnq7X3szKxO2NWWM0e48MwOHAacDk3LY8DhzWzjo/BS4EnszThcCG1Zki\n4i+S1gL+KWkA8Aqwc0S8nbdlZv2L2xuzXqaIaHQd+iRJinxwJO0GnBIRCzwB0N02W2VE3HHImV0q\nY9Bxo7unMmZtkDQxIkY1uh7NoMztjVlndPQ61VZ709S3ljpL0v5Ai6TB+ZPPGOBfDa6WmTUhtzdm\nXeNAprYLgRbSEwKTSbfgjmlojcysWbm9MeuC/jBGpsPy9z8c0uh6mFnzc3tj1jXukTEzM7PSciBj\nZmZmpeVbS32MVhrqp47MrFe4vbFm4B4ZMzMzKy0HMmZmZlZaDmTMzMystBzImJmZWWk5kDEzM7PS\n8lNLfczrTz7CxXuv26PbGHPppB4t38zKoTfaG7OKnrr2uEfGzMzMSsuBjJmZmZWWAxkzMzMrLQcy\nZmZmVloOZMzMzKy0HMiYmZlZaTmQMTMzs9JyIGNmZmal5UDGzMzMSsuBTBVJwyW9J+loSVMkrSrp\nekmTJE2TdL6kQTnvAEknS3pS0lRJf5K0aE7bUNItefmTko5p7J6ZWV/kNsesaxzI1LYQMDAiRgC/\nBJ6NiHWB9YGtgINyvm8Bo4CRwHrAh4BjJQ0B/gpcFhHrAFsDR0n6VK2NSTpYUouklrfmvNuT+2Vm\nfVOvtTlub6zZ+LeWWnd1/juGfJwiYqakCcDaOW134PyImAsgaWfgXWA7YOGIOCOv94Kki4B9gH9W\nbygizgLOAlhj6cHRY3tkZn1Zr7Q5bm+s2TiQad1r+e+2wA8krQq8B6wEnJ/TlgXeqKwQEW8DSFoK\nGCppcqG8QcBdPV1pMysttzlmneBApg25u/Y64Ejg7Ih4X9JlhSwvAcsV8g8DhgDPAa9FxMjerK+Z\nlZvbHLOO8xiZtg0kNRItuUHZGtgSWDynXwmMlTRY0gDgPOAwYALwjqQxAJIGSjpF0na9vwtmViJu\nc8w6yD0ybYiI6ZKOB66X9DpwM3AEcJ6kR4GTgVWAqcDbwJ3AiRExR9IXgd9IOgEQ8Hfg9gbshpmV\nhNscs45ThMd69SVrLD04ftVzyOgAAA+zSURBVPqZ4T26jTGXTurR8q1/kDQxIkY1uh7Web3R3phV\ndOXa01Z741tLZmZmVloOZMzMzKy0HMiYmZlZaTmQMTMzs9JyIGNmZmal5cev+5il11ifMZe2NLoa\nZtYPuL2xZuAeGTMzMystBzJmZmZWWg5kzMzMrLQcyJiZmVlpOZAxMzOz0nIgY2ZmZqXlH43sYyS9\nBUxpdD3qsCzwaqMrUSfXtWeMiIihja6EdV6J2pvuUKb3Vlc1476uFhHL1Urw98j0PVPK8IvCklrK\nUE9wXXuKJH8BSfmVor3pDmV6b3VVf9pX8K0lMzMzKzEHMmZmZlZaDmT6nrMaXYE6laWe4Lr2lDLV\n1WrrT/9D72uT8mBfMzMzKy33yJiZmVlpOZAxMzOz0nIg08skbS7pLkmPS5okab9W8u2X0x+XdLek\nzRtQ1+3ytidLekzSUTXyjJU0K+epTDf0cj2HS4qqOkyWtGyNvN+VNDUf139KWrOX6/rxGvV8VdLZ\nVfkadlwlHSxppqRjCsuWlfRnSU/kc+HXkmq2H/We49Y4zf4/aq3t6sh5XCaSlpT0vKRxeb4p97NV\nEeGplyZgUeA5YJ88vxbwBvCRqnwb5uVr5/m9gWeBRXqxrisCM4Dt8vyawFvAVlX5xgLjG3xch6dT\nud18OwLPAMvn+e8BExpc90HAZGDzvnBcgd8BlwETgWMKy68AzgAELAa0AIfVWL+uc9xT46Zm/x+1\n1XbVex6XbQLOBZ4ExuX5ptzP1qbmjdD6pu0AIuKS/Pdx4HpgTFW+LwPXR8RjOd+lpBNydK/VFN4D\nvhIR/8h1eAJ4lBRkldV+wPkR8XKe/y2wiaR1GlinHwC3RMQ9DaxD0cURsRep4QdA0lBgF+DkSGYC\nZ5LO02r1nuPWOM3+P2qt7dqC+s/j0pC0IykYvSDPd+T92hQcyPSukcBjVcumAuvXyDe1atljNfL1\nmIh4JSKursznWzAbAHfUyP5hSTdKmiLp75K26K16Fkk6X9LDklokfaVGlvmOa0TMIn0y7bXjWiRp\nBeBQ4EetZOn14xoRt9dYvHb++0RhWa3zFuo/x61xmvp/1EbbdV9eVM95XAqSlgJOA/YH3s+LO/J+\nbQoOZHrXYsDbVcvezss7k69XSPoQ8BfgFxHxcFXy48CfSbdC1gWuA27Ib7DeMgP4E/DriNgAOBI4\nU9InqvL1qeMKfAe4ICJerJHWF45rxWLA3Ih4v7CstePW146xLajf/I+KbRcQ1H8el8VpwO8iovjB\ntyPv16bgQKZ3zQAGVy1bLC/vTL4eJ2lT4N/AuRGxQM9BRNweEd+OiJci4v2IOBV4F9i6t+oYEa9G\nxIERcX+lTsC1wE5VWfvScR1AutV1Xq30vnBcC2YAi1YNFmztuPWZY2yt6hf/oxptV0fO4z5P0heB\nNYBTq5Kaaj/r4UCmdz0CVI/HWBd4sEa+EZUZSSJ1B1fn61G5IbgBODIift5KnlUlrVi9GHinp+tX\nqMPSktaqWrxQjTpUH9ehwCrAQz1bw5q2BeZExL21EvvCcS2YShp3UDzGtc5bqP8ct8Zp+v9RK21X\nR87jMtibFMg8KWkaqSd6D1LvdDPtZ/saPdq4P03AQOBpYP88vxHpaYF1qvKtR+EpAuAg0v3OhXux\nroPyNndvJ98vgX8CQ/L8/sALwBK9WNcvAC+RfuYd0v3wt4CPVuX7LPA88KE8/1Pg1gadCz8Ebu6r\nxxUYz/xPLV0EjCMFU0sCD1fO46r16jrHPTVuavb/UVttV73ncRkn4ATmPbXUtPtZa/JPFPQySRsD\nvweWA2YDJ0TElZJOAmZGxE9yvjGkJ1oWIV3ADosFx6f0ZD3HkEbBVw8KvIT0+ObMiPiJpEGk+7Tb\nkW59vAh8O1rpaejB+n4L+HqenQ38LCIukXQe6amgc3K+o0gDbBci7dvBEfFcb9Y11+NMYNGIGFtY\n9sE50Ijjmm93PZJnVyV1Rb8OXE0aY/B/wMakT3uXAMdHREjaldRI7pTLqXmO91S9reOa+X/UTtt1\nGq2cx71ayR4g6QRgeESMzWPpmnI/a3EgY2ZmZqXlMTJmZmZWWg5kzMzMrLQcyJiZmVlpOZAxMzOz\n0nIgY2ZmZqXlQMbMzMxKy4FMPyFpmqTnJU2umg6qc909urEuoyW9X6jDY5Ieqacu7ZS7haSJhfkD\nJQ3Jr0+S9IOu1r1qe5GPTWU/puQfrNy1zvUXlvT19nPWXPf0fNz+IWnJwvKlJT0kaZnOlGvWFX2w\nnQlJR9ZI69HvHal+b+f36We7sfzqNnSypKmSbpL0kTrLWFPSDp3Y9pL5R2wfk/T7qrRtJP2to2V2\n1cK9vUFrqG9FxBWNrkQ2KyJGVmYkjQTulPRURPy9MwVGxARgs1zeAOAU0g8vzoqIY7uhzrXsEREt\nlRlJWwN/l7RRzP9DbrVsAhwOnNGRDUrahPTtz+sAx5G+9feUnHwqcGJEvNaRMs26UV9qZ54Bfijp\nqoh4phe3O997OyK264FtVLehIv0Q7XXAanWsv1vOd0MHt7s/cFtEfFrSPyVtEhH35S/x/A3pZxJ6\nlXtkDABJu0m6v9BDsl8r+Q6WNEnSo3nar5B2aE57QtLdkjard/sRMRm4Hvh8LmstSTfnXo5Jkk6R\ntGhO2yX3OkzK6d/Jy0dLqvww2t3AUODfknaQNC73YnxW0ss50KnU+xuS7sivt5U0IX+6mSLpKx05\njhFxB/AysGEu70OSrs91nSbpfEmDJK1N+mHLtfMxX1bSapL+nLf9tKRfaP4ffqtYF7g7f0vnBNLv\ncCFpe2DpiLi4I3U26y0NaGeeIn2D8e/aqFOr7/ncw/CEpCclXSDpbEmn57SOvLenSdpD0omSrqra\n/hWSftqJfZtPbg8uB1aVtGwu7xN53x6V9JSk7+bl+5K+OX5fSTe0dxyqrAvclV/fQ25/SD+RcGFE\nPFFvnbtNo38jwVPvTMA0Uu9BrbRhwBxgyzy/PenHCZcurgssTvq66+F5+WqkN+xAYHfm/72j/UiN\nyEI1tjcamFFj+aXAiaTfB7kXOC4vXwxoAY7O8y8Do/PrpYBrgGWL5QLDgQCWzfPjgNOBAaSv+9+u\nsN3bgUOADwEzgR3y8pGk32xaq5XjFsCowrxIP+Q2i3m/53Qp8IfCfjwOHJHnxwIPF9Z9EPhZfr0E\ncB9wQI3t7g38Ir/egfSpb3HSj19uDFwM/APYp9Hnnaf+NfXBdmY86SdVpgB7FtIi/231PZ/biudI\nPw8D8EngbeD0PF/Xe7tq39bP7cPiefnQPD+iE/s2o2rZIqQe2YfyvEg/b7NPnl8vH9fKb/iNK+xL\n3W1fbm8q+X6R26NNSMHN5qQPpDcDG/XWeecemf7lNC1473q7iJgODIuISpT9T9Jtx+FV688B/gt8\nXdLIiHg6InaKiHeAPYHzI+JpgIg4j9QgbVlPxSRtRbooX05quDYGTs5lzST9ousXc/YXgK8q/V7M\n9IjYJSJerWc7EVH53ZG98nY/DGxKapR2BCZFxA0572TgpkreVlxROZakhuAQ4NMx7/ebxgDfLOzH\nBGDtGuWMAD4C/DSSN0m/lbJPjbwtwDaSFib9EOYEUgD0B1JjeTOwK/BrSYu3e1DMulefamciYg7p\n99VOkzSsKrmt9/w6wCrAuTntFtKHi4p639vFujxC+hXuSlu2MynwmNKJfRtSOL6Pk9qfJUgBYiVS\nWw24LM8/SgqUatWxI23fBGC73P5sCzwAnEU6xr8GvkFqj05t61h0J4+R6V9q3rvOty+OkbQnMBh4\nPyfNF+hGxDuStgGOBW6TNAv4fxHxf6SekU9J2rGwynvA8q3UZUi++Fe28wKwV6R7rVsCsyNiRiH/\na8AK+fUXgP8h3QseKOk3EfHTeg5AdgFwg6TDSG/U6yPiDaUfWhtZqBekT1rT2ijrgzEyki4G5kTE\nnYX0bYEfSFqVdDxWAs6vUc5SpB6eeyRVli1C6j2aT0Q8IelaUg/MvcCzwAHAEaT73edHxJuSniZ9\nCpvQRv3NultfamcqZd4i6SZSD8IhhaS23vNLkdqhmYW04jibet/b1S4gBS0Xk3ozzivUpSP79sEY\nGaXxKZOAf0XE84U8BwAH5gDufdKPhNbqwOhI23cRcA6p/RlH6kn6e0TcL+nDEfGUpOeBj7ZS727n\nQMYgdWEeBmydT8IhpOh+ARExCdhP6Wr7BeBqSeNJXbD3Rv2DaucbqFblBWCQpKER8VZetlxeTu7t\nOAw4TGlw7XWS/s28hrFNEdEi6XVSQ7Q38JOc9Bzp09FWde5Dte8BkyT9MSLuyMfxOuBI4OyIeF/S\nZa2sW+nB2TgiZtexDycBJ+UG7G5S93GoMPaH1DX+bif3xay7NaKdKToGeFTSeYVlrb7nJW0ALCpp\n0dyrA6mH5tUOvrerXQycIGkV0u2qsYW6dGrfImK20ljB30m6Jn8wG016CGCriLg/79PLrRRRd9uX\nj8WXcnkjST0+W+TkSpDUq22Pby0ZpHvXLwJP5wvh94C5pG7ND0jaSNJfc4ARpAvoHFJPwuXAGEkr\n5bxrSLpc0uBO1OcZUk/Dt3JZSwAHAldJWl7SvyWtmPPeB7xRo4y5+e/SrWzjQuBgUrf2jXnZTaRP\nJR/P211c0p9U5+OMkZ6K+BWpMRlAuqc/BGjJDd3WpG7iynGdCwyVtHBEPEvqOTkqb3shST+QNKad\nzR4PXJkbfkjjbLZUeiR7deCxeupu1gsa2s5ExOvA0aTbIBVtveenAq8y76I9mjQWBDrw3q5Rj+fz\nPv2K1JNRecKwS21o7gWbzLwPZsOAN0k9NUg6gtTLW6zjUnUch5pyD9sfSeOCKh++XpK0OvAxUlvU\nO3prMI6nxk60PQhvGdKguGmkUejbky70L5BGqE8jjb0Q8FPSoLZH8/T1QjmHA4+Q3jgPkG4V1dre\naGoM9q3KsybpzTUpTycCC+e0b5IG703Kf39UXW6u699IA9a+RmFgW05fg9SD85uq7W5LCigm5+nH\n1Bhsl/PON9g3LxtCutXzzTx/HPAf4GHS/eOdgenAd0kD7J4iNZYbkO5n/yXv01TSJ7el2jhGm5AC\nuYGFZauQGsmnKnXw5Km3pj7Yzoyvsfwm8hCSPN/qe57UG/R43tYZpNtCv81pHXlvz3dcSLd8Atit\nqm5dakNJYwvn5rZhEdKDEM/mdmI/Ujv6GqknaIdc3yntHYdW6nAEcFbVsk/ldR8Dtumt805542Zm\nZlZFkiJfKPPYtPsi4vgGV8sKfGvJzMysBkm3kG6BkQf1bgv8q6GVsgW4R8bMzKwGSRsCZ5Jui0Ea\n2PvzBlbJanAgY2ZmZqXlW0tmZmZWWg5kzMzMrLQcyJiZmVlpOZAxMzOz0nIgY2ZmZqX1/wEWliZ2\nLBywbAAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "\u003cFigure size 576x216 with 2 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "scores_constrained_test = model_constrained.predict(text_test)\n",
        "plot_metrics(labels_test, scores_constrained_test, groups_test, group_names,\n",
        "             \"Constrained Model\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EfKo5O3QWs9-"
      },
      "source": [
        "Below, we present a side-by-side comparison of the plots for the unconstrained and constrained models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 219
        },
        "id": "IFLLircZWs9-",
        "outputId": "b84e2351-dd4b-4352-d823-3275c02d3030"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAADKCAYAAABZu2FAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd7gdVdn+8e+dAkkgCYFQoxB6KC8o\nhFcQXg1iQaSDQCxIDYgFEAQREPCnNKmCICgY6UVpUpUSlU5CDeQkoQSIBqQjNSF5fn+sdchkZ5+W\nnH32PnPuz3XNdWZmrZm1ZrL3kzVrZs9SRGBmZmZWFr3qXQEzMzOzzuTGjZmZmZWKGzdmZmZWKm7c\nmJmZWam4cWNmZmal4saNmZmZlYobNz2IpJA0ssr6iZL2qEOVWiVpJ0krLOC2d0j6cifXZ3g+h0Or\npI3KaUdXSdsmpx3bmWVWyTtN0s4dLcOsq0j6hqQHJU2R9IykeyTtWOMyN5T02QXc9gRJR9WgTi1+\nV/P3/Y4q65eS9KGkcZ1dZkW+sZLOXpAyGkmfelfArBU/B74D/LujG0bEFp1fnTY9D+wh6Rcx7wuk\n9sppZj2WpMOA7wO7RsR9ed0XgSslLR4RF9Wo6L2A54B7O7phRBzR+dVpl5UlrRwRzxXWfQuYUaf6\ndDvuubF5SBon6SeSbs5XVo9JWiun9ZZ0mqRn85XXhZIWzWkjJd0tabKkpyQdKUk5bZqk/fO+X8hX\na8vltM0kjc/bTJX0K0m9JP0JWBv4k6R9JR0r6XJJt0u6NG+7o6RHJTXlbXcvHMfHVyltlN9f0hn5\neJ6XdI2kpXPaIpJ+J2m6pIeB7ds4fdOA/wKbF+qxNLAxcHdhXV9JJ+Z6T5J0Z+Ect1qmpM8Xrnwn\nS/p2x/6FzbqepCWAY4AxzQ0bgIi4HdgauD/nW1bS1fnz3STpj5IG57SxOT5cmT/7T0v6v5y2nKRb\n8vqpkq6TNDQ3qPYEDsvfq1E5rp2W999b0rqS/p5j0LScpkKZZ7dVfk7fP3+fn5H0gKQNC2k/yPHl\nSUnHteOU3ZDrXbQncH3Fef2WpCfysTwiaZv2lClpJUnXF+LeyZLK1R6ICE89ZAICGFll/URgjzw/\nLi8vmZevBv6Q538E/ANYhNTr91fgWKA/8C9g95xvaeAFYKe8PA24I+frTQpkx+S0Bwtl9wMuBtau\nrG8u5y1gjbw8GPgQ2DgvbwnMKtR7GrBzO8o/C7gTGEBq7P8euCin7Q1MBgbmtLG5TkOrnMNR+dwd\nAlxaWP8j4Oy87bF53YHAI8DgvPwT4HFArZUJfAJ4F9gqbzeC1JharfKYPXlqpAn4KvBOO/Jdm7+D\nvXKM+QtwVk4bS+q5WCkvnwLcledPBsYW9nM0sFueHwccmudHAR8A3ynkfQD4SZ5fNseZbQplnt2O\n8ncCXi6k7U7qLeoFrAbMZG5c2xf4qKXvav6+b0iKob3yug2AJ4A9gHF53aeBd4AReXkT4D1gWGtl\n5jjzOHBinh+U49FelcfcnadytdSss1wfEa/n+QnAinl+J+CyiJgZER8B2wG/BD4D9I3crRwRrwBX\nANsU9nlFRLwfEbOBRwv7nAHsLGlj4KOI+HZEPNVCvZ6KiCm5jLdIjYP7c9qdpGA4vIVtWyr/68Bp\nEfFeRMwBTgd2yVcxW+Rz8d+c9ruWT9nHLga2yVeqkK62xlbk2QH4fT4GSA2sdYGV2ihza2BSRNyc\nz0ETcCuwSzvqZVZPSwIvtZZBUh9SzDg9IubkGHMu88aROyOi+RbveOaNI5tI+pqkgRHx/yLiihaK\nWhS4srC8KamhQkS8DDwJrN7Cti2V/3Xg4ua0HAsXJ/XafgF4tBDXLiQ1NFoUEROAN4Dm5wb3Av5Y\nkW074JYcB4jUIzYR+FIbZa4J/A/wy0jeJjUod2utTt2Nn7npWYLqtyL7ALMLy28U5j8i9XZA6j14\n8+OdRbwPqUsYeLVin6+RvkAt7bNfnt+D1HNxMbC0pAuBwyNiVpV6vtY8kxsfh0r6OqlHZk5OaqnB\n3lL5Q4CzJZ3SvGvS1dBSeXq8sN0rLez7YxHxH6WHAb8p6cG8bnzu5W62XHFfEfGupA9JV42tlTkE\nGCGpqbBuMVKPjVkjexUYJqlXbrRXM5QUa4qf+ddI34tmLcWmM/Pyz4D1Jd0KHBAR1Z7XezciPigs\nbw8cJGlZUhxcEbimhTq2VP4Q4AuSti6kzwaWIX2nmy8WiYjZkl6nbRcCe0u6i9R4Wp/UQ91snjiS\nNZ+vXq2UOYT0f8FDhbi0CG00PrsbN256lhnAyqRbQQAoPTOzMu174PVl0i2n5m0Hk27nzCA1TBS5\nXzPna/Pht4h4AzgcOFzSuqRu6ImkL3ZrdgcOADaNiOckDSDdsumo6aQgeFtlgqQ3SLe/mi3Xzn1e\nCBxJum1UebUF6bwsUyhnIKmxNYMUPFsqczrwRERs0s56mDWK+0j/2e8I/KmYIGkr0m3YP5EaDMsA\n/8nJ7Y0jc0g9oGdJWob0HTyRFCdaJGkV4Crgq80xoPmipIOmAw9HlQeQ88Xf4MJyH1JDri2XAL8A\nvgk8FBEvVVwkzWDeC0iYe74GtFLm9Pz3UxWNvFLxbame5TTgSEnDID1MSwoI91J44LUVfyb9Gqi/\npN7ARaQGxoOk51++mfe7HDCalq9+yPkWkXS/pBF5VRPzBrJZpO7sagaTrjSez3U5nHSPefF2HEfR\n1cD3JS2S67StpJNy2t+BbSUtlnuK9mnnPm8BPkm6jXdJlfRrSFdkA/PyQaQu7hfbKPNWUs/NZrmu\niys91F0Z4MwaSr71cSSpl/QLzevz/FhgVr5lfANwoJJFSPGl1TiS93Oe8sP1EfEfoHhreyYtx5GB\npF6M8Xk/O5FubS9IHBktafm8n1WUHozuT3pOcQNJa+S8+9CO/3sj4jXSd/545r+1DXAdsGXzfiV9\njnTL6a+tlRkRL5Ji9sF5u16SjpI0uoPH3NDcc9OznEZqhNzc/J856YuwYytdxUW/IT2sNgV4n9Qo\nOj4i3pe0HXCGpCNJt4hOiYgbW9tZRMyUdAbpF1G9SV28t5NuUQFcClwr6VdVNr+E9OzKs6Su2aNJ\nV36XF4NnO/wcOAl4XFKQrhh/mNP+QLpnPpnUrf5boM1fJ0XER5IuAdaJiGpdvecCywMPKl2KPUd6\n+DEktVhmRLwiaXvgNEmD8r6uIj0jYNbQIuJMSS8BJ0paitTomA7sEhHjcrbvki64JuXlcaQfE7Tl\nHOAcpfdMzSF9p/bNaVcBZyq94+v4ijo9Jun3pO//a6S4cgxwgqSpHTi2W3Ocuj1flMwkPdPyPvBU\njou3S3qH9NB0Uyu7K7qQ9PzMDVXKfEzSGFL8XIR0O33HHHNeaqPMXUkNzcmkW/ETSPG9NDT3LoKZ\nmZlZ9+fbUmZmZlYqbtyYmZlZqbhxY2ZmZqXixo2ZmZmVihs3BoDSWEdvSvp8vetST5KGKI1nM1Vp\nHKoz8y+5UBrf6i2lcVyap/neWpx/4t5UMb0g6Zmc3k/S6Urjurwo6RJJi+W0lZTGpXla0k8r9vtN\nSRd0xXkw62yOMUlnxJjCvjaT9Jyk+X6ZKumTku7Kv5Yqru8RMcaNGyP/jPAbpHfFjKlzdertTNK7\nL9YG1gA+RXoPTbNrI2JEYdq3cgd5eIpinhGk99eck7P8lPRz7w1IQy70Jr1wDNJYVOcAawHfljQU\nPh6A8wjg0M49XLPac4yZx0LHGABJuwG/pspo5/n9NncCD1XZtEfEGDduDNL7Yh4gvU13c0kfv/BK\nrY8E/hmlEaqnKo1I+8W8/tjilYTSiOGR5zs6Ku/iSqPxPpvLOVHJ/ZIOKpTRS9K/JBVfT74gtgLO\niYhZ+e2dZ5KC8gLL52X9vK/mMi6IiHfy+4V+VShjLeD+PPzE48wd4+Ys0sCbxde/m3UXjjFzdVaM\neRr4LFDtfTwzgc2Bm6uk9YgY48aNAexPGsjxA9KL8L5TSDsQGEkaSmBt0sjUR0jqRxoq4WcRsXrO\nd43mvnW3NcOAx/JVyWzgAtIAcGuTBuHcmzRIJKSXbi0KrEpqIGyd6zeWNChls8+RXkb1t2JBkjbQ\n/LeImqclmF8wd7wYgLdJV1fN1s9dvVMkXa+5bwBtzUnAUXkgwJbKWFLpxWazC2kC5kjahjQ8w1uS\nbpN0g6QVMes+HGPm6pQYExHjWxo+ISKmRcT0amn0lBhTbahwTz1nIr2u+3mgV15ejzT6dnP6PcD+\nheX+QF/SyLMvVexrUP57LHBjYf3I9FELgFGkL3e/QnofoE9h+V7gR3n+X8CWhbTFSF/MJUhvSR6Z\n1/+W9FbkhT0fFwK35XIGkV79PiunbQ/8P9LQD32BU0lvEu7Tyv62Bh6pWPcz0sjkQ0lB9Zx8ToYB\nxwE/IL3+fRIp0D+Z0ybmOu0J/KHenx1PntozOcbMdz46O8bMcy4q0kYB71Ss6xExxj03th9pFNnX\nJb1JGpNkDaVxSqDKSOCRujPnWZ/T3m5nmdVG5R2Xu4SbSPegmz+bleW/GxGzI+JN0tgqeykNCrcT\n1Qep7KgfAf8mNT5uAu4ijwQcEddFxNER8VY+B0cDq5GuOFuyF3OHk2h2Eul++L2k8z2eFIzfJN1D\n/yqpC/8E0qvgf00aUPC/+RzfQ7r6NOsOHGPm1dkxpqN6RIzx2FI9WO723R1YOyKeLaw/kPTQ3z9o\neSTwl4Gh0tyRwCWtTPrSFrs9AYa0Uoe2RuWtLH8oadiQV0hjP11KChDTI+KJKvvfALisheI3zgHs\nY3n5465oSd8ijbuCpNWBV2PuPWnlaVYLx7YYsCVwWEUZH5IC3I9yvs2AyRHxLmlk863y+s1J98PH\nkMaiatabFIjMGppjTG1jzIKINCBn6WOMe256tq8DzxaDTnY1sEN+6K+lkcDvIX3hdgGQtBHp4bRF\nSaNbr6X0k+fewB6t1KGtUXn/DOyXHwrsRxold9ucdjvwAXB2rtd8IuLhqPjlUmF6szK/pFMlnZbn\nlyT9uuO3Ofl44Kx8FQep0fIk6cG+ajYgdcXPky7pIElXFI7puEIZzXkGkB7wG5MD+0ukQD+I9BDh\n4y2UadZIHGMqdHKMWWCljzH1vi/mqX4TcDdwSAtp/yT9PLEPcDIpmEwhPWTXP+fZAHgYeAZ4jHzf\nmnQv+RbSiL/3kka1jpw2ivnvAZ9Huu/9OOnL/F1SN/E3SVdwFxbKP5V87z5vezwpAC7bSedkJdJI\nxM+RRhw/sJA2FLiSFGgmAzcCq+a0YaRRd5co5B8NTKtSxhDgemBano4nD2JbyHMq8NOKdd8g/TLi\nCWBEvT8/njy1NTnGVD3uTokxpB6lJuBV0ojgTcAdOe3AvPwCaZT0pjwNK5RV6hjjUcGtW5O0N7Bt\nRGxX77qYWfk4xnRPfubGuq380+nDgKovuTIzWxiOMd2Xn7mxbknSkcAjpHdn/KPe9TGzcnGM6d58\nW8rMzMxKxT03ZmZmVipu3JiZmVmp+IHiBjN06NAYPnx4vathNp8JEya8GhFLt53TGpljjDWqzowx\nbtw0mOHDhzN+/Ph6V8NsPpKer3cdbOE5xlij6swY49tSZmZmVipu3JiZmVmpuHFjZmZmpeJnbhpM\n07/fZuNjbqt5Ofcf95Wal2FmjaezY4xjiTUi99yYmZlZqbhxY2ZmZqXixo2ZmZmVihs3ZmZmVipu\n3JiZmVmpuHFTQ5LGSjq7ct7MrDM4xphV58aNmZmZlUrpGzeSvixpgqQmSRMl7SPpMkmnVeQbL2lf\nSb0kHS1psqTnJN0hadWcZ5SkZySdlvfXW9K6kv4u6SlJ03Ka6nO0ZtbVHGPMGk+pGzeSlgeuAQ6J\niBHAdsDpwGRg5+YAIWk1YB3gKuBgYDSwSUSsDPwd+ENht8OAxyJiRETMBi4AbomItYHPAHsDW3fF\n8ZlZfTnGmDWmUjdugK8AkyJiHEBEPAPcAvQF+gMb53y7ATdExFvA14FzIuL1nHY6sJmkFfLyosCV\nhTI2BU7J+38ZeBJYvSOVlDQmX9WNn/XeWx07QjOrJ8cYswZU9sbNcsArFeteA5YiBY+v53W7Ahfl\n+SHAEblLuAl4CHg17wvg3Yj4oLC/7YFxkqbm/J+ig+c1Is6PiJERMbLvgMEd2dTM6ssxxqwBlX1s\nqRnAMhXrlgYeB/4GXCHpgpynebCV6cBVEXFe5c4kjapYXoXUzfzViLgtr3uwMw/AzBqaY4xZAyp7\nz81twJqS/g9A0prAlsD1EXEfMAs4Crg8Ij7K21wN7CNpUN5mI0l/bOEBvoFAAONz3p2A4cDitTsk\nM2sgjjFmDajUPTcR8ZKkHYHTJS1GCjT7RsRjOctlwNHAhoXNfkfqHn5AUgDvAD+JiKiMPRHxmKTf\nA49Leg24BDgGOEHS1Foem5nVn2OMWWNSRNS7Dlaw+AprxLr7nlXzcu4/7is1L8PKRdKEiBhZ73rY\nwunsGONYYp2lM2NM2W9LmZmZWQ/jxo2ZmZmVihs3ZmZmViqlfqC4OxqxwiDfwzazmnGMsZ7APTdm\nZmZWKm7cmJmZWam4cWNmZmal4saNmZmZlYofKG40/54Ax1Z7C7tZG471CzmtHbpjjPFn2zrIPTdm\nZmZWKm7cmJmZWam4cWNmZmal4saNmZmZlYobNx0kaQ9JE/P8DpJuKKT9oH41M7MycIwxW3hu3CyE\niLg2IrYFkLQMcHydq2RmJeIYY7Zguk3jRlJvSb+RNFXSJEkPSBopqb+kMyRNkfS8pGskLZ23uVTS\nbwv7GCtpbJ6fJmnnQtrZhbRBki7L5Twn6S+Shlap0x6SJkoaAIwHFpPUJOlASY9W5D1R0hW1ODdm\ntvAcY8zKo9s0boCvAF8C1o6ItYBfAjsDJwPrAZ8CVgZeB07N23wf+JqkTSRtDowCftiOso4ElgDW\nAdbI80e0lDki3gN2B96LiBHAH4E1JW1QyLYrcFG7jtTM6sExxqwkutNL/F4Glge+I+nGiLgBuEHS\nS8A++cuPpNOBCZL2iIg3JO0LnAf0BfaMiLfbUdZPgEUiYg4wR9I/gXXbW9GIeFPSdcBewMOSNgb6\nA7dVyy9pDDAGYMXB7S3FzDqZY4xZSXSbnpuImADsAuwEPCvpQUmbAkOAs3NXbRNwDfAOsFTe7lbS\ncX4YEXe1s7j1gD/lbugmUlDo6LkaC3xD0qKkK6pLI2J2C8d2fkSMjIiRSw/oYClm1ikcY8zKo9s0\nbgAi4paI+CopqNwKXA5MB/aLiBF5WjMihkbEKwCS9iNdkb2br7CazQZ6F5aHFOavBx4ldU+PAC5c\ngOr+DXgX+BrwddxdbNbwHGPMyqHbNG4k7SXpTEm9IuJ94P6cdDXwfUmL5HzbSjopz68C/BzYF9gH\nOEHSynm7F4ENcr7hwBaF4gYDD0fER5LWIgWPxduo4kxgEUmLAeTu5otJ9+Zfi4jHFvjgzazmHGPM\nyqPbNG6Aa4HlgKmSngSOA75NCizTgMclTQIOAS6T1IvUbXtSRDwbEZOAXwNjc9oxwE65S/gk4NJC\nWT8GzpL0FHA4sD+wvqSzWqnfY8BTwL8kbZnXjQWG4ysqs+7AMcasJBTh0VZrRdLipC7tNSPi5fZs\nM3IFxfgxta2XlVSNR06WNCEiRta0EOuQHhNjPCp4j9CZMaY79dx0R8cBf25v0DEz6yDHGLMq3Lip\nAUnrSHoOWBs4tN71MbNycYwxa113es9NtxERT5Je9mVm1ukcY8xa554bMzMzKxX33DSaFTaEY8fX\nuxZmVlaOMdYDuOfGzMzMSsWNGzMzMysVN27MzMysVPzMTYN5+s2pbHvd12qy7xu2v6km+zWz7mNh\nYoxjiHUX7rkxMzOzUnHjxszMzErFjRszMzMrFTduzMzMrFTcuDEzM7NSaYjGjaRRkt7ppH01SVqn\nhbRxkhZ6kDlJe0sakOdPkHRUnl9K0jcWdv9m1rkcY8x6ltL9FDwiRtRy/5J6A6cD1wPvRcQRheQv\nAN8ALqtlHcysfhxjzBpfmz03kn4g6SVJkyQdJukpSTvntPUk3SVpiqRni1cs+QrmJ5JulvSMpMck\nrZXTBku6RtKLkh4EPltR5o45/9OSnpD0lUJaSPpxvnqaZ7tC+sg8v3Mue7Kk3xaPN18BXZTr/oKk\nCyT1z2nH5rTf5uN+UdJOedMHgIHAfZK2kjRW0tmSRgHnAqNy3a+QdEZF3e6XtH9b59ysJ3GMcYwx\n62ytNm4kjSBdQXwpItYC+gOr5LQBwF+BqyJiDWBT4GBJXyjs4lvAtyJiVWAKcFhefyiwVN7X/wGb\nF8rcCLgY2DMiVgN+AFwtaXBhv6tExIiIuLeVug8G/gAcEBFrAtcwb4Abm49/bWA1YAWgeIW0A/C7\nfNwnAafk9Tvnv5tExM3NmSNiHHA2MC4i1s/7/6akRXJ9hgPrA1e2VGeznsYxxjHGrBba6rnZHHgk\nIp7Iyycz91bW54A+EXEuQETMIHWV7lbY/vqIeD3PTwBWzPNbAFdGxKyI+JD0JW22E3BLRDyc9zsO\nmAwUX6l5XTuO7TPA+xFxW97PX4HnACQtBmwFnBgRH0XETFLQKNb98YiYkOfHF+reXn8DPgS2zcu7\nATdFxBuVGSWNkTRe0viZb8/sYDFm3ZpjTOIYY9aJ2nrmZgjwavNCRLwv6dVC2kBJTYX8/YD7C8vF\nL9lHQO88vxTweiHtlYoyt6jY78C8TbPX2qh3tTKK5QwiNeyulTQ7r+ud699S3Tv08HVEzJZ0MbAX\n8CdgV+BnLeQ9HzgfYInVBkdHyjHr5hxj5tbdMcask7TVuHmb9CUFQNKiwNC8OB14bQEfrnsDKHYB\nL1eYnw7cGhGjF2C/rZVRLOdlYBawdURMXshyWjMWmChpU2AYcEsNyzLrjhxjFs5YHGPM5tPWlcLd\nwEaS1sjLPyZ9YQEeBGZJGg0gqa+k0yVt0Y5y/w7sIqlPDma7F9KuAbaUtGbe79KSrpS0fDuPqdmD\nwBKSNs/72ZLc7RsRc3I5B0tSTh8j6eB27Le5T3fJFtKWaN5nDmoPAZcAl0fERx08BrOyc4yZn2OM\n2UJqtXETEY8CvwDukPQIqQv25ZQUHwLbAPtJmgw8CSxKClZtOQl4j3R/+l6g+NDck8C+wJWSJgHj\ngH/m++3tlu/D7wNckOu3M+kedbPvAYsDTTl9O1IwassM4HZggqR9KtJuBVYHZkgakteNBYYDF3Wk\n/mY9gWNMVY4xZgtJEa3ffpWkKGSS9BawfUTcVevKlUG+yjwt/7qhTUusNjg+d8pmNanLDdvfVJP9\nWs8gaUJEjKzBfh1jFkJXxhjHEKulzowxbf0UfDDwhqRN8vJ2gIBHO6PwssvvtDiOuT/xNLMCx5iF\n4xhjVl2rDxRHxFuS9gX+kO/xfgDsWu2nhjYvSXsAPwf+HBEX17k6Zg3JMWbBOcaYtazN4Rci4mrg\n6i6oS6lExFjmfbeGmVXhGLNgHGPMWla6saW6u9WWWN33tc2sZhxjrCdoiFHBzczMzDqLGzdmZmZW\nKm7cmJmZWam4cWNmZmal4geKG0zM+C8f/HzcQu2j389GdUpdzKx8HGOsJ3DPjZmZmZWKGzdmZmZW\nKm7cmJmZWam4cWNmZmal0qMbN5KOlXRjnv++pHPbsc0Okm6ofe3MrLtzjDGrD/9aKouIs9uZ71rg\n2hpXx8xKxjHGrOuUrudG0nBJsyUdImmypJUkXSRpiqQXJF0gqX+V7YpXWIMkXSfpJUkPSNpf0js5\nbQ9JEwvbHSLpKUlNku6X9Nm8fpSkZyV9T9LDeV8ndNV5MLPacIwxa3yla9xkvYC+EbEmcHZeXhtY\nDVgBOKKN7Y8AlgJWBL4M7Fktk6TtgIOBL0bECOAM4FpJA3KWYUCviNgA+CLwE0krLcyBmVlDcIwx\na2BlbdxACgCLAVsBJ0bERxExkxSIdmtj2y8AV0TEzIh4C7ighXw7AJdHxL8BIuKKvP5/898+wPk5\nbSLwISmYzUPSGEnjJY1/5d232n+EZlZPjjFmDarMz9y8BgwiNeCulTQ7r+8N9Gtj2yHAq4XlF1rI\ntxzwVMW614FlgZeB9yPiw0LaR7n8eUTE+eQAteGwNaONuplZY3CMMWtQZW7cQPryzwK2jojJHdju\nbVLQajashXwzgGWaFyQJGJrXl7lXzMwSxxizBlTqL0dEzAGuAQ7OQaG5e/bgNja9G9hNUm9Jg4C9\nWsh3DTBa0rJ5+VvAB8BDC197M2t0jjFmjansPTcA3wPOBJpy7HkaOKCNbX4JXAo8m6dLgfUqM0XE\nXyStBtwpqTfwCrBdRLyfyzKz8nOMMWswivDt12okKfLJkbQjcHpE1PxXCBsOWzPu2e+8hdqHR+y1\nWpA0ISJG1rseZeEYYzavzowxpb4ttaAk7QmMl9Q/Xy2NBv5R52qZWUk4xpjVlhs31V0KjCf9SqGJ\ndPvu0LrWyMzKxDHGrIZ6wjM3HZbfVbFfvethZuXkGGNWW+65MTMzs1Jxz02D0fID/bCemdWMY4z1\nBO65MTMzs1Jx48bMzMxKxY0bMzMzKxU/c9NgXn/2SS7fda2alzP6ykk1L8PMGk+tY4xjizUC99yY\nmZlZqbhxY2ZmZqXixo2ZmZmVihs3ZmZmVipu3JiZmVmpuHFjZmZmpeLGjZmZmZWKGzcVJA2XNFvS\nIZImS1pR0k2SJkmaJuliSf1y3t6STpP0rKQpki6UtGhOW0/SXXn9s5IOre+RmVkjcIwxqz03bqrr\nBfSNiDWBXwEvRsRawDrAJsC+Od+BwEhgBLA28AngCEkDgL8CV0XEGsCmwMGSvtC1h2FmDcoxxqyG\n/Ibill2b/44mn6eIeFfSg8DqOW0n4OKImAkgaTvgI2ALoE9EnJu3myHpMmA34M7KgiSNAcYADB3g\nfxKzHsIxxqxG/Clv2Wv57+eBoyStCMwGlgcuzmlDgTebN4iI9wEkDQEGSmoq7K8fcH+1giLifOB8\ngFWW7B+deAxm1rgcY8xqxI2bVuSu3xuBg4ALImKOpKsKWV4Gli7kHwwMAKYDr0XEiK6sr5l1L44x\nZrXhZ25a15cUSMbnoLMpsDGweE7/M7CHpP6SegMXAQcADwKzJI0GkNRX0umStuj6QzCzBuYYY1YD\n7rlpRUS8JekY4CZJrwO3AfU3tIAAAAuySURBVD8ALpL0FHAaMAyYArwP3AscHxEfStoG+LWkYwEB\ntwN31+EwzKxBOcaY1YYifPu1kayyZP/45ZeG17yc0VdOqnkZVi6SJkTEyHrXwxZOrWOMY4stqM6M\nMb4tZWZmZqXixo2ZmZmVihs3ZmZmVip+oLjBLLnKOoy+cny9q2FmJeUYYz2Be27MzMysVNy4MTMz\ns1Jx48bMzMxKxY0bMzMzKxU3bszMzKxU/IbiBiPpv8DketejHYYCr9a7Em3oDnWE7lPPNSNiYL0r\nYQvHMaZTdYc6QvepZ6fFGP8UvPFM7g6vuJc0vtHr2R3qCN2rnvWug3UKx5hO0h3qCN2rnp21L9+W\nMjMzs1Jx48bMzMxKxY2bxnN+vSvQTt2hnt2hjuB6WtfqLv+O3aGe3aGO0APr6QeKzczMrFTcc2Nm\nZmal4sZNF5O0kaT7JT0taZKk3VvIt3tOf1rSA5I26uJ6bpHLbZI0VdLBVfLsIem9nKd5urkL6zhc\nUlSU3yRpaJW8h0maks/nnZJW7cJ6blaljq9KuqAiX13Op6Qxkt6VdGhh3VBJ10t6Jv/7nyqparxo\n72fauoZjTKfW0TGmc+rX9TEmIjx10QQsCkwHdsvLqwFvAv9TkW+9vH71vLwr8CKwSBfVczngHWCL\nvLwq8F9gk4p8ewDj6ng+h6ePcJv5tgZeAJbJy4cDD9ax3v2AJmCjep9P4DfAVcAE4NDC+j8B5wIC\nFgPGAwdU2b5dn2lPXfbv6RjTufV0jFn4utQlxrjnpmttARARV+S/TwM3AaMr8n0LuCkipuZ8V5I+\nAKO6qJ6zgW9HxB25/GeAp0gBsTvaHbg4Iv6Tl88CPi1pjTrV5yjgroh4qE7lF10eEbuQ/mMBQNJA\nYHvgtEjeBc4jfS4rtfczbV3DMaY+HGNaVpcY48ZN1xoBTK1YNwVYp0q+KRXrplbJVxMR8UpEXNu8\nnLtX1wXuqZL9k5JukTRZ0u2S/rcr6lgk6WJJEyWNl/TtKlnmOZ8R8R7pSqBLzmeRpGWB/YHjWsjS\npeczIu6usnr1/PeZwrpqn1No/2fauoZjTA04xiy4esUYN2661mLA+xXr3s/rFyRfzUn6BPAX4OSI\nmFiR/DRwPamrcy3gRuBmSUO6qHrvABcCp0bEusBBwHmSPleRr2HOJ/Bj4JKIeKlKWr3PZ7PFgJkR\nMaewrqXz1Ujn1hxjOptjTG3UPMa4cdO13gH6V6xbLK9fkHw1JWkD4D7gjxEx31VARNwdET+KiJcj\nYk5EnAF8BGzaFfWLiFcjYu+IeLS5PsANwLYVWRvlfPYmdV9fVC293uez4B1g0YqH+1o6Xw1xbu1j\njjGdyDGmZmoeY9y46VpPApX3YNcCHq+Sb83mBUkidc1V5quZHHRuBg6KiJNayLOipOUqVwOzal2/\nXP6SklarWN2rSvmV53MgMAx4orY1nM/ngQ8j4uFqifU+nwVTSM9EFM9ttc8ptP8zbV3DMaYTOcbU\nTO1jTFc+Nd3TJ6Av8DywZ15en/TU9xoV+dam8DQ4sC/p3mSfLqpnv1zeTm3k+xVwJzAgL+8JzAAG\ndVE9vwa8DKyUl9clPbT2mYp8Xwb+BXwiL/8S+Hsd/v2PBm5rxPMJjGPeXzJcBowlBb4lgInNn9uK\n7dr1mfbUZZ8xx5jOradjTOfVrUtjTJeeeE8B8CngXtIDUk80f7mBE4CjCvlGk1qsU4F/AOt2YR1H\nk1rVTRXTscV65gB1Huk+blP+8G7QxefzwEL9HmXuzwUvKn5RgIOByfl83twchLq4rucBYyvW1e18\nAr0L5+494D95/gRgCPBn0n9AU4CfM/eN5jsAN7T1mfZUn8kxptPr6hiz4PWpW4zx8AtmZmZWKn7m\nxszMzErFjRszMzMrFTduzMzMrFTcuDEzM7NScePGzMzMSsWNGzMzMysVN25KTtI0Sf+S1FQx7dvO\nbXfuxLqMkjSnUIepkp5sT13a2O//SppQWN5b0oA8f4Kkoxa27hXlRT43zccxOQ+ot0M7t+8j6bsL\nWPbZ+bzdIWmJwvolJT0haakF2a/ZgnKMcYxpSF39kiFPXTsB04Cdu3rbFvY3CninYt0I4HXgi51U\nRm/gbWBoDc9pACMr1m1KGsytzTfzAhsBExeg3E+T3i4q4Bjg4ELaRcDoen3OPPXcyTGmJufUMWYh\nJ/fc9HCSdpT0aOEqZ/cW8o2RNEnSU3navZC2f057RtIDkjZsb/kR0QTcBHw172s1SbflK5VJkk6X\ntGhO2z5fOUzK6T/O60dJah5E7QFgIHCfpK0kjc1XIl+W9B+lgeWa6/19Sffk+c9LelDSlLzvb3fk\nPEbEPaS3b66X9/cJSTfluk6TdLGkfpJWJw28t3o+50MlrSTp+lz285JO1rwDyjVbC3ggUqR5kBS0\nkbQlsGREXN6ROpt1BccYx5i6qHfrylNtJ1q5MgIGAx8CG+flLUkDqC1Z3BZYnPSq9OF5/UqkL09f\nYCfmHXtld+A5oFeV8kZRcVWV118JHE+6WngY+FlevxgwHjgkL/8HGJXnhwDXAUOL+wWGk656hubl\nscDZpKutl4AtCuXeDewHfAJ4F9gqrx9BGj9mtRbO2zxXVbneu5JeL/6JwjH9tnAcTwM/yMt7kK+q\n8raPAyfm+UHAI8BeVcrdFTg5z28FnJv/bZ4gvZ78cuAO8uvhPXnqiskxxjGmEae6V8BTjf+BU/D4\nF/OP4bJFTu9XyLtI/lJtUNh25xxgXgVOAkZU7P8K4JSKda8An61Sl/kCD7BJ/pJ/OgeNOcDihfQD\ngHF5/jHgD/lL1qvaflsKPHn+DOC8PP/JHCiWAPYHxlfU62rgpy2c08jnpjhmyp3FYyY9z7ZIYfky\n4Nd5vhh4RuT9DSzk/R7w1yrlrgrcB/TJx7InKah+D/hF3u+g/O+9eLW6e/LU2ZNjjGNMI06+LdUz\nHBgRIyqmO3K35KGSHpM0hblDyM/zuYiIWcD/AcsD/8zdmvvk5CHA7io8SEi6AlumhboMKOSdQrqa\n2CUiHgGWAz6IiHcK+V8Dls3zXyPdc74RmCHpyA6eh0uAHXK38S7ATRHxZj6GERXHsHFe35Kdm88l\ncD3wQkTcW0j/PHBL7oZvynWv9n0bQgo8DxXKPoR0tTSPiHiGdDX7BLA08CJphNxzgA2B+yLibdII\numu385yYdQbHmMQxpkH0qXcFrK52J121bBoRzyk9/f9utYwRMYkUYET6El0raRwwHXg4Io5oZ5nv\n5S9rNTOAfpIGRsR/87ql83oiYnqu7wGSNgVulHQf6UqsTRExXtLrpKCwK+lKhHwMT0TEJu08hkqH\nA5Mk/S4i7snn8UbgIOCCiJgj6aoWtp2e/34qIj5oxzGcAJwgqR/p3v9uERHF+/yk7vGPFvBYzDqT\nY0ziGNPF3HPTsw0m3SN+Pn9wDwdmUtGil7S+pL/mgBCkD/yHpKuBq4HRkpbPeVeRdLWk/gtQnxdI\n98MPzPsaBOwNXCNpGUn3SVou530EeLPKPmbmv0u2UMalwBhS1/Ited2tpKuqzXK5i0u6UNL/tKfS\nEfECcArwm3we+wIDSN3Qc3KQ3Ji553UmMFBSn4h4kfTg3sG57F6SjpI0uo1ijwH+nP9DgHRFvLHS\nTzdXBqa2p+5mNeYYkzjGdLV63xfzVNuJ1h/2WwoYl/M8RHrY71LSVcxazL0fLuCXpAfWnsrTdwv7\n+R7wJDCJdM96lxbKG0WVh/0q8qxKCgST8nQ80Cen/RCYnNdPBo6r3G+u699I99j3oXA/PKevQroK\n+3VFuZ8nBYDme9w/p8oDiznvPA/75XUDSF24P8zLPwP+DUwETgW2A94CDiM9XPgc6RmDdUkPT/4l\nH9MU0kN7Q1o5R58mBd6+hXXDSP8hPNdcB0+eumJyjHGMacRJudJmZmZmpeDbUmZmZlYqbtyYmZlZ\nqbhxY2ZmZqXixo2ZmZmVihs3ZmZmVipu3JiZmVmpuHFjZmZmpeLGjZmZmZWKGzdmZmZWKv8fqZBi\nTduN31kAAAAASUVORK5CYII=\n",
            "text/plain": [
              "\u003cFigure size 576x216 with 2 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "# Evaluate all evaluation metrics.\n",
        "overall_error1 = error_rate(labels_test, scores_unconstrained_test)\n",
        "overall_error2 = error_rate(labels_test, scores_constrained_test)\n",
        "\n",
        "overall_fpr1 = false_positive_rate(labels_test, scores_unconstrained_test)\n",
        "overall_fpr2 = false_positive_rate(labels_test, scores_constrained_test)\n",
        "\n",
        "group_fprs1 = group_false_positive_rates(labels_test, scores_unconstrained_test, groups_test)\n",
        "group_fprs2 = group_false_positive_rates(labels_test, scores_constrained_test, groups_test)\n",
        "\n",
        "# Plot FPRs and FNRs.\n",
        "fig, ax = plt.subplots(1, 2, figsize=(8, 3))\n",
        "\n",
        "fpr_percentages1 = [x * 100 for x in [overall_fpr1] + group_fprs1]\n",
        "fpr_percentages2 = [x * 100 for x in [overall_fpr2] + group_fprs2]\n",
        "\n",
        "ax[0].barh(-np.arange(len(group_names) + 1), fpr_percentages1, color=cb_palette)\n",
        "ax[0].set_yticks(-np.arange(len(group_names) + 1))\n",
        "ax[0].set_yticklabels([\"overall\"] + group_names)\n",
        "ax[0].set_xlabel(\"False Positive Rate %\")\n",
        "ax[0].set_xlim([0, max(10, max(fpr_percentages1) + 1)])\n",
        "ax[0].set_title(\"Unconstrained Model\\nAccuracy = %.2f%%\" % ((1 - overall_error1) * 100),\n",
        "                fontsize=13)\n",
        "\n",
        "ax[1].barh(-np.arange(len(group_names) + 1), fpr_percentages2, color=cb_palette)\n",
        "ax[1].set_yticks(-np.arange(len(group_names) + 1))\n",
        "ax[1].set_yticklabels([\"overall\"] + group_names)\n",
        "ax[1].set_xlabel(\"False Positive Rate %\")\n",
        "ax[1].set_xlim([0, max(10, max(fpr_percentages2) + 1)])\n",
        "ax[1].set_title(\"Constrained Model\\nAccuracy = %.2f%%\" % ((1 - overall_error2) * 100),\n",
        "                fontsize=13)\n",
        "\n",
        "fig.tight_layout()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IxshklMhWs-B"
      },
      "source": [
        "Compared to the unconstrained model, the constrained model yields significantly lower false positive rates for the sexuality-related comments, and does so with only a slight dip in the overall accuracy."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ojUJFvHpWs-B"
      },
      "source": [
        "## Train with robust optimization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sjaERfx_Ws-C"
      },
      "source": [
        "We have seen that by using constrained optimization, we can learn a model that has low false positive rates (FPRs) for all groups. However, the models we have trained so far all have high false negative rates (FNRs). We could try training a model with additional constraints on the per-group FNRs, but this is likely to overconstrain the model and produce a trivial solution, such as e.g. a model that makes the same prediction on all inputs.\n",
        "\n",
        "Instead, a more reasonable solution would be to train a model by optimizing for the worst-case  metric, i.e. minimizing the worst among all the metrics we care about. In the following, we seek to optimize the mean of the worst-case FPR and the worst-case FNR, i.e. the mean of the maximum among *the overall FPR* and *the FPRs for individual groups*, and the maximum among *the overall FNR* and *the FNRs for individual groups*. More formally, this can be stated as:\n",
        "\n",
        "$$\n",
        "minimize \\text{       }\\frac{1}{2}max\\big\\{OverallFPR, FPR(sexuality), \\ldots\\big\\} + \\frac{1}{2}\\max\\big\\{OverallFNR, FNR(sexuality), \\ldots\\big\\}.\n",
        "$$\n",
        "\n",
        "Since the goal here is to be robust to a set of different evaluation metrics, problems of this form are often referred to as *robust optimization problems*.\n",
        "\n",
        "Below, we solve the above robust optimization problem using the TF constrained optimization library. We once again use two streams of minibatches to better handle the small group sizes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ei947kcxWs-D"
      },
      "outputs": [],
      "source": [
        "# Set random seed.\n",
        "np.random.seed(121212)\n",
        "# You may need to use \"tf.set_random_seed(212121)\", instead of the below, in\n",
        "# earlier versions of TensorFlow.\n",
        "tf.random.set_seed(212121)\n",
        "\n",
        "# Features tensor.\n",
        "batch_shape = (hparams[\"batch_size\"], hparams['max_sequence_length'])\n",
        "features_tensor = tf.Variable(np.zeros(batch_shape, dtype='int32'), name='x')\n",
        "features_tensor_sen = tf.Variable(np.zeros(batch_shape, dtype='int32'), name='x_sen')\n",
        "\n",
        "# Label tensors.\n",
        "batch_shape = (hparams[\"batch_size\"], 1)\n",
        "labels_tensor = tf.Variable(np.zeros(batch_shape, dtype='float32'), name='labels')\n",
        "labels_tensor_sen = tf.Variable(np.zeros(batch_shape, dtype='float32'), name='labels_sen')\n",
        "\n",
        "# Group tensors.\n",
        "batch_shape = (hparams[\"batch_size\"], num_groups)\n",
        "groups_tensor_sen = tf.Variable(np.zeros(batch_shape, dtype='float32'), name='groups_sen')\n",
        "\n",
        "# Create model and predictions functions.\n",
        "model_robust = create_model()\n",
        "\n",
        "def predictions():\n",
        "  return model_robust(features_tensor)\n",
        "\n",
        "def predictions_sen():\n",
        "  return model_robust(features_tensor_sen)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5Uc6biFCWs-F"
      },
      "source": [
        "We formulate a rate minimization problem by creating a list of rate metrics we are interested in and using `tfco.upper_bound` to define the maximum of these metrics as the optimization objective."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mcoozVktWs-F"
      },
      "outputs": [],
      "source": [
        "# Set up separate contexts for the two minibatch streams.\n",
        "context = tfco.rate_context(predictions, lambda: labels_tensor)\n",
        "context_sen = tfco.rate_context(predictions_sen, lambda: labels_tensor_sen)\n",
        "\n",
        "# List of metrics we are interested in.\n",
        "fprs = []\n",
        "fnrs = []\n",
        "\n",
        "# Add overall FPR and overall FNR, computed using the first stream.\n",
        "fprs.append(tfco.false_positive_rate(context))\n",
        "fnrs.append(tfco.false_negative_rate(context))\n",
        "\n",
        "# Add group-specific FPRs and FNRs, computed using the second stream.\n",
        "for ii in range(num_groups):\n",
        "  context_sen_subset = context_sen.subset(\n",
        "      lambda kk=ii: groups_tensor_sen[:, kk] \u003e 0)\n",
        "  # We pass the group index ii as a default argument to the the function, so\n",
        "  # that the function retains a local copy of this variable, and is unaffected\n",
        "  # by changes to the variable outside the function.\n",
        "  fprs.append(tfco.false_positive_rate(context_sen_subset))\n",
        "  fnrs.append(tfco.false_negative_rate(context_sen_subset))\n",
        "\n",
        "# Set up optimization problem with the \"max\" of all the metrics as the\n",
        "# objective.\n",
        "problem = tfco.RateMinimizationProblem(0.5 * tfco.upper_bound(fprs) +\n",
        "                                       0.5 * tfco.upper_bound(fnrs))\n",
        "\n",
        "# Set up constrained optimizer.\n",
        "optimizer = tfco.ProxyLagrangianOptimizerV2(\n",
        "    optimizer=tf.keras.optimizers.Adam(learning_rate=hparams[\"learning_rate\"]),\n",
        "    num_constraints=problem.num_constraints)\n",
        "\n",
        "# List of variables to optimize include the model weights,\n",
        "# and the trainable variables from the rate minimization problem and\n",
        "# the constrained optimizer.\n",
        "var_list = (\n",
        "    model_robust.trainable_weights + list(problem.trainable_variables) +\n",
        "    optimizer.trainable_variables())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zW-1LU78Ws-H"
      },
      "source": [
        "During training, we periodically snapshot the model parameters and evaluate the worst-case metric value (i.e. the \"max\" objective) on the validation set. At the end of training, we choose the iterate that yields the minimum worst-case metric.\n",
        "\n",
        "**Note**: The following code cell may take several minutes to run."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "id": "6IdlsJltWs-H",
        "outputId": "084f4427-cb4d-4b74-ca0f-6211cecd0140"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Iteration 747 / 747: Elapsed time = 1126 secs, Max objective = 0.199 "
          ]
        }
      ],
      "source": [
        "# Create temporary directory for snapshotting models.\n",
        "temp_directory = tempfile.mktemp()\n",
        "os.mkdir(temp_directory)\n",
        "\n",
        "# Indices of sensitive group members.\n",
        "protected_group_indices = np.nonzero(groups_train.sum(axis=1))[0]\n",
        "\n",
        "num_examples = text_train.shape[0]\n",
        "num_examples_sen = protected_group_indices.shape[0]\n",
        "batch_size = hparams[\"batch_size\"]\n",
        "\n",
        "# Number of steps needed for one epoch over the training sample.\n",
        "num_steps = int(num_examples / batch_size)\n",
        "\n",
        "# Number of steps after which to snap-shot the model.\n",
        "skip_steps = int(num_steps / 10)\n",
        "\n",
        "# List of recorded robust optimization objectives.\n",
        "objectives_list = []\n",
        "\n",
        "start_time = time.time()\n",
        "\n",
        "# Loop over minibatches.\n",
        "for batch_index in range(num_steps):\n",
        "    # Indices for current minibatch in the first stream.\n",
        "    batch_indices = np.arange(batch_index * batch_size, (batch_index + 1) * batch_size)\n",
        "    batch_indices = [ind % num_examples for ind in batch_indices]\n",
        "\n",
        "    # Indices for current minibatch in the second stream.\n",
        "    batch_indices_sen = np.arange(batch_index * batch_size, (batch_index + 1) * batch_size)\n",
        "    batch_indices_sen = [protected_group_indices[ind % num_examples_sen]\n",
        "                         for ind in batch_indices_sen]\n",
        "\n",
        "    # Assign features, labels, groups from the minibatches to the respective tensors.\n",
        "    features_tensor.assign(text_train[batch_indices, :])\n",
        "    labels_tensor.assign(labels_train[batch_indices])\n",
        "\n",
        "    features_tensor_sen.assign(text_train[batch_indices_sen, :])\n",
        "    labels_tensor_sen.assign(labels_train[batch_indices_sen])\n",
        "    groups_tensor_sen.assign(groups_train[batch_indices_sen, :])\n",
        "\n",
        "    # Gradient update.\n",
        "    optimizer.minimize(problem, var_list=var_list)\n",
        "\n",
        "    # Once in every skip_steps steps, snapshot model parameters\n",
        "    # and evaluate robust optimization objective on the validation set.\n",
        "    all_fprs = []\n",
        "    all_fnrs = []\n",
        "\n",
        "    if batch_index % skip_steps == 0:\n",
        "        # Evaluate on validation set.\n",
        "        scores_vali = model_robust.predict(text_vali)\n",
        "\n",
        "        fpr = false_positive_rate(labels_vali, scores_vali)\n",
        "        all_fprs += [fpr]\n",
        "\n",
        "        fnr = false_negative_rate(labels_vali, scores_vali)\n",
        "        all_fnrs += [fnr]\n",
        "\n",
        "        group_fprs = group_false_positive_rates(labels_vali, scores_vali, groups_vali)\n",
        "        group_fprs = [group_fprs[jj] for jj in range(num_groups)]\n",
        "        all_fprs += group_fprs\n",
        "\n",
        "        group_fnrs = group_false_negative_rates(labels_vali, scores_vali, groups_vali)\n",
        "        group_fnrs = [group_fnrs[jj] for jj in range(num_groups)]\n",
        "        all_fnrs += group_fnrs\n",
        "\n",
        "        objectives_list.append(0.5 * max(all_fprs) + 0.5 * max(all_fnrs))\n",
        "\n",
        "        # Save model weights to temporary directory.\n",
        "        model_robust.save_weights(\n",
        "            temp_directory + \"/robust_\" +\n",
        "            str(int(batch_index / skip_steps)) + \".h5\")\n",
        "\n",
        "    # Display most recently recorded objective.\n",
        "    elapsed_time = time.time() - start_time\n",
        "    sys.stdout.write(\n",
        "        \"\\rIteration %d / %d: Elapsed time = %d secs, Max objective = %.3f \" %\n",
        "        (batch_index + 1, num_steps, elapsed_time, objectives_list[-1]))\n",
        "\n",
        "# Select the best model from the recorded iterates. Pick the one with the\n",
        "# minimum \"max objective\", and load its weights.\n",
        "best_index = np.argmin(objectives_list)\n",
        "model_robust.load_weights(temp_directory + \"/robust_\" + str(best_index) + \".h5\")\n",
        "\n",
        "# Remove temporary directory.\n",
        "shutil.rmtree(temp_directory)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I_PdmG2qWs-J"
      },
      "source": [
        "We plot various evaluation metrics for the trained model on the test set."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 254
        },
        "id": "0emCk3Tvwj7U",
        "outputId": "c80c1c11-3c30-4227-c470-6b5cd36da129"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAisAAADtCAYAAAB+gDWoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd7wcVf3/8debAIFAEkIvCpEaitSg\nICIRVBCR3iLIlw5ShAgW/CFNpakUQUEUjfTepAsSESnhBkJLIQRCr0ESkkBC+fz+OGfJZLO35t7s\n3M37+XjsY3dnzsycmd357GfOOburiMDMzMysrOardwXMzMzMWuJkxczMzErNyYqZmZmVmpMVMzMz\nKzUnK2ZmZlZqTlbMzMys1JysWLcmaZikqLpNk/SkpEMltfk9Lql/Xn7HrqxzW0nqJek4SU/kfXpf\n0sN5v3p0wfb2zft/ezPz55P0ci7Tfw63NVLS0HaUHybppg5u64Bc50s6svy8TtK2kh6XNF3SS5KO\nqVHmZ5JezGWekPTtNqx3fknn5Ndm3xrze0u6WNLE/P6/U9JqhfnLSbpf0mRJt0latGr5pfKy63dw\n161EnKxYI7gTWK5wWw/4C/B7YEgd61WTpB0lDWulTG/gfuAQ4LfAF4GNgMuAXwE3tjdhyQnCoFaK\nTQO+KWnpGvO2ABZvzzZLYh/gaWAnSb3qXZnuRNKXgVuAB4H1gZ8Cp0o6oFDmSOAXwM+AdYFrgZsl\nrdfCepcC/gm0lNQMBTYDdgE2Bd4H7pa0cJ7/S2Ai8CVgMeDoquVPBa6NiJFt2VcrNycr1gimR8Qb\nhdu4iPg98Gfg5/WuXA2btKHM6cCKwKYRcWlEjI+IZyPifGArYGvgh23doKRFgHXaUPRtYBwwuMa8\nvYH/tHWbZSBpJWBz4AigB1CKVrNu5AhgPHBERIyOiCuBs0mJCZJESmDOi4grI2JsRPwKGA4c28J6\n9wLeA7asNVPS6sDOwNERMSwingAOAJYGvpeLfTNvdwzwt/y8svyGefnjO7bbVjZOVqyRPQ0sLmmh\nygRJP5L0nKQZkl6TdLaknlXLLSbpOklTJL0t6eTC8idJeq9YOLeUfNY1ImlVSbdIeic3Xz8maYc8\nbygpuG+RlxlUXencnL0fcFZEvF49PwfuS0kfJJVlQtLhkv6eu4smSbowN7X3B6aQPqzva61VB7gN\n+H5VnXqSrnBvq1HfvSU9k7sA3pb0N0n9CvNXyc31H0oaL2m2REjSepLuys32kyRdLWmZ5iooaYKk\nc1rZD/J+jI+IfwM3kxKu6nWtIunW/Hq/LumCYpeCpG9JelTSB/m9M6Qwb7buqdy1MaHwPCQdIWm4\npFfztIUknZu3N0PS85J+XrWepSVdLul/+b10haRlJA3I6/xuVfmN8/SBbTgubbUW8GDM+lPntwKr\nSvo8MABYAbirarl7SUl1c24iJRNTm5m/FfARcF9lQkRMBpoK610OeCk/fhFYtrD8ucApEfFOC3Ww\nbsTJijWyAcCbEfEhgKQfkpqGTwHWBA4nfZidW7XcCcDdwAbA74ATJO3aju1eAiwMfB1YG7gOuD4n\nDUeRuq0eIgXbB2ssv1FevqVWjLuAlSV9rjDtOGBkrvfRpCvRY4CXgW/kMruQPiRachWwkaQ1C9O2\ny/d3FwtK2p60v38h7etgYBBwZaHY1aTuo83ztvcGPl9YxzLAv4BPSVfaWwOrAbfkK/daNiZ1PbTm\n+6TEjnw/SxdX7ha6G5hB6mrYhXSF/oc8f13Sh/M/SF0cxwGnSdqvDdsu+iFwDvDl/PwUUuvCYGBV\n4CfAiZL2ytsVKbnqT3rttgJWAa7JLQkPMbOFoWIXYFRENFVvXNJeORlr7rZ5M/WeH/i4atrEfL9y\nrjvAC1VlXgCWyy16s4mICdHyf72sCrwWEdNrrLcybkWk9wz5XgA5GV6C/BpaY5i/3hUw62ySFgC2\nBQ4EzirMOgoYGhGVgZbjJa0C/LJ4tQzcExEX5cenS/oesAcp6WiLdYBfRsRT+fmpuTVjYkS8L2k6\nMCMi3mhm+eXy/UvNzC/OWwF4JT8eHRFn58fP5dac3SLiDEmVD5h3I+LdliofESMkjSN90Feu9vcm\nXQ1Xf3gcBdxdtd0fA9fm5GwhUvK1XUQ8CmkgL1Dc9wOBnsAe+eoZpTERj5HGLDxQo45vt7QPeR1f\nBlYnjfOBlJS8C+xJGs8EKQnrT+pueysvdzSwvdLg7EOBsRFxSi4/Lu/XZy1HbfRMRFxReP4b4IKI\nqHzIvyTpWFJicjkpGdsEGBgRI3K9jgQOl9SX1O1xjqRFI2JKXsfOwF+b2f4twCMt1O/VZqaPI71+\nRZXuxN75Bmk8SVGlTn1ovvWkJb1rrLOy3j758Ruk1pTnSefBazn5PJOUqH9R0oV53j3AwRExowN1\nsRJwy4o1gu2KV4mkQaJDSVeyJwLkAL8y8HDVsk2kD9TVC9OqWzseBdZoR33uIF0lny5pc0nzR8SD\nEVEr+NZSueL8qIUylYGinxamVdd7JGncS0dcDeylpB8p+bu6RrkNqX1MIQ10rrTOPFGZmRONFwvl\nNwIeqyQquczjwCTSoM6O2ofUAvGSpMqF2TXM2hW0EfBKJVHJ2741Ig6OiE8rdSuuNCJ+ExHFJLgt\nHqt6/hEp8Ribu72mkBKUShK0Eel98Nng0IgYHhH/FxGTmPla7AQg6YuklpdKYjaLiHg/Ip5r4fZB\nM/W+CNhA0hBJPSWtQ2oFquxDPf0b2EfSgqRWqn+RxtI8FhF3k8asXUpqpVkNOLheFbU552TFGsF9\npA+1yu0BYBRwfP7AgZlXgJOrln2/an6tMlOBms3ZzdiP1N20PekbPW/kLqi2ejnff6GFMpVulFcK\n0yZVlZlC+pZER1xJSnS2AHbN67qnRrnetHxMK2M/ptWoW0Uf4CvVXRN5+rJ0QP4A24PUtfNR4XYE\nsLHSAE5IyUFLSWRr89uqeh1Xkd4np+Q6rs/MJK+y3WkR8UmtleXE7nrShzSkVpX7IuKVWuU7Kn/o\nDyG9n6eRusN+mWe/w8z3XJ+qRfvm++r3ZFtNqrHOynor6zyJlER/QHqf3ELqbvuR0rfpNgIuz93A\n15G6J62bcrJijWBq8SqRNF5jE9KHQUXlA7Vv1bK1gmp1YrIoMz9cg9w3XjX/MxExLSJOjYi1SFe7\nlwPnStqmjfvzeN7eHi2U2QJ4tmoAbnW9ewP/a+M2ZxERo4CngN2A3YEbIqLWlfQkWj6mlSv26q8M\nF5eZROqiWL/qtjqpdawjts3b+BqpxaJ4G8/M1pW3ablLp7X5rb4fquVWvq2BkyPi8ogYld+3xYT5\nbWCR3KXZnL8BW+WWr51JY4ea22ZHx6wQEeeQxhytSHo/v0EaxzKG1E1Enl60GvByRFQnqW01Dlhe\nhcHxhfWOyfV6DlgJWC4i1iN98+fCiBjPzCT9/cJ9RxN3KwEnK9Zw8rdl/kIab9IvT5tMCoCbVRXf\nlNRy8mxhWnXgXh8YnR9PBhat+hD5UuWBpH5K347pkbf7fEQcRfpAXruN9f+A1Px+oGr8+Foe9Lk3\n6SukRdX13gAY25ZtNuNK0u9gfI3aXUAAI6h9TIOUdFW2v3FlpqSVKQywJXWzrUL6cCsmnQswczBn\ne+0DDIuI/0REU/FGusqutEiMJH0orlSo3zaS7sutMyPz/lCYf5yk8/PTycyezHyJllXeO599U0XS\nBqQus0riU+n++UqhzPqSHpC0fJ40jNSy9jNSF+f1LWzzFmZPBou32Qbl5m2uK2n3iPggIl7NLZW7\nA/+KiKkR8Sxp0Gt1Ir4NqTu0o+4mfXutMjCcPDB6w+J6I3lL6Vt1XyH9BhHMvPhYIt8vQwcTdyuJ\niPDNt257IwXsm2pMX4r0Ow4XFKb9gDRA9CBScN+dFMDOzPP7kz5kx5NaZVYlffsjgO1zmS3y8xPz\nOvYjjceIvPzipFaRP5K+9rkScCTwCfDlvI6hwGukZuqlm9mvRUhjUF7N21iFdFX5A9KH3FWACuUj\nlz02l9uf1O1xaNW+/QpYp5lt7gtMKDz/Ql7mTaBH1Xr65+ffJo2bOS7XcRtSN9Y1eb5Iid6TeX83\nIH0QvUUa7Azpg2QiKTlajzQ+6Dekbof+tV7n/Pr2bmY/Fs+v86HNzN8478NXSInDi6Sv2q5JapEb\nReo+gJRgfkT6Vlh/0hiRKcBBef6Jef72+bj/hpSgTah6bY6uqsMLpO7L1UgfyI+SvhY+Flgxl3kw\nH7f1Sd9Euh/4b9V6Ktv/exedX7uQ3ruH5v0/IL8uXymU2Z/UgrZ7LnMa6QJg1UKZe4Hjql6jZUmt\nZ0EaqL0ssGyhzCWkC4xN82tzB/AMsEBVHXuQzsF9qqY/DpyRX8PRwGH1jle+zcF7sd4V8M23OblV\nf4hVzftRDrQDC9OGkL49MIP0jZpTmP2DeA/ghhxw3wJ+XrXeU0kf4FNIv9a5DbN+gH81f7BMzmVG\nAHsWlt+UlKxMI31bp7l960kazPhELvs+acDoAcB8VWWDdIX9Z1KS9h5wXmXfcpnLgA+BEc1sb18K\nH7J52sPMmvBVjlH/wrS9SB/wM/JxOR/oVZi/dl7P9Hzsv5ePz9BCmfVIv2g6lfTBdz/w1eZeZ2AC\ncE4z+/GD/Lov08KxfRH4Y368BimBmkrq4rgI6FMou11+DT4kJbLHFub1ze+VyaSum1+RErcJhTK1\nkpXNSR+80/Kx2ZD0len3gKdymeVIrUCTScnc1aQuj+J61snr37ILz7Ef5eP9IanFZ9saZYbkYzqd\n1EqzRdX8WV6v/HpGrVuhTC/gT6RvcE0ljZdZsca2D8vHUFXTNyAle5OBi4EFu+oY+db1N+UX1cy6\nMUkBDIk0vsDmEZKOI33FfO1wMLcG5t9ZMTPrZiQtSxpL9Atgbycq1uicrJiZdT//IX176ISIuKHe\nlTHrau4GMjMzs1LzV5fNzMys1JysmJmZWak5WTEzM7NSc7JiZmZmpeZkxczMzErNyYqZmZmVmpMV\nMzMzKzUnK2ZmZlZqTlbMzMys1JysmJmZWak5WTEzM7NSc7JiZmZmpeZkxczMzErNyYqZmZmVmpMV\nMzMzKzUnK2ZmZlZqTlbMzMys1JysmJmZWak5WTEzM7NSm7/eFbBZLbnkktG/f/96V8OsVSNGjHgn\nIpaqdz2sZY4p1l20FFOcrJRM//79aWpqqnc1zFol6cV618Fa55hi3UVLMcXdQGZmZlZqTlbMzMys\n1JysmJmZWak5WTEzM7NSc7JiZmZmpeZvA5XMmNcms8mJd9W7GvOsh0/eut5VMOtUjinl5FjTPm5Z\nMTMzs1JzsmJmZmal5mTFzMzMSs3JipmZmZWakxUzMzMrNScrZmZmVmpOVszMzKzUnKx0IUlDJZ1f\n/djMrCMcU2xe5WTFzMzMSq3hkxVJ35I0QtIYSU9LOlDSFZLOqirXJOkgSfNJ+oWksZJekHSvpFVy\nmUGSxks6K6+vh6R1JP1b0ihJE/I81WdvzayrOaaYzX0NnaxIWg64ATgmIgYAOwBnA2OBXSsBQNKq\nwNrANcAQYDCwaUR8Afg38LfCalcAnoiIARHxCXAxcEdErAV8GTgA2K6d9Tw4B7amj6ZN6vgOm1mX\nckwxq4+GTlaArYHRETEMICLGA3cACwALA5vkcnsCt0TEJGA34I8R8W6edzbwVUnL5+c9gasL29gM\n+G1e/5vAM8Bq7alkRFwUEQMjYuACvfq2bw/NbG5yTDGrg0ZPVpYF3q6aNhFYghQcdsvT9gAuyY/7\nAcflJtkxwKPAO3ldAFMj4sPC+nYEhkkal8uvT+MfV7N5lWOKWR00+r8uvw4sXTVtKeBJ4J/AVZIu\nzmUqf0v6CnBNRPypemWSBlU9X5nUzPvtiLgrTxvemTtgZqXimGJWB42erd8FrCFpcwBJawDbADdH\nxEPAR8DxwJUR8XFe5lrgQEl98jIbS/p7MwPcegMBNOWyuwD9gUW7bpfMrI4cU8zqoKFbViLiDUk7\nA2dLWoQUSA6KiCdykSuAXwAbFRb7M6l59hFJAUwBfhYRUR1bIuIJSX8BnpQ0EbgMOBE4TdK4rtw3\nM5v7HFPM6kMRUe86WMGiy68e6xx0Xr2rMc96+OSt612FbkPSiIgYWO96WMscU8rJsWZ2LcWURu8G\nMjMzs27OyYqZmZmVmpMVMzMzKzUnK2ZmZlZqTlbMzMys1Br6q8vd0YDl+3iUuJl1GscUawRuWTEz\nM7NSc7JiZmZmpeZkxczMzErNyYqZmZmVmpMVMzMzKzV/G6hsXhsBJ9X6M1azueAk/1dYw3FMmcnv\n727LLStmZmZWak5WzMzMrNScrJiZmVmpOVkxMzOzUnOyYmZmZqXmZMXMzMxKzcmKmZmZlZqTlXaS\ntK+kp/PjnSTdUph3ZP1qZmbdleOKWcucrMyBiLgxIrYHkLQ0cGqdq2Rm3Zzjitnsuk2yIqmHpD9I\nGidptKRHJA2UtLCkcyQ9K+lFSTdIWiovc7mkCwvrGCppaH48QdKuhXnnF+b1kXRF3s4Lkv4hacka\nddpX0tOSegFNwCKSxkg6StLIqrKnS7qqK46NmXWM44pZ99BtkhVga+CbwFoRsSbwa2BX4ExgXWB9\n4AvAu8Dv8jJHAN+RtKmkrwODgB+2YVv/D1gMWBtYPT8+rrnCETEN2AeYFhEDgL8Da0jasFBsD+CS\nWstLOlhSk6Smt6e1oXZm1lkaMq44plij6U7/DfQmsBzwf5JujYhbgFskvQEcmE9sJJ0NjJC0b0T8\nT9JBwJ+ABYD9ImJyG7b1M2DBiPgU+FTSf4B12lrRiHhP0k3A/sBjkjYBFgbuaqb8RcBFAAOXl/+8\nwmzuaci44phijabbtKxExAhgd2AX4HlJwyVtBvQDzs/NpGOAG4ApwBJ5uTtJ+zk9Iu5r4+bWBa7L\nTcBjgINp/7EaCnxPUk/S1c/lEfFJO9dhZl3IccWse+g2yQpARNwREd8mBYw7gSuBV4BDImJAvq0R\nEUtGxNsAkg4hXT1NzVdDFZ8APQrP+xUe3wyMJDUNDwD+2oHq/hOYCnwH2I1muoDMrL4cV8zKr9sk\nK5L2l3SupPki4gPg4TzrWuAISQvmcttLOiM/Xhk4BTgIOBA4TdIX8nIvAxvmcv2BrQqb6ws8FhEf\nS1qTFBgWbaWKM4AFJS0CkJt6LyX1c0+MiCc6vPNm1iUcV8y6h26TrAA3AssC4yQ9A5wMfJ8UNCYA\nT0oaDRwDXCFpPlKT6RkR8XxEjAZ+DwzN804EdsnNsWcAlxe29WPgPEmjgJ8ChwLrSTqvhfo9AYwC\nXpW0TZ42FOiPr37MyspxxawbUITHXnUVSYuSmpPXiIg327LMwOUVTQd3bb3MmnVS2+OBpBERMbAL\na2M1tDeuOKYUtOP9bXNfSzGlO7WsdEcnA9e3NVExM2sDxxWb5zhZ6QKS1pb0ArAWcGy962Nm3Z/j\nis3LutPvrHQbEfEM6YekzMw6heOKzcvcsmJmZmal5mTFzMzMSs3dQGWz/EZwUlO9a2FmjcIxxRqA\nW1bMzMys1JysmJmZWak5WTEzM7NSc7JiZmZmpeZkxczMzErN3wYqmefeG8f2N32n3tXoVm7Z8bZ6\nV8GstBxTrLPUM9a6ZcXMzMxKzcmKmZmZlZqTFTMzMys1JytmZmZWak5WzMzMrNScrJiZmVmpOVkx\nMzOzUitFsiJpkKQpnbSuMZLWbmbeMEnHdsI2DpDUKz8+TdLx+fESkr43p+s3sznnuGLWOBruR+Ei\nYkBXrl9SD+Bs4GZgWkQcV5i9JfA94IqurIOZzV2OK2b11WrLiqQjJb0habSkn0gaJWnXPG9dSfdJ\nelbS88Wri3y18TNJt0saL+kJSWvmeX0l3SDpZUnDga9UbXPnXP45SU9J2rowLyT9OF/pzLJcYf7A\n/HjXvO2xki4s7m++Wrkk1/0lSRdLWjjPOynPuzDv98uSdsmLPgL0Bh6StK2koZLOlzQIuAAYlOt+\nlaRzqur2sKRDWzvmZo3OccVxxaw9WkxWJA0gZfvfjIg1gYWBlfO8XsDdwDURsTqwGTBE0paFVewN\n7B0RqwDPAj/J048Flsjr2hz4emGbGwOXAvtFxKrAkcC1kvoW1rtyRAyIiAdbqHtf4G/AYRGxBnAD\nswavoXn/1wJWBZYHilczOwF/zvt9BvDbPH3XfL9pRNxeKRwRw4DzgWERsV5e/16SFsz16Q+sB1xd\no64HS2qS1DRj8ozmdsmsITiudH1ccUyxRtNay8rXgccj4qn8/Exmdh19DZg/Ii4AiIjXSc2UexaW\nvzki3s2PRwAr5sdbAVdHxEcRMZ10AlbsAtwREY/l9Q4DxgLFP7e4qQ379mXgg4i4K6/nbuAFAEmL\nANsCp0fExxExgxQQinV/MiJG5MdNhbq31T+B6cD2+fmewG0R8b/qghFxUUQMjIiBC/ZZsJ2bMet2\nHFeSLosrjinWaFobs9IPeKfyJCI+kPROYV5vSWMK5RcCHi48L55AHwM98uMlgHcL896u2uZWVevt\nnZepmNhKvWtto7idPqRE7UZJn+RpPXL9m6t7uwYjR8Qnki4F9geuA/YATmjPOswalOPKzLo7rpi1\nQWvJymTSCQiApJ7AkvnpK8DEDg48+x9QbH5dtvD4FeDOiBjcgfW2tI3idt4EPgK2i4ixc7idlgwF\nnpa0GbACcEcXbsusu3BcmTNDcVyxeUxrWf0DwMaSVs/Pf0w6GQGGAx9JGgwgaQFJZ0vaqg3b/Tew\nu6T5c6DapzDvBmAbSWvk9S4l6WpJy7VxnyqGA4tJ+npezzbkJteI+DRvZ4gk5fkHSxrShvVWOoAX\nb2beYpV15oD1KHAZcGVEfNzOfTBrRI4rs3NcMWtBi8lKRIwEfgXcK+lxUvPnm2lWTAe+CxwiaSzw\nDNCTFIhacwYwjdTX+yBQHFD2DHAQcLWk0cAw4D+577rNcp/2gcDFuX67kvp7Kw4HFgXG5Pk7kAJN\na14H7gFGSDqwat6dwGrA65L65WlDgf7AJe2pv1mjclypyXHFrAWKiJYLSIpCIUmTgB0j4r6urlwj\nyFeEZ+WR/K1abNW+8bXffrWLa9VYbtnxtnpXYZ4kaUREDOzgso4rc6A9ccUxxTpLV8falmJKa19d\n7gv8T9Km+fkOgICRnV7LBpR/X+FkZn490Wye57gyZxxXbF7U4gDbiJgk6SDgb7m/9ENgj1pfv7VZ\nSdoXOAW4PiIurXN1zErDcaXjHFdsXtXqz+1HxLXAtXOhLg0lIoYy6+88mFnmuNIxjis2ryrFHxma\nmZmZNcfJipmZmZVaw/3rcne36mKr+dstZtZpHFOsEbhlxczMzErNyYqZmZmVmpMVMzMzKzUnK2Zm\nZlZqTlbMzMys1PxtoJKJ19/nw1OG1bsa86SFThhU7yqYdTrHFCuTjsZZt6yYmZlZqTlZMTMzs1Jz\nsmJmZmal5mTFzMzMSs3JipmZmZWakxUzMzMrNScrZmZmVmrzdLIi6SRJt+bHR0i6oA3L7CTplq6v\nnZl1N44pZl3DPwqXRcT5bSx3I3BjF1fHzLo5xxSzztNwLSuS+kv6RNIxksZKWknSJZKelfSSpIsl\nLVxjueIVUR9JN0l6Q9Ijkg6VNCXP21fS04XljpE0StIYSQ9L+kqePkjS85IOl/RYXtdpc+s4mFnn\ncEwxq7+GS1ay+YAFImIN4Pz8fC1gVWB54LhWlj8OWAJYEfgWsF+tQpJ2AIYA34iIAcA5wI2SeuUi\nKwDzRcSGwDeAn0laqcZ6DpbUJKnp7amT2renZjY3OKaY1VGjJiuQTvBFgG2B0yPi44iYQQo0e7ay\n7JbAVRExIyImARc3U24n4MqIeA0gIq7K07+U7+cHLsrzngamk4LVLCLioogYGBEDl1qkb9v30Mzm\nJscUszpp5DErE4E+pITsRkmf5Ok9gIVaWbYf8E7h+UvNlFsWGFU17V1gGeBN4IOImF6Y93Hevpl1\nP44pZnXSyMkKpJP7I2C7iBjbjuUmk4JSxQrNlHsdWLryRJKAJfP0Rm61MptXOaaY1UFDv/kj4lPg\nBmBIPukrfblDWln0AWBPST0k9QH2b6bcDcBgScvk53sDHwKPznntzaxsHFPM6qPRW1YADgfOBcbk\n2PIccFgry/wauBx4Pt8uB9atLhQR/5C0KvAvST2At4EdIuKDvC0zazyOKWZzmSKi3nUoJUmKfHAk\n7QycHRGzjbrvbButsEb895A/dfVmrIaFThhU7yp0K5JGRMTAeteju3BMMWs5zrYUUxq6G6ijJO0H\nNElaOF/dDAbur3O1zKybckwxmzNOVmq7HGgijcofQ+ouO7auNTKz7swxxWwOzAtjVtot/3bCIfWu\nh5k1BscUsznjlhUzMzMrNScrZmZmVmruBioZLdfb30oxs07jmGKNwC0rZmZmVmpOVszMzKzUnKyY\nmZlZqTlZMTMzs1JzsmJmZmal5m8Dlcy7zz/DlXusWe9qNITBV4+udxXM6s4xxSq6c0x0y4qZmZmV\nmpMVMzMzKzUnK2ZmZlZqTlbMzMys1JysmJmZWak5WTEzM7NSc7JiZmZmpeZkxczMzErNyYqZmZmV\nmpOVKpL6S/pE0jGSxkpaUdJtkkZLmiDpUkkL5bI9JJ0l6XlJz0r6q6Seed66ku7L05+XdGx998zM\n6sVxxWzOOFmpbT5ggYhYA/gN8HJErAmsDWwKHJTLHQUMBAYAawGfA46T1Au4G7gmIlYHNgOGSNqy\n1sYkHSypSVLT+9M/7sr9MrP6mWtxxTHFGo3/G6h5N+b7weTjFBFTJQ0HVsvzdgEujYgZAJJ2AD4G\ntgLmj4gL8nKvS7oC2BP4V/WGIuIi4CKAlRdfOLpsj8ys3uZKXHFMsUbjZKV5E/P9FsDxklYEPgGW\nAy7N85YE3qssEBEfAEjqB/SWNKawvoWAh7u60mZWao4rZh3gZKUFudn1VuBo4OKI+FTSNYUibwJL\nFcr3BXoBrwATI2LA3KyvmZWf44pZ+3nMSssWIAWJphxQNgM2ARbN868H9pW0sKQewCXAYcBw4CNJ\ngwEkLSDpbElbzf1dMLOScVwxaye3rLQgIiZJOhG4TdK7wF3AkcAlkkYBZwErAM8CHwAPAqdGxHRJ\n3wV+L+kkQMA9wAN12A0zK+Md3MgAAA4HSURBVBHHFbP2U4THXpXJyosvHL/+Zv96V6MhDL56dL2r\n0NAkjYiIgfWuh7XMMcUqyh4TW4op7gYyMzOzUnOyYmZmZqXmZMXMzMxKzcmKmZmZlZqTFTMzMys1\nf3W5ZBZfeW0GX91U72qYWYNwTLFG4JYVMzMzKzUnK2ZmZlZqTlbMzMys1JysmJmZWak5WTEzM7NS\nc7JiZmZmpeY/MiwZSe8DY+tdj06yJPBOvSvRiRppfzpjX1aKiKU6ozLWdRospnSGRjqPO0tZjkmz\nMcW/s1I+Yxvln2wlNTXKvkBj7U8j7Yu1qmFiSmfwe3923eGYuBvIzMzMSs3JipmZmZWak5Xyuaje\nFehEjbQv0Fj700j7Yi3zaz0rH4/Zlf6YeICtmZmZlZpbVszMzKzUnKyYmZlZqTlZKQlJG0t6WNJz\nkkZL2qfedeoISf0lhaQxVbcl61239pB0sKSpko4tTFtS0s2SxksaJ+l3krrFOdTM/kyQ9FLV67Rt\nPetpnadRYsqcarRzeU5I2krSI/lcHydpSJ5e+uPh31kpAUk9gRuBYyPiKkmrAk2SHo+Ip+pcvQ6J\niAH1rkNHSfoDsBQwpmrWhcBrwI5AL+DfwKHAH+dqBduphf0B2Ccihs3dGllXa8SY0hGNdi7PCUnL\nAjcDO0TEvZJWAUZKehg4hpIfj1JlTvOwrQAi4qp8/xxwGzC4npWah10ZEbsD71cmSOpNOpHPimQq\n8Cdg7zrVsT1m2x9reI4pSaOdy3PiE+D7EXEvQESMB0YBX6IbHA8nK+UwABhXNe1ZYO061KVTSLpU\n0tOSmiR9v971aY+IeKDG5NXy/fjCtG7xGjWzPxVD8ms0WtKpkhacaxWzrtRwMaUjGu1cnhMR8XZE\n3Fh5nltW1gEez5NKfTzcDVQOiwAfVE37IE/vbqYAfwXOi4iRkr4K3C3pxYi4v851mxOLADMi4tPC\ntO76GlVcBzwMXA98DrgT+BA4pZ6Vsk7RSDGlszXiudwukj4H/AM4Ewi6wfFwy0o5TAEWrpq2SJ7e\nrUTEOxFxQESMzM8fAG4Btq9vzebYFKBn1aCzbvkaVUTEsRFxXW76fRk4j+7/OlnSMDGlCzTcudwe\nkjYEHgL+HhEn002Oh5OVcngGWL1q2prAk3WoyxyRtHgezFc0H/BRPerTiZ4l9fkW961bvkYAkhaS\ntG7V5EZ4nSxpmJjSBRrqXG6PnKjcDhwdEWfkyd3ieDhZKYf7gI8l7QcgaT3gW8Blda1Vx2wK/FfS\nSgCS1gG+DdxU11rNoTzo7Drg50oWAw4D/lbfmnXYosBDkr4NIKkfcBBwQ11rZZ2lkWJKp2rAc7lN\nJC0EXAscHhHXV6Z3l+Phn9svCUnrk74mthRp3MBJxTdUdyLpKOAH+emHwOmVbyWUnaQepKtSgBVJ\nTaHvkr4GeibwF2B90pXIVcCJUeKTqJX9uRc4HegNfEoKWCdHxMd1qKp1skaKKR3RaOfynJI0mJSs\nVg+8vgo4l5IfDycrZmZmVmruBjIzM7NSc7JiZmZmpeZkxczMzErNyYqZmZmVmpMVMzMzKzUnK2Zm\nZlZqTlbmEZImSHpV0piq20FtXHbXTqzLIEmfFuowTtIzbalLK+v9kqQRhecHSOqVH58m6fg5rXvV\n9iIfm8p+jM1/CrhTG5efX9IPWi9Zc9nz83G7N/+IU2X64pKekrRER9Zr1poSxpKQdHSNeV36uxzV\n528+F7/VieuvjpNjJD0r6U5JX2zjOlaRtG0Htr2YpHtyjPlj1bzNJf2zveucU/4jw3nLURFxXb0r\nkU2LiAGVJ5IGAA9KeiEi7unICiNiOLBRXl8P4Gzg5ryt4zqhzrXsGhFNlSeSNgPukbReRDzbyrIb\nAIcDF7Rng5I2ANYi/Zz6CcB+pH0FOAc4NSImtmedZu1UpljyEvALSTdExEtzcbuznL8RsVUXbKM6\nTgr4MXArsFIblt85l7u9ndvdD/hPRHxD0r8kbRARj+dfwf090GkJZ1u5ZcUAkLSzpJGFlo59mil3\nsKTRkkbl2z6FeYfmeeMlPSJpo7ZuPyLGALeRfpofSatKuiu3VoyWdLaknnnejrn1YHSe/+M8fZCk\nyp9vPUL6ZdaHJG0raWhujfiWpLdyMlOp9xGS/psfbyFpeL6CGSvp++05jhHxX+AtYN28vs9Jui3X\ndYKkS5X+l2c10h88rpaP+ZKSVpJ0c972i5LO1Kx/LlaxJvBI/nXJ4cCAvK1tgMUj4sr21NmsM9Uh\nlrxA+qXeP7RQp2bP69xSMF7S85Iuk3SxpPPzvPacvxMk7SrpVEk3VG3/Okm/7sC+zSKf89cCK0pa\nMq/va3nfRkl6QdJP8vS9gOOBvSTd3tpxqLIm6R/ZAR4lxxjgJODyiBjf1jp3mojwbR64ARNIrQC1\n5vUFpgOb5OfbkP7QbvHisqT/k/kE6J+nr0Q6YRcAdgHeBFbK8/YhBZH5amxvEDClxvSrgVMBAY8B\nJ+TpiwBNwDH5+VvAoPy4H+l/h5YsrhfoT/rr8yXz86HA+UAP4A1gq8J2HwAOAT4HTAW2zdMHAO8D\nqzZz3AIYWHguYA9gGvC5wj5dWNiP54Aj8/N9gacLyz5J+vl7AX2Ax4H9a2x3D+DM/Hhb0pXdosBT\npJ/LvpL0U/p71vt951vj3UoYS4YBPYGxwG6FeZHvmz2vczx4BTgsz/s68AFwfn7epvO3at/WzjFg\n0Ty9d36+Rgf2bUrVtAVJradP5ecCXq+c66QW10+AL+bnQwv70ub4lmNKpdyZOeZsQEpgNiZdWN4F\nrDe33nduWZm3nKvZ+5m3iohJQN+IqGTS/yJ1EfavWn468D/gB5IGRMSLEbF9RHwE7AZcGhEvAkTE\nJaSAtElbKiZpU9IH77WkwLU+cFZe11Tgr8B3c/HXgf9T+u+TSRGxY0S805btRETlfy92z9v9PLAh\nKShtB4yOiNtz2THAnZWyzbiucixJgeAQ4BsR8UqePxj4YWE/hgOr1VjPGsAXgV9HMpn0Xx171ijb\nBGwuaX7Sn9MNJyU5F5KC5V3ATsDvJC3a6kExa79SxZKImA4cmuvVt2p2S+f16sAKwN/zvPtIFwkV\nbT1/i3V5hvRPxpV4tQMpuRjbgX3rVTi+z5FiTB9SEljJxlYCrsnPR5GSoVp1bE98Gw5slWPMFsAT\nwEWkY/w74AhSzDmnpWPRmTxmZd5Ss585dzUcK2k3YGHSn9pBVTdhRHwkaXPgOOA/kqYBv4yIv5Ba\nOLaUtF1hkU+ApZupS6/8AV/ZzuvA7pH6RTcBPoyIKYXyE4Fl8uPvAD8n9dsuIOn3EfHrthyA7DLg\ndkmHkU7U2yLiPaV/Hh5QqBekq6kJLazrszErkq4EpkfEg4X5WwDHS1qRdDyWAy6tsZ5+pJaaRyVV\npi1IagWaRUSMl3QLqSXlMeBlYH/gSFLf9KURMVnSi6QrreEt1N+sI8oUSyrrvE/SnaSWgEMKs1o6\nr/uRYs3UwrziuJe2nr/VLiMlJleSWiUuKdSlPfv22ZgVpfEio4H7I+LVQpn9gQNykvYp6Y8razVE\ntCe+XUH61+WnSK0zuwD3RMRISZ+PiBckvQp8uZl6dzonKwapKfIwYLP8JuxFyuBnExGjgX2UPlG/\nA9woaRipKfWxaPtA1lkGjlV5HVhIUu+IeD9PWypPJ7daHAYcpjSg9VZJDzEzMLYoIpokvUsKRHsA\nv8qzXiFdAW3axn2o9lNgtKQ/R8R/83G8FTgauDgiPpV0TTPLVlpi1o+ID9uwD6cBp+UA9gipGThU\nGItDauL2Pyjb3FSPWFJ0LDBK0iWFac2e15LWAXpK6plbZyC1tLzTzvO32pXASZJWIHUt7VuoS4f2\nLSI+VBqf9wdJN+ULrEGkwfWbRsTIvE9vNbOKNse3fCy+l9c3gNRy86U8u5IIzdX44m4gg9TP/Abw\nYv6w+ykwg9Q8+RlJ60m6OycRQfqQnE5qEbgWGCxpuVx2ZUnXSlq4A/V5idRicFReVx/gAOAGSUtL\nekjSsrns48B7NdYxI98v3sw2LgcOJjVP35Gn3Um68vhq3u6ikv6qNn5NMNI3EX5LCiY9SP3vvYCm\nHOg2IzX3Vo7rDKC3pPkj4mVSC8iQvO35JB2v9LfuLTkRuD4HfkjjXjZR+jrzF5j97+DNulJdY0lE\nvAscQ+qyqGjpvH4WeIeZH8yDSGMzoB3nb416vJr36bekFonKt/PmKE7m1qwxzLzA6gtMJrW4IOlI\nUotssY792nAcasotZX8mjdOpXES9KekLwFdI8WbumFuDY3yr742WB8UtQRqkNoE08nsb0of566RR\n4RNIYyEE/Jo0yGxUvv2gsJ7DgWdIJ84TpG6dWtsbRI0BtlVlViGdXKPz7VRg/jzvh6TBdKPz/cnV\n6811/SdpANmBFAaa5fkrk1pifl+13S1IScOYfDuFGoPfctlZBtjmab1I3TI/zM9PAF4Dnib19e4A\nTAJ+Qhrw9gIpWK5D6nv+R96nZ0lXZ/1aOEYbkJK1BQrTViAFyRcqdfDNt868lTCWDKsx/U7ykI78\nvNnzmtSq81ze1gWkLpzz8rz2nL+zHBdS90wAO1fVbY7iJGk834x8/i9I+oLByzkW7EOKlRNJLTrb\n5vqObe04NFOHI4GLqqZtmZcdB2w+t953yhs3MzObJ0lS5A/DPB7s8Yg4sc7VsgJ3A5mZ2TxL0n2k\n7iryQNotgPvrWimbjVtWzMxsniVpXeBPpC4sSINpz6hjlawGJytmZmZWau4GMjMzs1JzsmJmZmal\n5mTFzMzMSs3JipmZmZWakxUzMzMrtf8PCBbuXlg+WZkAAAAASUVORK5CYII=\n",
            "text/plain": [
              "\u003cFigure size 576x216 with 2 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "scores_test = model_robust.predict(text_test)\n",
        "plot_metrics(labels_test, scores_test, groups_test, group_names, \"Robust Opt Model\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mVZ23Q2bWs-N"
      },
      "source": [
        "Compared to the two previous models, we can see a significant reduction in the false negative rates. This however comes at the cost of a lower accuracy and higher false positive rates. Indeed the trained model performs the best on the training objective that we sought to minimize.\n",
        "\n",
        "Note that if low false positive rates are more important than low false negative rates, one can modify the training objective to have a higher weight on the maximum FPR term than the maximum FNR term."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lfpQE4MSWs-O"
      },
      "source": [
        "## Performance on finer groups"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ECi_IZiFWs-O"
      },
      "source": [
        "We close this tutorial by noting that while constrained and robust optimization are effective means to enforce different fairness goals, with both these approaches, you at best get what you ask for!\n",
        "\n",
        "For example, with the robust optimization problem that we solved above, we observed that none of the four topic groups incurred a false positive rate or a false negative rate of more than 35%. However, the same may not hold for the finer sub-groups within these groups. Let's take a closer look at sub-groups within the *sexuality* topic group."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        },
        "id": "32x3i5WOWs-P",
        "outputId": "e59cfb5c-2b15-4d09-cb9e-5024077909eb",
        "scrolled": true
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\u003cdiv\u003e\n",
              "\u003cstyle scoped\u003e\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "\u003c/style\u003e\n",
              "\u003ctable border=\"1\" class=\"dataframe\"\u003e\n",
              "  \u003cthead\u003e\n",
              "    \u003ctr style=\"text-align: right;\"\u003e\n",
              "      \u003cth\u003e\u003c/th\u003e\n",
              "      \u003cth\u003eSub-group\u003c/th\u003e\n",
              "      \u003cth\u003eSub-group proportion\u003c/th\u003e\n",
              "      \u003cth\u003eLabel proportion\u003c/th\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/thead\u003e\n",
              "  \u003ctbody\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e0\u003c/th\u003e\n",
              "      \u003ctd\u003egay\u003c/td\u003e\n",
              "      \u003ctd\u003e0.58%\u003c/td\u003e\n",
              "      \u003ctd\u003e55.9%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e1\u003c/th\u003e\n",
              "      \u003ctd\u003elesbian\u003c/td\u003e\n",
              "      \u003ctd\u003e0.06%\u003c/td\u003e\n",
              "      \u003ctd\u003e35.0%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e2\u003c/th\u003e\n",
              "      \u003ctd\u003ebisexual\u003c/td\u003e\n",
              "      \u003ctd\u003e0.03%\u003c/td\u003e\n",
              "      \u003ctd\u003e0.0%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e3\u003c/th\u003e\n",
              "      \u003ctd\u003ehomosexual\u003c/td\u003e\n",
              "      \u003ctd\u003e0.24%\u003c/td\u003e\n",
              "      \u003ctd\u003e39.5%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e4\u003c/th\u003e\n",
              "      \u003ctd\u003estraight\u003c/td\u003e\n",
              "      \u003ctd\u003e0.46%\u003c/td\u003e\n",
              "      \u003ctd\u003e7.6%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "    \u003ctr\u003e\n",
              "      \u003cth\u003e5\u003c/th\u003e\n",
              "      \u003ctd\u003eheterosexual\u003c/td\u003e\n",
              "      \u003ctd\u003e0.03%\u003c/td\u003e\n",
              "      \u003ctd\u003e11.1%\u003c/td\u003e\n",
              "    \u003c/tr\u003e\n",
              "  \u003c/tbody\u003e\n",
              "\u003c/table\u003e\n",
              "\u003c/div\u003e"
            ],
            "text/plain": [
              "      Sub-group Sub-group proportion Label proportion\n",
              "0           gay                0.58%            55.9%\n",
              "1       lesbian                0.06%            35.0%\n",
              "2      bisexual                0.03%             0.0%\n",
              "3    homosexual                0.24%            39.5%\n",
              "4      straight                0.46%             7.6%\n",
              "5  heterosexual                0.03%            11.1%"
            ]
          },
          "execution_count": 47,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# We treat each identity term related to \"sexuality\" as a separate sub-group.\n",
        "terms_sexuality = terms[\"sexuality\"]\n",
        "\n",
        "# Create a sub-group membership matrix.\n",
        "comments_test = data_test[\"comment\"]\n",
        "subgroups_test = np.zeros((comments_test.shape[0], len(terms_sexuality)))\n",
        "for ii in range(len(terms_sexuality)):\n",
        "    subgroups_test[:, ii] = comments_test.str.contains(terms_sexuality[ii], case=False)\n",
        "\n",
        "# Display sub-group statistics.\n",
        "group_stats = []\n",
        "for ii in range(len(terms_sexuality)):\n",
        "    group_stats.append([terms_sexuality[ii],\n",
        "                        \"%.2f%%\" % (subgroups_test[:, ii].mean() * 100),\n",
        "                        \"%.1f%%\" % (labels_test[subgroups_test[:, ii] == 1].mean() * 100)])\n",
        "group_stats_df = pd.DataFrame(\n",
        "    group_stats, columns=[\"Sub-group\", \"Sub-group proportion\", \"Label proportion\"])\n",
        "group_stats_df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I4LJAIrGWs-S"
      },
      "source": [
        "The performance of the robust optimization model within each of these subgroups is shown below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 254
        },
        "id": "uoUYq4b5tNK4",
        "outputId": "5ddcdaac-0ae6-44b9-9fd0-2c1c1a0e6d7e"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAADtCAYAAAClO5+KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd7wcVf3/8dc7gQAhEBJCVxMgkNCk\nBaVKAEGK0pEmXboIKKDwRYkNkB8KKAgiaAQ0NGnSFQhN2g0dEkILNXQILZBAPr8/zlky2extuWXv\n3Pt+Ph77uHfmTDkzs/PZz5w5s6uIwMzMzKyMetW7AmZmZmZzyomMmZmZlZYTGTMzMystJzJmZmZW\nWk5kzMzMrLScyJiZmVlpOZHppiSNlRRVr48lPSrpIEktPvaShuT5t+3IOreUpL6SjpX0SN6mDyTd\nm7erdwesb++8/dc3Ut5L0kt5miFtXNfDkka3Yvqxkq5qyzrnRI33VvE1up51K9Txy5LOkvScpE8l\nfZjfJwdKUr3q1ZEkbZKPwR31rksZSfq6pLskfSLpNUknVccUSftImpjfUxMl7dnCZR8j6TNJo2qU\nzSXpVEmT87rvlvS1Qnk/SddIej+XLVk1/7z5ff6dOdz0UnMi073dCCxReK0KnAf8ATiyjvWqSdK2\nksY2M80CwB3AgcCpwCrAmsBFwK+BK1ubzOTkYWQzk30MbCpp0RplGwIDW7PObuLnzPr+qrwOz+Xb\nA3vXo2KSVgMeBpYjvVeWBzYAbgHOAv5Wj3p1gj2Bx4H1JQ2ud2XKJF+E/Bd4DfgasBdwEDCqMM13\nSDH0dOCrwO+A8yVt2cRy55d0CXAo8Fkjk/0W+B7wfWAN4CHgZklL5PIjgEVzvSYDv6ya/8fAxIj4\nd4s2truJCL+64QsYC1zVSNlZwNutWNYQIIBtO7jOJwNjm5nmLOANYIkaZasCnwJHtmKd85OCy8gm\nptkbmAQ8CRxeo/x8UtIYwJA27oOHgdHtcZw7+FgFcERnr7eFdesFPAHcBPSuUX4g6cNq+XrXtZ23\nuy/wAbAj8AJwXL3rVKYX8CvgXWD+wriD8j6dNw//D7i4ar6LgDubWO62wF2kROQ9YFRV+YLAJ8BB\nhXG98jH8VR6+Hdg3//9N4IXCtEvleq9Q731Yr5dbZHqmx4GBkuatjJD0I0nPSJom6VVJp0map2q+\nhSRdnpvo35T0i8L8oyS9V5w4t7B8cbtF0tDcPPpWviX0oKRtctlo4CfAhnmekdWVltQP2Af4fURM\nri6PiEeAC4EfFOYJSYdK+nu+BTVF0jm5KXcI8CHQG7itudYg4Dpgj6o6zQPskMuq6/s9SU/kJug3\nJf1N0oBC+bKS7shNyc9K2rXGMlaVdJOkt3PdL5G0WDP1LM4/l6QTlW59TZP0vKTjK7dWJI3M+2gN\nSTfmY/uCpANbuo4m1v3FraWWrkfSTpIeyOWv5qb9uQvlIekHku6X9Eojq94IWBH4WUR8XqP8XGDJ\niJhYVbftc53G5PH9JJ0t6fW87yYU66uZtxwXKoxbrfj+lTRa0p2S9pD0Yn4v3CtppcI8m0q6L78/\n38vHe8Va+7EZ25MSzOuAfwK7V08gaVFJ/5D0bj4P/1l8P0laU9Lt+fx8UdJvJM1V2JaHq5Z3hKQo\nDE+SdEI+xlOVWiN65XEvSJqe34u/UyG+5H19lqQ38vv8OqV4sVBezmFV611M0ueSdmzBfmmpFYGH\nI+KjwrhrgX7AWpL6AmuTEuSiW4B1Jc3XyHIfADaOiDcaKV8PmKe43IiYQbpI2SSPWgJ4Mf//ArB4\nYf7fki5+xje+ad1cvTMpvzrmRdMtMmcArxWGf0i6ItgTWBbYDngLOCeXDyEFyOeAA0jN9T/N43bM\n04wC3qtaz7YUWilIVzP/Id0OWho4jtQaMgToD9yQp1kc6FOj3hvm5a3XxHbvlKf5Uh4O4GXSrbSh\npERoOilp6k0KFEH6EBjYyDL3JrXIrJmnXaFQtgPpKmtY1bZuDcworPebwPPAjYV5G0hJ5Vqk1qTr\ngLfJLTLAYnn4hly+NvAgcB+g5o5zLv898A6wTT62+wJTgZ/m8pG53ncVpjkv76OvNLHcZltkinVr\nyXqATfM++y3pPVZ5H55Std6JwG6VY1xjvSeQWubUwnOlUre7SU33i+bxV+Rjtkk+hkfl+u1SeF8E\nsFBhWavlcSPz8GhSC+K1+RiuBTxGat0TMAD4iHSbdBlgJeAS4OnCMR5YXEcT23Ez8Nf8/4q5HmsU\nygXck7dzzVyf+4Dbc/kSpCv7s4Hh+Xi8BfyisC0PV63zCCAKw5NIceJHwGBSy8Ih+XhsA3w5L/cd\n4P8K810MTADWz+u+Nu+DuUlJ2T1V6z0oL2OeGvthA9IFSmOv3RvZf1cD/6kaN1/ej3uR4tYXx7bG\n+2elFhyjWi0yhwGfU9V6SIqpb+b/nyYlQ5DOnWn5/3WB11vy/ujOr7pXwK8OOrA1PuByUNgmB85f\nFcY/S05aCuOOIn3gzcfMRObcqmkeBS7L/4+i+UTmfeDoqmnWBRbI/19FE7eWgF3y8r7cxDRfz9N8\nPQ9HjeB0FdCQ/5/lg6eRZe4NTMr/TwROLJRdSQrwlX1U2dZbKCQtedyOlWlIwTqArQrli+SANjoP\n/18OvAsWplk9z7d+Y8e5MO28+Vj/tGr8mYXtGZmX98NC+fJ53NZN7JM5TWQaXQ8pyb2zahlH5m2Y\np7DeK5tZ7znAs604Vyp1O7ww7it53C5V015beY/S8kRmBoWki5nnxcqk/hABrFUoX5CUUPVqxTYs\nmd87GxXGPUhqvawMfy2va82qcX8nXUgcQ0oO5i6U70N+v9PyROahqmkGUHUbj5S43Jb/XzzXfYdC\n+dLAP0jJ3aa53ssWym+mKmYVyuYjJZ6NvRZoZL5TSUlBcfvXyuv+ASlWzbL/8jQj8vh1WnCcaiUy\nxwEf1Jj2KODT/P+dwG75/2/k/SxSa88BpKTxVuAVUnwb0NL3Tnd4+dZS9/bt3ET/oaQPSR1WR5M6\nqp0AIKk/KVjcWzVvA+mDcPnCuP9VTfMAqSWipW4ATpB0sqQNJM0VEf+LiA9aOH/kv9ObmKZv/juj\nMK663g+TPqjmxCXA7koGAFvmcdXWoPY+hXQlvEL+/5FKYUS8SWo2rlgTeDAi3i9M8xAwhfSB2Zxh\npP1Rqx6D87GvrhvAm/nvAJr22+L7q/A6pIl5mlrPmsBtVdOPzduwXGHcg83UawY13iP5tk2xnudU\nTVJc7ur5b619t2oz66/2UkS8XBiu3J75Cqll5kXgUqXbuytHxPsRcX+k2wsttTupE+hdSrcT5yK1\nZOyimZ3fKy2KX9weyuvZKyKm5PLHImJ6ofxvEXFcK7e3+vh8nOvxeOUYkFoyK8d9dVLLzRfzRcTz\nEbF7RDxHuih4kdQKRz7vNgIuqLXyiJgaEc808Wos3pwHDAJOVnoycjCpRfNjmo45neF2UtzpQ2o5\nv5WUZM5FqvfvgHGki6RppIugHsOJTPd2G+kDr/K6ixQ4jy8EyQXy3/er5v2gqrzWNB+ROsu21D7A\niaTbLncAr0n6YSvmfyn/XbqJab6c/xY/OKZUTfMhsBBzZgzpA2hDUgvLh6QnHaotQNP7tF/+/+Ma\ndatYkHTvfZZEIY9fnOa15tgW61FJGJt7RPn/Mev7q/L6RxPzNLWeBYGfVG3rXbmsuL3NJb4vAV/W\n7E+vrVmo4xOkRL2ouNym9t0CtE6t9x+klpxPSLdTrgeOBh7LfXE2oXX2IHX6nEb60J1OOj5LMLOf\nxQDg46jdb6hS3tKLiqZUL+MMUqvDWaTbPqsB11Stt9Z8wBf9Rf7OzD4/W5NaFKsvUNokIiYAuwL7\n5brcD/yRdFvsLWYexwWrZq1cEFQf55aaAvSt8X7tX1jmaaS4M5XUQnUGKZb+MO+fkcA/cxI6Jg/3\nGE5kurePilcipGbgtUkJRUUlUPevmrfWyVmdtPRjZlAOZv/g61cciIiPI+LEiFiRdJ/3H8AZkjZv\n4fY8lNe3cxPTbEh6DLHYGbi63guQ+gK0WkQ8SerjsBPwXeCK4hVswRSa3qdT8/99G5mmMt19zJ4o\nLE9qVWtO5di15NjOibcaueJtS0A/i1m3dVVSa8w9rVjObaT9Osv3HkXEs4VzYWrNOWetC9Ted5Wy\nWglfP2ZX6/0H+T0YES9FxKGk20Nrk1ofrpHUokf6lR41X4XUyrFW1etO0mO9kFrA5leh83SVN2m6\nFa7Zc7wR3wX+FBFnR8Rjef8XO8a2pAVwNLC8pFVI/dkubGzC3Npbq6Ww8pqtE3RFRFxKerpoMOl4\njM31epzU9+dzUuwqWo7U1+/ZJurflKdJn8XVF2jLkfoNERFvR8QqpMR0CClxHRsRd+Zpi0noB8z5\nhVopOZHpQSI91XMeqel0QB73PulEWq9q8nVILS4TC+M2qJpmNaDSU/59oF9VkCx+odMApad4euf1\nPhcRh5M+FFaiBSJiKumJk++rxhfPSfoqKWifVlVUXe/Vgadass5GjAG2IN2rrnVbCVIzb619GqSE\nrLL+tSqFkpZhZosSpFt3y5JuTRQT0rlJnYCb8xQp8atVj6aa2OvlAWDpqm19ndRP4KNm5v1CRNxL\nuiU0Kj9pMgulp9+WnG3GWT1I7lheNX4dZt4CqVwEFD+Av8bshkgqHtfKbauJ+cmcb+d6R0TcR+qr\n0pemWx6L9iQdzysioqH4It1e2i7vh8otpXUrMyo9ZXWX0hesPQysVnz6RunL3/5V2N7qZKPW9lbr\nQ2rRqCxzCdKtoUpS9DgpESjWa7FcrzUgxQtSK+5+wGY0clspa6B2S2HldU2tmZSeItwnIqZFxMu5\n5eq7pAujp3L8uROovvDaHLg1Ij5tdk/Udhcp1n6x3HwLaWPS7fgvRHryaTnSfji6UDQFWDj/vxhz\neKFWWvXsoONXx71opBMoqUPpe8DZhXEHk5pP9yf1l/ku6UQ4JZcPIQX1Z0mtOUOBY5m1o+aGefiE\nvIx9SP0/Kp1bB5I+VP9EeqJiMDN761c65o4GXiXdAli0ke2an9Tn5ZW8jmVJJ/bBpGB5MYWnVfL6\nXyF1nFuO9NTOdPJ3NhS27dfAyo2sc29y59g8vHSe53XykwbM3tl3C1JfjWNzHTcn3fK4NJeLlAQ+\nmrd3dVIHxjeY/amlMaSWiWGk2wUfF9ZT8zgX6vrbfCx3zMflwHysD8nlI3O9VyvMs1Aet3cTy53T\nzr6Nrof0AfVZ3mfL5/1yA+k2UK+WrjdPNzTv73GkDu6DSQnzgaT38TvkjrG16pbHX0JqHdksL++4\nfEy3yuXLkN6/f87HeFvSe7O6s+9bpFtHq5I++B8H7s3l3yK9Hw8mva+Gkb6s701gvjxNo08tkZ68\new04uZHyxXIdKx1F/5ffc6uRvtDtDuDuXDaIlKz8M2/PJqR+N7/J5Xvlbft+Lv8xqcUgCuubBJxe\nVYfb8jFcmZQI3kd6T7+bt1d5nS+TEsdhwL/ycZq3sJy98r66vYNi5lr5+I4inc/bk2LlToVpvkl6\njx6S31M/zMMbFKa5ADizMNyfdGt0cVLCcWphuBI/fkk69yvvtdGk+DJbp13S040/rxp3Jen7bJYn\n9ac5pb32Sxleda+AXx10YJt+muVHObiNKIw7ktR0Oo0UvH/J7B/SO5MeSf0on3THVS33xHzyfQhc\nRvrwLn64r58D5/t5mnEUngrJQe5V0gf1Tk1s2zykq9ZH8rQfkG497EfVkx55/T8F/pKD0nuk+969\nC9NcRHr8fFwj69ubQiKTx93LrMlgZR8NKYzbndQnaVreL2cCfQvlK+XlfJr3/W55/4wuTLMq6Wme\nj0i3Q+4gP7HU3HHO5XPl4/Iy6UPgGWZ9cmgkXSSRyeN2zMd1Wj5WF1N4Sq0l6y1MuzBwCqlV8ZO8\nvHGkZHtgYbrZ6pbHz096AuqNXJ/HmP0pph/kfftxPk6Vp1xG5vLRpJaOXUkfzJ+SkonlC8v4PumD\nfiopcb2JWZ8savQYkxLmoPDUU41p7gCuz/8vAVxOOgffJiVrSxSmXZt0Lk0lxYGTyV+FQGoJPI+U\nBL6X981eNJ/IrETqbzI178NNSbfCJpOSvD6kfifn5zpNISWww6qWsyAp0di3Jcd/Tl6kFt0J+f3y\nFLBXjWl2y2XTSOf3DlXlsxyv/B6IRl6V2NiLdJ6+ltc9Flilxrq3zPt4vqrxQ0iP1X9IemqpRz2O\nXfmeArNuKX9Z15ER0ZI+JWbtSumLHleLiJY8ZWZNUPrCyLNIj7JXd5K3HmyuelfAzMysMZIGkb7O\n4Azg105irJoTGTMz68rGkPpLnU9KZsxm4VtLZmZmVlp+/NrMzMxKy4mMmZmZlZYTGTMzMystJzJm\nZmZWWk5kzMzMrLScyJiZmVlpOZExMzOz0nIiY2ZmZqXlRMbMzMxKy4mMmZmZlZYTGTMzMystJzJm\nZmZWWk5kzMzMrLScyJiZmVlpOZExMzOz0nIiY2ZmZqXlRMbMzMxKy4mMmZmZlZYTGTMzMyutuepd\ngZ5g0KBBMWTIkHpXw6zdjBs37q2IWKTe9eiJHE+su2lrPHEi0wmGDBlCQ0NDvath1m4kvVDvOvRU\njifW3bQ1nvjWkpmZmZWWExkzMzMrLScyZmZmVlpOZMzMzKy0nMiYmZlZafmppU4w4dX3WfuEm+pd\njRa79xffqncVzKwRnRlPHAusDNwiY2ZmZqXlRMbMzMxKy4mMmZmZlZYTmVaSNFrSmdX/m5m1luOJ\nWds5kTEzM7PSKmUiI2kzSeMkTZD0uKTvS/qnpN9XTdcgaX9JvST9TNJTkp6XdIukZfM0IyU9K+n3\neXm9Ja0s6XZJT0qalMtUn601s47keGJWbqVLZCQtAVwB/DgihgPbAKcBTwE7VgKEpKHASsClwJHA\nrsA6EbE0cDvwt8JilwIeiYjhEfE5cD5wQ0SsCHwd2A/4dmdsn5l1HscTs/IrXSIDfAsYHxFjASLi\nWeAGYG5gPmDtPN0uwDURMQXYCfhTRLyTy04D1pe0ZB6eB7iksI71gFPz8l8HngCWa00lJR2Qr+Aa\npn88pXVbaGadxfHErOTKmMgsDrxZNe5tYGFS8Ngpj9sZuCD/PwA4Njf1TgAeAN7KywL4KCI+KSxv\nW2CspKfz9KvRyn0VEedGxIiIGDF33/6tmdXMOo/jiVnJlfGbfScDi1aNWwR4FPgPcLGk8/M0la+/\nfBm4NCL+XL0wSSOrhpchNR9vERE35XH3t+cGmFmX4XhiVnJlbJG5CRgmaQMAScOAzYGrI+IeYDpw\nPDAmIj7L81wGfF/SgnmetST9vZEOdwsAATTkaXcAhgD9Om6TzKxOHE/MSq50LTIR8Zqk7YHTJM1P\nCjT7R8QjeZJ/Aj8D1izM9hdSs+99kgL4EPhpRER17ImIRySdBzwq6W3gIuAE4CRJT3fktplZ53I8\nMSs/RUS969Dt9Vty+Vh5/z/Wuxot5h+Ks+ZIGhcRI+pdj56oM+OJY4F1hrbGkzLeWjIzMzMDnMiY\nmZlZiZWuj0wZDV9yQTfRmlm7cDwxm5VbZMzMzKy0nMiYmZlZaTmRMTMzs9JyH5nO8Oo4GOUfu+1x\nRvmrDawDtCae+D1oPYBbZMzMzKy0nMiYmZlZaTmRMTMzs9JyImNmZmal5UTGzMzMSsuJjJmZmZVW\nj01kJB0m6TVJ4yUdI+lJSTtK+oak+/Pw85KOydMfJOnhqmWcLOni+myBmXUljilm9dEjExlJw4HT\ngE0jYgVgPmAZQMAlwO8jYkVgK+AkSasAFwPDJK1RWNTOwAWNrOMASQ2SGt78uAM3xszqrqNjiuOJ\nWeN6ZCIDbAQ8FBGP5eFTSF8OGMBg4FKAiHgSeB1YLiLeA64C9gWQtDYpWN1UawURcW5EjIiIEYv0\n7chNMbMuoENjiuOJWeN6aiIzAHirMhARUwvD+wL3SZooaQKwCDP302hgN0nzkK6c/hERn3darc2s\nq3JMMauTnvoTBe8DC1YGchAZlAdPA9aJiIdz2RuF+f4DfERqHt4p/zUzc0wxq5Oe2iJzF7CWpOXz\n8NHAdGB+UkAaD6nzHtAH6AcQETOAC4HfAW9HxCOdXG8z65ocU8zqpEcmMvnK6NfALZIeAt4h3bf+\nBLgHeCaPnwL8CfidpI3y7KOBITTSydfMeh7HFLP66am3lgB+FRG/rAxIOgl4IyK2rTHtcYX/XyUF\no4s6uH5mVi6OKWZ10CNbZCT1B96VtE4e3ob0mOTDTc6Y/AL4V0S83oFVNLMScUwxq58e2SITEVMk\n7Q/8TZJIzb87R8S7jc0jaSXgWmACsFvn1NTMysAxxax+FBH1rkO3N2LEiGhoaKh3NczajaRxETGi\n3vXoiRxPrLtpazzpkbeWzMzMrHtwImNmZmal5UTGzMzMSqtHdvbtbM+89zRbX+Uv7GzMNdteV+8q\nmJlZSblFxszMzErLiYyZmZmVlhMZMzMzK61SJDKShkgKSYOan7rm/HtLeryRsqUkTZC0UNtqaWZl\n4Zhi1n30+M6+EfEKMLze9TCz7sExxaxzlaJFpkjShpLulzRR0lOS9iiUHSBpvKQn82vPqnlPyeWT\nJB2ax81yZSZpe0kP5yuqp4vLyPMdJGmspBcl3S1p8c7adjNrf44pZuVWtkRmXuB6YFRELA9sA/xJ\n0lBJ/YCzgS0iYkVgC2BHSXPneYcDD0bECsD2wOmSBhcXnn/4bQxwUEQMBw4Dzpc0sDDZTnnZSwO9\ngQM7aFvNrOM5ppiVXNkSmb2B8RFxPUBETABuBL4LfAq8CxwsaXhEvBARW0fE9DzvlIi4OM/3IPA0\nsH5x4RExBegfEffmUbeSbr8NKUx2cURMjYjPSb9s+5VaFc1Xcg2SGqa9P62t221mHWNvShBTivHk\nzTffbI/tNus2ypbICBiem2gnSJoArA0MyMFlA2AJ4E5JL0j6fmHet6qW9S5QvCpCUi/gKEmPSJoI\nPJqLelXNV/EZ6QpqNhFxbkSMiIgRfRbs08rNNLNOUoqYUowniyyyyBxspln3VbbOvpOAxyJinVqF\nETEe2FOSgK2AKyWNzcUDqyYfwOyBaE/gEGC9iHheUl/go3aqu5l1PZNwTDErtbK1yNxIunpaH0BS\nP0l/lbSKpFUl3SxpgYgI4D5S03DkeQdI2j7PNwIYCtxRtfz+wGvAC5J6Az8BpgH9OnzLzKweHFPM\nSq5siUwA2wK/z03ADcDLwBOkJtsHgIckPQncDhwdEc/meR8E1slllwMH58ckiy4C3geeA+4F7snT\njpG0QodumZnVg2OKWckpXWhYR1poaP/4xqnrNz9hD+UfjSwfSeMiYkS969ETjRgxIhoaGupdDbN2\n09Z4UrYWGTMzM7MvOJExMzOz0irbU0ulNHSh5Xz7xMzMrAO4RcbMzMxKy4mMmZmZlZYTGTMzMyst\n95HpBDH5Az755dh6V8Nsjsz785H1roKZWaPcImNmZmal5UTGzMzMSsuJjJmZmZWWExkzMzMrrS6V\nyEgaIikkDapRdoukzepRr8Y0VV8zqy/HE7OeoTRPLUXEJvWug5l1D44nZt1Hl2qRKdhV0sOSXpZ0\nrqS5JU2StCOApAMkjZf0ZH7tWZlR0kG57FlJ90laM4/fX9JjkubOw3vn6eaVNFrSmYVl7ChpUmH4\nBEkTJD0t6QFJa3XanjCztnI8MevGumois3xErAasDGwG7FIpkNQPOBvYIiJWBLYAdszBaQfgF8Dm\nEbEscBZwuaReEfEX4CXgx7np9iRgj4j4pKmKSNoCOAT4ekQsB9wCnNPcBuTg2CCp4c2PprR6B5hZ\nu+le8eTNN1u9A8y6s66ayPwFICLeA64HvlEo+xR4FzhY0vCIeCEito6I6cBOwIUR8UKe/wKgH7B2\nnnc/4FDgAuDPEdHQXEUi4gZgcERUspFbgeVaMN+5ETEiIkYsMn//5rfYzDpK94oniyzS/Bab9SBd\nNZEpXnK8CwysDOQAswGwBHCnpBckfT8XDwD2zM22EyRNAD4HFs3zTgauATYlXV01S9IA4Ky8vKeA\n8+i6+83MZud4YtaNddXOvgOByfn/AcBbxcKIGE8KMAK2Aq6UNBZ4GXgwIo6ttVBJqwBbk4LH6cDu\nuehzoHdh0gGF/08DlgTWiogPJG0JXDrnm2ZmnczxxKwb66pXAvvCF1cvW5HuI5PHrSrpZkkLREQA\n95GahwO4jNSxb4k87TKSLpM0n6Q+wIXA0cCRwFqVzn6ke92rSuotqS+wa6Eu/YHxOegsRGpO7lPp\n5GdmXZ7jiVk31lUTmfGSHgIeBf4N/KtQ9ijwAPCQpCeB24GjI+LZiLgR+H/AfyWNB64ELouIqcAJ\nwIsRcXHukLc/8CdJiwHn5mU/R7qHfnlhfScD38rNwJcDo/J04zpgu82s/TmemHVjShch1pHWXGpY\n3H3gn+tdDbM5UuvXryWNi4gRnV8bGzFiRDQ0NNuv2Kw02hpPumqLjJmZmVmznMiYmZlZaXXVp5a6\nFS2xQM3meTMzM2sbt8iYmZlZaTmRMTMzs9JyImNmZmal5T4yneCd555gzM4r1LsaXdKul4yvdxXM\nSqXs8cTnvLU3t8iYmZlZaTmRMTMzs9JyImNmZmalNUeJjKQhkkLSoPauUNnk/eCvajebQ44nMzme\nmLWeW2TMzMystNqayGwu6V5Jb0u6QFIvSUMl3STpKUnjJZ0maR4ASWMlHS/pZkmvSfq3pBGSbpc0\nWdKlknrlaZtazraSHsvjn5J0dKVCkraX9IikZ/I038rjN5P0qqSF8vBGkl6XtKikUZKuLSxjhKQo\nDB8k6Ym8rsclbdHG/WZms3M8MbNWa2siMzwi1gaGATsAGwKXAndHxDBgBLAB8IPCPNsC2wHDgY2A\n3wCbASsAWwIbSFIzyzkXOCwiVgDWBtaTNEjSWsCFwD4RMRQ4DLhMUv+IuBm4Cjg5B7BzgAMj4o2m\nNlDSCsAZwBa5LmfmdZhZ+3I8MbNWa2siMxogIt4CXgTWB1YDfp/HfwT8FfhOYZ4bIuKjiHgPeAm4\nLiI+LQx/BRjczHImA3tJWg2YEhHb5jrskJf/YJ5vLPAUsFWe72hgY2AMcE9EXNXcBkbEeKB/RLyY\nR90KLCxpYFPzSTpAUoOkhiaEup8AABfjSURBVA8+/ay51ZiZ40mjHE/MGtfWROa9wv+fA9OBTyLi\nw8L4t4HFCsNTquZ5v2q4N7B4M8vZCpgKXAtMlvR/efwAYBNJEyovYElgYfgigP2FdAV3Rks2MF9t\nnVhpCgZuyEVN7ruIODciRkTEiAXm8fcOmrWA40kjHE/MGtfeZ8Q8wLySFoiID/K4RUhXPK0xuanl\nRMTLwCHAIZLWA66VdA/wMnBjROxaa6GSlgCOAP4fcIakkRExg5kBr2JA4f9jgU2AjSLiDUkrAk+0\ncnvMrPUcT8ysWe391NKLwIPA4QCSFgT2A65or+XkznT3SFo8T/sQM6/kriB1GByW51tE0iU54ACc\nB/wR+AkQwI/y+JeAFSTNK6k3sHehLv2B53PQmRc4NI/v18ptMrPWcTwxs2a1d4tMADsDZ0mq/KDG\nlaSOcC1fSERIqrmciPhM0hjgdkkzSMnYBRFxK4Ck/YFLchPuDODsiJicxy8BnJqXvz9wv6QbgMty\nvZ8hBb2zgd3yes8ELpX0DPA6KRh+FRgrafXW7R4zawXHEzNrliKi+amsTZYZOF/8ZtMh9a5Gl+Qf\nkCsnSeMiwl/cVgdljyc+561aW+OJvxDPzMzMSsuJjJmZmZWWn+PrBAOXWYldL2modzXMrBtwPDGb\nlVtkzMzMrLScyJiZmVlpOZExMzOz0nIfmU7w5jPv8OdtLqp3NdrkwKu/V+8qmBntE098Plt34hYZ\nMzMzKy0nMmZmZlZaTmTMzMystJzImJmZWWmVMpGRtLCk3Zqfsua820m6pgXTjZT0YRPlm0hacU7q\nYGZdi2OKWXmVMpEBNmbmr8m2SkRcGRFbt0MdfgQ46Jh1D44pZiXVpRMZSb0lnSXpaUnjJd0n6Sjg\nbGCkpEfydCHpaEkTJK0raUFJ/8zzPC/p35IG5Wn3lvR4/r+PpPMkvSnpQUl75GUNKtRhL0njJL0t\n6eQ87gxgM+AMSaM6ebeY2RxyTDHrfrr698h8C9gUWDEipkvaGlgXOBMYERHfLky7TEQMB5D0W2Ah\nYCWgN3ArcCzw46rl7wNsCAwFpgKXV5XPC8wXEWvmJt8nJJ0TEYdL2gY4KiKq5yHX4QDgAICB8y08\nZ1tvZu2tlDHF8cSscV26RQZ4HVgC2EvS4hFxTUT8tJFpryr8/1Ngu4iYERHTgTuB5WrMszFwVURM\niYhppGBWJOCvABHxJPAJ8OWWVDwizo2IERExol+fBVsyi5l1vFLGFMcTs8Z16UQmIsYB3wV2AJ6T\ndL+k9RqZ/O3C/18FLpc0UdIE0pVMrW0dALxVGH6xqnxqDkYVn5OuxsyshBxTzLqfLp3IAETEDRGx\nBbAwcCMwpgWzXQ08TGo+Hk6+AqrhfaB4ebNUW+pqZl2fY4pZ99KlExlJ+0o6Q1KviJgK3JuLpgEL\nSVIjs/YHHoyIzyStAGwF9Ksx3V3AdpL6SuoDHNKK6k0DBrZiejOrM8cUs+6nSycywJXA4sDTkp4A\nfgHsQbqKWg6YLGlAjfmOBv4o6UngJ8BBwKqS/lg13bnAY8BE4DZm3hOPFtRtDHC6pPNat0lmVkeO\nKWbdjCJacn51X5IUeSdIWgO4H5g3Ij5rr3UMXmiZOG7DX7bX4urCv5ZrRZLGRcSIetejK+romNIe\n8cTns3UlbY0nXb1FpkNJ2gSYJKnSnPs94J72TGLMrOdwTDHrfF39e2Q62q3AhcA4SZ+RnjDYt71X\nssjQgb4CMusZOjymOJ6YzapHJzK5+ff4/DIzaxPHFLPO16NvLZmZmVm5OZExMzOz0urRt5Y6y/RH\nH+WVpVr0ywalt9QrL9W7CmbdWk+KJ9a9tdfnhVtkzMzMrLScyJiZmVlpOZExMzOz0nIiY2ZmZqXV\nbCIjaYikkDSoNQuWtLCk3ea8al2fpFGSrq13PczKxDGlcY4pZq3XkS0yGwPdOuiYWadyTDGz2bQm\nkdlc0r2S3pZ0gaRekgZLulrSREkvSDoljx8JnA2MlPQIgKSvSrotT/ucpKMqC5Y0VtJJkh6RdEQe\n92NJT0qakNe7bh4/v6SL83KeystcJpctnOs2UdKLks6XNF+u0x2SflpY522SRuX/Q9KIQtm1hbIv\nSbpO0nhJkyRdKGneOdvdZlbgmOKYYtZmrUlkhkfE2sAwYAdgQ+DfwPg8bhVgU2DviBgLnAmMjYhV\nJfUFbgYujYjlgfWAIyVtXFj+ZsA6EXG6pG2AI4FvRsRw4HTgyrycvYBFgWERMQwYA2ydlzE6b9OK\nwFBgSeDYiJgB7A38SNLSkvYB+gG/bsF2/w54KSJWAFYC1gH2b/luM7NGOKY4ppi1WWsSmdEAEfEW\n6YfQ1icFmt9E8j5wHrBLjXm/AcwVEWfnZUwG/lk17c0R8XH+fztgTES8mqe/OI//GjCZFFR2kjQw\nIs7NgWp+YEvg5Ij4LCKmkQLfLnkZzwE/A/4G/BLYs4W/SLsr8MO8jI+A+4HlmptJ0gGSGiQ1vD1j\nRgtWY9bjjAbHFFoQUxxPzBrXmm/2fa/w/+fAZ0AAD0iqjO8DvFZj3gHAApImFMbNC9xbGH678P/i\nwJNVy3gHWCwiLskB5hDgAkn/Aw7N9etFusr6PM/TO6+nYjRwEnBnRIxvfFNnsSFwvKSvkLZ7CdKv\n2zYpIs4FzgVYtU+faOG6zHoSx5QWxhTHE7PGteUnCirzrhYRnzQz7cvA27lJtyUmk5p6AVCKaoPy\neCLiIuAiSf1JzbR/BkYC04FvR8RTjSx3FHAD8HVJm0XEzXn8DFKAqhiQ19sXuBY4Ajg/ImZIurSF\n22BmreOYYmat1panll4iNYkeCZA7vx0vaddcPg1YKAeM+4HplTJJc0s6TdImjSz7CmBXSYvl4e8B\nn5Cu1H5W6WAXEVOAB/P/M/J8R+Z1VppjK/VbJy/nUOBA4HxJCxW2ZY083Qhg1Tx+bqAv0JADznrA\n2qR74WbWvhxTzKzV2vr49c7AupKeAiaQOq7dmMtuJN33nUw6cb8DHJinfQKYB7ir1kIj4t/AqcCt\nuen4AGCbiJgK/B3YSNIzkp4k3a8+JM96KCkgTMjr2Qa4Il8F/R04PCLei4hbgP+Q7ncDHAMcK+kJ\nUqe7K3M9pgAnANdJehzYHjgM2E7SMW3Yb2ZWm2OKmbWKIny7taOt2qdPXL/IYs1P2A341697Bknj\nImJE81Nae+tJ8cS6t8rnRVvjiX+iwMzMzErLiYyZmZmVVlueWrIWmvurX2WphoZ6V8PMugHHE7NZ\nuUXGzMzMSsuJjJmZmZWWExkzMzMrLScyZmZmVlpOZMzMzKy0nMiYmZlZafmbfTuBpA+Axn50rqsb\nBLxV70rMIde94wyOiEXqXYmeqOTxpEy6+jnYHVT2cZviib9HpnM8Vdavc5fU4Lp3vjLX3TpcaeNJ\nmfgc7HjttY99a8nMzMxKy4mMmZmZlZYTmc5xbr0r0Aaue32Uue7Wsfze6Bzezx2vXfaxO/uamZlZ\nablFxszMzErLiYyZmZmVlhOZDiRpLUn3SnpG0nhJe9a7Ts2RdICkjyQdVRg3SNLVkp6V9LSk30nq\nMu8dSZtIuk/ShFy/I/P4Ll1vAEmbS2rIdR8v6fA8vsvX3TpXGeNJVydpiKTI51/xNUjSspJuzefg\nREnH1Lu+ZdTazxRJvfLwM/l1taRBTa3DgbGDSJoHuBI4PSKGAt8B/iBplfrWrHGSzgK+CUyoKjoH\neBUYCqwGbAgc1Lm1q03S4sDVwHERMRzYHPilpHXowvWGL+p+OXB4rvtWwK8kbUAXr7t1rjLGkzKJ\niOFVr7eAi4GbI2JZYF3gB5K2rG9Ny2UOP1MOycOrAssBrwB/amo9TmQ6ziYAEXFx/vsMcB2waz0r\n1YwxEfFd4IPKCEkLANsCv4/kI+DPwPfqVMdqnwN7RMQtABHxLPAk8DW6dr0BAtg9Iu4GiIjngKdJ\nJ3ZXr7t1rjLGk9KStCLpg/QPADmxuRCfg601J58pewJ/joiPIj2NdBqwnaT5G1uJE5mOM5z0oVQ0\nEVipDnVpkYi4q8bo5fLfZwvjusx2RMSbEXFlZVjSssDKwEN5VJesN0BEvB4RV1eGJW0MDAb+l0d1\n2bpbpytdPCkTSRdKejzf5t2DtL9fiYiPC5N5f7fSHH6mDM/DFKbrBSzf2Hr8EwUdZ35gatW4qXl8\nmcwPTIuIGYVxXXI7JH0J+DdwCqm1oyz13pJ0RTIfqXm1LyWpu3Wa7hJPupoPgb8Cf4yIhyWtD9wM\n/Bbv747S3GfKLO/1iJgh6VOa2Pdukek4H5I+mIrmz+PL5ENgnqqOpl1uOyStAdwD/D0ifkFJ6g0Q\nEddHxJeBDYCTgfUpSd2t03SXeNKlRMRbEbFfRDych+8CriHd3vD+7hjNxeZZ3uuSegPz0MS+dyLT\ncZ5g9qawFYBH61CXtphI6ocytDCuS21HTmKuB46IiN/m0WWo9zBJW1eGI2I8KYh+jS5ed+t03SWe\ndCmSBkoaWjW6F3A/8CVJfQvjvb/bR3Ox+QlgWKFsGPAZTfziuxOZjnMb8JmkfQAkrQpsBlxU11q1\nUu6IdTlwnJKFSL3K/1bfmiWS5gUuAw6NiH9Vxnf1emcDgH9I+ipAruM3SX1kunrdrXN1i3jSBa0D\n3C1pMICklYEtgNOBB4Cf5PFfAfbA52CbtSA2jyY9IdZfkoBjgYsjovpW3xf8EwUdSNJqpMfGFgE+\nAUYVP2y7ktx890Qe/AqpGe8d0iOfpwDnkZ6m+Zz0WOIJ0QXePJJ2JQXz6o6QFwNn0EXrXZG/C+T/\ngN6ASC0yxwD96OJ1t85VpnhSJvm7mw7Og58AJ0fExTm5OQ9YGphO6kfT5GPANtOcfqbk5OUkYAdS\nTGwADoyIKY2uy3HRzMzMysq3lszMzKy0nMiYmZlZaTmRMTMzs9JyImNmZmal5UTGzMzMSsuJjJmZ\nmZWWE5luRtIkSa9ImlD12r+F8+7YjnUZKWlGoQ5PS3qiJXVpZrlfkzSuMLxf5Rs4JZ0k6fi21r1q\nfZH3TWU7nso/LrddC+efS9LBzU9Zc94z8367JX9xVGX8QEmPSVp4TpZr1pQuGEdC0hE1yjr0+0Oq\nz918Hm7WjsuvjpETJE2UdKOkVVq4jGXz77W1dt0LSfpvji9/qirbQNJ/WrvMevGPRnZPh0fE5fWu\nRPZxRAyvDEgaDvxP0vMR8d85WWBE3A+smZfXm/Qz71fndR3bDnWuZceIaKgMSFoP+K+kVSNiYhPz\nAawOHAqc3ZoVSlodWJH01fQ/B/YhbSukbx49MSLebs0yzVqhK8WRF4GfSboiIl7sxPXOcu5GxCYd\nsI7qGCngaOBaYHAL5t8+T3d9K9e7D3BnRHxT0q2SVo+Ih/K3pf8BaLdktKO5RaaHkbS9pIcLLSR7\nNjLdAZLGS3oyv/YslB2Uy56VdJ+kNVu6/oiYAFxH+hpwJA2VdFNu5Rgv6TRJ8+SybXOrw/hcfnQe\nP1JS5QfE7gMWAO6RtKWk0bkVYzNJb+REp1LvH0i6O/+/oaT789XPU5L2aM1+jIi7gTeAys8LfEnS\ndbmukyRdKGleScuRvq13ubzPB0kaLOnqvO4XJJ2iWX9ArWIF4L78bb73k37eHkmbAwMjYkxr6mzW\nXuoQR54nfavxWU3UqdFzOrcwPCvpOUkXSTpf0pm5rDXn7iRJO0o6UdIVVeu/XNJv5mDbZpHP98uA\nr0galJf3jbxtT0p6XtIxefzuwPHA7pKub24/VFkBuDf//wA5vgCjgH9ExLMtrXPdRYRf3egFTCK1\nHtQq6w98CqydhzcnffX2wOK8pK/H/xwYkscPJp3Qc5O+Nvp1YHAu25MUZHrVWN9I4MMa4y8BTiR9\n/fSDwM/z+PlJX0f94zz8BjAy/z8AuAoYVFwuMAQIYFAeHg2cSfrK/9eATQrrvQs4EPgS8BGwZR4/\nHPgAGNrIfgtgRGFYwM7Ax8CXCtt0TmE7ngEOy8N7A48X5n2U9CvXAhYEHgL2rbHenYFT8v9bkq4K\n+wGPkb7aewxwC7BLvd93fnWvVxeMI2NJv4D8FLBToSzy30bP6RwLXgYOyWUbAVOBM/Nwi87dqm1b\nKZ///fL4BfLwsDnYtg+rxvUhtbg+locFTK6c56RW2s+BVfLw6MK2tDi25XhSme6UHG9WJyU3a5Eu\nOG8CVq33+7G5l1tkuqczNPu97U0i/VZF/4ioZOG3km4vDqma/1PgXeBgScMj4oWI2DoipgM7ARdG\nxAsAEXEBKWCt3ZKKSVqH9KF8GSmwrQb8Pi/rI+CvwHfy5JOBvZR+Y2ZKRGwbEW+1ZD0RUfn9ju/m\n9X4ZWIMUtL4NjI+I6/O0E4AbK9M24vLKviQFigOBb0bEy7l8V+CHhe24H1iuxnKGAasAv4nkfdJv\njuxSY9oGYANJc5F+IPB+UgJ0DimY3gRsB/xOUr9md4pZ63SpOBIRnwIH5Xr1rypu6pxeHlgK+Hsu\nu4108VDR0nO3WJcnSL/iXIlV25ASj6fmYNv6FvbvM6T4siApQaxkaoOBS/Pwk6REqVYdWxPb7gc2\nyfFlQ+AR4FzSPv4d8ANSvDm9qX3RFbiPTPdU8952vn1xlKSdgPmAGbloloQ2IqZL2oD0q6N3SvoY\n+FVEnEdqGdlY0rcLs3wOLNpIXfrmD//KeiYD3410L3Zt4JOI+LAw/dvAYvn/rYDjSPeK55b0h4j4\nTUt2QHYRcL2kQ0gn8nUR8Z6kAcDwQr0gXYlNamJZX/SRkTQG+DQi/lco3xA4XulXcj8HlgAurLGc\nAaQWngckVcb1IbUezSIinpV0DakF5kHgJWBf4DDS/fALI+J9SS+QrtLub6L+Zq3VleJIZZm3SbqR\n1IJwYKGoqXN6ACnOfFQoK/azaem5W+0iUtIyhtSacUGhLq3Zti/6yCj1TxkP3BERrxSm2RfYLydw\nM0g/HFqrIaI1se2fpF+cfozUqrMD8N+IeFjSlyPieUmvAF9vpN5dhhOZnmVP0s+lr5ffpH1J2f9s\nImI8sKfSp+1WwJWSxpKaaB+MlneqnaUjW5XJwLySFoiID/K4RfJ4cmvHIcAhSp1rr5V0DzMDZ5Mi\nokHSO6RAtTPw61z0MunqaZ0WbkO1nwDjJf0lIu7O+/Fa4Ajg/IiYIenSRuattOCsFhGftGAbTgJO\nygHuPlLzcqjQ94fUdP7ZHG6LWWvVI44UHQU8KemCwrhGz2lJKwPzSJont+pAaqF5q5XnbrUxwChJ\nS5FuV+1dqMscbVtEfKLUF/AsSVflC6+RpE7+60TEw3mb3mhkES2ObXlf7JaXN5zU4vO1XFxJkkoR\nW3xrqWfpT7ryfyF/EP4EmEZq9vyCpFUl3ZwTjCB9gH5Kakm4DNhV0hJ52mUkXSZpvjmoz4uklobD\n87IWBPYDrpC0qKR7JC2ep30IeK/GMqblvwMbWcc/gANIzd435HE3kq5a1s/r7Sfpr2rh446Rnpo4\nlRRsepPu+fcFGnIgXI/UjFzZr9OABSTNFREvkVpOjszr7iXpeEm7NrPaE4B/5Q8GSP1s1lZ6JHtp\n4OmW1N2sHdQ1jkTEO8CPSbdBKpo6pycCbzHzQ3skqS8ItOLcrVGPV/I2nUpqyag8QdimGJlbwSYw\n88KrP/A+qaUGSYeRWnGLdRzQgv1QU25h+wupX1Dl4up1SUsD65JiTdfWmR1y/Or4F0130luY1Glu\nEqmX+uakD/rJpB7sk0h9LwT8htTp7cn8OriwnEOBJ0gn1iOkW0W11jeSGp19q6ZZlnTyjc+vE4G5\nctkPSZ37xue/v6hebq7rf0gd2r5PoeNbLl+G1ILzh6r1bkhKKCbk1y+p0RkvTztLZ988ri/pVs8P\n8/DPgVeBx0n3l7cBpgDHkDrgPU8KpiuT7nf/O2/TRNKV3YAm9tHqpERu7sK4pUhB9PlKHfzyq71e\nXTCOjK0x/kZyF5I83Og5TWoNeiav62zSbaE/5rLWnLuz7BfSLZ8Atq+qW5tiJKnv4LR87vchPejw\nUo4De5Li5NuklqAtc32fam4/NFKHw4Bzq8ZtnOd9Gtig3u/H5l7KlTYzM+u2JCnyB17ue/ZQRJxQ\n52pZO/CtJTMz69Yk3Ua6BUbu1LshcEddK2Xtxi0yZmbWrUn6KvBn0m0xSB17f1vHKlk7ciJjZmZm\npeVbS2ZmZlZaTmTMzMystJzImJmZWWk5kTEzM7PSciJjZmZmpfX/AdoUiokE3ZL4AAAAAElFTkSu\nQmCC\n",
            "text/plain": [
              "\u003cFigure size 576x216 with 2 Axes\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "plot_metrics(labels_test, scores_test, subgroups_test,\n",
        "             group_names=terms_sexuality, title=\"Robust Opt Model on Finer Groups\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PrqVCJRhWs-U"
      },
      "source": [
        "Notice that comments that contain the term \"heterosexual\" only constitute 0.03% of the dataset. It's therefore unsuprising that the robust optimization model has a false negative rate of 100% for this subgroup. Also notice that the comments containing the term \"gay\" and the comments containing the term \"bisexual\" have a higher false positive rate than the larger group of sexuality-related comments.\n",
        "\n",
        "While one could consider training a model by directly constraining these tiny subgroups, such a model is likely to generalize very poorly to new data. A better strategy would be to instead collect more training data for each of these subgroups, and then control performance on these subgroups through a constrained or robust optimization set-up."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "last_runtime": {
        "build_target": "",
        "kind": "local"
      },
      "name": "Wiki_toxicity_fairness.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.5rc1"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}