predict-next-purchase/Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting a customer's next purchase using automated feature engineering\n",
    "\n",
    "<p style=\"margin:30px\">\n",
    "    <img width=50% src=\"https://www.featuretools.com/wp-content/uploads/2017/12/FeatureLabs-Logo-Tangerine-800.png\" alt=\"Featuretools\" />\n",
    "</p>\n",
    "\n",
    "**As customers use your product, they leave behind a trail of behaviors that indicate how they will act in the future. Through automated feature engineering we can identify the predictive patterns in granular customer behavioral data that can be used to improve the customer's experience and generate additional revenue for your business.**\n",
    "\n",
    "In this tutorial, we show how [Featuretools](https://github.com/alteryx/featuretools) can be used to perform feature engineering on a multi-table dataset of 3 million online grocery orders provided by Instacart. We will generate a feature matrix that can be used to train a machine learning model to predict what product a customer buys next.\n",
    "\n",
    "*Note: This notebook requires a dataset from Instacart. You can download the dataset [here](https://www.kaggle.com/c/instacart-market-basket-analysis/data). Once you have downloaded the data, be sure to place the CSV files contained in the archive in a directory called `data`. If you use a different directory name, you will need to update the code below to point to the proper location.*\n",
    "\n",
    "## Highlights\n",
    "\n",
    "* We automatically generate features using Deep Feature Synthesis and select the 20 most important features for predictive modeling\n",
    "* We demonstrate how to generate features in a scalable manner using [Dask](http://dask.pydata.org/en/latest/)\n",
    "* We automatically generate label times using [Compose](https://github.com/alteryx/compose) which can be reused for numerous prediction problems\n",
    "* We develop a model for predicting what a customer will buy next, starting with a sample of data and then scaling to the full dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## You must have Featuretools version 1.4.0 or greater installed to run this notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.4.0'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import warnings\n",
    "warnings.simplefilter(\"ignore\")\n",
    "\n",
    "import os\n",
    "import composeml as cp\n",
    "import featuretools as ft\n",
    "import dask.dataframe as dd\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import utils\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import cross_val_score\n",
    "\n",
    "from dask.distributed import Client\n",
    "ft.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1. Load and preprocess data\n",
    "\n",
    "First, we will create a Dask distributed client so we can track the progress of our computation on the Dask dashboard that is created when the client is initialized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\"> </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px;\">Client</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Client-cb830540-85ed-11ec-99fe-acde48001122</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "\n",
       "        <tr>\n",
       "        \n",
       "            <td style=\"text-align: left;\"><strong>Connection method:</strong> Cluster object</td>\n",
       "            <td style=\"text-align: left;\"><strong>Cluster type:</strong> distributed.LocalCluster</td>\n",
       "        \n",
       "        </tr>\n",
       "\n",
       "        \n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\"></td>\n",
       "            </tr>\n",
       "        \n",
       "\n",
       "        </table>\n",
       "\n",
       "        \n",
       "            <details>\n",
       "            <summary style=\"margin-bottom: 20px;\"><h3 style=\"display: inline;\">Cluster Info</h3></summary>\n",
       "            <div class=\"jp-RenderedHTMLCommon jp-RenderedHTML jp-mod-trusted jp-OutputArea-output\">\n",
       "    <div style=\"width: 24px; height: 24px; background-color: #e1e1e1; border: 3px solid #9D9D9D; border-radius: 5px; position: absolute;\">\n",
       "    </div>\n",
       "    <div style=\"margin-left: 48px;\">\n",
       "        <h3 style=\"margin-bottom: 0px; margin-top: 0px;\">LocalCluster</h3>\n",
       "        <p style=\"color: #9D9D9D; margin-bottom: 0px;\">6c4b1db6</p>\n",
       "        <table style=\"width: 100%; text-align: left;\">\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Dashboard:</strong> <a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a>\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Workers:</strong> 2\n",
       "                </td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total threads:</strong> 16\n",
       "                </td>\n",
       "                <td style=\"text-align: left;\">\n",
       "                    <strong>Total memory:</strong> 16.00 GiB\n",
       "                </td>\n",
       "            </tr>\n",
       "            \n",
       "            <tr>\n",
       "    <td style=\"text-align: left;\"><strong>Status:</strong> running</td>\n",
       "    <td style=\"text-align: left;\"><strong>Using processes:</strong> True</td>\n",
       "</tr>\n",
       "\n",
       "            \n",
       "        </table>\n",
       "\n",
       "        <details>\n",
       "            <summary style=\"margin-bottom: 20px;\">\n",
       "                <h3 style=\"display: inline;\">Scheduler Info</h3>\n",
       "            </summary>\n",
       "\n",
       "            <div style=\"\">\n",
       "    <div>\n",
       "        <div style=\"width: 24px; height: 24px; background-color: #FFF7E5; border: 3px solid #FF6132; border-radius: 5px; position: absolute;\"> </div>\n",
       "        <div style=\"margin-left: 48px;\">\n",
       "            <h3 style=\"margin-bottom: 0px;\">Scheduler</h3>\n",
       "            <p style=\"color: #9D9D9D; margin-bottom: 0px;\">Scheduler-58bd1277-dde4-4e16-879a-b340b68caa5d</p>\n",
       "            <table style=\"width: 100%; text-align: left;\">\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Comm:</strong> tcp://127.0.0.1:54775\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Workers:</strong> 2\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Dashboard:</strong> <a href=\"http://127.0.0.1:8787/status\" target=\"_blank\">http://127.0.0.1:8787/status</a>\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total threads:</strong> 16\n",
       "                    </td>\n",
       "                </tr>\n",
       "                <tr>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Started:</strong> Just now\n",
       "                    </td>\n",
       "                    <td style=\"text-align: left;\">\n",
       "                        <strong>Total memory:</strong> 16.00 GiB\n",
       "                    </td>\n",
       "                </tr>\n",
       "            </table>\n",
       "        </div>\n",
       "    </div>\n",
       "\n",
       "    <details style=\"margin-left: 48px;\">\n",
       "        <summary style=\"margin-bottom: 20px;\">\n",
       "            <h3 style=\"display: inline;\">Workers</h3>\n",
       "        </summary>\n",
       "\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 0</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:54783\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 8\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:54786/status\" target=\"_blank\">http://127.0.0.1:54786/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 8.00 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:54778\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /Users/nate.parsons/dev/open_source_demos/predict-next-purchase/dask-worker-space/worker-fppdmzna\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "        <div style=\"margin-bottom: 20px;\">\n",
       "            <div style=\"width: 24px; height: 24px; background-color: #DBF5FF; border: 3px solid #4CC9FF; border-radius: 5px; position: absolute;\"> </div>\n",
       "            <div style=\"margin-left: 48px;\">\n",
       "            <details>\n",
       "                <summary>\n",
       "                    <h4 style=\"margin-bottom: 0px; display: inline;\">Worker: 1</h4>\n",
       "                </summary>\n",
       "                <table style=\"width: 100%; text-align: left;\">\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Comm: </strong> tcp://127.0.0.1:54784\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Total threads: </strong> 8\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Dashboard: </strong> <a href=\"http://127.0.0.1:54785/status\" target=\"_blank\">http://127.0.0.1:54785/status</a>\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Memory: </strong> 8.00 GiB\n",
       "                        </td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td style=\"text-align: left;\">\n",
       "                            <strong>Nanny: </strong> tcp://127.0.0.1:54779\n",
       "                        </td>\n",
       "                        <td style=\"text-align: left;\"></td>\n",
       "                    </tr>\n",
       "                    <tr>\n",
       "                        <td colspan=\"2\" style=\"text-align: left;\">\n",
       "                            <strong>Local directory: </strong> /Users/nate.parsons/dev/open_source_demos/predict-next-purchase/dask-worker-space/worker-wf7i7dzw\n",
       "                        </td>\n",
       "                    </tr>\n",
       "\n",
       "                    \n",
       "\n",
       "                    \n",
       "\n",
       "                </table>\n",
       "            </details>\n",
       "            </div>\n",
       "        </div>\n",
       "        \n",
       "\n",
       "    </details>\n",
       "</div>\n",
       "\n",
       "        </details>\n",
       "    </div>\n",
       "</div>\n",
       "            </details>\n",
       "        \n",
       "\n",
       "    </div>\n",
       "</div>"
      ],
      "text/plain": [
       "<Client: 'tcp://127.0.0.1:54775' processes=2 threads=16, memory=16.00 GiB>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client = Client(n_workers=2)\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will specify our input and output directories and set the blocksize we will be using to read the raw CSV files into Dask dataframes. When running on a machine with 16GB of memory available for two Dask workers, a `100MB` blocksize has worked well. This number may need to be adjusted based on your specific environment. Refer to the [Dask documentation](https://docs.dask.org/en/latest/best-practices.html) for additional info."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dir = os.path.join(\"data\")\n",
    "output_dir = os.path.join(\"data\", \"dask_data\")\n",
    "blocksize = \"100MB\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will read our data into Dask dataframes. This operation will complete quite fast as we are not actually bringing the data into memory at this stage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "order_products = dd.concat([dd.read_csv(os.path.join(data_dir, \"order_products__prior.csv\"), blocksize=blocksize),\n",
    "                            dd.read_csv(os.path.join(data_dir, \"order_products__train.csv\"), blocksize=blocksize)])\n",
    "orders = dd.read_csv(os.path.join(data_dir, \"orders.csv\"), blocksize=blocksize)\n",
    "departments = dd.read_csv(os.path.join(data_dir, \"departments.csv\"), blocksize=blocksize)\n",
    "products = dd.read_csv(os.path.join(data_dir, \"products.csv\"), blocksize=blocksize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the next few cells, we will perform some required preprocessing to clean up our data. We will merge together some of the raw dataframes and add absolute order time information from the relative times used in the raw data. This will allow us to use cutoff times as part of the Deep Feature Synthesis process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "order_products = order_products.merge(products).merge(departments)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def add_time(df):\n",
    "    df.reset_index(drop=True)\n",
    "    df[\"order_time\"] = np.nan\n",
    "    days_since = df.columns.tolist().index(\"days_since_prior_order\")\n",
    "    hour_of_day = df.columns.tolist().index(\"order_hour_of_day\")\n",
    "    order_time = df.columns.tolist().index(\"order_time\")\n",
    "\n",
    "    df.iloc[0, order_time] = pd.Timestamp('Jan 1, 2015') +  pd.Timedelta(df.iloc[0, hour_of_day], \"h\")\n",
    "    for i in range(1, df.shape[0]):\n",
    "        df.iloc[i, order_time] = df.iloc[i - 1, order_time] \\\n",
    "            + pd.Timedelta(df.iloc[i, days_since], \"d\") \\\n",
    "                                    + pd.Timedelta(df.iloc[i, hour_of_day], \"h\")\n",
    "\n",
    "    to_drop = [\"order_number\", \"order_dow\", \"order_hour_of_day\", \"days_since_prior_order\", \"eval_set\"]\n",
    "    df.drop(to_drop, axis=1, inplace=True)\n",
    "\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "orders = orders.groupby(\"user_id\").apply(add_time, meta={\n",
    "    \"order_id\": int,\n",
    "    \"user_id\": int,\n",
    "    \"order_time\": np.datetime64(0, 'ns').dtype})\n",
    "order_products = order_products.merge(orders[[\"order_id\", \"order_time\"]])\n",
    "order_products[\"order_product_id\"] = order_products[\"order_id\"] * 1000 + order_products[\"add_to_cart_order\"]\n",
    "order_products = order_products.drop([\"product_id\", \"department_id\", \"add_to_cart_order\"], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the preprocessing work is complete, we will save the results to disk. This will allow us to start from this point in the process in the future, without having to repeat all of the preprocessing steps. If you have already saved the results to disk previously, you can skip the cell below.\n",
    "\n",
    "#### Note: The process of saving to CSV is computationally intensive and may take 45 minutes or more, depending on the system you are using. You can use the Dask dashboard to monitor the progress."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save preprocessed data to disk\n",
    "# orders.to_csv(os.path.join(output_dir, \"orders-*.csv\"), index=False)\n",
    "# order_products.to_csv(os.path.join(output_dir, \"order_products-*.csv\"), index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have already performed the preprocessing steps and saved the processed files to disk, you can read them in with the commands in the following cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read preprocessed data from disk\n",
    "orders = dd.read_csv(os.path.join(output_dir, \"orders-*.csv\"), blocksize=blocksize)\n",
    "order_products = dd.read_csv(os.path.join(output_dir, \"order_products-*.csv\"), blocksize=blocksize)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>order_id</th>\n",
       "      <th>user_id</th>\n",
       "      <th>order_time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2539329</td>\n",
       "      <td>1</td>\n",
       "      <td>2015-01-01 08:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2398795</td>\n",
       "      <td>1</td>\n",
       "      <td>2015-01-16 15:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>473747</td>\n",
       "      <td>1</td>\n",
       "      <td>2015-02-07 03:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2254736</td>\n",
       "      <td>1</td>\n",
       "      <td>2015-03-08 10:00:00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>431534</td>\n",
       "      <td>1</td>\n",
       "      <td>2015-04-06 01:00:00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   order_id  user_id           order_time\n",
       "0   2539329        1  2015-01-01 08:00:00\n",
       "1   2398795        1  2015-01-16 15:00:00\n",
       "2    473747        1  2015-02-07 03:00:00\n",
       "3   2254736        1  2015-03-08 10:00:00\n",
       "4    431534        1  2015-04-06 01:00:00"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "orders.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>order_id</th>\n",
       "      <th>reordered</th>\n",
       "      <th>product_name</th>\n",
       "      <th>aisle_id</th>\n",
       "      <th>department</th>\n",
       "      <th>order_time</th>\n",
       "      <th>order_product_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>800</td>\n",
       "      <td>1</td>\n",
       "      <td>Organic Egg Whites</td>\n",
       "      <td>86</td>\n",
       "      <td>dairy eggs</td>\n",
       "      <td>2015-01-11 10:00:00</td>\n",
       "      <td>800003</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>800</td>\n",
       "      <td>0</td>\n",
       "      <td>Grilled Chicken Breast Strips</td>\n",
       "      <td>49</td>\n",
       "      <td>meat seafood</td>\n",
       "      <td>2015-01-11 10:00:00</td>\n",
       "      <td>800004</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>800</td>\n",
       "      <td>0</td>\n",
       "      <td>Nux Vomica 6C Homeopathic Pellets</td>\n",
       "      <td>47</td>\n",
       "      <td>personal care</td>\n",
       "      <td>2015-01-11 10:00:00</td>\n",
       "      <td>800005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>800</td>\n",
       "      <td>0</td>\n",
       "      <td>Kava Kava Root, Liquid Herbal Extract</td>\n",
       "      <td>47</td>\n",
       "      <td>personal care</td>\n",
       "      <td>2015-01-11 10:00:00</td>\n",
       "      <td>800006</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>800</td>\n",
       "      <td>0</td>\n",
       "      <td>Sedalia Homeopathic Medicine Stress Relief Qui...</td>\n",
       "      <td>47</td>\n",
       "      <td>personal care</td>\n",
       "      <td>2015-01-11 10:00:00</td>\n",
       "      <td>800011</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   order_id  reordered                                       product_name  \\\n",
       "0       800          1                                 Organic Egg Whites   \n",
       "1       800          0                      Grilled Chicken Breast Strips   \n",
       "2       800          0                  Nux Vomica 6C Homeopathic Pellets   \n",
       "3       800          0              Kava Kava Root, Liquid Herbal Extract   \n",
       "4       800          0  Sedalia Homeopathic Medicine Stress Relief Qui...   \n",
       "\n",
       "   aisle_id     department           order_time  order_product_id  \n",
       "0        86     dairy eggs  2015-01-11 10:00:00            800003  \n",
       "1        49   meat seafood  2015-01-11 10:00:00            800004  \n",
       "2        47  personal care  2015-01-11 10:00:00            800005  \n",
       "3        47  personal care  2015-01-11 10:00:00            800006  \n",
       "4        47  personal care  2015-01-11 10:00:00            800011  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "order_products.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: Create a Featuretools entityset\n",
    "\n",
    "When using Dask dataframes to create an entityset, type inference can be computationally intensive compared to entitysets created from pandas dataframes because of needing to pull the distributed data into memory. As a best practice, users should specify the [Woodwork logical types](https://woodwork.alteryx.com/en/stable/guides/logical_types_and_semantic_tags.html#Logical-Types) for all of the columns in the dataframes that make up the entityset when using Dask. In the following cell we define the data types for the `order_products` and `orders` dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "order_products_types = {\n",
    "    \"order_id\": \"Categorical\",\n",
    "    \"reordered\": \"BooleanNullable\",\n",
    "    \"product_name\": \"Categorical\",\n",
    "    \"aisle_id\": \"Categorical\",\n",
    "    \"department\": \"Categorical\",\n",
    "    \"order_time\": \"Datetime\",\n",
    "    \"order_product_id\": \"Integer\",\n",
    "}\n",
    "\n",
    "order_types = {\n",
    "    \"order_id\": \"Integer\",\n",
    "    \"user_id\": \"Integer\",\n",
    "    \"order_time\": \"Datetime\",\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have defined the data types, we can create the entityset and establish the relationship between the two dataframes. For our initial run we will use a sample of the full data to determine what features are the best predictors. Once we have the feature importances established we will rerun on the full dataset using only the most important features.\n",
    "\n",
    "First we will sample our data - grabbing the orders for 1000 different customers. Because we cannot pass a Dask series to `.isin()` we must call `.compute()` on the ids to convert this into a pandas series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "orders_sample = orders[orders['user_id'].isin(range(1001,2001))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will get the order products associated with the orders we sampled. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "order_products_sample = order_products[order_products['order_id'].isin(orders_sample['order_id'].compute())]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have sampled our data, we can create an entityset from these sampled dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: instacart_sample\n",
       "  DataFrames:\n",
       "    order_products [Rows: Delayed('int-45548309-29ce-49d0-9e2b-17872e86dc68'), Columns: 7]\n",
       "    orders [Rows: Delayed('int-e0bff2a5-4009-4b22-b953-071bc05bbf2e'), Columns: 3]\n",
       "  Relationships:\n",
       "    order_products.order_id -> orders.order_id"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es = ft.EntitySet(\"instacart_sample\")\n",
    "es.add_dataframe(dataframe_name=\"order_products\",\n",
    "                 dataframe=order_products_sample,\n",
    "                 index=\"order_product_id\",\n",
    "                 logical_types=order_products_types,\n",
    "                 time_index=\"order_time\")\n",
    "\n",
    "es.add_dataframe(dataframe_name=\"orders\",\n",
    "                 dataframe=orders_sample,\n",
    "                 index=\"order_id\",\n",
    "                 logical_types=order_types,\n",
    "                 time_index=\"order_time\")\n",
    "\n",
    "es.add_relationship(\"orders\", \"order_id\", \"order_products\", \"order_id\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will normalize the `orders` dataframe to create a new `users` dataframe that we will later use as the target dataframe during the deep feature synthesis process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.44.1 (20200629.0846)\n",
       " -->\n",
       "<!-- Title: instacart_sample Pages: 1 -->\n",
       "<svg width=\"248pt\" height=\"356pt\"\n",
       " viewBox=\"0.00 0.00 248.00 356.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 352)\">\n",
       "<title>instacart_sample</title>\n",
       "<polygon fill=\"white\" stroke=\"transparent\" points=\"-4,4 -4,-352 244,-352 244,4 -4,4\"/>\n",
       "<!-- order_products -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>order_products</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"16.5,-211.5 16.5,-347.5 223.5,-347.5 223.5,-211.5 16.5,-211.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"120\" y=\"-332.3\" font-family=\"Times,serif\" font-size=\"14.00\">order_products</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"16.5,-324.5 223.5,-324.5 \"/>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-309.3\" font-family=\"Times,serif\" font-size=\"14.00\">order_id : Integer; foreign_key</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-294.3\" font-family=\"Times,serif\" font-size=\"14.00\">reordered : BooleanNullable</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-279.3\" font-family=\"Times,serif\" font-size=\"14.00\">product_name : Categorical</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-264.3\" font-family=\"Times,serif\" font-size=\"14.00\">aisle_id : Categorical</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-249.3\" font-family=\"Times,serif\" font-size=\"14.00\">department : Categorical</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-234.3\" font-family=\"Times,serif\" font-size=\"14.00\">order_time : Datetime; time_index</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-219.3\" font-family=\"Times,serif\" font-size=\"14.00\">order_product_id : Integer; index</text>\n",
       "</g>\n",
       "<!-- orders -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>orders</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"16.5,-98.5 16.5,-174.5 223.5,-174.5 223.5,-98.5 16.5,-98.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"120\" y=\"-159.3\" font-family=\"Times,serif\" font-size=\"14.00\">orders</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"16.5,-151.5 223.5,-151.5 \"/>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-136.3\" font-family=\"Times,serif\" font-size=\"14.00\">order_id : Integer; index</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-121.3\" font-family=\"Times,serif\" font-size=\"14.00\">user_id : Integer; foreign_key</text>\n",
       "<text text-anchor=\"start\" x=\"24.5\" y=\"-106.3\" font-family=\"Times,serif\" font-size=\"14.00\">order_time : Datetime; time_index</text>\n",
       "</g>\n",
       "<!-- order_products&#45;&gt;orders -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>order_products&#45;&gt;orders</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M120,-211.35C120,-211.35 120,-184.54 120,-184.54\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"123.5,-184.54 120,-174.54 116.5,-184.54 123.5,-184.54\"/>\n",
       "<text text-anchor=\"middle\" x=\"97\" y=\"-186.75\" font-family=\"Times,serif\" font-size=\"14.00\">order_id</text>\n",
       "</g>\n",
       "<!-- users -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>users</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"0,-0.5 0,-61.5 240,-61.5 240,-0.5 0,-0.5\"/>\n",
       "<text text-anchor=\"middle\" x=\"120\" y=\"-46.3\" font-family=\"Times,serif\" font-size=\"14.00\">users</text>\n",
       "<polyline fill=\"none\" stroke=\"black\" points=\"0,-38.5 240,-38.5 \"/>\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-23.3\" font-family=\"Times,serif\" font-size=\"14.00\">user_id : Integer; index</text>\n",
       "<text text-anchor=\"start\" x=\"8\" y=\"-8.3\" font-family=\"Times,serif\" font-size=\"14.00\">first_orders_time : Datetime; time_index</text>\n",
       "</g>\n",
       "<!-- orders&#45;&gt;users -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>orders&#45;&gt;users</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M120,-98.41C120,-98.41 120,-71.76 120,-71.76\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"123.5,-71.76 120,-61.76 116.5,-71.76 123.5,-71.76\"/>\n",
       "<text text-anchor=\"middle\" x=\"100\" y=\"-73.89\" font-family=\"Times,serif\" font-size=\"14.00\">user_id</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x7febae1d3c40>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es.normalize_dataframe(base_dataframe_name=\"orders\", new_dataframe_name=\"users\", index=\"user_id\")\n",
    "es.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To finish up creation of the entityset we will add last time indexes and set some interesting values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "es.add_last_time_indexes()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "es.add_interesting_values(dataframe_name=\"order_products\",\n",
    "                          values={\n",
    "                              \"department\": ['produce', 'dairy eggs', 'snacks', 'beverages', 'frozen', 'pantry', 'bakery', 'canned goods', 'deli', 'dry goods pasta'],\n",
    "                              \"product_name\": ['Banana', 'Bag of Organic Bananas', 'Organic Baby Spinach', 'Organic Strawberries', 'Organic Hass Avocado', 'Organic Avocado', 'Large Lemon', 'Limes', 'Strawberries', 'Organic Whole Milk']\n",
    "                          })"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3. Use Compose to generate our cutoff times dataframe\n",
    "\n",
    "In the cells that follow we will demonstrate how [Compose](https://github.com/FeatureLabs/compose) can be used to generate the label times dataframe that will be used as cutoff times for deep feature synthesis.\n",
    "\n",
    "First we define a labeling function to add a label if a user has bought a specific product or not, and then we will create our `LabelMaker` using this function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bought_product(df, product_name):\n",
    "    purchased = df.product_name.str.contains(product_name).any()\n",
    "    return purchased"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Need to convert the `user_id` column type due to an issue in compose\n",
    "es['users'].ww.set_types(logical_types={'user_id': 'Integer'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "lm = cp.LabelMaker(\n",
    "    target_dataframe_name='user_id',\n",
    "    time_index='order_time',\n",
    "    labeling_function=bought_product,\n",
    "    window_size='4w',\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def denormalize(es):\n",
    "    df = es['order_products'].merge(es['orders']).merge(es['users'])\n",
    "    return df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compose does not currently work on Dask dataframes, so we must first run `.compute()` on the denormalized entityset to switch to pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = denormalize(es).compute()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can create our labels, indicating whether or not a user has purchased Bananas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed: 00:07 | Remaining: 00:00 | Progress: 100%|███████████████████████████████████████████████████████████████| user_id: 1266/1266 \n"
     ]
    }
   ],
   "source": [
    "label_times = lm.search(\n",
    "    df.sort_values('order_time'),\n",
    "    minimum_data='2015-03-15',\n",
    "    num_examples_per_instance=2,\n",
    "    product_name='Banana',\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>time</th>\n",
       "      <th>bought_product</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1002</td>\n",
       "      <td>2015-09-27</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1003</td>\n",
       "      <td>2015-03-15</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1006</td>\n",
       "      <td>2015-04-12</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1007</td>\n",
       "      <td>2015-03-15</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1008</td>\n",
       "      <td>2015-03-15</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id       time  bought_product\n",
       "0     1002 2015-09-27           False\n",
       "1     1003 2015-03-15           False\n",
       "2     1006 2015-04-12           False\n",
       "3     1007 2015-03-15           False\n",
       "4     1008 2015-03-15            True"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "label_times.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see above the our training examples contain three pieces of information: a user id, the last time we can use data before feature engineering (called the \"cutoff time\"), and the label to predict. These are called our \"label times\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use `describe` to print out the distribution with the settings and transforms that were used to make these labels. This is useful as a reference for understanding how the labels were generated from raw data. Also, the label distribution is helpful for determining if we have imbalanced labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label Distribution\n",
      "------------------\n",
      "False     376\n",
      "True      157\n",
      "Total:    533\n",
      "\n",
      "\n",
      "Settings\n",
      "--------\n",
      "gap                                    None\n",
      "maximum_data                           None\n",
      "minimum_data                     2015-03-15\n",
      "num_examples_per_instance                 2\n",
      "target_column                bought_product\n",
      "target_dataframe_name               user_id\n",
      "target_type                        discrete\n",
      "window_size                              4w\n",
      "\n",
      "\n",
      "Transforms\n",
      "----------\n",
      "No transforms applied\n",
      "\n"
     ]
    }
   ],
   "source": [
    "label_times.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4. Run Deep Feature Synthesis\n",
    "\n",
    "With our label times created, we are ready to run deep feature synthesis to generate our feature matrix. This will execute quickly and the resulting feature matrix will be returned as a Dask dataframe. This process does not cause the feature matrix to be computed or brought into memory.\n",
    "\n",
    "When we use DFS, we specify\n",
    "- `target_dataframe_name` - the table to build features for - `users` in this case\n",
    "- `cutoff_time` - the point in time to calculate the features\n",
    "\n",
    "A good way to think of the `cutoff_time` is that it let's us \"pretend\" we are at an earlier point in time when generating our features so we can simulate making predictions. We get this time for each customer from the label times we generated above.\n",
    "\n",
    "For this initial run we will not specify any primitives, which will result in all the default primitives being used to create features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Built 58 features\n",
      "Elapsed: 00:36 | Progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████\n"
     ]
    }
   ],
   "source": [
    "feature_matrix, features = ft.dfs(target_dataframe_name=\"users\",\n",
    "                                  cutoff_time=label_times,\n",
    "                                  entityset=es,\n",
    "                                  verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's compute the feature matrix and take a look at what we created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COUNT(orders)</th>\n",
       "      <th>COUNT(order_products)</th>\n",
       "      <th>NUM_UNIQUE(order_products.aisle_id)</th>\n",
       "      <th>NUM_UNIQUE(order_products.department)</th>\n",
       "      <th>NUM_UNIQUE(order_products.product_name)</th>\n",
       "      <th>PERCENT_TRUE(order_products.reordered)</th>\n",
       "      <th>DAY(first_orders_time)</th>\n",
       "      <th>MONTH(first_orders_time)</th>\n",
       "      <th>WEEKDAY(first_orders_time)</th>\n",
       "      <th>YEAR(first_orders_time)</th>\n",
       "      <th>...</th>\n",
       "      <th>COUNT(order_products WHERE department = frozen)</th>\n",
       "      <th>COUNT(order_products WHERE product_name = Bag of Organic Bananas)</th>\n",
       "      <th>COUNT(order_products WHERE department = canned goods)</th>\n",
       "      <th>COUNT(order_products WHERE department = produce)</th>\n",
       "      <th>COUNT(order_products WHERE department = deli)</th>\n",
       "      <th>COUNT(order_products WHERE product_name = Large Lemon)</th>\n",
       "      <th>COUNT(order_products WHERE product_name = Organic Whole Milk)</th>\n",
       "      <th>COUNT(order_products WHERE department = dry goods pasta)</th>\n",
       "      <th>user_id</th>\n",
       "      <th>bought_product</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8</td>\n",
       "      <td>58</td>\n",
       "      <td>21</td>\n",
       "      <td>9</td>\n",
       "      <td>34</td>\n",
       "      <td>0.413793</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2015</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>17</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>1007</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>42</td>\n",
       "      <td>22</td>\n",
       "      <td>7</td>\n",
       "      <td>40</td>\n",
       "      <td>0.047619</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2015</td>\n",
       "      <td>...</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1008</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>0.100000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2015</td>\n",
       "      <td>...</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1030</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>101</td>\n",
       "      <td>36</td>\n",
       "      <td>15</td>\n",
       "      <td>58</td>\n",
       "      <td>0.425743</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2015</td>\n",
       "      <td>...</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>1040</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>16</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "      <td>0.187500</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2015</td>\n",
       "      <td>...</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1053</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 60 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   COUNT(orders)  COUNT(order_products)  NUM_UNIQUE(order_products.aisle_id)  \\\n",
       "0              8                     58                                   21   \n",
       "1              4                     42                                   22   \n",
       "2              3                     10                                    8   \n",
       "3              7                    101                                   36   \n",
       "4              4                     16                                   12   \n",
       "\n",
       "   NUM_UNIQUE(order_products.department)  \\\n",
       "0                                      9   \n",
       "1                                      7   \n",
       "2                                      5   \n",
       "3                                     15   \n",
       "4                                      9   \n",
       "\n",
       "   NUM_UNIQUE(order_products.product_name)  \\\n",
       "0                                       34   \n",
       "1                                       40   \n",
       "2                                        9   \n",
       "3                                       58   \n",
       "4                                       13   \n",
       "\n",
       "   PERCENT_TRUE(order_products.reordered) DAY(first_orders_time)  \\\n",
       "0                                0.413793                      1   \n",
       "1                                0.047619                      1   \n",
       "2                                0.100000                      1   \n",
       "3                                0.425743                      1   \n",
       "4                                0.187500                      1   \n",
       "\n",
       "  MONTH(first_orders_time) WEEKDAY(first_orders_time) YEAR(first_orders_time)  \\\n",
       "0                        1                          3                    2015   \n",
       "1                        1                          3                    2015   \n",
       "2                        1                          3                    2015   \n",
       "3                        1                          3                    2015   \n",
       "4                        1                          3                    2015   \n",
       "\n",
       "   ...  COUNT(order_products WHERE department = frozen)  \\\n",
       "0  ...                                                0   \n",
       "1  ...                                                6   \n",
       "2  ...                                                2   \n",
       "3  ...                                                9   \n",
       "4  ...                                                1   \n",
       "\n",
       "   COUNT(order_products WHERE product_name = Bag of Organic Bananas)  \\\n",
       "0                                                  2                   \n",
       "1                                                  0                   \n",
       "2                                                  0                   \n",
       "3                                                  0                   \n",
       "4                                                  0                   \n",
       "\n",
       "   COUNT(order_products WHERE department = canned goods)  \\\n",
       "0                                                  0       \n",
       "1                                                  0       \n",
       "2                                                  0       \n",
       "3                                                  3       \n",
       "4                                                  0       \n",
       "\n",
       "   COUNT(order_products WHERE department = produce)  \\\n",
       "0                                                17   \n",
       "1                                                13   \n",
       "2                                                 1   \n",
       "3                                                 2   \n",
       "4                                                 1   \n",
       "\n",
       "   COUNT(order_products WHERE department = deli)  \\\n",
       "0                                              1   \n",
       "1                                              0   \n",
       "2                                              0   \n",
       "3                                              4   \n",
       "4                                              1   \n",
       "\n",
       "   COUNT(order_products WHERE product_name = Large Lemon)  \\\n",
       "0                                                  0        \n",
       "1                                                  1        \n",
       "2                                                  0        \n",
       "3                                                  0        \n",
       "4                                                  0        \n",
       "\n",
       "   COUNT(order_products WHERE product_name = Organic Whole Milk)  \\\n",
       "0                                                  0               \n",
       "1                                                  0               \n",
       "2                                                  0               \n",
       "3                                                  0               \n",
       "4                                                  0               \n",
       "\n",
       "   COUNT(order_products WHERE department = dry goods pasta)  user_id  \\\n",
       "0                                                  6            1007   \n",
       "1                                                  0            1008   \n",
       "2                                                  0            1030   \n",
       "3                                                 10            1040   \n",
       "4                                                  0            1053   \n",
       "\n",
       "   bought_product  \n",
       "0           False  \n",
       "1            True  \n",
       "2           False  \n",
       "3           False  \n",
       "4            True  \n",
       "\n",
       "[5 rows x 60 columns]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fm = feature_matrix.ww.compute()\n",
    "fm.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we use this feature matrix to build a predictive model, we will first encode any categorical features using `ft.encode_features()`. Note, at this time `ft.encode_features()` does not work with a Dask feature matrix, so we will use the pandas version we computed above.\n",
    "\n",
    "Also note that `encode_features` expects a feature matrix with Woodwork initialized as the first input. To retain Woodwork typing information we called compute through the `.ww` Woodwork accessor above rather than calling it directly on the feature matrix dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of features 62\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COUNT(orders)</th>\n",
       "      <th>COUNT(order_products)</th>\n",
       "      <th>NUM_UNIQUE(order_products.aisle_id)</th>\n",
       "      <th>NUM_UNIQUE(order_products.department)</th>\n",
       "      <th>NUM_UNIQUE(order_products.product_name)</th>\n",
       "      <th>PERCENT_TRUE(order_products.reordered)</th>\n",
       "      <th>MAX(orders.COUNT(order_products))</th>\n",
       "      <th>MAX(orders.NUM_UNIQUE(order_products.aisle_id))</th>\n",
       "      <th>MAX(orders.NUM_UNIQUE(order_products.department))</th>\n",
       "      <th>MAX(orders.NUM_UNIQUE(order_products.product_name))</th>\n",
       "      <th>...</th>\n",
       "      <th>user_id</th>\n",
       "      <th>bought_product</th>\n",
       "      <th>DAY(first_orders_time) = 1</th>\n",
       "      <th>DAY(first_orders_time) is unknown</th>\n",
       "      <th>MONTH(first_orders_time) = 1</th>\n",
       "      <th>MONTH(first_orders_time) is unknown</th>\n",
       "      <th>WEEKDAY(first_orders_time) = 3</th>\n",
       "      <th>WEEKDAY(first_orders_time) is unknown</th>\n",
       "      <th>YEAR(first_orders_time) = 2015</th>\n",
       "      <th>YEAR(first_orders_time) is unknown</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8</td>\n",
       "      <td>58</td>\n",
       "      <td>21</td>\n",
       "      <td>9</td>\n",
       "      <td>34</td>\n",
       "      <td>0.413793</td>\n",
       "      <td>20.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1007</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>42</td>\n",
       "      <td>22</td>\n",
       "      <td>7</td>\n",
       "      <td>40</td>\n",
       "      <td>0.047619</td>\n",
       "      <td>23.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1008</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>0.100000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1030</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>101</td>\n",
       "      <td>36</td>\n",
       "      <td>15</td>\n",
       "      <td>58</td>\n",
       "      <td>0.425743</td>\n",
       "      <td>28.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1040</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>16</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "      <td>0.187500</td>\n",
       "      <td>6.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1053</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 64 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   COUNT(orders)  COUNT(order_products)  NUM_UNIQUE(order_products.aisle_id)  \\\n",
       "0              8                     58                                   21   \n",
       "1              4                     42                                   22   \n",
       "2              3                     10                                    8   \n",
       "3              7                    101                                   36   \n",
       "4              4                     16                                   12   \n",
       "\n",
       "   NUM_UNIQUE(order_products.department)  \\\n",
       "0                                      9   \n",
       "1                                      7   \n",
       "2                                      5   \n",
       "3                                     15   \n",
       "4                                      9   \n",
       "\n",
       "   NUM_UNIQUE(order_products.product_name)  \\\n",
       "0                                       34   \n",
       "1                                       40   \n",
       "2                                        9   \n",
       "3                                       58   \n",
       "4                                       13   \n",
       "\n",
       "   PERCENT_TRUE(order_products.reordered)  MAX(orders.COUNT(order_products))  \\\n",
       "0                                0.413793                               20.0   \n",
       "1                                0.047619                               23.0   \n",
       "2                                0.100000                                4.0   \n",
       "3                                0.425743                               28.0   \n",
       "4                                0.187500                                6.0   \n",
       "\n",
       "   MAX(orders.NUM_UNIQUE(order_products.aisle_id))  \\\n",
       "0                                             15.0   \n",
       "1                                             15.0   \n",
       "2                                              4.0   \n",
       "3                                             24.0   \n",
       "4                                              6.0   \n",
       "\n",
       "   MAX(orders.NUM_UNIQUE(order_products.department))  \\\n",
       "0                                                8.0   \n",
       "1                                                6.0   \n",
       "2                                                3.0   \n",
       "3                                               11.0   \n",
       "4                                                4.0   \n",
       "\n",
       "   MAX(orders.NUM_UNIQUE(order_products.product_name))  ...  user_id  \\\n",
       "0                                               20.0    ...     1007   \n",
       "1                                               23.0    ...     1008   \n",
       "2                                                4.0    ...     1030   \n",
       "3                                               28.0    ...     1040   \n",
       "4                                                6.0    ...     1053   \n",
       "\n",
       "   bought_product  DAY(first_orders_time) = 1  \\\n",
       "0           False                        True   \n",
       "1            True                        True   \n",
       "2           False                        True   \n",
       "3           False                        True   \n",
       "4            True                        True   \n",
       "\n",
       "   DAY(first_orders_time) is unknown  MONTH(first_orders_time) = 1  \\\n",
       "0                              False                          True   \n",
       "1                              False                          True   \n",
       "2                              False                          True   \n",
       "3                              False                          True   \n",
       "4                              False                          True   \n",
       "\n",
       "   MONTH(first_orders_time) is unknown  WEEKDAY(first_orders_time) = 3  \\\n",
       "0                                False                            True   \n",
       "1                                False                            True   \n",
       "2                                False                            True   \n",
       "3                                False                            True   \n",
       "4                                False                            True   \n",
       "\n",
       "   WEEKDAY(first_orders_time) is unknown  YEAR(first_orders_time) = 2015  \\\n",
       "0                                  False                            True   \n",
       "1                                  False                            True   \n",
       "2                                  False                            True   \n",
       "3                                  False                            True   \n",
       "4                                  False                            True   \n",
       "\n",
       "   YEAR(first_orders_time) is unknown  \n",
       "0                               False  \n",
       "1                               False  \n",
       "2                               False  \n",
       "3                               False  \n",
       "4                               False  \n",
       "\n",
       "[5 rows x 64 columns]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fm_encoded, features_encoded = ft.encode_features(fm,\n",
    "                                                  features)\n",
    "\n",
    "print(\"Number of features %s\" % len(features_encoded))\n",
    "fm_encoded.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5. Machine Learning\n",
    "\n",
    "Using the default parameters, we generated dozens of potential features for our prediction problem. With a few simple commands, this feature matrix can be used for machine learning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = fm_encoded.drop([\"user_id\"], axis=1)\n",
    "X = X.fillna(0)\n",
    "y = X.pop(\"bought_product\").astype('bool')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COUNT(orders)</th>\n",
       "      <th>COUNT(order_products)</th>\n",
       "      <th>NUM_UNIQUE(order_products.aisle_id)</th>\n",
       "      <th>NUM_UNIQUE(order_products.department)</th>\n",
       "      <th>NUM_UNIQUE(order_products.product_name)</th>\n",
       "      <th>PERCENT_TRUE(order_products.reordered)</th>\n",
       "      <th>MAX(orders.COUNT(order_products))</th>\n",
       "      <th>MAX(orders.NUM_UNIQUE(order_products.aisle_id))</th>\n",
       "      <th>MAX(orders.NUM_UNIQUE(order_products.department))</th>\n",
       "      <th>MAX(orders.NUM_UNIQUE(order_products.product_name))</th>\n",
       "      <th>...</th>\n",
       "      <th>COUNT(order_products WHERE product_name = Organic Whole Milk)</th>\n",
       "      <th>COUNT(order_products WHERE department = dry goods pasta)</th>\n",
       "      <th>DAY(first_orders_time) = 1</th>\n",
       "      <th>DAY(first_orders_time) is unknown</th>\n",
       "      <th>MONTH(first_orders_time) = 1</th>\n",
       "      <th>MONTH(first_orders_time) is unknown</th>\n",
       "      <th>WEEKDAY(first_orders_time) = 3</th>\n",
       "      <th>WEEKDAY(first_orders_time) is unknown</th>\n",
       "      <th>YEAR(first_orders_time) = 2015</th>\n",
       "      <th>YEAR(first_orders_time) is unknown</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8</td>\n",
       "      <td>58</td>\n",
       "      <td>21</td>\n",
       "      <td>9</td>\n",
       "      <td>34</td>\n",
       "      <td>0.413793</td>\n",
       "      <td>20.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>42</td>\n",
       "      <td>22</td>\n",
       "      <td>7</td>\n",
       "      <td>40</td>\n",
       "      <td>0.047619</td>\n",
       "      <td>23.0</td>\n",
       "      <td>15.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>23.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>10</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>9</td>\n",
       "      <td>0.100000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>101</td>\n",
       "      <td>36</td>\n",
       "      <td>15</td>\n",
       "      <td>58</td>\n",
       "      <td>0.425743</td>\n",
       "      <td>28.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>28.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>16</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>13</td>\n",
       "      <td>0.187500</td>\n",
       "      <td>6.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 62 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   COUNT(orders)  COUNT(order_products)  NUM_UNIQUE(order_products.aisle_id)  \\\n",
       "0              8                     58                                   21   \n",
       "1              4                     42                                   22   \n",
       "2              3                     10                                    8   \n",
       "3              7                    101                                   36   \n",
       "4              4                     16                                   12   \n",
       "\n",
       "   NUM_UNIQUE(order_products.department)  \\\n",
       "0                                      9   \n",
       "1                                      7   \n",
       "2                                      5   \n",
       "3                                     15   \n",
       "4                                      9   \n",
       "\n",
       "   NUM_UNIQUE(order_products.product_name)  \\\n",
       "0                                       34   \n",
       "1                                       40   \n",
       "2                                        9   \n",
       "3                                       58   \n",
       "4                                       13   \n",
       "\n",
       "   PERCENT_TRUE(order_products.reordered)  MAX(orders.COUNT(order_products))  \\\n",
       "0                                0.413793                               20.0   \n",
       "1                                0.047619                               23.0   \n",
       "2                                0.100000                                4.0   \n",
       "3                                0.425743                               28.0   \n",
       "4                                0.187500                                6.0   \n",
       "\n",
       "   MAX(orders.NUM_UNIQUE(order_products.aisle_id))  \\\n",
       "0                                             15.0   \n",
       "1                                             15.0   \n",
       "2                                              4.0   \n",
       "3                                             24.0   \n",
       "4                                              6.0   \n",
       "\n",
       "   MAX(orders.NUM_UNIQUE(order_products.department))  \\\n",
       "0                                                8.0   \n",
       "1                                                6.0   \n",
       "2                                                3.0   \n",
       "3                                               11.0   \n",
       "4                                                4.0   \n",
       "\n",
       "   MAX(orders.NUM_UNIQUE(order_products.product_name))  ...  \\\n",
       "0                                               20.0    ...   \n",
       "1                                               23.0    ...   \n",
       "2                                                4.0    ...   \n",
       "3                                               28.0    ...   \n",
       "4                                                6.0    ...   \n",
       "\n",
       "   COUNT(order_products WHERE product_name = Organic Whole Milk)  \\\n",
       "0                                                  0               \n",
       "1                                                  0               \n",
       "2                                                  0               \n",
       "3                                                  0               \n",
       "4                                                  0               \n",
       "\n",
       "   COUNT(order_products WHERE department = dry goods pasta)  \\\n",
       "0                                                  6          \n",
       "1                                                  0          \n",
       "2                                                  0          \n",
       "3                                                 10          \n",
       "4                                                  0          \n",
       "\n",
       "   DAY(first_orders_time) = 1  DAY(first_orders_time) is unknown  \\\n",
       "0                        True                              False   \n",
       "1                        True                              False   \n",
       "2                        True                              False   \n",
       "3                        True                              False   \n",
       "4                        True                              False   \n",
       "\n",
       "   MONTH(first_orders_time) = 1  MONTH(first_orders_time) is unknown  \\\n",
       "0                          True                                False   \n",
       "1                          True                                False   \n",
       "2                          True                                False   \n",
       "3                          True                                False   \n",
       "4                          True                                False   \n",
       "\n",
       "   WEEKDAY(first_orders_time) = 3  WEEKDAY(first_orders_time) is unknown  \\\n",
       "0                            True                                  False   \n",
       "1                            True                                  False   \n",
       "2                            True                                  False   \n",
       "3                            True                                  False   \n",
       "4                            True                                  False   \n",
       "\n",
       "   YEAR(first_orders_time) = 2015  YEAR(first_orders_time) is unknown  \n",
       "0                            True                               False  \n",
       "1                            True                               False  \n",
       "2                            True                               False  \n",
       "3                            True                               False  \n",
       "4                            True                               False  \n",
       "\n",
       "[5 rows x 62 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1     True\n",
       "2    False\n",
       "3    False\n",
       "4     True\n",
       "Name: bought_product, dtype: bool"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's train a Random Forest and validate using 3-fold cross validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:    4.0s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'AUC 0.78 +/- 0.01'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = RandomForestClassifier(n_estimators=400, n_jobs=-1)\n",
    "scores = cross_val_score(estimator=clf,X=X, y=y, cv=3,\n",
    "                         scoring=\"roc_auc\", verbose=True)\n",
    "\n",
    "\"AUC %.2f +/- %.2f\" % (scores.mean(), scores.std())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see this model predicts the next purchase better than guessing. \n",
    "\n",
    "Next we will identify the top 20 features so we can use them to later perform machine learning on the whole dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1: Feature: COUNT(order_products WHERE product_name = Organic Hass Avocado), 0.066\n",
      "2: Feature: COUNT(order_products WHERE department = snacks), 0.054\n",
      "3: Feature: MAX(orders.COUNT(order_products)), 0.043\n",
      "4: Feature: WEEKDAY(first_orders_time) is unknown, 0.041\n",
      "5: Feature: COUNT(order_products WHERE department = pantry), 0.037\n",
      "6: Feature: YEAR(first_orders_time) = 2015, 0.030\n",
      "7: Feature: MIN(orders.PERCENT_TRUE(order_products.reordered)), 0.027\n",
      "8: Feature: COUNT(order_products WHERE department = bakery), 0.027\n",
      "9: Feature: MIN(orders.NUM_UNIQUE(order_products.aisle_id)), 0.025\n",
      "10: Feature: SUM(orders.NUM_UNIQUE(order_products.product_name)), 0.024\n",
      "11: Feature: PERCENT_TRUE(order_products.reordered), 0.024\n",
      "12: Feature: YEAR(first_orders_time) is unknown, 0.024\n",
      "13: Feature: DAY(first_orders_time) = 1, 0.023\n",
      "14: Feature: MONTH(first_orders_time) is unknown, 0.023\n",
      "15: Feature: NUM_UNIQUE(orders.DAY(order_time)), 0.022\n",
      "16: Feature: MIN(orders.NUM_UNIQUE(order_products.product_name)), 0.022\n",
      "17: Feature: MIN(orders.NUM_UNIQUE(order_products.department)), 0.021\n",
      "18: Feature: NUM_UNIQUE(order_products.product_name), 0.021\n",
      "19: Feature: STD(orders.COUNT(order_products)), 0.021\n",
      "20: Feature: COUNT(order_products), 0.020\n"
     ]
    }
   ],
   "source": [
    "clf.fit(X, y)\n",
    "top_features = utils.feature_importances(clf, features_encoded, n=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To persist these features, we can save them to disk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "ft.save_features(top_features, os.path.join(data_dir, \"top_features.txt\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Auto Modeling\n",
    "Instead of just training a Random Forest, we can use [EvalML](https://evalml.alteryx.com/en/stable/) from Alteryx to build multiple models, compare them, and identify the best one\n",
    "\n",
    "<p align=\"center\">\n",
    "<img width=50% src=\"https://evalml-web-images.s3.amazonaws.com/evalml_horizontal.svg\" alt=\"Featuretools\" />\n",
    "</p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "import evalml\n",
    "from evalml import AutoMLSearch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "automl = AutoMLSearch(X, y, problem_type=\"binary\", objective=\"auc\")\n",
    "automl.search()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With EvalML, we are able to get an AUC of .82. We can view the main features below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>feature</th>\n",
       "      <th>importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NUM_UNIQUE(orders.WEEKDAY(order_time))</td>\n",
       "      <td>0.072730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>MEAN(orders.COUNT(order_products))</td>\n",
       "      <td>0.059999</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>COUNT(order_products WHERE product_name = Bag ...</td>\n",
       "      <td>0.056083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>COUNT(order_products WHERE product_name = Banana)</td>\n",
       "      <td>0.053959</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>COUNT(order_products WHERE department = bevera...</td>\n",
       "      <td>0.032120</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>COUNT(order_products WHERE product_name = Limes)</td>\n",
       "      <td>0.028700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>NUM_UNIQUE(order_products.department)</td>\n",
       "      <td>0.026770</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>MAX(orders.PERCENT_TRUE(order_products.reorder...</td>\n",
       "      <td>0.025788</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>MAX(orders.COUNT(order_products))</td>\n",
       "      <td>0.025641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>SUM(orders.PERCENT_TRUE(order_products.reorder...</td>\n",
       "      <td>0.024423</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             feature  importance\n",
       "0             NUM_UNIQUE(orders.WEEKDAY(order_time))    0.072730\n",
       "1                 MEAN(orders.COUNT(order_products))    0.059999\n",
       "2  COUNT(order_products WHERE product_name = Bag ...    0.056083\n",
       "3  COUNT(order_products WHERE product_name = Banana)    0.053959\n",
       "4  COUNT(order_products WHERE department = bevera...    0.032120\n",
       "5   COUNT(order_products WHERE product_name = Limes)    0.028700\n",
       "6              NUM_UNIQUE(order_products.department)    0.026770\n",
       "7  MAX(orders.PERCENT_TRUE(order_products.reorder...    0.025788\n",
       "8                  MAX(orders.COUNT(order_products))    0.025641\n",
       "9  SUM(orders.PERCENT_TRUE(order_products.reorder...    0.024423"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pipeline = automl.best_pipeline\n",
    "pipeline.fit(X, y)\n",
    "\n",
    "pipeline.feature_importance[0:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Understanding feature engineering in Featuretools\n",
    "\n",
    "Before moving forward, take a look at the features we created. You will see that they are more than just simple transformations of columns in our raw data. Instead, they perform aggregations (and sometimes stacking of aggregations) across the relationships in our dataset. If you're curious how this works, learn about the Deep Feature Synthesis algorithm in our documentation [here](https://featuretools.alteryx.com/en/stable/).\n",
    "\n",
    "DFS is so powerful because with no manual work, the library figured out that historical purchases of bananas are important for predicting future purchases. Additionally, it surfaces that purchasing dairy or eggs and reordering behavior are important features.\n",
    "\n",
    "Even though these features are intuitive, Deep Feature Synthesis will automatically adapt as we change the prediction problem, saving us the time of manually brainstorming and implementing these data transformation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 6. Scale to Full Dataset\n",
    "\n",
    "Now that we have established the most important features, we will repeat the process of creating a feature matrix, using only these features, and then make predictions using our full dataset.\n",
    "\n",
    "To start, we will create a new entityset containing our full dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: instacart_full\n",
       "  DataFrames:\n",
       "    order_products [Rows: Delayed('int-94bedfb4-bc39-4e2d-a056-9e3171aab907'), Columns: 7]\n",
       "    orders [Rows: Delayed('int-9280aeb9-e44f-4380-b32a-1c135cd9140b'), Columns: 3]\n",
       "  Relationships:\n",
       "    order_products.order_id -> orders.order_id"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es = ft.EntitySet(\"instacart_full\")\n",
    "es.add_dataframe(dataframe_name=\"order_products\",\n",
    "                 dataframe=order_products,\n",
    "                 index=\"order_product_id\",\n",
    "                 logical_types=order_products_types,\n",
    "                 time_index=\"order_time\")\n",
    "\n",
    "es.add_dataframe(dataframe_name=\"orders\",\n",
    "                 dataframe=orders,\n",
    "                 index=\"order_id\",\n",
    "                 logical_types=order_types,\n",
    "                 time_index=\"order_time\")\n",
    "\n",
    "es.add_relationship(\"orders\", \"order_id\", \"order_products\", \"order_id\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: instacart_full\n",
       "  DataFrames:\n",
       "    order_products [Rows: Delayed('int-9c22cb15-ceb7-4072-96c6-c14bd3e1c0f9'), Columns: 7]\n",
       "    orders [Rows: Delayed('int-cfcf2f86-e9b0-44ff-b883-ef3d619729d8'), Columns: 3]\n",
       "    users [Rows: Delayed('int-321216b7-d713-4aed-aecd-d24cab4458bc'), Columns: 2]\n",
       "  Relationships:\n",
       "    order_products.order_id -> orders.order_id\n",
       "    orders.user_id -> users.user_id"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es.normalize_dataframe(base_dataframe_name=\"orders\", new_dataframe_name=\"users\", index=\"user_id\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "es.add_last_time_indexes()\n",
    "es[\"order_products\"][\"department\"].interesting_values = ['produce', 'dairy eggs', 'snacks', 'beverages', 'frozen', 'pantry', 'bakery', 'canned goods', 'deli', 'dry goods pasta']\n",
    "es[\"order_products\"][\"product_name\"].interesting_values = ['Banana', 'Bag of Organic Bananas', 'Organic Baby Spinach', 'Organic Strawberries', 'Organic Hass Avocado', 'Organic Avocado', 'Large Lemon', 'Limes', 'Strawberries', 'Organic Whole Milk']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use our previously defined Compose label maker to create cutoff times for the full dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = denormalize(es).compute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed: 55:30 | Remaining: 00:00 | Progress: 100%|███████████████████████████████████████████████████████████| user_id: 262420/262420 \n"
     ]
    }
   ],
   "source": [
    "label_times = lm.search(\n",
    "    df.sort_values('order_time'),\n",
    "    minimum_data='2015-03-15',\n",
    "    num_examples_per_instance=2,\n",
    "    product_name='Banana',\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>time</th>\n",
       "      <th>bought_product</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>2015-07-05</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2015-08-02</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>7</td>\n",
       "      <td>2015-08-02</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10</td>\n",
       "      <td>2015-04-12</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13</td>\n",
       "      <td>2015-03-15</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id       time  bought_product\n",
       "0        1 2015-07-05           False\n",
       "1        2 2015-08-02            True\n",
       "2        7 2015-08-02           False\n",
       "3       10 2015-04-12           False\n",
       "4       13 2015-03-15           False"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "label_times.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we will read in the top 20 features we identified previously and calculate a feature matrix using only these features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<Feature: COUNT(order_products WHERE product_name = Organic Hass Avocado)>,\n",
       " <Feature: COUNT(order_products WHERE department = snacks)>,\n",
       " <Feature: MAX(orders.COUNT(order_products))>,\n",
       " <Feature: WEEKDAY(first_orders_time) is unknown>,\n",
       " <Feature: COUNT(order_products WHERE department = pantry)>,\n",
       " <Feature: YEAR(first_orders_time) = 2015>,\n",
       " <Feature: MIN(orders.PERCENT_TRUE(order_products.reordered))>,\n",
       " <Feature: COUNT(order_products WHERE department = bakery)>,\n",
       " <Feature: MIN(orders.NUM_UNIQUE(order_products.aisle_id))>,\n",
       " <Feature: SUM(orders.NUM_UNIQUE(order_products.product_name))>,\n",
       " <Feature: PERCENT_TRUE(order_products.reordered)>,\n",
       " <Feature: YEAR(first_orders_time) is unknown>,\n",
       " <Feature: DAY(first_orders_time) = 1>,\n",
       " <Feature: MONTH(first_orders_time) is unknown>,\n",
       " <Feature: NUM_UNIQUE(orders.DAY(order_time))>,\n",
       " <Feature: MIN(orders.NUM_UNIQUE(order_products.product_name))>,\n",
       " <Feature: MIN(orders.NUM_UNIQUE(order_products.department))>,\n",
       " <Feature: NUM_UNIQUE(order_products.product_name)>,\n",
       " <Feature: STD(orders.COUNT(order_products))>,\n",
       " <Feature: COUNT(order_products)>]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_features= ft.load_features(os.path.join(data_dir, \"top_features.txt\"))\n",
    "top_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Having read in the top features we want to use, we can now create our feature matrix on the full dataset with a call to `ft.calculate_feature_matrix()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Elapsed: 00:14 | Progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████\n"
     ]
    }
   ],
   "source": [
    "fm = ft.calculate_feature_matrix(top_features, entityset=es, cutoff_time=label_times, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will compute our feature matrix to bring the results into memory, allowing us to encode categorical features and make our predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "fm = fm.ww.compute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of features 20\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>COUNT(order_products WHERE product_name = Organic Hass Avocado)</th>\n",
       "      <th>COUNT(order_products WHERE department = snacks)</th>\n",
       "      <th>MAX(orders.COUNT(order_products))</th>\n",
       "      <th>WEEKDAY(first_orders_time) is unknown</th>\n",
       "      <th>COUNT(order_products WHERE department = pantry)</th>\n",
       "      <th>YEAR(first_orders_time) = 2015</th>\n",
       "      <th>MIN(orders.PERCENT_TRUE(order_products.reordered))</th>\n",
       "      <th>COUNT(order_products WHERE department = bakery)</th>\n",
       "      <th>MIN(orders.NUM_UNIQUE(order_products.aisle_id))</th>\n",
       "      <th>SUM(orders.NUM_UNIQUE(order_products.product_name))</th>\n",
       "      <th>...</th>\n",
       "      <th>DAY(first_orders_time) = 1</th>\n",
       "      <th>MONTH(first_orders_time) is unknown</th>\n",
       "      <th>NUM_UNIQUE(orders.DAY(order_time))</th>\n",
       "      <th>MIN(orders.NUM_UNIQUE(order_products.product_name))</th>\n",
       "      <th>MIN(orders.NUM_UNIQUE(order_products.department))</th>\n",
       "      <th>NUM_UNIQUE(order_products.product_name)</th>\n",
       "      <th>STD(orders.COUNT(order_products))</th>\n",
       "      <th>COUNT(order_products)</th>\n",
       "      <th>user_id</th>\n",
       "      <th>bought_product</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>10.0</td>\n",
       "      <td>False</td>\n",
       "      <td>2</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12</td>\n",
       "      <td>4.0</td>\n",
       "      <td>56.0</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>6</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>23</td>\n",
       "      <td>2.048034</td>\n",
       "      <td>56</td>\n",
       "      <td>13</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>False</td>\n",
       "      <td>3</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>6.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>4</td>\n",
       "      <td>9.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>36</td>\n",
       "      <td>1.414214</td>\n",
       "      <td>40</td>\n",
       "      <td>23</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>15.0</td>\n",
       "      <td>False</td>\n",
       "      <td>3</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>8.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>3</td>\n",
       "      <td>9.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>30</td>\n",
       "      <td>3.055050</td>\n",
       "      <td>37</td>\n",
       "      <td>44</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>17.0</td>\n",
       "      <td>False</td>\n",
       "      <td>5</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>5</td>\n",
       "      <td>4.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>42</td>\n",
       "      <td>5.594640</td>\n",
       "      <td>48</td>\n",
       "      <td>74</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>18.0</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "      <td>True</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>30.0</td>\n",
       "      <td>...</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>27</td>\n",
       "      <td>7.106335</td>\n",
       "      <td>30</td>\n",
       "      <td>78</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 22 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   COUNT(order_products WHERE product_name = Organic Hass Avocado)  \\\n",
       "0                                                  0                 \n",
       "1                                                  0                 \n",
       "2                                                  0                 \n",
       "3                                                  0                 \n",
       "4                                                  0                 \n",
       "\n",
       "   COUNT(order_products WHERE department = snacks)  \\\n",
       "0                                                2   \n",
       "1                                                0   \n",
       "2                                                2   \n",
       "3                                                1   \n",
       "4                                                0   \n",
       "\n",
       "   MAX(orders.COUNT(order_products))  WEEKDAY(first_orders_time) is unknown  \\\n",
       "0                               10.0                                  False   \n",
       "1                               12.0                                  False   \n",
       "2                               15.0                                  False   \n",
       "3                               17.0                                  False   \n",
       "4                               18.0                                  False   \n",
       "\n",
       "   COUNT(order_products WHERE department = pantry)  \\\n",
       "0                                                2   \n",
       "1                                                3   \n",
       "2                                                3   \n",
       "3                                                5   \n",
       "4                                                1   \n",
       "\n",
       "   YEAR(first_orders_time) = 2015  \\\n",
       "0                            True   \n",
       "1                            True   \n",
       "2                            True   \n",
       "3                            True   \n",
       "4                            True   \n",
       "\n",
       "   MIN(orders.PERCENT_TRUE(order_products.reordered))  \\\n",
       "0                                                0.0    \n",
       "1                                                0.0    \n",
       "2                                                0.0    \n",
       "3                                                0.0    \n",
       "4                                                0.0    \n",
       "\n",
       "   COUNT(order_products WHERE department = bakery)  \\\n",
       "0                                               12   \n",
       "1                                                3   \n",
       "2                                                1   \n",
       "3                                                0   \n",
       "4                                                3   \n",
       "\n",
       "   MIN(orders.NUM_UNIQUE(order_products.aisle_id))  \\\n",
       "0                                              4.0   \n",
       "1                                              6.0   \n",
       "2                                              8.0   \n",
       "3                                              3.0   \n",
       "4                                              1.0   \n",
       "\n",
       "   SUM(orders.NUM_UNIQUE(order_products.product_name))  ...  \\\n",
       "0                                               56.0    ...   \n",
       "1                                               40.0    ...   \n",
       "2                                               37.0    ...   \n",
       "3                                               48.0    ...   \n",
       "4                                               30.0    ...   \n",
       "\n",
       "   DAY(first_orders_time) = 1  MONTH(first_orders_time) is unknown  \\\n",
       "0                        True                                False   \n",
       "1                        True                                False   \n",
       "2                        True                                False   \n",
       "3                        True                                False   \n",
       "4                        True                                False   \n",
       "\n",
       "   NUM_UNIQUE(orders.DAY(order_time))  \\\n",
       "0                                   6   \n",
       "1                                   4   \n",
       "2                                   3   \n",
       "3                                   5   \n",
       "4                                   4   \n",
       "\n",
       "   MIN(orders.NUM_UNIQUE(order_products.product_name))  \\\n",
       "0                                                4.0     \n",
       "1                                                9.0     \n",
       "2                                                9.0     \n",
       "3                                                4.0     \n",
       "4                                                1.0     \n",
       "\n",
       "   MIN(orders.NUM_UNIQUE(order_products.department))  \\\n",
       "0                                                2.0   \n",
       "1                                                6.0   \n",
       "2                                                7.0   \n",
       "3                                                2.0   \n",
       "4                                                1.0   \n",
       "\n",
       "   NUM_UNIQUE(order_products.product_name)  STD(orders.COUNT(order_products))  \\\n",
       "0                                       23                           2.048034   \n",
       "1                                       36                           1.414214   \n",
       "2                                       30                           3.055050   \n",
       "3                                       42                           5.594640   \n",
       "4                                       27                           7.106335   \n",
       "\n",
       "   COUNT(order_products)  user_id  bought_product  \n",
       "0                     56       13           False  \n",
       "1                     40       23           False  \n",
       "2                     37       44           False  \n",
       "3                     48       74           False  \n",
       "4                     30       78           False  \n",
       "\n",
       "[5 rows x 22 columns]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fm_encoded, features_encoded = ft.encode_features(fm, top_features)\n",
    "\n",
    "print(\"Number of features %s\" % len(features_encoded))\n",
    "fm_encoded.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = fm_encoded.drop([\"user_id\"], axis=1)\n",
    "X = X.fillna(0)\n",
    "y = X.pop(\"bought_product\").astype('bool')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:   25.0s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'AUC 0.67 +/- 0.02'"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = RandomForestClassifier(n_estimators=400, n_jobs=-1)\n",
    "scores = cross_val_score(estimator=clf,X=X, y=y, cv=3,\n",
    "                         scoring=\"roc_auc\", verbose=True)\n",
    "\n",
    "\"AUC %.2f +/- %.2f\" % (scores.mean(), scores.std())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1: Feature: STD(orders.COUNT(order_products)), 0.120\n",
      "2: Feature: PERCENT_TRUE(order_products.reordered), 0.113\n",
      "3: Feature: NUM_UNIQUE(order_products.product_name), 0.092\n",
      "4: Feature: COUNT(order_products), 0.092\n",
      "5: Feature: SUM(orders.NUM_UNIQUE(order_products.product_name)), 0.090\n",
      "6: Feature: COUNT(order_products WHERE department = snacks), 0.081\n",
      "7: Feature: MAX(orders.COUNT(order_products)), 0.080\n",
      "8: Feature: COUNT(order_products WHERE department = pantry), 0.073\n",
      "9: Feature: COUNT(order_products WHERE department = bakery), 0.064\n",
      "10: Feature: NUM_UNIQUE(orders.DAY(order_time)), 0.061\n",
      "11: Feature: MIN(orders.NUM_UNIQUE(order_products.product_name)), 0.044\n",
      "12: Feature: MIN(orders.NUM_UNIQUE(order_products.aisle_id)), 0.034\n",
      "13: Feature: COUNT(order_products WHERE product_name = Organic Hass Avocado), 0.030\n",
      "14: Feature: MIN(orders.NUM_UNIQUE(order_products.department)), 0.027\n",
      "15: Feature: WEEKDAY(first_orders_time) is unknown, 0.000\n",
      "16: Feature: YEAR(first_orders_time) = 2015, 0.000\n",
      "17: Feature: MIN(orders.PERCENT_TRUE(order_products.reordered)), 0.000\n",
      "18: Feature: YEAR(first_orders_time) is unknown, 0.000\n",
      "19: Feature: DAY(first_orders_time) = 1, 0.000\n",
      "20: Feature: MONTH(first_orders_time) is unknown, 0.000\n"
     ]
    }
   ],
   "source": [
    "clf.fit(X, y)\n",
    "top_features = utils.feature_importances(clf, top_features, n=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have finished, we can close our Dask client."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "While this is an end-to-end example of going from raw data to a trained machine learning model, it is necessary to do further exploration before claiming we've built something impactful.\n",
    "\n",
    "Fortunately, Featuretools makes it easy to build structured data science pipeline. As a next steps, you could\n",
    "\n",
    "- Further validate these results by creating feature vectors at different cutoff times\n",
    "- Perform feature selection on a larger subset of the original data to improve results\n",
    "- Define other prediction problems for this dataset (you can even change the dataframe you are making predictions on!)\n",
    "- Save feature matrices to disk as CSVs so they can be reused with different problems without recalculating\n",
    "- Experiment with parameters to Deep Feature Synthesis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Built at Alteryx Innovation Labs\n",
    "\n",
    "<p>\n",
    "<a href=\"https://www.alteryx.com/innovation-labs\">\n",
    "    <img width=\"75%\" src=\"https://evalml-web-images.s3.amazonaws.com/alteryx_innovation_labs.png\" alt=\"Alteryx Innovation Labs\" />\n",
    "</a>\n",
    "</p>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}