GAMENet: Graph Augmented MEmory Networks for Recommending Medication Combination

As mentioned before, a visit $\bm{x}_{t}$ consists of $[\bm{c}_{d}^{t},\bm{c}_{p}^{t},\bm{c}_{m}^{t}]$ where each of $\bm{c}_{\ast}^{t}$ is a multi-hot vector at the $t^{th}$ visit. The multi-hot vector $\bm{c}_{\ast}^{t}$ is binary encoded showing the existence of each medical codes recorded at the $t^{th}$ visit. Like [\citeauthoryearChoi et al.2016b] used a linear embedding of the input vector, we derive medical embeddings for $\bm{c}_{d}^{t},\bm{c}_{p}^{t}$ separately at the $t^{th}$ visit as follows:

where $\bm{W}_{\ast,e}\in\mathbb{R}^{|\mathcal{C}_{\ast}|\times d}$ is the embedding matrix to learn. Thus a visit $\bm{x}_{t}$ is transformed to $\hat{\bm{x}}_{t}=[\bm{e}_{d}^{t},\bm{e}_{p}^{t},\bm{c}_{m}^{t}]$.

To enable personalized medication recommendation which is tailored using patient EHR data, we design a Dual-RNN to learn patient representations from multimodal EHR data where each RNN encodes only one type of medical codes. The reason is that it is quite possible for a clinical visit to have missing modality (e.g. only diagnosis modality without procedure). Because of that, we model diagnosis and procedure modalities separately using two RNNs. Formally, for each input vector in transformed clinical history $[\hat{\bm{x}_{1}},\hat{\bm{x}}_{2},\cdots,\hat{\bm{x}}_{t}]$, we retrieve $\bm{e}_{m},\bm{e}_{p}$ and utilize RNN to encode visit-level diagnosis and procedure embeddings respectively as follows:

Thus, the RNNs accept all patient history visit medical embeddings $\{\bm{e}_{\ast}^{t^{\prime}}\}(t^{\prime}\leq t)$ to produce hidden states $\bm{h}_{\ast}^{t}$ for further generating query (a.k.a. patient representation) in Eq. 3.

To leverage drug knowledge, we construct a graph augmented memory module that not only embeds and stores the EHR graph and the DDI graph as facts in Memory Bank (MB), but also inserts patient history to Dynamic Memory (DM) key-value form to fully capture the information from different views. Inspired by  [\citeauthoryearWeston, Chopra, and Bordes2015], four memory components I, G, O, R are proposed which mimics the architecture of modern computer in some way:

• •

  I: Input memory representation converts inputs into query for memory reading. Here we can use hidden states from Dual-RNN to generate query as follows:

  $\displaystyle\bm{q}^{t}=f([\bm{h}_{d}^{t},\bm{h}_{p}^{t}])$

  (3)

  where we concatenate hidden diagnosis state $\bm{h}_{d}^{t}$ and procedure state $\bm{h_{p}}^{t}$ as the input patient health state. $f(\cdot)$ is the transform function which projects hidden states to query and is implemented as single hidden layer fully connected neural network.

• •

  G: Generalization is the process of generating and updating the memory representation. We design the memory module by storing graph augmented memory representation as facts in Memory Bank (MB) and insert patient history to Dynamic Memory (DM) as key-value pairs to fully capture the information from different view.

  For Memory Bank (MB) $\bm{M}_{b}$, two adjacency matrices $\bm{A}_{e},\bm{A}_{d}$ are used. Following the GCN procedure  [\citeauthoryearKipf and Welling2017], each $\bm{A}_{\ast}$ is preprocessed as follows:

  $\displaystyle\tilde{\bm{A}}_{\ast}=\tilde{\bm{D}}^{-\frac{1}{2}}(\bm{A}_{\ast}+\bm{I})\tilde{\bm{D}}^{-\frac{1}{2}}$

  (4)

  where $\tilde{\bm{D}}$ is a diagonal matrix such that $\tilde{\bm{D}}_{ii}=\sum_{j}\bm{A}_{ij}$ and $\bm{I}$ are identity matrices.

  Then we applied a two-layer GCN on each graph to learn improved embeddings on drug combination usage and DDIs respectively. The output $\bm{M}_{b}$ is generated as a weighted sum of the two graph embeddings.

  		$\displaystyle\bm{Z}_{1}$	$\displaystyle=\tilde{\bm{A}}_{e}\text{tanh}(\tilde{\bm{A}}_{e}\bm{W}_{e1})\bm{W}_{1}$		(5)
  		$\displaystyle\bm{Z}_{2}$	$\displaystyle=\tilde{\bm{A}}_{d}\text{tanh}(\tilde{\bm{A}}_{d}\bm{W}_{e2})\bm{W}_{2}$
  		$\displaystyle\bm{M}_{b}$	$\displaystyle=\bm{Z}_{1}-\beta\bm{Z}_{2}$

  where $\bm{W}_{e1}$, $\bm{W}_{e2}\in\mathbb{R}^{|\mathcal{C}_{m}|\times d}$ are medication embeddings from EHR graph and DDI graph (each contains $|\mathcal{C}_{m}|$ number of d-dimensional vectors), $\bm{W}_{1}$, $\bm{W}_{2}\in\mathbb{R}^{d\times d}$ are hidden weight parameter matrices. All $\bm{W}_{\ast}$ are updated during training phase. Then, graph node embeddings $\bm{Z}_{1}$, $\bm{Z}_{2}\in\mathbb{R}^{|\mathcal{C}_{m}|\times d}$ are generated using GCN. Finally we combine different node embeddings as Memory Bank $\bm{M}_{b}\in\mathbb{R}^{|\mathcal{C}_{m}|\times d}$ where $\beta$ is a weighting variable to fuse different knowledge graphs.

  For Dynamic Memory (DM) $\bm{M}_{d}^{t}$, the combined patient $\{\bm{q}^{t^{\prime}}\}(t^{\prime}<t)$ (the keys) associated with corresponding multi-hot medication vector $\{\bm{c}_{m}^{t^{\prime}}\}$ (the values) are inserted into DM as key-value pairs. This kind of design provides a way to locate most similar patient representation over time and retrieve the proper weighted medications set. Specifically, we can incrementally insert key-value pair after each visit step and treat $\bm{M}_{d}^{t}$ as a vectorized indexable dictionary as follows:

  $$\bm{M}_{d}^{t}=\{\bm{q}^{t^{\prime}}\colon\bm{c}_{m}^{t^{\prime}}\}_{1}^{t-1}$$

  (6)

  where $\bm{M}_{d}^{t}$ is empty when $t=1$. For clarity, we use $\bm{M}_{d,k}^{t}=[\bm{q}^{1};\bm{q}^{2};\cdots;\bm{q}^{t-1}]\in\mathbb{R}^{|t-1|\times d}$ to denote the key vectors and $\bm{M}_{d,v}^{t}=[\bm{c}_{m}^{1};\bm{c}_{m}^{2};\cdots;\bm{c}_{m}^{t-1}]\in\mathbb{R}^{|t-1|\times|\mathcal{C}_{m}|}$ to denote the value vectors at $t^{th}$ visit.

• •

  O: Output memory representation produces outputs $\bm{o}_{b}^{t}$ and $\bm{o}_{d}^{t}$ given the patient representation $\bm{q}^{t}$ (the query) and the current memory state $\bm{M}_{b},\bm{M}_{d}^{t}$. Here, we apply attention based reading procedure to retrieve most relevant information with respect to query $\bm{q}^{t}$ as outputs $\bm{o}_{b}^{t},\bm{o}_{d}^{t}$ as follows:

  		$\displaystyle\bm{o}_{b}^{t}$	$\displaystyle=\bm{M}_{b}^{\intercal}\overbrace{\text{Softmax}(\bm{M}_{b}\bm{q}^{t})}^{\bm{a}_{c}^{t}}$		(7)
  		$\displaystyle\bm{o}_{d}^{t}$	$\displaystyle=\bm{M}_{b}^{\intercal}\overbrace{({\bm{M}_{d,v}^{t}})^{\intercal}\underbrace{\text{Softmax}(\bm{M}_{d,k}^{t}\bm{q}^{t})}_{\bm{a}_{s}^{t}}}^{\bm{a}_{m}^{t}}$

  where $\bm{o}_{b}^{t}\in\mathbb{R}^{d}$ is directly retrieved using content-attention $\bm{a}_{c}^{t}$ based on similarity between patient representation (query) and facts in $\bm{M}_{b}$.

  For $\bm{o}_{d}^{t}\in\mathbb{R}^{d}$, it firstly considers similar patient representation from patient history records $\bm{M}_{d,k}^{t}$ with temporal-attention $\bm{a}_{s}^{t}$. Then $\bm{a}_{s}^{t}$ is utilized to generate history medication distribution $\bm{a}_{m}^{t}$ by weighted sum of history multi-hot medication in $\bm{M}_{d,v}^{t}$. Finally, we can get $\bm{o}_{d}^{t}$ by further retrieved information from $\bm{M}_{b}$ using $\bm{a}_{m}^{t}$ from temporal aspect.

  In addition, the attention based reading procedure makes the model differentiable so that it can be updated end-to-end using back propagation.

• •

  R: Response is the final step to utilize patient representation and memory output to predict the multi-label medication as follows:

  $\displaystyle\hat{\bm{y}}_{t}=\sigma([\bm{q}^{t},\bm{o}_{b}^{t},\bm{o}_{d}^{t}])$

  (8)

  where $\sigma$ is the sigmoid function.

We used EHR data from MIMIC-III [\citeauthoryearJohnson et al.2016]. Here we select a cohort where patients have more than one visit. In practice, if we use all the drug codes in an EMR record, the medication set can be very large, each day in hospital, the doctor can prescribe several types of medications for the patient. Hence, we choose the set of medications prescribed by doctors during the first 24-hour as the first 24-hour is often the most critical time for patients to obtain correct treatment quickly. In addition, we used DDI knowledge from TWOSIDES dataset [\citeauthoryearTatonetti et al.2012a]. In this work, we keep the Top-40 severity DDI types and transform the drug coding from NDC to ATC Third Level for integrating with MIMIC-III. The statistics of the datasets are summarized in Table 2.

We consider the following baseline algorithms.

• •

  Nearest will simply recommend the same combination medications at previous visit for current visit (i.e., $\hat{Y}_{t}=Y_{t-1}$)

• •

  Logistic Regression (LR) is a logistic regression with L2 regularization. Here we represent the input data by sum of one-hot vector. Binary relevance technique  [\citeauthoryearLuaces et al.2012] is used to handle multi-label output.

• •

  Leap [\citeauthoryearZhang et al.2017] is an instance-based medication combination recommendation method.

• •

  RETAIN [\citeauthoryearChoi et al.2016b] can provide sequential prediction of medication combination based on a two-level neural attention model that detects influential past visits and significant clinical variables within those visits.

• •

  DMNC [\citeauthoryearLe, Tran, and Venkatesh2018] is a recent work of medication combination prediction via memory augmented neural network based on differentiable neural computers (DNC) [\citeauthoryearGraves et al.2016].

To measure the prediction accuracy, we used Jaccard Similarity Score (Jaccard), Average F1 (F1) and Precision Recall AUC (PRAUC). Jaccard is defined as the size of the intersection divided by the size of the union of ground truth medications $Y_{t}^{(k)}$ and predicted medications $\hat{Y}_{t}^{(k)}$.

where $N$ is the number of patients in test set and $T_{k}$ is the number of visits of the $k^{th}$ patient. Average Precision (Avg-P) and Average Recall (Avg-R), and F1 are defined as:

where $t$ means the $t^{th}$ visit and $k$ means the $k^{th}$ patient in test dataset.

To measure medication safety, we define DDI Rate as percentage of medication recommendation that contain DDIs.

where the set will count each medication pair $(c_{i},c_{j})$ in recommendation set $\hat{Y}$ if the pair belongs to edge set $\mathcal{E}_{d}$ of the DDI graph. Here $N$ is the size of test dataset and $T_{k}$ is the number of visits of the $k^{th}$ patient.

The relative DDI Rate ($\triangle$ DDI Rate %) is defined as the percentage of DDI rate change compared to DDI rate in EHR test dataset given DDI rate of the algorithm:

We randomly divide the dataset into training, validation and testing set in a $2/3:1/6:1/6$ ratio. For LR, we use the grid search technique over typical range of hyper-parameter to search the best hyperparameter values which result in L2 norm penalty with weight as 1.1. For our methods, the hyperparameters are adjusted on evaluation set which result in expected DDI Rate $s$ as 0.05, initial temperature $Temp$ as 0.5, weight decay $\epsilon$ as 0.85 and mixture weights $\bm{\pi}=[0.9,0.1]$. For all deep learning based methods, we choose a gated recurrent unit (GRU) [\citeauthoryearCho et al.2014] as the implementation of RNN and utilize dropout [\citeauthoryearSrivastava et al.2014] with probability of an element to be zeroed as 0.4 on the output of embeddings. The embedding size and hidden layer size for GRU is set as 64 and 64 respectively, word and memory size for DMNC model is 64 and 16 which is the same as [\citeauthoryearLe, Tran, and Venkatesh2018]. Training is done through Adam [\citeauthoryearKingma and Ba2014] at learning rate 0.0002. We fix the best model on evaluation set within 40 epochs and report the performance in test set.