The Thirty-First AAAI Conference on Innovative Applications of Artificial Intelligence (IAAI-19)

DEFSI: Deep Learning Based Epidemic Forecasting with Synthetic Information

Lijing Wang,1 Jiangzhuo Chen,2 Madhav Marathe2,3

1
Computer Science, Virginia Tech
2
Biocomplexity Institute & Initiative, University of Virginia
3
Computer Science, University of Virginia
[email protected],{chenj,marathe}@virginia.edu

Abstract tion as the surveillance data; and high-resolution forecast-

ing to denote the forecasting with a higher geographical res-
Influenza-like illness (ILI) is among the most common dis- olution than provided in surveillance data. To be concrete,
eases worldwide. Producing timely, well-informed, and reli-
able forecasts for ILI is crucial for preparedness and optimal
in this paper flat-resolution means state level while high-
interventions. In this work, we focus on short-term but high- resolution means county level, since the highest resolution
resolution forecasting and propose DEFSI (Deep Learning CDC surveillance data is at state level.
Based Epidemic Forecasting with Synthetic Information), an A challenge in high-resolution disease forecasting is
epidemic forecasting framework that integrates the strengths the lack of surveillance data at a finer spatial scale. The
of artificial neural networks and causal methods. In DEFSI, data driven methods suffer from this limitation, includ-
we build a two-branch neural network structure to take both ing statistical methods such as ARIMA (Benjamin, Rigby,
within-season observations and between-season observations and Stasinopoulos 2003), ARGO (Yang, Santillana, and
as features. The model is trained on geographically high-
Kou 2015; Yang et al. 2017) and artificial neural net-
resolution synthetic data. It enables detailed forecasting when
high-resolution surveillance data is not available. Further- works (ANN) methods such as Long Short Term Memory
more, the model is provided with better generalizability and (LSTM) (Volkova et al. 2017; Venna et al. 2017; Wu et al.
physical consistency. Our method achieves comparable/better 2018). Even for a few states where county level surveillance
performance than state-of-the-art methods for short-term ILI data is available, training ANN methods for them is diffi-
forecasting at the state level. For high-resolution forecasting cult due to the small size of the data. Causal methods have
at the county level, DEFSI significantly outperforms the other recently been introduced to enable high-resolution forecast-
methods. ing (Yang, Karspeck, and Shaman 2014; Nsoesie et al. 2013;
Zhao et al. 2015). They estimate the parameters of the un-
Introduction derlying disease model from the surveillance data. Then ILI
incidence prediction is made from the output of simulations
Influenza-like illness (ILI) poses a serious threat to global using the identified disease model. Calibrating these models
public health. Worldwide seasonal influenza causes three to is challenging due to the high dimensionality of the param-
five million cases of severe illness and 290,000 to 650,000 eter space.
deaths annually. Traditionally, ILI surveillance data from the Another challenge to the epidemic forecasting domain is
Centers for Disease Control and Prevention (CDC) has been the generalization and physical consistency of data driven
used as reference data to predict future ILI incidence. The models due to the paucity of representative samples in na-
surveillance data is provided with a coarse resolution and ture. Meanwhile it is difficult for causal models to repre-
is usually updated regularly. For example, in the USA it sent complex processes that are not conceptually well under-
has been provided previously at HHS region level and re- stood using known scientific knowledge. This motivates our
cently at state level. Considering the heterogeneity between work to address the shortcomings of data driven and causal
different subregion locations and populations, accurate pre- models by using synthetic data generated by epidemiolog-
dictions with a finer resolution, e.g. at county level in the ical theories in conjunction with observed data to improve
USA, is crucial for local public health decision making, op- epidemic forecasts. In a recent independent effort, Karpatne
timal mitigation resource allocation among subregions, as et al. (Karpatne et al. 2017) formally conceptualized the
well as household or individual level preventive actions in- paradigm of theory-guided data science (TGDS) that seeks
formed by neighboring prevalence. We focus on the problem to exploit the promise of data science without ignoring the
of high-resolution ILI incidence forecasting based on ILI treasure of knowledge accumulated in scientific principles.
surveillance data of coarse resolution. Our work starts from a similar perspective but is different in
In this paper we use flat-resolution forecasting to de- that multi-agent models enable high-resolution forecasting
note the forecasting of ILI incidence with the same resolu- and better generalizability of our models. This is important
Copyright c 2019, Association for the Advancement of Artificial due to behavioral uncertainty of a social technical system.
Intelligence (www.aaai.org). All rights reserved. To address the above challenges, we propose a novel epi-

9607
 Problem Setup
Given an observed time-series of weekly ILI incidence for
a specific region, we focus on predicting ILI incidence for
both the region and its subregions in short-term. Without
loss of generality, in this paper we consider making predic-
tions for a state of the USA and all counties of the state,
using CDC state level ILI incidence data (CDC 2018a).
In this setting, state level forecasting is flat-resolution,
while county level forecasting is high-resolution. Let y =
hy1 , y2 , · · · , yT , · · · i denote the sequence of weekly state
level ILI incidence, where T is the last week of which the ILI
Figure 1: DEFSI framework. In this framework, a region-
incidence is given. Similarly, yC = hy1C , y2C , · · · , yTC , · · · i
specific disease parameter space is constructed based on
denotes the sequence of weekly ILI incidence for a partic-
surveillance data. Then the synthetic training data consist-
ular county C within the state. Assume that there are K
ing of both state level and county level weekly ILI incidence
counties D = {C1 , C2 , · · · , CK } in the state. Let ytD =
curves is generated by simulations parameterized by sam-
{ytC |C ∈ D} denote ILI incidence of all counties in the
ples from the parameter space. A two-branch deep neural
state at week t. TheP consistency constraint on county level
network model is trained on the synthetic data. The trained
incidence is yt = ytC . The problem is defined as pre-
model takes surveillance data as the input during forecast- C∈D
ing. dicting both state level and county level incidence at week t,
where t = T + 1, denoted as zt = (yt , ytD ), given historical
state level incidence.
demic forecasting framework, called Deep Learning Based
Epidemic Forecasting with Synthetic Information (DEFSI). DEFSI
It combines multi-agent system and deep neural network Framework DEFSI framework consists of three major
techniques from artificial intelligence (AI). The idea is to components (shown in Fig. 1): (1) Disease model parameter
model the non-linear relationship between the past higher space construction: Given an existing disease simulator, we
level (state) ILI incidences and the future higher (state) and estimate a marginal distribution for each model parameter
lower level (county) ILI incidences with a deep neural net- based on the state-specific surveillance data; (2) Synthetic
work. The novelty of our approach is to generate the train- training data generation: We generate a synthetic training
ing data using a multi-agent simulation that is based on a dataset at both flat-resolution and high-resolution scales for
synthetic population and contact network, where agent het- that state by running simulations parameterized from the pa-
erogeneities and unstructured interactions among agents are rameter space; (3) Deep neural network training and fore-
modeled. With a multi-agent model, individual or household casting: We design a two-branch deep neural network model
level behavior can also be modeled as well as the public trained on the synthetic training dataset and using surveil-
health intervention measures changing the disease dynam- lance data as inputs for predictions. We will elaborate the
ics. The purpose is to use simulations to create training data details in the following subsections.
as similar as possible to the surveillance data observed in the
real world. SEIR-based Epidemic Simulation The SEIR disease
To the best of our knowledge, DEFSI is the first to com- model is widely used for ILI diseases (Kuznetsov and Pic-
bine a realistic multi-agent model with deep learning for cardi 1994). Each person is in one of the following four
epidemic forecasting. Our major contributions are as fol- health states at any time: susceptible (S), exposed (E), in-
lows: (1) DEFSI enables accurate high-resolution forecast- fectious (I), recovered or removed (R). A person v is in the
ing with flat-resolution observations as inputs. (2) DEFSI susceptible state until he becomes exposed. If v becomes
proposes a two-branch neural network model for ILI fore- exposed, he remains so for pE (v) days, which is called
casting. It combines within-season observations (observed the incubation period, during which he is not infectious.
data points of the current season that characterize the on- Then he becomes infectious and remains so for pI (v) days,
going epidemic) and between-season historical observations which is called the infectious period. Finally he becomes re-
(observed data points from similar weeks of the past seasons moved (or recovered) and remains so permanently. While
that characterize general trends around the current week). (3) the SEIR model characterizes within-host disease progres-
DEFSI constructs region-specific training datasets at multi- sion, between-host disease propagation is modeled by trans-
ple spatially fine-grained scale with low costs. We initialize mission from person to person with a probability parameter
region-specific simulations with realistic parameter settings τ , through either complete mixing or heterogeneous connec-
learned from the corresponding surveillance data. (4) Exten- tions among people.
sive experiments on ILI incidence forecasting for two states In this work, we adopt the agent-based simulator Epi-
of the USA show that DEFSI achieves comparable/better Fast (Bisset et al. 2009). The outputs are individual level
performance than the state-of-the-art methods at state level. infected cases with the infected days of a simulated season.
For high-resolution forecasting at county level, DEFSI sig- They can be aggregated to any temporal and spatial scale,
nificantly outperforms the comparison methods. such as daily (weekly) state (county) level ILI incidence.

9608
Vaccine intervention IV (i.e. quantity of vaccines applied to
the population and the timing of the application) is applied
in our simulations. A distribution on the parameter space
P(pE , pI , τ, NI , IV ) is estimated from CDC historical data,
where NI denotes the number of infections at the beginning
of a flu season.
Disease Model Parameter Space Of the parameters in P,
(pE , pI ) are known from literature (Marathe et al. 2011).
We assume (τ, NI , IV ) follows distributions that can be es-
timated from historical data. For clarity, we define an epi-
demiological week in a calendar year as ew, and a seasonal
week in a flu season as sw. The historical time-series of CDC
surveillance data used to construct parameter space is seg-
mented into seasons by cutting at ew40 of each year (i.e. Figure 2: DEFSI neural network architecture. This archi-
ew40 of a calendar year corresponds to sw1 of a flu season). tecture consists of two branches. The left branch consists
Firstly, we collect observations of each parameter value of stacked LSTM layers that encode within-season obser-
by the following: (1) Transmissibility (τ ): We compute sea- vations x1, and the right branch is designed to be LSTM
son attack rate ar (i.e. fraction of population getting in- based layers that encodes between-season observations x1.
fected in the season) of each historical season for the tar- A merge layer is added to combine two branches.
get state and its neighbor states (i.e. geographically contigu-
ous states). We calibrate a transmissibility value for each of
ar as the solution to minτ |AR(EpiF ast(τ )) − ar|, where incidences of the same sw from the past b seasons. They
AR(·) computes attack rate from the output of EpiF ast(·). are used as the surrogate information to improve forecasting
(2) Initial Case Number (NI ): We collect the ILI inci- performance.
dence of the first week of each season for the target state The Long Short Term Memory (LSTM) network (Hochre-
and its neighbors. (3) Vaccine Intervention (IV ): We col- iter and Schmidhuber 1997) is adopted in our neural network
lect 6 vaccination schedules of the past six influenza seasons architecture to capture the dynamic temporal behavior of the
in USA (CDC 2018b). Each schedule consists of timing and observations. An LSTM layer consists of a sequence of a(b)
percent coverage of vaccine application throughout the sea- cells, of which the current cell takes one ILI incidence from
son. We assume that the state level vaccine schedule is the x1(x2) as well as the output and the cell state of its previous
same as the nationwide schedule. cell as inputs. We design a two-branch deep neural network
model to combine within-season observations and between-
Secondly, for τ (NI ), we fit the collected samples to sev-
season observations. As shown in Fig. 2, the left branch con-
eral distributions including normal, exponential, gamma,
sists of stacked LSTM layers that encode within-season ob-
and uniform. Then we run KS-test to choose a distribution
servations x1 = hyt−a , · · · , yt−1 i. The right branch is a sin-
with highest significance (refer to the supplementary file
gle LSTM layer that encodes between-season observations
(Wang, Chen, and Marathe 2019) for more details). For IV ,
x2 = hyt−`∗b , · · · , yt−`∗1 i. A merge layer is added before
we assume the 6 vaccination schedules follow a discrete uni-
the dense layer to combine the outputs from two branches. In
form distribution. In this way, a region-specific parameter
DEFSI model, the left and right LSTM-based branches take
space P is constructed.
x1 and x2 as inputs respectively. The merge layer requires
Training Dataset from Simulations Let ` denote the the outputs of two branches must be of the same dimension.
length (number of weeks) of each flu season simulated by The final output dimension is the same as zt .
EpiFast. For each run of a simulation, a specific parame- We are interested in a predictor f , which predicts the cur-
ter setting is sampled from P, and the simulator is called rent week’s state level and county level incidences zt based
to generate state and county level weekly incidence, called on the previous a weeks of within-season state level ILI in-
synthetic epicurve. Week 1 in the synthetic epicurve cor- cidences x1 and the previous b seasons of between-season
responds to sw1 of a flu season. Large volumes of high- state level ILI incidences x2:
resolution synthetic data are generated by repeating the sam- ẑt = f ([x1, x2]t , θ) (1)
pling and simulating process. where θ denotes parameters of the predictor, ẑt denotes the
DEFSI Neural Network Model In traditional time-series prediction of zt . The loss function L is defined as mean-
modeling problem, ILI incidences of the few previous square-error, with consistency constraint on outputs:
weeks are used as the observations for the prediction of
X
min L(θ) = ||zt − f ([x1, x2]t , θ)||2 ,
the current week. In DEFSI, we use two kinds of obser- θ
t
vations: (1) Within-season observations, denoted as x1 = X (2)
hyt−a , · · · , yt−1 i, are ILI incidence from previous a weeks with consistency constraint ŷt = ŷtC
of the current season. They are used as the main observa- C∈D
tions to follow the weekly trend. (2) Between-season ob- Adam optimization algorithm is used to learn θ. An activity
servations, denoted as x2 = hyt−`∗b , · · · , yt−`∗1 i, are ILI regularizer is added to ẑt for consistency constraint.

9609
Variants of DEFSI Model The two-branch neural net- Performance of Flat-resolution Forecasting We forecast
work architecture is flexible for multiple variants: (1) state level ILI incidence for VA, 2017-2018 and NJ, 2017-
DEFSI: Two-branch neural network as shown in Fig. 2. (2) 2018. Fig. 3 shows the forecasting performance on RMSE,
DEFSI-L: Only the left branch is used to take within-season MAPE, PCORR. (1) Performance on RMSE (left column
observations. (3) DEFSI-RDENSE: Changing the LSTM of Fig. 3): In VA, DEFSI, DEFSI-L, DEFSI-RDENSE,
layer of the right branch with Dense layers, which means ARIMA, LSTM achieve similar performances that are bet-
that the model does not care about the temporal relationship ter than EpiFast, AdapLSTM. In NJ, DEFSI and its variants
between between-season data points. We will discuss the re- consistently outperform others across the horizon, which on
sults of different variants in Experiments. average have 15%-64% improvement on RMSE. (2) Perfor-
mance on MAPE (middle column of Fig. 3): In VA, ARGO
Multi-step Forecasting In practical situations, we are in- performs the best among all methods except with horizon
terested in making predictions for several weeks ahead. In 1 where DEFSI-L performs the best. Meanwhile, DEFSI,
DEFSI, the left branch of the model appends the most recent DEFSI-L are comparable with ARGO. In NJ, DEFSI-
state level prediction to the input for predicting the target at RDENSE achieves the best performance closely followed
the upcoming week, and the right branch uses the state level by DEFSI. (3) Performance on PCORR (right column of
ILI incidences from the past seasons with sw equals to the Fig. 3): In both VA and NJ, DEFSI-L outperforms others on
upcoming week number. Pearson correlation (i.e. around 0.96 with horizon 1) which
on average have 20%-100% improvement compared with
Experiments others, followed by DEFSI, DEFSI-RDENSE, and ARGO.
Datasets CDC ILI incidence : The CDC surveillance of Overall, DEFSI and its variants make comparable/better pre-
weekly ILI incidence at state level from ew40, 2010 to dictions than the comparison methods at state level.
ew18, 2018. ILI Lab tested flu positive counts of New
Jersey : The state level and county level ILI Lab tested flu Performance of High-resolution Forecasting The per-
positive counts of season 2017-2018 in NJ. The data is avail- formance of county level forecasting is evaluated on NJ
able from ew40 to the next year’s ew20. Google data and 2017-2018. The horizon is extended to 10 for better ob-
Weather data are collected for comparison methods (see servations. In Fig. 4, we show county level ILI forecast-
the supplementary file (Wang, Chen, and Marathe 2019) for ing performance on each metric. The metric value of each
details). node in the figure is the average value across 21 counties
in NJ. Our method consistently outperforms the compari-
Comparison Methods Our method is compared with son method EpiFast on RMSE (about 53% reduction) and
5 state-of-the-art methods. They are LSTM (single layer PCORR (about 60% increase). However, EpiFast performs
LSTM) (Hochreiter and Schmidhuber 1997) and AdapLSTM better than our method on MAPE with horizon less than
(CDC + Weather data) (Venna et al. 2017) from artificial 4, while the error increases dramatically as the horizon in-
neural network methods; ARIMA (classic ARIMA) (Ben- creases. Overall, our method significantly outperforms the
jamin, Rigby, and Stasinopoulos 2003) and ARGO (CDC comparison method on county level forecasting.
+ Google data) (Yang, Santillana, and Kou 2015) from sta-
tistical methods; and EpiFast (Beckman et al. 2014) from Discussion In general, AdapLSTM and EpiFast do not
agent-based causal models. AdapLSTM, LSTM, ARGO and perform very well in our experiment compared with other
ARIMA are used for state level forecasting. EpiFast is ap- methods. For AdapLSTM, weather factors are considered
plied for both state level and county level forecasting. for post adjustment of LSTM outputs. As stated in (Venna et
al. 2017), the weather factors are estimated using time delays
Experiment Setup Our experiments are performed on computed by apriori associations and selected by the largest
two states: Virginia (VA) and New Jersey (NJ). For each confidence. However, in our experiment, they all show very
state, we separate the time sequence into 8 flu seasons low confidences (less than 0.3). This may cause arbitrary
from 2010-2011 to 2017-2018 by cutting at ew40. We use adjustment for predictions and consequently poor perfor-
the dataset of 2010-2011 to 2016-2017 seasons as training mance. For EpiFast, one possible reason is that we did not
dataset; and use the dataset of 2017-2018 season for testing. find a good estimate of the underlying disease model for a
At each time step in the testing season, each model makes specific region and season due to the noisy CDC observa-
predictions 5 weeks ahead, i.e. horizon = {1, 2, 3, 4, 5}. tions. As we discussed in the Introduction, if the observed
In DEFSI, the training dataset is used to estimate disease data is too noisy then the learned underlying model tends
parameter space, while for other comparison methods, it to make predictions with large errors. Through the results,
is used for training forecasting model directly. The col- DEFSI enables high-resolution forecasting that outperforms
lected county level ILI data of NJ is only used for evalu- EpiFast. Meanwhile, it achieves comparable/better perfor-
ating. More detailed settings (including estimated parameter mance than the comparison methods at state level forecast-
space in DEFSI, parameter settings for comparison meth- ing. Our experiments demonstrate that DEFSI integrates the
ods) are elaborated in the supplementary file (Wang, Chen, strengths of ANN methods and causal methods to improve
and Marathe 2019). The metrics used to evaluate the fore- epidemic forecasting.
casting performance are: root mean squared error (RMSE),
mean absolute percentage error (MAPE), Pearson correla- The Significance of IV in P Any intervention actions
tion (PCORR). taken during a flu season will obviously affect the disease

9610
 (a) VA, 2017-2018

(b) NJ, 2017-2018

Figure 3: State level ILI incidence forecasting performance on RMSE, MAPE, PCORR. A log y-scale is used in RMSE and
MAPE.

Figure 4: County level ILI incidence forecasting performance on RMSE, MAPE, PCORR for NJ, 2017-2018. A log y-scale is
used in RMSE and MAPE.

spread. Our method described above constructs a parameter Our experiments show the significance of realistic interven-
space P(pE , pI , τ, NI , IV ) that includes the vaccine inter- tions in P for good forecasting performance of our method.
vention IV . In this section, we investigate how IV affects Our proposed framework is extensible for further available
DEFSI model by generating two synthetic training datasets: realistic interventions, such as school closure and quaran-
(1) vaccine-case: simulations with IV (the training dataset tine, to further improve forecasting performance.
used in Experiments); (2) base-case: simulations that share
the common settings of pE , pI , τ, NI with vaccine-case ex- Conclusion
cept IV = ∅.
In this paper we propose DEFSI, a novel epidemic fore-
We train DEFSI on vaccine-case and base-case respec- casting framework combining deep neural network meth-
tively with the same settings described in Experiment Setup, ods with causal models. In DEFSI, a two-branch neu-
denoted as DEFSI-vac and DEFSI-base. State level forecast- ral network model and its variants are designed to com-
ing on VA 2017-2018 is evaluated. Fig. 5 shows the perfor- bine within-season and between-season observations. The
mance on RMSE, MAPE, and PCORR. DEFSI-vac consis- model is trained on a region-specific synthetic dataset con-
tently outperforms DEFSI-base on RMSE and PCORR, es- structed at multiple spatially fine-grained scale. The trained
pecially with small horizon less than 5. On MAPE, DEFSI- model enables accurate high-resolution forecasting with flat-
vac performs better than DEFSI-base with small horizon less resolution observations as inputs. Extensive experiments on
than 5, while the situation inverses as the horizon increases. NJ and VA showed that DEFSI achieved comparable/better

9611
Figure 5: Performance comparison between DEFSI models trained on base-case synthetic training dataset (DEFSI-base) and
vaccine-case synthetic training dataset (DEFSI-vac). VA, 2017-2018.

performance than the state-of-the-art methods on state level Kuznetsov, Y. A., and Piccardi, C. 1994. Bifurcation anal-
forecasting and consistently better performance than others ysis of periodic SEIR and SIR epidemic models. Journal of
on county level forecasting. A direction for future work is Mathematical Biology 32(2):109–121.
to further investigate the use of synthetic data generated by Marathe, A.; Lewis, B.; Chen, J.; and Eubank, S. 2011.
social, epidemiological and behavioral theories in conjunc- Sensitivity of household transmission to household contact
tion with observed data to improve epidemic forecasts. structure and size. PLoS ONE 6.
Nsoesie, E. O.; Beckman, R. J.; Shashaani, S.; Nagaraj,
Acknowledgment K. S.; and Marathe, M. V. 2013. A simulation optimization
The authors would like to thank members of the Network approach to epidemic forecasting. PLOS ONE 8(6):1–10.
Simulation Science and Advanced Computing (NSSAC) Venna, S. R.; Tavanaei, A.; Gottumukkala, R. N.; Raghavan,
Division. This work has been partially supported by De- V. V.; Maida, A.; and Nichols, S. 2017. A novel data-driven
fense Threat Reduction Agency (DTRA) Grant HDTRA1- model for real-time influenza forecasting. bioRxiv.
17-D-0023, National Institutes of Health (NIH) Grant
1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Volkova, S.; Ayton, E.; Porterfield, K.; and Corley, C. D.
DIBBS Grant ACI-1443054. 2017. Forecasting influenza-like illness dynamics for mil-
itary populations using neural networks and social media.
PLOS ONE 12(12):1–22.
References
Wang, L.; Chen, J.; and Marathe, M. 2019. DEFSI: Deep
Beckman, R. J.; Bisset, K. R.; Chen, J.; Lewis, B. L.;
Learning Based Epidemic Forecasting with Synthetic In-
Marathe, M. V.; and Stretz, P. E. 2014. ISIS: a networked-
formation (Supplement). https://github.com/christa60/defsi/
epidemiology based pervasive web app for infectious dis-
blob/master/supp IAAI DEFSI.pdf.
ease pandemic planning and response. In KDD.
Wu, Y.; Yang, Y.; Nishiura, H.; and Saitoh, M. 2018. Deep
Benjamin, M. A.; Rigby, R. A.; and Stasinopoulos, D. M.
learning for epidemiological predictions. In SIGIR.
2003. Generalized autoregressive moving average models.
Journal of the American Statistical Association 98(461): Yang, S.; Santillana, M.; Brownstein, J. S.; Gray, J.; Richard-
214–223. son, S.; and Kou, S. C. 2017. Using electronic health records
and internet search information for accurate influenza fore-
Bisset, K. R.; Chen, J.; Feng, X.; Kumar, V. A.; and Marathe,
casting. BMC Infectious Diseases 17(1):332.
M. V. 2009. EpiFast: A fast algorithm for large scale realistic
epidemic simulations on distributed memory systems. In Yang, W.; Karspeck, A.; and Shaman, J. 2014. Compari-
Proceedings of the 23rd ICS, 430–439. ACM. son of filtering methods for the modeling and retrospective
forecasting of influenza epidemics. PLOS Computational
CDC. 2018a. Fluview interactive. https://www.cdc.gov/flu/
Biology 10(4):1–15.
weekly/fluviewinteractive.htm. Accessed April 20, 2018.
Yang, S.; Santillana, M.; and Kou, S. C. 2015. Accurate
CDC. 2018b. Historical seasonal influenza vaccine sched-
estimation of influenza epidemics using Google search data
ule. https://www.cdc.gov/flu/professionals/vaccination/
via ARGO. PNAS 112(47):14473–14478.
vaccinesupply.htm. Accessed November 1, 2017.
Zhao, L.; Chen, J.; Chen, F.; Wang, W.; Lu, C. T.; and Ra-
Hochreiter, S., and Schmidhuber, J. 1997. Long short-term
makrishnan, N. 2015. SimNest: Social Media Nested Epi-
memory. Neural Computation 9(8):1735–1780.
demic Simulation via Online Semi-supervised Deep Learn-
Karpatne, A.; Atluri, G.; Faghmous, J. H.; Steinbach, M.; ing. Proceedings of IEEE ICDM 2015:639–648.
Banerjee, A.; Ganguly, A.; Shekhar, S.; Samatova, N.; and
Kumar, V. 2017. Theory-guided data science: A new
paradigm for scientific discovery from data. IEEE Trans-
actions on Knowledge and Data Engineering 29(10):2318–
2331.

9612