Operations Research for Health Care 26 (2020) 100266

Contents lists available at ScienceDirect

Operations Research for Health Care

journal homepage: www.elsevier.com/locate/orhc

A hybrid system dynamics, discrete event simulation and data

envelopment analysis to investigate boarding patients in acute
hospitals
∗
Leila Keshtkar , Wael Rashwan, Waleed Abo-Hamad, Amr Arisha
3S Group, College of Business, Technological University Dublin, Dublin, Ireland

article info a b s t r a c t

Article history: Timely access to health services has become increasingly difficult due to demographic change and
Received 7 November 2018 aging people growth. These create new heterogeneous challenges for society and healthcare systems.
Received in revised form 8 June 2020 Congestion at acute hospitals has reached unprecedented levels due to the unavailability of acute
Accepted 9 July 2020
beds. As a consequence, patients in need of treatment endure prolonged waiting times as a decision
Available online 15 July 2020
whether to admit, transfer, or send them home is made. These long waiting times often result in
Keywords: boarding patients in different places in the hospital. This threatens patient safety and diminishes the
Boarding patients service quality while increasing treatment costs. It is argued in the extant literature that improved
Acute medical assessment unit communication and enhanced patient flow is often more effective than merely increasing hospital
Hybrid simulation capacity. Achieving this effective coordination is challenged by the uncertainties in care demand, the
Data envelopment analysis availability of accurate information, the complexity of inter-hospital dynamics and decision times.
Healthcare management A hybrid simulation approach is presented in this paper, which aims to offer hospital managers a
chance at investigating the patient boarding problem. Integrating ‘System Dynamic’ and ‘Discrete
Event Simulation’ enables the user to ease the complexity of patient flow at both macro and micro
levels. ‘Design of Experiment’ and ‘Data Envelopment Analysis’ are integrated with the simulation
in order to assess the operational impact of various management interventions efficiently. A detailed
implementation of the approach is demonstrated on an emergency department (ED) and Acute Medical
Unit (AMU) of a large Irish hospital, which serves over 50,000 patients annually. Results indicate
that improving transfer rates between hospital units has a significant positive impact. It reduces the
number of boarding patients and has the potential to increase access by up to 40% to the case study
organization. However, poor communication and coordination, human factors, downstream capacity
constraints, shared resources and services between units may affect this access. Furthermore, an
increase in staff numbers is required to sustain the acceptable level of service delivery.
© 2020 Elsevier Ltd. All rights reserved.

1. Introduction bed-blocking and insufficient free beds [11]. This limits accepting
new patients to the ED. Boarding problems occur in other units
Patient boarding is a prevalent phenomenon within hospi- across the hospital, such as in the Intensive Care Unit (ICU), where
tals, especially within the emergency department (ED) [1–4]. A critically ill patients require intensive care. However, they are
‘boarded patient’ in the ED is described as a patient who must stay placed in an alternative subspecialty unit [12,13].
in the ED while awaiting transfer to an inpatient unit after a deci- Moreover, boarding occurs when delays in discharging pa-
sion for hospitalization has been made [5,6]. This phenomenon is tients who no longer require acute care services and are waiting
also identified as access blocked [7,8], exit block [9] and patient for post-acute care outside the hospital setting [11]. In 2017,
blocking [10], where a patient requires further inpatient care,
the Health Service Executive (HSE) in Ireland reported 201,977
however, a hospital bed within a reasonable time is unavail-
bed days were classified as lost due to delays in discharges [15].
able. These patients, who have completed their treatments and
Furthermore, 12,201 patients waiting on trollies in the EDs or on
are medically ready to leave the unit, are the main reason for
additional beds placed in the corridors throughout the hospitals
∗ Corresponding author. (Fig. 1). The red dotted curve estimates the boarding trend using
polynomial regression of order 2 in time. The trend shows under
E-mail addresses: [email protected] (L. Keshtkar),
[email protected] (W. Rashwan), [email protected] the current system conditions; the boarding trend is growing very
(W. Abo-Hamad), [email protected] (A. Arisha). fast.

https://doi.org/10.1016/j.orhc.2020.100266
2211-6923/© 2020 Elsevier Ltd. All rights reserved.
2 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

Fig. 1. The trend of the boarded patient in Ireland.

Source: Data source: [14].

Evidence increasingly confirms that boarding and admission in complex ways. Due to the stochastic nature of the health-
delays are associated with a higher mortality rate of critically ill care systems, its complex dynamics and interactions of their
patients [12,16]. Patients boarded for less than 2 hours have a inputs, activities and outputs, healthcare managers require tools
lower mortality rate than patients boarding 12 hours or more [17]. which can enable them to understand this complexity and thus
Boarding prolongs hospital stays and increases hospital occu- to enhance their system performance. Over the years, research
pancy, which in turn, exacerbates boarding [18,19]. has shown that the use of modeling approaches significantly
Moreover, prolonged boarding creates additional stress on improves decision-making for healthcare management at various
already suffering patients, families and staff [4]. These stress- levels if successfully applied [27]. The following modeling ap-
ful atmospheres result in increased medical errors, risks, and proaches were utilized: Discrete Event Simulation (DES), System
decreased quality of care [20]. Moreover, boarding has adverse Dynamics (SD), Agent-Based Simulation (ABS), and Monte Carlo
financial consequences due to the revenue lost from patients (MC) simulation. Research has indicated that the use of hybrid
leave without being seen, as well as ambulance diversions [5]. simulation would improve the capabilities of simulation solu-
The extra costs related to delayed transfers from ICUs also result tions. Hybrid simulation, by combining two or more simulations,
in revenue losses [21]. not only enables the symbiotic realization of the strengths of
There are many factors contributing to the boarding problem. individual methods but also reduce the limitations of a single
Demographic changes, population growth, aging and increased method [28]. The main restrictions of DES lie in its inability to
life expectancy all contribute to the considerable increase in the capture the feedback dynamics related to the holistic structure
number of emergency visits. This, in turn, increases the pressure of a system, a substantial amount of accurate data is required to
on hospital EDs [22]. The demand for care services in Ireland has build a model, and it is time-consuming to develop and run [29,
grown at a rate of 3.1% in patients over 65-years old and 4.2% 30].
in patients over 85-years old, between 2015 and 2016 [23]. The On the other hand, SD models fail to capture detailed com-
lack of inpatient beds for EDs and ICUs patients is recognized as plexities and individual movements through queues and activities
one of the primary drivers of boarding [8,24]. Patients are expe- within the system [31]. Morgan et al. [32] also discussed all the
riencing delays in their transfers due to insufficient ward beds or intended objectives of the study cannot be accomplished with a
understaffing [16]. Staff availability, staff skillset, as well as timely single simulation method. This is due to the model requiring sev-
transferring patients, are other causes of boarding [25]. The lack eral assumptions regarding system behavior. Therefore, a hybrid
of efficiency in the continuity of care through step-down and simulation provides a more realistic picture of the systems from
appropriate alternative care provisions also causes bed-blocking, different perspectives [33], which allows fewer assumptions and
which in turn exacerbates critical care resource shortages [11]. increased accuracy of outcomes, without oversimplifying some
Ultimately, hospital management is looking for ways to reduce aspects [34].
overcrowding and boarding patients [26]. The boarding problem This paper presents an integrated hybrid approach to investi-
is not only an ED-based problem, but it is an indication of the gate the impact of the boarding problem on patients’ ability to
dysfunction of interrelated parts of a broader system. It is chal- access other units in the hospital while considering the intra-
lenging to draw clearly defined boundaries around a unit in order departmental and inter-departmental interactions in both up and
to address this problem. Especially where different contributing down-stream hospital facilities. The proposed method integrates
factors affect each other. There is a need to adopt a system- DES, SD, Design of Experiments (DoE) and Data Envelopment
wide approach to address patient boarding. This includes the Analysis (DEA). Firstly, the DES component offers a better rep-
investigation of multi-stage patient flows, throughout the entire resentation of the complexity of the process in detail, includ-
process of care, as well as considering the interactions and in- ing patients flow, processes and underlying relationships with
terdependencies between various hospital units. Thus, given the supporting units in the hospital. Secondly, the SD component
complexity of the problem, it is not possible to use only a single captures the cross-boundary interactions between the hospital
method to study the problem in detail and at the same time allow facilities. This represents a holistic view of the feedback between
a holistic system view. elements of the system. Finally, DoE and DEA are used to explore
Furthermore, healthcare systems are human-based systems the most efficient system configuration according to a set of
that involve multiple stakeholders who interact with each other predefined key performance indicators (KPIs). It is also envisaged
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 3

this effort will have a positive impact on the overall issue of may result in shifting the boarding issue to other parts of the
patient boarding, within the hospital context. system. This, in turn, could block access to appropriate clinical
The remainder of the paper is organized as follows. Section 2 areas [53]. Therefore, some studies focus on improvement options
reviews the extant literature, focusing on studies, which ad- outside the ED, especially in downstream units such as inpatient
dressed the problem of boarding and overcrowding. The proposed wards. Improving inpatient discharge timing by shifting the time
methodology is discussed in Section 3. Section 4 presents a case of discharge to an earlier time in the day [26,54,55], can im-
study in the context of the Irish healthcare system. It provides prove resources utilization and alleviate the boarding problem.
a detailed application of the proposed approach from Section 3. Additionally, effective scheduling strategies can also be applied to
Section 5 presents the findings and discusses the results. Finally, manage the demand for hospital beds from competing admission
Section 6 concludes the paper. sources. These strategies can be designed to offset unscheduled
ED admission requests with elective arrivals and hence reduce
2. Literature review boarding times [1]. For example, considering different strategies
to schedule elective patients while reserving a fixed capacity
2.1. Solution strategies for ED admissions [56]. Moreover, demographic data and clinical
information can be used to estimate the likelihood of patient ad-
Several improvement options are investigated in the literature mission accurately. These predictions enable hospital managers to
in order to alleviate the patient boarding problem. These are improve their estimation of required resources and the process of
mainly divided into ED-based and hospital-wide solutions. The assigning beds to patients [25]. Finally, researchers indicate that
improvements in the first category are mainly related to the a frequent assessment of boarders, effective communication and
increased capacity of rooms, beds and staff. While increasing the coordination between the ED and other related departments fa-
number of ED trollies and inpatient beds might seem to mitigate cilitate patient transfers and result in lower patient boarding [25,
the boarding issue intuitively, many studies have reported that 57].
increased capacity is not only ineffective but also an expensive
solution. The process of increasing the transfer rate of ED patients 2.2. Solution methods
to inpatient beds (i.e., unlocking the access block from the ED to
inpatient beds) would have a more significant impact on reducing Simulation paradigms, particularly DES and SD, are widely
the number of ED boarders, than the increase of medical staff used to capture and understand the dynamics of a system. SD is
or assessment cubicles [35,36]. Bed management also plays a mainly used at more strategic levels in order to gain insight into
critical role to improve transfer rate and patient flows [37,38]. As the interrelations between the different parts of the healthcare
a function, bed management has several responsibilities includ- system [58]. However, SD models are less powerful in capturing
ing placing emergency and elective admissions into appropriate the level of granularity and less flexible in modeling individual
beds and enabling patient discharge and transfer by coordinating entities of the system [59].
services which patients may require [39]. Bed management is Due to the dynamism and multi-disciplinary nature of the
widely used to reduce boarding and overcrowding in the ED. This healthcare system, DES is a valuable tool in assisting healthcare
can be achieved by classifying patients as soon as they enter the managers in decision-making. It captures more of the detailed
ED, which enables the staff to understand the require inpatient complexity [60]. DES has been proven to be a useful tool for pro-
beds [37,40] and also predicting daily attendance [41]. Further- cess modeling and improvement [61]. Healthcare managers can
more, improving patient flow from the ED to inpatient areas by apply DES to assess a systems current performance, predicting
considering both emergency and elective patient flows [38], and the impact of operational changes and examining the trade-offs
analyzing the impact of critical bed status on ED patient flow [42] between system variables [62]. Review studies on modeling and
supports to alleviate the problem. simulation methods within the healthcare context have reported
Many hospitals have also started to use Acute Medical Units a vast number of DES models for modeling service operations, ca-
(AMUs) and Acute Surgical Units (ASUs) as alternative models of pacity, process and workforce issues in different units in hospitals
care for patients who present to EDs [43,44]. These units aim and clinics [63–66].
to restructure acute patient flow in the ED and provide better Furthermore, DES has been used to study patients boarding
hospital services. The benefits of AMUs and ASUs are derived from problems and their implications on patient experience. De Boeck
streamlining of medical and surgical patients to a location, where et al. [67] explored the impact of boarding patients on the ED
they can be seen without delay. Moreover, these units are staffed system performance and compared different priority policies for
by experienced consultants and multidisciplinary teams, which ED physicians decision. The tradeoff between increasing the phys-
allow the patients to be comprehensively assessed and managed ical capacity of the ED and reducing patient boarding times was
before being either discharged or transferred to the inpatient care examined in Khare et al. [35]. The impact of inpatient boarding
setting [45]. If admission is required, this will occur within a de- on the ED’s efficiency was also explored in Bair et al. [68]. The
fined period, and the patient is admitted to the most appropriate relationship between inpatient discharge times and ED boarding
clinical area in the hospital. Therefore, these units operate as the of admitted patients was presented by Powell et al. [26] using a
interface between primary services and the downstream medical cross-sectional computer model. Levin et al. [1] utilized DES in
specialty wards. In terms of locations, these units are usually co- investigating the effect of bed demand of cardiology admission
located with EDs, which eases access to diagnostic services such sources on the ED boarding. Pines et al. [5] applied DES and re-
as radiology and enhances service delivery [46]. gression analysis to study the financial implication of ED boarding
In the UK, Australia and New Zealand, many hospitals have on the overall hospital revenues. The study of Roh et al. [69]
implemented AMUs/ASUs or units with synonymous names. Re- developed a DES to understand the flow of mental health pa-
cently, several studies reported significant improvements in ser- tients within the ED and to inpatient settings. Several scenarios
vice delivery levels such as decreased mortality rates [47], reduc- designed to specify the percentage of increase in beds necessary
tion in length of stay [48–50], reduced waiting times in EDs [51], in reducing mental health ED boarding times The model of Shi
decreased admission rates [24] and improved patient and staff et al. [70] explored the operations within the inpatient wards
satisfaction [52]. However, the introduction of these units with- and their relation to the ED in order to reduce boarding time.
out improving the streamlining of patient flow across the hospital This study linked boarding times to the imbalance between the
4 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

daily number of arrivals and discharges numbers, as well as a mis- Furthermore, ABS and DES has been used to model emergency
match between the discharge timing and hourly arrival pattern. medical services (EMS) [85], radiology center [86], and analyze
Furthermore, Mustafee et al. [71] used DES to investigate the bed sustainable planning strategies for EMS [87], study patient choice
management strategies in reducing bed blocking in specialized and behavior in the healthcare system [88].
and integrated care units. The DES, in Crawford et al. [72], has Data envelopment analysis (DEA) is a mathematical program-
been used to model patient pathways in an acute care hospital in ming model, which is used to evaluate and compare the effi-
order to investigate the effects of discharge timing on ED waiting ciency of decision-making units (DMUs) [89]. DEA overcomes
and boarding time. many drawbacks of other performance evaluation approaches,
Recently, there has been increasing attention in using hybrid such as ratio-based analysis (RBA), least-squares regression (LSR),
simulation to address healthcare system challenges [59,73]. Hy- total factor productivity (TFP) and stochastic frontier analysis
brid simulation is a form of mixing methodologies and is defined (SFA). DEA is a non-parametric technique, which can provide
as a modeling approach, which combines two or more simulation a consistent benchmark for all inputs and outputs, predict the
methods (e.g., DES, SD and ABS) to model complex systems [74]. best performance or the most efficient relationships and identify
They potentially provide a more realistic picture of systems from individual inefficient units [90,91].
different perspectives, while reducing the constraints of a single Additionally, DEA has proven to be a useful tool for evaluating
method. Furthermore, hybrid simulation provides improved in- the efficiency of hospitals [92] and in measuring performance
sights into complex systems as they offer a holistic approach to efficiency within the healthcare system [93]. DEA can be applied
system analysis [33]. to evaluate the simulation results and facilitate the search pro-
Several studies discussed how various simulation methods cess [94]. However, few studies have examined the use of DEA
can be combined in a hybrid model [59,74–76]. Different studies in simulation analysis. The integration of DES and DEA has been
also used hybrid simulation within the context of healthcare. used to improve the quality of care in ED by modeling different
Hybrid simulation (i.e. DES-SD) was applied to improve outpa- errors from nurses and technicians [95], analyzing ED efficiency
tient scheduling [77]. DES was used to investigate the influence from eight hospitals [96], allocating ED resources efficiently [97]
of a new scheduling approach on patient cycle time, while SD and improving patient flow [98].
was used to understand the relationship between the scheduling The use of hybrid models in healthcare modeling is growing.
system, patient demand and service capacity. Rohleder et al. [78] Research in modeling of healthcare systems illustrates that the
presented a hybrid simulation model in order to redesign and im- simulation models if integrated appropriately, can be used in
plement new healthcare facilities. They used DES to design a more healthcare settings as an active decision support system for the
management team. Without compromising patient safety, man-
useful set of facilities and made recommendations for resource
agers can practice decision-making in specific clinical situations
changes. SD was also used to predict new patterns in demand
and develop reasoning for new strategies.
and examine the possible adverse effects on a new system. Viana
et al. [29] built a hybrid simulation to address the problem of the
3. Material and methods
sexually transmitted infection chlamydia. DES was used to model
the hospital outpatient clinic where patients get treated, and SD
The presented approach utilizes various methods of data col-
was used to model the infection process in the community. The
lection to incorporate multiple perspectives, so a mixture of qual-
impact of developing integrated patient pathways was explored
itative and quantitative has been engaged (Fig. 2). Interviews and
by Zulkepli and Eldabi [79]. In this study, DES was developed to
observations have a qualitative nature. This has the significant
model assessment and intermediate care processes. While in turn,
benefit to understand and model the workflow (processes) in
SD was used to capture the effect of patient readmission on the the healthcare facility. This form of data allows for the incor-
care process. Hybrid simulation has also been used to forecast poration of practitioners’ view to enrich understanding. On the
healthcare demand [80]. The SD model simulates the continuing other hand, quantitative data has a factual nature and depends
evolution of the population and DES generates patient arrival on verifiable information. This type of data is collected from
times and the prevalence of needs for service in the healthcare various sources, including the Hospital Information System (HIS)
system. and local databases. In order to collect this data, institutional
More specifically, DES-SD simulation is used to improve the ED approval was granted from the Ethics Research Committee of the
process. Ahmad et al. [81] used DES to model the complexities of Technological University of Dublin. They confirmed that there
the integrated ED system. In turn, SD was utilized in capturing were no ethical issues regarding the project. Furthermore, the
the interdependency between the ED and other sub-units within anonymity of the participants and the confidentiality of data
a hospital. The impact of strategic changes on demand levels was is maintained at all times and no private information will be
examined by Bell et al. [82]. The variation in demand for care, in disseminated.
particular for unplanned demand, was simulated by SD. DES was The data analytics component helps to assess and analyze
built to represent patient activity through urgent care services. the patient volume, their severity mix and patterns of patient
Similarly, Chahal et al. [83] used hybrid simulation to explore and presentation. Descriptive analytics play a key role in the analysis
evaluate the effect of a whiteboard on the workflow of an ED. The of the historical data, which is used to capture insightful infor-
DES model was used to capture the detailed complexity of the mation from the data. The outputs of descriptive analytics are the
ED’s processes. The variation in ED performance in response to probability distributions and parameters, which are required for
the whiteboard information flow was presented in the SD model. simulation.
This hybrid model was able to capture the detailed operational Conceptual modeling is a significant step in the building of
level, as well as the impact of information flows throughout the a simulation model, and it is potentially the most significant
ED process. stage in a simulation study. In order to modeling the underly-
The combination of DES-ABS-SD has also been recognized as ing business processes, knowledge from the individuals directly
beneficial in analyzing hospital process and workflows [84]. In involved in service delivery is required. The modeler uses various
this study, DES was applied to represent processes, ABS and state methods (e.g., interviews, direct observation and focus group)
charts were used to reflect individual behavior at the micro-level. to get as much information as possible, without influencing or
On the other hand, SD was used to model abstract and continuous manipulating the problem definition. Since model conceptual-
structures. ization is an iterative process, it requires close interaction with
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 5

Fig. 2. Model formulation and understanding.

experts and practitioners in order to obtain holistic insights into To support simulation as a tool for experimentation, DoE is
the aspects of the system under scrutiny. There are two outputs necessary. DoE allows the evaluation of scenarios to identify the
of this component: a detailed process representation of the real most significant factors affecting the overall performance. DoE is
system for DES and, a feedback conceptual model for SD. These a useful tool with many theoretical developments and practical
processes are mapped into a conceptual model using one of the applications in various fields [53].
well-developed modeling languages such as a flowchart and or The Taguchi method of DoE facilitates robust designs when
state chart diagram. This is where sub-processes and activities are selecting variables. This is achieved through applying different
identified. Feedback conceptualization is an essential activity for orthogonal arrays (OA) according to the number and level of
modeling and captures the dynamic behavior of the system. Key parameters. The Taguchi DoE decreases experimental errors and
variables and factors can be identified through a series of inter- increases both the efficiency and reproducibility of different ex-
views, focus groups and secondary data mined from the literature. periments. It also considers two-way interaction factors which
Causal loop diagrams and sub-system diagrams are regularly used simplify the interpretation of results. This gives a better insight
to captured feedback loops. A fundamental principle is modeling into the overall process analysis [99]. The results of DoE are then
and conceptualization should be focused on a problem instead of used as an input to DEA to evaluate and rank the best scenarios
a system and guided by a clear purpose and objectives. in improving an acute medical assessment unit (AMAU) perfor-
Modeling patient flow across hospital units necessitates the mance. Due to the complexity and variety of measures in health-
integration of the flow between downstream and upstream facil- care contexts, the output results of DoE are analyzed by DEA to
ities, which display a high degree of interdependency. Therefore, measure the efficiency of different designs and recommend the
a hybrid SD-DES simulation model is developed to address the most appropriate decision for the problem (Fig. 4).
consequences of patient boarding problem in a hospital setting
(Fig. 3). The upstream units represent the demand sources of the 4. Case study
specific unit under investigation (e.g., ED, AMU, and ICU), while
the downstream components model the patient disposition. Pa- The National Acute Medicine Program is a clinician-led ini-
tient disposition refers to two cases in any patient boarding tiative incorporating the Irish Health Service Executive’s (HSE)
problem in a given hospital unit (i.e., Hospital unit i). Firstly, Clinical Strategy and Programs Directorate, and the Royal Col-
patients who finished their care episode in that unit and are lege of Physicians of Ireland (RCPI), among others [100]. It has
waiting for beds in other downstream units. Secondly, patients developed a framework to mitigate the pressure which Irish
have been boarded to that unit from other upstream units. In the EDs face and to minimize the LOS. This is an effort to reduce
proposed hybrid simulation, patient flow within the hospital unit overcrowding by introducing AMUs to work in parallel with the
with boarding problem is modeled using a DES. This allows simu- EDs. The framework aims to provide medical patients presenting
lating the unit activities and processes in detail, while considering to the ED, with a fast track to decisions regarding their treatment
resources interaction. Downstream and upstream hospital opera- journey in the hospital. Patients who present in these units see a
tions are explained in the SD model. This enables managers to senior medical doctor (MD), who can make treatment decisions
envisage the impact of changes using feedback loops between within almost one hour of admission.
the different activities. The two simulation models run simulta- However, shortly after opening, the boarding problem shifted
neously, and the information is exchanged between models in the from the EDs and to these units. Which, in turn, restricted access
runtime with a parallel interaction. This is enabled by AnyLogic to the AMU by medical patients. Therefore, hospital managers and
software. executives requested a formal assessment of this problem and
6 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

Fig. 3. Hybrid simulation.

Fig. 4. Experimentation and decision-making.

on how these units may cope with the unpredicted increase in works as a 12-h’ unit. It is open from 9 am to 9 pm, however, it
workload and demand. As a response, a project was carried out only accepts patients up until 6 pm to allow beds to be released
within the HSE to provide managers with a tool to investigate for the next day. The SSU also works as a short stay ward, on a
the boarding problem. This tool also enabled the assessment of a 24/7 basis.
dedicated unit within a hospital and proposed a model to assist The AMAU is staffed by physicians, dedicated multidisciplinary
with resource planning. medical and support teams. The only access to the AMAU is
through the ED [53]. Patients are triaged in the ED and assigned a
4.1. Phase 1: Formulation and understanding triage category, according to the Manchester Triage System (MTS)
that uses a five-level scale for classifying patients per their care
The partner hospital for this study has one of the busiest ED requirements [101]. The triage nurse usually contacts the AMAU
nationally that operates 24 h a day, seven days a week throughout consultant or registrar so that they can accept or reject the case.
the year. In 2012, 41,781 unscheduled adult patients visited this Patients routed to the AMAU are those medical patients who have
ED. This number increased by 9% in 2015 and has seen sustained been assigned a triage category of 2 or 3 (i.e., very urgent and
growth in the above 65 age group, which has increased by 41% urgent patients respectively), who do not require resuscitation or
over the last six years. There is a higher acuity associated with isolation facilities. The patients are only transferred to the AMAU
this age group with a higher likelihood of admission, longer LOS if a trolley is available. Patients presented to these units will see a
and a higher incidence of both influenza and Noro-virus. This senior MD, who treats and discharge the patients within almost
results in a considerable number of bed days lost due to isolation one hour of admission. The AMAU, SSU and ED share resources
requirements. among them and share other resources with the hospital. When
The AMU is divided into two sub-units: the acute medical as- the AMAU was first introduced, there was an increase in the
sessment unit (AMAU) and, the short-stay unit (SSU). The AMAU proportion of patients discharged within 24 h and also a decrease
is considered to be the first gateway for acute medical patients in LOS and overall medical bed day usage [102].
who are referred from the ED, while the SSU is used by patients The capacities of the SSU and AMAU are 24 beds and 11
who need to be admitted to the hospital but their LOS is esti- trolley spaces, respectively. While the SSU has 24 beds, only 12 of
mated to be less than five days. Patients can also be admitted them are under the governance of an acute medical consultant.
directly to hospital clinical wards from the AMAU (Fig. 5). Re- The remaining 12 beds are under the management of standard
cently, our partner hospital opened both an AMAU and SSU. The medical consultants in the hospital. The AMAU in this study faces
AMAU was opened as a discontinuous healthcare service which two types of boarding problems: ED boarding and internal AMAU
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 7

Fig. 5. Generic patient pathways through a hospital with the AMAU/SSU.

boarding. In the ED boarding case, patients occupied between one with the peak times being between 11 am and 3 pm. There
to six AMAU trollies for a maximum of 12 h. On the other hand, was an hourly average in arrivals of 7.9. These patterns pro-
internal AMAU boarding occurs when patients from the AMAU vide an overview regarding the demand volumes for services
require a hospital bed in another downstream unit (e.g., med- in the ED and different temporal scales for the patient arrival
ical wards or SSU) for further treatment. As stated previously characteristics.
the AMAU opens from 9 am to 9 pm, and stop accepting new The hourly arrival data for the ED was consolidated by the
patients at 6 pm. Over six months, the number of medical patients hour of the day. This allowed the presentation of arrival rates that
presented to the ED with the triage categories 2 and 3 was 3753. are required as inputs for the model. The impact of monthly and
However, only 40% of those patients accessed the AMAU with an weekly variation have been smoothed. The inter-arrival times for
average LOS (i.e., the total time from the patient entrance to the each hour of the day was used to fit the exponential distribution
AMAU, until they exited the unit) of 4.45 h. using the maximum likelihood estimator (MLE). This analysis
results in 24 different exponentially fitted distributions. Patients
4.1.1. Data collection and analysis differ according to their medical complaints and the severity of
The data collected for this project utilized both quantitative their care needs. It was, therefore, essential to understand their
and qualitative data types. The quantitative data was collected different arrival patterns to reflect the overall characteristics and
from the historical ED logs, electronic patient records (EPRs) from needs of various groups of patients.
the ED’s IT system, and direct observation. The direct and indirect The patients are clustered based on their triage category,
time per activity are not stored in their IT system and were col- which enables the differentiation of those patients who will
lected from interviews and observations with staff. Furthermore, be directed to the AMAU. Urgent patients (triage category 3)
the qualitative data such as patient pathways and the process represent the largest group of new attendees to the ED annually
of conceptual modeling has been gathered through observation, (51% on average). They present to the hospital with a wide range
interviews and focus groups. The sources of each data element of medical complaints.
are summarized in Table 1. This data has been de-identified, so As mentioned previously, the AMAU in this study faces two
the patient ID was replaced by a generated number. This en- kinds of boarding issues: ED boarding and, internal AMAU board-
ables the tracking of patient pathways without identifying them. ing. In ED boarding, patients occupy between one to six AMAU
Demographic data included only age and gender. trollies, for a maximum of 12 h. In internal boarding case, patients
The sample data from all anonymized acute patients was gath- in the AMAU are waiting for inpatient ward beds or SSU beds in
ered retrospectively for six months for patients that presented order to release AMAU trollies.
to the ED and AMAU between January 1st, 2014 and June 30th,
2014. A total of 20,493 de-identified patient records from ED and 4.1.2. AMAU patient flow (DES model)
1520 patient records from the AMAU was collected through the Upon arrival at the ED and registration, walk-in patients (self-
hospital’s information system. This system is used by the staff referral or GP referral) remain in the waiting area to be triaged.
(e.g., administrators, doctors, and nurses) to record data about When a patient’s name is called, depending on the availability
each patient, through each stage of their care. All diagnostic of triage staff, the patient is assessed by a triage nurse. Based
and procedure types were considered and no exclusions were on patient condition and triage assessment by MTS criteria, each
made. Patient records were analyzed and qualitative information patient is assigned a triage category. Then, based on their severity
about patients’ arrival patterns, patient grouping and allocation level, medical patients can be directed to either the ED or the
were extracted. All days displayed high patient arrival numbers AMAU. Medical patients are eligible for the AMAU path if they
8 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

Fig. 6. Patient’s flow in AMAU.

Table 1
The data sources.
Data Source
• Patient arrival times, patient acuity, diagnosis and demographic data. • Historical data gathered from the EPR, ED and AMAU logs.
• Activities: registration, triage, seeing a doctor, treatment. • Historical data gathered from EPR.
• Duration of direct activities per patient. • Observations, shadowing, and interviews and group discussion.
• Patient flow: pathways, routing probabilities, conceptual modeling. • Historical data collected from EPR, interviews and observations.
• Human resource and non-human resource capacities: nurses, consultants, doctor, etc. • Interviews and group discussion.
• Number of AMAU and ED boarders. • AMAU logs.

arrive between 9 am and 6 pm and if they are assigned a triage arrivals depends on a variety of factors and the characteristics
category 2 or 3. Once these requirements are met, the triage of the surrounding catchment area. The rate of discharge back
nurse calls the AMAU’s consultant to check trolley availability. to the community depends on patients’ medical health condition.
The patient goes back to the ED path if a trolley is unavailable. This is a function of the patients’ average LOS in the hospital ward
The majority of patients in the AMAU are medical patients, which and bed occupancy. The factors affecting both the arrival rate (de-
accounting 96% of the patients presented to the unit. mand side) and the average LOS in hospital ward are not modeled
Following the triage process, a patient who is directed to the in detail. Instead, they are modeled as exogenous variables. To
AMAU will be registered in the system, interviewed by a nurse, reduce the pressure on the ED, medical patients can be dispatched
where their blood pressure and vitals are measured and recorded. to the AMAU pathway, subject to the AMAU’s trolley availability.
Then they wait to be assessed by a doctor. Next, the AMAU The majority of the patients (75%) which present to the ED are
doctor will discuss the case with the unit’s consultant, who either discharged back to the community after receiving their treatment.
If further inpatient care is required, patients will wait in the ED
asks for more tests, requests an opinion, or decides whether the
for an inpatient bed. The rate of admission depends on both bed
patient needs to be admitted or discharged. These are the primary
management and hospital bed occupancy. Due to the high bed
care stages, which are relevant for all AMAU’s patients, whether
occupancy of inpatients beds, patients that need inpatient care
they are discharged from or admitted to the hospital. Secondary
are boarded (delayed) in the ED while blocking ED trollies.
patient stages are steps involved in the care of some, but not all A similar situation occurs in the AMAU when patients require
patients, such as diagnostics (e.g., MRIs and CTs). The steps of the a further stay in the hospital. They should move to other down-
AMAU is depicted in the flowchart in Fig. 6. stream units such as the SSU or inpatient beds. To free-up the ED’s
blocked trollies, ED management transfers some patients to the
4.1.3. Interdepartmental interaction (SD model) AMAU overnight. This management practice just moves the ED
The casual loop shown in Fig. 7 can be conceptualized with bed-blocking partially from the ED to the AMAU. Consequently,
two main areas: The community area (i.e., outside of the hospital) the AMAU has an average of 3.2 blocked trollies due to the
and, the hospital area (i.e., the ED, AMAU, SSU and inpatient transferred ED boarded patients. The situation becomes worse
wards). Regarding the community, patients present to the ED when it is combined with the AMAU boarded patients. In turn,
from the hospital’s surrounding catchment area and many are these limit access to the AMAU and subsequently increase the
then discharged back to the community. The rate of patient pressure on the ED.
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 9

Fig. 7. A simplified causal loop diagram.

Fig. 8. The interaction between the DES and SD models. (For interpretation of the references to color in this figure legend, the reader is referred to the web version
of this article.)

4.2. Phase 2: Modeling complexity and dynamics simulation package. DES components include all detailed aspects
of the AMAU and the SD to capture the dynamic complexity of
Analyzing the patient boarding problem in the AMAU necessi- the inter-departmental interactions. The two simulation models
tates integrating with downstream and upstream facilities which run simultaneously, and the information is exchanged from both
show high interdependency, in particular, the ED and inpatient models in the runtime with a parallel interaction. In this hybrid
wards. A better understanding of the problem and its implica- simulation model, DES interacts dynamically with the wider SD
tions can be accomplished when system integration is considered. model. SD captures patients’ arrivals to ED, as well as the inter-
Therefore, a hybrid SD-DES simulation model is developed to ad- dependencies and relationships between capacities, along with
dress the consequences of patient boarding in the AMAU (Fig. 8). the LOS for patients in various treatment units (e.g., SSU and
Red arrows feed information into the DES from SD, blue arrows ward). The outputs of SD provides the daily demand for the DES
indicate change events from DES to SD, while black shows the model in terms of the patients which are referred from the ED
flow of changes inside the SD. to the AMAU in order to complete their medical processes. This
The upstream component is the ED, which is the demand data becomes the parameters for the arrival distributions at the
source of the AMAU. While the downstream components model AMAU. Moreover, the DES used to model patient dispositions in
the patient disposition. Patient disposition refers to two cases in the AMAU. In return, the main outputs of DES such as the number
AMAU: First, patients who are waiting for beds in other units of patients admitted to the wards, and the number of patients
to release AMAU trollies. Second, patients who been transferred admitted to the SSU are sent to the SD model automatically.
to the AMAU overnight in order to free-up ED’s blocked trollies. Thus, continually changing elements represented by the SD cause
Patient flow in the AMAU is modeled using a DES to simulate changes in the discrete variables, and discrete variables cause
the unit’s activities and processes in detail, taking into account changes in the continuous elements.
the interaction between resources. Downstream and upstream All model inputs are stored in a database attached to the sim-
hospital operations are explained in the SD model, which enables ulation model. The model output is exported to an excel database
managers to envisage the impact of changes using feedback loops for further analysis and validation. The simulation model also
between the different activities. Based on the analysis and con- considers different types of medical staff, including nurses, senior
ceptualization, a comprehensive simulation model for the ED and house officers (SHOs), registrars, and consultants in the ED and
AMAU is constructed using the ‘‘Any Logic 7 University Researcher’’ AMAU. Furthermore, non-staff resources have also been included,
10 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

Table 2
Model validation.
Run LOS (Min) Patient access Discharge home Admitted
1 279.77 1749 795 954
2 276.39 1697 765 932
3 277.20 1708 803 905
4 275.65 1746 811 935
5 277.52 1735 770 965
6 271.96 1668 753 915
7 276.00 1688 740 948
8 280.79 1703 739 964
9 277.42 1700 748 952
10 275.23 1704 745 959
Summary statistics
Mean 277 1710 767 940
Stdev. 2.44 25.92 27.04 18.24
Half CI 1.74 18.52 19.32 12.97
LB 275.05 1691.48 747.68 952.97
UB 278.53 1728.52 786.32 927.02
Actual 267.6 1520 734 786
% Diff 3.4 12.5 4.4 19.5

such as beds, physical spaces, and diagnostic rooms. Access to Table 3

diagnostic services and inpatient care has also been modeled. The Simulation results for a baseline (95% confidence intervals).
model tracks the individual patients through their journey in the KPIs Baseline scenario
hospital in order to quantify the amount of care required and for Avg. LOS(Minutes) 277 ± 1.74
their interactions with the staff. Avg. Patient access 1710 ± 18.52
Avg. Number of discharged patients 767 ± 19.32
To reduce the model development cycle time and to increase
Avg. Number of admitted wards 460 ± 7.92
confidence in the results, verification and validation was executed Avg. Number of admitted SSU 480 ± 10.06
throughout the development phases of the simulation model.
After each development phase, the model was verified and val-
idated in conjunction with completed phases. For the verification
process, the model logic is verified to ensure that patients follow 4.3. Phase 3: Experimentation and decision-making
the correct care path as expected. This was achieved by tracing
and debugging functions besides the visual tracking of patients. Different scenarios including boarding experiments, and stock
It was also achieved through checking intermediate output values and flow interventions, which impact the boarding problem, were
such as queue lengths and waiting times between processes. The tested. The DEA model is then used to rank the scenarios and
conceptual model was documented and validated by circulating choose the best strategy to alleviate the problem.
the document among the ED and AMAU senior consultants and
senior nursing staff (more details of the model is provided in 5. Findings and results discussion
Appendix A).
The final results of the simulation model were validated using The simulation results of the baseline show that on average
the two following techniques: face validation and, comparison 1710 patients had access to the AMAU unit over 24 weeks. In
testing. Face validation is performed by interviewing the ED and the current situation, the AMAU works 9h/5days a week; from
AMAU senior consultants and nursing staff in order to validate 9 am to 6 pm. According to the baseline, only 40% of the medical
the final results of the simulation model. The verification and patients in the ED could access the AMAU. However, the goal of
validation were carried out through all the development phases the ED’s managers is to increase the flow to the AMAU to absorb
of the model. Visual tracking was used to verify the model logic to most if not all medical patients who attend to the ED during its
ensure that patients follow the correct care path as expected and opening hours. The average LOS of the baseline – presented in
are going to the expected units. Before running the simulation Table 3 – is currently 277 min and 1710 patients did access to
model, the following Key Performance Indicators (KPIs) were used the AMAU. Of that 1710 patients, 767 patients were discharged
for the AMAU: the average of AMAU LOS, and patient access to home, while 460 and 480 patients moved to inpatient wards
(i.e., the total number of patients enter the AMAU). The main and the SSU, respectively.
reason for choosing patient access as one of the main KPIs is that
increasing patients access to the AMAU can mitigate overcrowd- 5.1. Boarding experiments
ing in the ED [53]. The second approach was comparison testing,
which is achieved by comparing the output of the simulation In the AMAU, there are three types of patients. The first
model with the real output of the system under identical input type is medical patients who are in the process of treatment
conditions. A sample output of selected KPIs for the AMAU is in the unit. The second type is the medical patients who have
given in Table 2. Prior to the application of this approach, the completed their treatment in the unit and are waiting to be
number of replications was determined using both graphical and admitted to an appropriate inpatient bed. This type of patient
confidence interval methods [103,104] (more details are provided is referred to as ‘‘internal AMAU boarders’’. The third category
in Appendix B). is the ‘‘ED boarders’’ who are clinically unnecessarily transferred
The model runs for a twenty-four week period with a warm- to the AMAU in order to free up ED trollies. Several scenarios
up period of eight weeks. The model is also run ten times for were designed to investigate the boarding problem in the AMAU
each scenario. Each run has different random seed checks to and identify the factors which contribute to LOS and patients’
ensure that the results have an accurate reflection and precisely access to this unit. Two factors are examined in this experiment
represent the AMAU. Then, the average of ten runs is reported for ED boarders and internal AMAU boarders (Table 4). The value of
the discussion of the results. ‘YES’ to the ED boarders means allowing the ‘ED boarded’ patients
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 11

Table 4
Boarding scenario design.
Allow ED boarders Allow internal AMAU boarders
Base scenario Yes Yes
Scenario 1 No Yes
Scenario 2 Yes No
Scenario 3 No No

Table 5
Simulation results for boarders scenarios (95% confidence intervals).
KPIs Base Scenario 1 Scenario 2 Scenario 3
O/Pa (%) O/Pa (%) O/Pa (%)
Avg. LOS (Minutes) 277 272 ± 2.9 −1.8 251 ± 1.5 −9.3 286 ± 1.8 3.2
Avg. Patient access 1710 2152 ± 49.0 25.9 2337 ± 57.1 36.7 3052 ± 13.4 78.5

a. O/P: simulation output; % percentage of increase/decrease relative to the base scenario.

Table 6
Scenarios stock and flow factor analysis.
Scenarios Stock factors Flow factors KPI
Number of SSU beds Number of ward beds LOS SSU LOS ward Patient access
Base 24 520 105 162 1710
1 18 390 79 122 1678
2 18 520 105 162 1189
3 18 650 131 203 724
4 24 520 131 122 1350
5 24 650 79 162 1795
6 30 650 105 122 1865

to be transferred to the AMAU. On the other hand, the value

‘YES’ for the ‘internal AMAU boarders’ means allowing the AMAU
admitted patients to be boarded in the unit when they waited for
an inpatient bed. The value ‘NO’ for both factors corresponds to
the immediate transfer of patients downstream. This value can be
seen as a relaxation of the capacity constraint in the downstream
hospital facilities.
Scenario one assumes constant inpatient beds availability for
ED patients, in order to prevent ED boarders from being trans-
ferred to the AMAU. This scenario resulted in a significant in-
crease in patient access to the AMAU by 25.9% (i.e., from 1710
to 2152 patients). This improvement is attributed to the increase
in the availability of AMAU trollies as a result of less waiting time.
However, there is an insignificant reduction in the LOS of AMAU Fig. 9. Percentage of waiting times during different procedures in AMAU.
patients (−1.8% of the baseline). The second scenario (Table 5)
seems to be more efficient than the first one, regarding a patient
access increase (from 1710 patients to 2337 patients) and reduc-
tion in LOS (9.3%). Scenario 3 is the ideal situation for the system, factors investigated are the number of beds in SSU and number
of inpatient ward beds. Through discussion with stakeholders,
in which there is no ED boarding and internal boarding. In this
the flow factors included in these experiments are the average
scenario, there is a noticeable increase of 78.5% in patient access.
LOS in both the SSU and inpatient ward. Table 6 summarizes the
However, LOS has not been significantly improved, and on the
scenarios used to examine their impact.
contrary, it was slightly increased (3.2%). The substantial increase
The evaluation of these scenarios is presented in Fig. 10. Re-
in patient access can partially explain this. There are two ways
sults show that the SSU beds are positively linearly correlated
to achieve this scenario: either increase bed capacity (stock inter-
to the patient access, with R2 = 42.9% (Fig. 10a). The results
ventions) or, reduce LOS of patients in downstream facilities/units confirm that more SSU beds facilitate in reducing the internal
(flow interventions). This will be explored in Section 5.2. boarding and therefore increases the patient access. However, the
A thorough analysis of the AMAU queues (Fig. 9) shows that capacity expansion of the inpatient ward has insignificant linear
the majority of patients are waiting for transfer, (from the ED to effect with R2 = 1.6% (Fig. 10b). The outcomes of this capacity
the AMAU namely ‘Patient pass’), and waiting to be seen by a expansion policy in the model reveal that increasing inpatient
consultant. The results have raised the need for further experi- bed capacity only provides a temporary solution, and its im-
mentations to identify the most significant factors which deter- pact is insignificant over longer periods [11]. When the inpatient
mine the performance of the AMAU. Therefore, a DoE along with bed capacity is increased, simply more patients are admitted.
multivariate analysis was conducted and presented in Section 5.3. Therefore, the effect of that capacity expansion can only offer a
temporary solution. As soon as the additional capacity is used
5.2. Stock/flow interventions up, the situation gradually deteriorates again. Capacity expansion
is an example of stock interventions, which have time-limited
In this analysis, various scenarios are examined to explore effects and may even stimulate more demand.
the impact of stock and flow factors identified in the SD level On the other hand, the average LOS in both the SSU beds
on patient access to the AMAU defined in the DES model. Stock and inpatients beds are examples of flow interventions. Fig. 10c
12 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

Fig. 10. Scatter plots of the stock and flow factors against the patient access to the AMAU.

and d show a negative linear relationship between patient access In order to test the significance of the input parameters,
and both average LOS in SSU (R2 = 48.3%) and inpatient ward MANOVA was used. MANOVA is very similar to ANOVA for testing
(R2 = 43.1%). The results showed that flow interventions can
the significant differences between two or more groups of partici-
be more effective in increasing patient access, and are more
likely to be effective in improving healthcare system performance pants. However, MANOVA is appropriate when the study involves
than implementing stock interventions separately. Creating new more than one criterion variable. In this study, MANOVA analysis
channels to reduce emergency admissions and the average LOS is performed on responses in order to indicate the influence or
can make a significant improvement.
significance of input parameters on the AMAU KPIs. The MANOVA
table for all responses and variable interactions are presented in
5.3. Multivariate analysis of variance
Table 8.
An efficient DoE can reduce the experimental effort and sim- By considering p-value = 5% and within the selected L27’s
plifying the task of interpreting the results by identifying the range, the results indicate the number of porters, consultants,
factors which are essential for each response. Also, DoE helps to trollies, ED boarding time, the time waiting for SSU and ward
specify the different effects on the responses and factor inter-
beds, are significantly related to the LOS and patient access. How-
action [105] by using seven different factors, with three levels
(i.e., L27). The Taguchi orthogonal array (OA) factorial designs ever, the average number of blocked trollies in the ED boarding
provide the possibility to consider a subset of combinations of case by p-value = 0.5783790 is insignificant. This result shows
multiple factors run at different numbers of levels. OAs are bal- that the blocking time of trollies has a higher impact on the KPIs
anced to ensure that all levels of all factors are considered equally.
than the number of blocked trollies. Also, two-way interactions
Table 7 summarizes the factors used and designs develop the DoE.
Parameters are divided into two groups: 1. Controllable ones, applied to the data in the investigation of interactions among
which are directly observable such as the number of porters or the independent variables. According to the results, the number
trollies and, 2. Uncontrollable ones, which require statistical in- of porters is significantly related to LOS and patient access. The
ference such as an average number of ED boarders. Two responses
number of porters also has a significant interaction (p < 0.05)
which were used in this experiment, include the average LOS
in the AMAU and the average number of patients accessing the with the number of consultants and the number of trollies which
AMAU. In these designs, for example, scenario one means there implies the effect of trollies and consultants to improve KPIs
is one porter, nine trollies, and one consultant while one trolley is depends on the number of porters. In addition, a significant inter-
blocked for 4 h due to the transferring of patients from the ED to
action (p < 0.05) were found between the number of consultants
the AMAU overnight. Also, the average waiting time for accessing
ward beds and average waiting time for accessing SSU beds, (in and trollies. However, according to p-values, no other interactions
order to release AMAU trolley) is one hour. have been seen between other factors.
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 13

Table 7
A description of DoE variables and responses with L27 simulation designs.
Factors
Type Variable Description Levels
P1 : Avg. ED boarding time 3L: 4, 8 and 12
Uncontrollable P2 : Avg. number of blocked trollies in ED boarding 3L: 1, 3 and 6
P3 : Avg. waiting time for inward beds (i.e., Internal boarding) 3L: 1, 2 and 3
P4 : Avg. waiting time for SSU beds (i.e., Internal boarding) 3L: 1, 2 and 3
X1 : Number of porters 3L: 1, 2 and 3
Controllable
X2 : Number of trollies 3L: 9, 11 and 16
X3 : Number of consultants 3L: 1, 2, and 3
Responses Variables (Predictors):
Y1 : Avg. LOS in AMAU
Y2 : Avg. Patient access
L27 Simulation Designs
Controllable Variables Uncontrollable Variables KPIs
Design X1 X2 X3 P1 P2 P3 P4 LOS Patient access
1 1 9 1 4 1 1 1 272 ± 1.86 2239 ± 34.82
2 1 9 1 4 3 2 2 305 ± 1.98 2118 ± 31.89
3 1 9 1 4 6 3 3 332 ± 2.63 1984 ± 13.79
4 1 11 2 8 1 1 1 260 ± 1.46 2200 ± 8.67
5 1 11 2 8 3 2 2 290 ± 1.07 2112 ± 11.06
6 1 11 2 8 6 3 3 323 ± 2.42 2047 ± 18.26
7 1 16 3 12 1 1 1 265 ± 3.47 2386 ± 15.77
8 1 16 3 12 3 2 2 295 ± 2.98 2354 ± 25.39
9 1 16 3 12 6 3 3 329 ± 1.14 2307 ± 15.34
10 2 11 1 12 1 2 3 297 ± 3.31 1846 ± 26.53
11 2 11 1 12 3 3 1 282 ± 5.88 1931 ± 11.63
12 2 11 1 12 6 1 2 267 ± 3.28 1954 ± 13.78
13 2 16 2 4 1 2 3 290 ± 2.15 2488 ± 6.95
14 2 16 2 4 3 3 1 273 ± 3.08 2493 ± 16.09
15 2 16 2 4 6 1 2 257 ± 4.32 2496 ± 18.87
16 2 9 3 8 1 2 3 287 ± 6.04 1873 ± 22.15
17 2 9 3 8 3 3 1 269 ± 2.55 1955 ± 10.79
18 2 9 3 8 6 1 2 253 ± 2.40 1989 ± 14.86
19 3 16 1 8 1 3 2 300 ± 3.34 2395 ± 19.68
20 3 16 1 8 3 1 3 287 ± 2.32 2422 ± 11.33
21 3 16 1 8 6 2 1 266 ± 1.84 2421 ± 10.84
22 3 9 2 12 1 3 2 285 ± 2.28 1550 ± 15.57
23 3 9 2 12 3 1 3 271 ± 1.68 1629 ± 10.16
24 3 9 2 12 6 2 1 255 ± 2.94 1627 ± 10.27
25 3 11 3 4 1 3 2 287 ± 1.35 2422 ± 17.36
26 3 11 3 4 3 1 3 272 ± 2.14 2436 ± 19.40
27 3 11 3 4 6 2 1 254 ± 1.51 2453 ± 15.93

Table 8
MANOVA for LOS and patient access and two-way interactions between different factors.
Factors D.F. Pillai Approx. F Num. D.F. Den D.F. Pr. (>F)
X1 (Porter) 1 0.97173 68.75 2 4 0.0007992 ***
X2 (Consultant) 1 0.98223 110.52 2 4 0.0003159 ***
X3 (Trolley) 1 0.99816 1082.95 2 4 3.398e-06 ***
P1 (ED boarding time) 1 0.99709 685.88 2 4 8.453e-06 ***
P2 (Number of blocked trollies in ED boarding) 1 0.23949 0.63 2 4 0.5783790
P3 (Waiting for ward beds) 1 0.99388 324.68 2 4 3.748e-05 ***
P4 (Waiting for SSU beds) 1 0.99441 355.63 2 4 3.127e-05 ***
X1 :X2 1 0.86957 13.33 2 4 0.0170112 *
X1 :X3 1 0.98366 120.38 2 4 0.0002671 ***
X1 :P2 1 0.31838 0.93 2 4 0.4646111
X1 :P3 1 0.24724 0.66 2 4 0.5666508
X2 :X3 1 0.90862 19.89 2 4 0.0083500 **
X2 :P2 1 0.56800 2.63 2 4 0.1866280
X2 :P3 1 0.46795 1.76 2 4 0.2830789
X2 :P4 1 0.44654 1.61 2 4 0.3063215
X3 :P2 1 0.56393 2.59 2 4 0.1901581
X3 :P3 1 0.02828 0.06 2 4 0.9442465
X3 :P4 1 0.04237 0.09 2 4 0.9170541
P1 :P2 1 0.01317 0.03 2 4 0.9738276
P1 :P3 1 0.12482 0.29 2 4 0.7659392
P5 :P6 1 0.34373 1.05 2 4 0.4306851

Significance codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1.
14 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

Table 9
The score for each scenario by output-oriented BCC model.
Rank DMU/Design Score Rank DMU/Design Score
1 D27 1.301238 15 D1 1
2 D18 1.289810 16 D8 0.986507
3 D15 1.244203 17 D9 0.966728
4 D4 1.158426 18 D17 0.961707
5 D21 1.093559 19 D2 0.946221
6 D24 1.083322 20 D5 0.925746
7 D12 1.082793 21 Baseline 0.921769
8 D20 1.081651 22 D23 0.905874
9 D14 1.072572 23 D6 0.897512
10 D7 1.070592 24 D3 0.886189
11 D19 1.069948 25 D11 0.842808
12 D13 1.067622 26 D16 0.836520
13 D26 1.053364 27 D10 0.808481
14 D25 1.048632 28 D22 0.785705

After completing DoE and analyzing different scenarios and scenario, the number of AMAU trollies did not change. However,
results, the designs are used as an entry for the DEA model to two porters and two consultants were added. Six trollies are
rank the scenarios and choose the best strategy. blocked for 4 h while in the current situation between 1 to 6
AMAU trollies are blocked for a maximum of 12 h each. Also,
5.4. DEA results waiting time for accepting a patient to the inward or SSU are 2
and 1 h, respectively. This scenario results in an 8.2% decrease
The ultimate objective of this study is to see the trade-off in the LOS (from 276.8 to 253.9) and 43.42% increase in patient
between scenarios and determine the most efficient scenario. access (from 1710 to 2452.6). Scenario 18 with an efficiency equal
Therefore, DEA is used to rank the scenarios and select the most to 1.28 ranks second, which leads to a reduction in the LOS by
efficient one. DEA, as a mathematical approach, is one of the most 8.5% and 16.3% increase in patient access. To compare scenario 18
robust methodologies used to evaluate the efficiency of decision- with scenario 27, the number of porters is two, and the number of
making units (DMU), as it considers multiple inputs and outputs. trollies decreases from 11 to 9. Also, six AMAU trollies are blocked
Also, due to the complexity and variety of measures in a health- for 8 h while waiting for inpatient ward beds is 1 h. Therefore,
care context, DEA is considered as a useful method to provide a the blocking time of trollies plays a critical role and improving
valid model for decision-making [89]. More details about different waiting times for SSU beds are effective than improving waiting
DEA models is provided in Appendix C. In constant returns to for ward beds (i.e., internal boarding case) to DMUs efficiency.
scale (CCR) and variable returns to scale (BCC) DEA models, DMU Furthermore, in both scenarios decreasing the ED boarding times
is efficient if w0 = 1 and DMUs with w0 < 1 consider inefficient. and internal boarding times as well as improving staffing levels,
In some cases, there is a possibility that multiple DMUs reach is more efficient than increasing trolley capacity in comparison
w0 = 1. Therefore, to overcome this problem, the super-efficiency with other scenarios. It can be concluded that improving the
technique is used by ranking efficient DMUs. The best DMU has flow between up and downstream units, as well as increasing the
the highest super-efficiency score [106] baseline and 27 scenarios porter and the consultant staff, enhances the patient access more
are considered as DMUs in the DEA model. As the focus of this than increasing the number of AMAU trollies.
study is on the outputs of the model, so the output-oriented DEA
models are applicable. The output-oriented BCC performs better 5.5. The managerial implication of study
compared with other models; it is thus chosen for ranking. It is
essential to fix the parameters of the problem with the structure Due to the complex and systemic causes for patient flow de-
of the model before using the model to calculate efficiencies and lays, simple local solutions are found to be ineffective. This study,
rank scenarios. The output-oriented BCC model maximizes the firstly, offers useful insights to hospital managers in order to
outputs of the model (larger the better type). However, in this better comprehend the underlying dynamic interactions between
model, the LOS should be smaller. Therefore, equation one is the different elements of patient flow and enhance their under-
applied to transform larger the better type to smaller the better standing of their systems. Thus, such models are instrumental in
type (e.g., LOS) and vice versa [107]. assessing various flow delays, and possible interventions. They
max (x) − x are also essential in finding the main causes of poor patient
x= (1) flow and offering solutions to support patient flow management.
max (x) − min(x) This study also highlights the benefit of focusing on care pro-
Eq. (1) normalizes the data and gives the values between 0 and cesses and considering their interdependency at various care
1 and also change their type. Now, by using the output-oriented stage. This is achieved using system-wide approaches to improve
BCC model on the standardized data, it is possible to calculate patient flow delays, through the continuum of care. In other
efficiencies of different DMUs. The initial results showed multiple words, removing barriers to improve the patient flow process
DMUs reach w0 = 1, so to solve this problem and discriminate requires including all the relevant departments and their interde-
among efficient DMUs, the super-efficiency technique is applied pendencies, rather than examining the role of each department
to rank them (Table 9). The results for current and optimal sce- individually. Therefore, it is recommended the hospital admin-
narios show that the current design of the AMAU has a lower istrators and healthcare planners attempt to provide solutions,
performance than 20 defined DMUs. which consider consecutive steps in the care process, in order
DMU 27 is the best with the efficiency score of 1.30. It is to ensure continuous care delivery. Finally, the insights from this
ranked as the DMU with the best overall performance. In this study provide opportunities for future research and practice. It
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 15

may guide future researchers and hospital managers in adopting Acknowledgments

hospital-wide approaches and exploring the relationship between
downstream and upstream departments. There also should be This research is funded by the Fiosraigh Scholarship Pro-
a focus on shared resources, services between units, enhancing gramme of the Technological University Dublin (TU Dublin). We
communication and coordination across the hospital in order to thank Prof. Garry Courtney (AMP Program Lead, HSE) and Prof.
successful patient flow improvement. Sean Kennelly (Consultant Physician in Geriatric and Stroke
Medicine) for their support to facilitate the data collection.
6. Conclusion
Appendix A. Model documentation
Fundamentally, boarding patient problem has two interre-
lated issues. The first issue concerns considerable delays that
occur when transferring patients to their appropriate medical A.1. Objectives
care units. This matter has become a silent feature of over-
crowded EDs, where patients wait in areas which are not desig- A.1.1. Purpose of the model
nated for clinical care such as treatment rooms, corridors, chairs, This study presents an integrated hybrid approach to inves-
or on trollies. The second issue is related to the inappropriate tigate the consequences of patient boarding in the AMAU and
placement of admitted patients. This has a direct relationship improve the flow of patients across hospital units.
to the overcrowding that is increasing the pressures to meet
the national targets. Therefore, due to the aforementioned con- A.1.2. Model outputs
straints, decisions to move patients to other areas which are There are two main KPIs for this study:
not suitable for patient conditions can be argued. Although the
ED’s boarding problem has received considerable attention in the 1- Average patient LOS (i.e., the total time from patients’ entrance
literature recently, the problem in other hospital units has not to the unit until their exits the unit).
been adequately addressed.
2- Average patient access (i.e., the total number of patients enter
This paper presents a system-wide approach to examine the
the AMAU).
boarding problem in order to improve the flow of patients across
hospital units. A hybrid system of DES, SD, DoE and DEA is devel-
A.1.3. Experimentation aims
oped to provide a comprehensive understanding of the dynamic
relationships between hospital units and their interdependencies. Several scenarios were designed to investigate the board-
SD component captures the interactions and feedback dynamics ing problem in the AMAU by considering the interdependency
between the AMU and connected units effectively (i.e., ED and between downstream and upstream facilities, in particular, ED
medical wards). The DES element enables the identification of and inpatient wards. These including baseline scenario, boarding
bottlenecks, which impede system performance. Incorporating experiments, stock and flow intervention scenarios; and the com-
DoE and DEA allows unit managers to evaluate the efficiency of bination of first group scenarios with adding different resources.
a diversity of capacity, staffing and patient flow scenarios. The Moreover, to support simulation as a tool for experimentation,
proposed methodology provides a practical guide for users of hy- the DoE was used for the evaluation of scenarios and identify
brid system that can help understand and model the complexity the most significant factors affecting the overall performance.
of healthcare systems. Finally, DEA was applied to evaluate and rank the best scenarios
This study also highlights that the patient flow, as a prop- in improving AMAU performance.
erty of the entire care system, can only be truly optimized at
the system level. Indeed, flow improvement requires the design A.2. Logic
and evaluation of interventions in different parts of the system.
Therefore, to support patient flow improvement, hospital admin- A.2.1. Base model overview diagram
istrators and healthcare planners should view the whole care Fig. 5 demonstrations the generic patient pathways through
process while considering many interactions between patients, the hospital with the AMAU/SSU which is the broad conceptual
clinicians, different services and resources. Results also indicate model for the simulation. Fig. 6 also present the patient’s flow in
focusing on improving the linkages and connections between AMAU while Fig. 7 present a simplified causal loop diagram of the
different care process steps, provide new opportunities to alle- SD model.
viate the patient flow bottlenecks without more investment in
expanding capacity. A.2.2. Base model logic
Unnecessary delays in discharging patients from inpatient
wards do increase the number of boarders in connected upstream SD basic model logic:
units (i.e., units competing for inpatient beds such as ED, AMU, First, patients arrive at the ED. Then, patients are triaged and
and ICU). Improving the process of patient discharge (on-time assigned a triage category according to the MTS. Patients routed
discharge of patients and the reduction of patient LOS), can to the AMAU are those medical patients triaged as category 2
accelerate transfer rates between hospital units and minimize the or 3 (i.e., very urgent and urgent patients respectively) who do
inappropriate placement of patients. Sustaining efficient flow of not require resuscitation or isolation facilities. The triage nurse
patients requires adequate staffing levels to deal with the upsurge usually contacts the AMAU consultant or registrar so that they
in the rate of patients. can accept or reject the case. These patients only transferred to
the AMAU if a trolley is available. Otherwise, patients should
CRediT authorship contribution statement stay in the ED to finish their treatment. All the patients from
the ED finally will be discharged to the home or transfer to
Leila Keshtkar: Conceptualization, Methodology, Software, the inpatient setting. In this model inpatient wards also receive
Analysis, Writing - original draft. Wael Rashwan: Conceptualiza-
elective patients and patients from the AMAU. In the last stage,
tion, Methodology, Software, Data collection, Validation, Writing
patients after completing their treatment in the inpatient setting
- review & editing. Waleed Abo-Hamad: Supervision, Investiga-
will leave the hospital.
tion, Writing - review & editing. Amr Arisha: Project administra-
tion. DES basic model logic:
16 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

After transferring the patients from ED, patients presented in List of resources:
the AMAU will get to see a senior medical doctor, who should Registrar, AMAU nurses, senior house officers (SHOs), AMAU con-
be able to treat and discharge patients, within a specific period sultant, porter, cleaners and AMAU trollies.
of admission. AMAU acts as the first gateway for acute medical
patients referred from the ED, while the SSU is used by patients Schedules:
who need to be admitted to the hospital but their estimated AMAU opening hours: 6 am to 9 pm.
length of stay is below a certain threshold. Patients from AMAU Patient access hours to AMAU: 6 am to 6 pm.
after completing different processes can be discharge home, sent
to SSU or admitted directly to hospital clinical wards. List of queues:
Seize a trolley: First in first out.
Renege: If a trolley is not available.
A.2.3. Scenario logic
Registration: First in first out.
In the AMAU, there are three types of patients. The first type
Preparation in AMAU: First in first out.
is medical patients who are in the process of treatment in the
unit. The second type is the medical patients who completed their Assessment by AMAU doctor: First in first out.
treatment in the unit and waiting to be admitted to an appro- Different test (e.g., blood test, ECG, X-ray): First in first out.
priate inpatient bed. This type of patient is referred as ‘‘internal Results of a test: First in first out.
AMAU boarders’’. The third category is the ‘‘ED boarders’’ who Consultant: First in first out.
are clinically unnecessarily transferred to AMAU in sake to free
Specialty team stage: First in first out.
up ED trolleys. Several scenarios were designed to investigate the
boarding problem in the AMAU. First group scenarios consider as Decision: First in first out.
‘‘boarding experiments’’ which mainly focus to explore the effect Accept to ward: First in first out.
of existing and removing different types of boarding on KPIs. Accept to SSU: First in first out.
Second group scenarios, consider as ‘‘stock/flow interventions’’
Entry/Exit Points:
which investigate the effect of changing the number of beds in
SSU and number of inpatient ward beds as well as changing LOS Entry point: Patients are sent from ED (in the SD model) to AMAU
in both the SSU and inpatient ward on KPIs. Third group scenarios, Exit Points: Home, SSU and Wards.
combine the first group scenarios with adding different resources.

A.2.4.2. SD.
A.2.4. Components Stocks: Stocks (i.e., state variables) are the accumulations that
A.2.4.1. DES.
characterize the system’s state.
Entities:
–Patients in Triage: Accumulation of the patients after their ar-
The main entities of the model are patients that arrive in the
rival in order to continue the care in the ED or AMAU.
ED from the community and each of them is assigned a set of
–Patients in ED: Those patients that are not medical patients with
attributes such as a triage category and clinical groups (medical
or nonmedical). triage category 2 or 3, as well as the patients renege from the
In this model, patients flow through the system from ED (in AMAU to ED are accumulated in this stock.
the SD model) and medical patients (triage category 2 and 3) –ED patient waiting for the ward admission: Store the patients
will send to AMAU. These patients pass different care stages from ED to admit to the wards.
while resources are identified and assigned to them during their
-AMAU patient waiting SSU bed: Patients from AMAU accumulate
journey. Next, after finishing their treatment in the AMAU, they
in this stock in order to receive a bed in SSU.
will leave the unit to home, SSU or inpatient wards.
–AMAU patient waiting for the ward admission: Patients from
List of activities:
AMAU accumulate in this stock in order to receive a bed in the
–Transfer to AMAU: Porter transfers the patient from ED to
AMAU. inpatient setting.
–Registration: Patient is registered and seize a trolley. –Scheduled elective waiting: Elective patient baulk in this stock
–Preparation in AMAU in order to receive a bed in the wards.
–Assessment by AMAU doctor –Patients in SSU: Patient baulk in SSU for a specific period to
–Different test: Some patients require to do the different test
complete their treatment.
(e.g., blood test, ECG, X-ray).
–Patients in the wards: Patient baulk in the wards for a specific
–Consultant: The results from the previous stage will discuss by a
consultant. The consultant will decide the patient may require a period to complete their treatment.
further text (e.g., MRI, CT) or should assess by the specialty team –ED patient waiting for AMAU admission: Patients from ED accu-
as well. Some patients also discharge to home directly. mulate in this stock in order to receive a bed in the wards.
–Specialty team stage
–Decision: Decision regarding discharge destination. Flows: Flow variables are rates or control variables that can
–Discharge: Patient will discharge to home, SSU or wards. change the state (i.e., the stocks) of the system.
–Cleaning –List of inflows with the equation:
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 17

ED patient arrival Arrival rate/time scale Graphical functions/lookup tables: N/A

Renegade rate to ED Number Renege
Source and sinks: A source represent the inflow that comes from
Patient in triage to ED (Patient in triage * (1-AMAU the cloud. Sinks are the outflows that go to the cloud.
Admit Fraction))/(Triage time* –List of sources:
Time Scale)
ED arrival, renegade patients, scheduled patient in the wards and
Ward admission rate Patients in ED * ED Admit other ED daily admission.
from ED Fraction/(Patient Time In ED* –List of sinks:
Time Scale) Discharge patient from ED, discharge patient from SSU, discharge
Admission to ward Min (Waiting Ward Admission patient from the ward and elective cancellation.
/Delay Time to Move Patient,
Available Beds)
A.3. Data
SSU Admission Min (Waiting SSU Admission,
SSU Available Beds)
AMAU to SSU Min (AMAU Await SSU Bed, SSU A.3.1. Data sources
admission rate Available Beds — SSU Table 1 provides the details of the elements of data sources.
Admission)
AMAU to ward rate Min (AMAU Await ward Bed, A.3.2. Input parameters
(Available Beds-Ward Input parameters for DES model summarized in the Table A.1.
Admission Rate))/Delay Time
Ward
The actual elective Min (Min (Desired Elective A.4. Experimentation
admission rate Admission Rate, Available
Beds-Ward Admission A.4.1. Initialization
Rate-AMAU to Ward Rate), The model runs for twenty-four weeks’ period with a warm-
Scheduled Elective Waiting) up period of 8 weeks. This period was found enough so that the
Schedule admission Desired Elective Admission model achieves a steady-state condition. No initial conditions are
rate Rate/(24* Time Scale) used for the DES model.
Other ED admission Daily Other ED Admission/(24* Initial values for SD model:
Time Scale) Patient in ward 400
Scheduled elective patients 200
–List of outflows with the equation:

ED discharge rate Patients In ED*(1- ED Admit A.4.2. Run length

Fraction)/(Patient time in ED* Time All point estimates are based on the average of 10 replications
scale) of a model run (More details provided in Appendix B). Each run
Sending patients to Admission AMAU has different random seed.
AMAU
SSU discharge rate Patient in SSU/(LOS SSU* Time scale)
A.5. Implementation
Ward discharge rate Patient in Ward/(LOS Ward* Time
scale)
A.5.1. Software or programming language
Feedback loops: DES and SD models are developed using AnyLogic 7 university.
Bed occupancy, waiting for the ward admission and waiting for
SSU admission.
A.5.2. Random sampling
Constants/auxiliary variables: All pseudo-random number generator was based on the de-
fault random number generator. The default random number
Arrival Rate Table Function Arrivals generator is an instance of the Java class Random, which is a
Week hour ((t/timescale)%168) Linear Congruential Generator (LCG).
AMAU Admit Fraction AMAU Fraction*AMAU Opened
T time () A.5.3. Model execution
SSU Available Beds Max (0, SSU bed Capacity-Patient One replication takes 1.89 min.
In SSU)
LOS SSU base SSU LOS + Change in SSU A.5.4. System specification
LOS* base SSU LOS
The model was run on iMac, with a 3.1 GHz intel Core i5
LOS ward base Ward LOS + Change in Ward processor and 16 GB of memory and Macintosh HD.
LOS*base Ward LOS
Ward Available beds Max (bed Capacity-Patient in
Ward, 0) A.6. Code access

Daily Other ED Monthly Other ED Admission /30

Admission Models are available on request. (See Fig. A.1).
18 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

Fig. A.1. A screenshot of hybrid simulation model in Anylogic software.

Table A.1
DES model input parameter.
Activity Distribution Min Most likely Max
Registration Triangular distribution 1 5 7
Transfer patient Triangular distribution 3 5 10
Preparation Triangular distribution 5 10 15
Interview Triangular distribution 20 30 50
Processing Time (min) AMAU
Discussion Triangular distribution 5 10 15
Consult Triangular distribution 10 15 20
Specialty team Triangular distribution 90 180 300
Consult Decision Triangular distribution 5 10 15
Exit Triangular distribution 1 5 7
Clean Triangular distribution 1 5 7
Radiology (X-ray) Triangular distribution 30 40 60
Radiology Radiology (MRI) Triangular distribution 10 30 40
Radiology (CT) Triangular distribution 10 30 40
Radiology (US) Triangular distribution 10 30 40
Lab Blood test Triangular distribution 5 10 20

Appendix B. Finding the number of replications For a more accurate estimation of the number of replications,

the confidence interval was calculated for the initial number

To determine the number of replications both graphical and
confidence interval methods were applied. The graphical method of replications (Table B.1). As shown in Table B.1, the mean of
was used to provide a plot of the cumulative mean of the simu-
confidence intervals becomes narrower as more simulation repli-
lation output. As more simulation replications are performed, the
cumulative mean graph converts to a flat line (Fig. B.1). cations are performed. Therefore, ten replications were estimated.
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 19

Table B.1
Results from 11 replications for patient experience time.
Significance level 5.0% confidence interval
Cumulative. mean Standard Lower Upper %
Replication Result LOS Average Deviation Interval Interval Deviation
1 279.77 279.77 n/a n/a n/a n/a
2 276.39 278.08 2.390 256.61 299.55 7.72%
3 277.2 277.79 1.765 273.40 282.17 1.58%
4 275.65 277.25 1.794 274.40 280.11 1.03%
5 277.52 277.31 1.558 275.37 279.24 0.70%
6 271.96 276.42 2.589 273.70 279.13 0.98%
7 276 276.36 2.369 274.16 278.55 0.79%
8 280.79 276.91 2.696 274.66 279.16 0.81%
9 277.42 276.97 2.528 275.02 278.91 0.70%
10 275.23 276.79 2.446 275.04 278.54 0.63%
11 273.35 276.48 2.542 274.77 278.19 0.62%

Fig. B.1. The plot of cumulative mean and 95%confidence interval.

s m
Appendix C. Different DEA models ∑ ∑
ur yrj − vi xij + C0 ≤ 0 j = 1, 2, . . . , n (8)
r =1 i=1
Various DEA models have been used in the healthcare context
ur , vi ≥ 0, r = 1, 2, . . . , s and i = 1, 2, . . . , m (9)
including CCR and BCC models as the classic forms of DEA [93],
which are defined as follows. The CCR model presents a constant In the BCC model the use of data that present negative values are
return to scale: possible, which can happen in stochastic simulation models. Also,
s
this model in contrast to CCR model is invariant to the application
∑ of linear transformations to the inputs and outputs values [106].
max w0 = ur yr0 (2)
Both BCC and CCR model can be input or output-oriented. The
r =1
input-oriented model is used to minimize inputs while keeping
Subject to: the outputs at their current levels. The output-oriented model in-
m tent to maximize outputs while using no more than the observed
∑
vi xi0 = 1 (3) amount of any inputs [97].
i=1 To achieve sufficient discrimination of the DMUs in a tra-
∑ s m
∑ ditional DEA model, the number of DMUs should satisfy the
ur yrj − vi xij ≤ 0 j = 1, 2, . . . , n (4) following equation:
r =1 i=1
n ≥ Max{(ms), 3(m + s)} (10)
ur , vi ≥ 0, r = 1, 2, . . . , s and i = 1, 2, . . . , m (5)
where m is the total number of inputs and s is total the number of
where r ∈ { 1, . . . , s } is the output index, and s is total the number outputs; n is the total number of DMUs [108]. In CCR/BCC models
of outputs; i ∈ { 1, . . . , m } is the input index and m is the total DMU is efficient if w0 = 1 and DMUs with w0 < 1 consider
number of inputs; yrj is the rth output for the jth DMU; xij is the inefficient. In some cases, there is a possibility that multiple
ith input for the jth DMU; ur is the weight associated with the rth DMUs reach w0 = 1. Therefore, to overcome this problem, the
super-efficiency technique is used by ranking the efficient DMUs.
output; vi is the weight associated with the ith input; w0 is the
In fact, the best DMU has the highest super-efficiency score [106].
relative efficiency of DMU0 , which is the DMU under evaluation;
and yr0 and xi0 are respectively the outputs and inputs for DMU0 . References
To consider variable return to scale, BCC model was developed
by the addition of free variable C0 . [1] S.R. Levin, R. Dittus, D. Aronsky, M.B. Weinger, J. Han, J. Boord, D.
France, Optimizing cardiology capacity to reduce emergency department
s
∑ boarding: A systems engineering approach, Amer. Heart J. 156 (6) (2008)
max w0 = ur yr0 + C0 (6) 1202–1209.
r =1 [2] P. Walsh, V. Cortez, H. Bhakta, Patients would prefer ward to Emergency
Department boarding while awaiting an inpatient bed, J. Emerg. Med. 34
subject to: (2) (2008) 221–226.
m [3] M.J. Hodgins, N. Moore, L. Legere, Who is sleeping in our beds? Factors
∑
vi xi0 = 1 (7) predicting the ED boarding of admitted patients for more than 2 h, J.
Emerg. Nurs. 37 (3) (2011) 225–230.
i=1
20 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

[4] C.Y. Han, C.C. Lin, S. Goopy, Y.C. Hsiao, A. Barnard, L.H. Wang, Waiting [29] J. Viana, S.C. Brailsford, V. Harindra, P.R. Harper, Combining discrete-event
and hoping: a phenomenographic study of the experiences of boarded simulation and system dynamics in a healthcare setting: A composite
patients in the emergency department, J. Clin. Nurs. 26 (5–6) (2017) model for Chlamydia infection, European J. Oper. Res. 237 (1) (2014)
840–848. 196–206.
[5] J.M. Pines, R.J. Batt, J.A. Hilton, C. Terwiesch, The financial consequences of [30] M. Helal, L. Rabelo, J. Sepulveda, A. Jones, A methodology for integrating
lost demand and reducing boarding in hospital emergency departments, and synchronizing the system dynamics and discrete event simulation
Ann. Emerg. Med. 58 (4) (2011) 331–340. paradigms, in: Proceedings of the 25th International Conference of the
[6] S. Simpson, J. Joesch, I. West, J. Pasic, Who’s boarding in the psychiatric System Dynamics Society, Vol. 3, No. 3, 2007, pp. 1–24.
emergency service? West. J. Emerg. Med. 15 (6) (2014) 669–674. [31] A. Greasley, A comparison of System Dynamics (SD) and Discrete
[7] D.M. Fatovich, Y. Nagree, P. Sprivulis, Access block causes emergency Event Simulation (DES), in: 17th International Conference of the System
department overcrowding and ambulance diversion in Perth, Western Dynamics Society, 1999, pp. 20–23.
Australia, Emerg. Med. J. 22 (5) (2005) 351–354. [32] J. Morgan, S. Howick, V. Belton, Designs for the complementary use of
[8] R. Forero, S. McCarthy, K. Hillman, Access block and emergency system dynamics and discrete-event simulation, in: Proceedings of the
department overcrowding, Crit. Care 15 (2) (2011) 216. 2011 Winter Simulation Conference, 2011, pp. 2710–2722.
[9] Z. Wang, X. Xiong, S. Wang, J. Yan, M. Springer, R.P. Dellinger, Causes of [33] E. Tillal, B. Mariusz, S. Brailsford, N. Mustafee, R.E. Nance, B.S. Onggo,
Emergency Department overcrowding and blockage of access to critical R.G. Sargent, Hybrid simulation: Historical lessons, present challenges and
services in Beijing: A 2-year study, J. Emerg. Med. 54 (5) (2018) 665–673. futures, in: Proceedings of the 2016 Winter Simulation Conference, 2016,
[10] T. Bountourelis, L. Luangkesorn, A. Schaefer, L. Maillart, S.G. Nabors, G. pp. 1388–1403.
Clermont, Development and validation of a large scale ICU simulation [34] K. Chahal, T. Eldabi, T. Young, A conceptual framework for hybrid system
model with blocking, in: Proceedings of the 2011 Winter Simulation dynamics and discrete event simulation for healthcare, J. Enterprise Inf.
Conference, 2011, pp. 1143–1153. Manag. 26 (1) (2013) 50–74.
[11] W. Rashwan, W. Abo-Hamad, A. Arisha, A system dynamics view of the [35] R.K. Khare, E.S. Powell, G. Reinhardt, M. Lucenti, Adding more beds
acute bed blockage problem in the Irish healthcare system, European J. to the emergency department or reducing admitted patient boarding
Oper. Res. 247 (1) (2015) 276–293. times: Which has a more significant influence on emergency department
congestion? Ann. Emerg. Med. 53 (5) (2009) 575–585.
[12] S. Al-Qahtani, A. Alsultan, S. Haddad, A. Alsaawi, M. Alshehri, S. Alsolamy,
A. Felebaman, H.M. Tamim, N. Aljerian, A. Al-Dawood, Y. Arabi, The [36] W. Abo-Hamad, A. Arisha, Simulation-based framework to improve pa-
association of duration of boarding in the emergency room and the tient experience in an emergency department, European J. Oper. Res. 224
outcome of patients admitted to the intensive care unit, BMC Emerg. (1) (2013) 154–166.
Med. 17 (1) (2017) 34. [37] M. Resta, M. Sonnessa, E. Tànfani, A. Testi, Unsupervised neural networks
for clustering emergent patient flows, Oper. Res. Health Care 18 (2018)
[13] A.M. Nunn, J.S. Hatchimonji, D.N. Holena, M.J. Seamon, B.P. Smith, L.J.
41–51.
Kaplan, N.D. Martin, P.M. Reilly, C.W. Schwab, J.L. Pascual, Boarding ICU
[38] P. Landa, M. Sonnessa, E. Tànfani, A. Testi, Multiobjective bed manage-
patients: Are our rounding practices subpar? Amer. J. Surg. 215 (4) (2018)
ment considering emergency and elective patient flows, Int. Trans. Oper.
4–9.
Res. 25 (1) (2018) 91–110.
[14] INMO, The Irish nurses and midwives organisation, 2018, https://www.
[39] N.C. Proudlove, R. Boaden, Using operational information and information
inmo.ie/.
systems to improve in-patient flow in hospitals, J. Health Organ. Manag.
[15] HSE, National service plan 2017, 2017, https://www.hse.ie/eng/services/
19 (6) (2005) 466–477.
publications/servi.
[40] J.S. Peck, J.C. Benneyan, D.J. Nightingale, S.A. Gaehde, Predicting emer-
[16] E.F. Long, K.S. Mathews, The boarding patient: Effects of ICU and hospital
gency department inpatient admissions to improve same-day patient
occupancy surges on patient flow, Prod. Oper. Manage. 27 (12) (2017)
flow, Acad. Emerg. Med. 19 (9) (2012) 1045–1054.
2122–2143.
[41] M. Afilal, F. Yalaoui, F. Dugardin, L. Amodeo, D. Laplanche, P. Blua,
[17] A.J. Singer, H.C. Thode, P. Viccellio, J.M. Pines, The association between
Emergency department flow: A new practical patients classification and
length of emergency department boarding and mortality, Acad. Emerg.
forecasting daily attendance, IFAC-PapersOnLine 49 (12) (2016) 721–726.
Med. 18 (12) (2011) 1324–1329.
[42] S. Liu, C. Hobgood, J.H. Brice, Impact of critical bed status on emergency
[18] E. Rabin, K. Kocher, M. McClelland, J. Pines, U. Hwang, N. Rathlev, department patient flow and overcrowding, Acad. Emerg. Med. 10 (4)
B. Asplin, N.S. Trueger, E. Weber, Solutions to emergency department (2003) 382–385.
‘boarding’ and crowding are underused and may need to be legislated,
[43] I. Scott, L. Vaughan, D. Bell, Effectiveness of acute medical units in
Health Aff. 31 (8) (2012) 1757–1766.
hospitals: a systematic review, Int. J. Qual. Health Care 21 (6) (2009)
[19] B.A. White, P.D. Biddinger, Y. Chang, B. Grabowski, S. Carignan, D.F.M. 397–407.
Brown, Boarding inpatients in the emergency department increases [44] M.R. Cox, L. Cook, J. Dobson, P. Lambrakis, S. Ganesh, P. Cregan, Acute
discharged patient length of stay, J. Emerg. Med. 44 (1) (2013) 230–235. Surgical Unit: a new model of care, ANZ J. Surg. 80 (6) (2010) 419–424.
[20] S.W. Liu, S.J. Singer, B.C. Sun, C.a Camargo, A conceptual model for assess- [45] J.A.C. Bokhorst, T. van der Vaart, Acute medical unit design – The impact
ing quality of care for patients boarding in the Emergency Department: of rearranged patient flows, Socio-Econ. Plan. Sci. 62 (2018) 75–83.
structure–process–outcome, Acad. Emerg. Med. 18 (4) (2011) 430–435. [46] D. Byrne, B. Silke, Acute medical units: Review of evidence, Eur. J. Intern.
[21] D.W. Johnson, U.H. Schmidt, E.A. Bittner, B. Christensen, R. Levi, R.M. Pino, Med. 22 (4) (2011) 344–347.
Delay of transfer from the intensive care unit: A prospective observational [47] S. Mayor, Acute medical units reduce deaths and stay in hospital, studies
study of incidence, causes, and financial impact, Crit. Care 17 (4) (2013) show, BMJ 337 (2008) 1865.
128. [48] E.D. Moloney, K. Bennett, B. Silke, Effect of an acute medical admission
[22] HSE, Health service executive annual report and financial statements, unit on key quality indicators assessed by funnel plots, Postgrad. Med. J.
2012, http://www.hse.ie/eng/services/Publications/corpor. 83 (984) (2007) 659–663.
[23] HSE, Building a better health service, annual report and finan- [49] V.S. Noble, G. Davies, D. Bell, Improving continuity of care in an acute
cial statements 2016, 2016, https://www.hse.ie/eng/services/publications/ medical unit: initial outcomes, QJM: Int. J. Med. 101 (7) (2008) 529–533.
corpo. [50] G.B.S. McNeill, D.H. Brahmbhatt, A.T. Prevost, N.J.B. Trepte, What is the
[24] S.M. Lo, K.T.Y. Choi, E.M.L. Wong, L.L.Y. Lee, R.S.D. Yeung, J.T.S. Chan, S.Y. effect of a consultant presence in an acute medical unit? Clin. Med. 9 (3)
Chair, Effectiveness of emergency medicine wards in reducing length of (2009) 214–218.
stay and overcrowding in emergency departments, Int. Emerg. Nurs. 22 [51] T. Rooney, E. Moloney, K. Bennett, Impact of an acute medical admission
(2) (2014) 116–120. unit on hospital mortality: a 5-year prospective study, QJM: Int. J. Med.
[25] D. Golmohammadi, Predicting hospital admissions to reduce emergency 101 (6) (2008) 457–465.
department boarding, Int. J. Prod. Econ. 182 (2016) 535–544. [52] E. Relihan, S. Glynn, D. Daly, B. Silke, S. Ryder, Measuring and benchmark-
[26] E.S. Powell, R.K. Khare, A.K. Venkatesh, B.D. Van Roo, J.G. Adams, G. ing safety culture: application of the safety attitudes questionnaire to an
Reinhardt, The relationship between inpatient discharge timing and acute medical admissions unit, Irish J. Med. Sci. 178 (4) (2009) 433–439.
emergency department boarding, J. Emerg. Med. 42 (2) (2012) 186–196. [53] W. Rashwan, H. Heba, A. Arisha, An integrated approach of multi-
[27] A. Arisha, W. Rashwan, Modeling of healthcare systems: Past, current and objective optimization model for evaluating new supporting program in
future trends, in: Proceedings of the 2016 Winter Simulation Conference, Irish hospitals, in: Proceedings of the 2016 Winter Simulation Conference,
2016, pp. 1523–1534. 2016, pp. 1904–1915.
[28] K. Chahal, T. Eldabi, T. Young, A conceptual framework for hybrid system [54] S. Khanna, J. Boyle, N. Good, J. Lind, Unravelling relationships: Hospital
dynamics and discrete event simulation for healthcare, J. Enterprise Inf. occupancy levels, discharge timing and emergency department access
Manag. 26 (1) (2013) 50–74. block, Emerg. Med. Australas. 24 (5) (2012) 510–517.
 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266 21

[55] S. Khanna, D. Sier, J. Boyle, K. Zeitz, Discharge timeliness and its impact on [80] B.I. Mielczarek, J. Zabawa, Modelling population growth, shrinkage and
hospital crowding and emergency department flow performance, Emerg. aging using a Hybrid Simulation approach: Application to healthcare,
Med. Australas. 28 (2) (2016) 164–170. in: 2016 6th International Conference on Simulation and Modeling
[56] J.M.H. Vissers, I.J. Adan, N.P. Dellaert, Developing a platform for compari- Methodologies, Technologies and Applications, 2016, pp. 1–9.
son of hospital admission systems: An illustration, European J. Oper. Res. [81] N. Ahmad, N.A. Ghani, A.A. Kamil, R.M. Tahar, Modelling the complexity of
180 (3) (2007) 1290–1301. emergency department operations using hybrid simulation, Int. J. Simul.
[57] R. Stephens, M. Cudnik, E. Patterson, Barriers and facilitators to timely Process Model. 10 (4) (2015) 360–371.
admission and transfer of patients from an emergency department to an [82] D. Bell, C. Cordeaux, T. Stephenson, H. Dawe, P. Lacey, L. O’Leary, Design-
intensive care unit, in: Proceedings of the Human Factors and Ergonomics ing effective hybridization for whole system modeling and simulation in
Society Annual Meeting, Vol. 55, No. 1, 2011, pp. 763–767. healthcare, in: 2016 Winter Simulation Conference, 2016, pp. 1511–1522.
[58] K. Katsaliaki, N. Mustafee, Improving decision making in healthcare [83] K. Chahal, T. Eldabi, A. Mandal, Understanding the impact of whiteboard
services through the use of existing simulation modelling tools and new on A & E department operations using hybrid simulation, in: Proceedings
technologies, Transform. Gov. People Process Policy 4 (2) (2010) 158–171. of the 27th International Conference of the System Dynamics Society,
[59] J.S. Morgan, S. Howick, V. Belton, A toolkit of designs for mixing Discrete 2009, pp. 1–19.
Event Simulation and System Dynamics, European J. Oper. Res. 257 (3) [84] A. Djanatliev, F. Meier, Hospital processes within an integrated system
(2017) 907–918. view: A hybrid simulation approach, in: Proceedings of the 2016 Winter
[60] S.H. Jacobson, S.N. Hall, J.R. Swisher, Discrete-event simulation of health Simulation Conference, 2016, pp. 1364–1375.
care systems, in: Patient Flow: Reducing Delay in Healthcare Delivery, [85] A. Anagnostou, A. Nouman, S.J.E. Taylor, Distributed hybrid agent-based
2006, pp. 211–252. discrete event emergency medical services simulation, in: Proceedings of
[61] S.C. Brailsford, P.R. Harper, B. Patel, M. Pitt, An analysis of the academic the 2013 Winter Simulation Conference: Simulation: Making Decisions in
literature on simulation and modelling in health care, J. Simul. 3 (3) a Complex World, 2013, pp. 1625–1636.
(2009) 130–140. [86] M. Abdelghany, A.B. Eltawil, S. Abdou, A discrete-Event and agent-Based
[62] A.P. Wierzbicki, Modelling as a way of organising knowledge, European hybrid simulation approach for healthcare systems modeling and anal-
J. Oper. Res. 176 (1) (2007) 610–635. ysis, in: Proceedings of the 2016 International Conference on Industrial
[63] A. Salmon, S. Rachuba, S. Briscoe, M. Pitt, A structured literature review Engineering and Operations Management, 2016, pp. 1921–1928.
of simulation modelling applied to Emergency Departments: Current [87] M. Fakimi, A. Anagnostou, L. Stergioulas, S.J.E. Taylor, A hybrid agent-
patterns and emerging trends, Oper. Res. Health Care 19 (2018) 1–13. based and discrete event simulation approach for sustainable strategic
[64] M.M. Günal, M. Pidd, Discrete event simulation for performance mod- plannig and simulation analytics, in: Proceedings of the 2014 Winter
elling in health care: A review of the literature, J. Simul. 4 (1) (2010) Simulation Conference, 2014, pp. 1573–1584.
42–51. [88] V.A. Knight, J.E. Williams, I. Reynolds, Modelling patient choice in health-
[65] N. Mustafee, K. Katsaliaki, S.J.E. Taylor, Profiling literature in healthcare care systems: development and application of a discrete event simulation
simulation, Simulation 86 (8–9) (2010) 543–558. with agent-based decision making, J. Simul. 6 (2) (2011) 92–102.
[66] S.A. Paul, M.C. Reddy, C.J. Deflitch, A systematic review of simulation [89] S.J. Weng, B.S. Tsai, L.M. Wang, C.Y. Chang, D. Gotcher, Using simulation
studies investigating emergency department overcrowding, Simulation 86 and data envelopment analysis in optimal healthcare efficiency alloca-
(8–9) (2010) 559–571. tions, in: Proceedings of the 2011 Winter Simulation Conference, 2011,
[67] K. De Boeck, R. Carmen, N. Vandaele, Needy boarding patients in pp. 1295–1305.
emergency departments: An exploratory case study using discrete-event [90] Y.A. Ozcan, Health care benchmarking and performance evaluation: An
simulation, Oper. Res. Health Care 21 (2019) 19–31. assessment using data envelopment analysis (DEA), 2008.
[68] A.E. Bair, W.T. Song, Y.C. Chen, B.A. Morris, The impact of inpatient [91] W.W. Cooper, L.M. Sciford, K. Tone, Data Envelopment Analysis: A Com-
boarding on ED efficiency: A discrete-event simulation study, J. Med. Syst. prehensive Text with Models, Applications, References and DEA-Solver
34 (5) (2010) 919–929. Software, Springer, 2001.
[69] T. Roh, V. Quinones-Avila, R.L. Campbell, G. Melin, K.S. Pasupathy, Evalua- [92] J.H. Choi, S.M. Fortsch, I. Park, I. Jung, Efficiency of U.S. hospitals between
tion of interventions for psychiatric care: a simulation study of the effect 2001 and 2011, Manage. Decis. Econ. 38 (8) (2017) 1071–1081.
on emergency departments, in: 2018 Winter Simulation Conference, [93] D. Wang, Q. Gao, Efficiency assessment and resource allocation for hospi-
2018, pp. 2507–2517. tals by data envelopment analysis, in: 2017 3rd International Conference
[70] P. Shi, M.C. Chou, J.G. Dai, D. Ding, J. Sim, Models and insights for hospital on Information Management, ICIM, 2017, pp. 289–293.
inpatient operations: Time-dependent ED boarding time, Manage. Sci. 62 [94] R.C. Lin, M.Y. Sir, K.S. Pasupathy, Multi-objective simulation optimization
(1) (2015) 1–28. using data envelopment analysis and genetic algorithm: Specific applica-
[71] N. Mustafee, T. Lyons, P. Rees, L. Davies, M. Ramsey, M.D. Williams, tion to determining optimal resource levels in surgical services, Omega
Planning of bed capacities in specialized and integrated care units: 41 (5) (2013) 881–892.
Incorporating bed blockers in a simulation of surgical throughput, [95] A. Azadeh, M. Pourebrahim Ahvazi, S. Motevali Haghighii, A. Kera-
in: Proceedings of the 2012 Winter Simulation Conference, 2012, pp. mati, Simulation optimization of an emergency department by modeling
835–846. human errors, Simul. Model. Pract. Theory 67 (2016) 117–136.
[72] K. Crawford, J. Morphet, T. Jones, K. Innes, D. Griffiths, A. Williams, Initia- [96] Y.N. Marmor, B. Golany, S. Israelit, A. Mandelbaum, Designing patient
tives to reduce overcrowding and access block in Australian emergency flow in emergency departments, Trans. Healthc. Syst. Eng. 2 (4) (2012)
departments: A literature review, Collegian 21 (4) (2014) 359–366. 233–247.
[73] S. Brailsford, T. Eldabi, M. Kunc, N. Mustafee, A.F. Osorio, Hybrid sim- [97] W.M.W.M. Aminuddin, W.R. Ismail, Integrated simulation and data envel-
ulation modelling in operational research: A state-of-the-art review, opment analysis models in emergency department, AIP Conf. Proc. 1784
European J. Oper. Res. (2018). (1) (2016) 050003, http://dx.doi.org/10.1063/1.4966822.
[74] K. Chahal, T. Eldabi, Applicability of hybrid simulation to different modes [98] R. Carmen, M. Defraeye, I. Van Nieuwenhuyse, A decision support system
of governance in uk healthcare, in: Proceedings of the 2008 Winter for capacity planning in emergency departments, Int. J. Simul. Model. 14
Simulation Conference, 2008, pp. 1469–1477. (2) (2015) 299–312.
[75] N. Mustafee, S. Brailsford, A. Djanatliev, T. Eldabi, M. Kunc, A. Tolk, [99] K.K. Prasad, S.V. Mohan, R.S. Rao, B.R. Pati, P.N. Sarma, Laccase produc-
Purpose and benefits of hybrid simulation: Contributing to the conver- tion by Pleurotus ostreatus 1804: Optimization of submerged culture
gence of its definition, in: Proceedings of the 2017 Winter Simulation conditions by Taguchi DOE methodology, Biochem. Eng. J. 24 (1) (2005)
Conference, 2017, pp. 1631–1645. 17–26.
[76] J. Zulkepli, T. Eldabi, Towards a framework for conceptual model hy- [100] HSE, Report of the National Acute Medicine Programme, Royal
bridization in healthcare, in: Proceedings of the 2015 Winter Simulation College of Physicians of Ireland, 2010, https://www.hse.ie/eng/services/
Conference, 2015, pp. 1597–1608. publications/clinical-strategy-and-programmes/report-of-the-national-
[77] R.E. Giachetti, E.A. Centeno, M.A. Centeno, R. Sundaram, Assessing the acute-medicine-programme.pdf.
viability of an open access policy in an outpatient clinic: a discrete-event [101] J.G. Cronin, The introduction of the Manchester triage scale to an
and continuous simulation modeling approach, in: Proceedings of the emergency department in the Republic of Ireland, Accid. Emerg. Nurs.
2005 Winter Simulation Conference, 2005, pp. 2246–2255. 11 (2) (2003) 121–125.
[78] T.R. Rohleder, D.P. Bischak, L.B. Baskin, Modeling patient service centers [102] W. Rashwan, An Integrated Framework for Staffing and Shift Scheduling
with simulation and system dynamics, Health Care Manag. Sci. 10 (1) in Hospitals, Dublin Institute of Technology (DIT), 2017.
(2007) 1–12. [103] S. Robinson, Simulation: The practice of model development and use,
[79] J. Zulkepli, T. Eldabi, Developing integrated patient pathways using hybrid 2004.
simulation, AIP Conf. Proc. 1782 (1) (2016) 040022, http://dx.doi.org/10. [104] J. Banks, B.L. Nelson, J.S. Carson, D.M. Nicol, Discrete-Event System
1063/1.4966089. Simulation, Pearson, 2005.
22 L. Keshtkar, W. Rashwan, W. Abo-Hamad et al. / Operations Research for Health Care 26 (2020) 100266

[105] S.M. Sanchez, H. Wan, Work smarter, not harder: A tutorial on de- [107] M.M. Nasiri, R. Yazdanparast, F. Jolai, A simulation optimisation approach
signing and conducting simulation experiments, in: Winter Simulation for real-time scheduling in an open shop environment using a composite
Conference, 2012, pp. 1795–1809. dispatching rule, Int. J. Comput. Integr. Manuf. 30 (12) (2017) 1239–1252.
[106] R.C. Miranda, J.A.B. Montevechi, A.F. da Silva, F.A.S. Marins, Increasing the [108] R.D. Banker, A. Charnes, W.W. Cooper, J. Swarts, D. Thomas, An intro-
efficiency in integer simulation optimization: Reducing the search space duction to Data Envelopment Analysis with some of its models and their
through data envelopment analysis and orthogonal arrays, European J. uses, Res. Governmental Nonprofit Account. 5 (1989) 125–163.
Oper. Res. 262 (2) (2017) 673–681.