Perspectives

Ten key considerations for the successful

implementation and adoption of large-scale
health information technology
Kathrin M Cresswell,1 David W Bates,2,3 Aziz Sheikh4
1
The School of Health in Social ABSTRACT change, selecting a system, implementation plan-
Science, The University of The implementation of health information technology ning, and maintenance and evaluation (figure 1).
Edinburgh, Edinburgh, UK
2
Department of Medicine,
interventions is at the forefront of most policy agendas
Brigham and Women’s internationally. However, such undertakings are often far TEN KEY CONSIDERATIONS FOR THE
Hospital, Harvard Medical from straightforward as they require complex strategic SUCCESSFUL IMPLEMENTATION OF HEALTH
School, Boston, Massachusetts,

Downloaded from [Link] by guest on 30 April 2021

planning accompanying the systemic organizational INFORMATION TECHNOLOGY
USA changes associated with such programs. Building on our
3
The Department of Health 1. Clarify what problem(s) the technology is
Policy and Management, experiences of designing and evaluating the designed to help tackle
Harvard School of Public implementation of large-scale health information Many health information technology procurements
Health, Boston, Massachusetts, technology interventions in the USA and the UK, we are based on assumed benefits, which are often
USA highlight key lessons learned in the hope of informing
4
eHealth Research Group, poorly specified. This can result in difficulties agree-
Centre for Population Health the on-going international efforts of policymakers, health ing on a shared vision across the healthcare organ-
Sciences, The University of directorates, healthcare management, and senior ization. While terms like ‘improved quality of care’
Edinburgh, Edinburgh, UK clinicians. and ‘improved efficiency’ are often used, detailed
outcomes resulting from specific functionality are
Correspondence to
K Cresswell, The School of hard to measure and to anticipate as most imple-
Health in Social Science, INTRODUCTION mentations require fundamental changes to oper-
The University of Edinburgh, Large-scale, potentially transformative, implementa- ational processes and many organizations do not
Edinburgh EH8 9DX, UK; even attempt this.3 4 52–54 Thus, organizations often
[Link]@[Link] tions of health information technology are now
being planned and undertaken in multiple coun- encounter difficulties conceptualizing the required
Received 30 January 2013 tries.1 2 The hope is that the very substantial finan- short-, medium-, and long-term transformations.
Revised 4 March 2013 cial, human, and organizational investments being A thorough mapping of existing local processes
Accepted 1 April 2013 before implementation can mitigate this risk and
made in electronic health records, electronic pre-
Published Online First
scribing, whole-system telehealthcare, and related help to identify existing problems as well as areas
18 April 2013
technologies will streamline individual and organ- for improvement. In an ideal scenario, this ground-
izational work processes and thereby improve the work would result in agreement on the problem(s)
quality, safety, and efficiency of care. The reality is, to be addressed by a specific functionality (eg,
however, that these technologies may prove frustrat- duplication of information) and, based on this, the
ing for frontline clinicians and organizations as the development of a long-term strategic vision (eg, a
systems may not fit their usual workflows, and the common patient record that is populated by all
anticipated individual and organizational benefits health professionals). However, new technology
take time to materialize.3 4 In this article, we reflect may not always be the answer. It is therefore also
on our mapping of the literature (see box 1) and important to assess if, and to what extent, existing/
complement this with our experiences of studying a new health information technology can support
range of national evaluations of various large-scale these strategic goals and whether other approaches
health information technology systems in the UK may also need to be considered.
and USA to provide key pointers that can help
streamline implementation efforts.4 52–54 In so 2. Build consensus
doing, we hope to inform policy and practice devel- Professional, managerial, and administrative con-
opment to support the more successful integration sensus needs to be built around the strategic vision,
of technology into complex healthcare environ- in addition to creating the means to support the
ments. This is particularly timely given the US realization of this vision.56 This may involve con-
Health Information Technology for Economic and sidering whether to aim for radical changes across
Clinical Health (HITECH) Act, which includes a the organization (eg, through implementing elec-
$19 billion stimulus package to promote the adop- tronic health record functionality), or whether to
tion of electronic health records and associated focus on streamlining specific processes (eg, elec-
functionality.55 tronic prescribing) initially and then expanding
Editor’s choice
Scan to access more This paper complements a previous publication functionality over time. Many authors in the field
free content by Bates and colleagues on ‘Ten commandments of organizational change have highlighted that
for effective clinical decision support’,11 which high-level strategic leadership of senior manage-
To cite: Cresswell KM, focused on lessons learned in relation to clinical ment including both administrative and clinical
Bates DW, Sheikh A. J Am decision support systems. We have developed a leaders is vital, and this is accurate, but it is also
Med Inform Assoc 2013;20: technology lifecycle approach to highlight key con- essential to involve and get the buy-in of different
e9–e13. siderations at four stages: establishing the need for professional stakeholder groups (eg, doctors,

Cresswell KM, et al. J Am Med Inform Assoc 2013;20:e9–e13. doi:10.1136/amiajnl-2013-001684 e9

 Perspectives

which need specific attention from different professional stake-

Box 1 Factors associated with effective implementation holders. For the latter, efforts promoting the participation and
identified in the literature5–51 empowerment of different groups by actively searching for
inclusive solutions, have the highest potential to achieve coordi-
nated implementation efforts.59 Nurses, for instance, will have
Technical: usability, system performance, integration and
different needs to doctors, but all groups tend to agree that the
interoperability, stability and reliability, adaptability and
provision of high standards of care should be the focus of activ-
flexibility, cost, accessibility and adaptability of hardware
ities. Patient-centered discussions could therefore be a point of
Social: attitudes and concerns, resistance and workarounds,
convergence between different professional viewpoints.
expectations, benefits/values and motivations, engagement and
user input in design, training and support, champions,
integration with existing work practices 3. Consider your options
Organizational: getting the organization ready for change, Once the need for a technological system has been established,
planning, leadership and management, realistic expectations, it is important to commit adequate time and resources to thor-
user ownership, teamwork and communication, learning and oughly consider different options in terms of which system(s) to
evaluation

Downloaded from [Link] by guest on 30 April 2021

choose. We have found that this aspect of planning and the asso-
Wider socio-political: other healthcare organizations, industry, ciated writing of business cases and procurement considerations
policy, professional groups, independent bodies, the wider are sometimes under-estimated and often rushed.11 52 It is, for
economic environment, international developments example, important to be aware of the full range of system pro-
viders, and network with potential suppliers in order to under-
stand the ethos and values of the companies with which the
nurses, administrative staff, managers) in order to facilitate organization is considering embarking on a long-term relation-
co-ownership and ensure commitment.57 58 From our experi- ship. Visiting other healthcare settings that have implemented
ence, this balance is best achieved through the creation of a similar technology can prove very helpful.52
high-level strategic group that not only includes senior man- Once available commercial systems have been appraised, it is
agers, but also clinical and administrative leads who represent appropriate to reflect on whether to build a customized system
different end-user groups. tailored to local needs, whether to customize an existing system,
An important factor to keep in mind is that attempts to align or whether to use an ‘off-the-shelf ’ standardized solution.58
perspectives through, for example, consensus building activities, The literature and our experiences indicate that there are inher-
need to be skillfully handled with cognizance of the means to ent risks and benefits to each of these approaches. For example,
overcome rather than perpetuate existing professional hierarch- although ‘home-grown’ customized systems tend to be better
ies. One approach that we have successfully used is to identify accepted by local users than standardized solutions, they are
domains in which there is already broad agreement versus those also not a cheap option and often do not easily integrate with

Figure 1 Summary of the lifecycle

stages of health information
technology and the ten key
considerations.

e10 Cresswell KM, et al. J Am Med Inform Assoc 2013;20:e9–e13. doi:10.1136/amiajnl-2013-001684

 Perspectives

other technological systems in the organization.4 52–54 In add-

ition, considerable time is needed to customize systems, and Box 2 Examples of ‘failures’ in implementation
such efforts are often led by individuals or small groups of
enthusiasts and so may not have longevity. In the USA, for
Example 1: Rejection by users61
example, commercial systems have markedly improved in recent
May and colleagues evaluated the implementation of a
years and now dominate the marketplace. Internationally, most
videophone, which was intended to be used by primary care
organizations will, we anticipate, also choose commercial
physicians to refer patients to a community mental health
systems in the future due to cost and interoperability considera-
facility. The team conducted qualitative interviews and
tions. Commercial systems are cheaper to purchase (as they are
observations with clinicians, managerial staff, and patients in
not customized to individual organizations and can therefore be
order to explore the acceptability of the technology. They found
produced in bulk) and they are also likely to be interoperable
that some professional groups, including community psychiatric
due to common data standards and architectures (such as, for
nurses and occupational therapists, resisted using the system as
example, the Health Level Seven International interoperability
they felt that it impacted adversely on the therapeutic
standards).
relationship with their patients.
Example 2: Bandwidth undermining system performance

Downloaded from [Link] by guest on 30 April 2021

4. Choose systems that meet clinical needs and are affordable
(authors’ own experience)
Once a decision on the basic type of system has been made, it is
At one large hospital, operational management was told by
important to base the final choice not only on organizational,
the information technology leadership that the hospital’s
but also on clinical needs.60 A system should be both fit for
network was at maximal bandwidth at budget time. The
organizational purpose and fit for clinical practice. There are
management decided that the hospital would wait a year to
countless examples of systems that have been procured but
upgrade the network, and instead purchased an expensive new
never used (eg, if they are perceived to undermine professional
imaging technology. However, several months later the
values) or are deployed in unintended ways, which will then
institution’s systems began to ‘brown out,’ and it was taking up
typically result in a failure to realize the hoped for improve-
to 30 minutes for a single screen change. Although the
ments (see box 2, example 1).4 61
leadership immediately reversed field and authorized a network
A system therefore needs to fulfill a range of requirements on
upgrade, this took several months to implement and care
a variety of levels. It needs to be usable for end-users (not cum-
delivery was substantially impaired in the interim.
bersome for clinicians and beneficial for patients), cost-effective
Example 3: The importance of user feedback (authors’ own
for organizations, and interoperable to allow secondary uses of
experience)
data. These purposes are often difficult to align as requirements
Getting an ‘early fix’ on how long a new system is taking to
of different domains may result in trade-offs for others.60 For
use is especially critical. At one hospital which was a pioneer in
example, it has repeatedly been found that many health infor-
order entry, a system was implemented and users were told
mation technologies slow down the work practices of users,
they had to use it. However, the leadership had no clear idea
despite improving overall organizational efficiency.62 Speed is of
how long it was actually taking front-line clinical users to do
the essence and any initiative that slows down key clinical tasks
their work—something that took an hour before
is likely to be strongly resisted by frontline staff. This issue can
implementation was now taking several hours, which resulted in
to some extent be addressed by purchasing systems that allow a
an unworkable situation for front-line users. This eventually
large degree of customization, but these are often expensive to
resulted in a computer monitor being thrown through a hospital
acquire and run, necessitating a careful balancing act between
window, and a work action by the clinical users. That got the
affordability and desired functionality. Our experiences suggest
leadership’s attention and major changes were made.
that the associated system costs are often under-estimated, par-
Example 4: Tracking system performance (authors’ own
ticularly those relating to infrastructure, support, and
experience)
maintenance.
Maintenance includes tracking how the system is performing,
and how the decision support within it is performing. Such
5. Plan appropriately
tracking is much easier if it is built in from the beginning. At
It takes both targeted and reflective efforts to plan for trans-
one large hospital, the allergy over-ride rates were initially very
formative organizational ventures of any kind. Although flexibil-
low. However, a series of apparently innocuous changes in the
ity in strategy is required, there are some general pointers that
decision support system were made by the responsible
tend to characterize effective preparation across organizations
committee with the result that several years later, large
and technologies. These include the aforementioned necessity to
numbers of alerts were being delivered, with nearly all being
engage extensively with potential suppliers and other organiza-
over-ridden. After these data were reviewed, the system could
tions who have already implemented, but also the decision to
be tuned, and the unimportant alerts were turned off.
prioritize the implementation of functionality that can bring
benefits to the greatest number of end-users as early as pos-
sible.4 52–54 Other factors relate to the avoidance of ‘scope- paper and electronic) wherever possible, as this tends to increase
creep’ (ie, the tendency to increase the scope of a project when workloads for end-users and may inadvertently introduce new
it is already underway) and maintaining open channels of com- threats to patient safety.4 52–54
munication between management and users.
Implementation strategies need to be tailored to organiza-
tional circumstances and systems, whether they involve ‘phased’ 6. Don’t forget the infrastructure
or ‘big-bang’ implementation approaches. The former relates to Developing the right infrastructure is an essential part of plan-
introducing incremental functionality slowly, while the latter ning activity. If this is not afforded sufficient attention, then
relates to introducing functionality across the organization all at software systems may perform sub-optimally (eg, if wireless net-
once. We suggest avoiding the running of parallel systems (both works are unavailable or bandwidth is too narrow), or may be

Cresswell KM, et al. J Am Med Inform Assoc 2013;20:e9–e13. doi:10.1136/amiajnl-2013-001684 e11

 Perspectives

inaccessible to users altogether (eg, if there is a lack of available infrastructure, and system upgrades), but also costs relating to
hardware). Again, this increases the possibility that systems are potential system changes as the strategic aims of organizations
not used at all or used in ways other than intended, potentially and therefore the capabilities of existing technological systems
compromising benefits and increasing risks associated with are likely to change over time (box 2, example 4).
technological systems.4 We have repeatedly found that inappro-
priate infrastructure can negatively shape user attitudes towards 10. Stay the course
software systems themselves, as it can impact on usability and The benefits of major transformative ventures are notoriously dif-
performance. Inappropriate infrastructure, such as a slow wire- ficult to measure and may take a long time to materialize.3 4 52–54
less connection, may for example, reduce the speed of a system, However, this is not to say that they are non-existent, rather they
which is an important (if not the most important) factor in need to be tracked by appropriate evaluation work assessing how
determining adoption (box 2, example 2).11 the new system is used and re-invented locally. This also requires
an appreciation of the timelines surrounding the realization of
7. Train staff expected benefits, allowing enough time for technologies to
Trained users tend to be more satisfied with new technologies embed and data to be exploited for secondary uses.60 Our work
than those who have not been adequately trained.63 This may has shown that in many cases the expectations of organizations

Downloaded from [Link] by guest on 30 April 2021

be due to a lack of understanding of system capabilities, which and individual users far exceed what is achievable in the short
can in turn lead to workarounds whereby the new systems are term. The managing of expectations is, therefore, important as
used in unintended ways—or worse still—avoided completely. otherwise there is a danger that stakeholders disengage with the
The most effective training is that which is tailored to the initiative and negative attitudes may emerge.4 52–54
individual roles of users, without being too restrictive as this can
undermine understanding of how the whole system functions. CONCLUSIONS
Training needs to allow users to practice ‘hands-on’ and as Careful planning and on-going, critical evaluation of progress
closely simulate the actual working environment as possible.58 64 are central to the successful implementation of major health
It is also ideally conducted shortly before the implementation as information technology. Taking a lifecycle perspective on the
otherwise staff may forget important functions. There may be a implementation of technological systems will, we hope, help
need for compulsory (eg, in relation to approaches to maintain- organizations to avoid some of the all too commonly encoun-
ing patient confidentiality) as well as voluntary components, tered pitfalls and improve the likelihood of successful imple-
and some individuals may need more training than others. For mentation and adoption (see figure 1). It is, however, important
instance, older users may never have used a computer and may to keep in mind that, although the stages and considerations dis-
therefore require more basic training than younger individuals, cussed here were depicted in a linear manner, they may to some
who tend to be more accustomed to computers. For infrequent extent overlap. This is consistent with the complex nature of
users and in relation to systems that are subject to regular large-scale health information technology implementations,
upgrades, continuous training may be necessary. From our where a range of different inter-related factors are at play.
experience, training should typically total about 40% of an
implementation budget, but is the area most often left short. Acknowledgements We are very grateful to all participants who kindly gave their
time and to the extended project and program teams of work we have drawn upon.
We are also grateful to two anonymous expert peer reviewers who commented on a
8. Continuously evaluate progress
previous version of this manuscript.
Although it is now widely recognized that evaluation is import-
Contributors AS conceived this work. AS is currently leading a National Institute
ant when considering new technologies, the reality is that it is
for Health Research-funded national evaluation of electronic prescribing and
still, more often than not, an afterthought as immediate imple- medicines administration systems. KMC is employed as a researcher on this grant
mentation activities take priority.65 Real-time, longitudinal data and led on the write-up and drafting of the initial version of the paper, with DWB
collection strategies providing formative feedback are desirable and AS commenting on various drafts.
as emerging results can be incorporated in on-going implemen- Funding This work has drawn on data funded by the NHS Connecting for Health
tation activity, but this is costly and time-consuming. However, Evaluation Programme (NHS CFHEP 001, NHS CFHEP 005, NHS CFHEP 009, NHS
it is essential to capture user feedback about problems that are CFHEP 010) and the National Institute for Health Research (NIHR)-funded
Programme Grants for Applied Research scheme (RP-PG-1209-10099). The views
identified and respond to it in a timely manner (box 2, example expressed are those of the author(s) and not necessarily those of the NHS, the
3). In our experience, investments in evaluation activities are NIHR, or the Department of Health.
always worth it. These should begin with assessing existing and Competing interests None.
anticipated organizational and individual workflows, monitoring
Provenance and peer review Not commissioned; externally peer reviewed.
desired and undesired consequences, and tracking new innova-
tive ways of working.4 52–54 It is also crucially important that
this work is carried out over an appropriate length of time, as it REFERENCES
1 Morrison Z, Robertson A, Cresswell K, et al. Understanding and contrasting
may well take years for benefits and consequences to emerge.60 approaches to nationwide implementations of electronic health record systems:
Following developments over the long term can further help England, the USA and Australia. J Healthc Eng 2011;2:25–42.
identify when systems have become obsolete and when there is 2 Bates D. Using information technology to reduce rates of medication errors in
a need for new solutions. hospitals. BMJ 2000;320:788.
3 European Commission. Interoperable eHealth is Worth it. Securing benefits from
Electronic Health Records and ePrescribing. Brussels: European Commission, 2010.
9. Maintain the system 4 Sheikh A, Cornford T, Barber N, et al. Implementation and adoption of nationwide
Maintenance is in many ways related to all of the above points electronic health records in secondary care in England: final qualitative results from
as these issues need to be re-visited periodically throughout the prospective national evaluation in “early adopter” hospitals. BMJ 2011;343:d6054.
technology lifecycle (see figure 1). Nevertheless, maintenance 5 Adler KG. How to successfully navigate your EHR implementation. Fam Pract Manag
2007;14:33–9.
deserves particular attention as it is often under-estimated in 6 Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task,
relation to associated activities and cost.66 This is not only the technology and individuals: a fit framework and a case study. BMC Med Inform
case in relation to on-going costs (eg, pertaining to support, Decis Mak 2006;6:13.

e12 Cresswell KM, et al. J Am Med Inform Assoc 2013;20:e9–e13. doi:10.1136/amiajnl-2013-001684

 Perspectives

7 Ash J, Berg M. Report of conference Track 4: socio-technical issues of HIS. Int J 36 Lorenzi NM, Smith JB, Conner SR, et al. The success factor profile for clinical
Med Inform 2003;69:305–6. computer innovation. Stud Health Technol Inform 2004;107:1077–80.
8 Austin CJ, Hornberger KD, Shmerling JE. Managing information resources: a study 37 Lu Y-C, Xiao Y, Sears A, et al. A review and a framework of handheld computer
of ten healthcare organizations. J Healthc Manag 2000;45:229–38. adoption in healthcare. Int J Med Inform 2005;74:409–22.
9 Bali RK, Wickramasinghe N. Achieving successful EPR implementation with the 38 Ludwick DA, Doucette J. Adopting electronic medical records in primary care:
penta-stage model. Int J Healthc Technol Manag 2008;9:97–105. lessons learned from health information systems implementation experience in seven
10 Bates DW, Ebell M, Gotlieb E, et al. A proposal for electronic medical records in U. countries. Int J Med Inform 2009;78:22–31.
S. primary care. J Am Med Inform Assoc 2003;10:1–10. 39 Mehta NB, Partin MH. Electronic health records: a primer for practicing physicians.
11 Bates DW, Kuperman GJ, Wang S, et al. Ten commandments for effective clinical Cleve Clin J Med 2007;74:826–30.
decision support: making the practice of evidence-based medicine a reality. J Am 40 Miranda D, Fields W, Lund K. Lessons learned during 15 years of clinical
Med Inform Assoc 2011;10:523–30. information system experience. Comput Nurs 2001;4:147–51.
12 Beuscart-Zephir MC, Anceaux F, Crinquette V, et al. Integrating users’ activity 41 Moen A. A nursing perspective to design and implementation of electronic patient
modeling in the design and assessment of hospital electronic patient records: the record systems. J Biomed Inform 2003;36:375–8.
example of anesthesia. Int J Med Inform 2001;64:157–71. 42 Nikula RE. Why implementing EPR’s does not bring about organizational changes–a
13 Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records qualitative approach. Stud Health Technol Inform 2001;84:666–9.
by physicians from systematic review to taxonomy and interventions. BMC Health 43 Ovretveit J, Scott T, Rundall TG, et al. Improving quality through effective
Serv Res 2010;10:231. implementation of information technology in healthcare. Int J Qual Health Care
14 Bossen C. Test the artefact–develop the organization. The implementation of an 2007;5:259–66.

Downloaded from [Link] by guest on 30 April 2021

electronic medication plan. Int J Med Inform 2007;76:13–21. 44 Pagliari C. Implementing the national programme for IT: what can we learn from
15 Callen JL, Braithwaite J, Westbrook JI. Contextual implementation model: a the Scottish experience? Inform Prim Care 2005;13:105–11.
framework for assisting clinical information system implementations. J Am Med 45 Pare G. Implementing clinical information systems: a multiple-case study within a
Inform Assoc 2008;15:255–62. US hospital. Health Serv Manage Res 2002;15:71–92.
16 Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information 46 Pare G, Sicotte C, Jaana M, et al. Prioritizing the risk factors influencing the success
technology on quality, efficiency, and costs of medical care. Ann Intern Med of clinical information system projects. A Delphi study in Canada. Methods Inf Med
2006;144:742–52. 2008;47:251–9.
17 Clemmer TP. Computers in the ICU: where we started and where we are now. J Crit 47 Pendergast DK, Buchda VL. Charting the course. A quality journey. Nurs Adm Q
Care 2004;4:201–7. 2003;27:330–5.
18 Crosson JC, Stroebel C, Scott JG, et al. Implementing an electronic medical record 48 Puffer MJ, Ferguson JA, Wright BC, et al. Partnering with clinical providers to
in a family medicine practice: communication, decision making, and conflict. Ann enhance the efficiency of an EMR. J Healthc Inf Manag 2007;21:24–32.
Fam Med 2005;3:307–11. 49 Quinzio L, Junger A, Gottwald B, et al. User acceptance of an anaesthesia
19 Dagroso D, Williams PD, Chesney JD, et al. Implementation of an obstetrics EMR information management system. Eur J Anaesthesiol 2003;20:967–72.
module: overcoming user dissatisfaction. J Healthc Inf Manag 2007;21:87–94. 50 Räisänen C, Linde A. Technologizing discourse to standardize projects in
20 Davidson E, Chiasson M. Contextual influences on technology use mediation: a multi-project organizations: hegemony by consensus? Organization
comparative analysis of electronic medical records systems. Eur J Info Syst 2004;11:101–21.
2005;14:6–18. 51 Rose J, Jones M, Truex D. Socio-theoretic accounts of IS: the problem of agency.
21 De Mul M, Berg M, Hazelzet JA. Clinical information systems: careSuite from Picis. J Scand J Info Syst 2005;17:133–52.
Crit Care 2004;19:208–14. 52 Cresswell K, Coleman J, Slee A, et al. Investigating and learning lessons from early
22 Duggan C. Implementation evaluation. HIM professionals share their experiences experiences of implementing ePrescribing systems into NHS hospitals: a
bringing health IT online. J AHIMA 2006;77:52–5. questionnaire study. PLoS One 2013;8:e53369.
23 Fenton SH, Giannangelo K, Stanfill M. Essential people skills for EHR 53 Ash JS, Stavri PZ, Kuperman GJ. A consensus statement on considerations for a
implementation success. J AHIMA 2006;77:60. successful CPOE implementation. J Am Med Inform Assoc 2003;10:229–34.
24 Ferneley E, Sobreperez P. Resist, comply or workaround? An examination of different 54 Health Information Technology Evaluation Toolkit. [Link]
facets of user engagement with information systems. Eur J Inf Syst 2006;15:345–56. unintended-consequences/sites/default/files/pdf/[Link] (accessed 3 Jan 2013).
25 Giuse DA, Kuhn KA. Health information systems challenges: the Heidelberg 55 Blumenthal D. Stimulating the adoption of health information technology. New Engl
conference and the future. Int J Med Inform 2003;69:105–14. J Med 2009;360:1477–9.
26 Goroll AH, Simon SR, Tripathi M, et al. Community-wide implementation of health 56 Markoczy L. Consensus formation during strategic change. Strategic Manage J
Information technology: the Massachusetts eHealth collaborative experience. J Am 2001;22:1013–31.
Med Inform Assoc 2009;16:132–9. 57 Cresswell K, Morrison Z, Crowe S, et al. Anything but...engaged: user involvement
27 Granlien MF, Hertzum M, Gudmundsen J. The gap between actual and mandated in the context of a national electronic health record implementation. Inform Prim
use of an electronic medication record three years after deployment. Stud Health Care 2012;19:191–206.
Technol Inform 2008;136:419–24. 58 Dagroso D, Williams PD, Chesney JD, et al. Implementation of an obstetrics EMR
28 Halamka J, Aranow M, Ascenzo C, et al. E-Prescribing collaboration in module: overcoming user dissatisfaction. J Healthc Inform Manage 2007;21:87–94.
Massachusetts: early experiences from regional prescribing projects. J Am Med 59 Checkland P. Systems thinking, systems practice. Chichester: Wiley, 1981.
Inform Assoc 2006;13:239–44. 60 Cresswell K, Sheikh A. Effective integration of technology into health care needs
29 Hendy J, Reeves BC, Fulop N, et al. Challenges to implementing the national adequate attention to sociotechnical processes, time and a dose of reality. JAMA
programme for information technology (NPfIT): a qualitative study. BMJ 2012;307:2255.
2005;331:331–6. 61 May C, Gask L, Atkinson T, et al. Resisting and promoting new technologies in
30 James D, Hess S, Kretzing JE Jr, et al. Showing “what right looks like”–how to clinical practice: the case of telepsychiatry. Soc Sci Med 2001;52:1889–901.
improve performance through a paradigm shift around implementation thinking. J 62 Aarts J, Doorewaard H, Berg M. Understanding implementation: the case of a
Healthc Inf Manag 2007;21:54–61. computerized physician order entry system in a Large Dutch University Medical
31 Jones M. Learning the lessons of history? Electronic records in the United Kingdom Center. J Am Med Inform Assoc 2004;11:207–16.
acute hospitals, 1988–2002. Health Informatics J 2004;10:253–63. 63 Yusof MM, Kuljis J, Papazafeiropoulou A, et al. An evaluation framework for Health
32 Karsten H, Laine A. User interpretations of future information system use: a Information Systems: human, organization and technology-fit factors (HOT-fit). Int J
snapshot with technological frames. Int J Med Inform 2007;76:S136–40. Med Inform 2008;77:386–98.
33 Keddie Z, Jones R. Information communications technology in general practice: 64 Sicotte C, Pare G, Moreault M-P, et al. A risk assessment of two interorganizational
cross-sectional survey in London. Inform Prim Care 2005;13:113–23. clinical information systems. J Am Med Inform Assoc 2006;13:557–66.
34 Keshavjee K, Bosomworth J, Copen J, et al. Best practices in EMR implementation: 65 Greenhalgh T, Stramer K, Bratan T, et al. Introduction of shared electronic
a systematic review. AMIA Annu Symp Proc 2006;982. records: multi-site case study using diffusion of innovation theory. BMJ
35 Lium JT, Tjora A, Faxvaag A. No paper, but the same routines: a qualitative 2008;337:1786.
exploration of experiences in two Norwegian hospitals deprived of the paper based 66 McGowan JJ, Cusack CM, Poon EG. Formative evaluation: a critical component in
medical record. BMC Med Inform Decis Mak 2008;8. EHR implementation. J Am Med Inform Assoc 2008;15:297–301.

Cresswell KM, et al. J Am Med Inform Assoc 2013;20:e9–e13. doi:10.1136/amiajnl-2013-001684 e13