Implementation Science BioMed Central

Research article Open Access

Patterns of research utilization on patient care units
Carole A Estabrooks*1, Shannon Scott1, Janet E Squires1, Bonnie Stevens2,
Linda O'Brien-Pallas3, Judy Watt-Watson3, Joanne Profetto-McGrath1,
Kathy McGilton4, Karen Golden-Biddle5, Janice Lander1, Gail Donner3,
Geertje Boschma6, Charles K Humphrey7 and Jack Williams8

Address: 1Faculty of Nursing, University of Alberta, Edmonton, Canada, 2Faculty of Nursing, University of Toronto and Hospital for Sick Children,
Toronto, Canada, 3Faculty of Nursing, University of Toronto, Toronto, Canada, 4Toronto Rehabilitation Institute, Toronto, Canada, 5School of
Management, Boston University, Boston, USA, 6Faculty of Nursing, University of British Columbia, Vancouver, Canada, 7Data Library, University
of Alberta, Edmonton, Canada and 8Institute of Clinical Evaluative Sciences & Clinical Epidemiology and Health Services Research Program,
Sunnybrook Health Sciences Centre, Toronto, Canada
Email: Carole A Estabrooks* - [email protected]; Shannon Scott - [email protected];
Janet E Squires - [email protected]; Bonnie Stevens - [email protected]; Linda O'Brien-Pallas - [email protected];
Judy Watt-Watson - [email protected]; Joanne Profetto-McGrath - [email protected];
Kathy McGilton - [email protected]; Karen Golden-Biddle - [email protected]; Janice Lander - [email protected];
Gail Donner - [email protected]; Geertje Boschma - [email protected];
Charles K Humphrey - [email protected]; Jack Williams - [email protected]
* Corresponding author

Published: 2 June 2008 Received: 11 August 2007

Accepted: 2 June 2008
Implementation Science 2008, 3:31 doi:10.1186/1748-5908-3-31
This article is available from: http://www.implementationscience.com/content/3/1/31
© 2008 Estabrooks et al; licensee BioMed Central Ltd.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract
Background: Organizational context plays a central role in shaping the use of research by
healthcare professionals. The largest group of professionals employed in healthcare organizations
is nurses, putting them in a position to influence patient and system outcomes significantly.
However, investigators have often limited their study on the determinants of research use to
individual factors over organizational or contextual factors.
Methods: The purpose of this study was to examine the determinants of research use among
nurses working in acute care hospitals, with an emphasis on identifying contextual determinants of
research use. A comparative ethnographic case study design was used to examine seven patient
care units (two adult and five pediatric units) in four hospitals in two Canadian provinces (Ontario
and Alberta). Data were collected over a six-month period by means of quantitative and qualitative
approaches using an array of instruments and extensive fieldwork. The patient care unit was the
unit of analysis. Drawing on the quantitative data and using correspondence analysis, relationships
between various factors were mapped using the coefficient of variation.
Results: Units with the highest mean research utilization scores clustered together on factors such
as nurse critical thinking dispositions, unit culture (as measured by work creativity, work efficiency,
questioning behavior, co-worker support, and the importance nurses place on access to continuing
education), environmental complexity (as measured by changing patient acuity and re-sequencing
of work), and nurses' attitudes towards research. Units with moderate research utilization
clustered on organizational support, belief suspension, and intent to use research. Higher nursing

Page 1 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

workloads and lack of people support clustered more closely to units with the lowest research
utilization scores.
Conclusion: Modifiable characteristics of organizational context at the patient care unit level
influences research utilization by nurses. These findings have implications for patient care unit
structures and offer beginning direction for the development of interventions to enhance research
use by nurses.

Background innovation adoption include: organizational complexity

Investigators have described the difficulties and complex- [24], centralization [25], size [25,26], presence of a
ities of implementing change in practice [1], and increas- research champion [27,28], traditionalism [29,30],
ingly we see calls for the design of more theory-informed organizational slack [31], access to and amount of
interventions [2-4]. While calls to make nursing practice resources [19,29,32,33], constraints on time [34-36] and
more research-based are common, research utilization staffing [15,36], professional autonomy [35,37,38], geo-
investigators in nursing have argued that the use of graphic location (i.e., urban versus rural) [39], and organ-
research evidence is often not reflected in the delivery of izational support [11,12,35,40,41].
nursing care despite the benefits of adopting research-
based practices, and the increased availability of research Over the past decade, nurse investigators in the United
to health professionals [5-7]. As a result, patients often do Kingdom (UK) have called for more attention to contex-
not receive optimal or effective nursing care. In response tual factors in promoting research use by healthcare pro-
to this, we have seen accelerated efforts to develop inter- viders [42-44]. They define context as 'the environment or
ventions to increase the use of research in practice. How- setting in which the proposed change is to be imple-
ever, relatively few reports exist about intervention studies mented' and understand it to be comprised of three core
in the area of research utilization for nurses, and those dimensions: culture, leadership, and evaluation [42].
available have often not yielded positive results [8,9].
(One reason for this, we argue, is a failure to systemati- McCormack et al., in a concept analysis of context in rela-
cally account for the factors that influence nurses' use of tion to research implementation, define culture as the
research, or stated another way, to systematically account defining prevailing beliefs and values, consistency in val-
for the determinants of research utilization behaviour ues, and receptivity to change, among members of an
within the work context (i.e., organizational setting) of organization or group [45]. Organizational culture, at
nurses. least theoretically, affects clinician behaviors such as the
adoption of research findings in practice. While positive
Various individual, organizational, and most recently, effects of culture on research utilization have been sug-
contextual, factors have been argued as influencing the gested by several scholars in the field [42,46-49], to date,
use of research by healthcare providers. Traditionally, the we have relatively little empirical evidence to support
factors studied in nursing have tended to be determinants these assertions.
of research use that could be characterized as individual –
such as age [10,11], attitude [11-13], clinical area [12,14], Leadership refers to the 'nature of human relationships'
education [14-17], prior knowledge [15], employment with effective leadership being proposed to give rise to
status [10,16,17], experience [11,14,15], journals read clear roles, effective teamwork and effective organiza-
[18,19], and recently, critical thinking dispositions [20]. tional structures, as well as staff involvement in decision-
In a systematic review of the literature on the individual making and approach to learning [45]. The effect of lead-
determinants of research utilization by nurses, Estabrooks ership has received much attention. Previous research has
and colleagues identified a positive attitude toward shown that leadership is instrumental for cultural change
research as both the most frequently studied individual and has a strong effect on overall organizational perform-
determinant and the only one with a consistently positive ance [45,50,51]. There is also evidence that leadership is
effect [21]. Findings for all other individual determinants critical to nurses' decision-making processes [15,52]. and
in that review were equivocal. to creating a culture for evidence-based practice [6]. Addi-
tionally, research conducted in magnet hospitals in the
Less attention has been paid to the role of organizations United States (US) indicate that nurse leaders play a criti-
and context in promoting research use [21-23]. Histori- cal role in developing environments (i.e., contextual set-
cally, a number of organizational factors thought to influ- tings) that support nursing excellence and improved
ence innovation adoption in industry and health services patient outcomes [53-55].
have been studied. Those shown to have an influence on

Page 2 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

Evaluation, the third proposed core dimension of context, display. However, magnet hospital research in the US
refers to feedback mechanisms (individual and system does give us some idea of what such an ideal unit would
level), sources, and methods for evaluation [45]. Audit look like from staff retention and quality patient care per-
coupled with a feedback mechanism, where data is fed spectives. Consistently reported contextual and individual
back to a unit's providers in the form of some kind of nurse characteristics of magnet hospitals include effective
report, is one of the most commonly applied evaluation leadership (i.e., leaders who are visionary, enthusiastic,
mechanisms used in healthcare to implement the adop- supportive, value education and professional develop-
tion of research-based practices, and has been shown to ment, maintain open lines of communication with staff
have modest effects with physicians [56]. While its effect nurses), the ability of staff nurses to establish and main-
on nurses has been relatively untested, in one trial inves- tain therapeutic nurse-patient relationships, nurse auton-
tigators reported that audit and feedback together with omy and control, and collaborative nurse-physician
educational outreach and printed materials results in relationships at the unit level [54,66,67]. The 'ideal nurs-
moderate improvements in nursing care [57], lending ing unit' for research utilization may exhibit similar indi-
support to the importance of evaluation as a contextual vidual and contextual characteristics, although this is yet
predictor. to be empirically tested.

Additional support for investigating the role of context in In summary, while an understanding of research utiliza-
research utilization comes from studies correlating spe- tion in nursing is growing, there are gaps in what is known
cific contextual factors with research utilization behaviors about the factors that predict nurses' use of research.
of nurses. A number of investigators have correlated the Knowledge of those factors would inform the develop-
impact of organizational structures, roles, and policies ment of interventions to increase the use of research in the
designed to promote research use with the actual use of service of improved patient care. Individual determinants
specific research-based practices by nurses [13,14,26,58- of research use have been studied most frequently but
60]. Studies examining the impact of context on research findings are equivocal, making it difficult to plan inter-
implementation in both the nursing [e.g., [52,61,62]] and ventions to facilitate research use, even at the individual
organizational behaviour literature [e.g., [63]] also sup- level. Organizational determinants have been studied in
port the importance of contextual factors to research utili- industries beyond health; relatively few studies have been
zation, while stressing the interactivity among different conducted in hospital settings or with nurses. Further,
contextual factors. within healthcare organizations, nursing work is com-
monly organized at the patient care unit level, indicating
Despite growing support for the importance of organiza- a need to understand contextual factors at sub-levels (i.e.,
tional context to research utilization, little is known about patient care units) within the organization. Few reports
which contextual factors are important for research utili- examine work at the patient care unit level. Before inter-
zation by nurses and how these factors operate. This lack ventions to increase research use among nurses working
of certainty was evident in the findings from a Cochrane in hospitals can be optimally designed, investigators need
systematic review [64] on organizational infrastructures to identify and understand factors at both the hospital
for promoting research-based nursing interventions. The and the unit-levels [68]. In the study reported here, we
authors were not able to identify any studies meeting focused at the patient care unit level.
Cochrane standards.
Purpose
A more recent review [65] that was not restricted to rand- The purpose of this study was to identify and examine
omized control trials also assessed contextual factors and individual and contextual factors at the unit level that
research utilization in nursing staff. These investigators influence research utilization among nurses working in
reported that contextual factors (e.g., role, access to acute care hospitals, and to identify any differences
research, a favorable organizational climate towards between adult and pediatric units. The specific purpose of
research use, material support to attend conferences, time the analyses reported in this paper was to conceptually
to read research, and organizational educational activities model an ideal patient care unit, i.e., a patient care unit
such as mini-courses) had statistically significant but displaying features optimal for research use. We used a
inconsistent associations with research use. These findings descriptive approach to develop an organizational arche-
suggest that while the contexts in which nurses work may type to examine determinants of research utilization at the
be important to research use, further study in this area is patient care unit level. Using this approach, a framework
needed. for unit level research utilization was constructed based
on our understanding of a model nursing environment.
Little consensus exists among researchers on the features
that an 'ideal nursing unit' for research utilization would

Page 3 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

Theoretical framing tions, unit workload, unit environmental complexity, and

Rogers' diffusion of innovations theory [29,69]. has pro- unit culture. The only inclusion criterion for participants
vided valuable insight into the field of research utiliza- was to be a registered nurse employed in one of the seven
tion. This theory explains the spread of new ideas using participating units. Sealed questionnaire packages were
four main elements: the innovation, communication sent to all nurses working in the seven units, with two to
channels, time, and a social system. That is to say, diffu- three weeks allowed for completion. Participation was
sion is a process by which an innovation is communicated voluntary and anonymity was maintained. Posters, pam-
through certain channels over a period of time among phlets, and informal communication with on-site data
members of a social system. It is not a single all-encom- collectors during observation work were used as remind-
passing theory; rather it consists of four theoretical per- ers to complete the questionnaires and return them to a
spectives that relate to the overall concept of diffusion: centrally established location on the unit. Response rates
innovation-decision process theory, the individual inno- varied with each instrument according to the time (i.e.,
vativeness theory, the rate of adoption theory, and the the- month one or month six) of data collection (see Addi-
ory of perceived attributes. tional File 2). Across the seven units, 176 nurses partici-
pated at month one and 117 at month six. Analysis was
While the study reported here does not represent an performed on a sample of N = 235 [i.e. time one (N = 176)
empirical test of the diffusion of innovation theory, we + time two (N = 117) minus nurses at time two who
did use selected components of Rogers' [29] classical Dif- already filled out a survey at time one (N = 58)]. We
fusion of Innovation work (i.e., characteristics of the adopter excluded nurses at time two who already replied at time
and characteristics of the environment) to guide the selec- one so not to bias the findings by placing a greater weight
tion of variables for the original survey [70] of which an on the responses from individuals responding twice. Due
abbreviated form was used in this study. For example, to the short time frame (six months) between times one
characteristics of the adopter included individual varia- and two, we also elected to combine responses from both
bles such as age and experience while characteristics of the periods. Further, our qualitative analyses during this six
environment included organizational and contextual var- month interval did not show any evidence that the con-
iables such as unit culture and workload levels. See Addi- text of the units had changed and thus supported combin-
tional File 1 for a complete listing of all variables included ing time one and time two responses. Table 1 provides the
in the abbreviated version of the survey utilized in this demographic profile of the nurses who participated in the
study. study, and Table 2, a demographic profile of participating
units.
Methods
Design and Sample Instruments
Two adult surgical units (units one and two) and five Six instruments were used to collect the quantitative data:
pediatric surgical and specialty units (units three to seven) A Demographic (DEM) Inventory, a Research Utilization
embedded in four metropolitan, tertiary level hospitals in Survey, the Environmental Complexity Scale (ECS), the
two Canadian provinces, Alberta and Ontario, partici- Nursing Unit Cultural Assessment Tool (NUCAT) Version
pated in the study. Ethical approval for the study was 3, the Project Research in Nursing (PRN) 80, and the Cal-
obtained from the Universities of Alberta and Toronto ifornia Critical Thinking Disposition Inventory (CCTDI).
human research ethics committees and relevant univer- These are described briefly in sections that follow. The
sity-affiliated institutional research ethics boards. ECS and PRN were both completed by research associates
on the unit during the two separate week-long quantita-
Data Collection tive data collection periods, while the remainder of the
Consistent with an ethnographic approach, both qualita- instruments were self-administered by the nurses. A sam-
tive and quantitative data were collected. On each unit, ple of the items and scales used to measure the study var-
fieldwork (participant observation, interviews, and focus iables and corresponding reliability coefficients for scales
groups) was conducted over a six-month period yielding are shown in Additional File 1.
qualitative data on nurses, physicians, other health pro-
fessionals, patients and their families. Selected findings of Demographic (DEM) inventory
the qualitative analysis have been reported elsewhere The DEM developed for this study, included questions on
[71,72]. gender, age, education, hours of work per week, length of
shift, years working in nursing, membership in nursing
In months one and six of observations on each unit, two organizations or groups, and the number of years worked
one-week periods of quantitative data collection occurred. on the unit.
Using survey instruments, data were collected on research
use, organizational measures, critical thinking disposi-

Page 4 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

Table 1: Demographic characteristics of participant nurses by unit (N = 235)

Variables Unit 1 Unit 2 Unit 3 Unit 4 Unit 5 Unit 6 Unit 7 Overall

(N = 37) (N = 45) (N = 15) (N = 20) (N = 19) (N = 77) (N = 22) (N = 235)

Gender (%) Female 91.9 88.9 93.3 95.0 89.5 98.7 95.5 94.0
Male 8.1 11.1 6.7 5.0 10.5 1.3 4.6 6.0
Education (%)a LPN 14.3 0 13.3 10.0 5.3 0.0 0.0 4.3
RN Diploma 57.1 44.2 66.7 80.0 47.4 39.0 40.9 48.9
Bachelor's Degree 28.6 53.5 20.0 10.0 47.4 50.6 50.0 42.0
Master's Degree 0.0 2.3 0.0 0.0 0.0 9.2 9.1 4.3
Age (years) Mean (SD) 39.1 (10.6) 35.5 (8.8) 47.5 (9.3) 45.5 (7.6) 38.1 (9.6) 37.5 (8.4) 35.1 (7.8) 38.7 (9.5)
Years in Nursing Mean (SD) 12.9 (9.8) 10.5 (9.1) 20.9 (8.6) 20.6 (8.4) 13.1 (7.9) 12.8 (8.9) 10.0 (8.1) 13.4 (9.4)
Usual Shift Length Mean (SD) 10.6 (1.9) 11.6 (1.0) 11.1 (1.6) 8.0 (0.0) 11.4 (1.4) 11.8 (0.8) 11.2 (1.7) 11.1 (1.6)
(hours)

aNumbers may not add up to 100% due to missing values.

SD = standard deviation
Research utilization survey items divided into three subscales: unanticipated changes
The Research Utilization Survey was first developed and in patient acuity, re-sequencing planned in nursing work
reported by Estabrooks [70,73]. A shortened version of to accommodate others, and influence of students. Indi-
the original research utilization survey was used in this vidual items on each subscale were coded 0–10 (high
study. The shortened version consisted of 22 questions increase to high decrease) and summated to obtain final
divided into three sections: research utilization, kinds and subscale scores.
sources of knowledge for practice, and organizational
characteristics. Nursing unit cultural assessment tool v3 (NUCAT3)
The NUCAT3 was developed by Coeling [77,78]. The pri-
Environmental complexity scale (ECS) mary purpose of this tool is to describe and understand
The ECS [74-76] was designed to assess the amount and nurses' immediate work group in a unit setting. A list of 50
degree of work disruption experienced by nurses over the items in the form of questions, representing various
course of a shift. Since its original publication in 1997, the behaviours is listed mid-page in the questionnaire. A five-
scale has undergone several pilot tests, reviews, and mod- point scale on the left and right of each item allows nurses
ifications. The version used in this study consisted of 23 to indicate how important the behaviour is to them per-

Table 2: Hospital (N = 4) and unit (N = 7) profile.

Unit Profile

Unit 1 There were 37 RNs, including 17 full time and 14 part time RNs. The nurse manager was in charge of the unit. The majority of
patients was older than 50 years and stayed on average 4–5 days.
(adult)
Unit 2 There were 39 full time RNs, 17 part time RNs, and 10 casual RNs. The nurse manager was the leader on the unit. The patients
stayed 1–3 weeks on average.
(adult)
Unit 3 (pediatric) Weekdays 4 nurses and 2 support staff worked the day shift. On nights and weekends, staff consisted of 2 nurses with support
people. The clinical supervisor was the clinical leader on the unit; the unit manager took care of the managerial responsibilities
for the unit.
Unit 4 (pediatric) There were 17 full time RNs, 6 part time RNs, 2 LPNs and 11RNs relief in this unit. At the time of the study, the unit did not
have a manager which was partly compensated for by the senior operating officer and the patient care director. The majority of
the patients were discharged at that same day.
Unit 5 (pediatric) Altogether there were 29 permanent nurses on this unit including 1 nurse educator and 2 LPNs. Local clinical leadership was
provided by the clinical supervisor, while the unit manager performed the general administrative and leadership role, with some
guidance from the senior operating officer. The average length of patient stay was 3 days.
Unit 6 (pediatric) There was over 100 nursing staff in this unit, including 65 full time staff nurses, 25 part time staff nurses, 23 special assignment
staff, 12 resource persons and 9 nurse specialists. The unit was administered by the unit manager working collaboratively with
the medical clinical directors and the child health services manager.
Unit 7 (pediatric) There was 37 nursing staff including the unit manager and the child health services manager. The average daily admissions were
4–5.

The seven pediatric and adult acute care units were embedded in four urban, tertiary level hospitals in two cities, each affiliated with a university. Of
the four hospitals: one was a dedicated pediatric center, one had adult and pediatric units, and two were dedicated adult care hospitals. The seven
units included five pediatric units and two adult surgical units.

Page 5 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

sonally (left) and to the group as a whole (right). Based on group mean square, WMS is the within-group mean
the responses to the 50 items, five subscales were concep- square, and K is the number of subjects per group. The
tually created to reflect specific cultural indicators reflec- average K for unequal group size was calculated as K = (1/
tive of the behaviours for the nurses in this study. These [N - 1]) (ΣK - [ΣK2/ΣK]); 2) interclass correlation ICC (2)
subscales were co-worker support, questioning behaviour, = (BMS - WMS)/BMS; 3) η2 = SSB/SST, where SSB is the
continuing education, work values-creativity, and work sum of squares between groups and SST is the sum of
values-efficiency. squares total; and 4) ω2 = (SSB - [N - 1]WMS)/(SST +
WMS). For each nursing characteristic analyzed, there was
Project research in nursing 80 workload measurement tool strong agreement among nurses in each given unit when
(PRN) ICC(1) was greater than 0.1. Aggregated data were consid-
The PRN is a Canadian classification system used to meas- ered reliable when the F statistic from the ANOVA table
ure the level of nursing care required by patients in hospi- was statistically significant (p < 0.05) and/or ICC(2) was
tals and nursing homes [79]. It consists of seven major greater than 0.60 [82]. An indicator of effect size was η2,
categories: respiration, feeding and hydration, elimina- which was the proportion of variance in the individual
tion, hygiene and comfort, communication, treatments, factor accounted for by group membership [83], and ω2
and diagnostic procedures. Each category provides a list of was a measure of the relative strength of the aggregated
patient related needs, which are assigned a point value variable at the group level [84]. Table 3 contains the relia-
based on frequency and complexity. The total score, deter- bility and validity values of the data aggregated at the unit
mined by summing up the points from each of the seven level. Both η2 and ω2 are measures of validity of the aggre-
categories, is multiplied by five minutes to determine the gated data at the patient care unit level.
direct care time estimate for each patient. The higher the
point value the greater the amount of direct care required. To index diversity across units, a coefficient of variation
The PRN method of measuring care required has been was computed and used in a correspondence analysis. A
tested extensively and has undergone several iterations coefficient of variation is a quotient of standard deviation
since its development in 1972. In 1978, Chagnon, over the mean, and allows distributions among different
Audette, Lebrun, and Tilquin reported its construct and units to be compared [85]. It is expressed as a percentage,
predictive validity [80]. which constitutes a relative measure of dispersion. In
order to assess the relationship between various factors
Critical thinking dispositions inventory (CCTDI) across the seven units, the coefficient of variation was
The CCTDI is a 75-question, six-point 'agree/disagree' Lik- computed and the resulting quotient was multiplied by
ert-type scale. There are seven subscales to the inventory: 100 and denoted in the variation index. Variation indices
truth-seeking, open-mindedness, inquisitiveness, system- are commonly used in research for making comparisons
aticity, maturity, self-confidence, and analyticity. The [86-88]. In this study, the variation index matrix was then
maximum overall score attainable on this tool is 420, with analyzed using correspondence analysis, which is a statis-
each subscale contributing a maximum of 60 points. The tical visualization method for picturing the associations
standard scores for each subscale and all scales combined among the variables of a two-way contingency table. The
are 40 and 280 respectively. A score less than 40 on any object of a correspondence analysis is to obtain a graphi-
subscale or less than 280 overall indicates limitations or cal display in the form of a spatial map of rows (units) and
weakness, whereas subscale scores of 50 or higher and columns (factors), not only with respect to their marginal
overall scores at 350 or higher indicate a strength in criti- profile, but also among each other. Here, we used corre-
cal thinking dispositions [81]. spondence analysis to explore the association between the
pattern of factors (or determinants) and units. It should
Analysis be noted however that correspondence analysis is an
While research utilization and possible explanatory varia- exploratory technique, based on a philosophical orienta-
bles were measured at the individual level, the unit of tion that emphasizes the development of models that fit
analysis in this study was the patient care unit. To create the data, rather than the rejection of hypotheses based on
unit scores, data collected at the level of the individual the lack of fit (Benzecri's 'second principle'). Therefore,
nurse were aggregated to the level of the patient care unit statistical significance tests are not customarily applied to
by calculating group means. When Cronbach alpha was the results of a correspondence analysis, and are not
assessed, this was done at the individual level. One-way needed for the clustering of factors produced in a corre-
analysis of variance (ANOVA) was performed for each var- spondence analysis [89,90].
iable using the unit as the group variable. The source table
from the one-way ANOVA was used to calculate the fol-
lowing indices: 1) interclass correlation ICC (1) = (BMS -
WMS)/(BMS + [K - 1] WMS), where BMS is the between-

Page 6 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

Table 3: Reliability and validity of data aggregated at the unit level

Variable ANOVA Degrees of Freedom ICC(1) ICC(2) η2 ω2 Alpha

Overall RU 5.83** 6,264 0.11 0.83 0.12 0.00 --

Authority 2.85* 6,303 0.04 0.65 0.05 0.00 --
Attitude 1.08 6,303 0.00 0.07 0.02 0.00 --
Intent 2.34* 6,298 0.03 0.57 0.05 0.00 --
Belief 2.43* 6,285 0.03 0.59 0.05 0.00 0.85
People support 4.60** 6,181 0.09 0.78 0.14 0.00 0.89
Organizational support 21.56** 6,204 0.34 0.95 0.40 0.28 0.85
Re-sequencing 12.21** 6,359 0.19 0.92 0.17 0.06 0.81
Students 1.57 6,133 0.02 0.36 0.07 0.00 0.75
Acuity 16.15** 6,364 0.24 0.94 0.21 0.11 0.84
Coworker support 2.36* 6,149 0.06 0.58 0.09 0.03 0.72
Education 1.46 6,144 0.02 0.32 0.06 0.00 0.64
Behavior 1.62 6,152 0.03 0.38 0.06 0.00 --
Creativity 0.86 6,155 0.01 0.11 0.03 0.00 --
Efficiency 0.92 6,154 0.01 0.12 0.04 0.00 --
Total PRN 260.32** 6,1334 0.59 1.00 0.54 0.48 --
Total CT 1.54 6,140 0.03 0.36 0.06 0.00 --

(a) Analysis of variance (ANOVA): Measure used to compare differences in mean scores across seven units;
(b) p value for ANOVA F-statistics:* p < .05; **p < .01. The denominator, degree of freedom, differs for some variables owing to different
instruments;
(c) ICC = interclass correlation;
(d) η2: proportion of total information in a given factor at the individual level, which is captured by aggregated data;
(e) ω2: provides a relative measure of the strength of an independent variable, small effect < 0.06; medium effect, 0.06–0.15; large effect > 0.15

Results overall research utilization scores increased significantly

Reliability of aggregated nursing measures from the first to the second question (p < 0.001), and from
The reliability properties of the aggregated nursing data at the second to the third question (p < 0.05). Adjusted over-
the unit level are shown in Table 3. These properties sup- all research utilization scores were obtained by taking a
ported the reliability of the aggregated data at the unit weighted average of the score obtained from the three
level for over half of the variables: overall research utiliza- times. The first inquiry was given a weight of 1/6, the sec-
tion, authority, intent, belief, people support, organiza- ond was given a weight of 2/6, and the third was given a
tional support, re-sequencing, acuity, co-worker support, weight of 3/6. We assigned higher weights to the research
and total PRN. Statistically significant (p < .05) F statistics utilization question each time it appeared in the question-
and/or ICC(2) values greater than 0.60 indicate greater naire because participants learned more about research
reliability and justification for aggregating the variables at utilization over the course of questionnaire completion.
the unit level. The ICC(1) values greater than 0.00 indi- We reasoned that their answers were more reflective of
cate some degree of perceptual agreement of nurses about their true scores each time they encountered the question,
the mean values within each unit. That is, the nurses' per- thus requiring a greater weight be placed on later inquir-
ceptions about their own unit were highly similar. How- ies. Figure 1 shows the adjusted overall research utiliza-
ever, the relative effect sizes for both η2and ω2 values were tion scores with 'used research on about half the shifts'
smaller, with η2 indices ranging from 0.02 to 0.54 and ω2 (five on the seven-point scale) as a reference line across
indices ranging from 0.00 to 0.48. Negative ω2 indices are the seven units. Analysis of variance indicated that statis-
reported as 0.00 [84,91]. The smaller η2and ω2 indices tically significant differences existed among units on the
suggest that, as we aggregated data, our ability to assign overall research utilization score (p < 0.001).
the same meaning for a variable at the unit level that we
had at the individual level lessened considerably. As illustrated in Figure 1, the seven units fell into three
main groupings with respect to research utilization which
Research utilization we categorized as low (units one and four), moderate
Adjusted overall research utilization scores were used. (units three and five), and high (units two, six, and seven)
Overall research utilization was assessed with a single research utilization units. Units seven (pediatric), two
question asked at three different points in the question- (adult) and six (pediatric) had the highest mean scores of
naire: 'Overall, in the past year, how often have you used research utilization with means of 5.55 (SD = 1.31), 5.77
research in some aspect of your nursing practice?' (SD = 1.22) and 5.78 (SD = 1.10) respectively. We found
Repeated measures analysis of variance revealed that the no statistically significant difference between units two,

Page 7 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

p < 0.001). Generally speaking, adult units scored higher

than pediatric units (see Table 4). The overall mean score
for re-sequencing of work was 29.45 (SD = 7.94). Unit
two (adult) scored the highest (mean = 35.39, SD = 7.96)
and unit five (pediatric) scored the lowest (mean = 24.78,
SD = 5.75). The overall mean score for influence of stu-
dents was 11.77 (SD = 3.35). Unit one (adult) scored the
highest (mean = 14.33, SD = 5.30) and unit four (pediat-
ric) scored the lowest (mean = 10.00, SD = 0.00). The
overall mean score for changing patient acuity was 55.76
(SD = 13.72). Unit two (adult) scored the highest (mean
= 67.30, SD = 11.93) and unit three (pediatric) scored the
lowest (mean = 48.01, SD = 9.95).

Unit culture
Note: reference line = “half of the shifts” = 5 on the 7-point likert scale.
The NUCAT3 assesses and describes unit culture on five
subscales: co-worker support, questioning behavior, con-
Figure 1 utilization scores by unit
Research
Research utilization scores by unit. Note: reference line tinuing education, work values – creativity, and work val-
= "half of the shifts" = 5 on the 7-point likert scale. ues – efficiency. Units two (adult) and six (pediatric) had
the highest aggregated mean scores on three of these
dimensions of group behavior: work values – creativity,
six, and seven however on research utilization scores work values -efficiency, and continuing education. Units
(ANOVA, p > 0.05). In contrast, units one (adult) and four three (pediatric) and five (pediatric) had the highest
(pediatric) were the only units with mean scores of aggregated mean scores on questioning behavior and co-
research utilization less than 5. Again, there was also no worker support respectively. Differences between adult
statistically significant difference between units one and and pediatric units were not noted to be statistically signif-
four on research utilization scores (ANOVA, p > 0.05). icant.

Factors influencing research utilization Workload

Table 4 displays the mean scores of selected variables from The overall PRN aggregated mean score for each unit
the Research Utilization Survey, Environmental Complex- ranged from 149.69 (unit four – pediatric) to 592.04 (unit
ity Scale (ECS), Nursing Unit Cultural Assessment Tool v3 six – pediatric). Statistically significant differences
(NUCAT3), as well as total scores for the Project Research between adult and pediatric units were noted for the total
in Nursing (PRN) 80, and the Critical Thinking Disposi- score (p < 0.001).
tions Inventory (CCTDI).
Critical thinking
Research Utilization Survey The overall aggregated mean scores of critical thinking dis-
With respect to the Research Utilization Survey, unit six positions (CCTDI) for the seven units ranged from 256.71
(pediatric) had the highest aggregated mean scores for (unit three – pediatric) to 291.00 (unit seven – pediatric).
three of the six subscales: people support, belief suspen- Comparisons of critical thinking dispositions showed that
sion, and organizational support. In contrast, unit four adult and pediatric units did not differ significantly with
(pediatric) had the lowest aggregated mean scores for four respect to overall aggregated mean critical thinking scores.
of the six subscales: people support, attitude, intent, and
organizational support. Comparisons of research utiliza- Correspondence analysis
tion measures showed that adult and pediatric units did The full set of variables (except individual nurse demo-
not differ significantly. graphic variables) was entered into a correspondence
analysis, revealing a space (see Figure 2) structured along
Environmental complexity scale (ECS) two dimensions, which captured two thirds of the varia-
There are three subscales on the ECS: re-sequencing of bility (65.99%). As illustrated in Figure 2, critical thinking
work, influence of students, and changing patient acuity. dispositions and unit culture (as measured by work values
Statistically significant differences were noted between the – creativity, work values – efficiency and questioning
seven units on the three subscales (re-sequencing of work behavior) were found to be close to unit two (adult), a
– ANOVA F-test statistic = 13.352, p < 0.001; influence of high research utilization score unit with a research utiliza-
students – ANOVA F-test statistic = 2.615, p = 0.020, tion mean of 5.77, indicating an association between
changing patient acuity – ANOVA F-test statistic = 16.575, these factors and this unit. Unit culture (as measured by

Page 8 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

Table 4: Mean scores and standard deviations by unit

Unit 1 Unit 2 Unit 3 Unit 4 Unit 5 Unit 6 Unit 7 Overall

(Adult) (Adult) (Pediatric) (Pediatric) (Pediatric) (Pediatric) (Pediatric)

Research Utilization Survey

People Support 17.94 (7.05) 20.70 (6.51) 18.79 (6.32) 16.44 (7.97) 20.29 (7.87) 21.18 (6.47) 20.30 (6.31) 19.94 (6.87)
(Max score = 30)
Autonomy/Authority 2.52 (0.81) 2.86 (0.95) 3.11 (0.81) 2.53 (1.01) 2.96 (0.74) 2.59 (0.82) 2.96 (0.74) 2.72 (0.86)
(Range is 0–4)
Attitude (Range is 0–4) 2.91 (0.92) 3.19 (0.83) 3.00 (0.75) 2.72 (0.96) 2.92 (0.95) 3.02 (0.82) 2.93 (0.92) 3.00 (0.87)
Intent 1.78 (0.42) 1.76 (0.43) 1.53 (0.51) 1.44 (0.51) 1.52 (0.51) 1.67 (0.49) 1.70 (0.47) 1.68 (0.48)
(Range is 0–2)
Belief Suspension 2.13 (0.99) 2.37 (0.95) 2.47 (1.15) 2.29 (1.13) 2.37 (1.13) 2.50 (0.87) 2.11 (0.87) 2.34 (0.97)
(Range is 0–4)
Organizational Support 11.70 (4.23) 13.61 (5.15) 11.94 (5.32) 7.89 (2.65) 11.13 (2.85) 15.28 (4.14) 14.89 (2.36) 13.30 (4.61)
(Max. Score = 25)
Overall Research 3.94 (1.78) 5.43 (1.50) 4.47 (1.99) 3.59 (1.54) 4.43 (1.99) 5.18 (1.61) 5.16 (4.41) 4.80 (1.75)
Utilization #1
Overall Research 4.67 (1.85) 5.51 (1.61) 5.21 (1.89) 4.12 (1.87) 5.24 (1.81) 5.69 (1.39) 5.59 (1.60) 5.30 (1.68)
Utilization #2
Overall Research 4.83 (1.91) 5.83 (1.25) 5.06 (1.82) 5.19 (1.72) 5.56 (1.78) 5.93 (1.30) 5.59 (1.42) 5.56 (1.57)
Utilization #3
Adjusted (weighted) 4.62 (1.62) 5.77 (1.22) 5.05 (1.82) 4.63 (1.34) 5.28 (1.63) 5.78 (1.10) 5.55 (1.31) 5.24 (1.43)
Overall research
Utilization Score

Environmental Complexity Scale

Re-sequencing of work 28.50 (9.66) 35.39 (7.96) 28.24 (6.53) 30.0 (8.98) 24.78 (5.75) 27.21 (6.03) 30.72 (7.94) 29.45 (7.94)
(Range is 0–50)
Influence of Students 14.33 (5.30) 12.18 (1.78) 11.37 (3.13) 10.00 (0.00) 10.91 (2.79) 12.61 (3.72) 11.00 (2.61) 11.77 (3.35)
(Range is 0–20)
Changing patient acuity 54.77 (18.68) 67.30 (11.93) 48.01 (9.95) 52.35 (13.70) 50.70 (10.53) 53.27 (12.27) 57.05 (11.74) 55.76 (13.72)
(Range is 0–90)

Nursing Unit Cultural Assessment Tool (Group's Behavior)

Co-worker support 7.56 (2.20) 8.42 (1.69) 7.83 (1.75) 8.00 (1.25) 9.00 (1.07) 7.15 (1.74) 7.71 (1.49) 7.78 (1.78)
(Range is 0–10)
Questioning behavior 4.04 (0.82) 4.21 (0.83) 4.58 (0.52) 4.36 (0.67) 4.47 (0.64) 4.23 (0.84) 3.83 (0.92) 4.21 (0.81)
(Range is 0–5)
Continuing education 14.39 (2.74) 15.65 (2.98) 14.83 (2.67) 14.44 (2.60) 15.73 (2.21) 15.96 (2.27) 14.94 (2.07) 15.32 (2.52)
(Range is 0–20)
Work values 3.62 (0.98) 3.96 (0.89) 3.58 (0.79) 3.27 (0.79) 3.93 (0.70) 3.60 (0.92) 3.53 (0.91) 3.66 (0.89)
(creativity) (Range is
0–5)
Work values 4.31 (0.84) 4.36 (1.00) 4.08 (1.00) 4.10 (0.74) 4.13 (0.35) 4.24 (0.72) 3.78 (0.94) 4.19 (0.82)
(efficiency) (Range is
0–5)

Project Research in Nursing 80

Total PRN 255.42 248.37 188.54 149.69 217.41 592.04 307.94 303.84
(108.15) (82.98) (81.70) (24.59) (83.17) (157.84) (124.86) (184.41)

Critical Thinking Dispositions Inventory

Total CCTDI 286.26 281.65 256.71 283.86 288.60 279.61 291.00 281.78
(Max score = 420) (28.39) (31.38) (15.96) (25.63) (25.57) (25.54) (29.33) (27.58)

Page 9 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

co-worker support) appeared to have a close relationship When the research utilization scores in the high group
with units six (pediatric) and seven (pediatric), also high (adult unit two, pediatric units six and seven) are superim-
research utilization units. Another cluster included posed onto the correspondence analysis map they appear
authority to use research, unit culture (as measured by close to one another in physical proximity (see Figure 2)
importance of access to continuing education), environ- suggesting they share similar characteristics. However
mental complexity (as measured by work re-sequencing, units six and seven were closer to each other than to unit
changing patient acuity), attitude toward research, people two indicating there may be subtle differences between
support, belief suspension, and intent to use research, sug- factors that determine research use in adult compared to
gesting this cluster of factors are consistently associated pediatric units. The following factors clustered around the
with each other. An additional factor, influence of stu- three high research utilization units: changing patient acu-
dents, was far from all of the other factors, reflecting dis- ity, re-sequencing of work, attitude toward research, criti-
similarity with the other factors across the seven units. cal thinking dispositions, importance of access to
Unit four (pediatric) was also far from other units, but continuing education, work values (creativity and effi-
close to the factor of people support. We also observed ciency), authority, questioning behavior, and co-worker
that nursing workload (i.e., total PRN score) was more support, indicating an association between high research
associated with unit one (adult), and organizational sup- utilization units and these factors. Some of these factors
port with unit five (pediatric). clustered more closely around the units than others indi-
cating a possible stronger relationship with research use:
Superimposing the research utilization scores onto the unit culture [as measured by work values (creativity and
correspondence analysis map efficiency), authority, questioning behavior], and critical
Superimposing findings from the research utilization thinking dispositions.
scores onto the correspondence map revealed interesting
results. Using the results from the overall research utiliza- After superimposing the research utilization scores onto
tion scores, the units cluster in three distinct groups: low the correspondence analysis map we also realized that the
(units one and four), medium (units three and five) and units in the low group (units one and four) were unlike
high (units two, six, and seven). These are summarized in the other units. Units one and four had the lowest levels
Table 5. of overall research utilization scores and subsequently
plotted farther away from the other units (and each other)

Figure 2
Overall correspondence analysis map illustrating unit clustering with contextual factors
Overall correspondence analysis map illustrating unit clustering with contextual factors.

Page 10 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

Table 5: Mapping of correspondence analysis results onto unit groups based on research utilization scores

FACTORS Low Group Medium Group High Group

Units 1 and 4 Units 3 and 5 Units 2, 6, and 7

Influence of students (ECS) X

People support (RU) X
Total PRN score (PRN80) X
Organizational support (RU) X
Belief suspension (RU) X
Intent (RU) X
Changing patient acuity (ECS) X
Re-sequencing of work (ECS) X
Attitude (RU) X
Continuing education (NUCAT3) X
Critical thinking (CCTDI) X
Work values: Creativity (NUCAT3) X
Work values: Efficiency (NUCAT3) X
Authority (RU) X
Questioning behavior (NUCAT3) X
Coworker support (NUCAT3) X

The three groupings (low, medium, high) were based on the aggregated research utilization scores for each unit
'X' means that the factor sat closest to the respective unit group

in the correspondence analysis map. Nursing workload have been previously investigated. However, our unit of
(i.e., total PRN) and people support clustered close to unit analysis was the patient care unit, and therefore, the com-
one and unit four, respectively, indicating these two fac- parisons described between the findings of this study and
tors may be associated with lower research utilization past research where the individual nurse was the unit of
units. analysis should be interpreted with caution.

When units in the medium research utilization group Some of our findings are consistent with previous work in
(units three – pediatric and five – pediatric) were superim- the field. For instance, our finding that patient care units
posed onto the correspondence analysis plot we discov- with high and moderate levels of research use had the
ered a third clustering. In particular, we saw that these highest levels of co-worker and organizational support
units are not like the units in the two other groups. Organ- respectively is not new. Champion and Leach [11] found
izational support, belief suspension, and intent to use support from the unit director, chairperson, and director
research clustered more proximally to the medium group of nursing to be positively correlated with nurses' use of
than the other two cluster patterns, indicating an associa- research in their practice nearly 20 years ago. Hatcher and
tion between units with moderate research utilization and Tranmer [40] also reported small positive significant asso-
these three factors. ciations between the amount of organizational support
nurses perceived and their use of research in practice. In
Discussion addition, Varcoe and Hilton [60] demonstrated that the
This discussion focuses on individual and contextual fac- use of specific research-based practices was correlated with
tors and their role in research utilization by nurses. This organizational support.
study was exploratory in nature. Data were collected from
nurses employed on seven units. The unit of analysis was Our finding that patient care units with the highest levels
the patient care unit and our sample size was thus seven. of research utilization had, on aggregate, nurses with
Findings and interpretations must therefore be inter- more positive attitudes about research use is also not new.
preted cautiously and premature generalizations avoided. Nurses' positive attitude towards research has been con-
sistently shown to be associated at statistically significant
Other research utilization investigators have explored sev- levels with research use [21].
eral of the factors that we studied in this project. In partic-
ular, links between research utilization and attitudes Authority to use research was also associated with higher
toward and beliefs about research [11,60,92], continuing levels of research utilization. While there is no literature
education [10,19,93], critical thinking [20], and support that directly associates authority and research utilization,
for research use [11,40,60] at the individual nurse level there is support for this concept in the 'barriers to research

Page 11 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

utilization' literature in nursing. Several investigators have definitive statement on its value as an intervention to
noted that one of the most consistently reported barriers increase research use in practice can be made. In addition
to using research in practice for nurses is 'lack the author- to continuing education, recent work by Belkhodja et al.
ity to implement change based on research findings' [94- [48] found specific aspects of unit culture, such as the
102]. unit's research culture (i.e., research as the preferred
source of information) and the intensity of use of research
Our findings run counter to the work of some investiga- sources by the unit's members to also be positively corre-
tors. For example, Profetto-McGrath et al. [20] reported a lated (p < 0.05) with research utilization by healthcare
statistically significant positive correlation between criti- professionals on hospital units.
cal thinking dispositions and research utilization. Given
the work by Profetto-McGrath and colleagues, we would Pepler et al. [49] in a multiple case-study of research utili-
expect to see high critical thinking dispositions scores for zation on eight acute care units also found unit culture to
nurses on units two, six, and seven (i.e., high research uti- be a principal factor linked to patterns of research utiliza-
lization units) in comparison to the nurses on the other tion. However, while Pepler and colleagues identified sev-
study units. However, there were no statistically signifi- eral aspects of unit culture that were important to research
cant differences between nurses on total critical thinking utilization (e.g., harmony of research perspectives, moti-
dispositions scores even though critical thinking disposi- vation to learn, goal orientation, creativity, critical
tions did cluster around the 'high' research utilization inquiry, mutual respect, and maximization of resources)
units in the correspondence analysis. The Profetto- they also reported that the components of unit culture
McGrath, et al. work was conducted on a subset of the data were tightly intertwined resulting in a complexity which
used for this study. However, their unit of analysis was at represents a distinctive culture for each unit. While this
the level of the individual nurse, possibly accounting for represents early support for unit culture as a factor in
differences. It may be that critical thinking dispositions research utilization behaviours of nurses, further empiri-
are most productively studied as an individual level phe- cal support is needed before a statement regarding the
nomenon, as suggested by our non-significant ANOVA F- association between unit culture and research use can be
statistic and ICC(2) of < 0.60 for the total critical thinking confidently made.
dispositions aggregated mean scores (see Table 3).
In addition to the factors discussed above, we reported a
The culture of a unit defines the behavior of nurses number of other factors that have not been previously
through observable artifacts, values (i.e., norms, social studied with respect to nurses' research utilization behav-
principles and ideologies), beliefs, and attitudes [46,103]. ior. For example, links between research utilization and
As such, it constitutes a potential contextual determinant nursing workload, patient acuity, and re-sequencing of
of research utilization. In this study, 'high' research utili- work have not been previously explored, suggesting fruit-
zation units had the highest aggregated mean unit culture ful new avenues of inquiry. While we located no reports of
scores (as measured by importance of access to continuing these concepts having been studied in relation to research
education, work values – creativity, work values – effi- utilization, there are many studies reporting on nurse per-
ciency, questioning behavior, and co-worker support) ceived barriers to using research. Among these, investiga-
indicating that variables associated with unit culture tors consistently report a lack of time to read research and
reflect the vitality with which research utilization can be implement findings as one of the most frequently identi-
promoted within patient care units. Positive effects of cul- fied barriers [37,97,99]. Little clarification of what is
ture on research utilization have been suggested by several meant by time has been offered in these studies, although
scholars in the field [42,46,47] but, to date, we have rela- an implicit assumption is that nurses' lack of time pre-
tively little empirical evidence to support these assertions. vents research use. Our findings suggest this may not be
For example, while several previous studies have exam- the case. Two of the units with the highest workloads in
ined continuing education, an element of unit culture as the study reported here were units one and two (both
measured by the NUCAT3, in relation to nurse research adult units). Unit one was classified as a 'low' research uti-
utilization behavior, findings have been equivocal. lization unit and unit two, a 'high' research utilization
McCleary and Brown [104] found taking a course about unit, making it difficult to ascertain the direction of the
research design was positively associated with research relationship between workload and research use. How-
utilization. Rodgers [36] found that the number of study ever, these findings do lead us to propose that there may
days attended was associated with using more research in be contextual differences between units (e.g., primary ver-
practice. However, other investigators, have not found sus team nursing models, patient case mix, patient care
similar associations [10,36,105]. Further research examin- acuity, healthcare team composition) that influence
ing the link between nurse research utilization and con- nurses' research use.
tinuing education will be necessary before a more

Page 12 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

In addition to the unit contextual and individual factors Study Limitations

identified in the correspondence analysis as important to While this was a multi-centre study, the sample size in the
research use, the 'high' research utilization units (i.e., analyses reported here was relatively small and may have
units two, six, and seven) also had the highest proportions been inadequate to detect differences between units for
of baccalaureate and master prepared nurses and the some of the variables. This study was also exploratory in
youngest nurses (see Table 1). Education and age have nature and the findings drawn from seven units and the
been investigated in numerous previous research utiliza- nurses employed on those units. The results must be inter-
tion studies and investigators have reported equivocal preted with caution and are not generalizable either to
effects, at best, on nurse research utilization behaviours. nurses or units. While this study sheds light on the factors
For example, several studies showed no statistically signif- that may influence research use at the patient care unit
icant association between education and research use level, further research is needed to expand on this knowl-
[60,92,106]. while others showed the use of research in edge. In particular, contextual factors (nursing workload,
practice to be higher among nurses with baccalaureate/ patient acuity, and re-sequencing of work) that have not
masters degrees compared to those with registered nurse been previously reported in relation to research use sug-
diplomas [10,36,105]. Similarly, age has not been dem- gest directions for study.
onstrated to predict research use [10,19,92]. For this rea-
son, and because we were interested in identifying While we were able to identify and build a model of an
modifiable, or at least more readily modifiable, factors ideal patient care unit from a research utilization perspec-
influencing research use, we chose not to enter age and tive from our analyses, it is important to note that we did
education into our correspondence analysis. Other indi- not collect data on several potentially important contex-
vidual characteristics such as questioning behaviours and tual factors. For example, Greenhalgh et al. [107], in a
belief suspension were entered in the correspondence review of the diffusion of service innovations, identified
analysis because we postulated they would be modifiable several structural factors that have been shown to influ-
through continuing education. Age is not modifiable and ence the likelihood of innovation adoption (e.g., size, bed
education, while modifiable, would require long-term capacity, functional differentiation, decision-making
commitment. structure, slack resources). Future research examining
research utilization patterns at the unit level should incor-
The archetypical unit porate such structural factors.
The specific purpose of the analyses presented in this
paper was to model an ideal patient care unit. In such an Aggregating individual nurse scores on a variable of inter-
ideal or archetypical patient care unit factors would be est to obtain scores for the unit on that characteristic can
optimized to facilitate research use. We identified a also introduce bias into the findings if the variable takes
number of such modifiable factors or characteristics that on a different meaning and thus has different effects at
were associated in this study with patient care units that various levels of analysis. Reliability and validity measures
reported greater research use (see Table 5). In such units, for the following variables of interest raise questions
these characteristics included unit culture, (specifically: about their suitability for aggregation: attitude, critical
co-worker support, questioning behavior, importance of thinking dispositions, workload (influence of students),
access to continuing education, work values – creativity, and some unit culture variables (e.g., importance of access
work values – efficiency), environmental complexity, to continuing education, work values – creativity, work
workload, authority to use research, positive attitudes values – efficiency, questioning behavior).
towards research, and stronger critical thinking disposi-
tions. These findings illustrate both the complex nature of Finally, we adjusted the research utilization score used in
research utilization and the shortcomings of models that the correspondence analysis by taking a weighted average
address only individual or unit level dimensions. Either of of the score obtained from asking the question on three
these dimensions (individual, unit/contextual), while separate occasions throughout the survey. We assigned
necessary, is insufficient to adequately explain the com- higher weights to the research utilization question each
plex behavior changes required by nurses who use time it appeared in the questionnaire because we hypoth-
research optimally and appropriately. Importantly, our esized participants learned more about research utiliza-
modeling of such an archetypical patient care unit, tion over the course of questionnaire completion.
allowed us to identify contextual factors (e.g., importance However, it is also possible that participants may have
of access to continuing education, co-worker support, obtained higher scores on the question each subsequent
questioning behavior) that can be modified to increase time it appeared because they learned how to answer the
research use. question.

Page 13 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

Conclusion participated in data analysis and interpretation, provided

Our findings offer preliminary support for the argument critical commentary and served as senior advisor to the
that context matters. Contextual factors at the patient care team and principal investigator. All authors read and
unit level, in addition to individual nurse characteristics, approved the final manuscript.
were important to promoting research utilization by
nurses. By studying several different patient care units, we Additional material
were able to suggest modifiable components of context at
the patient care unit level that may be important determi-
nants of nurses' use of research. We were also able to Additional file 1
additional table 1. Instrument Properties.
model an archetypical patient care unit, that is, a patient Click here for file
care unit displaying features optimal for research use. [http://www.biomedcentral.com/content/supplementary/1748-
Contextual features identified for such a unit included: 5908-3-31-S1.doc]
higher reported unit culture [as measured by importance
of access to continuing education, work values (creativity Additional file 2
and efficiency), questioning behavior, and co-worker sup- additional table 2. Number of Nurses Participating by Unit
port] and lower reported environmental complexity (as Click here for file
[http://www.biomedcentral.com/content/supplementary/1748-
measured by changing patient acuity and re-sequencing of 5908-3-31-S2.doc]
work). These factors represent modifiable conditions in
the hospital environment and have important practical
implications for work and unit structures and for organiz-
ing nursing service delivery to enhance nurses' use of Acknowledgements
research findings to improve patient outcomes. This work was supported by grants-in-aid from the Canadian Institutes of
Health Research (CIHR) and the Alberta Heritage Foundation for Medical
Competing interests Research (AHFMR). We would also like to thank William Midodzi and Lin-
The authors declare that they have no competing interests. glong Kong, University of Alberta, Canada for their assistance with data
analysis.
Authors' contributions
CAE conceived the study and its design, secured funding, References
1. Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M: Planning
provided leadership and coordination for the two projects and studying improvement in patient care: the use of theo-
and participated in data analysis and interpretation, writ- retical perspectives. Milbank Q 2007, 85:93-138.
ing, and final approval of the submitted manuscript, SS, 2. Bonetti D, Eccles M, Johnston M, Steen N, Grimshaw J, Baker R,
Walker A, Pitts N: Guiding the design and selection of inter-
KMcG, and JPM participated in data collection and con- ventions to influence the implementation of evidence-based
current data analysis, SS participated in drafting the man- practice: an experimental simulation of a complex interven-
tion trial. Soc Sci Med 2005, 60:2135-47.
uscript, JES made substantial contributions to data 3. Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the
analysis and interpretation and made major contributions behavior of healthcare professionals: The use of theory in
to writing of the manuscript, BS participated in conceptu- promoting the uptake of research findings. J Clin Epidemiol
2005, 58:107-12.
alization of study, securing grant funding, in the start-up 4. The Improved Clinical Effectiveness through Behavioural Research
of the study (with data collection) and served as a lead Group (ICEBeRG): Designing theoretically-informed imple-
investigator for the pediatric study, coordinating one of mentation interventions. Implementation Science 2006, 1:.
5. Rogers PL: Barriers to adopting emerging technologies in edu-
the participating sites, JWW participated in conceptualiza- cation. Journal of Educational Computing Research 2000, 22:455-472.
tion of study, securing grant funding, in the start-up of the 6. Stetler CB, Brunell M, Giuliano KK, Prince L, Newell-Stokes V: Evi-
dence-based practice and the role of nursing leadership. J
study (with data collection) and served as a lead investiga- Nurs Adm 1998, 28:45-53.
tor for the adult study, coordinating one of the participat- 7. Veeramah V: Utilization of research findings by graduate
ing sites, JL participated in conceptualization of study, nurses and midwives. J Adv Nurs 2004, 47:183-191.
8. Hodnett ED, Kaufmann K, O'Brien-Pallas L, Chipman M, Watson-
securing grant funding and in the start-up of the study MacDonell J, Hunsburger W: A strategy to promote research-
(with data collection), LOP participated in study concep- based nursing care: Effects on childbirth outcomes. Res Nurs
Health 1996, 19:13-20.
tion, served as a senior advising member on work envi- 9. Thompson DS, Estabrooks CA, Scott-Findlay S, Moore K, Wallin L:
ronment measures, and funded the collection of Interventions aimed at increasing research use in nursing: A
workload data in two hospitals. KGB participated in inter- systematic review. Implementation Science 2007, 2:15.
10. Butler L: Valuing research in clinical practice: A basis for
pretation of the findings, GD participated in study con- developing a strategic plan for nursing research. Can J Nurs
ception, data collection and interpretation, GB Res 1995, 27:33-39.
participated in start-up of the study helping to shape the 11. Champion VL, Leach A: Variables related to research utilization
in nursing: An empirical investigation. J Adv Nurs 1989,
sampling and data collection activities, CKH coordinated 14:705-710.
the data linkage activities, participated in data analysis 12. Bostrom J, Suter WN: Research utilization: Making the link to
practice. J Nurs Staff Dev 1993, 9:28-34.
and interpretation, and provided critical commentary, JW

Page 14 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

13. Coyle LA, Sokop AG: Innovation adoption behavior among 39. Bogdan-Lovis EA, Sousa A: The contextual influence of profes-
nurses. Nurs Res 1990, 39:176-180. sional culture: certified nurse-midwives' knowledge of and
14. Michel Y, Sneed NV: Dissemination and use of research findings reliance on evidence-based practice. Soc Sci Med 2006,
in nursing practice. J Prof Nurs 1995, 11:306-311. 62:2681-93.
15. Hancock HC, Easen PR: The decision-making processes of 40. Hatcher S, Tranmer J: A survey of variables related to research
nurses when extubating patients following cardiac surgery: utilization in nursing practice in the acute care setting. Can J
an ethnographic study. Int J Nurs Stud 2006, 43:693-705. Nurs Adm 1997, 10:31-53.
16. Tranmer JE, Lochhaus-Gerlach J, Lam M: The effect of staff nurse 41. Ring N, Malcolm C, Coull A, Murphy-Black T, Watterson A: Nursing
participation in a clinical nursing research project on atti- best practice statements: an exploration of their implemen-
tude towards, access to, support of and use of research in the tation in clinical practice. J Clin Nurs 2005, 14:1048-58.
acute care setting. Can J Nurs Leadersh 2002, 15:18-26. 42. Kitson A, Harvey G, McCormack B: Enabling the implementa-
17. Tsai S-L: The effects of research utilization in-service pro- tion of evidence based practice: a conceptual framework.
gram on nurses. Int J Nurs Stud 2003, 40:105-114. Qual Health Care 1998, 7:149-58.
18. Rutledge DN, Greene P, Mooney K, Nail LM, Ropka M: Use of 43. Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B,
research-based practices by oncology staff nurses. Oncol Nurs Titchen A: An exploration of the factors that influence the
Forum 1996, 23:1235-1244. implementation of evidence into practice. J Clin Nurs 2004,
19. Rodgers SE: A study of the utilization of research in practice 13:913-24.
and the influence of education. Nurse Educ Today 2000, 44. Rycroft-Malone J: The PARIHS framework – A framework for
20:279-287. guiding the implementation of evidence-based practice. J
20. Profetto-McGrath J, Hesketh KL, Lang S, Estabrooks CA: A study of Nurs Care Qual 2004, 19:297-304.
critical thinking and research utilization among nurses. West 45. McCormack B, Kitson A, Harvey G, Rycroft-Malone J, Titchen A,
J Nurs Res 2003, 25:322-37. Seers K: Getting evidence into practice: The meaning of 'con-
21. Estabrooks CA, Floyd JA, Scott-Findlay S, O'Leary KA, Gushta M: text'. J Adv Nurs 2002, 38:94-104.
Individual determinants of research utilization: A systematic 46. Scott-Findlay S, Golden-Biddle K: Understanding how organiza-
review. J Adv Nurs 2003, 43:506-520. tional culture shapes research use. J Nurs Adm 2005, 35:359-65.
22. Dobbins M, Ciliska D, Mitchell A: Dissemination and Use of Research 47. Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K,
Evidence for Policy and Practice by Nurses: A Model of Development and Titchen A, Estabrooks C: Ingredients for change: Revisiting a
Implementation Strategies. Toronto, ON: Author 1998. conceptual framework. Qual Saf Health Care 2002, 11:174-180.
23. Estabrooks CA: Translating research into practice: Implica- 48. Belkhodja O, Amara N, Landry R, Ouimet M: The extent and
tions for organizations and administrators. Can J Nurs Res organizational determinants of research utilization in Cana-
2003, 35:53-68. dian health services organizations. Sci Commun 2007,
24. Damanpour F: Organizational complexity and innovation: 28:377-417.
Developing and testing multiple contingency models. Manage 49. Pepler CJ, Edgar L, Frisch S, Rennick J, Swidzinski M, White C, Brown
Sci 1996, 42:693-716. TG, Gross J: Unit culture and research-based nursing practice
25. Kimberly J, Evanisko M: Organizational innovation: The influ- in acute care. Can J Nurs Res 2005, 37:66-85.
ence of individual, organizational, and contextual factors on 50. Tourangeau AE, Lemonde M, Luba M, Dakers D, Alksnis C: Evalua-
hospital adoption of technological and administrative inno- tion of a leadership development intervention. Nurs Leadersh
vations. Acad Manage J 1981, 24:689-713. 2003, 16:91-104.
26. Brett JLL: Organizational integrative mechanisms and adop- 51. Ovretveit J: Leading improvement. Journal of Health Organization
tion of innovations by nurses. Nurs Res 1989, 38:105-110. and Management 2005, 19:413-430.
27. Howell J, Higgins C: Champions of change: Identifying under- 52. Angus J, Hodnett E, O'Brien-Pallas L: Implementing evidence-
standing, and supporting champions of technological innova- based nursing practice: A tale of two intrapartum nursing
tions. Organ Dyn 1990, 19:40-55. units. Nurs Inq 2003, 10:218-228.
28. Markham S, Green S, Basu R: Champions and antagonists: Rela- 53. Lewis CK, Matthews JH: Quality watch. Magnet program desig-
tionships with R&D project characteristics and manage- nates exceptional nursing services. Am J Nurs 1998, 98:51-52.
ment. Journal of Engineering and Technology Management 1991, 54. Upenieks VV: Assessing differences in job satisfaction of nurses
8:217-242. in magnet and nonmagnet hospitals. J Nurs Adm 2002,
29. Rogers E: Diffusion of Innovations New York: The Free Press; 1995. 32:564-576.
30. Scott S, Bruce R: Determinants of innovative behaviour: A 55. Edgar L, Herbert R, Lambert S, MacDonald JA, Dubois S, Latimer M:
path model of individual innovation in the workplace. Acad The Joint Venture Model of Knowledge Utilization: A guide
Manage J 1994, 37:580-607. for change in nursing. Nurs Leadersh (Tor Ont). 2006, 19(2):41-55.
31. Damanpour F: Organizational innovation: A meta-analysis of 56. Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD:
effects of determinants and moderators. Acad Manage J 1991, Audit and feedback: Effects on professional practice and
34:555-590. health care outcomes. Cochrane Database Syst Rev
32. Pettengill MM, Gillies DA, Clark CC: Factors encouraging and 2006:CD000259.
discouraging the use of nursing research findings. Image J Nurs 57. Cheater FM, Baker R, Reddish S, Spiers N, Wailoo A, Gillies C, Rob-
Sch 1994, 26:143-147. ertson N, Cawood C: Cluster randomized controlled trial of
33. McCaughan D, Thompson C, Cullum N, Sheldon TA, Thompson DR: the effectiveness of audit and feedback and educational out-
Acute care nurses' perceptions of barriers to using research reach on improving nursing practice and patient outcomes.
information in clinical decision-making. J Adv Nurs 2002, Med Care 2006, 44:542-51.
39:46-60. 58. Crane J: Factors associated with the use of research-based
34. Royle J, Blythe J, Ciliska D, Ing D: The organizational environ- knowledge in nursing. In PhD thesis University of Michigan; 1990.
ment and evidence-based nursing. Can J Nurs Leadersh 2000, 59. French B: Contextual factors influencing research use in nurs-
13:31-37. ing. Worldviews Evid Based Nurs 2005, 2:172-83.
35. Oranta O, Routasalo P, Hupli M: Barriers to and facilitators of 60. Varcoe C, Hilton A: Factors affecting acute-care nurses' use of
research utilization among Finnish registered nurses. J Clin research findings. Can J Nurs Res 1995, 27:51-71.
Nurs 2002, 11:205-13. 61. Hunt M: The process of translating research findings into
36. Rodgers SE: The extent of nursing research utilization in gen- practice. J Adv Nurs 1987, 12:101-110.
eral medical and surgical wards. J Adv Nurs 2000, 32:182-193. 62. Davies S, McDonnell A, Brown J, Shewan J: Practice nurses' use of
37. Funk SG, Champagne MT, Wiese RA, Tornquist EM: BARRIERS: evidence-based research. Nurs Times 1999, 95:57-60.
The barriers to research utilization scale. Appl Nurs Res 1991, 63. Ferlie E, Fitzgerald , Wood M: Getting evidence into clincial
4:39-45. practice: An organisational behaviour perspective. J Health
38. Walczak JR, McGuire DB, Haisfield ME, Beezley A: A survey of Serv Res Policy 2000, 5:96-102.
research-related activities and perceived barriers to 64. Foxcroft DR, Cole N: Organisational infrastructures to pro-
research utilization among professional oncology nurses. mote evidence based nursing practice. Cochrane Database Syst
Oncol Nurs Forum 1994, 21:710-715. Rev 2003:CD002212.

Page 15 of 16
(page number not for citation purposes)
Implementation Science 2008, 3:31 http://www.implementationscience.com/content/3/1/31

65. Meijers JM, Janssen MA, Cummings GG, Wallin L, Estabrooks CA, 89. Greenacre MJ: Theory and Applications of Correspondence Analysis
Halfens R: Assessing the relationships between contextual fac- Orlando, FL: Academic Press; 1984.
tors and research utilization in nursing: Systematic litera- 90. Panagiotakos DB, Pitsavos C: Interpretation of epidemiological
ture review. J Adv Nurs 2006, 55:622-35. data using multiple correspondence analysis and log-linear
66. Laschinger HK, Almost J, Tuer-Hodes D: Workplace empower- models. Journal of Data Science 2004, 2:75-86.
ment and magnet hospital characteristics: Making the link. J 91. Meyers LS, Gamst G, Guarino AJ: Applied multivariate research:
Nurs Adm 2003, 33:410-22. Design and interpretation. Thousand Oaks: SAGE Publications;
67. Scott JG, Sochalski J, Aiken LH: Review of magnet hospital 2006.
research: Findings and implication for professional nursing 92. Estabrooks CA: The conceptual structure of research utiliza-
practice. J Nurs Adm 1999, 29:9-19. tion. Res Nurs Health 1999, 22:203-16.
68. Li YF, Lake ET, Sales AE, Sharp ND, Greiner GT, Lowy E, Liu CF, 93. McCleary L, Brown GT: Association between nurses' education
Mitchell PH, Sochalski JA: Measuring nurses' practice environ- about research and their research use. Nurse Educ Today 2003,
ments with the revised nursing work index: evidence from 23:556-565.
registered nurses in the Veterans Health Administration. 94. Carroll D, Greenwood R, Lynch K, Sullivan J, Ready C, Fitzmaurice J:
Res Nurs Health 2007, 30:31-44. Barriers and facilitators to the utilization of nursing
69. Rogers EM: Diffusion of Innovations New York: The Free Press; 1983. research. Clin Nurse Spec 1997, 11:207-212.
70. Estabrooks CA: Research utilization in nursing: An examina- 95. Dunn V, Crichton N, Roe B, Seers K, Williams K: Using research
tion of formal structure and influencing factors. In PhD thesis for practice: A UK experience of the BARRIERS Scale. J Adv
University of Alberta, Faculty of Nursing; 1997. Nurs 1997, 26:1203-10.
71. Estabrooks CA, Rutakumwa W, O'Leary K, Profetto-McGrath J, Mil- 96. Funk SG, Champagne MT, Wiese RA, Tornquist EM: Barriers to
ner M, Levers M, Scott-Findlay S: Sources of practice knowledge using research findings in practice: The clinician's perspec-
among nurses. Qual Health Res 2005, 15:460-476. tive. Appl Nurs Res 1991, 4:90-95.
72. Thompson DS, O'Leary K, Jensen E, Scott-Findlay S, O'Brien-Pallas L, 97. Nilsson Kajermo K, Nordstrom G, Krusebrant L, Bjorvell H: Barri-
Estabrooks CA: The relationship between busyness and ers to and facilitators of research utilization, as perceived by
research use: It is about time. J Clin Nurs 2008, 17:539-548. a group of registered nurses in Sweden. J Adv Nurs 1998,
73. Estabrooks CA: Modeling the individual determinants of 27:798-807.
research utilization. West J Nurs Res 1999, 21:758-772. 98. Nolan M, Morgan L, Curran M, Clayton J, Gerrish K, Parker K: Evi-
74. O'Brien-Pallas L, Irvine D, Peereboom E, Murrary M: Measuring dence-based care: Can we overcome the barriers? Br J Nurs
nursing workload: Understanding the variability. Nurs Econ 1998, 7:1273-8.
1997, 15:171-182. 99. Parahoo K: Barriers to, and facilitators of, research utilization
75. O'Brien-Pallas L-L, Doran DI, Murray M, Cockerill R, Sidani S, Lauri- among nurses in Northern Ireland. J Adv Nurs 2000, 31:89-98.
Shaw B, Lochhaas-Gerlach J: Evaluation of a client care delivery 100. Retsas A: Barriers to using research evidence in nursing prac-
model, Part 1: Variability in nursing utilization in community tice. J Adv Nurs 2000, 31:599-606.
home nursing. Nurs Econ 2001, 19:267-276. 101. Retsas A, Nolan M: Barriers to nurses' use of research: An Aus-
76. O'Brien-Pallas L-L, Irvine Doran D, Murray M, Cockerill R, Sidani S, tralian hospital study. Int J Nurs Stud 1999, 36:335-43.
Laurie-Shaw B, Lochhaas-Gerlach J: Evaluation of a client care 102. Rutledge DN, Ropka M, Greene PE, Nail L, Mooney KH: Barriers to
delivery model, Part 2: Variability in client outcomes in com- research utilization for oncology staff nurses and nurse man-
munity home nursing. Nurs Econ 2002, 20:13-21. agers/clinical nurse specialists. Oncol Nurs Forum 1998,
77. Coeling H: Nursing Unit Cultural Assessment Tool. Version 3 Kent, OH: 25:497-506.
Kent State University; 1991. 103. Schein EH: Organizational Culture and Leadership San Francisco: Jossey-
78. Coeling HVE, Simms L: Facilitating innovation at the nursing Bass; 1992.
unit level through cultural assessment, Part I: How to keep 104. McCleary L, Brown GT: Barriers to paediatric nurses' research
management ideas from falling on deaf ears. J Nurs Adm 1993, utilization. J Adv Nurs 2003, 42:364-72.
23(4):46-53. 105. Tsai S-L: Nurses' participation and utilization of research in
79. Tilquin C, Carle J, Saulnier D, Lambert P: Collaborators: Le Systeme PRN the Republic of China. Int J Nurs Stud 2000, 37:435-444.
Montréal, QC: La Mesure du Niveau des Soins Infirmiers Requis; 106. Bostrom J, Suter WN: Research utilization: Making the link to
1981. practice. J Nurs Staff Dev 1993, 9:28-34.
80. Chagnon M, Audette LM, Lebrun L, Tilqu in C: The PRN 76 System: 107. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffu-
User Guide of the PRN 76 Form [Montreal, QC]: Project de Recherche sion of innovations in service organizations: Systematic
en Nursing; 1978. review and recommendations. Milbank Q 2004, 82:581-629.
81. Facione NC: Critical Thinking Assessment in Nursing Education Programs:
An Aggregate Data Analysis Millbrae, CA: California Academic Press;
1997.
82. Glick WH: Conceptualizing and measuring organizational and
psychological climate: Pitfalls in multilevel research. Acad
Manage Rev 1985, 10:601-616.
83. Rosenthal R, Rosnow RL: Essentials of Behavioural Research. Methods
and Data Analysis New York: McGraw Hill; 1991.
84. Keppel G: Design & Analysis. A Researcher's Handbook Englewood Cliffs,
NJ: Prentice-Hall; 1991.
85. Snedecor GW, Cochran WG: Statistical Methods Arnes, IA: Iowa State Publish with Bio Med Central and every
University; 1989.
86. Ammenwerth E, Wolff AC, Knaup P, Ulmer H, Skonetzki S, van Bem- scientist can read your work free of charge
mel JH, McCray AT, Haux R, Kulikowski C: Developing and eval- "BioMed Central will be the most significant development for
uating criteria to help reviewers of biomedical informatics disseminating the results of biomedical researc h in our lifetime."
manuscripts. J Am Med Inform Assoc 2003, 10:512-4.
87. Bertini A, Di Bello V, Pedrinelli R, Giorgi D, Talini E, Dell'Omo G, Sir Paul Nurse, Cancer Research UK
Mariani M: P-527: Cyclic variation of the myocardial inte- Your research papers will be:
grated backscatter signal and geometric remodeling in
essential arterial hypertension. Am J Hypertens 2001, available free of charge to the entire biomedical community
14:207A-208A. peer reviewed and published immediately upon acceptance
88. Umene K, Nunoue T: Genetic diversity of human parvovirus
B19 determined using a set of restriction endonucleases rec- cited in PubMed and archived on PubMed Central
ognizing four or five base pairs and partial nucleotide yours — you keep the copyright
sequencing: use of sequence variability in virus classification.
J Gen Virol 1991, 72(Pt 8):1997-2001. Submit your manuscript here: BioMedcentral
http://www.biomedcentral.com/info/publishing_adv.asp

Page 16 of 16
(page number not for citation purposes)