Home 52/9 Assgn

52/9 Assgn

The collection of evidence is an activity that occurs with an endgame in mind. For example, law enforcement professionals collect evidence to support a decision to charge those accused of criminal activity. Similarly, evidence-based healthcare practitioners collect evidence to support decisions in pursuit of specific healthcare outcomes.

In this Assignment, you will identify an issue or opportunity for change within your healthcare organization and propose an idea for a change in practice supported by an EBP approach.

To Prepare:

Reflect on the four peer-reviewed articles you critically appraised in Module 4.
Reflect on your current healthcare organization and think about potential opportunities for evidence-based change.

The Assignment: (Evidence-Based Project)

Part 5: Recommending an Evidence-Based Practice Change

Create an 8- to 9-slide PowerPoint presentation in which you do the following:

Briefly describe your healthcare organization, including its culture and readiness for change. (You may opt to keep various elements of this anonymous, such as your company name.)
Describe the current problem or opportunity for change. Include in this description the circumstances surrounding the need for change, the scope of the issue, the stakeholders involved, and the risks associated with change implementation in general.
Propose an evidence-based idea for a change in practice using an EBP approach to decision making. Note that you may find further research needs to be conducted if sufficient evidence is not discovered.
Describe your plan for knowledge transfer of this change, including knowledge creation, dissemination, and organizational adoption and implementation.
Describe the measurable outcomes you hope to achieve with the implementation of this evidence-based change.
Be sure to provide APA citations of the supporting evidence-based peer reviewed articles you selected to support your thinking.
Add a lessons learned section that includes the following:
A summary of the critical appraisal of the peer-reviewed articles you previously submitted
An explanation about what you learned from completing the Evaluation Table within the Critical Appraisal Tool Worksheet Template (1-3 slides)

The Connection Between Evidence-Based
Medicine and Shared Decision Making

Evidence-based medicine (EBM) and shared decision
making (SDM) are both essential to quality health care,
yet the interdependence between these 2 approaches
is not generally appreciated. Evidence-based medicine
should begin and end with the patient: after finding
and appraising the evidence and integrating its infer-
ences with their expertise, clinicians attempt a deci-
sion that reflects their patient’s values and circum-
stances. Incorporating patient values, preferences, and
circumstances is probably the most difficult and poorly
mapped step—yet it receives the least attention.1 This
has led to a common criticism that EBM ignore s
patients’ values and preferences—explicitly not its
intention.2

Shared decision making is the process of clinician
and patient jointly participating in a health decision af-
ter discussing the options, the benefits and harms, and
considering the patient’s values, preferences, and cir-
cumstances. It is the intersection of patient-centered
communication skills and EBM, in the pinnacle of good
patient care (Figure).

One Without the Other?
These approaches, for the most part, have evolved in
parallel, yet neither can achieve its aim without the other.
Without SDM, authentic EBM cannot occur.3 It is a
mechanism by which evidence can be explicitly brought
into the consultation and discussed with the patient.
Even if clinicians attempt to incorporate patient prefer-
ences into decisions, they sometimes erroneously
guess them. However, it is through evidence-informed

deliberations that patients construct informed prefer-
ences. For patients who have to implement the deci-
sion and live with the consequences, it may be more per-
tinent to realize that it is through this process that
patients incorporate the evidence and expertise of the
clinician, along with their values and preferences, into
their decision-making. Without SDM, EBM can turn into
evidence tyranny. Without SDM, evidence may poorly
translate into practice and improved outcomes.

Likewise, without attention to the principles of EBM,
SDM becomes limited because a number of its steps are
inextricably linked to the evidence. For example, discus-
sions with patients about the natural history of the con-
dition, the possible options, the benefits and harms of
each, and a quantification of these must be informed by

the best available research evidence. If SDM does not in-
corporate this body of evidence, the preferences that pa-
tients express may not be based on reliable estimates
of the risks and benefits of the options, and the result-
ing decisions not truly informed.

Why Is There a Disconnect?
A contributor to the existing disconnect between EBM
and SDM may be that leaders, researchers, and teach-
ers of EBM, and those of SDM, originated from, and his-
torically tended to practice, research, publish, and col-
laborate, in different clusters. Some forms of SDM have
emerged from patient communication, with much of its
research presented in conferences and journals in this
field. A seminal paper in 19974 conceptualized SDM as
a model of treatment decision making and as a patient-
clinician communication skill. However, it did so with-
out any connection to EBM—perhaps not surprisingly, be-
cause EBM was in its infancy.2

Conversely, with its origins in clinical epidemiology,
much of the focus of EBM has been on methods and
resources to facilitate locating, appraising, and synthe-
sizing evidence. There has been much less focus on dis-
cussing this evidence with patients and engaging with
them in its use (sometimes even disparagingly referred
to as “soft” skills). Most of the EBM attention has
involved scandals (eg, unpublished data, results “spin,”
conflicts of interest) and the high technology mile-
stones (eg, systems to make EBM better and easier).
Information about using evidence in decision-making
with patients has been scant.

D i s c o n n e c t b e t w e e n t h e 2 a p –
proaches is also evident in, and main-
tained by, the teaching provided to
clinicians and students, again often
ref lec ting the backgrounds of their
teachers. Opportunities to attend EBM
teaching abound with content largely

focused on forming questions and finding and criti-
cally appraising evidence.5 Learning how to apply and
integrate the evidence is usually absent, or mentioned
in passing without skill training.

Realizing the Connection Between EBM and SDM
A logical place to start is by incorporating SDM skill train-
ing into EBM training. This will help to address not only
the aforementioned deficits in EBM training but also the
lack of SDM training opportunities presently available.
Additionally, it may facilitate the uptake of SDM and,
more broadly, evidence translation. Recent calls for SDM
to be routinely incorporated into medical education pre-
sent an immediate opportunity to capitalize on closely
aligning the approaches.

Without shared decision making, EBM
can turn into evidence tyranny.

VIEWPOINT

Tammy C. Hoffmann,
PhD
Centre for Research in
Evidence-Based
Practice, Faculty of
Health Sciences and
Medicine, Bond
University, Queensland,
Australia; and
University of
Queensland, Brisbane,
Australia.

Victor M. Montori,
MD, MSc
Knowledge and
Evaluation Research
(KER) Unit, Mayo Clinic,
Rochester, Minnesota.

Chris Del Mar, MD,
FRACGP
Centre for Research in
Evidence-Based
Practice, Faculty of
Health Sciences and
Medicine, Bond
University, Queensland,
Australia.

Viewpoint page 1293

Corresponding
Author: Victor M.
Montori, MD, MSc,
Knowledge and
Evaluation Research
Unit, Mayo Clinic, 200
First St SW, Plummer
3-35, Rochester, MN
55905 (montori.victor
@mayo.edu).

Opinion

jama.com JAMA October 1, 2014 Volume 312, Number 13 1295

Downloaded From: https://jamanetwork.com/ by a Walden University User on 01/12/2020

Another place to start to bring EBM and SDM together is the
development and implementation of clinical practice guidelines.
Whereas most guidelines fail to consider patients’ preferences in
formulating their recommendations,6 some advise clinicians to
talk with patients about the options but provide no guidance
about how to do this and communicate the evidence in a way
patients will understand. Shared decision making may be strongly

r e c o m m e n d e d i n g u i d e l i n e s w h e n t h e o p t i o n s a r e c l o s e l y
matched in their advantages and disadvantages, when uncer-
tainty in the evidence impairs determination of a clearly superior
approach, or when the balance of benefits and risks depends on
patient action, such as adherence to medication, monitoring, and
diet in patients using warfarin.

Conclusions
Links between EBM and SDM have until recently been largely ab-
sent or at best implied. However, encouraging signs of interaction
are emerging. For example, there has been some integration of the
teaching of both,7 exploration about how guidelines can be adapted
to facilitate SDM,8,9 and research and resource tools that recog-
nize both approaches. Examples of the latter include research agenda
and priority setting occurring in partnership with patients and cli-
nicians to help provide relevant evidence for decision making; and
a new evidence criterion for the International Patient Decision Aids
Standards requiring citation of systematically assembled and up-
to-date bodies of evidence, with their trustworthiness appraised,10

thus aligning the development of SDM tools with contemporary re-
quirements for the formulation of evidence-based guidelines. Also,
independent flagship conferences focused on the practice of evi-
dence-based health care and on the science of shared decision mak-
ing are now convening joint meetings.

Medicine cannot, and should not, be practiced without up-to-
date evidence. Nor can medicine be practiced without knowing
and respecting the informed preferences of patients. Clinicians,
researchers, teachers, and patients need to be aware of and
ac tively facilitate the interdependent relationship of these
approaches. Evidence-based medicine needs SDM, and SDM needs
EBM. Patients need both.

ARTICLE INFORMATION

Conflict of Interest Disclosures: All authors have
completed and submitted the ICMJE Form for
Disclosure of Potential Conflicts of Interest.
Dr Montori reported serving on the board of the
International Society for Evidence-based
Healthcare; serving as Chair of the Seventh
International Shared Decision Making Conference in
2013; that he is a member of the Steering
Committee of the International Patient Decision
Aids Standards; and that he is a member of the
GRADE Working Group. The KER Unit (Dr Montori’s
research group) produces and tests evidence-based
shared decision making tools that are freely
available at http://shareddecisions.mayoclinic.org.
Dr Hoffmann reported that she is supported by a
National Health and Medical Research Council of
Australia (NHMRC)/Primary Health Care Research
Evaluation and Development Career Development
Fellowship (1033038), with funding provided by
the Australian Department of Health and Ageing.
Drs Hoffmann and Del Mar reported that they are
coeditors of a book on evidence-based practice, for
which they receive royalties.

Additional Information: Additional information
abut evidence-based medicine and shared decision
making is available online in Evidence-Based
Medicine: An Oral History at http://ebm
.jamanetwork.com.

REFERENCES

1. Straus SE, Jones G. What has evidence based
medicine done for us? BMJ. 2004;329(7473):987-
988.

2. Sackett DL, Rosenberg WM, Gray JA, Haynes RB,
Richardson WS. Evidence based medicine: what it is
and what it isn’t. BMJ. 1996;312(7023):71-72.

3. Greenhalgh T, Howick J, Maskrey N; Evidence
Based Medicine Renaissance Group. Evidence
based medicine: a movement in crisis? BMJ. 2014;
348:g3725.

4. Charles C, Gafni A, Whelan T. Shared
decision-making in the medical encounter: what
does it mean? (or it takes at least two to tango).
Soc Sci Med. 1997;44(5):681-692.

5. Meats E, Heneghan C, Crilly M, Glasziou P.
Evidence-based medicine teaching in UK medical
schools. Med Teach. 2009;31(4):332-337.

6. Montori VM, Brito JP, Murad MH. The optimal
practice of evidence-based medicine: incorporating
patient preferences in practice guidelines. JAMA.
2013;310(23):2503-2504.

7. Hoffmann TC, Bennett S, Tomsett C, Del Mar C.
Brief training of student clinicians in shared decision
making: a single-blind randomized controlled trial.
J Gen Intern Med. 2014;29(6):844-849.

8. Decision Aids. MAGIC website. http://www
.magicproject.org/decision-aids/. Accessed July 24,
2014.

9. van der Weijden T, Pieterse AH,
Koelewijn-van Loon MS, et al. How can clinical
practice guidelines be adapted to facilitate shared
decision making? a qualitative key-informant study.
BMJ Qual Saf. 2013;22(10):855-863.

10. Montori VM, LeBlanc A, Buchholz A, Stilwell
DL, Tsapas A. Basing information on
comprehensive, critically appraised, and up-to-date
syntheses of the scientific evidence: a quality
dimension of the International Patient Decision Aid
Standards. BMC Med Inform Decis Mak. 2013;13
(suppl 2):S5.

Figure. The Interdependence of Evidence-Based Medicine and Shared
Decision Making and the Need for Both as Part of Optimal Care

Evidence-based
medicine

Optimal patient care

Patient-centered
communication skills

Shared decision
making

Opinion Viewpoint

1296 JAMA October 1, 2014 Volume 312, Number 13 jama.com

Copyright 2014 American Medical Association. All rights reserved.
Downloaded From: https://jamanetwork.com/ by a Walden University User on 01/12/2020

Shared Decision-Making in Intensive Care Units
Executive Summary of the American College of Critical Care Medicine

and

American Thoracic Society Policy Statement

Shared decision-making is a central component of patient-centered
care in the intensive care unit (ICU) (1–4); however, there remains
confusion about what shared decision-making is and when
shared decision-making ought to be used. Further, failure to
employ appropriate decision-making techniques can lead to
significant problems. For example, if clinicians leave decisions
largely to the discretion of surrogates without providing adequate
support, surrogates may struggle to make patient-centered
decisions and may experience psychological distress (5).
Conversely, if clinicians make treatment decisions without
attempting to understand the patient’s values, goals, and
preferences, decisions will likely be predominantly based on the
clinicians’ values, rather than the patient’s, and patients or
surrogates may feel they have been unfairly excluded from
decision-making (1, 2). Finding the right balance is therefore
essential. To clarify these issues and provide guidance, the
American College of Critical Care Medicine (ACCM) and
American Thoracic Society (ATS) recently released a policy
statement that provides a definition of shared decision-making
in the ICU environment, clarification regarding the range of
appropriate models for decision-making in the ICU, a set of skills
to help clinicians create genuine partnerships in decision-making
with patients/surrogates, and ethical analysis supporting the
findings (6).

To develop a unified policy statement, the Ethics Committee of
the ACCM and the Ethics and Conflict of Interest Committee of the
ATS convened a writing group composed of members of these
committees. The writing group reviewed pertinent literature
published in a broad array of journals, including those with a focus
in medicine, surgery, critical care, pediatrics, and bioethics, and
discussed findings with the full ACCM and ATS ethics committees
throughout the writing process. Recommendations were generated
after review of empirical research and normative analyses published
in peer-reviewed journals. The policy statement was reviewed,
edited, and approved by consensus of the full Ethics Committee
of the ACCM and the full Ethics and Conflict of Interest Committee
of the ATS. The statement was subsequently reviewed and approved
by the ATS, ACCM, and Society of Critical Care Medicine leadership,
through the organizations’ standard review and approval processes.

ACCM and ATS endorse the following definition: Shared
decision-making is a collaborative process that allows patients, or
their surrogates, and clinicians to make health care decisions
together, taking into account the best scientific evidence
available, as well as the patient’s values, goals, and preferences.

Clinicians and patients/surrogates should use a shared
decision-making process to define overall goals of care (including
decisions regarding limiting or withdrawing life-prolonging
interventions) and when making major treatment decisions that
may be affected by personal values, goals, and preferences (7, 8).
Once clinicians and the patient/surrogate agree on general goals of
care, clinicians confront many routine decisions (e.g., choice of
vasoactive drips and rates, laboratory testing, fluid rate). It is

logistically impractical to involve patients/surrogates in each of
these decisions. Partnerships in decision-making require that the
overall goals of care and major preference-sensitive decisions be
made using a shared decision-making approach. The clinician then
has a fiduciary responsibility to use experience and evidence-based
practice when making day-to-day treatment decisions that are
consistent with the patient’s values, goals, and preferences.
Throughout the ICU stay, important, preference-sensitive choices
often arise. When they do, clinicians should employ shared
decision-making.

Clinicians should generally start with a default shared decision-
making approach that includes the following three main elements:
information exchange, deliberation, and making a treatment
decision. This model should be considered the default approach to
shared decision-making, and should be modified according to the
needs and preferences of the patient/surrogate. Using such a model,
the patient or surrogate shares information about the patient’s
values, goals, and preferences that are relevant to the decision at
hand. Clinicians share information about the relevant treatment
options and their risks and benefits, including the option of
palliative care without life-prolonging interventions. Clinicians and
the patient/surrogate then deliberate together to determine which
option is most appropriate for the patient, and together they agree
on a care plan. In such a model, the authority and burden of
decision-making is shared relatively equally (9). Although data
suggest that a preponderance of patients/surrogates prefer to share
responsibility for decision-making relatively equally with clinicians,
many patients/surrogates prefer to exercise greater authority in
decision-making, and many other patients/surrogates prefer to
defer even highly value-laden choices to clinicians (10–13).
Ethically justifiable models of decision-making include a broad
range to accommodate such differences in needs and preferences.

In some cases, the patient/surrogate may wish to exercise
significant authority in decision-making. In such cases, the clinician
should understand the patient’s values, goals, and preferences to a
sufficient degree to ensure the medical decisions are congruent
with these values. The clinician then determines and presents the
range of medically appropriate options, and the patient/surrogate
chooses from among these options. In such a model, the
patient/surrogate bears the majority of the responsibility and
burden of decision-making. In cases in which the patient/surrogate
demands interventions the clinician believes are potentially
inappropriate, clinicians should follow the recommendations
presented in the recently published multiorganization policy
statement on this topic (14).

In other cases, the patient/surrogate may prefer that clinicians
bear the primary burden in making even difficult, value-laden
choices. Research suggests that nearly half of surrogates of critically
ill patients prefer that physicians independently make some
types of treatment decisions (10–13). Further, data suggest that
approximately 5–20% of surrogates of ICU patients want clinicians
to make highly value-laden choices, including decisions to limit or

1334 American Journal of Respiratory and Critical Care Medicine Volume 193 Number 12 | June 15 2016

EDITORIALS

withdraw life-prolonging interventions (12, 13). In such cases,
using a clinician-directed decision-making model is ethically
justifiable (15–24).

Employing a clinician-directed decision-making model
requires great care. The clinician should ensure that the surrogate’s
preference for such a model is not based on inadequate
information, insufficient support from clinicians, or other
remediable causes. Further, when the surrogate prefers to defer a
specific decision to the clinician, the clinician should not assume
that all subsequent decisions are also deferred. The surrogate
should therefore understand what specific choice is at hand and
should be given as much (or as little) information as the surrogate
wishes. Under such a model, the surrogate cedes decision-making
authority to the clinician and does not need to explicitly agree
to (and thereby take responsibility for) the decision that is made.
The clinician should explain not only what decision the clinician is
making but also the rationale for the decision, and must then
explicitly give the surrogate the opportunity to disagree. If the
surrogate does not disagree, it is reasonable to implement the care
decision (19–24). Readers may review references 19–24 for detailed
descriptions and ethical analyses of clinician-directed decision-making.

The statement was intended for use in all ICU environments.
Patients and surrogate decision-makers have similar rights both
to participate in decision-making when appropriate and to rely more
heavily on providers when they wish to do so, regardless of the type
of ICU. Similarly, the statement is equally applicable in pediatric and
neonatal settings, where decision-making partnerships between
parents and the ICU team are equally important. As noted in the
statement, including children in some decisions can often be
appropriate as well. The statement is also intended to be applicable
internationally. Although patient and surrogate decision-making
preferences may differ globally, the default approach presented
and the recommendation to adjust the decision-making model to fit
the preferences of the patient or surrogate are universal. Both
ACCM and ATS are international organizations, and the literature
review included publications from many countries. The statement
focuses on the ICU environment because critically ill patients are
often, but not always, unable to participate in decision-making
themselves, and because many decisions in the ICU are value-
sensitive. The recommendations in the statement, however, could be
equally applicable in all patient care settings.

To optimize shared decision-making, clinicians should be
trained in specific communication skills. Core categories of
skills include establishing a trusting relationship with the
patient/surrogate; providing emotional support; assessing
patients’/surrogates’ understanding of the situation; explaining
the patient’s condition and prognosis; highlighting that there
are options to choose from; explaining principles of surrogate
decision-making; explaining treatment options; eliciting patient’s
values, goals, and preferences; deliberating together; and making
a decision. The full policy statement provides significant guidance
and examples in these areas (6).

Finally, ACCM and ATS recommend further research to
assess the use of various approaches to decision-making in the
ICU. The use of decision aids, communication skills training,
implementation of patient navigator or decision support counselor
programs, and other interventions should be subjected to
randomized controlled trials to assess efficacy. Considerations
regarding the cost and time burdens should be weighed against

anticipated benefits from such interventions when determining
which efforts to implement. n

Author disclosures are available with the text of this article at
www.atsjournals.org.

Acknowledgment: The views expressed in this article represent the official
position of the American College of Critical Care Medicine, the Society
of Critical Care Medicine, and the American Thoracic Society. These views
do not necessarily reflect the official policy or position of the U.S. Department
of the Navy, U.S. Department of Defense, U.S. National Institutes of Health,
U.S. Department of Veterans Affairs, U.S. Food and Drug Administration,
or U.S. Government.

Alexander A. Kon, M.D.
Naval Medical Center San Diego
San Diego, California

and

University of California San Diego
San Diego, California

Judy E. Davidson, D.N.P., R.N.
University of California Health System
San Diego, California

Wynne Morrison, M.D.
Children’s Hospital of Philadelphia
Philadelphia, Pennsylvania

Marion Danis, M.D.
National Institutes of Health
Bethesda, Maryland

Douglas B. White, M.D., M.A.S.
University of Pittsburgh School of Medicine
Pittsburgh, Pennsylvania

References

1. Carlet J, Thijs LG, Antonelli M, Cassell J, Cox P, Hill N, Hinds C, Pimentel
JM, Reinhart K, Thompson BT. Challenges in end-of-life care in the ICU:
statement of the 5th International Consensus Conference in Critical Care:
Brussels, Belgium, April 2003. Intensive Care Med 2004;30:770–784.

2. Thompson BT, Cox PN, Antonelli M, Carlet JM, Cassell J, Hill NS, Hinds
CJ, Pimentel JM, Reinhart K, Thijs LG; American Thoracic Society;
European Respiratory Society; European Society of Intensive Care
Medicine; Society of Critical Care Medicine; Sociètède Rèanimation
de Langue Française. Challenges in end-of-life care in the ICU:
statement of the 5th International Consensus Conference in Critical
Care: Brussels, Belgium, April 2003: executive summary. Crit Care
Med 2004;32:1781–1784.

3. Davidson JE, Powers K, Hedayat KM, Tieszen M, Kon AA, Shepard E,
Spuhler V, Todres ID, Levy M, Barr J, et al.; American College of
Critical Care Medicine Task Force 2004-2005, Society of Critical Care
Medicine. Clinical practice guidelines for support of the family in the
patient-centered intensive care unit: American College of Critical Care
Medicine Task Force 2004-2005. Crit Care Med 2007;35:605–622.

4. Lanken PN, Terry PB, Delisser HM, Fahy BF, Hansen-Flaschen J,
Heffner JE, Levy M, Mularski RA, Osborne ML, Prendergast TJ, et al.;
ATS End-of-Life Care Task Force. An official American Thoracic
Society clinical policy statement: palliative care for patients with
respiratory diseases and critical illnesses. Am J Respir Crit Care Med
2008;177:912–927.

5. Gries CJ, Engelberg RA, Kross EK, Zatzick D, Nielsen EL, Downey L,
Curtis JR. Predictors of symptoms of posttraumatic stress and
depression in family members after patient death in the ICU. Chest
2010;137:280–287.

Editorials 1335

EDITORIALS

http://www.atsjournals.org/doi/suppl/10.1164/rccm.201602-0269ED/suppl_file/disclosures

http://www.atsjournals.org

6. Kon AA, Davidson JE, Morrison W, Danis M, White DB; American
College of Critical Care Medicine; American Thoracic Society. Shared
decision making in ICUs: an American College of Critical Care
Medicine and American Thoracic Society policy statement. Crit Care
Med 2016;44:188–201.

7. O’Connor AM, Llewellyn-Thomas HA, Flood AB. Modifying
unwarranted variations in health care: shared decision making using
patient decision aids. Health Aff (Millwood) 2004;Suppl Variation:
VAR63-72.

8. O’Connor AM, Wennberg JE, Legare F, Llewellyn-Thomas HA, Moulton
BW, Sepucha KR, Sodano AG, King JS. Toward the ‘tipping point’:
decision aids and informed patient choice. Health Aff (Millwood) 2007;
26:716–725.

9. Charles C, Whelan T, Gafni A. What do we mean by partnership in
making decisions about treatment? BMJ 1999;319:780–782.

10. Heyland DK, Cook DJ, Rocker GM, Dodek PM, Kutsogiannis DJ, Peters
S, Tranmer JE, O’Callaghan CJ. Decision-making in the ICU:
perspectives of the substitute decision-maker. Intensive Care Med
2003;29:75–82.

11. Anderson WG, Arnold RM, Angus DC, Bryce CL. Passive decision-
making preference is associated with anxiety and depression in
relatives of patients in the intensive care unit. J Crit Care 2009;24:
249–254.

12. Johnson SK, Bautista CA, Hong SY, Weissfeld L, White DB. An
empirical study of surrogates’ preferred level of control over value-
laden life support decisions in intensive care units. Am J Respir Crit
Care Med 2011;183:915–921.

13. Madrigal VN, Carroll KW, Hexem KR, Faerber JA, Morrison WE,
Feudtner C. Parental decision-making preferences in the pediatric
intensive care unit. Crit Care Med 2012;40:2876–2882.

14. Bosslet GT, Pope TM, Rubenfeld GD, Lo B, Truog RD, Rushton CH,
Curtis JR, Ford DW, Osborne M, Misak C, et al.; American Thoracic
Society ad hoc Committee on Futile and Potentially Inappropriate

Treatment; American Thoracic Society; American Association for
Critical Care Nurses; American College of Chest Physicians;
European Society for Intensive Care Medicine; Society of Critical
Care. An Official ATS/AACN/ACCP/ESICM/SCCM policy statement:
responding to requests for potentially inappropriate treatments in
intensive care units. Am J Respir Crit Care Med 2015;191:
1318–1330.

15. Beauchamp TL, Childress JF. Principles of biomedical ethics, 5th ed.
New York: Oxford University Press; 2001.

16. Loewy EH, Loewy RS. Textbook of healthcare ethics, 2nd ed. Boston,
MA: Kluwer Academic Publishers; 2004.

17. Lo B. Resolving ethical dilemmas, 2nd ed. Philadelphia, PA: Lippincott
Williams & Wilkins; 2000.

18. Thompson DR, Kaufman D, editors. Critical care ethics: a practice
guide, 3rd ed. Mount Prospect, IL: Society of Critical Care Medicine;
2014.

19. Curtis JR, Burt RA. Point: the ethics of unilateral “do not resuscitate”
orders: the role of “informed assent”. Chest 2007;132:748–751.

20. Kon AA. Informed nondissent rather than informed assent. Chest 2008;
133:320–321.

21. Kon AA. The “window of opportunity:” helping parents make the most
difficult decision they will ever face using an informed non-dissent
model. Am J Bioeth 2009;9:55–56.

22. Kon AA. The shared decision-making continuum. JAMA 2010;304:
903–904.

23. Kon AA. Informed non-dissent: a better option than slow codes when
families cannot bear to say “let her die.” Am J Bioeth 2011;11:22–23.

24. Curtis JR. The use of informed assent in withholding cardiopulmonary
resuscitation in the ICU. Virtual Mentor 2012;14:545–550.

1336 American Journal of Respiratory and Critical Care Medicine Volume 193 Number 12 | June 15 2016

EDITORIALS

Provider perspectives on the utility of a colorectal
cancer screening decision aid for facilitating shared
decision

making

Paul C. Schroy III MD MPH,* Shamini Mylvaganam MPH� and Peter Davidson MD�
*Director of Clinical Research, Section of Gastroenterology, Boston Medical Center, Boston, MA, �Study Coordinator, Section of
Gastroenterology, Boston Medical Center, Boston, MA and �Clinical Director, Section of General Internal Medicine,

Boston

Medical Center, Boston, MA,

USA

Correspondence

Paul C. Schroy III, MD MPH

Boston Medical Center

85 E. Concord Street

Suite 7715

Boston

MA 02118

USA

E-mail: paul.schroy@bmc.org

Accepted for publication

8 August 2011

Keywords: decision aids, informed
decision making, shared decision

making

Abstract

Background Decision aids for colorectal cancer (CRC) screening

have been shown to enable patients to identify a preferred screening

option, but the extent to which such tools facilitate shared decision

making (SDM) from the perspective of the provider is less well

established.

Objective Our goal was to elicit provider feedback regarding the

impact of a CRC screening decision aid on SDM in the primary care

setting.

Methods Cross-sectional survey.

Participants Primary care providers participating in a

clinical trial

evaluating the impact of a novel CRC screening decision aid on

SDM and adherence.

Main outcomes Perceptions of the impact of the tool on decision-

making and implementation issues.

Results Twenty-nine of 42 (71%) eligible providers responded,

including 27 internists and two nurse practitioners. The majority

(>60%) felt that use of the tool complimented their usual approach,

increased patient knowledge, helped patients identify a preferred

screening option, improved the quality of decision making, saved

time and increased patients� desire to get screened. Respondents
were more neutral is their assessment of whether the tool improved

the overall quality of the patient visit or patient satisfaction. Fewer

than 50% felt that the tool would be easy to implement into their

practices or that it would be widely used by their colleagues.

Conclusion Decision aids for CRC screening can improve the

quality and efficiency of SDM from the provider perspective but

future use is likely to depend on the extent to which barriers to

implementation can be addressed.

doi: 10.1111/j.1369-7625.2011.007

� 2011 John Wiley & Sons Ltd 27
Health Expectations, 17, pp.27–

Introduction

Engaging patients to participate in the decision-

making process when confronted with prefer-

ence-sensitive choices related to cancer screening

or treatment is fundamental to the concept of

patient-centred care endorsed by the Institute of

Medicine, US Preventive Services Task Force

and the Centers for Disease Control and Pre-

vention.
1–3

Ideally, this process should occur

within the context of shared decision making

(SDM), whereby patients and their health-care

providers form a partnership to exchange

information, clarify values and negotiate a

mutually agreeable medical decision.
4,5

SDM,

however, has been difficult to implement into

routine clinical practice in part owing to lack of

time, resources, clinician expertise and suitabil-

ity for certain patients or clinical situations.
6,7

The use of patient-oriented decision aids outside

of the context of the provider–patient interac-

tion has been proposed as a potentially effective

strategy for circumventing several of these bar-

riers.
3,8

Decision aids are distinct from patient

education programmes in that they serve as

tools to enable patients to make an informed,

value-concordant choice about a particular

course of action based on an understanding of

potential benefits, risks, probabilities and sci-

entific uncertainty.
9–11

Besides facilitating

informed decision making (IDM), decision aids

also have the potential to facilitate SDM by

improving the quality and efficiency of the

patient–provider encounter and by empowering

users to participate in the decision-making

process.
11

Studies to date have demonstrated

that while decision aids enhance knowledge,

reduce decisional conflict, increase involvement

in the decision-making process and lead to

informed value-based decisions, their impact on

the quality of the decision, satisfaction with the

decision making process and health outcomes

remains unclear.
11

Besides enabling patients to make informed

choices, decision aids also have the potential to

facilitate SDM by improving the quality and

efficiency of the patient–provider encounter.

Relatively few studies have examined the utility

of decision aids for promoting effective SDM

from the perspective of the provider. Studies to

date have largely focused on provider perspec-

tives on the quality of the decision tools

themselves or issues related to implementation

into clinical practice.
11–15

The overall objective

of this study was to elicit provider feedback

regarding the extent to which the use of a novel

colorectal cancer (CRC) screening decision aid

facilitated SDM in the primary care setting

within the context of a randomized clinical

trial.

Methods

Brief overview of decision aid and randomized

clinical trial

Details of the decision aid, recruitment process,

study design and secondary outcome results

have been previously published.
16

The overall

objective of the trial was to evaluate the impact

of a novel computer-based decision aid on SDM

and patient adherence to CRC screening rec-

ommendations. The decision aid uses video-

taped narratives and state-of-the-art graphics in

digital video disc (DVD) format to convey key

information about CRC and the importance of

screening, compare each of five recommended

screening options using both attribute- and

option-based approaches, and elicit patient

preferences. A modified version of the tool also

incorporated the web-based �Your Disease Risk
(YDR)� CRC risk assessment tool (http://
www.yourdiseaserisk.wustl.edu). To assess its

impact on SDM and screening adherence,

average-risk, English-speaking patients 50–

75 years of age due for CRC screening were

randomized to one of the two intervention arms

(decision aid plus the YDR personalized risk

assessment tool with feedback or decision aid

alone) or a control arm, each of which involved

an interactive computer session just prior to a

scheduled visit with their primary care provider

at either the Boston Medical Center or the

South Boston Community Health Center. After

completing the computer session, patients met

with their providers to discuss screening and

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson

� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35

identify a preferred screening strategy. Although

providers were blinded to their patients� ran-
domization status, they received written notifi-

cation in the form of a hand-delivered flyer from

all study patients acknowledging that they were

participating in the �CRC decision aid study� to
ensure that screening was discussed. Outcomes

of interest were assessed using pre ⁄ post-tests,
electronic medical record and administrative

databases. The study to date has found that the

tool enables users to identify a preferred

screening option based on the relative values

they place on individual test features, increases

knowledge about CRC screening, increases sat-

isfaction with the decision-making process and

increases screening intentions compared to non-

users. The study also finds that screening

intentions and test ordering are negatively

influenced in situations where patient and pro-

vider preferences differ. The tool�s impact on
patient adherence awaits more complete follow-

up data, which should be available in early

2011.

Study design

We conducted a cross-sectional survey of

primary care providers participating in the ran-

domized clinical trial in January and February

of 2009. At the time of the survey, 725 eligible

patients had been randomized to one of the three

study arms. The surveys were distributed just

prior to monthly business meetings conducted

by the Sections of General Internal Medicine

and Women�s Health at Boston Medical Center
and Adult Medicine at the South Boston Com-

munity Health Center. Respondents were asked

to sign an attestation sheet if they completed the

survey to identify providers not in attendance.

For those who were not in attendance, the sur-

vey was distributed electronically as an email

attachment; respondents were asked to return

the survey via facsimile to preserve anonymity.

Two email reminders with attached surveys were

sent 2 weeks apart after the initial email to

optimize response. The study was deemed

exempt by the Institutional Review Boards at

both participating institutions.

Subjects

The survey sample included board-certified

primary care providers (general internists and

nurse practitioners) at Boston Medical Center

and the South Boston Community Health Center

who had referred patients to the randomized

clinical trial. Of the 50 providers who had referred

patients to the study since its commencement in

2005, 42 were still practicing at the participating

sites at the time of the survey. All had exposure to

at least one patient in an intervention arm and at

least one patient in the control arm; all but two of

the targeted providers had multiple patients in

each arm. None of the participants had formally

reviewed the content of the decision aid nor

received special training in SDM.

Practice settings

The Boston Medical Center is a private, non-profit

academic medical centre affiliated with the Boston

University School of Medicine, which serves a

mostly minority patient population (only 28%

White, non-Hispanic). The South Boston Com-

munity Health Center is a community health centre

affiliated with BMC, which serves a mostly White,

non-Hispanic, low-income patient population.

Survey instrument

The survey instrument included a cover letter, 23

closed-ended questions and two open-ended

questions. Much of the content was derived from

instruments used in previously published studies

by Holmes-Rovner et al. and Graham et al.
6,15

The cover letter briefly described the purpose of

the study, a statement that participation was

completely voluntary, the approximate amount

of time required to complete the survey, and a

statement that all responses are anonymous and

confidential. The closed-ended questions include

one item related to eligibility [confirmation of

participation in the clinical trial (yes ⁄ no)], two
items related to demographics (provider degree

and year of graduation), 12 items related to

perspectives on the impact of the tool on various

patient and provider components of SDM for

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson
� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35

CRC screening (see Table 1), and eight items

related to perspectives on implementation or

content modification (see Tables 2 and 3). The

framing of the questions inferred a comparison

between patients exposed to the decision aid and

those not exposed, i.e., standard care patients,

regardless of their involvement in the study. All

of the items related to SDM used a 5-point Likert

scale ranging from 1 (strongly disagree) to 5

(strongly agree). Six of the items related to

implementation or content modification also

used the same 5-point Likert scale, and two used

a single best answer format. The two open-ended

questions inquired about suggestions for

improving the decision aid and complaints. The

questionnaire took �10 min to complete.

Statistical analyses

Descriptive statistics were used to characterize

the study population and response data for all

closed-ended questions. Frequency data for the

5-point Likert scale items were collapsed into

three categories: �agreed ⁄ strongly agreed�, �neu-

tral� and �disagreed ⁄ strongly disagreed�. Mean
response scores ± standard deviations were

also calculated for the same data using Micro-

soft Excel functions. Responses to open-ended

questions were summarized according to themes.

Results

Study population

In total, 29 of the 42 (71%) possible providers,

including 27 physicians and two nurse practitio-

ners, responded to the survey and acknowledged

that they had referred patients to the randomized

clinical trial. Of the 29 respondents, 4 (14%) had

received their degrees between 2000 and 2009, 15

(52%) between 1990 and 1999, and 6 (28%)

before 1990; two declined to answer the question.

Perspectives on SDM

As shown in Table 2, the majority of providers

(>60%) agreed or strongly agreed that the

decision aid complemented their usual approach

Table 1 Provider perspectives on the utility of the decision aid for facilitating SDM

From my clinical perspective, the decision aid

Response category, n (%)

Mean item

score (SD)*

Strongly

agree ⁄ agree Neutral

Strongly

disagree ⁄
disagree

4. Complemented my usual approach to CRC screening 24 (86) 4 (14) 0 4.3 ± 0.7

5. Improved my usual approach to CRC screening 16 (59) 8 (30) 3 (11) 3.7 ± 1.0

6. Helped me tailor my counselling about CRC

screening to my patient�s needs
12 (44) 11 (41) 4 (15) 3.5 ± 1.0

7. Saved me time 18 (64) 6 (21) 4 (14) 3.8 ± 1.0

8. Improved the quality of patient visits 14 (52) 9 (

) 4 (15) 3.6 ± 1.0

9. Increased my patients� satisfaction with my care 10 (40) 13 (52) 2 (8) 3.4 ± 0.8
10. Is an appropriate use of my patient�s clinic time 27 (93) 1 (3) 1 (3) 4.1 ± 0.6
11. Increase patient knowledge about the different

CRC screening options

26 (90) 3 (10) 0 4.3 ± 0.6

12. Helped patients understand the benefits ⁄ risks
of the recommended screening options

24 (83) 5 (17) 0 4.1 ± 0.7

13. Helped patients in identifying preferred

screening option

21 (72) 7 (24) 1 (3) 4.0 ± 0.8

14. Improved the quality of the decision making 22 (79) 6 (21) 0 4.0 ± 0.7

15. Increased patients� desire to get screened 21 (75) 5 (18) 2 (7) 3.9 ± 0.9

CRC, colorectal cancer; SD, standard deviation; SDM, shared decision making.

*1 = strongly disagree; 5 = strongly agree.

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson
� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35
30

to CRC screening, was an appropriate use of

their patient�s clinic time, saved them time,
increased patient knowledge about the various

CRC screening options and their risks and

benefits, helped the patients identify a preferred

screening option, improved the quality of deci-

sion making, and increased their patients� desire
to get screened. Providers were more neutral in

their assessment of the decision aid�s utility for
improving their usual approach to CRC

screening, helping them tailor their counselling

style to their patients� needs, improving the
quality of patient visits, and increasing patient

satisfaction with their care. Relatively few pro-

viders disagreed or strongly disagreed with any

of these measures.

Perspectives on clinical use and content

modification

There was less consensus when asked about

implementation of the tool into routine clinical

practice. As shown in Table 2, <50% of

respondents agreed or strongly agreed that the

decision aid would be easy to use in their prac-

tice outside of a research setting or that it would

be used by most of their colleagues. A slim

majority (58%) also believed that implementa-

tion would require reorganization of their

practice. Respondents mostly agreed or were

neutral in their assessment of whether the deci-

sion aid should be disseminated as an Internet-

or DVD-based tool. When asked to identify a

preferred time for having their patients review

the tool (Table 3), 72% chose prior to initiating

the CRC screening discussion, 21% chose after

initiating the screening discussion, and 7% chose

both. Among the 21 providers who chose the

pre-visit approach, 13 preferred that the tool be

used in the office just prior to the pre-arranged

visit, five preferred at home use and three pre-

ferred both; among the six providers who chose

the post-visit approach, five preferred in-office

use and one preferred at home use.

There was also a lack of consensus when

asked about content modification. Whereas 50%

of respondents agreed or strongly agreed that

the decision aid should include a discussion of

costs,

% disagreed or strongly disagreed

Table 2 Provider perspectives on decision aid implementation

The decision aid

Response category, n (%)
Mean item
score (SD)*
Strongly
agree ⁄ agree Neutral
Strongly
disagree ⁄
disagree

16. Would be easy to use in my practice

outside of a research stetting

12 (48) 9 (36) 4 (16) 3.4 ± 1.0

17. Use would require reorganization of my

practice for routine clinical use

14 (58) 6 (25) 4 (17) 3.6 ± 1.1

18. Is likely to be used by most of my colleagues 11 (41) 12 (44) 4 (15) 3.4 ± 0.9

19. Should include a discussion of costs 13 (50) 5 (19) 8 (31) 3.5 ± 1.2

20. Should be disseminated as an Internet-based tool 17 (63) 8 (30) 2 (7) 3.7 ± 0.9

21. Should be disseminated as a DVD-based tool 15 (56) 8 (30) 4 (15) 3.6 ± 0.9

DVD, digital video disc; SD, standard deviation.

*1 = strongly disagree; 5 = strongly agree.

Table 3 Preferences for clinical use and content modification

Item N (%)

22. When would you want your patient to

view the decision aid:

Before initiating CRC screening discussion

(pre-visit)

21 (72)

After initiating CRC discussion (post-visit) 6 (21)

Both 2 (7)

23. Would you prefer the decision aid to

contain information about:

All of the recommended screening options 15 (52)

A more restricted list of options 12 (41)

No opinion 2 (7)

CRC, colorectal cancer.

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson
� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35
31

(Table 2). Similarly, whereas 52% of providers

preferred that the decision aid include a discus-

sion of all of the recommended screening

options, 41% preferred a more restricted list of

options and 7% had no opinion on the issue

(Table 3).

Only seven providers made suggestions for

improving the current decision aid. These

included creating non-English versions of the

tool (n = 2), clearly distinguishing colonoscopy

as the best screening option (n = 2), enabling

patients to print out their preferred screening

option (n = 2), and taking into consideration

that patients may not have access to the Internet

at home if the decision aid was to be dissemi-

nated as a web-based tool (n = 1). There were

no complaints.

Discussion

Decision aids are evidence-based tools that

enable patients to make informed, value-con-

cordant choices, but the extent to which such

tools facilitate SDM from the perspective of the

provider is less well established. In an effort to

gain new insight into the issue, we conducted a

survey of primary care providers participating in

a clinical trial evaluating the impact of a novel,

DVD-formatted decision aid on SDM and

adherence to CRC screening. Our study finds

that a majority of providers perceived that the

tool was a useful, time-saving adjunct to their

usual approach to counselling about CRC

screening and increased the overall quality of

decision making. Moreover, providers also felt

that review of the tool just prior to a scheduled

office visit was an appropriate use of patient�s
time as it enabled the patient to make an

informed choice among the different screening

options. Together, these findings suggest that

much of the tool�s perceived utility was related
to its ability to better prepare patients for the

screening discussion outside of the clinical

encounter and, in so doing, increased both the

efficiency and quality of the interaction.

Few studies have explored provider perspec-

tives on the utility of decision aids for improving

SDM. A trial by Green et al. evaluating the

effectiveness of genetic counselling vs. counsel-

ling preceded by use of a computer-based deci-

sion aid for breast cancer susceptibility found

that although there were no significant differ-

ences in perceived effectiveness, use of the tool

saved time and shifted the focus away from basic

education towards a discussion of personal risk

and decision making.
17

A second study by Sim-

inoff et al. found that a decision aid for breast

cancer adjuvant therapy facilitated a more

interactive, informed discussion and helped

physicians understand patient preferences.
13

Similarly, Brackett et al. also found that pre-

visit use of decision aids for prostate and CRC

screening was associated with greater physician

satisfaction, as it saved time during the visit and

changed the conversation from one of the

informational exchanges to one of the values

and preferences.
18

A fourth study by Graham

et al. explored provider perceptions of three

decision aids prior to their actual use.
15

Although responses were based on perceptions

alone and not on clinical experience, their find-

ings were similar to our own. A majority agreed

or strongly agreed that the decision aids could

meet patients� informational needs about risks
and benefits and enable patients to make

informed decisions. Similarly, although many

felt that the decision aids were likely to com-

plement their usual approach, responses were

more neutral when asked about the overall

impact of the tools on the quality of the patient

encounter, patient satisfaction and issues related

to implementation. The most striking difference,

however, was that relatively few of the respon-

dents in the study by Graham et al. felt that use

of the tool saved time, which could be a reflec-

tion of either the complexity of the decisions

under consideration and ⁄ or the lack of explicit
instructions regarding how the tools were to be

used with respect to the timing of the interven-

tion and ⁄ or need for provider involvement.
Our findings also corroborate a more exten-

sive body of literature on barriers to the imple-

mentation of decision aids into clinical

practice.
14

Even though our study design cir-

cumvented many of the barriers related to

workflow, accessibility and costs, only 48% of

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson
� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35

providers felt that actual implementation of the

decision aids into their practices outside of the

context of a clinical trial would be easy. Based

on their feedback, however, most preferred that

the tool be used prior to initiating the screening

discussion rather than after initiation of the

discussion. Moreover, regardless of the timing, a

majority preferred that the tool be used in the

office rather than at home. Although it is quite

possible that their preferences reflected their

personal experiences with our study protocol,

Brackett et al. also found that pre-visit use was

preferred over post-visit use.
18

One of the most commonly cited barriers to

implementation of SDM is the time requirement.

Although studies to date have provided con-

flicting data regarding the impact of decision

aids on consultation time for other condi-

tions,
17–22

we postulated that by educating

patients about the risks and benefits of the dif-

ferent screening options and facilitating IDM

prior to the provider–patient encounter, our

decision aid would have the potential of

improving the efficiency of SDM and thus save

time, as noted by Green et al. and Brackett

et al.
17,18

We found that although a majority of

providers agreed or strongly agreed that pre-visit

use of the tool saved time, 21% were neutral on

the issue and 14% disagreed or strongly dis-

agreed. It is conceivable that this diversity of

opinion might be a reflection of the extent to

which provider and patient preferences agreed or

disagreed. In instances where there was concor-

dance between preferences, as was often the case

that since colonoscopy was preferred by major-

ity of both patients and providers,
16

one would

expect that the time required for deliberation

and negotiation would be substantially shorter

than in situations where there was discordance.

Alternatively, these differences might reflect

differences in case mix with respect to patient

factors, such as literacy level or desired level of

participation in the decision-making process.

A secondary objective of our study was to

elicit provider feedback regarding content and

format preferences to gain insight into potential

modifications that might enhance future uptake.

Because of an ongoing debate in the CRC

screening literature,
23–27

we focused on content

issues related to cost information and number of

screening options to include in the decision aid.

Both questions elicited a divergence of opinions.

Whereas nearly 50% of respondents felt that

cost information should be included, the

remainder was either neutral or opposed to its

inclusion. Similarly, when asked about the

number of screening options to include, �50%
preferred the full menu of options and �40%
preferred a more limited menu. This diversity of

opinion highlights some of the key challenges in

designing tools with broad dissemination

potential. In the light of recent evidence sug-

gesting that the number of screening options

may influence test choice but not interest in

screening and that the importance of out-of-

pocket costs declines as the number of screening

options discussed increases,
26

one approach

would be to develop one tool that presents the

full menu of screening options without cost

information and a second that includes a more

limited set of options with cost information. A

more appealing approach would be to develop a

more comprehensive tool that includes both the

full menu of options and cost information in a

format that permits navigation so that patients

could tailor their use to fit their own informa-

tional needs and ⁄ or recommendations of their
provider. Internet-based tools are ideally suited

for this purpose but, as noted by several par-

ticipants in our study, access remains a potential

barrier for a sizeable, albeit declining, propor-

tion of the target population. Providers in our

study felt that both Internet- and DVD-for-

matted tools were viable options for dissemina-

tion, even though the DVD-formatted tool

offers less navigation potential.

Our study has several notable limitations.

First, the survey was conducted among primary

care providers at only two institutions, and

hence, the findings may not be generalizable to

providers in other health care settings. It is

noteworthy, however, that the study was con-

ducted among a diverse patient population with

respect to both race ⁄ ethnicity and educational
status.

16
Second, as participating providers

never formally reviewed the decision aid, we

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson
� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35
33

were unable to assess their opinions with respect

to actual content or format. Third, the content

of our survey instrument did not allow us to

tease out the extent to which use of the decision

aid impacted on individual steps of the SDM

process.
4,5

Even though satisfaction with the

decision-making process was universally high

among patients participating in the clinical

trial,
16

especially those in the intervention

groups, only a relative minority of providers felt

that use of the tool helped them tailor their

counselling about CRC screening to their

patients� needs or increased patient satisfaction
with their care. Fourth, the anonymous nature

of our survey precluded any attempt to correlate

response data with exposure rates. It is con-

ceivable that the perceptions of providers

exposed to multiple patients in the intervention

arms might differ from those exposed to only a

few patients. Lastly, we cannot rule out the

possibility of social response bias, whereby

respondents may have felt compelled to offer

more positive responses than they actually

believed.

In conclusion, our study finds that a majority

of providers perceived that pre-clinic use of our

decision aid for CRC screening was a useful,

time-saving adjunct to their usual approach to

counselling about CRC screening and increased

the overall quality of decision making. Never-

theless, many of the providers felt that imple-

mentation of the decision aid into their practices

outside of the context of a clinical trial would be

challenging, thus highlighting the need for cost-

effective strategies for addressing provider,

practice and organizational level barriers to

routine use. We speculate that Internet-based

tools with enhanced navigation functionality

have the greatest dissemination potential, as

they offer a feasible, low-cost solution to many

of the structural barriers to implementation, as

well as a way to reconcile the diversity of opin-

ion related to content.

Acknowledgement

None.

Conflicts of interest

The authors have no conflict of interests.

Funding

This study was supported by grant RO1

HS013912 from the Agency for Healthcare

Research and Quality.

References

1 Institute of Medicine. Crossing the Quality Chasm: A

New Health System for the 21st Century. Washington,

DC: National Academy Press, 2001.

2 Sheridan SL, Harris RP, Woolf SH. Shared decision

making about screening and chemoprevention. A

suggested approach from the U.S. Preventive Services

Task Force. American Journal of Preventive Medicine,

2004; 26: 56–66.

3 Briss P, Rimer B, Reilley B et al. Promoting informed

decisions about cancer screening in communities and

healthcare systems. American Journal of Preventive

Medicine, 2004; 26: 67–80.

4 Charles C, Gafni A, Whelan T. Shared decision-

making in the medical encounter: what does it mean?

(or it takes at least two to tango). Social Science and

Medicine, 1997; 44: 681–692.

5 Charles C, Gafni A, Whelan T. Decision-making in

the physician-patient encounter: revisiting the shared

treatment decision-making model. Social Science and

Medicine, 1999; 49: 651–661.

6 Holmes-Rovner M, Valade D, Orlowski C, Draus C,

Nabozny-Valerio B, Keiser S. Implementing shared

decision-making in routine practice: barriers and

opportunities. Health Expectations, 2000; 3: 182–191.

7 Legare F, Ratte S, Gravel K, Graham ID. Barriers

and facilitators to implementing shared decision-

making in clinical practice: update of a systematic

review of health professionals� perceptions. Patient
Education and Counseling, 2008; 73: 526–535.

8 Rimer BK, Briss PA, Zeller PK, Chan EC, Woolf SH.

Informed decision making: what is its role in cancer

screening? Cancer, 2004; 101: 1214–1228.

9 International Patient Decision Aid Standards

(IPDAS) Collaboration. IPDAS Collaboration

Background Document, 2005. Available at: http://

ipdas.ohri.ca/IPDAS_Background , accessed 26

October 2010.

10 Elwyn G, O�Connor A, Stacey D et al. Developing a
quality criteria framework for patient decision aids:

online international Delphi consensus process. BMJ,

2006; 333: 417.

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson
� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35

11 O�Connor AM, Bennett CL, Stacey D et al. Decision
aids for people facing health treatment or screening

decisions. Cochrane Database of Systematic Reviews,

2009; 3: CD001431.

12 O�Connor AM, Llewellyn-Thomas HA, Sawka C,
Pinfold SP, To T, Harrison DE. Physicians� opinions
about decision aids for patients considering systemic

adjuvant therapy for axillary-node negative breast

cancer. Patient Education and Counseling, 1997; 30:

143–153.

13 Siminoff LA, Gordon NH, Silverman P, Budd T,

Ravdin PM. A decision aid to assist in adjuvant

therapy choices for breast cancer. Psycho-Oncology,

2006; 15: 1001–1013.

14 O�Donnell S, Cranney A, Jacobsen MJ, Graham ID,
O�Connor AM, Tugwell P. Understanding and over-
coming the barriers of implementing patient decision

aids in clinical practice. Journal of Evaluation in

Clinical Practice, 2006; 12: 174–181.

15 Graham ID, Logan J, Bennett CL et al. Physicians�
intentions and use of three patient decision aids.

BMC Medical Informatics and Decision Making,

2007; 7: 20.

16 Schroy PC III, Emmons K, Peters E et al. The impact

of a novel computer-based decision aid on shared

decision making for colorectal cancer screening: a

randomized trial. Medical Decision Making, 2011; 31:

93–107.

17 Green MJ, Peterson SK, Baker MW et al. Use of an

educational computer program before genetic coun-

seling for breast cancer susceptibility: effects on

duration and content of counseling sessions. Genetics

in Medicine, 2005; 7: 221–229.

18 Brackett C, Kearing S, Cochran N, Tosteson AN,

Blair Brooks W. Strategies for distributing cancer

screening decision aids in primary care. Patient

Education and Counseling, 2010; 78: 166–168.

19 Whelan T, Sawka C, Levine M et al. Helping patients

make informed choices: a randomized trial of a

decision aid for adjuvant chemotherapy in lymph

node-negative breast cancer. Journal of the National

Cancer Institute, 2003; 95: 581–587.

20 Bekker HL, Hewison J, Thornton JG. Applying

decision analysis to facilitate informed decision

making about prenatal diagnosis for Down syn-

drome: a randomised controlled trial. Prenatal Diag-

nosis, 2004; 24: 265–275.

21 Butow P, Devine R, Boyer M, Pendlebury S, Jackson

M, Tattersall MH. Cancer consultation preparation

package: changing patients but not physicians is not

enough. Journal of Clinical Oncology, 2004; 22: 4401–

4409.

22 Nannenga MR, Montori VM, Weymiller AJ et al. A

treatment decision aid may increase patient trust in

the diabetes specialist. The Statin Choice randomized

trial. Health Expectations, 2009; 12: 38–44.

23 Leard LE, Savides TJ, Ganiats TG. Patient prefer-

ences for colorectal cancer screening. Journal of

Family Practice, 1997; 45: 211–218.

24 Pignone M, Bucholtz D, Harris R. Patient preferences

for colon cancer screening. Journal of General Internal

Medicine, 1999; 14: 432–437.

25 Lafata JE, Divine G, Moon C, Williams LK. Patient-

physician colorectal cancer screening discussions and

screening use. American Journal of Preventive

Medicine, 2006; 31: 202–209.

26 Griffith JM, Lewis CL, Brenner AR, Pignone MP.

The effect of offering different numbers of colorectal

cancer screening test options in a decision aid: a pilot

randomized trial. BMC Medical Informatics and

Decision-Making, 2008; 8: 4.

27 Jones RM, Vernon SW, Woolf S. Is discussion of

colorectal cancer screening options associated

with heightened patient confusion? Cancer

Epidemiology, Biomarkers and Prevention, 2010; 19:

2821–2825.

Colorectal cancer screening decision aid, P C Schroy, S Mylvaganam and P Davidson
� 2011 John Wiley & Sons Ltd
Health Expectations, 17, pp.27–35
35

Copyright of Health Expectations is the property of Wiley-Blackwell and its content may not
be copied or emailed to multiple sites or posted to a listserv without the copyright holder’s
express written permission. However, users may print, download, or email articles for
individual use.

Measuring Return on Investment for
Professional Development Activities
Implications for Practice

Cathleen Opperman, DNP, RN, NEA-BC, CPN ƒ Debra Liebig, MLA, BSN, RN-BC ƒ
Judith Bowling, MSN, MHA, RN-BC ƒ Carol Susan Johnson, PhD, RN, NE-BC ƒ
Mary Harper, PhD, RN-BC

What is the return on investment (ROI) for the time and

resources spent for professional development activities? This

is Part 2 of a two-part series to report findings and demonstrate

how financial analysis of educational activities can drive

decision-making. The resources consumed for professional

development activities need to be identified and quantified

to be able to determine the worth of such activities. This

article defines terms and formulas for financial analysis

for nursing professional development practitioners to use in

analysis of their own programs. Three fictitious examples

of common nursing professional development learning

activities are provided with financial analysis. This article

presents the ‘‘how to’’ for the busy practitioner.

A
s nursing professional development (NPD) prac-
titioners, we are challenged by the question
‘‘What is the return on investment (ROI) for

professional development activities?’’ As described in Part
1 of this series, NPD practitioners are often the first to be
called when a problem exists and the first to have funding
restricted when budgets are tight. In Part 1, we discussed
the Kirkpatrick, Phillips, and Paramoure program evalua-
tion models, followed by a summary of the literature
reporting on ROI for professional development activities.

The synthesis of the studies on educational interven-
tions providing a calculation of financial aspects shows
no consistent method to describe financial and clinical im-
pact of professional development activities (Opperman,
Liebig, Bowling, Johnson, & Harper, 2016). The trend of
reporting outcomes associated with learning activities
has given rise to the next level of expectation: demonstra-
tion of financial impact of educational interventions.

This article defines the concepts of an economic assess-
ment including simple cost analysis, benefitYcost ratios,
cost-effectiveness analysis (CEA), and ROI, as well as pro-
viding formulas to calculate each. Three fictional examples
of various-sized educational programs are used to demon-
strate how to make these calculations and use them for
decision-making.

COST ANALYSIS, BENEFIT–COST RATIOS,
AND COST-EFFECTIVENESS ANALYSIS
Prior to discussing ROI, an understanding of the concepts
of cost analysis, benefitYcost ratios, and CEA is essential
when calculating the actual financial impact of professional
development activities.

From a financial perspective, cost analysis is the initial
consideration when developing an educational program.
Cost analysis simply determines the least expensive option.
The formula for cost analysis is to add all the costs for the
program and divide it by the number of participants to ob-
tain the cost per participant. See Figure 1 for the formulas
for cost analysis. When considering multiple learning mo-
dalities, this simple cost per participant can be compared.

Although cost analysis provides information on the effi-
ciency or least expensive modality, it does not consider
program outcomes. The benefitYcost analysis compares
program benefits to program costs as a ratio using dollars.
The first step is clearly identifying desired program out-
comes that can be observed and measured.

The next step is calculating all program costs. The
benefitYcost ratio formula uses all benefits (i.e., increased
productivity, quality, safety improvements, reduced turn-
over, increased patient volumes) and all costs (i.e.,
program development time, faculty costs, training supplies,

Cathleen Opperman, DNP, RN, NEA-BC, CPN, is Nurse Specialist, Profes-
sional Development, Nationwide Children’s Hospital, Columbus, Ohio.

Debra Liebig, MLA, BSN, RN-BC, is Director, Nursing Retention, Truman
Medical Center, Kansas City, Missouri.

Judith Bowling, MSN, MHA, RN-BC, is Clinical Learning Educator,
Baptist Health South Florida, Miami, Florida.

Carol Susan Johnson, PhD, RN, NE-BC, is NCC MagnetA Appraiser and
member, Commission on Accreditation, Fort Wayne, Indiana.

Mary Harper, PhD, RN-BC, is Director, Nursing Professional Develop-
ment, Association for Nursing Professional Development.

The authors have disclosed that they have no significant relationship with,
or financial interest in, any commercial companies pertaining to this article.

ADDRESS FOR CORRESPONDENCE: Cathleen Opperman, Nation-
wide Children’s Hospital, 255 East Main St., Columbus, OH 43205
(oppermancs@gmail.com).

DOI: 10.1097/NND.0000000000000274

176 www.jnpdonline.com July/August 2016

equipment costs, facility fees, salary cost for employee
attendance/replacement cost) to determine the financial re-
turn from the program (Warren, 2013). See Figure 1 for
the formulas for cost analysis, benefit-cost ratio and re-
turn on investment.

CEA goes one step further in economic assessment, be-
cause it compares two or more different educational
interventions and their outcomes. The NPD practitioner
may have an option of a self-study that takes the learner
an average of 2 hours to complete or a 90-minute live work-
shop with the same content. Both modalities are intended
to accomplish the same outcome. The costs must be mon-
etary values and calculated as cost analysis for each
possible intervention. The outcomes, however, do not
need to be monetary values; consider them the benefits
gained from the educational intervention. For example,
nonmonetary outcomes might be ‘‘increased patient en-
gagement’’ or ‘‘fewer staff reporting incivility.’’ The
combination of the cost per participant (cost analysis) and
the benefits, whether monetary (calculated as a benefitY
cost ratio) or nonmonetary, are used to determine the CEA.

BenefitYcost ratio and CEA collectively impact decisions
about program changes and resources. Although cost anal-
ysis alone may demonstrate efficiency through lower costs,
the benefitYcost analysis may demonstrate that the same
program is not as effective in achieving desired outcomes.
Clearly, comprehensive program evaluation requires
consideration of both componentsVefficiency and effec-
tiveness (Kettner, Moroney, & Martin, 2013).

ROI CALCULATION
In the economic assessment of a program, calculation
of ROI provides further data for administrative decision-
making. Calculating ROI (a) provides information for

justification of programs for budgetary planning, (b)
contributes to clinical decision-making and resource al-
location, and (c) demonstrates the value of education.

Because of the complexity of determining ROI for pro-
grams, pragmatically, it is used in only about 5%Y10% of
program planning processes for priority decision-making
like regulatory, higher-risk, or more expensive programs
(DeSilets, 2010.)

Steps in calculating ROI:
1. Identify program desired outcomes.
2. Describe educational interventions proposed to meet

these outcomes.
3. Plan the logistics of the educational intervention with

sufficient detail to identify expenses.
4. Calculate program costs (planning time, supplies, setup

time, faculty and staff time, etc.).
5. Calculate potential savings (cost of turnover, pressure

ulcer, litigation, inefficiency of program changes).
6. Compare costs to savings (efficiency).
7. Determine specific outcomes using observable and

measurable terms (effectiveness).
In order to calculate the benefit of educational inter-

ventions, an outcome must be quantified. For example,
changes in orientation should lead to greater new em-
ployee competence, confidence, and satisfaction, therefore
reducing turnover. Another example is that an education
activity on the catheter-associated urinary tract infections
(CAUTI) bundle should lead to reduction of CAUTIs.
When calculating the benefit of the educational interven-
tion for either of these examples, the cost (of a new RN
leaving or the average cost of a CAUTI) should be used
to counter the cost of the program. For examples of
published average costs per case of poor outcomes, see
Table 1. The formula for calculating the ROI is found in
Figure 1.

EXAMPLES OF FINANCIAL IMPACT ANALYSIS
Consider these examples of fictitious educational activities
and how financial impact can be calculated through cost
analysis, benefitYcost ratios, CEA, and ROI analysis.

Example 1: One-hour self-study compared to live class.
Situation. The organization considers requiring a

1-hour Web-based self-study module for 650 learners
on changes in the procedure for pressure ulcer preven-
tion bundle.

Background. The hospital incidence of hospital-
acquired pressure ulcers (HAPU) is 25%, which is above
the national average of 17% (Roe & Williams, 2014). The
Centers for Medicare and Medicaid Services no longer re-
imburses facilities for patients with newly acquired Stage 3
and Stage 4 pressure ulcers.

According to the Agency for Healthcare Research and
Quality (2014b), a full-thickness pressure ulcer costs an
average of $17,286 per incident to treat. This does not

FIGURE 1 Formulas for cost analysis, benefitYcost ratio, and return on
investment.

Journal for Nurses in Professional Development www.jnpdonline.com 177

include the additional emotional and physical burden
for the patient.

According to the American Faculty Association
(2012), the time needed to develop educational programs
is 4 hours of preparation for each hour of class presented.
This varies widely from Kapp and Defelice (2009) that
estimates 40 (self-instructional print), 43 (stand-up class-
room training), and 49 (instructor-led, Web-based
training) hours per hour of training are needed for devel-
opment. For purposes of these fictitious scenarios, it is
assumed that the NPD practitioner is well informed of
changes in pressure ulcer care and is an experienced
NPD practitioner; consequently, the number of develop-
ment hours is less.

Assessment. The program costs of a Web-based self-
study module for supplies, salaries, and equipment are
calculated as follows. The computers and software are
in place; thus, no further initial expense for equipment
is needed.

Cost analysis. The cost for a Web-based module is
$32.78 per participant, and the cost of the live classes is

35.03. The Web-based class costs $2.25 less per partici-
pant. Additional considerations for cost analysis include
the cost of educating new hires. If the organization hires
an additional 50 nurses over the course of a year, the only
expense for the Web-based course is the hourly salary of
the new nurses. No additional program costs are incurred.
If presented in a live format, in addition to the newly hired
nurses’ salary, additional costs would include the NPD
practitioner salary for each class and the additional admin-
istrative support salary.

BenefitYcost ratio. To calculate benefitYcost ratio, bene-
fits are divided by total costs. Using $17,286 as the per case
cost to treat a full thickness pressure ulcer, prevention of
two pressure ulcers results in cost savings of $34,572. This
results in a positive benefitYcost ratio for both Web-based
and live class formats.

BenefitYcost ratio:
Using Web-based self-study module:

$34,572 = 1.62 BCR
$21,310

Using live class format:
$34,572 = 1.52 BCR

$22,775

(G1 = negative impact, 9 = positive impact)
Cost-effectiveness analysis. The difference in cost be-

tween the Web-based modules at $21,310 and the live
classes at $22,775 is $1,465. The savings for the Web-
based format is only positive if the Web-based course and
the live classes are comparable in outcomes. CEA requires
an evaluation of effectiveness of each modality in achiev-
ing the same outcomes. In this scenario, the Web-based
course was determined to be equal in effectiveness re-
sulting in similar outcomes to the live presentation. As a
result, the Web-based course is more cost-effective.

ROI. To evaluate the ROI in this example, the cost of
the pressure ulcer treatment must be compared to the
cost of the education. As previously stated, if this educa-
tional intervention prevents two pressure ulcers, $34,572
is saved. The ROI for the Web-based self-study is 62.2%,
and the live class format is 51.7%.

Using Web-based self-study module:
$34,572j $21,310 � 100 = 62.2% ROI

$21,3107
Using live class format:

$34,572j $22,775 � 100 = 51.7% ROI
$22,775

Recommendation. In both the live and Web-based
courses, the ROI is positive and easily justifies the educa-
tion. The Web-based course, however, shows a higher
ROI. The cost analysis, benefitYcost ratio, CEA, and ROI
all demonstrate a more positive financial impact with the
Web-based course. As a result, the NPD practitioner rec-
ommends development of a Web-based educational
program on prevention of pressure ulcers.

One-Hour Self-Study Expenses

Item Hours X hourly pay Total

NPD practitioner
salary (development)

40 hours � $35/hour $1400

NPD practitioner
salary (coordination)

6 hours � $35/hour $210

Admin support salary 4 hours � $20/hour $80

IT support salary 4 hours � $30/hour $120

Participants salaries 1 hour � 650 participants
� $30/hour

$19,500

Total cost $21,310

Cost per participant
($21,310 / 650)

$32.78

Additional Costs for a Live Class

Item Hours X hourly pay Total
NPD practitioner
salary (classroom time)

35 classes � $35/hour $1,225

Admin support salary
(record-keeping)

12 hours � $20/hour $240

Total additional costs $1,465

Live program total cost
(from above + additional)

$22,775

Cost per participant
($22,775 / 650)

$35.03

178 www.jnpdonline.com July/August 2016

TABLE 1 Known Costs of Outcomes
Outcome Reported average cost Sources
Active surveillance screening for
MRSA

Universal surveillance screening cost-effectiveness
ratio of $14,955 per MRSA

Kang, Mandsager, Biddle, and
Weber (2012)

Adverse drug events Estimated extra cost per case $5,000 Agency for Healthcare
Research

and Quality (2014a)

Asthma/COPD treatment $1,681Y$8,533 annual mean expenditure per person Agency for Healthcare
Research and Quality (2014b)

Breast milk Biological mother’s milk ranged $0.051 to $7.93,
depending on the volume pumped daily
Donor human milk cost was $14.84
Commercial formula was $3.18

Jegler et al. (2013)

Cancer treatment $5,631Y$21,573 annual mean expenditure per person Agency for Healthcare
Research and Quality (2014b)

Care for child with autism spectrum
disorder

Intensive behavior intervention $4,000/month
Gluten-free diet, $150 every 2 weeks

Fletcher, Markoulakis, and Bryden
(2012)

Catheter-associated urinary tract
infections (CAUTI)

Additional $1,000 per admission Agency for Healthcare Research
and Quality (2014a)

Catheter-related bloodstream
Infections (CRBSI)

CRBSI, $11,971Y$56,167
Adult ICU, $33,000Y$44,000
Surgical ICU, $54Y$75,000
Pediatric ICU, $48,379
Multicenter study, $20,647
General wards, $20,647

Hollenbeak (2011)

Central line-associated bloodstream
infections (CLABSI)

Additional $17,000 per admission Agency for Healthcare Research
and Quality (2014a)

Employee musculoskeletal injuries $28,866 per strain
$33.528 per sprain

OSHA Safety Pays Program
Estimator (2016)
ANA’s Handle with Care Program
(2016)

Employee needle sticks $22,716 per incident OSHA Safety Pays Program
Estimator (2016)

Falls Estimated extra cost $7,234 per case
Mean cost of hospitalization related to a fall is
$17,483 per event

Agency for Healthcare
Research and Quality (2014a)
Trepanier and Hilsenbeck (2014)

Family support network for child
with cancer

$2,776 Canadian dollars for 3 months Tsimicalis et al. (2013)

Healthcare-acquired infection data
dates 1999Y2007

CAUTI, $758/weight adjusted mean cost estimate
MRSA, $6,400/MRSA infection
C-difficile, $5,042/infection
CLABSI, $12,000/infection
Surgical never events, $62,000/event
Falls prevention, $4,233/event
VTE prevention,
$10,804/event-DVT
$16,644/event-PE

Pressure ulcer prevention, $1,878/event

Schifalacqua, Mamula, and
Mason (2011)

Heart disease $4,349Y$14,492 annual mean expenditure per person Agency for Healthcare Research
and Quality (2014b)

Continued

Journal for Nurses in Professional Development www.jnpdonline.com 179

TABLE 1 Known Costs of Outcomes, Continued
Outcome Reported average cost Sources
Hospital-acquired pressure ulcer Estimated extra cost $17,286 per case Agency for Healthcare Research

and Quality (2014a)

Hospital-based violence
intervention program (VIP)

Savings of $4,100 for 100 individuals
Average hospital costs post recidivism (base case)
$6,513 (range, $1,996Y$100,000)

Juillard et al. (2015)

Hospital-centered violence
intervention programs

Cost of VIP, $2,810
Average hospital costs post recidivism with standard
referrals, $18,722

Chong et al. (2015)

Hospitalizations for pediatric mental
health disorders

Total resource utilization charges/mean charges per
visit (Pediatrics):
Depression, 1.33 billion/$13,200
Bipolar, 702 million/$17,058
Psychosis, 540 million/$19,676
Externalizing disorder, 264 million/$18,784
Anxiety disorder, 149 million/$19,118
ADHD, 133 million/$19,118
Eating disorder, 108 million/$46,130
Substance abuse, 102 million/$12,098
Reaction disorder, 100 million/$8,444

Bardach et al. (2014)

Infection with clostridium difficile Outpatient and inpatient setting
Total of $11,314.70

Kuntz et al. (2012)

New RN orientation cost $49,000Y$92,000 (includes replacement costs) Trepanier, Early, Ulrich, and
Cherry (2012)

Nonmedical out-of-pocket expenses
venous thromboembolism (VTE)

VTE annual cost, $1.5 billion
Estimated total cost (in Australian dollars):
Baseline, $5,078,522
12 months after implementation, $4,833,083
Prophylaxis implementation:
Baseline, $104,311; 12 months $142,846
LMWH regimen:
Baseline, $71,313; 12 months, $92,295
LDUH regimen:
Baseline, $32,998; 12 months, $50,569
DVT treatment:
Baseline, $2,375,532; 12 month, $2,143,767
PE treatment:
Baseline, $470,284; 12 months, $420,180
Major bleeds:
Baseline, $762,057; 12 months, $828,977
HIT:
Baseline, $118,605; 12 months, $180,298
Postthrombotic syndrome:
Baseline, $1,247,732; 12 month, $1,116,997

Duff, Walker, Omari, and Stratton
(2013)
Data from January 2010 to
January 2011

OB adverse events Estimated extra cost per case $3,000 Agency for Healthcare Research
and Quality (2014a)

Postop venous thromboembolism Estimated extra hospitalization cost $8,000 Agency for Healthcare Research
and Quality (2014a)

RN turnover RN replacement cost $22,000Y$64,000 with average
cost per RN leaving $36,567

Robert Wood Johnson Foundation
(2010)

Continued

180 www.jnpdonline.com July/August 2016

Example 2: Eight-hour live class on workplace violence.
Situation. On the basis of an identified professional

practice gap, a continuing education program on work-
place violence is planned for 25 hourly employees from
the Emergency Department.

Background: At a large, central city acute care facility,
gun violence is a concern. The United States has the
greatest number of gun-related injuries per capita com-
pared to all other industrialized nations at 10.3 per
100,000 (fatal and nonfatal) occurring in 2011 (Jena, Sun,
& Prasad, 2014). The average cost per incident is $18,722

(Chong et al., 2015). Escalation of violent behaviors
resulted in 11 reported incidents in the Emergency De-
partment last year. Because domestic violence frequently
involves gun injuries, the community is at high risk for
gun-related injuries, and escalation of violent behaviors
has occurred at increasing frequency in the Emergency
Department, an educational program is proposed to help
increase employee safety.

Assessment. In calculating the costs for this program,
the NPD department purchased predeveloped content for
this course, so development time was reduced. A content
expert was used to review potential programs for pur-
chase, select one, and prepare to facilitate the course
with the identified development time. Instead of 43 hours
per hour of content (43 � 7 = 301 development hours)
required to develop this course, 70 hours were needed
(7 content hours � 10 hours = 70 hours to review, select,
and prepare to facilitate this program).

Cost analysis. Simple cost analysis is the total cost of the
educationalinterventiondivided bythenumber ofstaff mem-
bers participating in the education. Cost of the class per
person: $15,350/25 participants = $ 614.00 per participant.

BenefitYcost ratio. When calculating the benefitYcost
ratio, the total benefit is divided by the total cost. In this sce-
nario, if one incident of violence is prevented in 1 year at
$18,722 average cost per incident (Chong et al., 2015), the
net benefit is $18,722. See Table 1 for published costs of
outcomes. A positive benefit to the organization is noted
with the calculation:

$18,722 = 1.22 BCR
$15,350

Cost-effectiveness analysis. In this example, the cost-
effectiveness compares the cost of providing the 8-hour
program with the current practice of no educational

TABLE 1 Known Costs of Outcomes, Continued
Outcome Reported average cost Sources
Subcutaneous drug delivery Administration:

Subcutaneous, $30.19
Intravenous, $113.13

Dychter, Gold, and Haller (2012)

Surgical site infections Estimated extra cost per case $21,000 Agency for Healthcare Research and
Quality (2014a)

Treatment of mental disorders
(adult)

$1,849Y$6,003 annual mean expenditure
per person

Agency for Healthcare Research and
Quality (2014b)

Treatment of trauma-related
disorders

$2,609Y$12,,975 annual mean expenditure
per person

Agency for Healthcare Research and
Quality (2014b)

Ventilator-associated
pneumonia

$21,000 per incident Agency for Healthcare Research and
Quality (2014a)

Copyrighted ANPD.
All resources were accessed between June 22, 2015 and December 30, 2015.

Eight-Hour Live Class Expenses

Item Hours X hourly pay Total
Predeveloped
content purchase

$6,000

Content expert salary
(development)

70 hours � $35 $2,450

Content expert
salary (event)

8 hours � $35 $280

NPD practitioner
salary (event)

8 hours � $35 $280

Admin support salary 4 hours � $20 $80

Participant salary 8 hours � 25 participants
� $30

$6,000

Supplies $8/person� 25 participants $200

Marketing (internal) 3 hours � $20 $60

Total $15,350

Journal for Nurses in Professional Development www.jnpdonline.com 181

program. The cost comparison is $614.00 per participant
versus no educational expense. The outcome is the num-
ber of incidents reported. If the educational intervention
demonstrates a reduced incidence of workplace injury
from violence, that outcome is better that the current data
of 11 incidents last year.

Return on investment. If the proposed program pro-
duces a modest result of one less reported incident of
workplace violence, the ROI is 22%.

During the year following education, two fewer inci-
dents are reported:

$18,722j $15,350 � 100 = 22% ROI
$15,350

Recommendation. With the increased incidents of
workplace violence, employers must demonstrate due
diligence to protect employees, patients, and visitors
by preventing these incidents. From the combination of
CEA, benefitYcost ratios, and ROI calculations, this pro-
gram is clearly recommended.

Example 3: Frequency of a multi-day orientation.
Situation. An organization conducts a 7-day interpro-

fessional orientation 10 times per year. It is considering
increasing to 12 times per year to accommodate more
timely incorporation of newly hired employees.

Background. The number of participants per cohort
has ranged from 30 to 70. The significant range of cohort
sizes makes it difficult to plan for room size, number of
stations for skills lab, computer training rooms, faculty
schedules, and handout preparation. In addition, par-
ticipant satisfaction drops with decreased learner
engagement in large classes. If decreased engagement
leads to poor socialization and increased turnover,
Robert Wood Johnson Foundation (2010) places the
average cost for replacing an RN at $36,567. An increase
to offering orientation 12 times a year eliminates cohorts
with more than 45 participants and saves last minute
planning time related to human resources communica-
tion, room scheduling, class coordinating, and faculty
availability. Because the classes are already developed
and current, additional development time is not needed.

Additional Expenses for Large Cohorts

Item Hours X hourly pay Total

SALARIES for additional
coordination:

NPD practitioner
coordination

15 hours � $35 $525

Clerical staff support (OT) 10 hours � $30 $300

Human resources 2 hours � $30 $60

Room scheduling 2 hours � $20 $40

Informatics nurse (2 additional days) $480

Addition equipment rental $175

Transport of supplies and equipment to university $280

Room rental from university $1,000

Added faculty for skills (twice the number faculty) $680

Total additional cost $3,540

Multi-day Orientation Expenses

Item Hours X hourly pay Total
SALARIES for
Development:

NPD practitioner
coordination

6 hours � $35 $210

Clerical Support 3.5 hours � $20 $70

NPD practitioner post
program

3 hours � $35 $105

SALARIES for Presenters:

NPD practitioner
classroom

34 hours � $35 $1,190

CNO/Administration 1 hour � $60 $60

Shared governance rep. 1 hour � $30 $30

Social worker 1 hour � $35 $35

Risk Management 1 hour � $45 $45

QI 2 hours � $30 $60

Pharmacy 1 hour � $40 $40

Informatics Nurse 16 hours � $30 $480

Subtotal $2,325

SALARIES for Skills
Stations Faculty:

Lab tech 4 hour � $30 $120

Respiratory therapist 4 hour � $35 $140

Lactation specialist 4 hour � $35 $140

Epidemiology RN 4 hour � $35 $140

Code team RN 4 hour � $35 $140

Subtotal $680

Consumable supplies $8 � 45 participants $360

Total cost $3,365

182 www.jnpdonline.com July/August 2016

The comparison is between the current state of 10 offerings
a year and the proposed change to 12 offerings per year.

Assessment. Program costs are more extensive for a
7-day program. Salaries, supplies, equipment, and even
rental of space are considered.

Additional costs incurred when a cohort is over 45 par-
ticipants includes two additional Informatics Nurse
instructor-days for the electronic medical record class
due to lack of computers and doubling the skills stations
on the skills day, requiring more faculty, equipment, and
space. For illustration, when a cohort is 65 instead of 45,
the following costs are added:

The cost of orienting a large cohort (65 participants) is
calculated by starting with the costs of the 45 participant
cohort and adding expenses incurred for the larger group.

Large class per orientation cost (65 in cohort):
$3,365 (first 45 participants)
$160 (supplies for 20 more participants)

$3,540
(added salaries for coordination and faculty,
room, and equipment rental)

$7,065 (for 65 participants)
Cost analysis. The cost per participant is calculated

by adding the expenses from a year’s worth of orienta-
tion classes and dividing it by the total number of people
oriented.

10 scheduled courses for 510 new employees/year
7 months � $3,365 (45 participants) = $23,555
3 months � $7,065 (65 participants) = $21,195
10 months (510 participants) = $44,750

Total cost $44,750/510 = $87.74/participant

12 scheduled courses for 510 new employees/year
12 months � $3,365 (G45 participants) = $40,380

Total cost $40,380 / 510 = $79.17/participant
BenefitYcost ratio. If offering orientation 12 times per

year improves the socialization, confidence, and compe-
tence of new nurses resulting in two fewer nurses
leaving before their first anniversary, a savings of $73,134
($36,567 � 2) is realized. On the basis of 12 offerings, the
benefitYcost ratio is positive and reflects positive organiza-
tional impact.

Calculation of the benefitYcost ratio:
12 offerings per year

$73,134 = 1.81 BCR
$40,380

Cost-effectiveness analysis. This proposal indicates a cost
of $79.17/participant for the planned 12 offerings, which is
less than $87.74 /participant for 10 offerings with three large
groups. If the pattern seen from large orientation classes is a
higher turnover rate by first year anniversary of employ-
ment, improving socialization to the institution through
more personal contact in the first weeks of orientation

should improve retention. With CEA, the outcomes of both
interventions (10 offerings and 12 offerings) must be com-
pared. A conservative cost of turnover is $36,567 per
employee (Robert Wood Johnson Foundation, 2010). See
Table 1 for further published costs of outcomes including
new RN orientation cost.

Return on investment. The ROI for offering 12 orienta-
tions per year is calculated by using the average cost for
replacing two RNs of $73,134 and the cost of 12 months
of offering the orientation at $40,380 in the ROI formula.
The result is an ROI of 81.11%.

Return on investment:
12 offerings

$73,134j $40,380 � 100 = 81.11% ROI
40,380

Recommendation. By combining the calculations for
cost analysis, benefitYcost ratio, CEA, and ROI, strong sup-
port for increasing the frequency of the offerings of
orientation is noted. The decrease in cost per participant
from $87.74 to $79.17 is a financial argument, yet when
the ROI of reducing turnover is considered, it becomes a
strong recommendation. Smaller cohorts allow more small
group exercises to be incorporated and require fewer skills
stations. Smoother centralized orientation, offered at closer
intervals, should improve the new employee experience
and contribute to satisfaction and retention.

This example was conservative on the benefit calcula-
tion both in averaging the cost of replacement and in
estimating the number of retained staff after 1 year as a re-
sult of this educational intervention. The recommendation
to reduce the cohort size and increase the frequency of of-
ferings from 10 to 12 times per year is based on the financial
and clinical impact as manifested in the better outcome
of higher retention and less cost per participant.

IMPLICATIONS FOR FUTURE RESEARCH
No consistent method is routinely reported in the literature
to describe the financial and clinical impact of professional
development activities. Researchers and NPD practitioners
reporting on educational program evaluations must regu-
larly calculate financial impact when disseminating and
publishing results. This evidence can be used to guide de-
cisions for limited resources and to better position NPD
as integral in the decision-making process in healthcare
organizations.

CONCLUSIONS
NPD practitioners must measure the impact of education
interventions to demonstrate the success of professional
development activities. One seldom addressed aspect is
the financial impact measurement. The two articles in this
series show how routine approaches have been used (e.g.,
Kirkpatrick’s Levels of Evaluation, Phillips’ Five-Level ROI
Framework, and Paramoure’s Measurable Instructional

Journal for Nurses in Professional Development www.jnpdonline.com 183

Design) to measure ROI in professional development.
Critical appraisal of the literature, both quantitative and
qualitative, revealed the importance of reporting more
than participant satisfaction.

Four methods for evaluating the financial impact of
educational activities were reviewed, including cost analy-
sis, benefitYcost ratio, CEA, and ROI; plus examples were
given using these methods. More consistent measuring and
reporting of the financial and clinical impact of NPD activ-
ities is warranted.

The NPD practitioners must proactively demonstrate the
value of educational programs. During lean economic times,
participant attendance and satisfaction are not adequate
metrics to convince leaders of the organizational value of
educational activities.

References
Agency for Healthcare Research and Quality. (2014a). Interim update

on 2013 annual hospital-acquired condition rate and estimates of
cost savings and deaths averted from 2010 to 2013. Retrieved from
http://www.ahrq.gov/sites/default/files/wysiwyg/professionals/
quality-patient-safety/pfp/interimhacrate2013

Agency for Healthcare Research and Quality. (2014b). The concen-
tration of health care expenditures and related expenses for
costly medical conditions, 2012 (Agency for Healthcare Research
& Quality Medical Expenditure Panel Survey Statistical Brief #455).
Retrieved from http://meps.ahrq.gov/mepsweb/data_files/
publications/st455/stat455

American Faculty Association. (2012, February 8). Hours for teaching
and preparation rule of thumb: 2Y4 hours of prep for 1 hour of
class. Retrieved from http://americanfacultyassociation.blogspot.
com/2012/02/hours-for-teaching-and-preparation-rule.html

ANA’s Handle With Care Program (2016). Safe patient handling and
mobility. Retrieved from http://www.nursingworld.org/handle
withcare

Bardach, N. S., Coker, T. R., Zima, B. T., Murphy, J. M., Knapp, P.,
Richardson, L. P., I Mangione-Smith, R., (2014). Common and
costly hospitalizations for pediatric mental health disorders.
Pediatrics, 133(4), 602Y609.

Chong, V. E., Smith, R., Garcia, A., Lee, W. S., Ashley, L., Marks, A.,
I Victorino, G. P. (2015). Hospital-centered hospital violence
intervention programs: A cost-effectiveness analysis. American
Journal of Surgery, 209, 597Y603.

DeSilets, L. D. (2010). Calculating the financial return on educational
programs. Journal of Continuing Education in Nursing, 41(4),
149Y150.

Duff, J., Walker, K., Omari, A., & Stratton, C. (2013). Prevention of
venous thromboembolism in hospitalized patients: Analysis
of reduced cost and improved clinical outcomes. Journal of
Vascular Nursing, 31(1), 9Y14.

Dychter, S., Gold, D., & Haller, M. (2012). Subcutaneous drug delivery.
The Art and Science of Infusion Nursing, 35(3), 154Y160.

Fletcher, P. C., Markoulakis, R., & Bryden, P. J. (2012). The cost of
caring for a child with autism spectrum disorder. Issues in
Comprehensive Pediatric Nursing, 35, 45Y69.

Hollenbeak, C. (2011). The cost of catheter-related bloodstream
infections. The Art and Science of Infusion Nursing, 1(5), 309Y313.

Jegler, B. J., Johnson, T. J., Engstrom, J. L., Patel, A. L., Loera, F.,
& Meier, P. (2013). The institutional cost of acquiring 100 mL of
human milk for very low birth weight infants in the neonatal intensive
care unit. Journal of Human Lactation, 29(3), 390Y399.

Jena, A. B., Sun, E. C., & Prasad, V. (2014). Does the declining lethality
of gunshot injuries mask a rising epidemic of gun violence in the
United States? Journal of General Internal Medicine, 29(7),
1065Y1069.

Juillard, C., Smith, R., Anaya, N., Garcia, A., Kahn, J. G., & Dicker, R. A.
(2015). Saving lives and saving money: Hospital-based violence
intervention is cost-effective. The Journal Trauma Acute Care
Surgery, 78(2), 252Y257.

Kang, J., Mandsager, P., Biddle, A. K., & Weber, D. J. (2012). Cost-
effectiveness analysis of active surveillance screening for
methicillin-resistant staphylococcus aureus in an academic
hospital setting. Infection Control and Hospital Epidemiology,
33(5), 477Y486.

Kapp, K., & Defelice, R. (2009). Time to develop one hour of training.
Retrieved from https://www.td.org/Publications/Newsletters/
Learning-Circuits/Learning-Circuits-Archives/2009/08/Time-to-
Develop-One-Hour-of-Training

Kettner, P., Moroney, R., & Martin, L. (2013). Designing and
managing programs: An effectiveness-based approach.
Washington, DC: Sage Publications, Inc.

Kuntz, J. L., Johnson, E. S., Raebel, M. A., Petrik, A. F., Yang, X.,
Thorp, M. L., I Smith, D. H. (2012). Epidemiology and health-
care costs of incident clostridium difficile infections identified
in the outpatient healthcare setting. Infection Control and Hos-
pital Epidemiology, 33(10), 1031Y1038.

Opperman, C., Liebig, D., Bowling, J., Johnson, C., & Harper, M.
(2016). Measuring return on investment for professional
development activities: A review of the literature. Journal for
Nurses in Professional Development, 32(3), 122Y129.

OSHA Safety Pays Program Estimator. (2016). Estimated costs
of occupational injuries and illnesses and estimated impact
on a company’s profitability worksheet. Retrieved from
https://www.osha.gov/dcsp/smallbusiness/safetypays/
estimator.html

Robert Wood Johnson Foundation. (2010). Wisdom at work:
Retaining experienced nurses. Retrieved from http://www.rwjf.
org/en/library/research/2010/07/wisdom-at-work–retaining-
experienced-nurses.html

Roe, E., & Williams, D. L. (2014). Using evidence-based practice to
prevent hospital acquired pressure ulcers and promote wound
healing. The American Journal of Nursing, 114(8), 61Y65.

Schifalacqua, M.M.,Mamula,J., &Mason, A.R.(2011). Returnon invest-
ment imperative. Nursing Administration Quarterly, 35(1), 15Y20.

Trepanier,S.,Early,S.,Ulrich,B.,&Cherry,B.(2012).Newgraduatenurse
residency program: A costYbenefit analysis based on turnover
and contract labor usage. Nursing Economic$, 30(4), 207Y214.

Trepanier, S., & Hilsenbeck, J. (2014). A hospital system approach at
decreasing falls with injuries and cost. Nursing Economic$, 32(3),
135Y141.

Tsimicalis, A., Stevens, B., Ungar, W. J., Greenberg, M., McKeever, P.,
Agha, M., I Moineddin, R. (2013). Determining the costs of
families’ support networks following a child’s cancer diagnosis.
Cancer Nursing, 36(2), E8YE19.

Warren, J. I. (2013). Program evaluation and return on investment. In
Bruce, S. L. (Ed.), Core curriculum for nursing professional
development (4th ed. pp. 547Y68). Chicago, IL: Association for
Nursing Professional Development.

184 www.jnpdonline.com July/August 2016

http://www.ahrq.gov/sites/default/files/wysiwyg/professionals/quality-patient-safety/pfp/interimhacrate2013

http://meps.ahrq.gov/mepsweb/data_files/publications/st455/stat455

http://americanfacultyassociation.blogspot.com/2012/02/hours-for-teaching-and-preparation-rule.html

http://www.nursingworld.org/handlewithcare

https://www.td.org/Publications/Newsletters/Learning-Circuits/Learning-Circuits-Archives/2009/08/Time-to-Develop-One-Hour-of-Training

https://www.osha.gov/dcsp/smallbusiness/safetypays/estimator.html

http://www.rwjf.org/en/library/research/2010/07/wisdom-at-work–retaining-experienced-nurses.html

Module 5 (week 9) – Intervention

As you continue to work on your assignment, you will be pulling in some information from your work throughout this course. For one part of this presentation, you will be identifying the current problem (or opportunity for change). This was part of your discussion in the week 2 assignment PowerPoint.

You will also propose an evidence-based intervention to address this particular problem. This intervention should be derived from the literature you have found and presented in your critical appraisal template.

As you have seen, these assignments have provided you the ability to identify a problem, develop a PCIOT question, search for evidence related to this PICOT, critically appraise the evidence for a solution to the problem, and now you will identify the solution and disseminate the results.

You are well on your way to becoming evidence-based practitioners!

Week 9!

Nice work on last week’s discussion. As you have discovered, decision aids can be very helpful when providing information for patients and families.

This week, you will continue to work on your assignment for this module. This will be an 8-9 slide PowerPoint presentation in which you will recommend an evidence-based practice change. Review the 4 articles you critiqued to determine what practice change is supported by the literature.

Some of the content for this assignment will be taken from your previous work and some will be new. This PowerPoint is a total of 8-9 slides.

Please review the full assignment details located under the learning resources for module 5.

Please let me know if you have questions

David

Turn in your highest-quality paper
Get a qualified writer to help you with

“ 52/9 Assgn ”

Get high-quality paper

NEW! AI matching with writer

Hire a Writer

Client Reviews

4.9

Sitejabber

4.6

Trustpilot

4.8

Our Guarantees

100% Confidentiality

Information about customers is confidential and never disclosed to third parties.

Original Writing

We complete all papers from scratch. You can get a plagiarism report.

Timely Delivery

No missed deadlines – 97% of assignments are completed in time.

Money Back

If you're confident that a writer didn't follow your order details, ask for a refund.

New to Your Trusted Assignment Help Service? Sign up & Save

Calculate the price of your order

Type of paper needed:

Pages:

You will get a personal manager and a discount.

Academic level:

We'll send you the first draft for approval by at

Total price:

$0.00

Power up Your Academic Success with the
Team of Professionals. We’ve Got Your Back.

Power up Your Study Success with Experts We’ve Got Your Back.

Order Now Order Now