Home Uncategorized CASE STUDY

Uncategorized

CASE STUDY

– 7 page (not including cover page, citations, and reference page)

– APA FORMAT

– NO PLAGIARISM

– NEED PLAGIARISM REPORT

– INTEXT CITATION

– QUESTION IS ATTACHED and SOURCE TOO (TWO DIFFERENT FILES)

SYSTEMATIC REVIEW Open Access

Engaging patients to improve quality of
care: a systematic review
Yvonne Bombard1,2*, G. Ross Baker1, Elaina Orlando1,3, Carol Fancott1, Pooja Bhatia1, Selina Casalino2,
Kanecy Onate1, Jean-Louis Denis4 and Marie-Pascale Pomey5

Abstrac

Background: To identify the strategies and contextual factors that enable optimal engagement of patients in the
design, delivery, and evaluation of health services.

Methods: We searched MEDLINE, EMBASE, CINAHL, Cochrane, Scopus, PsychINFO, Social Science Abstracts, EBSCO,
and ISI Web of Science from 1990 to 2016 for empirical studies addressing the active participation of patients,
caregivers, or families in the design, delivery and evaluation of health services to improve quality of care. Thematic
analysis was used to identify (1) strategies and contextual factors that enable optimal engagement of patients,
(2) outcomes of patient engagement, and (3) patients’ experiences of being engaged.

Results: Forty-eight studies were included. Strategies and contextual factors that enable patient engagement were
thematically grouped and related to techniques to enhance design, recruitment, involvement and leadership action,
and those aimed to creating a receptive context. Reported outcomes ranged from educational or tool development
and informed policy or planning documents (discrete products) to enhanced care processes or service delivery and
governance (care process or structural outcomes). The level of engagement appears to influence the outcomes of
service redesign—discrete products largely derived from low-level engagement (consultative unidirectional
feedback)—whereas care process or structural outcomes mainly derived from high-level engagement (co-design or
partnership strategies). A minority of studies formally evaluated patients’ experiences of the engagement process
(n = 12; 25%). While most experiences were positive—increased self-esteem, feeling empowered, or independent—some
patients sought greater involvement and felt that their involvement was important but tokenistic, especially when their
requests were denied or decisions had already been made.

Conclusions: Patient engagement can inform patient and provider education and policies, as well as enhance service
delivery and governance. Additional evidence is needed to understand patients’ experiences of the engagement process
and whether these outcomes translate into improved quality of care.

Registration: N/A (data extraction completed prior to registration on PROSPERO).

Keywords: Patient engagement, Patient involvement, Quality of care, Quality improvement, Health services,
Health delivery, Systematic review

* Correspondence: yvonne.bombard@utoronto.ca
1Institute of Health Policy, Management and Evaluation, University of
Toronto, 155 College Street 4th Floor, Toronto, Ontario M5T 3M6, Canada
2Li Ka Shing Knowledge Institute, St. Michael’s Hospital, 30 Bond St, Toronto,
Ontario M5B 1W8, Canada
Full list of author information is available at the end of the article

© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Bombard et al. Implementation Science (2018) 13:98
https://doi.org/10.1186/s13012-018-0784-z

Background
Patient engagement has become a cornerstone of quality
of care [1–6] and is a frequently stated goal for healthcare
organizations. Traditionally, and most commonly, this en-
gagement has focused on the relationship between pa-
tients and providers in making care decisions or how to
improve patient efforts to manage their own care [7].
However, there are growing efforts to integrate patients i

broader ways, including efforts to improve or redesign ser-
vice delivery by incorporating patient experiences [8–12].
These efforts are due in part to an increased recognition
and acceptance that users of health services have a rightful
role, the requisite expertise, and an important contribu-
tion in the design and delivery of services [4]. While the
nature of patient engagement may vary from including pa-
tients as members of a board to time-limited consultation
with patients on service redesign, its aims are consistent—
to improve the quality of care [11, 13, 14].
Healthcare organizations have a long tradition of

measuring the experience of patients, and health service
“users” including families, caregivers, and clients, with
their services. Yet, traditional satisfaction surveys often
prove difficult to translate into improved service delivery
[15, 16]. Indeed, research on patient engagement has
pointed to the importance of augmenting traditional sur-
veys and complaint processes, moving towards fuller en-
gagement of patients in reviewing and improving the
quality of service delivery in institutions and in the com-
munity [17–25]. This recognition has been accompanied
by a growth in the development of instruments to meas-
ure and improve the quality of care patients receive.
Over the past two decades, assessments of quality of
care from the patient perspective have shifted from pa-
tient satisfaction to patient experiences [26]. Increasing
literature indicates that it is not only feasible to involve
patients in the delivery or re-design of health care [9]
but that such engagement can lead to reduced hospital
admissions [27], improved effectiveness, efficiency and
quality of health services [28–31], improved quality of
life, and enhanced quality and accountability of health
services [9]. Frameworks of patient involvement have
been developed that move from the traditional view of
the patient as a passive recipient of a service to an inte-
gral member of teams re-designing health care [8, 11].
For example, one framework developed by Bate and
Robert (2006) describes a continuum of patient involve-
ment, which ranges from complaints, giving information,
listening, and consulting towards experience-based co-de-
sign of services [8]. Low-level engagement, such as con-
sulting, comprises largely unidirectional feedback (e.g.,
focus groups, surveys, interviews), whereas high-level en-
gagement, like co-design, represents a partnership in the
design or evaluation of services. A more recent framework
developed by Carman et al. describes various levels of

engaging patients and families in health and health care,
from consultation or involvement to partnership and shared
leadership in various activities including direct care,
organizational design, and governance to policy-making
[11]. Carman’s continuum of engagement was influenced by
Arnstein’s formative “ladder of citizen participation,” a con-
tinuum of public participation in governance ranging from
limited participation to a state of collaborative partnership
in which citizens share leadership or control decisions [32].
Governments and health care institutions are urged by

some experts to engage patients and other service users, in-
cluding caregivers and relatives in more robust ways [8, 33]
where patients are actively involved as partners or co-leads
in organizational re-design and evaluation of health care
delivery, as depicted by the red section in Carman’s frame-
work (Fig. 1). Despite the substantive body of research on
strategies to engage patients and their effects on patients
and health services, the literature is dispersed and has not
been recently synthesized into a coherent overview. If the
benefits of engaging patients in the design or delivery of
health care are to be realized at an organization or system
level, then effective strategies and the contextual factors en-
abling their outcomes need to be identified so that learning
can be generalized. We conducted a systematic review of
international English language literature on strategies for
actively engaging patients and families in improving or
redesigning health care and the contextual factors influen-
cing the outcomes of these efforts. The explicit questions
that guided our review were:

1. What are the strategies and contextual factors th

enable optimal engagement of patients in the
design, delivery, and evaluation of health services?

2. What are the outcomes of patient engagement on
services?

3. What are patients’ experiences of being engaged?

Methods
Approach
We took a comprehensive approach in our systematic
search and included all empirical qualitative, quantita-
tive, and mixed methods study designs across all settings
of care to address our narrow research questions. Our
review did not fit into typologies of literature reviews
[34, 35], given that we included qualitative and quantita-
tive studies (to capture the breadth of studies in this
area), employed a thematic analysis (given the multipli-
city of designs), and applied a quality appraisal. We
followed the PRISMA reporting criteria for Systematic
Reviews and Meta-Analyses (Fig. 2) [36].

Search strategy
In accordance with the core principles of systematic
review methodology [37], we conducted a systematic

Bombard et al. Implementation Science (2018) 13:98 Page 2 of 22

review of relevant literature with the help of a librarian
using the electronic databases of: MEDLINE, EMBASE,
CINAHL, the Cochrane Library, Scopus, PsychINFO,
Social Science Abstracts, AbiInform Business Source
Premier (EBSCO), and ISI Web of Science. We searched

the databases using the following subject headings related
to patient engagement—combinations of “patient”, “user”,
“client”, “caregiver”, “family” and “engage*”, “participat*”,
“involve*”, “consult*”; for those related to designing, evalu-
ating and delivery of services—combinations of “design”,

Fig. 1 Patient engagement frameworks used for the selection and analyses of studies included in our review. The red box indicates the level of
engagement along the continuum that is the focus of our studies included in our review [11]. The organizing framework used for analyzing the
studies reviewed [8]

Bombard et al. Implementation Science (2018) 13:98 Page 3 of 22

“deliver*”, “evaluat*”, “outcome”, “develop*”, “plan*” and
“health services”, “health care”, “health”, “service”. We in-
cluded a combination of search terms from each category
for each search, for example, “patient” AND “engage*”
AND “design” AND “health services”).

Criteria for selection
Studies were eligible for inclusion if they were available
empirical articles that explicitly investigated the partici-
pation of patients, caregivers, or families in the design,
delivery, and evaluation of health services, which aligns
with involving or partnering/sharing leadership with pa-
tients in organizational design and governance, reflecting
Carman’s framework (Fig. 1) [11]. Searches were re-
stricted to qualitative, quantitative, or mixed methods
articles published in English between January 1990 and
March 2016. We chose 1990 as this coincided with the
emergence of patient engagement particularly in mental
health services and the broader quality of care discourse.
All settings of care were eligible. We excluded articles
that did not explicitly address patient engagement, as
well as those that did not pertain to the broader design,
delivery, and evaluation of health services (e.g., directly
engaging patients in patient safety activities such as

challenging staff who treat them to wash their hands or
monitor the use of a safety checklist in their care, or in
their self-management or treatment decisions, or studies
pertaining to patient involvement in health research,
community development, or health promotion). We also
excluded articles that did not describe the outcome of
the engagement of patients and those in which the out-
comes did nor pertain to the design, delivery, or evalu-
ation of health services (e.g., those that related to
developing questionnaires or conceptual frameworks, in-
sights on how to engage patients or work collabora-
tively). We focused on studies that consulted, involved,
partnered, or co-designed health services with patients,
informed by Bate and Robert’s [8] and Carman et al.’s
[11] frameworks on patient engagement (Fig. 1). Finally,
theoretical or conceptual articles as well as those focused
on guideline development, instrument development, or
broader organizational issues were excluded.
Titles and abstracts of the papers were examined to

decide if the full article should be retrieved (Fig. 2). EO
and CF were the primary reviewers who examined the ti-
tles and abstracts, applied inclusion criteria to the articles,
and abstracted the data using an abstraction form. Any
disagreement and uncertainties regarding inclusion were

Fig. 2 Flow diagram for search and selection process. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting
Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6 (6): e1000097. https://doi.org/10.1371/journal.pmed1000097

Bombard et al. Implementation Science (2018) 13:98 Page 4 of 22

discussed and agreed upon by an additional reviewer (YB

)

on the abstraction form. We conducted calibration exer-
cises to ensure reliability in applying the selection criteria.
Reviewers independently screened the titles and abstracts,
and discrepancies were discussed and reviewed by the
third reviewer. There was a 95.46% observed agreement
and 85.75% expected agreement between primary re-
viewers, with a kappa statistic of 0.703 (standard error,
0.021; 95% confidence interval, 0.662–0.744), which is
relatively high compared to other knowledge synthesis
protocols reporting 50% consistency rates [34].

Data abstraction and synthesis
Data abstraction forms were used to describe the studies’
population, location (i.e., country), goals, methodology, and
outcomes (Table 1); contextual factors influencing engage-
ment (i.e., leadership and specific barriers and facilitators
to patient engagement) (Table 2); and patients’ experience
with the engagement and evaluation of study quality
(Tables 2 and 3). Studies were then categorized by the level
of patient engagement using Bate and Robert’s (2006) con-
tinuum of patient involvement [8]. Consistent with our
aims to review strategies for actively engaging patients and
families in improving or redesigning health care, we
focused on studies using co-design or those consulting
patients but also using elements of co-design—i.e., the
more active levels of engagement on the Bates and Robert
continuum. We classified changes or products of engaging
patients as “quality of care outcomes” and the impact of
the engagement on patients as “patients experience
outcomes” (Table 1). Quality of care outcomes were
categorized into one of the following: developing education
or a service-related tool, informing policy or planning
documents, and enhancing services or governance. Study
quality was assessed by one person and two verifiers using
a quality appraisal tool that systematically reviews disparate
forms of evidence and methodologies on a scale from “very
poor,” “poor,” “fair,” and “good” [38], which reflected the
mixed methods articles in our review. Verification involved
systematically checking and confirming the fit between
each criterion of the assessment tool and the conceptual
work of analysis and interpretation of study quality among
a subset of studies. We also assessed the possible impact of
study quality on the review’s findings (akin to a “sensitivity
analysis” conducted for meta-analyses).

Data analysis
Data were analyzed to address the three research ques-
tions, with the intention of (1) identifying strategies and
contextual factors that enable optimal engagement of pa-
tients in the design, delivery, and evaluation of health ser-
vices; (2) identifying the outcomes of patient engagement;
and (3) exploring patients’ experiences of being engaged.
YB analyzed the data using quantitative (i.e., frequency

analysis) and qualitative methods. YB used thematic ana-
lysis to identify the strategies and contextual factors (i.e.,
barriers and facilitators), outcomes, and experiences of op-
timal patient engagement. This process involved identify-
ing prominent or recurring themes in the literature
(relevant to our research questions) and summarizing the
findings of different studies under thematic headings using
summary tables. A coding framework was developed to
thematically describe the strategies and contextual factors
enabling patient engagement. YB and RB refined the
framework as new data emerged during the analysis.

Results
Included studies
We found a total of 20,957 studies about involving pa-
tients in the design, delivery, or evaluation of health care.
Of these, we excluded 20,909 because they did not report
outcomes related to health care delivery, design, or evalu-
ation (n = 67) or only informed/consulted with patients, as
opposed to engage them in co-design (n = 91) (Fig. 2;
Additional file 1: Table S3 & Additional file 2: Figure S1).
Our final sample of studies included 48 papers involving
patients, families, and caregivers along with service users,
health care providers, staff, board members, health care
managers, administrators, and decision-makers (Table 1).
The publication date of the included studies spanned from
1993 to 2016, and interestingly, co-design was employed
as early as 1993 to as recently as 2015 in published stud-
ies. Of the 48 included studies, 27 were qualitative studies;
3 were quantitative; 13 constituted mixed methods stud-
ies, which included qualitative, quantitative methods; and
5 comprised user panels or advisory meetings (Table 4).
We restricted our analysis to articles actively engaging pa-
tients. Half of the articles (n = 24) included consultative
activities typical of low-level engagement (i.e., where pa-
tients provided input on research design or measures as
part of the research or administrative team). The other
half were co-design (high-level engagement—i.e., delibera-
tive, reflexive processes where patients and providers work
together to create solutions [39]) (Table 4). Engagement
efforts spanned a range of services, including pediatrics,
community and primary care, and most frequently oc-
curred in mental health services (n = 17; 35%—Tables 4
and 1). Studies originated from various countries, with
most deriving from the UK (n = 26; 54%) (Tables 4 and 1).
Few studies formally evaluated patients’ experiences of the
process of being engaged (n = 12; 25%) (Additional file 3:
Table S1).

Strategies for optimal patient engagement to improve
quality of care
We identified various strategies that contributed to opti-
mal patient engagement, which were mediated by key
contextual factors that enabled or constrained the

Bombard et al. Implementation Science (2018) 13:98 Page 5 of 22

T
a
b
le

1
O
ve
rv
ie
w

o
f

p
at
ie
n
t
en

g
ag
em

en
t

st
u
d
ie

s
to

p
ro
ve

q
u
al
it
y

o
f

ca
re

St
u
d
y

C
o
u
n
tr
y

In
te
rv
en

ti
o
n

Ty
p
e
o
f

se
rv
ic
e

Sa
m
p
le
si
ze

Le
ve
l
o
f
en

g
ag
em
en
t

O
u
tc
o
m
es

o
n

q
u
al
it
y
o
f
ca
re

Im
p
ac
t
o
n
in
st
it
u
tio

A
cr
i
et

al
.2
01
4
[6
5]

U
SA

C
o
m
m
u
n
it
y

co
lla
b
o
ra
ti
o
n

m
o
d
el

M
en

ta
l

h
ea
lt
h

se
rv
ic
es

C
o
n
su
lt
at
iv
e

co
-d
es
ig
n

D
ev
el
o
p
ed

an
d

p
le

m
en

te
d

sc
re
en

in
g

,e
d

u
ca
ti
o
n

,a
n
d
an

p
o
w
er

m
en

in
te
rv
en

ti
o
n

fo
r

p
ee
r-
d
el
iv
er
ed

se
rv
ic
es

ta
rg
et
ed

at
im

p
ro
vi
n
g

em
o
ti
o
n
al
h
ea
lt
h
o
f

lo
w
-in

co
m
e

n
ew

m
o
th
er
s

U
se
rs
h
ad

an
eq

u
al
vo
ic
e

th
ro
u
g
h
o
u
t

al
l

as
p
ec
ts
o
f

b
u
ild
in
g

th
e

in
te
rv
en

ti
o
n
,

h
ic
h

u
al
iz
ed

th
e
p
o
w
er

d
iff
er
en

ti
al

M
ee
ti
n
g
s

Ba
rn
es

20
00

[7
5]

U
K

D
o
cu
m
en

t
an
al
ys
is

in
te
rv
ie
w
s

q
u
es
ti
o
n
n
ai
re

M
en

ta
l
h
ea
lt
h

se
rv
ic
es

N
o
t
cl
ea
r

C
o
-d
es
ig
n

D
ev
el
o
p
ed

a
n
ew

p
ro
g
ra
m

A
ch
ie
ve
d
a

cu
lt
u
re

ch
an
g
e

to
w
ar
d
s
va
lu
in
g

u
se
rs

’

kn
o
w
le
d
g
e,
ex
p
er
ti
se
,

co
n
tr
ib
u
ti
o
n
s,
an
d
g
re
at
er

p
o
w
er
-s
h
ar
in
g

Ba
rn
es

an
d

W
is
to
w

19
94

[4
0]

U
K

U
se
r

p
an
el
s

C
o
m
m
u
n
it
y
m
en

ta
l

h
ea
lt
h
an
d
d
is
ab
ili
ti
es

se
rv
ic
es

N
o
t

d
is
cu
ss

C
o
n
su
lt
at
iv
e
w
it
h

so
m
e
co
-d
es
ig
n

d
en

ti
fie
d
st
ra
te
g
ie
s
fo
r

u
se
r

in
vo
lv
em

en
t;
so
m
e
ev
id
en

ce
o
f
se
rv
ic
e
im

p
ro
ve
m
en

ts
;

d
ev
el
o
p
ed

a
“c
h
an
g
e

g
en

d
a”

A
ch
ie
ve
d
a
cu
lt
u
re
ch
an
g
e

vi
a

“t
o
p
-d
o
w
n
”
ap
p
ro
ac
h

fr
o
m

lo
ca
l
au
th
o
rit
y

Be
rg

et
al
.2
01
5
[5
0]

N
o
rw

ay
U
se
r

b
o
ar
d

C
o
m
m
u
n
it
y

H
IV

/A
ID
S

p
re
ve
n
ti
o
n

7
C
o
-d
es
ig
n

C
re
at
ed

a
d
es
ig
n
fo
r
an

o
u
tp
at
ie
n
t
cl
in
ic

Em
p
o
w
er
m
en

t
an
d

au
to
n
o
m
y
o
f
u
se
rs
th
ro
u
g
h

“a
ct
iv
e
ci
ti
ze
n
sh
ip
”
an
d
a

“e
g
al
it
ar
ia
n
sp
iri
t”

Bl
ic
ke
m

et
al
.2
01
3
[9
2]

U
K

Fo
cu
s

g
ro
u
p
s

M
en

ta
l
h
ea
lt
h
se
rv
ic
es

8
in

in
te
rv
ie
w
s

C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n
D
ev
el
o
p
ed

an
d
te
st
ed

a
p
ro
to
co
l
fo
r
PL
A
N
S,
a

co
m
m
u
n
it
y-
b

as
ed

re
fe
rr
al

sy
st
em

C
lo
se

en
g
ag
em

t
o
f

p
o
te
n
ti
al
u
se
rs

re
su
lt
ed

in
a

g
ra
ss
ro
o
ts

u
n
d
er
st
an
d
in
g

o
f

th
e
su
p
p
o
rt
va
lu
ed

b
y

in
d
iv
id
u
al
s

In
te
rv
ie
w
s

Su
rv
ey
s

6
fo
cu
s
g
ro
u
p
s
(t
o
ta
l

n
u
m
b
er

n
o
t

d
is
cu
ss
ed
)

Bo
n
e
et

al
.2
01
3
[9
3]

U
SA

C
o
m
m
u
n
it
y-

ac
a

d
em

ad
vi
so
ry

co
m
m
it
te
e

C
an
ce
r

sc
re
en

in
g
fo
r

A
fr
ic
an

A
m
er
ic
an
s

N
o
t
d
is
cu
ss
ed

C
o
-d
es
ig
n
D
ev
el
o
p
ed

a
co
m
m
u
n
it
y

h
ea
lt
h
w
o
rk
er

m
o
d
el
to

en
g
ag
e
A
fr
ic
an
-A
m
er
ic
an

co
m
m
u
n
it
ie
s
in

ca
n
ce
r

sc
re
en

in
g
an
d
ca
re

Id
en

ti
fie
d
th
e
co
m
m
u
n
it
y-

ac
ad
em
ic
ad
vi
so
ry
co
m
m
it
te
e

as
vi
ta
l
to

d
es
ig
n
in
g
th
e

se
rv
ic
e

an
d
en

su
rin

g
it
s

ef
fe
ct
iv
en

es
s

Br
o
o
ks

20
08

[6
4]

U
K
Fo
cu
s
g
ro
u
p
s

G
en

er
al
h
ea
lt
h

se
rv
ic
es

52
C
o
-d
es
ig
n

Pa
ti
en

t
in
vo
lv
em

en
t
in

au
d
it
in
g

p
ro
ce
ss

es
,

d
ev
el
o
p
m
en

t
o
f
p
at
ie
n
t

q
u
es
tio

n
n
ai
re
s,
p
o
lic
ie
s,
an
d

fr
am

ew
o
rk
s

Re
in
fo
rc
ed

th
e
im

p
o
rt
an
ce

o
f

p
at
ie
n
t
n
ar
ra
ti
ve
s
an
d

kn
o
w
le
d
g
e
in

o
rg
an
iz

at
io
n

an
d

d
el
iv
er
y

o
f
h
ea
lt
h
ca
re

In
te
rv
ie
w
s

O
b
se
rv
at
io
n
s

Bu
ck

20
04

[7
6]

U
S

C
it
iz
en

ad
vi
so
ry
b
o
ar
d
G
en
er
al
h
ea
lt
h
se
rv
ic
es

(f
o
r
h
o
m
el
es
s

in
d
iv
id
u
al
s)

7
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n
D
ev
el
o
p
ed

in
fo
rm

at
io
n
al

b
ro
ch
u
re
s
an
d

re
co
m
m
en

d
at
io
n
s

fo
r

lo
ca
l

in
te
rv
en

ti
o
n
s

an
d
se
rv
ic
es

to
im

p
ro
ve
g
en
er
al
h
ea
lt
h
se
rv
ic
es

fo
r
th
e
h
o
m
el
es
s

A
ch
ie
ve
d
co
lla
b
o
ra
ti
o
n

an
d

m
u
tu
al
ed

u
ca
ti
o
n

Bombard et al. Implementation Science (2018) 13:98 Page 6 of 22

T
a
b
le
1
O
ve
rv
ie
w

o
f
p
at
ie
n
t
en

g
ag
em

en
t
st
u
d
ie
s
to

im
p
ro
ve

q
u
al
it
y
o
f
ca
re

(C
o
n
tin
u
ed
)

St
u
d
y
C
o
u
n
tr
y
In
te
rv
en
ti
o
n

Ty
p
e
o
f
se
rv
ic
e

Sa
m
p
le
si
ze
Le
ve
l
o
f
en
g
ag
em
en
t
O
u
tc
o
m
es

o
n
q
u
al
it
y
o
f
ca
re

Im
p
ac
t
o
n
in
st
it
u
tio
n

C
ar
ls
o
n
an
d
Ro

se
n
q
vi
st

19
90

[5
1]

Sw
ed

C
o
n
su
lt
at
io
n

m
ee
ti
n
g
s

D
ia
b
et
es

ca
re

24
3

C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n

p
le
m
en

ta
ti
o

n
o
f

ca
re

im
p
ro
ve
m
en

t
p
ro
g
ra
m
s
an
d

p
at
ie
n
t
in
fo
rm

at
io
n
Id
en

ti
fie
d
p
ro
b
le
m
s
an
d
st
ep

s
to

so
lv
e
th
em

In
te
rv
ie
w
s

Tr
ai
n
in
g
co
u
rs
e

C
aw

st
o
n
20
07

[6
9]

U
K
Fo
cu
s
g
ro
u
p
s

Pr
im

ar
y
ca
re

72
in

fo
cu
s
g
ro
u
p
s;

37
2
vi
a

q
u
es
ti
o
n
n
ai
re
s

C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n

Re
co
m
m
en

d
at
io
n
s
an
d
so
m
e

ch
an
g
es

m
ad
e
to

im
p
ro
ve

d
ia
b
et
es

m
an
ag
em

en
t
in

p
rim

ar
y
ca
re
C
re
at
ed

re
se
ar
ch
-c
o
m
m
u
n
it
y

p
ar
tn
er

sh
ip
s
in

ev
al
u
at
in
g

se
rv
ic
es

b
u
t
h
ad

a
m
o
d
es
t

im
p
ac
t
o
n
se
rv
ic
e
ch
an
g
e

In
te
rv
ie
w
s

Q
u
es
ti
o
n
n
ai
re

C
o
ad

20
08

[4
1]

U
K

Yo
u
th

C
o
u
n
ci
l

Pe
d
ia
tr
ic
s—

ac
u
te

ca
re

yo
u
th

se
rv
ic
es

17
C
o
-d
es
ig
n

D
em

o
n
st
ra
te
d
im

p
ac
t
o
f

yo
u
th

co
u
n
ci
l
o
n
sp
ec
ifi
c

ar
ea
s
o
f
im

p
ro
ve
m
en
t
Id
en

ti
fie
d
w
ay
s
o
f
p
ro
m
o
ti
n
g

fu
rt
h
er

in
vo
lv
em
en
t

C
o
ke
r
et

al
.2
01
4
[7
2]

U
SA

C
o
m
m
u
n
it
y
ad
vi
so
ry

b
o
ar
d

Pe
d
ia
tr
ic
s

3
C
o
-d
es
ig
n

D
ev
el
o
p
ed
ca
re

m
o
d
el
s
fo
r

w
el
l-c
h
ild

ca
re
N
o
t
d
is
cu
ss
ed

w
el
l

20
14

[4
8]

U
K

G
ro
u
p
m
ee
ti
n
g
s

A
cu
te

ca
re
N
o
t
d
is
cu
ss
ed
C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n
D
ev
el
o
p
ed
an
d
im
p
le
m
en
te
d
ca
re

p
at
h
w
ay
s
fo
r
ce
llu
lit
is

ca
re

in
th
e
h
o
sp
it
al

U
se
r
in
vo
lv
em

en
t
cr
ea
te
d
th
e

d
es
ire

to
ch
an
g
e
in

th
e

o
rg
an
iz
at
io
n

A
lig
n
ed

u
se
r
in
vo
lv
em

en
t

w
it
h
st
ra
te
g
ic
d
ire
ct
io
n
s

En
n
is
et

al
.2
01
4
[7
0]

U
K

Fo
cu
s
g
ro
u
p
s,

in
te
rv
ie
w
s,

se
rv
ic
e

u
se
r

p
la
n
n
in
g

co
m
m
it
te
e,
su
rv
ey
s,

u
sa
b
ili
ty

te
st
in
g

M
en
ta
l
h
ea
lt
h
se
rv
ic
es

12
1
u
se
rs
vi
a
su
rv
ey
s,

u
n
cl
ea
r
n
u
m
b
er

vi
a

fo
cu
s
g
ro
u
p
s,

8
u
se
rs

vi
a
u
sa
b
ili
ty

te
st
in
g
,4

u
se
rs
vi
a
se
rv
ic
e
u
se
r

p
la
n
n
in

g
co
m
m
it
te
e

C
o
n
su
lt
at
iv
e

to
–
co
-d
es
ig
n

D
ev
el
o
p
ed

el
ec
tr
o
n
ic

p
er
so
n
al

h
ea
lt
h
re
co
rd

fo
r
m
en

ta
l
h
ea
lt
h
p
at
ie
n
ts

N
o
t
d
is
cu
ss
ed

En
riq

u
ez

et
al
.2
01
0
[6
7]

U
SA
Fo
cu
s
g
ro
u
p
s
q
u
es
ti
o
n
n
ai
re
s
H
IV

an
d
in
ti
m
at
e

p
ar
tn
er

vi
o
le
n
ce

p
re
ve
n
ti
o
n

7
u
se
r

p
ar
tn
er
s
in

d
es
ig
n
;3
1
p
ar
ti
ci
p
an
ts

in
fe
as
ib
ili
ty

st
u
d
y

C
o
-d
es
ig
n

Fe
as
ib
ili
ty

o
f
n
ew

se
rv
ic
e

w
as

es
ta
b
lis
h
ed

h
ic
h
im

p
ro
ve
d

p
ro
te
ct
iv
e
h
ea
lt
h
b
eh

av
io
rs
,

se
lf-
es
te
em

,s
o
ci
al
su
p
p
o
rt
,

an
d
at
ti
tu
d
es

to
w
ar
d
s
p
ar
tn
er

vi
o
le
n
ce

D
el
iv
er
y
o
f
in
te
rv
en

ti
o
n
w
as

d
ee
m
ed

fe
as
ib
le

,c
o
m
m
u
n
it
y-

p
ro
vi
d
er

p
ar
tn
er
sh
ip

w
as
w
el
l

re
ce
iv
ed

an
d
en

h
an
ce
d

ac
ce
p
ta
b
ili
ty

o
f
th
e

in
te
rv
en
ti
o
n

Er
w
in

et
al
.2
01
6
[7
1]

U
SA
Fo
cu
s
g
ro
u
p
s
Pe
d
ia
tr
ic
s—

as
th
m
a

20
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n
D
ev
el
o
p
ed
n
ew

p
ro
to
co
l
an
d

to
o
lf
o
r
p
at
ie
n
t
d
is
ch
ar
g
e

C
o
lla
b
o
ra
ti
ve

m
o
d
el

en
h
an
ce
d
th
e
p
er
ce
p
tio

n
o
f

ED
cl
in
ic
ia
n
s
as

p
ar
tn
er
s
in

as
th
m
a
co
n
tr
o
l

In
te
rv
ie
w
s
Su
rv
ey
s

Fa
ct
o
r
20
02

[5
7]

U
SA
Fo
cu
s
g
ro
u
p
s

Su
b
st
an
ce

u
se
rs

29
C
o
-d
es
ig
n

D
ev
el
o
p
m
en

t
o
f
a
“s
u
rv
iv
al

g
u
id
e”

to
im
p
ro
ve

ac
ce
ss

to
tr
ea
tm

en
t
C
re
at
ed

an
d
m
ai
n
ta
in
ed

th
e

p
ar
ti
ci
p
at
io
n
o
f

u
se
rs
in

al
l

as
p
ec
ts
o
f

g
u
id
e

d
ev
el
o
p
m
en
t

Fe
rr
ei
ra
-P
in
to

19
95

[5
8]

M
ex
ic
o

In
te
rv
ie
w
s

C
o
m
m
u
n
it
y
H
IV
/A
ID
S

p
re
ve
n
ti
o
n

10
5

C
o
-d
es
ig
n
D
ev
el
o
p
m
en
t
an
d

im
p
le
m
en

ta
ti
o
n
o
f

p
re
ve
n
ti
o
n
p
ro
g
ra
m

In
cr
ea
se
d
se
lf-
ef
fic
ac
y
an
d

se
lf-
es
te
em

o
f
co
m
m
u
n
it
y

p
ar
tn
er
s,
b
ey
o
n
d
th
e

p
ro
g
ra
m
’s

o
u
tc
o
m
es

Q
u
es
ti
o
n
n
ai
re
s

Bombard et al. Implementation Science (2018) 13:98 Page 7 of 22

T
a
b
le
1
O
ve
rv
ie
w
o
f
p
at
ie
n
t
en
g
ag
em
en
t
st
u
d
ie
s
to
im
p
ro
ve
q
u
al
it
y
o
f
ca
re
(C
o
n
tin
u
ed
)
St
u
d
y
C
o
u
n
tr
y
In
te
rv
en
ti
o
n
Ty
p
e
o
f
se
rv
ic
e
Sa
m
p
le
si
ze
Le
ve
l
o
f
en
g
ag
em
en
t
O
u
tc
o
m
es
o
n
q
u
al
it
y
o
f
ca
re
Im
p
ac
t
o
n
in
st
it
u
tio
n

Fi
tz
g
er
al
d
20
11

[7
8]

U
K

“S
er
io
u
s

g
am

e”
M
en

ta
l
h
ea
lt
h
se
rv
ic
es

25
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n

U
se
r
re
-d
es
ig
n
o
f
la
yo
u
t
an
d

fu
rn
is
h
in
g
s
o
f
a
n
ew

se
rv
ic
e

u
n
it;

d
es
ig
n
o
f
a

m
ed

ic
at
io
n

d
is
p
en

si
n
g
sy
st
em

A
ch
ie
ve
d
fle
xi
b
ili
ty

an
d

in
cl
u
si
vi
ty

th
ro
u
g
h
a
g
am

e
fo
rm

Fr
az
ie
r
20
07

[4
2]

U
SA

Sc
h
o
o
l-b

as
ed
p
ro
g
ra
m
M
en
ta
l
h
ea
lt
h
se
rv
ic
es
N
o
t
d
is
cu
ss
ed
C
o
-d
es
ig
n
D
ev
el
o
p
ed

a
sc
h
o
o
l-b

as
ed
m
en

ta
l

h
ea
lt
h
se
rv
ic
e

p
ro
g
ra
m

w
it
h
ac
ti
ve

en
g
ag
em

en
t
o
f
co
m
m
u
n
it
y

p
ar
tn
er
s
an
d
cl
in
ic
ia
n
s

A
ch
ie
ve
d
su
cc
es
sf
u
l

co
lla
b
o
ra
ti
o
n

b
et
w
ee
n

co
m
m
u
n
it
y
an
d
cl
in
ic
ia
n
s
o
n

cu
rr
ic
u
lu
m

d
ev
el
o
p
m
en
t

G
ib
so
n

20
05

[6
0]

U
K

In
te
rv
ie
w
s,

q
u
es
ti
o
n
n
ai
re
s,

w
o
rk
sh
o
p
s

Pe
d
ia
tr
ic
o
n
co
lo
g
y

40
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n
Re
co
m
m
en

d
at
io
n
s
fo
r

st
ru
ct
u
re

an
d

p
ro
ce
ss
es

o
f
se
rv
ic
e
N
o
t
d
is
cu
ss
ed

G
o
d
fr
ey

et
al
.2
01
3
[7
3]

U
K

Fa
ci
lit
at
ed

w
o
rk
sh
o
p
s,

fo
cu
s
g
ro
u
p
s,
in
te
rv
ie
w
s
A
cu
te

ca
re
—
d
el
iri
u
m

3
d
el
iri
u
m

p
re
ve
n
ti
o
n

te
am

m
em

b
er
s,

u
n
sp
ec
ifi
ed

in
te
rv
ie
w
s
C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n

Pr
ev
en

ti
o
n
o
f
D
el
iri
u
m

(P
O
D
)

p
ro
g
ra
m

En
h
an
ce
d
cu
lt
u
re

o
f
ca
rin

g
am

o
n
g
st
af
f

H
al
l
20
11

[9
4]

U
K

In
te
rv
ie
w
s,
m
o
d
el
in
g

ex
er
ci
se

C
an
ce
r

18
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n
D
ev
el
o
p
ed

a
m
o
d
el
fo
r

sh
ar
ed

ca
re

o
f
se
co
n
d
ar
y

ca
n
ce
r
fo
llo
w
-u
p
w
it
h

g
en

er
al
p
ra
ct
it
io
n
er
s

su
p
p
o
rt
ed

b
y
sp
ec
ia
lis
ts

In
vo
lv
ed

se
rv
ic
e
u
se
rs
an
d

d
el
iv
er
er
s’
ex
p
er
ie
n
ce
s
in

a
m
o
d
el
in
g
ex
er
ci
se

H
ig
g
in
s
et

al
.2
01
6
[9
5]

Ire
la
n
d

A
ct
io
n
re
se
ar
ch

g
ro
u
p
,f
o
cu
s
g
ro
u
p
s,

p
ee
r
fa
ci
lit
at
o
r,

st
ee
rin

g
co
m
m
it
te
e
M
en
ta
l
h
ea
lt
h
se
rv
ic
es

30
u
se
rs
vi
a
fo
cu
s

g
ro
u
p
s,
21

fa
m
ily

m
em

b
er
s
vi
a
fo
cu
s

g
ro
u
p
s
C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n
D
ev
el
o
p
ed

a
p
ee
r
an
d

cl
in
ic
ia
n
-le
d
ed

u
ca
ti
o
n
p
ro
g
ra
m
N
o
t
d
is
cu
ss
ed

H
o
p
ki
n
s
an
d
N
ei
m
ie
c

20
06

[5
2]

U
K
In
te
rv
ie
w
s
q
u
es
ti
o
n
n
ai
re
s

H
o
m
e
tr
ea
tm

en
t
se
rv
ic
es

70
C
o
-d
es
ig
n

Se
rv
ic
e
im

p
ro
ve
m
en

t
su
rv
ey

Sh
ar
ed

an
d
n
eu
tr
al
iz
ed

p
o
w
er

to
in
cr
ea
se

in
cl
u
si
ve
n
es
s

th
ro
u
g
h
u
se

p
ar
ti
ci
p
at
io
n
in

p
ro
ce
ss

Ie
d
em

a
et

al
.2
01
0
[3
9]

A
u
st
ra
lia

In
te
rv
ie
w
s

Em
er
g
en

cy
h
ea
lt
h

se
rv
ic
es

40
C
o
-d
es
ig
n

Re
co
m
m
en
d
at
io
n
s
fo
r

im
p
ro
vi
n
g
p
ro
ce
ss
es

an
d

fa
ci
lit
ie
s
in

th
e
ED

C
re
at
ed

a
d
el
ib
er
at
iv
e
sp
ac
e

fo
r
p
at
ie
n
ts
,c
lin
ic
ia
n
s,
an
d

st
af
f
to

d
is
cu
ss
p
er
so
n
al

ex
p
er
ie
n
ce
s
an
d
d
es
ig
n

p
ro
ce
ss
es

to
am

el
io
ra
te

is
su
es

.P
ro
ce
ss

d
ev
el
o
p
ed
n
ew

co
m
p
et
en

ci
es

an
d
sk
ill
s

am
o
n
g
p
ar
ti
ci
p
an
ts

Jo
n
es

et
al
.2
00
8
[6
2]

U
K
Fo
cu
s
g
ro
u
p
s,
in
te
rv
ie
w
s,

w
o
rk
g
ro
u
p
s

St
ro
ke

se
rv
ic
es

92
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n

In
fo
rm

at
io
n
p
ac
ka
g
e
fo
r

p
at
ie
n
ts
,r
ec
o
m
m
en

d
at
io
n
s

fo
r
im

p
ro
ve
m
en

ts
to

re
h
ab
ili
ta
ti
o
n
p
ro
g
ra
m
;

p
rio

rit
iz
at
io
n
o
f
h
ea
lt
h
ca
re

is
su
es

fo
r
st
ro
ke

p
at
ie
n
ts
an
d

d
ev
el
o
p
m
en

o
f
se
rv
ic
es

A
ch
ie
ve
d
m
ea
n
in
g
fu
l
u
se
r

p
ar
ti
ci
p
at
io
n
in
se
rv
ic
e
d
ev
el
o
p
m
en

t
th
ro
u
g
h

ex
te
rn
al
fa
ci
lit
at
io
n

Bombard et al. Implementation Science (2018) 13:98 Page 8 of 22

20
10

[9
6]

U
SA

A
d
vi
so
ry

m
ee
ti
n
g
s

in
te
rv
ie
w
s,
u
se
r
te
st
in
g

Sm
o
ki
n
g
ce
ss
at
io
n
fo
r

d
ea
f
in
d
iv
id
u
al
s

10
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n
D
ev
el
o
p
ed
an
d
te
st
ed

an
In
te
rn
et
-b
as
ed

sm
o
ki
n
g

ce
ss
at
io
n
in
te
rv
en
tio
n
fo
r
d
ea
f

in
d
iv
id
u
al
s
in
co
n
su
lta
tio

n
w
ith

m
em

b
er
s
o
f
th
e
d
ea
f

co
m
m
u
n
ity

In
vo
lv
ed

se
rv
ic
e
u
se
rs
in

al
l

p
h
as
es

o
f
d
ev
el
o
p
m
en

t
an
d
te
st
in
g

Lo
ft
er
s
et

al
.2
01
5
[4
3]

C
an
ad
a

C
o
m
m
u
n
it
y
ad
vi
so
ry

g
ro
u
p
,c
o
m
m
u
n
it
y

w
o
rk
sh
o
p
s,
co
n
ce
p
t

m
ap
p
in
g

,i
n
te
rv
ie
w
s

C
an
ce
r
sc
re
en

in
g

24
vi
a
co
n
ce
p
t

m
ap
p
in
g
C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n
D
ev
el
o
p
ed

a
cu
lt
u
ra
lly

ap
p
ro
p
ria
te

ca
n
ce
r
sc
re
en

in
g
p
ro
g
ra
m

fo
r
So
u
th

A
si
an

co
m
m
u
n
it
y
d
el
iv
er
ed

vi
a

co
m
m
u
n
it
y
o
rg
an
iz
at
io
n
s

In
cr
ea
se
d
ca
p
ac
it
y
to

im
p
le
m
en

t
ev
id
en

ce
-b
as
ed

in
te
rv
en
ti
o
n
s

Lo
rd

et
al
.1
99
4
[6
3]

C
an
ad
a
D
o
cu
m
en

t
an
al
ys
is
,

co
n
su
lt
at
io
n
fe
et
in
g
s,

fo
cu
s
g
ro
u
p
s,
q
u
es
ti
o
n
n
ai
re
M
en
ta
l
h
ea
lt
h
se
rv
ic
es
N
o
t
d
is
cu
ss
ed
C
o
-d
es
ig
n

Im
p
ro
ve
d
se
rv
ic
es

A
ch
ie
ve
d
o
rg
an
iz
at
io
n
al

cu
lt
u
re

ch
an
g
e

an
d
p
at
ie
n
t

re
p
re
se
n
ta
ti
o
n
o
n
b
o
ar
d
b
y

b
ro
ad
en

in
g
p
o
w
er

an
d

co
n
tr
o
l
th
ro
u
g
h

st
ak
eh

o
ld
er

in
vo
lv
em
en
t

M
ac
d
o
n
n
el
l
et

al
.

20
13

[4
5]

C
an
ad
a

Br
ai
n
st
o
rm

in
g
,

fa
ci
lit
at
ed

d
is
cu
ss
io
n
s

Pe
d
ia
tr
ic
s—

n
eo

n
at
al

in
te
n
si
ve

ca
re
3
C
o
-d
es
ig
n
D
ev
el
o
p
ed

a
fa
m
ily

in
te
g
ra
te
d
ca
re

p
ro
g
ra
m

Bu
ilt

p
o
si
ti
ve

re
la
ti
o
n
sh
ip

b
et
w
ee
n
u
se
rs
an
d
st
af
f

M
ac
N
ei
ll
20
09

[9
7]

U
K
In
te
rv
ie
w
s,

o
b
se
rv
at
io
n
s

Pe
d
ia
tr
ic
s

29
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n

N
ew

m
o
d
el
o
f
p
ar
ti
ci
p
at
io
n

to
im
p
ro
ve

p
at
ie
n
t-
st
af
f

re
la
ti
o
n
sh
ip
an
d
p
at
ie
n
t

u
n
d
er
st
an
d
in
g
o
f
p
ro
g
ra
m

G
re
at
er

in
vo
lv
em

en
t
o
f
u
se
rs

th
ro
u
g
h
d
em

o
cr
at
ic
p
ro
ce
ss

o
f
p
ar
ti
ci
p
at
io
n
,t
h
o
u
g
h
u
se
rs

ad
o
p
te
d
a
p
as
si
ve

ro
le

M
en
d
en

h
al
l
et

al
.
20
10

[7
7]

U
SA
C
o
lla
b
o
ra
ti
ve

ed
u
ca
ti
o
n
al
p
ro
g
ra
m

D
ia
b
et
es

52
C
o
n
su
lt
at
iv
e
to

C
o
-d
es
ig
n
C
o
lla
b
o
ra
ti
ve
d
es
ig
n
o
f
a

“F
am

ily
Ed
u
ca
ti
o
n
D
ia
b
et
es

Se
rie
s”
p
ro
g
ra
m
,w

h
ic
h
d
em
o
n
st
ra
te
d
im
p
ro
ve
d
o
u
tc
o
m
es
A
ch
ie
ve
d
co
lla
b
o
ra
ti
o
n

b
et
w
ee
n
el
d
er
s
an
d
p
ro
vi
d
er
s

in
d
es
ig
n

an
d

im
p
le
m
en

ta
ti
o
n
o
f
p
ro
g
ra
m

th
ro
u
g
h
u
se

o
f
ta
lk
in
g
ci
rc
le
s,

st
o
ry
te
lli
n
g
,d

an
ce
,s
h
ar
ed

m
ea
ls
,a
n
d
ac
ti
ve

ro
le
in

in
te
rv
en
ti
o
n

M
u
rp
h
y
et

al
.2
01
5
[4
4]

Ire
la
n
d

Q
u
al
it
y
im

p
ro
ve
m
en

w
o
rk
in
g

te
am

s
M
en

ta
l
h
ea
lt
h
se
rv
ic
es

10
C
o
-d
es
ig
n

En
h
an
ce
d
ex
p
er
ie
n
ce
s
o
f

ca
re

fo
r
u
se
rs
re
fe
rr
ed

to
co
m
m
u
n
it
y
m
en

ta
l
h
ea
lt
h
se
rv
ic
es

A
ck
n
o
w
le
d
g
em

en
t
th
at

u
se
r/

fa
m
ily
in
vo
lv
em

en
t
n
ee
d
s
to

g
o
b
ey
o
n
d
in
vo
lv
em

t
to

tr
u
e
co
-p
ro
d
u
ct
io
n
ex
er
ci
se
s

p
er
ce
iv
ed

as
m
ea
n
in
g
fu
l
b
y

al
l
p
ar
ti
ci
p
at
in
g
st
ak
eh

o
ld
er
s

O
w
en

s
20
11

[5
6]

U
K

W
o
rk
sh
o
p
s

M
en
ta
l
h
ea
lt
h
se
rv
ic
es

12
C
o
-d
es
ig
n

D
ev
el
o
p
ed

a
te
xt
-b
as
ed

in
te
rv
en

ti
o
n
fo
r
p
at
ie
n
ts
w
h
o

se
lf-
h
ar
m

In
vo
lv
ed
u
se
rs
in

th
e
d
es
ig
n

p
ro
ce
ss
,w

h
ic
h
ch
an
g
ed

th
e

n
at
u
re

o
f
th
e
in
te
rv
en

ti
o
n

d
ra
m
at
ic
al
ly

Pi
lg
rim

an
d
W
al
d
ro
n

19
98

[5
9]

U
K
C
o
n
su
lt
at
io
n

m
ee
ti
n
g
s,

o
b
se
rv
at
io
n
s
M
en
ta
l
h
ea
lt
h
se
rv
ic
es

14
C
o
-d
es
ig
n

Im
p
ro
ve
d
se
rv
ic
e:
ex
te
n
d
ed

o
p
en

in
g
h
o
u
rs
,e
m
p
lo
ye
d
a

m
en

ta
l
h
ea
lt
h
ad
vo
ca
te
,

p
u
b
lis
h
ed

an
in
fo
rm

at
io
n

b
o
o
kl
et

A
ch
ie
ve
d
d
ire
ct

n
eg

o
ti
at
io
n
s

fo
r
ch
an
g
e
b
et
w
ee
n
u
se
rs

an
d
p
ro
fe
ss
io
n
al
s

Bombard et al. Implementation Science (2018) 13:98 Page 9 of 22

Re
ev
e
et

al
.2
01
5
[7
4]

A
u
st
ra
lia
Fo
cu
s
g
ro
u
p
s,
w
o
rk
sh
o
p
s
Pr
im
ar
y
ca
re

6
C
o
-d
es
ig
n

G
en

er
at
ed

n
ew

d
el
in
ea
ti
o
n

o
f
ro
le
s
an
d
re
sp
o
n
si
b
ili
ti
es

b
et
w
ee
n
an

A
b
o
rig

in
al

co
m
m
u
n
it
y-
co
n
tr
o
lle
d
h
ea
lt
h

se
rv
ic
e
an
d
lo
ca
l
A
u
st
ra
lia
n

h
ea
lt
h
se
rv
ic
e

Tr
u
st
in
g
re
la
ti
o
n
sh
ip

b
et
w
ee
n

co
m
m
u
n
it
y
an
d
p
ro
vi
d
er
s
as

a
re
su
lt
o
f
ex
te
n
si
ve

co
m
m
u
n
it
y
co
n
su
lt
at
io
n

Ro
se

20
03

[9
8]

U
K
Q
u
es
ti
o
n
n
ai
re
M
en
ta
l
h
ea
lt
h
se
rv
ic
es

22
1

C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n

Im
p
ro
ve
d
co
o
rd
in
at
io
n
o
f

ca
re
g
en

er
al
ly
lin
ke
d
to

im
p
ro
ve
d
u
se
r
sa
ti
sf
ac
ti
o
n

Ve
ry

fe
w

u
se
rs
w
er
e
in
vo
lv
ed

o
r
aw

ar
e
o
f

th
e
n
ew

co
o
rd
in
at
io
n
p
ro
ce
ss

Sw
ar
b
ric
k
et

al
.2
00
6
[5
5]

U
SA
G
ro
u
p
m
ee
ti
n
g
s
M
en
ta
l
h
ea
lt
h
se
rv
ic
es
N
o
t
d
is
cu
ss
ed
C
o
-d
es
ig
n
Im
p
le
m
en

ta
ti
o
n
o
f
th
e

Re
co
ve
ry

N
et
w
o
rk

Pr
o
g
ra
m
,a

u
se
r-
le
d
w
el
ln
es
s
an
d
re
co
v-

er
y
tr
ai
n
in
g
p
ro
je
ct

Es
ta
b
lis
h
ed

a
co
lla
b
o
ra
ti
ve

p
ar
tn
er
sh
ip

b
et
w
ee
n
p
ee
r

ed
u
ca
ti
o
n
an
d
h
o
sp
it
al
st
af
f

vi
a
u
se
r
tr
ai
n
in
g

Th
o
m
so
n
et

al
.2
01
5
[6
8]

U
K

“F
u
tu
re
”
g
ro
u
p
s

M
u
lt
ip
le
sc
le
ro
si
s

5
C
o
-d
es
ig
n

Re
co
n
ce
p
tu
al
iz
ed

se
rv
ic
e
fo
r

o
u
tp
at
ie
n
ts

C
re
at
ed

a
p
o
si
ti
ve

w
o
rk
in
g

en
vi
ro
n
m
en

t
w
it
h
m
u
tu
al

re
sp
ec
t
an
d
in

eq
u
al

p
ar
tn
er
sh
ip

To
d
d
et

al
.2
00
0
[4
7]

U
K
In
te
rv
ie
w
s

In
te
lle
ct
u
al

d
is
ab
ili
ty

se
rv
ic
es
N
o
t
d
is
cu
ss
ed
C
o
-d
es
ig
n

In
flu
en

ce
d
im

p
le
m
en
ta
ti
o
n
o
f

se
rv
ic
e
st
ra
te
g
y

A
ch
ie
ve
d
a
sh
ift

in
th
in
ki
n
g
,

co
lla
b
o
ra
ti
o
n
an
d
co
n
su
m
er

p
ar
ti
ci
p
at
io
n
in
p
la
n
n
in
g

To
lly
fie
ld

20
14

[5
3]

U
K

C
o
-d
es
ig
n
m
ee
ti
n
g
s

A
cu
te

ca
re
—
cr
it
ic
al

ca
re

19
C
o
-d
es
ig
n

M
u
lt
ip
le
in
-u
n
it
q
u
al
it
y

im
p
ro
ve
m
en

t
in
it
ia
ti
ve
s

St
af
f
re
co
n
n
ec
te
d
co
re

va
lu
es

o
f
ca
rin

g
an
d
co
m
p
as
si
o
n

To
o
ke

20
13

[4
9]

U
K

Se
rv
ic
e
u
se
r
re
vi
ew

p
an
el
s
D
em

en
ti
a

14
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n
D
ev
el
o
p
m
en
t
o
f

o
rg
an
iz
at
io
n
al
p
rio

rit
ie
s
an
d

p
ro
ce
ss
es

fo
r
p
at
ie
n
ts
w
it
h

d
em

en
ti
a,
d
ev
el
o
p
m
en

t
o
f

ev
al
u
at
io
n
to
o
ls

En
h
an
ce
d
u
n
d
er
st
an
d
in
g
o
f

ef
fe
ct
iv
e
w
ay
s
fo
r
st
af
f
to

co
m
m
u
n
ic
at
e
w
it
h
u
se
rs

Va
n
St
aa

et
al
.2
01
0
[6
6]

N
et
h
er
la
n
d
s

In
te
rv
ie
w
s
at

a
d
is
co

p
ar
ty

A
cu
te
ca
re
fo
r

ch
ro
n
ic
al
ly
ill
p
at
ie
n
ts

34
C
o
-d
es
ig
n
to

co
n
su
lt
at
iv
e

Re
co
m
m
en
d
at
io
n
s
fo
r

en
g
ag
in
g
yo
u
th

in
d
es
ig
n

an
d
ev
al
u
at
io
n
o
f
h
ea
lt
h

se
rv
ic
es

In
vo
lv
in
g
u
se
rs
w
as

fe
as
ib
le

an
d
ap
p
re
ci
at
ed

b
y
u
se
rs
b
u
t

d
id

n
o
t
im

p
ro
ve
q
u
al
it
y

W
al
sh

an
d
H
o
st
ic
k

20
05

[9
9]

U
K
Q
u
es
ti
o
n
n
ai
re
M
en
ta
l
h
ea
lt
h
se
rv
ic
es
10
C
o
n
su
lt
at
iv
e
to
co
-d
es
ig
n

Im
p
ro
ve
d
ca
re

fa
ci
lit
y,

d
ev
el
o
p
m
en

t
o
f
se
rv
ic
e

st
ra
te
g
y,
an
d
ca
re

g
u
id
e

A
ch
ie
ve
d
u
se
r
o
w
n
er
sh
ip

th
ro
u
g
h
ex
te
rn
al
fa
ci
lit
at
io
n

W
ei
n
st
ei
n
20
06

[4
6]

U
K
D
o
cu
m
en
t
an
al
ys
is
,
m
ee
ti
n
g
s,
q
u
es
ti
o
n
n
ai
re
M
en
ta
l
h
ea
lt
h
se
rv
ic
es

72
C
o
n
su
lt
at
iv
e
to

co
-d
es
ig
n

Pl
an

to
im
p
ro
ve
se
rv
ic
e
d
el
iv
er
y

To
p
-d
o
w
n
ap
p
ro
ac
h
o
f
th
e

fir
st
ca
se

re
su
lt
ed

in
le
ss

u
se
r

o
w
n
er
sh
ip
,w

h
er
ea
s
th
e

co
lla
b
o
ra
ti
ve
,u
se
r-
le
d

ap
p
ro
ac
h
o
f
th
e
se
co
n
d
ca
se

le
d
to

th
e
n
ew

ap
p
ro
ac
h
to

se
ek
in
g
u
se
rs
’v
ie
w
s
an
d

ac
h
ie
ve
d
h
ig
h
er

re
sp
o
n
se

Bombard et al. Implementation Science (2018) 13:98 Page 10 of 22

an
d
Ba
rn
es

19
93

[6
1]

U
K
C
o
n
su
lt
at
io
n

m
ee
ti
n
g
s,
p
at
ie
n
t

co
u
n
ci
l,
q
u
es
ti
o
n
n
ai
re

M
en

ta
l
h
ea
lt
h
an
d

d
is
ab
ili
ty
se
rv
ic
es
N
o
t
d
is
cu
ss
ed
C
o
-d
es
ig
n

Im
p
ro
ve
d
ac
ce
ss

to
se
rv
ic
es
:

co
m
m
it
m
en

t
to

ad
d
re
ss

is
su
e,
em

p
lo
ym

en
t
su
p
p
o
rt

u
n
it
cr
ea
te
d

In
cr
ea
se
d
u
se
rs
’v
o
ic
e
in

th
ei
r

ca
re
,w

h
ic
h
im

p
ro
ve
d
th
e

se
n
si
ti
vi
ty

o
f
se
rv
ic
es

to
in
d
iv
id
u
al
n
ee
d
s
an
d

in
fo
rm

at
io
n
ab
o
u
t
se
rv
ic
es

X
ie
et

al
.2
01
5
[5
4]

U
SA

In
te
rv
ie
w
s,
m
ee
ti
n
g
s

A
cu
te
ca
re

1
p
ar
en

t,
14

st
ak
eh
o
ld
er
s
C
o
-d
es
ig
n
D
ev
el
o
p
ed

ch
ec
kl
is
t
fo
r

fa
m
ily
-c
en

te
re
d
ro
u
n
d
s

C
re
at
ed

b
u
y-
in

fo
r
th
e
fa
m
ily
–

ce
n
te
re
d
ro
u
n
d
s
p
ro
ce
ss

an
d

n
ee
d
fo
r
m
u
tu
al

u
n
d
er
st
an
d
in
g

Bombard et al. Implementation Science (2018) 13:98 Page 11 of 22

effectiveness of the engagement. These strategies were
thematically grouped as techniques to enhance (1) de-
sign, (2) recruitment, (3) involvement, (4) creating a re-
ceptive context, and (5) leadership actions. Here, we
describe the strategies and contextual factors that enabled
optimal patient engagement (see also Additional file 3:
Table S1).

Techniques to enhance design of engagement
In designing engagements, several studies pointed to the
importance of clarifying the objectives, roles, and expec-
tations of the engagement for patients/carers [40–45].
Approaches that gave users specific roles or engaged
them in a formal structure such as a steering committee
[45] or that enabled patients to set the agenda, develop

Table 2 Summary of facilitators and barriers of patient engagement

Facilitators Barriers

Design of engagement

1. Techniques for enhancing patient/carer input

● Enable patients or carers to set the agenda
● Enable patients or carers to participate in all/most stages of the
research (participatory action research)

● Include higher proportions of patients versus providers to
enhance patient voices

● Offer flexibility in the levels and approaches of involvement
● Build in reward mechanisms such as feedback and evaluation
● Set opportunities for interaction at regular frequencies

● Overly complex discussions
● Onerous, time-intensive involvement
● Inclusion of:
○ A disproportionate number of patients compared to
providers

○ Providers who previously cared for the patients in the
meeting/committee

○ Groups of individuals with existing hierarchical structures

2. Creating a receptive context

● Use of democratic dialog to build consensus
● Use of external facilitation and trained facilitators
● Conduct training sessions prior to engagements to clarify roles,
objectives, develop skills, increase sensitivity to cultural or
community issues and reduce power imbalances

● Maintain flexibility in aims, design, and outcomes in response
to patients’ input

● Enable time to develop strong and trusting relationships
● Create environment where participants are able to
communicate in the language of their preference

● Lack of clarity on:
○ Roles
○ Objectives
○ Responsibilities

3. Leadership actions

● Secure institutional commitment and sponsorship for
engagement

● Involve institutional leadership
● Conduct engagements before decision have been made
● Establish mechanisms to act on issues raised and to continue
involvement

● Demonstrate progress occurring between meetings

● Engagements conducted by consultative groups, not
decision-makers

● Lack of response or plans to address issues raised
● Lack of follow-up with patients after their participation
● Policies and procedures misaligned with participation,
recommendations or outcomes

Sampling of participants

1. Techniques for enhancing patient/carer input

● Have patients conduct interviews with fellow patients, when
possible

● Strive for a wide representation of patients at all stages
● Identify and recruit users through providers, existing users,
networks

● Offer incentives (monetary and other), stipends, reimbursement
of expenses

● Provider- or patient-led recruitment can introduce biases
● Inclusion of self-selected, participants:
○ Confident patients
○ Those who have fewer symptoms or family care duties

● Inclusion of proxy groups:
○ Parents to represent children
○ Carers to represent patients

● Ethical concerns regarding recruitment and consent of
participants with intellectual or physical disabilities

2. Creating a receptive context

● Consider setting: engage patients at home, in their facilities or
in environments outside where services are delivered to
increase participation and comfort

● Lack of participant commitment
● Lack of participant confidence
● Inclusion of providers:
○ Who are skeptical towards involving patients
○ Who feel threatened by devolving power
○ Whose behavior does not promote user participation

3. Leadership actions

● Emphasize to patients that there is organizational
commitment/sponsorship of the engagement of patients

Bombard et al. Implementation Science (2018) 13:98 Page 12 of 22

shared mission and purpose statements and partici-
pate in all/most stages of the planning, administra-
tion, and evaluation made participants feel
comfortable with the team and process, maintained
patient involvement throughout the course of the
process, and improved the quality of outcomes [41,
45–50]. These techniques occurred in mental health,
HIV, and pediatric service settings where patients
were engaged to improve access to, and quality of,
care or promote a culture change in the develop-
ment and delivery of services.

An important strategy used in pediatric, diabetes, and
home care settings was holding training sessions to pre-
pare staff and patients, which provided clarity on roles
and responsibilities, helped patients or carers understand
how they could best contribute, sensitized participants
to the contextual and cultural issues, and increased pa-
tients’ confidence and commitment to the engagement
process [41, 51, 52]. Training also offered the benefit of
building positive relationships between users, facilitators,
and staff [43, 45, 49, 53, 54], which also served to medi-
ate a key barrier identified: providers’ skepticism towards

Table 3 Quality of care outcomes and levels of engagement

Type of outcomes Level of engagement Studies

Co-design Consultative to co-design

Education or tool development

Information packages for patients,
peers, and providers

Consultative to co-design,
co-design

Pilgrim and Waldron 1998 [59],
Factor 2002 [57]

Jones 2008 [62], Carlson and
Rosenqvist 1990 [51], Buck 2004
[76], Ennis 2014 [70], Erwin
2016 [71]

Service improvement surveys

Co-design, consultative to
co-design

Hopkins and Neimiec 2006 [52],
Brooks 2008 [64], Xie 2015 [54]

Tooke 2013 [49]

Informed policy or planning products

Clinical care models Consultative to co-design Hall 2011 [94]

Service/care strategies Co-design Brooks 2008 [64]

User involvement models Co-design, consultative to
co-design

Coad 2008 [41] MacNeill 2009 [97], Van Staa 2010
[66], Barnes and Wistow 1994 [40]

Service policy implementation Co-design Todd 2000 [47]

Plans or recommendations to improve
service delivery and care

Co-design, consultative to
co-design

Iedema 2010 [39] Jones 2008 [62], Weinstein 2006
[46], Walsh and Hostick 2005 [99],
Gibson 2005 [60], Buck 2004 [76],
Cawston 2007 [69]

Enhanced care process or service delivery

Extended opening hours

Co-design Pilgrim and Waldron 1998 [59]

Employment of a dedicated mental
health advocate

Co-design Pilgrim and Waldron 1998 [59]

Improved/developed care facilities,
services, programs, or intervention

Co-design, consultative to
co-design

Ferreira-Pinto 1995 [58], Frazier
2007 [42], Barnes 2000 [75],
Owens 2011 [56], Sawbrick
2006 [55],
Lord 1998 [63], Coad 2008 [41],
Fitzgerald 2011 [78], Macdonnell
2013 [45], Bone 2013 [93], Berg
2015 [50], Thomson 2015 [68],
Reeve 2015 [74], Tolleyfield 2014
[53], Murphy 2015 [44]

Fitzgerald 2011 [78], Jones,
2010 [96], Mendenhall 2010 [77],
Walsh and Hostick 2005 [99],
Carlson and Rosenqvist 1990 [51],
Barnes and Wistow 1994 [40],
Cawston 2007 [69], Rose 2003 [98],
Godfrey 2013 [73], Blickem 2014
[92], Acri 2014 [65], Higgins 2016
[95], Lofters 2015 [43]

Improved access to service

Co-design Wistow and Barnes 1993 [61]

Creation of an employment
support unit

Co-design Wistow and Barnes 1993 [61]

Creation of new services Co-design Enriquez et al. 2010 [67]

Improved governance

Patient representation on board Co-design Lord et al. 1998 [63]

Auditing policy and frameworks Co-design Brooks 2008 [64]

Commitment to improve services Co-design Wistow and Barnes 1993 [61]

Organizational culture change Co-design Lord et al. 1998 [63]

Bombard et al. Implementation Science (2018) 13:98 Page 13 of 22

engaging patients and devolving power to them [42, 55].
Therefore, these techniques helped to create a level play-
ing field and support staff in their efforts to be partners.

Techniques to enhance representation
With respect to sampling and recruitment, several studies
stressed the importance of ensuring diversity and repre-
sentation consistent with the broader population across
different professional backgrounds and skills [43, 45, 54].
These studies endorsed recruiting patients through pro-
viders, [42] existing patients [56], and those with broader
networks or previous working relationships with staff
[45, 54, 57, 58]. These techniques proved useful in en-
gaging patients in the context of HIV/AIDS prevention,
interventions to reduce repetition of self-harm and sub-
stance use, and identifying barriers to mental health ser-
vices [42, 56–58]. One caveat with this approach is that it
needs to be weighed against the potential for introducing
biases or including self-selected participants. Offering
stipends, financial compensation (e.g., child care, transpor-
tation), or other incentives encouraged participation
[42, 45, 50, 54, 55, 58, 59]. One study in the HIV set-
ting used creative techniques to incentivize participation
beyond monetary incentives, such as counseling, access to
medical care, and granting diplomas [58].

Techniques to enhance involvement
Several authors also endorsed flexible approaches for in-
volving patients [45, 49, 53, 54]. For example, Gibson et
al. [60] used peer reporter interviews (where patient
pairs interviewed each other), headline generation
(where phrases were created to capture important is-
sues), group discussion (using a Who, Why, When,
What, How structure), a written exercise, and question-
naires for non-attendees to find out what youth would
like from their follow-up pediatric oncology services.
Other techniques identified in studies were the inclusion
of higher proportions of patients compared to providers
or staff to give patients a stronger voice in the discussion
and process [61] and building in debriefing to provide
feedback on how suggestions were acted upon to in-
crease the accuracy of the findings and offer an oppor-
tunity for additional input. These techniques proved
useful in engaging patients to prioritize stroke service is-
sues and document the process of change of a mental

Table 4 Characteristics of patient engagement studies

Study characteristics (n = 48) Number Percent

Country

UK 26 54

USA 11 23

Canada 3 6

Australia 2 4

Ireland 2 4

Mexico 1 2

Sweden 1 2

Netherlands 1 2

Norway 1 2

Type of service

Mental health 17 35

General health/community/primary care 5 10

Pediatric/maternity care 6 13

Acute care/emergency 6 13

Cancer 3 6

HIV/AIDS 3 6

Diabetes 2 4

Smoking cessation/substance abuse 2 4

Physical and intellectual disability 1 2

Elderly/home treatment 1 2

Stroke 1 2

Multiple sclerosis 1 2

Design

Qualitative 27 56

Mixed methods 13 27

Quantitative 3 6

Other 5 11

Level of engagement

Co-design 24 50

Consultative to co-design 24 50

Type of quality of care outcome*

Discrete product

Education/tool development 11 23

Enhanced policy or planning document 15 31

Care process or structural outcome

Enhanced care process or service delivery 35 73

Enhanced governance 5 10

Evaluation of patient experiences of engagement process

Formal 12 25

Informal; anecdotal reports 11 23

None 25 52

Table 4 Characteristics of patient engagement studies
(Continued)

Study characteristics (n = 48) Number Percent

Evaluation of engagement methods

Yes 25 52

No 23 48

*The total outcomes exceed the number of studies because some studies
reported more than one outcome

Bombard et al. Implementation Science (2018) 13:98 Page 14 of 22

health organization [62, 63]. Others built in regular up-
dates to patient support group to elicit more views,
thereby broadening the reach and involvement of pa-
tients and providing opportunities to raise and discuss
issues of concern in informal settings [48, 54]. One cre-
ative technique was a buddy system for users/families to
ensure their participation at meetings and throughout
implementation/evaluation of a quality improvement
project in mental health services [44].

Techniques to create a receptive context
Several studies across general medicine, diabetes, mental
health, and emergency services highlighted the import-
ance of creating a receptive context by giving each of the
stakeholder groups equal say, using techniques such as
deliberation and democratic dialog, [39, 51] values and
beliefs exercise [48], and narratives to facilitate shared
understandings, generate consensus, or find common
ground [54, 64]. These techniques created a level playing
field and supported staff in their efforts to be partners.
Other studies focused on the empowerment and auton-
omy of users through “active citizenship” in an “egalitar-
ian spirit” [50, 65] which was found to foster a culture of
respect [54]. Ensuring that users had an equal voice
throughout all aspects of building the intervention was
found to help equalize the power differential that often
arises in professionally delivered services [65]. Finally,
location influenced participation—some studies held
consultation outside of the hospital setting such as a
disco to appeal to youth [66]. Others conducted meet-
ings in participants’ homes [58] and childcare commu-
nity sites [67].
External facilitation [39, 63] catalyzed receptive con-

texts that encouraged user involvement by creating a
positive working environment with mutual respect and
equal partnership [53]. Finally, attention was also paid to
the physical environment (e.g., cleanliness, chair ar-
rangement [53]) and use of physical props, and visual
mapping, which supported participants’ discussion and
interactions as well as demonstrated to service users the
importance of their contribution [68].

Leadership actions
A key facilitator of successful engagement was actions and
involvement by organizational leaders. This occurred in a
variety of ways including top-down approaches and at
community levels where local champions led initiatives or
were actively engaged to ensure their success. Top-down
approaches included institutional- or executive-level com-
mitment and sponsorship, which was readily apparent
across mental health, HIV, and pediatric care settings
[41, 44–46, 50, 63]. Having managers and executives
recognize and advocate for the importance of patient
involvement fostered a sense of empowerment and

commitment among patients and ensured organizational
sustainability of the engagement. This was a goal of two
mental health studies, where the senior level of a local au-
thority took a “top-down” approach to promote user in-
volvement, which resulted in a reported culture change
throughout the authority [40, 63]. This was highlighted in
one study’s “ideological and policy commitment to mean-
ingful involvement of people affected with HIV” as dem-
onstrated by ongoing contact with management and
executives and a head clinician open to changes that
would disturb traditional relationships and power dispar-
ities between service users and providers [50]. Leadership
action was also shown to help align the engagement find-
ings or recommendations and ensure that they are ad-
vanced within the organization’s relevant strategic plans
and policies in primary care [69]. Timing is also an im-
portant factor—ensuring that the engagement occurs
prior to decision-making, rather than providing input on
proposals to which services are already committed was
stressed in a number of studies [45]. Otherwise, the
engagement could run the risk of being perceived as
tokenistic by the users.

Outcomes of engaging patients to improve quality of
care
Discrete outcomes of improved quality of care
Most studies noted more than one type of outcome on
the quality of care, including enhanced care or service
delivery (n = 35), development of specific policy or plan-
ning documents (n = 15), and enhanced governance and
education or tool development (n = 5 and 11, respectively).
Examples of educational materials, tools, policy, and plan-
ning documents included evaluation tools [49], electronic
personal health records for mental health users [70], a
new tool for discharge [71], creation of models of care
[72], and organization priorities and processes [49]. Exam-
ples of care process, service delivery, and governance in-
cluded the creation of a prevention of delirium program
[73], family integrated program in NICU [45], and care
pathway for cellulitis that reduced admissions to hospital
[48]. Other engagements in this category led to complete
organizational redesign of an outpatient HIV clinic in
Southern Norway [50], reconceptualized service for outpa-
tients [68], and revisions to the delineation of roles and re-
sponsibilities between an Aboriginal community-controlled
health service and local Australian health service [74].
We conceptualized the development of educational

materials, tools, policy, and planning documents as
“discrete products,” whereas enhanced care process, ser-
vice delivery, and governance constituted “care process
or structural outcomes.” Interestingly, discrete products
were more likely to derive from studies using lower
levels of engagement (i.e., mostly consultative with ele-
ments of co-design), while care process or structural

Bombard et al. Implementation Science (2018) 13:98 Page 15 of 22

outcomes were more likely to result from higher levels
of engagement (i.e., co-design) (Table 3).

Impact of engaging patients on the institution
Engaging patients can also change the culture of staff
and care settings. The experiences reported in these arti-
cles included shifts in organizational culture promoting
further patient participation in service design and deliv-
ery, [40, 63, 75] achieving collaboration and mutual
learning, [42, 47, 76, 77] and sharing or neutralizing
power among patients and providers or staff, [52] as well

as developing new competencies and negotiating for ser-
vice changes [39, 59] (Table 4). Interestingly, these out-
comes tended to arise in mental health settings and
from co-design engagements (Table 5). Further analysis
of the methods used in these studies revealed key enab-
ling factors including creating deliberative spaces to
share experiences, including external facilitation; broad-
ening power and control to include users, values, and
beliefs exercises; conducting user/staff/provider training;
and implementing a top-down approach from the local
authority (Table 5).

Table 5 Examples of studies reporting the impact of engaging patients in institutions

Reference Level of engagement Service type Patient engagement outcome Method/facilitator

Acri et al. 2014 [65] Consultative to co-design Mental health Shared/neutralized power Equal voice of users and
organization

Barnes 2000 [75] Co-design Mental health Culture change Educational program

Barnes and Wistow
1994 [40]

Consultative to co-design Mental health Culture change Top-down approach from the
local authority

Buck 2004 [76] Consultative to co-design General health Collaboration and mutual
learning

Citizen advisory board

Elwell 2014 [48] Consultative to co-design Acute care Organizational impetus to
change

User group meetings

Frazier 2007 [42] Co-design Mental health Collaboration between
community and clinicians

Service model development

Godfrey et al. 2013 [73] Consultative to co-design Acute care Culture change

Program development

Hopkins and Neimec
2006 [52]

Co-design Home tx services Shared/neutralized power Users conducted research/
interviews

Iedema 2010 [39] Co-design Emergency services Development of new
competencies

Created deliberative space to
share experiences

Jones 2008 [62] Consultative to co-design Stroke services Meaningful user participation External facilitation

Lord 1994 [63] Co-design Mental health Culture change Broadening power and control

Macdonnell et al.
2013 [45]

Co-design Pediatrics Enhanced relationship between
users and providers

Program development

Mendenhall 2010 [77] Consultative to co-design Diabetes Collaboration between
community and providers

Talking circles, storytelling, giving
users active role

Pilgrim and Waldron
1998 [59]

Co-design Mental health Direct negotiations for change Empowering users and external
facilitation

Reeve et al. 2015 [74] Co-design Primary care Enhanced relationships between
community and providers

Extensive community
consultation

Swarbrick 2006 [55] Co-design Mental health Collaborative partnership User training

Thomson et al. 2015 [68] Co-design Multiple sclerosis Mutual understanding Program development

Todd 2000 [47] Co-design Intellectual disability Shift in thinking, collaboration,
and participation

Higher proportion of users to
providers, training, and clarity of
roles

Tollyfield 2014 [53] Co-design Acute care Reconnection to core values of
caring and compassion

Ongoing co-design meetings

Tooke 2013 [49] Consultative to co-design Dementia Enhanced communication
between users and providers

Service user panels

Walsh and Hostick
2005 [99]

Consultative to co-design Mental health User ownership External facilitation

Xie 2015 [54] Co-design Acute care Commitment and mutual
understanding

Familiar, experienced user
representatives, establishing
common ground and updating
users on progress

Bombard et al. Implementation Science (2018) 13:98 Page 16 of 22

Patients’ experiences of being engaged to improve
quality of care
Twenty-three of the 48 studies provided information on
the patients’ experiences of their engagement, though
only 12 studies formally evaluated patients’ experiences
in the process of being engaged to improve quality of
care. Of those that evaluated experiences, ten studies re-
ported positive views, while in two studies, patients re-
ported negative experiences and two studies reported
both positive and negative experiences (Additional file 3:
Table S1). Of the positive experiences, patients and carers
expressed satisfaction with the engagement processes
[43, 78] were interested in continuing their involvement in
the longer term, [75] felt the experience to be educational,
[52] and felt that participation highlighted issues that would
have otherwise been ignored [39, 64, 75]. Positive experi-
ences were linked to feeling empowered and independent
as a result of skills development and positive recognition
[58, 59, 63]. Some patients reported increased self-esteem
from contributing [41, 58, 66] and improved self-efficacy
and self-sufficiency [76] and that the experience encouraged
peer educators to pursue formal training [55]. In another
study, staff reported learning about user participation [46].
Patient feedback in other engagement studies was not as

positive. Some studies found that patients were satisfied
but felt the engagement demanded considerable energy
and time [66]. Others felt that their involvement was
tokenistic because decisions had been made in advance or
was used to justify decisions that had already been made
[47, 61]. Some participants felt that their requests were de-
nied or that managerial support was lacking [47], while
others were dissatisfied with their lack of involvement in
analyzing the findings and creating the final report [46].

Quality appraisal
The average quality of the studies was “fair,” based on a
quality appraisal tool that systematically reviews disparate
forms of evidence and methodologies on a scale from “very
poor,” “poor,” “fair,” and “good” [38] (Additional file 4:
Table S2). We also assessed the possible impact of study
quality on the review’s findings (akin to a “sensitivity ana-
lysis” conducted for meta-analyses). There were only 6 (of
48) “poor” quality studies. Removing the six poor quality
studies reduced the number/range of examples provided
for our findings on the strategies/contextual factors that
contributed to optimal patient engagement (research ques-
tion 1), their outcomes on services (research question 2),
and patients’ experience of being engaged (research ques-
tion 3), but deletion of these studies from the analysis did
not alter the substance of the findings.

Discussion
This study provides a comprehensive review of the strat-
egies used to engage patients in service planning, design,

and evaluation. It also identifies the outcomes and con-
textual factors shaping optimal patient engagement to
improve quality of care. Strategies and contextual factors
that enabled patient engagement included techniques to
enhance design, recruitment, involvement, and leader-
ship action, and those aimed at creating a receptive
context. Reported outcomes ranged from developing
education or tools for patients and providers and
informing policy or planning documents (discrete prod-
ucts) to enhanced care, service delivery, and governance
(care process or structural outcomes). Interestingly, the
level of engagement appears to influence the outcomes
of service redesign: discrete products largely derived
from low-level (consultative) engagement, whereas care
process or structural outcomes mainly derived from
high-level (co-design) engagement. Surprisingly, only a
minority of studies (n = 12; 25%) formally evaluated pa-
tients’ experiences of the engagement activities. While
most experiences were positive, some patients sought
greater involvement and felt that their involvement was
important but tokenistic, especially when requests were
denied or when the engagement was used to justify deci-
sions that had already been made. However, it remains
unclear how these initiatives affect patients and whether
these improvements translate into improved quality of
care at a system level.
There were several limitations to this review. Despite

the large number of initial search results, there was only
a small number of studies focused on involving patients
in co-designing health service improvement. Therefore,
despite our best attempts, the specificity of our search
criteria was modest, a problem familiar to systematic re-
views in health services research, which typically crosses
many disciplinary boundaries [38]. Future searches
would benefit from improved keywords or MeSH terms on
the topic of patient engagement. In addition, studies char-
acterized health service users and their involvement differ-
ently, ranging from user-centeredness, patient-centered
care, and user involvement to patient involvement or par-
ticipation. Indeed, “user” was a common term used in the
UK, whereas other terms such as “patient” and “caregiver”
are commonly used in the USA and Canada. These differ-
ent conceptualizations might signify important distinc-
tions, and the use of different terms, and the publication of
these papers across many different journals, raises chal-
lenges in identifying and analyzing this literature. We ad-
dressed this limitation by using multiple terms and search
strategies across multiple disciplinary databases that incor-
porated terms used in similar reviews. We deliberately
sought out the terminology used in key articles to expand
our search though may not have captured the entire
breadth of terms, such as “consumer,” a popular term used
in Australian health services research. We echo previous
work that identified this “conceptual muddle” as “one of

Bombard et al. Implementation Science (2018) 13:98 Page 17 of 22

the greatest barriers to truly integrating patient involve-
ment into health services, policy, and research” [79].
There was also significant variation in sample sizes

and populations included in these engagement studies.
Samples sizes ranged from 3 to 372 participants and in-
cluded a variety of patients, families, caregivers, service
users, health care providers, staff, board members, health
care managers, administrators, and decision-makers.
Many studies did not provide details on their sample.
These variations illuminate the absence of a standard ap-
proach for designing and reporting engagement initia-
tives. This variation may also reflect the variety of
journals in which this research is reported. Additional
limitations include the variety of methods used and the
limited evaluation of the engagement methods them-
selves. Where there was no explicit evaluation of engage-
ment, other information including authors’ discussion of
strengths and limitations was used to assess the effect-
iveness of engagement. However, this does not specific-
ally comprise evaluation of the engagement process or
its outcomes on care. Development of evaluative metrics
and frameworks for the procedural and substantive out-
comes of engagements appears warranted. A final im-
portant limitation is that our search ended in 2016, and
therefore, these insights may differ in the future given
the rapidly growing field of patient engagement. This is
a limitation familiar to systematic reviews but a future
review may be warranted.
Despite these limitations, our study revealed key in-

sights into the factors that influence the ability of health
care organizations and decision-makers to create oppor-
tunities for engagement that are not provided in individ-
ual studies, which cross disciplines and geographical
boundaries. We found that successful patient engage-
ment resulted in culture change within the organization,
meaningful collaboration and mutual learning, and
shared or neutralized power, which tended to arise in
settings where co-design is used. Optimal engagement
often includes some of the following strategies: use of
deliberative spaces to share experiences, external facilita-
tion, broadening power and control to include users in
all aspects of the process, flexible approaches for involv-
ing users, user training, clarity of roles and objectives,
providing feedback, leadership by local champions and
securing institutional and/or executive level commit-
ment, and sponsorship from local authority by way of
dedicated resources and on-going contact with manage-
ment and executives. Leadership is key, but there may
be a potential temporal trend in leadership actions;
top-down approaches to patient engagement tended to
be reported in earlier studies [40, 63] whereas more clin-
ician or community-driven initiatives emerged from
more recent studies [42, 77]. Another important factor
is the timing of engagement. If the engagement occurred

after a decision had been made, the success (or even
function) of the engagement became highly questionable
from the patient’s perspective. Taken together, this ana-
lysis suggests that co-design methods supported by ex-
ecutive sponsorship or driven by local champions that
use externally facilitated, deliberative, experience-based
discourse with trained users can promote successful pa-
tient engagement and outcomes.
Mental health settings emerged as a frequent venue

for patient engagement in our review. The earliest re-
ports in our review [61, 63, 80] are in this setting, sug-
gesting that the therapeutic approaches, the nature of
the population, or the orientation of mental health ser-
vices might encourage greater patient participation in
this area. Indeed, enabling service user involvement in
care planning is a key principle of contemporary mental
health guidance in the UK [81] and a potentially effect-
ive method of improving the culture and responsiveness
of mental health services in light of a service history
founded on aspects of containment and compulsion, and
the stigmatization of those using mental health services
[82]. Many of the co-design engagement activities that led
to staff and organizational changes such as improved col-
laboration and mutual learning [42, 47, 76, 77], sharing or
neutralizing power among patients and providers or staff
[52], developing new competencies, and negotiating for
service changes [39, 59] also occurred in mental health.
While patient engagement is now occurring in many set-
tings, the experiences in mental health settings serve as
important examples of effective patient engagement.
Ultimately, the effectiveness of any patient engagement

should be judged by its impact on patient care. There is
a growing body of literature that indicates that engaging
patients can lead to improved effectiveness, efficiency,
quality of care [28–31], health outcomes, and cost-effect-
ive health service utilization [27, 83, 84]. The outcomes
reported in our review spanned beyond improved care
to include enhanced governance and informed policies
and organizational planning, which illustrates the
breadth of quality of care initiatives that might be sought
through patient engagement. However, drawing causal
associations between engaging patients in health services
improvement and health outcomes is difficult. Further-
more, it remains unclear whether these improvements
translate into sustained or improved quality of care be-
yond local settings at a system level. Indeed, one study
found a lack of evidence that patient involvement leads to
the implementation of patient-centered care [85]. Some
evaluative tools are emerging [86], yet more studies are
needed that assess the conditions on which these tools
and strategies can sustain the quality of care systemically.
Our review builds upon previous reviews in this field by

providing insight into the associations between quality im-
provement methods and the varying system-level

Bombard et al. Implementation Science (2018) 13:98 Page 18 of 22

outcomes they yield. Indeed, our review echoes previous
research indicating that patient engagement can lead to a
multiplicity of health services outcomes with sufficient
role definition, training, and alignment of patient-provider
expectations but that the quality of the reporting has been
poor and the full impact of patient engagement is not fully
understood [87–89]. Previous reviews have been limited
to specific countries [87], care settings (e.g., mental health
[89]), hospitals [90], or study design (e.g., qualitative
studies [88]). In this way, our review provides a compre-
hensive perspective of optimal strategies used internation-
ally, across care settings and using multiple methodologies
to engage patients, caregivers, and relatives in quality of
care improvement initiatives. Our review also provides
novel insights into how the level of engagement influences
the outcomes, namely, discrete products (e.g., development
of tools and documents) largely derived from low-level en-
gagement (consultative unidirectional feedback), whereas
care process or structural outcomes (e.g., improved gov-
ernance, care or services) mainly derived from high-level
engagement (co-design or partnership strategies). If the
benefits of engaging patients in the design or delivery of
health care are to be realized at an organization or system
level, then effective strategies and the contextual factors
enabling their outcomes need to be identified so that learn-
ing can be generalized. Importantly, our review provides
guidance on the effective strategies and contextual factors
that enable patient engagement including techniques to
enhance the design, recruitment, involvement, and leader-
ship action, and those aimed to create a receptive context.
Future research would benefit from greater consistency

in the conceptual, methodological, and evaluative frame-
works employed. Greater emphasis is also needed on a
procedural evaluation that assesses group composition,
group cohesion or collaboration, equality of the participa-
tion, and the level of deliberation/reasoning. Such assess-
ments are being developed in the deliberative democracy
field [91] and could be informative in patient engagement
initiatives. The limited evaluation of patients’ experiences
is particularly ironic given the intent of these services to
be patient-centered. Additional evaluative metrics should
be developed to examine patients’ experiences. Finally,
since it is difficult to draw causal relationships between
patient engagement and health outcomes, future research
should incorporate longitudinal measures and approaches
to explore the impact of patient co-design on quality
of care.
Several practice implications also emerge and reflect

factors linked to the success of quality improvement ini-
tiatives more generally. Senior leadership support is crit-
ical to success since it increases the likelihood that the
relevant decision-makers will implement the findings,
and dedicated resources may encourage staff commit-
ment to these efforts.

Conclusions
Despite the substantive body of research on strategies to
engage patients and their effects on patients and health
services, the literature is varied and dispersed. This study
provides a comprehensive review of the strategies used
to engage patients in service planning and design, identi-
fies the outcomes, and contextual factors shaping opti-
mal patient engagement to improve quality of care.
Patient engagement can inform education, tools, plan-
ning, and policy (discrete products) as well as enhance
service delivery and governance (care process or struc-
tural outcomes). The level of engagement appears to
influence the outcomes of service redesign; discrete
products are largely derived from low-level (consultative
to co-design) engagement, whereas care process or struc-
tural outcomes mainly derived from high-level (co-design)
engagement. Further evidence is needed to understand pa-
tients’ experiences of the engagement process and whether
these outcomes translate into improved quality of care.

Additional files

Additional file 1: Table S3. PRISMA checklist. (DOC 63 kb)

Additional file 2: Figure S1. PRISMA diagram. (DOC 57 kb)

Additional file 3: Table S1. Analysis of patient engagement strategies
to improve quality of care. Identification of facilitators and barriers to
patient engagement and subsequent evaluation of patient experiences.
(DOCX 160 kb)

Additional file 4: Table S2. Quality appraisal. Quality appraisal of r
eview articles based on Hawker S, Payne S, Kerr C, Hardey M, Powell J.
Appraising the evidence: reviewing disparate data systematically.
Qual Health Res. 2002;12(9):1284–99. (DOCX 117 kb)

Acknowledgements
We thank the Canadian Foundation for Healthcare Improvement for the
funding for this study. Yvonne Bombard was funded by a Postdoctoral
Fellowship and a New Investigator Award from the Canadian Institutes of
Health Research (CIHR) and CIHR Strategic Training Fellowships of “Public
Health Policy” and “Health Care, Technology and Place” during the conduct
of this research. Jean-Louis Denis holds a Canada Research Chair on
governance and transformation of health systems and organizations. We
thank Drs. Sharon Strauss, Andrea Tricco, and Monika Kastner for the advice
on systematic review methodology.
None of these funding agencies played any role in the study design; in the
collection, analysis, and interpretation of data; in the writing of the report;
and in the decision to submit the article for publication.

Funding
The Canadian Foundation for Healthcare Improvement provided funding for
this study but were not involved in the conception or conduct of the
systematic review.

Availability of data and materials
The dataset(s) supporting the conclusions of this article is(are) included
within the article (and its additional file(s)).

Authors’ contributions
YB and GRB conceived of the study and participated in its design and
coordination. KO and PB retrieved the records. EO, CF, and PB screened the
records. EO and CF extracted the data from the eligible articles. YB and GRB
developed the initial interpretations of the data and participated in the data
analysis. SC conducted the quality appraisal. YB drafted the manuscript. YB

Bombard et al. Implementation Science (2018) 13:98 Page 19 of 22

and GRB revised the manuscript. J-LD and M-PP were involved in the study
design and oversight; they reviewed the initial data analyses and suggested
revisions to the versions of the
manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate
Not applicable.

Consent for publication
Not applicable.

Competing interests
The author(s) declare that they have no competing interests. All authors
have completed the Unified Competing Interest form at (available on
request from the corresponding author) and declare no support from any
organization for the submitted work; no financial relationships with any
organizations that might have an interest in the submitted work in the
previous 3 years; and no other relationships or activities that could appear to
have influenced the submitted work.

Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in
published maps and institutional affiliations.

Author details
1Institute of Health Policy, Management and Evaluation, University of
Toronto, 155 College Street 4th Floor, Toronto, Ontario M5T 3M6, Canada. 2Li
Ka Shing Knowledge Institute, St. Michael’s Hospital, 30 Bond St, Toronto,
Ontario M5B 1W8, Canada. 3Niagara Health System, 1200 Fourth Avenue, St.
Catharines, Ontario L2S 0A9, Canada. 4Professor of Health Policy and
Management, School of Public Health, Université de Montréal-CRCHUM &
Canada Research Chair in Health System Design and Adaptation, 900, Saint
Denis Street, Pavillion R, Montreal, Quebec H2X 0A9, Canada. 5Départment
de Gestion, d’Évaluation et de Politique de Santé, École de santé Publique,
Université de Montréal, Centre de recherche du CHUM, Carrefour de
l’innovation et de l’évaluation en santé, 850 rue Saint-Denis, Montréal,
Quebec H2X 0A9, Canada.

Received: 10 January 2018 Accepted: 20 June 2018

References
1. Darzi A. High quality care for all: our journey so far. London: Department of

Health; 2008.
2. Ontario Health Quality Council. QMonitor: 2009 report on Ontari’s health

system. Toronto: Government of Ontario; 2009.
3. Institute of Medicine. Crossing the quality chasm: a new health system for

the 21st century. Washington, DC: National Academy of Sciences; 2001.
4. Bradshaw PL. Service user involvement in the NHS in England: genuine user

participation or a dogma-driven folly? J Nurs Manag. 2008;16(6):673–81.
5. Say RE, Thomson R. The importance of patient preferences in treatment

decisions—challenges for doctors. BMJ. 2003;327(7414):542–5.
6. Coulter A. What do patients and the public want from primary care? BMJ.

2005;331(7526):1199–201.
7. Coulter A, Ellins J. Effectiveness of strategies for informing, educating, and

involving patients. BMJ. 2007;335(7609):24–7.
8. Bate P, Robert G. Experience-based design: from redesigning the system

around the patient to co-designing services with the patient. Qual Saf
Health Care. 2006;15(5):307–10.

9. Crawford MJ, Rutter D, Manley C, Weaver T, Bhui K, Fulop N, et al.
Systematic review of involving patients in the planning and development
of health care. BMJ. 2002;325(7375):1263.

10. Johnson B, Abraham M, Conway J, Simmons L, Edgman-Levitan S,
Sodomka P, et al. Partnering with patients and families to design a
patient- and family-centered health care system. Bethesda: Institute of
Family-Centered Care; 2008.

11. Carman KL, Dardess P, Maurer M, Sofaer S, Adams K, Bechtel C, et al. Patient
and family engagement: a framework for understanding the elements and
developing interventions and policies. Health Aff. 2013;32(2):223–31.

12. Boivin A, Lehoux P, Lacombe R, Burgers J, Grol R. Involving patients in
setting priorities for healthcare improvement: a cluster randomized trial.
Implement Sci. 2014;9:24.

13. Parand A, Dopson S, Renz A, Vincent C. The role of hospital managers
in quality and patient safety: a systematic review. BMJ Open.
2014;4(9):e005055.

14. Pomey MP, Flora L, Karazivan P, Dumez V, Lebel P, Vanier MC, et al. The
Montreal model: the challenges of a partnership relationship between patients
and healthcare professionals. Sante Publique. 2015;27(1 Suppl):S41–50.

15. Nelson EC, Batalden PB. Patient-based quality measurement systems.
Qual Manag Health Care. 1993;2(1):18–30.

16. Nelson EC, Wasson JH. Using patient based information to rapidly redesign
care. Healthc Forum J. 1994;37(4):25–9.

17. DLB S. Engaging patients as vigilant partners in aafety a systematic review.
Med Care Res Rev. 2010;67(2):119–48.

18. Uding N, Kieckhefer G, Trams C. Parent and community participation in
program design. Clin Nurs Res. 2009;18(1):68–79.

19. Vincent CA, Coulter A. Patient safety: what about the patient? Qual Saf
Health Care. 2002;11(1):76–80.

20. Cornwell J. Exploring how to improve patients’ experience in hospital at
both national and local levels. Nurs Times. 2009;105:26.

21. Coulter A, Elwyn G. What do patients want from high-quality general
practice and how do we involve them in improvement? Br J Gen Pract.
2002;52:22–5.

22. Hibbard JH. Engaging health care consumers to improve the quality of care.
Med Care. 2003;41(1):61–70.

23. Conway J, Johnson B, Edgman-Levitan S, Schlucter J. Partnering with
patients and families to design a patient and family centred health care
system: a roadmap for the future—a work in progress. Bethesda: Institute
for Family Centered Care; 2006.

24. Pomey M-P, Pierre M, Veronique G. La participation des usagers à la gestion
de la qualité des CSSS : un mirage ou une réalité ? La revue de l’innovation : La
Revue de l’innovation dans le secteur. Publica. 2009;14(2):1–23.

25. Goodrich J, Cornwell J. Seeing the person in the patient: the point of care
review paper. The King’s Fund: London; 2008.

26. Elwyn G, Buetow S, Hibbard J, Wensing M. Measuring quality through
performance. Respecting the subjective: quality measurement from the
patient’s perspective. BMJ. 2007;335(7628):1021–2.

27. Simpson EL, House AO. Involving users in the delivery and evaluation of
mental health services: systematic review. BMJ. 2002;325(7375):1265.

28. Favod J. Taking back control… giving patients information about their drug
regimes improves compliance. Nurs Times. 1993;89:68–70.

29. Sheppard M. Client satisfaction, extended intervention and interpersonal
skills in community mental health. J Adv Nurs. 1993;18(2):246–59.

30. Beech P, Norman IJ. Patients’ perceptions of the quality of psychiatric
nursing care: findings from a small-scale descriptive study. J Clin Nurs.
1995;4(2):117–23.

31. Connor H. Collaboration or chaos: a consumer perspective. Aust N Z J Ment
Health Nurs. 1999;8(3):79–85.

32. Arnstein SR. A ladder of citizen participation. J Am Plan Assoc.
1969;35(4):216–24.

33. Berwick DM. What patient centered care should mean: confessions of an
extremist. Health Aff. 2009;28(4):555–65.

34. Kastner M, Tricco AC, Soobiah C, Lillie E, Perrier L, Horsley T, et al. What is
the most appropriate knowledge synthesis method to conduct a review?
Protocol for a scoping review. BMC Med Res Methodol. 2012;12:114.

35. Pare G, Trudel MC, Jaana M, Kitsiou S. Synthesizing information
systems knowledge: a typology of literature reviews. Inf Manage.
2015;52(2):183–99.

36. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting
items for systematic reviews and meta-analyses: the PRISMA statement.
J Clin Epidemiol. 2009;62(10):1006–12.

37. Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards
systematic reviews that inform health care management and policy-making.
J Health Serv Res Policy. 2005;10(Suppl 1):35–48.

38. Hawker S, Payne S, Kerr C, Hardey M, Powell J. Appraising the evidence:
reviewing disparate data systematically. Qual Health Res. 2002;12(9):
1284–99.

39. Iedema R, Merrick E, Piper D, Britton K, Gray J, Verma R, Manning N.
Codesigning as a discursive practice in emergency health services: the
architecture of deliberation. J Appl Behav Sci. 2010;46(1):73–91.

Bombard et al. Implementation Science (2018) 13:98 Page 20 of 22

40. Barnes M, Wistow G. Achieving a strategy for user involvement in
community care. Health Social Care. 1994;2:347–56.

41. Coad J, Flay J, Aspinall M, Bilverstone B, Coxhead E, Hones B. Evaluating the
impact of involving young people in developing children’s services in an
acute hospital trust. J Clin Nurs. 2008;17(23):3115–22.

42. Frazier SL, Addul-Adil J, Atikins MS, Gathright T, Jackson M. Can’t have one
without the other: mental health providers and community parents
reducing barriers to services for families in urban poverty. J Community
Psychol. 2007;35(4):435–46.

43. Lofters A, Virani T, Grewal G, Lobb R. Using knowledge exchange to build
and sustain community support to reduce cancer screening inequities.
Prog Community Health Partnersh. 2015;9(3):379–87.

44. Murphy L, Wells JS, Lachman P, Bergin MA. Quality improvement initiative
in community mental health in the Republic of Ireland. Health Sci J.
2015;9(13):1–11.

45. Macdonell K, Christie K, Robson K, Pytlik K, Lee SK, O’Brien K. Implementing
family-integrated care in the NICU: engaging veteran parents in program
design and delivery. Adv Neonatal Care. 2013;13(4):262–9. quiz 70-1

46. Weinstein J. Involving mental health service users in quality assurance.
Health Expect. 2006;9(2):98–109.

47. Todd S, Felce D, Beyer S, Shearn J, Perry J, Kilsby M. Strategic planning and
progress under the All Wales Strategy: reflecting the perceptions of
stakeholders. J Intellect Disabil Res. 2000;44(Pt 1):31–44.

48. Elwell R. Developing a nurse-led integrated ‘red legs’ service. Br J Community
Nurs. 2014;19(1):12. 4-9

49. Tooke J. Involving people with dementia in the work of an organisation:
service user review panels. Qual Ageing Older Adults. 2013;14(1):56–65.

50. Berg RC, Gamst A, Said M, Aas KB, Songe SH, Fangen K, et al. True user
involvement by people living with HIV is possible: description of a user-driven
HIV clinic in Norway. J Assoc Nurses AIDS Care. 2015;26(6):732–42.

51. Carlson A, Rosenqvist U. Locally developed plans for quality diabetes care:
worker and consumer participation in the public healthcare system. Health
Educ Res. 1990;5(1):41–52.

52. Hopkins C, Niemiec S. The development of an evaluation questionnaire for
the Newcastle Crisis Assessment and Home Treatment Service: finding a
way to include the voices of service users. J Psychiatr Ment Health Nurs.
2006;13(1):40–7.

53. Tollyfield R. Facilitating an accelerated experience-based co-design project.
Br J Nurs. 2014;23(3):136–41.

54. Xie A, Carayon P, Cartmill R, Li Y, Cox ED, Plotkin JA, et al. Multi-stakeholder
collaboration in the redesign of family-centered rounds process. Appl Ergon.
2015;46 Pt A:115–23.

55. Swarbrick MP, Brice GH Jr. Sharing the message of hope, wellness, and
recovery with consumers psychiatric hospitals. Am J Psychiatr Rehabil.
2006;9:101–9.

56. Owens C, Farrand P, Darvill R, Emmens T, Hewis E, Aitken P. Involving
service users in intervention design: a participatory approach to developing
a text-messaging intervention to reduce repetition of self-harm. Health
Expect. 2011;14(3):285–95.

57. Factor SH, Galea S, de Duenas Geli LG, Saynisch M, Blumenthal S, Canales E,
et al. Development of a “survival” guide for substance users in Harlem,
New York City. Health Educ Behav. 2002;29(3):312–25.

58. Ferreira-Pinto JB, Ramos R. HIV/AIDS prevention among female sexual
partners of injection drug users in Ciudad Juarez, Mexico. AIDS Care.
1995;7(4):477–88.

59. Pilgrim D, Waldron L. User involvement in mental health service
development: how far can it go? J Ment Health. 1998;7(1):95–104.

60. Gibson F, Aslett H, Levitt G, Richardson A. Follow up after childhood
cancer: a typology of young people’s health care need. Clin Eff Nurs.
2005;9:133–46.

61. Wistow G, Barnes M. User involvement in community care: origins, purposes
and application. Public Adm. 1993;71:279–99.

62. Jones SP, Auton MF, Burton CR, Watkins CL. Engaging service users in the
development of stroke services: an action research study. J Clin Nurs.
2008;17(10):1270–9.

63. Lord J, Ochocka J, Czarny W, MacIlivery H. Analysis of change within a
mental health organization: a participatory process. Psychiatr Rehabil J.
1994;21(4):327–39.

64. Brooks F. “When I was on the ward”: the contribution of patient narratives
to public involvement in health care decision-making. Neurol Rehabil.
2008;14(1):24–30.

65. Acri M, Olin SS, Burton G, Herman RJ, Hoagwood KE. Innovations in the
identification and referral of mothers at risk for depression: development of
a peer-to-peer model. J Child Fam Stud. 2014;23(5):837–43.

66. van Staa A, Jedeloo S, Latour JM, Trappenburg MJ. Exciting but exhausting:
experiences with participatory research with chronically ill adolescents.
Health Expect. 2010;13(1):95–107.

67. Enriquez M, Cheng AL, Kelly PJ, Witt J, Coker AD, Kashubeck-West S.
Development and feasibility of an HIV and IPV prevention intervention
among low-income mothers receiving services in a Missouri Day Care
Center. Violence Against Women. 2010;16(5):560–78.

68. Thomson A, Rivas C, Giovannoni G. Multiple sclerosis outpatient future
groups: improving the quality of participant interaction and ideation tools
within service improvement activities. BMC Health Serv Res. 2015;15:105.

69. Cawston PG, Mercer SW, Barbour RS. Involving deprived communities in
improving the quality of primary care services: does participatory action
research work? BMC Health Serv Res. 2007;7:88.

70. Ennis L, Robotham D, Denis M, Pandit N, Newton D, Rose D, et al.
Collaborative development of an electronic personal health record for
people with severe and enduring mental health problems. BMC Psychiatry.
2014;14:305.

71. Erwin K, Martin MA, Flippin T, Norell S, Shadlyn A, Yang J, et al. Engaging
stakeholders to design a comparative effectiveness trial in children with
uncontrolled asthma. J Comp Eff Res. 2016;5(1):17–30.

72. Coker TR, Moreno C, Shekelle PG, Schuster MA, Chung PJ. Well-child care
clinical practice redesign for serving low-income children. Pediatrics.
2014;134(1):e229–39.

73. Godfrey M, Smith J, Green J, Cheater F, Inouye SK, Young JB. Developing
and implementing an integrated delirium prevention system of care: a
theory driven, participatory research study. BMC Health Serv Res.
2013;13:341.

74. Reeve C, Humphreys J, Wakerman J, Carroll V, Carter M, O’Brien T, et al.
Community participation in health service reform: the development of an
innovative remote aboriginal primary health-care service. Aust J Prim
Health. 2015;21(4):409–16.

75. Barnes D, Carpenter J, Bailey D. Partnerships with service users in
interprofessional education for community mental health: a case study.
J Interprof Care. 2000;14(2):189–200.

76. Buck DS, Rochon D, Davidson H, McCurdy S. Involving homeless persons
in the leadership of a health care organization. Qual Health Res.
2004;14(4):513–25.

77. Mendenhall T, Berge JM, Harper P, GreenCow B, LittleWalker N,
WhiteEagle S, BrownOwl S. The Family Education Diabetes Series (FEDS):
community-based participatory research with a midwestern American
Indian community. Nurs Inq. 2010;17(4):359–72.

78. Fitzgerald MM, Kirk GD, Bristow CA. Description and evaluation of a serious
game intervention to engage low secure service users with serious mental
illness in the design and refurbishment of their environment. J Psychiatr
Ment Health Nurs. 2011;18(4):316–22.

79. Forbat L, Hubbard G, Kearney N. Patient and public involvement: models
and muddles. J Clin Nurs. 2009;18(18):2547–54.

80. Barnes M, Wistow G. Learning to hear voices: listening to users of mental
health services. J Ment Health. 1994;3:525–40.

81. Government HMs. No health without mental health: a cross-government
outcomes strategy. In: Health Do. London: Her Majesty’s (HM) Government; 2011.

82. Munro K, Ross KM, Reid M. User involvement in mental health: time to face
up to the challenges of meaningful involvement? Int J Ment Health Promot.
2006;8:37–44.

83. Hunter R, Cameron R, Norrie J. Using patient-reported outcomes in
schizophrenia: the Scottish Schizophrenia Outcomes Study. J Psychiatr Serv.
2009;60(2):240–5.

84. Coulter A. The Autonomous Patient: Ending Paternalism in Medical Care.
London: Stationery Office (for the Nuffield Trust); 2002.

85. Groene O, Sunol R, Klazinga NS, Wang A, Dersarkissian M, Thompson CA, et al.
Involvement of patients or their representatives in quality management
functions in EU hospitals: implementation and impact on patient-centred care
strategies. Int J Qual Health Care. 2014;26(Suppl 1):81–91.

86. Abelson J, Li K, Wilson G, Shields K, Schneider C, Boesveld S. Supporting
quality public and patient engagement in health system organizations:
development and usability testing of the public and patient engagement
evaluation tool. Health Expect. 2016;19(4):817–27. https://doi.org/10.1111/
hex.12378.

Bombard et al. Implementation Science (2018) 13:98 Page 21 of 22

87. Mockford C, Staniszewska S, Griffiths F, Herron-Marx S. The impact of patient
and public involvement on UK NHS health care: a systematic review.
Int J Qual Health Care. 2012;24(1):28–38.

88. van C, McInerney P, Cooke R. Patients’ involvement in improvement
initiatives: a qualitative systematic review. JBI Database System Rev
Implement Rep. 2015;13(10):232–90.

89. Bee P, Price O, Baker J, Lovell K. Systematic synthesis of barriers and facilitators
to service user-led care planning. Br J Psychiatry. 2015;207(2):104–14.

90. Liang L, Cako A, Urquhart R, Straus SE, Wodchis WP, Baker GR, et al. Patient
engagement in hospital health service planning and improvement: a
scoping review. BMJ Open. 2018;8(1):e018263.

91. De Vries R, Stanczyk A, Wall IF, Uhlmann R, Damschroder LJ, Kim SY.
Assessing the quality of democratic deliberation: a case study of public
deliberation on the ethics of surrogate consent for research. Soc Sci Med.
2010;70(12):1896–903.

92. Blickem C, Kennedy A, Vassilev I, Morris R, Brooks H, Jariwala P, et al. Linking
people with long-term health conditions to healthy community activities:
development of Patient-Led Assessment for Network Support (PLANS).
Health Expect. 2013;16(3):e48–59.

93. Bone LR, Edington K, Rosenberg J, Wenzel J, Garza MA, Klein C, et al.
Building a navigation system to reduce cancer disparities among urban
black older adults. Prog Community Health Partnersh. 2013;7(2):209–18.

94. Hall SJ, Samuel LM, Murchie P. Toward shared care for people with cancer:
developing the model with patients and GPs. Fam Pract. 2011;28(5):554–64.

95. Higgins A, Hevey D, Gibbons P, O’Connor C, Boyd F, McBennett P, et al. A
participatory approach to the development of a co-produced and co-delivered
information programme for users of services and family members: the EOLAS
programme (paper 1). Ir J Psychol Med. 2016;34(01):19–27.

96. Jones EG, Goldsmith M, Effken J, Button K, Crago M. Creating and testing
a deaf-friendly, stop-smoking web site intervention. Am Ann Deaf.
2010;155(1):96–102.

97. MacNeill V. Forming partnerships with parents from a community
development perspective: lessons learnt from sure start. Health Soc Care
Community. 2009;17(6):659–65.

98. Rose D. Partnership, co-ordination of care and the place of user
involvement. J Ment Health. 2003;12(1):59–70.

99. Walsh M, Hostick T. Improving health care through community OR.
J Oper Res Soc. 2005;56:193–201.

Bombard et al. Implementation Science (2018) 13:98 Page 22 of 22

Case Analysis Operational Problems Engaging Patients – Part 1

In a single Word document 7 full pages (excluding cover page and citations), APA style, answer the following questions. Use the week’s reading materials and video to guide your responses.

1. Identify the most important facts surrounding the case.

2. Identify the key issue or issues.

3. Specify alternative courses of action.

4. Evaluate each course of action.

5. Recommend the best course of action.

Link –

Turn in your highest-quality paper
Get a qualified writer to help you with

“ CASE STUDY ”

Get high-quality paper

NEW! AI matching with writer

Hire a Writer

Client Reviews

4.9

Sitejabber

4.6

Trustpilot

4.8

Our Guarantees

100% Confidentiality

Information about customers is confidential and never disclosed to third parties.

Original Writing

We complete all papers from scratch. You can get a plagiarism report.

Timely Delivery

No missed deadlines – 97% of assignments are completed in time.

Money Back

If you're confident that a writer didn't follow your order details, ask for a refund.

New to Your Trusted Assignment Help Service? Sign up & Save

Calculate the price of your order

Type of paper needed:

Pages:

You will get a personal manager and a discount.

Academic level:

We'll send you the first draft for approval by at

Total price:

$0.00

Power up Your Academic Success with the
Team of Professionals. We’ve Got Your Back.

Power up Your Study Success with Experts We’ve Got Your Back.

Order Now Order Now