A7 Logic Model Nursing in the Community

From the suggested reading materials, select a home visiting or case management program and summarize in 250 words or less the program goals and objectives and what interventions are provided. Focus on nursing activities when possible. Develop a logic model that would reflect that program. Ensure the critical elements of inputs, outputs, outcomes or impacts are included.  There will be aspects of the logic model that will require research so references are required using APA style.  Place the detail references on a separate page with the logic model. Click here for a guide to developing and using logic models (https://www.cdc.gov/dhdsp/docs/logic_model ). Click here for an example of logic model (https://www.cdc.gov/prc/pdf/prc-logic-model ).

The assignment should be presented in logic model format with APA formatted citations and references. At least two scholarly sources, other than the textbook and provided materials are required.

JOGNN I N F O C U S
Effects of Home Visiting and Maternal
Mental Health on Use of the Emergency
Department among Late Preterm Infants
Neera K. Goyal, Alonzo T. Folger, Eric S. Hall, Robert T. Ammerman, Judith B. Van Ginkel, and Rita S. Pickler

Correspondence
Neera K. Goyal, MD
3333 Burnet Ave. ML 7009
Cincinnati, OH 45229.
neera.goyal@cchmc.org

Keywords
emergency department
home visit
late preterm
maternal

mental health

ABSTRACT

Objective: To describe use of the emergency department (ED) among late preterm versus term infants enrolled in a
home visiting program and to determine whether home visiting frequency was associated with outcome differences.

Design: Retrospective, cohort study.

Setting: Regional home visiting program in southwest Ohio from 2007–2010.

Participants: Late preterm and term infants born to mothers enrolled in home visiting. Program eligibility requires �
one of four characteristics: unmarried, low income, < 18 years, or suboptimal prenatal care.

Methods: Data were derived from vital statistics, hospital discharges, and home visiting records. Negative binomial
regression was used to determine association of ED visits in the first year with late preterm birth and home visit

frequency, adjusting for maternal and infant characteristics.

Results: Of 1,804 infants, 9.2% were born during the late preterm period. Thirty-eight percent of all infants had at least
one ED visit, 15.6% had three or more. No significant difference was found between the number of ED visits for late

preterm and term infants (39.4% vs. 37.8% with at least one ED visit, p = .69). In multivariable analysis, late preterm
birth combined with a maternal mental health diagnosis was associated with an ED incident rate ratio (IRR) of 1.26,

p = .03; high frequency of home visits was not significant (IRR = .92, p = .42).
Conclusions: Frequency of home visiting service over the first year of life is not significantly associated with reduced
ED visits for infants with at-risk attributes and born during the late preterm period. Research on how home visiting can

address ED use, particularly for those with prematurity and maternal mental health conditions, may strengthen program

impact and cost benefits.

JOGNN, 44, 135-144; 2015. DOI: 10.1111/1552-6909.12538
Accepted July 2014

Neera K. Goyal, MD, is an
assistant professor in the
Department of Pediatrics,
Cincinnati Children’s
Hospital Medical Center,
Cincinnati, OH.

Alonzo T. Folger, PhD, is a
senior epidemiologist in the
Department of Pediatrics,
Cincinnati Children’s
Hospital Medical Center,
Cincinnati, OH.

(Continued)

The elevated risk of mortality and morbidity for late
preterm infants (LPIs) born at 34 weeks 0 days to
36 weeks 6 days gestation, who represent more
than 70% of all preterm infants, has been increas-
ingly well described (Bird et al., 2010; Engle,
Tomashek, & Wallman, 2007; Martin, Kirmeyer,
Osterman, & Shepherd, 2009; Raju, Higgins,
Stark, & Leveno, 2006). Compared with infants
born full term (� 37 weeks), LPIs have higher
rates of hospitalization and emergency depart-
ment (ED) use in the neonatal period and through
the first year of life (Escobar et al., 2005; Jain &
Cheng, 2006; McLaurin, Hall, Jackson, Owens, &
Mahadevia, 2009). Importantly, for certain condi-
tions like neonatal jaundice, risk of hospitalization
for LPIs is higher compared with full-term infants
as well as infants born at earlier gestational ages
(Ray & Lorch, 2013), suggesting an interplay of

immature physiology and current systems of care
for this population. In contrast to very preterm
infants, LPIs are often discharged home from the
hospital without a prolonged period of observation
(Goyal, Fager, & Lorch, 2011), and many are not
seen by any health care professional during the
first week home (Hwang et al., 2013). Moreover,
the majority of these infants are not enrolled in
systematic, high-risk infant follow-up programs,
which generally focus on very early preterm in-
fants (Walker, Holland, Halliday, & Badawi, 2012).
For LPIs, therefore, further research is needed to
develop models of follow up care that can improve
outcomes (National Perinatal Association, 2012;
Premji, Young, Rogers, & Reilly, 2012).

One potential strategy to address these concerns
is home visiting, a voluntary service delivered

The authors report no con-
flict of interest or relevant
financial relationships.

http://jognn.awhonn.org C© 2014 AWHONN, the Association of Women’s Health, Obstetric and Neonatal Nurses 135

I N F O C U S Effects of Home Visiting and Maternal Mental Health on Use of the Emergency Department Among Late Preterm Infants

Given the late preterm birth rate among at-risk infants, practices
and policies related to their care have the potential for a large

public health impact.

in a family’s home to provide care coordination,
parenting education, and social support for
at-risk child-bearing women and their children
(American Academy of Pediatrics Council on
Child and Adolescent Health, 1998; Kitzman
et al., 1997; Sweet & Appelbaum, 2004). Several
national models of home visiting, including Nurse
Family Partnership and Healthy Families America,
have developed specific program curricula and
protocols; qualifications of home visitors range
from nurses to social workers to paraprofessionals
(U.S. Department of Health and Human Services,
2013). Currently, an estimated 400 publicly and
privately funded home visiting programs serve at
least 500,000 families in the United States, and
an additional $1.5 billion was allocated through
the Patient Protection and Affordable Care Act
to expand these services (Astuto & Allen, 2009;
Health Resources and Services Administration,
2010). Despite significant public investment in this
intervention, to date, a paucity of literature on out-
comes such as ED use for preterm infants enrolled
in such programs (Goyal, Teeters, & Ammerman,
2013).

Eric S. Hall, PhD, is an
assistant professor in the
Department of Pediatrics,
Cincinnati Children’s
Hospital Medical Center,
Cincinnati, OH.

Robert T. Ammerman,
PhD, is a professor in the
Department of Pediatrics,
Cincinnati Children’s
Hospital Medical Center,
Cincinnati, OH.

Judith B. Van Ginkel, PhD,
is a professor in the
Department of Pediatrics,
Cincinnati Children’s
Hospital Medical Center,
Cincinnati, OH.

Rita S. Pickler, RN, PhD, is
a professor of nursing in the
Department of Pediatrics,
Cincinnati Children’s
Hospital Medical Center,
Cincinnati, OH.

The study objectives were to characterize ED
use over the first year of life among late preterm
and full-term infants enrolled in home visiting
and to determine whether increased frequency
of home visiting participation is associated with
improvement in this outcome. Our logic model
for this study was based on the social-ecological
model of child health that underpins the role of
home visiting for at-risk families. A strong body
of literature has linked social and environmental
risk factors with adverse child health outcomes,
including avoidable hospitalizations and ED
visits, that may be mitigated through early detec-
tion, parental education, and care coordination
(McLaren & Hawe, 2005; Paul, Phillips, Widome, &
Hollenbeak, 2004; Shanley, Mittal, & Flores, 2013;
Shonkoff & Garner, 2012). Given the known con-
tribution of LPIs to pediatric morbidity and health
care costs and the fact that preterm birth is likely
to disproportionately affect at-risk mothers eligible
for home visiting, a more detailed understanding
of program effectiveness for LPIs may be critical
to addressing gaps in care for this important
population.

Methods
Setting and Par ticipants
In this retrospective, cohort study we examined
ED use among late preterm and term infants born
to at-risk, first-time mothers enrolled in a well-
established, regional home visiting program serv-
ing southwest Ohio. This community-based home
visiting program, which has to date served more
than 19,000 families, comprises 11 local home vis-
iting agencies which adhere to program, training,
and evaluation standards established by a cen-
tral office at Cincinnati Children’s Hospital Medi-
cal Center (CCHMC). To track and document pro-
cess and outcome measures within and across
agencies, the program uses rigorous continuous
quality improvement procedures under the super-
vision of CCHMC quality improvement staff and
is facilitated by a web-based data entry system
(Ammerman et al., 2007).

In addition to being first- time mothers, women
eligible for this program must have at least one
of four risk characteristics: unmarried, low income
(up to 300% of poverty level, receipt of Medicaid,
or reported concerns about finances), < 18 years of age, or suboptimal prenatal care. Participants may be enrolled during pregnancy or postde- livery, before their child reaches age 3 months. Referrals to the program may be self-initiated or come from clinics, hospitals, and other commu- nity sources. Home visits are provided by social workers, child development specialists, or other professionals who employ a core program curricu- lum that is based on the Healthy Families America model of home visiting. The overall goals of the program are to (a) provide nutrition education and substance use reduction during pregnancy; (b) support parents in providing children with a safe, nurturing, and stimulating home environment; (c) optimize child health and development; (d) link families to health care and other services; and (e) promote economic self-sufficiency. To achieve these goals as outlined within the curriculum, the home provider offers printed materials for fami- lies but primarily focuses on interactive sessions with parents that may address curriculum content as well other issues or concerns specific to the family. Screening inventories for home safety, par- enting stress, substance use, and other items are also performed at scheduled intervals to identify and address risks and to generate appropriate service referrals. Expected visit frequency consis- tent with the curriculum is weekly through the first 3 months of infancy, tapering to biweekly through the remainder of the first year.

136 JOGNN, 44, 135-144; 2015. DOI: 10.1111/1552-6909.12538 http://jognn.awhonn.org

Goyal, N. K. et al. I N F O C U S

For this analysis, infants born prior to 34 weeks
gestation were excluded, resulting in 1,852 late
preterm and term infants born during the years
2007 to –2010 whose mothers enrolled in home
visiting either prenatally or within 3 months af-
ter delivery. Of these infants, 43 additional infants
were excluded from analysis due to major congen-
ital anomalies, as their patterns of health care use
were expected to vary significantly from otherwise
healthy infants. For similar reasons, six infants who
died of any cause before their first birthdays were
also excluded from analysis.

Data Sources
Home visiting data were abstracted from the pro-
gram’s web-based data entry system described
above. This system contains detailed information
on each participant including enrollment timing,
home visit history, and maternal demographic and
psychosocial screening information. Enrolled par-
ticipants consented to data being used for the
purpose of quality improvement benchmarking
and research. These data were linked to Ohio vi-
tal statistics, available from the Ohio Department
of Health, and birth-related hospital discharge
records of mother and infant, available from the
Ohio Hospital Association. Because no common
unique identifier across data sources was avail-
able, record linkage was accomplished using
LINKS (University of Manitoba), a SAS-based
probabilistic and deterministic matching program.
Selected variables used for linking included ma-
ternal and infant dates of birth, hospital of birth, de-
livery method, sex, and maternal address. Further
details of linkage for these data sources has been
previously described (Hall et al., 2014). The result-
ing data set contained information regarding ma-
ternal/child health including demographics, social
factors, pregnancy-related conditions, and infant
characteristics. Lastly, this data core was linked
to electronic health record data at CCHMC for
outcome measures of hospital service use. The
Ohio Department of Health and CCHMC Institu-
tional Review Boards approved this study.

Covariates and Key Predictors
As described previously, data for maternal co-
variates were obtained through a combination of
linked vital statistics, hospital discharge records,
and home visiting data (Hall et al., 2014). These
variables included race, ethnicity, payer source,
maternal age, education level, marital status,
substance use, household membership, and
paternal involvement. Indicator variables for
relevant clinical factors were constructed using

International Classification of Diseases, 9th Revi-
sion, Clinical Modification (ICD-9-CM) codes and
vital statistics data (Centers for Disease Control
and Prevention, 2014). The ICD-9-CM codes used
to derive a composite maternal mental health
diagnosis were obtained from the maternal birth
hospitalization record.

Late preterm birth was defined as infant birth from
34 weeks 0 days to 36 weeks 6 days gestation;
gestational age measures were obtained from vi-
tal statistics and represented the best clinical es-
timates. Additionally, as a sensitivity analysis we
repeated analyses using a combined gestational
age estimate from vital statistics rather than the
clinical gestational age estimate, as prior studies
have demonstrated discordance between these
measures (Wingate, Alexander, Buekens, & Vahra-
tian, 2007). To measure home visiting service in-
tensity, we adapted a prior approach from Duggan
et al. (2004), counting the number of home

visits

conducted over the first year of life and then di-
viding this by the number of expected home visits
over the infant’s first year per the program cur-
riculum to calculate a percentage of expected vis-
its. Mother/infant pairs were classified as receiving
a high dose of service if they received �75% of
expected visits, a commonly used cutoff for ser-
vice evaluation in home visiting programs (Healthy
Families New York, 2014). Timing of program en-
rollment was dichotomized as enrollment prena-
tally or after birth of the infant.

Analysis
Bivariate analyses using chi-squared or t tests
were used to identify covariates associated with
any ED use and number of unique ED visits over
the first year after birth. Factors deemed to be
empirically or statistically important (p values less
than 0.25) were considered and tested using
step-wise multivariable modeling to derive parsi-
monious models. To account for overdispersion
of the ED visit data due to excess zeros, we used
a random-effects negative binomial regression
model as an alternative to standard Poisson
regression, adjusting for clustering by individual
home visiting agency. Models were tested for
goodness of fit using Akaike Information Criterion
values and link tests for model specification.
Multicollinearity was also assessed, with variance
inflation factors for all retained variables < 1

0

(O’brien, 2007).

The final multivariable model included the follow-
ing variables: infant sex, maternal race, ethnicity,

JOGNN 2015; Vol. 44, Issue 1 137

I N F O C U S Effects of Home Visiting and Maternal Mental Health on Use of the Emergency Department Among Late Preterm Infants

Home visiting for at-risk families may reduce unnecessary
emergency department use through care coordination,

education, and social support.

insurance status, maternal age, paternal living ar-
rangement, smoking status, and mental health di-
agnosis. To prevent individual patients from bias-
ing rates and to reduce measurement error, a small
number of infants with >8 ED visits in the first year
(1% of the sample) were deemed to be outliers
based on visual assessment of the data and were
omitted from the final analysis. Interaction terms
between late preterm status with number of home
visits, maternal age, smoking status, and mental
health diagnosis were added and tested for sta-
tistical significance using likelihood ratio tests and
only an interaction term with mental health diag-
nosis was retained. All statistical tests were two
sided, and type I error was controlled at 0.05. Anal-
yses were performed using STATA 11.0.

Results
Of the 1,804 infants meeting study inclusion cri-
teria, 9.2% were born late preterm. No significant
differences in maternal characteristics were ob-
served among late preterm versus full term in-
fants (56% vs. 52% with maternal age <20 years, p = .34), 96% versus 95% with single marital status (p = .56), and 81% versus 80% insured by Med- icaid (p = .96). Forty-eight percent of the sample enrolled in the home visiting program prenatally. Approximately 17.5% of infants were classified as receiving a high dose of home visiting, with no significant difference between late preterm com- pared with term infants (18% vs. 17%, p = .82).

The number of ED visits in the first year of life for
all infants ranged from 0 to 17, with 38% of infants
having at least one ED visit and more than 15% of
infants having three or more ED visits. As shown
in Figure 1, the distributions of primary diagnoses
accounting for more than 80% of ED visits were
similar, although late preterm compared with term
infants had a higher incidence of visits for feeding
difficulty (3.3% vs. 0.7%) and asthma/wheezing
not otherwise specified (2.2% vs. 0.8%).

Unadjusted Analysis
As shown in Table 1, bivariate comparisons
demonstrated no significant difference in any ED
use by gestational age category (term vs. late
preterm birth). In univariable analysis, late preterm
birth was also not associated with a significantly

Table 1: Bivariate Comparisons of Predic-
tors with any Emergency Department (ED)
use in the First Year of Life

No ED use Any ED use p value

Gestational age, % (n)

Late preterm 60.6 (100) 39.4 (65) 0.70

Full term 62.2 (1019) 37.8 (619)

Home visiting service, % (n)

<75% expected 60.7 (903) 39.3 (585) 0.01

visits

�75% expected 68.4 (216) 31.7 (100)

visits

Timing of enrollment, % (n)

Prenatal 59.2 (514) 40.8 (354) 0.02

Postnatal 64.6 (605) 35.4 (331)

Infant gender, % (n)

Female 64.1 (579) 35.9 (324) 0.07

Male 59.9 (540) 40.1 (361)

Race, % (n)

White 65.8 (379) 34.2 (197) 0.60

Black 59.5 (672) 40.5 (458)

Asian/Pacific 66.7 (10) 33.3 (5)

Islander

Multirace 67.3 (37) 32.7 (18)

Other 74.1 (20) 25.9 (7)

Ethnicity, % (n)

Hispanic 82.8 (135) 17.2 (28) <.001

Non-Hispanic 60.0 (984) 40.0 (657)

Insurance, % (n)

Medicaid 60.0 (861) 40.0 (573) 0.002

Private 67.5 (187) 32.5 (90)

Self-pay 76.3 (58) 23.7 (18)

Other 81.8 (9) 18.2 (2)

Maternal age, % (n)

<20 years 61.4 (578) 38.6 (364) 0.02

20–30 years 62.5 (530) 37.5 (318)

>30 years 78.6 (11) 21.4 (3)

Mental health diagnosis, % (n)

Yes 53.4 (95) 46.6 (83) 0.01

No 63.0 (1024) 37.0 (602)

(Continued)

138 JOGNN, 44, 135-144; 2015. DOI: 10.1111/1552-6909.12538 http://jognn.awhonn.org

Goyal, N. K. et al. I N F O C U S

Table 1: Continued

No ED use Any ED use p value

Smoking status, % (n)

Yes 56.5 (309) 43.5 (238) 0.001

No 64.4 (810) 35.6 (447)

Lives with infant’s father, % (n)

Yes 68.3 (198) 31.7 (92) .0

5

No 60.4 (836) 39.6 (549)

increased number of ED visits (incident rate ra-
tio (IRR) 1.06, 95% confidence interval (CI) [0.91,
1.24]). High intensity of home visits over the first
year of life was associated with a reduced inci-
dence of any ED visit (31.7% vs. 39.3% among
the low-intensity home visiting group); however,
as shown in Table 2 this association was not ob-
served for multiple ED visits, with an IRR = .88,
95% CI [0.72, 1.08]. The association of prenatal
versus postnatal enrollment was significant in uni-
variable analysis, both when modelled as any ED
use (Table 1) and as a count of ED visits, IRR =
1.18, 95% CI [1.02, 1.36].

Multivariable Analysis
Table 3 depicts results of negative binomial re-
gression analysis adjusting for clustering by home

visiting agency as well as covariates. Although
the incidence of mental health diagnosis in this
sample was not significantly different between late
preterm and term infants (7% vs. 10%, p = .15),
we observed a significant modifying effect of men-
tal health diagnosis on the association between
late preterm birth and ED visits. Compared with
term infants of mothers without a diagnosis, LPIs
of mothers with a mental health diagnosis had a
2.26 IRR for ED visits in the first year, 95% CI [1.10,
4.67]. Maternal mental health was also a statisti-
cally significant predictor for term infants; how-
ever, the effect size was smaller (IRR = 1.27, 95%
CI [1.01, 1.60]). In this multivariable model, neither
timing of enrollment nor frequency of home visiting
service during the first year was significantly asso-
ciated with a reduced rate of ED use. Other covari-
ates in the model that were statistically significant
included Hispanic ethnicity (adjusted odds ratio
[AOR] = .52, 95% CI [0.34, 0.80]), maternal smok-
ing (AOR = 1.26, 95% CI [1.07, 1.49]), Black race
(AOR = 1.23, 95% CI [1.01, 1.49]), and maternal
age >30 years (AOR = .41, 95% CI [0.20, 0.81]).
Results of a sensitivity analysis using multivariable
logistic regression were similar and therefore not
depicted.

Discussion
The vulnerability of LPIs in terms of mortality,
morbidity, and increased health care use in the
neonatal period and later in infancy and early

0
5

10

15

20

25

P
er

ce
nt

o
f E

m
er

ge
nc

y
V

is
its

Late preterm

0
5
10
15
20
25
P
er
ce
nt
o
f E
m
er
ge
nc
y
V
is
its

Full term

Figure 1. Primary diagnoses accounting for >80% of emergency department visits in the first year among term and late

preterm infants.

JOGNN 2015; Vol. 44, Issue 1 139

I N F O C U S Effects of Home Visiting and Maternal Mental Health on Use of the Emergency Department Among Late Preterm Infants

Table 2: Bivariate Comparisons of Predic-
tors with � 3 Emergency Department (ED)
Visits in the First Year of Life

<3 ED visits � 3 ED visits p-value

Gestational age, % (n)

Late preterm 84.2 (139) 15.8 (26) 0.88

Full term 84.7 (1387) 15.3 (251)

Home visiting service, % (n)

<75% expected 84.5 (1257) 15.5 (231) 0.77

visits

� 75% expected 85.1 (269) 14.9 (47)

visits
Timing of enrollment, % (n)

Prenatal 83.1 (721) 16.9 (147) 0.08

Postnatal 86.0 (805) 14.0 (131)

Infant gender, % (n)

Female 86.5 (781) 13.5 (122) 0.03

Male 82.7 (745) 17.3 (156)

Race, % (n)

White 89.2 (514) 10.8 (62) 0.001

Black 81.9 (926) 18.1 (204)

Asian/Pacific 100.0 (15) 0.0 (0)

Islander

Multi-Race 83.6 (46) 16.4 (9)

Other 88.9 (24) 11.1 (3)

Ethnicity, % (n)

Hispanic 92.6 (151) 7.4 (12) 0.003

Non-Hispanic 83.8 (1375) 16.2 (266)

Insurance, % (n)

Medicaid 83.7 (1200) 16.3 (234) 0.09

Private 87.0 (241) 13.0 (36)

Self-pay 90.8 (69) 9.2 (7)

Other 100.0 (11) 0.0 (0)

Maternal age, % (n)

<20 years 85.7 (806) 14.3 (135) 0.02

20–30 years 82.7 (672) 17.3 (141)

>30 years 95.9 (47) 4.1 (2)

Mental health diagnosis, % (n)

Yes 78.1 (139) 21.9 (39) 0.01

No 85.3 (1387) 14.7 (239)

(Continued)

Table 2: Continued

<3 ED visits � 3 ED visits p-value Smoking status, % (n)

Yes 81.7 (447) 18.3 (100) 0.03

No 85.8 (1079) 14.2 (178)

Lives with infant’s father, % (n)

Yes 87.9 (247) 12.1 (34) 0.80

No 83.8 (1160) 16.3 (225)

childhood, has been established in previous liter-
ature (Bird et al., 2010; Engle et al., 2007; Martin
et al., 2009; Medoff-Cooper et al., 2012; Raju et al.,
2006). However, a gap remains in the evidence to
support models of postdischarge care that may
improve outcomes for this population (Premji et al.,
2012). The majority of LPIs are not enrolled in
high-risk infant follow-up programs that generally
focus on very preterm infants (Walker et al., 2012).
Additionally, recent evidence suggests that many
families of LPIs do not access timely primary care
follow-up (Hwang et al., 2013). Moreover, given
known social and environmental risks associated
with preterm birth, LPIs compared with term in-
fants may be more likely to be affected by poverty,
social isolation, and other factors that place them
at further risk for adverse health and developmen-
tal outcomes (Eunice Kennedy Shriver National
Institute of Child Health and Human Development,
2014). We evaluated ED visit outcomes in a cohort
of late preterm and term infants enrolled in a
home visiting program, an intervention aimed
at addressing social determinants of health for
disadvantaged families through parent education,
social support, and care coordination. Prior litera-
ture on at least one national model of home visiting
(Nurse Family Partnership) suggests that infants
receiving home visits may incur fewer hospital-
izations for injuries and ingestions compared with
those not receiving home visits (Kitzman et al.,
1997). Our results demonstrate an association
between intensity and timing of home visits and
ED use in the first year; however, this relationship
does not persist after adjusting for other clinical
and social factors. We did not find evidence to
support that LPIs benefit differently from this inter-
vention than do other at-risk infants born full term.

We did not detect an independent association be-
tween late preterm birth and ED use, perhaps due
to relatively small sample size as well as the over-
all high level of use across the cohort regardless
of gestational age. Importantly, we did find that
within this at-risk cohort, maternal mental health

140 JOGNN, 44, 135-144; 2015. DOI: 10.1111/1552-6909.12538 http://jognn.awhonn.org

Goyal, N. K. et al. I N F O C U S

Table 3: Random Effects Negative Binomial
Regression Analysisa of Emergency Depart-
ment Visits in the First Year of Life

Incident 95% confidence

rate ratio interval

Late preterm 0.97 0.74, 1.27

Maternal mental

health

diagnosis

1.27 1.01, 1.60

Late preterm 2.26 1.10, 4.67

mental health
diagnosis

� 75% expected

visits

0.92 0.74, 1.13

Prenatal enrollment 1.05 0.91, 1.22

Female 0.79 0.69, 0.92

Race

White reference

Black 1.23 1.01, 1.49

Asian/Pacific 1.42 0.57, 3.52

Islander

Multi-Race 1.15 0.71, 1.85

Other 0.71 0.29, 1.71

Hispanic 0.52 0.34, 0.80

Insurance

Medicaid reference

Private 0.85 0.68, 1.06

Self-pay 0.63 0.39, 1.02

Other 0.72 0.18, 2.81

Maternal age, % (n)

<20 years 1.00 0.86, 1.16

20–30 years reference

>30 years 0.41 0.20, 0.81

Smoking 1.26 1.07, 1.49

Lives with infant’s

father

0.84 0.68, 1.05

Note. a Adjusts for clustering by home visiting agency with
an exchangeable correlation structure; 11 outlier observations
(>8 ED visits in the first year) were omitted from analysis.

conditions were a significant modifier on the as-
sociation between late preterm birth and ED use
in infancy. That is, LPIs whose mothers had di-
agnosed mental health conditions had more than
double the rate of ED visits in the first year of life

than full-term infants of mothers without diagnosed
mental health conditions. Prior work on maternal
mental health has demonstrated a link between
maternal depression and anxiety with increased
infant acute and emergency visits (Chung, Mc-
Collum, Elo, Lee, & Culhane, 2004; Mandl, Tronick,
Brennan, Alpert, & Homer, 1999; Minkovitz et al.,
2005; Sills, Shetterly, Xu, Magid, & Kempe, 2007).
In a recent study of term infants, the authors noted
that the degree of association with infant ED visits
differed by timing of maternal depression and anx-
iety with even higher use for those infants whose
mothers’ mental health conditions began during
the postpartum period (Farr et al., 2013). To our
knowledge, the differential effect of maternal men-
tal health on ED visits for late preterm versus term
infants has not been previously evaluated. How-
ever, prior researchers have described the rela-
tionship between late preterm birth and maternal
anxiety, particularly relating to feeding difficulty
(DeMauro, Patel, Medoff-Cooper, Posencheg, &
Abbasi, 2011; McDonald et al., 2013). Combined
with the fact that LPIs have rates and patterns of
feeding dysfunction in the first year of life that are
similar to those of very preterm infants, this result
likely contributes to our finding of higher incidence
of ED visits for feeding problems compared with
term infants.

Finally, our current findings with this cohort
enrolled in a home visiting program are generally
consistent with previously reported data on in-
creased risk of ED use among late preterm versus
term infants (Jain & Cheng, 2006). However,
overall rates of ED visits in this sample are high
compared with national data on ED use (published
estimates for young children age 0–4 years are
17% with at least one ED visit, and 11% with two or
more visits). This finding likely reflects the higher
risk level of families eligible for and receiving home
visiting services, as many sociodemographic fac-
tors including poverty, single-parent status, Black
race, and Medicaid coverage are associated
with more frequent ED use (Bloom, Cohen, &
Freeman, 2011; Halfon, Newacheck, Wood, & St
Peter, 1996). Maternal smoking status was also
a significant factor in our analysis, consistent
with prior literature demonstrating its association
with respiratory and gastrointestinal disorders in
infancy (Carroll et al., 2007; Shenassa & Brown,
2004). Given that more than 30% of mothers
in this sample were classified as smokers, this
behavior may be a critical point of targeted inter-
vention within home visiting prenatally and during
infancy.

JOGNN 2015; Vol. 44, Issue 1 141

I N F O C U S Effects of Home Visiting and Maternal Mental Health on Use of the Emergency Department Among Late Preterm Infants

Further efforts to develop and refine community-based
programs serving late preterm infants and their families may

need to address maternal mental health conditions.

Limitations
Several limitations related to use of adminis-
trative data in this retrospective analysis are
acknowledged. Complications and comorbidities
identified using vital statistics and hospital dis-
charge data may result in a misclassification bias
(Hsia, Krushat, Fagan, Tebbutt, & Kusserow, 1988;
Iezzoni et al., 1992; Romano & Mark, 1994). In par-
ticular, the mental health diagnosis variable, which
was based on the list of discharge diagnoses from
the birth hospital, has high specificity but lower
sensitivity in that there are likely women in the
sample with uncoded or undiagnosed conditions,
which could drive risk estimates towards the null
(Yasmeen, Romano, Schembri, Keyzer, & Gilbert,
2006). Moreover, this measure would not include
new mental health conditions that emerged after
the birth hospitalization period. However, to further
assess the validity of this variable, we compared
it to maternal scores on the Interpersonal Support
Evaluation List (ISEL), a validated screening tool
collected by home visitors at enrollment that mea-
sures perceived social support and correlates
with stress and negative effect (Merz et al., 2014);
as expected, the mental health variable was
significantly associated with lower (worse) ISEL
scores in our bivariate assessments, p = .001. An-
other potential limitation is that outcome measures
were reliant on data captured through the CCHMC
system, the region’s only pediatric ED services
provider. This system sees nearly 90% of pedi-
atric admissions within its eight- county primary
catchment zone, with that percentage increas-
ing for the youngest patients. These data may
underestimate rates of ED visits and introduce
selection bias, though our findings are robust
to adjustment for geographic clustering by zip
code. Another limitation may be generalizability
due to the regional population represented in the
study. Finally, though home visitors place a strong
emphasis on identification of a medical home and
adherence to well child care visits for program
participants, we were unable to assess individual
data on primary care use as a predictor of ED
visits. These limitations are offset by strengths of
the study, which include our ability to evaluate
ED use for a regional, at-risk population and link
this outcome to predictors not otherwise captured
in hospital administrative data (i.e., gestational
age, social risk factors). Outcomes were obtained

through electronic heath system records and
therefore not subject to parental reporting bias.

Practice Implications
Specific implications for practice based on these
findings may include enhanced screening and
support systems for maternal mental health issues
prior to hospital discharge, at postpartum follow-
up, and in pediatric settings. Within home visiting
programs, targeted counseling and guidance for
mothers to reduce maternal stress and improve
coping after delivery of a preterm infant may serve
as a useful curriculum enhancement. Additional
components of an enhanced home visiting cur-
riculum targeting LPIs may include emphasis on
elevated health risks associated with late preterm
birth, particularly with regards to respiratory con-
ditions and feeding difficulty. Finally, for all infants
enrolled in home visiting programs, further pro-
gram refinements focused on modifiable risk fac-
tors for ED use, such as improved connection to
primary care and a more systematic approach
to maternal smoking cessation, may increase the
cost benefits of this intensive, preventive service.

Conclusions
Late preterm infants have been previously shown
to be at higher risk for ED use and rehospitalization
compared with full-term infants in the first year of
life. This study is one of the first to focus on the util-
ity of a community-based program to mitigate out-
comes in this population. We observed that within
a socially at-risk population of infants enrolled in
home visiting, late preterm birth in combination
with maternal mental health conditions is associ-
ated with more than twofold higher rate of ED visits
compared with infants lacking these risk factors.
Utilization outcomes in this population did not ap-
pear to improve with early program engagement
or high intensity of service after adjustment for clin-
ical and social factors. Further research may focus
on development and refinement of approaches to
address needs of LPIs and their families in a home-
based setting.

REFERENCES
American Academy of Pediatrics Council on Child and Adolescent

Health. (1998). The role of home-visitation programs in improving

health outcomes for children and families. Pediatrics, 101(3 Pt

1), 486–489.

Ammerman, R. T., Putnam, F. W., Kopke, J. E., Gannon, T. A., Short,

J. A., Van Ginkel, J. B., . . . Spector, A. R. (2007). Development

and implementation of a quality assurance infrastructure in a

multisite home visitation program in Ohio and Kentucky. Journal

142 JOGNN, 44, 135-144; 2015. DOI: 10.1111/1552-6909.12538 http://jognn.awhonn.org

Goyal, N. K. et al. I N F O C U S

of Prevention & Intervention in the Community, 34(1/2), 89–107.

doi:10.1300/J005v34n01_05

Astuto, J., & Allen, L. (2009). Home visitation and young children: An

approach worth investing in? Society for Research in Child De-

velopment, Social Policy Report, 23, 3–22.

Bird, T. M., Bronstein, J. M., Hall, R. W., Lowery, C. L., Nugent, R., &

Mays, G. P. (2010). Late preterm infants: Birth outcomes and

health care utilization in the first year. Pediatrics, 126(2), e311–

319. doi:10.1542/peds.2009-2869

Bloom, B., Cohen, R. A., & Freeman, G. (2011). Summary health statis-

tics for U.S. children: National Health Interview Survey, 2009.

Vital and Health Statistics, 10(247), 1–82.

Carroll, K. N., Gebretsadik, T., Griffin, M. R., Dupont, W. D., Mitchel,

E. F., Wu, P., . . . Hartert, T. V. (2007). Maternal asthma

and maternal smoking are associated with increased risk

of bronchiolitis during infancy. Pediatrics, 119(6), 1104–1112.

doi:10.1542/peds.2006-2837

Centers for Disease Control and Prevention. (2014). International clas-

sification of diseases, ninth revision, clinical modification (ICD-9-

CM). Retrieved from http://www.cdc.gov/nchs/icd/icd9cm.htm.

Chung, E. K., McCollum, K. F., Elo, I. T., Lee, H. J., & Culhane, J. F.

(2004). Maternal depressive symptoms and infant health prac-

tices among low-income women. Pediatrics, 113(6), e523–529.

DeMauro, S. B., Patel, P. R., Medoff-Cooper, B., Posencheg, M., &

Abbasi, S. (2011). Postdischarge feeding patterns in early-

and late-preterm infants. Clinical Pediatrics, 50(10), 957–962.

doi:10.1177/0009922811409028

Duggan, A., McFarlane, E., Fuddy, L., Burrell, L., Higman, S.

M., Windham, A., & Sia, C. (2004). Randomized trial of a

statewide home visiting program: Impact in preventing child

abuse and neglect. Child Abuse & Neglect, 28(6), 597–622.

doi:10.1016/j.chiabu.2003.08.008

Engle, W. A., Tomashek, K. M., & Wallman, C. (2007). “Late-preterm”

infants: A population at risk. Pediatrics, 120(6), 1390–1401.

doi:10.1542/peds.2007-2952

Escobar, G. J., Greene, J. D., Hulac, P., Kincannon, E., Bischoff,

K., Gardner, M. N., . . . France, E. K. (2005). Rehospitalisa-

tion after birth hospitalisation: Patterns among infants of all

gestations. Archives of Disease in Childhood, 90(2), 125–131.

doi:10.1136/adc.2003.039974

Eunice Kennedy Shriver National Institute of Child Health and Hu-

man Development. (2014). What are the risk factors for

preterm birth and labor? Retrieved from http://www.nichd.nih.

gov/health/topics/preterm/conditioninfo/pages/who_risk.aspx

Farr, S. L., Dietz, P. M., Rizzo, J. H., Vesco, K. K., Callaghan, W. M.,

Bruce, F. C., . . . Berg, C. J. (2013). Health care utilisation in the

first year of life among infants of mothers with perinatal depres-

sion or anxiety. Paediatric and Perinatal Epidemiology, 27(1),

81–88. doi:10.1111/ppe.12012

Goyal, N. K., Fager C., & Lorch, S. A. (2011). Adherence to discharge

guidelines for late-preterm newborns. Pediatrics, 128(1), 62–71.

doi:10.1542/peds.2011-0258

Goyal, N. K., Teeters, A., & Ammerman, R. T. (2013). Home visiting and

outcomes of preterm infants: A systematic review. Pediatrics,

132(3), 502–516. doi:10.1542/peds.2013-0077

Halfon, N., Newacheck, P. W., Wood, D. L., & St Peter, R. F. (1996).

Routine emergency department use for sick care by children in

the United States. Pediatrics, 98(1), 28–34.

Hall, E. S., Goyal, N. K., Ammerman, R. T., Miller, M. M., Jones, D.

E., Short, J. A., & Van Ginkel, J. B. (2014). Development of

a linked perinatal data resource from state administrative and

community-based program data. Maternal and Child Health

Journal, 18(1), 316–325. doi:10.1007/s10995-013-1236-7

Health Resources and Services Administration. (2010). Maternal,

infant, and early childhood home visiting. Retrieved from

http://mchb.hrsa.gov/programs/homevisiting/index.html

Healthy Families New York. (2014). Multi-site policy and pro-

cedures manual: Critical element 4. Retrieved from

http://www.healthyfamiliesnewyork.org/admin_docs.cfm

Hsia, D. C., Krushat, W. M., Fagan, A. B., Tebbutt, J. A., &

Kusserow, R. P. (1988). Accuracy of diagnostic coding

for Medicare patients under the prospective-payment sys-

tem. New England Journal of Medicine, 318(6), 352–355.

doi:10.1056/NEJM198802113180604

Hwang, S. S., Barfield, W. D., Smith, R. A., Morrow, B., Shapiro-

Mendoza, C. K., Prince, C. B., . . . McCormick, M. C. (2013).

Discharge timing, outpatient follow-up, and home care of late-

preterm and early-term infants. Pediatrics, 132(1), 101–108.

doi:10.1542/peds.2012-3892

Iezzoni, L. I., Foley, S. M., Daley, J., Hughes, J., Fisher, E. S., & Heeren,

T. (1992). Comorbidities, complications, and coding bias. Does

the number of diagnosis codes matter in predicting in-hospital

mortality? Journal of the American Medical Association, 267(16),

2197–2203. doi:10.1001/jama.1992.03480160055034

Jain, S., & Cheng, J. (2006). Emergency department visits and rehospi-

talizations in late preterm infants. Clinics in Perinatology, 33(4),

935–945; abstract xi. doi:10.1016/j.clp.2006.09.007

Kitzman, H., Olds, D. L., Henderson, C. R., Jr., Hanks, C., Cole, R., Tatel-

baum, R., . . . Barnard, K. (1997). Effect of prenatal and infancy

home visitation by nurses on pregnancy outcomes, childhood in-

juries, and repeated childbearing. A randomized controlled trial.

Journal of the American Medical Association, 278(8), 644–652.

doi:10.1001/jama.1997.03550080054039

Mandl, K. D., Tronick, E. Z., Brennan, T. A., Alpert, H. R., & Homer,

C. J. (1999). Infant health care use and maternal depression.

Archives of Pediatrics & Adolescent Medicine, 153(8), 808–813.

doi:10.1001/archpedi.153.8.808

Martin, J. A., Kirmeyer, S., Osterman, M., & Shepherd, R. A. (2009).

Born a bit too early: Recent trends in late preterm births. National

Center for Health Statistics Data Brief, 24, 1–8.

McDonald, S. W., Benzies, K. M., Gallant, J. E., McNeil, D. A., Dolan,

S. M., & Tough, S. C. A (2013). Comparison between late

preterm and term infants on breastfeeding and maternal men-

tal health. Maternal and Child Health Journal, 17(8),1468–77.

doi:10.1007/s10995-012-1153-1

McLaren, L., & Hawe, P. (2005). Ecological perspectives in health re-

search. Journal of Epidemiology and Community Health, 59(1),

6–14. doi:10.1136/jech.2003.018044

McLaurin, K. K., Hall, C. B., Jackson, E. A., Owens, O. V., & Mahadevia,

P. J. (2009). Persistence of morbidity and cost differences be-

tween late-preterm and term infants during the first year of life.

Pediatrics, 123(2), 653–659. doi:10.1542/peds.2008-1439

Medoff-Cooper, B., Holditch-Davis, D., Verklan, M. T., Fraser-Askin,

D., Lamp, J., Santa-Donato, A., . . . Bingham, D. (2012). New-

born clinical outcomes of the AWHONN late preterm infant

research-based practice project. Journal of Obstetric, Gyneco-

logic & Neonatal Nursing, 41(6), 774–785. doi:10.1111/j.1552-

6909.2012.01401.x

Merz, E. L., Roesch, S. C., Malcarne, V. L., Penedo, F. J., Llabre,

M. M., Weitzman, O. B., . . . Gallo, L. C. (2014). Validation

of Interpersonal Support Evaluation List-12 (ISEL-12) scores

among English- and Spanish-speaking Hispanics/Latinos from

the HCHS/SOL Sociocultural Ancillary study. Psychological As-

sessment, 26(2), 384–394. doi:10.1037/a0035248

Minkovitz, C. S., Strobino, D., Scharfstein, D., Hou, W., Miller, T., Mis-

try, K. B., & Swartz, K. (2005). Maternal depressive symptoms

JOGNN 2015; Vol. 44, Issue 1 143

I N F O C U S Effects of Home Visiting and Maternal Mental Health on Use of the Emergency Department Among Late Preterm Infants

and children’s receipt of health care in the first 3 years of life.

Pediatrics, 115(2), 306–314. doi:10.1542/peds.2004-0341

National Perinatal Association. (2012). Multidisciplinary guide-

lines for the care of late preterm infants. Retrieved from

http://www.nationalperinatal.org/latepreterm

O’brien, R. (2007). A caution regarding rules of thumb for vari-

ance inflation factors. Quality and Quantity, 41, 673–690.

doi:10.1007/s11135-006-9018-6

Paul, I. M., Phillips, T. A., Widome, M. D., & Hollenbeak, C. S. (2004).

Cost-effectiveness of postnatal home nursing visits for preven-

tion of hospital care for jaundice and dehydration. Pediatrics,

114(4), 1015–1022. doi:10.1542/peds.2003-0766-L

Premji, S. S., Young, M., Rogers, C., & Reilly, S. (2012). Transitions in the

early-life of late preterm infants: Vulnerabilities and implications

for postpartum care. Journal of Perinatal & Neonatal Nursing,

26(1), 57–68. doi:10.1097/JPN.0b013e31823f8ff5

Raju, T. N., Higgins, R. D., Stark, A. R., & Leveno, K. J. (2006). Op-

timizing care and outcome for late-preterm (near-term) infants:

A summary of the workshop sponsored by the National Institute

of Child Health and Human Development. Pediatrics, 118(3),

1207–1214. doi:10.1542/peds.2006-0018

Ray, K. N., & Lorch, S. A. (2013). Hospitalization of early preterm,

late preterm, and term infants during the first year of

life by gestational age. Hospital Pediatrics, 3(3), 194–203.

doi:10.1542/hpeds.2012-0063

Romano, P. S., & Mark, D. H. (1994). Bias in the coding of hospital dis-

charge data and its implications for quality assessment. Medical

Care, 32(1), 81–90.

Shanley, L., Mittal, V., & Flores, G. (2013). Preventing dehydration-

related hospitalizations: A mixed-methods study of parents, in-

patient attendings, and primary care physicians. Hospital Pedi-

atrics, 3(3), 204–211. doi:10.1542/hpeds.2012-0094

Shenassa, E. D., & Brown, M.-J. (2004). Maternal smoking and infan-

tile gastrointestinal dysregulation: The case of colic. Pediatrics,

114(4), e497–e505. doi:10.1542/peds.2004-1036

Shonkoff, J. P., & Garner, A. S. (2012). The lifelong effects of early

childhood adversity and toxic stress. Pediatrics, 129(1), e232–

246. doi:10.1542/peds.2011-2663

Sills, M. R., Shetterly, S., Xu, S., Magid, D., & Kempe, A. (2007). As-

sociation between parental depression and children’s health

care use. Pediatrics, 119(4), e829–836. doi:10.1542/peds.2006-

2399

Sweet, M. A., & Appelbaum, M. I. (2004). Is home visiting an effective

strategy? A meta-analytic review of home visiting programs for

families with young children. Child Development, 75(5), 1435–

1456. doi:10.1111/j.1467-8624.2004.00750.x

U.S. Department of Health and Human Services. (2013). Home visit-

ing evidence of effectiveness (HomVEE) review. Retrieved from

http://homvee.acf.hhs.gov/

Walker, K., Holland, A., Halliday, R., & Badawi, N. (2012). Which high-

risk infants should we follow-up and how should we do it? Journal

of Paediatrics and Child Health, 48, 789–793.

Wingate, M. S., Alexander, G. R., Buekens, P., & Vahratian, A. (2007).

Comparison of gestational age classifications: Date of last men-

strual period vs. clinical estimate. Annals of Epidemiology, 17(6),

425–430. doi:10.1016/j.annepidem.2007.01.035

Yasmeen, S., Romano, P. S., Schembri, M. E., Keyzer, J. M., & Gilbert,

W. M. (2006). Accuracy of obstetric diagnoses and procedures

in hospital discharge data. American Journal of Obstetrics &

Gynecology, 194(4), 992–1001. doi:10.1016/j.ajog.2005.08.058

144 JOGNN, 44, 135-144; 2015. DOI: 10.1111/1552-6909.12538 http://jognn.awhonn.org

CDCDivision for

Heart Disease and Stroke Prevention

State Heart Disease and Stroke Prevention Program

EvaluationEvaluation
GuideGuide

Developing and Using a
Logic Model

Department of Health and Human Services
Centers for Disease Control and Prevention

National Center for Chronic Disease Prevention
and Health Promotion

Acknowledgements

This guide was developed for the Division for Heart Disease and Stroke Prevention
under the leadership of Susan Ladd and Jan Jernigan in collaboration with Nancy
Watkins, Rosanne Farris, Belinda Minta, and Sherene Brown.

State Heart Disease and Stroke Prevention programs were invaluable in the
development and fine-tuning of this guidance document. Their review contributed
significantly to the clarity and utility of this guide. Special thanks are extended to:
Susan Mormann (North Dakota Department of Health),
Ghazala Perveen (Kansas Department of Health and Environment),
Ahba Varma (North Carolina Department of Health and Human Services), and
Namvar Zohoori (Arkansas Department of Health and Human Services).

We encourage readers to adapt and share the tools and resources in the document to
meet program evaluation needs. For further information, contact the Division for Heart
Disease and Stroke Prevention, Applied Research and Evaluation Branch at
cddinfo@cdc.gov or
(990) 488-2424.

mailto:cddinfo@cdc.gov

Heart Disease and Stroke Prevention Program Evaluation Guides

Introduction

Purpose

The Heart Disease and Stroke Prevention (HDSP) Program Evaluation Guides are a series of
evaluation technical assistance tools developed by the Centers for Disease Control and
Prevention (CDC), Division for Heart Disease and Stroke Prevention, to assist in the evaluation of
heart disease and stroke prevention activities within states.

The guides are intended to offer guidance, consistent definition of terms, and aid skill building on a
wide range of general evaluation topics and selected specific topics. They were developed with the
assumption that state health departments have varied experience with program evaluation and a
range of resources allocated to program evaluation. In any case, these guides clarify approaches
to and methods for evaluation, provide examples specific to the scope and purpose of the state
HDSP programs, and recommend resources for additional reading. Some guides will be more
applicable to evaluating capacity building activity and others more focused on interventions.
Although examples provided in the guides are specific to HDSP programs, the information might
also prove valuable to other state health department programs, especially chronic disease
programs.

Background

Heart disease and stroke, the primary components of cardiovascular disease (CVD), are leading
causes of death and disability in the United States. As the burden of heart disease and stroke
continues to increase, these conditions are projected to remain the number one and two causes of
death worldwide through the year 2020. In the United States alone, CVD affects 61.8 million
Americans and claims nearly 1 million lives annually among people of all racial/ethnic groups and
ages.

In 1998, the U.S. Congress provided funding for CDC to initiate a national, state-based heart
disease and stroke prevention program. As of July 2005, CDC funds heart disease and stroke
prevention programs in 32 states and the District of Columbia. The priority areas for State activities
are:

• Increase control of high blood pressure.
• Increase control of high cholesterol.
• Increase awareness of signs and symptoms of heart attack and stroke and the need

to call 9-1-1.
• Improve emergency response.
• Improve quality of care.
• Eliminate disparities.

Many factors increase the risk of developing heart disease and stroke. State-based programs must
therefore use strategies that target multiple risk factors in many different settings, including health
care settings, work sites, communities, and school worksites to be effective.

States are encouraged to build capacity, use evidence-based approaches when they exist, and
develop innovative interventions to address heart disease and stroke prevention. CDC-funded
states are charged with providing evidence of capacity, of intervention, and of change within their
state and are encouraged to build evidence for innovative and promising practices.

Introduction to Guides

In 2003, CDC convened key public health partners, including state programs, to develop
A Public Health Action Plan to Prevent Heart Disease and Stroke. The Action Plan identifies
targeted recommendations and specific action steps necessary to reduce the health and economic
toll caused by heart disease and stroke and supports the identification of innovative ways to
monitor and evaluate policies and programs. The Action Plan is available online at
http://www.cdc.gov/cvh/Action_Plan/pdf/action_plan_full

Using the guides
The guides are intended to be companion pieces to existing program evaluation documents. The
CDC State Heart Disease and Stroke Prevention Program Evaluation Framework is located on the
Internet at http://www.cdc.gov/cvh/library/evaluation_framework/index.htm. The document is also
available on CDROM by contacting ccdinfo@cdc.gov or your CDC project officer.

The guide topics are divided broadly into two categories, fundamentals and capacity building- or
intervention-related. The guides in the fundamentals series will be completed first and will cover
general evaluation topics using specific HDSP examples.
Capacity building- and intervention-related guides will provide the tools and techniques to evaluate
capacity building activities, like the effectiveness of partnerships, and interventions in the health
care, work site, and community settings. Some of the guides will be developed for evaluations of
specific interventions and others will focus on tools for evaluating interventions.

Because states have different levels of experience and involvement with evaluation, the series of
guides will range from very basic to more advanced topics. Depending on the evaluation capacity
of state programs, some guides will be more useful to program staff than others.

The guides are expected to be distributed over time. They will be posted online for easy review
and access. State programs should review the guides as they are distributed and determine which
are most applicable given current resources and activities. The series will be expanded and
enhanced as additional needs are identified and as state evaluation capacity is increased. States
are encouraged to provide feedback to the Evaluation Team on the utility of guides and suggested
topics for future guides.

Bibliography

American Heart Association. Heart Disease and Stroke Statistics – 2006 Update. Dallas, Tex:
American Heart Association; 2006.

Centers for Disease Control and Prevention. Prevention Works: CDC Strategies for a Heart–
Healthy and Stroke–Free America. Atlanta, GA: U.S. Department of Health and Human Services;
2003. Available at http://www.cdc.gov/cvh/library/prevention_works/index.htm

Introduction to Guides

http://www.cdc.gov/cvh/Action_Plan/pdf/action_plan_full

http://www.cdc.gov/cvh/library/evaluation_framework/index.htm

mailto:ccdinfo@cdc.gov

http://www.cdc.gov/cvh/library/prevention_works/index.htm

Heart Disease and Stroke Prevention Program Evaluation Guide

Developing and Using a Logic Model

The evaluation guide “Logic Models” offers a general overview of the development and use of logic
models as planning and evaluation tools. A feedback page is provided at the end of this guide. We
will appreciate your comments.

Logic models are tools for planning, describing, managing, communicating, and evaluating a
program or intervention. They graphically represent the relationships between a program’s
activities and its intended effects, state the assumptions that underlie expectations that a program
will work, and frame the context in which the program operates. Logic models are not static
documents. In fact they should be revised periodically to reflect new evidence, lessons learned,
and changes in context, resources, activities, or expectations.

Logic models increase the likelihood that program efforts will be successful because they:
• Communicate the purpose of the program and expected results.
• Describe the actions expected to lead to the desired results.
• Become a reference point for everyone involved in the program.
• Improve program staff expertise in planning, implementation, and evaluation.
• Involve stakeholders, enhancing the likelihood of resource commitment.
• Incorporate findings from other research and demonstration projects.
• Identify potential obstacles to program operation so that staff can address them early on.

State programs should develop logic models to describe:

• The State HDSP program as a whole.
• A more detailed view of any specific intervention or component of a program, such as

developing a state plan or a health communication campaign.

Electronic logic model templates can be created fairly easil

y in either a Microsoft Word table or a
Microsoft Excel work sheet. A sample template is provided as an appendix.

Components of a Logic Model

As with many aspects of evaluation, people use a variety of terms to describe logic models and
their components. A logic model can also be visually represented in a variety of ways, including as
a flow chart, a map, or a table. The only “rule” for a logic model is that it be presented on one page.
The basic components of a good logic model are:

•

Displayed on one page.

•

Visually engaging.

•

Audience specific.

•

Appropriate in its level of detail.

•

Useful in clarifying program activities and expected outcome

s.
•

Easy to relate to.

•

Reflective of the context in which the program operates.

Logic Models Page 1

A basic logic model (Figure 1) typically has two “sides”—process and outcome. The process
section describes the program’s inputs (resources), activities, and outputs (direct products). The
outcome section describes the intended effects of the program, which can be short term,
intermediate, and/or long term. Assumptions under which the program or intervention operates,
and the contextual factors can also be included in a logic model. They are often noted in a box
below or on the left side of the logic model diagram. Figure 1, below, illustrates the components of
a logic model.

Figure 1. Layout of a General Logic Model

OUTCOMES PROCESS

short intermediate long outputs activities inputs

Assumptions/Contextual Factors

Component Definitions

Inputs are the resources that go into a program or intervention—what we invest. They include
financial, personnel, and in-kind resources from any source. For example, inputs could include:

Various funding sources for your program.
Your partners.
Staff time and technical assistance.

Activities are events undertaken by the program or partners to produce desired outcomes—what
we do. You could include a clear identification of “early” activities and “later” activities. Examples of
activities include:

Create a state-level partnership.
Train health care partners and staff in clinical guidelines.
Develop a community health communication campaign on signs and symptoms of stroke,

and to call 9-1-1.

Outputs are the direct, tangible results of activities—what we get. These early work products often
serve as documentation of progress. Examples include:

State-level partnerships created.
Health care professionals trained in clinical guidelines.
Community health communication campaigns developed.

Outcomes are the desired results of the program—what we achieve.
Describing outcomes as short, intermediate, or long term depends on the objective, the length of
the program, and expectations of the program or intervention. What is identified as a long-term
outcome for one program could be an intermediate outcome for another.

Logic Models Page 2

Short-term outcomes are the immediate effects of the program or intervention activities. They
often focus on the knowledge and attitudes of the intended audience. Examples include:

Increase partner knowledge of HDSP priorities and strategies.
Increase physician knowledge of clinical guidelines.
Increase knowledge of signs and symptoms of stroke and of the need to call 9-1-1.

Intermediate outcomes are behavior, normative, and policy changes. Examples include:
HDSP State Plan has been developed and published with partner involvement.
Health systems implement clinical guidelines.
Decrease transport time to treatment for stroke victims.

Long-term outcomes refer to the desired results of the program and can take years to
accomplish. Long-term outcomes include:

Increase in statewide policy and environmental strategies for HDSP.
Increase in blood pressure control in a health center population.
Increase in early treatment for stroke.

Impacts refer to the ultimate impacts of the program. They could be achieved in a year or take
10 or more years to achieve. These may or may not be reflected in the logic model, depending
on the purpose and audience of the logic model. A logic model that portrays an HDSP
intervention may show expected long-term outcomes, such as a state-level system change,
and impact, such as a population-wide reduction in death rate. Examples of impacts include:

Decrease in the rate of death due to heart disease.
Eliminate disparities in treatment for stroke between general and priority populations.

Assumptions are the beliefs we have about the program or intervention and the resources
involved. Assumptions include the way we think the program will work—the “theory” we have used
to develop the program or intervention. (See the subsequent section on Theories of Change.)
Assumptions are based on research, best practices, past experience and common sense. The
decisions we make about implementing a program or intervention are often based on our
assumptions. Examples of assumptions we sometimes make include:

• Funding will be secure throughout the course of the project.
• Because we teach information, it will be adopted and used in the way we intended.
• Professionals will be motivated to attend learning sessions.
• External funds and well-placed change agents can facilitate institutional change.
• Staff with the necessary skills and abilities can be recruited and hired.
• Partnerships or coalitions can effectively address problems or reach into areas we cannot.
• Policy adoption leads to individual behavior change.

In developing your logic model, you should explore and discuss the assumptions you are making.
Often, an in-depth discussion is included as a narrative that accompanies your logic model.
Inaccurate or overlooked assumptions could be a reason that your program or intervention did not
achieve the expected level of success.

Contextual Factors describe the environment in which the program exists and external factors
that interact with and influence the program or intervention. These factors may influence
implementation, participation, and the achievement of outcomes. Contextual factors are the
conditions over which we have little or no control that affect success.

Logic Models Page 3

Examples include:
• Competing or supporting initiatives sponsored by other agencies.
• Socioeconomic factors of the target audience.
• The motivations and behavior of the target population.
• Social norms and conditions that either support or hinder your outcomes in reaching

disparate populations, such as the background and personal experiences of
participants.

• Politics that support or hinder your activities.
• Potential barriers or supports that could affect the success of your project.

In program or intervention planning and development, we should consider contextual factors that
are likely to affect our activities and either address them or collect data on them as part of the
process evaluation.

Steps for developing a logic model
1. Determine the purpose of the logic model, who will use it and for what? Is your purpose to

develop a work plan, to talk with stakeholders about the program or intervention, or to
develop an evaluation plan?

2. Convene stakeholders. Who should participate? Program planners and managers,
epidemiologists, and groups with a stake in program outcomes.

3. Determine a focus for the logic model. Will the logic model depict a single intervention, a
multiyear intervention, or a comprehensive picture of your HDSP program? Determine what
level of detail is needed to make this a useful tool.

4. Understand the situation. Use the program objective or goal as your anchor. Set
priorities and clarify expectations.

5. Explore the research, knowledge base, and what others have done/are doing. Compile
research findings and lessons learned, applicable program theory, and resources. Identify
and discuss assumptions you are making and contextual factors.

6. Construct a series of linked activities and outcomes or statements using a “left-to-
right” or “right-to-left” approach. Then connect the activities with arrows to show
linkages.

One way to proceed is using a “left-to-right” process by connecting a series of “If, then” statements
that help you identify and connect activities and anticipated outcomes.

Ask yourself how you can complete the following to describe your program:
If we have and , we can (do) ______ and ______, which will
result in and

The first two blanks list the resources available to conduct your program, the third and
fourth blanks describe the activities to be conducted, and the final two blanks list the
expected outputs of those activities.

Example:
“If we have program funding and participating clinics, we can inform our clinic
partners of the need to implement clinical practice guidelines and sponsor training
for clinic teams on the chronic care model, which will then increase the number of
clinic teams who are aware of clinical practice guidelines and who implement the
chronic care model.

Logic Models Page 4

A Series of “If…Then” Statements

If you have
access to
them, then
you can use
them to
accomplish
your planned
activities

If you
accomplish
your planned
activities to
the extent
you intended,
then your
participants
will benefit in
certain ways

If these
benefits are
achieved,
then certain
changes in
groups or
communities
are expected
to occur

If you
accomplish
your planned
activities, then
you will
hopefully
deliver the
amount of
service that
you intended

Certain
resources
are needed
to operate
your program

Resources/
Inputs Activities Output Outcome Impact

Your Planned Work Your Intended Results

By asking other similar questions, you can determine your short-, intermediate-, and long-
term outcomes.
If we educate clinic teams and train them in the chronic care model in clinics, then

we will see and occur in the short-term.

Example:
“If we educate clinic teams and train them in the chronic care model in clinics, we

will see implementation of the chronic care model resulting in appropriate treatment
for patients with high blood pressure.”

Continuing with the flow of the logic model, you should next complete:
If clinics implement the chronic care model and have an increase in appropriate

treatment for high blood pressure (short-term outcomes), then we will see
occur (intermediate outcomes).

Example:
“If clinics use the chronic care model and increase appropriate treatment for patients

with high blood pressure, then we will see an increase in the number of patients with
high blood pressure under control.”

Next, consider what the accomplishment of intermediate outcomes will lead to:
If there is an increase in the number of current clinic patients whose high blood

pressure is under control, then we expect that to lead to (long-term
outcomes).

Example:
“If there is an increase in the number of current clinic patients whose high blood

pressure is under control, then we will see a reduction in heart disease and stroke
among these patients.”

Logic Models Page 5

Finally, identify contextual factors and assumptions that should be considered and stated
when developing the logic model and interventions. In the example above, although we
expect that controlling high blood pressure in an individual will reduce their risk for heart
disease and stroke, when we apply this theory to a population, there are a number of
confounding factors:

• Risk factors for high blood pressure such as obesity and diabetes are increasing in
prevalence. This is likely to cause an increase in the prevalence of high blood
pressure and the number of heart disease or stroke patients.

• We assume in this model that once control of high blood pressure has been
achieved, it will be maintained. This might not be the case.

• We assume that once the chronic care model is implemented and clinic-based
changes occur, the changes are maintained.

If we put this all together in a logic model, it would look like this:

Activities Outputs Short-term

Outcomes

Intermediate
Outcomes

Long-term
Outcomes

Inputs

Funding Educate
Clinic
teams
about
clinical
guidelines

Clinic
teams
educated
about
clinical
guidelines

Increase in
appropriate
treatment
for HBP

Decrease in
heart disease &
stroke among
clinic patients

Increase in #
of patients
with HBP
under control

Clinic
teams
implement
CCM

Clinic
teams
trained in
CCM

Provide
training to
clinic
teams in
the CCM

Clinic
Partners

Assumptions: CCM changes are maintained by clinics.
Patients maintain blood pressure control.

Contextual factors: Prevalence of risk factors and
hypertension increasing.

As you develop your logic model, remember the amount and types of resources, activities, and
outcomes depicted can vary and are particular to each program. Some programs will have an
abundance of resources that allow a variety of activities and other programs may choose to
conduct fewer activities. The activities and expected outcomes are based on the type of program
or intervention you are implementing, the resources you have available and their distribution, the
needs and desires of your program or department, and your partners.

Logic Models Page 6

Theories of change
In a logic model, arrows are drawn to indicate the links between resources, activities, and
outcomes. A theory of change is used to provide a rationale for the expected links between
program resources, activities, and outcomes. It explains how and why activities are expected to
lead to outcomes in the particular order depicted.

Health promotion and prevention activities are based on numerous theories of change — a
reasonable explanation of why and how a certain set of activities leads to certain outcomes. These
theories are based on our beliefs, expectations, experience, and conventional wisdom. They
describe the set of assumptions that explain both the steps that lead to long-term objectives and
the connections between program activities and outcomes that occur at each step of the way.
Several common theories of change are used in health programming. To learn more about theories
of change, the following Web sites will be useful:

• http://www.csupomona.edu/~jvgrizzell/best_practices/bctheory.html.
• http://www.cacr.ca/news/2002/0212elder.htm.
• http://www.cancer.gov/theory/pdf.

Theories of change allow us to hypothesize that a program’s intermediate and long-term outcomes
are a result of short-term outcomes, which are a result of the activities implemented. The logic
model for the State Heart Disease and Stroke Prevention Program is based on a socio-ecological
model that links environmental and policy or systems changes with individual-level behavioral
changes. The “systems” interventions of HDSP result in policy or environmental change that can
lead to changes in knowledge and attitudes that reinforce behavior change among individuals and
gatekeepers. For example, implementing the Chronic Care Model in a health care system would
include use of electronic medical records that remind physicians of services needed to increase the
number of patients who have their high blood pressure under control. This, in turn, leads to
changes in patient behavior that result in better management of their high blood pressure.

Use of the logic model as a planning tool
As a planning tool, a logic model clarifies the sequence of outcomes and the relationship between
activities and specific outcomes. It helps you:

• Examine/refine the program mission and vision, goals and objectives, preferably with
stakeholders.

• Identify the most important desired outcomes.
• Identify the “critical path.” If efforts must be reduced, which paths are most effective, are

likely to get you there quickest, and/or are most cost-effective?
• Identify existing and needed, or weak and strong, components of the program and ways to

enhance performance.
Much of the benefit of constructing program logic models comes from the process of discussing,
analyzing, and justifying the expected relationships and linkages between activities and expected
outcomes with staff and partners.

Use of the logic model as an evaluation tool
A logic model is often used to guide evaluation planning. It can help you:

• Determine what to evaluate.
• Identify appropriate evaluation questions based on the program.
• Know what information to collect to answer these questions—the indicators.
• Determine when to collect data.
• Determine data collection sources, methods, and instrumentation.

Logic Models Page 7

http://www.csupomona.edu/~jvgrizzell/best_practices/bctheory.html

http://www.cacr.ca/news/2002/0212elder.htm

http://www.cancer.gov/theory/pdf

Using a logic model we can identify four areas, or domains, on which we can focus evaluation
activities. The four evaluation domains embedded within the logic model shown in Figure 2 are:

1. Implementation (Process): Is the program or intervention implemented as planned? Were
all of the activities carried out as expected?

2. Effectiveness (Outcome): Is the intervention achieving its intended short-, intermediate-,
and/or long-term effects/outcomes?

3. Efficiency: How much “product” is produced for a given level of inputs/resources?
4. Causal Attribution: Is progress on outcomes due to your program or intervention? In public

health practice, causal attribution is often difficult to ascertain, especially for your more
distant outcomes. However, determining causality between your activities/outputs and your
short-term outcomes can often be accomplished without too much effort. Usually, surveys
and interviews, or analysis of records can establish causality at that level. And the brief time
duration for short-term outcomes usually insures that causal results can be determined in a
relatively small amount of time. By using theories of change to develop your logic model
you can assume, with more confidence, that intermediate and long-term outcomes are a
result of your short-term outcomes. Therefore, it is important to establish causality between
at least the activities (and resulting outputs) you carry out and the short-term outcomes.

Figure 2: Evaluation Domains

Evaluation Domains
sdf

Activities Inputs Outputs
Intermediate

Effects/

Outcomes

Short-term
Effects/

Outcomes

Long-term
Effects/

Outcomes

Efficiency
(link between boxes)

Causal Attribution
(progression between boxes)

Implementation
Effectiveness

The boxes and arrows in Figure 2 indicate evaluation points or places where it is logical to ask
evaluation questions. As the program or intervention progresses through the logic model—as the
intervention matures—new series of evaluation questions can be identified. Outcome evaluation
looks back over the entire model. If based on a good process evaluation, the logic model can help
identify reasons for less than successful interventions by asking “where did the model break down?”

Using this thinking, the logic model can facilitate mapping evaluation questions and indicators as
shown in Figure 3.

Logic Models Page 8

Figure 3: Mapping Evaluation Questions and Indicators to the Logic Model

Mapping Evaluation Questions and
Indicators to a Logic Model

Outcome Process

Short-term
Outcomes

Intermediate
Outcomes Long-term

HDSP Program Logic Model
The Healthy People 2010 Objectives for Heart Disease and Stroke are national goals to unify and
focus work done by states, federal agencies, and non-profit agencies. State HDSP programs are
not directly responsible for these long-term, high-level outcomes; however, state interventions and
accomplishments contribute to achieving them. Typically, surveillance data are used to track
progress on such long-term outcomes.

The CDC HDSP program logic model is provided in Appendix 1. The logic model was developed to
describe the processes and events that are expected from combined state and federal resources
and activities to prevent heart disease and stroke. CDC and State activities are outlined in terms of
capacity building, surveillance, and interventions. These activities and outcomes result in changes
in policy and environmental supports (intermediate outcomes), which in turn influence system or
population changes and improve health status (long-term outcomes). A population decrease in
premature death and disability (impact) is the ultimate result of program activities. As programs
focus efforts on disparate populations, these activities are also expected to eliminate disparities

etween general and priority populations. b

Activities Inputs Outputs Outcomes

Are
resources
adequate to
implement
program?

Is program
implemented
as planned?

How many,
how much
was
produced?

Change in
knowledge,
policy,
environment?

Change
in system,
behavior?

Change
in
health
status?

What will
be
measured?

What will
be
measured?
What will
be
measured?
What will
be
measured?

Indicators

Impacts

Evaluation Questions
Change in
population
health
status?

What will
be
measured?
What will
be
measured?
What will
be
measured?

Logic Models Page 9

Bibliography and Additional Resources
To learn more about logic models, the following sources are helpful:

• Taylor-Powell, E., Jones, L., & Henert, E. Enhancing Program Performance with Logic
Models; 2002 Retrieved November 2005 from the University of Wisconsin-Extension Web
site: http://www1.uwex.edu/ces/lmcourse/.

• Kellogg Foundation Logic Model Development Guide. Retrieved from W.K. Kellogg
Foundation Evaluation Toolkit: Retrieved October 2005 from
http://www.wkkf.org/default.aspx?tabid=101&CID=281&CatID=281&ItemID=2813669&NID=
20&LanguageID=0.

• US Department of Health and Human Services. Centers for Disease Control and
Prevention. Office of the Director, Office of Strategy and Innovation. Introduction to
Program Evaluation for Public Health Programs: A Self Study Guide. Atlanta, GA: Centers
for Disease Control and Prevention; 2005.

Logic Models Page 10

http://www1.uwex.edu/ces/lmcourse/

http://www.wkkf.org/default.aspx?tabid=101&CID=281&CatID=281&ItemID=2813669&NID=20&LanguageID=0

http://www.wkkf.org/default.aspx?tabid=101&CID=281&CatID=281&ItemID=2813669&NID=20&LanguageID=0

Logic Models Page 11

Appendices:
Logic Models

Appendix 1: CDC HDSP Program Logic Model

Appendix 1: CDC HDSP Program Logic Model

Logic Models Page 12

HDSP State Program Logic

Logic Models Page 12

Appendi

Logi

x 2: Logic Model Template

c Models Page 13

____________ State HDSP Logic Model

O u t c o m e s
INPUTS ACTIVITIES OUTPUTS SHORT INTERMEDIATE LONG

Assumptions Contextual Factors

HDSP Evaluation Guide Comments

The Program Services Branch and the Applied Research and Evaluation Branch will
appreciate your comments and feedback on this Evaluation Guide.

______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________
______________________________________________________________________________

Please return your comments by fax to:
HDSP Evaluation Team at

770-488-8151 (fax)
Or to your CDC HDSP Project Officer

Evaluation Plan Page 14

Visit our website at:

http://www.cdc.gov/DHDSP/index.htm

http://www.cdc.gov/DHDSP/index.htm

Purpose
Background
Bibliography
Electronic logic model templates can be created fairly easil
Components of a Logic Model
Displayed on one page.
Visually engaging.
Audience specific.
Appropriate in its level of detail.
Useful in clarifying program activities and expected outcome
Easy to relate to.
Reflective of the context in which the program operates.

CONTEXTUAL CONDITIONS
(e.g., health services and service gaps, socioeconomic conditions)

National, Regional, or Local Health
Priorities and Health

Disparities

Motivating Conditions for Developing and
Maintaining Relationships (e.g., Trust)

Research and
Evidence-Based
Programs and

Interventions, e.g.
• Individual Level
• Community Level
• Policy and

Environmental
Level

Contributes
to

Improved
Community

and
Population

Health
and

Elimination
of Health

Disparities

Enhanced
Community Capacity
for Health Promotion

and Disease
Prevention

Translation of
Research

to Practice
and Policy

Widespread Use
of Evidence-

Based Programs
and Policies

Skilled
Public Health

Professionals and
Community Members

Expanded
Resources

Increased
Recognition of

and Support for
PRCs and

Prevention Research

Logic Model for the Prevention Research Centers Program
at the Centers for Disease Control and Prevention (CDC)

Last Revision
January 13, 2009 (5)

CDC PRC Program Office Oversight
and Support

Establish
a Research

Agenda

Communicate and
Disseminate

Activities and
Findings

Provide Training,
Technical Assistance,

and Mentoring, e.g.
• Researchers
• Practitioners
• Students
• Community Members

Engage the

Community

Conduct Core and
Other Research Using

Sound Research
Methods

OUTCOMESINPUTS ACTIVITIES OUTPUTS

Evaluation

IMPACT

Recipients of
Training or
Technical

Assistance

Research and
Evaluation Findings
and Other Products, e.g.
• Publications
• Presentations
• Media
• Intervention Tools

Community
Committee

and
Partner

Community

Other Partners, e.g.
• Public Health

Organizations
• Government Agencies
• Non-Governmental

Organizations
• Private Sector

Academic
Institution

Resources and Capacities, e.g.
• Expertise in Key Areas (e.g. Research,

Community Engagement and Partnerships,
Training, Communication and Dissemination,
and Evaluation)

• Facilities, Infrastructure, and Administrative
Capacity

• Funding
• Experience in Community-Based Participatory

Research and Public Health Practice
• Community Relationships and Accessibility