SW: Assignment 2X
Please see attachments
The threat of public stigma, as well as self-stigma, can prevent individuals from receiving the mental health treatment they need. In this Assignment, you analyze the influence of stigma on experiences with and treatment of mental illness.
To prepare: Watch the TED Talk by Sangu Delle (YOU CAN TYPE Watch the TED Talk by Sangu Delle on YOUTUBE) and then review the readings for this week. Focus on Delle’s examples illustrating Corrigan’s model about the stages of stigma and the hierarchy of disclosure. Consider Delle’s experience against that model.
Submit a 3-page paper that addresses the following:
- Briefly explain Corrigan’s model of the stages of stigma and his recommendations and hierarchy about recovery.
- Explain whether Delle’s experience follows that model. Use specific examples to argue your perspective. If you agree, identify which stage of recovery Delle is in.
- Analyze Delle’s reports about his own experiences with both types of stigma. Provide specific examples, and in your analysis consider the following questions:
Does one type of stigma predominate in his talk?
Which of Delle’s personal values or beliefs were challenged by his internalizations about his own illness and help-seeking?
What strengths does he exhibit?
What was the primary benefit of his diagnosis?
Do you think his experience would be different if his culture was different? Explain why or why not?
www.LaRCP.ca464 W La Revue canadienne de psychiatrie, vol 57, no 8, août 2012
CanJPsychiatry 2012;57(8):464–469
Key Words: self-stigma,
stigma reduction, mental
illness, empowerment
Manuscript received and
accepted January 2012.
In Review
On the Self-Stigma of Mental Illness: Stages, Disclosure, and
Strategies for Change
Patrick W Corrigan, PsyD1; Deepa Rao, PhD, MA2
1 Distinguished Professor and Associate Dean for Research, College of Psychology, Illinois Institute of Technology, Chicago, Illinois.
Correspondence: Illinois Institute of Technology, 3424 South State Street, Chicago, IL 60616; corrigan@iit.edu.
2 Research Assistant Professor, Department of Global Health, University of Washington, Seattle, Washington.
People with mental illness have long experienced prejudice and discrimination.
Researchers have been able to study this phenomenon as stigma and have begun to
examine ways of reducing this stigma. Public stigma is the most prominent form observed
and studied, as it represents the prejudice and discrimination directed at a group by the
larger population. Self-stigma occurs when people internalize these public attitudes and
suffer numerous negative consequences as a result. In our article, we more fully define the
concept of self-stigma and describe the negative consequences of self-stigma for people
with mental illness. We also examine the advantages and disadvantages of disclosure in
reducing the impact of stigma. In addition, we argue that a key to challenging self-stigma
is to promote personal empowerment. Lastly, we discuss individual- and societal-level
methods for reducing self-stigma, programs led by peers as well as those led by social
service providers.
W W W
Les personnes souffrant de maladie mentale font depuis longtemps l’objet de préjugés
et de discrimination. Les chercheurs ont pu étudier ce phénomène comme étant celui
des stigmates, et ont commencé à examiner des façons de réduire ces stigmates. Les
stigmates du public sont la forme prédominante qui a été observée et étudiée, car elle
représente les préjugés et la discrimination dirigés vers un groupe par l’ensemble de la
population. L’auto-stigmatisation se produit lorsque les gens internalisent ces attitudes
du public et par la suite, souffrent de nombreuses conséquences négatives. Dans notre
article, nous définissons plus complètement le concept de l’auto-stigmatisation et décrivons
les conséquences négatives que l’auto-stigmatisation provoque chez les personnes
souffrant de maladie mentale. Nous examinons aussi les avantages et désavantages de
la divulgation pour réduire l’effet des stigmates. En outre, nous alléguons qu’un moyen
de défier l’auto-stigmatisation consiste à promouvoir l’habilitation personnelle. Enfin, nous
présentons des méthodes au niveau individuel et sociétal de réduire l’auto-stigmatisation,
des programmes menés par les pairs ainsi que ceux menés par des prestataires de
services sociaux.
In making sense of the prejudice and discrimination experienced by people with mental illnesses, researchers
have come to distinguish public stigma from self-stigma.1
Public stigma is what commonly comes to mind when
discussing the phenomenon, and represents the prejudice and
discrimination directed at a group by the population. Public
stigma refers to the negative attitudes held by members
of the public about people with devalued characteristics.
Self-stigma occurs when people internalize these public
attitudes and suffer numerous negative consequences as
a result.2 In our article, we seek to more fully define self-
stigma, doing so in terms of a stage model. We will argue
that a key to challenging self-stigma is to promote personal
empowerment. One way to do this is through disclosure, the
strategic decision to let others know about one’s struggle
toward recovery. Then, we will discuss individual and
societal level methods for reducing self-stigma.
Defining Self-Stigma
While acknowledging the role of societal and interpersonal
processes involved in stigma creation, social psychologists
study stigma as it relates to internal and subsequent
behavioural processes that can lead to social isolation
and ostracism.3 Stereotypes are the way in which
humans categorize information about groups of people.
Negative stereotypes, such as notions of dangerousness or
incompetence, often associated with mental illness, can be
harmful to people living with mental illnesses. Most people
http://crossmark.crossref.org/dialog/?doi=10.1177%2F070674371205700804&domain=pdf&date_stamp=2012-08-01
www.TheCJP.ca The Canadian Journal of Psychiatry, Vol 57, No 8, August 2012 W 465
On the Self-Stigma of Mental Illness: Stages, Disclosure, and Strategies for Change
Clinical Implications
• Self-stigma can significantly impact overall health
outcomes.
• Techniques and interventions have been developed to
help a person reduce self-stigma.
Limitations
• Although interventions have been developed to help
a person reduce self-stigma, there are limited studies
on the validation of these interventions and their
adaptations for specific populations.
• Future research needs to continue to evaluate
programs that promote empowerment to reduce
self-stigma.
people have knowledge of particular stereotypes
because they develop from, and are defined by, societal
characterizations of people with certain conditions. Although
broader society has defined these stereotypes, people may
not necessarily agree with them. People who agree with
the negative stereotypes develop negative feelings and
emotional reactions; this is prejudice. For example, a
person who believes that people with schizophrenia are
dangerous may ultimately describe feeling fearful of those
with serious mental illness (SMI). From this emotional
reaction comes discrimination, or the behavioural response
to having negative thoughts and feelings about a person in a
stigmatized outgroup. A member of the general public may
choose to remain distant from a person with mental illness
because of their fear (prejudice) and belief (stereotype) that
the person with mental illness is dangerous.
People who live with conditions such as schizophrenia are
also vulnerable to endorsing stereotypes about themselves,
which is self-stigma. It is comprised of endorsement of these
stereotypes of the self (for example, “I am dangerous”),
prejudice (for example, “I am afraid of myself”), and the
resulting self-discrimination (for example, self-imposed
isolation). Once a person internalizes negative stereotypes,
they may have negative emotional reactions. Low self-
esteem and poor self-efficacy are primary examples of
these negative emotional reactions.4 Self-discrimination,
particularly in the form of self-isolation, has many
pernicious effects leading to decreased health care service
use, poor health outcomes, and poor quality of life.5,6 Poor
self-efficacy and low self-esteem have also been associated
with not taking advantage of opportunities that promote
employment and independent living.7 Link et al8 called this
modified labelling theory; contrasting classic notions of
the label (see Gove9,10), Link et al noted that people who
internalize the stigma of mental illness worsen the course
of their illness because of the harm of the internalized
experience, per se. Self-stigmatization diminishes feelings
of self-worth, such that the hope in achieving goals is
undermined. Thus the harm of self-stigma manifests itself
through an intrapersonal process, and ultimately, through
poor health outcomes and quality of life.2,4
A Stage Model of Self-Stigma
Self-stigma has often been equated with perceived stigma;
for example, a person’s recognition that the public holds
prejudice and will discriminate against them because of their
mental illness label.7 In particular, perceived devaluation
and discrimination is thought to lead to diminished self-
esteem and -efficacy. We believe this to actually be the first
stage of a progressive model of self-stigma (Figure 1). As
such, we see the process of internalizing public stigmas as
occurring through a series of stages that successively follow
one another.2,4,10,11 In the general model, a person with an
undesired condition is aware of public stigma about their
condition (Awareness). This person may then agree that
these negative public stereotypes are true about the group
(Agreement). Subsequently, the person concurs that these
stereotypes apply to him- or herself (Application). This may
lead to harm and to significant decreases in self-esteem and
-efficacy (Harm). Unlike other research on self-stigma,12,13
the stage model shows that pernicious effects of stigma on
the self do not occur until later stages. Not until the person
applies the stigma, does harm to self-esteem or -efficacy
occur.
One of the challenges of a stage model of self-stigma
is sorting out the effects of later stages from those of
depression, which is frequently experienced among people
with SMI.14 Other staged models of behaviour suggest
that any individual stage is most strongly influenced by
the immediately preceding one.15 Thus to fully understand
stigma’s contribution to poor health outcomes, research
must crosswalk specific stages with common antecedents
of poor outcomes, such as depressive symptoms. In this
way, the effects of internalized stigma on self-esteem can
be partialled out from other causes of depression.
The Why Try Effect
A related consequence of self-stigmatization is what has
been called the why try effect, in which self-stigmatization
interferes with life goal achievement.11 Self-stigma
functions as a barrier to achieving life goals. However, self-
esteem and -efficacy can reduce the harmful results of self-
stigma. Diminished self-esteem leads to a sense of being
less worthy of opportunities, which undermines efforts at
independence, such as obtaining a competitive job.
Why should I seek a job as an accountant? I am
not deserving of such an important position. My
flaws should not allow me to take this kind of a
job from someone who is more commendable.
Alternatively, decrements to self-efficacy can lead to a why
try outcome based on a person’s belief that he or she is
incapable of achieving a life goal.
Why should I attempt to live on my own? I am not
able to be independent. I do not have the skills to
manage my own home.
Why try is a variant of modified labelling theory,8 in which
the social rejection linked to stigmatization contributes to
www.LaRCP.ca466 W La Revue canadienne de psychiatrie, vol 57, no 8, août 2012
In Review
low self-esteem. Modified labelling theory also suggests
avoidance as a behavioural consequence of devaluation.
When people perceive devaluation, they may avoid
situations where public disrespect is anticipated.
Challenging Self-Stigma
There is a paradox to self-stigma.16 Some people with
mental illness internalize it and suffer the harm to self-
esteem, self-efficacy, and lost goals. However, many
others seem oblivious to its effects and report no pain. Yet
another group is especially interesting: people who seem
to report righteous indignation at the injustice of stigma.
It is this third group that may suggest an antidote to self-
stigma: personal empowerment. Empowering people seems
to be an effective way of reducing self-stigmatization,
encourage people to believe they can achieve their life
goals, and circumvent further negative consequences that
result from self-stigmatizations. In a sense, empowerment
is the flip side of stigma, involving power, control, activism,
righteous indignation, and optimism. Investigations have
shown empowerment to be associated with high self-
esteem, better quality of life, increased social support,
and increased satisfaction with mutual-help programs.17–19
Thus empowerment is the broad manner by which we can
reduce stigma. In the remainder of our article, we describe
the specific mechanisms that are involved with empowering
people as ways to decrease self-stigma.
Disclosure: The First Step
Many people deal with self-stigma by staying in the closet;
they are able to shelter their shame by not letting other
people know about their mental illness. One way for a
person to promote antistigma and counter the shame is to
come out, to let other people know about their psychiatric
history. Research has interestingly shown that coming out
of the closet with mental illness is associated with decreased
negative effects of self-stigmatization on quality of life,
thereby encouraging people to move toward achieving their
life goals.20 When people are open about their condition,
worry and concern over secrecy is reduced; they may soon
find peers or family members who will support them, even
after knowing their condition, and they may find that their
openness promotes a sense of power and control over
their lives.21 Still, being open about one’s condition can
have negative implications. Openness may bring about
discrimination by members of the public, any relapses
may be more widely known than preferred, and therefore
more stressful, and in some cases, disclosure may be more
isolating. For example, in India, documentation of mental
illness is grounds for divorce, a situation that some would
consider a form of institutionalized stigma.22,23 A person
with mental illness in India may feel doubly stressed by
the threat of divorce and further public discrimination.
Deciding to disclose is ultimately a very personal decision,
closely tied to the cultural context, and requires thorough
consideration of the potential benefits and consequences.
Coming out is not a black-and-white decision. There are
strategies that vary in risk for handling disclosure, which
are summarized in Figure 2.24,25 At the most extreme, people
may stay in the closet through social avoidance. This means
keeping away from situations where people may find out
about one’s mental illness. Instead, they only associate with
other people who have mental illness. It is protective (no one
will find out the shame) but obviously also very restrictive.
Others may choose not to avoid social situations but instead
to keep their experiences a secret. An alternative version
of this is selective disclosure. Selective disclosure means
there is a group of people with whom private information is
disclosed and a group from whom this information is kept
secret. While there may be benefits of selective disclosure,
such as an increase in supportive peers, there is still a
secret that could represent a source of shame. People who
choose indiscriminant disclosure abandon the secrecy. They
make no active efforts to try to conceal their mental health
history and experiences. Hence they opt to disregard any
of the negative consequences of people finding out about
their mental illness. Broadcasting one’s experience means
educating people about mental illness. The goal here is to
seek out people to share past history and current experiences
with mental illness. Broadcasting has additional benefits,
compared with indiscriminant disclosure. Namely, it fosters
Figure 1 The stage model of self-stigma
Figure 1: The Stage Model of Self Stigma
Awareness:
The public believes
people with mental
illness
are weak.
Harm:
Because I am weak, I
am not worthy or
able.
Agreement:
That’s right. People
with mental illness
are weak.
Application:
I am mentally ill so I
must be weak.
Why try…
To pursue a job; I am not worthy.
To live on my own; I am not able.
www.TheCJP.ca The Canadian Journal of Psychiatry, Vol 57, No 8, August 2012 W 467
On the Self-Stigma of Mental Illness: Stages, Disclosure, and Strategies for Change
their sense of power over the experience of mental illness
and stigma.
Methods of Reducing Stigma
There are other strategies that people living with mental
illness can use to cope with the negative consequences of
self-stigmatization. A caution needs to be sounded first.
In trying to help people learn to overcome self-stigma,
advocates need to make sure they do not suggest that the
stigmatization is the person’s fault, that having self-stigma
is some kind of flaw like other psychiatric symptoms that
the person needs to correct. Stigma is a social injustice and
an error of society. Hence eradicating it is the responsibility,
and should be the priority, of that society. In the meantime,
people with mental illness may wish to learn ways to live
with, or compartmentalize, that stigma. However, curing it
lies with the community in which one lives. Hence erasing
public stigma may be a broad-based fix of the stigma
problem. What we broach here are more narrowly focused
efforts to help people who are bothered by internalized
stigma.
Manualized approaches to self-stigma reduction for people
with mental illnesses are in development. One promising
approach is the Ending Self-Stigma intervention,26 which
uses a group approach to reduce self-stigmatization. The
intervention meets as a group for 9 sessions, with materials
covering education about mental health, cognitive-
behavioural strategies to impact the internalization of public
stigmas, methods to strengthen family and community ties,
and techniques for responding to public discrimination.
The cognitive-behavioural strategies rest on insights from
cognitive therapy27 that frame self-stigma as irrational self-
statements (for example, “I must be a stupid person because
I get depressed”) that the person seeks to challenge (for
example, “Most other people do not think depressed people
are stupid”). These kinds of challenges lead to counters—
pithy statements people may use the next time they catch
themselves self-stigmatizing.
There I go again. Just because I got depressed last
fall does not mean I am stupid and incapable of
handling a job. I have struggles just like everyone
else.
A pilot study of the intervention showed that internalized
stigma was reduced and perceived social support increased
after participation in the weekly intervention.26
A good example of a societal-level approach that may also
benefit a person is the In Our Own Voice program, developed
by the National Alliance on Mental Illness in the United
States. This intervention involves a manualized group
approach for targeted groups of the general population.
Testimonials by people with mental illness are the key
to stigma reduction in this program. Participants of the
intervention can be, for example, health care professionals,
church congregations, and students. Research has shown
the program’s effectiveness in reducing negative attitudes
toward people with mental illness, in its long and short
versions.11 If programs such as these help to reduce public
stigmas around mental illness, possible prejudices that
a person with mental illness perceives and internalizes
would be reduced, thus indirectly impacting self-stigma. In
addition, the people providing testimonials as part of the
intervention feel empowered by the activist role they play
in advocating for themselves, thereby reducing self-stigma
as the program is implemented.
Peer Support
Consumer-operated programs offer another way for people
with SMI to enhance their sense of empowerment.28 Groups
such as these provide a range of services, including support
for people who are just coming out, recreation and shared
experiences that foster a sense of community within a
larger hostile culture, and advocacy and (or) political
efforts to further promote group pride.28 Several forces
have converged during the past century to foster consumer-
operated services for people with psychiatric disabilities.
Some reflect dissatisfaction with mental health services
that disempower people by providing services in restrictive
settings. Others represent a natural tendency of people to
seek support from others with similar problems. Recently,
various consumer-operated service programs have
developed, including: drop-in centres, housing programs,
homeless services, case management, crisis response,
Figure 2 A hierarchy of disclosure strategies
Figure 2. A hierarchy of disclosure strategies
Social avoidance:
Stay away from others so they do
not have a chance to stigmatize me!
Secrecy:
Go out into the world—work and go
to church—but tell no one about my
illness.
Selective disclosure:
Tell people about my illness who
seem like they will understand.
Indiscriminant disclosure:
Hide it from no one.
Broadcast:
Be proud. Let people know.
www.LaRCP.ca468 W La Revue canadienne de psychiatrie, vol 57, no 8, août 2012
In Review
benefit acquisition, antistigma services, advocacy, research,
technical assistance, and employment programs.28,29 Results
of a qualitative evaluation of consumer-operated programs
showed that participants in these programs reported
improvements in self-reliance and independence; coping
skills and knowledge; and feelings of empowerment.29
Future research needs to isolate the active ingredients of
consumer-operated services that lead to positive change.
Conclusions
Stigma is a societal creation—what social psychologists
have come to describe as prejudice and discrimination.
Unfortunately, some people with SMI internalize the stigma
and suffer significant blows to self-esteem and -efficacy.
However, self-stigma is not an inevitable curse. People
in a stigmatized group do not necessarily turn that stigma
onto themselves. Consider research about racism affecting
the African-American community. Classic psychological
models believed African Americans to have lower
self-esteem than White Americans because the former
internalized the biases and prejudices about them that
dominated in the culture of the latter.30,31 Research
consistently fails to show this, and, in fact, may suggest
the obverse; African Americans may have higher self-
esteem than White Americans.32–37 How can this be?
African Americans will report they are aware of White
Americans prejudice but do not believe it actually applies to
themselves. In fact, many African Americans report White
Americans ignorance can be a personal rallying cry for
their personal sense of empowerment and a wake-up call
for their community.
The lesson seems to apply to self-stigma for mental illness,
too. Internalizing prejudice and discrimination is not a
necessary consequence of stigma. Many people recognize
stigma as unjust and, rather than being swept away by it,
take it on as a personal goal to change. Many others are
unaware or unmotivated by the phenomenon altogether.
However, there are people who seem to apply the prejudice
to themselves and suffer lessened self-esteem and -efficacy.
These people may benefit from structured programs to
learn to challenge the irrational statements that plague
their self-identity. They may benefit from joining groups of
peers who have successfully tackled the stigma. They may
benefit from a strategic program to come out about their
stigma. Research needs to continue to identify and evaluate
programs that promote empowerment at the expense of
self-stigma.
Acknowledgements
This work was supported, in part, by US National Institutes
of Mental Health grant 08598–01. Dr Rao is supported by a
US National Institutes of Health career development award,
K23 MH 084551.
The Canadian Psychiatric Association proudly supports the
In Review series by providing an honorarium to the authors.
References
1. Corrigan PW. On the stigma of mental illness: practical strategies
for research and social change. Washington (DC): American
Psychological Association; 2005.
2. Corrigan PW, Watson A, Barr L. The self-stigma of mental illness:
implications for self-esteem and self-efficacy. J Soc Clin Psychol.
2006;25(9):875–884.
3. Phelan JC, Link BG, Dovidio JF. Stigma and prejudice: one animal
or two? Soc Sci Med. 2008;67(3):358–367.
4. Watson AC, Corrigan PW, Larson JE, et al. Self-stigma in people
with mental illness. Schizophr Bull. 2007;33(6):1312–1318.
5. Sirey JA, Bruce ML, Alexopoulos GS, et al. Stigma as a barrier
to recovery: perceived stigma and patient-rated severity of illness
as predictors of antidepressant drug adherence. Psychiatr Serv.
2001;52(12):1615–1620.
6. Sirey JA, Bruce ML, Alexopoulos GS, et al. Perceived stigma as a
predictor of treatment discontinuation in young and older outpatients
with depression. Am J Psychiatry. 2001;158(3):479–481.
7. Link B. Understanding labeling effects in the area of mental
disorders: an assessment of the effects of expectations of rejection.
Am Sociol Rev. 1987;52:96–112.
8. Link B, Cullen F, Struening E, et al. A modified labeling theory
approach to mental disorders: an empirical assessment. Am Sociol
Rev. 1989;54(3):400–423.
9. Gove WR. Labelling and mental illness: a critique. In: Gove WR,
editor. The labeling of deviance: evaluating a perspective. 2nd ed.
Beverly Hills (CA): Sage; 1980. p 53–99.
10. Gove WR. The current status of the labeling theory of mental illness.
In: Gove WR, editor. Deviance and mental illness. Beverly Hills
(CA): Sage; 1982.
11. Corrigan P, Larson J, Rusch N. Self-stigma and the “why try” effect:
impact on life goals and evidence-based practices. World Psychiatry.
2009;8(2):75–81.
12. Corrigan PW, Watson AC. The paradox of self-stigma and mental
illness. Clin Psychol Sci Pract. 2002;9:35–53.
13. Ritsher JB, Otilingam PG, Grajales M. Internalized stigma of mental
illness: psychometric properties of a new measure. Psychiatry Res.
2003;121:31–49.
14. Ritsher JB, Phelan JC. Internalized stigma predicts erosion of morale
among psychiatric outpatients. Psychiatry Res. 2004;129:257–265.
15. Butler A, Hokanson J, Flynn H. A comparison of self-esteem lability
and low trait self-esteem as vulnerability factors for depression.
J Pers Soc Psychol. 1994;66:166–177.
16. Perz C, DiClemente C, Carbonari J. Doing the right thing at the
right time? The interaction of stages and processes of change in
successful smoking cessation. Health Psychol. 1996;15(6):462–468.
17. Corrigan PW, Faber D, Rashid F, et al. The construct validity
of empowerment among consumers of mental health services.
Schizophr Res. 1999;38:77–84.
18. Rogers E, Chamberlin J, Ellison M. A consumer-constructed scale
to measure empowerment among users of mental health services.
Psychiatr Serv. 1997;48:1042–1047.
19. Rogers ES, Ralph RO, Salzer MS. Validating the empowerment
scale with a multisite sample of consumers of mental health
services. Psychiatr Serv. 2010;61:933–936.
20. Corrigan P, Morris S, Larson J, et al. Self-stigma and coming out
about one’s mental illness. J Community Psychol. 2010;38:259–272.
21. Corrigan P, Roe D, Tsang H. Challenging the stigma of mental
illness. Hoboken (NJ): John Wiley and Sons, Ltd; 2011.
22. Thara R, Kamath S, Kumar S. Women with schizophrenia
and broken marriages—doubly disadvantaged? Part I: patient
perspective. Int J Soc Psychiatry. 2003:49(3):225–232.
23. Thara R, Kamath S, Kumar S. Women with schizophrenia
and broken marriages—doubly disadvantaged? Part II: family
perspective. Int J Soc Psychiatry. 2003;49(3):233–240.
24. Corrigan PW, Lundin RK. Don’t call me nuts! Coping with the
stigma of mental illness. Tinley Park (IL): Recovery Press; 2001.
www.TheCJP.ca The Canadian Journal of Psychiatry, Vol 57, No 8, August 2012 W 469
On the Self-Stigma of Mental Illness: Stages, Disclosure, and Strategies for Change
25. Herman NJ. Return to sender: reintegrative stigma-management
strategies of expsychiatric patients. J Contemp Ethnogr.
1993;22:295–330.
26. Lucksted A, Drapalski A, Calmes C, et al. Ending self-stigma:
pilot evaluation of a new intervention to reduce internalized
stigma among people with mental illnesses. Psychiatr Rehabil J.
2011;35(1):51–54.
27. Beck, AT. Cognitive therapy and the emotional disorders. New York
(NY): International Universities Press Inc; 1975.
28. Clay S, Schell B, Corrigan PW. On our own, together: peer
programs for people with mental illness. Nashville (TN): Vanderbilt
University Press; 2005.
29. Van Tosh L, del Vecchio P. Consumer/survivor-operated self-help
programs: a technical report. Rockville (MD): Department of Health
and Human Services; 2000.
30. Allport GW. The nature of prejudice. Cambridge (MA): Addison-
Wesley; 1954.
31. Allport GW. The nature of prejudice. Reading (MA): Addison-
Wesley; 1979.
32. Hoelter JW. Factorial invariance and self-esteem: reassessing race
and sex differences. Soc Forces. 1983;61:834–846.
33. Jensen GF, White CS, Gelleher JM. Ethnic status and adolescent self
evaluations: an extension of research on minority self-esteem. Soc
Probl. 1982;30:226–239.
34. Porter JR, Washington RE. Black identity and self-esteem: a review
of studies of black self-concept, 1968–1978. Annu Rev Sociol.
1979;5:53–74.
35. Verkuyten M. Self-esteem among ethnic minority youth in western
countries. Soc Indic Res. 1994;32:21–47.
36. Verkuyten M. Self-esteem, self-concept stability, and aspects
of ethnic identity among minority and majority youth in the
Netherlands. J Youth Adolesc. 1995;24:115–175.
37. Wylie R. The self-concept. Vol 2. Lincoln (NE): University of
Nebraska Press; 1979.
Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=wecd20
Journal of Ethnic & Cultural Diversity in Social Work
ISSN: 1531-3204 (Print) 1531-3212 (Online) Journal homepage: https://www.tandfonline.com/loi/wecd20
The Stigma of Depression: Black American
Experiences
Rosalyn Denise Campbell & Orion Mowbray
To cite this article: Rosalyn Denise Campbell & Orion Mowbray (2016) The Stigma of Depression:
Black American Experiences, Journal of Ethnic & Cultural Diversity in Social Work, 25:4, 253-269,
DOI: 10.1080/15313204.2016.1187101
To link to this article: https://doi.org/10.1080/15313204.2016.1187101
Published online: 13 Jun 2016.
Submit your article to this journal
Article views: 4219
View related articles
View Crossmark data
Citing articles: 7 View citing articles
https://www.tandfonline.com/action/journalInformation?journalCode=wecd20
https://www.tandfonline.com/loi/wecd20
https://www.tandfonline.com/action/showCitFormats?doi=10.1080/15313204.2016.1187101
https://doi.org/10.1080/15313204.2016.1187101
https://www.tandfonline.com/action/authorSubmission?journalCode=wecd20&show=instructions
https://www.tandfonline.com/action/authorSubmission?journalCode=wecd20&show=instructions
https://www.tandfonline.com/doi/mlt/10.1080/15313204.2016.1187101
https://www.tandfonline.com/doi/mlt/10.1080/15313204.2016.1187101
http://crossmark.crossref.org/dialog/?doi=10.1080/15313204.2016.1187101&domain=pdf&date_stamp=2016-06-13
http://crossmark.crossref.org/dialog/?doi=10.1080/15313204.2016.1187101&domain=pdf&date_stamp=2016-06-13
https://www.tandfonline.com/doi/citedby/10.1080/15313204.2016.1187101#tabModule
https://www.tandfonline.com/doi/citedby/10.1080/15313204.2016.1187101#tabModule
The Stigma of Depression: Black American Experiences
Rosalyn Denise Campbell and Orion Mowbray
School of Social Work, University of Georgia, Athens, GA, USA
ABSTRACT
While stigma is a large barrier for all racial/ethnic groups,
research suggests that stigma has a particularly strong impact
on the help-seeking behaviors and service use patterns of Black
Americans. In this qualitative study, in-depth, semi-structured
interviews were conducted with 17 Black American men and
women, ages 21 to 57, who have experienced depression.
Several thematic categories emerged: (a) race/culture-specific
aspects of dealing with depression; (b) the impact of the diag-
nosis on depression sufferers; and (c) pushing back against the
stigma. These findings highlight the impact of sociocultural
factors like stigma on illness experiences, help-seeking beha-
viors, and service use patterns among Black Americans. They
also offer targets for intervention to increase service rates and
improve care for Black Americans with depression.
KEYWORDS
Black Americans; depression;
illness experiences; mental
health and illness; stigma
Individuals with identities not considered “normal” in society often experience
stigma. Stigma can be defined simply as “an attribute that is deeply discredit-
ing” (Goffman, 1963, p. 3). It is a characteristic, behavior, or identity seen as
“incongruous with our stereotype of what a given type of individual should be”
(Goffman, 1963, p. 3). Stigmatizing behavior from others can take many forms,
including social distancing and discrimination. Individuals might also experi-
ence a reduction or loss of social status (Link & Phelan, 2001).
Stigma is a problem common to many who have suffered from mental
illness. In many ways, it can exacerbate their distress (Yang et al., 2007). Even
though this experience of stigma is universal, the impact and implications
can be different, and arguably worse, for racial/ethnic minorities (Bailey,
Blackmon, & Stevens, 2009; Mishra, Lucksted, Gioia, Barnet, & Baquet,
2009; United States Department of Health and Human Services
[USDHHS], 2001). Historically, people of color have been stigmatized and
marginalized because of their racial and/or ethnic identity. Adding another
stigmatizing identity in the form of a mental illness can have serious reper-
cussions for people of color (Bolden &Wicks, 2005; Conner, Copeland, et al.,
2010; Gary, 2005; Matthews, Corrigan, Smith, & Aranda, 2006).
CONTACT Rosalyn Denise Campbell rdcampb@uga.edu University of Georgia, School of Social Work, 279
Williams Street, Room 346, Athens, GA 30602, USA.
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK
2016, VOL. 25, NO. 4, 253–269
http://dx.doi.org/10.1080/15313204.2016.1187101
© 2016 Taylor & Francis
Although depression and stigma have been investigated a great deal,
research focusing on the stigma experienced by Black depression sufferers
is sparse by comparison. In this study, we address this gap in the literature
through interviews with 17 Black American men and women about their
experiences with depression and the stigma they have experienced. Through
these interviews, we can understand more about how stigma is understood,
experienced, and combated in Black communities as well as how stigma can
operate as a barrier for those seeking help for depression.
Stigma in Black communities
Perhaps one of the largest social barriers studied in mental health service use
research is stigma. Actual experiences or the fear of being stigmatized keep a
large number of people who might benefit from treatment out of services
(Anglin, Link, & Phelan, 2006; Corrigan, 2004; USDHHS, 1999). Although
stigma is a large barrier for all racial/ethnic groups, research has found that
stigma is more of a factor in the illness and treatment experience of Black
Americans than it is for many other racial/ethnic groups (Anglin et al., 2006;
Bailey et al., 2009; Cruz, Pincus, Harman, Reynolds III, & Post, 2008; Rusch,
Kanter, Manos, & Weeks, 2008). Some Black Americans have discussed the
existence of social proscriptions to keep matters private and not to discuss
their struggles with others (Carpenter-Song et al., 2010; Conner, Copeland,
et al., 2010). Others believe that they will be treated negatively, discriminated
against, or socially excluded if others find out about their mental health
problems (Carpenter-Song et al., 2010; Mishra et al., 2009). Regardless of the
type experienced or feared, stigma stops a large number of Black Americans
from seeking treatment.
Many Black Americans believe that the stigma attached to mental illness is
felt more strongly in Black communities (Bailey et al., 2009; Conner,
Copeland, et al., 2010; Moran, 2004). Individuals whose mental illness is
known by others have reported experiencing stigmatizing attitudes and
behaviors from people in their communities (Conner, Lee, et al., 2010).
Similarly, individuals whose mental illness is not known by others have
expressed reluctance to reveal that they have mental disorders for fear that
they will be subjected to ridicule and rejection (Carpenter-Song et al., 2010;
Mishra et al., 2009). A desire to avoid being labeled as “crazy” or subjected to
social exclusion can cause some to hide their illness and/or the fact that they
seek treatment (Black, Gitlin, & Burke, 2011; Conner, Lee, et al., 2010;
Matthews et al., 2006; Mishra et al., 2009). For others, it might cause them
to delay seeking services or avoid treatment altogether (Alvidrez, Snowden, &
Kaiser, 2008; Black et al., 2011; Calloway, 2006; Mishra et al., 2009).
Some believe that stigma is stronger in Black communities because indi-
viduals simply lack information about mental disorders and treatment
254 R. D. CAMPBELL AND O. MOWBRAY
(Conner, Copeland, et al., 2010). However, even after interventions to edu-
cate Black Americans about mental illness and reduce stigma, stigmatizing
views can continue to be held and in some cases strengthened (Rao,
Feinglass, & Corrigan, 2007). The precarious social status of Black
Americans in the United States might contribute to stronger stigmatizing
attitudes and behaviors in Black communities. Problems with mental illness
are viewed as “reduc[ing] one’s social standing and life opportunities”
(Matthews et al., 2006, p. 262). For groups like Black Americans who occupy
statuses that have historically pushed them to the margins of many social
spaces, adopting another potentially stigmatizing identity is not something
done easily because it can come with serious threats to one’s status or
reputation (Black et al., 2011; Bolden & Wicks, 2005; Conner, Copeland,
et al., 2010; Gary, 2005; Matthews et al., 2006). Some research has found that
African-Americans more than Latinos or Whites desire larger physical and
social distance from those with mental illness (Rao et al., 2007). It might be
that Black Americans who do not suffer from mental illness fear losing their
tenuous status because they are associated with someone who is ill (Abdullah
& Brown, 2011; Rao et al., 2007).
The fear of being found guilty by association can also produce stigmatizing
attitudes and behaviors from family members of those diagnosed with mental
illness. Many Black Americans suffering from mental illness have expressed
fears of experiencing stigmatizing attitudes or social exclusion from family
and friends when experiencing a mental health problem (Alvidrez et al.,
2008; Cruz et al., 2008).
However, this is not the experience of all. For some Black Americans,
having a serious mental illness does not bring familial exclusion. In their
study of ethno-cultural variations in the experience and meaning of mental
illness, Carpenter-Song and colleagues (2010) found that Blacks were the
least likely of the racial/ethnic groups studied to believe that their family
would be disappointed in them because of their illness.
Specific study aims
Most of the research on stigma and mental illness focuses on mental illness in
general or on mental illnesses other than depression. A small subset of that
research examines the experiences of Black Americans, and only a few of
those have examined depression and stigma. We sought to add to this body
of research and designed a study that would investigate, among other topics,
the stigma attached to depression and mental health service use among Black
Americans. Through qualitative interviews, we explored Black Americans’
experiences with depression, including experiences of stigma in Black com-
munities. Although we highlight the experiences of stigma among Black
sufferers of depression, we do not argue that stigma itself is unique to
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 255
Black Americans who are depressed. Rather, we seek to explore how the
stigma Blacks encounter and experience is unique. It is important to recog-
nize and understand these differences when designing and implementing
interventions targeting Black Americans.
Gathering information on this topic can be accomplished through a number
of means. For this study, the first author and primary researcher utilized
the qualitative interviewing approach. Intensive interviews allowed her to
(a) move beyond the surface of experiences and extract more details;
(b) explore various statements and/or topics; and (c) ask about feelings,
thoughts, perceptions, and behaviors (Charmaz, 2006). This approach was
also useful because of the population being studied. Research has shown that
when collecting data from African-American research participants, this
group prefers in-person, face-to-face interviews as opposed to telephone or
mail surveys (Burlew, 2003). Conducting interviews also gives the researcher
an opportunity to explore the richness of the ethnic/racial minority experi-
ence and illuminate some of its nuances that, as a method, quantitative
analysis is not designed to capture. Particularly in health research, the
qualitative approach provides more detailed information about the complex
experience of health and nature of health behaviors (Stewart, Makwarimba,
Barnfather, Letourneau, & Neufeld, 2008).
Sample/study participants
For this study, the first author wanted to speak with individuals 18 years of age
or older who identified as Black or African-American and responded “yes” to at
least one of the following: (a) has felt sad, empty, or depressed for two weeks or
more during their life; (b) has been told by a doctor, pastor, coworker, family
member, or friend that they were depressed; or (c) has seen a doctor, counsel-
lor, or mental health professional for depression. A total of 17 Black American
men and women from a Midwestern community were recruited using flyers
posted at a variety of locations including local colleges and universities,
libraries, area churches, social service agencies, doctors’ offices, and outpatient
mental health organizations. As shown in Table 1, the majority of the 17 parti-
cipants were women (N = 13). Participants’ ages ranged from 21 years of age to
57, with the majority of participants falling in the 22–29 and 40–49 age ranges.
Nine participants were diagnosed with major depression by a mental health
professional, three were diagnosed by a medical physician, and five were self-
diagnosed. Collectively, the study participants had achieved a high level of
education, with five completing or pursuing post-bachelor’s education. Two
participants were college graduates, six reported completing some college, three
256 R. D. CAMPBELL AND O. MOWBRAY
finished high school or received a GED, and one participant, the eldest in the
study, did not graduate high school, finishing the eleventh grade.
Interview procedures
Participants were recruited during the summers of 2007 and 2008. No
compensation was offered during the summer 2007 recruitment period,
Table 1. Participant characteristics.
Name
Age (in
years) Gender Employment Status Level of Education Method of Diagnosis
Carol 24 Female Employed Completed high school;
currenta undergraduate
student
Self-diagnosed
Denise 26 Female Student Earned a bachelor’s
degree; currenta
graduate student
Clinically diagnosed (by a
friend who was a mental
health professional
Derrick 40s Male Employed Earned a bachelor’s
degree
Clinically diagnosed by a
mental health professional
Devon 32 Female Employed Some college Self-diagnosed
Donna 37 Female Unemployed Earned a bachelor’s
degree; career-specific
training
Self-diagnosed
Doris 57 Female Unemployed; collects
Supplemental Security
Income (SSI)
Completed the 11th
grade
Clinically diagnosed by a
physician
Drucilla 49 Female Employed Completed high school Clinically diagnosed by a
physician
Elisa 28 Female Student Earned a master’s degree;
currenta graduate
student
Clinically diagnosed by a
mental health professional
Janelle 27 Female Employed Earned a master’s degree Self-diagnosed
Kamille 26 Female Student Earned a master’s degree;
currenta graduate
student
Clinically diagnosed by a
mental health professional
Keith 55 Male Unemployed Earned a bachelor’s
degree
Clinically diagnosed by a
physician
Laura 21 Female Student Completed high school;
currenta undergraduate
student
Clinically diagnosed by a
mental health professional
Margie 49 Female Employed Earned a professional
graduate degree;
currenta graduate
student
Clinically diagnosed by a
mental health professional
Miller 43 Male Employed GED Clinically diagnosed by a
mental health professional
Richard 39 Male Unemployed Some college Clinically diagnosed by a
mental health professional
Shalesa 35 Female Student Earned a master’s degree;
currenta graduate
student
Clinically diagnosed by a
mental health professional
Sidney 28 Female Employed Completed high school;
currenta undergraduate
student
Self-diagnosed
aStatus at the time of study participation.
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 257
which included eight college-educated and/or professional women. During
the second recruitment period, five women and four men were recruited and
completed interviews. These participants were given a $10 gift card to a
grocery or discount store for their participation. The first author conducted
these semi-structured, open-ended interviews in locations of the respondents’
choosing and included respondents’ homes and local coffee shops. The
interviews were conducted using an interview guide designed to have parti-
cipants reflect on their experience(s) with depression, the cultural messages
they received regarding depression, and how those messages shaped their
thoughts about depression and help-seeking. Interviews were audio-recorded
with the participants’ permission and ranged from 30 minutes to 2.5 hours.
All participants’ names were changed to maintain confidentiality.
Analysis
The interviews were transcribed verbatim by the first author, trained research
assistant, or professional transcriptionist. Many of the “ums,” “likes,” and
“you knows” were removed when appropriate to improve the clarity of the
message. Even though the interviews yielded a wealth of information about
Black Americans’ experience with depression, the first author wanted to
focus on the topic of stigma.
The first author performed a thematic analysis on the participants’ tran-
scripts to detect relevant themes. This approach, as outlined by Braun and
Clarke (2006), was selected because it views analysis as a process that allows
the researcher to identify, refine, and report themes within the data, similar
to the grounded theory approach developed by Glaser and Strauss (1967).
However, grounded theory insists that the analysis contribute heavily to
theory development which may not be the desired goal or outcome of
some qualitative research. A simpler “thematic analysis means researchers
need not subscribe to the implicit theoretical commitments of grounded
theory if they do not wish to produce a fully worked-up grounded-theory
analysis” (Braun & Clarke, 2006, p. 8). For a topic as understudied as Black
American mental health, the freedom to simply explore the richness of the
data without pointedly contributing to the theory construction or reframing
is welcomed and refreshing.
While describing thematic analysis as “simple,” we do not mean to imply
that this method is easy or plain. According to Braun and Clarke (2006), a
number of factors must be considered: (a) Will the analysis be a “rich
description of the data set or a detailed account of one particular aspect”
(p. 11); (b) Will one be using an inductive or deductive approach to the
analysis? (c) Is the level of analysis explicit or interpretive? (d) Is the
researcher using an essentialist or a constructionist epistemology? Even
though a number of topics were discussed during these qualitative interviews,
258 R. D. CAMPBELL AND O. MOWBRAY
the thematic analysis focused on one particular aspect of the data: stigma—
how it was encountered, experienced, and understood by participants. The
first author approached the data inductively, allowing the themes to come
from the data itself, not fitting the data into pre-set categories. The level of
analysis was more interpretive in that the goal was to “to identify or examine
the underlying ideas, assumptions, and conceptualizations—and ideologies—
that are theorized as shaping or informing the semantic content of the data”
(Braun & Clarke, 2006, p. 13). In other words, in examining the experiences
of stigma, the first author wanted to better understand what informed how
participants interpreted, made sense of, and responded to these experiences.
The first author also made sure to bring a constructivist orientation to this
work, an epistemology that “does not seek to focus on motivation or indivi-
dual psychologies, but instead seeks to theorize the socio-cultural contexts,
and structural conditions, that enable the individual accounts that are pro-
vided” (Braun & Clarke, 2006, p. 14).
The first author moved through the analysis in the six phases of analysis as
outlined by Braun and Clarke (2006):
(1) Familiarize yourself with the data and formulate some ideas and
thoughts about the data.
(2) Generate initial codes.
(3) Sort through the codes, searching for and then constructing themes.
(4) Review the themes, selecting those that best reflect the data.
(5) “Define and refine” themes, “identifying the ‘essence’ of what each
theme is about (as well as the themes overall), and determining what
aspect of the data each theme captures” (p. 22).
(6) Write the results section.
After becoming familiar with the data and focusing keenly on the partici-
pants’ experiences, particularly around stigma, the first author began the
process by developing inductive codes, described by Boyatzis (1998) as
codes that arise based on how the researcher reads and interprets the data.
These codes were then sorted and grouped by using loose themes that
contextualized the coded data. The first author then constructed solid themes
around those codes. The themes were intentionally kept broad to ensure that
the themes remained reflective of the data and not too interpretative. The
first author then reviewed these themes and then collapsed them once again,
creating even broader thematic categories. In order to increase the rigor and
interpretive value of the research, the first author sought out another
researcher, the second author, to independently review the transcripts,
codes, and themes. The first and second author then met to discuss the
final themes. While the second author agreed with how the themes were
developed, he recommended not collapsing the themes into just three
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 259
categories, stating that it potentially undermined the richness of the partici-
pants’ experiences. The first author agreed but felt that having too many
themes did not capture the elements that were shared among experiences.
The authors then decided to include both the three broad, overarching
themes as well as the “richer” themes, listing them as subthemes (see
Table 2).
Participants spoke strongly about the experience and impact of stigma during
the active phases of their depression. Their responses were divided into three
main categories: race/culture-specific aspects of dealing with depression, the
impact of the diagnosis on those suffering from depression, and pushing back
against the stigma. Subthemes of these broader themes are also included and
identified as “aspects,” “consequences,” or “methods.”
Race/culture-specific aspects of depression experience
It is no surprise that many Black individuals experiencing depression fear
being seen as “crazy” and/or “weak.” What is perhaps most surprising is that
for many Black Americans there is a racializing and gendering of “crazy” and
weakness, a notable aspect of how the depression experience is interpreted
and understood. For some respondents, being “crazy” was not only an
indictment on one’s mental status but also of one’s racial status. Janelle
stated that many Blacks characterized other Blacks with depression as
being “crazy like those White people.” Respondents believed that weakness
was a characterization entangled with race and sometimes gender. Shalesa
believes that there is an “image of the strong Black woman or the strong
Black man” that many Black Americans feel they must emulate. Janelle feels
that it is this image that forces some people to “deny” or “not acknowledge”
the fact they are depressed, “especially Black women.” She went on to say that
it is “’cause it’s always ‘oh, you’re the matriarch, you have to be the strong
one’ and you know, it’s kinda like you’re not allowed to be depressed
Table 2. Major themes and subthemes.
Major Themes Subthemes
Race/Culture-Specific Aspects of Depression
Experience
Racializing and Gendering of Depression
Stigma Felt More Strongly in Black Communities
Experience and Disclosure of Depression as Taboo
Impact of Stigma Diminished Help-Seeking
“Outted and Ousted”
Shame and Internalized Stigma
Pushing Back Against the Stigma Seeking Treatment
Disclosing to Other Black Americans
Study Participation
260 R. D. CAMPBELL AND O. MOWBRAY
almost.” Kamille felt that her “image of what depression is” was shaped by
the cultural messages she received about depression. She said that it was not
until later that she came to understand that depression affected many and the
image was not simply “White women who can’t deal with life.” Keith had this
to say when describing his depression:
I don’t know if you asked, another manifestation of the depression is tears. I’d be
just driving, you know, and then all of a sudden I’m crying. I don’t know if it’s a
song on the radio, but the tears, you know? I’m just, I’m thinking about this or
that, and for an African-American male to do that in public, you feel very
vulnerable, you know? It’s a sign of weakness. You know, it’s like an Achilles
heel. And especially if you came up in the streets, that’s something you never want
to do because, you know, it’s like that little triple wildebeest: you don’t stand a
chance. [laughs]
Although the stigma attached to mental illness is not unique to Black
Americans, many respondents felt that their experiences were indeed differ-
ent from those of other races, suggesting that stigma may be stronger in
Black communities. Richard observed that “not only in the general populace
is there a stigma, but it’s there in our community on a very strong level.”
Other respondents also shared this belief, indicating another strong aspect of
the depression experience of Black Americans. Denise stated that depression
and treatment were “even more stigmatized in the Black community than in
any other community.” Kamille’s response suggested that individuals with
predominately Black social networks may be less willing to discuss being
depressed than those with predominantly White or mixed-race networks:
Like, the core of people that I socially interact with are Black. I think that would’ve
made me less likely to come and be like, ‘hey, guess what? I’m depressed.’ So, I
think that made me even more hesitant to broach the subject or to really talk about
the real deal about how I was feeling.
Similarly, other respondents believed that people in White communities
were more accepting of depression. Laura stated that she believed “for
Caucasians, in general, [depression] is more accepted.” Elisa agreed, stating,
[Whites] would be understanding and sympathetic towards it, and they would
listen to me and not tell me, ‘oh you should just do this’ or ‘you should get over it’
and things like that ’cause I feel like that’s how a lot of my family members reacted
to my mom when she talked about her depression.
Respondents also described a cultural environment in which symptoms of
depression, including increased suicidal ideation, were not only stigmatized.
There also were strong taboos concerning discussion of the topic. Therefore,
the taboo nature of disclosure is another important aspect of Black
Americans’ depression experiences. Laura stated that depression “isn’t really
a widely talked about issue in the African-American community” and that “it
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 261
was pretty taboo within the African-American community itself.” Janelle also
talked about depression as being “a taboo subject in the Black community.”
Miller also used the term “taboo” to describe why be believes Black
Americans found it difficult to talk about or seek help for their depressed
feelings. Kamille stated that the taboo around mental health problems
extends beyond depression to suicide: “I think it’s extremely taboo to talk
about suicide, or like there’s a perception that Black people don’t commit
suicide, like it’s not even an option, you know?”
The impact of stigma
Many of the respondents stated that the impact of stigma can have real
consequences concerning whether an individual seeks help for depression.
Richard believed stigma diminished help-seeking for Blacks with depression,
stating, “I mean, they don’t even want to go to a doctor. [laughter] The
stigma again. [Others will] think you are weak or, you know, different.” He
went on to say that even when an individual does seek help, “all they’re
gonna do is give me some medication and people are going to tease me.” In
addition, Sidney said that Blacks might be less willing to discuss their mental
health problems because they “don’t want people talking about them” or they
“don’t want people to treat you differently, like you have a problem.”
These fears of being stigmatized led some respondents to hide their
depression. Carol felt that the stigmatization of depression and its symptoms
may make many individuals reluctant to “tell anyone ’cause people might
think [they’re] crazy.” Also, depressed Black Americans may have more to
lose if/when their depression diagnoses were revealed. Kamille addressed the
consequence of being what the first author would like to term as “outted and
ousted,” stating the following:
I think that my perception of or my experience of being a Black woman in a
department full of White people, one of my fears was that I wouldn’t be taken
seriously as a student if it was widely known that I was depressed. Or there would
be a perception that I can’t perform to the best of my ability or get things done.
And I think my experience as a Black woman colors that perception of the
situation for me. I think I didn’t want it to be another strike, so to speak, on my
record. And I feel like they might be able, or certain people might be willing to give
Suzie [laughs] White grad student more leeway than they would me automatically,
so for me coming in talkin’ about I’m depressed, and I need special concessions, I
think I just thought that I wasn’t likely to get that. In fact, I thought that people
might just look at me as I’m just incompetent and don’t belong here.
Kamille’s thoughts highlight an issue that is shared between Blacks and
Whites, although for different reasons: that Blacks are not expected to be
depressed. From national studies that applaud Black Americans’ ability to
“sustain a high degree of mental health” (USDHHS, 2001, p. 54) to
262 R. D. CAMPBELL AND O. MOWBRAY
laypersons who believe that Black Americans have endured so much histori-
cally (Thompson, Bazile, & Akbar, 2004), there is almost an expectation that
Black Americans can deal with anything. For Kamille, as a Black woman, a
failure to keep up appearances or “to appear a certain way one hundred
percent of the time” had very real consequences in the wider, Whiter world:
“. . . people aren’t gonna take me seriously. Or they’re gonna try to prevent
me from, you know, finishing this degree or whatever.” For others, it meant
strong assumptions about one’s identity. Laura stated the following:
I think with Caucasians, it’s easier for them to admit that they’re depressed, but for
African-Americans, there’s more of a sense of a struggle within the community,
and to be depressed for what seems like no reason seems almost shameful.
This idea that depression can be shameful has serious implications for
those who are depressed. Taboos concerning the symptoms of depression as
well as the shame associated with seeking treatment can create another
consequence of stigma—internalized stigma. For example, when Margie
was asked about how she felt about the statement that individuals with
depression were weak, she stated,
I still think, unfortunately, there are sort of stereotypes and myths that are well-
ingrained in our society and therefore well-ingrained, some more than others
though, especially the one about depressive people are weak. I mean, just like we
have internalized racism, I think I have internalized feelings about myself based on
having emotional problems. And I think they’re very hard to overcome, if you
could ever overcome them. I mean, you’re kind of like always recovering. Sort of
like a recovering alcoholic, you’re kind of recovering stigma, you know?
This internalized view can damage both the individual and others around
them. Shalesa believed that her father-in-law’s negative views about indivi-
duals with depression came from his own “shame” around taking
antidepressants.
Pushing back against the stigma
Many of the respondents reported that even though they felt individuals with
depression are stigmatized, they found ways to push back against the stigma.
One method many participants discussed was simply seeking treatment.
Richard said that he “learned you’ve got to look over the stigma, look over
the whatever, because you’re doing something to help yourself.” Denise said,
I kinda felt weird about having formal sessions with a psychotherapist initially
because I feel like depression is really stigmatized and I feel like if people know that
you’re going to a psychologist, they think that you’re crazy, so I think now I feel
better about that kinda thing.
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 263
Others believed stigma must be confronted by disclosing their depression
to other Blacks. Thus, sharing one’s experience became another method of
combating stigma. Janelle said that “once you talk about it, get it out on the
table, and people start to understand it, I think it’ll be less of a taboo.”
Kamille talked about how she eventually told her family and close friends
about her depression. She said,
It almost seemed like it was becoming like a big secret, and I didn’t want it to be
like that. I didn’t want to have something else on my plate where I accidentally
mentioned going to therapy or, like, taking medication, and people are like ‘what?’
You know? Like, ‘when did that happen?’ So, I didn’t want it to be like a secret
because I didn’t feel like I had anything to hide from them.
Coming to terms with her depression helped her realize and acknowledge
“I’m still Black. I’m still the same person I was before. It doesn’t make me
weak.” Some respondents recognized that their personal attempts to push
back against the stigma could also push against the stigma that existed in
Black communities at large. Elisa described her process:
I think the fact that. . . after a while I insisted on talking about it as if it was
something real and not like some whiny BS like my mother might’ve said, like
‘White people talk about stuff like that.’
Derrick addressed the image of strength and said that treatment could
support or aid those cultural images: “If depression is a feeling that’s going to
make you ill, then treatment is going to come in and show you how to avoid
it, or correct it, or make you feel stronger.” Drucilla challenged the image of
people with depression as “crazy” people who see a “shrink,” saying that
therapy is simply “talking to somebody ’cause you’re depressed.”
A few respondents expressed that this study and their participation in it
was also a method of challenging the stigma around mental illness. Carol
said, “It’s really a great thing that you’re doing, and it’s very needed.” Shalesa
“really appreciate[d] this opportunity to talk about depression because it
doesn’t come along often enough at all.” She went on to say that she was
“really excited” when she saw the flyer for the study “because [she] wish[ed]
that [she] had access to something years ago when [she] was really struggling,
and [she] hope[d] that whatever becomes of this research that it, it helps
someone else.” However, one respondent suggested that instead of focusing
on differences in experiences, we should instead focus on the universality of
the disorder:
I don’t know why do society in general have to divide, redline, different cultures,
you know? It just seems like to me that everybody gets depressed every now and
then or whatever. Why would another culture be depressed more than or be more
susceptible to depression than another? I don’t understand why society deal with
people like that.
264 R. D. CAMPBELL AND O. MOWBRAY
Stigma is a major barrier keeping many people with mental health problems
out of services. For Black Americans, the impact of stigma can be more
detrimental in that it is not the only marginalizing marker they must endure
(Bolden & Wicks, 2005; Conner, Copeland, et al., 2010; Gary, 2005; Matthews
et al., 2006). Through this study, we sought to learn more about Black
Americans’ depression experience. We found that stigma was a major part
of that experience and discovered the various ways stigma is both experi-
enced and combated.
The results of this study highlighted the stigmatizing messages Black
Americans can receive about depression, help-seeking, and treatment.
Respondents discussed their belief that Black Americans in general see people
with depression as “crazy” and/or “weak.” As shown in previous research
(Alvidrez et al., 2008; Calloway, 2006; Mishra et al., 2009), these beliefs made
it difficult for those experiencing depressive symptoms to seek help for their
problems. Respondents’ reports of hiding their symptoms or the fact that they
sought treatment to avoid being labeled or treated differently echoed findings
of other studies (Conner, Lee, et al., 2010; Matthews et al., 2006; Mishra et al.,
2009). Some respondents attributed the stigmatization of those with depres-
sion to a lack of education about the disorder. However, other respondents
believed that individuals themselves can hold these beliefs, internalize them,
and thus, believe their social statuses and/or standings in their community are
at risk or being challenged. It was these fears that respondents stated forced
some to hide their depression and others to turn to those outside of their Black
social networks and communities to seek support.
One of the more novel ideas to emerge from this study was the fact that
the negative characteristics of those with depression were often racialized
and/or gendered. A number of respondents talked about how being “crazy,”
“weak,” or depressed itself were traits of other races, namely Whites and
particularly White women. The idea that depression was something that did
not, or should not, affect Blacks was clearly implied if not stated outright.
While previous research by Borum (2012) identified this racializing and
gendering of depression as protective for Black women, these participants
found that these views of depression and subsequent help-seeking compli-
cated their depression and help-seeking experiences. By using racialized and
gendered language to describe depression and the depressed, some were
forced not only to contemplate the existence of a mental illness, but also
their identity as Black American men and women. For them, it was not
enough to be weak or crazy because of their illness, but also less of a “Black”
man or woman. These characterizations of and responses to depression made
many respondents believe that depression and treatment were more stigma-
tized in Black communities. They also shared that talk of depression and
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 265
treatment was taboo in Black communities, creating feelings of shame in
some. The oppressive environment created for many respondents forced
some to turn to individuals outside of their family, friends, and community
to seek understanding and consolation.
Although the belief that depression was more stigmatized in Black com-
munities is similar to findings in previous research (Bailey et al., 2009;
Conner, Copeland, et al., 2010; Moran, 2004), several findings presented
here go beyond the current literature. For example, our interviews suggest
that Blacks may resort to turning to those outside of their families and racial/
ethnic communities for support while hiding their depression within the
community to maintain social ties. This suggests that Black Americans with
depression might believe they must navigate multiple identities in various
spaces, which can in turn add to their (di)stress. Respondents also discussed
the importance of fighting the stigma experienced and seeking help when
needed. Some talked about how they combated stigma at the individual or
personal level, but they also expressed the need for community-level inter-
vention to educate people about depression and the importance of treatment.
Many said that studies like this aided that goal.
Limitations and future directions
Even though this study offers a substantial contribution to the study of the
stigmatization of depression in Black communities, it is not without its
limitations. One major limitation of this study is that respondents were
allowed to self-identify as depressed. Although we tried to use participation
criteria that reflected diagnostic criteria, we might have captured individuals
who would not be viewed as “depressed” in a clinical setting. Another
limitation is that beliefs about Black communities were entirely based on
the perceptions of the respondents. Even though it is important to know
about how individuals with depression believe they are being viewed and
treated, the work did not interview members of the wider Black American
community, including those who did not identify as having depression.
The limitations of this study not only illuminate areas of concern but
also highlight directions to take future work in this area. As previously
mentioned, future directions in research may extend beyond talking to
depression sufferers themselves and include the broader community’s
beliefs about depression and help-seeking. Although it is important to
understand the perceptions Black Americans diagnosed with depression
have about depression and treatment, it is equally important to assess the
actual community climate about depression and treatment by talking to
those in the community who have not experienced depression. These
climate-assessment studies can also be replicated to examine the “tempera-
ture” within families and in mental health care settings. Future studies
266 R. D. CAMPBELL AND O. MOWBRAY
should also look more closely at the practice of going outside of Black
social networks and accessing spaces that are non-Black for depression
support as well as how Blacks with depression negotiate their positions in
both spaces.
Conclusion
Even though the stigma of mental illness is a barrier for many seeking services
for mental health treatment, it is a particularly strong impediment for Black
Americans. It is important that research continue to better understand Blacks’
experiences with depression and stigma both inside and outside of Black com-
munities. By understanding their perceptions of their community and its beliefs
about them we can look toward designing and implementing interventions that
reduce stigma and increase understanding and support within Black commu-
nities for those that struggle with depression and other mental health problems.
Abdullah, T., & Brown, T. L. (2011). Mental illness stigma and ethnocultural beliefs, values,
and norms: An integrative review. Clinical Psychology Review, 31(6), 934–948. doi:10.1016/
j.cpr.2011.05.003
Alvidrez, J., Snowden, L. R., & Kaiser, D. M. (2008). The experience of stigma among Black
mental health consumers. Journal of Health Care for the Poor and Underserved, 19, 874–
893. doi:10.1353/hpu.0.0058
Anglin, D. M., Link, B. G., & Phelan, J. C. (2006). Racial differences in stigmatizing attitudes
toward people with mental illness. Psychiatric Services, 57(6), 857–862. doi:10.1176/
ps.2006.57.6.857
Bailey, R. K., Blackmon, H. L., & Stevens, F. L. (2009). Major depressive disorder in the
African American population: Meeting the challenges of stigma, misdiagnosis, and treat-
ment disparities. Journal of the National Medical Association, 101(11), 1084–1089.
doi:10.1016/S0027-9684(15)31102-0
Black, H. K., Gitlin, L., & Burke, J. (2011). Context and culture: African-American elders’
experiences of depression. Mental Health, Religion & Culture, 14(7), 643–657. doi:10.1080/
13674676.2010.505233
Bolden, L., & Wicks, M. N. (2005). Length of stay, admission types, psychiatric diagnoses, and
the implications of stigma in African Americans in the nationwide inpatient sample. Issues
in Mental Health Nursing, 26, 1043–1059. doi:10.1080/01612840500280703
Borum, V. (2012). African American women’s perceptions of depression and suicide risk and
protection: A womanist exploration. Affilia: Journal of Women and Social Work, 27(3),
316–327. doi:10.1177/0886109912452401
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code
development. Thousand Oaks, CA: Sage Publications.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in
Psychology, 3(2), 77–101. doi:10.1191/1478088706qp063oa
Burlew, A. K. (2003). Research with ethnic minorities: Conceptual, methodological, and
analytical issues. In G. Bernal, J. E. Trimble, A. K. Burlew, & F. T. L. Leong (Eds.),
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 267
http://dx.doi.org/10.1016/j.cpr.2011.05.003
http://dx.doi.org/10.1016/j.cpr.2011.05.003
http://dx.doi.org/10.1353/hpu.0.0058
http://dx.doi.org/10.1176/ps.2006.57.6.857
http://dx.doi.org/10.1176/ps.2006.57.6.857
http://dx.doi.org/10.1016/S0027-9684(15)31102-0
http://dx.doi.org/10.1080/13674676.2010.505233
http://dx.doi.org/10.1080/13674676.2010.505233
http://dx.doi.org/10.1080/01612840500280703
http://dx.doi.org/10.1177/0886109912452401
http://dx.doi.org/10.1191/1478088706qp063oa
Handbook of racial & ethnic minority psychology (pp. 179–197). Thousand Oaks, CA: Sage
Publications.
Calloway, N. C. (2006). The mental health of Black men: A problem of perception. Challenge,
12(1), 55–65.
Carpenter-Song, E., Chu, E., Drake, R. E., Ritsema, M., Smith, B., & Alverson, H. (2010).
Ethno-cultural variations in the experience and meaning of mental illness and treatment:
Implications for access and utilization. Transcultural Psychiatry, 47(2), 224–251.
doi:10.1177/1363461510368906
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative
analysis. Thousand Oaks, CA: Sage Publications.
Conner, K. O., Copeland, V. C., Grote, N. K., Rosen, D., Albert, S., McMurray, M. L.,. . .
Koeske, G. (2010). Barriers to treatment and culturally endorsed coping strategies among
depressed African-American older adults. Aging & Mental Health, 14(8), 971–983.
doi:10.1080/13607863.2010.501061
Conner, K. O., Lee, B., Mayers, V., Robinson, D., Reynolds, C. F., III, Albert, S., & Brown, C.
(2010). Attitudes and beliefs about mental health among African American older adults
suffering from depression. Journal of Aging Studies, 24, 266–277. doi:10.1016/j.
jaging.2010.05.007
Corrigan, P. W. (2004). How stigma interferes with mental health care. American
Psychologist, 59(7), 614–625. doi:10.1037/0003-066X.59.7.614
Cruz, M., Pincus, H. A., Harman, J. S., Reynolds, C. F., III, & Post, E. P. (2008). Barriers to
care-seeking for depressed African Americans. The International Journal of Psychiatry in
Medicine, 38(1), 71–80. doi:10.2190/PM.38.1.g
Gary, F. A. (2005). Stigma: Barrier to mental health care among ethnic minorities. Issues in
Mental Health Nursing, 26(10), 979–999. doi:10.1080/01612840500280638
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for
qualitative research. Chicago, IL: Aldine.
Goffman, E. (1963). Stigma: Notes on the management of spoiled identity. New York, NY:
Simon & Schuster, Inc.
Link, B. G., & Phelan, J. C. (2001). Conceptualizing stigma. Annual Review of Sociology, 27(1),
363–385. doi:10.1146/annurev.soc.27.1.363
Matthews, A. K., Corrigan, P. W., Smith, B. M., & Aranda, F. (2006). A qualitative explora-
tion of African-Americans’ attitudes toward mental illness and mental illness treatment
seeking. Rehabilitation Education, 20(4), 253–268. doi:10.1891/088970106805065331
Mishra, S. I., Lucksted, A., Gioia, D., Barnet, B., & Baquet, C. R. (2009). Needs and
preferences for receiving mental health information in an African American focus group
sample. Community Mental Health Journal, 45, 117–126. doi:10.1007/s10597-008-9157-4
Moran, M. (2004). Culture, history can keep Blacks from getting depression treatment.
Psychiatric News, 39(11), 12–50. doi:10.1176/pn.39.11.0390012
Rao, D., Feinglass, J., & Corrigan, P. W. (2007). Racial and ethnic disparities in mental illness
stigma. The Journal of Nervous and Mental Disease, 195, 1020–1023. doi:10.1097/
NMD.0b013e31815c046e
Rusch, L. C., Kanter, J. W., Manos, R. C., & Weeks, C. E. (2008). Depression stigma in a
predominantly low income African American sample with elevated depressive symptoms.
Journal of Nervous Mental Disorders, 196, 919–922.
Stewart, M., Makwarimba, E., Barnfather, A., Letourneau, N., & Neufeld, A. (2008).
Researching reducing health disparities: Mixed methods approaches. Social Science &
Medicine, 66, 1406–1417. doi:10.1016/j.socscimed.2007.11.021
268 R. D. CAMPBELL AND O. MOWBRAY
http://dx.doi.org/10.1177/1363461510368906
http://dx.doi.org/10.1080/13607863.2010.501061
http://dx.doi.org/10.1016/j.jaging.2010.05.007
http://dx.doi.org/10.1016/j.jaging.2010.05.007
http://dx.doi.org/10.1037/0003-066X.59.7.614
http://dx.doi.org/10.2190/PM.38.1.g
http://dx.doi.org/10.1080/01612840500280638
http://dx.doi.org/10.1146/annurev.soc.27.1.363
http://dx.doi.org/10.1891/088970106805065331
http://dx.doi.org/10.1007/s10597-008-9157-4
http://dx.doi.org/10.1176/pn.39.11.0390012
http://dx.doi.org/10.1097/NMD.0b013e31815c046e
http://dx.doi.org/10.1097/NMD.0b013e31815c046e
http://dx.doi.org/10.1016/j.socscimed.2007.11.021
Thompson, V. L. S., Bazile, B., & Akbar, M. (2004). African Americans’ perceptions of
psychotherapy and psychotherapists. Professional Psychology: Research and Practice,
35(1), 19–26. doi:10.1037/0735-7028.35.1.19
U.S. Department of Health and Human Services (USDHHS). (1999). Mental health: A report
of the surgeon general. Rockville, MD: National Institute of Mental Health.
U.S. Department of Health and Human Services (USDHHS). (2001). Mental health: Culture
race and ethnicity—a supplement to mental health: A report of the surgeon general.
Rockville, MD: U.S. Department of Health and Human Services, Public Health Service,
Office of the Surgeon General.
Yang, L. H., Kleinman, A., Link, B. G., Phelan, J. C., Lee, S., & Good, B. (2007). Culture and
stigma: Adding moral experience to stigma theory. Social Science & Medicine, 64(7), 1524–
1535. doi:10.1016/j.socscimed.2006.11.013
JOURNAL OF ETHNIC & CULTURAL DIVERSITY IN SOCIAL WORK 269
http://dx.doi.org/10.1037/0735-7028.35.1.19
http://dx.doi.org/10.1016/j.socscimed.2006.11.013
- Abstract
Introduction
Stigma in Black communities
Specific study aims
Method
Sample/study participants
Interview procedures
Analysis
Results
Race/culture-specific aspects of depression experience
The impact of stigma
Pushing back against the stigma
Discussion
Limitations and future directions
Conclusion
References
Schizophrenia Research 173 (2016) 45–
53
Contents lists available at ScienceDirect
Schizophrenia Research
journal homepage: www.elsevier.com/locate/schres
Therapeutic intervention for internalized stigma of severe mental illness:
A systematic review and meta-analysis
Hector W.H. Tsang ⁎, S.C. Ching, K.H. Tang, H.T. Lam, Peggy Y.Y. Law, C.N. Wan
Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong
⁎ Corresponding author at: Neuropsychiatric Rehabilita
Rehabilitation Sciences, the Hong Kong Polytechnic Un
Hong Kong.
E-mail address: hector.tsang@polyu.edu.hk (H.W.H. Ts
http://dx.doi.org/10.1016/j.schres.2016.02.013
0920-9964/
© 2016 Published by Elsevier B.V.
a b s t r a c t
a r t i c l e i n f o
Article history:
Received 3 August 2015
Received in revised form 5 February 2016
Accepted 5 February 2016
Available online 9 March 2016
Objective: Internalized stigma can lead to pervasive negative effects among people with severe mental illness
(SMI). Although prevalence of internalized stigma is high, there is a dearth of interventions and meanwhile a
lack of evidence as to their effectiveness. This study aims at unraveling the existence of different therapeutic in-
terventions and the effectiveness internalized stigma reduction in people with SMI via a systematic review and
meta-analysi
s.
Methods: Five electronic databases were searched. Studies were included if they (1) involved community or hos-
pital based interventions on internalized stigma, (2) included participants who were given a diagnosis of
SMI N 50%, and (3) were empirical and quantitative in nature.
Results: Fourteen articles were selected for extensive review and five for meta-analysis. Nine studies showed sig-
nificant decrease in internalized stigma and two showed sustainable effects. Meta-analysis showed that there
was a small to moderate significant effect in therapeutic interventions (SMD = −0.43; p = 0.003). Among the
intervention elements, four studies suggested a favorable effect of psychoeducation. Meta-analysis showed
that there was small to moderate significant effect (SMD = −0.40; p = 0.001).
Conclusion: Most internalized stigma reduction programs appear to be effective. This systematic review cannot
make any recommendation on which intervention is more effective although psychoeducation seems most
promising. More Randomized Controlled Trials (RCT) on particular intervention components using standard out-
come measures are recommended in future studies.
© 2016 Published by Elsevier B.V.
Keywords:
Psychoeducation
Severe mental illness
Self-stigma
Internalized stigma
Psychosocial intervention
1. Introduction
Internalized stigma, also called self-stigma, is the process of adopting
the public’s stigmatizing opinions into own thoughts. Its prevalence
among people with severe mental illness (SMI) stands at 41.7%
(Gerlinger et al., 2013). Meta-analysis (Livingston and Boyd, 2010)
found that high levels of internalized stigma were significantly correlat-
ed with hopelessness, poorer self-esteem, reduced empowerment/mas-
tery, and reduced self-efficacy. Internalized stigma significantly
predicted poor social functioning over time (Fung et al., 2008; Tsang
et al., 2010; Yanos et al., 2012a, 2012b). Higher internalized stigma
was also associated with poorer quality of life on all the domains of
WHOQOL-Brief (Mosanya et al., 2014). In addition, higher levels of in-
ternalized stigma were significantly related to more severe psychiatric
symptoms, poorer treatment adherence, and lower utilization of mental
health services (Rüsch et al., 2009). Up to 20% of people may even dis-
continue treatment prematurely due to internalized stigma (Corrigan,
tion Laboratory, Department of
iversity, Hung Hom, Kowloon,
ang).
2014). Unfortunately, poorer treatment adherence was related to
poorer treatment outcomes, more re-hospitalization, and increased
health costs (Lacro et al., 2002). All of the above studies suggest that in-
ternalized stigma has tremendous negative impact on functional out-
comes of people with SMI and increases societal burden in taking care
of these individuals.
Given the close negative relationship between internalized stigma
and recovery (Yanos et al., 2008), interventions that attempt to reduce
it among those with SMI so as to improve their prospect for recovery
are important both in psychiatric rehabilitation and community
integration.
Although a conceptual framework (Hayward and Bright, 1997) on
internalized stigma reduction is available, there is a dearth of interven-
tions and meanwhile a lack of evidence as to their effectiveness. The
programs usually consist of a combination of different elements of inter-
vention such as psychoeducation, cognitive behavioral therapy, and so-
cial skills training. Currently there are no widely accepted treatment
protocols on internalized stigma reduction. Numerous studies on inter-
nalized stigma were published in the past few years. There are a few
reviews (e.g., Mittal et al., 2012) on this area. But no attempt on meta-
analysis has ever been conducted. To fill the knowledge gap, this
systematic review and meta-analytical study focuses on studies which
http://crossmark.crossref.org/dialog/?doi=10.1016/j.schres.2016.02.013&domain=pdf
mailto:hector.tsang@polyu.edu.hk
http://dx.doi.org/10.1016/j.schres.2016.02.013
www.elsevier.com/locate/schres
46 H.W.H. Tsang et al. / Schizophrenia Research 173 (2016) 45–53
are randomized control trials, clinical trials, or experimental trials in na-
ture and reported therapeutic interventions to reduce internalized stig-
ma in people with SMI.
2. Methods
The PRISMA Statement Criteria (Liberati et al., 2009) was adopted
throughout the review to report our results.
2.1. Data sources
Systematic search was done to locate studies from PubMed, PsychInfo
(1806-Present), SCI (1970-Present), SSCI (1970-Present), and Scopus on
October 3rd, 2014. The search terms were generated based on the four
types of eligibility criteria mentioned above. They are presented in
Supplementary Appendix 1. In addition, emails were sent to a few
prominent researchers on mental illness stigma that had close collabo-
rations with the first author. One replied and suggested an in press arti-
cle that we missed in the systematic search and eventually included in
the systematic review.
2.2. Study selection
Studies were included if the following criteria were met: (1) studies:
randomized clinical trials, clinical trials, and experimental studies
studying internalized stigma reduction in people with SMI which was
operationally defined as mental illness having a chronic course and
leading to significant social and occupational dysfunction such as
Schizophrenia, Psychotic disorder, Psychosis, Delusional disorder,
Schizoaffective, Bipolar disorder and Personality disorder; (2) partici-
pants: more than 50% studies participants were given a diagnosis of
SMI,; (3) intervention: studies that compared community or hospital
based therapeutic interventions with conventional treatment; (4) out-
come measures: studies that used validated instruments for screening
and assessing the severity of internalized stigma. Qualitative studies
and literature reviews were excluded.
There were no limitations in the follow up period. Search was re-
stricted to publications in English but there was no limit to the years
of publication.
Two reviewers independently conducted the search in five electron-
ic databases. The results were screened on titles and abstracts by all re-
viewers independently. Disagreements were resolved via discussion
between reviewers with the facilitation of the corresponding author.
2.3. Data extraction and analysis
Data extracted from selected studies included: (1) characteristics of
the study (e.g., aims, study design, intervention setting, participants’
characteristics, randomization procedures, inclusion and exclusion
criteria, etc.); (2) outcome measures (e.g., outcome measure instru-
ments, assessment period, dropout rate, etc.); (3) interventions
(e.g., intervention approach, trainers’ quality, etc.); and (4) study results
(e.g., significant effects, sustainability, etc.). To ensure accuracy, five re-
viewers assessed two to three studies independently and the data were
cross-checked by another reviewer. Also, emails were sent to the
authors for clarifying missing or unclear data.
2.4. Assessment of methodological quality
The methodological risk of bias for each trial was evaluated using the
risk of bias table (Higgins and Green, 2011). Five reviewers worked in-
dependently to determine the adequacy of randomization sequence,
blinding of patients and assessors, and the extent of follow up. Disagree-
ments were resolved in discussion.
2.5. Data synthesis and analysis
Review Manager 5.3, developed by the Cochrane Collaboration
(2014), was used for meta-analysis of the results of included studies.
The outcomes across the trials were expressed in Cohen’s D. Standard-
ized mean differences (SMDs) were calculated for the pooled effects.
SMDs were interpreted using the “rule of thumb”: 0.2 represents a
small effect, 0.5 represents a moderate effect, and 0.8 represents a
large effect (Cohen, 1988). Heterogeneity was tested with an X2 test. I2
was also reported. I2 statistic N75% was considered to have high degree
of heterogeneity while I2 statistic of 25%–50% was considered to have a
low degree of heterogeneity (Higgins et al., 2003). We used random-
effects model for heterogeneity (p b 0.05) and a fixed effects model
for heterogeneity (p N 0.05). Sensitivity analyses were conducted with
psychoeducation group and professional-led or peer-led intervention.
Publication bias was examined using funnel plot. A value of p b 0.05
was considered statistically significant.
Since the outcome of internalized stigma reduction program was
assessed by more than one tool in a trial, only the primary outcome
(i.e., internalized stigma reduction) was included in the meta-analysis.
Emails were sent to corresponding authors for clarifying missing data.
3. Results
3.1. Results of literature search
Eight hundred nineteen articles were retrieved. Three hundred
eighty duplicated and 421 irrelevant articles were excluded after initial
screening of title and abstract. Full reports of 22 studies were acquired
and eight were further excluded for the following reasons: (1) not a
clinical trial, (2) b50% participants given diagnoses of SMI, (3) explor-
atory research, and (4) duplicated studies (Fig. 1). (See Figs. 2– 4.)
3.2. Description of included studies
Fourteen studies, including seven RCTs (Corrigan et al., 2015;
Çuhadar and Çam, 2014; Fung et al., 2011; McCay et al., 2007;
Russinova et al., 2014; Rüsch et al., 2014; Yanos et al., 2012a, 2012b),
three controlled clinical trials (Roe et al., 2014; Sibitz et al., 2013;
Uchino et al., 2012), and four uncontrolled studies without a control
group (Costain et al., 2014; Lucksted et al., 2011; Lysaker et al., 2012;
Staring et al., 2013) met inclusion criteria. These studies were originated
from nine countries across Americas, Europe, and Asia. Most programs
adopted psychoeducation approach with inclusion of a combination of
other components such as CBT, social skills training, goal attainment
program, and narrative therapy. The duration of these programs ranged
from 10 to 40 sessions (Russinova et al., 2014; Sibitz et al., 2013). Other
than the above more conventional therapeutic elements, there were
two evolving innovative interventions. Both of them were peer led
group interventions. One was Coming Out Proud (Corrigan et al.,
2015; Rüsch et al., 2014). It consisted of group discussion focusing on
topics of secrecy and disclosure of own mental illness. The other was
Photovoice (Russinova et al., 2014). In the group, individuals photo-
graph objects or events in their daily lives were used to generate narra-
tives for group discussion. The characteristics of included studies are
summarized in Table 1.
Participants were given a diagnosis of schizophrenia, schizophrenia
spectrum disorder, bipolar disorder, or major mood disorder. One
study included patients with schizophrenia only (Fung et al., 2011)
and one included patients with bipolar disorder only (Çuhadar and
Çam, 2014). Participants were mostly given a diagnosis by psychiatrists
according to DSM IV or ICD-10 (Yanos et al., 2012a, 2012b). Two studies
recruited participants according to their self-reported diagnosis. Only
one study reported the use of the structured interview procedure in ver-
ifying the diagnosis of the research participants (Lysaker et al., 2012).
Fig. 1. Selection process of included studies.
47H.W.H. Tsang et al. / Schizophrenia Research 173 (2016) 45–53
Sample size of the studies varied from 21 to 205 participants, with a
total of 1131 participants including 879 participants in the experimental
groups and 452 participants in the control groups.
Controlled group design was employed in ten studies. Treatment as
usual (TAU) was applied in seven studies (McCay et al., 2007; Roe et al.,
2014; Russinova et al., 2014; Rüsch et al., 2014; Sibitz et al., 2013;
Uchino et al., 2012; Yanos et al., 2012a, 2012b), conventional treatment
group (i.e., newspaper reading) in one study (Fung et al., 2011), no ac-
tive treatment in one study (Çuhadar and Çam, 2014), and waitlist con-
trol in one study (Corrigan et al., 2015). All studies provided information
Fig. 2. A forest plot of the meta-analysis of three studies comp
on the duration of treatment in patient intervention groups. Thirteen in-
cluded studies examined the effect of internalized stigma reduction and
one examined the effect of negative symptoms immediately following
the internalized stigma reduction intervention. In addition, seven stud-
ies (Corrigan et al., 2015; Costain et al., 2014; Fung et al., 2011; Lysaker
et al., 2012; Russinova et al., 2014; Rüsch et al., 2014; Yanos et al., 2012a,
2012b) examined the sustainability of intervention effect after follow-
up periods ranging from three weeks to six months.
Regarding outcome measures, internalized stigma was assessed as
the primary outcome in all the included studies except one which
aring psychoeducation to TAU for changes in ISMI score.
Fig. 3. A forest plot of the meta-analysis of three studies comparing self-stigma reduction program to TAU for changes in ISMI total score.
48 H.W.H. Tsang et al. / Schizophrenia Research 173 (2016) 45–53
treated internalized stigma as a mediator of dysfunctional beliefs
(Staring et al., 2013). The internalized stigma scales applied in the stud-
ies included Internalized Stigma of Mental Illness (ISMI) (Costain et al.,
2014; Çuhadar and Çam, 2014; Lucksted et al., 2011; Lysaker et al.,
2012; Roe et al., 2014; Russinova et al., 2014; Rüsch et al., 2014; Sibitz
et al., 2013; Yanos et al., 2012a, 2012b), short form of Self-stigma of
Mental Illness Scale (SSMIS) (Corrigan et al., 2015), Chinese Self-
stigma of Mental Illness Scale (CSSMIS) (Fung et al., 2011), Link Per-
ceived Stigma Questionnaire (LPSQ) (McCay et al., 2007), and
Japanese version of Social Distance Scale (SDS-J) (Uchino et al., 2012).
Other outcome measurements were also applied to assess the interven-
tion effect, for examples, the Positive and Negative Syndrome Scale
(PANSS) and the Rosenberg Self-Esteem Scale (RSES). These outcome
measures varied substantially across studies.
3.3. Results of individual studies
Nine studies reported significant internalized stigma reduction.
Studies that reported significant effects of therapeutic intervention
other than internalized stigma reduction are shown in Table 1. Seven
studies had follow-up periods but only two studies showed sustainable
effects for one to three months (Corrigan et al., 2015; Russinova et al.,
2014).
3.4. Effects of intervention approaches on internalized stigma
Meta-analysis was performed among studies which used similar
treatment approaches (e.g., psychoeducation, CBT, etc.) and similar out-
come measures. Following this, six controlled studies were identified to
use ISMI as outcome measure. However, one of the studies was eventu-
ally excluded (Çuhadar and Çam, 2014) because it used the Turkish ver-
sion of ISMI which was not comparable to the original version.
Consequently, five trials were included in meta-analysis (Roe et al.,
2014; Russinova et al., 2014; Rüsch et al., 2014; Sibitz et al., 2013;
Yanos et al., 2012a, 2012b).
3.4.1. Comparison 1: psychoeducation element
Four out of seven trials which contained psychoeducation were
RCTs. Two trials indicated significant effects when compared with con-
trolled group with no active treatment (Çuhadar and Çam, 2014) and
TAU (Russinova et al., 2014). The former focused on patients with bipo-
lar disorder and the latter focused on patients with schizophrenia
Fig. 4. A forest plot of the meta-analysis of four studies comparing self-stigma reduct
spectrum disorder and bipolar disorder. ISMI total scores from two
RCTs and one controlled trial were pooled and the pooled SMD
was −0.40 [−0.64, −0.16], indicating a small-moderate effect (p =
0.001). There was a low degree of heterogeneity (I2 = 17%).
3.4.2. Comparison 2: cognitive behavioral therapy element
Two out of five trials containing CBT element were RCTs. Both trials
suggested no intergroup difference when compared to newspaper read-
ing group (Fung et al., 2011) and TAU (Yanos et al., 2012a, 2012b). The
former focused on patients with schizophrenia and the latter focused on
patients with schizophrenia, schizoaffective, and bipolar disorder.
3.4.3. Comparison 3: group discussion element
Three out of the four trials which had group discussion element were
RCTs. These RCTs included innovative interventions, namely photovoice
(Russinova et al., 2014) and Coming Out Proud (COP) (Corrigan et al.,
2015; Russinova et al., 2014). Photovoice was a discussion group on
the selected photographs regarding to stigma while COP was a peer-
led group intervention which empowered people with mental illness
to decide on secrecy and disclosure of illness. When compared to TAU,
the trial on photovoice indicated significant effect (Russinova et al.,
2014). For COP, one of them suggested significant effect on reduction
in internalized stigma when compared to waitlist control (Corrigan
et al., 2015). Another trial, however, suggested no group x time effect
(Rüsch et al., 2014). All of them focused on patients with serious mental
illness.
3.4.4. Comparison 4: social skills training element
One out of two trials that contained social skills training element was
RCT. It suggested no intergroup difference on patients with schizophre-
nia when compared to newspaper reading group (Fung et al., 2011).
3.4.5. Comparison 5: narrative enhancement and cognitive therapy (NECT)
element
This approach focused on the process of telling and sharing personal
narratives about their experience of mental illness. Two out of three tri-
als containing NECT element were RCTs. When compared to TAU, one
trial indicated significant effect (Russinova et al., 2014) while another
suggested no intergroup difference (Yanos et al., 2012a, 2012b). Both
focused on patients with schizophrenia spectrum disorder and bipolar
disorder. Moreover, the former focused on depressive disorder.
ion program to TAU for changes in average of the sum of sub-scales 1–4 of ISMI.
49H.W.H. Tsang et al. / Schizophrenia Research 173 (2016) 45–53
3.4.6. Other intervention approaches
There was one RCT which reported the effect of group psychothera-
py on internalized stigma reduction among individuals with schizo-
phrenia spectrum disorder (McCay et al., 2007). No intergroup
difference was reported. The other two trials were uncontrolled clinical
trials involving genetic counseling (Costain et al., 2014) and vocational
rehabilitation (Lysaker et al., 2012).
3.5. Meta-analysis
Five studies were eligible for meta-analysis (Roe et al., 2014;
Russinova et al., 2014; Rüsch et al., 2014; Sibitz et al., 2013; Yanos
et al., 2012a, 2012b). ISMI included 5 subscales: alienation, stereotype
endorsement, discrimination experience, social withdrawal, and stigma
resistance. The total scores from three psychosocial interventions were
pooled and the pooled SMD was −0.43 [−0.72, −0.14] which indicated
a small-moderate effect (p = 0.003) of psychosocial interventions.
There was a low degree of heterogeneity (I2 = 22%). As two studies ex-
cluded the 5th subscale (i.e., stigma resistance) of ISMI score (Roe et al.,
2014; Rüsch et al., 2014), we pooled the ISMI scores by taking average of
the sum of sub-scales 1–4 from four studies. Similar to the result of total
ISMI score, it indicated a small to moderate effect (SMD = −0.33
[−0.54, −0.12]; p = 0.002). The results from both professional led
and peer led psychosocial intervention were pooled. Result of meta-
analysis showed moderate significant effect professional led interven-
tion (SMD = −0.51 [−0.79, −0.23]; p b 0.001) but not peer led
intervention.
Quality assessment for each trial is presented in Table 2. None of the
studies met all of the criteria for high quality. Inclusion and exclusion
criteria were clearly defined except one trial (Roe et al., 2014). Seven
studies were RCTs with randomization method reported clearly except
one study (Corrigan et al., 2015). Common problems included inade-
quate blinding of patients and outcome assessors, lack of information
on sample size estimation, and intention-to-treat analyses not applied.
4. Discussion
This paper presents the most updated systematic review and meta-
analysis on internalized stigma reduction. Half of the reviewed studies
were published from 2013 to 2015 which were not included in earlier
reviews. The studies were conducted in nine countries across America,
Europe, and Asia which suggest cross-cultural some degree of generaliz-
ability of the results.
As pointed out in the introduction, programs to reduce internalized
stigma consist of many components. Our review reveals the beneficial
effects of the psychoeducation component in internalized stigma reduc-
tion when compared to TAU. For COP and NECT, effects were inconsis-
tent. For CBT and social skills training, RCTs reported no significant
effects.
Interpretation and generalization of results should be cautiously
treated due to limited number and methodological problems of studies.
Because of bias among studies, the results of meta-analysis should also
be read with caution. The studies used different diagnostic criteria for
participants, including self-reported and diagnoses given by psychia-
trists. Moreover, the proportion of participants with SMI was slightly
less than 50% in the control group in one of the studies (Rüsch et al.,
2014). Both of these factors limit the comparability between results
for the participants with SMI in the studies. The baseline internalized
stigma level among participants varied, with mean total scores of ISMI
at baseline ranging from 1.63 to 2.23 out of 4.0. Although there does
not have any evidence, this variation in baseline level of internalized
stigma might affect the effectiveness of programs and hence the compa-
rability of effectiveness between different studies. Some of the studies
only presented group × time effect or main effect. As intergroup differ-
ence was not presented in all studies, the significance of between group
differences was unclear. Randomization was involved only in seven
studies. Twelve trials did not adhere to the intention-to-treat principle.
These two factors lead to possible over-estimation of the treatment ef-
fect. There is also a possibility of publication bias. Among the limited
number of studies, different combinations of intervention approaches
and outcome measurements were adopted. It is therefore hard to
draw an accurate conclusion based on our meta-analysis. Finally, as
few trials involved post-intervention follow up or the follow-up period
was short, it is difficult to make conclusion on the sustainability of the
interventions.
To date, a variety of interventions are available but there is a lack of
commonly accepted framework on internalized stigma reduction. The
framework proposed by Hayward and Bright (1997) adopted CBT as
the main intervention with the role of psychosocial factors and self-
management skills also taken into consideration.
There are some implications for further research. Our meta-analysis
shows that psychoeducation is common among the studies and seems
to have significant effect. However, among these three studies, only
two (Roe et al., 2014; Russinova et al., 2014) mentioned clearly the con-
tent of psychoeducation, which included understanding of stigma and
strategies to cope with stigma. Given that psychoeducation was not
the sole intervention in these studies and the follow-up was short, it is
difficult to make a solid recommendation as to the optimal duration of
psychoeducation program. It is however suggested that future studies
should focus on studying the effectiveness of psychoeducation as the
only intervention with longer follow up period to evaluate the sustain-
ability of its outcome. As the content of psychoeducation varies, it is also
necessary to study the specific content that actually contributes to its ef-
fectiveness. Our systematic review found that there were only limited
studies meeting our inclusion criteria. Most studies had numerous lim-
itations in measurement tools and methodological qualities. These
added to our difficulties in making conclusion based on our results. Fol-
lowing this, we suggest the use of common outcome measures among
studies such as ISMI, and improved methodological qualities of future
studies to facilitate similar reviews in the future.
Two evolving innovative interventions, Coming Out Proud and
Photovoice, deserve more attention. The results from trials of COP
were inconclusive. Yet the study on Photovoice had a significant effect
in reduction of internalized stigma and was shown sustainable in a pe-
riod of three months. As new and promising interventions, more re-
search on their effectiveness is needed. As both of these two
innovative interventions involve peer-led activities, the therapeutic
values of this type of activities should be more carefully discerned.
Other than research, this study has some implications for clinical
practice. First, the serious consequences of leaving internalized stigma
untreated should be brought to the awareness of clinicians. Second, in-
ternalized stigma reduction programs have to be delivered by health
and rehabilitation professionals. Relevant training courses should be
made available to those professionals who show interest in putting
these programs into their clinical and rehabilitation practice. Third, if re-
courses are limited, practitioners should be recommended to select
those programs with a psychoeducation element because it is found to
be the most evidence-based in reducing internalized stigma based on
existing evidence. Finally, clinicians should be encouraged to explore
more innovative approaches to counteract internalized stigma because
existing interventions are limited and their evidence is not entirely
compelling.
5. Conclusion
Most programs we reviewed showed significant effects in reducing
internalized stigma. As an emerging area, studies were still limited.
The programs are implemented by professionals including clinicians,
psychiatric nurse, social worker, and non-professionals who are trained
according to the program manuals. Among different intervention ap-
proaches, psychoeducation seems to be more promising and two
novel techniques (i.e., coming out proud and photovoice) deserve
Table 1
Summary of the effects of self-stigma reduction program.
Author,
year
Study Participants’
diagnosis
Intervention Outcomes Result
Design; Country Inclusion Sample Size
(Pre−/Post)
Approach (frequency) Control Duration of
follow-up
Outcome measures
Corrigan et al.,
2015
RCT; U.S. People with serious
mental illness who
have life disabilities
EG: 107/51
CG: 98/75
Coming Out Proud (COP) (2 h
session, 3 sessions, over 3 days
or 1 day long group)
Waitlist 1 month 1. (Short form) SSMIS 2. Stigma
Stress Scale 3. CES-D
Effect in subscales under SSMIS:
1. Group × time: no significant effect in change in ‘harm”
comparing COP group to control group. Significant reduction in
“harm” from pre to posttest (p b 0.01) and pre to follow-up for
COP group. (p b 0.05)
2. Group × time: significant interaction for change in applying
stereotypes to self across groups (F(2,98) = 3.74, p b .05).
Significant reduction in applying stereotypes to self from pre to
posttest (p b 0.05) and at one month follow-up (p b 0.05)
3. Group × time: significant interaction for agreeing on stereotype
4. Group × time: no significant change in “awareness”
Stigma Stress Scale
1. Group × time: significant in stigma-related harm
2. Group × time: no significant for resources to cope with stigma
Depression
1. Group × gender × time: significant difference in CES-D
Costain et al.,
2014
Uncontrolled
clinical trial; Canada
Schizophrenia
Schizoaffective disorder
25/22
(21 for FU)
Genetic counseling
(1 session with mean 46.4 [SD
11.1] minutes)
NA 7 weeks 1. ISMI 1. Decrease in self-stigma (p = 0.0111)
2. Reduction in self-blame (p = 0.0401)
3. Improvement in understanding of the empiric recurrence risk
(p = 0.0090)
4. Reduction in associated concern (p = 0.0020)
5. Significant increase in subjective (p = 0.0007) and objective
(p = 0.0103) knowledge
Çuhadar and
Çam, 2014
RCT; Turkey Bipolar disorder 63/47 Psychoeducation program
(90 min session; once a week;
7 sessions)
No
treatment
NA 1. ISMI (Turkish Version)
2. BDFQ
1. Significant decrease in total ISMI score (p b 0.001) and sub-scale
1–4
(p b 0.001)
2. Significant gain in domestic relationship subscales (p = 0.03)
Fung et al.,
2011
RCT; Hong Kong Schizophrenia CG: 32/30
EG: 34/34
Self-stigma reduction program
(twice a week, 6 weeks; new
life: once 1 week, 12 weeks)
Conventional
therapy
3 months 1.CSSMIS
2.CAQ-SPMI
3.PTCS
4.BPRS
5.GAF
6.SUMD
7.CGSS
1. Group × time: significant improvement in the self-esteem
decrement subscale of CSSMIS *others subscale: stereotype
agreement and self-concurrence show trend of improvement
2. Group × time: significant changes in stages of change
continuous score of CAQ-SPMI and the participation subscale of
PTCS
3. Group × time: no significant changes found in SUMD and CGSS
Lucksted
et al., 2011
[26]
Pilot study; U.S. Self reported:
Schizophrenia
Schizoaffective disorder
Major mood disorder
EG: 50/34 Ending self-stigma (ESS) with
lectures and group
discussions.
(90 min session; once a week,
9 weeks)
NA NA 1. ISMI
2. MHRM
3. MSPSS
4. Empowerment Scale
(10 out of 28 items)
1. Group × time: significance in self stigma reduction.(p b 0.01) 2.
Group × time: recovery orientation (MHRM) and social support
(MSPSS) significantly increase
3. Group × time: mean empowerment increased but
insignificantly
(p = 0.0556 N 0.05) 4. Group × time: all ISMI subscale scores
increased significantly except stigma resistance
Lysaker et al.,
2012
Quasi-experimental;
U.S.
Schizophrenia
Schizoaffective disorder
EG: 70/70 Vocational rehabilitation:
part-time paid work
placement (non-specified
frequency)
NA 5 months 1. ISMI (5th scale excluded)
2. MSEI
3. PANSS
1. Group × time: modest but significant decrease in internalized
stigma
(p = 0.007) 2. Group effect: significant for both emotional
discomfort and self-esteem 3. Time effect: significant for
emotional discomfort and self-esteem 4. Group × time: significant
for self-esteem
5. No group or time effect for positive symptoms
McCay et al.,
2007
RCT; Canada Schizophrenia
Schizophreniform
disorder
Schizoaffective disorder
EG: 67/47 Psychotherapy group
(1.5 h session; once a week;12
weeks)
TAU NA 1.MES
2.TSCS
3.RSES
4.SES
5.LPSQ
6.QLS
7.MHS
8.PANSS
9.GAF
1. Group × time: no significance improvement in self concept, self
esteem, self efficacy, and stigma
2. Group × time: significant changes on measure of engulfment,
hope and quality of life measures compared with control
5
0
H
.W
.H
.Tsan
g
et
al./
Sch
izoph
ren
ia
R
esearch
17
3
(2016
)
45–
53
Russinova
et al., 2014
RCT; US Schizophrenia
spectrum disorder
Bipolar disorder
Depressive disorder
Other diagnosis
EG: 40/38
CG: 42/37
Antistigma Photovoice (90
min per session, 10 weeks)
TAU 3 months 1. ISMI
2. ACSS
3. PGRS
4. Empowerment scale
5. CES-D
6. GPSES
1. Significant decrease in overall ISMI score (p = 0.03) in
experimental group
2. Significant more likely to report using proactive strategies:
challenging (p = 0.04), educating others (p = 0.01) to cope with
perceived stigma.
3. Significantly greater increases in community activism and
autonomy (p = 0.02) on empowerment scale
4. Significant increases in overall level of PRGS (p = 0.04)
Roe et al.,
2014
Controlled clinical
trials; Israel
SMI ≥ 50% EG: 222/119 NECT
(1-h session; once a week; 20
sessions)
TAU NA 1. ISMI
2. MANSA
3. ADHS
4. RSE
1. A significant group × time interaction (F(4114) = 2.68,
p b 0.04) and with higher improvement in self-stigma in the
treatment group.
2. A significant group × time interaction (p = 0.043) and with
higher improvement in quality of life in the treatment
group.
3. No significant group × time effect
4. A significant group × time interaction (p = 0.003) and with
higher improvement in self-esteem in the treatment group.
Rüsch et al.,
2014
Pilot RCT;
Switzerland
Self reported:
Schizophrenia
spectrum disorder
Bipolar disorder
Depressive disorder
CG: 50/47
EG:50/40
Coming Out Proud (COP): with
peer-led group discussion.
(2 h session, once a week, 3
sessions)
TAU 3 weeks 1. ISMI (5th scale excluded)
2. Empowerment scale
3. Link’s 5-item secrecy scale
4. COMIS (7 items)
5. CAS (8 item)
6. Stress appraisal score
7. CES-D (15-item German version)
1. Group × time: no significant effect detected on self-stigma as
primary outcome.
2. Group × time: no significant effect of treatment on
empowerment or on disclosure-related self-efficacy.
3. Group × time: significant pre-post effect in some secondary
outcome including decreasing stigma stress and secrecy as
compared to the control group, with a medium-effect size for
stigma stress reduction
4. The effect is only partly sustainable. Disclosure-related
distress decreased significantly in the COP group from baseline
to post-intervention and follow up. Perceived benefits of being out
increased at post-intervention and remained stable for follow up.
Sibitz et al.,
2013
Controlled clinical
trial; Austria
Schizophrenia,
Schizoaffective
disorder, other F2
disorders
EG: 46/40
CG: 51/40
Recovery oriented day clinic
treatment
(8 am-3:30 pm, Mon–Friday, 2
months)
TAU NA 1. ISMI
2. WHOQOL-BREF
1. Significant group effect on ISMI subscale, discrimination
experience and social withdrawal, which internalized stigma
within control group at both baseline and follow-up were
higher.
2. Significant group, time, and group by time effect in QoL of
experimental group and improved in greater extent over time.
3. Significant time, group by time interaction in which significant
reduction in psychopathology over time in both groups.
Starin et al.,
2013
Uncontrolled
clinical trial;
Netherlands
Schizophrenia
Schizoaffective disorder
Schizophreniform
disorder
Psychotic disorder NOS
CBT-n
(45-min session, once a week,
20 sessions)
NA NA 1. PANSS
2. A combined score of BDI-4, 10, 12,
13, 15, 19, 20, 21; BHS-2, 6 inverted,
17; DAS-DPA-6, 14, 15; ISMI
1. Time effect: the improvement on the primary outcome measure
was highly significant and clinically relevant.
Uchino et al.,
2012
Controlled clinical
trial; Japan
Schizophrenia
Schizoaffective disorder
EG: 29/29
CG: 27/27
Weekly psychoeducation
program
(Once a week, 6 weeks)
TAU NA 1. SDS-J
2. KIDI
3. DAI-10
4. BPIS
5. GAF
1. Group effect: significant change between 2 groups for the SDS-J,
KIDI and BPIS.
2. Correlation: significant correlation found between SDS-J and
KIDI, BPIS and DAI-10.
3. No significant difference for DAI-10, GAF, age, and duration of
treatment.
Yanos et al.,
2012a,
2012b
RCT; U.S. Schizoaffective
Schizophrenia
Bipolar I
Bipolar II
Major depression
EG: 21/15
CG: 18/NR
NECT
(1-h session, once a week, 20
sessions)
TAU 3 months 1. ISMI
2. BHS
3. RSES
4. CSC
5. QLS
6. PANSS
7. SUMD
1. No significant differences between the NECT and TAU groups.
CG: Control Group, EG: Experimental Group, NA: Non Applicable; ACSS: Approaches to Coping with Stigma Scale, ADHS: Adult Dispositional Hope Scale, BDI: Beck Depression Inventory, BHS: The Beck Hopelessness Scale, BPIS: Birchwood’s Psychosis
Insight Scale, BPRS: The Brief Psychiatric Rating Scale, CAS: Cognitive Appraisal of Stigma, CAQ-SPMI: The Change Assessment Questionnaire for People with Severe and Persistent Mental Illness, CBT-n: Cognitive Behavioral Therapy for negative
symptoms, CES-D: Center for Epidemiological Studies Depression scale, CGSS: The Chinese General Self-efficacy Scale, COMIS: Coming Out with Mental Illness Scale, CSC: The Coping with Symptoms Checklist, CSSMIS: The Chinese Self-stigma of
Mental Illness Scale, DAI-10: Drug Attitude Inventory, DAS-DPA: Dysfunctional Attitude Scale — Defeatist Performance Attitude, GAF: Global Assessment of Functioning Scale, GPSES: Generalized Perceived Self-Efficacy Scale, ISMI: Internalized Stigma
of Mental Illness Scale, KIDI: Knowledge of Illness and Drugs Inventory, LPSQ: Link Perceived Stigma Questionnaire, MANSA: The Manchester Short Assessment of Quality of Life, MES: Modified Engulfment Scale, MHRM: The Mental Health Recovery
Measure, MSEI: Multidimensional Self-Esteem Inventory, MSPSS: Multidimensional Scale of Perceived Social Support, NECT: Narrative Enhancement/Cognitive Therapy, PANSS: The Positive and Negative Syndrome Scale, PGRS: Personal Growth and
Recovery Scale, PTCS: The Psychosocial Treatment Compliance Scale, QLS: The Quality of Life Scale, RSES: Rosenberg Self-Esteem Scale, SDS-J: Japanese version of Social Distance Scale, SES: Self-Efficacy Scale, SUMD: The Scale for Assessing Unaware-
ness of Mental Disorder, TAU: Treatment as usual, TSCS: Tennessee Self-Concept Scale. 5
1
H
.W
.H
.Tsan
g
et
al./
Sch
izoph
ren
ia
R
esearch
17
3
(2016
)
45–
53
Ta
b
le
.2
Q
u
al
it
y
A
ss
es
sm
en
t
of
th
e
in
cl
u
d
ed
st
u
d
ie
s.
St
u
d
ie
s
R
an
d
o
m
iz
at
i
o
n
em
p
lo
ye
d
R
an
d
o
m
iz
at
io
n
m
et
h
o
d
s
In
cl
u
si
o
n
an
d
ex
cl
u
si
o
n
cr
it
er
ia
P
at
ie
n
t
B
li
n
d
ed
O
u
tc
o
m
e
as
se
ss
o
rs
b
li
n
d
ed
W
it
h
d
ra
w
al
an
d
d
ro
p
o
u
ts
Sa
m
p
le
si
ze
es
ti
m
at
io
n
In
te
n
ti
o
n
-t
o
-t
re
at
an
al
ys
is
In
te
rv
en
ti
o
n
d
es
cr
ib
ed
Q
u
al
ifi
ca
ti
o
n
o
f
th
e
tr
ai
n
er
C
o
rr
ig
an
et
al
.,
2
0
1
5
Y
N
Y (i
n
cl
u
si
o
n
o
n
ly
)
N
N
Y
Y
N
Y
N
C
o
st
ai
n
et
al
.,
2
0
1
4
N
A
N
A
Y
N
N
Y
N
N
Y
N
Ç
u
h
ad
ar
an
d
Ç
am
,2
0
1
4
Y
R
an
d
o
m
d
ig
it
n
u
m
b
er
ta
b
le
Y
N
N
Y
Y
N
Y
Y
Fu
n
g
et
al
.,
2
0
1
1
Y
G
en
er
at
io
n
o
f
ra
n
d
o
m
n
u
m
b
er
b
y
SP
SS
Y
N
Y
Y
N
N
Y
N
Lu
ck
st
ed
et
al
.,
2
0
1
1
N
A
N
A
Y (I
n
cl
u
si
o
n
o
n
ly
)
N
N
Y
N
N
Y
N
A
Ly
sa
ke
r
et
al
.,
2
0
1
2
N
A
N
A
Y
N
N
Y
N
N
Y
N
A
M
cC
ay
et
al
.,
2
0
0
7
Y
B
y
ra
n
d
o
m
ly
p
u
ll
in
g
gr
o
u
p
as
si
gn
m
en
t
(g
ro
u
p
vs
co
n
tr
o
lf
ro
m
an
en
ve
lo
p
e)
Y
N
N
Y
N
N
N
Y
R
u
ss
in
o
va
et
al
.,
2
0
1
4
Y
C
o
m
p
u
te
r-
ge
n
er
at
ed
p
ro
gr
am
st
ra
ti
fi
ed
o
n
ge
n
d
er
&
ra
ci
al
-e
th
n
ic
m
in
o
ri
ty
st
at
u
s
Y
N
N
Y
N
Y
Y
N
R
o
e
et
al
.,
2
0
1
4
N
N
N
N
N
Y
N
N
Y
Y
R
ü
sc
h
et
al
.,
2
0
1
4
Y
R
an
d
o
m
n
u
m
b
er
ta
b
le
,a
n
d
en
ve
lo
p
es
w
it
h
n
u
m
b
er
s
w
er
e
d
ra
w
n
b
y
re
se
ar
ch
as
si
st
an
ts
w
h
o
h
ad
n
o
kn
o
w
le
d
ge
w
h
ic
h
n
u
m
b
er
w
as
in
ea
ch
en
ve
lo
p
e
Y
N
N
Y
Y
N
Y
N
Si
b
it
z
et
al
.,
2
0
1
3
N
N
A
Y
N
U
n
b
li
n
d
ed
Y
N
N
Y
Y
St
ar
in
g
et
al
.,
2
0
1
3
N
A
N
A
Y
U
n
b
li
n
d
ed
N
Y
N
Y
Y
Y
U
ch
in
o
et
al
.,
2
0
1
2
N
o
N
A
Y
N
N
N
N
N
Y
N
Y
an
o
s
et
al
.,
2
0
1
2
a,
2
0
1
2
b
Y
Se
p
ar
at
ed
ra
n
d
o
m
iz
at
io
n
fo
r
tw
o
se
tt
in
gs
b
as
ed
o
n
co
m
p
u
te
ri
ze
d
n
u
m
b
er
ge
n
er
at
in
g
sy
st
em
.
Y
N
N
Y
N
N
Y
Y
N
:
n
ot
re
p
or
te
d
;
N
A
:
n
ot
ap
p
li
ca
b
le
;
Y
:
ye
s.
52 H.W.H. Tsang et al. / Schizophrenia Research 173 (2016) 45–53
more attention. In future, more innovative approaches to reducing in-
ternalized stigma should be developed and more RCTs on particular in-
tervention components using standard outcome measure should be
conducted so that meta-analysis could be conducted and effects of
the intervention could be compared. All of the above adds to
evidence-based practice in internalized stigma reduction.
Supplementary data to this article can be found online at http://dx.
doi.org/10.1016/j.schres.2016.02.013.
References
Cohen, J., 1988. Statistical Power Analysis for the Behavioral Sciences. second ed. Law-
rence Erlbaum Associates, Hillsdale, NJ.
Corrigan, P.W., 2014. The impact of mental illness stigma on seeking and participating in
mental health care. Psychol. Sci. Public Interest 15 (2), 37–70. http://dx.doi.org/10.
1177/1529100614531398.
Corrigan, P.W., et al., 2015. Diminishing the self-stigma of mental illness by coming out
proud. Psychiatry Res. 229 (1–2), 148–154. http://dx.doi.org/10.1016/j.psychres.
2015.07.053.
Costain, G., Esplen, M.J., Toner, B., Scherer, S.W., Meschino, W.S., Hodgkinson, K.A.,
Bassett, A.S., 2014. Evaluating Genetic Counseling for Individuals With Schizophre-
nia in the Molecular Age. Schizophr. Bull. 40 (1), 78–87. http://dx.doi.org/10.1093/
schbul/sbs138.
Çuhadar, D., Çam, M.O., 2014. Effectiveness of psychoeducation in reducing internalized
stigmatization in patients with bipolar disorder. Arch. Psychiatr. Nurs. 28 (1), 62–66.
http://dx.doi.org/10.1016/j.apnu.20 13.10.008.
Fung, K.M., Tsang, H.W., Cheung, W.M., 2011. Randomized controlled trial of the self-
stigma reduction program among individuals with schizophrenia. Psychiatry Res.
189 (2), 208–214. http://dx.doi.org/10.1016/j.psychres.2011.02.013.
Fung, K.M., Tsang, H.W., Corrigan, P.W., 2008. Self stigma of people with schizophrenia as
predictor of their adherence to psychosocial treatment. Psychiatr. Rehabil. J. 32 (2),
95–104.
Gerlinger, G., et al., 2013. Personal stigma in schizophrenia spectrum disorder: a system-
atic review of prevalence rates, correlates, impact and interventions. World Psychi-
atry 12 (2), 155–164. http://dx.doi.org/10.1002/wps.20040.
Hayward, P., Bright, J., 1997. Stigma and mental illness: a review and critique. J. Ment.
Health 6 (4), 345–354.
Higgins, J.P.T., Green, S., 2011. Cochrane handbook for systematic reviews of interven-
tions version 5.1.0. The Cochrane Collaboration (Retrieved from www.cochrane-
handbook.org).
Higgins, J.P.T., Thompson, S.G., Deeks, J.J., Altman, D.G., 2003. Measuring inconsistency in
meta-analyses. BMJ 327 (7414), 557–560. http://dx.doi.org/10.1136/bmj.327.7414.
557.
Lacro, J.P., Dunn, L.B., Dolder, C.R., Leckband, S.G., Jeste, D.V., 2002. Prevalence of and risk
factors for medication nonadherence in patients with schizophrenia: a comprehen-
sive review of recent literature. J. Clin. Psychiatry 63 (10), 892–909.
Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Loannidis, J.P.A., … Moher,
D., 2009. The PRISMA statement for reporting systematic reviews and meta-analyses
of studies that evaluate health care interventions: explanation and elaboration. PLoS
Med. 6 (7), 1–28. http://dx.doi.org/10.1016/j.jclinepi.2009.06.006.
Livingston, J., Boyd, J., 2010. Correlates and consequences of internalized stigma for peo-
ple living with mental illness: a systematic review and meta-analysis. Soc. Sci. Med.
71 (12), 2150–2161. http://dx.doi.org/10.1016/j.socscimed.2010.09.030.
Lucksted, A., Drapalski, A., Calmes, C., Forbes, C., DeForge, B., Boyd, J., 2011. Ending self-
stigma: pilot evaluation of a new intervention to reduce internalized stigma
among people with mental illnesses. Psychiatr. Rehabil. J. 35 (1), 51–54. http://dx.
doi.org/10.2975/35.1.2011.51.54.
Lysaker, P.H., Roe, D., Ringer, J., Gilmore, E.M., Yanos, P.T., 2012. Change in self-stigma
among persons with schizophrenia enrolled in rehabilitation: associations with
self-esteem and positive and emotional discomfort symptoms. Psychol. Serv. 9 (3),
240–247. http://dx.doi.org/10.1037/a0027740.
McCay, E., Beanlands, H., Zipursky, R., Roy, P., Leszcz, M., Landeen, J., … Chan, E., 2007. A
randomised controlled trial of a group intervention to reduce engulfment and self-
stigmatisation in first episode schizophrenia. Adv. Ment. Health 6 (3), 212–220.
http://dx.doi.org/10.5172/jamh.6.3.212.
Mittal, D., Sullivan, G., Chekuri, L., Allee, El, Corrigan, P.W., 2012. Empirical studies of self-
stigma reducation strategies: a critical review of the literature. Psychiatr. Serv. 63
(10), 974–981.
Mosanya, T.J., Adelufosi, A.O., Adebowale, O.T., Oqunwale, A., Adebayo, O.K., 2014. Self-
stigma, quality of life and schizophrenia: An outpatient clinic survey in Nigeria. Int.
J. Soc. Psychiatry 60 (4), 377–386. http://dx.doi.org/10.1177/0020764013491738.
Roe, D., Hasson-Ohayon, I., Mashiach-Eizenberg, M., Derhy, O., Lysaker, P.H., Yanos, P.T.,
2014. Narrative enhancement and cognitive therapy (NECT) effectiveness: a quasi-
experimental study. J. Clin. Psychol. 70 (4), 303–312. http://dx.doi.org/10.1002/
jclp.22050.
Rüsch, N., Abbruzzese, E., Hagedorn, E., Hartenhauer, D., Kaufmann, I., Curschellas, J., …
Corrigan, P.W., 2014. Efficacy of coming out proud to reduce stigma’s impact
among people with mental illness: pilot randomised controlled trial. Br.
J. Psychiatry 204, 391–397. http://dx.doi.org/10.1192/bjp.bp.113.135772.
Rüsch, N., Corrigan, P.W., Wassel, A., Michaels, P., Larson, J.E., Olschewski, M., Wilkniss, S.,
Batia, K., 2009. Self-stigma, group identification, perceived legitimacy of discrimina-
tion and mental health service use. Br. J. Psychiatry 195 (6), 551–552. http://dx.doi.
org/10.1192/bjp.bp.109.067157.
http://dx.doi.org/10.1016/j.schres.2016.02.013
http://dx.doi.org/10.1016/j.schres.2016.02.013
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0010
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0010
http://dx.doi.org/10.1177/1529100614531398
http://dx.doi.org/10.1177/1529100614531398
http://dx.doi.org/10.1016/j.psychres.2015.07.053
http://dx.doi.org/10.1016/j.psychres.2015.07.053
http://dx.doi.org/10.1093/schbul/sbs138
http://dx.doi.org/10.1093/schbul/sbs138
http://dx.doi.org/10.1016/j.apnu.20 13.10.008
http://dx.doi.org/10.1016/j.psychres.2011.02.013
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0045
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0045
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0045
http://dx.doi.org/10.1002/wps.20040
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0055
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0055
http://www.cochraneandbook.org
http://www.cochraneandbook.org
http://dx.doi.org/10.1136/bmj.327.7414.557
http://dx.doi.org/10.1136/bmj.327.7414.557
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0070
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0070
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0070
http://dx.doi.org/10.1016/j.jclinepi.2009.06.006
http://dx.doi.org/10.1016/j.socscimed.2010.09.030
http://dx.doi.org/10.2975/35.1.2011.51.54
http://dx.doi.org/10.1037/a0027740
http://dx.doi.org/10.5172/jamh.6.3.212
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0110
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0110
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0110
http://dx.doi.org/10.1177/0020764013491738
http://dx.doi.org/10.1002/jclp.22050
http://dx.doi.org/10.1002/jclp.22050
http://dx.doi.org/10.1192/bjp.bp.113.135772
http://dx.doi.org/10.1192/bjp.bp.109.067157
53H.W.H. Tsang et al. / Schizophrenia Research 173 (2016) 45–53
Russinova, Z., Rogers, E.S., Gagne, C., Bloch, P., Drake, K.M., Mueser, K.T., 2014. A random-
ized controlled trial of a peer-run antistigma photovoice intervention. Psychiatr. Serv.
65 (2), 242–246. http://dx.doi.org/10.1176/appi.ps.201200572.
Sibitz, I., Provaznikova, K., Lipp, M., Lakeman, R., Amering, M., 2013. The impact of
recovery-oriented day clinic treatment on internalized stigma: preliminary report.
Psychiatry Res. 209 (3), 326–332. http://dx.doi.org/10.1016/j.psychres.2013.02.001.
Staring, A.B.P., Ter Huurne, M.B., van, D.G., 2013. Cognitive behavioral therapy for negative
symptoms (CBT-n) in psychotic disorders: a pilot study. J. Behav. Ther. Exp. Psychia-
try 44 (3), 300–306. http://dx.doi.org/10.1016/j.jbtep.2013.01.004.
The Cochrane Collaboration, 2014. Review manager (RevMan) [computer program]. Ver-
sion 5.3 Retrieved from http://tech.cochrane.org/revman.
Tsang, H.W.H., Fung, K.M.T., Chung, R.C.K., 2010. Self-stigma and stages of change as pre-
dictors of treatment adherence of individuals with schizophrenia. Psychiatry Res. 180
(1), 10–15.
Uchino, T., Maeda, M., Uchimura, N., 2012. Psychoeducation may reduce self-stigma of
people with schizophrenia and schizoaffective disorder. Kurume Med. J. 59 (1–2),
25–31.
Yanos, P.T., Roe, D., Markus, K., Lysaker, P.H., 2008. Pathways between internalized stigma
and outcomes related to recovery in schizophrenia spectrum disorders. Psychiatr.
Serv. 59 (12), 1437–1442.
Yanos, P.T., Roe, D., West, M.L., Smith, S.M., Lysaker, P.H., 2012a. Group-based treatment
for internalized stigma among persons with severe mental illness: findings from a
randomized controlled trial. Psychol. Serv. 9 (3), 248–258. http://dx.doi.org/10.
1037/a0028048.
Yanos, P.T., West, M., Gonzales, L., Smith, S., Roe, D., Lysaker, P., 2012b. Change in internal-
ized stigma and social functioning among persons diagnosed with severe mental ill-
ness. Psychiatry Res. 200 (2–3), 1032–1034. http://dx.doi.org/10.1016/j.psychres.
2012.06.017.
http://dx.doi.org/10.1176/appi.ps.201200572
http://dx.doi.org/10.1016/j.psychres.2013.02.001
http://dx.doi.org/10.1016/j.jbtep.2013.01.004
http://tech.cochrane.org/revman
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0160
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0160
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0160
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0165
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0165
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf0165
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf5165
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf5165
http://refhub.elsevier.com/S0920-9964(16)30069-X/rf5165
http://dx.doi.org/10.1037/a0028048
http://dx.doi.org/10.1037/a0028048
http://dx.doi.org/10.1016/j.psychres.2012.06.017
http://dx.doi.org/10.1016/j.psychres.2012.06.017
- Therapeutic intervention for internalized stigma of severe mental illness: A systematic review and meta-�analysis
1. Introduction
2. Methods
2.1. Data sources
2.2. Study selection
2.3. Data extraction and analysis
2.4. Assessment of methodological quality
2.5. Data synthesis and analysis
3. Results
3.1. Results of literature search
3.2. Description of included studies
3.3. Results of individual studies
3.4. Effects of intervention approaches on internalized stigma
3.4.1. Comparison 1: psychoeducation element
3.4.2. Comparison 2: cognitive behavioral therapy element
3.4.3. Comparison 3: group discussion element
3.4.4. Comparison 4: social skills training element
3.4.5. Comparison 5: narrative enhancement and cognitive therapy (NECT) element
3.4.6. Other intervention approaches
3.5. Meta-analysis
4. Discussion
5. Conclusion
References