Home discussion4-2

discussion4-2

The push toward accountability seen throughout the P-12 educational settings is now evidenced within teacher preparation programs, ensuring that universities and colleges are adequately preparing future teachers for the task. Such pressures are even more rampant within the field of special education. Special education teachers are tasked with the responsibility of not only ensuring students, individual needs are being met with meaningful curriculum and instruction, but that they are making adequate progress toward their goals and objectives. Student learning rests at the core of accountability within educational systems (Fullan & Quinn, 2016). While internal accountability focuses on individuals’ and groups’ efforts at continuous improvement and success for all students, external accountability focuses on leaders’ responsibility to be transparent with the public about the system’s ability to meet expectations and requirements. Both internal and external accountability are integral to the Coherence Framework, strengthening the vision and collaboration of the organization, thus establishing deepening of learning goals. For this Discussion, you will analyze both internal and external accountability and their role in whole systems change. As the special education leader, how will you secure accountability?

To Prepare:

Review Chapter 5, “Securing Accountability,” in the Fullan and Quinn text, focusing on the aspects of internal and external accountability. How would you distinguish between internal and external accountability?
Reflect on the Norman and Sherwood (2015) article, considering how internal and external accountability measures have shaped teacher preparation programs.
Reflect on the plans you have created for the case scenario viewed in Modules 2 and 3 so far and reflect on how you would secure accountability if you were a leader of the staff in this scenario.

Post an explanation of how you would build internal accountability based on the case scenario. What steps would you recommend to ensure effective implementation of external accountability? Include an analysis of each concept and evidence from the case scenario, as well as other learning resources to support your rationale.

Please use the file enclosed plus step by step explanations.

ALSO use the SCENARIO s tracking data and mandate meeting

22 EDUCATION CANADA • December 2015 | Canadian Education Association www.cea-ace.ca/educationcanada

MY COLLEAGUES AND I have been working on “whole
system change” (how all schools in a province/state/
country can improve) since we carried out the evalua-
tion of England’s literacy and numeracy strategy from
1998-2002.1 We then applied the lessons from England to
Ontario’s reform strategy that began in 2003. In an earlier
article for CEA2 I identified the “big ideas” as:

1. All children can learn
2. A small number of key priorities
3. Resolute leadership
4. Collective capacity
5. Strategies with precision
6. Intelligent accountability
7. All means all.

We have learned a great deal about whole system
change, which we have captured in a complete case study

of Ontario, 2003-2015.3 As we examined and worked with
systems around the world – some that were relatively cen-
tralized and some relatively decentralized – we began to
search for a more powerful way to seek whole system
success regardless of the starting point.

The answer, and the focus of this article, is “Leadership
from the Middle” (LftM), first identified by Hargreaves
and Braun4 in their evaluation of the implementation of a
special education initiative in Ontario. For this initiative,
the government allocated $25 million to the Council of
Ontario Directors of Education to lead implementation
across all 72 districts. The government, if you like, asked
“the middle” – the districts – to lead system change.

LftM and its rationale
In education system terms, the top is the state, the mid-
dle is districts or regions, and the bottom is schools and

Michael Fullan, Order
of Canada, is Professor
Emeritus at OISE/
University of Toronto
and has served as
special adviser to two
Premiers and Ministers
of Education in Ontario
since 2003. He advises
governments around
the world on whole
system change (for his
latest books see www.
michaelfullan.ca).

Leadership
from the Middle

A system strategy

By Michael Fullan

THE CHALLENGE TO CHANGE
ILLU

STR
ATIO

N
: ISTO

C
K

www.cea-ace.ca/educationcanada Canadian Education Association | December 2015 • EDUCATION CANADA 23

24 EDUCATION CANADA • December 2015 | Canadian Education Association www.cea-ace.ca/educationcanada

communities. Top-down leadership doesn’t last even if you get a lot
of the pieces right, because it is too difficult to get, and especially
to sustain, widespread buy-in from the bottom. In many ways the
Ontario strategy was led from the top (the government), and although
it did contain many strong partnership ideas, it ultimately will not be
embedded enough to establish sustainable system change (see the
discussion of New Pedagogies for Deep Learning (NPDL) and Ontario
below). Similarly, bottom-up change (e.g. school autonomy) does not
result in overall system improvement; some schools improve, others
don’t and the gap between high and low performers grows wider.

The key question, then, is how can we achieve the strongest
system coherence, capacity and commitment resulting in sustained
improvement?

Leadership from the Middle can be briefly defined as: a deliberate
strategy that increases the capacity and internal coherence of the middle
as it becomes a more effective partner upward to the state and downward
to its schools and communities, in pursuit of greater system perform-
ance. The goal of LftM is to develop greater overall system coherence
by strengthening the focus of the middle in relation to system goals
and local needs. Thus, it is not a standalone, but rather a connected
strategy. This approach is powerful because it mobilizes the middle
(districts and/or networks of schools), thus developing widespread
capacity, while at the same time the middle works with its schools
more effectively and becomes a better and more influential partner
upward to the center.

The LftM strategy is being used in several systems around the
world, and my colleagues and I are currently involved in initiatives
in California (districts working with each other on system goals), Con-
necticut (districts working in cohorts), and Quebec (again districts
working together on local and province-wide priorities). For this
article I will draw on two examples: one from the relatively decentral-
ized system of New Zealand; the other from the relatively centralized
province of Ontario.

System change in New Zealand
In 1989, New Zealand passed a radical (at the time) piece of legisla-
tion entitled Tomorrow’s Schools that abolished regional authority
and created individual school autonomy, with each school having
its own school council. Assessing its impact is beyond the scope of

this article, but we can say that by and large, improved performance
of the overall system did not ensue (for example, the gap between
high- and low-performance schools increased). In 2014, the current
government passed another initiative, called Investing in Education
Success, that provided a substantial new budget of 369 million NZ dol-
lars in order to set up networks of schools that would work together to
leverage improvement. There are some 2,500 schools in New Zealand;
it was expected that all schools would participate in networks of 5-20
schools. Initially the proposal was imposed on the system and was
greeted with widespread opposition.

Over the past year and a half, the system has worked on a resolu-
tion that I would essentially call an LftM solution. For example, the
government and the primary school teachers’/principals’ federations
worked out guidelines in something called the “Joint Initiative.” Here
are its five fundamental principles:5

1. Children are at the centre of a smooth and seamless whole of
educational pathway, from earliest learning to tertiary options.

2. Parents who are informed and engaged are involved in their
children’s education and part of a community with high expect-
ations for and of those children.

3. Teachers and education leaders, supported by their own profes-
sional learning and growth, and those of their colleagues, will
systematically collaborate to improve educational achievement
outcomes for their students.

4. Teachers and education leaders will be able to report measur-
able gain in the specific learning and achievement challenges of
their students.

5. Teachers and leaders will grow the capability and status of the
profession within clearly defined career pathways for develop-
ment and advancement.

Within these overarching principles, New Zealand is working out
additional requirements to guide the work of emerging networks.
These guidelines are consistent with eight criteria that Santiago
Rincon-Gallardo and I formulated in relation to LftM networks of
schools or districts. We have identified eight essential ingredients of
effective networks:6

1. Developing high-trust relationships
2. Focusing on ambitious student learning goals linked to measur-

able outcomes
3. Continuously improving instructional practice
4. Using deliberate leadership and skilled facilitation
5. Frequently interacting and learning inwards
6. Connecting outward to learn from others
7. Forming new partnerships among students, teachers and

families
8. Securing adequate resources to sustain the work.

EN BREF
Le changement issu d’une approche descendante « top-down »
des gouvernements n’engendre pas un changement systémique
(amélioration de toutes les écoles), pas plus que le changement
issu d’une approche ascendante « bottom-up », venant de la base
où chaque école jouit d’autonomie, n’amènent de grands progrès.
Pour rehausser la cohérence et l’impact à l’échelle du système, une
nouvelle stratégie beaucoup plus prometteuse voit le jour : le lea-
dership provenant du milieu (« LpM »). Le LpM renforce les conseils
scolaires et les réseaux d’écoles, qui collaborent pour régler des
problèmes précis en vue d’accroître des capacités pédagogiques
et des compétences collectives ayant des effets mesurables sur
l’engagement des élèves. Dans cet article, l’auteur cite la Nouvelle-
Zélande et l’Ontario comme exemples de la mise en pratique de
cette stratégie mobilisatrice. Le LpM est mieux adapté à l’innovation,
à la diffusion, à l’engagement des élèves et des enseignants, ainsi
qu’à des apprentissages en profondeur tels le développement de
la personnalité, l’éducation à la citoyenneté, la collaboration, la
créativité et la pensée critique. Bref, le LpM consiste à mobiliser
tout le système en vue des nouveaux modes d’apprentissage plus
profonds requis pour le 21e siècle.

www.cea-ace.ca/educationcanada Canadian Education Association | December 2015 • EDUCATION CANADA 25

It is too early to assess the impact of New Zealand’s LftM strategy,
but it does provide a clear example of deliberately trying to mobilize
the middle for system success.

New Pedagogies for Deep Learning (NPDL) in Ontario
Ontario makes for a particularly interesting case because it has had
strong success using an assertive strategy from the government
combined with partnerships with its districts. This has served the
province well on basic measures of literacy, numeracy (though less
impressively), and high school graduation. What becomes evident is
that such a model may not be suitable for innovation and its related
21st century skills.

My colleagues and I are pursuing, in seven countries including
Canada, a strategy that we call New Pedagogies for Deep Learning
(www.npdl.global). New pedagogies refer to developing learning part-
nerships between and among students, teachers, and families. We are
currently defining and developing the details of these partnerships,
which essentially are based on proactive learning roles for students
and for teachers using the latest pedagogical practice.

Deep learning is the 6Cs: Character education, citizenship, col-
laboration, communication, creativity, and critical thinking. Again
we are defining and developing instruments to assess and support
these outcomes.

There are currently some 15 districts in Ontario and Manitoba
working to implement and disseminate these ideas in practice. Let
me be explicit how this represents Learning from the Middle:
• The center of gravity of NPDL is districts and schools, not the

province. The ideas are compatible with government policy but
not led by the province. This is especially interesting in Ontario
because after some eight years of central leadership (2003-2011),
districts are having more opportunity to lead change. I cannot
say that this is a result of deliberate policy but can observe that
many districts are showing new initiative in their own right and
in teaming up with other districts, with NPDL being a prime
example.

• The essence of this LftM work consists of innovation and dis-
semination. Having established a basic instructional capacity in
implementing literacy, many schools and districts are now going
deeper into new pedagogies that engage students and teachers
in real-life problems.

• This model of change is one that Maria Langworthy and I formu-
lated in A Rich Seam. We described the process as: “Directional
vision, letting go, and reining in.”7 Clearly this represents a
dynamic model, but it also requires a degree of “discipline.”
First, directional vision shapes the direction (our NPDL defin-
ition for example); second, letting go encourages people to do
new things within the broad new direction. We are currently

Top-down leadership doesn’t
last even if you get a lot of the
pieces right, because it is too
difficult to get, and especially
to sustain, widespread buy-in
from the bottom.

26 EDUCATION CANADA • December 2015 | Canadian Education Association www.cea-ace.ca/educationcanada

documenting examples of this work in action as students and
teachers, for example, are tackling local problems and working
together to come up with innovative solutions; third, reining
in is built into the process of co-learning. Again we are docu-
menting what this looks like but it consists of the use of targeted
questions and protocols to arrive at new meaning and new
assessment.8 Because the model is laden with transparency,
precision of action, assessment of what is being learned includ-
ing outcomes, and continuous exchanges and deliberation, good
ideas get sorted out and retained. This is not a linear model but
more or less simultaneous. It generates and assesses a great deal
of innovation. These ideas can be further sorted in relation to
provincial frameworks and assessments.

In NPDL we are assessing, capturing and spreading what is being
learned. It sounds messy and to a certain extent it is, but it promises
to produce better ideas, more quickly, with greater local capacity
and ownership. At the same time, this is played out within an overall
mindset that we have called “systemness”: a commitment to con-
tributing to, and benefitting from, the larger system. LftM cultivates
activities and co-learning that constantly place people in the context
of interaction at their own level, and also beyond it as ideas are sorted
out in regional, provincial, and in the case of NPDL, international
exchanges.

In short, NPDL is a strong, specific example of LftM oriented to
innovation and the future of learning.

The promise of LftM
It is crucial to say that every time the middle gets together, it is not
automatically a good thing. We referred earlier to our eight criteria for
collaboration.9 Thus having a strong moral purpose focus, working
on deep new pedagogies and learning outcomes, affecting the whole
system, and so on, are all essential components.

LftM is a new concept, and has not been fully tested and assessed.
But there are at least three big reasons why it holds great promise:

1. It appeals instantly to a critical mass of people who want a role
and have hitherto not been able to see where they fit. When
people become aware of LftM ideas they quickly identify with
its potential because it is a strategy that finally gives people in
the middle a prominent role to play.

2. It can be used in a variety of ways and is especially suited to
breakthrough innovations that are so sorely lacking in public
school systems. Traditional school systems have become stodgy
and boring for students and for educators. LftM enables and
unleashes badly needed innovation on a large scale while at

the same time helping to assess and sort out what should be
retained and spread.

3. By definition, it implicates the whole system starting from the
middle out, up and down. In addition to our system-use of the
concept, LftM can and should be used at other levels. Schools,
for example are the middle if you use a within-district focus.
Teachers, students and families are the middle when you think
of intra-school and community work.

Conclusion
Governments have become less and less effective at leading system
change.

The old model – prioritize and implement – is no longer suitable. It
cannot generate innovation and learning fast enough for the demands
of the 21st century. For the latter you need continuous innovation
in real time generated and assessed through co-learning (laterally
within and across classrooms, schools and districts; and hierarchic-
ally school to district to province). For this kind of innovation, the
middle is essential.

Learning from the Middle represents a new and powerful way of
thinking that frees us from outdated and limited models that depend
on top-down versus bottom-up thinking. It liberates a greater mass of
people to become engaged in purposeful system change, and ultim-
ately to own the changes that they create together. EC

NOTES
1 L. Earl, M. Fullan, K. Leithwood, and N. Watson, Watching & Learning: OISE/UT

evaluation of the national literacy and numeracy strategies (London, England:
Department for Education and Skills, 2003).

2 M. Fullan, “Big Ideas Behind Whole System Reform,” Education Canada 50, No. 3
(2010): 24-27.

3 M. Fullan and S. Rincon-Gallardo, “Developing High Quality Education in Canada:
The case of Ontario,” in Adamson, Astrand, and Darling-Hammond, Eds., (London:
Routledge, in press).

4 A. Hargreaves and H. Braun, Leading for All (Ontario: Council of Ontario Directors
of Education, 2010).
www.ontariodirectors.ca/downloads/Essential_FullReport_Final

5 New Zealand Education Institute, Ministry of Education, Joint Initiative Agreement
(Wellington NZ: 2015).

6 S. Rincon-Gallardo and M. Fullan, “Essential Features of Effective Networks and
Professional Collaboration,” Journal of Professional Capital and Community (in
press).

7 M. Fullan and M. Langworthy, A Rich Seam: How new pedagogies find deep
learning (London: Pearson, 2014), 48.

8 M. Fullan, “Leadership in a Digital Age,” Presentation at the Deep Learning Hub
(October, 2015). www.npdl.global

9 See also our “coherence framework”: M. Fullan and J. Quinn, Coherence: Putting
the right drivers in action (Thousand Oakes, CA: Corwin Press, 2015).

Leading from the Middle
unleashes badly needed
innovation on a large scale
while at the same time helping
to assess and sort out what
should be retained and
spread.

Overview

Our team has been immersed in ‘whole system change’ for the past few years
in Ontario, Canada; California; Australia and New Zealand; and elsewhere. Our main
mode of learning is to go from practice to theory, and then back and forth to obtain
more specific insights about how to lead and participate in transformative change in
schools and school systems.

In this workshop we take the best of these insights from our most recent
publications: Stratosphere, The Professional Capital of Teachers, The Principal,
Freedom to Change, and Coherence and integrate the ideas into a single set of
learnings.

The specific objectives for participants are:

1. To learn to take initiative on what we call ‘Freedom to Change’.
2. To Understand and be able to use the ‘Coherence Framework’.
3. To analyze your current situation and to identify action strategies fro making

improvements.
4. Overall to gain insights into ‘leadership in a digital age’.

We have organized this session around six modules:

Module I Freedom From Change 1-

Module II Focusing Direction 5-

Module III Cultivating Collaborative Cultures 11-

Module IV Deepening Learning 15-2

Module V Securing Accountability 23-

Module VI Freedom To Change 31-

References 3

Please feel free to reproduce and use the
material in this booklet with your staff and others.

201

Freedom From Change

Shifting to
the Right Drivers

Right Wrong

§ Capacity building

§ Collaborative work

§ Pedagogy

§ Systemness

§

Accountability

§ Individual teacher and

leadership quality

§ Technology

§ Fragmented strategies

Freedom:

If you could make one

change in your school or

system what would it be?

What obstacles stand in

your way?

What would you change? What are the obstacles?

Trio Talk:

§ Meet up with two colleagues.

§ Share your choice and rationale.

§ What were the similarities and differences in the choices?

Module 1

The Concepts of Freedom § Freedom to is getting rid of the constraints.

§ Freedom from is figuring

out what to do when you

become more liberated.

Seeking Coherence § Within your table read the seven quotes from Coherence and circle
the one you like the best.

§ Go around the table and see who selected which quotes.

§ As a group discuss what ‘coherence’ means.

Coherence: The Right Drivers in Action for Schools, Districts, and Systems

Fullan, M., & Quinn, J. ( 2015). Corwin & Ontario Principals’ Council.

# Quote

1. There is only one way to achieve greater coherence, and that is through purposeful action and interaction,
working on capacity, clarity, precision of practice, transparency, monitoring of progress, and continuous
correction. All of this requires the right mixture of “pressure and support”: the press for progress within
supportive and focused cultures. p. 2

2. Coherence making in other words is a continuous process of making and remaking meaning in your own
mind and in your culture. Our framework shows you how to do this. p. 3

3. Effective change processes shape and reshape good ideas as they build capacity and ownership among
participants. There are two components: the quality of the idea and the quality of the process. p.14

4. … that these highly successful organizations learned from the success of others but never tried to imitate
what others did. Instead, they found their own pathway to success. They did many of the right things, and
they learned and adjusted as they proceeded. p.

5. Most people would rather be challenged by change and helped to progress than be mired in frustration.
Best of all, this work tackles “whole systems” and uses the group to change the group. People know they
are engaged in something beyond their narrow role. It is human nature to rise to a larger call if the
problems are serious enough and if there is a way forward where they can play a role with others.
Coherence making is the pathway that does this. p. ix

6. What we need is consistency of purpose, policy, and practice. Structure and strategy are not enough. The
solution requires the individual and collective ability to build shared meaning, capacity, and commitment
to

action.

When large numbers of people have a deeply understood sense of what needs to be done—
and see their part in achieving that purpose—coherence emerges and powerful things happen. p. 1

7. Coherence pertains to people individually and especially collectively. To cut to the chase, coherence
consists of the shared depth of understanding about the purpose and nature of the work. Coherence,
then, is what is in the minds and actions of people individually and especially collectively. p. 1-2

Freedom From Change

The Coherence Framework

Securing
Accountability

Focusing
Direction

Deepening
Learning

Cultivating
Collaborative

Cultures

Leadership

Module 1

Notes:

Focusing Direction

Purpose Driven:
Quick Write

Clarify your own moral purpose by reflecting and recording your

thoughts about these four questions using the quick write protocol:

§ What is your moral purpose?

§

What actions do I take to realize this moral purpose?

§

How do I help others clarify their moral purpose?

§ Am I making progress in realizing my moral purpose with

students?

Share your thoughts with other members of your team and discuss

themes that emerge.

Focusing Direction

§ Purpose Driven
§ Goals That Impact
§ Clarity of Strategy
§ Change Leadership

Deepening
Learning

Securing
Accountability

Cultivating
Collaborative
Cultures
Leadership

Module 2

What is my moral purpose?

What actions do I take to realize this moral purpose?

How do I help others clarify their moral purpose?

Am I making progress in realizing my moral purpose with students?

Focusing Direction

Clarity of Strategy § Successful change processes are a function of shaping and

reshaping good ideas as they build capacity and ownership.

§ Clarity about goals is not sufficient. Leaders must develop shared

understanding in people’s minds and collective action. Coherence

becomes a function of the interplay between the growing

explicitness of the strategy and the change culture. The two

variables of explicitness of strategy and quality of the change

culture interact creating four different results.

Change Quality Protocol

1. Superficiality

When the strategy is not very precise, actionable or clear (low explicitness) and people are comfortable

in the culture, we may see activity but at very superficial levels.

2. Inertia

This quadrant represents the history of the teaching profession—behind the classroom door, where

teachers left each other alone with a license to be creative or ineffective.

Innovative teachers receive little feedback on their ideas, nor do these ideas become available to others

and isolated, less than effective teachers get little help to improve.

3. Resistance

When innovations are highly prescribed (often detailed programs bought off the shelf) but culture is

weak and teachers have not been involved sufficiently in developing ownership and new capacities, the

result is pushback and resistance. If the programs are sound, they can result in short term gains

(tightening an otherwise loose system), but because teachers have not been engaged in shaping the

ideas or the strategy there is little willingness to take risks.

4. Depth

A strong climate for change with an explicitness of strategy is optimal. People operating in conditions of

high trust, collaboration, and effective leadership, are more willing to innovate and take risks. If we

balance that with a strategy that has precision, clarity, and measures of success, changes implemented

will be deep and have impact.

Module 2

Change Quality Quadrant

Change Climate (vertical axis):

§ Describes the degree to which a culture supports change by

fostering trust, nonjudgmentalism, leadership, innovation, and

collaboration.

Explicitness (horizontal axis):

§ Describes the degree of explicitness of the strategy, including

precision of the goals, clarity of the strategy, use of data, and

supports.

Change Quality Protocol

1. Brainstorm individually all the changes you are implementing in

your school or district and place each idea on a post-it along with

your initial.

2. Consider evidence of explicitness of the strategy and the strength

of the culture for each initiative. Mark the post-it as belonging to

quadrant 1, 2, 3 or 4.

3. When the first two steps are completed, all peers should place their

post-its on the quadrants at the same time.

4. Review each post-it looking for similarities or differences. Discuss

the evidence that led to the placement.

5. Select two or three important changes and discuss:

§ What is effective/ineffective about the explicitness of the

strategy?

§ What is effective/ineffective about the culture for change?

Focusing Direction

Three Keys to Maximizing
Impact

The Lead Learner:
The Principal’s New Role

To increase impact, principals should use their time differently: they

should direct their energies to developing the group.

The Principal’s New Role To lead the school’s teachers in a process of learning to improve their
teaching, while learning alongside them about what works and what

doesn’t.

Module 2

Notes:

Cultivating Collaborative Cultures

Cultivating Collaborative
Cultures

Within-School Variability

§ Variability of performance between schools is 36%, while variability

within schools is 64%. —OECD (2013)

Turn and Talk § Read the excerpt from John Hattie and discuss what the meaning
of ‘within school variability’ is.

Introduction

Hattie, J. (2015). What Works Best in Education: The Politics of Collaborative Expertise, pp. 1-2, Pearson.

The Largest Barrier to Student Learning: Within-School Variability

If we are to truly improve student learning, it is vital that we identify the most important barrier to such

improvement. And that barrier is the effect of within-school variability on learning. The variability between schools

in most Western countries is far smaller than the variability within schools (Hattie 2015). For example, the 2009

PISA results for reading across all OECD countries shows that the variability between schools is 36 per cent, while

the variance within schools is 64 per cent (OECD 2010).

There are many causes of this variance within schools, but I would argue that the most important (and one that we

have some influence to reduce) is the variability in the effectiveness of teachers. I don’t mean to suggest that all

teachers are bad; I mean that there is a great deal of variability among teachers in the effect that they have on

student learning. This variability is well known, but rarely discussed, perhaps because this type of discussion would

necessitate potentially uncomfortable questions. Hence, the politics of distraction are often invoked to avoid

asking them.

Cultivating
Collaborative Cultures
§ Culture of Growth
§ Learning Leadership
§

Capacity Building

§ Collaborative Work

Deepening
Learning
Securing
Accountability
Focusing
Direction
Leadership

Module 3

Overcoming Variability Through Collaborative Expertise

There is every reason to assume that by attending to the problem of variability within a school and increasing the

effectiveness of all teachers there will be a marked overall increase in achievement. So the aim is to bring the effect

of all teachers on student learning up to a very high standard. The ‘No Child Left Behind’ policy should have been

named ‘No Teacher Left Behind’.

This is not asking teachers and school leaders to attain some impossibly high set of dream standards; this is merely

asking for all teachers to have the same impact as our best teachers. Let’s consider some analogies: not all doctors

have high levels of expertise, and not all are in an elite college of surgeons; not all architects are in royal societies;

and not all engineers are in academies of engineers. Just because a doctor, architect or engineer is not a member

of these august bodies, however, does not mean that they are not worth consulting. They may not have achieved

the upper echelon, but they will still have reached a necessary level of expertise to practise.

Similarly, the teaching profession needs to recognise expertise and create a profession of educators in which all

teachers aspire to become members of the college, society or academy of highly effective and expert teachers.

Such entry has to be based on dependable measures based on expertise. In this way, we can drive all upwards and

not only reduce the variability among teachers and school leaders but also demonstrate to all (voters, parents,

politicians, press) that there is a ‘practice of teaching’; that there is a difference between experienced teachers and

expert teachers; and that some practices have a higher probability of being successful than others. The alternative

is the demise of teacher expertise and a continuation of the politics of distraction.

So, my claim is that the greatest influence on student progression in learning is having highly expert, inspired and

passionate teachers and school leaders working together to maximise the effect of their teaching on all students in

their care. There is a major role for school leaders: to harness the expertise in their schools and to lead successful

transformations. There is also a role for the system: to provide the support, time and resources for this to happen.

Putting all three of these (teachers, leaders, system) together gets at the heart of collaborative expertise.

§ Human Capital

§ Social Capital

§ Decisional Capital

What has a greater

impact

on teaching and learning?

§ Teacher appraisal?

§ Professional Development

§ Collaborative Cultures

Cultivating Collaborative Cultures

School Cultures § Talented schools improve weak teachers

§ Talented teachers leave weak schools

§ Good collaboration reduces bad variation

§ The sustainability of an organization is a function of the quality of

its lateral relationships

Freedom To Means § Autonomy & Cooperation

Balancing Autonomy
& Cooperation

§ If you choose

being on your

own you lose the

human

connection

necessary for life.

§ If you succumb to

the extreme of

being absorbed

in a group, you

lose your identity.

Struggle between Autonomy
and Cooperation

§ Countries granting schools independent status freer from

traditional bureaucracies find pockets of innovation among a larger

number of pockets of failure.

§ What is needed for success is to combine flexibility with

requirements for cooperation.

Forms of Cooperation § Building collaborative cultures

§ Participating in networks of schools or districts to learn from each

other

§ Relating to state policies and priorities

Groupthink § …situations where groups are cohesive, have highly directive
leadership, and fail to seek external information. Such groups

strive for unanimity, failing to consider alternative courses of

action.

Module 3

Point & Go? Meet up with a colleague from another table group.

§ Discuss a time you were part of groupthink. What impact did it

have on the group and you personally?

§ What is the power of autonomy?

§ How do you balance autonomy and cooperation?

Notes:

Deepening Learning

Stratosphere

Deep Learning
Competencies

§ The 6C’s provides an advance organizer for thinking about Deep

Learning Competencies as identified by New Pedagogies for Deep

Learning. The placemat organizer can be used to activate prior

knowledge about the 6C’s or to look for examples of the 6C’s

using video exemplars.

Exciting new learning
needs to be:

§ Irresistibly engaging

§ Elegantly efficient

§ Technology ubiquitous

§ Steeped in real life problem solving

§ Involves deep learning

STRATOSPHERE

Deepening Learning

§ Clarity of Learning Goals
§ Precision in Pedagogy
§ Shift Practices Through

Capacity Building
Focusing
Direction
Securing
Accountability
Cultivating
Collaborative
Cultures
Leadership

Module 4

The 6C’s Protocol § Form groups of six with each peer assigned one of the 6C’s.

§ Review the descriptors of the six deep learning competencies. Each

group member will take one competency and provide an example

of what that competency might look like and sound like in practice

or how it is being developed in their classroom or school.

§ Share the examples within the group of six.

§ Select a video of classroom practice and analyze it for examples of

how the six deep learning competencies are being developed. Use

the same graphic organizer to record evidence.

§ Discuss ways to incorporate one or more competencies in future

learning designs.

The 6C’s Protocol

Communication

§ Coherent communication using a range of modes

§ Communication designed for different audiences

§ Substantive, multimodal communication

§ Reflection on and use of the process of learning to improve communication

2. Critical thinking
§ Evaluating information and arguments

§ Making connections and identifying patterns

§ Problem solving

§ Meaningful knowledge construction

§ Experimenting, reflecting, and taking action on ideas in the real world

Collaboration

§ Working interdependently as a team

§ Interpersonal and team-related skills

§ Social, emotional, and intercultural skills

§ Management of team dynamics and challenges

Creativity

§ Economic and social entrepreneurialism

§ Asking the right inquiry questions

§ Considering and pursuing novel ideas and solutions

§ Leadership for action

5. Character
§ Learning to learn

§ Grit, tenacity, perseverance, and resilience

§ Self-regulation and responsibility

§ Empathy for and contributing to the safety and benefit of others

Citizenship

§ A global perspective

§ Understanding of diverse values and worldviews

§ Genuine interest in human and environmental sustainability

Deepening Learning

The 6 C’s of Learning Goals

Communication

Creativity

Critical Thinking

Character

Collaboration

Citizenship

Module 4

My Learning

Deepening Learning

Fullan, M., & Quinn, J. (2015). Coherence, pp. 95-96. Corwin & Ontario Principals’ Council.

My Learning

The first element refers to the need for students to take responsibility for their learning and to

understand the process of learning, if it is to be maximized. This requires students to develop skills in

learning to learn, giving and receiving feedback, and enacting student agency.

§ Learning to learn requires that students build metacognition about their learning. They begin to

define their own learning goals and success criteria; monitor their own learning and critically

examine their work; and incorporate feedback from peers, teachers, and others to deepen their

awareness of how they function in the learning process.

§ Feedback is essential to improving performance. As students make progress in mastering the

learning process, the role of the teacher gradually shifts from explicitly structuring the learning task,

toward providing feedback, activating the next learning challenge, and continuously developing

the learning environment.

§ Student agency emerges as students take a more active role in codeveloping learning tasks and

assessing results. It is more than participation; it is engaging students in real decision-making and a

willingness to learn together.

Deepening Learning

My Belonging

The second element of belonging is a crucial foundation for all human beings who are social by nature

and crave purpose, meaning, and connectedness to others.

§ Caring environments help students to flourish and meet the basic need of all humans to feel they

are respected and belong.

§ Relationships are integral to preparing for authentic learning. As students develop both

interpersonal connections and intrapersonal insight, they are able to move to successively more

complex tasks in groups and independently. Managing collaborative relationships and being self-

monitoring are skills for life.

My Aspirations

Student results can be dramatically affected by the expectations they hold of themselves and the

perceptions they believe others have for them (see also Quaglia & Corso, 2014).

§ Expectations are a key determinant of success, as noted in Hattie’s research. Students must believe

they can achieve and also feel that others believe that. They must codetermine success criteria and

be engaged in measuring their growth. Families, students, and teachers can together foster higher

expectations through deliberate means—sometimes simply by discussing current and ideal

expectations and what might make them possible to achieve.

§ Needs and interests are a powerful accelerator for motivation and engagement. Teachers who tap

into the natural curiosity and interest of students are able to use that as a springboard to deeply

engage students in tasks that are relevant, authentic, and examine concepts and problems in

depth.

Teachers, schools, and districts that combine strategies to unlock the three elements in their students

will foster untapped potential and form meaningful learning partnerships.

How good is your school
at addressing the three
‘mys’?

§ My learning (scale 1-10) = __________

§ My belonging (scale 1-10) = __________

§ My aspirations (scale 1-10) = __________

Reflect on what you can do to accelerate meaningful learning partnerships with students in you school.

Module 4

Deepening Learning

Students, Computers, and
Learning

§ Countries that invest more heavily in ICT do less well in student

achievement.

—OECD, 2015

Early Insights about
Leadership for NPDL:
Direction, Letting Go,
Consolidating

§ A cycle of trying things and making meaning

§ Co-learning dominates

§ Leaders spent a lot of time listening, learning, asking questions

§ Leaders help articulate what is happening, and how it relates to

impact

§ The role of tools is to provide focus and shape without

suffocating context

§ Ultimately you need people to take charge of their own learning

in a context of individual and collective efficacy

Module 4

Notes:

Securing Accountability

Sec uring Ac c ountability

Accountability

Fullan, M., & Quinn, J. (2015). Coherence, pp. 110-111. Corwin & Ontario Principals’ Council.

Simply stated, accountability is taking responsibility for one’s actions. At the core of accountability in

educational systems is student learning. As City, Elmore, Fiarman, and Teitel (2009) argue, “the real

accountability system is in the tasks that students are asked to do” (p. 23). Constantly improving and

refining instructional practice so that students can engage in deep learning tasks is perhaps the single

most important responsibility of the teaching profession and educational systems as a whole. In this

sense, accountability as defined here is not limited to mere gains in test scores but on deeper and more

meaningful learning for all students.

Internal ac c ountability occurs when individuals and groups willingly take on personal, professional,
and collective responsibility for continuous improvement and success for all students (Hargreaves &

Shirley, 2009). “ p. 110-111

External ac c ountability is when system leaders reassure the public through transparency,
monitoring, and selective intervention that their system is performing in line with societal expectations

and requirements. The priority for policy makers, we argue, should be to lead with creating the

conditions for internal accountability, because they are more effective in achieving greater overall

accountability, including external accountability. Policy makers also have direct responsibilities to

address external accountability, but this latter function will be far more effective if they get the internal

part right.

Securing Accountability

§ Internal Accountability
§ External Accountability

Focusing
Direction
Deepening
Learning
Cultivating
Collaborative
Cultures
Leadership

Module 5

Coherence: The Right Drivers in Action for Schools, Districts, and Systems

Fullan, M., & Quinn, J. ( 2015). Corwin & Ontario Principals’ Council, pp. 117-118.

# Quote

1. Accountability is now primarily described as an accountability for student learning. It is less about some
test result and more about accepting ownership of the moral imperative of having every student learn.
Teachers talk about “monitoring” differently. As they engage in greater sharing of the work, they talk
about being accountable as people in the school community know what they are doing and looking to
see what is changing for students as a result. And as they continue to deprivatize teaching, they talk about
their principal and peers coming into their classrooms and expecting to see the work [of agreed-upon
practices] reflected in their teaching, their classroom walls, and student work. (Anonymous, personal
communication, November 2014)

2. Teachers and administrators talk about accountability by deprivatizing their practice. If everyone knows
what the other teacher or administrator is working on and how they are working on it with students, it
becomes a lot easier to talk about accountability. When everyone has an understanding of accountability,
creating clear goals and steps to reach those goals, it makes it easier for everyone to talk and work in
accountable environments. (Elementary principal, personal communication, November 2014)

3. We are moving to define accountability as responsibility. My district has been engaged in some important
work that speaks to intrinsic motivation, efficacy, perseverance, etc., and accountability is seen as doing
what is best for students . . . working together to tackle any challenge and being motivated by our
commitment as opposed to some external direction. (Superintendent, personal communication,
November 2014)

4. I do believe that a lot of work remains to be done on building common understanding on the notion of
accountability. Many people still believe that someone above them in the hierarchy is accountable. Very
few take personal accountability for student learning and achievement. There are still those who blame
parents and students’ background for achievement. (Consultant, personal communication, November
2014)

5. In one school, the talk about accountability was pervasive as the school became designated as
underperforming. The morale of the school went down significantly, and the tension was omnipresent at
every meeting. The team switched the conversation to motivation, innovation, and teamwork and the
culture changed. The school is energized and the test scores went up in one year. The team is now
committed to results and continuous improvement. (Consultant, personal communication, November
2014)

Securing Accountability

T hree Step Interview

1. Form teams of three and letter off A, B. and C.

2. Read the excerpt on ‘Accountability’ from Coherence above and the

five quotes. Think about the responses to the questions below.

3. Begin the cycle with person A as the Interviewer, B as the

Respondent and C as the Recorder using the Advance Organizer.

4. Provide five minutes for each Respondent to respond and then

continue the cycle until all participants have been interviewed.

Question Person A Person B Person C

1. How would

you distinguish

between

Internal and

External

Accountability?

2. Describe

strategies your

school/district

uses to build

Internal

Accountability?

3. What steps

will you take to

ensure the

effective

implementation

of External

Accountability?

Module 5

Know T hey Impac t Turn and Talk:

§ Read the excerpt from John Hattie. What does your school

specifically do to develop a culture of evidence?

Know They Impact!

Hattie, J. (2015). What Works Best in Education: The Politics of Collaborative Expertise,

pp. 15-16. Pearson.

The model advanced here is that the school leader is responsible for asking on a continual basis about the impact

of all the adults on the learning of the students. Of course, I am not forgetting that the students are players in

improving their learning. But that is the bonus, the compound-interest component. What is requested is that

school leaders become leaders in evaluating the impact of all in the school on the progress of all students; the

same for teachers; and the same for students.

School leaders need to be continually working with their staff to evaluate the impact of all on student progression.

Leaders need to create a trusting environment where staff can debate the effect they have and use the information

to devise future innovations. And leaders need to communicate the information on impact and progression to the

students and parents. Schools need to become incubators of programs, evaluators of impact and experts at

interpreting the effects of teachers and teaching on all students.

In short, we need to develop an evaluation climate in our education system.

Experience has shown that ten- to twelve-week cycles of evaluation are about optimal. Fewer weeks tend to lead

to over-assessment or insufficient time to detect change; more weeks and the damage or success is done. We

should know this and react appropriately. It does mean asking teachers to be clear about what success or impact

would look like before they start to teach a series of lessons.

Of course, this must start by asking the questions, ‘Impact on what? To what magnitude? Impact for whom?’

Evaluating impact requires analyses of what a year’s growth looks like, and it is likely it may differ depending on

where the student begins in this growth. Evaluating impact asks schools and systems to be clearer about what it

means to be good at various disciplines, to be clearer about what a year’s progress looks like and to provide staff

with collaborative opportunities to make these decisions.

This is the hardest part of our work, as teachers we have been so ingrained to wait and see what the students do,

to see which students attend and then to pick out examples of successful progress. Our alternative model asks that

teachers be clearer about what success would look like and the magnitude of the impact, and we ask them to

prepare assessments to administer at the end – before they start teaching. The bonus of this latter preparation is

that it ensures that teachers understand what success is meant to look like before they start teaching, and it

increases the likelihood that teachers communicate these notions of success to the students.

There is also a need to include the student voice about teacher impact in the learning/teaching debates; that is, to

hear the students’ view of how they are cared about and respected as learners, how captivated they are by the

lessons, how they can see errors as opportunities for learning, how they can speak up and share their

understanding and how they can provide and seek feedback so they know where to go next. As the Visible

Learning research has shown, the student voice can be highly reliable, rarely includes personality comments and,

appropriately used, can be a major resource for understanding and promoting high-impact teaching and learning.

Securing Accountability

Developing a c ulture of evidenc e

Janet Clinton and I have used the theories of empowerment evaluation to spell out many of these mind frames (in

Clinton and Hattie 2014). Empowerment evaluation is based on the use of evaluation concepts, techniques and

findings to foster improvement. It increases the likelihood that programmes will achieve results by increasing the

capacity of stakeholders to plan, implement and evaluate their own programmes. We argued that we need to

teach educators:

§ to think evaluatively;

§ to have discussions and debates in light of the impact of what they do;

§ to use the tools of evaluation in schools (such as classroom observations of the impact of teachers on students,

interpreting test scores to inform their impact and future actions, and standard setting methods to clarify what

challenge and progression should look like in this school);

§ to build a culture of evidence, improvement and evaluation capacity-building;

§ to develop a mind frame based on excellence, defined in multiple ways, and for all;

§ and to take pride in our collective impact.

Empowerment evaluation helps to cultivate a continuous culture of evidence by asking educators for evidence to

support their views and interpretations and to engage in continual phases of analysis, decision-making and

implementation.

Note to Self How would I describe our evidence based culture?

Module 5

Freedom as Learning Feedback: A Gold Mine of Potential Growth

1. People don’t like feedback and want to be free from it.

2. Feedback is one of the key interacting simplifiers for individuals and

groups wanting to change.

3. To think in terms of active seeking means to think first and foremost

in terms of what receivers of feedback need and can do.

4. Giving and taking feedback are both challenging.

Feedbac k Forum Meet up with another colleague from a different district. Use the
following questions as the basis for your discussion

§ Think of a time when you received powerful feedback. Why was it

powerful? What did you learn from it?

§ What are the challenges of giving feedback?

§ Describe feedback that inspires growth.

Notes

Securing Accountability

Freedom To World § If we recast its role, feedback can become one of the most powerful
forces for the betterment of the individual and the organization.

Best Advic e § Take a risk and seek feedback, both because you will be worse off if
you do nothing and because you will learn from it.

C ultures that Value
Feedbac k

T urn and T alk Does our organization have a culture to support providing/receiving
feedback?

What, if anything, could we do to improve the culture for feedback?

Freedom To:
Ac c ountability

If you are seeking feedback and using feedback as an opportunity to

learn with respect to important goals, you are already on the path of

accountability: a willingness to accept responsibility for your own

actions.

Module 5

Notes:

Freedom to Change

Exploration vs
Engagement

§ What’s out there?

§ Who should we partner with

—Pentland, 2014

C riteria for Effec tive
Networking

1. A small number of ambitious goals (pre-school to tertiary)

2. Leadership at all levels

3. Cultures that produce ‘Collective Efficacy’

4. Mobilizing data and effective practices as a strategy for

improvement

5. Intervention in a non-punitive manner

6. Being transparent, relentless and increasingly challenging

—Rincón-Gallardo & Fullan, in press

New Zealand:

Joint Initiative Agreement

Read the Joint Initiative Agreement

§ What do you like about it?

§ What questions do you have?

§ Discuss implications for your work.

—Rincón-Gallardo & Fullan, in press

New Zealand Education Institute, Ministry of Education

Following up to Working Party Report

Working Party Report – O verarc hing Princ iples

1. Children are at the centre of a smooth and seamless whole of educational pathway from earliest learning to

tertiary options.

2. Parents who are informed and engaged are involved in their children’s education and part of a community

with high expectations for and of those children.

3. Teachers and education leaders, supported by their own professional learning and growth, and those of their

colleagues will systematically collaborate to improve educational achievement outcomes for their students.

4. Teachers and education leaders will be able to report measurable gain in the specific learning and

achievement challenges of their students.

5. Teachers and leaders will grow the capability and status of the profession within clearly defined career
pathways for development and advancement.

Key Learnings From the Working G roup Were:

1. Self-identified Communities of Learning should form around clear learner pathways from early childhood to

secondary education and may, over time, extend to include tertiary learning.

2. Each Community of Learning’s purpose is to enhance student achievement for educational success as set out

in the Vision of the National Curriculum documents; and the Community of Learning should define its own

achievement challenges, learning needs and areas of focus that enable it to support that purpose.

Module 6

3. Self-identified Communities of Learning should form around clear learner pathways from early childhood to

secondary education and may, over time, extend to include tertiary learning.

4. Each Community of Learning’s purpose is to enhance student achievement for educational success as set out

in the Vision of the National Curriculum documents; and the Community of Learning should define its own

achievement challenges, learning needs and areas of focus that enable it to support that purpose.

5. Each Community of Learning will be able to use data, evidence and research to target their efforts and

resources and demonstrate impact on the learning growth of its students.

6. Each Community of Learning should determine its own leadership and teaching, collaboration and support

functions that align with its achievement challenges, making the best use of its own and new resourcing. Some

leadership and teaching roles and their functions will be required for all Communities of Learning; other

functions may be particular to the Community.

7. Any appointment to a leadership role with the required functions will be made by the Community of Learning

in conjunction with an external professional adviser.

8. Successful collaboration changes and evolves, and Communities of Learning must have sufficient flexibility to

enable this rather than limit it.

9. In recognising these factors, each Community of Learning will access its own and new resources to support the

attainment of its goals.

10. A Community of Learning’s success will be dependent on ‘whole of Community of Learning collaboration’.

Therefore, allocation of sufficient time and resources to support participants in the Community of Learning is

critical.

11. The parties commit to undertake further work on Māori, Pasifika, Early Childhood Education, Support Staff,

Special Education and Professional learning and Development to build on the work begun in the Working

Group in the next and final stage of the Joint Initiative Development. The parties acknowledge this may lead

to additional changes in future collective agreement bargaining rounds.

12. Leadership, teaching, collaboration and support roles within Communities of Learning should align with career
pathways for principals, teachers, support and specialist staff to ensure continuous development of leadership

and teaching capacity.

Leadership from the
Middle

§ Where is the coherence—where is the glue?

We find it “in the middle”.

What Ac tions are you going to take home as a result of this workshop?

References

Fullan, M. (2011). Choosing the wrong drivers for whole system reform. Seminar Series 204. Melbourne:
Center for Strategic Education.

Fullan, M. (2013). Great to excellent: Launching the next stage of Ontario’s education reform.
www.edu.gov.on.ca/eng/document/reports/FullanReport_EN_07

Fullan, M. (2013). Stratosphere: Integrating technology, pedagogy and change knowledge. Toronto:
Pearson.

Fullan, M. (2014). The principal: Three keys to maximizing impact. San Francisco: Jossey-Bass.

Fullan, M. (2015). Freedom to change: Four strategies to put your inner drive into overdrive. San
Francisco, CA: Jossey-Bass.

Fullan, M., & Donnelly, K. (2015). Evaluating and assessing tools in the digital swamp. Bloomington, IN:
Solution Tree Press.

Fullan, M., & Quinn, J. (2015). Coherence: The right drivers in action for schools, districts, and systems.
Thousand Oaks, CA: Corwin; Toronto, ON: Ontario Principals’ Council.

Fullan, M., & Rincón-Gallardo, S. (in press). Developing high quality public education in Canada: The case
of Ontario. In F. Adamson, B. Astrand, & L. Darling-Hammond (Eds.), Global education reform:
Privatization vs public investments in national education systems. New York, NY: Routledge.

Fullan, M., Rincón-Gallardo, S., & Hargreaves, A. (2015). Professional capital as accountability. Education
Policy Analysis Archives, 23(15), 1-18.

Hargreaves, A. & Fullan, M. (2012). Professional capital: Transforming teaching in every school. New York:
Teachers College Press.

Hattie, J. (2015). What works best in education: The politics of collaborative expertise. London, UK:
Pearson.

Kirtman, L., & Fullan, M. (2015). Leaders who lead. Bloomington, IN: Solution Tree.

New Pedagogies for Deep Learning (NPDL). (2015). Retrieved from www.NPDL.global

November, A. (2012). Who owns the learning? Preparing students for success in the digital age.
Bloomington, IN: Solution Tree.

Nunan, D. (2003). Practical English language teaching. New York, NY: McGraw-Hill.

Organisation for Economic Cooperation and Development. (2013). Teachers for the 21s century: Using
evaluation to improve teaching. Paris, France: Author.

Pentland, A. (2014). Social physics: How good ideas spread—the lessons from a new science. New York,
NY: Penguin.

Quaglia, R.J., & Corso, M.J. (2014). Student voice: The instrument of change. Thousand Oaks, CA:
Corwin.

Rincón-Gallardo, S., & Fullan, M. (in press). Essential features of effective networks and professional
collaboration. Journal of Professional Capital and Community.

Michael Fullan,OC, is professor emeritus at the Ontario
Institute for Studies in Education, University of Toronto. He

served as special adviser in education to Ontario premier

Dalton McGuinty from 2003 to 2013, and now serves as one

of four advisers to Premier Kathleen Wynne. He has been awarded honorary doctorates from the

University of Edinburgh, University of Leicester, Nipissing University, Duquesne University, and the

Hong Kong Institute of Education. He consults with governments and school systems in several

countries around the world.

Fullan has won numerous awards for his more than thirty books, including the 2015 Grawemeyer

prize with Andy Hargreaves for Professional Capital. His books include the best sellers Leading in a

Culture of Change, The Six Secrets of Change, Change Leader, All Systems Go, Motion Leadership,

and The Principal: Three Keys to Maximizing Impact. His latest books are Coherence: The Right

Drivers in Action (with Joanne Quinn), Evaluating and Assessing Tools in the Digital Swamp (with

Katelyn Donnelly), Leadership: Key Competencies (with Lyle Kirtman), and Freedom to Change.

Special thanks to Joanne Quinn and Eleanor Adam for their training design contributions.

Produced by Claudia Cuttress

Cover Design by BlinkBlink

Please visit our website

michaelfullan.ca

Chapter 4

Deepening Learning

Develop Clarity of Learning Goals

The first step in building precision and consistent practices is to be clear about the learning goals. For the last quarter-century, education has been giving superficial lip service to 21st-century skills without much-concerted action or impact. The energy has been invested in describing sets of skills without much robust implementation or effective ways to measure them. If we want to mobilize concerted action and a profound shift in practice, then governments, districts, and schools need to develop clarity of outcomes and build a shared understanding of these by educators, students, and parents. The CCSS is a step in the direction of more in-depth learning.

NPDL is developing clarity of learning goals for what it calls

deep learning

. Deep learning involves using new knowledge to solve real-life problems and incorporates a range of skills and attributes. The global partnership is working to define with specificity six deep learning competencies (the 6Cs), describe what the learning would look like for each of these, identify the pedagogies that foster those competencies and design new measures to assess student progress in developing them. Their deep learning competency framework and initial descriptors of each competency and its dimensions are displayed in Figure 4.

Citizenship

Communication

Character

6Cs

Critical Thinking

Creativity

Collaboration

Communication:

• Coherent communication using a range of modes

• Communication designed for different audiences

• Substantive, multimodal communication

• Reflection on and use of the process of learning to improve communication

Critical Thinking

• Evaluating information and arguments

• Making connections and identifying patterns

• Problem-solving

• Meaningful knowledge construction • Experimenting, reflecting, and taking action on ideas in the real-world

Collaboration

• Working interdependently as a team

• Interpersonal and team-related skills

• Social, emotional, and intercultural skills

• Management of team dynamics and challenges

Creativity

• Economic and social entrepreneurialism

• Asking the right inquiry questions

• Considering and pursuing novel ideas and solutions

• Leadership for action

Character

• Learning to learn

• Grit, tenacity, perseverance, and resilience

• Self-regulation and responsibility

• Empathy for and contributing to the safety and benefit of others

Citizenship

• A global perspective

• Understanding of diverse values and worldviews

• A genuine interest in human and environmental sustainability

• Solving ambiguous, complex, and authentic problems

The overall purpose of the 6Cs is the well-being of the whole student but also the well-being of the group and society as a whole. Learning becomes the development of competencies for the successful negotiation of an uncertain world. Learning is about developing the personal and interpersonal and cognitive capabilities that allow one to diagnose what is going on in the complex, constantly shifting human and technical context of real-world practice and then match an appropriate response (Fullan & Scott, 2014).

In this context, Fullan and Scott (2014) suggest that well-being and success in life incorporate two big Es: entrepreneurialism and ethics. Increasingly in what we might call the citizen of the future and indeed the present, there should be no distinction between being able to work with your hands and your mind. Entrepreneurialism is being able to resolve complex personal and societal challenges locally and globally. Entrepreneurialism does not just pertain to business endeavors. Every time a group tries to solve a social problem (youth crime, homelessness, bullying, and so on), they require the entrepreneurial skills of critical thinking, problem-solving, innovative ideas, collaboration and communication, and the qualities of character.

The mark of an educated person is that of a doer (a doing-thinker; a thinker-doer)—they learn to do and do to learn. They are impatient with lack of action. Doing is not something they decide to do—daily life is doing, as natural as breathing the air. Along with doing is an exquisite awareness of the ethics of life. Small-scale ethics is how they treat others; large-scale ethics concern humankind and the evolution of the planet. When we change our education system and when hordes of people are acting individually and collectively in entrepreneurial and ethical ways, the world changes and keeps on changing with built-in adaptation. sIn strong critical thinking skills such as interpretation, analysis, synthesis, and evaluation. Students will be not only able to think deeply and independently but also be able to articulate the “why” behind their learning. Students are stretched to use concepts rather than memorize them. Further, these strategies are based on the belief that if students are to flourish in the 21st century, they must take an active role in their education (Hamilton, personal communication, November 2014).

Build Precision in Pedagogy

Schools and districts that make sustained improvement in learning for all students develop explicit frameworks or models to guide the learn- ing process. This instructional guidance system (Bryk, Bender Sebring, Allensworth, Luppescu, & Easton, 2010) is crucial because it represents the “black box” of implementation. The history of education is heavily weighted toward lofty goals and outcomes (usually poorly assessed) but weak on pedagogy. Our Coherence Framework makes pedagogical precision a priority and a driving force.

Instructional or pedagogical (we use the terms interchangeably) systems must include the development of at least the following four components:

• Build a common language and knowledge base. Cultivate system-wide engagement by involving all levels of the system to capture and create a model for learning and teaching. Identify the learning goals and principles that underlie the learning process. This collaborative approach builds language to promote meaningful conversations about practice.

• Identify proven pedagogical methods. The process typically begins with an analysis of best practices currently used in the district and an examination of the research to validate the model. Ownership and commitment emerge at all levels of the system study, work, and learn together.

• Build capacity. Provide consistent and sustained capacity building based on research-proven practices to build precision in pedagogy. Teachers need “a deep multidimensional knowledge that allows them both to assess situations quickly and to draw upon a variety of repertoires for intervention. Individual teachers possess such knowledge but it is largely invisible to the field as a whole. There are few ways for it to be gathered, codified and shared” (Mehta, Schwartz, & Hess, 2012). Collective capacity building and the collaborative work processes in previous chapters make the knowledge and skills accessible and visible to all.

• Provide clear causal links to impact. Pedagogies should specify the two-way street between learning and assessment. Such a process serves to strengthen the specificity of instructional practice and its causal efficacy in making a difference to learning. This is what Hattie (2012) is getting at with his mantra “know thy impact.” Knowing your impact is not just a matter of being responsible for outcomes but it also reverberates back to clarify how teaching and learning can be strengthened.

In the NPDL work, we have identified three strands of expertise that teachers need to weave together if they are to support deeper learning. These are precision in pedagogical partnerships that engage students in codesigning authentic, relevant learning,

learning environments

that foster risk-taking and 24/7 connections, and leveraging digital, so it accelerates learning.

We examine each of the three strands of the NPDL depicted in Figure 4.3 and then

Pedagogical Partnerships

The first strand recognizes that teachers must possess deep expertise in instructional and assessment practices if they are to maximize the impact and use of digital to accelerate learning. These new pedagogies build on the foundation of proven pedagogical practices but fuse them with emerging innovative practices that foster the creation and application of new ideas and knowledge in real life. Educators must hone a deep understanding of the learning process and a repertoire of strategies if they are to use digital as an accelerator. The magic is not in the device but the scaffolding of experiences and challenges finely tuned to the needs and interests of students and maximized through relevance, authenticity, and real-world connections.

with a culture that fosters learning for all. If the adults are not thinking at high levels, it is unlikely the students will be either. Districts and schools that get results have clarity about the elements of their instructional system. They build knowledge from the research combined with best practices in their context and then ensure that everyone has the skills and resources to apply them appropriately.

Schools and districts who want to build a common language and knowledge base and identify proven pedagogical practices may want to consider the work of John Hattie in Visible Learning (2009). He reviews the impact of instructional strategies and concludes that what is needed to raise the bar and close the gap is consensus and skill development by all teachers engaged with groups of students around the most impactful strategies. He differentiates the role of teachers as facilitators that has a .17 impact on learning with the role of teachers as an activator at .87. The role of teachers as activators is far more powerful as it is more active in engaging student learning and challenging the next practice.

No learning-teaching process is complete without addressing the black box of assessment. In our NPDL work, we are not only identifying the pedagogies that affect learning but also creating new tools and measures for student success. We are shifting from measuring what is easy to measure what matters. If we want students to develop the 6Cs of communication, critical thinking, collaboration, creativity, character, and citizenship, we need to be able to define and measure those competencies. To that end, we have created learning progressions that describe the pathway many students would follow in developing a competency. These tools become the anchor for meaningful discussion as groups of teachers design more meaningful learning based on the competencies; students and teachers develop success criteria, monitor progress, and evaluate growth. Teams of teachers then collaboratively examine student work and processes to analyze the quality of both the learning design and student progress. They use these data to identify the next appropriate learning challenge.

The new pedagogies go deeper than changing pedagogy between students and teachers; they explore more deeply new roles for students. One of the most distinctive differences between traditional learning and the new pedagogies is the role students play and “the new learning partnerships” that emerge student to student, student to teacher, and student to the external world. New learning goals require changes in how relationships between students, teachers, families, and communities are structured. The shift toward active learning partnerships requires students to take greater charge of their own and each other’s learning inside and outside the classroom. The new learning partnerships have the potential to create more authentic and meaningful learning locally, nationally, and globally. This more active role increases student engagement. The shift to a new balance in decision making is inevitable because students are no longer willing to be passive recipients of learning defined by someone else, are digitally connected to massive amounts of new ideas and information, and respond to traditional academic approaches with passivity once they have foundational skills.

Schools and districts that embrace the new learning partnership are seeing exponential growth in student engagement and success. We filmed recently in W. G. Davis Middle School in Ontario, wherein 2009 students were disengaged, disruptive behaviors were on the rise, and achievement was dropping. The principal and staff collaborated over several months to find a solution. They eventually determined that their students needed better role models and the kind of digital engagement they valued outside the school. They realized that they were the ones who had to become role models for their students. This began a process of implementing more authentic learning using problem-based units that crossed traditional content boundaries and implementing a new bring your device (BYOD) policy. The shift to cross-disciplinary planning and increased technology use caused teachers to move outside of their comfort zone. They began using new strategies for co-planning and using digital resources supporting one another and feeling supported to take risks and even fail at first. Almost immediately, they noticed their roles with students were changing dramatically. Students were more engaged and teacher time in the classroom was spent on giving feedback and challenging the next step in learning rather than in delivering content. As they focused on meaningful, relevant learning using what we are calling the new pedagogies, they also saw more than a 20 percent leap in reading and writing scores over three years on the provincial testing (Video: W. G. Davis,

www.michaelfullan.ca

The new learning partnerships we saw at W. G. Davis take time and expertise to develop. Meaningful learning partnerships with students can be accelerated when teachers understand the three elements of the student learning model, depicted in Figure 4.4.

This model goes beyond the notions of student voice and agency to combine both internal development and external connections to the world. We are not talking here about student forums or interest surveys (although they may be part of the approach) but about a deeper engagement of students as codesigners and co-learners. The three elements of the model all contribute to the development of students as active, engaged learners who are prepared to learn for life and experience teaching as life. Educators need to be aware of these critical elements to design learning and environments that maximize student potential to thrive. Moreover—and this is crucial—none of these three components are fixed variables. They can be altered through intervention. This domain represents a vastly underutilized set of factors that would be very high yield (low cost, high impact). The student learning model then focuses on the three elements of student development and the ways they become active participants in my learning, my belonging, and my aspirations.

My Learning

The first element refers to the need for students to take responsibility for their learning and to understand the process of learning if it is to be maximized. This requires students to develop skills in learning to learn, giving and receiving feedback, and enacting student agency. • Learning to learn requires that students build metacognition about their learning. They begin to define their own learning goals and success criteria, monitor their learning and critically examine their work, and incorporate feedback from peers, teachers, and others to deepen their awareness of how they function in the learning process. • Feedback is essential to improving performance. As students make progress in mastering the learning process, the role of the teacher gradually shifts from explicitly structuring the learning task, toward providing feedback, activating the next learning challenge, and continuously developing the learning environment. • Student agency emerges as students take a more active role in co-developing learning tasks and assessing results. It is more than participation; it is engaging students in real decision making and a willingness to learn together.

My Belonging

The second element of belonging is a crucial foundation for all human beings who are social by nature and crave purpose, meaning, and connectedness to others.

• Caring environments help students to flourish and meet the basic need of all humans to feel they are respected and belong. • Relationships are integral to preparing for authentic learning. As students develop both interpersonal connections and intrapersonal insight, they are able to move to successively more complex tasks in groups and independently. Managing collaborative relationships and being self-monitoring are skills for life. My Aspirations Student results can be dramatically affected by the expectations they hold of themselves and the perceptions they believe others have for them .

• Expectations are a key determinant of success, as noted in Hattie’s research. Students must believe they can achieve and also feel that others believe that. They must codetermine success criteria and be engaged in measuring their growth. Families, students, and teachers can together foster higher expectations through deliberate means— sometimes simply by discussing current and ideal expectations and what might make them possible to achieve. • Needs and interests are a powerful accelerator for motivation and engagement. Teachers who tap into the natural curiosity and interest of students are able to use that as a springboard to deeply engage students in tasks that are relevant, authentic, and examine concepts and problems in depth.

Learning Environments

The second strand that fosters the transformation to deep learning is a shift in the learning environment. Quality learning environments that use the pedagogical practices and build the learning partnerships described previously need to meet four criteria: be irresistibly engaging for students and teachers, allows 24/7 access to learning, cultivate social learning, and foster risk-taking and innovation. Students thrive in this type of learning environment and so do teachers.

How, then, do we transform today’s classrooms from the traditional status quo to places of energy, curiosity, imagination, and deep learning? A recent video by the inventor of the Rubik’s cube, Erno Rubik, sheds light on the dilemma when he asks, “How do we get teachers to stop teaching answers but instead to help students generate questions that are waiting for answers?” There is no one recipe for creating classrooms that provoke deep learning, but as we look across the early innovators, we see a few common characteristics. In schools on the pathway to deepening knowledge, we see the following:

• Studentsaskingthequestions.Theyhaveskillsandlanguagetopur- sue inquiry and are not passively taking in the answers from teachers. • Questions valued above solutions. The process of learning, discovering, and conveying is as essential as a result. • Varied models for learning. The selection of approaches is matched to student needs and interests. Students are supported to reach for the

next challenge. • Explicitconnectionstoreal-world application.Learningdesignsare

not left to chance but scaffolded and built on relevance and meaning. • Collaboration. Students possess skills to collaborate within the

classroom and beyond.

• The assessment of learning is embedded, transparent, and authentic. Students define personal goals, monitor progress toward success criteria, and engage in feedback with peers and others.

Leveraging Digital

The third strand of the deep learning trio is leveraging digital. We have purposely moved away from the term technology to signal that this discussion is not about devices but about learning that can be amplified, accelerated, and facilitated by interaction with the digital world. This demands a rethinking of the ways we use technology. It’s not about putting a device in front of every student and leaving them to learn independently. That will only result in students who are digital isolates. It is about bringing the digital world inside the process of learning and building collaboration, within and outside the classroom, in ways that are authentic and relevant. Alan November (2012), a pioneer in the meaningful use of technology for over three decades, describes this new view of the digital world as “transforming learning beyond the $1000 pencil.” Just adding devices is not enough; mindsets and behaviors need to change for both students and teachers. He emphasizes that students must be taught how to use technology appropriately, safely, and ethically to gain understanding at the highest levels (Bloom’s taxonomy or depth of knowledge). Teachers then “guide students in the complex tasks of innovation and problem solving, and in doing work that makes a contribution to the learning processes of others” (November 2012, p. 18).

The challenge for leaders is to help educators move from uses of technology as a substitution to methods of digital that provide value. If I’m a student studying a unit on poverty and I use technology to create a PowerPoint instead of handwriting a report, there may be little value-added. In contrast, if I interview people in four global communities who are living in poverty, synthesize that information, and create my report, there has been tremendous value-added through the layers of critical thinking, communication, character, and global citizenship.

Making the New Pedagogies “gel”

Building capacity in all three strands of the new pedagogies takes persistence and commitment. We find an excellent example of sustained focus that gets new and better results in our work with Napa Valley Unified School District. The district is making progress in building on powerful pedagogical practices—particularly problem-based learning and leveraging digital. Napa has developed a clear instructional focus on what they term their 4Cs and combines that with the growing use of digital. The approach began more than a decade ago at New Tech High but has evolved to engage the entire district. Napa intentionally built the capacity of teachers in every school, over time, to use the new pedagogy and then used the addition of digital devices to enrich the thinking and learning. They have taken an approach to innovation by starting with some schools but using that learning in rapid cycles of reflection and doing to diffuse the learning to all schools. Each year they host an “Educators Exchange” to share the knowledge they are gaining with their schools but also laterally with other school districts. Schools and districts need to foster collaborative inquiry into the three strands of the new pedagogies: pedagogical partnerships, learn- ing environments, and leveraging digital. There is no simple recipe; this is a job for professional educators who must develop the expertise and knowledge base that is a foundation for fostering deeper learning. The simplexity is knowing the elements and integrating them so that every child has the learning experience that challenges and supports them. The challenge for schools and districts is to build momentum across all classrooms.

Once districts and schools have clarified the learning goals and developed precision in pedagogical practices, they must focus on the “how” of shifting practice. They need to identify the processes that will support a shift in practice for all educators. We will highlight the key attributes and then illustrate with examples in action. As we look at districts that are making the shift to support deep learning, we see that several conditions are in place. Superintendents strategies noted in Chapter 3. • The model being lead learners. They don’t send people to capacity building sessions but learn alongside them. • They shape a culture that fosters an expectation of learning for everyone, taking risks and making mistakes but learning from them. • They build capacity vertically and horizontally in the organization with persistence and single-mindedness until it affects learning. How do schools and districts tackle the shift to deep learning? The first step in making a change is to assess the starting point. We offer a few questions for reflection about your capacity to shift the practices in your school, district, or state.

Assessing Capacity

Teachers: 1. Do teachers possess knowledge and skills in pedagogical practices? 2. Do teachers have knowledge and skills to develop new learning partnerships? 3. Do teachers have the knowledge and skills to create learning environments that move beyond the traditional classroom? 4. Do teachers have the knowledge and skills to use digital resources to accelerate learning?

Schools:

1. Do school leaders have the knowledge and skills to create a culture of learning for teachers and students? 2. Do schools have collaborative learning structures and process? 3. Do schools have access to models of effective practice and opportunities to share laterally and vertically?

2. Districts:

3. 1. Does the district have clarity of learning goals? 2. Have high-yield pedagogical practices been identified and shared? 3. Does the district create a culture of learning for all educators? 4. Does the district provide resources for collaborative learning structures and processes to thrive?

We use examples to illustrate how schools and districts can use the elements of the Coherence Framework to assess their starting point and then either focus on continuous improvement of the basic literacies or sustain those basics while innovating with deeper learning. The first school example is Cochrane Collegiate Academy in North Carolina that in 2008 lacked clarity of goals, had little precision or consistency in pedagogy, and had weak capacity and culture to support change. They needed to focus relentlessly on continuous improvement of the basics. The second school example is Park Manor Senior Public School in Ontario, which had some clarity of goals, good pedagogy, and teacher capacity but was underperforming. They combined continuous improvement with innovating with deep learning and digital and saw their writing scores soar.

Cochrane Collegiate Academy

We look first to a school that was able to engage an underperforming student population with dramatic results using pedagogical precision and capacity building. In 2007, Cochrane Collegiate Academy in Charlotte, North Carolina, was listed as one of the 30 lowest-performing schools in North Carolina. By 2011, the number of students performing at grade level had doubled and the achievement gap had been reduced by 35 per- cent in reading and math. Most notable was that their growth was 3.5 times that of North Carolina in mathematics and twice the rate of growth in reading. Cochrane serves a population of 640 students in grades 6 through 8. Eighty-seven percent qualify for free or reduced lunch, 60 percent are African American, and 30 percent are Latino. In a recent Edutopia (n.d.) video, teachers described the situation in 2008 as out of control with students running and screaming in the halls, weak performance at 20 percent in reading and math, and good teachers choosing to leave the profession. Staff attributes their success to their principal who brought out their potential using five key components: 1. Use quality professional development that is research-based, consistent, convenient, relevant, and differentiated. 2. Use time wisely by flipping faculty meeting time to focus on learning, not administration. 3. Trust your teachers to determine the professional learning they need next. 4. Facilitate, don’t dictate by providing teachers with what they need and allowing them to make decisions. 5. Expect the best by holding everyone to high standards. Guided by research, they identified their top 10 teaching practices and engaged weekly in professional learning to help them implement the practices more effectively. Their non-negotiable list of strategies included the following: essential questions, activating strategy, relevant vocabulary, limited lecture, graphic organizer, the student movement, higher-order thinking questions, summarize, rigorous, and student-centered.

What differentiates this school is not which top 10 instructional strategies they selected but the fact that they built a common language, knowledge base, and set of practices about quality learning and teaching. They instituted pr.actices and processes such as weekly professional learning targeted to this instructional guidance system. Strong professional relationships, collaborative work, and learning partnerships with their students are making the difference. They have work still to be done but are on a trajectory for success.

Park Manor Senior Public School

The second school example is Park Manor, which serves grades 6, 7, and 8 students just outside of Toronto. It is a normal school with the same standard resources of all schools in that district. In Stratosphere (2013c), Fullan profiled the innovations at Park Manor for two reasons. First, they increased scores on the Ontario assessment, which measures higher-order skills, from 42 percent to 83 percent in just four years. Second, they applied what we are calling the three strands of the new pedagogies to shift practice across the entire school. Park Manor’s stated mission is to develop “global critical thinkers collaborating to change the world.” The goal is clear and concise, and everyone shares it. Many schools have inspiring goals, but Park Manor was an early innovator in developing a clear strategy for moving forward. Their approach was to build a collaborative culture that was learning together how to do this work. James Bond, the principal, and Liz Anderson, the learning coordinator, facilitated a process where they and the teachers developed clarity about what learning needed to be like to serve their students. They developed as a staff what they call an accelerated learning framework to guide the transition from goals to action (see Figure 4.5). Over two years, they developed several versions of the framework and still see it as a work in progress. Teachers explained the following:

We begin with the student and then embed the 6Cs into everything. From there, we develop the learning goals, success criteria, productive learning tasks and then make decisions about the most appropriate pedagogy. Only then do we consider the digital tools and resources that will accelerate the learning? (Video at www.michaelfullan.ca) While they are committed to incorporating digital, they learned early on that pedagogy had to be the driver with digital acting as an accelerator. Visitors to the school are always impressed that every student can articulate their learning goals and success criteria, the reasons for the digital or pedagogical strategy they may be using, and how the tools are meeting their learning needs.

Three indicators of success have evolved: first, gains in student achievement have been significant; second, the school uses success criteria and evidence to determine the effectiveness of the framework as it relates to student learning; and third, the notion of developing a learn- ing framework has been taken up by other schools across North America. Schools and districts are seeing the development of a learning framework as a powerful process to build shared language, knowledge, and expertise. The framework serves to clarify the small number of goals, identify the pedagogical practices that need to be in every teacher’s repertoire, and provide a focus for capacity building that gets results.

deep learning

pedacogical parternships

leveraging digital

learning environments

Full Terms & Conditions of access and use can be found at
https://www.tandfonline.com/action/journalInformation?journalCode=utne20

The New Educator

ISSN: 15

7-688X (Print) 1549-9243 (Online) Journal homepage: https://www.tandfonline.com/loi/utne20

Using Internal and External Evaluation to Shape
Teacher Preparation Curriculum: A Model for
Continuous Program Improvement

Patricia J. Norman & Sara A. S. Sherwood

To cite this article: Patricia J. Norman & Sara A. S. Sherwood (2015) Using Internal and
External Evaluation to Shape Teacher Preparation Curriculum: A Model for Continuous Program
Improvement, The New Educator, 11:1, 4-23, DOI: 10.1080/1547688X.2015.1001263

To link to this article: https://doi.org/10.1080/1547688X.2015.1001263

Published online: 12 Feb 2015.

Submit your article to this journal

Article views: 533

View related articles

View Crossmark data

Citing articles: 2 View citing articles

https://www.tandfonline.com/action/journalInformation?journalCode=utne20

https://www.tandfonline.com/loi/utne20

https://www.tandfonline.com/action/showCitFormats?doi=10.1080/1547688X.2015.1001263

https://doi.org/10.1080/1547688X.2015.1001263

https://www.tandfonline.com/action/authorSubmission?journalCode=utne20&show=instructions

https://www.tandfonline.com/doi/mlt/10.1080/1547688X.2015.1001263

http://crossmark.crossref.org/dialog/?doi=10.1080/1547688X.2015.1001263&domain=pdf&date_stamp=2015-02-12

https://www.tandfonline.com/doi/citedby/10.1080/1547688X.2015.1001263#tabModule

The New Educator, 11:4–23, 2015
Copyright © CCNY and ATE
ISSN: 1547-688X print/1549-9243 online
DOI: 10.1080/1547688X.2015.1001263

Using Internal and External Evaluation to
Shape Teacher Preparation Curriculum: A

Model for Continuous Program Improvement

PATRICIA J. NORMAN AND SARA A. S. SHERWOOD
Trinity University, San Antonio, Texas, USA

The same high stakes accountability measures that have shaped
the K-12 educational context over the past decade have become
part of the university teacher preparation landscape. National
accrediting organizations have created more rigorous, out-
comes-based accountability measures for teacher preparation pro-
grams. Educational stakeholders offer mixed reviews about the
value and impact of these accreditation efforts. In this article,
however, we argue that such external evaluation measures when
paired with our own internal evaluation processes have gener-
ated meaningful changes to our elementary teacher preparation
program.

INTRODUCTION

Although well-crafted teacher preparation programs impact preservice
teacher learning (Brouwer & Korthagen, 2005; Hart, 2002; Levin, 2003; Yao,
2010), ongoing questions about the effectiveness of teacher preparation pre-
vail (Greenberg & Walsh, 2010; Washburn, Joshi, & Cantrell, 2011). The
same high stakes accountability measures (NCLB, 2008) that have shaped
the k-12 educational context over the past decade have become part of the
university teacher preparation landscape. Concerned about the perceived
inadequacy of teacher preparation, the U.S. Secretary of Education has urged
programs “to make better outcomes for students the overarching mission that
propels all their efforts” (Duncan, 2009).

In response to these concerns, organizations like the National Council
for the Accreditation of Teacher Education (NCATE) and the Teacher

Address correspondence to Patricia J. Norman, Trinity University, One Trinity Place, San
Antonio, TX 78212-7200, USA. E-mail: pnorman@trinity.edu

mailto:pnorman@trinity.edu

Using Internal and External Evaluation 5

Education Accreditation Council (TEAC)— now consolidated to form the
Council for Accreditation of Educator Preparation (CAEP)— have created
more rigorous, outcomes-based accountability measures for teacher prepa-
ration programs (NCATE, 2008). These standards assess programs across
multiple domains and require them to develop and implement assess-
ments systems that clearly demonstrate that their candidates have developed
the knowledge, skills and dispositions to be effective classroom teachers.
In addition, these organizations routinely review and revise their standards
to reflect the latest research in best practices in teacher preparation and
teaching (Curtis, Bordelon & Teitelbaum, 2010).

Educational stakeholders offer mixed reviews about the value and
impact of NCATE (Dillon & Silva, 2011).1 Some find NCATE accreditation
a meaningful process that encourages programs to set priorities and to
monitor progress (Dhonau, 2007; Meyer, 2010); others describe NCATE’s
standards as internally inconsistent and ineffective in improving teacher train-
ing and quality (Fein, 2004; Levine, 2006). We find that NCATE’s requirements
and standards have served to strengthen our already nationally recognized
program.2 Notably, the assessment process helps to systematize the data
collection process and improvement strategies already in place within our
department.

This article describes and analyzes how combining NCATE’s external
evaluation with our own internal evaluation processes has generated mean-
ingful changes to the elementary teacher preparation program. In particular,
we describe our approach to continuous program improvement, offering
examples of how both internal and external evaluation measures have
helped us strengthen the elementary program. We conclude by exploring
the benefits and challenges encountered through the process.

TRINITY UNIVERSITY’S MASTER OF ARTS
IN TEACHING PROGRAM

Trinity University is a small, private liberal arts university in the south central
United States with an enrollment of 2,600 students. Trinity serves a predom-
inately White (62%) student body whose median SAT score nears 1300. The
Education Department at Trinity offers a five-year Master of Arts in Teaching
(MAT) program leading to certification in one of three levels: (a) early child-
hood through grade six, (b) grades four through eight, and (c) grades eight
through twelve. As undergraduates, students at Trinity major in a content

1Given the recent consolidation of NCATE and TEAC into CAEP in July 2013, the impact of
this new accreditation agency is not yet known.

2Although NCATE is now part of CAEP, our teacher preparation program is accredited by
NCATE through spring, 2018.

6 P. J. Norman and S. A. S. Sherwood

area and take a dozen hours of supporting education coursework including
two practica, one field seminar, one human growth and development course,
and one special education course. Thus there is no undergraduate major in
education offered. After receiving their bachelor’s degree, students enroll in
the Department’s one-year MAT program where they complete a five-week
summer program before entering a one-year, unpaid, mentored internship
at a local elementary school. Throughout their internship, candidates com-
plete graduate coursework designed to introduce them to best practices as
well as help them both prepare for and make sense of their field-based
experiences.

The Education Department typically graduates 35-45 students per year
across all three teacher preparation programs and 10-15 within the elemen-
tary program. Historically, the program has had a 100 percent pass rate on
its state certification exams and 100 percent placement of its students in
teaching jobs after graduation. Given the small, “boutique” nature of Trinity’s
three preparation programs, we find it relatively easy to make programmatic
changes since there are so few faculty who teach within the MAT program.
Pat has served as the elementary program coordinator for the past thirteen
years, teaching two required undergraduate courses, four of six required
graduate courses, and supervising students in their year-long internships.
Sara has taught as an adjunct professor in the elementary program for eight
years as both an undergraduate and graduate instructor and university field
supervisor to MAT students.

CONTINUOUS PROGRAM IMPROVEMENT MODEL

As illustrated in Figure 1, internal and external evaluations run parallel to one
another, and both impact decisions about how to maintain, refine and revise
the program. Our model for continuous program improvement begins with
the EC-6 MAT curriculum that is implemented across the graduate students’
ten-month program. The curriculum generates a pool of yearly collected
data that drives both internal and external evaluation. In the case of inter-
nal evaluation, we routinely collect a variety of data to support ongoing
assessment and program development including assessment results of candi-
date work, student/instructor course evaluations, written reflections gathered
at the end of every weekly graduate seminar, and an end-of-year focus
group interview held with the elementary MAT candidates. Selected data,
including information gathered from our own direct experience teaching the
curriculum, is then used to drive our analysis and reflection. For instance,
the two of us might engage in a conversation to discuss the feedback gen-
erated during the candidates’ focus group interview. Those conversations
and independent analyses lead us to explore implications for programmatic

Using Internal and External Evaluation 7

t
e

r
n

a
l

E
x

t
e
r
n
a
l

Exploration of

Implications

Synthesis of Data

External Feedback

Data Analysis and Reflection

Maintenance, Refinement or Revision of Curriculum

Data Pool

Implementation of Early Childhood-Grade 6

Master of Arts in Teaching (MAT) Curriculum

FIGURE 1. Model for Continuous Program Improvement.

change. At this stage, possibilities for revision are identified and a deci-
sion is ultimately made to implement an explicit revision, refine an existing
curricular component, or maintain an aspect of the program as is.

In the case of external evaluation, NCATE requires that programs
address individual specialty areas, such as the elementary standards designed
by the Association for Childhood Education International (ACEI). To address
these standards, we collect data from a minimum of six different assessments
that address the following: licensure, content knowledge in elementary edu-
cation, candidates’ ability to plan instruction, student teaching, candidates’
effect on student learning, and an additional assessment that addresses
ACEI’s standards. The specific data that we collect to address these six
assessment areas include: the state’s Generalist Early Childhood – Grade
6 teacher certification exam; grade point average on undergraduate content
coursework or content exam such as the Praxis; a curricular unit that uses
Understanding by Design principles; an exit portfolio that addresses Trinity’s
Standards for Professional Practice; a Teacher Work Sample; a Dispositional
Assessment; and a set of Teaching for Understanding entries that demon-
strates candidates’ ability to plan, teach and assess students’ learning in all
content areas.

8 P. J. Norman and S. A. S. Sherwood

Data analysis and synthesis are largely driven by NCATE requirements
and include:

● A brief description of the assessment and its use in the program;
● A description of how each assessment specifically aligns with the

standards it cites;
● A brief analysis of the data findings;
● An interpretation of how that data provides evidence for meeting

standards; and
● Assessment documentation, including assessment tool itself; the scoring

guide for the assessment; and charts that provide candidate data derived
from the assessment (NCATE, 2013).

ACEI formally reviews the program’s formal synthesis of data and provides
external feedback, which we explore and use to maintain, refine or revise
our program.

In the following two sections, we provide specific examples of how
internal and external evaluation drive program improvement in Trinity’s
elementary MAT program.

INTERNAL EVALUATION: ANOTHER LOOK AT CLASSROOM
MANAGEMENT

A series of recent examples of how we have engaged in internal evaluation
to strengthen the MAT elementary curriculum focuses on classroom man-
agement. We believe that the best place to learn how to teach— including
managing students’ behavior and learning— is in a classroom under the
thoughtful guidance of a mentor teacher who can induct novices into the
practice of teaching. Candidates are able to learn about management not only
through their careful observation and study of their mentor’s practice but also
through their ongoing efforts to establish their moral authority and manage
students’ behavior and learning as they assume increased teaching responsi-
bility across their year-long internship. Interns receive direct support in learn-
ing to teach across the year from their mentor teacher and university supervi-
sor. In addition, their graduate coursework is designed to help them prepare
for and make sense of their field-based experiences. Thus one venue for sup-
porting their ability to manage a classroom learning community is through
their graduate coursework where we deliberately create space for candidates
to examine best practices and reflect on their own emerging practice.

Ongoing activities and assessments connected to candidates’ graduate
courses and their classroom-based internship create a diverse data pool to
support internal evaluation. In the following sections, we define the spe-
cific data we analyzed and how that reflection on the data led us to revise,

Using Internal and External Evaluation 9

refine and maintain different aspects of the MAT curriculum as it relates to
classroom management.

An Example of Program Maintenance

An ongoing way that students reflect on and evaluate their classroom man-
agement skills is through the weekly process of Checking In. In both the
fall and spring semesters, the 10-15 interns in the elementary cohort attend a
weekly three-hour seminar. One weekly ritual that we devote 20-30 minutes
to is “Checking In.” This is a time for each candidate to share something
for our collective examination and problem solving, or to get a different
perspective. Interns are given a minute or two to collect their thoughts
about one pressing issue, celebration, question, or concern connected to
their classroom-based experience. Each intern first takes several minutes
to describe a field-based experience then the group responds. They often
describe difficult encounters with a particular child or the entire class that
involve management issues. Their fellow interns and course instructor serve
as helpful resources not only for making sense of what transpired but also
possible next steps to take.

Several weeks into the fall semester, for example, an intern in a first
grade classroom used Checking In to share a concern about a child in
her classroom who was “in constant motion” and made continuous sounds
throughout the day. The intern explained that sensory information seemed
difficult for the child to manage. Although the child was a phenomenal
reader, oral communication was challenging for her. In addition, transitions—
be they between lessons or between different locations in the school—
presented significant difficulty for her. The intern was concerned that several
students were becoming increasingly frustrated with the child who some-
times inadvertently interrupted their learning with the noises and body
movements she made. In response to her concern, we suggested that learn-
ing how to manage her own emotional response to a situation is an important
part of helping other students manage their own responses. In addition, we
gave her the titles of two picture books by Jennifer Veenendall, one about a
boy with Autism and the other about a girl with a sensory processing disor-
der. We encouraged her to read one or both books aloud to help the child’s
classmates better understand children who may act different from them.

During Checking In the following month, the intern reported that she
felt increasingly comfortable working with the student who had initially
intimidated her. The intern now recognized what a wonderful sense of
humor the child had, something she hadn’t initially appreciated or noticed.
Although the intern had not shared the suggested picture books as read
alouds, she had borrowed language from the picture books when privately
consulting with two students who frequently grew frustrated with the child

10 P. J. Norman and S. A. S. Sherwood

making noises in order to understand how they could all learn and work
together even though they may have different needs.

We have received consistently positive feedback about Checking In. For
example, during a recent focus group interview held at the end of the MAT
experience, one student stated: “I liked taking time to check in—it was a
really important part of class. It gave everyone a chance to offer suggestions
and get it all out there.” A second student chimed in:

Checking-in helped me to clarify my own thinking about what was going
on in my classroom. Talking through a situation helped me better under-
stand what was going on myself, and it also allowed others to support
me with encouragement and suggestions. It was a regular opportunity to
share, so you didn’t have to go asking for feedback or begging others for
advice. You knew the time would be there every week, so it was a relief to
not have to try to find time to have those important conversations. I always
felt better after checking in because I felt like everyone was not only root-
ing for me, but they were invested in my situation, and cared about the
outcome and how I got there.

Because of interns’ steady appreciation for “Checking In” as well as our
own observation that they frequently act on the feedback they receive once
back in their classrooms, we have maintained this component. Checking In
enables candidates to get support and feedback from both their peers and
professors in order to grow their practice.

An Example of Program Refinement

In the fall semester, a second way that interns explore classroom manage-
ment is through their first formal inquiry project called The First Days of
School. Anchored by the question “How does my mentor teacher begin the
process of creating a classroom learning community?” the inquiry addresses
these sub-questions: (1) What kind of classroom learning community does
my teacher value? Why are these qualities or features important? (2) How do
the arrangement of space and the uses of time support this kind of learning
community? (3) What rules, routines and procedures need to be established
for the classroom to function smoothly? How does my teacher go about this
during the first days/weeks of school? and (4) What kind of learning does
my teacher value? How is this communicated to students?

Interns collect a variety of data during their investigation, including for-
mally interviewing their mentor and observing their mentor and students.
They create a map of the physical lay out of the classroom then use a proto-
col to examine whether and how the decisions that a teacher makes about
the physical environment reflect her values and beliefs about teaching and
learning. In addition, interns study the uses of time in classrooms, comparing

Using Internal and External Evaluation 11

and contrasting the typical daily schedule in their mentors’ classrooms and
how the mentors’ decisions about the uses of time reflect their priorities
and values. Interns record and analyze how rules, routines and procedures
are introduced and contribute to the functioning of the mentor’s classroom
learning community. Finally, the interns investigate how a culture of learning
is created in their mentor’s classroom, including whether and how students
are encouraged to learn with and from each other. The inquiry culminates in
a written reflective memo that interns formally submit for evaluation.

We have used the First Days of School inquiry project with elementary
interns since 2001 and have analyzed data collected from a variety of sources
including focus group interviews and course evaluations. For example, when
asked in the formal course evaluation to “comment on the assignment from
which you learned the most and why,” five of the eleven interns in a recent
cohort named the First Days of School inquiry. One student wrote, “The
First Days of School assignment helped me realize how much/little class
community there was in my class and gave me an idea of what I want my
ideal classroom to look like.” Another commented, “After drawing a map of
my classroom and having to show it to other members of the class I realized
how important set-up can be.” Yet another intern wrote, “I learned a great
deal during the First Days of School Inquiry. I really had to examine my
mentor’s actions. I learned that everything she says and does has a specific
purpose.” Finally, one intern explained, “It was so rewarding to be able
to synthesize the work we had done at the beginning of the year in our
reflective memo, to recognize how our actions, teaching, and practice were
affecting our students and the classroom community, and to discuss and
think about what we could do next.”

While this formal data suggests that interns find this initial inquiry into
classroom learning community a worthwhile learning opportunity, we have
made refinements to this program element based on our own professional
judgment. For example, we have cut down on the amount of formal writing
required. In years past, we asked interns to formally write and submit com-
mentary on the physical map of their mentor’s classroom, the way students’
time is scheduled in their classroom, and the rules/routines introduced in
their classroom as well as whether/how a culture of learning is established.
Because we have added additional teaching responsibilities and graduate
coursework connected to the teaching of specific content in the fall semester,
something had to give. Past experience has taught us that we need to take
care in helping interns balance their internship-based responsibilities and
their graduate course commitments. Thus we now use graduate class ses-
sions to examine classroom maps, analyze the introduction of rules/routines,
and together consider a set of questions designed to help interns explore
the culture of learning in their mentor’s classroom. We no longer, however,
require that interns submit written essays on these aspects. Instead, we only
require them to submit a final reflective memo that includes both an analytic

12 P. J. Norman and S. A. S. Sherwood

description of what interns have come to understand about the emerging
classroom community and 2-3 insights and/or questions that probe beneath
the surface of classroom life.

An Example of Program Revision

Across their MAT experience, Trinity students engage in additional learning
opportunities around classroom management. Until 2010, for example, we
introduced interns to non-traditional approaches to classroom management
during the five-week summer term that they complete before entering their
internship. Specifically, interns studied Beyond Discipline: From Compliance
to Community (Kohn, 2006) to launch our investigation. We did so out of
the belief that most candidates would be steeped in traditional notions of
management once they entered their internships. We wanted to offer an alter-
native vision of doing “with” rather than “to” students before being immersed
in punishment and reward systems both in their internships and their initial
years of teaching.

An exit focus group interview with the 2010-11 cohort created an impor-
tant data source to analyze. During the interview, candidates are asked to
reflect on strengths of the MAT program and to offer suggestions for improve-
ment. Candidates are given a few minutes to gather their thoughts before
they share their ideas with one another while we script their responses.
In addition, we often share our own ideas for program improvement at the
end of the focus group interview to elicit candidates’ constructive feedback.

When asked what suggestions the candidates had for improving
the program, a candidate noted, “Kohn was too radical.” Her classmate
responded:

I liked Kohn because it was so radical but maybe balancing it with
some other types would be helpful. Reading that right before going into
the classroom held me back from doing some things I would have done
management-wise. It was always in my mind, ‘Am I doing it the Alfie
Kohn way?’ It would help to have more discussions on different philoso-
phies of classroom management, and it might help to talk about our
classroom management beliefs before reading Kohn.

Based in part on candidate feedback and subsequent conversations, we
realized that introducing candidates to Kohn’s ideas before starting their
internship created significant challenges for the MAT students. The stu-
dents’ feedback enabled us to see that by starting with Kohn’s treatment
of conventional management practices, the interns often found themselves
placed in classrooms where traditional approaches to management were
the norm. This left interns feeling uncomfortable adopting management
practices based on punishments or rewards. However, they were unable
to establish alternative practices, whether from fear of undermining their

Using Internal and External Evaluation 13

mentor teacher’s established rules and expectations and/or the lack of skills
needed to do so.

Jointly reflecting on this feedback, we significantly revised the way
we addressed classroom management across the program the following
year. In the summer course, we now use Teaching Children to Care:
Classroom Management for Ethical and Academic Growth K-8, by Ruth
Charney (2002). Our intention is to equip interns with concrete manage-
ment strategies that maintain children’s dignity. Our goal is to introduce
candidates to a set of management practices that enable them to establish
their moral authority, build classroom learning community, and establish
classroom rules and student expectations. Once candidates begin their year-
long internship, we support students’ exploration of classroom management
in several ways. First, they continue to complete the First Days of School
inquiry project. Second, candidates continue to share their own challenges
in managing students’ behavior and learning during “Checking in” in the
weekly graduate seminar. In addition, we assigned and discussed Learning to
Trust: Transforming Difficult Elementary Classrooms Through Developmental
Discipline (Watson, 2003) in the graduate seminar. The text chronicles how a
classroom teacher struggles to enact principled management practices in her
classroom. Finally, in the spring semester, after interns have had a chance to
establish their moral authority and help their mentor maintain rules and pro-
cedures for the smooth operation of the classroom, we examine Alfie Kohn’s
less conventional ideas.

This revised curricular sequence seems to have worked quite well these
past two years. From candidates’ written course feedback, they felt well pre-
pared to begin their internship after the summer session. For example, one
wrote: “I loved our examination of classroom management. It really helped
me prepare for the internship. I now feel better with my authority in the
class and the organization of the room.” Another commented: “I feel like
we have talked about a variety of different management options and I am
prepared and excited to take them into my classroom this fall.” In our own
observations of interns’ classroom-based work during the fall semester, the
interns seemed much more comfortable and able to establish their teacher
presence and address discipline problems than interns have in years past.
Moreover, during a recent exit focus group interview, one candidate men-
tioned, “We took a really good look at classroom management and that was
really helpful.”

EXTERNAL EVALUATION: PLANNING, TEACHING,
AND ASSESSING IN THE CONTENT AREAS

In addition to continually collecting and reflecting on data to revise and
improve the program internally, we have found that the NCATE accreditation

14 P. J. Norman and S. A. S. Sherwood

process creates another important opportunity, be it more formal and system-
atic, to collect and reflect on data in order to strengthen the MAT program.
Similar to our internal evaluation process, external evaluation begins with
the gathering and analysis of data that grows from the curriculum. The dif-
ference is that while we can select any data to analyze during the internal
process, including our own professional judgment that grows out of our
direct experience teaching the curriculum, ACEI/NCATE provide clear guide-
lines about what data is to be collected and synthesized for formal reporting.
Furthermore, ACEI/NCATE provides formal external feedback that we are
required to address. In other words, during internal evaluation, when we
get to the point of exploring implications for our program, this process is
completely voluntarily whereas with external evaluation it is mandatory.

As stated earlier, NCATE requires programs to collect data from a mini-
mum of six different assessments that address licensure, content knowledge
in elementary education, candidates’ ability to plan instruction, student teach-
ing, candidates’ effect on student learning, and an additional assessment that
addresses the specific standards with which we align our program, in this
case ACEI. We submitted Trinity’s elementary education program report to
ACEI in March, 2010.

We received ACEI’s response, much of which was positive, four months
later. In their national recognition report, they wrote:

The program is very well designed and has structured a thoughtful set
of major assessment points. Results are strong, and program review of
assessment data is systematic. Of particular note is the clarity with which
assignments, assessments and the program are described to candidates
(and program reviewers).

Against this glowing feedback, however, the reviewers identified a significant
concern within the ACEI Curriculum standards. These standards assess can-
didates in terms of their content knowledge of and ability to plan and teach
reading, writing and oral language; science; mathematics; social studies; the
arts; health education; and physical education. Specifically, the reviewers
commented:

The program assessments show clearly the alignment between the stan-
dard and the candidates’ assignments. The resulting data, however, are
reported by the domain or section, where multiple ACEI standards are
aligned to one domain or section. Activities or requirements within assess-
ments must be set off in such a way that each task or requirement is specif-
ically designed to measure competency on a single standard or standard
element. Because the assessment and/or rubric are designed to evaluate
performance across ACEI multiple standards in some cases, the resultant
data cannot be used as evidence of successful candidate performance on
individual standards or elements . . . Assessments in their entirety must

Using Internal and External Evaluation 15

demonstrate evidence that candidates “know, understand and use” the
concepts in Standards 2.1-2.7. None of the assessments cited for this stan-
dard evaluate candidates on curriculum-specific performance in terms of
instructional planning or implementation.

In other words, ACEI reviewers raised concerns that although the assess-
ments that we used— namely the state’s EC-6 Generalist and EC-6 Pedagogy
and Professional Responsibilities certification exams—addressed all seven
curricular areas, the assessments did not assess candidates’ ability to plan
and teach those seven content areas.

Initially this feedback was challenging to digest. The state’s EC-
6 Generalist exam is designed to assess not only content knowledge across
all curricular domains but also knowledge of instruction and assessment
within those domains. Trinity elementary candidates have scored at least
23 points higher than the state average on this exam for the past three years.
Moreover, our teacher candidates’ initial pass rate on both certification exams
has been 100 percent for the past six years, much higher than the 80 percent
initial pass rate that NCATE requires. How were we going to create seven
separate assessments to evaluate candidates’ content knowledge of and abil-
ity to plan and teach within ACEI’s identified seven content areas? This is
exactly what we had to do if we wanted to move the elementary program
from being “nationally recognized with conditions” through August, 2012 to
being “nationally recognized” until 2017.

An Example of Program Improvement in Response to External
Evaluation

The NCATE process gives programs up to eighteen months to address “con-
ditions” identified in the national recognition report. After taking the first
two months to process this feedback and develop a plan, we then imple-
mented and refined it over the following fourteen months. We first identified
ways that we already internally assess candidates’ ability to plan for, teach
and assess student learning across specific content areas. For example, at
the end of their internship, interns are required to develop an “exit port-
folio” which includes a series of entries demonstrating that interns have
met Trinity’s Standards for Professional Practice. These standards include
“Planning for Students’ Learning” and “Teaching and Assessing: Engaging
All Students in Learning.” For the previous six years, as part of this sum-
mative assessment, interns had been required to develop four Teaching for
Understanding entries in reading, mathematics, social studies and science.
For each of the four entries, interns included a unit or lesson plan that they
had developed and taught then described why this content was important for
students to learn. In addition, they had to describe one or more of the learn-
ing activities implemented, explaining how these tasks fit identified goals

16 P. J. Norman and S. A. S. Sherwood

and how learning differences were accommodated. Finally, candidates gath-
ered several student work samples created from the lesson/unit, outlined
criteria used to assess students’ performance, assessed each student, and
then explained how this assessment of students’ work helped them further
support children’s learning.

Because these four Teaching for Understanding entries were not inde-
pendently assessed but were rather part of a summative assessment including
additional exit portfolio entries, ACEI did not accept the exit portfolio as
demonstrating interns’ ability to plan for, teach and assess students’ learn-
ing in the seven content areas (reading, mathematics, social studies, science,
health, physical education, and the arts). Thus we first decided to decouple
the Teaching for Understanding entries from the exit portfolio.

In the first year, we spaced the four Teaching for Understanding entries
across the spring semester as part of students’ field-based experience and
graduate coursework. We revised the entry descriptions so that candidates
had clearer expectations for what was required of these now stand-alone
projects. In addition, we added a fifth entry—Integration—where interns
are required to develop a lesson or unit that integrates health, physical
education and the arts then to collect and assess students’ work generated
during the lesson(s). Finally, we developed a single formal assessment rubric
for all five entries with the following criteria: goals for student learning;
discipline-specific unit or lesson plan; discipline-specific teaching methods;
and assessment of student learning. Language was provided for what it
means to approach, meet and exceed expectations across the four criteria.

Based on feedback we received from candidates during an exit focus
group interview, we made several revisions the following year. First, we
spread the five entries across the summer, fall and spring semesters after
learning from the interns that it was too challenging to complete all five
in a single semester on top of their classroom teaching responsibilities and
completion of other graduate coursework. To do so, interns completed the
reading Teaching for Understanding entry during the summer session which
focuses on reading development, instruction and assessment. In addition, we
revised the structure of one of their fall graduate courses so that each month
was devoted to the examination of a particular content area: mathematics in
September, social studies in October, and science in November. Candidates
completed those three entries during that corresponding month in the fall.
We also structured their classroom teaching responsibilities to coincide with
the planning, teaching and assessment of those content areas by month as
well. Candidates completed the Integration entry in the spring semester in
order to leave the bulk of their attention to “lead teaching” where they
assume primary responsibility for planning, teaching and assessing students’
learning for six consecutive weeks.

A final revision that we made in the second year was to stop approach-
ing the Integration entry as a requirement that had no merit other than

Using Internal and External Evaluation 17

to meet ACEI/NCATE requirements. We have always prided ourselves on
ensuring that program requirements are relevant, worthwhile and directly
supportive of novices’ learning to teach. Initially, we could not see the bene-
fit of asking interns to complete the Integration entry other than for external
accreditation. However, the high quality lessons that they developed and
taught in response to this new requirement completely changed our minds.
As a brief example, an intern in a fourth grade classroom not only inte-
grated health, the arts and PE but did so in the context of teaching reading.
Her reading goal centered on helping students understand how to glean
and use information in procedural texts and documents, including deter-
mining the sequence of activities needed to carry out a procedure. After
providing instruction on how to read a “how to” text, the intern introduced
two strategies to manage stress (e.g. breath technique and personal mantra),
thus incorporating health and PE goals. The text she provided them focused
on how to create a piece of origami. Students were expected first to read
through the text in its entirety before creating their own piece of origami
by following the instructions. If they experienced frustration while creating
the art, they were encouraged to take responsibility for their personal health
and well being by using the breath and/or mantra technique the intern had
taught them.

The intern drew on several forms of assessment to determine whether
students had met her learning goals, including recording anecdotal notes. For
example, the first step was for students to read the text from beginning to end
before starting to create their own origami. The intern could observe right
away who did or did not read the text before attempting their origami. She
also assessed students’ reading comprehension by viewing the final origami
product. In addition, students completed a self-assessment rubric. In terms
of the health and PE goals, the intern observed whether the students used
the stress reducing strategies before asking for her help. We observed the
intern teach this lesson and along with the intern were struck by the high
level of engagement that students demonstrated. The lesson provided a very
practical and engaging way to not only explore and unpack procedural text
but also to learn important PE and health content/skills.

After submitting the five Teaching for Understanding entries along with
an analysis of data findings from the first two years of implementation to
ACEI, last fall we received word that the EC-6 MAT program has become
nationally recognized. In their response to our revisions, ACEI commented:

The report provided the necessary evidence to show that the area of
candidates’ ability to understand and apply pedagogical and profes-
sional content knowledge, skills, and dispositions had been carefully
addressed and documented. This report shows that Trinity University used
the information provided in the earlier ACEI SPA report to make signif-
icant changes to improve their program and to respond to suggestions

18 P. J. Norman and S. A. S. Sherwood

for improvement. The changes have been implemented and the program
meets the requirements . . .

DISCUSSION

Educators need both formative and summative feedback to improve the
quality of their instruction and students’ learning (Marzano, 2010). Teacher
educators are no different. As our model (see figure 1) demonstrates,
although we draw from a diverse pool of data, the internal and external
evaluation processes work together to inform our practice and strengthen
the MAT program over time. Our own internal evaluation measures create
important formative assessment points. In contrast, our external evaluation
measures, namely the NCATE accreditation process, provide essential sum-
mative feedback by asking us to track and analyze multiple data points over
time. Each form of evaluation provides its own benefits.

Internal evaluation measures give students an important and central
voice in programmatic improvement. Although we use our professional judg-
ment when making program revisions, it is essential to hear directly from
candidates themselves how they experience those decisions. Their ongoing
written and oral feedback provides an important window into their thinking.
For instance, we feel strongly that Alfie Kohn’s notion of working “with”
children rather than doing “to” them is essential to establishing a classroom
learning community. While the interns appreciated Kohn’s ideas, we learned
during an exit focus group interview that implementing his ideas from the
outset of their internship was challenging. This insight led us to revamp our
approach to teaching classroom management, including moving our exami-
nation of Alfie Kohn’s ideas to the end of their internship rather than keeping
it at the beginning.

A second benefit to internal evaluation is that it fuels our efforts to reflect
on and improve our own practice. A major premise of our teacher prepara-
tion program is that good teachers are students—of children and childhood,
learners and learning, curriculum and pedagogy. In other words, teaching is
“the learning profession” (Darling-Hammond & Sykes, 1999) where contin-
ual inquiry is central to a teacher’s personal and professional growth over a
lifetime of practice (Barth, 1990). We want our teacher candidates to become
reflective practitioners, routinely engaging in the analysis of their teaching
and students’ learning. We deliberately model how we ourselves make sense
of our own pedagogy and their learning, including how we collect and
process their feedback— daily/weekly/over time— to make programmatic
changes. As discussed earlier, a case in point involves the exit focus group
interview that we hold with interns. We elicit their thoughts about strengths
of the current program and suggestions for improvement while also shar-
ing changes that we are already considering making based on their ongoing

Using Internal and External Evaluation 19

feedback throughout the year. We then ask them to consider whether/how
those proposed changes make sense given their experience in the program.

A third benefit to internal evaluation is that it provides space for and
validates our own professional judgment. All of the weekly Checking Ins,
the weekly observations and debriefings of interns’ classroom practice, the
end-of-class written reflections, the continuous email and face to face con-
versations and consultations that we hold with them continuously inform
whether/how we are creating educative learning opportunities for candi-
dates. In this sense, just as we want our own candidates to feel that they are
doing “with” us rather than being done “to” in the program, we appreciate
having a voice in determining whether/how to respond to data we believe
is important to collect and analyze, not just the data points that we are
required to collect and analyze for national accreditation. We have become
so data driven in education that it is easy to feel that only numbers matter;
in doing so, however, we ignore at our peril educators’ direct knowledge
of students drawn from years of practice and routinely conducted informal
assessments. These less formal assessment arenas are central to engaging in
program improvement because they address aspects of teacher candidates’
teaching and learning that are not easily quantifiable.

That said external evaluation has its own benefits. NCATE provides a
second set of eyes and a broader group of colleagues to think about and
analyze our program’s success and progress. When analyzing one’s own pro-
gram, it can be challenging to identify blind spots: “When you are already
familiar with a culture or group or school, your angles of vision are nar-
rowed by preformed assumptions about what is going on” (Glesne, 1999,
p. 25). External evaluation measures force us to think outside of our own
context and to consider concerns that are not otherwise apparent as an
insider. For instance, we never would have considered making the Teaching
for Understanding entries stand-alone assessments had the external concern
not been raised that we were not demonstrating that our candidates possess
content-specific planning, teaching and assessment skills. Doing so, how-
ever, created greater coherence with the way the program is structured.
Assignments now have a meaningful time and purpose within the program.
In addition, the Teaching for Understanding entries are more rigorous now
that there is greater clarity with what a high-quality practice looks and sounds
like in a given content area. Moreover, the intern receives specific feedback
on her own planning, teaching and assessment efforts in relationship to that
quality.

External evaluation also allows us to model to our teacher candidates
how to approach high-stakes testing measures. We want them to see the
role that state standards and assessments, along with other forms of data,
can play in informing their instructional decision-making. In this data driven
world, it is not just about collecting required data but also using those data
to inform decisions about curriculum and instruction. We have to actively

20 P. J. Norman and S. A. S. Sherwood

seek out external feedback and make sense of it, remaining true to our own
values/beliefs while also being responsive to concerns raised. We have been
transparent with them how we have processed NCATE feedback and used it
to make programmatic improvements, pointing out that this process parallels
what they must do as classroom teachers in light of standards-based student
achievement data.

Finally, external evaluation measures hold teacher educators account-
able. In order to meet requirements of national accreditation, a program
must demonstrate that it has done what it said it was going to do. Making
program results public means that we must move from a stance of “this is
what I hope the program is” to “this is what the program is” by providing
concrete outcome-based evidence to support those claims.

While we advocate a model of program improvement that embraces
both internal and external evaluation components, such a model is not with-
out challenge. Although both forms of evaluation increase rigor in our pro-
grammatic decision-making, they also create enormous stress on students to
complete all formal and informal assessments. On the one hand, the time that
our teacher candidates spend documenting and demonstrating the quality of
their teaching and children’s learning creates both important reflection points
for candidates and important data points for program evaluation. On the
other hand, that writing and reflecting time means less time for candidates’
actual classroom instruction. Determining the most beneficial combination of
classroom teaching and documentation of instructional performance remains
an ongoing tension that we envision we must learn how to manage rather
than resolve over time. This tension reflects the ever-present difficulty that
classroom teachers face. Because of increased accountability measures, so
much time is devoted to administering tests that there is less time to actually
instruct students. It is also equally challenging to find time to use the results
of such measures to inform future instruction in meaningful ways.

As teacher educators, we struggle to score all of the required assess-
ments, both those needed for national accreditation and those that we have
deemed important for our candidates’ growth, in a timely fashion. Keeping
the administrative work manageable is another challenge.

We strive to ensure that our program improvement model informs our
practice rather than becomes it. As labor intensive as the model is, both
our professional experience and our external accreditor tells us that our
candidates leave our program with the knowledge, skills and dispositions
needed to plan, teach and assess elementary students’ learning effectively.

IMPLICATIONS

Even though our model of program improvement grows out of our work
in a small teacher preparation context, larger institutions can experience

Using Internal and External Evaluation 21

many of the same benefits from combining internal and external evalua-
tion efforts. Using an external accrediting body in concert with collecting
formative assessment data from students and faculty insure that program
improvement is responsive to the voices of both candidates and faculty while
also addressing external organization standards. Many of the tensions that we
face will undoubtedly surface for larger teacher preparation institutions as
well. Any institution, regardless of size, must attempt to balance teacher can-
didates’ clinical experiences with the time needed to analyze and write about
those experiences as part of formal and informal data points. Managing the
ongoing stress that both students and faculty face in generating and assessing
those data points is another shared tension.

That said we imagine that the size of larger schools and departments
of education also create unique challenges. In our program, our internal
and external coordination efforts involve less than one dozen faculty and
staff. In larger programs, a significant challenge lies in using internal and
external assessment efforts to create shared understandings and program-
matic changes beyond any individual educator’s practice or coursework.
Coordination efforts not only need to address the collaboration of multi-
ple instructors within a single course but also engage faculty across the
entire teacher preparation curriculum to ensure that over time, the curriculum
reflects teacher educators’ evolving understanding of the alignment between
program goals, coursework, and learning outcomes for candidates as well as
the ongoing feedback that candidates and external evaluators provide.

Larger teacher preparation programs must help all stakeholders develop
a shared vision for teaching and learning that undergird the program. They
must also construct shared understandings of the knowledge, skills and dis-
positions that candidates must demonstrate all while knowing how any given
component of the program enables those learning outcomes. Given the sheer
number of faculty involved in larger teacher preparation programs, a chal-
lenge lies in developing that shared vision and assessment system without
becoming overly prescriptive about how teacher educators teach. In the
same way that we want to avoid creating teacher candidates who simply
become implementers of what someone else has declared that they should
do to students, leaders within large teacher preparation programs must grap-
ple with the question of how to avoid de-professionalizing their own teacher
educators.

REFERENCES

Barth, R. (1990). Improving schools from within: Teachers, parents, and principals
can make the difference. San Francisco, CA: Jossey-Bass.

Brouwer, N., & Korthagen, F. (2005). Can teacher education make a difference?
American Educational Research Journal, 42(1), 153–224.

22 P. J. Norman and S. A. S. Sherwood

Charney, R. (2002). Teaching children to care: Classroom management for ethical
and academic growth. Turner Falls, MA: Northeast Foundation for Children.

Curtis, D., Bordelon, K., & Teitelbaum, K. (2010). Keep a focus on meaningful reform
efforts instead of political agendas. Planning and Changing, 41(3/4), 133–146.

Darling-Hammond, L. & Sykes, G. (Eds.). (1999). Teaching as the learning profession:
Handbook of policy and practice. San Francisco, CA: Jossey-Bass.

Dhonau, S. (2007). Creating a culture for the preparation of an ACTFL/NCATE
Program Review. Foreign Language Annals, 40(2), 247–259.

Dillon, E., & Silva, E. (2011). Grading the teachers’ teachers: Higher education comes
under scrutiny. Phi Delta Kappan, 93(1), 54–58.

Duncan, A. (2009, October). Teacher Preparation: Reforming the Uncertain
Profession. Speech presented at Teachers College, Columbia University,
N.Y. Retrieved from http://www.ed.gov/news/speeches/teacher-preparation-
reforming-uncertain-profession

Fein, A. (2004). What’s good for the goose . . . Inconsistency in the NCATE Standards.
Multicultural Education, 11(4), 54–55.

Greenberg, J., & Walsh, K. (2010). Evaluating the Fundamentals of Teacher Training
in Texas. Washington DC: National Council on Teacher Quality. Retrieved from
http://www.nctq.org/edschoolreports/texas/docs/report

Hart, L. C. (2002). Preservice teachers’ beliefs and practice after participating in an
integrated content/method course. School Science and Mathematics, 102(1),
1–14.

Kohn, A. (2006). Beyond discipline: From compliance to community 2nd edition.
Alexandria, VA: Association for Supervision and Curriculum Development.

Levine, A. (2006). Educating school teachers. Washington, DC: The Education
Schools Project. Retrieved from http://www.edschools.org/pdf/Educating_
Teachers_Report

Levin, B. (2003). Answering the “So what?” question: What do these case studies tell
us? Case studies of teacher development: An in-depth look at how thinking about
pedagogy develops over time (pp. 233–299). Mahwah, NJ: Lawrence Erlbaum
Associates.

Meyer, C. (2010). Program Assessment: Ways to make it meaningful while waiting
for NCATE. National Social Science Journal, 34(2), 105–109.

Marzano, R.J. (2010). Formative assessment and standards-based grading.
Bloomington, IN: Marzano Research Laboratory.

National Council for the Accreditation of Teacher Education (NCATE). (2013).
Guidelines on assessment. Retrieved from http://www.ncate.org/Accreditation/
ProgramReview/GuidelinesAndProcedures/GuidelinesonAssessment/tabid/446/
Default.aspx

National Council for the Accreditation of Teacher Education (NCATE). (2008).
Professional Standards for the Accreditation of Teacher Preparation Institutions.
Washington DC: Author. Retrieved from http://www.ncate.org/Portals/0/
documents/Standards/NCATE%20Standards%202008

No Child Left Behind Act of 2001, 20 U.S.C. § 6319 (2008). Retrieved from http://
www2.ed.gov/policy/elsec/leg/esea02/index.html

http://www.ed.gov/news/speeches/teacher-preparation-reforming-uncertain-profession

http://www.nctq.org/edschoolreports/texas/docs/report

http://www.edschools.org/pdf/Educating_Teachers_Report

http://www.ncate.org/Accreditation/ProgramReview/GuidelinesAndProcedures/GuidelinesonAssessment/tabid/446/Default.aspx

http://www.ncate.org/Portals/0/documents/Standards/NCATE%20Standards%202008

http://www2.ed.gov/policy/elsec/leg/esea02/index.html

Using Internal and External Evaluation 23

Washburn, E.K., Joshi, R.M., & Cantrell, E.B. (2011). Are preservice teachers prepared
to teach struggling readers? Annals of Dyslexia, 61, 21–43.

Watson, M. (2003). Learning to trust: Transforming difficult elementary classrooms
through developmental discipline. San Francisco, CA: Jossey-Bass.

Yao, Y., & Williams, W. (2010). A comparison of three teacher preparation programs
in terms of their impact on teacher competencies. Action in Teacher Education,
32(3), 15–24.

ABSTRACT

INTRODUCTION

Trinity University’s Master of Arts in Teaching Program
Continuous Program Improvement Model
Internal Evaluation: Another Look at Classroom Management

An Example of Program Maintenance
An Example of Program Refinement
An Example of Program Revision

External Evaluation: Planning, Teaching, and Assessing in the Content Areas

An Example of Program Improvement in Response to External Evaluation

Discussion

Implications
REFERENCES

Tracking Data

Tracking Data
Program Transcript

[MUSIC PLAYING]

FEMALE SPEAKER: Thank you all for meeting with me today. As you’ve heard,
the federal government has mandated states to set up accountability systems for
students who are on IEPs and IFSPs to ensure they’re making progress.

FEMALE SPEAKER: This came up at the state special ed directors meeting, so
we all knew this was coming.

FEMALE SPEAKER: I’m glad you have some background on this information.
Progress reporting on this level is going to be a big change.

MALE SPEAKER: Oh, great. Our early childhood special education staff are
already overwhelmed with paperwork. Now what else do they have to do?

FEMALE SPEAKER: The states mandated that the districts set up a database to
collect ongoing data about IEP goals and objectives for all students in special
education. This system will monitor progress and keep special education
professionals accountable for creating intervention plans for those students not
making progress.

MALE SPEAKER: I don’t see how this is going to help.

FEMALE SPEAKER: It really is in the best interest of the students. I’m hopeful
that this process will be helpful for the education teachers and not a hindrance.

MALE SPEAKER: I’m glad you can be positive but I doubt the staff will see it that
way.

FEMALE SPEAKER: In any case, I’m asking you to work together to create a
strategic plan for how you’ll deliver this news at our next staff development day.
And plan a staff training session soon after so they can learn the new system.

FEMALE SPEAKER: We can handle it.

MALE SPEAKER: Yeah, we could put something together.

FEMALE SPEAKER: It’s important that this be presented in a positive way, with
clear goals, so that we can make this change as smooth as possible.

The Mandate Meeting

The Mandate Meeting
Program Transcript

MALE SPEAKER: Welcome everyone. We have a lot of information to cover this
morning, so let’s get started. I’ve invited special education director Carolina
Reyes to join us this morning to help present the first topic.

CAROLINA REYES: Good morning everybody.

MALE SPEAKER: As many of you have heard, the state has mandated that each
district put in place a database to collect ongoing data about IEP goals and
objectives for every special education student.

CAROLINA REYES: To put this in perspective, this is coming down to states
from the federal government. Recent research has shown that the students aren’t
making adequate progress in special education. So the intent of this new system
is to increase the skills of students with exceptionalities. This tracking system will
be directly tied to student IEPs.

MALE SPEAKER: The special education case managers will be responsible for
putting the data into the district database. But it’s really going to be a team effort
for everyone who works with students with exceptionalities to monitor the data
and implement interventions.

CAROLINA REYES: I can tell by some of the reactions on your faces that this
may seem overwhelming. But I want you to know that I have already been
through the training. And I can tell you this system is easy to navigate. I am
confident that the data you receive will do great things for our students. And it will
allow us to better rapport both data and progress to parents.

MALE SPEAKER: Principals have also had an orientation on the system. And I
agree with Ms. Reyes. The system is set up to provide teachers with excellent
data on any skills that any of the students are working on.

MALE SPEAKER: What all does the system do? Like, I’m afraid this is going to
take a lot of my time.

CAROLINA REYES: In addition to collecting data and graphing it for teachers,
the system has extensive lists of student goals ideas to help with IEP writing, and
has a tool to find evidence-based interventions for when a student isn’t
progressing as expected. It really is cool. I wish I had this system when I was
teaching in the classroom.

MALE SPEAKER: The district has set up training sessions that will occur in the
next couple of weeks. And there’s going to be a trained support person in every
building to help navigate the new database system.

The Mandate Meeting

CAROLINA REYES: I have other locations to visit today, but we still have time for
some questions.

FEMALE SPEAKER: I have a question. Will the special education case
managers be given more time in their daily schedules to input all of this
information?

MALE SPEAKER: You’ll need to build this time into your current schedules and
prep times.

FEMALE SPEAKER: How can we possibly do that, given our current case loads
and the amount of paperwork we’re already doing?

MALE SPEAKER: It has to have already been an expectation that special
education staff keep data on student IEPs goals and objectives. The only
difference is now we have a uniform system for everyone.

MALE SPEAKER: I’m just as concerned as Rebecca. I know I can speak for most
of us when I’m saying that we’re already spending hours outside of our
contracted time completing the paperwork we do now. All this paperwork is
taking away from our students and time away from our families.

MALE SPEAKER: I know this will take some time to adjust to. And I know that
change can be difficult. I’m hoping that everyone will come to this with an open
mind, and that you will move forward with a positive attitude.

MALE SPEAKER: Excuse me. Speaking from the general education side. I’m
also concerned about the amount of time this is going to take. We track data from
all the other curriculum. And now you want us to do this as well?

CAROLINA REYES: How your teams set up the data collection process and the
needs of individual students will determine how much tracking will be done by
general education teachers.

MALE SPEAKER: In this building since we already have weekly teaming time.
Those teams will simply add this to their weekly agenda.

MALE SPEAKER: Our weekly planning time is already consumed with
discussions concerning our general education students, and their progress
towards state testing. This is just ridiculous.

MALE SPEAKER: We have a great staff here. And we have always been able to
pull together to work on any of the changes that we as educators are constantly
facing. I feel confident that we can address all of your issues as they arise, and
that collaboratively we can figure out ways to address those issues to make the
process easier.

The Mandate Meeting

CAROLINA REYES: Again, I just want to reinforce that this system has several
benefits and should make progress tracking easier. The bottom line is that this is
for the benefit of our students. And that is why we are all here.

MALE SPEAKER: I’ll be working closely with Ms. Reyes. And I will be taking your
input to the leadership as thing arise as we unfold the new system. I want to
thank Ms. Reyes for being here today and to talking with us today.

Now we have a lot more information to get through today. So I want to suggest
that we take a break. Then we’ll come back and look at the other items on the
agenda.

CAROLINA REYES: Thank you for having me, and have a great rest of the day.

COHERENCE Chapter 5

Securing Accountability

Internal Accountability Simply stated, accountability is taking responsibility for one’s actions. At the core of accountability in educational systems is student learning. As City, Elmore, Fiarman, and Teitel (2009) argue, “the real accountability system is in the tasks that students are asked to do” (p. 23). Constantly improving and refining instructional practice so that students can engage in deep learning tasks is perhaps the single most important responsibility of the teaching profession and educational systems as a whole. In this sense, accountability as defined here is not limited to mere gains in test scores but on deeper and more meaningful learning for all students. Internal accountability occurs when individuals and groups willingly take on personal, professional, and collective responsibility for continuous improvement and success for all students (Hargreaves & Shirley, 2009).

External accountability is when system leaders reassure the public through transparency, monitoring, and selective intervention that their system is performing in line with societal expectations and requirements. The priority for policymakers, we argue, should be to lead with creating the conditions for internal accountability, because they are more effective in achieving greater overall accountability, including external accountability. Policymakers also have direct responsibilities to address external accountability, but this latter function will be far more effective if they get the internal part right.

Existing research on school and system effectiveness and improvement (DuFour & Eaker, 1998; Marzano, 2003; Pil & Leana, 2006; Zavadsky, 2009) and our own work with educational systems in the United States and internationally (Fullan, 2010; Hargreaves & Fullan,). suggests that internal accountability must precede external accountability if lasting improvement in student achievement is the goal. Richard Elmore (2004) conducted a series of intensive case studies of individual schools—some that failed to improve and some that improved their performance. Relative to the former, schools that failed to improve were not able to achieve instructional coherence, despite being in systems with strong external accountability. A minority of schools did develop internal coherence together and showed progress on student achievement. The main feature of successful schools was that they built a collaborative culture that combined individual responsibility, collective expectations, and corrective action—that is, internal accountability. Transparent data on instructional practices and student achievement were a feature of these cultures. As these cultures developed, they were also able to more effectively engage the external assessment system. Highlighting the fundamental role of internal accountability on school improvement, Elmore (2004) pointed out the following: It seems unlikely to us that schools operating in the default mode—where all questions of accountability related to student learning are essentially questions of individual teacher responsibility—will be capable of responding to strong obtrusive accountability systems in ways that lead to systematic deliberate. improvement of instruction and student learning. The idea that a school will improve, and therefore, the overall performance of its students, implies a capacity for collective deliberation and action that schools in our sample did not exhibit. Where virtually all decisions about accountability are made by individual teachers, based on their individual conceptions of what they and their students can do, it seems unlikely that these decisions will somehow aggregate into overall improvement for the school. (p. 19). Internal accountability is based on the notion that individuals and the group in which they work can transparently hold themselves responsible for their performance. We already know that current external accountability schemes do not work because, at best, they tell us that the system is not performing but does not give a clue about how to fix the situation. As Elmore (2004) observes, if people do not know how to fix the problem and so cannot do so, then the following will occur: Schools will implement the requirements of the external accountability system in pro forma ways without ever internalizing the values of responsibility and efficacy that are the nominal objectives of those systems. (p. 134) Elmore (2004) then concludes this:

investments in internal accountability must logically precede [emphasis added] any expectation that schools will respond productively to external pressure for performance. (p. 134) “Logically precede,” yes, but more to the point of our framework, internal accountability must strategically precede engagement with external accountability. This is why focusing direction, cultivating collaborative cultures, and deepening learning precedes accountability in our Coherence Framework. There are two messages here: One is that policymakers and other leaders are well advised to establish conditions for developing cultures of internal accountability. The second is that there are things other people can do when the hierarchy is not inclined to move. The answer is to “help make it happen in your own situation”—that is, develop collaborative work with your peers and push upward for this work to be supported. The history of the teaching profession is laced with assumptions of and conditions for isolated, individual responsibility. But atomistic responsibility, detached from any group, can never work. In a nutshell, the cultural shift needed is to shift to collaborative cultures that honor and align individual responsibility with collective expectations and actions. Elmore discusses several schools that he and his team studied. Most of them exemplify the individualistic model. Teachers work away on their own and periodically grapple or clash with external accountability requirements. But Elmore also discusses two cases where the schools have developed more or less “collaborative” cultures. The first case is St. Aloysius Elementary School Without exception, teachers described an atmosphere of high expectations. Some stressed a high priority on “reaching every child” and “making sure that no one is left behind” while others referred to a serious and supportive environment where everyone is expected to put forth excellent work. (Elmore, 2004, p. 164) It sounds ideal, but what happens when things don’t go as expected? At another school, Turtle Haven, Elmore (2004) asked teachers, “What happens when teachers do not meet the collective expectations?” He reports that “most teachers believed that a person who did not meet . . . expectations, or conform to a culture created by those expectations would first receive a great deal of support from the principal and other colleagues” (p. 183). If this approach failed to produce results, most Turtle Haven teachers said that the teacher in question would not be happy at the school and eventually would either “weed themselves out [or]. . . if there was a sense in the community that a certain number of children were not able to get the kind of education that we say we’re committed to providing . . . we would have to think whether the somebody belongs here or not” (Elmore, 2004, p. 183). This kind of culture is not foolproof, but we would say it stacks up well against the external accountability thinking that creates demands that go unheeded or can’t be acted on. In the collaborative cultures, the internal accountability system is based on visible expectations combined with consequences for failure to meet set expectations.

Such cultures, says Elmore (2004), are much better equipped to deal with external accountability requirements, adding that a school with a strong internal accountability culture might respond to external assessments in a number of ways, “including accepting and internalizing it; rejecting it and developing defenses against it, or incorporating just those elements of the system that the school or the individuals deem relevant” (p. 145). What is coming through in this discussion is that collaborative cultures with an eye to continuous improvement establish internal processes that allow them to sort out differences and to make effective decisions. At the level of the microdynamics of school improvement, Elmore (2004) draws the same conclusion we do at the system level: investing in the conditions that develop internal accountability is more critical than beefing up external accountability. The Ontario Reform Strategy, which we discussed in previous chapters, offers an illustrative example of the importance of internal accountability preceding external accountability systemwide. The Canadian province of Ontario, with 4,900 schools in 72 districts serving some two million students, started in 2004 to invest in building capacity and internal accountability at the school and district levels. The initial impulse for the reform came from leadership at the top of the education system—Dalton McGuinty, the premier of the province at the time—through the establishment of a small number of ambitious goals related to improvements in literacy, numeracy, and high school retention. However, the major investments focused on strengthening the collective capacity of teachers, school principals, and district leaders to create the conditions for improved instructional practice and student achievement (Glaze, Mattingley, & Andrews, 2013). There was little overt external accountability in the early stages of the Ontario Reform Strategy. External accountability measures were gradually introduced in the form of assessment results in grades 3 and 6 in literacy and numeracy, and in high school, retention numbers, transparency of data, and a school turnaround support-focused policy called Ontario Focused Intervention Program (OFIP) for schools that were underperforming. This system has yielded positive and measurable results in literacy that has improved dramatically across the 4,000 elementary schools and in high school graduation rates that have climbed from 68 percent to 84 percent. across the 900 high schools. The number of OFIP schools, formerly at over 800, has been reduced to 69 schools even after the criteria to identify a school as in need of intervention had widened to include many more schools (Glaze et al., 2013; Mourshed, Chijioke, & Barber, 2010). An evaluation of the reform strategy in 10 of Ontario’s 72 school districts that concentrated particularly on the special education aspects of the reform pointed to a significant narrowing of the achievement gap in writing scores for students with learning disabilities (Hargreaves & Braun, 2012). Concerns were expressed among teachers who were surveyed about some of the deleterious consequences of standardized testing in grades 3 and 6— that the tests came at the end of the year at a point that was too late to serve a diagnostic function, that they were not sufficiently differentiated in order to match differentiated instructional strategies, and that principals in some

The most intriguing finding though was that special education resource teachers, whose role was moving increasingly to providing in-class support, welcomed the presence of transparent objective data. They saw it as a way of drawing the attention of regular classroom teachers to the fact and the finding that students with learning disabilities could, with the right support, register valid and viable gains in measurable student achievement. Together, these findings point to the need to review the nature and form of high-stakes assessments—more differentiated, more just-in-time, and more directed at the needs of all students, perhaps—but also to the value of having transparent data that concentrate everyone’s attention on supporting all students’ success along with diagnostic data and collaborative professional responsibility for all students’ learning, development, and success A similar approach to whole system improvement can be found in U.S. districts that have been awarded the prestigious Broad Prize for Urban Education, granted to urban school districts that demonstrate the great- est overall performance and improvement while reducing achievement gaps based on race, ethnicity, and income. In her in-depth study of five such districts, Zavadsky (2009) finds that, while diverse in context and strategies, these districts have addressed the challenge of improving student performance systemwide following remarkably similar approaches: investing in, growing, and circulating the professional capital of schools (what they term building capacity) to improve instructional practice by fostering teacher collaboration and collective accountability. These successful schools set high instructional targets, attracting and developing talent, aligning resources to key improvement priorities, constantly monitoring progress, and providing timely targeted support when needed.

The solid and mounting evidence on the fundamental impact of internal accountability on the effectiveness and improvement of schools and school systems contrasts sharply with the scarce or null evidence that external accountability, by itself or as the prime driver, can bring about lasting and sustained improvements in student and school performance. There is, indeed, a growing realization that external accountability is not a capable driver of school and system effectiveness. At best, external accountability does not get its intended results. At worst, it produces undesirable and sometimes unconscionable consequences, such as the cheating scandal in Atlanta (Hill, 2015). We frequently ask successful practitioners that we work with how they themselves handle the “accountability dilemma” (direct accountability doesn’t work; indirect may be too soft). What follows are a few responses that we have personally received to this question: What is effective accountability? Not surprisingly, these views are entirely consistent with Elmore (2004): Accountability is now primarily described as an accountability for student learning. It is less about some test result and more about accepting ownership of the moral imperative of having every student learn. Teachers talk about “monitoring” differently. As they engage in greater sharing of the work, they talk about being accountable as people in the school community know what they are doing and looking to see what is changing for students as a result. And as they continue to deprivatize teaching, they talk about their principal and peers coming into their classrooms and expecting to see the work [of agreed-upon practices] reflected in their teaching, their classroom walls, and student work. (Anonymous, personal communication, November 2014).

Teachers and administrators talk about accountability by deprive- sizing their practice. If everyone knows what the other teacher or administrator is working on and how they are working on it with students, it becomes a lot easier to talk about accountability. When everyone has an understanding of accountability, creating clear goals and steps to reach those goals, it makes it easier for every- one to talk and work in accountable environments. (Elementary principal, personal communication, November 2014).

I spoke with my staff about accountability versus responsibility in brainstorming, about what is our purpose and who is responsible for what . . . being explicit and letting teachers collectively determine what our responsibilities are. (Secondary school principal, personal communication, November 2014) We are moving to define accountability as responsibility. My district has been engaged in some important work that speaks to intrinsic motivation, efficacy, perseverance, etc., and accountability is seen as doing what is best for students . . . working together to tackle any challenge and being motivated by our commitment as opposed to some external direction. (Superintendent, personal communication, November 2014).

When you blow down the doors and walls, you can’t help but be evermore accountable. (Superintendent, personal communication, November 2014) I do believe that a lot of work remains to be done on building common understanding on the notion of accountability. Many people still believe that someone above them in the hierarchy is accountable. Very few take personal accountability for student learning and achievement. There are still those who blame parents and students’ background for achievement. (Consultant, personal communication, November 2014) In one school, the talk about accountability was pervasive as the school became designated as underperforming. The morale of the school went down significantly, and the tension was omnipresent at every meeting. The team switched the conversation to motivation, innovation, and teamwork and the culture changed. The school is energized and the test scores went up in one year. The team is now committed to results and continuous improvement. (Consultant, personal communication, November 2014) In short, internal accountability is far more effective than external accountability. The bottom line is that it produces forceful accountability in a way that no hierarchy can possibly match. We have shown this to be the case for teachers, and we can make a parallel argument for students. If we want students to be more accountable, we need to change instruction toward methods that increase individual students’ responsibility for assessing their methods that increase individual students’ responsibility for assessing their own learning and for students to work in peer groups to assess and provide feedback to each other under the guidance of the teacher. We still need external accountability, and we can now position it more effectively.

External Accountability

External accountability concerns any entity that has authority over you. Its presence is still essential, but we need to reposition external accountability so that it becomes more influential in the performance of individuals, groups, and the system as a whole. We first take the perspective of external authorities and then flip back to local entities. External Authorities The first thing to note is that if the external body invests in building widespread internal accountability they will be furthering their own goals of greater organization or system accountability. The more that internal accountability thrives, the greater the responsiveness to external requirements and the less the externals have to do. When this happens, the center has less need to resort to carrots and sticks to incite the system to act responsibly. Dislodging top-down accountability from its increasingly miscast role has turned out to be exceedingly difficult. People at the top do not like to give up control. They cling to it despite obvious evidence that it does not work. And attacks on the inadequacy of top-down accountability have failed because they have only focused on the “from” side of freedom. Critics seem to be saying that accountability requirements do not work, so remove them. That is not the complete solution because it takes us back to nothing. The answer is found in our argument in this chapter—rely on developing the conditions for internal accountability and reinforce them with certain aspects of external accountability. In particular, central authorities should focus their efforts on two interrelated activities:

1. Investing in internal accountability

2. Projecting and protecting the system

By the first, I mean investing in the conditions that cause internal accountability to get stronger. The beauty of this approach, as we have seen, is that people throughout the system start doing the work of accountability. Though indirect, this form of accountability is more explicit, more present, and, of course, more effective. We have already suggested its components:

• A small number of ambitious goals, processes that foster shared goals (and even targets if jointly shaped)

• Good data that are used primarily for developmental purposes

• Implementation strategies that are transparent, whereby people and organizations are grouped to learn from each other (using the group to change the group)

• Examination of progress in order to problem solve for greater performance

The center needs to invest in these very conditions that result in greater focus, capacity, and commitment at the level of day-to-day practice. They invest, in other words, in establishing conditions for greater local responsibility. In this process, the center will still want goals, standards, assessment, proof of implementation, and evidence of progress. This means investment in resources and mechanisms of internal accountability that people can use to collaborate within their units and across them.

With strong internal accountability as the context, the external accountability role of the system includes the following:

1. Establishing and promoting professional standards and practices, including performance appraisal, undertaken by professionally respected peers and leaders in teams wherever possible and developing the expertise of teachers and teacher-leaders so that they can undertake these responsibilities. With the robust judgments of respected leaders and peers, then getting rid of teachers and administrators who should not be in the profession will become a transparent collective responsibility.

2. Ongoing monitoring of the performance of the system, including direct intervention with schools and districts in cases of persistent underperformance.

3. Insisting on reciprocal accountability that manages “up” as well as down so that systems are held accountable for providing the resources and supports that are essential in enabling schools and teachers to fulfill expectations (e.g., “failing” schools should not be closed when they have been insufficiently resourced, or individual teachers should be evaluated in the context of whether they have been forced into different grade assignments every year or have experienced constant leadership instability).

tors of student performance and well-being. These would include measures of social capital in the teaching profession such as extent of collaboration and levels of collegial trust. Outcome measures for students should also, as previously stated, include multiple measures including well-being, students’ sense of control over their own destiny (locus of control), levels of engagement in learning, and so forth. The Perspective of locals

4. Adoptingandapplyingindicatorsoforganizationalhealthasacontext for individual teacher and leader performance, such as staff retention rates, leadership turnover rates, teacher absenteeism levels, numbers of crisis-related incidents, and so on, in addition to outcome indicators of student performance and well-being. These would include measures of social capital in the teaching profession such as extent of collaboration and levels of collegial trust. Outcome measures for students should also, as previously stated, include multiple measures including well-being, students’ sense of control over their destiny (locus of control), levels of engagement in learning, and so forth.

The Perspective of locals

We have drawn on numerous relatively successful examples in this book. They all established strong degrees of internal accountability (people being self and group responsible) that served them well in the external ).

accountability arena. Such systems strengthened accountability by increasing focus, connecting dots and otherwise working on coherence, building capacity (so people could perform more efficaciously), being transparent about progress and practices, and engaging the external accountability system. As districts increase their capacity, they become stronger in the face of ill-advised external accountability demands as the following two extended examples reveal from Laura Schwalm, former superintendent of Garden Grove).

Example One: garden grove Handles External Pressure In the words of Laura Schwalm: Shortly after we completed our audit and instituted a district-wide mandate and system to place students in college prep (a–g) courses, Ed Trust and several other advocacy groups, with support from the California Department of Education (CDE), began “calling out” the low college readiness statistics in large urban districts in California. Every large urban district, including Garden Grove, was called out (rightfully so) with one exception of a district in the north, which was held as a model solution because they had made the age requirement mandatory for every student and claiming they had eliminated all other courses with absolutely no effect on their graduation rate. Based on this example, the advocacy groups started a very public campaign and got a majority of school boards, including LAUSD, to adopt the policies of this northern district with the pledge that they would achieve 100 percent a–g achievement with no increase in dropout rate within four to five years. When Garden Grove refused to comply (Long Beach did as well), we were more strongly targeted and pressured (the approach we had adopted was to not eliminate all support courses that were not college prep but rather to eliminate a few and to align the rest in a way to provide an “on ramp” to college prep courses while at the same time using individual student-by-student achievement data, rather than the former practice of “teacher recommendation” for placement in college prep courses) (one of the shameful things our audit revealed, which did not surprise me, was that if you were an Asian student with mean achievement on the California Standards Tests, you had about a 95 percent chance of being “recommended for placement in a-g courses”—conversely, if you were a Latino male with the exact same scores, you had less than 30 percent chance of being recommended for placement in these courses). As the pressure continued to adopt a policy of mandating an exclusive a–g curriculum, I met with a few of the key advocates and explained that while we shared the same goal of increasing our unacceptably low a–g completion rate, we strongly felt the approach they were suggesting was ill advised. Putting students in a course for which they were absolutely not prepared, based on very objective data, and then expecting them to pass the course with a grade of C or better was unfair to both students and teachers. They kept focusing on the district up north, which led me to point out to them that the data from that district did not support what they were claiming. If their approach was truly working, then their achievement scores, as measured by the state, should be outperforming ours, and in fact, they fell far short of ours, for all subgroups. Additionally, a neighboring district that had adopted the same policy now claimed a 90 percent a–g completion rate, yet 65 percent of their high school students scored below the mean on the state standards test. It clearly pointed out that all was not as it looked on the surface, and while I had no desire to criticize another district’s approach, I was not about to follow it. That caused the advocates to pause and finally to leave us alone. Our rate, both in terms of a–g completion and student achievement data by subgroup, continued to climb. Within a few years, we surpassed all the others, and over time, the policy the CDE and advocates had pushed into districts quietly vanished.

Example Two: garden grove Deals With the bureaucracy Again in Schwalm’s words:

Another example occurred during one of the CDE’s three-year systemwide compliance reviews. While I accepted the state’s responsibility to oversee that we were not using specially designated funding for inappropriate uses, as well as to assure we were following laws around equity and access for all students, the process they had was unnecessarily burdensome, requiring us to dedicate significant staff to collecting, cataloging, and preparing documentation that filled dozens and dozens of boxes. When the state team came—usually about 10 to 12 people, each looking at different programs with one person loosely designated as team lead—the expectation was that you treat them like royalty and that they had enormous authority. My view was somewhat different. I respected that they had a job to do, but just because they did not like the way we displayed something did not mean we needed to do it differently or because they would have used another approach—our approach if appropriately supported with data—was not out of bounds. At one of the first reviews early on in my superintendency, we drew a particularly weak but officious team with a very weak lead. They came up with some particularly lame findings (i.e., one team member commended us on how we used data to identify areas of focus for targeted groups of students, while another team member marked us as noncompliant in this area because we did not put it on a form that she had developed—and other equally ludicrous examples). At the end of the process, the superintendent was required to sign an agreement validating the team’s findings as well as a plan and timeline to bring things into “compliance.” I very professionally told them that I did not agree with their findings and thus could not sign either document—I was not going to pretend to fix something that I had no intention of doing because there was nothing wrong with it in the first place. What I did do was sign a document, which we drafted, acknowledging that the team had, in fact, been there and that we agreed to a couple of specific areas where we needed to and would make some changes, but I did not agree with the majority of the report and would not agree to take any action other than what was previously specified. This seemed pretty fair to me, but apparently it shocked them and the system, which was the beginning of my unpopularity with many in CDE. Probably this was made worse when the story got out (not by my telling), and other superintendents realized that they could do the same thing (although I advised those who contacted me—and a number did—that their life would not be particularly easy for a while and also that they should have the data and results to back their stand) (L. Schwalm, personal communication, 2014). You can see why in another book (where I cited an even more egregious example of defiance), I referred to Laura as a “rebel with a cause” (Fullan, 2015). There are two lessons here with what I have called both the freedom-from problem and the freedom-to problem. You need to attend to both. The freedom-from problem is what Laura did—refusing to comply with ridiculous demands. But she was backed up by her freedom- to actions in which she built a culture of coherence, capacity, and internal accountability. If you do the latter, you are in good shape to contend with the external accountability system, including acting on external performance data that do show that you need to improve,

In California as a whole, they currently face the freedom-to problem. The wrong drivers are on the way out the door. Jerry Brown, the governor, has suspended all statewide student tests for at least two years on the grounds that it is better to have no tests than to have the wrong test. So far so good, but getting rid of bad tests is not enough for securing accountability. New tests—Smarter Balanced Assessment Curriculum (SBAC)— are being piloted relative to CCSS. Districts would be well advised to use our Coherence Framework to build their focused accountability. They will then perform better and be in a better position to secure their own accountability as they relate to the ups and downs of external accountability. External accountability as wrong as it can get sometimes is a phenomenon that keeps you honest. Leaders need to be skilled at both internal and external accountability and their interrelationship.

Turn in your highest-quality paper
Get a qualified writer to help you with

“ discussion4-2 ”

Get high-quality paper

NEW! AI matching with writer

Hire a Writer

Client Reviews

4.9

Sitejabber

4.6

Trustpilot

4.8

Our Guarantees

100% Confidentiality

Information about customers is confidential and never disclosed to third parties.

Original Writing

We complete all papers from scratch. You can get a plagiarism report.

Timely Delivery

No missed deadlines – 97% of assignments are completed in time.

Money Back

If you're confident that a writer didn't follow your order details, ask for a refund.

New to Your Trusted Assignment Help Service? Sign up & Save

Calculate the price of your order

Type of paper needed:

Pages:

You will get a personal manager and a discount.

Academic level:

We'll send you the first draft for approval by at

Total price:

$0.00

Power up Your Academic Success with the
Team of Professionals. We’ve Got Your Back.

Power up Your Study Success with Experts We’ve Got Your Back.

Order Now Order Now