Question answer from the book

Textbook Exercises

Chapter 5 Hashed Data Structures

1, 5, 9, 10, 11 (a)(b), 25, 26, 28

Chapter 6 Recursion

2 (a,b,c,d), 11(a, b, c, d)

2

DATA STRUCTURES
AND ALGORITHMS USING
JAVA ™
William McAllister
St. Joseph’s College
JONES AND BARTLETT PUBLISHERS
Sudbury, Massachusetts
BOSTON TORONTO LONDON SINGAPORE
3

World Headquarters
Jones and Bartlett Publishers
40 Tall Pine Drive
Sudbury, MA 01776
978-443-5000
info@jbpub.com
www.jbpub.com
Jones and Bartlett Publishers
Canada
6339 Ormindale Way
Mississauga, Ontario L5V 1J2
Canada
Jones and Bartlett Publishers
International
Barb House, Barb Mews
London W6 7PA
United Kingdom
Jones and Bartlett’s books and products are available through most bookstores and online
booksellers. To contact Jones and Bartlett Publishers directly, call 800-832-0034, fax 978-
443-8000, or visit our website, www.jbpub.com .
Substantial discounts on bulk quantities of Jones and Bartlett’s publications are available to
corporations, professional associations, and other qualified organizations. For details and
specific discount information, contact the special sales department at Jones and Bartlett via
the above contact information or send an email to specialsales@jbpub.com .
Copyright © 2009 by Jones and Bartlett Publishers, LLC
All rights reserved. No part of the material protected by this copyright may be reproduced or
utilized in any form, electronic or mechanical, including photocopying, recording, or by any
information storage and retrieval system, without written permission from the copyright
owner.
Production Credits
Publisher: David Pallai
Acquisitions Editor: Timothy Anderson
Editorial Assistant:Melissa Potter
Production Director: Amy Rose
Production Editor: Katherine Macdonald
Senior Marketing Manager: Andrea DeFronzo
V.P., Manufacturing and Inventory Control: Therese Connell
Composition: Northeast Compositors, Inc. and International Typesetting and Composition
Cover Design: Kristin E. Parker
Cover Image: © Vitezslav Halamka/ShutterStock, Inc.
Printing and Binding:Malloy, Inc.
Cover Printing:Malloy, Inc.
Library of Congress Cataloging-in-Publication Data
McAllister, William (William James)
Data structures and algorithms using Java / William McAllister.—
4

mailto:info@jbpub.com

http://www.jbpub.com

http://www.jbpub.com

mailto:specialsales@jbpub.com

1st ed.
p. cm.
Includes index.
ISBN-13: 978-0-7637-5756-4 (pbk.)
ISBN-10: 0-7637-5756-X (pbk.)
1.Computer algorithms. 2. Data structures (Computer science) 3. Java (Computer
program language) I. Title.
QA76.9.A43M373 2008
005.13’3–dc22
2008013120
6048
Printed in the United States of America
12 11 10 09 08 10 9 8 7 6 5 4 3 2 1
5

Dedication
To my best friend and wife, Gretchen (a.k.a Maggie),
who supplied endless encouragement and hundreds of commas
6

Contents
Preface
Chapter 1 Overview and Java Review
1.1 Data Structures
1.1.1 What Is Data?
1.1.2 What Is a Data Structure?
1.2 Selecting a Data Structure
1.2.1 The Data Structure’s Impact on Performance
1.2.2 Determining the Performance of a Data Structure
1.2.3 The Trade-Off Process
1.3 Fundamental Concepts
1.3.1 Terminology
1.3.2 Access Modes
1.3.3 Linear Lists
1.3.4 Data Structure Operations
1.3.5 Implementing a Programmer-Defined Data Structure
1.3.6 Procedural Abstractions and Abstract Data Types (ADTs)
1.3.7 Encapsulation
1.4 Calculating Speed (Time Complexity)
1.4.1 Big-O Analysis (O Standing for Order of Magnitude)
1.4.2 Algorithm Speed
1.4.3 Relative Speed of Algorithms
1.4.4 Absolute Speed of an Algorithm
1.4.5 Data Structure Speed
1.5 Calculating Memory Overhead (Space Complexity)
1.6 Java Review
1.6.1 Arrays of Primitive Variables
1.6.2 Definition of a Class
1.6.3 Declaration of an Object
1.6.4 Accessing Objects
1.6.5 Standard Method Name Prefixes
1.6.6 Shallow and Deep Copies
1.6.7 Declaration of an Array of Objects
1.6.8 Objects that Contain Objects as Data Members
1.6.9 Classes that Extend Classes, Inheritance
1.6.10 Parent and Child References
1.6.11 Generic Types
Knowledge Exercises
7

Programming Exercises
Chapter 2 Array-Based Structures
2.1 The Built-in Structure Array
2.1.1 Multidimensional Arrays
2.2 Programmer-Defined Array Structures
2.2.1 Unsorted Array
2.2.2 Sorted Array
2.2.3 Unsorted-Optimized Array
2.2.4 Error Checking
2.3 Implementation of the Unsorted-Optimized Array Structure
2.3.1 Baseline Implementation
2.3.2 Utility Methods
2.4 Expandable Array-Based Structures
2.5 Generic Data Structures
2.5.1 Design Considerations
2.5.2 Generic Implementation of the Unsorted-Optimized Array
2.5.3 Client-Side Use of Generic Structures
2.5.4 Heterogeneous Generic Data Structures
2.6 Java’s ArrayList Class
Knowledge Exercises
Programming Exercises
Chapter 3 Restricted Structures
3.1 Restricted Structures
3.2 Stack
3.2.1 Stack Operations, Terminology, and Error Conditions
3.2.2 Classical Model of a Stack
3.2.3 A Stack Application: Evaluation of Arithmetic Expressions
3.2.4 Expanded Model of a Stack
3.3 Queue
3.3.1 Queue Operations, Terminology, and Error Conditions
3.3.2 Classical Model of a Queue
3.3.3 Queue Applications
3.3.4 Expanded Model of a Queue
3.4 Generic Implementation of the Classic Stack, a Methodized
Approach
3.4.1 Generic Conversion Methodology
3.5 Priority Queues
3.6 Java’s Stack Class
8

Knowledge Exercises
Programming Exercises
Chapter 4 Linked Lists and Iterators
4.1 Noncontiguous Structures
4.2 Linked Lists
4.3 Singly Linked Lists
4.3.1 Basic Operation Algorithms
4.3.2 Implementation
4.3.3 Performance of the Singly Linked List
4.3.4 A Stack Implemented as a Singly Linked List
4.4 Other Types of Linked Lists
4.4.1 Circular Singly Linked List
4.4.2 Double-Ended Singly Linked List
4.4.3 Sorted Singly Linked List
4.4.4 Doubly Linked List
4.4.5 Multilinked List
4.5 Iterators
4.5.1 Implementation of an Iterator
4.5.2 Multiple Iterators
4.6 Java’s LinkedList Class and ListIterator Interface
Knowledge Exercises
Programming Exercises
Chapter 5 Hashed Data Structures
5.1 Hashed Data Structures
5.2 Hashing Access Algorithms
5.2.1 A Hashing Example
5.3 Perfect Hashed Data Structures
5.3.1 Direct Hashed Structure
5.4 Nonperfect Hashed Structures
5.4.1 Primary Storage Area Size
5.4.2 Preprocessing Algorithms
5.4.3 Hashing Functions
5.4.4 Collision Algorithms
5.4.5 The Linear Quotient (LQHashed) Data Structure
Implementation
5.4.6 Dynamic Hashed Structures
Knowledge Exercises
Programming Exercises
9

Chapter 6 Recursion
6.1 What Is Recursion?
6.2 Understanding Recursive Algorithms
6.2.1 n Factorial
6.2.2 The Code of a Recursive Algorithm
6.2.3 Tracing a Recursive Method’s Execution Path
6.3 Formulating a Recursive Algorithm
6.3.1 Definitions
6.3.2 Methodology
6.3.3 Practice Problems
6.4 Problems with Recursion
6.4.1 Dynamic Programming Applied to Recursion
6.5 Backtracking, an Application of Recursion
6.5.1 A Generalized Backtracking Algorithm
6.5.2 Algorithm Adaptation Methodology
Knowledge Exercises
Programming Exercises
Chapter 7 Trees
7.1 Trees
7.1.1 Graphics and Terminology of Trees
7.2 Binary Trees
7.2.1 Terminology
7.2.2 Mathematics
7.3 Binary Search Trees
7.3.1 Basic Operation Algorithms
7.3.2 Performance
7.3.3 Implementation
7.3.4 Standard Tree Traversals
7.3.5 Balanced Search Trees
7.3.6 Array Implementation of a Binary Search Tree
7.3.7 Performance
7.3.8 Java’s TreeMap Data Structure
Knowledge Exercises
Programming Exercises
Chapter 8 Sorting
8.1 Sorting
8.2 Sorting Algorithm Speed
8.2.1 Minimum Sort Effort
10

8.2.2 An Implementation Issue Affecting Algorithm Speed
8.3 Sorting Algorithms
8.3.1 The Binary Tree Sort
8.3.2 The Bubble Sort
8.3.3 The Heap Sort
8.3.4 The Merge Sort
8.3.5 Quicksort
Knowledge Exercises
Programming Exercises
Chapter 9 Graphs
9.1 Introduction
9.1.1 Graphics and Terminology of Graphs
9.2 Representing Graphs
9.2.1 Representing Vertices
9.2.2 Representing Edges
9.3 Operations Performed on Graphs
9.4 Implementing Graphs in the Vertex Number Mode
9.5 Traversing Graphs
9.5.1 Depth-First Traversal
9.5.2 Breadth-First Traversal
9.6 Connectivity and Paths
9.6.1 Connectivity of Undirected Graphs
9.6.2 Connectivity of Directed Graphs
9.6.3 Spanning Trees
9.6.4 Shortest Paths
Knowledge Exercises
Programming Exercises
Appendices
Appendix A ASCII Table
Appendix B Derivation of the Average Search Length of a
Nondirect Hashed Data Structure
Appendix C Proof That If an Integer, P, Is Not Evenly
Divisible by an Integer Less Than the Square Root of P, It Is a
Prime Number
Appendix D Calculations to Show That (n + 1) (log2(n + 1) −
2) Is the Minimum Sort Effort for the Binary Tree Sort
Glossary
11

Index
12

Preface
Introduction
Data Structures and Algorithms Using Java is an undergraduate-level
textbook for a computer science curriculum. It covers the entire syllabus of
the Association of Computing Machinery standard curriculum courses
CS103i and CS103o, “Data Structures and Algorithms” (CS2001
evolutions of the traditional CS2 course). As such, it is intended to be used
as the primary text for these courses. The book is built upon my academic
and industry experience, combining the pedagogical techniques I’ve
developed and refined during 29 years of teaching with my practical
knowledge of the computer field acquired during 28 years in the industry.
The resulting integration of theory and application will help students at all
levels to understand core concepts that will enhance their success in later
courses, and ultimately, in their careers.
Why Another Data Structures Book
My primary reason for writing this book was to produce a text that
was more readable, engaging, and instructive than those currently in print
without compromising the scope of the ACM CS103 course material. I
wanted the text to engage students outside the classroom in the process of
investigative discovery. The motivation for this was based partially on the
findings from two National Science Foundation grants my colleagues and I
received, whose purpose was to investigate ways of attracting students into
technical areas of national need (including computer professionals) and
retaining them through graduation. “Data Structures and Algorithms,” a
sophomore-level course, is usually the “make-or-break” course for
computer science majors. As a co-principal investigator on both of these
grants, I realized that a highly accessible Data Structures and Algorithms
text, with improved pedagogy, would help retain students majoring in
computer science.
Advantages of This Text
I’ve endeavored to present the minimal amount of Java code
necessary to illustrate the implementation of the learned concepts. This
minimal code set leaves plenty of room for meaningful programming
assignments, while being thorough enough that a computer professional
13

could use the text as a source code reference book.
The book is more comprehensive in some topic areas than current
books in print, which makes it more appealing to highly capable students.
The clearer language, simple examples, and abundance of instructional
figures, including nearly 300 illustrations and more than 50 tables, make it
more accessible to the majority of students who might otherwise struggle
to grasp these and other, more challenging, concepts.
The text is aimed at both the computer scientist who implements,
refines, or improves the classic data structures and the software engineer
who selects and integrates library implementations of these data structures
into particular applications. From the computer scientist’s viewpoint, the
design of each of the classic data structures is discussed and their
algorithms’ pseudocode developed and implemented. In addition, the
characteristics of each structure that give rise to its speed and space
complexity advantages (or disadvantages) are examined. From the
software engineer’s viewpoint, the text presents the relative merits of the
classic data structures to the extent that an optimum choice could be made
for a particular application. This is accomplished by deriving the speed and
memory requirements of each of the classic data structures in a quantified
manner, with the performance of each structure compared in a summary
table at the end of each chapter. In addition, multivariable graphs are
presented to demonstrate how to locate optimum design points from a
memory requirement point of view. Finally, the use of the Java API
implementation of the data structures is demonstrated.
Other significant advantages of this book are: a methodized approach
to recursive algorithm development aimed at teaching students to “think”
recursively; an introduction to the techniques and benefits of dynamic
programming; and line-by-line discussions of the significant portions of
the implementation code, which include the techniques for encapsulating
data within a structure and the techniques for implementing generic
structures.
Programming Language Background
The implementation language used in the text is Java, but since the
first chapter includes a review of objects and the Java constructs and
syntax necessary to understand the book’s code examples, students with a
programming background in other high-level languages should find the
book approachable. The Java review guides students through an
14

understanding of the fundamental concepts essential to data structures that
are particular to object-oriented languages, such as Java’s use of reference
variables, its memory allocation model, arrays of objects, shallow and
deep object copies, parent-to-child references, composition
(containership), and generic features of Java 1.4.
Organization
The text employs a unique, student-friendly pedagogical approach
and organizational structure. The chapters that present the classic data
structures (Chapters 2 , 3 , 4 , 5 , 7 , and 9 ) use a common template, rooted
in the object paradigm, as the organizational basis of each chapter. The use
of this template, embellished with topics particular to each structure,
permits the student to “anticipate” the material presented in each chapter,
which in turn makes the text more readable. The template begins by stating
the shortcomings of the structures studied to that point and identifies the
ways in which the new structure addresses some of these shortcomings.
Then the structure is visually introduced, and its initialization and
operation algorithms are developed in pseudocode. These algorithms are
then fully implemented in Java as an encapsulated homogenous structure,
and the structure’s use is then demonstrated in a telephone information
request application. Next, the performance of the structure is quantified,
discussed, and compared to the performance of the previously studied
structures. Finally, the use of the Java API implementation of the structure
is discussed and demonstrated.
Chapter 1 is an introduction to the basic concepts of data structures
and a review of Java. The first part of the chapter introduces the student to
the concept of a data structure and its impact on the performance of a
software product. The terminology and fundamental concepts common to
all data structures are discussed, including encapsulation, abstraction, and
algorithm performance. This section concludes with a discussion of speed
complexity, space complexity, and Big-O analysis. The second part of the
chapter is a review of the Java concepts and constructs necessary to
understanding the implementations presented in the text. The topics
include array declarations, class definitions and objects, containership,
inheritance, shallow and deep copies, and the use of the generic features of
Java 1.4 to write generic methods and classes.
Chapter 2 discusses the implementation of the built-in structure array
and presents three programmer-defined, array-based structures. A major
15

pedagogical advantage of this chapter is that it utilizes the simplicity of
arrays and the students’ familiarity with them to facilitate a discussion of
the concepts common to all data structures. In the first part of the chapter,
the student is encouraged to view arrays from a new perspective: a data
structures perspective. Once viewed from that perspective, they can be
used as a case study to illustrate the techniques of memory modeling and
performance trade-offs that were employed in their design. The second
part of the chapter begins the book’s study of programmer-defined data
structures by presenting three data structures built upon arrays. The
simplicity of these structures allows the student to focus on learning the
concepts common to all data structures. Then, having gained an early
understanding of these concepts, students are free to concentrate on the
more complicated conceptual details of the structures presented in
subsequent chapters. The chapter also presents techniques for expanding
arrays at run-time and for converting the array-based implementation
developed in the chapter to a fully generic data structure using the generic
features of Java 1.4. It concludes with a discussion of the use of the
ArrayList class included in the Java API.
Chapter 3 presents the restricted structures Stack and Queue. The first
part of the chapter discusses what they have in common with other data
structures and encourages the student to consider what about them is
restricted. Thus the student understands that Stack and Queue are part of
the broader data structure landscape, rather than two isolated computer
science topics. The second part of the chapter presents the details of Stack
and Queue, including a full implementation of each structure. The Queue
implementation is used as the basis of presenting a methodized approach
to converting a data structure class to a fully generic class using the
generic features of Java 1.4. The chapter concludes with a discussion of
priority queues and the use of the Stack class included in the Java API.
Chapter 4 introduces the student to linked lists, their use in the
implementation of a stack, and iterators. The first part of the chapter
discusses singly linked lists and their implementation. It also includes a
discussion of several variations of a singly linked list, doubly linked lists,
and multi-linked lists. The second part of the chapter introduces the
concept of an iterator, a discussion of the advantages of its use, and a full
implementation of an iterator class. As part of this implementation, the
concept of an inner class is introduced, which includes a discussion of why
classes are coded as inner classes. The chapter concludes with a discussion
of the use of the LinkedList and ListIterator classes included in the Java
16

API.
Chapter 5 deals with the topic of hashing, but it significantly extends
the traditional treatment of this topic. It treats the pervasive data structures
that use hashing algorithms as a separate data structure category, Hashed
Structures. The first part of the chapter illustrates the advantages of
hashing via an example, and then it presents the algorithms of a data
structure based on a perfect hashing function. The second part of the
chapter discusses nonperfect hashing functions, collision algorithms and
their performance, and string preprocessing algorithms used by data
structures that utilize nonperfect hashing functions. Included is a
discussion of optimum hash table size and a 4k + 3 prime number
generator. All these concepts are then applied in the implementation of a
data structure based on the division hashing function and high performance
collision algorithm. The chapter concludes with a discussion of expandable
hashed structure and the use of the HashTable class included in Java’s API.
Chapter 6 presents the topic of recursion. The first part of the chapter
introduces recursion and provides the student with an understanding of
how a recursive method works. The student is taught to think recursively
in the second part of the chapter via a methodized approach to recursive
algorithm development. Several examples are presented whose recursive
solutions can be discovered by directly applying the methodology. Once
comfortable with the methodology, the student is presented with other
examples that require modification in order to discover their recursive
solutions. The chapter concludes with a discussion of the speed and space
complexity problems of recursive algorithms and the application of
dynamic programming to alleviate these problems. Other applications of
recursion appear in the chapters on trees, sorting, and graphs.
Chapter 7 introduces the student to the topic of trees. After a
discussion of the terminology of trees in general and binary trees in
particular, including the mathematics of binary trees, the algorithms of the
binary search tree are introduced. All three cases of the delete algorithm
are presented. The structure is implemented using a linked
implementation, and a subsequent analysis of its performance leads to a
discussion of self-balancing trees, specifically the AVL and re-black trees.
The second part of the chapter presents the array-based implementation of
a binary tree, which includes the pseudocode of its Insert and Fetch
algorithms. The poor performance of the Delete algorithm under this
implementation is discussed along with a suggested remedy. The chapter
concludes with a discussion of the use of the red-black tree
17

implementation TreeMap included in Java’s API.
Chapter 8 presents the topic of sorting. It begins with a discussion of
the motivation for sorting and demonstrates the need for efficient sorting
algorithms. Included is a discussion of the parameters and techniques used
to determine the speed of these algorithms. The first algorithm presented is
the binary tree sort, which uses the student’s knowledge of search trees as a
stepping stone to the analysis of sorting algorithms. The remainder of the
chapter presents four other classic sorting algorithms, and it implements
two of the better performers—the merge and quick sort—recursively. As
each algorithm is discussed, its speed and space complexity are determined
and summarized in a table for comparative purposes.
Chapter 9 presents the topic of graphs. In the first part of the chapter,
they are presented as part of the data structure landscape. After a
discussion of the terminology and the mathematics of graphs, the
techniques for representing them are introduced and compared, and the
algorithms used to operate on them are developed and implemented. These
algorithms included both depth-first and breadth-first traversals. The first
part of the chapter concludes with an implementation of a graph structure
that includes a depth-first traversal. The second part of the chapter
discusses the issues of connectivity and paths along with the associated
algorithms. These algorithms include the determination of connectivity,
spanning trees, and shortest paths based on Warshall’s, Dijkstra’s, and
Floyd’s algorithms.
Supplemental Materials
The pedagogical features of the text are significantly enhanced by
animation courseware that can be run on any Java-enabled browser. These
animations demonstrate the functionality of the algorithms associated with
each data structure presented in the text by presenting the changes that take
place in main memory as the algorithms execute. Other ancillary materials
provided with the text are PowerPoint lecture outlines, the text’s source
code, spreadsheets that can be used to perform design performance trade-
offs between various data structures, and solutions to selected exercises.
These supplements can be found at Jones and Bartlett’s catalog page,
http://www.jbpub.com/catalog/9780763757564 .
Acknowledgments
I would like to thank the team at Jones and Bartlett that aided me in
18

http://www.jbpub.com/catalog/9780763757564

the preparation of this book and guided me through the publication
process: my acquisitions editor, Tim Anderson; his editorial assistant,
Melissa Potter; Melissa Elmore, Associate Production Editor; Kat
Macdonald, Production Editor; my copy editor, Diana Coe; and the entire
production staff.
I’d also like to thank the reviewers for taking the time to review the
manuscript and providing many valuable suggestions, most of which have
been incorporated into the textbook: Tom Murphy, Contra Costa College,
and Victor Shtern, Boston University.
Most importantly, I’d like to thank my family, friends, and colleagues
who not only offered encouragement but also endless patience and
understanding during the times when I was too busy to be me. Finally, I’d
like to thank St. Joseph’s College for the sabbatical that allowed me to get
this textbook off the ground.
To the Students
It’s my hope that you enjoy reading this book, and that you will
experience the joy of accomplishment as you learn material that is
fundamental to the remarkable discipline in which we have chosen to
immerse ourselves. The pedagogy on which it is based evolved through the
classroom experiences I’ve had with hundreds of students much like you.
By the time you finish this book, you will be way ahead of my son-in-law
(a lawyer) who says he’d rather wait for Data Structures: The Movie . I
imagine each of you reading and learning from this book (even before the
movie version is released), and I wish you much success in your future
careers.
William McAllister
19

CHAPTER 1
Overview and Java Review
OBJECTIVES
The objectives of this chapter are to familiarize the student with the
concepts common to all data structures and to review the Java constructs
used to implement them. More specifically, the student will be able to
Understand the basic terminology of data structures, including data
abstraction, encapsulation, linear lists, complexity, homogeneity, and
the four basic operations performed on structures in both the key field
and the node number mode.
Explain the difference between programmer-defined and built-in
data structures, shallow and deep copies, absolute and relative algorithm
speed, and static and nonstatic methods.
Understand the performance of a data structure, the factors that
affect it, and the techniques used to quantify it, including the application
of a Big-O analysis.
Understand the binary search algorithm and be able to explain and
verify its O(log2 n ) performance.
Understand the significance of a data structure’s density and be able
to calculate it.
Understand and be able to declare primitive and reference variables
in Java, arrays of primitive and reference variables, and understand their
underlying memory models.
Understand and implement methods, classes, declarations, and
operations on objects, and the passing of arguments by value.
Implement a node definition class and an application class that
creates nodes and operates on them.
Understand and be able to implement inheritance and containership.
20

Generalize a method or class using the generic features of Java.
1.1 Data Structures
A thorough knowledge of the topic of Data Structures is essential to
both software design and implementation. Its subject matter embodies
techniques for writing software that performs efficiently, as opposed to
software that merely produces correct results. Properly designed data
structures can increase the speed of a program and reduce the amount of
memory required to store the data that the program processes. In addition,
it serves as a bridge between a basic programming course and more
advanced topics such as operating systems, networking, and computer
organization.
When asked to reflect on their undergraduate careers, most computer
scientists consider the core topic of data structures to be the most
challenging course in the undergraduate curriculum. Not only is the sheer
volume of the material burdensome, but the concepts and algorithms
introduced in the course are quite novel. Furthermore, implementation of
the concepts into functional software often tax their novice programming
abilities. It is not unusual for students to struggle at the beginning of this
course until their coding skills improve and they can then focus all of their
attention on the underlying concepts and algorithms.
After a brief discussion of data, data structures, and their roles and
importance in the field of computer science, this chapter will present topics
common to all data structures. It concludes with a review of the Java
language, including the topics of objects, inheritance, and generics, which
are necessary to implement the concepts presented in subsequent chapters.
1.1.1 What Is Data?
Data is information.
Information such as your name, your address, and your age are all
examples of data. The source of the information can be any device
attached to a computer system, such as a keyboard, a mouse, a modem, a
network line, a disk drive, or some other mass storage device. Sometimes
the program itself can be the source of the data if, during processing, it
generates data (Figure 1.1 ).
21

Figure 1.1 Some of the Sources of a Program’s Input Data
All programs, when viewed on a macro level, are the same: when
executed they accept input data, process the data, and then output a result
(Figure 1.2 ). In some special cases, the input or output data set is empty,
but more often it is not. Occasionally, although the input or output
information appears to be empty, it is not. For example, consider a
program that when executed simply generates a random number. It appears
to have no input data; however, random number algorithms require a seed
value , and although the value is not input by the program user, it still must
be supplied for the algorithm to function properly. This seed value is often
the current time supplied to the program by the computer’s system clock.
Studies show that programs spend 80% of their execution time
searching through memory to locate the data they process. When the data
set is small, this time is usually insignificant. For example, a program that
outputs the largest of 10 numbers stored in memory will identify the
largest number after 10 memory accesses. Assuming a memory access
takes 100 nanoseconds (10−9 seconds), the 10 memory accesses will be
complete and the result will be available in 1,000 nanoseconds. It is
difficult to imagine a user growing impatient in one millionth of a second.
However, if the information was needed to make a critical mid-course
correction to a space probe, or to shut down a runaway nuclear reactor, the
time to locate the datum could be significant, even though the data set is
small.
22

Figure 1.2 Macro View of a Program
When programs deal with large data sets, the time spent searching for
a datum is more likely to be significant. Consider a program that searches
sequentially through 400 million social security records. If the information
requested is the 400 millionth record, and each memory access takes 100
nanoseconds, it will take 40 seconds to locate the information (400 × 106
accesses * 100 × 10−9 accesses per second). Studies show users grow
impatient in one to two seconds; they find 40 seconds intolerable (Figure
1.3 ). Thus, the speed at which a program locates the data to be processed
is usually a concern when designing programs that operate on large data
sets, and it must also be considered when designing programs that operate
on small data sets under stringent time constraints.
Another program design concern is efficient use of main memory.
Programs execute most rapidly when both the data that they operate on and
their processing instructions reside simultaneously in main memory.
However, even on modern computer systems, main memory is a limited
resource. Therefore, it is important that the amount of storage required in
order for the data to be processed (as well as the program instructions) be
kept to a minimum. Techniques for minimizing the data storage
requirements of a program and the speed at which the program locates the
data it processes are fundamental to the topic of data structures. Data
Structures is the area of computer science that addresses these issues in
order to ensure that the storage requirements and speed at which the data is
accessed is consistent with the requirements of the application.
Figure 1.3 A User’s Reaction to a Slow Program
1.1.2 What Is a Data Structure?
23

The National Institute of Standards and Technology defines a data
structure as:
“A data structure is an organization of information, usually in memory, for
better algorithm efficiency.”
Therefore, the study of data structures is the study of how to organize
the information that a program processes in a way that improves the
program’s performance. Specifically, the information is organized in a way
that facilitates the operations the program’s algorithm performs on the data.
For example, suppose salespersons used a program to look up the
price of an item in a store’s inventory. A good organization of the data
would be to place the most popular items at the beginning of the data set,
since a sequential search would then rapidly locate them. However, this
organization of the data would not as be attractive if the function of the
program were to output the store’s inventory in alphabetic order.
To improve the program’s efficiency, the organization scheme often
requires additional storage over and above the size of the program’s data
set. This extra storage is commonly referred to as overhead . Returning to
our store example, if we wished to perform rapid price checks and rapidly
produce inventory printouts, an organization scheme might store the data
set twice: ordered by an item’s popularity and ordered in alphabetic order.
Although this would facilitate the operations performed on the data set
(price check and inventory output) it would double the data memory
required by the application.
Standard for Goodness
A good data structure is one that organizes the data in a way that facilitates
the operations performed on the data, while keeping its total storage
requirement at, or close to, the size of the data set.
There are two types of data structures: built-in data structures and
programmer-defined data structures. Built-in data structures are schemes
for storing data that are part of a programming language. Examples of
these are memory cell (variable ) declarations, arrays, and Java’s String
class.
Programmer-defined data structures are schemes for storing data that
are conceived and implemented by the programmer of a particular
program. These data structures are constructed from the built-in data
structures. Examples of these are parallel arrays, class definitions, hashing
schemes, linked lists, trees, stacks, and queues. As programming languages
24

evolve, the built-in data structures have expanded to include some of the
programmer defined data structures. For instance, Java’s Application
Programmer Interface includes implementations of hashing schemes,
linked lists, trees, stacks, and queues. Many of these data structures are
also implemented in the C++ Standard Template Library .
1.2 Selecting a Data Structure
The selection of a data structure for a particular application can have
a significant effect on the performance of the program. Techniques
borrowed from other engineering disciplines are used to ensure that the
structure selected satisfies the performance criteria of the application and
has a minimal impact on the overall implementation cost of the
application.
1.2.1 The Data Structure’s Impact on Performance
In an introductory programming course, the programs students write
operate on small data sets and execution time constraints are not
considered. As such, these programs are evaluated using the following two
criteria (standards for goodness):
• The “penmanship” of the program is good.
• For all sets of valid inputs, the program produces the correct outputs.
Here, we will consider penmanship to include such issues as an
absence of syntax errors, good variable naming, proper indentation, good
organization of the code (including the use of subprograms), and proper
use of the language constructs (e.g., appropriate use of switch statements
vs. if-else-if statements).
These criteria are adequate for programs that operate on small data
sets. However, as discussed, programs that process large sets of data, or
have stringent speed constraints previously imposed on them, must also
consider two additional criteria:
• Efficient use of storage—both main memory and external storage.
• Speed of execution.
There is additional criteria that brings us to the study of Data
Structures, because the scheme that is selected for storing large data sets
can have a significant impact on both speed and memory utilization.
To illustrate this point, consider a program that would service
25

telephone number information requests for a large city. Aside from
information requests, listings will be both added to, and deleted, from the
data base as residents move into and out of town. In addition, listings will
be updated as customers relocate within the town. The conditions and
assumptions of the application are given in Table 1.1 , as is the operation
speed requirement, 0.10 seconds.
Suppose that four software firms have developed a program for this
application, each of which uses a different data structure to store the data,
as specified in Table 1.2 .
Table 1.1
Conditions and Assumptions for a Telephone Information
Request Application
Condition Problem Assumptions
Number of phone listings 100,000,000
Size of each person’s name and each
listing
16 bytes and 50 bytes
Time to fetch 4 bytes from memory 100 nanoseconds (100 × 10−9
seconds)
Time to execute an instruction 2 nanoseconds (2 × 10−9 seconds)
Maximum time to perform an
operation
0.10 seconds
Table 1.2
Data Structures Used by Four Telephone Information Request
Programs
Program Data Structure
1 Unsorted Array
2 Sorted Array
3 Hashing
4 Perfect Hashing
Based on the first set of criteria, all the programs perform equally
well: for valid inputs, each program produces valid outputs, and they all
are written with good penmanship. However, since they operate on a large
data set with a specified time constraint—perform an operation within one-
half second—we must also consider the two additional evaluation criteria:
efficient use of memory and speed of execution.
To evaluate the speed and memory utilization of the programs,
26

assume each program is put into service for several days. The average time
needed to perform four common operations on the data set is monitored, as
is the additional storage required by the programs above that used to store
the 100,000,000 listings. The results, summarized in Table 1.3 , show that
the performances of the candidate programs differ greatly. The shaded
cells in the table indicate unacceptable performance. Only programs 3 and
4 meet the speed requirement of the problem, and program 4 requires an
unacceptable amount of additional memory—memory above that required
to store the 100,000,000 listings. (The techniques used to calculate the data
in Table 1.3 are discussed in another section of this chapter.) By
examining the data in the table, we can clearly see that the choice of a data
structure can have a large effect on the execution speed and memory
requirements of a program.
1.2.2 Determining the Performance of a Data Structure
In our hypothetical telephone listing problem, we indicated that the
four programs were actually put into service to evaluate their performance.
However, this is most often not the way the merits of candidate data
structures (i.e., the alternative structures being considered) are evaluated. It
would be much too costly to code an entire program just to determine
whether the performance of a candidate data structure is acceptable.
Rather, this evaluation must take place during the early stages of the
design of the program, before any code is written.
To do this, we borrow tools from other engineering disciplines.
Consider a group of civil engineers responsible for evaluating several
candidate designs for a bridge. Certainly, they would never consider
fabricating each design so that they can be tested to determine which
design is the best. Cost issues aside, this build-and-test approach to design
27

trade-offs would be too time-consuming. Instead, civil engineers perform
detailed calculations early in the design process to evaluate candidate
designs. Based on the results of these calculations, they select the lowest
cost design that satisfies the performance criteria.
Software engineers have adopted this design trade-off technique to
evaluate candidate data structures early in the program design process.
Consistent with the definition of a data structure, two sets of calculations
are performed on each candidate data structure during the trade-off
process:
• Calculations to determine the speed of the operations to be performed
on the data.
• Calculations to determine the amount of extra storage (overhead)
associated with the data structure.
These two calculations are considered to be a measure of the
performance of a data structure. A high-performing data structure is one
that
• Rapidly operates on the data.
• Minimizes storage requirements.
Unfortunately, due to the architecture of modern computer systems,
rapid operation and minimal storage are usually mutually exclusive. Data
structures that are very fast normally require high storage overhead; this
was the case with our fourth telephone program’s data structure. Data
structures that minimize memory overhead can be slow; this was the case
with our first and second telephone programs’ data structure. Thus, the
selection of the best structure for a particular application is usually a
compromise, or trade-off, between speed and overhead and one other very
important factor: cost.
1.2.3 The Trade-Off Process
Once the performance of the candidate data structures has been
calculated (i.e., their speed and memory requirements), the trade-off
process begins aimed at selecting the best data structure for the
application. The selection of the best data structure should always be based
on the following guideline:
Select the least expensive data structure that satisfies the speed
requirements and storage constraints of the application.
Thus, there are three factors to consider in the trade-off: cost, speed,
28

and memory overhead. The process is illustrated in Figure 1.4 .
Speed requirements can vary widely from one application to another.
A program that is monitoring the temperature of a nuclear reactor may
have to operate on its data within a few hundred nanoseconds to prevent a
meltdown, while a program that updates a bank account balance may have
several seconds to perform its operation. When the data processing is
performed to update a display viewed by humans, an operation time of 0.1
seconds is more than adequate. Studies show that faster response times are
imperceptible to humans. Whatever the speed requirements for a particular
problem are, good software engineering practices mandate that they be
specified before the program is designed and that they be documented in
the project’s Requirements Document .
The cost of a data structure is primarily the labor cost associated with
developing the code to implement the data structure. This includes its
design, coding, testing, and documentation. Some data structures can be
implemented in a few lines of code, while others take thousands of lines.
Software engineering studies indicate that the cost of software is directly
proportional to the number of lines of code that the software contains.
Typical software costs for large software projects are illustrated in Figure
1.5 for various burdened labor rates. 1 The cost shown includes the cost of
design, implementation, testing, and documentation. Thus, the most cost-
effective data structures are those whose implementations require a
minimal amount of code and utilize builtin data structures in their design.
Figure 1.4 Data Structure Selection Process Involving Four
Candidate Structures
29

Figure 1.5 Cost of Software that is Part of a Large Project
Using data similar to that presented in Figure 1.5 , the designer of a
data structure can estimate its cost by simply estimating the number of
lines of code required to implement the data structure. Once this is done,
the most inexpensive structure is selected from those that meet the speed
criteria and demonstrate an acceptable level of memory overhead.
To illustrate the trade-off process depicted in Figure 1.4 , we will
return to our telephone listing application. During the Requirements Phase
of the project, our system analysis team has met with the customer and has
consolidated their findings in the project’s Requirements Document.
Assume that the conditions specified in Table 1.1 have been reproduced
from that document. As previously mentioned, four candidate data
structures have been proposed for the project (see Table 1.2 ). These will
be passed through the trade-off process illustrated in Figure 1.4 .
To begin this process, the speed of the operations are calculated for
each of the candidate structures using techniques explained later in this
text. The results of these calculations are presented in Table 1.3 . Since the
required maximum operation time was specified to be less than 0.10
seconds (see Table 1.1 ), the first and second data structures are eliminated
from further consideration. (Their maximum operation times are 6 seconds
and 10 seconds respectively.)
Next, calculations are performed to determine the additional storage
required by the two remaining candidate data structures that are necessary
to store the 100,000,000 telephone listings. The results of these
calculations are presented in the rightmost column of Table 1.3 . The
overhead associated with the fourth structure (1.7 × 1031 bytes) is
unacceptably large, and so it is eliminated, leaving us with the third
30

structure. Its additional memory requirements are only 10% larger than the
minimum required to store the phone listings (5,000,000,000 bytes =
100,000,000 listings × 50 bytes per listing).
Finally, a cost analysis is performed on the third structure. As
previously discussed, one simple cost analysis technique is to estimate the
number of lines of code required to implement the data structure. Then, a
chart similar to Figure 1.5 can be used to estimate the implementation cost.
If this cost is acceptable, the structure would be used for the application. If
not, other candidate structures would have to be proposed, or the
requirements of the project, specified in Table 1.1 , would be renegotiated
with the customer.
1.3 Fundamental Concepts
All data structures share a common set of concepts that include:
• A common terminology .
• The manner in which the information stored in the structures is
accessed .
• The manner in which the information is operated on .
• The manner in which the structures are implemented .
• The concepts of abstraction and encapsulation .
We will begin our study of these topics with a discussion of the
common terminology of data structures.
1.3.1 Terminology
Before proceeding further with our study of data structures, it is
necessary to gain an understanding of five terms fundamental to the study
of data structures: field, node, key field, homogeneous structure, and linear
list.
Field:
A field is an indivisible piece of data.
Our phone listings consist of three fields: a person’s name, the
person’s address, and the person’s phone number. By indivisible , we mean
that if the field were divided further, it would lose all meaning. For
example, a seven-digit phone number could be divided into seven separate
integers, but in the context of our phone directory, those seven separate
integers would have no meaning in that they would no longer represent a
31

phone number.
Node:
A node is a group of related fields.
In our phone book problem, a single listing is called a node. In some
programming languages, node definitions are coded using the structure
construct (in C++ the keyword struct). Java programmers code node
definitions using the class construct, with the node’s fields coded as the
class’ data members. The relationship between the fields of a node must be
a belongs-to relationship, so if the name field of a phone book node
contains the name “Al Smith,” then the address field must contain Al
Smith’s address, and the phone number field must contain Al Smith’s
phone number.
Key Field:
A key field is a designated field in a node whose contents are used to
identify, or to refer to, the node.
In the case of our phone book information request application, the key
field would normally be the name field. The contents of the name field are
used to identify which listing is to be fetched from the data set. There can
be more than one key field. If the application was also used by the police
to trace a call, the phone number field would also be designated as a key
field.
The definitions of field, node, and key field are illustrated in Figure
1.6 , which refers to the telephone listing application.
Homogeneous data set:
A set of nodes in which all the nodes have identical fields (number and
type)
Figure 1.6 A Node and Its Fields
The data set in our telephone listing application is a homogeneous
data set because each node in the data set has the same three fields (a
32

person’s name, address, and telephone number), and these fields in all the
nodes are of the same type (String). Most data sets are homogeneous.
1.3.2 Access Modes
Access is the process by which we locate a node in memory. After the
node is accessed or located, then an operation can be performed on it.
There are two generic ways or modes used to specify the node to be
accessed:
• The node number mode.
• The key field mode.
In the node number mode, the number of the node to be operated on
is specified (e.g., fetch back the third node). In the key field mode, the
contents of the designated key field are specified (e.g., fetch back the node
whose key field contains the value “Al Smith”). Most data structures
utilize the key field access mode. However, as we will see, there are some
important applications in which the node number mode is used.
1.3.3 Linear Lists
An implicit assumption in the node number mode is that nodes are
stored in a linear fashion, called a linear list . A collection of n nodes is a
linear list if:
• There is a unique first node, N 1 .
• There is a unique last node, N n .
• For any other node, N i , a unique (one and only one) node, N i −1 ,
comes just before it, and a unique (one and only one) node, N i +1 ,
comes just after it.
Figure 1.7 illustrates a group of nodes organized in two different
ways. The organization of the nodes depicted on the left side of the figure
satisfies the linear list condition stated above. A is the unique first node, D
is the unique last node, and the other nodes have a unique predecessor and
ancestor. The organization depicted on the right side of the figure is not a
linear list. It does not have a unique first or last node.
1.3.4 Data Structure Operations
The operations performed on data structures can be arranged in a
33

hierarchy. The most fundamental operation is Insert , the operation which
is used to add a node to the data structure. It is considered to be the most
fundamental operation because without it all data sets would be empty.
Therefore, this operation must be available for all data sets.
At the next level of the operation hierarchy are the operations Fetch
and Delete . The Fetch operation returns a node from the data set, and the
Delete operation eliminates a node from the data set.
Figure 1.7 Example of a Linear and a Non-Linear List
One level above these two operations is the Update operation. It is
used to change the contents of all the fields of an existing node. It is
considered a higher level operation because it can be implemented as a
Delete operation (to eliminate the existing node and its contents) followed
by an Insert operation (to add a node that contains the new information).
Stated from an implementation viewpoint, Update can be implemented
with two lines of code: an invocation of the delete method, followed by an
invocation of the insert method. Although simple, this is not as efficient as
alternative implementations we will discuss, and all fields of a node must
be supplied to the Update operation, even if only one of them is to be
updated .
At higher levels of the operational hierarchy, operations are added to
accommodate the needs of a particular application. One data structure may
provide an operation to output the contents of a node, while another may
provide an operation to output all nodes in sorted order. Both of these
operations would use the Fetch operation. The sorted output operation
could be used in our telephone number application to print the telephone
34

directory. The action of the Insert, Delete, Fetch, and Update operations
are illustrated in Figure 1.8 .
To summarize, the four most fundamental operations performed on
data structures are:
• Insert , to add a node to the data set.
• Delete , to eliminate a node from the data set.
• Fetch , to retrieve a node from the data set.
• Update , to change the contents of a node in the data set.
These operations can be performed in either the node number or key
field access mode. The particular operations provided and the access mode
utilized for a given application are specified during the systems analysis
phase of the software project.
Figure 1.8 Action of the Insert, Delete, Update, and Fetch Operations
Most beginning programmers have used the Fetch and Update
operations on a data structure in the node number mode without realizing it
and without realizing that these operations are part of the larger Data
Structures picture. Consider the built-in structure array that is introduced
in all basic programming courses. It is a homogeneous data structure: all of
the elements of an array must be the same type (e.g., integers), whose
nodes contain one field (e.g., an integer), and the structure is accessed in
the node number mode (the index is the node number).
There are two operations allowed on the data structure: Fetch and
Update (Insert and Delete are not allowed). One common syntax of the
Update operation used in programming languages is to code the node
name on the left side of the assignment operator (e.g., Node[3] = 24). This
syntax causes the contents of node 3 to be updated to the value 24. A
common syntax of the Fetch operation is to code the name of the node on
the right side of the assignment operator (e.g., myBalance = Node[3]).
This syntax fetches back the contents of a node and then stores the value in
35

the variable myBalance.
1.3.5 Implementing a Programmer-Defined Data Structure
Data structures are implemented in object-oriented programming
languages using the class construct. The memory required for the data
structure is specified as the class’ data members and the operations
performed on the information stored in the data structure are coded as the
class’ methods (subprograms). From a terminology viewpoint, the
statement: “George implemented the data structure” simply means that
George coded and tested the data structure class.
The best way to implement a data structure is to implement it as a
generic data structure. Generic data structures are data structures
implemented in such a way that they can be used for multiple applications,
even though these applications do not have the same node structure (i.e.,
number of fields and type of information). Generic implementations
reduce the cost of software development because once coded for a
particular application, they do not need to be recoded for subsequent
applications. All of the implementations presented in this book will be
based on generic concepts.
To implement a data structure in a generic way, the implementation
must follow a few simple guidelines. They are:
• The node definition and the data structure are coded as separate
classes.
• The node definition class (sometimes referred to as the interface class):
• Always contains a data member for each field in a node.
• Usually contains a toString method to facilitate the output of a node’s
fields.
• The data structure, often referred to as the implementation class:
• Allocates storage to maintain the structure (commonly referred to as
overhead ).
• Allocates storage for the data set.
• Provides initialization methods (coded as constructors) to initialize the
data structure.
• Provides methods to perform the required operations on the data set.
• Usually provides a method to display the contents of all of the nodes.
All data structures coded in this text will use a consistent set of
36

signatures for the basic operations (Insert, Fetch, Delete, and Update).
With the exception of the Fetch method, all of the methods return a
Boolean value 2 that is set to false if they cannot successfully complete
their operation. The fetch method returns a reference to the requested node
or a null reference if the node requested is not in the data set. The
signatures for the operations in the key field mode are:
where:
NodeType is the name of the class that defines the node,
newNode is (a reference to) the node to be inserted,
targetkey is the contents of the key field of the node to be operated on,
and is of type keyType, and,
newContents is (a reference to) a node containing the new node
contents.
The signatures for the operations in the node number mode are:
where
nodeNumber is the number of the node to be operated on.
1.3.6 Procedural Abstractions and Abstract Data Types
(ADTs)
“Viewing an entity as an abstraction” refers to the idea that we do not
need to know the details of how the entity is implemented in order to use
it. Very few people know how to implement an internal combustion
engine, yet we all use automobiles. Therefore, most drivers have an
abstract view of a car. They know what the car does, and how to use it, but
they don’t know how it works. Abstractions are functional views of an
entity.
In computer science, we encounter two abstractions: procedural
abstractions and data abstractions. The term procedural abstraction means
that we do not need to know the implementation details of a method
(procedure) in order to use it. We simply need to know what the method
37

does (so we know when to use it) and the details of its signature or heading
(so we know how to use it). Armed with this superficial understanding of
methods, we can integrate their functionality into our programs.
Similarly, the term data abstraction means that we do not need to
know the implementation details of a data structure in order to use it. We
simply need to know what the operation methods do (so we know when to
use them) and their signatures (so we know how to use them). We do not
need to know the details of how the data is physically stored in memory,
nor do we need to know the algorithms of the basic operations methods. A
data structure that can be used with this superficial level of understanding
is called an abstract data type . To most programmers, arrays are abstract
data types. We know how to use an array to store data, but we do not know
how it does it. For example, to use an array in our programs, we do not
need to know that all of its elements are stored in contiguous memory, nor
do we need to know how it calculates the memory address of an element
given its index.
The term standard abstract data type refers to a data structure whose
operation method signatures conform to a consistent format. The benefit of
this standardization is that an application programmer can easily change
the data structure used in an existing application by simply changing one
line of the application: the line that declares the data structure object. Since
the signatures of the basic operation methods are the same for the original
and new data structures, the invocations of the basic operations need not
be changed. With the exception of the implementation of the Restricted
Data Structures discussed in Chapter 4 , the structures implemented in this
text will utilize the standard method signatures discussed in the previous
section. Standardizing abstract data types reduces the cost of software.
1.3.7 Encapsulation
The principal of encapsulation is a topic of great importance in
computer science because it can greatly reduce the cost of software
development and maintenance by eliminating many difficult hours of
debugging. In addition, encapsulation produces code that is more easily
reused, thereby reducing the cost of future software products. Simply
stated,
Encapsulation is the idea that we write code in a way that establishes
compiler enforced protocols for accessing the data that the program
processes.
38

These protocols usually restrict access to the data. The enforcement
of the protocols by the compiler is manifested in the compiler’s inability to
translate a program that violates the protocols.
As an example, suppose that in a particular application, Tom’s age is
stored in a memory cell named tomsAge, and we wish to set his age to
sixteen. If the data is not encapsulated, we can simply write:
This statement could be coded anywhere in the application. With no
encapsulation, all code in an application has free access to the memory
cells that store the program’s data.
However, if the data are encapsulated, direct access to these data
could be limited to the code of the program’s data structure module. In
computer science, we would say that encapsulation limits the scope of the
program statements that can access a data item. Then, if the statement:
appeared anywhere else in the program outside this scope, the
compiler would issue a syntax error and terminate the translation of the
program. To avoid the compile error, the following line of code would be
used anywhere outside the data structure module to set Tom’s age to
sixteen:
and the assignment of Tom’s age to sixteen would be done inside the
update method of the data structure class.
The pitfalls of writing code with no encapsulation are not obvious
until you have participated in the development of a large program written
by a team of programmers. Consider a 1,000,000-line program written
with a person’s age datum (e.g., tomsAge) fully encapsulated inside a data
structure. Within this large application, George is assigned to write a small
obscure module that rarely executes. To temporarily store the age of an
antique piece of T hom as son furniture, he writes:
and neglects to declare the variable tomsAge in his module.
Coincidentally, he has chosen the name of the variable inside the data
structure that stores Tom’s age.
If Tom’s age datum is fully encapsulated, a compile error is issued
indicating that the variable tomsAge is inaccessible. This is sufficient for
39

George to realize he neglected to declare the variable in his module, and he
quickly corrects the problem. George then receives his own variable
named tomsAge, distinct from the encapsulated variable of the same name
used to store Tom’s age.
If Tom’s age datum were not encapsulated, a compile error would not
be issued, and George would deliver his module without a declaration for
his local variable: tomsAge. Then, after ten years of trouble-free operation,
circumstances cause George’s module to execute for the first time, and
Tom is suddenly 200 years old.
The type of error that George made—forgetting to declare a variable
—is unfortunately quite common and, without encapsulation, very time-
consuming to find. Yet, it can be nipped in the bud by encapsulation. This
is one reason for the rise in popularity of object-oriented programming
languages because, through the use of their class construct, we can easily
encapsulate data.
The class construct allows us to encapsulate more than data. The code
that is allowed to manipulate the data is also encapsulated inside the class
construct or module. For example, the code of the update method used to
change Tom’s age to 16 would be part of the class that contained the
declaration of the data member used to store Tom’s age. The advantage of
this modularization is that if the data are not being manipulated properly,
the focus of the debugging process can be isolated to the code of the
methods encapsulated inside the class module or to the arguments sent to
these methods. No other portion of the program code needs to be
examined. Furthermore, if the classes are designed properly, they can be
developed and tested as independent modules and then easily integrated
into an application or reused in multiple applications. The reduction in
debugging time and the reusability issues associated with encapsulation of
data and methods in one entity greatly reduce the time and cost required to
develop a software product.
1.4 Calculating Speed (Time Complexity)
To evaluate the merits of candidate data structures during a project’s
design phase, we must calculate the speed of these data structures, which is
related to the speed at which the code of the basic operations (i.e., the
insert, fetch, delete, and update methods) execute. Paradoxically, during
the design phase we have not yet written this code, so the pseudocode
versions of the operation algorithms are used in this analysis.
40

Intuitively, we may consider the speed of an algorithm to be the time
it takes to execute (in seconds). However, this measure of speed, which is
commonly referred to as wall (clock) time can vary widely from one
execution of the algorithm to another because it is not only dependent on
the algorithm but also on many other factors. These factors include the
speed of the hardware it is running on, the efficiency of the language
translator and operating system, and the number of other processes the
platform is executing. To eliminate these platform-dependent factors, a
technique called complexity analysis is used to analyze algorithms, not
only from a speed viewpoint (commonly referred to as time complexity ),
but also from a storage requirement viewpoint (commonly referred to as
space complexity ).
The time complexity of an algorithm is expressed as a mathematical
function T(n ), where n is usually the number of pieces of data the
algorithm processes. As we will see later in this chapter, often there are
several terms in this time complexity function, which are determined by
analyzing the algorithm. This analysis can be greatly simplified through
the use of a mathematical tool called Big-O Analysis . Using this tool the
algorithm analysis is reduced to determining only one term, the dominant
term of the function. Before proceeding with our discussion of algorithm
speed and the method to determine the time complexity function of an
algorithm, we will examine the technique of Big-O analysis.
1.4.1 Big-O Analysis (O Standing for Order of Magnitude)
Big-O analysis is an analysis technique used to set a bound on the
upper limit of a mathematical function. The analysis technique is based on
the assumption that one of the terms of the function will dominate , or
contribute, all but a negligible portion of the value of the function, usually
as the independent variable(s) gets large. Under this assumption, if we are
only interested in the approximate value of the function at a large value of
the independent variable, we can simply evaluate the function’s dominant
term and neglect all of the other terms.
As an example, consider the function y = 20,000n + 5000 log2 n + n
2
. Table 1.4 presents the values of this function and its three terms for
increasing values of n . The highlighted values of the terms in the table are
those terms that contribute more than one percent of the value of the
function.
41

For small values of n , n ≤ 106 , various combinations of the three
terms make significant contributions to the value of the function. Beyond
this value of n , the only term contributing more than one percent of the
value of the function is the third term. If we were to extend the table
beyond n = 108 , this trend would continue. Thus, the term 2n 2 dominates
the function y = 2000n + 5000log2 n + n
2 as n gets large, so to
approximate the value of this function for n > 106 we simply need to
evaluate its third term.
The dominance of one term in a multitermed function is the basis for
Big-O analysis. As a result, the following guideline can be used to evaluate
the functional relationships associated with an algorithm’s time and space
complexity.
To approximate the value of a function of n , as n gets large, it is sufficient
to identify its dominant term and evaluate it for adequately large values of
n .
In an extension of this concept, constants that multiply the dominant
term of a function (e.g., the 2.4 in the term 2.4n 2 ) are neglected when we
express the proportional relationship between the function and large values
of its independent variable. Thus, we would say that the function y =
5000n + 3000log2 n + 2n
2 is proportional to n 2 as n gets large. In Big-O
notation, this is written as: y is O(n 2 ), for large values of n .
Big-O analysis is used to evaluate functional relationships that arise
42

in all fields of engineering. However, its use in software engineering to
evaluate the speed of an algorithm is simpler than its use in other
engineering disciplines because there are a limited number of functions
that result from the analysis of algorithm complexity. Table 1.5 presents
the terms that appear in most algorithm speed functions ordered by their
relative magnitude (for large values of n ) from smallest to largest.
Table 1.5
Relative Dominance of Common Algorithm Complexity Terms
Dominant Term in an
Algorithm’s Speed
Function
Name of the
Dominant
Term
Relative Magnitude of the
Dominant Term for Large Values
of n
c , a constant Constant
log2 n Logarithmic
n Linear
n log2 n Linear
logarithmic
Powers of n : n 2 < n 3 <…< n i (i < n ) Polynomial c n Exponential n ! Factorials Figure 1.9 Growth Rate of Common Functions as n Gets Large Figure 1.9 presents the magnitude of several of these terms as n gets large. It can be seen from this figure that not only is the magnitude of the terms towards the bottom of Table 1.5 larger at a given value of n , but 43 also the difference in the magnitude of these terms increases as n increases. It should be noted that there are two other analysis techniques used to determine the approximate value of a mathematical function, Big-Omega (Ω) and Big-Theta (Θ). Big-Omega is on the opposite end of the analysis spectrum from Big-O. Whereas Big-O analysis determines the upper bound of a function, Big-Omega determines the lower bound of a function. In terms of algorithms, Big-Omega analysis is used to determine the minimum execution speed of an algorithm. For example, the function y = 20,000n + 5000log2 n + n 2 has a Big-O value of O(n 2 ) while its Big- Omega value is Ω(log2 n ). Big-Theta analysis is used to analyze functions whose upper and lower bounds are of the same order of magnitude. For example, the function y = 10 + n log2 n has a BigTheta value of Θ(n log2 n ). Not all functions lend themselves to Big-Theta analysis. In the remainder of the textbook we will use Big-O analysis in our discussions of algorithm complexity. Having gained an understanding of Big-O analysis, we will now return to our discussion of the evaluation of an algorithm's speed. 1.4.2 Algorithm Speed When discussing algorithm speed, there are two factors to consider: • The relative speed of the algorithm (relative to other algorithms). • The absolute speed of the algorithm. Relative speed is used to determine whether an algorithm (or code segment) is faster or slower than other algorithms. Absolute speed is used to determine the actual execution time, in seconds, of an algorithm. As we have indicated, absolute speed is a much more difficult calculation, since it is dependent on many platform-specific parameters. The analysis performed to determine both of these speeds can be simplified using Big-O analysis; however, the approximation inherent in this technique may invalidate its use in the calculation of absolute speed for some time-critical applications. We shall first turn our attention to the topic of relative speed. 1.4.3 Relative Speed of Algorithms To determine the relative speed of several candidate algorithms, we analyze each algorithm's pseudocode, to determine the dominant term in 44 each algorithm's speed function, T(n ). The fastest algorithm is the algorithm whose dominant term occupies the highest position in Table 1.5 and the slowest algorithm is the one whose dominant term occupies the lowest position in Table 1.5 . Stated another way, the algorithms are ranked in order of speed by their relative positions in Table 1.5 ; the faster the algorithm, the closer its dominant speed term is to the top of the table. For example, suppose we are trying to evaluate the relative speed of three candidate algorithms A, B, and C, and that through an analysis of their pseudocode, we determine the dominant terms in their speed function (TA , TB , and TC ) to be: 3 Examining Table 1.5 , we find that log2 n occupies the highest position in the table (row 2). Further down the table, we see that n 2 and n 3 occupy a lower position in the table (row 5), but n 3 is to the right of n 2 on that row. Therefore the relative speed order of the algorithms from fastest to slowest is: TA (the fastest), followed by TC , followed by TB (the slowest). Suppose a more extensive analysis of the algorithms was performed to determine all of terms of their speed functions as: Substituting n = 106 nodes into these equations we obtain speeds of 6.98 × 102 , 1.00 × 1018 , and 5.00 × 1014 for TA , TB , and TC respectively, verifying that our Big-O analysis did indeed identify the fastest algorithm. It is interesting to note that for small values of n , algorithm C is actually faster than algorithm A, but for values of n larger than 8.1, algorithm A is the fastest. In our Big-O analysis, we do not need to determine the constant multipliers of the dominant terms in the speed functions (e.g., TA was O(log2 n ), not O(20 log2 n )). This speeds up the analysis of the algorithms because determining the constant multiplier requires a close inspection of the algorithm, which can be a time-consuming process. The multipliers only need be determined when the level of dominance of the terms is the same. For example, if the dominant terms of two speed equations are both O(n 3 ), then the multipliers will determine which 45 algorithm is the fastest; the higher the multiplier, the slower the algorithm. In summary, to use Big-O analysis to determine the relative speed of a group of algorithms we must: Examine each algorithm's pseudocode to determine the most dominant term in their speed equation, ignoring constant multipliers. Rank them from fastest to slowest based on the position their dominant terms occupy in Table 1.5 (faster dominant terms are toward the top of the table). If the dominant terms are the same, the constant multipliers must be determined. The higher the constant multiplier, the slower the algorithm. Relative Speed Case Study: the Binary Search Algorithm The binary search algorithm is a technique for rapidly finding a data item stored in an array. Given the data item, called a search value , the algorithm returns the index of the array element where the search value is stored. In the context of data structures, the search value is usually the contents of the key field of a node, and the array is used to store references to the collection of nodes. The algorithm assumes the data in the array are sorted . For this example, we will assume the data are sorted in ascending order. If the array did in fact store nodes, the nodes would be stored in sorted order based on the contents of the key fields of the nodes. The algorithm defines a sub-array as a portion of the given array that includes the search value. The search value can be anywhere in the array, and so initially the sub-array is the entire array. The element accessed is always the middle element of the sub-array. If this is not the desired search value, the algorithm eliminates half of the elements in the sub-array by simply determining if the accessed element is greater than or less than the search value. This process continues until the search value is found. Assuming that the array is named data, it contains n elements, and that the variables low, high, and searchValue are defined as follows: low stores the index of the lowest element in the sub-array, high stores the index of the highest element of the sub-array, and searchValue stores the value being searched for (assumed to be in the array). 46 The algorithm assumes that the value being searched for is in the array. When the algorithm terminates, the variable i contains the index of the array element that stores the value searched for. Having discovered the algorithm, we will now perform a Big-O analysis to determine its relative speed so that it can be compared to other candidate search algorithms. If the search value is stored in the middle of the array, we will find it without executing the loop. This is the best-case scenario. In a worst-case scenario, the search value is the first (or last) element of the array. Assuming the search value is the first element (index 0) and that the array contains 16 elements, then the first value looked at would be the 8th element, then the 4th, then the 2nd, and finally the 1st element. Thus, the loop would execute three times. Extending this reasoning, if there were 32 elements in the array, the elements accessed when looking for an item stored in first element would be the 16th, 8th, 4th, 2nd, and 1st. In this case, the loop would execute 4 times. Extrapolating this reasoning further, Table 1.6 presents the maximum number of times the loop executes T for array sizes between 16 and 1024 elements. Examining the data in this table, we can deduce the functional relationship between T and n by observing n = 2(T+1) . To solve for the time complexity function T(n ) we take the log2 of both sides of the equation, which yields log2 (n ) = log2 (2 (T+1) ) = T + 1. This means that T = (log2 n ) − 1. Therefore, in the worst case , the loop executes (log2 n ) − 1 times, or the algorithm speed is, at worst, O(log2 n ). Since only one item can be located in one loop iteration, while two items can be found in two loop iterations, four in three iterations, etc., the average number of 47 comparisons is weighted towards the maximum value of O(log2 n ). Table 1.6 Maximum Number of Loop Executions in the Binary Search Algorithm when the Search Value is in the First or Last Element of the Array Number of Loop Executions Size of the Array T n 3 16 4 32 5 64 6 128 7 256 8 512 9 1024 If we were comparing its speed to other algorithms whose speed were also O(log2 n ), we would need to determine the multipliers of log2 n . To do this, we need to further analyze the algorithm to determine the number of instructions executed inside the loop. Inspection of the algorithm reveals that statements 4, 5, and 9 will execute every time through the loop, as will either statements 6 or 8. Therefore the multiplier is four statements, and our refinement to the Big- O analysis of this algorithm would be to say its dominant speed term is 4(log2 n − 1). Stated another way, this algorithm will locate an item in an n element array after executing, at worst, 4(log2 n − 1) instructions as n gets large. This is a fast search algorithm, because for an array of 106 elements we can locate a given value after executing, at most, 4(log2 10 6 − 1) = 76 Java instructions. What makes the algorithm fast is that with every look into the array, we eliminate the need to examine half of the remaining elements of the array. 1.4.4 Absolute Speed of an Algorithm Although Big-O analysis identifies the fastest algorithm from several candidates, it could be that the absolute speed of the fastest algorithm is not fast enough for a particular application. In this case, the algorithm 48 discovery process must continue. Conversely, if the fastest algorithm is too costly to implement, or its storage requirements are unacceptably large, perhaps the absolute speed of a slower, but less costly, algorithm would meet the speed and storage requirements of the application. The most reliable way to determine the absolute speed of an algorithm is to code it, translate it into the machine language of the platform on which it will run, and then measure its wall time as it processes a representative set of input data on that platform. However, this is not often practical during the design process, and so it is useful to perform a more convenient but less accurate analysis technique to approximate absolute speed. Assuming the CPU is dedicated to the execution of the algorithm, we can express execution time, t , of the algorithm as the sum of the execution time of the individual instructions in the machine language translation of the algorithm. Assuming the translation consisted of n instructions, we have: where: n is the total number of machine language instructions in the translation of the algorithm, and t mi is the time required to execute the i th machine language instruction. In order to obtain a more workable form of the previous equation, we categorize the machine language instructions into groupings based on the number of clock pulses that are required to execute the instruction. Assuming there are g groupings, the above equation becomes: where: g is the number of instruction groupings, t i is the time required to execute an instruction in the ith grouping, and n i is the number of instructions in group i . In the simplest case, instructions are assumed to be in one of two groups (g = 2): those that access memory, and those that do not. The reason for distinguishing between instructions that access data in memory from those that do not is that memory access instructions execute much slower than nonaccess instructions. Typically, the speed of memory is orders of magnitude slower than the time it takes the CPU to perform math 49 or logic operations, or to transfer data between its registers. To illustrate this speed difference, consider a system with a 2- gigahertz clock and an average memory access time (the time to read or write a single memory word) of 50 nanoseconds. 4 Assuming that the CPU executes one nonaccess instruction (i.e., math, logic, or register transfer instructions) in one clock pulse, the time to execute a nonaccess instruction, t na , would be The time to execute a memory access instruction t a would be which is 100 times slower than the 5 × 10−10 seconds to perform a nonaccess instruction. With g = 2, equation 1.1 , which gives the absolute speed of an algorithm in seconds: where: t na is the time to execute a memory nonaccess instruction, n na is the number of memory nonaccess machine language instructions executed, t a is the time to execute a memory access instruction, and n a is the number of memory access machine language instructions executed. The execution times (t na and t a , in equation 1.2 ) can be determined from the specification of the computer system hardware. To determine the number of instructions in each group (n na and n a ), we examine the algorithm line-by-line to determine which grouping each line falls into. We can then make an assumption regarding the number of machine instructions per pseudocode line. These techniques are illustrated in the following case study. Absolute Speed Case Study As an example, consider the following code sequence to sum the 50 elements of the two-dimensional array data consisting of n rows and n columns. To determine n na and n a , we inspect the algorithm, revealing that Line 1 will execute once; Lines 2, 3, and 8 will execute n times; and Lines 4, 5, and 6 will execute n 2 times. Next we need to identify which of these are not memory access instructions. Modern optimizing compilers will recognize the value of storing variables (such as loop variables or counters) that are accessed repeatedly inside of loops, in CPU registers. Therefore, Lines 1, 2, 3, 4, 6, and 8 are not considered memory access instructions. Line 5, however, refers to a different element of the array every time through the inner loop, and therefore requires a memory access. Line 5 also performs an addition operation. Assuming one CPU operation translates into one machine language instruction, we have: Assuming the same hardware as in the previous example (t na = 5 ×10−10 seconds per nonaccess instruction and t a = 5 × 10 −8 seconds per memory access instruction), the execution time, t a , of the algorithm would be: For a 1000 × 1000 element two dimensional array (n = 103 ), this equation evaluates to t = 0.0515 seconds. It is interesting to note that a Big-O analysis would have resulted in a relative speed of O(n 2 ), and if the memory access instruction execution time were used for each instruction executed, the algorithm execution time would have been 0.050 seconds (10002 instructions * 5 × 10−8 seconds per instruction). This is within 3% of the time determined by the more complex absolute speed analysis. 51 To summarize, the procedure to calculate absolute speed of an algorithm (to within a good approximation) is: 1. Analyze the algorithm to determine the number of instructions that will be executed. For instructions in loops, this will be a function of n , the number of times the loop executes. 2. Identify the memory access and nonaccess instructions. 3. Add up the number of access and nonaccess instructions to determine n a and n na respectively. 4. Determine the hardware dependent time to execute the access and nonaccess instructions, t a and t na respectively. 5. Calculate the speed of the algorithm as: t = t na × n na + t a × n a . 1.4.5 Data Structure Speed As previously discussed, the speed of a data structure depends upon the speed of its operation algorithms. However, after calculating either the relative or the absolute speed of the algorithms of each candidate data structure, it still may not be obvious which data structure is the fastest. Consider the following case study. Suppose that two candidate data structures, A and B, are being considered for an application in which only the Insert and Fetch operations will be required. As part of the design process, the absolute speed of these algorithms has been calculated for each of the structures as shown in Table 1.7 . Table 1.7 Insert and Fetch Operation Speed for Two Candidate Data Structures Data Structure Insert Speed (seconds) Fetch Speed (seconds) A 2.0 0.10 B 0.2 0.11 By examining the speeds of the operations of the two structures presented in the table, we would probably conclude that B is the best structure. After all, Structure B is 10 times faster in insertion speed, and the two structures are very close in fetch speed. However, the data presented in the table is insufficient to allow conclusions about which data structure is best for this application. We 52 need another piece of information: the frequency of each operation. Suppose that during a typical day, 1,000,000 fetch operations and no insert operations are performed. In this case, A would be the faster structure. This would be the case for an application in which, after the initial data set is stored, no new nodes are added to the structure. An example of this kind of a data set would be a language dictionary, where the two fields in the nodes are a word and its definition. As illustrated in this example, to determine the best structure for an application we must consider not only the speed of the operations but also the frequency at which the operations are performed over a given period of time. A good parameter that does this is a frequency-weighted average operation time, t avg , defined as: where: ti , is speed of the i th operation (i = 1, 2, 3,…, n ), n is the number of different operations available on the structure, and f i , is frequency of the i th operation (i = 1, 2, 3,…, n ). Applying this to our case study involving the dictionary database involves two operations: Substituting the data from Table 1.7 for t 1 and t 2 ; and substituting frequencies of 1,000,000 fetches per day for f 1 , and 0 inserts per day for f 2 , the average speeds for Structures A and B are: The equation for t avg is normally expressed in a simpler form. The denominator is divided into each term of the numerator to give: This form of the equation is more convenient, because rather than specifying the number of times each operation is performed over a period of time, we simply specify the probability of the operation occurring (e.g., 10% = 0.1). Keep in mind that the sum of the probabilities for all 53 operations must equal one. Applying this form of the equation to our test case, and realizing the probability of an insert operation is 0% (0.0) and the probability of a fetch operation is 100% (1.0), we obtain: As a special case, if all operations are equally probable (the p i 's are equal) then the equation for t avg is reduced to an arithmetic average of the operation speeds. To illustrate this, consider four operations, all with equal probabilities (25% = 0.25). Assuming the operation times were 1, 2, 3, and 4 seconds, the arithmetic average would be 10 / 4 = 2.5, and the above formula would yield: 1.5 Calculating Memory Overhead (Space Complexity) All encapsulated data structures must allocate sufficient memory to store the information inserted into the data structure. In addition to that storage, they must also allocate storage that is used by the operation algorithms to maintain the structure. This extra memory, in excess of that required to store the nodes' information, is called overhead . Some structures require a minimal amount of overhead storage, while others require a large amount. Since memory is a coveted resource, data structures that minimize overhead are more desirable and are said to be more memory efficient . The parameter (or metric) used to specify how efficiently a data structure uses memory is density , D, defined as: D = information bytes / total bytes = information bytes / (information bytes + overhead bytes) where: information bytes is the amount of memory required to store the information stored in the structure, in bytes, total bytes is the total amount of memory allocated to the structure, in bytes, and 54 overhead bytes is the amount of memory required to maintain the structure, in bytes. From the previous equation we see that the density, D, equals one when the overhead is zero; otherwise, D is always less than one. In fact, as the overhead memory required by a data structure gets large, D approaches zero, and since all data structures have some overhead, the range of density, D, is: 0 < D < 1 Stated another way, memory-efficient data structures have a density close to one, while memory-inefficient data structures have densities that approach zero. Consider our telephone listing nodes, where each node has 50 information bytes and the number of nodes in the structure, n , is 100,000,000. If these nodes are stored in a data structure that utilizes an additional 10 bytes per node to maintain the structure, its density would be 0.83, calculated as: The overhead memory required by the data structure in the above example is a multiple of the number of nodes in the structure, n . When this is the case, the density of the structure D is not a function of the number of nodes in the structure, since n can be eliminated from both the numerator and the denominator of the density equation. The density of our 100,000,000 node structure would calculate to 0.83 whether there are 100,000,000 nodes or 10 nodes stored in the structure. Some data structures require a set (or constant) amount of overhead, O, independent of the number of nodes. The density for these types of structures increases as the number of nodes in the data structure increases. To demonstrate this, assume that each node has w information bytes, and thus the density is: Since the overhead, O, was assumed constant and the node width, w is constant for a particular data structure, as n gets large, O / (w * n ) approaches zero, and therefore D approaches 1 = 1 / (1 + 0). 55 Often data structures are a combination of the two cases discussed previously: they have a set amount of memory overhead, O, as well as an additional amount of overhead per node. The density of these structures also increases, as the number of nodes increases. 1.6 Java Review There are several Java constructs, operations, and concepts that are used in the implementation of most data structures. These are: • The declaration of an array of primitive values. • The definition of a class. • The declaration of an object. • Accessing an object. • Deep and shallow copies. • The declaration of an array of objects. • Objects that contain objects as data members. • Objects that extend objects: inheritance. • Parent and child references. • Generic methods and classes. Familiarity with these topics, which are usually covered in the first course in computer science, is assumed in the subsequent chapters of this book. However, some of these topics are introduced late in the first computer science course, if they are covered at all. As a result, some students have only a superficial knowledge of these topics or no knowledge at all. Therefore, a formal review of these topics is useful before proceeding to subsequent chapters. 1.6.1 Arrays of Primitive Variables Primitive variables are single instances of integral or real types of information. They are declared using the Java data types boolean, byte, short, int, long, char, float, and double. Assuming dataType represents a primitive type, arrays of these primitive variables are declared using the syntax 56 Figure 1.10 Allocation of Storage for an Array of Primitives where arraySize is an integer that specifies the number of elements in the array, and it must be a numeric literal (e.g., 3) or an integer variable (that has been assigned a value). As an example, to declare an array named numbers to store three integers we write: It is very useful to understand the memory allocation process resulting from each of the above statements. The first statement (int [] numbers;) allocates a reference variable 5 (one that can store a memory location) named numbers initialized to the value null . The second statement allocates three contiguous memory cells, each of which can store an integer value, and also stores the address of the first integer memory cell into the reference variable numbers. Finally, each cell is assigned an index value starting from zero (in ascending order), and each of the integer memory cells is initialized to zero. Figure 1.10 illustrates the two-step memory allocation process resulting from the execution of the two statements. The shaded parts of the figure are a result of the first statement's execution. All other parts of the figure are produced by the second statement. An alternate syntax can be used to code the two statements as a single statement: The memory allocation of the single-statement syntax is the same as when the two-statement grammar is used. 57 The array reference variable, like all variables in Java, can only be allocated once in a program, or a duplicate definition compile error will result. Therefore, if the size of the array is to change as the program executes, the two-line grammar is required. The code results in a syntax error because the reference variable, numbers, on the above line of code has been declared a second time. 1.6.2 Definition of a Class A class is a programmer-defined type that consists of data definitions and methods (subprograms) that operate on that data. The name of the class is the name of the newly defined type. The definition of the class (the class statement) does not allocate any memory. As is the case with the built-in primitive types, it simply provides a template for the memory that will be allocated when an instance of this newly defined type (or class) is declared. The simplest grammar for defining a class is: where ClassName is the name of the class. The keyword public is called an access modifier , and it will be used in all the class definitions in this text, 6 Its use allows a method in any application to declare a reference variable of this class type. In addition, throughout this text the data members of a class will be specified using the private access modifier that instructs the translator to enforce the encapsulation of the class' data members. 58 Figure 1.11 The Class Person As an example, consider a class named Person (shown in Figure 1.11 ) that will contain two data members, age and weight. In addition, the class will contain two methods: a two-parameter constructor that will initialize the data members and a toString method to facilitate the output of the data members' values. The translator considers the method Person (Lines 6–9 in Figure 1.11 ) to be a constructor method because it has the same name as the class. A constructor method executes automatically when an instance of the class, an object , is declared. 7 Aside from the indentation used in this code, there are two penmanship issues to be followed when coding classes: • Names of classes should always begin with an uppercase letter (e.g., Person). • Names of data members and methods (except for the constructors) should always begin with a lowercase letter (e.g., age, weight, toString) and an uppercase letter is used when a new word begins within the name (e.g., toString). Parameter names used in method headings (e.g., a and w on Line 6) should not be the same as the names of the data members (age and weight on Lines 3 and 4). When they are the same, the compiler creates a new local variable (within the method) with the same name as the data member. All references to the variable within the method are to the local variable and not to the data member. This issue will be discussed further in the next section. 59 1.6.3 Declaration of an Object Just as we can declare instances of the built-in type int to store an integer, we can declare instances of the programmer-defined type Person to store a person's information. An instance of a built-in type is called a variable, while the instance of a class is called an object . Objects can be created in Java using a two-line grammar: As an example, consider the class Person defined in Figure 1.11 . To declare an object in this class named tom that is 25 years old and weighs 187.6 pounds, we write: Again, it very useful to understand the memory allocation process resulting from each of the previous statements. The first statement (Person tom;) allocates a reference variable named tom that can store the address of a Person object. The value null is stored initially in this memory cell. The second statement allocates an object in the class Person and places the address of the object into the memory cell tom. Inside the newly created object, two memory cells are allocated: an integer called age and a double called weight. Then, since the second statement supplied two arguments (25, 187.6), the two-parameter constructor executes. As coded on Lines 6–9 of Figure 1.11 , this constructor places the values 25 and 187.6 into the data members of the newly created object with the statements: age = a; weight = w;. Figure 1.12 illustrates the two-step memory allocation process resulting from the execution of the two statements. The shaded parts of the figure are a result of the first statement's execution. All other parts of the figure are produced by the second statement. 60 Figure 1.12 Storage Allocation for a Person Object The name of the reference variable is considered to be the name of the object. It is good penman-ship to begin the names of objects with lowercase letters, using an uppercase letter when a new word begins (e.g., myBalance). The two line declaration of the object tom discussed previously can be coded as a single statement: However, as with arrays, the two-line grammar must be used if the object is to be declared more than once in a program. Although the contents of the reference variable, tom, can be changed to refer to different objects as the program executes, it (like any other variable), can be declared only once. The sequence of code that declares the object is referred to as client code . This term comes from the idea that just as clients come to a vendor to obtain products, clients come to the class to obtain objects. Therefore, object-oriented programs contain two types of code: client code and class definition code. The authors of the two may be, and often are, different programmers. For example, most beginning programmers are clients of the class String, in that they declare String objects in their program by writing: or String name = new String(“bob”); 8 Each object declared in a program is allocated its own memory locations to store the values of its data members. Consider the client code: Here, two objects (and, of course, two reference variables) are created and initialized as shown in Figure 1.13 . Each object, tom and mary, contains two memory cells. One named age and one named weight. What is sometimes confusing is that when we examine the line of code age = a; in the constructor of the class Person (Figure 1.11 , Line 7), we cannot tell which of the memory cells named age depicted in Figure 1.13 is being assigned. The constructor, Person, simply assigns the first argument to the variable age (and the second 61 argument to the variable weight) without mentioning an object's name. The determination of which object receives the argument values is decided by the client code. This is typical of object-oriented programming. When the client code creates an object, it specifies the name of the object being created (in this case, tom or mary) and that object's data members are assigned by the constructor. Therefore, to answer the question of which object's age is being assigned a value by the constructor code age = a; we must look at the constructor invocation in the client code. For example, if the client code was Person tom = new Person(25, 187.6); then the variables age and weight in Tom's object would be assigned by the constructor. Figure 1.13 Storage Allocation for Two Person Objects 1.6.4 Accessing Objects After an object is declared, the client code can invoke any publicly accessible method declared in the class. Publicly accessible methods are methods that are declared using the public access modifier in their heading. In the class Person, the method toString is a public method since its code begins with the keyword public (e.g., public void toString()). Once again, if we examine the code of the method toString, we cannot tell which object's age and weight the lines: are referring to. That reference is specified in the client code. In object-oriented languages, the client specifies the object that the method is to operate on (or access ) by mentioning the object name in the method invocation statement 9 using the following grammar: Thus, to output the contents of the age and weight data members of 62 the object mary, the client code would be Accessing information stored in objects is slower than accessing information stored in primitive variables. 10 Suppose that an integer is stored in a primitive variable at location 2000. The integer is fetched by simply accessing the contents of location 2000. However, if the integer is a data member of an object, it is stored inside the object and accessing the integer then becomes a two-step process. First the reference variable that stores the address of the object must be accessed to locate the object, and then the integer can be accessed. Referring to Figure 1.13 , to access Tom's age, first the object's address, 100, stored in the reference variable tom must be accessed, and then, knowing the address of the object, the contents of the integer age can be accessed. Thus, in Java, accessing information in objects requires two memory accesses, while accessing information in primitive variables requires only one memory access. 1.6.5 Standard Method Name Prefixes One advantage of programming in a high-level language is that the code is much more readable. To further promote readability, Java programmers have adopted a prefix convention for naming methods. The prefix gives insights into the source or destination of the data the method processes. Four of these prefixes are input, show, set , and get . The prefixes input and show are used to indicate that the information is flowing between an I/O device and the method. Often this I/O device is a user-interface device (e.g., keyboard or monitor). The other two prefixes, set and get, indicate that the information is flowing between the client code and the method. A set method typically contains a parameter list, and a get method is typically a nonvoid method that contains a return statement. Figure 1.14 illustrates the information source and destination implied by these prefixes (as well as toString). Consistent with these prefixes, a method named getAge added to the class Person would be a method that returned a person's age to the client code, while the method setWeight would be a method the client code could use to store a person's weight in a Person object. Their code would be: 63 Figure 1.14 Class Method Prefixes Without set and get methods, the client code would not be able to access an object's data members, since good programming practice dictates that all class data members are declared with private access to encapsulate the data inside of the class' objects. Assuming a Person object named tom stores information about Tom, the following client code fetches Tom's age from his object and then sets his weight to 200 pounds. One more comment on standardized names: The class Person contains an example of a method named toString, which is a standardized name for a method that returns a string containing the annotated values of a class' data members. As illustrated in Figure 1.14 , it is similar to methods that use the get prefix in that it returns information to the client code. 1.6.6 Shallow and Deep Copies Copying information from one primitive variable to another is done with the assignment operator. As an example, assume we have two integer variables i and j that store 10 and 20, respectively. The code i = j; copies the value from j into i, leaving j unchanged. After the operation, both i and j store the value 20. When we are dealing with objects, the copy operation is not as simple, because there are two types of copy operations we can perform on objects: shallow and deep copies. As we demonstrated, when an object is allocated, the address of the object is stored in a reference variable. To help understand the difference between shallow and deep copies, it is useful to imagine the reference variable floating on the surface of water, 64 with the object (weighed down by its data) sunk to a greater depth. Figure 1.15 depicts two Person objects, tom and mary, using this analogy. Figure 1.15 Shallow and Deep Copy Water Analogy Figure 1.16 A Shallow Copy of the Object mary into the Object tom When a shallow copy is performed, the memory affected by the copy is at the shallow depths (the surface of the water). Therefore, shallow copies only affect the contents of the (shallow floating) reference variables. The Java syntax to perform a shallow copy is: Thus, to shallow copy mary into tom, we write: The action of the statement is the same as when we copy one primitive variable into another. The value 200 stored in the variable mary is copied into the variable tom, and mary would remain unchanged (Figure 65 1.16 ). As a result, both reference variables then refer to the same object. Since the address of the object at location 100 is no longer stored in a reference variable, it would be returned to the available storage pool by the Java memory manager. After the shallow copy, the statements would both output an age of 21 and a weight of 127.3, since they both refer to the same object. When a deep copy is performed, the memory affected by the copy is at deeper depths. Deep copies only affect the contents of objects; the contents of the reference variables remain unchanged. The action of a deep copy is to copy the contents of the data members from one object into the data members of the other object. The result of a deep copy of Mary's object into Tom's object is shown in Figure 1.17 . After the deep copy, the statements Figure 1.17 A Deep Copy of the Object mary into the Object tom would both output an age of 21 and a weight of 127.3 pounds (as they did after the shallow copy was performed). However, in the case of the deep copy, the output is coming from two different objects. Since the reference variables are unaffected by the deep copy, both objects are still referred to (or referenced), and Java's memory manager does not recycle either object. Both objects continue to exist after a deep copy. Java does not provide an operator to perform a deep copy. Rather, a method must be coded in the class of the objects that copies every data member of one object into the corresponding data member of the other object. For example, one version of a method to deep copy Person objects 66 (coded in the class Person) would be: Then, to perform a deep copy of object mary into object tom, we would code: An alternate approach to the coding of a deep copy method is often used in the implementation of data structures. In this approach, the method performs the deep copy into a newly created object and returns a reference to the new object. Assuming the method was to deep copy a Person object, its code would be: As we have discussed, the age and weight mentioned in this method is the age and weight of the object that invokes the method. Therefore, to perform a deep copy of object mary into object tom, we would code: Using this approach, after the deep copy is performed, the reference variable tom contains the address of a newly created object whose data members contain the same values as the object mary. Since the variable tom references the newly created object, Figure 1.17 does not accurately depict the action of this version of a deep copy method. There are three objects involved here, the object referenced by mary, the object referenced by tom, and the newly created object returned from the deepCopy method whose address will be assigned to the variable tom. 1.6.7 Declaration of an Array of Objects An array of objects is declared using a three-step process. Suppose, for example, we wish to allocate an array of 10 objects. First, as with an array of primitives, we must declare a reference variable in which to store the location of the first element of the array. Second, we must declare the 10-element array. In the case of an array of 10 primitives, this step gives us the 10 primitive storage locations. However, in the case of an array of objects, this step gives us an array of 10 reference variables that will store 67 the address of (or point to ) our 10 objects. It is in the third step that we finally allocate the 10 objects, setting their locations into the array of 10 reference variables. Specifically, the three steps necessary to allocate an array of n objects are: • Declare a reference variable in which to store the location of the first element of the array. • Declare an array of n reference variables that will store the address of the n objects. • Declare the n objects and set their locations into the array of n reference variables. The following code declares an array of three Person objects named employee, each initialized to 21 years old and 187.6 pounds. Figure 1.18 Storage Allocation for an Array of Three Person Objects The memory allocation associated with the previous code is illustrated in Figure 1.18 . Alternately, the code could be abbreviated as: 68 1.6.8 Objects that Contain Objects as Data Members Suppose your friend Bob was asked to conduct several classes at a weight loss clinic, each containing three students, and he has asked you to computerize his record keeping. Obviously, each student in Bob's classes is a person, and each has an age and a weight. In other words, Bob's classes will contain Person objects. Being an object-oriented programmer, you realize that a good model for Bob's venture would be a class named WeightLossClass that contains Person objects. The class would initialize the weight and height of each person and be able to output their data. The code of this class would be: Having established this class, the client code for Bob's application could create two of Bob's classes with the statements: The objects class1 and class2 would each contain three Person objects. These objects are referenced by the variables declared on Line 3 of the WeightLossClass class, and are allocated by the constructor on Lines 6–8. The constructor's parameters are the age and initial weight of the three students. Figure 1.19 shows the memory allocated by the constructor for the class1 object. Naturally, other methods would have to be added to the class such as a member method to change the weight of the students as they (ideally) lose weight. 69 Figure 1.19 A WeightLossClass Object Containing Three Person Objects 1.6.9 Classes that Extend Classes, Inheritance Suppose that your friend Bob asked you to improve his WeightLossClass application by adding another piece of information about a student: goal weight. Thus, weight loss clinic attendees may now be considered to be an extension of a person in that they have the attributes of a person (age and weight) but also have the additional attribute of goal weight. Being an object-oriented programmer, you realize you can easily extend the class Person to construct a new class, WeightLossClient, using the object-oriented programming concept of inheritance . The new class would have all the data members and methods of the Person class as well as the new data member, goal weight. In addition, it would contain a constructor and a toString method to accommodate the additional data member. The code of the new class would be: 70 The clause extends Person in the heading of the class WeightLossClient indicates that the class inherits all the attributes and methods of the class Person. We say that the class Person is the parent (or super class) of the child class WeightLossClient. Line 3 of the class adds the new (third) data member that will store a person's goal weight. The code super (a,w) on Line 6 invokes the parent class' (Person's) constructor to initialize the age and weight data members, and Line 10 invokes the parent class' toString method to fetch the annotated values of the person's age and weight. Line 7 initializes the goal weight, and Line 11 adds the value of the goal weight to the annotated string returned from the Person class' toString method. To use the new class in Bob's application, the Person objects contained in the class WeightLoss-Class would be replaced with WeightLossClient objects, and the class' constructor would be expanded to process the goal weight. The revised code follows. 71 Figure 1.20 A WeightLossClass Revised Object Containing Three Initialized WeightLossClient Objects When a WeightLossClass object is created, the goal weight, along with the age and weight, are specified as arguments to the constructor. Thus, the client code to create Bob's class1 object is now: The resulting object is shown in Figure 1.20 . 1.6.10 Parent and Child References When a new class is created that is an extension of an existing class, we say the new class is a child (or sub ) class and that the existing class is a parent (or super ) class. In Java, a parent class reference variable can contain the address of a child object; however, a child class reference variable cannot contain the address of a parent object. If the reference variables p and c are declared as: 72 Figure 1.21 Allowable Assignments Between Parent, p, and Child Reference, c, Variables the statement p = c; is valid, and it places the address of the child object into the parent reference variable. The statement c = p; would be invalid because child reference variables cannot refer to parent objects. If, however, a parent reference actually stores the address of a child object, the statement c = (WeightLossClient)p; is valid syntax for placing (or coercing) the address of the child object stored in p, into the child reference variable c. The allowable assignments involving parent and child reference variables are illustrated in Figure 1.21 . To summarize, using an old-fashioned parenting analogy, a parent can point to a child (object), but it is improper for a child to point to a parent (object). 1.6.11 Generic Types A generic type, or generic typing, is the defining new feature in Java 5.0. It is a language feature that allows the author of a method or class to generalize the type of information with which the method or class will deal. The choice of the type of information is left to the invoker of the method or the declarer of an object. Thus, a generic method can be invoked in one part of a program to process an array of Integer objects, and the same method can be invoked in another part of the program to process an array of Double objects. In the case of a generic class, the type 73 of the class' data members and the parameters of the class' methods can be different for each object instance that the program declares. Generic Methods Consider the code shown below that outputs an array of Integer objects passed to it. Assuming this method exists, the invocation statement outputIntegerArray(ages); would be used to output the contents of the array ages that stores five Integers declared as However, it could not be used to output the contents of the array weights that stores a group of Doubles. For example, the following statements would result in a compile error because the method is expecting an Integer array and it is sent an array of Doubles. Using generics we could generalize the method so that it could be used to output any type of numeric object. The generic version of the output method, renamed outputNumericArray consistent with its generic nature, is shown below. Comparing the generic version of the method to the original version, we see that two simple changes have been made to make the method generic. Both of the changes are made to the method's signature (Line 1). Just before the return type specification, which for this method is void, a type placeholder is coded. 11 This indicates to the compiler that this
method will use a generic type T in its code, which it does in the parameter
list. (The parameter type Integer in the nongeneric version of the method
has been change to T in the generic version.) The actual type that T
represents will be the type of the argument that appears in an invocation of
the generic method. Thus, the following statements would effectively
74

replace T with the type Integer during the first invocation (Line 3), and T
would be replaced with the type Double during the second invocation
(Line 4).
Figure 1.22 The Person Class Shown in Figure 1.11 Written
Generically
One restriction placed on the use of generics is that within the code of
the method objects cannot be declared in the generic type (e.g., T
myObject = new T(); is not allowed).
Generic Classes
Type placeholders can be used in the definition of classes to defer the
decision as to the type of information that the class deals with until a
specific object is declared. Figure 1.22 presents the code of a class named
PersonGeneric, which is the code of the class Person (presented in Figure
1.11 modified to make it generic).
As shown on Line 1 of Figure 1.22 , to make a class generic we code
the generic placeholder(s) at the end of the class’ heading. In this case, two
placeholders are specified (separated by a comma), which indicates to the
compiler that this class’ code will use generic types T and E in its code.
These will be the types of the class’ two data members, and so Line 3 has
been modified to use type T, and Line 4 has been modified to use type E.
Finally, on Line 6, the types of the constructor’s parameters have been
changed to the generic types T and E.
With these modifications in place, we can declare an object and
75

specify the types of the age and weight data members for that object when
it is declared. For example, the syntax to declare an object tom whose age
and weight are both doubles would be:
To declare an object bill whose age and weight were they same types
as a Person object (integer and double), we would write:
It should be mentioned that the above declarations of the objects tom
and bill take advantage of the autoboxing feature added to the Java
language in Java 5.0 to simplify the coding examples. The object
declaration for Bill’s object indicates that placeholders T and E will be
references to Integer and Double objects respectively. Therefore, the
constructor on Line 6 of the PersonGeneric class (Figure 1.21 ) must be
sent a reference to objects of these types when the object bill is created.
However, the argument list used to create Bill’s object contains primitive
constants (i.e., 1 and 15.1), not references to Integer and Double objects.
To prevent a compile error, Java 5.0’s auto-boxing feature creates these
Integer and Double objects for us. Without the autoboxing feature, the
code to declare Bill’s object would have to be
because primitive types (e.g., int and double ) cannot be substituted
for type placeholders (e.g., T and E in the class PersonGeneric).
EXERCISES
Knowledge Exercises
1. What is data?
2. Define the term Data Structure.
3. What is a built-in data structure?
4. What three criteria are used to determine whether a data structure is
acceptable for a particular application?
76

5. What factors determine the cost of software?
6. It has been estimated that a program will consist of 300,000 lines of
code. If the burdened cost of a programmer’s efforts is $150 per hour,
determine the cost of the program.
7. Put the following terms in size order: node, field, data set.
8. An application is to be written that would allow students to find out
their GPA (a double) and their total number of credits (an integer), given
their student number (an integer).
(a) Draw a picture of the node used in this application. Include field
names and data types.
(b) Give the node width in bytes.
(c) Which field would be the key field?
9. You have been asked to write a nonvoid, one-parameter method to
access nodes in a data set. What will be the argument passed to your
method, and what will be the type of the returned value if the access
mode is:
(a) the key field mode?
(b) the node number mode?
10. Nodes are stored in a linear list. What node comes just before, and
just after, node 6?
11. Give the four basic operations performed on data structures, and
tell what each operation does.
12. The nodes in a data set are objects of type: Listing. Give the
signatures (the method headings that include types and parameters) for
the four basic operations if they are performed in the:
(a) Node number mode.
(b) Key field mode (assuming the key field is a String).
13. Define the terms procedural abstraction and data abstraction.
14. Define the term encapsulation.
15. What is the Java keyword that encapsulates data inside an object?
16. Three algorithms A, B, and C, are under consideration for the
Insert operation of a particular data set. Through an analysis of these
algorithms, their speed functions have been determined to be:
Algorithm A: 23n + 36n 2 ; Algorithm B: 6 + n log2 (n ) + n ;
77

Algorithm C: log2 n + 36n
2 .
(a) Calculate the value of these three functions when n , the number of
nodes in the data structure, is equal to 1,000,000.
(b) Using Big-O analysis, calculate the value of each function.
(c) Determine the percent difference between the values calculated in
parts (a) and (b) for each algorithm.
17. A 1000 element array is used to store integers in ascending order.
The array is to be searched using the binary search algorithm for the
two integers 5215 and 7282. How many elements of the array would be
examined by the algorithm to locate
(a) the integer 5215 stored in element 499?
(b) the integer 7282 stored in element 686?
18. What is the maximum and minimum number of times the search
loop will execute when searching through an array of 1,048,576
integers if the search algorithm is
(a) the binary search?
(b) the sequential search?
19. Half of the integers stored in the array data are positive, and half
are negative. Determine the absolute speed of the following algorithm,
assuming: the time to execute a memory access is 100 nanoseconds and
that all other operations (arithmetic, register accesses, etc.) take 10
nanoseconds.
20. Observations of the “traffic” on a data structure over a certain
period of time indicate that 500 Insert operations, 500 Delete
operations, 700 Fetch operations, and 200 Update operations were
performed on a data set. If Insert operations take 10 nanoseconds,
Delete operations take 250 nanoseconds, Fetch operations 200
nanoseconds, and Update operations take 300 nanoseconds, determine:
(a) the probability of performing a Fetch operation over the observation
period.
(b) the average speed, in nanoseconds, of the data structure over the
observation period.
78

21. Calculate the density of a data structure whose data set consists of
1,000,000 nodes, assuming the structure requires 1,000,000 bytes of
overhead to maintain itself, and:
(a) each node in the data set contains 2000 information bytes.
(b) each node in the data set contains 20 information bytes.
22. Repeat the above exercise assuming the overhead is 10 bytes per
node.
23. State the Java code to declare an array of 100 integers named ages.
24. State the Java code to declare an array of three Listing objects
named data that are initialized with the no-parameter constructor.
25. Draw a picture of the storage allocated in the previous exercise.
Assume the array is stored at location 20, and the three objects are
stored at locations 60, 70, and 50.
26. Two objects, objectA and objectB, are objects in the class Listing.
The object objectA is copied to objectB. How many objects exist after
the copy, if the copy is performed as:
(a) a deep copy?
(b) a shallow copy?
27. Of the two types of copies discussed in the previous exercise,
which one produces a clone (an exact duplicate of an existing object)?
28. Give the signature of a method named deepCopy, that clones an
object in the class Listing sent to it as a parameter and returns a
reference to the clone.
29. Give the Java invocation to clone the object objectA (using the
method discussed in the previous exercise) and store a reference to the
clone in the variable: newListing.
Programming Exercises
30. Write a generic method that could output three numbers of any
primitive numeric type sent to it, and include a driver program that
demonstrates it functions properly.
31. Write a Java program to accept an item’s name and price from the
user and output them to the console.
32. Write a Java program to accept the names of three items and their
prices from the user and outputs them and the average price to a
79

message box.
33. Write a Java program to accept three item names and prices, and
output them. Also, output the average price if one of the items is named
Peas (not case sensitive) otherwise output: “no average output”.
34. Write a Java program to accept an unlimited number of item names
and prices, and output them. In addition, output the average price if one
of the items is named Peas (not case sensitive) otherwise output: “no
average output”. The inputs will be terminated by a sentinel price of
−1. (Do not include the −1 in the average.)
35. Write a Java program to accept a given number of item names and
prices and then output them in the reverse order in which they were
input. In addition, output the average price if one of the items is named
Peas (not case sensitive) otherwise output: “no average output”. (The
first user input will be the number of items to process.)
36. Add a static method to the program described in one of the two
previous exercises that that outputs “Welcome To St. Joseph’s College”
before the items and prices are output. The method main should invoke
this method.
37. Add a method to the program described in the previous exercise
that accepts and returns a single item name that is input by the user.
The prompt presented to the user should be passed to the method as an
argument.
38. Modify the program described in one of the four previous
exercises so that the average price is calculated by a separate method
and returned to the invoker.
39. Modify the program in the previous example so that after it
produces its output, it asks the users if they would like to process
another group of items. The program should terminate when the user
enters “no” (not case sensitive). Otherwise, it continues processing
names and ages.
40. Write a class named Listing that contains two data members, name
(a String) and number (an integer). The class should have a no-
parameter constructor, a two-parameter constructor, and a toString
method to facilitate the output of the values of the data members. It
should also contain set and get methods for each data member. Write a
driver program that is progressively developed (i.e., expands as
methods are added to the class Listing) that demonstrates each method
80

in the class functions properly.
41. Write the class whose diagram is shown below and write a driver
program (progressively developed) to test the class (verify that all the
methods function properly). The default (no-parameter constructor)
should initialize the String data member name to “ ” and numeric data
member to zero. The input method should allow the user to input the
values of an object’s data members.
42. Write an application that declares an array of three Listings
(defined in one of the two previous examples) whose contents are input
by the user. After being input, the listings should be output in reverse
order.
43. Draw the memory allocated to the Listing objects described in the
previous example.
44. Code the definition of the class whose diagram follows. The
constructors should allocate the array (data) of Listing object
references sized to either 100 elements (the no-parameter constructor)
or the number of elements specified by the argument sent to the one-
parameter constructor. The method, addListing, will add a new Listing
object to the array at index next. The showAll method will output the
values of the data members of all the Listing objects. Write a
progressively developed driver program to demonstrate that the class
functions properly. The class listing is described in Exercise 41.
81

45. Write a program to count the number of times the search loop of
the binary search algorithm executes when searching for an integer
contained in a one-dimensional array of sorted integers in the range 1
to 65,000. Use the Java method random in the Math class to generate
the integer, targetInt, to be located (i.e., targetInt = 1 + (int)
Math.random()*65,000). The program should accept an input, n, the
number of integers to be located, and then output the average number
of times the loop executed after locating the n randomly generated
target integers. For comparative purposes, also output the log2 65,000.
46. Write a class named DataStructure that contains a no-parameter
constructor and the four basic operation methods inset, fetch, delete
and update in the node number mode. The operation methods should
use the standard signatures presented in this chapter. Assume that the
nodeType is an object in the class Listing. Each method, when
invoked, should just output a message to the console that it was
invoked. The fetch method should return a null reference to Listing
object. The class Listing can be the class described in Exercise 41 or
coded as an empty class: i.e., public class Listing {}. If you use the key
field mode signatures, assume a String key type. Write an application
that demonstrates that your class is coded properly.
1 Burdened labor rates are the cost of one programmer working one hour and include
salary, profit, and all associated expenses (e.g., employee benefits, rent, electric, heat,
supplies, etc.).
2 Returning a Boolean value is simple, but it does not allow the client to determine
which of several errors has occurred. Alternatives would either be to return an integer, or for
the operation method to throw an exception that contains error information.
3 Most often the independent variable, n , in the speed function is the number of nodes
in the data structure, because the number of nodes in the structure usually determines how
many instructions the algorithm executes.
4 For simplicity, it will be assumed that all of the data are resident in the same part of
the system’s memory hierarchy and, therefore, one memory fetch speed can be used for all
82

memory fetch instructions. In fact, data resident in cache memory can be fetched faster than
data stored in RAM memory, which can be fetched faster than data stored in other parts of the
memory hierarchy.
5 In other programming languages, reference variables are often called pointers.
6 Other Java access modifiers are private , protected , and package .
7 Constructor methods are used to construct new objects.
8 In the special case of a String object we can declare objects using a third grammar:
String name = “bob”;
9 An exception is a static method, declared using the keywords: public static , which is
invoked by mentioning the class name followed by the method name (e.g., Math.sqrt(9)).
10 Primitive variables store integers, doubles, etc.
11 The name of the generic placeholder (i.e., T) is arbitrary. Any valid Java identifier
can be used.
83

CHAPTER 2
Array-Based Structures
OBJECTIVES
The objectives of this chapter are to familiarize the student with the
use of the built-in structure array, the implementation techniques common
to all data structures, and the use of these techniques to develop three
array-based structures. More specifically, the student will be able to
Understand the memory-model programming languages used to
implement arrays, and the advantages and disadvantages of this
representation.
Explain which of the four basic operations are allowed on the built-
in structure array and the Java syntax for using these operations.
Understand the techniques for designing data structures that make
them application independent, and understand the techniques for
designing applications that allow them to easily change the data
structures they use to store their data.
Understand the advantages and disadvantages of array-based
structures.
Understand and be able to quantify the performance differences of
three array-based structures, and the role of sorting in the performance
of these structures.
Fully implement an array-based structure in the key field mode.
Explain the error conditions associated with the four basic
operations, and be able to detect them in a data structure
implementation.
Understand the implementation techniques used to fully encapsulate
a data structure and be able to explain the underlying memory model.
Expand an array-based structure at run-time, and understand the
84

performance penalty associated with that expansion.
Convert a data structure to a generic implementation using the
generic features of Java, and understand how to declare a homogeneous
or a heterogeneous object in a generic data structure class.
Understand Java interfaces and their role in generic data structures,
and be able to write and implement an interface.
Develop an application that declares objects in Java’s ArrayList
class, and understand the advantages and disadvantages of the class.
2.1 The Built-in Structure Array
Early programming languages were, among other things, designed to
evaluate mathematical formulas. Since subscripted variables are used
extensively in mathematical formulas, these languages provided a built-in
data structure for storing them called an array. An array is a data structure
rooted in the mathematical concept of subscripted variables, with single
subscripted variables, e.g., x 1 , x 2 ,…, x i , modeled by one-dimensional
arrays. Virtually all modern programming languages continue to include
the structure array as part of their language standard.
It was anticipated that the structure would be widely used, therefore,
its designers recognized the need for both high-speed and low-memory
overhead in its implementation. From a data structures viewpoint, these
two objectives are usually mutually exclusive. They could not have been
achieved without some compromises, or restrictions, placed on the
structure. A group of compromises was suggested by the characteristics of
subscripted variables themselves and the way in which mathematicians use
them:
• All of the values stored in the variables are of the same type (e.g.,
integer, real, etc.).
• Each variable is distinguished by a unique ordinal subscript.
• There is a minimum and maximum value of the subscript.
Since each value stored in the variables is of the same type, the data
structure array could be restricted to a homogeneous structure. Only node
number access would be supported, since the unique ordinal subscript
could be thought of as a node number. In addition, since the structure
would have a specified number of members (within the range of the
minimum and maximum value of the subscript) and therefore not grow or
85

shrink, Insert and Delete operations would not be allowed.
The designers of the structure imposed two other restrictions on the
way arrays would be stored in memory. First, they restricted the storage of
the array members (nodes) to contiguous memory.
Figure 2.1 The Storage of Five Nodes in the Array x
Furthermore, within this contiguous memory, the nodes would be
stored sequentially based on the node number. 1 Figure 2.1 illustrates the
storage of a five-element array, x , consistent with these restrictions. Each
element of the array x is assumed to contain 10 bytes.
Storing arrays in this way permits rapid access to the nodes since the
byte address of a node can be expressed as a simple mathematical function
of the node number, N . Specifically, the function is: 2
where
A o is the byte address of the first node, called the base address,
w is the node width (number of bytes per node), and
N is the number of the node, starting from N = 0.
The function: A N = A o + N * w, called the linear list access function
, can be rapidly evaluated by the CPU since it involves only two arithmetic
operations. 3 The ability to calculate the location of an element of the
array from its node number (for a given base address and node width) is
what gives the structure array its speed. In addition, since the only
overhead associated with structure is the storage for the values of A o and
w, the design goal of low overhead was also achieved.
Returning to the mathematical concept of subscripted variables, the
only remaining issue was which subscripted variable would be stored in
86

node 0, node 1, etc. Figure 2.2 shows a typical assignment between the
subscript and the node number in which x 0 is stored in node 0, x 1 in node
1, etc.
Figure 2.2 The Mapping of Subscripts to Node Numbers for the
Array x
The relationship between subscript number and node number is called
a mapping function , and for the assignments depicted in Figure 2.2 it is: 4
N = i
where
N is the node number, and
i is the variable’s subscript.
Combining this mapping function with the linear list access function,
we obtain a function to calculate the address of the i th subscripted
variable as
As a check of our mapping function, let’s calculate the address of the
variable x 2 . Here i = 2, and assuming the base address is A o = 100 and
the node width is w = 10 (as depicted in Figure 2.2 ), we calculate the
address of the variable to be
which is correct, as verified by inspecting the figure.
The inconvenience of typing subscripts demanded a friendlier syntax
for arrays. Therefore, subscripts were replaced with indices enclosed in
either parentheses or brackets. Java uses brackets.
87

Figure 2.3 The Syntax of Array Indices in Java
Thus, in Java the variable x i is coded as array element x[i]. The array
notation for subscripted variables is shown in Figure 2.3 .
Consistent with the restrictions placed on them by their designers,
only two of the four basic operations can be performed on the data
structure array: Fetch and Update. The syntax of the Fetch operation is
simply to code the name of the array element. Thus, to fetch the value of
x[2] and output it, we code System.out.println(x[2]). The syntax of the
Update operation is to place the name of the element of the array on the
left side of an assignment operator. Therefore, to update the value of x[1]
to 24, we write x[1] = 24.
All array indices in Java start at zero. Some programming languages
allow the minimum index to start at any integer, i o . When this is the case,
the mapping function for a one-dimensional array becomes:
2.1.1 Multidimensional Arrays
Most programming languages also include the ability to store
multisubscripted variables, (e.g., x ijk ). These variables are assigned node
numbers in one of two ways: row major order or column major order.
Consider a variable with two subscripts, x ij , with 0 <= i <= 1, and 0 <= j <= 2. Figure 2.4 illustrates how each scheme assigns variables to node numbers. Referring to Figure 2.4 , the mapping functions used to calculate the node number, N, given the subscripts i and j are: 5 88 Figure 2.4 Comparison of Row Major and Column Major Orders where i and j are the subscripts of the variable, j max is the maximum value of the index j , and i max is the maximum value of the index i . Substituting these expressions for node number into the linear list access function, A N = A o + N * w, we obtain the address of x ij to be Once again, these functions can be rapidly evaluated since they involve only basic arithmetic operations. To verify our mapping functions, let us calculate the row major and column major addresses of x 02 shown in Figure 2.4 . The figure assumes the maximum values of i and j are 1 and 2 respectively, A o = 100, and the node width is w = 10. Under these conditions, we calculate which agrees with the addresses shown in Figure 2.4 . The Java syntax for the variable x ij is x[i][j] where the indices i and j always start at zero. Usually programmers visualize two-dimensional 89 arrays arranged in rows and columns, with i representing the row number and j representing the column number. Since the maximum number of rows (n r ) and columns (n c ) of this visualization is always one more than the maximum subscripts of the variables i max and j max , respectively, the address of the array element x[i][j] can be expressed as where n r is the number of rows in the array, and n c is the number of columns in the array. Since arrays are built-in structures in modern programming languages, they are treated as an abstract data structure by most programmers. This means that programmers need not know the mapping functions. The only exception to this would be the programmers who write compilers. They need to know all the details of the structure in order to include the mapping function into the compiler's code. These details would include where the base address and number of bytes per node will be stored, whether the nodes will be stored in ascending or descending order, and whether row or column major order will be used for two-dimensional arrays. 2.2 Programmer-Defined Array Structures By design, the built-in data structure array is fast and its memory overhead is low. However, also by design, its operations are restricted to Fetch and Update. Insert and Delete operations are not allowed and the key field access mode is not supported. These restrictions are acceptable when we are using arrays to emulate subscripted variables, but they are not consistent with requirements of most applications that process data sets. Most applications access the data they process in the key field mode (e.g., fetch the telephone listing of Al Jones ), and use the Insert and Delete operations to add nodes to, or eliminate nodes from, the data set (e.g., Mr. Jones has moved into, or out of, town). In addition, nodes in a data set are typically made up of several fields of information rather than a single value. Despite their limitations, many data structures that support the four basic operations and permit access in the key field mode utilize arrays as 90 their underlying storage scheme. The speed and low overhead of arrays make them an attractive foundation for higher level structures to build upon. For example, arrays can be used to store a multifield data set by declaring the array to be an array of node objects, 6 each with multiple data members. In addition, key field mode access can be implemented on array- based structures if the class that defines the node objects has a method to fetch the key from a node object. Although not suited for all applications, data structures that build upon, or enhance, the structure array are in wide use. In the remainder of this chapter, we'll examine three of these array- based data structures: • The Unsorted array. • The Sorted array. • The Unsorted-Optimized array. As we will see, all of these data structures • Can store data sets of multifield nodes. • Are accessed in the key field mode. • Permit all four basic operations to be performed on the data set. • Use an array of objects to store the data set. • Are fully encapsulated. What distinguishes one from the other is that each one uses a unique approach to the basic operation algorithms. The study of these three structures is valuable from several points of view. First, it will demonstrate the use of our calculation and trade-off techniques to uncover the strengths and weaknesses of each structure. Second, the study of these structures will lead us to a discussion of the relative merits of two very basic key field search algorithms: the Sequential and Binary Search algorithms. Finally, the simplicity of these structures will allow us to focus on the implementation techniques common to all the data structures. After we have studied and compared the three, we will fully implement the best performing data structure as the first of the classic data structures presented in the text. The algorithms presented for the three data structures will not include error checking (e.g., the node to be operated on is not in the structure or there is insufficient memory to perform an Insert operation) since it does not affect the relative performance of the three structures and its exclusion will simplify the development of their pseudocode. The ability to check for errors will be added to the pseudocode of the best performing structure before it is implemented. 91 2.2.1 Unsorted Array This array-based structure, accessed in the key field mode, uses an array of objects to store the data set. The array (named data) is sized to the maximum number of nodes that will reside in the structure, n . An additional integer type memory cell, next, is used to store the index of the array where the next insert will be performed. Initially, all the elements of the array are set to null , and next is set to zero. Figure 2.5 shows the structure in its initialized state. To perform an Insert operation, a new node object is allocated with its contents initialized to the contents of the node to be inserted, and then a reference to it is placed into the array at data[next]. Then, next is incremented by one to prepare for the next insert. Thus, assuming we are to insert the node newNode into the structure and that a method to perform a deep copy of a node (named deepCopy) exists, the algorithm is: Figure 2.5 The Unsorted Array Structure in Its Initialized State Figure 2.6 shows the structure after five nodes have been inserted into the initialized structure. The node with key field Phil was inserted first, then Bill, followed by Carol, then Vick, and finally Mike. 92 The Delete operation of this structure uses a sequential search to locate the node to be deleted. The search begins at element zero of the array and terminates when the node with the given key is located. Once the node is located, it could be eliminated from the structure in one of two ways: 1. Set the array element that stores the reference to the node to null . 2. Move all the node references below the deleted node up one element in the array and then decrement the memory cell next. The first technique would be the faster approach because it does not spend time moving up the node references; however, our Insert algorithm would never reuse the array element associated with the deleted node, since it always performs an insert at data[next], which is already below this element. In effect, every time a node was deleted from the structure, the maximum capacity of the structure would be reduced by one node, because the vacated element of the array would never be used again. Therefore, the second technique: Figure 2.6 The Unsorted Array Structure after Five Nodes Are Inserted Move all the node references below it up one element in the array, and then decrement the memory cell next, which is considerably slower, will be used in our Delete algorithm. The idea of reclaiming the storage associated with a deleted node (in data structures jargon) is called garbage collection and all data structures that 93 support the Delete operation must provide some form of garbage collection. The Delete algorithm (that includes garbage collection) is presented below. It assumes the node with key field contents targetKey is to be deleted and that the pseudocode data[i].key refers to the contents of the key field of the node referenced by element i of the array. Figure 2.7 Unsorted Array Structure after Carol's Node Is Deleted Figure 2.7 shows the data structure after the deletion of the node with key field Carol. Comparing it to Figure 2.6 , we see that the references to Vick's and Mike's nodes have each been moved up one element in the 94 array. Carol's reference, which was stored in data[2], has been overwritten with the contents of data[3] (Vick's reference 44). Similarly, data[3] has been overwritten with the contents of data[4]. Now examine the Fetch algorithm. To perform a fetch of the node with the key targetKey, the Fetch algorithm searches sequentially down the array starting at element zero. Once the object with the given key is found, we simply return a deep copy of it. The pseudocode of the Fetch algorithm is: Figure 2.8 Fetching a Deep Copy of Mike's Node from the Unsorted Array Structure Figure 2.8 illustrates the deep copy operation that takes place after a requested node (in this case Mike's node) has been located using the sequential search coded as the while loop in the fetch algorithm. 95 For the purposes of brevity, we will assume that the algorithm to update the node with the key field targetKey to the contents of the node newNode uses the Delete and Insert algorithms, previously discussed. Under this assumption, the Update algorithm becomes: A more efficient implementation of the Update algorithm would be one in which the actual code to access the node and change its fields would be included in the update method because this approach saves the time required to invoke the delete and insert methods. This implementation is left as an exercise for the student. Speed of the Structure To analyze the speed of the structure, we will perform a Big-O analysis to determine the approximate speed as n (the number of nodes stored in the structure) gets large. Since the Big-O analysis is an approximate technique and the time to perform a memory access instruction is typically considerably longer than the time to perform a non- access instruction, only memory access instructions will be included in our analysis. Consistent with this approach, instructions inside of loops that repeatedly access the same memory cell (e.g., Line 4 of the Delete algorithm) will also not be included in our speed analysis since modern compilers store these variables in CPU registers. Assuming that the CPU has sufficient register space to allow this, the contents of these variables can be used without accessing memory. Examining the Insert algorithm, its two lines perform three memory accesses: one to fetch the variable next, one to write the object's address into data[next], and one to overwrite the variable next. 7 Fortunately, these two lines are only executed once per Fetch operation regardless of how many nodes are in the structure. Therefore, three memory accesses are required per Insert operation, and the dominant (and only) term in the Insert operation's speed function is 3, which is O(1). The Delete algorithm uses a sequential search (Lines 3–5) to locate a node, given its key. Sometimes the node will be at index 0, and line 3 will execute only once. Other times the node will be at index n − 1, and line 3 will execute n times. Since all locations are equally probable (on the 96 average), the node to be deleted will be in the middle of the array, and the loop will execute an average of approximately n / 2 times. Once found, the node is deleted by moving up all of the references in the array below the deleted node (Lines 7 and 8). Assuming, on the average, the reference to the deleted node is located in the middle of the array, this loop will also execute an average of approximately n / 2 times. Within the two loops, Line 3 requires two 8 memory accesses and Line 8 requires two memory accesses. Therefore, the dominant term of the speed equation is (2 + 2) (n / 2) = 2n , which is O(n ). The Fetch algorithm also uses a sequential search (Lines 3–5) to locate the node to be fetched. Assuming the fetched node is (on the average) referenced from the middle of the array, the search loop executes approximately n / 2 times. Line 3 requires two memory accesses inside the search loop to access a node and return its key. Line 7 of the algorithm contains a memory access instruction, but it is only executed once, so it does not contribute to the dominant term in the speed equation. Therefore, the two memory accesses inside the loop result in a dominant speed equation term of 2(n / 2) = n , which is O(n ). Table 2.1 summarizes the number of memory accesses (associated with the dominant term in the speed equation) to perform the basic operations on the Unsorted Array structure. The Update operation speed was calculated as the sum of the Delete and Insert speeds. We will assume the average speed of this structure is too slow for our application and consider another array-based structure: the Sorted Array. 97 Figure 2.9 Sorted Array Structure Containing Five Nodes 2.2.2 Sorted Array Examining the results presented in Table 2.1 , we see that the Delete, Fetch, and Update algorithms are slow, O(n). To improve the speed of these operations, we can focus our attention on the Delete and Fetch algorithms since an improvement to the Delete algorithm will produce a corresponding improvement in the Update algorithm. Both the Delete and the Fetch operations use a sequential search access algorithm, which is why they are slow. A faster access algorithm will speed up these two operations (as well as the Update operation). In Chapter 1 , we examined the Binary Search algorithm and calculated its speed to be O(1og2 n ) which, as shown in Figure 1.9 , is much faster than O(n ). However, the algorithm assumes the information in the array being searched is arranged in sorted order. Therefore, as the name of this structure (Sorted Array) implies, in order to take advantage of the speed of the Binary Search algorithm, it will store the nodes in sorted order. Naturally this will require modifications to the Insert algorithm of the previous structure. Before discussing these modifications, let us first turn our attention to the Fetch and Delete algorithms. Figure 2.9 shows the data structure after the nodes with key fields Phil, Bill, Carol, Vick, and Mike have been inserted into the structure. 98 Since the nodes are stored in sorted order based on the contents of their key field, the Fetch algorithm can use the Binary Search algorithm to locate a node given its key. Once located, it returns a deep copy of the node, just as the unsorted structure did. The new Fetch algorithm becomes: Lines 2–11 are the Binary Search algorithm. They set the variable i to the index of the array element that references the node to be fetched. Line 12 returns a deep copy of this node. Next, let's consider the Delete algorithm, which also uses the Binary Search algorithm. Once the node to be deleted is found, as in the case of the previous structure, all the nodes below it must be moved up to reclaim the unused storage. Figure 2.10 show the structure after Carol's node has been deleted from the data set depicted in Figure 2.9 . The Delete algorithm is: 99 Figure 2.10 Sorted Array Structure after Carol's Node Is Deleted Finally, let's consider the revised Insert algorithm. It was not our intent to modify this algorithm, since it was very fast. However, the new Delete and Fetch algorithms use a binary search to locate a node, and so the nodes must be stored in sorted order based on the key field. A newly inserted node can no longer be inserted into the array as the last node (at index next). Rather, it must be placed in its correct sorted position. 100 Figure 2.11 Sorted Array Structure after Ron's Node Is Inserted To find the “correct location” for an inserted node, the Insert algorithm performs a binary search. It continues its search until it finds two adjacent keys that “bracket” the new node's key. Then, it moves all the nodes below these two nodes and the larger of the two nodes down one element to “open up” a spot for the new node. Finally, the contents of the node to be inserted are deep copied into a newly created node, and then a reference to that node is placed in the array element that has been opened up. Figure 2.11 shows the structure after Ron's node is inserted into it. Compare it to Figure 2.10 to visualize action of the algorithm. The pseudocode of the algorithm follows. Since we are interested only in the performance of the algorithm, it ignores the special cases of an Insert operation being performed when the structure contains one or two nodes, which are not significant from a speed complexity viewpoint. 101 Speed of the Structure To analyze the speed of this structure, we will perform a Big-O analysis to determine the approximate speed of its operation algorithms as n (the number of nodes stored in the structure) gets large. Again, we will only consider memory access instructions, which typically take considerably longer to execute than non-access instructions, and will ignore instructions in loops that repeatedly access the same memory cell. Beginning with the Fetch algorithm, Lines 5–11 represents a binary search loop. As discussed in Chapter 1 , this executes (in the worst case) approximately log2 n times. Lines 5 and 6 require two memory accesses to fetch data[i] and the key field of the node it references. Therefore, the dominant term in this loop's speed equation is 2log2 n , which is O(log2 n ). Turning our attention to the Delete algorithm, Lines 5–11 are the same binary search loop as in the Fetch algorithm, so their term in the speed equation is again 2log2 n . The loop on Lines 13–15 reclaims the vacated element of the array by moving the node references up in a sequential manner. As such, it executes an average of approximately n / 2 times. 10 Since Line 14 in this loop requires two memory accesses, the dominant term in this loop's speed equation is 2 * n / 2, which is equal to n . The speed function for this operation is therefore 4log2 n + n , with n being the more dominant term (see Figure 1.9 ). Therefore, the speed 102 function for the Insert algorithm is O(n ). Lines 6–12 of the Insert algorithm use the same binary search loop as in the Fetch algorithm except Line 6 performs two additional memory accesses to fetch data[i - 1] and the key of the node it references. Therefore, Lines 6–12's term in the speed equation is 4 log2 n . The loop which opens up a “spot” for the new node (Lines 14–16) moves the node references down in a sequential manner. As such, it executes an average of approximately n / 2 times. Since Line 15 in this loop requires two memory accesses, the speed equation term for this loop is 2n / 2, which is equal to n . The speed function for this operation is therefore 4log2 n +n , with n being the more dominant term (see Figure 1.9 ). Therefore, the speed function for the Insert algorithm is O(n ). Table 2.2 summarizes the number of memory accesses needed to perform the basic operations on the Sorted Array structure and compares it to the previously discussed structure. Comparing the speed of the operations presented in Tables 2.1 and 2.2 , we see that the Binary Search algorithm used in the Sorted Array structure considerably reduced the number of memory access instructions required to perform a Fetch operation. However, the need to maintain the array in sorted order minimized the improvement on the Delete algorithm and made the Insert operation less efficient. Because of this, the average speed of this structure when populated with 10,000,000 nodes (presented in the rightmost column of Table 2.2 ) is approximately the same. This realization leads us to consider our third array-based structure: the Unsorted-Optimized Array. 2.2.3 Unsorted-Optimized Array 103 As the name of this structure implies, it is an optimized version of the Unsorted Array structure. Examining the data presented in Table 2.2 , the Unsorted structure's Insert operation is the fastest of all the operation algorithms studied so far. Therefore, the new structure will retain this Insert algorithm. The Delete and Fetch algorithms of the new structure will be optimized versions of the Unsorted structure's Delete and Fetch algorithms, optimized to improve their speed. Both the Delete and Fetch algorithms of the Unsorted Array structure use a sequential search to locate a node, given its key field contents. Since this search starts at the top of the array (element zero) it would execute quickly if the nodes accessed were always referenced by one of the first few elements of the array. This is an unrealistic situation if all nodes in the structure have equal probability of being accessed. However, most often there are some “favorite” nodes that are operated on much more frequently than the others. For example, the phone number of a town's most popular bakery, Maggie's Delights, is accessed much more frequently than the phone number of the Cockroach Cafe. To take advantage of this characteristic of many data sets, we modify the Unsorted Fetch algorithm to position the references to the most accessed nodes toward the top of the array. This modification is surprisingly simple. After a node is fetched, the position of its reference is swapped with the node reference just above it. Thus, every time a node is accessed, it moves up one position in the array. Eventually, the nodes with the highest probability of being accessed will be positioned at the beginning of the array. Figure 2.6 shows the data structure after five nodes (Phil, Bill, Carol, Vick, and finally Mike) have been inserted into it using the Unsorted structure's Insert algorithm. Next, let's assume that Carol's node is fetched. Figure 2.12 shows the structure after the Fetch operation is complete. The reference to Carol's node has been moved up one element in the array, switched with the reference to Bill's node. A subsequent (sequential) search for Carol's node will be slightly quicker. The modified Fetch algorithm is: 104 Figure 2.12 Unsorted-Optimized Array after Carol's Node Is Fetched Lines 8–10 of the algorithm is the additional code that moves the node reference up one element in the array unless the node being fetched is referenced by element zero (Line 7). The Unsorted Delete algorithm will also be modified to improve its speed. It will still perform a sequential search to locate the node to be deleted. However, instead of moving all the node's references up to reclaim the deleted node's array element, we will simply move the last node reference into the deleted node's position. Since the nodes are not 105 stored in sorted order, any node can be moved into this position. The modified Delete algorithm is: Figure 2.13 Unsorted-Optimized Array after Carol's Node Is Deleted Figures 2.12 and 2.13 , respectively, show the data structure before and after Carol's node is deleted. After the deletion, the reference to Mike's node has been written into the element of the array that referenced Carol's node. Speed of the Structure We will now examine the speed of the Unsorted-Optimized structure. The Insert algorithm is the same as the Unsorted structure, and therefore the dominant term in its speed equation (as presented in Table 2.1 ) is 3, which is O(1). 106 When all the nodes in the structure have an equal probability of being accessed, the speed of the new Fetch operation will be the same as that of the Unsorted structure's Fetch operation, n . But, whenever some nodes are more frequently accessed than others, the number of memory accesses to perform a Fetch operation will be less than n . In the extreme case when the same node is fetched all the time, eventually its reference will find its way to the first element of the array, and only two memory accesses will be required to fetch it. Therefore, the number of memory accesses for our new fetch algorithm is <= n , which is < O(n ). In the modified Delete algorithm, we eliminated the need to reposition half of the nodes in the structure in order to reclaim the storage of the deleted node. Therefore, only the search loop (Lines 3–5), which executes n / 2 times, contributes to the speed equation. Line 3 of the loop requires two memory accesses, so the speed equation is 2 * n / 2 (equal to n ), which is O(n ). However, if the deleted nodes are those most likely to be fetched, the search loop will execute less than n / 2 times, and the speed of the algorithm will be < n . Table 2.3 presents the speed of this structure and, for comparative purposes, the speed of the other two array-based structures. As indicated in the rightmost column of the table, even when all operations on the data set are equally probable, the average speed of the Unsorted-Optimized structure is faster than the other two structures, and its speed advantage increases when some nodes have a higher fetch frequency than others. However, from a Big-O analysis viewpoint, all three structures are equivalent (all O(n )). The Unsorted-Optimized structure has the best- performing Insert, Delete, and Update algorithms. However, the Sorted Array Structure has the best Fetch algorithm. Therefore, if, after the data set is initially inserted into the data structure, the only operation performed is Fetch, then the preferred structure is the Sorted structure. 107 Having determined the speed of these three array-based structures, we will now turn our attention to their storage requirements. Density of the Structure Density is the measure of how efficiently a data structure utilizes storage. In Chapter 1 , it was defined as The information bytes for all three array structures is simply the product of the number of nodes, n , and the number of bytes per node, w (n * w). The total bytes allocated to the structure is the sum of the information bytes and the structure's overhead. The overhead of these structures is the array of n reference variables that point to the node objects, plus the integer variable next. In Java, reference variables and integers occupy 4 bytes. Therefore, the overhead is 4 * n + 4. Thus, the density of these structures, DA , can be expressed as: where w is the information bytes per node (called the node width), and n is the number of nodes in the structure. As n increases, the term 4 / (w * n ) in the denominator tends toward zero and can be neglected. 13 Therefore, the density of our three array- based structures can be expressed as 108 Figure 2.14 presents a graph of 1 / (1 + 4/w) vs. node width, w. The figure demonstrates that good densities (0.80 or higher) are achieved for an array-based structure whenever the number of bytes in a node is greater than sixteen. Figure 2.14 Density Variation of Array-Based Structures with Node Width Table 2.4 summarizes the overall performance (speed and density) of the fastest of our array-based structures: the Unsorted-Optimized array. It is the first of the classic data structures we will implement in this text. Before implementing it, however, we will modify the pseudocode of its basic operations to include error checking. Error checking was not considered when we developed the pseudocode of the three array-based structures because it does not affect the relative performance of these structures, and its exclusion simplified their pseudocode. 2.2.4 Error Checking Considering the four basic operation algorithms, there are three errors that can occur: • During an Insert, the structure is full (every element of the array contains a reference to a node object). • During a Fetch, Delete, or Update operation, the node to be operated on is not in the structure. • During an Insert operation, there is insufficient memory for the deep copy of the client's node. 109 The ability to deal with the first two of these errors will be incorporated in the pseudocode of the Unsorted-Optimized Array operation algorithms. The third error will be considered during the implementation of the structure, since the detection of this error is implementation-language specific. In the case of the Fetch, Delete, and Update algorithms, a Boolean value of true will be returned if the operation completes without an error. Otherwise, the operations will return a value of false . As written, the Fetch algorithm returns a reference to the deep copy of the client's node inserted into the structure. Consistent with this, the modified code will return a null reference if there is insufficient array (or system) memory space to complete its operation. The pseudocode of the Insert algorithm expanded to include error checking follows. The changes to the original version are shaded and assume that the size of the array is stored in the variable size. As indicated on Line 1 of the algorithm, when the structure is full, next is equal to size. In this case, the algorithm terminates on Line 2 and returns a value of false . Otherwise, it performs the insert and returns a value of true on Line 5. The following pseudocode is the Fetch algorithm expanded to include error checking. The changes to the original version are shaded. They assume that the size of the array is stored in the variable size. The Sequential Search performed on Lines 3–5 of the algorithm uses the variable i to index through the array. Since the variable next stores the index just beyond the last used index of the array, the condition for a node not present in the structure is when i equals next. Thus, the loop condition 110 on Line 3 has been modified to continue the search only while i is less than next. After the loop terminates, if the node has not been found, i will equal next. Lines 6–7 have been added to detect this, terminate the algorithm, and return a value of null . The pseudocode of the Delete algorithm expanded to include error checking follows. The modifications to this algorithm are the same as the modifications to the Fetch algorithm except that Line 7 returns false instead of null when a node is not found. In addition, Line 11 has been added to the algorithm to return a value of true after a deletion is performed. To include error checking in the Update algorithm, the errors returned from the invocations of the Delete and Insert operations are tested (Lines 1 111 and 3 of the pseudocode below). If either operation returns a value of false , the Update algorithm returns false (Lines 2 and 4). Otherwise, a value of true is returned on Line 6. 2.3 Implementation of the Unsorted-Optimized Array Structure The Unsorted-Optimized data structure will be implemented in this section. 14 However, since the code of a complete implementation of any data structure can be a bit overwhelming, our first implementation of this structure will be a minimal implementation tied to a particular application (our telephone book listing problem). Once an understanding of this baseline implementation is gained, it will be easier to understand the enhancements that bring it to a full implementation. When fully implemented, the data structure will be coded in a generic way so that it can be used to store nodes particular to any application, and can be easily integrated into these applications. In addition, several utility methods will be added to the baseline implementation to make the structure easier for the client to use. The data in both the baseline and in the full implementation will be encapsulated within the data structure. The only way the client will be able to access the data set will be through the structure's methods. Another feature that the baseline and full implementations will share is that the code that defines the nodes to be stored in the structure will not be part of the class that implements the data structure. Rather, the node definition will be implemented as a separate class. This design feature will be utilized in all implementations presented in this text. Separating the node definition from the data structure class is a first step toward generics, because the data structure is not tied to a particular application's node structure. 112 2.3.1 Baseline Implementation We will use a telephone information directory application to demonstrate the functionality and use of our implementations of the Unsorted-Optimized Array structure. In this application, each telephone directory listing will have three fields of String information as shown in Figure 2.15 , with the Name field designated as the key field. Typically, a node definition class contains the data declarations for the node fields and the methods that operate on these fields. Consistent with the idea of a baseline implementation, our node class will initially contain a minimum number of methods. Aside from a constructor, normally a toString method is provided to facilitate the output of a node. In addition to these, the pseudocode of the four basic operations requires two other methods, one to perform a deep copy and the other to determine if a node's key field is equal to the contents of a given key. Finally, a set method will be provided to demonstrate the encapsulation of the nodes inside the data structure. In summary, the following five methods will be coded in the node definition class: Figure 2.15 Telephone Directory Node • A three-parameter constructor to create a node. • A toString method to return the annotated contents of a node. • A deepCopy method (used by the data structure to maintain encapsulation). • A method to compare a given key to the contents of the name field of a node. • A method to set the value of a node's address field (used as a pedagogical tool to demonstrate data encapsulation). Node Definition Class The code of the class that defines our telephone directory node, named Listing, depicted in Figure 2.15 is given in Figure 2.16 . 113 Figure 2.16 The Class Listing that Defines a Telephone Listing Node Lines 2–4 declare the fields of a node. The data access is private to encapsulate the data members. By specifying private access, only the code of the methods in this class can directly access the data members. All other attempts to directly access the data, either from the code of the data structure class or the application that uses the data structure, will result in a compile error. Lines 5–9 are the code of a three-parameter constructor. When an object is created in this class, the client code must specify the name, address, and phone number of the listing: e.g., Listing bill = new Listing(“Bill”, “1st Avenue”, “453 3434”;). Lines 10–14 are the code of the toString method that returns a string containing the annotated contents of a listing. The deep copy method is coded on Lines 15–18. It copies all the data (from the object that invokes the method) into a newly created object and returns the address of the new object (Line 17). Line 16 does most of the work. First, it creates a new Listing reference variable clone. Then the Java operator new creates a new Listing object, and invokes the three-parameter constructor. The values of the data members of the object that invoked the method are sent to the constructor (as arguments), and the constructor sets them into the data members of the newly created object (Lines 6–8). 114 Finally, the address of the new Listing object, returned from the operator new , is set into the variable clone. If there is insufficient storage available to create the new Listing object, the new operator returns null . Lines 19–21 are a method to determine if the key field (name) of a telephone listing is equal to a given key (targetKey). Since the key field of a telephone listing is a String, the method can use Java's compareTo method to perform its work. This method, invoked on Line 20, returns a value of zero if two strings are equivalent. Finally, Lines 22–24 are a set method to change the contents of the address field of a Listing object. Normally it would not be included in a minimal implementation, but is included here for pedagogical reasons (to demonstrate the encapsulation of the Unsorted-Optimized structure's implementation). Data Structure Class The class, UnsortedOptimizedArray, is the class that implements the Unsorted-Optimized structure's pseudocode (error checking included). It allocates and initializes all the storage for the structure which includes an array named data, a memory cell next to keep track of where to perform the next insert operation, and a memory cell size to store the size of the array. Its methods, which include a no-parameter constructor, perform the four basic operations Insert, Fetch, Delete, and Update in the key field mode. For purposes of simplicity, the maximum capacity of the data structure in this implementation, which is presented in Figure 2.17 , is fixed at 100 nodes. The code of the operation methods is simply the Java version of the Insert, Delete, Fetch, and Update pseudocode previously discussed but with one exception: The insert method checks to make sure that there is sufficient system memory to perform the deep copy of the client's node. Lines 2–4 create three fully encapsulated data members: the integer next, which will store the array index of the next Insert operation, the integer size, which will store the size of the array, and the array reference data, which will store the address of the array of node references. Both integer variables are initialized by the constructor (Lines 6–10) when a data structure object is created, and the variable data is set pointing to an array of 100 reference variables (Line 9) that can store the addresses of 100 Listing objects. 115 Figure 2.17 The Code of the Class SortedOptimizedArray To utilize this structure, a typical application would code: which would create a data structure object named boston. Its data would be initialized as shown in Figure 2.18 , which assumes that the array data's base address is 100. For brevity, the variable size is omitted from the 116 figure. The insert Method Lines 12–20 implement the Insert operation as the Java equivalent of the pseudocode previously developed, with two additional lines (Lines 16– 17) that check for insufficient system memory. If there is not sufficient memory to perform the deep copy of the client's node, the Listing class' deepCopy method returns a null reference (Line 15). In this case the insert method returns false and terminates (Line 17). The client (application) code to insert a Listing object bill into the data structure object boston could be: Figure 2.18 An Initialized Unsorted-Optimized Array Structure Named boston Line 15 deep copies the listing into a new Listing object and places the address of the new object into the next available element of the encapsulated array data. Since the client does not know this address and cannot access the array containing the address, the node is fully encapsulated. The encapsulation is accomplished by “hiding” the address of the newly created listing. Figure 2.19 shows the process of inserting a copy of the Listing object bill into the data structure boston. It assumes the new Listing object, created by the deepCopy method, is stored at location 1000. To illustrate the effectiveness of the encapsulation, suppose that the client code used the class Listing's setAddress method to set the address 117 field of the Listing object bill to “2nd Street,” coded as: Then, as shown in Figure 2.20 , the address contained in the Listing object bill located at address 5000 would change. However, since the information stored in the data structure is in a different Listing object, specifically the Listing object located at location 1000 created by the deepCopy method, the address of the phone listing stored inside the data structure is unchanged. Consistent with the concept of data encapsulation, the only way the client can change the address field of a listing stored inside the data structure is to invoke the data structure's update method. An alternate, but undesirable, way of coding the insert method is to change Line 15 of Figure 2.17 to data[i] = newListing. This subtle change would, in effect, unencapsulate the data structure, since Line 15 would then perform a shallow copy. Thus, data[i] would store the address of the client's Listing object, bill, sent over as an argument to the insert method (see Figure 2.21 ). The data structure and the client code now reference the same Listing object, and the client statement Figure 2.19 The Data Structure boston, After Inserting Bill's Listing 118 Figure 2.20 The Data Structure boston After the Client Changes the Address Stored in the Client Listing bill Figure 2.21 The Data Structure boston, with a Shallow Copy Coded as Line 15 of the insert Method changes the address of the listing stored (actually referenced from) “inside” the data structure. All subsequent fetches of the listing from the data structure would contain the new address. Effectively, the client has changed the data stored in the data structure without invoking the update method—a violation of encapsulation. The fetch Method 119 Lines 22–41 of Figure 2.17 implement the pseudocode of the Fetch operation. The method heading, Line 22, indicates that the method returns a reference to a Listing object. This reference will be the address of a deep copy of the node whose key field targetKey, is passed to the method as a parameter. It is important that the method return a reference to a deep copy of the requested listing, since, if it returned a shallow copy, the client would then know the address of the object stored inside the data structure. Knowing this object's address, the client would have direct access to it, a violation of encapsulation. Figure 2.22 shows the data structure with three listings stored in it, before and after the client fetches Tom's listing, using the statement tom = boston.fetch(“Tom”). Notice the change in the contents of the second and third elements of the array after the fetch operation is complete. The reference to the fetched node has begun to bubble up to the front of the array. Figure 2.22 The Data Structure, boston, Before and After Tom's Listing Is Fetched The delete Method Lines 43–56 of Figure 2.17 implements the pseudocode of the Delete operation. It returns the Boolean value true if it successfully deletes a listing. Lines 46–48 perform the sequential search for the requested listing. If found (Line 49), Line 52 eliminates the listing from the data structure by 120 overwriting the reference to it stored in the array data with the address of the last listing in the structure. After this overwriting, the address of the deleted listing is not stored in any reference variable, which causes Java's memory manager to return the storage allocated to the deleted listing's object to the available memory pool. Figure 2.23 shows the data structure before and after Bill's listing is deleted using the statement: boolean success = boston.delete(“Bill”). Notice the change in the contents of the first and third elements of the array after the deletion is complete. The reference to the deleted node has been overwritten with the address of Tom's node, and the address of Tom's node is no longer stored in the third element of the array. In addition, the variable next has been decremented to “collect the garbage.” Figure 2.23 The Data Structure, boston, Before and After Bill's Listing Is Deleted The update Method Lines 58–65 of Figure 2.17 implement the pseudocode of the Update operation. The value of the key of the node to be deleted (targetKey) and the new contents of the node (contained in the object newNode) are passed into the method (Line 58). It is important to understand that the object containing the old information is not actually updated. Rather, that object is deleted from the data structure (Line 59), and a new Listing object is inserted into the structure (Line 61). Thus, the key field of the node could also be updated during an Update operation. Figure 2.24 illustrates the data structure before and after Bill's listing is updated by the client statement: boolean success = boston.update(“Bill”, 121 billsNewListing). The graphics in the figure assume Bill's name has not changed, his new address is “4th Avenue,” and his new phone number is “676-7878” (all specified in the object billsNewListing). Notice that the reference to Tom's listing has moved to the front of the array as a result of the deletion of Bill's old listing (the first step in the Update algorithm). Furthermore, the reference to Bill's new listing is at the end of the used portion of the array because it was added with an invocation to the insert method (the second step in the Update algorithm). Using the Baseline Implementation Figure 2.25 presents an application program that demonstrates the use of the data structure class UnsortedOptimizedArray. The output it produces is shown in Figure 2.26 . Line 3 of Figure 2.25 creates the data structure boston, an object in the class UnsortedOptimizedArray. Lines 6–8 create three telephone listings (Listing objects) two of which are inserted into the data structure (Lines 12–13) and then fetched back and output (Lines 15–18), verifying the class's constructor, insert method, and fetch method. Line 14 verifies that the fetch method returns a null reference when the requested listing (Tom's) is not in the structure. Figure 2.24 The Data Structure, boston, Before and After Bill's Listing Is Updated Lines 21–25 demonstrate the data structure's encapsulation. Line 22 changes the contents of the address field of the client's object mary to 9th 122 Avenue. Line 23 outputs the modified client object to verify that the contents of the address field have been changed. This should not affect Mary's listing in the data structure because it is encapsulated inside the structure. Line 24 fetches back the listing with key field contents “Mary” from the structure, which is then output on Line 25. Since the output is the original listing with the address field unchanged, the data structure's encapsulation is verified. Lines 28–31 demonstrate the use of the delete method. First, an attempt is made to delete a listing not in the structure (Tom's listing, Line 29), which produces an output of false . Then Bill's listing is deleted (Line 30), which produces an output of true because Bill's listing was in the structure. The subsequent attempt to fetch Bill's listing (Line 31) results in an output of null because Bill's listing has been deleted from the structure. Lines 34–40 demonstrate the use of the update method. First, an attempt is made to update a listing not in the structure (Tom's listing, Line 35), which produces the correct output, false . Then, Bill's listing is reinserted into the structure (Line 36) and successfully updated to the contents of Tom's listing (Line 37 produces an output of true). After the Update operation is performed, the subsequent attempt to fetch a listing whose key field is Bill (Line 38) is unsuccessful (null output), while the fetch of the listing whose key field is Tom (Line 39) is successful and the listing is output. 123 Figure 2.25 An Application that Demonstrates the Use of the Class UnsortedOptimizedArray 124 Figure 2.26 The Output from the Application Program Shown in Figure 2.25 2.3.2 Utility Methods The motivation for beginning with a baseline implementations of the UnsortedOptimizedArray structure and a class that described a node in a telephone listing data base was purely pedagogical. It allowed us to gain an understanding of the basic implementation techniques within a minimal amount of code. Having gained that understanding, in this section we will expand the code of the class Listing, shown in Figure 2.16 , and the class UnsortedOptimizedArray, shown in Figure 2.17 , adding some common 125 utility methods that make them easier for clients to use. An input method will be added to the class Listing that, when invoked by the application program, will allow the user to input the contents of a telephone listing. The code shown below is an example of how the method could be used by a client application to add 50 telephone listings, input by the user, to the data structure boston. Two methods will be added to the DataStructure class: • A constructor to permit the client to specify the maximum number of nodes that can be stored in the structure when it is created. • A showAll method that outputs the contents of the entire data structure. The relationship between the three new methods and the application's user is depicted in Figure 2.27 . Figure 2.27 The Three-Method Expansion of the Baseline Implementation of the Unsorted-Optimized Array Structure 126 Figure 2.28 Recoding of the Class Node to Include a Default Constructor and an input Method The revised code of the Listing class, which has been renamed Node, is given in Figure 2.28 . The new input method is coded on Lines 25–29, which places the user input directly into the data members name, address, and number. The revised code of the UnsortedOptimizedArray class, which has been renamed UOAUtilities, is given in Figure 2.29 All references to the class Listing now refer to its expanded version, the class Node (e.g., Lines 4 and 18). The new constructor is coded as Lines 12–16. It uses an integer passed into it (the parameter on Line 12) to size the structure's array on Line 14. Since the value of s is also saved in the class' data member size (Line 15), no changes are required to the error checking performed in basic operation methods (e.g., Line 19). 127 128 Figure 2.29 The Unsorted-Optimized Array Implementation with an Additional Constructor and a showAll Method The showAll method is coded as Lines 72–75. It outputs the nodes by invoking the Node class' toString method Line (74) inside a for loop. The loop uses the variable next (Line 73) to decide when the output is complete. The following code is an example of how the new constructor and the showAll method could be used by the client application to declare and output all the nodes in a 100,000 listing structure named boston. Having completed the expansions to the baseline implementation of our Unsorted-Optimized, array-based structure, we will now discuss an additional feature that will make the structure easier for the client to use. 2.4 Expandable Array-Based Structures 129 When any array is created, its size must be specified. Thus, the array- based implementations developed in this chapter either required the client to specify the maximum number of nodes that would be stored in the structure, or the maximum was set by the structure's constructor to a default value. In either case, the maximum value was used to size the structure's array, and it was checked by the Insert operation before it attempted to add a node to the structure. There are many applications in which the maximum number of nodes in the structure cannot be anticipated (with any certainty) at the time the data structure is created. And so, there is an alternate implementation of the insert method aimed at these applications. When the structure's array is full, the insert method copies the entire contents of the structure's array into a larger array and then inserts the node into the expanded structure. Under this scheme, the only time the Insert operation would return false is when the system memory is exhausted. In some programming languages, copying an array of objects from one array to another is a time consuming process. This is not the case in Java, because Java implements an array of objects as an array of reference variables that point to the objects (see Figure 2.24 ). Therefore, only the reference variables need to be copied into the expanded array, making the process as efficient as copying an array of primitive values. The pseudocode of the algorithm written to expand a Java array follows. It assumes data is the array to be expanded, larger is the expanded array, and temp is an array reference variable. The comment on Line 3 of the algorithm must be replaced with the code that copies the object references from temp into data. This can be done using a loop or by invoking the Java method arraycopy, which is contained in the System class. When a loop is used, the following code would replace Line 3. where temp.length returns the number of elements in the array to be 130 expanded, temp. Java's arraycopy method copies a specified number of items from a source array to a destination array. The source and destination starting indices are supplied to the method, as is the number of elements to be copied. Since we want to copy the entire contents of the array temp starting at index zero into the array data starting at index zero, the invocation would be: The use of the method arraycopy is preferred because it is about 50% faster than the loop technique. On an absolute speed basis, tests performed on a PC using an AMD Athlon XP 3000+ processor with a 2.17-gigahertz clock showed that the arraycopy method was able to copy a 500,000 element array of object references in 0.06 seconds. What this means is that an Insert operation performed at a time when an array-based structure is full would require an extra 0.06 seconds to expand the structure's 500,000 element array. This level of performance would be adequate for most applications. Still, there are many real-time applications that could not tolerate a 0.06-second delay. One other potential downside is that expanding an array can result in a somewhat unpredictable “java.lang.OutOfMemoryError: Java heap space” run-time error. All things considered, for small- to moderately-sized array-based structures, an insert method that can expand the size of the structure's array greatly enhances the usefulness of these structures. A good middle ground implementation would add a Boolean parameter to the structure's constructor to allow the client to specify whether or not the expandable feature of the insert method should be employed for a particular application. This implementation is left as an exercise for the student. 2.5 Generic Data Structures The development of software is an expensive and time-consuming process. Therefore, whenever possible, we should write reuseable software. In the context of data structures, reusability is achieved by writing data structures that are application independent, or generic . Generic data structures, by definition, can store any kind of node and, therefore, can be used in any application. (Naturally, the performance of the structure would have to be consistent with the requirements of the application.) Stated another way, if the final implementation of our Unsorted-Optimized array structure (the class UOAUtilities) was generic, 131 it could not only be used to store the data for a telephone listing application, but also an employee record application, a store inventory application, or any other type of data intense application without modifying its code . Generic implementations of data structures fall into two groupings: homogeneous and heterogeneous (nonhomogeneous) generic data structures. Both groups can store data sets comprised of any type of node (e.g., telephone listings or employee records). However, in a homogeneous generic data structure, all nodes stored in a particular data structure object must be of the same type (e.g., all nodes in the data set are telephone listings or they are all employee records). In a heterogeneous structure, the nodes in a single data structure object need not be all of the same type (e.g., employee records and telephone listings can be stored in the same object). The difference between homogeneous and heterogeneous structures is illustrated in Figure 2.30 . Figure 2.30 Phone Listings and Employee Records Stored in Homogeneous and Nonhomogeneous Data Structures In the remainder of this section, we will examine the features of a data structure's design that makes it generic and show how to implement these features in Java. 2.5.1 Design Considerations 132 The first consideration in designing and implementing generic data structures is to separate the node definition and the data structure's definition into two separate classes since different applications deal with different types of nodes. Typically, the nodes in different applications have a different number of fields, along with different field names, widths, and data types. By defining the node in a separate class we need not rewrite the data structure class when the node composition changes. In addition, the task of coding the node class is appropriately passed on to the application programmers who know the composition of the nodes. In the baseline and expanded implementations of the array-based structures presented in this chapter, the node definition was coded in a separate class. Therefore, both of these implementations conformed to this generic design feature. A second consideration is that the data structure's code must not mention the names of the data fields that make up a node. If it did, it would again be tied to a particular node composition of a particular application. The two implementations of the Unsorted-Optimized Array structure presented in Figures 2.17 and 2.29 , both conformed to this generic design feature. Neither class mentioned the variables name, address, or number; the field names of our telephone listings. The second implementation avoided mentioning these in its showAll method by invoking the node definition class' toString method. A third consideration is that if the structure is going to be encapsulated, a method to perform the deep copy must be part of the node definition class. Otherwise, the data structure class would have to have knowledge of the node composition in order to create the clone node (see Line 16 of Figure 2.28 ). Since the node definition classes Listing and Node provided a deepCopy method, both of our implementations satisfied this consideration. A fourth consideration in designing and implementing generic structures is that if the structure is going to be accessed in the key field mode, a method to determine if a given key is equal to the key of a node must be provided in the node definition class. It cannot be coded in the data structure class, because how we determine that two keys are equal depends on the key's type, which varies from one application to another. For example, if the keys are numeric, the relational operator == can be used, but it cannot be used if the keys are String objects. Since the node definition classes Listing and Node provided a compareTo method to determine if two keys were equal, both of our implementations satisfied this generic consideration. 133 The final two design considerations address the fact that nodes can be instances of any class (e.g., Listing objects, or Node objects, or Employee objects, or CarType objects, etc.), and their key fields can be any type (not just a String object). Neither of our two implementations addressed these issues. In the first implementation, the name of the node definition class had to be Listing because many lines of the data structure's code specifically mention that type (e.g., Lines 4, 9, 12, 22, etc. of Figure 2.17 ). Similiarly, the second implementation shown in Figure 2.29 can only store nodes that are instances of the class Node (see Lines 4, 9, 14, 18, 28, etc.). Both implementations assume the key field sent to the fetch, delete, and update methods is an object in the class String (e.g., Lines 28, 49, and 64 of Figure 2.29 ). Our final implementation of the Unsorted-Optimized, array-based structure will address these last two issues, making it a generic implementation. Before proceeding to that implementation, it is useful to summarize the design and implementation features that make a data structure class generic: 1. The node definition and the data structure are coded as two separate classes. 2. The data structure cannot mention the names of the data fields that make up a node. 3. If the structure is going to be encapsulated, a method to perform a deep copy of a node must be coded in the node definition class. 4. If the structure is going to be accessed in the key field mode, a method to determine if a given key is equal to the key of a node must be coded in the node definition class. 5. The data structure code cannot mention the name of the node class. 6. The data structure code cannot mention the type of the key. 2.5.2 Generic Implementation of the Unsorted-Optimized Array As previously mentioned, in our two implementations of the Unsorted-Optimized structure, we have incorporated the first four design features of a generic implementation. In this section, we will develop a completely generic implementation of this structure by incorporating the fifth and sixth design features into the code of the class UOAUtilities (presented in Figure 2.29 ). The revised structure will be able to store nodes of any type, whose key field is of any type. Prior to the release of 134 Java 5.0, this was accomplished by changing the types of the node and key field references in the data structure class to Object references. However, for reasons we will discuss later, the generic typing features in Java 5.0 offer a better alternative. As mentioned in Chapter 1 , generic typing is a language feature that allows the author of a class to generalize the type of information with which a class (or a method) will deal. Then, the specific type of information is specified by the client when an object in that class is declared. For example, if the name of a generic data structure class were UOA, then the following two declarations of the data structure objects boston and newYork would be used to store Listing objects in the data structure boston, and Node objects in the data structure newYork. Although generic typing is the best alternative for coding generic data structures, its use is not as simple as substituting generic type placeholders for the data member types as presented in the Java review example in Chapter 1 . The class in that example did not allocate new objects of the generic type, and it did not invoke any methods to operate on objects of the generic type. Both of these complications must be dealt with to complete the conversion of our data structure class into a generic class. In the interest of simplicity, we will discuss the conversion of one section of the class UOAUtilities' code at a time, so that that we can focus on one conversion issue at a time. However, the line numbers in each section of converted code (presented in Figures 2.31 -2.35 ) will be sequential and will roughly parallel the line numbers of the class UOAUtilities presented in Figure 2.29 . To begin with, we will consider modifications to the data declarations and the class' constructors (Lines 1– 16 of Figure 2.29 ), which are highlighted in the revised code presented in Figure 2.31 . The name of the new (and eventually generic) class is assumed to be UOA. As was discussed in Chapter 1 , to make data members generic, a generic placeholder is added to the end of the class heading. In this case, the placeholder has been added to the end of Line 1, and the type
Node on Line 4 has been replaced with the generic type, T. However, we
cannot simply replace the type Node on Lines 9 and 14 with the generic
type T because the translator will not allow us to declare an array of
references to a generic type. To do so would imply that there is a
constructor in the class T, which at this point is an class.
135

Therefore, the arrays on these lines are declared to be arrays of Object
references. The class Object is predefined in Java, and since the class
Object is the parent class of all Java classes, references will be able to
store the address of any type of node. Finally, on Lines 9 and 14 the
location of the array of Object references must be coerced into the location
of an array of type T since the variable data on Line 4 is now declared to
store the location of an array of type T.
Figure 2.31 The Changes Made to the Data Members and
Constructors of the Class UOAUtilities (Figure 2.29 ) to Make Them
Generic
Figure 2.32 The Changes Made to the insert Method of the Class
UOAUtilities (Figure 2.29 ) to Make It Generic
Next we will consider modifications to the insert method, Lines 18–
26 of Figure 2.29 . These changes are highlighted in the revised code
presented in Figure 2.32 . Referring to that figure, one line has been added
(Line 19), and two lines have been changed (Lines 18 and 22). The change
to Line 18 is simply to change the parameter type Node to the generic type
placeholder T. The change to Line 22 necessitates the addition of Line 19,
136

so we will discuss Line 22 first.
The invocation to the method deepCopy (Line 22) can no longer
operate on the object referenced by newNode, as it did on Line 21 of
Figure 2.29 , because newNode is now of type T (Line 18) and the
translator cannot look into the class T to verify the method’s signature. 15
The remedy is to reference the object to be operated on with the variable
node, which is defined on the left side of Line 19 to be a KeyMode
reference, and then place the address of the node to be inserted into the
variable node (right side of Line 19). However, the translator still wants to
verify the signature of the method deepCopy, and at this point KeyMode is
an symbol. Therefore, the programmer of the data structure
must code a Java interface named KeyMode and define the signature of the
method deepCopy in it. The actual implementation of the method will be
coded where we have always coded it, in the application’s node definition
class. One restriction is that the node definition classes must add the
phrase “implements KeyMode” to its heading. The coercions (KeyMode)
and (T) on Lines 19 and 22 respectively are necessary to match the types
on the left side of the assignment operators.
The interface KeyMode would be written (by the coder of the data
structure class) as:
As we discuss the changes to the other methods in the class
UOAUtilities, we will add two more method signatures to this interface.
Next we will consider modifications to the fetch method, Lines 28–47
of Figure 2.29 . These changes are highlighted in the revised code
presented in Figure 2.33 . Referring to that figure, one line has been added
(Line 36), and five lines have been changed (Lines 29, 30, 31, 34, and 41).
137

Figure 2.33 The Changes Made to the fetch Method of the Class
UOAUtilities (Figure 2.29 ) to Make It Generic
Most of the changes here are made for the same reasons as the
changes we made to Line 22 of the insert method: A method cannot
operate on objects referenced by variables of type T. In the case of the
fetch method, there are two invocations. The method compareTo is
invoked on Line 33 of Figure 2.29 , and the method deepCopy is invoked
on Line 39. Both of these methods operate on the entity referenced by
data[i]. Since the variable data now stores a T reference (Line 4 of Figure
2.31 ), the translator cannot verify the method signatures. The remedy is
the same as that discussed for the insert method. Both invocations are
changed to operate the KeyMode reference variable node (Lines 34 and 41
of Figure 2.33 ), whose declaration is on Line 30, and the signature of the
method compareTo is added to the interface KeyMode. Line 36 was added
so that the reference stored in the variable node is changed each time
through the loop. Since the fetch method returns the reference stored in the
variable node, and node’s type has been changed to be of type KeyMode
(Line 30), the fetch method’s returned type (on Line 29) has been changed
from Node to KeyMode. Finally, the type of the insert method’s parameter
on Line 29 has been changed to an Object reference so that the key is not
restricted to String objects.
The expanded KeyMode interface is now:
138

Next we will consider modifications to the delete method (Lines 49–
62 of Figure 2.29 ). These changes are highlighted in the revised code
presented in Figure 2.34 . Referring to that figure, one line has been added
(Line 57), and three lines have been changed (Lines 51, 52, and 55). The
rationale for these changes is the same as that discussed for the analogous
changes made to the insert method: to direct the translator to the interface
KeyMode to verify the signature of the compareTo method and to allow
any type key to be passed into the method.
Figure 2.34 Changes Made to the delete Method of the Class
UOAUtilities (Figure 2.29 ) to Make It Generic
Figure 2.35 Changes Made to the update Method of the Class
UOAUtilities (Figure 2.29 ) to Make It Generic
The modifications to the update method (Lines 64–71 of Figure 2.29 )
are highlighted in the code presented in Figure 2.35 . Only the types of the
method’s parameters were changed so that the key field need not be a
139

reference to a String object, and to make the invocation on Line 71
compatible with the insert method’s new signature. The modified insert
method is expecting an argument of type T (Line 18 of Figure 2.32 ).
Therefore, the type of the argument (newNode) passed it has been changed
on Line 68 of Figure 2.35 .
Finally, although the toString method in the class UOAUtilities need
not be changed, its signature is added to the interface KeyMode to remind
the application programmer to include it in the node definition class. For
convenience, all of the code in the generic implementation of the
Unsorted-Optimized array is presented in Figure 2.36 , and the code of the
expanded interface KeyMode is presented in Figure 2.37 .
140

141

Figure 2.36 Fully Implemented Generic Version of the Unsorted-
Optimized Array-Based Structure
Figure 2.37 The Interface KeyMode
2.5.3 Client-Side Use of Generic Structures
To use our generic data structure class UOA, the application
programmer simply declares an object (or objects) in the class UOA,
specifying the type of the nodes it will store, and implements the interface
KeyMode in the node definition class. This means that the node definition
class must include a coding of a deepCopy method, a compareTo method,
and a toString method whose signatures are those specified in the interface
(Figure 2.37 ). Naturally, the node definition class should also include the
142

declaration of the fields of a node, constructors as required by the
application, and any utility methods that the application uses to operate on
the nodes (e.g., an input method for the application’s user to input the
values of a node). The translator will check that the phrase “implements
KeyMode” appears in the class’ heading and that the class includes the
code of all the methods mentioned in the interface. As previously
mentioned, the coding of the deepCopy and compareTo methods is
properly left to the application programmer who knows the character of
the nodes and what it means to say that two key objects are equal.
To declare two Unsorted-Optimized, array-based structures named
carDealer1 and carDealer2, each able to store a maximum of 700 objects
of the class Car, the client code would be:
As part of Java’s generic feature, these declarations are compiler
enforced homogeneous structures. Any attempt to store anything other
than a Car object in these structures, for example,
results in a compile error.
As an example, Figures 2.38 and 2.39 present the code of a telephone
directory application. Figure 2.38 presents the class PhoneListing that
defines the nodes used the application. Since the application will use our
unsorted-optimized generic structure UOA, it implements the interface
KeyMode. Figure 2.39 presents the code of the application and the output
it produces.
143

Figure 2.38 The Node Definition Class PhoneListing
Figure 2.39 A Phone Book Application That Uses the Generic Class
UOA and Its Output
2.5.4 Heterogeneous Generic Data Structures
144

A generic data structure can be used as a heterogeneous structure if
the type parameter is left out of the object declaration. For example, if the
declaration for the data structure given in the previous section were
changed to:
then any type of objects whose definition class implements the
interface KeyMode (Car objects, Truck objects, etc.) could be stored in it.
In this case, the statements
would not result in a compile error. It should be noted that for most
applications it is better programming practice to declare two homogeneous
data structures that each store one type of node, rather than one
heterogeneous structure that stores two different types of nodes.
2.6 Java’s ArrayList Class
The Java Application Programmer Interface provides a class named
ArrayList that is similar to the language’s array construct but in fact has
much more in common with the array-based structures developed in this
chapter. Like arrays, access into the structure is in the node number mode
16 via an integer index whose minimum value is zero, and the structure is
unencapsulated . Unlike arrays, the structure ArrayList supports all four
basic operations (Insert, Fetch, Delete, and Update), is implemented using
generics, cannot store primitive types, 17 and can be used either as
compiler-enforced homogeneous or heterogeneous structure. Considering
these characteristics, an ArrayList is actually an unencapsulated node
number mode access version of our generic array-based structure.
The ArrayList structure does have one additional feature that neither
arrays nor our array-based structures possess. Although we can specify an
initial number of nodes to be stored in the structure,
an ArrayList object can expand at run-time beyond its initial size to
accommodate unanticipated Insert operations. As discussed, Insert
operation implementations that support run-time expansion run more
slowly than those that do not.
145

Table 2.5 presents the names of the basic operation methods in the
class ArrayList and some examples of their use. It assumes that the
ArrayList structure boston was declared as a heterogeneous structure with
the statement:
that the variables tom and mary store references to Node objects and
temp can store a Node reference.
EXERCISES
Knowledge Exercises
1. Which of the four basic operations does the built-in structure array
support?
2. Draw a picture of the memory allocated to a five-element, one-
dimensional array of integers called numbers. Assume the first element
of the array is stored at location 500, the array is stored in ascending
memory locations, and that the integers occupy 4 bytes each. On the left
side of the array, indicate the number of each element, and on the right
side of the array indicate the memory location number (beginning at
500).
3. Give the mapping function used to calculate the address of the k th
element of a one-dimensional array of integers, assuming element zero
is stored at location 500, the array is stored in ascending memory
locations, and that an integer occupies 4 bytes of memory.
146

4. 20 integers are stored in a two-dimensional array of four rows and
five columns. The name of the array is ages.
a) Draw a picture of the rows and columns of the array, indicating the
column numbers across the top of the array and the row numbers along
the left side.
b) Place an x in the cell of your picture whose contents is modified by
the statement: ages[2][3] = 20;
c) Give the memory location of the cell ages[2][3] assuming the array is
stored in row major order in ascending memory locations beginning at
location 500. Assume each cell is 4 bytes.
5. Give the change to Line 7 of an Unsorted Array structure’s Fetch
algorithm to unencapsulate the structure after the Fetch operation is
completed.
6. A data structure is to be chosen that will allow the customers to look
up the price of an item, given its item number. The data set will only be
loaded once and never modified. Of the three array-based structures
discussed in this chapter, which would be best suited for this
application?
7. The Sorted Array structure accesses nodes using the Binary Search
algorithm, and the Unsorted-Optimized Array structure uses a
Sequential Search to access nodes. Knowing that a binary search is
significantly faster than a sequential search, explain why it is that the
average speed of the Unsorted-Optimized array structure is faster than
that of the Sorted Array structure. (Assume all four basic operations are
equally probable.)
8. Referring to the data structure depicted in Figure 2.8 , what memory
location would be returned by the Fetch algorithm if Mike’s node was
requested and the structure was not encapsulated?
9. Give the differences in the basic operation algorithms of the Unsorted
and Unsorted-Optimized Array structures that made the latter structure
faster. Under what conditions would both structures’ Fetch algorithms be
equivalent from a speed viewpoint?
10. The Unsorted-Optimized array structure is used to store a data set.
Calculate its density if:
a) Each of the client’s nodes contains 8 bytes of information and there
are 50 nodes in the data set.
b) Each of the client’s nodes contains 200 bytes of information and there
147

are 1,000,000 nodes in the data set.
11. Give the average number of memory accesses of the Unsorted-
Optimized array structure whose data set is described in part (b) of the
previous exercise:
a) Assuming all operations on the data set are equally probable.
b) Assuming only Insert operations will be performed on the data set.
12. Plot the variation in density with the number of nodes, n , in an
array-based structure. Assume each node contains 10 information bytes
and that the range of n is 2 <= n <= 100. 13. Define the data structures term garbage collection. 14. Describe the garbage collection method for the Unsorted- Optimized array structure. 15. You have coded an application for your friend's business that uses an Unsorted-Optimized array to store the venture's data. One day your friend informs you that the speed of the operations performed on the data set seems to be getting faster and faster. Explain how this could happen. 16. An application's data set will consist of five different types of nodes. Is the data set homogeneous or heterogeneous? 17. Give the six design features that should be followed when designing generic data structures. 18. Give the pseudocode of an update method for the Unsorted structure that does not invoke the delete and insert methods and does check for errors. Programming Exercises 19. A database is to be developed to keep track of student information at your college. It will include names, identification numbers, and grade point averages. The data set will be accessed in the key field mode, with the student's name being the key field. Code a class named StudentListings that defines the nodes. Your class should include all the methods in the class shown in Figure 2.28 except for the setAddress() method. It should also include a no-parameter constructor. Test it with a progressively developed driver program that verifies the functionality of all of its methods. 20. Code a class that implements the Sorted Array structure, and write 148 a progressively developed driver program that verifies the functionality of all of its methods. Assume that it is to store a data set whose nodes are described in Exercise 19. Include error checking in the code of the basic operation methods, a constructor to permit the client to specify the maximum size of the data set, and a method to display the contents of entire data set in sorted order. 21. Code an application program that keeps track of student information at your college: names, identification numbers, and grade point averages in a fully e ncapsulated (homogeneous) Sorted array- based data structure. When launched, the user will be asked to input the maximum size of the data set, the initial number of students, and the initial data set. Once this is complete, the user will be presented with the following menu: Enter: 1 to insert a new student's information, 2 to fetch and output a student's information, 3 to delete a student's information, 4 to update a student's information, 5 to output all the student information in sorted order, and 6 to exit the program. The program should perform an unlimited number of operations until the user enters a 6 to exit the program. 22. A database is to be developed to keep track of faculty information at your college. Faculty member's names, departments, areas of expertise, and e-mail addresses will all be included. The data set will be accessed in the key field mode, with the area of expertise being the key field. Code a class named ProfessorListing that defines the nodes. Your class should include all the methods in the class shown in Figure 2.28 except for the setAddress() method. It should also include a no- parameter constructor. Include a progressively developed driver program that verifies the functionality of all of the class's methods. 23. Code an application program to store a data set consisting of StudentListing and ProfessorListing objects (see Exercises 19 and 22) in two Sorted Array structures. When launched, the user will be asked to input the maximum size of both data sets, the initial number of students, the initial number of professors, and the initial data set. Once this is complete, the user will be presented with the following menu: 149 Enter: 0 to exit the program. 1 to insert a new student's information, 2 to fetch and output a student's information, 3 to delete a student's information, 4 to update a student's information, 5 to insert a new professor's information, 6 to fetch and output a professor's information, 7 to delete a professor's information, 8 to update a professor's information, and 9 to output the entire data set in sorted order. The program should perform an unlimited number of operations until the user enters a 0 to exit the program. 24. Modify the class presented in Figure 2.29 so that the Insert operation doubles the size of the array when it senses the structure is full. The feature should be activated by a Boolean parameter set by the client code when the structure is declared. Write a driver program to demonstrate that the class functions properly. 25. Code a GIU program that visually demonstrates the changes to the contents of the array, and the other data members that make up an Unsorted-Optimized array object, when each of the four basic operations is performed. When the program is launched, the structure should be shown in its initialized state. Six buttons should be available to the user: one button for each of the four basic operations, a reinitialize button, and a quit button. Text boxes should be available to allow the user to input a node's information and key field contents. 26. Code the application described in Exercise 21, using a generic implementation of the data structure described in Exercise 20. 27. Code the application described in Exercise 23, but use the Java API class ArrayList as the application's data structure. (Will there be a need to ask the user for the maximum size of both data sets?) 1 This sequential ordering is referred to as a linear list, in which there is a unique first node (node 0 ), a unique last node (node max ), and any other node (node i ) is preceded by a unique node (node i −1 ), and followed by a unique node (node i +1 ). 2 Assumes the nodes are stored in ascending memory locations as shown in Figure 2.1 . 150 For descending memory location storage, A = A 0 − N × w. 3 Actually, many CPUs (e.g., the Intel Processors) can evaluate this function in one machine instruction. 4 This assumes the subscript begins at zero. 5 Again we assume the subscripts start at zero. 6 More accurately, the array is an array of reference variables that stores the address of the objects that contain the nodes' information. 7 For simplicity, the time to execute the method deepCopy() is ignored. 8 As previously stated, accessing a data member of an object requires two memory accesses: one to fetch the object's address from the array and one to fetch the data member from the object. 9 Assumes all operations are equally probable and is therefore calculated as the arithmetic average of the four operation speeds. 10 This assumes, on the average, the node referenced from the middle of the array is the node being deleted. 11 This assumes all operations are equally probable and is therefore calculated as the arithmetic average ofthe four operation speeds. 12 Assumes all operations are equally probable and is therefore calculated as the arithmetic average of the four operation speeds. 13 For n > = 100, and w > = 5,4 / (w n ) < = 0.008, which is less than 1% of the denominator. 14 This implementation will include error checking. 15 The generic type T is not associated with a class until an application declares an object in the class UOA. 16 The one exception is that the Insert operation assigns the next node number to the inserted node, which makes the structure a little awkward to use. 17 Java 5.0 gives the appearance of allowing primitives to be inserted into an ArrayList object but it actually wraps the primitive in a Wrapper object before inserting it. 151 CHAPTER 3 Restricted Structures OBJECTIVES The objectives of this chapter are to familiarize the student with the features, implementation, and uses of the restricted structures, Stack and Queue, and to master a methodized approach to converting a data structure to a generic implementation. More specifically, the student will be able to Understand the operation and access mode limitations implicit in restricted structures and the motivation for accepting those restrictions. Explain the operations and access modes of the Stack and Queue data structures, and be able to implement an array-based version of them that includes error checking. Write an application program that uses a stack or a queue to store the data it processes. Understand and be able to quantify the performance of restricted structures. Understand the advantages of postfixed arithmetic expression notation and be able to write an application that evaluates these arithmetic expressions. Expand a Stack or Queue implementation to include a reinitialization operation, a peek operation, tests for full and empty, and the ability to expand these structures dynamically at run-time. Recognize several common applications of restricted structures. Understand a methodology used to convert a data structure to a generic implementation using the generic features of Java, and be able implement a generic data structure using the methodology. More fully understand the implications of a Java interface and its role in coding generic data structures. 152 Understand the operations of a priority queue. Develop an application that declares objects in Java's Stack class, and understand the advantages and disadvantages of this API class. 3.1 Restricted Structures All of the array-based structures presented in Chapter 2 have high density and support all four of the basic operations: Insert, Delete, Fetch, and Update. As such, they can be used to store a data set for any application as long as their speed is within the performance parameters of the application. Restricted structures are not as ubiquitous in that they are not appropriate for all applications. By design, they target a small subset of applications (see Figure 3.1 ) that share a common “pattern” of operations and are so ideally suited for this subset of applications that no other data structure would even be considered during the design process. A good analogy would be a very talented rock band whose music appeals to a very small percentage of the population. Although their music does not appeal to everyone, the band has a small cult-like following who would not even consider listening to anyone else's music. Figure 3.1 Limited Use of Restricted Structures Figure 3.2 The Two Basic Operations Allowed on Restricted Structures Restricted structures receive their name because they place severe restrictions on the way the application program accesses its data set. Both 153 the operations performed (Insert, Delete, Fetch, and Update) and the mode in which these operations are performed (node number or key field mode) are restricted. The restrictions placed on the operations performed on the data set, as illustrated in Figure 3.2 are: • The Update operation is not supported. • The Fetch and Delete operations are combined into one operation. • The Insert operation is supported. Combining the Fetch and Delete operations implies that whenever a node is fetched from the data set, it is automatically removed from the structure. Thus, restricted structures are aimed at applications that recall a node once, and only once. In addition, since the Update operation is not supported, applications targeted by restricted structures are those that never change the contents of the nodes stored in the data set. The restrictions placed on the access modes are even more severe than those placed on the operations in that: • The key field access is not supported. • The node number access is supported, but is severely restricted. Under the restricted form of node number mode access associated with these structures, node number has no meaning. The client code does not specify the node number of the node inserted into the data set, nor does it specify the node number of the node to be fetched (and deleted). Rather, the client code simply specifies that the Insert operation, or combined Fetch-and-Delete operation, be “performed.” The Insert operation “arranges” the nodes in a chronological insertion order. Then, depending on the restricted structure we are dealing with, the combined Fetch-and- Delete operation operates on the node that has been in the structure either the shortest or the longest amount of time. Figure 3.3 A Restricted Structure after Nodes A, then B, then C, and Finally D Have Been Inserted 154 Figure 3.4 A Queue and a Stack Before and After a Fetch-and-Delete Operation Is Performed For example, consider the nodes with key fields A, B, C, and D inserted into a restricted structure—first A, then B, then C, and finally D (see Figure 3.3 ). Because the structure is a restricted structure, we can't say “fetch-and-delete B,” nor can we say “fetch-and-delete the second node.” All we can say is “fetch-and-delete” and depending on the restricted structure, either A will be returned and then deleted from the structure, or D will be returned and then deleted from the structure. The two most widely used restricted structures are Queue and Stack. In the case of a Queue, when a Fetch-and-Delete operation is performed, the node that has been in the structure the longest amount of time is fetched and then deleted. In the case of a Stack, the node that has been in the structure the shortest amount of time is fetched and then deleted (see Figure 3.4 ). A third restricted structure, Deque (not as widely used as Queues and Stacks), combines the functionality of a Stack and a Queue into one structure. Although extremely restrictive, the operational behavior of these restricted structures mimics the manner in which some important and common application programs access their data set. As a result, restricted structures are an important topic in computer science. We will begin our study by examining the structure Stack. 3.2 Stack The structure Stack obtains its name from the analogy that the nodes are stacked one on top of the other in the structure just as coins would be stacked (see Figure 3.5 ). A new coin, or node, is always placed on the top of the stack. Since the coins are stacked, the only way to remove a coin (or node) without toppling them is to remove the one that is on the top of the stack, which is the one that has been on the stack the least amount of time. 155 Removal of any other coin (or node) from the stack is not allowed since it would cause the stack to fall. Thus, the last node inserted into the structure is always the node returned on the next combined Fetch-and-Delete operation. Because of this, the structure is referred to as a Last-In-First- Out structure, and the acronym LIFO is often used to describe the structure Stack. 3.2.1 Stack Operations, Terminology, and Error Conditions The Insert operation on a Stack structure is called a Push operation, and the combined Fetch-and-Delete operation is called a Pop operation. Nodes are said to be stored on a stack and removed from a stack. Thus, we say that nodes are “pushed onto” a stack and “popped from” a stack. The last node pushed onto a stack is said to be at the top of the stack and the first node pushed onto the stack is said to be at the bottom of the stack. When there are no nodes on a Stack, the stack is said to be empty . If the stack has reached its maximum node capacity, it is said to be full . A Pop operation cannot be performed on an empty stack, and a Push operation cannot be performed on a full stack; both situations result in an error. The error associated with a Pop operation on an empty stack is called an underflow error, and the error associated with a Push operation on a full stack is called an overflow error. Figure 3.6 illustrates the results of a series of Push and Pop operations on a Stack structure that has a maximum capacity of three nodes. Figure 3.5 A Stack of Four Coins 156 Figure 3.6 Several Operations on a Stack that Has a Maximum Capacity of Three Nodes, and the Results They Produce 3.2.2 Classical Model of a Stack In the classical model of a Stack, the Push and Pop operations are the only operations allowed on the structure. In addition, the maximum number of nodes to be stored in the structure (size of the stack) is specified when the structure is created. This classical model is often expanded to include other operations and features such as: • The ability to reinitialize the stack to empty. • The ability to test for an empty stack (underflow condition). 157 Figure 3.7 Typical Depiction of the n Element Array data Used to Store References to Nodes Pushed onto a Stack • The ability to test for a full stack (overflow condition). • The ability to pop a node from the stack without deleting it from the structure. 1 • The ability to store an “unlimited” number of nodes on the stack. 2 In the interest of simplicity, we will ignore these expanded features for now and limit our discussion of the structure Stack to the classical model and its implementation. Once an understanding of the classical model is obtained, we will discuss the changes necessary to incorporate the expanded features. The last feature, the ability to store an unlimited number of nodes on the stack, will be discussed again when we study linked lists in Chapter 4 . For now, an attempt to push more than a specified maximum number of nodes onto the stack will result in an overflow error. Throughout this chapter, an n element array of object references named data will be part of our Stack object, and it will be used to store the locations of the nodes pushed onto the stack. Typically, this array will be shown with its first element (index 0) at the bottom of the graphic, and its last element (index n − 1) at the top of the graphic (see Figure 3.7 ) In Chapter 4 we will discuss a second implementation of a stack that does not utilize an array. Operation Algorithms As we have stated, the classical model of a Stack is a fully encapsulated structure that supports two operations: Push and Pop. In addition to the array data used to store the locations of nodes pushed onto the stack, the algorithms for these two operations require that each Stack object contains two other integer data members: 158 • size, used to store the maximum number of nodes the stack can hold. • top, used to store the index into the array, data, where the last Push operation was performed. Figure 3.8 Data Members of a Stack Object Capable of Storing Four Nodes in Its Initialized State Stated another way, when a Push operation is performed, data[top] will store the address of the node pushed onto the stack. The memory cell top, as its name implies, will be used to “keep track” of the top of the stack. To determine the initial state of the Stack structure, we will assume that nodes pushed onto the stack will be stored in the array data sequentially, in the order in which they are pushed, beginning at element zero. Thus, the location of the first node pushed onto the Stack object is stored in data[0], the location of the second node pushed is stored in data[1], the location of the third node pushed is stored in data[2], etc. Initializing the memory cell top to −1 and incrementing it before the location is set into data[top], guarantees that the first node will be stored at location zero and that all subsequent nodes pushed will be store sequentially. Thus, the initial condition for the variable top (which will also indicate that the stack is empty) is: Initial (Empty Stack) Condition top = −1 Figure 3.8 shows all of the data members of a Stack object capable of storing four nodes in their initialized state. Turning our attention to the operation algorithms, our Push operation will not only insert a node into the structure but it will also check for an overflow error. Since top stores the index of the last node pushed, the test for overflow is when top is equal to the maximum index of the array, size- 159 1. Thus, the Push algorithm that adds the node whose location is stored in the memory cell newNode into the Stack object depicted in Figure 3.8 is: Push Algorithm Line 5 of the algorithm accomplishes the encapsulation of the data structure by performing a deep copy of the node added to the structure and storing the location of the copy of the client's node in the structure. Figure 3.9 shows the data members of a four element Stack object in its initial state, after three Push operations have been performed (Mike's node, Vick's node, and finally, Carol's node), and in its full state (after Bill's node is placed on the stack). In its full state, the variable top is equal to 3, or size - 1, so the next Push operation will not be performed. Rather, the Push operation will end after returning false on Line 2. Let us now turn our attention to the Pop operation. The Pop operation fetches and deletes a node from the structure. The node operated upon is the node that has been in the structure the least amount of time. The variable top stores the index of the element in the array (data) that contains a reference to this node. A successful Pop operation returns this reference to the client. To complete the Pop operation, top must be set to the index of the next node to be popped from the stack and the fetched node must be deleted from the structure. Since the nodes are stored in chronological order, decrementing top sets it to the index of the next node to be popped. That is, if data[2] was at the top of the stack, after a pop operation data[1] would be the new top of the stack. Decrementing top also, effectively, deletes the fetched node from the structure. Since the deleted node's reference is now stored “above” the top of the stack it cannot be accessed by subsequent Pop operations as they always decrement top. The only way the variable top can ever store the index of a popped node's location is after a Push operation which increments top (Line 4 of the Push algorithm). However, before the Push operation ends, the reference to the previously popped node is overwritten with the location of the newly 160 pushed node (Line 5 of the Push algorithm). When the last node is popped from the structure, the decrementing of the variable top returns it to its initial (empty) condition, −1. Thus, the test for underflow (an empty stack) is when top contains −1. When an underflow error occurs, the Pop operation returns a null reference to the client, as shown in the following pseudocode version of the algorithm. Figure 3.9 A Stack Object in its Initial, Intermediate, and Full States Pop Algorithm 161 Figure 3.10 shows the changes in the data members of a four element Stack object after performing two Pop operations on a stack that contains two objects: Mike's node and Vick's node. The first Pop operation returns the location of Vick's node to the client, and the second Pop operation returns the location of Mike's node to the client. As shown in the figure, the only change made to the Stack object's data members during the Pop operation is the decrementing of the data member top. The locations of the popped nodes are still stored in the array data. However, since top has been decremented during the Pop operations, these node locations cannot be accessed by subsequent Pop operations. Their inaccessibility to subsequent Pop operations has effectively removed the two nodes from the stack. The location of these popped nodes will eventually be overwritten by two Push operations (e.g., after Carol's and Bill's nodes are pushed as shown in Figure 3.11 ). 162 Figure 3.10 Pop Operations Performed on a Four Element Stack Object Containing Two Nodes It should be noted that, unlike the Fetch algorithms discussed in Chapter 2 , the Pop algorithm does not return a deep copy of the node popped (fetched) but rather passes the address of the actual node stored in the structure to the client. This is not a violation of encapsulation since the Pop operation, unlike a Fetch operation, deletes the node from the 163 structure. Thus, if the client subsequently makes changes to the popped node it does not affect the database stored in the stack. Performance of the Structure The performance of a Data Structure is dependent on the speed of its operations and the additional memory (above that necessary to store the nodes) required by the structure. We will discuss these two factors separately as we examine the performance of the structure Stack. Speed of the Structure We will perform a Big-O analysis to determine the approximate speed of the structure Stack, as n , the number of nodes stored in the structure, gets large. Once again we will only consider memory access instructions in this approximation technique because they are much slower than arithmetic, logic, and register transfer instructions. Examining the Push operation algorithm, Line 1 performs two memory accesses to fetch the contents of the memory cells top and size. When an overflow occurs, these are the only memory accesses performed. However, during a successful Push operation, Line 4 performs an additional memory access to store top's incremented value, and Line 5 performs an additional memory access to store the location of the copy of the pushed node in the array data. Therefore, the worst case from a speed viewpoint is a successful Push operation which requires four memory accesses (two for Line 1, one for Line 4, and one for Line 5) and the dominant (and only) term in the Push operation's speed function is 4, which is O(1). Examining the Pop algorithm, Line 1 requires one memory access to fetch the contents of the memory cell top. When an underflow occurs, this is the only memory access performed. However, when a Pop operation is successful, one memory access is performed on Line 5 to overwrite the value of memory cell top. In addition, Line 6 performs one memory access to fetch the location of the popped node from the array data. 3 Therefore, the worst case scenario is a successful Pop operation which requires three memory accesses (one for Line 1, one for Line 5, and one for Line 6) and the dominant (and only) term in the Pop operation's speed function is 3, which is O(1). 164 Figure 3.11 The Overwriting of Popped Node Locations by Push Operations 165 Figure 3.12 Density Variation of an Array-Based Stack with Node Width Density of the Structure The analysis of the density of the structure Stack is similar to the analysis performed to determine the density of the array-based structures presented in Chapter 2 . Density is defined as The information bytes are simply the product of the number of nodes stored in the structure, n , and the number of information bytes per node, w. Therefore, the information bytes is n * w. The total bytes allocated to the structure is the information bytes plus the overhead storage required to maintain the structure. The overhead is the array of n reference variables that point to the node objects, plus the integer variables top and size. In Java, reference variables and integers occupy four bytes. Therefore, the overhead is 4 * n 4 + 4, and the density of the structure, DS , can be expressed as which is approximately equal to 1 / (1 + 4/w) as n gets large. Figure 3.12 presents a graph of an approximation of DS for n greater than 100. The figure demonstrates that good densities (0.80 or higher) are achieved whenever the number of information bytes in a node, w, is greater than 16 bytes. Table 3.1 summarizes the performance of the Stack structure and includes the performance of the previously studied Unsorted-Optimized Array structure for comparative purposes. From a density viewpoint, the performance of the two structures is the same. However, the Stack structure is much faster than the Unsorted-Optimized Array structure since 166 the speed of its operations is not proportional to the number of nodes stored in the structure, n . Implementation The implementation of the classical Stack structure is presented in Figure 3.13 as the Java class Stack. It is implemented as a homogeneous, fully encapsulated structure that stores Listing objects. Its code, and the code of the class Listing (see Figure 3.14 ), are consistent with all of the generic design features discussed in Chapter 2 except that the stack can only store Listing class objects. A fully generic implementation of the structure, using the generic features of Java 5.0, will be presented later in the chapter. Line 2 of the class Stack shown in Figure 3.13 , declares the array data to be an array of references to Listing objects. As was the case for the structure implementations presented in Chapter 2 , the code of the class Stack does not mention any of the data members of a Listing object, which is a first step toward a generic implementation of this data structure. 167 Figure 3.13 Implementation of a Classical Stack Structure The class has two constructors that begin on Lines 5 and 10 that initialize the structure's data members and allocate the array data. The first constructor is a default constructor that allows for a maximum of 100 nodes to be stored on the stack before an overflow error occurs. The array of 100 reference variables is declared on Line 8. The second constructor includes an integer parameter (n on Line 10) which allows the client to specify the maximum number of nodes to be stored on the stack. Line 13 of this constructor allocates the array of n reference variables. The push and pop methods, which begin on Lines 15 and 24, are simply the Java equivalent of the Push and Pop algorithms' pseudocode previously presented in this chapter. The push method's heading (Line 15) 168 includes a Listing parameter which the client uses to specify the location of the node to be pushed onto the stack. The method returns a Boolean value set to false to indicate an unsuccessful operation (stack overflow). The heading of the Pop operation (Line 24) indicates that the method returns a reference to the Listing object, the object popped from the stack (null if the stack is empty). As mentioned in the discussion of the Pop algorithm, since the popped node is also deleted from the structure, returning a shallow copy of the node (Line 31) does not compromise the structure's encapsulation. The showAll method (Lines 34−37) outputs the nodes from the top of the stack to the bottom. This is done by initializing the output loop's variable (Line 35) to top and decrementing it every time through the loop. Figure 3.14 presents the class Listing that defines the nodes to be stored on the stack. It provides a method to deep copy a node and a toString method; implicit assumptions in the code of the class Stack. It is a shorter version of the class Listing presented in Chapter 2 (see Figure 2.16 ). Since restricted structures do not support the key field mode of access, a compareTo method need not be included in the class and (for brevity) it does not include a setAddress method for demonstrating encapsulation. Figure 3.15 presents a Stack application to demonstrate the use of the class Stack. The output it produces is presented in Figure 3.16 . 169 Figure 3.14 The Class Listing Consistent with the Three Assumptions Implicit in the Class Stack Figure 3.15 An Application Program that Utilizes the Class Stack 170 Figure 3.16 The Output Generated by the Application Program Presented in Figure 3.15 3.2.3 A Stack Application: Evaluation of Arithmetic Expressions The structure Stack is used extensively in computer science because the LIFO processing of information appears in the algorithms of many important applications. These applications include artificial intelligence, tree traversals, graph traversals (demonstrated in Chapter 7 and 9 ), and compilers. Specifically, compilers use stacks to pass information to and from subprograms, to “remember” the return addresses of nested subprogram invocations, and to evaluate arithmetic expressions. In most programming languages, arithmetic expressions are written in 171 infixed form because this is the way we are taught to write arithmetic strings in grammar school. When an arithmetic expression is written in infixed form, the operator is always written in between the operands. Thus, the infixed notation to add 2 and 4 is: 2 + 4. Arithmetic expressions involving more than one term written in infixed notation can be ambiguous. For example, consider the infixed expression: 2 + 3 * 4. This could evaluate to 20 = (2 + 3) * 4, or 14 = 2 + (3 * 4). When parentheses are not used, ambiguities are resolved by a set of rules called precedence rules. These rules of precedence state (among other things) that multiplication is always performed before addition. Thus, when we consider the rules of precedence, the multiplication contained in the infixed expression 2 +3*4isperformedfirst, and the expression evaluates to 14. If we want the addition to be performed first, we enclose the term 2 + 3 in parentheses. Parentheses override the rules of precedence. Thus, the infixed expression: (2 + 3) * 4 evaluates to 20. Dealing with the rules of precedence and groupings of parentheses at run-time would be a very time consuming process which would slow down the execution of programs involving multiple arithmetic expressions. As a result, modern language compilers translate arithmetic expressions into alternate notations at compile time. These alternate notations eliminate the ambiguities implicit in infixed notations, and thus eliminate the need to consider precedence rules and parentheses at run-time. The result is executable modules that run faster. Stated another way, programmers write arithmetic expressions in infixed notation using parentheses to override the rules of precedence. Compilers produce executable modules in which arithmetic expressions have been translated into a nonambiguous alternate notation that does not require precedence rules and/or parentheses to resolve ambiguities. The two alternate notations used by compilers are postfixed or prefixed notations. A post-fixed notation places the operator after the two operands, and a prefixed notation places the operator before the two operands. Thus, the postfixed notation for adding two and four is 2 4 +, while the same arithmetic expression is written in prefixed notation as + 2 4. The nice thing about these two notations is that they do not suffer from ambiguities. As a result they do not have precedence rules and parentheses need not be used to override these rules. Of these two notations, most compilers use postfixed notation in their executable modules. The infixed arithmetic expression: 2 + 3 * 4 is written in postfixed notation as 2 3 4 * +, which evaluates to 14, while the infixed 172 notation expression (2 + 3) * 4 is written in postfixed notation as: 2 3 + 4 * which evaluates to 20. After checking the syntax of an infixed notation arithmetic expression, a two-step process is used to generate the code used to evaluate it at run-time. The compiler: 1. Converts the infixed notation expression to the equivalent postfixed notation expression. 2. Applies the postfixed notation evaluation algorithm to the resulting expression. The data structure Stack is used in both of these steps. In this section, we will demonstrate the use of a stack in the second step of this process, the postfixed notation evaluation algorithm. The postfixed notation evaluation algorithm begins on the left side of the postfixed expression and searches through the expression until an operator is encountered, or the end of the expression is reached. When an operator is encountered, the operator is applied to the previous two operands in the expression, and the arithmetic result replaces the two operands and the operator in the postfixed expression. This process is continued from that point rightward replacing each operator encountered, and the two operands that precede it, with the result of applying the operator to the two operands. When the end of the expression is reached, the value of the entire expression is the numeric value just preceding the end of the expression. As an example, consider the postfixed expression 2 3 + 4 *, which should evaluate to 20. We begin on the left looking for operators. As such, we skip the 2 and the 3, and arrive at the operator +. This is applied to the two previous operands, 2 and 3, to obtain 5. The value 5 replaces the + operator and the 2 and 3 (its two previous operands) in the postfixed string, reducing it to the postfixed string: 5 4 *. Proceeding to the right looking for operators, the operand 4 is skipped and then the operator * is encountered. It is applied to the two previous operands, 5 and 4, which yields 20. This value replaces the two operands, 4 and 5, and the operator * to yield the new postfixed string, 20. Proceeding again to the right we encounter the end of the string, and so the value of the string is the previous (and only remaining) numeric value in the string, 20. The steps involved in the evaluation of the post-fixed string discussed in this paragraph (2 3 + 4 *) are shown in Table 3.2 along with the evaluation of the postfixed arithmetic expression 2 3 4 * +. Because an operator always operates on the two operands last 173 encountered, and then these operands are never used again (can be deleted), the data structure, Stack, is ideally suited for storing postfixed expression operands. As we proceed from left to right through the postfixed string, we push the operands onto a stack until we encounter an operator. Then the stack is popped twice to obtain the two previous operands, and the result of applying the operator to these two operands is pushed onto the stack. If the postfixed expression is syntactically correct, when we encounter the end of the expression string, there will be only one operand on the stack which is the value of the arithmetic expression. Figure 3.17 presents the postfixed arithmetic expression evaluation algorithm using a stack, s, to store the operands. In it, a single operand, or operator, is referred to as a token . Figure 3.18 shows the progression of the contents of the stack, s, as the postfixed string: 2 3 4 * + is evaluated using the algorithm depicted in Figure 3.17 . 3.2.4 Expanded Model of a Stack As previously mentioned, in the classical model of a Stack the Push and Pop operations are the only operations allowed on the structure. This model is often expanded to include methods to perform some, or all of the following operations: • To reinitialize the stack to empty. 174 • To test for an empty stack (underflow condition). • To test for a full stack (overflow condition). • To pop a node from the stack without deleting it from the structure. • To expand the stack at run-time within the limits of system memory. The method to reinitialize a stack to empty simply sets the data member top to its initial condition: top = -1. With top set to −1, a Pop operation will return an underflow error (which it should, since the stack has been reinitialized). The elements of the array data need not be set back to their initial condition (null ). To do so would be a waste of time because subsequent Push operations will overwrite the references to the nodes stored in the array before the stack was reinitialized. 5 Figure 3.17 The Postfixed Arithmetic Expression Evaluation Algorithm 175 Figure 3.18 The Progression of the Contents of a Stack used to Evaluate the Postfixed Arithmetic Expression: 2 3 4 * + The method to determine if a stack is empty simply tests the variable top: as does the method to determine if a stack is full: where size is the maximum number of nodes the structure can store. The method to fetch a node from the structure without deleting it is typically named: Peek . It is the same algorithm as the Pop algorithm except the variable top is not decremented during the operation (Line 30 of Figure 3.13 ). To allow the stack to expand at run-time, the algorithm used to expand arrays discussed in Chapter 2 (Section 2.4 ) is added to the Push operation. When a Push operation would result in an overflow condition, instead of returning false (Line 17 of Figure 3.13 ) the push method would expand the size of the stack's array, overwrite the variable size, and then add the node to the expanded structure. The only time the method would return false is when the system memory is exhausted. Chapter 4 will present an alternative implementation of an expandable stack that is not array-based. The implementation of all of these expanded features is left as an exercise for the student. 3.3 Queue The data structure Queue, like Stack, is a restricted data structure that shares an equally important role in computer science. The structure obtains its name from the concept of a waiting line that, by definition, is a queue. The term queue is not often used in normal conversation in the United States, but in England it is a very commonly used word. A person trying to cut into a ticket holder's line for a play in London would be told to “get at 176 the end of the queue ,” and the first person to be seated would be the person at the front of the queue. Similarly, the first node inserted into a Queue structure is always the first node returned from the structure by the combined Fetch-and-Delete operation, and nodes are inserted at the end of a queue. Because of this, the structure is referred to as a First-In-First-Out structure (FIFO). A proper understanding of the difference between a queue and a stack is that a queue is a “fair” waiting line, whereas a stack is a very “unfair” waiting line. 3.3.1 Queue Operations, Terminology, and Error Conditions We say that nodes are inserted into a queue and removed from a queue. As with all restricted structures, nodes are stored in chronological order in a queue. The node that has been in the structure the longest amount of time is said to be at the front of the queue, and the node that has been in the structure the least amount of time is said to be at the rear of the queue. The Insert operation on a Queue structure is called an Enqueue operation and the combined Fetch-and-Delete operation is called a Dequeue operation. Figure 3.4 illustrates the operational difference between the Queue and Stack data structures. It assumes that prior to performing the combined Fetch-and-Delete operation, the insertion order of the nodes into the structures were first A, then B, then C, and finally D. When there are no nodes in a queue, it is said to be empty . If the queue has a maximum node capacity, when this capacity is reached the queue is said to be full . A Dequeue operation cannot be performed on an empty queue, and an Enqueue operation cannot be performed on a full queue; both situations result in an error. The error associated with a Dequeue operation on an empty queue is called an underflow error, and the error associated with an Enqueue operation on a full queue is called an overflow error. A superficial difference between the Stack and Queue structures is the way they are abstractly depicted. As the name Stack would imply, nodes on a stack are always shown in a vertical arrangement (see Figure 3.5 ), while nodes in a queue are represented in a horizontal arrangement, as shown in Figure 3.19 . There is also a difference in the terminology used to refer to the first and last node inserted into these two structures. As we have learned, consistent with the vertical image of a stack, the last node inserted into a stack is said to be at the top of the stack, and the first node inserted into a 177 stack is said to be at the bottom of the stack. Consistent with the idea of a waiting line and the horizontal depiction of a queue, the first node inserted into the structure (e.g., Node A in Figure 3.20 ) is said to be at the front of the queue and the last node inserted into a queue (e.g., Node D in Figure 3.20 ) is said to be at the rear (or back ) of the queue. Enqueue (Insert) operations are always performed at the rear of a queue, and Dequeue (combined Fetch-and-Delete) operations are always performed at the front of a queue. Figure 3.21 shows the queue depicted in Figure 3.20 after node E is inserted and then after a Dequeue operation is performed. Figure 3.19 Abstract Depiction of a Queue Figure 3.20 The Nodes at the Front and Rear of a Queue Figure 3.21 A Queue After an Enqueue and Dequeue Operation 178 3.3.2 Classical Model of a Queue In the classical model of a Queue, the Enqueue and Dequeue operations are the only operations allowed on the structure, and the maximum number of nodes to be stored in the structure (size of the queue) is set when the structure is created. As is the case with the structure Stack, this classical model of a Queue is often expanded to include other operations and features such as: • The ability to reinitialize the queue to empty. • The ability to test for an empty queue (underflow condition). • The ability to test for a full queue (overflow condition). • The ability to fetch a node from the queue without deleting it from the structure. 6 • Expand the queue at run-time within the limits of system memory. In the interest of simplicity, we will ignore these expanded features for now and limit our discussion of the structure Queue to the classical model and its implementation. Once an understanding of the classical model is obtained, we will discuss the changes necessary to incorporate the expanded features. The last feature, the ability to store an unlimited number of nodes, will be discussed again when we study linked lists in Chapter 4 . For now, an attempt to add more than a specified maximum number of nodes to the queue will result in an overflow error. Operation Algorithms Our classical model of a Queue will be a fully encapsulated structure that supports two operations: Enqueue is used to insert nodes into the structure, and Dequeue is used to fetch (and delete) nodes from the structure. An n element array of object references named data will be used to store the locations of the nodes inserted into the queue. In addition to this array, the two operation algorithms require that each Queue object contains four other integer data members: • size, used to store the maximum number of nodes the queue can hold. • numOfNodes, used to store the number of nodes currently stored in the structure. • front, used to store the index into the array data where the next Dequeue operation will be performed. • rear, used to store the index into the array data where the next Enqueue operation will be performed. Thus, when an Enqueue operation is performed, a reference to the 179 newly inserted node is stored in data[rear], and data[front] stores the address of the next node to be fetched (and deleted) from the queue. Figure 3.22 shows all of the data members of a Queue object capable of storing four nodes 7 shown in their initialized state. Figure 3.22 The Data Members of a Four-Node Queue in their Initialized State 180 Figure 3.23 A Queue Object in its Initial, Intermediate, and Full States The locations of the nodes inserted onto the queue are stored in the array data sequentially in the order in which they are inserted, beginning at element zero. Thus, the location of the first node inserted into a queue object is stored in data[0], the location of the second node inserted is stored in data[1], etc. Initializing the memory cell rear to 0, and incrementing it after the inserted node's location is set into data[rear] guarantees that the location of the first node inserted will be stored at location zero, and that all subsequent nodes inserted into the queue will be stored sequentially. Figure 3.23 shows the data members of a four-element Queue object in its initial state, after three Enqueue operations have been performed 181 (first Mike's node was added, then Vick's node, and finally Carol's node), and in its full state (after Bill's node is inserted into the queue). The test for a full queue is when the memory cells size and numOfNodes are equal. When we implement a queue, a problem develops that does not surface during the implementation of a stack. Consider the full queue depicted at the bottom of Figure 3.23 . Since front contains the element number of the array data that stores the location of the node to be fetched next, when a Dequeue (Fetch-and-Delete) operation is performed, the node referenced by data[0] is returned and front is incremented to prepare for the next Dequeue operation. After this operation the queue is no longer full, and element zero of the array data is now available to store the location of the next node inserted into the queue. However (as depicted in Figure 3.23 ) every time a node is inserted into the queue the variable rear is simply incremented, and therefore it will never return to its initial value, 0. Thus, the next insert will not utilize element zero of the array data. Worse yet, simply incrementing the value of rear has set it to 4 (as shown in the lower portion of Figure 3.23 ), which is an illegal (out of bounds) index into the array data. The solution to this dilemma is the concept of a circular queue. All queues implemented using arrays utilize this concept. When the memory cell rear of a four-element circular queue reaches the value 3, it is not incremented during the Enqueue operation. Rather, it is reset to zero to “reclaim” the unused portion of the array data. Therefore, the memory cell rear, associated with a four-element queue, takes on the values 0, 1, 2, 3, 0, 1, 2, 3, 0, etc. There are two ways of implementing this. One is: The alternate method, which is the method more commonly used is: where % is the modulus (remainder) operator. Thus, the Enqueue algorithm to add the node whose location is stored in the memory cell newNode is: Enqueue Algorithm 182 Line 5 of the algorithm enforces the encapsulation of the data structure by performing a deep copy of the node to be added to the structure and then storing the location of the cloned node in the structure. Let us now turn our attention to the Dequeue (combined Fetch-and- Delete) operation algorithm. Since nodes are stored in chronological order in the array data, after a node is fetched from the queue, the element of the array just “after” the node fetched from the queue contains the location of the next node to be fetched. For example, if the node whose location is stored in data[1] was just fetched from the queue, the location of the next node fetched would be stored in data[2]. Thus, the memory cell front must be incremented during a Dequeue operation to keep track of the next node to be fetched from the queue. Eventually, front will store a value that, if incremented, will exceed the bounds of the array. To prevent this from happening, just as in the Enqueue algorithm, the modulus operator is used during the incrementing of the variable front: Its use causes the variable front to remain in the range 0 to size - 1, and prevents an “array-outof-bounds” error from occurring during a Dequeue operation. A successful Dequeue operation returns the location of the node at the front of the queue; otherwise, a null value is returned. The test for an empty queue is when the variable numOfNodes contains a zero. Thus, the Dequeue algorithm is: Dequeue Algorithm 183 Figure 3.24 shows the changes in the data members of a four-element queue after performing two Dequeue operations on a queue that contains three objects: Mike's node, Vick's node, and Carol's node. The first Dequeue operation returns the location of Mike's node to the client, and the second Dequeue operation returns the location of Vick's node. As shown in the figure, the only changes made to the Queue object's data members during these Dequeue operations are the incrementing of the data member front and the decrementing of the memory cell numOfNodes. The locations of the fetched nodes are still stored in the array data; however, the changes made to front and numOfNodes during the Dequeue operations prevent these elements from being accessed until they are overwritten by subsequent Enqueue operations. The fact that they are inaccessible until they are overwritten has effectively removed the two nodes from the queue. 184 Figure 3.24 Data Member Changes as a Result of Two Dequeue Operations The overwriting process is shown at the bottom of Figure 3.25 , which depicts the changes to the data members as a result of the insertion of Ryan's node into the queue. After Bill's node was inserted into the structure (middle section of the figure), rear has recycled to zero (as a result of the modulo size arithmetic on Line 6 of the Enqueue algorithm). 185 This reclaims the element zero of the array data, previously used to store a reference to Mike's node, which is then overwritten with the location of Ryan's node. Before ending our discussion of the Dequeue algorithm, it should be noted that unlike the Fetch algorithms associated with the array-based structures discussed in Chapter 2 , and like the Pop operation discussed previously in this chapter, the Dequeue algorithm does not return a deep copy of the node fetched. Rather it passes the address of the actual node stored in the structure to the client. This is not a violation of encapsulation since the Dequeue and Pop operations, unlike a Fetch operation, deletes the returned node from the structure. Thus, if the client subsequently makes changes to the node it does not affect the database stored in the queue. Performance of the Structure We will now turn our attention to the performance of the structure Queue considering first the speed of the structure, and then the additional memory (above that necessary to store the nodes' information), required to maintain the structure. Speed of the Structure In the interest of consistency, we will again include only memory access instructions in our Big-O analysis since it is an approximation technique and the time to perform nonmemory access instruction is negligible within this approximation. Examining the Enqueue operation algorithm, Line 1 performs two memory accesses (one to fetch the contents of the memory cell numOfNodes, and the other to fetch the contents of the memory cell size). When an overflow occurs, these are the only memory accesses performed. However, when an Enqueue operation is successful, Line 4 performs one additional memory access (to rewrite the incremented value of numOfNodes). Line 5 requires two memory accesses (one to fetch the value of the memory cell rear, and the other to store the location of the copy of the inserted node in the array data). Finally, Line 6 requires one additional memory access to store the incremented value or rear. Therefore, the worst case scenario from a speed viewpoint is a successful Enqueue operation which performs six memory accesses (two for Line 1, one for Line 4, two for Line 5, and one for Line 6) and the dominant (and only) term in the Enqueue operation's speed function is 6, which is O(1). The dominant term in the Dequeue operation's speed function is also 186 O(1). When an underflow error occurs, only one memory access is performed (Line 1). However, when the structure is not empty, Line 4 requires one additional memory access to fetch the value of the memory cell front, and Line 5 requires two memory accesses (one to access the memory cell size, and one to overwrite the value of memory cell front). Line 6 requires one memory access to overwrite the contents of the memory cell numOfNodes, and Line 7 requires one memory access to fetch the location of the fetched node from the array data. In all, six memory accesses (one for Line 1, one for Line 4, two for Line 5, one for Line 6, and one for Line 7) are performed, O(1), to complete a successful Dequeue operation. 187 Figure 3.25 Data Member Changes as a Result of Two Enqueue Operations Density of the Structure The analysis of the density of the structure Queue is the same as the analysis previously performed to determine the density of the Stack structure, except that a Queue uses an additional eight bytes of overhead. Both structures require n reference variables to point to the client's node objects, and both structures use an integer variable size to 188 store the size of the array data. A stack requires one addition variable, top, to keep track of the top of the stack, while three variables numOfNodes, front, and rear are required to maintain a queue. Remembering that reference variables and integers occupy four bytes, the overhead is 4 * n + 4 * 4 bytes, and the density of the structure Queue, DQ , can be expressed as: where w is the width of the client's nodes. As the number of nodes, n , gets large 16 / (n * w) approaches zero and DQ ≈ (1 + 4/w). This is the same expression we obtained for the density of the structure Stack as n gets large (> 100), which is plotted in
Figure 3.12 . The figure demonstrates that good densities (0.80 or higher)
are achieved whenever the number of bytes in a node is greater than
sixteen.
Table 3.3 summarizes the performance of the Queue structure and
includes the performance of the two previous structures for comparative
purposes. From a density viewpoint, the performance of all three structures
is the same. The Queue structure is only slightly slower than the Stack
structure (with a calculated average speed of 6 accesses vs. 3.5 accesses);
however, like the Stack, the Queue is much faster than the array structure
since its speed is also not a function of n , the number of nodes stored in
the structure.
Implementation
The implementation of the classical Queue structure is presented in
Figure 3.26 as the class Queue. It is implemented as a homogeneous, fully
encapsulated structure that stores Listing nodes. Its code, and the code of
the class Listing (see Figure 3.14 ), are consistent with all of the generic
design features discussed in Chapter 2 except that the stack can only store
Listing class objects. A fully generic implementation of the structure,
using the generic features of Java 5.0 and the methodology presented in
Section 3.4 , is left as an exercise for the student.
Line 2 of the class Queue declares the array data to be an array of
references to Listing objects. As was the case for the previous structures
we implemented, the code of the class does not mention any of the data
189

members of a Listing object, which is a first step toward a generic
implementation of this data structure.
190

Figure 3.26 Implementation of a Classical Queue
The class has two constructors that begin on Lines 7 and 14. The first
of these constructors is a default constructor that allows for a maximum of
100 nodes to be stored in the Queue before an overflow error occurs. The
array of 100 reference variables is declared on Line 12. The second
constructor includes an integer parameter (n on Line 14) that allows the
client to specify the maximum number of nodes to be stored in the Queue.
191

Line 19 of this constructor allocates the array of n reference variables.
The enque and deque methods, which begin on Lines 21 and 31
respectively, are simply the Java equivalent of the pseudocode versions of
the Enqueue and Dequeue algorithms presented earlier in this chapter. The
method enque’s heading (Line 21) includes a Listing parameter that the
client uses to pass the location of the node to be inserted into the method.
A Boolean value is returned to indicate overflow (false if an overflow
occurs). The heading of the deque method (Line 31) indicates that it
returns a reference to the Listing object fetched from the queue (null if the
queue is empty). As mentioned in the discussion of the Stack structure,
since the fetched node is also deleted from the structure, returning a
shallow copy of the node (Line 39) does not compromise the structure’s
encapsulation.
The showAll method (Lines 42−48) outputs the nodes from the front
of the Queue to the rear. This is done by counting the number of nodes
output until the total is equal to the number of nodes in the structure (Line
44). Each time through the loop, the index i, which was initialized to the
front of the queue on Line 43, is incremented using modulo size arithmetic
(Line 46).
As mentioned previously, Figure 3.14 presents the class Listing that
describes the nodes stored by our Queue implementation. It provides a
method to deep copy a node and a toString method, an assumption implicit
in the code of the class Queue. Figure 3.27 presents an application
program to demonstrate the use of the class Queue. The output it produces
is presented in Figure 3.28 .
3.3.3 Queue Applications
Queue’s FIFO characteristic models a waiting line (which is how the
structure received its name). As such, it is used extensively in computer
applications where information is to be processed in the order it is
received. One such application is a print spooler for a multitasking
operating system. When several applications that produce output are
running at the same time 9 and share a single printer, the output they
produce is buffered to disk files. When an application completes its output,
the disk file is added to a queue of files that are being written to the
printer. As a result, the output from these applications appears on the
printer in the order in which these applications complete their execution.
Queues are also used extensively in the field of operations research,
192

in some sorting algorithms, in the graph traversal algorithms discussed in
Chapter 9 , and in the algorithm used to convert infixed strings to postfixed
strings.
Figure 3.27 An Application Program that Utilizes the Class Queue
3.3.4 Expanded Model of a Queue
In the classical model of a Queue, the Enqueue and Dequeue
operations are the only operations permitted on the structure. Often, as
previously discussed, restricted structures are expanded to include
additional features (repeated here for convenience).
• To reinitialize the queue to empty.
• To test for an empty queue (underflow condition).
• To test for a full queue (overflow condition).
• To fetch a node from the queue without deleting it from the structure.
• To expand the queue at run-time.
193

The method to reinitialize a queue simply sets the variables front,
rear, and numOfNodes back to their initial values. The elements of the
array data need not be set back to their initial condition (null ). To do so
would be a waste of time because subsequent Enqueue operations will
overwrite the references to the nodes stored in the array before the queue
was reinitialized.
Figure 3.28 The Output Generated by the Application Presented in
Figure 3.27
The method to determine if a queue is empty simply tests the variable
numOfNodes to determine if it is zero. If the variables size and
numOfNodes are equal, the queue is full. The method to fetch a node from
the structure without deleting it, typically named peek, is the same
algorithm as the deque method except the variable front is not incremented
and numOfNodes is not decremented.
194

To allow the queue to expand at run-time, the algorithm used to
expand arrays discussed in Chapter 2 (Section 2.4 ) is added to the enque
method. When an Enqueue operation would result in an overflow
condition, instead of the method returning false (Line 23 of Figure 3.26 ) it
would expand the size of the queue’s array, adjust the variables front, rear,
and size and then add the node to the expanded structure. The only time
the method would return false is when the system memory is exhausted.
Chapter 4 will present an alternative implementation of an expandable
restricted structure that is not array-based.
The implementation of all of these expanded features is left as an
exercise for the student.
3.4 Generic Implementation of the Classic
Stack, a Methodized Approach
Earlier in this chapter we implemented a classic Stack structure
(Figure 3.13 ) in a way that could be easily modified into a generic
implementation. It separated the node definition and the data structure into
two separate classes, and the data structure class did not mention the field
names of the nodes. In addition, the node definition class contained a
toString method and a deepCopy method. As coded, however, it can only
store objects that are instances of a class named Listing. In this section we
will modify the class to make it fully generic; that is, it will be able to store
objects of any class. The generic typing features of Java 5.0 discussed in
Chapters 1 and 2 will be used to generalize the class. In describing this
generalization, we will take a different pedagogical approach than we did
in Chapter 2 (Section 2.4 ). Here, we will methodize the process of
converting the code (presented in Figure 3.13 ) into a four-step process.
3.4.1 Generic Conversion Methodology
Step One Since we are trying to eliminate the dependence of the
class’ code on the type Listing, the first step in this process is to add a
generic placeholder (in this case ) at the end of the method’s heading
(Line 1 in Figure 3.29 ). This will be the generic name of the node type
stored in the structure.
Step Two The next step is to replace all references to the type Listing
with our generic type. This gives us the highlighted replacements on the
left side of Lines 2, 8, 13, 15, and 24 of Figure 3.29 . (For now, ignore the
195

fact that the code on Lines 8 and 13 have been commented out and recoded
on the right side of these lines.) It would be nice if this were all we had to
do, but Java is a heavily typed language and so the process continues.
Figure 3.29 Some of the Generic Revisions of the Class Stack
Presented in Figure 3.13
Step Three Wherever the keyword new operates on the placeholder
T, we must substitute the class Object for the placeholder T. The translator
will not let us generate a reference to the type T; it is an type.
This is the case on Lines 8 and 13. The fully corrected versions of lines 8
and 13 appear on the right side of the lines, which also includes coercion,
because the variable data is now declared on Line 2 to be a T reference.
196

Figure 3.30 The Interface GenericNode
Once we substitute Object for T on these lines, the translator gives us
some additional help via an incompatible type error: “found:
java.lang.Object[], required: T[]”; type T[] is required to match the new
type of the variable data.
Step Four Wherever a method is invoked that operates on an object
of type T, we can expect a “cannot find symbol” compile error. This is
because the translator cannot look into the class T to verify the existence,
or at least the signature, of the method. The invocation of the method
deepCopy on Line 20 in Figure 2.39 is an example of this problem,
because the type of newNode was changed to T on Line 15 in Step 2. The
remedy here is to collect all the method signatures into an interface, and
implement the interface 10 in the node definition class. The corrected
version of Line 20 is coded on the right side of the line, which requires that
the symbol node be defined as a local variable in the push method (and
that the reference returned from the method deepCopy be coerced into the
array data, T). Assuming the name of the interface is GenericNode, the
declaration of the variable node would be:
which must be added to the push method. It not only defines the
symbol node, but also directs the translator to look into the interface
GenericNode for deepCopy’s signature (which was our Step Four
problem). The only remaining issue is to code the interface GenericNode
and add the phrase “implements GenericNode” to the end of heading of the
node definition class (problem solved).
The code of the interface is given in Figure 3.30 , and the final
version of the generic code, the class GenericStack, is given in Figure 3.31
. All the additions and revisions to the code of the class presented in Figure
3.13 to make it fully generic are highlighted.
Consistent with the idea of divide and conquer, it is often easier to
code a fully generic data structure, as we did, in two steps. First, the data
structure class and a node class used to debug it are coded using the design
features incorporated into the classes depicted in Figures 3.13 and 3.14 .
197

Once the data structure class is debugged, then the four-step methodology
developed in this section is used to convert it to a fully generic data
structure class.
Figure 3.31 Generic Version of the Class Stack Presented in Figure
3.13
3.5 Priority Queues
Priority queues can be thought of as a restricted data structure in that
they normally support just an Insert operation (often named Add) and a
combined Fetch-and-Delete operation (often named Poll). In addition, the
access to the nodes in the structure is restricted in that they are fetched and
198

deleted in an order based on a priority assigned to each node. Normally,
the node with the lowest priority is returned from a Poll operation,
although some priority queues return the node with the highest priority.
Various strategies are used to decide which node should be returned in the
event of a priority “tie”. In many cases priority ties are resolved by
returning the node that has been in the structure the longest, thus the name
Priority Queue .
Priority Queues can be implemented in a variety of ways, but one of
the most efficient ways is to use a scheme called a heap, which will be
examined in Chapter 8 . The implementation of a Priority Queue is left as
an exercise for the student in that chapter.
3.6 Java’s Stack Class
The Java Application Programmer Interface provides a class named
Stack that is implemented as a generic , expandable , but unencapsulated
data structure. The structure stores objects , not primitive types, 11 and can
be used either as a compiler enforced homogeneous or a heterogeneous
structure. An EmptyStackException is thrown when a Pop operation is
performed on an empty Stack object. The structure is dynamic and
expands at run-time to accommodate an unlimited number of Push
operations. A no-parameter constructor is provided to create Stack objects.
For example, the declaration
creates a compiler enforced empty stack named NYC that can only
store Node objects.
Table 3.4 presents the names of the basic operation methods in the
class Stack and some examples of their use. It assumes that the Stack
structure boston was declared as a heterogeneous structure
that the variable tom, mary, and temp can store references to Node
objects, and that the coding examples are performed in the order shown
(from the top row to the bottom row).
199

EXERCISES
Knowledge Exercises
1. Which of the two access modes cannot be used to access restricted
structures?
2. Which of the four basic operations do restricted structures support
(indicate if any are combined)?
3. Name two restricted structures.
4. Tell what the following acronyms stand for and which restricted
structures they are associated with:
a) LIFO
b) FIFO
5. Give the names of the operations that can be performed on the
following structures, and tell what the operations do.
a) A Stack
b) A Queue
6. Nodes A, B, and C are placed on a stack in the order first A, then B,
and finally C.
a) Draw a picture of the stack using the standard abstract graphic.
b) What would be stored in the variable top?
200

c) A Pop operation is performed. What node is returned?
7. What error occurs if
a) A Pop operation is performed on an empty stack?
b) A Push operation is performed on a full stack?
8. Describe the action of the stack Peek operation.
9. In the implementations of the Stack operation presented in this
chapter, what does the memory cell top store, the index of the array
where the nextPush or the nextPop will be performed?
10. Rewrite the stack Push algorithm presented in this chapter
assuming that top was initialized to 0 instead of −1.
11. Rewrite the stack Pop algorithm presented in this chapter assuming
that top was initialized to 0 instead of −1.
12. Give the line numbers of the code presented in Figure 3.13 that
perform the garbage collection for the structure Stack.
13. Evaluate the following arithmetic expressions written in postfixed
notation:
a) 45 3 21 + − 10 *
b) 3 6 * 45 2 + *
c) 12 3 * 2 /
14. Write the following arithmetic expressions in postfixed notation:
a) 45 + 6/2
b) (3 + 4 + 7) / 2
c) (b2 − 4 * a c) / (2 a)
15. Nodes A, B, and C are placed on an initialized queue in the order
first A, then B, and finally C.
a) Draw a picture of the queue using the standard abstract graphic.
b) Indicate the position of the rear of the queue.
c) Indicate the position of the front of the queue
d) A Dequeue operation is performed. What node is returned?
16. In the implementations of the structure Queue presented in this
chapter, what is stored in the memory cells front and rear?
17. Give the integer range of the values that rear can assume after the
statement: rear = x % 54; executes (assume x is an integer).
201

18. Give the line numbers of the code presented in Figure 3.26 that
perform the garbage collection for the structure Queue.
Programming Exercises
19. Expand the implementation of the class Stack presented in Figure
3.13 to include methods to: reinitialize the stack to empty, test for an
empty stack (underflow condition), test for a full stack (overflow
condition), and to perform a Peek operation. Include a progressively
developed driver program to demonstrate the functionality of each new
method.
20. Expand the implementation of the class Stack presented in Figure
3.13 so that it expands every time a Push operation is performed that
would cause an overflow. Initially, it should accommodate a maximum
of three nodes. Include a progressively developed driver program to
demonstrate the functionality of each new class.
21. Write a program to evaluate an arithmetic expression written in
postfixed notation. The arithmetic expression will be input as a String
(by the user) and will contain only integer operands. Use the following
code sequence to parse (i.e., remove) the integers and the operators
from the input string, mathExpression:
22. Expand the implementation of the class Queue presented in Figure
3.26 to include methods to reinitialize the queue to empty, test for an
empty queue (underflow condition), test for a full queue (overflow
condition), and to perform a Peek operation. Include a progressively
developed driver program to demonstrate the functionality of each new
method.
23. Using the generic capabilities of Java 5.0, modify the
implementation of the structure Queue presented in Figure 3.26 to
make it generic. Include a driver program that demonstrates the
functionality of the class with two homogeneous Queue objects that
store two different kinds of nodes.
24. Write a program to convert an infixed arithmetic expression that
202

includes nested parentheses to the equivalent postfixed string. The
infixed expression will be input from the keyboard, and the postfixed
expression should be output to the console.
25. Code a GUI program that visually demonstrates the changes to
contents of the array, and the other data members that make up a
Queue object (see Figure 3.23 ) when an Enqueue or a Dequeue
operation is performed. When the program is launched, the structure
should be shown in its initialized state. Four buttons should be
available to the user: an “Enqueue” button, a “Dequeue” button, a
“reinitialize” button, and a “quit” button. Provide text boxes for the
input of the nodes’ fields and for the output of a fetched node.
26. Code a GUI program that visually demonstrates the changes to
contents of the array, and the other data members that make up a Stack
object (see Figure 3.9 ) when a Push or a Pop operation is performed.
When the program is launched, the structure should be shown in its
initialized state. Four buttons should be available to the user: a “Push”
button, a “Pop” button, a “reinitialize” button, and a “quit” button.
Provide text boxes for the input of the nodes’ fields and for the output
of a fetched node.
27. Marty the mouse is to navigate his way through a ten-foot by
twenty-foot rectangular room whose floor is made of red and white
one-foot square tiles. Marty hates red tiles and will only walk on white
tiles. There is a piece of cheese at the exit. Write a program that
outputs the row and column numbers of the white tiles Marty walks on
as he finds his way to the cheese. Model the floor of the room using a
two-dimensional array of integers, one element per tile. A value of 0
indicates a white tile, a value of 1 indicates a red tile. Assume the maze
is navigable, and that the maze entrance location and exit location will
be input by the user. (Hint: you will need two stacks for this
application. One will store the path Marty has followed so he can
backtrack if he encounters a dead end. A special location is pushed on
this stack whenever Marty is at a tile from which he can proceed in
more than one direction. As Marty steps onto a tile, if there are
alternate tiles he could have stepped onto they are pushed onto the
second stack.)
1 This operation is commonly referred to as a Peek operation.
2 That is, within the limits of the system’s available storage.
3 An optimizing compiler would store the variable topLocation used on Lines 4 and 6 of
the algorithm in a CPU register. Thus, no memory accesses would be required to assign, or
203

use, the value stored in it.
5 The only advantage to setting all elements of the array data to null is that it more
rapidly returns the copies of the client’s nodes to the Java memory manager.
6 This operation is commonly named a Peek operation.
7 Although the standard graphic of a queue exhibits the nodes in a horizontal
arrangement, the array data used to store references to the nodes will be shown arranged
vertically inside a queue object.
9 Actually applications only appear to be running at the same time on a multitasking
operating system. In fact they share the CPU in sequential time slices, but the time slices are
so small that the applications appear to be running simultaneously.
10 A class that implements an interface codes all of the methods whose signatures are
defined in the interface. In addition, the phrase “implements interfaceName” must appear at
the end of the class’ heading.
11 Java 5.0 gives the appearance of allowing primitives to be inserted into a Stack
object, but it actually wraps the primitive in a Wrapper object before inserting it.
204

CHAPTER 4
Linked Lists and Iterators
OBJECTIVES
The objectives of this chapter are to familiarize the student with the
features, implementation, and uses of linked structures and list iterators.
More specifically, the student will be able to
Explain the advantages of linked structures and be able to quantify
their performance.
Understand the memory model programmers use to represent linked
structures and the resulting advantages and disadvantages of this
representation.
Understand the classic linked structures Singly Linked list (and its
circular, sorted, and double-ended variations), Doubly Linked list, and
Multilinked lists.
Implement a fully encapsulated version of any of the classic linked
structures.
Understand the ways in which a linked structure can be used to
implement a dynamically expandable Stack or Queue and the
advantages of that implementation.
Understand list iterators; be familiar with their access advantages
and the classic operations used to position iterators, be able to identify
applications that benefit from their use, and be able to quantify their
performance advantages.
Write an application that attaches an iterator to a data structure and
use the iterator to access the structure.
Implement several types of iterator classes including one that allows
the application programmer to declare multiple iterator objects, attach
them to the application’s structure, and access the structure using them.
205

Develop an application that declares objects in Java’s API
LinkedList class, understand the advantages and disadvantages of the
class, and operate on the structure using the class’ operation methods.
Develop an application that attaches a Java API ListIterator object to
a Java LinkedList object and access the linked list structure using the
iterator.
4.1 Noncontiguous Structures
All of the data structures we have discussed in the previous chapters
use an array to store references to the data set. When an array is created the
system’s memory manager must locate a contiguous portion of memory
large enough to accommodate the array, since arrays are always stored in
contiguous memory. In order to do this, the size of the array must be
known. That is why the array-based implementations developed in the
previous chapters either required the client to specify the maximum
number of nodes that would be stored in the structure, or the maximum
was set to a default value by the structure’s constructor. In either case, the
maximum value was used by the memory manager to locate and allocate
the contiguous portion of memory for the array’s elements.
A data structure that does not require contiguous memory can, at
times, be very advantageous. Consider three application programs: A, B,
and C, all running on a system at the same time with the system’s RAM
memory shared as shown in Figure 4.1 . Although there are still 60
kilobytes (KB) of unused memory, it is fragmented into three 20KB
sections. Thus, the largest RAM resident array-based structure that could
be declared by any of the three applications is one that could store 5000
nodes, because the structure’s 5000 element array of reference variables
requires 20KB of storage (4 bytes per reference variable). An attempt to
declare a larger array would either produce an “insufficient system
memory” error or cause the operating system to allocate the storage further
up the memory hierarchy, thus slowing down the application. However,
any of the applications using a noncontiguous structure could expand by
60KB within RAM memory since it could use all three of the unused
portions of RAM.
In this chapter, we will study the most fundamental category of
noncontiguous data structures: linked lists. These structures, like all linked
structures, can rapidly expand to accommodate a virtually unlimited
number of nodes, one node at a time. Aside from its compatibility with
206

fragmented memory, a subtle but important advantage this structure has
over an array-based structure is that it always uses all the memory assigned
to it. Consider an array-based structure that can accommodate one million
nodes with only half of the nodes currently inserted into it. At this point in
the structure’s life, it has been allocated 2MB (= 4 bytes × 500,000
elements) of storage it really does not need since half of the elements of
the array are not being used. Linked lists, like noncontiguous structures,
are more frugal. They are only assigned as much storage as they currently
need.
Figure 4.1 Three Applications Sharing Main Memory
There are many applications in which the maximum number of nodes
in the structure cannot be anticipated with any certainty at the time the data
structure is created. Consider an application that stores information
transmitted from a deep space probe whose mission is to discover new
stars and planets. The very nature of the mission precludes our ability to
predict the maximum number of heavenly bodies that will make up the
data set. These types of applications require dynamic data structures;
structures that can be expanded at run-time. If the implementation
language does not provide the ability to rapidly expand an array (as Java
does), then noncontiguous structures are the only viable alternative.
Linked lists, implemented in all programming languages, are so easily
expanded that at run-time their insert algorithm expands the structure
every time a node is added to it.
Because of their ability to utilize fragmented memory, utilize all the
memory assigned to them, and expand rapidly regardless of the
implementation language, noncontiguous structures such as linked lists are
207

widely used.
4.2 Linked Lists
There are several types of linked lists, but all of them share two
common characteristics:
• Every node in the structure has at least one field, called a link field,
that stores the location of another node (with the exception of the unique
last node, if there is one).
• Each node’s location is stored in at least one other node (with the
exception of the unique first node, if there is one).
Figure 4.2 a illustrates these two characteristics using a data set that
contains four nodes: X, B, T, and G. Rather than storing the nodes
addressed in a fixed size array, each node has two link fields that contain
the addresses of other nodes. Considering Node T to be the unique first
node (see Figure 4.2 b ), we can follow either the first (left) or the second
(right) link field contents (beginning at Node T) to locate all the other
nodes. For example, starting at Node T and “following” the second link
fields, Node G, stored at location 973, is encountered next. This is
followed by Node X stored at location 20, and finally Node B, stored at
location 332. Since the node locations can be determined in this way (by
traveling “through” the other nodes), we no longer need to store the node
references in contiguous memory, an assumption implicit in the access
function used to locate array elements. Link fields free us from the
constraint of contiguous memory; however, the sequential nature of
accessing linked lists through the link fields tends to reduce the speed of
these structures. The process of moving from the first node to the second
node to the third node, etc., by using the contents of the link fields to
locate the next node, is called traversing the list.
208

Figure 4.2a A Linked List with Two Link Fields
Figure 4.2b One Sequential Access Path Through the Nodes of a
Linked List
4.3 Singly Linked Lists
The simplest form of a linked list is the singly linked list. A singly
linked list is one in which:
• Each node has one link field.
• The nodes form a linear list:
209

• there is a unique first node, n 1 ,
• there is a unique last node n j , and
• any other node n k is proceeded by node n k −1 and followed by node n k +1 .
As an example, Figure 4.3 shows the nodes X, B, T, and G arranged
in a singly linked list. Each node has one link field, and Node T is the
unique first node. Its address is stored in a single reference variable called
the list header . Node B is the unique last node. Each of the other two
nodes, X and G, have a unique node just before it (T is before X, and X is
before G) and a unique node just after it (G is after X, and B is after G).
Although Figure 4.3 accurately depicts the arrangement of the nodes
in memory, most often singly linked lists are not illustrated this way.
Rather, the “standard” graphical depiction of a singly linked list, shown in
Figure 4.4 , is used to depict the nodes. In the standard depiction, the nodes
are shown horizontally, in traversal order, with the unique first node on the
left and the unique last node on the right. The list header is drawn above
the unique first node. Arrows emanating from the link fields are used to
point to the next node in the sequence. A series of dots indicates that one
or more nodes in the list have not been drawn.
Figure 4.3 Four Nodes: X, B, G, and T Stored in a Singly Linked List
Representing a linked list in this way often makes it easier to develop
and understand the basic operation algorithms for singly linked lists. What
is sometimes forgotten when using this standard graphical depiction is
that, although the nodes are shown “next” to each other, they are indeed
210

scattered around in memory. This fact is more easily remembered when we
add memory addresses to the standard depiction of a singly linked list, as
shown in Figure 4.5 . (For brevity, the list header reference variable is
simply denoted as h in this figure.)
4.3.1 Basic Operation Algorithms
The basic operation algorithms for a singly linked list can be a bit
confusing because they all involve manipulating the link fields of the
nodes in the structure. The best way to approach the discovery of these
algorithms, or any other algorithm associated with linked lists, is to first
draw a picture of the structure and then modify the picture to incorporate
the changes required to perform the operation. After the operation
algorithm has been developed graphically, it is then verified and
transformed into pseudocode.
Figure 4.4 Standard Depictions of Singly Linked Lists
Figure 4.5 A Standard Depiction of a Singly Linked List with Node
Addresses and Link Field Contents Shown
As an example, consider an algorithm to position a new node,
referenced by the variable r, into our four-node singly linked list between
211

Nodes X and G (referenced by q and p). The first step in developing the
algorithm is to draw a picture of the structure before the operation is
performed (see Figure 4.6 ). The modifications necessary to perform the
insert are then added to the picture. Since the new node is to be inserted
between Nodes X and G, two modifications are necessary:
1. The arrow coming from the link field of Node X must be changed to
point to the new node.
2. The link field of the new node must be made to point to the Node G.
These modifications are shown as the dotted arrows in Figure 4.7 ,
numbered 1 and 2.
Before coding the algorithm, we verify it graphically by moving
down (or traversing) the linked list to make sure that the adjusted link
fields have the effect of inserting the new node in between Nodes X and G.
Starting the list header, we first encounter (or visit) Node T. T’s link field
brings us to Node X, X’s link field now brings us to the new node, the new
node’s link field brings us to Node G, and G’s link field brings us to Node
B. Thus, the order of the nodes is T, X, the new node, G, and B, which
verifies the algorithm.
Figure 4.6 A Four-Node Linked List before the Insertion of the Node
Referenced by r
212

Figure 4.7 The Graphical Representation of the Two-Step Algorithm
to Insert the Node Referenced by r Between Nodes X and G
We are now ready to code the two steps of our algorithm shown
graphically in Figure 4.7 . Assuming the name of the link field is: next,
and that it can be accessed with the notation: reference-Variable.next, the
code of the algorithm is:
Figure 4.8 Inclusion of a Dummy Node into the Standard Depiction
of a Singly Linked List
These two lines of pseudocode place the address 700 (stored in the
variable n) in Node X’s link field, and stores the address 973 (stored in the
variable p) in the newly inserted node’s link field.
Before using this graphical technique to discover and verify the basic
operation algorithms, two changes to the standard graphical depiction of a
singly linked list will be discussed. The first change is to insert a dummy
node between the list header and the real first node. The dummy node will
not store client information (thus the term dummy). It is added to the
structure to simplify the code of the Insert and the Delete algorithms
because without it, performing these operations on an empty structure
213

becomes a special case. The grayed area of Figure 4.8 reflects the changes
to the linked list depiction presented in Figure 4.5 necessary to incorporate
the dummy node.
The second change is to introduce a lower level of detail into Figure
4.8 consistent with the manner in which the operation algorithms will be
implemented. In the implementation, the left field of each node that makes
up the singly linked list will not contain the client’s information. Rather, it
will contain a reference to a deep copy of the information. Thus, both
fields of the “nodes” in our singly linked list will be reference variables.
The left field will be named l, and the right field will be named next. l will
point to a deep copy of the client’s information (a Listing object), and next
will point to the subsequent node (a linked list Node object) in the singly
linked list. This implementation level view of the structure is shown in
Figure 4.9 .
At the implementation level, the nodes in the linked list are analogous
to the array of reference variables that was part of the structures presented
in previous chapters in that they store the addresses of the deep copies of
the client’s objects. However, since it is a linked list, each node also stores
the address of the next node in the linked list. A thorough understanding of
the material presented in Figure 4.9 is necessary to understand the
structure implementations presented in this chapter.
Initialization
One of the ironies of linked lists is that, although they can contain a
virtually unlimited number of nodes (within the limits of system storage),
the class that defines them usually contains only one data member: the list
head h. In addition, after the structure is initialized, it only contains one
linked node, the dummy node. Consistent with our graphical technique for
developing the operation algorithms, Figure 4.10 shows a linked list object
before and after its initialization algorithm is executed.
214

Figure 4.9 Implementation-Level View of a Singly Linked List
Figure 4.10 Singly Linked List Before and After Initialization
To verify the initialization algorithm, we traverse the list shown in
Figure 4.10 starting at the list head. The list head points us to the dummy
node, and then the dummy node’s link field, being null , ends the traverse.
As long as we accept the condition of an empty list as a null value stored
in the link field of the dummy node, the algorithm is verified. Assuming
the left and right fields of the dummy node are named l and next,
respectively, and that the dummy node is an object in the class Node, the
pseudocode of the verified initialization algorithm can be written as:
215

Figure 4.11 The Steps to Insert T’s Node into the Singly Linked List
Depicted in Figure 4.9
The Singly Linked List Initialization Algorithm
Insert Algorithm
In order to encapsulate the structure, the Insert algorithm will add a
deep copy of the client’s information to the data set. Since the access to the
nodes is sequential, the fastest and simplest way to add the deep copy to
the structure is to insert the new node at the beginning of the linked list,
just after the dummy node. This approach to the insertion algorithm will
cause the structure to resemble a stack in that the most recently inserted
node will be the most readily accessible node, an issue we will return to
later in this chapter.
Figure 4.11 shows the changes required to insert a deep copy of
client’s Listing, T, at the beginning of a linked list. The circled numbers in
the figure indicate the order in which the changes are made, as described
below.
1. Create a new linked list Node object.
2. Add the new linked list Node object to the beginning of the linked
list.
a) Set the next field of the linked list Node object to the contents of the next field of the
dummy node.
216

b) Place the address of the new linked list Node object into the next field of the dummy node.
3. Create a deep copy of the client’s information, and reference it from
the l field of the new linked list Node object.
To verify the graphical version of the Insert algorithm depicted in
Figure 4.11 , we traverse the list starting at the list head. This brings us to
the dummy node, and the dummy node’s link field (next) now brings us to
the new linked list node (by following the dotted arrow from the next field
of the dummy node). Then, the next field of the new linked list node brings
us to the remainder of the linked list (by following the dotted arrow from
the next field of the new linked list node). Also, the l field of the new
linked list node references the deep copy of the client’s information. Since
the newly inserted node is accessible via the dummy node, the other nodes
in the structure are still accessible, and the deep copy of the client’s
inserted information is also accessible, we have verified the graphical
representation of the algorithm. It is important to note that if Step 2b is
performed before 2a, the entire linked list, except for the newly inserted
node, is effectively deleted from the structure.
The next step in the development of the algorithm is to translate the
graphical representation into pseudocode. Following the order of the
numbered changes to the linked list depicted in Figure 4.11 , the four-step
pseudocode version of the verified Insert algorithm is written as:
The Singly Linked List Insert Algorithm
where newListing is an object that contains the information to be
inserted into the linked list. It should be noted that, because of the presence
of the dummy node in the list, the Insert algorithm functions properly on
an empty (initialized) list as well. The verification of this is left as an
exercise for the student.
The Fetch Algorithm
Because the only access to the information stored in the structure (the
deep copies of the client’s listings) is through the nodes in the singly linked
list, in order to locate a listing we must traverse the linked list. Traversing
is performed using a linked node reference (e.g., p), which is initialized to
217

reference the first node in the linked list. Then p is moved down the linked
list by repeatedly assigning it the contents of the next field of the node it
references, until the listing is found. Once found, a deep copy of the listing
is returned. If the requested listing is not in the list, then p will eventually
assume a null value when it reaches the last node in the linked list, which
terminates an unsuccessful search.
Figure 4.12 The Singly Linked List Fetch Algorithm (Illustrated to
Fetch G’s Listing)
Figure 4.12 shows the singly linked list depicted in Figure 4.9 ,
modified (gray boxes) to include the graphical representation of the Fetch
algorithm. In this case, the listing to be fetched is G. The four dashed
arrows in the figure indicate that the algorithm is copying information
from one area of memory to another. The circled numbers indicate the
order of the operations required to accomplish a Fetch operation, which
are:
1. Initialize the Node reference, p, to reference the first node in the list.
2. Traverse p down the list until it locates the information to be fetched.
3. Return a deep copy of the information to be fetched.
To verify the graphical representation of the algorithm depicted in
Figure 4.12 we first observe that by copying the contents of the next field
of the dummy node (180) into p, it is set to point to the first node in the
218

linked list. Then, by repeatedly copying the contents of the next field of
the node that p references into p (first 300, and then 54), p traverses the list
and locates the listing to be fetched (G). At this point, we observe that the
location of the listing to be fetched is contained in the l field of the node p
references, and a deep copy of it is returned to the client. This verifies the
graphical version of the algorithm.
The pseudocode version of the three-step graphical algorithm,
expanded to include an unsuccessful search, is:
The Singly Linked List Fetch Algorithm
The algorithm returns null if the node to be fetched is not in the
structure and assumes the key searched for is targetKey.
The Delete Algorithm
The first two steps of the Delete algorithm are identical to the first
two steps of the Fetch algorithm in that it begins by initializing a reference
variable, p, to the first node in the linked list and then traverses p down the
list in order to locate the listing to be deleted. To delete the listing from the
structure, the linked list node that references it is deleted from the linked
list. To accomplish this, the next field of the node preceding it is modified
to “jump over” it. As illustrated in Figure 4.13 (where it is assumed that G
is the listing to be deleted), the jump is performed by resetting the
preceding linked node’s next field to the location of the node after the
deleted node.
219

Figure 4.13 Deleting the Listing G by “Jumping” Around Its Linked
List Node
Figure 4.14 The Delete Algorithm After p and q Have Been
Positioned At, and Before, the Node to Be Deleted, G
One problem arises: when the traverse is complete and p is
referencing the node to be deleted, there is no way of determining the
location of the preceding node. The next field references point forward,
not backward, in the linked list. The remedy is to use another node
reference, q, to store the location of the proceeding node. The reference
variable q follows p down the list. Initially, q is set pointing to the dummy
node, which is another reason the dummy node was included in the
structure. Figure 4.14 shows the positioning of the reference variables, p
and q, after a successful traversal to locate the node G.
Figure 4.14 also shows the graphical representation of the portion of
the Delete algorithm that is different from the three-step Fetch algorithm;
220

the jump over the linked node to be deleted. As indicated by the circled 3
in the figure, this jump replaces Step 3 of the Fetch algorithm. The only
other changes to the Fetch algorithm are to expand Steps 1 and 2 to
initially store the location of the dummy node in the variable q (in Step 1)
and then to set q to p (in Step 2).
To verify the graphical version of the algorithm depicted in Figure
4.14 , we traverse the list beginning at the dummy node referenced by h.
The next field of the dummy node brings us to the linked node that
references Node T. Then, the next field of the node that references Node T
brings us to the linked node that references Node X. Finally, the next field
of the linked node that references Node X brings us to the linked node that
references Node B, which completes the traverse. Since Node G was not
encountered in the traverse, and all the other nodes were, the algorithm is
verified.
One final point should be made regarding the garbage collection
process used in this algorithm. Because of the “jump over” step of this
algorithm, the deleted linked node is no longer referenced (by a linked
node). Therefore it, and the deleted Listing object, are returned to the
available memory pool by the Java memory manager.
Having verified the graphical representation of the Delete algorithm,
we will now present the pseudocode version. As discussed above, the first
two steps of the algorithm are the first two steps of the Fetch algorithm
expanded to include the use of the trailing reference variable, q. The
pseudocode returns false if the node to be deleted is not in the structure
and assumes the key of the listing to be deleted is targetKey.
The Linked List Delete Algorithm
The Update Algorithm
221

Once again, the Update algorithm will be an invocation of the Delete
algorithm to eliminate the listing to be updated from the structure,
followed by an invocation of the Insert algorithm to place a clone of the
new information into the structure. Therefore, the Update algorithm to
change the listing whose key is targetKey to the contents of the listing
newNode is:
The Singly Linked List Update Algorithm
4.3.2 Implementation
The implementation of the singly linked list structure is presented in
Figure 4.15 as the Java class, SinglyLinkedList. It is implemented as a
homogeneous, fully encapsulated structure. The code presented in Figure
4.15 is consistent with many of the concepts of generics presented in
Chapter 2 . For example, it does not mention the names of any of the fields
of the client’s nodes, and the definition of these nodes is coded as a
separate class (see Figure 2.16 ). The class provides a deepCopy method in
order to encapsulate the structure, a compareTo method to determine if a
given key is equal to the key of a client node stored in the structure, and a
toString method to return the contents of a node. The implementation is
not fully generic in that the client’s node class must be named Listing, and
the key field must be a String. A fully generic implementation of the
structure, using the generic features of Java 5.0 and the techniques
described in Chapters 2 and 3 , Section 2.5 and 3.4 , is left as an exercise
for the student.
222

223

Figure 4.15 The Implementation of the Singly Linked List Structure
Line 2 declares the list header, h, that will store the address of the
dummy node, an object in the class Node. This class defines the objects
that will make up the linked list, and its code appears as Lines 57–63.
Node objects have two data members: l, a reference to a Listing object,
and next, a reference to a Node object (Lines 58–59). The class Node is
defined as an inner class 1 of the class SinglyLinkedList because:
• The code of the class SinglyLinkedList can then directly access the
two data members, l and next, of the class Node (e.g., Lines 5–6).
• Only the code of the class SinglyLinkedList will declare objects in the
class Node.
The singly linked list initialization algorithm is coded on Lines 3–7,
the outer class’ constructor. Line 4 creates the dummy node referenced by
the list header. The insert, delete, and fetch operation methods (Lines 8–
42) are the Java equivalent of the pseudocode algorithms presented in the
previous section with error checking added to the insert method (Lines 10
and 11). The only nuance is on Lines 21 and 32, in that they use the
Listing class’ compareTo method to compare the String keys. Lines 43–49
are the update method (which is the same coding of the update methods
presented in Chapter 2 ). It invokes the delete and insert methods (Lines 44
and 46) to perform its operation. Finally, the showAll method (Lines 50–
56) traverses the list outputting each listing until p reaches the end of the
linked list (p == null; on Line 52).
224

Figure 4.16 A SinglyLinkedList Telephone Listing Application
To demonstrate the use of the class SinglyLinkedList, an application
program that processes a telephone listing data set is presented in Figure
4.16 . The output it generates is presented in Figure 4.17 . Notice that the
listings output by invoking the showAll method on Line 10 are in the
reverse order (compared to the order in which they were inserted into the
structure on Lines 7–9) because, as we have mentioned, new listings are
inserted at the beginning of the linked list. The class that defines the
telephone listings, presented in Figure 4.16 , complies with the
assumptions implicit in the coding of the class SinglyLinkedList.
4.3.3 Performance of the Singly Linked List
As we have discussed, the major advantage of a singly linked list is
its ability to rapidly expand to accept a virtually unlimited number of
nodes in a fragmented memory environment, regardless of the
implementation language. We would suspect the major disadvantage of the
structure would be its speed, since it uses a sequential search to locate a
node. To evaluate the overall performance of the structure
SinglyLinkedList, we will first examine the speed of its operation
algorithms, and then the amount of overhead memory required to
implement the structure.
Speed
To analyze the speed of the structure, we will perform a Big-O
225

analysis to determine the approximate speed as n , the number of listings
stored in the structure, gets large. Because the time to perform a memory
access instruction is typically considerably longer than the time to perform
a nonaccess instruction, only memory access instructions will be included
in our analysis. In addition, instructions inside of loops that repeatedly
access the same memory cell will not be included in our speed analysis
either, since modern compilers store these variables in CPU registers.
Figure 4.17 The Output Generated by the SinglyLinkedList
Telephone Listing Application shown in Figure 4.16
Examining the Insert algorithm, all four lines of pseudocode access
memory. However, they are only executed once per Insert operation,
independent of the number of nodes in the structure. Therefore, six
memory accesses are required per Insert operation (one on Line 1, three on
Line 2a, 2 one on Line 2b, and one on Line 3). Thus the dominant (and
only) term in the Insert operation’s speed function is 6, which is O(1).
226

The Fetch algorithm uses a sequential search (beginning on Line 2) to
locate a listing given its key, targetKey. Sometimes it will be at the
beginning of the linked list and the loop will not execute, and other times
the listing will be at the end of the list and the loop will execute n − 1
times. Since all locations are equally probable, the loop will execute an
average of approximately n / 2 (≈(n − 1) / 2) times.
Line 1 and the lines after (and including) Line 3 are not in the search
loop so they will be ignored. Inside the loop, a total of three memory
accesses are performed: one to fetch p.l, one to fetch the key, and one to
fetch p.next (assuming p would be stored in a CPU register). Since these
accesses are performed approximately n / 2 times, the dominant term in the
speed equation is 3(n / 2) = 1.5n , which is O(n ).
The Delete algorithm also uses a sequential search (beginning on Line
2) to locate the node to be deleted. Assuming the variables p and q are
stored in CPU registers, there are three memory accesses performed inside
the search loop: one to fetch p.l, one to fetch the key, and one to fetch
p.next. The lines outside of the search loop contain memory access
instructions, but they are only executed once so they do not contribute to
the dominant term in the speed equation. As in the case of the Insert
algorithm, the sequential search loop will be executed an average of
approximately n / 2 times. Therefore, the three memory accesses
performed inside the loop result in a dominant term of 3(n / 2) = 1.5n ,
which is O(n ).
Overhead
Let us now turn our attention to the overhead of the structure.
Referring to Figure 4.9 , the overhead of the structure is the storage
associated with the list header and the nodes that form the linked list,
including the dummy node. The list header, the fields of the dummy node,
and the fields of the n nodes on the linked list are all reference variables.
Therefore, the overhead is one header reference variable, plus two dummy
node reference variables, plus 2n linked node reference variables, for a
total of 3 + 2n reference variables. Since reference variables occupy four
bytes, the total overhead storage required by this structure is 4(3 + 2n )
bytes.
Density is defined as
where the information bytes is simply the product of the number of
227

Listing objects, n , and the number of information bytes per Listing object,
w. The total bytes allocated the structure is the sum of the information
bytes, n * w, and the overhead bytes, 4(3 + 2n ). Therefore the density can
be expressed as
(which is approximately equal to 1 / (1 + 8 / w) as n gets large).
Figure 4.18 Density of the SinglyLinkedList Structure Containing
More than 100 Nodes
Figure 4.18 presents a graph of this function for n > 100 (which
makes the term 12 / (n * w) negligible). The figure demonstrates that good
densities (0.80 or higher) are achieved whenever the number of bytes in a
node is greater than 33. Table 4.1 summarizes the performance of the
singly linked list structure and includes the performance of the previously
studied structures for comparative purposes. While not quite as fast as the
Unsorted-Optimized array structure, its noncontiguous feature makes it an
attractive alternative to the array-based structure for node widths greater
than 33 bytes.
4.3.4 A Stack Implemented as a Singly Linked List
Often, a singly linked list is used to implement the data structure
Stack as an alternative to the array-based implementation presented in
Chapter 3 . The advantage of this alternate implementation is that it can be
rapidly expanded in all programming languages to accommodate a
virtually unlimited number of Push operations, and it is compatible with a
fragmented memory environment.
228

Figure 4.19 A Singly Linked List After A, B, C, and D Are Inserted
(or Pushed)
The singly linked list Insert operation can be thought of as a stack
Push operation, if we consider the front of the singly linked list to be the
top of the stack. Since the inserted node is always placed in the front of the
linked list, the nodes are stored in a “stack-like” reverse order. This is
illustrated in Figure 4.19 which shows a linked list after the nodes A, B, C,
and finally D are inserted.
The singly linked list Fetch operation however, appears to be far from
a stack Pop operation. It is performed in the key field mode and does not
delete the fetched node from the structure. Aside from these problems,
neither the fetch nor the insert methods are named properly. Stack
operation methods should be named pop and push.
The best remedy is to define a new stack structure class, copy the
SinglyLinkedList data members, constructors, inner class Node, and the
insert method code into it and then rename the insert method pop. The pop
method in this class would be a new method that has the standard (no
229

parameter) push signature, and always returns and deletes the first node in
the list. Figure 4.20 presents the graphical representation of this Pop
algorithm which places the address of the popped listing into the variable
p. The circled numbers in the figure indicate the sequence of the two steps
of the algorithm, and the dash-dot arrow indicates the writing of address
649 into P. Figure 4.21 presents the equivalent pseudocode with the test
for underflow added to it.
The implementation of this linked list-based stack is left as an
exercise for the student.
An Alternative Linked Stack
As an alternative to coding a new class to implement a dynamic stack,
there is a “trick” we can use to force the linked list Fetch operation to
always fetch the first node from the structure and to also force the Delete
operation to remove it from the structure. Thus, utilizing this trick, a Pop
operation can be accomplished by performing a Fetch operation followed
by a Delete operation.
Figure 4.20 A Pop Operation for a Singly Linked List-Based Stack
230

Figure 4.21 The Pop Operation for the Dynamic Implementation of a
Stack
The trick is to set the key field of each of the client’s nodes inserted
into the structure to the same value (e.g., “X”) before they are inserted.
Naturally, if the definition of the nodes is not changed, the contents of the
key fields will be lost. This problem will be dealt with later in this section.
Figure 4.22 shows a linked list after nodes A, B, C, and finally D are
inserted into the structure. Then, to perform a Pop operation, we invoke
the fetch method followed by the delete method and specify this common
value of the key field as the argument sent to the fetch and delete methods
(e.g., fetch(X), delete(X)). Since both methods start their sequential search
at the first node in the linked list, and this node (and all nodes in the
structure) contain the key value X, the first node in the linked list is
returned and then deleted.
Figure 4.22 Singly Linked List after A, B, C, and D Are Inserted
(Pushed) with their Key Fields Set to X
Considering the data stored in the stack depicted in Figure 4.22 , the
fetch method invocation: fetch(“X”) will return D. The delete method
invocation: delete(“X”) will remove D from the structure. Since D was the
last node inserted into the structure, these two invocations have performed
a stack Pop operation. If the two invocations are repeated three more
times, C will be fetched and deleted, followed by B, and then A. Thus, the
structure is behaving like the restricted LIFO Stack structure.
Figure 4.23 shows an application program that uses our trick to make
the object boston, an object in the class SinglyLinkedList, behave like a
stack. The code is followed by the output it produces. Consistent with the
231

trick, the key fields of the nodes declared on Lines 4–6 have all been set to
“X”. The nodes are pushed onto the structure using the insert method
(Lines 8–10). Then they are popped from the structure and output using
the fetch, delete, and toString methods (Lines 12–20). The key “X” is used
in each Fetch-and-Delete operation. The LIFO sequence of the output
verifies that our trick has indeed caused the SinglyLinkedList object,
boston, to behave like a stack.
Although this trick can be used in application programs to simulate a
stack that accommodates a virtually unlimited number of Push operations,
its use is undesirable for four reasons:
• To retain the contents of the key field, an additional field must be
added to the node.
• It requires the application programmer to have knowledge of the
“trick.”
• The push and pop operation methods are not named push and pop.
• If the client neglects to invoke the delete method after each fetch
invocation, the stack “simulation” breaks down (the Pop operation
becomes a Peek operation).
232

Figure 4.23 A Program That Uses a Singly Linked List Structure As
a Stack and Its Output
A more desirable approach to utilizing the trick is to write a new
class, StackSLL, that includes a SinglyLinkedList object as a data
member. Knowledge of the trick is then imbedded into the push and pop
methods of the StackSLL class, which unburdens the application
programmer. As shown in the top part of Figure 4.24 , the client will pass
StackListing objects to, and from, the push and pop methods respectively.
These objects will contain the client’s information to be stored in the
structure. The push method will invoke the singly linked list insert method,
and the pop method will invoke the singly linked list’s fetch and delete
233

methods. Since the linked list methods pass Listing objects, the dummy
key and a reference, r, to a deep copy of the client’s StackListing object
will be combined by the push method to form a redefined two-field Listing
object before it invokes insert. The pop method will invoke the fetch
method to fetch a Listing object from the front of the singly linked list.
The location of the StackListing object, r, contained in the Listing object
will be returned to the client by the pop method after it invokes delete.
Figure 4.24 Invocation Sequence and Argument Flow for a Stack
Implemented Using the Class SinglyLinkedList
The top portion of Figure 4.25 shows the redefined fields of the
singly linked list Listing object consistent with this approach. The
rightmost field of the object, r, is a reference to a StackListing object. It
also shows the fields of a StackListing object for a client application that
stores telephone listings on our stack. Figure 4.26 shows a StackSLL
structure after the four nodes A, B, C, and finally D have been pushed onto
the structure. The implementation of the class StackSLL is left as an
exercise for the student.
4.4 Other Types of Linked Lists
A singly linked list is the simplest form of a linked list. Other types of
linked lists include circular singly linked lists, double-ended singly linked
lists, sorted singly linked lists, doubly linked lists, circular doubly linked
lists, and multilinked lists. We will now briefly discuss these structures.
234

Figure 4.25 The Fields of the Listing and StackListing Objects
Mentioned in Figure 4.24 (for a Phone Listing Application)
Figure 4.26 The Memory Model for the Stack Implemented as
Shown in Figure 4.24
235

Figure 4.27 A Circular Singly Linked List
4.4.1 Circular Singly Linked List
A circular singly linked list is a singly linked list in which the next
field of the last node in the list references the first node in the list. When a
dummy node is used in the implementation, the last node in the list
references the dummy node. Otherwise, it references the actual first node.
Figure 4.27 shows a circular singly linked list (whose implementation uses
a dummy node) with four client nodes, T, X, G, and B stored in it.
The implementation of a circular singly linked list is basically the
same as a singly linked list with a few minor additions to account for the
last node’s circular reference to the dummy node. When the list is created,
the next field of the dummy node is set to reference itself (h.next = h). The
only other change is to the sequential search performed in the Fetch and
Delete algorithms. Rather than an unsuccessful search ending at a null
reference, it ends when a reference to the dummy node is encountered.
Therefore, the comparison of p with null on Lines 2 and 3 of both the
Fetch and Delete pseudocode algorithms are changed to comparisons of p
with h (the list header) as
4.4.2 Double-Ended Singly Linked List
A double-ended singly linked list is a singly linked list in which
236

insertions are permitted at the front (as usual) and rear (a new feature) of
the list. A newly inserted linked node can become either the new first node
or the new last node in the list.
Figure 4.28 A Double-Ended Singly Linked List
To implement this structure a reference variable, rear, is added as a
data member to the SinglyLinkedList class along with an additional insert
method. The variable rear references the last node in the structure (Figure
4.28 ), and the additional insert method is used to insert nodes at the end of
the linked list. 4 When the list is empty (all the nodes have been deleted)
rear is null . When a node is inserted at the end of the list, the last node’s
next field and the variable rear are changed to reference the new (last)
node. An empty list is treated as a special case in the new Insert algorithm.
Assuming the new linked node is referenced by the variable newLink, the
pseudocode to add a new linked node at the end of the list is:
To complete the Insert algorithm, the l field of the newly added
linked node is set to reference a clone of the client’s listing.
Deleting the last node in the list is also treated as a special case.
Referring to Step 3 of the singly linked list Delete algorithm, prior to
returning true , the following is added:
237

Often a double-ended singly linked list is used to implement the
structure Queue. Nodes are added at the rear of the queue using the new
insert method. Nodes are removed (fetched) from the front of the queue
using the algorithm depicted in Figure 4.21 (the Pop operation for a stack
implemented as a singly linked list).
4.4.3 Sorted Singly Linked List
A sorted singly linked list is one in which the nodes are positioned in
the list in sorted order based on the contents of their key field. Figure 4.29
shows a sorted singly linked list after four client nodes (B, G, X, and
finally T) have been inserted.
To arrange them in sorted order, the Insert algorithm must include a
sequential search down the list to find the “correct location” for a newly
inserted node. The search continues until a node is encountered with a key
greater than the new node’s key, or the end of the list is encountered.
Assuming the reference variable used to traverse the list is p, newNode
references the item to be inserted into the structure, and the client’s node
definition class included a getKey method, the traversal would continue as
follows:
The compareTo method would have to return a positive integer
whenever the key of the object that invoked it (newNode) is greater than
the argument passed to it (p.l.getKey()). 5 The new linked node is inserted
between the node referenced by p and its predecessor, or as the new last
node. The technique for inserting a node in between two nodes is
illustrated in Figure 4.7 , and the sequential search to find the new item’s
correct position is similar to the sequential search in the Delete algorithm
in that a reference variable q trails the variable p during the traversal.
238

Figure 4.29 A Sorted Singly linked List
The major advantage of a sorted singly linked list structure is that its
showAll method outputs the nodes in sorted order based on their key field
contents. In addition, when the delete or fetch method is invoked to
operate on a listing that is not in the structure, their average speed is
doubled. Since the listings are stored in sorted order, the search portion of
these algorithms can be modified to end when it encounters a key greater
than the key of the listing to be operated on. This, on the average, will
occur after traversing halfway down the linked list.
On the negative side, the sequential search added to the Insert
algorithm decreases the speed of both the Insert and Update operations.
Therefore, for some applications that require sorted output and perform
many Insert operations, it is more efficient to store the nodes in an
unsorted singly linked list and then simply sort the nodes inside the
showAll method before they are output. Sorting methods used in this
approach are discussed in Chapter 8 .
4.4.4 Doubly Linked List
A doubly linked list is a singly linked list in which each node in the
list has an additional linked reference field that refers to the node just
before it in the list. Figure 4.30 shows a doubly linked list with three nodes
T, X, and G stored in it. The additional reference variable is named back.
As shown in the figure, the dummy node also contains the additional
reference field, back.
The Fetch algorithm of this structure is the same as the singly linked
list’s Fetch algorithm. Since inserts are still performed at the front of the
list, the Insert algorithm is modified to set the back reference of the first
239

node in the list to the location of the inserted (new) node, and to set the
back reference of the inserted node to null . Assuming the new doubly
linked list node is referenced by the variable newLink, the additional code
in the Fetch algorithm is:
Figure 4.30 A Doubly Linked List
The singly linked list Delete algorithm must also be modified to
implement this structure. When a node is deleted from a singly linked list,
the next field of the node that precedes it is reset to effectively jump
around the deleted node (see Figure 4.13 ). An analogous section of code
must be added to the Delete algorithm to adjust the back reference of the
node following the deleted node; it too must also be made to jump around
the deleted node. To accomplish this, the Delete algorithm places the
address of the node preceding the deleted node into the back field of the
node that follows the deleted node. Deletion of the last node in the list is
treated as a special case in which the addition code is not executed. Thus,
the code added to the Delete algorithm is:
Doubly linked list structures are used for applications that require a
backward traversal through the list. The density of the structure is slightly
lower than that of the singly linked list due to the additional four bytes of
overhead associated with the back field of each node. Specifically, the
overhead is one header reference variable, plus three dummy node
reference variables plus 3n doubly linked node reference variables, for a
total of 4 + 3n reference variables. Since reference variables occupy four
bytes, the total overhead is 4(4 + 3n ) bytes.
240

To compute the density of this structure we recall that density is
defined as
The information bytes is simply the product of the number of nodes, n
, and the number of bytes per client listing, w. The total bytes allocated to
the structure, the sum of the information bytes and the overhead bytes, is n
* w + 4(4 + 3n ). Therefore the density can be expressed as
which is approximately equal to 1 / (1 + 12/w) as n gets large.
Figure 4.31 presents the variation of the density of this structure with
node width (for n ≥ 100), and includes the density of the singly linked list
for comparative purposes. The figure shows that the doubly linked list
structure achieves a density greater than 0.80 for node widths greater than
60 bytes.
Figure 4.31 Density of a Doubly Linked List that Stores 100 Nodes
4.4.5 Multilinked List
Multilinked lists are linked lists in which the nodes are stored in a
way that allows more than one traversal path through the nodes (e.g.,
Figure 4.2 b ). Consider our phone book listings. Suppose an application
required the listings to be output in both name and phone number order. A
multi-linked list that allowed a traversal in alphabetic name order and in
phone number order would be ideally suited for this application.
Figure 4.32 presents a multilinked data structure implemented as two
241

sorted singly linked lists. The first singly linked list, which is shown at the
top of the figure and whose header is h1, orders the listings in name order.
Assuming the lowest phone number is T’s followed by B’s, G’s, and finally
X’s, the second singly linked list (shown at the bottom of the figure and
whose header is h2) orders the listings in phone number order.
In the implementation of this structure, two sorted singly linked
objects (e.g., list1 and list2) are declared as data members of the new
multilinked structure class. Although a listing inserted into the multilinked
data structure would be fully encapsulated, the implementation of the
insert method of the sorted singly linked lists would be modified to
unencapsulated the listings. This would permit (as shown in Figure 4.32 )
the two singly linked list objects to share access to the same clone of the
client’s information. The insert method of the multilinked structure would
clone the client’s listing, and send a reference to it to the sorted singly
linked list’s insert method which would shallow copy the cloned listing
into the list.
A second modification to the singly linked list’s insert method would
be necessary. An integer parameter would be added to its parameter list to
indicate which field in the cloned listing is to be used as the sort field. Its
expanded signature would be:
Figure 4.32 A Multilinked List with Two Orderings
242

The class that defines the client’s listings would have to include two
getKey methods (e.g., getKey1 and getKey2) used by the linked list’s
insert method to fetch the contents of one sort field, or the other. The
decision as to which one to invoke would be based in the value of the new
sortField parameter sent to it. Assuming that the listing to be inserted is an
object in the class Listing, and that it is referenced by the variable
newNode, the multilinked structure’s insert method would be:
where list1 and list2 are the two sorted singly linked list objects
declared as data members of the new multilinked structure class. The
sorted singly linked list insert method would be modified to contain the
additional code:
and to use the contents of the variable key to position the node in the
sorted singly linked list objects.
The showAll method of the multilinked list class would also contain
an additional integer parameter that the client could set to indicate which
of the two sorted orders would be used in the output. Its signature would
be: void showAll(int listNumber). The client would invoke the method in
one of two ways: myList.showAll(1) or myList.showAll(2). The
multilinked structure’s showAll method would then invoke the Listing
class’ showAll method using the code sequence:
If the application required that a Delete operation be performed on the
data set, the simplest implementation would be to require the client to
specify two key values, key1 and key2, when the delete method is
invoked. Then, the delete method would eliminate the node from the data
structure by invoking the sorted singly linked class’ delete method twice:
243

A more involved (but more client-friendly) implementation would
only require the client to pass one key to the multilinked class’ delete
method and the method would fetch the other key from the node before it
was deleted from one of the lists. Then the delete method would be
invoked again with that key to delete the node from the other list.
The density of this structure is lower than the other linked structures
presented in this chapter because of the overhead associated with the
second sorted singly linked list. As we have previously shown, the
overhead associated with a singly linked list, containing n listings, is 4(3 +
2n ) bytes. To generalize our calculation of density for this structure, we
will assume that the nodes are sorted on L different fields, requiring our
structure to contain L singly linked lists. 6 Thus the total overhead for the
structure is 4L(3 + 2n ) bytes. Remembering that the information bytes in
the structure is simply the product of the number of nodes, n , and the
number of information bytes per client node, w, its density is:
which is approximately equal to 1 / (1 + 8L / w) as n gets large.
Figure 4.33 , which assumes a node width n ≥ 100, presents the
density of this structure for values of L equal to 2 and 4. For comparative
purposes it includes the density variation of the singly and doubly linked
structures. A density of 0.80 is achieved for node widths greater than 65
bytes for two orderings, and a node width greater than 130 bytes for four
orderings.
244

Figure 4.33 Densities of Multilinked Lists that Store more than 100
Nodes in Two or Four Orderings
4.5 Iterators
An iterator is an object that the client can use to sequentially access
the nodes in a linear list. Singly linked lists and arrays are examples of
linear lists. Typically, the iterator’s class provides methods for positioning
the iterator object at the first item in the list, advancing the iterator to the
next item, and determining if the iterator is at the end of the list. In
addition, the iterator’s class provides methods to operate on the item at the
iterator’s current position.
Iterators can be a very convenient means of access for certain
applications. For example, suppose we wanted to add an area code to every
listing in a telephone directory stored in a singly linked list (see Figure 4.9
). In this case, an Update operation performed in the key field mode is not
particularly useful because we would have to generate the names of all of
the telephone customers in order to operate on each node. In addition, even
if the Update method was coded in the node number mode, using it to
change the area code of every item in the database would be a time
consuming process since each invocation begins its traversal at the front of
the list. When updating the first listing, one node would be traversed. Two
nodes would be traversed to update the second listing, three nodes for the
third listing, etc. Thus, the total number of nodes traversed to update each
of n nodes is: 1 + 2 + 3 +…+ n = n (n + 1) / 2, which is O(n 2 ).
An iterator object, on the other hand, retains its position in the list
after an operation is performed. Therefore, if the iterator’s class contained
an update method, after updating one listing, it would simply traverse one
more node in order to update the next listing. The total number of nodes
traversed to update each of n nodes would be: 1 + 1 + 1 +…+ 1 = n , which
is O(n ). In summary, the use of the iterator would speed up the update of
the listings by n times.
Table 4.2
Several Iterator Methods of an Iterator Class to be Added to the
Class SinglyLinkedList
Iterator
Method
Description
public void positions the iterator at the dummy node
245

reset()
public boolean
hasNext()
returns true if there is a node after the iterator’s current
position
public Listing
next()
moves the iterator to the next node and then returns a
reference to a clone of its listing
public void
set(Listing
newListing)
replaces the listing, stored at the iterator’s current
position, with a clone of newListing. The iterator’s
position is not changed.
Most iterator classes provide methods, whose signatures are
somewhat standardized, to operate on the information stored in a linear
list. These methods typically included the method add (to perform an
Insert operation), the method next (to perform a Fetch operation), the
method remove (to perform a Delete operation), and the method set (to
perform an Update operation). Other common iterator methods are reset,
used to position the iterator at the beginning of the list, and hasNext to
determine if the iterator is at the end of the list. Iterator classes that operate
on doubly linked lists include a method previous, for traversing the iterator
backward through the list, and a method hasPrevious, to determine if the
iterator is at the beginning of the list.
To illustrate the use of some of these methods, let us again consider
the problem of adding an area code to a telephone listing data set. We will
assume that the iterator class containing the methods described in Table
4.2 has been added to the class SinglyLinkedList (see Figure 4.15 ), and
that an iterator object, i, has also been added to the class as a public data
member.
We will also assume that the methods getNumber and setNumber
have been added to the class Listing (see Figure 2.16 ) to fetch
(getNumber) and change (setNumber) the contents of a Listing object’s
phone number field. Under these assumptions, the iterator object, i, is used
in the following code sequence to efficiently add the area code “631” to
every listing in the telephone directory boston, a SinglyLinkedList object:
246

As previously discussed, the efficiency of this code resides in the first
line of the while loop (listing = boston.i.next()) because the iterator, i,
retains its position in the list after the line executes. This eliminates the
need to begin a new traversal through the list during each loop iteration.
4.5.1 Implementation of an Iterator
There are two common techniques used to implement an iterator
class. One technique defines the class inside the data structure class (i.e.,
as an inner class), and the iterator object (as previously suggested) is a
public data member of the data structure class. This technique is used
when the data structure is encapsulated, since it maintains the
encapsulation of the structure. In this implementation, the Iterator class’
methods insert and return clones (deep copies) of the client’s listings.
In the second technique, the iterator class is defined outside of the
data structure class, and the iterator object is declared in the client code.
This technique is used when the data structure is not encapsulated, since it
gives the client direct access to the information contained in the structure.
In this implementation, the iterator class’ methods insert and return shallow
copies of the client’s listings.
Figure 4.34 presents the code of a class named
SinglyLinkedListIterator, which is an expanded version of the class
SinglyLinkedList presented in Figure 4.15 . 7 The expansion includes the
definition of an iterator class, named Iterator, that implements the four
methods presented in Table 4.2 and the declaration of an Iterator object as
a data member in the class. The inner class implementation of the iterator
was used since the class SinglyLinkedList is a fully encapsulated structure.
Line 3 declares the Iterator reference variable, i, as a public data
member. Its access is public so that the client can use it to access the
Iterator class’ methods. The actual Iterator object is created on Line 6 of
the class’ constructor. Lines 7–9, the code of the four basic operation
methods (Lines 10–51), the showAll method (Lines 52–58), and the
definition of the class Node (Lines 59–65) are the same as the code of the
class SinglyLinkedList presented in Figure 4.15 .
247

Lines 66–87 is the code of the inner class Iterator. It contains one data
member ip, a singly linked Node reference variable declared on Line 67.
This variable will be used to store the position of the iterator. Initially it
refers to the dummy node (Line 69).
The reset method (Lines 71–73) re-initializes the variable ip (Line 72)
to again position the iterator at the dummy node. The hasNext method
(Lines 74–79) determines if the iterator is not at the end of the list by
testing the next field of the node at the iterator’s location for a non-null
value (Line 75). If it is null , the method returns false indicating there is no
next node.
248

249

Figure 4.34 The Class SinglyLinkedList (Figure 4.15 ) Expanded to
Include an Iterator Object
250

Figure 4.35 The Class Listing2 with Methods to “Get” and “Set” a
Listing’s Phone Number
Lines 80–83 are the code of the method next, which fetches a Listing
object from the structure after moving the iterator. Line 81 moves the
iterator to the next node in the linked list, and then Line 82 returns a deep
copy of the listing referenced by that node. Similarly, the set method
(Lines 84–86) stores a reference to a deep copy of a Listing object at the
current iterator position (Line 85). The invocation of the Listing class’
method deepCopy, on Lines 82 and 85, maintains the encapsulation of the
structure.
Figure 4.35 presents the code of the Listing class presented in Figure
2.16 (less the setAddress, renamed Listing2, method) expanded to include
the setNumber and getNumber methods (Lines 22–27). These two methods
permit the client to access the phone number fields of a listing. Figure 4.36
is an application program that creates a phone listing data set stored in the
SinglyLinkedList-Iterator object boston (Lines 3–10). Then the object’s
iterator is used to output the data set 8 (Lines 12–14), add an area code to
each listing’s phone number (Lines 16–23), and output the revised data set
(Lines 25–29). The two sets of output it generates is presented in Figure
251

4.37 .
Figure 4.36 An Application that Uses an iterator, i, to Efficiently Add
a 631 Area Code to Phone Directory Listings
252

Figure 4.37 The Output Generated by the Application Shown in
Figure 4.36 , Demonstrating the Addition of a 631 Area Code
4.5.2 Multiple Iterators
Some applications require two or more iterators to be operating on a
list simultaneously. Programs with multiple threads often have this
requirement. For example, an iterator in one thread could be used to
transfer a data set over a modem, while an iterator in a second thread could
be used to output the data set to a printer. One alternative is to add more
iterator references to the data structure class as data members, and then to
create the corresponding iterator objects in the data structure’s constructor.
For example, to increase the number of iterators in the class
SinglyLinkedListIterator (Figure 4.34 ) from one to three, the code:
would be added to the class after Line 3, and the code:
would be added to the class’ constructor after Line 6. Then, the client
code could position the three iterators i, j, and k at different locations in the
253

list.
A more flexible alternative is to add a parameter to the
SinglyLinkedListIterator class’ constructor to allow the client to specify
the number of iterators required for a particular application. In this
approach, the iterators are implemented as an array of iterator references.
Line 3 of the class would become:
The constructor’s heading, Line 4 would become:
and Line 6 of the constructor would be replaced with:
where numberOfIterators is the name of the parameter the client
would use to specify the number of iterators used in the application. Then
the client code to allocate a singly linked list structure with 10 iterators and
to output the list using the third iterator would be:
External Iterators
Another approach to implementing multiple iterators is to implement
them in a way that allows the client code to declare the iterator object
references (e.g., Iterator iterator1, iterator2;). The advantage to this
approach is that the client can choose the names of the iterators (as well as
the number of iterators). In this implementation, the Iterator class can no
longer be an inner class (i.e., coded inside the class
SinglyLinkedListIterator). Once the iterator class is removed from this
class, the class Node (Lines 59–65 of Figure 4.34 ) must also be removed
from the class SinglyLinkedListIterator because otherwise the iterator
class could not declare a Node reference to store the iterator’s current
position (Line 67 of Figure 4.34 ).
Figures 4.38 , 4.39 , and 4.40 present the Node, Iterator, and
SinglyLinkedListIterator classes as three separate classes renamed
254

NewNode, SLLIterator, and SllExternalIterator respectively.
Figure 4.38 The class NewNode with Package Access Data Members
Figure 4.39 The External Iterator Class, SllIterator
Some subtle, but important changes have been made to the code of all
three classes to make them syntactically correct.
To begin with, Lines 12 and 19 of Figure 4.39 and Lines 5 and 6 of
Figure 4.40 (among several others), will not compile since the variables
next and l are declared in the class NewNode, which has been separated
from the other two classes. The solution to this problem is to change the
access modifier of the variables next and l from private (see Lines 60 and
61 of Figure 4.34 ) to package access. This is done by removing the
keyword private from the declaration of these variables as shown on Lines
2 and 3 of the class NewNode (Figure 4.38 ), and placing all three classes
in the same Java package (directory). A data member of a class with
package access can be accessed from any other class defined in the
255

package.
256

Figure 4.40 The Singly Linked List Class, SllExternalIterator, that
Supports an External Iterator Class, SllIterator
Separating the class definitions causes a second syntax problem. The
list header, h, defined on Line 2 of Figure 4.40 is no longer accessible
from the Iterator class (e.g., Line 5 of Figure 4.39 ). The solution to this
problem is to add a method (typically) named iterator to the class
SllExternalIterator (Lines 58–60 of Figure 4.40 ). This is the method the
client will use to create the external iterator objects. For example,
assuming the name of the client’s singly linked list object is boston, in
order to create two iterator objects(iterator1 and iterator2) the client would
invoke the method iterator as shown below:
The method iterator then invokes the SllIterator class’ constructor
(Line 59 of Figure 4.40 ), passes it the contents of the singly linked list
header, h, and returns the location of the newly created SllIterator object to
the client. When the code of the SllInterator class constructor executes
(Lines 4–7 of Figure 4.39 ), it stores the location of the singly linked list
passed to it as the parameter h (Line 4 of Figure 4.39 ) not only in the
variable, ip, but also in its own NewNode reference variable h (Line 6 of
Figure 4.39 ), declared on Line 3 of Figure 4.39 . This is the variable used
by the reset method to reinitialize the iterator to the beginning of the list
(Line 9 of Figure 4.39 ).
Figure 4.41 presents an application program that declares three
(external) SllIterator references i1, i2, and i3 (Line 12), to operate on the
data structure boston, an object in the class SllExternalIterator (Line 3).
The iterators are “attached” to the structure boston on Lines 13–15. Iterator
257

i1 is used on Lines 17–20 to traverse the linked list structure and output
original data set. Iterator i2 is used on Lines 22–29 to traverse the structure
and add the area code “631” to all the phone numbers, and finally iterator
i3 is used on Lines 31–34 to traverse the structure and output the modified
data set.
Figure 4.41 An Application That Declares Its Own Iterators to
Operate on a Singly Linked List
4.6 Java’s LinkedList Class and ListIterator
Interface
The Java class LinkedList, contained in the package java.util, is an
258

unencapsulated generic implementation of a double-ended, doubly linked
list structure. The structure does not support key field mode access, but
rather is accessed in the node number mode or through the use of a client
declared external iterator. The class contains methods to insert and fetch
objects (add and get), to easily allow a LinkedList object to be used as a
Stack or a Queue (addFirst, getFirst, removeFirst, addLast), and to
attached a client defined iterator object to a LinkedList object. The class
implements ListIterator, using techniques similar to those presented in
Figures 4.39 and 4.40 , providing methods for performing forward and
backward traversals through a LinkedList object.
Table 4.3 presents a description of some of the methods in the class
LinkedList, and Table 4.4 presents some of the methods specified in the
ListIterator interface. Figure 4.42 presents a brief application program (and
the output it produces) that demonstrates the use of the iterator methods
add, hasPrevious, previous, and next to access and operate on a data
structure named dataBase, a LinkedList object. The items inserted into the
structure are objects in the class Listing2 (Figure 4.35 ). The application
also demonstrates the structure’s lack of encapsulation.
259

260

Figure 4.42 An Application that Demonstrates the use of the Java
API class LinkedList and the ListIterator Interface Methods
EXERCISES
Knowledge Exercises
1. What is one advantage of a linked list structure over array-based
261

structures?
2. What is one advantage of array-based structures over a linked list
structure?
3. Explain the term “fragmented memory.”
4. What is “dynamic” about dynamic data structures?
5. What is the only condition that would cause the Insert operation of a
dynamic data structure to return a “data structure full”error?
6. There are many types of linked lists; however, the nodes in all of
them have one thing in common. What is it?
7. The last node in a singly linked list, by definition, does not store the
address of another node. Instead, what does it store?
8. All singly linked lists contain a reference variable called a “list
header”. What is sorted in it?
9. What is the advantage of implementing a stack using a singly linked
list?
10. A data set consisting of four information nodes A, X, P, and C, is
stored in a singly linked list. A field in each node named next is used to
“link” them together. The memory locations of the nodes are: 200, 30,
500, and 60, respectively. The memory cell h is the list header. The
nodes are to be stored in alphabetic order in the linked list.
a) Draw a picture, similar to Figure 4.2 a , showing the relative position
of the nodes in main memory.
b) Add arrows to the picture drawn in part (a) to show the ordering of
the nodes starting at the list header (see Figure 4.3 ).
c) Give the contents of the list header.
d) Give the contents of the next field of each node.
e) Draw the standard depiction (see Figure 4.4 ) of the four nodes in the
linked list.
f) Add the node locations to your answer to part (e), as shown in Figure
4.5 .
g) Assuming a field named back, was added to each node to order them
in reverse alphabetic order, give the contents of this field for each node.
11. Draw the implementation level depiction (see Figure 4.9 ) of the
nodes described in the previous example. (Make up the memory
location of the dummy node and the locations of the other linked nodes
262

that are used to implement the structure.)
12. Figure 4.10 depicts an initialized singly linked list. Verify that the
singly linked list Insert algorithm is correct when it is used to add a
new Listing to an empty singly linked list.
13. Suppose that when a node is added to a singly linked list, it
becomes the new last node. Assuming the list header is named h, and
the link field is named next, give the pseudocode algorithm for this
approach to the Insert algorithm.
14. Give the dominant term in the speed function of the Insert
algorithm described in the previous exercise.
15. The reference variable p references a node in the middle of a long
singly linked list, and q points to the node just before the node that p
references.
a) Give the standard graphical representation of the list including the
reference variables p and q.
b) Modify the graphic to show the deletion of both the node p points to
and the node that follows it.
c) Give the pseudocode to accomplish the deletion of the two nodes.
16. Two linked lists, L1 and L2, are pointed to by the lists headers h1
and h2 respectively. A reference variable, p, stores the address of a
node in list L1. The entire list L2, is to be inserted into list L1 just after
the node referenced by p.
a) Give the standard graphical representation (Figure 4.4 ) of the two
lists. Include the reference variable, p.
b) Modify the graphic to show the steps necessary to accomplish the
insertion of the list L2 into the list L1.
c) Give the pseudocode to accomplish the insertion of the list L2.
17. For the two linked lists described in the previous example, give an
algorithm to “shuffle” the two linked lists into one linked list. After the
shuffle operation the odd nodes (i.e., the first, third, fifth, etc.) will be
the nodes from L1, and the even nodes (i.e., the second, fourth, sixth,
etc.) will be the nodes from L2. (Hint: develop the algorithm
graphically and then translate it to pseudocode.)
18. Give the pseudocode of the four basic operation algorithms of a
sorted singly linked list. (Hint: develop the algorithm graphically and
then translate it to pseudocode.)
263

19. Give the speed functions of the four basic operations for a sorted
singly linked list and the average operation speed (assume all
operations are equally probable).
20. Give the ratio of the average speed of an Unsorted-Optimized
array structure to the average speed of a SinglyLinkedList structure,
assuming each structure contains one million information nodes and all
operations are equally probable.
21. Describe the garbage collection method for the SinglyLinkedList
structure, and give the line number of the code presented in Figure 4.15
that accomplishes (i.e., actually initiates) the “garbage collection.”
22. A SinglyLinkedList structure is used to store a data set. Calculate
its density if:
a) Each of the client’s information nodes contains 8 bytes of information,
and there are 50 nodes in the data set.
b) Each of the client’s information nodes contain 200 bytes of
information, and there are one million nodes in the data set.
23. Give a plot showing the variation in density with the number of
nodes, n , stored in a Singly-LinkedList structure. Assume each node
contains 10 information bytes and that the range of n is 2 ≤ n ≤ 100.
24. Give an example of when it would be more efficient to use an
iterator to access the nodes in a singly linked list.
Programming Exercises
25. A database is to be developed to keep track of student information
at your college. It will include their names, identification numbers, and
grade point averages. The data set will be accessed in the key field
mode, with the student’s name being the key field. Code a class named
Listing that defines the nodes. Your class should include all the
methods in the class shown in Figure 2.28 . Test it with a progressively
developed driver program that demonstrates the functionality of all of
its methods.
26. Using the generic capabilities of Java 5.0, modify the
implementation of the structure SinglyLinkedList presented in Figure
4.15 to make it fully generic. Include a driver program that
demonstrates the functionality of the class with two homogeneous
SinglyLinkedList objects that store two different kinds of nodes.
264

27. Code an application program that keeps track of student
information at your college (see Exercise 25). Include their names,
identification numbers, and grade point averages in a fully
encapsulated, homogeneous singly linked list. When launched, the user
will be asked to input the initial number of students and the initial data
set. Once this is complete, the user will be presented with the following
menu:
Enter: 1 to insert a new student’s information,
2 to fetch and output a student’s information,
3 to delete a student’s information,
4 to update a student’s information,
5 to output all the student information, and
6 to exit the program.
The program should perform an unlimited number of operations until
the user enters a 6 to exit the program. If the user requests an operation
on a node not in the structure, the program output should be “node not in
structure.” Otherwise, the message “operation complete” should be
output.
28. Code a class that implements a homogeneous stack structure using
a singly linked list. The Push operation will insert a node at the front of
the linked list, and the Pop operation will fetch and delete the node at
the front of the linked list. Include a progressively developed driver
program to demonstrate the functionality of the Push and Pop
operations.
29. Code an application program that keeps track of student
information at your college. Include their names, identification
numbers, and grade point averages in a fully encapsulated,
homogeneous sorted singly linked list structure. When launched, the
user will be asked to input the initial number of students and the initial
data set. Once this is complete, the user will be presented with the
following menu:
Enter: 1 to insert a new student’s information,
2 to fetch and output a student’s information,
3 to delete a student’s information,
4 to update a student’s information,
265

5 to output all the student information in sorted order, and
6 to exit the program.
The program should perform an unlimited number of operations until
the user enters a 6 to exit the program. If the user requests an operation
on a node not in the structure, the program output should be “node not in
structure.” Otherwise, the message “operation complete” should be
output.
30. Redo the application described in the previous exercise using a
doubly linked list to store the nodes. Add a seventh user option to
output all the nodes in descending order.
31. Implement the dynamic version of a stack illustrated in Figures
4.24 -4.26 using the class SinglyLinkedList presented in Figure 4.15 .
Provide a driver program that demonstrates that the Push and Pop
operations function properly.
32. Code a class named SLLQueue that uses a double-ended singly
linked list to implement a Queue as described in this chapter. Provide a
driver program that demonstrates the constructor, enqueue, and
dequeue methods function properly.
33. Redo Exercise 27, but this time use the Java API class LinkedList
to store the nodes. The list should be accessed in the node number
mode. Add a seventh option to the class to increase all the student
GPA’s by a given amount using an iterator object.
34. Code a GIU program that visually demonstrates the changes that
take place to the linked nodes, and the other data members, that make
up a SinglyLinkedList object when each of the four basic operations
are performed. When the program is launched, the linked list should be
shown in its initialized state. Six buttons should be available to the
user: one for each of the four basic operations, a “reinitialize” button,
and a “quit” button. Text boxes should be provided to permit the user
to enter a node’s information prior to an Insert or Update operation, and
to display the results of a Fetch operation.
1 An inner class is a class defined within a class.
2 Once n and h are accessed on lines 1 and 2a, modern compilers would store them in
CPU registers for the remainder of the algorithm.
4 Alternately, a parameter can be added to the singly linked list insert method to
indicate which end of the list is to receive the new node.
5 The compareTo method in the Strmq class, which is invoked by the compareTo
method in the Listing class ( Figure 2.16 ), does this.
266

6 L is equal to 2 in the structure depicted in Figure 4.32 .
7 Better programming practice would be to implement the class
SinglyLinkedListIterator as an extension of the class SinglyLinkedList, but that would require
the reader to refer to the code of both classes during subsequent discussions in this chapter.
8 The iterator, i, and the toString methods are used in place of the showAll method in
order to illustrate the iterator’s use.
267

CHAPTER 5
Hashed Data Structures
OBJECTIVES
The objectives of this chapter are to familiarize the student with the
features, uses, and implementation of hashing and hashed data structures,
and to understand how to convert these data structures to generic
implementations. More specifically, the student will be able to
Explain the advantages and disadvantages of hashed structures and
be able to quantify their performance.
Understand the various memory models programmers use to
represent static and dynamic hashed structures, and understand the
advantages and disadvantages of these representations.
Understand the role of key preprocessing algorithms, hashing
functions, and collision algorithms in hashed data structures and be
familiar with the features of these that lead to good performance.
Understand basic numeric and alphanumeric key preprocessing
algorithms, hashing functions, and collision algorithms and be able to
implement them.
Implement a fully encapsulated perfect hashed structure accessed in
the key field mode, and be able to understand and quantify its
performance.
Implement a fully encapsulated nonperfect hashed structure accessed
in the key field mode, and be able to understand and quantify its
performance.
Convert a hashed data structure to a generic implementation, and
understand the role of the Java hashCode method in this conversion.
Develop an application whose data structure is an object in Java’s
API HashTable class, understand the advantages and disadvantages of
the class, and operate on the structure using the class’s operation
268

methods.
Understand the techniques used to dynamically expand a hashed
structure at run-time, and the performance of dynamic hashed
structures.
5.1 Hashed Data Structures
Each of the data structures we have studied so far—array-based
structures, restricted structures (Stacks and Queues), and linked lists—
have strengths and weaknesses. All of these structures have high densities.
Restricted structures are fast, but they do not support access in the key
field mode. The array-based and linked list structures do offer access in the
key field mode, but are slow because they use a sequential search to locate
a node.
Hashing is an alternate search technique (more accurately a set of
techniques) for locating a node in the key field mode. Unlike the
Sequential Search algorithm, hashing algorithms are fast. When you want
speed, think hashing. Because of their ability to rapidly locate a node, data
structures that use hashing access algorithms are in wide use.
In this chapter we will study several hashing algorithms and the data
structures that use them. These data structures, called hashed data
structures, vary in speed. However, when properly implemented, all of
them are faster than the structures we have studied thus far.
There is a down side to hashed structures. For some applications,
their overhead can be very high. However, there are many applications
where the overhead of even the fastest hashed structure approaches zero.
With their guarantee of speed and the possibility of low overhead, hashed
data structures should always be considered in our designs.
5.2 Hashing Access Algorithms
Hashing access algorithms are a collection of algorithms that share a
common characteristic: the given key is used to compute an index or a
location into a primary storage area. Since the primary storage area is a
group of sequentially numbered storage cells, it is normally implemented
as an array. Sometimes the nodes themselves are stored in the primary
storage area at the computed location (see Figure 5.1 a ), and sometimes
paths to the nodes are stored there. The paths, stored in the primary storage
area array, can be a reference variable that stores a node’s location (see
269

Figure 5.1 b ), the beginning of a linked list that contains nodes (see Figure
5.1 c ), or the location of a secondary array that stores a group of nodes
(see Figure 5.1 d ). 1
Figure 5.1 Four Uses of the Primary Storage Area
This ability to compute an index into the primary storage area from
the given key is what gives hashing access algorithms their speed because
computations are performed rapidly by modern CPU’s. In contrast,
sequential access algorithms perform time-consuming memory accesses to
fetch keys from the data structure in order to compare them to a given key.
Memory accesses are slow, computations are fast; thus, sequential
algorithms are slow, hashing algorithms are fast.
The computation of the index into the primary storage area is
performed using a mathematical function, h , that uses the given key as the
independent variable. This function, called the hashing access function , is
expressed as:
where: i p is the index (in Java, an integer greater than or equal to
zero) into the primary storage area,
h is the hashing (or mapping) function of a particular hashing access algorithm, and
k is the contents of the key field of the node being accessed.
Figure 5.2 illustrates the hashing process used to determine a primary
storage index.
270

Figure 5.2 The Hashing Process
The function, h , is said to hash or map a key into a primary storage
area location. There are many hashing functions. Two of the most
simplistic functions used for numeric keys are the Division Hashing
function and the Direct Hashing function. These functional relationships
are:
i p = h (k ) = k mod N =
k % N
Division Hashing
function
i p = h (k ) = k Direct Hashing
function
where: N is the number of storage locations allocated to the primary
storage area
k is the given key.
Comparing the two hashing functions, the Division function uses the
division remainder as the primary index, while the Direct Hashing
function uses the key as the primary index. The number of storage
locations allocated to the primary storage area, N , should not be confused
with the maximum number of nodes that will be stored in the structure, n
max . There are times when they are equal, but most often they are not. In
hashing jargon, the ratio n max /N is referred to as the maximum loading
factor .
If the key is negative or non-numeric (e.g., −36 or “Jones”) then some
form of preprocessing is performed on the key to convert it to a numeric,
non-negative value called a pseudo key and then the pseudo key, pk , is
used as the independent variable in the hashing function. Figure 5.3
illustrates the expanded process of converting a key to a primary storage
area location when preprocessing is required.
Preprocessing algorithms, and other motivations for using them, will
be discussed in subsequent sections of this chapter.
Figure 5.3 The Hashing Process with Preprocessing
271

Figure 5.4 A Stadium Ticket Database Node
Table 5.1
Comparison of the Division and Direct Hashing Algorithms
Ticket Number
(Key Value)
Calculated Location Using the Two Access Algorithms
Division Algorithm (N = 10,054) Direct Algorithm
342556 720 342,556
000000 0 0
999999 4463 999,999
211854 720 211,854
5.2.1 A Hashing Example
To illustrate the use of the Division and Direct Hashing algorithms,
we will consider a stadium ticket database that will store the ticket number
(key field) and the purchaser’s name for an event to be held in a 10,054
seat stadium. The ticket number is a six digit encoding of the seat number,
the event number, and the event date in the range 000000 to 999999.
Figure 5.4 depicts a typical node and the information for ticket number
342556.
For this example, we will assume that all 10,054 tickets will be stored
in the data structure, and that there will be 10,054 storage locations (one
per ticket) allocated to the primary storage area for the division algorithm
(N = 10,054). Under these assumptions, the index computed for ticket
number 342556 by our two hashing functions is:
i p = h (k ) = k % N = 342,556 %
10,054 = 720
Division Hashing
function
i p = h (k ) = k = 342,556 Direct Hashing
function
These indices and the indices computed for several other ticket
numbers are tabulated in Table 5.1 . The ticket numbers presented in this
table were selected to illustrate two issues that arise when using these two
access algorithms. The first issue is the Division algorithm’s inability to
map keys into unique indices, and the second issue is the potential of high
272

overhead associated with the Direct Hashing algorithm. We will first turn
our attention to the problem of nonunique indices associated with the
Division algorithm.
Figure 5.5 The Hashing Process with Preprocessing and Collision
Resolution
As shown on rows 1 and 4 of Table 5.1 , the Division algorithm maps
the keys 342556 and 211854 into the same index, 720 (342,556 % 10,054
= 720 and 211,854 % 10,054 = 720). The mapping of two keys into the
same index is called a collision , and the resolution of a collision is left to
another collection of algorithms called (you guessed it) collision
algorithms . Unfortunately, collision algorithms reduce the speed of access
since they require another processing step usually involving multiple
memory accesses. Figure 5.5 illustrates the inclusion of a collision
algorithm into the process of converting a key to a primary storage area
location.
There is a class of hashing algorithms, called perfect hashing
algorithms that map every key into a unique primary storage area index.
This unique mapping means that collisions cannot occur, and therefore
structures that use perfect hashing algorithms do not include collision
algorithms. As a result, they perform better from a speed viewpoint. Our
Direct Hashing function is an example of a perfect hashing algorithm since
a unique ticket number becomes the unique index.
However, all that glitters is not gold. As shown in the third column of
Table 5.1 , the information for the lowest and highest ticket numbers,
000000 and 999999, are stored in primary storage locations 0 and 999,999
respectively. Therefore, we would have to allocate 1,000,000 locations to
the primary storage area even though only 10,054 of them (one for each
seat in the stadium) would be used. The maximum loading factor would be
low (0.01 = 10,054 / 1,000,000), which is indicative of a low density
structure. In contrast, the Division algorithm can only generate indices in
the range 0 to N − 1 (the range of the remainder when dividing by N ), and
since N was chosen to be the number of tickets, its memory requirements
are much more modest.
If properly designed, however, the maximum loading factors of
273

perfect hashing functions for some applications can approach a value of
one. A perfect hashing function that minimizes the unused portions of the
primary storage area is called a minimum perfect hashing function.
Unfortunately, designing a hashing function that even approaches a
minimal function is not a simple process. Aside from the complexity of
their design, a minimal hashing function is usually tied to a particular
application, requires that the designer know the particular subset of the
application’s keys that will be stored in the data structure, and is valid only
for that subset of keys. For example, to design a minimal perfect hashing
function for our stadium application we would have to know the subset of
ticket numbers that correspond to the 10,054 tickets for a particular event
number and date. Once designed, the hashing function would only be valid
for that (static ) set of ticket numbers and, therefore, it could not be used
for other events.
These restrictions limit the use of perfect hashing functions and a
further discussion of their design process is, with one exception, beyond
the scope of this book. The exception is a set of applications where perfect
hashing can be easily applied because the Direct Hashing function can be
used without producing low densities; these applications share common
characteristics which make the Direct Hashing algorithm approach a
minimal perfect hashing algorithm. In the next section, we examine the
features of the applications that make this possible and develop a perfect
hashed data structure based on the Direct Hashing algorithm. The data
structures discussed in the remainder of this chapter will utilize nonperfect
hashing algorithms.
5.3 Perfect Hashed Data Structures
Before we examine the features of an application that make the Direct
Hashing algorithm approach a minimal hashing algorithm, we will
formally state the definition of a perfect hashing function. Consistent with
our previous discussions, we can define a perfect hashing function as:
Perfect Hashing Function
A perfect hashing function is a function that maps each key, in a static set
of keys,
into a unique index in the primary storage area.
A static set of keys is the subset of all possible values of the key for
which the function is valid. For example, if the keys were comprised of
letters, a static set of keys would be the keys “Bob,” “Mary,” “Alice,” and
274

“Tim” and the hashing function would be designed to process only these
four keys. As a result, there would be no guarantee that the function would
produce a valid index when processing a key outside of the predefined
subset (e.g., “Harry”).
By unique index we mean that each location in the primary storage
area is dedicated to a particular value of the key. No other key is mapped
into that location by the hashing function. When preprocessing is used, the
combination of the preprocessing algorithm and hashing function must
produce a unique index for each key. The unique mapping eliminates the
occurrence of collisions, which degrade the speed performance of hashed
data structures. As a result, hashed data structures based on perfect hashing
algorithms, which we will refer to as perfect hashed structures , are the
fastest of all the data structures we will study.
To identify the feature of an application that permits the Direct
Hashing function to approach a minimal perfect hashing function we will
revisit the use of this algorithm in our stadium ticket database example.
The reason the loading factor was low for that application was that there
were 1,000,000 possible ticket numbers (key values) and a place in the
primary storage area had to be provided for each of them, even though
only 10,054 of these keys would actually by stored. Now consider the case
where the 10,054 ticket numbers no longer include an encoding of the
event number and date. In this case, the ticket numbers can be the seat
numbers and there are only 10,054 possible key values; meaning, we only
need to allocate 10,054 locations in the primary storage area, all of which
will be used. Thus, an application’s feature that permits the Direct Hashing
function to approach a minimal perfect hashing function is:
Condition Under Which the Direct Hashing Function Approaches a
Minimal Perfect Hashing Function
Of the set of all possible key values, most (or all) of them will be stored in
the structure.
Even when conditions are such that a moderate percentage of all
possible keys will be represented in the data structure, a Direct Hashing
algorithm-based structure can produce acceptable densities. For example,
when only half the keys are represented, a density greater than 0.8 is
achieved as long as the node width is greater than 32 bytes. One caveat is
in order: when the keys are nonnumeric, the percentage of the possible key
values represented in the structure is usually too low to produce acceptable
densities.
275

We will now develop a data structure based on the Direct Hashing
function.
5.3.1 Direct Hashed Structure
Before developing the operation algorithms for this structure, we will
discuss a numeric key preprocessing algorithm commonly used with this
structure to process numeric keys. We will not discuss a string key
preprocessing algorithm because string preprocessing for perfect hashing
schemes that produce acceptable densities is highly application specific
and beyond the scope of this book.
The Subtraction Preprocessing Algorithm
In a numeric key application when the minimum key value (k min ) is
nonzero, a subtraction preprocessing algorithm is used to compute a
pseudo key. As shown in Figure 5.3 , the calculated pseudo key is then
used in the Direct Hashing algorithm. This prevents the Direct Hashing
algorithm from generating negative indices when the minimum key is
negative, and improves the density when the minimum key is positive. In
the latter case, without preprocessing, the minimum index generated by the
direct hashing algorithm would be k min , and the low area of primary
storage (indices 0 to k min − 1) would be unused. The Subtraction
Preprocessing algorithm is:
Subtraction Preprocessing Algorithm
pk=k–k min
276

where:
pk is the calculated pseudo key,
k is the given key that can assume negative values, and
k min is the minimum value the key can assume.
Table 5.2 illustrates the use of the algorithm for minimum key values
of −3 and 112, which prevents a negative index from being produced by
the Direct Hashing function.
Primary Storage Area
As mentioned in Section 5.2 , the primary storage area is normally
implemented as an array, but, we must decide what will be stored in the
array. We have two choices in Java: Each element of the primary storage
array can store a reference to a single node (just as they did in the
Unsorted-Optimized array structure discussed in Chapter 2 ), or each
element can store a path to a group of nodes (see Figures 5.1 c and d ). We
will use the first option: each element of the array will store a reference to
a single node, because it is simpler. The second option will be discussed
later in the chapter.
When an array of references is created in Java, each element is
initialized to null . Realizing this, we will use a null element to indicate
that the node whose key maps into the element is not in the structure.
Figure 5.6 shows the primary storage area in its initialized state.
Figure 5.6 Primary Storage Area in its Initialized State
Operation Algorithms
277

Having decided on what will be stored in the array and the condition
for a nonexistent node, we can now examine the basic operation
algorithms. The four basic operation algorithms for our perfect hashed
structure all begin the same way. We use the Direct Hashing function,
preceded by a preprocessing algorithm when necessary, to determine the
index into the primary storage area, i p . Then, using this element of the
array as the reference to the node, we perform the operation on the node.
To insert a node into the structure, we deep copy the node into a
newly created node, and store a reference to the new node in element i p of
the primary storage array. Assuming the name of the primary storage array
is data, that the node to be inserted is referenced by newNode, and its key
is targetKey, the pseudocode of the Inset algorithm is:
Direct Hashed Insert Algorithm
This is an encapsulated Insert operation because the address of the
newly created node is stored inside the data structure, not the address of
the client’s node. Figure 5.7 shows the structure after two telephone listing
nodes have been added to the structure. It assumes that the preprocessing
and hashing algorithms map the keys “Bill” and “Bob” into indices 4 and 1
respectively.
Figure 5.7 A Direct Hashed Structure after Bill’s and Bob’s Listings
have Been Inserted
278

Figure 5.8 The Process of Fetching Bill’s Listing
The Fetch operation returns a deep copy of the requested node or a
null value if the requested node is not in the structure. Returning a deep
copy hides the address of the listing stored inside the data structure from
the client and thus maintains the structure’s encapsulation. Figure 5.8
illustrates the process of fetching Bill’s listing. Assuming the name of the
primary storage array is data, that the key of the node to be deleted is
targetKey, the pseudocode of the Fetch algorithm (which defers the
language-specific checking for insufficient system memory to the
implementation) is:
The Direct Hashed Fetch Algorithm
To perform a Delete operation, we simply set element: data[ip] to
null . The pseudocode of the Delete algorithm is:
The Direct Hashed Delete Algorithm
279

Figure 5.9 shows the data structure after Bob’s listing is deleted, but
before the deleted node’s storage is collected by Java’s run-time memory
manager.
As we have done with previously studied data structures, for the
purposes of brevity, we will use the Delete and Insert operations in the
Update algorithm. Assuming that the node with the key field targetKey is
to be updated to the contents of the node newNode, the pseudocode of the
Update algorithm (which defers the language-specific checking for
insufficient system memory to the implementation) is:
Figure 5.9 Data Structure after Bob’s Listing is Deleted
The Direct Hashed Update Algorithm
Performance
280

We will now examine the performance of the Direct Hashed data
structure, by examining the speed of the basic operation pseudocode, and
the amount of overhead memory required to implement this code.
Speed of the Structure
To analyze the speed of the structure, we will perform a Big-O
analysis to bound the number of memory access instructions executed as n
, the number of nodes stored in the structure, gets large. We will assume
that an optimizing compiler will use a CPU register to store a memory
cell’s value after the first time it is accessed.
Looking at the Insert, Fetch, and Delete algorithms, Line 2 of each
algorithm invokes the preprocessing algorithm. The number of instructions
executed in this algorithm is not a function of the number of nodes in the
structure, and an optimizing compiler would store most of the code’s
variables in CPU registers. The one exception would be if the keys were
non-numeric, which is a case we are not considering. When our
Subtraction Preprocessing algorithm is used, a total of two memory
accesses is required to preprocess a key; one memory access to fetch the
key, and one to fetch the maximum key value.
Examining the remainder of the pseudocode instructions in the Insert,
Fetch, and Delete algorithms, we see that the number of instructions
executed is also not dependent on the number of nodes in the structure.
Rather, the algorithms perform one access into the primary storage array
on Line 5 of each algorithm to access data[i], and the Delete algorithm
performs an additional memory access on Line 8 to overwrite data[i].
Therefore, a total of one memory access is performed during an Insert or
Fetch operation (three with preprocessing), and two memory accesses are
performed during a Delete operation (four with preprocessing). Thus the
speed complexity for these operations, including the preprocessing term, is
O(1).
Overhead and Density
Certainly, the speed of this structure is very good news. Let us now
turn our attention to the overhead and density of the structure. As we have
previously discussed, this could be very high or very low depending on the
application. The source of the overhead for this structure is the same as
that of the array-based structures. It is the array of reference variables, the
primary storage area array. Unlike the array-based structures however,
281

where the array was sized to the maximum number of nodes that would be
stored in the structure at one time, the Direct Hashed structure’s array is
sized to the maximum value of the key (or pseudo key when preprocessing
is performed). Thus, the overhead can become significant if the number of
nodes that will be stored in the structure is small compared to the size of
the set of all possible keys.
The density of any data structure, D, is defined as:
with:
Using the notation:
n is the number of nodes stored in the structure,
w is the width of a node, in bytes,
N is the number of elements in the primary storage array, and
w a is the width of each element of the array, in bytes,
the information bytes and total bytes of the data structure becomes:
Thus, the density of our Direct Hashed structure, Ddh , becomes:
Dividing the numerator and denominator of the right side of this
equation by n * w we obtain:
Examining the previous equation, we see that the density is dependent
on two ratios:
• N / n , the ratio of the size of the array to the number of nodes stored
in the structure, (the inverse of the loading factor).
• wa / w, the ratio of the size of the array elements to the node width.
The term N / n (the inverse of the loading factor) in the denominator
of this equation was not present in the density equation of the array-based
structures, and the density of this structure is highly dependent on it (and
thus the loading factor). When the loading factor is one, the density of this
structure is the same as the density of the array-based structures and we
have the best of both worlds: speed and high density. 2
282

In Java, reference variables occupy 4 bytes, and so wa = 4.
Substituting this value for wa in the above density equation and replacing
N / n with the the reciprocal of the loading factor l , we obtain:
Density of a Direct Hashed Data Structure
Ddh = 1/(1+4/(l *w))
Where:
w is the width of a node, in bytes, and
l is the loading factor (n / N ).
Figure 5.10 presents the variation in density with a loading factor for
the Direct Hashed structure for various client node widths, w. Substituting
a density of 0.8 into the density equation and solving for w * l , we see that
a density of 0.8 or better is achieved when w * l ≥ 16, or w ≥ 16/l . Figure
5.11 presents the client node widths that result in a density of 0.8 or greater
at various loading factors. This figure demonstrates that there is a there is a
wide range of loading factors that produce good density as long as the
client’s nodes are wide enough.
Table 5.3 summarizes the performance of the Direct Hashed
structure, and includes the performance of the previously studied structures
for comparative purposes. Its speed is the fastest of all the data structures
we have studied, or will study, and its density for some applications can be
high. The implementation of this perfect hashed structure is left as an
exercise for the student.
Figure 5.10 Density of the Direct Hashed Structure
283

Figure 5.11 Node Widths and Loading Factors that Result in a
Density of 0.8
5.4 Nonperfect Hashed Structures
When the conditions of an application require high speed but are such
that perfect hashing cannot be used (e.g., the key set is not static, the
density would be too low, or an efficient perfect hashing function cannot
be discovered), a hashed structure based on a nonperfect hashing algorithm
provides the next fastest alternative. These structures, which we will refer
to simply as hashed structures , do not provide a unique location in the
primary storage area for every allowable value of the key. As a result, two
or more keys can map into the same primary storage index. When this
happens, we say that a collision has occurred and so the mapping process
must be expanded (as depicted in Figure 5.5 ) to include a collision
resolution algorithm.
284

Figure 5.12 Hashed Data Structure Flowchart
There are many collision algorithms, preprocessing algorithms, and
285

hashing functions used by hashed data structures to determine the primary
storage area location for a given key. Regardless of the particular
algorithms used, they are generally executed as shown in Figure 5.12 . If
the array index generated by the preprocessing and hashing algorithms
result in a collision, a collision algorithm is repeatedly executed (see
bottom of Figure 5.12 ) until a correct location (which, in the case of the
Insert operation, is an unused location) is found.
For example, suppose that Bill’s listing was to be inserted into a
hashed data structure that already contained Bob’s listing referenced by
element 1 of the primary storage array. Let us assume, as shown on the left
side of Figure 5.13 a , that the key “Bill” is preprocessed and hashed into
index 4, and that element 4 of the primary storage array was unused. Bill’s
listing would then be stored in the structure referenced by element 4 (see
Figure 5.13 a ).
Figure 5.13a Inserting Bill’s Information
286

Figure 5.13b The First Unsuccessful Attempt to Insert Tom’s
Information
Now suppose that Tom’s listing is to be inserted next and that the key
“Tom” is also preprocessed and hashed into index 4 (see Figure 5.13 b ). A
collision has occurred. We cannot reference Tom’s listing with element 4
since it is already pointing to Bill’s listing. Technically speaking, when
Bill’s information is inserted first, index 4 is no longer the correct place to
store Tom’s information. Therefore, the collision algorithm would be
executed to determine a correct (unused) location for Tom’s information.
Where Tom’s information will be inserted depends on the collision
algorithm used. As shown on the left side of Figure 5.13 c , some collision
algorithms use the incorrect index to generate the next location to be
considered. It is possible that the index produced by the collision algorithm
also has some other node stored in that element (see Figure 5.13 c ). In this
case, this new index is also incorrect, and the collision algorithm continues
to execute until a correct, or unused, element of the primary storage area is
located (see Figure 5.13 d ).
Continued execution of the collision algorithm is the weakness of
hashed structures, because each execution of the collision algorithm
involves memory accesses. Consider the case where half the nodes are
accessed before a correct location is found. Then the speed of the
algorithm, in Big-O notation, is O(n / 2). This is approximately the speed
of the array-based and linked list structures we previously studied, and the
speed advantage of hashed structures is lost.
Thus, it is important that the preprocessing, hashing, and collision
algorithms of a hashed data structure be carefully chosen in order to
minimize the number of collisions. Before studying three specific
algorithms and the features that minimize collisions, we will examine the
strategy used to size the primary storage area array for a hashed structure
because the size of this array can also have an effect on the number of
times the collision algorithm executes. This discussion will begin with a
definition of the term search length .
5.4.1 Search Length
Search length, L , a parameter used to describe the speed of all data
structures is defined as:
Search Length
287

The number of memory accesses required to locate the node.
As we have seen, perfect hashed data structures dedicate a unique
location in the primary storage area for each allowable value of the key.
There are no collisions, and thus the search length for perfect hashed
structures is always one (one access into the primary storage area to fetch
the location of the node).
For a hashed structure, when an operation is performed without a
collision, the search length is also one. Most often, however, there are
multiple collisions. Since the number of collisions varies from one
operation to another, an average search length, L avg , is used as a measure
of the speed of a hashed data structure. One would hope that the average
search length would be well below n / 2, since n / 2 would be the average
search length of the (slow) sequential search. As we shall now discuss, the
size of the primary storage area of a hashed data structure can greatly
influence the average search length of the structure.
Figure 5.13c The Second Unsuccessful Attempt to Insert Tom’s
Information
288

Figure 5.13d The Successful Attempt to Insert Tom’s Information
5.4.2 Primary Storage Area Size
A guideline for sizing the primary storage area of a hashed data
structure is that it be a small percentage higher than the maximum number
of nodes that the structure will contain, n max . Thus, we have:
Size of the Primary Storage Area, N , for a Hashed Structure
size of the primary storage area = N = n max + p * n max
where:
n max is the maximum number of nodes to be stored in the structure, and
p is a percentage expressed in decimal form (i.e., for 10%, p = 0.10).
Usually, for reasons we will discuss shortly, a value of p = 33%
results in optimum performance. Applying this guideline to our stadium
ticket database problem in which 10,054 nodes are to be stored (n max =
10,054), the size of the primary storage array would be 13,371 elements
(10,054 + 10,054 * 0.33).
Optimum Loading Factor
For hashed structures we normally talk about two loading factors, the
current loading factor, l , and the maximum loading factor l max . The
current loading factor is computed using the number of nodes currently in
the structure, n , and the maximum loading factor is computed using the
maximum number of nodes that will be stored in the structure, n max .
289

Loading Factors
l = n / N (current loading factor)
l max = n max / N (maximum loading factor)
where:
n is the number of nodes the structure currently contains,
N is the number of elements in the primary storage array, and
n max is the the maximum number of nodes that will be stored in the structure.
As nodes are added to a hashed structure, the current loading factor
increases until finally, when every node in the database is stored in the
data structure, the two loading factors are equal.
When we size the primary storage area of a hashed data structure
using the optimum value of p (33%), the maximum loading factor is fixed
at 0.75:
From a density viewpoint, a higher loading factor would be better.
However, from a speed viewpoint the number of collisions that occur are
lower when the loading factor is low because more of the primary storage
area is unused. The optimum value (l max = 0.75 with p = 33%), was
chosen as a middle ground where the number of collisions (search length)
is acceptably low and the density is still relatively high. This middle
ground is possible because, while the density decreases in a linear fashion
with loading factor, the average search length decreases in a more rapid
nonlinear fashion 4 (see Figure 5.14 ). As a result, at a loading factor near
1.0 (where the density is high) a decrease in loading factor produces a
much larger decrease in search length than it does a decrease in density.
This trend continues until we reach a maximum loading factor of 0.75
(called the optimum loading factor ) at which we reach a point of
diminishing returns. At this point, as shown in Figure 5.14 , the average
search length is between three and four memory accesses. Further
reductions in the loading factor produce relatively insignificant decreases
in the average search length while producing significant decreases in
density.
290

Figure 5.14 Variation of Average Search Length with Loading Factor
for Hashed Structures
To demonstrate that the density is relatively high at the optimum
loading factor, we will derive an expression for the density of a hashed
data structure, Dh , in terms of the loading factor and evaluate it for a
loading factor of 0.75. The density can be expressed as
where:
n is the number of nodes stored in the structure,
w is the number of information bytes per node,
N is the size of the primary storage area array, and
w a is the number of bytes per primary storage area array element.
Dividing numerator and denominator by n , and realizing that wa is 4
(i.e., reference variables in Java occupy 4 bytes), Dh becomes
where l is the loading factor, n /N . (This is the same density equation
derived for the Direct Hashed structure in Section 5.3.1 .) Substituting a
loading factor of 0.75, we obtain
This function, plotted in Figure 5.15 , yields densities greater than 0.8
for client node widths greater then 22 bytes per node.
291

Therefore, to achieve good speed (average search lengths of between
three and four memory accesses) while maintaining relatively high
densities, hashed data structures are designed so that the maximum loading
factor is 0.75. At this loading factor the primary storage area array is 33%
larger than n max (the maximum number of nodes stored in the data
structure at any given time).
Prime Numbers
There is one other issue to be considered when sizing the primary
storage area. For reasons that we will discuss later, if the size of the
primary storage area (N ) is a prime number, 5 then the number of
collisions for certain hashing and collision algorithms is minimized. Euclid
proved, in the third century BC, that there are an infinite number of prime
numbers. Table 5.4 presents the prime numbers less than 7000.
Figure 5.15 Density of Hashed Structures at the Optimum Loading
Factor (0.75)
The performance of some hashing and collision algorithms improves
even further if we choose only primes of the type “4k + 3”. Only a subset
of the prime numbers are 4k + 3 primes. A prime, P , is a 4k + 3 prime if
there is an integer k such that P = 4 * k + 3. Therefore, to determine if a
prime is a 4k + 3 prime we set the prime equal to 4 * k + 3, solve for k ,
and see if k is an integer. If it is, the prime is a 4k + 3 prime. Otherwise, it
is not.
To illustrate this test, we will arbitrarily choose P to be the prime 61.
Then, setting 61 = 4 * k + 3 and solving for k we have
292

Since k is not an integer, 61 is not a 4k + 3 prime.
Alternately, if we choose the prime 67, we find that
Therefore, since k is an integer, 67 is a 4k + 3 prime.
To take advantage of the speed increase associated with prime
numbers, we will increase the size of the primary storage area array to be
the 4k + 3 prime just above that calculated with our general sizing
guideline (N = 1.33n max ). This minor increase in N does not produce a
significant decrease in density. Thus, the optimum size of a hashed data
structure’s primary storage area array is:
293

Optimized Size of the Primary Storage Area, N, for a Hashed
Structure
N = next highest 4k + 3 prime 33% above the maximum number of nodes
that will exist in the structure at any given time, n max .
As an example, suppose that we were going to store 5000 nodes in a
hashed data structure. To calculate the size of the primary storage area,
first we multiply 5000 by 1.33 (to add 33%) and obtain 6650. Referencing
the table of primes, the first prime greater than 6650 is 6653. Testing 6653
to determine if it is a 4k + 3 prime, we find it is not (k = (6653 − 3) / 4 =
294

1662.5). The next highest prime is 6659, which is a 4k + 3 prime (k =
(6659 − 3) / 4 = 1664). Thus, the size of the primary storage area array
would be 6659 elements.
The code presented in Figure 5.16 calculates, and returns, the next
highest 4k + 3 prime a given percent (pct) above a given integer (n).
The method determines the requested 4k + 3 prime by testing
successively higher odd candidate integers the given percentage above n to
determine if they are prime numbers. When a prime number is found, the
prime is then tested to determine if it is a 4k + 3 prime. The process
continues, until a 4k + 3 prime is found.
Lines 6–8 make an initial guess of the 4k + 3 prime to be the next odd
integer (2 is the only even prime) the given percent above n . Lines 9–27 is
an outer loop that continues until the prime, found in the inner loop (Lines
10–20) is a 4k + 3 prime. The test for a 4k + 3 prime is performed on Line
21.
The test to determine if the candidate is, in fact, a prime is performed
by the for loop of Lines 12–15. The candidate prime is divided by every
integer between the square root of the candidate prime and 2. If the
remainder of the division is zero (Line 13) then the candidate is not a
prime. In this case, the candidate prime is increased to the next highest odd
integer (Line 17) and this new candidate prime is tested.
Starting the divisor, d, in the for loop at the square root of the
candidate prime, saves many iterations through the loop. A more brute
force approach would start the divisor at one less than the candidate prime.
For instance, for a candidate prime of 10,000,001, d would be initialized to
10,000,000, and the loop would execute 9,999,998 times. However, if we
initialize d to 3263 (the square root of 10,000,001) the loop executes only
3262 times.
To determine the validity of this technique, we must verify that if an
integer, p, is not evenly divisible by an integer less than the square root of
p, it is not evenly divisible by an integer greater than the square root of p.
The proof of this statement is in Appendix C , but we can gain an intuitive
understanding of its validity by considering the following example:
295

Figure 5.16 Method to Calculate a 4k + 3 Prime Number
Suppose the candidate prime was 100. Its square root is 10. If the for
loop were allowed to proceed through all the integers at and below 10, it
would consider divisors of 10 through 2 and would determine that 10, 5, 4,
and 2 are all evenly divisible into 100. If it were to examine the integers
above the square root of 100, it would find that 50, 25, and 20 are all
evenly divisible into 100. But 50 * 2 = 100, which means that since 50 is
an even divisor, 2 would also have to be (100 / 2 = an integer, 50). Thus,
by testing 2 we eliminate the need to test 50. Similarly by testing 4 we are,
in effect, also testing 25, 5 also tests 20, with 10 (the square root of the
candidate prime) being a symmetry (pivot) point.
Having gained an understanding of how to size the primary storage
area array in a way that minimizes collisions while producing acceptable
densities, we will now turn our attention to a discussion of the
preprocessing, hashing, and collision algorithms used in hashed data
structures.
296

5.4.3 Preprocessing Algorithms
As shown in Figure 5.5 , a preprocessing algorithm maps keys into
pseudo keys. Most hashing functions require that some form of
preprocessing be performed when the key field is alphanumeric or it can
assume negative values, because their independent variable must be a
positive integer.
When the keys can assume negative values, the Subtraction algorithm
discussed in Section 5.3.1 can be used, although hashed structures employ
a wide variety of algorithms to preprocess both numeric and alphanumeric
keys. Generally speaking, most string preprocessing algorithms combine
the bits that make up the characters of the key into four bytes, and the
resulting bit pattern is then evaluated as a positive integer. One of these
preprocessing algorithms is coded into the method hashCode(), discussed
in Section 5.4.6 , which is a member of the Java String class.
Naturally, a good preprocessing algorithm will produce pseudo keys
that infrequently collide. Three widely used preprocessing algorithms used
in hashed structures are: folding, Pseudorandom Averaging, and Digit
Extraction. They are often used to preprocess both numeric and
nonnumeric keys. We will begin our discussion of these algorithms with a
folding algorithm.
Fold-Shifting Preprocessing Algorithm
Generally speaking, folding algorithms divide the key field into
groups of bits, with the size (number of bits) of each group being the
desired size of the pseudo key. Then the groupings are treated as numeric
values, and arithmetically added to produce the pseudo key. Typically, one
grouping near the middle of the key, called the pivot , is selected as the
first operand in the addition. To maintain the size of the pseudo key,
arithmetic overflow from the higher order bit is ignored. Thus, folding can
be used to convert keys of any size into integer pseudo keys.
A popular version of a folding algorithm, called Fold-shifting , is
illustrated in Figure 5.17 . The top half of the figure is presented purely for
pedagogical purposes in that the algorithm’s operands and additions are
shown in the decimal (base 10) system. The 9 digit numeric key
987845369 is transformed into the 3 digit pseudo key 201, using 845 as
the pivot. A more realistic but slightly more complicated application of the
algorithm is shown at the bottom of the figure. The nonnumeric key “Al
McAllister” (with the blank removed), is converted into a 32 bit integer
297

pseudo key. The middle four characters, “Alli”, are used as the pivot point
and the Unicode bit patterns 6 of the four characters to the left and right of
it are added to it. The resulting integer, 4,132,249,406, is used as the 32 bit
integer pseudo key.
A variation on this basic technique reverses the characters in the
groupings before the addition is performed.
Figure 5.17 Fold-Shifting Key Conversion
The Java coding of the fold-shifting algorithm for non-numeric keys
is presented in Figure 5.18 . It preprocesses String keys of any length into
a 32 bit integer pseudo key. It does not ignore white space and is case
sensitive (e.g., “Bob Smith”, “BobSmith”, and “bob smith” map to the
pseudo keys 1,780,691,972, 706,950,148 and 1,344,743,749, respectively).
It should be kept in mind, however, that there are many keys that, when
processed by this algorithm, yield the same pseudo key (e.g.,
“MaryLynnTodd” and “MaryToddLynn” both produce the pseudo key
297,122,485) so the algorithm is not suited for use in perfect hashed
structures.
The loop on Lines 7–17 process all the characters of the key. Starting
on the left side of the key, Lines 8–10 build four character groupings and
298

place them into the variable grouping. Line 11 checks to see if the
grouping is complete; that is if grouping contains four characters or
contains the (possibly incomplete) last grouping. Line 12 builds the pseudo
key by adding the grouping to it. When the last grouping is processed, the
absolute value of the pseudo key is returned on Line 18. 7
Figure 5.18 A Fold-Shifting Non-Numeric Key Conversion Method
Pseudorandom Average Preprocessing
Pseudorandom preprocessing is a technique used to distribute keys
somewhat randomly about the primary storage area array. Generally
speaking, introducing randomness into the preprocessing algorithm tends
to reduce collision frequencies. The algorithm is:
Pseudorandom Preprocessing
pk = p 1 * k + p 2
where:
pk is the pseudo key,
k is the key, and
p 1 and p 2 are prime numbers.
The prime numbers are usually small and, once chosen, retain their
values over the life of the data structure. As an example, consider the
numeric key: 89,351. Assuming the primes p 1 and p 2 were chosen to be
299

13 and 53, respectively, then the pseudo key would be
pk = 13 * 89351 + 53 = 1,161,616.
Usually arithmetic overflow occurring in this calculation is ignored.
Pseudorandom preprocessing can also be applied to non-numeric key
applications after the key is converted to a numeric equivalent, perhaps
using the folding technique previously discussed.
Digit Extraction Preprocessing
Digit extraction, like folding, is a technique used to reduce the length
of multiposition (or multi-digit) keys, and also to introduce randomness
into the pseudo keys it generates. Several key positions (e.g., m of them)
are retained to form the pseudo key; all other key positions are ignored.
Thus, the multiposition key is mapped into an m position pseudo key
formed by concatenating the retained m key position values.
Key positions that add uniqueness to the keys are usually passed onto
the pseudo key. For example, consider keys with fifteen positions and
every other position’s value common to all keys. Two typical keys would
be: 12 34 56 78 90 12 34 5 and 92 84 36 28 10 42 54 2. These common
value positions would be ignored and only the first, third, etc. key
positions would be retained. In this example, m would be 8, and the key
123456789012345 would map into the pseudo key, 13579135. This
technique can be used to preprocess non-numeric as well as numeric keys.
Naturally, non-numeric keys would require further preprocessing (e.g.,
Fold-shifting).
In the case of non-numeric keys the bit patterns of the characters that
make up the keys are examined to determine which bits are not
contributing any uniqueness. These bit locations are then ignored. For
example, Java’s Unicode representation of lowercase letters all begin with
0000 0000 011 (e.g., “a” is 0000 0000 0110 0001 and “b” is 0000 0000
0110 0010). Therefore, if the keys were comprised of just lowercase
letters, only the rightmost five bits of the key’s characters would be
retained and processed through the Fold-shifting algorithm.
5.4.4 Hashing Functions
There are many hashing functions used in hashed structures to
compute the index into the primary storage area, i p . These include both
the Direct Hashing function and the Division Hashing function previously
300

discussed. Assuming preprocess is performed, they are:
i p = h (pk ) = pk Direct Hashing
function
i p = h (pk ) = pk mod N =
pk % N
Division Hashing
function
where:
N = the number of storage locations allocated to the primary storage
area, and
pk is the given key.
Figure 5.19 Open Addressing Collision Resolution of “Tom”
Colliding with “Bill” During an Insert Operation
When preprocessing is not performed, 8 the pseudo key, pk , in the
previous equations is replaced with the value of the key. Most hashed data
structures, however, do perform some form of preprocessing. Of the two
algorithms, the Division algorithm is more commonly used in hashed
structures.
5.4.5 Collision Algorithms
Collision algorithms can be divided into two categories: those
algorithms that compute an index into the primary storage area and those
that do not. The former group is called open addressing collision
algorithms and the latter group is called non-open addressing collision
algorithms.
When open addressing is used, each element of the primary storage
301

area stores a reference to a single node , and the collision algorithm always
generates indexes into the primary storage area array. During an Insert
operation, if the location generated by the preprocessing/hashing
algorithms is occupied, an open addressing collision algorithm looks for
the unused or open address in the primary storage area array. Thus the
name: open addressing. Figure 5.19 illustrates the resolution of the
collision of the key “Tom” with “Bill” by an open addressing collision
algorithm.
Figure 5.20 Non-Open Addressing Collision Resolution of “Tom”
Colliding with “Bill” During an Insert Operation
When non-open addressing is used, each element of the primary
storage area can store a reference to multiple nodes (see Figures 5.1 c and
5.1 d ). This reference locates all the nodes that the preprocessing /hashing
algorithms map into the element (synonyms ). Thus, during an Insert
operation, if the location generated by the preprocessing/hashing
algorithms is already occupied, a non-open addressing collision algorithm
stores the reference to the inserted node outside the primary storage area
array. Figure 5.20 illustrates the resolution of the collision of the key
“Tom” with “Bill” by a non-open addressing collision algorithm.
We will begin our study of collision algorithms with a study of open
addressing collision algorithms.
Linear and Quadratic Probing Collision Algorithms
Linear Probing and Quadratic Probing are two similar open
302

addressing collision algorithms, expressed as:
i p = i p +
1
Linear Probing collision
algorithm
i p = i p +
p 2
Quadratic Probing collision
algorithm
where:
i p is the index into the primary storage area array, and
p is the pass number (1, 2, 3,…) through the collision algorithm loop
depicted in the lower portion of Figure 5.12 .
Table 5.5
Indices Generated by the Linear and Quadratic Collision
Algorithms, for an Initial (Home) Address of 4
Collision Algorithm Pass
Number, p
Index Generated by the Collision
Algorithm
Linear Quadratic
1 5 5
2 6 9
3 7 18
4 8 34
5 9 59
6 10 95
7 11 144
8 12 208
9 13 289
10 14 389
11 15 510
12 16 654
13 17 823
14 18 1019
15 19 1244
16 20 1500
17 21 1789
18 22 2113
Both algorithms calculate the index of the next candidate location in
the primary storage area by adding an increment to the current,
unsuccessful location. In the case of the Linear algorithm the increment is
303

1, which amounts to a sequential search upward through the primary
storage area. The increment added to the current location in the Quadratic
algorithm is the square of the pass number through the collision
algorithm’s loop. Assuming the preprocessing/hashing algorithm produced
an initial, or home , index of 4, Table 5.5 gives the subsequent indices
generated by both algorithms, for the first 18 passes through each
algorithm.
Since both algorithms continue to add a positive increment to the
unsuccessful index it will continue to grow as the pass number increases,
and eventually it will exceed the maximum allowable index of the primary
storage area array, N − 1. For example, if N is 19 and the home address is
4, then the Linear algorithm generates indices that are out of bounds after
pass 14 and the Quadratic algorithm produces invalid indices after pass 3
(see the shaded cells in Table 5.5 ). To prevent this from happening, the
algorithms are modified to perform modulo N arithmetic which guarantees
that the calculated index remains in the allowable range of the primary
storage array, 0 ≤ i p ≤ N − 1. The modified algorithms are:
ip = (ip + 1) %
N
Linear Probing collision
algorithm
ip = (ip + p
2 )
% N
Quadratic Probing collision
algorithm
where:
i p is the index into the primary storage area array, and
p is the pass number (1, 2, 3,…) through the collision algorithm loop
depicted in the lower portion of Figure 5.12 .
For N = 19, both modified algorithms produce indices in the range 0
to 18, and the indexing into the primary storage area array is never out of
bounds. Table 5.6 presents the indices for the first 18 passes through the
revised algorithms.
Although modulo N arithmetic has solved the index-out-of-bounds
problem, these two collision algorithms are rarely used. The data in Table
5.6 reveals three undesirable characteristics intrinsic to these two
algorithms that renders them two of the poorer performers in the world of
open addressing collision algorithms. These undesirable characteristics
are:
• Multiple accesses of an element of the primary storage array
(Quadratic algorithm)
304

• Primary clustering (Linear algorithm)
• Secondary clustering (Quadratic algorithm)
The rightmost column of Table 5.6 illustrates the problem of multiple
accesses. Beginning with a home address of 4, and after 18 passes through
the Quadratic collision algorithm, the algorithm has generated the index 4
twice (pass numbers 9 and 18) and the indices 9 and 18 three times each
(pass numbers 2, 10, 15, and 3, 8, and 16, respectively). This is a waste of
eight passes through the algorithm, because if the locations 4, 9, and 18
were unacceptable the first time they were generated, there is no need to
consider them again. Furthermore, indices 1, 7, 10, 13, 14, and 17 have not
been generated.
The Linear Collision algorithm inevitably leads to the problem of
primary clustering. Primary clustering occurs when the nodes mapped into
the same home address by the preprocessing/ hashing algorithms, are
located in a tight cluster near the home address. Consider the insertion of
six nodes, A, B, C, D, E, and F whose collisions will be resolved using the
Linear collision algorithm. Let us assume that A, B, C, D, and E’s home
address is 4, and that F’s home address is 6.
Table 5.6
Indices Generated by the Modified Linear and Quadratic
Collision Algorithms, for an Initial (Home) Address of 4
Pass Number, p
Collision Algorithm
Linear Quadratic
1 5 5
2 6 9
3 7 18
4 8 15
5 9 2
6 10 0
7 11 11
8 12 18
9 13 4
10 14 9
11 15 16
12 16 8
13 17 6
14 18 12
305

15 0 9
16 1 18
17 2 3
18 3 4
Furthermore we will assume that the nodes are inserted in the order
A, B, C, D, E, and finally F. If the structure was empty to begin with, A
would be inserted without a collision at location 4. Referring to the middle
column of Table 5.6 , B would be inserted (after one pass through the
collision algorithm) at location 5; C (after two passes) would be inserted at
location 6, etc. Figure 5.21 depicts the structure after nodes A through E
have been inserted.
The grouping of nodes A through E near their home location (4) is a
primary cluster. There are two problems with primary clustering. Not only
do they slow down operations on the nodes whose home address is the
home address of the cluster (e.g., nodes B, C, D, and E), but they also slow
down operations on nodes whose home address is anywhere within the
cluster. For example, in the absence of the primary cluster, F (whose home
address is 6) would be inserted without a collision. However, because of
the primary cluster, three passes through the collision algorithm are
required to insert F; location 7 is tried first, then location 8, and finally
location 9. If nodes B, C, D, and E were scattered about primary storage, F
would probably be inserted at location 61 without a collision.
Figure 5.21 A Primary Cluster Produced by the Linear Collision
Algorithm after Inserting Five Nodes with the Same Home Address (4)
306

Secondary clustering occurs when nodes with the same home address,
although scattered throughout the primary storage area, generate the same
sequence of collision addresses. The Quadratic collision algorithm
generates secondary clusters. Consider the insertion of five nodes, A, B, C,
D, and E. We will assume that the nodes are inserted in the order A, B, C,
D, and E; that the Quadratic collision algorithm is used to resolve
collisions; and that the home address of all five nodes is 4. If the structure
was empty to begin with, A would be inserted without a collision at
location 4. As shown in the rightmost column of Table 5.6 , B would insert
(after pass 1 through the collision algorithm) at location 5, C (after pass 2)
would insert at location 9, etc., resulting in the situation depicted in Figure
5.22 . Because all of these nodes follow the same collision path, clustering
has occurred. But since they are scattered about the primary storage area
array, the cluster is referred to as a secondary cluster (rather than a primary
cluster). Secondary clusters only suffer from one of the primary cluster
problems: they slow down the operations on nodes whose home address is
the home address of the cluster (e.g., 4 in our example).
Figure 5.22 A Secondary Cluster Produced by the Quadratic
307

Collision Algorithm after Inserting Five Nodes with the Same Home
Address (4)
The way to reduce clustering is to make the collision algorithm
dependent on the key (or pseudo key). Then, nodes with different keys that
hash into the same home address will follow different collision paths.
Using this technique, many open addressing collision algorithms greatly
reduce the probability of the occurrence of primary and secondary
clustering. One of these is the Linear Quotient collision algorithm, which
also eliminates the problem of multiple accesses into the primary storage
area.
The Linear Quotient Collision Algorithm
The Linear Quotient (or LQ) collision algorithm, like the Linear and
Quadratic collision algorithms, uses modulo N arithmetic to keep the
calculated indices within the bounds of the primary storage area array. In
addition, like these algorithms, it computes the next location by adding an
offset to the previously computed location. Unlike these algorithms,
however, the offset is a function of the key (or pseudo key) which tends to
minimize secondary clustering.
As the name of the algorithm suggests, the offset is a division
quotient and the functional relationship involving this quotient is a linear
one. Specifically, the quotient is the result of dividing the key (or pseudo
key) by N , with one exception. When the quotient is evenly divisible by N
, it is replaced with a somewhat arbitrarily chosen 4k + 3 prime.
Referring to the quotient as q , the algorithm is stated as
Linear Quotient Collision Algorithm
where:
q = pk / N ,
N is the number of elements in the primary storage area (a 4 k + 3
prime),
pk is the value of the integer pseudo key (or key when no
308

preprocessing is performed), and
p 4 k +3 is any 4 k + 3 prime other than N .
The unusual part of this algorithm involves the quotient, q . Its initial
computation is straightforward (pk / N ). However, if the remainder of the
division of q by N is zero, then a default 4k + 3 prime (other than N ) is
used in the collision algorithm. Once picked, the default value remains
constant for the entire life of the data structure. To see why the default
prime is a necessary part of the algorithm, we rewrite the first part of the
algorithm as
Since i p is always calculated using modulo N arithmetic, i p is always
in the range 0 to N − 1 and therefore i p % N = i p . Therefore, the previous
equation becomes
This means that when q % N = 0 the above equation degenerates to i p
= i p and all passes through the collision algorithm produce the same index
(the home address).
For example, let us assume N is 19, and a quotient of 57 which is
evenly divisible by 19. Furthermore, we will assume a home index of 6,
which resulted in a collision. Then, if q were used in the collision
algorithm, the next address calculated would also be 6:
However, using a default prime, p 4 k +3 , in the collision algorithm,
rather than the quotient 57, guarantees a nonzero offset since prime
numbers are not evenly divisible by any number other than themselves.
For example, if the default prime chosen for our data structure was 23, the
next index calculated by the collision algorithm would 10:
Table 5.7 further illustrates the use of the LQ collision algorithm. It is
a tabulation of the collision algorithm pass number vs. the indices
calculated during the first 18 passes through the algorithm for 3 pseudo
keys. N was assumed to be 19 and the default prime, p 4 k +3 , was
assumed to be 23. The values of the 3 pseudo keys are 593, 5058, and 251
for the indices tabulated in columns 2 (Case 1), 3 (Case 2), and 4 (Case 3)
respectively. All 3 keys map into the same home address, 4, as calculated
309

by the Division Hashing algorithm (593 % 19 = 5058 % 19 = 251 % 19 =
4).
To begin the collision algorithm we calculate the quotient, q . As
shown in the headings of the table, the quotients are 31 (= 593 / 19), 266
(= 5058 / 19), and 13 (= 251 / 19). Next we calculate q % N for each of the
keys to determine the offset to be used in the collision algorithm. If it
evaluates to zero, q is evenly divisible by N and the default prime will be
used as the offset. Otherwise, q will be used as the offset. The values of q
% 19 and the resulting offset are shown in the column headings of the
table. The key value 5058 uses the default prime as an offset (266 % 19 =
0). The resulting three equations for the primary storage index i p is also
present in the column headings of the table. These equations were used to
generate the indices presented in the table.
It is a useful exercise for the student to calculate the home address
(4), the offsets (31, 23, and 31), and the first two or three indices presented
in the table for each of the three pseudokeys. (Note: the Division Hashing
algorithm is used to calculate the home address.)
Having gained an understanding of the calculation processes of the
Linear Quotient collision algorithm, we will now examine it from the
viewpoint of multiple accesses and clustering, beginning with the problem
of multiple accesses.
Multiple accesses occur when the collision algorithm produces an
index already visited on a previous pass (probe). Inspecting the indices
presented in the Table 5.7 , we see that for all three cases the indices
between 0 and 18 (with the exception of index 4 which is the previously
visited home address) are generated once, and only once. The data
presented in the table is typical of the performance of this algorithm; it has
the remarkable (and very desirable) feature of never recalculating a
location in the primary storage area, for all possible values of the offset.
Therefore, the algorithm does not produce multiple accesses.
310

Now let us turn our attention to the problem of primary clustering.
Primary clustering occurs whenever the collision path generates a set of
sequential indices. For the LQ collision algorithm, this will occur when the
term q % N , or p 4 k +3 % N , equals 1. Since there are N − 1 possible
values for these terms (the range of the remainders when dividing by N ),
the probability of them evaluating to 1 is 1 / (N − 1). As N gets large, 1 /
(N − 1) approaches zero and, therefore, the LQ hashing has a low
probability of primary clustering.
Finally, let us consider the problem of secondary clustering.
Secondary clustering occurs when two keys with the same home address
follow the same nonsequential collision path. The home address for all
311

three pseudo keys presented in Table 5.7 is the same, 4. Yet, when we
examine the indices presented in the table, we see that all three pseudo
keys follow a different collision path. If this is true for all keys that map
into the same home address, then this algorithm does not exhibit secondary
clustering.
Consider the key 954. For N = 19, its quotient is 50 and its home
address (4) is the same as the keys presented in Table 5.7 . 9 Since 50 %
19 is not 0, the index equation in the collision algorithm is
During the first pass through the collision algorithm this equation
yields an index of 16 (i.e., (4 + 50) % 19 = 16). The second pass produces
an index of 9 (i.e., (16 + 50) % 19). Unfortunately, these and all
subsequent passes yield the same collision path as pseudo key 593 (see
Table 5.7 ) and the algorithm, under these conditions, exhibits secondary
clustering. Clustering occurs because the offset 50 (for key 954) when
divided by 19 yields the same remainder, 12, as when the offset 31 (for
key 593) is divided by 19. Since both calculations start from the same
home address, 4, all calculated indices will be the same.
Therefore, whenever two pseudo keys map into the same home
address, and their offsets divided by N yield the same remainder, the
algorithm exhibits secondary clustering. There are N − 1 choices for this
remainder (1 to N − 1), and when we use the optimum loading factor to
size the array, three-fourths of the home addresses will be occupied. Thus,
the probability of secondary clustering occurring for any two keys is ≤ [¾
* 1 / (N − 1)]. As N gets large, 1 / (N − 1) approaches zero and, therefore,
the LQ collision algorithm has a low probability of secondary clustering.
The low probability of clustering, the ability to generate unique
indices each pass through the algorithm, and its simplicity make the Linear
Quotient algorithm an excellent candidate for resolving collisions in
hashed data structures. Before demonstrating its use in an implementation
of a hashed data structure, we will examine a problem that all open
addressing collision algorithms demonstrate: the Delete problem.
312

Figure 5.23 A Hashed Structure After Nodes A and B are Inserted, B
Colliding with A
The Delete Problem
To illustrate this problem, let us assume that we try to fetch node B
from an empty data structure, and its home index, as determined by the
preprocessing and hashing algorithms, is 4. Finding a null reference stored
in that element of the array, we would correctly conclude that B is not in
the structure. Now suppose that nodes A and B are inserted into the empty
structure in the order, first A, and then B. Let us also assume that they both
have the same home address, 4, and that the collision algorithm placed B
at location 9 (see Figure 5.23 ).
Now consider what happens when we try to fetch node B from the
structure. The first place we look is its home address, 4. Finding node A
there, we cannot conclude that node B is not in the data structure since,
when it was inserted, it could have (and in fact did) collide with node A.
Therefore, we use the collision algorithm to probe further into the primary
storage area. On the first pass through the collision algorithm we find it, or
313

if it is not in the structure, we encounter a null reference.
Figure 5.24 The Data Structure Shown in Figure 5.38 after Node A
has been Deleted
Now suppose that node A is deleted and that the reference to it, at
index 4, is set to null . A subsequent searched for node B uncovers a null
reference at its home address, index 4, and we would conclude
(erroneously) that node B is not in the data structure. This example
illustrates the Delete problem which occurs when a node on a collision
path is deleted (i.e., A) and then an operation is attempted on a node
further down the collision path (i.e., B). By setting the deleted node’s
reference in the primary storage area to null , we have effectively lost the
record that the other nodes have collided with it.
The usual solution to this problem is to set the deleted node’s
reference in the primary storage area array to a value other than its initial
null value. Referring to the initial value (null ) as v 1 and the other value
as v 2 , v 1 indicates there was never a node reference at that location, while
v 2 indicates that there was once a node there, but it has been deleted. Both
314

v 1 and v 2 must be chosen to be nonlegitimate node references. Usually v 1
is chosen to be null , and v 2 is chosen to be a pointer to a dummy node.
Assuming values of null and 9999 for v 1 and v 2 respectively, Figure 5.24
shows the data structure after A has been deleted. Now, when we try to
locate node B, the home address contains a pointer to the dummy node
indicating that there once was a node at this location which B could have
collided with. Therefore, we must continue the search for B by invoking
the collision algorithm. In this case, on the first pass through the collision
algorithm, B is located.
Suppose we tried to locate some other node, C, which is not in the
structure but whose home address is also 4. Finding a reference to the
dummy node in element 4, we would invoke the collision algorithm
repeatedly until a null reference was encountered which would terminate
the search. The solution to the Delete problem for all four basic operations
is summarized in Figure 5.25 .
The Delete Problem Solution
• Initialize the primary storage area to a value, v 1 (usually null ).
• The Fetch and Delete operations continue searching primary storage
until the node is found, or the element of primary storage contains v 1 .
• When deleting a node, its reference is set to v 2 (usually a reference to a
dummy node).
• The Insert operation continues searching primary storage until a value
of v 1 or v 2 is found.
Having discussed all aspects of the hashing process depicted in Figure
5.5 , and the preprocessing, hashing, and collision algorithms, we are now
in a position to implement a hashed structure.
5.4.6 The Linear Quotient (LQHashed) Data Structure
Implementation
In this section we will implement a hashed data structure LQHashed.
The keys will be assumed to be strings of any length, therefore
preprocessing will be necessary to map then into numeric pseudo keys.
The Fold-shifting algorithm, coded as the method stringToInt (see Figure
5.18 ), will be the preprocessing algorithm.
The client will specify the maximum number of nodes to be stored in
the structure. This value, adjusted by the optimum loading factor (0.75),
315

will be used by the method fourKPlus3 (see Figure 5.16 ) to determine the
size the primary storage area array, N . Since the preprocessing algorithm
returns the absolute value of a 32 bit signed integer, the range of the
pseudo keys will be 0 to 2,147,483,648. To avoid indexing beyond the
bounds of the primary storage area array, the Division Hashing algorithm
will be used to map the pseudo keys into indices. The value of v 1 will be
null , and v 2 will be the address of a dummy node. As previously stated,
collisions will be resolved with the Linear Quotient collision algorithm.
The default quotient will be the (arbitrarily chosen) k 4 + 3 prime, 9967.
As usual, the update algorithm will be implemented as an invocation
to the delete method, followed by an invocation to the insert method. The
flowcharts for the other three operation algorithms, Insert, Fetch, and
Delete (which are adaptations of the algorithm depicted in Figure 5.12 )
are given in Figures 5.25 , 5.26 , and 5.27 respectively. The following
notation is used in these figures:
316

Figure 5.25 The Insert Algorithm of the LQHashed Structure
k is the given key; pk is the numeric pseudo key;
data is the name of the primary storage area array;
ip is used as an index into the primary storage area;
pass is a count of the number of times the collision algorithm (loop) executes;
v 1 is the value initially stored in the primary storage array elements ( null ); and
v 2 is the reference to a dummy node.
317

Figure 5.26 The Fetch Algorithm of the LQHashed Structure
318

Figure 5.27 The Delete Algorithm of the LQHashed Structure
Our implementation will be a fully encapsulated homogeneous
implementation of a hashed data structure that stores nodes in the class
Listing. The code, which is presented in Figure 5.28 , is consistent with the
many of the concepts of generics presented in Chapter 2 . It does not
mention the names of any of the fields of the nodes, and the definition of
the nodes to be stored in structure is coded as a separate class (see Figure
319

2.16 ). This class provides a deepCopy method in order to encapsulate the
structure, a method compareTo to determine if a given key is equal to the
key of of a node in the structure, and a method toString to return the
contents of a node. Since this is the first structure we have studied whose
Insert algorithm needs access to the key field of the node being inserted (in
order to preprocess and hash the key), a getKey method will be added to
the class Listing. The code of the method follows:
320

321

322

Figure 5.28 Implementation of the Hashed Data Structure LQHashed
The implementation is not fully generic in that the node class must be
named Listing and the key field must be a String. A fully generic
implementation of the structure, using the generic features of Java 5.0 and
the techniques described in Chapters 2 and 3 (Section 2.5 and 3.4 ) will be
left as an exercise for the student.
Lines 121–149 is the method that generates the 4k + 3 prime
(previously presented in Figure 5.16 ) used to size the primary storage
area. Lines 150–169 is the preprocessing method to convert string keys, of
any length, into numeric pseudo keys (previously presented in Figure 5.18
).
323

The default prime is set to the 4k + 3 prime 9967 on Line 4, and the
loading factor is set to the optimum value of 0.75 on Line 5. This will
minimize collisions while maintaining an acceptable density. Lines 9–16,
the class’ one parameter constructor, allocates the primary storage area
array (Line 12), and allocates a dummy node (Line 13) that will be used
for the value of v 2 (i.e., to indicate that a node has been deleted). The
client will use the method’s parameter (length) to specify the maximum
number of nodes to be stored in the structure. The size of the primary
storage area (a 4k + 3 prime) is calculated on Line 11. Finally, on Lines 14
and 15, the constructor initializes all elements of primary storage to v 1
(chosen to be null ). 10
The insert method on Lines 18–48, the fetch method on Lines 50–75,
and the delete method on Lines 77–105 are the Java equivalent of the
algorithms presented in Figures 5.25 , 5.26 , and 5.27 , respectively, except
for the code added to the insert and delete methods to keep track of the
current loading factor and to not allow it to exceed the optimum value
(0.75). Each of these methods uses the method stringToInt to preprocess
String keys into numeric pseudo keys (Lines 22, 54, and 81). As
previously mentioned, they presuppose that the class Listing, presented in
Figure 2.16 , has been expanded to include a method getKey that returns
the key field of a Listing object.
Line 23 of the insert method will only allow an Insert operation to be
performed if the current loading factor (n / N ) is below the optimum
loading factor. The while loop (Lines 30–37) performs a search for an
unused location in the primary storage area: a null reference or a reference
to the dummy node (Line 31). When found, the variable hit is set to true
(Line 32), and a deepCopy of the node is inserted into the structure (Line
39). Then, Line 40 increases the number of nodes in the structure by 1, to
keep track of the current loading factor.
Line 72 of the fetch method returns a deepCopy of the requested
node, thus maintaining the structure’s encapsulation. When a node is
deleted, Line 99 of the delete method resets the reference to it in the
primary storage area to the dummy node. References to this node are
considered unused by the insert method (Line 31) but do not cause the
search loops of the fetch and delete methods to terminate (Lines 62 and
89). When a node is deleted, the number of nodes in the structure, n, is
decremented by the delete method (Line 100) to keep track of the current
loading factor. Finally, the Boolean expression on the right side of Line
324

117 prevents the showAll method from outputting the dummy node.
Performance of the LQHashed Structure
The speed and density of the LQHashed data structure will now be
examined to determine its performance.
Speed
To analyze the speed of the structure, we will again perform a Big-O
analysis to determine the approximate number of memory access
instructions executed as n , the number of nodes stored in the structure,
gets large. We will ignore instructions in loops that repeatedly access the
same memory cell because optimizing compilers store these values in CPU
registers. Each of the algorithms presented in Figures 5.25 -5.27 performs
preprocessing on non-numeric keys. During the preprocessing each
character of the key has to be fetched from memory to be fold-shifted into
the pseudo key. Assuming there are m characters in the key, m memory
accesses would be performed during the preprocessing. In addition, the
calculation of the quotient requires one memory access to fetch the value
of N and an optimizing compiler would fetch the reference to the dummy
node, v 2 prior to entering the loop. This would require one memory
access. Thus, prior to entering the collision loop, m + 2 memory accesses
are performed.
The remaining term in the speed equations is contributed by the
collision loop in the Insert, Fetch, and Delete algorithms (bottom of
Figures 5.25 -5.27 ). Since the structure’s maximum loading factor is the
optimum loading factor (0.75), Figure 5.14 indicates we can expect
average search lengths (resulting from repeated collisions) to be
approximately 4.
In fact, the search length of this structure is even better than that
illustrated in Figure 5.14 since the formula the figure is based upon, as
stated in Appendix B , assumes that “All N primary storage area locations
are equally probable to be generated each pass through the collision
algorithm.” This is not the case for the Linear Quotient collision algorithm.
As we have seen, once a location is generated by the algorithm it is not
generated again (see Table 5.7 ). When all locations have an equal
probability of being generated (as Appendix B assumes), the probability of
a collision on the i th pass through the collision algorithm is n / N .
However, if we do not revisit locations as we proceed down a collision
325

path, the probability of a collision on the i th pass through the collision
algorithm is (n − i ) / (N − i ), since i locations have been eliminated from
consideration. Since (n − i ) / (N − i ) is less than n / N for all positive
values of i 11 the collision probability of the LQ algorithm decreases with
each probe, resulting in slightly shorter search lengths.
Returning to our speed calculation, the Insert algorithm performs one
memory access every pass through the loop to fetch the reference stored at
data[ip]. Thus, the loop’s speed term (in the speed equation) is four, 12
which, when combined with the memory access performed prior to
entering the loop, (m + 2) gives a speed equation of m + 6.
Both the Fetch and the Delete algorithms perform two memory
accesses every pass through the loop. One to fetch data[ip], and one to
fetch the key of the node referenced by data[ip]. Thus, the loop’s speed
term (in these operations’ speed equation) is eight, 13 which, when
combined with the memory access performed prior to entering the loop, (m
+ 2) gives a speed equation of m + 10.
Before ending our discussion of the speed of this structure, it should
be mentioned that there is one situation that causes the speed of the Fetch
and Delete operations to fall off sharply: when a significant portion of
unused primary storage contains the value v 2 and the node to be fetched or
deleted is not in the structure. To see why this happens, we must remember
that the Fetch and Delete algorithms can only end on one of three
conditions (see Figures 5.26 and 5.27 ):
1. The node to be operated on is located.
2. A value of v 1 (null ) is encountered.
3. The collision loop executes N − 1 times.
Since the node to be operated on is not in the structure, condition 1
cannot end the algorithms. In addition, when the traffic on the structure is
such that nodes are often deleted and others inserted, there may be no null
values in the primary storage array. The deletions have overwritten them
with the reference to the dummy node, v 2 . Therefore, condition 2 cannot
end the algorithms and the only way they can terminate is on condition 3.
In this case, the collision loop executes N − 1 times performing (N − 1) * 2
time-consuming memory accesses. This is as slow as a sequential search.
One remedy for this situation is for the Fetch and Delete algorithms to
temporarily suspend operations on the data structure when the number of v
326

2 primary storage references becomes excessive. Then, all the nodes are
inserted into a newly declared array, except for the references to the
dummy node. The array reference data is set to reference the new array.
Finally, the count of the v 2 references can be set to zero because the null
references in the newly created array were not overwritten with the
references to the dummy node. The code to accomplish this, which
assumes the new array is named temp, is given below.
To keep track of the number of the number of v 2 references in the
data structure, a counter is incremented whenever a node is deleted from
the structure, and the counter is decremented whenever a node is inserted
into an element previously written to v 2 . Then, when the sum of this
counter and n (the number of nodes in the structure) produce a loading
factor above 0.75, operations on the structure are temporarily suspended
and the previously discussed remedy is performed.
Figure 5.29 Density of the LQHashed Structure for Various Node
Widths
Density
327

Our LQHashed structure uses the optimum loading factor (0.75) to
size the primary storage area array which, as we have discussed, strikes a
balance between good speed and space complexity. To demonstrate this, in
Section 5.4 of this chapter, we derived a formula for the density of a
hashed data structure in terms of loading factor:
where:
w is the node width, in bytes,
N is the size of the primary storage area array, and
l is the loading factor.
Substituting our structure’s loading factor (0.75) into this equation, its
density DLQ becomes
Figure 5.15 , which is a plot of this function, demonstrates that the
density of the LQHashed structure, like all hashed data structures with
loading factors of 0.75, is good (> 0.80) for node widths of 23 bytes or
more. Figure 5.29 , which presents the density of a hashed structure for
various loading factors and node widths, shows that only minimal
improvements in density can be made by increasing the loading factor
above 0.75.
Table 5.8 presents the overall performance of this structure, along
with the performance of the structures we have previously studied for
comparative purposes. Its speed is not quite as good as that of the Direct
Hashed data structure whose search length is always 1. However, its
ability to deliver densities of 80% for all applications (whose node widths
are above 23 bytes), and its ability to process String keys of any length
make it a more than acceptable alternative when the density of the Direct
Hashed structure is unacceptable, or when the key field is a long string.
328

For example, in our stadium ticket case study in which the seat
number, date, and event number is encoded into a six digit ticket number,
the loading factor of the DirectHashed structure was shown to be 0.01
(Section 5.2.1 ) and, therefore, the above equation for density yields D =
0.074 (the node with for this application was 32 bytes). In most cases, this
would be a prohibitively low density. In contrast, since the node width is
greater than 23 bytes, the LQHashed structure has a density greater than
0.80. From a speed complexity viewpoint, both structures are O(1).
Generic Implementation Considerations
Although the implementation of the LQHashed structure presented in
Figure 5.28 is not fully generic (it can only store objects in the class
Listing), it is consistent with many of the concepts of generics presented in
Chapters 2 and 3 . It does not mention the names of any of the fields of the
nodes, and the definition of the nodes to be stored in the structure is
defined in a separate class (Figure 2.16 was expanded to include a getKey
method). To convert it to a full generic class, the techniques discussed in
Chapters 2 and 3 (Sections 2.5 and 3.4 ) that make use of the generic
features of Java 5.0 can be used. However, these techniques will have to be
expanded slightly if the generic implementation of this structure is to allow
keys that are not String objects because the preprocessing method
stringToInt (in the data structure class) assumes that the key field is a
329

String object. If the key is other than a String object, the method would
have to be rewritten to produce integer pseudo keys for that type of object.
There are two approaches that could be taken to accomplish this
rewrite. One approach would be to define a new data structure class that
extends the class LQHashed (using OOP inheritance techniques, see
Section 1.6 ) and overwrite the stringToInt method (which would now be
improperly named, but we’ll deal with that in a moment). Another
approach would be to remove the stringToInt method from the class
LQHashed and require that it be defined as a method in the class of the
key. In this case, its signature is usually changed so that it no longer
operates on an argument sent to it, but on the object that invoked it. For
example, if the key were an object in the class CarType, and the name of
the method was a generic name like hashCode, then the signature of the
method would be
and the method would be coded in the class CarType. The invocation
in the data structure class to determine the pseudo key, pk, (e.g., Line 22 of
Figure 5.28 ) for the CarType key convertible would be changed to:
As we will see later in this chapter, this is the approach taken in the
implementations of the generic hashed data structures included in the Java
Application Programmer Interface. In fact, many of the classes included in
the Java API (e.g., Integer, Double, and String) include an implementation
of a method named hashCode, whose signature is public int hashCode().
Therefore, if the invocations of the method stringToInt in Figure 5.28 are
changed to invocations of the method hashCode, the structure LQHashed
could store nodes whose keys were objects in the class Double, Integer, or
String. For example, Line 22 of Figure 5.28 would be changed to
5.4.7 Dynamic Hashed Structures
The hashed structures studied so far require the client to specify the
maximum number of nodes to be stored in the structure when it is created.
The primary storage area array is sized using this value and an attempt to
insert more than the specified number of nodes into the structure results in
an insert error. A more flexible alternative is to allow the data structures to
dynamically expand, while the data structure is in service. The number of
nodes specified by the client is typically used to establish the initial size of
the structure, but the structure will be allowed to grow beyond its initial
330

size to accommodate additional nodes. This approach can make the speed
of hashed data structures available to applications where the maximum
number of nodes cannot be accurately predicted.
Dynamic expansion of the data structure can be accomplished in a
variety of ways. The size of the primary storage area array can be
increased to accommodate additional node references, or the additional
storage can be provided outside of the primary storage area. Hybrid
structures combine both approaches. These various schemes are illustrated
in Figure 5.30 . Two popular approaches to providing additional storage
outside of the primary storage area is to provide either a linked list or an
array to store the additional node references. Whichever approach is taken,
the ability of these structures to accommodate virtually an unlimited
number of nodes and still provide the speed of a hashed data structure,
makes them very attractive for many applications.
Figure 5.30 Three Approaches to Dynamic Hashed Structures
We will conclude our study of hashed data structures by briefly
examining two dynamic hashed data structures. The first one, which is
implemented in a Java Application Programmer Interface package, will be
examined from the client’s viewpoint. The other structure will be examined
at a lower level, to gain insights into the implementation of dynamic
structures.
Java’s Dynamic Hashtable Data Structure
The Java class Hashtable, contained in the package java.util, is an
implementation of a dynamic hashed data structure. As such, it expands
beyond its initial size to accommodate virtually an unlimited number of
nodes. As implemented, it is a nonhomogeneous, unencapsulated generic
331

data structure accessed in the key field mode. The key can be any type of
object.
The structure is a dynamic hybrid; it expands its primary storage area
and resolves collisions by providing additional storage for node references
outside of the primary storage area. When the number of references stored
in the primary storage area array is such that the loading factor exceeds a
specified maximum, the size of the primary storage area is increased.
Although keys can be any type of object, as previously discussed, the key’s
class should overwrite the method hashCode. This method is invoked by
the operation methods in the Hashtable class to preprocess a key into an
integer. The key’s class should also overwrite the method equals. The
signature of both of these methods is given in the description of the class
Object to be:
The classes String, and the wrapper classes for the numeric types (i.e.,
Byte, Double, Float, Integer, Long, and Short) all overwrite these methods.
Therefore, keys that are instances of these classes can be properly
processed into pseudo keys. 15
The class Hashtable has four constructors. The default constructor
sets the maximum loading factor to 0.75 and the initial size of the primary
storage area to 101 elements. The one parameter constructor also defaults
the maximum loading factor to 0.75, but allows the client to specify the
initial size of the primary storage area. The client can specify both the
maximum loading factor and the initial size of the primary storage area
using the class’ two parameter constructor. A fourth constructor initializes
the structure to a copy of an existing Hashtable structure by copying the
references in the primary storage area array of the existing structure into
the newly created structure.
The Insert, Fetch, and Delete operation methods are named put, get,
and remove, respectively. The methods get and remove return null if the
specified key is not in the structure. The structure recalculates the loading
factor every time an insert is performed. When the loading factor exceeds
the specified maximum, an internal reorganization takes place. A
protected method named rehash is invoked to expand the primary storage
area, and all the nodes are reinserted into the expanded structure. This
process reduces the speed of the structure and so it is advisable to specify
the initial size of the structure in a way that minimizes the number of times
the structure is reorganized. Descriptions of the other methods in the class
332

are given in the Java online documentation
(http://java.sun.com/j2se/1.3/docs/api/java/util/Hashtable.html ).
An application that uses the Hashtable data structure, and the output it
produces, is given in Figure 5.31 . On Line 3, the application uses the Java
5.0 generic type parameters (i.e., ) to declare the structure
to be a homogeneous structure that can store only Listing objects whose
key field is a String object. The code of the class Listing is presented in
Figure 2.16 .
The lack of the structure’s encapsulation is demonstrated on Lines
21–24. If the structure were encapsulated, Line 23 would have output Bill’s
address that was updated to “99th Street” on Line 18. Instead it outputs the
address “18 Park Avenue” that the client set using the setAddress method.
If the generic type parameters were not used in the declaration of the
object dataBase on Line 4, then the structure would be heterogeneous and
any type node with any type key could be stored in the data structure. In
this case, the reference returned from a get operation would have to be
coerced before they were assigned to reference variables; e.g., b =
(Listing) dataBase.get(“Bill”);.
Linked Hashed Data Structures
A common design of a dynamic hashed structure is the Linked
Hashed structure. 16 It does not expand primary storage to accommodate
additional nodes, but rather provides additional storage outside of the
primary storage area in the form of multiple linked lists.
Each of the N locations of the primary storage area array (commonly
called a bucket ) is a header of a singly linked list. All of the nodes that
hash into a given location of primary storage are stored in the linked list
associated with that (home) location. For example, if nodes A, B, and C all
mapped into location 4, and node D mapped into location 10, the nodes
would be stored as shown in Figure 5.32 . 17 Thus, not only are the linked
lists used to dynamically grow the structure, but they are also used to
resolve collisions. Collision resolution is performed through a sequential
search down the linked lists, rather than a search through the primary
storage area.
When a collision occurs during an Insert operation, the length of the
linked list (headed by the key’s home address) is increased by one node.
Normally, for speed considerations the new node is inserted at the
333

http://www.java.sun.com/j2se/1.3/docs/api/java/util/Hashtable.html

beginning of the linked list, and so the elements of primary storage refer to
the most recently inserted node that hashed into that element.
Figure 5.31 An Application Program that Uses the Java Hashtable
Data Structure
334

Figure 5.32 A Dynamic Linked Hashed Structure (with Nodes A, B,
and C Hashing into the Same Location of Primary Storage)
Most implementations of this structure permit the client to specify a
best guess at the maximum number of nodes to be stored in the structure, n
. Then the primary storage area is sized to a percentage of n . The size of
primary storage area, N , does not necessarily have to be greater than n for
this structure since the nodes are stored in the linked lists. In the extreme
case, when N is 1, the structure degenerates into a singly linked list.
The overhead of this structure is the storage associated with the
primary storage area array (N * 4) and the storage associated with the
implementation of the linked list. The storage associated with the linked
list implementation includes two reference variables for each node in the
structure; one variable to store the node’s address and one to store the
address of the next item in the linked list (see Figure 4.9 with the dummy
node eliminated). Assuming reference variables occupy 4 bytes, the
density of this structure, DLH , can be expressed as
335

where:
n is the number of nodes in the structure,
w is the information bytes per node,
N is the size of the primary storage area (= number of linked lists), and
each reference
variable occupies 4 bytes.
Dividing the numerator and denominator of the right side of this
equation by n * w we obtain
Substituting 1 / lp , for N / n the density of the Linked Hashed
structure becomes
where:
l p is a pseudo loading factor n / N .
Examining the fraction on the right side of the equation, we see that
as the node width or the pseudo loading factor increases, the denominator
decreases, and the density improves. This is typical of the hashed
structures studied thus far. What is atypical of this structure, and the reason
we use the term pseudo loading factor, is that the ratio n / N can be
allowed to exceed 1, since we no longer have to provide a location in the
primary storage area array for each node. We only need to provide a
location for each linked list.
The variation in the density with the pseudo loading factor for various
node widths, w, is shown in Figure 5.32 . For a given node width, the
increase in density with loading factor appears to be insignificant beyond a
loading factor of three. Now, assuming the preprocessing and hashing
function distributes the keys’ home addresses evenly over the primary
storage area, the average number of nodes in each of the N linked lists is n
/ N . For example, 30 nodes (n = 30) distributed over 10 lists (N = 10)
results in 3 nodes per list (30 / 10). Since the average number of nodes in
336

the linked lists adversely affects the speed of the structure, and further
increases in the loading factor do little to improve the density, a loading
factor of 3 is a good balance between speed and density for this structure.
At a loading factor of 3, the density is 0.8 for node widths (w) greater than
or equal to 37 bytes.
Now let us examine the speed of the structure in more detail
beginning with the Insert algorithm. In Chapter 4 we determined that an
average of six memory accesses are required to insert a node at the
beginning of a linked list. However, this structure requires one less
memory access because the preprocessing and hashing algorithms
calculate the location of the memory cell that references the new node
rather than using the list header to locate a dummy node. Thus, an Insert
operation is performed with an average of five memory accesses.
The average search length for a Fetch or Delete operation is related to
the average number of nodes in the linked lists. A goal of the
preprocessing and hashing algorithms used in this structure is to keep the
length of the linked lists equal or balanced by randomly mapping the keys
over the primary storage area. In this case, as previously stated, each
linked list will contain n / N nodes since the number of linked lists is the
size of the primary storage area, N .
Assuming an average node would be halfway down one of the linked
lists, (n / N ) / 2 nodes would be traversed to locate it. As discussed in
Chapter 4 , traversing a linked list requires two memory accesses per node
traversed, therefore the average number of memory accesses required to
locate a node would be
Since n /N is, by definition, the loading factor, the average number of
memory accesses during a Fetch or Delete operation is equal to the loading
factor. Therefore, if the linked lists can be kept balanced, the x -axis in
Figure 5.33 is also the average number of memory accesses (speed) of the
Fetch-and-Delete operations, not considering the memory accesses
required to preprocess nonnumeric keys. When we consider the number of
preprocessing memory accesses, m , and the fact that little gain in density
is realized if we increase the loading factor beyond a value of 3, the
number of memory access for a Fetch or Delete operation becomes m + n
/N = m + 3.
Table 5.9 presents the overall performance of this structure, compared
to structures previously studied. Not only is its speed and performance
337

excellent (assuming the linked lists remain balanced), but its dynamic
characteristic make it an excellent choice for many applications.
An interesting variation of the Linked Hashed structure is a hybrid
structure, expandable in two directions. In addition to the size of a linked
list increasing every time a collision occurs, the primary storage area is
also expandable. When the average length of the linked lists grows to a
point where the speed of the structure is unacceptable, the primary storage
area is expanded and the nodes are reinserted into the structure.
Figure 5.33 Variation of Density with Loading Factor for a Linked
Hashed Structure
338

EXERCISES
Knowledge Exercises
1. Give the major advantage of any hashed data structure over array-
based and linked list structures?
2. What is the potential downside to hashed data structures compared to
array-based and linked list structures?
3. What characteristic common to all hashing access algorithms makes
them fast in the key field mode?
4. For a hashed data structure implemented in Java, give the two
alternatives for what is is stored in the primary storage area array?
5. Define loading factor.
6. Define the term collision in the context of data structures.
7. True or false, collisions improve the performance of a hashed data
structure?
339

8. Keys are numeric values between 0 and 10,000. Assuming 200 nodes
are to be stored in a hashed data structure that uses the direct hashing
function, how many elements will be in the primary storage area array?
9. Assuming the Direct Hashing function and the Subtraction
preprocessing algorithm is used to map keys into indices, give the index
it maps the key 2000 into, assuming:
a) The range of the keys is 0 to 999,999.
b) The range of the keys is 100 to 999,999.
10. Give the Division Hashing function and the index it maps the key
2000 into, assuming a primary storage area array size of 61 elements
and:
a) The range of the keys is 0 to 999,999.
b) The range of the keys is 100 to 999,999.
11. Nodes are to be stored in a hashed data structure that utilizes the
direct hashing function. Assuming the key field was an integer ranging
from 2000 to 100,000 and the structure will store a maximum of
60,000 nodes:
a) Compute the loading factor of the structure.
b) Compute the density of the structure assuming a node width of 100
bytes.
c) Give the node width that results in a density of 0.7.
12. A key maps into index 200 of a Direct Hashed data structure’s
primary storage array. A Fetch operation is to be performed. How will
we tell if the node is not in the structure?
13. What would be the density and maximum loading factor of the
structure described in Exercise 11 if 9000 nodes were to be stored in
the structure?
14. Which data structure is faster: a perfect or a nonperfect hashed
structure?
15. Define search length.
16. What is the average search length of a perfect hashed data
structure?
17. Considering density and speed, what is the optimum loading factor
for a hashed data structure that uses a nonperfect hashing function, and
what is the average search length at that loading factor?
340

18. Which of the following integers are 4k + 3 primes?
a) 4726
b) 2003
c) 9109
19. Pseudo keys are in the range 0 to 200,000 and a maximum of 9000
nodes will be stored in a nonperfect hashed data structure. Give the size
of the primary storage area array (number of elements) that will
produce the optimum loading factor.
20. Give the density of the structure described in Exercise 19 assuming
the node width is:
a) 10 bytes
b) 200 bytes
21. Assuming keys are comprised of upper- and lowercase letters, give
the bit pattern and base 10 numeric value of the pseudo key that would
result from processing the key “Mary” using the algorithm illustrated
in Figure 5.17 .
22. What would be the base 10 value of the pseudo key produced by
fold-shifting the key “Bob-Jones” to produce a 16-bit numeric pseudo
key. Use the characters “bJ” as a pivot.
23. A pseudorandom preprocessing scheme is being used in a hashed
data structure with the primes p 1 = 11 and p 2 = 5. Give the pseudo
keys for the following keys:
a) 198
b) 24
24. State the difference between an open addressing and a non-open
addressing collision algorithm.
25. The primary storage area in a nonperfect hashed data structure is a
103 element array. A key has been mapped into index 102, and a
collision has occurred. Give the indices calculated by the next three
passes through the collision algorithm if the collision algorithm is the
modified version of the:
a) Linear Probing
b) Quadratic Probing
26. Define the terms:
a) Primary clustering
341

b) Secondary clustering
27. Why is the array used for a restricted hashing scheme always sized
to a 4k + 3 prime?
28. Give the Linear Quotient collision algorithm and describe how it
gets its name?
29. Describe the “delete problem” associated with nonperfect hashed
data structures and state the standard remedy.
30. Give the density of a hashed structure with a loading factor of 0.6
assuming each node contains 30 information bytes.
31. Objects to be stored in a data structure each have 1000 bytes of
information, which includes the key field comprised of 5 digits. A
maximum of 1500 nodes will be in the structure at one time. How
many elements will be in the primary storage area array, if the nodes
are stored using a:
a) Direct hashing function?
b) Division hashing function?
32. Give the density of the two structures described in Exercise 31.
33. A maximum of 300 nodes are to be stored in a hashed data
structure. Give the size of the primary storage area that would
maximize the performance of the structure.
34. A 23 element array has been allocated to store nodes using the
LQHashed data structure discussed in this chapter. Give the array index
used to store the nodes with the following keys, assuming they are
inserted in the order given:
a) 4618
b) 391
c) 6941
d) 547
Note: Use the 4k + 3 prime 19 to resolve any problems with the
quotients.
35. Give the advantage of dynamic hashed structures over nondynamic
hashed structures.
36. Give an advantage and a disadvantage of the array-based approach
to a dynamic hashed structure over the linked approach.
37. In the context of dynamic hashed structures, what is a bucket?
342

38. Assuming 30,000 nodes are equally distributed over a dynamic
linked hashed structure, give the size of the primary storage array for
optimum performance.
39. Java contains a class that implements a dynamic hashed data
structure.
a) Give the name of the class.
b) Give the import statement necessary to use the class.
c) How does it resolve collisions?
d) Is the structure a fully encapsulated structure?
Programming Exercises
40. Implement the perfect hashed data structure discussed in this
chapter and provide an application that demonstrates that its four basic
operation methods function properly. Your application should store
nodes for a stadium ticket application where the ticket numbers range
from 2000 to 100,000 for a 60,000 seat stadium. The ticket number will
be the key field and the nodes will also store the purchaser’s name. See
Figure 5.4 for a description of the field widths.
41. Code a method that outputs the prime numbers between any two
given integers. Supply a driver program that allows the user to input
the range of the prime numbers.
42. Code a method that outputs the density of a Direct Hashed data
structure given the number of nodes in the structure, the node width in
bytes, and the number of elements in the primary storage area.
43. Code a method that maps an automobile license plate number into
a unique integer. License plates can consist of any permutation of up to
four capital letters and the digits 0 through 9. Supply a driver program
that demonstrates that the method functions properly.
44. Write a program that demonstrates that the Linear Quotient
collision algorithm generates every primary storage array index before
repeating one. The size of the primary storage area and the initial home
address will be input by the user. Supply a driver program that
demonstrates that the method functions properly.
45. A database is to be developed to keep track of student information
at your college: their names, identification numbers, and grade point
averages. The data set will be accessed in the key field mode, with the
343

student’s identification number being the key field. Code a class named
Listing that defines the nodes. The class must comply with the five
conditions (assumptions) of a generic, homogeneous, fully
encapsulated data structure. As such, your class should include all the
methods in the class shown in Figure 2.28 . Test it with a progressively
developed driver program that demonstrates the functionality of all of
its methods.
46. Code an application program that keeps track of student
information at your college: their names, identification numbers (the
key field), and grade point averages in a fully encapsulated, Direct
Hashed data structure. When launched, the user will be asked to input
the maximum size of the data set, the initial number of students, and
the initial data set. Once this is complete, the user will be presented
with the following menu:
Enter: 1 to insert a new student’s information,
2 to fetch and output a student’s information,
3 to delete a student’s information,
4 to update a student’s information,
5 to output all the student information, and
6 to exit the program.
The program should perform an unlimited number of operations until
the user enters a 6 to exit the program. Assume the identification
numbers will be in the range 1000 to 50,000.
47. Code the application described in Exercise 46 using a hashed
structure based on the Quadratic collision algorithm, but now the
student’s name will be the key field.
48. Code the application described in Exercise 46 using a Java’s
Hashtable data structure.
49. Code the application described in Exercise 46 using a Linked
Hashed data structure.
50. The code presented in Figure 5.28 processes nodes whose keys are
Strings. Modify its code so that it processes keys that are objects in any
class. Provide a driver program to demonstrate the modifications
function properly.
51. Modify the code presented in Figure 5.28 so that it is a fully
generic class using the generic features of Java 5.0. Is should be able to
344

processes nodes and keys that are objects in any class. Provide a driver
program to demonstrate the modifications function properly.
1 Of the four schemes depicted in Figure 5.1 only the middle two can be implemented in
Java.
2 Assuming the node width, w (as shown in Chapter 2 ), is greater than 16.
4 The actual functional relationship is: average search length = L avg ≤ 1/(1 − l ). See
Appendix B for its derivation, which assumes that all N primary storage area locations are
equally probable to be generated each pass through the collision algorithm.
5 A prime number is a positive integer that can only be divided evenly by itself and one.
6 Java uses 32 bit Unicodes to represent characters. The first 256 characters in the
Unicode table (hexadecimal 0000 to 00FF) are the ISO 8859-1 extension of the ASCII code
set. Figure 5.17 assumes that the eight leading zeros of each Unicode character in the key have
already been stripped away.
7 Returning the absolute value of the pseudo key effectively ignores overflow into the
sign (leftmost) bit of the pseudo key.
8 When the Direct Hashing function is used and preprocessing is either not performed
or the preprocessing algorithm produces unique pseudo keys, a hashed data structure is a
perfect hashed data structure.
9 Its home address is 4 because we have assumed the Division algorithm is used for the
hashing algorithm, 4 = 954 % 19.
10 Actually, in the language Java, all reference variables are initialized to null when
created (Line 12). Lines 14–15 therefore, are redundant, but are included to make the
initialization more obvious to the reader.
11 ( n − i ) / ( N − i ) decreases as i increases since n is always less than N and, thus, n
reach zero before N .
12 One access times four passes.
13 Two accesses times four passes.
15 If a key’s class does not overwrite these two methods the implementation in the class
Object is used and the pseudo key returned from the method hashCode is based on the node’s
location rather than the node’s key.
16 Also called a Chained Hashed structure.
17 We have assumed a linked list implementation without a “dummy” node between the
list header and the list.
345

CHAPTER 6
Recursion
OBJECTIVES
The objectives of this chapter are to introduce the student to the topic
of recursion, to teach the student how to think recursively, and how to
formulate and implement recursive solutions. More specifically, students
will be able to
Understand the concept of recursion and recursive definitions.
Understand how recursive algorithms produce their result, and
understand the execution path of methods that implement them.
Implement a recursive algorithm, and understand its iterative
counterpart.
Think recursively. Understand a methodized approach to formulating
many recursive algorithms that includes the discovery of a base case ,
reduced problem , and general solution , and understand the flowchart
that integrates these into a recursive solution.
Extend their ability to think recursively by generalizing the
methodized approach to include problems that require multiple base
cases, reduced problems, and general solutions.
Understand, and be able to explain, the advantages of recursive
solutions as well as the limitations and problems associated with
recursion.
Understand a technique called dynamic programming , and its role in
improving the speed complexity of recursive algorithms.
Understand a general problem solving technique called backtracking
that utilizes recursion, understand its use in the solution of the Knights
Tour problem, and be able to apply the technique to the solution of other
problems such as the Queens Eight problem.
346

6.1 What Is Recursion?
Most of us at some point in our lives have been told not to define
something in terms of itself. We were told that the word we are defining
should be not be used in its definition. Usually, this is good advice. When
asked to define a dog, the statement “a dog is a dog” gives no insight into
what a dog really is. Yet there are times when it is not only allowable to
use the word we are defining in its definition, but actually advisable; when
the word is used to only partially define itself.
For example, to define the word ancestor it is not permissible to say
“an ancestor is an ancestor,” but the statement
“An ancestor is a parent or an ancestor of a parent”
is a permissible definition even though the word ancestor is used in
the definition. The phrase “ancestor of a parent” is only part of the
definition; the other part is the phrase “a parent.”
This type of definition is called a recursive definition. Typically,
recursive definitions consist of something we are familiar with (e.g., a
parent), and some sort of recurrence relation involving the word we are
trying to define and the familiar entity (e.g., ancestor of a parent ). The
strength of a recursive definition is that the recurrence relation, or
repetitive part of the definition, is used to succinctly expand the scope of
the definition.
For example, suppose we wish to determine the meaning of the
statement
“Mary is an ancestor of George.”
Using the nonrepetitive portion of the definition of ancestor (an
ancestor is a parent) the statement becomes “Mary is a parent of George,”
and its meaning is clear. However, the word ancestor in the original
statement could also mean “an ancestor of a parent.” In this case the
statement becomes
“Mary is an ancestor of a parent of George.”
To find the meaning of this statement we could use the nonrepetitive
portion of the definition of ancestor (an ancestor is a parent). Then the
statement becomes “Mary is a parent of a parent of George,” and the
original statement means Mary is George’s grandparent. Alternately, we
could again use the repetitive portion of the definition of ancestor (an
ancestor is an ancestor of a parent). In this case, the statement becomes
“Mary is an ancestor of a parent of a parent of George.”
347

This could mean that Mary is George’s great grandparent or a more
distant relative;
“Mary is an ancestor of a parent of a parent of a parent of George.”
Thus, we see that our original recursive definition of ancestor “an
ancestor is a parent or an ancestor of a parent” is a succinct way of saying
an ancestor is a parent, or a grandparent, or a great grandparent, or a great
great grandparent, etc.
Recursive definitions are also used in computer science in the
statement of algorithms. A recursive algorithm is an algorithm defined at
least partially in terms of itself. Not all problems have recursive solutions.
However, for those that do, the recursive algorithm often offers an elegant
and succinct alternative to iterative algorithms. 1
When most people are introduced to the concept of a recursive
algorithms, they usually find them difficult to understand and even
somewhat magical. After an initial examination of a recursive algorithm,
we are often convinced it will not work, that it is incomplete, or that it only
addresses a subset of the problem. This is because only a small percentage
of the population has the innate ability to think recursively. The majority
of us have to acquire this cognitive ability. We have to be trained to think
recursively.
There are two levels of recursive thinking we should aspire to. They
are, the ability to understand a recursive algorithm and, at a deeper level,
the ability to formulate a recursive algorithm; the former skill being a
necessary foundation for the latter.
6.2 Understanding Recursive Algorithms
One way of learning how to understand recursive algorithms is to
compare a nonrecursive definition of an algorithm to its recursive
definition. Another way is to code the recursive version of the algorithm
and trace its execution path. Let us begin with the former technique and
consider the problem of calculating n factorial.
6.2.1 n Factorial
The function n factorial, written as n !, is defined for non-negative
values of n as:
348

This definition is a nonrecursive definition of n factorial because the
factorial operator (!) is not used in its own definition. Using this
nonrecursive definition, we calculate 0! = 1 and 4! = 4 * 3 * 2 * 1 = 24.
The second portion of this definition, n ! = n * (n − 1) * (n − 2) * (n −
3) *…* 1, can be written recursively by realizing that
Using this equality in the second part of the nonrecursive definition it
becomes
which is the recursive definition of n !. It is a recursive definition
because unless n = 0, n factorial is defined in terms of another factorial, (n
− 1)!. The nonrecursive definition explicitly tells you how to calculate n !
as: n * (n − 1) * (n − 2) * (n − 3) *…* 1. The recursive definition does not
explicitly tell you how to calculate n!, but rather seems to assume that you
already know something about the factorial function. That is, that you
already know how to calculate (n − 1)!.
Like most recursive definitions, this definition of n ! seems to beg the
question “how do I calculate n !?” It can be likened to you asking your
friend John the value of n factorial. John, being blessed with the ability to
think recursively, begs the question by responding “I’ll call Mary and ask
her the value of n − 1 factorial, and then I’ll have the answer for you.”
What John plans to do is multiply Mary’s answer (the value of n − 1
factorial) by n to determine the answer to your question. Thus, for n = 4,
once John calls Mary and she tells him that the value of 3 factorial is 6, he
can then calculate 4 factorial as: 4! = 4 * 3! = 4 * 6 = 24.
The only remaining question is how will Mary know the value of 3
factorial? Strangely enough, the answer is that she doesn’t need to know it
because Mary, like John, is blessed with the ability to think recursively.
Therefore, Mary also has an uncontrollable urge to beg questions. Using
the same thought process John used, Mary calls her friend Sue and asks
her the value of 2 factorial, who asks Bob the value of 1 factorial, who (at
long last) asks Al the value of 0 factorial. Finally, Al no longer needs to
beg the question because Al, like everyone else in the world, knows that 0
factorial is 1 (by definition).
349

Figure 6.1 The Ten-Step Recursive Process to Evaluate 4!
Al responds immediately with his answer 1, which Bob uses to
calculate 1! as 1! = 1 * 0! = 1 * 1 = 1, which is used by Sue to calculate 2!
= 2 * 1! = 2 * 1 = 2, which is used by Mary to calculate 3! = 3 * 2! = 3 * 2
= 6, which is finally used by John to calculate 4! = 4 * 3! = 4 * 6 = 24.
This ten-step recursive process performed by John, Mary, Sue, Bob,
and Al to determine the value of 4 factorial is illustrated in Figure 6.1 .
Each person that took part in our recursive algorithm, except for John,
is called a level in the recursive solution. In our example, there were four
levels of recursion: Mary, Sue, Bob, and Al, with Al said to be at the
deepest level of recursion. Typically, in a recursive process we enter
deeper and deeper recursive levels with problems that get progressively
closer and closer to the nonrepetitive part of the recursive algorithm; the
portion of the definition whose solution is “known to everyone.” In the
jargon of recursion, this part of the algorithm is called the base case . In
our example, Bob’s problem (1!), was closer to the nonrepetitive part of
the definition, 0!, than Sue’s problem (2!), which was closer to 0! than
Mary’s problem (3!), etc. Once we reach the base case, the recursive
process works its way back up the recursive levels, most often using
solutions supplied by the level just below, to determine the solution to the
original problem.
6.2.2 The Code of a Recursive Algorithm
Let us now turn our attention to the alternate way of learning how to
understand recursive algorithms, coding the algorithm, and tracing its
execution path. To begin with, we will express the recursive algorithm for
n ! in pseudocode.
350

Figure 6.2 The Coding of the Recursive Algorithm for n Factorial
Lines 1 and 2 are the portion of the algorithm that is known to
everyone (the base case), and Lines 3–6 are the recurring portion of the
algorithm.
In most programming languages, recursive algorithms are coded as
subprograms (methods). Some languages use a special syntax to indicate
that the subprogram is implementing a recursive algorithm. This is not the
case in Java. Any Java method can implement a recursive algorithm.
Figure 6.2 presents the class Factorial with one method, nFactorial, that
implements our pseudocode algorithm for n !. The value of n is sent to it as
an argument and the method returns the value of n !.
Line 2 of Figure 6.2 declares the method, nFactorial, to be a static
method. When a static method is invoked, it is not preceded by an object
name. This is appropriate for this method since it does not operate on, or
use, an object’s member data. 2 The method invokes itself on Line 7 to
determine the values of (n − 1)!. This is one of the indications that a
method implements a recursive algorithm; it invokes itself. When this is
the case, the method is said to be directly recursive . Alternately, a method
could be indirectly recursive . An indirectly recursive method invokes
some other method that invokes it, or the other method invokes another
method that invokes it, etc.
Figure 6.3 is an application program that uses the method nFactorial
to calculate 4!. The output it produces appears at the bottom of the figure.
351

Line 4 invokes the method nFactorial to determine the value of 4
factorial. Since the method is a static method (Line 2 of Figure 6.2 ), Java
requires the name of the class be coded before the name of the method
when it is invoked (Line 4).
6.2.3 Tracing a Recursive Method’s Execution Path
In order to more easily trace the execution path of the recursive
method shown in Figure 6.2 , we will add statements to it that generate a
significant amount of additional output. These statements will produce an
output whenever:
• the method is invoked (Line 2 of Figure 6.2 executes),
• the method returns a value (Lines 5 or 8 of Figure 6.2 executes), and
• the method invokes itself (Line 7 of Figure 6.2 executes).
Figure 6.3 An Application Program to Calculate 4! and its Output
Although we are more interested in the output than the details of the
code that produces it, in the interest of completeness the revised code is
shown in Figure 6.4 . The expanded method is named nFactorialTrace, and
the expanded class is named FactorialTrace.
The data members count and time (Lines 2 and 3) have been added to
the class to count the number of times the method nFactorialTrace is
invoked and to simulate the passage of time. These variables are
incremented on Lines 6 and 7 every time the method is invoked. In
addition, the variable time is also incremented whenever a value is
returned (Lines 12 and 21). Lines 8 and 9 have been added to produce an
output every time the method is invoked. Lines 13, 14, 22, and 23 have
been added to produce an output every time the method returns a value.
Line 19 produces an output every time the method invokes itself.
Let us now turn our attention to the output produced by the
application program shown in Figure 6.3 , assuming the invocation on
352

Line 4 has been changed to invoke our revised method. 3 The output it
produces, when calculating 4!, is shown in Figure 6.5 . Assuming that one
increment of the variable time is a second, we see that the application takes
10 seconds to execute (at time 1, at time 2, etc.). As indicated by the
second line of the figure, at time 1a, nFactorial is invoked for the first time
by the application program to calculate 4 factorial (n = 4). This invocation
does not complete (return the value 24) until 10 seconds have past (see the
bottom of Figure 6.5 ), because the algorithm is recursive and it has
invoked itself at time 1b to determine 3!. With each tick of the clock the
program enters a deeper level of recursion.
At time 6 the base case (0!) is encountered and the recursion begins to
unwind , returning the value of 0! from the fifth invocation to the fourth
invocation. At time 7, 1! is returned from the fourth invocation to the third
invocation; at time 8, 2! is returned from the third invocation to the second
invocation; and at time 9, 3! is returned from the second invocation to the
first invocation.
Figure 6.4 The Class FactorialTrace
Finally, at time 10, the value of 4! is returned to the application
353

program from the first invocation of nFactorialTrace and the application
program outputs this value. Thus, 10 seconds has passed from the time of
the first invocation to nFactorialTrace is initiated (to determine the value
of 4!) to the time that the invocation terminates (returning 24). During this
time, four other invocations of nFactorialTrace are initiated, executed, and
terminated.
Figure 6.6 illustrates the five invocations of the method
nFactorialTrace (or nFactorial) for n = 4. The circles numbered 1 through
5 indicate the five invocations of the method (i.e., the winding up of the
recursion), and the circles 6 through 10 indicate the returns from the five
invocations (the unwinding of the recursion).
Figure 6.5 Trace of the Recursive Invocations when nFactorialTrace
Calculates 4!
One final analogy to aid our understanding of how recursion works is
354

shown in Figure 6.7 . The invocations and the returns from the invocations
can be likened to a cascading row of dominos that fall from left to right
during the recursive invocations, and then are reset upright from right to
left during the returns from the invocation. The left hand domino
(analogous to the first invocation of the method) stays down the longest
waiting to be set upright (analogous to the first method receiving its
returned value).
Figure 6.6 The Execution Sequence of the Method nFactorial for n =
4
355

Figure 6.7 Recursive Invocation and the Return Falling Domino
Analogy
6.3 Formulating a Recursive Algorithm
Having gained an understanding of recursive algorithms and how
they execute, we now turn our attention to the techniques for formulating
recursive algorithms. These techniques can be methodized, and the
methodology can be directly applied to the formulation of recursive
algorithms for some problems. However, there is still a certain amount of
“art” remaining in the development of recursive algorithms because our
methodology requires a bit of creativity to adapt it to certain problems, and
a good deal of creativity to adapt it to many other problems. That being
said, an understanding of the methodized approach is still a valuable skill
to acquire because it not only furthers our understanding of recursion but
can also take us far into the development of most recursive algorithms.
More importantly, it reveals insights into the thought processes of those
blessed with the innate ability to think recursively.
356

6.3.1 Definitions
Before examining the methodology of developing recursive
algorithms, we will define the terms base case , reduced problem , and
general solution used in this methodology. (Don’t worry; we won’t define
them recursively.) The problem of computing n ! will be used to illustrate
the definitions.
Base Case
The base case is the known portion of the problem solution . It is the
portion of the solution that “everyone knows,” the nonrecursive portion of
the solution. In the n factorial problem, the base case is 0! = 1. Often the
base case is referred to as the escape clause because it allows the
algorithm to escape from the recursive invocations.
Reduced Problem
The reduced problem is a problem very “close” to the original
problem, but a slight bit closer to the base case . In the n factorial
problem, the reduced problem is (n − 1)!, which is as close to the original
problem (n !) as we can get, and it is closer to the base case (0!). One
important quality of the reduced problem is that when it is repeatedly
reduced, it degenerates into the base case. By repeatedly reduced, we mean
that the relationship between the original problem and the reduced problem
is repeatedly applied to the reduced problem. In our case, (n − 1)! becomes
(n − 2)!, which becomes (n − 3)!…, which becomes (0)! the base case. If it
does not eventually degenerate into the base case, the algorithm will enter
an infinite loop. This is one of the dangers of an incorrectly chosen
reduced problem (often referred to as a false base case because the
statement “the reduced problem degenerates into the base case” is false).
General Solution
The general solution is the solution to the original problem expressed
in terms of the reduced problem . In other words, the general solution uses
the solution to the reduced problem to solve the original problem. In the n
factorial algorithm, the general solution uses the reduced problem as a
multiplier of n , in that the general solution in the n ! recursive algorithm is
n * (n − 1)!.
357

The base case, reduced problem, and general solution for the n !
recursive algorithm are summarized in Table 6.1 . Having gained an
understanding of these terms, we will now be able to explore the
methodology of recursive thinking.
6.3.2 Methodology
To begin with, it is worth repeating that not all recursive algorithms
can be formulated using this methodology. Although it can be followed
verbatim to produce a recursive algorithm for some problems with
recursive solutions, it must be modified to solve most problems. However,
it does serve as starting point for all problems with recursive solutions, and
can usually be “tweaked,” morphed, or modified to accommodate most
problems.
The methodology consists of four steps:
1. Determine the base case.
2. Determine the reduced problem.
3. Determine the general solution.
4. Combine the base case, reduced problem, and general solution to form the recursive
algorithm.
General Flowchart
Before we examine the techniques involved to complete the first three
steps, we will turn our attention to Step 4 of the methodology. The
completion of Step 4 can be a trivial effort because many times the base
case, reduced problem, and general solution are combined as shown in the
flowchart presented in Figure 6.8 . Although the shadowed box in the
flowchart looks as innocent as the others, it isn’t. What you see is just the
“tip of the iceberg” because it is the recursive portion of the algorithm. If
we enter this box of the flowchart, the algorithm begins again, initiating
another level (a lower level) of recursion.
The first three boxes of the flowchart will be repeatedly executed
until the second box determines that the current level of recursion is the
base case. At this point we “escape” from the repeated execution of boxes
358

1-3, return the base case solution, and the recursion begins to unwind. If
the reduced problem does not eventually reach the base case, then the
algorithm will never end. This is why, in the definition of the reduced
problem we stated that “one important quality of the reduced problem, is
that when it is repeatedly reduced, it degenerates into the base case. ”
Figure 6.8 A Common Approach to Step 4 of the Methodology for
Formulating Recursive Algorithms
To demonstrate how the flowchart is used, let us assume we have
already completed Steps 1, 2, and 3 (using techniques not yet discussed) to
determine the base case, reduced problem, and general solution for n !
shown in Table 6.1 . Then, mechanically following the flowchart to
combine these three parts of our problem solution, the pseudocode
algorithm for n ! would be:
359

When this algorithm is coded into a Java method (e.g., Figure 6.2 ),
the term (n − 1)! on Line 4 of the algorithm is replaced with an invocation
to the method, and Lines 5 and 6 of the algorithm can be combined into
one (return) statement.
Let us now turn our attention to Steps 1, 2, and 3 of the methodology:
1. Determine the base case.
2. Determine the reduced problem.
3. Determine the general solution.
Determining the Base Case
Determining the base case is the easiest of these three steps. Here, we
try to identify a particular instance of the problem, a special case, whose
solution is known. Often the base case is trivial. For example, the base
case for a recursive algorithm to output a string is to output a string of
length 1, a single character. Sometime it is a defined solution. This is the
case for n ! because 0! is defined as 0! = 1. When the problem involves an
integer variable n , the base case often occurs when n is 0 or 1. If both 0
and 1 are acceptable values for n , usually the base case is the one that is
farthest from the original problem, n = 0.
Many list search algorithms (e.g., the Binary Search algorithm) can
be written recursively. Two common base cases for a recursive search
algorithm is when the item we are currently examining in the list is the
item we are looking for, or the list is empty.
Determining the Reduced Problem
Once we determine the base case, the reduced problem is usually
determined next. Of the four steps in our methodology, this is usually the
most difficult and requires the most imagination and creativity. The best
technique for this is to consider the original problem and the base case in
the definition of a reduced problem. The reduced problem should
• be a problem “similar” to the original problem,
• be a small “step” away from the original problem in the “direction” of
the base case, and
• degenerate into the base case when repeatedly reduced.
360

In other words, the reduced problem should be “in between” the
original problem and the base case, but much closer to the original
problem than to the base case, and eventually become the base case.
For example, problems similar to n ! that are in the direction of the
base case, 0! are (n − 1)!, (n − 2)!, (n − 3)! etc. Which one should we pick
for the reduced problem? The first of these is certainly the closest to the
original problem, but if n = 1,000,000 then all three of them, when
compared to the base case, are very close to the original problem. To select
the correct reduced problem from these we turn to the last part of the
definition of a reduced problem which states it should “degenerate into the
base case when repeatedly reduced.”
Suppose we were to choose (n − 2)! and n were odd, say n = 5. Then
the reduced problem would be (5 − 2)! = 3!. Considering 3! to be an
original problem and reducing it again, we obtain (3 − 2)! = 1! One more
reduction gives (1 − 2)! = −1!. Since these repeated reductions of the
reduced problem do not become the base case (0!), (n − 2)! is not a valid
reduced problem. The only reduced problem that, when repeatedly
reduced, degenerates into the base case for all values of n , is (n − 1)!.
Therefore, it is selected as the reduced problem.
Determining the General Solution
To determine the general solution, we ask ourselves the question,
“how can I use the reduced problem to generate, or solve, the original
problem?” In the case of n !, where the reduced problem is (n − 1)!, the
answer is to multiply the reduced problem by n . Therefore, the general
solution to n ! is n * (n − 1)!.
Summary of the Methodology
Table 6.2 summarizes the techniques used in our four step
methodology to formulate recursive algorithms. Although this
methodology does not produce a recursive algorithm for many problems, it
does train us to think recursively. It also serves as a starting point for all
problems and can be modified to accommodate most problems.
6.3.3 Practice Problems
As we have discussed, most recursive algorithms do not fall into our
“cookie cutter” methodology. They often have multiple base cases, they
361

use the reduced problem several times in the general solution, the reduced
problem is not a “small step” from the original problem, or they have
alternate general solutions. The discovery of these recursive algorithms
require some creativity and, as a result, can be quite challenging. However,
the old axiom “practice makes perfect” certainly applies to the learning
process of how to think recursively; practice can help move us toward that
set of people who naturally think recursively.
To minimize the frustration associated with this practice, it is
advisable to start with problems that follow our methodology, then
progress to those that “almost” follow it, and finally, venture into what
some students refer to as “wide open territory.” Table 6.3 presents practice
problems that require different levels of innovation and creativity. The
problems in the leftmost column of the table require the least amount of
creativity, while the problems in the rightmost column require the most
creativity. Before concluding this section, we will formulate and
implement recursive algorithms for four of these problems (two from the
first column of the table, one from the second column, and one from the
third column). The remaining problems are left as an exercise for the
student. The base case(s), reduced problem(s), and general solution(s) for
the all the problems are given in Table 6.8 , which is presented at the end
of Section 6.4 .
Table 6.2
The Four-Step Methodology for Developing Recursive
Algorithms
Step Techniques
4 By “repeatedly reduced,” we mean that the relationship between the
original problem and the reduced problem is repeatedly applied to the
reduced problem.
1. Determine the base case. Look for a known solution, a trivial
case, a special case, a defined value,
or when n = 0 or 1
(e.g., 0! is defined as 1; an empty
list; or a search item is found).
2. Determine the reduced problem. Look for a problem
• similar to the original problem,
• in between the original problem
and the base case,
• much closer to the original problem
362

than to the base case, and
• that, when repeatedly reduced,4
degenerates into the base case (e.g.,
(n − 1)!; search a sub list).
3. Determine the general solution. Think of how the reduced problem
can be used to solve original
problem
(e.g., n ! = n * (n − 1)!).
4. Combine the base case, reduced
problem, and general solution to
form the recursive algorithm.
Use Figure 6.8 or morph it to
accommodate the problem
(e.g., for n ! use it verbatim).
X n
Let’s consider the problem of determining x n for all non-negative
integer values of n . To discover a recursive algorithm for this problem,
our methodology can be followed verbatim. It involves a positive integer
variable n , and so the base case often occurs when n is 0 or 1.x 0 and x 1
are both legitimate candidates for the base case because their values, 1 and
x respectively, are known by definition. However, as we have stated,
usually the base case is the one that is farthest from the original problem
which is x n . Since, for positive values of n , 0 is further from n than 1 is,
the base case for this problem is usually chosen to be x 0 = 1.
Table 6.3
Practice Problems to Improve our Ability to Think Recursively
Increasing Order of Difficulty
Generally
Follows the
Methodology
Multiple Base
Cases and
Reduced
Problems
Uses the
Reduced
Problem
Twice Multiple Base Cases
n ! Generate the n
th term in the
triangular series
Towers of
Hanoi
Finding the greatest
common denominator
of two positive integers
A number raised
to a positive
integer power, x
n
Binary search of
an array
Traversing a
binary tree
(see Chapter 7
)
Sum of the Quick Sort
363

integers from n
to 1
(see Chapter 8
)
Product of two
integers, m * n
Merge Sort
(see Chapter 8
)
Generate the nth
term in the
triangular series
Output the
characters of a
string in reverse
order
Our reduced problem should be similar to the original problem, x n ,
between it and the base case, x 0 , and much closer to x n than to x 0 .
Candidate reduced problems are x n −1 , x n −2 , etc. Of these, x n −1 is the
only one that will degenerate into x 0 for all n when repeatedly reduced.
Therefore, the reduced problem is chosen to be x n −1 .
To determine the general solution, we ask ourselves the question,
“how can I use the reduced problem to generate, or solve, the original
problem?” In this case the question becomes “what can I do to x n −1 to
make it x n ?” The answer is to multiply it by x . Therefore the general
solution is x n = x * x n −1 .
Figure 6.9 A Recursive Method to Compute xn
364

Table 6.4 summarizes the base case, reduced problem, and general
solution for x n and includes the problem of n ! for comparative purposes.
Finally, the base case, reduced problem, and general solution are
combined as shown in Figure 6.8 to produce the recursive algorithm for x n
. Figure 6.9 presents the code of a method xToN that implements the
algorithm.
n th Term of the Fibonacci Sequence
Calculating the n th term of the Fibonacci sequence is a good problem
to study next because although the discovery of its recursive algorithm
generally follows our methodology, there are two nuances. It is an
example of a problem with multiple base cases and two reduced problems;
however, both are easily detected because the series is usually defined
recursively. The Fibonacci sequence, f n is defined as
the first term, f 1 = 1,
the second term, f 2 = 1, and
all other terms, f n = f n −1 + f n −2 .
Thus, the terms of the sequence are: 1, 1, 2, 3, 5, 8, 13, 21, 34….
From its definition we see the nuance it presents to our methodology.
There are two trivial solutions, when n = 1 and when n = 2. These become
our base cases.
The next steps in our methodology are to determine the reduced
problem and the general solution. Since the definition is recursive, they are
easy to discover. Two reduced problems are used to calculate f n , f n −1 ,
and f n −2 , and our general solution is f n = f n −1 + f n −2 .
365

Figure 6.10 A Recursive Method to Determine the n th Term of the
Fibonacci Sequence
Table 6.5 summarizes the base case, reduced problem, and general
solution for this problem and includes the problems of n ! and x n for
comparative purposes.
Finally, the base cases, reduced problems, and general solution are
combined as shown in Figure 6.8 , to produce the recursive algorithm for
the n th term of the Fibonacci sequence. Figure 6.10 presents the code of
the method fibonacci that implements the algorithm.
Reverse a String, s , of Length n
To discover a recursive algorithm for this problem, our methodology
can be followed verbatim. However, this problem is more challenging
because the general solution is a little more difficult to determine.
We will assume that the problem will be to output a given string s of
length n in reverse order. Thus, the string “Bill” should be output as “lliB”.
Once again the problem involves a positive integer variable n , and so the
base case often occurs when n is 0 or 1. Both n = 0 and n = 1 are legitimate
base cases; for n = 0, output nothing and for n = 1, output the string s .
However, we will again choose the base case to be the one that is furthest
from the original problem, n = 0. In this case, we would not produce an
output.
Our reduced problem should be similar to the original problem
(reverse a string, s , of length n ), between it and the base case (reverse a
string, s , of length 0), and much closer to the original problem than it is to
the base case. Candidate reduced problems are a reversed string of n − 1
characters, n − 2 characters, etc., but of these, the first is the only one that
will degenerate into the base case for all n when repeatedly reduced. For
example, if the reduced problem reverses a string of n − 2 characters and n
is 3, two reductions (3 − 2 = 1, and 1 − 2 = −1) would skip the base case.
366

Therefore, the reduced problem is chosen to output a string of n − 1
characters in reverse order.
To determine the general solution, we ask ourselves the question,
“how can I use the reduced problem to generate, or solve, the original
problem?” The reduced problem can be used to output n − 1 characters of
the string backward. The remaining character must be output by the
general solution. But which character would that be? The answer lies in
which group of characters the general solution uses the reduced problem to
output. If it’s the first n − 1 characters, then the general solution must
output the last character, and vice versa. Either approach will work. We
will take the former approach. Since the last character must be output first,
the general solution is to output the last character and then use the reduced
problem to output the first n − 1 characters in reverse order.
Table 6.6 summarizes the base case, reduced problem, and general
solution for this problem, and includes the problems of n ! and x n for
comparative purposes.
Finally, the base case, reduced problem, and general solution are
combined as shown in Figure 6.8 to produce the recursive algorithm
outputting a string backward. Figure 6.11 presents the code of a method
stringReverse that implements the algorithm. The method charAt on Line
5 is a method in the Java String class that returns a character in a string,
with the first character being at position 0. Thus, the last character in an n
character string is at position n − 1.
367

Figure 6.11 A Recursive Method to Reverse the Characters of a
String
Towers of Hanoi
This problem is a more challenging one because both the reduced
problem and general solution are a little more difficult to determine.
However, our general methodology is still applicable. There is another
important reason for discussing this problem. The previous problems
discussed are more easily solved using loops rather than recursion. Their
recursive solutions were developed for pedagogical reasons. However, the
Towers of Hanoi problem is typical of problems in which the recursive
solution is much simpler than the iterative solution. This class of problems
is the reason we study recursion.
The Towers of Hanoi is a puzzle conceived by the French
mathematician Eduardo Lucas in 1883. The objective is to move n rings
from one tower (designated “S” for source tower) to another tower
(designated “D” for destination tower). A third tower (designated “E” for
extra tower) is available for the temporary storage of rings. The rings are
each of a different radius and are initially stacked on tower S in sized
order, with the largest radius ring on the bottom. When moving the rings
three rules apply.
1. The rings must be moved one at a time.
2. A small ring can never be placed on top of a larger ring.
3. When a ring is removed from a tower it must be placed on a tower before another ring is
removed.
Our problem will be to discover a recursive algorithm that outputs the
moves to relocate n rings from tower S to tower D without violating the
rules.
Applying our methodology, we first look for a base case. As with the
previous problems, we are again dealing with an integer variable, n , the
number of rings to be relocated. Thus, we would suspect that the base case
is when there are no rings, n = 0, or there is one ring, n = 1, to move.
368

Considering the rules of the puzzle, n = 1 is a trivial case whose solution
everyone knows. Simply move one ring from tower S to tower D; end of
puzzle.
The next step is to find the reduced problem. Since the base case is
for one ring and the original problem involves n rings, we would suspect
(as in the previous problems) that the reduced problem would involve n −
1 rings. So far, so good. But there is a problem. In the previous problems
the reduced problem was identical to the original problem except that n − 1
was substituted for n . In this problem, however, if we move n − 1 rings
from tower S to tower D then the largest ring is left on tower S and it can’t
be moved to tower D without violating Rule 2. The solution here is to
generalize the reduced problem so that it could be used to move n − 1 rings
from any tower to any other tower.
Now we will examine the general solution. According to our
methodology, the general solution should use the reduced problem to solve
the original problem. And so the question is, if we have n rings on tower S
and the ability to move n − 1 rings from any tower to any other tower,
which towers would we select? For our answer we look back to our initial
discussion of n ! where we said that if we could find someone who knew
(n − 1)! we would know n ! (= n * (n − 1)!). An analogous argument is
applied twice in this problem. If we could find someone who could move n
369

− 1 rings from tower S to E and then someone who could move the n − 1
rings from E to D, and in between move the n th (largest) ring from tower
S to D, we would have our general solution. The unique thing here is that
it was not obvious that the reduced problem had to be generalized and used
twice in the general solution. Once to move n − 1 rings from tower S to E,
and a second time to move them from tower E to D.
Table 6.7 summarizes the base case, reduced problem and general
solution for this problem and includes the previously discussed problems
for comparative purposes.
Finally, the base case, reduced problem, and general solution are
combined as shown in Figure 6.8 to produce the recursive algorithm for
the Towers of Hanoi puzzle. Figure 6.12 presents the code of a method
hanoi that implements the algorithm. The method’s first parameter, n , is
the number of rings to be moved. Its last three parameters accept the
integer tower numbers of the source (S), destination (D), and extra (E)
towers. Also included in the figure is the output of the method for three
rings (n = 3) being moved from tower number 1 to 2, with tower 3 being
the extra tower.
Figure 6.12 A Recursive Method to Solve the Towers of Hanoi
Puzzle and its Output for Three Rings
6.4 Problems with Recursion
370

The elegance of the Towers of Hanoi recursive algorithm often
attracts us to recursive solutions. But all that glitters is not gold. Aside
from the fact that the ability to think recursively does not come naturally to
most people, there are two major problems associated with the use of
recursive algorithms in our programs:
• They tend to run slowly.
• At run-time, they can consume an unacceptably large amount of main
memory.
Both of these problems are amplified by the manner in which modern
compilers transfer the execution path, and the shared information, to and
from methods. When a method is invoked, the values of the shared
information 5 (arguments and returned value) must be passed between the
invoker and the method. In addition, in order to continue the program after
the method completes execution, the address of the line of code after the
method invocation (often referred to as the return address ) and the
contents of the CPU’s registers at the time of the invocation (often referred
to as the state of the machine ) must be “remembered.” Finally, storage
must be allocated for the method’s local variables.
Figure 6.13 Changes to the Run-time Stack as Method A Invokes
Method B, and then Method B Completes Execution
The memory allocated to this information is part of a stack called the
run-time stack , and the collection of information is referred to as a stack
frame . Every time a method is invoked a new stack frame is pushed onto
the stack, and when an invocation is complete the information is popped
off the stack. Figure 6.13 shows the progression of the run-time stack after
method A begins execution (stack status 1), after it invokes method B
371

(stack status 2), and after method B completes execution (stack status 3).
The pushing and popping of the run-time stack takes time, and the
stack occupies main memory. For a program that invokes nonrecursive
methods, this time and memory use is most often negligible 6 because the
nesting of non-recursive method calls usually does not exceed a depth of
10 methods. However, recursive methods often call themselves hundreds
or thousands of times as would be the case to calculate 2000!. Every time
we enter a lower level of recursion, a new stack frame is built. The time to
create and push each frame, and the memory associated with all the frames
can become excessive. At best, a recursive method runs slowly and
consumes memory. At worst, programs that invoke recursive methods run
unacceptably slowly and, since most operating systems dedicate a limited
amount of memory to the run-time stack, can terminate in a run-time
“Stack Overflow” error.
The safe alternative is to abandon the elegance of a recursive
algorithm in favor of an iterative algorithm. Iterative algorithms are
algorithms involving loops, and any recursive algorithm has an iterative
counterpart. For example, the iterative version of n ! is
Tests run on a AMD XP 3000 Windows-based platform running at
2.17 GHz with 448 MB of RAM, show that the iterative solution for n !
runs three times faster than the recursive solution, and since the iterative
solution eliminates recursive invocations it cannot end in a run-time stack
overflow error.
6.4.1 Dynamic Programming Applied to Recursion
Incorporating dynamic programming into recursive algorithms allows
us to retain the elegance of recursive solutions while approaching the
speed and run-time stack requirements of iterative solutions. It is a
technique for efficiently computing solutions to problems that involve
recurrences by storing partial results. To apply dynamic programming to
recursive algorithms we look for parts of the algorithm that repeatedly
calculate the same values. Then, the first time these values are calculated,
they are stored in a memory resident table. From that point on, rather than
372

recalculate these values they are simply fetched from the table.
For example, consider a program that calculates the value of 4! and
then 5! with two invocations to a method that implements a recursive
algorithm of the factorial function. As illustrated in Figure 6.6 , in the
process of calculating the value of 4! the recursive algorithm also
calculates the value of 3!, 2!, and 1!. A dynamic version of this algorithm
would store these calculated values (as well as the value of 4!) in a table
during the first invocation of the method. Then, when 5! is calculated, 4!
would not be recalculated but simply read from the table. This saves not
only computational time, but also reduces stack overflow errors by
reducing the number of levels of recursion for all but the first invocation of
the recursive method.
In the case of n !, a one-dimensional array could be used to store the
calculated values with the value of n used as an index into the array. Thus,
1! would be stored at index 1, 2! stored at index 2, etc. The first column in
Figure 6.14 shows the array in its initialized state, and the second shows
the array contents after the recursive algorithm calculates the value of 4!.
Then, when 5! is calculated, the value 4! is simply read from the table and
multiplied by 5, eliminating the need for recursion. Before ending, the
method would write the value of 5! (120) to the table, as shown in the third
column of Figure 6.14 .
Figure 6.14 The Changes to the Contents of an Array used in the
Dynamic Recursive Algorithm for n !
In summary, although recursive algorithms are not all glitter they do
have an important place in algorithm design. For applications whose levels
of recursion do not overflow the run-time stack and are not time critical,
373

recursive algorithms offer concise and elegant solutions to some rather
difficult problems (e.g., the Towers of Hanoi, tree traversal discussed in
Chapter 7 , and sorting algorithms discussed in Chapter 8 ). In addition, the
inclusion of dynamic programming into many recursive algorithms can
eliminate the speed and stack overflow problems inherent in recursive
solutions.
As an example of the use of dynamic programming in recursive
methods, consider the code shown in Figure 6.15 that calculates and
outputs the first 45 terms of the Fibonacci sequence. Each term of the
series is calculated on Line 5 using an invocation to the method fibonacci
defined on Lines 9–17. The program takes 45 seconds to execute when run
on an AMD Athlon XP 3000+ processor with a 2.17 GHz clock. When
recoded using the techniques of dynamic programming (see Exercise 21),
it executes in less than one second on the same processor.
6.5 Backtracking, an Application of Recursion
Backtracking is a problem solving technique that can be used to
resolve a class of problems in which the objective is to proceed to a goal
from a starting point by successively making n correct choices from a
finite selection of choices. The backtracking algorithm either determines a
set of n correct choices that achieves the goal, or it determines that the goal
is unattainable. Typical goals are to reach the exit of a maze from a given
starting point, Sudoku puzzle solutions, to place eight queens on a
checkerboard such that no queen can capture another queen (the Queens
Eight problem), or to move a knight from a given starting point around a
checkerboard such that it moves to every square on the board once, and
only once (the Knights Tour problem).
374

Figure 6.15 A Program to Output the First 45 Terms of the Fibonacci
Sequence using a Recursive Method to Calculate the Terms
The algorithm uses a trial-and-error technique to decide the n correct
choices that attain the goal. At each of the n decision points, it selects or
tries a choice from among the choices it has not yet tried at that decision
point, and then it proceeds to the next decision point and repeats the
process. If the choice at a decision point does not lead to the goal, another
choice is tried at that decision point. When the choices at a decision point
are exhausted, the algorithm backs up (thus the name backtracking )
through the previous decision points until it finds a decision point with an
untried choice. That choice is substituted for the backtracked decision
point’s previous choice, and the algorithm again proceeds to the next
decision point. If the algorithm backs up to the first decision point and
there are no untried choices, it concludes that the goal is unattainable.
The decision choices can be depicted as shown in Figure 6.16 , which
is referred to as a decision tree . The problem’s starting point, S, is
represented by the top circle (called the root of this inverted tree) with
each level of circles in the tree representing a decision point. Referring to
Figure 6.16 , A and B (called the root’s children ) represent the set of
choices for the first decision point, the choices available at the starting
point. A’s children, C, D, and E, represent the choices available if A is
chosen at the first decision point (the starting point), and B’s children, F
and G, represent the choices available if B is chosen at the first decision
point. Generally speaking, the children of a node represent the choices for
the next decision point assuming their parent (e.g., A or B) was chosen at
375

a previous decision point. As decisions are made by the backtracking
algorithm, it proceeds down the levels of the decision tree. If none of the
children of a parent chosen at a given level lead to the goal, the algorithm
backtracks up to the previous level and makes an alternate decision at that
level (here comes the “r” word), recursively. Assuming that the choices
depicted at each level of the tree shown in Figure 6.16 were selected in a
left to right order, the choices made by the backtracking algorithm to
arrive at H would be A, C, backtrack, D, backtrack, E, backtrack,
backtrack, B, F, H.
376

Figure 6.16 Depiction of a Decision Tree
For example, consider the problem of traveling along the squares of a
maze from a given starting square to a goal square. The set of choices for
each move are to proceed straight, turn left, or turn right. However, for the
maze depicted in Figure 6.17 not all of these moves are valid for every
square. Although all three choices are valid for the square in row 0,
column 1 (denoted 0,1), only two of them are valid for the square 1,1 (i.e.,
to proceed straight or to turn left). Turning right when at square 1,1 is not a
valid choice. After entering any of the squares in column 0, there are no
valid choices. The only option after entering any of these squares would be
to backtrack.
Figure 6.17 A Nine Cell Maze
377

Figure 6.18 The Decision Tree for the Valid Next Move Choices of
the Maze Shown in Figure 6.17
The decision tree shown in Figure 6.18 represents all the valid
choices for the squares of the maze depicted in Figure 6.17 when it is
entered at square 0,1. The placement of the nodes in the tree assumes the
choices are selected in a left to right order, and that proceeding forward is
always considered first, then a left turn, and then a right turn. If the goal
was to arrive at square 1,2, the backtracking algorithm would traverse the
maze as follows: 0,1; 1,1; 2,1; 2,0; backtrack to 2,1; backtrack to 1,1; 1,0;
backtrack to 1,1; backtrack to 0,1; 0,0; backtrack to 0,1; 0,2; 1,2.
At each point in the decision tree we encounter a new problem similar
to the original problem—the problem of making correct choices in order to
reach a goal. However, the new problem is one choice closer to the goal
because we have just made a choice at the previous decision point.
Considering this, and also considering the goal to be our end point or base
case, most often a backtracking algorithm is coded recursively. The
reduced problem is considered to be the problem of making the remaining
correct choices.
378

Figure 6.19 The First Four Levels of the Decision Tree for the Valid
and Invalid Next Move Choices of the Maze Shown in Figure 6
Before presenting a generalized version of the backtracking algorithm
that can be adapted to a variety of problems, we should discuss one more
point about decision trees. The two decision trees presented previously in
this chapter contained only valid choices. For example, after moving onto
square 1,1 from square 0,1 (in the maze depicted in Figure 6.17 ), moving
right is not an option, so square 1,2 was not included as a child of square
1,1 in the decision tree. Generally speaking, it is usually better to include
all choices in the decision tree, and allow the backtracking algorithm to
determine the validity of each choice because it tends to generalize our
algorithm. This approach will allow us to rearrange the walls of the maze
and still use the same decision tree. The first four levels of the expanded
version of the maze’s decision tree are presented in Figure 6.19 .
6.5.1 A Generalized Backtracking Algorithm
A generalized recursive version of the backtracking algorithm, named
makeNextDecision, is given below. It is generalized in that several of its
lines simply indicate what is to be done because the decision as to how it is
to be done is particular to a specific problem. The algorithm is passed the
choice made at the last decision point so that only that choice’s children (in
the decision tree) will be considered during the next decision. The
selection from among these choices is performed on Line 4 by a method
named nextChoice that returns the next choice at a decision point from
among the decision point’s untried choices.
379

Generalized Recursive Backtracking Algorithm
The algorithm begins by initializing the Boolean variable atGoal to
false and the variable choiceNumber to zero. Line 3 begins a loop that
works its way sequentially through the choices at a decision point looking
for one that eventually leads to the goal. Each pass through the loop the
variable choiceNumber is incremented (Line 16), and the loop continues
while the goal has not been reached (atGoal == false ) and then there are
more choices to consider at this decision point (choiceNumber < numberOfChoices). The variable choiceNumber is passed to the method nextChoice on Line 4 to request that the next choice (initially the 0th choice) be determined and returned. The method nextChoice is assumed to exist and is particular to the problem being solved. (Lines 5, 6, 7, and 12 are also particular to the problem being solved.) Normally the choice is not only a function of the choice number, but also a function of the previous decision, which is also passed to the method as the second argument. At the beginning of the maze problem depicted in Figures 6.17 and 6.19 , the previous decision would be to start at square 0,1, and 1,1 would be returned as the next choice (since, as specified in the decision tree, the move order is forward, left, and then right). 380 The code of the Boolean condition on Line 5 would be particular to an application, and in most cases coded as a separate function. When Line 5 determines that a choice is not valid, the body of the if statement (Lines 6–15) does not execute and the choice is ignored. Then, Line 16 increments the choiceNumber, and the next choice is considered. Line 6 and Line 12 are also particular to a specific problem. When a choice is valid, Line 6 makes a record of the choice. For the maze problem, the choice could be recorded by storing the chosen square's row and column number in an array, and unrecorded by eliminating them from the array. The choice is recorded so that if the goal is reached, the sequence of correct choices can be displayed. Another reason for recording the choice is that, for some problems, the determination of a valid choice performed on Line 5 depends on the choices made at previous decision points. For example, if a square on the Knight's Tour checkerboard was visited by the knight as a result of a choice made at a previous decision point, that square is no longer a valid choice for subsequent decision points. Line 12 eliminates the choice from the record of choices if the next decision point results in a backtrack. Line 7 is the base case, i.e., the current choice has achieved the goal. Its Boolean condition is also particular to an application. In the case of the maze problem, the goal is reached when the choice is the goal square. In the Queens Eight problem the goal has been reached when eight queens have been placed on the board. If the goal has not been reached, the algorithm is invoked recursively on Line 10 (the reduced problem) to continue the decision process. If the method returns false , indicating that the goal could not be reached using the current choice, the choice is unrecorded (Line 12), and the choice number is incremented (Line 16) so that the next choice at this decision point can be considered. Adapting the generalized backtracking algorithm to a particular application requires a bit of creativity. However, the amount of creativity can be reduced by methodizing the adaptation process. 6.5.2 Algorithm Adaptation Methodology As previously mentioned, Lines 4, 5, 6, 7, and 12 are particular to a given problem. The scheme used to represent the choices is also particular to a given problem. Therefore, when applying the backtracking algorithm to a problem we have to decide how to represent the choices, how to determine if the goal has been reached (for Line 7 of the algorithm), how 381 to record and unrecorded a choice (for Lines 6 and 12 of the algorithm), how to determine the next choice (for Line 4 of the algorithm), and how to determine if a choice is valid (for Line 5 of the algorithm). In the spirit of divide and conquer, once we have made decisions on these issues we code a data structure to represent the choices, and code five methods that can be invoked on Lines 4, 5, 6, 7, and 12 of the algorithm. Next, the algorithm itself is coded and, finally, an application program is written to perform initialization, record the starting point, and invoke the backtracking algorithm to determine the choices that lead to the goal. These steps are summarized in the following methodology. Methodology for Applying the Generalized Backtracking Algorithm 1. Decide how to represent the n choices, and declare a data structure that will store the n choices. 2. Decide how to determine if the goal has been reached, and write a method that can be invoked on Line 7 of the algorithm. 3. Decide how to record and unrecord a choice, and write a method that can be invoked on Lines 6 and 12 of the algorithm. 4. Decide how to determine the next choice, and write methods that can be invoked on Line 4 of the algorithm. 5. Decide how to determine if a choice is valid, and write a method that can be invoked on Line 5 of the algorithm. 6. Write the generalized backtracking algorithm, and include invocations to the methods developed in the previous steps on Lines 4, 5, 6, 7, and 12. 7. Write an application that performs initialization, records the starting point, invokes the generalized backtracking method, and then outputs the choices that lead to the goal. We will now illustrate the application of the methodology by using it to code a solution to the Knights Tour problem. The Knights Tour, an Adaptation Example The Knights Tour problem begins by placing a knight on a square of a checkerboard. Then, using only valid knight moves, we are to determine a sequence of moves during which the knight arrives at each square of the checkerboard once, and only once. The shaded boxes in Figure 6.20 show the eight valid moves for a knight, as per the rules of chess, assuming that the knight is initially positioned in the box K. (The rules of chess also 382 demand, of course, that after the move, the knight is on the checkerboard.) Figure 6.20 The Eight Valid Moves for the Knight Positioned at Square K The eight moves have been arbitrarily assigned a sequential choice number (0 through 7) as indicated by the digit in the shaded boxes of Figure 6.20 . When the moves are chosen in this order (i.e., move to the right two boxes and up one box as a first choice, move to the right one box and up two boxes as a second choice, etc.), the sequence of valid moves for a tour beginning in row 4 and column 0 of a 5 × 5 checkerboard is depicted in the top part of Figure 6.21 . The numbers in the squares of the checkerboard represent the move number during which the knight landed on the squares (1 being the starting point, 25 being the end point). The bottom part of the figure presents the number of times the algorithm backtracked to each of the 25 decision points. For example, the 4 in square 4,3 of the lower part of the figure indicates that the decision point that determined the eighth move (see square 4,3 of the upper part of the figure) was backtracked to four times during the solution. Similarly, square 3,1 in the figure indicates that the decision point that determined the ninth move was backtracked to eight times. 383 Figure 6.21 A Valid Knights Tour Beginning at Row 4, Column 0 and the Amount of Backtracking Performed The five recorded choices for the eighth move (four of which are unrecorded as a result of the four backtracks to the eighth decision point), and the nine recorded choices for the ninth move (eight of which are unrecorded), are depicted in Figure 6.22 . Each of the five checkerboards in the figure depict one of the five recorded choices for the eighth move, denoted as 8/1 (the first recorded choice) on the top checkerboard, 8/2 (the second recorded choice) on the second from the top checkerboard, and 8/3, 8/4, and 8/5 on subsequent checkerboards. Also shown in each checkerboard are the positions chosen for the first seven moves, denoted in the squares as 1 to 7. The seventh move positions the knight on square 2,2. The first choice for move 8 is to move the knight from that square to square 1,4, as shown in the top of the figure. From that square, the first valid choice for move 9 is square 0,2. However, all subsequent combinations of choices for the tenth, eleventh, twelfth,…twenty-fifth moves do not produce a solution, which results in the first backtrack and the move to square 0,2 is 384 unrecorded. The only other valid choice for the ninth move from square 1,4 is to square 3,3. This also does not lead to a solution, resulting in the second backtrack to the ninth decision point. Since there are no other valid choices for the ninth move from square 1,4 the first backtrack to the eighth decision point is performed, and the move to square 1,4 is unrecorded. Referring to the second checkerboard, the next valid choice for the eighth move (to square 0,1 from square 2,2) is recorded, and then the knight is moved from that square to the first valid choice for the ninth move, square 2,0. (It should be noted at this point that the square 0,3 is not a valid choice for the eighth move because the square had been visited on the fourth move.) Moving to square 2,0 or square 1,3 from square 0,1 (the only valid choices for the ninth move) do not result in a solution. After backtracking to the eighth decision point, the remaining three valid choices for the eighth move are tried. As indicated on the checkerboard at the bottom of the figure, the fifth, and last, valid choice for the eighth move results in a solution. If it had not, the algorithm would have backtracked to the seventh decision point, the move to square 2,2 would have been unrecorded, and the only remaining valid choice for the seventh move, square 4,2, would have been tried to see if it led to a solution. It is suggested that an understanding of the move's choices and the backtracking illustrated in Figure 6.22 be mastered before proceeding to the application of the methodology to this problem. In Step 1 of our methodology we determine how we will represent the eight move choices. To accomplish a move we simply add an increment to the knight's current row and column numbers. For example, if the knight is currently at row 4 and column 0 (and the row and column numbers are assigned as depicted in Figure 6.21 ), move choice 0 would be accomplished by adding a −1 to the row number and a +2 to the column number. Thus, to store the choices we simply store the row and column increments for the eight moves. The simplest data structure to store the increments is a two-dimensional array of integers (named rowColIncrement) composed of eight rows and two columns, and then use the choice number as the index into the array's row. The scheme is depicted in Figure 6.23 where the column increment is stored in column 0 of the array, and the row increment is stored in column 1 of the array. The row assignment of the increments in the array reflects the move order depicted in Figure 6.20 . 385 Figure 6.22 The Choices Recorded for Moves Eight and Nine during the Four Backtracks to Decision Eight 386 Figure 6.23 The Representation of the Eight Valid Knight Move Increments Depicted in Figure 6.20 The Java code to declare and initialize the array is: The next step (Step 2) of the methodology is to decide how to determine if the goal has been reached. For a checkerboard with n rows and n columns, the goal has been reached when we make n 2 successful moves (which include the starting square). Assuming the number of rows and columns are stored in the variables nRows and nCols, and that the number of moves made is stored in the variable thisMoveNumber, the goal is reached when thisMoveNumber = nRows * nCols. This step is concluded with the coding of the method goalHasBeenReached given below. It will be invoked by Line 7 of the generalized backtracking algorithm. In Step 3 of our methodology, we decide how to record and unrecord 387 a choice. To record the move we will “mark” the square of the checkerboard the knight lands on. In order to display the sequence of the moves, we will mark the square with the move number as depicted in the upper part of Figure 6.21 . We will represent the checkerboard as a two- dimensional array of integers named board, consisting of nRows and nCols defined as Assuming that an unvisited square is marked with a zero, this step concludes with the coding of the two methods recordThisDecisionChoice and unrecordThisDecisionChoice given below. These methods will be invoked from Lines 6 and 12 of the generalized backtracking algorithm, respectively. In Step 4 of our methodology, we decide how to determine the next choice for a move when the current move does not lead to the goal. The simplest thing to do is to select the moves sequentially in the order given in Figure 6.20 , which is consistent with the fact that every time through the while loop of the generalized backtracking algorithm, the variable choiceNumber is incremented by one. The row and column of the next move will simply be determined by adding the row and column increments stored in the array rowColIncrement (depicted in Figure 6.23 ) at row index choiceNumber, to the row and column numbers that represent the knight's current location (the location we are moving from). The two methods nextChoiceRow and nextChoiceColumn, which would be invoked by Line 4 of the generalized backtracking algorithm, accomplish this. 388 In Step 5 of our methodology, we decide how to determine if a choice is valid. Since our moves will be fetched from the array depicted in Figure 6.23 that represents valid knight moves (see Figure 6.20 ), the only way they could be invalid is if the move places the knight off the board or on a square already visited. A square whose row and column numbers are stored in row and col is on the board if and it has not been visited if The method thisDecisionChoiceIsValid, which follows, and would be invoked from Line 5 of the generalized backtracking algorithm, tests these conditions. In Step 6 of our methodology, we code the generalized backtracking algorithm, the method makeNextDecision, incorporating into it the methods developed in the first five steps of our methodology. Before coding it, we have to decide on the parameter(s) of the method that will represent the previous (last) decision choice. In the Knights Tour problem that would be the row and column number of the knight's current position. We will also include the last move number as a third parameter because it is incremented and then used in the method recordThisDecisionChoice (to keep track of where the knight has been). It is also used in the goalHasBeenReached method (to determine if the solution is complete). Thus, the signature of the method becomes 389 Having defined the method's parameters, we complete Step 6 of the methodology by coding the method. Lines 31–55 of Figure 6.24 presents the Java implementation. The invocations to the methods developed in the previous steps of our methodology are highlighted on Lines 39–43 and 48. In addition, Lines 33 and 34 have been added to perform initialization particular to this problem. Finally, in Step 7 we complete our solution by writing an application that performs some initialization, records the starting point, invokes the generalized backtracking method, and then outputs the choices that lead to the goal. The application is presented in Figure 6.24 as Lines 1–29, which includes the declaration of the data structures previously discussed (Lines 4–5) and the initialization of the valid knight moves (Lines 9–16). The methods developed in Steps 2-5 of the methodology are included in Figure 6.24 as Lines 57–92. The Queens Eight Problem Table 6.9 summarizes the decisions made during Steps 2-5 of the methodology to adapt the generalized backtracking algorithm to the Knights Tour problem. In addition, it presents the analogous decisions that adapt the algorithm to the Queens Eight problem, assuming that each queen will be placed in a different column and therefore the backtracking algorithm is only used to determine queens' row numbers. The coding of the solution to this problem is left as an exercise for the student. (The entire solution of the maze problem, the decisions made during Steps 2-5 of the algorithm adaptation methodology, and the coding of the solution, is left as an exercise for the student.) 390 391 Figure 6.24 The Solution to the Knights Tour Problem 392 When coding the method thisDecisionChoiceIsValid for the Queens Eight problem (see the last column of the Table 6.9 ) we make use of two facts regarding the diagonals of a checkerboard: 1. For each square on an upper left to lower right running diagonal, the difference of the square's row and column numbers is a constant (different for each diagonal). For example, on the diagonal from 1,0 to 4,3 the difference is 1. 2. For each square on a lower left to upper right running diagonal, the sum of the square's row and column numbers is a constant (different for each diagonal). For example, on the diagonal from 3,0 to 0,3 the sum is 3. These sums and differences can be used to index into two one- dimensional arrays used to record which diagonals have queens on them. For example, if a queen had been placed on square 2,1 then element 1 of one of the arrays would be marked occupied, as would element 3 of the other array. A third one-dimensional array can be used to keep track of the rows that have queens on them using the queen's row number as an index into this array. EXERCISES Knowledge Exercises 1. State the definition of recursion. 393 2. True or false: a. All algorithms can be stated recursively. b. Some algorithms can only be stated recursively. c. For most people, recursive algorithms are more difficult to discover and understand than nonrecursive algorithms. d. All recursive algorithms must have an escape clause. e. Recursive algorithms, if not properly coded, can terminate with a stack overflow error; if true, why? 3. The menu of Juan's Taco Tower states “when ordering a Sombrero Meal you get a taco, and a soda, and your choice of an ice cream sundae or a Sombrero Meal.” a. What part of the Sombrero Meal's description is recursive? b. What is the base case (escape clause) in the description of a Sombrero Meal? c. Can I order a Sombrero Meal consisting of: 1. 1 taco, 1 soda, and 1 ice cream sundae? 2. 2 tacos, 1 soda, and 1 ice cream sundae? 3. 2 tacos, 3 sodas, and 1 ice cream sundae? 4. 1 taco and 1 soda? 5. 1 taco, 1 soda, and 2 ice cream sundaes? 4. Define: a. Base case b. Reduced problem c. General solution 5. Give the generic flowchart for a recursive method that has one base case. 6. Give the four steps in the methodized approach to formulating a recursive algorithm. 7. Give an advantage of a recursive algorithm over an iterative algorithm. 8. Give two advantages of an iterative algorithm over a recursive algorithm. 9. A recursive method is used to calculate 64 . The base case is 60 . 394 a. What is the first value returned from the recursive invocations? b. What is the second value returned from the recursive invocations? 10. Draw a figure similar to Figure 6.6 that shows the values calculated by a recursive method invoked to determine the value of 53 . 11. Give the base case, reduced problem, and general solution of the recursive algorithm for: a. n ! b. x y c. The product of two integers a and b . d. The sum of the integers from a to b , a > b .
e. Outputting an array of characters, c , given the starting and ending
indices.
f. Binary search of an array to locate the value aKey.
12. Give the signatures of the methods that implement the recursive
algorithms of Exercises 11(a)-(f).
13. Give the pseudocode of the methods that implement the recursive
algorithms of Exercises 11(a. through 11(f).
Programming Exercises
14. Code a recursive solution for Exercise 11(d), and provide a driver
program to test your solution.
15. Code a recursive solution for Exercise 11(e), and provide a driver
program to test your solution.
16. Code a recursive solution for Exercise 11(f), and provide a driver
program to test your solution.
17. Code a recursive solution for Exercise 11(c), and provide a driver
program to test your solution.
18. Code a recursive method that calculates the greatest common
denominator of two integers and provide a driver program that
demonstrates that it functions properly.
19. Apply the techniques of dynamic programming to improve the
performance of the method coded in Exercise 14, and provide a driver
program that demonstrates that it is faster than the nondynamic version
of the code in Exercise 14.
395

20. Code a program that outputs the first 45 terms in the Fibonacci
sequence by invoking a non-dynamic recursive method that calculates
the n th term of the series.
21. Apply the techniques of dynamic programming to improve the
performance of the method fibonacci that is presented in Figure 6.15
and provide a driver program that demonstrates that it functions
properly.
22. Determine the value of n such that the dynamic version of the
recursive algorithm for x y is 20 seconds faster than the nondynamic
version. Include a description of the platform (CPU make and model,
clock speed, amount of RAM memory, and the operating system).
23. Your pet mouse Marty can find his way out of a maze, unassisted,
much to the delight of his fans. The maze is divided into a two-
dimensional grid of six rows and six columns of black and white tiles.
Tiles are identified by their row number, followed by their column
number. Rows and columns are numbered sequentially from 0 to 5
with box 0,0 in the upper left corner. Black tiles cannot be stepped on
so Marty must proceed along the white tiles of the maze to reach the
exit. Marty always enters the maze from below, stepping onto tile 5,1
which is always a white tile. From there he can advance to any adjacent
white tile, but cannot travel diagonally from tile to tile. If possible,
Marty will always continue straight ahead. Being left handed his
second choice is always to turn left, then right, and finally (if he has
reached a dead end), his last and only remaining choice is to retreat
backward. The maze is modeled by a 6 × 6 array of integers with a
black tile represented as a 1, and a white tile represented as a 0. Write a
program that invokes a recursive backtracking method and outputs the
row and column numbers of the tiles Marty steps on as he finds his
way through the maze. Include all the boxes backtracked onto because
Marty chose a path that ended in a dead end.
24. Each square of the following 10 × 10 checkerboard has a toll
associated with it that must be paid when you enter the square. You
wish to travel from the bottom-most row to the topmost row and
minimize the total of the tolls along the way. Write a program to output
the row and column numbers of a route that minimizes the tolls. When
making a move, the row number must increase by 1 and the column
number can change by −1, 0, or +1. Tolls range from 0 to 9, and row 1
and column 1 is the lower leftmost square of the checkerboard. The
program should invoke a dynamic recursive method, minToll, to
396

determine the minimum toll to get to each of the top row cells, i = 10,
of the checkerboard calculated as
minToll[i ][j ] = toll[i ][j ] + minimum of (minToll[i − 1][j − 1],
minToll[i − 1][j ], minToll[i − 1][j + 1])
25. There is a kind of puzzle formatted as a 9 by 9 grid of cells
grouped into nine 3 by 3 boxes. To solve this puzzle the numbers 1
through 9 must appear once, and only once, in each row, column, and 3
by 3 box. The puzzle begins with several cells filled in. Use the
generalized backtracking algorithm to solve this kind of puzzle.
1 Iterative algorithms are algorithms that involve loops.
2 Recursive methods can operate on an object’s member data, in which case the method
is coded as a nonstatic method.
3 Line 4 of Figure 6.3 would become: FactorialTrace.nFactorialTrace(n);
5 Or the locations of the shared information if the parameters are nonprimitive types
(i.e., objects).
6 When it is not, many compilers give the programmer the option of replacing the
invocation with the actual code of the method. The C++ keyword to accomplish this is
“inline.”
397

CHAPTER 7
Trees
OBJECTIVES
The objectives of this chapter are to familiarize the student with the
features, implementation, and uses of tree structures. More specifically, the
student will be able to
Understand the standard graphics used to depict trees, the
terminology of trees, and the mathematics of binary trees.
Understand the linked and array-based memory models
programmers use to represent binary tree structures, and the advantages
and disadvantages of these representations.
Understand the classic binary search tree structure and its operation
algorithms, including their use of a method that locates a node in the
tree.
Explain the advantages and disadvantages of binary search tree
structures, and be able to quantify their performance.
Implement a fully encapsulated version of a binary search tree.
Understand the standard tree traversal techniques (including inorder,
preorder, and postorder), the use of these techniques, and be able to
implement them recursively.
Understand the extension of the binary search tree structure to an
AVL tree and a Red-Black tree, and understand the techniques these
structures use to keep the tree balanced.
Understand the topic of recursion more fully by examining several
recursive tree algorithms.
Develop an application that declares a structure in Java’s API
TreeMap class, understand the advantages and disadvantages of the
class, and operate on the structure using the class’ operation methods.
398

7.1 Trees
In the previous chapters we have studied three types of data structures
that allow access in the key field mode: array-based structures, linked lists,
and hashed structures. Each of these structures has unique attributes that
make them attractive in certain applications. The hashed structures are, by
far, the fastest structures when we consider all four basic operations. The
array-based structures are the simplest to implement and the speed of two
of their operations (Insert for the unsorted version and Fetch for the sorted
version) match the hashed structures. The linked list structures do not
require contiguous memory and, therefore, are the most efficient structures
to expand. In addition, the speed of the Insert operation for unsorted linked
list structures approaches the speed of the hashed structures.
Tree structures, like linked lists, do not require contiguous memory,
making them just as efficient to expand. In addition, when properly
implemented, the speed of these structures greatly exceeds that of the
array-based and linked structures and approaches the overall speed of the
hashed structures. From a density viewpoint, they require higher node
widths than the other structures to reach acceptable densities. However,
they are the only structure we will study that can easily model data in a
hierarchical (as opposed to a linear) form and can process nodes in sorted
order without sacrificing operation speed. These attributes make them
attractive for many applications.
7.1.1 Graphics and Terminology of Trees
Before proceeding to a discussion of the algorithms used to operate
on trees, it is necessary to gain an understanding of the depictions and
terminology of trees. The standard depiction of a tree is shown in Figure
7.1 , which presents a tree containing eight nodes. Each node in the tree is
represented by a circle with lines emanating from it. These lines connect a
node to other nodes in the structure that come after it, and are analogous to
the arrows in the standard depiction of a linked list. The annotation inside
the circle normally refers to the contents of the key field of the node. As
was the case with linked lists, these graphical depictions will be used
extensively in this chapter to develop the algorithms used to operate on
trees and to facilitate our understanding of them.
We will now turn our attention to the terminology of trees, beginning
our discussion with the term directed tree . We will see that most tree
399

terminology employs two analogies; the terms root, leaf, forest, and, of
course, tree are terms borrowed from nature, and the terms parent, child,
and grandparent are terms borrowed from the traditional family unit.
Figure 7.1 Standard Depiction of a Tree
Directed (or General) Tree
A directed (or general tree), is a structure in which
• There is a designated unique first node,
• Each node in the structure, except for the unique first node, has one
and only one node before it, and
• Each node in the structure has 0, 1, 2, 3,…nodes after it.
Consider the tree depicted in Figure 7.1 . If we designate node A to be
the unique first node, then it is a directed tree because
• There is a unique first node, A, which has no node before it,
• Nodes B, C, and D are preceded only by node A; node E is preceded
only by node B; node F is preceded only by node D; and nodes G and H
are preceded only by node E, and
• Nodes C, F, G, and H have 0 nodes after them; nodes B and D have
one node after them; node E has two nodes after it; and node A has three
nodes after it.
Root Node
A root node is the unique first node in the structure. The root node
does not have a node before it. Each tree has one, and only one, root node
that is always drawn at the top of the standard depiction of a tree. Node A
is the root node of the tree presented in Figure 7.1 .
Leaf Nodes
Leaf nodes are nodes in the tree that have no (0) nodes after them.
400

Nodes C, F, G, and H in Figure 7.1 are leaf nodes.
Parent (or Father) Nodes
An analogy to a family is often used to describe the relationship
between the nodes of a tree. A parent node is a node’s unique predecessor.
Referring to the tree shown in Figure 7.1 , node A is the parent of nodes B,
C, and D; node B is the parent of node E; node D is the parent of node F;
and node E is the parent of nodes G and H. A consequence of this
definition is that all nodes in a tree are parent nodes except for the leaf
nodes. Sometimes in the literature, a patriarchal analogy is used in which
case the term father is substituted for the term parent.
Child (or Son) Nodes
A child node is a node that comes directly after a node in a tree.
Referring to the tree presented in Figure 7.1 , nodes B, C, and D are the
children of node A; node E is the child of node B; node F is the child of
node D; and nodes G and H are the children of node E. A consequence of
this definition is that all nodes in a tree are child nodes except for the root
node, and when we also consider the definition of a parent node we can
conclude that all nodes in a tree are both child and parent nodes except for
the leaf nodes and the root node. When the patriarchal analogy is used, a
child is referred to as a son .
Grandchild Nodes, Great Grandchild Nodes, etc.
A grandchild of a node is a child of a child of a node. Referring to the
tree presented in Figure 7.1 , nodes E and F are the grandchildren of node
A, and nodes G and H are the grandchildren of node B. Extending this
genealogy, nodes G and H are also the great grandchildren of node A.
Grandparent Nodes, Great Grandparent Nodes, etc.
A grandparent of a node is the parent of the parent of a node.
Referring to the tree presented in Figure 7.1 , node A is the grandparent of
nodes E and F, and node B is the grandparent of nodes G and H. Extending
this genealogy, node A is the great grandparent of nodes G and H.
Outdegree of a Node
The outdegree of a node is the number of children it has. Referring to
the tree presented in Figure 7.1 , node A has outdegree three, node E has
401

outdegree two, node D has outdegree one, and nodes C, F, G, and H have
outdegree zero. Leaf nodes always have outdegree zero.
Figure 7.2 The Level Numbers of a Tree With Four Levels
Outdegree of a Tree
The outdegree of a tree is equal to the largest outdegree of any of the
nodes in the tree. The tree presented in Figure 7.1 has outdegree three.
Levels of a Tree
The levels of a tree are a sequential numbering assigned to the
descendents of the root node, with the root node at level 0. The root node’s
children are at level 1, the root node’s grandchildren are at level 2, etc.
Since the level numbers begin at 0, the number of levels of a tree is always
one more than the highest level number in the tree. Figure 7.2 presents the
levels of the tree depicted in Figure 7.1 , which has a total of four levels.
When we begin our study of trees, this level numbering system can be
a bit confusing since the lowest level number is at the top of the standard
tree depiction, and vice versa.
Visiting a Node
We visit a node in a tree by first locating the node, and then
performing some operation on it. Typical operations performed on a node
are to fetch the contents of the node, output the node, determine if it has a
child, etc.
Traversing a Tree
Traversing a tree is the process of visiting each node in the tree once,
and only once. For example, if the operation to be performed on the nodes
of the tree presented in Figure 7.1 was to output the key values of each
402

node, then a valid traversal would output the keys in the order: A, B, C, D,
E, F, G, and finally, H. An equally valid output traversal would be to
output the nodes in the order: A, B, E, G, H, C, D, and finally, F. Although
it is true that many traversal orders are possible, three of these possibilities
(named preorder , inorder , and postorder traversals) are most often used.
These popular traversal orders will be discussed later in this chapter.
Outputting the nodes in the order A, B, E, G, H, A, C, D, and finally, F
would be an invalid traversal since node A is visited twice.
7.2 Binary Trees
Most of the material in the remainder of this chapter deals with binary
trees. They are a subset of directed trees that occupy a very important
place in the field of computer science. A binary tree , by definition, is a
directed tree with maximum outdegree two. This implies that each node in
a binary tree can have a maximum of two children. That is, nodes in a
binary tree can only be leaf nodes, have one child, or have two children.
The tree presented in Figure 7.1 is not a binary tree because the root node
has three children. Figure 7.3 presents six valid binary trees. Before
beginning our study of the algorithms associated with binary trees, we will
discuss some the terminology and mathematics particular to binary trees.
7.2.1 Terminology
Left Child (Left Son) of a Node
When the standard tree graphic is used to depict a binary tree, a
node’s left child is the child of the node to the viewer’s left. For example, in
tree f presented in Figure 7.3 , node B is A’s left child, and node G is C’s
left child. When the patriarchal analogy is used, a left child is referred to as
a left son .
403

Figure 7.3 Valid Binary Trees
Right Child (Right Son) of a Node
When the standard tree graphic is used to depict a binary tree, a
node’s right child is the child of the node to the viewer’s right. For
example, in tree f presented in Figure 7.3 , node C is A’s right child, and
node G is C’s right child. Again, when the patriarchal analogy is used, a
right child is referred to as a right son .
Left Subtree of a Node
The left subtree of a node is the tree whose root is the left child of the
node. Referring to tree a in Figure 7.3 , the left subtree of node B is the
empty tree (containing no nodes), while the left subtree of node A contains
the nodes B, E, G, and H, with node B being the root of the subtree.
Right Subtree of a Node
The right subtree of a node is the tree whose root is the right child of
the node. Referring to tree a in Figure 7.3 , the right subtree of node B is
the tree containing the nodes E, G, and H, with E being the root of the
subtree.
404

Complete
A binary tree is complete if all the levels of the tree are fully
populated. Thus, except for the leaf nodes, all the nodes in a complete tree
have two children. Tree f in Figure 7.3 is the only tree in the figure that is
complete because at least one nonleaf node in all of the other trees has less
than two children.
Balanced
A tree is balanced if all the levels of the tree below the highest level
are fully populated. This implies that all complete trees are balanced. Trees
b , e , and f in Figure 7.3 are examples of balanced trees.
Complete Left (or Right)
A tree is complete left if it is balanced and all the nodes at the highest
level are on the left side of the level. Tree e presented in Figure 7.3 is
complete left. A tree is complete right if it is balanced and all the nodes at
the highest level are on the right side of the level. Tree b presented in
Figure 7.3 is complete right.
This concludes our discussion of the terminology of binary trees. We
will now turn our attention to the mathematics of binary trees, which is
used later in this chapter when we evaluate the performance of these
structures.
7.2.2 Mathematics
Maximum Number of Nodes at Level l of a Binary Tree
Consider tree f in Figure 7.3 . All of its levels are fully populated; that
is, they contain the maximum number of nodes that could exist at these
levels. Counting the number of nodes at each level, we find that for any
binary tree there is a maximum of
1 node at level l = 0,
2 nodes at level l = 1, and
4 nodes at level l = 2.
Extrapolating these observations (or alternately fully populating
higher levels of the tree and counting the nodes) we find that there would
be a maximum of
8 nodes at level l = 3,
405

16 nodes at level l = 4, and
32 nodes at level l = 5.
It is now easy to deduce the functional relationship between the
maximum number of nodes at a level, nl max , and the level number, l , to
be:
Maximum Number of Nodes, nl max , at Level l of a Binary Tree
Thus, the maximum number of nodes at level 15 of a binary tree is
215 , or 32,768 nodes.
Maximum Number of Nodes in a Tree with L Levels
Atree with L levels will contain a maximum number of nodes when
all of its levels are fully populated; that is, the tree is complete. Figure 7.4
presents three complete trees comprised of 1, 2, and 3 levels.
Table 7.1 tabulates the number of levels in each of these trees (L ) vs.
the total number of nodes they contain (nL max ) extrapolated to include
complete trees with 4 and 5 levels. (This extrapolation can be verified by
drawing complete 4 and 5 level trees and counting the total number of
nodes.)
Figure 7.4 Complete Binary Trees with 1, 2, and 3 Levels
Table 7.1
Maximum Number of Nodes, nL max , in a Binary Tree With L
Levels
L nL max
1 1
2 3
3 7
406

4 15
5 31
From the data presented in Table 7.1 , we can deduce the functional
relationship between nL max and L to be
Maximum Number of Nodes, nL max , a Binary Tree with L Levels
which is also the number of nodes in a complete binary tree with L
levels. Thus, a complete binary tree with 16 levels would have a total of
216 − 1 = 65,535 nodes with slightly over half these nodes (32,768 = 215 )
at the highest level (level l = 15).
Minimum Number of Levels in a Tree with N Nodes
The number of levels in a tree with N nodes will be a minimum when
the tree is balanced because all levels except the lowest level are fully
populated (e.g., compare trees e and c in Figure 7.3 ). Assuming N is such
that the tree is not only balanced but also complete (e.g., tree f in Figure
7.3 ), then N = nL max and using Equation 7.2 we obtain
or
Remembering, from the definition of a logarithm, x y = z implies y =
logx z we can solve the above equation for L to obtain
This is the number of levels in a complete binary tree containing N
nodes.
Number of Levels in a Complete Binary Tree Containing N Nodes
If N is such that the tree is balanced but not complete (N < nL max ), then the highest level of the tree would not be fully populated and Equation 7.3 will not yield an integer value. In this case, in order to include the highest level of the tree in the computed value of L , we use the ceiling of the previous function to determine the minimum number of 407 levels in the tree. Thus, we have Minimum Number of Levels in a Tree Containing N Nodes (which is also the number of levels in a balanced binary tree) For example, a binary tree containing 600 nodes must have at least 10 levels (= ciel(log2 (600 + 1)) = ciel(9.23) = 10), and the highest level of the tree would not be fully populated because log2 (600 + 1) is not an integer. Armed with knowledge of tree graphics, tree terminology, and the mathematics of binary trees, we are now prepared to explore the algorithms used to operate on trees. We will begin with a discussion of a special kind of binary tree, a binary search tree . 7.3 Binary Search Trees Consider the tree shown in Figure 7.5 consisting of six nodes, each containing a one letter key. Let us assume we are to fetch the node with key field P. We begin our search, as we begin all searches for nodes stored in trees, at the root node. The key field of this node is fetched and examined, and our search ends successfully after one memory access. Now suppose we are searching for the node with key field K. Starting at the root node, the first memory access returns the key P, and the search must continue. Now we need to make a decision whether to visit the left child (node N), or right child (node V), of node P. Let us assume we make the correct choice and visit node N next, and then make another correct choice visiting node N's left child next. Under these assumptions we will locate node K after three memory accesses, searching through the nodes in the order P, N, K. Figure 7.5 Six Nodes with One Letter Key Each Stored in a Binary Tree 408 But suppose we make incorrect choices at every decision point. In this case a typical search order could be: P, V, X, N, O, K. Under this “worst case” scenario, six memory accesses are required to locate the node K. Obviously deciding to visit N after P, and then K after N would be a better choice, but we can only be certain of making a correct decision at each decision point if we have some way of knowing that K is in the left subtrees of P and N. Enter the binary search tree . A binary search tree is a binary tree that allows us to always identify the “correct” child (subtree) to visit after an unsuccessful memory access. This is made possible by the manner in which the nodes are arranged in a binary tree as stated in its definition. Binary Search Tree A binary search tree is a binary tree in which the key field value of the root node is greater than the key field values of all of the nodes in the root's left subtree , and less than the key field values of all of the nodes in the root's right subtree . In addition, each subtree in the tree is also a binary search tree. Consider the tree presented in Figure 7.6 whose nodes have integer key values. Under our definition, this tree is a binary search tree. The value of the key field of the tree's root node, 50, is greater than all the keys in its left subtree (40, 35, 47, and 43), and it is also less than all the keys in its right subtree (63, 55, 70, 68, and 80). In addition, all subtrees in the tree are also binary trees, which can be verified by inspecting them. For example, consider the subtree whose root node is 63. This subtree is also a binary search tree because all keys in 63's left subtree (55) are less than 63, and all of the keys in the root's right subtree (70, 68, and 80) are greater than 63. Other examples of binary search trees are trees b and d in Figure 7.3 , the leftmost tree in Figure 7.4 , and the tree shown in Figure 7.5 . Conversely, because of the arrangement of the nodes in the trees depicted in Figure 7.3 , trees a , c , e , and f are not binary search trees, nor are the trees depicted in the middle and right side of Figure 7.4 . 409 Figure 7.6 A Binary Search Tree Figure 7.7 The Progressive Build-up of a Six Node Binary Search Tree The positioning of the nodes in a binary search tree consistent with its definition is performed by the Insert operation. The following five step process accomplishes this. 1. The first node inserted becomes the root node. 2. For any subsequent node, consider the root node to be a root of a subtree, and start at the root of this subtree. 3. Compare the new node's key to the root node of the subtree. 3.1. If the new node's key is smaller , then the new subtree is the root's left subtree. 3.2. Else, the new subtree is the root's right subtree. 410 4. Repeat Step 3 until the new subtree is empty. 5. Insert the node as the root of this empty subtree. As an example, consider six nodes with integer key fields to be inserted into an empty binary search tree: first key 50, then 40, followed by 47, 63, 55, and finally 43. Figure 7.7 depicts the growth of the tree as the nodes are inserted. Since the tree is initially empty, as per Step 1 of the process, 50 becomes the root node (see Figure 7.7 a ). The node with key 40 is to be inserted next. Step 2 of the process tells us to consider the tree whose root is 50 as a subtree. Comparing the root of this subtree to the key value 40 (Step 3 of the process) we find it is less than the root, and the new subtree is therefore the left subtree of 50 (Step 3.1). Since this subtree is empty, Step 5 tells us to make the new node the root of the empty left subtree of 50 (see Figure 7.7 b ). The node with key 47 is to be inserted next. Step 2 of the process tells us to consider the tree whose root is 50 as a subtree. Comparing the root of this subtree to the key value 47 (Step 3 of the process) we find it is less than the root, and the new subtree is therefore the left subtree of 50 (Step 3.1). This subtree's root is 40, and since it is not empty, Step 4 tells us to repeat Step 3. Comparing the root of this subtree to the key value 47 (Step 3 of the process) we find it is greater than the root, and the new subtree is therefore the right subtree of 40 (Step 3.2). Since this subtree is empty, Step 5 tells us to make the new node the root of the empty right subtree of 40 (see Figure 7.7 c ). The node with key 63 is to be inserted next. Step 2 of the process tells us to consider the tree whose root is 50 as a subtree. Comparing the root of this subtree to the key value 63 (Step 3 of the process) we find it is greater than the root, and the new subtree is therefore the right subtree of 50 (Step 3.2). Since this subtree is empty, Step 5 tells us to make the new node the root of the empty right subtree of 50 (see Figure 7.7 d ). The remaining nodes are inserted into the tree structure in a similar fashion, following the steps of the insertion process, and the tree grows as shown in Figure 7.7 e -f . The subsequent addition of four more nodes to this tree: key 70 first, then keys 80 and 35, and finally, key 68, produces the tree shown in Figure 7.6 . Armed with an understanding of how nodes are arranged in binary search trees inherent in their definition, the process of deciding which subtree to visit after an unsuccessful access is simple. If the key value of the node just accessed is less than the key of the node being searched for, 411 we visit the right subtree; otherwise we visit the left subtree. Knowing which subtree to visit significantly improves the time required to locate a node in a balanced binary tree. Consider a balanced tree with 65,535 nodes and assume we are trying to fetch a node stored in the highest level of the tree. Equation 7.4 tells us that the tree has 16 levels. By making a correct subtree decision at each level, the node would be located after only 16 memory accesses. However, there are times when even knowing what subtree to visit does not significantly reduce the number of memory accesses performed when searching for a node. Consider a tree that is not complete or one that is not balanced. Rather, it is highly skewed, like tree c in Figure 7.3 , with one node at each level of the tree. In this case, there would be 65,535 levels in the tree, and even though we make the correct subtree decision (proceed to the left subtree) after each unsuccessful access, it will take 65,535 memory accesses to locate a node at the highest level of the tree. Thus, although binary search trees do allow us to make a correct subtree decision when searching for a node, the speed advantages associated with this decision-making process can only be realized if the tree is balanced, or close to balanced. Figure 7.8 An Imbalanced Binary Search Tree The most basic binary search tree operation algorithms make no attempt to keep the tree balanced, and they can produce skewed trees like the tree depicted in Figure 7.8 . (This tree would be generated if the keys depicted in Figure 7.7 were inserted into the tree in the order: 70, 63, 68, 80, 50, 57, 47, 40, 44, and finally 35.) However, a characteristic common to most data sets (that will be identified when we study the performance of binary search trees) often produces trees that are close to (or actually) balanced. As a result, even the most basic search tree structure is a 412 practical structure and, in the interest of simplicity, we will begin our study of search trees by examining its operation algorithms. Once we have gained an understanding of this structure, we will discuss more complicated binary tree structures, such as AVL trees and Red-Black trees, whose operation algorithms keep the tree balanced for all data sets. 7.3.1 Basic Operation Algorithms Before discussing the pseudocode of the basic operations on binary search trees, we need to expand our understanding of the circle symbol used in the tree graphics presented thus far in this chapter. A deeper level of understanding of what this symbol represents is essential to the development of the pseudocode and its implementation. The Graphical Circle Symbol Meaning We have previously interpreted the circles in the graphical representations of a tree as the symbol that represents a node, and the annotation inside of it as the contents of the node's key field. At the implementation level, the circle symbol has two other interpretations, each tied to one of the two standard implementations of a binary tree; the linked implementation and the array implementation. The meaning of the circle symbol under the linked implementation of a binary tree will be discussed first, since this implementation is in much wider use. The array implementation will be discussed in a subsequent section of this chapter in which we will also compare the advantages of the two implementations. Figure 7.9 The Linked Implementation Level Meaning of the Circle Symbol In the linked implementation, the circle represents two objects: a TreeNode object and a Listing object (see Figure 7.9 ). The TreeNode object has three data members, all of which are reference variables. 413 Referring to Figure 7.9 , the left- and rightmost data members, named lc and rc respectively, store the location of the nodes' left and right children. The other data member, node, contains the location of the information stored in the structure, a Listing object. Figure 7.10 shows the standard graphic of a three node tree (the left side of the figure) and the actual storage it represents (the center and right portions of the figure) under the linked implementation. The information that has been inserted into the tree structure (three Listing objects) are shown at the right of the figure. The contents of the key fields are 50, 63, and 40 which correspond to the key field values shown in the standard tree graphic at the left side of the figure. As shown in the figure, the items inserted into the tree are not actually arranged in a tree structure. Rather, the TreeNode objects (shown in the center of the figure) form the tree structure. 1 The null values in the lc and rc data members of the lower two TreeNode objects indicate that they do not have any children. It is important that we become familiar with this lower level representation of the circle symbols (presented in Figures 7.9 and 7.10 ) before moving on to the development of the operation algorithms. The best way to approach the coding of these algorithms (or any other operation to be performed on a tree structure) is to first draw these lower-level graphics and then modify them to incorporate the changes necessary to perform the operation. Once the operation algorithm has been graphically developed and verified, it is much more easily coded. Figure 7.10 Actual Storage Represented by the Standard Depiction of a Three-Node Tree 414 Having gained an understanding of the circle symbol under the linked implementation of a binary tree, we are now ready to develop the operation algorithms for a binary search tree under the linked implement. Initialization Algorithm One of the ironies of a binary tree structure (as was the case with linked lists) is that, although a binary tree can store virtually an unlimited number of nodes, initially it contains only one data member, root , a reference variable. This variable will store the location of the tree's root node as depicted in Figure 7.11 . Figure 7.11 A Ten-Node Binary Tree Figure 7.12 A Binary Search Tree Object Before and After Initialization Consistent with the graphical technique for developing the operation algorithms employed in Chapter 4 , Figure 7.12 shows a binary search tree object, before and after its initialization algorithm is executed. The initialization sets the variable root to null . Thus, the initialization algorithm is: Binary Search Tree Initialization Algorithm Whenever the tree is empty, the value of root will be null . We will now turn our attention to the Insert, Fetch, Delete, and Update algorithms. 415 Insert Algorithm and the findNode Method As depicted in Figure 7.7 , the process of inserting nodes into a binary search tree places all newly inserted nodes into the tree as leaf nodes. Therefore, the Insert algorithm must first decide which of the nodes in the tree will be the new leaf's parent. This decision process is depicted in Figure 7.13 in which the node with a key value of 52 is being inserted into the tree. The process involves repositioning two reference variables: P (for parent) and C (for child) until C contains a null value. The reference variables P and C are first set pointing to the root node (see Figure 7.13 a ). The key of the node to be inserted, 52, is compared to the node referenced by C. Since 52 is greater than the key of the node referenced by C, we know the new node belongs in the right subtree of C, and its parent, therefore, must be in the right subtree of the node referenced by C. Therefore, C is set to reference the right child of C (see Figure 7.13 b ). Again, 52 is compared to the node referenced by C. Since 52 is now less than the key of the node referenced by C, we know the new node belongs in the left subtree of C. P is set to C, but now C is set to reference the left child of C (see Figure 7.13 c ). This process continues moving through the levels of the tree until C contains a null reference (see Figure 7.13 d ). At this point the search for the new node's parent ends with P containing a reference to what will become the new node's parent. 416 Figure 7.13 Finding the Parent of a Newly Inserted Node with Key Value 52 The only remaining question is whether the new node will be the parent's left or right child. This is easily determined by comparing the new node's key to the key of the parent node. If the new node's key is less than its parent's key, the new node becomes the parent's left child, otherwise it becomes the parent's right child. To express the steps of the Insert algorithm depicted in Figure 7.13 in pseudocode, it is useful to assume that a method, findNode , exists that locates a node in a binary search tree given its key value, targetKey . Let us also assume the method has two parameters: C and P, and that it sets C to the location of the node with the given key and sets P to the location of C's parent. Thus, if the method were searching the tree depicted in Figure 7.13 for the node with key field 55, then its search process would be as depicted in parts a ,b , and c. Now for some implementation trickery. Let us further assume that when the method was given a key that was not in the tree, it halted its search when the parameter C assumed a null value. Under this assumption, if it were searching for the node with key value 52, its search process would be as depicted in Figures 7.13 a , b , c , and d . Thus (and here is the 417 implementation trickery), the unsuccessful search mode of this method could be used to locate the parent of a node to be inserted into a binary search tree 2 if the key sent to it was the new node's key. Although C would return set to null , P would contain the location of the parent of the new node (see Figure 7.13 d ). Remembering the steps discussed above and depicted in Figure 7.13 , the pseudocode of the method findNode that returns a Boolean value of true after a successful search for the key targetKey is: Pseudocode of the findNode Algorithm (Iterative Version) Alternately, the findNode can be coded recursively. Following our recursive methodology we need to identify the original problem, bases cases, reduced problem, and general solution. The original problem is to locate the node whose key is targetKey in a tree whose root is given, and set C pointing to the node and P pointing to its parent. Stated more succinctly, findNode(root, targetKey, P, C). The base cases would be when the tree is empty or when the key is found (C is referencing the targetKey). The reduced problem would be just one step closer to the base cases: to look for the key in either the right or left subtree of C. The pseudocode of findNode's recursive algorithm follows, assuming the location of the tree is initially stored in root. 418 The recursion is on Line 11 where the algorithm invokes itself and passes to it the root of either the left subtree (Line 8) or the right subtree (Line 10). Returning to the Insert algorithm, after using the method findNode to locate the parent of the new node to be inserted into the search tree, we must place a reference to a deep copy of the inserted information (a Listing object) into a TreeNode object and add the TreeNode object into the tree structure. Assuming the key 52 is to be inserted into the tree depicted in Figure 7.13 d , Figure 7.14 shows the process to complete its insertion. Figure 7.14 Inserting a Node whose Key Value is 52 into a Binary Tree as P's Left Child Finally, the pseudocode version of this graphical representation of the 419 algorithm is given below. It inserts a deep copy of the Listing object referenced by newListing into the tree structure whose root is referenced by root. The line numbers correspond to the circled step numbers in Figure 7.14 . Fetch Algorithm The graphical representation of the Fetch algorithm is essentially depicted in Figure 7.13 a , b , and c , which actually illustrates the process performed by the findNode method to position the reference variable C on the item to be fetched (in this case, key 55). Once located, the Fetch algorithm returns a deep copy of the Listing . In the following pseudocode version of the algorithm, the key value of the item to be fetched is targetKey and the root of the tree is referenced by root . As developed earlier in the chapter, the method findNode returns true if it locates a Listing whose key value is targetKey (Line 1), and sets the argument C to the location of the TreeNode object that references the Listing . In this case, Line 3 returns a reference to a deep copy of the node. If the node is not found, a null value is returned (Line 5). 420 Figure 7.15 Case 1 of the Binary Search Tree Delete Algorithm Delete Algorithm This algorithm is the most complicated algorithm studied thus far in this text. In the spirit of “divide and conquer,” it is traditionally broken down into three cases: • Case 1: the node to be deleted has no children , is a leaf . • Case 2: the node to be deleted has one child , or subtree. • Case 3: the node to be deleted has two children , or subtrees. The portion of the algorithm associated with each case becomes more complex as we move from Case 1 to Case 3. Therefore, the algorithm will be developed in that order. Case 1: The Node to be Deleted has No Children (is a Leaf ) This portion of the Delete algorithm, depicted in Figure 7.15 , simply breaks the connection between the deleted node, C, and the parent node, P, by setting P's reference to the node to null . Since the deleted node is a leaf node, and therefore has no children, we do not have to concern ourselves with retaining the location of the nodes in the deleted node's subtree; it has no subtree. The top half of Figure 7.15 illustrates the process of deleting a leaf node that is a left child, and the bottom half of the figure assumes the node to be deleted is a right child. Assuming the key value of the item to be deleted is targetKey and the tree is referenced by root , the pseudocode version of Case 1 of the algorithm (depicted in Figure 7.15 ) is: Binary Search Tree Delete Algorithm, Case 1 421 The algorithm begins by using the findNode algorithm (Line 1) to set C and P pointing to the node to be deleted and to its parent, respectively. Line 4 verifies that the node to be deleted has no children (Case 1). Then the algorithm decides if the node is a left or right child (Line 5) by comparing the left reference of the parent node to the location of the node, C. Finally, it sets the appropriate reference to the node to null (Lines 6 and 8). Case 2: The Node to be Deleted has One Child or Subtree This portion of the Delete algorithm separately considers the following four possibilities (depicted in Figure 7.16 ) involving the node to be deleted (C) and its parent (P). 2a. C is a left child of P, and C has a left child or subtree (see Figure 7.16 a ). 2b. C is a left child of P, and C has a right child or subtree (see Figure 7.16 b ). 2c. C is a right child of P, and C has a left child or subtree (see Figure 7.16 c ). 2d. C is a right child of P, and C has a right child or subtree (see Figure 7.16 d ). Simply stated, the node to be deleted could be either a left or right child, and it could have either a left or right child (or subtree). The Delete algorithm associated with the four possibilities depicted in Figure 7.16 , is shown in Figure 7.17 . In each of the four possibilities, the node is deleted by resetting the parent's reference to it, to reference the deleted node's child (the root of its subtree). 422 Before writing the pseudocode version of this algorithm, we will discuss one subtle, but important, point. In the upper right portion of Figure 7.17 , the deletion causes the deleted node's subtree to change from a right subtree (of C) to a left subtree (of P). This is precisely what should happen since, although all the keys in the subtree were greater than C's key (the subtree was to the right of C), they must be less than P's key or the subtree would not have been to the left of P. More specifically, all the keys in the subtree depicted in the upper right portion of Figure 7.17 , must be between 33 and 69. Therefore, they should become a left subtree of P. An analogous situation occurs in the possibility depicted in the lower left portion of Figure 7.17 . In this case, all of the keys in 80's subtree must be between 71 and 79. Figure 7.16 The Four Possibilities of Case 2 of the Binary Search Tree Delete Algorithm Assuming the key value of the item to be deleted is targetKey and the root of the tree is referenced by root , the pseudocode version of Case 2 of the algorithm (depicted in Figure 7.17 ) is: 423 Figure 7.17 Case 2 of the Binary Search Tree Delete Algorithm 424 The algorithm begins by using the findNode algorithm (Line 1) to set C and P pointing to the node to be deleted and to its parent, respectively. Line 3 verifies that the node to be deleted has one, and only one, child (Case 2). Then it determines if the node to be deleted is a left or right child (Line 4) by comparing the left reference of the parent node to the location of the node. Finally, it eliminates the node to be deleted (as shown in Figure 7.17 ) by setting the parent's appropriate reference (left child or right child) to the left or right child of the node to be deleted (Lines 6, 8, 12, or 14). Figure 7.18 The Four Possibilities of Case 3 of the Binary Search Tree Delete Algorithm Case 3: The Node to be Deleted has Two Children or Subtrees The final portion of the Delete algorithm separately considers the four possibilities (depicted in Figure 7.18 ) involving the node to be deleted (C) and its parent (P). The triangle symbol in the figure represents a subtree. 3a. C is a left child of P, and C's left child has a right subtree (see Figure 7.18 a ). 425 3b. C is a right child of P, and C's left child has a right subtree (see Figure 7.18 b ). 3c. C is a left child of P, and C's left child has no right subtree (see Figure 7.18 c ). 3d. C is a right child of P, and C's left child has no right subtree (see Figure 7.18 d ). Simply stated, the node to be deleted could be either a left or right child, and its left child could have either a right subtree or not. In the first two of these four possibilities (depicted in Figures 7.18 a and 7.18 b ) the node referenced by C is deleted from the tree using the same process. For the tree depicted in Figure 7.18 a , the process is illustrated in Figure 7.19 . 3 The node to be deleted (in this case the node with key value 70) is simply replaced with a node relocated from its left child's right subtree (in this case the node with key value 65). This node is chosen as the replacement node because its position in the tree guarantees that it is the largest key in 70's left subtree, and it is less than all of the keys in 70's right subtree. Thus, it can be relocated to the deleted node's position without violating the definition of a binary search tree. In general, the replacement node is always chosen to be the node positioned at the far right side of the deleted node's left subtree, which is the largest node in the deleted node's left subtree. To locate this node, two reference variables l and nl (standing for “largest” and “next largest”), are initially positioned as shown in Figure 7.19 a . They then traverse through their right children (shown in Figures 7.19 b and 7.19 c ) until l is referencing a node with no right child. Once this node is located, it is copied into the node to be deleted referenced by C (see Figure 7.19 d ). (More accurately, a reference to it is copied into the node field of the TreeNode referenced by C.) Finally, to prevent two copies of the node referenced by l from being in the structure, nl's right child reference is set pointing to the left subtree of the relocated node (see Figure 7.19 e ). Because of its position in the tree, the keys of all of the nodes in this subtree are greater than the key of the node referenced by nl , making it a legitimate right subtree of this node. The remainder of the Case 3 portion of the Delete algorithm (Subcases 3c and 3d) addresses the other two possibilities for a node with two children; the left child of the node to be deleted does not have a right subtree (see the bottom half of Figure 7.18 ). The graphical depiction of this portion of Case 3 is shown in Figure 7.20 . The left side of the figure assumes the node to be deleted is a left child, and the right side of the 426 figure assumes the node to be deleted is a right child. In both cases, the node to be deleted and its parent is referenced by the variables C and P, respectively. Once again, a reference variable nl is set to reference the left child of the node to be deleted (see Figures 7.20 a and 7.20 b ). To keep from losing the right subtree of the node to be deleted, it becomes nl 's right subtree, which was previously empty (see Figures 7.20 c and 7.20 d ). Finally, the deleted node is removed from the tree by making P's left child (see Figure 7.20 e ), or P's right child (see Figure 7.20 f ), the left child of the deleted node. Assuming the key value of the item to be deleted is targetKey and the tree is referenced by root, the pseudocode version of Case 3 of the Delete algorithm follows. It combines the processes presented in Figures 7.19 and 7.20 . For clarity, the names of the reference variables nl and l have been changed to nextLargest and largest respectively. 427 Figure 7.19 Part “a” of Case 3 of the Binary Search Tree Delete Algorithm (the node to be deleted has a key of 70) 428 Figure 7.20 Parts “c” and “d” of Case 3 of the Binary Search Tree Delete Algorithm 429 The algorithm begins by using the findNode algorithm (Line 1) to set C and P pointing to the node to be deleted and its parent, respectively. Line 3 verifies that the node to be deleted has two children (Case 3). Then, the locations of the node's left child, and the child's right child are set into the variables nextLargest and largest, respectively (Lines 4 and 5). If the left child has a right subtree (Line 6), then Lines 7–13 execute, which is the pseudocode of the algorithm shown in Figure 7.19 . Otherwise, Lines 14–21 execute, which is the pseudocode version of the algorithm shown in Figure 7.20 . Referring to Figures 7.19 a -c , the while loop (Lines 7–10) locates the node that will replace the node to be deleted. Once this node is located, Lines 11–12 perform the deletion (see Figure 7.19 d ) and reclaim the repositioned node's left subtree (see Figure 7.19 e ). Figure 7.21 shows the changes to the relevant TreeNode objects (see the unshaded fields in the figure) made by Lines 11 and 12 of Case 3 of the Delete algorithm at the implementation level. Since the deleted node, and the TreeNode object that was associated with the largest key in the subtree (65), is no longer referenced by a variable in the structure, they will be recycled by Java's memory manager. When the deleted node's left child does not have a right subtree (see Figure 7.20 ), Line 15 of the algorithm performs the action depicted in Figures 7.20 c and 7.20 d , and Lines 17 and 19 actually delete the node from the structure (see Figures 7.20 e and 7.20 f ). 7.3.2 Performance The performance of a data structure is dependent upon the speed of its operations and the additional memory (above that necessary to store the clones of the client's information) required by the structure. In this section, we will discuss the speed of the structure's operations first, and then discuss the structure's overhead. Throughout our speed discussions, we 430 will assume that the binary search tree is balanced, which, as we have seen, has a significant impact on the speed of the structure. For data sets that do not produce balanced binary search trees, the speed of the operations would be slower than that presented in this section. At the end of this chapter, we will consider alternate binary tree structures that always result in balanced (or close to balanced) trees. Figure 7.21 Memory Changes Resulting from Lines 11 and 12 of Case 3 of the Delete Algorithm Speed of the Structure As we have done in the previous chapters, we will perform a Big-O analysis to determine the approximate speed of the binary search tree structure as n , the number of nodes stored in the structure, gets large. Since a Big-O analysis is a bounding technique, and the time to perform a memory access instruction is typically considerably longer than the time to perform a nonaccess instruction, only memory access instructions will be included in our analysis. Since all of the basic operation algorithms use the findNode algorithm, we will analyze it first. Pseudocode of the findNode Algorithm (Iterative Version) 431 Assuming that the variables P and C will be stored in CPU registers, Lines 1 and 2 require one memory access to fetch the variable root . Although targetKey is used inside the while loop to descend through the levels of the search tree (see Figure 7.13 ), since its value does not change, it would only be fetched once and then stored in a CPU register. Examining the remainder of the loop, Line 4 requires two memory accesses to fetch C's key. The variables involved in Lines 7 and 8 have already been used in previous statements so they are already in the registers of the CPU and, therefore, do not require any additional memory accesses. Finally, either Line 9 or Line 11 executes requiring one additional memory access to access either C's left or right child reference. Therefore, the number of memory accesses performed by this algorithm is: 2 + 3T , where T is the number of times the loop executes. To determine the value of T , we recall that every time the loop executes we move one level higher into the tree. This means that T will be in the range 1 (when locating the root node) to log2 (n + 1) (when locating a node at the highest level of our, assumed to be, balanced tree: see equation 7.4 ). At most, half the nodes in a balanced tree are at the highest level of the tree. 4 This occurs when the tree is not only balanced, but also complete (the highest level is fully populated). Therefore, at most, half the nodes will require log2 (n + 1) passes through the loop, and the other half 432 of the nodes will require less passes (1, or 2…, or log2 (n + 1) − 1). This makes T <= log2 (n + 1), and the number of accesses to locate a node in balanced binary tree using the findNode algorithm is: The Number of Memory Accesses Performed by the findNode Algorithm Having developed an expression for the speed of the findNode algorithm, we will now determine the speed of the basic operation algorithms beginning with the Insert Algorithm. For convenience, it is presented again as follows: Lines 1, 2, 3, and 4 require a total of four additional accesses: one access per line to store values in the variables n, n.node, n.lc, and n.rc . Next, Lines 5a-b or c-h execute. Lines 5a-b require less memory accesses (i.e., two: one to fetch the variable root and one to store n in root) than 5c- h so we will ignore them. We have previously determined that Line 5d requires less than 2 + 3log2 (n + 1) memory accesses. Line 5e requires four memory accesses (one to access P, one to access P.node, one to fetch P.node's key, and one to fetch newListing 's key). Finally, one access is required to store n in either P.lc or P.rc (Lines 5f and 5h). Therefore, assuming the tree is balanced , the Insert algorithm requires <= 11 + 3log2 (n + 1) accesses: four for Lines 1−4, (2 + 3log2 (n + 1), for Line 5d, four for Line 5e, and one for either Lines 5f or 5h) which is O(log2 n ). 433 We will now analyze the speed of the Fetch algorithm, which is repeated for convenience. Binary Search Tree Fetch Algorithm As we have shown, Line 1 requires less than 2 + 3log2 (n + 1) memory accesses. Examining Line 3, we see that it requires two memory accesses: one to access C and 1 to access C.node. Thus, the Fetch algorithm requires less than 4 + 3log2 (n + 1) memory accesses which is O(log2 n ). Table 7.2 Number of Nodes in the Levels of a Six-Level Complete Binary Tree Level Number, l Nodes at Level l Total Number of Nodes Below this Level 0 1 0 1 2 1 2 4 3 3 8 7 4 16 15 5 32 31 Since there are three parts to the Delete algorithm, in order to analyze its performance we will need to determine what percentage of the nodes in a balanced binary tree fall into Cases 1, 2, and 3 of the algorithm; that is, how many nodes have 0, 1, or 2 children. Then, these percentages will be used to calculate a weighted average speed of the algorithm. Consider the complete binary tree with three levels shown in Figure 7.3 f . Counting the nodes in this tree, we find that Level 0 contains 1 node; Level 1 contains 2 nodes; and Level 3 contains 4 nodes. Table 7.2 tabulates this data and extrapolates it to a six-level complete binary tree. In addition, its rightmost column presents the total number of nodes below each level, l , of the six-level tree. Comparing the data in the rightmost 434 column to that in its middle column, we can observe that each level of the tree contains one more node than the total number of nodes in all the levels below it. This is typical of all complete binary trees. Thus, if there are n nodes in a complete binary tree, and x represents the number of nodes in the highest level of the tree, then the total number of nodes in all the other (lower) levels of the tree is x − 1 and the total number of nodes in the tree, n , is n = x + x − 1. Now let us add one more level to our complete tree, which we will assume, on the average, will be half populated. The tree is no longer complete, but it is balanced. Being half populated, instead of containing twice as many nodes as the level below it, 2x , the newly added level only contains 2x /2 = x nodes. Thus, the total number of nodes, n becomes: Figure 7.22 Distribution of the Number of Nodes in a Balanced Binary Tree Whose Highest Level is Half Populated (n is the number of nodes in the tree) where x is the number of nodes at the next to highest level of the new tree, l h−1 . Solving this equation for x we obtain x = (n + 1)/3, and therefore: For Any Balanced Binary Tree Containing n Nodes Whose Highest Level is Half Populated For example, consider the balanced binary tree shown in Figure 7.22 whose highest level is half populated. The total number of nodes in the tree 435 is 11. Counting the number of nodes at each level of the tree we can verify that there are (11 + 1) / 3 = 4 nodes at each of the two highest levels, and a total of (n + 1) / 3 − 1 = 3 nodes in the levels below the two highest levels. We are now in a position to determine how many of the nodes in a balanced binary tree, whose highest level is half populated, have (most probably) zero, one, or two children. Equation 7.5 indicates that (approximately) one-third of the nodes reside in the highest level of the tree. Since all of the nodes at this level are leafs, these one-third n nodes have no children. Equation 7.6 indicates that (approximately) one-third of the nodes reside in the levels below the next to the highest level, l h−1 . Since all of these level and level lh−1 are fully populated, these (one-third n nodes) all have two children each. The remaining one-third of the nodes reside at level lh−1 . It can be demonstrated that, since the level below it is half populated, most probably, half of these nodes will have one child. With half of the nodes at level l h−1 having one child, the only way to half populate the highest level is if half of the remaining nodes at level l h−1 have two children, and the rest of the nodes at l h−1 must have zero children. Consistent with the previous analysis, Table 7.3 summarizes the most probable distribution of the nodes with zero, one, or two children in a balanced binary tree whose lowest level is half populated. The bottom row of the table is a weighted sum of the three rows above it. As indicated in this row, when deleting nodes from a balanced binary tree (whose highest level is half populated), 42% of the time we will be deleting a node with 436 zero children (Case 1 of the Delete algorithm), 16% of the time we will be deleting a node with one child (Case 2 of the Delete algorithm), and 42% of the deleted nodes will have two children (Case 3 of the Delete algorithm). Having determined the weighting factors to be applied to the speeds of Cases 1, 2, and 3 of the Delete algorithm (42%, 16%, and 42%, respectively), we will now analyze each case individually to determine their speed. The analysis will begin with an observation: each of the three cases of the Delete algorithm begins with an invocation of the findNode method which, when the three cases are combined into one Delete method, will only be invoked once. Therefore, we will ignore this line during our analysis of the three cases and include its 2 + 3log2(n + 1) memory access after we have determined the weighted average speed of the three cases. In order to more easily analyze Case 1 of the Delete algorithm, its pseudocode, developed earlier, follows: Binary Search Tree Delete Algorithm, Case 1 Ignoring findNode 's accesses, Line 4 requires three memory accesses to access C, C.lc , and C.rc. Line 5 requires two additional memory accesses to access P , and then P.lc . Lines 6 or 8 require one memory access to store a null value in either P.lc or P.rc . Thus, Case 1 of the delete algorithm requires a total of six memory accesses (which is O(1)). The pseudocode of Case 2 of the Delete algorithm, previously developed, is: 437 Line 3 requires three memory accesses to access C, C.lc , and then C.rc . Line 4 requires two additional memory accesses to access P , and then P.lc . Either Lines 5–9 or Lines 11–15 execute next. Each group of code requires the same number of memory accesses. We will analyze Lines 5–8. The only additional memory access performed in this section of pseudocode is the writing into the memory cell P.lc on either Line 6 or 8. Therefore, ignoring findNode 's accesses, Case 2 of the Delete algorithm requires six memory access (which is O(1)). The pseudocode of Case 3 of the Delete algorithm, previously developed, is: 438 We will again ignore Line 1 of the algorithm, the invocation of the findNode method, because it will only be invoked once for all three cases of the Delete algorithm. The remainder of the algorithm can be divided into three portions. The upper portion (Lines 3–6), the middle portion (Lines 7–13) which executes if the left child of the deleted node has a right subtree, and the lower portion (Lines 14–20) which executes if the left child of the deleted node does not have a right subtree. Examining the upper portion, Line 3 requires three memory accesses to access C, C.lc , and then C.rc . The variables nextLargest (first used on Line 4 and largest (first used on Line 5 and again on Line 6) are variables local to this algorithm. An optimizing compiler would store them in CPU registers, and therefore accessing them does not require memory access. However, Line 5 does require one additional memory access to fetch nextLargest.rc . Therefore, the upper portion of the algorithm requires a total of four memory accesses. Examining the middle portion of the algorithm (Lines 7–13), the loop on Lines 7–10 require one memory access to fetch the memory cell 439 largest.rc every time the loop executes. Lines 11 and 12 require a total of four memory accesses to access C.node, largest.node, nextLargest.rc , and largest.lc . This give a total of 1T + 4 memory accesses, where T is the number of times the loop that begins on Line 7 executes. To find T , we must find the average number of nodes traversed as we move down the right subtree of the left child of the deleted node (see Figure 7.19 ). Since we are in Case 3 of the algorithm, the deleted node cannot be at the highest level of the tree because Case 3 nodes have two children, and the nodes on the highest level have no children. Furthermore, since we are in the middle portion of the Case 3 algorithm, the deleted node's left child must have a right subtree. Therefore, the deleted node cannot be on the next to the highest level either, since the left children of nodes at this level are leafs (they are on the highest level). Thus, this portion of the Case 3 algorithm deals only with nodes below the next to the highest level of the tree. Figure 7.23 shows a tree with 10 levels. Suppose we were deleting the root node of the tree. Then the variables nextLargest and largest (nl and l of Figure 7.19 ) would be positioned at nodes 40 and 45 respectively. Therefore, to move l to the highest level of the tree, the loop would execute 7 times. This can be verified by counting the number of nodes from node 45 to the lowest level of the tree shown in Figure 7.23 . Similarly, if the node to be deleted was on level 1 of the tree, the variables nextLargest and largest would be positioned at levels 2 and 3, respectively, and the loop would execute six times. Extending this logic, Column (b) of Table 7.4 presents the number of times the loop executes to delete a node at each level of a 10-level tree operated on by the middle portion of Case 3 of the algorithm. Column (c) of the table presents the number of nodes at each level of the tree, and column (d) presents the number of times through the loop to delete all the nodes at a level of the tree. Assuming all the nodes in the tree are equally likely to be deleted; we can compute a weighted average number of times through the loop by dividing the sum of Column (c) by the sum of Column (d). Thus, for our 10-level tree, the most probable number of times the loop in the middle portion of the algorithm executes is: 247 / 255 = 0.97 times, or T is approximately 1. This result may seem surprising, but it is typical of balanced binary search trees of any size. Since most of the nodes in a balanced binary tree reside in the higher levels of the tree, their deletion requires very few passes through the loop. In fact, as illustrated in the dark shaded row of 440 Table 7.4 , the majority of the nodes deleted by this portion of the Case 3 algorithm (128 of the 255 nodes) require zero passes through the loop. Substituting an average value of T = 1 into the expression for the number of memory accesses performed by the middle portion of the algorithm (1T + 4, as derived previously), we find that an average of five memory accesses are performed to delete a node processed by the middle portion of the Case 3 algorithm. Figure 7.23 A 10-Level Binary Tree 441 The lower portion of the algorithm (Lines 14–20) requires one memory access to write into next-Largest.rc (Line 15), one access to write into P.lc (Line 16), and one access to write into P.lc or P.rc (Lines 17 or 19). Thus, the lower portion of the Case 3 algorithm requires three memory accesses. To summarize our findings, the upper, middle, and lower portion of Case 3 of the delete algorithm requires four, five, and three memory accesses, respectively. We will use the worst case performance scenario for the Case 3 algorithm, which is when the upper portion and the middle (rather than the lower) portion of the algorithm execute. This execution path requires a total of nine memory accesses to delete a node: four memory accesses performed for the upper portion and five performed by the middle portion of the algorithm, which is O(1). To combine all three cases of the Delete algorithm, we add the number of memory accesses required by Line 1 of the algorithm 2 + 3log2(n + 1) (the invocation of findNode ), to the number of memory accesses required by Cases 1, 2, and 3 of the Delete algorithm (six, six, and nine, respectively) with the weighting factors of Table 7.3 (0.42, 0.16, 0.42) applied to them. The result is that the most probable number of memory accesses for the Delete algorithm is: which is O(log2 n ) memory accesses per deletion. Density Let us now turn our attention to the overhead of a binary search tree structure. The overhead is the storage associated with the TreeNodes (see Figure 7.9 ) that actually make up the binary tree and the reference variable, root , that stores the address of the root of the tree. Therefore, the total overhead is 1 reference variable associated with the variable root plus 3n reference variables associated with the client's n information Listings in the structure. This gives a total of 1 + 3n reference variables of overhead. Since reference variables occupy 4 bytes, the total overhead is therefore 4(1 + 3n ) bytes. Density is defined as: 442 Figure 7.24 Density of the Binary Search Tree Structure for n > 100
The information bytes is simply the product of the number of client
Listings n and the number of bytes per Listing, w. Therefore, the density
can be expressed as:
which is approximately equal to 1/(1 + 12/w) as n gets large. Figure
7.24 presents a graph of the approximation of this function for n > 100.
The figure demonstrates that good densities (0.80 or higher) are
achieved whenever the number of information bytes per node is greater
than 48.
Table 7.5 summarizes the performance of the balanced Binary Search
Tree structure, and includes the performance of the previously studied
structures for comparative purposes. When storing large data sets, the
speed of the binary tree structure (presented in the next to the last column
of the table) approaches the speed of the hashed data structures. The speed
advantages of this structure, its ability to expand to accommodate an
unlimited number of nodes, and (as we will see) its ability to process nodes
in sorted order make it a popular data structure for many applications.
7.3.3 Implementation
This implementation, named BinaryTree , will be a fully
encapsulated homogeneous implementation of a binary search tree
structure. The code, presented in Figure 7.25 , is consistent with the many
of the concepts of generics presented in Chapter 2 in that it does not
mention the names of any of the fields of the nodes and the definition of
the nodes to be stored in the structure is defined in a separate class (see
443

Figure 2.16 ). The node definition class provides a deepCopy method in
order to encapsulate the structure, a compareTo method to determine if a
given key is equal to the key of a node in the structure, and a toString
method to return the contents of a node. Like the hashed data structures
studied in Chapter 5 , this structure’s Insert algorithm needs access to the
key field of the node being inserted. Therefore, a getKey method will have
to be added to the class Listing. The code of the method is given as
follows:
444

445

446

Figure 7.25 Listing of the Implementation of a Binary Search Tree
Structure
A fully generic implementation of the structure, using the generic
features of Java 5.0 and the techniques described in Chapters 2 and 3
(Sections 2.5 and 3.4 ) will be left as an exercise for the student.
Lines 97–104 are a definition of the class TreeNode . This class
447

defines the objects that will make up the binary search tree. A TreeNode
object contains three reference variables (Lines 98–100): node (a reference
to a Listing object ), lc , and rc (both references to TreeNode objects).
The variable node will reference the deep copy of the client’s information
inserted into the structure, while lc and rc are the references to the tree
node’s left and right children, respectively. The class is defined as an inner
class (of the class BinaryTree ) for two reasons. Most importantly, this
allows the code of the class BinaryTree to directly access a TreeNode
object’s data members (e.g., Lines 13–15). In addition, only the code of
this class will need to declare TreeNode objects.
Line 2 declares the reference variable, root , that will store the
address of the root TreeNode . It is initialized to the empty tree condition
on Line 4 by the class’ constructor.
The findNode method is coded on Lines 105–123. It is the Java
coding of the iterative pseudocode version of the algorithm developed
earlier in this chapter, with two exceptions. First, lines 109 and 110 have
been added to check for an empty tree. Second, aside from a Boolean
value, this method must also return two node locations: the location of the
node whose key is targetKey, and the location of its parent. The only way
for a Java method to return more than one item, is by way of its parameter
list. However, since parameters in Java are always value parameters, the
arguments passed into the method by the client are unchanged when the
method returns back to the invoker. The solution to this dilemma is to
make the items to be changed by the method data members of an object.
References to these objects are passed to the method which then changes
the data members in the objects. Thus, the parameters are not the items to
be modified, but references to the objects containing the items to be
modified. A Java class created for this purpose, to allow a method to return
a changed value via a parameter, is called a wrapper class.
This explains why the second and third parameters of method
findNode (Line 105-106) are of type TreeNodeWrapper rather than type
TreeNode . The class TreeNodeWrapper is defined as a second inner
class on Lines 124–135. TreeNodeWrapper objects contain one data
member, a reference to a TreeNode (Line 125). This variable can be set,
or fetched, using the methods of the class coded on Lines 129–134. The
method findNode uses these methods to begin its search for the node with
the given key at the root of the tree (Lines 107 and 108), and to move the
TreeNodeWrapper objects’ parent and child references through the levels
of the tree (Lines 115, 117, and 119). The Listing class’ method
448

compareTo is used to decide if the node has been found and, if not, to
decide if the search should proceed into the left, or right, subtree of the
node just examined (Line 16).
The Insert, Fetch, and Delete operation methods on Lines 6–89, are
the Java coding of the pseudocode algorithms previously developed in this
chapter. However, they pass TreeNodeWrapper objects to the method
findNode (e.g., Lines 7, 8, and 19), and use the get method (e.g., Lines 21
and 34) of the class TreeNodeWrapper to access the returned information
(the location of the parent of the node to be operated on, or the location of
node itself). Lines 90–96 is the update method, which is the same coding
of the update methods presented in previous chapters. It invokes the
delete and insert methods (Lines 91 and 93) to perform its operation.
Finally, the class does not contain a showAll method. Outputting all
the nodes in a binary tree structure is not as simple as when the nodes are
stored in the structures previously discussed. The showAll method will be
presented in the next section after a detailed discussion of the techniques
used to traverse binary trees.
To demonstrate the use of the class BinaryTree , an application
program that processes a telephone listing data set is presented in Figure
7.26 , and the output it generates is presented in Figure 7.27 . The Listing
class that defines the telephone listings, would be similar to the class
Listing presented in Figure 2.16 of Chapter 2 , modified to include a
getKey method that returns the key field of a Listing object.
7.3.4 Standard Tree Traversals
Traversing a data structure is the process of performing a processing
operation on each node in the structure, once and only once. When we
perform the processing instruction on a node, we are said to have “visited”
the node. Typical processing operations are to modify the contents of a
particular field of the nodes, output the contents of the nodes, or to count
the nodes to determine how many are in the structure.
We have already studied traversal methods. The showAll method
coded in the data structure implementations presented in the previous
chapters, is an example of an output traversal. The linear nature of these
data structures allowed us to traverse them using a loop construct. For
example, in the case of the array-based structures, the hashed structures,
and the restricted structures the loop variable simply indexed sequentially
through the array associated with these structures from the first to the last
449

element. The linked list traversal used a while loop to sequentially travel
through the nodes stored in the structure via the next (or link) field, until
the last node in the structure (the node with a null next field) was output.
The algorithm for traversing a tree structure is not as simplistic
because a tree is not a linear structure. After visiting the unique first node
(the root node) it is not clear which node to visit next. For example, the left
child of the root node could be visited second, and the root’s right child
third, or vice versa. Actually, any of the nodes in the tree could be
considered the second node in the traversal, and in fact, the root node does
not necessarily have to be the first node operated on.
Furthermore, once we proceed into one of the two subtrees of a node
another problem develops; the location of the subtree not visited is no
longer available. Thus, the traversal algorithm must maintain a history of
the subtree locations not visited (as it proceeds down the levels of the tree)
in order to insure that all nodes of the tree are visited. For example, if the
traversal always chose the left subtree first, the locations of all the right
subtrees would have to be stored.
450

Figure 7.26 An Application that Uses a Binary Search Tree Structure
451

Figure 7.27 The Output Generated by the Application Program
Presented in Figure 7.26
Although it is true that many traverse orders are possible, 5 . most of
them are rarely, if ever, used. The most often used traverses fall into two
groups:
• Those that visit all the nodes at a given level (siblings) before
proceeding to the next level, called breadth-first traverses.
• Those that visit all children of a node, before visiting the nodes
siblings, called depth-first traverses.
The most often used traverses in the depth-first group have been
given names indicative of the order in which they visit the nodes. Consider
the binary tree shown in Figure 7.28 .
Designating the left subtree as L , the root node as N , and the right
subtree as R , a traversal that first visited all the nodes in the left subtree,
then visited the root node, and then visited all the nodes in the right subtree
is named an LNR traversal, or LNR scan . This scan, and the other
possible scans designated by the five other permutations of these three
letters, are defined below.
LNR traverse the l eft subtree, then visit the root n ode, then traverse
the r ight subtree.
LRN traverse the l eft subtree, then traverse the r ight subtree, then visit
the root n ode.
452

NLR visit the root n ode, then traverse the l eft subtree, then traverse
the r ight subtree.
NRL visit the root n ode, then traverse the r ight subtree, then traverse
the l eft subtree.
RLN traverse the r ight subtree, then traverse the l eft subtree, then visit
the root n ode.
RNL traverse the r ight subtree, then visit the root n ode, then traverse
the l eft subtree.
Figure 7.28 A Binary Tree
The three traversals that visit the subtrees in a left-to-right order:
LNR, LRN, and NLR have alternate names. (By “left-to-right” order we
mean that during these traversals the left subtree is always traversed before
the right subtree.) The prefixes pre , in , and post are used in the alternate
names to indicate when the root node is visited during the traversal. Thus:
• NLR is called a pre order scan (because the root node is visited before
visiting the left and right subtrees).
• LNR is called an in order scan (because the root node is visited in
between visiting the left and right subtrees).
• LRN is called a post order scan (because the root node is visited after
visiting the left and right subtrees).
Returning to the discussion of the six scans, a question arises; how do
we traverse the subtrees during these scans? That is, what is the order in
which the nodes in the left or right subtrees are visited? The answer is
simple. The subtrees are traversed in the same order as the original tree.
Therefore, if an LNR traversal is being performed on the tree, then the
subtrees are traversed using an LNR traversal. The LNR definition (and
that of the other five traversals) is recursive in that an LNR traversal uses
an LNR traversal to traverse the subtrees. The LNR traversal is more
accurately stated as:
LNR Traversal
453

• Traverse the entire left subtree using an LNR traversal recursively.
• Visit (operate on) the root node.
• Traverse the entire right subtree using an LNR traversal recursively .
Figure 7.29 Four Binary Search Trees
The base case of this recursive definition is when the subtree is
empty. In this case we do nothing (which most everyone is good at). Thus,
the recursive LNR algorithm to traverse the tree whose root node is
referenced by the variable root , whose left child is referenced by lc , and
whose right child is referenced by rc , is:
LNR Recursive Traversal Algorithm Named LNRtraversal (assumes
LNRtraversal is passed the root of the tree to be traversed)
To gain an understanding of the recursion in the algorithm, we will
examine the algorithm’s execution for the trees shown in Figure 7.29 . For
simplicity, we will assume that the operation performed on a node by Line
3 of the algorithm (when a node is visited) is to output the node’s key field.
First let us consider the simple case of performing an LNR traversal
454

on a tree with just one node, a root node (see Figure 7.29 a ). The if
statement on Line 1 evaluates to false (the root node does not have a left
subtree) and Line 2 does not execute. Line 3 operates on the root node, and
the root node’s key (50) is output. Finally, the if statement on Line 4
evaluates to false (the root node’s right subtree is empty) and the algorithm
ends. This traversal does not execute the recursive part of the LNR
algorithm (Lines 2 and 5). The confusing part of a traversal is the use of
recursion to output the nodes in the subtrees.
Consider the LNR traversal of the tree with two nodes shown in
Figure 7.29 b . This time the if statement on Line 1 evaluates to true (the
root has a left subtree) and Line 2 executes. Line 2 is recursive. It invokes
LNRtraversal recursively to perform an LNR traversal on the tree whose
root is the node with key field 40, and the algorithm begins again. In this
invocation, the variable root refers to the node whose key is 40.
The if statement on Line 1 evaluates to false (the tree whose root is
40 does not have a left subtree), and Line 2 does not execute. Line 3
operates on the root node, and the key 40 is output at this point. Finally,
the if statement on Line 4 evaluates to false (40’s right subtree is empty),
and the recursive invocation of the algorithm ends.
Having completed Line 2 of the original execution of the algorithm,
Line 3 operates on the root node of the tree, and the key 50 is output at this
point. Finally, the if statement on Line 4 evaluates to false (50’s right
subtree is empty), and the algorithm ends.
If the right subtree were not empty (as in Figure 7.29 c ) the if
statement on Line 4 would perform an LNR traversal on the tree whose
root is the node with key field 63. In this invocation, the variable root
refers to the node whose key is 63. The if statement on Line 1 evaluates to
false (the root node, 63, does not have a left subtree), and Line 2 does not
execute. Line 3 operates on the root node, and the key 63 is output.
Finally, the if statement on Line 4 evaluates to false (63’s right subtree is
empty) and the recursive invocation of the algorithm ends. This completes
Line 5 of the original execution of the algorithm, and it ends.
Finally, consider the tree presented in Figure 7.29 d with nodes
whose key fields are 50, 40, 63, 47, and 55. The LNR algorithm’s traversal
process would be (reading from left to right):
Line 2 Line 3 Line 5
Traverse all the nodes in the
left subtree of 50, in LNR
then output
50 (the root
then traverse all the nodes in the
right subtree of 50, in LNR
455

order recursively node) order recursively
Thus, to output the entire tree using an LNR traversal, we must first
traverse the left subtree of 50 in LNR order. To accomplish this, we treat
this subtree as an independent tree whose root has a key value of 40 to be
traversed also (recursively) in LNR order. Thus, the traversal process to
traverse the left subtree of 50 (first level of recursion) is (reading from left
to right):
Line 1 Line 3 Line 5
The left subtree of 40
is empty, skip Line 2
output 40
(the root
node)
then traverse all the nodes in the right
subtree of 40, in LNR order recursively
Line 1 of the algorithm determines that the left subtree of 40 is empty,
and Line 2 does not execute. Line 3 outputs the root node of the subtree,
40, producing the first output. To complete the traversal of the left subtree
of 50, Line 5 traverses the right subtree of 40 using an LNR traversal. To
accomplish this, we treat the subtree as an independent tree whose root has
a key value 47 (see Figure 7.29 d ) to be traversed also (recursively) in
LNR order. Thus, the traversal process to traverse the right subtree of 40
(second level of recursion) would be (reading from left to right):
Line 1 Line 3 Line 4
The left subtree of 47 is
empty, skip Line 2
then output 47 (the
root node)
The right subtree of 47 is
empty, skip Line 5
Line 1 of the algorithm determines that the left subtree of 47 is empty,
and Line 2 does not execute. Line 3 outputs the root node of the subtree,
47, producing the second output. To complete the traverse of the right
subtree of 40, Line 4 of the algorithm determines that 47’s right subtree is
empty, and Line 5 does not execute. This completes the second level of
recursion (used to output the right subtree of 40), as well as the first level
of recursion (to output the left subtree of 50), and so the root of the
original tree, 50, is output (the third output).
To complete the LNR scan of the tree in Figure 7.29 d , we must
traverse the right subtree of 50 in LNR order. To accomplish this, we treat
this subtree as an independent tree whose root has a key value 63 to be
traversed also (recursively) in LNR order. Thus, the traversal process to
traverse the right subtree of 50 (first level of recursion) is (reading from
left to right):
Line 2 Line 3 Line 4
Traverse all the nodes in the left then output 63 The right subtree of
456

subtree of 63, in LNR order
recursively
(the root node) 63 is empty, skip Line
5
Line 2 traverses the left subtree of 63 using an LNR traversal. To
accomplish this, we treat this subtree as an independent tree whose root
has a key value 55 (see Figure 7.29 d ) to be traversed also (recursively) in
LNR order. Thus, the traversal process to traverse the left subtree of 63
(second level of recursion) is (reading from left to right):
Line 1 Line 3 Line 4
The left subtree of 55 is
empty, skip Line 2
then output 55 (the
root node)
The right subtree of 55 is
empty, skip Line 5
Line 1 of the algorithm determines that the left subtree is empty, and
Line 2 does not execute. Line 3 outputs the root node of the subtree, 55,
producing the fourth output. To complete the traverse of the left subtree of
63, Line 4 of the algorithm determines that the right subtree of 55 is
empty, and Line 5 does not execute. This completes the second level of
recursion (to traverse the left subtree of 63).
Returning to the first level of recursion used to traverse the right
subtree of 50, Line 3 executes and outputs the root node of 50’s right
subtree, 63 (producing the fifth output). Next, Line 4 executes and, since
the right subtree of 63 is empty, the first level of recursion ends. This
completes the execution of Line 5 of the initial invocation of the algorithm
(to traverse the tree whose root is 50) and the LNR traversal of the tree
depicted in Figure 7.29 d is complete. The output produced is 40, 47, 50,
55, 63.
To summarize the execution of the tree traversal algorithms, the
subtrees are traversed using a recursive tree traversal whose escape clause
is an empty subtree. Figure 7.30 shows the results of LNR, NLR, and RNL
output traversals of an expanded version of the tree presented in Figure
7.29 d . This figure can be studied to gain an understanding of these
recursive traversal algorithms.
As shown in Figure 7.30 , the LNR and RNL traversals output the
nodes in ascending and descending key order respectively. This is not the
case for LNR or RNL output traversals performed on all binary trees, but
only those binary trees that are also binary search trees. Examining the tree
presented in Figure 7.30 , we can verify that it is, in fact, a binary search
tree. Therefore, aside from good performance, binary search trees have the
additional feature that LNR and RNL scans process the nodes in key field
sorted order.
457

Figure 7.30 Three Output Traversals of a Binary Tree
Figure 7.31 The LNR Output Traversal
Figure 7.32 The ShowAll method for a Binary Search Tree
The showAll Method
Now that we have gained an understanding of tree traversal
techniques, we can complete the coding of the class BinaryTree by coding
its showAll method used by the client to output the contents of all of the
nodes stored in the structure. We will arbitrarily select an LNR traversal to
visit all the nodes in the tree, and so the nodes will be output in ascending
order based on the values of the key fields of the nodes. Both the traversal
method and the showAll method are normally included as member
functions of the data structure class (see Figure 7.25 ).
The implementation of the LNR output traversal shown in Figure
7.31 is simply the Java version of this algorithm with console output
substituted for Line 3 of the algorithm (Line 4 of the method).
458

The code of the showAll method is presented in Figure 7.32 . If the
tree to be output is not empty (checked on Line 2), the method simply
invokes the LNRoutputTraversal method, passing it the location of the
root of the tree (Line 5).
7.3.5 Balanced Search Trees
As previously discussed, the speed of a balanced binary search tree is
O(log2 n ). With the exception of hashed structures, they are faster than all
the other data structures presented thus far. Balancing a tree minimizes the
number of levels in the tree, and since one comparison is made per level to
locate a node in a search tree, balancing a tree also maximizes the speed of
the structure’s operations.
The binary search tree Insert algorithm presented in this chapter made
no attempt to keep the tree balanced, and so for some data sets the O(log2
n ) speed of the structure is not realized. Consider the four node data set
with one letter keys: A, B, C, and D. Let us assume that the node whose
key is A is inserted first, then B, then C, and finally the node with key D is
inserted. Because each newly inserted node’s key is greater than all the
nodes currently in the tree, each node is inserted as the rightmost
descendent and the tree develops (as shown in Figure 7.33 ) highly skewed
to the right.
Figure 7.33 A Binary Search Tree after Inserting Keys A, then B,
then C, and then D
Whenever nodes are inserted into a binary search tree in sorted key
order, each node inserted creates a new level of the tree, making the
number of levels in the tree equal to the number of nodes in the tree. Under
these conditions, the speed of the structure degenerates to that of a singly
linked list, O(n ). To prevent this from happening, the data set can be
randomized before it is initially inserted into the data structure.
Although randomizing the nodes before the tree is built usually
459

results in an initial search tree that is balanced (or close to balanced), once
the data structure is “in service” subsequent insertions and deletions can
cause it to become highly imbalanced. One way to prevent this from
happening is to keep track of the number of nodes, n , and the number of
levels, l , in the tree. Whenever an Insert (or Delete) operation is
performed, the number of levels in the tree, l , is compared to the number
of levels in a balanced binary tree with n nodes: ciel(log2 (n + 1)). If the
difference between the actual number of levels in the tree is larger than
that of a balanced tree by some specified tolerance, dL max , all the nodes
are removed from the tree, randomized, and reinserted into the tree. This
usually brings the tree back into balance within the desired tolerance.
When this technique is used to keep a binary search tree balanced, the
Insert and Delete operation uses the parameter dL = l − ciel(log2 (n + 1)) to
monitor how far the tree is out of balance. The tree is rebuilt when this
parameter is greater then the chosen value of dL max . The choice of the
value of dL max is not self evident. Although the speed of the operation
algorithms is optimized when the tree is balanced, dL = 0, rebuilding the
tree takes time. If dL max is chosen too small, the tree is frequently rebuilt,
and the processing time associated with the rebuilding degrades the overall
performance of the structure. Conversely, if dL max is chosen too large, the
speed of the basic operations degrades as dL increases. Ultimately, the
optimum value of dL max for a particular application is dependent on the
character of the data set and the distribution of the operations performed
on the structure. The modification of the Binary Search Tree
implementation developed in this chapter (see Figure 7.25 ) to keep the
tree balanced within a client specified tolerance is left as an exercise for
the student.
AVL Trees
A more efficient technique for keeping a binary search tree balanced
was developed by two mathematicians, G. M. Adelson-Velskii and E. M.
Landis, in the 1960s. This specialized binary search tree, named an AVL
Tree in honor of its inventors (A delson-V elskii and L andis) is always
kept close to balanced in that the height of the left and right subtrees of the
root do not differ by more than one. The balancing is accomplished by
expanding the Insert and Delete algorithms to rebalance the tree without
having to randomize the nodes and reinsert them back into the tree. The
460

Fetch and Update algorithms are the same as the binary search tree
algorithms previously presented in this chapter.
Figure 7.34 The AVL Balance Factors for Nodes in Balanced and
Imbalanced Binary Search Trees
A parameter called a balance factor is used to decide when to
rebalance the tree. Each node in the tree has a balance factor associated
with it, which is calculated and stored as an additional piece of information
for each node. The value of a node’s balance factor is, by definition, the
difference in the number of levels in the node’s left and right subtrees.
Figure 7.34 presents several trees with the value of each node’s balance
factor shown to its left (in parentheses). The trees are arranged in the
figure such that all the trees in the upper portion of the figure are balanced,
and the trees in the lower portion of the figure are imbalanced.
Examining the trees in the figure, we see that the balance factors of
all of the nodes in the balanced trees are 1, 0, or −1, whereas this is not the
case for the imbalanced trees. This characteristic, which is a consequence
of the definition of a balanced tree and the definition of a balance factor, is
typical of all balanced and imbalanced AVL trees.
To see why, consider the complete trees shown in Figures 7.3 f and
7.34 c . Because each level of a complete tree is fully populated, the left
and right subtrees of every node in the tree contain the same number of
levels. As a result, all nodes in a complete tree (which is one type of
461

balanced tree) have balance codes of 0. The other type of balanced tree is
one in which a nonfully populated level is added to a complete binary tree
(see Figure 7.34 d ). All of the nodes at the added level are leafs and so
their balance factors are zero. The addition of the new level can only cause
the balance factors of the nodes at the other levels of the tree to increase or
decrease by at most one, since only one level was added to the tree. The
balance factor of all of these nodes was zero before the new level was
added, and therefore their new balance factors must be in the range ±1.
In AVL jargon, a node with a +1 balance factor is said to be left high ,
a node with a 0 balance factor is said to be even high , and a node with a
−1 balance factor is said to be right high . Left high, even high, and right
high are denoted LH, EH, and RH, respectively. The AVL tree algorithms
check to make sure that an insertion (or a deletion) has not caused the
balance factors of the nodes in the tree to exceed ±1; that is, all the nodes
are either LH, EH, or RH. If this is the case, the tree is still balanced, and
the operation is complete.
When a node is inserted or deleted, we start at the node’s parent and
work our way up the tree looking for a node with an unacceptable balance
factor (other than LH, EH, or RH). This node and its descendants are
subjected to a rotation to rebalance the tree. After the rotation, we continue
to work our way up the tree, performing other rotations wherever
necessary.
For example, consider the insertion of a node whose key value is 33
into the tree shown in Figure 7.34 d . Before the insertion, all the nodes are
EH (even high) with the exception of the tree’s root node, 50, which is LH
(left high). When the root node of a tree is left high, we say the tree is left
high. Figure 7.34 g shows our left high tree after key value 33 is inserted
(using the binary search tree insertion algorithm) along with the new
values of the balance factors. Working our way up the tree beginning at the
parent of the inserted node we move from key 35, to 40, and finally find a
node with a balance factor out of the range ±1 (the node 50). This indicates
that the tree, rooted by 50, is out of balance. In addition, the insertion of
node 33 has also made the left subtree of 50 left high (i.e., node 40’s
balance factor has changed to +1). When this is the case we say that the
tree has become left-of-left ; the operation created a left high subtree in a
left high tree.
To restore balance to a left-of-left tree, a right rotation is performed
on node 50. The rotation is a right rotation because there are too many
levels in 50’s left subtree as indicated by its +2 balance factor. Beginning
462

with the tree shown in Figure 7.35 a , the right rotation positions the nodes
as shown in Figure 7.35 b . (This leaves 40’s original right child 47
without a parent, which is a problem we will resolve momentarily.) The
root node 50 is moved (rotated) rightward and downward making 40 the
new root of the tree, and 50 is now its right child. Since the rotation moves
the left subtree of 50 up one level, the resulting tree has one less level. The
balance factors of the right rotated tree, as shown in Figure 7.35 b , are
now in the range ±1. This means this tree is in balance.
For some trees, this rotation completes the rebalancing operation. But
since node 40 had a right child (47, or possibly a larger right subtree) in
the original tree, the right rotation algorithm continues. The right rotation
of any node (in our case 50) always causes it to have no left child. In
addition, since it was originally to the right of the orphaned node (47) or
subtree, it must be larger than the orphaned node (or all of the nodes in the
orphaned subtree). Therefore, node 47 (or the subtree whose root 47) can
become the left subtree of 50 (Figure 7.35 c ) and the resulting tree is still
a binary search tree. This repositioning of the subtree whose root is 47
completes the rebalancing of our tree. The balance factors of the resulting
tree shown in Figure 7.35 c , verifies that the right rotation of node 50 has
produced a search tree that is once again balanced.
Figure 7.35 The AVL Single Right Rotation to Rebalance a Left
High Tree with a Left High Subtree (the Tree of Figure 7.34d After the
Node with Key Field 33 Is Inserted)
The AVL tree insertion algorithm includes three other rotation
algorithms to deal with imbalanced trees whose balance factor
distributions are different than that depicted in Figure 7.35 a . The
rebalancing procedure described is used when an operation on a node
creates a left high subtree (the subtree rooted by node 40 in Figure 7.34 g )
of a previously left high tree (the tree rooted by node 50 in Figure 7.34 d ).
The other three cases are when the operation creates a
463

• right high subtree in a right high tree,
• right high subtree in a left high tree, or
• left high subtree in a right high tree.
When the operation creates a right high subtree in a right high tree (as
depicted in the left and center portions of Figure 7.36 , a rebalancing
process similar to the one described above is performed except that the
right rotation is replaced with a left rotation (see the center and left
portions of Figure 7.36 ).
When the tree’s imbalance results from an operation that produces a
right high subtree of a left high tree, or a left high subtree of a right high
tree, then two rotations are necessary to restore an AVL tree to a balanced
condition. Consider the left high search tree depicted in the upper left
portion of Figure 7.37 . After key 44 is inserted (see the upper right portion
of Figure 7.37 ), node 50’s balance factor become +2 requiring a rotation.
Unlike the situation depicted in the left portion of Figure 7.35 , this time,
the left subtree of 50 has gone right high. A right high subtree of a left
high tree has been created requiring two rotations. First, a left rotation is
performed on node 40 resulting in the tree depicted in Figure 7.37 c . Then
a right rotation is performed on node 50 resulting in the tree depicted in
Figure 7.37 d .
Figure 7.36 The Rebalancing of a Right High Tree with a Right High
Subtree
464

Figure 7.37 Double Rotation to Rebalance a Left High Tree with a
Right High Subtree
When the operation creates a left high subtree in a right high tree, first
a right rotation is performed on the root of the subtree, and then a left
rotation is performed on the root of the tree.
Both the Insert and Delete algorithms are usually expressed, and
coded, recursively. As previously mentioned, the Fetch and Update
algorithms of an AVL tree are the same as the binary search tree
algorithms previously developed in this chapter.
Red-Black Trees
Another form a self balancing binary tree is a Red-Black tree, which
was invented by Rudolf Bayer in 1972. Originally named a Binary B-tree,
it has become one of the more popular self balancing tree algorithms.
Red-Black trees and AVL trees share many characteristics. Their
Fetch and Update algorithms are the same (the binary search tree
algorithms previously presented in this chapter), and both trees incorporate
465

rotations into the binary search tree’s Insert and Delete algorithms to
resolve imbalance. Neither an AVL tree nor a Red-Black tree is perfectly
balanced, although an AVL tree typically comes closer to a balanced tree
than a Red-Black tree. In fact, a Red-Black tree can contain twice as many
levels (2 * log2 (n + 1) levels) as a balanced tree. That being said, the Red-
Black tree does outperform the AVL tree because its rebalancing process
is more efficient. The Java API structure TreeMap is a Red-Black tree.
The tree gets its name from the fact that the algorithm assigns each
node in the tree one of two colors, red or black. With the exception of the
root node which is always black, during the insertion or deletion of a node
the color of the other nodes in the tree can—and often do—change during
the rotations these operations perform. Aside from the normal ordering
that all binary search trees comply to (small keys in the left subtree, large
keys in the right subtree) Red-Black trees must comply to additional
ordering conditions. If a node is red, its children must be black, and every
path from a node to a null link (a leaf’s left or right null reference) must
contain the same number of black nodes. All of the conditions of a Red-
Black tree are summarized as follows:
Ordering Conditions of a Red-Black Tree
1. Every node in the tree must be red or black.
2. The root of the tree is always black.
3. If a node is red, its children must be black.
4. Every path from a node to a null link (a leaf’s left or right null
reference) must contain the same number of black nodes.
5. The tree must be a binary search tree. (Every node in a node’s left and
right subtree is less than and greater than it, respectively.)
In an AVL tree when the balance factors exceed ±1, balancing
rotations are performed. In a Red-Black tree, lack of compliance with
Conditions 3 and 4 initiates rotations that keep the tree near balanced. For
example, an implication of Condition 4 is that a newly inserted node must
be red or it will initiate a balancing rotation. If it were black, then the black
path to it would be increased by 1, making the path longer than the other
black paths in the tree. This would violate Condition 4. If the new node
were inserted as a red node (and thus not initiate a rotation), when a
subsequent node was inserted as its child, a rotation would be initiated.
This rotation would be necessary because the newly inserted node would
have to be red to comply with Condition 4, but its parent is also red which
466

violates Condition 3.
The rotations performed by Red-Black trees are somewhat similar to
the rotations performed by AVL trees, but are more complicated because
they also involve color changes. The important feature of Red-Black trees
is that the coloring of the nodes adds sufficient information to the tree to
initiate color adjustments and rotations as we move down through the tree
looking for the insertion point (or the node to be deleted). These rotations
keep the tree near balanced making the additional upward traversal
required to balance an AVL tree unnecessary. The fact that Red-Black
trees perform only one traversal during their Insert and Delete operations
is what gives Red-Black trees their performance advantage.
A further discussion of the rotational procedures and color changes
performed during the insertion and deletion downward traversals is beyond
the scope of this text. However, our discussion of the details of the AVL
rotations provide a good foundation for future study.
7.3.6 Array Implementation of a Binary Search Tree
For some applications, an array-based implementation of a binary tree
offers a more efficient means of representing a binary tree than the linked
approach utilized up to this point in this chapter. Rather than storing the
address of the client’s information Listing object in a TreeNode object,
which also stores the address of the left and right children (see Figure 7.10
), in this alternate implementation the Listing ‘s address is stored in an
element of an array of references variables (see Figure 7.38 ).
The location of the information at the root of the tree is always stored
at index 0 of the array. Therefore, for the array depicted in Figure 7.36 the
root Listing object has a key value of 50. Since each element of the array
of reference variables can only store one address, that of a single Listing
object, the implementation makes an assumption about where in the array
the addresses of the children are stored. The assumption, called the “2i + 1,
2i + 2 rule,” is stated as:
Children Locations in the Array Implementation of a Binary
Tree
For a Listing whose address is stored at index i of the array, its left
child ‘s address will be stored at index 2i + 1, and its right child ‘s address
will be stored at index 2 i + 2.
467

Figure 7.38 Array Implementation of a Binary Tree
Figure 7.39 The Binary Tree “View” of the Array Presented in
Figure 7.38
For example, in Figure 7.38 we can locate the addresses of the two
children of the root (stored at index i = 0) by examining the contents of
elements 1 = 2 * 0 + 1, and element 2 = 2 * 0 + 2. Thus, the root’s children
are the Listings with key values 40 and 63. Since the address of the node
with key value 40 is stored at element 1 of the array, its two children’s
addresses are stored in elements 3 = 2 * 1 + 1, and 4 = 2 * 1 + 2. Using
this rule, any array can be viewed as a tree. The tree represented by the
array shown in Figure 7.38 is depicted in Figure 7.39 .
When the tree is completely empty, all elements of the array are set to
null . In addition, if a Listing does not have a left or right child, then the
reference variable used to store the address of the nonexistent child is null
. Again referring to the array implementation of a binary tree depicted in
Figure 7.38 , we know that the Listings with keys 63, 35, and 47 do not
have any children since the array elements that the 2i + 1, 2i + 2 rule
associates with their children, elements 5, 6, 7, 8, 9, and 10, all contain
null values.
468

Figure 7.40 Four Binary Trees and Their Array Representations
Not only can an array be viewed as a binary tree using the 2i + 1, 2i +
2 rule, but the rule is often used to store a binary tree in an array. For
example, consider the four binary trees shown in Figure 7.40 . The array
representations of the four trees are given just to their right. In each of the
arrays, all of the elements after element 7 contain a null value. To build
the arrays, we begin by placing the address of the root into element 0 of
the array. Then to place the address of the other Listings into the array, we
use the 2i + 1, 2i + 2 rule to calculate their indices. Referring to Figure
7.40 c , since B is the right child of A, its address is stored in element 2 = 2
* 0 + 2, and since C is the right child of B, its address is stored in element
6 = 2 * 2 + 2. An equivalent—and often easier—method for placing the
addresses into the array is to go across the levels of the tree from left to
right, starting at level 0, and then working our way down the levels of the
tree placing the Listing objects’ addresses into sequential elements of the
array. Referring to Figure 7.40 b , A’s address would be stored in element
0, B’s in element 1, C’s in element 2, D’s in element 3, etc. If a child were
469

missing, the element would receive a null value (see element 3 of Figure
7.40 a ).
Mapping the Listing objects into array elements by working our way
across the tree levels from left to right beginning at level 0, is much more
convenient then using the 2i + 1, 2i + 2 rule. However, this is only a
graphical technique. Whenever we are coding an array implementation of
a binary tree, we always use the 2i + 1, 2i + 2 rule to determine the index
of a node. This distinction will become clearer when we develop the
pseudocode of an array-based implementation in the next section and when
we study the Heap Sort algorithm in Chapter 8 (which uses the array
implementation to store a binary tree).
Operation Algorithms
In this section we will develop the pseudocode algorithms for the
Insert and Fetch operations of an array-based binary search tree structure.
As we will see, one advantage this scheme has over the linked approach is
that these algorithms are simpler, and when the tree is balanced their
performance is better. A disadvantage is that the Delete operation is very
inefficient from either a speed or density viewpoint, depending upon how
it is implemented. As a result, this structure is suited for applications
where deletions are not necessary, or they are held to a minimum. With
this in mind, we will develop an array-based tree structure that just
supports Insert and Fetch operations. After analyzing these algorithms we
will discuss the problems with the Delete algorithm and the alternatives for
implementing it.
Insert Operation As with the array-based structures studied in
Chapter 2 , an array named data and an integer variable named size will be
allocated as part of the structure. The array will store the references to the
client’s nodes inserted into the tree, and the variable size will store the
number of elements in the array. As previously mentioned, when the tree is
empty all the elements of the array are set to null . Figure 7.41 shows the
structure in its initialized state.
The Insert algorithm for this implementation is fundamentally the
same as the Insert algorithm for the linked implementation depicted in
Figures 7.7 and 7.13 . When the tree is empty, the inserted node becomes
the root node. Otherwise, we repeatedly move into the left or right subtree
depending on the relationship between the root node’s key and the key of
the node to be inserted. The 2i + 1, 2i + 2 rule is used to determine the
location of the root of the subtree. The search for an insertion point
470

continues until a null reference is located in the array or the calculated
index is out of bounds. In the former case, the node is inserted and the
algorithm returns true , otherwise it returns false . Assuming the listing to
be inserted is referenced by the variable newListing , the pseudocode of
this process is given as follows:
Figure 7.41 The Array-Based Binary Tree in its Initialized State
Lines 2–7 perform the search for the insertion point assigning the
variable i to either 2i +1or2i + 2. When the tree is empty, data[0] is null ,
471

the loop does not execute, i remains 0, and the first node inserted into the
tree is stored in data[0]. Line 8 checks to make sure the loop did not end
on an out-of-bounds condition and if not, Lines 10–12 perform the
encapsulated Insert operation.
It should be noted that there are two ways for this structure to return
false , indicating that the node could not be added to the structure. Both are
detected by the Boolean expression i < size on Line 2. The first way, is that every element in the array is used. The second way, is that the node insertion order is such that the tree is imbalanced or highly skewed as is the tree depicted in Figure 7.34 e . When this happens, the array is far from full when the index calculated by the 2i + 1, 2 i + 2 rule is out of bounds. For example, the Insert operation will return false when an attempt is made to insert the eleventh (and largest) node into a highly right skewed structure whose array size is 1023 elements. This unfortunate characteristic will be discussed in more detail when we consider the density of this structure. Fetch Operation The Fetch operation uses the same search technique as the Insert algorithm except that the search can now end on an additional condition, the node is found. A null reference encountered during the search indicates that the given key is not in the structure. The pseudocode for the Fetch operation follows, which assumes the key of the node to be fetched is targetKey . The Boolean condition on Line 2 has been expanded so that the search ends if the given key is found. The test for an unsuccessful search 472 on Line 8 has also been expanded to include the case when the index is not out of bounds but the node is not in the structure, data[i] == null . A successful search ends on Line 11, where a deep copy of the located node is returned to the client. Implementation The implementation of this structure will be left as an exercise for the student. With the exception of the class' constructor, the implementation is a line-for-line translation of the pseudocode algorithms previously presented. The constructor requires some further discussion. As we mentioned when we discussed the pseudocode of the Insert operation, the density of this structure can degrade rapidly if the tree is imbalanced. However, there is a range of imbalance within which the density is good and, if the speed of the structure is acceptable to the client, the structure can be quite serviceable. Therefore, the constructor's parameter list will include all the information necessary to size the array in a way that the density of the structure meets the client's minimum needs. The size of the array, N , will be calculated by the constructor as where: N is the size of the array (N >= n ),
D BSA is the desired density (D BSA <= (1 / (1 + 4 / w)), n is the maximum number of nodes to be stored in the structure, and w is the node width in bytes. The derivation of the equation will be presented when we discuss the performance of this structure. Since there are three independent variables in this equation, the class' constructor will contain three parameters. Assuming the name of the array is data and the name of the class is BinaryTreeArray, the code of the constructor is given below. 7.3.7 Performance As we have indicated, the performance of this structure can be very 473 good or very bad depending on how well the tree is balanced. Generally speaking, as the tree becomes more imbalanced its speed degrades to that of a singly linked list, as was the case for our linked implementation. Unlike the linked implementation, however, the density of this structure also degrades rapidly as the tree becomes more and more imbalanced. Speed We will first examine the speed of the Insert algorithm. Again, assume that an optimizing compiler will make use of the CPU's registers to minimize the number of memory accesses required to fetch (or assign) local, and other, variables used several times in the algorithm. Lines 2 and 3 require two memory accesses: one to fetch the variable data[;] and one to fetch its key. (The variables size and newListing.key would be stored in CPU registers during the loop's execution.) Lines 4 and 6 do not require any memory accesses because the variable i would also be stored in a CPU register. Therefore, a total of two memory accesses are performed during each iteration of the loop, which executes once per level of the tree. If the tree is highly skewed, there will be n levels in the tree and the loop will execute an average of n / 2 times. If the tree is balanced, the loop will execute, on the average, <= log2 (n + 1) times (as shown is Section 7.3.2 ). Therefore, the number of memory accesses performed by Lines 2–7 will be between 2log2 (n − 1) and 2n depending on how well the tree is balanced. The remainder of the algorithm (Lines 8–12) accesses memory one additional time to write the reference to the cloned node into data[i] (Line 11). This brings the total number of memory accesses for the Insert algorithm to between 1 + 2log2 (n − 1) and 1 + 2n /2 depending on how well the tree is balanced (between O(log2 (n )) and O(n )). The search process for the Fetch algorithm (Lines 2–7) is identical to that of the Insert algorithm except that the memory access required to fetch the key of the node referenced by data[i] is performed on Line 2 instead of Line 3. Thus, the search portion of both algorithms requires the same number of memory accesses. The remainder of the Fetch algorithm does not require any additional memory accesses. Therefore, the total number of memory accesses for the Fetch algorithm is between 2log2 (n − 1) and n depending on how well the tree is balanced (between O(log2 (n )) and O(n )). 474 Density To calculate the density, D, we recall that: The information bytes are simply the product of the number of nodes, n , and the number of bytes per node, w. The total bytes allocated the structure is the sum of the information bytes and the overhead bytes. Assuming there are N elements in the array, and that reference variables occupy 4 bytes, the overhead is 4N . Thus, the density of the array-based structure, DBSA is where: N is the size of the array, n is the maximum number of nodes to be stored in the structure, and w is the node width in bytes. The value of N in the above formula for density depends on how well balanced the tree is. Its minimum value, which produces the maximum density for this structure, is n because we must provide an element in the array to store the address of each of the n nodes in the tree. However, N can be considerably larger than n . Consider the trees shown in Figure 7.40 a , c , and d . The parent- child relationships in these trees are such that there are elements of the array that are not used, and so N > n . The worst case is when the tree is
highly skewed to the right. In this case, assuming the tree contains n nodes
(and therefore n levels), the array must be sized to N = 2n − 1 elements.
For example, the tree shown in Figure 7.38 c has only 3 (= n ) nodes but
the array must be sized to 7 (= N ) elements.
475

Figure 7.42 Density of the Linked and Array-Based Implementations
of a Binary Tree
The best case scenario occurs when the tree is left balanced or
complete (see Figure 7.40 b ). This is the case when N = n . Thus the range
of N for an array-based tree containing n nodes is n ≤ N ≤ 2n − 1.
Substituting these limiting values for N into this equation, this density of a
left balanced tree, DLB , and that of a right skewed tree, DRS becomes
and
Because the term 2n dominates the terms in the denominator of the
equation for DRS , it approaches zero even for moderate values of n .
Therefore, from a memory overhead viewpoint, the array-based
implementation is not practical for trees that are far out of balance.
However, when the tree is left balanced (or complete), the density of the
array-based structure is better than the linked implementation. Figure 7.42
compares the density of the linked implementation (from Figure 7.24 ) and
the array-based implementation for left balanced (or complete) trees.
Before we end our discussion of density, it is useful to gain some
insights into the range of imbalance for which the array-based structure
still offers good density. As derived previously, the density of this
structure is
Figure 7.43 shows the variation in density with the parameter N / n ,
the ratio of the size of the array to the number of nodes in the structure, for
various values of node widths. As shown in this figure, densities above
0.80 can still be obtained in array-based binary trees when N / n = 32 as
long as the node widths are greater than 640 bytes.
The ratio of N / n is related to the degree of imbalance in the tree, in
that, when N / n is 2, we have provided twice as many “spots” in the tree
as will be occupied. This means that the tree is one level larger than it
would have to be if the tree were balanced. When N / n = 4, we have
provided four times as many spots in the tree as will be occupied. This
means that the tree is two levels larger than it would have to be if the tree
were balanced. Following this logic, the number of extra levels in a tree
containing n nodes is log2 N / n . Therefore, for a given node width and
476

required minimum density, the log2 of N / n (as determined from Figure
7.43 ) can be used as the upper bound of range of imbalance for which the
density meets the required minimum. For example, for a node width of 640
bytes and a minimum required density of 0.84, Figure 7.43 indicates that N
/ n must be 32. Thus, the tree can be imbalanced by as many as 5 (= log2
32) levels and the density of the structure will still be within the desired
range ≥ 0.84).
Figure 7.43 Variation in Density of the Array-Based Binary Tree
with N/n and Node Width
Another handy formula is one that can be used to determine the
degree of imbalance to achieve a density of 0.8 for a given node width.
Substituting a density of 0.8 into the density equation and solving for N / n
we obtain
This function is plotted in Figure 7.44 . Finally, from the density
equation we can determine the size of the array for a desired density, given
the node width, and number of nodes as
It should be noted that there is an upper limit to the desired density of
any array-based structure because even if all elements of the array are
used, N / n = 1, the overhead is nonzero (4 * N ). Substituting N = n in the
previous equation and solving for the density we obtain
477

Figure 7.44 Conditions for a Density of 0.8 for an Array-Based
Binary Tree
Table 7.6 presents the performance of the array-based binary search
tree and includes the performance of the previously studied structures for
comparative purposes. When balanced, its speed is only exceeded by the
hashed structures, and it is the best performing structure from a density
viewpoint. However, the structure does not support the Delete and Update
operations.
Delete Operation Alternative
If we were to use the linked implementation’s Delete algorithm for the
array-based implementation, its speed would be very slow for all but the
first case of the algorithm: deleting a leaf node. To see why, consider the
case where we want to delete the root node from the tree shown in Figure
7.40 d . The node B would become the root node so its reference would
have to move from data[1] to data[0]. Then, to maintain the parent-child
relationship, C’s reference would have to be moved to data[1], and D’s to
data[3]. For all but very small trees, the number of memory accesses
required to maintain the parent-child relationships make the linked
implementation’s Delete algorithm prohibitively slow for an array-based
tree.
An alternative algorithm adds a Boolean field to each node to indicate
that a node has been deleted from the structure. The Insert algorithm sets
this field to false . To delete a node, this field is simply set to true . The
search portions of Delete and Fetch are modified to ignore any node whose
deleted field is true . The downside of this scheme is that the storage
occupied by a deleted node is never reclaimed. However, it does offer a
viable scheme for applications where only a few nodes will be deleted.
The implementation is left as an exercise for the student.
478

7.3.8 Java’s TreeMap Data Structure
The Java class TreeMap, contained in the package java.util, is an
implementation of a Red-Black tree. As implemented, it is an
unencapsulated generic data structure accessed in the key field mode. The
key can be any type of object, however, the key’s class must implement
Java’s Comparable interface. That is, it must contain a method whose
signature is
479

that compares two key objects and determines whether invoking
object’s key is less than (returns a negative integer), equal to (returns zero),
or greater than (returns a positive integer) aKey. The String class
implements this interface. The other alternative is that the key must have a
Comparitor. The wrapper classes for the numeric types (i.e., Byte, Double,
Float, Integer, Long, and Short ) have Comparitors . Therefore, keys
that are instances of Strings or numeric wrappers can be used in
TreeMaps without the need to implement an interface.
The class TreeMap has four constructors. The default constructor
sorts the nodes into the Red-Black tree according to the key’s natural order
(i.e., as determined by the key’s compareTo method). The following
statement declares a TreeMap structure that stores Listing objects in a
Red-Black tree named dataBase , which is ordered based on a String key
:
This declaration uses the Java 5.0 generic type parameters (i.e.,
) to declare the structure dataBase to be a
homogeneous structure that can store only Listing objects whose key field
is a String object.
The TreeMap class’ Insert, Fetch, and Delete operation methods are
named put, get, and remove , respectively. The methods get and remove
return null if the specified key is not in the structure. Descriptions of the
other methods in the class are given in the Java online documentation
(http://java.sun.com/j2se/1.5.0/docs/api/java/util/TreeMap.html ). An
application that uses a TreeMap data structure is given in Figure 5.31 with
the exception that Line 4 of the application would be changed to the above
TreeMap object declaration. The code of the class Listing is presented in
Figure 2.16 .
The following code outputs the keys (String objects ) and the nodes
(Listing objects ) stored in the TreeMap object dataBase in sorted order
based on their key values. The highlighted items are application specific.
EXERCISES
480

http://www.java.sun.com/j2se/1.5.0/docs/api/java/util/TreeMap.html

Knowledge Exercises
1. Give an advantage of a binary search tree structure over:
a. A linked list structure
b. A stack
c. An array-based structure
d. A hashed structure
2. Define the terms:
a. Root node
b. Binary tree
c. Parent
d. Leaf node
e. Level 0 of a tree
f. Highest level of a tree
g. Balanced tree
h. Complete tree
i. Left-balanced
3. What is the maximum number of nodes that can be stored in a binary
tree with 10 levels?
4. What is the minimum number of levels in a binary tree containing
6023 nodes?
5. What is the maximum number of levels in a tree containing 6023
nodes?
6. What is the maximum number of nodes that can be stored in level 16
of a binary tree?
7. If there are nl nodes in level l of a balanced binary tree, give an
expression for the number of nodes at level l + 1 (assuming level l + 1 is
not the highest level of the tree).
8. True or false:
a. The highest level of a complete binary tree contains more nodes than
all of the other levels combined.
b. The root node is always at the highest level of the tree.
c. Level 9 is the highest level in a 10 level tree.
9. State the significance of the word binary in the term binary tree (i.e.,
481

why is a binary tree called a binary tree?).
10. Define the term “binary search tree.”
11. Draw the binary search tree resulting from inserting the nodes with
the integer key values: 68, 23, 45, 90, 70, 21, and 55 into the tree.
Assume key 68 is inserted first, then key 23, etc.
12. The client objects inserted into a Binary Search Tree structure are
not actually stored in a binary tree, true or false?
13. Give the data members of the class TreeNode defined in this
chapter and state what is stored in each data member.
14. Draw a picture of a binary search tree, implemented using the
linked implementation, in its initialized state.
a. Using the standard tree graphic.
b. At the implementation level.
15. Under what conditions are the Fetch and Insert operations
performed on a binary search tree structure fast?
16. State the three cases of the binary search tree Delete algorithm.
17. The linked implementation is used in the coding of a Binary
Search Tree structure. Calculate the structure’s density assuming that it
contains 200 nodes and:
a. Each node contains 10 bytes of information.
b. Each node contains 300 bytes of information.
18. Repeat the above exercise for the array implementation of the
Binary Search Tree, assuming it is:
a. Left balanced.
b. Skewed to the right.
19. A balanced binary search tree stores 1,000,000 nodes. Assuming a
memory access takes one nanosecond, calculate the average time
required to fetch a node from the tree for the:
a. Linked implementation.
b. Array-based implementation.
20. Give the output produced by an NLR output scan of the tree shown
in Figure 7.3 f .
21. State the advantage of an AVL tree over a Binary Search Tree
Structure.
482

22. Give the AVL balance factors for each of the nodes in Figure 7.3 a
.
23. Of the 11 nodes in an AVL tree, 6 of the nodes have balance
factors of 1, 4 nodes have balance factors of 0, and 1 node has a
balance factor of −1.
a. Is the tree balanced?
b. Is the tree complete?
24. An array contains the values 100, 20, 34, 200, 6, 10, and and 31.
Element 0 contains 100, element 1 contains 20, etc. Draw the binary
tree representation of the array.
25. In the array implementation of a binary tree:
a. Where is the root node stored?
b. Where is the right child of the node in element 10 stored?
c. How can we determine if the node stored at element 22 does not have
a left child?
26. True or false, a binary tree containing 20 nodes can always be
stored in a 20 element array?
27. Give the size of an array that could store any tree containing10
nodes.
28. Demonstrate that the algorithm used to delete the node referenced
by C in Figure 7.18 a can also be used to delete the node referenced by
C in Figure 7.18 b .
Programming Exercises
29. Write a method to perform an NLR scan on a binary tree. Assume
the method will operate on trees implemented as arrays. Provide a
driver program to demonstrate that the method functions properly. Use
the trees shown in Figure 7.3 as test cases.
30. A database is to be developed to keep track of student information
at your college. Their names, identification numbers, and grade point
averages will be included. The data set will be accessed in the key field
mode, with the student’s name being the key field. Code a class named
Listing that defines the nodes. The class must comply with the
guidelines that permit student information nodes to be stored in the
fully encapsulated BinaryTree structure discussed in this chapter. As
such, your class should include all the methods in the class shown in
483

Figure 2.16 and include a getKey method. Test it with a progressively
developed driver program that demonstrates the functionality of all of
its methods.
31. Code an application program that keeps track of student
information at your college. Include their names, identification
numbers, and grade point averages in a fully encapsulated,
homogeneous, linked-based binary search tree. When launched, the
user will be presented with the following menu:
Enter:
1 to insert a new student’s information,
2 to fetch and output a student’s information,
3 to delete a student’s information,
4 to update a student’s information,
5 to output all the student information in descending order, and
6 to exit the program.
32. Do Exercise 31 but use an array-based implementation of a binary
search tree. Include only user options 1, 2, 5, and 6. Your program
should allow the user to specify the minimum density of the structure.
33. Expand the program described in Exercise 32 to allow the user to
delete and update nodes.
34. Code the generic version of the linked-based Binary Search Tree
using the generic features of Java 5.0, and provide a driver program to
demonstrate that the method functions properly. The driver program
should declare two binary search tree objects: one to store Listing
objects as defined in Figure 2.16 with a getKey method added to the
class, and the other to store Student objects as described in Exercise 30.
35. Redo Exercise 31 using an expanded version of the linked
implementation of the binary search tree presented in this chapter that
keeps the tree balanced within a client specified tolerance (number of
extra levels).
36. Repeat Exercises 31 using an AVL tree.
1 This is analogous to the implementation of a singly linked list (see Figure 4.9 ) in
which the Node objects were arranged in the linked list, not the Listing objects.
2 This use of the findNode method assumes that we will never insert two Listings with
the same key value into the structure (an assumption used in the development of all data
structures presented in this text).
484

3 It is left as an exercise for the student to verify that the process is identical for the tree
depicted in Figure 7.18 b .
4 This is proven in the appendix using Equations 7.1 and 7.3 .
5 For any tree containing n nodes, n ! different traverses are possible
485

CHAPTER 8
Sorting
OBJECTIVES
The objectives of this chapter are to familiarize the student with the
topic of sorting and the classic sorting algorithms, and to be able to select
the best sorting algorithm for a particular application. More specifically,
students will be able to
Explain the motivation for sorting data sets.
Understand that the speed complexity of a sorting algorithm is
dependent on both the number of comparisons it performs and the
number of data items it swaps.
Understand that the theoretical minimum number of comparisons
required to sort n items is n log2 n , and be able to calculate the
minimum time required to sort n items.
Understand, and be able to implement, the classic sorting algorithms:
Binary Tree Sort, Bubble Sort, Heap Sort, Merge Sort, and Quicksort,
and be able to explain the strengths and weaknesses of each algorithm.
Determine the speed and space complexity of the classic sorting
algorithms (and any other sorting algorithm), and be able to select the
best sorting algorithm for a particular application.
Explain the characteristics of a binary tree that make it a heap, and
understand the algorithm that transforms a binary tree into a heap.
Understand the topic of recursion more fully by examining the
recursive parts of the Merge Sort and Quicksort algorithms.
8.1 Sorting
Sorting is the process of ordering a set of items. The two most
common orderings are ascending order and descending order. In the
context of data structures, it is the process of arranging nodes in an order
486

based on the contents of one of the fields in the nodes. For example, the
nodes that comprise a collection of telephone listings are usually sorted in
ascending order based on the contents of the name field, while the results
of a weight lifting contest would possibly be sorted in descending order
based on the contents of the maximum weight lifted field.
Most often, nodes are sorted for one of two reasons:
1. To produce sorted output listings.
2. To improve the speed of a data structure.
Let us first consider sorted output listings. When nodes are output in
sorted order, not only are they more pleasant to read, but it is also much
easier to find a particular listing from among the printed listings. Anyone
who has used a phone book would agree that an unsorted phone book
would be very difficult to use. The search process would be a time
consuming, manual sequential search, instead of the binary search we all
intuitively use on a standard sorted phone book.
There is another important reason for outputting nodes in sorted
order. Drawing conclusions based on the data presented in sorted order
often becomes self-evident. Consider again the output listing of a weight
lifting contest that is sorted based on the maximum weight lifted field. It
would be obvious who won the competition because the analysis necessary
to determine the winner is the sorting process itself. Sorted listings can
make conclusions self-evident.
Now, let us turn our attention to the second motivation for sorting: to
improve the speed of a data structure. Often the speed of the fetch method
can be improved if the data set is stored in a sorted order. We saw an
example of this in Chapter 2 when we studied the Sorted Array structure in
which the nodes were stored in sorted order based on their key field. This
allowed us to use the binary search algorithm to locate a node which is an
efficient, O(log2 n ), search algorithm. Thus, even if the data set is never
output in sorted order, sorting algorithms are important because they can
speed up one or more of the basic operations performed on a data set.
Considering the advantages of sorting, the question arises: Why aren’t
all output listings and data sets sorted? One answer is that sorting takes
time. However, through the use of efficient sorting algorithms and good
sorting strategies, the additional time needed to sort can be held to
acceptable levels.
As an example of a sorting strategy, consider the following scenario.
487

When a data set is to be output in sorted order, two alternatives are
available: sort the nodes just prior to outputting them, or store the nodes in
the data set in a sorted order. The former approach is attractive if the nodes
are rarely output in sorted order, or if the field on which the nodes are
sorted often changes. If, however, the nodes are often output in sorted
order, then it may be more efficient to store the nodes in sorted order,
especially if the contents of the field on which they are sorted rarely
changes.
In this chapter, we will study several of the classic sorting algorithms
and examine the advantages of each. Some of the algorithms require more
memory than others, while some execute faster than others. It can be
shown, however, that there is a theoretical minimum number of
comparisons required to sort a set of n items, which we will use as a
standard for goodness when we analyze the performance of the algorithms
presented in this chapter. Some data sets exhibit special characteristics
that allow some sorting algorithms to perform faster than the theoretical
minimum. It turns out that these characteristics are not that uncommon, as
we shall see. Aside from speed and memory differences, some of the
algorithms are much easier to code than others. Thus, the selection of the
best sorting algorithm for a particular application is usually based on the
speed and memory constraints of the application, the character of the data
set being sorted, and the coding skills of the programmer.
For each of the classic sorting algorithms discussed in this chapter,
we will:
• Present a pseudocode version of the algorithm.
• Learn how it performs a sort by tracing its execution while sorting a
set of integers.
• Determine its performance, space, and time complexity.
• Discuss particular data set characteristics, if any, under which it
performs best.
• Enter its performance in a summary chart useful in determining which
sorting algorithm to use for a particular application.
Some of the algorithms will be implemented in this chapter, and the
remaining implementations will be left as exercises for the student.
Before we proceed to a detailed discussion of the classic sorting
algorithms, we will expand our discussion of sorting algorithm
performance. As we have previously stated, the measure of a sorting
algorithm’s performance is based on its memory overhead and its speed.
The techniques for determining a sorting algorithm’s memory overhead are
488

the same techniques we have used to determine the overhead of the data
structures studied in the previous chapters. However, the techniques we
used to determine the speed of data structures will require a minor
refinement in order to be applied to sorting algorithms.
8.2 Sorting Algorithm Speed
All sorting algorithms have two features in common. Two data items
are compared in order to determine their relative position in the sorted list,
and data items are swapped (based on these comparisons). Both of these
features require memory accesses that can be counted using the algorithm
analysis techniques discussed in the previous chapters. However, in the
case of sorting algorithms, it is useful to know not only the total number of
memory accesses, but also how many of them were a result of swap
operations and how many of them were the result of comparison
operations. This more detailed level of analysis allows us to better
understand why some algorithms are faster than others. A parameter used
in this analysis is sort effort , which is defined as:
Sort Effort
Sort Effort = SE = number of comparisons required to sort n items.
To illustrate the use of this parameter and our expanded analysis
techniques, consider the following code segment extracted from a sorting
algorithm. The number of items to be sorted is represented by n .
The comparison on Line 3 is executed n times as part of the inner
loop (Lines 3–8), and the inner loop is executed n times inside the outer
loop (Lines 2–9). Therefore, n * n comparisons are performed by this line
of code. Assuming these were the only comparisons performed in the
sorting algorithm, its sort effort would be n 2 . The variable items[j] is
accessed during each of these n 2 comparisons (j changes each pass
through the inner loop) and items[i] is accessed n times (i changes every
pass through the outer loop). As a result n 2 + n memory accesses are
489

associated with the algorithm’s comparisons. Thus, from a Big-O analysis
viewpoint, the number of memory accesses and the sort effort are
equivalent (both O(n 2 )). This is typical of sorting algorithms.
To complete the speed analysis, Lines 4–6 perform a data swap
operation. If the Boolean expression on Line 3 is always false , no swaps
are performed. Conversely, if the condition is always true , n 2 swaps are
performed. The actual number of swaps performed is a function of the data
set (and ultimately the rest of the sorting algorithm). For this example we
will assume that half the time the Boolean condition is true , so n 2 / 2
swaps are performed. During each swap items[j] is overwritten. However,
an efficient translator would store items[i] in a register during the
execution of the inner loop and only write it to memory each of the n times
the inner loop ends. Therefore, a total of n 2 / 2 + n memory accesses are
performed during the swap operations.
Considering both the comparisons and the swaps performed by this
algorithm, its speed is therefore O(n 2 ). More specifically it is 3/2n 2 + 2n
(= n 2 + n comparison accesses + n 2 / 2 + n swap accesses), with the
comparison portion of the algorithm contributing n 2 / 2 more memory
accesses than the swap portion.
8.2.1 Minimum Sort Effort
As previously mentioned, there is an approximate theoretical
minimum number of comparisons required to sort n randomly arranged
items. This number of comparisons is referred to as the minimum sort
effort , SEmin , and is calculated as:
Minimum Sort Effort
The derivation of this formula is beyond the scope of this text.
However, it is useful to know the minimum effort because it can be used to
identify efficient algorithms from a number-of-comparisons viewpoint. For
example, consider the algorithm whose sort effort (number of
comparisons) is O(n 2 ). Since the minimum effort is O(n log2 n ), we
know there is considerable room for improvement.
To make the performance difference between an O(n 2 ) and O(n log2
490

n ) sorting algorithm more tangible, and to demonstrate the need for
efficient sorting algorithms, let us assume the two algorithms are used to
sort 1,000,000 items. In addition, we will assume they are executed on a
system that performs a comparison in one nanosecond, and each
comparison requires one memory fetch that takes 40 nanoseconds. Then,
the time to execute the O(n 2 ) algorithm would be
However, the time to execute the O(n log2 n ) algorithm would only
be
Clearly, if 1,000,000 items were going to be sorted often, we would
not want to use a sort algorithm whose sort effort is n 2 . However, we
should be sensitive to the fact that even sorting algorithms whose sort
effort is equivalent to the theoretical minimum still require a significant
amount of computing time when the number of items to be sorted, n , is
large. This is illustrated in Figure 8.1 , which presents the minimum time
required to sort a set of n items. This figure truly presents an overall
minimum sorting time, since it assumes the sorting algorithm performs n
log2 n comparisons and no swaps. The figure shows that to sort
300,000,000 social security records would require 5.8 minutes.
Figure 8.1 Minimum Sort Time
8.2.2 An Implementation Issue Affecting Algorithm Speed
Before we begin our study of the classic sorting algorithms, we will
examine an implementation issue common to all sorting algorithms that
491

can greatly affect their speed. As we have discussed, the sorting process
involves swap operations. When we are sorting primitives, we simply
swap the contents of the two memory cells used to store the primitives.
However, when we are sorting nodes, the process of swapping the contents
of the memory cells that store the nodes’ member data can be very time
consuming because (unlike primitives) objects can consist of many data
members, each containing a significant number of bytes.
The remedy is to perform a shallow copy of the node objects during
the sorting process. Thus, the contents of the reference variables (4 bytes
each) that store the location of the nodes are swapped, rather than
swapping the contents of the data members of the objects (which contain
multiple groupings of 4 bytes per data member). This is always the
preferred technique when nodes are being sorted.
Figure 8.2 illustrates the difference between the time-consuming deep
copy approach and the more efficient shallow copy approach to
positioning nodes C and B in sorted order. The figure assumes the
references to the nodes are stored in an array; however, the same
advantages will be realized if the node references were stored in a tree
structure.
One last introductory comment is in order before we proceed. When
the items to be sorted are encapsulated inside a data structure, the sorting
algorithm is coded as a member function of the data structure class in
order to maintain the encapsulation of the structure.
8.3 Sorting Algorithms
We will now begin our study of the classic sorting algorithms with
the Binary Tree Sort algorithm. This is a good starting point, since the
algorithm places the items to be sorted in a binary search tree, and we are
already familiar with search trees. Therefore, we can focus most of our
attention on the techniques used to evaluate algorithm performance, rather
than the development of the algorithm itself.
492

Figure 8.2 Two Techniques for Swapping Objects C and B During a
Sorting Algorithm
8.3.1 The Binary Tree Sort
The Binary Tree Sort algorithm is basically the algorithm used in
Chapter 7 to insert nodes into a binary search tree. Assuming we are
sorting n items, the algorithm is:
To illustrate the algorithm, let us assume we are to sort a group of 10
integers given in the order: 50, 40, 47, 63, 55, 43, 70, 80, 35, and 68. The
integer 50 would become the root node as per Step 1 of the algorithm (see
Figure 8.3 a ). The second integer, 40, would then be compared to 50 (Step
3), and since it is less than 50 we would proceed to the left (Step 3.1). Step
4 of the algorithm would cause 47 to be inserted as 50’s left child (see
Figure 8.3 b ).
493

Figure 8.3 Progression of the Binary Tree Sort Algorithm when
Sorting the Integers 50, 40, 47, 63, 55, 43, 70, 80, 35, and 68
Next, 47 would be compared to the root value, 50. Again we would
proceed to the left since 47 is less than 50 (Step 3.1). Since there is a node
to the left of 50, we would compare 47 to that node, 40 (Step 3). Since 47
is greater than 40, this time we would proceed to the right (Step 3.2). Step
4 of the algorithm would cause 47 to be inserted as 40’s right child (see
Figure 8.3 c ). The growth of the tree during the sorting of the first six
integers is illustrated in Figures 8.3 a -f , with the final sorting shown in
Figure 8.3 g .
Once the items are inserted into the binary tree, they can be listed in
either ascending or descending order using the tree traversal algorithms
discussed in Chapter 7 , Section 7.2 .
494

Figure 8.4 The 10-Level Tree Resulting from Sorting the Integers 80,
70, 68, 63, 55, 50, 47, 43, 40, and 35 with the Binary Tree Sort
Speed
Now let us examine the speed of the algorithm. The speed of this
algorithm is dependent upon how well the binary tree is balanced, because
as the items are inserted into the tree, one comparison is made at each level
of the tree (Step 3 of the algorithm). As shown in Chapter 7 (Equation 7.3
), a complete binary tree can stores n items in a log2 (n + 1)
1 level tree
(see Figure 7.4 ), while an imbalanced skewed tree stores n items in an n
level tree (see Figure 8.4 ). Whether or not the tree is balanced depends on
the order in which the items are processed by the algorithm. If they are
processed in a random order, as when Figure 8.3 was generated, the tree
will usually be balanced (see Figure 8.3 g ) or close to balanced. However,
if the integers are processed in sorted order, the tree will be highly skewed
to the right or left (see Figure 8.4 ).
As a result, the speed of this algorithm varies between the speed
required to sort a random set of items (which we will designate SEBTmin )
and the speed to sort an already sorted set of items (SEBTmax ). Let us
begin by determining the performance of the algorithm when sorting a set
of items that are already sorted.
When the items to be sorted are already sorted in descending order,
the tree is skewed to the left, as shown in Figure 8.4 . The sorting process
495

proceeds in the following way. The first item entered becomes the root
node. The second item is compared to the first item and then inserted into
the tree. Thus, one comparison is required to insert the second item. The
third item is initially compared to the first item, then compared to the
second item, and then inserted into the tree. Thus, two comparisons are
required to insert the third item. Continuing in this way, three comparisons
are required to insert the fourth item, four comparisons are required to
insert the fifth item, etc. Therefore, the total number of comparisons for
the skewed tree—which is the maximum sort effort for the Binary Tree
Sort—is
Figure 8.5 A Four-Level Tree Formed by Sorting the 15 Integers: 50,
40, 47, 63, 55, 43, 70, 80, 35, 68, 37, 49, 61, 51, and 10 using the Binary
Tree Sort
The minimum sort effort for the Binary Tree Sort, SEBTmin occurs
when the items are processed in an order that produces a balanced binary
tree. Figure 8.5 presents such a tree, which is the result of processing 15
items in the order shown in the title of the figure.
To derive the minimum sort effort, let us as sume that n items are
sorted to form a complete binary tree. Then the highest level of the tree
contains ½(n + 1) items. Since each level in a balanced complete binary
tree contains twice as many items as the level below it, the level just below
the highest level contains ½(½(n + 1)) = ¼((n + 1) items, the level below
that contains ½(¼(n + 1)) =
(n + 1) items, etc.
The level number of the highest level of a balanced complete binary
tree is log2 (n + 1) − 1, which is also the number of comparisons required
to place a single item at that level. To place a single item at one level
496

below the highest level requires one less comparison, or log2 (n + 1) − 2
comparisons; to place an item at the next lower level requires log2 (n + 1)
− 3 comparisons; etc.
If N is the number of items at a level of the tree, and C is the number
of comparisons required to place a single item at that level, then the total
number of comparisons necessary to fill a level of the tree is N * C .
Therefore, the total number of comparisons required to place all the items
in the tree containing n items, T n , is the sum of N * C for each level of the
tree:
where
It can be shown that the series a and b converge to 1 and 2,
respectively, as the number of terms in their equations increases. Since
there is one term in each series for each level in the tree (except level 0
since no comparisons are necessary to place an item at level 0), and the
number of levels in a tree containing n items is log2 (n + 1), there will be
log2 (n + 1) − 1 terms in these two series. For values of n greater than
18,434, both series are within 1% of their terminal values. Therefore, we
can approximate T n as
Since the total number of comparisons required to sort n items is, by
definition, the sort effort, a good approximation of the sort effort of the
Binary Tree Sort when sorting n items that produces a balanced complete
binary tree would be:
Appendix D contains a table of calculations comparing the minimum
sort effort calculated using the above approximation and the actual
497

minimum sort effort. Aside from demonstrating good agreement, an
examination of the numbers presented in the table can aid in the
understanding of the minimum sort effort derivation.
Having derived expressions for the minimum and maximum sort
efforts of the algorithm, we can express the range of the sort effort as:
Sort Effort of the Binary Tree Sort
The algorithm’s speed is close to the theoretical minimum when
sorting a random set of items (resulting in a balanced binary tree) and very
slow, O(n 2 ), when sorting an already sorted set of items. However, the
speed can be improved when sorting an ordered, or almost ordered, set of
items by processing the items through the algorithm in random order.
Memory Overhead
Assuming the linked implementation of a binary tree, the memory
overhead required by this algorithm—above that necessary to hold the
items being sorted—is the storage required for the reference variables that
point to the location of the left and right children. Since there will be n
items in the tree, the algorithm requires 2n reference variables of
additional storage. Assuming each reference variable occupies 4 bytes, the
total overhead for the algorithm is 8n bytes.
If the binary tree is implemented using an array, and the items to be
sorted are primitives, then the primitives can be sorted within the array
used to store the items. If the tree is balanced, then there are no unused
elements in the array, and the overhead is 0. However, if the tree is
skewed, then there will be n levels in the tree and (as we have seen in
Chapter 7 ), (2n − 1) −n elements of the array will be unoccupied. This is
an unacceptably large amount of overhead. For this reason, the linked
implementation is used in this sorting algorithm.
Table 8.1 summarizes the performance of the Binary Tree Sort
algorithm, its speed, and its memory overhead. As noted in the comments
column, this algorithm is fast when the items to be sorted are introduced
into it in a random order because the binary tree it produces is balanced. In
this case, the sort effort of the algorithm approaches the theoretical
minimum, O(n log2 n ). However, when the items the algorithm processes
498

are already sorted, the algorithm produces a highly skewed tree, and its
performance is extremely slow, O(n 2 ).
As indicated in Table 8.1 , the memory overhead of this algorithm is
8n bytes (because it normally uses the linked implantation of a binary
tree), which is high compared to the other sorting algorithms presented in
this chapter.
8.3.2 The Bubble Sort
The Bubble Sort is the simplest sorting algorithm presented in this
chapter. For data sets that are already sorted, or close to sorted, it offers
good performance. Because of its simplicity, it is easy to code and,
therefore, is also used for randomized data sets with few members.
This algorithm, like all sorting algorithms, executes its sorting
process inside a loop. In the jargon of sorting, each iteration through this
loop is referred to as a “pass through the sorting algorithm.” Just as gas
bubbles rise to the surface of a liquid, during each “pass” through this
algorithm the smaller items rise, or bubble upward, toward the top of the
array of primitive or reference variables. Thus, the name Bubble Sort.
Each pass places one item into its final position in the array. When
coded to sort items in ascending order, the first pass places (or bubbles) the
smallest item into element 0; the second smallest item is bubbled into
element 1 during the second pass; the third smallest item is bubbled into
element 2 during the third pass; etc. Figure 8.6 presents the contents of an
array of integers after the end of each of the first four passes through the
algorithm. The shaded cells in the table indicate the integers that have been
placed in their sorted position in the array after each pass.
499

Figure 8.6 Results of the First Four Passes Through the Bubble Sort
when Sorting an Array of Integers
Figure 8.7 Changes to an Array of Integers during the First Four
Passes Through the Bubble Sort
At the beginning of each pass through the algorithm, two integers, b
and t, are set to the lowest two indices of the array. For example, when
sorting seven items, b would be set to 6 and t would be set to 5. The
elements at these indices are then compared. If they are not in sorted order,
their positions in the array are swapped or flipped . Regardless of whether
or not a flip is performed, the integers b and t are decremented (b–, t–),
and a comparison is made to determine if the two elements at these
incremented indices should be flipped. This process continues until the
index stored in t reaches the portion of the array already sorted on previous
passes (i.e., t stores the index of the item sorted on the previous passes). As
a result, one fewer comparison is made on each successive pass through
the algorithm.
This process is depicted in Figure 8.7 , which presents the details of
the changes to the contents of the array of integers presented in Figure 8.6
during the first four passes through the algorithm. Moving from left to
right as the algorithm proceeds, the shaded cells in each column of the
500

array indicate the two integers that are compared as a pass proceeds. For
example, as part of the first pass through the algorithm, 9 and 1 are
compared, then 1 and 3 are compared, then 1 and 10 are compared, etc.
The comparison of 1 and 3 in the second column of the table initiates the
first flip. These two integers are shown in their new positions in the third
column of the table.
It turns out that for the initial ordering of the integers presented in
Figure 8.6 , the entire array is, in fact, sorted after four passes through the
algorithm. This is not always the case. As previously stated, the algorithm
only guarantees that one item is correctly positioned during each pass.
However, for some data sets, several items are swapped into their final
positions during a single pass. For example, the integers 10, 9, 8, and 3
were all positioned in their final locations during pass 3.
To take advantage of the fact that sometimes the algorithm completes
the sort “early,” a Boolean variable flip is set to false before each pass
through the algorithm. Then whenever a flip is performed, the variable flip
is set to true . If, at the end of a pass, flip is still true , no two adjacent
elements of the array were out of sorted order, and therefore, the entire
array is sorted. At this point the algorithm terminates. Assuming n is the
number of items to be sorted, the algorithm is:
We will now consider the speed of the bubble sort algorithm.
Speed
Like the Binary Tree Sort algorithm, there are particular
characteristics of the data to be sorted that result in very fast or very slow
Bubble Sort speeds. Ironically, one case that was very slow for the Binary
Tree Sort—when the information to be sorted is initially in ascending
501

order—is very fast for the Bubble Sort.
The sort effort for any sorting algorithm is the sum of the number of
comparisons performed during each pass of the algorithm. For this
algorithm the sum is easily deduced by examining Figure 8.7 , in which 7
(n = 7) integers were processed through four passes of the algorithm. Since
the pairs of shaded cells in the figure represent the comparisons, we simply
need to count them to determine the comparisons for each pass and then
total these to obtain the sort effort. Pass 1 performed six comparisons; pass
2, five comparisons; pass 3, four comparisons; and pass 4 performed three
comparisons.
Figure 8.8 Number of Comparisons per Pass when Processing n = 7
Items Through the Bubble Sort
The number of comparisons for each of the four passes is given in
Figure 8.8 , which includes an extrapolation out to the sixth pass. Although
six passes are not necessary to sort this set of integers, six passes is the
maximum number of passes necessary to sort seven items using the
algorithm. Intuitively, most students conclude seven passes would be the
maximum number of passes necessary to sort seven items. However, the
flip performed on the sixth pass (assuming it is necessary) properly
positions the next-to-the-largest and the largest item in the array.
When the character of the seven items to be sorted are such that six
passes through the algorithm are required to complete the sort, then the
data presented in Figure 8.8 indicates that the total number of comparisons
required to sort the seven items would be 6 + 5 + 4 + 3 + 2 + 1 = 21, which
is the sum of the integers from 7 − 1 to 1. Alternately, if the character of
the items to be sorted were such that only one pass were required to
complete the sort, then the data presented in the figure indicates that only
6, or 7 − 1, comparisons would be require to complete the sort.
Generalizing these results for n items, we find the minimum and maximum
502

sort effort for the Bubble Sort to be:
Thus the range of the sort effort for this algorithm is:
Sort Effort of the Bubble Sort
The minimum sort effort is realized when the items to be sorted are
already in ascending order. When this is the case, no flips are performed
on the first pass through the algorithm, and the algorithm ends after pass 1.
The maximum sort effort is required when the items to be sorted are
initially sorted in descending order. When this is the case, n − 1 passes are
required to complete the algorithm. Thus, from a speed viewpoint, the
algorithm performs best when sorting nodes that are close to sorted, or
already sorted, in ascending order.
We will make one last point on the speed of this algorithm before
turning our attention to its overhead. The algorithm includes swap
operations, which require memory accesses. A reasonable assumption
would be that, on the average, half of the comparisons result in swaps.
Therefore, the number of swaps performed by the algorithm when sorting
n items is half the sort effort.
Memory Overhead
When this algorithm is presented with an array of primitive values or
an array of object references, the primitive values or the references are
swapped within the array. Thus, the only extra storage required to perform
the algorithm is one extra memory cell (temp), used in the classic
swapping algorithm,
Table 8.2 summarizes the performance of the Bubble Sort algorithm
(its speed and memory overhead) and includes the performance of the
Binary Tree Sort for comparative purposes. As noted in the comments
column, this algorithm is fast when the items to be sorted are already
sorted (or close to sorted) in ascending order. In this case, the speed of the
503

algorithm is faster than the theoretical minimum, n log2 n . However, when
the items the algorithm processes are already sorted (or nearly sorted) in
descending order, its performance is extremely slow, O(n 2 ). As indicated
in Table 8.2 , the memory overhead of this algorithm is 4 bytes, which is
the lowest overhead of the sorting algorithms discussed in this chapter.
The sort efforts of the two algorithms we have studied so far are
dependent on the character of the data set to be sorted ranging from very
fast, O(n ), to slow, O(n 2 ). Still, as we have seen, under the correct
circumstances, both algorithms do have their niche. Our familiarity with
binary search trees, and the simplicity of the Bubble Sort, made these
sorting algorithms a good starting point for our studies. The Heap Sort,
which we will study next, is an algorithm whose sort effort is close to the
theoretical minimum for all data sets.
8.3.3 The Heap Sort
A heap is a binary tree in which the nodes exhibit a special
relationship to each other. Although only small subsets of all binary trees
are heaps, there are some very useful algorithms in computer science that
can only be used if the tree they process is a heap. One of them is the Heap
Sort algorithm. To understand this algorithm, we must first become
familiar with heaps.
A Heap
A heap is binary tree in which the value of each parent in the tree is
greater than both of its children ‘s values.
In the context of heaps that store primitives, the term value refers to
the value of a primitive, and in the context of heaps that store objects, the
504

term value refers to the value of the object’s key field. Figure 8.9 presents
several examples of binary trees that are heaps. The definition of a heap
excludes them from being binary search trees because in a heap, a node’s
right child cannot be greater than its parent, which is a necessary condition
for a binary search tree.
Figure 8.9 Examples of Binary Trees That Are Heaps
Examining the trees present in Figure 8.9 , we observe that the value
of the root node in each tree is the largest value in the tree. This condition
is not only true for these heaps, but true for all heaps since, in a heap, the
root node must be greater than both of its children, who are greater than
both of their children, etc. Thus, the root node being the largest item in the
tree is a direct consequence of the definition of a heap and, as we shall see,
is an implicit assumption of the Heap Sort algorithm.
Examining the heaps presented in Figure 8.9 , another observation
can be made. Each subtree in these trees is also a heap. Once again, this
condition is not only true for these heaps, but true for all heaps, since, if
we start at the subtrees at the highest levels of a heap, the children must be
less than their parent, who is less than its parent, etc. Our two observations
for any heaps are summarized as follows:
For any heap,
505

1. The largest node in the heap is the root node.
2. All subtrees in a heap are themselves heaps.
Having gained an understanding of heaps, we will now begin our
study of the Heap Sort algorithm.
The Heap Sort algorithm consists of three steps that sort an array of
items into ascending order.
The Heap Sort Algorithm
1. Place all the items to be sorted in a left balanced binary tree.
2. Build the initial heap (i.e., reposition the items in the tree to make it a
heap).
3. Repeatedly:
3.1 Swap the root node into its “proper” position, and
3.2 Rebuild the remaining 2 items into a heap.
If a team of three programmers were each to implement one step of
this algorithm, the lucky programmer would be the one assigned Step 1,
place all the items to be sorted in a left balanced binary tree . The reason
is that, as we discovered in Chapter 7 , all arrays can be viewed as left
balanced binary trees using the 2i + 1, 2i + 2 rule. Therefore, the
programmer assigned this step does nothing, which is something most of
us are very good at.
Figure 8.10 Viewing an Array as a Left-Balanced Binary Tree using
the “2n + 1, 2n + 2” rule
Figure 8.10 presents an array of integers and the left balanced binary
tree that the array represents when viewed using the 2i + 1, 2i + 2 rule.
506

As mentioned in Chapter 7 , the easiest way to view the binary tree
that represents an array is to index through the array in sequential order
starting at element 0, which becomes the tree’s root, and then fill in the
higher levels of the tree from left to right. Therefore, the array is already a
left balanced binary tree, so long as, in Steps 2 and 3 of the algorithm, we
use element 0 as the root node and locate the left and right children of
element n as elements 2i + 1 and 2i + 2, respectively. Nice job, Step 1
programmer; take a break.
It is much easier to understand the remaining two steps of the
algorithm if we depict these steps in the binary tree graphical
representation of the array. However, it is important to realize that the
Heap Sort algorithm simply swaps the items to be sorted between the
elements of the array during the sorting process, just as the Bubble Sort
algorithm did. Therefore, we will refer to two graphics during the
discussion of the algorithm: one graphic will show the changes to the tree
representation of the array as the algorithm executes (to facilitate an
understanding of the algorithm), and the second graphic will show the
changes to the array contents actually made by the algorithm.
In Step 2 of the algorithm, we reposition the nodes in the tree to make
it a heap . In Heap Sort jargon, this is referred to as building the initial
heap . To do this, we examine the highest level of the tree that has parent
nodes (level 2 in Figure 8.10 ). If there are several parent nodes at this
level, we locate the level’s rightmost parent. This node is referred to as the
highest-level-rightmost-parent. Referring to the tree in Figure 8.10 , the
highest-level-rightmost-parent would be the node whose value is 2. If all
the nodes on this level were parents, then the highest-level-rightmost-
parent would be the node whose value is 1.
There is an arithmetic expression that can be used to determine the
index of the highest-level-rightmost-parent in the array, which is a
consequence of viewing the tree using the 2i + 1, 2i + 2 rule. The
expression is:
Index of the Highest-Level-Rightmost-Parent in a Left Balanced
Binary Tree
In a left balanced binary tree with n nodes, the index of the highest-level-
rightmost-parent is floor[(n / 2) − 1].
To demonstrate the validity of the technique, consider the tree
depicted in Figure 8.10 . There are nine nodes in the tree (n = 9) and,
therefore, the index of the highest-level-rightmost-parent is calculated to
507

be floor[(9 / 2) − 1] = 3. Inspecting the contents of the array and the tree
depicted in Figure 8.10 , we observe that this index (3) is, in fact, the index
of the highest-level-rightmost-parent (the item whose value is 2). If we
were to add a tenth node to the tree, it would be added as a child of the
node whose value is 3 (to keep the tree left balanced), which is located at
index 4 in the array. The numerical technique again yields the correct
index, 4 = floor([10 / 2] − 1).
Once we have located the highest-level-rightmost-parent we are ready
to build the initial heap. The changes to the tree (depicted in Figure 8.10 )
that occur as the initial heap is built are shown in Figure 8.11 , and the
corresponding changes to the array contents are shown in Figure 8.12 .
First, the highest-level-rightmost-parent is compared to both of its children
(the shaded nodes in Figure 8.11 a and Figure 8.12 a ). If the parent is
larger than both of its children, then no action is taken because it and its
children are already a heap. If it is not larger than both of its children (as is
the case in Figure 8.11 a ), it is swapped with the larger of its two children
to form a heap with the swapped child becoming the root of the heap (see
the lower left side of Figure 8.11 b and elements 3 and 7 of Figure 8.12 ,
column b ).
This process is repeated for all the other parent nodes decrementing
through parents in sequential index order. For the tree depicted in Figure
8.11 , the order would be indices 3, 2, 1, and finally, index 0. Thus, the
next two parents considered are the nodes whose values are 36 and 17 (the
shaded parents in Figure 8.11 b and c ), which are at indices 2 and 1 of the
array (see Figure 8.12 , columns b and c ). In both cases, a swap is
performed.
After swapping the values 17 and 19 (see Figure 8.12 d ), a situation
occurs that has not arisen up to this point in our study of the algorithm. As
shown in Figure 8.11 d , after the value 17 is moved downward in the tree,
it moves into a position in which it is still has children. Since it was
smaller than the children’s previous parent, 19, it could be smaller than one
of its new children (e.g., one of the new children could have the value 18).
Therefore, before proceeding to the next higher parent (25), 17 is
compared to its new children (see the shaded items in Figure 8.11 d and
Figure 8.12 , column d ) to see if it should be swapped with either of them
(i.e., moved further down the tree). In this case, it remains where it is.
508

Figure 8.11 Progression of the Tree during Step 2 of the Heap Sort,
Building the Initial Heap
Next, 25 is compared to both of its children (see the shaded nodes in
Figure 8.11 e and Figure 8.12 , column e ) and swapped with 100. Since
25, in its new position, has children, it is compared to them (see the shaded
nodes in Figure 8.11 f and Figure 8.12 , column f ) to determine if it
should be moved further down the tree. In this case, it has to be swapped
with 36, resulting in the tree shown in Figure 8.11 g and column g of
Figure 8.12 . Since 25, in its new position, has no children, its path
downward is completed.
Since, after considering 25, we have examined all the parent nodes in
the tree, Step 2 of the algorithm is complete. The resulting tree, Figure
8.11 g , is the initial heap, which is equivalent to the ordering of the
integers in Figure 8.12 , column g . By examining this tree, we can verify
that each parent in the tree is greater than both of its children, and
therefore the array contents shown in Figure 8.12 , column g , constitute
our initial heap.
509

Figure 8.12 Progression of the Array Contents during Step 2 of the
Heap Sort, Build the Initial Heap
The process used above to build the initial heap—comparing a parent
to each of its children and then moving the parent downward in the tree
until it is greater than both of its children—is called the Reheap Downward
algorithm. It gets its name from the situation that developed when the
parent node, 25, was swapped with one of its children (Figure 8.11 e ). The
swap caused the subtree it moved into to no longer be a heap, because the
root (25) was less than one of its children (36). When this occurs, the
subtree is rebuilt into a heap by moving the node further downward in the
tree (Figures 8.11 f and g ) until it becomes a leaf node, or is greater than
both of its children. It is useful to develop the Reheap Downward
algorithm separately, since, as we will see, its process is also used in Step
3 of the Heap Sort algorithm.
The Reheap Downward algorithm builds a heap out of any tree whose
root, P, is the only node in the tree preventing it from being a heap, As
such, the algorithm assumes that all the subtrees in the tree are already
heaps. Under this assumption, the pseudocode version of the Reheap
Downward algorithm is:
Reheap Downward Algorithm
510

Step 2 of the Heap Sort algorithm is performed by repeatedly
applying the Reheap Downward algorithm to each subtree in the tree,
beginning with P set to the highest-level-rightmost-parent, and ending with
P set to the tree’s root node. By working our way up from the bottom of the
tree, the subtrees of P have already been transformed into heaps.
Therefore, if the tree rooted by P is not a heap, the only node preventing it
from being a heap is the root node itself, P. Thus, the assumption of the
algorithm is satisfied.
We will now turn our attention to Step 3 of the Heap Sort algorithm:
“Repeatedly:
Swap the root node into its proper position, and rebuild the
remaining items into a heap.”
Its starting point, the initial heap built in Step 2 of the algorithm is
depicted in Figure 8.13 a and column a of Figure 8.14 . Remembering the
first of the two observations we made about heaps earlier in this chapter,
the largest node in a heap is the root node , the first part of Step 3, swap
the root node into its “proper” position , is simple. Since the root (100 in
our heap) is the largest item in the tree, and we are sorting in ascending
order, its proper position is at the end of the array. Therefore, it is swapped
with the last item in the array (7, as depicted in Figures 8.13 b and 8.14 ,
column b ).
Next, we rebuild the remaining items into a heap . The phrase
remaining items refers to all of the items that have not been placed into
their proper position in the array. Since 100 is in its proper position it is
excluded from the heap, and the remaining n − 1 items (array elements 0
through n − 2) are rebuilt into a heap. Since only the root (7) of this n − 1
node tree has been repositioned, it is the only item that could be preventing
this tree of n − 1 nodes from being a heap. Therefore, the Reheap
Downward algorithm (with item 100 excluded) can be invoked to rebuild
this tree into a heap.
The changes to the tree (and to the array it represents) as the heap is
rebuilt is shown in Figures 8.13 c -e and 8.14 , columns c-e . The arc
drawn just above the node 100 in Figures 8.13 b -e , and the heavy
horizontal line drawn just above item 100 in Figure 8.14 , columns b-e
indicate that 100 is no longer considered part of the tree that is being
rebuilt into a heap; it has been carved out of the tree. Node 7 is first
swapped with node 36, and then it is swapped with node 25. Figures 8.13 e
and 8.14 , column e show the rebuilt heap (excluding node 100). This
511

rebuilding of the heap completes the first pass through Step 3 of the Heap
Sort algorithm.
The two parts of Step 3 described above are repeated until all the
nodes are in their sorted positions in the array. Each iteration, or pass,
through Step 3 operates on a tree containing one less node than the
previous pass. Changes to the tree during the second pass through Step 3
are shown in Figures 8.13 f -i and Figures 8.14 , columns f-i . The process
begins by placing the root of the heap (36) in its proper spot in the array
(Figures 8.13 f and 8.14 , column f ), carving it out of the tree, and then
rebuilding the heap (Figures 8.13 g ,h , and i ). The changes to the contents
of the array during the next three passes through Step 3 of the algorithm
are shown in Figure 8.14 , columns j through t . The changes to the array
during the remaining passes through Step 3 are left as an exercise for the
student.
Figure 8.13 Progression of the Heap During Step 3 of the Heap Sort
Speed
Now let us consider the speed complexity of the algorithm by
512

examining its sort effort (the number of comparisons required to sort n
items). The total number of comparisons performed by this algorithm is
the sum of comparisons performed during the three steps of the algorithm.
As previously discussed, nothing is done during Step 1 of the algorithm,
since we simply use the 2i + 1, 2i + 2 rule to view the array as a binary
tree. Thus,
During Step 2 of the Heap Sort, no more than n / 2 parent nodes are
processed, since n / 2 is the maximum number of parent nodes in a
balanced binary tree containing n nodes. To decide if a parent node should
be moved down a level in the tree, two comparisons are necessary to
determine if it is less than either of its children. If it is, it descends a level
in the tree. The number of levels a parent can descend through a balanced
tree containing n nodes is always less than, or equal to, the maximum level
number in the tree, floor[log2 (n )]. (Actually, only the root node can
descend this many levels.) Since a parent is compared to two children, it is
reasonable to assume that one-third of the time the parent will be larger
than both of its children. Therefore, two-thirds of the n / 2 parents descend
a maximum of floor[log2 (n )] levels, making two comparisons at each
level and:
Figure 8.14 Progression of the Array Contents during Step 3 of the
Heap Sort
The analysis of Step 3 of the algorithm is similar to that used in Step
2. During Step 3.2 (rebuilding the heap) two comparisons are made to
determine if the root node should be moved down a level in the tree. As we
have previously stated, the maximum number of levels a node can move
513

down in a balanced binary tree containing n nodes is floor[log2 (n )]. Since
Step 3.2 is repeated for each node of the n nodes in the tree, a total of two
comparisons are made at most floor[log2 (n )] times
3 for each of the n
nodes in the tree. Thus,
Adding together the number of comparisons for Steps 1, 2, and 3 of
the Heap Sort, we obtain:
or
This sort effort is O(n log2 n ), which means it approaches the
theoretical minimum sort effort and, since the character of the items to be
sorted was not considered in its derivation, the Heap Sort algorithm
achieves these speeds for all data sets.
This algorithm includes swap operations, which require memory
accesses. Since, in our sort effort analysis for this structure we assumed
that two-thirds of the comparisons (a parent compared to both of its
children) resulted in a swap, the number of swaps is two-thirds the number
of comparisons = 2 / 3 (Sort effort) = 2n log2 n .
Memory Overhead
Like the Bubble Sort, when this algorithm is presented with an array
of primitive values or an array of references to objects, the primitive
values or the object references are swapped within the array. Thus, the
only extra storage required to perform the algorithm is four bytes for the
memory cell, temp, in the standard swapping algorithm,
Table 8.3 summarizes the performance of the Heap Sort algorithm (its
speed and memory overhead) and includes the performance of the Binary
Tree Sort and Bubble Sort for comparative purposes. As noted in the
comments column, this algorithm exhibits the best overall performance of
the sorting algorithms presented thus far in this chapter. It is fast for all
data sets, approaching the theoretical minimum speed, and its memory
overhead is as low as the overhead of the Bubble Sort.
514

8.3.4 The Merge Sort
The Merge Sort is based on the idea that two sorted sublists, A and B,
can be merged into one sorted list, T, by comparing the minimum items in
each sublist. The smaller of the two items is transferred to list T. This
process is continued until either sublist A or sublist B is empty. Then, the
remaining items from the nonempty sublist are transferred to T.
Figure 8.15 illustrates the process. The numbers inside the circles
indicate the order in which the integers are transferred from the sublists A
and B to the list T. First the 3 in sublist A is compared to the 30 in sublist
B, and the smaller of the two (3) is copied into list T. Then 21 from sublist
A is compared to 30 from sublist B, and the smaller of the two (21) is
copied into list T. This process continues until 93 from sublist A is
compared to 99 from sublist B and the 93 is copied into list T. At this point
all the values in sublist A have been copied into list T, and the remaining
values in sublist B (99 and 107) are copied into list T.
The next logical question is: given a list of eight unsorted items how
did we sort them into two four item lists (A and B) so that the merge
process could be applied to them? The answer is: using the merge process
on two sublists of two items each to produce list A, and on two other
sublists of two items each to produce list B. Finally, you may ask, given a
list of eight unsorted items, how do we produce four sublists of two items
each? Once again the answer is using the merge process, this time on eight
sublists of one item each. If it sounds recursive, your ear is improving,
because it is.
515

Figure 8.15 The Process of Merging Two Sorted Sublists into a
Sorted List
Figure 8.16 Repeated use of the Merge Process to Sort a List of Eight
Unsorted Integers
Figure 8.16 illustrates the process of sorting the original eight-item
list (93, 3, 50, 21, 85, 107, 99, and 30 depicted at the bottom of the figure)
using the merge process seven times. In all, 24 items are moved in the
order given by the circled numbers, with each of the seven pairs of circled
numbers being an application of the merge process. Initially, eight sublists
contained one item each, as shown at the bottom of the figure. These eight
lists were merged into four lists of two items each. The four lists of two
items were merged into two lists of four items, and the two lists of four
items are merged into one list of eight items.
The sublist length always begins at one, because this guarantees that
the sublist is sorted, which is a condition of the merge process.
516

Assuming the original set to be sorted is stored in the array items and
that they are copied into the array temp, the array items would be the set of
eight numbers at the bottom level of Figure 8.16 , and the array temp
would be the set of numbers just above it. This means we need two arrays
just to produce sublists of length two. To minimize the number of arrays
needed to finish the sort, the array temp is copied back into the array items
before the next merging of the sublists. Thus, the items are always merged
from the array items into the array temp. Just before the sort ends, temp is
copied into items one last time so that the sorted list is contained in the
original array.
As implied in its description, the Merge Sort algorithm is usually
expressed recursively. Using our recursive methodology, we will identify
the original problem, the base case, the reduced problem, and the general
solution. The original problem is to sort a set of n items stored in the array
items, between a left index (leftIndex, initially 0) and a right index
(rightIndex, initially n − 1), using a temporary array (temp). Stated more
succinctly, the original problem is,
Since the problem involves an integer n , the base case is probably
when n = 0 or 1. In our case, when n = 1, one item is to be sorted and
nothing needs to be done. Therefore, we will use n = 1 as the base case, in
which case the algorithm does nothing and ends.
The reduced problem should be a problem like the original one
(sorting n items), but closer to the base case (sorting one item). But how
close should it be to the base case? Should it be sorting n − 1 items, or n −
2 items, or…? A clue to the answer lies in the top part of Figure 8.16
which actually depicts the general solution. The merge process is applied
to two sublists of n /2 nodes to produce the solution to the original problem
(sorting a list of n items). Therefore, our reduced problem will be to sort a
list of n / 2 items.
The general solution will use the reduced problem twice. Once to sort
the n / 2 items in the left half of the original data set, and again to sort the
items in the right half of the original data set. Once this is done, the merge
process can be used on the two lists of n / 2 sorted items to produce the
final sorted list. Assuming middleIndex = (rightIndex + leftIndex) / 2 the
general solution is:
Assuming the arrays are named items and temp, and the indices of the
517

first and last item to be sorted are leftIndex (initially set to 0) and
rightIndex (initially set to n − 1), the recursive pseudocode of the
algorithm is:
The algorithm of the merge process (Line 7 of the algorithm) is
depicted in Figure 8.15 .
Implementation
The Java implementation of the Merge Sort pseudocode is given in
Figure 8.17 , as is the code of the method merge that implements the
merge process, depicted in Figure 8.15 .
Lines 1–11 are the code of the recursive Merge Sort algorithm, and
Lines 12–48 are the code of the method merge that it invokes on Line 9.
Lines 18–43 of the merge method moves the items to be sorted from the
array items to the array temp. Lines 18–29 repeatedly move the minimum
item from one of the sublists of the array items into the array temp (Lines
20 and 25) until one sublist is empty. The determination of which sublist
contains the minimum item is performed on Line 19. When one of the
sublists is empty, Lines 30–43 transfer the remaining items from the
nonempty sublist into the array temp. The actual data transfer takes place
on Lines 32 and 39. The determination of which sublist is not empty is
performed on Line 30. Finally, Lines 44–47 copy the contents of the array
temp into the array items.
Speed
Now let us consider the speed complexity of the algorithm by
examining its sort effort. The total number of comparisons performed by
this algorithm can be expressed as the number of passes through the
algorithm times the number of comparisons made during each pass. To
518

find the number of passes, we can simply examine Figure 8.16 .
Ignoring the original data set depicted at the bottom level of Figure
8.16 , a completion of a pass through this algorithm is depicted as a level
in the figure. Thus, three passes through the algorithm are required to sort
eight items. If we assume that the original data set contained four items
(for example the integers 93, 3, 50, and 21, depicted on the left side of the
lower level of the figure) then, as shown in the figure, two passes would be
required to complete the sort. Finally, if the original data set had contained
only two items (for example the integers 93 and 3, shown on the left side
of the lower level of the figure) then, (as depicted in the figure) one pass
would be required to complete the sort. From these observations, which
are tabulated and extrapolated (last two rows) in Table 8.4 , we can deduce
that the number of passes through this algorithm can be expressed as:
519

Figure 8.17 The Merge Sort
Table 8.4
Number of Passes through the Merge Sort Algorithm Required to
Sort n Items
Number of Items Sorted, n Number of Passes, P MS
0 0
520

2 1
4 2
8 3
16 4
32 5
Number of Passes Through the Merge Sort Algorithm
To find the number of comparisons made on each pass through the
algorithm, we consider two cases that represent the minimum and
maximum number of comparisons per pass.
1. All the items in one sublist are less than all the items in the other
sublist.
2. The sublist location of the item written into the array temp alternates
between the sublists.
An example of the first case would be the sublists 3, 21, 30, 50, and
85, 93, 99, 107. In this example, after n / 2 comparisons (3 compared to
85, 21 compared to 85, etc.), all of the items in the first sublist would be
moved into the array temp, and the pass would end after the items in the
second list were simply copied into temp.
An example of the second case would be the sublists 3, 30, 85, 99 and
21, 50, 93, 107. In this example, after n − 1 comparisons (3 compared to
21, 30 compared to 21, 30 compared to 50, 85 compared to 50, 85
compared to 93, etc.), all of the items in both sublists, except 107, would
have been moved into the array temp, and the pass would end after 107
was copied into temp. Since the two cases represent the minimum and
maximum number of comparisons performed to complete a pass, to
approximate the average number of comparisons for the Merge Sort, CAms
, we will simply average these two extremes.
Approximate Average Number of Comparisons Per Pass Through the
Merge Sort
Expressing the sort effort, SEMS , as the number of passes times the
average number of comparisons per pass, using Equations 8.1 and 8.2 we
obtain:
521

During each pass through the algorithm, all n items are swapped from
the array items into the array temp and back again resulting in 2n swaps
per pass. Since log2 n passes are performed, the total number of swaps is
Memory Overhead
The memory overhead associated with this sorting algorithm is
essentially the storage associated with the n element array temp. Each
element of temp stores either a primitive value (the items being sorted) or
a reference to the objects being sorted. In either case, each element of the
array is 4 bytes wide, resulting in an overhead of 4n bytes.
Table 8.5 summarizes the performance of the Merge Sort algorithm,
its speed and memory overhead, and includes the performance of the other
sort algorithms we have studied for comparative purposes. As noted in the
comments column, this algorithm’s speed (like the Heap Sort) is fast for all
data sets. However, it does require a moderate amount of overhead (4n
bytes).
8.3.5 Quicksort
The Quicksort algorithm, if not the most popular sorting algorithm, is
certainly the one most written about. A recent search of the internet with a
popular browser revealed that there were 3.8 million more hits for
information on this algorithm than the combined hits for all the other
algorithms discussed in this chapter. Its popularity is based on the
simplicity of its recursive implementation (it can be coded in 20
executable Java statements) and its average speed. Still, there are data sets
for which the other sorts we have studied far outperform it.
During each pass through this algorithm, the data item in the middle
of the unsorted array is chosen to be a pivot value . By the end of the pass
the item is positioned into its proper sorted place in the array, partitioning
it into two parts. In addition, the other items in the array have been
positioned such that the values of the items to the left of the pivot value are
all less than it, and the values to the right of the pivot are all greater than it.
To complete the algorithm, the left and right partitions are considered
unsorted arrays, and the algorithm operates each of them (I hope you
guessed it) recursively.
522

Figure 8.18 Data Set after One Pass Through the Quicksort
Figure 8.18 shows an array of integers before and after the first pass
through the algorithm. The integer in the middle of the original data set,
30, was chosen as the pivot value. When the pass is complete, 30 is in its
final sorted order position. In addition, the integers 21, 93, and 85 have
been relocated so that every integer to the left of 30 (said to be in the left
partition ) is less than it, and every integer that is greater than 30 is to the
right of it (said to be in the right partition ). As mentioned above, the two
partitions are then operated on by the algorithm recursively.
The steps required to reposition the items during a pass is illustrated
in Figure 8.19 using the set of integers depicted in Figure 8.18 . To start a
pass through the algorithm, the pivot value is set to the value of the item in
the middle of the array, 30. Two variables i and j are set to the highest and
lowest index of the array (0 and 8). For convenience, we will refer to i and
j as pointers and say that they are set “pointing to” elements of the array.
Thus, initially the pointers are set to point to the items stored in the highest
and lowest elements of the array.
After their initialization, the pointers are moved toward each other (i
523

is incremented, j is decremented) until each one is pointing to an item in
the wrong partition. (In our case, an item is in the wrong partition if it is ≥
30 and pointed to by i, or ≤ 30 and pointed to by j.) When both pointers
have located an item in the wrong partition (93 and 21), the values are
swapped. The contents of the array and the position of the pointers just
before the first swap are shown at the bottom of Figure 8.19 .
After the swap, the pointers are moved toward each other to locate
two other items to be swapped. The contents of the array and the position
of the pointers just before the second swap are shown in the middle portion
Figure 8.19 . This process continues until the pointers cross. The top
portion of the figure shows that the movement of the pointers has caused
them to cross (i to the right of j), which ends the pass.
As we have stated, the algorithm is unusually simple when expressed
recursively. Following our recursive methodology, the original problem is
to sort the data contained in an array (items) contained between two given
indices (leftIndex and rightIndex). Stated more succinctly,
Since the problem involves a given number of items to be sorted
(rightIndex – leftIndex +1), the base case once again is when there is only
one item to be sorted. In this case nothing is done. The reduced problem is
to apply the sorting algorithm to the two partitions. The left boundary of
the left partition is still leftIndex, and the right boundary of the right
partition is still rightIndex. The two other boundaries of the partitions can
be seen in the top part of Figure 8.19 . Since the algorithm ends when the
pointers have crossed, the right boundary of the left partition is j, and the
left boundary of the right partition is i. Thus, the reduced problems are:
Figure 8.19 The Pointer Locations and Swaps during the First Pass of
the Quicksort
524

The general solution is to build the partitions and then invoke the
reduced problems. The pseudocode of the algorithm is given below, which
assumes the items to be sorted are stored between the indices leftIndex and
rightIndex in the array items.
Lines 1–3 of the algorithm are the base case which ends the algorithm
(Line 3) when the number of items is <= 1. Lines 5–7 determine the pivot value and initialize the left and right pointers, i and j. Lines 10–18 constitute a loop that continues until the pointers cross (Line 18). The pointers are repositioned on Lines 10–13, and the array elements are swapped on Line 15, assuming the pointers have not crossed (Line 14). Line 16 moves the pointers after a swap. Lines 20 and 21 sort the left and right partitions recursively. Implementation The Java implementation of the Quicksort pseudocode is given in 525 Figure 8.20 . With the exception of the method heading and the variable declarations on Line 2, it is a line-for-line translation of the pseudocode into Java. Speed Now let us consider the speed complexity of the algorithm by examining its sort effort. The total number of comparisons performed by this algorithm can be expressed as the number of passes through the algorithm times the number of comparisons performed during each pass. When sorting n items, n 2 comparisons are made during each pass though the algorithm. We can see this by examining the first pass shown in Figure 8.19 , which is typical of the other passes. The top row of the figure shows the pointers i and j in their final positions. A total of four comparisons were made to position i at 99 (the values 93, 3, 85, and 99 were all compared to the pivot value 30). Similarly, a total of seven comparisons were made to position the pointer j at 30 (the values 40, 107, 21, 50, 30, 99, and 30, were all compared to the pivot value 30). In all, 11 (= 4 + 7) comparisons were made (= n + 2 for our nine item list). The number of passes made depends on the character of the data set. If the data set is such that the pivot values' correct positions are always in the middle of the partitions, then the left and right partitions will always be the same size (± 1). In this case, one item is positioned on the first pass, two additional items (one per partition) are positioned on the second pass (1 + 2 = 3 in total), four additional items (one per partition) are positioned on the third pass (3 + 4 = 7 in total), eight additional items (one per partition) are positioned on the fourth pass (7 + 8 = 15 in total), etc. Assuming that p is the pass number, this means that a total of 2p − 1 items are positioned after pass p . Since the sort ends when a total of n items are positioned, the sort ends when n = 2p − 1. Solving this for p we find that p = log2 (n − 1). Therefore, when the character of the data set is such that the pivot value's correct position is always in the middle of the partitions, log2 (n − 1) passes are made through the algorithm. 526 Figure 8.20 The Quicksort When the character of the data set is such that the pivot values' correct positions are always at one end of the partitions, then one of the partitions will contain one item, and the other will contain all the other items. One item is positioned on the first pass. We will assume it is positioned as the leftmost element of the array, making the length of the left partition one. On the second pass through the algorithm, since the length of the left partition is one, there are no additional items in the left partition to position; therefore, only one additional item (one from the right partition) is positioned on the second pass. Since this is once again positioned at the left end of the partition, the third pass will also position only one item. This one item partition dilemma continues for all subsequent passes through the algorithm. As a result, when the character of the data set is such that the pivot values' correct positions are always at an end of the partition, then n passes are made to complete the sort. Thus the number of passes, p , required by the Quicksort to sort n items is: log2 (n − 1) ≤p ≤n . There is, however, some good news. Empirical studies show that the average number of passes to sort is 1.45 log2 n , which means that the data set characteristics that produce the one item partitions do not occur very often. Combining this result with the number of comparisons per pass, we 527 find the average sort effort for the Quicksort, SEQSAvg , to be: which is O(n log2 n ). On average, the sort effort of this algorithm approaches the theoretical minimum. When we consider the swaps performed by the algorithm it is reasonable to assume that, on average, only half of the n / 2 items (n / 4 items) distributed over the partitions will need to be swapped during each pass through the algorithm. Remembering that the algorithm requires an average of 1.45 log2 n passes, the total number of swaps is n /4 * 1.45 log2 n = 0.25SEQSAvg . Overhead The overhead associated with this sorting algorithm is the storage required to keep track of the ends of the partitions. Specifically, a pair of the algorithm's pointers (i and j) would be allocated on the runtime stack for each level of recursion the algorithm enters. When the data set is such that the number of passes though the algorithm is log2 (n − 1), the deepest level of recursion is log2 (n − 1). Thus, a total of 2 log2 (n − 1)pointers are allocated during the recursive decent to the base case. Adding the two pointers that are allocated before the first recursive invocation, the total number of pointers is 2 + 2log2 (n − 1). Assuming 4 bytes per pointer variable, the overhead is 8+8 log2 (n − 1) bytes, O(log2 n ). Table 8.6 summarizes the performance of the Quicksort algorithm, its speed and memory overhead, and includes the performance of the other sorting algorithms studied in this chapter for comparative purposes. As noted in the comments column, this algorithm is the fastest (for most data sets), its overhead is low, and it is easy to code. These attributes account for its popularity. We must remember, however, that for some rare data sets, the speed of this algorithm is O(n 2 ) and its overhead is 8n. Subtle but important changes can be made to each of the O(log2 n ) algorithms to improve their speed. Studies indicate that when these changes are incorporated into the algorithms, the Merge Sort and the Quicksort algorithms emerge as the fastest sorting algorithms with the best performer being dependent on the number of items being sorted and the character of the data set (random, almost sorted, or sorted). 528 EXERCISES Knowledge Exercises 1. Define the term “sorting.” 2. Give two reasons for sorting a data set. 3. Give a condition under which it would not be advantageous to store a set of nodes in sorted order. 4. Give three factors to be considered in selecting a sorting algorithm for a particular application. 5. Define the term “sort effort.” 6. Give an expression for the minimum number of comparisons required to sort n items. 7. My friend has told me that he has discovered an algorithm that performs 5,000,000 comparisons to sort 1,000,000 items, regardless of the character of the data set. Should I believe him, and why? 8. Calculate the time (in minutes and seconds) required to sort 10,000,000 items on a system that performs a comparison in two 529 nanoseconds, assuming the sorting algorithm's sort effort is (don't consider swaps): a. n b. n log2 (n ) c. n 2 9. To reposition two nodes into sorted order, is it more desirable to perform a shallow or deep copy of the nodes? Why? 10. Under what conditions is the Binary Tree Sort fast? 11. Calculate the minimum and maximum number of comparisons required to sort 1,000,000 items using the Binary Tree Sort. 12. Give the times, (in minutes and seconds) to perform the sort described in the previous exercise on a machine that performs one comparison in one-half nanosecond (don't consider swaps). 13. The integers 65, 80, 70, 18, 86, 6, and 37 are to be sorted using the Binary Tree Sort. Assume the integers are processed by the algorithm in the order given. a. Show the binary tree that results from sorting. b. As the integers are placed in the tree, count the number of comparisons made. What is the total number of comparisons? 14. Repeat parts (a) and (b) of the previous exercise, but this time process the integers in the order 86, 80, 70, 65, 37, 18, and finally 6. 15. Why is the variable flip included in the Bubble Sort algorithm? 16. The integers 65, 80, 70, 18, 86, 6, and 37 are stored sequentially in an array with 65 stored in element 0 and 37 stored in element 6. The Bubble Sort is used to sort them. Trace the execution of the sort by constructing a table similar to the one presented in Figure 8.7 . Shade the elements being compared. 17. Count the number of comparisons made on each pass through the sort performed in the previous exercise and present the result as a tabulation of pass number vs. number of comparisons. 18. If a Bubble Sort does not end early, how many comparisons are required to sort n items? 19. What is the minimum number of comparisons necessary to sort n items using the Bubble Sort? 20. Give an example of a 10-item data set that would be sorted quickly 530 by the Bubble Sort. 21. True or false? The memory overhead associated with the Binary Tree Sort is less than that of the memory overhead associated with the Bubble Sort. 22. The integers 65, 80, 70, 18, 86, 6, and 37 are stored sequentially in an array with 65 stored in element 0 and 37 stored in element 6. Draw the binary tree represented by the array. 23. Define the term “heap.” 24. Is the tree discussed in Exercise 20 a heap? If not, identify the contents of the array that is preventing it from being a heap. 25. How does the Reheap Downward algorithm get its name? 26. The integers 65, 80, 70, 18, 86, 6, and 37 are stored sequentially in an array with 65 stored in element 0, and 37 stored in element 6. The Reheap Downward algorithm is used to arrange the integers into a heap. Trace the execution of the algorithm by drawing a sequence of trees similar to the ones depicted in Figure 8.11 . In your figures, shade the elements being compared. 27. Show the changes to the array discussed in the previous exercise in a table similar to the one shown in Figure 8.12 . 28. Draw the tree representation of the array shown in column t of Figure 8.14 . 29. Show the changes made to the tree drawn in the previous exercise in order to sort the remaining four integers (7, 3, 2, and 1) using the Heap Sort. 30. Show the changes to the array of integers shown in column s of Figure 8.14 that reflect the changes to the trees drawn in the previous exercise. Produce a table similar to the one shown in Figure 8.14 . 31. A 16 element array stores the integers 81, 16, 2, 89, 54, 23, 76, 25, 37, 107, 1, 74, 45, 16, 31, and 58 in elements 0 through 15, respectively. They are to be sorted using the merge sort. a. How many passes will be performed to complete the sort? b. Show the contents of the array at the beginning of the sort and after each pass. c. Count the number of comparisons that were made to sort the 16 items. d. Count the number of swaps made to complete the sort of the 16 531 items. e. Compare your answers in parts (c) and (d) of this question to results obtained when the formulas presented in Table 8.6 are used. 32. What characteristic of a data set makes the Quicksort slow? 33. The array described in Exercise 31 is to be sorted using the Quicksort. a. How many passes will be performed to complete the sort? b. Show the contents of the array at the beginning of the sort and after the first two passes. c. How many partitions will there be after pass three? d. Give the total number of comparisons and swaps performed by the algorithm using the formulas presented in Table 8.6 . Programming Exercises 34. Code the Bubble Sort in a static method whose parameter is an array of integers to be sorted. Provide a driver program to demonstrate that the method functions properly. 35. Modify the method described in the above exercise so that it counts and outputs the number of comparisons and swaps performed during each pass through the algorithm. 36. Code the Reheap Downward algorithm in a static method and code a driver program to demonstrate that it functions properly. The array to be built into a heap should be passed to the method. Use the tree shown in Figure 8.13 b and e (less item 100) as test data for your driver program. 37. Modify the Reheap Downward method coded in the previous exercise so that the index of the root node of the tree and the size of the array are passed to the method as parameters: reheapDown(int [] array, int root, int size). Provide a driver program to demonstrate that the method functions properly. Use the arrays shown in Figure 8.12 c and Figures 8.14 b and f as test data for your method. In each case, output the contents of the array before and after the method is invoked. The invocations for the three test cases should be: 38. Code the Heap Sort algorithm and provide a driver program to 532 demonstrate that it functions properly. (Hint: complete the previous exercise first.) 39. Code the Merge Sort in such a way that it outputs the number of comparisons and the number of swaps performed when sorting a random set of items. Then use it to sort 1000, 5000, 10,000, and 100,000 integers. Tabulate the results and compare it to the number of comparisons and swaps calculated using the formulas given in Table 8.6 . 40. Repeat Exercise 39 for the Quicksort. 41. Write a GUI application that demonstrates the changes to a six- element array of integers as it is sorted by the Bubble Sort (see Figure 8.7 ). The user should be able to enter the initial data set and be able to interact with the program using GUI buttons to perform the follow functions: a. Initiate the sort from the beginning of any pass to completion. b. Step through the sort, one comparison at a time, from the beginning of any pass to the sort's completion. c. Reset the sort to its initial condition. 42. Write a GUI application that demonstrates the changes to an eight- element array of integers as it is sorted by the Heap Sort (see Figures 8.12 and 8.14 ). The changes to the heap tree should also be depicted (see Figures 8.11 and 8.13 ). The user should be able to enter the initial data set and be able to interact with the program using GUI buttons to perform the follow functions: a. Initiate the sort from the beginning of any pass to completion. b. Step through the sort, one comparison at a time, from the beginning of any pass to the sort's completion. c. Reset the sort to its initial condition. 43. Write a GUI application that demonstrates the changes to a sixteen-element array of integers (and the temporary array) as the integers are sorted by the Merge Sort (see Figure 8.16 ). The user should be able to enter the initial data set and be able to interact with the program using GUI buttons to perform the follow functions: a. Initiate the sort from the beginning of any pass to completion. b. Step through the sort, one comparison at a time, from the beginning of any pass to the sort's completion. 533 c. Reset the sort to its initial condition. 44. Write a GUI application that demonstrates the changes to an eight- element array of integers as it is sorted by the Quicksort (see Figure 8.19 ). The user should be able to enter the initial data set and be able to interact with the program using GUI buttons to perform the follow functions: a. Initiate the sort from the beginning of any pass to completion. b. Step through the sort, one comparison at a time, from the beginning of any pass to the sort's completion. c. Reset the sort to its initial condition. 45. Implement a Priority Queue structure using a heap. Provide an application that demonstrates your structure functions properly. 1 If the highest level of the tree is not full, the number of levels should be rounded to the next highest integer: ceil ((log 2 ( n + 1)) 2 The term “remaining items” means the items not yet positioned into their sorted order location (or “proper” position) in the array by Step 3.1 of the algorithm. 3 As nodes are put into their proper place in the tree, the number of levels the root node moves down decreases. 534 CHAPTER 9 Graphs OBJECTIVES The objective of this chapter is to familiarize the student with the features, implementation, and uses of graph structures. More specifically, the student will be able to Understand that trees and linked lists are subsets of graphs, and understand the characteristics of trees and linked lists that restrict them to subsets. Understand the standard graphics used to depict graphs, the terminology of graphs, and the mathematics of graphs. Understand the array-based, linked, and hybrid memory models programmers use to represent graph structures, and the advantages and disadvantages of these representations. Be able to determine the best representation for a particular graph. Understand the differences between digraphs and undirected graphs, weighted and unweighted graphs, and their uses in problem modeling. Understand the modes used to access nodes stored in graphs and the basic operations performed on graphs, including depth-first and breadth- first traversals. Implement a fully encapsulated version of a graph structure that includes a traversal operation, and use the implementation in an application program. Understand, and be able to explain, the concepts related to graph connectivity and path (including spanning trees and minimum spanning trees), and be familiar with a set of problems that involve these concepts. Understand and be able to implement the classic graph algorithms 535 related to connectivity and path (Warshall's algorithm, spanning and minimum spanning tree algorithms, Dijkstra's algorithm, and Floyd's algorithm), and be able to identify applications of these algorithms. 9.1 Introduction To put the material of this chapter into the context of what we have discussed so far in this text, graphs are very similar to trees and linked lists. In fact, trees and linked lists are subsets of the broader topic, graphs. Like trees and linked lists, graphs are composed of a collection of nodes, which are called vertices in graph theory. Vertices, like the nodes of trees and linked lists, store information. In trees and linked lists there is an ordering to the nodes that implies certain nodes are adjacent to other nodes. For example, in a tree, the adjacency relationship is expressed as a parent-child relationship, while in linked lists, the adjacency relationship is a predecessor-next relationship. The vertices of a graph also have an ordering to them, and the relationship is called adjacency . If two nodes that are positioned in a tree as parent and child were to appear in a graph, they would be said to be adjacent vertices . In the case of trees and linked lists, adjacent nodes are depicted with lines or arrows drawn between them. Similarly, depictions of graphs use lines and arrows between adjacent vertices. In graphs, these lines and arrows are called edges . Where graphs differ from trees and linked lists has to do with the graph's edges. As we will see, in a graph there are no restrictions on which vertices can be adjacent. An edge can connect any two distinct vertices in the graph. 1 That was not the case for trees and linked lists. For example, two siblings in a tree could not be adjacent (have an edge between them), and at least one of the nodes in a tree had to have no children (be a leaf node). In addition, the edges in a graph can also contain a piece of information, called the edge weighting, that is not relevant to a tree or a linked list. To illustrate the similarities and differences between graphs, trees, and linked lists consider the six graphs depicted in Figure 9.1 . Figure 9.1 a depicts a graph that is also a tree and a linked list. Viewing it as a tree, its root node would be node A, and B would be A's left child. Viewing it as a linked list, we could consider A to be the first node in the list, and B the last node in the list. Figure 9.1 b adds another edge and another vertex, C, to the graph. This graph is no longer a linked list since there are now two nodes after node A. However, it is still a tree with vertex A being the 536 parent of vertices B and C. Figure 9.1 c depicts the graph with another edge added to it between vertices B and C. This graph is no longer a tree because B does not have a unique predecessor. For example, we can reach B from A or C. However, in a graph, an edge can connect any two nodes, so connecting B and C is valid. Figure 9.1 Six Graphs, Two Trees, and One Linked List Figures 9.1 d , e , and f depict three more valid graphs in which the edges possess features that cannot be present in trees or linked lists. In Figure 9.1 d , the arrows representing the edges have been replaced with lines. Although this looks like a tree, it is not, because a line connecting two vertices in a graph eliminates the parent-child relationship essential to all trees. 2 Node A can no longer be considered a parent node. Figure 9.1 e illustrates the concept of an edge weighting , which is not a concept used in trees or linked lists. In this graph, the edge between C and B carries more weight then the other two edges in the graph (i.e., 9 > 5 and 6).
Finally, in Figure 9.1 f the edge between vertices A and B, and the edge
between vertices B and C have been eliminated. In a graph, unlike a tree or
a linked list, a vertex need not have an adjacent vertex.
Of the problems in computer science whose solutions use graphs,
many do not need the additional edge properties that are not available in
trees and linked lists. We have seen examples of these problems in the
preceding chapters that discussed these structures. However, there is a set
of very interesting and important problems whose solutions are greatly
simplified when they are modeled using these additional properties. For
example, a GPS navigation system uses graphs in the algorithm that
determines the shortest route to a destination. The road intersections are
represented as the graph’s vertices, and its edges represent the roads
between the intersections. The distances between the intersections are the
weightings. A similar problem is the determination of the cheapest way to
fly from Indianapolis to Miami. In this problem, the hub airports would be
represented by the graph’s vertices, the edges would represent the flights
between them, and the edge weightings would be the cost of the flights.
In the 18th century, the mathematician Leonard Euler examined one
537

of the earliest problems involving graph theory. In this problem, known as
the Bridges of Koningsberg, the residents of the town wanted to know if
there was a way to stroll across all seven bridges that connected an island
to the rest of the town, and return to the starting point, without crossing the
same bridge twice (see Figure 9.2 ). Euler modeled the problem using the
graph shown in the left portion of Figure 9.2 , using the vertices to
represent the land masses and the graph’s edges to represent the bridges.
Then, using graph theory, Euler was able to prove that because of the
arrangement of the bridges and the land masses, the stroll was impossible.
In fact, he produced a more general solution that states that a stroll like this
is possible if and only if the number of edges incident on (leading to) each
vertex is even. Since an inspection of the graphical representation of the
problem shown in the left portion of Figure 9.2 reveals that this is not the
case (e.g., the number of edges incident on vertex D is 3), the Koningsberg
stroll is impossible.
Figure 9.2 The Seven Bridges of Koningsberg and its Graphical
Representation
Before proceeding to a discussion of how graphs are represented in
memory and the algorithms used to operate on them, it is necessary to gain
an understanding of the depictions and terminology of graphs, and also
two mathematical implications of the terminology.
538

9.1.1 Graphics and Terminology of Graphs
Standard Depiction of a Graph
In the standard depiction of a graph the vertices are represented by
circles and the edges are represented by either lines or arrows drawn
between two vertices. Figures 9.1 and 9.2 present examples of the standard
graph depictions, which typically contain either all arrows or all lines to
represent the graph’s edges. Arrows and lines are never mixed in a
depiction of a graph. Arrows are used to indicate the allowed one way
direction of travel between the two vertices the edge connects. Lines
indicate that bidirectional travel is allowed along the edge.
Graph
A graph is a set of vertices and a set of edges that connect pairs of
vertices.
Undirected Graph
An undirected graph is a graph in which all edges permit bidirectional
travel. In these graphs the edges are depicted as lines (not arrows). The
graphs shown in Figures 9.1 d and Figure 9.2 are examples of undirected
graphs. The wires and solder joints on a printed circuit board form an
undirected graph (the solder joints being the graph’s vertices and the wires
being the graph’s edges) because electricity can flow between the solder
joints in either direction.
Directed Graphs (Digraphs)
A directed graph, also known as a digraph , is a graph in which all of
the graph’s edges permit travel in only one direction. To depict the
direction of travel, the edges of a digraph are drawn as arrows in their
graphical representation. With the exception of Figure 9.1 c , all of the
graphs shown in Figure 9.1 are examples of directed graphs. The one-way
grid of streets in a large city would be an example of a digraph with each
street being an edge of the graph and the intersections being its vertices. It
could be the case that travel is permitted in both directions between two
vertices in a digraph. When this is the case, the vertices are considered to
have two distinct edges represented by two opposite-facing arrows in their
graphical depiction. An example of this would be a large city in which
some of the streets were two-way streets. In much of the literature, the
539

undirected graphs are referred to simply as graphs, and directed graphs are
referred to as digraphs.
Weighted Graphs
A weighted graph is a directed or undirected graph in which each
edge has a weighting factor assigned to it. The graph depicted in Figure
9.1 e is a weighted graph. The weighting factors could be used to represent
the lengths of the interstate highways between the points at which they
intersect.
Adjacent Vertices
Two vertices are said to be adjacent if there is an edge between them.
All of the vertices in the graphs depicted in Figures 9.1 a -e have at least
one adjacent vertex. However, vertices B and C in Figures 9.1 b and 9.1 d
are not adjacent, nor are the vertices A and B, or vertices B and C in the
graph depicted in Figure 9.1 f .
Figure 9.3 An Undirected Graph
Path
A path is a sequence of edges that connects two vertices in a graph.
Referring to the undirected graph depicted in Figure 9.3 whose eleven
edges have been named a , b , c , d , e , f , g , h , i , j , and k , the edges c,f,g
are a path from vertex E to vertex H, as are the edges c,k,e,f,j .
Simple Path
A path in which all vertices encountered along the path, except
possibly the first and last vertices, are distinct . Referring to Figure 9.3 ,
the path c,f,g is a simple path, as is the path a,h,i,j,k . However, the path
c,k,e,f,j is not a simple path because the vertex C is encountered twice.
540

Cycle
A cycle is a path in which the first and last vertex is the same. The
path a,h,i,j,k in Figure 9.3 is a cycle, beginning and ending with vertex A,
as is the path a,c,f,g,j,k .
Simple Cycle
A simple cycle is a simple path in which the first and last vertex is the
same. The path a,h,i,j,k in Figure 9.3 is a simple cycle beginning and
ending with vertex A. However, the path a,c,f,g,j,k is not a simple cycle
because the path is not simple (vertex C is encountered twice).
Path Length
In an unweighted graph, path length is the number of edges that make
up a path. Referring to Figure 9.3 , the path length of the path a,c,j,i is 4. In
a weighted graph, path length is the sum of the weighting factors of the
edges that make up a path. Referring to Figure 9.1 e , the path length from
vertex A to vertex B is either 5 or 15 depending on whether we go through
C to reach B.
Connected Vertices
Two vertices are said to be connected if there is at least one path
between them. All pairs of vertices in the graphs depicted in Figure 9.1 are
connected, except for vertices B and C in Figure 9.1 b , d , and f , and
vertices A and B in Figure 9.1 f . All pairs of vertices in the graph depicted
in Figure 9.3 are connected.
A Connected Graph
A graph is a connected graph if, given any two of its vertices vi and vj
, there is a path between them. The graphs depicted in Figure 9.2 and 9.3
are connected, as are the graphs shown in Figure 9.1 , except for the graph
shown in Figure 9.1 f .
A Complete Graph
A graph is said to be complete if the path length between any two
distinct vertices is 1. The graph in Figure 9.3 is not complete because there
is no edge connecting vertex A to B, H to B, etc. The two graphs depicted
in Figure 9.4 , are complete. Stated another way, for an undirected graph to
541

be complete there must be an edge connecting every vertex to every other
vertex. For a directed graph to be complete there must be two edges
connecting every vertex to every other vertex, so that the path length from
vertex vi to vj is 1, and the path length from vj to vi is also 1. Thus, a
complete digraph with n vertices will have twice as many edges as a
complete undirected graph with n vertices. Both of these situations are
depicted in Figure 9.4 .
To derive an expression for the number of edges in a complete
undirected graph with n vertices, we simply draw all the edges and count
them. Beginning our count at any vertex (call it vertex A), we count n − 1
edges emanating from it to make its path length 1 to each of the other n − 1
vertices. Moving to any other vertex, B, and ignoring the edge between A
and B which has already been counted, we would count another n − 2
distinct edges connecting it to the other n − 2 vertices. Continuing in this
way, and only counting edges that have not been already counted, the sum
of the edges would be (n − 1) + (n − 2) + (n − 3) +….. 1 = n (n − 1) / 2.
Therefore, in a complete undirected graph containing n vertices, there are
n (n − 1) / 2 edges. Since (as we have already observed) a complete
digraph with n vertices will have twice as many edges as a complete
undirected graph with n vertices, a complete graph must contain n (n − 1)
edges (= 2 * n (n − 1) / 2). Thus we have:
Figure 9.4 Complete Graphs
The Number of Edges in a Complete Graph
These equations can be verified for n = 4 by counting the edges in the
graphs depicted in Figure 9.4 . The directed graph in the figure has six
542

edges (= 4(4 − 1) / 2), and the digraph has 12 edges (= 4(4 − 1)).
9.2 Representing Graphs
As we have discussed, a graph is composed of vertices and edges that
are incident on the vertices. Stated a bit more formally, a graph G is a set
of vertices V, and a set of edges, E. Thus, to represent (or store) the graph
in memory we must store the set of vertices and the set of edges.
9.2.1 Representing Vertices
The set of vertices are often stored as an array of reference variables
each pointing to a vertex object. The vertex object stores the information
contained in the vertex. For example, if the vertices represent cities, the
vertex objects could be String objects each containing the name of a city.
The scheme is depicted in Figure 9.5 a for a graph containing five vertices
that represent cities. Being array-based, the maximum number of vertices
would have to be specified at the time the graph structure is created. When
the structure is implemented in the key field mode, an integer variable next
is included to keep track of the next place to add a new vertex.
If it is impossible to accurately predict the maximum number of
vertices the graph will contain, or if the number of vertices in the graph
varies over a wide range during the application, the array can be expanded
dynamically as the application proceeds. As discussed in Section 2.4 , the
expansion is most efficiently accomplished in Java with the API arraycopy
method. Sometimes, depending on the application and the implementation
language, it is more efficient to represent the vertices using a singly linked
list with each node in the list referencing a vertex object. This scheme is
depicted in Figure 9.5 b for a graph containing five nodes. It should be
kept in mind, however, that the noncontiguous memory representation of
linked list-based structures always makes their Fetch and Delete operations
slower than array-based structures.
543

Figure 9.5 Array-Based and Linked Representations of Graph
Vertices (That Are Cities)
9.2.2 Representing Edges
The edges of a graph are represented using two different schemes.
The choice of which representation is used often depends on the operations
that will be performed on the graph. One scheme is called an adjacency
matrix , and the other scheme is called an adjacency list . We will examine
them separately starting with the adjacency matrix.
Adjacency Matrix
An adjacency matrix is a square matrix in which each element stores
one edge of the graph. The matrix is represented in memory as a two-
dimensional array consisting of n rows and n columns, where n is the
number of vertices in the graph. This scheme can be used when the
vertices are represented using an array, as in Figure 9.5 a , but is not used
when the vertices are represented using a linked list as in Figure 9.5 b .
The rows of the adjacency array are considered parallel to the vertex array
in that all of the edges that emanate from the vertex stored in element 0 of
the vertex array, are stored in row 0 of the adjacency array. The column
numbers are the vertex numbers that the edges are incident on . Thus, the
544

element in row 3, column 5 of the adjacency array stores the information
for the edge going from vertex 3 to vertex 5. Since a vertex in a graph
cannot have an edge to itself, the elements along the diagonal from
element [0][0] to element [n-1][n-1] are not used. An entry of 1 in element
[i][j] of the matrix indicates that the edge from vertex i to vertex j is
present in the graph. Otherwise, the element is set to 0. Figure 9.6 depicts a
five vertex graph and its corresponding adjacency matrix.
Figure 9.6 An Undirected Graph and its Adjacency Matrix
Figure 9.7 A Digraph and its Adjacency Matrix
Since, in an undirected graph, the travel along the edges is
bidirectional, if element [i][j] is 1, element [j][i] is also 1. Consequently,
the adjacency matrix for an undirected graph is always symmetric (see
Figure 9.6 ). This fact can save some time in algorithms that process
undirected graphs because only half the array elements need to be fetched
from memory. The values of the other half of the elements can be
determined from those by reversing the indices (i.e., edge [i][j] = edge [j]
[i]).
Travel along the edges of a digraph is not bidirectional and, therefore,
the adjacency matrix of a directed graph is usually not symmetric. The
matrix of a digraph is symmetric only when there are two edges between
545

pairs of vertices. Figure 9.7 depicts a digraph and its adjacency matrix.
The combination of a graph’s vertex array and its adjacency matrix
would be a complete representation of the graph. The complete
representation of the graph depicted in Figure 9.6 is shown in Figure 9.8 .
The variable edge stores a reference to the two-dimensional edge array.
Figure 9.8 A Graph Object Representing the Graph Depicted in
Figure 9.6 Using an Adjacency Matrix to Represent the Edges
Adjacency List
Aside from the four basic operations (Insert, Fetch, Delete, and
Update) a common operation performed on graphs is to determine which
vertices are adjacent to a given vertex, for example, vertex v i . To
accomplish this in an undirected graph, we simply examine the elements of
row i of the adjacency matrix. 3 . If there is a nonzero entry stored in
element [i][j] of the array, then the graph contains an edge from v i to v j ,
and the two vertices are adjacent. Although the algorithm is
straightforward, it does perform n memory accesses for a graph with n
vertices; the algorithm is O(n ). Thus, even if there were only one edge
emanating from a vertex in a graph that contained 1000 vertices, the
algorithm would perform 1000 memory accesses to locate the adjacent
vertex.
From a space complexity viewpoint an adjacency matrix can be an
inefficient way to represent a graph. Consider the case when each of the
1000 vertices in a directed graph has two adjacent edges. In this case only
546

2000 of the 1,000,000 elements of the matrix would contain a 1. The
remaining 998,000 elements would store a 0. A matrix such as this, in
which most of its elements contain a default value (in our case 0, to
represent no edge) is called a sparse matrix . From a space (and time)
complexity viewpoint, sparse matrices are better represented as a set of
linked lists. Enter the adjacency list.
An adjacency list is a set of n linked lists, one list per vertex, which
are considered parallel to the vertex array. The first linked list stores the
edges emanating from vertex 0, the second linked list stores edges
emanating from vertex 1, etc. Each node on the linked list contains at least
two pieces of information, the vertex number of the edge it is incident
upon and, of course, the location of the next node in the linked list. Figure
9.9 shows the adjacency list for the undirected graph depicted in Figure 9.6
, and Figure 9.10 shows the adjacency list for the directed graph depicted
in Figure 9.7 .
Figure 9.9 The Adjacency List for the Graph Depicted in Figure 9.6
Figure 9.10 The Adjacency List for the Digraph Depicted in Figure
9.7
To determine which vertices are adjacent to a given vertex under this
representation of the edges, we simply traverse the vertex’s linked list.
547

Two memory accesses are required at each node in the linked list, one to
fetch the adjacent vertex number and one to fetch the location of the next
node in the list. Assuming there are an average of n l vertices adjacent to
each vertex in the graph, the linked lists will contain (on the average) n l
nodes; therefore, 2n l memory accesses are required to determine the
adjacent vertices. Since the adjacency matrix scheme requires n memory
accesses to locate adjacent vertices, the speed of the two schemes are
equivalent when n = 2n l or n l = n / 2. Thus, for values of n l < n / 2 the adjacency list scheme provides better speed performance. In a complete graph that contains n vertices, n l would be equal to n − 1 (each vertex would be adjacent to the other n − 1 vertices). In this case, n l = n − 1 > n /
2 and the adjacency matrix scheme would be the better scheme. However,
since most graphs are far from complete, n l is usually significantly less
than n / 2 and the adjacency list is, therefore, usually the favored scheme.
Figure 9.11 A Dynamic Adjacency List for the Graph Depicted in
Figure 9.9
When the vertices of a graph are represented using a singly linked list
as depicted in Figure 9.5 b , the edges are always represented using an
adjacency list. Consistent with the conditions that dictate the choice of a
linked representation of the vertices, the array edges, depicted in Figures
9.9 and 9.10 , are replaced with a singly linked list. This allows the number
of adjacency lists to grow dynamically as the number of vertices grows.
Figure 9.11 shows the linked version of the adjacency list depicted in
Figure 9.9 . The linked list used to store the headers of the adjacency lists
548

is considered parallel to the linked list used to represent the vertices, in that
the n th node in both linked lists is dedicated to vertex n .
Figure 9.12 summarizes the three combinations of vertex and edge
representations most often used to represent graphs. The two schemes on
the left side of the figure use arrays that must be expanded at run-time if
the maximum number of vertices in the graph cannot be accurately
predicted. Of these two schemes, the lower scheme is preferred when the
average number of adjacent vertices in an n vertex graph is less than n / 2.
When the maximum number of vertices in the graph cannot be accurately
predicted and the implementation language does not support a fast array
copy operation, the dynamic structure on the right side of Figure 9.12 is
used to represent the graph.
Figure 9.12 Array-Based and Linked Representations of a Graph’s
Vertices and Edges
9.3 Operations Performed on Graphs
The operations performed on graphs expand the basic operation set
performed on the previous structures we have studied. The reason for
expanding the operation set is that, not only do the nodes hold information
(i.e., client listings) but, as we have discussed, the edges also hold
information. Typical information held “in” the edges include which
vertices they connect, which direction of travel is allowed, and the
549

weighting factor of the edge. In addition, the ability to traverse a graph is
so fundamental to many graph applications that it is usually considered a
basic operation. A typical set of fundamental operations, therefore,
provides the ability to operate on both vertices and edges, and to perform a
traversal operation.
Turning our attention first to vertex operations, the functionality of
the Insert, Fetch, and Update operations is the same as that of the previous
structures presented in this text. They are used to add a vertex to the
structure and store information in it (Insert), retrieve the information stored
in a vertex (Fetch), and modify the information stored in a vertex (Update).
The Delete operation, however, extends the functionality of the previous
structures, in that, when we delete a vertex from a graph, the edges
emanating from it, and incident upon it, must also be deleted. Edges must
connect two vertices.
The operations performed on the edges of a graph typically include an
Insert operation that adds an edge between two existing vertices, a Delete
operation that eliminates an edge that connects two vertices, and a Fetch
operation that returns the edge’s weighting factor. The Update operation is
used to change the value of an edge’s weighting factor.
As far as access modes are concerned, graphs can be accessed in the
key field mode or in the node number mode. Generally speaking, when the
node number mode is used, the client specifies the number of the node to
be operated on. In the case of a graph, the vertex number would be
specified, and we will refer to this mode as the vertex number mode. A
typical client invocation to insert the Listing object P into the graph g as
vertex 2 would be g.insertVertex(2, P), and the client’s statement
g.fetchVertex(2) would be used to fetch back a reference to the
information. Many applications that utilize graph structures lend
themselves to node number mode access. Alternately, if the key field mode
is used, the invocations become g.insertVertex(P) and
g.fetchVertex(targetKey), assuming targetKey is the contents of the key
field of the information to be retrieved.
To begin with, we will develop and implement the pseudocode of just
the Insert operation on the vertices and edges in the vertex number mode.
The graph will be assumed to be an unweighted digraph. Then we will
expand the set of operations to include a traversal operation. The
implementations that include the remaining operations, access in the key
field mode, and a dynamic expansion of the structure is left as an exercise
for the student.
550

In the interest of simplicity, the vertices will be represented as a one-
dimensional array and we will store the edges in an adjacency matrix (see
Figure 9.12 a ). Assuming the vertex and edge arrays are named vertex and
edge, respectively, and that the graph can initially store a maximum of five
vertices, Figure 9.13 is a depiction of a graph object after it is initialized.
Figure 9.13 A Graph Object that can Contain Five Vertices in its
Initialized State
To insert a vertex into the structure we simply set a reference to a
deep copy of the information into the vertex array. Assuming the vertex
number specified by the client is v and the listing to be inserted is
referenced by newListing, the pseudocode is simply:
The Insert Vertex Algorithm (Assumes a Figure 9.13 graph
representation)
Assuming the graph is a directed graph and its edges are unweighted,
the pseudocode to insert an edge from vertex from , to vertex to is simply:
551

If the graph was an undirected graph, the statement edge[to][from] =
1 would be added to the algorithm after Line 3.
9.4 Implementing Graphs in the Vertex
Number Mode
Figure 9.14 presents a class called SimpleGraph, which is the Java
implementation of the graph representation scheme depicted in Figure 9.12
a and the previous pseudocode insert algorithms. The code initializes the
graph to the state shown in Figure 9.13 ; however, the maximum number
of vertices the graph will contain is specified by the client and a data
member, numberOfVertices, has been added to keep track of the number
of vertices the graph contains. In addition, the class includes methods to
output the information stored in a vertex and to output the incident vertex
numbers of all of a vertex’s edges.
Figure 9.14 The Implementation of a Simple Directed Graph in the
Vertex Number Mode
552

The class SimpleGraph is a fully encapsulated data structure whose
vertices store objects in a class named Listing. When a graph object is
created, the client passes the maximum number of vertices the graph will
contain into the class’ constructor, Line 5. Lines 6, 7, and 8 then allocate
and initialize the vertex array, the adjacency matrix array, the vertex count,
and save the maximum number of vertices. The methods insertVertex and
insertEdge, that begin on Lines 10 and 16, respectively, are the Java
equivalent of the pseudocode versions of these algorithms previously
discussed, except that Line 13 of the insertVertex method increments the
vertex count.
The vertex and edge output methods (showVertex and showEdges)
are coded as Lines 22–24 and Lines 25–30, respectively. The vertex
method outputs the contents of a Listing object (a vertex) by implicitly
invoking the toString method (Line 23) of the Listing class. The edge
output method indexes its way across the columns of a given vertex’s row
in the adjacency matrix (Line 26) outputting the incident vertex number
(the adjacency matrix column number) on Line 28 whenever a 1 is
encountered in the row.
An airline hub application that demonstrates the use of the class
SimpleGraph is presented in Figure 9.15 and the output it generates in
shown in Figure 9.16 . The application uses the graph’s vertices to
represent the airline’s hub cities, and the connections between the cities are
represented by the graph’s edges. The airline’s routes are given in Figure
9.7 , with hubs in Philadelphia (vertex 0), New York (vertex 1), Boston
(vertex 2), Los Angeles (vertex 3), and Houston (vertex 4). Consistent with
the coding of the class SimpleGraph, the definition of the hub city objects
stored in the vertices is given in a class named Listing (presented as Figure
9.17 ). The Listing class has one String data member used to store the hub
city’s name.
Lines 4–21 of the application loads the hub airport names and routes
into the SimpleGraph object flyUS declared on Line 3. Lines 22–27 output
the hubs and the routes that originate from them.
9.5 Traversing Graphs
In general, traversing a data structure is the process of performing a
processing operation on each item in the structure, once and only once.
When we perform the processing operation on an item, we are said to have
visited the item. Typical processing operations are to modify the contents
553

of a particular field, output all the fields, or to count the items to determine
how many are in the structure.
Figure 9.15 An Airline Hub Application That Uses a SimpleGraph
Object to Store the Hub Cities and Routes
Of the structures we have examined in previous chapters, array-based
structures and linked list structures are the simplest to traverse. The linear
nature of these data structures allow us to traverse them using a loop
construct that sequentially indexes through the elements of the array, or
that moves through the members of a linked list until a null reference is
found. As we have seen in Chapter 7 , the algorithm for traversing a binary
tree structure is not as simplistic because a binary tree is, generally
speaking, not a linear structure. Most often, there is more than one node, or
subtree, succeeding a node. Therefore, after visiting a node in a tree, a
decision has to be made as to which subtree to enter next. In the case of the
NLR traversal, the left subtree is entered next, while in the case of the
NRL traversal the right subtree is entered next.
554

Figure 9.16 The Output generated by the Airline Hub Application
Shown in Figure 9.15
Figure 9.17 The Class Listing for the Airline Hub Application Shown
in Figure 9.15
When we traverse graphs, the decision process is more complicated
because in a graph that contains n vertices there could be as many as n − 1
edges emanating from each vertex. The two most common traversal
techniques for graphs are the depth-first traversal (DFT) and the breadth-
first traversal (BFT). In both of these traversals any one of the graph’s
vertices can be designated to be visited first. Then, in a depth-first traversal
for each vertex visited, all of its adjacent vertices (its descendents ) are
visited before any of its siblings are visited. In the context of graph
traversals, siblings are vertices that have an edge from a common vertex,
assuming the common vertex was the vertex previously visited. 4 . Thus, if
555

vertex 0 in Figure 9.18 has just been visited, then vertex 1’s siblings would
be vertices 3 and 4. Conversely, in a breadth-first traversal all of a visited
vertices’ siblings are visited before any of its decedents are visited.
Figure 9.18 An Undirected Graph and its Adjacency Matrix
9.5.1 Depth-First Traversal
To illustrate the depth-first traversal process, consider the tree
depicted in Figure 9.18 (which also shows its adjacency matrix). We will
assume that vertex 3 (V 3 ) was arbitrarily selected to be visited first. Its
descendents are its adjacent vertices V 0 , V 1 , and V 4 , easily determined
by indexing our way through row 3 of the adjacency matrix. One of these
vertices will be visited next. Although the choice is arbitrary, in the
interest of speed and to simplify the coding process, the one chosen is
usually the vertex with the highest vertex number, in our case V 4 .
5 .
To see why this vertex is selected we must recall that the decedents of
the selected vertex (V 4 ) will be visited before any its siblings (V 1 , V 0 )
are visited, which necessitates “remembering” these unvisited siblings
while we visit V 4 ‘s descendents. In the case of a DFT, for reasons we will
mention later, a stack is used to remember the unvisited siblings. As we
index our way across row 3 of the adjacency matrix, we push all the
siblings onto the stack in the order encountered: first V 0 , then V 1 , and
finally V 4 . Then, to determine the next vertex visited we simply pop the
stack, and since V 4 is at the top of the stack, it is visited next.
The reason a stack is used in this algorithm is that if V 4 has
descendents (adjacent nodes that are not siblings), in a DFT they must be
visited before V 4 ‘s siblings (V 0 and V 1 ). Adding them to a stack
556

guarantees that they will be visited before V 0 and V 1 since they are
pushed onto the stack after V 0 and V 1 , and the vertices are visited as they
are popped from the stack. For example, if V 4 had two more adjacent
vertices that were not adjacent to any other vertex in the graph, these
vertices would be visited before V 0 and V 1 .
Figure 9.19 A Depth-First Traversal of A Graph Beginning at Vertex
V3
Figure 9.19 presents the order in which the vertices of the graph
presented in Figure 9.18 are visited, using the depth-first traversal process,
assuming vertex 3 is the first vertex visited. It also shows the progression
of the stack as the traversal takes place, and the operations performed on
the stack after a vertex is visited.
First V 3 is visited and row 3 of the adjacency matrix is examined to
determine V 3 ‘s descendents, found to be V 0 , V 1 , and V 4 . These vertex
numbers are pushed onto the stack. Next, V 4 is popped off the stack and
visited, and row 4 is examined to determine V 4 ‘s descendents. However,
V 4 ‘s descendents, V 0 and V 3 , are either already on the stack (V 0 ) or
already visited (V 3 ). When this is the case, the descendent vertices are not
557

pushed onto the stack because we do not want to visit a vertex twice.
Next, V 1 is popped off the stack and visited, and row 1 is examined to
determine V 1 ‘s descendents. The only descendent of V 1 that has not been
already pushed onto the stack (or not already visited) is V 2 , so it is added
to the stack. Then V 2 is popped off the stack and visited. V 2 ‘s descendent,
V 1 , has already been visited so it is not pushed onto the stack. Finally, V 0
is popped from the stack and visited. Since all of V 0 ‘s descendents (V 1 , V
3 , and V 4 ) have been visited, nothing is pushed onto the stack. The stack
is now empty, which signals the end of the algorithm.
The pseudocode version of this process follows. For convenience, the
first node to be visited is pushed onto the stack before it is visited. It is
assumed that the graph is represented, as shown in Figure 9.12 a , using
arrays named vertex and edge. Also, each vertex has the ability to store a
record of whether or not it has been pushed onto the stack in a Boolean
data member named pushed (see Line 2 of the algorithm). The name of the
stack object used in the algorithm is stack and the number of the first
vertex visited is stored in the variable firstVertex. When reading the
algorithm it is helpful to remember that the row and column numbers of
the adjacency matrix, edge, represent vertex numbers.
The time complexity of the above algorithm is O(n 2 ) because the
inner and outer loops on Lines 6 and 3 each execute n times.
558

Implementation
Figure 9.20 presents the implementation of a class SimpleGraphDFT,
which is the code of the class SimpleGraph presented in Figure 9.14 ,
expanded (see Lines 11–32) to include a method DFT that performs the
depth-first traversal. On Line 13, the DFT method declares an object stack
that is an instance of the class Stack defined the package java.util (see Line
1). This generic class implements a traditional stack (LIFO) structure.
Lines 15–18 set the data member pushed (of all the Listing objects the
vertices represent) to false . This data member, and the methods that set
and fetch its value, are part of the client’s class Listing2, which is
presented in Figure 9.21 (see Lines 4, 15, and 18). This class is an
expanded version of the class Listing, shown in Figure 9.17 , that also
includes a method visit() to perform the client defined operation on the
vertex being visited (Line 21). In this case, the Listing2 object stored at the
vertex is simply output.
559

Figure 9.20 The Extension of the Class SimpleGraph (Presented in
Figure 9.14 to Include a Depth-First Traversal Operation
560

Figure 9.21 An Expansion of the Class Listing Presented in Figure
9.17 to Include a Data Member and Methods Necessary to the DFT
Method of the Class SimpleGraphDFT
The remainder of the code of the DFT method (Lines 19 through 32)
is the Java equivalent of the pseudocode version of the depth-first traversal
presented above. The first vertex to be visited is passed to the method DFT
as an argument (Line 11).
An application that demonstrates the use of the method and the output
it produces is presented in Figure 9.22 . The graph used in the application
is the graph presented in Figure 9.19 . It performs a traversal of the graph
starting at vertex 3 (Line 30).
561

Figure 9.22 An Application that Performs a Depth-First Output
Traversal and its Output
It should be noted that SimpleGraphDFT is coded in a generic way, in
that it can perform any client defined operation on the nodes stored in the
graph during its traversal operation as long as the node definition class,
Listing2:
• Contains a Boolean data member, and the methods setPushed, and
getPushed to set and return its value.
• Contains a method visit that carries out the operation to be performed
on each vertex during the traversal.
562

9.5.2 Breadth-First Traversal
In a breadth-first traversal, for each vertex visited all of its siblings
are visited before any of its adjacent vertices (descendents) are visited. In
the contexts of graph traversals, siblings are vertices that have an edge
from a common vertex, assuming the common vertex has just been visited.
6
To demonstrate this traversal technique, we will use the graph
depicted in Figure 9.23 and assume the traversal starts at vertex V 3 . V 3 is
visited, and since V 3 has no siblings (no other node has been visited), we
move to its descendents V 0 , V 1 , and V 4 . One of these will be visited
next and the others will be remembered on a to-be-visited list and visited
later. Let us assume that V 0 will be visited next and that V 1 and V 4 are
remembered by placing them on the to-be-visited list. After V 0 is visited,
its descendents must be remembered since they will be visited after V 0 ‘s
siblings (V 1 and V 4 ) are visited. To insure the correct order of visitation
(siblings before descendents) the descendents of the visited vertex must be
placed on the to-be-visited list after the siblings. This is easily done if the
list is a queue . Therefore, a breadth-first traversal replaces the stack used
in a depth-first traversal with a queue. Otherwise, the two algorithms are
identical.
Figure 9.23 presents the order in which the vertices of the graph
presented in Figure 9.19 are visited using the breadth-first traversal
process, assuming vertex 3 is the first vertex visited. It also shows the
progression of the queue as the traversal takes place and the operations
performed on the queue after a vertex is visited.
The implementation of a breadth-first traversal is left as an exercise
for the student.
9.6 Connectivity and Paths
Two vertices in a graph are said to be connected if there is a way of
reaching one from the other by traveling along the graph’s edges. The
sequence of edges we travel along from one vertex to another is called a
path. Vertices 2 and 4 in the directed graph presented in Figure 9.24 a are
connected by path b,e,f . In the graph presented in Figure 9.24 b , these
two vertices are connected by several paths, e.g., edges b,e,f ; edges b,a,d ;
and edges b,a,c,f .
563

Figure 9.23 A Breadth-First Traversal of a Graph Beginning at
Vertex V3
Figure 9.24 A Directed and an Undirected Version of a Graph
An undirected graph, and a directed graph whose edge directions are
ignored , is said to be connected if, given any two vertices, there is a way
of reaching one from the other by traveling along the graph’s edges. The
graph shown in Figure 9.24 a and 9.24 b are both connected graphs. If a
graph is not connected it is said to be disjoint . The graph shown in Figure
9.1 f is disjoint.
564

A directed graph is said to be strongly connected if, considering the
direction of its edges, and given any two vertices, there is a way of
reaching one from the other by traveling along the graph’s edges. The
directed graph show in Figure 9.24 a is not strongly connected because,
when we consider the direction of the edge between vertex 2 and 1, there
is no way to reach vertex 2 from vertex 1 (or from any other vertex in the
graph). This type of connected digraph is said to be weakly connected . If
we were to delete vertex 2 from the graph, then it would be strongly
connected.
In an unweighted graph, the path length is the number of edges that
make up the path connecting two vertices. The length of the path from
vertex 2 to 4 in Figure 9.24 a is 3. In a weighted graph, the path length is
the sum of the weighting factors of the edges that make up the path. The
path length between vertex A and B in the graph depicted in Figure 9.1 e is
either 5 or 15 depending on which path is taken to reach vertex B.
Many interesting problems involve the consideration of connectivity
and path. For example, suppose we are building roads between towns in an
isolated area. Once constructed, we may ask:
• “Can any town be reached from any other town?” (The Connected
Undirected Graph problem.)
• “Can we still reach every town if some of the roads are changed to one
way streets?” (The Strongly Connected Directed Graph problem.)
• “Which roads can be closed for repair such that travelers will still be
able to reach every town?” (The Spanning Tree problem.)
• “Which roads can be closed for repair such that every town can be
reached and the total road mileage is minimized?” (The Minimum
Spanning tree problem.)
• “What is the route that minimizes the mileage traveled between two
towns?” (The Shortest Path problem.)
• “What is the route that minimizes the number of roads traveled?”
• “Is there a route we can travel such that we pass through each town
once but never visit a town twice?” (The Hamiltonian Path problem.)
• “Are there routes that travel across all the roads just once, and is there
one of these routes that will return to the starting point?” (The Bridges of
Koningsberg problem.)
• And finally, “what is the shortest route to visit all towns once and
return back to the starting town?” (The Traveling Salesman problem.)
These problems are not only applicable to roads connecting towns but
also to a variety of other problems in electronics, computer science,
565

operations research, and many other fields.
Figure 9.25 A Connected and a Disjoint Undirected Graph
9.6.1 Connectivity of Undirected Graphs
Let us begin our study of connectivity and paths by considering the
first problem in our list of questions, the problem of determining if any
town can be reached from any other town by traveling along bidirectional
roads that connect the towns. If the towns are represented by the vertices
of an undirected graph, and the roads by its edges, then the problem
becomes one of determining if the directed graph formed by the vertices
and edges is connected. If it is, then any town can be reached from any
other town. We can determine if an undirected graph is connected by
simply traversing the graph (using either a DFT or a BFT) starting at any
of the graph’s vertices, and if all the graph’s vertices have been visited, the
graph is connected. Otherwise, the traversal identifies the vertices that are
connected.
Consider the two graphs shown in Figure 9.25 . A simple inspection
reveals that any of the “towns” represented by the vertices of the
connected graph on the left, can be reached from any other, but that is not
the case for the towns represented by the disjoint graph on the right side of
the figure (e.g., Towns 1 and 2 cannot be reached from Towns 0, 3, and 4,
and vice versa). The depth-first traversal of the graph on the left side of the
figure that begins (arbitrarily) at vertex 1, visits the vertices 1, 3, 4, 0, and
finally, 2. Since this list includes all the vertices, the graph is connected.
However, the depth-first traversal of the graph on the right side of the
figure that begins (again arbitrarily) at vertex 1, visits the vertices 1, then
2. Since vertices 0, 3, and 4 are not on the “visited” list, the graph is not
566

connected (disjoint).
9.6.2 Connectivity of Directed Graphs
Now let us consider the case where the roads connecting our towns
are one-way streets. Again, the towns will be represented as vertices of a
graph and the roads will be represented as the graph’s edges. Since the
direction of travel along the roads is not bidirectional, the vertices and
edges form a digraph and, therefore, any town can be reached from any
other town if and only if the graph is strongly connected. To determine if
the graph is strongly connected, it is not sufficient to perform a DFT (or a
BFT) beginning at any vertex and then checking to see if all the vertices
are visited. This is easily demonstrated by performing a depth-first
traversal on the graph shown in Figure 9.26 beginning at vertex 1. The
traversal visits all the vertices in the order, vertex 1 followed by vertex 3,
4, 0, and finally, vertex 2. Yet there is no way to travel from vertex 2 to
another vertex in the graph. Therefore, a DFT beginning at any vertex is
not sufficient to determine if a digraph is strongly connected.
Figure 9.26 A Weakly Connected Digraph and its Adjacency Matrix
There is hope, however. If a traversal is initiated at every vertex in a
directed graph and each of these traversals visit every vertex in the graph,
then the digraph is strongly connected. For the graph depicted in Figure
9.26 , the DFT initiated at vertex 2 would only visit vertex 2 demonstrating
that the graph is not strongly connected. For a graph containing n vertices,
the speed of an algorithm that performs n depth-first traversals is O(n 3 )
since the traversal algorithm is itself O(n 2 ). In addition, as we have
discussed, each of the n traversals must be examined to determine if all n
vertices were visited, which is an O(n 2 ) operation.
Warshall’s Algorithm
567

Warshall’s Algorithm presents an alternative method for determining
if a directed graph is strongly connected and, if it is not, also affords a
rapid way of determining which vertices have paths connecting them. The
algorithm begins by copying the array that represents the graph’s
adjacency matrix into another array, t. Then, it modifies t by placing a 1 in
column j of row i if there is a path from vertex i to vertex j. The path could
be a single edge (a path length of 1, which would already be present in the
adjacency matrix) or the path could consist of a sequence of edges (a path
length > 1 which would not appear in the adjacency matrix). The resulting
modified version of the matrix t is called the transitive closure or
reachability matrix. A 1 in row i, column j of the adjacency matrix
indicates that there is an edge between vertices v i and v j , however, a 1 in
the corresponding element in the transitive closure matrix indicates that
there is a path from vertex v i to vertex v j .
Figure 9.27 A Directed Graph’s Adjacency and Transitive Closure
Matrices
For example, the transitive closure matrix for the graph depicted in
Figure 9.26 is shown in Figure 9.27 , along with its adjacency matrix and
the graph itself. Warshall’s Algorithm has changed the 0 in columns 2 and
4 of row 0 of the adjacency matrix (which indicates that there is no edge
from vertex 0 to vertex 2 or 4), to a 1 in the transitive closure matrix
(which indicates that there is a path from vertex 0 to vertex 2, and a path
from vertex 0 to vertex 4). Other rows of the transitive closure matrix
reflect similar changes. The only row unchanged is row 2, since there are
no paths from it to any other vertex in the graph.
After generating the transitive closure matrix for a graph, we can
determine if the graph is strongly connected by examining its elements. If
all the elements of the matrix are 1, except for the elements along the main
(upper-left-to-lower-right) diagonal, then the graph is strongly connected.
568

In addition, if the graph is not strongly connected, we can rapidly
determine if there is a path from vertex i to vertex j by simply testing t[i][j]
to determine if it is 1. If so, a path exists between the two vertices.
The basis of Warshall’s Algorithm is the transitive property in
mathematics: if a = b and b = c , then a = c. The algorithm reasons that if
there is a path from vertex v b to vertex v c then there is a path to v c from
every vertex that can reach v b . Consistent with this reasoning, the
algorithm examines each element of the adjacency matrix working its way
across the columns beginning with row 0. When it finds an element with a
value of 1, (e.g., vertex[b][c] = 1, indicating a path exists from vertex v b
to vertex v c ), it indexes its way down column b of the matrix to find the
vertices with a path to v b ; e.g., vertex[a][b] = 1, indicating that there is a
path from v a to v b . If this is the case, there must be a path from v a to v c
(through vertex v b ) so vertex[a][c] is set to 1. Provisions are made in the
algorithm to not place a 1 along the main diagonal of the matrix.
Figure 9.28 presents the code of the method transitiveClosure which
implements Warshall’s Algorithm. It returns the transitive closure of the
array adjacency passed to it as a parameter (Line 2). The parameter n is the
number of rows (and columns) in the adjacency matrix. Lines 4–6 copy the
adjacency matrix into the transitive closure matrix, t. Lines 8–19 is the
coding of Warshall’s Algorithm. Line 8 indexes through each vertex in the
graph. For each vertex, b , Lines 9–10 find a vertex, c , it is connected to.
Then Lines 11–13 locate all the vertices, a , connected to b , and mark a
path in the transitive closure matrix from a to c .
569

Figure 9.28 A Method that Determines the Transitive Closure Matrix
of a Given Matrix
Figure 9.29 presents an application that determines the transitive
closure matrix of the graph depicted in Figure 9.27 . The program output
(shown at the bottom of the figure) is the returned transitive closure
matrix, t (which is also shown in Figure 9.27 ).
The three nested loops on Lines 8, 9, and 11 of Figure 9.28 make the
speed complexity of Warshall’s Algorithm no better than performing n
depth-first traversals to determine if a directed graph is strongly connected.
However, Warshall’s Algorithm provides an additional piece of
information: a permanent record of all the possible paths via the transitive
closure matrix.
9.6.3 Spanning Trees
A simple cycle is a simple path in a graph that begins and ends at the
same vertex. The paths a,c,e ; c,d,f ; and d,f,e,a in Figure 9.30 a are simple
cycles. A tree is a connected graph that does not contain simple cycles.
Eliminating the edges c and f from the graph depicted in Figure 9.30 a
make it a tree because it is still connected and has no simple cycles. For
simplicity, we will refer to simple cycles as cycles.
A graph’s spanning tree is a tree that contains all of the vertices of the
graph connected by a subset of the graph’s edges. The edges are chosen
such that is there is a path from each vertex to every other vertex, and
570

(since it is a tree) there are no cycles. Most graphs have more than one
spanning tree. Two of the spanning trees for the graph shown in Figure
9.30 a are shown in Figure 9.30 b .
Figure 9.29 A Program to Demonstrate the Use of the Method
transitiveClosure
Figure 9.30 An Undirected Connected Graph and Two of its
Spanning Trees
There is always one, and most often more than one, spanning tree for
571

every connected undirected graph. Since there are no cycles in a spanning
tree, or trees in general, they always contain one less edge than the number
of vertices in the graph. Thus, the spanning tree of an undirected connected
graph is a subgraph that contains the minimum number of the graph’s
edges and still allows a path from any vertex to any other.
These characteristics give spanning trees an important role in many
applications. Consider the problem of deciding which bidirectional roads
connecting towns to plow first after a major snowstorm so that all of the
towns could be reached. If the towns are represented as the vertices of a
graph and the roads its edges, then any spanning tree of the graph presents
a solution. For this problem, a more interesting solution would be the
spanning tree that offers the shortest plow route. As we will see, a
spanning tree exhibiting this characteristic is called a minimum spanning
tree . In the remainder of this section, we will examine the techniques for
determining a graph’s spanning trees and its minimum spanning trees.
To find a spanning tree of a connected undirected graph we can
simply use a depth-first traversal and record the edge between each node
visited and its descendents as the descendents are pushed onto the
algorithm’s stack. Since the vertices are only pushed onto the stack once,
only one edge to each vertex will be recorded, and since all the vertices are
pushed onto the stack, the edges will include an edge to each vertex. These
edges will be the edges of the spanning tree. Alternately, a breadth-first
traversal could be used and then we would record the edge to each vertex
added to the queue used by the BFT algorithm. Since, when building a
spanning tree, we don’t actually operate on the nodes, we eliminate the
code that visits the nodes from the DFT and BFT traversals.
The pseudocode to find a subset of a graph’s edges that are included
in one of its spanning trees follows. It is a modification of the DFT
pseudocode presented earlier in this chapter, which assumed that the graph
is represented using a vertex array named vertex and an adjacency matrix
named edge (see Figure 9.13 ). These, as well as the starting vertex
number, firstVertex, are supplied to the algorithm. It produces an
adjacency matrix, st, that represents the edges of the spanning tree. The
modifications to the DFT algorithm are to eliminate the traversal’s visit of
a node (Line 5 of the DFT algorithm) and to add two lines (the following
Lines 9 and 10), which place the edge to a descendent vertex into the
spanning tree’s adjacency matrix, st.
572

The implementation of the spanning tree algorithm is left as an
exercise for the student. As presented, it, like the implementation of the
DFT algorithm, would be coded as an operation method in a class that
defines a graph object. The most efficient way of doing this would be to
extend the class SimpleGraphDFT (shown in Figure 9.20 ). The new class
would support both depth-first traversals and the generation of spanning
trees.
Minimum Spanning Trees
Minimum spanning trees are spanning trees that consider an
additional piece of information associated with the edges of a connected
undirected graph. That piece of information is called an edge weighting .
For example, consider the graph whose vertices represent towns and
whose edges represent the roads between the towns. A typical edge
weighting could be the length of the roads. Other edge weightings could be
the toll charged to travel along the roads, or the amount of snow on the
roads. A graph whose edges carry weightings is called a weighted graph .
In the typical depiction of a weighted graph (see Figure 9.31 ), the
values of the weighting factors are shown along the graph’s edges. In this
graphical depiction, no attempt is made to make the relative length of the
edges correspond to their relative weights. For example, the edge with the
highest weight that connects vertex 3 and 0 is not the longest edge in the
573

graph. As shown in Figure 9.31 , the weights are stored in a matrix using
the same row and column assignment scheme used in the adjacency
matrix. That is, the weight of the edge from vertex v i to vertex v j is stored
in row i, column j of the weight matrix. Nonexistent edges (e.g., between
vertex 0 and 2) are represented by an impossibly low or impossibly high
value of the weighting factor depending on the application. Most
programming languages provide a predefined constant that can be used
(e.g., Java’s Integer.MIN_VALUE and or Integer.MAX_VALUE). For
convenience, the impossibly low value of the weight for the graph depicted
in Figure 9.31 is 0.
Figure 9.31 A Weighted Graph and its Weight Matrix
A minimum spanning tree is the spanning tree whose edges are
selected to minimize the sum of the weighting factors of the edges that
make up the tree. It can be shown that, if there are no two edges in a
weighted graph with the same weighting factor, then there is only one
minimum spanning tree for the graph.
To find a minimum spanning tree of a connected undirected graph,
we begin by placing vertex 0 in the tree. Then we consider all the vertices
currently in the tree (initially only vertex 0), and select the edge emanating
from them with the minimum weight. This edge, and its incident vertex, is
added to the tree and the process is repeated until all the vertices are
added. During the process, an edge to a vertex already in the tree is not
considered. Figure 9.32 illustrates the process of generating the minimum
spanning tree for the graph depicted in Figure 9.31 . The vertices that have
been added to the tree are shown as gray circles, and the edges added to
the tree are shown as colored lines.
First, vertex 0 would be added to the tree (Figure 9.32 a ). Then the
edges emanating from vertex 0 would be considered (dashed edges in
Figure 9.32 b ) and the edge with weight 5 (the minimum of weights 8, 9,
574

and 5) would be selected. It and its incident vertex, vertex 4, would be
added to the tree. Next, the edges emanating from vertices 0 and 4 would
be considered (dashed edges in Figure 9.32 c ), and the edge with weight 2
(the minimum of weights 8, 9, and 2, with 5 not considered since it is a
weighting of an edge to a vertex already in the tree) would be selected. It
and its incident vertex, vertex 3, would be added to the tree. Then, the
edges emanating from vertices 0, 4, and 3 would be considered (dashed
edges in Figure 9.32 d ) and the edge with weight 3 (the minimum of
weights 8 and 3 with 9, 2, and 5 not considered since they are weightings
of edges to vertices already in the tree) would be selected. It and its
incident vertex, vertex 1, would be added to the tree. Finally, the edge
emanating from vertices 0, 4, 3, and 1 would be considered (dashed edge
in Figure 9.32 e ) and the edge with weight 1 would be selected (the edges
with weights 2, 3, 5, 8, and 9 are not considered since they are the
weightings to vertices already in the tree).
575

Figure 9.32 Process of Building a Minimum Spanning Tree
Having added all the vertices to the tree, the algorithm ends. The
minimum spanning tree it generated is shown in the lower right portion of
Figure 9.32 . The sum of the weightings of the edges of the tree is 11. This
means that if the edge weights represented the length of the roads
connecting five towns, the shortest plow route to make all the towns
accessible after a snow storm would be the 11 mile route shown in Figure
9.32 f . In addition, the best place to locate a snow plow garage would be
576

town 0 or town 2.
The pseudocode version of the process that generates a minimum
spanning tree follows. It uses three arrays. One array, verticesIncluded,
stores the vertices of the graph added to the tree as the algorithm proceeds;
another array aCopy is assumed to be initialized to the graph’s weight
matrix; the third array mst is the product of the algorithm, the weight
matrix of the graph’s minimum spanning tree; noEdge is an impossible
high weighting value.
Lines 1–4 of the pseudocode perform an initialization. Vertex 0 is
added to the tree (Line 1), and then the variable numVerticesIncluded is
set to 1 (Line 2) to indicate that one vertex has been added to the tree.
Lines 3–4 eliminate all the edges to Vertex 0 by setting their entry in the
array aCopy to noEdge, a value selected to be higher than any of the edge
weightings in the graph. The edges to a vertex are eliminated from the
array aCopy when a vertex is added to the minimum spanning tree because
once a vertex is added to the tree, we no longer have to consider edges that
lead to the vertex; it is already part of the tree. Figure 9.33 shows the
contents of the arrays verticesIncluded and aCopy after the initialization
performed by Lines 1–4 is complete.
577

Figure 9.33 Arrays of the Minimum Spanning Tree Algorithm in
their Initialized State (* is the impossibly high edge weight)
It also shows the array mst in its initial state. In the figure, an asterisk
in an element of a matrix indicates that the element has been set to the
impossibly high value of the weighting factor, noEdge.
Line 5 begins a while loop that terminates on Line 15 after all the
vertices have been added to the tree. Figure 9.34 shows the changes to the
contents of the arrays as the loop executes. The progression of the array
aCopy is shown in the center portion of the figure with the arrays mst and
verticesIncluded on the right and left. The first row of arrays in the figure
depicts the array contents during the first pass through the while loop, with
subsequent passes shown sequentially in the rows below it. The status of
the array, verticesIncluded, before each pass through the loop begins, is
shown on the left side of each row.
During each pass through the loop, a vertex and an edge is added to
the minimum spanning tree. Line 6 uses a method findMinWeightedEdge
(assumed to exist) to locate the minimum weighted edge emanating from
those vertices currently included in the tree (initially just vertex 0), and
returns the row (rowMin) and column (colMin) of that edge’s weighting in
the array aCopy. It also returns the value of the edge’s weighting factor
(weightMin).
The rows of aCopy searched each time through the loop by the
method appear in color in the leftmost depiction of the array in Figure 9.34
, and the minimum weighted edge located by the method each time
578

through the loop is colored in the center depiction of aCopy. Lines 8–10
eliminate the minimum weighted edge, and all the other edges leading to
the vertex added to the tree from the array aCopy. It does this by setting
the vertex’s column of the array to the value noEdge (see the colored
column of the rightmost depiction of the array aCopy in Figure 9.34 ). As
stated above, all these edges are eliminated because once a vertex is added
to the tree we no longer have to consider edges that lead to the vertex; it is
already part of the tree.
Figure 9.34 Array Contents during the Minimum Spanning Tree
Algorithm’s while Loop Execution (* is an impossibly high edge weight)
579

Before the loop ends, Lines 11–12 include the minimum weighted
edge in the minimum spanning tree’s weight matrix, mst, (as shown on the
right side of Figure 9.34 ). Then, Line 13 adds the vertex’s number the
edge is incident upon, to the list of vertex numbers included in the tree (see
the left side of the next row of Figure 9.34 ). Finally, Line 14 increments
the number of vertices included in the tree.
The coding of the minimum spanning tree algorithm and the method
findMinWeightEdge is left as an exercise for the student. Both methods
could be added to the class SimpleGraph presented in Figure 9.14 .
Assuming the name of the method that implements the minimum spanning
tree algorithm is minSpanningTree its signature would be:
The returned two-dimensional array would be the weight matrix of
the minimum spanning tree. Thus, the client invocation to determine the
minimum spanning tree of a SimpleGraph object g would be:
where minTree is a two-dimensional integer array reference.
9.6.4 Shortest Paths
There are many applications in which we would like to know the
shortest path length from one vertex of a graph to another. For example,
suppose the vertices of the graph shown in Figure 9.33 represent cities and
we want to know the shortest trip between the two cities represented by
vertex 0 and vertex 1. Assuming the graph’s edges represent the roads
connecting the cities and the edge weightings represent the miles between
them, the shortest trip would be along the eight mile road connecting the
two cities. On the other hand, the shortest trip between the cities
represented by vertex 0 and vertex 3 would be the indirect trip of seven
miles that would pass through the city represented by vertex 4.
Often, when students are introduced to the problem of determining
the shortest path between two vertices, they believe the solution is to travel
along the edges of the graph’s minimum spanning tree; especially if they
have just concluded a study of minimum spanning trees (ring any bells?).
However, the minimum spanning tree algorithm produces the shortest
route connecting all cities, which may not include the shortest route
between two cities. For example, the minimum spanning tree for the graph
presented in Figure 9.33 is the tree whose edges are represented by the
bold lines in Figure 9.34 . The roads included in it are the roads from
580

vertex 0 to 4, vertex 4 to 3, vertex 3 to 1, and, finally, from vertex 1 to 2.
Therefore, the trip from vertex 0 to 1 would be a 10 mile trip passing
through vertices 4 and 3 before arriving at vertex 1. However, as we have
previously discussed, it is clear that the shortest trip from vertex 0 to 1 is
the eight mile trip along the edge connecting them. Since this edge is not
included in the minimum spanning tree, it is apparent that we will need
another algorithm to determine the shortest path between any two vertices.
Enter Edsger Dijkstra. In 1959, Edsger Dijkstra discovered an algorithm
that determines the shortest path between any two vertices in a connected
undirected graph or a connected digraph. The algorithm is aptly named the
Dijkstra Shortest Path algorithm. As we will see, the algorithm not only
determines the shortest path from any vertex A to any other vertex B, but it
also determines the shortest path from vertex A to all the other vertices in
the graph, and the path lengths along these paths. The tree comprised of all
the vertices of the graph and the edges that form the shortest path from
vertex A to all the other vertices, is called the shortest path tree from
vertex A.
Dijkstra’s Algorithm
Dijkstra’s Shortest Path algorithm is very similar to the minimum
spanning tree algorithm. They both begin by placing the starting vertex
into the tree. Then they consider all the vertices currently in the tree
(initially only the starting vertex), and select an edge emanating from them
based on a “consideration” of the edge weightings. The selected edge, and
its incident vertex are then added to the tree and the process is repeated
until all the vertices are added. During the process, an edge to a vertex
already in the tree is not considered.
Where the algorithms differ is in the consideration of the edge
weightings used to determine which edge to add to the tree. As we have
seen, to build a minimum spanning tree, the edges emanating from the
vertices currently in the tree with the minimum weighting is selected. In the
Shortest Path algorithm the edge emanating from the vertices currently in
the tree that produces the shortest path length to the vertex it is incident
upon is selected.
For example, suppose we are to build a minimum spanning tree
(MST) starting at vertex 0, and a shortest path tree (SPT) from vertex 0 for
the graph depicted in Figure 9.32 a . Let us assume that the vertices 0, 3,
and 4 and the edges 5 and 2 have been added to both trees as shown in
Figure 9.32 d . The next edges to consider for both trees are the edges not
581

already in the tree, emanating from these three vertices that are incident
upon vertices not already in the tree. Thus, as depicted in Figure 9.32 d ,
the edges with weightings 3 and 8 would be considered (because vertex 1
is not in the tree) and the edges with weightings 2, 5, and 9 would not be
considered (because vertices 0, 3, and 4 are already in the tree).
Now things get different. In the case of the minimum spanning tree,
the edge with minimum weighting , 3, is selected for inclusion in the tree.
In the case of the shortest path tree the edge with weighting 8 would be
selected, because if 3 were selected the path length from the starting vertex
(0) to vertex 1 would be larger than 8, i.e., 10 = 5 + 2 + 3. Figure 9.35
shows the graph depicted in Figure 9.32 a along with its minimum
spanning tree and the shortest path tree from vertex 0.
The complete process of building the shortest path tree from vertex 0
for the graph depicted in Figure 9.35 is illustrated in Figure 9.36 . The
vertices that have been added to the tree are depicted as gray circles, and
the edges added to the tree as colored lines. The dotted lines are edges
under consideration for inclusion into the tree.
First, vertex 0 (the starting vertex) is added to the tree (see Figure
9.36 a ). Then the edges emanating from vertex 0 would be considered (the
dashed edges in Figure 9.35 b ) and the edge with weight 5 (the minimum
path length of the paths from vertex 0 to vertices 1, 3, and 4) would be
selected. It and its incident vertex, vertex 4, would be added to the tree.
Next, the edges emanating from vertex 0 and 4 that are incident upon
vertices not yet in the tree (vertices 1 and 3) would be considered (see
Figure 9.35 c ). The edge with weight 2 would be selected because it
would complete a path from vertex 0 to vertex 3 whose length is 7, which
is shorter than the direct path length 9 to vertex 3, or 8 to vertex 1. Thus,
the edge with weight 2 and its incident vertex, vertex 3, would be added to
the tree. Next, the edges emanating from the tree’s vertices 0, 4, and 3 that
are incident upon vertices not yet in the tree (vertex 1) would be
considered (see Figure 9.32 d ). The edge with weight 8 would be selected
because it gives a path length from vertex 0 to vertex 1 of 8 which is
shorter than 10 (= 5 + 2 + 3 the path from vertex 0 to 4 to 3 to 1). It and its
incident vertex, vertex 1, would be added to the tree. Finally, the edges
emanating from vertex 0, 4, 3, and 1 that are incident upon vertices not yet
in the tree (vertex 2) would be considered (see Figure 9.32 e ). The edge
with weight 1 would be selected (the edges with weights 2, 3, 5, 8, and 9
are not considered because they are either already in the tree or incident
upon vertices already in the tree). Having added all the vertices to the tree,
582

the algorithm ends.
Figure 9.35 A Graph’s MST and SPT (from Vertex 0)
The shortest path tree from vertex 0 generated by the algorithm is
shown in Figure 9.35 f . The tree gives the minimum paths from vertex 0
to any other vertex in the tree. This means that if the edge weights
represent the length of the roads connecting five towns, the roads included
in the graph would be the shortest routes to any town from the starting
point (the town represented by vertex 0).
The pseudocode version of the algorithm, which is a modification of
the pseudocode version of the minimum spanning tree algorithm, is
presented below. The algorithm not only determines the shortest path tree,
but also the path lengths of the shortest paths between the starting vertex
and each of the tree’s other vertices. It uses four arrays. Three of these
arrays serve the same function as in the minimum spanning tree algorithm.
The array verticesIncluded stores the vertices of the graph added to the
shortest path tree as the algorithm proceeds; the array aCopy is initialized
to the graph’s weight matrix, and the array spt (named mst in the minimum
spanning tree algorithm) is the weight matrix of the shortest path tree
produced by the algorithm.
583

Figure 9.36 The Process of Building a Shortest Path Tree from
Vertex 0
The fourth array, minPathLengths, has been added to the algorithm to
store the path lengths of the shortest paths between the starting vertex and
each of the tree’s other vertices as these path lengths are generated by the
algorithm. The path lengths in this array are initialized to an impossibly
high value, noPath. When the algorithm ends, the shortest path length from
the starting vertex to vertex 0 will be in element 0 of the array, the shortest
584

path length from the starting vertex to vertex 1 will be in element 1, etc.
Lines 1 to 7 of the pseudocode generalize the initializations
performed by the minimum spanning tree algorithm so that any vertex,
whose vertex number is contained in the variable startVertex, could be the
starting vertex. Line 1 adds the starting vertex to the list of vertex numbers
included in the tree, and then Line 2 sets numVerticesIncluded to 1 to
indicate that one vertex has been added to the tree. Lines 3 to 6 is a loop
that initializes the path lengths, (Line 4), to an impossible value (noPath),
and eliminates all the edges to the starting vertex (Line 5) by setting their
entry in the array aCopy to noEdge, an impossible value of the of the edge
weightings. The edges to a vertex are eliminated from the array aCopy
when a vertex is added to the shortest path tree because once a vertex is
added to the tree we no longer have to consider edges that lead to the
vertex; it is already part of the tree. Line 7 completes the initialization
process by setting the path length to the starting vertex (from the starting
vertex) stored in the array minPathlengths to 0. The four arrays
verticesIncluded, aCopy, spt, and minPathlengths are shown in their
initialized state in Figure 9.37 . The figure depicts the values of noPath and
noEdge as an as asterisk, and vertex 0 is assumed to be the starting vertex.
585

Figure 9.37 Arrays of the Shortest Path Tree Algorithm in their
Initialized State
Line 8 of the algorithm begins a while loop that terminates on Line
19 after all the graph’s vertices have been added to the tree. During each
pass through the while loop, a vertex and an edge is added to the shortest
path tree. Figure 9.38 shows the changes to the contents of the arrays as
the loop executes. The first row of arrays in the figure depicts the array
contents during the first pass through the loop, with subsequent passes
shown sequentially in the rows below it. The status of the array
verticesIncluded before each pass through the loop begins, is shown on the
left side of each row.
Line 9 of the algorithm uses a method findMinPath (assumed to exist)
to determine the minimum of the path length from the starting vertex to the
vertices adjacent to those vertices currently included in the tree (initially
just vertex 0). It returns the location in the array aCopy (rowMin and
colMin) of the edge that completes the minimum of these paths. It also
returns the value of the edge’s weighting factor (weightMin) and the
minimum path length (minPath) from the starting vertex to the incident
vertex (colMin).
The rows of aCopy searched by the method each time through the
loop appear in color in the leftmost depiction of the array aCopy in Figure
9.38 . These are the rows that store the edge weightings from the vertices
already included in the tree. The minimum of the sum of each of these
edge weightings and the path length to each vertex (stored in the
586

corresponding row of the array minPathLengths and intialized to noPath)
determines which edge is added to the tree each pass through the loop. The
weighting of the edge added to the tree is colored in the center depiction of
the array aCopy.
Figure 9.38 Array Contents during the Shortest Path Tree
Algorithm’s while Loop Execution (* is an impossibly high edge weight or
is an impossibly high path length)
Lines 11–13 eliminate all the edges leading to the vertex included in
the tree from the array aCopy. It does this by setting the vertex’s column in
the array to the value noEdge (see the colored column of the rightmost
depiction of the array aCopy in Figure 9.38 ). As stated, all of these edges
are eliminated because once a vertex is added to the tree we no longer have
to consider edges that lead to the vertex; it is already part of the tree.
587

Lines 14 and 15 add an edge to the tree by writing its weighting into
the tree’s weight matrix, spt, (as shown on the right side of Figure 9.38 ).
Line 16 places the path length to the vertex added to the tree during this
pass through the while loop into the minPathLength array (see the colored
entry on the far right of Figure 9.37 ).
Finally, Line 17 adds the incident vertex to the tree by writing its
vertex number into the list of the tree’s vertices (see the left side of the next
row of Figure 9.38 ), and Line 18 increments the number of vertices
included in the tree.
The coding of the shortest path algorithm and the method
findMinPath is left as an exercise for the student. Both methods could be
added to the class SimpleGraph presented in Figure 9.14 . Assuming the
name of the method that implements the shortest path algorithm is
shortestPath, its signature would be:
The returned two-dimensional array would be the weight matrix of
the shortest path tree containing the edges that produce the shortest paths
from the starting vertex. The first parameter would be used by the client to
specify the starting vertex number. The second parameter returns the
shortest path lengths from that vertex to all the other vertices after the
method completes its execution. Thus, the client invocation to determine
the shortest path tree from vertex 1 of a SimpleGraph object g would be:
where shortestPathTree is a two-dimensional integer array reference
and minPathLength is a one-dimensional integer array reference.
Floyd’s Algorithm
As we have seen, Dijkstra’s algorithm determines the shortest path
from one vertex to all the other vertices in the graph. During the period
from 1959 to 1962, Bernard Roy and Robert Floyd developed a
remarkably simple algorithm that took the determination of the shortest
paths in a graph to a higher level, in that the algorithm determines the
shortest path between all the pairs of vertices in a directed weighted graph.
In addition, its speed is equivalent to the speed of Dijkstra’s algorithm.
The algorithm is most commonly known as Floyd’s Algorithm,
although it is also referred to as the Roy-Floyd Algorithm, or the All-Pairs
Shortest Path Algorithm. While it is true that Floyd’s Algorithm does not
determine the weight matrix that describes the edges involved in the
588

shortest path as Dijkstra’s Algorithm does, its ability to treat every vertex
as a starting vertex in one pass through the algorithm makes it a very
useful algorithm for many applications.
The basis of the algorithm is the idea that the shortest path length
from vertex A to C is the shortest of the following two path lengths:
1. The path length from A to C;
2. The path length from A to B plus the path length from B to C, for all
B .
In other words, the algorithm looks for an intermediate vertex, B, to
travel through on its way to C, such that the sum of the path lengths from
A to B, and B to C, is shorter than the path length from A to C.
As an example, consider the graph shown in Figure 9.38 and suppose
we want to find the shortest path from vertex 0 to 3. The edge connecting
them has a path length of 9. If we consider the intermediate vertex to be
vertex 1, the total path length would be 11 (= 8 + 3) which is longer than 9,
and so vertex 1 would not be accepted as an intermediate vertex for the trip
from 0 to 3. However, when vertex 4 is considered the intermediate vertex,
the path length is 7 (= 5 + 2) so the algorithm would, from this point
forward, consider the path length from vertex 0 to vertex 3 to be 7. The 9
in row 0, column 3 of the weight matrix would be overwritten with a 7.
Proceeding in this way, the algorithm considers each vertex to be a
candidate intermediate vertex for every possible point-to-point trip. When
a path length that includes an intermediate vertex B is found to be shorter
than the entry in the weight matrix for the path length between vertices A
and C, the shorter path length is written into the Ath row and Cth column
of the weight matrix. This process eventually transforms the weight matrix
into the all-points shortest path matrix.
Figure 9.39 shows a weighted graph, its weight matrix, and the all-
points shortest paths matrix generated by Floyd’s Algorithm. This
algorithm once again uses an impossibly large value of an edge weight to
indicate the nonexistence of an edge between two vertices (e.g., Java’s
Integer.MAX_VALUE). In the interest of simplicity, Figure 9.39 uses an *
symbol as the impossibly large value. The entries in the shortest paths
matrix, the product of the algorithm, are the shortest paths between each
pair of vertices in the graph. For example, the shortest path between vertex
3 and vertex 2 is 6 (the colored element of the matrix). The path is from
vertex 3, to 4, to 1, to 2 (although this path is not produced by Floyd’s
Algorithm).
589

Figure 9.39 A Graph and its All-Pairs Shortest Path Matrix (* is an
Impossibly High Edge Weight)
The generation of the all-pairs shortest paths matrix for the graph
depicted in Figure 9.39 is illustrated in Table 9.1 . The leftmost column of
each row of the table contains the number of the intermediate vertex, B,
that was used for the point-to-point paths identified in the other 16 cells of
the row. 7 . For example, the first intermediate-vertex considered is vertex
0, and the first point-to-point path examined is the path from vertex 1 to
vertex 1 (denoted as 1 → 1 in the table’s first row and second column). In
some cells of the table there is a number below the path entry. This
indicates that the path between the two vertices that included the
intermediate vertex was shorter than the path between the two vertices
currently stored in the weight matrix. The value in the cell is the path
length between the two vertices that includes the intermediate vertex. The
current (longer) path length stored in the weight matrix is overwritten with
this value.
Let us again consider the trip indicated in the first row and second
column of the table, from vertex 1 to vertex 1 with vertex 0 as the
intermediate vertex. This is the first case processed by Floyd’s Algorithm,
and so the weight matrix is in its initialized state as depicted in Figure 9.39
. Since the path length for this trip (from A = 1 to C = 1) in the weight
matrix is initially impossibly high (* in row 1, column 1 of the weight
matrix) and the trip through the intermediate vertex (B = 0) is 20 (the path
length from vertex 1 to 0 is 10, and the path from vertex 0 to 1 is 10), the *
in the weight matrix is overwritten with the shorter path length 20. The 20
in the first row and second column of the table reflects the overwriting
process. The 20 is highlighted in the table, to indicate that it is also
overwritten later in the algorithm, first in row 3, column 7, and then again
in row 5, column 7.
590

Next the algorithm considers the trip from vertex 1 to vertex 2, with
vertex 0 as an intermediate vertex (see the first row, third column of the
table). Examining the weight matrix, the path length from vertex 1 to 2 is
currently 3. For the alternate trip, the trip that travels through the
intermediate vertex 0, the path length of the trip from 1 to 0 and 0 to 2 is
15 (= 10 + 5). Since this is greater than the current value of the path length
stored in the weight matrix (3 in row 1, column 2) the value is not
overwritten. Thus, there is no path length entry in the second row, third
column of the table.
In total, 33 new path length values are written into the weight matrix,
with 18 of them overwritten (see the 18 highlighted path lengths in the
table) by shorter path lengths as the algorithm proceeds.
One of the attractive things about Floyd’s Algorithm is that it is
amazingly simple to code. For an intermediate vertex, B, the algorithm
simply compares the path length from A to C, for all A and C, to the path
length from A to B to C. The minimum of these two path lengths is written
into the weight matrix as the (shortest) path from A to C. This process is
repeated with B set to each vertex in the graph. The pseudocode of the
algorithm, including the initialization of a copy of the weight matrix,
aCopy, is given below. When the algorithm ends, the shortest path
between all pairs of vertices is stored in the weight matrix, aCopy.
591

Line 1 generates all possible intermediate vertices, b. Lines 2 and 3
generate all starting vertices, a, and all destination vertices, c. Line 4
compares the path length currently stored in the weight matrix aCopy from
vertex A to C to the path length from vertex A to B to C. When the latter
path length is shorter, Line 5 stores it in the array aCopy. The coding of
the algorithm is left as an exercise for the student.
It should be noted that Floyd’s Algorithm can be used to find the all-
pairs shortest paths in an undirected or in a directed graph. In a directed
graph, it could be the case that a path does not exist between two vertices.
The directed graph shown in Figure 9.40 illustrates this point in that there
is no way of reaching vertex 2 from any other vertex in the graph, and
vertex 3 can only be reached from vertex 2. In addition, none of the
vertices can be reached from vertex 3. When a path does not exist between
two vertices in a directed graph, Floyd’s Algorithm leaves the impossibly
high value of the weighting factor in the shortest paths matrix, as indicated
by the colored elements of the matrix on the right side of Figure 9.40 .
Figure 9.40 A Directed Graph and All-Pairs its Shortest Path Matrix
(* is an impossibly high edge weight or path length)
592

EXERCISES
Knowledge Exercises
1. Draw a binary tree that contains six nodes. Then modify the figure so
that it is no longer a tree but it is:
a. A connected graph.
b. A disjoint (unconnected) graph.
2. Draw a singly linked list that contains four nodes. Then modify the
figure so that it is no longer a singly linked list but it is:
a. A connected graph.
b. A disjoint (unconnected) graph.
3. Define the terms:
a. Graph
b. Undirected graph
c. Directed graph
d. Path
e. Path length
f. Unconnected (disjoint) graph
g. Cycle
h. Complete graph
i. Simple path
j. Strongly connected digraph
4. Calculate the number of edges in a complete graph containing five
vertices, and then draw the graph and count the edges to verify your
calculation assuming the graph is:
a. An undirected graph.
b. A digraph.
5. Give the adjacency matrix for the graphs A and B shown below.
593

6. Give the adjacency list representation of the graphs A and B, shown
above, assuming:
a. The number of vertices is fixed.
b. The number of vertices can expand to a potentially large value and
expanding arrays is a slow operation.
7. Draw the graph whose edges are represented by the following
matrices. Assume the vertices are named V 0 through V 4 .
8. Consider the graphs in the previous exercise. Which of them are:
a. Disjoint?
b. Directed?
c. Weighted?
d. Undirected?
9. The number of vertices and edges for two undirected and two
directed graphs are given in the following table. Fill in the last column
of the table to indicate which representation presented in Figure 9.12
would be the best performing representation for each graph from a time-
complexity viewpoint. Assume array expansion is slow.
594

10. Assuming that an undirected, unweighted graph is represented as
shown in Figure 9.12 a and that operations are performed in the vertex
number mode, give the signature and describe the actions of a method
that:
a. Deletes an edge from the graph.
b. Deletes a vertex from the graph.
c. Updates an edge in the graph.
11. Repeat the previous exercise assuming the graph is:
a. Directed and unweighted.
b. Undirected and weighted.
12. Assume a graph is represented as shown in Figure 9.13 and that
operations are performed in the vertex number mode. Give the errors
that could occur during:
a. An insert vertex operation.
b. An insert edge operation.
c. A show vertex operation.
13. Give the changes to the code presented in Figure 9.14 so that it
could store a digraph rather than an undirected graph.
14. Is an NLR traversal of a binary tree a depth-first or breadth-first
traversal?
15. Which traversal algorithm uses a queue, breadth-first, or depth-
first, and why?
16. Assuming vertex 1 in graph A of Exercise 5 is visited first, give
the order in which the vertices of the graph are visited if the traverse is
a:
a. Depth-first traversal.
b. Breadth-first traversal.
17. In a connected undirected graph, there is always a path from vertex
A to B, for all A and B. True or false?
595

18. In a connected digraph, there is always a path from vertex A to B,
for all A and B. True or false?
19. Identify an algorithm for determining if an undirected graph is
connected.
20. Identify an algorithm for determining if a digraph is strongly
connected.
21. Given a digraph’s transitive closure matrix, how can we determine
if there is a path from vertex v i to v j , and a path from vertex v j to v i ?
22. State the difference between a spanning tree and a minimum
spanning tree.
23. It is possible to have more than one minimum spanning tree for a
graph, true or false?
24. What does Dijkstra’s Algorithm do that Floyds Algorithm cannot
do?
25. What does Floyd’s Algorithm do that Dijkstra’s Algorithm cannot
do?
26. Dijkstra’s Algorithm is to operate on a graph containing n vertices.
Using Big-O notation, give the speed of the algorithm.
27. Floyd’s Algorithm is to operate on a graph containing n vertices.
Using Big-O notation, give the speed of the algorithm.
28. Draw the graph obtained when Dijkstra’s Algorithm operates on
the following graph, assuming vertex 4 is the starting vertex.
29. Give the contents of the returned array when Floyd’s Algorithm
operates on the graph shown in Exercise 28.
30. True or false, Floyd’s Algorithm can only operate on directed
graphs?
31. Of the algorithms we studied, which would be used to determine
596

the two-way roads to close a connecting group of towns, and still allow
access to all towns?
32. Of the algorithms we studied, which would be used to determine
the toll roads to travel to minimize the tolls when traveling from a
given town to all other towns?
33. Of the algorithms we studied, which would be used to determine if
there is a way to pass through all towns connected by one-way streets?
34. Of the algorithms we studied, which would be used to determine
the cheapest fares between all the cities that an airline flies to?
Programming Exercises
35. Extend the class SimpleGraph, shown in Figure 9.14 , so that it
expands the size of the arrays whenever necessary. Write a simple
application to demonstrate it functions properly.
36. We wish to expand the class SimpleGraph, shown in Figure 9.14 ,
to include methods to delete a vertex, delete an edge, fetch a vertex,
fetch an edge, update an edge, update a vertex.
a. Give the pseudocode of the methods’ algorithms.
b. Extend the code of the class SimpleGraph, to include the new
methods and write a simple application to demonstrate they function
properly.
37. Modify the class SimpleGraph so that it can store a weighted graph
and expand its methods to include those mentioned in Exercise 36.
38. Change the class SimpleGraph, shown in Figure 9.14 so that the
access mode of the revised class is the key field mode. Write a simple
application to demonstrate it functions properly.
39. Revise the class SimpleGraph, shown in Figure 9.14 , so that it
represents a graph using the scheme depicted in Figure 9.12 b . Write a
simple application to demonstrate it functions properly.
40. Revise the class SimpleGraph, shown in Figure 9.14 , so that it
represents a graph using the scheme depicted in Figure 9.12 c . Write a
simple application to demonstrate it functions properly.
41. Add a method to the class SimpleGraphDFT, shown in Figure 9.20
, that performs a breadth-first traversal. Write a simple application to
demonstrate it functions properly.
597

42. Add a method to the class SimpleGraph, shown in Figure 9.14 ,
that implements Warshall’s Algorithm. Write a simple application to
demonstrate it functions properly.
43. Modify the spanning tree method presented in this chapter so that
the client can specify the starting vertex. Write a simple application to
demonstrate it functions properly.
44. Add a method to the SimpleGraph, shown in Figure 9.14 , that
implements the minimum spanning tree algorithm. Write a simple
application to demonstrate it functions properly.
45. Implement the minimum spanning tree algorithm for the graph
representation depicted in Figure 9.12 b . Write a simple application to
demonstrate it functions properly.
46. Implement Dijkstra’s Algorithm for the graph representations
shown in Figures:
a. 9.12a
b. 9.12b
c. 9.12c
Write a simple application to demonstrate your implementation(s)
function properly.
47. Implement Floyd’s Algorithm for the graph representations shown
in Figures:
a. 9.12a
b. 9.12b
c. 9.12c
Write a simple application to demonstrate your implementation(s)
function properly.
48. We wish to supply water to a group of n villages. Given the
distances that separate each village from every other village, write a
program that determines the minimum total length of water pipe that
can be run between the villages, such that each village will be supplied
water.
49. A set of roads connect towns with multiple routes available from
each town to the others. Write a program that determines the minimum
plowing time necessary to clear the roads after a snowstorm so that any
of the area residents will be able to travel to all towns. The snow plow
travels 20 miles an hour, less 1 mile per hour for each 6 inches of snow
598

on the road. The number of towns, the roads that connect them, the
distance between the towns, and the amount of snow (in inches) on the
roads will input to the program. You may assume that the residents will
stay off the roads until the roads are plowed.
50. Given the flight times for a set of connecting flights between n
cities, write a program to determine and output the maximum flight
times between all pairs of cities.
1 This implies the edge does not begin and end at the same vertex.
2 It is only for graphical convenience that standard tree graphics (like those presented in
Chapter 7 ) do not use arrows. Rather, the parent-child relationship between two nodes is
implied by their relative position in the graphic.
3 In a directed graph, the elements in column i would also have to be examined.
4 This implies that the first vertex visited has no siblings unless the graph is a forest (see
Figure 9.1 f ).
5 When an adjacency list is used to represent the vertices, the next node visited is
usually the last vertex along the visited vertex’s adjacency list.
6 This implies that the first vertex visited has no siblings unless the graph is a forest (see
Figure 9.1 f )
7 For a five-vertex graph there are a total of 25 point-to-point trips: 0 to 0, 0 to 1, 0 to 2,
0 to 3, 0 to 4, 1 to 0, 1 to 1, etc. Only 16 of these are shown in each row of Table 9.1 because
for the trips that begin or end with the intermediate vertex, the indirect trip can only be longer
than the direct trip. For example, consider the trip from 0 to 1 with 0 as the intermediate
vertex. The trip from 0 to 0 plus the trip from 0 to 1 must be longer than the trip from 0 to 1.
Similarly, consider the trip from 1 to 0 with 0 as the intermediate vertex. The trip from 1 to 0
plus the trip from 0 to 0 must be longer than the trip from 1 to 0. Thus, the following 9 trips
can be eliminated from the search for an indirect shorter trip when vertex 0 is the
intermediate vertex (row 1 of the table): 0 to 0, 0 to 1, 0 to 2, 0 to 3, 0 to 4, 1 to 0, 2 to 0, 3 to
0, and 4 to 0.
599

APPENDIX A
ASCII Table
(First 127 Unicode characters)
(ASCII = A merican S tandard C ode for I nformation I nterchange)
600

601

602

APPENDIX B
Derivation of the Average Search Length of a
Nondirect Hashed Data Structure
The derivation will be based on the following assumptions:
• There are n nodes in the structure, and the size of the primary storage
area is N .
• All N primary storage area locations are equally probable to be
generated each pass through the collision algorithm.
The easiest way to present the derivation for average search length is
to consider the search for an unused location in which to perform an Insert
operation. By our first assumption, n of the N locations are used.
Therefore, the probability of hashing into an occupied location is n /N . For
example, if there are 700 nodes in the structure, and the size of the primary
storage area array is 1000, then n /N is 0.70, meaning there is a 70%
chance that we will hash into an occupied location. Under our second
assumption, every probe into the primary storage area has the same
probability, n /N , of hashing into an occupied location. This implies that
the previously hashed location is just as likely as any other to be hashed
into.
As an aside, if the collision algorithm were sophisticated enough to
not revisit a location after it was found to be occupied (as some are), then
the probability of a collision would decrease after each pass through the
collision algorithm. The collision probability of the first pass would still be
n /N , but since the location probed would no longer be considered, the
total number of remaining occupied spots would be one less, and the total
of the primary storage area would also be one less. Therefore, the
probability of a collision on probe two would be (n − 1)/(N − 1), on the
third pass (n − 2)/(N − 2), and on the i th pass (n − i − 1)/N − i − 1). Since
N is greater than n , this expression approaches zero as i increases.
Under our second assumption however, the probability of the hashing
and collision algorithms calculating an occupied location is n /N , for every
probe into the primary storage area. Since a location can only be occupied
or not occupied, the sum of the probability of finding the location
603

occupied, p o , and unoccupied, p u , must be equal to 1. Thus p o + p u = 1,
or p u = 1 − p o . Therefore, the probability of finding an unoccupied
location on any probe is (1 − n /N ). Returning to our example, if the
probability of a collision is 0.70 (70%), then the probability of finding an
unused location (a noncollision) is 0.30 (30%).
The probability of finding an empty spot by the i th probe, means that
the first i − 1 probes resulted in a collision, while the i th probe resulted in
a noncollision. The probability of the first i − 1 probes resulting a collision
is (n /N )probe1 * (n /N )probe2 *….* (n/N)probe i −1 = (n /N )
i −1 .
The probability of finding an empty spot on the i th probe is
therefore:
which is the probability of i − 1 collisions, followed by a
noncollision. Returning to our example again, the probability of finding an
open spot on the fourth probe is:
The average search length, for a group of operations, is the total
number of memory accesses to perform the operations divided by the
number of operations. To determine a formula for the average search
length, it is useful to consider a case where 10 Insert operations are
performed. Three of these find an empty spot with no collisions, and so the
number of memory accesses for each of these operations is one. Four
inserts resulted in one collision each, and so the number of memory
accesses for each of these inserts is two. Finally, three inserts resulted in
two collisions, and so they each take three memory accesses. The average
search length, L avg , per operation is:
Examining the above equation, we observe for this example:
3/10 is the probability of finding an open location on the first probe,
4/10 is the probability of finding an open location on the second probe,
3/10 is the probability of finding an open location on the third probe,
and that the multiplier of these probabilities are the probe numbers.
604

Generalizing this observation we have:
or, in general,
where: P 1 is the probability of finding an empty spot on the first
probe
P 2 is the probability of finding an empty spot on the second probe, etc.
From our earlier result, P i = (n /N )
i − 1 * (1 − n /N ), and so the above
can be written as:
But since i * (n /N )i − 1 is positive, extending the summation from
zero to infinity will produce a larger result. Therefore,
Since the summation in the above equation can be shown to be 1
equal to 1/(1 − n /N )2 the average search length is:
1 In the absence of a proof, use an Excel ® spreadsheet to calculate
and compare it to 1/(1 − n / N ) 2 for any n < N . Here's the proof (thanks to David Holtzman of St. Joseph's College, New York): Let n / N = x . Note that the right-hand side becomes 1/(1 − 2 x + x 2 ). Let us cross- multiply the infinite sum on the left by the quadratic denominator on the right with the hope of getting 1. This would prove the identity. Now 605 Here we have just collected the coefficients of each power of x from 2 up to infinity. (Note that the linear terms of 2 x and −2 x cancel out.) Now observe that i − 2 i − 2 + i + 2 = 0. Thus, we end up with 1, as desired. 606 APPENDIX C Proof That If an Integer, P , Is Not Evenly Divisible by an Integer Less Than the Square Root of P , It Is a Prime Number Preliminary Proof Given: R = the square root of P . Prove: If P = H * L , either H = L = the square root of P , or H or L is < R and the other is > R .
Proof :
If both were > R , then their product would be greater then P , and if both were less than R ,
their product would be less than P .
Desired Proof
Given:
There are no integers less than R (the square root of P ) that divide evenly into P .
Prove:
There are no integers greater than R that divide evenly into P .
Proof:
Assume: H is an integer > R and that H divides evenly into.
Then: Define L as L = P / H .
L is an integer, since L = P / H and H divides evenly into P
L must divide evenly into P since H is an integer and H = P / L .
Since P = H * L and H > R , then L < R (as per preliminary proof). The assumption implies L is an integer that divides evenly into P and is less than R , which contradicts the given fact. Therefore, the assumption cannot be true. 607 APPENDIX D Calculations to Show That ( n + 1) (log 2 ( n + 1) − 2) Is the Minimum Sort Effort for the Binary Tree Sort 608 609 Glossary A Abstract data type A set of data and the operations that can be performed on the data. Abstraction The idea of knowing how to use something without the underlying knowledge of how it functions. Access specifier A key word in a language that determines the scope of an entity. Access mode See key field mode or node number mode. Adjacency list A set of linked lists, one list associated with each vertex in a graph, in which each one of the nodes in a list stores information about one of the edges emanating from the vertex it is associated with. The information includes the adjacent vertex number and, in a weighted graph, the edge's weight. Adjacent vertices The vertices in a graph that have an edge between them, or in the case of a digraph, two edges between them (one from vertex a to b , and another from vertex b to a ). Adjacency matrix A square matrix in which each element stores one edge of the graph. ADT A bstract D ata T ype (see data abstraction). Algorithm A step-by-step solution to a problem that a computer can execute. Algorithm complexity A measure of the efficiency of an algorithm. See time and space complexity. Ancestor In a tree, a node's ancestor is any of the nodes encountered in moving from the root to the node, including the root node. Ancestor class See parent class. Application code In the context of data structures, it is the code that declares an object in the data structure class. Arc A directed edge in a graph. Argument A piece of information passed to a method. Array A technique for naming groups of memory cells that share a common first name and a unique last name. In Java, the unique last name, called an index, must be a literal integer or a variable that stores an integer. From a data structures perspective, an array is a data structure whose major design goal was speed. It is stored in contiguous memory, accessed 610 in the node number mode, and supports the Fetch and Delete operations. Array-based structures Those data structures that utilize an array as their underlying structure (at their lowest level). ASCII A merican S tandard C ode for I nformation I nterchange. ASCII is a table of characters and the bit patterns that represent them. AVL tree A self-balancing binary search tree that is always balanced to within one level. B Balanced binary tree A binary tree in which all the levels of the tree below the highest level are filled. Base case The base case is the known portion of a recursive problem solution. It is often called the escape clause. Base class See parent class. BFT See breadth-first traversal. Big-O analysis An analysis technique that approximates the upper bound of a function. It is used to determine the complexity of an algorithm. Big-O notation O(n 2 ) is read as order n 2 . Binary search A search technique used on a sorted list of n data items in which the middle item is accessed, half the list is discarded, and the list is replaced with the remaining half of the list. This process is repeated until the item being searched for is found. Its speed complexity is O(log2 n ). Binary search tree A binary tree in which the nodes are arranged such that, for every node in the tree, all the keys in a node's left subtree are less than its key, and all the keys in a node's right subtree are greater than its key. Binary tree A tree in which every node in the tree has, at most, two children. Bit Binary digit. A single on-off switch of storage, which, when on is designated 1, and when off is designated 0. Boolean A variable type that can assume one of two values, true or false. Breadth-first traversal A graph traversal technique; for each vertex visited all of its siblings are visited before any of its descendents are visited. In a graph, siblings are vertices that have an edge from a common vertex, assuming the common vertex has already been visited. 611 Bubble sort A sorting technique that compares adjacent items in a list and exchanges them when they are not in their proper sorted position relative to each other. During each pass through the algorithm at least one item is positioned in its final sorted position in the list. Bucket Elements of the primary storage area in a hashed structure that are not used to store references to a single node, but rather to multiple nodes. For example, they are used as a header of a singly linked list of nodes or a reference to an array of nodes. Built-in data type A data type that is defined as part of a language standard. Byte Eight contiguous bits of storage. C Ceiling x Rounding a floating point value x up to the next highest integer. Child class A class that inherits from (or is derived from) another class. Child node A child node is a node that comes directly after a node in a tree. Any node in a tree that is not the root node. Circuit A cycle that passes through every vertex of a graph once. Circular doubly linked list A doubly linked list in which the forward link field of the last node references the first node, and the back link field of the first node references the last node. Circular singly linked list A singly linked list in which the forward link field of the last node references the first node. Class A programming construct that permits the programmer to define a type consisting of data definitions and the methods that operate on that data. A template for an object. Client code The code that declares an instance of an ADT, or invokes a method. Clone A newly created object whose data fields contain the same values as an existing object. Clustering The tendency of nodes in a hashed data structure not to be randomly distributed over the primary storage area. Collision When two keys map into the same position in the primary storage area of a hashed structure. Collision resolution The process of finding an alternate position in a hashed structure after a collision has occurred. 612 Compile time That point in time when the correctness of a program's syntax and semantics is verified and, if correct, the program is translated into a lower-level language. Complete tree A tree that contains the maximum number of nodes for its height. Complete graph A graph is said to be complete if the path length between any two distinct vertices is 1. Complexity See algorithm complexity. Connected graph A graph is a connected graph if, given any two of its vertices v i and v j , there is a path from v i to v j , ignoring the direction of the edge. Constructor A method in a class that has the same name as the class and is used by a client to create an object. Contiguous Adjacent; side-by-side. Cycle A path in a graph that begins and ends with the same vertex. D Data Information. Data abstraction The idea that we need not know the details of how data is stored in order to access it. Data encapsulation Utilizing compiler-enforced protocols to restrict access to the data that a program processes. Data member A variable definition that is part of a class. Each instance of the class will be allocated storage for the variable. Data structure A data structure is an organization of information, usually in memory, for better algorithm efficiency. Data type A name that defines a set of values and the operations that can be performed on them. Descendent class See child class. Deep copy The process of copying the contents of all of the data members of one object into the data members of a second object. Default constructor A constructor with no parameters that sets a newly created object's data members to default values. Delete A fundamental operation performed on data structures that removes a node from the structure. Depth-first traversal A graph traversal technique in which for each vertex visited all of its adjacent vertices (its descendents ) are visited before any of its siblings are visited. In a graph, siblings are vertices that 613 have an edge from a common vertex, assuming the common vertex has already been visited. Dequeue An operation performed on a queue that fetches and deletes the node that has been in the queue the longest amount of time. Derived class See descendent class. Descendent class A class that inherits from another class. Descendent of a node A node encountered in a path from the node. Digraph A graph for which the direction of travel along the graph's edges is specified. Directed edge An edge for which the direction of travel is specified. Directed graph See digraph. Disjoint graph A graph that is not connected. Division Hashing function A hashing function in which the key is divided by an integer and the remainder is used as the key's home address. Doubly-linked list A linear linked list in which each node contains the address of its successor node and its predecessor node. Dummy node A node in a data structure that will not be used to store client information. Dynamic memory allocation The allocation of storage for variables at run-time. Dynamic structure A data structure that can expand or contract at run-time via dynamic memory allocation. E Edge A bidirectional path between two vertices in a graph. Edge weight A value assigned to an edge in a weighted graph. Encapsulation The concept of combining data and the operations that operate on the data as one entity. Access to the data is restricted, normally by compiler-enforced protocols, to the use of the encapsulated operations. Enqueue An operation performed on a queue that inserts a node at the end of the queue. Exponential complexity Algorithm complexity that is a function of the power of the number of items (n ) being processed, denoted O(c n ). Exception An out-of-the-ordinary event that occurs during the execution of a program. F 614 Factorial of n The product of the integers from n to 1 . Denoted n !. Factorial complexity Algorithm complexity that is a function of the factorial of the number of items (n ) being processed, denoted O(n !). Fetch A fundamental operation performed on data structures that returns a node from the structure. Fibonacci sequence A sequence in which each term's value is the sum of the values of the two terms before it, with the exception of the first two terms whose values are 1. Floor x Rounding a floating point value x down to the next lowest integer. Field An indivisible piece of data. First-in-first-out (FIFO) The idea that the first item added to a data structure will be the first item fetched from the data structure. Flag A variable used by a program to indicate that an event has occurred. Floating point number A number with a fractional part; a real number. Flowchart A graphical representation of an algorithm aimed at depicting execution path. Folding The process of dividing a key into groups of bits and then arithmetically adding the groups to produce a pseudokey. Fold shifting A folding algorithm. Four-k-plus-three prime A prime number that, when reduced by 3 and then divided by 4, produces an integer value. Front of a queue The position of the next item to be fetched (and deleted) from a queue. Function A method that returns a value via a return statement. G Garbage collection The process of returning memory assigned to a program, or to a data structure, that is no longer in use to an available memory pool. General solution The part of a recursive algorithm that uses the reduced problem to solve the original problem. H Hashed structure A data structure that uses a hashing algorithm to locate a node. 615 Hashing A process in which a node's key is used to determine the node's probable location in a data structure without searching through memory. Hashing function A function that maps keys into node locations. Header The reference variable that stores the location of the first node in a linked list. Heap A binary tree in which the key of each node in the tree is larger than the keys of both of its children. Heap sort A sorting algorithm that transforms a list of items to be sorted into a heap. Then it repeatedly swaps the root into its correct sorted position and rebuilds the heap out of the remaining items. Height balanced See balanced binary tree. Height of a tree The number of nodes in the longest path from the root to any of the leaf nodes. Home address The location produced by preprocessing and hashing a key. I Indegree The number of directed edges incident upon a vertex of a graph (or node of a tree). Index An item (an integer in Java) used to specify a particular element of an array. Infix notation A technique used for writing mathematical expressions in which the operator is placed between the two operands it operates on. Inheritance The object-oriented concept of incorporating all of the member data and methods of an existing class into a new class. The new class can add additional methods and data. Inline function The inclusion of a copy of the executable code of a method into a program's executable module wherever the method is invoked (as opposed to transferring the execution to a single copy of the method at run-time). Inner class A class that is defined inside of another class. Inorder traversal A binary tree traversal technique in which a node's entire left subtree is visited before the node, then the node is visited, and then the node's entire right subtree is visited recursively. Insert A fundamental operation performed on data structures that adds a node to the structure. 616 Integer A number without a fractional part. Interface A Java construct that specifies the signatures of methods, not their code. It is a “promise” to the translator that methods with these signatures will be written in the future. Once the interface is defined, the methods can be invoked on instances of the interface without a compile error, and an instance of a class that codes the methods can be referenced by an interface reference. Instance of a class An object in a class (created with the new operator in Java). Instantiation The process of creating an object. Iteration A single pass through a loop construct. Iterative solution An algorithm whose solution involves loop(s), not recursion. Iterator An item that moves sequentially through a linear list and retains its position between moves. J Java An object-oriented language whose design goal was platform independence. K Key field A designated field in a node whose contents is used to identify the node. Key field mode An access mode in which the contents of the key field is used to specify which node in a data structure is to be operated on. L Last-in-first-out (LIFO) The idea that the last item added to a data structure will be the first item fetched from the data structure. Leaf A node in a tree that has no children. Left child In the standard depiction of a binary tree, the node that is below and to the left of a node. Left subtree In the standard depiction of a binary tree, all of the nodes in a binary tree below and to the left of a node. Level of a node The number of nodes in the path from the root to the node (not counting the node). Level of recursion The number of times a recursive method has invoked itself. Linear collision algorithm A collision algorithm that adds 1 to the 617 current primary storage area location to determine the next location to be examined (usually using modulo arithmetic). Linear list A set of nodes in which there is a unique first and last node, and every other node has a unique predecessor and successor. Linear Quotient collision algorithm A collision algorithm that adds the quotient obtained by dividing the key by the size of the primary storage area to the current primary storage area location to determine the next location to be examined (usually using modulo arithmetic). When the quotient is an even multiple of the size of the primary storage area, a predetermined 4k + 3 prime is used in place of the quotient. Link field A field in a node that stores the address of (reference to) another node in the structure. Linked list A linear list in which each node contains at least one reference to another node in the list. List An ordered set of nodes. LNR traversal See inorder traversal. Loading factor The ratio of the number of nodes in a hashed structure to the size of the primary storage area. Logarithmic complexity Algorithm complexity that is a function of the base-two logarithm of the number of items (n ) being processed, denoted O(log2 n ). M Member method A method that is coded inside the definition of a class. Merge sort A sorting technique that divides a list into two halves, sorts each half, and then merges the two sorted lists into one sorted list. The merge is performed by indexing through each list, comparing their members, and copying the smaller member into the sorted list. Each half of the list is sorted the same way. Method A programming language construct (or subprogram) that consists of a sequence of instructions to perform a specific task (or set of tasks). The transfer of the execution path to and from it, and the transfer of the information shared between it and the invoking code, is provided for by the compiler. Minimum Spanning Tree A spanning tree of a weighted graph whose edges are chosen such that the sum of their weightings is minimized. 618 Modulo operator An operator that determines the remainder of division. N NLR traversal Stands for n ode-l eft subtree-r ight subtree. See preorder traversal. Node A collection of related fields. Node number mode An access mode in which the number of a node is used to specify which node in a data structure is to be operated on. A node's number usually corresponds to its position in a linear structure. Null The value stored in a reference variable when it does not reference an object. O Object A particular occurrence of a class. The memory allocated for the data members and the associated methods of the class. Open Addressing collision algorithm A collision algorithm that resolves collisions by producing an alternate position in the primary storage area (as opposed to a position in a linked list, or an index into an array referenced by the primary storage area). Operand The item in an expression on which an operation will be performed. Operator The item in an expression that specifies the action to be performed on one or more of the operands in the expression. Outdegree The number of directed edges emanating from a vertex of a graph (or node of a tree). Outer class A class that has another class (an inner class) defined inside it. Overflow The error that occurs when an attempt is made to insert an item into a full restricted structure. P Package ac cess The default Java access specifier (i.e., public, private, or protected is not specifically declared) for a class' member data and methods. Their scope is the code of any class in the same package. Parameter The type of a piece of shared information and the name used to refer to the information while a method (subprogram) is in execution. Parent class A class that is inherited from. Parent of a node A node's unique predecessor in a tree. 619 Path A sequence of edges that connect two vertices in a graph. Perfect Hashed structure A hashed structure that uses a Perfect Hashing function. Perfect Hashing function A hashing function that maps each key, or pseudokey, into a unique location in the primary storage area. Pivot value The item in a list of items to be sorted by the Quicksort algorithm, which will divide the partitions used by the algorithm. Pointer Another name for a reference variable (used in C and C++). Polynomic complexity Algorithm complexity that is a function of the number of items processed, n , raised to a power (e.g., O(n 2 ), O(n 3 ),…). Pop An operation performed on a stack that fetches and deletes the node that has been in it the shortest amount of time. Postfix notation A technique used for writing mathematical expressions in which the operator is placed after the two operands it operates on. Postorder traversal A binary tree traversal technique in which a node's entire left subtree and then its entire right subtree is visited before the node is visited, recursively. Predecessor node Generally speaking, the node that comes before a node in a structure. In a linked list, the node that contains the address of a node is its predecessor. In a directed graph, the vertex whose directed edge is incident upon a vertex is its predecessor. Prefix notation A technique used for writing mathematical expressions in which the operator is placed before the two operands it operates on. Preorder traversal A binary tree traversal technique in which a node is visited, and then its entire left subtree is visited, and then its entire right subtree is visited, recursively. Preprocessing The action performed on a key in a hashed data structure to convert it to the type of the hashing function's independent variable, or to more evenly distribute nodes over the primary storage area, or both. Prime number A number that is only evenly divisible by 1 and itself. Private access Limiting the scope of a class' data members or methods to code of the class' methods. Primary clustering In a hashed structure, when the nodes that map into the same home address are located in close proximity to the home 620 address. Primary storage area In a hashed data structure, the storage indexed by the home address generated by the hashing function. The storage is usually implemented as an array, and often it is referred to a hash list or a hash table. Primitive data type In Java, the built-in types boolean , byte , short , int , long , char , float , and double . Primitive variable A named memory cell that stores the value of a primitive data type. Priority queue A restricted data structure that fetches (and deletes) nodes in an order based on a priority assigned to each node. Probe In a hashed structure, one “look” into the structure. Procedural abstraction The concept of knowing how to use a method (subprogram) without knowing the underlying algorithm or implementation. Programmer defined data type A data type whose definition is specified as part of a program (as opposed to a built-in data type). Pseudocode A sequence of statements used to specify an algorithm that resembles a programming language but exhibits less precise syntax. Pseudokey The result of processing a key through a preprocessing algorithm. Pseudorandom preprocessing A preprocessing algorithm that attempts to introduce more randomness into the keys before they are processed by a hashing function. The objective is to reduce clustering and thereby reduce the search length. Public access Setting the scope of a class' member method or data to the code of any method in a program. Push An operation performed on a stack that inserts a node into the stack. Q Quadratic collision algorithm A collision algorithm that adds the square of the probe number to the current primary storage area location to determine the next location to be examined (usually using modulo arithmetic). Quadratic complexity Algorithm complexity that is a function of the number of items processed (n ) squared, denoted O(n 2 ). Query An inquiry. 621 Queue A restricted data structure that operates on a first-in-first-out basis. QuickSort A sorting technique in which a list of items is partitioned into two sublists divided by a pivot item. Then, the items in the left and right sublists are repositioned such that each of the items in the left and right partitions are less than and greater than the pivot value, respectively. This process is applied to each of the partitions recursively. R Random number A number chosen from a set of numbers under the condition that each number in the set has an equal probability of being chosen. Real number A number with a fractional part. Real-time Processing events at the time they occur, usually so rapidly that the processing appears to be instantaneous. Rear of a queue The position in a queue where the next node inserted will be placed. Recursion Defining the solution to a problem, at least partially, in terms of similar problems that are closer to a known solution. Recursive method A method x is a recursive method if it invokes itself, or if it invokes a method that eventually leads to an invocation of method x . Recycle See garbage collection. Recurrence relation An algorithm for generating the terms of a sequence by using the terms previously determined. See the Fibonacci sequence. Reduced problem In a recursive algorithm, a problem similar to a problem to be solved that is closer to the base case and eventually becomes the base case. Red-Black tree A self-balancing binary search tree that assigns a color (red or black) to each node in the tree as part of its balancing algorithm. Reference variable A variable that stores a memory address. Restricted structure A set of data structures whose operations and access modes are severely restricted. Among other restrictions, the Fetch and Delete operations are combined, key field mode access and Update are not allowed, and node number mode is restricted. Return A programming language statement that terminates the 622 execution of a nonvoid method and returns a value to the invoking method. Right child In the standard depiction of a binary tree, the node below and to the right of a node. Right subtree In the standard depiction of a binary tree, all of the nodes below and to the right of a node. Root The unique first node in a tree (that has no predecessor). Rotation A repositioning of nodes performed by AVL and Red-Black trees as part their balancing algorithms. Run-time The time during which a program is in execution. S Scope of an item That part of a program in which the item can be accessed. Search tree A tree whose nodes are positioned to facilitate the operations performed on them (e.g., binary search trees, AVL trees, Red- Black trees). Secondary clustering In a hashed structure, when nodes with the same home address (although scattered throughout the primary storage area) generate the same sequence of collision addresses. Sentinel A signal. Sentinel value An unrealistic value of a data item. Sentinel loop A loop that terminates when a sentinel value is detected. Sequential search A search technique in which the items of a linear list are searched in item number order. Shallow copy The process of copying the address of an object from one reference variable to another. Both variables then refer to the same object. A new object is not created. Shortest path The path between two vertices in a directed graph that the minimizes the sum of the edge weights. Siblings In a tree, nodes with the same parents. In a graph, siblings are vertices that have an edge from a common vertex, assuming the common vertex has already been visited. Simple cycle A simple path in which the first and last vertex is the same. Simple path A path in which all vertices encountered along the path, except possibly the first and last vertices, are distinct. Singly linked list A linked list in which each node in the list contains 623 one link field. Software engineering A branch of computer science that deals with the techniques of developing software products that are fault free, within budget, delivered on time, and satisfy the clients' needs now and in the future. Software eng ineer A programmer that utilizes the techniques of software engineering to develop a software product. Space complexity The efficiency of an algorithm from a memory overhead viewpoint. Spanning tree The set consisting of a graph's vertices and a subset of its edges chosen such that each vertex can be reached from every other vertex and there are no cycles. Stack A restricted data structure that operates on a last-in-first-out basis. Stack frame The collection of information on a stack maintained by an operating system that is particular to a specific invocation of a method (subprogram). Strongly connected digraph Given any two vertices in a digraph there is a way of reaching one from the other by traveling along the graph's edges, considering the direction of the edges. Subclass See child class. Subtree of a node All of the nodes in a tree whose root is a child of the node. Successor Generally speaking, the node that comes after a node in a structure. In a linked list, a node's successor is the node whose address is stored in its link field. In a directed graph, vertex a is vertex b 's successor if there is a directed edge from vertex b to vertex a . Super class See parent class. Symmetric matrix A matrix, in which, for every pair of elements a ij and a ji , a ij = a ji . Synonym Two keys that map into the same home address of a hashed structure's primary storage area. T Time complexity The efficiency of an algorithm from an execution speed viewpoint. Token An item in a string delimited by white space. Top of a stack The position in a stack were the next pop will be 624 performed. Traversal The process of locating and visiting each node in a structure once and only once. Tree A connected graph with no cycles. U Underflow The error that occurs when an attempt is made to fetch (and delete) an item from an empty restricted structure. Undirected graph A graph in which bidirectional travel is permitted along its edges. Update A fundamental operation performed on data structures that changes the contents of the fields of a node stored in the structure. User The person interacting with a program while it is executing. Unicode A table of characters and the 16 bit patterns that represent them. The ASCII characters are the first 128 entries in the Unicode table with eight zero's added to the high order side of the ASCII bit representations. Java represents characters internally (i.e., in storage) using the Unicode table. V Variable A named memory cell that stores a specific type of information, and whose contents can change during the execution of a program. Vertex A node in a graph. Visiting a node The process of performing an operation on a node in a data structure. Void method In Java, a method that does not return a value. In other programming languages, a method that does not return a value by way of a return statement. W Wall time The passage of time as measured by a clock (on the wall). Weakly connected graph A connected digraph that is not strongly connected. Weighted graph A graph whose edges are each assigned a value. Weight of an edge See edge weighting. 625 Index The index that appeared in the print version of this title does not match the pages in your eBook. Please use the search function on your eReading device to search for terms of interest. For your reference, the terms that appear in the print index are listed below. A absolute speed case study methodology abstract data type (ADT) abstraction access function access modes key field node number access modifiers adjacency list vs.adjacency matrix adjacency matrix adjacent vertex ADT. See abstract data type algorithm complexity. functions space ( see also density) time (absolute speed, relative speed) algorithm speed absolute 23 relative algorithms Dijkstra's Floyd's Warshall's all-pairs shortest path algorithm. See Floyd's algorithm API classes ArrayList Hashtable LinkedList 626 ListIterator Stack System TreeMap application code array based structures generic data structure application graph structures hashed structures iterators linked structures queue stack trees application programmer interface. See also API classes array access mode abstraction base address column-major order declaration of design goals expanding index Java syntax mapping function memory model multidimensional of objects of primitives one-dimensional mapping function operations allowed row-major order subscript two-dimensional mapping function two dimensional syntax viewed as a binary tree viewed as a data structure array-based structures binary tree ( see binary search tree array-based) errors graphs ( see graphs) 627 hashed structures ( see direct hashed structures; LQHashed structure) performance comparison sorted array unsorted array ( see also unsorted array-based structures) unsorted-optimized array ( see also unsorted-optimized array-based structures) ArrayList class ASCII table AVL trees balance factor delete algorithm extent of imbalance fetch algorithm insert algorithm performance vs.red-black rotations update algorithm B backtracking application of methodology decision points decision trees generalized backtracking algorithm Knights Tour problem Knights Tour solution maze example methodology Queens Eight problem recursion (use of) summary of methodology adaptation balanced trees advantage of AVL trees parent-child node distribution red-black trees search trees base case for several problems basic operations action of standard signatures 628 belongs-to relationship BFT. See breath-first-traversal Big-O algorithm complexity analysis notation Big-Omega ( ) Big-Theta (Θ) binary search performance use of binary search trees. See also AVL; red-black trees advantages of array-based ( see binary search tree array-based) balancing client code definition of delete algorithm density example of fetch algorithm findNode algorithm implementation (linked) initialization insert algorithm node representation operation algorithms (linked) performance performance comparison positioning nodes in showAll method skewing speed complexity traversal TreeNode class TreeNodeWrapper class binary search tree array-based children's index rule (2i + 1, 2i + 2) compared to linked implementation delete algorithm 629 density density if balanced density if imbalance fetch algorithm imbalance (tolerance for) initialization insert algorithm performance performance comparison root node index size of the array speed complexity update algorithm binary trees. See also binary search trees density (linked) definitions and terms mathematics of standard graphics traversal ( see binary tree traversal) binary tree representations array linked binary tree sort algorithm overhead performance performance comparison sort effort sorting process speed complexity binary tree traversal definition of examples of LNR traversal standard traversals trace of LNR visiting a node breadth-first traversal (BFT) algorithm example of visitation order queue use 630 Bridges of Koningsberg bubble sort algorithm early termination overhead performance performance comparison sort effort sorting process speed complexity swap effort bucket hashing. See also linked hashed structures built-in structure types burdened labor rates C child and parent references child class circuits. See cycles circular singly linked list class naming convention data members definition of definition code method naming convention methods classes BinaryTree Factorial FactorialTrace GenericStack Listing Listing2 LqHashed NewNode Node Person PersonGeneric PhoneListing 631 Queue SimpleGraph SimpleGraphDFT SinglyLinkedList SinglyLinkedListIterator SllIterator SllExternalIterator Stack TreeNode TreeNodeWrapper UnsortedOptimizedArray UOA (generic) UOAUtilities WeightLossClass classes in the API. See API Classes client code. See also application code error detection generic data structure application client side code. See client code clock pulse clone. See deep copy clustering primary clustering secondary clustering collision. See also non-perfect hashing collision algorithms. See also non-perfect hashed structures clustering linear linear-quotient multiple accesses non-open addressing open addressing primary clustering problems with quadratic secondary clustering column-major order Comparable interface Complexity. See also algorithm complexity complexity functions compareTo method 632 complete trees connected graph connectivity determination of speed complexity constant complexity constructor containership memory model contiguous copies. See shallow copy;deep copy current loading factor C++ CPU register cycles D data data abstraction data encapsulation data structure basic operations client side declaration client side error checking design process error checking generic guidelines ( see also generic data structures) implementation class implementation of memory selection process speed standard for goodness terminology data structure performance calculation of comparison of frequency weighted average importance of data structures. See also array-based structures built-in ( seealso graphs;hashed structures; linked implementation of) 633 programmer defined performance comparison ( see queue; stack;trees) decision points decision trees deep copy in data structures methods syntax water analogy default constructor default prime delete operation delete problem solution deleting edges deleting nodes array-based structures binary search trees graphs hashed structures linked structures queues stacks deleting vertices density constant vs.variable overhead definition of examples formula for density vs.node width array-based structures binary tree (array-based) binary tree (linked) doubly linked list hashed structures (array-based) hashed structures (linked) multi-linked list 210 queue ( see also density vs.node width, stack) singly linked list stack depth first traversal (DFT) algorithm 634 client side code example of implementation stack use dequeue algorithm method derived class. See child class descendent class node vertex design process. See trade-off DFT. See depth first traversal digit extraction preprocessing Dijkstra's shortest path algorithm algorithm process application of client side use of example of implementation guidelines pseudocode speed complexity trace vs.Floyd's algorithm vs.minimum spanning tree directed edge directed graph direct hashed structure delete algorithm density encapsulation fetch algorithm garbage collection hashing function initialized state insert algorithm loading factor overhead performance primary storage area speed 635 string preprocessing subtraction preprocessing update algorithm direct recursion directed tree disjoint graph division hashing function dominate terms dot operator (.) doubly linked list dummy nodes in hashed structures in linked structures duplicate definition error dynamic hashed structures dynamic memory allocation dynamic programming advantage of Hashtable class ( see API classes) hybrid approach memory model dynamic structures array-based binary tree hashed linked E edges definition of information stored in insert algorithm operations performed on representation of weighs edge array. See adjacency matrix encapsulation advantages of data demonstration of in classes enqueue 636 algorithm method error handling in applications errors client side detection of demonstration of detection of ( see specific data structure) in basic operations out-of-bounds queue overflow and underflow stack overflow and underflow system memory expanding arrays at run time algorithm speed exponential complexity extending classes extends key word F factorial factorial complexity factorial methods iterative recursive fetch operation fetching nodes array-based structures binary search trees graphs hashed structures linked structures queues stacks Fibonacci sequence field FIFO findNode method first-in-first-out (FIFO) Floyd's algorithm basis of algorithm 637 pseudocode shortest path matrix speed complexity trace vs.Dijkstra's algorithm folding preprocessing speed complexity four-k-plus-three prime (4k + 3) frequency weighted average time front of a queue G garbage garbage collection hashed structure Java memory manager linked list queue structure sorted array-based structure unsorted array-based structure unsorted-optimized array-based structure general solution for several problems generic classes generic data structures. See also generics advantages of application code array-based structure conversion class GenericStack class UOA client side coding guidelines conversion methodology conversion to design considerations hashed heterogeneous data set interfaces node definition stack generic methods GenericNode interface 638 generics. See also generic data structures classes Java features methods object declaration parameters primitive arguments type placeholders GenericStack class get prefix get methods getAge getKey graph access modes vertex number key field graph connectivity definitions of in directed graphs minimum spanning tree shortest paths ( see Dijkstra's algorithm; Floyd's algorithm;) spanning trees in undirected graphs Warshall's algorithm graph operations insert algorithms methods on edges on vertices graph representations adjacency list adjacency matrix edges impact on performance selection criteria (summary) vertices graph terminology adjacency complete connected graph connected vertex cycle 639 directed (digraph) disjoint edges path path length shortest path simple path strongly connected graph undirected graph vertex weakly connected graph weighted weights graph traversals BFT (breadth first traversal) DFT (depth first traversal) visitation graphs adjacency adjacency lists adjacency matrix all-pairs shortest path applications of Bridges of Koningsberg breadth-first traversal client side code compared to trees and linked lists connectivity ( see graph connectivity) definition of depiction of depth-first traversal implementation using arrays maximum number of edges minimum spanning tree operations ( see graph operations) representing edges ( see graph representations) representing vertices ( see graph representations) shortest path ( See also Dijkstra's and Floyd's algorithms) spanning tree terminology ( see graph terminology) transitive closure matrix ( see also Warshall's algorithm) 640 traversing ( see traversing graphs) H hashed structures. See also direct hashed structure and LQHashed structure advantages of direct hashed dynamic ( see dynamic hashed structures) LQHashed structure non-perfect LQHashed structure ( see also non-perfect hashed structures) operation algorithms perfect ( see also perfect hashed structures) performance of primary storage area primary storage area size schemes hashcode method hashing. See also non-perfect hashed structures access function alphanumeric keys ( see preprocessing algorithms) buckets collisions ( see collision resolution algorithms) current loading factor linear probing ( see collision algorithms) linear quotient probing ( see collision algorithms) loading factor maximum loading factor numeric keys ( see preprocessing) open addressing ( see collisions) perfect ( See also perfect hashed structures and direct hashed structure) preprocessing ( see preprocessing) process quadratic probing ( see collision algorithms) hashing access functions direct division perfect Hashtable class client code constructors 641 encapsulation (lack of) equals method generic nature of hashCode method key types methods use of hasNext method header heap array representation definition of examples implications of heap sort algorithm definition of a heap highest-level right-most parent initial heap initial heap process overhead performance performance comparison re-heap down algorithm sort effort sorting the heap sorting the heap process speed complexity swap effort hashed structures height of a tree heterogeneous data set in a generic structure high probability of access ordering homogeneous data set homogeneous structure and generics I imbalanced trees implementations binary search tree 642 graph generic optimized array generic stack hashed structure iterator optimized array queue singly linked list stack implements key word index indirect recursion information bytes inheritance initial heap initialization of structures array-based binary trees graphs hashed structures linked list queues stacks inner classes inorder traversal input method insert operation inserting edges inserting nodes array-based structures binary search trees graphs hashed structures linked structures queues stacks interfaces Comparable KeyMode GenericNode interface iterative solutions 643 iterator advantages of client side multiple iterators client side single iterator external implementation of multiple iterators implementation of single iterator internal type Java's ListIterator class methods multiple iterators objects operations performance improvements singly linked list iterator classes J Java API data structure classes. See API classes Java built in types Java character representations Java code. See also methods and classes array-based structure binary search tree structure deep copy depth-first traversal factorial Fibonacci sequence graph structure hashed structure initial data base loading merge sort node definition classes queue Quicksort shallow copy singly linked list node class singly linked list structure singly linked list structure with Iterator stack Java generic typing. See also Java review-generics Java Interface construct Java memory manager 644 garbage collection Java methods arraycopy in ArrayList class compareTo in Hashtable class hasMoreTokens length in LinkedList class in ListIterator class nextToken in Stack class in TreeMap class Java primitive types Java review arrays of objects arrays of primitives classes clones ( see deep copy) containership deep copies generics get methods inheritance input method method naming prefixes objects return set methods void and non-void methods K key key field key mode access Knights Tour problem L last-in-first-out (LIFO) leaf deletion of left subtree 645 length of an array levels of a binary tree levels of recursion LIFO. See last-in-first-out linear collision algorithm linear complexity linear list access function linear logarithmic complexity linear probing. See linear collision algorithm linear-quotient collision algorithm default prime example of multiple accesses primary clustering secondary clustering speed complexity link field linked hashed structures buckets collision resolution density hybrid performance performance comparison primary storage area size speed considerations linked implementation of graphs hashed structures queues singly linked list stacks trees linked list nodes. See singly linked list linked lists algorithm discovery methodology circular singly linked double ended singly linked doubly linked iterators ( see iterators) multi-linked 646 non-contiguous memory performance of singly linked ( see singly linked list) sorted singly linked linked representation of trees linked structures advantages of LinkedList class client side list iterator. See also iterators advantages of Listing class Listing2 class ListIterator class client side LNR loading factor direct hashed structures LQHashed structure optimum logarithmic complexity LQHashed structure alphanumeric key preprocessing collision resolution algorithm default prime delete algorithm density use of dummy node fetch algorithm generic considerations getKey method hashing algorithm insert algorithm implementation implementation restrictions loading factor operation algorithms performance performance comparison performance degradation preprocessing primary storage area sizing 647 ShowAll method speed complexity update method LRN M macro level of a program maximum loading factor memory fragmentation memory model arrays array of objects array of primitives containership deep copy dynamic hashed structures objects shallow copy merge sort algorithm implementation merging process overhead performance performance comparison speed complexity sort effort sorting process swap effort merging sorted sublists method naming conventions prefixes methods arrayCopy compareTo deepCopy fourKplus3 prime generator fibonacci findNode getAge getKey getPushed 648 Hanoi hashCode input LNRoutputTraversal mergeSort nFactorial outputIntegerArray outputNumericArray (generic) quickSort setAddress setPushed setWeight showAll showEdges showVertex stringReverse stringToInt toString transitiveClosure (Warshall's algorithm) xToN minimum spanning tree algorithm algorithm process algorithm trace application of client side code definition of modularization modulo arithmetic multiple access problem N n factorial friends analogy iterative method non-recursive definition performance recursive definition recursive method trace of method nanosecond new operator 649 action of next field nextToken method nlog2n complexity NLR node in binary tree child depiction of in doubly linked list dummy leaf parent reference root in singly linked list visiting Node class node deletion. See deleting nodes node fetch. See fetching nodes node insertion. See inserting nodes node update. See updating nodes node width non-access instruction non-open addressing non-perfect hashed structures. See also LQHashed structure alphanumeric keys collisions collision resolution algorithms the delete problem density general flow hashing functions implementation of ( see also LQHashed structure) optimum loading factor preprocessing algorithms primary storage area optimum sizing rule primary storage area sizing issues prime number generator search length use of prime numbers in 650 non-void methods null reference in error detection O object accessing (.operator) arrays of and classes creating deep copy of memory allocation reference to shallow copy of sorting speed vs.primitives Object class open addressing linear probing linear quotient probing problems with quadratic probing operators dot (.) new optimized array-based structure. See UnsortedOptimized array-based structure optimizing compiler optimum loading factor order of magnitude analysis. See Big-O overhead calculation of definition of example fixed vs.variable overflow P package access parameters parent class node 651 parent and child object references parsing strings StringTokenizer class partition pointers partitions path peek operation perfect hashed structures. See also direct hashed structure definition of overhead perfect hashing minimum hashing function static key set and subtraction preprocessing unique mapping performance. See also data structure performance;algorithm complexity pivot value pointer polynomial complexity pop operation. See also stack array-based; stack linked based post-fixed notation evaluation algorithm post-order traversal preorder traversal preprocessing preprocessing algorithms alphanumeric key method digit extraction fold shifting pseudorandom subtraction primary clustering. See clustering primary storage area schemes primary storage area sizing direct hashed structure LQHashed structure non-perfect hashed structures prime numbers four-k-plus-3 (4k + 3) generator algorithm generator method 652 table of use in hashed structures primitive data types. See also built-in arrays of priority queue private access probability of operations procedural abstraction program standard for goodness pseudo key pseudo loading factor pseudorandom preprocessing public access push operation. See also stack array-based; stack linked based Q quadratic complexity quadratic probing clustering multiple accesses short comings Queens Eight problem queue analogy applications of array based in breadth-first traversals circular dequeue empty enqueue expanded features FIFO front full link based operations and errors overflow performance performance comparison peek operation priority queue 653 rear representation of vs.stack queue array-based client code density dequeue algorithm encapsulation enqueue algorithm front garbage collection (recycling) implementation initialized state memory model numOfNodes overflow condition performance rear size speed underflow condition Queue class Quicksort algorithm implementation overhead partition pointers partitions performance performance comparison pivot value popularity of speed complexity sort effort sorting process swap effort R reachability matrix. See transitive closure matrix record recurrence relation recursion 654 base case binary search tree traversal definition domino analogy direct dynamic programming flow chart of formulating recursive algorithms general solution indirect iterative alternative levels of merge sort methodology n factorial performance considerations performance comparison vs.iteration practice problems problems with quicksort reduced problem run-time stack implications Towers of Hanoi recursion practice problems Fibonacci sequence reverse a string of length n table of problems table of solutions Towers of Hanoi x n recursive algorithms backtracking Fibonacci sequence findNode merge sort n factorial quicksort reverse a string of length n Towers of Hanoi tree traversal x n 655 red-black trees deletions extent of imbalance fetch algorithm ( see binary search tree insert) inserts maximum number of levels ordering conditions of nodes performance vs.AVL re-balancing condition rotations reduced problem for several problems reference variables null references re-heap down algorithm re-initialize operation relative speed case study methodology representing graphs. See graph representations requirements phase restricted structures application subset difference in the fetch-delete operation queue ( see queue) restrictions on operations restrictions on access stacks ( see stack) retrieve operation. See fetch operation return statement right child right subtree root of a heap of a tree rotations in AVL trees in red-black trees row-major order run-time stack 656 S searching arrays binary trees binary search graphs ( see sequential search;linked lists; traversing graphs) linked lists sequentially search length linear quotient collision algorithm at optimum loading factor secondary clustering. See clustering seed value self-balancing trees AVL trees red-black trees sequential search performance of setAddress method set prefix shallow copy in a data structure syntax water analogy shortest path. See Dijkstra's shortest path algorithm;Floyd's algorithm showAll method sibling node (children of same parent) vertex singly linked list algorithm discovery methodology client side defined delete algorithm delete traversal density dummy node fetch algorithm fetch traversal header implementation implementation level graphic 657 initialization insert algorithm l field linked Nodes next field Node (inner) class Node objects operation algorithms overhead performance performance comparison showAll method speed complexity stack standard graphic traversals update algorithm skewed tree problems with remedies software cost software engineering sorted array. See also sorted array-based structure binary search Sorted array-based structure binary search ( see also binary search) delete algorithm density encapsulation of errors fetch algorithm garbage collection initialization insert algorithm operation algorithms performance speed complexity update algorithm sorted linked list sorting. See also sorting algorithms comparisons minimum sort effort 658 minimum time to sort n items motivation for performance sort effort sorting objects swaps sorting algorithms Binary Tree ( see also binary tree sort) Bubble ( see also bubble sort) Heap ( see also heap sort) Merge ( see also merge sort) performance comparison Quicksort ( see also quicksort) sorting algorithm performance sorting the heap space complexity spanning tree. See also minimum spanning tree algorithm application of definition of determination of examples of implementation guidelines sparse matrix Stack class stack applications analogy array-based expanded features frame in depth-first traversals last-in-first-out LIFO operations and errors overflow peek operation performance performance comparison pop operation push operation 659 in post-fixed evaluation vs.queue use in recursion run-time top underflow stack array-based client code density encapsulation garbage collection generic implementation implementation initialized state memory model overflow condition performance pop algorithm push algorithm speed top size underflow condition Stack class generic implementation Java's API class stack link based as singly linked list using SinglyLinkedList class Standard abstract data Standard Template Library state of the machine static methods String class string preprocessing StringTokenizer class subarray subscripted variables subtraction preprocessing subtree super class. See parent super key word 660 swapping in fetch algorithms in sorting algorithms System class T telephone information request case study node description time complexity token top toSting method total bytes Towers of Hanoi trade-off process criteria ( see also density vs.node width) factors speed transitive closure matrix. See also Warshall's algorithm traversing graphs breadth-first depth-first use in determining connectivity traversing lists traversing trees. See binary tree traversals TreeMap class client side Comparable interface trees. See also binary trees;binary search trees advantages of array based representation AVL balanced ( see binary trees) binary tree binary tree traversals directed graphical depiction height of implementation of levels linked representation 661 operation algorithms red-black skewed terminology triangular series 2 i + 1, 2 i + 2 rule type place holders U unbalanced trees. See imbalanced trees underflow undirected graphs. See also graphs Unicode ASCII subset Unsorted array-based structure delete algorithm density encapsulation of errors expansion of fetch algorithm garbage collection initialization insert algorithm operation algorithms performance speed complexity update algorithm Unsorted-Optimized array-based structure client code delete algorithm delete optimization density encapsulation of errors expansion of fetch algorithm fetch optimization garbage collection implementation (baseline) implementation (with utilities) initialization 662 insert algorithm operation algorithms performance showAll method sorted on probability of being fetched speed complexity update algorithm utility methods updating nodes V vertex adjacent descendent insert operations on representing siblings visiting vertices ( see also traversing graphs) visiting a node ( see also traversing graphs) void (vs.non-void) methods W Warshall's algorithm algorithm client side code implementation speed complexity transitive closure matrix transitive property in mathematics weighted edges weighted graphs wrapper classes implementation of 663 664 Table of Contents Cover Page 664 Title Page 3 Copyright Page 4 Dedecation Page 6 Contents 7 Preface 13 Chapter 1 - Overview and Java Review 20 1.1 - Data Structures 21 1.1.1 - What is Data? 21 1.1.2 - What is a Data Structure? 23 1.2 - Selecting a Data Structure 25 1.2.1 - The Data Structure's Impact on Performance 25 1.2.2 - Determining the Performance of a Data Structure 27 1.2.3 - The Trade-Off Process 28 1.3 - Fundamental Concepts 31 1.3.1 - Terminology 31 1.3.2 - Access Modes 33 1.3.3 - Linear Lists 33 1.3.4 - Data Structure Operations 33 1.3.5 - Implementing a Programmer-Defined Data Structure 36 1.3.6 - Procedural Abstractions and Abstract Data Types (ADTs) 37 1.3.7 - Encapsulation 38 Chapter 2 - Array-Based Structures 84 2.1 - The Built-in Structure Array 85 2.1.1 - Multidimensional Arrays 88 2.2 - Programmer-Defined Array Structures 90 2.2.1 - Unsorted Array 92 2.2.2 - Sorted Array 98 2.2.3 - Unsorted-Optimized Array 103 2.2.4 - Error Checking 109 665 2.3 - Implementation of the Unsorted-Optimized Array Structure 112 2.3.1 - Baseline Implementation 113 2.3.2 - Utility Methods 125 Chapter 3 - Restricted Structures 152 3.1 - Restricted Structures 153 3.2 - Stack 155 3.2.1 - Stack Operations, Terminology, and Error Conditions 156 3.2.2 - Classical Model of a Stack 157 3.2.3 - A Stack Application: Evaluation of Arithmetic Expressions 171 3.2.4 - Expanded Model of a Stack 174 3.3 - Queue 176 3.3.1 - Queue Operations, Terminology, and Error Conditions 177 3.3.2 - Classical Model of a Queue 179 3.3.3 - Queue Applications 192 3.3.4 - Expanded Model of a Queue 193 3.4 - Generic Implementation of the Classic Stack, a Methodized Approach 195 3.4.1 - Generic Conversion Methodology 195 3.5 - Priority Queues 198 3.6 - Java's Stack Class 199 Knowledge Exercises 200 Programming Exercises 202 Chapter 4 - Linked Lists and Iterators 205 4.1 - Noncontiguous Structures 206 4.2 - Linked Lists 208 4.3 - Singly Linked Lists 209 4.3.1 - Basic Operation Algorithms 211 4.3.2 - Implementation 222 4.3.3 - Performance of the Singly Linked List 225 4.3.4 - A Stack Implemented as a Singly Linked List 228 4.4 - Other Types of Linked Lists 234 4.4.1 - Circular Singly Linked List 236 4.4.2 - Double-Ended Singly Linked List 236 666 4.4.3 - Sorted Singly Linked List 238 4.4.4 - Doubly Linked List 239 4.4.5 - Multilinked List 241 Chapter 5 - Hashed Data Structures 268 5.1 - Hashed Data Structures 269 5.2 - Hashing Access Algorithms 269 5.2.1 - A Hashing Example 272 5.3 - Perfect Hashed Data Structures 274 5.3.1 - Direct Hashed Structure 276 5.4 - Nonperfect Hashed Structures 284 5.4.1 - Primary Storage Area Size 287 5.4.2 - Preprocessing Algorithms 289 5.4.3 - Hashing Functions 297 5.4.4 - Collision Algorithms 300 5.4.5 - The Linear Quotient (LQHashed) Data Structure Implementation 301 5.4.6 - Dynamic Hashed Structures 315 Chapter 6 - Recursion 346 6.1 - What is Recursion? 347 6.2 - Understanding Recursive Algorithms 348 6.2.1 - n Factorial 348 6.2.2 - The Code of a Recursive Algorithm 350 6.2.3 - Tracing a Recursive Method's Execution Path 352 6.3 - Formulating a Recursive Algorithm 356 6.3.1 - Definitions 357 6.3.2 - Methodology 358 6.3.3 - Practice Problems 361 6.4 - Problems with Recursion 370 6.4.1 - Dynamic Programming Applied to Recursion 372 6.5 - Backtracking, an Application of Recursion 374 6.5.1 - A Generalized Backtracking Algorithm 379 6.5.2 - Algorithm Adaptation Methodology 381 Knowledge Exercises 393 Programming Exercises 395 667 Chapter 7 - Trees 398 7.1 - Trees 399 7.1.1 - Graphics and Terminology of Trees 399 7.2 - Binary Trees 403 7.2.1 - Terminology 403 7.2.2 - Mathematics 405 7.3 - Binary Search Trees 408 7.3.1 - Basic Operation Algorithms 413 7.3.2 - Performance 430 7.3.3 - Implementation 443 7.3.4 - Standard Tree Traversals 449 7.3.5 - Balanced Search Trees 459 7.3.6 - Array Implementation of a Binary Search Tree 467 7.3.7 - Performance 473 7.3.8 - Java's TreeMap Data Structure 479 Knowledge Exercises 481 Programming Exercises 483 Chapter 8 - Sorting 486 8.1 - Sorting 486 8.2 - Sorting Algorithm Speed 489 8.2.1 - Minimum Sort Effort 490 8.2.2 - An Implementation Issue Affecting Algorithm Speed 491 8.3 - Sorting Algorithms 492 8.3.1 - The Binary Tree Sort 493 8.3.2 - The Bubble Sort 499 8.3.3 - The Heap Sort 504 8.3.4 - The Merge Sort 515 8.3.5 - Quicksort 522 Chapter 9 - Graphs 535 9.1 - Introduction 536 9.1.1 - Graphics and Terminology of Graphs 539 9.2 - Representing Graphs 543 9.2.1 - Representing Vertices 543 9.2.2 - Representing Edges 544 668 9.3 - Operations Performed on Graphs 549 9.4 - Implementing Graphs in the Vertex Number Mode 552 9.5 - Traversing Graphs 553 9.5.1 - Depth-First Traversal 556 9.5.2 - Breadth-First Traversal 563 Appendices 600 Appendix A - ASCII Table 600 Appendix B - Derivation of the Average Search Length of a Nondirect Hashed Data Structure 603 Appendix C - Proof That If an Integer, P, is not Evenly Divisible by an Integer Less Than the Square Root of P, It is a Prime Number 607 Appendix D - Calculations to Show That (n + 1) (log2(n + 1) − 2) is the Minimum Sort Effort for the Binary Tree Sort 608 Glossary 610 Index 626 1.4 - Calculating Speed (Time Complexity) 40 1.4.1 - Big-O Analysis(O Standing for Order of Magnitude) 41 1.4.2 - Algorithm Speed 44 1.4.3 - Relative Speed of Algorithms 44 1.4.4 - Absolute Speed of an Algorithm 48 1.4.5 - Data Structure Speed 52 1.5 - Calculating Memory Overhead (Space Complexity) 54 1.6 - Java Review 56 1.6.1 - Arrays of Primitive Variables 56 1.6.2 - Definition of a Class 58 1.6.3 - Declaration of an Object 60 1.6.4 - Accessing Objects 62 1.6.5 - Standard Method Name Prefixes 63 1.6.6 - Shallow and Deep Copies 64 1.6.7 - Declaration of an Array of Objects 67 1.6.8 - Objects that Contain Objects as Data Members 69 1.6.9 - Classes that Extend Classes, Inheritance 70 1.6.10 - Parent and Child References 72 669 1.6.11 - Generic Types 73 Knowledge Exercises 76 Programming Exercises 79 2.4 - Expandable Array-Based Structures 129 2.5 - Generic Data Structures 131 2.5.1 - Design Considerations 132 2.5.2 - Generic Implementation of the Unsorted-Optimized Array 134 2.5.3 - Client-Side Use of Generic Structures 142 2.5.4 - Heterogeneous Generic Data Structures 144 2.6 - Java's ArrayList Class 145 Knowledge Exercises 146 Programming Exercises 148 4.5 - Iterators 245 4.5.1 - Implementation of an Iterator 247 4.5.2 - Multiple Iterators 253 4.6 - Java's LinkedList Class and ListIterator Interface 258 Knowledge Exercises 261 Programming Exercises 264 Knowledge Exercises 339 Programming Exercises 343 Knowledge Exercises 529 Programming Exercises 532 9.6 - Connectivity and Paths 563 9.6.1 - Connectivity of Undirected Graphs 566 9.6.2 - Connectivity of Directed Graphs 567 9.6.3 - Spanning Trees 570 9.6.4 - Shortest Paths 580 Knowledge Exercises 593 Programming Exercises 597 670 Cover Page Title Page Copyright Page Dedecation Page Contents Preface Chapter 1 - Overview and Java Review 1.1 - Data Structures 1.1.1 - What is Data? 1.1.2 - What is a Data Structure? 1.2 - Selecting a Data Structure 1.2.1 - The Data Structure's Impact on Performance 1.2.2 - Determining the Performance of a Data Structure 1.2.3 - The Trade-Off Process 1.3 - Fundamental Concepts 1.3.1 - Terminology 1.3.2 - Access Modes 1.3.3 - Linear Lists 1.3.4 - Data Structure Operations 1.3.5 - Implementing a Programmer-Defined Data Structure 1.3.6 - Procedural Abstractions and Abstract Data Types (ADTs) 1.3.7 - Encapsulation Chapter 2 - Array-Based Structures 2.1 - The Built-in Structure Array 2.1.1 - Multidimensional Arrays 2.2 - Programmer-Defined Array Structures 2.2.1 - Unsorted Array 2.2.2 - Sorted Array 2.2.3 - Unsorted-Optimized Array 2.2.4 - Error Checking 2.3 - Implementation of the Unsorted-Optimized Array Structure 2.3.1 - Baseline Implementation 2.3.2 - Utility Methods Chapter 3 - Restricted Structures 3.1 - Restricted Structures 3.2 - Stack 3.2.1 - Stack Operations, Terminology, and Error Conditions 3.2.2 - Classical Model of a Stack 3.2.3 - A Stack Application: Evaluation of Arithmetic Expressions 3.2.4 - Expanded Model of a Stack 3.3 - Queue 3.3.1 - Queue Operations, Terminology, and Error Conditions 3.3.2 - Classical Model of a Queue 3.3.3 - Queue Applications 3.3.4 - Expanded Model of a Queue 3.4 - Generic Implementation of the Classic Stack, a Methodized Approach 3.4.1 - Generic Conversion Methodology 3.5 - Priority Queues 3.6 - Java's Stack Class Knowledge Exercises Programming Exercises Chapter 4 - Linked Lists and Iterators 4.1 - Noncontiguous Structures 4.2 - Linked Lists 4.3 - Singly Linked Lists 4.3.1 - Basic Operation Algorithms 4.3.2 - Implementation 4.3.3 - Performance of the Singly Linked List 4.3.4 - A Stack Implemented as a Singly Linked List 4.4 - Other Types of Linked Lists 4.4.1 - Circular Singly Linked List 4.4.2 - Double-Ended Singly Linked List 4.4.3 - Sorted Singly Linked List 4.4.4 - Doubly Linked List 4.4.5 - Multilinked List Chapter 5 - Hashed Data Structures 5.1 - Hashed Data Structures 5.2 - Hashing Access Algorithms 5.2.1 - A Hashing Example 5.3 - Perfect Hashed Data Structures 5.3.1 - Direct Hashed Structure 5.4 - Nonperfect Hashed Structures 5.4.1 - Primary Storage Area Size 5.4.2 - Preprocessing Algorithms 5.4.3 - Hashing Functions 5.4.4 - Collision Algorithms 5.4.5 - The Linear Quotient (LQHashed) Data Structure Implementation 5.4.6 - Dynamic Hashed Structures Chapter 6 - Recursion 6.1 - What is Recursion? 6.2 - Understanding Recursive Algorithms 6.2.1 - n Factorial 6.2.2 - The Code of a Recursive Algorithm 6.2.3 - Tracing a Recursive Method's Execution Path 6.3 - Formulating a Recursive Algorithm 6.3.1 - Definitions 6.3.2 - Methodology 6.3.3 - Practice Problems 6.4 - Problems with Recursion 6.4.1 - Dynamic Programming Applied to Recursion 6.5 - Backtracking, an Application of Recursion 6.5.1 - A Generalized Backtracking Algorithm 6.5.2 - Algorithm Adaptation Methodology Knowledge Exercises Programming Exercises Chapter 7 - Trees 7.1 - Trees 7.1.1 - Graphics and Terminology of Trees 7.2 - Binary Trees 7.2.1 - Terminology 7.2.2 - Mathematics 7.3 - Binary Search Trees 7.3.1 - Basic Operation Algorithms 7.3.2 - Performance 7.3.3 - Implementation 7.3.4 - Standard Tree Traversals 7.3.5 - Balanced Search Trees 7.3.6 - Array Implementation of a Binary Search Tree 7.3.7 - Performance 7.3.8 - Java's TreeMap Data Structure Knowledge Exercises Programming Exercises Chapter 8 - Sorting 8.1 - Sorting 8.2 - Sorting Algorithm Speed 8.2.1 - Minimum Sort Effort 8.2.2 - An Implementation Issue Affecting Algorithm Speed 8.3 - Sorting Algorithms 8.3.1 - The Binary Tree Sort 8.3.2 - The Bubble Sort 8.3.3 - The Heap Sort 8.3.4 - The Merge Sort 8.3.5 - Quicksort Chapter 9 - Graphs 9.1 - Introduction 9.1.1 - Graphics and Terminology of Graphs 9.2 - Representing Graphs 9.2.1 - Representing Vertices 9.2.2 - Representing Edges 9.3 - Operations Performed on Graphs 9.4 - Implementing Graphs in the Vertex Number Mode 9.5 - Traversing Graphs 9.5.1 - Depth-First Traversal 9.5.2 - Breadth-First Traversal Appendices Appendix A - ASCII Table Appendix B - Derivation of the Average Search Length of a Nondirect Hashed Data Structure Appendix C - Proof That If an Integer, P, is not Evenly Divisible by an Integer Less Than the Square Root of P, It is a Prime Number Appendix D - Calculations to Show That (n + 1) (log2(n + 1) − 2) is the Minimum Sort Effort for the Binary Tree Sort Glossary Index 1.4 - Calculating Speed (Time Complexity) 1.4.1 - Big-O Analysis(O Standing for Order of Magnitude) 1.4.2 - Algorithm Speed 1.4.3 - Relative Speed of Algorithms 1.4.4 - Absolute Speed of an Algorithm 1.4.5 - Data Structure Speed 1.5 - Calculating Memory Overhead (Space Complexity) 1.6 - Java Review 1.6.1 - Arrays of Primitive Variables 1.6.2 - Definition of a Class 1.6.3 - Declaration of an Object 1.6.4 - Accessing Objects 1.6.5 - Standard Method Name Prefixes 1.6.6 - Shallow and Deep Copies 1.6.7 - Declaration of an Array of Objects 1.6.8 - Objects that Contain Objects as Data Members 1.6.9 - Classes that Extend Classes, Inheritance 1.6.10 - Parent and Child References 1.6.11 - Generic Types Knowledge Exercises Programming Exercises 2.4 - Expandable Array-Based Structures 2.5 - Generic Data Structures 2.5.1 - Design Considerations 2.5.2 - Generic Implementation of the Unsorted-Optimized Array 2.5.3 - Client-Side Use of Generic Structures 2.5.4 - Heterogeneous Generic Data Structures 2.6 - Java's ArrayList Class Knowledge Exercises Programming Exercises 4.5 - Iterators 4.5.1 - Implementation of an Iterator 4.5.2 - Multiple Iterators 4.6 - Java's LinkedList Class and ListIterator Interface Knowledge Exercises Programming Exercises Knowledge Exercises Programming Exercises Knowledge Exercises Programming Exercises 9.6 - Connectivity and Paths 9.6.1 - Connectivity of Undirected Graphs 9.6.2 - Connectivity of Directed Graphs 9.6.3 - Spanning Trees 9.6.4 - Shortest Paths Knowledge Exercises Programming Exercises