ACQ 203 Part B

Attached doc Study Info contains the questions

Course	ACQ203V_20313-Virtual Instructor-Led Training; ACQ 203V Intermediate Systems Acquisition, Part B

Question 1

0 out of 1 points

For questions 1 through 5 (worth 1 point each), use the data (In $Millions) below from the latest Integrated Program Management Report (IPMR) from your contractor: BCWS BCWP ACWP BAC EAC 7 6 6 10 10 Calculate the Cost Variance (CV) and the Schedule Variance (SV) then pick the true statement from the answers below:

Answers: a.   The contractor is on budget and behind schedule b.  The contractor is over budget and ahead of schedule. c.  The contractor is under budget and ahead of schedule. d.  The contractor is over budget and behind schedule.

·

Question 2

0 out of 1 points

Use the IPMR data (In $Millions) to answer the question below. BCWS BCWP ACWP BAC EAC 7 6 6 10 10 Calculate the Schedule Performance Index (SPI). Based on the SPI:

Answers: a.  The contractor seems efficient in terms of schedule. b.  The contractor has finished 85% of the work. c.  The contractor will meet their Estimate at Completion (EAC) of 10. d.  The contractor seems inefficient in terms of schedule.

· Question 3

1 out of 1 points

Use the data (In $Millions) below from the latest Integrated Program Management Report (IPMR) from your contractor: BCWS BCWP ACWP BAC EAC 7 6 6 10 10 Calculate the Cost Performance Index (CPI). Based on the CPI:

Answers: a.  The contractor seems inefficient in terms of cost. b.  The contractor is on track to meet the Budget at Completion (BAC). c.  The contractor has spent 85% of the budget. d.  The contractor will finish on time.

·

Question 4

0 out of 1 points

Use the data (In $Millions) below from the latest Integrated Program Management Report (IPMR) from your contractor: BCWS BCWP ACWP BAC EAC 7 6 6 10 10 Pick the true statement from the answers below:

Answers: a.  Based on the Percent Spent and the Percent Complete, the contractor is on track to complete the work within budget. b.  Based on the Percent Spent and the Percent Complete, the contractor will run out of money before the work is completed. c.  Based on the Percent Scheduled and Percent Spent, the contractor will complete the work on time. d.  Based on the Percent Scheduled and Percent Complete, the contractor will complete the work on time.

·
·

· Question 5

0 out of 1 points

Use the data (In $Millions) below from the latest Integrated Program Management Report (IPMR) from your contractor: BCWS BCWP ACWP BAC EAC 7 6 6 10 10 Based on the IPMR data on this component, what should we tell the Program Manager?

Answers: a.  The contractor is right on budget, but there are tasks that are currently behind schedule. b.  The contractor will complete this project well ahead of schedule. c.  The contractor will complete this project under budget. d.  The contractor will complete this project ahead of schedule and will only be 5% over budget.

· Question 6

0 out of 3 points

The program office for a Navy shipboard communications system early in the TMRR phase just conducted an affordability analysis of their key performance parameters.  The program can save 10% on overall life cycle cost by trading 5% of point to point surface communications range KPP for greater component reliability.  The Navy CDD sponsor agreed to include the requirements change as part of staffing for the CDD-V.  For questions 6 through 9 (worth 3 points each), choose the greatest potential impact of the changed requirements on the  given functional area  (i.e., which is the greatest impact on Logistics, on T&E, on Funds Management etc.): The greatest impact on Logistics will be:

Answers: a.  Changes to manufacturing methods b.  Modification of the contract c.  Additional testing to verify that new requirements are met without impacting other requirements d.  Changes to maintenance and supply planning based on the new component

· Question 7

3 out of 3 points

The program office for a Navy shipboard communications system early in the TMRR phase just conducted an affordability analysis of their key performance parameters.  The program can save 10% on overall life cycle cost by trading 5% of point to point surface communications range KPP for greater component reliability.  The Navy CDD sponsor agreed to include the requirements change as part of staffing for the CDD-V.  For questions 6 through 9 (worth 3 points each), choose the greatest potential impact of the changed requirements on the  given functional area  (i.e., which is the greatest impact on Logistics, on T&E, on Funds Management etc.): The greatest impact on Test and Evaluation will be:

Answers: a.  Changes to DT and OT test objectives for range and reliability b.  Changes to spares and support equipment c.  Generation of an unfunded requirement d.  Changes to the system configuration

· Question 8

3 out of 3 points

The program office for a Navy shipboard communications system early in the TMRR phase just conducted an affordability analysis of their key performance parameters.  The program can save 10% on overall life cycle cost by trading 5% of point to point surface communications range KPP for greater component reliability.  The Navy CDD sponsor agreed to include the requirements change as part of staffing for the CDD-V.  For questions 6 through 9 (worth 3 points each), choose the greatest potential impact of the changed requirements on the  given functional area  (i.e., which is the greatest impact on Logistics, on T&E, on Funds Management etc.): The greatest impact on Systems Engineering will be:

Answers: a.  Shortage of procurement funds to cover communications testing b.  Changes to the system performance specifications c.  Changes to repair procedures d.  Changes to the contract incentives for EMD

· Question 9

0 out of 3 points

The program office for a Navy shipboard communications system early in the TMRR phase just conducted an affordability analysis of their key performance parameters.  The program can save 10% on overall life cycle cost by trading 5% of point to point surface communications range KPP for greater component reliability.  The Navy CDD sponsor agreed to include the requirements change as part of staffing for the CDD-V.  For questions 6 through 9 (worth 3 points each), choose the greatest potential impact of the changed requirements on the  given functional area  (i.e., which is the greatest impact on Logistics, on T&E, on Funds Management etc.): The greatest impact on Contracting will be:

Answers: a.  Changes to the statement of objectives for the request for proposal for the EMD contract b.  Significant changes to the test schedule for IOT&E c.  Additional O&M funding for the TMRR contract effort d.  Modification to preventive maintenance procedures

·

· Question 10

3 out of 3 points

If any task on a project’s critical path increases, this means…?

Answers: a.  The Program Manager must re-baseline the whole program. b.  The total project will also increase in length. c.  There has been a breach of the acquisition program baseline. d.  Hardware risk has been mitigated.

·

· Question 11

0 out of 3 points

A recent ACQ 203 graduate touring a contractor facility stops to speak with a worker on the manufacturing floor. “Any chance you could make that widget a little bigger?” She asks the worker. “Sure, no problem,” says the worker and he makes the widget bigger. The contracting officer receives an invoice for the widget a few weeks later that is significantly higher than she expected. This is an example of a/an…?

Answers: a.  Supplemental Agreement b.  Reprogramming Action c.  Change Order d.  Unauthorized Commitment

·

· Question 12

3 out of 3 points

Cybersecurity planning activities, including writing the Cybersecurity Strategy annex to the Program Protection Plan, should begin____________________.

Answers: a.  With the implementation of performance based support b.  After the Capability Development Document Validation c.  During the Material Solution Analysis Phase prior to Milestone A d.  In full rate production

·

· Question 13

0 out of 3 points

Consider the program structure chart when answering the questions 13 – 15 Which of the following changes should be made to the phases and decisions depicted on the program schedule?

Answers: a.  Move the MDD to the end of the Materiel Solution Analysis phase. b.  The FRP-DR should be after IOT&E at the end of LRIP. c.  The TMRR phase should end at the DRFPRD. d.  IOC should coincide with Milestone C.

·

· Question 14

0 out of 3 points

Which of these program events is scheduled too early in the lifecycle?

Answers: a.  TMRR Prototype Deliveries b.  IOT&E c.  PDR d.  CDR

·

· Question 15

0 out of 3 points

Which one of the following changes should be made to program funding?

Answers: a.  Fund the EMD phase with procurement dollars. b.  Eliminate RDT&E funding in FY-4. c.  Start O&M funding in FY-2 to fund testing. d.  Add procurement dollars in FY-4 to fund production.

(March 2020) Version 2.0
Intermediate Systems
Acquisition Course
ACQ 203

DAU strives to make you, the defense acquisition
workforce, better at what you do. Your work is important to
the nation and your learning is important to us. We
constantly work to make the course more effective for you.
Please let your instructor know how we can improve this
course or feel free to send me an email.
Thanks,
Matt Ambrose
ACQ 203 Course Manager
matt.ambrose@dau.mil
DAU Learning Resources Available at www.dau.mil
Interactive DAU Catalog http://icatalog.dau.mil/
Continuous Learning Center – http://www.dau.mil/clc/
Defense Acquisition Portal – https://dap.dau.mil/
Defense Acquisition Guidebook – https://dag.dau.mil/
Milestone Document Identification Tool – https://dap.dau.mil/mdid/

http://www.dau.mil/

http://icatalog.dau.mil/

http://www.dau.mil/clc/

https://dap.dau.mil/

https://dag.dau.mil/

https://dap.dau.mil/mdid/

Student Assessment
In the computer-based portion of the
course (ACQ 202), you learned about
the business, technical, and
management processes involved in
defense systems acquisition. In ACQ
203, you will work in an integrated
product team environment to apply
what you learned in the computer-
based course to solve a variety of
problems. Your performance in the
classroom portion of the course will
be evaluated on a pass/fail basis. You
must achieve at least 80% mastery of
the ACQ 203 learning objectives in
order to pass the entire course.
Should you not achieve the required
80% overall, you will be required to
repeat ACQ 203. ACQ 203 classroom
performance assessment is based on
these factors.
1. Assessments (70 Points)
Content and Analysis
Questions. On the second and
fourth day of class, you will
answer some multiple choice
questions based on the
material covered in ACQ 203.
All assessments are individual
efforts. You are encouraged to
refer to your notes, lesson
summaries, and other written
references. Each assessment
contains 15 questions and is
worth 35 points
2. Participation (30 Points)
Class participation will be
assessed through instructor
observation of teamwork,
leadership and discussions.
You are expected to be in class
on time, actively participate in
group and class discussions, and
rotate leadership responsibility
among the members of your team
(30 points). Behavior that could
cause a student to lose
participation points includes but is
not limited to: tardiness, lack of
attention, texting, sidebar
conversations and disruption of
class or team exercises and
discussions.
3. Briefing
Each student is required to give a
5-10 minute briefing for their
team. The briefing is a
requirement for graduation but is
not graded for points. This is an
opportunity to develop and
practice your briefing skills in a
low threat environment.
4. Attendance
Attendance all class days is
mandatory for graduation. Under
special circumstances, such as a
medical emergency, you may be
excused from the course for up to
two (2) hours with the
instructor’s permission. (Early
flights on Friday are not
considered a valid reason to miss
class.)
5

6

Exercise 1.1
IPT Leadership & Barriers – 9
Exercise 1.2
Ethics & Acquisition – 13
Exercise 1.3
Acquisition Strategy – 21
Firebird II DRAFT CDD – 43
Exercise 1.4
Materiel Solution Analysis – 51
Exercise 2.1
Source Selection Planning – 61
Exercise 2.2
Systems Engineering – 77
Exercise 2.3
Test Planning – 89
Exercise 2.4
Technical Performance Measures – 97
Exercise 2.5
Contractor Planning, Scheduling and
Resourcing – 101
Exercise 3.1
Source Selection Process – 109
Exercise 3.2
Contractor Performance Analysis – 115
Exercise 3.3
Software & Interoperability – 125
Exercise 3.4
Reliability Issue – 133
Exercise 3.5
Contract Change – 139
Exercise 4.1
Supportability Issue – 149
CBT Summaries – 157
(See the Index on the Next Page)
TABLE
OF
CONTENTS
7

ACQ 202 Summaries Index
Lesson Title Page
1.1 Considering the Costs 158
1.2 Selecting the Best Approach 162
2.1 Integrated Product and Process 164
Development (IPPD)
2.2 Developing the Acquisition 167
Approach
2.3 Developing the Life Cycle 172
Sustainment Plan
2.4 Risk Management 176
2.5 Developing the TEMP 181
2.6 ESOH Issues 186
2.7 Programming Funds 188
2.8 RFP Preparations (Part I) 192
2.9 RFP Preparations (Part II) 196
3.1 Source Selection 201
3.2 Technical Risk Management 207
3.3 Design for Supportability/ 210
Trade-Off Analysis
3.4 Software Design 214
3.5 Commercial & NDI 216
3.6 Role of Manufacturing 219
Lesson Title Page
3.7 Earned Value Management 221
3.8 Budgeting Process 225
4.1 Design Changes 227
4.2 Software Problems 231
4.3 APB Breaches 235
4.4 Reprogramming Funds 236
4.5 Reviews, Simulations & Tests 238
4.6 Contractor Performance 241
Measurement
4.7 Integrated Baseline Review 246
4.8 Budget Execution 248
4.9 Operational & Live Fire Tests 250
5.1 Best Manufacturing Practices 252
5.2 Unauthorized Commitments 254
5.3 Production and Follow-On 256
Support
5.4 Contract Modification 258
6.1 Contract Dispute 261
6.2 Life Cycle Product Support 263
6.3 Leadership and Ethics 266
8

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 1.1
______________________________________________________
Lesson Title Integrated Product Team (IPT) Leadership & Barriers
______________________________________________________
Lesson Time 1 hour
______________________________________________________
Terminal and Enabling Learning Objectives
TLO Determine how IPT leadership concepts can be used to overcome barriers to effective teamwork, based on real world experience.
ELO Relate key tenets of IPPD to planning and executing an acquisition program.
ELO Identify the aids and barriers to successful IPT implementation.
ELO Identify the Supervisory, Participative and Team leadership styles.
ELO Describe how different leadership styles impact the effectiveness of an IPT.
ELO Identify the behaviors and characteristics of effective teams.
______________________________________________________
Assignments Review the following ACQ 202 CBT Lesson Summaries:
• Lesson 2.1, Integrated Product and Process Development
• Lesson 6.3, Leadership and Ethics
______________________________________________________
Estimated Student
Preparation Time N/A
______________________________________________________
Assessment Class participation; oral presentation
______________________________________________________
Related Lessons CBT Lesson 2.1, Integrated Product and Process Development
CBT Lesson 6.3, Leadership and Ethics
______________________________________________________
9

Intermediate Systems Acquisition Course March 2020
Self Study
References
• DoD Guide to Integrated Product and Process Development,
(Version 1.0), February 5, 1996.
• Rules of the Road: A Guide for Leading Successful Integrated
Product Teams, Oct 1999. Available at
http://www.defenselink.mil/nii/org/cio/pa/rulesoct1999
• DoD Integrated Product and Process Development Handbook,
August 1998.
Insert Slides
10

Supervisory Participative Team
From Leading Teams, Mastering the New Role, by Zenger, Musselwhite, Hurson and Perrin
Leadership Styles
Direct people Involve people Build trust and inspire teamwork
Explain decisions Get input for decisions Facilitate and support team decisions
Train individuals Develop individual performance Expand team capabilities
Manage one-on-one Coordinate group effort Create a team identity
Contain conflict Resolve conflict Make the most of team differences
React to change Implement change Foresee and influence change
The Standards
Clarity
Accuracy
Relevance
Logic
Breadth
Precision
Significance
Completeness
Fairness
Depth
The Elements
Purpose
Questions
Points of View
Information
Inferences
Concepts
Implications
Assumptions
The Intellectual Traits
Intellectual Humility
Intellectual Autonomy
Intellectual Integrity
Intellectual Courage
Intellectual Perseverance
Confidence in Reason
Intellectual Empathy
Fairmindedness
Must be
applied
to
As we
learn to
develop
Dr. Paul and Dr.
Elder’s model
shows critical
thinkers routinely
apply standards to
the elements of
reasoning to
develop
intellectual traits
Critical Thinking—
The art of analyzing
and evaluating
thinking with a view
to improving it.—
Paul and Elder
Effective Team Leaders
Enable & Encourage Critical Thinking
The IPPD Process Provides Good Conditions for 
Critical Thinking ‐ ACQ 203 Provides Multiple 
Opportunities to Apply Critical Thinking
11

A general agreement by all team members that they
can live with and be committed to a particular course
of action.
Consensus
When the output of a team is greater than the sum of
the contributions of its individual members.
Synergy
12

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 1.2
______________________________________________________
Lesson Title Ethics and Acquisition
______________________________________________________
Lesson Time 1 hour
_____________________________________________________
Terminal and Enabling Learning Objectives
TLO Resolve an acquisition-related dilemma by prioritizing ethical values and considering how choices impact the welfare of others.
ELO Identify the characteristics of a “successful” defense acquisition program from a variety of perspectives.
ELO Identify core ethical values critical to decision making in the acquisition environment.
ELO Identify the steps of the Principled Decision Making Model
ELO Resolve an ethical dilemma by applying the steps of the Principled Decision Making Model.
______________________________________________________
Assignments Review the following ACQ 202 CBT Lesson Summary:
• Lesson 6.3, Leadership and Ethics
______________________________________________________
Estimated Student
Preparation Time N/A
______________________________________________________
Assessment Class participation
______________________________________________________
Related Lessons CBT Lesson 6.3. Leadership and Ethics
______________________________________________________
Self Study
References N/A
______________________________________________
13

Intermediate Systems Acquisition Course March 2020
14

Intermediate Systems Acquisition Course March 2020

Exercise 1.2 Successful Acquisition Program

What is a successful defense acquisition program?

It depends upon your point of view:

_______________
A successful program delivers a system that meets the
user’s technical performance requirements on time and
within budget.

_______________

A successful program is profitable; it provides a positive
cash flow and return on investment.

_______________
A successful program provides capability in a system that
is available, effective, and easy to operate in wartime and
peacetime.

_______________
A successful program balances social, environmental and
defense needs. It provides a fair distribution of defense
dollars by state.

Whose perspectives are indicated above? Fill in the blanks.

15

• Consider the welfare of all stakeholders.
• Give precedence to
ethical values over
non-ethical values.
• Prioritize based on
what will bring the
most good and
least harm to
others.
Principled Decision Making Model
• Trustworthiness
• Respect
• Responsibility
• Justice/Fairness
• Caring
• Civic Virtue/Citizenship
Ethical Values
16

• Profit Motive
• Career Progression
• Power
• Position
Non-Ethical Values
17

Intermediate Systems Acquisition Course March 2020

18

Intermediate Systems Acquisition Course March 2020

Case 1.2, An Ethics Dilemma

Read the following case and discuss the three questions with your team:

Brigadier General Burt Goodguy is the Program Executive Officer (PEO) for five military
programs. Tomorrow he is to testify before the House Armed Services Committee (HASC)
regarding a very sophisticated and expensive weapons system considered a very high priority by
his service secretary. The prime contractor is Mogul Systems, located in the district of Rep.
Allen, chairman of the HASC.

The system is in trouble because Congress is desperately looking to make large cuts in the
defense budget, and the program is almost one year behind schedule. In addition, several
significant technical problems were uncovered in the most recent tests. Several members of
Congress have publicly advocated canceling the system before it goes into full production.
Mogul insists it has solved the problems and is confident that the system will pass its next test
with flying colors. Mogul asserts it can go into full production within nine months. The
Secretary of Defense has thus far been strongly supportive of the system in his public statements,
but some think he is privately wavering for political reasons.

Col. Wantit, Program Manager for the system, briefs BGen. Goodguy and tells him that he is not
sure that Mogul has solved the problems yet. BGen. Goodguy grimaces at this news and says
sarcastically, “Can’t you bring me good news? You aren’t helping the cause, you know.”

Col. Wantit recently heard disturbing rumors, which he has not yet tried to verify, that the chief
scientist on the program is seriously ill (possibly with cancer) and that several top engineers are
about to quit. If either of the rumors is true, the likelihood that Mogul will solve its problems
before the next test is much less likely. However, he still believes the problems are temporary.
Since the information is shaky and so potentially volatile, Col. Wantit decides not to tell BGen.
Goodguy about the rumors for fear that he might have to mention it to Congress, and some
politicians and the press would blow the program.

1. Who are the stakeholders in his decision?

2. What ethical values are involved in Col. Wantit’s decision to withhold his information
about the rumors?

3. What would you have done in his place?

____________________________________________
©1997 Josephson Institute of Ethics – Reprinted with permission
19

Intermediate Systems Acquisition Course March 2020
Case 1.2, An Ethics Dilemma (continued)
An hour after briefing BGen. Goodguy, Col. Wantit receives a call from Barbara Leake, a top
manager at Mogul who has known Col. Wantit for 10 years.
Leake: George, it’s Barbara Leake. How are things going for you?
Wantit: Things are pretty hectic around here, as usual. How about you?
Leake: “Well, this isn’t for publication, but I wanted you to know I’m going to be leaving
Mogul. If you know of any appropriate openings, let me know.”
Wantit: “I’ve got to know more. Is the program in any way endangered? Are there
problems I should know about?”
Leake: “Probably, but you simply can’t use this yet: it will be traced to me. Even if you
sniff around they will suspect me, and it would kill any chance I have to land another job.
I’ve already told you too much, and it really isn’t a big thing. Really. I’ll tell you the whole
story if you hold it confidential for a week or so.”
Wantit: “I can’t promise that. But I need to know, and you need to tell me. I’ll protect you
as a source as best I can.”
Leake: “I’m sorry, I just can’t risk it, but you’ll know whatever you need to know in a few
days, I imagine. It’s just not that serious. Look, I’ve got to go to a meeting now;
goodbye…”
1. Did Col. Wantit handle this properly?
2. Who are the major stakeholders?
3. What ethical values are involved?
4. What would you have done in Col. Wantit’s position?
5. What, if anything, should Col. Wantit tell BGen. Goodguy?
6. Should BGen. Goodguy want to know about this and similar information? Would you?
7. If BGen. Goodguy wanted his people to tell them everything that might be relevant to a
program, what could he do to increase the likelihood?
____________________________________________
©1997 Josephson Institute of Ethics – Reprinted with permission
20

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 1.3
______________________________________________________
Lesson Title Acquisition Strategy Development
______________________________________________________
Lesson Time 8.5 hours
______________________________________________________
Terminal and Enabling Learning Objectives
TLO
Prepare an acquisition strategy program structure chart showing
appropriate interrelationship(s) of the various business and technical
functions involved in planning and executing the program:
ELO
Given an acquisition program scenario with information on technology
maturity, funding and JCIDS documentation, identify the correct starting
point for the program in the acquisition lifecycle
ELO Identify the correct type of appropriated funds needed by phase and work effort
ELO Given an acquisition program structure chart identify the correct sequence and timing of technical reviews by phase and work effort
ELO
Given an acquisition program structure chart identify the correct sequence
and timing of developmental and operational test events by phase and
work effort
ELO
Given an acquisition program structure chart identify the correct sequence
and timing of lifecycle logistics planning and execution efforts by phase
and work effort
ELO Given an acquisition program structure chart, identify the appropriate contract types by phase and work effort
ELO Given an acquisition program structure chart, identify the timing of major hardware deliverables by phase and work effort
ELO Relate the capability documents (ICD,CDD) to the correct phases of the acquisition system
ELO Identify the evolutionary acquisition strategy approach
ELO Identify the single step acquisition strategy approach
TLO Modify, present, and defend an acquisition strategy to accommodate a change in program funding levels
ELO Identify the proper response to a program funding cut
ELO Given a program funding cut, identify the potential impacts on industry.
______________________________________________________
21

Intermediate Systems Acquisition Course March 2020
Assignments  Scan Pages 2-17 of the DoDI 5000.02T
 Read Pages 18-32 DoDI 5000.02T
 Review the following ACQ 202 CBT Lesson Summary:
– Lesson 2.2, Developing the Acquisition Strategy
______________________________________________________
Estimated Student
Preparation Time 60 minutes
______________________________________________________
Assessment Class participation; oral presentation
______________________________________________________
Related Lessons CBT Lesson 2.2, Developing the Acquisition Strategy
Classroom Exercise 1.4, Materiel Solution Analysis
______________________________________________________
Self Study
References
 DoDD 5000.01, The Defense Acquisition System, 12 May 2003
 DoDI 5000.02T, Operation of the Defense Acquisition System,
Defense Acquisition Guidebook, Chaps 2 & 4
______________________________________________
22

DoDD 5000.01: The Defense Acquisition System
DoDI 5000.02:  Operation of the Adaptive Acquisition Framework
1. Simplify Acquisition Policy
2. Tailor Acquisition Approaches
3. Empower Program Managers
4. Data Driven Analysis
5. Active Risk Management
6. Emphasize Sustainment
< 5 years < 1 year Cy be rs ec ur ity   Path Selection Defense  Business  Systems Middle Tier of  Acquisition DoDI 5000.80 Acquisition  of Services Major  Capability Acquisition Urgent Capability Acquisition DoDI 5000.81 OP ER AT IO NS  A ND   SU ST AI NM EN T Business Capability Acquisition Cycle < 2 years Software Acquisition (In Process) Rapid Prototyping Capability Need Identification Solution Analysis Functional Requirements and  Acquisition Planning Acquisition Testing and  Deployment Capability Support Pl an ni ng Ph as e I1 I2… MVP MVCR Rn O D Rapid  Fielding 10 Material Solution Analysis Technology Maturation and  Risk Reduction Engineering and  Manufacturing Development Production and Deployment MDD MS A MS B MS C IOC FOC ATP ATP ATP ATP In In In Execution Phase O D < 5 years D D Legend: ATP: Authority to Proceed DD: Disposition Decision FOC: Full Operational Capability I: Iteration IOC: Initial Operational Capability MDD: Material Development Decision MS: Milestone MVP: Minimum Viable Product MVCR: Minimum Viable Capability Release OD: Outcome Determination R: Release 1 Form the Team 2 Review Current Strategy 3 Perform Market Research 4 Define Require‐ ments 5 Develop Acquisition Strategy 6 Execute Strategy 7 Manage Performance PLAN DEVELOP EXECUTE Middle  Tier Adaptive Acquisition Framework Tenets of the Defense Acquisition System Revised DoDI 5000.02 will include an Adaptive Acquisition Framework (AAF) with 6 tailorable acquisition pathways 5000.02T 23 Urgent Operational Needs DoDI 5000.81 To field  capabilities to  fulfill urgent  existing and/or  emerging  operational  needs or quick  reactions in  less than 2  years. • Identified and approved for resolution by designated authorities. • Estimated cost below MDAP thresholds • Processes, reviews, and documents are aggressively streamlined • Planning in a few weeks; development and production in months.  Transition documentation  complete within 2 Years Middle Tier of Acquisition 5 Innovative technologies,  Fieldable prototypes, New Capabilities * LCSP at program start is required for Major Systems in the Rapid Fielding Pathway only.  PMs, with the support of the product support managers,  will develop and implement sustainment programs addressing each of the integrated product support elements to deliver affordable readiness. Transition to Rapid Fielding or  New or Existing Program. ADM Program Start Develop/  Approve  Requirement Develop  Acquisition  Strategy  Full Funding Rapid Prototyping – 2 to 5 Years to Completion ADM Outcome Determination  (OD)  within 60 Days Fieldable  Prototypes Develop  Cost  Estimate &  LCSP* CAE Assigns  PM & PSM Transition to Operations and  Sustainment. Rapid Fielding  ‐ 2 to 5 Years to Completion ADM Outcome Determination (OD)within 60 Days Fielding Complete Proven technologies,  Minimal development New Production Start Production  within 6 Months MTA  Planning MTA Execution ‐MTA programs are not subject to JCIDS or major capability acquisition requirements. PMs will “tailor‐ in” reviews, assessments, and relevant  documentation that results in an acquisition strategy customized to the unique characteristics and risks of their program.   ‐MTA programs will include a process for demonstrating and evaluating performance to include cybersecurity and interoperability as applicable.  24 Capabilities- Based Assessment The Defense Acquisition Management System Relationship to JCIDS Acquisition ProcessJCIDS Strategic Guidance Joint Concepts Technology Opportunities & Resources Technology Opportunities & Resources User NeedsUser Needs OSD FCB A CB Technology Maturation & RR Production & Deployment Materiel Solution Analysis CDD Draft CDD MDD Engineering & Manufacturing Development CDD O&SICD PPBE DASJCIDS “If the Materiel Development Decision is approved, the MDA will designate the lead DoD Component; determine the acquisition phase of entry; and identify the initial review milestone.” JCS Update if Needed The Defense Acquisition Management System  The Materiel Development Decision precedes entry into any phase of the acquisition management system  Entrance Criteria met before entering phase  Evolutionary Acquisition or Single Step to Full Capability A CB LRIPTechnology Maturation & Risk Reduction Production & Deployment DRFPRD Materiel Solution Analysis CDD-V CDD ICD Draft CDD Operations & SupportMateriel Development Decision IOC FRP Decision Sustainment DisposalFOC Engineering & Manufacturing Development PDR CDR Model 1: Hardware Intensive Program Initial Capabilities Document (ICD) RELATIONSHIP TO JCIDS DRAFT CDD  PDR: Preliminary Design Review  CDR: Critical Design Review  CDD-V: CDD Validation  LRIP: Low Rate Initial Production  FRP: Full Rate Production  DRFPRD: Development Request For Proposals Release Decision  IOC: Initial Operational Capability  FOC: Full Operational Capability Capability Development Document (CDD) CDD Update if Needed 25 Achieving Full Capability Two strategy approaches to full capability: evolutionary and single-step.  Particular approach chosen depends on: o Availability of time-phased capabilities in the CDD o Technology maturity o Cost/benefit of incremental fielding vs. single step o Cost of fielding multiple configurations • Retrofit decisions & cost • Training • Supportability  Acquisition strategy shall address chosen approach  Evolutionary acquisition is the preferred strategy for rapid acquisition of mature technology PPBE Phases  Planning o Review threat / assess capabilities o Develop guidance o Defense Planning Guidance (DPG)  Programming o Turn guidance into achievable and affordable packages / programs o Program Objective Memorandum (POM) o Future Years Defense Program(FYDP)  Budgeting o Budget Estimate Submission (BES) o Scrub budget year o First year of FYDP  Execution o Measure performance against plan o Assess effectiveness of resource allocations PPBE DAMSJCIDS The Program Team Program Management Test and Evaluation Logistics Contracting Systems Engineering Financial Management 26 “Colors” of Money Appropriation Category Life Operation & Maintenance (O&M) 1 year MILPERS 1 year RDT&E 2 years Procurement (excluding SCN) 3 years SCN (Shipbuilding & Conversion, Navy) 5 years MILCON 5 years PPBE DAMSJCIDS * All appropriation categories are good for period of obligation plus five years for paying bills The Program Team The Program Team Program Management Test and Evaluation Logistics Contracting Systems Engineering Financial Management 27 Risk and Contract Types FFPCPFF Greatest Cost Risk to the Contractor Greatest Cost Risk to the Government FPAFFPI (F)CPAFCPIF Technical requirements defined; fair & reasonable prices determinable Vague technical requirements; labor and material costs uncertain Technical Risk Contractor Delivers “Best Effort” Contractor Delivers Acceptable Product “Typical” Contract Types by Phase PPBE DASJCIDS CPFF CPFF= Cost Plus Fixed Fee CPAF= Cost Plus Award Fee   CPFF, CPIF, CPAF, FPIF, FPAF  CPIF= Cost Plus Incentive Fee FPIF= Fixed Price Incentive Firm    FFP, FPIF FP (EPA) FFP = Firm Fixed Price FP (EPA) = Fixed Price Economic Price Adjustment     FPIF FFP A CB LRIPTechnology Maturation & Risk Reduction Production & Deployment DRFPRD Materiel Solution Analysis CDD-V CDD ICD DraftCDD Draft CDD Draft CDD Operations & SupportMateriel Development Decision IOC FRP Decision Sustainment DisposalFOC Engineering & Manufacturing Development PDR CDR CDD FFP Update if Needed 28 The Program Team The Program Team Program Management Test and Evaluation Logistics Contracting Systems Engineering Financial Management Technical  Systems Engineering─the overarching process that a program team applies to transition from a stated capability to an operationally effective and suitable system  Test and Evaluation─process by which a system or components are exercised and results analyzed to provide performance-related information o Program/contractor systems engineers o Developmental and Operational test communities  Supportability─includes design, technical support data, and maintenance procedures to facilitate detection, isolation, and timely repair and/or replacement of system anomalies o Program/contractor systems engineers o Program/contractor logistic engineers 29 Technical Reviews and Testing  Alternative Systems Review (ASR)  Systems Requirements Review (SRR)  System Functional Review (SFR)  Preliminary Design Review (PDR)  Critical Design Review (CDR)  Test Readiness Review (TRR)  System Verification Review (SVR)  Functional Configuration Audit (FCA)  Production Readiness Review (PRR)  Operational Test Readiness Review (OTRR)  Physical Configuration Audit (PCA)  Technology Readiness Assessment (TRA)  In-Service Review (ISR)  Developmental Testing (DT)  Early Operational Assessment (EOA)  Operational Assessment (OA)  Initial Operational Test & Evaluation (IOT&E)  Follow on Operational Test and Evaluation (FOT&E) ASRASR SRRSRR SFRSFR PDRPDR CDRCDR SVRSVR PCAPCA ISRISR FCAFCA PRRPRR TRATRATRATRATRATRAS EP TE M P DT&E OA IOT&EIOT&E FOT&EEOAEOA TRRTRR OTRROTRR A CB LRIPTechnology Maturation & Risk Reduction Production & Deployment DRFPRD Materiel Solution Analysis CDD-V CDDICD Draft CDD Operations & SupportMateriel Development Decision IOC FRP Decision Sustainment Disposal FOC Engineering & Manufacturing Development CDD PRRPRR Update if Needed Logistics/Sustainment  Technical and management activities conducted to ensure supportability implications are considered early and throughout the acquisition process to minimize support costs and to provide the user with the resources to sustain the system in the field. o Evaluate product support capabilities o Develop, demonstrate, and implement product support strategy • Logistics footprint control • Reliability, Availability, and Maintainability • Training, spares, technical manuals, transportation • Performance Based Logistics (PBL) agreements  Major Defense Acquisition Programs are now required by law to have a Product Support Manager. 30 Logistics/Sustainment Planning Performance Based Logistics (PBL), is the required DoD approach for product support. It allows us to manage program and system outcomes such as materiel availability and reliability as opposed to actions and transactions such as repairs and parts. “a strategy for weapon system product support that employs the purchase of support as an integrated performance package designed to optimize system readiness. It meets performance goals for a weapon system through a support structure based on performance agreements with clear lines of authority and responsibility.” Initiate LCSP LCSP Update Performance Based Logistics Implementation Demonstrate Product Support Capability A CB LRIPTechnology Maturation & Risk Reduction Production & Deployment DRFPRD Materiel Solution Analysis CDD-V CDD ICD DraftCDD Draft CDD Draft CDD Operations & SupportMateriel Development Decision FRP Decision Sustainment Disposal FOC Engineering & Manufacturing Development IOC CDD LCSP Update & ILA LCSP Update & ILA LCSP Update & ILA Update if Needed Cybersecurity & the Acquisition Lifecycle Integration Tool (CALIT) CALIT Ver 3.1 Sep 2018 31 Materiel Solution Analysis  ENTER: Approved ICD, study guidance for conducting the AoA and an approved AoA plan. AoA study guidance for MDAPs and AoA plan approval will be provided by CAPE.  ACTIVITIES: Establish PM & PMO, Conduct AoA, user writes draft CDD, develop initial: • Acquisition Strategy • Test & Evaluation Master Plan (TEMP) • Systems Engineering Plan (SEP) • Life Cycle Sustainment Plan (LCSP) • Cyber Security Strategy • Other Program Documentation as Required  GUIDED BY: ICD and AoA Plan  EXIT: Completed the necessary analysis and activities to support a decision to proceed to the next decision point and desired phase in the acquisition process. PURPOSE: to conduct the analysis and other activities needed to choose the concept for the product that will be acquired A Materiel Solution Analysis ICD DraftCDD Materiel Development Decision Technology Maturation and Risk Reduction  ENTER: MDA approved materiel solution and Acquisition Strategy, initial major program documentation and funding in the FYDP  ACTIVITIES: Competitive prototyping of critical subsystems, SE Trade- off analysis, develop contracting strategy, conduct CDD Validation, conduct Preliminary Design Review (PDR), conduct Development RFP Release Decision, begin source selection for EMD  GUIDED BY: Acquisition Strategy & Draft CDD/Approved CDD, SEP & TEMP  EXIT: Demonstration that technology, engineering, integration, manufacturing, sustainment, and cost risks risk have been adequately mitigated to support a commitment to design for production, Validated capability requirements, full funding in the FYDP, and compliance with affordability goals for production and sustainment PURPOSE: to  reduce  technology,  engineering, integration, and  life cycle cost risk  to the point that  a decision to  contract for EMD  can be made with confidence  in successful  program  execution for  development,  production, and  sustainment A B Technology Maturation & Risk Reduction DRFPRD CDD-V CDDDraft CDD Final RFP PDR Final RFP 32 Engineering and Manufacturing Development  ENTER: Adequate Risk Reduction; Approved Requirements; Full Funding in FYDP  ACTIVITIES: Complete detailed design, system-level CDR, integrated testing, establish product baseline, demonstrate manufacturing processes and supportability  GUIDED BY: CDD, Acquisition Strategy, SEP & TEMP  EXIT: (1) the design is stable; (2) the system meets validated capability requirements demonstrated by developmental and initial operational testing as required in the TEMP; (3) manufacturing processes have been effectively demonstrated and are under control; (4) industrial production capabilities are reasonably available; and (5) the system has met or exceeds all directed EMD Phase exit criteria and Milestone C entrance criteria PURPOSE: to develop, build,  and test a  product to verify that all  operational and  derived  requirements  have been met  and to support  production or deployment  decisions B Engineering & Manufacturing Development CDR C PDR? CDD Update if Needed Production and Deployment  ENTER: Acceptable performance in DT & OA; mature software; no significant manufacturing risks; approved CDD; acceptable interoperability and operational supportability; demonstration of affordability; fully funded  ACTIVITIES: Low Rate Initial Production, IOT&E, LFT&E (If Required) and interoperability testing of production-representative articles; Full-Rate Production Decision; fielding and support of fielded systems; IOC/FOC  GUIDED BY: CDD, TEMP, SEP, LCSP  EXIT: Full operational capability; deployment complete PURPOSE: to produce and deliver requirements- compliant products to receiving military organizations LRIP Production & Deployment FRP Decision FOC IOC C Full Rate Production CDD Update if Needed 33 Operations and Support  ENTER: Approved CDD; approved LCSP; successful FRP Decision  ACTIVITIES: LCSP implementation; Performance-Based Life-Cycle Product Support (PBL) implementation, and management; initiate system modifications as necessary; continuing reviews of sustainment strategies, Demilitarize and dispose of systems IAW legal and regulatory requirements, particularly environmental considerations and explosives safety • GUIDED BY: CDD/Acquisition Strategy/LCSP PURPOSE: Execute a support program that meets materiel readiness and operational support performance requirements, and sustains the system in the most cost- effective manner over its total life cycle. Operations & Support Sustainment Disposal FOC The Defense Acquisition Management System  The Materiel Development Decision precedes entry into any phase of the acquisition management system  Entrance Criteria met before entering phase  Evolutionary Acquisition or Single Step to Full Capability A CB LRIPTechnology Maturation & Risk Reduction Production & Deployment DRFPRD Materiel Solution Analysis CDD-V CDD ICD Draft CDD Operations & SupportMateriel Development Decision IOC FRP Decision Sustainment DisposalFOC Engineering & Manufacturing Development PDR CDR Model 1: Hardware Intensive Program Initial Capabilities Document (ICD) RELATIONSHIP TO JCIDS DRAFT CDD  PDR: Preliminary Design Review  CDR: Critical Design Review  CDD-V: CDD Validation  LRIP: Low Rate Initial Production  FRP: Full Rate Production  DRFPRD: Development Request For Proposals Release Decision  IOC: Initial Operational Capability  FOC: Full Operational Capability Capability Development Document (CDD) CDD Update if Needed 34 Intermediate Systems Acquisition Course March 2020 Warm Up Exercise For each of the following situations, determine where on the acquisition life cycle model would recommend the Milestone Decision Authority (MDA) authorize entry into the defense acquisition management framework? 1. An Initial Capabilities Document (ICD) was validated and approved for a joint war fighting capability to intercept and attack ballistic missile reentry vehicles in mid-course, prior to reentering the earth's atmosphere. The ICD identified several possible materiel approaches to provide the required capability including an air launched missile interceptor. Market research determined that the technology is feasible, but the various possibilities need to be analyzed to determine the best missile and launch platforms before the appropriate technology can be demonstrated. The MDA also wants to designate a lead DoD Component for this joint war fighting system, needs a strategy for rapid fielding using evolutionary acquisition, and wants to encourage maximum innovation and competition for the best system(s) from private industry. CAPE has issued AoA study guidance and approved an AoA study plan. 2. Senior leaders in the U.S. Army are anticipating protracted times of constrained budgets and limited opportunities to train. Army leaders are looking for technology solutions that will greatly improve accuracy when firing side arms with limited training. There is a recently approved CDD leveraging an already existing ICD for Soldier small-arms capability needs. The CDD requires a new Soldier side-arm solution that includes an integrated targeting LASER with significantly improved first shot accuracy. Multiple commercial vendors offer pistols with integrated targeting LASERs; three vendors in particular have existing contracts and running productions lines supplying the U.S. Marine Corps and U.S. Special Operations Forces. Field evaluations from the Marines and SOF combat units indicate effectiveness and suitability of the firearms, particularly accuracy, which meets the CDD thresholds. The program has full procurement funding. 3. An ICD has been validated and approved for a capability to intercept and attack ballistic missile reentry vehicles in mid-course, prior to reentering the earth's atmosphere. Air Force will be lead service to develop this capability. An analysis of alternatives and an acquisition strategy have been completed and the Air Force has selected as the best system a laboratory proposal for a laser mounted on an existing airplane. Funding for the effort was included in the latest update to the FYDP. The concept is promising, however, the technology has not been matured and there are significant performance risks. The user has provided a draft CDD based on the ICD. 4. A Navy Lab has developed a protective eye shield/mask that will guard the wearer's eyes against the full spectrum of current lasers directed from any angle. The Navy Lab has coordinated with the users, who have produced an ICD and CDD that have both been validated and approved by the Chief of Naval Operations. The Navy Acquisition Executive agreed to fully support this initiative in the upcoming budget review, and has identified specific offsets in other programs to provide the funding. The technology appears to be mature and technical risks are assessed as low. However, the system has yet to be tested outside of the lab. It also has not been integrated with other components of a helmet system. 35 Intermediate Systems Acquisition Course March 2020 Capstone Exercise (to be completed and briefed Friday) – Background Firebird II unmanned air vehicles (UAV’s) have reached FOC, and have been successfully carrying out military operations around the world. The survivability enhancements provided in Firebird II have reduced the loss rate from heat-seeking shoulder-launched missiles to less than 10% per engagement. However, because of deeper defense budget cuts and further consolidation, the need has emerged for Firebird to provide more persistent Intelligence, Surveillance, and Reconnaissance (ISR) capability. As a result, the Services want to increase the Firebird loiter time from 3 hrs to 4.5/6.0 hrs (threshold/objective). This increase in loiter time is to be provided while achieving the original Firebird II requirement for range of 250 KM/300 KM (threshold/objective). The next increment, dubbed “Firebird III”, is planned to address the capability needs and include engineering changes to address reliability degraders. The MDA has approved the Materiel Development Decision for Firebird III and an AoA has been completed. The following new requirements have been included in the draft CDD for Firebird III: DRAFT REQUIREMENTS FOR FIREBIRD III 1. Firebird III will have a loiter capability of 4.5/6.0 hrs (threshold/objective). This is a Key Performance Parameter (KPP). 2. Firebird III will have a range of 250/300 KM (threshold/objective). This is a Key Performance Parameter (KPP). 3. Firebird III will provide Link 17 capability for real-time transfer of compressed digital video intelligence to the DoD Information Network (DoDIN). All other requirements from the Firebird II CDD remain unchanged. You can assume that the program will be fully funded in the FYDP in time for the next milestone and the CDD will be approved in time for a DRFPRD. Situation Ms. Connie Smith, former contracting officer for the Firebird program, has been promoted and is the newly appointed interim Program Manager for Firebird III. She has asked your team to develop a program structure chart for the Acquisition Strategy. She has provided the following information: - Firebird III will be a joint, ACAT II program with the Army as the lead service. - Initial Operational Capability (IOC) objective date is 42 months from program initiation. IOC threshold date is 48 months from program initiation (Milestone B). 36 Intermediate Systems Acquisition Course March 2020 The user’s requirement for IOC is 2 operational Firebird III systems (2 ground stations and 8 UAVs). - Retrofit of 400 Firebird air vehicles is required to meet FOC for all the military services combined. Responses to an RFI have provided the Program Manager with several possible acquisition strategy alternatives. She wants to explore three different acquisition strategy approaches: Approach 1. Two contractors responding to the Request for Information have indicated that they have integrated a more fuel efficient, lightweight engine into a commercial variant of the Firebird. The contractors have already successfully flown these prototype Firebird air vehicles using this engine which indicates the potential to meet the increased loiter time and range requirements. However, the prototypes must be further refined and developed before production representative units suitable for Operational Testing can be fully designed, integrated, and built. Approach 2. Recent advances in lightweight material technology research look promising. MIT has been working on a new material, called Litex that may be suitable for aircraft skins due to its combination of extreme light weight and strength. The Air Force Research Laboratory has published a white paper describing potential future applications of this technology in UAVs. According to the paper, the Firebird airframe could be retrofitted with Litex to meet the increased loiter time and range requirements if this technology pans out. However, it also states that Litex is not yet mature enough to use in aircraft. Six months of development and testing is necessary to determine durability and temperature limits before the material is ready for integration into aircraft and actually flown at representative altitudes and airborne profiles. Several U.S. contractors have the necessary technical and manufacturing capabilities to apply this technology to the Firebird. Approach 3. Mannheim Technologies, a small German company, has proposed adding a “probe and drogue type” aerial refueling capability to Firebird. According to Mannheim, UAV to UAV refueling is feasible using GPS technology, automated flight controls, and optical tracking systems to approach, link-up, and complete the refueling procedure. If successful, the ability to refuel would significantly increase loiter time and range of the Firebird without the need to change the engine or airframe materials. In addition, it may be possible to provide more persistent ISR capabilities with fewer Firebird UAV’s. Although DARPA and other U.S. defense contractors are working on UAV to UAV refueling as well, significant effort remains before the technology is considered mature and it has yet to be demonstrated in flight test. Note to Students: All the situations allow competitive acquisition and two of the approaches should consider using use competitive prototyping in TMRR. In order to do that, the text mentions commercial variants of the Firebird. These are demilitarized versions (without weapons, some sensors and secure data links) that are sold internationally for use by border patrols, coast guard, fisheries and wildlife managers, conservationists, police and scientific communities. The variants allow the situation that companies other than CyboRaptor have access to Firebirds to develop prototypes/EDMs with the required capabilities. 37 Intermediate Systems Acquisition Course March 2020 Exercise Introduction One team member will serve as the IPT leader. The IPT leader will be responsible for guiding the efforts of the other team members and for briefing the program structure chart. The other team members will assist the IPT leader by providing functional area expertise (contracting, systems engineering, test and evaluation, logistics, and financial management). Prior to your IPT meeting on Friday, all team members should review the assigned approach and consider the questions related to each functional area in the development of the program structure chart. No more than 2 hrs will be provided Friday morning to complete the program structure chart, so advance research and coordination with team members will be necessary. Remember to include activities related to all Firebird III changes. You may leverage and expand upon work completed in previous lessons and exercises. In addition, the Program Manager needs to ensure that we have addressed concerns expressed by the Program Executive Officer. Each team will be assigned one of the areas of concern to address in detail as part of the exercise briefing. Assignment 1: Your team will be assigned to explore one of the above approaches. You are now in FY-1 of the Firebird III effort. Assume that it will be at least three months before your Acquisition Strategy will be approved. Based on the information above for your assigned approach, determine in your team: - At what point in the life cycle will your program enter systems acquisition? - What phases and work efforts will be included? Lay out on a notional timeline the acquisition life cycle phases, work efforts, and major milestone/program decision reviews needed to execute and oversee your program. Be prepared to present your timeline to the class, including the rationale for your decisions and any assumptions your team made (as part of your Friday briefing). Do not go on to Assignment 2 until after you have presented your timeline to one of the instructors and received the instructor’s approval to proceed. Your team should review Assignment 1 with an instructor Thursday morning at the latest. Assignment 2: Using the timeline you developed, fill out the rest of the elements of the program structure chart for your approach. It may be helpful to complete the program structure chart through a series of steps for each phase of the program, as outlined below. If necessary, make assumptions about the technology, the operational requirements, and the political and economic situation in order to complete your program structure chart. List your supporting assumptions on butcher paper as you go. The instructor may change or add to those assumptions before you complete your acquisition strategy. 38 Intermediate Systems Acquisition Course March 2020 The IPT leader will present to the class a 15-minute overview of your program structure chart, your assumptions, and the rationale for your decisions. Use the following questions to help frame your team’s thinking as you put together the details of your acquisition strategy. Step 1 - Programmatic Issues: Consider overall programmatic issues as you begin developing your acquisition strategy: - What are the major program risks regarding cost, schedule, and performance? - What can you do to mitigate those risks? - How much concurrency will be in your acquisition strategy? You might save time by overlapping activities, but you might also increase risk. - What new environmental issues, if any, might need to be addressed? - Is international cooperative development feasible? - When will initial operational capability (IOC) be achieved? Step 2 - Contracting Issues: Consider contractual issues for your acquisition strategy: - How many contractors will develop and produce Firebird III in each phase of your acquisition strategy? - How will you address competition? - What types of contracts will be used in each phase of your acquisition strategy? - What are your planned dates for RFP release(s) and contract award(s)? Step 3 - Technical Management Issues: Determine how you will address technical management aspects of your acquisition strategy: - Technical reviews and audits: which ones, when, and how many? - How can modeling and simulation be used to support the program? - Will interoperability with other systems be affected? The increment cannot disrupt any interchanges required between Firebird and other systems. - To what extent will you use open systems architecture? Why? - What types of testing will be conducted, and when it will take place? How will interoperability and reliability upgrades be tested? Step 4 - Logistics Issues: Consider how you will address supportability aspects of your acquisition strategy. - What new supportability issues arise in the transition from Firebird II to Firebird III? - What supportability planning needs to occur and when? - What testing needs to be done to confirm the required reliability is achieved prior to fielding? When should the testing be done? 39 Intermediate Systems Acquisition Course March 2020 - How will performance based life cycle product support be performed? Step 5 - Production Issues: Consider the articles required for conducting various tests, including both LRIP and full rate production: - What quantities of items will be produced? - What is the purpose of articles produced? - When will they be delivered? Step 6 - Financial Management: Address financial management aspects of your acquisition strategy. - What appropriation(s) will be used during each phase of your program? Indicate them at the bottom of your program structure chart. - What impacts, if any, will the end or beginning of a fiscal year have on your funding strategy? - How will you deal with the PPBE process, including getting initial funding? Note: The PMO and the user agreed to the following life cycle cost objectives (in the current base fiscal year). These costs reflect agreed-to affordability goals: - RD&TE: $300M - Procurement: $410M based on the following estimates: - 400 air vehicle retrofit kits @ $1M each = $400M - 100 ground station retrofit kits @$100K each = $10M - O&M: $1.2B over 20 years of system life. If necessary, make assumptions concerning the costs of your program. Program Executive Officer - Areas of Concern: Also address these areas of concern for the PEO. Technology Maturity - What specific actions do you intend to take during Technology Maturation and Risk Reduction phase to ensure that your program is ready for Engineering & Manufacturing Development phase? Schedule Risk - You may be able to compress your schedule if you plan to conduct some activities concurrently (e.g., development and initial production, DT and OT, design reviews, etc.). How much concurrency (overlapping activities) is appropriate for your strategy? What specific efforts might be good candidates for a concurrent approach? Test Efficiency - How does your strategy address integrated testing (DT/OT)? What actions are necessary now and throughout your strategy to ensure we take full advantage of integrated testing? How would you use modeling and simulation? 40 Intermediate Systems Acquisition Course March 2020 Competition - How does your strategy address competition in each phase? What are some actions we can take as part of the initial effort to enable greater competition in subsequent phases? Are there opportunities to compete at a subsystem level and how would we plan for that? Operational Suitability - How does your strategy address reliability and maintainability? What specific actions/efforts should we consider in each phase to ensure we develop a suitable system? How will we know if we are on track? Logistics Support Strategy - What kind of PBL strategy should we pursue? What actions would we need to take in the initial phase of work to facilitate this strategy for deployment? Assignment 3: In this assignment, the IPT leader (with limited assistance of other team members) will present to the class a 15-minute overview of your program structure chart, your assumptions, and the rationale for your decisions. The presentation of your acquisition program chart should address at a minimum: • Key milestones, reviews and phases • Number of contractors in each phase and your rationale regarding competition • Contract types • RFP release and contract award dates • Technical reviews and audits • Production deliverables • Major test events • IOC date • Appropriation category of funding required for each phase of the acquisition • Logistics/life cycle sustainment events and deliverables Be prepared to discuss the major risks regarding cost, schedule, and performance and what you can do to mitigate those risks. The Program Manager will be most interested in what you consider to be your top technical risk. You will also be asked to evaluate and question the acquisition strategies presented by the other teams and provide constructive feedback. 41 Intermediate Systems Acquisition Course March 2020     42 Intermediate Systems Acquisition Course March 2020 DRAFT FIREBIRD II UNMANNED AERIAL VEHICLE (UAV) SYSTEM CAPABILITY DEVELOPMENT DOCUMENT (CDD) (Excerpt of Performance Requirements) 43 Intermediate Systems Acquisition Course March 2020 44 Intermediate Systems Acquisition Course March 2020 UNCLASSIFIED DRAFT CAPABILITY DEVELOPMENT DOCUMENT FOR FIREBIRD II UNMANNED AERIAL VEHICLE (UAV) SYSTEM Increment: II ACAT: II Validation Authority: JROC Approval Authority: Army Milestone Decision Authority: Assistant Secretary of the Army (Acquisition, Logistics and Technology) Joint Staffing Designator: JCB Interest Prepared for Milestone B Decision [Subparagraphs in italics are omitted because they are not applicable for academic purposes] Executive Summary [Simplified for academic purposes]: Firebird II is the second increment to the baseline Firebird in response to expected but initially requirements to increase range and improve survivability. This increment will be accomplished as part of the Firebird’s evolutionary acquisition strategy. Revision History: Omitted: Not required for classroom activities. Table of Contents: Omitted: Not required for classroom activities. Points of Contact: Omitted: Not required for classroom activities. 1. Capability Discussion. a. Firebird losses due to shoulder-launched missiles are much higher than planned, exceeding the ability of support systems to sustain the system. This has resulted in unacceptably low operational availability and unplanned costs. b. The initial increment of Firebird does not have sufficient range to conduct reconnaissance operations out to 250 KM. This range is needed due to the earlier-than- anticipated loss of another military asset (classified), which will create a gap in coverage. 2. Analysis Summary. [Simplified for academic purposes] The AoA conducted during Materiel Solution Analysis identified several technical approaches that could achieve the desired improvement in range and survivability. The Acquisition Strategy recommended that a Technology Maturation and Risk Reduction phase be used to further develop these approaches and reduce the risk to an acceptable level. 3. Concept of Operations Summary. [Simplified for academic purposes] The intent is to field a joint UAV, Firebird II, with improved Range and Survivability over the current Firebird. The system will perform the same general reconnaissance, surveillance and target acquisition 45 Intermediate Systems Acquisition Course March 2020 missions as well as the capability to detect, track and launch a weapon to destroy a moving vehicle or fixed target. 4. Threat Summary. The system will be directed against lightly armored, mobile ground targets, such as Scud missile launchers, SA-9 Surface-to-Air Missiles and other mobile artillery weapons. It will also be used against small lightly armored water-borne targets (e.g., gunboats). 5. Program Summary. This is the second increment in the Firebird program. a. The first increment provided a system capable of locating and destroying lightly armored enemy ground targets from an unmanned aerial vehicle (UAV) when directed from the ground and/or naval ships by friendly forces during daylight hours. Each system consists of four recoverable unmanned air vehicles and a ground station equipment package. System is used by all U.S. services. b. This second increment will address range and survivability improvements. c. The third increment will address improved loiter time. 6. System Capabilities Required for the Current Increment a. The UAV shall be capable of being deployed from a mobile launcher unit. The launcher shall be capable of propelling the aerial vehicle from a standing stop to airborne within a distance of 25 (objective) to 30 (threshold) feet. b. The UAV shall be recoverable, with or without munitions on board, onto an unimproved landing surface (threshold). It shall be capable of being re-used in subsequent missions. c. Minimum range of the UAV shall be 250/300 km (threshold)/(objective). d. The UAV shall have the ability to cruise at speeds between 40 and 80 kilometers per hour (KPH). Once within the patrol area, the system should be able to loiter for at least three (3) hours (threshold) in a search and destroy mode. The system, with its munitions mounted on the UAV, shall have an explosive force comparable to 200 (threshold) to 500 (objective) lbs. of TNT, with the system having a CEP1 of 10 (threshold)/5 (objective) meters. e. The UAV will have a single optical target acquisition system (threshold) for daytime operations. f. The UAV shall be capable of transmitting video images in real time, throughout the mission, to a ground control unit beyond the line of sight (LOS), other military airborne 1 CEP, the circular error of probability, refers to the radius around the target within which the munitions must fall 50% of the time. 46 Intermediate Systems Acquisition Course March 2020 surveillance and targeting units, and receiving and responding to avionics commands from the ground control terminal (threshold). g. The UAV shall be able to link and exchange data with the DoDIN and other systems as defined in Annex A, Net-Ready KPP Products. h. Firebird II will incorporate improved survivability measures such that the probability of survivability during a single engagement by a shoulder-launched heat-seeking missile is greater than or equal to 90%. i. Materiel Reliability - Mean Time between Critical Failure (MTBCF) shall be no less than 150 (threshold)/200 (objective) hours. j. Mean Time to Repair (MTTR) shall not exceed 3 (threshold)/2.5 (objective) hours. Table 4.1 Key Performance Parameter Table Tier 1 & Tier 2 JCAs Key Performance Parameter Development Threshold Development Objective Omitted: Not required for classroom activities Range 250 Km 300 Km Survivability : Expected survival rate when engaged by shoulder-launched heat-seeking missiles The probability of survivability during a single engagement by a shoulder- launched heat-seeking missile shall be greater than or equal to 90%. Same Loiter 3 hours Same Explosive Force 200 lbs TNT 500 lbs TNT Accuracy 10 Meter CEP1 5 Meters CEP1 Net-Ready System supports military ops, is entered on the network and effectively exchanges information Same Sustainment Materiel Availability ( Am) of .80 .85 Training (not required for classroom activities) Energy (not required for classroom activities) Force Protection Not applicable: Firebird II is not a manned system Survivability Not applicable: Firebird II is not a manned system Table 4.2 Key System Attributes Table Additional Performance Attribute Development Threshold Development Objective Mobile Launch Distance 30 Feet 25 feet Recovery Conditions with Munitions Unimproved landing surface Same Cruising Speed 200 KPH 300 KPH Target Acquisition System One optical system suitable for daytime operations Same Materiel Reliability Mean Time between Critical Failure (MTBCF) 150 hours 200 hours Mean Time to Repair (MTTR) 3 hours 2.5 hours Operations and Support Costs Omitted: Not required for class 47 Intermediate Systems Acquisition Course March 2020 7. Family of Systems and Systems of Systems Synchronization. – Omitted: Not required for classroom activities. 8. Information Technology and National Security Systems Supportability. - Omitted: Not required for classroom activities. 9. Intelligence Supportability - Omitted: Not required for classroom activities. 10. Electromagnetic Environmental Effects (E3) and Spectrum Supportability – Omitted: Not required for classroom activities. 11. Assets Required to Achieve Initial Operational Capability (IOC) - Omitted: Not required for classroom activities. 12. Schedule and IOC and Full Operational Capability (FOC) Definitions. The program should take no longer than 48 months (threshold)/42 months (objective) from initiation to IOC. IOC is defined as two combat-ready systems (8 UAVs and two ground stations) with properly trained and equipped personnel. FOC is 400 Firebird air vehicles retrofitted and 100 ground stations modified. 13. Other DOTMLPF and Policy Considerations [Simplified for academic purposes] Logistics and Facilities Considerations a. Maintenance Planning: Maintenance shall be limited to two levels: operator maintenance and depot repair. Repair parts shall be commercially available to the maximum extent practical. b. Ground Stations: The ground station shall consist of a launcher, a ground control unit, commercially available hand tools, and associated documentation. The ground control unit shall include built-in-test equipment to verify flight control circuitry and it shall contain simulation flight control software to be used as a training tool. c. Human Systems Integration: The system shall be capable of set up, operation, and tear down by a crew of no more than four (threshold) or three (objective) trained personnel. d. Transportation and Basing: The system shall be capable of being moved within the theater by aircraft (CH-47 and larger) or vehicle (2 1/2 ton truck and larger). 14. Other System Attributes – Omitted: Not required for classroom activities. 15. Program Affordability. [Simplified for academic purposes] RDT&E Objective $325M Threshold $350M 48 Intermediate Systems Acquisition Course March 2020 Procurement Objective $650M Threshold $720M (Dollars are Then Year) Mandatory Appendices. Appendix A. Net-Ready KPP Products (Not required for classroom activities) Appendix B. References (Not required for classroom activities) Appendix C. Acronyms (Not required for classroom activities) 49 Intermediate Systems Acquisition Course March 2020 50 Intermediate Systems Acquisition Course March 2020 LESSON ASSIGNMENT SHEET Lesson Number Exercise 1.4 ______________________________________________________ Lesson Title Materiel Solution Analysis ______________________________________________________ Lesson Time 2 hours ______________________________________________________ Terminal and Enabling Learning Objectives TLO Evaluate alternative approaches to meet a needed capability based on affordability, schedule and technical considerations ELO Given a user’s requirement and selected concept, select an appropriate approach from the perspective of the system developer, to meet the requirement. ELO Identify the three major dimensions of program risk used to analyze technical approaches during the Materiel Solution Analysis Phase (cost, schedule and performance) ELO Identify the concept of affordability goals in relation to an acquisition program. ELO Relate the concept of affordability goals to the planning of an acquisition program. ELO Working in a student-led IPT, demonstrate the behaviors and characteristics of an effective team. ______________________________________________________ Assignments Review the following ACQ 202 CBT Lesson Summaries: • Lesson 1.1, Considering the Costs • Lesson 1.2, Selecting the Best Approach • Lesson 2.2, Developing the Acquisition Strategy ______________________________________________________ Estimated Student Preparation Time None ______________________________________________________ Assessment Class participation; oral presentation ______________________________________________________ 51 Intermediate Systems Acquisition Course March 2020 Related Lessons CBT Lesson 1.1, Considering the Costs CBT Lesson 1.2, Selecting the Best Approach Classroom Exercise 1.3, Acquisition Strategy ______________________________________________________ Self Study References • DoDD 5000.01, The Defense Acquisition System • DoDI 5000.02T, Operation of the Defense Acquisition System Defense Acquisition Guidebook ______________________________________________ 52 Intermediate Systems Acquisition Course March 2020 Exercise 1.4 Materiel Solution Analysis This lesson is divided into two activities, A and B. Everyone should read both activities. However, half of the student teams will be assigned Activity A, and the other half will be assigned Activity B. Activity A – Enhanced Survivability Scenario Firebird unmanned aerial vehicles (UAVs) are nearing the end of fielding, and the Services have used them extensively in a number of conflicts. When this first increment of the Firebird was in development, a second increment was planned to provide additional survivability from current and projected threats from heat seeking shoulder-launched missiles. Although military operators are extremely pleased with Firebird’s combat capabilities, they are unhappy with its poor availability due to higher-than-anticipated combat losses. Most of these losses have been from heat-seeking shoulder-launched missiles. Records show that several air vehicles were shot down by these missiles over the last few years. The losses have created a serious air vehicle shortage, leading to unacceptably low operational availability. The second increment is now in the early part of the Materiel Solution Analysis Phase. This increment will significantly increase Firebird’s survivability against the shoulder-launched missile threat. This increment and a future 3rd increment are supported by a time-phased requirement, originally documented in the approved Firebird I Capability Development Document (CDD). This increment of the program, dubbed “Firebird II,” is now being planned to meet the new survivability requirement. This activity is supported by the Acquisition Strategy created in Firebird I’s development. The users have drafted the following requirement language: DRAFT REQUIREMENT FOR FIREBIRD SURVIVABILITY ENHANCEMENT Capabilities required: 1. Firebird II will incorporate improved survivability measures such that the probability of survivability during a single engagement by a shoulder-launched heat-seeking missile is greater than or equal to 90%. This is a Key Performance Parameter (KPP). 2. Firebird II must meet all unamended requirements in the CDD for the first increment. 53 Intermediate Systems Acquisition Course March 2020 Using the draft requirement language as a guide, three alternative approaches for enhancing survivability have been studied by Mitronix, a Federally Funded Research and Development Center (FFRDC). Their report is provided below. FIREBIRD II UAV ALTERNATIVE APPROACHES FOR INCREASING SURVIVABILITY Approach 1: Modify Firebird to fly high enough to avoid shoulder-launched missiles. Research shows that existing shoulder-launched missiles have an effective ceiling of 15,000 ft., but intelligence sources indicate that near-term improvements are expected to increase the ceiling to 18,000 feet. Analysis indicates that increasing Firebird’s ceiling to 20,000 ft. when loitering in the threat zone will meet the new survivability requirement. Increasing Firebird’s operational altitude will require some changes to existing control software in both the vehicle and ground station. The software effort should have minimal impact on the overall time and cost for the upgrade and is considered low risk. Higher altitudes will necessitate a modified or new propulsion system, redesign of fuel systems, and upgrades to vehicle sensor packages. Technology in these areas is fairly mature resulting in low hardware risk. Research and development (R&D) costs are expected to be $140M (RDT&E appropriation) due to the extensive testing and work required to design all the modifications. Production costs are estimated at $285M (Procurement appropriation). Operations and Support (O&S - a combination of O&M and MILPERS appropriations) costs with this upgrade are estimated at $32.5M per year. Disposal cost is estimated to be $65M. It is expected this approach will take 34 months from program initiation to initial operational capability (IOC). However, if new engine technology now in advanced development does not mature as planned, IOC would end up slipping to 36 months. Approach 2: Add on-board countermeasures (flares) and pre-programmed evasive maneuvering to avoid heat-seeking targeting systems of incoming missiles. Flares are missile decoy devices that are released from air vehicles when a heat-seeking threat is detected. When combined with evasive maneuvering, flares are effective survivability enhancers that have been successfully used for years by manned aircraft. Adding flares and evasive maneuvering would allow Firebird to meet the new survivability requirement without increasing altitude. This approach requires integration of new control capabilities into both the ground control console and the air vehicle. Also, the addition of missile sensing and evasive maneuvering capabilities necessitates writing and rigorously testing a large amount of new software. Missile sensing technology is widely available and considered a low risk. Some of the existing flight control software will be re-usable for the evasive maneuvering. Past experience shows that flight control software complexity is often underestimated. Therefore, this approach entails moderate software risk. Required integration of both the mechanical operation and the physical characteristics (size, weight, attachments, etc.) of new countermeasures into the air vehicle is considered moderate risk. R&D costs are estimated to be $150M (RDT&E). Most of those costs are due to the extensive software effort required. Production costs are expected to be $300M 54 Intermediate Systems Acquisition Course March 2020 (Procurement). Operations and support costs with this upgrade are estimated at $28.75M (O&S) per year. Disposal cost is estimated to be $58M This approach will take 30 months from initiation to IOC. There is a relatively low risk that the flight control software will not be reusable, which would add three months and $4M (RDT&E) to this approach. Approach 3: Reduce the heat signature of the vehicle. Reducing the heat signature of air vehicles to increase survivability has a proven track record in numerous existing aircraft. Heat signature reduction techniques and materials in use are relatively mature and cost-effective. If Firebird’s heat signature can be reduced sufficiently it will meet the new survivability requirement. This option requires significant hardware redesign of portions of the airframe structure, mostly in the engine exhaust area. While the technology is mature it is expected that current techniques and materials cannot reduce Firebird’s signature sufficiently to meet the requirement. Also, characteristics of Firebird, such as low speed, small size, and the need for short takeoffs and landings, will make this a high-risk hardware redesign effort. Extensive testing will be required to prove performance and reliability, but much of the data should be available from the labs and/or modeling and simulation. Software risk is low since this approach will require only minimal changes to existing software code. R&D costs are expected to be $160M (RDT&E) due in large part to the redesign challenges. Production costs are estimated at $320M (Procurement). Most of those costs are driven by the anticipated need for expensive materials and unique manufacturing processes. Operations and support costs with this upgrade are estimated at $26.25M (O&S) per year. Disposal cost is estimated to be $53M. This approach is projected to take 32 months from initiation to IOC. There is a moderate risk that a new propulsion system or major redesign of the existing system will be required. If that happens, both R&D and production costs will increase 30% and the schedule will stretch 6 months. For teams assigned Activity A (Increased Survivability), here is your tasking: 1. Choose a team leader/briefer. Use the information in the Mitronix report to build a matrix that shows the cost, schedule and technical characteristics of each of the three approaches. 2. Preliminary discussions among the PM, PEO, service officials and other stakeholders indicate that money and time are tight, as usual, but the user has a valid need for the survivability enhancement. Taking these discussions into account, along with an affordability analysis and the urgency of the requirement, the PM has provided the following guidance: “This survivability enhancement should not take longer than 36 months from initiation to IOC, the R&D affordability goal is $160M (RDT&E), and the Production affordability goal is $320M (Procurement). O&S costs should be no more than $32.5M per year. Total Life Cycle Cost (LCC) should not exceed $1.19B. Also, cost, schedule and technical risks should be weighted equally when considering alternative approaches. Assume a 20-year operational life for the Firebird II UAVs.” 55 Intermediate Systems Acquisition Course March 2020 Your team has been asked to help assess technology that is currently under development to determine applicability to the Firebird II. Assuming the draft requirement language will be approved as written, and considering the PM’s guidance: - Use your matrix to help you rank each of the three approaches based on the overall risk of meeting the new requirement within schedule and affordability goals - Discuss and list any assumptions your group feels are necessary. For this academic exercise, do not create new approaches or combine elements of different approaches. 3. Prepare a 10-minute briefing to the class that: - Explains how you built your matrix. - Lists and explains any assumptions made by your team. - Lists and explains the rationale behind your approach rankings. 56 Intermediate Systems Acquisition Course March 2020 Activity B - Increased Range Scenario The first increment of Firebird brought a much-needed capability to the operational forces. When this first increment of the Firebird was in development, it was recognized that additional range would be required in the future. The second increment is now in early part of the Materiel Solution Analysis Phase. The user has found new and innovative uses for Firebird and, simultaneously, the military is losing a critical aviation mission asset much earlier than expected due to increases in operational tempo, budget cuts, and consolidation. Loss of this asset will create a gap in reconnaissance coverage within four years. All the Services want to fill the resulting gap by increasing Firebird’s range from 100 to 250 KM (threshold)/300 KM (objective). As part of the acquisition process, the Services are planning to execute the next increment to increase Firebird’s range. This increment and a future 3rd increment are supported by a time-phased requirement, originally documented in the approved Firebird I Capabilities Development Document (CDD). This increment of the program, dubbed “Firebird II,” is now being planned to meet this new requirement. This activity is supported by the Acquisition Strategy created in Firebird I’s development. The users have drafted the following requirement language: DRAFT REQUIREMENT FOR FIREBIRD II INCREASED RANGE Capabilities required: 1. Firebird II will have a range of 250/300 KM (threshold/objective). This is a Key Performance Parameter (KPP). 2. Firebird II must meet all unamended requirements in the CDD for the first increment. Using the draft requirement language as a guide, three alternative approaches for increasing range have been studied by Mitronix, a Federally Funded Research and Development Center (FFRDC). Their report is provided below. 57 Intermediate Systems Acquisition Course March 2020 FIREBIRD II UAV ALTERNATIVE APPROACHES FOR INCREASING RANGE Approach 1: Use new propulsion system to provide more range Any new propulsion system must be significantly more efficient to achieve the additional range. There are several possible commercial and Non-Developmental Item (NDI) solutions, but none can be easily integrated into the current air vehicle configuration due to compromises made with non-standard interfaces in the original design. A new engine will need extensive testing, both in the lab and in the air, but some of the data needed should be available through modeling and simulation, depending on the design chosen. All of these factors lead to moderate hardware risk. Some software will need to be rewritten for control of the propulsion system, but that should be a fairly straightforward, low risk effort. R&D costs are estimated at $180M (RDT&E). Production costs should be $410M (Procurement). Operations and Support (O&S - a combination of O&M and MILPERS appropriations) costs with this upgrade are estimated at $35M per year. Disposal cost is estimated at $70M. This effort should take 34 months from initiation to IOC. There is a moderate risk that the integration of the new engine will be more difficult than planned, requiring an additional $20M (RDT&E) and 3 more months to make IOC. Approach 2: Increase wing span and fuel capacity of air vehicle This approach will require redesign of a significant portion of the airframe. Increasing the range to 250-300KM will require nearly twice as much fuel capacity. Lengthening the wing span will provide room for more fuel, but will also add weight. Extensive flight testing will be necessary to ensure the new design meets all operational and safety requirements. Some of the required flight test data should be obtained from wind tunnel tests or modeling and simulation which will reduce the cost and time required for actual flight tests. Design techniques and production processes that will be used for this approach are relatively mature. Overall hardware risk is considered moderate. Flight control software will need to be modified, and portions may need to be completely rewritten. Past experience on Firebird indicates that flight control software complexity is often underestimated. For this approach, it is expected that very little software will be re-usable. The software risk for this approach is expected to be high. R&D costs are estimated at $200M (RDT&E). Most of that cost is due to the extensive software effort and risk mitigation required. Production costs are estimated at $350M (Procurement). Operations and support costs with this upgrade are estimated at $38.75M (O&S) per year. Disposal cost is estimated at $78M. This effort will take 34 months from initiation to IOC. There is a low risk that additional flight testing will be required, adding $30M RDT&E and 2 months to the schedule. Approach 3: Streamline design and decrease weight of air vehicle This approach will require a significant redesign of the air vehicle. The degree of technical difficulty in reducing weight will largely depend on how well the original designers incorporated weight-reduction elements in the current Firebird. It is expected that weight reduction will need to be supplemented with a more streamlined aerodynamic design, further complicating the 58 Intermediate Systems Acquisition Course March 2020 development. Anytime this is attempted, difficult technical tradeoffs must be made. These factors, combined with the expected need for special materials and production processes, indicate high hardware risk for this approach. In addition to the air vehicle redesign, the flight control software must be modified. However, much of the software should be re-usable, and the software effort can be minimized through computer-aided vehicle design, so the overall software risk for this approach is considered moderate. R&D costs are estimated at $220M (RDT&E), production costs at $395M (Procurement). Operations and support costs with this upgrade are estimated at $30M (O&S) per year. Disposal cost is estimated at $60M. This effort will take 36 months from initiation to IOC. For teams assigned Activity B (Increased Range), here is your tasking: 1. Choose a team leader/briefer. Use the information in the Mitronix report to build a matrix that shows cost, schedule and technical risks of each of the three approaches. 2. Preliminary discussions among the PM, PEO, service officials and other stakeholders indicate that money and time are tight, as usual, but the user has a valid need for the additional range. Taking these discussions into account, along with an affordability analysis and the urgency of the requirement, the PM provides the following guidance: “This range enhancement should not take longer than 36 months from initiation to IOC, the R&D affordability goal is $220M (RDT&E), and the Production affordability goal is $420M (Procurement). O&S costs should be no more than $35M (O&S) per year. Total Life Cycle Cost (LCC) should not exceed $1.41B. Also, cost, schedule and technical performance should be weighted equally when considering alternative approaches. Assume a 20-year operational life for the Firebird II UAVs”. Your team has been asked to help assess technology currently under development to determine applicability to the Firebird II. Assuming the draft requirement language will be approved as written, and considering the PM’s guidance: - Use your matrix to help you rank each of the three approaches based on the overall risk of meeting the new requirement. - Discuss and list any additional assumptions your group feels are necessary. For this academic exercise, do not create new approaches or combine elements of different approaches. 3. Prepare a 10-minute briefing to the class that: - Explains how you built your matrix. - Lists and explains any assumptions made by your team. - Lists and explains the rationale behind your approach rankings. 59 Intermediate Systems Acquisition Course March 2020 60 Intermediate Systems Acquisition Course March 2020 LESSON ASSIGNMENT SHEET Lesson Number Exercise 2.1 ______________________________________________________ Lesson Title Source Selection Planning ______________________________________________________ Lesson Time 2 hours ______________________________________________________ Terminal and Enabling Learning Objectives TLO Develop portions of a source selection plan, including source selection criteria ELO Identify how the Government communicates performance requirements in solicitations. ELO Identify the role of various IPT members in developing the solicitation. ELO Identify the purpose of evaluation criteria and how the criteria are developed. ELO Develop evaluation criteria in a source selection. ELO Identify methods of pre-solicitation communication with defense contractors. ______________________________________________________ Assignments Review the following ACQ 202 CBT Lesson Summary: • Lesson 3.1, Source Selection Process ______________________________________________________ Estimated Student Preparation Time 10 minutes ______________________________________________________ Assessment Class participation; oral presentation ______________________________________________________ Related Lessons • CBT Lesson 2.8, RFP Preparation, Part I • CBT Lesson 2.9, RFP Preparation, Part II • CBT Lesson 3.1, Source Selection ______________________________________________________ Self Study References FAR Part 15 ______________________________________________ 61  1974: Hughes Missile Systems was sole designer, developer & producer of Navy’s AIM-54, Phoenix Missile  Early 1980’s: Production/unit cost were approx. $1M  Mid 80’s: DoD developed 2nd source after full & open competition o Raytheon won 2nd source development contract  Late 80’s: Navy held limited competition o 1 contract to win minimum production quantity to maintain production line o 1 contract to win majority production quantity o Raytheon won majority; Hughes won minimum production quantity  1 year later, Navy held head-to-head and Hughes won. o Hughes’ production per unit cost was $499K o Competition drove cost down by just over 50% Why Competition is Important 62 Purpose:  Provide a structured, fair, impartial evaluation of offerors  Maximize competition, innovation  Select best source Source Selection Nominal Source Selection Process Source Selection Planning RFP to Industry Proposals From Industry Evaluation of Proposals Establish Competitive Range Final Proposal Revisions Source Selection Contract Award Hold Discussions Evaluation of FPRs Debriefings If Needed 63 Evaluates offerings IAW RFP Reviews SSEB Findings, compares offerors, and makes best value recommendation to SSA Selects best value offeror(s) SSEB SSAC COST TEAM TECHNICAL PAST PERFORMANCE SMALL BUSINESS (if needed) Required if equal to or more than $100M Purpose  Uniform procedures across DoD  Simplify source selection process  Require standardized rating criteria and descriptions for technical and past performance factors  Require appointment of SSAC on source selections valued over $100M DoD Source Selection Procedures, 1 April 2016 (No Foolin’) 64  Details all aspects of source selection process  Key elements: o Organization/Personnel o Conduct o Criteria for Proposal Evaluation  Prepared by Contracting Officer/IPT  Approved by SSA Source Selection Plan  Ensure technical requirements are approved and stable  Establish technical specifications  Develop SOW, SOO or PWS  Allocate Resources to support SSP  Assist in establishing SST  Assist in development of evaluation criteria PM/Rqmts Office Roles & Responsibilities 65  Market Research  Industry Day  Request For Information (RFI)  Draft Request For Proposal Pre-RFP Communication with Industry  Request For Proposal o SOO o PWS o SOW o System Spec o CDD (some organizations) Communicating Requirements 66  Meaningful discrimination among offerors  Examples: o Cost o Technical o Past Performance o Small Business Participation (if needed)  Tailored to the acquisition  Level of detail, number will vary (minimize) Evaluation Factors/Subfactors Factor:  Technical Subfactors:  Weapon Accuracy  Range  IED Protection Evaluation Factor/Subfactors Example 67  Choose carefully Will this factor help me select a winner by discriminating from the less capable offerors?  Use sparingly Evaluation Factors/Subfactors  Basis for contractor selection  Ensure contractor: o Can perform work o Understands requirement  Included in the RFP (Section M) Evaluation Factors/Subfactors (continued) 68  How is it determined?  Do we include in the RFP? Factor Relative Importance “Numerical or percent- age weight- ing of the relative importance of evalua- tion factors and sub- factors shall not be used.” Example “The Technical area is significantly more important than Cost, which is more important than Past Performance.” Factor Relative Importance 69  Developed for each factor/subfactor  Used to determine how well a proposal meets a factor/subfactor  Can use words and/or colors  Must be clearly defined & understood by SSEB  Not included in the RFP Evaluation Rating Guidelines Table 1. Combined Technical/Risk Ratings Color Rating Description Blue Outstanding Proposal indicates an exceptional approach and understanding  of the requirements and contains multiple strengths, and risk  of unsuccessful performance is low. Purple Good Proposal indicates a thorough approach and understanding of  the requirements and contains at least one strength, and risk  of unsuccessful performance is low to moderate. Green Acceptable Proposal meets requirements and indicates an adequate approach and understanding of the requirements, and risk of  unsuccessful performance is no worse than moderate. Yellow Marginal Proposal has not demonstrated an adequate approach and  understanding of the requirements, and/or risk of unsuccessful  performance is high. Red Unacceptable Proposal does not meet requirements of the solicitation, and  thus, contains one or more deficiencies, and/or risk of  unsuccessful performance is unacceptable. Proposal is  unawardable. Standardized Source Selection Evaluation Ratings 70  Evaluation Factors/Subfactors o e.g. Quality/Reliability  Factor/Subfactor Importance o e.g. Quality is more important than Cost  Evaluation Standard & Rating Example Source Selection Criteria Summary Subfactor Standard Rating MTBCF 180 hrs ≤ MTBCF    Blue 165 ≤ MTBCF < 180 Purple 150 ≤ MTBCF < 165 Green 135 ≤ MTBCF < 150 Yellow MTBCF < 135 hrs Red Best Value Continuum Greater GreaterLesser LesserImportance of Price Importance of non‐Price or Cost Factors FAR Part 15  Ba sis  fo r A w ar d “Best value” means the expected outcome of an acquisition that, in the Government’s  estimation, provides the greatest overall benefit in response to the requirement.” FAR 2.101  Lowest Price/ Technically Acceptable (LPTA) Non‐Price Factor(s):   Pass/Fail Only Price Factor:   Lowest Price No Tradeoff Lowest Price that is  Technically Acceptable  [“Pass” on all non‐ price factors]. Value Adjusted Total  Evaluated Price  (VATEP) Bidders receive a price  evaluation credit for  achieving defined,  (preferably measurable)  performance beyond  threshold values Tradeoff Limited to subjective non‐ VATEP evaluated factors  only. Must document basis  for award to other than  lowest evaluated price. Combination  Approach ‐Some factors pass/fail ‐Other factors used in tradeoff For example‐‐ Technical:  Pass/Fail; Tradeoff only allowed  between Past Perf. and Price Tradeoff Limited to NON‐pass/fail  factors only.  Must document  basis for award to other than  lowest price or other than  highest rated on non‐price  factors. Subjective  Tradeoff Can award to offer that is  other than lowest price or  other than highest rated on  non‐price factors.  Tradeoff  is among price and non‐price  factors. Tradeoff All factors in tradeoff.  Must  document basis for award to  other than lowest price or  other than highest rated on  non‐price factors. 71 Intermediate Systems Acquisition Course March 2020 Background: The Army Acquisition Executive has granted Milestone A approval for Firebird II. A Program Management Office (PMO) has been fully staffed to support this program; you and your IPT have been assigned to carryout the effort. We are now in the Technology Maturation and Risk Reduction phase. In accordance with the acquisition strategy, contracts were awarded through full and open competition to two contractors to develop competitive prototypes for the Firebird II. The two contractors will compete with their prototypes in developmental testing and a fly-off just before Milestone B. The fly-off will be followed by a down-select to one contractor. The competitors will be given a final Request For Proposals (RFP) immediately following the Development RFP Release Decision (DRFPRD). The successful competitor will then execute the program through the Engineering and Manufacturing Development phase. A team has been assembled to write a Source Selection Plan (SSP) including the draft RFP addressing the down-select. This exercise will focus on that SSP. One of the factors to be evaluated at the down-select is technical performance. The Firebird II source selection team has already developed technical subfactors based on the draft CDD and a performance work statement. Those subfactors are: range, launch, survivability, Mean Time to Repair (MTTR), weapons accuracy and Mean Time Between Critical failure (MTBCF). Assignment: 1. For each of the six technical subfactors, determine their relative importance to each other. Remember that subfactors can have equal importance, or one subfactor can be more important, slightly more important, or significantly more important than another subfactor. The user has determined that survivability is the most important requirement for Firebird II. 2. Your instructor will assign one or more of the subfactors (on the following pages) to your team for analysis. Develop a clear set of standards for evaluating the subfactor(s) assigned to your team, using a rating system where: 72 Intermediate Systems Acquisition Course March 2020 For your subfactor, determine the standard based on the rating definitions above. The subfactors are based on the draft CDD parameters (shown on the next page), so refer to the Firebird II CDD as necessary. Use the subfactor table example to display your results. Be prepared to present your standards to the class. Note that numerical standards are only one portion of subfactor evaluation. The source selection plan will also address how to evaluate risk at for each subfactor to arrive at a final color rating. Also note that red is considered unawardable. In other words the performance is so bad that you would not consider awarding to that contractor. Use a table to depict your answer. Subfactor Standard Rating Blue Purple Green Yellow Red Threshold____________ Objective__________________ 73 Intermediate Systems Acquisition Course March 2020 DRAFT FIREBIRD II CDD PARAMETERS NOTE: These values come from the draft CDD. KPPs and other requirements should be proven through testing prior to the Milestone C decision. 1. Range (KPP): Threshold: The vehicle must be able to fly out to 250 KM and return to base. Objective: The vehicle must be able to fly out to 300 KM and return to base. 2. Launch: Threshold: The UAV must launch from a stationary mobile launcher unit and be safely airborne within a distance of 30 feet. Objective: The UAV must launch from a stationary mobile launcher unit and be safely airborne within a distance of 25 feet. 3. Survivability (KPP): Threshold/Objective: The UAV must have a probability of survival against shoulder- launched heat-seeking missiles of at least 90%. 4. Mean Time to Repair (MTTR) for the UAV must be no more than: Threshold: 3 hours Objective: 2.5 hours 5. Weapons Accuracy: (KPP) Threshold: 10M Circular Error Probable (CEP) Objective: 5 M Circular Error Probable (CEP) Note: CEP, the circular error of probability, refers to the radius around the target within which the munitions must fall 50% of the time. 6. Mean Time Between Critical Failure (MTBCF) for the UAV must be no less than: Threshold: 150 hours Objective: 200 hours 74 Intermediate Systems Acquisition Course March 2020 LESSON ASSIGNMENT SHEET Lesson Number Exercise 2.2 ______________________________________________________ Lesson Title Systems Engineering ______________________________________________________ Lesson Time 1 hour ______________________________________________________ Terminal and Enabling Learning Objectives TLO Apply the iterative SE steps to develop outputs of the systems engineering process in order to verify they meet a given requirement ELO Given a summary Capability Development Document (CDD) and a system concept, determine whether the concept addresses all user requirements. ELO Identify the overall purpose of the systems engineering process ELO Identify the technical processes that make up the overall systems engineering process ELO Identify the technical management processes used to control and manage the overall systems engineering process ELO Identify the main inputs and outputs of the overall systems engineering process ELO Given an acquisition scenario within an IPT environment, develop and present selected outputs of the systems engineering process steps. ______________________________________________________ Assignments Review the following ACQ 202 CBT Lesson Summary: • Lesson 3.2, Technical Risk Management • Lesson 3.3 Design for Supportability, Trade Off Analysis • Lesson 3.4 Software Design • Lesson 4.5 Reviews, Simulations and Test • Lesson 4.9 Operational and Live Fire Testing ______________________________________________________ Estimated Student Preparation Time 45 minutes ______________________________________________________ Assessment Class participation; oral presentation ______________________________________________________ 75 Intermediate Systems Acquisition Course March 2020 Related Lessons CBT Lessons 3.2, Technical Risk Management, 3.3 Design for Supportability, Trade Off Analysis, 3.4 Software Design, 4.5 Reviews, Simulations and Test, 4.9 Operational and Live Fire Testing Classroom Exercise 1.3 Materiel Solution Analysis Classroom Exercise 2.3, Test Planning Classroom Exercise 2.1 Source Selection Planning Classroom Exercise 3.1 Source Selection Process ______________________________________________________ Self Study References • ACQ 101 Lesson 17, Systems Engineering Process • Defense Acquisition Guidebook, Chapter 4, Systems Engineering ______________________________________________ 76 “Systems engineering (SE) establishes the technical framework for delivering materiel capabilities to the warfighter. SE activities begin before a program is officially established and are applied throughout the acquisition life cycle. Any effective SE approach should support and be integrated with sound program management.” Standardized SE Terminology Defense Acq Guidebook (DAG) Chapter 3-2 Technical Processes  Stakeholder Requirements Definition  Requirements Analysis  Architecture Design  Implementation  Integration  Verification  Validation  Transition Disciplined application of SE processes enables • sound decision making • increased product knowledge and system maturity • risk reduction Technical Management Processes  Decision Analysis  Technical Planning  Technical Assessment  Requirements Management  Risk Management  Configuration Management  Technical Data Management  Interface Management • This triangle is a convenient way to illustrate the Systems Engineering Process as described in Chapter 3 of the Defense Acquisition Guidebook (DAG). • SE processes are not meant to be performed in a particular time-dependent or serial sequence. Apply the processes iteratively, recursively and in parallel (as applicable) throughout the life cycle. o The Systems Engineer is responsible for developing the plan and applying the SE processes across the program, monitoring execution throughout the life cycle, and taking necessary steps to improve process efficiency and effectiveness. o The Program Manager and Systems Engineer should apply appropriate resources with requisite skill sets to ensure successful execution of each process. Systems Engineering Process 77 Systems Engineering Processes The eight Technical Management Processes form a “tool kit” used to help manage, control, and provide balance in the execution of the eight Technical Processes Firebird II will incorporate improved survivability measures such that the probability of survivability during a single engagement by a shoulder-launched heat-seeking missile is greater than or equal to 90%. Firebird II Survivability Requirement The warfighter is considering an increase in the survivability requirement to greater than or equal to 92%. 78 Intermediate Systems Acquisition Course March 2020 Exercise 2.2 Systems Engineering Introduction: This exercise has been designed to give you “hands on” experience in exercising portions of the systems engineering and technical management process. Before beginning the exercise, there will be a short review of the systems engineering process. Background: Flyin-Hyer and CyboRaptor, each with significant UAV experience, proposed different technical solutions. The warfighter has proposed a further increase in the survivability requirement. The Program Manager wants your IPT to evaluate CyboRaptor’s technical solution for enhancing survivability to be sure that their physical architecture is affordable, technically feasible, and traceable to the user’s increased requirement, which is to be specified in the CDD as follows: Firebird losses due to shoulder-launched missiles are much higher than planned, exceeding the ability of support systems to sustain the system. This has resulted in unacceptably low operational availability and unplanned costs… Firebird II will incorporate improved survivability measures such that the expected loss rate from heat-seeking shoulder-launched missiles is no greater than 8% (threshold)/ (objective). CyboRaptor proposes to upgrade the existing engine to extend the range of the air vehicle and increase survivability by adding self-defense enhancements. The air vehicle will be equipped with a sensor to detect incoming missiles. It will release flares as decoys to draw heat-seeking missiles away from the vehicle. It will also be equipped with software upgrades to give Firebird II greater maneuvering capability, further reducing its vulnerability to missile attacks. In order to meet the warfighter’s increased requirement, CyboRaptor will add a laser to jam the guidance system of approaching missiles so they cannot engage the air vehicle. The Program Manager wants you to use the systems engineering process to analyze the requirement for enhanced survivability and evaluate CyboRaptor’s technical solution to meet that requirement. First you will perform Stakeholder Requirements Definition to determine what the Firebird II system must be able to do, how well, and under what conditions/constraints. You will then use Requirements Analysis to determine what functions must be performed and define a functional architecture for the system. Finally, using Architecture Design, you will evaluate the technical solution (physical architecture) proposed by CyboRaptor to accomplish the functions identified during Functional Analysis, and verify that it will satisfy the functional architecture and system requirements. 79 Intermediate Systems Acquisition Course March 2020 Assignment: The PM has asked your IPT to conduct a trade-off analysis for the increased CDD requirement, using CyboRaptor’s proposed technical solution for Firebird II survivability. He wants you to use the Systems Engineering Process as a tool to verify that the proposed technical solution will meet the requirement, and at what cost, in terms of time, money, and risk. Step 1: Stakeholder Requirements Definition Your instructor will identify the system requirements derived from the CDD requirement to enhance survivability. Working with your team, list at least 5 questions you would need to ask the user to clarify the requirement before you can evaluate the proposed system concept. Step 2: Requirements Analysis Based on the requirements identified above, your instructor will identify the system level functions that will need to be performed in order to enhance survivability of the air vehicle. Analyze each system-level function to break it down one level into sub-functions. After you complete your analysis of functions, the instructor will discuss with you how to allocate the requirements identified in Step 1 to the functions identified in Step 2. Step 3: Architecture Design Solution Based on your requirements analysis, compare CyboRaptor’s proposed physical architecture for enhancing survivability of the air vehicle (see the next page) to the functional architecture that you developed in the step above. Don’t analyze the entire Firebird system; just use the shaded blocks under “Self Defense System.” Your instructor will provide a matrix to guide you in comparing functions to physical components. Are any required functions for the air vehicle not being accomplished by an element in the proposed physical architecture? Are any hardware or software elements in the proposed physical architecture not accomplishing a required function? Based on your comparison, what would you recommend to the warfighter regarding their proposed higher survivability requirement? 80 Intermediate Systems Acquisition Course March 2020 CyboRaptor Technical Solution CyboRaptor’s technical solutions for Firebird II is to increase survivability by adding self- defense enhancements. The air vehicle will be equipped with an electronic sensor to enable it to detect the threat of approaching missiles. Additional software upgrades will provide Firebird II with greater maneuvering capability, further reducing its vulnerability to missile attacks and increasing the air vehicle survivability rate. It will also be equipped with flares, which will serve as decoys to draw heat-seeking missiles away from the air vehicle. In order to meet the higher survivability, a laser is being proposed to jam the guidance system of approaching missiles. CyboRaptor is also incorporating an upgraded engine to give Firebird II the extended range required. The physical architecture for CyboRaptor’s technical solution is shown below in Figure 1. This architecture will be further expanded and modified during the Engineering and Manufacturing Development Phase. CyboRaptor Firebird II Physical Architecture (Figure 1) 81 Intermediate Systems Acquisition Course March 2020 Note: Flyin-Hyer proposes to replace the existing engine with a new technology engine. The new engine will both extend the range of the air vehicle and increase its survivability by allowing Firebird to fly beyond the threat envelope of shoulder fired munitions. The Program Manager has reviewed Flyin-Hyer’s approach and is confident that their technical soltution, based on using a new engine, has potential to meet both range and survivability requirements. Flyin-Hyer has not yet responded to the government’s request for a proposed solution to the warfighter’s increased survivability requirement. You do not need to evaluate their technical solution; however it is provided for your information on the following page. You may need to refer to this information in the upcoming exercise on Technical Performance Measures. Flyin-Hyer Technical Solution Flyin-Hyer’s proposed technical solution is to increase survivability using upgrades to Firebird to allow it to fly outside the threat envelope of shoulder fired munitions (>18,000 ft). A new
technology engine will enable Firebird II to fly above 18,000 ft. The new engine also provides
the UAV with extended range capability. An upgraded infrared (IR) camera with high-
resolution optics will be added for enhanced night vision capability. Figure 2 below illustrates
Flyin-Hyer’s physical architecture for their technical solution. This architecture will be further
expanded and modified during the Engineering and Manufacturing Development Phase.
NOTE: This technical solution has already been evaluated and approved by the Program
Manager.
Flyin-Hyer Firebird II Physical Architecture
(Figure 2)
82

• What is the system supposed to do?
• Where will the products of the system be used?
• Under what conditions will the products be used?
• How often? How long?
• Who will use the products of the system?
Stakeholder Requirements Definition
 Analyze functions
 Decompose higher level functions to lower level
functions
 Allocate performance requirements to the
functions
Requirements Analysis
This step
answers
the question:
“HOW?”
Using
“Action
Verbs”
83

A feedback loop to ensure that:
 All requirements are covered
by at least one function
 All functions are justified by
a valid requirement (no
unnecessary duplication)
Requirements Traceability
Requirements
Management
(technical
management
process) is
key to the
control
and trace-
ability of
requirements
throughout
the design,
development
and fielding
of a system.
Defines the physical architecture:
 Each part must perform at
least one function
 Some parts may perform
more than one function
Architecture Design
What performs
the function(s)?
NOUNS are used
to describe
hardware
and/or software
elements of the
design
84

A feedback loop to ensure that:
 All functions are covered by at least one hardware
or software element
 All elements of the physical architecture are
justified by a valid functional requirement (no
unnecessary duplication)
Functional/Physical
Architecture Crosswalk
• Determines how the elements of the
design will be carried out
o Will software or hardware be used?
o Will it involve a new design (hardware
or software)?
o Can components (hardware or
software elements) be reused?
o Are COTS products feasible/available?
Implementation
Developing
supporting
documentation,
such as inter-
face require-
ments, opera-
tions and
maintenance
manuals, and/
or installation
instructions,
is a key part
of this SE
process step.
85

Integration
 Incorporates lower level system elements
into a higher level system element in the
physical architecture
o Involves linkage of hardware and software
elements
o Analogous to the process of “rolling up” lower
level Work Breakdown Structure (WBS)
elements into the next higher (subsystem or
system) level
Interface
management
(one of the SE
technical
management
processes) is
especially
important to
this step of the
SE Process.
 Each requirement must be verifiable
 The Verification Process ensures that the solution meets
the specified (specification) requirements
 The Validation Process ensures that the solution meets
the user’s needs
 “Verification” can be accomplished by:
– Inspection – Analysis
– Demonstration – Test
Verification and Validation
In the triangle
Model for
Systems
Engineering,
“Verification”
consists of
two separate
processes,
Verification
(Development
al Testing, or
DT) and
Validation
(Operational
Testing, or OT)
86

 Process of moving one element in the physical
architecture to the next higher level e.g. component
to system.
o For the end item (system) this is the process which
fields the system to the user in the operational
environment
o May require integration of the end item with other
systems via the defined external interfaces
Transition
3 Successive Iterations of the
Configuration Management Baseline
Systems Engineering: An Iterative*
and Recursive** Process
Functional Baseline (SFR)
Inputs Outputs
System Level
Allocated Baseline (PDR)
Inputs Outputs
Subsystem Level
(Configuration Item Performance
Specifications)
Initial Products
Baseline (CDR)
Inputs Outputs
Detail Design Level
(including processes and materials)
*Iterative:
overall SE process
is repeated
multiple times
over the life cycle;
technical
processes are also
repeated as
necessary with
feedback loops to
earlier processes
**Recursive:
the SE technical
processes are
applied at each
(successively
lower) level of
systems
development
(i.e., system-
subsystem-
module-
component)
87

The objective of the SE process is to develop, produce,
test and field a solution that meets user needs.
Systems Engineering Process
88

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 2.3
______________________________________________________
Lesson Title Test Planning
______________________________________________________
Lesson Time 1.0 hour
______________________________________________________
Terminal and Enabling Learning Objective
TLO
Given a program schedule, explain the role of test and evaluation
(DT&E, OT&E, LFT&E) in the systems engineering and acquisition
management processes.
ELO Identify the characteristics and purposes of Developmental Test and Evaluation (DT&E)
ELO Identify the characteristics and purposes of Operational Test and Evaluation (OT&E)
ELO Identify the characteristics and purposes of Live Fire Test and Evaluation (LFT&E)
ELO Given a test event description, correctly identify the type of testing being accomplished
ELO Given a program schedule, correctly identify opportunities for combined DT/OT
ELO Identify the risks and benefits associated with combining DT and OT events
______________________________________________________
Assignments Review the following ACQ 202 CBT Lesson Summaries:
• Lesson 3.2, Technical Risk Management
• Lesson 2.5, Developing the TEMP
• Lesson 4.5, Reviews, Simulations and Tests
• Lesson 4.9, Operational/Live-Fire Testing
______________________________________________________
Estimated Student
Preparation Time 30 minutes
______________________________________________________
Assessment Class participation; oral presentation
__________________ ____________________________________
89

Intermediate Systems Acquisition Course March 2020
Related Lessons CBT Lesson 3.2, Technical Risk Management
CBT Lesson 2.5, Developing the TEMP
CBT Lessons 4.5, Reviews, Simulations and Tests
CBT Lesson 4.9, Operational/Live-Fire Testing
Classroom Exercise 1.4 Materiel Solution Analysis
Classroom Exercise 2.1 Source Selection Planning
Classroom Exercise 3.1 Source Selection Process
______________________________________________________

Self Study
References
N/A
______________________________________________

Interoperability certification testing by DISA /JITC is part
of DT&E and OA prior to MS C as well as IOT&E after MS C.
LFT&E is also accomplished in both OT&E and DT&E.

90

Typical DT & OT Testing by Phase
A CB
LRIP
Technology
Maturation &
Risk Reduction
Production &
Deployment
DRFPRD
Materiel
Solution
Analysis
CDD-V
CDD
ICD DraftCDD
Draft
CDD
Draft
CDD
Operations &
SupportMateriel
Development
Decision
FRP
Decision
Sustainment
DisposalFOC
Engineering &
Manufacturing
Development
CDD
Developmental
Testing
Operational
Testing
TRRTRR
DT&E
OA FOT&EEOA IOT&E
EOA
TRRTRR
OTRROTRRTRRTRR
IOC
Effectiveness and 
Suitability 
Assessment
Effective / 
Suitable
LRIP items 
usually used 
for IOT&E
Update if
Needed
 An integral part of the Systems Engineering process
(Verification)
 Assesses component and system performance against
system specifications
 Equipment is usually operated by contractors/engineers
in a controlled environment
 Overseen by the Deputy Assistant Secretary of Defense
for DT&E and the Program Office
 Conducted by the contractor and the service
developmental test agencies (Army Evaluation Center
[AEC], AFMC, Navy Systems Commands, MARCORSYSCOM)
Developmental Test & Evaluation
(DT&E)
91

 Environmental Effects Testing
 Captive Engine Tests
 Wind Tunnel Testing
 Component Reliability Testing
 Blue Team Cyber Vulnerability Assessments
 Materials Testing (hardness, corrosion resistance etc.)
 Hardware in the Loop
DT&E Examples
 In the context of Systems Engineering determines
operational effectiveness and suitability
(Validation)
 Assesses the system performance against the
users requirements as stated in the capability
documents
 Equipment is operated by warfighters in an
operational environment
 Overseen by the OSD Director, Operational Test &
Evaluation (DOT&E)
 Conducted by the service operational test
agencies (ATEC-OTC, AFOTEC, COMOPTEVFOR &
MCOTEA)
Operational Test & Evaluation
92

Early Operational Assessment 
(EOA)
Performed on prototypes to help decision 
makers assess the proposed concepts.
Operational Assessment (OA) Conducted during the EMD Phase to assess 
the system’s potential to meet mission 
requirements. Supports a Low Rate Initial 
Production (LRIP) decision.
Initial Operational Test and 
Evaluation (IOT&E)
Conducted on production or production 
representative articles to support a Full Rate 
Production Decision Review.
Follow‐on Operational Test and 
Evaluation (FOT&E)
Conducted after the system is in production 
and may continue throughout the lifecycle.
Types of OT&E
• Weapons Accuracy and Lethality
• Communications Effectiveness
• Mission Effectiveness (Many Possible Dimensions)
• System and Weapons Operational Range
• Red Team Cyber Penetration
• Recovery and Repair Procedures
• System Reliability
Examples of What is Tested in OT&E
93

 LFT&E assesses 2 major dimensions, Survivability
and Lethality
 Covered Systems – LFT&E is a statutory
requirement for systems that are covered under
the law, these include:
o Any major system that provides some degree of protection
to its occupants in combat.
o Any major conventional munitions or missile program; or
one that will acquire 1,000,000 rounds or more.
o A modification to a covered system that is likely to affect
significantly the survivability or lethality of such a system.
 A waiver from full up system LFT&E must be
approved at Milestone B.
Live Fire Test & Evaluation (LFT&E)
Early
 Component testing
 Lethality Effects
 Strength of System Materials
Under Fire
Mature System
 Full Up System Live Fire
Testing
 Ship Level Shock Test
 Aircraft Crew Survivability
 Vehicle Crew Survivability
 Missile Lethality
LFT&E Examples
94

 Integrated DT/OT testing is an expected best
practice within the services and DoD
 Benefits –
o Schedule and cost savings through better use of test
resources/data
o Early identification of operational issues (before IOT&E)
o Early Warfighter feedback to influence design
 Risks/Issues –
o Independent testers see your system when it is immature
o Independent OT agency may not want contractors
operating equipment
o DT/OT environments are often not the same
Integrated Testing
 Each requirement must be verifiable
 The Verification Process ensures that the solution meets
the specified (specification) requirements
 The Validation Process ensures that the solution meets
the user’s needs
 “Verification” can be accomplished by:
– Inspection – Analysis
– Demonstration – Test
Verification and Validation
In the triangle
Model for
Systems
Engineering,
“Verification”
consists of
two separate
processes,
Verification
(Development
al Testing, or
DT) and
Validation
(Operational
Testing, or OT)
95

Intermediate Systems Acquisition Course March 2020
Exercise 2.3 Test Planning
Background:
The Firebird II program office is busily putting the final touches on our updated Test and
Evaluation Master Plan (TEMP) for the DRFPRD. The Program Manager wants to ensure that
we have robust testing of Firebird II to support program decisions in EMD. He also wants to
make sure we have integrated developmental and operational testing where it makes sense on the
schedule. In this exercise, your team will be asked to help evaluate Firebird II test planning.
For both technical approaches Firebird II will incorporate the same weapons and support
equipment that were used in the first increment.
Assignment:
1. For the test events described below come to a team consensus on whether they are DT&E,
OT&E, or integrated DT/OT. Be prepared to discuss your answer with the class.
a) The FB II air vehicle will be put through several tests using the wind tunnels at Arnold
Engineering Development Complex in Tullahoma, TN.
b) Soldiers will conduct several missions over 3 days using an integrated FB II system at
White Sands, NM.
c) The FB II will drop a laser-guided bomb on a test range with a hardened infantry
emplacement as the target. The target will be instrumented.
d) The FB II operators and maintainers will run through several repair and scheduled
maintenance procedures on the integrated system.
e) Thermal imaging of the FB II flare deployment system (Cyboraptor Concept) will be
conducted in a laboratory.
f) The FB II engine will be bench tested to obtain reliability and fuel efficiency data.
2. Identify opportunities for integrated test events (DT/OT) during the EMD phase on the FB II
schedule. Be prepared to discuss your conclusions with the class:
3. In your opinion, is LFT&E applicable to Firebird II? Why or why not? Who would make the
final determination?
96

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 2.4
______________________________________________________
Lesson Title Technical Performance Measures
______________________________________________________
Lesson Time .5 hour
______________________________________________________
Terminal and Enabling Learning Objectives
TLO
Analyze actual versus planned technical performance data in risk
areas to indicate potential problems that may prevent a system from
being operationally effective and suitable.
ELO Identify potential risk areas based on technical performance data
ELO Identify the role of technical performance measures in the systems engineering process.
______________________________________________________
Assignments Review the following ACQ202 CBT Lesson Summaries:
• Lesson 3.2, Technical Risk Management
• Lesson 2.5, Developing the TEMP
• Lesson 4.5, Reviews, Simulations and Tests
• Lesson 4.9, Operational/Live Fire Testing
______________________________________________________
Estimated Student
Preparation Time 45 minutes
______________________________________________________
Assessment Class participation; oral presentation
_______________________________________________________
Related Lessons CBT Lesson 3.2, Technical Risk Management
CBT Lesson 2.5, Developing the TEMP
CBT Lessons 4.5, Reviews, Simulations and Tests
CBT Lesson 4.9, Operational/Live Fire Testing
Classroom Exercise 1.4 Materiel Solution Analysis
Classroom Exercise 2.1 Source Selection Planning
Classroom Exercise 3.1 Source Selection Process
______________________________________________________
Self Study
References N/A
97

Analyze technical data to identify risks and
ensure a system will be operationally effective
and suitable.
Ex. 2.4 Learning Objective
TPMs are used to track
Progress over Time
TPMs in the Systems Engineering
Process
The eight
Technical
Management
Processes
form a “tool
kit” used to
help manage,
control, and
provide
balance in the
execution of
the eight
Technical
Processes
98

Intermediate Systems Acquisition Course March 2020
Exercise 2.4 Technical Performance Measures

Background:
CyboRaptor and Flyin-Hyer intend to upgrade or replace the Firebird engine in order to achieve
extended range. For the past six months, the contractors have been perfecting their engines and
conducting test flights to collect data on range as well as other parameters.

During developmental testing, contractors use critical technical parameters (CTPs), derived from
the CDD, to determine whether thresholds and objectives are being met. The Program
Management Office can monitor attainment of these CTPs by using technical performance
measurement (TPM) data reported by the contractors. Recall that TPMs compare the actual
values obtained during test flights against planned or expected values over time.

The raw flight test data received to date from each contractor is summarized at Figure 1.
This data is plotted on TPM charts at Figure 2 for CyboRaptor and at Figure 3 for Flyin-Hyer.

Assignment:
Analyze the test data provided by the two contractors to determine if the technical solution will
meet the user’s requirements. What can you conclude about the probability of each contractor
achieving the required range? What concerns, if any, do you have with each contractor? Be
prepared to discuss the risks associated with each contractor’s technical solution.
Figure 1
CyboRaptor Test Flight Data

Test Flight
Numbers
Average Distance
Range @ Radius
Time of
Test Flights
1 – 10 150 km SEP
11 – 30 180 km OCT – NOV
31 – 75 200 km DEC
76 – 100 240 km JAN – FEB
101 -110 240 km MAR – APR

Flyin-Hyer Flight Test Data

Test Flight
Numbers
Average Distance
Range @ Radius
Time of Test
Flights
1 – 18 150 km SEP
19 – 35 200 km OCT – DEC
36 – 60 300 km JAN – MAR

99

Technical Performance Measurement-
Range
CyboRaptor
KPP: Range
Objective
Threshold
SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG
(#11‐30)
(#31‐75)
(#76‐100)
(#101‐110) 
Achieved to
Date
Planned
Profile
(#1‐10)
SEP OCT
100
150
200
250
300 KM
Technical Performance Measurement-
Range
Objective
Threshold
SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG SEP OCT
100
150
200
250
300
Flyin – Hyer
KPP: Range
(#1 – 18)
(#19 – 35)
(#36 – 60)
Achieved
to date
Planned
Profile
350 KM
100

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 2.5
______________________________________________________
Lesson Title Contractor Planning, Scheduling and Resourcing
______________________________________________________
Lesson Time 1.5 hours
______________________________________________________
Terminal and Enabling Learning Objectives
TLO
Given a segment of contract work and associated tasks, plan and
schedule the tasks and resources necessary to complete contract work
within cost and schedule constraints.
ELO Apply the fully burdened rate to labor hours to correctly calculate contractor’s costs
ELO Distinguish correctly between direct and indirect costs on a contract
ELO Given a simple Gantt chart with defined task relationships, identify the critical path
ELO Given a completed Gantt chart with the critical path identified, identify cost and schedule risks in the plan
ELO Given a completed Gantt chart with the critical path identified, explain cost and schedule risks in the plan
______________________________________________________
Assignments Review the following ACQ202 CBT Lesson Summaries:
• Systems Engineering Exercise 2.3
• Source Selection Process – CBT lesson 3.1
• Technical Risk Management – CBT lesson 3.2
• Earned Value Management – CBT lesson 3.7
• Integrated Baseline Review – CBT lesson 4.7
______________________________________________________
Estimated Student
Preparation Time 45 minutes
______________________________________________________
Assessment Class participation; multiple choice exam
_______________________________________________________
101

Intermediate Systems Acquisition Course March 2020
Related Lessons CBT Lesson 2.3 – Systems Engineering
CBT Lesson 3.1 – Source Selection Process
CBT Lesson 3.2 – Technical Risk Management
CBT Lesson 3.7 – Earned Value
CBT Lesson 4.7 – Integrated Baseline Review
Classroom Exercise 2.1 Source Selection Planning
Classroom Exercise 2.2 Systems Engineering
Classroom Exercise 2.4 Technical Performance Measures

______________________________________________________

Self Study
References N/A
______________________________________________

102

Control Account
is the level at
which work,
schedule, and
budget come
together
Contract Work Breakdown Structure/
Organization Breakdown Structure
Integration
Control
Account
Software
Engineering
Work
Packages
Planning
Packages
Marketing
FUNCTIONAL
ORGANIZATION
SELECTED
WBS
ELEMENTS
Hardware
Engineering
Engineering
Operations
VP/GM
Defines
Product
Defines Work
Defines
Responsibility
Firebird
II
1.1
Air
Vehicle
1.2
Ground Control
Terminal
1.3
Launcher
1.1.1
Weapons
Delivery
System
1.1.2
Air
Frame
1.1.3
Engine
1.1.4
Command
& Control
System
1.1.5
Self
Defense
System
1.5
System Test
& Evaluation
Control
Account
Control
Account
Control Account
A Key Management Control Point
 Responsible for allocating resources and planning
schedules to accomplish the tasks within a control
account (associated with an element of the work
breakdown structure)
 Work/planning packages the CAM develops within their
control account become part of the Integrated Master
Schedule (IMS) and the Performance Measurement
Baseline (PMB)
Control Account Manager (CAM)
103

 Direct Costs – A cost that can be tracked directly to one
contract or other unit of work (cost objective) for which
an accounting system accumulates and measures costs.
o Examples: touch labor, purchased parts, computer time
 Indirect Costs – A cost identified with two or more
contracts (cost objectives), but not identifiable directly
to a single contract. Indirect costs must be captured in
appropriate discrete cost pools. Overhead and General &
Administrative (G&A) are commonly used indirect cost
pools.
o Overhead: Indirect costs that support a specific part or function of
the company but not the entire company.
• Examples: Factory maintenance, material handling
o General and Administrative: Indirect costs incurred or allocated to a
business unit for the general management and administration of the
business unit as a whole.
• Examples: Senior management salary, independent research
and development
Contractor Costs
Contractor Costs Example
(Fully Burdened Rate)
Category Rate Cost Note
Salary and Benefits $180,000 Direct Cost
Engineering O/H 1.44 $259,200 Direct Cost X Eng. O/H Rate.
Indirect Cost.
G&A 0.11 $48,312 (Direct Cost + Eng. O/H ) X G&A
Rate. Indirect Cost.
Total Indirect (O/H + G&A) $307,512 ($259,200 + $48,312)
Total Cost (Direct +
Indirect)
$487,512 ($180,000 + $307,512)
$259,200
$48,312
$180,000
Overhead
G&A
Direct
104

 The critical path is the sequence of activities that
determine the length of a project
 Generally, if any task on the critical path increases in
length, the project also increases in length
Critical Path
Project X
1. System Design
2. Acquire Raw Materials
3. Acquire Components
4. Prototype Fabrication
5. Prototype Test
Tasks 0 5 10 15 20 25 Days
105

Intermediate Systems Acquisition Course March 2020

Exercise 2.5 Contractor Planning,
Scheduling and Resourcing

After consulting his technical staff,
Cyboraptor’s PM decided to address the range
problem by adding a turbocharger to the engine
to increase power and fuel efficiency. This will
be part of Cyboraptor’s proposal in response to
the Firebird II final RFP for EMD
developmental effort. The Cyboraptor team
must now plan out the top level tasks to show
that the work can be done with the funds
available, within a tight schedule constraint and
at an acceptable level of risk.

You are now the contractor team at Cyboraptor
that will develop the schedule and budget for
this part of the program. Cyboraptor’s PM gave
you the following top level constraints:

– The work must be completed in 45 work days.
– The budget goal for this work is $270K.
– Schedule is more important than cost because
this work is on the critical path of the program
and necessary for a major developmental test
for which the flight test range has already been
scheduled.
– Materials risk is based on the quality of the
materials which will most affect performance

Cyboraptor engineers developed the following
11 tasks with associated time and resources.

Task Sequencing Time and Labor/material costs
1) Design
Turbocharger
Starts immediately 12 Days with 6 engineers, 14 days with
5 engineers, 16 days with 4 engineers
or 20 days with 3 engineers
2) Acquire GFE Airflow
Mass Sensor
Starts when task
#1 is complete
16 days, no cost to contractor
3) Acquire
Turbocharger
Materials
Starts when task
#1 is complete
6 days at a cost of: low risk-$30K,
moderate risk-$25K or high risk-$20K
4) Turbocharger
Fabrication
Starts when task
#3 is complete
8 days with 2 machinists
5) Integrate
Turbocharger and
Engine
Starts when task
#2 and #4 are both
complete
4 days with 1 engineer and 2 machinists
6) Test Integrated
Turbocharger and
engine
Starts when task
#5 is complete
6 days with 3 engineers
7) Design, modeling
and virtual
prototyping of
airframe modification
Starts when task
#1 is 50%
complete
10 days with 2 engineers
8) Purchase Airframe
Modification
Materials
Starts when task
#7 is complete
5 days at a materials cost of $25K
9) Modify prototype
airframe
Starts when task
#8 is complete
10 days with 4 machinists, 12 days with
3 machinists or 14 days with 2
machinists
10) Wind tunnel test
airframe
Starts when task
#9 is complete
6 days with 3 Engineers
11) Integrate and test
engine and airframe
Starts when tasks
#6 & #10 are
complete.
4 days with 1 engineer and 2 machinists

106

Intermediate Systems Acquisition Course March 2020

107

Intermediate Systems Acquisition Course March 2020
Engineers cost $500 per day (Direct Labor) X 2.4 (includes overhead and G&A) = $1200 per day
fully burdened rate.
Machinists cost $300 per day (Direct Labor) X 2.2 (includes overhead, benefits etc.) = $660 per
day fully burdened rate.
Add a 20% materials overhead (shipping, handling, storage etc.) to all materials costs to get the
full cost.
The materials risk for task 3 is in terms of performance.
Assignment:
1) Using the blank Gantt chart provided, develop a schedule for this work package with all 11
tasks sequenced meeting the PM’s cost and schedule constraints at an acceptable level of risk.
2) Determine your critical path.
3) What is the biggest risk in this plan and how you would mitigate it?
4) Will Cyboraptor’s proposal for the post MS B contract have to change based on this effort?
5) Be prepared to discuss your results with the class.
108

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 3.1
______________________________________________________
Lesson Title Source Selection Process
______________________________________________________
Lesson Time 1 hour
______________________________________________________
Terminal and Enabling Learning Objectives
TLO Select a best value contractor by comparing contractor proposals and test results to source selection criteria
ELO Apply evaluation criteria in a source selection.
ELO Identify the best value approach to source selection
ELO Apply a selected tool (e.g. comparison matrix) to resolve a problem
______________________________________________________
Assignments Review the following ACQ 202 CBT Lesson Summary:
• Lesson 3.1, Source Selection Process
______________________________________________________
Estimated Student
Preparation Time 10 minutes
______________________________________________________
Assessment Class participation; oral presentation
______________________________________________________
Related Lessons • CBT Lesson 2.8, RFP Preparation, Part I
• CBT Lesson 2.9, RFP Preparation, Part II
• CBT Lesson 3.1, Source Selection Process
______________________________________________________
Self Study
References FAR Part 15
______________________________________________
109

Intermediate Systems Acquisition Course March 2020
110

Intermediate Systems Acquisition Course March 2020
Exercise 3.1 Source Selection Process
Background:
We are now conducting the down-select of the contractor for the Engineering and Manufacturing
Development phase. Flyin-Hyer and CyboRaptor have each submitted a proposal for continued
design and development of Firebird II based on the final RFP. The Government technical team
has analyzed the contractors’ test results and proposals addressing the technical subfactors
established in accordance with the source selection plan. The Government technical team
evaluation is as follows:
Government Technical Evaluation of CyboRaptor Technical Proposal
Demonstrated Performance Risk Findings
1. Range: 240 KM
Design modification plan in place –
evaluated as a moderate risk of not
reaching the 250 km threshold
2. Survivability: 92% against shoulder-
launched heat-seeking missiles
Low risk due to success in testing and
system adaptability for future
improvements
3. Weapons Accuracy: 10 meters CEP Low risk due to mature technology
4. Launch: Launched from a stationary
mobile launcher unit and safely airborne in
28 feet
Low risk due to mature technology
5. Mean Time between Critical Failure
(MTBCF): 160 hours Low risk due to demonstrated performance
6. Mean Time to Repair (MTTR): 3.1
hours
Improvements in MTTR planned for
through low impact design modifications –
moderate risk of not reaching 3 hour
threshold
Government Technical Evaluation of Flyin-Hyer Technical Proposal
Demonstrated Performance Risk Findings
1. Range: 320 KM Very low risk due to test performance
2. Survivability: 85% against shoulder-
launched heat-seeking missiles
Design changes proposed by Flyin-
Higher should result in improvement to
the 85% survivability achieved in testing,
however, the risk of not reaching the
threshold of 90% is rated as high
3. Weapons Accuracy: 8 meters CEP Low risk due to demonstrated performance
4. Launch: Launched from a stationary
mobile launcher unit and safely airborne
in 32 feet
Improvements in launch distance planned
for – moderate risk of not reaching 30 foot
threshold
111

Intermediate Systems Acquisition Course March 2020
5. Mean Time between Critical Failure
(MTBCF): 140 hours
Improvements in MTBCF expected due to
design changes for reliability – moderate
risk of not reaching 150 hour threshold
6. Mean Time to Repair (MTTR): 3.3
hours
Improvements in MTTR planned for
through design modifications – high risk
of not reaching 3 hour threshold

Your IPT is part of the Source Selection Evaluation Board (SSEB). As a part of the SSEB, you
will rate one contractor, using the standards you developed and the proposal analysis results
provided by the technical team. Your evaluation will be provided to the Source Selection
Advisory Council (SSAC) and ultimately to the Source Selection Authority (SSA), who will pick
the winning contractor. Once you have completed your work as the SSEB you will switch roles
and become the SSAC. As the SSAC you will make a best value recommendation to the SSA.

Assignment:

1. Using the proposal analysis information above, apply the standards you and the other teams
developed in Exercise 2.2, Source Selection Planning, as a starting point to rate your assigned
contractor in each subfactor. According to the Source Selection Plan, the color rating may be
adjusted one color up or down by the SSEB based on the risk information for each factor.

2. Once you reach consensus as a class on the technical performance ratings the instructor will
reveal the proposed cost and past performance rating for each offeror. Using the matrix,
compare the 2 contractors and come up with a team recommendation of the best value to the
government. Be prepared to defend your recommendation to the class.

112

Intermediate Systems Acquisition Course March 2020
DRAFT FIREBIRD II CDD PARAMETERS

NOTE: These values come from the CDD approved at the CDD Validation decision. KPPs and
other requirements should be proven through testing prior to the Milestone C decision.

1. Survivability (KPP):

Threshold: The UAV must have a probability of survival against shoulder-launched heat-
seeking missiles of at least 90%.

2. Range (KPP):

Threshold: The vehicle must be able to fly out to 250 KM and return to base.
Objective: The vehicle must be able to fly out to 300 KM and return to base.

3. Weapons Accuracy: (KPP)

Threshold: 10M Circular Error Probable (CEP)
Objective: 5 M Circular Error Probable (CEP)

Note: CEP, the circular error of probability, refers to the radius around the target within
which the munitions must fall 50% of the time.

4. Launch:

Threshold: The UAV must launch from a stationary mobile launcher unit and be safely
airborne within a distance of 30 feet.
Objective: The UAV must launch from a stationary mobile launcher unit and be safely
airborne within a distance of 25 feet.

5. Mean Time between Critical Failure (MTBCF) for the UAV must be no less than:

Threshold: 150 hours
Objective: 200 hours

6. Mean Time to Repair (MTTR) for the UAV must be no more than:

Threshold: 3 hours
Objective: 2.5 hours

113

Intermediate Systems Acquisition Course March 2020

114

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 3.2
______________________________________________________
Lesson Title Contractor Performance Measurement
______________________________________________________
Lesson Time 1.5 hours
______________________________________________________
Terminal and Enabling Learning Objectives
TLO The student will be able to analyze contractor performance indicators to identify trends and problems
ELO Given earned value data calculate cost variance, schedule variance, cost performance index and schedule performance index
ELO Given cost variance, schedule variance, SPI & CPI explain the program’s cost and schedule status
______________________________________________________
Assignments Review the following ACQ 202 CBT Lesson Summaries:
• Lesson 3.4 Technical Risk Management
• Lesson 3.7, Earned Value Management
• Lesson 4.6, Contractor Performance Measurement
• Lesson 4.7, Integrated Baseline Review
______________________________________________________
Estimated Student
Preparation Time 45 minutes
______________________________________________________
Assessment Class participation; oral presentation
______________________________________________________
Related Lessons Exercise 2.2, Systems Engineering
Exercise 2.4, Technical Performance Measures
Exercise 2.5, Contractor Planning, Scheduling and Resourcing
Exercise 3.1, Source Selection Process
______________________________________________________
115

Intermediate Systems Acquisition Course March 2020

Self Study
References
• DoDD 5000.01, The Defense Acquisition System
• DoDI 5000.02, Operation of the Defense Acquisition System,
• Defense Acquisition Guidebook
______________________________________________
116

Work Breakdown Structure
1.1
Air Vehicle
1.1
Air Vehicle
1.2
Ground Control
Terminal
1.2
Ground Control
Terminal
1.3
Launcher
1.3
Launcher
1.2.1
Radio
Transceiver
1.2.1
Radio
Transceiver
1.2.2
Control
S/W
1.2.2
Control
S/W
1.2.3
TV
Receiver
1.2.3
TV
Receiver
1.1.1
Weapons
Delivery
System
1.1.1
Weapons
Delivery
System
1.1.2
Air Frame
1.1.2
Air Frame
1.1.3
Engine
1.1.3
Engine
1.1.4
Command
& Control
System
1.1.4
Command
& Control
System
1.1.5
Self
Defense
System
1.1.5
Self
Defense
System
1.1.4.1
Radio
1.1.4.1
Radio
1.1.4.2
TV Camera
1.1.4.2
TV Camera
1.1.4.3
Avionics
1.1.4.3
Avionics
1.1.5.1
Flares
1.1.5.1
Flares
1.1.5.2
Flight Control
Mechanism
1.1.5.2
Flight Control
Mechanism
1.1.5.3
Missile
Sensor
1.1.5.3
Missile
Sensor
Firebird
II
Firebird
II
117

c. TYPE
8. PERFORMANCE DATA
ITEM Current Period Cumulative to Date Reprogramming
Adjustments At Completion(l) Budgeted Cost ACWP Variance Budgeted Cost ACWP Variance
WS WP SCH COST WS WP SCH COST CV SV Budget Budgeted Estimated Variance
1.1 Air Vehicle 2 14.235 12.975 14.942 -1.260 -1.967 42.704 38.879 43.678 -3.825 -4.799 115.881 112.468 3.413
1.1.1 Weapons Delivery 3 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
1.1.2 Air Frame 3 0.620 0.650 0.643 0.030 0.007 1.860 1.950 1.930 0.090 0.020 6.497 6.340 0.157
1.1.3 Engine 3 7.328 5.577 8.111 -1.752 -2.534 21.985 16.730 23.184 -5.255 -6.454 58.920 57.299 1.621
1.1.4 C2 System 3 1.115 1.245 1.066 0.130 0.179 3.344 3.689 3.198 0.345 0.491 8.962 8.471 0.491
1.1.4.1 Radio 4 0.700 0.732 0.680 0.032 0.052 2.100 2.150 2.040 0.050 0.110 5.628 5.527 0.101
1.1.4.2 TV Camera 4 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
1.1.4.3 Avionics 4 0.415 0.513 0.386 0.098 0.127 1.244 1.539 1.158 0.295 0.381 3.334 2.944 0.390
1.1.5 Self Defense 3 5.172 5.503 5.122 0.332 0.381 15.515 16.510 15.366 0.995 1.144 41.502 40.358 1.144
1.1.5.1 Flares 4 1.844 2.009 1.819 0.165 0.190 5.531 6.026 5.456 0.495 0.570 14.823 14.253 0.570
1.1.5.2 Flight Control 4 2.829 2.970 2.808 0.142 0.162 8.486 8.911 8.424 0.425 0.487 22.677 22.185 0.492
1.1.5.3 Sensor 4 0.499 0.524 0.495 0.025 0.029 1.498 1.573 1.486 0.075 0.087 4.002 3.920 0.082
1.2 Grnd Cont. Terminal 2 4.002 4.087 3.942 0.086 0.146 12.005 12.262 11.825 0.257 0.437 34.025 33.010 1.015
1.2.1 Radio 3 1.835 1.836 1.835 0.001 0.001 5.505 5.507 5.504 0.002 0.003 16.605 15.610 0.995
1.2.2 Control Software 3 2.167 2.252 2.107 0.085 0.145 6.500 6.755 6.321 0.255 0.434 17.420 17.400 0.020
1.2.3 TV Camera 3 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
a. QUANTITY c. ESTIMATED COST OF AUTHORIZED
UNPRICED WORK
d. TARGET PROFIT/
FEE
e. TARGET
PRICE
f. ESTIMATED
PRICE
h. ESTIMATED CONTRACT
CEILING
UNCLASSIFIED
UNCLASSIFIED
1. CONTRACTOR
a. NAME
c. EVMS ACCEPTANCE
b. LOCATION (Address and ZIP Code)
2. CONTRACT
a. NAME
b. NUMBER
3. PROGRAM
a. NAME
4. REPORT PERIOD
a. FROM (YYYYMMDD)
b. TO (YYYYMMDD)
d. SHARE RATIO
DOLLARS IN
LOCAL REPRODUCTION AUTHORIZED.
Millions
CyboRaptor
1100 Carey Ave
Waynesville VA 21345
FIREBIRD II
FDS601-20006C-DO23
FPI (F) 60/40
FIREBIRD II
FORM APPROVED
Update to OMB No. 0704-0188
The public reporting burden for this collection of information is estimated to be 3.1 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and
reviewing the collection of information. Send comments regarding this burden or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense Washington Headquarters Services
Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any
penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THIS ADDRESS. SUBMIT COMPLETED FORMS IN ACCORDANCE WITH
CONTRACTUAL REQUIREMENTS.
b. PHASE EMD
b. COST i. DATE OF OTB/OTS
(YYYYMMDD)
5. CONTRACT DATA
6. ESTIMATED COST AT COMPLETION 7. AUTHORIZED CONTRACTOR REPRESENTATIVE
a. BEST CASE
b. WORST CASE
c. MOST LIKELY
MANAGEMENT ESTIMATE
AT COMPLETION
(1)
VARIANCE
(3)
CONTRACT BUDGET
BASE
(2)
a. NAME (Last, First, Middle Initial) b. TITLE
c. SIGNATURE d. DATE SIGNED
(YYYYMMDD)
Page 1 of 2
INTEGRATED PROGRAM MANAGEMENT REPORT
FORMAT 1 – WORK BREAKDOWN STRUCTURE
0.00$230.00
$211.60
$230.00
$221.87 $230.00 $8.129
$251.67$27.37 / 11.9% $283.10N / A
NO YES YYYY/MM/DD
$257.37
YYYY0901
YYYY0930
UPDATE FROM DD FORM 2734/1, MAR 05, PENDING APPROVAL
N/A
118

INTEGRATED PROGRAM MANAGEMENT REPORT
UPDATE FROM DD FORM 2734/1 MAR 05 PENDING APPROVAL
UNCLASSIFIED
UNCLASSIFIED
DOLLARS IN
LOCAL REPRODUCTION AUTHORIZED.
Page 2 of 2Millions
The public reporting burden for this collection of information is estimated to average 3.1 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and
reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services,
Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any
penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THIS ADDRESS. SUBMIT COMPLETED FORMS IN ACCORDANCE WITH
CONTRACTUAL REQUIREMENTS.
8. PERFORMANCE DATA
ITEM Current Period Cumulative to Date Reprogramming
Adjustments At Completion(l) Budgeted Cost
ACWP
Variance Budgeted Cost
ACWP
Variance
WS WP SCH COST WS WP SCH COST CV SV Budget Budgeted Estimated Variance
1.3 Launcher 2 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
1.4 Sys Prog Mgmt 2 4.315 3.952 4.623 -0.362 -0.671 12.944 11.857 13.303 -1.087 -1.446 51.786 52.825 -1.039
1.4.1 Proj Mgmt 3 1.361 1.361 1.423 0.000 -0.063 4.082 4.082 4.270 0.000 -0.188 14.730 14.832 -0.102
1.4.2 Sys Engineering 3 2.954 2.592 3.200 -0.362 -0.608 8.862 7.775 9.033 -1.087 -1.258 37.056 37.993 -0.937
1.5 Sys T&E 2 0.884 0.802 0.844 -0.081 -0.042 2.651 2.240 2.532 -0.411 -0.292 8.633 8.684 -0.051
1.5.1 Dev T &E 3 0.480 0.407 0.443 -0.073 -0.036 1.441 1.222 1.329 -0.219 -0.107 4.897 5.049 -0.152
1.5.2 Oper T&E 3 0.184 0.176 0.184 -0.008 -0.008 0.553 0.528 0.551 -0.025 -0.023 1.440 1.412 0.028
1.5.3 Mockups 3 0.219 0.219 0.217 0.000 0.002 0.657 0.490 0.652 -0.167 -0.162 2.296 2.223 0.073
1.6 Sys Data 2 0.294 0.280 0.293 -0.014 -0.012 0.882 0.841 0.878 -0.041 -0.037 2.967 3.062 -0.095
1.6.1 Eng Data 3 0.136 0.133 0.094 -0.003 0.039 0.407 0.399 0.282 -0.008 0.117 2.025 1.933 0.092
1.6.2 Mgmt Data 3 0.158 0.147 0.199 -0.011 -0.051 0.475 0.442 0.596 -0.033 -0.154 0.942 1.129 -0.187
1.7 Pec Support Equip 2 0.557 0.526 0.624 -0.031 -0.098 1.671 1.577 1.872 -0.094 -0.295 4.986 5.372 -0.386
1.7.1 Test & Measure 3 0.228 0.240 0.323 0.011 -0.083 0.685 0.719 0.969 0.034 -0.250 2.794 3.229 -0.435
1.7.2 Support & Handling 3 0.329 0.286 0.301 -0.043 -0.015 0.986 0.858 0.903 -0.128 -0.045 2.192 2.143 0.049
1.8 Common Supt Equip 2 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
1.9 Spares & Rep 2 0.036 0.033 0.030 -0.003 0.003 0.107 0.099 0.091 -0.008 0.008 6.422 6.450 -0.028
b. Cost of Money 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000
c. GENERAL AND ADMINISTRATIVE 3.203 2.962 3.069 -0.242 0.107 9.610 8.885 9.206 -0.725 -0.321 29.539 29.167 0.372
d. UNDISTRIBUTED BUDGET 0.000 0.000 0.000
e. SUBTOTAL (PMB) 24.321 22.656 25.298 -1.665 -2.642 72.964 67.755 74.179 -5.209 -6.424 224.700 221.871 2.829
f. MANAGEMENT RESERVE 5.300
g. TOTAL 24.321 22.656 25.298 -1.665 -2.642 72.964 67.755 74.179 -5.209 -6.424 230.000 221.871 8.129
9. RECONCILIATION TO CONTRACT BUDGET BASE
a. VARIANCE ADJUSTMENT 0.000 0.000 0.000
b. TOTAL CONTRACT VARIANCE -5.209 -6.424 8.129
119

Intermediate Systems Acquisition Course March 2020

Exercise 3.2 Contractor Performance Measurement

This exercise is comprised of two parts. During the first part your team will analyze Integrated Program Management Report (IPMR)
data and answer questions regarding the current status of the Firebird II program. You will examine the report for specific work
breakdown structure (WBS) elements to determine if the program is on track. During the second part of this exercise, you will review
the integrated program status including both IPMR and technical performance measurement (TPM) data to identify program risk and
ways to manage that risk.

Background: The Engineering and Manufacturing Development contract awarded to CyboRaptor calls for a 24-month, $230
million effort to complete final development. Under the contract, CyboRaptor will continue developmental testing using
design/production representative prototypes and prepare for production of the required modification kits for the UAV.

CyboRaptor will submit monthly IPMRs to report their progress to the Firebird II Program Office.

Assignment 1:

Use the latest IPMR to answer the following questions based
on the “Cumulative to Date” and “At Completion” columns
only:

1. In general, how is our project doing in terms of cost and
schedule?

2. Which element of the WBS is of greatest concern? Why?

3. What level of WBS should the Government pay attention
to?

4. What is the Schedule Performance Index (SPI) of the air
vehicle? What does that tell us about the contractor’s
efficiency for this element?
5. What is the Cost Performance Index (CPI) for the air
vehicle? What does that tell us about the contractor’s
efficiency for this element?

6. What is the percent spent for the air vehicle? What does
this mean?

7. What is the percent complete for the air vehicle? What
does this tell us when compared to percent spent?

8. How will development of the air vehicle turn out if the
current trend continues? (Hint: Calculate a Government
EAC)

9. Are the variances for the air vehicle consistent with the
contractor’s projections at complete? What does this
indicate?
120

Intermediate Systems Acquisition Course March 2020

EXPLANATIONS AND PROBLEM ANALYSIS REPORT

PROJECT: Firebird II – Air Vehicle

CONTRACT: FDS601-20006-D023

DATE: 30 September

SCHEDULE VARIANCE: – 3.825 COST VARIANCE: – 4.799

COMPLETION: BAC 115.881 EAC 112.468 VARIANCE 3.413

PROBLEM ANALYSIS:

COST:
Higher labor costs than planned due to using overtime to investigate anomalies revealed during
flight tests. The software conversion/enhancement has been much more complex than early
estimate.

SCHEDULE:
Test delayed at Army Test Facility. Recent late receipt of GFE caused a slip in finalizing design
of mock-ups. We are currently one month behind schedule.

PROJECTED IMPACT:
Cost and schedule overrun not anticipated.

CORRECTIVE ACTION:
Continue to monitor and aggressively seek solutions to potential problems. Additional data is
being gathered on possible link in software interoperability problem.

121

Intermediate Systems Acquisition Course March 2020

Assignment 2:

The PM is concerned about the cost overrun and schedule slip in the air vehicle and wants to pin
point the problem and take corrective action. Using the IPMR and TPM data provided on the
last page, answer the following questions:

1. What does the TPM indicate?

2. Combined with the information provided in the IPMR, what is the greatest area of risk in
the project?

3. Given the latest IPMR and the TPM, what is your opinion of the contractor’s
estimate at completion?

4. What is your confidence in the contractor’s ability to complete this project on time
and within budget?

5. What are the implications to the overall program (e.g., Acquisition Program
Baseline, program master schedule, requirements?) What should we do?
122

Intermediate Systems Acquisition Course March 2020
Technical Performance Measurement – Range
CyboRaptor
KPP: Range
Objective
Threshold
SEP OCT NOV DEC JAN FEB MAR APR MAY JUN JUL AUG
(#11-30)
(#31-75)
(#76-100)
(#101-110) (#116-118)
Achieved to Date
Planned
Profile
(#1-10)
(#111-115) (#119-120)
SEP OCT
100
150
200
250
300 KM
123

Intermediate Systems Acquisition Course March 2020

124

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 3.3
______________________________________________________
Lesson Title Software & Interoperability
______________________________________________________
Lesson Time 2 hours
______________________________________________________
Terminal and Enabling Learning Objectives
TLO
Given a scenario, apply key software acquisition management
principles needed to make sound decisions for planning and
executing an acquisition program.
ELO Recognize the importance of fully integrating cybersecurity into programs early and throughout the system lifecycle.
ELO Identify “Best Practices” that may be appropriate for the acquisition of software-intensive systems.
ELO Identify the interoperability requirements in the Net Ready KPP as they apply to acquisition of Information Technology
ELO Identify the benefits and risks associated with using Commercial Off The Shelf (COTS) software
ELO Explain the relationship between software development activities and the systems engineering process.
ELO Explain the impact of a new requirement on various functional areas
ELO
Identify the impacts of a new program requirement on the following
functional areas: Program Management, Systems Engineering,
Contracting, Lifecycle Logistics, Financial Management, Software
Acquisition Management, & Test and Evaluation
_________________________________________________
Assignments Read Case 3.5, Software & Interoperability Requirement for
Firebird, DoD Information Network (DoDIN)
Review the following ACQ202 CBT Lesson Summary:
• Lesson 2.4 Risk Management
• Lesson 3.4, Software Design
• Lesson 3.5, Commercial and NDI
• Lesson 4.1, Design Changes
• Lesson 4.2, Software Problems
______________________________________________________
125

Intermediate Systems Acquisition Course March 2020
Estimated Student
Preparation Time 30 minutes
______________________________________________________
Assessment Class participation; oral presentation
______________________________________________________
Related Lessons • CBT Lesson 3.4, Software Design
• CBT Lesson 3.5, Commercial and NDI
• CBT Lesson 4.1, Design Changes
• CBT Lesson 4.2, Software Problems
______________________________________________________
Self Study
References
• Software Program Managers Network: http://www.spmn.com
• Collection of various lists of DoD “Best Practices”:
https://www.dau.edu/cop/it/Pages/Default.aspx
______________________________________________
126

http://www.dau.mil/cop/it

The ability of systems, units, or forces to provide data, information,
materiel, and services to and accept the same from other systems,
units, or forces and to use the data, information, materiel, and
services so exchanged to enable them to operate effectively
together.
Interoperability
(DAU Glossary Definition)
1) IT must be able to support military
operations.
2)IT must be able to be entered and
managed on the network.
3)IT must effectively exchange
information.
IT = Information Technology
CJCSI 6212.01F para. 4. b.
The 3 NR-Requirement Attributes
127

What is Cybersecurity??
(DoDI 8500.01, 14 March 2014)
Prevention of damage to, protection of and restoration of computers, electronic
communication systems, electronic communication services, wire communication,
and electronic communication, including information contained therein, to ensure its
confidentiality, integrity, availability, authentication, and nonrepudiation.
Confidentiality – Is my data safe against disclosure to unauthorized entities?
Integrity – Is my data accurate and consistent over its entire lifecycle?
Availability – Can I access my data/network resources as required?
Authentication – Are data, transactions, communications or documents (electronic or
physical) genuine?
Non-Repudiation – Assurance that the sender of data is provided with proof of delivery
and that the recipient is provided with proof of the sender’s identity, so neither can
later deny having processed the data.
128

Intermediate Systems Acquisition Course March 2020

Case 3.3, Software & Interoperability Requirement for Firebird, Joint
Integrated Environment DoD Information Network (DoDIN)

Background:

The Source Selection Authority selected CyboRaptor to continue with Engineering &
Manufacturing Development Phase of Firebird II. However, after the CDR a new
requirement surfaced…

DoD Information Network (DoDIN)

The latest increment of the DoDIN has defined link 17 as the standard for joint sharing of
surveillance video.

Link 17 enables the transmission and integration of large amounts of digitized video information
from various battlefield and national strategic assets for use in intelligence analysis. Link 17 will
enable Firebird video to be entered onto multiple classified networks supporting the DoD
intelligence community. Link 17 incorporates three current commercial standards: streaming
video, data compression, and public key encryption. The international community as well as the
United States has accepted these three standards.

The U. S. Joint Service community has been impressed with the potential of the Firebird air
vehicle as a key data sensor source for video. Based on the recommendation of the
Configuration Steering Board (CSB), the Joint Requirements Oversight Council (JROC)
amended the Capability Development Document (CDD) for Firebird II to require that it transmit
compressed video using the Link 17 standard. An Army working group on intelligence, tasked
to study potential enhancements to video intelligence distribution, recommended that the JROC
designate Firebird II as a special interest program due to the upgrade involving significant
software modifications. It may also present some complex integration, interoperability, and
reuse issues.

This new requirement was presented at a classified briefing to the Firebird PM and the Milestone
Decision Authority (MDA). The Joint Intelligence community also expressed a strong desire to
see this new capability demonstrated by Firebird II in the upcoming Joint Warfighter
Interoperability Exercise, scheduled 11 months from now. The MDA directed the PM to go back
and see what could be done to accomplish this new requirement, on the proposed schedule, and
within current program resources.

The diagram outlined below illustrates the connections to the DoDIN of most immediate concern
to the members of the Firebird II IPT:
129

Intermediate Systems Acquisition Course March 2020

Subset of DoDIN and Interfaces to External Systems

Situation:

The Firebird II IPT considered three options to make Firebird interoperable with Link 17:
(1) Upgrade the current Firebird commercial off-the-shelf (COTS) communications
software package,
(2) Replace the current Firebird communications software package with several
integrated (COTS) software products, or
(3) Develop new, custom software.

Option 1: Upgrade Current Firebird Communications Software

Steve Larson, the Firebird Systems Engineer, met with Poore Associates, the vendor of
COMVID (version 5.1), the current Firebird COTS video communications package. Mr. Poore
told Steve that the next release of COMVID (version 5.3) would have Link 17 capability.
Sam Robins, the Firebird PMO’s software engineer, obtained a beta copy of COMVID 5.3 for
purposes of government suitability assessment. Sam knew that a preliminary technical
evaluation was a best practice when considering new COTS releases. Sam used DoDIN test data
with the new COMVID software. Sam determined that the COTS software would meet the basic
requirements for Firebird II; however, he found numerous anomalies and reported the problems
to Steve and to Poore Associates.
130

Intermediate Systems Acquisition Course March 2020

When Steve next met with Poore Associates the conversation was cordial. Mr. Poore said that
Sam’s reported problems would be given due consideration. However, based on the lack of a
specific plan to correct the problems, Steve was skeptical that Poore could fix all the anomalies
by the announced commercial release date (eight months from now). Poore also mentioned the
licensing cost to the Firebird program for the new software release would be $600,000.

Option 2: Replace with Alternative COTS Software

Sam discovered, through some calls to associates, that the R&D center in Huntsville, AL was
running Link 17 experiments through other COTS packages. Sam obtained the test data from the
software life cycle support center (an element of the Huntsville R&D center). Sam found that
the center could emulate the full functionality of Link 17 by using several COTS products with
additional integration software (affectionately nicknamed “glue code”). Sam was advised that
the licensing costs to Firebird for the COTS packages alone would cost approximately
$1,000,000. This does not include the price of the glue code from the R&D center or any follow-
on support.

Sam ran the same tests as with the beta COMVID software and had success, with the exception
of slow response by Firebird to the DoDIN. When Sam showed these test results to Steve, Steve
noted that the measured response times from Firebird did not meet the real-time performance
constraints for the DoDIN. Despite the fact that the alternative COTS suite had all the required
functionality, Steve reluctantly ruled out this option because of the poor response time results.

Option 3: Develop New Custom Software

Steve asked Sam to research the possibility of using an available Federal Supply Schedule (FSS)
Information Technology (IT) Indefinite Delivery, Indefinite Quantity (IDIQ) contract to custom
develop the software for Firebird II Link 17 capability. Sam found two software companies
under an IDIQ contract that could do the work and requested offers for delivery of a software
product that would meet Firebird II’s requirements.

• Roman & Associates responded to the software development requirement with an
offer of $1,500,000 and 10 months to deliver the specified software product to
Firebird.
• Dynasoft responded to the same requirement with an offer of $1,750,000 and 8
months to deliver the specified software product to Firebird.

Sam noted that Roman & Associates was rated Level 3 on the Capability Maturity Model
Integration (CMMI) scale, and Dynasoft was rated at Level 4. Both companies emphasized that
their offers entitled the Government to government purpose data rights. Also, neither offer
addressed the cost or availability of follow-on maintenance.

131

Intermediate Systems Acquisition Course March 2020
Assignment:

1. Your instructor will assign each team one of the options to evaluate. Consider the question
below that pertains to your assigned option:

• Option 1 – Software Upgrade. What are the issues associated with relying on Poore’s
new COMVID release?

• Option 2 – Alternative COTS. Was Steve premature in ruling out the alternate COTS
software option? Why or why not?

• Option 3 – New Development. What are the issues with developing new custom software
to meet Firebird II’s interoperability requirements?

2. Identify the pros and cons associated with your assigned option.

3. For your assigned option, what would be the risks if the PM decided to go this way? In
determining risk factors, you should consider the functional areas listed below.
(Note: any risks addressed should be program specific, not general risks such as cost,
schedule, and performance.)

Acquisition Logistics Test and Evaluation
Systems Engineering Software Development
Contracts Management Funds Management

4. Recommend a mitigation strategy for at least one of your identified risks.

5. Recommend a cybersecurity strategy for enabling the following for your assigned option.

Option 1 – confidentiality of information (FB II video)

Option 2 – integrity of FB II’s video data in the DoDIN

Option 3 – availability of FB II video to support military operations

6. Would you recommend this option?

132

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 3.4
______________________________________________________
Lesson Title Reliability
______________________________________________________
Lesson Time 2 hours
______________________________________________________
Terminal and Enabling Learning Objectives
TLO Analyze a reliability problem from multiple perspectives and select and defend a solution
ELO Explain the interrelationship between selected functional areas (e.g., contracting, finance, systems engineering) and acquisition logistics.
ELO Explain why it is important to influence system design for supportability.
ELO
Explain the relationship of Reliability, Availability, and Maintainability
(RAM) to Acquisition Logistics, and its impact on system performance,
operational effectiveness (including support), logistics planning, and life-
cycle cost.
ELO
Identify and the impacts of a supportability problem on the following
functional areas: Program Management, Systems Engineering,
Contracting, Lifecycle Logistics, Financial Management, Quality
Assurance & Manufacturing, & Test and Evaluation
ELO Explain how instability of requirements, design, and production processes impact program cost and schedule.
______________________________________________________
Assignments Review the following ACQ 202 CBT Lesson Summaries:
• Lesson 1.2, Selecting the Best Approach
• Lesson 2.3, Developing the Life Cycle Sustainment Plan
• Lesson 3.3, Design for Supportability/Trade-Off Analysis
______________________________________________________
Estimated Student
Preparation Time 10 minutes
______________________________________________________
Assessment Class participation; oral presentation
______________________________________________________
133

Intermediate Systems Acquisition Course March 2020
Related Lessons • CBT Lesson 1.2, Selecting the Best Approach
• CBT Lesson 3.3, Trade-Off Analysis
• CBT Lesson 3.6, Role of Manufacturing
______________________________________________________
Self Study
References N/A
______________________________________________
DoDI 5000.02T
12. RELIABILITY AND MAINTAINABILITY (R&M)
“a. The Program Manager will formulate a comprehensive R&M
program using an appropriate strategy to ensure reliability and
maintainability requirements are achieved.….
b. For MDAPs, the Program Manager will prepare a preliminary
Reliability, Availability, Maintainability and Cost Rationale (RAM-
C) Report in support of the Milestone A decision. This report
provides a quantitative basis for reliability requirements, and
improves cost estimates and program planning. This report will
be attached to the SEP at Milestone A, and updated in support of
the Development RFP Release Decision Point, Milestone B, and
Milestone C.
c. Reliability growth curves (RGCs) will reflect the reliability
growth strategy and be employed to plan, illustrate, and report
reliability growth. RGCs will be included in the SEP at Milestone A
and updated in the draft SEP submitted at the Development RFP
Release Decision Point and in the final approved SEP and Test
and Evaluation Master Plan submitted at Milestone B…”
134

Intermediate Systems Acquisition Course March 2020
Lesson 3.4 – Reliability Issue
Background:
The Full Rate Production Decision Review (FRPDR) for Firebird II is only two months away, when a
potentially serious problem develops with the countermeasures control module in the air vehicle.
Tom, from Test & Evaluation, receives conflicting data from CyboRaptor on whether the control
module they plan to use will meet the Mean Time Between Critical Failure (MTBCF) minimum
requirement of 300 hours for this component. Limited developmental testing of the module indicates
that 300 operating hours MTBCF is attainable. However, a computer-based parametric model
predicts an MTBCF of only 200 hours. This presents a potential reliability issue.
Tom consults with Larry from Logistics in the next cubicle, who convinces Tom to bring their
dilemma to Steve, the Systems Engineer. After some discussion with Steve, they decide the best
solution is to conduct three months of additional testing to better determine the module’s
reliability. However, when they take this recommendation to the PM, COL Cole, he says there
may not be enough time or money for additional testing. Instead, COL Cole directs them to
talk with the contractor and user reps and then prepare a discussion paper with various options
for handling this risk and the tradeoffs involved.
Tom contacts the user reps, who state categorically that they are “tired of getting burned by poor
reliability” and are unwilling to reduce the MTBCF requirement. They also make it clear that
they don’t want any schedule slips.
Steve calls Zeke, his counterpart at CyboRaptor. Zeke says the best solution from his
perspective is to go with a better module. It will cost more, but it will provide increased
reliability. Zeke agrees to fax details to Steve; then, before hanging up, adds that whatever the
Government decides, they must do it quickly to avoid costly delays. “Indecision on this issue
will quickly put us behind schedule” says Zeke. Steve, Tom and Larry meet again and prepare
the following discussion paper for the PM:
135

Intermediate Systems Acquisition Course March 2020
Discussion Paper
From: Engineering Design Team
To: Colonel Cole
Subj: Control Module Options
Computer-based modeling conducted by CyboRaptor revealed a potential reliability issue with
the UAV countermeasures control module. While limited environmental testing indicates the
module will operate for 300 hours, parametric models predict a problem that will reduce Mean
Time Between Critical Failure (MTBCF) to as low as 200 hours, 33% below the requirement.
CyboRaptor could apply special coatings and heat sinks to the module during production, but
they say this additional process would be inefficient. Instead, CyboRaptor recommends the use
of a more reliable, and more expensive, solid state module. The user reps are adamantly opposed
to reducing the MTBCF requirement, since all their supportability analyses, planning and
resultant funding profiles for support costs depend on the module operating properly for 300
hours. They are tired of battling numerous reliability problems with current systems and don’t
want Firebird modifications to add any more problems.
Below is a summary of the options we came up with after discussions with CyboRaptor and
military user reps:
Option 1 – Stay with the current module.
If the module proves reliable, this option has the advantage of maintaining the current cost
and schedule baseline. We estimate the probability of meeting the required 300 hour MTBCF
is 80%, based on the conflicting results of the limited testing and the parametric models.
However, if this option is chosen and the reliability falls short, major operational and
maintenance problems will result, including system performance degradation due to poor
reliability, availability and maintainability. In the current design, replacement of modules
later on would require extensive dismantling and reassembly of the air vehicle. No funds
have been programmed to allow for additional maintenance costs, and currently there is no
time or money for additional testing.
Option 2 – Modify the existing module.
This will require production changes to ensure the module meets the 300 hour MTBCF.
Some new manufacturing equipment and changes to the planned manufacturing process will
be needed in preparation for the upcoming Full Rate Production Decision Review. The
resultant schedule slip is expected to be 60 days. In addition, this option will increase
production costs by $3,750 per air vehicle for the 400 retrofit kits. No funds have been
budgeted to pay for these changes.
136

Intermediate Systems Acquisition Course March 2020
Option 3 – Replace the module.
This is easier than altering the manufacturing process, but the new module will cost an
additional $7,500 per vehicle for the 400 retrofit kits. There is no money for a more expensive
module. In addition, our market research indicates the three available vendors cannot meet our
initial quantity requirements due to temporary shortages from high demand, but they promise
that this will not happen once the current shortage is overcome. We estimate this will result in
a production delay of at least 90 days. However, this commercially available module is
guaranteed to work for over 500 hours before failure.
Signed,
Steve, Larry and Tom
COL Cole reads the discussion paper and thinks to himself, “What should I do now? I guess
this is why I get the big bucks…”
Assignment:
1. Pick a group leader/briefer. Each team member should take on the role of a different
functional area expert or stakeholder (e.g., user, logistician, systems engineer, funds
manager, tester, production and quality manager).
2. Discuss in your team the pros and cons of each option. Then reach consensus on the
best approach to recommend to the PM, based on the information provided and any
assumptions you feel are necessary.
3. For your recommended approach, examine the impact in terms of the user,
logistician, systems engineer, funds manager, tester, and PQM manager.
4. Prepare a 5-10 minute brief explaining your recommended choice, assumptions, and
supporting rationale.
137

Intermediate Systems Acquisition Course March 2020

138

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 3.5
______________________________________________________
Lesson Title Contract Change
______________________________________________________
Lesson Time 1 hour
______________________________________________________
Terminal and Enabling Learning Objectives
TLO Recognize an unauthorized commitment situation and avoid giving inappropriate direction to a contractor.
ELO Explain the interrelationship between selected functional areas (e.g., life cycle logistics, finance, systems engineering) and contracting.
ELO Identify the causes and consequences of unauthorized commitments
ELO
Identify the complementary roles and responsibilities of the contracting
officer and the program manager in their partnership throughout the
acquisition process.
TLO Given a scenario, apply the procedures, rules and public laws associated with the execution of DoD budgets.
ELO
Identify the public laws (i.e., Misappropriation Act, Anti-deficiency
Act, Bona Fide Need Rule) that apply to the use of appropriated funds in
DoD acquisition.
ELO
Select the appropriate public law (i.e., Misappropriation Act, Anti-
deficiency Act, Bona Fide Need Rule) that applies to the use of
appropriated funds under specific circumstances.
______________________________________________________
Assignments Review the following AC202 CBT Lesson Summaries:
• Lesson 4.4, Reprogramming Funds
• Lesson 5.2, Unauthorized Commitments
• Lesson 5.4, Change Orders
______________________________________________________
Estimated Student
Preparation Time 15 minutes
______________________________________________________
Assessment Class participation; oral presentation
______________________________________________________
139

Intermediate Systems Acquisition Course March 2020
Related Lessons • CBT Lesson 4.4, Reprogramming Funds
• CBT Lesson 5.2, Constructive Changes
• CBT Lesson 5.4, Change Orders
______________________________________________________

Self Study
References N/A
______________________________________________

140

Intermediate Systems Acquisition Course March 2020
Lesson 3.5 – Contract Change
Part 1: Contractual Direction and Authority
Background:
After receiving the discussion paper, COL Cole considered his options. He knew money was
extremely tight, and that a lot was riding on his ability to get Firebird II to the fighting forces on
time and within budget. He decided to have CyboRaptor stay with the original control module
and accept the 20% risk that it would not meet the MTBCF requirement in spite of the model’s
conflicting predictions.
A New Problem:
Three weeks after the start of full rate production (FRP), Connie, Firebird’s Procuring
Contracting Officer, receives an unexpected package in the morning’s mail. The cover letter
begins: “CyboRaptor is pleased to submit the enclosed Request for Equitable Adjustment for the
changes described herein.” Slightly stunned, and growing angrier by the minute, Connie’s eyes
drift to the bottom of the page and she sees the words, “…increase in production contract price
by $3,000,000 and extends the delivery schedule by 90 days.” “Get the PM on the horn right
away,” she calls to her secretary as she mutters to herself, “Why am I always the last to know?”
The Story:
Connie has supported the Firebird program since its inception. After discussions about the
technical complexities, programmatic uncertainties, and fiscal risk of the Firebird II upgrade, she
and the PM agreed that the best contract type for full rate production lot options would be firm
fixed price (FFP). Both felt that this type of contract would adequately distribute risks of the
program between the parties based on the maturity of the technologies and the availability of
adequate cost data.
CyboRaptor, the prime contractor for both the original Firebird and Firebird II, is well known in
the industry for quality products. The company’s management recently promoted Howard
Hagan to the position of program manager for Firebird II, due to his extensive familiarity with
UAV technology. While Howard had worked periodically on various stages of Firebird’s
original development, this is his first assignment as a program manager, and he is determined to
please his customer and make this project successful for both himself and his company.
United States Indo-Pacific Command (USINDOPACOM) has a high level of interest in the
development and eventual fielding of Firebird II. Consequently, USINDOPACOM recently
selected and permanently stationed an O-5 in the PMO to represent the interests of their
Operational Commands and to serve as liaison officer (LNO) with the “developers.” CDR Flyboy
has no acquisition experience but was chosen because of his extensive experience with operational
units currently employing Firebird.
Upon reporting as the LNO, Flyboy learned of the risk of reliability problems in the UAV control
modules, and he decided to take action. Feeling strongly about his professional responsibilities to
141

Intermediate Systems Acquisition Course March 2020
the sailors, soldiers, airmen and marines in operational units, he telephoned RADM Adams, his
boss at USINDOPACOM. After a brief discussion of the problem, Flyboy asked the Admiral’s
permission to have CyboRaptor make the system more reliable for its ultimate warfighting
customers by using the higher quality, more expensive module. RADM Adams replied that he
didn’t understand the issue well enough to comment, but he would rely on Flyboy’s knowledge of
the program and his warfighting expertise to proceed as he thought best for all concerned.
Flyboy convinced COL Cole to let him visit CyboRaptor’s production facility and take a tour of
their plant. Cole called Howard, who said he would be glad to have Flyboy visit. Cole then
explained to Flyboy that his visit was simply intended to provide him with fundamental
information about, and orientation with, the current status of the program. He went on to explain
that the FRP contract options were firm fixed price, meaning that Government control of contractor
operations was minimal, due to the relatively certain nature of the technical requirements and
associated work.
Before touring the production facility, Flyboy was taken to Howard Hagan’s office for a courtesy
call. Over coffee, Flyboy expressed his concern about the reliability of the control module. He
said he felt strongly that using a better module would maximize the effectiveness and survivability
of the system. Flyboy also said he hoped Hagan would do whatever was necessary to make sure
Firebird II performs with the required reliability. Hagan nodded frequently appearing interested
and sympathetic to Flyboy’s concerns.
After the meeting with Hagan, Flyboy took a tour of the plant with one of the production
supervisors. It was a worthwhile visit, with Flyboy asking lots of questions and gaining valuable
insight into the Firebird program.
The next day, eager to please his customer, Hagan put his staff to work developing and
implementing a solution consistent with Flyboy’s direction. Hagan instructed his staff to proceed
with any actions necessary to satisfy this revised requirement for better reliability. Sensing the
urgency in Hagan’s voice, they immediately placed orders for the more expensive module to
minimize impact on the delivery schedule. Hagan assumed that COL Cole would be pleased with
his initiative and resultant action taken to avoid a future problem.
142

Intermediate Systems Acquisition Course March 2020
Assignment:
Consider and develop answers to the following questions. Be prepared to explain your answers to
the class.
1. What is the authority and responsibility of:
– CDR Flyboy?
– COL Cole?
– RADM Adams?
– Connie?
2. What role should the Administrative Contracting Officer (ACO) have played in this scenario?
3. Is the Government liable for Flyboy’s actions?
4. Should Howard Hagan have relied on Flyboy’s direction?
5. If you were COL Cole, what actions would you take?
6. Who messed up?
143

Intermediate Systems Acquisition Course March 2020
Part 2 – Funding Issue
PM decides to use the new module after all
COL Cole, after a lot of soul-searching and some additional indications that the current module
would not meet MTBCF requirements, decides that the right thing to do is use the more expensive
module and pay the claim from CyboRaptor. He sweet-talks the users into giving him an
additional 90 days of schedule. The Government contracting official (the Head of the Contracting
Activity) agrees to ratify the change once the PM certifies that appropriate funds are available and
Connie, with legal counsel concurrence, recommends payment. But where will the money come
from???? He puts in a call to Faye, his trusted funds manager…..
Intern to the Rescue
Faye is away on TDY, but her energetic new intern comes up with what he believes is a solution
to the money dilemma. He says that the Mustang program down the hall in the same PEO has
enough RDT&E money to pay for the cost of the request for equitable adjustment from
CyboRaptor and is willing to fork it over since they can’t use before it expires.
Assignment
Discuss and develop an answer to the following question in your group and be prepared to
explain your answer to the class.
Did the intern come up with a good funding solution? Why or why not?
144

You are hereby notified that I DO NOT have
the authority to direct you in any way to alter
your contractual obligation. Further, if the
Government, as a result of the information
obtained from today’s discussion DOES
desire to alter your requirements, changes
will be issued in writing and signed by the
contracting officer. You should take no action
on any change unless and until you receive
such a contract modification.
Statement of Limitation of Authority
 Supplemental Agreement
 Change Order
 Unauthorized Commitment
Types of Contract Modifications
145

 Contract mod based on prior agreement of parties
regarding the change
 Incorporates equitable adjustment to contract
cost and/or schedule as a result of the change
Supplemental Agreement
 Written order issued by CO directing contractor to
make a change without prior agreement
 Creates “undefinitized” Government liability
 Contractor may be entitled to equitable adjustment
 Authorized by the “Changes Clause”
Change Order
Change Order
must be within
the scope of the
contract –
limited to
 Drawings,
designs, or
specifications
 Method of
shipping or
packing
 Increase or
decrease in Govt
Furnished
Property
 Place of delivery
 FAR 52.243-1&2
146

 An Unauthorized Commitment is an agreement
that is not binding solely because the government
representative who made it lacked the authority
to enter into that agreement on behalf of the
government.
 Must involve:
 Change in performance beyond minimum contract
requirements, and
 Word or deed by Government representative which
requires contractor effort that is not a necessary part
of the contract
 Requires Ratification by Head of Contracting
Activity
Unauthorized Commitment
All functional areas will be affected by a contract
change, not just the contracting officer. What are
some possible repercussions in these areas?
 Program Management
 Financial Management
 Logistics
 Test and Evaluation
 Systems Engineering
Contract Change Effects
147

Use of Appropriated Funds
Appropriated Funds
 Misappropriation Act (Purpose)
o Requires funds to be used only for the purposes and
programs for which the appropriation was made.
 Anti-Deficiency Act (Amount)
o Prohibits making or authorizing an obligation in excess
of the amount available.
 Bona Fide Need Rule (Time)
o Requires funds to be used only for needs in the year of
the appropriations obligation
148

Intermediate Systems Acquisition Course March 2020
LESSON ASSIGNMENT SHEET
Lesson Number Exercise 4.1
______________________________________________________
Lesson Title Supportability
______________________________________________________
Lesson Time 2 hours
______________________________________________________
Terminal and Enabling Learning Objectives
TLO
Analyze the impact of supportability issues on system
readiness/performance and other functional areas. E.g. contracts,
finance, systems engineering and acquisition logistics
ELO Synthesize several approaches to solving a program supportability issue (obsolescence).
ELO Evaluate approaches to solving a program supportability issue (obsolescence).
ELO Recommend the best to solving a program supportability issue (obsolescence).
ELO Identify the proper DoD Appropriation Category to be used to budget for each of the three phases of a Product Improvement Program.
ELO Assess the impact of the failure to execute funds in accordance with program plans.
ELO Recognize how configuration management impacts all functional disciplines (e.g., test, logistics, manufacturing, etc.)
ELO Demonstrate the interrelationship between selected functional areas, e.g., contracting, finance, systems engineering, and life cycle logistics.
ELO
Identify tools/best practices/techniques available in the systems
engineering process to achieve the principal goals of supportability
analyses.
ELO Apply performance based metrics to a program supportability problem (e.g. obsolescence)
ELO Apply performance or outcome based logistics principles to solving a program obsolescence issue.
______________________________________________________
Assignments Review the following ACQ 202 CBT Lesson Summaries:
• Lesson 2.3, Developing the Life Cycle Sustainment Plan
• Lesson 3.3, Design for Supportability/Trade-Off Analysis
• Lesson 3.6, Role of Manufacturing
149

Intermediate Systems Acquisition Course March 2020
• Lesson 6.2, Logistical Support
______________________________________________________

Estimated Student
Preparation Time
30 minutes
______________________________________________________

Assessment

Class participation; oral presentation
______________________________________________________

Related Lessons

• Lesson 2.3, Developing the Life Cycle Sustainment Plan
• Lesson 3.3, Design for Supportability/Trade-Off Analysis
• Lesson 3.6, Role of Manufacturing
• Lesson 6.2, Logistical Support
______________________________________________________
Self Study
References N/A
_____________________________________________

150

Obsolescence
 Performance Based Logistics = Performance Based Life
Cycle Product Support (PBL)
 A Life Cycle Management (LCM) implementation strategy
 An outcome-based product support strategy that plans and
delivers an integrated, affordable performance solution
designed to optimize system readiness
 Establishes performance goals for a
weapon system through a support
structure
 Based on long-term performance
agreements with clear lines of
authority and responsibility to
continuously meet the users needs
Recommended Reference: The PBL
Guidebook – https://www.dau.edu/tools/t/Performance-
Based-Logistics-(PBL)-Guidebook
So What Exactly is PBL?
Focuses
system
support on
what’s
Important to
the war
fighter:
capability and
performance
151

 Produce OUTCOMES, not OUTPUTS!
 Performance as a package, vice transactional goods and
services
 Document performance, support, & resource
requirements in Performance Based Agreements (PBA)
 Establish Product Support Integrator (PSI) to integrate
& manage all (contract and organic) sources of support
 Establish incentives to promote “win-win” relationships
and achievement of performance outcomes
 Leverage Public-Private Partnerships to make best use
of organic and commercial capabilities in long-term
collaborative relationships
Fundamental PBL Tenets
PBL is NOT
Outsourcing
– it “is not
synonymous
with CLS nor
does it
require a
private
sector
integrator”
(AFI 63-
107)
• DoD obtains comprehensive performance package
– Not individual parts, transactions, or “spares & repairs”
• Approach totally reverses vendor incentive
– Fixed price “pay for performance” contracts motivate vendor to reduce failures/
consumption
– Incentivizes “less I use, the more profit I can make” vice a “more spares and
repairs I can sell, the more profit I can make” mentality
– Long term commitment enables vendor to balance risk vs. investment
• Improves Parts Support
– Material availability increases + Logistics Response Time (LRT) decreases
resulting in Improved Readiness
• Optimizes Depot Efficiency
– Repair Turn Around Time (RTAT), Awaiting Parts (AWP), & Work in Process
(WIP) decrease
• Incentive to Invest in Reliability
– Mean Time Between Failure (MTBF) improves
• Incentive to Invest in DMSMS & Obsolescence Mitigation,
Improve Repair Processes, Reduce Costs, and Support the
Warfighter
Why PBL Works
Focus on the
Performance
“End-State” …
NOT the “How
To”
152

The program office’s logistics manager will most often
perform the role of the statutorily required Product Support
Manager.
The PBL Team
Support Performance “Outcomes”
Requirement Need Performance Measure
Materiel
Availability
Is the system
ready?
• Mission Capability Rate
• Reduced Down Time
Materiel
Reliability
Will the system be
effective?
• Mission Completion Rate
(sorties, etc.)
• Time on Wing
• Mean Time Between Failures
(MTBF)
Ownership
Cost
How much will it
cost?
• Operating Cost (per flight
hour, mile, steaming hour,
etc.)
Mean Down
Time
How long does it
take to meet the
demand?
• Customer wait time
• Mean Logistics Delay Time
(MLDT)
• Mean Time to Repair
These 4 Life
Cycle
Sustainment
Outcome
Metrics Are
Universal
Across All
Programs And
Are Essential
To Effective
Sustainment
Planning
153

Fund
Development
& Test
with …..
NOYES
NO
YES
Product Improvement
Funding Decision Tree
Independent
DT or OT
required?
System in
Production?
RDT&E Procurement O&M
NO
YES
Fund All Mod Kit for
End Items
& Installation of Kits with ….
Mod to
increase
performance?
Procurement
 The work of a program office is never done.
Issues arise even after a system is fielded.
o Follow-on operational testing may reveal weaknesses in
the system
o Advancement may present opportunities to improve or
extend the service life of the system
o Logistical support can impact other functional areas such
as contracting, funds management, configuration
management and test & evaluation.
o By taking a long-term view, considering total life-cycle
costs, and using standardized components and open
systems designs, we will make the most cost effective
decisions.
Conclusion
http://www.dmea.osd.mil/
154

Intermediate Systems Acquisition Course March 2020
Exercise 4.1, Firebird II Supportability

Firebird II has finally reached Full Operational Capability (FOC) when the item manager for the
X-651 Field Programmable Gate Array (FPGA) integrated circuit chip announces that the chip
has reached end of life. System Danner, the manufacturer of the FPGA states they will cease
production of the X-651 in 12 months. The X-651 FPGA is used in the Firebird Inertial
Navigation System (INS). The Air Force manages this stock item for all the services.
Operational units order the INS through normal Air Force supply channels using O&M dollars.

The INS in which the X-651 is used is one of only a few original GFE items that remain from the
original Firebird system. The INS was originally manufactured in accordance with a detailed
specification provided by the Air Force and is used in all of the current Firebird and Firebird II
fleet. System Danner, who has been the supplier of the chip to the INS manufacturer since the
mid-1990’s, is the sole supplier for the X-651.

The item manager sends an e-mail to Larry, the Logistics Manager in the Firebird II program
office, to let him know about the FPGA situation. The item manager mentions that the
manufacturer has a couple of possible solutions they would like to offer the
Government:

• System Danner is willing to execute an “ End of Life” buy, where the Government would
be given priority to purchase the X-651’s in a quantity that the Air Force believes would
provide enough FPGAs to last for about 4 years beyond the current year assuming current
usage rates.

• System Danner has also offered to sell the technical data package (TDP) for the
integrated circuit if the Government wants to buy it, which would allow the Government
the opportunity to find another manufacturer to make the X-651. The manufacturer is
willing to allow the Government to purchase the TDP for approximately the same price
as the value of the ICs on hand.

Larry, worried about the readiness impact, asks the item manager what the Air Force intends to
do about the X-651 FPGA integrated circuits for their other UAVs. The item manager states that
this isn’t a problem for the Air Force as with the exception of Firebird, they haven’t ordered any
of these FPGAs for over a year now. Instead, the Air Force has been buying FPGAs for all their
service unique UAVs using a performance specification. The performance specification uses
open standard interfaces that specify standard FPGAs provided by lots of different vendors. This
was part of an Air Force initiative several years ago to go to an open systems design for all of
their UAVs to save money and support broader competition.

Upon further investigation, Larry learns that Firebird INS motherboard is the lowest replaceable
assembly (LRA), and is unique in that newer FPGAs are not compatible. So, Larry has a
supportability dilemma, and he needs your help to decide how best to ensure that the Firebird
system has a viable INS for the future.

155

Intermediate Systems Acquisition Course March 2020
Assignment:

1. Identify three possible alternatives that could meet the services’ requirements for Inertial
Navigation System (INS) for Firebird II, given that the supplier will stop producing the FPGA
within 12 months.

2. Select your choice among the alternatives and be prepared to explain why it is the alternative
your team would recommend to the Logistics Manager.

3. Analyze your selected alternative in terms of the following considerations (if they apply) and
the necessary actions that would need to be taken.
• Funding impacts
• Contracting impacts
• Configuration management issues
• Test and Evaluation impacts
• Supportability issues/concerns [i.e., RAM/Operational Availability (Ao)/Applicable
Elements of Support]
• Manufacturing issues

4. Choose a performance based metric that you would use in the contract for your selected
alternative.

5. Be prepared to brief your results of the above four steps to the class.

156

Intermediate Systems Acquisition Course October 2018
ACQ – 202
Intermediate Systems
Acquisition Course
CBT Lesson Summaries
157

ACQ 202 Summaries Index
Lesson Title Page
1.1 Considering the Costs 158
1.2 Selecting the Best Approach 162
2.1 Integrated Product and Process 164
Development (IPPD)
2.2 Developing the Acquisition 167
Approach
2.3 Developing the Life Cycle 172
Sustainment Plan
2.4 Risk Management 176
2.5 Developing the TEMP 181
2.6 ESOH Issues 186
2.7 Programming Funds 188
2.8 RFP Preparations (Part I) 192
2.9 RFP Preparations (Part II) 196
3.1 Source Selection 201
3.2 Technical Risk Management 207
3.3 Design for Supportability/ 210
Trade-Off Analysis
3.4 Software Design 214
3.5 Commercial & NDI 216
3.6 Role of Manufacturing 219
Lesson Title Page
3.7 Earned Value Management 221
3.8 Budgeting Process 225
4.1 Design Changes 227
4.2 Software Problems 231
4.3 APB Breaches 235
4.4 Reprogramming Funds 236
4.5 Reviews, Simulations & Tests 238
4.6 Contractor Performance 241
Measurement
4.7 Integrated Baseline Review 246
4.8 Budget Execution 248
4.9 Operational & Live Fire Tests 250
5.1 Best Manufacturing Practices 252
5.2 Unauthorized Commitments 254
5.3 Production and Follow-On 256
Support
5.4 Contract Modification 258
6.1 Contract Dispute 261
6.2 Life Cycle Product Support 263
6.3 Leadership and Ethics 266
158

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Considering the Costs | Lesson Summary 1
1.1 Lesson Summary
Considering the Costs
Learning Objectives
The following learning objectives are covered in this unit:
• Given an Initial Capabilities Document (ICD) and a summary Analysis of Alternatives, (AoA),
select an appropriate concept, from the perspective of the system developer, to meet the user’s
needs.
• Given a cost breakdown, determine Development Cost, Flyaway Cost, Weapon System Cost,
Procurement Cost, Acquisition Cost, and Life Cycle Cost.
• Select an appropriate method to estimate the cost of an acquisition program.
• Select an appropriate approach (e.g., Analogy, Parametric, Top-Down, Engineering (Bottom-Up),
Actual, Delphi) to estimate the cost and schedule for a software-intensive system.
• Relate the typical distribution of software lifecycle costs to the planning of an acquisition
program.
• Recognize the impact and interrelationship of logistics support and life cycle cost.
Analysis of Alternatives (AoA)
An Analysis of Alternatives (AoA) is used to help select the best, most cost-effective way to meet a
capability need. An AoA:
• Is a study of operational effectiveness and life-cycle cost
• Is mandatory for all acquisition category (ACAT) programs
• Provides objective feedback on cost and effectiveness of alternatives
• Is based on information from the Initial Capabilities Document (ICD), including:
o Functional areas
o Range of military operations
o Time
o Key attributes defined by measures of effectiveness
The AoA can be used to justify starting, stopping or continuing an acquisition program based on cost,
performance and schedule factors.
Cost Estimation
Cost Estimation is an analysis of costs of hardware, software or services derived from historical cost,
performance, schedule and technical data of similar items or services and performance, schedule, and
technical data for the new system.
Methods used to prepare cost estimates include:
159

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Considering the Costs | Lesson Summary 2
• Analogy – A new system or component is compared with a similar, existing system or
component. Generally this type of analysis can be done quickly and inexpensively. Analogy
estimates are commonly used early in the acquisition process, but are subjective and less precise
than other methods.
• Parametric – Statistical analysis of a database of similar systems is used to develop a
relationship between cost and one or more performance or design characteristics of the systems.
The resulting Cost Estimating Relationship can then be used to estimate the cost of a new
system. This method is inexpensive and used relatively early in the acquisition process. The cost
estimating relationship is very useful in analyzing “what-if” scenarios, but it is only as valid as the
statistical correlation and the database used to create it.
• Engineering (Bottom-Up) – Detailed analysis of all of the materials, parts and labor required to
produce a system is performed from the bottom up. This analysis is very accurate and is more
objective than other methods, but it can be expensive and very time-consuming. Engineering
estimates are typically used for elements identified as cost drivers in the later stages of system
development.
• Actual Costs – Costs of future efforts are extrapolated based on the previous cost of identical or
nearly identical systems. This method also refers to the use of actual cost data from completed
portions of the program to update the program’s life cycle cost estimate.
Methods used to prepare cost estimates for software development include the analogy, parametric, and
the bottom-up methods above, plus:
• Top-Down – a systems-level view of the project
• Delphi – a team of experts combine different approaches to arrive at a collective judgment.
Generally, a life cycle cost estimate will use a combination of different methods. The type of estimating
method that is used on each of the cost elements that comprise the estimate should be based on the
type and accuracy of available data, the stage of the acquisition life cycle, and the relative importance of
the cost element. Each of these methods can be used independently or in combination.
Cost Terms and Definitions
Cost terms and definitions are provided in DoD 5000.4-M, and include seven types of acquisition-based
costs:
• Development Cost – Cost of all research and development-related activities that are necessary
to design and test the system. Funded with RDT&E appropriation.
• Flyaway Cost (a.k.a. rollaway) – Cost of producing prime mission equipment such as ships or
tanks. Funded with procurement appropriations
• Weapon System Cost – Sum of flyaway cost and cost of associated support equipment and
services (including initial training). Also funded by procurement appropriations.
• Procurement Cost – Sum of weapon system cost and cost of the system’s initial spares. Funded
with procurement appropriations.
• Program Acquisition Cost – All costs associated with developing, procuring and housing a
weapon system. Procurement, RDT&E, and MILCON appropriations are used to fund this cost.
• Operating and Support – All costs for personnel, equipment, supplies, software, and services
associated with operating, maintaining, supplying and providing ongoing training for any DoD
160

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Considering the Costs | Lesson Summary 3
system. Most O&S costs are funded with the O&M and MILPERS appropriations, although
procurement, RDT&E and MILCON appropriations may also be used when appropriate.
• Life-cycle Cost – Total cost of an acquisition program from beginning to end, including program
acquisition, operating and support, and disposal costs.
All of these costs are funded by specific appropriations, generally referred to as “colors of money.” As you
recall from ACQ-101, the appropriations used by DoD generally fall under five categories:
1. Research, Development, Test and Evaluation (RDT&E)
2. Procurement
3. Operations and Maintenance (O&M)
4. Military Construction (MILCON)
5. Military Personnel (MILPERS)
Distribution of Life Cycle Cost
Generally, costs related to the Operation and Support of a system once it is fielded represent the largest
proportion of its life cycle cost, generally between 70-80%, although the specific percentage varies by
system type and service life. This holds true for software as well as hardware. The largest distribution of
software costs will be late in the program’s cycle. The primary costs of software are related to upgrades
and the maintenance that it takes to keep the software running.
Version 1.0 (11 July 2014)
161

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Selecting the Best Approach | Lesson Summary 1
1.2 Lesson Summary
Selecting the Best Approach
Learning Objectives
The following learning objectives are covered in this unit:
• Determine the applicability of science and technology activities to the acquisition of a system.
• Relate the concepts of Affordability and Should Cost to the planning of an acquisition program.
• Given a draft Capability Development Document (CDD) and a summary Analysis of Alternatives
(AoA), select an appropriate concept, from the perspective of the system developer, to meet the
user’s requirements.
Science and Technology
Science and Technology (S&T) provides the capabilities that give us combat advantages over our
adversaries. S&T activities are divided into three groups based on funding activities: basic research,
applied research, advanced technology development.
Basic Research
Basic Research (Budget Activity 1 funds) involves innovation and discovery aimed at increasing scientific
knowledge. It is usually conducted at academic, laboratory, or other research facilities.
Applied Research
Applied Research (Budget Activity 2 funds) applies Basic Research findings to real-world problems in
order to generate and test new technologies with potential military utility.
Advanced Technology Development
Advanced Technology Development (Budget Activity 3 funds) demonstrates the technology maturity and
the military utility of completed applied research projects and provides realistic assessment of their
potential for transition to an acquisition program. There are two types of demonstrations: Advanced
Technology Demonstrations (ATDs) and Joint Capability Technology Demonstrations (JCTDs).
Advanced Technology Demonstrations (ATDs)
ATDs demonstrate the feasibility and maturity of new technology, and reduce technical risk and
uncertainty, before that technology is incorporated in a formal acquisition program.
Joint Capability Technology Demonstrations (JCTDs)
JCTDs are used in the field to gain understanding and evaluate utility of technology, develop concepts
of operation for that technology, and expedite delivery of new capabilities to combat forces. JCTDs
promote the rapid transition of the new technology into the appropriate phase of a formal acquisition
program.
162

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Selecting the Best Approach | Lesson Summary 2
The point at which a JCTD enters the acquisition life cycle will vary depending on how much work
remains to be done before production begins. A Management Plan, Transition Plan, and Funding Plan
ensure that the necessary preparations are made to facilitate movement into the acquisition process
without loss of momentum. The Transition Plan considers such issues as contracting strategy,
supportability, test and evaluation, affordability and interoperability of the JCTD with other systems to
ease the transition to a formal acquisition program.
Summary of Affordability and Should Cost information
Should Cost entails setting cost objectives that balance mission needs against projected out-year resources
and making tradeoffs in performance and/or schedule to meet capability needs within available resources.
Under the Should Cost approach, available funds are considered to be the independent variable or
constraint, while performance and schedule may be adjusted (within thresholds established in the CDD) to
determine the most cost effective and affordable solution to meet mission requirements.
• Should Cost performance tradeoffs are made within the trade space between thresholds and
objectives established by the user and documented in the CDD.
• The user, developer and support communities must actively participate in the Should Cost
process.
• Should Cost helps refine the CDD by determining what threshold and objective values should be
associated with particular operational performance parameters.
• The best time to reduce life cycle cost is early in the acquisition process. However, Should Cost
principles can be applied throughout the acquisition life cycle to achieve an affordable and
effective system.
Version 1.1 (30 June 2015)
163

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Integrated Product and Process Development (IPPD) | Lesson Summary 1
2.1 Lesson Summary
Integrated Product and Process Development (IPPD)
Learning Objectives
The following learning objectives are covered in this unit:
• Relate the key tenets of Integrated Product and Process Development (IPPD) to planning and
executing an acquisition program.
• Identify the barriers to successful Integrated Product Team (IPT) implementation.
• Identify key acquisition best practices related to IPPD and IPT processes, including commercial
practices that impact the relationship between government and industry.
IPPD Key Tenets
Integrated Product and Process Development (IPPD) stresses cross-functional communication throughout
the acquisition process and includes the following key tenets:
• Customer focused: meet the customer’s needs better, faster, and cheaper.
• Concurrent development of products and processes: processes used during all phases
should be considered throughout product design and development.
• Early and continuous life cycle planning: should begin with science and technology efforts
and extend throughout the entire acquisition life cycle.
• Maximize flexibility for optimization and use of various contractor approaches:
contracts should be designed to allow contractors to apply IPPD principles and make use of
effective commercial standards, practices, and processes.
• Encourage robust design and process improvement capability: techniques should be
used that achieve quality through design, focus on process capability, and stress continuous
process improvement.
• Event-driven scheduling: scheduling should relate program events to their respective
accomplishments and accomplishment criteria.
• Multidisciplinary teamwork: decision-making should be based on input from the entire team,
to reduce risk and create a work environment that is more likely to result in successful
suggestions.
• Empowerment: team members should have the authority to make decisions at the lowest
possible level commensurate with risk.
• Seamless management tools: a management framework should be established that helps
show the interrelationship of all products and process.
• Proactive identification and management of risk: risk analyses and user needs should be
evaluated to identify critical cost, schedule and technical parameters.
164

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Integrated Product and Process Development (IPPD) | Lesson Summary 2
Integrated Product Teams
IPPD is implemented through Integrated Product Team (IPT) members who represent technical, business,
and support functions. The following guiding principles will improve the productivity of any IPT:
• Chartering, launch, and initiation: To get the team off to a good start, prepare a charter
documenting the mission, timeframe, and membership of the IPT; train participants in IPT
principles and the role of each team member; and prepare a Plan of Action and Milestones
(POA&M).
• Goal alignment: Team leaders should ensure that the goals and objectives of each team
member are consistent with the goals of the project. Effective feedback mechanisms should be
put in place to facilitate this.
• Open discussions with no secrets: Due to the unique design of IPTs, in which each member
has expertise in a specific area, free and open communication among all members is essential.
• Empowered, qualified team members: Team members should have the authority to
represent their superiors in the decision-making process. They should remain in close
communication with their bosses to ensure their advice is sound and will not be changed later,
barring unforeseen circumstances.
• Dedicated/Committed, Proactive Participation: Because team success hinges on
participation by members with institutional knowledge of functional areas, IPTs should be
organized so that all key stakeholders can contribute effectively. In many cases, this means
minimizing membership to enhance communication and trust.
• Issues Raised and Resolved Early: All issues should be raised openly and discussed at the
earliest possible opportunity, and solved through team consensus and discussion, not isolated
conversations “offline.”
IPPD Barriers
If IPPD is not implemented properly, barriers can arise that will impact the quality, effectiveness, and
timeliness of the overall process. Some of these barriers include:
• Lack of commitment from top management, which can hurt team member motivation and
impact their ability to achieve results.
• Need for significant cultural change due to the inherent hierarchical structure of the military,
which contrasts with the philosophy set forth in the IPPD process.
• Lack of adaptation to the IPPD process by functional organizations, thereby reducing
everyone’s performance.
• Lack of planning, which causes teams to rush to catch up, thus impacting quality.
• Poor or non-existent education/training in the IPPD process.
• No effort to identify and/or share best practices in IPPD implementation.
• A “not invented here” mentality that can arise due to the many functional areas involved in
the IPPD process, which leads to a lack of information sharing.
• Contractually-imposed practices that hinder a contractor’s flexibility.
165

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Integrated Product and Process Development (IPPD) | Lesson Summary 3
• Use of IPPD by the contractor but not by DoD, resulting in morale problems and less
effective working relationships.
• Awarding of contracts to traditional approach contractors who are not familiar with the
IPPD process, even if it is specified in the Request for Proposal (RFP).
• Unrealistic promises by contractor to implement IPPD.
• Poor contract award fees or incentives that don’t encourage IPPD.
• Poorly run meetings or reviews, resulting in over-emphasis of a particular topic or functional
area to the exclusion of others.
Version 1.1 (30 June 2015)
166

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Acquisition Strategy | Lesson Summary 1
2.2 Lesson Summary
Developing the Acquisition Strategy
Learning Objectives
The following learning objectives are covered in this unit:
• Identify the information required for a decision review and recognize the significance of the
Acquisition Program Baseline, Key Performance Parameters, and Acquisition Strategy.
• Identify the advantages and disadvantages of international armaments cooperative development
in an acquisition strategy.
• Identify the intelligence organizations of the Intelligence Community (IC) and recognize how the
IC provides support to an acquisition program.
Key Performance Parameters (KPPs)
Key Performance Parameters (KPPs) are capabilities and characteristics considered by the user to be the
most essential in successfully accomplishing a capability need. KPPs:
• Should be a minimum number of Performance Parameters necessary to adequately describe the
required capability of the system (generally eight or fewer).
• Are defined using threshold and objective values as a way to describe performance capabilities.
While trade-offs among cost, schedule, and performance might have to be made during the program’s life
cycle, KPP thresholds are typically non-negotiable.
• Threshold values can be lower or higher than objective values, depending on the parameter
involved. For example, for a lighter and faster vehicle, the threshold speed would be lower, and
the weight higher, than the objective values.
• Threshold values establish the minimum acceptable operational value of a given
parameter, below which the utility of the system becomes questionable.
• Unless otherwise specified, the objective value for performance is the same as the
threshold value. For schedule, the threshold typically is the objective value plus six
months, while the threshold cost typically is the objective value plus 10 percent.
• Objective values are the ideal performance parameters desired for the acquisition program,
and are usually defined in operationally meaningful, time-critical, and cost-effective increments
above the threshold values. Ideally, the difference between the threshold and objective values
should diminish as the acquisition program advances.
167

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Acquisition Strategy | Lesson Summary 2
Acquisition Program Baseline (APB)
The Acquisition Program Baseline (APB) establishes the cost, schedule and performance targets for an
acquisition program. Specifically, the APB:
• Serves as a formal agreement between the Program Manager (PM) and the Milestone Decision
Authority (MDA)
• Defines the space between the KPP objectives and thresholds in which trade-offs can be made
between cost, schedule and performance without requiring MDA or user approval, as appropriate
• Can only be changed at milestone reviews, program reviews, or in the event of an unrecoverable APB
breach
Performance Criteria
Only those performance criteria that influence operational effectiveness, suitability, cost and schedule should
be included. In addition to program-specific threshold/objective values for performance, there are six
mandatory Key Performance Parameters (KPPs) that must be addressed by each program: Force Protection;
System Survivability; Training; Energy; Sustainment; and Net-Ready.
Schedule Parameters
Schedule parameters should include program initiation, major milestone decision points, initial operating
capability (IOC) and other critical program dates.
Cost Constraints
This section of the APB shows program-related costs in base year dollars, based on careful risk assessment
and cost estimating. The APB will also include affordability caps for unit production and sustainment costs.
Acquisition Strategy
Development of an Acquisition Strategy is usually done by an Acquisition Strategy IPT, which includes
representation from all functional areas, end users, and key stakeholders. A well-defined acquisition
strategy will include information on:
• Contracting: Number and types of contracts, timing, competition, potential sources, source
selection approach, and Item-Unique Identification (IUID) implementation
• Funding: Type and year of appropriations, funding source agreements, and affordability analysis
• Cost: Cost objective and threshold values derived from Affordability goals, Should Cost objectives,
and cost estimation activities for typical major cost metrics such as total RDT&E cost, total
procurement cost, program acquisition unit cost, average procurement unit cost, and life cycle cost
• Systems Engineering: Technology and product solutions, including commercial and non-
developmental items; open systems architectures; modeling and simulation; environmental, safety
and occupational health considerations; baseline system performance thresholds and objectives;
corrosion prevention and control; and interoperability
• Test & Evaluation: Types of testing, timing of testing, test articles including quantities and sources,
modeling and simulation, and resources such as test ranges
168

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Acquisition Strategy | Lesson Summary 3
• Software development: System integration, sources, re-use, open systems architecture, data
rights, and computer resource life cycle management
• Support Strategy: Life cycle sustainment addressing design for supportability, all applicable
support requirements, and Performance Based Logistics (PBL) approach
• Production: Design for producibility; low-rate initial production (LRIP) schedule; and production
quantities, including long lead procurement items
• Management: Risk management, including planning, assessment, handling, and monitoring of cost,
schedule and performance risk; earned value management reports, if required, to track contractor
progress; and any international considerations related to the program
Much of this functional information can be found in the Program Structure Chart, used to show specific
dates for critical events, including acquisition program phases, decision milestones, program and technical
reviews, major deliveries, T&E periods, RFP/contract information, and other important scheduling
information.
The sequence and interrelationship of activities as the team progresses through the acquisition program is of
significant importance in the program structure chart. The program structure chart should be consistent with
the schedule parameters in the APB. The demonstration of program interrelationships is at the heart of the
IPT approach, where actions and expertise of each team member can either help or hinder the team’s
overall ability to deliver a successful end product.
Remember, as in any IPT-based program, the team can be made up of different members depending on
the nature of the acquisition program itself, and the expertise needed to make it successful. Because the
Firebird is an ACAT II program, the management chain will include the PM, the Program Executive Office,
and the Army Service Acquisition Executive (SAE). The ARMY SAE will be acting as the MDA.
Lifecycle Intelligence Analysis Requirements
The threat intelligence community provides a range of support over the lifecycle of a particular capability
shortfall identification process and the resulting system acquisition program. Capstone Threat Assessments
(CTAs) inform the capability shortfall identification process during early phases of system acquisition prior
to the generation of a Validated Online Lifecycle Threat (VOLT) Report / System Threat Assessment (STA).
The CTAs project foreign capabilities, in particular warfare areas, out 20 years.
At the beginning of the Material Solution Analysis (MSA) phase, the program office or capability sponsor
should contact the appropriate intelligence production center to support integration of validated threat
information into the Acquisition Strategy (AS). Threat information may come from DIA-validated CTA or
other DIN Service-validated VOLT / STAs that align with the capability mission, concept of operations
(CONOPs), and employment timeline.
Capstone Threat Assessments
CTAs provide the bedrock analytical foundation for threat intelligence support to the defense acquisition
process. CTAs, covering major warfare areas, present the DoD Intelligence Community (IC) validated
position with respect to those warfare areas and will constitute the primary source of threat intelligence for
the preparation of Initial Threat Environmental Assessments, VOLT / STAs, and threat sections of
documents supporting the Joint Capabilities Integration and Development System (JCJDS) process.
169

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Acquisition Strategy | Lesson Summary 4
In order to effectively support both the capability development and acquisition processes, CTAs are not
specific to existing or projected U.S. systems, cover the current threat environment, and, in general,
project threats out 20 years from the effective date of the CTA.
Listing of Capstone Threat Assessments
Warfare Area Primary Production Office or Center
Air Warfare National Air and Space Intelligence Center (NASIC)
Chemical, Biological and Radiological Defense Defense Intelligence Agency (DIA)
Information Operations DIA/Joint Information Operations Threat Working Group
Land Warfare National Ground Intelligence Center (NGIC)
Missile Defense Defense Intelligence Agency (DIA)
Naval Warfare Office of Naval Intelligence (ONI)
Space Warfare National Air and Space Intelligence Center (NASIC)
Validated Online Lifecycle Threat (VOLT) Report / System Threat Assessment
(STA)
The Defense Intelligence Agency (DIA) provides validation for the Validated Online Lifecycle Threat
(VOLT) Report, prepared by the appropriate Service, to support Acquisition Category (ACAT) ID/ Major
Defense Acquisition Programs (MDAPs). Appropriate Defense Intelligence organization(s), identified by the
component headquarters intelligence organizations, prepare the VOLT Report.
The assessment should be kept current and validated throughout the acquisition process. DoD Instruction
5000.02, Operation of the Defense Acquisition System, requires that MDAPs have a VOLT Report in place
at Milestone A, it is updated / re-validated at subsequent milestones and the full rate production decision.
A VOLT must be in place at program initiation for shipbuilding programs. The assessment should be
system specific, to the degree that the system definition is available at the time the assessment is being
prepared, and should address projected adversary capabilities at system initial operating capability (IOC)
and at IOC plus 10 years.
DIA will co-chair the TSGs for ACAT ID VOLT Report with the producing command or center. VOLT Reports
for ACAT IC MDAPs and System Threat Assessments (STAs) for ACAT II non-MDAPs are prepared and
validated by the lead Service in accordance with service regulations. DIA Instruction 5000.002, Intelligence
Threat Support for Major Defense Acquisition Programs, describes the required VOLT Report elements and
format.
Critical Intelligence Parameters (CIPs)
CIPs are established and examined through the joint and collaborative efforts of the intelligence, capability
sponsor, and acquisition management community to aid in developing intelligence production requirements
to support an acquisition program. CIPs are those key performance thresholds of foreign threat systems,
which, if exceeded, could compromise the mission effectiveness of the U.S. system in development.
Adversary military doctrine, tactics, strategy, and expected employment of systems should be considered
in the CIPs. Program specific CIPs, and their associated production requirements, are a key part of a VOLT
Report and TTRA will be required for validation. The inclusion of CIPs is also encouraged for STAs.
170

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Acquisition Strategy | Lesson Summary 5
International Cooperation
International Cooperation involves the collaboration of foreign governments and related organizations
during any stage of the acquisition cycle. Congress requires DoD to determine if there are allied or
other friendly nations with whom we can cooperate on major systems development. Also, the
acquisition strategy should address the potential for international cooperative research, development,
production, logistics support, or sale. Some of the possible attractions of international involvement
include:
• A foreign government sharing in the cost of development
• An opportunity to incorporate emerging technology from abroad
• Possible lower production costs through increased foreign competition, by encouraging international
producers to compete with domestic sources
• Promoting interoperability of our systems with those of our allies, providing a warfighting advantage
in multi-national warfighting coalitions
Some of the possible problems with international involvement include:
• Political differences or economic problems with partners that can delay programs
• Possible dependency on foreign sources
• Security issues associated with technology transfer between countries can take a long time to
resolve, which can lead to program delays
• Economic considerations for the US industrial base when foreign competition is introduced
• Legal and administrative requirements for international participation including coordination with the
State Department
Version 1.3 (30 June 2016)
171

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Life Cycle Sustainment Plan | Background Information 1
2.3 Lesson Summary
Developing the Life Cycle Sustainment Plan
Learning Objectives
The following learning objectives are covered in this unit:
• Identify long term supportability and sustainment strategies through the application of the
Product Support Business Model (PSBM).
• Capture the Product Support Strategy and specific planning and execution details in the Life Cycle
Sustainment Plan (LCSP).
Product Support Business Model (PSBM)
Hierarchical Framework
The PSBM defines the hierarchical framework in which the planning, development, implementation,
management, and execution of product support for a weapon system component, subsystem, or system
platform will be accomplished over the life cycle. The PSBM effectively describes the methodology by
which DoD intends to ensure achievement of optimized product support through balancing maximum
weapon system availability with the most affordable and predictable total ownership cost.
Roles and Responsibilities
The Program Manager (PM) –
The PM is assigned Total Life Cycle Systems Management (TLCSM) responsibility and is accountable for
the implementation, management, and oversight of all activities associated with development,
production, sustainment, and disposal of a system across its life cycle. As part of this, the PM has the
responsibility to develop an appropriate sustainment strategy to achieve effective and affordable
operational readiness consistent with the Warfighter resources allocated to that objective.
The Product Support Manager (PSM) –
Consistent with 10 USC 2337, all weapons system programs are required to have a PSM, and the PSM is
delegated responsibilities for oversight and management of the product support functions. The PSM then
typically leads the development, implementation, and top-level integration and management of all sources
of support to meet Warfighter sustainment and readiness requirements.
Product Support Implementing Agents –
Consistent with the model’s emphasis on PBL and a performance/outcome based product support
approaches, there may be a requirement for one or more Product Support Integrators (PSI) who are
chartered with integrating sources of support from Product Support Providers (PSP) (public or private),
defined within the scope of their implementing arrangements, to achieve the documented outcomes.
172

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Life Cycle Sustainment Plan | Background Information 2
Product Support Strategy Alternatives
Product Support Levels –
Product support may be categorized into three levels select each one for more information: system,
subsystem, and component,
System – A “system” is defined as a weapons platform, such as a tactical aircraft, an M1 Abrams tank,
or an AEGIS ship (there are circumstances where a system may house or support another system
managed by a different PM).
Subsystem – A “subsystem” is an integrated critical subsystem that is part of a war fighting platform,
such as an aircraft engine, a ground tactical vehicle fire control system, or on-board radar.
Component – A “component” is generally defined as an item that can be readily removed and
replaced. Components can be repairable assemblies or a commodity item requiring little or no
repair, such as aircraft tires.
Integrated Product Support (IPS) Elements–
The range of product support is generally defined by the scope of the IPS elements. A listing of the twelve
IPS elements is included below:
1. Product Support Management
2. Design Interface
3. Sustaining Engineering
4. Supply Support
5. Maintenance Planning & Management
6. Packaging, Handling, Storage, and Transportation (PHS&T)
7. Technical Data
8. Support Equipment
9. Training and Training Support
10. Manpower and Personnel
11. Facilities & Infrastructure
12. Computer Resources
These items are addressed in more detail in the IPS Element
Guidebook: https://acc.dau.mil/ips- guidebook.
12-Step Product Support Strategy Process Model
Process Overview –
The development of, or revision to, a product support strategy follows the logical methodology captured in
the 12-step model. The 12-step Product Support Strategy Process Model represents the major activities
required to implement, manage, evaluate, and refine product support over the life cycle. It is not a rigid one-
time process, but rather a flexible continuing, iterative process in which the sustainment of a system (or
systems) is adapted and evolved to optimally support the needs and requirements of the Warfighter in an
effective and affordable manner. The first two steps set the stage and set each program up for success:
173

https://acc.dau.mil/ips-guidebook

https://acc.dau.mil/ips-guidebook

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Life Cycle Sustainment Plan | Background Information 3
Step One – Integrate Warfighter Requirements and Support
Understanding Warfighter requirements in terms of performance is the essential first step in developing a
meaningful support strategy.
Step Two – Form the Product Support Management Integrated Product Team
Effective performance based product support strategies require the participation and consensus of all
stakeholders in developing the optimum sustainment strategy. Establishing the Integrated Product Team
(IPT) team, led by the PSM, is also a critical step in the Product Support Strategy Process Model.
Details on the remaining steps and other information about the Product Support Strategy Process Model is
the PSM Guidebook: https://acc.dau.mil/psm-guidebook.
Life Cycle Sustainment Plan (LCSP)
Format –
The LCSP is to be a streamlined document consistent with the attached annotated outline. The outline is
designed to be a tool for programs to effectively and affordably satisfy life-cycle sustainment
requirements. This plan articulates the product support strategy, and it must be kept relevant as the
program evolves through the acquisition milestones and into sustainment. The LCSP outline emphasizes
early-phase sustainment requirements development and planning, focuses on cross-functional integration
– most critically with systems engineering – and highlights key sustainment contract development and
management activities.”
The annotated LCSP outline is available on the DAU website at the following address:
https://acc.dau.mil/lcsp-outline.
Overview
An overview of the product support strategy and sustainment-related arrangements shall be included in
the Acquisition Strategy, to include information regarding performance based arrangements (both
contracts and intra-Governmental agreements). Also, there must be alignment among the LCSP and
other critical program documents, including the Systems Engineering Plan, and Technical Data Rights
Strategy. It should also be noted that the Sustainment Quad Chart is the primary vehicle for summarizing
the program’s product support planning to stakeholders outside the program, and the LCSP must support
and provide the detail behind the summary information presented on the chart.
Requirements
Public law 10 USC §2337 requires that the PSMs of major weapon systems e responsible for
developing the product support strategy to be included in the LCSP.
DoD Instruction (DoDI) 5000.02 requires that a LCSP be developed and provided as part of the
program approval process to document how the sustainment strategy is being implemented.
DoDI 5000.02 also provides the following information regarding the LCSP:
a. The Under Secretary of Defense for Acquisition, Technology and Logistics (USD(AT&L)) will
approve acquisition category (ACAT) ID, ACAT IAM, and USD(AT&L) – designated special interest
program LCSPs at each decision point.
174

https://acc.dau.mil/psm-guidebook

https://acc.dau.mil/lcsp-outline

http://uscode.house.gov/view.xhtml?req=granuleid%3AUSC-prelim-title10-section2337&amp%3Bnum=0&amp%3Bedition=prelim

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the Life Cycle Sustainment Plan | Background Information 4
b. The Component Acquisition Executive (CAE), or designee, will approve LCSPs for ACAT IC, ACAT
IAC, and ACAT II and below programs at each acquisition decision point after the Materiel
Development Decision.
c. The LCSP will be updated at each decision point to reflect the increased maturity of the product
support strategy, any changes in the corresponding product support package, current risks, and
any cost reduction activities.
The Defense Acquisition Guidebook (DAG) Section 5.1.2.2 states that the LCSP is a living
document describing the approach and resources necessary to develop and integrate sustainment
requirements into the system’s design, development, testing and evaluation, fielding and operations.
Performance Based Logistics
DoD 5000.02 also requires that programs employ effective PBL planning, development, implementation,
and management in developing a system’s product support arrangements. PBL (also known as
performance-based life-cycle product support) ties objective metrics delivered logistical system
performance to incentives that will motivate the support provide.
Living Document
As depicted in the graphic below, requirements for LCSP content change over time, starting at Milestone A
and continuing through the Operations and Support phase (NOTE: Review the links on Page 14, LSCP
Evolution Tab, for more detailed information about the purpose/content of the LSCP at various points).
Milestone A – Focus on development of sustainment metrics to influence design and product support
strategy, and on actions that can be taken before Milestone B to reduce future operating & support costs.
DRFPDR & Milestone B – Focus on finalizing the sustainment metrics, integrating sustainment
considerations with design and risk management activities, and refining the execution plan.
Milestone C – Focus on ensuring operational supportability and verifying performance.
FRP Decision or Full Deployment – Focus on how sustainment performance is measured, managed,
assessed and reported.
Initial Operational Capability (IOC) – LCSP will be principle document governing the system’s
sustainment.
Version 1.5 (31 December 2015)
175

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Risk Management | Lesson Summary 1
2.4 Lesson Summary
Risk Management
Learning Objectives
The following learning objectives are covered in this unit:
• Identify the five activities of the risk management process.
• Use the risk assessment process to identify the major areas/sources of risk in an acquisition program
strategy.
• Recognize how implementing program protection, cybersecurity, and counterintelligence
requirements into the acquisition strategy help to reduce overall program risk.
Risk Management Process Model
The Risk Management Process has five activities designed to help identify and manage risk during the
acquisition process:
Risk Planning
The first activity in the risk management process is to conduct risk planning, i.e.; completion of the activities
to develop, implement, and document the risk management process. Effective planning should outline each of
the risk management steps and should be summarized in the System Engineering Plan (SEP) and the Risk,
Issue, and Opportunity (RIO) Management Plan (these three items may be captured in separate documents).
This activity answers the question, What is the program’s risk management process? This activity determines
what planning, budget, requirements and contractual changes are needed, provides a coordination vehicle
with management and other stakeholders, directs the integrated product teams to execute the defined, and
approved risk mitigation plans, outlines the risk reporting requirements for on-going monitoring, and
documents the change history.
Risk Identification
The second activity in the risk management process is to identify risks by answering the question, What can
go wrong? This step involves examining the technical aspects of a program to determine risk events (root
causes) that may have negative cost, schedule, and performance impacts. To help identify risk root causes,
the program team should ask “why” the event or condition is a risk, and then repeat the question “why” to
reveal the causal factors that lead to the underlying root cause. The team continues the activity until it
identifies the risk root cause as well as the specific causal factors.
Risk identification begins as early as possible in successful programs and continues throughout the program
with regular reviews and analyses of Technical Performance Measurements (TPMs), schedule, resource data,
life-cycle cost information, Earned Value Management (EVM) data/trends, progress against critical path,
technical baseline maturity, safety, operational readiness, and other program information available to
program IPT members.
176

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Risk Management | Lesson Summary 2
Risk Analysis
Risk analysis answers the question, How big is the risk? or What are the likelihood or consequence of the risk?
It is an iterative process that examines the cost, schedule, and performance parameters of risks in order to
determine their likelihood and consequence to achieving program objectives. This activity examines each
identified risk to refine the description of that risk, isolate its cause, determine the effects, and aid in setting
risk mitigation priorities.
Risk analysis refines each risk in terms of its likelihood, its consequence, and its relationship to other risk
areas or processes. Analysis begins with a detailed study of the risks that have been identified. The
objective is to gather enough information about future risks to judge their root causes, their likelihood, and
their consequences, if the risk occurs.
Risk Handling
Risk handling includes the handling options or combination of options and the specific implementation
approach to addressing risks. A risk handling approach answers the question, What is the plan to address the
risk? or Should the risk be accepted, avoided, transferred, or mitigated? After the program’s risks have been
analyzed, the program personnel should develop a strategy to manage risks by evaluating the four risk handling
options and selecting those best fitting the program’s circumstances.
The selected handling approaches for program-level risks should be reflected in the program’s SEP and
Acquisition Strategy and include the specifics of what should be done, when it should be accomplished, who is
responsible, the cost, schedule, and performance impact, and the funding/resources required to implement the
risk handling plan. The most appropriate program approach is selected from the handling options listed below
and documented in a RIO mitigation plan. One or more of these handling options may apply:
• Risk Acceptance – Program acknowledges risk event or condition may be realized – risk should be
continued to be tracked to ensure that accepted consequences do not change.
• Risk Avoidance – Program reduces or eliminates the risk event or condition by taking an alternate
path, i.e., eliminates the source of the risk and replaces it with a lower risk solution.
• Risk Transfer – Program is able to reassign the risk responsibility to another entity (could be to
another program, to the prime contractor, to another Government agency, etc.). Programs should
recognize that transference of the risk does NOT eliminate all responsibility and the risks must still
be monitored for potential consequences.
• Risk Mitigation – Program actively seeks to reduce risk to an acceptable level. This option generally
entails taking action to reduce likelihood, and at times the consequence, of a risk to as low as possible
to minimize potential program impacts.
Risk Monitoring
Risk monitoring answers the question, How has the risk(s) changed? Risk monitoring includes a continuous
process to systematically track and evaluate the performance of risk mitigation approaches against
established metrics throughout the acquisition process. During this time, the program office should
reexamine and conduct assessments with the risk mitigation approaches to determine effectiveness.
Successful risk monitoring includes timely, specific reporting procedures as part of effective communications
among the program office, contractor, and stakeholders.
177

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Risk Management | Lesson Summary 3
Program Risk Areas
The risk management process should be integrated with other program management and systems
engineering functions and associated tools during all phases of the program. Examples of program
management tools include the Work Breakdown Structure (WBS), Integrated Management Plan (IMP),
Integrated Master Schedule (IMS) and Earned Value Management (EVM). Technical Performance
Measures (TPMs) are an example of a relevant systems engineering tool.
Framing Assumptions and Ground Rules
Program leadership should consider and document program framing assumptions because of the risks they
may introduce should the assumptions prove invalid. Examples of framing assumptions may include priority
of requirements, schedule dependencies, or accuracy of models and simulations. In addition, the program
should document key risk management ground rules to be used across the risk management program.
Typical ground rules for risk management relevant to programs include time frame, time of risk event, and
WBS level:
• Time Frame: Current probability and impact estimates are based upon the status of the item or
event as assessed, not upon projected or planned activities.
• Time of Risk Event: In order to properly analyze a risk, the time at which the risk hypothetically
will occur should be considered and documented, as occurrence or realization timing is most likely a
primary factor in the impact.
• WBS Level: Hardware and software risk events should be identified to the lowest level possible to
facilitate identification of causal factors and handling strategies.
Potential Risk Areas
Program Risk Areas can come in a variety of forms from any functional area. These potential sources of risk
include:
• Threat: Foreign intelligence collection efforts, program uncertainty due to changes in the threat, and
degree of change in system design.
• Capability Needs: Level of sensitivity to uncertainty in user needs.
• Design: Ability of program’s system configuration to meet objectives based on available tools,
technology, etc.
• Test and Evaluation: Capability of the T&E program to assess performance specifications.
• Modeling and Simulat ion (M&S): Capability of M&S to support program using validated models and
simulations.
• Technology: May change rapidly during the program’s life; opt for mature technology that has been
demonstrated and can meet the program’s objectives.
• Logistics: Ability of system configuration to meet logistics objectives.
178

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Risk Management | Lesson Summary 4
• Sources of Support: Ability of the support strategy to ensure the system will be operationally
suitable in its intended environment.
• Production: How well program production objectives can be met based on system design and
manufacturing processes.
• Concurrency: Sensitivity to uncertainty resulting from poorly-planned lifecycle phases or activities.
• Capability of Developer: Developer’s ability to design, develop and manufacture the system.
• Cost/Funding: Achieving objectives within given resource and funding parameters.
• Management: Degree to which program plans and strategies can meet objectives.
• Schedule: Can the program accomplish its goals within a reasonable time frame?
The PMO should also be advised of risk in areas including, but not limited to, manpower, environmental
impact, systems safety/occupational health, and systems engineering. Within an IPT, it is each team
member’s responsibility to best identify these potential risks within their area of expertise, and help develop
consensus on how to tackle them before they grow unchecked.
Program Protection
By definition, Program Protection is the integrating process for mitigating and managing risks to advanced
technology and mission-critical system functionality from foreign collection, design vulnerability, or supply
chain exploitation/insertion, battlefield loss, and unauthorized or inadvertent disclosure throughout the
acquisition lifecycle.
At its core, the purpose and focus of Program Protection is protect technology, components, and
information from compromise through the cost effective application of countermeasures to mitigate risks
posed by threats and vulnerabilities. In a simple sense, Program Protection seeks to defend warfighting
capability by “keeping secret things from getting out” and “keeping malicious things from getting in.”
Where the capability is derived from advanced or leading-edge technology, Program Protection mitigates
the risk that the technology will be lost to an adversary; where the capability is derived from integration of
commercially available or developed components, Program Protection mitigates the risk that design
vulnerabilities or supply chains will be exploited to degrade system performance.
Program Protection Plan (PPP)
The PPP is the milestone acquisition document that describes the plan, responsibilities, and decisions for all
Program Protection activities. A PPP is required by DoDI 5000.02, Operation of the Defense Acquisition
System, and DoDI 5200.39 Critical Program Information (CPI) Protection within DoD. In accordance with
DoDI 5000 .02, the PPP will be submitted for MDA approval at each milestone review beginning with
Milestone A.
Additionally, Program Managers will submit the Program’s Cybersecurity Strategy as part of every PPP. The
purpose of the PPP is to help programs ensure that they adequately protect their technology, components,
and information. Cybersecurity is a critical component of this plan.
179

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Risk Management | Lesson Summary 5
Cybersecurity
DoD systems and networks are constantly under cyber-attack. Nearly all defense systems incorporate
information technology (IT) in some form, and must be resilient from cyber adversaries. This means that
cybersecurity applies to weapons systems and platforms; Command, Control, Communications, Computers,
Intelligence, Surveillance, and Reconnaissance (C4ISR) systems; and information systems and networks.
Cybersecurity is a critical priority for DoD, and is a vital aspect of maintaining United States’ technical
superiority.
Cybersecurity is the prevention of damage to, protection of, and restoration of computers, electronic
communications systems, electronic communications services, wire communication, and electronic
communication, including information contained therein, to ensure its availability, integrity, authentication,
confidentiality, and nonrepudiation.
Effective Cybersecurity in DoD acquisition programs encompasses all of the necessary actions taken to
ensure the Confidentiality, Integrity and Availability (CIA) of system information to enable warfighting
operations.
PMs should integrate Cybersecurity, including required resources, into the system’s acquisition lifecycle
activities, e.g., the Systems Engineering Technical Reviews, and artifacts, e.g., Systems Engineering Plan,
PPP which includes the Cybersecurity Strategy, Test and Evaluation Master Plan (TEMP), Request for
Proposal (RFP); and activities, e.g., testing, cost estimates.
Counterintelligence (CI)
Counterintelligence (CI) from a DoD policy perspective are those activities undertaken as part of an
integrated DoD and national effort to detect, identify, assess, exploit, penetrate, degrade, and counter or
neutralize espionage, intelligence collection, sabotage, sedition, subversion, assassination, and terrorist
activities conducted for or on behalf of foreign powers, organizations, persons, or their agents directed
against U.S. national security interests or DoD and its personnel, information, materiel, facilities, and
activities.
Cybersecurity and the Risk Management Framework (RMF)
The management of the inherent risks associated with Information Systems (IS) and Platform Information
Technology (PIT) is now captured in the program’s Cybersecurity Strategy (required at Milestone A) as part
of the overall Program Protection Plan (PPP). Cybersecurity is a special area of risk that requires an added
emphasis and in fact has separate and complementary policies to govern its implementation and
management.
The RMF describes the DoD process for identifying, implementing, assessing, and managing cybersecurity
capabilities and services, expressed as security controls, and authorizing the operation of IS and PIT
systems.
The RMF is designed to be complementary to and supportive of DoD’s acquisition management system
activities, milestones, and phases. RMF activities should be initiated as early as possible in the DoD
acquisition process to increase security and decrease cost. Requirements development, procurement, and
T&E processes should be considered in applying the RMF to the acquisition of DoD IT.
Version 1.5 (30 June 2016)
180

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the TEMP | Lesson Summary 1
2.5 Lesson Summary
Developing the TEMP
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify the primary test and evaluation (T&E) products required at each acquisition milestone.
• Identify the key T&E support organizations within DoD.
• Identify the key Operational Test and Evaluation (OT&E) activities that must be coordinated with
the Director, Operational Test and Evaluation (DOT&E) staff and the Operational Test Agencies
(OTAs).
• Identify the requirements for interoperability testing.
• Recognize how the Test and Evaluation Master Plan (TEMP) generation, staffing and approval
process integrates all functional disciplines throughout the acquisition life cycle.
• Identify issues affecting T&E resource requirements, test planning, and test execution activities
in support of a program’s acquisition strategy.
TEMP Purpose
The Test and Evaluation Master Plan (TEMP) outlines the structure and objectives of the test and
evaluation program. It must be developed by and staffed with a wide variety of functional experts to
ensure the plan addresses all necessary technical, business, and resource issues. Moreover, the TEMP
links together, and must be consistent with, a number of related program documents such as the:
• Initial Capabilities Document (ICD)
• Analysis of Alternatives (AoA)
• Capability Development Document (CDD)
• Acquisition Program Baseline (APB)
• Acquisition Decision Memorandum (ADM)
• Threat Assessment (type determined by program Acquisition Category)
o Validated Online Lifecycle Threat (VOLT) Report or;
o System Threat Assessment (STA)
The TEMP is required at Milestone A and must be updated periodically to ensure that it stays current
and integrates the various disciplines as the program evolves through the life cycle. At a minimum,
these updates would coincide with subsequent milestones and other major decision reviews.
181

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the TEMP | Lesson Summary 2
TEMP Information
The TEMP is mandated by DoD policy. The recommended TEMP format contains the following
information:
• Four parts that serve as a starting point for organizing a successful test and evaluation program:
System Introduction, Test Program Management Schedule, Test and Evaluation Strategy and
Resource Summary.
• Critical Technical Parameters (CTPs) are key parameters and developmental testing criteria that
are derived from Warfighter capabilities specified in the Capability Development Document (CDD) and
from technical performance measures as specified by the System Engineering Plan. The CTPs are
developed, coordinated and approved by the T&E Integrated Product Team (IPT) within the
Program Management Office. Examples of CTPs are an aircraft’s engine thrust, cruising speed,
range and altitude. CTPs are measurable criteria that, if not achieved during DT&E, preclude
fulfillment of desired operational performance capabilities.
• Critical Operational Issues (COIs), are key operational effectiveness or suitability issues that
must be examined in operational test and evaluation to determine the system’s capability to
perform its mission. COIs must be relevant to the required capabilities and of key importance to
the system being operationally effective, operationally suitable and survivable, and represent a
significant risk if not satisfactorily resolved. COIs are critical elements or operational mission
objectives that must be examined, are related to Measures of Effectiveness (MOE) and
Measures of Suitability (MOS), and are included in the Test and Evaluation Master Plan (TEMP).
Generally, the resolution of each Critical Operational Issue (COI) during OT&E is in terms of the
evaluation of some MOE (and/or MOSs).
• Measures of Effectiveness (MOEs) are operational measures used to determine the degree to
which the system performs its mission. The data used to measure the military effect (mission
accomplishment) that comes from using the system in its expected environment. That environment
includes the system under test and all interrelated systems, that is, the planned or expected
environment in terms of weapons, sensors, command and control, and platforms, as appropriate,
needed to accomplish an end-to-end mission in combat.
• Measures of Suitability (MOSs), which are used to determine the degree to which the system is
usable in its intended environment, is a measure of an item’s ability to be supported in its
intended operational environment. MOS’s typically relate to readiness or operational availability
and, hence, reliability, maintainability, and the item’s support structure.
Types of T&E
Test and evaluation falls into several categories including developmental, operational, live fire, and
interoperability. Each plays a different role within the acquisition life cycle.
Developmental Test and Evaluation (DT&E):
• Is an integral part of the systems engineering process.
• Is conducted throughout design and development to ensure the system attains Critical Technical
Parameters (CTPs).
182

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the TEMP | Lesson Summary 3
Operational Test and Evaluation (OT&E):
• Helps determine system operational suitability and effectiveness.
• Addresses Critical Operational Issues (COIs) that are defined by the user.
Whenever possible, the Program Manager and test team should try to integrate DT and OT to save both
cost and schedule time that would otherwise be lost in a serial testing process.
Live Fire Test and Evaluation (LFT&E):
• Determines survivability of crew and/or system vulnerability.
• Confirms lethality of munitions/missiles against intended target set.
Interoperability Testing:
• Confirms interoperability requirements have been met through actual performance testing.
Milestone Test-Related Products
Various test-related products are required prior to each milestone decision:
Milestone A
• Approved TEMP
Milestone B
• Updated and Approved TEMP
• Identification of LRIP Quantities
• Live Fire T&E Waiver (when required)
• Early Operational Assessment (EOA) results (when required)
• DT&E Report
Post Critical Design Review Assessment
• Early Operational Assessment (EOA) results (when required)
Milestone C
• Updated and Approved TEMP
• DT&E Report
• Operational Assessment results
Full Rate Production Decision Review
• Updated and Approved TEMP
• DOT&E Report on IOT&E
• Live Fire T&E Report
• Interoperability Certification (certification of actual performance testing results by JITC)
183

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the TEMP | Lesson Summary 4
Developmental T&E Support Organizations

There are four key developmental T&E support organizations within DoD:
• Army Test and Evaluation Command (ATEC) – Developmental Test Command (DTC) supports
the materiel acquisition process for defense materiel by: Planning and conducting tests and
simulations across the full spectrum of environments (arctic, tropic, desert, shock, vibration,
electromagnetic, nuclear, underwater, live fire).
• Navy Systems Commands (NAVAIR, NAVSEA, SPAWAR, NAVFAC, and NAVSUP) – each of the
systems commands provides full life-cycle support for a specific category of military hardware
or software, including research and development, design, procurement, testing, repair, and in-
service engineering and logistics support. NAVSEA is concerned with ships and submarines,
NAVAIR – naval aircraft, SPAWAR – information technology, NAVFAC – shore facilities, and
NAVSUP – supply chain management.
• Marine Corps Systems Command (MARCORSYSCOM) serves as the Department of the Navy’s
systems command for Marine Corps ground weapon and information technology system
programs in order to equip and sustain Marine forces with full-spectrum, current and future
expeditionary and crisis-response capabilities.
• Air Force Materiel Command (AFMC) conducts research, development, test and evaluation, and
provides acquisition management services and logistics support necessary to keep Air Force
weapon systems ready for war.
Operational T&E Support Organizations

There are several key operational T&E support organizations within DoD:
• Army Test and Evaluation Command (ATEC) – Operational Test Command (OTC) is the
independent Operational Test and Evaluation (OT&E) authority for the U.S. Army and conducts
independent operational testing to inform acquisition and fielding decisions for Army and select
multi-service Warfighting systems.
• Navy Operational Test and Evaluation Force (OPTEVFOR) is the independent Operational Test
and Evaluation (OT&E) authority for the U.S. Navy and is responsible to the Chief of Naval
Operations (CNO) to provide objective assessments on the effectiveness and suitability of
systems under test in support of Navy and DoD acquisition programs, and how those systems
affect mission accomplishment.
• Marine Corps Operational Test and Evaluation Activity (MCOTEA) is the independent Operational
Test and Evaluation (OT&E) authority for the U.S. Marine Corps. MCOTEA plans, executes, and
evaluates testing of material solutions against warfighter capabilities, under prescribed realistic
conditions and doctrine, to determine Operational Effectiveness, Operational Suitability, and
Operational Survivability (OE/OS/OSur) of all new equipment for the Marine Corps in support
of its acquisition process.
184

https://en.wikipedia.org/wiki/Naval_ship

https://en.wikipedia.org/wiki/Submarines

https://en.wikipedia.org/wiki/Naval_aviation

https://en.wikipedia.org/wiki/Command_and_control

https://en.wikipedia.org/wiki/Naval_base

https://en.wikipedia.org/wiki/Military_supply_chain_management

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Developing the TEMP | Lesson Summary 5
• Air Force Operational Test and Evaluation Center (AFOTEC) is the independent Operational Test
and Evaluation (OT&E) authority for the U.S. Air Force. AFOTEC is responsible for testing,
under operationally realistic conditions, new systems being developed for Air Force and multi-
service use. They seek to answer questions about how safe, effective, reliable, maintainable,
compatible and logistically supportable new Air Force systems will be.
Interoperability Capabilities
Most systems today must be able to exchange information with other systems. For example, joint and
combined military operations require National Security Systems (NSS) that are interoperable across the
services and compatible with our allies. DoD acquisition policy requires such systems to establish a Net-
Ready Key Performance Parameter (NR-KPP) that identifies specific interoperability capabilities. The NR-
KPP ensures that the systems are able to:
• Provide and accept data, information, materiel, and services from other systems, units or forces
• Use interchangeable systems that operate effectively together
• Exchange information directly between themselves and/or their users
The program’s NR-KPP (requirement for interoperability capabilities) must be certified and validated
by the Joint Staff. Once the capability need is validated, the performance of the systems must be tested
to ensure that they meet the established interoperability capability needs.
The Joint Interoperability Test Command (JITC) is the DoD’s Joint Interoperability Certifier and the only
non-Service Operational Test Agency for Information Technology (IT/National Security Systems (NSS).
Through the use of risk-based test, evaluation and certification services, tools, and environments, JITC
assesses system interoperability performance and certifies full end-to-end interoperability of systems
upon successful completion of the testing.
Version 1.3 (30 June 2016)
185

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Environment Safety and Occupational Health Issues | Lesson Summary 1
2.6 Lesson Summary
Environment Safety and Occupational Health Issues
Learning Objectives
The following learning objectives are covered in this unit:
• Identify the information required for a milestone review regarding environment, safety, and
occupational health issues.
• Identify key federal and DoD policies governing environment, safety, and occupational health
issues associated with defense systems acquisition.
PESHE
There are many federal laws, Executive Orders, and other guidelines designed to minimize an acquisition
program’s impact on the environment. To ensure awareness, proper planning, and compliance, the
Programmatic Environment, Safety, and Occupational Health Evaluation, usually referred to as PESHE, is
an acquisition policy requirement for all ACAT programs. Support for a PESHE requires analyses in the
following areas:
• ESOH Compliance- describes procedures for determining compliance, defines compliance
requirements, and analyzes impact of compliance on the program’s cost, schedule and
performance.
• Safety and Occupational Health – describes procedures used to identify and eliminate hazards,
defines risk levels, and summarizes the impact of potential health and safety hazards, including
loss of life or program units.
• Hazardous Materials Management – outlines the goals of the hazardous materials program and
related issues, and includes the process for identifying, tracking, handling and disposing of
hazardous materials that cannot be eliminated.
• Pollution Prevention – describes pollution prevention initiatives and process for preventing or
minimizing impacts on natural resources.
• National Environmental Policy Act – NEPA requires preparation of detailed statements on major
federal actions significantly affecting the quality of the human environment.
The order of priority for handling hazardous materials is as follows:
1. Source reduction/eliminat ion by using alternative materials or processes.
2. Recycling or purification and reuse of material.
3. Treatment to neutralize waste products so that they are no longer hazardous.
4. Disposal through burning, landfills, or other means.
As a last resort, PMs can use remediation to clean up material that was improperly disposed of.
186

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Environment Safety and Occupational Health Issues | Lesson Summary 2
NEPA
The National Environmental Policy Act (NEPA) requires program managers to evaluate the environmental
impact of an acquisition program before making major decisions that could affect the environment. It
must be completed prior to a milestone review for programs that may affect the quality of the human
environment. The Component Acquisition Executive (CAE) is the final approval authority for system-
related documentation pertaining to NEPA and environmental Executive Orders.
Documenting these potential environmental impacts can take three different forms:
• Categorical Exclusion (CATEX): A document that indicates that neither an Environmental
Impact Statement (EIS) nor an Environmental Assessment (EA) is required.
• Environmental Assessment (EA): Considers any elements of the environment that might be
potentially impacted by the acquisition program. Typically, the EA is prepared in much the same
way as an Environmental Impact Statement (EIS), but is much shorter (often 25-50 pages) in
length. Generally an EA is required if the PM cannot determine the extent of the program’s
impact on the environment.
• Environmental Impact Statement (EIS): If significant environmental impacts are identified,
an EIS is drawn up to document the scope, cost, and potential damage of these impacts. This is
typically an extensive document of at least a couple hundred pages. It includes a Notice of Intent
that alerts the public to the fact that the Government is contemplating an action that could impact
the environment.
These documents are generally prepared by outside contractors with expertise in environmental issues.
EO 13693 (Greening the Government)
Starting in 1998, the last four Presidents have issued environmental policies via Executive Orders (EOs),
that direct DoD and all Government agencies to conduct their missions in an environmentally friendly
manner. This is sometimes referred to as “Greening the Government”.
The 1998 EO was superseded by EO 13423 in 2007. In 2009, the President issued EO 13514 which
supplemented and expanded EO 13423. In March 2015, the President issued EO 13693, Planning for
Federal Sustainability in the Next Decade. Although this new EO revoked EOs 13423 and 13514, it
expanded on the topics covered in the previous EOs and refined the required goals and objectives for
government agencies.
The gist of these EOs is to direct all Federal Agencies to increase energy efficiencies, reduce greenhouse
gases, conserve and protect water resources, prevent pollution and reduce the quantities of toxic and
hazardous chemicals and materials acquired, used, or disposed of by the agency. The agencies are charged
to develop plans and policies to acquire environmentally preferable products and services.
Although the EOs give DoD the authority to exempt weapon systems from their provisions, the waivers are
not automatic, and we are still charged to pursue these goals in systems acquisition, whenever possible.
Version 1.2 (30 June 2016)
187

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Programming Funds | Lesson Summary 1
2.7 Lesson Summary
Programming Funds
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify the basic flow of the financial management process, to include cost analysis, the
Planning, Programming, Budgeting and Execution (PPBE) process, Congressional enactment, and
program execution.
• Relate the following building blocks to the PPBE process: Future Years Defense Program (FYDP);
Major Force Program (MFP); and Program Element (PE).
• Identify the key events in the programming phase, including the preparation, review and decision
process associated with the two primary documents of the phase: Program Objectives
Memorandum (POMs) and Resource Management Decision (RMD).
• Given programming and budgeting documents, relate the applicable funding policies to each of
the five DoD appropriation categories of greatest interest to acquisition programs.
• Identify two exceptions to the full funding policy.
• Identify the concept of escalation in submitting program and budget documents.
Financial Management Process
The financial management process for defense systems acquisition operates as follows:
• It begins with the operational user’s capability need, first documented in the Initial Capabilities
Document (lCD) and later in the Capability Development Document (CDD).
• Following lCD approval, an Analysis of Alternatives (AoA) is conducted, Cost as an Independent
Variable (CAIV) trade-offs are made, and a program cost estimate is prepared to project resource
requirements.
• Cost, schedule and performance targets are identified in the Acquisition Program Baseline.
• The PPBE process is then used to translate plans and programs into a budget that the President
submits to Congress.
• Congress in turn authorizes programs and appropriates funds.
• Finally, budget authority is allocated through a series of steps to the services and defense
agencies, enabling them to execute their missions.
188

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Programming Funds | Lesson Summary 2
PPBE Process
The PPBE process is a calendar-driven process that helps DoD determine how to allocate resources. It
consists of the following:
• Planning phase: Planning examines national defense from a broad perspective in terms oflong-
term strategies, policies, and objectives. The end product of planning is the Defense Planning
Guidance (DPG), which provides input for the Programming phase.
• Programming phase: Programming translates planning decisions into time-phased resource
requirements. Through programming, military departments and defense agencies allocate
resources to support their roles and missions for the next five years in terms of money and next
eight years in terms of manpower. They submit their requirements in a Program Objectives
Memorandum (POM), which is amended and approved by OSD in Resource Management
Decisions (RMDs).In turn, programming decisions provide input during the concurrent Budgeting
phase.
• Budgeting phase: This phase is conducted concurrently with the review of the POMs from the
Programming Phase. Budgeting translates programming decisions into detailed resource
requirements for the next fiscal year. Each Military Department and Defense Agency produces a
Budget Estimate Submission (BES) derived from the first year of their POM. When approved by
the DEPSECDEF via RMDs, these ultimately become the DoD portion of the President’s Budget.
• Execution Review: The final activity of the PPBE process is the Execution review, which is
accomplished concurrently with the Program and Budget Reviews. The purpose of the Program
Review is to prioritize the programs which best meet military strategy needs; the purpose of the
Budget Review is to decide how much to spend on each of these programs; and the purpose of
the Execution Review is to assess what is received for the money spent (i.e., actual output versus
planned performance). Performance metrics are developed and used to assess actual output
against planned performance. These metrics are used to adjust resources to achieve goals.
Note: PPBE is an internal DoD process, but guidance from Congress in the form of ongoing
Congressional actions, e.g., passing an Appropriations Act that impacts the next PPBE cycle or directed
program terminations or program enhancements may impact the overall PPBE process.
Programming Phase Products
The most important products of the Programming phase are the Program Objectives Memoranda (POMs)
and the Resource Management Decisions (RMDs):
• Program Objectives Memoranda (POMs): Each year, the military departments and defense
agencies submit a combined POM and BES to OSD. The POM proposes a five year allocation of
resources to satisfy the Defense Planning Guidance (DPG). These POMs are reviewed by the Joint
Staff, who issue the Chairman’s Program Assessment (CPA), and by the OSD staff, who
recommend program changes through POM Issue Papers. The military departments and agencies
can comment on or reclama the issues raised by OSD.
• Resource Management Decisions (RMDs): The Deputy Secretary of Defense (DEPSECDEF)
makes decisions on the POMs and BESs submitted by the Services and defense agencies, and
documents his decisions in RMDs. The RMDs will be reflected in the Defense portion of the
President’s Budget submission.
189

Programming Funds | Lesson Summary 3
ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Programming and Budgeting Tools
There are several tools that provide data and structure for programming and budgeting.
• Future Years Defense Program (FYDP): The FYDP is a single database that summarizes all
forces, resources, and equipment associated with programs approved by the Secretary of
Defense. In addition to showing past and current funding and manpower levels, it shows funding
requirements for the next five years, as well as manpower requirements for the next eight years.
• Major Force Programs (MFP): The FYDP breaks data into eleven different major programs
that contain the total aggregation of resources necessary to achieve a mission objective, such as
General Purpose Forces or Research and Development. Each MFP is divided into program
elements.
• Program Elements (PE): PEs are the primary units of data in the FYDP, the smallest
aggregation of resources controlled by OSD. Represented by an eight to ten digit code, PEs are
considered to be the ”building blocks” of the programming and budgeting process.
Funding Policies
Funding policies are used to govern the PPBE process, and different policies apply to different
appropriation categories:
• Annual funding policy: Governs Operations and Maintenance (O&M) and Military Personnel
(MILPERS) funds. Annual funding policy requires that we request only the dollars that we need to
spend in order to operate, maintain, or pay the forces in a given fiscal year. This generally
pertains to routine expenses, for example equipment maintenance and labor costs.
• Incremental funding policy: Governs RDT&E funds and requires you to budget only for the
research and development effort that is needed during a given fiscal year. Emphasis is on
covering only those expenses to be incurred, based on the work expected to be accomplished
during that year.
• Full funding policy: Governs PROCUREMENT, MILCON, and SCN funds and provides for the
procurement of useable end items which must be delivered within a 12- month period after
delivery of the first item. Full funding requires us to budget sufficient funds to cover the total cost
to deliver a quantity of usable end items, such as aircraft, missiles, ships, or vehicles that can be
delivered in a future 12 month delivery period. Piecemeal procurement of systems is not
permitted.
Full Funding Policy Exceptions
There are two exceptions to the full funding policy:
• Advance procurement funds are set aside to buy certain components, material, or effort
before an end item is procured in order to avoid a serious break of continuity. For example,
advanced procurement might be used to obtain a long- lead time item to prevent a break in
production, or to maintain critical skills that might otherwise be lost between early and later
stages of a manufacturing process. Advance procurement funds are budgeted as a separate line
item, usually one fiscal year in advance of the funds budgeted for the related end item.
190

Programming Funds | Lesson Summary 4
ACQ 202 Intermediate Systems Acquisition Course (ISAC)
• Multiyear procurement can be used to acquire multiple years’ worth of equipment with a
single contract in order to reduce cost and maintain stability in the acquisition process. The
Government makes a commitment to the contractor to procure a specific quantity of a weapon
system over several years, thus giving the contractor incentive to realize savings, particularly
through economic order quantity (EOQ) purchases and investment in productivity enhancements.
Congress must approve all multiyear procurements.
Escalation
Since Program and Budget requests are projections into the future, they must take into consideration
possible market forces that will influence the economy. Escalation allows us to make predictions about
expected inflation and outlay rates for each year of the program. There are two types of dollars referred
to when we talk about escalation:
• Constant, or Base Year, dollars are tied to a specific year with no inflation across the life of a
program. Constant dollars are usually used for cost estimates because it makes it easy to make
changes across the year without considering the impact on the cost of money over time.
• Current, or Then Year, dollars include inflation and outlay rates to account for when the money is
actually supposed to be outlayed from the Treasury. This type of dollars is used for program and
budget documents and is found in the FYDP.
Escalation Indices
There are two types of indices used when we apply escalation:
• Compound, or Raw, indices relate price levels for each year to a baseline year. This is annual
compounding of inflation, similar to the way interest is received on a savings account. The
compound indices are used to convert dollars in one Base Year to dollars in another Base Year.
• Composite, or Weighted, indices factor in the historical outlay pattern of the appropriation and
inflation rates associated with the fiscal years when cash flows out of the US Treasury. Based on
this rate of outlay, appropriation expenses can be loosely predicted to provide a more accurate
budgeting picture. The composite indices are used to convert Base Year dollars to Then Year
dollars.
DoD publishes escalation indices at least twice a year for the services and defense agencies to use in
preparing PPBE input. Program and budget documentation is initially prepared in constant or Base Year
dollars and then escalated into current or Then Year dollars so that the funding requested in those future
years will be sufficient to pay for expenses that will be incurred in those years.
Version 1.0 (11 July 2014)
191

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part I) | Lesson Summary 1
2.8 Lesson Summary
RFP Preparations (Part I)
Learning Objectives
The following learning objectives are covered in this unit:
• Identify the complementary roles and responsibilities of the Contracting Officer and the Program
Manager in their partnership throughout the acquisition process.
• Identify the role of various Integrated Product Team (IPT) members in conducting market
research and developing the solicitation.
• Recognize the purpose and formats of the Integrated Program Management Report (IPMR).
• Select appropriate contract type based upon program risk.
• Identify current socioeconomic programs and determine their contractual consequences.
Program Manager and Contracting Officer Roles
The Program Manager is ultimately responsible for an acquisition program, but the PM must rely on the
Contracting Officer to enter into the business agreements needed to carry out that program. The
Contracting Officer serves as business advisor and is responsible for the following actions:
• Prepare and release solicitations (e.g., Request for Proposals (RFPs).
• Communicate with potential offerors and conduct negotiations with contractors.
• Ensure consistency with the Federal Acquisition Regulation (FAR), the Defense Federal
Acquisition Regulation Supplement (DFARS), and all other regulations, policies and laws.
• Prepare, award, and administer contracts and any modifications to the contracts, and terminate
contracts.
Market Research
The government must conduct appropriate market research before soliciting offers from potential
contractors. Various IPT members can participate. For example, technical Integrated Product Team (IPT)
members can evaluate existing commercial products and non-developmental items (NDI), which must be
considered as a primary source of supply. Cost analysts can provide input on proper contract pricing
information. The extent of market research will vary depending upon the value, complexity, and urgency
of the procurement.
192

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part I) | Lesson Summary 2
Socioeconomic Programs
The government has implemented a series of targeted socioeconomic programs to help small and
disadvantaged businesses related to historical economic disadvantage and underutilization of minority
and women-owned small businesses. These include the following:
• Small Business: A business which is independently owned and operated, but not dominant its
field, and meets the size requirements specified in Federal Acquisition Regulation (FAR) 19.102.
(The Small Business Administration (SBA) establishes size standards on an industry-by-industry
basis).
• Small Disadvantaged Business (SDB): A small business, which is at least 51% owned and
managed by a person or persons who are socially and economically disadvantaged.
• 8(a) Business: A SDB which has been approved by the SBA for participation in the 8(a)
program. Majority owners must be socially disadvantaged individuals, that is, members of a
group that has been subject to racial or ethnic prejudice or cultural bias.
• Economically Disadvantaged Women-Owned Small Business (EDWOSB): A small
business that is 51% owned by one or more women who are economically disadvantaged.
• Service-Disabled Veteran-Owned Small Business (SDVOSB): A small business that is
owned by a veteran who has incurred a service-related disability.
• Historically Underutilized Business Zone (HUB Zone) Business: A small business that
operates in a HUB Zone and 35% of its employees reside in zone.
Contracts greater than $3,000 but less than or equal to $150,000 are set-aside exclusively for small
businesses if at least two responsible small businesses can be expected to submit offers.
Contract Types
The type of contract determines how cost risk is shared between the government and the contractor, and
it can provide effective contractor incentives. The tradeoffs associated with contract type must be weighed
carefully before a solicitation is released.
Cost-Reimbursement Contracts
In cost-reimbursement contracts, the government pays all allowable, allocable, and reasonable costs
incurred on the contract, while the contractor promises to put forth their best effort.
Fixed-Price Contracts
In fixed-price contracts, the contractor promises to deliver on time and to meet contract specifications for
a negotiated price. As we move from cost-reimbursement towards fixed-price contracts, the contractor
assumes more of the cost risk and the government assumes less. On the other hand, cost-reimbursement
contracts require more government monitoring and administration than fixed-price contracts.
193

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part I) | Lesson Summary 3
Within these two broad categories of contract type are a number of common variations:
Firm Fixed-Price (FFP):
• Negotiated fixed-price is not subject to any adjustment, regardless of the cost.
• Contractor bears all cost risk; has maximum incentive to control cost.
• Minimum administrative burden for contractor and government.
• Most appropriate when the requirement is well-defined and a fair and reasonable price can be
established at the outset.
Fixed-Price Incentive (FPI):
• Parties negotiate a target cost, target profit, share ratio and ceiling price prior to contract award.
• Government pays all allowable, allocable, and reasonable costs up to the ceiling price.
• Based on the contractor’s cost overrun or underrun and the share ratio, the target profit is
adjusted upward or downward upon contract completion.
• Government will not pay beyond the negotiated ceiling price regardless of cost incurred.
• Contractor must deliver on time and meet all specifications.
Cost-Plus-Fixed-Fee (CPFF):
• Contractor is reimbursed for all allowable, allocable and reasonable costs incurred plus the
negotiated fee.
• Fee is negotiated prior to contract award and is not adjusted regardless of cost incurred.
• Contractor has minimum incentive to control costs.
Cost-Plus-Incentive-Fee (CPIF):
• Parties negotiate a target cost, target fee, share ratio, maximum fee and minimum fee prior to
contract award.
• Contractor agrees to provide “best effort” to deliver the product or service.
• Based on contractor’s cost overrun or underrun and the share ratio, the target fee is adjusted
upward or downward upon contract completion.
• Based on the will not be paid fee exceeding the negotiated maximum fee but will be paid all
allowable, allocable and reasonable costs
• Contractor is assured the minimum fee regardless of the extent of the cost overrun and is paid
for all allowable, allocable and reasonable costs.
194

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part I) | Lesson Summary 4
Cost-Plus-Award-Fee (CPAF):
• Consists of a base fee ranging from 0 to 3% and an award fee pool allocated to award fee
periods.
• Each award fee period emphasizes different elements on which the contractor should focus
• Government makes subjective, unilateral decision on how much fee to award for each award fee
period.
• Requires government to make periodic performance evaluations of the contractor.
• Highly administratively burdensome to the government.
• Award fee may also be used as an add-on-incentive with other types of contracts.
Acquisition Regulations and Policy
The PM is responsible for managing the program in accordance with the DoD 5000 series of acquisition
policy, while the Contracting Officer is responsible for contract management in accordance with the FAR.
Thus, the two must work closely together and understand their respective roles throughout the life of the
program.
Integrated Program Management Report
Earned value management reports are available to help the PM track the contractor’s cost, schedule, and
performance against a Performance Measurement Baseline.
The Integrated Program Management Report (IPMR):
• Required for all cost or incentive contracts greater than or equal to $20 million. For contracts
below $20 million, the decision to use EVM is based on a risk assessment.
• Contains seven formats of information.
o The government requires EVM System (EVMS) reporting data (Integrated Program
Management Report (IPMR) – all 7 Formats) for cost and incentive contracts greater than
or equal to $50 million.
o EVMS reporting data in the form of IPMR Formats 1, 5, 6 and 7 are required for cost and
incentive contracts greater than or equal to $20 million but less than $50 million (Formats
2, 3, and 4 are included at the optional discretion of the Program Manager).
o In some circumstances, the Program Manager may require EVMS data for cost or incentive
contracts below $20 million. Although there is no requirement, a recommended optional
application would include only Formats 1, 5 and 7. In some instances, Format 6 may be
recommended as well.
o EVM reports are discouraged on Firm-Fixed Price and Time and Material contracts.
The IPMR is useful in providing objective data about the status of contractor performance. It identifies
current problems, emerging problems, and their potential cost and schedule impact. The Program
Manager should determine the formats to be reported based on such considerations as value of the
contract, complexity of the effort, and past performance of the contractor.
Version 1.2 (31 December 2015)
195

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part II) | Lesson Summary 4
2.9 Lesson Summary
RFP Preparations (Part II)
Learning Objectives
The following learning objectives are covered in this unit:
• Identify the aspects of the Joint Capabilities Integration and Development System (JCIDS) as it
applies to acquisition of Information Technology (e.g., interoperability, architecture, and
reuse).
• Identify the policy and concepts involved in the acquisition of data rights.
• Identify key laws and software acquisition management policies that are required for the
acquisition of a DoD automated information system.
• Identify “best practices” that may be appropriate for the acquisition of software-intensive
systems.
• Identify key discriminators for selecting the most capable software developer.
• Identify DoD policy regarding Basic Quality Systems and the role of ISO 9001.
Quality Systems
ISO 9001
As a result of acquisition reform in 1994, the ISO 9001 series of International Quality Standards has been
implemented by many contractors, shifting the focus to preventing problems in quality rather than
repairing them after they have occurred.
ISO 9000 deals with the fundamentals of quality management systems, including the eight management
principles on which the family of standards is based (customer focus; leadership; involvement of people;
process approach; systems approach to management; continual improvement; factual approach to
decision making; and mutually beneficial supplier relationships). ISO 9001 deals with the requirements
that organizations wishing to meet the standard have to fulfill. Third party certification bodies provide
independent confirmation the organizations meet the requirements of ISO 9001.
Contractor’s Choice
However, DoD guidance allows contractors to use the quality assurance process of their choice, as long
as it meets program objectives and does the following;
• Establishes capable processes
• Continuously improves processes
• Monitors and controls critical processes and product variation
• Has feedback mechanisms in place to assess field product performance
• Implements effective root cause analysis and corrective action systems
196

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part II) | Lesson Summary 4
Although the Government cannot require that a contractor be ISO 9001 compliant, a contractor can be
asked to provide an equivalent quality assurance system in place, with similar characteristics to those
listed above. The intent of the ISO 9000 series of standards, and other quality standards is to require
companies to manage quality as a fundamental focus of their business.
Automated Information System Risk Reduction
There are special risks associated with the acquisition of an automated information system. As a result,
DoD guidance states that it is preferable for software developers too:
• Have a successful past performance record, experience in the domain or product line, a mature
software development process, and evidence of adequate training in software development tools
and environments.
• Develop system architectures that support open system concepts, exploit existing commercial
products, and provide for incremental improvements based on modular, reusable and extensible
software.
• Identify and exploit software reuse opportunities before beginning new development initiatives.
• Select a programming language based on overall life-cycle costs, risks, and interoperability
potential.
• Use DoD standard data.
• Use a software measurement process to plan and track the software development program.
Software Developer Capability Evaluation
The CMMI models can be used throughout the acquisition lifecycle by industry as well as Government.
Although process areas depict the characteristics of an organization committed to process improvement, you
must interpret the process areas using an in-depth knowledge of CMMI, your organization, the business
environment, and the specific circumstances involved (STUDENT NOTE: Refer to the CMMI-DEV document
linked to this lesson for greater detail on CMMI-DEV model).
To interpret practices, it is important to consider the overall context in which these practices are used and to
determine how well the practices satisfy the goals of a process area in that context. CMMI models do not
prescribe nor imply processes that are right for any organization or project. Instead, CMMI describes minimal
criteria necessary to plan and implement processes selected by the organization for improvement based on
business objectives.
CMMI practices purposely use nonspecific phrases such as “relevant stakeholders,” “as appropriate,” and “as
necessary” to accommodate the needs of different organizations and projects. The specific needs of a project
can also differ at various points in its life.
197

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part II) | Lesson Summary 4
When selecting a contractor to develop software, the Government can evaluate their capability using a
Standard Capability Maturity Model Integrated (CMMI) Appraisal Method for Process Improvement
(SCAMPI). Based on the CMMI model used, a SCAMPI evaluation enables a sponsor to:
• Gain insight into an organization’s capability by identifying the strengths and weaknesses of its
current processes relative to appraisal reference model(s)
• Focus on improvements (correct weaknesses that generate business risks) that are most beneficial
to the organizations given its current level of process implementation
• Identify risks relative to achieving capability or maturity targets
The Government can also use the SCAMPI to assess the maturity of their own internal software
acquisition and development processes.
Using CMMI Appraisals
Many organizations find value in measuring their progress by conducting an appraisal and earning a maturity
level rating or a capability level achievement profile. These types of appraisals are typically conducted for
one or more of the following reasons:
• To determine how well the organization’s processes compare to CMMI best practices and identify
areas where improvement can be made
• To inform external customers and suppliers about how well the organization’s processes compare to
CMMI best practices
• To meet the contractual requirements of one or more customers
Appraisals of organizations using a CMMI model must conform to the requirements defined in the
Appraisals Requirements for CMMI (ARC) document. Appraisals focus on identifying
improvement opportunities and comparing the organization’s processes to CMMI best practices.
Appraisal teams use a CMMI model and ARC-conformant appraisal method to guide their evaluation of
the organization and their reporting of conclusions. The appraisal results are used (e.g., by a process
group) to plan improvements for the organization.
Software Acquisition Best Practices
DoD has identified a number of key best practices to follow in the acquisition of software. They include:
• Identify and manage risk continuously throughout the life of the system.
• Estimate cost and schedule empirically.
• Use metrics to monitor risk, identify problems, and base decisions.
• Track earned value.
• Establish quality targets and track defects against those targets.
• Treat people as your most important resource.
• Implement a sound configuration management process.
198

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part II) | Lesson Summary 4
• Manage and trace requirements to the lowest level.
• Use system-based software design to document and evaluate the process.
• Ensure data and database interoperability.
• Define and control all internal and external interfaces.
• Design twice, code once.
• Address the risks of reusing existing software, whether commercial or non-developmental items.
• Inspect requirements and design, subject to configuration management products to
formal inspection.
• Conduct continuous testing based on plans, pass-fail criteria, and traceable procedures.
• Compile and smoke test frequently.
Interoperability Requirements
In today’s military environment, systems must be interoperable in order to be effective; that is, they must
be able to exchange data. To ensure interoperability, all systems acquired by DoD that will produce, use,
and exchange information must be consistent with the DoD Information Technology Standards Registry
(DISR).
The DISR provides a common set of mandatory standards for information processing, transfer, modeling,
interfaces, and systems security. The interoperability requirements are captured in the program’s Net-
Ready KPP. In addition to compliance with DISR, all systems, regardless of ACAT, must undergo a two-
step oversight process to ensure all interoperability capabilities are identified and met.
Interoperability Capability Certification
This process, based on the capability needs identified by the user in the ICD and CDD, ensures that we
consider interoperability from the very beginning. Before the capabilities can be approved for a system,
the Joint Staff must certify that interoperability capabilities are identified and consistent with joint
policy, architectural integrity, and interoperability standards.
Interoperability Certification
This process is used to demonstrate, based on actual performance tests conducted in the field,
that interoperability capabilities have been met. After completion of this testing, the Joint
interoperability Test Command (JITC) issues a Letter of Certification to document that the
required level of interoperability performance was achieved.
Data Rights – Basic Principles
As outline in the program’s Intellectual Property (IP) Strategy, the Government should acquire the
appropriate rights to data, software, intellectual property, and other documentation to facilitate
competition over the life of the system. The key to success is ensuring requirements and strategies
are identified and acted on early in the acquisition to avoid unexpected costs to procure, reformat,
or deliver data. Important basic principles are summarized on the following page:
199

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
RFP Preparations (Part II) | Lesson Summary 4
• Data deliverables or data items are delivered in response to contract requirements for data that has
pre-determined content and format. Generally, there is no automatic delivery of data, nor does the
Government have any automatic rights to technical data.
• To receive data or have access to data the Government must explicitly request it in the contract
using a Contract Data Requirements List (CDRL).
• In general, the contractor continues to own the actual data or intellectual property; government
owns the deliverable and has license rights that govern how the data may used.
Data rights fall under the following categories:
Unlimited rights
If the Government has funded the entire development of an item, then it is entitled to unlimited rights to
use, duplicate, or disclose technical data for any purpose.
Limited rights
If a contractor has developed an item entirely at its own expense, then the government is only entitled to
limited rights, within the Government itself, and normally cannot release the data to other parties outside
the Government.
Restricted Rights
These rights only apply to noncommercial computer software, and are similar to limited rights.
An example would be restricting usage of a computer program to only one computer at a time.
Government Purpose Rights
When technical data is developed with mixed funding (part contractor and part government), government
purpose rights allow the Government to use the technical data for Government purposes as described in
limited rights and for other purposes such as competition, but not for commercial applications.
Government purpose rights are automatically effective for five years and revert to Unlimited Rights upon
expiration of the five-year period.
Version 1.3 (30 Septemeber 2015)
200

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Source Selection | Lesson Summary 1
3.1 Lesson Summary
Source Selection
Learning Objectives
The following learning objectives are covered in this unit:
• Identify the complementary roles and responsibilities of the Contracting Officer and the Program
Manager in their partnership during source selection.
• Differentiate among the various types of interaction between the Government and contractors,
e.g., discussions, clarifications, deficiencies, communications, and exchanges.
• Identify the role and responsibility of the participants in fact-finding and negotiations.
• Identify how to prepare for and conduct a fact finding activity.
• Identify how to prepare for and support a negotiation.
• Recognize the importance of contractor finance principles to the defense acquisition process.
• Identify how the balance sheet and income statement portray the operating characteristics and
health of a business.
• Differentiate generally between direct cost and indirect cost.
• Identify how indirect costs are allocated to a contract.
• Identify the five bases for cost allowability.
• Recognize the purpose and application of forward pricing rates to government contracts.
• Explain how corporations use their organization to implement their business strategies.
• Recognize how company organizations can create a competitive advantage.
• Recognize different typical company organizational models used between small and large
businesses.
201

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Source Selection | Lesson Summary 2
Source Selection Information Exchanges
Throughout the source selection process, IPT members must take care to protect the interests of both
the Government and the contractors competing for the work. Government personnel must be careful not
to disclose procurement sensitive or proprietary information to unauthorized personnel and to avoid any
exchange that would give an advantage to any one offeror.
Fact-Finding Information Exchanges
After proposals are received and initially evaluated against the source selection factors and subfactors by
the Source Selection Evaluation Board, the Contracting Officer determines whether or not to hold
discussions with the offerors in order to achieve the best value to the government. Only the most highly
rated proposals are included in the “competitive range.” Throughout the process, the Contracting Officer
conducts fact-finding activities to gain a complete understanding of the proposals and identify specific
areas of concern which include ambiguity, weaknesses, or deficiencies. There are several types of
information exchanges involved in fact-finding.
Clarification
If no discussions are anticipated, then the Government may request comments from the offeror on any
negative past performance information to which they have not seen or been allowed to comment on
previously. These are called clarifications and are also used to clarify minor clerical errors.
Communication
In order to establish the competitive range of the most highly rated proposals the Contracting Officer
may have exchanges known as communications. Communications can be used to resolve uncertainties
about specific proposals, to correct minor clerical errors, and to explain any negative past performance
information prior to establishing the competitive range.
202

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Source Selection | Lesson Summary 3
Discussion, Negotiation, Bargaining
Negotiations
Negotiations are exchanges, in either a competitive or sole source environment, between the government
and offerors. The intent of negotiations is to allow offerors to revise their proposals.
Bargaining
Negotiations may include bargaining. Bargaining includes the use of persuasion, the potential alteration
of assumptions and positions, give-and-take, and may apply to price, schedule, technical requirements,
contract type, or other terms of a proposed contract.
Discussion
When negotiations are conducted in a competitive environment, they take place after establishment of
the competitive range and are called discussions. Discussions are tailored to each offeror’s proposal and
are conducted by the contracting officer with each offeror in the competitive range. The purpose is to
indicate or discuss significant weaknesses, deficiencies, and other aspects of the offeror’s proposal in
order to allow the contractor to make changes to their proposal. These changes to the proposal may
enhance the offeror’s potential for award. The primary objective of discussions is to maximize the
government’s ability to obtain best value based on the capability need and source selection evaluation
factors.
Communication and negotiations between the government and the contractor must always go through
the Contracting Officer
Allowable Costs
During the source selection process, IPT members may be called upon to help evaluate price and cost-
related factors. This information helps ensure that the contractor selected has the financial means
necessary to perform the work. If a firm already has an existing, forward pricing rate agreement, their
contract rates don’t need to be evaluated for later contracts. However, the costs included in a contract
must be evaluated to determine whether they are allowable.
For a cost to be allowable, it must meet five criteria. The cost must:
• Be reasonable, that is, the cost does not exceed the cost that a prudent business person would
incur in a competitive environment for a similar item.
• Be allocable to the contract, that is, meet any one of the following conditions:
o The cost is incurred specifically for the contract;
o The cost is beneficial to both the contract and to other work, and it can be distributed
between the two in reasonable proportion; or
o The cost is necessary to the overall operation of the business although a direct relationship to
a particular contract cannot be shown.
• Comply with applicable Government Cost Accounting Standards (CAS) and Generally Accepted
Accounting Principles (GAAP). These are rules normally used for estimating and reporting costs.
• Be consistent with the terms of the contract. The Government and the contractor can agree that
certain costs will be considered unallowable.
203

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Source Selection | Lesson Summary 4
• Be consistent with the cost principles identified in the Federal Acquisition Regulation (FAR), which
designate certain costs as allowable, partially allowable, or unallowable.
Forward Pricing Rate Agreements (FPRA)
Forward pricing rate agreements (FRPA) are written agreements negotiated between a contractor and
the Government to make certain rates available during a specified period for use in pricing contracts or
modifications. These rates represent reasonable projections of specific costs that are not easily estimated
for, identified with, or generated by a specific contract, contract end item, or task. These projections may
include rates for such things as labor, indirect costs, material obsolescence and usage, spare parts
provisioning, and material handling. They provide consistency in proposal pricing and save time.
• Purpose–An FPRA is used to ensure a fair and reasonable price earned by a contractor and to
protect a government agency from being charged unfairly. The contractor estimates these rates
based on reasonable standards. The contracting government agency must approve them before
the agreement is signed.
• Process–Typically, forward pricing rates are estimated by using a percentage or ratio. The
percentage or ratio is based upon unforeseen differentials in prices. When the bill is issued, the
costs are multiplied by this percentage or ratio. The rate protects the contractors by allowing an
extra amount above and beyond the estimated quoted prices. The rate represents costs
projected for material and labor costs, for example.
• Guidelines–A contractor should submit an FPRA proposal each year. These agreements should
also state what time period the rate is good for. The rate should be fair and be used only for
labor, indirect costs, material and other items that are not easy to estimate.
Direct and Indirect Costs
Costs incurred by a contractor can be classified as direct or indirect.
• A direct cost is a cost incurred by the contractor due to a single contract. Direct costs are often
divided into direct material and direct labor costs. An example of a direct cost is the cost of a
component purchased exclusively for use on a Government contract.
• An indirect cost is a cost incurred by the contractor that cannot be attributed solely to a single
contract. Indirect costs include support costs for continued operations. There are two categories
of indirect costs: overhead and general & administrative.
• Overhead costs support a specific part or function of the company but not the whole company.
An example of an overhead cost is the cost of factory maintenance that can be shared
proportionally between specific manufacturing jobs.
• General and Administrative (G&A) costs are required to support operation of the entire
company. An example of a G&A cost is the salary of the chief executive officer.
Financial Statements
Financial statements can help the Government assess the financial health of a company. Two key
financial statements are the:
204

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Source Selection | Lesson Summary 5
• Balance Sheet shows in monetary terms a company’s assets (things of value owned by the
firm), liabilities (claims against those assets) and owners’ equity, at a particular point in time.
• Income Statement shows a company’s revenue and expenses incurred over a period of time,
such as a fiscal year.
Financial Indicators
Two helpful indicators of a company’s financial condition are the profitability ratios of return on sales, or
ROS, and return on total assets, or ROA:
• Return on Sales (ROS): Also known as profit margin, ROS is calculated by dividing net income
for an accounting period by revenue. For example, if net income was $15,000 and sales were
$300,000, then ROS would be 15,000/300,000 or 5%.
• Return on Assets (ROA): ROA measures the efficiency of the firm’s investment in assets and
their ability to generate revenue. It is calculated by dividing net income for an accounting period
by the total dollar value of the assets shown on the balance sheet at the end of the year. For
example, if net income was $6,000 and total asset value at the end of the year was $150,000,
ROA would equal 6,000/150,000 or 4%.
Both ROA and ROS should be used carefully. Both calculations provide an indicator of a firm’s financial
health, but variations may be due to unusual accounting events. If a firm has an unusually low ROA or
ROS compared with the overall industry, it is important to find out why.
Business Acumen
Organizational Structure
A company’s organizational structure facilitates the implementation of its strategy. The company should
be organized so that its efforts and resources are focused on implementing and achieving its strategic
goals. A company that is organized to remain focused on its mission and with the built-in flexibility to
respond to change is going to have a competitive advantage over companies with less effective
organizational structures.
Types of Organizational Structures
• Functional structure classifies people according to the function they perform in the
organization and may include positions such as President, Sales Department, Customer Service
Department, etc.
• Divisional structure is based on different divisions within an organization, such as different
product lines, different markets, or different geographical locations.
• Matrix structure is a combination of the functional and divisional structure. This creates an
efficient organizational structure; however, it is the most complex because often there is blurring
of the lines of authority.
205

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Source Selection | Lesson Summary 6
Company Size and Organizational Structure
• Smaller companies often concentrate functions in staff organizations to support all business
areas or product lines, or outsource core functions such as human resources, accounting, or
legal.
• Larger companies often decentralize staff functions, dedicating them to each business or
product organization.
Version 1.0 (11 July 2014)
206

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Technical Risk Management | Lesson Summary 1
3.2 Lesson Summary
Technical Risk Management
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify the role of systems engineering in balancing cost, schedule and performance
throughout the lifecycle.
• Use Technical Performance Measures to track progress in program risk areas during
systems development.
• Identify the role of modeling and simulation as a tool in the systems engineering process.
• Recognize the importance of modeling and simulation in the defense acquisition process.
• Identify the role of the WBS in the systems engineering process.
• Identify how T&E supports the systems engineering process.
Systems Engineering Process
Systems engineering is a problem-solving process that translates capability needs into designs to
provide a new or improved capability. This process must take into consideration such factors as
producibility, supportability, testability and interoperability to achieve a well-balanced design. Applied
within the Integrated Product and Process Development (IPPD) management process, systems
engineering brings multiple disciplines together to determine the optimal solution to satisfy capability
needs. It is an iterative process throughout a system’s development that evolves through a series of
steps, from stakeholder requirements definition through architecture design to verification, validation
and then transition of the developed system to the User.
The systems engineering process is used to manage the technical risk inherent in development and
production of a system. While technical risk has a direct impact on the performance of a system, it also
affects program cost and schedule. Done properly, systems engineering can also be used to reduce
program cost and schedule risk. A number of tools are available to help mitigate technical risk, including
modeling and simulation, work breakdown structure, and technical performance measurement.
Modeling and Simulation
Modeling and simulation is an essential part of Simulation Based Acquisition (SBA). SBA involves
integrating modeling and simulation across many functional disciplines throughout the acquisition life
cycle. Thus, modeling and simulation can be used to support capability needs definition, concept
refinement, system design, manufacturing, and testing.
207

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Technical Risk Management | Lesson Summary 2
Modeling and simulation offer a number of advantages. Virtual prototypes and simulations provide a
common vision of a system, show the complex interactions among parts of a design, and identify the
potential effects of alternative approaches without physically changing a system. They allow designers,
logisticians and manufacturers to collaborate on the same design using a common platform or shared
database.
Through modeling and simulation, IPT members can better understand the relationships among
components and evaluate alternatives in a virtual environment. As a result, modeling and simulation can
save time and money, improve the quality of hardware and software, produce integrated product
designs, and help make better program decisions.
Work Breakdown Structure
A work breakdown structure (WBS) can be used to support a wide range of technical, business, and
management functions. The WBS displays and defines the product to be developed, breaking down the
overall system into its component parts. For technical management, it helps to identify and assess high-
risk elements, establish key interface control requirements, evaluate Engineering Change Proposals
(ECPs), and determine the number and type of technical reviews and audits required. The WBS also is
used to develop the Statement of Work (SOW) and determine the contract line items (CLINs) that
specify contract deliverables.
One of the outputs of the systems engineering process is a draft physical architecture, which serves as
the basis for the “product” part of the WBS. In a typical WBS, the products are displayed vertically on
the left hand side, while the processes that support those products are displayed horizontally on the
right.
Technical Performance Measures
Technical Performance Measures (TPMs) reduce technical risk by tracking certain selected performance
parameters over time to identify potential performance problems during system development. TPMs are
used to monitor the progress of the most critical, high-risk technical areas. For example, speed and
weight might be tracked as TPMs in the development of a new land combat vehicle. TPMs compare
actual values against expected values over time to identify problems before they become too difficult or
costly to solve.
208

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Technical Risk Management | Lesson Summary 3
Test and Evaluation (T&E)
The program manager and systems engineer use the technical management processes as insight and
control functions for the overall technical development of the system throughout the acquisition life cycle.
They use the technical processes to design, create, and analyze the system, system elements, and enabling
system elements required for production, integration, test, deployment, support, operation, and disposal.
As part of the overall systems engineering process, Test and Evaluation is the process by which a system or
components are compared against requirements and specifications through testing. The results are
evaluated to assess progress of design, performance, supportability, etc.
• Verification determines whether or not the system meets its “build-to” specifications. In other
words, “Did you build it right?” Verification is usually associated with Developmental Testing
(DT).
• Validation determines whether or not we “built the right thing” and entails the performance
of the system in its intended (operational) environment. Validation is usually associated
with Operational Testing (OT).
Developmental test and evaluation is an engineering tool used to reduce risk throughout the defense
acquisition cycle. Operational test and evaluation is the actual or simulated employment, by typical users,
of a system under realistic operational conditions.
Version 1.1 (30 June 2016)
209

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Trade-Off Analysis | Lesson Summary 1
3.3 Lesson Summary
Designing for Supportability/Trade-Off Analyses
Learning Objectives
The following learning objectives are covered in this unit:
• Identify the role of life cycle product support and systems engineering in balancing cost
schedule, and performance throughout the life cycle.
• Identify the key policy provisions that relate to how life cycle product support and systems
engineering are performed in the Department of Defense.
• Apply life cycle product support and systems engineering efforts through the Designing
for Supportability approach to determine a design solution to meet an operational need
that demonstrates the balancing of Affordability objectives and technical activities.
• Identify key acquisition best practices, including commercial practices that impact the relationship
between Government and industry.
• Identify life cycle product support resource requirements and understand why it is important
to influence system design for supportability.
• Identify Product Support Analyses (PSA) tools/best practices/techniques available in
the systems engineering process to achieve the principal supportability goals.
• Relate design, operational and supportability characteristics to systems readiness
objectives.
• Identify the relationship of Reliability, Availability, and Maintainability (RAM) and
Supportability to life cycle product support, and its impact on system, subsystem and
component performance, operational effectiveness, logistics footprint and life-cycle cost.
• Select appropriate management methods and techniques to achieve Reliability, Availability,
and Maintainability (RAM) and Supportability parameters.
• Apply the trade-off study process to evaluate alternatives.
• Apply a selected quantitative tool (e.g., decision matrix) to support a decision.
Supportability
Supportability refers to the inherent design characteristics of reliability and maintainability and the efficacy of
the product support package required for operating and maintaining the systems, equipment or component
throughout its life cycle
Life cycle product support requirements drive the design and development of reliable, maintainable
and affordable systems through the continuous application of the systems engineering methodology.
210

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Trade-Off Analysis | Lesson Summary 2
Supportability directly affects operational readiness as well as operations and maintenance costs. In
general, over 65 – 80% of system costs are incurred after the system is fielded/deployed, and most of
those costs are already fixed by the time first milestone approval is obtained. Therefore, we must
consider system support early and continuously throughout a system’s development.
Designing for Supportability
Designing for supportability incorporates the principles of systems engineering throughout the system
life cycle to design, develop, produce, and sustain operationally reliable, supportable, and effective
systems.
Trade-Off Analyses
Trade-off analyses examine alternatives among requirements and designs at the appropriate level of
detail to support decision making and lead to a proper balance between performance and cost.
Open Systems
One design approach that promotes supportability is open systems architecture, which enables us to use
standard design features and interfaces that are compatible with products from multiple suppliers. This
approach uses non-proprietary interfaces and protocols and industrial standards to provide interoperable
components and portability. Open systems design facilitates technology insertion and product
modification by taking advantage of standardization. It also results in lower life cycle costs, with a greater
number of suppliers available to compete to meet our needs.
Reliability, Availability Maintainability, and Supportability
Reliability, Availability Maintainability (RAM) and Supportability are important design and operational
characteristics should be addressed early in the acquisition process and throughout the life cycle. The
goals of RAM and Supportability are higher operational effectiveness and lower life cycle costs.
Reliability
Reliability is how long an item or system will perform its function before it breaks. It is measured in Mean
Time Between Failure (MTBF). Reliability is the probability that a system will perform its function within
stated time and performance conditions. Poor reliability will reduce readiness, increase logistics support
requirements, increase life cycle costs, and waste manpower. However, redundancy, the use of back-up
systems or parts, can increase reliability.
Maintainability
Maintainability is how quickly, easily and cost effectively a system can be returned to operational status
after preventative or corrective maintenance is performed. It is measured by Mean Time to Repair
(MTTR), or how quickly and easily a system can be fixed. Maintainability is a consequence of the design
process, so initial engineering efforts are vital to creating a maintainable product.
Human Systems Integration
One determinant of maintainability is Human Systems Integration (HSI), which has several aspects:
• Accessibility: Can the part be easily accessed for repair?
• Visibility: How easily can you see the part being worked on?
211

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Trade-Off Analysis | Lesson Summary 3
• Testability: How easy is it to test and detect faults?
• Standardization: Are parts interchangeable, and can standard tools be used?
The more user-friendly the design, the faster the repair and upkeep can be performed.
Supportability
Supportability is the degree to which a system’s design and planned logistics resources support its
readiness needs and wartime utilization. Unlike reliability or maintainability, supportability
includes activities and resources (such as fuel) that are necessary whether the system fails or not. It
also includes all resources, such as personnel and technical data that contribute to the overall support
cost.
The PM can address the long versus short term issue by designing for the optimal balance between
performance (technical and supportability), life-cycle costs, schedule, and process efficiency. The Affordable
System Operational Effectiveness (ASOE) concept is important because it is what the user sees in
terms of how well the system is able to perform its missions over a sustained period as well as the ability to
surge given the user’s operating budget. In this concept the emphasis is not only on the system’s ability
to execute its mission or its reliability and maintainability, but also on the cost effective responsiveness of
the supply chain.
• The concept of ASOE is used to address the interplay between system performance, availability
(reliability, maintainability, and supportability), process efficiency (system operations, maintenance,
and life cycle product support), and system life cycle cost. This overarching perspective provides a
context for the “trade space” available to a PM to increase reliability and drive down life cycle costs.
• ASOE is the composite of performance, availability, process efficiency and Life Cycle Cost (LCC).
The objectives of ASOE are to influence early design via the combination of the System Design for
Operational Effectiveness (SDOE) approach and the Supply Chain Model (SCM) as contributors to
LCC. SDOE focuses on the impact of reliability and maintainability design parameters and their
role in operational effectiveness and suitability requirements; SCM focuses on the logistics
activities that enable effective sustainment.
Availability
Availability is the heart of mission readiness. Obviously, the more reliable and maintainable an item,
the greater its availability
The presence of a solid RAM and Supportability ensures system readiness by ensuring Operational
Availability (AO). AO is measured as a ratio of the time a system is able to be up and running to the total
time a system is required (AO = Uptime/Total Time). When a system is not able to be up and running,
its downtime can be attributed to:
• Logistics delays−parts out of stock
• Administrative delays−personnel or paperwork delays
• Corrective maintenance −making repairs
• Preventive maintenance −routine service
Product Support Analysis (PSA)
Because RAM and Supportability are so important, we must evaluate them throughout the design and
development process. Supportability analyses are jointly conducted by the Systems Engineering and Life Cycle
Logistics communities to address the impact of the design characteristics of reliability and maintainability on
212

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Trade-Off Analysis | Lesson Summary 4
the design and the product support system. The product support package equates to the resources and
infrastructure required to support systems in the field. PSA, an iterative and multidisciplinary process, is used
as part of the life cycle product support and systems engineering processes to influence design as well as
determine the most cost effective way to support the system throughout its life.
A number of tools are available to evaluate supportability, including:
• Failure modes and effects criticality analysis (FMECA): Examines each failure to
determine and classify its effect on the entire system.
• Reliability centered maintenance (RCM): Uses a scheduled maintenance approach
to identify failures before they degrade system effectiveness.
• Level of Repair Analysis: Level of Repair Analysis LORA is a prescribed procedure for defense
logistics planning. LORA is performed to determine the best, most efficient location where an item
can be repaired.
Analysis of product support – this portion of the PSA process commences at the system level to affect design
and operational concepts; identify gross product support resource requirements of alternative concepts; and
to relate design, operational, and supportability characteristics to system readiness objectives and goals.
Trade-Offs
Creating a supportable design that is also producible, testable, and affordable involves making trade-offs
among competing features. A decision matrix can be used to systematically compare choices by
selecting, weighting and applying criteria.
A decision matrix has eight steps:
1. Identify the items to be compared.
2. Establish evaluation criteria (e.g., reliability, cost, etc.).
3. Assign weight to each criteria based on its relative importance.
4. Establish a quantitative rating scheme (e.g., scale from 1 to 5).
5. Rate each item on each criteria using the established rating scheme.
6. Multiply the rating for each item by the assigned weight for each criteria.
7. Add the totals for each item.
8. Find the highest score which determines the best value.
Choices Reliability (MTBF) 0.6
Cost ($)
0.3
Maintainability (MTTR)
0.1 Totals
BIT 01
(MTBF = 150; $ = 8K; MTTR = 3 hrs) 1 X 0.6 = 0.6 3 X 0.3 = 0.9 1 X 0.1 = 0.1 1.6
BIT 02
(MTBF = 175; $ = 10K; MTTR = 2
hrs)
2 X 0.6 = 1.2 2 X 0.3 = 0.6 2 X 0.1 = 0.2 2.0
BIT 03
(MTBF = 250; $ = 11K; MTTR = 1
hrs)
3 X 0.6 = 1.8 1 X 0.6 = 0.3 3 X 0.1 = 0.3 2.4
Version 1.2 (31 December 2015)
213

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Software Design | Lesson Summary 1
3.4 Lesson Summary
Software Design
Objectives
The following learning objectives are covered in this lesson:
• Recognize the relationship between software development activities and the systems engineering
process.
• Identify common ways that software-intensive projects have gotten into trouble.
• Given a software-intensive system (such as a telecommunications or guidance system), select an
appropriate software development methodology.
• Identify typical software development lifecycle activities and standards.
• Using DoD Practical Software Measurement methodology principles, select appropriate software
measures to make sound decisions regarding acquisition of software intensive systems.
System Architecture
The structure or architecture of a defense information system can be viewed in three different ways.
• Operational Architecture describes how the system meets the end-user’s or warfighter’s
information needs.
• Systems Architecture shows the “physical” structure and information flows.
• Technical Architecture describes how hardware and software components interact to satisfy
user requirements.
Software Development Considerations
The development and integration of software is a complex and challenging aspect of system acquisition.
Some points to consider:
• All new and upgraded command, control, communications, computer, and intelligence (C4I)
systems must be in compliance with the DoD Information Technology Standards Registry (DISR).
• Commercial software components that are already DISR-compliant can be used to save time and
are usually easier to maintain and upgrade.
• Identifying system requirements is one of the most important aspects of software development.
• Software modification doesn’t just affect the software itself: hardware issues also need to be
explored to determine the impact of software modifications on the total system.
214

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Software Design | Lesson Summary 2
Sources of Software Problems
Software development can become difficult due to a variety of problems, many of which are within the
control of the program manager. Typically, software problems come from the following sources:
• Poor requirements definition
• Lack of user involvement
• Poorly-defined architecture and interfaces
• Overlooking hardware deficiencies
• Failure to establish a functional team of vendors, experts, and end users
Software Development Methods
Tradeoffs must be made when selecting the engineering/development approach to take in acquiring
software. For example, we might make a choice between modifying existing software or undertaking a
new development. In addition, different methods can be used to develop software:
• The Waterfall method is based on a top-down approach. It requires extensive formal
documentation, which can be time consuming. This approach is often used late in the life cycle,
and it is best used for systems with relatively stable requirements.
• The Incremental method requires strong configuration and requirements management. It is
best utilized when budget or schedule constraints impact the final product such that additional
features could be added later, if needed.
• The Spiral model incorporates extensive prototyping to ensure proper risk management. It is
best used in situations where the system is unstable and user capability needs are not clear or
have not been properly defined.
Software Measurement Techniques
Different measurement techniques are available to track software development progress. These
measurement techniques fall into three categories:
• Process metrics deal with the maturity and robustness of organizational processes that are
used to develop software. They examine qualifies such as process maturity, developer
productivity, amount of rework required, and the impact of technology.
• Quality metrics are concerned with software product attributes that can impact performance,
user satisfaction, supportability, and ease of change. They are used to track attributes such as
software integrity, reliability, usability, maintainability, interoperability, and flexibility.
• Management metrics compare actual progress against plans. These indicators can suggest
trends, detect trouble early, or trigger the need to make adjustments to plans so that they are
more realistic. Management metrics deal with questions regarding scheduling, personnel,
requirements volatility, cost performance, and individual work unit progress.
Version 1.1 (30 Sep 2015)
215

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Commercial and NDI | Lesson Summary 1
3.5 Lesson Summary
Commercial and NDI
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify key issues regarding Test and Evaluation (T&E) of commercial and non-developmental
(NDI) items.
• Identify the role of Early Operational Assessment (EOA) in reducing program risk.
• Recognize key logistics related acquisition policies and their impact (e.g., life-cycle cost,
contractor logistics support, commercial and non-developmental items).
Non-Developmental Items and Commercial Items
Non-developmental items (NDI) are previously developed items used exclusively for governmental
purposes by federal, state, local, or allied governments. Commercial items are generally used for non-
governmental purposes and are offered for sale, lease or license to the general public.
Benefits from Using NDI and Commercial Items
The use of non-developmental items and commercial products is encouraged to reduce life cycle costs
associated with having to develop new products or systems. Use of these types of products doesn’t
completely eliminate testing and supportability issues, but it can drastically cut development costs.
The benefits of using NDI and commercial products include:
• Reduced cycle time
• Reduced/eliminated R&D cost
• Reduced technical, cost and schedule risk
• Availability of product samples for source selection process
• Availability of state-of-the-art technology
Drawbacks from Using NDI and Commercial Items
On the other hand, there can be drawbacks to using NDI and commercial products:
• Difficulty in integrating components
• Long-term logistics support problems
• Lack of engineering and test data
216

Commercial and NDI | Lesson Summary 2
ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Testing
The amount and type of testing required for an NDI or commercial item depends on how the item will be
used, whether any modifications are needed, and the availability of previous test results.
• If the item will be used in the same environment for which it was originally designed,
developmental testing is usually not necessary. However, operational testing will be required if
the item will be maintained by the Government.
• If the item will be used in a different environment than that for which it was originally designed,
some developmental testing may be required to ensure the item meets specifications or make
sure the manufacturing process is effective. Operational testing, including early operational
assessment (EOA) and operational assessment (OA), will be required to verify effectiveness and
suitability.
• If the item will be integrated into a system, developmental testing will be required on a test
sample before the item is integrated into the system. Pre-production testing of the complete
system, including both hardware and software, may be conducted. Operational testing of the
complete system will also be required.
• If the item will be modified, both developmental testing and operational testing will be conducted
to insure the modification meets all the requirements.
Modifications
Making government unique modifications to commercial or non-developmental items may invalidate
testing and usage data. The more we modify these items, or change the way in which they will be used,
the more additional testing we will need to conduct.
Early Operational Assessment
Operational testing and evaluation (OT&E) is the primary means of assessing weapon system
performance. One type of OT&E, Early Operational Assessment (EOA), is conducted to forecast and
assess potential operational effectiveness and suitability of the weapon system during development. It is
used to detect deficiencies that may impact the performance, availability, and supportability of a system.
Thus, EOA increases our confidence in the NDI or commercial item, thereby reducing our probability of
failure, which in turn reduces risk.
The use of NDI and commercial items raises long-term supportability issues. For example, we could face
a situation where the vendor changes the product line or discontinues making replacement parts. In
addition, there may be problems with design interface and the interoperability of parts with the overall
system. Furthermore, service unique logistics capability needs may be difficult to meet with commercial
and NDI products.
Organic or Contractor Logistics Support
When deciding to use commercial or NDI items, we must determine how best to support the system once
it is fielded; i.e., whether to use organic support using military personnel or to contract out logistics
support.
Both options have their merits and drawbacks, and determining these can be done by taking into account
the following circumstances:
217

Commercial and NDI | Lesson Summary 3
ACQ 202 Intermediate Systems Acquisition Course (ISAC)
• How much modification is required to make the item fully operational? If significant changes are
required before the item is used by the military, then government (organic) logistics support
might be the best approach.
• How or where will the item be used? If the environment will be hostile or austere, it could affect
the contractor’s ability to support the item due to safety concerns, and government (organic)
logistics support might be the best approach.
• What is the projected service life? For short-term items, contractor logistics support is often
more appropriate.
• How stable is the design or configuration? If constantly changing configurations are inevitable,
especially due to advances in technology, then contractor logistics support is likely to be the
better option.
Version 1.1 (30 September 2015)
218

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Role of Manufacturing | Lesson Summary 1
3.6 Lesson Summary
Role of Manufacturing
Learning Objectives
The following learning objectives are covered in this lesson:
• Recognize the impact of manufacturing on cost, schedule, and performance.
• Recognize the relationship of manufacturing to the systems engineering process.
• Identify the methods and objectives of manufacturing that influence system design.
• Distinguish from among the types of trade-offs that may be required to attain a producible
design.
Benefits of a Producible Design
Manufacturing considerations impact the systems engineering process by influencing the design for
producibility. This results in a more robust, balanced design that is cheaper and easier to build. A
producible design is more stable and leads to a higher quality product that can be introduced more
quickly at lower overall cost. Manufacturing a product with high producibility will reduce assembly errors,
repair costs, labor time and wasted material. By designing for producibility up front, manufacturing costs,
which usually account for about 13%-25% of total system life cycle costs, can be significantly reduced.
Designing for Producibility
The following methods may be used to achieve a producible design:
• Use standard components
• Use parts designed for ease of fabrication
• Use multifunctional parts
• Use a modular approach
• Minimize assembly and handling requirements
• Minimize the total number of parts
A balanced design must take into consideration the inevitable trade-offs that must be made among
various functional areas. Some considerations include:
• Changes made late in the development process or early in the production process are usually the
most expensive.
• The highest risk of failure is most likely to occur in the transition from system development to
production.
219

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Role of Manufacturing | Lesson Summary 2
• A product can usually be produced by different methods, each with its own set of costs, and the
optimum method should be determined early in the design process.
• Most costs associated with manufacturing are inherent in the design.
Manufacturing Trade-Offs
Manufacturing trade-offs are made throughout the design process among three areas: producibility, cost,
and operational requirements. Changes in one can affect the other two, so each trade-off needs to be
fully considered before being implemented. In doing so, trade-offs between different product
characteristics need to be evaluated. Trade-offs in cost, for example, involve examining the development
of alternative designs, required technology and the required industrial base capability. Environmental
concerns, factory and support facilities, and the 5 manufacturing elements or “5 Ms” (manpower,
machinery, measurement, methods, and materials) are also important trade-off considerations.
Version 1.1 (30 September 2015)
226220

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Earned Value Management | Lesson Summary 1
3.7 Lesson Summary
Earned Value Management
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify the steps in the development of the initial Performance Measurement Baseline (PMB).
• Identify the relationship of the PMB to program objectives.
• Identify the purpose and content of the Integrated Program Management Report (IPMR).
• Identify performance report tailoring considerations and their effect on reporting.
• Recognize the importance of Earned Value as a management tool.
Earned Value
As you learned in a previous lesson, earned value is an important management tool that is used to
monitor and manage the contract and/or project performance by emphasizing the planning and
integration of program cost, schedule and performance factors. Although the contractor may choose
whatever management system it deems necessary, that system must comply with established American
National Standards Institutes’ earned value management guidelines (ANSI/EIA-748).
Performance Measurement Baseline
Earned value provides one of the best ways to identify problems, take corrective action, and measure the
actual cost of the work accomplished against the planned schedule and cost of the project. This requires
the establishment of a Performance Measurement Baseline (PMB), which integrates the integrated master
schedule (IMS), the contract work scope, and the contract budget. This baseline is also referred to as the
Budgeted Cost of Work Scheduled (BCWS).
The initial basis for the PMB comes from the negotiated cost of the contract and does not include profit or
fee. Prospective contractors will estimate cost, schedule, and performance risks after reviewing the scope
of work as defined by the Government in the Statement of Work (SOW) or Performance Work Statement,
as appropriate. Upon being awarded the contract, the negotiated contract cost provides a good starting
point in developing the PMB. Development of the PMB needs to take place immediately to help manage the
project.
221

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Earned Value Management | Lesson Summary 2
PMB Development
Using this initial estimate, the PMB is then developed in three steps:
Step 1
In Step 1, the contractor defines or scopes all work to the control account level, using a Work Breakdown
Structure (WBS). The control account level is the lowest level of functional responsibility within the
contractor’s organization. Each control account is assigned to a Control Account Manager (CAM).
Step 2
In Step 2, the contractor creates a detailed schedule or time phased work plan. Each CAM builds a PMB
for their respective control account by breaking the work down into work packages and planning
packages. Work packages list the detailed job or material items that will be needed to accomplish the
required work in the control account, while planning packages identify and budget work expected to be
done in the future.
Step 3
In Step 3, the contractor develops a budget for the work scheduled. Each CAM establishes a budget
estimate for their control account, which is compared to the other CAM estimates relative to the
negotiated cost of the contract. The contractor’s project manager then assigns dollar amounts to each
CAM based on a comparison of budgeted needs versus available funds.
During this process, the contractor’s project manager may withhold a small amount of the overall budget
to cover any unknown costs that might arise later in the project. This budgeted dollar amount is known
as Management Reserve, or MR.
Integrated Program Management Report
Integrated Program Management Report (IPMR) is used to report earned value data and contractor
performance to the PM on all cost or incentive contracts greater than or equal to $20M. The IPMR has
seven formats, which provide information on different aspects of the contractor’s performance:
Format 1: Work Breakdown Structure
Format 1 contains current and cumulative performance element data broken out by Contract WBS. Any
schedule or cost variances that exceed the negotiated dollar or percentage thresholds require a narrative
explanation on Format 5.
Format 2: Organizational Categories
In Format 2 contracting efforts are broken down by organizational category.
222

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Earned Value Management | Lesson Summary 3
Format 3: Baseline
Format 3 displays time-phased budgets, showing current period, cumulative value to date, the next six
months, and five additional specified periods which take the contract to completion. Changes to future
budget periods, application of management reserve, and distribution of Undistributed Budget are also
listed here.
Format 4: Staffing
In Format 4 staffing projections for the organizational categories found in Format 2 are listed here, as
well as the data for the current period, cumulative, the next six months, and five specified periods
extending to contract completion.
Format 5: Explanation and Problem Analyses
Format 5 explains the history of the current status and any actions being taken to address problems that
have arisen. It addresses the overall contract status, significant schedule and cost variances between
planned and actual achievements, reasons for baseline changes, and rationale for use of management
reserve. In response to the contract requirements, the contractor program manager should provide future
risk management assessments. This information provides input to the government program manager for
future program risk management.
Format 6: Integrated Master Schedule
Format 6 defines and contains the contractor’s Integrated Master Schedule (IMS) and is mandatory for all
contracts requiring EVMS. The IMS shall include, at a minimum, discrete tasks/activities, consistent with
all authorized work, and relationships necessary for successful contract completion.
Format 7: Electronic History and Forecast File
Format 7 defines the supplemental historical and time-phased information in the DoD-approved electronic
XML format, by WBS, provided at the same level as the Format 1 (unless otherwise specified in the CRDL)
and is mandatory for all contracts requiring EVMS. This time-phased historical and forecast cost
submission data is intended to enhance Government analysis beyond the information provide in Format 5
and is required to be submitted at least annually.
IPMR Purposes
DoD will also use the IPMR data for the following purposes:
• Integrate cost and schedule performance data with objective technical measures of performance.
• Identify the magnitude and impact of realized and potential performance problems area that may
cause significant cost and schedule variances.
• Provide valid, timely, and accurate contract status information to Government leadership.
EVM is NOT required on Firm Fixed Price (FFP) contracts. EVM is NOT required for any contract with a
period of performance of less than 12 months, regardless of value. The use of EVM is NOT recommended
for non-schedule based contracts such as Level of Effort (LOE) and Time and Material (T&M) contracts.
223

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
A summary of IPMR Format requirements by contract type and value is provided below:
CONTRACT IPMR REQUIRES DURATION
Type Value ($) Formats Period of Performance
Firm-Fixed Price (FFP)* At Any Value REQUIRED None Not Applicable
FFP – Level of Effort (LOE) At Any Value REQUIRED None Not Applicable
Other – Time & Material (T&M) At Any Value REQUIRED None Not Applicable
Any Cost Plus and/or Fixed-
Price Incentive
Greater than or
equal to
$100 Million**
REQUIRED
All Formats (1 thru 7)
Greater than
12 months
Any Cost Plus and/or Fixed-
Price Incentive
Greater than or
equal to
$50 Million BUT
less than
$100 Million
REQUIRED
All Formats (1 thru 7)
Greater than
12 months
Any Cost Plus and/or Fixed-
Price Incentive
Greater than or
equal to
$20 Million BUT
less than
$50 Million***
REQUIRED
Formats 1, 5, 6, & 7;
OPTIONAL
Formats 2, 3, & 4
Greater than
12 months
Any Cost Plus and/or Fixed-
Price Incentive
Less than
$20 Million
REQUIRED
None (optional)
If optioned for use by
PM – Formats 1, 5, & 7
are recommended;
Format 6 may be
recommended as well
Greater than
12 months
* Format 6 may be required for FFP contract.
** $100M or greater requires formal EVMS validations and performance of DCMA surveillance.
*** All cost plus and/or fixed-price incentive contracts greater than $20 million must be compliant with
EIA-748.
Version 1.3 (31 December 2015)
Earned Value Management | Lesson Summary 4
224

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Budgeting Process | Lesson Summary 1
3.8 Lesson Summary
Budgeting Process
Learning Objectives
The following learning objective is covered in this lesson:
• Identify the key events in the budgeting phase, including the preparation, review and decision
process associated with the three major documents of the phase: Budget Estimate Submission
(BES), Resource Management Decision (RMD), and Reclamas.
Budgeting Phase
The budgeting phase of the PPBE process focuses on program execution to determine near-term funding
requirements. Budgeting is a calendar-driven process, resulting in the DoD portion of the President’s
Budget, which is submitted to Congress in February each year.
Program Objectives Memorandum and Budget Estimate Submission
The services prepare their combined Program Objectives Memorandum (POM) and Budget Estimate
Submission (BES). The POM and BES update the Future Years Defense Program (FYDP). The BES covers
one year (such as FY 12).
The BES is submitted to the OSD Comptroller. Occasionally the OSD Comptroller will send a list of
“Advance Questions” about specific areas of the budget.
Draft Resource Management Decision
In the fall, after receiving responses to the advance questions, analysts from the OSD Comptroller and
the Office of Management and Budget (OMB) hold hearings to review appropriations or specific
programs. The analysts typically examine program pricing and phasing, compliance with funding policies,
and budget execution. After reviewing these areas, the OSD Comptroller analyst may prepare a draft
Resource Management Decision (RMD). The draft RMD is used to make adjustments to the BES,
generally reducing the amount of funding.
Reclama
The draft RMD is provided to the services and defense agencies for comment, at which point they are
allowed to provide an alternate position, known as a reclama. A reclama provides an opportunity to
explain problematic areas in the budget and refute proposed budget cuts. Reclamas should always be
based on fact and provide an objective evaluation of the implications of the proposed cuts.
225

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Budgeting Process | Lesson Summary 2
Final RMD
After considering the reclama, the OSD analyst makes the decision whether to withdraw, amend, or
submit the original version of the RMD. If not withdrawn, this final draft version of the RMD will include
all information regarding the original RMD and the associated reclama. It is then sent to the DEPSECDEF,
who ultimately makes the decision to sign off, thus finalizing the RMD.
Execution Review
While programming and budgeting are ongoing, the Execution Review phase is also ongoing. The results
of execution review will be used to make decisions about how to best allocate resources.
President’s Budget
The RMD and changes that occur during programming will be incorporated as part of the DoD portion of
the President’s Budget. The FYDP is then updated to reflect the President’s Budget, thus ending the
budgeting phase of the PPBE process.
Version 1.0 (11 July 2014)
226

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Design Changes | Lesson Summary 1
4.1 Lesson Summary
Design Changes
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify how instability of user capability needs, design, and production processes impact
program cost and schedule.
• Identify the purpose of specific technical reviews and their relationship to the acquisition process.
• Identify the roles, responsibilities, and methods for interface control and technical data
management.
• Recognize how configuration management impacts all functional disciplines (e.g., test, logistics,
manufacturing, etc.).
• Identify the impact on configuration management when commercial items are used in the
system.
• Relate the different types of program unique specifications to their appropriate configuration
baselines and technical review requirements.
• Trace the maturation of system design information as it evolves through the acquisition life cycle
of a system.
• Identify the relationship between configuration baselines, specifications, and configuration
management planning.
• Identify key acquisition best practices, including commercial practices that impact the relationship
between government and industry.
Technical Reviews
Technical reviews are conducted throughout the acquisition life cycle to reduce program risk. They are
event-driven, not schedule-driven, and help determine whether to proceed with development or
production.
With the exception of the PDR and CDR, which are mandatory technical reviews for any program that is
not initiated at MS C, not every type of technical review is required for every program. The type and
number of technical reviews can be tailored and the key to this tailoring is determining what technical
reviews are appropriate given the circumstances of your particular program.
Technical reviews are used to clarify design requirements, assess design maturity, and evaluate the
system configuration at various points in the development process. They provide a forum for
communication across different disciplines in the system development process and establish common
configuration baselines from which to proceed to the next level of design.
227

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Design Changes | Lesson Summary 2
Some of the Major Types of Technical Reviews
• System Requirements Review (SRR), a formal system-level review in which the system
specification is evaluated to ensure that system requirements are consistent with the preferred
concept and available technologies and that a mutual understanding between the government
and the contractor exists.
• Systems Functional Review (SFR), a formal review of the conceptual design of the system to
establish its capability to satisfy the requirements. It establishes the Functional baseline.
• Preliminary Design Review (PDR), in which the top-level design for each configuration item
function and interface is evaluated to determine if it is ready for detailed design. It confirms the
Allocated baseline. Unless waived by the MDA, a PDR is required prior to MS B and Program
Initiation.
• Critical Design Review (CDR), assesses design maturity, design build-to-code or code-to
documentation and remaining risks and establishes the initial product baseline. During the EMD
phase, the CDR will be used as the decision point in which the detailed Product Baseline is
evaluated to determine if system design documentation is good enough to begin production
(hardware) or final coding (software). It confirms the initial Product baseline.
• Test Readiness Review (TRR), in which test objectives, procedures and resources are evaluated
to determine if the system is ready to begin formal testing.
Configuration Management
Configuration management is one of the technical management processes that is used in the systems
engineering process to control the design of a product as it evolves from a top-level concept into a highly
detailed design. Through configuration management, we ensure that designs are traceable to
requirements, interfaces are well defined and understood, change is controlled and documented, and
product documentation is consistent and current.
Configuration management involves development of program unique specifications and other technical
data to document the design. As design requirements are finalized at different levels of detail,
configuration baselines are established to formally document those requirements and to define an item’s
functional and physical characteristics. The baselines progress from the overall system level (functional
baseline) to the more specific configuration item level (allocated baseline) and then down to the detailed
level (product baseline).
The highest level baseline is the functional baseline. The functional baseline includes the overall system
specifications, such as the mission and technical performance requirements. It also includes system to
system, or external interface definitions. This level of baseline defines system level requirements and is
documented primarily by the ‘system’ specification.
The next level of baseline is the allocated baseline. The allocated baseline defines design requirements for
configuration items below the system level. This baseline includes Item Performance specifications, which
describe the performance characteristics of a “configuration item,” including its form, fit, and function. This
level of baseline, often referred to as the ‘design to’ spec, does not include detailed requirements, but
focuses on the specifications necessary for the system to meet performance parameters, including interface
requirements.
228

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Design Changes | Lesson Summary 3
Finally, the lowest level of baseline that concerns the Government is the product baseline. The product
baseline includes the design requirements that specify how to build the system, including details of
processes (hardware and software), procedures, and materials used in fabrication. This level of baseline is
often referred to as the ‘build to’ baseline and is usually controlled by the contractor.
Baseline Specifications Examples for Firebird UAV
Functional
(“system” specification)
Overall system performance
requirements, including interfaces
Hardware and software requirements
analysis for the Night vision
requirement
Allocated
(“design to” specification)
Item performance specification.
Performance characteristics of
specific configuration items,
including form, fit, function
requirements.
Specific light level and resolutions
that are required of a digital camera
for the night vision capability.
Hardware and software interface
requirements for camera to attach
and interoperate with air vehicle.
Product
(“build to” specification)
Item detail specifications.
Hardware and Software process,
procedure, material details,
technical documentation.
Camera shutter design details.
Video transport circuit detailed design
including required software coding.
Drawing showing locking mechanism
for camera body.
The Government must determine which baselines should come under Government control. Generally
speaking, the Government maintains control of the functional, or system-level baseline; either the
Government or contractor; can maintain the allocated baseline; and the contractor is usually
responsible for the product, or ‘build-to’ level baseline and below.
Interface Management
Interface management involves the control and definition of the boundaries at which product subsystems
come into contact with other components of the system. Effective interface management involves
identifying, developing and maintaining the external and internal interfaces necessary for system
operation. Interface management can become a configuration management challenge when a product is
modified.
The contractor is usually responsible for design and control of internal interfaces, while the Government
is responsible for external interfaces.
Interface Control Working Group
An Interface Control Working Group (ICWG) is often used to establish formal communication links
between Government and contractor personnel involved in system interface design.
229

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Design Changes | Lesson Summary 4
Supporting the System
Once a system is fielded, configuration management documentation becomes the basis for supporting the
system, whether that support is provided by the contractor or by the Government. Interoperability and
maintenance issues can become very problematic if configuration management isn’t done properly. Even
minor changes to a commercial item can create configuration challenges and impact logistics, testing,
production and other functional areas.
Technical Data Package
The contractor will ultimately document the functional, performance, and physical characteristics of their
product in a Technical Data Package (TDP). Ensuring that the TDP is comprehensive and updated
regularly is especially important if the Government is going to maintain or modify the system.
Version 1.2 (31 March 2015)
230

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Software Problems | Lesson Summary 1
4.2 Lesson Summary
Software Problems
Learning Objectives
The following learning objectives are covered in this lesson:
• Apply a generic problem-solving model to an acquisition situation.
• Apply one or more selected qualitative tools (e.g., fishbone diagram) to resolve a problem.
• Identify developer practices essential for creation of high quality software.
• Identify the requirements for interoperability testing.
Cause and Effect Diagram (Fishbone Diagram)
One problem-solving technique is the cause and effect diagram or “fishbone” diagram. By analyzing all
the possible causes of a problem, the fishbone diagram focuses on determining the root cause of a
problem, rather than on symptoms or solutions. Typically, the fishbone diagram begins with a statement
of the problem in a box on the right side of the diagram-the “head” of the fish. Then categories of major
causes are identified and drawn to the left-the “bones” of the fish. These major causes are broken down
into all the related causal factors that might contribute to the major causes. Finally, the causal factors are
examined and narrowed down to the most significant elements of the problem to determine the ultimate
cause or causes.
231

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Software Problems | Lesson Summary 2
Software Best Practices
The Software Program Managers Network has identified several software best practices based on
interviews with software experts and industry leaders. Here is a synthesized list of some of those
characteristics, which are essential for the creation of high quality software:
Adopt Continuous Program Risk Management
Risk management is a continuous process beginning with the definition of the concept and ending with
system retirement. Risks need to be identified and managed across the life of the program.
Estimate Cost and Schedule Empirically
Initial software estimates and schedules should be looked on as high risk due to the lack of definitive
information available at the time they are defined.
Use Metrics to Manage
All programs should have in place a continuous metrics program to monitor issues and determine the
likelihood of risks occurring. Metrics information should be used as one of the primary inputs for program
decisions.
Track Earned Value
Earned value requires each task to have both entry and exit criteria and a step to validate that these
criteria have been met prior to the award of the credit. Earned value credit is binary with zero percent
being given before task completion and 100 percent when completion is validated.
Track Defects against Quality Targets
All programs need to have pre-negotiated quality targets, which is an absolute requirement to be met
prior to acceptance by the customer. Programs should implement practices to find defects early in the
process and as close in time to creation of the defect as possible and should manage this defect rate
against the quality targets. Meeting quality targets should be a subject at every major program review.
Treat People as the Most Important Resource
A primary program focus should be staffing positions with qualified personnel and retaining this staff
through the life of the project. The program should not implement practices (e.g., excessive unpaid
overtime) that will force voluntary staff turnover. The effectiveness and morale of the staff should be a
factor in rewarding management.
232

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Software Problems | Lesson Summary 3
Adopt Life Cycle Configuration Management
All programs, irrespective of size, need to manage information through a preplanned configuration
management (CM) process. This discipline requires as a minimum:
• Control of shared information
• Control of changes
• Version control
• Identification of the status of controlled items (e.g., memos, schedules) and
• Reviews and audits of controlled items
Manage and Trace Requirements
Before any design is initiated, requirements for that segment of the software need to be agreed to.
Requirements need to be continuously traced from the user requirement to the lowest level software
component.
Use System-Based Software Design
All methods used to define system architecture and software design should be documented in the system
engineering management plan and software development plan and be frequently and regularly evaluated
through audits conducted by an independent program organization.
Ensure Data and Database Interoperability
All data and database implementation decisions should consider interoperability issues and, as
interoperability factors change, these decisions should be revisited.
Define and Control Interfaces
Before completion of system-level requirements, a complete inventory of all external interfaces needs to
be completed. Internal interfaces should be defined as part of the design process. All interfaces should be
agreed upon and individually tested.
Design Twice, Code Once
Traceability needs to be maintained through the design and verified as part of the inspection process.
Design can be incrementally specified when an incremental release or evolution life cycle model is used
provided the CM process is adequate to support control of incremental designs.
Assess Reuse Risks and Costs
The use of reuse components, COTS (Commercial Off-The-Shelf), GOTS (Government Off- The-Shelf) or
any other non-developmental items (NDI) should be a primary goal, but treat any use as a risk and
manage it through risk management.
233

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Software Problems | Lesson Summary 4
Inspect Requirements and Design
All products that are placed under CM and are used as a basis for subsequent development need to be
subjected to a formal inspection defined in the software development plan. The program needs to fund
inspections and track rework savings.
Manage Testing as a Continuous Process
All testing should follow a preplanned process, which is agreed to and funded. Every test should be
described in traceable procedures and have pass-fail criteria.
Compile and Smoke Test Frequently
Smoke testing should qualify new capability or component only after successful regression test
completion. All smoke tests should be based on a traceable procedure and run by an independent
organization (not the engineers who produced it). Smoke test results should be visible and provided to all
project personnel.
Interoperability Testing
Interoperability problems can best be identified through the use of actual, live systems to mitigate risk.
Joint interoperability is defined as the ability of systems to provide services to and accept services from
other systems and to use the services exchanged to enable them to operate effectively together.
Through actual testing, Joint Interoperability Test Command (JITC) is responsible for verifying the
interoperability of systems to the performance parameters outlined in the lCD, CDD and ISP.
Version 1.1 (01 December 2014)
234

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
4.3 Lesson Summary
APB Breaches
The following learning objectives are covered in this Lesson:
• Identify when program deviations occur and the actions that should be taken by the Program
Manager.
• Relate the Acquisition Program Baseline (APB) to planning, control and risk management in
attaining cost, schedule and performance goals.
Program Deviation
A program deviation occurs when the Program Manager has reason to believe that the current estimate
for a given cost, schedule or performance parameter does not meet the threshold value specified for that
parameter in the Acquisition Program Baseline.
The PM must follow certain procedures whenever this occurs:
• The PM must immediately inform the Milestone Decision Authority (MDA) when a program
deviation occurs.
• Within 30 days of the deviation, the PM must explain to the MDA the reason for the deviation
and what steps need to be taken to bring the program back on track.
• Within 90 days of the deviation, one of the following scenarios must take place:
o The program is brought back on track; or
o A new APB is approved, changing only the parameters that were deviated; or
o An OIPT-level review is conducted to evaluate the PMs proposed baseline revisions, and
feedback is given to the MDA, or in the case of a major program, to the Defense Acquisition
Executive; or
o If it’s not possible for at least one of these actions to take place within 90 days, then the
MDA should hold a formal program review to determine the status of the program.
Interrelatedness of APB Parameters
Cost, schedule, and performance parameters are interrelated, and a change in one parameter can affect
the others. For example, the materials needed for a lighter aircraft may cost more and take longer to
design and manufacture than materials in a heavier aircraft. In that case, performance would affect both
cost and schedule parameters. Therefore it is important to involve all the key stakeholders when
considering changes to the APB.
Version 1.1 (30 June 2015)
APB Breaches | Lesson Summary 1
235

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Reprogramming Funds | Lesson Summary 1
4.4 Lesson Summary
Reprogramming Funds
The following learning objectives are covered in this lesson:
• Select the appropriate public law (i.e., Misappropriation Act, Anti-Deficiency Act, Bona Fide Need)
that applies to the use of appropriated funds under specific circumstances.
• Given a funding shortfall, apply the rules governing reprogramming of appropriated funds in each
appropriation category to resolve the problem.
• Identify the role of Early Operational Assessment (EOA) and Operational Assessment (OA) in reducing
program risk.
• Identify the risks and benefits associated with integrated Developmental Testing/Operational Testing
(DT/OT).
Appropriated Funds Laws
Congress has passed laws to ensure the proper use of the funds they make available for defense
acquisition programs:
Misappropriation Act
The Misappropriation Act states that funds appropriated by Congress can only be used for the programs
and purposes for which the appropriation was made. Using Research, Development, Test and Evaluation
(RDT&E) funds to pay for the procurement of items, for example, would violate the Misappropriation Act.
Anti-Deficiency Act
The Anti-Deficiency Act prohibits the obligation of funds in excess of an appropriated amount or in
advance of receiving an appropriation. In other words, you can’t spend more funds than you have or
before you have them. Incurring a contractual obligation without having the funds to cover it, for
example, would violate the Anti-Deficiency Act.
Bona Fide Need Rule
The Bona Fide Need Rule states that funds appropriated for a particular area can only be used during the
period in which the appropriation is available for new obligations. If a research and development contract
were awarded with FY13 RDT&E funds, and a new requirement arises inFY15 beyond the scope of that
contract, then using FY13 RDT&E funds to pay for the new requirement would violate the Bona Fide Rule.
236

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Reprogramming Funds | Lesson Summary 2
Reprogramming Funds
Although there are strict rules governing the use of appropriated funds, Congress recognizes that there
are certain situations where some flexibility is needed. Reprogramming is the use of funds for purposes
other than those intended by Congress at the time originally appropriated. Note that reprogramming only
applies to funds that have already been appropriated by Congress.
Below-Threshold Reprogramming
Prior approval from Congress is required to move funds between appropriations, to increase the quantities of
major systems procured, new starts, or for designated special interest items. However, most reprogramming
actions in DoD are approved at the service or agency level, without the involvement of Congress, using
below-threshold reprogramming. Below-Threshold reprogramming allows the transfer of funds among
programs within an appropriation category, subject to certain limitations. Up to $20 million of procurement
funds can be transferred into a line item, and up to $10 million of RDT&E funds can be transferred into a
program element, through below-threshold reprogramming.
Version 1.2 (30 June 2015)
237

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Reviews, Simulations, and Test | Lesson Summary 1
4.5 Lesson Summary
Reviews, Simulations, and Test
The following learning objectives are covered in this lesson:
• Recognize the importance of modeling and simulation in the defense acquisition process.
• Distinguish among various types of Developmental Test and Evaluation (DT&E) (e.g., Production
Qualification Tests, Production Acceptance Test and Evaluation, etc.).
• Recognize the relationship between risk management and exit criteria.
• Identify the information required for a milestone review.
Exit Criteria
One way to effectively manage acquisition risk is through the use of exit criteria, which serve as a litmus
test as to whether the program is on track to achieve its goals. In order for exit criteria to be meaningful,
they must be unique to not only the program itself, but to each phase of the program. Exit criteria are
proposed by the Program Manager and approved by the Milestone Decision Authority (MDA).
Exit criteria can take many forms. However, the criteria should be measurable and reflect progress made in
high risk areas of the program. Examples include the achievement of technical capabilities as seen in test
results or the maturity of a manufacturing process. Thus, exit criteria are event-driven and considered at
program reviews throughout the life of a program. They are critical “show-stoppers;” failure to meet an exit
criterion could prevent a program from making further progress.
Milestone Reviews
Milestone reviews are conducted by the MDA to initiate technology maturation and risk reduction, to
authorize program initiation and entry into the EMD phase, and later to commit to production and
deployment. Information for milestone reviews may be required by statute or regulation. The specific
information required for each milestone review can be found in Enclosure 1 of DoDI 5000.02.
Modeling and Simulation
The use of modeling and simulation (M&S) can be very helpful during the acquisition process. Used as a
predictor of future capabilities, M&S can be an inexpensive way to test various capabilities. Models and
simulations can also be modified and reused later in the acquisition process, which should avoid costs in the
long run.
However, M&S should not be used as a substitute for good test data. While M&S can be very effective,
simulations only provide predictions of a system’s performance and effectiveness. Thus, by combining M&S
data with the empirical, measurable data provided by T&E, the two processes enhance each other and
should result in long term efficiencies and cost savings.
238

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Reviews, Simulations, and Test | Lesson Summary 2
Developmental Testing and Evaluation
Developmental Testing and Evaluation (DT&E) can take many forms during the acquisition process,
depending upon what stage of the life cycle the program is in.
Component Testing
Component tests take place on individual system parts before the parts are merged into the system as a
whole. Component testing is conducted both on hardware items and on software items before they are
integrated with system hardware.
Integration Testing
Integration testing is used to assess compatibility of individual hardware and software components as they
are aggregated to form subsystems or systems.
Environmental Testing
Environmental testing, sometimes referred to as the “shake-rattle-roll” part of the testing process, attempts
to define how different components react under various conditions, such as temperature and shock.
Production Qualification Testing
Production Qualification Testing (PQT is conducted on initial production articles to verify the effectiveness of
the manufacturing process.
Production and Acceptance Testing and Evaluation
Production and Acceptance Testing and Evaluation (PAT&E) is conducted on production items to verify that
these items have met contract requirements.
Operational Assessments
Early Operational Assessments (EOA) can be conducted at any time during development, but at least one is
typically conducted sometime before the Critical Design Review occurs in the Engineering and
Manufacturing Development (EMD) phase. Using prototype systems, the EOA identifies potential
operational effectiveness and suitability issues during system development. An Operational Assessment
(OA) is conducted before Milestone C. Using production representative articles, the OA provides
operational effectiveness and suitability data before low rate initial production is begun.
239

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Reviews, Simulations, and Test | Lesson Summary 3
Combined Developmental and Operational Testing
Sometimes developmental and operational testing are integrated to save resources, time and money. DT
and OT are typically integrated when the data, resources, objectives, test scenarios, and measures of
effectiveness of both tests are similar and compatible.
DoD policy encourages integrated testing as long as the objectives of both types of testing are met.
Integrated testing eliminates redundant activities and raises operational concerns in time to make
changes in the system design. However, integrated tests require extensive coordination, are more
difficult to design, and risk compromising test objectives.
Integrating DT and OT does not remove the requirement to conduct initial operational test and evaluation
(lOT&E), which is required by law for ACAT I and ACAT II programs. lOT&E uses mature production-
representative systems and typical user personnel in a scenario that is as realistic as possible. Successful
lOT&E is required for the milestone decision authority to make the full-rate production decision.
Modification Testing
Modification testing can be used during production, or following system deployment, to determine the need
for or benefits of any system changes.
Live Fire Test and Evaluation
Live Fire Test and Evaluation (LFT&E) provides a realistic assessment of weapon platform/crew vulnerability
and lethality of conventional munitions/missiles. LTF&E is required for all ACAT I and ll programs or
modifications that impact the system’s vulnerability or lethality in combat. It is mandated by Congress, and
funded by the program office. Results must be reported to Congress prior to a Full Rate Production Decision
in the LFT&E Report.
Version 1.1 (30 June 2015)
240

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contractor Performance Measurement | Lesson Summary 1
4.6 Lesson Summary
Contractor Performance Measurement
The following learning objectives are covered in this lesson:
• Given performance data, select and compute appropriate performance status indicators.
• Given performance data, detect and analyze the impact of significant problem areas, based on
the status indicators.
• Given performance data, calculate an estimate of cost at completion.
• Recognize the importance of Earned Value data in external reporting.
Performance Status Indicators
There are various performance status indicators used in earned value management to tell whether a
program is on track or not.
Budgeted Cost of Work Scheduled (BCWS)
BCWS indicates the value of work planned to be accomplished or planned value.
Budgeted Cost of Work Performed (BCWP)
BCWP indicates the value of work accomplished or the earned value.
Actual Cost of Work Performed (ACWP)
ACWP indicates the cost of work accomplished or actual cost.
Schedule Variance (SV)
SV equals the difference between the value of work accomplished and the value of work planned to be
accomplished. It is calculated by subtracting the budgeted cost of work scheduled from the budgeted
cost of work performed.
SV = BCWP – BCWS
A negative schedule variance is unfavorable and indicates that less work was accomplished than planned,
while a positive schedule variance shows that more work was accomplished than planned. The program’s
critical path schedule must be reviewed to determine the impact of these schedule variances to the
program. (Note that the schedule variance is denominated in dollars.)
241

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contractor Performance Measurement | Lesson Summary 2
Cost Variance (CV)
CV indicates whether the work accomplished cost more or less than planned. It is calculated by
subtracting the actual cost of work performed from the budgeted cost of work performed.
CV = BCWP – ACWP
A negative cost variance is unfavorable and indicates that more money was spent for the work
accomplished than was planned. This has the potential to put the program over budget if the trend
continues, and may require the government to provide additional money to complete the program. A
positive cost variance is favorable and indicates that the work accomplished cost less than planned
Performance Trends
We can also identify performance trends to see whether performance is improving or worsening over
time and at what rate. This can be done for the overall program or for a specific activity within the
program.
Schedule Performance Index (SPI)
SPI indicates the efficiency with which the work has been accomplished in comparison to the work
planned. For example, we may be functioning at only 0.8 or 80% efficiency of what we had planned to
accomplish. It is calculated by dividing the budgeted cost of work performed by the budgeted cost of
work scheduled.
SPI = BCWP/BCWS
Cost Performance Index (CPI)
CPI tells the cost efficiency. It compares the budgeted cost of work that has been accomplished to the
actual cost of the accomplished work. For example, if our CPI is 0.75, we are accomplishing only 75 cents
worth of work for every dollar we spend. It is calculated by dividing the budgeted cost of work performed
by the actual cost of work performed.
CPI = BCWP/ACWP
Ideal CPI for a project is 1.0. Any activity with a CPI of less than 1.0 will rarely be improved over time. In
fact, a program’s CPI performance of less than 1.0 is often non-recoverable.
Cumulative CPIs and SPIs
Cumulative CPIs and SPIs are usually less than 1.0 for most programs. Current period SPIs and CPIs for
individual tasks can exceed 1.0, and exhibit positive and negative elements. When cumulative
performance (CPI and SPI) falls below 1.0, the government needs to discuss the performance status with
the contractor as part of risk management. Earned Value industry guidelines specifically state that
management reserve will NOT be used to offset negative variances.
242

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contractor Performance Measurement | Lesson Summary 3
Budget at Completion
Budget at Completion (BAC) is the sum of all authorized budgets for the contract scope of work. The
project’s scope of work forms the performance measurement baseline (PMB), which projects the cost to
complete the entire program. The BAC equals the sum of all the allocated budgets plus any undistributed
budget (management reserve and profit/fee not included). We use the BAC to determine the percent of
the program spent and completed.
Percent Spent (% Spent)
Percent Spent indicates how much of the program budget has been spent to date relative to the total
amount of the project’s budgeted funds. It is calculated by dividing the actual cost of work performed to
date by the total amount expected to be spent on the program (the budget at completion).
% Spent = (ACWP/BAC) * 100
Percent Complete (% Complete)
Percent Complete indicates how much of the total program has been completed to date relative to the
total amount of work to be performed. It is calculated by dividing the budgeted cost of work performed
to date by the total amount expected to be spent on the program (the budget at completion).
% Complete = (BCWP/BAC) * 100
Percent Scheduled (% Scheduled)
Percent Scheduled indicates where the program should be based on a point in time. It is calculated by
dividing the budgeted cost of work scheduled to date by the budget at completion.
% Scheduled = (BCWS/BAC) * 100
If the Percent Spent is greater than the Percent Complete, the program is going to run out of funds
before the end of the project if it continues on the current trend. Conversely, if the Percent Complete is
greater than or equal to the Percent Spent, the project has sufficient funds if it continues on the current
trend.
For example, if the percent complete is 50% and percent spent is 66%, we know we have a problem
because we are spending at a faster rate than the project’s work is being completed.
Note: Don’t confuse Percent Spent and Percent Complete with the SPI and CPl. Percent Complete and
Percent Spent indicate program status, looking at the entire program from beginning to end. SPI and CPI
indicate efficiency trends and look at a program up to a certain point in time.
243

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contractor Performance Measurement | Lesson Summary 4
Estimate at Completion
Estimate at Completion (EAC) is the ‘current’ estimate of what the program will cost when completed.
The EAC is based on the actual cost of work performed to date plus an estimate of the work remaining. It
is calculated by adding the actual cost of work performed (ACWP) to the estimated cost to complete the
remaining work of the program.
EAC = ACWP + Estimated Cost to Complete
The EAC can be calculated as follows: EAC is equal to the ACWP plus the BAC minus the BCWP divided by
a performance factor. One method to calculate EAC is called EAC(CPI) , or EAC(Low), where the
performance factor is just the cumulative CPI. This is an optimistic EAC and represents to lower end of
the EAC range.
EAC(CPI) = ACWP + [(BAC – BCWP)/(CPI)]
Another method to calculate EAC is called EAC(COMPOSITE), or EAC(High), where the performance factor is
the CPI multiplied by the SPI. Since CPI and/or SPI are usually below 1, this is a pessimistic EAC and
represents the higher end of the EAC range.
EAC(COMPOSITE) = ACWP + [(BAC – BCWP)/(CPI x SPI)]
Both the Government and the contractor calculate EACs. The contractor’s EAC is often referred to as the
Latest Revised Estimate (LRE). Under normal circumstances, The contractor’s EAC should typically fall
somewhere between the EAC(CPI) and EAC(COMPOSITE) values.
To-Complete Performance Index
To-Complete Performance Index {TCPI (target)} is a powerful but often misunderstood EVM metric. The
TCPI is an EVM metric computed by dividing the value of the work remaining by the value of the cost
target remaining. The cost target remaining value is tied to some financial goal set by management
(Government or Contractor).
In other words the TCPI metric represents the cost efficiency from the present time or time now, until the
end of the contract required to achieve management’s financial goal. The management goals are usually
defined as either the Contractor’s EAC (also known as LRE), the contract’s BAC, or the Government’s
“Most Likely” EAC.
TCPI (Target) = Work Remaining / Cost Remaining
or
(BAC – BCWP)/(COST TARGET-ACWP)
Note: To determine the TCPI for any of the cost targets listed above, simply replace the Cost Target
value with either BAC, EAC, or the LRE value.
244

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contractor Performance Measurement | Lesson Summary 5
TCPI for BAC
TCPI for the Budget at Completion {TCPI (BAC)} is an index that shows what efficiency is required to
accomplish the remaining work within the contract budget.
TCPI for LRE
TCPI for the Latest Revised Estimate {TCPI (LRE)} is an index that shows what efficiency is required by
the Contractor to accomplish the remaining work within their expected cost target estimate.
TCPI for EAC
TCPI for Estimate at Completion {TCPI (EAC)} is an index that shows what efficiency the Government
thinks is required to accomplish the remaining work within some identified cost target estimate
(Government’s “Most Likely” EAC).
The TCPI is correlated with the cumulative CPI; it takes the cost efficiency experienced to date, as
reflected by the cumulative CPI, and determines what level of performance efficiency will be required to
complete the project within available budget.
If the cumulative CPI is 0.8 or 80%, in order to stay within our budget, we must achieve a performance
factor of 1.2, or work at an efficiency of 120% for all the remaining work in order to complete the project
at the BAC. This means the contractor must work 40% more efficiently than its current cumulative CPI of
80%.
To calculate TCPI (BAC) we divide the budgeted cost of the work not yet completed by the amount of
budget remaining. In other words, we subtract the BCWP from the BAC then divide that difference by the
difference between the BAC and the ACWP.
TCPI (BAC) = (BAC – BCWP)/(BAC – ACWP)
DoD analysts have determined that after 20% into a program, the cumulative CPI rarely improves.
Therefore, achieving a TCPI that is greater than 5% (or 0.05) of the CPI is unlikely; this means we may
have to restructure the program in order to obtain an executable program.
Version 1.0 (11 July 2014)
245

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Integrated Baseline Review | Lesson Summary 1
4.7 Lesson Summary
Integrated Baseline Review
The following learning objectives are covered in this lesson:
• Identify the primary factors that the government should review to evaluate the contractor’s
Performance Measurement Baseline (PMB) during an Integrated Baseline Review (IBR).
• Identify the three reasons for Performance Measurement Baseline (PMB) changes—contract
changes, internal re-planning, and formal reprogramming–and recognize the impact of each.
Performance Indices
The Cost Performance Index (CPI) and Schedule Performance Index (SPI) indicate the performance
efficiency factors that the contractor has achieved to date. Anytime the CPI or SPI are running
significantly below 1.0, rebaselining may be necessary in order to complete the program.
Generally, a CPI or SPI falling 10% or more below 1.0 is considered significant. The To-Complete
Performance Index (TCPI) indicates the efficiency factor that the contractor must achieve from “time
now” to meet the Budget At Completion (BAC) or Estimate At Completion (EAC).
A TCPI greater than 1.0 indicates the contractor must work more efficiently that they have in the past to
stay within the BAC or meet the EAC. These performance indices may indicate the need to conduct an
Integrated Baseline Review (IBR).
Integrated Baseline Review
The IBR assesses the validity of the PMB and identifies the risks associated with executing to the current
PMB. Participants in an IBR typically include the Government PM and technical staff, along with the
related contractor’s staff. During an IBR, the primary factors that are evaluated include:
• Technical scope of the PMB
• Program schedule requirements
• Effective resource allocation to ensure that the work can be accomplished
The IBR is not just a single event – it is a continuous process that revalidates the PMB. The revalidation of
the baseline is an integral part of Government/contractor management and maybe included as an agenda
item in subsequent program management reviews. IBRs should be used as necessary throughout the life
of a program to facilitate and maintain a mutual understanding of:
• The scope of the PMB consistent with authorizing documents
• The management control processes
• The Risks in the PMB associated with cost, schedules, and resources
• Corrective actions where necessary
246

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Integrated Baseline Review | Lesson Summary 2
PMB Changes
There may be considerable risks associated with the current PMB, indicating a need to rebaseline the
program in order to make it executable. Rebaselining is an unfortunate term that causes no end of
confusion among experienced practitioners and the inexperienced alike. Rebaselining the PMB can be
caused by any one of the following three reasons:
• Contract changes: only apply to changes/contract modifications directed by the Government, not
the contractor.
• Internal replanning: occurs when the contractor’s original plan needs adjustment in response to
problems or the opportunity to capitalize on efficiencies. The remaining in-scope work is then re-
planned by the contractor PM using the remaining budget and schedule.
• Formal reprogramming: occurs when the remaining budget and schedule is unrealistic; the
contractor requires more time and dollars; the PMB exceeds the contract target cost and an over
target baseline (OTB) occurs — the current contract budget is insufficient and the original
objectives cannot be met.
Replanning should happen whenever the shape of the future baseline needs revision to reflect realism, and
should never – theoretically – modify the shape of the baseline for time past unless that work was never
begun. Replanning does not alter scope, schedule, or budget at the PMB, although it changes the time-
phasing of the existing scope and budget. Revisions to the baseline that exceed the negotiated scope/budget
or schedule result in over-target baselines or over-target schedules (OTS). It is a fiction that if the
government does not approve an OTB or an OTS that it does not exist.
Formal reprogramming changes scope, schedule, budget, or all three. On a DOD contract, it requires
notification to the customer, which in practice becomes approval from the customer, although that is not
strictly the intent of the requirement. Formal reprogramming should never be undertaken to eliminate
variances for the sake of “starting over” unless the baseline is so unrealistic that retaining the variances
would forever skew performance data so much as to be unusable.
STUDENT NOTE: Be advised that in this instance, we are talking about reprogramming the PMB, not
reprogramming appropriated funds from one program to another; however, we may need to do that later in
order to cover any additional program scope.
Version 1.3 (30 June 2015)
247

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Budget Execution | Lesson Summary 1
4.8 Lesson Summary
Budget Execution
The following learning objectives are covered in this lesson:
• Given a scenario, track budget execution through the commitment, obligation, and expenditure
process.
• Identify the use and importance of obligation and expenditure plans.
• Assess the impact of the failure to execute funds in accordance with program plans.
Budget Execution Process
In the budget execution process, the following steps are taken:
Commitment
A commitment is an administrative reservation of funds, made upon receipt of a request for spending.
Commitment occurs upon certification that funds are available in the correct appropriation, in the correct
fiscal year, and in the correct amount to cover the anticipated obligation.
Obligation
An obligation is a “legal reservation” of funds, tying the government to a liability, such as a contract for
goods or services. Obligation occurs when a contract is signed or when orders are placed.
Expenditure
An expenditure is a payment of some part or all of an obligation. Expenditure occurs when a check is
issued, or when funds are electronically transferred, to a contractor in response to an invoice or bill for
costs incurred, services rendered, or products delivered.
Outlay
An outlay is a payment by the U.S. Treasury to the contractor. Outlay occurs when a check is cashed or
when funds are electronically transferred from the Government to the contractor. (In electronic funds
transfer, expenditure and outlay happen simultaneously.)
248

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Budget Execution | Lesson Summary 2
Participants in Budget Execution Process
A number of players are involved in the execution of funds. After the Comptroller commits the funds by
certifying their availability, the Contracting Officer obligates the funds by awarding the contract or signing
purchase orders.
Then the contractor performs the work and submits a Material Inspection and Receiving Report to the
Quality Assurance Representative (QAR) from the Contract Management Office, if deliverables are
received at the contractor’s plant, or to the Contracting Officer’s Representative (COR), if deliverables are
received at the program management office.
The QAR or COR verify that the deliverables were received and accepted and inform the Administrative
Contracting Officer (ACO). The contractor submits an invoice to the ACO.
The ACO certifies that the invoice is correct, and then forwards the invoice to the finance office to make
payment. The ACO also assures that the contractor gets paid in a timely manner.
The Finance and Accounting Office in tum expends the funds by check or electronic funds transfer.
Finally, the U.S. Treasury outlays the funds when the cash is provided to the contractor.
Spending Plan
Failure to make timely payment to a contractor can cause serious cash flow problems for the contractor.
In addition, poor expenditure or outlay rates are a bad reflection on a program and may jeopardize a
program’s current and future funding. To minimize this risk, the Program Management Office prepares a
spending plan that projects and tracks obligations and expenditures on a month-by-month basis.
A spending plan is required for each Procurement line item, RDT&E program element, and Operations
and Maintenance sub-activity group in the program. The PMO creates an obligation plan for each fiscal
year of funding that is available for new obligations and an expenditure plan for each fiscal year of
funding that has not been completely expended, even if the period of obligation availability has expired.
Spending plans serve as a tool to analyze program execution, an indicator of potential problems, and a
predictor of future program performance. Generally, a history of poor obligation, expenditure, or outlay
will cause a program to come under increased scrutiny or – worse – to lose funding. When a program
deviates from its spending plan, it risks becoming a source of funding for other programs through
reprogramming and runs the risk of having its funding cut in future years.
Version 1.0 (11 July 2014)
249

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Operational and Live Fire Tests | Lesson Summary 1
4.9 Lesson Summary
Operational and Live Fire Tests
The following learning objectives are covered in this lesson:
• Identify which organizations develop, coordinate, or approve Critical Operational Issues (COIs).
• Identify which organizations develop, coordinate, or approve Critical Technical Parameters (CTPs).
• Recognize how Measures of Effectiveness (MOE) and Measures of Suitability (MOS) are used
throughout the Test and Evaluation (T&E) process.
• Recognize the purpose and objectives of Live Fire Test and Evaluation.
• Distinguish among various types of Developmental Test and Evaluation (DT&E) (e.g., Production
Qualification Tests, Production Acceptance Test and Evaluation, etc.).
Initial Operational Test and Evaluation
Developmental test and evaluation is essential in determining a system’s readiness for initial operational
test and evaluation (IOT&E). The results of developmental testing are formally reviewed in an
Operational Test Readiness Review (OTRR) prior to proceeding with IOT&E.
Critical Technical Parameters
Critical Technical Parameters (CTPs) are key parameters and developmental testing criteria that are
derived from Warfighter capabilities specified in the Capability Development Document (CDD), and from
technical performance measures as specified by the System Engineering Plan (SEP). The CTPs are
developed, coordinated and approved by the T&E Integrated Product Team (IPT) within the Program
Management Office. Examples of CTPs are an aircraft’s engine thrust, cruising speed, range and altitude.
CTPs are measurable criteria that, if not achieved during DT&E, preclude fulfillment of desired
operational performance capabilities.
Developmental Testing
Two types of developmental testing become important as a system nears and enters production.
Production Qualification Testing
Production Qualification Testing (PQT) is conducted on a small number of initial production items to
evaluate the effectiveness of the manufacturing process.
Production Acceptance Testing and Evaluation
Production Acceptance Testing and Evaluation (PAT&E) is conducted on items as a form of quality
assurance to ensure that contractual obligations are being met.
250

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Operational and Live Fire Tests | Lesson Summary 2
Operational Test and Evaluation
Operational test and evaluation is conducted to determine if a system will successfully meet the user’s
capability needs.
Critical Operational Issues
Critical Operational Issues (COIs) indicate the operational effectiveness and operational suitability needs
of a system. They are expressed in the form of a question, developed by an independent operational test
agency, and broken down into quantifiable MOEs and MOSs. An example of a COI is: “Does the aircraft
accomplish its mission in the battlefield environment?”
Measures of Effectiveness
Measures of Effectiveness (MOEs) are specific, objective measures of system performance that are
closely related to mission accomplishment. An example of a MOE is: ”Number of targets destroyed”.
Measures of Suitability
Measures of Suitability (MOSs) are specific, objective measures of how well as system can be maintained
and utilized by the end user. They are written and approved by an independent operational test agency.
An example of a MOS is: “Aircraft Mean Time Between Failure (MTBF).”
In summary, COIs are the primary operational issues that must be answered by the testing program,
while MOEs and MOSs may be thought of as the quantifiable measures that can be used to determine
whether the COIs have been addressed successfully. In turn, CTPs provide developmental test data that
help support the MOEs and MOSs.
Live Fire Test and Evaluation
Live Fire Test and Evaluation is required by law for certain major systems before full-rate production can
begin:
Survivability testing is required for “covered” systems that are occupied by personnel and designed to
provide the personnel some degree of protection in combat situations.
Lethality testing is required for all major munitions and missile programs to determine whether the
weapon can reliably disable or destroy its target.
Live Fire Test and Evaluation results are sent to the Director, Operational Test and Evaluation (DOT&E),
acting as the OSD agent, who then reports them to Congress before a program can move forward
beyond Low-Rate Initial Production (LRIP) and on to full-rate production.
Version 1.2 (31 December 2015)
251

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Best Manufacturing Practices 1
5.1 Lesson Summary
Best Manufacturing Practices
Learning Objectives
The following learning objectives are covered in this lesson:
• Recognize the value of Lean Manufacturing.
• Identify methods of controlling manufacturing costs (e.g., process proofing, variability reduction,
and statistical process control).
• Distinguish between process and product structures.
Lean Manufacturing
Two main principles of lean manufacturing are minimization of waste and responsiveness to
change. By practicing lean manufacturing techniques, the contractor can control costs and more
effectively meet customer requirements.
• Waste can manifest itself in many forms, including:
• Inefficient layouts
• Defective equipment
• Excess inventory
• Inefficient production or assembly processes
By reducing the time needed to adjust or react to changes taking place, whether in the product or a
process, the contractor can reduce waste associated with these changes. To do so effectively requires
buy-in from everyone, from top management all the way to employees on the factory floor. Some
characteristics of organizations that have lean manufacturing processes include:
• Team-based approach
• Minimal inventory
• Customer-driven products and inventory quantities
• Concurrent product and process design
• Multi-skilled workforce
Reducing Costs
Manufacturing costs can be reduced utilizing a variety of tools, including:
• Process proofing: By examining and verifying the production process and support
infrastructure, early production problems can be eliminated.
• Variability reduction: Common cause variability, which is inherent in the production process, is
typically corrected by management. Special cause variability, as its name implies, is triggered by
a unique event and is often corrected at the worker level. Reducing variability improves product
cost, quality, and reliability.
252

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Best Manufacturing Practices 2
• Statistical process control: Involves using statistical analysis to track and measure variability.
By using SPC, the contractor can pinpoint causes of variability early, then eliminate them, thus
reducing costs and improving performance.
Optimal Process
For every given product, there is an optimal process structure that can be used to produce it. In order to
determine the process, the product type must be identified first. Products can be classified using a
continuum of product standardization and production volume. Production for highly standardized and
high volumes of products, such as bullets, require a different process than one of a kind products, such
as satellites. These product types are at the opposite ends of a continuum.
A process can be identified using a continuum of production flow that ranges from high, or continuous
flow, to low, or jumbled flow. One of a kind items fit under a jumbled flow, where specialized material
and flexible methods are required. On the opposite end of the continuum, high volumes of products
require a continuous flow, where interchangeable parts and standardization of assemblies are required.
What resources and procedures the contractor needs to most effectively produce a product should define
the process structure.
Version 1.0 (11 July 2014)
253

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Constructive Changes 1
5.2 Lesson Summary
Unauthorized Commitments
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify the relationship between the Program Management Office, the Procuring Contracting
Officer, the Administrative Contractor Officer, and Program Integrator.
• Identify the causes and consequences of unauthorized commitments.
ACO, CAO, PCO, and PI
The Administrative Contracting Officer (ACO) works for the Contract Administration Office (CAO) under
the head of the Defense Contract Management Agency. The primary responsibility of the ACO, as
delegated by the Procuring Contracting Officer (PCO), is contract administration, including:
• Contractor payment
• Administrative Contract modification
• Program technical support
• Quality assurance
• Property management
• Engineering and production surveillance
The primary responsibility of the ACO is
overseeing the day-to-day contractual activities
after contract award has been made and ensuring
that the contractor satisfies the terms and
conditions of the contract. As such, the ACO has a
direct line of communication with the Procuring
Contracting Officer (PCO).
The Program Integrator (PI), who also works for the CAO, provides support for the Program Management
Office. The PI’s duties are defined by the PM and written into the Memorandum of Agreement, which is
then signed by the PMO and CAO. It is essential to the PI’s job that he or she keeps in direct
communication with the PM. The PI:
• Acts as the “eyes and ears” of the PM
• Leads and directs the program support team (PST)
• Provides feedback and data to the PM
• Develops and implements program surveillance plans
Unauthorized Commitment
A Constructive Change is an oral or written act (or failure to act) by an authorized Government official in a
position of authority construed by the contractor as having the same effect as a written change order (on an
existing contract). It usually occurs when the contract work has been changed but proper procedures
254

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Constructive Changes 2
have not been followed (i.e. when a Government official, other than the Contracting Officer, interacts with
the contractor). However, a Contracting Officer acting outside his/her authority can effect a constructive
change.
Any unauthorized change that requires the contractor to perform beyond the requirements of the
contract can lead to unauthorized commitments. The only person authorized to make changes to a
contract is the Contracting Officer (CO). Occasionally a government employee will initiate a change
believing that he or she is empowered to do so. Sometimes the contractor makes these changes under
the assumption of apparent authority–that someone’s rank, title, or tenure authorizes that person to
make changes. Rank, title, or other indicators do not equal lawful contracting authority! Only the CO has
the authority to initiate and approve contract changes.
An Unauthorized Commitment is an agreement that is not binding solely because the Government
representative who made it lacked the authority to enter into that agreement on behalf of the
government (for an existing contract or no contract).
Other situations that can lead to constructive changes include:
• Technical terms that are “impossible to perform”
• The acceleration of work or performance despite a contractor’s valid claim of an excusable delay
• Government inspection that exceeds any reasonable interpretation of what a contract may
require
• Government failure to disclose its superior knowledge when such knowledge is essential to the
performance of required work
• Unauthorized technical direction by Government personnel
In extreme situations, a person who initiates an unauthorized commitment can be held personally liable
for the costs associated with this mistake. In this situation, the liability is determined by each respective
agency on a case-by-case basis.
Version 1.0 (11 July 2014)
255

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Production and Follow-On Support | Lesson Summary 1
5.3 Lesson Summary
Production and Follow-On Support
Learning Objectives
The following learning objectives are covered in this lesson:
• Recognize the value of the Cost of Quality.
• Identify where and when learning curve theory is applied.
• Recognize the impact of manufacturing on cost, schedule and performance.
• Recognize the considerations/concerns of the elements of manufacturing (5Ms) and how other
areas are affected.
Five Elements
The five elements of manufacturing–manpower, machinery, materials, methods, and measurement–
all contribute to the cost of production. The cost of producing a quality end product will vary depending
upon how that quality is achieved. By putting processes in place to prevent problems in the first place,
less money will be spent on correcting and rework of failures.
The cost of achieving quality can be broken down into three areas: prevention, appraisal and failure.
• Prevention – money spent on avoiding problems, such as utilizing process proofing. Ideally,
prevention should make up about 50% of the cost of achieving quality.
• Appraisal – money spent looking for errors through testing and inspection. Appraisal costs
should make up about 35% of the cost of achieving quality.
• Failure – money spent correcting errors, often in the form of rework or repair. Correcting failures
should only account for about 15% of the total cost of achieving quality.
Although more money may be spent to avoid costs up front, less money will be spent on production in
the long run.
Learning Curve
Learning curve theory states that as the production of an item doubles, the man-hours needed to
produce that item decrease at a fixed rate. In other words, the more items that are produced, the less it
should cost per item. Declining unit costs are a result of workers becoming more familiar with their tasks
and making process improvements based on their experience.
Learning curve theory is most applicable in situations where the following conditions exist:
• Uninterrupted serial production
• Consistent product design
• Management emphasis on productivity improvement
256

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Production and Follow-On Support | Lesson Summary 2
Plotting this reduction in cost onto a graph results in a curved line. If, for example, the cost has been
reduced by 20%, then there is an 80% learning curve. Thus, an 80% learning curve means that the cost
of a particular unit of production is 80% of the cost of the unit exactly halfway back in the production
sequence. The steeper the learning curve, the greater its impact.
Factors that influence the learning curve include:
• Manufacturing methods and processes
• Item complexity
• Workforce stability
• Production breaks
Version 1.0 (11 July 2014)
257

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contract Modification | Lesson Summary 1
5.4 Lesson Summary
Contract Modification
Learning Objectives
The following learning objectives are covered in this lesson:
• Contrast a Change Order with a Supplemental Agreement.
• Identify how instability of requirements, design, and production processes impact program cost
and schedule.
• Identify the proper DoD Appropriation Category to be used for each of the phases of a Product
Improvement Program.
Unstable Requirements
Requirements that are not stable can become very expensive, impacting schedule and cost because of a
“ripple effect” through the system’s entire configuration. Changing requirements late in the acquisition
process often requires re-design, re-fabrication, and re-testing of many system components. Usually, the
later the changes are made in the life cycle, the more expensive they are. Therefore, system
requirements should be stabilized well before production begins.
Acceptable Ways to Change a Contract
Undefinitized contract action means any contract action for which the contract terms, specifications, or price
are not fully agreed upon before performance is begun under the action. Examples are letter contracts,
orders under basic ordering agreements, and provisioned item orders, for which the price has not been
agreed upon before performance has begun.
Definitization means the agreement on, or determination of, contract terms, specifications, and price, which
converts the undefinitized contract action to a definitive contract.
A Change Order and a Supplemental Agreement are two acceptable ways to change an existing contract.
Change Orders
Government contracts contain a changes clause that permits the contracting officer to make unilateral
changes in certain areas that are within the scope of the contract. Those areas are: drawings, designs
and specification for supplies specifically produced for the government; method of shipment or packing of
supplies; place of delivery.
258

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contract Modification | Lesson Summary 2
The use of Change Orders are limited because there may be an impact to the contract terms and
conditions or cost that goes beyond the change which is not known when the change order is issued. It is
typically used when time is of the essence. If the contractor determines that the change has affected the
terms of the contract, including price, a request for equitable adjustment (REA) may be submitted within
30 days to the contracting officer. To issue a change order, the contracting officer needs to have direction
from the PM detailing the needed changes and certified funds to cover anticipated costs, if applicable.
The notional change order process is as follows:
1. Program Manager (PM) directs change
2. Details of change are identified (to include available funding
3. Issue change order (with not-to-exceed amount)
4. Contractor submits REA (if applicable)
5. Negotiate REA and sign Supplemental Agreement
Supplemental Agreements
A Supplemental Agreement is a bilateral agreement, signed by both parties, on what will be changed
and at what price. Under a Supplemental Agreement, a price for the work to be done is negotiated before
the work actually begins. This is the more preferred method, as long as there is enough time to reach an
agreement before the work begins.
The supplemental agreement process is as follows:
1. Government issues modified SOW or SOO
2. Contractor submits change proposal
3. Government evaluates and audits proposal
4. Government and contractor prepare negotiations
5. Government and contractor conduct negotiations
6. Government modifies contract
7. Government and contractor review and sign contract
Product or System Modifications
If any product or system modifications need to take place, funding must be used from the correct
appropriation category. The type of funds used for development and testing of the modification is directly
related to:
• The purpose of the modification;
• Whether or not extensive developmental or operational testing is required; and
• Where the system is in the life cycle
• If a modification increases the system’s performance capability, or if the testing will be done by
an independent government agency, funding for development and testing should come from
RDT&E appropriations.
• If the modification does not increase system performance and the system is still in production,
procurement appropriations should be used to fund research, development, and testing of the
mod.
• If the modification does not increase system performance, and if the system is no longer in
production, then Operations and Maintenance appropriations should be used to fund research,
development and testing of the mod.
259

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Contract Modification | Lesson Summary 3
Regardless of which appropriation is used to develop and test the mod, the fabrication and installation of
mod kits should be funded with procurement appropriations.
A modification is not considered to increase the performance capability if it only extends the system’s
years of usefulness. Likewise, improvements in maintainability or reliability are not considered to increase
system performance for the purposes of funding.
Version 1.1 (30 June 2015)
260

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Contract Dispute 1
6.1 Lesson Summary
Contract Dispute
Learning Objectives
The following learning objectives are covered in this lesson:
• Contrast the difference between termination for convenience, termination for default, and
termination for cause.
• Identify the process for resolving disputes between parties of a contract.
• Given a funding shortfall, apply the rules governing the use of expired funds to resolve the
problem.
Contract Terminations
Contract termination can occur for two main reasons: convenience or default.
• Termination for Convenience: allows the Government the unilateral right to completely or
partially terminate a contract if the work no longer needs to be done or there is no more funding
available. If a contract is terminated for convenience, the government must reimburse the
contractor for the cost of completed work, a reasonable profit for that work, and costs associated
with termination settlement.
• Termination for Default: allows the Government to completely or partially terminate a contract
for non-commercial items because the contractor fails to deliver on time, endangers a timely
delivery, or fails to comply with the terms or conditions of the contract. In this case, the
government is only responsible for paying for products delivered and accepted. The government
is also entitled to reimbursement for expenses incurred as a result of finding another contractor.
• Termination for Cause: A type of termination for default that applies only to contracts using
commercial item procurement procedures. Termination for cause allows the Government to
completely or partially terminate a contract for commercial items because the contractor fails to
deliver on time, endangers a timely delivery, or fails to comply with the terms or conditions of the
contract.
Contract Dispute Resolution
There are two options for resolving contract disputes: Litigation and Alternative Dispute Resolution. Both
parties of a contract can exercise these options. Disputes between the government and contractor can be
very costly for both parties, especially if the dispute results in litigation. Alternative Dispute Resolution
uses selected methods to resolve disputes without going to court, including the following:
• Mediation: A neutral third party listens to the issues, helps develop options, and works with the
disputing parties to obtain a negotiated settlement. Mediation helps preserve relationships. The
parties in the dispute maintain high level of control over the outcome.
• Fact-finding: A neutral technical expert renders an advisory decision to both parties based on
the facts presented by the disputing parties.
261

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Contract Dispute 2
• Mini-Trial: Senior-level management listens to both parties and renders a decision. A neutral
third party can help in clarifying and identifying issues, but senior management is ultimately
responsible for negotiating a settlement.
• Non-binding Arbitration: A neutral third party renders a non-binding decision based on
evidence presented by disputing parties. Arbitration is closest to litigation.
• The purpose of ADR is to resolve disputes in an environment that is collaborative, not
competitive. Alternative Dispute Resolution (ADR) should be the first resort to solve disputes
when appropriate, but there are circumstances where taking the dispute directly to court is
necessary. Court is most appropriate when:
o Dispute is over issues of law
o Full public record is required
o Fraud is suspected
o Other party is likely to falsely present their case
Expired Funds
Funds are considered “expired” when the obligation period for that fund has expired. For example,
RDT&E funds have a two-year obligation period. After this two-year obligation period is over, RDT&E
funds are available for expenditure for five more years but are considered expired. Expired funds still
retain their original appropriation category, year, line item and other accounting identifiers for the
expenditure time beyond the original obligation period. Expired funds can only be used for payment or
adjustments to the original obligations during the expired period and cannot be re-assigned to new
obligations.
Version 1.0 (11 July 2014)
262

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Life Cycle Product Support 1
6.2 Lesson Summary
Life Cycle Product Support
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify life cycle product support activities and requirements that deal with
fielding/deployment (e.g., planning, coordination, organizing deployment teams, materiel
release).
• Identify life cycle product support activities and requirements associated with post-production
support (e.g., planning, adequate sources of supply, spares modernization and sustaining system
readiness).
• Identify system supportability issues in planning and executing a defense acquisition program.
• Determine the impacts to a given acquisition program if supportability issues are not resolved.
Deployment Planning
The primary purpose of deployment planning is to ensure a smooth introduction of the system to the end
user. Deployment planning must take into account all of the parties involved in this process by specifically
defining responsibilities of each. Thus, successful system deployment is directly related to how well
deployment is planned, coordinated, negotiated and executed.
Deployment Planning usually begins in the Materiel Solution Analysis Phase, where a deployment team
drafts a deployment plan that is incorporated into the Life Cycle Sustainment Plan. To enable the
system to move smoothly from production to operation, all related support activities must be well
coordinated, requiring effective lines of communication. Some examples of these support activities
include:
• availability of training manuals and accurate technical data
• manpower to operate and support the system
• adequate supply support
• facilities & infrastructure support
• packaging, handling, storage and transportation
Good deployment planning also defines how system modifications might be tracked, determines how
training will be developed and implemented, and ensures the availability of spare parts, for example.
Overlooking these elements will lead to poor training, personnel turnover, continual system modifications,
and technical problems such as software anomalies – all of which will impede a smooth introduction of the
system and can potentially drive up the life cycle costs.
Additionally, deployment planning involves efforts to reduce the “footprint” of the system. Footprint
reduction includes minimizing the amount of supporting material (including parts inventory and special
tools), equipment and maintenance personnel required when forces are deployed, all of which are
intended to help drive down life cycle costs.
263

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Life Cycle Product Support 2
Supportability
Integrated Product Support (IPS) Elements
Supportability concepts define the overall logistics end state in achieving product support. The PM and the
Product Support Manager (PSM) develop logistics support concepts (e.g., organic, two-level, three-level,
contractor, partnering, etc.) early in the program, and refine the concepts throughout program
development. Using the IPS elements, support concepts for all systems should be structured to provide cost
effective, total life cycle logistics support. Every acquisition program, regardless of size, must plan for 12
Integrated Product Support Elements:
1. Product Support Management
2. Design Interface
3. Sustaining Engineering
4. Supply Support
5. Maintenance Planning & Management
6. Packaging, Handling, Storage, and Transportation (PHS&T)
7. Technical Data
8. Support Equipment
9. Training and Training Support
10. Manpower and Personnel
11. Facilities & Infrastructure
12. Computer Resources
Life Cycle Cost
Life cycle cost (LCC) includes the cost to develop, acquire, maintain, and dispose of a weapon
system over its entire life. LCC includes system:
• Research, development, test, and evaluation
• Investment (procurement, military construction (facilities))
• Operations and support (personnel, spares, support equipment, etc.)
• Disposal
Supportability Goals
The goal of supportability is to increase system capability while:
• Reducing ownership costs
• Reducing dependence on spares
• Requiring fewer support personnel
• Reducing the overall logistics footprint
Sustainment
One important aspect of good deployment planning is Sustainment, which includes delivery of required
IPS elements after the system is fielded that contribute to operational readiness. Whereas sustainment
used to fall to the Service’s sustainment community, the Program Manager is now accountable for
“cradle to grave” support, from development and production all the way to disposal. Although the PM
has Total Life Cycle Systems Management responsibility, this is often done in conjunction with the
contractor, Service materiel and sustainment commands and the Defense Logistics Agency (DLA).
264

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Life Cycle Product Support 3
Effective sustainment means minimizing problems up front, which requires a long-term outlook.
Current DoD policy that supports effective sustainment includes:
1. The PM is the single point of accountability:
o Each PM is charged with the accomplishment of program’s cost, schedule, and
performance objectives for the total life cycle, including sustainment.
2. Evolutionary acquisition:
o This is DoD’s preferred strategy for satisfying operational needs by the rapid acquisition of
mature technology. An evolutionary approach delivers capability in increments, recognizing,
up front, the need for future capability improvements.
3. Supportability and Sustainment as key elements of performance:
o Supportability and sustainment are essential components of battlefield effectiveness and
must be planned and implemented as early as possible in the life of a program. If a
weapon system is not supportable and sustainable, it cannot be considered as an effective
warfighting capability.
4. Performance-Based Logistics:
o PMs shall develop and implement performance based logistics (PBL) strategies that optimize
total system availability while minimizing cost and logistics footprint.
5. Performance-Based Life-Cycle Product Support:
o The PM will develop and implement an affordable and effective performance-based product
support strategy. The product support strategy will be the basis for all sustainment efforts
and lead to a product support package to achieve and sustain warfighter requirements.
6. Increased reliability and reduced logistics footprint:
o PMs must ensure the application of a robust systems engineering process to provide for
reliable systems with reduced logistics footprint and reduced total ownership cost (TOC).
7. Continuing reviews of sustainment strategies:
o Reviews must be conducted at defined intervals throughout the life cycle to identify needed
revisions and corrections, and to allow for timely improvements in these strategies to meet
performance requirements.
Even with effective sustainment, problems can pop up in many areas throughout the system, including
depletion of supply lines, system down-time due to a defective part, or ineffective training. All of these
problems affect readiness and will lead to poor operational availability. Resolving these issues after the
system has been deployed can result in increased life cycle costs. However, most of these problems
and the associated costs can be minimized through early and effective sustainment and deployment
planning.
Version 1.2 (31 March 2015)
265

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Leadership and Ethics 1
6.3 Lesson Summary
Leadership and Ethics
Learning Objectives
The following learning objectives are covered in this lesson:
• Identify core ethical values critical to decision making in the acquisition environment.
• Discover how different leadership styles impact the effectiveness of an Integrated Product Team
(IPT).
Ethics
Ethics may be thought of as a set of behavioral standards for a group of people or society. Ethics can
also be defined as standards of conduct that shape one’s behavior with respect to moral duties and
obligations. The extent to which a person fulfills those obligations is based on two aspects:
• Ability to distinguish right from wrong
• Level of commitment to doing what is right
Although ethical norms vary from organization to organization, and culture to culture, there are some
core values that have been identified by leaders in education, business, religion and government. These
include:
• Trustworthiness
• Respect
• Responsibility
• Justice and Fairness
• Caring
• Civic Virtue and Citizenship
Principled Decision-Making
Sometimes defense acquisition personnel encounter ethical dilemmas. Guidance for resolving those
dilemmas can be found in a number of classical models. The Golden Rule—do unto others as you would
have them do unto you—is simple and timeless advice. Immanuel Kant’s belief in the existence of
absolute “higher truths” provides a starting point for identifying one’s moral obligations. Consequentialism
recognizes the complexity of ethics issues and advocates basing decisions upon consequences that yield
the greatest good.
The Principled Decision-Making model combines aspects of all three classical models. It calls for decisions
to take into consideration the welfare of all stakeholders. It also expects ethical values, such as
trustworthiness and fairness, to take precedence over other values, such as efficiency or self-interest.
Finally, it offers help in prioritizing conflicting ethical values based on what will bring the most good and
the least harm to others.
266

ACQ 202 Intermediate Systems Acquisition Course (ISAC)
Lesson Summary | Leadership and Ethics 2
Leadership Styles
Integrated Product Teams are an important part of the acquisition process, and effective leadership of
those teams is essential to their success. There are three primary types of leadership styles found in
today’s workplace:
• Supervisory-style: Typical in line-level supervision, this leadership style is characterized by
directing individual workers, providing them with one-on-one training, and resolving conflicts.
These leaders most often react to change, rather than initiate it.
• Participative-style: Effective in an IPT environment, this leadership style involves getting
multiple inputs prior to making decisions, developing team member performance, coordinating
group efforts, and implementing productive change.
• Team leadership-style: Highly effective in an IPT environment, leaders with this style create
team identity and maximize a group’s performance by capitalizing on the diversity of its
members. These leaders foresee and influence change to constantly expand the team’s
capabilities.
Version 1.0 (11 July 2014)
267

1.0 ACQ-203 SB 3-19-20
Intermediate Systems
Acquisition Course
ACQ 203
DAU strives to make you, the defense acquisition workforce, better at what you do. Your work is important to the nation and your learning is important to us. We constantly work to make the course more effective for you. Please let your instructor k…
Thanks,
Matt Ambrose
ACQ 203 Course Manager
matt.ambrose@dau.mil
Milestone Document Identification Tool – https://dap.dau.mil/mdid/
Student Assessment
1.1 ACQ-203 SB 3-19-20
1.2 ACQ-203 SB 3-19-20
Case 1.2, An Ethics Dilemma
____________________________________________
Case 1.2, An Ethics Dilemma (continued)
1.3 ACQ-203 SB 3-20-20 & CDD
Background
Situation
– How many contractors will develop and produce Firebird III in each phase of your acquisition strategy?
Assignment 3:

1.4 ACQ-203 SB 3-20-20
Review the following ACQ 202 CBT Lesson Summaries:
Assessment
Activity A – Enhanced Survivability
Scenario

2.1 ACQ-203 SB 3-20-20
Background:
2.2 ACQ-203 SB 3-20-20
Background:
2.3 ACQ-203 SB 3-20-20
1. For the test events described below come to a team consensus on whether they are DT&E, OT&E, or integrated DT/OT. Be prepared to discuss your answer with the class.
2. Identify opportunities for integrated test events (DT/OT) during the EMD phase on the FB II schedule. Be prepared to discuss your conclusions with the class:
2.4 ACQ-203 SB 3-20-20
Insert TPM Slides
2.5 ACQ-203 SB 3-20-20
3.1 ACQ-203 SB 3-20-20
 Lesson 3.1, Source Selection Process
Background:
3.2 ACQ-203 SB 3-20-20
Assignment 2:
3.3 ACQ-203 SB 3-18-20
Background:
Situation:
Option 1: Upgrade Current Firebird Communications Software
Option 3: Develop New Custom Software
Assignment:
Systems Engineering Software Development

3.4 ACQ-203 SB 3-20-20
Background:
3.5 ACQ-203 SB 3-18-20
Part 1: Contractual Direction and Authority
PM decides to use the new module after all
Intern to the Rescue

4.1 ACQ-203 SB 3-20-20
ACQ 202 summaries
1.0 Introduction
Milestone Document Identification Tool – 30Thttps://dap.dau.mil/mdid/30T
Student Assessment
1.1 IPPD
1.2 ACQ-203 Ethics
Case 1.2, An Ethics Dilemma
Case 1.2, An Ethics Dilemma (continued)
1.3 Acquisition Strategy
The Defense Acquisition Management System
PPBE Phases
“Typical” Contract Types by Phase
Technical Reviews and Testing
Logistics/Sustainment Planning
MSA Phase
TMRR Phase
EMD Phase
P&D Phase
O&S Phase
Warm Up Exercise
Capstone Exercise
1.4 Materiel Solution Analysis
Activity A – Enhanced Survivability
Activity B – Increased Range
2.1 Source Selection Planning
2.2 Systems Engineering
2.3 Test Planning
2.4 Technical Performance Measures
2.5 Scheduling
3.1 Source Selection Process
3.2 Contractor Performance Measurement (EVM)
3.3 Software & Interoperability
3.4 Reliability
3.5 Contract Change
4.1 Supportability
ACQ 202 Summaries
ACQ202 Considering_the_Cost
Learning Objectives
Analysis of Alternatives (AoA)
Cost Estimation
Cost Terms and Definitions
Distribution of Life Cycle Cost
ACQ202 Selecting_the_Best_Approach
Learning Objectives
Science and Technology
Basic Research
Applied Research
Advanced Technology Development
Advanced Technology Demonstrations (ATDs)
Joint Capability Technology Demonstrations (JCTDs)

Summary of Affordability and Should Cost information
ACQ202_M2_L03_Summary_Developing_the_Life_Cycle_Sustainment_Plan (12-31-15)
ACQ202_M2_L04_Summary_Risk_Management (6-30-16)
Learning Objectives
Risk Management Process Model
Risk Planning
Risk Identification
Risk Analysis
Risk Handling
Risk Monitoring
Program Risk Areas
Framing Assumptions and Ground Rules
Potential Risk Areas
Program Protection
Program Protection Plan (PPP)
Cybersecurity
Counterintelligence (CI)
Cybersecurity and the Risk Management Framework (RMF)

ACQ202_M2_L05_Summary_Developing_the_TEMP (6-30-16)
Learning Objectives
TEMP Purpose
TEMP Information
Types of T&E
Developmental Test and Evaluation (DT&E):
Operational Test and Evaluation (OT&E):
Live Fire Test and Evaluation (LFT&E):
Interoperability Testing:
Milestone Test-Related Products
Milestone A
Milestone B
Post Critical Design Review Assessment
Milestone C
Full Rate Production Decision Review
Developmental T&E Support Organizations
Operational T&E Support Organizations
Interoperability Capabilities
ACQ202_M2_L06_Summary_ESOH_Issues (6-30-16)
PESHE
NEPA
EO 13693 (Greening the Government)
ACQ202_M2_L07_Summary_Programming_Funds (6-11-14)
Learning Objectives
Financial Management Process
PPBE Process
Programming Phase Products
Programming and Budgeting Tools
Funding Policies
Full Funding Policy Exceptions
Escalation
Escalation Indices
ACQ202_M2_L08_Summary_RFP_Preparations_Part_I (12-31-15)
Learning Objectives
Program Manager and Contracting Officer Roles
Market Research
Socioeconomic Programs
Contract Types
Cost-Reimbursement Contracts
Fixed-Price Contracts
Firm Fixed-Price (FFP):
Fixed-Price Incentive (FPI):
Cost-Plus-Fixed-Fee (CPFF):
Cost-Plus-Incentive-Fee (CPIF):
Cost-Plus-Award-Fee (CPAF):
Acquisition Regulations and Policy
Integrated Program Management Report
ACQ202_M2_L09_Summary_RFP_Preparations_Part_II (9-30-15)
ACQ202_M3_L01_Summary_Source_Selection (9-30-15)
Learning Objectives
Source Selection Information Exchanges
Fact-Finding Information Exchanges
Clarification
Communication
Discussion, Negotiation, Bargaining
Negotiations
Bargaining
Discussion

Allowable Costs
Forward Pricing Rate Agreements (FPRA)
Direct and Indirect Costs
Financial Statements
Financial Indicators
Business Acumen
Organizational Structure
Types of Organizational Structures
Company Size and Organizational Structure

ACQ202_M3_L02_Summary_Technical_Risk_Management (6-30-16)
Learning Objectives
Systems Engineering Process
Modeling and Simulation
Work Breakdown Structure
Technical Performance Measures
Test and Evaluation (T&E)
ACQ202_M3_L03_Summary_Designing_for_Supportability (12-31-15)
Learning Objectives
Supportability
Designing for Supportability
Trade-Off Analyses
Open Systems
Reliability, Availability Maintainability, and Supportability
Reliability
Maintainability
Human Systems Integration
Supportability
Availability
Product Support Analysis (PSA)
Trade-Offs
ACQ202_M3_L04_Summary_Software_Design (9-30-15)
Objectives
System Architecture
Software Development Considerations
Sources of Software Problems
Software Development Methods
Software Measurement Techniques
ACQ202_M3_L05_Summary_Commercial_and_NDI (9-30-15)
ACQ202_M3_L06_Summary_Role_of_Manufacturing (9-30-15)
Learning Objectives
Benefits of a Producible Design
Designing for Producibility
Manufacturing Trade-Offs
ACQ202_M3_L07_Summary_Earned_Value_Management (12-31-15)
Learning Objectives
Earned Value
Performance Measurement Baseline
PMB Development
Step 1
Step 2
Step 3
Integrated Program Management Report
Format 1: Work Breakdown Structure
Format 2: Organizational Categories
Format 3: Baseline
Format 4: Staffing
Format 5: Explanation and Problem Analyses
Format 6: Integrated Master Schedule
Format 7: Electronic History and Forecast File
IPMR Purposes
ACQ202_M3_L08_Summary_Budgeting_Process (6-11-14)
Learning Objectives
Budgeting Phase
Program Objectives Memorandum and Budget Estimate Submission
Draft Resource Management Decision
Reclama
Final RMD
Execution Review
President’s Budget

ACQ202_M4_L01_Summary_Design_Changes (3-31-15)
ACQ202_M4_L02_Summary_Software_Problems (12-1-14)
Learning Objectives
Cause and Effect Diagram (Fishbone Diagram)
Software Best Practices
Adopt Continuous Program Risk Management
Estimate Cost and Schedule Empirically
Use Metrics to Manage
Track Earned Value
Track Defects against Quality Targets
Treat People as the Most Important Resource
Adopt Life Cycle Configuration Management
Manage and Trace Requirements
Use System-Based Software Design
Ensure Data and Database Interoperability
Define and Control Interfaces
Design Twice, Code Once
Assess Reuse Risks and Costs
Inspect Requirements and Design
Manage Testing as a Continuous Process
Compile and Smoke Test Frequently
Interoperability Testing
ACQ202_M4_L03_Summary_APB_Breaches (6-30-15)
Program Deviation
Interrelatedness of APB Parameters
ACQ202_M4_L04_Summary_Reprogramming_Funds (6-30-15)
Appropriated Funds Laws
Misappropriation Act
Anti-Deficiency Act
Bona Fide Need Rule
Reprogramming Funds
Below-Threshold Reprogramming
Operational Assessments
Combined Developmental and Operational Testing
ACQ202_M4_L05_Summary_Reviews_Simulations_and_Tests (6-30-15)
Exit Criteria
Milestone Reviews
Modeling and Simulation
Developmental Testing and Evaluation
Component Testing
Integration Testing
Environmental Testing
Production Qualification Testing
Production and Acceptance Testing and Evaluation
Modification Testing
Live Fire Test and Evaluation
ACQ202_M4_L06_Summary_Contractor_Performance_Measurement (6-11-14)
Performance Status Indicators
Budgeted Cost of Work Scheduled (BCWS)
Budgeted Cost of Work Performed (BCWP)
Actual Cost of Work Performed (ACWP)
Schedule Variance (SV)
SV = BCWP – BCWS
Cost Variance (CV)
CV = BCWP – ACWP

Performance Trends
Schedule Performance Index (SPI)
SPI = BCWP/BCWS
CPI = BCWP/ACWP
Cumulative CPIs and SPIs
Budget at Completion
Percent Spent (% Spent)
% Spent = (ACWP/BAC) * 100
% Complete = (BCWP/BAC) * 100
% Scheduled = (BCWS/BAC) * 100

Estimate at Completion
EAC = ACWP + Estimated Cost to Complete
EAC(CPI) = ACWP + [(BAC – BCWP)/(CPI)]
To-Complete Performance Index
TCPI (Target) = Work Remaining / Cost Remaining or
TCPI for LRE
TCPI for EAC
TCPI (BAC) = (BAC – BCWP)/(BAC – ACWP)

ACQ202_M4_L07_Summary_Integrated_Baseline_Review (6-30-15)
Performance Indices
Integrated Baseline Review
PMB Changes
ACQ202_M4_L08_Summary_Budget_Execution (6-11-14)
Budget Execution Process
Commitment
Obligation
Expenditure
Outlay
Participants in Budget Execution Process
Spending Plan
ACQ202_M4_L09_Summary_Operational_and_Live_Fire_Tests (12-31-15)
ACQ202_M5_L01_Summary_Best_Manufacturing_Processes (6-11-14)
Learning Objectives
Lean Manufacturing
Reducing Costs
Optimal Process
ACQ202_M5_L02_Summary_Unauthorized_Commitments (6-11-14)
Learning Objectives
ACO, CAO, PCO, and PI
Unauthorized Commitment
ACQ202_M5_L03_Summary_Production_and_Follow-On_Support (6-30-15)
Learning Objectives
Five Elements
Learning Curve
ACQ202_M5_L04_Summary_Contract_Modification (6-30-15)
Learning Objectives
Unstable Requirements
Acceptable Ways to Change a Contract
Change Orders
Supplemental Agreements
Product or System Modifications
ACQ202_M6_L01_Summary_Contract_Dispute (6-11-14)
Learning Objectives
Contract Terminations
Contract Dispute Resolution
Expired Funds
ACQ202_M6_L02_Summary_Life_Cycle_Product_Support (3-31-15)
ACQ202_M6_L03_Summary_Leadership_and_Ethics (6-11-14)
Learning Objectives
Ethics
Principled Decision-Making
Leadership Styles