MILSTD-6 b (USAF

4 JUNE l=
● SUPERSEDING
)’ MIL-SD-l~A lUSAm
,
1 JUNE 1976

MILITARY STANDARD

TECHNICAL REVIEWS AND AUDITS

mR SYSTEMS, EQUIPMENTS, AND
COMPUTER SOFIWARE
I }

AM:C IuO.!=?637 AaEA CMAN

— iA.---
~!~~,i~’H;C?i u ,ewifi~T A. ~pp~veo torpUIJtIC
r’eleaae:
Clisvlcunon
ISUnllmltM.
 MIL-STD-1521B

D&PAR’34ENT OF DEFZNSZ
WAS.HIWTON, D.C. 203C!

Technical itevie”~san~ Audits for Syster.s, Equipments, and Computer

Software

1. This Milltary Standard is approved for use by all Departments

and Agencies of the’ Department of Defense.

2. Beneficial conmients (recommendat ions, additions, deletions)

and any pertinent data which may be of usa in improving this
document should be aedressed to: Commander, liq tXD/ALET, Hanscom
AFB, MA 01731 by using the se~f-addressed Standardization Document
Improvement Proposal (DD Form 1426) appearing at the ●nd of this
document or by letter.

1-

●
 MIL-STD-1521B

FOREWORD

This standard has been designed ts take advantage of Cur::n:

technological advanceme!?t and mafiage!ne?tprocedures :n conducting
reviews and autiits.

‘o
i .’
i;;/~,,
.—-,.
I

: 141L-STD-1521B

CONTENTS

Paragraph Page
SCOPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
::1 Purpose ....................................... 1
1.2 Classification . ............................... 1
1.3 Application ............. ...................... 1
2. RSFERSNCED DOCU?4EW”
*S.................... ...... 3
3. DEFINITIONS ................................... 5
3.1 System Requirements Review (SIR) ..............
3.2 System Design Review (SIR) .................... :
3.3 Software Specification Review (SIR) ... ........ 5
3.4 Preliminary Design Review (PDRI ............... 5
3.5 Critical Design Review (cOR) ..................
3.6 Test Readiness Review (TM) ............... .... :
Functional Configuration Audit (FCA) ..........
H Physical Configuration Audit (PA) ............ :
3.9 Formal Qualification Review (FAR) ............. 6
3.10 Production Readiness Review (PAR) ............. -i
4. - GEIJSRAL REQUI RsFIENTs . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.1 Contractor Participation and Responsibilities.
4.1.1 Subcontrnc%ors and Suppliers .................. ;
4.1.2 Location ......................................
4.1.3 Contractor Requirements. ...................... :
4.2 Contracting Agency Participation .............. 10
4.3 Sample Forms .................................. 11
5. DSTAILED REQUIRSMSNTS. . . . . . . . . . . . . . . . . . . . . . . . 13
NOTES ................. . . . . . . . . . . . . . . . . . . . . . . . 15
Intended use .......... . . . . . . . . . . . . , . . . . . . . . . . 15
Data requirements list and cross reference. ... 15
Changes from previous sue . ............. ..... 1s

v
 HIL-StD-lS21B

F IGURSS

Section Page

Figure 1 i%qlneerlng and .est Flow.... ................. 16

Figure 2 Sample Action Icem Fem .......................
Figure 3 Sample Certification Attachment (?O.) ......... ;;
Figure 4 Sample Certification Attachment (Pm) ......... 96

APPENDIXES
Appendix Page
A. System Requirements Review (SRR) .............. 19
B. System Design Review (SDR) . . . . . . . . . . . . . . . 23
c. Software Specification Review SIR) ........... 31
D. Preliminary Design Review (PDR ............... 33
E. Critical Design Review (CDR).. ............... 53
F. Test Readiness Reviev (TAR) ................... 69
G.
H.
Functional Configuration
Physical Configuration
Audit (FCA) ..........
Audit (PA).............
71
75
●
I. Formal Qualification Review (FAR) ............. 83
J. Application Guide for Tailoring MIL-sTD-1521. . 117
K. Production Readiness Review (P!7R)........... ... 123

vi

1
 MI L-STD-J5Z1S

10
In
SE~lO?J 1

SCOPE

1.1 Purpose. This standard prescribes the requirements for :he

conduct or Technical Re%iews and Audits on Systens, Equipments,
and Computer Software.

1.2 Classification. The following technical reviews and audits

shall be selected by the program manager a: the appropriate phase
of program development. Zach review/audit is generally described
in Section 3, Definitions, and more specifically defined in a
separate appendix.
Systam Requirements Revie-~ (S2R)
System Design Review (SDR)
Software Specification Review (SSR)
Preliminary Design Review (?DR]
Critical Design Revie’4 (CiIR)
Test Readiness Review (TRR)
Functional Configuration Audit (FCA)
?hysical Configuration Audit (PCA)
Formal Qualification Review (FQR)
Production Readiness Review (PRR)
NCITE: A typicaI ●ngineering end test flow relative to program
activities is illustrated in Figure 1.
1.3 ADV lication. Technical Rev iews and Audits defined herein
shalr be conducced in accordance with this standard to the extent
specified in the contract clauses, Statement of Work (sow) , and
the Contract Data Requirements List. Guidance in applying this
standard is provided in Appendix J. The contracting agency shall
railer this standard to requi re only vhat is needed for each
individual acquisition.

10
1/2
I 141L-srD-ls21n

REFERENCED DOCUMENTS

2.1 Reference documents are not included in this document. The

Statement of Work shall be referenced for applicable documents.

(Copies of specif.icacions, standards, dravings, and

publications required by contractors in connection with
specific procurement functions should be obtained f?om the
contracting agency or .as directed by the contracting
officer).

‘o
3/4
 l’llLSllJ 15215

Q SECTION

DEFINITIONS
3

TSCHNICAL REVISW AND AUDiTS

3.1 System Requirements iteview (sRR) . The objective of this

review IS to ascertain th~qua~the contractor’s efforts in
defining system requirements. 1: will be conducted when a
significant portion of the system functional requirements has been
●stablished.
I
3.2 System Design Reyiew. (SDR). This review shall be conducted to
evaluate the optlmlzacion, correlation, Completeness, and risks
associated with the al~o~ated technical requirements. Also
included is a summa ry review of the system engineering process
which produced the allocated technical requircmencs and of the
1“ engineering planning for che nmxt phase of effort. 9asic
manufacturing consi~erations will be reviewe~ and planning for
1 production engineering in subsequent phases will be addressed.
This review will be conducted when the system definition
has proceeded to the point vhe re system
●ffort
characteristics are
defined an~ the configuration items are identified.
3.3 Softwere Specification Review (SSR). A review of the
finalized computer software Co-a:~a (CSCI ) require.meats
and operational concept. The SSR is conducted when CSC I
,0 requirements have been sufficiently defined to ●valuate. the
contractor’s responsiveness to and interpretation of the system,
segment, or prime item Level requirements. A successful SSR is
predicated upon the contracting agency’s determination that the
Software Requirements Specification, Interface Requireme!?ts
Specification(S), and Operational Concept Document form
satisfactory basis for proceeding into preliminary softvar~
design.
.

;iidu%+%? Wcxra- iK~~Lso~e”’&r%~~ S;

configurateion itema to (1) ●valuate the progress, technical
adequacy, and risk resolution (on a technical, cost, and scheduie
basis) of the
compatibility with
‘elect$rf%Xe ap~;~chGn&;!eZ~~eZ~~ia2;~
requirements of the Hardware Configuration Item (HwCI) development
specification, (3) -evaluate the degree of definition and assess
the technical risk associated with tI.e selected manufacturing
methods/processes, and (4) establish the existence and
Compatibility of the physical and functional interfaces amcr.g :he
con~igurat ioii item and other items of equipment, facilities,
c~mputer so:tware, and personnel. For CSCIS, this review will
focus on: (~) the evaluation of the progress, consistency, and
technical adeqtiacy of the selected top-level design and test
approach, (2) comparability b:::een software requ~reme:;s and
preliminary design. ant? (3) on .. preliminary version the
,0 5
 MIL-STD-1521B

operation anti support documents.

3.5 Critical DeSicn a:view, lCDR). This review shall be conducted

for each co-at Ion item when detail design is ●ssentially
complete. The purpose of this review will be to . (1) detemaine
that the detail design of the configuration item under review
satisfies the performance and engineering specialty requirements
of the HWCI development specifications, (2) ●stablish the detail
design compatibility among the configuration item and other it ems
of equipment, facilities, computer software and personnel. (3)
assess configuration item risk areas (on a technical, cost , and
schedule basis). (4) assess the results of the producibility
analyses conducted on system hardware, and (5) review the
preliminary hardware product specifications. For CSCIS, this
review will focus on the determination of the acceptability of the
detailed design, performance, and test characteristics of the
design solution, and on the adequacy of the operation and support
documents.
3.6 Test Readiness Review (TRR). A review conducted for ●ach CSCI
to d=mlne whether the s~e test procedures are complete and
to assure that the contractor is prepared for fonrial CSCI testing.
Software test procedures are evaluated for compliance with
software test plans and descriptions, and for adequacy in

I
accomplishing test requirements.
also reviews the results of
updates
At TRU, the contracting agency
informal software
to the operation and support documents.
zest ing and any
A successful TRR
●
is predicated on the contracting agency’s determination that the
software test procedures and informal test results form a.
satisfactory basis for proceeding into formal CSCI testing.
3.7 Functional Configuration Audi t (FCA). A formal audit to
valiaate that the development ~f a configuration item has been
completed satisfactorily and that the configuration item has
achieved che performance and functional characteristics specified
in the functional or allocated configuration identification. In
addition, the completed operation and support documents shall be
reviewed.
3.8 Ph sical Confi uration Audit (pCA)
of +ignat~ conf;-i~td ‘~Zhn;z~~;/xZ~!a?%
configuration item “As conforms to technical
documentation which defines the configuration item.
3.9 Formal Qualification Review (FQR). The test, inspection, or
anal= process by ~ a group of configuration items
comprising the system are verified to have met specific
contracting agency contractual performance requirements
(specifications or equivalent). This review does not ~cA a;:;y to
hardware or software requirements verified at che
individual configuration item,
●
6
 +fIL-STD-1521B

3.10 Production Readiness Review (PRR). This rev ● v is intended

to d~the stacus~c~ion of the specific actions
which must be satisfactorily aceompiis;ed prior to executing 8
production go-ahead decision. The review is accomplished in an
incremental f;;~~;;l during the Full-Scale Development phase.
usually t.Vo rev ievs and one final review to assess the
risk in exercising the production go-ahead decision. In its
earlier stages the PRR concerns itself with gross level
manufacturing concerns such as the need for identifying high
risk/low yield manufacturing processes or materials ar zhe
requirement for manufacturing developmentefforz to,satisfy aesign
requirements. ‘The reviews become more refined as the design
matures, dealing With such concerns as ‘production planning,
facilities. allocation, incorporation of producibility-oriented
changes, identification and fabrication of tools/test equipment .
long lead item acquisition etc. Timing of the increment&L PRRs is
a function of program posture and is not specifically locked in to
other reviews.
OTHER DEFINITIONS
3.11 For further guidance on cost terminology see the latest
edition of DOD I 5000.33, Uniform Budget/Cosz Terms and
Definitions.

● 1
3.12 New titles are being phased in for Eke levels of maintenance.
They are (with their
(Organizational), Off Equipment
former
- On Site
terms) : On Equipment
(Intermediate), off
*~~nt - Off Site (Depot). See the latest edition of AFR
Equipment 14nincenance Policies, Objectives, and
Responsibilities.
3.13 For definitions of the various levels of repair, see the
latest edition of MIL-STD-280A, Definition of item Levels, Item
Exchangeability, Models, and Related Terms.
3.14 Configuration item. Hardware or software, or an aggregation
of both. which is designated by the contracting agency for
configuration management.

‘o.
7/8
,1

MIL-STD-1521B

SKTION 4

GE.NSRAL R5QUIREMWTS

4.1 Contractor ?artici~ation and Responsibilities. The contractor

shall be responslDle for ~ductlng the Technical Reviews and
Audits in acco~dance with the followi;g requi-rements except :as
I amended by the contract.

I 4.1.1 Subcontractors
responsible for
anC Suppliers.
in~ing
The contractor
that subcontractors,
shall
vendors,
be
and
suppliers participate in formal Reviews/Audits, as appropriate.
.
4.1.2 Location. Unless otherwise specified in” the Statement of
Work, the Reviews/Audits shall be conducted at the contractor’s
facility or at a designated subcontractor facility, if approved by
the contracting agency. Accordingly, the contractor shall be
required to provide the necessary resources and material to
perform the Review/Audit effec~ively. This includesa~~e following
items to zhe ● xtent appropriate for the type scope of
Review/Audit required by the contract:
a. Meeting agenda/plahs
b. Conference room(s)
c. Applicable system engineering data, specifications.
drawings, manuals, schedules, and design and test data
d. Specialty study results
e. Trade study results
f. Risk analysis results

9. Mockups, breadboards. in-process hartivare, and finished

hardware
h. Test methods and data
i. Meeting minutes
4.1.3 Contractor Requirements. The contractor shall be
responsible for eszabi:snlng :fie time, place and acjenda for each
Revicv/Aud:: in consonance with the master .milestone schedule,
subject co coordiadation with the contracting agency. Thls should
be accomplished sufficiently in qdvance of each Review/Audit to
a~lo” adequaxe preparation for the nee:in~ by both the contractor
and the contracting agency (see 6.2). In addition, che contractor
shall:

● 4.1.3.1 Insure that each Fteviev/A,udiEscheduie is compatible

the availability of the necessary information and
with,
contract
 141L-sTD-1521B

articles, e.g., system

producibility
engineering
analysis results,
data,
risk
trade study results,
analysis results.
●
specificati~ns. cenzals, t!rawicqs, reports, hard=a:e. soft-~sre, or
mockups.

4.1.3.2 Prepare for ●ach ReviewlAudit in sufficient detail

consistent with the scope and magnitude of the Review/Audit.
4.1.3.3 Designate a Co-Chairperson for each Review/Audit.
Participating contractor and subcontractor personnel or chose
chosen to make presentations shall be prepared to discuss in
technical detail any of the presented material within the scope of
the review.
4.1.3;4 Provide a stenographer or other acceptable method to
record inputs to official meeting minutes. Minutes shall be
recorded only as dictated by either Co-Chairperson and shall
consist of signif icant questions and answers. action items,
deviations, conclusions, recommended courses of act ion resulting
from presentations or discussions. Conclusions from discussions
conducted auring side meetings shall be summarized in the main
meeting at an appointed time, and appropriate comments shall be
read into the offlc~al minutes. Recommendac ions not accepted
should also be recorded together with the reason for
non-acceptance. The minutes of each daily seszion shall be
available for review by both the contractor.a.nd contracting agent
personnel at the conclusion of each day’s session (see 6.2). *
4.1.3.5 Clearly record all ection items in che minutes and
identify whether contracting agency and/or contractor action is
required for its resolution. (See Figure 2 fur Sample Action Item
Form) .
4.1.3.6 Publish and dis:rihu:e official minuzes.

4.2 con:ractin~ ~ ParKici~ation.

4.2.1 Serves as CO-Chairperson.
4.2.2 Provides the name, organization, and security clearance of
each participating individual to the Contractor prior to each
Review/Audi C.
4.2.3 Reviews the daily minutes and ensures that they reflect all
sicjnificant con:Yacting agency inputs.
4.2.4 Provides formal acknowledgement to the contractor of the
accomplishment of each Review/Audit after receipt of Review/Audit
minutes (see 6.1). The contracting agency establishes the
adequacy of the contractor’s review performance by notification
of:
a. Approval ‘- to indicste that the review was satisfactory
a
10
 ZMIL-STD-1.521B

completed.
t
b. contingent approval -- co indicate that the rev iaw is not
considered accomplished until the satisfactory ccmple:ion of
resultant ac:ion items.

c. Disapproval ‘- to indicate that the review .4

as seriously
inadequate.
4.3 .samDle Forms. A sample action item form and sample
certi~on attachment are provided ‘for guida%e ‘purposes (;see
Figures 2, 3 and 4).

11/12
 ..

)41L-STD-1521B
*

SECTICN 5

DETAILED RSQUIWMENTS

5.1 The appropriate Reviews or Audits will be conducted as

specified in the following appendices (as selec:ed and/or modified
in the contract):
5.1.1 system !7equirenents aeview. See Appendix A.
5.1.2 System Design aeview. See Appendix B.

5.1.3 Software Specification Review. See Appendix C.

5.1.4 Preliminary Design Review. See Appendix D.

5.1.5 Critical Design Review. See Appendix E.
5.1.6 Test Readiness Review. See Appendix F.
5.1.7 Functional Configuration Audit. See Appendix G.
5.1.s Physical Configuration Audit. See Appendix H.
5.~.9 Formal Qualification Review. See Appendix I.
5.1:10 Application Guide For Tailoring )41L-ST9-1521. . See
Appendix J.
,5.~.11 Production Readiness Review. See Appendix K.

13/14
 MIL-STD-1521B

●✿ !
SECf’ION 6

No’!’”ss

6.1 Intended use. This standard prescribes the requirements for

contlu~ Tecnnlca~
— Reviews and Audits on Sys:ems, Equipments,
and Computer Software. Official acknowledgement by the
contracting agency of the accomplishment of a Fteview/Audi: is =ot
to be incerpreced as approval of statements made in the minutes or
of matters discussed at the aeview/Audit and does not relieve the
contractor fro= requirements which are -a part ‘of the contract.
6.2 Data requirements list and cross reference. Whea this
stand= is used In an~ui~io~ch Incorporates a DD Form
1423: Contract Data Requirements List ICDRL), the data
requirements identified below shall be developed as specified by
an approved Data Item Description (DD Form 1664) and delivered in
accordance with the approved CDRL incorpora:cd iato the contract.
When the provisions of the DOD FAR c labse on data requireme~ts
(currently DOD FAR Supplemen: 52.227-7031) are invoked and the DD
Form 1423 is not ur,ed. the data snecified below shall be delivered
bv the contractor in accordance with the contract or ourchase
o;tler requirements. Deliverable data reauired bv this stahdard is
cited in-the following paragraphs.

●
I
Paraqraph No
-— Data Requirement Ti:le AP plicable — No
DID -
4.1.3 Conference Agenda i)I-A-7088
4.1.3.4 Conference Minutes DI-A-7089

(Data item descriptions related to this standard, and identi i ed

section 6 wiil be approved end listed as such in DOD
i~O0.19-L. , vol. II, AMSDL. Cop”ies of data item descript ons
required by the contractors in connection with spec fic
acquisition ‘functions should be obtained from the - Naval
Publications and Fo rMS Center or as directed by the contracting
officer. )
6.3 Chanoes from previous issue. Asterisks or verzica: lines%re
not ~ ~ revision ~ntify changes wi:h respect to the
previous issue due to the expensiveness of :he chazses.

15
 ●
1

●
(

“o

,.
 MIL-STD-1521 B

Camsa. !lmlBm.—.

IaA= or .-X I SCW7: @UT:ON I

I
USIGND ‘lw: .XIG2YATC!R

b-- - ‘-” ‘-: I

1------ ““ ~ ““-’ I

17/18
 MI L-STD-1521B
<APPENDIX A

Review (SRR).

10.1 Generai. The SiGts ● re normally conducted during the system

Conceut 5xp10ration or Demonstration and validation phase. Such
revieifs say” be conducted at any time M t normaljy vill be
conducted af:er the accomplishment of functional analysis and
preliminary requirements allocation (to
operational/maintenance/training Hardware Configuration Items
[HWCIS), Computer Software Configuration Items (CSCIS), facility
config~ration items, +uanuf-acturing considerations, -personnel and
human factors) to determine initi-al direction and progress of the
contractor’s System Engineering Management effort and his
. . convergence upon an optimum and complete configuration. .
10.2 Purpose. The total System Engineering Management activity
and lts output shall be reviewed for responsiveness to the
Statement of Work an~ system/segment requirements. Contracting
agency direction to the contractor vill be provided: as necessary,
for continuing the technical program and system optimization.
10.3 Items to be Revieved. Representative items to be revieved
inclu-e=e=lcs of the following, as appropriate:
a. Mission and Requirements Analysis
● }
b. Functional F1OW Analysis
c. Preliminary Requirements Allocation

I d. System/Cost Effectiveness Analysis

e. Trade Studies (e.g., addressing system functions in
hardware/firmware/software)
f. Synthesis
I
9. Logistics Support fialysis
h. Specialty Discipline Studies (i.e., hardware and Softvare
,reliability analysis, maintainability aiIialysis, anaament
integration, electromagnetic compatibility,
.survivability/vul.ner.abilizy (including nuclear), inspection
methods/teck3iques analysis, energy management,
-environmental considerations).
.
i. System Interface Studies

]. Generation of Specifications
k. Program Risk Analysis

o .
1
.. Integrated Te.Si ?ianning

19
 MI L-STD-1521B
APPENDIX A

m. Producibility Analysis Plans

n. Technical Performance Measurement Planning

o. Engineering Integration

P. Data Management Plans

q. Configuration uanaqemenc ?lans

r. System Safety
s. Human Faccors Analysis

t. Value Engineering Studies

u .. Life Cycle Cost Analysis

v. Preliminary Manufacturing Plans
w. Manpower Requirements/Personnel Analysis
x. Milestone Schedules
10.3.: The contractor shall describe his progress and problems in:

10.3.1.1 Risk identification and risk ,ranking (the a

interrelationship among system effectiveness analysis, technical
performance measurement, intended manufacturing methods, and costs
shsll be discussed, as appropriate) .
10.3.1.2 Risk avoida>ce/reduction and control (the
interrelationships with trade-off studies, test planning, hardware
proofing, and technical performance measurement shall be
discusses, es appropriate).
10.3.1.3 Significant trade-offs among stated system/segmenz I
specification requirements/cons:ra ints and resulting engineering
design requirements/constraints ! manufacturing methods/process
constraints, and loglstic/cost of ownership
requirements/constraints and unit production cost/design-to-cost
objectives.
10.3.1.4 Identifying computer resources of the syszem and
partitioning t?e system in:o hwC IS and CSCIS. Include any
:rade-off studies concucted co eva:.~are alternative approaches and
methcds f~r meeting operational needs and to determine the ●ffects
of constraints on the system. Also include any evaluations of
logistics, technology. cost , schedule, resource limitations,
ir.teliigence ●stimates, ● ::., made to de:ermi.ne tneir inpact on
:he system. Ia aadition, address the following specific tradeoffs
:e~ated to :omputer reSCurCes:

20
●
 ●✎ Candidate programming languages and computer architectures
●valuated in light of DoD requirements for approved higher
order languages and stantiard instruction set architectures.
b. Alternative approaches ●valuated for iriplementing secxrity
requirements. If an approach has been selected, discuss how
it is the most economical balance of ●lements which meet the
total system requirements.
c. Alternative approaches identified for achieving the
operational and support concepts, and, for joint service
‘programs, opportunities for interser~ice support.
10.3.1.5 Producibility and ‘manufacturing considerations which
could impact the program decision such as critical components,
materials and processes, tooling and test ●quipment development.
production testing methods ! long lead items, and
facilities/personnel/ski 11s requirements.
10.3.1.6 Significant hazard consideration should be made here to
develop requirements and constraints ro eliminate or control these
system associated hazards.
10.3.2 Information ‘tihichthe contractor identifies as being useful
to his analysis and available through the contracting agency shall
● \
be requested prior
operational/suppOrt
plan(s), etc.).
to this
factors,
review
cost
(e.g.,, prior
factors, safety
studies,
data,
A separate SXR may be conducted for each of the
test
operational support subsystems depending upon the nature and
complexity of the program.
10.4 Post Review Action. After completing the SRR, the contractor
shall~~~and distribute copies of Review minutes. The
contracting agency officially acknowledges completion of the SRR
as indicated in paragraph 4.2.4.

! o 21/22

—
 MIL-STD-1521B
APPSNDIX B

20. System Desiqn Review (SDR).

20.1 Generai. The SDR shall be conducted to evaluate the

“—
oqcimlzatlon, traceability, correlation, ~ompleteness, and the
risk of the allocated requirements, including the corresponding
test requirements in fulfilling the system/segment requirements
(the functional baseline). The review ●ncompasses the total
systaa requirements, i.a., operations/maintenance/test/traininq
hardware, computer software, facilities, personnel, preliminary
logistic support considerations. AISO included shall be a summary
review of the System Sn’ginee’r ing Management Activities (e.g.,
mission land requirements analysis, functional analysis,
requirements allocation, manufacturing merhods/process selection,
prug ram risk analysis, system/cosc ●ffectiveness analysis,
logistics support analysis, trade studies, intra- and inter-
syatem intarface studies, integrated test planning, specialty
discipline studies, and Configuration Management) which produced
the above system definition products. A technicai understanding
shall be reached on the validity and the dearee of completeness of
the following information
a. System/Segment Spec fication
b. The engineering des gn/cost of the system (see Section 3,
Definitions) .
c. Preliminary Dperetional Concept Document
d. Preliminary Software Requirements Specification>
●. Preliminary Interface Requirements Specification(s)
$.
. Aa appropriate:

(1) Prime Item Development Specification

(2) Critical Item Development Specification
An SDR shall be conducted as “the final review prior
%“2t%%%%ittal of the ~ns~~;;~on and valida;~ye;ph;~;
products or as the initial Development
sys t●ms noc requiring a formal Demonstration and Validation Phase
but sufficiently complex to warrant the formal assessment of the
allocated requirements (and the basis of these requirements)
before proceeding vish tune preliminary design of :3WCIS or the
detailed requirements analysis for CSCIS. The SDR is primarily
concerned vi:h the overall review of the operational/support
requirements (i.e., the mission requirements), updated/completeti
System/Segment Specification requirements, allocated performance
requirements, programming and manufacturing
methods/processes/planning, and the accomplishment of the System
Zngi.qeering Management activities ta insure :hat the definition

23
 MIL-STD-1521B
APPENDIX B
●
●ffort products are necessary a!td sufficient. The ;ar?oses of the
SDR are to:
20.2.1 Insure that the updatedlcompleted Sysrem/Segment
Specification is adequate and Cost ,eff~ctive j~
satisfying
validated mission requirements.

20.2.2 Insure that the allocated requirements represent a complete

and optimal synthesis of the system requirements.

20.2.3 Insure that the technical program risks are identified,

ranked, avoided, and reduced through:

a. Adequate trade-offs (particularly for sensitive mission

requirements versus engineering realism and manufacturing
feasibility to satisfy the anticipated production quantities
of corresponding performance requirements):
b. Subsysrem/cOmponent hardware proofing;
C. A responsive test program; and
d. Implementation of comprehensive engineering disciplines
“(e.~., vors t case analysis, failure mode
analysis, maintainability analysis,
and standardization. )
and effects
producibility analysis ●
20.2.4 Identify how the final combination of operations,
manufacturing, maintenance, logistics and test and activation
requirements have affected overall progrem concepts: quantities
and equipment , unit product cost (see Section 3,
Defin;%~s,O~a:ag’raoh 3.12), compu:er software, personnel, anti
facilities.
20.2.5 Insure tha;n~ technical understanding of requirements has
been reached technical direction is provided to the
contractor.
20.3 Items to be Reviewed. The SDR shall include a review of the
follo~i=m~ as appropriate:
20.3.1 System Engineering Management Activities, e.rj.:
a. t4ission and Requirements Analyzis
b. Functional Analysis

c. Requirements Allocation
d. 5ystem/Cost Effectiveness
e. Synthesis

1, 2&
●✍✎
MIL-STD-i5ZlB
APPENDIX B

f. Survivabilisy~J’alterability (including nuclear)

9. Reliability/Maintainabi lity/Availability R/M/A )

h. Electromagnetic Co~patibiLity
i. Logistics Supporx Analysis (to address appropriate,
integrate~ logistics support including l~istlcs support
concept, maintenance, zsupply, :softvare su~port ftici.lities,
etc.)-

j. System Safety (emphasis shall be placed on system hazard

analysis and itienzification of safety test requirements)
k. Security
1. Human Factors
m. Transportability [including Packaging and Har.dling)
n. system Mass Properties
o. Standardization.

P- Electronic Warfare

q. Value Engineering
r. System Grovth Capability
s. Program Risk Analysis
t. Technical Performance Measurement Planning
u. Protiucibility Analysis and Manufacturing
v. Life Cycle Cost/Design to Cost Goals
w. Quality Assurance’Program
x. Environmental Conditions (Temperature, vibration, “Shock,
Humidity, etc.)

Y. Training and Training Support

z. Milestone Schedules

.s8. Software Development Procedures

20.3.2 Results of significant trade studies, for example:

,a. Sensitivity of selected miss on :equiremerts versus

25
 MIL-STD-1521B
APPSNDIX B

realistic performance parameters and cost estimates.

b. Operations desi5n versus maintenance design

c. System centralization versus decentralization

d. Automated versus manual operation

e. Reliability/Maintainability/Avai lability

f. Commercially available items versus new developments

9. National Stock Number (NSN) ite!es versus new development

h. Testability trade studies (Allocation of fault
detection/isolat ion capabilities between ●lements of
built-in test, on boarWon-site fault dezection/isolation
subsystems, separate support equipment, and ma~ual
procedures)
i. Size and weight

j. Desired propagation characteristics versus retiuction in

interference to other systems (optimum selection of
frequencies)
●
k, Performance/logiszics trade studies
1. Life cycle cost reducticn for different computer programming
languages
m. Functional allocation between hardware, software, firmware
and personnel,’procedures
n. Life Cycle Cost/system performance trade studies to include
sensitivity of performance parameters to cost.
o. Sensitivity of performance parameters versus cost

P. Cost versus performance

~. Design versus manufacturing consi~eration

r. Hake versus buy
s. Software development schedule
20.3.3 Up~ated design requirements for 9peracions/maintenance
functions and item.
20.3.+ Updated requirements for manufacturing sethods ant!
processes.

26
 MI L-sTD-1521B
APPENDIX B

‘( 20.3.5 Updated operations/maintenance requirements for facilities.

20.3.6 Updated requirements for Operationslmaintenance personnel

and training.

20.3.7 Specific actions to be performed include evaluations of:

I a. System design feasibility and systemlcost effectiveness

b. Capability o’f the selected configuration to tmeet
requis.ements of the System/Segment Specification

c. Allocations of System requirements to

subsystems/configuration items
d. Use of commercially available and standard parts
e, Allocated inter- and intra- system interface requirements
f. Size, weight, and configuration of HWCIS to permit
●conomical and ●ffective transportation, packaging, and
hsndling consistent with applicable specifications and
standards

9. Specific “design concepts whic”h may require development

toward ●dvancing the state-of-the-art
h. Specific subsystems/components which may require ‘hardware
proofing” and high-risk long-lead time items
i. The ability of inventory items to meet overall system
requirements. and their compatibility vith configuration
item interfaces
.
1, The planned system design in view of provitiing multi-mode
functions, as applicable
I k. Cansideraticns given to:
(1) Interference caused by the external environment to the
system and the system to the external ●nvironment.
(2) Allocated performance characteristics of all Sys:em
transmitters and receivers :0 identify potential
intra-system electromagnetic (SM) incompatibilities.

(3) Non-design, spurious and ha;ym~; system Performance

characteristics and their on electromagnetic
environments of operational deployments.

●✎ 1. Value Engineering studies, preliminary Value Engineering

Change P=oposals (VECPS) and VECPS (as applicable).
27’
 ..

MIL-STD-1521B
APPENDIX B
●
20.3.8 Review the Preliminary Operational Concept Document, azd
sections 1.0, 2.0, 3.0, 5.0, 6.0, and 10.0 of the Sys:em/Segment
Specification, all avaiiable HUC I Development Specifications,
preliminary Software Requirements, and Interface Requirements
Specifications for format, content, technical adequacy,
completeness and traceability/correlation to the validated
mission/support requirements. All entries marked “not applicable
(N/A)” or “to be determined (TBD)- are identified and ex?lained by
the contractor.

20.3.9 Review section 4.0 of the System/Segment Specification, all

available hardware Development Specifications, and preliminary
Software Requirements and Interface Requirements Specifications
for format, content, technical adequacy, and completeness. All
available test documentation: including NCI/subsystem and system
test plans, shall be revleved to insure that the proposed test
program satisfies the test requirements of section 4.0 of all
~~~;;$able specifications. All entries labeled ‘not applicable
or “to be determined (T3D)” in sect ion 4.0 of any
applicable speci fication are identified and explained by the
contractor.
20.3.10 Review the system, HWCI, and CSCI design for interactim.
with the natural environment. If any effect or interaction is no
ch~w~et;~~ understock and further study is required, or it is
not completely compensated for in the design, the
proposed method of resolution shall ~lSO b= reviewed. All
proposed environmental tests shall be reviewed for compatibility
with the specified natural environmental conditions.
20.3.11 Maintenance funct ions developed by the contractor to
determine that support concepts are valid, technically feasible,
and understood. In particular, attention is given to:
a. R/H/A considerations in the updated System/Segment
Specification
b. Maintenance design characteristics of the system

c. Corrective and preventive maintenance requirements

.S. Special eqaipment, tools, or material required
●. Requirements or planning for automated.maintenance analysis

f. Item Maintenance Analysis compatibility with required

maintenance program when weapon is deployed

9. Specific configuration item support requirements

h. Forms, procedural, and techniques for maintenance analysis ●

28
 FfIL-sTD-1521B
APPENDIX B

i. Maintenance related trade-off studies and findinas (ineluties

commercially available equipment, software faul~ diagnostic
techniques)

]. ~ogiscic cost impacts

K. Support procedures and tools for computer software which

facilitate software modification, improvements, corrections
and updates
1. Hardness critical items/processes
20.3.12 System compliance with nuclear, non-nuclear and laser
hardening requirements. High risk areas or design concepts
requiring possible advances of the s:ate-of-:he-art as a result of
survivability critieria shall be identified, and prepared

1 a9DrOaCh(tS) to the oroblem

shill be reviewed
specified
facilities.
threat environment
reviewed.
ior sufficiency
and
Prec.ared test nroarams
ancl compatibility ~it~ the
existing sinulacion test

20.3.13 The optimization, traceability, completeness, and risks

associated with the allocated technical :.equirements, and the

● ,
adequacy
proceeding
of allocated
with the
system requirements
development
as a
of hardware
configuration items. Include any available
basis for
and software
preliminary Software
Requirements and Interface Requirements Specifications.
20.3.14 Manufacturing (HUCIS onlv)
—-
20.3.14.1 Production feasibility and risk analyses addressed at
:he S.?..shall be updated and expardeti. This efforz should review
the progress made in reducing production risk and evaluate the
risk. remaining for consideration in the Full Scale Development
Phase. Estimates of cost and schedu:e inpacts shall be updated.
20.3.14.2 ReVieW of the Production Capability Assessment shall
include:
20.3.14.2.i A review of production capability shall be
accomplished which will constitute an assessment of the
facilities, ma:erials, methods, processes, equipment and skills
necessary to perform the full scale deveiopmen: acd procuc:ion
efforts. Identification of requirements to upgrade or develop
manufacturing capabilities shaii be made. Requirements for
Manufac?urinq Technology
. . (MANTXH 1 Drocrams
. . will also be
identified a= an element of this production assessment.

20.3.i4.3 Present the manasemenc controls “and the

design/manufacturing ●ngineering approach :0 assure chat the
equipment is producible.

29
 MIL-STD-1521B

I
APPENDIX B
●
20.3:14.4 Present ● review of trade-off studier for design
reqylrements against the requirement for producibility,
facilities, toolin~, production test ●quipment, inspection, and
capital equipment for intended production rates and volume.

20.3.14.5 The analysis, assessments and trade-off studies should

recommend any additional special studies or development efforts as
needed.

20.4 Post Review Action. After completing the SDR, the contractor
shal 1~ti~a~tribute copies of Review Minutes. The
contracting agency officially acknowledges completion of the SbR
as indicated in paragraph 4.2.4.

30
 MIL-STD-1521B
APPENDIX c

*
30. Softvare Specification Review (SSR).
I
30.1 General. The SS!? shall be a formal reviev of a CSCi”S
requi- specified the Software Requirements
Specification a% the Interfac~n Requirements Specification(s).
Normally, it shall be held after System Design Reviev but prior to
the start of CSCI preliminary design. A collective SSa for a
qro~p of configuration items, treating each configuration item
lndavidually, may be held when such an approach is advantageous to
the c?ntractinq agency. Its purpose is to establish the allocated
basellne for preliminary ‘CSCI design ~;e demonstrating ‘to the
contracting agency the adequacy of Software Requirements
~~R;~fication (SRS), Interface Requirements Specification(s)
, and Operational Concept Document (oCD).
30.2 Iteats to be reviewed. The contractor shall present the
follo~i=m= or review by the contracting agency:

a. Functional overview of the CSCI, including inputs.

processing and OUCPU:S of each function.
b. Overall CSCI performance requirements, including those for
execution time, storage requirements, and similar
constraints.

● ‘1
“
c. Control
functions
flow and data flow between
that comprise tne CSCI.
each of the software

d. All interface requirements between the CSCI and all other

configuration iteras both internal and external to the
system.
e. Qualification requirements that identify applicable levels
ana methods of testing for the software requirements zhat
comprise the CSC1.
~
f. -y special delivery requirements for the CSCI.

I 9. Quality factor requirements: i.e., Correctness, Reliab~l+ty,

Efficiency, Integrity, Usability, Haintainablllty.
Testability, Flexibility, Portability, Reusability, and
InterOperability.
h. .Mission requirements of the system and its associated
operational and suppor~ environments.

i. Functions and characteristics of the computer system within

the overall system.

j. Milestone schedules.

k. Updates since the last review to ali previously delivered

31
 1
! MIL-sTD-1521B
APPENDIX c
I
I‘ software related CDRL items.

1. tiy actions or procedures &eviating from approved plans.

I 30.3 Post Review Action.

shall~ti=a~tribute
After completing the SSR, the contractor
copies of Review Minutes. The
contracting agency officially acknowledges completion of the SSR
as indicated in paragraph 4.2.4.

1! 30.3.L The accomplishment of the SSR

configuration item Development
shall be recorded
Record
on the
by the contractor (see
UIL-STD-483, Appendix VII).
.
J

32
●
 PIIL-STD-15Z19
APPENDIX D

40. Preliminaryy Design Review (PDR)

40.1 General. The PDR shall be a formal technical review of the

basic design approach for a configuration item or for a
functionally related group of configuration item+. It shall be
heid after the hardware Development Specification(s), the Software
Top Level Design Document (STLDD), the Software Test Plan (STP),
the HUCI Test Plan, and preliminary versions of the Computer
System Operator’s Manual (CS2M), Soft”*are User’s Manual (SUM),
Computer System Diagnostic Manual (CSDM). and Computer Resources
Integrated “Support Document (CRISD) zare available, but prior to
the start of detailed design. For each configuration item the
actions described below may be accomplished as a single event. or
they may be sFread over several events, depending on the nature
and the extent of the development of the configuration item. and
on provisions specified in the contract Statement of Work. A
collective PDR for a group of configuration items, treating each
configuration izem individually, may be held when such an approach
is advantageous to the contracting agency: such a collective PDR
may also be spread over several events, as for a single
cor.figuration item. The overai 1 technical program r isks
associated with each configuration item shall also be reviewed on
a technical, cost, and schedule basis. For software, a technical
understanding shall be reached on the validity and the degree of
completeness of the STLDD, STP. and the preliminary versions of
the CSOM, SLE4, CSDt.1,and CRISD.
40.2 Items to be Reviewed. The contractor shall present the
follo~f~ Gview by the contracting agency:

40.2.1 HUCIS:
a. Preliminary design synthesis of the hardware Development
Specification for the iteE being reviewed.
b, Trade-studies and design studies resuits (see paragraph
20.3.2 of SDR for a representative listing).
c. Functional flow, requirements allocation data, and schematic
diagrams.
Equipment layout dmwings and preliminary drawings,
1 d.
including any proprietary
design/process/components and information.
or restricted

●. Environment control and thermal design aspects

f. Electromagnetic compatibility of the preliminary design

9. Power distribution and grounding design aspects

h. Preliminary mechanical and packaging design of consoles,

33
 I

M! L-STD-1S2LB
APPENDIx D

a
I racks, drawers, printed circuit boards, connectors, ● tc.

i. Safety engineering considerations

j. Security engineering considerations

k. Survivability/Vulnerabi lity (including nuclear)
considerateions

1. preliminsrg lists of materials, parts. and processes

m. Pertinent relability/maintainabi lity/availability data

n. Preliminary weight data
o. Development test data

P. Lnterface requireutencs contained in configuration item

Development Specifications and interface control data (e.g.,
interface control drawings) derived from these requirements

q. Configuration item development scheaule

r. Mock-ups, models, breadboards, or prototype hardware” when
●ppropriate
s. Producibility and Manufacturing Considerations (e.9.7
materials, toolxng, test ●quipment, processes, facilities,
skills, and inspection techniques). Identify single source,
sole .source, diminishing source.
t. Value Engineering Considerat ions, Preliminary VECPS and
VECPS (if applicable).
u. Transportability, packaginq, and handling considerations
v. Human Engineering and Biomedical considerateions (including
life support and Crew Station Requirements).
v. Stan&ardization considerations
I x. Description and characteristics of commercially available
equipment, including any optional capabilities such aS
special features, interface units, spec ia 1 instructions
controls, formats, etc. , (include limitations o:
commercially available equipment such as failure to mee~
human engineering, safety, end maintainability requirements
of the specification and identify deficiencies).

I r. Existing
manuals,
documentation
etc.,)
(technical
for commercially
orders , commercial
available eqq~ipment an
copies of contractor specifications used to procur
b
34
 ,
1’
! MI L-STD-15219
I APPENDIX D

iO,,- equipment shall be made available for review by the

contracting agency.
1,
z. Firmware to be provided wi:h the system: microprogram logic
diagrams and reprogrsuming/instruction translation al~orithm
descriptions, f“;;;ca;;~;~epackaging (integration technology
(e.g., LSI, types (e.g., 040S, PSIOS))r and
special equipment and support soft.aare needed for
developing, testing, and supporting the firmware.

.aa. Life Cycle Cost Analysis

ab. Armament compatibility
ac. Corrosion preventionlcontrol considerations
I
ad. Findings/Status of Quality Assurance Program

I 40.2.2 CSCIS:
a. Functional flow. The computer software functional flow
embed y ing all of che requirements allocated fram the
software Requirements Specification and Interface
Retirements Specification(s) to the individual Top-Level
Computer Software Components (TLCSCS) of the CSCI.
b. Storage allocation data. This information shall be
presented for each CSCI as a whole, describing the manner in
which available storage is allocated to individual TLcscs .
Timing, sequencing requirements, an~ relevant equipment
constraints used in determining the allocation are to be
included.
c. Centrol functions description. A description of the
executive control and start/recovery features for the CSCI
shall be available, including method of initiating system
operation and features enabling recovery from system
malfunction.
d. cSCI structure. The contractor shall describe the top-level
structure of the CSCI , the reasons for choosing the
components described, the development methodology which will
be used within the constraints of the available compu:er
resources, and any support programs vhich “~ill be required
in order to develop[maintain the CSCI structure .and
I allocation of data storage.
e. Security. An identification of unique security requirements
and a description of the techniques to be used for
implementing and maintaining securi:y within the CSCI shall
be provided.
a
. 35
 MIL-sTD-1521B
APPSNDIX D

f. Reentrance. An identification of reent~ancy e

requirements and 8 description of thean~echniques for
implementing reentrant routines shall be available.

9. Computer softvare de~elopment facilities. The availability,

adequacy, and planned utilization of the computer soft-dare
development facilities shall be addressed.

h. Computer software development facility versus the

operational system. The contractor shall provide
information relative to uziGue design features which may
exist in a TLCSC in order to allow use within the computer
software development facility, but which will not ●xist in
the TLCSC installed ifi the operational system. The
contractor shall provide information on the design of
support programs not explicitly required for the operational
system but which will be generated to aasist in the
development of the CSCI(S). The contractor shall also
provide details of the Software Development Library
control=.
I
i. Development took . The contractor shall describe any
special simulation, data reduction, or utility tools that
are not deliverable under the tenma of the contract, but
which are planned for use during software development.

j. Test tools. The contra:;;: shall describe any special

systems, test data, reduction
test
tools, test computer
●
software, or calibration and diagnostic software that are
not deliverable under terms Of the contract. but which are
planned for use during product development.
k. Description and characteristics of commercially available
computer resources, including any optional capabilities such
as spatial features, interface units, special instructions,
controls, formats, etc. Include limitations of commercially
available equipment such as failure to meet human
engineering, safe~y and maintainability requirements of the
specification and Identify deficiencies.
1. Existing documentation (technical orders, commercial
manuals, etc. ) for commercially available computer resources
and copies of contractor specifications used to procure
computer resourcaa shall be made available for review by the
contracting agency.

m. Support resources. The contractor shall describe those

resources necessary to support the software and firmware
during operational deployment of the system, such
operational and support har~vare and software, personne?~ .
special skills, human factors, configuration management ,

●
test , and facilities/space.

36
 1’ ML L-s”l’u-AaLLn
APPENDIX D

e.
\
n. Opera:ion and support documents. The preliminary versions
I of the CSO!4, SUM, CSDU , and CRISD shall be reviewed for
technical content and czmpatability with the top-level
design documentation.

o. Updates since the last review to all previously delivered

software related CDRL items.

P. Review considerations applicable to :c.2.l as appropriate.

40.2.3 .— ent (SE):

‘Sutmort-Eauismo
.
a. Review considerations .app.licable to pacagriaph 40.2.1 and
40.2.2 as appropriate.

b. ‘Ierify testability analysis resul:s. For example,

repairable integrated circuit boards are test po in~~
available so that failures can be isolated to the lowest
level of repair (See Section 3 Definitions, for ‘Levels of
repair-) .
c. Verify that the Government furnished SE is planned to be
used to the maximum extent possible.
d. Review progress of long-lead t ime SE items, identified
0 through interim release and SE Requ’ rements Document (SERD)
procedures.

e. Review progress toward tietermining total SE requirements for

installation, checkout, and test support requirements.
f. Review the reliability/maintainabi lity/availability of
support equipment items.

9. Id&ntify logistic support requirements for support equipment

items and rationale for their selection.
h. Review calibration requirements.
i. Describe technical manuals and data availability for support
equipment.

j. Verify compatibility of proposed support ecpipmen: ..-’i:> the

syszam maintenance concept.
k. If a Logistic SUppOrt Analysis (LSA) is not done , then
review the results 0: SE trade-off studies for each
alternative support concept. For existing SE and printed
circuit board testers, review Maintainability data resulting
frmn the field use of these equipments. Review the cost
difference between systems using single or multipurpose SE
10 Vs . proposed new SE. Examine technical feasibility in

I 37

I-_
 MI L-STn-1521B
APPSNDIX D

using ●zisting, developmental, and proposed new SE. For

mobile systems, review the mobility requirements of scpport
equipment.

1. Review the relationshipof the computer resources in the

system/subsystem with
those in Automatic Test Equipment
(ATE), Relate this to the development of Built In Test
Equipment (BITE) and try to reduce the need for complex
supporting SE.

40.3 Evaluation ~ Electrical, MechanicalL @ Loqical Desiqns

40.3.1 HUCIS. The material of paragraph 40.2.1 above shall be
evaluat~

a. Determine that the preliminary detail design provides the

capability of satisfying the performance characteristics
paragraph of the HWCI Development specifications.
b. Establish compatibility of the H’UCI operating
characteristics in each mode with overall system design
requirements if the HWCI is involved in multi-mode
functions.
c. Establish the ●xistence and nature of physical and
functional interfaces between t!teRUCI and other items of.0
equipment, computer s{ ftwar&, and facilities.
40.3.2 CSCIS. The material of paragraph 40.2.2 above shall be
evaluat~
a. Determine whether all interfaces between the CSC I and all
other configuration terns both internal and exrerna; co the
system meet the requi ements of the Software Requirements
Specification and 5nterface Requirements Specification(s).
b. Determine whether the top-level design embodies all the
requirements of the Software Requirements and Interface
Requirements Specifications.
c. Determine whether the approved design methodology has been
used for the top-level design.
d. Determine whether the appropriate Human Factors Engineering
(HFE) principals have been incorporated in the design.
●. Determine vhether timing and sizing constraints have been
met throughout the :op-level design.
f. Determine whether logic affecting system and nuclear safety
has been incorporated in the design.
 ..

)41L-STD-1521B
APPENDIX D

40.4 Electromagnetic Comnatibiliiy. Review HUCI design for

compliance Wl:.n electromagneclc compatibility/electromagnetic
interference (mc/!a41 ) requirements. Use Electromagnet ic
Compatibility Plan as the basis for this review. Check
application of MIL-STDS and MIL-Specs cited by the
system/equipment specification to the iiWCI/Subsystem design.
Raview preliminary ZMI test plans to assess adequacy to confirm
that E14C requirements have been met.
40.5 Design Reliability.

40.5.1 Identify the quantitative reliability requirements

specified in the hardware Development and Software Flequirements
Specification(s) , including design allocations, and the complexity
of the CSCIS.
40.5.2 Review failure rate sources, derating policies, and
prediction methods. Review the reliability mathematical models
and block diagrams as appropriate.
40.5.3 Describe planned actions when predictions are less than
specified requirements.
40.5.4 Identify and review parts or components which have a
o critical life or require special consideration, and general plan
for handling. Agencies so affected shall present planned act ions
CO deal with these components or parts.
40.5.5 Identify applications of redundant HWCI elements. Evaluate
the basis for their use and provisions fbr “on-line” switching of
the redundant element.
40.5.6 Rev iev critical signal paths to determine chat a
fail-safe/fail-soft design has been provided.
40.5.7 Review margins of safety for HUCIS between functional
requirements and design provisions for el~m~nts, such as: power
supplies , transmitter modules, motors, hydraulic pumps .
Similarly, review structural .eleinents; i.e. , antenna pedestals,
dish-s, and radomes to determine that adequate margrns of safety
shall :be provided between operational stresses and design
strengzhs.
I
40.5.S Review Reliability Design Guidelines fcr HWCIS :0 insure
that design reliability concepts shall be availabia and tised by
equipment designers. Reliability Design Guidelines shall include,
as a minimum, par: application guidelines !elec:rical dera:ing,
thermal derating, part parameter tolerances), part seiec:ion order
of preference. prohibited parts/mater:als, reliability
apportionments/preaict ions, and management procedures to ensure
compliance with the guidelines.

39
 141L-STD-1521B
APPEUDIX D

40.5.9 Review for fItiCIs preliminary reliability demonstration

plan: failure counting ground rules, accept-reject criteria,
number of test articles, test location and environment, planned
starting date, and test duration.

40.5.10 Review ●lements of reliability program plan to determine

that each task has bee n initiated cowa?d achieving specified
requirements.

40.5.11 Review s~contractor/supplier reliability controls.

40.6 Desiqn Maintainability

40.6.1 Identify the quantitative maintainability requirements

specified in the hardware Development and software Requirements
Specifications; if applicable, compare preliminary predictions
with specified requirements.
40.6.2 Review HWCI preventive maintenance schedules in terms of
frequencies, durations, and compatibility with system schedules.
40.6.3 Review repair rate sources and prediction methods.
40.6.4 Review planned act ions when predictions
specified requirements will not be attained.
indicate that
●
40.6.5 Review planned designs for accessibility, testability. and
ease of maintenance characteristics (including provisions for
automatic operator-controlled recovery from
failure/malfunc~;ons) to determine consistency with specified
requirements.
40.6.6 Determine if planned HWC I design indicates that parts,
assemblies, and components will be so placed that there is
sufficient space to use test probes, soldering irons, and other
tools without difficulty and that they are placed so -that
structural mambers of units do not prevent access to them or their
ease of removal.
40.6.7 RevieG provisions for diagnosing cause(s) of failure: means
for localizing source to lowest replaceable element: adequacy and
locations of planned test points: and planned system diagnostics
that provide a means for isolating faults to and within the
configuration item shall encompass on-line
diagnostics, off.lin~ d~~~~os~~~l~wand proposed technical orders
and/or commercial manuals.
40.6.8 Review for HWCIS the Design for Maintainability Checklist
to insure that listed design principles shall lead to a mature
maintainability design. Determine
●ngineers are using the checkiist.
that contractor design
●
40
I

MIL-sTD-1521B
●
--
APPENDIX D

40.6.9 Evaluate for !lWCIs the preliminary maintainability

demonstracicn P an, including number of maiczenance casks :hat
shall be acccmpl sbet!; accep:-reject criteria; general plans for
introducing fau. ts into the _CI and pe:sonnel involved in the
demortstrat ion.

40.6.10 Rev i●w elements of maintainability program plan

determine that each task has been initiated towards achievi~~
specified requirements.
40.6.11 Insure that consideration has been given to optimizing the
sy.steu+/item from a maintainability anti maixrexance vie-+oint and
that it is supportable within rhe maintenance concept as
developed. Also, for HWCIS insure ghat a Repair Level Analysis
(RLA) has been considered.
40.7 .—
?i~an ?UCtOrS

40.7.1 The contractor shall present evidence that substantiates

the functional allocation decisions. The Review shall cover all
operational and maintenance functions of the configuration item.
In particular, ● nsure that the approach to be followed emphasizes
the functions: integrity of the man with the machine to accomplish
a system operation.
40.7.2 Review desiqn tia:a, design descriptions and drawings on
system operations. ●quipments, and facilities to insure that human
performance requirements of the hardware Development and SOftvare
9equirernencs Specifications are me:. Zxamples of the types of
desisn information to be reviewed are:
a. Operatifig modes for each c?isplay szation, and for each mode,
the functions performed, the dis?lays and control used, etc.
b. The exact format and content of eech display, including data
locations, spaces, abbreviations, :he number of digits, all
special symbols (Pictographic) , alert mechanisms (e.g..
flashing rates). ate.

c. The control and data entry devices an~ formats including

keyboards, special function keys, cursor control, etc.
d. The format of all opera:or in?uts, zo~ether vith provisions
for error detection and correction.
I
e. All status, error, and dats printouts - includinq formats,
headings, &ata units, abbreviations, spacings, coi.umns, ● tc.

?hese should be presented in sufficient *etail allov

● contracting
usability
egency pe:sonnel
standpoint,
to judge
●nd design
atiequacy fro;” a human
personnel
required, and test personnel to prepare tes:s.
to know what is
 MIL-sTD-1521B
APPENDIX D
●
40.7.3 Make recommendations to update the System/Segment, or
Software Requirements Specification and Interface Requirements
Specification(s) in cases where requirements for human performance
need to be more detailed.
40.7.4 Review man/machine functions to insure that ma~’s
capabilities are utilized and that his limitations are not
exceeded.
40.8 Svstem Safety

40.8.1 Review resulzs of configuration item safety analyses, and

quantitative hazard analyses (if applicable).
40.8.2 Review results of systen sncl intra-system safety interfaces
and trade-off studies affecting the configuration item.
40.8.3 Review safety requirements levied on subcontractors.
40.8.4 Review known special areas” of safety, peculiar to the
nature of the system (e-q., fuel handlinq, fire pro~ection, high
levels of radiated ● nergy. high voltage protection. safety
interlocks, etc.).
40.8.5 Rev i● w
appropriate) .
results of preliminary safety tests (if ●
40.8.6 Generally review adequacy and completeness of configuration
item from design safety viewpoint.
40.8.7 Review compliance of commercially available configurateion
iterns or configuration item components with system safety
requiremen~s and identify modifications to such equipment, if
required.
40.9 Natural Environment
40.9.1 Review contractor’s planned design approach toward meeting
climatic conditions (~~~~~ting and non-operating ranges for
temperat.>~e, humidity, chat are specified in the Iiwc;
Development Specification.
40.9.2 Insure that the contractor clearly understands the ●ffect
of, and the interactions between, cne natural aerospace
environment: and HWCI design. In cases where the ● ffect and
ln:eraczions are not known or are ambiguous, insure that studies
are in progress or planned to make these determinations.
40.9.3 Current and forecast natural aerospace environment
parameters may be needed for certain configuration i:ems; e.g. ,
display o f airbase conditions in a command
calculation of impact point
and concrol
:or a missile. etc.
system,
Insure ●
 ‘141L-Sl!D-1521B
lo APPENDIX D
I
I compatibility between the config=razion i:em design and
appropriate meteorological communications by comparing
characteristics of the source [teletype, facsimiie, or data link)
I with thet of the configuration item.
plans to obtain needed
Insure that arrangements or
information> have been made and that
I adequate clisplay of natural environmental information shall be
provided.
( 40.10 Euui pment .—
and Part Standardization

40.10.1 —.
Eouinment and Components:

a. Reviev current and planned contractor actions to determine

that equipment or components for which standards or
specifications exist shall be used whenever practi=al.
(Standard item with NSN should have first preference).
b. Review specific trade-offs or modifications that may be
required of existing designs if existing items are, or will
be, incorporated in the fiUCI.
c. Existing designs will be reviewed for use or non-use based
on the potential impact m the overall program n the
following areas:
(1) Performartce
(2) cost

(3) Time
(4) Weight
(5) Size
(6) Reliability
(7) Maintainability
(8) Supportability
(9) Producibility
d. Reviav HWCI design to identify areas .-here a practical
design change would materially increase the number of
standard items that could be incorporated.
e. Insure that C~i:ical Item Specifications shall be prepared
for hardware it ems identified as engineering or log:stics
critical.

43
 MIL-sTD-1521B
APPE.NDIX D

40.10.2 Parts Standardization and Interchanqeabili:y:

—
a. Review procedures to determine if maximum practical use will
be made of parts built to approved standards or
specifications. The potential impact on the overall program
is to be evaluated when a part built :0 approved standards
and specifications cannot be used for any of the following
reasons:

(1) Performance
(2) Weight
(3) Size
(4) Reliability/Mantainabi lity/Availability
(5) Supportability
(6) Survivability (including nuclear)

b. Identify potential design changes that will permit a greater

use of standard or preferred parts and evaluate th
trade-offs.
@
c. Insure undarstantiing of parts control program operations for
selection and approval of parLs in new dasign or major
modifications.
d. Review status of the Program Parts Selection List.
e. Review status o: ail zoz-standa:d. par:s identified.

:. Review pending par:s control actions :hac may cause program

slippages, such as non-availability of tested parts.
of
40.10.3 Assignment — Official Nomenclature:
a. Insure understanding of procedure for obtaining assignment
of nomenclature and approvai of nameplates.
b. Determine that a nomenclature conference has been held and
agreement has been reached .zith the contracting agency on
the level of nomenclature: i.e., system, set. central,
group, componen:, sub-assembly, unit. etc.
40.11 Value Enaineerinq

40.11.1 Review the Can:ractor’s in-house incentive Value

Engineering Program, which may include but not be limited to th
foilowing: m
 MI L-STD-15Z1S
APPENDIX C

a, Contractor’s Value Er.q aeeriag Organization, poiicies arid

procedures.

b. Contractor’s ‘Jalue Erig neering Training Program.

c. ?otential Value Engineering projec:s, s:udies and VZC?S.

d. Schedule of.pl-anned Value Engineering zasks/evenzs.

e. Policies and procedures for subcan:rac:or ‘Jalue 2n5ineering
Programs.
40.~2 T~an~poy~abil~~v

40.12.1 Review HwCI :C determine if design meets contrac:s

requirements governinq size and weigh: to permit ● conomical
handling, loadino. securino. trans=ortino. and disassenblv for
shipment within ‘“existing-capabili~ies o~”military and comm~rcial
carriers. Identify potential outsized and overweight items.
Identify system/items defined as be ing hazardous. Ensure
packaging afforded hazardous items complies with hezardous
materials regulations.

● )
40.12.2 Identify HWCIS requiring special temperature ●nd
control or those possessing
humidity
sensitive and shock susceptibility
.’ characteristics. Determine special zransportat ion requirements
and availability for use with these HWCIS.
40.12.3 Review Transportability Analysis to determine t’ha
t
transportation conditions have been evaluated and thar these
condiiicns are reflected in the design of .crotecti,)e, ski;picq,
and handling devices. In addition size and weight
characteristics, determine that analysis incl~~es provisiaris for
temperature and humidity controls, rninitnizationof sensitivity,
susceptibility to shock, and transit damage.
40.13 Test
40.13.1 Review all changes to the System/Segment, Hwc I
Development , Software Requirements, an~ :n:erface Re.quiremenzs
Specifications s.~bsequent to the established ALiocate~ ‘Jaseline LO
determine whether See: ion 4.0 of ali these specif:ca:ions
adequately reflects these changes. .

40.13.2 Review information to be provided by the cort:ractor

regarding test concepts for Developmer.t Test an~ Evaluation (DTLE)
testing (both informal and formal) . Information shall include:

a. The organization and responsibilities of the group that wiil

● be responsible for test.

“.
 MI L-STD-1521B
APPENDIX D
●
b. The manege%e~t of his in-house development test effort
provides for:

(1) Test Mezhods (plans/?rocedures)

(2) Test Reports

(3) Resolution of problems and erro:s

(4) Retest procedure

(5) Change cont:ol and configuration management

(6) Identification of any special test tools that
are not deliverable under the contract.

c. The methodology to be used to meet quali:y assurance

requirements/qual ification requirements, includinq the test
repeatability charac:zristics and approach to regression
testing.
d. The progress/status of the test effort since the previous
reporting milestone.
40.13.3 Review status of all negative or provisional ●ntries
as ‘not applicable (tJ/A)” or ‘to be determined (TED)” in
such
Section
●
4.0 of the System/Segment, hardware Development, Software
Requirements or Interface Requirements Specifications. Review all
positive entries for technical adequacy. Insure that associated
test documentation includes zhese changes.
40.13.4 Review interface :est requirements specified in Section
4.0 of the hardware Development, Software Requirements, and
Interface Requirements Specifications for compatibility, currency,
technical adequacy, elimination of redundant test. Insure that
all associated test docunents reflect these interface
requirements.
40.13.5 Insure that all test planning documentation has been
updated to incluUe nev test s.apport requireciencs and provisions
for long-lead time support requirements.
40,13.6 Rev i●w contractor test Claza from prior zestinq to
determine if such daza negates the need for additional testing.
40.13.7 Examine all availabie breadboards, mock-ups, or devices
which will be us ●d in implementing the test program or which
affect the test program. for program impact.
40.13,8 Review plans far software Unit testing to ● nsure chat
they:
o

46
 tiIL-STD-1521B
APPSNDIX D

a. Address Unit level sizing, timing, and accuracy

requirements.

b. ?resent general anti specific reqcireaents that wiii be

demonstrated by Unit :estin~.

c. ‘Describe the required test-unique sup~o rt sof:.aare,

hardware, and facilities and the inzerrelatlonshi p of these
items.
d. Describe how, when, and from .vheee the test-unique support
items will be obtained.

e. Provide test scheduies consistent with higher level pians.

40.13.9 Review plans for CSC integration test n9 :0 ensure that
they:
a, Define the type of testinq required for each level of the
software struc:ure above the unit level
b. Pres6nt general and specific requirements that will be
demonstrated by CSC integration testing..
‘c. Describe the required test-unique support software,
hardware, and facilities.and the interrelationship of these
items.
d. Describe how, when, and, from vhere the test-unique support
items will be obtained.
e. Oescribe CSC integration test management, to include:
(1) Organi: ation and responsibilities of the test team
I
(2) Control procedures to be applied during rest
(3) Test reporting
(4) Review of CSC integration test results

(5) Generation of data to be used in CSC

integration testing.
.
.. ?rov itie:est schedules consistent “<ith higher ievel pians.
40.13.10 Review plans for formal CSCI testing to ensure tha: they:

a. Define tne objective of ● ach CSCI test, and relate Ehe test
to the soft.aare requirements being tested.

47
 .. MIL-sTD-1521B
APPENDIX D

b. Relate forma! CSCI tests to other test phases.

c. Describe support software. nardware, and facilities required

for CSCI ces:ins; and ho.~, ./hen, ar.d from where they will be
abtainet?.

d. Describe CSCI test roles and :espcnsibilities.

e. Describe requirements for Government-provided software,

hardware, facilities, data, and doc,umenzation.
f. Provide CSCZ tes: schedules consistent w: th higher-level
pla,ns.

9. Identify sof:.<are requiremer.:s that will be ver fied by each

formal CSC: test.
40.14 Maintenance and Maintenance
— Data
—— (HWCIS)
40.14.1 Describe System Maintenance concept for impact on *esign
and SE. Review adequacy of maintenance plans, Coverage shall be
provided for On Squipment (Organizational), off Equipment - On
Site (Incefmediate), off ZauiDment - off Site (DeDOt) level
maintenance of Government ‘Fu;nished Equipment (GbE) , and
Contractor Furnished Squipme.nt (CFE!. (See Section 3,
Definitions, para 3.12 for levels of maintenance. )
40.14.2 Determine degree @f understantiirtg of the background,
●
purpose, requirements, and usage of Maintenance (failure) Data
Collection and Historical/Status Records. (Ref D?ta Item titled,
“?.eliability and l.?ainzainability Data Reporting and Feedback
Failure Summary Report’s”).
40.:4,3 Oescr:be ne:ho~ of ?r~vic?i~g Maintenance, ?ai lure,
Reliability, !.taincainabiiityData co contracting agency.
40.14.4 Describe how requirements are submitted to the conz:ac:ing
agency for =uui~ment Classification lSQ/CL) Codes (formerly Work
Order Number Pref~x/Suffix Codes) when this requirement exis:s.
40.14.5 Review plans for (and stattis of) Work Unit Coding of the
equipment . Xcrk Unit codes shall be available for docume.n:i.-.g
Maintenance >a:a commencing .#ith configuration i:en/Subsysten
Testing. [Ref. Data Item ti:ied “Technical Orders” acd the
military specification on wo:’k ur.i: coding).

40.15 ——
Soa?es and Go.lernment
— Furnished ?roperty -(GF?)
40.15.1 Revie.d lcgis:ics and provisioning planning zc insure :’JIL
understanding of scope of requirements in tt.ese areas and that a
I reasonable time-phased p!an has been deveioped for accomplishment.
Of specific c:rcern are the areas of: provisionifiq requirements,
48
●
 MIL-sTD-1521B
APPfzNDIX D

,0
GFP usage. and spare parts. and support during installation,
checkout. and test.
40.15.2 Review provisioning actions and identify ●xisting or
potential provisioning problems - logistic critical and long-lead
time items are identified and evaluated against use of the interim
release requirements.
40.15.3 Review plans for maximum screening and usage of G?? , and
extent plans have been implemented.
40.15.4 Xeviev progress toward determining and acquiring total
installation, checkout, ..andtest support requirements.
40.16 Packagina/SDPE (SDecial Desiqn Protective Scuioment)
40.16.1 Anelyze all available specifications (System/Segment, HWCI
Development:, Soft.Jare Requirem.?nrs, Interface Fle.quirements,a~d
I Critical Items) for packaging (Sec%ion 5) requirements for each
product fabrication and material specification.
40.16.2 Evaluate user/operat ional support requirements and
I maintenance concepts for effect and influence on package design.
time phased plan for package design
●
40.16.3 Establish that
development is in consonance =ith the development of the equipment
design.
40.16.~fReview planned and/or preliminary equipment designs for
● ase packaging and simplicity of package design, and identify
areas where a practical design change would materially decrease
cost, weight, or volume of packaging required.
40.16.5 Reviev requirements for Si)?z necessary :0 effectively
support configuration item during transportation, handling and
storage processes. Insure SDPE is categorized as a configuration
item utilizing specifications conforming to the types and forms as
prescribed in the contract. Review SD?E development/product
specifications for adequacy of performance/interface requirements.
40.16.6 De:ennine initial package design baselines: concepts,
parameters, constraints, -etc.# to the txtent Possible at this
phase of the configuration item development process.
$0.16.7 :nsure previously cleveiooed and approved packaga design
data for like or similar cDnfiguratian items ;s bein~ utilized.

40.16.8 Establish plans for trade studies to Cetermine she most

economical and ( esirable packaging design approach needed to
satisfy the funct onal performance and iogistic requirements.

49
 MIL-STD-1S21B
APPENDIX D

40.16.9 Verify the adequacy of the prototype package design.

●
40.16.10 Review Section 5 of Specification to insure full
understanding by contractor for contractor requirements. Identify
package s?ecificacion used for hazardous materials.

40.17 Technical
— Manuals
—
40.17.1 Review status of the ‘Technical $lanual Publications Plan”
to insure :hat all aspects of the plan have been considered to the
extent that all concerned agencies are apprised of the technical
manual coverage to be obtained under this procurement. The
suitability of available commercial manuals and/07 modifications
there:o shall also be determined.
40.17.2 Review the availability of technical manuals for
validation/verification during the latter phases of DTCE testing.
40.17.3 If a Guidance Conference was not accomplished or if open
items resulted from it. then review as applicable provisions for
accomplishing TO in-process revjews, validation, verification,
prepublication. and poscpublicazion reviews.
40;18 System Allocation Document
40.18.1 Reviev the Draft System Allocation Document fo
completeness and technical adequacy to extent completed. *

40.18.2 The format shall provide the following minimum

information:
Drawing Number
;: Issue
Number of Sheets
;: Location
e. Configuration Item Number
f. Title
9. Part Number
h. Serial Number
i. Specification Number
j. Equipment Nomenclature
k. Configuration Item Quantity
1. ASSeMbly Drawing
40.19 Desion Producibility —
and Manufacturicq
40.19.1 The contractor shall demonstrate and present ●violence that
manufacturing engineering will be in:egraced iato the design
process.
. .

50
 kiIL-sTD-1521B
APPENDIX ‘D
,0
II a. The contracto:
oroauci~llity
shall provide
analyses
●v dence of perfoming
on development hartlvare trading of:
desi5s requirements aga i=s: mar..cfact.cri:.
g risk, cost ,
production, vol.ae, and existing capability/availa5il i:y.

Evidence of such analyses may be in zhe contractor” s own

fornat but must conclusively demonstrate that in-depth
analyses .+ere perfo-ed by qualified
organizations/ir.div idu.?is and the results of those analyses
will be incorporated in the design.

.b. ?reli~inary manufacturing ●ngineering an~ production

n:annlng demonstrations shall address: material and
component selection, preliminary prociuczion sequencing,
methods and flow concepts, new processes! manufacturing
risk, equipment and facility utilization for Intended rates
and volume, production in-process and acceptance zest and
inspection concepts. (Efforts :0 maximize productivity in
the above areas should be demonstrated. )
c. Management systems to be utilized will insure that
producibility and manufacturing considerations are
integrated throughout the FSD effort.

● 40.19.2 The producibility and manufacturing concerns identified

the SRR and the SDR shall be updated and expanded to:
in

a. PrOV ide evidence that cczcerns ideriti:ied in the

.Manufacturing Feasibility Assessment and the Production
Capability Estimate have been addressed and that resolutions
are planned or have been performed.
b. Make recommendat ions including manufacturing technology
efforts and provide a schedule of necessary actions cc tl?e
program office to resolve open manufacturing concerns and
reduce manufacturing risk.
40.20 ——
Post Review Action
—
40.20.1 After completing the PDR, the contractor shall publish and
distribute covies of Review minuzes. Tk,e contracting a~enc~
officially acknowledges completion of a PDR as indic~ted
paragraph 4.2.4.
40.20.2 The accomplishment of zhe ?DR shail oe recor<ed O!l
con firjuration item Development Xecar6 by :?.e zonzrac:or.

●
✼ 51/52

I
1!

Ii MIL-STD-1521B
APPSNDIX z

● 50. Critical Desiqn Review

I I 50.1 General. The CDR shall be conducted on each configuration
item prior to fabrication/production/coding release to insure that
the detail design solutions, as reflected in the Draf: Hardware
Product Specification, Soft”*are Detailed Design Document (SDDD),
Data Base Design Doc”uent(s) (DBDD(s)), Interface Design
Document(s) (IDD(s)), and ●ngineering drawings satisfy
1: r=?uir~ents established by the hardware Development Specification
and Software Top Level Design Document (STLDD). CDR shall be held
after the Computer Software Operator’s Manual (CSDM) , Software
User’s Manual (SUM), Cafuputer System Diagnostic Manual (CSDM),
Software Programmer’s Manual (SPM), and Firmware Support Uanua 1
(FSM ) have been updated or newly released. For complex/large
configuration items the CDR may be con~ucted on an incremental
basis. i.e., progressive reviews are conducted versus a single
CDR. The overall technical program risks associated with ● ach
configuration item shall also be reviewed on a technical (design
and manufacturing), cost and schedule basis. For software, a
technical understanding shall. be reached on the validity and the
degree of completeness of the SDDD, IDD(s], D8DD(s), STD. CRISD,
SPU , and FSU, and preliminary versions of the CSOM, SUM, and CSDM.
50.1.1 EuuiPment/?acilities configuration items. The detail
by the hardware
●
design Pr~ Specification,
dravings,a~che;;i~;~ mockups, e“tc., shall.be revie~ed against the
Ruc I Development Specification performance requirements. For
) other than facilities, the result of a successful WR shall be the
establishment of the design baseline for detailed
fabrication/production planning i.e., the contractor is permitted
to use the detail design as presented at CDR and reflected in the
hardware Product Specification for planning for production and, if
specifically authorized, for initial fabrication/production
efforts.
50.1.2 Corn uter Software configuration items (CSCIS). The CDR for
a CSCI ~besha a formal technlca= ew of tfieCSCI detail
design, including data base and i!tterfaces. The CDP. is normally
accomplished for the purpose of establishing integrity of computer
software design at the level of a Unit’s loa cal desicn .urior to
coding and ~esting, CDR may be accomplished at a ;ingie review
meeting or in increments during the development process
corresponding co periods at which components or groups of
components reach the completion of logical design. The primary
product of the CDii is a formal identification of specific software
documentation which will be released for coding and testing. By
mutual agreement between the contractor znd the contracting
agency, CDRS may be scheduled concurrently for t“+o or more CSCIS.
50.1.2.1 Since computer software development is an iterative
process, the completion of a CDR for a CSC: is not necessarily
● ,, sufficient for maintaining adequate visibiii:y into the remaining

53
\

I
 141L-STD-1521B
APPENDIX E

development ●ffort through testing.

50.1.2.2 Additional In-?rcgress Re”:ie=s aay be scheduied post-CDR

which address:

a. Response to outstanding action items

b. Modifications to design necessitated by approved ECPS or

design/program errors

c. Updating sizing and timing data

d. Updated design information, as applicable

e. Results obtained during in-house testing, including problems

encountered and solutions implemented or proposed.
50.2 Items to be Revieweti. The contractor shall present the
follo~-f= =view by the contracting agency:
50.2.1 ~
a. Adequacy of the detail design reflected in the draft
hardware ProduCt Specification in satisfying the
requirements of the HUCI Development Specification for the

b.
item being reviewed.
Detail engineering drawings for the HWCI including schematic
●
diagrams.
c. Adequacy of the detailed design in the following areas:
(1) Electrical design
(2) Mechanical design
(3) Environmental control an~ therms: aspeccs
(4) Electromagnetic compatibility
(5) Power generation anti grounding
(6) Elecr.rical and mechanical interface compatibility
(7) Mass properties

(8) Reliability/Maintainabi lity/Availability

(9) System Safe:y Engineering

(10 ) Security Engineering

54
 MI L-STD-.1521B
APPENDIX E

(11) Survivabi’..ty/Vulnerability
~ (including nuclear)

(12) Producibility and Manufacturing

(13) Transportability, Packaging and handling

(14) Human Ezgifieering and aiomet!ical Requirements (including

Life Support and Crew Station Requirements)
(15) Standardization
(16) DesiGn versus Logistics ?rade-o!fs
d. Interface control drawings

●. Mock-ups, breadboards, and/or prototype hardware

I f. Design analysis and test data

~- System Allocation Document for Huc 1 inclusion at each

scheduled location.

h. Initial Manufacturing Readiness (for examDle, manufacturing

●ngineering, tooling demonstrateions, development and
proofing of new materials, processes, methods, tooling, test
equipment , procedures, reduction of manufacturing risks to
acceptable levels).
i. Preliminary VECPS andlor formal VECPS

j. Life cycle costs

k. Detail design information on ail firmware to be provided
with the system.
1. Verify corrosion prevention/controi considerations to insure
materials have been chosen that will be compatible with
operating environment.
m. Findings/Status of Quality Assurance Program
50.2.2 CSC:S.
a. Software >etaiieti Designr Data 9ase Design, anti Interface
Design Document(s) . In cases where the CDR is =anducced in
increments. complete documents to support that increment
shall be available.
b. Supporting documente:ion describing :esul:s of an~lyses,
testing, ● tc. , as mutually agreeti by the contracting agency
and the contractor.

55
 HIL-STD-1521B
APPENDIX E ●
c. System Allocation Doc.ament for CSC I inclusion at each
scheduled location.

d. Computer Resources Integrated Support Document.

e. Software Programmer’s Manual

f. Firmware Support Manual

9. Progress on activities required by CSCI PDR (para 40.2.2 .

h. Updated operation and support documents (CSOM, SUM, CSDM

i. Schedules for remaining milestones.

j. Updates since the last review to all previously delivered

software related CDRL items.
50.2 3 Suuuort Equipment ~
a. aeview requirements (paragraphs 50.2.1 and 50.2.2) for SE.

b.
c.
Verify maximum considerations
Identify existing or potential
GFE SE
SE provisioning problams
●
d. Determine qualitative and quantitative adequacy of
provisioning drawings and data

e. Review reliability of SE
‘.
. Review logistic su?port req’airements for SE items

9. Review calibration requirements

h. Review documentation for SE.

50.3 Detailed Evaluation ~ Electrical, Mechanical, —and Loqical

Desiqns.
50.3.1 RWCIS. Detailed block diagrams, schematics. and log ic
diagram~ll be compared .ith interface control drawings to
determine system compatibility. Analytical and available test
data shall be revieved to insure the hardware Development
Specification has been satisfied.
50.3.1.1 The contractor shall provide information on firmware
which is included in commercially available equipment or to be
included in equipment developed under the contract.
this context
Firmware in
includes the microprocessor and associated sequence
of micro-instructions nacessary to perform the allocated tasks.
As a ❑inimum, the information presented during CDR shall provide
● ’

56
 ●
d41L-sTD-1521B
APPENDIX E

descriptions and status for the following:

1
a. Detailed logic flow diagrams

b. Processing algorithms

c. Circuit diagrams

d. clock and t.iming ‘data (e;g., :,tim

ing ~char.ts for
micro-instructions)

●✎ Memory (e.g., type (RAM, PROM), wore! length, size (total and
spare capacity))
f. Micro-instruction list and fomat
I 9. Device functional instruction set obtained by implementation
of firmware.
h. Input/outp~t data width (i.e., number of bits for data and
control. )

●
i. Self-test (diagnostics) within firmware.

J j- Support software for firmware development:

(1) Resident assembler
(2) Loader
(3) Debugging routines
(4) Executive (monitor)

I (5) Non-resident diagnostics

(6) Cross assembler and higher level language on host
computer
(7) Instruction .aimulator
50.3.2 CSCIS. The contractor shall present the detailed design
(inci:d~ rationale) of the CSCI :0 include:
a. The assignment of CSCI requirements to specific Lower-Level
Computer Software Components (LLcSCS) and Units, the
criteria andan~esign rules used to accomplish this
assignment, the traceability of’ Unit and LLCSC designs
to satisfy CSCI requirements, with ●mphasis on the necessity
●✌✌ and sufficiency
requirements.
of the units for implementing TLCSC design

57
 141L-STD-1521B
APPENDIX E

b. The overali information flow between software Units, the

method(s) by which each Unit gains control, and the
sequencing of Units relative to each other.

c. The design details of the CSCI, TLCSCS, LLCSCS , and Units

including ciata definitions, timing and sizing, data and
storage requirements and allocations.

d. The detailed design characteristics of all interfaces,

including their data saurce, tiestination, interface name and
interrelationships: and, if applicable, the design for
direct memory access. The contractor shall also give an
overview of the key design issues of the in:erfaca software
design, and indicate whether data flow formats are fixed or
subject to extensive dynamic changes.
●. The detailed characteristics of the data base. Data be.se
structure and detailed design, including all files, records,
fields, and items. Access rules, how file sharing will be
controlled, procedures for data base recovery/regeneration
from a system failure, rules for data base manipulation,
rules for maintaining file integrity, rules for usage
reporting, and rules governing the types and depth of access
shall be defined.
implementing
lanqu8ge
Data management rules and algorithms for
them shall be described. Details of the
r-yired by the user to access the data base shall
●
also be described.
50.4 Electromagnetic ComDatibiiitv:
a. Rev iew contractor EMC design of all HWCIS. Determine
compliance with requirements of the Electromagnetic
Compatibility Plan and h-JCI specifications.
b. Re,v;ev system EMC including ● ffects on the electromagnetic
● nvironment (inter-system EHC ) and intra-system *C.
Determine acceptability of EMC design and ?rogress toward
meeting contractual &UC requirements.
c. Review SMC test ~lans. Determine adequacy to con:inn EUC
design charac:erlstics of the system/EWCI/subsystem.
50.5 Desiqn Xeliabilitv:
50.5.1 Review the mosi. recent predictions of hardwa:e and software
reliability and compare against requirements specified in hardware
Development Specification and Software Requirements Specification.
For hardware, predictions are substantiated by review of parts
application stress data.
50.5.2 Review applications
minimum life, or those
of parts or configuration it ems wi:h
which require special consideration to ●
58
 ..

-MIL-STD-1521B
0 APPENDIX E

I
insure their effect on system perf~.mance is minimized.

S0.5.3 Review completeti Reliability Design !leviev Checklist to

insure principles have been satisfactorily reflected in the
configuration item design.

50.5.4 Review applications of redundant c~nfiguration item

elements or components co establish t~~~ expectations have
materialized since the PDR.

50.5.5 Reviev detailed WCI reliability demonstration plan for

compatibility with specified test requirements. The number of
tesr articles, schedules. Loca: ions, tes.c conditions, and
personnel involved are reviewe~v~~a;;sure a %utual understanding
of the plan and to provide . planning information to
activities concerned.
50.5.6 Rev+ev the failure data reporting procedures and met hods
for d?termlnation of failure trends.
50.5.7 Review the thermal altalysis of components, printed circuit
cards, modules, etc. Determine if these data are used in.
performing the detailed reliability stress predictions.
m 50.5.8 Review on-line diagnostic programs, off-line diagnostic
programs, support equipment, and preliminary technical orders
(and/or commercial manuals) for compliance with the system
maintenance concept and specification requirements.
50.5.9 Review software reliability prediction model and its
updates based upon test data and refined predictions of component
usage rates and complexity factors.
50.6 Design Maintainability
1
50.6.1 Review the alest rece>t predictions of quantitative
maintainability and compare these against :equiremcn:s s~ecified
in the H’UCi Development Specification and Software Requirements
Specification.

I 50.6.2 Review prev.entive ”aaintenance frequencies an? durations for

compatibility with overall syszem requirements and pianning
criteria.

50.6.3 Identify unique maintenance procedures req~::ed for the

configuration item during operational use and evaluate their roral
effec:s on system maintenance concepzs. Asscre :hac system is
I optimized from a maintenance and main:ainabi~i:y vievuoin:
....- and
.
COnfOrMS vlLh the planned m81ntenanCe concept. 731s shall Incluce
a review of prov-isions for au:omaric, sem~-automatic, and manual
recovery from hardware/sofr’#are failures and malfunctions.
“.
53
I

FIIL-STD-1521B
App~Ix ~

I
50.6.4 Identify design-for-maintainabi lity criteria provided by
the checklist in the design detail to insure that criteria have,
in fact been incorporated.

50.6.5 Detemine if parts, assemblies, and other it ems are so

placed that there is sufficient space to us ● test probes,
solder ng irons, and other tools without difficulty and that they
are p aced so that structural members of units do not prevent
access to them or their ease of removal.

50.6.6 Review detailed maintainability demonstration plan for

compa t bility vith specified test requirements. Supplemental
inform t ion is provided and reviewed to insure mutual
understanding of the plan and to provide overalla planning
information to activities concerned.
50.7 ——
Human Paccars
50.7.1 Review detail design” presented on drawings, schematics,
mockups, or actual hardware to determine that it meets human
performance requirements of the HWC1 Development Specification and
Software Requirements Specification, Interface Requirements
Specification(s), and accepted human ●ngineering practices.
50.7.2 Demonstrate by checklist or other formal means the ●dequac s
of design for human performance.
50.7.3 Review ● ach facet of design for man/machine compatibility.
Rev ie’+ time/cost/effectiveness considerations and forced
trade-offs of human engineering design.
S0.7.4 Evaluate the following human ●ngineering/biomedical design
factors:
a. Operator controls
b. Operator displays
c. Maintenance features
d. Anthropometry
e. Safety features and emergency cqui?ment

I :. Wo:k space layout

I cond ions (noise,
9. Internal environmental t ighting,
I ventilation, ● tc. )

h. Traini=g equipment
I
i. persanntl accomunodatiomi ●
60
I
I
MIL+TD-1S21B
APPENDIX .S

I 50.8 System Safety

50.8.1 i?evie”~configu:a:ion item detail design for compliance to

safety design requirements.
50.8.2 Reviev acceptance :es: req.uirenents .2
● insure adequate
safety req,~iremeflcsare reflected therein.
50.0.3 _~aluare :adequacy :of detailed design for safety -arid
protective equipment/c?evices.
50.8.4 Review Configuration iiem ope=aciorfal maintenance sa~ety
analyses and procedures.
50.9 Natural Environment
50.9.1 Review detail design to determine that “. meets natural
environment requirements of the hard;;re Development
Specification.
50.9.2 Insure that studies have been accomplished concerning
effects of the natural environment on, or interactions with, the
NWCI . Studies which have been in progress sha-ll be complete at
this time.
50.9.3 Determine whether arrangements have been made to obtain
current anti/or forecast natural ●nvironment information. when
needed for certain HWCIS. Assure conrpatibilicy of HUCI and source
of information by ccnparing electrical charac:eriscics and formats
for the source and the NUCI.
5C.1O Squiumer.: ——
a~d Parts Standar5izazion
50.10.1 z ui ment and Com onencs. Determine znat every reasonable
action Fi%%7TtZnto,u ~ the standardization req,tire;;;;;
for use of standard items (standard item with NS)J should be
preference) and to obtain app~oval for use of non-standard or
non-preferred items. Accordingly, the following criteria shall be
evaluated:

a. Data sources :ha: were reviewed.

b. ?ac:ors :nat were considered in the decis:on :0 rejecz kao=n
similar, exissing cesiqns.
c. Factors :ha: were considered in decisions :0 acceat any
exis:i~q designs which .~eye i>corpo:.a:ed, and t?&
trat!e-offs, if any, that had to be matie.

50.10.2 Parts

a. Determine “-he:her there are any outstar,ding non-standard or

 MIL-sTD-1521B
APPSNDIX E
●
ncn-preferred par:s approval requests and action necessary
for approval or ~isa;~r:...al
. (Sta:us of parts control
program operations).

b. Identify non-standar-a.Ron-Prefer:ed parts approvai proble!ns

and sta:us of actions towazd resolving tha problems.

c. Review potential fabrication/p?otiuction iine delays due co

non-availability of standard or preferred parts. In such
cases, dete=ine whether it is planned to request use of
parts which may be replaced by standard items during
subsequent support repair cycles. Assure that appropriate .
documentation makes note of these items and that standard
replacement items shall be provisioned for su>oort and used
for repair.
t. Require certification that max imum practical
interchangeability of parts exists among components,
assemblies, and ~cIs. Reservations concerning
interchangeability are identified, particularly for hardness
critical items.

e Sample preliminary drawings and cross check to insure that

parts indicated on the draw ngs are compatible with th
Program Parts Selection List. w
50. 0.3 Assignment ~ Official rJornencature.

a. Determine whether official nomenclature and approval of

r.ameplates have been obtained to extenz practical.
b. Determine whether DD Form 61, Request for Nomenclature, ~as
beefi pracessed to the agreed ievel of inden:ure.
c. Insure that approved nomenclature has been reflected in the
Development and Product Specifications.
d. Identify problems associated with nomenclature requests
(DD-61S) together with status of actibns towards resolving
the problems.
I e. Insure that a software invent~r-; nunberino s.~stem has been
~ agreed to and implemented to th~ CSCi lev;i.”

50.11 Vaiue E’naineerino (VS)

50.11.1 Review status of all VEC?S presented per the terms of the
contract .
50.~1.2 Review any new areas of potenciai V.alue Engineering
I considered profitable to cha~lenge.
●
62
 WL-STD-1521B
APPENDIX E

50.lL.3 If required by contract (funded VE prog?am), review the

actual Va lae Engineering acc~mpliskments against the planned VE
program.

50.i2 Trans=orcabilitv
50.~2.l Review tran~a~r~ability evaluations accomplished fcz those
items ide~tified is outsized, over-weight, sensitive, and/or
requiring special cemperatu:e and humidity controls.
50.12.2 Review actions taken ’as a “result of the above evaluation
to insure adequate facilities and military or commercial
transporting e~~ipnent are available ta sapport system
requirements during Production and Deplo’ymenc Phases.
50.12.3 Review design of spec ia 1 materials handling equipment,
when required, and accion taken to acquire equi?ment.

50.12.4 Insure DOD Certificates of Essentiality for movement of

equipment have been obtained for equipment exceeding limitations
of criteria established in contract requirements.
50.12.5 Insure transportability approval has been annotated on
design documents and shali remain as long as no design changas are
made that modify significant transportability parameters.
50.12.6 Identify equipne~t to be test loaded for air
I transpor:abi:ity of material i> Military A:rcrafr.
50.13 Test
50.13.1 Review updating changes co all specifications subsequent
to the PDR, to determine whether Section 4.0 of the specifications
adequately reflects these changes.
50.13.2 Review all available Cesr documentation for currency,
technical adequacy, and compatibility with Section 4.0 of all
Specificazi on requirements.
50.13.3 For any development model, prototype, etc., on which
testing may have been performed, exanine test ?esu;ts for design
compliance with hardware Development, Soft=are Requirements, and
Interfac= Requirements Specification requi~e-ents.
50.13.4 Rev iew quality assurance ?ro.3isions/quaiif ication
requirements in HWCI ?roduct, Software Requirements, or Interface
Requirements Specifications for completeness and technical
adequacy. Section 4.0 of these specifications shall include the
minimum requirements that che item, materiel, or process must meet
to be acceptable.
50.13.5 Review all test documentation requireti :0 suppo rt test
63
I
MiL-STD-i521B
APPENDIX E

I
requirements of Section 4.0 of HWCI Product Spec fications for
o
I compatibility, technical adequacy, and completeness.
1’
50.13.6 Inspect any breadboards, DOCkUDS, or Drototne
.. hardware
available for test-program implications.

50.13.7 Review Software Test Descriptions to e?sure they are

consis:enc vith the Software Test Plan &nd thev thoroughly
identify necessary parameters and prerequisites-” to enable
execution of each planned software test antimonitoring of test
results. As a minimum, test descriptions shall identify the
following for each test:

a. Required preset hard.~aee and Soft.-are conditions and the -

necessary input data, including the source for all data.

b. Criteria for evaluating test results.

c. Prerequisite conditions to be established or set prior to
test execution.
d. Expected or predicted test results.
50.14 Maintenance and Maintenance
— Data
50.14.1 Review adequacy of maintenance plans.
50.14.2 Review ststus of unresolved maintenance and maintenance
data problems since the PDR.
50.14.3 aeview status of compliance with Data Item titled
“Reliability, Maintainability Data Reporting and Feedback Failure
Summary Reports.”
50.15 Spare —.
Parts and Government Furnished Prooerty -(GFP)
50.15.1 Review provisioning planning through normal logistics
channels and Administrative Contracting Officer (ACO )
representative (Industrial Specialist) to insure its compatibility
(contant and time phasing) with contractual requirements (data and
SOW items). The end objective is to provision by a met hod which
shall iasure system supportability at operational date of the
first site. Also accomplish the folio-~ing:
a. Insure contractor understanding of contractual requirements,
inciuding time phasing, instructions from logistics support
agancies, interim release authority and procedure, end
responsibility to daliver spare/repair parts by need date.
b. Determine that scheduled provisioning actions, such tLS,
guidance meetinss, interim release and screening, are being
accomplished .sdequazely and on time.

. 64
●
●

I
 .,.
MI L-STD-1521B
APPENDIx E

~o
c. Identify existing or potentia: provisioning problems.
I
1! 5i).15.2 Jete:aine
Provisioning
quantitative
dra-~i~gs and data.
hrd qsa:itative at!equacy
Verify that Loaistics Critical
of

~tems are listed for-consideration anti t.ha- t ade~ate procedwes

exist for reflecting design change information in provisioning
I
documentation ant! Technical Orders.

I 50.15.3 Insure support :equirenen:s have been determined for

J installation, checkout, and tes: for approval by concracti>g
I agency. Insure screening has been accomplished and resul :s are
included into support requirements lists.
50.15.4 t)etemine tha: adequa:e s:ora~e space requirements >ave
been programmed for on-site handling of Installation and Checkout
(s&c), test support material, and a scheme has been developed for
‘down straamingm and joint use of insurance (high cos=) or
I catastrophic failure support items.
50.15.5 Assure that Acquisition Method Coding (AMC) is considered.
I 50.16 PackatJinq/SDPE
50.16.1 Review proposed pac’kage design to insure that adequate
i, protection to the HUCI , and che media on wh ch che CSCI is
recorded, is provided against natural and induced
! environments/hazards :0 which the equipment vi 1 be subjected
its life cycle. and to insure with
1 throughout
contractual requirements.
limited to, the folloving:
compliance
Such analysis shall include, but not be
I
a. Methods of preservation
b. -.
Physical/inechanical/snock protection including cushioning
shock mounting and isolation features, ioad factorsj
1 media,
support pa~s, cushioning &evices, blocking and bracing, etc.
c. Mounting facilities and sezuring/hold-down provisions
d. Interior and exterior container designs.
e. Handling provisions and compat ~Y with .airz?aft
materials handling system ($63L)
f. Container marking.

9. Consideration and identification dangerous/hazardous

commodities
50.16 2 Review ~esign of SDPE HWCI co determine if a cacegor:~ I
conta ner is required. The analysis of the proposed COnt8iner or

l\
e handl ngt sh.l?plng C<.Jlvalent shall encam?ass as a ~iniinum:

65
9
 MI L-STD-1521B
APPENDIX S
●
a. Location and type of inte:nal amunting or actactitng
provisions

b. Vibration - shock isc:ation features , based on the

pre-determined fragility rating (or other constraint of the
item %0 be sni?peti.)

c. Ser-~i:e items (indicators, relief .~a!ves, e:c.)

I d. Environmental control features

e. External handling, stacking -and tie-dobm provisions with

stress ratings

f. Dimensional and weitjht data (gross and net)

9. Bill-of-material
h. Marking ?rovisions including the center-of-gravity location
i, For wheeled SDPE (self-powered or tractOr/trailer) the
OVera~~ length, width, and height with mounted item, turning
radius, mobility, number of axles, unit contact load, number
of tires, etc.

I j. Position and travel of adjustable wheels, titling, or

adjustments to facilitate loading.
othe m

50.16.3 Review the results of trade studies, engineering analyses,

● tc., to substantiate selected package/SDPE design approach,
choice of mace:ials, handling pr~visions, environmental features,
etc.
50.16.4 Insure Chat packaqe/S3PE design provides reasonable
‘balance between cost and desirec? performance.
50.16.5 Review all preproduccion test results of the prototype
package design :0 insure that Lhe HWCI is afforded the proper
degree of protection.
50.16.6 Reviev Section 5, Packaging, of the !iWCI Product
Specification for correct format, accuracy an~ technical adequacy.
50.16.7 ?.ev~ew contractor procedures to assure that the
requirements of Section 5! ?repara:ion for Delivery of the
approved .HWCI ?roauc: Speciflca:ion, ‘Jill be incorporated into the
package tiesign data for provisioned spares.

●
50.17.1 Review maintenance of the System Allocation Document since
PDR .
66

I*
I
 1

MIL-STD-i521B
APPENDIX ‘E
p
50.17.2 Insure ?lans are initiated for Cocf guration item
re-allacacions that aa:y be necessary due :0 at :ions occ”~r:~ng
I prior co, or duri?g, CDR.

and .Yaaufac:uring
50.16 Desiun ?rsducibiiizy —
I
s~o~a.l ~ev~e., :he scacus of ali producibility (and productivity)
effar:s for :0s: and s=hed’uie considerations.

50.15.2 Review the status of efforts to resolve manufac:urixj

concerns identified in-previous technical reviews /and their cost
and schedule impact to the production-program.

53.18.3 9evie.~ :he status of *enufacturi>q ?echnoloSy >rograms an~

other previously recommended actions to reduce cost. manufacturing
risk and industrial base concerns.
5D.lB.4 Identify open manufacturing concerns :hitt require
additional direction/ef?or: co ninimize risk :0 the production
program.
50.1S.5 Review the status of manufact.~ring engineering efforts.
tool ing and test equipment demonstrations, proofing of new
materials, processes, methods, az~ special tooliz5/:est equipmen:.

●✌✌ 50.la.6 Review the intended manufacturing management system ctnci

organization for the production program in order to show how their
efforts will ● ffect a smooth transition into production.
50.19 Post
—— Review —
Action
50.19.1 After completing the CDFI, :ne contractor shall publish and
Distribute copies of Review mir.utes. The contracting agency
officially acknowledges completion of a CDR as indicated in
paragraph 4.2.4.
50.19.2 The accomplishment of the CDR shall be recorded on the
configuration item 3evelopmer.t Record by :he contractor.

I .. .67/’68
 MIL-STD-1521B
APPENDIX F

0.
\
1
60. Test Readiness
—— Reviev
— (TRR) .

60.1 General. The TRR

shall “m a formal review of the
contractor’s readiness to
begin formal CSC: testing. It is
conducted after software test procedures are available and CSC
integration testing is complete. The purpose of TRR is for the
contracting agency to determine .vhether the contractor is in fact
ready to begin CSCI testing. A technical understanding shail be
reached on the informal test resuits, and on the validity and the
degree of completeness of the Computer System Operator’s Manual
(Cstx.f), Software User’s Manual (SUM), and Compueer System
Diagnostic Manual (CSDM).
60.2 Items to be reviewed. The contractor shall present the
follo~f= Zview:
60.2.1 Requirements
Requirements Specl* Lcatlon ‘n~rcha!~~=r~~es~h~e~~~~~~
I Specification(s) .that have been approveti since ,
impact CSCI testing.
60.2.2 Desiqn chanaes. Any changes to the Softvare Top-Leve 1
Design Document, Software Detailed Design Document, Data Base
Design Document(s), or Interface Design Document(s) that have been

o)
made since PDR and CDR, and which impact CSCI testing.

60.2.4 Software test procedures. Test procedures ta be used in

conducting cscI— cesclng, lnc~uding retest procedures for test
anomalies and corrections.
60.2.5 ~ integration test cases. procedures, and results. Csc
integration test cases= ~ures used In ~ductLng ~nformal
CSC integration tests and the test results.
60.2.6 Software test resources. Status of the development
facility hardware,=overnment Furnished Software IGFS), test
personnel, and supporting test softvare and ‘materiaLs, including
software test tool qualification and review of the zracea~ility
between requirements and their associated tests.
60.2.7 Test limitations. Identification of all soft-aare test
limitat=.
60.2.8 Software problems. Summary of software problem status
including all known &lscrepancies of the CSCi and test support
software.
60.2.? Schedules. Schedules for remeinincj milestones.
‘ ●.
1. 69

I
 MIL-STD-1521B
APPENDIX F

60.2.10 Documentation Updates. Updates to all evolving an~

●
previously del].~ered CDRL Items (e.g,, CS31, SUM, CSDM).
60.3 ——
Post Review -
Action

60.3.1 After completing the TRR, the contractor shall publish and
distribute copies of Review Minutes. The contracting agency
officially acknowledges completion of a TRR as indicated in
paragraph 4.2.4.
60.3.2 The accomplishment of the TRR shall be recorded on the
configuration item De~elopment Record by ~he contractor.
.

70
 MI L-STD-1521B
APPSNDIX G

70. Furic:ioaai Con fi;urac ion katii~.

70.1 General. The o>:ecti”+e of :he Functional Configuration Audit

(FCA ) sha~i be to verify that the zcnfiguratioz item’s actual
performance complies .Jith its !-.arclwareOevelopmens or Software
ilequiremen:s and Interface Requirements .spezifications. ?est data
snail be :evie.ae~ :0 verify Lhaz the hardware or c~mpu:er software
performs as required by ics functional/allocated configuration
identification. Far ccmfi,g.urationitems ‘developed at Government
expense, an FCA shall be a prerequisite to acceptance of the
configuration item. For software, a technical understanding shall
be reached on the validity and :!vide5ree S: completeness of the
SoftvaFe Test aeporcs, arxi as appropriate, Conputer Systea
operator’s Manual (CSOH), ~oft”~are User’s Manual (SUM). and
Compu:er Sys:em Diacjnostic Manual (CSDM).
70.1.1 The FCA for a compiex configuration item may be conducted
on a progressive basis, when so specified by zhe contracting
agency, throughout :he configuratim item’s development and
culminates at the completion of the q)~alification testing o’f the
configuration item vi:h a review of all discrepancies at the final
FCA . The FCA shall be conduczed on that configuration of the
configurateion isam which is representative (prototype or
preprbduction) of the configuration to be released for production
of the operational inventory quantities. When a prototype or
preproduction article is not produced, the FCA shall be conducted
on a first production article. For cases where configuration item
qualification can only be determined through integrated system
testinac
. FCA’S for such confiauracion items wili no: be considered
complete until completion of ~uch integrated testing.
70.1.2 ~e:ar.?.ezda:i~ns of cor.figuration a:ev. acceptance or
non-acceptance :0 the local contract management agency are based
upon and governed by procedures and requirements outlined in
subsequent paragraphs.
70.2 Contract Requirements
70.2.1 The ‘schedules for the FCA shall be recorded on the
configura:io~ i:e~ &eveLoFmez: re:ord by :he contractor. A
:onfigura:ion item canno: be axditet! .W:::qc$ut:he contracting
agency authentication of the functional and allocaced baseiine.
In addition, tune contractor stiaii submit :F)e final Craft prod~~t
Specifica:ioc for the confiqura:ion i:ert :0 be audited to the
contracting a~ency for revie,~ pri3r to KA.

70.3 Contractor !lesoor,sibi

ii:-:
70.3.1 Prior to the FCA date (for configuration items to be

● audited) , the
to the contract
contractor shall provide the following information
ng ●gency (this information shall be provided in
✼ addition ta the general requirements of Section 4.):
71
●
 MIL-STD-1521B
APPENDIX G
●
a. Contractor rcpresentation (the Sest manager should be in
attendance) .

b. Identification of items to be a,udite~:

(1) Nomenclature

(2) Specification identification number

(3) Configuration Item number

(4) Current listing of all deviations/waivers against the
configuration item, ●ither requested of, or approved by
the contracting agency.

(5) Status of Test Programs to-test configured izeros with

automatic test equipment (when applicable).
70.4 Procedures and Requirements
—
70.4.1 The contractor’s test procedures and results shall be
reviewed for compliance with specification requirements.
70.4.2 The following testing
the FCA team.
information shall be available for
●
a. Test plans, specifications, descriptions, procedures, and
reporzs for the configuration item.
b. A COmplet!? list of successfully accomplished? functional
tests during which pre-acceptance data was recorded.
c. A complete list of successful functional tests if’ detailed
test data aye not recorded.
d. A complete list of functional tests required by the
specification but not yet performed. (To be performed as a
system or subsystem test).
e. Preproduction and production tast results
70.4.3 Testing accomplished with the approved test procedures and
validated data (witnessed) shall be sufficient to insure
configuration item performance as set forth in the specification
Section 3 and meet the quality assurance provisionsiquali:icat ion
requirements containeti i? the specification Section 4.
70.4.4 For those performance parameters which caznot completely be
verified during testing, adequate analysis or simulations shall
results of the
●
have been accomplished. The analysis or
simulations will be sufficient to insure configuration irem
performance as outiinad in the specification.
~~
 uIL-STD-1521B
APPENDIX G

70.4.5 Test reports, procedures, and data used by the FCA :em
shall be made a matter of record in the FCA minutes.
70.4.6 A list of the contractor’s internal docaunenca:ion
idrawings) of the configuration item shall be revieved to insure
that the contractor has documented the physical configuration of
the configuration item for wnich the test da:a are verified.
70.4,7 Drawings of HUCI parts which are to.be provisioned should
be selectively sampled to assure that test data ●ssential to
manufacturing are inclsaed on, or furnished with, the drawings.
70.4.0 Configuration items which fail to pass quality assurance
test provisions/qualificat ion requirements are to be analyzed as
to the cause of failure to pass. Appropriate corrections shall be
made before a configuration item is subjected to a
requalification.
70.4.9 A checklist shall be developed which identifies
documentation and hardware and computer software to be available
and tasks to be accomplished at the FCA for the configuration
item. See Pre-FCA checksheet.

● ,,
1
7IJ.4.1O Retesr.s or additional tests shall be ?erformed
compliance with paragraph 70.4.3.
tO assure

70.4.11 Acknowledge accomplishment of partial completion of the

FCA for those configuration items whose qualification is
continqe3t upon completion of integrated systems testing.
70.4.12 ?or CSCIS the following additional requirements shall
apply:

1 a. The contractor shall provide the FCA team with a briefing

for each CSCI being audited and shall delineate the test
results and findings for each CSCX. As a minimum, the
discussion shall include CSCI requirements that were not
met, including a proposed solution to each item, an account
of the ECPS incorporated and tested as .~ell as proposed. and
a general presentation of the .entire C5CI test effort
delineating problem areas us well as accomplishments.
I b. An audit of the formal test plans/descriptions/procedures
shall be made and compared against the official test data.
The results shall be checked for completeness and acsuracy.
Deficiencies shali be documented and made a part of the FCA
minutes. Completion dates for all discrepancies shall be
clearly established ant! documented.

‘o c. An audit of the Soft”aare Yes? Repor:s Shail be performed

validate :hat the reports are accurate
tO
and completely
. describe .the CSCI tests.
73
 ..

MIL-STD-1521B
APPENDIX G

d. Al: SC?S :tiaz have been approved shall be reviewed to ● nsure

:kst they have beefi technizaiiy incorporated and verified.

●. A1l ~gda:es to previously aelivered documents shall be

revie.~ed to ensure accuracy and consistency throughout the
documentation set.
f. Preliminary and Critical Desiqn aev iew rairiures shall be
examined to ensure that all findings F.ave been incorporated
ancl completed.

9. The interface :equiTemen:s a~d the :es:i?g of these

requirements shall be re~-ieved [or CSCIS.
h. Review data ba6e characteristics, storage allocation data
and timing, and sequencing characteristics for compliance
vith specifisd req-uiremen:s.
-
70.5 Post Audit —.
—— Actions
70.5.1 After completion of the !YA, the contractor shall publish
and disr.ribute copies of F~ minutes. The contracting agency
officially acknowledges completion of the FCA as in~ica:ed in
paragraph 4.2.4.
70.5.2 The accomplishment of the FCA shall be recorded on the
●
configuration item Development Record by thr contractor.

I ●
 MIL-STD-1521B
● APPCHDIX H

60. Physical Configuration Audit (PCA)

.—
80.1 General. The Physical Configuration Audit (pCA) shall be the
formal examlna:ion of the as-bui:t version of a configuration item
against its design documentation in order to establish the product
baseline. After successful completion of the audit, all
subsequent changes are processed by engineering chan5e action.
The PCA also determines that the acceptance testing requirements
prescribed by the “documentation is adequate for acceptance of
production units of a configuration item by quality assurance
activities. The PCA includes a detailed audit of engineering
drawings, specifications, technical da:a and ;ests utilized in
production of HW~~u~;~ a detailed audit of design documentation,
listinas. and for CSCIS. The review shall include an
1. audit ““o~ the released enaineerina documentation and quality
I control records to make sure- the as-built or as-coded
I configuration is reflected by this documentation.
the software Product Specification and Version
For software,
Description
Document shaAl be a part of the Pa review.
sO.~_l The PCA shall be conducted an the first article of
configurateion items and those that are a reprocurement of a
configuration item already in the inventory shall be identified
and selected jointly by the contracting agency and the contractor.
A PCA shall be conducted on the first configuration item to be
delivered bv a new contractor ● ven thouah PCA was Previously
I \ accom~lished- on the first article delive;ed by a ‘differen~
contractor.
i
80.1.2 Formal approval by the contracting agency of the
configuration tem Product spec, fication, and the satisfactory
cora:letion of a PCA results in establishment of the product
baseline.
ao.1.3 Recommendations of confiaurat ion item acceDcance or
nonacceptance to the responsible= contract adminstra;ion office
(CAO) are based upon and governed by procedures and requirements
outlined in subsequent paragraphs.
80.1.4 A final review shall be mat2e of all operation and support
documenzs (i.e., Compurer System Operatar’s Manual (CSOM),
Software User’s Manual (SUM), Computer System Diagnostic MaRual
(C.sml) , Software Programmer’s Manual (SPM), Firmware Sugport
Manual (FSM) ) to check format: completeness, and conformance with
applicable data item descriptions.
80.2 Contract Reouiremenzs

80.2.1 The schedules for the PCA shall be recorded on the

configuration item Development Record by the contractor. A
current set of listings shaLl be provided for each CSCI being
audited. The contractor shall s-it the final draft of the
75
 MIL-ST’D-1521B
APPSNDIX Ii

product spec fication for the configuration item to be aud ted to

the contract ng agency for review prior to PCA.

80.3 Contractor Responsibility

S0.3.1 The contractor shall Drovide the follovina inforsiation to
the contracting agency (this information sh~ll be provided in
accordance with the general inst:uctions of Section 4 and the
contractual requirements):

a. Contractor representation the test manager should be in

attendance). -
b. Identification of items to be accepted by:
(1) Nomenclature
(2) Specification Identification Number
(3) Configuration item Identifiers

[4) Serial Numbers

(5) Drawing and Part Numbers

(6) Identification Numbers

(7) Code Identification Numbers
(0) Software inventory numbering system
c.A list delineating all deviations/vaivers a~ainst the
configuration item either reques:ed or cor.tracting agency
approved.
80.3:2 The PCA cannot be performed unless data pertinent to the
conflquration item being audited is provided to the ?CA team at
time of the audit. The contractor shall compi le and make this
information available for ready reference. Required information
shall include:
● . Configuration item product specification.
5. A list delineating both approved and outstanding changes
against the configuration item.
c. Complete shortage list.
d. Acceptance test procedures and associated test data.

e. Engineering ckawing index including r,evis.ionletters.

o

76
1

lo
‘1 t
MIL-STD-15213
APPZf4DIX H

maintenance, I??2 illustrated parts breakdow.

II f. Qperating,
manuals.

! 9. Proposed DD F~rn 253, *Material Inspection and aeceiving

Report”.
h. Approved nomenclature and nameplates.
i, Software Programmer’s %anuals (SF?4S), Software User’s
Manuals (SUMS) Computer System Operators Manual (CSOM),
Computer System Diagnostic Mazual (CSiiM), and F irmnarc
SuppOrt Manual (FSX).

]. Software Version Description Document.

k. FCa minutes for each configuration item.

1. Findingz/Status of Quality Assurance Programs.

80.3 3 The contractor shall assemble and make available to the PCA
team at time of audit all data describing the item configuration.
Item configuration data shall include:

● a. Current approved
speci’ficatlon,
issue
Software
of hardware
Requirements
development
Specification, and
Interface Requirements Specification(s) to include approved
specification change not ices and approved
deviations/vaivers.

b. Identification of all changes actually made during test.

c. Identification of ali required changes not completed.
d. All approved drawings and documents by the top drawing
number as identified in i he configuration item product
specification. A1l drawings shall be of the category and
form specified in the contract.
● ✎ Manufacturing ins?r-~ction sheets for lWCi$ identified by the
contracting agency.

80.3. 4 The contractor shall identify any difference between zhe

physical configurations of the selected production unit and the
Development Unit(s) used for the FCA and shall certify or
demonstrate to the Government that these differences do not
degrade the functional characteristics of the selected units.
80.4 —
PCA Procedures —
and Reouireme>ts

● .
80.4.1 DravinS
Instructions:
and S4acc$acturinq ?nst:ucz ion Sheet Review

..
77
 MIL-STD-1521B
APPENDIX H

a. A representative number of drawings and associated

manufacturing instruction sheets for each item of hardware,
identified by the contracting agency Co-Chairperson, shall
be reviewed to de:enaine their accuracy and insure that they
include the authorized changes reflected in the engineering
dravings and tk.e hardware. Unless otherwise directed by the
contracting agency Co-Chairperson, inspection of drawings
and associateri manufacturing instruction sheets may be
accomplished on a valid sampling basis. The purpose of this
review is to insure the manufacturing instruction sheets
accurately reflect all design details contained in the
drawings. Since the hardvare is built in accordance vith
the manufacturing instruction sheets, any discrepancies
between the instruction sheets and the design details and
changes in the drawings will also be reflected in the
hardware.
b. The followiog minimum information shall be recorded for each
&raving reviewed:
(1) Drawing number/title (include revision letter)

(2) Date of drawing approval

(3) List Of manufacturing

change Letterltitles
instruction sheets (numbers with
and date of approval) associated
●
with-this draving.

(4) Discrepancies/comments
(5) Select a sample of part numbers reflected on the
dravinq. Check to irsure compatibility “~ith the Proqram
Parts Selection List, and examine the HUCI to insure
that the proper parts .are actually installed.
c. AS a minimum, the following inspect ions shall be
accomplished for each drawing and associated manufacturing
instruction sheets:
(1) Drawing number identified on manufacturing instruction
sheet should mazch latest released drawing.

(2) List of materials on manufacturing instruction sheets

should match materials identified on the drawing.

(3) All special instructions called on the draving should be

on the manufacturing instruction sheets.

(4) All dimensions, tolerances, finishes, ● tc., called out

on the drawin~ should be identified on the mancfaczurin
instruction sheets. a

78
 KIL-STD-1521B
APPENDIX H

(5) All special processes called out en the drawing should

be dentified on the manufacturing instruction sheets.

(6) Nomenclature descriptions. Dart numbers and serial

number markings caiied out“ - on the drawing should be
identified on the manufacturing instruction sheets.

(7) Review drawings and associated manufacturing instruction

I sheets to ascertain that all approved changes have been

incorporated into the configuration item.

(8” Check ‘release record to insure all drawings reviewed are

identified.
(9 Record the number of any drawings containing more than
five outstanding changes attached to the drawing.

10) Check the drawings of a major assembly/black box of the

hardware configuration item for continuity from top
drawing down to piece-part drawing.
80. 4.: Review of all records of baseline configuration for the
RUCI bv direct comparison with contractor’s enqineerinq release
system -and change ~ontrol procedures to ●stablish ;hat the
● )
configuration
engine-ring
provisioned
being produced
data.
prior
This
does accurately
includes
to PCA to ensure
interim
delivery
reflect released
releases of spares
of currently
\ configured spares.
8G.4.3 Audit of contractor’s engineering release and change
control system to ascertain that they are adequate to properly
control the processing and formal release of engineering changes.
The minimum needs and capabilities set forth belov are reauired of
his engine~:~ng release ~ecords system. The contractor’s ”fonnats,
systems, procedures are to be used. Information in addition
to the basic requirements is to be considered part of the
contractor’s internal system.*

80.4-3.1 As u alnirnmm, ‘the “Zollowfng information shall be

contained on one release record supplied by the contractor,
subcontractor, or vendor for.each drawing .nuuber, if applicable:
a. Serial numbers, top drawing number, specification number;

b. Drawing number”, title, code number, number of she?ts, date

● Contract Administration Office (CAo) Quality Assurance

Representat ive [QAR) records can be reviewed for purpose of
determining the contractor’s present and most recent past
performance.
79
 ,

MIL-STD-1521B
I APPENDIX H

I
I of release, change letter, date of change letter release,
●
●ngineering change order (ECO) number.
1
I 00.4.3.2 The contractor’s release function and documental on will
be capable of determining:

a. The composition of anv Dart at any level in erms of

subordi~ate part nu.nbe~s ~disregard s~andard parts);

b. The next higher assembly using the part number, except for
assembly into standard parts;

c. The composition of the configuration item or part number

with respec: to other configuration items or part numbers.:

d. The configuration itez and associated serial number on which

subordinate parts are used. (This does not apply to
contractors below prime level who are not producing
configuration items); .
e. The accountability of changes which ha-de been partially or
completely released against th? configuration item;
f. The configuration item .snd serial number effectively of any

●
change.

9. The standard specification number or standard part numbers

used within any nonstandard part number;
h. The contractor specification document and specification
control numbers ~ssociared with any subcontrac~or, vendor,
or supplier part number.
BO.4.3.3 The enginecrinq release system and associated
documentation shall be capable of:
a. Identifying changes and retaining records of superseded
configurations formally accepted by :he contracting agency;
b. Identifying all engineering changes released for production
incorporation. These changes shall be completely released
and incorporated prior to formal acceptance of the
configuration item:
c. Determining the configuration released for each
configuration item a: the time of formal acceptance.
80.4.3.4 Engineering da:a shall be released or processed through a
central authority :0 ensure coordinated acrion and preclude
unilateral release of data.
80.4.3.5 Engineering change control numbers shall be unique,

ao
●
 MIL-STD-1521B
APPENDIX H

● 80.4.4 Difference betveen the configuration of the configuration

item qualified and the configuration item being audited shall be a
matter cf record iz the minutes of the ?CA.

80.4.5 For Xl acceptance tests date and procedures shall comply

with its product specification. The PCA team shall determine any
acceptance tests to be reaccomplished, and reserves the
prerogative t~ have representatives of the contracting agency
witness all or any portion of the required audits, inspections, or
tests.
BO.4.6 WCIS which fail to Pass acceptance test requirements,shall
be repaired if necessary and be retested by the contractor an the
manner specified by the PCA team leader in accordance with the
product specification.

SC.4.7 The ~$tractor shall present data confirming the inspection

●nd test subcontractor equipment end iterns at point of
manufacture. Such data shall have been witnessed by Government
representative.
80.4.8 The PCA ~eam reviews the prepared back-up data (all initial
documentation which accompanies the configuration item) for
correct types and quantities to ensure adequate coverage at the

● ‘)
tiaw of shipment to the user.
80.4.9 Configuration items which have demonstrated compliance with
che product specification are approved for acceptance as follows:
a. The Pa team shal 1 certify by signature that the
configuration item has been buil: in accordance with the
drawings and specifications.
.90.4.10 AS a minimum, the following actions shall be performed by
the PCA team on each CSCI bein9 audited:
a. ~ev iew all documents .#hich will comprise the Software
Product Specification for format and completeness
b. Review FCA minutes for racorded discrepancies and actions
taken
c. !?eview the design descriptions for proper entries, symbols,
labels, taqs, references, and data ~escriPtlons.
d. Csrapare Tap-Leve: Computer Soft”~are Cocponen: (7LCSC) design
descriptions .~itb, LoMe:-Level Compu:er Software Components
(LLCSC) descriptions for consistency

e. Com?a re all lover-level design c!escriptions with a~~

software iistings for accuracy and completeness
e
 MIL-STD-1521B
APPENDIX H

I f. Check Software User’s Manual(s), Software Programmer’s a

Manual , Computer System Operator’s Manual, Firmware Support
Manual , and Computer System Diagnostic Manual for format
completeness and conformance with applicable data item
I descriptions. IFormal verification/acceptance of these
manua 1s should be withheld until system testing to ensure
I
that the procedural contents are correct)

9. Examine actual CSCI delivery media (card decks, tapes.

●tc.,) to insure conformance with Section 5 of the Software
Requirements Specification.
h. Review the annotated listings for compliance with approved
coding standards (e.g. Appendix C of DOD-STD-2A67).
80.5 ——
Post Audit —
Actions

80.5.1 Contracting agency acceptance or rejection of the

configuration item and the configurateion itern product
specification presented for-PCA must be furnished to the
contractor in vritinq by the responsible contract management
agency or other designated agency after completion of PCA.
80.5.2 After completion of the PCA, the contractor shall publish
and distribute copies of PCA minutes. The contracting agency
officially acknowledges completion of the
paragraph 4.2.4.
PCA as indicated in
●
80.5.3 The accomplishment o; the PCA shall be recorded on the
configuration item Development Record by the contractor.

—.

,1
 )(IL-STD-1521B
I APPENDIX I

‘ a.
\
90, PO ma 1 Qualification Review.
I
90.l”General. The objective of the FQR shal & to verify that
the actual performance of :::D;:nfiguration terns of the system as
determined throuah test with the hardware Developraen:
Specification, ~oft~are Reqbi~ements and nterface Requirements
I Specifications, and to identify the test report(s)/data which
document results of qualification tests of the configuration
items. The point of Government certification will be determined
I by the contracting agency and vill depend upon the neture of the
program, risk.aspects ’of the particular hard~are and software, and
I contractor progress in successfully verifying the requirements of
the configuration items. When feasible: the FQR shall be combined
1“ with the FCA ● t the end of configuration item/subsystam testing,
prior to PCA. If sufficient test results are not available et the
Fm to insure the configuration items will perform in their system
environment, the FOR shall be conducted (post PCA) during system
testing whenever the necessary tests have been successfully
comnleted
—_. ~—- .— to
-. enable certification of configuration items. For
non-combined Fti/FQRs, traceability, correla~ion, and completeness
of the FQR shall be maintain-d with the F~ and duplication of
● ffort avoided.

90.2 Requirements.
90.2.1 In cases where the FQR and the FCA can be accomplished in a
single combined Audit/Review, contractor and Government
‘certificacion- of the configuration items shall be accomplished
after completion of the FCA and such certification shall be
considered as accomplishment of the FQR.
90.2.2 When the agency responsible for qualification of the
configuration items at the contracting agency judges that the
system is not ready for FQR.az the time of ?C.A, the FQR will be
delayed until it is determined that sufficient information on the
system’s qualification is available. The FQR may be delayed up co
tha end of Systam testing if deemed necessary.
90.2.3 When a senarate FOR is necessarv, the contractor shal 1
notify the c~ntracti;g aqency of ‘the sufficiency of ,the
confi~uration items test results to substantiate a FQR and
coordinaca the agenda with the Deputy Director for Test and
Deployment. The FQR team will be assembled in the same manner as
that required for the FCA team. NO duplication of FCA effor:
shall occur at the FOR; however. the fallowing additior,al ef forts
must be accomplished:

90.2.3.1 A review af the FCA minutes must be per:ormed and the FQR
shall be considered as an extension of FCA . New/additional
qualification c!ata shall be audited and reviewed to insure
qualification of the configuration items against the
System/Seqment, Soft=are Requirements, and Interface Requirements
a
,. .83
 )41L-STD-1521B
APPSNDIX I

Specifics”” .

90.2.3.2 Any testing accomplished cgai~st canfigu:a:ion item

qualification during System testing shall be considered.

90.2.3.3 The contractor shall, after notification of certification

by the contracting agency enter :he date of system certification
of qualification and the identity 0: the test
reports/documentation which sets forth the results of the
associated test(s) in the confi~uration i:em Development ?iecord.

90.2.4 All other factors such as: agenda, team organization,

review procedures, data to be reviewed, etc., shall be
accomplished as delineated in the F= and General Requirements and
Procedures sections of this standard to the ● xtent necessary to
●ccomplish the FQR.
90.3 ——
Post Review Action
—
90.3.1 After th? conduct of the FOR.
-. the contractor shall c.ublish
and distribute copies of FQR minutes. The contracting agen;y will
officially acknowledge the conduct of the Review as indica?ed in
paragraph 4.2.4.

1-

84
 i

I HIL-sTD-1521B ‘“
APPENDIX I

PRE-F13 CHECK SF?E.V

I Nor.lENcQTLas
(
CONFIGURATION .lTES1NO. DATE

1. Waiver/Deviation List Pzepared —.

2. Qualification Test Procedures Submitted —.
i
3. Qualification TestinS Completed —.
I
4. Qualification Test Resuits Compiled 6
Available ——

a I
5.
6.
Facilities
~ualification
for Conducting FCA AvallaBle
Test Procedures Reviewed
—.

1. and Approved ——
I
7. Qualification Tes:ing Witnessed ——
8. Qualification Test Data end Resalts
Re.:iewed azc?A?pro-:eti ——

COFWWTS

Figure 3
Page 1 of li

es

I
 MIL-S’I’D-1S21B
MPENDIX I

SAMPLE CERTIFICATION ATTAC!IF!ENT

FUN~IONAL CONFIGURATIONN A~

FOR

CONFIGUTUTION ITEM NO. (sJ

CONTRACT

PRIME CONTRACTOR: EQUIPUENT MANUFACTURERS:

●
APPROVED BY APPROVED
[CONTRACTOR 1

DATE DATE

Figure 3
Page 2 of 11

86
 MIL-STD-1521B

●✍✎ APPENDIX I

DEF-INITIONS:

COMUENT : A note explaining, illustrating, ox Criticizing the

meaning of a writing. I:ems of tnis nature snould be expLored
by the contractor and/or the Contracting Aqency, Dut corrective
action is NOT necessary to successfully accomplish a FCA.

DEFICIEWY: Deficiencies conszst of two types: (1) conditions

of characteristics >n any nardware/software wnxcn are not in
compl~ance with specified configuration, or (’”2) inadeqtwste tor
erroneous] configuration, .i&entificatlon wnicn ‘has resulted. or
may result in configuration ~tema tnat do not fulfill xtpproved
operational requirements.

I ..
Figure 3
Page 3 of 11

87
 HIL-STD-1521E
APPENDIX I

SCOPE/PUR~

Scone:
~ Functional Configuration Audit (FCA) was conducted on the follwing
configuration it&m:
Configuration Item No. Nomenclature Part No. Serial Mo.

PURPOSE : The purpose of this FCA was to vezify that the confiquraticn
Ltem’s performance complies with the Type B Development Specification.

I
Figure 3 *
Page 4 of 11

BE
 141L-sTD-15219

● APPENDIX I

~
(per zquipmenc/CoEpu:er Sofware)

Contract: Date

Contractor:

Configuration IEem No.:

—
I CLialification Test Procedures and Resuizs. The quali~ication test/
analysls xesults have been re-:~cwed :0 ensure that tasting is adequat,
properly done. and certified. [All test procedures and interface
documents shall be reviewed to assure that the documents have been
zPProved by the Contracting Agency. All test aaca sheets shall be
reviewed to assure that the teat was witnes.qed by a representative of
the Contracting Ageacy. )
* Attached is a list of the documents reviewed.
Check One
Pracec?tiresand results zeviewed satisfy the requirements and
zre accepted. See Attachment _ for comments.
Attached is a list of deficiencies.
Signature(s) of FCA Team $lember(s)
●

Figure 3
Page 5 of 11
 MIL-STD-1521B
APPENDIX I

SPECIFICATION/TESTIXG REVIEW

Configuration Item No. Nomenclature

Specification No.
Test p=Oceti~res

Spec Ref
TP Ref Description Teat Result

!
Figure 3
●
Page 6 of 11
 .. . .. . .. . . . . . .. . .. . . .. -.’. . . . . ...... . . . . .

MIL-STD-1521B
APPENDIX I
● — . — —

— — — —

— — —

— — —

— — — —

— — — — —
Figure 3
Page 7 at U
 . .. . .. .. . . ..
I . . .. -.’-
. .. . . .. . . .

I
MIZ-STD-1521B
I APPENDIX I
1’

7LNCTIONAL CCNFIG”MTION .qLq&

~FRTIFICaTION SHEET No. 2

(For Equipment/Computer Software)

Contract: — Date
Contractor:
Configuration Item So.:

Review Deviations/iiaivers. A review of all deviations/waivers to

mli>tary Spec:f lcat~ons and standarZs that have been approved. The
purpose is to determine the extent to which the equipmentl sl[computer
Software undergoing FCA vary from applicable specifications and
standards and to form a basis for satisfactory compliance with these
specifications and standards.
In accordance with this paragraph, all applicable
have been reviewed with the following resulta:
deviations/w.sivers
●
Check One
O ;~~e~q;~pment(s) /computer software listed. on Certification
1 of this report complies with all applicable
specifications and standards. See Attachment — for comments.
O Attached is a list of discrepancies and/cr conunents.
SlgtICtture(S) of FCA Teem Member(s)
●

●Sub-Team Ch alrpe=son

Fiuure 3
Pa~e 8 of 11

I 92
l“”” . . . .

.MIL-STD-1521B

1!
● AFPENDIX I

A. Deviation/Waiver Review Team Inszrzctions. All approved

waivezs and deviations to mllltary specitlcations and standaraa
shall be reviewed and recorded. Also, record any Part of the
FCA which fails to meet specifications or standards but
— is not
an a?proved waiver/deviation.

I B. Results of Team Review. List the deviations~daivers against

the equipmen.t/computer:sofVaare being FCA’d that were reviewed.
I

Fiqure 3
Page 9 of 11

,0 .,
93
 MIL-sTD-1521B

●
APPENDIX I

— — — — — — — —

— — —

●
— — —

o
z

— — — — —
i... . . ...-.................’..
. . . . . .

I
HU-STD-1521B

●✎
.Xppzlmrx I

1
I
SAMPLE CERTIFICATION ATTACHMENT

FORM= OUALZFXCATION REVIECf

(For riqu.pme9VCompucer Software)

II Contract: Date

Contractor:
Configuration Item No.:

I
Formal Qualification Review. Qualification Test/Analysis results
have been rev~ewed to verify that the actual performance of the
~.
configuration item complies with its development or requirements
specification(s) and that sufficient test results are available to
ensure the configuration item will perform in its system environment.
I
Attached is a list of the documents reviewed.

● Check One
I
o Results reviswed=atisfy FQR requirements and the eonfiquration
item is qualified for ent~~ into the Government Invento&.
Results zeviewed are unsatisfactory/insufficient fOr FQR- FQR
El will be delaved until it is determined that sufficient
information & the configuration items Qualification is available.
Signature(s) of FCA Team Member(s)

Figure 3
Page 11 of 11

95
 nxL-sTD-1521B
APPENDIX I

SAMPLE CERTIFICATION AITACHFIENT

SAWLE PCA CHECKLIST

The following haz~ware, computer software, ~ocumentation shall be

available, and the following tasks shall be accomplished at the
PCA .

Elardware:

Computer Software:

Documentation:

(1) Approved final dzaft of ‘the configuration item

product specification. ——
(~) A list delineating both aPProved and OdtStanding
changes against the configuration item. ——
[3) Complete shortage list. ——
(4) Acceptance test procedures arid associated test
data. ——
(5) Engineering Drawing Index. ——
[6) Opezating, maintenance, an< illustrated parts
breakdown manuals. ——
[7) List of approvefl natezial :eview board actions
on waivers. ——
(8) Proposed DD Form 2S0, “Material Inspection and
Receiving Report-. ——
(9) Approved nomenclature and nameplate. ——
(10) Manuscript copy of all CSCI manuals. ——
(11) Computer Software Version Description Document. ——
(12) Curzent set of listings and updated desiqn
descriptions or other means of design portrayal
for each CSC1. —.
(13) FCA minutes for each configuration item. ——

Figure 4

●
Page 1 of 20

96
 141L-5T9-151LB
APP2NDIx 1

I
SAMPLE C==T?FICATION ATTACHMENT

SAMPLE PCA CHECKLIST [CONTINUED)

Yes
—. No

Tasks :
[1) Define Product B~seline. ——
(2) Specification Review and validation. .—
(3) Drawing review. .—
(4) 3eview acceptance test proced’ares and :esults. ——
(5) Review shortages and unincorporated ~csign
changes. .—
(6) Review deviations/waivers. .— -
(7) Examine proposed DD 250. ——
(8) Review contractor’s Engineering Release and
Change Control System. ——
(9) Review system allocation d~cument. ——

● [10) Review Software User’s klanuals, Software

programmer’s -nuals, Computer sYstem Diagnostic
Manual, Computer System Operator’s Manual, and
Firmware Support Manual. ——
(11) Review CSCIS for the following:
(a) Top-1evel and lower-level Computer Software
Component design descriptions or alternative
design portrayals. ——
(b) Top-level and lower-level Compute= Software
Component interface requirements. _—
(c) Data base characteristics, storage alloca-
tion charts and timing acd sequencing
characteristics. —— .
(12] Review packaging plan and requirements. —.
(131 Review status of Rights in Data. ——

Figure 4
Page la of 20

97
i.
,.
 141L-sTD-1521B
APPENDIX I

SAMPLE CERTIFICATION ATTACIME31T

WY sIcu c0NFIf3JRAT10t4AUDm rwi)

mR
CONFIGU’NiTION ITEM NO.(S)
CONTRA~ NO.

I PRIMS CONTRACTOR: EQUIPMENT MANUFACTURERS:

APPROVED BY (DESIGNEE) APPROVED BY (DESIGNEE)

●
CONTRA~OR CONTIUiCTING AGENCf

DATE DATE

Figure 4
Page Z of 20

98
 HIL-STD-152A8
APPENDIX x
●✎
DEFINITION OF TERMS

com’fENT - A note-explaining, illustrating, or criticizing the

~ of a writing. Items of this nature should be ●xplored
by the contractor and/or the Contracting Agency, but corrective
action is “NOT necessary .to.successfully .accomplish a PCA.
DISCREPAN&Y - i+not explaining, .illustrliting, or criticizing ths
difference between writings. A note shmfing the variance between
what exists and what is acceptable. Items of this nature shall
be rectified by the contractor prior to successful accomplishments
of a PCA.

1.

0 ,,

Figure 4
Page 3 of 20

! 0, 99
 MIL-STD-152LR
APPENDIX I

SCOPE/PCRPCX5E

A Physical Confis=xa:icn Audit (PCA) was condac:ed on the

following ead itens of eq.~ipme>tlcomputer softvare:

CONFIGURATION ITEN NO>~;JCLATr~E PART NUMBER SERIAL ?40. NSN

—

The pu-rpose of the PCA vaa to ensure accaracy of zhe identifying

documentation and to establish a product baseline.
The establishment of a pro~uct baseline for equipnent/computer
software is not to be construed as meeting Contracting Agency
requirements for delivery 5y the contractor of an operational
system meeting approved acceptance criteria.

Figure 4
Page 4 of 20
●
100
1“
I
t41L-STO-1521~
APPENDIX I
o
I 1 “’

CERTIFICATION SHEET NO. 1

(For Equipment/Computer Software)

Contract: Date
Contractor:

Product baseline. The following documents of the issue and date

shown comprLse the product baseline for the listed equipment(s)/
computer software:

EQPT./COMP.
ASSR4SLY TOP SOFTWARE CONFIGURATION
SPEC NO. DRAWING NO. ISSUE NOMENC’f.ATURE ITEM NC.
● )
\
signature(s) of PCA Team Member(s)

●☛

● “Team c halrperson
“Sub-Team Chairperson

Figure 4
.
Page S of 20
.. .

,,.,
.“.
 IYIL-S?I’D-1521B
APPENDIX I

~EYSICU CONFIGURATION AUDIT

~ERTIFICATTON SHE9 NO. 2

(For Equipment/Computer Software)

Conzzact: Date
Contractor:

spa cification Review and Validation. Specifications have been

rev~ewed and validated to assure that they adequately define the
configuration item and the necessary testing, mobility/transpor-
tability, and packaging requirements.
Check One
❑ The Type C Specifications are complete and adequately
define the configuration item. They shall, taerefore,
constitute the product baseline.
couments.
See Attachment _ for ●
I m
u
The Type C Specifications are unacceptable. Attached is a
list of Cisczepancies.

.signatu:e(s) of PCA Team Ne.mber(s)

*Sub-Team Chairperson

Figure 4
Page 6 of 20
●
iwi
 UIL-STD-1521B
-APPENDIX I
●
A. Soecificacion Review and Validation Instructions. The
detailed speclflcatlons llsted in paragraph B. Below shall
be reviewed for compliance with the applicable requirements.
I1 Each Specification shall serve as the basic document for
‘anfig~ration control of the subject configuration items.
The information contained within the specifications shall be
audited at the PCA.
I . .. . . ..Results:
.
B. Revlewana Vallaatlon
1. Specifications Reviewed ’and Validated
.

EQP7 ./COHP .
somwAR.E CONFIGURATION
SPEC NO. PART No. DATE NOMENC2ATL~ ITEM NO.

2. Specifications Reviewed and Disapproved:

(Provide attachment foz causes. )

Figure 4
Page 7 of 20

103
 HXL-STD-1521B
APPENDIX I

P!I’YSICRLCOSFIGL?W?ION xDD~

CERTIFICATION 5HSET F?O. 2

(Equip.ZencJ

Contzacr: Date

Contractor:

Drawi nq Review. Drawings have been compared with the equipment to

ensure that the latest drawing change letter has been incorporated
into the eqaipment, that part numbers sgree with the drawings. and
that the drawings a:e complete and accurately describe the equipment.
Attachment _ is x list of the drawings reviewed.
Check One ●
•1 The drawings axe complete and accurately describe the equipment.
See attachment — for comments.
D Attachment _ s a list of discrepancies.

Signature(s) Of pm Team Members(s)

●

“Su5-Tea.m Chairperson

●
Figure 4
Pa9e 8 of 20

104
 MIL-STD-1521 B
APPENDIX I

A. Drawinq Review Results. The following drawings were reviewed

by tlie PCA drawing reviewing sub-teams:

DOCUMENT NUMBER DOCUMENT TITLE

Figure 4
Page 9 of 20

105
 MIL-sTD-1521B
APPENDIX I

/WYSICAI. CONFIGURATION AUDIT

CERTIFICATION SHEE?T NO. 4

iEquipmenc )

Contract: Date

Contractor:

Acceptance Test Procedures and Results. The acceptance test results

have been reviewed to ensure that testing is adequate, properly done,
and certifie3.
Attachment _ is a list of the documents reviewed.
Check One
O Prcacefiuresand results reviewed satisfy the requirekts
and are accepted. See Attachment _ for comments.
O Attachment _ is a list of discrepancies.
Signature(s) of PCA Team Member(s)

●Sub -Team Chairperson

Figure 4
Page 10 of 20

106
 NIL-SID-1521B
APPENDIX I

Wfsx CA> CONFIGURATION A~m

CERTIFICATION SNEET NO. 5

(For ~uipmcnt/Computer Software)

Contract Date
Contractor:
.

Review of Shortawes and Unincorporated Desiqn Changes. The shortages

and unincorporated design changes listed on the proposed DD Form 250,
‘Wterial Inspection and Receiving Report-. and ocher records have
been reviewed.
Check One
There are no shortages or unincorporated design changes.
● ) Attachment is a list of shortages and/or unincorporated
design chan~s. and the recommended corrective action required.
Signature(s) of PCA Team Member(s)
●

*5ua-Tearn Cha~zperson

I
Figure 4
Page 11 of 20

1. ● 107
 I

I HIL-STD-1521B
II APPENDIX 1

,i
A. Review of Shortages and Unincorporated Design Changes.
All shortsges and unzcorporated design changes llsted
on the proposed DD Form 250, ‘Material Inspection and
Receiving Report”, shall be zeviewed by the Contracting
Agency or their designated representatives for a deter-
mination of what changes should be accomplished in the
field and what changes should be accomplished at the
contractor’s facility. The Contracting Agency shall
also determine if the reported shortages and unincor-
porated changes are complete.

B. Results. List the shortages and unincorporated design

changes that were reviewed in compliance with requirements.

?iqure 4
Page 12 of m

I 108
 I

I
m.
L-sTD-1521B
I APPENDIX I
~@’-
I
!
1 fJ?#T1~Ic&+rfjNs=
1 NO. 6
I (Fox Equ~pment/Computer Software)

I
Contract: Date
II
Contractor:

1.
Review Deviations/Waivers. A review o: all deviations/waivers to
mal~tary speclflcataons and standards that have been approved.
The purpose is to detarmine the extent to which the equipment(s)/
computer software undergoing PCA vaq from applicable specifica-
tions and standards and to form a basis for satisfactory compliance
with these specifications and standards.
In accordance with this paragraph, all applicable aviations/
waivers have been reviewed with the follwing results:

●,
Check One.
The ●quipment(s)/computer Software listed on Certification
Sheet No. 1 of this report complies with all applicable
specifications and stan~ards. See Attachment _ for comments.
Attachment _ is a list of discrepancies and/or comments.

Signa:ure(s ).of PCA Team Member(a)

*Sub-Team Chairperson

!0 Flqure 4
Page 13 of 20
109
 lJIIL-sTD-ls21B
APPENDIX I

A. Deviation/Waiver Review Team Lnstractlon. All approved waivers

and deviations to m~litary Speelxxcatxons and srandards snail be
~ reviewed and recorrled. Also, recora any pare of the PCA which
fails to meet specification or standards —
but is not an approved
waiver/deviation.
B. Results of Team Review.
I List the deviations/waivers against the
equ2pment/computer software being PCA’S that were reviewed. ,

Figure 4
Page 14 of 20

110
 nIL-sTD-ls22a
APPENDIX S

PXYSICAi. CONFIGLPLATION .aL~T?

I (For Equipment/Compuuer :oftware)

Contract: Date
Contzaetor;

Examination of the Proposed DD 250. The DD Form 250 has been examined
to ensure tha: ~t adequately ?eflnes the eqaipment/computer software
and that unaccomplished casks are inlc~ded as deficiencies.
Check One
D The .DD Form 250 adequately defines the equipment/computer
software and all unaccomplishe~ tasks are included as
deficiencies.
‘m
n Artac.hment _ is a list of discrepancies andlor comments.
Signature(s) of PCA Team Member(a)

I*
I

‘Sub-Team chairperson

Figure 4
Page 15 of 20

111

.
 MIL-STO-15219
APPENDIX 1

A. Examination of the Proposed DD Form 250. The FroPosed DD Form

and an accurate definition
of the equipment/computer software. Unaccomplished tasks,
shortages, and certain specified discrepancies uncovered at the
PCA shall be inclzded in the DD Form 250. If the equipment/
computer software is to 5e shipped from the plant, the Program
Office representative will recommend to the CAO that the DD Form
250 be executed i-naccordance with the terms of the contract.
I
a. Results. Include a statement that the proposed.DD Form 250 was
‘d and was recommended.

Figure 4
Page 16 of 20

112
●
 I
rA*A.-a 4”-..&*-
APPENDIX I

~=SICAL CO??FIGURAT ZON AL~IT

~RTIFI~~TON c-
~
(For Equipment/Computer Software)

Contract: Date

Cantractoz:

Review of Contractor’s Enaineerinc Release acd C5anae Control system.

The cantraczor’s engineering release system and change control proce-
dures have been reviewed to ensure that they are adequate to properly
control the ?rocessing and formal release cf engineering changes.

I Check One
The contractor’s engineering release system and change control
I procedures are adequate for the processing and formal release
o of engineering changes. See Attachment — for comments.
)
Attachment _ is a list of deficiencies.
SigmSture(S) of PCA Team Metier(s)
I ●

I
I

I “.Sub-Team Chairperson

Figure 4
Page 17 of 20

113
 MIL-STD-1521 B
APPENDIX I

r~ 3 NO. 9
(For EqulpmentjCompu~er Software)

Contract: Date:
Contractor:

System Allocation Document Review. The following System Allocation

book form drawings have been reviewed and validated to ensure that
they adequately identify, and are compatible with, the shipping
instructions .
I Check One
O The System Allocation Documenc is complete and adequately o
defines the equipment/computer software scheduled for each
location.
O The System A:locacion Document is unacceptable. Attached is
a list of discrepancies.

O This task is notrcqaired by contract.

Signature of PCA Team Member(s)
●

I
I
.

●Sub-Team Chaizperscn

Figure 4
Page 18 of 20
●
114
.
 MIL-S7’D-1S21B
APPENDIX I

I 1. The System Allocation Documents, both Part I and Part Ii,

applicable to ,the contract shall be reviewed to determine their
accuxa~ and insure chat they adequately desczibe the equipment/
compute; software.

2. The followiag information shall be recorded:

Part I.
a. System employment and configuration.
b. Specification reference.
e. Location.
d. Mission Eauiument.
Confiquraiio; Item Number
Short Title
Part Number
Serial Number
e. Installed equipment/computer software.
Configuration Item Number
Short Title

●
Part Number
Serial Number
Drawing Title and Number
1 Number of sheets
Issue Number.

I Part II.
Location.
Specification Numbe:
Equipment/computer software nomenclature.
Configuration Item Quantity.
Fkssembly Drawing Number
3. Insure that the System Allocation Documents are compatible .wi
I tke priorities and shipping inatrucciona.
a
-. ~.)~$--~ Alloc=ti~n Do-e~t Revi5w Res.:!ts. The fallowing Sjrs:ez!
AIL=cazion Dorxtimen:swere reviewed by the PCA ~eviewing Sub-Team:

DOCUMENT NUMBER DOCUMENT TITLE

I
Figure 4
!0 Page 19 of 20
.

11s
 M:L-STD-1S21B
APPENDIX I

mYSICU CCNTIG-URATION AUDIT

~TI?ICATION SliZET NO. 10
(Equipment)

Contract: Date

C3nzraetor:

1. Review of Louis’tics Suormrt Plan for Pre-operational Suooort.

The Logistics Support Plan ZOr Pre-operational Support has been
reviewed to ensure that it is adequate to support the acquisition
phase and is compatible with the operational phase maintenance concept
and support requirements.
Check One

c1 The contractor’s Logistic Plan for pre-operational SLIppCrt

will fulfill the acquisition phase req~izements and is com-
patible with operational phase needs.

c1 Attachment is a list of deficiencies.

●
—
Review of Lonq Lead Time Items and Provisione~ Items Processed to
P&. Long Lead Time items released anti items provisioned, prior to
= have been reviewed to ensure that obsolete items resulting from
pre-PCA design chanyes are ?urged from the system. “Where basic items
may be upgraded “by rework or modification these actions have been
verifie~ as accomplished or in process based upon design change notice.

Check One
❑ Long lead time itezs aad provisioned items processed, grior to PCA,
aze all of current configuration at time of PCA or are in work.
a Attachment _ is a list of deficiencies.
Signature(s) of PCA Team ?Iember(s)
●

●Sub-Team Chairperson

Figure 4
Page 20 of 20 ●
. 116
 UIL-STD-1521B
APPENDIX J

100. ApplicationtGuide for Tailoring 14:L-STD-1521

100.1 Scooe

This appendix sets forih guidance for the cost effective

application of the requirements of this standard when this
standard is contractually invoked during the acquisition process.
This appendix serves as guidance for the activity responsible for
the preparation of contract requirements and does not form ‘a parr
of the contract.
100.2 Purpose

The guidelines contained herein implement the Department af

Defense Directive 4120.21, Specification and Standards
Application. which requires all DOD components to ●pply
selectively and tailor military specifications and standards prior
to their contractual imposition and:
a. Eliminate inapplicable and unnecessary requirements.
b. Provide for adding/modifying necessary technical review and
audit factors not included in MIL-STD-1521.
c. Eliminate redundancy and inconsistency with other con:ract
specifications end standards.
100.3 Objective
The objective of this guide is to establish the applications and
limitations of tailoring MIL-STD-1521. MIL-STD-1521 is nut .s
stand-alone document. It is dependent upon the work effort
specified in the contractual requirements (e.g., SOW, etc.) The
taiioring of specifications shoulcf cake place in all phases of
military procurement, but is ●specially applicable to the initial
stages of solicitation package preparation and contract
negotiation. DeDendino uno n the tme ‘of ● rid-item(s) under
procurement, the r;vievs-and” audits outl-ined by MIL-sT9-1521 may
or may not be required for all programs.
100.4 Considerations —for ?ailorinq
100.4.1 .Relationshiu .—
to the Statement of .—
— work
The Program Manager must keep in mind that technics: ,reviews
proviUe visibility into the contractor’s implementation of the
work effort required under the terms of the SOW and the contract
to assure timely and effective a:tention to the technical
inrer?retation of contract ,requiremen:s. The key to tailoring
MIL-STD-1521 is to match the MIL-STD-1521 requirements against the
details of the applicable SOW/Contractual task requirements. It
-*i11 become immediately obvious that MIL-STD-1522 may contain
117
.
 MIL-STD-1521B
APPENDIX J

technical review factors that are not applicable to the contract —

under consideration. (For example, if a contract t!oes not include
computer softvaze, all references %0 the review of Computer
Software materials in MIL-sTD-1521 .,i~l not apply. ) When
MIL-STD-1521 is used, then a cask containing the applicable
requirements will be specified in the SOW. Review factors not set
forth in MIL-STD-1521 but considered necessary because of the
nature of the particular Drogram shoulr! be adced in the S3W. By
carefully going through th-is evaluative process the technical
review and audit requirements will become p?agram specific rather
than an all purpose document to be continually negotiated during
contract performance.
100.4.2 Elimination ~ iledflndanc~and
— Ambiquitv
While MIL-STD-1521 is the broad program document for :echnical
rev iews and audits, other standards in existence also require
technical reviews or audits. For example , MIL-STDS for
reliability, maintainability, system engineering and others can
require reviews and/or audits. Review of these aspects of the
design would also be required under MIL-STD-1521; therefore, if
such standards are contractually stipulated together with
MIL-STD-1521, the SOw should include a provision to show how and
whether the technical review requirements of these other standards

●
can be combined with technical reviews/audits in MIL-STD-1521.
Combining revievs does not nullify other MIL-STD(S), ‘Plans”, etc,
which contain requirements for reviews/audits. The contract
should require the minimal integrated, comprehensive technical
design review effort that will provide the desired visibility and
assurance of contract compliance.
100.4.3 Contractor Participation in Tailorina
‘When requiring a particular revie.~ or a“udic, it is important that
the topics to be reviewed are aligned zo the program requirements.
Therefore, the offeror should be given an opportunity to recommend
changes and identify topics/items he considers appropriate. The
program office should request, in the instructions for proposal
preparation, that the offeror recommend the MIL-sTD-1521
topics/items and their related details to be covered at the
various reviews or audits required by tha SOW. This will allow
the offeror to tailor the topicsiite!ns ard details ~y a~ditions
and deletions for the particular review/audit. In addition,. ii
must be recognized that effecive tailoring requires several poir,ts
of review. The requir~ment, hoveve:, for the rev!ev/audit must be
.
‘inalized prior :0 concract award.
100..4.4 Comalexitv

a. System/Segment/subsys:em/conf iguracion item complexity and

type of program is central in determining both the Reed for
and the number of such reviews.
218
When developin~ a small
●
i

-MIL-STD-15219
APPENDIX J

!’ non-complex sysrem some reviews nay not be required, or, if

re~zizet, =ay be -.”. ..
‘:-:”e~ i> S=z?e. Tke tailoring procedures
disczssed earlier should result either in the ●xclusion of
M;L-STD-1521 0: in a tailored MIL-STD-1521 that reflects a
limited scope technical review effort. Conversely, in a
very complex development the review process will increase in
levels and numbers of reviews.

b. In :addi:ion to the -above, the tiegree of application is

dependent upon the configuration item state of development
(example, ne,d &esign vs. commercially available) or the
deqree of any tnodifica:ions. if involved. For example: a
newly developed item may require the majority of the rev iew
topics/items and audits, while a commercially available
I configuration item with the appropriate documentation. i.e..,
verified test results, specifications, drawings, ●tc. may
require reviews or audits limited to its application to the
program and its interfaces. In the case of modified designs
one must consider the degree and effect of the
modifications. Reviews and ● udits may be limited to the
modifications and their interfaces.
100.5 Scheduling ~ Technical Reviews and —
—— Audit”s
● The schedule
important. [f
for Tech~ical
they ● re
Reviews
conducced too
and Audits
●erly, the
is ●xtremely .
item for review
will not be adequately defined. Conversely, if the review is too
late, the program commitments could have been made erroneously,
and correction vill be both difficult and costly. For planning
purposes, a good method for scheduling technical reviews is to
relate them to the documental ion requirements. For example,
schedule a PDR after the hard,~are Development Specification or
Software Top Level Design Document and Software Test Plan are
available, since the essence of the PDR is to assess the
contractor’s approach to meeting these requirements of these
documents. Scheduling of audits are dependent not only on
documentation availability but also on hard-~are/software
availability, and the completion of the acceptance qualification
tests. Table 1 contains a list of the primary documentation
associated wi:h ● ach review 07 aud it aed the ●stimated time
phaskng:
TABLE 1
SCHEDULING TZCXSICAL ?.E’JIZUSAND A2!31TS
Review Time
.— Phase Primary Documentation
SRE Usually accomplished in :he various anal:~sis and
“COnCept Exploration phase. trade study reports
However, may be used in used to develop zhe
other phases when the system/segmen:
 ..

MI L-STD-1521B
APPENDIX J

Review Time Phase ?rimarv Documen:atio7.

Concept Exploration phase re~irements for the

is not accomplished. specification.

SDR usually in the De!nonstration SystemlSetjme9t

and Validation phase. Specification,
preliminary Gpezational
Concept Document,
preliminary Software
Requirements and
Interface Rcquiremencs
s.pccificat~ons, analyses,
trade studies, Drawings
Level I DOD-D-1OOO.

SSR USUAllY early in Full Software Requirements

Scale Development Specification. Interface
Requiremen~s
Specifications,
Operational Concept
Document.

PDR usually accomplished in ~he

Demonstration and Validation
and/or Full Scale
Development, Type B
Performance
Specification,
●
Development Phase Drawings Level I
DOD-D-1OOO, Software
Top Level !2esign
Document, Software Test
Plan, preliminary
Computer Resources
Integrated Support
Document, preliminary
Computer System
Operator’s Manual,
preliminary Software
User’s manual,
preliminary Computer
System Diagnostic Manual.

CDR Usually accomplished in the Draft Product, Type C

Full Scale Development Specification, and
phase referenced documentation,
Drawings Level 1 or 11
DOD-D-1OOO, Software
Detailed Design Document,
Interface Design
Document(s), Data Base

●
Design Doc’.Jment(s),

:20
I
I
 m,- U2L-STD-1521B
APPs.VDIX J

1
Review Time
.— Phase Primarv Documentation

Software Test
Description, Computer
Resources Integrated
Support Doc”ument,
Seft.iare Programmer’s
I Manual, Firmware
;Support Manual, Informal
Test Descriptions/Test
Procedures, Software
Development File(s).
TRR Usually accomplished in Software Test Procedure,
the Full Scale Development Informal software test
phdSe results (of development
I tests) .
FCA .Usually accomplished at Test plans, test
end of Full Scale descriptions, test
Development procedures, Software
Test Reports, Computer
System Operator’s Manual,
Software User’s Manual,
Computer System
Diagnostic Manual.
PCA Usually accomplished early Final Part II
in the initial production Specification/Type C
when the developing Product Specifications
contractor is preselected and referenced documents
as the production and dravangs. Drawings
contractor. However, may Level 11 or 111
be accomplished at the end DDD-D-1OOO. Software
of Full Scale Development Product Specification,
when the developing Version Description
contractor is not Document.
preselected as the
?roduction contractor.
And the PCA is repeated
with each .saktseqaen:
contractor or break
in produc:ior..
Al:hough the time frame for reviews and audits is suggested above,
they may vary depending on the partic~lar proqram. The schedule
for each review or aadit may be requesteti from the cfferor as part
of his proposal, or as part of the system engineering management
plan (which can be part of the proposal).

●
121/122
 MIL-STD-:5219
APPENDIX K

~
●
“-
110. Production Readiness Review
—. (PRR)

110.1 For specific guidance, see AFSCR 64-2, Production Readiness

Rev iew.

● )

123
 MIL-STD-1521B

Custodian: Preparing Activity:

Air Force - 13 Air Force - 13

Review Activity: (Project CMAN-O-006)

Air Force - l13,~l,Bo,B5

124
9
I

I
●✍

111111
Orsslak ● swm
m.uv. ●om ●mws7m use uca BWdfNESS mm”8?REPLY
no.
●
MAIL
,,*”---- -C
MS?.oa
-al..c AIOV-99.***=*O===
● ● ●

COMMAMYJE?,
!I[ACOUAR7ER5
Electronic
SYSTEMS 3; ’J IS ION [HO ES3)
ATN: ALET, SYSTEt4S E!iGI !!EE?! !G C! ’?! SICt!
H.W.CCM AFL3. ‘a!!- ‘7711
..,-.
-

●
✎ .~
 ... . . .
~
I

=N4DARDIZATIOR DOCUMENT lMPR0VEMEN7 PROPOSAL

rk Im!nnuml - R&?w w)
I
a-v auaa I* 00---7 Tltm
I
?**S O* Onti”, zArto” ,m- -,

c1 tma ●

I ❑
o m“an,J-Oa,:

I .

. n—m.-. ,. ——:

.
● nlMAnr,a