Test and Evaluation Guideline For Liquid Rocket Engines

The following notice applies to any unclassified (including originally classified and now
declassified) technical reports released to "qualified U.S. contractors" under the provisions of
DoD Directive 5230.25, Withholding of Unclassified Technical Data From Public Disclosure.
NOTICE TO ACCOMPANY THE DISSEMINATION OF

EXPORT-CONTROLLED TECHNICAL DATA
1. Export of information contained herein, which includes, in some circumstances, release to

foreign nationals within the United States, without first obtaining approval or license from the
Department of State for items controlled by the International Traffic in Arms Regulations
(ITAR), or the Department of Commerce for items controlled by the Export Administration
Regulations (EAR), may constitute a violation of law.
2. Under 22 U.S.C.2778 the penalty for unlawful export of items or information controlled
under the ITAR is up to ten years imprisonment, or a fine of $1,000,000, or both. Under 50
U.S.C., Appendix 2410, the penalty for unlawful export of items or information controlled under
the EAR is a fine of up to $1,000,000, or five times the value of the exports, whichever is
greater; or for an individual, imprisonment of up to 10 years, or a fine of up to $250,000, or both.
3. In accordance with your certification that establishes you as a “qualified U.S. contractor,”
unauthorized dissemination of this information is prohibited and may result in disqualification as
a qualified U.S. contractor, and may be considered in determining your eligibility for future
contracts with the Department of Defense.
4. The U.S. Government assumes no liability for direct patent infringement, or contributory
patent infringement or misuse of technical data.
5. The U.S. Government does not warrant the adequacy, accuracy, currency, or completeness
of the technical data.
6. The U.S. Government assumes no liability for loss, damage, or injury resulting from
manufacture or use for any purpose of any product, article, system, or material involving reliance
upon any or all technical data furnished in response to the request for technical data.
7. If the technical data furnished by the Government will be used for commercial
manufacturing of other profit potential, a license for such use may be necessary. Any payments
made in support of the request for data do not include or involve any license rights.
8. A copy of this notice shall be provided with any partial or complete reproduction of these
data that are provided to qualified U.S. contractors.
DESTRUCTION NOTICE
For classified documents, follow the procedure in DoD 5220.22-M, National Industrial Security
Program, Operating Manual, Chapter 5, Section 7, or DoD 5200.1-R Information Security
Program Regulation, Chapter 6, Section 7. For unclassified, limited documents, destroy by any
method that will prevent disclosure of contents or reconstruction of the document.
Form Approved
REPORT DOCUMENTATION PAGE OMB No. 0704-0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing this collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis
Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a
collection of information if it does not display a currently valid OMB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.
1. REPORT DATE (DD-MM-YYYY) 2. REPORT TYPE 3. DATES COVERED (From - To)
11-10-2011 Conference Paper
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
DRAFT: Test and Evaluation Guideline for Liquid Rocket Engines 5b. GRANT NUMBER
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER

JANNAF Liquid Propulsion Subcommittee Test Practices and Standards Panel
5f. WORK UNIT NUMBER

50260651
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
REPORT NUMBER
Air Force Research Laboratory (AFMC)
AFRL/RZSE
4 Draco Drive
Edwards Air Force Base CA 93524-7160
9. SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S

ACRONYM(S)
Air Force Research Laboratory (AFMC)

AFRL/RZS 11. SPONSOR/MONITOR’S
5 Pollux Drive NUMBER(S)
Edwards AFB CA 93524-7048 AFRL-RZ-ED-TP-2011-401

12. DISTRIBUTION / AVAILABILITY STATEMENT
Distribution A: Approved for public release; distribution unlimited (PA#11877).
13. SUPPLEMENTARY NOTES

To be presented at the JANNAF 2011 Joint Subcommittee Meeting, Huntsville, AL, 5-9 Dec 2011. EXPORT CONTROLLED.
14. ABSTRACT
This current draft guideline is a work in progress written by members of the Joint Army Navy NASA Air Force (JANNAF) Liquid
Propulsion Subcommittee (LPS) Test Practices and Standards Panel (TPSP). The Acknowledgements page lists the current and
past members of the TPSP, as well as other contributors. The content herein does not represent the official or approved position of
any of the organizations involved in its preparation. It is intended as an initial starting point for further discussion within the
propulsion community regarding the appropriate specific test requirements for liquid rocket engines (LREs). This draft is intended
for submittal to the JANNAF TPSP and the greater United States propulsion community for detailed review, comment, and revision.
Further review, comment, and revision over multiple review periods are anticipated. Furthermore, the focus and content of this
current draft edition relates primarily to engine testing and integrated vehicle propulsion system level testing. Although testing at the
component level is not discussed in detail, this is recognized as also being important and valuable, and therefore it is anticipated
that detailed treatment of component testing will be incorporated into subsequent editions of the guideline. Similarly, although
reaction control systems (RCS) and satellite propulsion are not addressed in the current draft edition, it is anticipated that these
areas will also be incorporated into subsequent editions of the guideline.
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION 18. NUMBER 19a. NAME OF RESPONSIBLE
OF ABSTRACT OF PAGES PERSON
Dr. Gregory A. Ruderman
a. REPORT b. ABSTRACT c. THIS PAGE 19b. TELEPHONE NUMBER
SAR (include area code)
93
Unclassified Unclassified Unclassified N/A
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std. 239.18
JANNAF LRE T&E Guideline_v6.0

Test and Evaluation Guideline for Liquid Rocket Engines
Joint Army Navy NASA Air Force (JANNAF)

Liquid Propulsion Subcommittee (LPS)
Test Practices and Standards Panel (TPSP)
14 December 2010
Distribution Restrictions
Distribution Statement A: Approved for Public Release. PA Clearance Number 11877
Revision History
Document No.
Joint Army Navy NASA Air Force (JANNAF) Liquid Propulsion TBD
Subcommittee (LPS) Test Practices and Standards panel (TPSP)
Issue Date
TBD
Title As of: Superseding: Page:

Test and Evaluation Guideline for Liquid Rocket TBD New 2
Engines
NOTE:
PAGE STATUS AUTHORITY
Initial Issue TBD
2
****************************************************************************************************
Foreword (with Status and Disclaimer)
This current draft guideline is a work in progress written by members of the Joint Army Navy NASA
Air Force (JANNAF) Liquid Propulsion Subcommittee (LPS) Test Practices and Standards Panel
(TPSP). The Acknowledgements page lists the current and past members of the TPSP, as well as
other contributors. The content herein does not represent the official or approved position of any of
the organizations involved in its preparation. It is intended as an initial starting point for further
discussion within the propulsion community regarding the appropriate specific test requirements for
liquid rocket engines (LREs). This draft is intended for submittal to the JANNAF TPSP and the greater
United States propulsion community for detailed review, comment, and revision. Further review,
comment, and revision over multiple review periods are anticipated.
Furthermore, the focus and content of this current draft edition relates primarily to engine testing and
integrated vehicle propulsion system level testing. Although testing at the component level is not
discussed in detail, this is recognized as also being important and valuable, and therefore it is
anticipated that detailed treatment of component testing will be incorporated into subsequent editions
of the guideline. Similarly, although reaction control systems (RCS) and satellite propulsion are not
addressed in the current draft edition, it is anticipated that these areas will also be incorporated into
subsequent editions of the guideline.
****************************************************************************************************
3
Acknowledgements
Contributors
This draft guideline is the result of contributions received from many individuals representing different
organizations associated with the design, test, and operation of LREs. The names and organizations
of principal contributors and JANNAF LPS TPSP panel members, as well as those acting as the focal
point of contact for their organizations, are listed below:
Current JANNAF LPS Test Practice and Standards Panel Members:
Dave Krismer Aerojet, Redmond
Grant Hart Aerojet, Sacramento
Ian Murray Blue Origin
Robert Drake AFRL/PRSF Edwards AFB
Gabe Mendez AFRL/PRSF Edwards AFB
Sean Kenny AFRL/PRSF Edwards AFB
Bill Munsch Pratt & Whitney Rocketdyne (CA)
Steve Hobart Blue Origin
Steve Fisher Pratt & Whitney Rocketdyne (CA) (retired)
Bob Grabowski Pratt & Whitney Rocketdyne (FL)
Dan Guisinger Pratt & Whitney Rocketdyne (FL)
Matt Thomas CFD Research Corporation
Keith Coste Jet Propulsion Laboratory
Tom Vasek NASA Glenn Research Center
Doug Parkinson NASA Marshall Space Flight Center
Kendall Brown (chair) NASA Marshall Space Flight Center
Michael Mims NASA Marshall Space Flight Center Test
Shamim Rahman NASA Stennis Space Center
Harry Ryan NASA Stennis Space Center
Robert M. Cort NASA White Sands Test Facility
Holger Fischer NASA White Sands Test Facility
Robert R. Kowalski NASA White Sands Test Facility
Jacky Calvignac Northrop Grumman, NGST
Darrick Schneider Northrop Grumman, NGST
Ron Teeter Orbital Technologies Corporation
Jeff Emdee The Aerospace Corporation
Dewey George The Aerospace Corporation
Fred Peinemann The Aerospace Corporation
Wayne Van Lerberghe The Aerospace Corporation
Tom Turner U.S. Army Aviation and Missile Command
Drexel Smith Wyle Laboratories, Inc.
Former JANNAF LPS Test Practice and Standards Panel Members:

Ed Fox Aerojet, Sacramento
Jared Sass NASA Kennedy Space Center (formerly SSC)
Melvianna Cady Pratt & Whitney Rocketdyne (FL) (retired)
Other significant contributors from The Aerospace Corporation:

Ken Behring Liquid Propulsion
Ming Chen Launch Structures
Mark Dunbar Electrical & Electronic Systems
Dan Ehrlich Liquid Propulsion
Grant Henson Launch Structures
4
Jason Malburg Liquid Propulsion

Jeff Michlitsch Liquid Propulsion
Shel Rubin Structural Dynamics (pogo)
Patrick Yee Aerothermal
Industry/Government Reviewers:
Darren Hamer Lockheed Martin
Jeff Schnackel Lockheed Martin
Jan Monk NASA MSFC
Dick Weiss USAF SMC/LRD
5
Table of Contents
1 SCOPE ............................................................................................................................ 11
1.1 PURPOSE ....................................................................................................................... 11
1.2 APPLICATION ................................................................................................................. 11
1.3 MOTIVATION ................................................................................................................. 11
2 REFERENCE DOCUMENTS ............................................................................................... 12
2.1 GOVERNMENT APPLICABLE DOCUMENTS .................................................................... 12
2.2 NON‐GOVERNMENT APPLICABLE DOCUMENTS ........................................................... 12
2.3 GUIDANCE/REFERENCE DOCUMENTS .......................................................................... 13
2.4 ORDER OF PRECEDENCE ............................................................................................... 14
3 GENERAL TEST GUIDELINES ............................................................................................ 15
3.1 GENERAL TEST PHILOSOPHY ......................................................................................... 15
3.2 TAILORING OF GUIDELINES ........................................................................................... 15
3.3 ENGINE SAMPLES .......................................................................................................... 16
3.3.1 Number of Engine Samples ....................................................................................... 16
3.3.2 Engine Qualification Samples ................................................................................... 17
3.3.3 Integrated Propulsion System Samples ..................................................................... 17
3.4 NUMBER OF TOTAL TESTS ............................................................................................ 17
3.4.1 Statistically Relevant‐Based ...................................................................................... 19
3.4.2 Functional Objectives‐Based ..................................................................................... 20
3.4.3 Testing to Failure ...................................................................................................... 21
3.4.4 Modeling and Simulation .......................................................................................... 21
4 DEVELOPMENT AND QUALIFICATION TESTS ................................................................... 23
4.1 TEST TYPES AND RECOMMENDATIONS ........................................................................ 23
4.1.1 Component Tests....................................................................................................... 23
4.1.2 Engine ........................................................................................................................ 24
4.2 PERFORMANCE ............................................................................................................. 26
4.2.1 Steady State Performance Characterization ............................................................. 26
4.2.2 Repeatability ............................................................................................................. 26
4.2.3 Run‐Time Trends ....................................................................................................... 26
4.2.4 Steady State Analytical Models ................................................................................ 27
4.2.5 MR Excursion Tests ................................................................................................... 27
4.2.6 Thrust/MR Margin Demonstration ........................................................................... 27
4.2.7 Ignition System .......................................................................................................... 28
4.2.8 Turbomachinery Performance .................................................................................. 28
4.2.9 Combustion Device Performance .............................................................................. 29
4.3 LIFE ................................................................................................................................ 30
4.3.1 Operational Life/Durability ....................................................................................... 30
4.3.2 Single Burn Endurance Test ....................................................................................... 30
4.3.3 Life, Starts ................................................................................................................. 31
6
4.3.4 Acceptance Test Procedure Validation ..................................................................... 31
4.4 FUNCTIONAL CHARACTERISTICS ................................................................................... 31
4.4.1 Cold Shock Tests ........................................................................................................ 31
4.4.2 Cold Flow Tests .......................................................................................................... 32
4.4.3 Engine Propellant Inlet Conditions ............................................................................ 32
4.4.4 Transient Characterization ........................................................................................ 34
4.4.5 NPSP Margin and Cavitation ..................................................................................... 37
4.4.6 Pogo and Compliance Characterization .................................................................... 40
4.4.7 Ancillary Systems ...................................................................................................... 40
4.4.8 Thrust Vector and Gimballing ................................................................................... 41
4.5 CONTROLS ..................................................................................................................... 42
4.5.1 Functional Tests ........................................................................................................ 42
4.5.2 Engine Control System Malfunction Logic Check ...................................................... 43
4.5.3 Engine Health Management (EHM) System Validation ............................................ 43
4.6 OPERATIONS ................................................................................................................. 44
4.6.1 Pre‐Test Inspections and Checkouts .......................................................................... 44
4.6.2 Leakage ..................................................................................................................... 44
4.6.3 Post‐Test Inspections ................................................................................................ 45
4.6.4 Drying and Heated Purges ........................................................................................ 46
4.6.5 LRU Demonstrations ................................................................................................. 46
4.6.6 Reusability ................................................................................................................. 46
4.6.7 Operability ................................................................................................................. 47
4.6.8 Preflight Procedures and Flight Sequences ............................................................... 47
4.7 ENVIRONMENTS ............................................................................................................ 48
4.7.1 Thermal Environment ................................................................................................ 48
4.7.2 Climatic Tests ............................................................................................................ 48
4.7.3 Vibration/Shock/Acoustics ........................................................................................ 48
4.7.4 Modal Surveys/Testing ............................................................................................. 50
4.7.5 Vehicle Interface Loads ............................................................................................. 50
4.7.6 Electromagnetic Interference/Compatibility Tests ................................................... 50
4.8 DESIGN AND CONSTRUCTION ....................................................................................... 51
4.8.1 Proof Pressure ........................................................................................................... 51
4.8.2 FOD / DOD Tolerance ................................................................................................ 51
4.8.3 Structural Model Verification .................................................................................... 52
4.8.4 Margin Testing .......................................................................................................... 52
4.8.5 Human Rating (further description in Appendix D) .................................................. 53
4.8.6 Hardware Discrepancy Tracking ............................................................................... 53
4.9 PHYSICAL ....................................................................................................................... 54
4.9.1 Gas Liquefaction Control ........................................................................................... 54
4.9.2 External Icing ............................................................................................................. 54
4.9.3 Mass Properties ........................................................................................................ 54
4.10 UNIQUE REQUIREMENTS .............................................................................................. 55
4.10.1 System Unique Requirements ............................................................................... 55
4.10.2 New or Mission Unique Requirements .................................................................. 55
7
4.10.3 Delta‐Qualification Requirements ........................................................................ 55
5 ACCEPTANCE TESTS ........................................................................................................ 56
5.1 TEST TYPES AND RECOMMENDATIONS ........................................................................ 56
5.1.1 Component ................................................................................................................ 56
5.1.2 Engine ........................................................................................................................ 57
5.2 PERFORMANCE ............................................................................................................. 58
5.2.1 Steady State Performance Characterization ............................................................. 58
5.2.2 Repeatability ............................................................................................................. 59
5.3 LIFE ................................................................................................................................ 59
5.3.1 Operational Life/Durability ....................................................................................... 59
5.3.2 Single Burn Endurance Test ....................................................................................... 59
5.4 FUNCTIONAL CHARACTERISTICS ................................................................................... 59
5.4.1 Propellant Conditions ................................................................................................ 59
5.4.2 Transient Characterization ........................................................................................ 60
5.4.3 Ancillary Systems ...................................................................................................... 60
5.4.4 Thrust Vector and Gimballing ................................................................................... 61
5.5 CONTROLS ..................................................................................................................... 61
5.5.1 Component ................................................................................................................ 61
5.5.2 Engine ........................................................................................................................ 61
5.6 OPERATIONS ................................................................................................................. 62
5.6.1 Pre‐Test Inspection and Checkouts ........................................................................... 62
5.6.2 Leak Test ................................................................................................................... 62
5.6.3 Post‐Test Inspection .................................................................................................. 62
5.6.4 Drying Purges ............................................................................................................ 62
5.7 ENVIRONMENTS ............................................................................................................ 62
5.7.1 Component ................................................................................................................ 62
5.7.2 Engine ........................................................................................................................ 62
5.8 DESIGN AND CONSTRUCTION ....................................................................................... 62
5.8.1 Proof Test .................................................................................................................. 62
5.8.2 Human Rating ........................................................................................................... 62
5.8.3 Hardware Discrepancy Tracking ............................................................................... 63
5.9 PHYSICAL ....................................................................................................................... 63
5.9.1 Mass Properties ........................................................................................................ 63
5.9.2 RESERVED ...................................................................... Error! Bookmark not defined.
5.10 UNIQUE REQUIREMENTS .............................................................................................. 63
6 PRELAUNCH VALIDATION AND OPERATIONAL TESTS ..................................................... 64
6.1 GENERAL GUIDELINES TEST TYPES AND RECOMMENDATIONS .................................... 64
6.2 PRELAUNCH VALIDATION AND OPERATIONAL TEST DESCRIPTIONS ............................. 64
6.2.1 Receiving Inspection .................................................................................................. 64
6.2.2 Purges ....................................................................................................................... 64
6.2.3 Vehicle Readiness Test .............................................................................................. 65
6.2.4 Vehicle Tanking Test ................................................................................................. 65
8
6.2.5 Prelaunch Countdown ............................................................................................... 65
APPENDIX A: DEFINITIONS AND DESCRIPTIONS ..................................................................... 67
APPENDIX B: OTHER RELEVANT DISCUSSION ......................................................................... 73
B.1 OTHER ASPECTS OF DDT&E RELEVANT TO TEST ................................................................. 73
B.1.1 Design Heritage ........................................................................................................... 73
B.1.2 Risk Management ........................................................................................................ 73
B.1.3 Programmatic Considerations ..................................................................................... 73
B.2 TEST PLANNING .................................................................................................................. 73
B.2.1 Minimum Elements for Test Plan ................................................................................. 74
B.2.2 Requirements Definition .............................................................................................. 74
B.2.3 Requirements Verification Test Planning Elements ..................................................... 74
B.3 TEST OPERATIONS AND IMPLEMENTATION ....................................................................... 76
B.3.1 Test Preparation .......................................................................................................... 76
B.3.2 Post‐Test Activities ....................................................................................................... 76
B.3.3 Quality Assurance (QA) ................................................................................................ 77
B.3.4 Configuration Control/Management ........................................................................... 77
B.3.5 Documentation ............................................................................................................ 78
B.4 TEST DATA UNCERTAINTY AND TOLERANCES .................................................................... 78
B.4.1 Testing and Uncertainty Determination Overview ...................................................... 78
B.4.2 Specification Decision Making with Consideration of Uncertainties ........................... 80
APPENDIX C: REPRESENTATIVE CAMPAIGNS AND TEST TURN‐AROUND RATES FOR A VARIETY
OF R&D AND FLIGHT LRE TESTING ......................................................................................... 81
APPENDIX D: HUMAN‐RATING CONSIDERATIONS FOR LRE TESTING AND EVALUATION ........ 87
APPENDIX E: SAMPLE CASES OF USING TEST GUIDELINE TO PLAN TEST PROGRAM ............... 89
APPENDIX F: ACRONYMS ....................................................................................................... 90
List of Figures
FIGURE 3.4‐1: “LEARNING CURVE” FOR F‐1, J‐2, AND SSME PROGRAMS .................................................................................. 19
FIGURE 3.4‐2: DEMONSTRATED RELIABILITY TEST REQUIREMENTS VERSUS RANDOM FAILURE MODES BASED ON WEIBULL STATISTICAL
ANALYSIS METHODOLOGY. .......................................................................................................................................... 20
FIGURE 3.4‐3: DEMONSTRATED RELIABILITY TEST REQUIREMENTS VERSUS WEAROUT FAILURE MODES BASED ON WEIBULL STATISTICAL
ANALYSIS METHODOLOGY. .......................................................................................................................................... 20
FIGURE 4.2‐1: EXAMPLE THRUST‐MIXTURE RATIO BOXES: SPECIFICATION BOX, FLIGHT BOX, AND MARGIN BOX. ... ERROR! BOOKMARK NOT
DEFINED.
FIGURE 4.2‐2: THRUST MARGIN DEMONSTRATION FOR VARIOUS PAST ENGINE PROGRAMS ................. ERROR! BOOKMARK NOT DEFINED.
FIGURE 4.4‐1: NOTIONAL INLET BOXES FOR A LIQUID OXYGEN/LIQUID HYDROGEN ENGINE. ............................................................. 33
FIGURE B.4‐1: SYMMETRIC TWO‐SIDED RELAXED ACCEPTANCE AND STRINGENT REJECTION ............................................................. 80
FIGURE B.4‐2: STRINGENT ACCEPTANCE, SIMPLE REJECTION, AND A TRANSITION ZONE EXAMPLE ...................................................... 80
USING SYMMETRIC TWO‐SIDED GUARD BANDING ..................................................................................................................... 80
FIGURE C‐1: TOTAL NUMBER OF TESTS CONDUCTED (AS OF 12‐31‐04), BY STAND. ....................................................................... 82
FIGURE C‐2: TOTAL NUMBER OF HOT‐FIRE TEST SECONDS (AS OF 12‐31‐04), BY STAND. ............................................................... 83
FIGURE C‐3: AVERAGE NUMBER OF DAYS BETWEEN TEST DAYS, BY STAND. ................................................................................. 84
9
FIGURE C‐4: SSME DAYS BETWEEN TEST‐DAYS, BY NUMBER OF STANDS OPERATING. ................................................................... 84
FIGURE C‐5: TEST SECONDS COMPLETION METRICS, BY STAND ................................................................................................... 85
FIGURE C‐6: HISTORICAL TREND OF SSME TESTING COMPLETION RATE. ...................................................................................... 86
List of Tables
TABLE 4.1‐1: DEVELOPMENT AND QUALIFICATION TEST MATRIX .............................................................................................. 24
TABLE 4.3‐1: EXAMPLE OF LIFE DEMONSTRATION FACTORS USING MIL‐R‐5149B REQUIREMENTS. ..... ERROR! BOOKMARK NOT DEFINED.
TABLE 4.3‐2: ACTUAL DURATION LIFE DEMONSTRATION FACTORS FOR VARIOUS PAST PROGRAMS COMPARED TO THE SSME
REQUIREMENT .............................................................................................................. ERROR! BOOKMARK NOT DEFINED.
TABLE 5.1‐1: ENGINE AND ENGINE CLUSTER ACCEPTANCE TESTS MATRIX ................................................................................... 57
TABLE 6.1‐1: PRELAUNCH VALIDATION AND OPERATIONAL TESTS MATRIX .................................................................................. 64
10
1 SCOPE
1.1 PURPOSE
This document is intended as a test and evaluation guideline that establishes the recommended nature and
extent of testing to successfully develop, qualify, accept, and ready liquid propellant rocket engines and
propulsion systems for launch and space vehicles. The goal is to define a test program that will verify the
design, eliminate latent defects, and help ensure a high level of confidence in achieving successful space
missions.
1.2 APPLICATION
This guideline applies to liquid rocket engines (LREs) and LRE propulsion systems for expendable and re‐
usable applications, including non‐human rated and human‐rated systems including pump‐fed and
pressure‐fed, utilizing various propellant combinations including hydrogen/oxygen, hydrocarbon/oxygen,
storable, and mono‐propellants. This guideline addresses development, qualification, acceptance and pre‐
launch testing for main propulsion systems (i.e., thrust > 1000 pounds force[lbf]) for space launch vehicles
(including booster and upper stage propulsion). This current edition does not address launch vehicle
reaction/roll control systems (RoCS/ReCS), satellite, air‐breathing or combined cycle propulsion. This
guideline focuses on testing of an LRE at the individual engine and integrated vehicle propulsion system
levels. It does not address development and qualification of any system outside of these bounds, although
it is recognized that an LRE is a part of a larger system (a.k.a. stage). Furthermore, this current edition does
not discuss testing at the component level in detail.
1.3 MOTIVATION
The members of the JANNAF TPSP concluded there was a lack of established test guidelines to help
determine the appropriate level and types of testing for LREs. Although certain existing documents provide
guidance for certain related areas, thorough and specific guidance are noticeably lacking in many areas.
The most recent U.S. government specification intended for LRE qualification was MIL‐R‐5149B. This
specification, last updated in 1969, was used as a guide for formal qual of military rocket engines. It was
canceled in 1993. A standard published in 1996 by the Society of Automotive Engineers (SAE) primarily
concentrates on reliability certification requirements [SAE ARP4900 (1995)]. No programs have applied this
standard in practice. MIL‐STD‐1540 and the associated MIL‐HDBK‐340 describe qualification and
acceptance test guidelines for space systems, but do not adequately represent the nuances of rocket
engine testing, mainly because of the difficulty of reproducing engine environments at the engine
subsystem or component level. Existing surveys of past test programs for liquid oxygen (LOX)/kerosene
engines and LOX/hydrogen engines [Emdee (2001) A, Emdee (2001) B, Pempie (2001)] provide valuable
insight into typical test histories, but they are not sufficient by themselves to define the appropriate
requirements for a new engine test program.
Program Managers want a set of guidelines to help scope the engine test program because testing is one of
the largest expenses in a new launch vehicle system. The major costs are associated with the purchase of
engine hardware, test time on test facilities, and failure resolution. Unfortunately, the recent historical
record shows that propulsion systems account for over 50% of launch vehicle failures [Chang (2000)]. Such
a poor flight record suggests that test efforts should be re‐examined for adequacy or at least that a larger
proportion of resources be devoted to engine testing. The industry would benefit greatly from
establishment of practices and guidelines for engine testing that certify high reliability at the lowest
possible cost. This document attempts to fill that need.
LRE development, testing, and verification are an extensive undertaking that can be very expensive, but
absolutely critical for mission risk reduction. A benchmark understanding of the Test & Evaluation (T&E)
aspects of Design, Development, T&E (DDT&E) for LREs is needed as well as a benchmark approach to
reduce the uncertainty and variability in the outcomes of government and future commercial LRE products.
11
2 REFERENCE DOCUMENTS
2.1 GOVERNMENT APPLICABLE DOCUMENTS
The following documents form a part of this guideline to the extent referenced herein. Unless otherwise
noted, the latest revision is preferred.
 Aerospace Report TR‐2005(8583)‐1, “Electromagnetic Compatibility Requirements for Space
Equipment and Systems,” 8 Aug 2005.
 CPIA Publication 247, “Combustion Stability Specifications and Verification Procedures for Liquid
Propellant Rocket Engines,” Oct 1973.

 MIL‐A‐83577B (USAF), “Assemblies, Moving Mechanical, For Space and Launch Vehicles, General
Specification For.”
 MIL‐STD‐461 (DOD), “Requirements for the Control of Electromagnetic Interference Characteristics of
Subsystems and Equipment.”
 MIL‐STD‐464 (DOD), “Electromagnetic Environmental Effects Requirements for Systems.”
 MIL‐STD‐1522 (USAF), “Standard General Requirements for Safe Design and Operation of Pressurized
Missile and Space Systems.”
 MIL‐STD‐1541 (USAF), “Electromagnetic Compatibility Requirements for Space Equipment and
Systems” [Note: references within this document prefer the proposed MIL‐STD‐1541B as described
by Aerospace Report TR‐2005(8583)‐1, “Electromagnetic Compatibility Requirements for Space
Equipment and Systems,” 8 Aug 2005].
 MIL‐STD‐1546A (USAF), “Parts, Materials, and Processes Control Program for Space and Launch
Vehicles.”
 MSFC‐HDBK‐2221, “Verification Handbook Volume I: Verification Process” DCN 001 Change 1,
Sep 1994.
 MSFC‐HDBK‐2221, “Verification Handbook Volume II: Verification Documentation Examples,”
Aug 1994.
 MSFC‐SPEC‐164, “Cleanliness of Components for Use in Oxygen Fuel and Pneumatic Systems
Specification,” Nov 1996.
 NASA‐STD‐5012, “Strength And Life Assessment Requirements For Liquid Fueled Space Propulsion
System Engines.”
 SMC‐S‐016, “Test Requirements for Launch, Upper‐Stage, and Space Vehicles,” Space and Missile
Systems Center, Air Force Space Command, 13 June 2008.
2.2 NONGOVERNMENT APPLICABLE DOCUMENTS
 ANSI/IEEE C63.14, Standard Dictionary for Technologies of Electromagnetic Compatibility (EMC),
Electromagnetic Pulse (EMP), and Electrostatic Discharge (ESD).
 CPIA Publication 655, “Guidelines for Combustion Stability Specifications and Verification Procedures
for Liquid Propellant Rocket Engines,” Jan 1997.
 IEC 61000‐4‐2, “Testing and Measurement Techniques ‐ Electrostatic Discharge Immunity Test,”
International Electrotechnical Commission.
12
2.3 GUIDANCE/REFERENCE DOCUMENTS
The following documents provide additional guidance and/or reference material.
 Aerospace Report TOR‐2003(8583)‐2894, “Space Systems‐Structures Design and Test Requirements,”
2 Aug 2004.
 Aerospace Report TOR‐2005(8583)‐2, “Electrical Power Systems, Direct Current, Space Vehicle Design
Requirements.”
 AFSPCMAN 91‐710, Range Safety User Requirements Manual, Air Force Space Command.
 AFWAL‐TR‐83‐2079, “Weibull Analysis Handbook,” Nov 1983.
 AIAA S‐114‐2005, “Moving Mechanical Assemblies for Space and Launch Vehicles.”
 ANSI/NCSL Z540.3‐2006, “Requirements for the Calibration of Measuring and Test Equipment.”
 ASME‐B‐89‐7.3.1‐2000, “Guidelines for Decision Rules: Considering Measurement Uncertainty in
Determining Conformance to Specifications.”
 Butler, J.P., “The Materials Selection and Structural Development Process for Aircraft Structural
Integrity Under Fatigue Conditions,” Proceedings of the Conference on Fatigue and Fracture of
Aircraft Structures and Materials, Miami Beach (1969).
 Chang, I‐S, “Overview of World Space Launches,” J. Propulsion and Power, Vol. 16, No.5, Sept‐Oct,
pp. 853‐866, 2000.
 Coleman, H. W. and W. G. Steele, “Experimentation and Uncertainty Analysis for Engineers,” Wiley
Interscience, New York, NY, 1989.
 DOD 6055.9‐STD, “DOD Ammunition and Explosives Safety Standards.”
 Emdee, J.L., “A Survey of Development Test Programs for Lox/Kerosene Rocket Engines,”
AIAA‐2001‐3985, 37th AIAA/ASME/SAE/ASEE Joint Propulsion Conference, Salt Lake City, UT, July,
2001 [A].
 Emdee, J.L., “A Survey of Development Test Programs for Hydrogen Oxygen Rocket Engines,” AIAA‐
2001‐0749, 39th Aerospace Sciences Meeting & Exhibit, Reno, NV, January, 2001 [B].
 Evolved Expendable Launch Vehicle Standard Interface Specification, Version 6.0, EELV Program
Office in conjunction with The Aerospace Corp. and the USAF, SMC/MVS, 5 Sep 2000.
 George, D., “Chemical Propulsion: How To Make It Low Cost,” presented at Highly Reusable Space
Transportation Meeting, 25‐27Jul 1995.
 GEVS‐SE, “General Environmental Verification Specification for STS & ELV Payloads, Subsystems, and
Components,” NASA/GSFC, June 1996.
 Hardrath, H. F., “Fatigue and Fracture Mechanics,” AIAA paper 70‐512, Proceedings of the 11th
AIAA/ASME Structures, Structural Dynamics and Materials Conference, Denver (1970).
 ISO 10012:2003, “Measurement management systems – Requirements for Measurement
Equipment.”
 Kirchner, C., Morgan, J., and Rahman, S, “Final Report: NASA SSC Rocket Propulsion Testing Major
Statistics,” Central Engineering File, Study #6323‐06‐007, NASA John C. Stennis Space Center, May 25,
2005.
13
 Manual Of Regulations & Procedures For Federal Radio Frequency Management, National
Telecommunication And Information Agency (NTIA).
 MIL‐HDBK‐340, Volume I, “Test Requirements for Launch, Upper‐Stage, and Space Vehicles:
Baselines.”
 MIL‐HDBK‐340, Volume II, “Test Requirements for Launch, Upper‐Stage, and Space Vehicles:
Applications Guidelines.”
 MIL‐Q‐9858, “Quality Program Requirements” [Note: the most recent revision A was cancelled
1 Oct 1996].
 MIL‐R‐5149, “General Specification for Liquid Propellant Rocket Engine” [Note: the most recent
revision B was cancelled 20 Apr 1993].
 MIL‐STD‐810 (DOD), “Test Method Standard for Environmental Engineering Considerations and
Laboratory Tests.”
 MIL‐STD‐822 (DOD), “Standard Practice for System Safety.”
 MIL‐STD‐1540 (USAF), “Test Requirements for Launch, Upper‐Stage, and Space Vehicles”
[Note: references within this document prefer a proposed MIL‐STD‐1540E as described by SMC‐S‐
016, “Test Requirements for Launch, Upper‐Stage, and Space Vehicles,” 31 Jan 2004].
 MIL‐STD‐1568B (USAF), “Materials and Processes for Corrosion Prevention and Control in Aerospace
Weapons Systems.”
 MSFC‐SPEC‐522A, “Design Criteria for Controlling Stress Corrosion Cracking.”
 NIST Technical Note 1297, “Guidelines for Evaluating and Expressing the Uncertainty of NIST
Measurement Results,” the National Institute of Standards and Technology, 1994.
 NPR 8705.2A, “Human‐Rating Requirements for Space Systems,” NASA Procedural Requirement, 7
Feb 2005.
 Payne, A.O., “Determination of the Fatigue Resistance of Aircraft Wings by Full‐Scale Testing” in Full‐
Scale Fatigue Testing of Aircraft Structures, F.J. Plantema and J. Schijve eds., Pergamon Press (1961).
 Pempie, P. and Vernin, H., “Liquid Rocket Engine Test Plan Comparison,” AIAA 2001‐3256, 37th
AIAA/ASME/SAE/ASEE Joint Propulsion Conference, Salt Lake City, UT, July, 2001.
 RI/RD89‐171, “Reusable Rocket Engine Turbopump Health Monitoring System,” Contract NAS3‐25279
for NASA Lewis Research Center, Rockwell International/Rocketdyne Division, 14 March 1989.
 RTCA/DO‐160 (ISO 7137) (EUROCAE ED‐14), “Environmental Conditions and Test Procedures for
Airborne Equipment,” Radio Technical Commission for Aeronautics (RTCA), Washington, DC.
 SAE ARP4900, “Liquid Rocket Engine Reliability Certification,” 1995.
 Space Vehicle Design Requirements, The Aerospace Corp., 13 Jan 2005.
2.4 ORDER OF PRECEDENCE
In the event of conflict between the text of this document and the references cited herein, the text of this
document takes precedence. However, nothing in this document supersedes applicable laws and
regulations unless a specific exemption has been obtained.
14
3 GENERAL TEST GUIDELINES
3.1 GENERAL TEST PHILOSOPHY
The primary objective of any test program should be to maximize the probability that the tested design will
function properly and successfully when used in actual service for the intended application. Flight risks
should be mitigated via prudent and effective testing. Certain key tenets of testing have served the liquid
propulsion test community well as it has tried to accomplish that objective.
One particular “tried and true” philosophy is the “test‐like‐you‐fly” approach. “Test‐like‐you‐fly” means
that testing should demonstrate engine operation with flight representative hardware and under flight
representative conditions, including expected worst‐case conditions. The overall test program should
encompass and explore as much of the operational flight envelope as possible to accomplish the general
objective of avoiding operating any flight hardware configuration under any conditions for the first time in
flight. Specific tested conditions for expected operating ranges should include, but are not limited to,
general performance requirements (such as thrust and mixture ratio), interface conditions (such as
propellant inlet conditions, electrical, pneumatic and hydraulic connections), and environmental exposure
(such as thermal, acoustic, and vibration conditions). Some flight environments (acceleration for one)
cannot be replicated during ground test. Margin testing with respect to the expected flight conditions
should be included to protect against known and unknown uncertainties in the flight conditions
(e.g., ground‐to‐flight dispersions), as well as known and unknown hardware variations
(e.g., manufacturing tolerances, non‐conformances, and undetected deficiencies). If it is impractical to test
or simulate a particular flight condition on the ground, then additional margin might be appropriate.
Furthermore, testing should consider and account for engine hardware experience throughout all phases
of the required life cycle, including manufacturing, acceptance testing, transportation, handling, storage,
vehicle integration, checkout testing, launch preparations, liftoff, and flight.
A corollary to the above is the “fly‐as‐you‐test” approach. “Fly‐as‐you‐test” means that flight operation
should remain within demonstrated ground‐tested and qualified regimes, and that design and process
differences between qualified test hardware and flight hardware should be minimized. A successful
development and qualification program will anticipate all potential flight conditions and ensure those
conditions are validated by a robust test program.
Hot‐fire testing to verify that a LRE design is ready for flight typically consists of four phases of major
program activity: prototype testing, development testing, qualification, and integrated system testing. The
first three test phases can occur at the component level as well as the engine level. The integrated system
testing phase is performed at the propulsion system and/or vehicle level. After a LRE design has completed
the qualification program, each individual flight engine is typically acceptance tested by hot‐fire to verify
that specific engine’s suitability for flight. Prelaunch operational testing is performed prior to engine start
and liftoff to verify readiness for launch. Finally, additional testing can be performed or additional data
obtained for the system during the operational phase of a test flight or actual mission launch. Each of these
test phases is described below to provide a common set of definitions for the discussion.
3.2 TAILORING OF GUIDELINES
The test guidelines provided herein establish a baseline of test recommendations for consideration as part
of an overall verification program. Specific recommendations are based on experience and
industry/government best practices, and assume a new engine system with no design heritage.
The Panel anticipates that each program will tailor these general guidelines based upon specific
requirements and programmatic constraints. Not every program will utilize all test objectives described
herein, nor are the listed objectives all encompassing. The intent of these guidelines is to provide
recommendations and possible alternatives.
15
It is the recommendation of this Panel that rationale for any tailoring be established and documented,
especially in cases where a lower equivalent level of verification is being accepted, so a Program Manager,
Chief Engineer, and Chief Safety Officer can jointly make knowledgeable decisions regarding any additional
risk resulting from deviation from the established best practices.
3.3 ENGINE SAMPLES
Few aspects of a development and qual program have as great an impact on its scope, cost, and schedule
as the number of engine samples and number of tests to be incorporated into that test program.
Historically, testing and test hardware account for about 75% of the total full‐scale development costs
[George (1995)]. Thus, program costs scale quite directly with the number of engine and test samples. On
the other hand, as will be discussed below, programs typically do not have sufficient engine and test
samples to statistically demonstrate the high reliability and high confidence demanded for launch vehicle
propulsion systems. Thus, determining the appropriate scope of a test program can be difficult. The
following sections describe various aspects and approaches to consider when making these decisions.
3.3.1 Number of Engine Samples
To remain consistent with the NASA‐STD‐5012, six (6) unique engine samples holding similarity with the
flight design are required for verification. Each engine may have different test objectives associated with
it, thus introducing some leeway with regards to hardware configuration. The intent is to apply a sample
size large enough to begin to establish build‐to‐build variation with some level of confidence. The total
engine count to meet this recommended sample size can include the later development and qualification
samples as verification tends to demand a configuration closely related to flight. Alternate paths do exist
and have been exercised in the past.
Alternate approaches utilizing fewer samples are implemented to demonstrate the robustness of the
system and build the confidence necessary for spaceflight. The intent is to replicate all the major
functions of the integrated propulsion system to capture the complex system interactions
3.3.1.1 Number of Engine Samples (Pressure‐Fed)
A select set of pressure‐fed engine applications fall under the auspices of this guideline. It is
recommended for a new engine design, a minimum of three engine samples be included as part of the
engine verification activity. These are to include: one development article with a corresponding
successfully completed development test activity (addressing all intended usages, propellant
condition/interface conditions, representative operating duty cycles, and thermal and combustion
stability); and two engine qualification samples (encompassing and formally verifying the aspects of the
development thruster program). Size‐scaling of existing engine designs should be considered as
equivalent to a new engine design. Development or qualification engine activities which result in major
failures or anomalies requiring modification of the engine design should not be considered as part of the
three engine samples.
For a modification of an existing pressure‐fed engine design or a mission application which extends an
existing pressure‐fed engine design in terms of life or interface conditions, but for which existing analysis
shows the design to be capable of meeting all requirements, it is recommended that a minimum of two
engine samples, with a minimum of one qualification specification, be included as part of the engine
verification activity. Development or qualification engine activities which result in major failures or
anomalies requiring modification of the engine design should not be considered as part of the three
engine samples.
This approach is different than that of NASA‐STD‐5012, which was formulated to address pump‐fed
engine design. The above pressure‐fed engine guidelines reflect a historical precedence approach, just as
NASA‐STD‐5012 reflects a historical precedence approach for pump‐fed engines. The pressure engine
design has a reduced number of design elements which could contribute to variations in life and
16
performance, although mission specific application effects (propellant interface conditions, operating
duty cycles) may be much more significant for the engine design and subsequent verification.
3.3.2 Engine Qualification Samples
It is recommended that there be at least two (2) qualification engine samples. Detailed discussion for
specific objectives in later sections will note when more or less unique samples are recommended.
3.3.3 Integrated Propulsion System Samples
It is recommended that at least one integrated propulsion system sample be tested. The intent is to
replicate all the major functions of the integrated propulsion system to capture integrated propulsion
system interactions. This can sometimes be done using a battleship, although flight similarity is
encouraged.
Background/Rationale: The complexity of a rocket engine design results in hardware dimensional
variances from engine‐to‐engine. Despite the careful attention paid to identifying and controlling critical
tolerances in the design phase, engine testing will often identify significant engine‐to‐engine variations in
operating conditions and other responses. Common examples include pump cavitation characteristics,
turbine blade responses to forcing functions, bearing loading, pump chilldown characteristics, ignition
effectiveness, and self‐induced vibration. Adverse responses to variations can result in a design with
fewer margins than desired. Therefore, part of the function of the test program is to provide insight into
engine‐to‐engine variations and to verify that these variations are acceptable for the given design and
within the given application. Multiple engine samples demonstrating a test objective are required to
provide adequate confidence that the engine operation and its variations are well understood. It is
acceptable to use some rebuilt engines to reduce costs, but at the disadvantage of reducing the extent of
normal variation that will be observed. A significant number of development engines should be included
into the program to refine and reduce risk for the final flight design. The optimum number will depend on
design complexity, heritage, and risk tolerance. However, for the final verification and qualification of a
new engine, experience indicates that four (4) unique engine samples with the flight design are typically
the minimum required for most objectives. Qual programs should include at least two of these unique
flight design engines samples, with the remainder being incorporated into the development test program.
Some verification test objectives can be satisfied by the development effort (e.g., combustion stability
bomb tests). Based upon engineering assessment some of the development test effort can also provide
additional test samples to support verification and add confidence. Detailed discussion for specific
objectives in later sections will note when more or less unique samples are recommended. If an engine
includes reused parts, determination of whether that engine sample is “unique” depends on valid
engineering assessment based on knowledge of the engine build history and the specific objective under
consideration. For example, an ignition test sample would be unique if the igniter and injector were
changed during an engine rebuild, but for a pump chilldown test sample the uniqueness of the
turbopump unit would be of interest. One or more engine samples (new or reused from the engine level
testing) should undergo additional tests at the integrated systems level.
Fewer than four unique engine samples may be appropriate for a modified engine design with significant
design heritage and extensive existing test experience (e.g., an “evolved” engine or a slightly modified
new version of an existing engine). However, such reductions must be considered carefully and well
justified to ensure that prior design heritage and test experience is directly applicable to the new aspects
of the current engine design.
3.4 NUMBER OF TOTAL TESTS
The test program should include enough tests to verify all the specific engine system performance
requirements and functional objectives. Planning a sufficient number of tests to verify all the requirements
is difficult to determine. There is simply no “cookbook.” Experience, analytical capabilities, and deviation
from known, highly matured technologies/practices all play a role. The number of tests to satisfy a
particular requirement will be visited during the design and verification planning phase, and then revisited
17
during and after the test phase when the verification compliance paperwork is being filled out, telling the
story of how the requirement was satisfied and how the actual test program satisfied versus the planned.
Further, there will be programmatic influences that significantly shape the test campaign. Depending upon
these influences, there can be a number of paths available to obtain the necessary data to qualify the
design. Largely, the development path will have to do with interdependencies of components, sub‐
systems, and system (a.k.a. system interactions).
Looking back at previous large engine programs, a “learning curve” of sorts has been established
(Figure 3.4‐1). Even with recent programs like RS‐68, the trend was essentially in family. However, with one
significant difference, the total numbers of tests for the development program was significantly less than
the predecessors whose programs were used to generate the curve. Evidently, some of the key factors
which play a role in understanding the physical system and its interactions had evolved. Some of the
benefits this program had over the predecessors can be attributed to the increased analytical capability,
experience base, and mission (cost‐driven, non‐manned spaceflight). The complexity of Space Shuttle Main
Engine (SSME) and the long lasting ground test programs have allowed modeling of physical systems to
evolve significantly as did computing power. The cost‐driven aspects kept technology development to an
absolute minimum, which would be another significant driver for additional development tests.
Technology development areas will definitely add significant amounts of testing at lower levels through
system level (depending on technology area) to ensure it is understood sufficiently before flight.
Utilizing this information for test planning, it is recommended that the qualification portion of the test
program not be initiated until the latter portion of the total test program. There tends t o be efforts to
reduce test program durations by running engines in parallel on multiple facilities. This is viable, but
maintaining that a development test program should be through at least 80% of its tests or greater is a
good rule. The intent is to minimize any potential for issue, and this would logically be at the latter portion
of the program where there has been sufficient time to explore each of the functional requirements and
incorporate necessary design changes or operational refinements.
Continuing the focus on test planning, there are two potential approaches which can be used to establish a
total numbers of tests: the “statistically relevant based” approach or the “functional objectives based”
approach. A successful test program might include a combination of both of these approaches. There are
other approaches and, as always, the Program Manager, Chief Engineer, and Chief Safety Officer must
carefully weigh programmatic requirements and constraints versus acceptable risk levels when selecting
the desired test approach.
18
Based on Failures to Qual for F-1, J-2, and SSME

(Regenerated from Jan Monk Presentation)
100
90
80
% of Failures to Qual 70
60
50
40
30
20
10
0
0 10 20 30 40 50 60 70 80 90 100
% of Tests to Qual
Figure 3.4‐1: “Learning Curve” for F‐1, J‐2, and SSME Programs
3.4.1 Statistically RelevantBased
This approach relies on standard statistical analysis methods to determine the number of tests that must
be performed successfully to statistically demonstrate the specified engine reliability, for a defined
confidence level and with an assumed failure distribution. For example, some past programs have used
the Weibull statistical methodology as described in AFWAL‐TR‐83‐2079. Based on this methodology,
demonstrating 99% engine reliability at 90% confidence versus random failure modes (Weibull statistical
distribution with Beta = 1) requires, for example, testing of about 20 times the required engine
operational life on each of ten engine samples with zero failures (Figure 3.4‐2). Based on the same
methodology, demonstrating 99% engine reliability at 90% confidence versus wearout failure modes
(Weibull statistical distribution with Beta = 3 to 6) requires, for example, testing of about 2 to 4 times the
required engine operational life on each of four engine samples with zero failures (Figure 3.4‐3). The plots
show that demonstrated reliability increases fastest for random failure modes by testing additional units,
and for wearout failure modes by testing additional duration, as would be expected. Unfortunately, the
testing required to statistically demonstrate the high reliability and high confidence demanded for launch
vehicle propulsion systems against random failure modes is typically cost prohibitive and impractical. On
the other hand, demonstrating high reliability and high confidence against common wearout failure
modes is much more reasonable and achievable.
19
ZERO FAILURE TESTING:

Beta = 1 (Random), Reliability at 90% Confidence Level
50 99%
45 95%
90%
40
Test Time / Service Life

85%
35
30
25
20
15
10
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Number of Units Tested
Figure 3.4‐2: Demonstrated reliability test requirements versus random failure modes based on Weibull
statistical analysis methodology.
ZERO FAILURE TESTING:

Beta = 3 to 6 (Wearout), Reliability at 90% Confidence Level
7
99%, Beta 3
6 85%, Beta 3
99%, Beta 6
Test Time / Service Life
5 85%, Beta 6
99% reliability range
(Beta 3 to 6)
4
1
85% reliability range
(Beta 3 to 6)
0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Number of Units Tested
Figure 3.4‐3: Demonstrated reliability test requirements versus wearout failure modes based on Weibull
statistical analysis methodology.
3.4.2 Functional ObjectivesBased
This approach is preferred. The goal of the functional objectives based approach is to verify all the specific
engine system performance requirements and functional objectives of Table 4.1‐1: Development and
Qualification Test Matrix as efficiently as possible in the minimum number of tests. However, it is
recommended that enough tests be performed to adequately exercise the full range of engine operating
conditions, including nominal, off‐nominal, and extreme conditions (with margin where practical).
Furthermore, it is recommended that enough tests be performed for each of the various conditions to
adequately characterize the normal variability of the engine system for those conditions. Since this
approach generally will not include sufficient test samples to statistically demonstrate reliability
20
requirements against random failure modes and perhaps not against wear‐out failure modes, it is
necessary to verify significant margin against various key engine performance aspects, requirements, and
operating conditions to ensure adequate robustness in the design by identifying any design flaws and
failure modes that might exist. If sufficient margin is demonstrated, the functional objectives approach
provides protection against unknowns not addressed by the demonstrated reliability based approach.
Margin is relative to the maximum expected operating conditions. It may include an increase in level or
range, an increase in duration or cycles of exposure, or any other appropriate increase in severity. Other
technical disciplines have incorporated robust margin test requirements for their systems to reduce
historical failure rates. For example, standard practice for structural static load test qualification requires
a design ultimate load test of 1.4 times the limit load (LL) for human‐rated flight, and 1.25 times the LL for
non‐human rated flight (MIL‐STD‐1540, MIL‐HDBK‐340A). A global margin requirement is impractical to
apply for a LRE, since the complex and often very non‐linear interactions in these systems result in vastly
differing conditions throughout the engine. Thus, the test margin conditions must be carefully selected to
demonstrate robustness for critical aspects of the engine, without over testing other parts.
Recommended margin requirements for specific objectives and design aspects of LREs are discussed
within the respective sections that follow.
3.4.2.1 Functional Objectives‐Based (Pressure‐Fed)
In addition to the approach identified for section 3.4.2 (Functional Objectives‐Based), it is recommended
that pressure‐fed engines under the auspices of this guideline also address the application specific
combinations of propellant interface conditions, propellant quality and conditions, and operational duty
cycles of the mission. Particular consideration should also be given to thermal soak‐back conditions
possible for the engine for any restartable application or mission duty cycle, as well as any potential
environmentally‐induced thermal conditions imposed on the engine. It is recommended that verified
engine life meet or exceed a minimum life margin of 100% of the mission application (twice the required
mission application), in all appropriate potentially life‐limiting characteristics such as propellant through‐
put, thermal cycles, pressure cycles, etc.
3.4.3 Testing to Failure
Historically, testing to failure was originally employed since analytical tools were immature and
experience was limited. After multiple test‐fail‐fix cycles, the hardware demonstrated the necessary
margin and reliability for spaceflight. It can be a very effective technique as it produces quantifiable
results, especially for smaller or newer organizations entering the rocket business. However, it can
become costly as the samples necessary to perform these demonstrations is significant, driving program
cost and schedule.
3.4.4 Modeling and Simulation
Although details of this effort are excluded from this document, it is recognized that modeling and
simulation complement testing. Therefore, acquisition of data to validate and calibrate analytical models
for the ‘by analysis’ element of development and qualification verification of requirements of compliance
should be a major test objective. One of the fundamental purposes of testing is the validation and
calibration of physics‐based models. Much effort is placed in this area to increase fidelity of the initial
design based upon past experience and applying it to new or evolved propulsions system designs. Analysis
efforts that are inherently lower risk can be initiated earlier in the DDT&E effort and provide the capability
to explore many aspects of the design prior to hardware manufacture and assembly. However, test data is
necessary to validate the analytical assumptions. There is no substitution for test.
There are test objectives tied to validating and anchoring models, which can then be extrapolated or
interpolated with confidence to verify functionality and performance. The goal is to reduce the numbers
of tests necessary to provide the confidence necessary for flight.
Modeling and simulation serve several important functions. First, they provide early design guidance and
system characterization. Second, they give preliminary analytical indication of functional performance and
21
integrity, which helps maximize the probability of successful verification during subsequent testing.
Finally, in specific cases, they may provide sufficient analytical verification without testing. This is
especially critical for those aspects of engine performance and operation that are impossible or
impractical to adequately test on the ground. Furthermore, with the start of more powerful computing
capabilities, advanced simulation tools, and improved manufacturing processes, recent engine
development and qualification programs have successfully utilized more extensive and accurate upfront
robust design, sensitivity studies, and optimization to reduce the traditional and costly “test‐fail‐fix”
design cycle that has plagued many past programs. This allows an earlier entry into the verification phase
of the test program. Nevertheless, sufficient testing is required to verify the design since any design result
is subject to the validity of assumptions and proper consideration of all potential failure modes. Improved
analyses coupled with customary design verifications can yield better reliability for the same cost, or they
can be coupled with a reduced level of design verification to yield lower cost for the same level of
reliability.

22
4 DEVELOPMENT AND QUALIFICATION TESTS
The objective of the overall development and qualification test program is to yield a flight design,
manufacturing processes, and acceptance program that produces flight hardware/software that meet
specification and performance requirements with adequate margin to accommodate normal engine hardware
variations. Development testing is required to achieve design maturity, demonstrate capability, and to reduce
risk to the qualification program. Qualification testing is required to formally verify compliance of the flight
design with requirements. The guidelines below apply to both development and qualification testing, except
as noted. It is suitable to satisfy multiple objectives on any given single test if it can be accomplished safely
and effectively.
4.1 TEST TYPES AND RECOMMENDATIONS
Even though requirements levied upon a propulsion system can vary from system to system, the body of
types of tests is relatively well known. These establish the basic functionality and overall robustness of an
engine or integrated propulsion system. The tables in this section provide the different test types and
recommendations; the following section provides context for the recommendation.
4.1.1 Component Tests
Specific component test guidelines will be addressed in future editions. The focus of this current edition
is engine level testing, so no detailed discussion of component level testing will be provided. Readers
should refer to SMC‐S‐016, MIL‐HDBK‐340, MIL‐A‐83577B, and AIAA S‐114‐2005 for component level test
guidelines.
Many major engine components are subjected to component level development and qualification testing
to reduce risk for subsequent engine level testing and to verify flight readiness. SMC‐S‐016 and MIL‐HDBK‐
340 describe test guidelines for components such as valves and engine control units (ECUs). AIAA S‐114‐
2005, NASA‐STD‐5017, and MIL‐A‐83577B specify general requirements for the testing of moving
mechanical assemblies (MMAs). Test standards for more complex components, such as turbopump
assemblies (TPAs) and combustion devices, are not generally available. These more complex components
mainly undergo development testing to reduce risk for engine level testing. The significant level of engine
system interaction with these types of components makes it impractical to qualify these parts on an
individual basis, since the component‐level test conditions are not representative of engine conditions.
Often a robust sub‐scale test program is used to significantly reduce full‐scale component risk.
Furthermore, since the actual engine operating environments for various components may not be well
characterized prior to the engine level testing, it is prudent to include significant margin in the component
level design and testing.
The specific test plan and objectives for different types of components will vary. For example, testing of
the TPA and its components might emphasize piece part development, performance mapping, efficiency
characterization, and TPA health checkout, with a gradual buildup of the system from turbopump to
powerpack. In contrast, testing of combustion devices such as the thrust chamber assembly (TCA), gas
generator (GG), or preburner (PB) might emphasize c‐star performance evaluation and verification. The
intent would be to characterize these components, then push to systems test to truly verify the
requirements. The component stand will not provide FULL verification – the requirements will be verified
at the system level. Therefore, even after their respective component test programs, componentry will be
assembled into an engine to validate component qualification levels (where possible), and expose the
component to the interactive, or combined, system environments.
Some components, such as line replaceable units (LRUs), may undergo sufficient component acceptance
testing to preclude the need for engine hot‐fire testing. If this is the desired approach, it has to be
verified at the system level that the engine is insensitive to the manufacturing variability, or the variability
is predictable enough to not require engine level hot‐fire characterization prior to flight.
23
Following component testing, it often is valuable to include some additional sub‐system testing. This
provides further risk reduction prior to full‐up engine level tests. One common example is testing of the
powerhead (or powerpack) sub‐system, which includes the turbopump, GG or pre‐burner, injector, and
chamber. Depending upon cycle, the system may not allow you to test the main injector or chamber.
Typical powerhead test objectives include verifying pump performance and efficiencies, verifying turbine
efficiencies, developing purge and bleed requirements, and characterizing injector and chamber
performance. This test would be performed with propellant flow and would utilize flight representative
spin start system, if applicable, and could include various throttle conditions as required. Another
common example is testing of the engine control system, which might include valves, ECU, sensors, and
flow orifices. Typical engine control system objectives include validating control system logic, verifying
response times, exercising fault detection and accommodation, and characterizing control system
accuracy. This test could be performed with simulated fluid flow and resistances, or as a combination of
hardware and system response performance simulation.
4.1.2 Engine
The single engine test program (together with appropriate analysis) accomplishes most of the
demonstration and qualification necessary to verify requirements compliance. It is necessary because of
the significant component‐to‐component interactions, unknown environments, and non‐linearities that
cannot be adequately predicted nor replicated at lower levels. The engine unit includes an assembly of all
of the primary subsystems (e.g., TPA) as well as all major components (e.g., valves and combustion
devices). Engine level testing is critical to design verification since there are significant interactions
between the various components and subsystems that cannot otherwise be simulated or characterized.
Furthermore, it is difficult and often impossible to test components and subsystems in the actual engine
environment other than through their testing at the engine level itself. In several instances, the
component development and qualification testing addresses individual environmental stresses and the
impact to the functional capability, whereas the engine level introduces systems interactions and
combined environments (engine self induced). Most of the test recommendations will be pertinent to this
level of testing.
Table 4.1‐1 types of tests which are expected to be included in a DDT&E engine test program for a single
engine. These are to be utilized in conjunction test recommendations, and should not be treated as
mutually exclusive.
Table 4.1‐1: Development and Qualification Test Matrix
Component Engine
Section Objective Dev Qual Dev Qual
4.2.1 Steady State Performance Characterization   x x
4.2.2 Repeatability   x x
4.2.3 Run-time Trends  
4.2 Performance
x x
4.2.4 Engine Influence Coefficients   x x
4.2.5 Mixture Ratio Excursions   x x
4.2.6 Thrust/Mixture Ratio Margin Demonstration   x x
4.2.7 Ignition System x x x x
4.2.8 Turbomachinery x  x x
4.2.9 Combustion Devices x  x x
4.3.1 Operational Life/Durability x x x x
4.3.2 Single Burn Endurance Test   x x
4.3 Life
4.3.3 Service Life (SL), Number of Starts   x x

4.3.4 Acceptance Test Validation   x x
4.4.1 Cold Shock Tests   x 
4.4.2 Cold Flow Tests x x x 
Propellant Conditions
Characteristics
4.4 Functional
– Pre-start Chilldown x x
4.4.3 – Start Propellant Conditions   x x
– Steady State x x
– Shutdown Propellant Conditions x x
Transient Characterization 
4.4.4  
– Start Transient x x
24
Component Engine
Section Objective Dev Qual Dev Qual
– Restart x x
– Throttle Transient x x
– Shutdown Transient x x
– Abort Shutdown x x
4.4.5 NPSP Margin and Cavitation x x x
4.4.6 Pogo and Compliance Characterization x x x
Ancillary Subsystems
– Autogenous Pressurization x x
4.4.7 – Valve Actuation x x x x
– Purges x x x x
– Electrical Power and Ignition x x x x
Thrust Vector and Gimballing
– Gimbal Limits x x x x
– Roll Control Limits x x
– Ambient Environment x x
4.4.8 – Inspections x x x x
– Heat Flux x x
– Clearance x x
– Interface Compatibility x x
– Thrust Vector Alignment x x
4.5.1 Functional Tests x x x x
Contr
4.5.2 Eng Control System Malfunction Logic Check

4.5
ols
x x x x
4.5.3 Engine Health Management x x x x
4.6.1 Pre-Test Inspections and Checkouts x x x x
4.6.2 Leakage x x x x
4.6 Operations
4.6.3 Post-Test Inspections x x x x

4.6.4 Drying Purges x x
4.6.5 Line Replaceable Unit (LRU) Demonstrations x x
4.6.6 Reusability x x x x
4.6.7 Operability x x
4.6.8 Preflight Procedures and Flight Sequences x x
4.7.1 Thermal Environment x x x x
4.7.2 Climactic Tests x x x
4.7 Environments
Vibration/Shock/Acoustics
4.7.3 – External Vibration x x x
– Self-induced Vibration x x x
4.7.4 Modal Surveys/Testing x x
4.7.5 Vehicle Interface Loads x x
Electromagnetic Interference/Compatibility
4.7.6 x x x
Tests
4.8.1 Proof Pressure x x
4.8.2 FOD/DOD Tolerance
4.8 Design &
Construction
x x
4.8.3 Structural Model Validation x x
4.8.4 Margin Testing x x
4.8.5 Human Rating x x x x
4.8.6 Hardware Discrepancy Tracking x x x x
4.9.1 Gas Liquefaction Control x x x
4.9.2 External Icing Control
4.9 Physical
x x
Mass Properties
– Mass x x x x
4.9.3
– Center of Gravity x x x x
– Moments of Inertia x x x x
Requirements
4.10 Unique
25
4.2 PERFORMANCE
4.2.1 Steady State Performance Characterization
Test programs should conduct enough testing to sufficiently characterize the different performance
parameters and their sensitivities. Thrust, specific impulse, and mixture ratio are all examples of
performance parameters that must be measured or computed/derived from measured data during
engine test to thoroughly anchor analysis models over a range of input variations to minimize any
interpolation of flight data from the family of ground test data. Accurate measurement and
characterization is necessary as vehicle requires characterization of engine performance over time to
properly calculate propellant load and reserves, calculate vehicle guidance, navigation and control
parameters, etc.
4.2.1.1 Component
RESERVED
4.2.1.2 Engine
Steady state performance should be thoroughly characterized by varying various parameters to anchor
analytical models to minimize uncertainties and minimize flight risks. The engine physics‐based steady
state model should be anchored to test data such that 3‐sigma conditions can be evaluated and show
that build‐to‐build and run‐to‐run variations still falls within requirement limits. Establishing and
validating engine gains are inherently a part of this model anchoring activity. To accomplish this,
multiple engine tests on each of a family of engines are expected. Measuring performance tends to
occur during “steady state” time slices during a test where no commanded changes occur.
Measurements should be taken at all power levels with emphasis at power levels where the engine
spends a significant portion of the mission.
4.2.2 Repeatability
Run‐to‐run and engine‐to‐engine variations in thrust, mixture ratio (MR), Isp, and other key engine
operating parameters.
4.2.2.1 Component
RESERVED
4.2.2.2 Engine
Run‐to‐run and engine‐to‐engine variation characterization should be performed during a repeatability
test campaign. Tests program should be sufficient to have the steady state analytical model anchored to
enough data so that the predictions fall at or below 0.1% of measured values (once calibrated). Three
tests repeating the same profile should be run each of 3 engines minimum. For engines with multiple
power levels required, this must be taken into account as part of the profile (thrust, mixture ratio, etc.).
4.2.3 RunTime Trends
These are uncommanded and/or unintentional time varying operating conditions in pressures,
temperatures, or component efficiencies.
4.2.3.1 Component
RESERVED
4.2.3.2 Engine
Conduct characterization of time dependent parameters. If significant unexpected trends or trends
exceed specification limits, corrective action or specification changes are required.
26
4.2.4 Steady State Analytical Models
Steady state analytical models predict engine internal operational conditions (pressures, temperatures,
flowrates) based upon specified input conditions and expected hardware characteristics. Because the
model is physically‐based, exercising the model varying the input parameters results in influence
coefficients that represent changes in various engine performance parameters given the change in input.
4.2.4.1 Component
RESERVED
4.2.4.2 Engine
Determine (development testing) and verify influence coefficients based on inlet conditions, valve
positions if applicable, and other interface conditions for steady state engine performance (e.g., thrust,
Isp, MR) and key operating conditions (e.g., pump speed, chamber pressures and temperatures, pump
discharge pressures). Exact parameters will vary depending upon the engine design. Characterize
influence coefficient for other allowable propellants (like ancillary systems), if applicable.
4.2.5 MR Excursion Tests
The intent is to vary mixture ratio to disperse the family of data which will help characterize engine
behaviors and hardware durability. Depending on engine configuration, MR variation may be a desired
functionality for a number of reasons. However, start and run box variations that may exist on a stage can
also introduce variations which the engine has to contend with. Some systems may have control features
to minimize variation in combustion zones and/or turbomachinery (closed‐loop control), but others may
elect to tolerate these potential external variations (open‐loop control).
4.2.5.1 Component
RESERVED
4.2.5.2 Engine
The mixture ratio excursion test campaign should characterize the engine hardware sensitivity to
mixture ratio variations. Verify MR control and dispersion requirements are met. Test duration at each
MR condition should be equal to at least one times the maximum expected operational life exposure.
Develop MR and inlet condition relationships. Test minimum and maximum MR at nominal, off‐nominal
and worst‐case propellant inlets, including margin. Include demonstrations at throttle points and
repeatability tests.
4.2.6 Thrust/MR Margin Demonstration
The intent is to envelop extreme operating conditions and verify engine robustness via margin testing on
thrust and MR conditions beyond expected worst‐case ground‐test and flight conditions. Worst‐case
conditions should account for all possible dispersions and biases, including run‐to‐run variations,
acceptance test measurement uncertainty, ground‐to‐flight dispersions, operating biases (e.g., off‐
standard inlet conditions), run‐time trends, and post‐test engine hardware replacements. Minimum MR
and thrust margin should be 2% beyond the worst‐case MR and Thrust envelope. The test duration at
various margin conditions should be at least one times the maximum expected SL exposure time at the
respective worst‐case condition. Testing of greater margin during development testing is recommended
to increase the likelihood of successful qualification testing unless development data indicates otherwise.
To the extent possible, testing should also encompass the worst‐case expected flight operating range of
critical engine parameters (e.g., speeds, pressures, temperatures).
4.2.6.1 Component
RESERVED
4.2.6.2 Engine
The Panel recommends that engine thrust and mixture specification margins be demonstrated at the
engine system test level. Testing outside of the specification limits is desired to help understand the
27
engine’s off‐design perormance, significant testing outside the specification limits is unrealistic due to
the cost associated with damaging engine hardware and the test facility.
4.2.7 Ignition System
Verify reliable ignition of all combustion devices under nominal, off‐nominal, and worst‐case conditions
for propellant conditions (including mixture ratio, temperature, and pressure), chilldown conditions, input
voltage (if electronic excitation is utilized), and hardware temperatures. Ignition testing should take place
at sea level for ground start engines and at vacuum for altitude start engines. Tests for worst‐case
conditions should include ignition system variables with margin on top of the engine operational limits.
Electronic ignition must address spark production variables and delays. Hypergolic ignition systems must
address quantities and timing of fluid introduction. Pyrotechnic ignitions must address the pyrotechnic
device’s loading and timing.
4.2.7.1 Component
Significant component or subsystem type testing should be conducted on ignition systems to minimize
risk to engine level testing. The testing should be conducted within a environment that simulates the
engine operation.
4.2.7.2 Engine
Characterizing start transient timing and sensitivities will capture the ignition system performance as a
part of the engine test program. The engine hot‐fire program should vary parameters for propellants,
conditions, etc., sufficiently to capture ignition performance relative to start transient variation.
Altitude‐start engines should conduct altitude simulation to characterize ignition system performance in
the simulated environment, even though the individual ignition system should have been tested in
these conditions at the component level.

Background/Rationale: Hot‐fire verification of the ignition system includes demonstration that adequate
energy is produced to ignite each and all combustion devices reliably, under nominal, off‐nominal, and
worst‐case conditions for propellant conditions (including mixture ratio, temperature, and pressure),
input voltage (if electronic excitation is required), and hardware environments (or equivalently,
chilldown conditions). Worst‐case propellant conditions typically occur when a minimum amount of
oxidizer (low mixture ratio) is present at the igniter (typically minimum oxidizer net positive suction
pressure (NPSP) for cryogenic propellants). Very low pressure and cold propellants generally require
more available energy to ignite, so vacuum testing under conditioned environments are recommended
for upper stage engines. Extreme hardware environments may exist, particularly for upper stage engines
with multiple restarts required. Cold propellants at high pressure may also be difficult to ignite since
more exciter power is required to produce a spark (i.e., Paschens Law). Other factors that may affect the
amount, density, quality, or mixture ratio of propellants in the vicinity of the ignition location
(e.g., purges, known potential leaks) must be considered and tested appropriately. For hypergolic
ignition systems, the quantity of the hypergol delivered, its associated timing (e.g., characteristic fill
times, valve actuation times), and potential impacts on propellant delivery in the vicinity of the ignition
location (e.g., purges, known potential leaks) must be considered and tested appropriately. Tests for
worst‐case conditions should include margin and a minimum of four unique engine hardware samples.
4.2.8 Turbomachinery Performance
Analytical models for steady state performance utilize turbomachinery maps to predict engine
performance. These maps are predicted analytically and need validation with testing. Ideally, the testing
would span the ranges of expected 3‐sigma operating band with some excursions outside of that range.
Consideration must be given to performance over time in a single burn as well as over the engine’s life
expectancy. Tendency is for the engine to be concerned with turbine and pump efficiencies and net
positive suction head (NPSH) performance. Efficiencies will drive the need for turbine drive power, which
can translate into system performance losses (inefficiencies). Turbine flow area can be a parameter which
carries uncertainty that can affect engine performance and prediction capability. Vehicle systems like to
28
drive tank weight down by minimizing tank operating pressures, but a lower limit must be established as
turbomachinery will require a minimum NPSH to avoid pump cavitation, which can damage and fail
hardware. The secondary flows (bearing coolant, fluid film bearing flow) also need validation as can affect
performance, functionality, and margins in the system.
4.2.8.1 Component
Depending upon engine configuration, which drives complexity for the test facility, a full‐scale
turbomachinery test may be conducted to obtain much of the mapping information. This facility will
have some method to drive the turbine and control speed that may not match the turbine drive on the
engine. The pump may or may not pump same propellant as engine. However, it is preferred to match
drive gas and pumping fluid to engine as much a possible.
The full‐scale testing would be on top of the typical characterization of turbine and pump performance
at a subscale in an airflow test and water flow test, respectively. These types of lower level tests occur
to validate turbine flowpath and pump inducer performance.
4.2.8.2 Engine
Verify acceptable pump and turbine performance for all operating conditions. Any evidence of
significant pump cavitation must be assessed and appropriately tested to verify robustness against the
conditions. Map pump and turbine efficiencies for use in validation of analytical models. Loss of
efficiency may be related to bearing and/or seal wear, and seal leakage.
4.2.9 Combustion Device Performance
Verify combustion device performance under nominal, off‐nominal, and worst‐case conditions for
propellant conditions (including mixture ratio, temperature, and pressure), chilldown conditions, input
voltage (if electronic excitation is utilized), and hardware temperatures.
4.2.9.1 Component
At the component level, subscale testing is typical to characterize combustion efficiency and injector
delta pressure. If possible, there tends to be workhorse calorimeter spool pieces to simulate a
combustion chamber to also estimate heat loads/fluxes for MCC designs. All of this is to reduce risk to
the engine test program.
Combustion Stability: Either here, on a full‐scale component test or engine test, verify combustion
stability in combustion devices per the requirements of CPIA Publication 655.
Nozzle Sideloads: Subscale nozzle sideload testing should be consider for high area ratio designs,
particularly if those nozzles will be tested in an over‐expanded condition during ground testing. Further
validation should occur on full‐scale component and/or engine test.
Turbine Drive: These systems tend to manifest itself in a PB or GG. The purpose is the same, but engine
cycles and operating conditions tend to separate them. In addition to the conditions mentioned above,
the turbines tend to be sensitive to temperature distribution at the inlet, which can drive design
features into both PB and GG. These are added requirements to be validated and tested at a
component level prior to engine test.
4.2.9.2 Engine
Most of the performance characterization will occur on the engine as it couples the self‐induced
environments to combustion devices. Transient priming and purging characteristics should be explored
thoroughly (include nozzle sideloads). Gas‐ingestion testing falls in this category as it has historically
affected combustion stability and performance (both steady state and transient), particularly in
pressure‐fed systems. Depending upon component test program, bomb testing may have to occur on
the engine program. Coolant flow heat loads/fluxes verified here primarily. All operational power levels
including expected maximum variations should be explored and characterized. To name a few
characteristics that are analytically‐modeled and need validation/anchoring to match steady state
29
performance predictions to test and flight results include combustion efficiency, injector delta pressure,
and coolant channel heat load characterization.
Ablative Nozzle: Develop correlations for predictions and corrections to flight thrust and Isp for engines
that are not acceptance tested with the flight nozzle. Ablative chamber char and throat area erosion
characteristics should be included.
Background/Rationale: If the engine is not ground tested with the flight nozzle, then the measured
thrust and Isp must be adjusted for the absence of the nozzle or for the presence of a slave nozzle.
Slave nozzles include any attachments to the exit area of the MCC that provide additional performance,
such as water spray rings or witness rings (WRs), attached or not attached. Engine thrust and Isp with a
slave nozzle will differ from that which would be achieved with the flight nozzle.
4.3 LIFE
4.3.1 Operational Life/Durability
As a part of qualification of an engine for flight, it is recommended to show margin to maximum expected
operational life. This is one aspect of demonstrating hardware robustness. The intension is to show this
on multiple engines throughout the DDT&E program to build confidence in manufacturing processes and
build‐to‐build variation. Some programs have used a fleet leader program to ensure the part that flies is
well within the bounds of the experience base. Life limited parts are expected and can drive engine
maintenance intervals and total life.
4.3.1.1 Component
Component testing is expected to qualify several of the components for use on the engine. One of the
tests in the component qualification programs tends to be life (ref. MIL‐STD‐1540).
4.3.1.2 Engine
Testing should demonstrate margin on maximum specified operating life. Multiple engines are
required to verify manufacturing process stability and compliance with specified engine‐to‐engine
performance and operability variability limits. The margin should be between 2x and 4x (4x more
properly accounts for scatter in high cycle fatigue related failure modes); test conditions should include
specific thrust, mixture ratio, inlet box, and other flight conditions. Development of test margins may be
larger to provide confidence in qualification success.
Ablative Nozzle Life: Show 20% life margin (i.e., 1.2x) via test and/or analysis for all nozzle locations.
Verify acceptable erosion rates and uniformity. Verify acceptable structural interface and liner interface
temperature margins at the end of the extended duration, including during the soak‐back heating at
shutdown. If a WR or slave nozzle is to be used during acceptance testing to characterize the injector
environment for flight units, establish correlations of erosion characteristics between the WR and/or
slave nozzle versus the full‐scale flight nozzle, with sufficient full and extended duration testing to
determine engine‐to‐engine and nozzle‐to‐nozzle variability effects. Verification of nozzle erosion
acceptability should include post‐test dissection inspections.
Background/Rationale: For ablative nozzles, the primary concern is erosion and char depth margin and
not fatigue. Experience with ablatives has shown that a life demonstration factor (LDF) of 1.2x is
adequate to cover variations in erosion characteristics.
4.3.2 Single Burn Endurance Test
Intent is to qualify hardware to maximum expected mission single burn duration with margin. Other than
being able to repeatably show tolerance to several successive firings as expressed in the operational
life/durability tests, this type of test shows that the engine does not have issues meeting maximum burn
durations. Issues relating to HCF or self‐induced environments can be exposed and life limiting
components can result.
30
4.3.2.1 Component
Component testing can be used as part of component qualification testing.
4.3.2.2 Engine
Test the greater of a 10% burn time duration margin beyond the maximum expected mission duration
on a single continuous test or the design specification stated value. Include flight representative profiles
in thrust and mixture ratio. For multi‐burn applications, tests must demonstrate margin on cumulative
mission duration in a series of single burn continuous tests.
4.3.3 Life, Engine Starts
Engine starts are a significant hardware durability driver and a major component to life calculations.
Starting and stopping the engine tends to put significant thermal and pressure gradients across the engine
in a very short period of time, stressing hardware to the limits. Rocket engine life is typically quoted in
starts and seconds (run time). Engine testing will push the limits on number of starts because the cost of
hardware drives to use it as many times as possible before refurbishment.
4.3.3.1 Component
RESERVED
Note: Component qualifications should stress the hardware cycles.
4.3.3.2 Engine
Conduct testing to demonstrate margin relative to the specified service life requirement, including
multi‐burn applications. Fleet leader tends to be at least 2x the operational life. Qualification
demonstrations are out to 4x. Pressure‐fed engines are qualified to 2x operational life.
4.3.4 Acceptance Test Procedure Validation
Intent is to establish the ATPs to be used for acceptance test of flight units. Verify the performance data
slice provides stable and consistent engine operation at that time in the test sequence. Verify
performance meets the requirement for integration into the flight vehicle. Include vacuum testing if
needed. Mission profiles should be performed if specified by program requirements.
4.3.4.1 Component
Component level ATPs are derived for a manufacturing screen and to assure product readiness for an
engine build.
4.3.4.2 Engine
These tests should validate the processes and procedures that will be used during production level
acceptance of engines. Burn times should be sufficient enough to screen for infant mortality issues and
flex the engine enough to ensure that it is within qualification family with sufficient life remaining for
possible stage ground test and flight with margin.
4.4 FUNCTIONAL CHARACTERISTICS
4.4.1 Cold Shock Tests
Facility propellant systems and test article hardware should be exposed to propellants during intial
activation of the test facility and the test article, especially if the propellants are cryogenic. The facility
and test article will be examined for leaks and thermal distortion, and inspected for material compatibility
issues in test article hardware following the tests.
Background/Rationale: Cold shock is normally performed upon initial integration of the test facility and
first test article or two, as risk reduction prior to cold flow tests and hot‐fire testing. Most if not all, new
cryogenic propellant rocket engines should go through a cold shock test.
31
4.4.2 Cold Flow Tests
Flow test the facility prior to test article integration for validation of facility capability. This also
characterizes pressure drops, resistances, temperatures, and flow rates through the feed system.
Turbomachinery places restrictions on this test as spinning the turbomachinery may not be desirable.
Often these tests are performed only on the initial activation of a test facility or on the first test unit.
Background/Rationale: Cold flow tests should be performed early in the development test program to
characterize flow rates and feed system pressure drops, and to verify model results and predictions as risk
reduction prior to hot‐fire. This is especially important on the first test unit for which uncertainty in
system flow resistances may otherwise pose unreasonable risk to proceed to hot‐fire testing. Cold flow
tests may also be performed prior to the first test on each subsequent test series to verify system
integrity and readiness. Combustion devices may be removed and replaced by equivalent simulated
resistance if beneficial to the test implementation. Test results should be compared to predictions to
verify resistances, pressure drops, temperatures and flow rates are within expectations and design
requirements. Safety precautions, such as flowing only a single propellant at a time and abort system
modifications, should be employed to avoid unwanted mixing and combustion of propellants. Also, the
use of simulated propellants, such as the use of liquid nitrogen (LN2) for LOX, may be employed for safety
and/or cost reasons. Cold flow with an integrated engine including turbomachinery and regenerative‐
cooled combustion devices may be difficult to test and results may not match intended performance; in
these cases it may be acceptable to cold flow the large components (valves, injectors) rather than the
integrated rocket engine.
4.4.3 Engine Propellant Inlet Conditions
Through each phase of operation, it is important to characterize the propellant conditions expected. Pre‐
start, it is important to thermally condition the engine hardware to minimize thermal shock that could be
detrimental to hardware durability. Start transient is sensitive to propellant conditions due to NPSH
sensitivity of the turbomachinery, where severe enough cavitation can cause a failure to start and damage
hardware. This carries over to steady state where inlet conditions can vary over time due to stage
propellant control systems and heat load into the tank. At the end of burn, shutdown propellant
conditions can vary from the dominant part of the burn due to tank heat loads, ullage temperature, and
pressure control bands.
Testing must verify that any engine system will have proper and reliable operation for all propellant
conditions that might be supplied by the vehicle at the engine inlet. Most vehicles will have a specification
or interface control document that explicitly describes the expected and allowable propellant inlet
condition boxes that the vehicle propellant feed system must deliver to the inlet of the engine. Such
propellant inlet boxes are typically defined as a region within prescribed temperature and total pressure
boundaries. Generally there is one set of inlet boxes for engine start conditions and another set for steady
state engine operation. If the engine to be used on the vehicle will run at multiple power levels, then
there may be different steady state inlet boxes for the various power levels. As an example, Figure 4.4‐1
shows propellant inlet boxes for a LOX and liquid hydrogen (LH2) engine. Vehicle feed systems should be
replicated as closely as possible during propellant inlet condition testing.

32

Figure 4.4‐1: Notional inlet boxes for a liquid oxygen/liquid hydrogen engine.
4.4.3.1 Prestart Chilldown
Prior to start, the engine should be bled and chilled (cryogenic engine) to eliminate or minimize vapor,
thermally condition the hardware, and achieve the specified engine inlet temperature and pressure.
Cryogenics cause significant thermal loads that could be detrimental to hardware if not properly
thermally conditioned before operation.
4.4.3.1.1 Component
RESERVED
4.4.3.1.2 Engine
Verify the chilldown or conditioning procedure to be used during the launch preparation or mission.
Characterize chilldown system operation (flow rates, temperatures, valve operations) and hardware
thermal condition variability (temperature versus flow rate versus time). The minimum, nominal, and
maximum chilldown pressures, temperatures, and flow rates should be tested. Minimum and
maximum chilldown should be tested in combination with hardware at the maximum and minimum
predicted temperatures, respectively. Verify hardware temperatures meet requirements prior to
33
engine start. Demonstrate chilldown margin, especially for upper stage engines started in flight.
Include vacuum testing as appropriate to simulate the flight environment. Verify requirements are
met for fluid consumption rates. Verify hold durations and allowable chilldown duration, if applicable.
4.4.3.2 Start Propellant Inlet Conditions
These conditions exist immediately prior to start command. Chilldown covers the conditions when the
hardware is thermally conditioned; however, there is a small window that holds conditions in the “start
box” where the tanks are pressurized to run pressures. The start box tends to reside within the “run
box,” which is related to steady state propellant conditions. These conditions take into consideration
any propellant feed pressure “slump” (from feed system inertance and resistance) and possible
variation in local temperature conditions due to low circulation flow rates and feedline heat loads.
Consideration must also be given to the various ancillary systems and their relative operating boxes as
these effects are not trivial.
4.4.3.2.1 Component
RESERVED
4.4.3.2.2 Engine
Reference section 4.4.4.1, Start Transient, for testing expectations.
4.4.3.3 Steady State Propellant Inlet Conditions
Run boxes are to be established based on vehicle and engine configuration and documented in the
engine/vehicle ICD. Reference section 4.2, Performance.
4.4.3.3.1 Component
RESERVED
4.4.3.3.2 Engine
Reference section 4.2.1, Steady State Performance Characterization, and section 4.2.4 Engine
Influence Coefficients, for more details on testing expectations.
4.4.3.4 Shutdown Propellant Inlet Conditions
Engine performance and overall vehicle performance is very sensitive to MR. The durability of engine
components can be very sensitive to mixture ratio conditions and the associated environments. The
end‐of‐burn propellant density is more prone to vehicle tank heat loads and ullage temperature. This
introduces the MR issue. Further, the end‐of‐burn ancillary system conditions must be explored as valve
timing and purging/inerting of the engine can see variations. Particularly for upper stages, shutdown
transients are important for vehicle orbital positioning.
4.4.3.4.1 Component
RESERVED
4.4.3.4.2 Engine
Reference section 4.4.4.4, Shutdown Transient, for testing expectations.
4.4.4 Transient Characterization
Transients are critical to explore during the development of a rocket engine. Transients test hardware
durability, analytical modeling, and interactions between components/subsystems (i.e., system
interactions). Significant time should be spent characterizing the variations that could occur during any
given flight by varying propellant conditions (primary and secondary) and valve command timing.
4.4.4.1 Start Transient
Start transients on rocket engines are typically described as controlled explosions. Start transient
modeling is a significant portion of the hardware design criteria. This modeling must be physics‐based to
best represent the predicted hardware characteristics. Like the steady state performance models, the
34
testing in this area must also anchor the start transient model. Because of the complexities with the
time‐varying conditions of propellants and hardware condition, the models, in general, will not
represent the test data to the accuracy and precision that the steady state models will. Specifically in
this area, it is not unusual to see two independent models run in parallel. And because they are
physically based, model inputs include propellant and secondary/ancillary flow conditions and hardware
characteristics (resistances, pump maps, valve control volume (Cv) curves, valve slew rates, etc.), which
all need validation in parallel to providing output critical in the decision making process regarding how
to alter the commanded sequence to successfully start the engine, repeatably.
4.4.4.1.1 Component
Depending upon the component being tested and the complexity of the facility, the ability to mimic
the engine start transient can diminish. Obviously, it is desirable to duplicate where feasible, but not
mandatory. Typically, these lower level tests will have to derive a component specific transient, with
the tendency to minimize wear on the component. Objectives for this level of testing are not
necessarily tied to engine start transient verification.
4.4.4.1.2 Engine
This level of testing would dominate the start transient characterization. Timing is everything. Verify
that the final startup sequence produces a start transient that satisfies all requirements. Typical
approach is to break the start transient into logical steps, which will flow into a series of tests, prior to
obtaining mainstage operation with any substantial steady state duration. The logical steps are
dependent upon engine cycle and engine configuration. It is typical to see adjustments in valve slew
rates, orifices, and commanded sequence throughout these tests. Variations in propellant conditions
(both primary and secondary/ancillary), hardware thermal conditioning, electric power, valve
command timing, valve slew rates, orifices, and ambient conditions (if altitude start) must be
considered.
Tests for worst‐case conditions should include margin and a minimum of four unique engine hardware
samples. Margin values vary widely depending upon the actual engine cycle with criticality determined
from development testing and specification requirements. Vehicle and mission configurations, single
or multiple engines, are drivers for unique limits that must be demonstrated. Critical starting
conditions such as those discussed above must be demonstrated.
Characterization tasks include the following: Develop forcing functions for structural dynamics loads
analyses; Define/validate control valve timing; Establish purge system schemes, timing, and flow rates
(e.g., to mitigate potential reverse flow); Determine ignition overpressure (IOP) and structural loads
environment; Establish start commit criteria and launch commit criteria (LCC) (ground start engines
only).
Verification tasks include the following: Verify specification limits are met; Verify acceleration rates,
flow rates, propellant consumption, and side forces are acceptable; Validate initial spin‐up and/or
bootstrap method.
4.4.4.2 Restart
Depending upon the role for the engine, it is not out of the realm of possibility to have to restart an
engine during flight. J‐2 and RL10 are examples. Uncertainty in models is introduced the greater time
there is between initial burn cut off and restart. The concept of operations can make a significant
difference in the amount of testing required to anchor models.
4.4.4.2.1 Component
RESERVED
4.4.4.2.2 Engine
Restart or multiple restart capability should be validated with conditions as representative of the
predicted flight condition as possible. Depending upon the important restart conditions (including but
35
not limited to heat soak back, propellant settling, and propellant slosh), this testing may be more
appropriately reserved for higher level testing.
Because of variability that could be introduced by natural/external influences, any testing at this level
should include excursions to the worst‐case with margin (where possible). Ideally, flight would always
remain within test experience, or the extrapolation is minimized.
4.4.4.3 Throttle Transient
Demonstrate throttle rates and control capability to satisfy specification limits. Include all throttle points
and worst‐case control system response requirements. Simulate relevant external influences in flight
(e.g., rapid inlet pressure change due to rapid acceleration change) to the extent possible.
4.4.4.3.1 Component
RESERVED
Note: Can be explored and characterized somewhat on component stands.
4.4.4.3.2 Engine
Engine throttle transients should be characterized during engine system testing. Most of the
parameters influencing throttling are expected to be characterized as a part of normal start, steady
state, and shutdown characterization testing. Throttle transient characterization testing should be
incorparated into tests with other objectives to maximize development engine life.
Demonstrate throttle rates and control capability to satisfy specification limits. Include all throttle
points and worst‐case control system response requirements. Simulate relevant external influences in
flight (e.g., rapid inlet pressure change due to rapid acceleration change) to the extent possible.
4.4.4.4 Shutdown Transient
Shutdown transient must be repeatable as possible to provide guidance, navigation, and control the
best knowledge of where the vehicle exists in space relative to the expected/predicted flight trajectory.
Shutdown modeling must be physics‐based to best represent the predicted hardware characteristics.
Like the start transient and steady state performance models, the testing in this area must also anchor
the shutdown transient model. Because of the complexities with the time‐varying conditions of
propellants and hardware condition during burn, it is not expected that the model will represent the
test data to the accuracy and precision that the steady state model can provide. As with the start
transient model effort, it is not unusual to see two independent models run in parallel. Especially since
the start and shutdown transient models stem from the same modeling effort, and because they are
physically‐based, model inputs include propellant and secondary/ancillary flow conditions and hardware
characteristics (resistances, pump maps, valve Cv curves, valve slew rates, etc.), which all need
validation in parallel to providing output critical in the decision‐making process regarding how to alter
the commanded sequence to successfully shutdown the engine, repeatably.
4.4.4.4.1 Component
RESERVED
4.4.4.4.2 Engine
Establish (development testing) and characterize a baseline shutdown sequence. Demonstrate
shutdown from all potential power level conditions at nominal, off‐nominal, and worst‐case
conditions, including those that produce the slowest and fastest shutdown transient. Important engine
shutdown parameters to consider include impulse ranges and repeatability, propellant inlet
conditions, mixture ratio, repressurization flows, purges, and control system parameters (e.g., voltage,
pressure, valve timing). Realistic combinations of shutdown parameters that cannot be test verified
must be verified by analytical means. Tests for worst‐case conditions should include program‐specific
36
margins. The actual margin values are dependent on the type and application of engines being tested.
These values should be determined during development testing, be in compliance with specification
requirements, and should be tailored to the specific engine system being tested.
Verification tasks include the following: Verify specification limits are met; Verify deceleration rates,
flow rates, propellant consumption, shutdown impulses, impulse repeatability, and side forces are
acceptable; Validate spin‐down rates; Validate dynamic responses (e.g., chugging, “pops”) are within
allowable limits.
4.4.4.5 Abort Shutdown
Abort scenarios vary widely from vehicle to vehicle. Which stage the engine is on will also vary the
conditions an engine may see during an abort scenario. Complex systems tend to fail in complex ways,
therefore modeling all possible abort shutdown scenarios is rather difficult. Hazards and Failure Modes
and Effects Analysis (FMEA) documentation should be examined at all levels to best understand the
reasonable failure modes to test and evaluate. The test program should incorporate some testing of
abort scenarios to better anchor models that can be extrapolated to satisfy safety concerns.
4.4.4.5.1 Component
RESERVED
4.4.4.5.2 Engine
Verify the ability to safely abort and shutdown while on the pad. Launch site abort logic and shutdown
procedures should be used. Launch pad and launch vehicle interfaces should be simulated or
represented to the greatest extent possible. Demonstrate the planned post‐abort shutdown safing,
inspections, and turnaround activities.
4.4.5 NPSP Margin and Cavitation
NPSP is effectively the delta pressure between the total pressure minus the vapor pressure of the
propellant. For engines with turbomachinery, this is a key parameter to understand and thoroughly
characterize during development. Low inlet pressure to the pump can result in cavitation, and in turn
create head fall off and significantly degrade pump life. The tendency is to always maintain a minimum
amount of NPSP margin to prevent head fall off and hardware damage. The margin is analytically
predicted along with the turbo pump maps, which need to be anchored to test data, feeding into engine
steady state performance and transient models. It is possible to obtain much of the necessary information
off of component level testing, but this does not eliminate the need to conduct some of these excursions
on the full engine to validate the component level findings. Some programs have used “powerpack” type
configurations, which tends to include the turbomachinery and parts of the engine to test these items in
combination, prior to or in parallel with the full engine development test program.
4.4.5.1 Component
Test throughout the prescribed operating range to map structure, strength, and extent (operating
parameter range) of cavitation instabilities. The engine operating range should be parameterized and
mapped in terms of the dimensionless pump parameters which govern cavitation behavior, namely
cavitation number and flow coefficient, accounting for uncertainties derived from measurement
uncertainties, hardware variation, and ground‐to‐flight differences. Propellant inlet temperature,
pressure, and flow rate measurements for determination of cavitation number and flow coefficient
should be located as close as possible to flight measurement locations or corrected to flight
measurement locations. Inlet pressure measurements should be sufficiently far upstream to prevent
erroneous pressure readings that may be caused by local backflow near the inducer inlet. Quantify the
strength and oscillation frequencies associated with cavitation instabilities with pump housing axial and
radial accelerometers. If possible, include one or more close‐coupled dynamic pressure transducers at
the inducer inlet. Accelerometers and pressure transducers should have sufficient frequency response
37
and sample rate to resolve the frequencies associated with the highest frequency cavitation
instabilities. If instabilities are observed, a circumferential array of pressure transducers should be
employed to provide information on the structure and rotation rate of the cavitation disturbance.
Verify robustness and margin against any instability behavior that cannot be eliminated by
accumulating 4x duration versus worst‐case total SL duration. Worst‐case durations should account for
the often hysteretic behavior of cavitation instabilities. Test durations should be increased, or an
alternate test sample obtained, if cavitation response amplitudes on the intended test sample are well
below the maximum expected cavitation response amplitudes for all engines. Flight representative
feedline geometries should be employed and propellant cleanliness and dissolved gas content levels
should be representative of flight conditions.
Background/Rationale: Development and qualification test programs for LREs with turbomachinery
must include testing to demonstrate robustness of the engine design to the cavitation phenomena
commonly encountered in turbopumps. The inlet condition testing guidelines associated with cavitation
above are intended to primarily address engine performance degradation that prevents the engine from
meeting specified performance requirements. However, current turbopump designs typically employ
cavitating inducers to maximize cavitation performance. These inducers commonly experience various
forms of localized cavitation that are routinely present at operating conditions well inside the intended
flight operating range of propellant inlet conditions. Experience shows the localized cavitation regimes
can be very sensitive to small changes in propellant inlet conditions, hardware variations (e.g., inducer
tip clearances), and engine operating parameters (especially speed). While these localized cavitation
phenomena do not significantly impact pumping performance, they can lead to reduced system
reliability through the generation of elevated engine and vehicle vibration environments.
Localized turbopump inducer cavitation phenomena can take a variety of forms including partial blade
cavitation, tip vortex cavitation, backflow cavitation, surge cavitation, rotating cavitation, and higher
order surge and rotating cavitation. These complex cavitation phenomena produce both broadband and
discrete frequency excitation that can damage both turbopump and vehicle components. In addition,
they can provide the excitation and/or compliance necessary to initiate turbopump rotordynamic or
coupled propulsion/structure (pogo) instabilities.
A rigorous test program is absolutely essential for the identification and elimination of the potentially
harmful impacts of these phenomena on flight hardware. Initial characterization of inducer cavitation
performance and screening for cavitation instability behavior within the intended operating range
should be conducted in inducer and pump component tests, which will not be discussed here in detail.
However, it is important to consider that such tests are often conducted using subscale hardware at
scaled speeds using stimulant fluids (such as water) that may not adequately represent the true flight‐
operating environment. Even full scale turbopump component tests pumping flight propellants may not
identify all cavitation issues due to the variety and complexity of possible cavitation phenomena and the
potential for impact from unknown engine system effects. For these reasons, the engine development
test program should include a thorough mapping of the cavitation behavior of the pumps over an
operating range that envelops the intended flight operating range, including low NPSP and high and low
speed extremes.
Focused testing to map structure, strength, and extent (operating parameter range) of cavitation
instabilities over the prescribed operating range requires specific test procedures and careful selection
of instrumentation type, placement, and data acquisition strategy. To ensure that no potentially
damaging instability behavior is overlooked, the engine operating range should be parameterized and
mapped in terms of the dimensionless pump parameters which govern cavitation behavior, namely
cavitation number and flow coefficient. Propellant inlet temperature, pressure, and flow rate
measurements for determination of cavitation number and flow coefficient should be located as close
as possible to flight measurement locations or carefully corrected to flight measurement locations to
ensure test parameters are flight representative. Care must be taken to locate the inlet pressure
measurement sufficiently far upstream to prevent erroneous pressure readings that may be caused by
38
local backflow near the inducer inlet. Test ranges should be expanded sufficiently beyond the test range
of interest to account for cavitation number and flow coefficient uncertainties derived from
measurement uncertainties, hardware variation, and ground‐to‐flight differences. Particular interest
should be paid to temperature measurement uncertainty as cavitation number is acutely sensitive to
temperature, especially at low NPSP, due to the slope of propellant vapor pressure curves. In addition to
the instrumentation required to accurately establish the pump operating parameters, pump housing
axial and radial accelerometers are required, at a minimum, to quantify the strength and oscillation
frequencies associated with cavitation instabilities. Where possible, it is highly desirable to also include
one or more close‐coupled dynamic pressure transducers at the inducer inlet. These accelerometers and
pressure transducers should have sufficient frequency response and be sampled at a rate sufficient to
resolve the frequencies associated with the highest frequency cavitation instabilities. Higher‐order
cavitation modes have been observed at frequencies as high as 6 to 10 times shaft synchronous
frequency. In problematic hardware where additional information is required, a circumferential array of
pressure transducers can be employed to provide information on the structure and rotation rate of the
cavitation disturbance, which is useful in identification of the type of instability present. The objective of
cavitation instability mapping in development testing is the identification and elimination of cavitation
instabilities having frequencies that are coincident with fundamental engine, propulsion system, and
vehicle frequencies such as inducer blade resonant frequencies, propellant feedline resonant
frequencies, and vehicle mode frequencies.
While the goal of the development test program is to identify and eliminate potentially harmful
cavitation instabilities, it is typically not possible to eliminate all instability behavior from the engine
operating range. Consequently, qualification testing should verify margin by accumulating 4x duration
versus worst‐case total mission duration of operation for any remaining cavitation instabilities that
occur in the flight operating range of the engine. Worst‐case mission durations should be carefully
determined to account for the often hysteretic behavior of cavitation instabilities (the pressure at which
an instability is suppressed once initiated can be substantially different than that required for initiation).
Test durations may also need to be increased, or an alternate test sample obtained, if cavitation
response amplitudes on the intended test sample are well below the maximum expected cavitation
response amplitudes for all engines. Flight representative feedline geometries should be employed to
capture potential interactions between localized inducer cavitation and feedline dynamics. In particular,
a surge type of instability involves the dynamic characteristics of the overall feed system and so the
upstream facility feed system should be dynamically decoupled from the flight representative feedline
by using an isolation accumulator to create a dynamic pressure null at the accumulator position. In
addition, care should be taken to ensure that propellant cleanliness and dissolved gas content levels are
representative of flight conditions as these factors are known to impact cavitation behavior. Again,
proper selection of instrumentation type, placement, and data acquisition strategy is essential to ensure
adequate characterization of and qualification to flight cavitation environments.
4.4.5.2 Engine
If not obtained on the component test, the default becomes the engine test program. Powerpack test
configurations will fall in this category as a subsystem test program.
Demonstrate adequate steady state pump performance and acceptable head falloff characteristics
(i.e., no worse than 2‐3% degradation) with decreasing NPSP. Test margin outside of pump inlet box
and/or beyond the specified minimum NPSP requirement. A separate NPSP margin demonstration test
is recommended for each propellant to avoid risk of interactions and potential over test conditions. Test
durations at the minimum NPSP condition should be sufficiently long to collect steady state data. Also,
the minimum NPSP test point should be sufficiently low to collect data that explores the impact of
potential single point system failures (e.g., failed pressurization branch), but not lower than the 2% head
falloff region. Actual margin values should be the result of development test results and specification
requirements for the type of engine being tested.
39
4.4.6 Pogo and Compliance Characterization
Pogo is a fluid‐structural instability that can cause catastrophic loss of a vehicle. Characterization of
engine compliance by testing becomes an important part of anchoring the analytical models and helps
determine if a suppressor is required. Suppressors can be engine‐mounted or stage‐mounted. If engine‐
mounted, the majority of majority of engine development tests and all engine qualification testing should
be performed with the Pogo suppressor hardware installed and operating as designed during flight.
4.4.6.1 Component
RESERVED
Note: Turbopump compliance can be explored and characterized on component stands.
4.4.6.2 Engine
Engine testing is the one place in a development program that can offer an environment that generally
simulates vehicle provided inlet conditions. Determine pump compliance and pump gain, as a function
of cavitation index and flow coefficient at the pump inlet, to facilitate vehicle pogo modeling. Derive
parameters using pogo pulse testing and analytical modeling.
Verification of the pogo model of engine oscillatory behavior requires special engine development
testing to determine frequency response functions over the full range of engine operating conditions.
These frequency response functions express the amplitude and phase of engine inlet, pump discharge,
and Pc versus frequency per unit of sinusoidal flow oscillation inserted upstream of the engine inlet. This
testing is applied to each propellant circuit that remains liquid up to the MCC, such as LOX or a
kerosene‐like propellant. Testing for liquid hydrogen is not required because gasification occurs
upstream of the MCC.
Ideally, the overall test feed system should include a replication of the flight feed system or, as close
thereto as practical. The upstream facility system should be dynamically decoupled from the flight
representative feed system by using an isolation accumulator to create a dynamic pressure null at the
accumulator position. A pre‐test dynamic model of the overall test system should be created and the
design requirements for the facility accumulator determined.
Special pressure instrumentation should be included with the data acquisition ranged to accurately
determine small‐amplitude oscillations in the frequency range of interest. The pump inlet pressure
amplitude is intentionally kept below a few pounds per square inch (psi) to insure that its dynamic
response is linear with amplitude. The downstream pressure amplitudes are normally smaller than the
inlet pressure. After determining the frequency responses by test, parameters of the test system
dynamic model should be adjusted to best match the test data. The paramount parameters to be
verified are pump cavitation compliance and pump gain as a function of cavitation index and flow
coefficient at the pump inlet.
4.4.7 Ancillary Systems
Ancillary subsystems are secondary systems (mechanical, hydraulic, pneumatic, and electrical subsystems)
that make the engine function as it is intended. All functions are necessary in these systems and are no
less significant than the primary flowpath items. These systems tend to make up the majority of the
interfaces with the stage. Just as the primary propellants, there are operating boxes defined for each
ancillary system. Either coming from or sent to the stage, the ancillary systems influence performance and
functionality of the engine. Each ancillary system operational band must be characterized and models are
to be anchored through testing.
4.4.7.1 Autogenous Pressurization
Autogenous pressurization is when a small amount of the primary propellant is heated, expanded, and
then returned to the vehicle to be used for tank pressurization.
40
4.4.7.1.1 Component
RESERVED
4.4.7.1.2 Engine
Verify specification autogenous pressurization requirements. Determine influence on engine
operation. Test with margin on flow rates. Include nominal and worst‐case engine operating
conditions.
4.4.7.2 Valve Actuation
Variations in conditions for valve actuation directly affect repeatability of start and shutdown transient
characteristics. Anchor models to the family of test data gathered over the spread of the operational
boxes.
4.4.7.2.1 Component
RESERVED
4.4.7.2.2 Engine
Verify electrical, pneumatic, hydraulic operational box minimums and maximums do not significantly
affect engine or stage operation.
4.4.7.3 Purges
Purging ensures the engine remains clean prior to operation (prevents contaminants from entering),
and inerts the engine at the end of operation (expels remaining propellants). Proper sequencing (purge
sequence number or PSN) and flowrates are key to start and shutdown characteristics of the engine.
4.4.7.3.1 Component
RESERVED
4.4.7.3.2 Engine
Establish (development testing) appropriate purge schemes, flow rates, and timing, and validate
(qualification testing) purge effectiveness using dew point measurements or other appropriate means.
Verify effectiveness of purges at minimum, nominal, and maximum purge flow rates, temperatures
and pressures. Include operational sequences and procedures and abort situations.
4.4.7.4 Electrical Power and Ignition
Electrical power tends to be supplied by the stage/vehicle. These systems have power quality
requirements. Engine electrical systems include the engine controller, data systems, valve control, and
likely ignition systems. Variations in power quality can have detrimental effects to engine functionality.
4.4.7.4.1 Component
RESERVED
Note: Many times this functionality is checked at the lowest levels and several steps along the way.
4.4.7.4.2 Engine
Test electronics from utilizing end‐to‐end checkouts, valve sequence checkouts, etc., prior to hot‐fire.
Systems should run thorough checkouts over the full range of interface power conditions.
Ignition systems should also get timing and sequence checks relative to power variations.
4.4.8 Thrust Vector and Gimballing
Testing should verify capability of the integrated engine system and the thrust vector control (TVC)
system. TVC includes engine gimbal and roll control systems, or differential throttling in non‐conventional
configurations such as aerospike engines. Vehicle interfaces should be simulated as close as possible to
the flight design including the stiffness at attach points. If vehicle structural components (e.g., heat
41
shields, boots, or a boat tail) are significant in defining the interfaces or clearances, then those
components should also be included or simulated. Testing should be performed at engine firing and non‐
firing conditions.
4.4.8.1 Component
RESERVED
4.4.8.2 Engine
Verify that the TVC functional capability meets system requirements including maximum control range,
slew rate, acceleration, loads, and frequency response. Firing tests should be performed at worst‐case
combination propellant conditions for TVC loading, and at minimum and maximum thrust levels.
Gimbal Limits: Engine or chamber (as appropriate) should be gimbaled to and functionally operated at
its limit positions, slew rates, and accelerations during hot‐fire testing, where possible. Non‐hot‐fire
testing should include functional and hardware clearance checks; however, maximum slew rates and
accelerations tend to be reserved for hot‐fire.
Roll Control Limits: Roll control should be functionally operated at its limit positions during non‐firing
testing and during hot‐fire testing.
Ambient Environment: TVC should be demonstrated at sea level and/or vacuum conditions (as
appropriate). If vacuum testing is required (lubrication systems and frictional characteristics may be
vacuum sensitive), but not practical at the engine level, then this may be verified at the component
level.
Inspections: Verify through real time observation and post‐test inspection that all thermal protection
shields, flexible boots, gimbal hardware and adjacent structures remain properly configured/intact
through all physical movement.
Heat Flux: Verify that the engine induced heat flux on adjacent surfaces during hot‐fire TVC testing is
within acceptable limits.
Clearance: Validate the engine envelope clearance analyses at TVC limit positions. The recommended
clearance under dynamic conditions and with flight conditions applied is 1.0 inches (25.4 mm). Smaller
clearances may be considered acceptable upon review of the uncertainties associated with the analysis
of the differences in the ground to flight clearances.
Interface Compatibility: Verify gimbal block and roll control interface compatibility for mechanical
(e.g., bearings), hydraulic and electrical connections. Minimum and maximum control parameters
should be demonstrated.
Thrust Vector Alignment: Verify thrust vector alignment characteristics (arc minutes and offset).
Tolerance stack‐ups of hardware play a significant role in repeatability. Some gimbal bearings have
adjustments to reduce offset. All should be verified on the engine stand utilizing thrust measurement
systems with lateral measurement capability.
4.5 CONTROLS
The engine control system acts as the nervous system. It sends commands throughout the engine and
communicates with the stage/vehicle. It also collects and distributes engine data as necessary. This
subsystem is a vital contributor to the engine as a whole.
4.5.1 Functional Tests
These tests tend to be run initially off of the engine. Component and Subsystem labs are set up to
integrate software and electronic hardware.
4.5.1.1 Component
RESERVED
42
Note: Thoroughly test component and subsystem in laboratory environment. Hardware‐in‐the‐loop type
labs are common approaches for flight software and electronic hardware integration.
4.5.1.2 Engine
Verify the engine control system can reliably and accurately satisfy specification requirements for
startup, steady state operation, throttling, and shutdown. This includes command and control type
activities and data transfer activities.
Verify data collection and transfer characteristics.
Verify engine response time characteristics regarding sending and receipt of command to actual physical
movement of fluid, mechanical, and electronic driven devices.
Verify vehicle commands through vehicle computer simulator.
4.5.2 Engine Control System Malfunction Logic Check
These tests explore the various ways the control system can fail. Standard flight computer systems
contain two channels and in‐flight switchover is a standard redundancy feature on engines.
4.5.2.1 Component
RESERVED
Note: Thoroughly test component and subsystem in laboratory environment. Hardware‐in‐the‐loop type
labs are common approaches for flight software and electronic hardware integration.
4.5.2.2 Engine
Demonstrate fault detection and accommodation for engine control system faults. Demonstrate
channel switchover during the start transient, steady state, throttling, and shutdown for control systems
with redundancy. Demonstrate acceptable engine, engine controller, and control hardware responses to
malfunctions
4.5.3 Engine Health Management (EHM) System Validation
The engine control system communicates with the vehicle, accepts commands, transmits data, directs
engine operational functions based on commands; provides engine closed‐loop control if so configured;
and provides condition monitoring data, and if also so configured, manages engine health.
4.5.3.1 Component
RESERVED
Thoroughly test component and subsystem in laboratory environment. Hardware‐in‐the‐loop type labs
are common approaches for flight software and electronic hardware integration.
Demonstrate and validate that the EHM system can correctly detect and identify potentially hazardous
conditions, especially selected critical vehicle system scenarios identified by the system FMEA.
Demonstrate and validate that the EHM system can inform the appropriate controller in time to
acceptable engine corrective action and recovery, without exposing the test hardware and facility to
excessive risk. Verify that the EHM can evaluate start commit criteria and LCC and respond accordingly,
if separate, and independent ground facility systems do not perform this function. All actions must be
completed in time to prevent unacceptable damage or degraded performance.
4.5.3.2 Engine
Demonstrations at the engine level are required prior to active control during flight.
43
4.6 OPERATIONS
4.6.1 PreTest Inspections and Checkouts
Pre‐test inspections and checkouts primarily refer to visual inspections and functional checks. Both the
facility and test article must perform these checklist items in preparation for test. Items like
understanding of hardware configuration going into test, condition of hardware, removal of
covers/closures where appropriate, review of hardware changed/altered since previous test, functionality
checks like cycling the valves or end‐to‐end electrical checks all fall into this area.
4.6.1.1 Component
RESERVED
4.6.1.2 Engine
Generate (development testing) and validate basic inspection procedures. Key aspects to incorporate
into or consider for the pre‐test inspections and checkouts follow.
Perform all specified general checkouts. These may include, but are not limited to, the following: visual
examination of configuration, visual inspections for damage, visual examination of fits and clearances,
verification of safety precautions/requirements, verification of facility system readiness, and
confirmation of adequate consumables.
Engine‐to‐facility interfaces should be inspected to verify compliance with dimensional and surface
finish requirements. If main propellant inlet covers are removed, the internal area should be inspected
to verify foreign object debris (FOD)‐free conditions. Interfaces associated with internal components
sensitive to moisture intrusion should be verified to be dry.
Verify that all test procedures to be implemented are applicable for the intended test objectives and the
correct version.
Verify all facility and engine software to be used are the correct versions with the correct inputs.
Relevant checksums should be performed.
Perform all specified functional checkouts. These may include, but are not limited to, the following:
manual or commanded operation/movement of mechanical systems, electrical integrity verification,
control system checks, abort system readiness, data system readiness, and turbopump torque checks.
4.6.2 Leakage
The intent is to ensure that the system will not leak beyond its specified limits during pre‐launch and
flight/operation. There are various ways to perform leak tests and there may be different approaches at
different locations across the engine depending upon hardware configuration and operating pressures.
Some locations may need mass spec; others may only need bubble soap. The SMC‐S‐016, MIL‐STD‐1522,
and MIL‐STD‐1540 also address leakage.
4.6.2.1 Component
RESERVED
4.6.2.2 Engine
Integrated enigne fluid leak checks should be performed. Leak tests performed during engine assembly
and at any joint that is disturbed (i.e., removal and replacement of a component) during testing or
integration to larger assemblies.
Perform leak tests at pressure levels and durations sufficient to identify unacceptable leaks pre‐ and
post‐ engine hot‐fire. Acceptable leakage rates should be based on analysis and test experience. If the
engine contains pneumatic and hydraulic systems, which are integrated with the stage, these systems
44
should be tested at an integrated level for full verification. Different methods for quantifying leakage
include mass spec, bubble soap, pressure decay (other methods also exist that may be acceptable).
Integrated pneumatic and hydraulic sub‐systems should be leak tested per MIL‐STD‐1522 or
MIL‐STD‐1540 for pressurized systems.

4.6.3 PostTest Inspections
Inspections after a test is conducted contain valuable data that will help explain or elaborate on any
unexpected data, hardware life/durability concerns, etc. Depending upon test objective, the set of
inspections conducted can vary during the DDT&E program. However, there will be a set of inspections
that will be “standardized” for production (acceptance test) to validate the condition of hardware and
show the engine is ready for flight. These must be rehearsed and demonstrated to be a sufficient
screening mechanism for flight preparation.
4.6.3.1 Component
RESERVED
4.6.3.2 Engine
Post‐test inspection requirements for individual tests are defined below. In general, the test program
should use the same requirements intended for the flight units during acceptance testing. Activities in
addition to the normal activity intended for flight units are acceptable if required for specific risk
mitigation for subsequent testing. But, if the normal activity for flight units is found to be deficient then
it must be improved and re‐validated.
Establish (development testing) and validate procedures to evaluate engine health prior to subsequent
processing. Requirements for post‐test inspections following individual hot‐fire tests generally are fairly
similar to pre‐test inspections. Differences should be documented and verified during the test program.
Establish (development testing) and validate appropriate periodic inspection schedules and methods
(e.g., visual, throat impressions) to evaluate post‐test condition of combustion chambers and nozzles
and to identify evidence of unacceptable hot spots or erosion. Verify acceptable hardware conditions
before and following the engine testing for operational life, MR excursion testing, and thrust/MR margin
demonstration. Develop acceptance criteria for allowable chamber cracks and leakage rates, if any.
Establish (development testing) and validate procedures to secure and safe the engine for subsequent
processing.
Establish (development testing) and validate (qualification testing) procedures for post‐test inspection
data review and evaluation.
4.6.3.2.1 Post‐Test Series Teardown
Conduct a detailed engine teardown and inspection following each test series. Teardown to the piece‐
part level. Inspect for any signs of distortion, damage, excessive wear, or any other unexpected
discrepancies. Identify design modifications, operating constraints, or other corrective actions
required.
Background/Rationale: A detailed engine teardown and inspection is recommended following each
test series. The extent of the teardown preferably would be thorough engine disassembly to piece part
level. Assemblies and piece parts should be carefully inspected for any signs of distortion, damage,
excessive wear, or any other unexpected discrepancies. Such review of both “expended” and reusable
hardware is necessary to identify areas with low margin, if any, so design modification, operating
constraints, or other corrective action can be taken. If any hardware is found to be in particularly good
condition and is shown to have adequate life remaining, it may be appropriate to re‐use that hardware
45
for other test engines, provided the reused hardware does not compromise any objectives in the
respective test series that need unique hardware samples.
4.6.4 Drying and Heated Purges
These purges are intended to minimize the moisture (water) in the system. It is especially a concern with
cryogenics and propellants expecting to operate below the freezing point of water. Icing of sense lines,
effectively blocking the pressure from being measured, has been seen in engine hot‐fire testing. Some
materials need attention to prevent swelling of seals or rusting.
After a cryogenic engine hot‐fire, it is typical to see drying and heated purges to warm hardware for
inspection and preparation for the next test. Heated purge temperatures have a maximum due to
materials, and should be considered for each hardware configuration.
4.6.4.1 Component
RESERVED
4.6.4.2 Engine
Establish (development testing) drying procedures and verify (qualification testing) ability to dry engine
per specification limits. Check dew point at purge exit points at designated times to verify requirements
are met.

4.6.5 LRU Demonstrations
LRUs are components that are interchangeable and do not pose a threat to the understood performance
of the engine. Typically, the line is drawn such that the component can be change after acceptance testing
without significantly altering performance. However, there are different stages in the operational flow for
engines that could create variations in the LRU list. Prior to acceptance test complete, it is feasible to
change a much larger set of componentry if a calibration hot‐fire will follow.
SSME tends to be the extreme in this area by allowing almost all the components to be interchangeable.
However, most components have to undergo engine level hot‐fire acceptance series, which creates a
need for “slave” engines to hot‐fired for sustainment of this need. The engine hardware and control
architecture also enable this capability.
Expendable engines tend to head down the path of minimizing LRUs and accepting full engines. This is not
to mean that there are not logical LRUs incorporated into these programs, but it is all to keep engine costs
down.
4.6.5.1 Component
RESERVED
4.6.5.2 Engine
Verify LRU replacement procedures. Verify acceptable operation with approved LRUs via LRU
replacements.
Demonstrate potential LRUs during development by comparing performance variation for potential
LRUs to relevant performance uncertainty requirements.
4.6.6 Reusability
The best example of a system being close to reusable is the SSME. The engine is reused, but tends to carry
significant operational cost for inspection and maintenance to ensure the components remain flight
worthy for human spaceflight.
4.6.6.1 Component
RESERVED
46
4.6.6.2 Engine
Demonstrate two sets of two complete flight sequence simulations to the extent possible at the engine
level. The first set should demonstrate the nominal turnaround timeline. The second set should
demonstrate the minimum required turnaround timeline. Test objectives include the following:
 Demonstrate engine system start, steady state operation, and shutdown. Perform a
complete flight sequence during engine operation.
 Verify turnaround capability. Verify engine system turnaround procedures. Operational
turnaround and countdown procedures should be used.
 Demonstrate any required between‐flights hardware or software modifications.
 Perform post‐flight checkout and inspections, required health monitoring data reviews, and
pre‐flight checkout. Access limitations and restrictions, including vertical and horizontal
access as applicable and confined spaces, should be simulated to reflect operational physical
constraints.
 Demonstrate engine system re‐start, steady state operation, and shutdown. Perform a
complete flight sequence during engine operation.
 Perform post‐flight checkout and inspections, required health monitoring and data reviews.
Engine system should successfully meet all criteria.
4.6.7 Operability
The term is utilized to describe the ability to conduct operational procedures in a timely and effective
manner. Minimizing the level of effort required to change LRUs when necessary, conduct pre‐and post‐
test checkouts, electrical and mechanical checkouts, etc. This also manifests itself in operational timeline
optimizations. It helps decide the time allowed to perform the multitude of operational tasks on an
engine.
4.6.7.1 Component
RESERVED
4.6.7.2 Engine
Establish (development testing) and verify (qualification testing) practical servicing procedures,
operational readiness capabilities, maintenance access requirements, availability and suitability of
alternate parts/processes. Access to the propulsion system for verification should be similar to actual
flight‐related operations and replicate the engine servicing environment expected during full
operational status. Engine‐related operability verification should be performed according to the
intended engine servicing configuration (e.g., mounted on the flight vehicle, or removed and replaced)
and orientation (e.g., vertical or horizontal) during flight‐related operations. Operability verification
demonstrations should include or replicate vehicle elements similar to those needed for battleship
testing, and use available resources, including on‐ground maintenance equipment and logistics support
infrastructure, as close as possible to actual launch operations.
4.6.8 Preflight Procedures and Flight Sequences
This includes the pre‐flight countdowns and hardware preparation and configuration for flight. Much of
the vehicle operations are remotely operated at this point. However, there may be some flight day or
near flight day activities that would mean personnel around the engine, or vehicle.
4.6.8.1 Component
RESERVED
47
4.6.8.2 Engine
Because DDT&E tends to have significant time before 1st flight, the procedures known at this point are
likely significantly lower fidelity than what will be used on flight day (if in existence). However, notional
procedures can be put in place based on known vehicle architecture and concept of operations. To the
extent possible, demonstrate/simulate preflight procedures and operational procedures. Interfaces
with vehicle simulated command and data systems should be part of the facility configuration.
Demonstrate the flight sequence of events. Thermal conditioning may be simulated if it is impractical to
simulate long soak periods under space environments. The sequence of events may be compressed
through elimination of steady state operation provided all critical sequences are tested.
4.7 ENVIRONMENTS
Natural and induced environments are key factors in the design of hardware. The analytical estimates for
loads and environments essentially design the engine until test data is obtained. To validate the model, the
inputs need to be anchored to test data. In this section, various environments to be considered for test are
discussed.
This section outlines the tests necessary to validate analytical models used to define induced environments
and to verify compatibility with natural environments under both operating and non‐operating conditions.
4.7.1 Thermal Environment
Thermal environments are initially modeled at various levels. Vehicle analysis flows down natural and
vehicle induced environments to its constituents. The engine level model takes these inputs and combines
them with the self‐induced thermal environments from expected engine operation. All phases of
operation are taken into account. Output of the engine model [typically finite element model (FEM) in
nature] will flow to component thermal models, where detailed design will take this into account.
Model validation will utilize test data from thermocouples, resistive temperature devices (RTDs), and skin
temperatures to convey the thermal profile of the engine.
4.7.1.1 Component
RESERVED
4.7.1.2 Engine
Verify the engine thermal environments. Extrapolate between engine ground test environments and in‐
flight environmental predictions. The greatest wealth of thermal data is to be gathered during this
campaign.
4.7.2 Climatic Tests
Testing must verify that engine is robust against or adequately protected against expected exposure to
salt, fog, sand, and dust, per the requirements of SMC‐S‐016.
Many times these requirements can be verified by similarity to designs in the past. Full climatic test
programs are not necessarily needed.
4.7.2.1 Component
RESERVED
4.7.2.2 Engine
Refer to MIL‐STD‐1540 or Aerospace Report TOR‐2004(8583)‐1 for more detailed testing requirements
for engine hardware.
4.7.3 Vibration/Shock/Acoustics
Space launch vehicles experience severe vibration environments during liftoff, atmospheric ascent, and
space flight that can impose substantial dynamic loads on vehicle components and payloads. LREs also
48
experience self‐induced vibration environments inherent to the high‐speed high power turbomachinery,
high flow rate fluids, and complex combustion devices utilized.
4.7.3.1 Component
RESERVED
4.7.3.2 Engine
External Excitation: Refer to SMC‐S‐016 and MIL‐STD‐1540 for test requirements. External excitation is
not required if it can be shown that engine self‐induced vibration is much more severe. Following
external excitation vibration testing, it would be appropriate to functional test and hot‐fire the rocket
engine in a flight simulation to identify any hidden failure modes.
Background/Rationale: Traditional methods can be used to qualify engine hardware for external
environments induced during non‐operating phases of flight (e.g., boost phase loads on an upper stage
engine). The appropriate test requirements are described in SMC‐S‐016 and MIL‐STD‐1540.
Qualification of vehicle components or subsystems for external excitation environments is typically
achieved by affixing the test article to a shaker table and exposing it to a pre‐determined dynamic
environment that replicates or exceeds the expected flight environment. Predicted environments
usually are derived from flight data. They are simulated by an equivalent base‐shake input, generally
stationary random or sine dwell excitation, that induces dynamic loads in the component at frequencies
and amplitudes consistent with expected operational loads. In this manner, the test article is subjected
to fatigue damage potential equivalent to that experienced by the flight hardware. The duration of
exposure required to adequately demonstrate durability is determined based on margin requirements
and SL (including acceptance testing and flight duration).
Self‐Induced Vibration: The engine turbopump and other critical components should be instrumented
with accelerometers to characterize the dynamic environment during the development and
qualification program. The resulting accelerometer test data represent the qualified environment for
self‐induced engine vibration.
Background/Rationale: Self‐induced environments for LREs can be quite severe and are highly complex,
often including a large number of discrete frequency and narrow band random excitations spanning a
wide frequency range. Typical sources of self‐induced dynamic excitation include rotating imbalance,
pump and turbine blade passage, gear train and bearing dynamics, cavity flows, unsteady cavitation
phenomena, fluid‐structure interaction, and combustion instabilities. The complexity of engine systems
makes the dynamic environments to which engine hardware are exposed impossible to replicate in
traditional shaker‐type testing. Self‐induced dynamic excitations from multiple sources within a running
engine, each subject to variations in transmissibility, combine in complex ways to produce frequency
and location‐dependent loads throughout the engine system. Thus, identification of an equivalent base‐
shake input to reproduce operational loads in the components is not possible, and even if the
appropriate input could be determined, introducing vibration at a single point cannot accurately
reproduce the complex dynamic loads imposed on components in a running engine. Furthermore, since
qualification for self‐induced vibration by analysis typically requires complexity beyond current state‐of‐
the‐art capabilities, direct demonstration by engine level testing is left as the only means to qualify
engine component durability. Since the nature of the environments in conjunction with the size, cost,
and complexity of LREs makes vibration test qualification of engines by traditional methods impractical,
an engine‐unique methodology is required to qualify and ultimately accept engine hardware for the
self‐induced vibration environments.
Current engine development and qualification testing practices rarely adequately address the test
requirements for the complex self‐induced engine dynamic environments. Also, engine specification
limits for self‐induced environments are rarely defined, largely due to the inability of state‐of‐the‐art
analyses to predict these environments. Furthermore, there often is significant engine‐to‐engine
variability in various aspects of the self‐induced engine environments. As a result, engines and engine
49
components can be subjected to higher dynamic loads during acceptance testing than during
development and qualification testing, thereby introducing the potential to severely degrade life and
consequently posing an often‐unquantifiable level of risk to mission success. When these higher
environments and the resultant higher dynamic loads occur during acceptance testing, an assessment is
required to evaluate the health of engine components prior to flight and, if possible, identify potential
mission risks.
To facilitate clearance of self‐induced vibration encountered during acceptance testing, the durability of
engine hardware should be demonstrated via extended duration engine operation under flight
representative operating conditions during development and qualification testing. The engine
turbopump and other critical components should be well instrumented with accelerometers to
characterize the dynamic environment. The engine should be operated over a wide range of conditions
to envelope all inlet and mixture ratio excursions that will be encountered by the engine in service.
Post‐test inspections and functional tests are performed to verify proper component function and to
establish that no damage has been sustained during testing. The resulting accelerometer test data then
represents the qualified environment for self‐induced engine vibration and, as such, serves as a
comparison case to establish engine vibratory health for future production engines during acceptance
testing.
4.7.4 Modal Surveys/Testing
This is done to characterize the frequency response characteristics of the system by exciting the entire
engine system with a low frequency sine sweep. Extract engine modes and identify resonant excitation.
4.7.4.1 Component
RESERVED
4.7.4.2 Engine
A modal testing can be done for engine specific characterization only.

4.7.5 Vehicle Interface Loads
Vehicle‐to‐engine interfaces have a combined fluid‐structural response characteristics. Significant
structural loading is transferred from the engine to the stage/vehicle at the primary thrust take‐out points
(i.e., through gimbal bearing or other thrust structure). At the other interfaces there can be fluid loading
from propellant flow, secondary structural loading, and mechanical loading from gimballing. All these
conditions are to be taken into account for engine models and then flowed to component level designers
for detailed design. These models need anchoring from engine testing for engine generated loadings.
4.7.5.1 Component
RESERVED
4.7.5.2 Engine
Verify interface environments are within the Maximum Predicted Environments (MPE). Characterize
frequencies and amplitudes at the interfaces and validate engine testing encompasses expected vehicle
interface loading and dynamics.
4.7.6 Electromagnetic Interference/Compatibility Tests
Electronics are becoming more critical on flight systems as dependency on these items to control and
manage engine functionality increases (engine health and status, valve controls, data, etc.). These systems
must meet EMC limits as described in Aerospace Report No. TOR‐2005(8583)‐1 (proposed MIL‐STD‐
1541B), as well as additional documents referenced therein, subject to the tailoring below.
The above requirements must be tailored for a LRE. For example, only the electrical components are
usually subject to EMC requirements. In addition, ordnance, deployment and fuel systems must be
50
protected from, or made impervious to, inadvertent activation, due to high electric field strengths,
electrostatic discharge (ESD) and lightning. The following specific initial tailoring of
TOR‐2005(8583)‐1 is recommended for electrical equipment for launches:
(a) Requirement 6.15(RE102): Include any expected receiver notches not already called out in the
TOR.
(b) Requirement 6.16(RS103): The levels in bold face type should be adequate for a starting
point, but it may be beneficial to identify the possibility that they could go up.
(c) Requirement 6.12(CS06): Increase amplitude to 800V, and apply to all power and signal lines
(to address lightning indirect effects).
(d) Requirement 6.43c2(ESD): Contact discharge increased to 16kV, and applied to the pins of all
power and signal lines.
(e) Requirements should be made more stringent if necessary to verify acceptability for the
worst‐case inclement weather allowable for the launch. In addition, fuel lines need to have
low enough resistivity and adequate bonding to surrounding structures to avoid surface
electrostatic charging and discharge. Guidance may be obtained in Requirements 6.42 and
7.7.
4.7.6.1 Component
RESERVED
Note: Reference Aerospace Report Number TOR‐2005(8583)‐1 (proposed MIL‐STD‐1541B) for EMC
testing.
4.7.6.2 Engine
Not Applicable.
4.8 DESIGN AND CONSTRUCTION
4.8.1 Proof Pressure
Proof testing is not performed on engines due to the large pressure gradients throughout the engine
during operation. Typically, this is reserved for the component level as a part of manufacturing and
assembly. Refer to NASA‐STD‐5012 for additional detail regarding proof pressure testing and burst factor
demonstrations.
4.8.1.1 Component
RESERVED
Note: Pneumatic and Hydraulic Sub‐systems ‐ Integrated pneumatic and hydraulic sub‐systems should
be proof tested per the more stringent of MIL‐STD‐1522 or MIL‐STD‐1540 for pressurized systems.
4.8.1.2 Engine
Not Applicable
4.8.2 FOD / DOD Tolerance
Foreign and Domestic Object Debris (FOD/DOD) can either be self generated or come from the vehicle
and/or facility. It can be introduced as residual after cleaning processes, flowing propellants, general
operations (functional check‐outs like valve cycling), or through open interfaces during handling.
Prevention of FOD damage is accomplished by design features (clearances, material selections, filters,
etc.), or process control (cleaning, white rooms, protective covers, inspections). Analysis should be done
to identify the particulate size allowed through tightest clearances in the engine. Filters tend to be
incorporated where large sources of particulate or very small clearances are identified. Fluid cleanliness
and hardware cleanliness specification should be established for engine design. MSFC‐SPEC‐164 is a
reference typically used as there are MIL‐PRF documents, too.
51
4.8.2.1 Component
RESERVED
Note: Material characterization is used for oxygen compatibility.
4.8.2.2 Engine
Demonstrate tolerance to self‐generated particles due to wear (e.g., gear teeth, turbine seals). These
will naturally exist within the system during engine operation. Inspections should be performed to find
any significant FOD/DOD. Special inspections may be necessary in development to ensure system
cleanliness is maintained.
Confidence is gained from running multiple engines well past the expected operational life. Testing and
operational procedures verify system maintenance procedures are adequate and manufacturing and
assembly processes deliver clean parts. It will also verify that propellant quality and in‐place filters are
adequate.
However, some testing has been done in the past regarding introduction of contamination (metallic
particulate and gases as appropriate).
4.8.3 Structural Model Verification
Extensive hot‐fire testing of each individual engine provides data regarding structural integrity and
durability of the engine hardware. Engine qualification programs involve life testing that validates
structural models which involves low‐cycle fatigue (LCF)/HCF estimates. Structural models report margins
and are anchored to test data. Refer to NASA‐STD‐5012 for structural requirements for LREs.
4.8.3.1 Component
RESERVED
4.8.3.2 Engine
Refer to NASA‐STD‐5012 for structural verification for LREs. Margins should be reported for all major
components and interfaces.
Include strain gauge measurements for the purpose of structural model verification.

4.8.4 Margin Testing
Margin testing is utilizing test as a method to demonstrate the robustness of the engine. It tests the
engine a percentage beyond the expected maximum and minimum conditions. It is not necessary to
demonstrate all conditions outside of the expected operational box, but selected ones where the engine
design shows sensitivity or where the actual corner of a box may be a condition where the engine
operates for some period of time. This part of the test program should be well supported by analytical
models. Since not all model scenarios can be run on a test stand, the test points will anchor the model in
each margin area so interpolation between the tested points and the analytical case is minimized.
Margin testing provides confidence in the robustness of the design when significant numbers of tests and
significant numbers of samples are not an option. To be effective, the areas to demonstrate margin must
be selected carefully. Emphasis should be put on structural model anchoring to show hardware durability
to LCF and HCF ‐ the idea being to result in positive margins for flight. Functionality extremes like
propellant conditions, NPSP, and ancillary systems like valve actuation method (pneumatics, hydraulics,
electronics) should be exercised to characterize any impacts to performance (possibly identify “the cliff”).
4.8.4.1 Component
RESERVED
52
Component level margin testing can come in lots of flavors since component qualification test programs
many times occur prior to engine test. Being sensitive to over‐test as expressed in R‐8099.
4.8.4.2 Engine
Demonstrate the engine system’s margin capability to operate at conditions, which exceeds the worst‐
case maximum and minimum operation points required by the engine specification. Both a
development/verification engine assembly and a qualification engine assembly should be tested during
their respective phases of the program.
Development/Verification Phase Margin Tests: A final development/verification engine configuration
should be subjected to tests to demonstrate engine robustness beyond the operating limits established
by the engine specification. Demonstration should be a series of tests at 2% increments up to failure or
110% of minimum and maximum operating limits whichever comes first.
Qualification Phase Margin Tests: One qualification engine configuration should be subjected to tests to
demonstrate engine robustness beyond the operating limits established by the engine specification.
Demonstration should be a series of tests at 2% increments up to the value established during the
development/verification phase.
Background/Rationale: Engine operating margin has historically shown many examples where
unplanned flight anomalies have occurred where propulsion system operation outside the specified
performance requirements has been able to salvage the mission. Knowing the margins that the engine is
capable of provides the flight controllers with expanded options for saving the mission.
Additionally, other applications of the engine may be considered where above nominal operation of the
engine is required.
4.8.5 Human Rating (further description in Appendix D)
Without trying to over constrain design solutions, the NPR 8705.2 attempts to impose requirements
which continue to “raise the bar,” following the “each program builds on the lessons from the last”
philosophy. Most, if not all, historical engines would not meet the NPR 8705.2, including SSME. However,
even for Constellation Program (CxP) that levied the document, each of the designs were brought forward
showing proposed design approaches based on interpretation of these requirements. Although 2‐fault
tolerance is levied, there is also design for minimum risk (DFMR) or fault avoidance available as a possible
path. It is not logical or necessary to impose 2‐fault tolerance in its truest form, or the vehicle would not
be able to leave the ground. But, it is justifiable to go through the process of rationalizing and justifying
the design approach taken.
The design must demonstrate these capabilities and redundancies. Margins, environmental, and life
testing all come into play when demonstrating robustness of design. Functional testing should include
exposing redundancies to demonstrate that it is, in fact, a redundancy (i.e., little or no loss in capability).
4.8.5.1 Component
RESERVED
4.8.5.2 Engine
Testing should demonstrate redundancy, fault tolerance, increased margins in thrust and mixture ratio,
and durability (life and structural). Testing should demonstrate durability and thrust margin for engine‐
out conditions. Characterize functionality at degraded level of functionality or loss of redundancy.
4.8.6 Hardware Discrepancy Tracking
Although this is not specifically testing, it is good to identify expectations regarding hardware
discrepancies and how they can play into hardware durability, data from testing, etc. This is an important
part of the DDT&E effort as it is likely to see several manufacturing issues of varying criticality during
development due to low production levels.
53
Identify and resolve hardware or operational discrepancies, anomalies, or deficiencies. These may include,
but are not limited to, leakage, cracks, material erosion/ discoloration, part deformation, unexplained
elevated vibration, and unusual operating characteristics.
Background/Rationale: During the course of testing and/or operational experience, various hardware or
operational discrepancies, anomalies, or deficiencies may be identified. These may include, but are not
limited to, cracks, material erosion/ discoloration, part deformation, unexplained elevated vibration, and
unusual operating characteristics. It is recommended that such items be eliminated by a proper design
modification since, at a minimum, reduced margins are indicated. However, if sufficient testing and/or
analysis can show the increased risk is acceptable, then a programmatic decision may be made to accept
such items as a normal expectation for flight hardware. In this case, special inspections and/or data
review are recommended to ensure that the condition remains within the qualification experience and
bounds of analysis/testing.
4.9 PHYSICAL
4.9.1 Gas Liquefaction Control
This testing refers to liquid air formation and control of detrimental amounts generated by cryogenic
systems. Typically, foam insulation techniques are used for vacuum jackets. However, liquid air can still
form in areas which can cause dripping onto electrical components, cryo‐pumping, and insulation
damage. The control and mitigation methods need testing to demonstrate durability, repair techniques,
etc.
4.9.1.1 Component
RESERVED
4.9.1.2 Engine
Demonstrate on multiple units (at least 2) the life and durability characteristics to prove out application
processes and design sensitivities. Demonstrate repair techniques (if foam and tape) and effectiveness.
4.9.2 External Icing
Ice formation on the external surface of the hardware can act like an insulator or vibration dampener.
Environmental testing should simulate vehicle environments to mimic the atmospheric moisture
conditions, pre‐start conditioning, etc. On J‐2, ice formed on the augmented spark igniter (ASI) line and
acted as a dampener for ground test. When in flight, the ice did not form and the line ruptured due to
vibration.
External iceing should be characterized in the normal course of testing.
4.9.3 Mass Properties
Engine weight, center of gravity (CG), and moments of inertia must be understood and translated to
vehicle controls and trajectory analysts. Expect some level of testing to characterize unit‐to‐unit weights
and how that translates to CG and moments of inertia (MOI). This is to include an understanding of the
associated uncertainties around these values and weight growth allowances.
4.9.3.1 Component
RESERVED
54
4.9.3.2 Engine
The engine assemblies should be weighed in the as‐tested configuration as part of the engine test
program; with the CG and MOI calculated.
4.10 UNIQUE REQUIREMENTS
4.10.1 System Unique Requirements
RESERVED
4.10.1.1 Booster
RESERVED
4.10.1.2 Upper Stage
RESERVED
4.10.1.3 Cryogenic
RESERVED
4.10.1.4 Hypergolic
RESERVED
4.10.2 New or Mission Unique Requirements
New requirements or mission unique requirements levied on the engine following completion of
qualification must be evaluated carefully to determine whether or not the existing qualification
experience adequately addresses and verifies satisfaction of the new requirements. If not, additional
testing and/or analyses should be performed to qualify the engine for the new requirements.
4.10.3 DeltaQualification Requirements
Qualification testing should be performed on the final design, manufacturing processes, procedures, and
acceptance program to be used for flight units. Any deviations following completion of qualification must
be justified and the system re‐qualified (i.e., “delta‐qualification”) via suitable testing or analyses, unless it
can be verified that change does not affect form, fit, or function. Changes applicable to delta‐qualification
include design changes, configuration changes, modified processes, new suppliers, and revised
procedures.

55
5 ACCEPTANCE TESTS
Acceptance tests are conducted to demonstrate the acceptability of each deliverable item to meet
performance specification and demonstrate acceptable workmanship in manufacturing. Acceptance testing
of a LRE includes hot‐fire testing to ensure the article is acceptable for delivery. The criticality and complexity
of LREs require that performance be calibrated and measured. Verified performance parameters include
thrust, mixture ratio, and flow rate (for derivation of Isp). In addition, selected critical operating conditions,
key interface conditions, and other specification requirements must be demonstrated. Verification of
workmanship is a critical goal of the acceptance hot‐fire test since discrepancies are not always detected
during the build process. The extreme thermal, pressure, and dynamic environments during engine operation
can cause failure if significant workmanship flaws exist. Acceptance testing is intended to stress screen
hardware items to nominal operational levels to precipitate incipient failures due to latent defects in parts,
processes, materials, and workmanship prior to flight.
Although there will be an acceptance series of tests conducted on hardware during the DDT&E phase, this
section is primarily to address the production phase and acceptance of hardware during the flight operational
phase.
Further, this section focuses upon the aspects which need to be acceptance tested and may have significant
references back to previous sections.
5.1 TEST TYPES AND RECOMMENDATIONS
Acceptance tests are conducted to demonstrate the acceptability of each deliverable flight item to meet
performance specification and demonstrate acceptable workmanship in manufacturing.
5.1.1 Component
Specific component acceptance test guidelines will be addressed in future editions.
Depending upon the component, a mix of functional and performance tests will be conducted prior to
delivering a component to the next higher level of assembly. Items like turbopumps and combustion
chambers tend to go through manufacturing checks like torque, leakage, and proof pressure checks as a
part of manufacture and ATPs. Items like valves and flexible ducts may undergo more thorough series of
functional checks.
Since the standard practice is to hot‐fire test an engine to calibrate and “green run” it for acceptance, the
components will receive a level of testing over and above the component ATP. This reduces the risk of a
failure due to manufacture and assembly. Engine test also hones the performance of each particular build
as a contributor to the entire engine. Turbomachinery might have differences in efficiencies from build to
build, or maybe coolant flows, thrust balancing, etc. Injectors may have some variation in performance
and efficiency, or maybe require some tuning for outer row biasing for coolant flow, etc.
Largely, there is an expectation (like on SSME) that flight propulsion system hardware be hot‐fire tested
prior to flight. However, there are instances where components may be allowed to bypass this expected
norm (LRU type situations). Refer to the section on LRUs for more information, but there may be another
level of scrutiny required as a part of acceptance to ensure the proper functionality of these components
coming directly from the vendor.
Table 5.1‐1 shows the recommendations for acceptance testing, which might share the same title as
development and qualification testing, but its contents have been altered to be more commensurate with
the production/operational phase.
Background/Rationale: Component acceptance testing at the bench level serves to reduce risk for engine
testing, but it often cannot adequately simulate the engine environments. Thus, in general, components
should at a minimum be hot‐fire tested at the engine level to adequately reduce flight risk. If a
56
component is a LRU, then that component may be removed and replaced by a new unit without requiring
re‐acceptance hot‐fire test of the engine with the new unit. However, if that engine is not to be hot‐fired
again and the replacement unit normally includes an engine hot‐fire test as a standard part of its
acceptance test, then the replacement unit should be subjected to a hot‐fire test on some other engine.
Individual components are excluded from engine hot‐fire test requirements only when they can undergo
equivalent unit‐level acceptance testing. Equivalent testing should consider all appropriate environments
such as temperature, vibration, pressure, vacuum, and chemical, and such testing should demonstrate
functionality and performance of the unit under conditions similar to those achieved in the engine
acceptance test firing and flight.
5.1.2 Engine
Acceptance testing should to occur at the integrated engine system level. Because of the combination of
complex componentry, significant dependency on functional timing, and strict performance bands, the
engine testing tends to be a logical step in acceptance of flight propulsion systems. Single engine
acceptance is more “regular” as clusters tend to be seen as part of integrated stage acceptance.
The acceptance series functionally checks critical systems and performance should be “tagged” for flight
predictions. The acceptance series test parameters and instrumentation should fall within the
development and qualification experience.
New engines should have a minimum of two tests and a minimum total hot‐fire duration equal to the full
mission burn duration. If full mission duration is not possible on the engine system due to facility
limitations, then key life critical components may be green run separately to full duration, followed by
shorter duration engine system testing. As an engine system accumulates successful ground and flight
history, starts/duration requirements may be reduced provided that performance‐ and workmanship‐
verification objectives are not compromised.
5.1.2.1 Engine (Pressure‐Fed)
Pressure‐fed engine testing should be in accordance with the above paragraph, with the exception of the
requirements for a minimum of two tests and a minimum total hot‐fire duration equal to the full mission
burn duration. Historical precedent for pressure‐fed engine design is to not accumulate a significant
portion of the required mission life requirement as part of acceptance test.

Table 5.1‐1: Engine Acceptance Tests Matrix
Component
Engine
Section Objective
5.2.1 Steady State Performance Characterization x
5.2 Performance
5.2.2 Repeatability x
5.3.1 Operational Life/Durability x
5.3 Life
5.3.2 Single Burn Endurance Test x
Propellant Conditions
5.4 Functional – Pre-start Chilldown x
5.4.1
Characteristics – Start Propellant Conditions x
– Steady State Propellant Conditions x
57
– Shutdown Propellant Conditions x

Transient Characterization
– Start Transient x
– Restart x
5.4.2
– Throttle Transient x
– Shutdown Transient x
– Abort Shutdown x
Ancillary Subsystems
– Autogenous Pressurization x
5.4.3 – Valve Actuation x
– Purges x
– Electrical Power and Ignition x
Thrust Vector and Gimballing
– Gimbal Limits x
– Roll Control Limits x
– Ambient Environment x
5.4.4 – Inspections x
– Heat Flux x
– Clearance x
– Interface Compatibility x
– Thrust Vector Alignment x
– Functional Tests x x
5.5 Controls – Engine Control System Malfunction Logic Check x
– Engine Health Management x x
5.6.1 Pre-test Inspections and Checkouts x
5.6.2 Leakage x x
5.6 Operations
5.6.3 Post-test Inspections x
5.6.4 Drying Purges x
Vibration/Shock/Acoustics
5.7
– – External Vibration
Environments
– Self-induced Vibration x
5.8.1 Proof Pressure x
5.8 Design &
5.8.2 Human Rating x
Construction
5.8.3 Hardware Discrepancy Tracking x x
Mass Properties
– Mass x x
5.9 Physical 5.9.1
– Center of Gravity x x
– Moments of Inertia x x
5.10 Unique
Requirements

5.2 PERFORMANCE
5.2.1 Steady State Performance Characterization
The objective is to understand the performance of each build relative to the experience base set in
development and qualification. Typically, as a part of this, each engine will require some level of tuning to
ensure the performance is within the desired operational parameters. Performance parameters (primarily
thrust, Isp, and MR) can vary over the flight regime depending upon propellant and hardware conditions.
Flight profile tends to be mimicked to a certain extent, but is not required.
5.2.1.1 Component
RESERVED
5.2.1.2 Engine
Tag performance and calibrate engine to desired thrust and mixture ratio and Heat Exchanger/Tap‐Off
parameter, if required, based on hardware build performance as demonstrated during acceptance
tests. A single hot‐fire to mission duration is desired within the series as a minimum. If orifice or valve
position changes are required, or some performance‐related change/procedure is conducted, then a
58
validation run may be required. “Paper” calibration should be permitted provided adequate history is
provided to keep calculated error within calibration limits.
5.2.2 Repeatability
The objective is to characterize the run‐to‐run performance variation of a particular build, which aids in
flight predictions.
5.2.2.1 Component
RESERVED
5.2.2.2 Engine
This requires multiple hot‐fire tests to show comparison to DDT&E and production experience.
5.3 LIFE
5.3.1 Operational Life/Durability
Employ life limits (starts and durations) based upon DDT&E experience. This must be balanced with other
test objectives that can draw out the numbers and durations of tests. The intent is to fly with the
maximum life margin to minimize sensitivity to build‐to‐build hardware durability variation.
This should not become a reason to eliminate acceptance testing, but rather it will help ensure sufficient
margin in the system to handle the proper number of cycles over the expected life of the hardware.
5.3.1.1 Component
RESERVED
5.3.1.2 Engine
Pre‐ and post‐test inspections are expected. The data gathered through these activities in conjunction
with hot‐fire test data should reinforce that the hardware is not degrading faster than rates
experienced and accepted during development and qualification.
5.3.2 Single Burn Endurance Test
Test objective is to demonstrate that each build can endure a mission duration prior to flight.
5.3.2.1 Component
RESERVED
5.3.2.2 Engine
A single mission duration hot‐fire should be part of the acceptance series. It is not necessary to explore
margins with respect to time unless this is an expected operating condition. Flight profile is not
necessarily required but, if not used, the profile should be deemed equivalent with regards to duration
and performance output expected in flight.

5.4 FUNCTIONAL CHARACTERISTICS
5.4.1 Propellant Conditions
Since acceptance series includes hot‐fires, there are opportunities to validate chilldown characteristics,
start and run propellant conditions, and shutdown propellant conditions. The idea is not to significantly
vary conditions, but rather promote consistency to understand build‐to‐build variations.
5.4.1.1 Component
RESERVED
59
5.4.1.2 Engine
Prestart Chilldown: Ensure proper functionality of chilldown systems. Flowrates and time should be
tracked as it pertains to knowing systems have been adequately thermally conditioned.
Start: Acceptance series should attempt to replicate vehicle start conditions.
Steady State: Acceptance series should attempt to replicate vehicle start conditions. Vary interface
pressure to show any performance variation within run box conditions. Have a portion of the test held
as steady as possible for performance tagging.
Shutdown: Production testing should maintain consistent shutdown conditions to enable build‐to‐build
and run‐to‐run variations to be exposed. Varying interface propellant pressure is reasonable to bound
shutdown characteristics expected in flight if vehicle is significantly sensitive.
5.4.2 Transient Characterization
5.4.2.1 Component
RESERVED
5.4.2.2 Engine
Start: Understand where run‐to‐run variation lands within experience for each build.
Restart: Understand where run‐to‐run variation lands within experience for each build. Additional
parameters may come into play to simulate restart conditions that the engine may be sensitive to
during the mission.
Throttling: If required in flight, expect acceptance series to thoroughly exercise the control systems and
understand the response of each build relative to “family” of experience.
Shutdown: Understand where run‐to‐run variation lands within experience for each build.
5.4.3 Ancillary Systems
Acceptance test series should operate these systems under expected nominal range of operation. Some
controlled variation on interface conditions are allowed as it pertains to understanding performance and
functionality variation from build‐to‐build.
5.4.3.1 Component
RESERVED
5.4.3.2 Engine
For the most part, expect nominal interface conditions and control profiles. Assure variation lands
within experience for each build.
Autogenous Pressurization: Expect engine test stand configuration to vary from flight systems.
Valve Actuation: These systems can be responsible for significant variation of flight performance and
need thorough examination, which can drive some additional variation of interface conditions during
acceptance test depending upon system sensitivity.
Purges: These systems can be responsible for significant variation of flight performance and need
thorough examination, which can drive some additional variation of interface conditions during
acceptance test depending upon system sensitivity.
Electrical Power and Ignition: May vary interface conditions slightly, but still within expected nominal
operating range
60
5.4.4 Thrust Vector and Gimballing
5.4.4.1 Component
RESERVED
5.4.4.2 Engine
Dry, ambient hardware conditions can be used to check functionality of systems and fits/clearances of
hardware. Hot‐fire conditions can be used to help check proper TVC maximum control range at
expected slew rates, accelerations, loads, and frequency response.
Gimbal Limits: Flex engine to exercise flexible ducting, lines, cabling, and structure. Expect nominal
control profiles to be exercised.
Roll Control Limits: Expect nominal control profiles to be exercised.
Ambient Environment: Measure to understand effects on hardware relative to “family” of experience.
Inspections: Nominal pre‐ and post‐test inspections.
Heat Flux: Verify that the engine‐induced heat flux on adjacent surfaces during hot‐fire TVC testing is
within acceptable limits.
Clearance: Dry, ambient hardware conditions can be used to check fits/clearances.
Interface Compatibility: Expect nominal interface conditions and control profiles to be exercised.
Thrust Vector Alignment: Measure thrust vector during hot‐fire and quantify for stage integration and
vehicle guidance, navigation, and control.
5.5 CONTROLS
Expect strict adherence to software verification and validation (V&V) procedures that are established
during DDT&E. A significant amount of testing is done during the V&V at the component or subsystem level
in a laboratory setting. Engine and higher levels of assembly do little during actual hot‐fire testing to
validate all the software functions since test conditions are typically nominal. However, there are more
benign conditions on these assemblies when the system is not in operation to exercise more of the
software functionality and response of the physical aspects of the system (i.e., valves).
5.5.1 Component
Thoroughly test component and subsystem in laboratory environment. Hardware‐in‐the‐loop type labs
are common approaches for flight software and electronic hardware integration.
Functional Tests: RESERVED
Engine Control System Malfunction Logic Check: RESERVED
EHM System Validation: RESERVED
5.5.2 Engine
Control nominal profiles to be run during hot‐fire acceptance series.
Functional Tests: Prior to hot‐fire, expect a series of functional readiness tests (FRTs) to be run showing
the logic of the software matches expectations. Expect valves and purges to be exercised as a part of the
FRTs.
Engine Control System Malfunction Logic Check: FRTs should include checking fault detection and
response systems built into the engine control system. This may only be exercising redline type cut off
systems.
61
EHM System Validation: FRTs should also exercise any advanced logic that would be considered part of
the fault detection, isolation, and recovery system and the response desired/expected.
5.6 OPERATIONS
5.6.1 PreTest Inspection and Checkouts
See Section 4.6.1, Engine Development/Qualification. However, the verification of the procedures is
assumed complete and this phase focuses primarily on execution of the established procedures.
5.6.2 Leak Test
5.6.3 PostTest Inspection
5.6.4 Drying Purges
5.7 ENVIRONMENTS
The environmental testing investigations are done during development and qualification. However, some
level of screening is incorporated into the acceptance test series even if it is not a specific objective during
engine hot‐fire. Accelerometers, strain gages, skin temperatures, etc., are all utilized to characterize each
engine build’s response. Engine environments during operation tend to be the most extreme environments
on engine hardware.
5.7.1 Component
Acceptance testing for environmental acceptance of components is informed by SMC‐S‐016 or
MIL‐STD‐1540 type documents.
5.7.2 Engine
Largely, engine operating environments encompass the worst‐case environments for the engine
hardware. Thus, it tends to act as an adequate screen for hardware acceptance. For upper stage or in‐
space propulsion, there may be significant environments that exist, which drive design features that need
additional testing over and above engine hot‐fire. Many times, the engine hot‐fire can still serve as an
adequate screening environment.
5.8 DESIGN AND CONSTRUCTION
The design and construction standard have mainly been applied during the design phase and verified
during DDT&E. This section is the execution of procedures developed during that phase to be continued
screening for adequacy of hardware.
5.8.1 Proof Test
Proof testing should occur as part of manufacturing and assembly procedures at the component level.
Leakage is measured during engine and higher assembly build processes.
5.8.2 Human Rating
Tests should be incorporated to ensure the functionality of systems and controls pertaining to
redundancy, fault avoidance, DFMR, and caution and warning systems over and above what have been
discussed already in previous sections.
62
5.8.3 Hardware Discrepancy Tracking
Processes and procedures defined in DDT&E are expected to be utilized during the production phase.
Refer to Section 4.8.6, Development/Qualification.
5.9 PHYSICAL
5.9.1 Mass Properties
Mass properties characteristics should be provided at major component, and engine levels. Procedures
and analysis is accompanied by weighing individual pieces to establish mass and moments of CG inertia at
major assembly levels. Each build is expected to track these characteristics.
5.10 UNIQUE REQUIREMENTS
RESERVED
63
6 PRELAUNCH VALIDATION AND OPERATIONAL TESTS
6.1 GENERAL GUIDELINES TEST TYPES AND RECOMMENDATIONS
General prelaunch requirements are defined in SMC‐S‐016. The scope of these tests covers from receiving a
stage at the launch site to launch. This section is focused on augmenting these, where needed.
Table 6.1‐1: Prelaunch Validation and Operational Tests Matrix
Component
Engine
Section Objective
Receiving Inspection
– Software x x
6.2.1
– Functional Checkouts x
– Leak Checks x
6.2.2 Purges x
6.2.3 Vehicle Readiness Test
Vehicle Tanking Test
– Preflight Procedures
– Preflight Sequence of Events
6.2.4 – Propulsion System Thermal Conditioning x
– Launch Commit Criteria (LCC) Check x
– Thrust Vector Control (TVC) Test
– Post-test Inspections x
6.2.5 Prelaunch Countdown x

6.2 PRELAUNCH VALIDATION AND OPERATIONAL TEST DESCRIPTIONS
6.2.1 Receiving Inspection
An external inspection of the condition of hardware is to be conducted. Discrepancies must be noted and
dispositioned. Desiccants, covers, closures, acceleration monitors, and support equipment/cradles/etc.,
that “touch” flight articles should also be included as part of the receiving inspection, with the focus on
understanding if the condition of the hardware has changed as a result of transportation and handling.
Follow SMC‐S‐016 recommendations for shock, vibration, temperature and humidity monitoring.
Interfaces should be inspected to verify compliance with dimensional and surface finish requirements. If
main propellant inlets covers are removed, the internal area should be inspected to verify FOD‐free
conditions. Follow on inspections and checkouts regarding functionality and more thorough
understanding of internal hardware condition are expected per SMC‐S‐016.
Software: Any engine software should be verified to be the correct version and a checksum performed.
Functional Checkout: Reference SMC‐S‐016
Leak Checks: Reference SMC‐S‐016
6.2.2 Purges
Intent is to maintain engine internal environment (desired cleanliness and moisture levels) after any
protective covers and closures have been removed. Verify pressures, temperatures, flow rates, and fluid
quality (grade, moisture content, etc.) meet requirements.
64
For vehicle assembly and maintenance: While performing final vehicle assembly, maintenance, etc., in a
semi‐controlled environment, periods of exposure may be allowed. However, to ensure the propulsion
system internal environment remains clean, a sequencing of purges and moisture checks are likely to
follow to validate the internal environment.
In preparation for launch: This includes tanking and loading of propellants and pressurants. Purges are
necessary to on a more continuous basis to provide the confidence that the hardware internal
environment is maintained. Purge sequences are usually programmed in and rules for moving from one
sequence to another (forward or backward) are well established and software controlled.
6.2.3 Vehicle Readiness Test
Perform engine control system functional checks through the airborne flight control system with control
parameters at nominal values. Operate the control system per the flight sequence. Verify through real‐
time observation and post‐test inspection that all thermal protection shields, flexible boots, gimbal
hardware and adjacent structures remain properly configured/intact through all physical movement. Also
refer to SMC‐S‐016.
Background/Rationale: The vehicle readiness test is intended to verify readiness of the assembled vehicle
by performing a simulated flight sequence. If the engine control system is capable of being remotely
functioned, it should be functioned accordingly per the flight sequence. Control parameters should be at
nominal operating values.
6.2.4 Vehicle Tanking Test
This test is intended to verify readiness of the assembled vehicle and interfacing ground support
equipment. This test is intended to go as far as possible in the pre‐launch countdown procedures as
feasible without igniting the propulsion systems, including loading of propellants.
For pressure‐fed engine architectures involving Earth storable hypergolic propellants, it is desirable to not
load and subsequently unload hypergolic propellants in the associated flight propulsion system under
nominal operating conditions. As such, the identified guideline for a vehicle tanking test is not
recommended for systems utilizing these propellants. This avoids or reduces the potential for inadvertent
air/ground atmosphere and propellant reactions as well as propellant reaction product precipitate issues
within the flight propulsion system.
Preflight Procedures: Demonstrate preflight procedures and launch timelines. Operational procedures
should be used to the extent possible.
Preflight Sequence of Events: Demonstrate preflight sequence including any functional health checks such
as TVC and valve slewing and engine controller checkout.
Propulsion System Thermal Conditioning: Exercise thermal conditioning processes and procedures. Verify
performance of engine system passive and active thermal control (including chilldown, warming purges,
heaters, etc.).
LCC Check: Verify engine LCC can be satisfied prior to commit time.
TVC Functional Check: Demonstrate TVC operational envelope prior to introducing propellants.
Demonstrate TVC operational envelope after propellant load and engine thermal conditioning.
Post‐Test Inspections: Expect nominal post‐test inspections to be performed after vehicle and launch
facility is “safed.” Intent is to verify that the hardware resulting condition is satisfactory and ready for
flight.
6.2.5 Prelaunch Countdown
Refer to Vehicle Tanking Test for expected checks to be performed during nominal prelaunch countdown
(with exception to post‐test inspections). Checks for prelaunch countdown are in addition to these as they
65
are intended to verify propulsion system health immediately prior to liftoff. Primarily, this consists of
review of the LCC and detailed operating procedure (DOP) prior to executing the launch command. No
failure identifications (FIDs) should be left without disposition. Hardware must be thermally conditioned
and in proper configuration for launch.
66
Appendix A: Definitions and Descriptions
Acceptance Hardware: The item subjected to acceptance testing should be flight hardware and can include
software as applicable. Manufacture of the acceptance hardware must comply with all quality
requirements of the flight hardware.
Acceptance Phase: Phase in which hardware is tested and subsequently accepted for operational use.
Consists of a series of tests that screen for workmanship issues, checks functionality, and measures
the performance of each particular build.
Acceptance Test: Test conducted to demonstrate that flight hardware is free of workmanship defects,
meets specified performance requirements, and is acceptable for delivery. Generally, acceptance
test operating conditions should remain well within the operating envelope and, in most cases, are
fairly nominal. Margins are not tested except under extraordinary circumstances (e.g., specific
technical issue resolution) since the hardware being tested is intended for flight. The appropriate
numbers of starts and duration is a trade between obtaining sufficient ground test time to
accurately establish performance and reliably screen workmanship issues, versus minimizing the
expenditure of operating life. The most appropriate starts/duration will depend on the engine
design, performance repeatability, and manufacturing maturity, as well as the number and severity
of any unique persisting technical issues that require special testing to “clear” prior to flight.
Ambient Environment: The actual external environment surrounding an engine or subsystem. Will vary
depending on whether operation is during ground test or flight test.
Assembly: Completed functional subsystem, system, engine, vehicle, or other hardware, which itself is
assembled from smaller parts.
Battleship Test: A test configuration in which any one piece is designed with the intention of holding
significant margin regarding life, disregarding the thrust‐to‐weight target of the assembly
(a.k.a. workhorse).
Booster: The lowest stage of a multi‐stage launch vehicle that injects an upper‐stage space vehicle and
satellite into typically a sub‐orbital trajectory.
Breadboard: Representative components are integrated in a laboratory or facility test environment that is
representative of the functional relationship of the final system to understand , but is not
configured with in the final system configuration or with all of the
Burst Factor: A multiplying factor applied to the maximum expected operating pressure (MEOP) to obtain
the design burst pressure.
Burst Pressure: The maximum force per unit area that an object containing a fluid can withstand before the
stress created by that force causes the object to fail.
Burst Test: Test to identify the burst pressure.
Chamber Pressure (Pc): Force per unit area within the enclosed chamber where combustion takes place
between the injectors and throat. Often referenced as injector end Pc (static pressure at the
injector face), or nozzle stagnation Pc (calculated from injector end Pc and Rayleigh losses).
Component: Major subdivision or elementary part of a system (e.g., injector, chamber, turbopump, etc.).
Demonstrator (or Prototype) Program: Program to increase confidence in the likely success and provide risk
reduction for proposed new designs, concepts, applications, or technologies prior to a full
development program.
67
Design, Development, Test and Evaluation (DDT&E): The period during which a new program design or
concept is initiated, refined, and implemented up to manufacturing of qualification or flight
hardware. Activities during this phase will provide confidence that the new design and concepts
will accomplish mission objectives. Also refer to development phase.
Development Hardware: Generally full‐scale hardware and very similar to the flight hardware. The
hardware in the early development phase should be representative in flow path and function,
although it may not necessarily be flight‐weight hardware or meet flight quality requirements. This
is appropriate since design changes are often required during the development program as new
data are collected. However, by the time the development program is completed the test articles
should be completely or nearly completely equivalent to the flight hardware in all aspects of flow‐
path and design.
Development Phase: The development phase usually provides the first true demonstration of the
capabilities of a proposed design. Development tests are conducted as required to: (a) Validate
new design concepts or the application of proven concepts and techniques to a new configuration;
(b) Assist in the evolution of designs from the conceptual phase to the operational phase; (c)
Validate design changes; (d) Expand, update, and anchor models; (e) Reduce the risk involved in
committing designs to the fabrication of qualification and flight hardware; (f) Develop and validate
qualification and ATPs; and (g) Investigate problems or concerns that arise after successful
qualification. Requirements for development testing therefore depend upon the maturity of the
subsystems and units used and upon the operational requirements of the specific program. An
objective of development testing is to identify problems early in their design evolution so that any
required corrective actions can be taken prior to starting formal qualification testing. Development
tests should be used to confirm structural and performance margins, manufacturability, testability,
maintainability, reliability, life expectancy, and compatibility with system safety. Where practical,
development tests should be conducted over a range of operating conditions that exceeds the
design requirements to identify marginal capabilities and marginal design features.
Development Test: Tests conducted on representative articles to characterize engineering parameters,
gather data, and validate the design approach. Development test conditions should encompass
worst‐case conditions for the intended application. In addition, the development program is used
to verify by test margin outside the operating envelopes. Margin testing will allow a successful
development program to identify weak aspects in the design, identify, eliminate, and/or mitigate
failure modes prior to beginning qualification, and ultimately yield a much more robust design.
However, development testing should avoid conditions that violate acceptable safety margins or
cause unrealistic modes of failure.
Duty cycle: (1) Ratio of pulse duration time to pulse repetition time. (2) Required operating life.
Engine: See “Liquid Rocket Engine.”
Engine Cycles: A thermodynamic cycle that describes how liquid propellants are fed to the engine
combustion chamber. Types include pressure fed, expander, gas generator (GG), and staged‐
combustion.
Engine System: A term used to describe the liquid rocket engine (LRE) portion of the integrated stage;
whether it is a single engine or a multi‐engine configuration.
Envelope: (1) Boundary which encompasses all possible variations of a set of parameters (e.g., operating
conditions envelope). (2) Dimensional boundary which encompasses the component or system.
Expansion Ratio: Ratio of cross‐sectional area of rocket nozzle exit to area of nozzle throat.
Expendable Engine: An engine that is discarded after use on a single mission. The SL of an expendable
engine includes all ground testing and mission operation time.
68
Factor of Safety (FoS): A multiplying factor applied to the maximum expected operating loads in the design
analysis to account for uncertainties in actual load conditions, material properties, design
procedures and analyses, and manufacturing.
Flight Design: Final production design intended for the “as‐flown” hardware.
Flight Operational Phase: This phase begins at launch. It includes test flights prior to the first mission and
the actual mission flights themselves. Not all new vehicles will include test flights but, when they
do occur, they may be performed for a nominal mission or a selected mission of particularly high
interest or value. In some cases, special operational tests may be added to a mission flight
provided those tests do not adversely impact the mission objectives. Both test flights and mission
flights may be flown with special flight instrumentation along with the standard flight
instrumentation (SFI) suite to provide additional flight data for further operational characterization
and/or verification for aspects that could not be adequately ground tested. Results generated
during this phase may include performance reconstructions and detailed post‐flight data reviews,
with the goal of verifying in‐flight specification performance, interface compatibility and
predictions (e.g., engine and vehicle operating environments), calibration/control, and the ability
to meet future mission requirements. The accumulation of flight data generally leads to
refinement of flight simulations and revision of expected flight dispersions. If flight experience is
found to be significantly different than expectations and/or specified design requirements, then
further development and/or qualification ground testing may be warranted. No further detailed
discussion of flight operational testing is contained in this document, since objectives for this
phase are often program dependent.
Functional Test: Test that exercises the function of an item.
Guard Band: The magnitude of the offset from the specification limit to the acceptance or rejection zone
boundary.
Impulse: Integral of thrust and time over a specified time period.
Line Replaceable Unit (LRU): A unit (e.g., igniter or closed‐loop control valve) that may be removed and
replaced by a new unit without requiring engine removal and test firing.
Limit Load (LL): The highest load, or combinations of loads, that may be applied to a structure during its SL
and, acting in association with the applicable operating environments, produces a design or
extreme loading condition for the structure. When a statistical estimate is applicable, the LL is that
load not expected to be exceeded on at least 99% of flights, estimated with 90%conficence.
Liquid Rocket Engine (LRE): Launch or space vehicle propulsion subsystem device utilizing a combination of
components and liquid phase chemical reactants to provide thrust. These devices also can be
grouped or be part of an engine system to satisfy full stage and vehicle propulsion needs.
Margin: Capability in excess of worst‐case operating conditions.
Margin of Safety (MS): MS = 100% x allowable / (FS x actual) ‐ 1.
Maximum Expected Operating Pressure (MEOP): The highest gage pressure that an item in a pressurized
subsystem is required to experience during its SL and retain its functionality, in association with its
applicable operating environments. The MEOP is synonymous with limit pressure or maximum
operating pressure (MOP) or maximum working pressure (MWP). Included are the effects of
maximum ullage pressure, fluid head due to vehicle quasi‐steady and dynamic accelerations, water
hammer, slosh, pressure transients and oscillations, temperature, and operating variability of
regulators or relief valves.
Maximum and Minimum Expected Temperatures: The highest and lowest temperatures that an item is
expected to experience during SL, including all operational modes.
69
Mixture Ratio (MR): Ratio of the oxidizer mass flow rate to the fuel mass flow rate. For engine MR this is
measured at the engine inlets.
Non‐destructive Testing (NDT): Methods of testing for integrity that does not impair serviceability or future
life.
Operability: the ability to support required flight rates and schedules and to meet a variety of operational
characteristics while minimizing cost and risk
Operating Envelope: Extreme boundaries of conditions to which hardware may be subjected during
operation (e.g., thrust and mixture ratio boundaries).
Operating Environment: Thermal, pressure, dynamic and/or electromagnetic conditions to which the
system is exposed during its operational life.
Operational Life: The operational life of an engine is the total allowed starts and run‐time including ground
acceptance testing, on‐pad firings/aborts and flight exposure.
Part: A single piece, or two or more joined pieces, which are not normally subject to disassembly without
destruction or impairment of the design use.
Pogo Effect: Self‐excited, sustained vibration due to interaction of structural vibration and engine thrust
oscillation.
Prelaunch Operational Phase: This phase begins when the flight hardware and software are received at the
launch site and continues until launch. It includes all preparatory operations and checkout testing
to verify flight readiness. It may also include separate flight readiness static firings and/or
autonomous engine health monitoring and checkout during the engine startup and main stage
operation immediately prior to lift‐off. It is intended to ensure the readiness of the hardware,
software, personnel procedures, and mission interfaces to support launch and the program
mission. On some occasions, the prelaunch operations may include unexpected or out‐of‐sequence
inspection, testing, or modification of flight hardware to resolve identified concerns after the
hardware has been delivered to the launch site.
Prelaunch Tests: Testing following system delivery to vehicle factory or launch site prior to launch. Used to
verify system readiness for integration, to verify system integrity, safety and performance, and to
verify successful integration.
Pressure Vessel: A structural component whose primary purpose is to store pressurized fluids.
Proof Factor: A multiplying factor applied to the LL, or MEOP, to obtain the proof load or proof pressure for
use in a proof test.
Proof Load: Value established by taking the calculated maximum design condition, MEOP, etc., and
multiplying it by the proof factor.
Proof Pressure: Synonymous with the term proof load. Value established by taking the calculated maximum
design condition, MEOP, etc., and multiplying it by the proof factor.
Proof Test: An acceptance test used to prove the structural integrity of a unit or assembly, or to establish
maximum possible flaw sizes for safe‐life determination.
Propulsion System: The system required to produce thrust, which includes the engine system; propellant
tankage and feedlines; off‐engine valve, fill, vent, purge, chilldown and drain systems; pogo
suppression devices; and propellant tank pressurization systems, as applicable.
Prototype: First example build of item intended for production. Intended to be as representative of
definitive article as possible, but usually deficient in many respects. Many times the prototype will
70
be focused on replicating only specific parameters since its purpose is to guide future
development, permit customer evaluation, and demonstrate key new technologies.
Prototype Phase: Prototype testing (a.k.a. feasibility, risk reduction, or demonstration testing) often
precedes development and is intended to assist design definition by providing engineering data to
confirm analyses and/or help define expected operating conditions. Often this testing includes
Research & Development (R&D) to explore and/or validate new technologies that might be
beneficial to the engine system. Prototype hardware is typically designed to be more robust with
greater margins compared to flight hardware because the design and operating conditions have
higher uncertainty during this phase. The hardware may contain facility components in place of
flight components, modified components from earlier engine models, or component simulators to
gain the engineering information needed to complete the initial flight design. Breadboard and/or
bench‐level type engines or subsystems may be used in some cases, and subscale testing is also
common. Not all programs will utilize prototype hardware or perform prototype testing. The
decision will be based on a number of factors, including the design maturity, technology readiness
levels, and the program risk mitigation strategy. No further detailed discussion of prototype testing
is contained in this document since objectives for this phase are often technology dependent.
Qualification Hardware: Essentially first production articles that go through a series of qualification tests to
demonstrate readiness for flight operation. Qualification hardware should be produced from the
same drawings, using the same materials, tooling, manufacturing processes, and level of personnel
competency as will be used for actual flight hardware. Limited deviations are permitted, but only if
required to accommodate benign changes that are necessary to conduct the testing. These
changes may include adding instrumentation to record functional parameters for engineering
evaluation. Feedlines should also be as flight representative as possible, definitely at the integrated
propulsion system level testing, but preferably at engine level testing as well. Manufacture of the
qualification hardware should be controlled to the same quality requirements as the flight
production hardware. Often the qualification engines are the first engines off of the production
line.
Qualification Phase: Qualification testing provides formal verification that the final design, manufacturing
processes, and acceptance program produce flight hardware/software that meet specification and
performance requirements with adequate margin to accommodate expected variations in
hardware and engine operation. It generally follows completion of the development test program
to reduce program risk at this phase, and concurrent completion of development objectives and
qualification testing is not recommended. Testing should validate the planned acceptance
program, including test techniques, procedures, equipment, instrumentation, and software, as
well as potential rework and repeat test cycles.
Qualification Test: Tests conducted to demonstrate satisfaction of design requirements including margin
and product robustness for designs that have no demonstrated history. A full qualification
validates the planned acceptance program, in‐process stress screens, and retest environmental
stresses resulting from failure and rework. Qualification testing verifies compliance to engine
specification requirements and vehicle interface requirements over the range of expected
operating conditions. Overall qualification test conditions should encompass worst‐case conditions
for all intended applications. Selected margin conditions (e.g., operating life margin, thrust margin)
are also verified to demonstrate robustness. However, qualification testing should not create
conditions that violate acceptable safety margins or cause unrealistic modes of failure.
Restart: Engine start after previous shutdown without interruption of the environment or modification of
the hardware or setup.
Reuse: Use of engine after completion of one mission.
71
Reusable Engine: An engine that is to be used for multiple missions. The SL of a reusable engine includes all
ground testing, planned reuses (mission operation times), refurbishment, and retesting.
Reusable Item: An item that is to be used for multiple missions. The SL of reusable hardware includes all
testing, planned reuses, refurbishment, and retesting.
Service Life (SL): The SL of an item starts at the completion of fabrication and continues through all
acceptance testing, handling, storage, transportation, prelaunch testing, all phases of launch,
orbital operations, disposal, re‐entry or recovery from orbit, refurbishment, retesting, and reuse
that may be required or specified.
Similarity: The process of assessing by review of prior acceptance data or hardware configuration and
applications that the article is similar or identical in design and manufacturing process to another
article that has been previously qualified to equivalent or more stringent specifications.
Specific Impulse (Isp): Engine Isp is the instantaneous total thrust divided by the instantaneous total weight
flow rate of propellants through the engine inlet. Specified at specific altitude (e.g., sea level
and/or vacuum conditions).
Storage Life: The time that a unit can be stored after acceptance tests without replacement of parts and
subsequently operate within specification limits.
Subassembly: A unit containing two or more parts which is capable of disassembly or part replacement.
Ultimate Load (Design): A load or combination of loads that the structure must withstand without rupture
or collapse in the applicable operating environments. It is equal to the product of the LL and the
design ultimate FoS.
Upper‐stage Vehicle: An upper‐stage vehicle is one or more stages of a flight vehicle capable of injecting a
space vehicle or vehicles into orbit from the sub orbital trajectory that resulted from operation of a
launch vehicle booster.
Validation: To show to be accurate and correct (as in, validate requirements or validate results). Validation
can be by inspection, demonstration or analysis.
Verification: Confirmation that ground and flight hardware and software are in compliance with design and
performance requirements (as in, verify capability). Verification can be by inspection, test or
analysis.
Yield Load (Design): A load or combination of loads that the structure must withstand without experiencing
detrimental deformation in the applicable operating environments. It is equal to the product of the
LL and the design yield FoS.
72
Appendix B: Other Relevant Discussion
B.1 OTHER ASPECTS OF DDT&E RELEVANT TO TEST
Requirements definition specify and describe functional requirements, program requirements, technical
requirements document, interface control document, engine and system requirements. Test planning and
implementation cannot be completed properly if detailed requirements and necessary engine operating
characteristics are not well established. Programs should not over specify requirements or establish
unnecessary requirements. Instead it should set goals for aspects that do not have hard requirements to
allow for pushback on goals to make an overall more optimal system.
B.1.1 Design Heritage
Previous design heritage is important, but must be appropriate to the specific intended conditions and
application. Design heritage includes any past testing and operational use of engine components and sub‐
systems. Heritage can provide a significant advantage and a “head start” on the path toward thorough
understanding of engine operation and ultimate verification of the engine capabilities. Together,
modeling and simulation and design heritage help determine whether it is appropriate to qualify certain
aspects of the engine by similarity to existing systems, by analysis, or by test. For vehicle structures, well‐
defined “qualification by similarity” guidelines allow objective use of design heritage to assist with
qualification of new designs. In any event, care must be taken that the “similarity” being assumed is
indeed there. For example, a component ostensibly built to the same drawings and specifications, but by
a different vendor cannot be assumed to be the same without additional verification. Experience has
shown this to be increasingly important as the life cycle of the engine progresses.
B.1.2 Risk Management
Risk Management is intended to anticipate potential failures, mitigate the impact of failures, and solve
issues early in the program. Program Managers should avoid a “1‐strike‐and‐you’re‐out” mentality during
the test program (i.e., very adverse to near term political risk) and instead adopt a mission success‐
oriented mentality (i.e., mitigate long term technical risk). Impacts increase as risks are deferred, and it is
far better to perform robust testing and damage test hardware than to perform “safer” testing but lose a
mission payload due to a failure mode that should have been identified by a proper test program. It might
be possible to actively encourage a focus on mission success and mitigate the impact of political risk by
tying stakeholder metrics and rewards to long term mission success. Technology development tends to be
a part of every engine development program. Inherently there is risk to realizing all the benefits as
promised by the advancement in technology. This undoubtedly has programmatic risk, but also
performance and functionality risk. DDT&E programs must be careful in the classification of risk as
countermeasures for the two different types (i.e., programmatic versus technical) are totally different.
B.1.3 Programmatic Considerations
Apart from the technical aspects that this document strives to focus upon, there are significant
programmatic aspects that must be considered on each program/project. Consider politics, cost, and
schedule. Careful planning is necessary to provide the appropriate cost and schedule margins to testing
and evaluation activities.
B.2 TEST PLANNING
Test planning includes the process that establishes the test objectives from program and technical
requirements, and integrates those objectives with the product configuration, to identify test article
configurations, establish detail test data monitoring and analysis plans, and to identify requirements for
human resources, the facility, test equipment, safety, pre‐/post‐test inspections, and reporting.
Fundamental elements of the test planning process include identification of test objectives, success criteria,
and analyses plan. Supporting elements include instrumentation, facilities & special test equipment (STE),
safety, test risk assessment, scheduling, and test procedures. Data deliverables set up the necessary
73
submittal rate and level of detail needed at each submittal. For example, there is a need for validation of
the requirements early in the program. Verification matrices need to be set up with verification methods
and success criteria (top level) to validate the requirements as early as systems requirements review (SRR).
One reference to utilize is MSFC‐HDBK‐2221, Verification Handbook, which describes the verification
process in Volume I, and provides verification documentation examples in Volume II. Other discussion
below may be useful in scoping the documentation and planning effort in preparation of the test program.
It is not considered all inclusive.
B.2.1 Minimum Elements for Test Plan
The minimum requirements for a test plan include security classification, objectives/success criteria, test
matrix, test schedule, test risk assessment, required safety procedures, test article description, facility and
STE requirements, instrumentation list, list of all engine and facility abort parameters (by instrumentation
header), test conditions/set‐up/standard tolerances, detailed run procedures, and post‐test
requirements.
B.2.2 Requirements Definition
Planning must properly define requirements, including a description of the technical and contractual
requirements to be accomplished through testing. In addition, it should provide a link to the Systems
Engineering & Integration requirements verification planning activity. It should include data, documents,
reviews, propellant conditions, hardware, and data. Planning must consider how flight margins will be
accounted for (e.g., during ignition or minimum NPSP start demonstration, “testing of propellant tank
inlet conditions should demonstrate a minimum of 0.5 pounds per square inch differential (psid) start box
margin”).
Test Team
Planning must specify the roles and responsibilities of test team, as well as define the mechanism
for communicating changes to test plan.
Test Resources
Planning must establish resource allocation baselines, as well as providing spare hardware. The
plan must account for the likelihood of failures and damage to facility or test article hardware.
B.2.3 Requirements Verification Test Planning Elements
Objectives
Specific objectives should be generated to quantitatively describe what is to be accomplished in
the test or test series. All objectives should have measurable outcomes.
General Implementation
The most appropriate test methods should be selected. Candidates include Design of
Experiments, direct functional objective based approaches, or statistical methods.
Test Article/Assets
Actual engine hardware with the correct configuration must be defined and tested.
Success Criteria
Specific and well‐defined success criteria must be established for the measurable parameters of
various test objectives to ascertain whether a given test achieved its respective requirements
and goals. There should be success criteria defined for all objectives. Agreed upon definitions for
valid tests, premature cut‐offs, and aborts must be established.
Analysis Plan and Data Review
Analysis and data review requirements must be identified for each test. Items may include
inspection reports, low‐speed data analysis, high‐speed (hi frequency) data analysis, spectral
analyses, video/film, and comparison to system and transient models. Results should include
74
recommendations for subsequent testing, depending upon satisfaction of success criteria for the
preceding tests.
Instrumentation
A detailed instrumentation list must be generated that defines what must be measured to
evaluate the success of the test and safely operate the facility. The list should include all
instruments used in the test program, including those used on the test article and the facility.
Detailed specifications should be provided for the following: parameter description, instrument
name/callout, instrument type, instrument vendor/serial number, calibration life cycle, critical
instrument designation (i.e., required or desired for upcoming test), redline channel assignment
(if applicable), installation torque, placement/location, port requirement, cleanliness
requirement, timing (i.e., time periods of active recording), range, response, sample rate,
power/purge requirements, and designation (e.g., normal flight instrumentation, ground test
instrumentation [GTI], SFI).
Test Facilities and Equipment
All equipment needed to support the various tests must be specified and allocated with respect
to the schedule. Standard interfaces should be used to the maximum extent possible. Necessary
computer control and communication must be determined and established.
Environmental Impact
The potential impact of the test program to the surrounding environment and proposed
mitigating actions must be defined. The appropriate processes should be employed well in
advance of testing to ensure the required approval is received.
Safety
The safety procedures to be used during test article installation, operation, and removal must be
identified. Special precautions should be included as necessary. This should include provisions for
handling of hazardous materials and contaminated hardware.
Test Risk Management
Test risk assessment should be performed to identify and provide a summary of the risks
associated with the tests required by the test plan. Pre‐declared risk statements should be
generated for any tests including operating conditions that are either inherently dangerous or do
not exist during the mission (e.g., malfunction tests, tests to failure, safety limits testing,
operating limit survey), as well as known high risk tests that may not accurately represent true
flight conditions but if successful could mitigate mission risk. In addition, a clear plan for incident
handling is needed in case of significant test mishap.
Scheduling
The tasks and timing required to complete the test or series of tests must be identified and
incorporated into an integrated schedule. The schedule should be used to monitor progress, so
that any potential issues can be identified and mitigated in a timely manner.
Test Procedures
Detailed test procedures with instructions for performing each test should be generated. They
should provide line‐by‐line description of the test sequence to be performed. Potential test
procedures include facility and test article setup, countdown checklist, post‐test securing, and
next‐test preparations.
Hazards Analysis
An integrated facility and test article hazard analysis should be completed and reviewed prior to
first test to confirm that appropriate mitigations are in place.
75
B.3 TEST OPERATIONS AND IMPLEMENTATION
Processes must be put in place to efficiently and effectively turn tests around. When considering the
complexity and cost associated with each engine or higher level of test, it needs to be well orchestrated.
The following information is provided as guidance information, helping formulate an effective turn around
process.
B.3.1 Test Preparation
Test Readiness Review
Performing a test readiness review just prior to actual start of testing provides many benefits. It
brings together the entire test team to review a summary component and engine pedigree, the
planned testing, schedule and manpower constraints and open items that must be closed prior
to first engine test.
Test Performance Predictions
Performance analysis will provide the basis for understanding the expected operational profile of
the test article. Pressure, temperature, flow rates are all modeled to provide a pre‐test
prediction.
B.3.2 PostTest Activities
Data Review
A systematic review of the test data should be conducted concurrently within a test series as well
as upon the completion. This should include test article as well as test facility data. The basis for
the review should include comparison to specifications and interface requirements, identification
and disposition of out‐of‐family conditions and statistical outliers, and comparison to model
predictions. The data review should be conducted by the hardware experts and reviewed with
the multi‐disciplinary test team and other experienced and objective personnel. The latter
reviewers’ function is to provide a systems review and to identify potential cross‐discipline
issues. Future test constraints should be identified and resolved. A database of test anomalies
should be maintained to capture lessons learned. The individual test data review should be
limited to verification that success criteria or test objectives were met, identification of test
anomalies, and evaluation of future test safety based upon current test results. Extensive
analysis of test results for model validation etc., should be separated from the test turnaround
process.
Test Objectives Success Evaluation
Evaluate the gathered data relative to the test objectives, and then relate them back to
verification activities. The data should be scrutinized heavily as it is significant to determine
whether the success criteria has been met or if another test will have to pick up this objective.
Consider the data, calibration of instrumentation, and system uncertainties.
Anomaly Resolution
The anomaly resolution process should include a comprehensive assessment of potential root
causes utilizing a fault tree, fishbone analysis, or other similar techniques. The potential impacts
of the anomaly on the health of the hardware directly or indirectly impacted by the anomaly
should be systematically evaluated. Investigation of the anomaly may require additional
instrumentation and/or testing. Recovery from the anomaly may require design or operating
changes.
Hardware Functional Checkout and Inspection
Functional checkout of engine components is often valuable following the completion of higher
level testing. This testing serves to verify hardware viability and performance. Inspection of
hardware following completion of testing should entail disassembly to the extent that wear
76
and/or mechanical integrity can be evaluated. Assemblies that are in the process of life testing
can be subjected to an abbreviated inspection sufficient to confirm viability to continue the life
testing followed by a complete disassembly inspection at the conclusion of life testing. Inspection
and functional checkout may be waived if sufficient prior experience exists for the hardware
under similar test conditions. Qualification engines should be completely disassembled,
inspected, and displayed after completion of testing.
Test Plan Adjustments
Replanning is crucial to effective use of the test budget (numbers of tests and contingency
available for retest/reverify activities).
B.3.3 Quality Assurance (QA)
QA provides an independent check on the workmanship performed in all program phases. In design, they
ensure the proper features, criteria, and processes are included into the DDT&E phase, which will set the
stage for production and operational phases. In manufacturing and assembly, QA performs the function of
ensuring the design and process checks are properly implemented. QA oversees activities during test and
evaluation to ensure tasks are performed per instruction. QA also enforces the dispositioning of hardware
(and it associated paperwork) when something does not meet specification tolerances. This can occur in
any phase from manufacturing through test. QA Plans are written to document nominal and
dispositioning processes for a program to follow, and is expected to vary from program to program.
B.3.4 Configuration Control/Management
Discuss general goals, objectives, and value of effective configuration control. Configuration Management
Plans are written for each program to document information flow and management for a program to
follow, and is expected to vary from program to program. Look for future revisions of this document to be
more explicit on scope of configuration management relative to test and evaluation of liquid rocket
engines.
Test Article
Discuss general and specific requirements, with guidance on what are minimal acceptable
requirements. Describe potential processes to track/control configuration. Identify potential
allowable configuration deviations from flight for various phases. Discuss the implications of
design changes subsequent to development and qualification testing. Discuss technical value of
recording and documenting evolution of design, rationale for design decisions, and the
associated testing performed on various evolved designs. Identify classifications. Configuration
control must address various phases including design, manufacturing processes, system
integration, testing, handling/transport processes, and storage procedures.
Development/qualification processes must be consistent with future processes for production
and flight.
Test Facility
Discuss general and specific requirements, with guidance on what are minimal acceptable
requirements. Describe potential processes to track/control configuration.
Flight Design
Discuss need to provide and document direct traceability between test articles (especially
qualification) and flight articles.
Software
Discuss general and specific requirements, including the configuration control levels required
during various types and phases of testing. Discuss the configuration control requirement
differences between on‐vehicle software and test stand control software. Describe potential
processes to track/control configuration.
77
Methods/Procedures
Provide more details for the various activities to understand/track/validate changes through
physical configuration audit (PCA), functional configuration audit (FCA), drawings, schematics,
procedures, functional/similarity checkouts, end‐to‐end checks, and configuration control board.
B.3.5 Documentation
List and describe all the required documents for test program, which may include, but not be limited to,
the following: design requirements review, preliminary design review, critical design review, system
design review, test requirements document, critical experiment review, test plan, process specifications,
test procedures, propellant specifications, interface control document, system modeling analyses, test
readiness review, data reviews, test results briefings, and test reports. Nomenclature on reviews tends to
be facility‐specific, so there may be a need to define various types of documents and reviews.
B.4 TEST DATA UNCERTAINTY AND TOLERANCES
B.4.1 Testing and Uncertainty Determination Overview
Analytical engine performance must be verified with test data results, which are not equal to the true
performance values. Rather, the test data results approximate the true engine performance values within
measurement uncertainty bounds. Estimating those uncertainty bounds in the test‐planning phase is
recommended to ensure that the test results meet accuracy requirements.
Measurement uncertainty in desired results such as thrust, specific impulse, characteristic velocity (C*),
C* efficiency, and mixture ratio must often be estimated from more fundamental measurements, such as
temperature, pressure, mass flow rate, each with its own characteristic measurement uncertainty. The
composite uncertainty in a result is then determined by combining the uncertainties of more fundamental
measurements.
Fundamental measurement errors may be categorized as bias (systematic) errors, and precision (random)
errors. Bias errors are fixed and may be quantified and reduced by calibration. Precision errors are
variable in nature, may be quantified by statistical methods, and may be reduced by averaging multiple
readings. The composite uncertainty in a result is the combination of bias and precision uncertainties that
approximate the bias and precision errors of more fundamental measurements. Standard uncertainty
analysis methods [Coleman & Steel (1989); NIST Technical Note 1297 (1994)] may be used to calculate
composite uncertainty in result variables, which depend on fundamental uncertainties through sensitivity
(influence) coefficients. Sensitivity coefficients may be estimated analytically or numerically.
Calibration of fundamental measurement instrumentation against “primary” standards is required.
Primary standards include measures of mass, length (volume), and time traceable to the National Institute
of Standards and Technology (NIST). For example, a thrust load cell instrument may be calibrated against
a primary standard of mass if the local value of gravitational acceleration is known. There are bias
uncertainties associated with the mass standard and the gravitational acceleration constant that transfer
bias to the load cell instrument via the calibration process. Calibration against “transfer” standards may
also be advantageous. For example, pressure transducers may be periodically calibrated against an
electronic transfer standard which, in turn, is periodically calibrated against a primary standard traceable
to NIST. Maintaining control charts of calibration data is recommended to ensure that bias errors stay
within estimated bias uncertainty limits. It is also recommended that sources of bias uncertainty be
carefully evaluated to account for dependencies. Such bias dependencies can increase or decrease the
actual composite uncertainty of result variables. Bias correlation can be accounted using standards
methods [Coleman & Steel (1989); NIST Technical Note 1297 (1994)].
In addition to bias uncertainty in measured variables, additional ground‐to‐flight engine performance
biases may require estimation, including:
 Effects of hardware installed post‐test (e.g., nozzle extensions)
 LRUs change‐out
78
 Time/duration effects correction
 Mission unique adjustments
 Post‐qualification manufacturing changes
Estimation of such ground‐to‐flight biases may require additional development tests to quantify the
differences within the desired level of uncertainty.
Precision (random) uncertainties are estimated with test data using standard statistical methods
[Coleman & Steel (1989); NIST TN 1297 (1994)]. Precision uncertainties may be reduced by averaging
multiple readings since the standard deviation of a mean value is a factor of square root(N) lower than the
sample standard deviation (where N equals the number of data readings averaged). In many cases, data
from previous testing can be included to obtain a larger sample size; however, care must be taken to
ensure the previous data is statistically relevant.
There are many potential sources for uncertainties in measurements. Some are caused by human errors
such as improperly reading the instrument, chart, or record, and/or improperly interpreting or correcting
these data. Some are caused by instrument or system errors that may fall into four classifications: static
errors, drift errors, dynamic response errors, and hysteresis errors. Static errors are usually fixed errors
due to fabrication and installation variations. Static errors can usually be detected and mitigated by
careful calibration and an appropriate correction can be applied to the reading. Drift error is a varying
output reading over a period of time for the same input, often caused by changing biases of measurement
system or changes in the environmental conditions. Drift errors can also usually be detected and
mitigated by careful calibration and an appropriate correction. Dynamic response errors occur when the
measured quantity changes its value faster than the measurement system can accurately register.
Dynamic response errors can be mitigated by the careful selection of instrumentation with the
appropriate response characteristics. Hysteresis error is usually a combination of mechanical hysteresis
(output deviation at a certain input condition when that input is approached first with an increasing
reading and then with a decreasing reading) and temperature hysteresis (output deviation at a certain
input, before and after a temperature cycle). Hysteresis error can be mitigated by careful evaluation and
correction for such system characteristics.
Statistical Design of Experiments (DOE) is often used to reduce the number of test conditions and test
samples required to characterize the influence of individual variables in a multi‐variable system. For small
test articles, it may be appropriate and affordable to conduct many tests with many different hardware
samples. For the LREs considered in this Guideline, 1000 lbf thrust and greater, it is usually not fiscally
viable to test the number of unique hardware samples that a DOE designed test plan would suggest. This
should not be used as justification to not perform statistical analysis; rather, the proper application of
experimental uncertainty methodologies can yield important insights into the behavior of the engine. In
fact, knowledge of the experimental uncertainty can make a test program more efficient. For example,
postulating a fictitious rocket engine test campaign that includes tests to characterize a hardware
configuration change. However, the potential benefit is expected to be a factor of 4 less than the
uncertainty for a single test. In that case, the testers could either perform the test and accept a
confounded result, perform a statistically sufficient number of tests in each configuration, test with a
sufficient number of unique samples, or not perform the test. By having the experimental uncertainty
knowledge, the program can make a well‐informed decision.
Regardless of the level of uncertainty, these levels must be well characterized and acknowledged in the
test planning and test evaluation. Test planning must account for data uncertainty to ensure that test
conditions, objectives, and verification targets will be satisfied beyond the uncertainty range of the data.
Test evaluation must account for uncertainty to determine whether a certain test objective is
accomplished, a particular design specification requirement is satisfied, or an acceptance test result
meets mission requirements.
79
B.4.2 Specification Decision Making with Consideration of Uncertainties
LRE characterization is one of the most critical aspects of launch vehicle performance. As this section has
previously discussed, there are significant contributors to LRE performance uncertainty. The uncertainty
associated with the propulsion system uncertainty must be incorporated into the vehicle’s flight
performance reserve. Therefore, uncertainty contribution from the rocket engine acceptance test tag‐
value must be properly accounted for to ensure the mission is not subject to unknown risk or excessive
propellant margin. The American Society of Mechanical Engineers standard, ASME B89‐7.3.1‐2001 was
developed “to facilitate the development of understanding between suppliers and customers regarding
measurement uncertainty in the decision to accept or to reject a product.” Liquid propellant rocket
engines are provided in a customer‐supplier relationship to a launch vehicle or spacecraft. Even in cases
where the engines are developed with the same company, that relationship still exists. The engine is
provided as a major component and how its performance is integrated into the vehicle system is critical.
ASME B89‐7.3.1‐2001 does not dictate a singular way to consider conformance to specifications; rather, it
provides a set of definitions and alternative approaches to interpreting the acceptance of a product and
conformance to specifications. Any of the approaches will work if properly propagated through the
integrated performance model; however, the benefit of explicitly using one of these methods is the
consistency provided through clear communication of the confidence in the engine’s performance.
A customary practice today is to acceptance test an engine and to perform analysis during data reduction
equations to calculate a performance parameter (for example, calculating specific impulse using
measured thrust, measured mass flow rates, and making corrections for known biases, adjustments to
nominal inlet conditions, etc.); then compare the result to the contract specification. If it is within the
specification, the engine is accepted; if not, it is rejected until changes are made such that it is within the
specification zone. By explicitly incorporating the fact that measurement uncertainty exists, a set of
decision rules can be created. ASME uses the term guard band to represent the uncertainty estimate
associated with a specification parameter. The guard bands can either be inside the specification range or
outside of it, and then creates regions of stringent acceptance and relaxed acceptance. Figures B.4‐1 and
B.4‐2 represent two possible decision rules. Other decision rules include cases where a specification value
is a not to exceed or a greater than, i.e., an asymmetric or one‐sided decision rule. The other decision
rules can be found in the ASME Guideline.

Figure B.4‐1: Symmetric two‐sided relaxed acceptance and stringent rejection

Figure B.4‐2: Stringent acceptance, simple rejection, and a transition zone example
using symmetric two‐sided guard banding
80
Appendix C: Representative Campaigns and Test Turn

around Rates for a Variety of R&D and Flight LRE Testing
This Appendix is an extracted synopsis of an NASA SSC internal document entitled “SSC Rocket Propulsion
Testing Major Statistics,” dated 29 April, 2005, and filed as Study #6323‐06‐007, in NASA SSC Central
Engineering File, developed by co‐authors Kirchner, Morgan, and Rahman.
The following summarizes testing activities conducted at the John C. Stennis Space Center (SSC). The results
include the total number of tests performed, the total number of test seconds accumulated, the test
turnaround rate, and the test achievement rate for all test campaigns during the history of SSC (from 1966
until the end of 2004, i.e., four decades).
The primary benefit of collecting this information is to apply it as a planning and forecasting tool for future
test projects to indicate the “time it takes” (i.e., test windows) to conduct tests for either research and
development (R&D) or flight hardware (components, engines, stages).

Results and Discussion
The results are presented and discussed primarily in terms of the various major statistical outcomes
(number of tests, test seconds, test achievement rates or “completion rates,” etc.). The outcomes are
affected by a variety of factors and these are highlighted in the discussion in terms of how they likely
influence the respective outcomes.
For various types of large scale test articles at SSC, it is found that the test project teams can engineer and
operate the facilities to accomplish a test on an average of approximately every 11 calendar days, and will
generally test with better than 75% probability of completing that test. Further illustration and discussion of
these results by test stand, and by test project, is the subject of the following sections.
Details, caveats, and additional data including results categorized by complex, project, and thrust scale can
be found in the parent document (NASA SSC Study #6323‐06‐007).

Total Number of Tests at SSC
The best available information indicates that over 3000 hot‐fire tests have been performed at NASA SSC.
The data presented is counted up to and including December 31, 2004.
The projects reflected in the graph of Figure C‐1, include Apollo rocket propulsion tests, which were all
booster stages with multiple engines. The SSME testing in particular for Space Shuttle has occupied
multiple stands over the past three decades. Also note that the testing at E Complex is relatively recent,
most of it occurring during the past 6 years (1999 – 2005). The quantity of tests there is quite large,
numbering over 700, and the statistics represent a remarkable variety of R&D testing involving many
types of test articles (PBs, thrust chambers, GGs, pumps, powerheads, and engines), and encompasses a
large span of thrust scale (from less than 1000 lbf to as much as 650,000 lbf thrust scale). Testing at
E‐Complex is shown as the sum of all tests at stands E‐1, E‐2, and E‐3, the large majority of them being
conducted at E‐3 with relatively smaller test articles.
The well‐known heritage testing of the Saturn V booster included: (a) 15 first stage tests at B‐2 stand, and,
(b) 7 and 23 second stage tests at the A‐1 and A‐2 stands, respectively. Although few in number, the test
articles were quite large and complex since they were actual flight hardware (in most cases) with clusters
of engines. Five J‐2 engines powered the Apollo second stage and five F‐1 engines powered the Apollo
first stage. The large test stands were built in large part to ensure that man‐rated Apollo booster stages
were adequately ground‐tested prior to delivery to Cape Canaveral for launch.
81
Notes:
• Number of tests shown in parentheses
• B‐2 also has 27 engine tests of Fastrac 60K‐
lb engine (not shown)

Figure C‐1: Total number of tests conducted (as of 12‐31‐04), by stand.

Total Number of Test Seconds at SSC
Figure C‐2 is the complement of Figure C‐1, showing the accumulated test seconds from all test projects
over the years. It indicates that at least 850,000 hot‐fire seconds have been accumulated at NASA SSC
across all the test complexes. The majority of test seconds are for the SSME engine, including SSME Main
Propulsion Test Article (MPTA) at 765,366 seconds. When SSME flight seconds from launch were included
into the SSME total, along with some testing at other test sites (NASA Marshall Space Flight Center
[MSFC], Rocketdyne Santa Susana Field Laboratory [SSFL]), the “1 Millionth second test” was completed at
SSC in February 2004 and celebrated as a public event.
Testing of integrated engines and stages at A and B stands has been performed with low‐pressure
propellant feed, and for long durations that are representative of flight profiles (SSME, RS‐68, Apollo first
and second stages). Testing duration was anywhere from 1.5 seconds to 2017 seconds. An SSME flight
profile is approximately 8 minutes, and therefore 500 or greater seconds of test duration is typical. The
on‐stand propellant run tanks are augmented by ground‐based barge‐fed propellants to achieve much
greater test duration when that is a test requirement.
At the E‐Complex, where ultra‐high pressures of 8000 psia are needed in the propellant run tanks, the
durations are limited by run tank size and pressurant gases (up to 15,000 psi). Typically, 5 seconds to 50
seconds of run duration is likely based upon the test article thrust scale and its development test
objectives.
82
1,000,000
Note: Scale is logarithmic.
333,380
272,699
190,581
100,000
54,297
Seconds
10,000
6,526
1,000
100
A-1 A-2 B-1 B-2 E-Complex
Test Stand

Figure C‐2: Total number of hot‐fire test seconds (as of 12‐31‐04), by stand.

Test Turnaround Rates at SSC Major Test Complexes
Figure C‐3 illustrates and compares the so‐called “test turnaround rate” for different types of testing.
Historically, test customers have not specifically planned for a particular optimum test turnaround rate,
and have generally accepted the outcome. A current test project at E‐1 [Integrated Powerhead
Demonstration (IPD) engine system] has systematically addressed its own achievable project test
turnaround rate to optimize the use of resources.
It should be noted that this rate is the average number of days between successive tests (or mean time
between Tests) and is counted only from the first day that a test campaign begins to the last test of the
series. The year or more that it may take to prepare the test stand for a new test project or series is not
included as it would skew the data. (The preparation time is a separate consideration.) At all stands
except E‐3 stand, it is rare for multiple tests to be conducted on a single day, and usually a second test is
only performed because the first attempt was deemed an unsatisfactory test attempt. At E‐3 stand, due
to the smaller scale and relatively lesser complexity of test article, the norm is that multiple tests are
conducted on a single day. Therefore the “test turnaround” is more correctly interpreted as the number
of days between successive test days (or mean time between Test Days). Henceforth it is referred to as
“Days between Test Days” for convenience.
Interestingly, owing to the large database on SSME testing, a time history trend for this statistic can be
shown indicating year‐to‐year variations in the test turnaround rates. When this trend was examined for a
given stand (either A‐1 or A‐2), it generally highlighted a rapid turnaround rate R&D period in the late
1970s, and a more nominal‐turnaround‐rate production testing period since the early 1980s. When all
SSME single‐engine data from all three stands is combined as shown in Figure C‐4, it is clear that the test
rate can be increased by commissioning multiple test stands. A test every four days was possible when
three test stands were active to support SSME engine tests between 1988 and 1997. In contrast, a single
test stand is, at best, likely to conduct a test every 10 days on average.
83
60.0
56.4
Note:
To date, test projects have
50.0
historically not planned for
specific test turnaround rates.
40.0
Days
30.0
Multiple tests
20.0
per day typical
at E-3
13.2
11.2
9.8 9.0
10.0
6.9
4.0
0.0
A-1 A-2 B-1 B-2 E-1 Test Stand E-2 Test Stand E-3 Test Stand
Test Stand

Figure C‐3: Average number of Days between Test Days, by stand.

20.0 Loss of
18.0 Challenger
Days Between Tests
16.0 (1986) Loss of

Columbia
14.0
(2002)
12.0 1 Stand
10.0 2 Stands
8.0 3 Stands
3 Stands
6.0 Operating in operation
4.0
2.0
0.0
1975 1978 1981 1984 1987 1990 1993 1996 1999 2002 2005
Year
Figure C‐4: SSME Days between Test‐Days, by number of stands operating.

Test Completion Rates at SSC
Figure C‐5 shows two different test achievement metrics. Firstly, the “Actual vs. Planned Seconds
Completed” is a percentage measure of planned total test seconds that were actually achieved during
hot‐fire. Second, the “Tests Reaching Full Duration” is a percentage of the tests that were completed to
full (planned) duration without a so‐called redline cut (a “redline” provides for a programmed test
shutdown based upon a measured instrument parameter). If planned full duration was not achieved, this
84
may have been the result of either a test article anomaly or a test facility anomaly, or other scenario.
Therefore, it should not be viewed as either characteristic of the facility, test article, or process, but
simply as an attribute of R&D testing. Sometimes, aborts are performed during tests to test systems of the
test stands and of the test articles.
From different perspectives, it is possible to measure testing achievement both by the percentage of test
seconds successfully completed, or by the completion of the full planned test duration. If achieving steady
state hot‐fire duration is an important test objective, i.e., for production engines which must demonstrate
a specific flight profile and accumulated run duration, then “Tests Reaching Full Duration” is relevant. It
might also be assumed that any redline cut represents some anomaly or vulnerability that must be
mitigated before testing resumes. However, many tests meet all the objectives of customers and/or
stakeholders without reaching full duration – the planned duration may have been somewhat flexible, or
just not as important. Also, as an engine component or system reaches maturity, the achievable duration
usually increases. In this way, the “Actual vs. Planned Seconds Completed” is a useful metric indicating
design maturity.

100%
93% 95%
88% 90% 89%
90%
84% 86%
79% 80%
80% 77% 76%
69% Actual/Planned
70% 65% Seconds
63% Completed
60%
Tests Reaching
50% Full Duration
40%
30%
20%
10%
0%
A-1 A-2 B-1 B-2 E-1 Test E-2 Test E-3 Test
Stand Stand Stand
Test Stand

Figure C‐5: Test seconds completion metrics, by stand
Note: Rocketdyne has chosen not to release data, which would lead to calculation of completion
metrics for the RS‐68, out of a desire to keep such performance measures proprietary.

Due to the wealth of SSME testing information, a yearly average was generated for test completion rates
for the SSME test campaigns at SSC. The time history trend in Figure C‐6 shows a rapid rise in completion
rate during the early years of the program in the mid‐1970s (at 45 to 75% completion), with an eventual
asymptote at 80 to 100% as both the engine itself and the test program matured.
85
100%
90%
% Completion
80%
Seconds
70%
Full Dur.
60%
50%
40%
1975 1980 1985 1990 1995 2000 2005
Year

Figure C‐6: Historical trend of SSME testing completion rate.

Conclusions
The following notable findings and conclusions are highlighted as a result of this study.
 The test turnaround time, or “days between test days,” is significantly greater for the larger scale
test articles, either engines or engine components.
 Multiple tests per day have been achieved only at the subscale testing level, or for small size test
articles, such as with several test projects at the E‐3 test stand. Interestingly, E‐2 test stand has
also conducted multiple tests per day during a couple of test projects.
 For SSME, it appears that a 10‐day turnaround time per stand has been typical over the years for
high‐completion‐rate testing. Quicker turnaround is possible, but is associated with a lower test
completion ratio.
 During peak testing periods, turnaround time can be minimized by testing a given engine system
on multiple test stands. This was the case when SSME was testing an average of once every four
days on any one of three SSC test stands.
 It is clear from the data that flight engine test activity has a higher test completion rate
(e.g., > 80%) than the low‐to‐mid‐TRL R&D testing conducted for early development.
 Early in the flight engine program, the test completion rates are relatively low (even less than
50%), for a variety of reasons, and improve over time as the engine development and test
experience matures.
 Rocket propulsion test facilities at SSC are adaptable to handling a variety of related, but perhaps
not strictly propulsion, testing activities.

The data collected and presented in this study reflects an enormous amount of unique testing experience,
especially in long‐standing production engine programs such as SSME. As stated earlier, the information
indicates a global average test turnaround capability of every 11 calendar days, with significant variation
around this mean depending on test article maturity, size, complexity, type, and other such parameters.

86
Appendix D: HumanRating Considerations for LRE
Testing and Evaluation
The foundation for any approach for human‐rating a propulsion system comes from referencing NPR 8705.2B,
“Human Rating Requirements for Space Systems.” “Human‐rating consists of three fundamental tenets:
1. Human rating is the process of designing, evaluating and assuring that the total system can safely
conduct the required human missions.
2. Human rating includes the incorporation of design features and capabilities that accommodate
human interaction with the system to enhance overall safety and mission success.
3. Human rating includes the incorporation of design features and capabilities to enable safe
recovery of the crew from hazardous situations.
Human rating must be an integral part of all program activities throughout the life cycle of the system,
including design and development; test and verification; program management and control; flight readiness
certification; mission operations; sustaining engineering, and maintenance / upgrades; and disposal.”
While these tenets guide programmatic approach, interpretation and design implementation of these tenets
reside at much lower levels (i.e., propulsion system, LRE, or even component level). Fundamentally, a
propulsion system must operate as part of a larger vehicle system. Therefore, it must be understood that
human‐rating for a given LRE cannot be completely addressed independently from the integrated vehicle
architecture. Integration of systems across the interface is a key in understanding how as system could fail,
and what mitigations can be incorporated as prevention or avoidance measures. More explicitly, because an
engine is considered human‐rated on one vehicle, does not automatically imply that it is human‐rated on any
vehicle.
For LREs, design characteristics fall into four primary categories:
1) Fault avoidance – design out the failure mode
2) Design margins or DFMR – design with increased margin to add sufficient robustness
3) Redundancy – design redundancy capability into the system to be tolerant to failures
4) Caution and Warning Devices – provide detection capability to detect, warn, and provide other
systems to activate or respond to avoid loss of crew scenarios
To implement these features appropriately, it is required to have knowledge of how the system can fail, and
how those failures can most likely propagate through establishment of hazards, FMEA, and CIL.
Although each engine will have different and distinct characteristics, there is a generally an accepted T&E
approach which can be summarized by an excerpt from “Point Paper on Space Propulsion Technology
Necessary to Enable Human and Robotic Missions” by Robert Sackhiem:
1) “Selecting a sufficiently large number of specimens of the new product design to be tested;
2) Conducting a sufficient number of tests on each prototype and flight‐like specimens to demonstrate
adequate operating and performance capabilities under nominal operational and environmental
conditions;
3) Conducting significant off‐nominal and/or overstress conditions for testing to demonstrate adequate
new product design margins;
4) Design the new product systems with as much redundancy as feasible to minimize single‐point
failures; and finally
5) Use a logical optimized combination of all of the above approaches to establish sufficient confidence
in the human‐rated capabilities, reliability, and safety of the new products and their associated
integrated systems under all anticipated operational conditions.”
87
Number 5, the use of a logical optimized combination of all the approaches is generally accepted. This brings
out the point that the T&E approach of the LRE must be carefully constructed and will be as unique as the
design characteristics, which must be logically tied back to the vehicle architecture and program.
It can be stated that the expectation is that the T&E must scrutinize the validity of the design features which
add to the reliability of the propulsion system and safety of the crew. Do they really add significant safety to
crew return and mission success? Do these features negatively affect the crew safety in any way (foreseen or
unforeseen)? Redundancy tests should be done beginning at the component level and follow through to the
propulsion system level, as necessary. Margin tests should be conducted as discussed in earlier sections for
qualification (expected operating box plus margin). Caution and warning systems should be tested as this is
software dependent, which there are established V&V processes that scrutinize each revision. Must work and
must not work functions should be tested to avoid inadvertent activation or deactivation.
Thorough test programs that consist of risk reduction activities, component, subsystem, and engine testing are
required for human‐rating. Significant understanding of the system must be acquired during the T&E stage to
provide the confidence that the propulsion system and how it will react in a given situation is very well
understood; over and above test‐based reliability analysis results (demonstration of reliability is typically too
costly). Reliability is not the sole characteristic that determines whether a system is human‐rated or not.
Human‐rating promotes the “test‐as‐you‐fly” approach discussed earlier, where production and operational
phases should fly within experience base of ground test. Further, it is important to have consistency in
manufacture and resulting build‐to‐build performance. Acceptance testing should thoroughly screen the LRE
for defects which can affect crew safety or mission success. Significant variations in build‐to‐build
performance (design specific) can indicate a lack of understanding of the characteristics that drive the system,
therefore reducing confidence. Statistically relevant numbers of tests are typically not available so an
acceptance series must provide an adequate screening function.
Although not explicit, the expectation of human‐rating adds rigor to all phases of the DDT&E process and
carries through production/operation.

88
Appendix E: Sample Cases of Using Test Guideline to Plan
Test Program

To be considered in future revisions.

89
Appendix F: Acronyms
AIAA American Institute of Aeronautics and Astronautics
a.k.a. also known as
ANSI American National Standards Institute
ASEE American Society for Engineering Education
ASI augmented spark igniter
ASME American Society of Mechanical Engineers
ATP acceptance test procedures
C* characteristic velocity
CG center of gravity
CIL critical items list
CPIA Chemical Propulsion Information Agency
Cv control volume
CxP Constellation Program
DAR deviation approval requests
DDT&E design, development, test and evaluation
DFMR design for minimum risk
DoD Department of Defense
DOD domestic object debris
DOE design of experiments
DOP detailed operating procedure
E&M electrical and mechanical
ECU engine control unit
EHM engine health management
ELV expendable launch vehicle
EMC electromagnetic compatibility
EMP electromagnetic pulse
EOM end‐of‐mission
ESD electrostatic discharge
FCA functional configuration audit
FEM finite element model
FID failure identification
FMEA failure modes and effects analysis
FOD foreign object debris
FCA functional configuration audit
FRT functional readiness tests
FoS factor of safety
GG gas generator
GTI ground test instrumentation
HCF high‐cycle fatigue
IEEE Institute of Electrical and Electronics Engineers
IOP ignition overpressure
IPD integrated powerhead demonstration
Isp specific impulse
lbf pounds force
LCC launch commit criteria
LCF low‐cycle fatigue
LDF life demonstration factor
LH2 liquid hydrogen
LL limit load
LN2 liquid nitrogen
90
LOX liquid oxygen
LRE liquid rocket engine
LRU line replaceable unit
MCC main combustion chamber
MEOP maximum expected operating pressure
MIL military
MMA moving mechanical assemblies
MOI moments of inertia
MOP maximum operating pressure
MPE maximum predicted environments
MPTA main propulsion test article
MR material review
MS margin of safety
MSFC Marshall Space Flight Center
MWP maximum working pressure
N square root
NASA National Aeronautics and Space Administration
NCSL National Conference of Standards Laboratories
NIST National Institute of Standards and Technology
NPR NASA procedural requirement
NPSH net positive suction head
NPSP net positive suction pressure
PB preburner
Pc chamber pressure
PCA physical configuration audit
psi pounds per square inch
psia pounds per square inch absolute
psid pounds per square inch differential
PSN purge sequence number
QA quality assurance
qual qualification
R&D research and development
RCS reaction control systems
RLV reusable launch vehicles
RTD resistive temperature device
RTCA Radio Technical Commission for Aeronautics
SAE Society of Automotive Engineers
SFI special flight instrumentation
SL service life
SRR systems requirements review
SSC John C. Stennis Space Center
SSFL Santa Susana Field Laboratory
SSME space shuttle main engine
STE special test equipment
T&E test & evaluation
TBD to be determined
TCA thrust chamber assembly
TOR Technical Operating Report
TPA turbopump assembly
TVC thrust vector control
V&V verification and validation
WR witness ring
91

Test and Evaluation Guideline For Liquid Rocket Engines

Uploaded by

Copyright:

Available Formats

Test and Evaluation Guideline For Liquid Rocket Engines

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Test and Evaluation Guideline For Liquid Rocket Engines

Uploaded by

Copyright:

Available Formats

The following notice applies to any unclassified (including originally classified and now

NOTICE TO ACCOMPANY THE DISSEMINATION OF

1. Export of information contained herein, which includes, in some circumstances, release to

6. AUTHOR(S) 5d. PROJECT NUMBER

5f. WORK UNIT NUMBER

9. SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S

Air Force Research Laboratory (AFMC)

Edwards AFB CA 93524-7048 AFRL-RZ-ED-TP-2011-401

Distribution A: Approved for public release; distribution unlimited (PA#11877).

13. SUPPLEMENTARY NOTES

15. SUBJECT TERMS

Test and Evaluation Guideline for Liquid Rocket Engines

Joint Army Navy NASA Air Force (JANNAF)

Title As of: Superseding: Page:

PAGE STATUS AUTHORITY

Initial Issue TBD

Former JANNAF LPS Test Practice and Standards Panel Members:

Other significant contributors from The Aerospace Corporation:

Jason Malburg Liquid Propulsion

Based on Failures to Qual for F-1, J-2, and SSME

ZERO FAILURE TESTING:

Test Time / Service Life

ZERO FAILURE TESTING:

4.3.3 Service Life (SL), Number of Starts   x x

4.5.2 Eng Control System Malfunction Logic Check

4.6.3 Post-Test Inspections x x x x

– Shutdown Propellant Conditions x

Appendix C: Representative Campaigns and Test Turn­

16.0 (1986) Loss of

You might also like

Appendix C: Representative Campaigns and Test Turn