Management issues in systems engineering

MANAGEMENT ISSUES IN SYSTEMS ENGINEERING N9 3 MANAGEMENT ISSUES by Robert Shishko with contributions Robert Aster, and by Vincent Robert Bilardo, IN SYSTEMS strategy. divided complex, conquered. Kevin Complex systems into pieces that until they are simple This decomposition several system structures and the system that structures engineering the Forsberg, are are of sucless enough to be results in system"). These play important roles in systems and project management. Many of the remaining sections in this chapter devoted to describing some of these structures. Structures tem include, /p for describing the product producing system ("the produces ENGINEERING G. Chamberlain When applied to a system, the doctrine successive refinement is a divide-andconquer cessively 8 2 are key Hal Mooz; "voice of the and buyer" Ron Wade is heard. Determining the right requirements -- that is, only those that the informed buyer is willing to pay for is an important part of the neer's job. Second, system communicate to the design design ments chy system which engialso what to of project, architecture and prodconsists of the hierar- systems, segments, subsystems, etc. The work breakdown the systems requirements engineers and build (or code). As these requireare allocated, they become inexorably linked to the uct breakdown, also that describe the product sysbut are not limited to, the re- Lou Polaski ,}_. a hierarchical pieces project. Each structure structure of work in the (WBS) that necessary task elements, contains to complete WBS is the should be quirements tree, system architecture and certain symbolic information such as system drawings, schematics, and data bases. The traceable to one or more of the system requirements. Schedules, which are structured as networks, describe the time-phased activi- structures ties that result in the WBS The cost account product system in the structure needs to be directly work that describe the tem include the project's schedules, cost accounts These tives desired mental structures on their product harmony producing sys- work breakdown, and organization. provide common different perspecraison d'etre: the system. among Creating a fundathese structures is linked schedules The describes perform structures are are represented needs to-one to be established correspondence trees; tures, across and in other cases, several structures. point, to give principle. some cases by onetwo struc- by traceable It is useful, illustrations links at this of this key System requirements serve two purposes in the systems engineering process. First, they represent a hierarchical the buyer's desired product stood by the systems description of system as under- engineer. tion between the buyer and to develop these requirements The interac- systems engineer is one way the one for project structure sponsibility Project particular mental that in WBS work and the is done. project's organizational structure clusters of personnel assigned the work. These organizational essential for successful systems engineering and project management; this harmony in some between to the by which usually trees. as a matrix for line Sometimes they of two interlaced responsibilities, responsibilities. should allow In any identification for each WBS task. documentation is the WBS tasks. There are categories of project to the other case, the of re- product of two funda- documentation: baselines and archives. Each category contains information about both the product system and the producing system. The baseline, once established, contains information describing the current system and producing state system of the product resulting from 35 R EADINGS all IN SYSTEMS decisions ENGINEERING that have been ally organized as a collection tree cant structures, amount and should of cross-linking. made. It is usu- of hierarchical exhibit The a signifiarchives should contain all of the rest of the information that is worth keeping, only temporarily. tain all analyses and ture The archives assumptions, data that are relevant future (and should decisions. Inevitably, control) of the archives structure of reviews ciated control gates) activities associated the product down. The structure progress their asso- provides information of those same activities. breakdown MANAGING PROCESS: nical function that nical systems functions and on the the fi- and cost accounts. On it should be linked to the and/or the requirements PLAN management discipline is a techthat ensures engineering and all other are properly applied. tech- aspect. This engineer perform the necessary decomposition, ification and orchestrating 36 prothe project cycle, the project-level or lead engineer concentrates on managing technical systems formed) requires (or cause multiple that incorporating the to be perlayers of definition, integration, validation of the system, and techniques used design failure in systems engineer- include baseline traceability, managechange certification. Project Plan defines how the overall will be managed to achieve the pre- established grammatic requirements constraints. within defined proThe Systems Engi- neering Management subordinate document project participants technically managed established communicates must ment should skills system reviews, audits, document review boards, control gates performance The project by respond practices. describe Plan that (SEMP) defines is the to all how the project will within the constraints the Project Plan. to all participants be The SEMP how they to pre-established manageFor instance, the SEMP the means for both internal and trol. external (to Role of the SEMP the SEMP is the all participants tailored to the project's risks. While the ject manager concentrates on managing its The ing management ment, requirements The Each project should be managed in accordance with a project cycle that is carefully overall systems Each one of functions re- the project) rule book interface con- ENGINEERING MANAGEMENT engineering On reporting and assessbe directly linked to THE SYSTEMS THE SYSTEMS ENGINEERING Systems of system from its product breakstatus reporting and assessment WBS, schedules technical side, product tree. cross feasi- engineering. engineering quires application of technical analysis and tools to achieve the optimum solution. and though where (and concurrent systems control, control, the strucis much reflect the time-phased with the realization nancial side, the status ment structure should the the con- and supporting to past, present looser than that of the baseline, references should be maintained ble. The project's even if priate these verwhile appro- nically Center how project describes will to be tech- managed. The responsible NASA should have a SEMP to describe ho_" it will conduct its and each contractor describe how it will with both its contract management project: and dated the that for technical should management, have a SEMP manage in accordance and NASA's technical practices. Since contract-unique, each to significant the SEMP it must be is up- programmatic change or it will become outmoded and unused, and the project could slide into an uncontrolled state. The NASA Center should have its SEMP developed before attempting to prepare a "should-cost" estimate, since activities that incur cost, such as technical risk reduction, need to be identified and described first. The contractor should have its SEMP MANAGEMENT developed to costing during the and pricing) proposal because ISSUES IN SYSTEMS process (prior the SEMP de- • • The role Therole scribes the technical content of the project, the potentially costly risk management activities, and the verification and validation • • The role of specialty Applicable standards • Applicable techniques included estimates. • • Baselinecontrolprocess Changecontrolprocess • • Interfacecontrolprocess Control of contracted • engineering Data control Control Design • • Make-or-buy control Parts, materials and etc., de- • Quality with it. and • • Safety control Contamination • EMI/EMC • Technical • • Control Internal • • Integration Verification • Validation to be used, all in the preparation of which must be of project cost The project SEMP is the management document for other technical control the Interface Control Plan, Make-or-Buy Review pend The Plan, on the SEMP Plan, Control Since nical of the such Change Plan, Audit Plan, and must comply be comprehensive describe how a fully effort will be managed Contents documents, Technical SEMP should senior technical the project; all as integrated engineering and conducted. SEMP the SEMP management describes the project's techapproach, which is driven by the type of project, the phase in the project cycle, and the technical development risks, it must be specifically written for each to address these situations and issues. the specific content to the project, the listed below. Part and of the SEMP recommended Program Planning section should identify responsibilities and authority engineering management, in- clude control of contracted els of control established and design requirements, method is tailored content is used; technical engineering; levfor performance and the control should of the of the procedures training (or subcontracted) process process control control control performance gates technical measurement reviews control control control. Engineering Process. contain a detailed de- of the process the specific quirements and process II -- Systems section should scription engineering tailoring of the to be used, including of the process to the resystem and project; the project user study methodology; the types ical and/or simulation models system cost-effectiveness the generation This section of specifications. should describe System decomposition • • System System decomposition format definition process • • System analysis and Trade study process describe: • • System System integration verification process process office • • System System qualification acceptance process process • • System validation Risk management • • The role The role • The role of the Contracting cal Representative (COTR) assurance for design and and control of Office Techni- the trade of mathematto be used for evaluations; • progress methods; plans and schedules technical program reviews; documentation. This section Part This of systems engineering of design engineering procedures to be used in implementing process; in-house documentation; the I -- Technical Control. This organizational for systems project While ENGINEERING and the: process design process process process 37 READINGS IN SYSTEMS ENGINEERING . • • Life-cycle cost management Use of mathematical models Use of simulations • Specification • • Baseline Baseline • • Change control process Tools to be used. Part drawing management communication III gration. and -- section ious parts breakdown Specialty of the SEMP should plines during into the systems engineering each iteration of that process. is potential for overlap forts, the SEMP should define responsibilities and authorities This section should contain approach of the disciprocess Where of specialty • ef- the relative of each. the project's • • plines The participation The involvement • The • disciplines The participation role engineering phasing of specialty and of specialty of specialty responsibility • • disci- • • • Reliability Producibility Human engineering • Maintainability • • Safety Survivability/vulnerability • • Integrated logistics Quality assurance. The SEMP must • disciplines • of the • of specialty in system decomposition and definition The role of specialty disciplines in verification and validation Development (See and sections network Lessons Learned • the project cycle. A SEMP is a living document that must be updated as the project changes and kept consistent with the Project Plan. A meaningful SEMP must be the product of experts from all areas of the project. Projects with little or insufficient systems engineering discipline generally have major problems. Weak systems engineering, or systems engineering placed too low in the organization, cannot perform the functions as required. The systems engineering effort must be skillfully managed and well communicated to all the individuals. The systems engineering effort responsive to both the customer contractor interests. The SEMP's development butions from and technical project that requires knowledgeable experts from can must be and the significantly influence the project's outcome. The involvement of recognized experts is needed to establish a SEMP that is credible to the project manager and to secure the full commitment of the project team. SEMP be developed Managing concurrently the Systems Engineering with the Project Plan. In developing theSEMP, the technical approach to the project, and hence the technical aspect of the project Systems engineering organizations, cycle, cifically project-level systems 38 contri- programmatic all areas of the Summary developed. on work sched- frorn DoD Experience Process: are the A well-managed project requires a coordinated SEMP that is used through disciplines disciplines of specialty of that work. structures SEMP • Concurrent The activity determines de- to: • • ultimately ules.) Inte- tbe integration and coordination of the specialty engineering that and cost of the project. The developof the programmatic and technical management approaches of the project requires that the key project personnel develop an understanding of the work to be performed and the relationships among the var- structure scribe efforts there project length ment process process Engineering This the process This becomes the keel of and engineers, speare MANAGEMENT responsible the technical for managing aspect of the responsibility position and includes definition the integration, sequence and baselines of the baselines "build-to" are (or coded"), and projects through project cycle. This managing sequence, the decommanaging verification controlling and the validation technical project. Typically, these the functional, "design-to," "code-to"), "as-built" (or "as"as-deployed." Systems neering must ensure efficient and progression through these baselines. Systems system design-to engineering items and design of all lower have logical is responsible decomposition specifications figuration engi- been for until level produced. the con- Design engineering is then responsible for developing the build-to and code-to documentation that complies with baseline. Systems design and coding the approved engineering process and design-to audits design the gineering solutions for compliance higher level baselines. In performing responsibility, systems engineering ensure requirements traceability and ment the results in a requirements ity/verification matrix. Systems engineering for the overall tion, verification this role, Readiness is also the en- to all this must docu- systems engineering conducts Reviews and ensures that Test only verified configuration items are integrated into the next higher assembly for further verification. Verification is continued to the system level, is conducted after which system to prove compliance validation with user requirements. Systems concurrent engineering engineering services. ject's with also ensures that is properly applied through the project cycle by involving the required specialty engineering. The SEMP is the guiding document for these activities. STRUCTURE A WBS breakdown work is a hierarchical level necessary to complete a project. are products is a cognizant is built The such branch point vice elements subsystems, At the lowest as hardware items, items which or manager. the PBS by integration (e.g., there Branch adding, at each and verification integrated logistics support (ILS). WBS elements require similar equipment or software, then a higher level element might be defined to perform a buy or a development activity (e.g., "System Support Equipment"). shows the relationship between PBS and a WBS. A project WBS cost account risks detail (PBS), at the of the PBS, any necessary sersuch as management, systems engineering, WBS block pro- hierarchy should show how the are to be integrated. The WBS from (I&V), and If several the structure product(s) engineer points in the PBS elements hierarchical associated contain product breakdown the specified prime management's costs, balanced Figure a system, should be carried level appropriate to be managed. The for a cost account desire against 1 a down to to the appropriate level of is determined by to have visibility into the cost of planning and reporting. Contractors may have a Contract WBS (CWBS), which is appropriate to the contractor's needs to control costs. A summary CWBS, consisting of the upper els of the full CWBS, is usually the project WBS to report costs tracting WBS tle and the • agency. elements • should by a numbering following Identifies Shows the the included in to the con- be identified system lev- that by ti- performs functions: the level Identifies which ed of the it should software items and information documents, databases, etc.) for • THE WORK BREAKDOWN As such, ENGINEERING and top, and the systems, segments, etc. at successive lower levels. the responsible IN SYSTEMS WBS should be a product-based, division of deliverable items traceabil- management of the integraand validation process. In ISSUES higher WBS the cost of the WBS level element account element element will number into be integratof the element. 39 READINGS IN SYSTEMS ENGINEERING other interested parties. It fully describes the products and/or services expected from each WBS element. System Components !subsystems) heldtogetherby "glue (integration) Subsystem A Subsystem Sub. This section provides developing a WBS, and takes to avoid. The whole does more than the sum of itsparts C some points techniques out some for mis- system B Subsystem Role D _Str WBS A product-based structure for: BPr°duct reakdown ucture Shows the components from which the tern I of the WBS is the organizing \ , Project and technical planning and sched- uling formedwas system • __J__ Sub Cost estimation and budget formuIation (In particular, costs collected in a product-based WBS can be compared to historical data. This is identified as a primary WBSs.) syste_ A Work Breakdown Structure (WBS) All work components necessary to produce a complete system The individual system components objective by DoD standards for • Defining the scope of statements of work and specifications for contract efforts • Project status reporting, including sched- ule, cost and work force, technical performance, integrated costschedule data (such as earned value and estimated cost _V__ at completion) • Plans, such mentation as the SEMP, and other docuproducts, such as specifica- Subsystem Managemeat A Systems Engin- {&V tions II_ and drawings. eering It provides a logical outline and that describes the entire project Work to produce the individual system components Work to integrate the components into a system The whole takes more work than the sum of itsparts grates there WBS, WBS Cost Figure 1 The Relationship between a System, a Product Breakdown Structure, and a Work Breakdown Structure should dictionary identification and other any that also a companion WBS contains each element's title, number, objective, description, dependencies WBS have elements. (e.g., This receivables) dictionary elements impacts are are most more likely to be affected. accurately estimated. and these elements can for potential adverse Techniques for Developing pro- [ be consulted r impacts. z the WBS on vides a structured project description that is valuable for orienting project members and 4O information in a consistent way. If is a schedule slip in one element of a an observer can determine which other If there is a design change in one element of the WBS, an observer can determine which other WBS elements will most likely be affected, A WBS vocabulary and inte- Developing a successful to require several project cycle since project WBS is likely iterations through the it is not always obvious at MANAGEMENT the outset may be. what the Prior WBS, there the system extent to developing of the work developed PBS can be created. and associated WBS level by approach, level from systems the a project-level then top recursive ists down An to that be down. systems enlevel, lower WBS is then derived by adding services such as management engineering of a lower approach hierarchical breakdown services would still apply. level. WBS services such gration and ex- Common all There Error will ele- but there fies each delivery. At any point will be both active and inactive the WBS. A WBS for an Operational a immediately that identi- 1: The A WBS makes one formally only the will be integrated. operations system and It would hardware verified then and all produced on a routine for an operational facility of information products or manfor that ele- instance, in a a distributed at lower levels of a WBS. be inappropriate to separate software as if they were sepa- rate systems to be integrated at the system level. This would make it difficult to assign for integration the costs of integrating nents of a system. Error 3: The WBS and and the PBS. This makes it possible will not be fully implemented, avoided niques by using described one compo- its with that the PBS and generally process. errors are prevents the WBS above. to identify testing is inconsistent performing and organizing. in the PBS are once and then project there is typically software assohardware items that will be inte- 2. Each are in functions, responsible For with successfully planning but found describes This accountability in time there elements in Facility. a WBS errors in Figure A PBS consist inte- not be needed. common difference is that not necessarily basis. might as maintenance to the PBS, and may complicates the management Some examples of these integrated, a same, engineering, WBS for managing an operational facility such as a flight operations center is analogous to a WBS for developing a system. The the products completed the Error 2: The WBS has branch points not consistent with how the WBS grated will be one ex- hierarchy product(s) are architecture, ciated with multiple deliveries, are "rapid development," "rapid prototyping" and "incremental delivery." Such projects should also have WBS prime such added are in Developing three products. ments flight a Multiple Delivery Project. terms for projects that provide WBS, for an operational for developing facility verification are ager the products. tra level in the under the final to a development as systems Errors not product-based and/or WBSs: activWBS. are included, and all assembly/ and verification branches are A WBS for Some of the apply for an operational all products integration who WBS of products WBS also apply to a WBS facility. The techniques When this approach is taken, it is necessary to take great care to develop the PBS so that correct. The involvement of people be responsible for the lower level ments is recommended. that cusof a This is to define levels of a complete PBS in one design ity, and then develop the complete rules ENGINEERING provided to external the general concept except that services and user support are apand process is repeated until a WBS to the desired cost account level. alternative products However, The can gineer finalizes the PBS at the project and provides a draft PBS for the next level. The propriate service tomers. a preliminary should be some development architecture to the point where preliminary The PBS In this full ISSUES IN SYSTEMS the roles These WBS shown from in project errors are development tech- 41 READINGS IN SYSTEMS ENGINEERING _ Error I I Functions without Products Error 2 Inappropriate This WBS describes only functions, not the products I Error 3 ] This WBS has branch points that are not consistent with the way the WBS elements will be integrated Inconsistency This WBS Product with PBS is inconsistent Breakdown with the Structure := :_'_':--_ •-_-_ -_i ..............._-.>_......×. ;;.......................... _:_._::.:'.'...:_.._.+:+z_,_ _:_ _._._ _._._.z.:_-.-_:_ _.:_ -.:-: ....:_._:_:._:y2_'_.:,':_: :i_%,.'::::::i:i:::i:::i:i::>.'::.:_.:._:_:_j __2 _,::">.':_.':':_:;-_: 1 [_,,.':_::_:5:'::':-: :i:_:i:(-:_:>._: _:!::_:_$:: :): ..+.!:8-)_t>. :._ ........................ -_..................................... _ _ -- . Branches :_:: _: _: _,>.;y_N _?_._ :_ ............................... ;-:.:.:._ _:_,_-.:.:_:_:.._.._ __×-:_:_:-:'-_._-_ N_K_ |i::: _:_:>._ :_ _1: :_): ::_ :_:: :_:: ::'_g::.S_._t_ _i.......................... -.................................................._>.._,...._...._...× ..........._ Subsystem _i-..._-_.-,%/,',_ ................ :_.:::.:: >__>:__¢c¢.:_:,.,:_: :_;_. :_:_-_;_.: :-:-:.:-_:::-_ :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :_. ::::::::::::::::::::: ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ,.-_ _.:_.:...-:,>: <_.:._ _.v_ ×:_ ..======================================= i__:_::_:-_;_:_;_:;_._;_:_:_._:_:_._:_:_:_:_:_ - .;_ .%'xXJL .u..v.v......-.u.u.v.v - .v. xL L x:Lv_. X.T.V::LU.: - :::?.=::?XT.:U._ ._ A.. J=...... :: ::: .'.-:_::>- _f i._:.":_:!:i[]t Transm,tter The _*:_._._;_.i._;$:_i,i:;*_;l TWT Amphfier Work Breakdown 2 The Examples of WBS SCHEDULING Products described that orderly efficient and process requires place in a way take WBS time are the Development that these that respects An engineering activities take the underlying calculate project, ration much [:I z_'rAmplin,, Breakdown [ Structure Errors and how each ject WBS might complete network result to complete. systems [4 Product it will take, in the of activities H Structure Figure NETWORK I:;:;:;:;:_:;:;:!:_:_:l how which (i.e., spare element of the pro- affect other elements. schedule can be used A to long it will take to complete a activities determine that du- critical path activities), time (i.e., float) exists and how for all the time-precedence relationships among them. This is accomplished by creating a network other activities of the project. An standing of the project's schedule schedule, which explicitly takes into account the dependencies of each activity on other activities and receivables from outside sources. prerequisite for accurate project budgeting. Keeping track of schedule progress is an essential part of controlling the project, be- This cause section and the network discusses techniques schedule. Scheduling planning project. schedule standing 42 the role of scheduling for building is an essential a complete component needs to be done, how technical problems often show up first as schedule problems. Because work schedules show how each activity of and managing the activities of a The process of creating a network can lead to a much better underof what cost and underis a long netaf- fects other activities, they are essential for predicting the consequences of schedule slips or accelerations of an activity on the entire project. help Network managers scheduling accurately systems assess the also impact MANAGEMENT Critical Path and quence of activities collectively have float. If there is a delay in an activity in this sequence, then the path float for all subsequent activities is reduced by that amount. Free float exists when a delay in an activity will have no effect on any other activity. For example, if activity A can be finished in 2 days, and activity B requires 5 days, and activity C requires completion of both A and B, then A would have 3 days of free float. Float is valuable. Path float should be conserved where possible, so that a reserve exists for future activities. Conservation is much less data items. When ule, graphical useful. Two mats, shown cost and schedule Network Formats resource changes Data schedule data consist gram Evaluation (PERT) chart. The and Review second called • Dependencies • where an activity depends activity for a receivable) Products or milestones that • Duration of each activities activity. de- Technique precedence diagrams, has boxes that represent activities; dependencies are then shown by arrows. Due to its simpler visual format and reduced requirements on precedence mon computer diagram in recent Activity-on-Arrow "Y has resources, become the more com- years. Diagram 5 Activity A has been "artificially" broken into two separate activities. "r ! Activity Description Activity Duration #e.g., days) Precedence Diagram A Activity Description on the Activity Duration (e.g., days) SS5 Note: Each activity's description shouldcontain an action and the object of that action. i ! B Graphical 3 Activity-on-Arrow Diagrams for Network The precedence for simple depiction Activities of one or more and end of the of the Pro- and Precedence Schedules of: • sult sched- ThismeansthatActivity B start 5daysafter cab Activity A starts. and between products beginning and typical format Figure Network with pendencies at the arrow. This is the of a project. Schedule a network formats of these data are very general types of graphical forin Figure 3, are used. One has ]'_ and creating activities-on-arrows, important for free float. To determine the critical path, there is first a "forward pass" where the earliest start time of each activity is calculated. The time when the last activity can be completed becomes the end point for that schedule. Then there is a "backward pass," where the latest possible start point of each activity is calculated, assuming that the last activity ends at the end point previously calculated. Float is the time difference between the earliest start time and the latest start time of an activity. Whenever this is zero, that activity is on a critical path. technical ENGINEERING A work flow diagram (WFD) is a graphical display of the first three data items above. A network schedule contains all four Float Calculation The critical path is the sequence of activities that will take the longest to accomplish. Activities that are not on the critical path have a certain amount of time that they can be delayed until they, too are on a critical path. This time is called float. There are two types of float, path float and free float. Path float is where a se- of both ISSUES IN SYSTEMS activities upon occur (e.g., diagram of the format following allows logical relationships: another • Activity B begins when (Start-Start, or SS) • Activity B begins ends (Finish-Start, as a re- Activity only after or FS) A begins Activity A 43 READINGS • IN SYSTEMS ENGINEERING Activity B ends when (Finish-Finish, or FF). Each of these three may be modified to the relationship, It is possible Activity activity A ends • relationships by attaching a lag ( ÷ or - ) as shown in Figure 3. to summarize a number of low-level summary lationship activity activity and that extended nificant reports, low-level activities in a precedence diagram with a single activity. This is commonly referred to as hammocking. One takes the initial Ensuring attaches • For the cost account each product, listing the for its generation and steps within the work steps re- drawing process as a work flow diagram. Indicating the dependencies products, and any integration cation is downward to describe all sigproducts, including documents, hardware and software items. quired • WBS the among the and verifipackage. a activity to it using the first redescribed above. The summary is then attached to the final low- Step 2: Identify dependencies. any receivables and negotiate external External dependencies from outside of the cost are ac- level activity using the third relationship described above. Unless one is hammocking, count, of the the most common relationship dence diagrams is the second should occur to ensure that there is agreement with respect to the content, format and labeling of products that move across cost above. The represent used in preceone mentioned activity-on-arrow the identical as a precedence cial events and Establishing format time-precedence diagram activities by creating as needed. a Network can logic artifi- Schedule in the upper levels network schedules of the WBS. To that are consis- tent with the project's objectives, the following six steps are applied to each cost account at the lowest available level of the WBS. Step 1: Identify activities and dependen- cies needed to complete each WBS element. Enough activities should be identified to show exact schedule activities and other uncommon tified for that will Typically, the current dependencies WBS elements. to have about 100 activities and much subsequent years. Each updated with additional rent year. complished between It is not iden- the first year of a WBS element require 10 work-years per year. there is more schedule detail for year, This by: first step less detail for year, schedules detail for the are cur- is most account ensure boundaries. that lower This level step is designed schedules can to be integrated. Scheduling begins with project-level schedule objectives for delivering the products described develop and any deliverables that go outside cost account. Informal negotiations easily ac- Step 3: Estimate durations of all activities. Assumptions behind these estimates (work force, availability of facilities, etc.) should be written down for future reference. Step WBS gram 4: Enter the schedule data for the element into a suitable computer proto obtain a network schedule and an estimate (There software of the critical are many packages path for that element. commercially available for this function.) This step enables the cognizant engineer, leader, and/or systems engineer to team review z the schedule logic. It is not unusual at this point for some iteration of steps one to four to be required in order to obtain a satisfactory schedule. Reserve will also be added to critical path activities, dummy activity, commitments can ment. often in the to ensure that be met for this Step 5: Integrate schedules WBS elements, using suitable that ments all dependencies are correctly between included form of a schedule WBS ele- of lower level software, so WBS in a ele- project =- 44 -- MANAGEMENT ISSUES IN SYSTEMS ENGINEERING network. It is important to include pacts of holidays, weekends, etc., the imat this point. The critical path for the project covered at this step in the process. Step 6: funding justments force Review levels able. tions the work and funding Adjustments of activities to the levels targets activities, activities or finding or the include element, increasing that are ways parallel, rather the project-level justed, are • at the • • the work force on the critical activities in than in series. If necessary, targets may need to be adscope of the project may need to be reviewed. Again, it is good practice to have some schedule reserve, or float, as part of a risk mitigation strategy. ble The product of these last steps is a feasibaseline schedule for each WBS element that is consistent other these with the activities WBS elements, and schedules is consistent technical scope and the goals for the project. There should be enough float in this integrated master schedule so that schedule and associated cost risk are project and to the project's when WBS ed, this is done, time elements will have or work start sumed on some acceptable customer. elements will been originally of receivables. sequently, replanning is almost needed to meet the project's goals. Reporting Summary described which output to the Even estimates for many been underestimat- WBS as early as had due to late arrival not asCon- always recent For example, changes in float is usually example of a project of key manager type of the float activities. may 4 illustrates A heading that describes the WBS element, the responsible manager, the date of the baseline used, and the date that status was reported. A milestone section in the main body (lines 1 and 2). An activity section in the main body. Activity data: a. WBS elements (lines 3, 5, 8, 12, 16 and 20) b. Activities (indented from WBS elements) c. Current plan (shown as thick bars) d. Baseline plan (same as current plan, or if different, represented by thin bars under the thick bars) e. Status line at the appropriate date f. Slack for each activity (dashed lines above the current plan bars) g. Schedule slips from the baseline (dashed lines below the milestone on line 12) A note section, where the symbols in the main body can be explained. This Gantt summary worked for to tailor the pertinent know wish to chart shows only 23 lines, which is a of the activities currently being this WBS element. It is appropriate amount of detail to those items most at the time of status precisely been and how consumed whether consumed or latest reporting are on the Good capabilities scheduling to show time, and reporting. much schedule by critical reserves being period. information reserve. over is shown in Figure 4. Another format is a table that shows and • has ties, Techniques data about a schedule in Gantt charts, a good Charts of all the sum of all with both the schedule in Gantt and adding more deleting re- to do more Features The Gantt chart shown in Figure the following desirable features: reason- established project-level. This may activities to some WBS dundant for some level make final adso that work to the logic and the duramay be needed to conform schedule path, force profile over time, and to logic and durations is dis- Desirable rate preserved This table of change include to make profiles, shows work important an example activibeing in the provides of schedule systems provide resource requirements adjustments the schedule is feasible resource constraints over may reserve path are with time. force level, facilities, of an etc. unleveled so that respect to Resources funding Figure 5 resource profile. The objective is to move the start dates of tasks that have float to points where 45 READINGS IN SYSTEMS ENGINEERING Space Science& Instruments Approval Level Subsystem 3 Manager STIKSCAT System (Level 2) Achievement (Level Assembly 3) Statusas of:Jan 20,1991 RevisionDate: Dec 23,1990 tLevel 4) Level 4 Mana_r ] 1990 ACTIVITY NOV DEC Milestones - Subsystem _FSOR 2 1991 FY91 OCT t JAN • I DR i QuarterlyAssessments System Engineering 6 Assembly 7 Subassembly 8 Design 10 Fabricate 11 Test APR MAY [ JUN VCDR JUL CDR _ ] [ • rqmt.. 1 V : _ F REC REQ IS AUG SEP DELi .... • V IM Design Subassembly 9 MAR I 3 :Management 5 FEB •PDR - Assembly 4 PROJECT 1 • F I •i F I #1 le _l ........ 9 TO I&T i i u kT ..... 1 [ .. 12 Subassembly 13 Design 14 Fabricate 15 Test #2 ! [i J ..... • TO I&T II II £ 16 Subassembly 17 Design 18 Fabricate 19 Test #3 i ........ 20 Integration & Test 2t Plans 22 Procedures 23 Integrate • TO I&T i i '_ i i 1I I lima q REC •F', x7 ALL SUBASSY ..... • F V FIXTURE & Test ' l I NOTES: FLOATPositive or Negative - is shown above the activity bars and event symbols. The BASELINE plan is shown below the current plan, if they differ. This assembly isforthe PFM (WBS 49801) Assemblies forFM1 (WBS 49802) and FM2 (WBS 49803) are on Pg 2/2. Figure 4 An Example the not resource profile sufficient, then tions is the for resource-intensive BUDGETING Budgeting baseline 46 accordingly, AND RESOURCE and resource establishment budget and If that is task dura- activities be re-examined and, source levels changed. the feasible. assumed should the re- PLANNING planning involves of a reasonable project the capability to ana- of a Gantt Chart lyze changes to that baseline resulting from technical and/or schedule changes. The project's WBS, baseline schedule and budget should be viewed by the systems mutually dependent, reflecting engineer as the technical content, time, and cost of meeting the ect's goals and objectives. The budgeting process needs to take proj- account cost profile exists, whether a fixed exists. When no a baseline budget cost cap such cap or is developed or into profile from MANAGEMENT descoping the ISSUES project's goals design, and/or requirements, Note: Actidties j resulting in f violations of Units tion exists, -t3- res, •,, %d,u!ed, they {{ Resource 10 Limit " ! iiii !:!!i:I_i_ t |:!il:i: ii'i_" _i:|_il;iii !_ Time 0 ,::; !_ F:i :If i]: 20 ': i::|:{ii: :.:i 5 An Example of an Unleveled Profile WBS cally force and network schedule. involves combining and other resource appropriate work force cial and element programmatic estimates. and other finan- factors to obtain cost These elements of cost include: Direct • • Overhead costs Other direct costs • • • ing, etc.) Subcontract costs Material costs General and administrative • Cost • if applicable) Fee (if applicable) • Contingency there can ning made process. whether ule are after important At are included avoidance the and project worklevel, planning must also level of contingency to deal with unforeseen the Effect of Schedule Slippage Certain elements of cost, called fixed costs, are mainly time related, while others, called variable costs, are mainly product related. If a project's schedule is slipped, then the fixed costs of completing it increase. The variable costs remain the same in total (excluding inflation adjustments), but are deferred downstream, as in the figure below. $ (travel, (i.e., data processvARIABLE costs interest payments, FIXED T NOW cap or a fixed cost additional logic gates that before the systems engi- complete feasible costs An DEFERRED is a cost profile, there are must be satisfied neer risk costs of money profile II • When labor cost specifi- the project's work needs with the rates implementa- events. Resource This objectives, to control baselined. strategies. Assessing the and or fixed budgeting and resource ensure that an adequate Lii 30 funds Figure been as developing around : cap it is important have such ii:i i: "i-: "ii_ [;i; :i :. i: [ 10 a cost ENGINEERING aspect of cost control is project cost and schedule status reporting and assessment. Another is cost and schedule risk planning, -- [ i |i:.i!i_{ fililili!!::::-!:::: ;) H:!:lil i_: ; _ approach. Whether IN SYSTEMS the budgeting A determination the WBS and with cost caps and/or cost tems engineer needs and needs network respect to mandated profiles. If not, to recommend approaches for either stretching (usually at an increase in the planto be sched- the the sysbest out a project total cost) or To quickly assess the effect of a simple schedule slippage: • Convert baseline budget plan from nominal (real-year) dollars to constant dollars • Divide baseline budget plan into fixed and variable costs • Enter schedule slip implementation • Compute new variable costs including any work force disruption costs • Repeat last two steps until an acceptable implementation is achieved • Compute new fixed c_sts . Sum new fixed and variable costs • Convert from constant dollars to nominal (real-years) dollars. 47 READINGS IN SYSTEMS ENGINEERING RISK MANAGEMENT Risk management thought to the mitigation ward its comprises purposeful sources, magnitude and of risk, balanced management There and actions directed reduction. As such, is an integral part torisk of project management, and contributes directly objectives of systems engineering. are engineer Principal a number pleting a well-conceived process. activities is shown The first is the characterizing tive of this Risk NASA policy project risks are Risk Management • • objectives with Provide a disciplined and proach to risk management the project cycle Support regard expressed in NMI Policy. These are management decision of occurrence and separately, quences without methods ap- by • significance the decisions and safety to NASA of assessed made with concerns) management risk levels respect hence, are these used in and (e.g., high, according medium, or low), to severity of forms the by their rela- with both high adverse conse- ranked higher than those characteristics. The primary in this process are qualitative; engineering literature, (by phase) attention. to be given enough to elucidate the magnitude are not of or to allocate scarce risk reducRisk analysis is the process of quantifying both the likelihood of occurrence and consequences of potential future events (or "states of nature" in some texts). The Risk Analysis Figure 6 Risk Management 48 among This In some projects, qualitative methods adequate for making risk management decisions; in others, these methods are I and Characterization and by categorizing (in a consisuncertainties by the likelihood significant risks specific management Risk Management IRiskItioo / in Figure 6. process of identifying the project's risks. The objecis to understand what uncer- systems the problem, tion resources. to them. of these this step is sometimes called qualitative risk assessment. The output of this step is a list of precise the management structure tive riskiness. Uncertainties likelihood and severely providing integrated risk assessments (i.e., taking into account cost, schedule, performance Communicate The consequences. This categorization basis for ranking uncertainties to making risk systems objectives. and com- project faces, and which be given greater attention. is accomplished tent manner) 8070.4A, to: documented throughout step tainties the them should The term risk has different meanings depending on the context. Sometimes it simply indicates the degree of variability in the outcome or result of a particular action. In the context of risk management during the systems engineering process, the term denotes a combination of both the likelihood of various outcomes and their distinct consequences. The focus, moreover, is generally on undesired or unfavorable outcomes such as the risk of a technical failure, or the risk of exceeding a cost target. the program. Such a program encompasses several related activities during the systems engineering to the of actions can take to effect these among them is planning Structure MANAGEMENT ISSUES IN SYSTEMS ENGINEERING systems engineer risk identification needs to decide whether and characterization are adequate, or whether the increased of risk analysis is needed for some precision uncertain- ties. In making that determination, the systems engineer needs to balance the (usually) higher cost of risk analysis against the value of the additional information. Risk mitigation is the tion and execution economically reduce tivity of these strategies another the systems Heisenberg is also type. the project (Some The of risk considered Risk mitigation is efforts and expentype of risk may engineering Uncertainty tum mechanics). trade one type selec- of strategies designed to risk. Tracking the effec- part of risk mitigation. often a challenge because ditures to reduce one increase formulation, have called equivalent Principle and the of the in quan- systems needs niques that best fit the of each project. A risk management throughout the doctrine focus, however, in the (Phases early A and ing product C and D). ations again. the unique moves from concurrent "big Contingency planning Risk templates (e.g., DoD 4245.7-M) Probabilistic network schedules (e.g., PERT) Critical items/issues lists Lessons Probabilistic cost and effectiveness models (e.g., Monte Carlo models) Cost/schedule control VeSte ms and chnical Performance Measure (TPM) tracking and development pre-operations E and F), the risk management phase with its words, address plan, which elaborates SEMP of the and project a of Risk Management should cycle, at each • The programmatic aspects management activities (i.e., of the risk responsibil- ities, and • overall be updated contains: The project's tives • future part of for a project management 1 Techniques resources, tones, etc.) A description to be used • risk mitigation A description policy schedules of the for risk characterization, (Phases and oper- engineering. That (PRA) • tech- focus changes program is activities in a risk Risk Mitigation and Tracking Assessment FMECAs/ FMEAs/ Digraphs picture" ongoing risk issues and As such, it is a natural plan. Probabilistic Risk phases of the project cycle B) to more specific issues dur- Risk management should be documented program Independent assessment (cost, schedule and technical) is needed always forward-looking. In other risk management program should the project's uncertainties. the In keeping refinement, the Watchlists/ milestones on the requirements program project cycle. of successive design During (Phases A good to choose Decision analysis Table Several techniques have been developed for each of these risk management activities. The principal ones are shown in Table 1. The engineer Expert interviews Analysis engi- neer need to understand the system-wide effects of various strategies in order to make a rational allocation of resources. systems Risk learned i]les from previous projects this ability (or necessity) to for another means that manager Risk Identification and Characterization risk of the and objec- miles- tools and techniques identification and analysis, role and risk of risk manage- ment with respect to systems baseline change control, formal analysis, reviews, and status reporting and assessment Documentation requirements for risk The should overall management level be risk of risk product management consistent with policy established and each action. activities the project's in conjunction with its NASA Headquarters program office. At present, formal guidelines for the 49 READINGS IN SYSTEMS ENGINEERING classification of projects with respect to overall risk policy do not exist; such guidelines exist only mulgated NASA Payloads, Types There for NASA payloads. These are in NMI 8010.1A, Classification Attachment proof A. ways to describe ious types of risk a project engineer faces. Traditionally, the var- manager/systems project manag- ers and systems engineers have attempted to divide risks into three or four broad categorcost, safety More recently, con, including schedule, (and/or technical, hazard) others have the categories of project categories categories. also For supportability, risks. These expanded set of managers engineers who must NASA environment. operate Some represent example, and risks. entered the lexiof organization- al, management, acquisition, political and programmatic newer categories reflect the concerns first kind of uncertain- the unpredictability requirement for Freedom designs. is stochastic of the alternative While the in any particular logistics cycle, the probability distribution can be estimated for each design from reli- several ies namely, sometimes, of the ty occurs in spares upmass Space Station requirement of Risks are An example and in the of these systems current newer supersets of other the Defense Sys- ability of the theory second and empirical data. kind of uncertainty Examples occur in trying to predict whether a Shuttle accident will make resupp]y of Freedom impossible for a period of time greater whether life on Mars exists. Modern subjectivist than x months, or (also known as Bayesian) probability theory holds that the probability of an event is the degree of belief that a person has that it will occur, given his/her state of information. As that information improves (e.g., through the tion were known. In previous paragraph, then, is the probability the examples the only estimator's state tivists of information. Consequently, find the distinction between kinds of uncertainty political risks" programmatic significance. tivist's view useful in informal the broader category While these terms discussions, to be no formal taxonomy One reason, mentioned there project appears free of ambiguities. above, is that often one type of risk can be exchanged other. A second reason is that some categories cost risk of are move together, and political risk for anof these as for example, (e.g., the risk of cancellation). Another way some have categorized risk is by the degree of mathematical predictability in its underlying uncertainty. The distinction has been made between an uncertainty distribution, that has a known probability with known or estimated parameters, probability and one in which the underlying distribution is either not known, or its parameters quantified. 5O cannot be objectively even with of little of the difference, perceived tems Management College (DSMC) Systems Engineering Management Guide wraps "funding, schedule, contract relations, and into risks. acquisi- tion of data or experience), the subjectivist's estimate of a probability should converge to that estimated as if the probability distribu- subjecthe two or no practical The implication of the subjecfor risk management is that, little or no data, the systems engineer's subjective probability form a valid basis for risk decision Risk Identification estimates making. and Characterization Techniques A variety of techniques is available identification and characterization. for risk The thoroughness with which this step is accomplished is an important determinant of the F risk management program's Expert Interviews. ducted, expert source of insight ject's One success. When interviews and properly can information risks in the expert's key to a successful be cona on the major pro- area of knowledge. interview is in E MANAGEMENT identifying a risk issue an expert who is close enough to to understand it thoroughly, and at the same time, able (and willing) to step back and take an objective view of the probabilities and consequences. success is advanced of the interviewer. A second preparation This means as they apply to the project, oping methods for capturing acquired during the interview. the Initial interviews may yield tive information, which should follow-up rounds. Expert used to solicit quantitative ENGINEERING caution, risk templates an exhaustive list of risk every project, but risk identification. they are cannot issues for a useful input to to on the part having a list of risk issues to be covered in the developing a working knowledge issues key general provide ISSUES IN SYSTEMS Lessons Learned. learned files, A review data and of the reports lessons from previous interview, of these similar projects can produce insights and information for risk identification on a new and project. devel- information only qualitabe verified in interviews are also data and infor- For an example, technical risk it makes sense identification, as to examine pre- vious projects of similar function, architecture or technological approach. The lessons learned ellite Space from the Infrared Astronomical (IRAS) project might Infrared Telescope Sat- be useful to the Facility (SIRTF) mation ly rank for those risk issues that qualitativehigh. These interviews are often the project, even though the complexity is significantly major models later. source of inputs to risk built using the techniques to applying this technique is in recognizing what aspects are analogous in two projects, and what data are relevant to the new analysis described project. Even if the Independent Assessment. This technique can take several forms. In one form, it can be a review of project documentation, such as learned from previous statements component cation SEMP. of work, acquisition plans, tion of the WBS for completeness tency with the project's schedules. form, an independent assessment independent from verifi- plans, manufacturing plans and the In another form, it can be an evalua- cost an outside (and/or agency and consisIn a third can be an schedule) and/or estimate group. cable at the valuable data the iatter's greater. degree The documented projects of key lessons are not appli- system level, there may applicable at the subsystem be or level. FMECAs, FMEAs and Digraphs. Modes, Effects, and Criticality (FMECA), Failure Modes and Failure Analysis Effects Analy- sis (FMEA) techniques identification and digraphs are specialized for safety (and/or hazard) risk and characterization. These techniques focus on the hardware compo- Risk Templates. This technique consists of examining and then applying a series of previously developed risk templates to a current nents that make up the system. According to MIL-STD-1629A, FMECA is "an ongoing procedure by which each potential failure in project. particular methods a system Each template risk issue, for avoiding generally and then or reducing covers a describes that risk. is analyzed or effects thereof fy each potential to determine the on the system, failure mode The most widely recognized series of templates appears in DoD 4245.7M, Transition its severity." fied into four from Development the Risk Equation. • Category I - Catastrophic • ble death Category or system loss) II - Critical Failure responses learned enough described to Production... Many of the risks are based on Solving and risk lessons from DoD programs, but are general to be useful to NASA projects. As a major Failures severity injury are generally categories: or system results and to classiaccording to Failure classi- (possi(possible damage) 51 READI-NGS IN SYSTEMS Category ENGINEERING III minor injury radation) - Major Failure or mission (possible effectiveness Category IV - Minor Failure system maintenance, but does hazard to personnel or mission ness). deg- (requires not pose a effective- common to much of systems engineering, a complex uncertainty is decomposed into simpler ones, which are then treated separately. The decomposition a level at which be brought effectively. cally A complete mate FMECA of the also probability includes of each an esti- potential fail- ure. These probabilities are usually based, at first, on subjective judgment or experience factors from similar kinds of hardware components, but may be refined data as the system An FMEA is similar cally excludes category. from reliability development progresses. to an FMECA, but typi- the severity fault low, if quantitative needed. Risk Analysis trees, probability Digraphs resemble a described be- estimates are Techniques The tools and techniques heavily on the concept axioms and theorems) of risk analysis rely and "laws" (actually, of probability. The systems engineer needs to be familiar these in order to appreciate the full and limitations of these available software measures. The decision can also In brief, allows: that Decision technique maker ties. 52 Analysis. to help deal Using with the risks, allocating and risk re- Decision analysis is one the individual decision a complex set divide-and-conquer be of uncertainapproach analysis calculate decision analysis at of of dollar derived valfor complex dein currently software. a variety This of risk is a technique A systematic enumeration ties and encoding of their and outcomes • An explicit sion maker's • upper • of uncertainprobabilities characterization of the deciattitude toward risk, ex- bound expenditures Sensitivity mates and seeks on testing outcome Risk to measure information-gathering on probability dollar values. Assessment the risk esti- (PRA). inherent system's ing both design and operation by the likelihood of various accident A typical sequences and their consequences. PRA application is to determine the risk associated power plant. Within to = pressed in terms of his/her risk aversion A calculation of the value of "perfect information," thus setting a normative understanding dominant The assigned dollar value • PRA for tree. E Probabilistic of the a decision In most applications of decision analysis, these outcomes are generally assigned dollar products of risk analyses are generally quantitative probability and consequence estimates for various outcomes, more detailed improved capability duction resources. as branch points, called nodes, in a decision tree represent either decision points or chance events. Endpoints of the tree are the potential outcomes. with power techniques. represented can function can be graphi- each set of decisions. Even large, cision trees can be represented schematic diagram. The digraph technique permits the integration of data from a number of individual FMECAs/FMEAs, and can into to bear, or intuition The decomposition each outcome, the distribution ues (i.e., consequences) can Digraph analysis is an aid in determining fault tolerance, propagation and reliability be translated until it reaches information can values. From the probabilities each chance node, and the classification in large, interconnected systems. exhibit a network structure and continues either hard demonstrate, for with a specific NASA, PRAs example, the A in a quantifypossible nuclear are used relative MANAGEMENT An Example of a Decision Tree for Robotic ISSUES IN SYSTEMS Precursor Missions ENGINEERING to Mars In 1990, the Lunar/Mars Exploration Program Office (LMEPO) at JSC wanted to know how robotic prect_sor missions might reduce the risk of a manned Mars mission. Structuring the problem as a decision tree atlows the effects of different missions and chance events to be systematically and quantitatively evaluated. The portion of the decision tree shown here illustrates the calculation of the probabilities for three distinct outcomes: (A) a successful Mars landing, (B) a safe return without a landing, or (C) a disaster resulting in mission and crew loss, when no atmospheric or site reconnaissance robotic precursor mission,s were made and aerocapture at Mars was selected. As new information becomes available, the decision tree s data can be reviewed and updated. Probability ofEachOutcome _ = .8635| = .0600_"= 1.000 = .0765J /_x^ / _ Propulsive'_ _ -Capture No S!te Retort _ ...... .,,.,o° / ......... _ Crash0.01/ Catastrophic S/CLoss0.04 ,_,_- _ --- \ I [] o . on od. @ ""-.._ccess Land _V 0.099.L2._ A Land0 9 0.9 / / A O.o. A Crash 0.01___U_.._ 0.,, ohah,, y I Making the same calculations for every branch in the decision tree allows a determination of the best mix of roboticprecursor missions as an explicit function of: (a) the contribution of each robotic precursor mission to mannedmission risk reduction; (b) the cost, schedule and riskiness of each robotic mission; (c) the value of the manned mission; and (d) the science value of each robotic mission in the absence of a subsequent manned mission. Another benefit of this quantitative approach is that robotic precursors can be traded against other risk mitigation strategies in the manned mission architecture. For more information and Managers, 1971, safety RTGs ators). of launching (Radioisotope The facilitated search by initiating successes depict ways probability structure in for accident event trees, which the containing Gener- sequences is which depict combinations and fault of system trees, which system failures in an event tree can occur. When an event tree and its associated can be used to calculate the of each accident and mathematics to analysis, see de Neufville and Stafford, Systems et al., Handbook for Decision Analysis, 1977. spacecraft Thermoelectric events and and failures, represented integrated, fault tree(s) similar on decision and Barclay, that for sequence. The of these trees is decision trees. The Analysis for Engineers consequences of each accident sequence are generally measured both in terms of direct economic losses and in public health effects. Doing requiring other a PRA is itself a major a number of specialized than engineers PRAs also those provided and human require large by effort, skills reliability factors amounts engineers. of system design data at the component level and operational procedures data. [For additional information on PRAs, refer to the PRA Procedures Nuclear and Guide Society Electronic (1983) and Engineers by the Institute American of Electrical (IEEE).] 5:3 READINGS IN SYSTEMS ENGINEERING Probabilistic Probabilistic Risk Risk is generally assessment (PRA) sequence function Assessment defined as the -- that Pitfalls Models. in a probabilistic risk expected value of a conis: R = _s Ps Cs where Ps istheprobability ofoutcome s,and Cs is theconsequenceofoutcomes.To attachprobabilities to outcomes,eventtreesand faulttreesare developed.These techniques have been used since 1953, but by the late 1970s, they were under attackby PRA practitioners. The reasons includethefollowing: • Fa,,Ittreesare limitingbecause a complete set of failures is not definable • Common cause failures could not be captured properly. An example of a common cause failure is one where all the valves in a system have a defect so that their failures are not truly independent • PRA results are sometimes sensitive to simple changes in event tree assumptions • Stated criteria for accepting different kinds of risks are often inconsistent, and therefore not appropriate for allocating risk reduction resources • Many risk-related decisions are driven by perceptions, not necessarily objective risk as defined by the above equation. Perceptions of consequences tend to grow faster than the consequences themselves -- that is, several small accidents are not perceived as strongly as o_e large one, even if fatalities are identical There are difficulties in dealing with incommensurables, as for example, lives vs. dollars. view bilistic network (Program nique), Network schedules, Evaluation permit the to be treated as supplying PERT maximum activity, computed Schedules. and duration a and most a probability for project Proba- such as PERT Review of each Techactivity random variable. with the minimum, By likely duration for each distribution can be completion time. This can then be used to determine, for example, the chances that a project (or any set of tasks in the network) will be completed date. In this probabilistic setting, by a given however, a unique critical path may not exist. Some practitioners have also cited difficulties in obtaining probabilistic 54 meaningful network input schedules. data for These and models of a project's Effectiveness offer cost and a probabilistic effectiveness out- comes. This approach explicitly recognizes that single point values for these variables do not adequately represent the risk conditions inherent in a project. Risk Mitigation Techniques and Risk identification risk analysis Tracking and characterization provide a list and of significant project risks that require further management attention and/or action. Because risk mitigation actions are generally not costless, the systems engineer, in making recommendations to the project manager, must balance the cost (in resources and time) of such actions against Four responses available: accept the (3) avoid or reduce tingent action. risk their value to a specific (1) deliberately risk, (2) share participant, take the to the risk are do nothing, the risk with preventive risk, project. usually and and a co- action (4) plan with a co-participant, that partner or a contractor. situation, independent which ways with the goal is to reduce of what happens to incentive contracts and third and additional undertaken. fourth specific responses planning Typical technical include testing additional of subsystems ing redundancy, in risk is, with a In this NASA's to total may go up or down. There to share risks, particularly contractors. These include risk risk, are many cost risks, various warranties. The require that and actions be mitigation (and and usually systems, and building actions costly) designa full engineering model. Typical cost risk mitigation actions include using off-the-shelf hardware and providing sufficient funding during Phases A and B. Major E for con- The first response is to accept a specific consciously. Sometimes, a risk can be shared foreign Probabilistic Cost supportability z F MANAGEMENT risk mitigation actions sufficient initial spares include to meet availability a goal and capability (when cant factor). For mitigated by approach, commend the the financial a design resupply is a signifithat cannot be or schedule same should contingencies ject cycle, techniques be tracked through mentioned and compilation Milestones. of specific the The risk list their delay in the related area delivery of impact of long lead (production and mitigation strategy response. reevaluated The watchlist and items are cant system C/SCS Risk is a projected as appropriate. Should occur, the projected should be strategy updated revised and the A critical is similar to a watch- extensively items with on the signifi- consequences. TPM Tracking. discussed Management: Two very later. Summary • • a to be used setting, efforts Plan, or project issues. as needed. credible hedges and work which are activated upon a trigger- ing event. To be credible, hedges often require that additional resources be expended, which provide a return only if the triggering • in systems good-practice and consequences be given Reviews mitigation of life engirisk In a approach complete a risk man- agement program. Identify and characterize risks for each phase of the project. High risks, those for which the combined effects of likelihood and • a to: document in the triggering consequences risk is a fact To deal with it effectively, the needs a disciplined approach. project includes Contingency Planning. This technique is generally used in conjunction with a watchlist. The focus in contingency planning is on developing arounds, (CIL) safety and neering. manager items), the schedule), is periodically added, modified deleted event Lists. here. management activities. A typical watchlist also shows for each specific risk a triggering event or missed milestone (for example, risk reserves.) list, and has been used Shuttle program to track tracking--are consequences and early indicators of the start of the problem. The risks on the watchlist are those that were selected for management attention as a result of completed risk the contingency important risk tracking techniques--cost and schedule control systems (C/SCS) and Technical Performance Measure (TPM) pro- A watchIist risks, sense, Items/Issues Uncertainty Watchlists this for project items/issues mitigation strategy deal with the larger role of trade studies and system modeling in general. Some techniques for planning and are briefly term Critical and as required by NMI 8070.4A. for choosing a (preferred) tracking In ENGINEERING management margins. strategy and underlying rationale for a specific risk should be docuin a risk mitigation plan and its ef- fectivity occurs. IN SYSTEMS planning and resources act as a form of project insurance. (The term contingency here should not be confused with use of the systems engineer should reestablishment of reasonable and technical The selected mented robust transportation those risks event providing the system's ISSUES are significant, should specific management attention. conducted throughout the cycle should help to force out risk Apply qualitative and quantitative techniques to understand the dominant risks and to improve the allocation of risk reduction resources. This may include the development lysis models PRAs. Formulate handle each ment, where financial and technical of project-specific such as decision and risk, execute including a risk trees anaand strategy to establish- appropriate, of reasonable schedule contingencies and margins. 55 READINGS • IN SYSTEMS ENGINEERING Track the effectivity tion strategy. Good effort--that neers at of each risk management all is, managers levels of the involved. However, sponsibilities individuals. practices must be Successful often evolve risk requires risk mitiga- • a team • Definition (or specification) of the tional and performance requirements hardware, software and operations Interface definitions • • • Specialty engineering Verification plans Documentation trees. and systems project need engito be management re- assigned to specific risk management into institutional funcfor requirements Baseline management techniques: includes • Baseline and • Configuration trol, if needed) • Change • Traceability • • Data management Baseline communication. the following policy. BASELINE MANAGEMENT The baseline for a project contains technical requirements and related schedule mature change ager. parts: requirements to be accepted control by the that are sufficiently and placed under NASA project man- The project baseline c?nsists of two the technical baseline and the business responsible baseline. The for managing line and ensuring is consistent with the all of the cost and business systems engineer is the technical base- that the technical baseline the costs and schedules in baseline. Typically, the control office manages the business Baseline management requires mal agreement of both the buyer project baseline. the forand the seller to proceed according to the documented project requirements up-to-date, (as they exist cycle), at that to change a formal phase in the project the baseline requirements change control process. only by The buyer this same example, office is the buyer contractor, the Loral The project-level management in the next the NASA GOES must level systems be for project and the seller is GOES project office. 56 version con- in discrete steps Evolution project through baseline the evolves project life cycle. An initial baseline may be established when the toplevel user requirements expressed in the Mission Needs Statement are placed under configuration control. At each interphase control gate, increased technical detail is added to the maturing baseline. For a typical project, there are five sequential technical baselines: • engineer is Functional baseline at Program/Project Requirements Review (PRR, called development baseline) • sometimes • Design-to baseline at Preliminary Design Review (PDR) Build-to (or code-to) baseline at the Criti- • cal Design Production the responsible for ensuring the completeness and technical integrity of the technical baseline. The content of the technical baseline includes: (and and might be an external funding agency. For example, the buyer for the GOES project is NOAA and the seller is the NASA GOES project office. Baseline enforced at all levels, control approval control Baseline The definition line at (SAR) • the Operational Operational Risk early Review (CDR) (or as-built or as-coded) System Review (or as-deployed) baseline Acceptance Review (OAR). management and Acceptance base- continue activity must throughout at begin the | MANAGEMENT decomposition prove that process the of the core-level These early detailed be documented and archives, but cal baseline. they Configuration studies retained are not part of identifying and functional and formalizing characteristics item integrity changes at ages discrete for the the of the of a product's management design, and is evidenced requirements drawings, tions tion and the approved to for later progress identification by documents, code listings, material documentation is the process of approved baseline baseline. Typically, meets business project. The the to consider or technical project manager board chair, and the configurathe secretary, who skillfully process and records the official process. What is the proposed What What is the reason is the design • What is the • impact? What is the schedule • • What What • change? What is the risk • What is the impact • What is the and services? impact • What ments? • • What What • change? Is the buyer of the as specifications, process specificaConfigura- is not considered the . an rep- such specifications. the of the by of a change board that is same authority that pre- • . of a baseline documentation control to any board to the of the approved buyer. a num- man- is essential technical control requests until of the ENGINEERING In a change control board forum, ber of issues should be addressed: lication of an existing design. Configuration management often provides the information needed to track project. Configuration change change events configuration communication. to provide approved guides configuration. Conincludes configura- management viously is usually the tion manager the management action action by the baselines product the execution of an orderly development process, to enable the modification of existing by formal controlled discipline the baseline gate Configuration controlling changes and controlling As a functional documentation Figure 7.) Configuration in technical control techni- of maintaining tion or baseline identification, control and configuration (See the the physical of a configura- points purpose configuration evolution figuration of the is the of the product to the baseline. discipline, to sound. Management management evolution cycle are and tests must in the project Configuration tion project decisions ISSUES IN SYSTEMS part change? for the impact? effectiveness change? or performance impact? is the project life-cycle is the impact of not is the of making cost impact? making the the change? on operations? impact is the effectivity documentation supportive to support on equipment spares require- of the change? is affected by of the the change? of Configuration Management Configuration Identification Configuration Control Figure 7 Configuration Management Structure 57 READINGS IN SYSTEMS ENGINEERING A review of this well-informed information decision. should When lead this to a Audit tance informa- tion is not available to the change control board, unfounded decisions are made, often with negative consequences Change Control to the Board requirements Conduct mate review Configuration management control always of approved documentation, so configuration required on a no-change project frequently changing one. • information The data passes is available management managing and to the function archiving project staff. also encomsupporting at the of the product. This modifications needed to qualification-caused Computer Resources (CRWG)--ensures • corrective Working the is adequate for the job Configuration Review change changes board for adare Group development ronment Software software enviBoard-baseline = Software Development agement controlled ware development tools Library--manrepository documentation for softand _m! • Configuration management and configuration control embrace the function of data management, which ensures that only up-to-date baseline approved For disciplined software development, ditional configuration control methods recommended: includes baseline control is as well as a configuration follows all implement actions. • the previously PDR and CDR. The Formal Qualification Review control gate verifies that the as-built product is consistent with the as-built or ascoded documentation and describes the ulti- project. Objective: To review evaluations and then approve or disapprove proposed changes to the project's technical, operations or business baseline. Participants: Project manager (chair), projectlevel systems engineer, managers of each affected organization, configuration manager (secretary), presenters. Format: Presenter covers recommended change and _ -cusses related system impact. The presentath _s reviewed by the systems engineer for comp:eteness prior to presentation. Decision: The CCB members discuss the Change Request (CR) and formulate a decision. Project manager agrees or overrides. control gate verifies that the acceptest results are consistent with the test Software Development developer-controlled opment documentation The configuration following functions: Folder (SDF)-- repository for develand tools. manager performs • Conceives, configuration • Acts as secretary board (controls process) of the change control the change approval ing products conform to the intentions of the designers and to the standards established • Controls tion changes to baseline documenta- by preceding • Controls release of baseline documenta- analyses and trade study data, and keeping it convenient for project use. Configuration verification is part of configuration control. It ensures that the result- trol gate approved serves to review baselines. and Each challenge data presented for conformance to the viously established baseline constraints. Physical verifies Configuration that the physical product corresponds to) documentation the 58 CDR. The Audit control configuration con- gate of the to the build-to (or codepreviously approved at Functional Configuration manages system the tion the preThe documents and management the • Initiates dits. configuration verification Configuration communication of conveying to approved manner. baseline This is all involved progression essential to is the parties au- process the in a timely ensure that L MANAGEMENT forewarn developers only pursue options that are compatible with the approved baseline. of an impending the change con- change contractor funds on should a formal system. line and hence the product version tion. Class 2 changes are editorial Change Control and Version Plan. Class Overly burdensome Control a baseline is placed trol, any change change control chairs the under change con- change control board, while needs the board, and for ensuring that all ganizations are represented control board forum. Change contractor Changes contractor control and affected in the is essential at NASA Center determined must be to be referred or- release change .......... u to circumvent project. software control both the levels. Class 1 to the to the NASA viously erable multiple changes external can become so of the project the process. of the tailored It is change to the there control must process systems. It version on one is control for a development deliverable are only applicable to that item. However, for projects deliverables may become equally ob- one delivthat have of "identical" design, effective on the second production articles. In such a .......................... Disapprove Disapprove ! Disapprove Request Approve Class I identificachanges or to the change changes has only or subsequent ............................. base- projects, it is routine to use for both pre-release and post- deliverable Approved project that is dollars. approved However, important to maintain hardware-only systems. project manager for resolution. This process is described in Figure 8. The use of a preliminary Engineering Change Proposal (ECP) to ! try that the formality be appropriately of each For version the not "visible" always be an effective on every project. systems engineer or configuration manager is responsible for reviewing all material for completeness before it is presented to the contract affect formalized systems that members may essential process requires the approval of the board. The project manager significant 1 changes internal changes interfaces. team Once provides NASA contract This technique designed The project'sapproach to configuration management should be documented in the to save spend ECP. trol board for approval of any deviations considered necessary to further develop the project'sConfiguration Management ENGINEERING the project manager with sufficient preliminary information to determine whether the Communication also keeps developers knowledgeable of the approved baseline and the necessity of approaching [SSUES IN SYSTEMS IP*epare , i ! Prelim I Re,uest Change _,'IB_ ECP _ Approve Class12 F°rmalECP' _-_J-,Prepare ECP DecisionBuyer_preve Approve }L Concurrence J [_ I with Buyer i Classification Yes Change Record Status Indicates Buyer Action Figure 8 Contract 1 Change Control Process 59 READINGS IN SYSTEMS situation, decide the ENGINEERING the change control board must effectivity of the change, and the configuration control version control and as-built configuration mental common policy system must identification for each implementation in projects that of introducing maintain of the article. of have Incre- changes a deliberate is product or process improvements. 1972 plan held As an example, that each of the the Space original Shuttle orbiters would of the orbiters be identical. is different, In reality, each driven primarily by the desire to achieve the original payload requirement of 65,000 pounds. Proper version control documentation has been essential tenance to the of the sparing, Data Management Traceability Data ated Data fielding operational and is an essential data is for all management project library The data functions: retained, and project is essentially and reference manager the following performs Before tangible descriptions (drawings) formation). common documents the the and release project The project understanding docu- can using words, (i.e., symbolic icons in- team must have a of the words and an idea to spends about itself, time the system there are the symbolic inthe information it is in electron- ic or paper form, the data must be readily available in the most recently approved version to all members of the team. Second, symbolic durable. This means information that it must accurately every time and most current version of the baseline information grade with repeated paper files, This is not must be be recalled represent baseline. the The cannot change or access of the database and cannot degrade a trivial requirement, practices the only deor with time. poor data (e.g., allowing somecopy of a document or can allow controlled information to lost. Also, material must be retained for the life of the program (and possibly yond), and a complete set of documentation for each baseline change must be retained. Third, traceable the symbolic upward and information downward. be developed must be A data must maintained to show data the parentage of any requirement. base must also be able to display The all derived from traceability engineering study results and be- base a given requirement. must be provided reports that document and other decisions that role in the flowdown to trade played of requirements. It is the responsibility of the systems engineer to ensure the active, approved baseline is communicated to all those relying on it. This technique prised as to the produce a must produce engineer several vital characteristics formation must have. First, must be shareable. Whether a key library. icons in order to be able to go from a properly functioning system. 6O the system documenta- of baseline the project team product, engineering of the system and numbers manages systems with information than the system children Finally, documentation management Manages changes to baseline tion Manages Data official • • and use. the desk. Conceives, Manages mentation associ- available official the management one to borrow management. that official • • working rather drawing) become management controlled main- Requirements function to configuration management ensures baseline and fleet. Since keeps all participants apdistinction between what is frozen under formal can still be decided board approval. REVIEWS, AUDITS The and intent control gates policy should change without control and what change control AND CONTROL for reviews, be developed GATES audits and during MANAGEMENT ISSUES IN SYSTEMS ENGINEERING Phase A and defined in the mentation Plan. The tion of these activities Project specific should with, though not reviews and audits limited described The same tailoring reviews, audits and applies control to, project cycle that is of sufficient importance to be identified, defined and included in the implementabe consistent the types of in this section. to the gates. The purpose of a review forum and process to provide ment and their contractors Imple- timing of is to furnish the NASA manageassurance that the most satisfactory approach, plan or design has been selected, that a configuration item has been specified requirements, tion item is ready. management) an approach, produced to communicate an ability to meet requirements or establish help to develop a better or project channels, and management nues for solutions. the or that a configuraReviews (technical or are scheduled demonstrate among task communication to meet status. Reviews understanding participants, open alert participants of problems and open project schedule. tion to evaluate approval to proceed to the next event according to the Project tion Plan. GENERAL being review ling chair technical should Termination audit specifications. Audits are the convening manager normally Unless members. should not program authority, of the activity appoints the there are compel- The convenreview board The majority of the be directly associated or project under Reviews. members with the review. During or task, reviews it the is necessary that present trade studies, fined by the project of the performing provide systematic course analyses are also a formal, control The internal manager or the organization. held prior to participation gate review. reviews provide means for controlling the technical progress of the project. They also should be used to ensure that all interested parties are involved and quality assurance should attend internal reviews as active participants. audit) that NASA make program A control gate of verify gate is to provide a review or an management will use design/development and throughout tatives from the areas process early on, process. Thus, represensuch as manufacturing the They can then, for example, ensure that the design is producible and that quality is managed through to or project go/no-go decisions. is a management event in the in an excellent An audit may examine policies and procedures adherence to them. The purpose of a control scheduled event (either and manager Internal in the a of a to conduct technical examination of tangible evidence to determine adequacy, validity and effectiveness of the activity or documentation under review. documentation as well as the with problem areas to a peer group for evaluation and comment. The timing, participants and content of these reviews are normally de- NASA management and its contractors a thorough examination of adherence to program or project policies, plans, requirements and REVIEWS reasons to the contrary, not be directly associated approaches, is to management Implementa- the project or task under review. ing authority also names the Internal of an The the reviewed, board chair. reviews purpose Boards. supervises FOR examinato obtain ave- It should be noted that project termination, while usually disappointing to project personnel, may be a proper reaction to changes in external conditions or to an improved understanding of the system's projected cost-effectiveness. The PRINCIPLES Review who project internal Project It requires formal project status and Red the project cycle. In addition, some organizations utilize Team. This is an internal, independent, peer-level review conducted to identify a any 6_ READfNGS IN SYSTEMS deficiencies al responses, material ENGINEERING in requests for proposals, proposdocumentation or presentation prior to its release. The project or task manager is responsible for establishing the Red Team membership and for deciding which of their recommendations are to be implemented. Review tions Presentation using Material. existing specifications, ports may be pared materials be provided such review board and meeting attendees. Background information and review presentation material of use to board members should be distributed to the members early enough to enable them to examine it prior to the review. For major reviews, time may be as long as 30 calendar days. this Review con- sist Conduct. of oral All reviews presentations of the project requirements plans or designs that ments. These should applicable normally tor) project not being their personnel directly (NASA associated reviewed. This cross-disciplinary and with are is required expertise contrac- the design area under facturing assurance, review, and specialists and fabrication, reliability reviews may also both the contractor's ing officers. Prior to and require and during the Review has the necessary, velop quality Some action manre- requests Report. The responsibility a consensus are review board to develop, of the findings for action. will submit, on a timely basis, report, including recommendations tion, to the convening authority cognizant of the and deThe chair a written for acwith copies managers. Standing Review boards are selected Boards. Standing review for projects or tasks that members by the convening erally made from senior and management or advisors may SPECIFIC TYPES This section timing and members board as If the lifetime review of a pro- to select extra board active assignments to OF REVIEWS describes the content of most types, purpose, of the reviews that may occur during the conduct of projects or tasks. Review material should be keyed to project documentation when minimize separate efforts. change a concern, Purpose. ments Review in authority is genCenter technical staff. Supporting be added to the required by circumstances. board is to function over the Program/Project improvement of the recommendations board for action or engineering (ECR) that document the of the board, including an assessment risks associated with problem areas, submit requests requests 62 Post chair where may attendees design. responsibility of the review that adequate closure for each review, and or recommended in manu- the presence of NASA's contract- members deficiency review design should in the testing, and safety. It is the to ensure ject, it is advisable members and rotate cover needs. to utilize to identify any design shortfalls or recommend improvements. The review audience also include non-project specialists or have a high level of activity, visibility and/or resource requirements. Selection of board given by the cognizant design engineer or his/her immediate supervisor. It is highly recommended that in addition to the review board, the review audience include plan that the chair and cognizant understand the intent of quests. board to the and the approaches, satisfy those require- presentations ensure ager(s) approach, the review, these are screened by board to consolidate them and to as drawings, analyses and readequate. Copies of any pre(such as viewgraphs) should to the presented Following the review responses obtained. Presenta- documentation the Requirements The Program/Project (PRR) establishes available Review. the Requireproject to MANAGEMENT development ensures that: • The (i.e., project functional) objectives baseline. (particularly research and/or science been properly translated unambiguous ments. • systems • of have and require- well by all project techniques, resources participants Management • • Budget constraints Schedule • Risk procedures, to be utilized are At the Definition Phase completion (Phase prior to issuing the Source for Proposal. Agenda. The appropriate following review be addressed: followed by configuration PDR and reviews conitems (CIs). establishes ensures process the that it meets or PDR should: just Request items from checklist the ability of the selected deto meet the technical requirements. • the Establish the compatibility face relationships between configuration items. should Conceptu- Establish the sign approach item and • Establish the integrity design approach. • Establish • design. Assess • Status of action items from al Design Review (CoDR) • • Project Mission Plan objectives • • Research Science objectives objectives • • Design System criteria and approach trade analyses • • Design analyses and trade Final system specification • • • Preliminary interface specifications Software system requirements Work breakdown structure • • Preliminary Preliminary manufacturing plan ground operations plan Agenda. The appropriate following review items/data be addressed: • • Preliminary Preliminary payload integration plan flight operations plan • • • Preliminary Configuration data management management • • Reliability requirements and Quality assurance requirements • • System Project safety policy Prelimi- (PDR) is not a single reof reviews starting with The baseline The the program, project, system, subsystem specific CI baseline requirements. The Concept activities, Selection items/data Review. evaluated. of the B) Design Design Review but a number design-to ENGINEERING activities. the system PDR, ducted on specific • Timing. management Purpose. and IN SYSTEMS structure Preliminary understood between requirements can be properly made. The management agreements and • nary view requirements on the project elements and is sufficiently that trades constraints the objectives) into definite statements The impact of these design of the major It ISSUES the • Address tionships. status, • Establish the other interfacing of the operability compliance ance, reliability quirements. of the interthe specific with and of the selected quality assur- system schedule feasibility selected safety and of the cost rerela- approach. studies Timing. After are developed and ses are available. plan Hardware Review(s) plan plan requirements and and requirements and plan Status of action or • Final functional fications • Technical plan mance design-te after risk items specifications reduction analyitems checklist from Software the applicable Specification requirements justification from the should for and the speciperfor- specified 63 READINGS • • IN SYSTEMS ENGINEERING Experiment performance analysis, including an analysis of instrument accura- requirements and/or code-to cy requirements Design parameters whether and • Environmental • • • Interface design requirements Requirements traceability results Software standards to be applied • Design • be applied Results of technical • and testing Design optimization • • Discussion Compliance ofblock with ments specifications • and and Suitability ware design constraints safety Lists • processes Spares requirements standards to modeling analyses diagrams functional require- designs parts, Preliminary reduction data plans and hard- materials • • Preliminary Preliminary payload ground • Preliminary flight • Requirements equipment, and including flow integration operations operations requirements Interface requirements and specifications Design approach Assessment of hardware and software • plans and schedules (including verification tests or analyses to be performed) Present status of item under review, in- Critical Design analyses verification development The Critical De- CIs and ending with the system Purpose. The CDR verifies CDR. the suitabil- ity the 64 design in meeting specified The or fabrication appropriate of final items from checklist the should items and • • • Test procedures Producibility demonstration Scale model test results • Design ered • Reliability, • bility considerations Spares list specifications trades and results alternatives maintainability of the Conformance • and user requirements Conformance to environmental • requirements Differences between design the considand • sign Review (CDR) is not a single review but a number of reviews starting with specific of a CI equipment inheritance developments activities. Review. and of Design • management action purchase • • • plans Hardware Risk to allow for corrective total design freeze, the • support support analyses , includmode policy technical of producibilbe held early action Quality Assurance Hardware and/or • completion It should of PDR • • and plete and after ity demonstration. Status Preliminary cost and/or coding. design of a CI is com- • • cluding the items/data (GSE) software CI specifications and placed under may be then re- following review be addressed: plan Plan software of ana- and Preliminary reliability ing single-point failure safety the are updated control, and of of significant hardware. • system CDR, review leased for fabrication Timing. When enough before the that the estab- updated to the time CDR, the integrity through the with and verifies requirements and plan plan plans for ground and the the Agenda. management is compatible is verified test data. and drawings configuration philosophy • and the design lytical and Following of preliminary equipment lished at the PDR the CDR. During feasibility of inherited • and design specified requirements, design conforms to the requirements codes the and establishes its build-to baseline. The CDR determines opera- to functional design configuration item, system and subsystem performances in relation to the performances estimated at the PDR • Final ification hardware plans and software design ver- MANAGEMENT • • Detailed mechanical packaging, thermal, (including hydraulic matic) design Detailed electronic and electronic and pneu- electrical circuit design • • Detailed Interface • Mechanical and analysis results • software details Final design and agreements electronic reliability single-point reliability parts analyses, failure policy stress including analyses against • • System safety analyses Electronic parts classifications • screening Nonelectric specifications parts, materials the and and process- test compatibility testing and prior cation testing. to the • Description • • Test objectives Verification requirements tions of test • • Applicable Applicable • Test configuration diagrams • • Test Test test test Data • Configuration • preservation methods Quality assurance plan • • • Qualification and Calibration plan Data management • • Safety plan Test failure • Personnel tions • Present flow and data reduc- tion plan • Support equipment ments and plans • • Spares Ground provisioning operations plan plan • • Payload integration Flight operations plan plan • Present cluding • Test Risk and GSE management Readiness • The Test The TRR in- and/or system. TRR establishes Readi- the CIs, subsystems official sell-off assesses the deci- verificatesting and/or sysverification adequacy tion to be collected, of the meets and Formal and and review in- developments activities. Qualification Review. The System Formal (SFQR) establishes by that the system The qualificathat the system performance and operational specified margins. point for customer qualification After the qualification Qualificathe system verifying meets the specifications. demonstrates approval of the the design. qualifica- under technical requirements within the The SFQR is the decision Timing. lower-level block collection and of item management its functional responsibilities cost qualification tion testing specifica- procedures status Purpose. Review from the should and circuitry calibration production baseline system performance is not a single review but conducted prior to the testing of each test arti- sion point to proceed with planned tion (qualification and/or acceptance) of test articles, tems to acquire Risk System activities. Review. cle, CI, subsystem Purpose. The equipment equipment cluding status of item under review, cost and technical developments ness Review (TRR) a series of reviews start of verification data. require- and and • plans items checklist verifi- plans procedures Manufacturing and fabrication plans Quality assurance plans and procedures test ver- article • • plans the of official Agenda. The appropriate following review items/data be addressed: Materials Purchased acceptance ENGINEERING with start • • control specifications and IN SYSTEMS ification requirements and specifications. Timing. After completion of preliminary ing list and processing devices list planning ISSUES certification completion testing. Agenda. The appropriate following review items/data be addressed: of of all items from the checklist should 65 READINGS IN SYSTEMS ENGINEERING • Status CDRs of action items and TRRs from the • Description of system tested, all subsystems and functional applicable grams Qualification Qualification • specifications Description of test facilities • Description of test configurations • • Subsystem qualification test results System qualification test results • • . Qualification by similarity analysis Nonconformance reports/status Waivers and deviations • Open • Environmental retest following tive action of any failures Strength and fracture mechanics built hardware • • • Software Summary • Present • Risk cost of system and management and Purpose. Audit (FCA) A Functional verifies that item, Physical test review, develop- the (weight, center ertia, surface 66 Design • the subsystem in their physical article, should be specifications engineering orders item, subsystem results Material tions and electronic • • Materials process MaterialUtilization • • Installed non-flight Test results • Demonstration • • Nonconformance Results of each • ceptance Results and and system parts certifica- certifications List (MUL) hardware list results ! reports/status Configuration Item design Ac- Review (CIAR) of the SFQR. Acceptance Review. Purpose. The System Acceptance Review (SAR) provides the decision point to confirm that the design (PCA) CI, is ready for either integra- acceptance or replication. Timing. Following the completion of the SFQR and prior to the Multi-Unit Procurement Phase and/or the PreOperations Phase Agenda. The following project addressed: of inetc.) speci- ! con- (Phase E). appropriate items from documentation should requirements of gravity, moments finish, cleanliness, respective system and the tion, functional and specified in their test and drawings • System Configuration each as-built con- verifies that each as-built subsystem and/or system: specified fications CI, subsystem • documentation from figuration Inspection respective design-to specifications. A Physical Configuration Audit Satisfies • items • Configuration article, and/or system satisfies performance requirements Q testing. appropriate for as- activities. Functional Audit. figuration or to qualification Agenda. The project (SAR). For single may be held pri- • of all under etc. correc- qualifica- technical manuals, draw- Subsystem and system schematics block diagrams Design verification matrices for each manuals manuals status user Review FCA/PCA following addressed: status to separate code listings, in as-built System Acceptance unit projects, the documentation of qualification Operational Maintenance and documented Timing. Following the completion of the SFQR. Usually held in conjunction with the list subjected • • including ments test objectives test requirements development end items tion tests Is correctly ings, including block dia- • • work • • Briefdescriptionofsystem under • • • Verification requirements Results of the system FCA Results of the SFQR and review PCA the be MANAGEMENT • • • System verification and operation) System document • Safety • Present including ments Risk Safety lessons management learned review, develop- activities. Reviews. System of engineering criteria and safety within the safety constraints and occupational during the project and cycle. safety cycle, concurrently is the appli- and management printechniques to optimize effectiveness, time phases of the project Following and their reviews. mainten- of system under and technical cation ciples, held and status status cost with of operational cost through all A series of system reviews are held many of which are other project are descriptions of these relationship to the other Conceptual Agenda. following Safety Reviews. for these reviews are here. However, be aware that quirements flight and/or of the • • Design Safety requirements requirements • • Preliminary Preliminary • Safety ture • • Safety budget Schedule • Risk Conceptual Design Safety have been included 0 Safety Review. • conceptual The project baseline safety translated statements struc- The impact of these design of the major • constraints can The management agreements the safety pants are Timing. Definition and objectives into ensures have been definite and of requirements. un- requirements on the project elements and systems is sufficiently that trades between require- design and that a preliminary assessment of the potential hazards has been made. At several NASA Centers, the CoDSR is called the Phase management activities. requirements properly ambiguous on as the Design Safety safety require- in the or equip- safety plan analysis and management facility safety that: Review. Purpose. The Conceptual Review (CoDSR) ensures that ments the the reviews project safety personnel and understand specify from reviews. shipping and handling of pressure vessels or toxic or explosive materials. Early reviews any problem areas and ments to control them. (CoDR). items project, project hazard staffing Mission be addressed: Purpose ment The renot covered can impose requirements ground equipment, such should of the Studies Phase (Phase concurrently with the Design Review The appropriate list ENGINEERING Project Requirements Safety Review. Purpose. The Project Requirements Safety Review (PRSR) establishes the project the systems engineer should many occupational safety re- with Center occupational should be held to identify completion • • Occupational quirements At the Needs and Conceptual A). It should be held development analyses Timing. (qualification System acceptance report Final systems operations ance methods • • report ISSUES IN SYSTEMS well understood requirements and be properly techniques, and resources program by all made. procedures, to implement project partici- evaluated. At the completion Phase (Phase B) of the Concept activities just prior to issuing the Source Selection Request for Proposal. It should be held concurrently with the PRR. Agenda. the following • Purpose ment The appropriate subjects list should be addressed: of the project, facility from or equip- 67 IN SYSTEMS READINGS ENGINEERING • Status of action items • Design requirements • Safety • • Updated Updated • Safety ture • • Safety budget Schedule • Risk from the CoDSR Critical Design Safety Review. The Critical Design Safety Review (CDSR) is not a requirements single preliminary preliminary staffing project hazard and safety plan analysis management struc- review but a series activities. conduct- items, subsys- Purpose. The CDSR establishes the baseline for safety requirements, safety hazard controls and verification methods to be implemented management of reviews ed on specific configuration tems and the system. several in verifying NASA those Centers, the controls. CDSR At is called Preliminary Preliminary Design Safety Review. The Design Safety Review (PDSR) is the Phase II Safety Review. Timing. When the design of a configuration item is essentially complete and prior to not review total a single but a series of reviews conducted on specific configuration subsystems and the system. Purpose. The PDSR ensures proposed CI, subsystem signs satisfy the project items, that the and/or system and Center safety dere- quirements. At several NASA Centers, PDSR is called the Phase I Safety Review. the Timing. At the completion of prelimi- nary design and prior to the start of major detail design activities. It should be held concurrently with the PDRs. Agenda. the following • • The appropriate subjects list should be addressed: Description of design Status of safety-related under review action items applicable hardware cation reviews or software • • Updated Updated • Updated (sometimes project safety safety analysis preliminary called Preliminary Failure Analysis (FMEA) Modes • • Preliminary Critical Items List of limited-life items • • Accident Waiver tions • Present 68 Risk following • Description status of safety management activities, activities. appropriate list should of signifi- subjects under of safety-related review Status • • applicable hardware or software Final project safety plan Updated safety analysis reports Updated (sometimes • List • Accident • Waiver tions • Present ing cost • Risk action preliminary called the Modes Items oflimited-life hazard Phase and List items from PDSRs analyses II Hazard Effects Analysis items or mishap and from be addressed: of design from investigation deviation request reports disposiF status of safety activities and technical developments management includ- activities. Effects List (CIL). developments purchase • reports disposi- System Acceptance Safety Review. Purpose. The System Acceptance includ- Safety Review (SASR) provides the decision point to confirm that all project safety requirements have been satisfied and confirms the satisfactory ing cost and technical • the Analyses) Final Failure FinalCritical analyses I Hazard or mishap investigation and deviation request The • • specifi- and Agenda. the or fabrication of final hardbe held concurrently with the • Analyses) • freeze, from plan reports hazard the Phase design cant equipment, ware. It should CDRs, completion of verification items and several NASA Centers, the Phase III Safety all hazard open safety the SASR Review. control items. At is called MANAGEMENT Timing. SFQR ment Following and prior Phase and (Phase E). It with the SAR. the completion to the Multi-Unit the Pre-Operation should be held the Agenda. following • • Description ofdesignunder Status of safety-related of the ProcurePhase concurrently The appropriate subjects list should be addressed: from ISSUES IN SYSTEMS • • Critical items list Limited-life item list • • Accident or mishap investigation Nonconformance reports/status • Personnel • • Personnel training status Present status of safety activities, training ing cost and applicable • • hardware review action items or software Updated safety analysis Updated preliminary (sometimes called the • Accident • Waiver tions and or mishap • Present status ing cost and • Risk request of safety technical management Launch Reviews. investigation reports system objectives. er. activities. that Readiness meets Project A CI, subsystem or system all program and/or project ments. Assessing verts the • Approved controls hazards have been • All personnel involved and/or operation of the Timing. gration ground the the Following and prior operations. Agenda. following complies with safety require- to flight and and/or Brief • Safety description requirements of item • • Safety Hazard data compliance data analyses/reports and under inte- start of from review specifications package with supporting not management of the status conreport- processes. Status reporting of determining where the is the output useful changes, needed. assessing The appropriate subjects list should be addressed: • what training. installation does managers need of those plans in analytical process that of the reporting process form for the project coninto manager; namely, what are the future implications current trends? Lastly, the manager decide whether that future is acceptable, in the handling item under review required fiscal resource discussed earli- however, project progress and as WBS project stands in dimensions of interest such as cost, schedule and technical performance. a more safety goals such and were order to exercise proper trol. This is the purpose • received desired functions management, planning; into the ing and assessing is the process have the scheduling planning, Purpose. These reviews ensure the flight and/or ground operational safety of the item under review by certifying that: for all identified implemented. AND ASSESSMENT Planning end with visibility Safety developments activities. REPORTING preparation, requirements includ- developments or Operational technical includ- An important part of systems engineering planning is determining what is needed in time, resources and people to realize the disposi- activities, requirements management STATUS CDRs reports hazard analyses Phase III Hazard deviation Risk reports from Analyses) • ENGINEERING if any, in current These processes back loop depicted functions; one. decision together form in Figure 9. takes place on a continual the project cycle. This project porting loop is applicable hierarchy. data and basis at each Planning assessments hierarchy with appropriate each level; decisions cause data, must and plans Planning, status reporting, are systems engineering program control is a management of are and and/or making the This feedloop throughout level of the status flow up aggregation actions to rethe at be _9 READINGS IN SYSTEMS ENGINEERING Status Planning Reporting Assessing I Figure 9 ] Not OK techniques Making Reporting Cost and Status taken down the hierarchy. Managers at each level determine (consistent with policies establi=.hed at the next higher level of the project hierarchy) how often, and in what form, reporting be made. ing and principles data and assessments should In establishing these status assessment requirements, of good practice are: reportsome and and Use an agreed-upon • status Report • tent format at all project levels Maintain historical data for both • • of well-defined reporting variables these core variables identification • set and in cross-project Encourage a logical process status reporting variables, a consistrend analyses of rolling (e.g., use up the WBS for obligations/costs status reporting and PBS for mass status reporting) Support assessments with quantitative risk measures Summarize the condition of the project by using color-coded (red, yellow, and green) alert ables. zones for all core reporting periodic (e.g., status reporting mended, through ables should be there is rapid Key reviews, points at vari- which trends, some status reporting tracked more often status reporting should warning there variwhen if allowed measures be carefully scrusigns of potential be indications that to continue, yield an unfavorable outcome, should begin as soon as practical. 7O tracking of is recom- change or cause for concern. such as PDRs and CDRs, are and their trends tinized for early problems. Should existing monthly) variables replanning and systems Schedule schedules systems will additional inforand assessment schedules, and provides engineer pro- Measures assessment on the project visibility costs manager into how is tracking against its schedule targets. From a management point targets is on a par of view, achieving with meeting the cal performance requirements It is useful to think of cost reporting technical engineering Control well the project planned cost and and these techni- of the system. and schedule assessment as measur- ing the produces performance the system." NHB Reporting 9501.2B, Procedures for Contractor of Correlated Cost and Perfor- mance Data, of the provides "system specific that k E requirements for cost and schedule status reporting assessment based on a project's dollar and value and period of performance. Generally, the NASA Form 533 series of reports is applicable to NASA cost-type (i.e., cost reimbursement and fixed-price However, on larger which require Form incentive) contracts 533P, NHB contracts. (>$25M) 9501.2B al- lows contractors to use their own reporting systems in lieu of 533P reporting. The project manager/systems engineer may choose to evaluate Regular, the core and reporting status • provides reporting for costs performance, cess metrics. Status[OK Planning and Status Feedback Loop This section mation on status the completeness and quality of these reporting systems against criteria established by the project manager/systems engineer's own Center, or against the DoD's CostSchedule Cost System Criteria (C/SCSC). The latter are widely industry and government, and tools exist for their implementation. accepted a variety Assessment traditional Methods. The method of cost and comparing baselined schedule cost and against values. trol their terminology, actual a difference control schedule In program between by of is by plans conactual B MANAGEMENT performance and planned status is called a variance. Figure costs 10 illustrates ances and constructed two or schedule kinds some related concepts. work breakdown task and product a schedule cost. The (at any level BCWPt-ACWPt, of vari- project into discrete with each in the WBS) and a budgeted (i.e., planned) Budgeted Cost of Work Scheduled (BCWSt) for any budgeted cost ducts in those set of WBS elements the is the of all work on tasks and proelements scheduled to be com- technical ing Control BCWPt, also called Earned Value (EVt), is the budgeted cost for tasks and products that have actually been scope age between makes it very mate performance. the cost variance indicate (EACt) that of the budgeted cost. enable a program EAC at any point in schedule of the ENGINEERING baselines work are not and fully the cost data and schedule difficult (or impossible) current cost EAC of Variances Systems of the and Engineer. can linkdata to esti- project. the When the inte- grated, then cost and schedule variances still be calculated, but the incomplete pleted by time t. The Budgeted Cost of Work Performed (BCWPt) is a statistic representactual from types of variances to estimate the project cycle. ffthe cost and is the variances may at Completion is different These analyst IN SYSTEMS is called at time t. Such the cost Estimate A properly structure (WBS) divides the project work tasks and products. Associated ISSUES Role of the negative vari- produced (completed or in progress) at time t in the schedule for those WBS elements. The ances are large enough to represent a significant erosion of reserves, then management attention is needed to either correct the vari- difference, schedule ance, or to replan the project. It is important to establish levels of variance at which BCWPt-BCWSt, variance at time is called the t. action ECA Forecast of Cost Var/ance at is to be taken. ally lower when cost do not support Earned The first action _ nz//, ,::i: r..', :, / j!i I':::.:: _:_;:i'_"'" I m /y 0 Work Actual TIME Schedule] ACWP = BCWP = Budgeted Coet Work Performed EV BCWS EAC = Earned Value BudgetedCtmt Eati mate at Work Ctm'or*t Vato gener- and schedule baselines Value calculations. taken to control an variance is to have or systems engineer Ctmt of possible occur: reasons , A receivable • tory for some reason. A task is technically - ACWP) :.:.._ "_!!!::J:_ f/ negative manager number problems Schedule ] Variance ] to Date Date (BCWP are vestigate the problem, determine its and recommend a solution. There / ACWP '" |to levels Completion excessive cognizant C U M U L A T I V E These of requires planned. Performed of • Completion was more Unforeseeable events occurred, strike, late why or was very resources the incause are a variance unsatisfacdifficult than and originally (and unlikely to repeat) such as illness, a labor a fire or some other calamity. FigureI0 Costand ScheduleVariances Although The (ACWPt) Actual Cost of is a third statistic Work Performed representing the funds that have been expended on those WBS elements. The tween the budgeted and up to time t difference beactual costs, largely the an important their control. correct ance identification a program is control of variances function, is there is systems engineering role in That role arises because the assessment of why a negative vari- occurring greatly increases the 71 READINGS IN SYSTEMS ENGINEERING chances of successful assessment of the cost, that can engineer. control often requires schedule and only be actions. This provided by the systems trade the Estimate at Completion EAC can be estimated at any point in the project. The appropriate formula depends upon the the reasons associated for any variances that may exist. If a variance exists due to a one-time event, such as an accident, then EAC = BUDGET + ACEP - BCWP where BUDGET is the is assumed to continue the equation is: EAC BCWP). substitute for understanding causes of variances. Technical Performance reporting technical complements tracking manager the able that, of Selecting 72 and and performance verification like (attributes At into begin as soon to support however, in the project analysis as a base- mass that the full set not be avail- are TPMs can be meaningful to Structure [PBS] or reliability) are or unique meaningful only to spe- the whethits ties together engineering requireand valida- a butes engineer should track effectiveness (when maintains such performance that lower determine levels a measure) or technical PBS, In selecting TPMs, should focus on those measured during the surement indirectly analysis. available the TPMs. TPMs tracking can be identified through tional and performance requirements on each individual system, segment, systems the the and the attri- it, as top-level of the worth the funclevied etc. engineer that can be objectively project cycle. This mea- can be done directly by testing or by a combination of testing and Analyses are to determine = =$ cycle. In general, PBS. The systems measure of system activitiesmthat is, a TPM tracking program forges a relationship among systems analysis, functional ments definition tion activities. later TPMs. element, the tracking TPMs basic systems can generic (attributes that each Product Breakdown project principal visibility are signals to reand people re- sometimes new systems need to be initiated. TPMs until and assessment of the performance measures cost and schedule congains TPMs schedule of TPMs. cific PBS elements). The systems engineer needs to decide which generic and unique TPMs are worth tracking at each level of the er the delivered system will actually meet performance specifications (requirements). Beyond number Out-of-bounds" plan fiscal, requirehelp identify requirements. activities re- evaluation start of Phase C. Data of selected TPMs may, underlying TPMs, performance in quantitative anaperfor- line design has been established, which can occur as early as Phase B. A TPM tracking program should begin not later than the Measures system's and system's • Tracking to grow over time, and = BUDGET X (ACWP/ in systems the Functional sources; activities In a large project, a good EAC is the result of a variance analysis that may use a combination of these estimation methods on different parts of the WBS. A rote formula should not be used as a project performed ments definition activities verification and validation Verification and validation • It is also possible that EAC will grow at a greater rate than estimated by the above equation if there are a growing number of liens, action items or significant problems that will increase the difficulty of future work. Such factors could be addressed using risk management methods. By studies activities identify the or technical attributes system effectiveness; • sult original planned cost at completion. If a variance exists for systemic reasons, such as a general underestimate of schedule durations, or a steady redefinition of requirements, then the variance trol. Systems analysis key performance that determine lysis help quantify mance requirements. Computing Status system's (TPMs) • an understanding technical situation often the some only means high-level i MANAGEMENT TPMs such as system reliability, but the data used in such analyses should be based on demonstrated values to the maximum practical extent. performed These using the analyses same can project D, the demonstrated cycle proceeds measurement increasingly availability trade studinstead of values. through of TPMs more accurate of more "actual" As the Phases C and should become because of the data about the system. lect Lastly, those defined system the systems engineer TPMs that must fall (quantitative) effectiveness Usually these upper or lower example of such injected pability should sewithin well- limits for reasons or mission feasibility. limits represent either of a firm bound constraint. A typical a TPM for a spacecraft is its mass, which must of the selected Tracking injected mass is meant to ensure that not exceed the calaunch vehicle. as a high-level this control described be using estimated (or desired) performance or technical attributes, the models are exerusing Value does TPM method Methods. of assessing a time-phased comparing the The a TPM • • • • • • • • the the system, the demonstrations, technological maturity or related systems. dry mass tends during Phases C and 30 percent. A planned in the intersects requirement planned formance or of planned profile is asymptotic (or contract profile method measurement As an to grow D by as much as 25 to profile for spacecraft dry mass may try to compensate growth with a lower initial value. value schedule power demands by spacecraft science instruments may be as well. For launch • • • • • • • performance measures spacecraft include: vehicles, high-level TPMs subsystracked include: Total vehicle mass at launch Payload mass (at nominal altitude or orbit) Payload volume Injection accuracy Launch reliability In-flight reliability For reusable vehicles, percent of value recovered For expendable vehicles, unit production cost at the n th unit. is by establishing planned schedule of tests and and any historical exper- ience with similar example, spacecraft and traditional planned profile for it, and demonstrated value against include cost End-of-mission (EOM) dry mass Injected mass (includes EOM dry mass, baseline mission plus reserve propellant, other consumables and upper stage adaptor mass) Consumables at EOM Power demand (relative to supply) Onboard data processing memory demand Onboard data processing throughput time Onboard data bus capacity Total pointing error Mass and tems and separately not happen. that profile. The planned profile represents a nominal "trajectory" for that TPM taking into account a number of factors. These factors for earlier. High-level technical (TPMs) for planetary • Assessment method Examples of High-Level TPMs for Planetary Spacecraft and Launch Vehicles measurement methods or models used during ies. In TPM tracking, however, cised Earned ISSUES IN SYSTEMS ENGINEERING usually for this The final either to an allocated specification). The is the technical percounterpart to the A closely TPM relies related method on establishing margin requirement the actual margin for against The margin is generally ence between a TPM's and its allocated requirement may of assessing a time-phased a it and comparing that requirement. defined as the demonstrated requirement. be expressed differvalue The margin as a percent of the allocated requirement. The margin requirement generally declines through Phases C and zero at their Depending the systems reasonable planned quirements manager. ods is D, reaching completion. on which engineer's or method is chosen, role is to propose profiles or for approval by The value of either that exceptionnthat they allow is, only approaching margin re- the cognizant of these meth- management deviations by from 73 READINGS IN SYSTEMS planned ENGINEERING profiles ments quiring or margins below require- signal potential future problems replanning. If this occurs, then (a) renew cost, schedule and/or technical changes should be proposed. Technical changes may imply some new planned profiles. This lustrated for a hypothetical TPM l l(a). In this example, a significant strated in the Planned Profile New AllocaUon T P is il- • M Original V Allocation in Figure demon- remaining Margin the TPM management requirement. allocation for the margin requirement, placed the margin ment. cation value to a low number, new exceeding its allo- for example, five perand The for the same replanning The the probability of the than the allocation red alert zone. The in Figure t occurred ll(c). when final TPM value fell precipitously new higher the TPM resulted in a substantial ment in that probability. r Demonstrated Planned r Variance Profile for the tracking being less into the allocation for improve- 74 management method requires of the probability distribution final TPM program, value. when D a t e m TIME (b) Margin Management • [ Method DiscontJ _ Early in the TPM the demonstrated nuity /induced 4_./f by shift tonew allocation t M 0 Margin I Requirement • New Margln Requirement r I Replanning Occth_fed Here TIME P ro (c) Risk Management 1.0 Method ! b Green F i n Zone Alert I 4i T • • • • • s • • • i* Yellow J, A]ert ... Zone T P M _e V _iiiiiii] r Di .... t.i nui ty l:: _iii::iii::i::i :_;i:'_:_'_ induced by _hiR _i!;_;_i_!_ iii_:::i_}_ tO new ellocati.n [i::i!!i::] C _!:_:'_ ..................... _..._ ...... ;...i::::_::_l u i_iii-_:i_:_:i:':'_ii:_:i_:i:_:i_:i_-'._:_: _.!':i:':::_: :_:: :_:_::_:_ r u e ...... :i:::i-%'.::i::':':_-:i:i:i:':'_: @_:i-j-_ Z(_-"_ _" A 1 l o c i:'.' ie .... yello_ert are apply The risk an estimate en _eplannJng Occurred Here higher cent or less. A third method of reporting assessing deals with this risk explicitly. risk management method is illustrated example at time Original t TPM resulted in a higher but it also immediately in excess of that require- final J of as- The margin management method attempts to deal with this implicitly by establishing a margin requirement that reduces the of the • u Both of these methods recognize that the final value of the TPM being tracked is uncertain throughout most of Phases C and D. chances • tracking method The • • is illustrated for the same example in ll(b). The replanning at time t ocwhen the TPM fell significantly below margin m I -_ + margin sessing Figure curred the of • V• A new planned profile was to track the TPM over the time • Planned of the system resulted in replanning at time t. The replanning took the form of an increase in the allowed final value of the TPM program. The • t u e variance (i.e., unanticipated growth) TPM during design and development (the "allocation"). then established Method shown only "to _at and in (c}, (bL . _ip _i!!ii] D _i_::il t .:'.'!;_ e TIME Figure 11 Three TPM Assessment Methods = E MANAGEMENT ISSUES IN SYSTEMS ENGINEERING A n Example of the Rish Management Method Traching Spacecraft Mass for value is based tion, this statistical on indirect distribution variance means of estima- typically than later, has a larger when it is During PhasesC and D, a spacecraft's injected mass canbe consideredan uncertainquantity.Estimates ofeachsubsystem'sand eachinstrument's mass are, however,made periodically by thedesignengineers. These estimateschange and become more accurate as actual parts and components are built and integratedintosubsystems and instruments and are integratedintospacecraft. Injectedmass can alsochange duringPhases C and D as the quantity of propellantis fine-tunedto meet the mission design requirements. At each point during developmentthen,the spacecraft's injected mass is betterrepresentedas a probabilitydistribution ratherthan as a singlepoint. The mechanics of obtaining a probability distribution for injectedmass typicallyinvolve making estimatesof threepoints-- the lower and based on measured When a TPM upper bounds and the most likelyinjectedmass I value.These three values can be combined into parameters that completely define a probability! distribution like the one shown in the figure below. ment for describing the project's TPM tracking program. This description should include a master list of those TPMs to be tracked and the measurement and assessment methods Probabi]_ °-'l..... ....':: ..... Spacecraft Injected Mass, Kg The launch vehicle's "guaranteed" payload capability, designated the "LV Specification," is shown as a bold vertical line.The area under the probability curveto theleftofthe boldvertical line irepresentsthe probabilitythat the spacecraft's injectedmass will be lessthan or equal to the :launchvehicle's payloadcapability. Ifinjected mass isa TPM beingtrackedusing theriskmanagement method, this probabilitycould be plotted in a displaysimilartoFigure11(c). Ifthisprobability were nearly one, then the projectmanager might consider adding more objectives tothe missioninordertotake advantage ofthe "largemargin" that appearsto exist.In the above figure, however, the probability is significantlyless than one. Here, the project manager might considerdescopingthe project, for example,by removing an instrumentor otherwise changing mission objectives. The projectmanager couldalsosolvethe problem by requestinga larger launchvehicle! stays data, e.g., along its (or equivalently, when above the corresponding ment), bution the the narrowing should allow green alert zone Relationship to the SEMP. result. planned margin margin profile remains require- of the statistical distrithe TPM to remain in (in Figure its growth. The three different ways to assess cate that information whichever is chosen, failure should be the its a test 1 l(c)) despite methods represent TPMs and communi- to management, but the pattern of success or same for all three. of TPM Tracking The SEMP is the to be employed. If analytical models are used to measure level TPMs, then these need Program usual docu- methods and certain highto be identified. The reporting frequency and sessments should be specified timing as well. termining balance engineer must for accurate, these, the systems the project's needs timely and effective the cost of the TPM TPM tracking against tracking program. The TPM tracking program plan, rates on the SEMP, should TPM's allocation, time-phased file or margin as appropriate method. Systems requirement, to the Engineering of asIn de- which elabospecify each planned pro- and alert zones, selected assessment Process Metrics Status reporting and assessment of systems engineering process metrics provides additional visibilityinto the performance of the "system that produces the system." As such, these metrics supplement the cost and schedule control measures discussed earlier. Systems engineering process metrics try to quantify the effectivityand productivity of 75 READINGS IN SYSTEMS ENGINEERING the systems engineering process and organization. Within a single project, tracking these metrics allows the systems engineer to arises from the insights they provide into the process that cannot be obtained from cost and schedule control measures alone. Over better understand the health that project. Across projects hard and (and progress of over time), the tracking of systems engineering metrics allows for better estimation process of the cost and time of performing systems engineering functions. It also allows the systems engineering organization to demonstrate its commitment to the tinuous improvement. TQM Selecting Metrics Engineering Systems of con- metrics systems engineering fall into three the progress productivity. process categories: those that of the systems engi- Different in demonstrating the investment in systems training. Examples and on metrics staffing, levels dealing project ment progress and progress. A subsystem engineer metrics. Requirements development management and worth tracking moves through Collecting just a should Design and development Assessment as the project cycle. and maintaining Category S volatility Q Requirements approved per systems engineering hour p Specificationsplanned vs. completed S Processing of ECRs/ECOs Q Verification Validation and (V&V) the data Reviews planned S S V&V procedures planned vs. completed S Functional requirements approved vs. verified S V&V plans approved per systems engineering hour P V&V procedures completed per systems engineering hour P Processing of trouble reports Q Processing of Review Item Discrepancies (RIDs) Q Processing of action items Q project on the S = Progress, or schedule-related Q = Quality.related P = Productivity Table2 Systems EngineeringProcessMetrics these tions, metrics allow each NASA them in a common-sense the process itself. The system balance the value of each metric of these drawings V&V plans identifiedvs. approved intended 76 prois not S and effort engineer systems engineering process its collection cost. The value engineering That list Trade studies planned vs. completed systems engineering process is not without cost. Status reporting and assessment of systems engineering process metrics divert time from must Methods. Requirements identifiedvs. completed vs. approved Engineering vs.related few be systems engineer's engineering effort. process metrics change of invaluable of systems with systems risk manage- to focus on Which metrics also a source are potential returns from engineering tools and !Requirements major trade study systems engineer tracked depends on the role in the total systems The systems engineering be Systems Engineering Process Metric may focus on subsystem requirements and interface definition progress and verification procedures progress. It is useful for each systems process also which Table 2 lists some systems cess metrics to be considered. the qualmeasure engineering management are generally interested in different metrics. For example, a project manager or lead systems engineer may focus engineering can data, Process neering effort, those that measure ity of that process, and those that its these productivity Function Generally, metrics measure principle time, against metrics own to be exhaustive. processes. For Because for different Center needs way example, some of interpretato define that fits its each Center MANAGEMENT needs to completed or whether part determine what it meant by a versus an approved requirement, these terms are even relevant. As of this recognize example, more definition, it that not all need be lumped useful to track the rately for each of several requirements, for example. Quality-related is important requirements, for together. It may be same example, quantified in metrics several example, processing cumulative could ECRs serve ways. volatility of newly change to cost For can be identified (ECR) be tracked by comparing opened versus cumulative ECRs closed, or by plotting the age profile of of open ECRs, or by examining the number of ECRs opened last month versus the total number open. The systems engineer should number of systems to a particular not all systems the same, an productivity-related engineering appropriate an per is hours weighing to ensure systems engi- allows a current completed type. The latter against which the metrics can or graph of With quality- metrics, more important The most useful method trend on successful personal provide output personnel. generally snapshots. ment the function or activity. engineering hours Displaying schedule-related be accomplished in a table planned quantities vs. actuals. and picking method. more sophisticated the most common scheme should be developed comparability of hours across neering ENGINEERING metrics engineering unit of input. Although measures of input exist, the IN SYSTEMS judgment in and assessment Productivity-related indication of systems dedicated Because of changes to As another request personal reporting of systems engiand/or breakbe defined and or as the number requirements. engineering sepatypes should different requirements as the number requirements, already-approved metric different indicate when a part of the neering process is overloaded ing down. These metrics can tracked to apply status ISSUES than kind trends are isolated of assess- comparisons project with that project of the of the for a same provides a benchmark system engineer can judge efforts. 77 J