Safety Analysis of Safety

Critical Systems

1
 Outline
1. Introduction

2. Petri nets

3. A Case Study

4. Safety Analysis

5. Conclusions

2
 Introduction
Computers - used as passive (monitoring) and active (controlling)
components of real-time control systems.
− Power Plants, aerospace, medical systems, air traffic
control.

Modeling and analysis tools are desperately needed to aid in these

tasks as the standard tools and methods which currently exist do
not satisfy these requirements.

It is important to stress the system nature.

Catastrophic failures may lead to danger to human life.

3
 Introduction
Software does not harm anyone – only the hardware which it controls
can do damage.

Therefore, software safety must be considered as part of the

overall systems safety.

Disasters are often the result of multiple failures sequences which

involve hardware, software and human failures

Petri nets (PN) have been used to model and analyze systems for
many properties like deadlock and reachability.

4
 Introduction
Objective is to demonstrate how PN can be used in designing and
analyzing such properties as safety and fault-tolerance.

PN can be used because it is capable to model the hardware,

software, and human behavior within one model.

It is possible to do the impact analysis of fault on other components

(due to a specific component).

PN models can also be used to identify the critical functions.

It is possible to eliminate the hazards from a design without generating

the entire PN reachability graph.

5
 Petri Nets
6

concurrent, asynchronous, distributed, parallel, nondeterministic

and/or stochastic systems

graphical tool
• visual communication aid.

mathematical tool
• state equations, algebraic equations, etc.

communication between theoreticians and practitioners

6
 History
1962
C.A. Petri’s dissertation (U. Darmstadt, W. Germany)

1970
Project MAC Conf. on Concurrent Systems and Parallel Computation (MIT, USA)

1975
Conf. on Petri Nets and related Methods (MIT, USA)

1979
Course on General Net Theory of Processes and Systems (Hamburg, W. Germany)

1980
First European Workshop on Applications and Theory of Petri Nets (Strasbourg, France)

1985
First International Workshop on Timed Petri Nets (Torino, Italy)

7
 Petri Nets Applications
8
performance evaluation
communication protocols
distributed-software systems
distributed-database systems
concurrent and parallel programs
industrial control systems
discrete-events systems
multiprocessor memory systems
dataflow-computing systems
fault-tolerant systems
etc, etc, etc

8
 Petri Nets Definition
9
Directed, weighted, bipartite graph with Places, Transitions, arcs
(places to transitions or transitions to places) and weights
associated with each arc.

Initial marking
• assigns a non-negative integer to each place.

A transition t is enabled if each input place p has at least

w ( p , t ) tokens.

An enabled transition may or may not fire.

A firing on an enabled transition t removes w(p,t) from each input place

p, and adds w ( t , p’ ) to each output place p’.

9
 Petri Nets Firing Example

2H2 + O2 → 2H2O

2 t
H2
2

H2O
O2

10
 Petri Nets Firing Example

2H2 + O2 → 2H2O

2 t
H2
2

H2O
O2

11
 A Case Study (Reactor Protection System)

RPS is a complex control system comprising several safety electronic and

mechanical components, known as nuclear safety components, to safe
shutting down the nuclear reactor.

Shut down System

4 FAV in between the helium tank and helium header which services the
poison tanks.

12
A Case Study (Reactor Protection System)

13
 Safety Analysis
Properties that depend on the initial marking

Contains sensors, logic, actuators, GUI. These components can fail abruptly
and hence the consequences to hardware and software failure, undesired
environmental conditions must be modeled into the system design.

To do the safety analysis, an attempt must be made to identify all the system
hazards and assess their consequences with respect to their severity.

Some hazards may lead to the risks that are acceptable to a certain limit
while some hazards may lead to the risks that are unacceptable in any
conditions.

Hazard analysis technique.

14
Safety Analysis

15
 Safety Analysis

From the table 1, it is to be noted that we keep redundant information to track

the state of FAV because of its criticality for safety.

To identify the hazards, it is required to find out all the

possible reachable states and which can be derived from the
reachability graph.

The reachability graph can be constructed from the PN model

16
 Safety Analysis

Then we need to look into the behavioral states and failure states.

It is difficult to draw the complete reachability graph for a complex

system due to its size

We can analyze the safety without drawing the full reachability

graph by backtracking from the failure states to its originating state
and apply some design techniques to ensure that from the
originating state, that failure state cannot be reached.

17
Safety Analysis

18
Safety Analysis

19
Safety Analysis

20
Safety Analysis

21
 Safety Analysis

To meet this condition Petri net needs modification such that these risk states
can be avoided in the resultant reachability graph.

The precedence of the parallel transitions can be controlled through

interlocks, which can guarantee the firing of the parallel transitions as per the
defined sequence.

22
Safety Analysis

23
Safety Analysis

24
 Safety Analysis

The modified design can be validated by constructing the reachability graph

from the modified PN, given in figure 3(a). The constructed reachability
graph is shown in figure 3(b).

25
 Safety Analysis


26
 Safety Analysis


27
Safety Analysis

28
 Safety Analysis


29
 Safety Analysis

30
 Safety Analysis

RTOS provides a feature to the application software, through which the tasks
can be prioritized.

Therefore, the other way to impose the precedence among the parallel
transitions, to avoid risk states, is to enforce the firing of transitions as per the
defined priority.

31
 Safety Analysis


32
 Safety Analysis


33
 Safety Analysis

This logic can be achieved by the soft watchdog or hard

watchdog as:

• −In case of hard wired system or module, hard-wired logic

must send a periodic pulse to the watchdog. The moment the
constraint given in equation 1 fails, it must stop sending pulse
to the watchdog. The watchdog must maintain the current and
its previous state. Once watchdog stops getting periodic pulse,
it loads all the dynamic variables, which are of previous state.

34
 Safety Analysis

The other possibility is to reset the microprocessor IC through reset pin. This
will reset the system state.

• This watchdog can be implemented as a software watchdog, known as soft

watchdog or hard wired, known as hard watchdog.
• Similarly in case of software system or module, software must send a
periodic pulse to the watchdog to process in the same manner as hard
wired system does.

35
 Safety Analysis


36
 Safety Analysis


37
 Safety Analysis


38
 Safety Analysis

Now the design can be again validated by drawing the reachability graph of
the modified PN model, as shown in figure 4(a).

The resultant reachability graph is shown in figure 4(b). The transitions from
the state, marked with a solid circle on the upper left corner, have not shown
further because the subsequent states will not be reachable due to the
imposed timing constraints.

39
Safety Analysis

40
Safety Analysis

41
 Safety Analysis

Similarly, this analysis can be further investigated for more

issues.

It may be a time taking process and much effort may be

required but the benefits outweigh them, looking from the
safety perspectives.

42
 Conclusions
43
1. The analysis can be done in parts, may be component by
component or even function by function, in case the volume or
criticality of the function is much.

2. In the similar fashion, analysis can be done for the component

connectors as well.

3. Further all the risks do not have the equal probabilities.

43
 Bibliography
1. Lalit Singh, Hitesh Rajput, “Dependability analysis of Safety Critical Real-Time
Systems by using Petri nets” in IEEE Transactions on Control Systems
Technology, vol.PP, no.99, pp.1-12 doi: 10.1109/TCST.2017.2669147.
2. Lalit Singh, Gopika Vinod, A.K. Tripathi, "Design verification of Instrumentation
and Control systems of Nuclear Power Plants," in IEEE Transactions on Nuclear
Science, Vol.61(2), March 2014, pp.921-930.
3. Vinay Kumar, Lalit Singh, Pooja Singh, K.V. Singh, A.K. Maurya, A.K. Tripathi,
“Parameter Estimation for Quantitative Dependability Analysis of Safety-Critical
and Control Systems of NPP,” in IEEE Transactions on Nuclear Science,
(Accepted for Publication).
4. Lalit Singh, Gopika Vinod, A.K. Tripathi, “Early Prediction of Software
Reliability: A Case Study with a Nuclear Power Plant System”, in IEEE
Computer, Vol.49 (1), Jan 2016, pp.52-58.
5. Vinay Kumar, Lalit Singh, A.K. Tripathi, Pooja Singh “Safety Analysis of safety
critical systems using state space models”, in IEEE Software, Vol. 34(4), July
2017, pp.38-47.

44
 Bibliography (cont.)
6. Sumit, Sandeep Kumar, Kuldeep Kumar, Lalit Kumar Singh, “Evaluating
Technologies for Reliable Software in Consumer Electronics: Survey of
Component Failure Modeling based System Reliability Prediction Models,”
in IEEE Consumer Electronics Magazine, (under production).
7. Raj Kamal, Lalit Singh, Babita Pandey, “A Review of Security Analysis for
Electronic Power Systems,” in IEEE Consumer Electronics Magazine,
(under production).
8. Lalit Singh, Hitesh Rajput, “Ensuring Safety in Design of Safety Critical
Computer Based Systems,” in Annals of Nuclear Energy, Elsevier Vol.92,
June 2016, pp.289-294.
9. Vinay Kumar, Lalit Singh, A.K. Tripathi, “Transformation of deterministic
models into state space models for safety analysis of safety critical systems:
A case study of NPP,” in Annals of Nuclear Energy, Elsevier, Vol.105, July
2017, pp.133-143.
10.Raj Kamal, Lalit Singh, Babita Pandey, “Dependability Analysis of Safety
Critical Systems: Issues and Challenges,” in Annals of Nuclear Energy,
Elsevier, Vol.105, July 2017, pp.133-143 (Accepted for Publication).

45
 Bibliography (cont.)
11. Lalit Singh, Gopika Vinod, A.K. Tripathi, “An approach for
Parameter estimation in Markov model of software reliability for
early prediction: A case study”, in IET Software Vol.9 (3), June 2015,
pp.65-75.
12. Vinay Kumar, Lalit Singh, A.K. Tripathi, “Reliability Analysis of
safety-critical systems: A state-of-the-art review”, in IET Software,
2017, DOI: 10.1049/iet-sen.2017.0053 IET Digital Library,
http://digital-library.theiet.org/content/journals/10.1049/iet-
sen.2017.0053.
13. Lalit Singh, Hitesh Rajput, Gopika Vinod, A.K. Tripathi, “Computing
Transition Probability in Markov Chain for Early Prediction of
Software Reliability,” in Quality and Reliability Engineering
International, Vol.32, April 2016, pp.1253-1263. UR -
http://dx.doi.org/10.1002/qre.1793
14. Pramod Kumar, Lalit Singh, Chiranjeev Kumar, “Suitability Analysis
of Software Reliability Models for its Applicability on NPP Systems,”
in Quality and Reliability Engineering International, (Accepted for
Publication).

46
 Bibliography (cont.)
15. Pramod Kumar, Lalit Singh, Chiranjeev Kumar, “An Optimized
Technique for Reliability Analysis of Safety Critical Systems: A case
study of Nuclear Power Plant,” in Quality and Reliability Engineering
International, (Accepted for Publication).
16. Lalit Singh, Hitesh Rajput, “Verification of Safety in Safety Critical
Computer Based Systems: A Case Study of Nuclear Power Plant
System,” in Nuclear Technology, American Nuclear Society Vol.
195(3), September 2016 pp.301-309.
17. Vinay Kumar, Lalit Singh, A.K. Tripathi, “A Probabilistic Hazard
Assessment Framework for the Safety-critical and Control Systems: A
Case Study with NPP,” in Nuclear Technology, American Nuclear
Society Vol. 197(1), Jan 2017, pp.20-28.
18. Raj Kamal, Lalit Singh, Babita Pandey, “Security Analysis of Safety
Critical and Control Systems: A Case Study of Nuclear Power Plant
System,” in Nuclear Technology, American Nuclear Society,
Vol.197(3), Feb 2017, pp.296-307.

47
 Bibliography (cont.)
20. Lalit Singh, Hitesh Rajput, “Safety Analysis of Life Critical Software
Systems: A Case Study of Nuclear Power Plant,” in IETE Technical
Review, Taylor & Francis, Vol.34(3), 2017, pp.333-339.
21. Vinay Kumar, Lalit Singh, A.K. Tripathi, “Analysis of safety critical
systems using transformation of UML into state space models: A Case
Study of NPP”, in IETE Technical Review, Taylor & Francis, (under
minor revision).
22. Sumit, Sandeep Kumar, Lalit Singh, “A Petri net based approach for
reliability prediction of a safety system used in Nuclear Power Plant”,
in IET Software, (under major revision).
23. Lalit Singh, Gopika Vinod, A.K. Tripathi, “Reliability Prediction
through System Modeling”, in ACM SIGSOFT Soft. Engineering
Notes, vol. 38, Nov, 2013, pp.1-10.
24. Lalit Singh, Gopika Vinod, A.K. Tripathi, “Impact of Change in
Component Reliabilities on System Reliability Estimation,” in ACM
SIGSOFT Software Engineering Notes, June, 2014, pp.1-6.

48