Risk-based oversight

Gian Andrea Bandieri

Principal Standardisation Coordinator
EASA Flight Standards Directorate

20 March 2018
TE.GEN.00409-001
 PBE = PBR + RBO

EASA paper: A Harmonised European Approach to a Performance-Based Environment

20 March 2018 RBS Workshop - Lima 2
 What is RBO? Definitions:

Oversight: the function by means of which a

competent authority ensures that the
applicable requirements are met by
regulated entities
Surveillance: The State activities through which the State proactively
verifies through inspections and audits that aviation licence,
certificate, authorization or approval holders continue to meet the
established requirements and function at the level of competency and
safety required by the State. ICAO Annex 19, Second Edition, July 2016.

Risk Based Oversight:

A way of performing oversight, where
planning is driven by the combination of
Planning Execution
risk profile and safety performance; and
execution focuses on the management
of risks, besides ensuring compliance.

20 March 2018 RBS Workshop - Lima 3

 Why is RBO interesting? - 1

Regulatory
Ineffectiveness
Illegal Unsafe
Data driven
Targeted use of
resources
Focus on safety
No Common Unique
Risk Cause Cause Positive effect of
prescriptive
requirements is
Regulatory Regulatory SMS maintained
Inefficiency Effectiveness Effectiveness
adapted from M. Sparrow, ‘The Regulatory Craft’

20 March 2018 RBS Workshop - Lima 4

 Why is RBO interesting? - 2
DISCLAIMER
Qualitative assumption
based on successful
implementation

20 March 2018 RBS Workshop - Lima 5

 Practices

Simple
Practical
Easy to implement

Cross-domain team
FS1: Maintenance & Production
FS2: Air Operations
FS3: Aircrew & Medical
FS4: ATM/ANS & Aerodromes
FS5: project coordination

Available on EASA website:

http://www.easa.europa.eu/document-library/general-
publications/practices-risk-based-oversight

20 March 2018 RBS Workshop - Lima 6

 RBO Paper - TOC
2. Introduction I Risk profile - Best practices
•2.1 Why RBO
•2.2 Definitions, conceptual model and link with the EASA
rules • I - 1 AT as example of simple risk profile
•2.3 EASp, SSP and management system as drivers to RBO • I - 2 The Irish method – another simple model
•2.4 Limits of RBO • I - 3 ES more elaborated risk profile, collaboration
between components of the CAA
3. Risk Profile and oversight planning • I - 4 CH risk profile linked with SSP
• I - 5 Contribution from Finland

4. Enablers and tools

•4.1 Management of safety information II Tools supporting RBO
•4.2 Information sharing with other Competent Authorities
•4.3 Training and qualification of inspectors
• II - 1 SMICG tools
5. Conduct of Risk-Based Audits • II - 1 ICAO tools
•5.1 UK experience: transforming the CAA and strengthening
the people capability to operate in a RBO environment
III Questionnaire used for the collection of
•5.2 Risk based audit: issues for discussion
•5.3 Attitude during the conduct of risk based audits
the best practices
•5.4 Accountabilities and enforcement
• III – 1 Introduction
• III – 2 Questions on Risk Based Oversight
6. Experiences – success stories
•6.1 RBO in practice by Switzerland
•6.2 Success stories from UK-CAA IV Draft documents from the WG of ATM
•6.3 Success story from Ireland Competent authorities
•6.4 An approach scaled to one specific sector – helicopter
safety in Norway • IV - 1 Elements for establishing an RBO audit plan
•6.5 Experience from Sweden • IV - 2 Outcome of a questionnaire on ATM RBO
Processes

20 March 2018 RBS Workshop - Lima 7

 The starting point

ARx.GEN.305(b) • the results of past

For organisations certification and/or
certified by the oversight activities
competent authority, the required by
oversight programme ARO.GEN and
shall be developed taking ARO.RAMP, and
into account shall be based on the
• the specific nature of assessment of
the organisation, associated risks.
• the complexity of its
activities,

WHICH RISK SHOULD WE CONSIDER?

20 March 2018 RBS Workshop - Lima 8
 Types of Risk

Any event or issue that could occur and adversely impact the achievement of
Business risk the Agency’s political, strategic and operational objective. Lost opportunities are
also considered as risks.
A condition or an object with the potential to cause or contribute to an aircraft
Hazard
incident or accident.

The predicted probability and severity of the consequences or outcomes of a

Safety risk
hazard.

Operational
The safety risk connected with the performance of operations
risk
RISK BASED OVERSIGHT: a way of performing oversight allowing the
competent authority to:
i) prioritise and plan its activities based on compliance, risk profiling and
assessment of the safety performance; and
ii) verify compliance with a focus on management of operational risks.

WE CONSIDER THE OPERATIONAL RISK OF THE AUDITEE

20 March 2018 RBS Workshop - Lima 9
 Risk profile and safety performance

The elements of risk that are

Risk Profile inherent to the nature and the
operations of the regulated entity

20 March 2018 RBS Workshop - Lima 10

 Risk profile and safety performance

Safety The demonstration of how

effectively can a regulated entity
performance mitigate its risks

20 March 2018 RBS Workshop - Lima 11

Conceptual RBO Model

competence
Questioning Knowledge

Inspectors’
Tools
Risk Assessment Skills

Quantitative methods Attitude

SAFETY
RISK PROFILE
RISK BASED OVERSIGHT PERFORMANCE

Planning Short term

Long term analysis

Execution
Compliance Assessment of risk
verification management
 RBO Enablers

Management of Accountabilities,
Information
safety responsibilities
sharing and enforcement
information

Organisational Mature safety

Culture
requirements management

Inspector
Competence
and
qualifications
 • Oversight planning and determination of oversight cycle for each organisation should take
into consideration the risk profile and the assessment of the safety performance.
When the risk profile relies on expert judgment, the decision making should be made by
1 consensus by a team of experts.

• For each organisation, RBO parameters should be continuously monitored at an appropriate

frequency in order to identify any trend and to review the oversight programme, its cycle
and the safety objectives.
2 The competent authority should continuously follow-up and improve the overall RBO
system.

• The ICAO state safety programme (SSP) should be established and used as a background
framework for RBO and the competent authority should have a functioning management
3 system, as required by the rules.

• The state oversight system should be mature enough before it can be complemented by
RBO. This oversight approach should be linked to the objectives of the SSP and of the
management system of the competent authority.
4 EASp actions should also be taken in consideration.

• The management system of the competent authority should capture the different risk
profiles of the regulated entities according to a model.
When determination of risk profile relies on expert judgment, decision making should be
5 made by consensus by a team of experts.
 • RBO should be progressively deployed and extension of RBO to additional domains
should be consistent and appropriate. Initial introduction of RBO could be facilitated by a
6 dedicated team of “champions’ inspectors.

• A system in place for the collection, analysis, and exchange of safety data at the level of
State and regulated entity is a prerequisite for RBO, as well as safety management
principles and a just culture environment.
Exchange of information on safety risks between competent authority and regulated
7 entities should be established.
Development of an integrated risk picture in and across different domains should be
done in partnership with involved stakeholders.

• Competent Authorities should develop arrangements for cooperation on oversight,

exchange of collected safety information, sharing of RBO experience, feedback on
8 experience with the SSP etc…

• Initial and continuous training should be given to inspectors implementing RBO, to cover:
• development of proper culture when interacting with industry
• use of expert judgment, specially when safety performance and “gut feeling” are
blended
9 • use of RBO-specific tools available at the competent authority.
• Support and coaching should be available during the initial phase of RBO deployment.
 What is EASA doing

20 March 2018 RBS Workshop - Lima 16

 RBO applied to Continuing Airworthiness Organisations

Overall /
Feedback from EASA safety risk
Country Risk Intrinsic
the Team portfolio for EU
profile organisation
Leader Aviation
risk profile

Oversight period
& audit frequency   
Focus areas  
Confidence level  
Allocation of
oversight task  
Initial oversight
period & audit
frequency
  
20 March 2018 RBS Workshop - Lima 17
 RBO principles do not apply

For planning of initial investigation

For the allocation of the task (EASA or external
partner)
When ad-hoc oversight is needed due to high in the
following areas:
Country Risk Factor
E.g.: ICAO SSC flag or an EU Safety lists flag
Organisation Risk Factor
E.g.: in case of suspension or limitation of the approval
Team Leader Input Risk Factor
specific cases known by the team leader in charge

20 March 2018 RBS Workshop - Lima 18

 Profile & Performance

Ratings & Level of Number of Specialized Maintenance

Limitations maintenance staff service sites Organisation Performance Risk Profile

Scope of Approval

Intrinsic organisation risk profile

Non Use of the

Number of
Permanent approval &
years using Fabrication
Outsourcing staff & Part- Other
the EASA of parts
66 licensesd approval
approval
Engineers held

Organisation Activity
20 March 2018 RBS Workshop - Lima 19
 How and when we are going to use it

Conceptual development completed

Test cases run  good correlation

Operational deployment: end 2018, when

finding database will be ready

20 March 2018 RBS Workshop - Lima 20

 From RBO to PBO

Level of
Compliance-based Compliance

PROFILE
RISK
Risk-based

TARGETS
SPIs &
Performance based

20 March 2018 RBS Workshop - Lima 21

 To take home

Risk Based Oversight:

A way of performing oversight, where
planning is driven by the

competence
Questioning Knowledge

Inspectors’
combination of risk profile and safety

Tools
performance; and execution focuses Risk Assessment Skills

on the management of risks, besides Quantitative methods Attitude

ensuring compliance.

SAFETY
RISK PROFILE
RISK BASED OVERSIGHT PERFORMANCE

Planning Short term

Long term analysis

Execution
Compliance Assessment of risk
verification management
Thanks for your attention

For further information:

gian-andrea.bandieri@easa.europa.eu
 RBO Benefits – A Regulators View

Safety
Resilience Flexibility
management
regulatory framework focus on safety
risk management
capable of outcomes
capabilities in a
anticipating and self- encouraging
flexible framework
adapting to change innovation

response to increased
better allocation of complexity, new means to control
resources to address business models and specific risks not
the risks identified technological restricted in priority
development
“Without pioneers, the world will turn but
never move forward”