Risk-based oversight
Gian Andrea Bandieri
Principal Standardisation Coordinator
EASA Flight Standards Directorate
20 March 2018
TE.GEN.00409-001
PBE = PBR + RBO
EASA paper: A Harmonised European Approach to a Performance-Based Environment
20 March 2018 RBS Workshop - Lima 2
What is RBO? Definitions:
Oversight: the function by means of which a
competent authority ensures that the
applicable requirements are met by
regulated entities
Surveillance: The State activities through which the State proactively
verifies through inspections and audits that aviation licence,
certificate, authorization or approval holders continue to meet the
established requirements and function at the level of competency and
safety required by the State. ICAO Annex 19, Second Edition, July 2016.
Risk Based Oversight:
A way of performing oversight, where
planning is driven by the combination of
Planning Execution
risk profile and safety performance; and
execution focuses on the management
of risks, besides ensuring compliance.
20 March 2018 RBS Workshop - Lima 3
Why is RBO interesting? - 1
Regulatory
Ineffectiveness
Illegal Unsafe
Data driven
Targeted use of
resources
Focus on safety
No Common Unique
Risk Cause Cause Positive effect of
prescriptive
requirements is
Regulatory Regulatory SMS maintained
Inefficiency Effectiveness Effectiveness
adapted from M. Sparrow, ‘The Regulatory Craft’
20 March 2018 RBS Workshop - Lima 4
Why is RBO interesting? - 2
DISCLAIMER
Qualitative assumption
based on successful
implementation
20 March 2018 RBS Workshop - Lima 5
Practices
Simple
Practical
Easy to implement
Cross-domain team
FS1: Maintenance & Production
FS2: Air Operations
FS3: Aircrew & Medical
FS4: ATM/ANS & Aerodromes
FS5: project coordination
Available on EASA website:
http://www.easa.europa.eu/document-library/general-
publications/practices-risk-based-oversight
20 March 2018 RBS Workshop - Lima 6
RBO Paper - TOC
2. Introduction I Risk profile - Best practices
•2.1 Why RBO
•2.2 Definitions, conceptual model and link with the EASA
rules • I - 1 AT as example of simple risk profile
•2.3 EASp, SSP and management system as drivers to RBO • I - 2 The Irish method – another simple model
•2.4 Limits of RBO • I - 3 ES more elaborated risk profile, collaboration
between components of the CAA
3. Risk Profile and oversight planning • I - 4 CH risk profile linked with SSP
• I - 5 Contribution from Finland
4. Enablers and tools
•4.1 Management of safety information II Tools supporting RBO
•4.2 Information sharing with other Competent Authorities
•4.3 Training and qualification of inspectors
• II - 1 SMICG tools
5. Conduct of Risk-Based Audits • II - 1 ICAO tools
•5.1 UK experience: transforming the CAA and strengthening
the people capability to operate in a RBO environment
III Questionnaire used for the collection of
•5.2 Risk based audit: issues for discussion
•5.3 Attitude during the conduct of risk based audits
the best practices
•5.4 Accountabilities and enforcement
• III – 1 Introduction
• III – 2 Questions on Risk Based Oversight
6. Experiences – success stories
•6.1 RBO in practice by Switzerland
•6.2 Success stories from UK-CAA IV Draft documents from the WG of ATM
•6.3 Success story from Ireland Competent authorities
•6.4 An approach scaled to one specific sector – helicopter
safety in Norway • IV - 1 Elements for establishing an RBO audit plan
•6.5 Experience from Sweden • IV - 2 Outcome of a questionnaire on ATM RBO
Processes
20 March 2018 RBS Workshop - Lima 7
The starting point
ARx.GEN.305(b) • the results of past
For organisations certification and/or
certified by the oversight activities
competent authority, the required by
oversight programme ARO.GEN and
shall be developed taking ARO.RAMP, and
into account shall be based on the
• the specific nature of assessment of
the organisation, associated risks.
• the complexity of its
activities,
WHICH RISK SHOULD WE CONSIDER?
20 March 2018 RBS Workshop - Lima 8
Types of Risk
Any event or issue that could occur and adversely impact the achievement of
Business risk the Agency’s political, strategic and operational objective. Lost opportunities are
also considered as risks.
A condition or an object with the potential to cause or contribute to an aircraft
Hazard
incident or accident.
The predicted probability and severity of the consequences or outcomes of a
Safety risk
hazard.
Operational
The safety risk connected with the performance of operations
risk
RISK BASED OVERSIGHT: a way of performing oversight allowing the
competent authority to:
i) prioritise and plan its activities based on compliance, risk profiling and
assessment of the safety performance; and
ii) verify compliance with a focus on management of operational risks.
WE CONSIDER THE OPERATIONAL RISK OF THE AUDITEE
20 March 2018 RBS Workshop - Lima 9
Risk profile and safety performance
The elements of risk that are
Risk Profile inherent to the nature and the
operations of the regulated entity
20 March 2018 RBS Workshop - Lima 10
Risk profile and safety performance
Safety The demonstration of how
effectively can a regulated entity
performance mitigate its risks
20 March 2018 RBS Workshop - Lima 11
Conceptual RBO Model
competence
Questioning Knowledge
Inspectors’
Tools
Risk Assessment Skills
Quantitative methods Attitude
SAFETY
RISK PROFILE
RISK BASED OVERSIGHT PERFORMANCE
Planning Short term
Long term analysis
Execution
Compliance Assessment of risk
verification management
RBO Enablers
Management of Accountabilities,
Information
safety responsibilities
sharing and enforcement
information
Organisational Mature safety
Culture
requirements management
Inspector
Competence
and
qualifications
• Oversight planning and determination of oversight cycle for each organisation should take
into consideration the risk profile and the assessment of the safety performance.
When the risk profile relies on expert judgment, the decision making should be made by
1 consensus by a team of experts.
• For each organisation, RBO parameters should be continuously monitored at an appropriate
frequency in order to identify any trend and to review the oversight programme, its cycle
and the safety objectives.
2 The competent authority should continuously follow-up and improve the overall RBO
system.
• The ICAO state safety programme (SSP) should be established and used as a background
framework for RBO and the competent authority should have a functioning management
3 system, as required by the rules.
• The state oversight system should be mature enough before it can be complemented by
RBO. This oversight approach should be linked to the objectives of the SSP and of the
management system of the competent authority.
4 EASp actions should also be taken in consideration.
• The management system of the competent authority should capture the different risk
profiles of the regulated entities according to a model.
When determination of risk profile relies on expert judgment, decision making should be
5 made by consensus by a team of experts.
• RBO should be progressively deployed and extension of RBO to additional domains
should be consistent and appropriate. Initial introduction of RBO could be facilitated by a
6 dedicated team of “champions’ inspectors.
• A system in place for the collection, analysis, and exchange of safety data at the level of
State and regulated entity is a prerequisite for RBO, as well as safety management
principles and a just culture environment.
Exchange of information on safety risks between competent authority and regulated
7 entities should be established.
Development of an integrated risk picture in and across different domains should be
done in partnership with involved stakeholders.
• Competent Authorities should develop arrangements for cooperation on oversight,
exchange of collected safety information, sharing of RBO experience, feedback on
8 experience with the SSP etc…
• Initial and continuous training should be given to inspectors implementing RBO, to cover:
• development of proper culture when interacting with industry
• use of expert judgment, specially when safety performance and “gut feeling” are
blended
9 • use of RBO-specific tools available at the competent authority.
• Support and coaching should be available during the initial phase of RBO deployment.
What is EASA doing
20 March 2018 RBS Workshop - Lima 16
RBO applied to Continuing Airworthiness Organisations
Overall /
Feedback from EASA safety risk
Country Risk Intrinsic
the Team portfolio for EU
profile organisation
Leader Aviation
risk profile
Oversight period
& audit frequency
Focus areas
Confidence level
Allocation of
oversight task
Initial oversight
period & audit
frequency
20 March 2018 RBS Workshop - Lima 17
RBO principles do not apply
For planning of initial investigation
For the allocation of the task (EASA or external
partner)
When ad-hoc oversight is needed due to high in the
following areas:
Country Risk Factor
E.g.: ICAO SSC flag or an EU Safety lists flag
Organisation Risk Factor
E.g.: in case of suspension or limitation of the approval
Team Leader Input Risk Factor
specific cases known by the team leader in charge
20 March 2018 RBS Workshop - Lima 18
Profile & Performance
Ratings & Level of Number of Specialized Maintenance
Limitations maintenance staff service sites Organisation Performance Risk Profile
Scope of Approval
Intrinsic organisation risk profile
Non Use of the
Number of
Permanent approval &
years using Fabrication
Outsourcing staff & Part- Other
the EASA of parts
66 licensesd approval
approval
Engineers held
Organisation Activity
20 March 2018 RBS Workshop - Lima 19
How and when we are going to use it
Conceptual development completed
Test cases run good correlation
Operational deployment: end 2018, when
finding database will be ready
20 March 2018 RBS Workshop - Lima 20
From RBO to PBO
Level of
Compliance-based Compliance
PROFILE
RISK
Risk-based
TARGETS
SPIs &
Performance based
20 March 2018 RBS Workshop - Lima 21
To take home
Risk Based Oversight:
A way of performing oversight, where
planning is driven by the
competence
Questioning Knowledge
Inspectors’
combination of risk profile and safety
Tools
performance; and execution focuses Risk Assessment Skills
on the management of risks, besides Quantitative methods Attitude
ensuring compliance.
SAFETY
RISK PROFILE
RISK BASED OVERSIGHT PERFORMANCE
Planning Short term
Long term analysis
Execution
Compliance Assessment of risk
verification management
Thanks for your attention
For further information:
[email protected]
RBO Benefits – A Regulators View
Safety
Resilience Flexibility
management
regulatory framework focus on safety
risk management
capable of outcomes
capabilities in a
anticipating and self- encouraging
flexible framework
adapting to change innovation
response to increased
better allocation of complexity, new means to control
resources to address business models and specific risks not
the risks identified technological restricted in priority
development
“Without pioneers, the world will turn but
never move forward”
