Presentasi Penyegaran Auditor Ditjen Kereta API Oleh Kasubdit Kelaikudaraan
Presentasi Penyegaran Auditor Ditjen Kereta API Oleh Kasubdit Kelaikudaraan
Tatacara Audit
Keselamatan dan
Pembuatan Laporan
Audit
14 September 2021
OUTLINE
SUDUT PANDANG
RISK BASED
OVERSIGHT SYSTEMS
LOOPING
Regulator:
Controlled SDR • Effective program surveillance (systematic
oversight system)
• Approaching to safety culture (Safety through
NIL Accident/Incident safety risk management not compliance)
• Focus only particular area that still need
improvement, etc.
Deviation Event
High Audit
Risk Risk Based Surveillance
Ramp Check
MOR
Oversight Accident /
Incident
System SDR
MISR
2021 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
Sampling Witness
Inspection on Heavy
Maintenance
Implementasi AD
etc
PELAKSANAAN
LAPORAN SURVEILLANCE YA
& INPUT DATABASE
CORRECTIVE ACTION
ADA YA ACCEPTED YA
FINDING (S) FINDING (S)
? ?
TIDAK APPROVED
TIDAK
CORRECTIVE
FEEDBACK OPERATOR & INPUT TIDAK ACTION (CA)?
DATABASE
APPROVED by COORDINATOR YA
(DATABASE)
SELESAI
22
What is an Audit?
• The process Audit establishes whether “what” is actually done
“conforms to” what is said to be done’ or ‘what should be done’.
• Audit outcomes need to be reported to senior management to let
them know the degree to which the company, its systems and its
operations comply with documented requirements.
• Key element in this whole process is a formal program detailing just
Who will audit What, and When.
23
Surveillance
• The purpose of surveillance is to conduct inspections to ensure
compliance with the appropriate technical and operational approved
documents.
• Surveillance results may indicate the need for a non-scheduled audit
of all or part of an organization.
24
Area Audit
• The areas examined include at least a re-evaluation of:
- the operator's organization,
- management effectiveness and control,
- facilities,
- equipment,
- ground handling,
- aircraft maintenance,
- operational control and supervision,
- flight and duty time records,
25
Area Audit
- maintenance of flight and cabin crew standards,
- passenger and cargo safety procedures,
- dangerous goods procedures,
- operational and personnel records, training,
- company manuals,
- financial viability and record of compliance with the
provisions of the AOC,
- the associated operations specifications and
- pertinent operating regulations.
26
SI 8900-6.3 AOC AUDIT AREA
AIRWORTHINESS
a) Management and administration;
b) Company Maintenance Manuals (CMM);
c) Publication/ Library;
d) Maintenance Personnel Training Program and Record;
e) Maintenance Record System;
f) Maintenance Facilities;
g) Maintenance Contractual Arrangement;
h) Minimum Equipment List (MEL);
i) Maintenance Program;
j) Maintenance Inspection System & RII Evaluation;
k) Continuing Analysis and Surveillance Program;
l) Reporting Procedure and SDR Evaluation.
27
SI 8900-6.3 AOC AUDIT AREA OPERATIONS
a) Management and administration;
b) Operation Manual;
c) Publication/ Library;
d) Company check pilot program (Check pilot);
e) Training Program and Training Record (Flight crew, Flight Attendant, and Flight Operations
Officer);
f) Dispatch & Flight Watch/ Operation Control;
g) Flight Documentation / Trip Record;
h) Aircraft documentation;
28
SI 8900-6.3 AOC AUDIT AREA OPERATIONS
i) Flight Time and duty record;
j) Flight inspection and route check (Cockpit enroute);
k) Aircraft performance;
l) Cabin safety;
m) Station facility inspection;
n) Pilot proficiency check;
o) Minimum Equipment List (MEL);
p) Dangerous goods.
29
SI 8900-6.3 AOC AUDIT AREA SMS
a) Safety management system (SMS) manual;
b) SMS Implementation
c) Flight data analysis (FDA);
d) Reporting of incident;
e) Internal safety audit;
30
SI 8900-6.3 AOC AUDIT RISK INDICATOR
a) Financial Change;
b) Labour Difficulties
c) Management Practices
d) Poor internal Audit or Quality Assurance Program
e) Change in operational scope or additional Authorities
f) Changes in Contracting for services
g) High Turnover in personnel
h) Less of Key Personnel
i) Addition or change to product line
31
SI 8900-6.3 AOC AUDIT RISK INDICATOR
j) Poor Accident or Safety Record;
k) Merger or Takeover
l) Regulatory Record
32
SI 8900-6.3 PARAREL REPORT
When audit or surveillance findings are identified against CASR and/
or Aviation Act no. 1 year 2009, the audit manager or PAI/ POI will
prepare a parallel report for the DGCA management.
33
Term and Definitions
• Audit Criteria :
Policies, procedures or requirements used as the standard against
which objective evidence is measured. Audit criteria are used as a
reference against which audit evidence is compared.
• Audit evidence:
This can include records, statements of fact or other information
relevant to the audit criteria and which can be verified. This may be
qualitative or quantitative.
34
Terms and Definitions
• Audit findings:
The result once the audit evidence has been evaluated against the
audit criteria. The findings would reveal conformity or nonconformity
with audit criteria or present opportunities for improvement.
• Audit conclusion:
This is the outcome of the audit, provided by the audit team, after
consideration of the audit objectives and all the audit findings.
35
• Auditee : Organization being audited
• Auditor : Person with the competence to conduct an
audit
• Audit team: One or more auditors conducting an
audit, supported, if needed, by technical experts.
• Technical expert: Person who provides specific
knowledge or expertise to the audit team
• Competence: Demonstrated personal attributes and
demonstrated ability to apply knowledge and skills.
36
• Audit plan:
Description of the audit activities and arrangements
• Audit program:
Set of one or more audits planned for a specific time-frame and
directed towards a specific purpose. An audit program includes all
activities necessary for planning, organizing and conducting the
audits.
37
• Audit scope:
Extent and boundaries of an audit.
38
DGCA AUDIT Reff. SI 8900-6.3
• Audit Types:
• 1. Initial Certification Audit
• 2. Amendment / Additional Approval Audit
• 3. Routine Conformance Audit
• 4. Special Purpose Audit
39
Initial Certification
• Prior to the issue of an Air Operating Certificate all areas of a
company will be audited to ensure that the required standards will be
met. The company will be required to show how the applicable
standards will be met by describing all procedures in a Company
Maintenance, Operations and/or Training manual, as appropriate.
40
Request for Additional Authority
• Prior to granting additional authorities to amend the Air Operator
Certificate, full or partial audit may be conducted.
41
Special Purpose Audit
• An audit which was not previously identified as a requirement in the
DGCA Audit Plan, but deemed necessary due to operational
circumstances. These audits are non-scheduled in that they may be
the result of a deficiency identified by routine surveillance of the
organization.
42
Organization Renewal Audit
• Every two year, prior to the renewal of an Air Operator Certificate
43
Audit Objective
• Conformance
- this will provide confidence that agreed policies and working practices are being
maintained.
• Effectiveness
– this will ensure that the management is capable or realizing the objectives
which have been set.
• Improvement
– this will identify opportunities for improvement of products, services, processes
and working practices which will have a positive impact on customer satisfaction
and business result.
44
7 steps in the Auditing Process
3. Preparation on
1. Planning 2. Document review
site Audit
5. Preparing,
6. Completing the Approving and
4. Onsite Audit
Audit Distributing the
Audit Report
7. Follow up Audit
45
Three major components in
Management System Auditng
46
Management of an Audit Program
Authority for the audit program
Improving
Act the Audit
program Implementing the audit Competence and
program evaluation of
+Scheduling Audits auditors
+ Evaluating the auditors Do
+ Selecting audit teams
+ Directing audit activities Audit activities
+ Maintaining records
47
Authority for the Audit Program
• The senior management of an organization grants the authority for
managing the audit program.
• The responsible personnel must establish, implement, monitor,
review and improve the program as well as identify and provide
necessary resources.
48
Establishing The Audit Program
• Appropriate objectives for planning and carrying out the audits
should be established.
• The scope of an audit program may vary and will most likely depend
upon the size and complexity of the organization to be audited.
• Personnel : Assigned responsibility for managing an audit program
should have an understanding of the audit principles, competence
requirements for auditors and auditing techniques.
49
Establishing The Audit Program (cont)
• Resources: Need to be allocated taking into consideration the extent of the audit program,
competence of available auditors and technical experts, auditing techniques and financial
resources.
• Documented procedures should address such activities as:
• planning and scheduling,
• selection of audit teams,
• conducting audits,
• monitoring the effectiveness of the audit program,
• method of reporting to the senior management.
50
Implementing the Audit Program
• The implementation of an audit program should cover:
- Communications,
- Co-ordination and scheduling activities,
- Evaluation of the auditors and their continued
professional development,
- The selection of audit teams,
- Directing the audit activities, and
- Ensuring records are kept of all processes to demonstrate the implementation of
the audit program.
51
Monitoring and Reviewing the Audit Program
• The audit program should, at appropriate intervals, be reviewed to
assess whether the audit objectives have been met, and to identify
opportunities for improvement.
52
Competence and Evaluation of Auditors
• One of the functions of managing an audit program is to ensure that
the auditors and technical experts have the competence appropriate
to the particular audit.
• Competence is based on education, training, skills and experience. An
effective audit process depends on the competence of those
conducting the audit.
53
Competence and Evaluation of auditor (cont)
• Specific knowledge and skills of the auditors Auditors should have the
training and work experience needed to acquire the knowledge and
skills for the aviation industry being audited. There are times when
there is a specialist area to be covered in the audit and this would call
for the appropriate specialist to be included in the audit team.
54
Competence and Evaluation
of Auditors (cont)
55
Principles of Auditing :
• Ethical conduct – the foundation of professionalism
• Trust and integrity – confidentiality and discretion
• Fair presentation – report truthfully and accurately
• Due professional care – the application of diligence and judgment in auditing.
• Independence – the impartiality and objectivity of audit conclusions
• Evidence base approach – the rational basis for reaching reliable and
reproducible audit conclusions in a systematic audit process
• Objective – free from bias
56
Principles of Auditing
• Ethical conduct:
The very foundation of professionalism. Trust, integrity, confidentiality and
discretion are essential to auditing.
• Fair presentation:
The need to report truthfully and accurately. Audit findings, audit conclusions and
audit reports reflect accurately and truthfully the audit results. Significant
obstacles encountered during the audit and unresolved diverging opinions
between the audit team and the auditee are reported.
57
Principles of Auditing (cont)
• Due professional care
The application of diligence and judgment in auditing. Auditors
exercise care in accordance with the importance of the task they
perform and the confidence placed in them by audit clients and other
interested parties. Having the necessary competence is an important
factor.
58
Principles of Auditing (cont)
• Evidence-based approach
The rational method for reaching reliable and reproducible audit
conclusions in a systematic audit process must be based on samples
of the information available, since an audit is conducted during a
finite period of time and with finite resources. The appropriate use of
sampling is closely related to the confidence that can be placed in the
audit conclusions.
59
Principles of Auditing(cont)
• Independence
The basis for the impartiality of the audit and objectivity of the audit
conclusions is that auditors are independent of the activity being
audited and are free from bias and conflict of interest. Auditors
maintain an objective state of mind throughout the audit to ensure
that the audit findings and conclusions will be based only on the audit
evidence.
60
Procedures and Techniques
• Plan, organize and keep to the agreed time schedule
• Choose important items for main focus
• Listen, observe and review records and documents, and communicate well
• Use sampling techniques effectively Ascertain accuracy of collected information and appropriate
amount of evidence required to support audit findings
• Be aware of factors that may determine reliability of audit findings and conclusions
• Use appropriate working papers
• Maintain confidentiality and security of information presented
61
Participants of Audit
• Team Leader / Lead auditor
• Auditor
• Technical Expert
• Auditee / Auditee management
• Guides
62
Lead Auditors
Audit Team Leaders need additional skills, such as:
• Good resource management: Effective use of resources during
audit
• Effective communications: Leading audit team effectively in
communications & organization of audit
• Ability to guide and direct team: Ability to direct and guide any
trainee auditors in team
• Conflict resolution: Awareness of any need or assistance by
audit team to reach conclusions and/or prevent and resolve
conflicts
• Good reporting techniques: To assist audit team in completion
of audit report
63
Qualities of Auditors
• Sound knowledge of the aeronautical regulations and technical
expertise;
• Fundamental knowledge of the industry or activity being
regulated;
• Skills in communication;
• Be able to anticipate problems and assess the seriousness of
observe weakness;
• Be able to organize his/her work, to question to look for
objective evidence to construct checklists and use them
effectively without being limited by them.
• Have knowledge of the criteria against which the audited
object is to be evaluated
64
Qualities of Auditors (cont)
• In addition, inspectors involved in audit assignments require other skills and
qualities such as:
• Salesmanship – opening the job in a hostile environment, securing information,
the whole truth and selling new ideas or procedures;
• Ability to analyze - to break the whole in to parts, and to determine validity;
• Creativity - to develop a clear picture of a situation and to produce a solution;
• Personal Suitability - courtesy, appearance, integrity and competence
65
Auditor Evaluation
• Organizations should establish the levels of education, work experience, training and audit
experience that an auditor needs to gain knowledge and skills appropriate to the audit program.
• Having identified the levels of competence, a process of evaluation should be planned and
implemented.
• The evaluation of auditors against the criteria should be carried out:
• on those who wish to become auditors,
• as part of the audit team selection process,
• as a continuous evaluation of the auditor performance to identify needs for improvement.
66
Auditor Evaluation (cont)
• If the auditors become aware, prior to the audit, of specific areas in
the audit that are outside the audit team’s scope and expertise, then
they should source an area need specialist and, if necessary,
reschedule the audit for when the specialist is available.
• Audit team members should be selected based on their experience,
availability and have relevant knowledge of the area being audited.
67
Audit Program Objectives
• The purpose behind an audit could be:
- to determine the conformity or nonconformity with specified
requirements (i.e. CASR);
- to determine the capability of a supplier,
- to provide the auditee with an opportunity to improve the
management system,
- to meet regulatory requirements,
- to meet the requirements for certification to a management
system or regulatory standard.
69
Effective Auditing (cont)
The essential elements for an efficient audit program are:
• Planning. The audit program should be planned and systematic.
• Procedures approach. Audit procedures should be in place to ensure uniformity
of approach, authorities and responsibilities.
• Access to facilities. Audit staff should have reasonably unlimited access to
facilities, documents and personnel.
• Access to management. Audit staff must have the freedom of access to all levels
of management.
70
Completing the Audit
• Finalizing the Audit
“The audit is completed when all activities described in the audit
plan have been carried out and the approved audit report has been
distributed”
71
Preparing, Approving And
Distributing The Audit Report
• Content Audit Report
• Objectives and scope of the audit
• Who was involved in the audit
• The audit plan
• Dates and sites
• Audit criteria used
72
Preparing, Approving And
Distributing The Audit Report
• Content Audit Report (cont)
• The finding of the audit
• Audit conclusions
• Unresolved issues/differences of opinions
• Improvement opportunities
• Follow-up actions
• Distribution list
• Statement of confidentiality
73
The Audit Report should NOT include:
• Confidential information given in interviews
• Matters not raised or discussed at closing meeting
• Subjective opinions – verifiable facts only
• Ambiguous statements
• Antagonistic words or phrases
74
Conducting Audit Follow-up
• The completion and effectiveness of corrective action should be
verified.
This verification can be part of subsequent audit.
75
Conducting Audit Follow-up
The auditor could be looking for objective evidence of corrective action in
such :
• Amended written procedures
• Records
• Additional training
• Programme of corrective action
• Programme of preventive action
• Amended quality objectives
• Results of internal audit
• Report of discussion with supplier
76
Follow-up Auditor
- After providing the formal Audit Report and relevant
findings the auditor may be asked to suggest
solutions to the problems identified during the audit.
77
Follow-up Auditor
- The auditor can:
1. Help identify the cause
2.Review suggested options (perhaps using tool)
3.Verify that the non-conformance has been effectively
addressed
78
Follow-up Auditee
• This Follow Up is the responsibility of the company's management
who must:
- Investigate the findings presented by the auditor.
- Identify the cause of the non-compliance.
79
Follow-up Auditee
- Decide upon necessary Corrective Action that will prevent the
problem from re-occurring.
- Implement the corrective action.
- Monitor the result of the corrective action to ensure it has been
effective
80
REFF: SI 8900-6.3
• AUDIT REPORT
The audit report should normally be presented to the company within
ten working days. Any delay must be documented since the validity of
the audit will be compromised if the report is not presented in a
timely manner. Although draft audit finding forms may be left with
the company as a courtesy, this is not mandatory
81
REFF: SI 8900-6.3
• AUDIT REPORT
• Part I - Introduction, which summarizes the audit process and the content of the audit report;
• Part II - Executive Summary of Findings, which summarizes the most significant findings for the
information of the senior management of the audit and DGCA;
• Part III - Airworthiness, which contains the functional summaries for airworthiness;
• Part IV - Operations, which contains the functional summaries for operations;
• Part V - Safety, which contains the functional summaries for safety; and
• An Appendix, which contains the audit findings for both airworthiness and operations.
82
REFF: SI 8900-6.3
• AUDIT REPORT
The audit report will be a factual account of the audit and will not
include subjective statements, suggestions or recommendations.
83
REFF: SI 8900-6.3
• AUDIT REPORT
The Convening Authority will sign the covering letter and forward it,
with a copy of the audit report, to the company.
The letter will outline the procedure for responding to audit findings
and specify the required response time of 15 working days from the
time the company receives the report.
84
REFF: SI 8900-6.3
ROOT CAUSE ANALYSIS AND CORRECTIVE ACTION
A method of analysis that focuses on identifying the root cause(s) of an undesirable situation or
condition of finding (non-conformities).
The root cause analysis is the tool or method designed to help auditee and auditors describe what
happened during a particular finding (non-conformities), then determine how it happened and
understand why it happened.
85
REFF: SI 8900-6.3
ROOT CAUSE ANALYSIS AND CORRECTIVE ACTION
• A method of describing and schematically representing finding (non-conformities) sequence and its
contributing conditions.
• A method of identifying the critical events, conditions, and unsafe acts in the sequence. This is where
different root cause analysis techniques come into play (Ishikawa, 5-whys, etc.) with varying degrees of
complexity.
• Based on the identification of the critical events, conditions, and unsafe acts, a method for systematically
investigating the management and organisational factors that allowed the unsafe acts to occur i.e. a method
to uncover the root cause.
86
REFF: SI 8900-6.3
Type of Corrective Action
1. Immediate corrective action/Level 1 for non-compliance finding
2. Short-Term Corrective Action/Level 2 for Non-Conformance
finding.
3. Long-Term Corrective Action/Level 3 for Non-adherence finding
87
REFF: SI 8900-6.3
88
Bagaimana memiliki Auditor yang Kompeten?
• Di DKPPU kita memiliki SI 8900-1.3 tentang Inspector Training System
Inspector Training System
• Provides a comprehensive system for managing the training and
qualification of new inspectors that includes:
ICAO requirements
Formal classroom training course requirements
Inspector training profiles
Technical requirements for all inspector specialties
Inspector Training System
• Fully addresses:
Training system management
Planning
Scheduling
Resource Allocation
Final Inspector qualification requirements
On-The-Job Training
Database for recordkeeping
5 CORE TRAINING
1. Indoctrination
2. Certification
SI 8900-1.3 ITS
QUALIFIED REF. 4. Personnel Licensing
NON 5. Investigation
CASR
AVIATION
61, 63, 65
4 ADDITIONAL TRAINING
6. Job Skills
7. Dispatcher (OPS) &
Avionic (AIR)
8. Cabin Safety
9. Management
Inspector Qualifications
All Inspectors
General Requirements for New Hires
See ICAO Doc. 8335, Part I, Par. 6.2.3
JOB PERFORMANCE
QUALIFICATION
AUTHORIZATION
DAAO
Formal Course Inspector Positif
OJT Level 3 Credentials
Safety
Culture
520 Possible
Job Functions
OPS 250 / AIR 270
Structured OJT on
Assigned Tasks
• On-the-Job-Training is required by ICAO and is discussed in ICAO document 9734, Safety Oversight
Manual.
OJT = is presented in three stages or Levels of learning.
Level I Level II is a further
understanding of the function
Knowledge
and conduct of the task,
including observation or
Trainee: Study assistance of the task being
Instructor: Discuss performed.
Level II
Understanding
Level I is basic background
and knowledge of a task. Trainee: observe/Assist
Instructor: Demonstrate
Level III
Level III is where trainees
Performance
actually perform the task
themselves. Trainee: Perform
Instructor: Evaluate
These three levels provide for a progressive and structured field training experience for anyone who has a
need to learn a new job task.
3. ITS Formal Course Standards
Describes the minimum requirements for all formal
classroom style inspector training courses.
4. ITS Job Task Analysis
• Detailed description of each inspector job
function.
• Separate JTAs for OPS and AIR.
• Includes supporting documentation and step-by-
step subtasks.
5. ITS Training Record - Software
OJT /
Job Tasks
ITS Training Record
Worksheet or Software!
DIREKTORAT KELAIKUDARAAN DAN
PENGOPERASIAN PESAWAT UDARA