The Past, Present and

Future of Quality
Culture
Susan Schniepp,
Distinguished Fellow
Regulatory Compliance Associates

November 11, 2021

1
REDICA © 2021. ALL RIGHTS RESERVED.
Topics for Discussion
The Past:
Quality Metrics Journey

The Present:
Regulations
Potential Models for Measuring Quality

The Future:
Data Integrity & Quality Culture
Expectations of Regulators

2
REDICA © 2021. ALL RIGHTS RESERVED.
The Past
“Progress, far from consisting in change,
depends on retentiveness. Those who cannot
remember the past are condemned to repeat it.”
George Santayana (1863—1952)

2016 © Regulatory Compliance Associates Inc.

3
REDICA © 2021. ALL RIGHTS RESERVED. 3
In the Beginning….

A maximally efficient, agile, flexible,

pharmaceutical manufacturing sector
that reliably produces high quality drug
products without extensive regulatory
oversight.

Janet Woodcock, FDA, Center for Drug

Evaluation and Research October 5,2005

4
REDICA © 2021. ALL RIGHTS RESERVED.
 Drug Shortages to Quality Metrics to Quality Culture

• FDAISA (7/12)
– Title VII, Section 70
• risk based inspection
• Title VII, Section 70
– allows for records to be requested in advance or in lieu of
inspection
– Title X, Section 506C-
• FDA annual drug shortage report to Congres
• Federal Register 12 Feb 201
– Assist in drafting strategic plan on drug shortages

5
REDICA © 2021. ALL RIGHTS RESERVED.
s

Quality Metrics Journey

PDA Culture PDA Q Culture

Metrics Survey Assessment Pilot
PDA Metric
Conf #1

PDA Metric
Industry White Conf #4
Papers PDA Metric PDA Metric
Published Conf #2 Conf #3

ISPE Pilot ISPE Pilot ISPE Pilot

Initial Industry Announced
Responses Results Results
Wave 1 Wave 2

Feb Feb Feb

Nov
Feb 2013 May Aug Nov May Aug Nov May Aug Nov May Aug Feb May
2014 2015 2016 Nov
2017

FDA Draft FDA Technical

Brookings “Metrics of Guidance FDA Draft
FR Notice Stakeholder Conformance
Requesting Potential (rev 1) Guidance
Meeting Interest” Guidance (rev 2)
Metrics to
Prevent Drug
Shortages

6
REDICA © 2021. ALL RIGHTS RESERVED.
The Present…

Developing a Standard

7
REDICA © 2021. ALL RIGHTS RESERVED.
 Is there a set of Mature Quality Attributes that are a
surrogate for
Quality Culture Behaviors?

1. Is there a relationship between Quality Culture

Behavior scores and Mature Quality Attribute
scores?
2. Which Mature Quality Attributes relates to
Quality Culture behavior?

REDICA © 2021. ALL RIGHTS RESERVED.

The Culture Equation

Quality Attribute Quality Behavior Quality Cultur

Quantifiable and Difficult to measure and Defined by behaviors
easily measured may rely on observation and beliefs

Quality Attributes Quality Culture

REDICA © 2021. ALL RIGHTS RESERVED. 9

e

 • Higher MQA score the

higher the behavioral
score

• Given this is a Social

Science Analysis, this
is a strong relationship

PDA Quality Culture Survey analysis

http://journal.pda.org/content/69/5/631.full.pdf+html?sid=1d68365b-c441-4c68-943f-eb0f39ce084e

Yes! There is a relationship between Quality Culture Behavior

and Mature Quality Attribute

REDICA © 2021. ALL RIGHTS RESERVED.

Higher Quality Maturity is correlated with Higher Quality Behavior
St.Gallen confirms PDA‘s Quality Culture Survey outcome

PDA Survey Analysis 2014 St.Gallen Analysis 2017

Behaviors

Mature Quality Attribute

R2 = 0.34 R2 = 0.66

▪ 326 pharmaceutical sites of different size and focus within St.Gallen database confirm PDA

Dr. Thomas Friedli, Measuring Quality Systems & Quality Culture February 2017

REDICA © 2021. ALL RIGHTS RESERVED.

Most Impactful Attributes Identified
Top 10 Quality Attributes significant impact
1. Participation at conferences to stay current 11. Program to show how employee’s
2. Collecting Error Prevention Metrics specific goals contribute to overall
3. Management Communication that Quality is quality goals
Everyone’s Responsibility 12. Program to measure, share and discuss
4. Utilization of new proven technologies product quality performance and
5. Clear performance criteria for feedback and improvement from shop floor to
coaching executive management.
6. EH&S Environmental Program with trained staff 13. Continuous Improvement Program /
(risk assessments, emission controls, spill Plans with active support of CEO and
prevention and response) Corp Management of QMS
7. Site has formal quality improvement objectives and 14. Program that establishes quality
targets system maturity model and action plan
8. Quality topics included in at least half of “all hands” and tracking to measure progress
meetings
9. Collecting Management Review Metrics 15. Internal survey measuring a company/
10. Collecting Employee Turn Over Rate Metrics site quality culture

Voted by ~225 Conference Participants, Dec 2014

REDICA © 2021. ALL RIGHTS RESERVED.

The Attributes

REDICA © 2021. ALL RIGHTS RESERVED.

The ANSI Standard Development Process

14
REDICA © 2021. ALL RIGHTS RESERVED.
 The Proposal
Title of Proposed Standard:
Quality Culture Assessment Tool
Stakeholder Category being represented:

! General Interest
! Producer Interest members
! User Interest members
! Regulatory Interest members

Project Description:

The U.S. FDA continues to focus on the use of quality metrics to modernize pharmaceutical quality systems and advance
innovation. In a recently published document, FDA has called for routine assessment and management oversight of quality
culture. In addition, MHRA, PIC/S and the WHO have all issued guidance on data integrity that specifically call for
companies to address the issue of quality culture. There is currently no agreed-upon standardized way for companies to
effectively measure their quality culture. PDA has already designed a comprehensive Quality Culture Assessment Tool and
Training, designed to guide companies to a better understanding of quality culture, how to assess it, and what actions to
take to improve it. The tool helps a company effectively collect verifiable data that will help them to assess their culture at
all levels of their organization. The tool allows the company to facilitate positive culture changes and continuous
improvement within their organization. This tool will serve as the basis for the development of a consensus standard to
guide quality assessment and facilitate benchmarking both within and across organizations.

15
REDICA © 2021. ALL RIGHTS RESERVED.
Project Details

Project Need:
Provide a data driven assessment approach to allow companies to
effectively measure quality culture and its importance in providing
high quality medicinal products to patients.

Project Intent:
This proposed American National Standard (ANS) is intended to:

• address the challenges associated with establishing and maintaining a quality culture that
meets the expectations of the regulators;
• Provide a reasonable and effective way to collect and evaluate quality culture;
• Provide a way for companies to benchmark their culture through data gathered through the
use of the tool.

16
REDICA © 2021. ALL RIGHTS RESERVED.
Project Details (cont.’)

Scope, Summary or Abstract of Project:

The U.S. FDA continues to focus on the use of quality metrics to modernize pharmaceutical quality
systems and advance innovation. In a recently published document, FDA has called for routine
assessment and management oversight of quality culture. In addition, MHRA, PIC/S and the WHO have all
issued guidance on data integrity that specifically call for companies to address the issue of quality
culture.
PDA has designed a comprehensive Quality Culture Assessment Tool and Training, designed to guide
companies
to a better understanding of quality culture, how to assess it, and what actions to take to improve it.
The tool helps a company effectively collect verifiable data that will help them to assess their culture at all
levels of their organization. The tool allows the company to facilitate positive culture changes and
continuous improvement within their organization.
The goal is to ensure that a quality mindset and behaviors are embedded into the daily work of the
individuals involved in all functions to ultimately ensure the delivery of high-quality product to patients.
The current tool enables companies to collect vital feedback in the following key areas:
• Employee Ownership and Engagement
• Continuous Improvement
• Technical Excellence
• Leadership Commitment
• Communication and Collaboration

17
REDICA © 2021. ALL RIGHTS RESERVED.
ANSI Standards Action - October 11, 2019 - Page 17 of 73 pages

PDA (Parenteral Drug Association)

Contact: Christine Alston-Roberts, (301)-656-5900-, [email protected] Bethesda Towers, 4350 East-West
Highway, Bethesda, MD 20814
New Standard
BSR/PDA Standard 06-201x, Quality Culture Assessment Tool (new standard)
Stakeholders: Quality assurance, quality control, quality engineering, operations, production,
manufacturing, general interest, regulatory interest members.
Project Need: Provide a data-driven assessment approach to allow companies to effectively measure
quality culture and its importance in providing high-quality medicinal products to patients.
A comprehensive Quality Culture Assessment Tool and Training, designed to guide companies to a better
understanding of quality culture, how to assess it, and what actions to take to improve it. The tool helps a
company effectively collect verifiable data that will help them to assess their culture at all levels of their
organization. The tool allows the company to facilitate positive culture changes and continuous
improvement within their organization.

REDICA © 2021. ALL RIGHTS RESERVED.

 PDA® Standards Development: Call for Volunteers:
Standard 06-201x, Quality Culture Assessment Tool (new standard).

PDA is very pleased to announce the launch of the Parenteral Drug Association’s sixth standard! We are seeking volunteer participants to
assist in developing, writing, and fine tuning the following proposal:

Standard 06-201x, Quality Culture Assessment Tool (new standard).

A comprehensive Quality Culture Assessment Tool and Training, designed to guide companies to a better understanding of quality culture,
how to assess it, and what actions to take to improve it. The tool helps a company effectively collect verifiable data that will help them to
assess their culture at all levels of their organization. The tool allows the company to facilitate positive culture changes and continuous
improvement within their organization.

This proposed American National Standard (ANS) was presented by Susan Schniepp, Distinguished Fellow with Regulatory Compliance
Associates.
Those individuals involved in Quality Assurance, Quality Control, Quality Engineering, Operations, Production, and Manufacturing,
Regulatory, and General Interest are needed.

Nominations/Volunteers to serve as a member of the technical team (consensus body) must have some subject matter expertise, and
willing to help write/contribute to this standard. Applicants should apply by contacting the PDA Standards Manager at [email protected].
The deadline to submit notification of interest in serving on the consensus body is January 14, 2020.

REDICA © 2021. ALL RIGHTS RESERVED.

REDICA © 2021. ALL RIGHTS RESERVED.
Why is Culture Important

REDICA © 2021. ALL RIGHTS RESERVED.

 Data Integrity and Quality Culture: Regulatory Expectations

PIC/S: Good Practices for Data Management and Integrity in

Regulated GMP/GDP Environments

• 6.3 Quality Culture

• Management should aim to create a work environment (i.e. quality culture) that is
transparent and open, one in which personnel are encouraged to freely communicate
failures and mistakes. Organisational reporting structure should permit the
information flow between personnel at all levels.

22
REDICA © 2021. ALL RIGHTS RESERVED.
 Data Integrity and Quality Culture: Regulatory Expectations (cont.’)

MHRA ‘GXP’ Data Integrity Guidance and Definitions

3. The principles of data integrity

3.1 The organisation needs to take responsibility for the systems used and the data they generate. The
organisational culture should ensure data is complete, consistent and accurate in all its forms, i.e. paper
and electronic.

3.3 The impact of organisational culture, the behaviour driven by performance indicators, objectives
and senior management behaviour on the success of data governance measures should not be
underestimated. The data governance policy (or equivalent) should be endorsed at the highest levels of the
organisation.

23
REDICA © 2021. ALL RIGHTS RESERVED.
 Data Integrity and Quality Culture: Regulatory Expectations (cont.’)

World Health Organization: Guidance on good data and record

management practices

1. Introduction
1.4 Examples of controls that may require development and strengthening to ensure good
data management strategies include, but are not limited to:

• adoption of a quality culture within the company that encourages personnel to be

transparent about failures so that management has an accurate understanding of risks
and can then provide the necessary resources to achieve expectations and meet data
quality standards:

24
REDICA © 2021. ALL RIGHTS RESERVED.
 Data Integrity and Quality Culture: Regulatory Expectations (cont.’)

World Health Organization: Guidance on good data and record management

practices

4. Principles
4.7 Quality culture. Management, with the support of the quality unit, should establish and maintain a
working environment that minimizes the risk of non-compliant records and erroneous records and data. An
essential element of the quality culture is the transparent and open reporting of deviations, errors,
omissions and aberrant results at all levels of the organization, irrespective of hierarchy. Steps should be
taken to prevent, and to detect and correct weaknesses in systems and procedures that may lead to data
errors so as to continually improve the robustness of scientific decision-making within the organization.
Senior management should actively discourage any management practices that might reasonably be
expected to inhibit the active and complete reporting of such issues, for example, hierarchical constraints
and blame cultures.

25
REDICA © 2021. ALL RIGHTS RESERVED.
 Data Integrity and Quality Culture: Regulatory Expectations (cont.’)

Data Integrity and Compliance With Drug CGMP

Questions and Answers
Guidance for Industry

Meaningful and effective strategies should consider the design, operation, and monitoring of
systems and controls based on risk to patient, process, and product. Management’s
involvement in and influence on these strategies is essential in preventing and correcting
conditions that can lead to data integrity problems. It is the role of management with executive
responsibility to create a quality culture where employees understand that data integrity is an
organizational core value and employees are encouraged to identify and promptly report data
integrity issues. In the absence of management support of a quality culture, quality systems
can break down and lead to CGMP noncompliance.

26
REDICA © 2021. ALL RIGHTS RESERVED.
 Available Information
MHRA GMP Data Integrity Definitions and Guidance for Industry (March 2015)

MHRA: A GxP Data Integrity Definitions and Guidance for Industry Draft version for consultation (July 2016)

FDA Data Integrity and Compliance With CGMP Guidance for Industry Draft Guidance (April 2016)

WHO: Annex 5, Guidance on good data and record management practices (June 2016)

Draft PIC/S Guidance: Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments (August 2016)

Parenteral Drug Association (PDA): Elements of a Code of Conduct for Data Integrity
(2015 – Free on PDA Website)

FDA: Data Integrity and Compliance with Drug CGMP: Questions and Answers, Guidance for Industry (December 2018)

2016 © Regulatory Compliance Associates Inc.

27
REDICA © 2021. ALL RIGHTS RESERVED. 12
 Over 100 regulators from MHRA and USFDA have been
trained on PDA’s Culture Assessment Tool

28
REDICA © 2021. ALL RIGHTS RESERVED.
Regulator Responses

“[We] can apply the Quality Culture Attributes to improve how we assess firms in non-
compliance context”
“As industry becomes aware and comfortable with this tool… it can be a powerful tool for
evaluating CMOs and business partners.”
“Industry can use the PDA tool as part of internal/self-assessment for improvement of
Quality Systems.”
“Industry could use the PDA tool to be more open with regulators on quality culture.”
“I will consider quality culture when reviewing data from industry.”
“Nice to differentiate quality Culture from Quality Systems and emphasize the importance of
what we make relative to what we do.”
“This course does help identify quality culture issues in a company. This may help [us] to
evaluate the quality of a pharmaceutical company.”

REDICA © 2021. ALL RIGHTS RESERVED.

 4.3.3. New Inspection Protocol Project
Inspections have traditionally focused on facility compliance with current good
manufacturing practices (CGMPs), with particular attention to process
deviations and system failures. Such inspections have not been a reliable
predictor of the state of product quality. Consequently, FDA is working to
develop a new inspection and reporting paradigm to better assess and record
the state of quality in manufacturing facilities; this new paradigm is embodied
within an initiative referred to as the new inspection protocol project (NIPP).
NIPP is expected to provide a more quality-focused, semi- quantitative
approach with streamlined and structured inspection reports. The NIPP
protocols utilize expert investigator questions and assessment approaches.
NIPP is expected to increase the quality focus of investigator assessments,
so that facilities and behaviors found to exceed basic compliance can be
recognized as such. Following successful piloting, NIPP-developed protocols
will be incorporated into new mobile technology to capture investigator findings
and assessments and better support investigators while traveling and during
facility inspections.

White Paper: FDA Pharmaceutical Quality Oversight - One Quality Voice

REDICA © 2021. ALL RIGHTS RESERVED.
REDICA © 2021. ALL RIGHTS RESERVED.
REDICA © 2021. ALL RIGHTS RESERVED.
 AveXis takes the feedback contained in the 483 seriously and is
committed to comprehensively addressing the noted observations. The
company's goal moving forward is to ensure a robust culture of
quality and sustainable GxP compliance across AveXis. In light of the
data integrity issues mentioned above, and the 483, AveXis, with
significant input and oversight from the Novartis Group of companies
(Novartis), has developed-and is in the process of implementing-a
company-wide Compliance Action Plan (the CAP). As detailed in
Section II (pp. 10-20) of the enclosed response, the core elements of
the CAP are (A) the Quality Integration Plan and (B) the Data
Integrity Remediation Plan.

REDICA © 2021. ALL RIGHTS RESERVED.

Final Thought:
“Quality is always fashionable.”

Boy George (George Alan O’Dowd), Lead Singer of Culture Club

1961 - present
34
REDICA © 2021. ALL RIGHTS RESERVED.
 35
REDICA © 2021. ALL RIGHTS RESERVED.