Software Quality Assurance
B4M36ZKS + BE4M36ZKS
1
Quality
Assurance
2
Quality
/ˈkwɒləti/
noun
1. [uncountable, countable] the standard of
something when it is compared to other things like
it; how good or bad something is
2. [uncountable] a high standard
3. [countable] a thing that is part of a person’s
character, especially something good
3
Quality Assurance
5
Quality Assurance (QA) is a systematic method that
guarantees products or services meet special
requirements and requirements, aiming for customer
satisfaction. It involves the utility of methodologies,
processes, and standards throughout the improvement
lifecycle.
6
The center standards of QA offer a basis for building a
strong quality management machine.
QA is a crucial thing within the lifecycle of any services
or products, encompassing a �xed of methodologies,
tactics, and sports designed to save you defects and
enhance average �rst-class.
7
QA includes the established order of nicely-de�ned
processes and requirements at every level of
development or manufacturing. This proactive
technique facilitates corporations become aware of
and cope with ability troubles early on, in the end
leading to higher customer delight and more sturdy,
dependable products.
8
Importance of QA
10
The signi�cance of QA extends across various
industries, ranging from manufacturing to software
program improvement, healthcare to production.
Regardless of the area, QA serves as a foundation for
delivering services and products that meet or exceed
expectations.
11
Why QA is crucial in various industries:
1. Customer Satisfaction: QA guarantees that the
�nal service or product aligns with client desires
and expectancies.
2. Cost Reduction: Identifying and rectifying defects
early inside the process prevents costly transform,
decreasing overall production expenses.
3. Brand Reputation: Consistently delivering
excellent merchandise enhances a brand’s
popularity, building consider amongst consumers
and stakeholders.
12
4. Regulatory Compliance: Many industries have
strict regulations, and QA facilitates businesses
adhere to those requirements, averting criminal
headaches and making sure ethical practices.
5. Risk Mitigation: QA tactics become aware of
capability risks and vulnerabilities, permitting
agencies to proactively deal with problems earlier
than they enhance.
13
Examples
15
In software development
• Testing software to ensure it is free of bugs and
defects.
• Writing clear and concise documentation that
users can understand.
• Conducting user testing to ensure that the
software is user-friendly.
16
In manufacturing
• Inspecting products to ensure they meet quality
standards.
• Testing materials to ensure they are safe and
durable.
• Maintaining a clean and organized production
environment.
17
In healthcare
• Administering medications and treatments
correctly.
• Monitoring patients for signs of complications.
• Maintaining a clean and sterile environment.
18
In education
• Developing a curriculum that meets educational
standards.
• Providing students with access to quali�ed
teachers and resources.
• Assessing student learning to ensure they are
meeting expectations.
19
Principles
21
QA principles are fundamental guidelines and best
practices used to ensure the quality and reliability of
software products and services.
QA principles emphasize proactive defect prevention,
continuous improvement, customer satisfaction, and
adherence to quality standards throughout product/
service lifecycle.
22
In the �eld of software development principles
included are for example risk-based testing, test-driven
development, shift-left testing, and automation-�rst
approach to achieve higher quality outcomes.
23
Procedures in QA
25
The quality assurance process can be complicated, and
the list of steps can be very long. It can be integrated it
with the Plan-Do-Check-Act (PDCA) model for
simplicity. This model is a common tool used for the
management of continuous process improvement.
26
1. Plan
In this �rst crucial stage, a quality assurance technician
or manager will determine clear-cut goals to produce
high-quality products and suggest suitable processes to
execute those objectives. At this stage, the business
can predict any potential problems.
27
2. Do
As the name suggests, this stage allows the
implementation of the processes identi�ed in the
previous phase. The organization carries out its quality
plan, which includes establishing procedures, training
staff, and implementing quality controls.
28
3. Check
In Stage 3, the results of the tests are checked and
compared to what was expected. This helps to see if the
products meet the required standards. If they do, then
the experts move to the �nal stage. But if they don't,
they go back to the �rst stage to make necessary
improvements.
29
4. Act/Adjust
In this �nal stage, the organization takes action to
improve the quality plan based on the results of the
previous stage. This involves making changes to the
quality plan, implementing new procedures, and
continuing to monitor the quality results.
30
Process capability
32
Process capability is de�ned as a statistical measure of
the inherent process variability of a given
characteristic. You can use a process-capability study
to assess the ability of a process to meet speci�cations.
33
Process Capability Analysis
A process capability study uses data from an initial run
of parts to predict whether a manufacturing process
can repeatedly produce parts that meet speci�cations.
Can I rely on this process to deliver good
parts?
34
35
Process stability
36
37
Capability Maturity Model
Integration
39
The Capability Maturity Model Integration (CMMI)
helps organizations streamline process improvement,
encouraging a productive, ef�cient culture that
decreases risks in software, product, and service
development.
40
The CMMI was developed by the Software Engineering
Institute at Carnegie Mellon University as a process
improvement tool for projects, divisions, or
organizations.
41
Improvement efforts require a model of how your
organization works, which functions they need, and
how those functions interact. A model gives you an
understanding of organizational elements and assists in
discussions of how and what can and should be
improved.
42
A model offers the following bene�ts:
• Provides a common framework and language to
help communicate
• Leverages years of experience
• Helps users consider the large picture while
focusing on improvement
• Is often supported by trainers and consultants
• Can help solve disagreements by providing
agreed-upon standards
43
Maturity levels
44
Software Quality
46
Software quality is the "capability of a
software product to conform to
requirements." (ISO)
47
Software quality describes the desirable
attributes of software products
(American Society for Quality, ASQ)
48
Software quality is de�ned as a �eld of study and
practice that describes the desirable attributes of
software products.
49
Software's functional quality
How well the software complies with or conforms to a
given design, based on functional requirements or
speci�cations. The attribute can also be described as
the �tness for the purpose of a piece of software or
how it compares to competitors in the marketplace as a
worthwhile product. It is the degree to which
the correct software was produced.
50
Software structural quality
How the software meets non-functional requirements
that support the delivery of the functional
requirements, such as robustness or maintainability. It
has a lot more to do with the degree to which the
software works as needed.
51
Motivation for Software quality:
• Risk Management - impact of failure can range
from inconveniences to human fatalities.
• Cost management - as in any other �elds of
engineering, a software product or service
governed by good software quality costs less to
maintain, is easier to understand and can change
more cost-effective in response to pressing
business needs.
52
Software quality also often gets mixed-up with Quality
Assurance or Problem Resolution Management or
Quality Control or DevOps. It does over-lap with those
areas but is distinctive as it does not solely focus on
testing but also on processes, management,
improvements, assessments, etc.
53
When quality is sidetracked ...
55
The UK Post Of�ce has been using software called
Horizon for 20 years. It had bugs that caused it to
report that accounts under the employees’ control
were missing money. It looked like an employee stole
thousands. As a result 736 post of�ce operators were
convicted. People lost jobs, families, and one woman
was sent to prison while pregnant. One man committed
suicide after the system showed his account was
missing £100,000.
56
In 2020, three �ight loads were miscalculated. TUI
check-in software treated travelers identi�ed as
“Miss” as children. As the passengers’ weight is used to
estimate thrust during the take off, it led to an
unfortunate miscalculation. Children are counted as
35kg and adults as 69kg. Lower calculated weight
means lower thrust during take off. With an
unfavorable passenger list, such a case can lead to a
disaster. Fortunately, the �nal thrust value was within
the safety limit, and everyone traveled without issues.
57
Toyota had to recall more than 8 million cars due to
software errors. Some vehicles were accelerating, even
when the gas pedal was not touched. Investigation
showed that systems were badly designed, and had
poor quality and had various software bugs, including
memory corruption, buffer over�ow, unsafe casting,
race conditions, and others. Toyota claimed �rst that
the problem was caused by �oor mats. They got �ned
$1.2 Billion for concealing safety defects. The most
important acceleration related piece of code appeared
to have huge cyclomatic complexity, in practice making
it untestable.
58
• Reboot Your Boeing 787 Every 248 Days
• Open Of�ce Won't Print on Tuesdays
• Heathrow Terminal 5 Opening
• Crowdstrike and Blue Screen of Death
59
Software Quality Lifecycle
61
Quality from an end-user viewpoint when they are
actually using the software in real life and not in a lab
(context of use).
62
Internal quality has an in�uence on, but not a direct
correlation with, external quality, and that external
quality depends on internal quality.
You can have great code quality (internal quality) and
still have poor external quality (software behavior in
the test lab).
This makes sense in reverse too. The software might
work okay, but the internal quality could be terrible.
63
According to American Society for Quality (ASQ) there
are two main approaches to software quality defect
management and quality attributes.
64
Defect Management
66
A software defect can be regarded as any failure to
address end-user requirements. Common defects
include missed or misunderstood requirements and
errors in design, functional logic, data relationships,
process timing, validity checking, and coding errors.
67
68
Defect management (DM) is the process of identifying,
documenting, and tracking defects (bugs or issues) in a
software product. It is an important part of the
software development process that ensures defects
are identi�ed and addressed in a timely manner.
69
Less buggy software will be readily available in the
market if Defect management is conducted more
effectively and with full attention. Most businesses use
a DM process that includes defect discovery, removal,
and improvement.
70
The primary objectives of adopting DM for various
projects or organizations are:
• It offers operational help for �xing and retesting
discovered �aws.
• It provides information for a defect status and
progress report.
• It gives suggestions for guidance on defect release.
• It determines the primary cause of the condition
and suggests �xes.
71
More mature software development organizations use
tools, such as defect leakage matrices (for counting the
numbers of defects that pass through development
phases prior to detection) and control charts, to
measure and improve development process capability.
72
Why should you consider
Defect management?
A reductive process, defect data management requires
as much input as feasible. All stakeholders have
important fault input that needs documentation. It
functions best when everyone noti�es the system of
any uncovered �aws.
73
DM helps in several ways
• Removing false positives brought on by test
environment anomalies, test code or data
problems, or test process �aws.
• Deleting duplicate entries when a bug produces
results that appear to testers to be a collection of
unrelated issues.
• Removing items from the list won't deter people
from reporting defects on a timely basis.
74
• The process of managing defects encourages a
culture of continuous improvement.
• Defects and their resolutions are documented.
This provides a valuable knowledge base for future
projects.
75
Defect Lifecycle
76
1. Tester �nds the defect
2. Status assigned to defect - New
3. A defect is forwarded to Project Manager (PM) for
analyze
4. PM decides whether a defect is valid
5. Defect is not valid - a status is given “Rejected”
6. So, PM assigns a status rejected. If the defect is
not rejected then the next step is to check whether
it is in scope. Suppose we have another function-
email functionality for the same application, and
you �nd a problem with that. But it is not a part of
the current release when such defects are
assigned as a postponed or deferred status. 77
7. PM checks for similar defects raised earlier.
8. If no the defect is assigned to the developer who
starts �xing the code. During this stage, the defect
is assigned a status in-progress.
9. When the code is �xed a defect is assigned a status
�xed.
10. Next, the tester will re-test the code. In case, the
test case passes the defect is closed.
11. If the test cases fail again, the defect is re-opened
and assigned back to the developer.
78
Bene�ts of Defect
management
Defect management process optimizes the
organization's work�ows. Software tools involve the
detection or tracking of non-technical issues.
79
• Existence of defect tracking tools: Defect tracking
is one of the critical steps in the Defect
management process. Many defect tracking tools
are available to track �aws like Jira, Trello, Asana,
Hive, etc.
80
• Verify resolution: Defect management process
also aids in determining whether or not all issues
that are discovered or tracked have been
addressed. Simply say, it enables us to ensure the
recti�cation of tracked issues.
81
• Provide useful metrics: the DM process offers
defect metrics and automation tools. These
metrics are helpful for reporting and development.
82
Limitations
• When not handled correctly, there will be a
signi�cant cost increase over time.
• If faults or defects are not handled correctly at an
early stage, they may later cause more harm and
increase the cost of repair.
• DM can be a time-consuming process, especially if
there are a large number of defects to track,
prioritize, and resolve. This can potentially slow
down the development and release process.
83
• If the DM process does not execute correctly,
further drawbacks such as loss of revenue,
customers, etc., can occur.
84
Control charts
86
Walter A. Shewhart invented control charts while was
working for Bell Labs in the ’20s, and they have been
used in a variety of industries as part of a process
improvement methodology. Shewhart understood that,
no matter how well a process was designed, there will
always be variation within that process—and the
validation can have a negative effect if it keeps you
from meeting deadlines or quotas.
87
Control charts (Shewhart Charts or Statistical Process
Control Charts) are tools used to determine if a process
is in a state of statistical control, or how much variation
exists in a process.
88
89
• Points representing a statistic (e.g., a mean, range,
proportion) of measurements of a quality
characteristic in samples taken from the process
• A center line is drawn at the value of the mean or
median of the statistic
• The standard deviation of the statistic is calculated
using all the samples.
• Upper and lower control limits indicating the
threshold at which the process output is
considered statistically 'unlikely', typically drawn
at 3 standard deviations from the center line
90
91
As long as all of the points plotted on the chart are
within the control limits, the process is considered to
be in statistical control. There is no urgent need for
change. Improvements can be made, but operating
within the control limits is an admirable goal.
The points that fall outside of control limits indicate the
times that the process was out of control. If these out-
of-control points happen rarely, an analysis must be
conducted to �nd out what went wrong and to plan for
�xing them in the future. If the process hits out-of-
control points often, this could indicate a pattern that
needs to addressed.
92
How can a Control Chart be used?
• Analyze the team's past performance in a
retrospective,
• Measure the effect of a process change on the
team's productivity,
• Provide external stakeholders with visibility of the
team's performance, and
• For Kanban, use past performance to set targets
for the team.
93
Quality attributes
95
Software quality is not a single dimension, but a
complex and multifaceted concept that can be
measured by different attributes - for example
functionality, reliability, usability, ef�ciency,
maintainability, and portability.
Each attribute can be de�ned by speci�c criteria and
metrics that can be used to evaluate the software
product.
96
Non-functional requirements (NFRs) focus on the
quality aspects of the system. They don’t describe what
the system does but rather how well it performs its
intended functions.
97
98
The system must be driven by a set of quality goals. For
the system to be successful, it also must work within
restricted performance, availability, security, reliability,
and supportability.
A quality attribute is a property of a software system
that determines its usefulness and success from the
client and business standpoint. A quality attribute acts
as the driver for a software system.
99
Making the decision process more analytical and
predictable is challenging and costly. Quality attributes
are one way to help overcome this challenge.
They help to de�ne the system goals and as a starting
point drive system architecture in an informed manner
leading to smaller costs in the long run.
100
A single quality attribute ideally should be represented
by a discrete and measurable value. The measurement
tests should be automated and effortless to run. It is
rarely possible to have a single quality attribute
represented by a single value. In most cases, it is
composed of multiple values, and its complexity
depends on the business needs.
Introducing quality attributes gives the possibility to
target the quality attribute from the starting point. It
allows architects to focus on the goals that a given
system should aim for.
101
Software Quality Attributes (QA) classi�cation is
de�ned by the standard ISO/IEC 25010. There is also
an extensive list of QA at Wikipedia.
102
Non-Functional Testing
How well the system performs in certain
area is non-functional testing.
Non-functional testing is a type of software testing
which refers to various aspects of the software such as
performance, load, stress, scalability, security,
compatibility, etc.
103
Examples of non-functional testing
• Performance Testing
• Volume Testing
• Scalability
• Usability Testing
• Load Testing
• Stress Testing
• Compliance Testing
• Portability Testing
• Disaster Recover Testing
104
Software Quality Models
106
Software quality models are a well-accepted means to
support quality management of software systems.
• taxonomic models like the ISO 9126 are used to
de�ne quality,
• metric-based models like the maintainability index
(MI) are used to asses quality,
• stochastic models like reliability growth models
(RGM) are used to predict quality
107
De�nition models
Definition models are used in various phases of a
software development process.
During requirements engineering, they define quality
attributes and requirements for planned software
systems and thus constitute a method to agree with the
customer what quality means.
108
During implementation, quality models serve as basis
of modelling and coding standards or guidelines. They
provide direct recommendations on system
implementation and thus constitute constructive
approaches to achieve high software quality. Defects
found during QA are classified using the quality model.
Apart from their use during software development,
definitional quality models can be used to
communicate software quality knowledge during
developer training.
109
Assessment models
Assessment models often extend quality definition
model usage scenarios to control compliance.
During requirement engineering, assessment models
can be used to objectively specify and control stated
quality requirements.
110
During implementation, the quality model is the basis
for all quality measurements, i.e. for measuring the
product, activities and the environment.
During quality audits, models serve as a basis of the
performed audit procedure - internal measures that
might influence external properties are monitored and
controlled.
Assessment models furthermore constitute the basis
for quality certifications.
111
Prediction models
Prediction models are used during project
management. More specifically, such models are used
for release planning and in order to provide answers to
the classical “when to stop testing” problem.
112
Boehm’s model
Introduced by Barry Boehm and his team at the US
Defence Department in 1978. It represents a
hierarchical quality model to de�ne software quality
using a prede�ned set of attributes and metrics. Each
one contributes to the overall quality of software.
System quality
Product quality characteristics
levels
Utility, Reliability, Ef�ciency, As-is utility
Human engineering, Testability, Maintainability
Maintainability, Portability General utility
113
McCall’s model
Developed in the 1970s by James McCall and his
colleagues at the US Air Force. Similar to the Boehm’s
model, it identi�es quality factors for software
products and quality perspectives for software
systems.
Product quality System quality
characteristics perspectives
Correctness, Reliability,
Product operation,
Ef�ciency, Integrity, Usability,
Revision of the
Maintainability, Flexibility,
product, Product
Testability, Portability,
transition
Reusability, Interoperability
114
ISO/IEC 25010
This is one of the most widely used and well known
software quality models. It de�nes eight quality
characteristics for software products and four quality
characteristics for software systems.
System quality
Product quality characteristics
characteristics
Functional suitability, Reliability, Adaptability
Performance ef�ciency, Installability
Usability, Security, Replaceability
Compatibility, Maintainability, Co-existence
Portability 115
FURPS
Developed at Hewlett-Packard �rst publicly elaborated
by Grady and Caswell.
FURPS+ is now widely used in the software industry.
The + was later added to the model after various
campaigns at HP to extend the acronym to emphasize
various attributes.
116
• Functionality - Capability, Security
• Usability (UX) - Human Factors, Aesthetics,
Consistency, Documentation, Responsiveness
• Reliability - Availability, Predictability, Accuracy
• Performance - Speed, Ef�ciency, Resource
Consumption, Throughput, Capacity, Scalability
• Supportability - Testability, Flexibility,
Installability, Localizability
117
MoSCoW
Must have, Should have, Could have and Won’t have
(MSCW) and was used originally within the DSDM
(Dynamic System Development Method) to consider
the priorities from a relatively simple perspective and
is now applied ubiquitously for system requirements
prioritization.
118
• Mo - Must have – Requirements that the release
of the product or system MUST include.
• S – Should have – Requirements that are high
priority and SHOULD if possible, be included
within the release.
• Co – Could have – Requirements that are
desirable or ‘nice-to-have’ if they COULD be
included without too much divergence or cost.
• W - Won't have (this time) – Requirements that
are wanted but WILL NOT be included within the
current release.
119
or combination ...
120
Challenges
122
Lack of Clear Requirements
Clear and speci�ed necessities are crucial for the QA
crew to expand effective test instances and make
certain that the stop product meets the consumer’s
expectations. When requirements are uncertain, it
could lead to misunderstandings, neglected defects,
and delays within the QA method.
Solution - exchange channels among stakeholders,
builders, and QA teams have to be robust, and there
have to be a manner for clarifying requirements as
early as possible.
123
Resource Constraints
Resource constraints (time, �nances, or personnel) can
restrict the effectiveness of QA procedures. In some
cases, QA groups may not have get entry to to the vital
tools or environments to behavior thorough trying out.
Solution - allocate enough assets for QA, prioritize
trying out efforts based totally on vital functionalities,
and spend money on automation gear to streamline
repetitive responsibilities.
124
Rapid Changes in Technology
New technologies, frameworks, and improvement
methodologies emerge regularly, making it hard for QA
experts to hold up.
Solution - being proactive by staying informed about
enterprise tendencies, participating in non-stop getting
to know programs, and incorporating adaptability into
the procedures. Collaboration with improvement
teams is essential to know-how and trying out the
ultra-modern technology successfully.
125
Balancing Speed and Quality
Requirement to deliver the products quickly can result
in a compromise in quality. Balancing speed and �rst-
class is a sensitive mission for QA groups. It is essential
to discover a center �oor that ensures thorough trying
out without in�icting undue delays in product release.
Solution - Implementing agile methodologies, adopting
automation, and emphasizing collaboration between
improvement and QA groups to strike the proper
balance.
126
Resources
• 6 ways to optimize development with a control
chart
• How to make a control chart
• 11 of the most costly software errors in history
• Bad software examples - how much can poor code
hurt you?
• Drawing Parallels Of Software Quality With Other
Fields
• Software quality models: Purposes, usage
scenarios and requirements
127
