Model Quality Objectives

Embedded software development with MATLAB/Simulink

Author: MQO Working Group | Version: 1.0 | Release date: 09-2018
Model Quality Objectives | Version 1.0

Contents
1 Introduction 4
1.1 Abstract 4
1.2 Intended audience 4
1.3 Scope 4
1.4 Purpose 4
1.5 Background and motivation 5
1.6 References 6
1.7 Terminology 7
1.8 Abbreviations 7
1.9 Template 8
1.10 Authors 8
2 Software development with design models 9
2.1 Overview 9
2.2 Software planning phase 10
2.2.1 Scope definition 10
2.2.2 Tools definition 10
2.2.3 Standards definition 11
2.2.4 MQR identification and allocation 11
2.2.5 Strategy to achieve MQO 11
2.2.6 MQR conformance demonstration 11
2.3 Software requirements phase 11
2.3.1 Roles of the functional model 11
2.3.2 Main characteristics of the functional model 12
2.4 Software architectural design phase 12
2.4.1 Role of the architecture model 12
2.4.2 Main characteristics of the architecture model 12
2.5 Software component design and testing phase 13
2.5.1 Role of the component design model 13
2.5.2 Main characteristics of the component design model 13
2.6 Software component implementation and testing phase 14
2.6.1 Role of the component implementation model 14
2.6.2 Characteristics of the component implementation model 14
2.7 Relationship between design models 15

2
Model Quality Objectives | Version 1.0

3 Design Models Quality 17

3.1 Overview 17
3.2 Model Quality Requirements 19
3.2.1 Model layout 19
3.2.2 Model comments 19
3.2.3 Model links to requirements 20
3.2.4 Model testing against requirements 20
3.2.5 Model compliance with modeling standard 21
3.2.6 Model data 21
3.2.7 Model size 22
3.2.8 Model complexity 22
3.2.9 Model coverage 23
3.2.10 Model robustness 23
3.2.11 Generated code testing against requirements 24
3.2.12 Generated code compliance with coding standard 24
3.2.13 Generated code coverage 25
3.2.14 Generated code robustness 25
3.2.15 Generated code execution time 26
3.2.16 Generated code memory footprint 26

3
Model Quality Objectives | Version 1.0

1 Introduction

1.1 Abstract
This document, named Model Quality Objectives (MQO), presents quality objectives for models developed with
Simulink® at different phases of the software development lifecycle. It has been defined by a group of leading actors
from the automotive industry and MathWorks, the company that develops the MATLAB®, Simulink, and Polyspace®
products. Its purpose is to clarify and ease the collaboration when sharing Simulink models, e.g. between OEM and
suppliers, in the context of embedded software development to drive the production of higher quality and integrity
software.

1.2 Intended Audience

The intended audience of this document is Simulink users and project/quality/safety managers interested in estab-
lishing a standard approach to assess the quality of design models used at different phases of an embedded software
development.

1.3 Scope
The use of design models developed with the Simulink® software and its toolboxes in the context of embedded soft-
ware development with Model-Based Design.

1.4 Purpose
This document clarifies how Simulink design models contribute to accelerate development and verification activities
from software requirements specification to software implementation. Four types of design models with specific
purposes are introduced, each with a specific quality objective to control their proper usage. Each quality objective is
a set of measurable metrics with quantified satisfaction criteria to facilitate and standardize model quality
assessment.
The organizations that apply the concepts presented in this paper should experience the following benefits:
a. Shared understanding of Model-Based Design within the organization
b. Application of a quality model adapted to Model-Based Design projects and compatible with industry software
quality and safety standards
c. Assessment of model quality at different phases of projects
The organizations that also collaborate with partners to execute Model-Based Design projects should experience the
following benefits when applying the concepts presented in this paper:
a. Easiest split of responsibility between parties at the beginning of projects
b. Common understanding of model quality
c. Common expectation on model quality when sharing models

4
Model Quality Objectives | Version 1.0

1.5 Background and Motivation

Design models developed with the Simulink software are widely used in the industry to accelerate the development
of embedded software. Those models enable engineers to accomplish various engineering tasks such as frequen-
cy-domain analysis, desktop simulation, formally-based verification, and automatic code generation. This develop-
ment process is known as Model-Based Design.
Design models can be developed at a very early stage to validate requirements and quickly explore design solutions.
Such models can also be incrementally refined until they reach a level of maturity that is sufficient to generate code
that complies with international software safety standards. To incrementally increase the maturity of the design
models, different engineering disciplines need to be involved such as system engineering, control engineering and
software engineering. Collaborating with the same language, tools, and models is a great way to improve communi-
cation between engineers and reduce the project cost and development time. However, with different disciplines
using design models at different project phases, confusion may arise about the contribution of models and what they
represent.
An incorrect interpretation of what the models represent can lead to an incorrect use of those models and ultimately
impact the quality of the software produced. OEM and tier-one suppliers that participate in the definition of MQO
have shared many concrete use cases when underspecified models or models with insufficient maturity have been
prematurely promoted as “ready for coding”. Consequently, higher development effort than planned, bugs, and diffi-
cult conversations related to responsibilities would then take place. In order to avoid this situation, this document
proposes to clarify the role of design models for the development of embedded software and standardize measurable
criteria to verify their quality.
This approach has been inspired by the Software Quality Objectives (SQO) [1] defined by a group of automotive
actors and MathWorks in 2010, at a time when most exchanges between car manufacturers and suppliers were based
on textual specification and manual code. This approach also aims to go one step further in the formalization of
model sharing, as defined by Bosch [2] in 2014, and in the implementation of techniques and measures proposed by
software safety standards such as ISO26262-6. [3]

5
Model Quality Objectives | Version 1.0

1.6 References

Ref Description

[1] Patrick Briand (Valeo), Martin Brochet (MathWorks), Thierry Cambois (PSA Peugeot
Citroën), Emmanuel Coutenceau (Valeo), Olivier Guetta (Renault SAS), Daniel
Mainberte (PSA Peugeot Citroën), Frederic Mondot (Renault SAS), Patrick Munier
(MathWorks), Loic Noury (MathWorks), Philippe Spozio (Renault SAS), Frederic
Retailleau (Delphi Diesel System), Software Quality Objectives for Source Code, ERTS
2010-Conference, 2010.
[2] S. Louvet, Robert Bosch (France) SAS, Dr. U. Niebling, Dr. M. Tanimou, Robert Bosch
GmbH Model Sharing to leverage customer cooperation in the ECU software devel-
opment; Toulouse, ERTS 2014-Conference, 2014.
[3] ISO 26262 International standard for functional safety of electrical and/or electronic
systems in production automobiles defined by the International Organization for
Standardization (ISO) in 2011.
[4] RTCA/Eurocae, Software Considerations in Airborne Systems and Equipment
Certification, RTCA DO-331 / Eurocae ED-218, December 13, 2011.

[5] Automotive SPICE Process Assessment / Reference Model from VDA QMC Working
Group 13 / Automotive SIG

[6] Railway applications - Communication, signalling and processing systems - Software

for railway control and protection systems, EN 50128:2011

[7] Hersteller Initiative Software (HIS)is an initiative from German automotive manufactur-
ers whose goal is the production of agreed standards within the area of standard
software modules for networks, development of process maturity, software test, soft-
ware tools and programming of ECU’s. HIS specifies a fundamental set of Software
Metrics to be used in the evaluation of software.
[8] Modeling guidelines for MATLAB, Simulink and Stateflow (MAAB) to develop control
algorithms and defined by the MathWorks Automotive Advisory Board.

[9] MISRA C - Set of software development guidelines for the C programming language
developed by MISRA (Motor Industry Software Reliability Association). Its aims are to
facilitate code safety, security, portability and reliability in the context of embedded
systems, specifically those systems programmed in ISO C / C90 / C99.

6
Model Quality Objectives | Version 1.0

1.7 Terminology

Term Definition

Design model A model developed with MATLAB, Simulink and Stateflow to design software
architecture and algorithms for signal processing, communication or control
software. In the context of MQO, four types of design models are defined: the
functional model, the architecture model, the component design model, and the
component implementation model.
Model higher level A requirement satisfied by a design model.
requirement

Model-Based Design A process that systematically relies on the use of models at different phases of the
system and software development process.

Model Quality Objective A quality objective that applies to a type of design model.

Model Quality A textual expression that specifies a non-functional requirement of a design

Requirement model.

1.8 Abbreviations

MAAB MathWorks Automotive Advisory Board

MBD Model-Based Design
MQO Model Quality Objective
MQR Model Quality Requirement

OEM Original Equipment Manufacturer

SQO Software Quality Objectives

7
Model Quality Objectives | Version 1.0

1.9 Template
The following template is used to specify MQR in section 3.2.

Requirement ID Requirement title

Description A description including a measurable satisfaction criterion on model, generated

code or executable generated code.

Recommendation Level of recommendation for each model quality objective:

level • Empty i.e. N/A
• Recommended i.e. Recommended for early verification
• Mandatory

MQO-1 MQO-2 MQO-3 MQO-4

XXX XXX XXX XXX

Notes Further information to clarify the requirement description.

References / References or examples of techniques to implement the requirement with
Examples of MATLAB/Simulink.
techniques
Rationale Justification for quality
Last update MQO Version when requirement was last updated

1.10 Authors
This document was prepared by the MQO working group composed of representatives from MathWorks, automotive
OEMs and suppliers.

Jérôme Bouquet Renault

Stéphane Faure Valeo
Florent Fève Valeo
Matthieu Foucault PSA
Ursula Garcia Bosch
François Guérin MathWorks
Thierry Hubert PSA
Florian Levy Renault
Stéphane Louvet Bosch
Patrick Munier MathWorks
Pierre-Nicolas Paton Delphi
Alain Spiewek Delphi
Yves Touzeau Renault

8
Model Quality Objectives | Version 1.0

2 Software Development with Design Models

2.1 Overview
This document defines a development approach based on four types of design models supporting the left-hand side
of the V-cycle.

Figure 1: Model-Based Design/MQO software lifecycle

The Model-Based Design/MQO software development lifecycle involves five specific phases marked as 1 to 5 in
Figure 1 Sections 3.1 to 3.5 will provide greater details on the phases.
Figure 2 shows how the Model-Based Design/MQO software development lifecycle maps to other software develop-
ment lifecycles from the industry. The phases supported by design models are highlighted with a dark background,
and Model-Based Design is referred to as MBD.

9
Model Quality Objectives | Version 1.0

Figure 2: Model-Based Design / MQO software phases versus other industry standards [3], [4], [5], [6]

2.2 Software Planning Phase

This section defines the planning activities that must be carried out to prepare the use of design models. This is rec-
ommended for the use of functional models and mandatory for the use of architecture, component design, and com-
ponent implementation models. Most of these concepts are already imposed by safety standards such as DO-331 [5].

2.2.1 Scope definition

All design models may not be applicable to all projects. For instance, the scope of Model-Based Design can be
reduced to the development of a single software component or only used to support the software architectural design
specification. The project shall define the software development phases that will be supported by design models.
Each design model shall be managed independently as a work product of the software development phase it belongs
to.

2.2.2 Tools definition

The tools that support the development and verification of design models shall be identified and classified at the
beginning of the project. Those tools shall be qualified, if required by the project.

10
Model Quality Objectives | Version 1.0

2.2.3 Standards definition

The modeling standard used to support the development of design models shall be defined prior to entering the soft-
ware architecture phase. The coding standard used to support the development of design models shall be defined
prior to entering the software component implementation phase, or ideally, prior to entering the software component
design phase.

2.2.4 MQR identification and allocation

The MBD quality requirements (MQR) defined in 3.2 shall be identified and agreed to by the project stakeholders at
the beginning of the project. Some MQR shall be adapted to the project requirements (e.g. model and code coverage
criteria). Each MQR shall be allocated to a project stakeholder.

2.2.5 Strategy to achieve MQO

Once the MQR has been defined for the project, a strategy shall be defined to achieve the objective. Such a strategy
can include intermediate steps corresponding to project milestones, specific training, or a tools migration process.
For instance, it is recommended to gradually increase the coverage criteria and not wait for the final version of the
software to perform most of the test development effort.

2.2.6 MQR conformance demonstration

The conformance with the project MQR shall be planned and demonstrated at the end of the project. The verifica-
tion of each MQR shall lead to the production of a report produced by the project stakeholder responsible of the
MQR. Sufficient justifications must be provided when MQR are not met (e.g. missing coverage should be justified).
The person in charge of assessing the compliance shall have the necessary skills to understand the justifications.

2.3 Software Requirements Phase

This section focuses on the functional model developed during the software requirement phase.

2.3.1 Roles of the functional model

The role of the functional model is to clarify and refine complex dynamic behaviors that need to be translated into
software requirements.
In most cases, the functional model and the software requirements are concurrently developed by the person in
charge of the software requirements. This functional model engineer supports the stabilization of the software
requirements (the “what”) while identifying good design solutions (the “how”) that could be further elaborated
during the design and implementation phases. The functional model is often referred to as an executable specifica-
tion because it provides a functional behavior that satisfies the functional requirements. However, the functional
model does not replace the software functional requirements. The functional model contributes to the validation
activities of the software requirements.

11
Model Quality Objectives | Version 1.0

2.3.2 Main characteristics of the functional model

The functional model focuses on the correctness of algorithms and equations. It does not have to consider design
constraints related to embedded software development. However, when developing the functional model, it should
anticipate the main characteristics of the hardware platform and their impact on the software requirements.
The functional model may not be needed if the software functional requirements are simple to implement, nor does
it have to be representative of all the software functional requirements. Figure 3 shows an example of a functional
model using continuous time and is limited to a small function of a larger software.

Figure 3: An example of functional model (Anti-Lock Braking system)

2.4 Software Architectural Design Phase

This section focuses on the architecture model developed during the software architectural design phase.

2.4.1 Role of the architecture model

The role of the architecture model is to contribute to the specification of the software architectural design.
Graphical notation is naturally well-suited to defining an organization of components, representing interfaces and
connections, and specifying component scheduling. For a complex architecture, it is not conceivable to develop such
a diagram without a proper modeling language and a computer-aided design tool such as Simulink.

2.4.2 Main characteristics of the architecture model

The architecture model fully specifies the static software architectural design (e.g. component models, interfaces)
and provides links/references to the component design models that will be built or are already built. The architecture
design model is associated with a data dictionary that defines the data and interfaces of the software and its
components.

12
Model Quality Objectives | Version 1.0

The architecture model directly contributes to the design activities and is therefore subject to conformance with
industry quality standards, safety standards, and/or architecture standards (e.g. traceability to requirements, com-
patibility with architecture standard).
Next figure shows an example of an architecture model that references component models represented by model
references.

Figure 4: Example of architecture model

2.5 Software Component Design and Testing Phase

This section focuses on the component design model developed during the software component design and testing
phase.

2.5.1 Role of the component design model

The role of the component design model is to provide a complete specification of the software component design and
support its verification with dynamic and static analysis.
The use of a high-level modeling and programming language enables better management of the complexity of algo-
rithms and reduces the probability of design errors. The support of simulation and static analysis contributes to
elimination of design errors.

2.5.2 Main characteristics of the component design model

The component design model fully specifies the algorithms and equations that will be part of the embedded soft-
ware and excludes any elements used for debugging or prototyping such as measurement points or override mecha-
nisms. Each component design model is associated with a data dictionary that defines its interface, parameters, and
monitored signals.

13
Model Quality Objectives | Version 1.0

The component model directly contributes to the development activities and is therefore subject to conformance
with industry quality standards, safety standards, and/or design standards (e.g. conformance to modeling standard,
traceability to requirements).
Figure 5 shows an example of a component design model with fully defined interfaces and sub-functions implement-
ed with state machines.

Figure 5: Example of component design model

2.6 Software Component Implementation and Testing Phase

This section focuses on the component implementation model developed during the software component design and
testing phase.

2.6.1 Role of the component implementation model

The role of the component implementation model is to enable the generation of production code for a specific
embedded target and basic software.

2.6.2 Characteristics of the component implementation model

The component implementation model fully specifies the software component implementation. Implementation
details are added to the data dictionary to refine how to represent parameters and signals in the target memory.
Code configuration options and customization are defined to integrate the generated code with specific basic soft-
ware functions, so they match the target characteristics (e.g. byte ordering) and satisfy the component code memory
footprint and execution performance requirements allocated to the software component.
The generated code of the component implementation model directly contributes to the development activities and
is therefore subject to conformance with the industry quality standard, safety standard, and/or coding standard (e.g.
MISRA C® [9]). Each component implementation model is associated with a data dictionary that defines its interface
parameters and monitored signals. Figure 6 shows an example of target hardware configuration of Embedded Coder.

14
Model Quality Objectives | Version 1.0

Figure 6: Example of code generation configuration for the component implementation model

2.7 Relationship Between Design Models

Each design model shall be independently managed as a work product of the software development phase in which it
belongs. At the same time, design models can share design information and shall be consistent. For instance, the
component design model in Figure 5 share its interface definition with the architecture model of Figure 4 .Whenever
consistency is required, reuse is encouraged.
Figure 7 indicates which aspects can be reused between design models (“reuse” arrow). It also provides guidance on
which aspects of design models can be partially reused to accelerate development (“refine” arrow). The arrows on
Figure 7 can apply to the following modeling aspects of design models:
• Architectural aspect: interface, scheduling, partitioning, intercomponent control and data flow, etc.
• Algorithmic aspect: mathematical calculation, component control and data flow, state machine, truth table, etc.
• Code generation aspect: memory management, data access, function prototype, code optimization, etc.
The design models differ from each other’s by the level of maturity and importance of the different modeling aspects
described above. Figure 7 indicates the levels of maturity and importance based on the following definitions and
representations:
• Maturity level: High (Production) / Low (Prototyping)
• Importance level: Mandatory (plain line) / Recommended (dotted line)

15
Model Quality Objectives | Version 1.0

Figure 7: Design model relationships and contribution to prototyping and production development

The functional model shall have structured algorithms that can contribute to the validation of the software require-
ments with modeling and simulation. A model’s code generation configuration for rapid-prototyping can be useful
to validate the software requirements with a real-time environment. The development focus shall be on the software
requirement (not represented on the figure). The entire model shall be considered a prototype.
The architecture model shall define the component interface and scheduling of the software architectural design.
The architectural design aspect of the functional model can serve as a baseline to initiate the development of the
software architecture for production (1a). The prototype algorithms of the functional model can populate the archi-
tecture model to enable early dynamic verification of the model in simulation to evaluate the impact of the architec-
ture on the functional behavior (2a). A prototype code generation configuration representative of the software
architecture standard (e.g. AUTOSAR) can be created to achieve early verification of the impact of the functional
behavior in real time and its integration with software and hardware (e.g. AUTOSAR RTE).
The component design model shall fully define the software component design with its structure, scheduling, and
algorithms. The interface of the model shall be consistent with, and can be reused from, the architecture model (1b).
The prototype algorithms developed for the functional model can serve as a baseline to define the production algo-
rithms (2b). A prototype code generation configuration can be used for early verification of the non-trivial impacts
of the design model on the generated code (e.g. compliance with the coding standard, level of code coverage versus
model coverage, code expansion).
The component implementation model shall define both the software component design and implementation. The
structure, scheduling, and algorithms shall be reused from the software component design model (1c, 2c). The way
algorithms are implemented can be adapted to address non-functional requirements (e.g. optimization, safety). The
code generation configuration shall be used for production code generation and shall then be compatible with the
software coding standard and the target hardware.

16
Model Quality Objectives | Version 1.0

3 Design Models Quality

3.1 Overview
As design models are critical for software development using Model-Based Design, their quality must be carefully
assessed. Design models can automatically transform into other design artifacts such as documentation, source code,
or executables. Therefore, the quality objectives defined on the design models shall impact the models themselves as
well as their derived products. A specific quality objective is defined for each type of design model to account for
their specific role.

Design model name Quality Objective

Functional model MQO-1

Architecture model MQO-2
Component design model MQO-3

Component implementation model MQO-4

Table 1: Model Quality Objectives of design models

17
Model Quality Objectives | Version 1.0

Table 2 below provides the list of Model Quality Requirement (MQR) applicable to achieve the quality objective of
each type of design models. The details of each MQR are specified in section 3.2.

MQR ID MQR Title MQO-1 MQO-2 MQO-3 MQO-4

MQR-01 Model layout M M M M

MQR-02 Model comments M M M M
MQR-03 Model links to requirements M M M M
MQR-04 Model testing against requirements M R M M

MQR-05 Model compliance with modeling M M M

standard
MQR-06 Model data M M M

MQR-07 Model size M M

MQR-08 Model complexity M M

MQR-09 Model coverage M M

MQR-10 Model robustness M M

MQR-11 Generated code testing against R M

requirements
MQR-12 Generated code compliance with R M
coding standard
MQR-13 Generated code coverage R M

MQR-14 Generated code robustness R M

MQR-15 Generated code execution time M

MQR-16 Generated code memory footprint M

M: Mandatory
R: Recommended for early verification

Table 2: Overview of Model Quality Requirements of MQOs

Note: An additional MQR to verify the generated source code against the model can be required in
the context of DO-331.

18
Model Quality Objectives | Version 1.0

3.2 Model Quality Requirements

3.2.1 Model layout

MQR-01 Model layout

The model shall define Simulink and Stateflow ® diagrams that are completely visible on
Description
A4 paper size.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory Mandatory Mandatory

Fit to view with a zoom ratio smaller than 80% is harder to read on screen and likely not
Notes to be readable on A4 paper size.
The model zoom ratio is visible at the center of the model status bar below the diagram.
• Simulink subsystems
References /Examples • Stateflow sub-charts
of techniques • Simulink bus

Printing a Simulink model can be necessary to archive or share models as documents.

A model diagram that can be completely displayed on screen improves readability and
Rationale eases model review.
Reducing the size of the diagrams forces the model developer to better organize large
model and data into hierarchical structures of buses and model references or subsystems.

Last update 1.0

3.2.2 Model comments

MQR-02 Model comments

The model comments shall provide a description of the model itself and the following
types of elements:
• Simulink subsystem
Description
• Simulink function and S-function mask
• Stateflow chart, sub-chart, truth table, state transition table, and flowchart
• Simulink and MATLAB function blocks and sub-functions

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory Mandatory Mandatory

A comment can include a mix of text, equations, diagrams, and pictures.

A comment can be embedded in the model or a link can be established from the model
Notes to a separate and accessible document.
The quality of the comments is not in the scope of this requirement and shall be assessed
by peers during the model review.
• Insertion of blocks for documentation
• Description in Simulink subsystems masks
References /Examples
• Stateflow diagrams annotations
of techniques
• Comments in Simulink and MATLAB function codes

Like code, a model without comments is harder to understand by peers. Lack of descrip-
Rationale tion can negatively impact the efficiency of the peer review activity and maintenance
activities.

Last update 1.0

19
Model Quality Objectives | Version 1.0

3.2.3 Model links to requirements

MQR-03 Model links to requirements

The model elements that specify algorithms and calculations shall trace to the model
higher level requirements.
Description
The design model elements that specify interface shall trace to the software interface
requirements or software component interface requirements.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory Mandatory Mandatory

A model element is implicitly traced to a model higher level requirement if one of its par-
ents is traced (e.g. its parent subsystem).
The model shall trace to the right level of requirements:
• Functional model and architecture model shall trace to software requirements
• Component design model and component implementation model shall trace to soft-
Notes
ware component requirements
The correctness of the links to model higher level requirements is not in the scope of this
requirement and shall be assessed by peers during the model review.
When model references are used inside component design and implementation models,
each referenced model shall trace to its own model higher level requirements.

References /Examples • Bidirectional links between model and requirement tool

of techniques
Traceability to requirements eases static model verification against requirements. It
facilitates:
Rationale • Requirement coverage analysis
• Impact analysis on design following changes on requirements
• Identification of unintended or useless design to be present in the model

Last update 1.0

3.2.4 Model testing against requirements

MQR-04 Model testing against requirements

The model shall produce the expected outputs when exercised by tests derived from and
Description
traced to the model higher level requirements.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory Mandatory Mandatory

The model tests shall be derived from and traced to all model higher level requirements
which verification strategy is testing.
Each test shall have a defined procedure, stimuli, and expected outputs.
Notes
The model test environment shall not impact the behavior of the model under test.
The correctness of the tests and links to model higher level requirements are not in the
scope of this requirement and shall be assessed by peers during the tests review.
• Stimuli and expected outputs time series
References /Examples • Test sequences and test oracles
of techniques • Automation of test procedure, execution, and reporting

The simulation of the design model enables the discovery of design errors at design [Link]
Rationale can also contribute to refining model higher level requirements or correcting and validat-
ing requirement-based tests.

Last update 1.0

20
Model Quality Objectives | Version 1.0

3.2.5 Model compliance with modeling standard

MQR-05 Model compliance with modeling standard

The model shall be compliant with the modeling standard.

Description

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory Mandatory

The modeling standard shall be defined during the project software planning phase and
shall be compatible with the software safety standard, software design standard, coding
standard, and targeted hardware (e.g. floating-point support).
Notes Model compilation warnings and errors reported by Simulink diagnostics are considered
modeling standard violations.
The modeling standard could be adapted to software architectural design modeling and
software component design modeling.
• MathWorks modeling guidelines for high-integrity systems
References /Examples (Include compatibility with MISRA C® compliance)
of techniques • MathWorks Automotive Advisory Board Control Algorithm Modeling Guidelines
Using MATLAB, Simulink, and Stateflow [8]
The model standard can enforce best practices and define a subset of the modeling lan-
Rationale
guage that limits the possibility of incorrect use of the language.

Last update 1.0

3.2.6 Model data

MQR-06 Model data
The model I/O signals, calibrations, and observable signals shall be fully defined with the
following properties:
• Name
• Description
• Design min/max
Description • Initial value (output only)
• Data type (e.g. base type, fixed-point type, enumerated type, structured type)
• Size
• Physical unit
• Safety integrity level
• Memory storage

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory Mandatory

The compute method is necessary for data coming from external software, driver, or com-
munication network.
Notes An initial value or safe value can be added for output and safety critical data.
Memory storage only needs to be defined in the component implementation model.
Display format for measured signal and calibration for floating point is recommended.
• Simulink data objects
Examples of techniques • Simulink data dictionary

Model data are part of the design and need to be fully defined. For instance, incorrect or
Rationale unknown data integrity level or data design min/max can impact the model and software
reliability and robustness.

Last update 1.0

21
Model Quality Objectives | Version 1.0

3.2.7 Model size

MQR-07 Model size
The model shall have less than 500 elements including:
• The number of Simulink blocks
Description • The number of MATLAB executable lines of codes
• The number of Stateflow transition, states, and connections
• The number of truth tables decision

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory

The model reference block only counts as one element.

The company standard utility function (e.g. Simulink library block, MATLAB function file)
Notes
only counts as one element.
Please refer to MathWorks guidance on large-scale modeling in Simulink documentation.
References /Examples
of techniques
Very large models are more difficult to merge and are more likely to be modified
by several users at the same time.
Rationale
Smaller models are more likely to be reusable and easily configurable.
Generated code of very large models cannot be incrementally tested.

Last update 1.0

3.2.8 Model complexity

MQR-08 Model complexity
The model and its subsystems, Stateflow charts, and MATLAB functions shall have a local
Description
cyclomatic complexity lower or equal to “30”.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory

Local complexity is the cyclomatic complexity for objects at their hierarchical level.
Aggregated cyclomatic complexity is the cyclomatic complexity of an object
and its descendants.
Notes
The threshold of 30 for local cyclomatic complexity is a recommendation and can be
adapted on a project basis. The number 30 for cyclomatic complexity has been derived
from the HIS [7] code metric and adapted to Model-Based Design.
Cyclomatic complexity is a measure of the structural complexity of a model. It approxi-
mates the McCabe complexity measure for code generated from the model. The McCabe
complexity measure is slightly higher on the generated code due to error checks that the
model coverage analysis does not consider.
To compute the cyclomatic complexity of an object, such as a block, chart, or state,
model coverage uses the following formula:
Examples of techniques

N is the number of decision points that the object represents and on is the number of out-
comes for the nth decision point. The tool adds one to the complexity number for atomic
subsystems and Stateflow charts.
Cyclomatic complexity is a leading testability metric. Test harness can be created for
Rationale
simulation at model, subsystem, chart, and MATLAB function level.

Last update 1.0

22
Model Quality Objectives | Version 1.0

3.2.9 Model coverage

MQR-09 Model coverage

The model structure shall be fully covered by the test suite that is derived from and traced
Description
to the model higher level requirements.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory

The structural coverage criteria chosen shall be at least conformant to the structural cover-
Notes
age criteria imposed by the software safety integrity level.
Types of coverage analysis available on Simulink model:
• Execution Coverage (EC)
• Decision Coverage (DC)
References /Examples • Condition Coverage (CC)
of techniques • Modified Condition/Decision Coverage (MCDC)
EC, DC, CC, MCDC, saturation on integer overflow coverage, and relational boundary
coverage can be used to measure the model structural coverage.

Model coverage enables to identify untested design, untestable design,

Rationale or unintended design.

Last update 1.0

3.2.10 Model robustness

MQR-10 Model robustness

Description The model shall be robust in normal and abnormal operating conditions.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory Mandatory

In normal operating condition, inputs and tunable parameters values are within their
design ranges.
In abnormal operating condition, inputs, and tunable parameters values are outside their
design ranges.
Robustness shall prevent errors such as:
Notes
• Divisions by zero
• Integer overflows
• Out of design range
• Out of bound array
The level of robustness shall be compliant with the software safety integrity level.
• Test generation based on relational boundary coverage
Examples of techniques • Formally-based verification technique with abstract interpretation
• Defensive programming
Model robustness verification prevents edge case or incorrect use of model, which can
Rationale
cause unexpected results or simulation errors.

Last update 1.0

23
Model Quality Objectives | Version 1.0

3.2.11 Generated code testing against requirements

MQR-11 Generated code testing against requirements

The model generated code shall produce the expected outputs when exercised by tests
Description
derived from and traced to the model higher level requirements

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Recommended Mandatory

For MQO-03, tests can be run in software-in-the-loop.

Notes For MQO-04, tests shall be run in processor-in-the-loop. A representative hardware or an
emulator can be used in place of the actual processor.

References /Examples • Test reuse from component design model testing

of techniques • Test generation for back-to-back testing

Code testing is required to verify the output of the code generator and compiler
or cross-compiler, linker, load, and flash utilities.
Rationale
For MQO-3, code testing in software-in-the-loop increases confidence in the
code generator.

Last update 1.0

3.2.12 Generated code compliance with coding standard

MQR-12 Generated code standard compliance

Description The generated code shall be compliant with the coding standard.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Recommended Mandatory

The coding standard shall be defined during the project software planning phase and
shall be compatible with the software safety standard, software architecture standard,
Notes and targeted hardware (e.g. floating-point support).
The modeling standard shall anticipate the compliance with the coding standard.
The project coding standard can be tailored for generated code.
• MISRA C 2012 for safety
Examples of techniques • CERT C for cyber security

Rationale Coding standard verification is required to verify the output of the code generator.

Last update 1.0

24
Model Quality Objectives | Version 1.0

3.2.13 Generated code coverage

MQR-13 Generated code coverage

The model generated code structure shall be fully covered by all the tests that are derived
Description
from and traced to the model higher level requirements.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Recommended Mandatory

The structural coverage criteria shall be at least conformant to the structure coverage crite-
ria imposed by the software safety integrity level.
The model tests shall be reused to cover the structure of the generated code.
Notes
The code coverage can be different than the model coverage depending on the blocks
used (e.g. look-up table interpolation algorithm) or code generation optimization options
(e.g. for loop unrolling).
Types of coverage analysis available on the generated code:
• Statement Coverage for Code Coverage
References /Examples
• Condition Coverage for Code Coverage
of techniques
• Decision Coverage for Code Coverage
• Modified Condition/Decision Coverage (MCDC) for Code Coverage
Code coverage is required in addition to model coverage to verify that the code genera-
Rationale tor do not add unintended functionalities.

Last update 1.0

3.2.14 Generated code robustness

MQR-14 Generated code robustness

Description The model generated code shall be robust in normal and abnormal operating conditions.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Recommended Mandatory

In normal operating condition, inputs and tunable parameter values are within their
design ranges.
In abnormal operating condition, inputs and tunable parameter values are outside their
design ranges.
Robustness shall prevent errors such as:
Notes
• Divisions by zero
• Integer overflows
• Out of design range
• Out of bound array
The level of robustness shall be compliant with the software safety integrity level.
• Test generation based on relational boundary coverage
Examples of techniques • Formally-based verification technique with abstract interpretation
• Defensive programming

Rationale Code robustness verification is required to verify the output of the code generator

Last update 1.0

25
Model Quality Objectives | Version 1.0

3.2.15 Generated code execution time

MQR-15 Generated code execution time

The model generated code running on the production target shall be instrumented to mea-
Description
sure and verify the execution time.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory

Worst case execution time shall be specified during software architectural design phase.
The execution time shall include the generated code and its calling functions (e.g. basic
software services).
Notes
The production target can be an emulator or a representative hardware.
The model tests can be reused on the generated code running on the production target
(aka processor-in-the-loop) and the expected outputs shall still be obtained.

References /Examples • Profiling in processor-in-the-loop from Simulink

of techniques
The component software execution time shall be measured prior the component integra-
Rationale tion to verify compatibility with architecture requirements, avoid shortage of hardware
resource, and enable reuse of component on different architecture.

Last update 1.0

3.2.16 Generated code memory footprint

MQR-16 Generated code memory footprint

Description The model generated code memory footprint shall be measured and verified.

MQO-1 MQO-2 MQO-3 MQO-4

Recommendation level
Mandatory

Memory footprint, such as RAM, ROM, and stack, shall be specified during software
Notes architectural design phase. The memory footprint shall include the generated code and its
calling functions.
• Stack estimation tool
Examples of techniques

The component software memory footprint shall be measured prior the component inte-
Rationale gration to verify compatibility with architecture requirements, avoid shortage of hardware
resource, and enable reuse of component on different architecture.

Last update 1.0

© 2018 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See [Link]/trademarks for a list of additional trademarks.
Other product or brand names may be trademarks or registered trademarks of their respective holders.

26