Polyspace Test Product Presentation
Polyspace Test Product Presentation
By Author
Date
Software
Requirements C/C++
Design
Write / Fix Code Test on Host Test on Target
Run Static Code Analysis
Run static analysis / create unit tests to test functions and code components
– Check for coding standards (MISRA) and runtime errors
– Trace detailed design requirements to tests, confirm intended and no-unintended behavior
Bug Finder
– Identify software bugs and security vulnerabilities
– Check compliance (MISRA, AUTOSAR, CERT, others)
Code Prover
– Prove absence of critical runtime errors
– Improve software design and code understanding
Supported workflows
– Polyspace as You Code for IDE integration
– Server based analysis for CI workflows
– Review results using web browser (Polyspace Access)
– Integration with Simulink for generated code
– Certification support (ISO 26262, DO-178, others)
3
MathWorks Vision for Dynamic Testing
4
Benefits of the Unified Platform for Static + Dynamic
Identify code coverage gaps
– Static code analysis can provide insights into code coverage
– Highlight areas of code that cannot be reached
Demo Example
• See this video with explanations on presenting the how_many_tests introductory demo
• Go to this link for the source file and demo instructions
• Recommend presenting the demo live. It will be necessary to prepare VScode and PS Test in
advance to help with live presentation. This demo will take ~10mins.
• You can skip this demo and discussion about ChatGPT if desired
6
Let's Start with Requirements
7
How many
test-cases to achieve
100% decision coverage?
A. 1 test-case
B. 2 test-cases
C. 5 test-cases
D. 7 test-cases
E. >8 test-cases
8
Live Demo
9
How many
test-cases to achieve
100% decision coverage?
10
What Would ChatGPT Say?
C/C++
Write / Fix Code Test on Host Test on Target
16
xUnit Testing Workflow
17
What is xUnit?
18
xUnit Testing Framework Benefits
21
Example GUI Driven Testing Workflow
API GUI
• Integration and • User Interaction
Automation and Simplicity
• Efficiency and
• Visual Design
Performance
• Customization and Layout
and Extensibility • Quick
• Better C++ Prototyping and
support Rapid
• Headless or Development
Server-Side • Automated test
Execution (CI/CD)
authoring
23
Advanced Features and Capabilities
• Topics in this section are discussed in the GUI demo shown previously, therefore recommend
selecting specific slides to have deeper discussions
• You can skip parts of this section if features important to the customer were already discussed
during the GUI demo presentation
• Note that certain topics such as PS Access integration, CI/CD, etc. are not shown in the xUnit
or GUI demos, but are included in this section 24
Unified Platform for Static Analysis and Dynamic Test
25
Centralized Configuration for Static + Dynamic
26
Centralized Tool Execution for Static + Dynamic
Check for defects, analyze for run-time errors, and run tests
27
Centralized Dashboard to Review Static + Dynamic Results
28
Centralized Navigation for Detailed Static + Dynamic Results
Consolidated
Results for
Bug Finder,
Code Prover,
and Test
29
Polyspace Access – Review Static + Dynamic Results
30
Notes for Current Polyspace Bug Finder and Code Prover Users
New GUI has different look and feel compared to prior releases
Project migration
– You can migrate old projects from prior releases (.psprj) to the new format (.psprjx)
– Cannot convert from new format to old format
– Does not apply to results file
Existing GUI in Polyspace Bug Finder and Code Prover will be available for at
least 4 releases (until R2025b)
See documentation for limitations and differences between the older and new GUI
31
Analyze Code for Testing – Code Explorer
32
Analyze Code for Testing – Call Graph
Stubbed
Function
33
Mock and Stub
Mock
– Change behavior of defined
functions in your code
Stub
– Create definitions for undefined
functions in your code
34
Requirements Traceability
35
Automatic Test Generation
Limitations
– Compute intensive, may run for significant time
– May not successfully generate test cases to achieve objectives
– Some coding constructs may not be supported (workaround by modifying code for test generation purposes)
36
Automatic Test Generation – Coverage
Coverage levels
– Decision, condition, MCDC
Relational boundary
– For relational operators such as
>, <, ==, etc.
– Produce tests with equal operand values
or differing by absolute or relative tolerance
37
Automatic Test Generation – Boundary Values
Mode
– Minimal: fewest tests that cover all boundary values
– Exhaustive: tests to cover all boundary value combinations
with multiple inputs
Options
– Include (force) zero values
– Smallest float epsilon values
– Off by one specifies max-1, min+1, etc.
38
Automatic Test Generation – Capability Leverage
39
Add or Import External Tests
40
Code Coverage Metrics
View code
– Highlight fully covered or justified code
41
Code Coverage Types
MCDC
Condition
• Statement: each statement in the code executed
if (X && Y) { • Decision: every branch of control statements executed
• Condition: Boolean evaluated for both true and false
Z = 1; Decision
• Modified Condition Decision Coverage (MCDC)
} else { Statement
Z = -1;
}
42
Code Coverage Overview
Statement Coverage
– Every statement is executed once
– Does not consider Decisions and Conditions
43
Streamline Unreachable Code Justification
44
Profile for Execution Speed and Memory
45
Test Code Robustness
46
Polyspace from Coding to Continuous Integration (CI/CD)
Coding
Integration
Code
Review Polyspace Access
Polyspace plugin Polyspace review results in browser
Pre-qualification
for your IDE on your desktop Quality
Dashboard
Test
Continuous
Release
Integration
Build
48
Compare and Merge Polyspace Platform Projects
CLI: polyspace-test
Diff/Merge: polyspace-project
49
On-Target Testing with Target Boards or Instruction Set Simulators
Build
On-target test execution Process
Polyspace Test
– Leverages MathWorks Target
Framework (TF) Host-side Coms Launch
API
50
Target and ISS Support
License requirements
– Authoring TF for specific processor*
MATLAB and MATLAB Coder sufficient for implementing the TF
Embedded Coder needed to test the TF
– Executing with the TF
Polyspace Test
* When done by MathWorks consulting services, these additional licenses are not required
51
Certification
52
ISO 26262 - Software Unit Verification
IEC Certification Kit for ISO 26262, IEC 61508, and related standards
– Certified by TÜV SÜD for suitability for use
– Provides tool classification and tool qualification work products, templates, certificates, and validation suites
Polyspace Test
– –Support for Table 7 methods 1j-1l, Table 9 methods 1a-1c,Table 10 methods 1a-1c, Table 12 methods 1a-1b
– For requirements-based tests, coverage analysis, interface tests, fault injection tests
A B C D
1n + + ‡ ‡
1m + + + ‡
1l + + + ‡
1k ASIL A B C D‡ ‡ ‡ ‡
1j A B C D A B C D + + ‡ ‡‡ ‡ ‡ ‡
1i + + + + ‡ ‡ ‡ ‡ A B C D ‡ ‡ ‡ ‡+ + + + A B C D
1h ‡ ‡ ‡ ‡ + + + ‡ + + ‡ ‡ + ‡ ‡ ‡‡ ‡ ‡ ‡ + + + +
1g + ‡ ‡ ‡ + + + ‡ + + ‡ ‡ + ‡ ‡ ‡+ + ‡ ‡ ‡ ‡ ‡ ‡ A B C D
Topics
1f + ‡ ‡ ‡ ‡ ‡ ‡ ‡ + + ‡ ‡ + ‡ ‡ ‡+ + ‡ ‡ + + ‡ ‡ + ‡ ‡ ‡
1e + + ‡ ‡ A B C D + ‡ ‡ ‡ o o + + A B C D + + ‡ ‡o o + + A B C D + + ‡ ‡ A B C D + + ‡ ‡
1d + + ‡ ‡ + + + + + ‡ ‡ ‡ o o + ‡ + + + + ‡ ‡ ‡ ‡+ + ‡ ‡ + + + + A B C D ‡ ‡ ‡ ‡ + + + + A B C D + + ‡ ‡
1c ‡ ‡ ‡ ‡ + + ‡ ‡ + + + ‡ + + + ‡ + + ‡ ‡ ‡ ‡ ‡ ‡+ ‡ ‡ ‡ + ‡ ‡ ‡ + + + ‡ + + ‡ ‡ + ‡ ‡ ‡ A B C D + + ‡ ‡ A B C D + + ‡ ‡
1b ‡ ‡ ‡ ‡ ‡ ‡ + + ‡ ‡ ‡ ‡ + ‡ ‡ ‡ ‡ ‡ + + + ‡ ‡ ‡+ + + + + ‡ ‡ ‡ + ‡ ‡ ‡ ‡ ‡ ‡ ‡ + ‡ ‡ ‡ + + ‡ ‡ ‡ ‡ ‡ ‡ + + + ‡ + ‡ ‡ ‡
1a ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ + o o ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡‡ + O o ‡ ‡ ‡ ‡ ‡ ‡ + + ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ + + ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡ ‡
Table 1 Table 2 Table 3 Table 4 Table 5 Table 6 Table 7 Table 8 Table 9 Table 10 Table 11 Table 12 Table 13 Table 14 Table 15
Tables
Polyspace Bug Finder and Code Prover Polyspace Test 53
Qualification of MathWorks Tools
MathWorks
MathWorks
Pre-qualification based on
reference use cases / workflows
Report package
Templates, work products,
acc. to ISO 26262-8
Independent Assessment Test package
Validation tests harnesses,
example tests
Certification Kit
Tool User
Project-specific
adaptation
54
DO Qual Kit Support for Polyspace Test
DO-330* Compliant Artifacts
* Supplement for Software Tool Qualification Considerations: Guidance on the qualification of software
tools used in the development and verification of airborne systems
55
Qualifiable Products for Handwritten C/C++ Code
56
Requirements-Based Testing and Coverage
e
st d
Te Co
e d i e d ge
l i f i l ts l i f ra
ua su ua ve
Q Re Q Co
57
MathWorks Test and Verification Product Portfolio
58
MathWorks Portfolio of Test and Verification Tools
Polyspace Test
Dynamic testing Static analysis Bridge between Dynamic testing Static analysis
Model-Based Design
Requirements Toolbox and Code
Model-Based Design Handwritten and Mixed Code
59
Conclusion and Summary
60
Polyspace Test – Built with MathWorks Model Testing Experience
Testing workflows
Customer Success Stories
– Simulink Test: MIL, SIL, PIL www.mathworks.com/company/customer-stories.html
– Released 2015
Requirements traceability
– Requirements Toolbox
– Released 2009
Check unit tests with Polyspace Bug Finder / Code Prover to confirm unit test is robust
– Confirm that there are no issues with the code under test when tested under specific test scenarios
62
Installation and Licensing
63
Downloading and Installing Polyspace Test
Download
– Navigate to this URL mathworks.com/downloads
– Select Get Polyspace Test Products and download for your platform
Install instructions
– mathworks.com/help/polyspace_test/install-polyspace-test.html
64
Polyspace Test Licensing
Polyspace Test does not require MATLAB or any other products
Polyspace Test can be installed and used on the desktop and CI/CD servers
– Separate license and installation not required
– Note: Bug Finder and Code Prover for CI requires Polyspace Bug Finder Server and Polyspace Code Prover Server
Developing custom targets using Target Framework (TF) requires additional product licenses
– Authoring and testing TF for specific processor*
MATLAB and MATLAB Coder sufficient for implementing the TF
Embedded Coder needed in for testing the TF in context of authoring
– Executing with the TF
65
* When developed by MathWorks Consulting,
Polyspace Test these additional licenses are not required
Product Overview
Polyspace Test
66
Polyspace Test NEW PRODUCT
Develop, Manage, and Execute Tests for C and C++ Code in
Embedded Systems
Who it’s for:
Software Developers, Testers, and Quality Engineers
Responsible for unit testing and integration testing of handwritten C/C++ code or mixed code (hand code + generated code)
Where software quality has safety, security, or business-critical impact, or requires certification
Challenges:
Software quality metrics or certification require thorough testing and high code coverage
Tedium of writing, managing, and executing tests
A single solution unifying static code analysis and dynamic testing
67
Polyspace Test NEW PRODUCT
Develop, Manage, and Execute Tests for C and C++ Code in
Embedded Systems
68
Polyspace Test NEW PRODUCT
Develop, Manage, and Execute Tests for C and C++ Code in
Embedded Systems
69
Polyspace Test NEW PRODUCT
Develop, Manage, and Execute Tests for C and C++ Code in
Embedded Systems
70
Polyspace Test NEW PRODUCT
Develop, Manage, and Execute Tests for C and C++ Code in
Embedded Systems
Test on Target
71
Polyspace Test NEW PRODUCT
Develop, Manage, and Execute Tests for C and C++ Code in
Embedded Systems
72
Backup
73
Software Test Challenges
74