AI Software Development Kit

Introduction 1

Safety notes 2
Installing AI Software
Development Kit 3
Using AI Software
Development Kit 4
AI Software Development Kit Guideline for writing
5
pipeline components

Operating Manual

04/2023
A5E52031285-AE
Legal information
Warning notice system
This manual contains notices you have to observe in order to ensure your personal safety, as well as to prevent
damage to property. The notices referring to your personal safety are highlighted in the manual by a safety alert
symbol, notices referring only to property damage have no safety alert symbol. These notices shown below are
graded according to the degree of danger.

DANGER
indicates that death or severe personal injury will result if proper precautions are not taken.

WARNING
indicates that death or severe personal injury may result if proper precautions are not taken.

CAUTION
indicates that minor personal injury can result if proper precautions are not taken.

NOTICE
indicates that property damage can result if proper precautions are not taken.
If more than one degree of danger is present, the warning notice representing the highest degree of danger will
be used. A notice warning of injury to persons with a safety alert symbol may also include a warning relating to
property damage.
Qualified Personnel
The product/system described in this documentation may be operated only by personnel qualified for the specific
task in accordance with the relevant documentation, in particular its warning notices and safety instructions.
Qualified personnel are those who, based on their training and experience, are capable of identifying risks and
avoiding potential hazards when working with these products/systems.
Proper use of Siemens products
Note the following:

WARNING
Siemens products may only be used for the applications described in the catalog and in the relevant technical
documentation. If products and components from other manufacturers are used, these must be recommended
or approved by Siemens. Proper transport, storage, installation, assembly, commissioning, operation and
maintenance are required to ensure that the products operate safely and without any problems. The permissible
ambient conditions must be complied with. The information in the relevant documentation must be observed.

Trademarks
All names identified by ® are registered trademarks of Siemens AG. The remaining trademarks in this publication
may be trademarks whose use by third parties for their own purposes could violate the rights of the owner.
Disclaimer of Liability
We have reviewed the contents of this publication to ensure consistency with the hardware and software
described. Since variance cannot be precluded entirely, we cannot guarantee full consistency. However, the
information in this publication is reviewed regularly and any necessary corrections are included in subsequent
editions.

Siemens AG A5E52031285-AE Copyright © Siemens AG 2023.

Digital Industries Ⓟ 04/2023 Subject to change All rights reserved
Postfach 48 48
90026 NÜRNBERG
GERMANY
Table of contents

1 Introduction ........................................................................................................................................... 5
1.1 Overview of Industrial Edge ................................................................................................. 5
1.2 AI@Edge .............................................................................................................................. 7
1.3 AI Software Development Kit functionalities ......................................................................... 8
1.4 Information about the software license ................................................................................ 8
2 Safety notes ........................................................................................................................................... 9
2.1 Security information ............................................................................................................ 9
2.2 Note on use ......................................................................................................................... 9
2.3 Note regarding the general data protection regulation ....................................................... 10
3 Installing AI Software Development Kit .............................................................................................. 11
3.1 Install and run.................................................................................................................... 11
4 Using AI Software Development Kit .................................................................................................... 14
4.1 Preparing data for training ................................................................................................. 14
4.2 Training models ................................................................................................................. 15
4.3 Packaging models into an inference pipeline ...................................................................... 16
4.4 Test the pipeline configuration package locally................................................................... 20
4.5 Mocking the logger of AI Inference Server .......................................................................... 24
4.6 Deploy the packaged inference pipeline for AI@Edge.......................................................... 24
4.7 Create delta package and deploy to AI@Edge ..................................................................... 24
5 Guideline for writing pipeline components ........................................................................................ 25
5.1 Component definition ........................................................................................................ 25
5.2 The entrypoint ................................................................................................................... 26
5.3 Input data .......................................................................................................................... 27
5.3.1 Variable types .................................................................................................................... 27
5.3.2 Restrictions on type Object................................................................................................. 28
5.3.3 Custom data formats ......................................................................................................... 28
5.4 Processing data.................................................................................................................. 30
5.5 Python dependencies ......................................................................................................... 30
5.6 File resources..................................................................................................................... 31
5.7 Returning the result ........................................................................................................... 32
5.8 Adding custom metrics ...................................................................................................... 33
5.9 Pipeline parameters ........................................................................................................... 34

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 3
Table of contents

5.10 Use cases ........................................................................................................................... 35

5.10.1 Processing images ............................................................................................................. 35
5.10.2 Processing time series of signals......................................................................................... 36
5.10.3 Processing batch data ........................................................................................................ 38
5.11 Examples ........................................................................................................................... 39
5.12 Writing components for earlier versions of AI Inference Server ........................................... 42

AI Software Development Kit

4 Operating Manual, 04/2023, A5E52031285-AE
Introduction 1
1.1 Overview of Industrial Edge
Siemens Industrial Edge is the next generation of digital automation. With Industrial Edge,
you use intelligence and scalability of the cloud directly in your manufacturing - in a simple,
high-performance manner and without your data leaving the manufacturing process.
Industrial Edge combines local and high-performance data processing directly in automation
with the advantages of the cloud: app-based data analysis, data processing and
Infrastructure-as-a-Service concepts with central update functionality. In this way, you can
quickly integrate apps into manufacturing and manage them with a high degree of
automation.
Industrial Edge gives you the opportunity to make continuous changes to your automation
components and plants. It analyzes large volumes of automation data to implement
innovative functions, such as preventive maintenance, and to obtain maximum flexibility and
thus productivity over the entire machine life cycle.

Industrial Edge Hub

With the Industrial Edge Hub, an App Store is available to you where you can find all Siemens
apps and 3rd-party apps. From here, you can manage all licenses for your apps and devices
centrally. You can install updates for security issues, device firmware, apps and Industrial
Edge Management.
You can monitor and manage distributed Edge devices centrally in the Industrial Edge
Management. In this way, for example, new apps and software functions can be installed on
all connected Edge devices company-wide. The central software management thus minimizes
the effort to carry out maintenance and updates on individual devices.
On the individual Industrial Edge devices, you can start and run apps and keep statistics on an
Edge device, for example.
With the Industrial Edge Publisher, you can develop your own Edge apps and make them
available to other users in Industrial Edge Management.
Another component of the Industrial Edge ecosystem is the Industrial Edge Runtime, which is
installed on Edge Devices (IED) or Unified Comfort Panels (UCP) and on which the system
ultimately runs, with all applications.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 5
Introduction
1.1 Overview of Industrial Edge

Industrial Hub Industrial Edge Management Industrial Edge Runtime

on Industrial Devices
Platform for purchasing apps and Centralized control level for Software level for app container
software, and for monitoring managing devices, apps, and store
management systems floor users
• Central management location • Assignment of apps to the • Installation of scalable apps on
for apps, contributing to matching Edge devices many different Edge devices
corporate standardization (worldwide) • Supports use in industrial
• Management of all licenses in • Specification of user rights environments by:
use and thus easy cost
(e.g. app installations) – Ensuring security and
estimates reliability
• Just a few clicks for app setup – Providing comprehensive
• Overview of all management
and security update cycles user management to meet
system instances that are in
• Supervision of all operations the requirements of
use worldwide machine manufacturers
using the centralized Admin
and plant operators alike
view
– Adhering to Company
• Excellent usability for IT and Policy Compliance, e.g. user
OT users, helping to promote management integration or
user adaptation and self- IT/firewall specifications
service • Integrating device connectivity
to cloud and automation
systems

AI Software Development Kit

6 Operating Manual, 04/2023, A5E52031285-AE
 Introduction
1.2 AI@Edge

1.2 AI@Edge
Siemens Industrial Edge Ecosystem is enabled with the SIMATIC AI Launcher products for
industrial AI. With SIMATIC AI Launcher, the scalable industrial edge ecosystem is extended by
AI capabilities that facilitate the provision of AI models in the production environment on the
shopfloor.

Introduction of AI models into the shop floor

Customers can use the model training environment of their choice, cloud or on-premise.
Data scientists/AI engineers can use the AI framework of their choice.
Siemens offers an easy-to-use AI Software Development Kit (AI SDK) that has ready-made
features that generate the Siemens standard format with the designed AI pipelines. The
standard format is fully compatible with AI Inference Server for Industrial Edge.
AI Inference Server App is a ready-to-use inference runtime from Siemens that receives
AI pipelines as configuration packages (content deployment). This can be done manually
through the available user interface or automatically received for scaling via the AI Model
Manager – your extension of the Industrial Edge Manager for AI Fleet Management.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 7
Introduction
1.3 AI Software Development Kit functionalities

See also
Industrial Edge Homepage (https://new.siemens.com/global/en/products/automation/topic-
areas/industrial-edge.html)
AI@Edge Homepage (https://new.siemens.com/global/en/products/automation/topic-
areas/industrial-edge/production-machines.html)

1.3 AI Software Development Kit functionalities

AI Software Development Kit, or AI SDK for short, is a set of Python libraries, wihich provide
building blocks for automating the creation, packaging and testing of inference pipelines for
AI Inference Server.
The AI SDK is accompanied by project templates that deploy notebook-based workflows for
training models, package them for deployment, and test those packages.
The AI Software Development Kit assumes a machine learning workflow that includes the
following steps:
• Preparation of data for training
• Training models
• Packaging models into an inference pipeline
• Testing of packaged inference pipelines
• Converting and deploying packaged inference pipelines to AI@Edge
If you have already a trained model, you might directly begin with packaging. Please note
that the preceding chapters might provide contextual information which might aid in
understanding the concepts.
The AI Software Development Kit can be used both exploratively from interactive Python
notebooks and purely programmatically as part of an automated ML workflow.

1.4 Information about the software license

Software from third-party suppliers

AI Software Development Kit contains Open-Source Software and/or other software from
third parties.
Copyright © SIEMENS, 2023, and licensors. All rights reserved. Parts contain Open-Source
Software. More information can be found in the README_OSS.

AI Software Development Kit

8 Operating Manual, 04/2023, A5E52031285-AE
Safety notes 2
2.1 Security information
Siemens provides products and solutions with industrial security functions that support the
secure operation of plants, systems, machines and networks.
In order to protect plants, systems, machines and networks against cyber threats, it is
necessary to implement – and continuously maintain – a holistic, state-of-the-art industrial
security concept. Siemens’ products and solutions constitute one element of such a concept.
Customers are responsible for preventing unauthorized access to their plants, systems,
machines and networks. Such systems, machines and components should only be connected
to an enterprise network or the internet if and to the extent such a connection is necessary
and only when appropriate security measures (e.g. firewalls and/or network segmentation)
are in place.
For additional information on industrial security measures that may be implemented, please
visit (https://new.siemens.com/global/en/products/automation/topic-areas/industrial-
security.html).
Siemens' products and solutions undergo continuous development to make them more
secure. Siemens strongly recommends that product updates are applied as soon as they are
available and that the latest product versions are used. Use of product versions that are no
longer supported, and failure to apply the latest updates may increase customers' exposure to
cyber threats.
More information about network segmentation, firewall etc. is all on these pages.
To stay informed about product updates, subscribe to the Siemens Industrial Security RSS
Feed visit (https://new.siemens.com/global/en/products/automation/topic-areas/industrial-
security.html).

2.2 Note on use

Protection of the host computer

Customers are responsible for protecting their own host computers and preventing
unauthorized access to their host computers.
To protect the host computer Siemens suggests taking the following measures:
• Deploy the host computer only in isolated plant network, but not office network.
• Enable the screen saver and lock the screen when leave.
• Install suitable anti-virus software.
• Install updates and patches for the operating system and software on the host PC in time.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 9
Safety notes
2.3 Note regarding the general data protection regulation

Notes on protecting administrator accounts

A user with administrator rights has extensive access and manipulation options available to
the system.
Therefore, ensure there are adequate safeguards for protecting the administrator accounts to
prevent unauthorized changes. To do this, use secure passwords and a standard user account
for normal operation. Other measures, such as the use of security policies, should be applied
as needed.

Notes on the use

• Before installing AI Software Development Kit, it is recommended to verify the SHA-256
checksum of the distribution zip package against the checksum provided on Siemens
Online Industry Support.
• AI Software Development Kit can only be accessed from the host computer. Do NOT allow
other machines in the plant network to access AI Software Development Kit.
• The current AI Software Development Kit is only applicable for non-safety critical
application.
• AI Software Development Kit stores the project data without encryption on the host PC.
The customer is responsible for the CIA (Confidentiality, Integrity and Availability) of the
files created, stored, downloaded, or exported by AI Software Development Kit.
• AI Software development Kit might be used in conjunction with Jupyter Lab, which
includes a web server that can be accessed locally or remotely. The customer is
responsible for configuring Jupyter Lab with HTTPS enabled (https://jupyter-
notebook.readthedocs.io/en/stable/public_server.html#using-ssl-for-encrypted-
communication).
• If you use the AI Software Development Kit to create pipeline configuration packages,
make sure that you only include source code and Python packages from trusted sources.
• If you use the AI Software Development Kit to run pipeline configuration packages locally,
make sure that you only use pipeline configuration packages from trusted sources.

2.3 Note regarding the general data protection regulation

Siemens observes the principles of data protection, in particular the principle of data
minimization (privacy by design). For this product this means:
The product does not process / store any personal data, only technical functional data (e.g.
time stamp, IP addresses of the connected manufacturing devices). If the user links this data
with other data (e.g. shift plans) or stores personal data on the same medium (e.g. hard disk)
and thus establishes a personal reference, the user must ensure compliance with data
protection regulations.

AI Software Development Kit

10 Operating Manual, 04/2023, A5E52031285-AE
Installing AI Software Development Kit 3
3.1 Install and run
Depending on your background, you can choose to use AI Software Development Kit (AI SDK)
in a pure Python 3.8 environment or use the Jupyter Notebook-oriented project templates
with a notebook editor of your choice.
To familiarize yourself with the AI SDK, we recommend that you start with the project
templates. These are sample solutions that contain all the necessary dependencies and allow
for a quick and smooth start. Once you have left your interactive exploration phase behind,
you can consider switching to a pure Python approach.

Prerequisites
Before you begin, make sure you have internet access. If you reach the internet through a
proxy, such as if you are working in a corporate network directly or via VPN, please make sure
that you have configured the following tools to use the correct proxy.
• pip
• conda (If you also use conda)
Setting environment variables http_proxy and https_proxy covers both. A detailed
explanation about alternative solutions is provided in:
• Using a proxy server (https://pip.pypa.io/en/stable/user_guide/#using-a-proxy-server)
• Using Anaconda behind a company proxy (https://docs.anaconda.com/anaconda/user-
guide/tasks/proxy/)

Using the AI Software Development Kit without project templates

Just install the AI SDK from the Python wheel using pip in a Python 3.8 environment. This
ensures that any additional Python packages required are also installed.
pip install simaticai-1.4.0-py3-none-any.whl
Please note that by default, pip installs the latest available version of the required packages
that are compatible with the AI SDK and possibly other packages that are already installed. If
you want to make sure that you are using the versions listed in Readme_OSS, you can apply
the appropriate constraint during installation as follows:
pip install simaticai-1.4.0-py3-none-any.whl -c constraints.txt
Please note that this increases the probability that pip will not be able to resolve all
applicable restrictions or an older package and versions with security issues will be installed.
To use the Software Development Kit from your Python code, you must import modules from
the simaticai namespace. For more information, see the User Guide, Software
Development Kit API reference, or project Templates.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 11
Installing AI Software Development Kit
3.1 Install and run

Using the AI Software Development Kit with project templates

Standalone project template zip packages contain a prepared working directory with
notebooks and sources. You can use them in a Python or Jupyter Lab environment of your
choice. Prerequisites are:
• Python 3.8, installed either natively or via Conda.
• A notebook editor such as Jupyter Notebook, Jupyter Lab or Visual Studio Code.
We strongly recommend that you set up a separate Python environment specifically for the
project template, as described in the README of the templates. The notebooks in the project
templates assume their own Python environment with a predefined name and a matching
kernel name. For example, the State Identifier project template uses the environment name
state_identifier.
You can use your favorite Python environment manager to create the Python environment.
Here we show the commands for Conda and Python venv using the project template State
Identifier, as an example. For other project templates, you must replace the name
state_identifier as described in the according README.

# create Conda environment including Python and activate it

conda create -n state_identifier python=3.8.16
conda activate state_identifier

# create Python virtual environment on Linux and activate it

python -m venv ~/venv/state_identifier
source ~/venv/state_identifier/bin/activate

# create Python virtual environment on Windows and activate it

python -m venv %USERPROFILE%\venv\state_identifier
USERPROFILE%\venv\state_identifier\Scripts\activate.bat
Once the environment is created and activated, you must register it as an interactive Python
kernel so that it becomes available in your notebook editor. This is usually achieved with the
following commands:

# install and register interactive Python kernel

python -m ipykernel install --user --name state_identifier --
display-name "Python (state_identifier)"
Now your Python environment is ready to be used for the project template. Unzip the project
template from its package, change the working directory to the extracted project folder, and
execute the following command.

# install packages required for the template including the AI SDK and ipykernel
pip install ipykernel -r requirements.txt -f <directory path
containing simaticai wheel>
Please note that you have to specify a path to the directory containing the AI SDK wheel, not
a path to the wheel itself.

AI Software Development Kit

12 Operating Manual, 04/2023, A5E52031285-AE
 Installing AI Software Development Kit
3.1 Install and run

Once the required packages are installed, you can explore and execute them in your
notebook editor.
Please make sure that you select the appropriate interactive Python kernel to execute the
notebooks, which is in this example: Python (state_identifier).
Note that by default, pip installs the newest available version of required packages that are
compatible with the AI SDK and the project template. If you want to make sure to use the
versions that are listed in Readme_OSS, you can apply the appropriate constraint during
installation as follows:

pip install ipykernel -r requirements.txt -f <directory path

containing simaticai wheel> -c constraints.txt
Note that this increases the probability that pip installs older package versions containing
security issues.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 13
Using AI Software Development Kit 4
4.1 Preparing data for training
Preparing data for training the models is mostly out of scope for the AI Software
Development Kit (AI SDK). Prepared example data sets are available for the project templates,
allowing you to try out these templates without tedious data acquisition. Each project
template includes a notebook to help you download a sample data set.

Processing time series data

The State Identifier project template provides basic building blocks for building ML models
that process time series of aligned signals. Aligned signals mean that the input of the
processing pipeline consists of rows, containing a value for each signal. For example, a row
consisting of 3 signals and a time stamp would look like this:

Time stamp var1 var2 var3

09:50:23 1.2 202 25

The building blocks help you create a time series pipeline that processes a stream of such
rows according to the following pattern:

Here is the role of piping elements as follows:

• "Windowing" accumulates a given number of input rows in a processing window.
• The "feature extractor" calculates a number of characteristics for each window. A feature is
mathematically a value calculated from the values in the window.
• The "classifier" is the actual machine learning model that predicts a class for each window
based on the extracted characteristics.
Most real-world pipelines contain other processing elements, such as imputers to fill missing
values, or scalers that map an input to a given predefined range.

AI Software Development Kit

14 Operating Manual, 04/2023, A5E52031285-AE
 Using AI Software Development Kit
4.2 Training models

To train the classifier in such a pipeline, the input data must be routed through the
preprocessing steps during the training process.
Therefore, this processing pipeline must be defined as part of data preparation before
training. This is where the building blocks in the State Identifier project template come into
play.
These building blocks are based on the widely used machine learning Python package
scikit-learn. Scikit-learn provides a framework for defining pipelines that allows you to
combine data transformers with classifiers or other kinds of estimators. The building blocks
are located in the src/pipeline.py file in the State Identifier project template. The main
ones are:
• WindowTransformer, which transforms a series of input rows into a series of windows
of rows
• FeatureTransformer, which transforms a window of rows into the feature values
according to user-defined functions.
In addition to these transformers, there is a transformer called FillMissingValues, which
performs input data correction for simple cases. For more advanced cases, you should use a
more sophisticated imputer to correct your input.
For more details and concrete examples, please refer to the training notebooks in project
template State Identifier. We also recommend to study the documentation of scikit-learn if
you would like to understand in-depth how scikit-learn works or if you want to implement
own transformers.

Mapping predicted classes of data windows to data points

As described in the previous chapter, time series data is typically classified window by
window. This means that the class of a single data row is not defined per se. Nevertheless,
there are cases where it is convenient to map the classes defined window by window to the
data points themselves. One such opportunity could be if you want to visualize the data
points according to their classification, e.g., plot the data points color-coded with the class.
The State Identifier project template provides utility function back_propagate_labels in
the file src/pipeline.py to perform this mapping. For more details and concrete
examples, please refer to the training notebooks in project template State Identifier.

4.2 Training models

The AI SDK does not restrict how you train your models and save the trained model. You can
take the training notebooks in the project templates as an example, but you can select any
ML libraries that are compatible with AI Inference Server. The project templates include
examples of scikit-learn and TensorFlow.
Some ML frameworks, such as TensorFlow, use their own format for storing trained models.
Other frameworks, such as scikit-learn, rely on persistent Python runtime objects. In the latter
case, you need to make sure that the same versions of Python libraries exist when the objects
are stored after training and retrieved on the AI Inference Server. The packaging feature of
the AI SDK supports this by requiring exact version specifiers for required Python packages in
a pipeline package.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 15
Using AI Software Development Kit
4.3 Packaging models into an inference pipeline

4.3 Packaging models into an inference pipeline

The AI SDK provides the functionality to create a pipeline configuration package that wraps
trained models. These models can be converted to an edge configuration package using
AI SDK and then uploaded and run AI Inference Server on an Industrial Edge Device. The
related functions are located in the simaticai.deployment module.

Single or multiple components

From a deployment perspective, the inference pipeline can consist of one or more
components. This is independent of the logical structure of the inference pipeline. For
example, a typical time series pipeline that consists of multiple Scikit Learn pipeline elements
can be packaged into a single pipeline component for deployment:

Alternatively, you can deploy the same pipeline split into two components:

To keep things simple and less error-prone, you should usually deploy your inference pipeline
with as few components as possible.

AI Software Development Kit

16 Operating Manual, 04/2023, A5E52031285-AE
 Using AI Software Development Kit
4.3 Packaging models into an inference pipeline

In many cases, a single component is sufficient. However, there may be reasons why you
should consider using separate components, such as:
• You need a different Python environment for different parts of your processing, e.g., you
have components that require conflicting package versions.
• You want to exploit parallelism between components without implementing
multithreading.
• You want to modularize your pipeline and build it from a pool of component variants that
you can flexibly combine.

Creating an inference pipeline package

The AI SDK allows you to create pipeline components implemented in Python and assemble
linear pipelines from one or more such components.
The API is designed to anticipate future possible types of components that could be based on
a technology other than Python, such as ONNX or native TensorFlow serving. However, only
Python is currently supported.
The workflow for creating an inference pipeline package is as follows:
1. Write the Python code that wraps your trained model as an inference pipeline component.
2. Define the pipeline component.
3. Repeat the above steps if you have multiple components.
4. Configure the pipeline.
5. Save the pipeline configuration in a pipeline configuration package.

Creating pipeline components implemented in Python

Implementing an inference pipeline component in Python is a comprehensive topic in itself
and will be described in detail in the next chapter, Guideline for writing pipeline components
(Page 25). We only give an overview here.
A component consists of files and metadata.
Files contain:
• Python scripts
• trained models
Metadata includes:
• component name, component version
• required Python version and Python packages
• input and output variables
• the number of parallel executors
• the entry point
How you arrange the files in the file system is up to you. We recommend that you follow the
project templates for the AI SDK, where source code and stored trained models are organized
in a predefined structure. The idea is that you keep the same relative structure on the

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 17
Using AI Software Development Kit
4.3 Packaging models into an inference pipeline

AI Inference Server, which allows you to use the same relative references from the source
code to the stored models or other files.
Put together all the files for the components. Usually, it should be at least a Python script for
the entry point, the inference wrapper, and a saved model. Create the pipeline component by
running a Python script or notebook that provides the following functionality:
• creates a Python Component object with a specific name, component version, and
required Python version
• defines required Python packages
• defines input and output variables
• defines custom metrics
• defines the number of parallel executors
• adds Python scripts and saved models
• defines the entry point under the Python scripts
All this takes place with the corresponding functionality of the simaticai.deployment
module. For concrete examples, see the packaging notebooks in the project templates. For
more information and advanced options, see the AI SDK API reference manual.
Consider the following limitations:
• The AI SDK allows you to select a required Python version that is supported by different
versions of AI Inference Server.
• Make sure you select a Python version that is supported by the version installed on your
Industrial Edge target device. At the time of writing, this is Python version 3.8.
• The required Python packages must either be added as wheels to the pipeline component
or be available for download via pip for the target Inference Server.
• At the time of writing, there is a limitation that the entry point script must be in the root
folder of the package. This requirement can be relaxed with later versions of AI Inference
Server.
• AI Inference Server supports a maximum of 8 parallel executors.

Configuring and saving the pipeline

After you create the components, even if it is only a single one, you have to combine them
into a pipeline using a Pipeline object.
If the pipeline has only a single component, the situation is easy because the pipeline has the
same input and output variables as its sole component. You only need to specify a pipeline
name and version from which the file name is derived when you save the package. For
example, see the end of the package creation notebooks in the project templates.
To create a linear pipeline of multiple components, you can still mostly rely on the
constructor of Pipeline, which attempts to automatically connect the components passed
as a list:
• connecting pipeline input to the first component
• connecting inputs and outputs of subsequent components if the variable name matches
• connecting the last component to the pipeline output

AI Software Development Kit

18 Operating Manual, 04/2023, A5E52031285-AE
 Using AI Software Development Kit
4.3 Packaging models into an inference pipeline

In general, we recommend that you pass data from one component to another in a single
variable of type String and serialize and deserialize any data you have through a string.
The low-level methods of the Pipeline class allow you to define arbitrary wiring between
components and pipeline inputs and outputs, but the AI SDK cannot guarantee that the result
will behave on AI Inference Server as intended.
For information about pipeline input and output with different data types and defining
custom metrics, see the Guideline for writing runtime components (Page 25). It explains how
input and output data is passed between AI Inference Server and your entry point and
explains special considerations that apply for a continuous stream of time series data or for
bulk data.
Whether you created the pipeline with a single constructor call or with low-level methods,
you must save it as a final step. This creates the pipeline configuration package as a .zip file
and leaves the contents of the .zip file in the file system, which you can explore to
troubleshoot or see how your package creation calls are reflected in the contents of files and
directories.

Pipeline parameters
Advanced use cases might require that the behavior of the pipeline is modified after
deployment, for example by changing the parameters of the AI model. So AI SDK allows you
to define pipeline parameters.
In many respects, pipeline parameters are similar to pipeline inputs. But pipeline parameters
are handled separately and treated specially. Unlike input variables, pipeline parameters must
have a default value, which the parameter takes initially after deployment. Therefore, a
pipeline parameter's value is always defined.
Depending on the configuration, a pipeline parameter might be changed interactively
through the user interface of AI Inference Server or also be connected to an MQTT topic like
an input variable. In the latter case, the pipeline can receive parameter updates from other
system components through the External Databus.
The parameters defined for a pipeline apply to all components. This means that in a pipeline,
all components with parameters must be ready to receive parameter updates. A pipeline
component only needs to respect updates concerning parameters relevant for the given
component.
For details on how to define pipeline parameters and how to handle parameter updates in
the pipeline components, refer to Guideline for writing pipeline components (Page 25). For a
complete code example that shows how to define and use pipeline parameters, see the State
Identifier project template.

Parallel execution
By default pipeline components process the inputs sequentially, within the same Python
interpreter context. To increase the throughput of the component, you can instruct AI
Inference Server to run multiple instances of a pipeline component and distribute the inputs
among them. This way you can exploit the parallelism available in most multi-core CPUs.
If you specify parallel component execution, every instance is initialized separately and
receives only a fraction of the inputs. Therefore not all components are suitable for parallel
execution.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 19
Using AI Software Development Kit
4.4 Test the pipeline configuration package locally

For example, the single component of the pipeline given in project template State Identifier
cannot be executed by parallel instances because the component must process inputs
sequentially, one by one in order to form windows from the data.
Theoretically, you could separate State Identifier into two components, the first component
forming the windows and the second component calculating the features and the prediction.
Then, the second component could be run in multiple parallel instances, as it does not have
to remember previous inputs to calculate the output. (It is another matter if this complexity is
worth the effort in a given use case.)
In contrast, the component given in project template Image Classification can be executed by
parallel instances out-of-the-box. It is practically stateless, as the key global state the
component uses is the model loaded during initialization. Otherwise, the component only
needs the current input to calculate the output.
Please note that with parallel component execution, there is no guarantee that the outputs
are produced in the same order as the corresponding inputs arrive. It might happen that one
instance overtakes another even if the raw CPU time required is roughly the same for all
inputs, as the component instances are competing for CPU cores with other applications
running on the Industrial Edge device.
You can predefine the number of parallel component instances using AI SDK function
PythonCompoent.set_parallel_steps(). This setting can be overridden on the user
interface of AI Inference Server.

4.4 Test the pipeline configuration package locally

Once you have created your pipeline configuration package, you can proceed directly to the
conversion to an Edge configuration package and deployment to the AI Inference Server.
However, we strongly recommend that you test the pipeline configuration package
before converting and deploying it.
The advantages of local testing are the following:
• You can identify many potential issues more quickly because you need not to go through
a deployment cycle.
• You can diagnose and troubleshoot problems much more easily because you can inspect
artifacts in your development environment.
• You can validate your fixes faster and move on to other issues that have not been able to
arise due to previous issues.
• You can easily include the local pipeline tests in the test automation of your build process.

Two tools for local testing

You can apply state-of-the-art software engineering practices such as unit testing and test-
driven development.
This means that ideally, you already have automated unit or even integration tests that
ensure that the Python code and stored models work in isolation as expected. This helps you
localize errors when you assemble these parts and integrate them as a pipeline configuration
package.

AI Software Development Kit

20 Operating Manual, 04/2023, A5E52031285-AE
 Using AI Software Development Kit
4.4 Test the pipeline configuration package locally

The AI SDK package simaticai.testing provides two tools for local testing:
• A pipeline validator that performs static validation of the package for the availability of
required Python packages.
• A pipeline runner that allows you to simulate the execution of your pipeline in your
Python environment.
Note, that all of these testing features apply to pipeline configuration packages, not Edge
configuration packages. You must use it before you convert your pipeline configuration
package to an Edge configuration package using AI SDK.
Since the conversion itself is done automatically by AI Model Deployer, most of the potential
issues are already present in the package before the conversion, so a post-conversion
verification would only delay the identification of these issues.

Static validation of a pipeline package

You can pass your pipeline configuration package to the
validate_pipeline_dependencies function in the
simaticai.testing.pipeline_validator submodule to perform static checks. These
checks include:
• Verify that the Python version required in the package is supported by a known version of
AI Inference Server.
• Verify that all required Python packages are either included in the pipeline package itself
or available on pypi.org for the target platform.
For specific programming details, see the AI SDK API reference manual.

Local execution of a packaged pipeline

The LocalPipelineRunner class in the simaticai.testing.pipeline_runner
submodule can be used to locally mimic the behavior of the AI Inference Server for loading
and running inference pipelines. This is a quick and easy way to find programming or
configuration errors before deploying the package.
The local pipeline runner simulates the server environment as follows:
1. It unpacks the pipeline components into a test folder, similar to what would happen on the
inference server.
2. It creates a separate Python virtual environment for each component.
3. It installs required Python packages from the wheels, if provided in the package or by
pypi.org.
4. It installs the mock of log_module (see Mocking the logger of AI Inference Server
(Page 24))
5. It updates pipeline parameters if applicable.
6. It feeds the pipeline with input data by triggering the entry points of the components
accordingly.
7. It collects the sequence of pipeline outputs for a given sequence of pipeline inputs.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 21
Using AI Software Development Kit
4.4 Test the pipeline configuration package locally

You can also use the local pipeline runner to run your pipeline component by component.
You can feed individual components with inputs and verify the output produced.
If the pipeline contains parameters, the pipeline uses the default values for the parameters.
You can also change the parameter values using the update_parameters() method. In
this way you can test your pipeline with different parameters.
Note: You can only use update_parameters() method before calling run_component or
run_pipeline(), but you cannot change pipeline parameters while these methods are
running.
From a testing strategy and risk-based testing perspective, we recommend that you validate
the business logic within the pipeline components in unit tests as you would with any
ordinary Python program and use the local pipeline runner to cover test risks such as the
following:
• Mismatch between pipeline and component input and output variable names
• Required Python packages not covered by requirements.txt
• Source or other files are missing from the package
• Interface mismatch between subsequent pipeline components
• The entry point cannot process input data due to a mismatch in the data format
• Entry point generates output data in the wrong format
• For some reason, the pipeline does not work consistently as intended
A crucial point for making the local test faithful concerning data input and output formats is
to understand how data connections work in AI Inference Server. The following data
connection types are straightforward:
• Databus
• External Databus
• IE Vision Connector
For these data connection types, AI Inference Server passes the MQTT payload string directly
as the value of the connected pipeline input variable. In many use cases where you use this
data connection type, your pipeline has a single input variable of type string. This means that
you must pass local pipeline runner a Python dictionary with a single element.
For example, if you take the pipeline from the image classification project template, you have
a single input variable vision_payload. To run your pipeline on two consecutive input
images, you must call the pipeline runner as follows:
pipeline_input1 = { 'vision_payload': mqtt_payload1 }
pipeline_input2 = { 'vision_payload': mqtt_payload2 }
pipeline_output = runner.run_pipeline([pipeline_input1,
pipeline_input2])
For a complete code example that shows how to feed a pipeline with a single string input
variable in a local test, see the Local Pipeline Test Notebook in the Image Classification project
template.
The SIMATIC S7 Connector data connection type requires more attention. This connector is
typically used in time series use cases. Using this connection, the AI Inference Server
processes the MQTT payload used by the S7 Connector and only passes on the values of the

AI Software Development Kit

22 Operating Manual, 04/2023, A5E52031285-AE
 Using AI Software Development Kit
4.4 Test the pipeline configuration package locally

PLC variables, but not the metadata. So, if you intend to use your pipeline with the S7
connector, you need to feed it with dictionaries holding the PLC tag values.
Taking the pipeline from the State Identifier project template for example, you have input
variables ph1, ph2 and ph3, which are meant to be used with the SIMATIC S7 Connector data
connection type. To mimic how AI Inference Server feeds the pipeline, you must call the
pipeline runner like this:
pipeline_input1 = {'ph1': 4732.89, 'ph2': 4654.44, 'ph3': 4835.02}
pipeline_input2 = {'ph1': 4909.13, 'ph2': 4775.16, 'ph3': 4996.67}
pipeline_output = runner.run_pipeline([pipeline_input1,
pipeline_input2])
For a complete code example that shows how to feed a pipeline with an input line of PLC tag
values in a local test, see the Local Pipeline Test Notebook in the State Identifier project
template.

Restrictions of local pipeline execution

The local runner works with batches of input data and processes the whole input batch
component by component. In the case of a sequence of pipeline inputs, the entire sequence
is first processed by the first components, and only then the output of the first component is
processed by the second component.
This is different from the runtime environment on the AI Inference Server, where the
components in the pipeline potentially start consuming input as soon as the preceding
component has produced output.
You cannot fully test input and output data formats, as these depend on the data connection
settings in AI Inference Server, and you must provide the local runner the input data in the
representation that matches the output side of the connector. This means that if your
assumptions on the data connection settings or the resulting data formats are wrong, your
tests will provide misleading results, too. The local runner can only simulate linear pipelines,
where the pipeline input variables are only used by one component, each component uses
only the outputs of the previous components, and the pipeline output only consists of
variables from the last component.
Furthermore, the results obtained in local tests are not fully representative for AI Inference
Server, including but not limited to the following aspects:
• The local version of Python may be different from that in the Inference Server.
• The local architecture may be different, resulting in different builds of imported Python
packages being used.
• The local runner executes only one instance of the Python code, regardless of the
parallelism setting in the configuration.
Despite all these limitations, we recommend testing your pipeline locally before deployment.
Most likely, this will save you more time than skipping this step.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 23
Using AI Software Development Kit
4.5 Mocking the logger of AI Inference Server

4.5 Mocking the logger of AI Inference Server

The Python environment on AI Inference Server injects a Python module named
log_module, which the Python scripts can use for logging on the server. To be able to run
the same code in a local development environment on a PC, the AI SDK provides a mock of
log_module in a wheel, which you can install, import and use in the same way. This wheel
file must not be included in the pipeline package's dependencies.

4.6 Deploy the packaged inference pipeline for AI@Edge

After you create and test your pipeline configuration package, you have to convert it to an
edge configuration package before you deploy it to AI Inference Server. At the time of
writing, you must:
1. Use AI SDK to convert the pipeline configuration package created by the AI SDK to an edge
configuration package.
2. Deploy the Edge configuration package to an industrial Edge device by uploading it using
the AI Inference Server application.
In the future, there will be more options to perform the required conversion and deployment.

The reason for the required conversion is that while a pipeline configuration package fully
defines the inputs, outputs, and inner workings of an inference pipeline, it does not contain
all the components required to run in AI Inference Server. To make it complete for
deployment on AI Inference Server, the pipeline configuration package must be converted to
an edge configuration package.
Among other things, the conversion ensures that all the necessary Python packages for the
target platform are included. If any of the required packages, including transitive
dependencies, are not included in the pipeline configuration package for the target platform,
they will be downloaded from pypi.org.
The conversion function is available in AI SDK both as Python function and CLI command.
Please refer to the details of function convert_package in module
simaticai.deployment in the SDK API reference manual.

4.7 Create delta package and deploy to AI@Edge

The time of deployment strongly correlates with the size of edge configuration package. To
reduce the deployment time, the AI SDK provides the functionality to create a delta pipeline
package. A delta package contains the files which are updated or newly added compared to
the original version.
You can use function create_delta_package in module simaticai.deployment or
the corresponding CLI command. For more details, refer to the SDK API reference manual.
Note: You can deploy the delta configuration package in the same way as deploying Edge
configuration package. The original Edge configuration package must be deployed before the
delta configuration package.

AI Software Development Kit

24 Operating Manual, 04/2023, A5E52031285-AE
Guideline for writing pipeline components 5
AI Inference Server is an Industrial Edge application designed to execute your ML models on
an Industrial Edge device. The interface between your ML model and AI Inference Server
is a Python script that consumes incoming data, processes it and creates an output response.
This guideline walks you through the workflow for defining a pipeline component using
a Python script.

5.1 Component definition

In this context, a component means a pipeline step which consumes input data, processes
that data by using a model, and produces the output data. The model can mean, in a
narrower sense, an ML model such as a neural network or a random forest, or simply an
aggregator or other pre- or postprocessing logic. In every case, a Python script, hereinafter
referred to as the 'entrypoint', establishes the connection between the model and the
AI Inference Server. To make it work, the server needs to know what Python environment is
required to execute the code, including the required Python packages or file resources.

Essential information for execution of the code

The most essential information for execution of the code is:
• The Python script to which input data is passed
• The Python version required to run the Python script
• The input and output variables of the component

Example
The following code shows how to define component settings. The created configuration can
be checked in the pipeline-config.yml which can be found in the in Examples (Page 39)
section. Please note that this code is only used to create the pipeline configuration package,
but it is not contained in the package itself.
# create_pipeline_config_package.py
from simaticai import deployment

# defining basic properties of the pipeline component

# AI Inference Server version 1.4 supports Python 3.8
component = deployment.PythonComponent(name='classifier',
version='1.0.0', python_version='3.8')

# defining entrypoint python script

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 25
Guideline for writing pipeline components
5.2 The entrypoint

component.add_resources("../src","entrypoint.py")
component.set_entrypoint('entrypoint.py')

# defining input variable of component

component.add_input(name='input_1',_type='Double')
component.add_input(name='input_2',_type='Double')

# defining output variable of component

component.add_output(name='class_label',_type='Integer')
component.add_output(name='confidence',_type='Double')

In this example the code uses a pre-trained scikit-learn model that is stored in a joblib file.
The file acts as a resource file in this model. For more details about resource files, see the File
resources section. The code also uses external Python modules (Page 30) that must be
deployed and installed on the AI Inference Server.
# adding stored scikit-learn model as a resource file
component.add_resources('..','models/classifier-model.joblib')

# adding python dependency scikit-learn with version==1.0.1

component.add_dependencies([('scikit-learn','1.0.1')])

With this configuration, AI Inference Server collects data for input_1 and input_2. When
the data is ready, the server wraps it into a data payload and calls the process_input()
function from entrypoint.py. Once the data is processed and the class_label and
confidence results are calculated, the function generates a return value.

5.2 The entrypoint

AI Inference Server itself receives the data payload from the input data connection. With each
input, AI Inference Server triggers the process_input(data: dict) -> dict function
in the entrypoint module. After process_input() returns, the server forwards the output
to the next pipeline component, or emits it as pipeline output over the output data
connection.

Example
# entrypoint.py
import sys
from pathlib import Path

AI Software Development Kit

26 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.3 Input data

# when you import from source, the parent folder of the module ('./src') must be added to the
system path
sys.path.insert(0, str(Path('./src').resolve()))
from my_module import data_processor # should be adapted to your code

def process_input(data: dict) -> dict:

return data_processor.process_data(data["input_1"], data["input_2"])
In this case, it is assumed that business logic is encapsulated in
data_processor.process_data(). You can place the code into your package and
modify only the reference to your data processor.

5.3 Input data

AI Inference Server wraps the acquired input values into a dictionary and passes them to
process_input() as a single parameter. Each input variable is represented as a separate
element in the dictionary, for example:
{"input_1": 123.51, "input_2": 47.02}'
If you have multiple inputs, the process_input() might be triggered with incompleted
data. If inter signal alignment is enabled, the missing input variables are None in the
dictionary.
{"input_1": 123.51, "input_2": None}
Without inter signal alignment, inputs are passed to process_input() one by one, variable
by variable, so that the dictionary contains only one element for each call.
{"input_1": 123.51}
For further details, see Processing time series of signals (Page 36).

5.3.1 Variable types

Let's check again how we defined the input variable input_1
# defining input variable
component.add_input(name= 'input_1', _type='Double')
We defined it as Double, which is a data type of AI Inference Server. The
process_input() function receives the inputs converted to a Python data type, which is
float for AI Inference Server type Double.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 27
Guideline for writing pipeline components
5.3 Input data

In general, you need to define the types of input and output variables as AI Inference Server
types, but the Python script should use the appropriate Python type. The correspondence
between AI Inference Server data types and Python data types is displayed in the following
table. The table below also shows which data type is supported by which data connection in
AI Inference Server version 1.4.

AI Inference Python Databus S7 Connector Vision ZMQ

Server Connector
Bool bool I/O
Integer int I/O
Double float I/O
String str I/O I/O input
Object dict input output

External Databus connections support the same data types as Databus.

5.3.2 Restrictions on type Object

AI Inference Server version 1.4 imposes restrictions on the dictionaries returned by the
inference wrappers for output variables of type Object. The returned dictionary is required
to hold a metadata string and a binary sequence. The metadata and the binary sequence can
have an arbitrary key in the dict, but must be of type str and bytes respectively. See
details in section Returning the result (Page 32).

Note
The structure of dictionaries received as pipeline input differs from the dictionary structure
required as component output. See details in section Processing images (Page 35).

5.3.3 Custom data formats

To connect your pipeline to a custom application with its own data format, you can take one
of the following methods:
• Use String and connect input or output through Databus or External Databus. In this
case, you can use arbitrary textual data formats, for example, JSON, XML, CSV or any
combination of them.
• Use Object and connect output through ZMQ. In this case, AI Inference Server converts
the metadata dictionary to a JSON string and passes it to the receiver along with the
binary contents in a ZMQ multipart message. Refer to the AI Inference Server User Manual
for more details.

AI Software Development Kit

28 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.3 Input data

Specific variable types for images

AI Inference Server supports receiving URL encoded images through MQTT. The payload type
is str and can be extracted into PIL image as follows:
# define input
component.add_input("image", "String")
# extract payload
def process_input(payload: dict):
url_encoded_image = payload["image"]
with urlopen(url_encoded_image) as response:
assert response.headers["Content-type"] in ["image/png",
"image/jpeg"]
image_bytes = response.read()
pil_image =
Image.open(io.BytesIO(image_bytes)).resize(IMAGE_SIZE)
The other supported type for images is Object , which can be used to receive or send
images via ZMQ. When the input variable is defined with type Object, the AI Inference
Server takes the image from ZMQ and creates a specific payload format. In your code, this
format can be processed and extracted into a PIL Image. A concrete code example can be
found in project example Image Classification.
# define input
component.add_input("image", "Object")
# Object input format
payload = { "image":
{
"resolutionWidth": image.width,
"resolutionHeight": image.height,
"mimeType": ["image/raw"],
"dataType": "uint8",
"channelsPerPixel": 3,
"image": _swap_bytes(image.tobytes())
}
}
When the output variable is defined with type Object, the output must be provided in a
specific format. In your code, a dictionary must be created with a string and a bytes field.
They must contain the width and height information in a JSON string and the UINT8 bytes of
the raw image. A concrete code example can be found in project example Image
Classification.
# define output
component.add_input("image_with_filter", "Obejct")
# Object output format
"image_with_filter": {
"metadata": json.dumps( {
"resolutionWidth": image.width,
"resolutionHeight": image.height
}
),
"bytes": image.tobytes()
}

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 29
Guideline for writing pipeline components
5.4 Processing data

5.4 Processing data

The main part of your Python script is the logic of calculating the output from the input. This
is done by your Python code which can use config files and persisted ML models in a well-
defined Python environment. To define these resources, the PythonComponent class of the
simaticai.deployment module is featured to add dependencies or extra files. These files
are wrapped in the configuration package and extracted on the AI Inference Server as
follows:
• The dependencies are installed on the server via pip.
• The extra files will be copied into the component directory.

5.5 Python dependencies

The AI Inference Server executes every component of a pipeline in an isolated Python virtual
environment. For each component, you must specify which Python packages are required by
the Python scripts in that component, including the Python packages required to load
persisted Python objects.

Adding Python dependencies

The Python dependencies of a component can be added in a variety of ways:
Using a standard requirements.txt
In this way, the component.dependencies dictionary contains only the dependencies
defined in the file. The required wheels are downloaded in the course of converting the
pipeline configuration package to an edge configuration package using pip.
component.set_requirements('./requirements.txt')
As a standard wheel file or a zip file that contains standard wheel files
In both cases, the packages are added to the component.dependencies dictionary and
binaries are zipped into the configuration package.
component.add_python_packages('../packages/my_module-0.0.1-py3-any-
any.whl')
component.add_python_packages('../packages/MyPackages.zip')
By name using a list that contains the names of the Python modules
In this case, the method looks for the module in the current Python environment and adds
the package with its version and all of its transitive dependencies.
component.add_dependencies(['numpy','scikit-learn'])
The dependencies added to a component are installed on the AI Inference Server with the
defined version and can be imported into your Python code during execution.
# entrypoint.py or data_processor.py
import numpy as np
import pandas as pd
[...]

AI Software Development Kit

30 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.6 File resources

5.6 File resources

File resources can be any type of files required to execute the Python code, including the
Python sources themselves, such as configuration file, static data, or trained AI models stored
in joblib or pickle format.

Adding resources
For the configuration package to transport these files to the server environment, you must
specify them using the add_resources(base_dir, resources) method, as shown
below:

# the method adds 'prediction_model.joblib' from the '../models' directory file to the
component
# and the file will be extracted on the server into the component folder under 'models'
directory
component.add_resources(base_dir="..",
resources="models/prediction_model.joblib")
# same way we define a file 'model-config.yml' to bring into 'config' directory
component.add_resources(base_dir="..", resources="config/model-
config.yml")

Once the pipeline is imported into AI Inference Server and the component is installed, the
files in the server's file system are available in the component directory and can be accessed
by the Python scripts:
# data_processor.py
import yaml
import joblib
from pathlib import Path
# Our goal is to have identical relative path to the resources in the source repository and on
the server.
base_dir = Path(__file__).parents[1]
# file 'model-config.yml' is extracted into 'config' directory
config_path = base_dir / "config/model-config.yml"
model_config = yaml.load(config_path)
# file 'prediction_model.joblib' is extracted into 'models' directory
model_path = base_dir / "models/prediction_model.joblib"
with open(model_path, "rb") as model_file:
model = joblib.load(model_file)
As loading files can be time consuming, it is recommended to load files and ML models into
memory at initialization time of your Python code and not during the call to

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 31
Guideline for writing pipeline components
5.7 Returning the result

process_input(). The entrypoint process_input() should focus on processing the

incoming data as quickly as possible. We highly suggest you initialize the objects that are
used in this code at the beginning of the script, and then use them in the functions invoked
from process_input.
Please be aware that this approach can cause a huge memory load, so you have to make a
trade-off between memory consumption or CPU load and response time.
After loaded, the model can be used to process input data. For simple cases, this could be
directly in the entrypoint script. In the given example, we have factored this out into an own
module to illustrate how another module can be called from the entrypoint.
# data_processor.py
def process_data(width,height):
data=[width,height]
class_label,confidence=model.predict(data)
return{"class_label":class_label,"confidence":confidence}

5.7 Returning the result

If you want to return the results after the input data are processed, you must return them in a
dictionary. The keys should be the variable names of the component's outputs. In the
example in File resources (Page 31), function process_data() returns such a dictionary,
which can be directly returned from process_inputs() as well. The dictionary contains
an integer for class_label and a float that represents the confidence in the prediction.
If there is no output for a given invocation of process_input(), you should return None.
As a result, AI Inference Server will not trigger the next component in the pipeline, or if this is
the last component of the pipeline, the pipeline will emit no output from this component.
See Use cases (Page 35) for a concrete use case.
Returning Object
AI Inference Server version 1.4 restricts the type of dictionary that a pipeline component can
return in an output variable of type Object. The dictionary must hold a metadata string and
a binary sequence. The metadata and the binary sequence can have an arbitrary key in the
dict, but must be of type str and bytes respectively. In other words, any dictionary with
two elements will work if one of the elements is a str and the other is bytes.
For example, if you want to pass a processed version of the input image to a later component
or to ZMQ, you can do it as follows:
# define output
component.add_output("processed_image", "Object")
# Object output format
"processed_image": {
'metadata': json.dumps( {
"mode": image.mode,
"width": image.width,
"height": image.height
}
),
'bytes': image.tobytes()
}

AI Software Development Kit

32 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.8 Adding custom metrics

Note
You cannot pass on a dictionary received as pipeline input as a component output because
the structure of these dictionaries is different.

In the receiver pipeline component, you can decode the image as follows:
# define input
component.add_input("processed_image", "Object")
# construct PIL Object from metadata and binary data
def process_input(data: dict):
metadata = json.loads(data['processed_image']['metadata'])
image_data = data['processed_image']['bytes']
mode = metadata['mode']
width = metadata['width']
height = metadata['height']
image = Image.frombytes(mode, (width, height), image_data)

5.8 Adding custom metrics

You can implement arbitrary model metrics as component metrics which you can use to
assess the performance of the model on AI Model Monitor. The pipeline produces the metrics
automatically as outputs, which are automatically mapped to the required Databus topics. On
AI Inference Server you only need to select Databus as data connection for these metric
outputs.
A custom metric has to be defined on the component as follows:
component.add_metric("probability")
An output can be returned with the same name from the inference wrapper as follows:
def process_input(data: dict):

prediction, metric_value = predict(data)

return {
"prediction": prediction,
"metric_name": json.dumps({"values": metric_value}),
}
Once the pipeline is created, it will collect the metrics from all components and provide as
pipeline outputs, so the AI Inference Server can treat it as an output. The custom metric will
be visible on AI Inference Server as an output with a preconfigured topic which needs to be
connected to Databus.
Please note that you can also add custom metrics to a monitoring component provided by the
AI SDK Monitoring Extension. Refer to the AI SDK Monitoring Extension User Manual for more
details.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 33
Guideline for writing pipeline components
5.9 Pipeline parameters

5.9 Pipeline parameters

If a component is used in a pipeline with parameters, the component must provide a function
update_parameters() to handle parameter updates. AI Inference Server calls
update_parameters() at least once after the pipeline starts but before the first input is
passed to the component. update_parameters() might be called if pipeline parameters are
changed while the pipeline is running. AI Inference Server guarantees that calls to
update_parameters() and process_input() do not occur concurrently.

Parameters to be changed on the user interface

If you intend to let the operator changes pipeline parameters interactively on the user
interface of AI Inference Server, Siemens recommends you to use individual parameters. You
must define the name, default value and type for each parameter.
# define individual parameters for the pipeline
pipeline.add_parameter('windowSize', 300, 'Integer')
pipeline.add_parameter('windowStepSize', 75, 'Integer')
In this way, AI Inference Server provides individual entry fields on its user interface for each
parameter. In the handler function, the parameters can be retrieved from the function
parameters as individual dictionary items.
# entrypoint.py
def update_parameters(params: dict):
windowSize = params['windowSize']
windowStepSize = params['windowStepSize']

Parameters to be changed via MQTT

If you need to change pipeline parameters programmatically by sending MQTT messages,
Siemens recommends to you use a single compound parameter of type String and combine
multiple parameters in JSON. To let AI Inference Server supply parameter values from an
MQTT topic, you must enable this function by passing True as an additional, fourth
argument.
# define compound JSON parameter for the pipeline
pipeline.add_parameter('windowing', json.dumps({'windowSize': 300,
'stepSize': 75}), 'String', True)

As a result, AI Inference Server allows you to map this parameter to an MQTT topic, which is
similar to the method how input and output variables are mapped. In the handler function,
the parameters can be retrieved by first unfolding the JSON in the single formal pipeline
parameter into a dictionary, after which they can be accessed individually.
# entrypoint.py
def update_parameters(params: dict):
windowing = json.loads(params['windowing'])
windowSize = windowing['windowSize']
windowStepSize = windowing['stepSize']

AI Software Development Kit

34 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.10 Use cases

5.10 Use cases

The following sections provide you guidance on how to use inputs and outputs as intended in
selected typical ML use cases.

5.10.1 Processing images

AI Inference Server version 1.4 supports image input from the Vision Connector application.
The connection can be through Databus or ZMQ. Depending on the connection, the images
are either represented as input Strings or as input Objects.

Using ZMQ connection

For use cases with large images or high input rate, we recommend you use a ZMQ
connection, as the Databus is not designed to handle large quantities of data. For image
input through ZMQ, you have to define the input variable as Object as follows:
# define input
component.add_input('vision_payload', 'Object')
Currently, the MIME type, the data type and the number of channels is restricted to the
values defined in the example. AI Inference Server converts the Vision Connector payload
received through ZMQ into a Python image data dictionary with entries as follows:
# Object input format
image_data = { 'image':
{
'resolutionWidth': 640,
'resolutionHeight': 480,
'mimeType': 'image/raw',
'dataType': 'uint8',
'channelsPerPixel': 3,
'image': 'placeholder for binary image contents'
}
}
You can extract the binary image into a PIL Image as follows:
from PIL import Image

def process_input(data: dict):

image_data = data['vision_payload']

assert image_data['dataType'] == 'uint8' and

image_data['channelsPerPixel'] == 3

width = image_data['resolutionWidth']
height = image_data['resolutionHeight']
# image received with 'BGR' byte order
return Image.frombytes('RGB', (width, height), image_data['image'],
'raw', 'BGR')

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 35
Guideline for writing pipeline components
5.10 Use cases

Using Databus connection

For use cases with small images and low input rate, you can use Databus. In this case, you
have to define the type of the component input variable for passing the image as String,
which corresponds to a Python str in the input dictionary.
# define input
component.add_input('vision_payload', 'String')
AI Inference Server passes through the payload provided by the Vision Connector directly as a
string. This string is a JSON, which contains metadata and the image itself in data URL
encoded form as follows. Refer to the Vision Connector User Manual for more details.
# example image in Vision Connector MQTT JSON format
{
'timestamp': '2022-02-23T09:29:45.276338',
'sensor_id': 'a204dba4-274e-43ce-9a71-55de9e715e72',
'image':
'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAASwAAAGQCAIAAACbF8osA
A...QmCC', # note this URL encoded string is truncated
'status':
{
'genicam_signal': {'code': 3}
}
}
You can extract the URL encoded image into a PIL Image object as follows:
# extract payload
from urllib.request import urlopen
from PIL import Image
def process_input(data: dict):
payload = json.loads(data['vision_payload'])
url_encoded_image = payload['image']
with urlopen(url_encoded_image) as response:
assert response.headers['Content-type'] in ['image/png',
'image/jpeg']
image_bytes = response.read()
return Image.open(io.BytesIO(image_bytes))
Refer to Image Classification project template for a complete example for MQTT and ZMQ
connection variants.

5.10.2 Processing time series of signals

In time series use cases, the situation is typically as follows. You have a number of signals,
that you want to sample at a regular rate and feed your ML model with a time window of
multiple samples at once. For example, your model might be designed to expect a window of
5 samples for 3 variables.

Time stamp var1 var2 var3

09:50:23 1.2 202 25
09:50:28 1.3 230 5
09:50:33 1.2 244 18

AI Software Development Kit

36 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.10 Use cases

Time stamp var1 var2 var3

09:50:38 1.2 244 18
09:50433 1.2 244 18

Ideally, each signal is a variable that is read by the Industrial Edge data bus, as configured in
AI Inference Server. Often, the signals from PLCs are captured using the Industrial Edge S7
Connector, which samples these signals from given PLC tags and makes them available on
the data bus.
Unfortunately, data points for the signals do not necessarily arrive at a regular rate or
synchronously. So, by default, your Python script is usually called with a single variable filled
in and the others are None, which is not good for an ML model that expects an entire matrix
of multiple variables in an entire time window.

Time stamp var1 var2 var3

09:50:28 None 230 None

To ensure the regular rate and the synchronicity of inputs, AI Inference Server supports inter
signal alignment. You can specify a time interval and receive the inputs for all variables
stimulated in that interval. In our example, you would specify 5 seconds as time interval and
receive inputs like:

Time stamp var1 var2 var3

09:50:28 1.3 230 5

As there is no guarantee that the data source delivers a data point in each interval, there is
still a possibility that some values are missing and set to None in an input row. However, if
the sample rate of inter signal alignment does not exceed the data rate of the sources, your
Python script will mostly be passed complete rows of data.
For details on inter signal alignment, refer to the user manual of AI Inference Server user's
manual.
See the packaging notebook in project template State Identifier and the AI SDK API reference
to specify inter signal alignment to be applied to the input of an ML pipeline when it is
packaged for deployment to the AI Inference Server.
However, rows are not yet enough to feed time series ML models when they need data
windows consisting of multiple rows. Since the AI Inference Server does not support
accumulating windows, the Python script must take over this task. The key point of the
server's script interface here is that not all inputs in your Python script results in an output,
because the ML model can only produce output if the received input has just completed a
data window that can be passed to the model to calculate an output.
As described earlier in Returning the result (Page 32), the script can return None while it
accumulates input, and the model cannot calculate a value for the output. For reasons of
compactness, the following diagram shows this for a window size of two.

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 37
Guideline for writing pipeline components
5.10 Use cases

In real life, this can be even more complex, depending on whether the windows are
overlapping or disjoint. For a concrete implementation of such accumulation logic, see the
Python script provided in the State Identifier project template.
Please note that you cannot use parallel component execution if your component relies on
building up windows from subsequent data points, as the data points would be distributed to
different instances of the component. You can, however, separate the aggregation of data
windows and CPU intensive processing of the windows into an own component each, and
enable parallel execution for the latter only.
Please also note, however, that even in this case there is no guarantee that the processing of
the windows finishes in the original sequence of the data. If that is essential, you should
supply the windows with a sequence id in the aggregating component, which you pass on to
the output of the processing component. That way the consumer of the pipeline can recreate
the original sequence.

5.10.3 Processing batch data

In batch use cases, such as classification of discretely manufactured items, the input data is
often available in a single packet on the data bus in a text representation like a JSON
structure or CSV table. For example, data is provided through External Databus as a textual
payload. The Industrial Edge Vision Connector also delivers images embedded in a JSON
through Databus.
Such data is treated as a single variable of type String, which is passed then to
process_input() as a dictionary with a single element. You need to process this input
string according to which representation it uses, for example, json.loads() for JSON or a
combination of splitlines() and csv.reader() for CSV.
For a concrete code example that shows how to process a single input variable with a JSON
structure see the Image Classification project template.

AI Software Development Kit

38 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.11 Examples

5.11 Examples

Component definition example

The following script creates an example pipeline with a single component, two inputs and
two outputs.
from simaticai import deployment
component = deployment.PythonComponent(name='classifier',
version='1.0.0', python_version='3.8') # defines basic properties of the
pipeline step
component.add_input(name='input_1', _type='Double') # defines input
variable
component.add_input(name='input_2', _type='Double') # defines input
variable
component.add_output(name='class_label', _type='Integer') # defines
output variable
component.add_output(name='confidence', _type='Double') # defines output
variable
component.add_resources("../src/", "entrypoint.py")
component.set_entrypoint('entrypoint.py')
pipeline = deployment.Pipeline.from_components([component],
name='Example', version='1.0.0')
pipeline_package_path = pipeline.save('../packages')

The above code generates a pipeline-config.yml that contains, among others, the
following:
# pipeline-config.yml
components:
name: classifier
entrypoint: entrypoint.py
version: 1.0.0
runtime:
type: python
version: 3.8
inputType:
- name: input_1
type: Double
- name: input_2
type: Double

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 39
Guideline for writing pipeline components
5.11 Examples

outputType:
- name: class_label
type: Integer
- name: confidence
type: Double

The pipeline looks like this:

Image Classification
The following script creates an image classification pipeline that consists of a single
component. The pipeline processes images embedded in JSON strings and produces a
classification result as a string. This example is detailed in the Image Classification project
template.
from simaticai import deployment
# create pipeline component and define basic properties
component = deployment.PythonComponent(name='inference',
version='1.0.0', python_version='3.8')
component.add_input('vision_payload', 'String') # define single input
variable
component.add_output('prediction', 'String') # define single output variable
component.add_resources('..', 'entrypoint.py') # add Python script
component.set_entrypoint('entrypoint.py') # define above script as entrypoint
component.add_resources('..', 'src/vision_classifier_tflite.py') #
add classifier script used by entrypoint
component.set_requirements("../runtime_requirements_tflite.txt") #
define required Python packages
component.add_resources('..',
'models/classification_mobilnet.tflite') # add saved model used in classifier
component.set_parallel_steps(2) # set the number of parallel executors
# create and save pipeline consisting of single component

AI Software Development Kit

40 Operating Manual, 04/2023, A5E52031285-AE
 Guideline for writing pipeline components
5.11 Examples

pipeline = deployment.Pipeline.from_components([component],
name='Image_TFLite_package', version='1.0.0')
pipeline_package_path = pipeline.save('../packages')
# convert pipeline configuration package to edge configuration package
deployment.convert_package(pipeline_package_path)
The above code generates a pipeline_config.yml that contains, among other things:
dataFlowPipeline:
components:
- entrypoint: ./entrypoint.py
inputType:
- name: vision_payload
type: String
name: inference
outputType:
- name: prediction
type: String
runtime:
type: python
version: '3.8'
version: 1.0.0
pipelineDag:
- source: Databus.vision_payload
target: inference.vision_payload
- source: inference.prediction
target: Databus.prediction
pipelineInputs:
- name: vision_payload
type: String
pipelineOutputs:
- name: prediction
type: String
dataFlowPipelineInfo:
dataFlowPipelineVersion: 1.0.0
projectName: Image_TFLite_package

AI Software Development Kit

Operating Manual, 04/2023, A5E52031285-AE 41
Guideline for writing pipeline components
5.12 Writing components for earlier versions of AI Inference Server

The saved pipeline configuration package contains the files listed below. The main folder
contains the YAML files that describe the pipeline. The inference subfolder contains the
files that belong to this component.
Image_TFLite_package_1.0.0/pipeline_config.yml
Image_TFLite_package_1.0.0/datalink_metadata.yml
Image_TFLite_package_1.0.0/inference/entrypoint.py
Image_TFLite_package_1.0.0/inference/requirements.txt
Image_TFLite_package_1.0.0/inference/src/vision_classifier_tflite.py
Image_TFLite_package_1.0.0/inference/models/classification_mobilnet.
tflite

5.12 Writing components for earlier versions of AI Inference Server

AI Inference Server versions up to 1.1 requires the entrypoint to define a function named as
run() instead of process_input(). Not only the function name is different, the component
inputs and outputs are passed differently too.
AI Inference Server version 1.1 passes the input variables as a JSON string, which you must
convert to a dictionary. On the output side, you must pass the outputs as a JSON string as
well, and embed the whole into a dictionary with a ready flag.
The following code example shows how to wrap process_input() into a run() function
compatible with AI Inference Server 1.1.
# entrypoint.py
...
# compatibility run() wrapper for process_input()
def run(data: str) -> dict:
input_data = json.loads(data)

result = process_input(input_data)

if result is None:
answer = {"ready": False, "output": None}
else:
answer = {"ready": True, "output": json.dumps(result)}
return answer

AI Software Development Kit

42 Operating Manual, 04/2023, A5E52031285-AE