Application note No.

3: using high-level synthesis to

accelerate image processing algorithms
Project objectives
This is the final of three application notes for creating imaging solutions using the 96Boards ON
Semiconductor Dual Camera Mezzanine and Ultra96V2.

This project demonstrates how to leverage Xilinx Vitis HLS to implement image processing algorithms
using C / C++ algorithms.

The expected outcome is a project running on the Avnet Ultra96V2 board with the 96Boards ON
Semiconductor Dual Camera Mezzanine, which includes a simple Vitis HLS block that manipulates
the output video.

This application note is the final of three application notes based on creating imaging solutions using
the 96Boards ON Semiconductor Dual Camera Mezzanine and Ultra96V2, as this project requires the
use of embedded Linux. The project outlined in this application note is created on a Xilinx-supported
64-Bit Linux operating system.

Getting started: requirements

The following hardware is required to follow this series of application notes.

• Ultra96V2
• Ultra96V2 power supply
• 96Boards ON Semiconductor Dual Camera Mezzanine
• SD card
• JTAG / USB adaptor
• Display port capable monitor
• Mini display port cable
• 16 GB SD card

To implement this series of application notes, you need the following Xilinx software installed:

• Vitis 2020.1, includes Vivado 2020.1.

• PetaLinux 2020.1
• Git
• Terminal application

You’ll need a few licenses for this implementation. Bitstreams for the Xilinx ZU3EG device can be
generated license-free as part of the Xilinx Vivado webPACK. However, some of the IP used within
this design requires an additional license. This license is provided with the Ultra96V2 and is marked
“OEM Zynq ZU3 Ultra96 Vivado Design Edition with SDSoC Voucher Pack.” Redeem this license and
install it on the machine you’re using to implement the project. Once it is installed, you can use the
license manager to see the licensed IP, including the On-Screen Display (OSD).

What is Vitis HLS?

Vitis HLS enables the developer to create an IP module to be implemented in the Vivado design using
higher-level languages than Verilog and VHDL. Developers using Vitis HLS leverage the increased
productivity provided by using higher-level languages such as C and C++. To further increase
productivity, several HLS-compatible libraries are provided for a range of applications, such as
quantitative finance, linear Algebra and image processing, to name a few. These domain-specific
libraries are also supported by a range of common libraries.

The benefit of using C/C++ for development of IP is that it enables a much faster development time,
as C/C++ is inherently untimed and results in much faster verification. Synthesis from C / C++ to RTL,
such as Verilog or VHDL, is called high-level synthesis and takes three stages.

pg. 1
 1. Scheduling or assignment of operation in the untimed C/C++ operations to clock cycles
based upon the target device and the required clock period
2. Binding of the design to logic elements available in the target device (for example, BRAM,
DSP elements or LUT and Registers)
3. Control Logic Extraction or implementation of control structures necessary to control the
behavior of the IP block.

To control the module interfacing, throughput and latency performance of the implemented IP
modules in Vivado, the designer can leverage optimization pragmas that control the decisions made
by the HLS engine.

pg. 2
Creating the Vitis HLS IP core
1. Open the Vitis HLS tool and select Create Project.

2. Select a location to save the project and a project name.

3. Leave the Source and test bench file addition empty and click next on both.

pg. 3
4. For the board, select the ultra96V2.

5. Leave the solution configuration unchanged and click Finish.

pg. 4
6. Once the project opens, create two new files under the source tab by right clicking and
selecting New File. Name the first file main.c and the second file main.h. These will be the
files into which we are entering our design.

7. In the main.c file, copy and paste the code below

#include "main.h"
#include "stdint.h"

void video_top(AXI_STREAM& vidip, AXI_STREAM& vidop, ap_uint<1> invert){

#pragma HLS INTERFACE axis port=vidip
#pragma HLS INTERFACE axis port=vidop
AXITYPE dataInA;
AXITYPE dataOutB;
uint16_t pix1, pix2;

while(1){
dataInA = vidip.read();
dataOutB = dataInA; // copy all fields data, dest, keep, tlast, tuser, id, strb,
pix1 = (uint16_t) (dataInA.data >>16); //shift pixels by 16 bit
pix2 = (uint16_t) (dataInA.data);
if (invert ==1){ //do we want to wait for external trigger or free run
dataOutB.data = ((16384 - pix1) <<16) | (16384 - pix2);
}
else {
dataOutB.data = dataInA.data;
}
vidop.write(dataOutB);
}
}

8. Do the same with the code below for the main.h

#include <ap_fixed.h>
#include <ap_axi_sdata.h>
#include "ap_utils.h"
#include "hls_stream.h"

#define WIDTH 32

typedef ap_axiu< WIDTH, 1, 1, 1> AXITYPE;

typedef hls::stream<AXITYPE> AXI_STREAM;
void video_top(AXI_STREAM& vidip, AXI_STREAM& vidop,ap_uint<1> invert);

pg. 5
9. Once the code is entered, the next step is to define the top-level function that will be
synthesised. Open the project settings.

10. Enter the top function name, click browse and find video_top.

11. Run the HLS synthesis to generate the output RTL for export. You can run the synthesis by
clicking the Run Synthesis button on the menu bar.

pg. 6
12. With the synthesized RTL available, we are now ready to export the IP module for use in
Vivado. Click on the export IP button on the menu bar. Select an output location and click OK.

13. Close Vitis HLS.

pg. 7
Updating the Vivado Design
1. Open Vivado and the Ultra96V2 Camera project.

2. Once the project is opened, from the window option, select the IP Catalog.

3. Select the user Repository, right click and select from the menu that appears. Add IP to the
repository. Select the IP module just exported from Vitis HLS.

pg. 8
4. You should now be able to see the HLS module available under the Vivado HLS IP directory
in the IP Catalog. Double click on the IP core and select add to block design.

5. On the block design, you will notice the new IP block has been added. Insert the block
between the MIPI output and the AXI Subset convertor. Add in a constant block and connect
this to the AP_START input and the Invert Input.

6. Rerun the bitstream generation of the design by selecting the generate bitstream option from
the menu bar. When the bit stream is completed, select export hardware design from the file
menu.

pg. 9
7. Select a fixed platform type.

8. Select Include Bitstream.

pg. 10
9. Rename the exported XSA ULTRA96V2.

10. If you see a warning about the XSA already existing, select OK to overwrite it.

11. Close Vivado.

pg. 11
PetaLinux Update
1. Open a terminal window and change directory into the Ultra96V2 Camera MIPI project as
created in Lab2.

2. Enter the command petalinux-config --get-hw-description=/<path to XSA>

3. When the configuration dialog opens, close it without making any changes.

4. Build the petalinux project using the command petalinux-build

5. With the build complete, change directory to the images/linux directory.

6. Run the package command to build a new boot.bin file petalinux-package --boot --fsbl
zynqmp_fsbl.elf --u-boot u-boot.elf --pmufw pmufw.elf --fpga system.bit

7. Copy the image.ub, boot.bin onto the BOOT partition of the SD created in lab 2.

8. Insert the SD card into the assembled Ultra96V2 with the 96Boards ON Semiconductor Dual
Camera Mezzanine attached as well as the USB UART adaptor.

pg. 12
 9. Power on the Ultra96V2 and tune the command once logged in (root/root) run_1920_1080

10. Note the different image representation on the Display Port Monitor.

<image to be provided>

Conclusion
This project demonstrates how HLS can be used to create image processing IP, which can be easily
inserted into the Vivado Video Stream. While this example is simple, it provides the user with an
easily expandable framework that can be used to further experiment and implement embedded vision
processing algorithms.

pg. 13