OpTiMSoC

OpTiMSoC Enabled Demonstrator System with Fault-Tolerant NoC

Tue, 26 Jan 2021 00:00:00 +0000

Although it has been silent around OpTiMSoC in recent months, some development is still ongoing. Based on OpTiMSoC, a demonstrator system has been developed in the scope of the ARAMiS II research project and has since been further improved. This blog post describes the demonstrator system, its architecture and building blocks, and the alterations and extensions made to OpTiMSoC in order to build it.

Purpose of the demonstrator

The demonstrator was implemented to demonstrate the capabilities of a newly developed Network on Chip (NoC). This network is a hybrid packet-switched and Time-Division-Multiplexed (TDM) NoC using TDM channels with protection switching to implement fault-tolerance and hard real-time guarantees for safety-critical traffic in a mixed-critical environment.

The goal of the demonstrator is to:

demonstrate fault-tolerance of critical communication in the NoC
demonstrate isolation between critical and non-critical traffic in the NOC
visualize the current status of the NoC
allow interaction with the system and the applications running

Overview

The following picture gives an overview of the different parts of the demonstrator.

The demonstrator has two main parts: an OpTiMSoC-based tiled 3x3 many-core system implemented on FPGA and a “Host PC” to control the system and visualize its current state. The 3x3 system is implemented on a VCU108 Evaluation Board from Xilinx and connected to the Host PC via the Cypress EZ-USB FX3 peripheral controller.

A special bridge module connects the debug system to the NoC and acts as I/O tile to allow communication between the Host PC and the other tiles. The remaining 8 tiles are compute tiles that each contain an OpenRISC CPU. Three of these tiles run a pedestrian detection algorithm to represent a safety critical application, the remaining five tiles run an application that produces uniform random background traffic in the NoC. Furthermore, in order to demonstrate a globally asynchronous locally synchronous (GALS) architecture, the critical tiles, the non-critical tiles, and the NoC are all in different clock domains.

The application on the Host PC is split in two major parts: a control part to handle the communication with the target, load the applications into the local memory of the compute tiles, and handle the frame distribution for the pedestrian detection, and a monitoring part to observe and interact with the NoC. The control part is written in Python and makes heavy use of the glip Python bindings, the monitoring part is displayed in a browser using a local Flask server.

The video shows the demonstrator in action and the necessary steps to get it running.

OpTiMSoC extensions/alterations

Several new hardware modules were implemented and some existing OpTiMSoC modules altered for the demonstrator system. Most notably:

A completely new NoC has been implemented. The NoC uses TDM for critical traffic and packet-switching for non-critical traffic. The packet-switched part can use either source-routing or distributed routing (defined at design time). 1+1 protection switching can be used to guarantee fault-tolerance for critical communication.
An I/O tile allowing communication between a Host PC and the compute tiles of a system via the NoC.
Modules for fault-detection of NoC flits based on parity, as well as modules to generate the parity information for NoC flits.
Fault injection modules to test the fault-tolerance of the NoC.
A monitoring module to trace the utilization of individual NoC links for both TDM and packet-switched traffic.
Clock-domain-crossing capabilities between the NoC, the compute tiles, and the debug interconnect as well as for the reset signals.

So, what’s next?

As of now, the new and altered modules are not available in the public OpTiMSoC repository as some of them are specific to the demonstrator system and some are still being worked on. However, we plan to add the hybrid NoC, the I/O tile, and the support for different clock domains and will hopefully have a new release this year.

OpTiMSoC wins the Eurolab4HPC Open Source Project Award!

Sun, 16 Jun 2019 00:00:00 +0000

How exciting! At the Week of Open Source Hardware (WOSH) in Zürich, OpTiMSoC co-won the Eurolab4HPC Open Source Project Award! We’re very excited and honored to see the efforts we put into OpTiMSoC being more widely recognized. Thanks a lot to the sponsors of this award!

The other prices went to Nyuzi, a processor for highly parallel and GPGPU applications, and FuseSoC, a hardware package manager and build tool which we use in OpTiMSoC!

You can find OpTiMSoC, Nyuzi, FuseSoC, and many other great projects, listed at the Open Source contributions page.

2018.1 Release: Linux, Debugging, Automation, and Tons of Fixes

Thu, 20 Dec 2018 00:00:00 +0000

Woohoo! After more than two years of work and 479 git commits later we are very proud to present the all-new 2018.1 release of OpTiMSoC! A look at the statistics gives a first impression of how large this release is: diffstat tells us about 973 files changed, 133,697 lines inserted and 58,806 lines deleted. Or in other words, the code size increased by 74,891 lines! How do those lines of code translate into functionality, you may ask? Let’s have a closer look.

Debug Infrastructure

The debug infrastructure of OpTiMSoC is essential in getting code running on the system, in debugging it, and in performing measurements. Over the last year we have fully re-implemented the host software of the debug system as part of Open SoC Debug (OSD), and imported it back into OpTiMSoC. Together with this rewrite we extended and improved the OSD specification document to be even more complete, flexible, and robust.

Other changes:

A rock-solid off-chip interface is key: if developers cannot reliably access the SoC frustration levels can rise quickly. To avoid that, Max significantly improved the UART and USB3 (Cypress FX3) backends of GLIP, which provide the off-chip interface for OSD and consequently for OpTiMSoC.
The implementation of the off-chip interfaces are now free of vendor primitives, making the porting between FPGAs easier than ever.
New and improved Python bindings make it easy to interact with an OpTiMSoC system in an automated way.
Last but not least, a new implementation of the UART Device Emulation Module within Open SoC Debug by Thomas Leyk was the last remaining blocker to get Linux up and running on OpTiMSoC.

Refreshed Network on Chip (NoC) Implementation

In previous generations of OpTiMSoC we relied on a NoC implementation called LISNoC. Developed as part of a research project a couple years ago it has shown its age in various places. Stefan took up the job of reimplementing significant parts of it, leading to a design which is easier to understand and more flexible as a result.

Testing and Automation

OpTiMSoC is complex: hardware, software, tooling, all need to work together to form a feature-rich System on Chip. To ensure that we don’t accidentally break functionality we have added more tests on various levels. The tests can be executed manually by every developer, and are executed automatically: some on every check-in, some once a day.

Static analysis helps to catch bugs early. For hardware, we are making use of Spyglass Lint to check the code for common errors.
To test and verify the functionality of our hardware modules, we have added more cocotb tests to the tree.
All software imported from Open SoC Debug is tested extensively with unit tests and static analysis tools to avoid regressions and to catch bugs like memory leaks early. Look at the configuration in the opensocdebug/osd-sw repository for more details.
All steps in the tutorial within the user guide are now covered by automated tests. Even the tutorial showing how to build and flash Linux on an OpTiMSoC system running on an FPGA is fully performed and checked as part of a test. This allows us to say: our tutorials always work!
Finally, we have fully automated our test and build infrastructure using Travis (directly visible on GitHub) and GitLab CI (for nightly builds using internal infrastructure). Read more about that in a separate blog post.

User Experience

OpTiMSoC is used at TUM for research and teaching purposes. With many different people working on it it is essential to provide a great out-of-the-box user experience. Many small changes in this regard accumulated over the period of this release.

The steps needed to get started with OpTiMSoC have been reduced. For example, all dependencies can be now installed with a single script.
A new tool, optimsoc-pgm-fpga, has been added to help flashing a bitstream onto an FPGA.
Initially we shipped our own version of FuseSoC. Now that all of our patches have made it upstream we were able to rely on the upstream version, which is easily installable through pip.
Good documentation makes a developer’s life so much easier. This time around we added API documentation for our SoC software libraries, documented the NoC implementation, and added a guide how to port OpTiMSoC to new FPGAs.
It might seem like a odd entry in the category “user experience,” but it is an important one: we integrated a fan controller for the VCU108 board, significantly reducing the noise level produced by this board.
Finally, we added a script to automatically install an Eclipse IDE with all recommended settings to develop with and on OpTiMSoC.

Linux Support

Most users of OpTiMSoC run baremetal software on it, i.e. programming it like a microcontroller without any operating system. Supporting advanced operating systems has always been a dream, but getting there required a lot of work. But with all the small and large changes in this release and in the previous releases, we finally were able to make our dream come true: reliably boot Linux on OpTiMSoC. Partly responsible for this dream is also the work of Stafford Horne, who worked relentlessly to update the OpenRISC port of Linux upstream and to incorporate all the changes that accumulated over the years.

Contributors

This release would not have happened without the continued great work of our contributors. In this release the following people contributed.

Shivam Aggarwal
Franz Biersack
Annika Fuchs
Max Koenen
Pedro H. Penna
Philipp Wagner
Stefan Wallentowitz

Linux on OpTiMSoC: How many small steps unlock a whole new world

Wed, 19 Dec 2018 00:00:00 +0000

Some projects we take upon ourselves are done quickly: start, do the work, profit. Others take a bit longer. And then there are these projects which seem to linger forever in an “almost done” state. Just one more small thing and we’ll be done. A small fix here. An extension to a module there. A new component elsewhere. And so it goes on, and on, and on. For days, for weeks, for years. Adding Linux support to OpTiMSoC is such a story. But there’s a happy end: Linux support has finally arrived!

This blog post tells the story of how Linux can be now used on OpTiMSoC, and shines a bit of light on the obstacles along the way.

How it all looks in the end

A picture is always a good starting point, so let’s start with a picture.

The picture shows a simplified version of the whole system we’ve set up to run Linux on OpTiMSoC. On the left is the “Host PC” where the developer sits. On the bottom at right is the FPGA board, in this case a Nexys 4 DDR board (recently renamed to Nexys A7-100T) containing a Xilinx Artix 7 FPGA (the XC7A100T to be exact). The board is connected to the Host PC through USB, and the FPGA is programmed with a bitstream containing our SoC design.

The design (all described in Verilog) is shown on the top right. At its center is the Wishbone bus which connects a single OpenRISC CPU core, DDR memory, and a 16550 UART module (we’ll get to that in a minute). Supporting this infrastructure is the debug system. It connects to the memory and the UART and enables them to speak with the Host PC.

What we want to achieve (and what not)

Before we go into all the gory details, let’s first discuss the goals of this whole exercise.

We want to have a Linux-based operating system running on the OpenRISC CPU core on the FPGA.
We want to see the output produced by Linux (e.g. during boot), and we want to interact with it by typing commands and seeing their output. We call this way of interaction “virtual serial console.”
We also want to use the Host PC to control the system on the FPGA: we want to load our Linux operating system into the memory on the FPGA, and we want to start and reset it.

To avoid confusion, let’s also discuss the things we don’t want to achieve.

We don’t have a monitor connected to the FPGA. Our only way of interacting with the system is through the virtual serial console.
We don’t have persistent storage hooked up to the system, like an SD card or flash memory. Once the FPGA is turned off all software is gone and needs to be loaded onto the system again through the Host PC.

With the goals and non-goals clearly stated let’s have a look at how all this machinery can be brought to life.

Let’s cook up a Linux system on OpTiMSoC!

Before we can get started we first need to prepare our “ingredients”. Don’t worry for now if you don’t know where to get these “ingredients” from, we’ll discuss that later.

A bitstream for the FPGA containing the SoC design as shown in the picture.
Software on the Host PC (which also runs Linux) to interact with the FPGA.
Finally, we need a Linux root image. That’s the software which we will execute on the OpenRISC CPU.

With everything prepared let’s start our recipe, the steps we need to take to boot up Linux on OpTiMSoC are simple.

Program the FPGA with the bitstream.
Connect to the system through the debug infrastructure.
The host software detects the UART emulation module on the FPGA and provides a virtual serial port (e.g. /dev/pts/1).
On the Host PC, open a terminal application and connect to the virtual serial port.
Write the Linux root image to the DDR memory using the host software.

That’s it! Once the Linux root image has been written to the DDR memory the CPU starts executing it. In our case, that means Linux boots, and as soon as that’s done, greets you with a login screen: “Welcome to OpTiMSoC”! You can now log into the system as root user, and interact with it as you would do with any other Linux system.

Don’t believe me? Have a look at the video then.

Where do all the components come from?

Everything you’ve seen in the video is using published components of OpTiMSoC. Head to the download page to get the latest release. This gives you pre-built versions of all tools and bitstreams used in this demo. The only thing you need to build yourself it the Linux image.

The Linux root image we’re using is similar to a self-extracting ZIP file: it contains the Linux kernel together with an initial ramdisk (initramfs). That’s essentially a “hard drive image” loaded into memory, with all tools contained in it: a shell, tools like ps or ls, and additional software we chose to add.

To prepare such a image we are using buildroot, a project dedicated to exactly that purpose. Buildroot takes a configuration file as input, and produces a vmlinux file as output. That’s just normal ELF file which we program into the memory on the FPGA board.

All our custom configuration is contained in the optimsoc/optimsoc-buildroot repository. The user guide contains detailed instructions how to make it all work.

Why did it take so long?

The system shown above looks sufficiently simple, doesn’t it? That’s what we thought as well, and again and again. It turns out, to have something as complex as Linux running on a custom system actually requires getting a lot of small and by itself insignificant details right.

The off-chip connection must be fully reliable. Making sure that no packets drop, even in rare backpressure scenarios, took a while. Max did great work in creating stress tests for our UART and USB3 interfaces (all of them part of GLIP) and debugging their fallout. Some days and nights went into the firmware for the Cypress FX3 chip, which doesn’t always behave as the datasheet would make one think …
The second ingredient is of course Linux itself. Stafford Horne did a great job to pick up work on the upstream Linux port and to steadily improve it. Today, we can run an unpatched upstream Linux kernel on our system!
One of the last steps was to add the DEM UART module to Open SoC Debug. This module emulates a 16550 UART device so that Linux can use its standard drivers to communicate. But instead of sending all data to a physical UART port, the DEM UART module wraps it into Debug Packets, which are sent through the debug interconnect to the host PC, reusing the same infrastructure we use for loading the Linux root image into the memory. This last step was completed by Thomas Leyk a couple of weeks ago.
Finally, we needed a way to reproducibly build a Linux root image. Thankfully, that’s rather easy thanks to buildroot.

Not mentioned here is all the time spent on debugging small issues, fixing the build automation, and adding tests to ensure we don’t break our newly gained Linux functionality any more. Today we have a full Linux build happening every night, performing the exact same steps as you’ve seen in the video, and checking its outputs.

Get started

Do you want to build your own Linux image for OpTiMSoC? Do you want to try out what you’ve seen in the video? Then head over to the tutorial in the user guide!

With Linux running on OpTiMSoC we’ve unlocked a whole new world of possibilities – software can run with minimal porting effort, existing drivers can be reused, and much more. The future is bright, and we’re excited to explore more ways of using OpTiMSoC!

Code with Confidence: OpTiMSoC Always Works!

Tue, 18 Dec 2018 00:00:00 +0000

OpTiMSoC is a highly complex system. If all goes to plan, software, hardware and tooling work together to form a well-integrated SoC (framework). But as so often, the reality is less gloomy: changing a single line of code anywhere could lead to trouble anywhere else. Finding out about breakages only weeks of months after the fact makes debugging a nightmare. [1]

Not any more. After multiple years of despair and a lot of work we can finally say with confidence: “OpTiMSoC always works!” In this blog post we’ll have a look at how we achieved this goal.

The first ingredient to our solution are (automated) tests. OpTiMSoC comes with many tests on various levels up to the full system level; just type make test in your OpTiMSoC source tree to execute them. But: the best tests are useless if they’re not run. And that’s the problem: Compiling all parts of OpTiMSoC and running the tests takes multiple hours. Expecting developers to run all of them after every single change is unrealistic. So we need a better solution, and the answer is (as so often) automation. In this case, the automation is called Continuous Integration, or CI for short.

CI is is not a new concept, of course. It has been used in software development for almost two decades now. Unfortunately, testing and CI in the hardware world are a bit more complicated than in the software world.

The first challenge are tools: While many of the tools we’re using are available as open source, not all of them are. Particularly the tools to generate FPGA bitstreams (e.g. Xilinx Vivado) are closed source, and some of them require a license to operate. Another challenge is the hardware itself: running tests on an FPGA board requires (unsurprisingly) an FPGA board, and a way to connect it to the test machine. A third challenge is time: synthesizing an FPGA bitstream takes multiple hours. Hence waiting for the full build and test to complete significantly reduces developer productivity.

Despite these challenges, we found a way to have extensive test coverage and fast developer feedback. A two-staged solution gets us there. The first stage are tests in Travis CI. Travis is a commonly used continuous integration service which is free to use for open source projects. It integrates nicely into GitHub and can be configured to run a build and test script on every check-in, and on every pull request.

We configured Travis to build OpTiMSoC and to run all simulation-based tests using Verilator. If a Travis run indicates a successful build and test, we already know that all steps outlined in the user guide tutorial up to (and including) the Verilator-based examples work as expected.

The second stage goes beyond what Travis can provide, as we now need access to commercial tools, associated licenses, and real hardware (e.g. FPGA boards). Internally at TUM we have access to a hosted instance of GitLab, which comes with (among many other great things) an integrated continuous integration system, GitLab CI. GitLab CI lets you attach “runners” to it. A runner is nothing else than a PC with a piece of software installed, connecting it to the central GitLab server. We have made a couple machines in our lab a “GitLab runner”. All of them have access to our tools installed on a central NFS share, and some of them have FPGA boards connected to them.

Making use of this setup and combining it with over 200 lines of configuration file we have created a fully automated continuous integration system. Every night, a new build pipeline wakes up (so we don’t need to) and crunches data for more than four hours on up to four machines in parallel.

That’s how it looks in GitLab CI:

Looking closer, the build pipeline consists of the following steps.

Get latest OpTiMSoC source code from the master branch.
Run all cocotb-based hardware tests.
Run Spyglass Lint on all hardware.
Build OpTiMSoC itself (that’s mostly software tooling)
Build seven different Verilator-based simulations models, from a small single-core, single-tile model without debug system up to a 2x2 mesh system with two cores per tile.
Build four FPGA bitstreams for the Nexys 4 DDR (recently renamed to Nexys A7-100T) and VCU108 boards.
Run the system tests in simulation and on real hardware. On real hardware, the tests use the Nexys 4 DDR board to run baremetal software, and to build, flash and boot a full Linux!
If all went well, the resulting build artifacts (source files, simulation models, FPGA bistreams) are uploaded to Bintray, where anybody can download them.

With all this automation in place you can now write a short script to always get the latest nightly build of OpTiMSoC, including FPGA bitstreams.

# figure out the latest version of OpTiMSoC [2]
LV=$(curl -sL https://api.bintray.com/packages/optimsoc/nightly/optimsoc-src/versions/_latest  |  python -c 'import sys, json; print json.load(sys.stdin)["name"]')

# get the source archive
curl -sLO https://dl.bintray.com/optimsoc/nightly/optimsoc-$LV-src.tar.gz

# get the OpTiMSoC framework
curl -sLO https://dl.bintray.com/optimsoc/nightly/optimsoc-$LV-base.tar.gz

# and finally: get all examples with all bitstreams
curl -sLO https://dl.bintray.com/optimsoc/nightly/optimsoc-$LV-examples.tar.gz
curl -sLO https://dl.bintray.com/optimsoc/nightly/optimsoc-$LV-examples-ext.tar.gz

You can find all CI configuration used for Travis in our main repository (look for the .travis.yml and Dockerfile files). The results of the Travis runs are also visible online.

The configuration for the GitLab CI is contained in the optimsoc-autobuild repository (look for the .gitlab-ci.yml file). However, this configuration is heavily tailored towards our internal infrastructure and won’t run anywhere else as-is.

So for now, go and give it a try! Use OpTiMSoC to learn and teach, to explore and discover. Have fun!

[1] At least for light debugging. Heavy debugging, typically performed at night, avoids the problem of nightmares in an interesting way: no sleep, no nightmares.

[2] Getting the latest version first is a bit annoying, having a “latest” symlink would be easier of course. This seems to be a restriction Bintray has for their non-paying customers. Having an additional line in a script is a fair price to pay a free service.

What's going on at OpTiMSoC?

Wed, 20 Sep 2017 00:00:00 +0000

Each year in September or early October the OpTiMSoC team attends ORConf, and this year was no exception. In addition to exchanging a lot of ideas with friends and other open source enthusiasts, ORConf presents itself as a good opportunity to reflect on what has happened in OpTiMSoC land over the last year.

As it turns out, there’s a lot we forgot to talk about before!

OpTiMSoC got initial Linux support, thanks to the great work of Pedro during his Google Summer of Code. See this blog post for more details. Also, none of this work would have been possible without the great support of Stafford Horne, the maintainer of the OpenRISC architecture on Linux.
Our system-level tests are run continuously in Travis and make sure that the steps outlined in the tutorial documentation always work. Even though these tests are in simulation only, they are at least able to ensure that we don’t fundamentally break the code base when making changes.
We extended the Reference Manual and added auto-generated API documentation to the homepage.
Stefan reimplemented the NoC. While the functionality stayed largely the same, the code is now easier to maintain.
The Xilinx VCU108 board is now supported.
A lot of debugging work, sweat and tears went into GLIP, especially the USB 3.0 transport (Cypress FX3). A huge thanks to Max for leading the effort on this and spending weeks on debugging tiny little bugs between the FPGA, the FX3 chip, its firmware and the host. Using USB 3 we can more than 100 MByte/s trace streams off-chip, which helped significantly in getting the Linux port up and running.
Finally: We went over 1024 commits!

The future is always hard to predict (and history has told us many times that we’re particularly bad at predictions), but some serious things are cooking and might come to you soon.

Multi-core Linux support. The required Linux patches are currently under review and are scheduled to get into Linux in the 4.15 release.
Debug infrastructure improvements. Most of that work is happening over at Open SoC Debug, but we’re actively involved and are constantly merging when things get ready.
More NoC and network adapter improvements. We have quite a bit of research going on in this area, and we’ll try to make the results available as soon as they’re sufficiently usable in the general case.

As always, if you have questions, let us know! In addition to the mailing list you can find us on Gitter in the optimsoc/Lobby channel.

GSoC 2017 Project: Integration of the OpenRISC Linux Port into OpTiMSoC

Mon, 04 Sep 2017 00:00:00 +0000

Linux was ported to OpTiMSoC during the 2017’s Google Sumer of Code. This blog post details the work that was accomplished during the project; and as well the work that was left to be tackled.

To port Linux to OpTiMSoC, a new manycore configuration with a Host Tile was designed and implemented. The Host Tile runs OpenRISC Linux and communicates with applications running on Computing Tiles via message-passing through the Network-on-Chip (NoC). Overall, the communication infrastructure is exported to user-level through standard UNIX file system operations, thereby enabling user-level applications running on the Host Tile to rely on a high-level communication abstractions.

I. Design

Figure 1 presents and the design overview of this project. A manycore configuration featuring six tiles is depicted, one host tile and five compute tiles. The host tile has a UART device attached to it and runs the OpenRISC port of Linux; whereas compute tiles runs user-level applications bare metal. Processes running on the Host Tile and user-level applications running on the Compute Tile communicate with one another via message-passing.

Figure 1: Overview of the target manycore configuration.

The OpenRisc Linux kernel running on the Host Tile features a Network-on-Chip (NoC) driver on which processes rely to send/receive messages. Operations on this driver are exported to userland through the standard UNIX system calls for manipulating files. Code Snippet 1 illustrates how a process running on the Host Tile send messages based on this abstraction. As a side remark, note that the user-level application should explicitly build the message header. In future implementation, the idea is to have a user-level library that does this job for the application. On the other hand, in Compute Tiles, user-level applications rely on a baremetal message-passing library to carry out a communication with other tiles.

int main(int argc, char **argv)
{
    int fd;
    const char *devname = "/dev/noc";

    /* Open NoC device. */
    fd = open(devname, O_WRONLY);
    
    /* Write some data. */
    while (int i = 0; i < 100; i++) {
        unsigned dest = 0x8000000;
        unsigned data = 0x0000dead;
        unsigned msg = dest | data;
        write(fd, &msg, sizeof(unsigned));
    }
   
    /* Close NoC device. */
    close(fd);

    return (EXIT_SUCCESS);
}

II. NoC Driver Implementation

To enable inter-tile communication, a NoC device driver for Linux was written. This device driver was implemented as a dynamically loadable module and it exports operations on the NoC to userland through regular file system calls. The main internal routines of the implemented driver are discussed next.

/*
 * Opens a NoC endpoint. The device minor number is used to
 * identify the target endpoint to open. Access to an endpoint
 * is exclusive, ie. only one process may use a given endpoint
 * at a time.
 */
int optimsoc_open(struct inode *, struct file *);

/*
 * Releases a NoC endpoint. The device minor number is is used
 * to identify the target  endpoint to release. Underlying
 * resources associated with the endpoint are released.
 */
int optimsoc_release(struct inode, struct file *);

/*
 * Sends messages over an endpoint. The device minor number
 * is used to identify the target endpoint to use on the
 * communication. Sending messages does not block the
 * calling process. The size of messages should be aligned
 * on word size (32 bits). Message headers should be built
 * on user-space.
 */
ssize_t optimsoc_write(struct file *, const char *, size_t, off_t);

/*
 * Receives messages over an endpoint. The device minor
 * number is used to identify the target endpoint to use on
 * the communication. Sending messages blocks the calling
 * process. Flits of a message are buffered in kernel land.
 */
ssize_t optimsoc_read(struct file *, const char *, size_t, off_t);

III. Building the Project

Since the Linux source tree is not yet integrated into OpTiMSoC Project, you should follow the next steps to get it running on OpTiMSoC.

Part 1: Setup OpTiMSoC

Follow the upstream online instructions at: https://www.optimsoc.org/docs/master/user_guide/installation.html

In the end, set the OPTIMSOC environment variable to point to the installation location of OpTiMSoC.

Part 2: Get Development Tools

To build Linux, you will need a slightly diferent toolchain than the one used to build OpTiMSoC.

Musl Toolchain
Newlib Baremetal Toolchain

mkdir -p $HOME/toolchain

cd $HOME/toolchain

wget  https://github.com/openrisc/or1k-gcc/releases/download/or1k-5.4.0-20170218/or1k-elf-5.4.0-20170218.tar.bz2

tar -xjvf or1k-elf-5.4.0-20170218.tar.bz2

wget https://github.com/openrisc/or1k-gcc/releases/download/or1k-5.4.0-20170218/or1k-linux-musl-5.4.0-20170218.tar.bz2

tar -xjvf or1k-linux-musl-5.4.0-20170218.tar.bz2

wget https://github.com/openrisc/or1k-gcc/releases/download/or1k-5.4.0-20170218/or1k-linux-5.4.0-20170218.tar.bz2

tar -xjvf or1k-linux-5.4.0-20170218.tar.bz2

Part 3: Build Unit Tests (Optional)

You can test your Linux build in OpTiMSoC with a simple distributed application:

hello-linux: runs on a host tile on top of Linux and simply sends raw messages to a remote compute tile.
hello-baremetal: runs on a compute tile on baremental, and simply read messages that arrive at the local NoC adapater and print them on the screen.

export LIBS=$OPTIMSOC/soc/sw/lib/baremetal/libbaremetal.a

git clone https://github.com/optimsoc/linux-apps $HOME/linux-apps

cd $HOME/linux-apps/

export CC=$HOME/toolchain/or1k-linux-musl/bin/or1k-linux-musl-gcc

$CC hello-linux/hello.c -o hello-linux/hello

export CC=$HOME/toolchain/or1k-elf/bin/or1k-elf-gcc
export CFLAGS=”-I $OPTIMSOC/soc/sw/include/baremetal/”

$CC $CFLAGS hello-baremetal/hello.c -o hello-baremetal/hello $LIBS

Part 4: Build Linux

Setup toolchain.

export PATH=$PATH:$HOME/toolchain/or1k-linux/bin/
export ARCH=openrisc
export CROSS_COMPILE=or1k-linux-

(Optional) If you have built unit tests from Step 3, you should copy hello-linux to the the initramfs directory.

mkdir -p $HOME/linux/arch/openrisc/initramfs/
    cp $HOME/linux-apps/hello-linux/hello $HOME/linux/arch/openrisc/initramfs/

Clone and build Linux. Note that this is done in three steps: (i) build system configuration; (ii) build dynamic modules; and (iii) build the kernel.

git clone https://github.com/optimsoc/linux.git $HOME/linux

cd $HOME/linux

make optimsoc_defconfig
make modules
make

III. Conclusions

During this GSoC project I wrote a Linux NoC driver for OpTiMSoC-based platforms. This driver was implemented as a dynamically loadable module and used standard file system kernel calls to export the message passing of the underlying platform to user space. Concerning the current implementation, the following work remains open:

[Enhancement] Enable Endpoints to Be Multiplexed. In the current implementation, only process at a time may use a given endpoint. A nice feature to have would be to enable multiple processes to concurrently use an endpoint.
[Bug Fix] Properly Install the Interrupt Handler. In the current implementation, when the interrupt handler is registered, the boot sequence halts when running on Verilator/FPGA. Interrupt-firing behavior should be investigated and the interrupt handler setup should be fixed accordingly.

2016.1 Release: Heart Surgery for FuseSoC and Open SoC Debug

Fri, 02 Sep 2016 00:00:00 +0000

It’s OpTiMSoC release time! After a bit over half a year of work, we’re proud to announce our first release in this year, 2016.1. It comes with many great new features, but two are especially noteworthy: our switch to FuseSoC and the integration of Open SoC Debug. Both new features strengthen our collaborations with other projects – because sharing is caring!

Part 1: FuseSoC

Traditionally, every digital hardware project had its own build system and its own way of managing dependencies, and OpTiMSoC was no exception. Especially for people coming from the software world with all the nice package managers and module repositories specialized to the various programming languages, this seems a bit arcane. Fortunately, our friend Olof Kindgren has put a lot of work into FuseSoC, a tool that fills exactly that gap: a package manager, dependency solver and build system for digital hardware designs.

For this release, we took the chance to perform a bit of heart surgery: rip out our existing build infrastructure based on TCL files, and replace it with FuseSoC’s core description files.

While we restored all the existing functionality, i.e. to run Verilator, Vivado Synthesis and the various simulation tools through FuseSoC, we discovered a couple of issues with FuseSoC that are not all fixed upstream yet. While we work on getting all our changes upstream, we ship OpTiMSoC with a slightly modified FuseSoC version, which is available by calling optimsoc-fusesoc. One main reason is that we are using the new <vendor>:<library>:<name>:<version> (vlnv) naming scheme that we prototyped before it is now also getting into upstream. Beside that, we added Vivado and better xsim support, that we will also get into the official fusesoc version soon.

So what does that mean for you? Synthesizing a 2x2 tiled multi-core system with four compute tiles using Xilinx Vivado is now as simple as

optimsoc-fusesoc --cores-root $OPTIMSOC_SOURCE/examples build optimsoc:examples:system_2x2_cccc_nexys4ddr

Building and running a Verilator-based simulation of a single compute tile with two mor1kx cores is not much harder:

optimsoc-fusesoc --cores-root $OPTIMSOC_SOURCE/examples sim optimsoc:examples:compute_tile_sim --NUM_CORES 2

See the documentation for more examples.

By switching to FuseSoC, we are now able to much easier share hardware modules with other projects. In the old days, sharing meant copying HDL files around, and missing updates whenever there were new releases. With FuseSoC, it’s easy to simply point to a Git repository containing HDL files, or even download a ZIP file with HDL files from a web site (a great usage example of FuseSoC!).

We use this feature to factor out and share some of our modules.

We share our board support packages, like the one for the Nexys 4 DDR board, with others. You can find them here.
We also include the HDL files for GLIP (our host communication library) and Open Soc Debug (more about that below) from their upstream sources.

Right now, we ship all these sources inside our source tree to make changes and editing easier. If the rate of changes decreases over time, we can move to directly including these dependencies from upstream.

Part 2: Open SoC Debug (OSD)

OpTiMSoC has always included a rather sophisticated trace-based debug infrastructure. As it turns out, we’re not the only open source digital hardware project that needs such a infrastructure. In order to share effort, we founded, together with the LowRISC project and in close cooperation with the PULPino project, Open SoC Debug (OSD).

The goal of this project is to collaborate on debug and diagnosis components, as they are included in OpTiMSoC: components for instruction traces, system traces, memory initialization, and much more.

In this OpTiMSoC release, we see the first results of this collaboration. Our debug infrastructure is now based on OSD. All debug features that were available previously are still there, and some features are new:

System Traces (using printf() inside the SoC software, or the OPTISMOC_TRACE macro)
Function traces
Reading and writing of memories in the system (now with an additional memory test and verify feature)
Now you can also write ELF files directly into the memory, without using objdump to convert them first.
Not really a OSD feature, but we still profit from it by using GLIP: You can now use 12 MBaud/s UART to connect to the debug system, in addition to JTAG or USB (depending on the board)

The main visible difference today are new tools to communicate with the OpTiMSoC system. Instead of our own tools like optimsoc-cli, we now use the OSD tools, like osd-cli. You find much more about how to use them in the updated tutorials in the User Guide.

Part 3: FPGA Synthesis

Support for FPGA synthesis and prebuilt example FPGA bitstreams are available again. We currently focus on the Nexys 4 DDR and the KC705 boards. But as we have defined a good board abstraction layer now, it is a lot easier to support other boards. Please get in touch with us if you want to have another board mainline supported.

Part 4: The small things

There were much more small changes throughout the system. Some things without special order.

We now use much more SystemVerilog. That’s still a bit tricky, because every tool supports a slightly different subset of features, but in general the benefits to code readability are significant.
Our build system has improved a lot to the point that we now can build a full release including bitstreams with one command. We use this also to do automated builds on Travis for all checkins.

The future is bright

With this release we’ve layed a lot of groundwork for even faster progress in the future. Some things are already in the pipeline, including a cool demo with live video streams.

We currently plan a project sprint and plan the next release for end of October with the following improvements:

Re-activation of the Partitioned Global Address Space option
Support for the KC705
Improved Network-on-Chip configuration for better extendability
Re-activation of the multiple clock domains support and maybe dynamic clocks

Stay tuned and join us on IRC or the mailing list.

Contributors to this release

Without the work of many individuals, a project like OpTiMSoC would be impossible. We thank everybody who contributed to this release and the projects that we included into our release.

Nico Gutmann
Olof Kindgren (on FuseSoC)
Stefan Rösch
Wei Song (on Open Soc Debug)
Philipp Wagner
Stefan Wallentowitz

(This list is likely to be incomplete; if you’re missing, let us know!)

2015.1 Release and Back to Unified Tree

Wed, 30 Dec 2015 00:00:00 +0000

Good news first: We have a put out a regular release and we promise to do this more often. Until now there was not much sense to make releases, because everything was in a flow and there was not even an installation package. But this has changed, from now on regular releases can be found on github. The release numbers are numbered throughout the year (2015.1 on December 30 will hopefully not happen again..).

We have also accordingly updated the download instruction, pretty simple now, right?

Now to the kind of “bad” news: We are moving back from Google repo to a unified source tree, plus extra repositories for demo apps, FPGA target code, etc. It simply turned out that for people new to git, the usage of repo is too challenging and time-consuming. Hence, you can now find the unified source tree github. We have brought in all changes from the distributed repos again.

But to turn it into good news again, you can install from sources with one simple installer script (a better one to come in 2016):

git clone https://github.com/optimsoc/sources optimsoc-sources
./optimsoc-sources/tools/install.py -d /opt/optimsoc/current

We are looking forward to a more active 2016 and wish you all a Happy New Year!

Switch to github organization and Google repo

Sat, 16 May 2015 00:00:00 +0000

A few weeks ago we started cutting out the different parts of OpTiMSoC from the original repository to separate repositories at the new organization. The aim is to structure it better and have separation of different aspects of OpTiMSoC.

The method of separating subtrees from the original tree with keeping the history is straight forward using git filter-branch. Some parts of the history get lost, especially for the system software. Nevertheless, we now have cut the original large repository into smaller logical units. Some of them are only needed for installation if you simply want to run OpTiMSoC, others are needed to build systems or for debugging.

Finally, we struggled a bit with still allowing for a unified tree layout, which is desirable for the documentation and ease of use. We wanted a tree similar to the original layout and inspected git submodule and git subtree. Unfortunately both did not convince us entirely, as we don’t want people to learn new merge strategies etc.

We first started with a simple shell script that initially creates the layout. After some more search we came to Google repo, which is a git wrapper for Android developers. While we don’t plan to use the entire feature set, we use it to distribute the tree layout. The layout is distributed as xml files and it is possible to define groups for different feature sets.

To start a layout, simply create an empty folder, download the repo script and make it executable:

mkdir optimsoc
wget https://storage.googleapis.com/git-repo-downloads/repo
chmod a+x repo

Now you can initialize the repository layout. The script downloads some more files from Google’s repositories and reads the remote xml description into a local repository of the repository files, both in the hidden path .repo.

./repo init -u https://github.com/optimsoc/optimsoc-repo

You then synchronize the repositories and repo clones and puts them into “detached HEAD” state.

 ./repo sync

If you want to create a new feature branch for a part of OpTiMSoC you can use the repo infrastructure or simply do this with the usual git commands (create branch, stage, commit etc.). We are also still experimenting with the best development strategies.