Artifact Description: Automated Instruction Stream Throughput Prediction for Intel and AMD Microarchitectures

• Jan Laukemann, Computer Science University of Erlangen-Nürnberg, jan.laukemann@fau.de
• Julian Hammer, RRZE University of Erlangen-Nürnberg, julian.hammer@fau.de, +49 9131 85 20101
• Johannes Hofmann, Chair of Computer Architecture, University of Erlangen-Nürnberg, johannes.hofmann@fau.de
• Georg Hager (advisor), RRZE University of Erlangen-Nürnberg, georg.hager@fau.de
• Gerhard Wellein (advisor), RRZE University of Erlangen-Nürnberg, gerhard.wellein@fau.de

To model in-core performance behavior of throughput bound kernels, accurate prediction of scheduling and execution of instruction streams is unavoidable. Such in-core models are indispensable for analytical performance modeling, as done in the Roofline and ECM model, and allow a deep understanding of the performance-relevant interactions between hardware architecture and kernel code.

We present a static analysis tool for predicting sequential x86 instruction stream execution time under the assumption of an infinite first level cache and perfect on out-of-order scheduling. We show the process of building a model from available documentation and manual benchmarking, carried out for the latest Intel Skylake and AMD Zen micro-architectures.

To validate the gained model, we apply the methodology to several assembly kernels and compare runtime predictions with actual measurements on two distinct microarchitectures. In the end we outline the methodology to extend such models to new architectures.

• Compilation: gcc
• Binary: x86
• Hardware: Intel skylake, AMD zen
• Publicly available?: yes

Check out https://github.com/RRZE-HPC/OSACA

We ran on an AMD EPYC 7451 (Zen architecture) at 1.8 GHz (fixed, turbo disabled) and Intel I7-6700HQ (Skylake SP architecture) at 2.4 GHz (fixed, turbo disabled). The results should be reproducible on any Zen and Skylake SP processors. Fixing the frequency and disabling turbo is vital for experiment reproduction.

• Python >= 3.5, with the following packages installed: numpy, pandas, kerncraft
• likwid
• GNU GCC 7.2.0

On Ubuntu 17.10 install with:

apt install gcc-7 python3 python3-pip likwid
pip3 install numpy pandas kerncraft

None necessary, everything is part of the code.

Please install OSACA using pip:

pip3 install osaca==0.2

To validate our results use the following commands.

Download script and benchmark codes:

git clone https://github.com/RRZE-HPC/pmbs2018-paper-artifact
cd pmbs18-paper-appendix/

Fix frequencies and disable turbo mode on CPU (for 1.8 GHz, or which ever frequency your CPU will be stable on):

likwid-setFrequencies -t 0 -f 1.8

Generate predictions (can be gained on any architecture) results with

./run_predictions.sh

Generate performance (must be done on AMD Zen and Intel Skylake SP machines) results with

./run_measurements.sh

Fixing the frequency and disabling turbo is very important to verify our results.

We expect these numbers to exactly match those in the paper. If your numbers deviate you will mostlikly have used a different compiler. Please compare the generated assembly of your compiler (found in pi/*.s and triad/.s, respectively) with those we have generated for the paper (found in orig/pi/*.s and `orig/triad/.s).

The OSACA and IACA output of your measurements can be found in the results directory of the corresponding benchmark in the structure osaca.[architecture].[optimization flag].out.

Compare numbers to Table II, IV and V.

It outputs performance measurements in Time [s], MFlop/s and MLUP/s. MLUP/s can be easily translated to cy/it, as used in the paper: 1 / MLUP/s * Frequency. E.g., 1 / (362.6 MLUP/s) * 1.8 GHz = 4.96 cy/it.

We expect these numbers to lie within 10% of those in the paper, if run on the same micro architectures as mentioned. If your numbers are significantly faster, turbo mode or frequency scaling might be the reson. If they are slower, while running on a laptop or desktop machine, energy saving features may have interfered.

The measured results will be stored as measurements.[benchmark].txt.

Compare numbers to Table I, III and V.

None

None