clang Based Atomic Operations Parser & Analyzer

The source code attached for a master's thesis work carried out in 2021. Contained within is the source code for an Atomic Operations Parser and a Performance Analyzer.

Atomic Operations Parser (Operatioon Finder)

The Atomic Operations Parser is a C++ program which is meant to parse C-code source files, and extract atomic operations from them. The tool uses LLVM & clang. So precompiling the development libraries of those two is required.

Building LLVM & clang

A docker file which automatically compiles and installs the required dependencies is included for convenience.

Otherwise, the steps to installing clang are as follows:

Install the required dependencies via apt:

apt update -y
apt install -y git build-essential cmake ninja-build python3 python3-pip

Clone LLVM from the repo. Depth 1 makes the process faster. Also set up the various folders for building and installing.

cd ~
git clone --branch "release/11.x" --depth 1 https://github.com/llvm/llvm-project.git
mkdir ~/llvm-project/build
mkdir ~/llvm-install
cd ~/llvm-project/build

Run cmake to configure the project. Followed by ninja to install it. Note that when installing, you can modify -DCMAKE_INSTALL_PREFIX to specify where the libraries should be installed to. In this case, we'll put them into ~/llvm-install.

cmake ../llvm -G "Ninja" -DCMAKE_INSTALL_PREFIX=~/llvm-install -DLLVM_ENABLE_PROJECTS="clang" -DCMAKE_BUILD_TYPE=Release
ninja install

Building the Tool

Using the conan package manager is recommended. Otherwise you have to provide Catch2_ROOT and nlohmann_json_ROOT yourself.

The variables Clang_ROOT and LLVM_ROOT depend on the previous step. If you installed the libraries into your system, then you don't need to specify them. Otherwise, assuming an installation directory of ~/llvm_install, they'd look as follows:

-DClang_ROOT=~/llvm_install/lib/cmake/clang/
-DLLVM_ROOT=~/llvm_install/lib/cmake/llvm/

Now clone this repo and cd inside of it. Make a build directory and build the project:

mkdir build
cd build
conan install .. --build=missing
export CLANG_ROOT=~/llvm_install/lib/cmake/clang/
export LLVM_ROOT=~/llvm_install/lib/cmake/llvm/
cmake .. -DWITH_TESTS=ON -DClang_ROOT=${CLANG_ROOT} -DLLVM_ROOT=${LLVM_ROOT} -GNinja
ninja

You are now left with op-finder/op-finder and op-finder-tests/op-finder-tests executables.

Usage

Use op-finder --help for help.

The finder will process multiple source files and output them to a single JSON file. For example:

op-finder ./source1.c ./source2.c -o=project_opfinder.json

The above line will take the C-code source files of source1.c and source2.c, extract atomic operations from them, and output the JSON to the ./project_opfinder.json file. This file can then be given to the analyzer along with a gcov report.

Analyzer (Operation Summarizer)

The Analyzer is responsible for taking the Atomic Operations Finder report and a gcov code coverage report and combining them into a singular analysis of the codebase. In the present implementation, it will summarize all unique atomic operations. This can then be combined with a database of atomic operations and turned into a performance estimation.

The Analyzer is written in Python and requires no tooling beyond having Python 3 installed. gcov is needed to generate the simulation reports.

Prerequisites

Install the prerequisites from your package manager:

apt update -y
apt install gcov python3 python3-pip
pip3 install gcovr

Usage

Assuming Atomic Operations Parser was used in the previous usage example. The first step is to compile the C program with GCC and to acquire a code coverage report from it using gcovr. This is done as follows:

gcc -fprofile-arcs -ftest-coverage -fPIC -O0 ./source1.c ./source2.c -o project.out
./project.out
gcovr -r ./ --json-pretty -o project_gcov.json

This will output the coverage report in human-readable JSON into the project_gcov.json file.

Next, the Analyzer needs to be ran:

python3 op-summarizer/opsummarizer.py --gcov project_gcov.json --finder project_opfinder.json --output project_summarized.json source1.c source2.c

The list of files in the specifies which source files should be taken into consideration. If a file is not present, then that file will not be evaluated during the summarization.

The summarizer will then generate an output report in JSON, along with printing a human-readable version out on the screen.