From 254eb878890b95ae4c287c6187a5771883754693 Mon Sep 17 00:00:00 2001 From: Erki Date: Thu, 6 May 2021 16:14:48 +0300 Subject: [PATCH] Licensing and readme --- LICENSE | 2 +- README.md | 127 ++++++++++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 120 insertions(+), 9 deletions(-) diff --git a/LICENSE b/LICENSE index 204b93d..a7e1971 100644 --- a/LICENSE +++ b/LICENSE @@ -1,4 +1,4 @@ -MIT License Copyright (c) +MIT License Copyright (c) 2021 Erki Meinberg Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/README.md b/README.md index f605102..0783859 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,124 @@ -# masters-thesis +# clang Based Atomic Operations Parser & Analyzer -## Building +The source code attached for a master's thesis work carried out in 2021. +Contained within is the source code for an **Atomic Operations Parser** and +a **Performance Analyzer**. -Conan is recommended. Otherwise you have to provide `Catch2_ROOT` +## Atomic Operations Parser (Operatioon Finder) + +The **Atomic Operations Parser** is a C++ program which is meant to parse C-code +source files, and extract atomic operations from them. The tool uses LLVM & clang. +So precompiling the development libraries of those two is required. + +### Building LLVM & clang + +A docker file which automatically compiles and installs the required dependencies +is included for convenience. + +Otherwise, the steps to installing clang are as follows: + +Install the required dependencies via apt: +```shell +apt update -y +apt install -y git build-essential cmake ninja-build python3 python3-pip +``` + +Clone LLVM from the repo. Depth 1 makes the process faster. Also set up the various +folders for building and installing. +```shell +cd ~ +git clone --branch "release/11.x" --depth 1 https://github.com/llvm/llvm-project.git +mkdir ~/llvm-project/build +mkdir ~/llvm-install +cd ~/llvm-project/build +``` + +Run cmake to configure the project. Followed by ninja to install it. +Note that when installing, you can modify `-DCMAKE_INSTALL_PREFIX` to specify +where the libraries should be installed to. In this case, we'll put them into +`~/llvm-install`. +```shell +cmake ../llvm -G "Ninja" -DCMAKE_INSTALL_PREFIX=~/llvm-install -DLLVM_ENABLE_PROJECTS="clang" -DCMAKE_BUILD_TYPE=Release +ninja install +``` + +### Building the Tool + +Using the conan package manager is recommended. Otherwise you have to provide `Catch2_ROOT` and `nlohmann_json_ROOT` yourself. -```shell -> mkdir build -> cd build -> conan install .. --build=missing -> cmake .. -DWITH_TESTS=ON -DClang_ROOT=${CLANG_ROOT} -DLLVM_ROOT=${LLVM_ROOT} +The variables `Clang_ROOT` and `LLVM_ROOT` depend on the previous step. If you installed the libraries +into your system, then you don't need to specify them. Otherwise, assuming an installation directory of +`~/llvm_install`, they'd look as follows: ``` +-DClang_ROOT=~/llvm_install/lib/cmake/clang/ +-DLLVM_ROOT=~/llvm_install/lib/cmake/llvm/ +``` + +Now clone this repo and `cd` inside of it. Make a build directory and build the project: + +```shell +mkdir build +cd build +conan install .. --build=missing +export CLANG_ROOT=~/llvm_install/lib/cmake/clang/ +export LLVM_ROOT=~/llvm_install/lib/cmake/llvm/ +cmake .. -DWITH_TESTS=ON -DClang_ROOT=${CLANG_ROOT} -DLLVM_ROOT=${LLVM_ROOT} -GNinja +ninja +``` + +You are now left with `op-finder/op-finder` and `op-finder-tests/op-finder-tests` executables. + +### Usage + +Use `op-finder --help` for help. + +The finder will process multiple source files and output them to a single JSON file. For example: + +`op-finder ./source1.c ./source2.c -o=project_opfinder.json` + +The above line will take the C-code source files of `source1.c` and `source2.c`, extract atomic operations +from them, and output the JSON to the `./project_opfinder.json` file. This file can then be given to the +analyzer along with a gcov report. + +## Analyzer (Operation Summarizer) + +The Analyzer is responsible for taking the Atomic Operations Finder report and a gcov code coverage report +and combining them into a singular analysis of the codebase. In the present implementation, it will +summarize all unique atomic operations. This can then be combined with a database of atomic operations +and turned into a performance estimation. + +The Analyzer is written in Python and requires no tooling beyond having Python 3 installed. gcov is needed +to generate the simulation reports. + +### Prerequisites + +Install the prerequisites from your package manager: +```shell +apt update -y +apt install gcov python3 python3-pip +pip3 install gcovr +``` + +### Usage + +Assuming Atomic Operations Parser was used in the previous usage example. The first step is to compile +the C program with GCC and to acquire a code coverage report from it using gcovr. This is done as follows: +```shell +gcc -fprofile-arcs -ftest-coverage -fPIC -O0 ./source1.c ./source2.c -o project.out +./project.out +gcovr -r ./ --json-pretty -o project_gcov.json +``` + +This will output the coverage report in human-readable JSON into the `project_gcov.json` file. + +Next, the Analyzer needs to be ran: +```shell +python3 op-summarizer/opsummarizer.py --gcov project_gcov.json --finder project_opfinder.json --output project_summarized.json source1.c source2.c +``` + +The list of files in the specifies which source files should be taken into consideration. If a file is not present, then +that file will not be evaluated during the summarization. + +The summarizer will then generate an output report in JSON, along with printing a human-readable version out +on the screen.