Skip to content
Snippets Groups Projects
Commit d17ff8e8 authored by Akash Kothari's avatar Akash Kothari
Browse files

Merge branch 'hpvm-release-exp' of https://gitlab.engr.illinois.edu/llvm/hpvm into hpvm-release-exp

parents bb806560 bd8d02eb
No related branches found
No related tags found
No related merge requests found
......@@ -14,6 +14,8 @@ benchmark suite to test the compiler infrastructure.
[HPVM-C Language Specification](/hpvm/docs/hpvm-c.md)
[HPVM Compilation Process](/hpvm/docs/compilation.md)
## Dependencies
The following components are required to be installed on your machine to build HPVM.
......@@ -63,14 +65,16 @@ In order to manually build and install HPVM, GNU Make can be run using the follo
make -j<number of threads>
make install
```
With all the aforementioned steps, HPVM should be built, installed and ready for use!
With all the aforementioned steps, HPVM should be built, installed and ready for use.
## Benchmarks and Tests
We are providing the following HPVM benchmarks:
* Select benchmarks from the [Parboil](http://impact.crhc.illinois.edu/parboil/parboil.aspx) benchmark suite, located under [test/parboil](/hpvm/test/parboil).
* An edge detection pipeline benchmark, located under [test/pipeline](/hpvm/test/pipeline).
* A Camera ISP pipeline, located under [test/cava](/hpvm/test/pipeline), adapted from C code provided from our collaborators at [Harvard](http://vlsiarch.eecs.harvard.edu).
We are providing the following [HPVM benchmarks](/hpvm/test/benchmarks):
* Select benchmarks from the [Parboil](http://impact.crhc.illinois.edu/parboil/parboil.aspx) benchmark suite, located under [test/benchmarks/parboil](/hpvm/test/benchmarks/parboil).
* An edge detection pipeline benchmark, located under [test/benchmarks/pipeline](/hpvm/test/benchmarks/pipeline).
* A Camera ISP pipeline, located under [test/benchmarks/hpvm-cava](/hpvm/test/benchmarks/hpvm-cava), adapted from C code provided from our collaborators at [Harvard](http://vlsiarch.eecs.harvard.edu).
Benchmark descriptions and instructions on how to compile and run them are [here](/hpvm/test/benchmarks).
We are also providing [unit tests](/hpvm/test/unitTests) and [regression tests](/hpvm/test/regressionTests).
Benchmark descriptions and instructions on how to compile and run them are [here](/hpvm/test).
# HPVM Compilation Process
Compilation of an HPVM program involves the following steps:
1. `clang` takes an HPVM-C/C++ program (e.g. `main.c`) and produces an LLVM IR (`main.ll`) file that contains the HPVM-C function calls. The declarations of these functions are defined in `test/benchmark/include/hpvm.h`, which must be included in the program.
2. `opt` takes (`main.ll`) and invoke the GenHPVM pass on it, which converts the HPVM-C function calls to HPVM intrinsics. This generates the HPVM textual representation (`main.hpvm.ll`).
3. `opt` takes the HPVM textual representation (`main.hpvm.ll`) and invokes the following passes in sequence:
* BuildDFG: Converts the textual representation to the internal HPVM representation.
* LocalMem and DFG2LLVM_NVPTX: Invoked only when GPU target is selected. Generates the kernel module (`main.kernels.ll`) and the portion of the host code that invokes the kernel into the host module (`main.host.ll`).
* DFG2LLVM_X86: Generates either all, or the remainder of the host module (`main.host.ll`) depending on the chosen target.
* ClearDFG: Deletes the internal HPVM representation from memory.
4. `clang` is used to to compile any remaining project files that would be later linked with the host module.
5. `llvm-link` takes the host module and all the other generate `ll` files, and links them with the HPVM runtime module (`hpvm-rt.bc`), to generate the linked host module (`main.host.linked.ll`).
6. Generate the executable code from the generated `ll` files for all parts of the program:
* GPU target: `llvm-cbe` takes the kernel module (`main.kernels.ll`) and generates an OpenCL representation of the kernels that will be invoked by the host.
* X86 target: `clang` takes the linked host module (`main.host.linked.ll`) and generates the X86 binary.
# Using HPVM
The below benchmarks are provided with HPVM, along with a template Makefile for user projects. In order to be able to build the existing benchmarks, a new `Makefile.config` must be created in [include](/hpvm/test/include) based on the existing `Makefile.config.example`. This configuration file must set up the following paths:
* LLVM_BUILD_DIR: should point to your local `build` directory of HPVM.
* CUDA_PATH: should point to your local CUDA installation.
## Parboil
Instructions to compile and run Parboil are provided in the following [README](/hpvm/test/parboil).
## Harvard Camera Pipeline (HPVM-CAVA)
Instructions to compile and run HPVM-CAVA are provided in the following [README](/hpvm/test/hpvm-cava).
## Edge Detection Pipeline
Instructions to compile and run Pipeline are provided in the following [README](/hpvm/test/pipeline).
## Your own project
See `template/` for an example Makefile and config.
Include `hpvm.h` to use HPVM C api functions, found in the `test/include/hpvm.h`.
#Test Directory Organization
The test directory is organized as follows:
* unitTests: Includes unit tests for HPVM.
* regressionTests: Includes regression tests for HPVM.
* benchmarks: Includes the current benchmarks and a template, as well as directions for compiling and running HPVM benchmarks.
......@@ -5,8 +5,6 @@ Harvard Camera Pipeline with HPVM intrinsics
The camera pipeline is a simple five-stage image signal processor (ISP) which processes raw images (i.e., sensor inputs) into an image that can feed into a vision backend (e.g., a CNN).
See the original camera/vision pipeline repo (repo: `yaoyuannnn/cava`) for details on each stage.
## How to Build and Test
After building HPVM, the following steps are required to build and run the camera pipeline:
......@@ -15,6 +13,6 @@ After building HPVM, the following steps are required to build and run the camer
2. Run with `./cava-hpvm-<Target> example-tulip-small/raw_tulip-small.bin example-tulip-small/tulip-small`.
* `<Target>` can be either `seq` or `gpu` depending on what target is used to build.
* This processes the raw image `example-tulip-small/raw_tulip-small.bin`. Note that raw images are different from bitmaps, so you might need to obtain them using special software.
* This generates: `tulip-small.bin` and `tulip-small-<stage>.bin` where `<stage>` represents the stage of the pipeline.
3. Convert the binary outputs to a PNG with `./convert.sh example-tulip-small`
4. View the resulting PNG at `example-tulip-small/tulip-small.png`. (As well as all the intermediary images for each stage `tulip-small-<stage>.png`)
* This generates: `tulip-small.bin` and `tulip-small-<stage>.bin` where `<stage>` represents the stage of the pipeline, in the directory `example-tulip-small`.
3. Convert the binary outputs to a PNG with `./convert.sh example-tulip-small`.
4. View the resulting PNG at `example-tulip-small/tulip-small.png`. (As well as all the intermediary images for each stage `tulip-small-<stage>.png`).
......@@ -6,13 +6,13 @@ OPENCL_LIB_PATH=$(OPENCL_PATH)/lib64
# This path should be set to your HPVM build directory
LLVM_BUILD_DIR=PATH/TO/LOCAL/HPVM/BUILD
HPVM_TEST_DIR=PATH/TO/HPVM/TEST/DIR
HPVM_BENCH_DIR=PATH/TO/HPVM/BENCH/DIR
CC = $(LLVM_BUILD_DIR)/bin/clang
PLATFORM_CFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_TEST_DIR)/include
PLATFORM_CFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_BENCH_DIR)/include
CXX = $(LLVM_BUILD_DIR)/bin/clang++
PLATFORM_CXXFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_TEST_DIR)/include
PLATFORM_CXXFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_BENCH_DIR)/include
LINKER = $(LLVM_BUILD_DIR)/bin/clang++
PLATFORM_LDFLAGS = -lm -lpthread -lrt -lOpenCL -L$(OPENCL_LIB_PATH)
......
Compilation of a project with HPVM is a multi-step process.
Let's look at the compilation of the `pipeline` test for gpu as an example.
`clang` is used to produce an LLVM IR file that contains the HPVM intrinsics in the form of function calls.
```
/.../hpvm/build/bin/clang -Isrc/ -I -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -ffast-math -O3 -fno-lax-vector-conversions -fno-vectorize -fno-slp-vectorize -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -DDEVICE=GPU_TARGET -emit-llvm -S -o build/main.ll src/main.cc
```
`opt` is used to invoke the GenHPVM pass, which converts the HPVM function calls to LLVM intrinsics.
```
/.../hpvm/build/bin/opt -debug-only=genhpvm -load LLVMGenHPVM.so -genhpvm -globaldce -hpvm-timers-gen build/main.ll -S -o build/main.hpvm.ll
```
`opt` is used again to invoke the BuildDFG pass, which converts the textual representation to the internal HPVM representation.
```
/.../hpvm/build/bin/opt -debug -load LLVMBuildDFG.so -load LLVMLocalMem.so -load LLVMDFG2LLVM_NVPTX.so -load LLVMDFG2LLVM_X86.so -load LLVMClearDFG.so -localmem -dfg2llvm-nvptx -dfg2llvm-x86 -clearDFG -hpvm-timers-x86 -hpvm-timers-ptx -S build/main.hpvm.ll -o build/pipeline-gpu.host.ll
```
`llvm-cbe` is a C backend for LLVM. It is used here to create the OpenCL kernel.
```
/.../hpvm/build/bin/llvm-cbe -debug build/gpu/main.hpvm.ll.kernels.ll -o build/gpu/main.hpvm.ll.kernels.cl
```
`clang` is used again to compile a separate source file that contains I/O code.
```
/.../hpvm/build/bin/clang -Isrc/ -I -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -ffast-math -O3 -fno-lax-vector-conversions -fno-vectorize -fno-slp-vectorize -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -emit-llvm -S -o build/gpu/io.ll src/io.cc
```
`llvm-link` is used to link against the HPVM runtime.
```
/.../hpvm/build/bin/llvm-link build/gpu/pipeline-gpu.host.ll build/gpu/io.ll /.../hpvm/llvm/tools/hpvm/projects/hpvm-rt/hpvm-rt.ll -S -o build/gpu/pipeline-gpu.linked.ll
```
`clang++` is used to do the final linking against OpenCL and emit the binary.
```
/.../hpvm/build/bin/clang++ -O3 `pkg-config opencv --libs` -lm -lpthread -lrt -lOpenCL -L/software/cuda-9.1/lib64 build/gpu/pipeline-gpu.linked.ll -o pipeline-gpu
```
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment