From ede0c36cdc028c1f802da6dec76f00d2df091d81 Mon Sep 17 00:00:00 2001
From: Adel Ejjeh <aejjeh@tyler.cs.illinois.edu>
Date: Fri, 24 Jan 2020 17:26:17 -0600
Subject: [PATCH] Final fixes to all readmes
---
README.md | 12 +++---
hpvm/docs/compilation.md | 15 ++++++++
hpvm/test/README.md | 22 +++--------
hpvm/test/benchmarks/hpvm-cava/README.md | 8 ++--
.../include/Makefile.config.example | 6 +--
hpvm/test/benchmarks/template/README.md | 37 -------------------
6 files changed, 33 insertions(+), 67 deletions(-)
create mode 100644 hpvm/docs/compilation.md
delete mode 100644 hpvm/test/benchmarks/template/README.md
diff --git a/README.md b/README.md
index 6c709cbabf..345db7ce2a 100644
--- a/README.md
+++ b/README.md
@@ -14,6 +14,8 @@ benchmark suite to test the compiler infrastructure.
[HPVM-C Language Specification](/hpvm/docs/hpvm-c.md)
+[HPVM Compilation Process](/hpvm/docs/compilation.md)
+
## Dependencies
The following components are required to be installed on your machine to build HPVM.
@@ -63,14 +65,14 @@ In order to manually build and install HPVM, GNU Make can be run using the follo
make -j<number of threads>
make install
```
-With all the aforementioned steps, HPVM should be built, installed and ready for use!
+With all the aforementioned steps, HPVM should be built, installed and ready for use.
## Benchmarks and Tests
We are providing the following HPVM benchmarks:
-* Select benchmarks from the [Parboil](http://impact.crhc.illinois.edu/parboil/parboil.aspx) benchmark suite, located under [test/parboil](/hpvm/test/parboil).
-* An edge detection pipeline benchmark, located under [test/pipeline](/hpvm/test/pipeline).
-* A Camera ISP pipeline, located under [test/cava](/hpvm/test/pipeline), adapted from C code provided from our collaborators at [Harvard](http://vlsiarch.eecs.harvard.edu).
+* Select benchmarks from the [Parboil](http://impact.crhc.illinois.edu/parboil/parboil.aspx) benchmark suite, located under [test/parboil](/hpvm/test/benchmarks/parboil).
+* An edge detection pipeline benchmark, located under [test/pipeline](/hpvm/test/benchmarks/pipeline).
+* A Camera ISP pipeline, located under [test/cava](/hpvm/test/benchmarks/hpvm-cava), adapted from C code provided from our collaborators at [Harvard](http://vlsiarch.eecs.harvard.edu).
We are also providing [unit tests](/hpvm/test/unitTests) and [regression tests](/hpvm/test/regressionTests).
-Benchmark descriptions and instructions on how to compile and run them are [here](/hpvm/test).
+Benchmark descriptions and instructions on how to compile and run them are [here](/hpvm/test/benchmarks).
diff --git a/hpvm/docs/compilation.md b/hpvm/docs/compilation.md
new file mode 100644
index 0000000000..f9a8b57209
--- /dev/null
+++ b/hpvm/docs/compilation.md
@@ -0,0 +1,15 @@
+# HPVM Compilation Process
+Compilation of an HPVM program involves the following steps:
+
+1. `clang` takes an HPVM-C/C++ program (e.g. `main.c`) and produces an LLVM IR (`main.ll`) file that contains the HPVM-C function calls. The declarations of these functions are defined in `test/benchmark/include/hpvm.h`, which must be included in the program.
+2. `opt` takes (`main.ll`) and invoke the GenHPVM pass on it, which converts the HPVM-C function calls to HPVM intrinsics. This generates the HPVM textual representation (`main.hpvm.ll`).
+3. `opt` takes the HPVM textual representation (`main.hpvm.ll`) and invokes the following passes in sequence:
+ * BuildDFG: Converts the textual representation to the internal HPVM representation.
+ * LocalMem and DFG2LLVM_NVPTX: Invoked only when GPU target is selected. Generates the kernel module (`main.kernels.ll`) and the portion of the host code that invokes the kernel into the host module (`main.host.ll`).
+ * DFG2LLVM_X86: Generates either all, or the remainder of the host module (`main.host.ll`) depending on the chosen target.
+ * ClearDFG: Deletes the internal HPVM representation from memory.
+4. `clang` is used to to compile any remaining project files that would be later linked with the host module.
+5. `llvm-link` takes the host module and all the other generate `ll` files, and links them with the HPVM runtime module (`hpvm-rt.bc`), to generate the linked host module (`main.host.linked.ll`).
+6. Generate the executable code from the generated `ll` files for all parts of the program:
+ * GPU target: `llvm-cbe` takes the kernel module (`main.kernels.ll`) and generates an OpenCL representation of the kernels that will be invoked by the host.
+ * X86 target: `clang` takes the linked host module (`main.host.linked.ll`) and generates the X86 binary.
\ No newline at end of file
diff --git a/hpvm/test/README.md b/hpvm/test/README.md
index fbce8dce48..175d60ea63 100644
--- a/hpvm/test/README.md
+++ b/hpvm/test/README.md
@@ -1,17 +1,5 @@
-# Using HPVM
-The below benchmarks are provided with HPVM, along with a template Makefile for user projects. In order to be able to build the existing benchmarks, a new `Makefile.config` must be created in [include](/hpvm/test/include) based on the existing `Makefile.config.example`. This configuration file must set up the following paths:
-* LLVM_BUILD_DIR: should point to your local `build` directory of HPVM.
-* CUDA_PATH: should point to your local CUDA installation.
-
-## Parboil
-Instructions to compile and run Parboil are provided in the following [README](/hpvm/test/parboil).
-
-## Harvard Camera Pipeline (HPVM-CAVA)
-Instructions to compile and run HPVM-CAVA are provided in the following [README](/hpvm/test/hpvm-cava).
-
-## Edge Detection Pipeline
-Instructions to compile and run Pipeline are provided in the following [README](/hpvm/test/pipeline).
-
-## Your own project
-See `template/` for an example Makefile and config.
-Include `hpvm.h` to use HPVM C api functions, found in the `test/include/hpvm.h`.
+#Test Directory Organization
+The test directory is organized as follows:
+* unitTests: Includes unit tests for HPVM.
+* regressionTests: Includes regression tests for HPVM.
+* benchmarks: Includes the current benchmarks and a template, as well as directions for compiling and running HPVM benchmarks.
diff --git a/hpvm/test/benchmarks/hpvm-cava/README.md b/hpvm/test/benchmarks/hpvm-cava/README.md
index b5573603a8..4b8611fba7 100644
--- a/hpvm/test/benchmarks/hpvm-cava/README.md
+++ b/hpvm/test/benchmarks/hpvm-cava/README.md
@@ -5,8 +5,6 @@ Harvard Camera Pipeline with HPVM intrinsics
The camera pipeline is a simple five-stage image signal processor (ISP) which processes raw images (i.e., sensor inputs) into an image that can feed into a vision backend (e.g., a CNN).
-See the original camera/vision pipeline repo (repo: `yaoyuannnn/cava`) for details on each stage.
-
## How to Build and Test
After building HPVM, the following steps are required to build and run the camera pipeline:
@@ -15,6 +13,6 @@ After building HPVM, the following steps are required to build and run the camer
2. Run with `./cava-hpvm-<Target> example-tulip-small/raw_tulip-small.bin example-tulip-small/tulip-small`.
* `<Target>` can be either `seq` or `gpu` depending on what target is used to build.
* This processes the raw image `example-tulip-small/raw_tulip-small.bin`. Note that raw images are different from bitmaps, so you might need to obtain them using special software.
- * This generates: `tulip-small.bin` and `tulip-small-<stage>.bin` where `<stage>` represents the stage of the pipeline.
-3. Convert the binary outputs to a PNG with `./convert.sh example-tulip-small`
-4. View the resulting PNG at `example-tulip-small/tulip-small.png`. (As well as all the intermediary images for each stage `tulip-small-<stage>.png`)
+ * This generates: `tulip-small.bin` and `tulip-small-<stage>.bin` where `<stage>` represents the stage of the pipeline, in the directory `example-tulip-small`.
+3. Convert the binary outputs to a PNG with `./convert.sh example-tulip-small`.
+4. View the resulting PNG at `example-tulip-small/tulip-small.png`. (As well as all the intermediary images for each stage `tulip-small-<stage>.png`).
diff --git a/hpvm/test/benchmarks/include/Makefile.config.example b/hpvm/test/benchmarks/include/Makefile.config.example
index 06267a75ee..e7260886d0 100644
--- a/hpvm/test/benchmarks/include/Makefile.config.example
+++ b/hpvm/test/benchmarks/include/Makefile.config.example
@@ -6,13 +6,13 @@ OPENCL_LIB_PATH=$(OPENCL_PATH)/lib64
# This path should be set to your HPVM build directory
LLVM_BUILD_DIR=PATH/TO/LOCAL/HPVM/BUILD
-HPVM_TEST_DIR=PATH/TO/HPVM/TEST/DIR
+HPVM_BENCH_DIR=PATH/TO/HPVM/BENCH/DIR
CC = $(LLVM_BUILD_DIR)/bin/clang
-PLATFORM_CFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_TEST_DIR)/include
+PLATFORM_CFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_BENCH_DIR)/include
CXX = $(LLVM_BUILD_DIR)/bin/clang++
-PLATFORM_CXXFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_TEST_DIR)/include
+PLATFORM_CXXFLAGS = -I$(OPENCL_PATH)/include/CL/ -I$(HPVM_BENCH_DIR)/include
LINKER = $(LLVM_BUILD_DIR)/bin/clang++
PLATFORM_LDFLAGS = -lm -lpthread -lrt -lOpenCL -L$(OPENCL_LIB_PATH)
diff --git a/hpvm/test/benchmarks/template/README.md b/hpvm/test/benchmarks/template/README.md
deleted file mode 100644
index ca51cbb900..0000000000
--- a/hpvm/test/benchmarks/template/README.md
+++ /dev/null
@@ -1,37 +0,0 @@
-Compilation of a project with HPVM is a multi-step process.
-Let's look at the compilation of the `pipeline` test for gpu as an example.
-
-`clang` is used to produce an LLVM IR file that contains the HPVM intrinsics in the form of function calls.
-```
-/.../hpvm/build/bin/clang -Isrc/ -I -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -ffast-math -O3 -fno-lax-vector-conversions -fno-vectorize -fno-slp-vectorize -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -DDEVICE=GPU_TARGET -emit-llvm -S -o build/main.ll src/main.cc
-```
-
-`opt` is used to invoke the GenHPVM pass, which converts the HPVM function calls to LLVM intrinsics.
-```
-/.../hpvm/build/bin/opt -debug-only=genhpvm -load LLVMGenHPVM.so -genhpvm -globaldce -hpvm-timers-gen build/main.ll -S -o build/main.hpvm.ll
-```
-
-`opt` is used again to invoke the BuildDFG pass, which converts the textual representation to the internal HPVM representation.
-```
-/.../hpvm/build/bin/opt -debug -load LLVMBuildDFG.so -load LLVMLocalMem.so -load LLVMDFG2LLVM_NVPTX.so -load LLVMDFG2LLVM_X86.so -load LLVMClearDFG.so -localmem -dfg2llvm-nvptx -dfg2llvm-x86 -clearDFG -hpvm-timers-x86 -hpvm-timers-ptx -S build/main.hpvm.ll -o build/pipeline-gpu.host.ll
-```
-
-`llvm-cbe` is a C backend for LLVM. It is used here to create the OpenCL kernel.
-```
-/.../hpvm/build/bin/llvm-cbe -debug build/gpu/main.hpvm.ll.kernels.ll -o build/gpu/main.hpvm.ll.kernels.cl
-```
-
-`clang` is used again to compile a separate source file that contains I/O code.
-```
-/.../hpvm/build/bin/clang -Isrc/ -I -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -ffast-math -O3 -fno-lax-vector-conversions -fno-vectorize -fno-slp-vectorize -I/.../hpvm/llvm/include -I../include -I/.../hpvm/build/include -emit-llvm -S -o build/gpu/io.ll src/io.cc
-```
-
-`llvm-link` is used to link against the HPVM runtime.
-```
-/.../hpvm/build/bin/llvm-link build/gpu/pipeline-gpu.host.ll build/gpu/io.ll /.../hpvm/llvm/tools/hpvm/projects/hpvm-rt/hpvm-rt.ll -S -o build/gpu/pipeline-gpu.linked.ll
-```
-
-`clang++` is used to do the final linking against OpenCL and emit the binary.
-```
-/.../hpvm/build/bin/clang++ -O3 `pkg-config opencv --libs` -lm -lpthread -lrt -lOpenCL -L/software/cuda-9.1/lib64 build/gpu/pipeline-gpu.linked.ll -o pipeline-gpu
-```
\ No newline at end of file
--
GitLab