hpvm-cava
Harvard Camera Pipeline with HPVM intrinsics
Camera Pipeline
The camera pipeline is a simple five-stage image signal processor (ISP) which processes raw images (i.e., sensor inputs) into an image that can feed into a vision backend (e.g., a CNN).
See the original camera/vision pipeline repo (repo: yaoyuannnn/cava) for details on each stage.
HPVM DFG
The single-node HPVM DFG for the camera pipeline is as follows:
A multi-node DFG with one node per pipeline stage is forthcoming. (This will most likely be similar to the parboil/benchmarks/pipeline example in HPVM.)
Motivation: Pilot Project
This is an out-of-source C program with external C/C++ dependencies (e.g., the gem5-aladdin toolkit for SoC simulation). It is a pilot project to demonstrate the integration of HPVM into an existing external codebase with external dependencies for hardware characterization.
How to Build and Test
- Clone the HPVM repo (
mariaKt/hpvm) and build HPVM according to its instructions. - Set the source paths in the Makefile appropriately.
The remaining steps are the same as in the convenience shell script run-camera-pipeline.sh. (This script calls eog by default to view the output.)
- Build with
make TARGET=seq - Run with
./cava-visc example-face/raw_face_32x32.bin example-face/face.bin. (This processes the raw imageexample-face/raw_face_32x32.bin. Note that raw images are different from bitmaps, so you might need to obtain them using special software.) - Convert the binary output to a PNG with
python scripts/load_and_convert.py -b example-face/face.bin - View the resulting PNG at
example-face/face.png.
If all went well, example-face/face.png should look something like this:
Contributing
The main file of interest is: benchmarks/cava/src/visc/cam_pipe.c. This file defines the HPVM data flow graph (DFG), which must be defined in one file.
TODO
- Clearly describe any present limitations of the camera pipeline for processing certain raw input images.
- Copy details about camera pipeline stages from
yaoyuannnn/cavaREADME.