-
Yifan Zhao authoredYifan Zhao authored
Tensor Operations in HPVM
Tensor Implementation in HPVM
Tensors are referred to in HPVM IR with pointers (as opposed to SSA tensor values). The HPVM Tensor Runtime allocates tensors in memory and is responsible for managing them. The runtime assumes that the layout of tensors is NCHW.
Intrinsics for Tensor Operations
Tensor Add
Overview
Add tensor pointed to by input with the tensor pointed to by bias. Axis of size 1 will be broadcast. Rank of input and bias tensors must match. Return a pointer to the resultant tensor of the same size and shape.
i8* llvm.hpvm.tensor.add(i8* input , i8* bias)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| bias | Reference to the bias tensor |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Multiplication
Overview
Perform matrix multiplication between the tensor pointed to by input and the tensor pointed to by weight. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.mul(i8* input, i8* weight)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| weight | Reference to the weight tensor |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor ReLU
Overview
Perform the element-wise clipped ReLU (Rectified Linear Unit) operation on the tensor pointed to by input. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.relu(i8* input)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Clipped ReLU
Overview
Perform the element-wise clipped ReLU (Rectified Linear Unit) operation on the tensor pointed to by input. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.clipped.relu(i8* input)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Tanh
Overview
Perform the element-wise hyperbolic tangent operation on the tensor pointed to by input. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.tanh(i8* input)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Sigmoid
Overview
Perform the element-wise sigmoid function on the tensor pointed to by input. Sigmoid function: output = 1 / (1 + exp(-input). Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.sigmoid(i8* input)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Softmax
Overview
Perform the element-wise softmax operation on the tensor pointed to by input. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.softmax(i8* input)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Convolution
Overview
Perform convolution operation between the tensor pointed to by input and the tensor pointed to by filter. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.convolution(i8* input, i8* filter, i32 vpad, i32 hpad, i32 vstride, i32 hstride)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| filter | Reference to the filter tensor |
| vpad | Vertical pad |
| hpad | Horizontal pad |
| vstride | Vertical stride |
| hstride | Horizontal stride |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Group Convolution
Overview
Perform depthwise-convolution operation between the tensor pointed to by input and the tensor pointed to by filter. This operation entails performing 2D convolutions separately over each channel of the given input and filter tensors. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.group.convolution(i8* input, i8* filter, i32 vpad, i32 hpad, i32 vstride, i32 hstride)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| filter | Reference to the filter tensor |
| vpad | Vertical pad |
| hpad | Horizontal pad |
| vstride | Vertical stride |
| hstride | Horizontal stride |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Batchnorm
Overview
Perform batch-normalization operation on the tensor pointed to by input. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.batchnorm(i8* input, i8* gamma, i8* beta, i8* mean, i8* variance, double epsilon)
Batchnorm is computed using the following formula: output = beta + gamma * ((input - mean)/sqrt(epsilon + variance))
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| gamma | Reference to the scale tensor |
| beta | Reference to the bias tensor |
| mean | Reference to the mean tensor |
| variance | Reference to the variance tensor |
| epsilon | The Epsilon value in the batchnorm formula |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Pool Max
Overview
Perform reduction maximum function to all elements within the sliding window, and place the maximum value in the output tensor. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.pool.max(i8* input, i8* filter, i32 winWidth, i32 winHeight, i32 vpad, i32 hpad, i32 vstride, i32 hstride)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| filter | Reference to the filter tensor |
| winWidth | Width of the sliding window |
| winHeight | Height of the sliding window |
| vpad | Vertical pad |
| hpad | Horizontal pad |
| vstride | Vertical stride |
| hstride | Horizontal stride |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Pool Min
Overview
Perform reduction minimum function to all elements within the sliding window, and place the minimum value in the output tensor. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.pool.min(i8* input, i8* filter, i32 winWidth, i32 winHeight, i32 vpad, i32 hpad, i32 vstride, i32 hstride)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| filter | Reference to the filter tensor |
| winWidth | Width of the sliding window |
| winHeight | Height of the sliding window |
| vpad | Vertical pad |
| hpad | Horizontal pad |
| vstride | Vertical stride |
| hstride | Horizontal stride |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |
Tensor Pool Mean
Overview
Perform reduction mean function to all elements within the sliding window, and place the mean value in the output tensor. Return a pointer to the resultant tensor.
i8* llvm.hpvm.tensor.pool.mean(i8* input, i8* filter, i32 winWidth, i32 winHeight, i32 vpad, i32 hpad, i32 vstride, i32 hstride)
Operands
| Operand | Descrption |
|---|---|
| input | Reference to the input tensor |
| filter | Reference to the filter tensor |
| winWidth | Width of the sliding window |
| winHeight | Height of the sliding window |
| vpad | Vertical pad |
| hpad | Horizontal pad |
| vstride | Vertical stride |
| hstride | Horizontal stride |
Result
| Result | Descrption |
|---|---|
| output | Reference to the output tensor |