-
- Downloads
[SPARK-9358][SQL] Code generation for UnsafeRow joiner.
This patch creates a code generated unsafe row concatenator that can be used to concatenate/join two UnsafeRows into a single UnsafeRow. Since it is inherently hard to test these low level stuff, the test suites employ randomized testing heavily in order to guarantee correctness. Author: Reynold Xin <rxin@databricks.com> Closes #7821 from rxin/rowconcat and squashes the following commits: 8717f35 [Reynold Xin] Rebase and code review. 72c5d8e [Reynold Xin] Fixed a bug. a84ed2e [Reynold Xin] Fixed offset. 40c3fb2 [Reynold Xin] Reset random data generator. f0913aa [Reynold Xin] Test fixes. 6687b6f [Reynold Xin] Updated documentation. 00354b9 [Reynold Xin] Support concat data as well. e9a4347 [Reynold Xin] Updated. 6269f96 [Reynold Xin] Fixed a bug . 0f89716 [Reynold Xin] [SPARK-9358][SQL][WIP] Code generation for UnsafeRow concat.
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 19 additions, 0 deletions.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 2 additions, 0 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala 3 additions, 3 deletions...talyst/expressions/codegen/GenerateUnsafeProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeRowJoiner.scala 241 additions, 0 deletions...atalyst/expressions/codegen/GenerateUnsafeRowJoiner.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala 14 additions, 1 deletion...test/scala/org/apache/spark/sql/RandomDataGenerator.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeRowJoinerBitsetSuite.scala 147 additions, 0 deletions...ressions/codegen/GenerateUnsafeRowJoinerBitsetSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeRowJoinerSuite.scala 114 additions, 0 deletions...st/expressions/codegen/GenerateUnsafeRowJoinerSuite.scala
- sql/core/src/main/java/org/apache/spark/sql/execution/UnsafeFixedWidthAggregationMap.java 1 addition, 6 deletions...e/spark/sql/execution/UnsafeFixedWidthAggregationMap.java
- sql/core/src/test/scala/org/apache/spark/sql/execution/TungstenSortSuite.scala 3 additions, 0 deletions...la/org/apache/spark/sql/execution/TungstenSortSuite.scala
Loading
Please register or sign in to comment