-
- Downloads
[SPARK-22542][SQL] remove unused features in ColumnarBatch
## What changes were proposed in this pull request? `ColumnarBatch` provides features to do fast filter and project in a columnar fashion, however this feature is never used by Spark, as Spark uses whole stage codegen and processes the data in a row fashion. This PR proposes to remove these unused features as we won't switch to columnar execution in the near future. Even we do, I think this part needs a proper redesign. This is also a step to make `ColumnVector` public, as we don't wanna expose these features to users. ## How was this patch tested? existing tests Author: Wenchen Fan <wenchen@databricks.com> Closes #19766 from cloud-fan/vector.
Showing
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 0 additions, 4 deletions.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarArray.java 1 addition, 1 deletion.../apache/spark/sql/execution/vectorized/ColumnarArray.java
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarBatch.java 4 additions, 74 deletions.../apache/spark/sql/execution/vectorized/ColumnarBatch.java
- sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/ColumnarRow.java 0 additions, 26 deletions...rg/apache/spark/sql/execution/vectorized/ColumnarRow.java
- sql/core/src/test/scala/org/apache/spark/sql/execution/arrow/ArrowConvertersSuite.scala 1 addition, 1 deletion...ache/spark/sql/execution/arrow/ArrowConvertersSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala 0 additions, 24 deletions.../execution/datasources/parquet/ParquetReadBenchmark.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/vectorized/ColumnarBatchSuite.scala 0 additions, 52 deletions...e/spark/sql/execution/vectorized/ColumnarBatchSuite.scala
Loading
Please register or sign in to comment