-
- Downloads
[SPARK-1678][SPARK-1679] In-memory compression bug fix and made compression...
[SPARK-1678][SPARK-1679] In-memory compression bug fix and made compression configurable, disabled by default In-memory compression is now configurable in `SparkConf` by the `spark.sql.inMemoryCompression.enabled` property, and is disabled by default. To help code review, the bug fix is in [the first commit](https://github.com/liancheng/spark/commit/d537a367edf0bf24d0b925cc58b21d805ccbc11f), compression configuration is in [the second one](https://github.com/liancheng/spark/commit/4ce09aa8aa820bbbbbaa0f3f084a6cff1d4e6195). Author: Cheng Lian <lian.cs.zju@gmail.com> Closes #608 from liancheng/spark-1678 and squashes the following commits: 66c3a8d [Cheng Lian] Renamed in-memory compression configuration key f8fb3a0 [Cheng Lian] Added assertion for testing .hasNext of various decoder 4ce09aa [Cheng Lian] Made in-memory compression configurable via SparkConf d537a36 [Cheng Lian] Fixed SPARK-1678
Showing
- sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala 5 additions, 2 deletions...core/src/main/scala/org/apache/spark/sql/SQLContext.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/ColumnBuilder.scala 13 additions, 4 deletions...n/scala/org/apache/spark/sql/columnar/ColumnBuilder.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/InMemoryColumnarTableScan.scala 6 additions, 2 deletions...apache/spark/sql/columnar/InMemoryColumnarTableScan.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/NullableColumnBuilder.scala 2 additions, 2 deletions...org/apache/spark/sql/columnar/NullableColumnBuilder.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/compression/CompressibleColumnAccessor.scala 3 additions, 1 deletion...sql/columnar/compression/CompressibleColumnAccessor.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/compression/CompressibleColumnBuilder.scala 11 additions, 1 deletion.../sql/columnar/compression/CompressibleColumnBuilder.scala
- sql/core/src/main/scala/org/apache/spark/sql/columnar/compression/compressionSchemes.scala 1 addition, 1 deletion...e/spark/sql/columnar/compression/compressionSchemes.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala 1 addition, 1 deletion...main/scala/org/apache/spark/sql/execution/SparkPlan.scala
- sql/core/src/test/scala/org/apache/spark/sql/TestData.scala 11 additions, 0 deletionssql/core/src/test/scala/org/apache/spark/sql/TestData.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/InMemoryColumnarQuerySuite.scala 27 additions, 3 deletions...pache/spark/sql/columnar/InMemoryColumnarQuerySuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/compression/BooleanBitSetSuite.scala 6 additions, 1 deletion...e/spark/sql/columnar/compression/BooleanBitSetSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/compression/DictionaryEncodingSuite.scala 5 additions, 2 deletions...rk/sql/columnar/compression/DictionaryEncodingSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/compression/IntegralDeltaSuite.scala 6 additions, 1 deletion...e/spark/sql/columnar/compression/IntegralDeltaSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/compression/RunLengthEncodingSuite.scala 5 additions, 2 deletions...ark/sql/columnar/compression/RunLengthEncodingSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/columnar/compression/TestCompressibleColumnBuilder.scala 1 addition, 1 deletion.../columnar/compression/TestCompressibleColumnBuilder.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 1 addition, 1 deletion...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala 1 addition, 1 deletion...main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
Loading
Please register or sign in to comment