-
- Downloads
[SPARK-2554][SQL] CountDistinct partial aggregation and object allocation improvements
Author: Michael Armbrust <michael@databricks.com> Author: Gregory Owen <greowen@gmail.com> Closes #1935 from marmbrus/countDistinctPartial and squashes the following commits: 5c7848d [Michael Armbrust] turn off caching in the constructor 8074a80 [Michael Armbrust] fix tests 32d216f [Michael Armbrust] reynolds comments c122cca [Michael Armbrust] Address comments, add tests b2e8ef3 [Michael Armbrust] Merge remote-tracking branch 'origin/master' into countDistinctPartial fae38f4 [Michael Armbrust] Fix style fdca896 [Michael Armbrust] cleanup 93d0f64 [Michael Armbrust] metastore concurrency fix. db44a30 [Michael Armbrust] JIT hax. 3868f6c [Michael Armbrust] Merge pull request #9 from GregOwen/countDistinctPartial c9e67de [Gregory Owen] Made SpecificRow and types serializable by Kryo 2b46c4b [Michael Armbrust] Merge remote-tracking branch 'origin/master' into countDistinctPartial 8ff6402 [Michael Armbrust] Add specific row. 58d15f1 [Michael Armbrust] disable codegen logging 87d101d [Michael Armbrust] Fix isNullAt bug abee26d [Michael Armbrust] WIP 27984d0 [Michael Armbrust] Merge remote-tracking branch 'origin/master' into countDistinctPartial 57ae3b1 [Michael Armbrust] Fix order dependent test b3d0f64 [Michael Armbrust] Add golden files. c1f7114 [Michael Armbrust] Improve tests / fix serialization. f31b8ad [Michael Armbrust] more fixes 38c7449 [Michael Armbrust] comments and style 9153652 [Michael Armbrust] better toString d494598 [Michael Armbrust] Fix tests now that the planner is better 41fbd1d [Michael Armbrust] Never try and create an empty hash set. 050bb97 [Michael Armbrust] Skip no-arg constructors for kryo, bd08239 [Michael Armbrust] WIP 213ada8 [Michael Armbrust] First draft of partially aggregated and code generated count distinct / max
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala 342 additions, 2 deletions...rg/apache/spark/sql/catalyst/expressions/Projection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Row.scala 1 addition, 1 deletion...scala/org/apache/spark/sql/catalyst/expressions/Row.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SpecificRow.scala 307 additions, 0 deletions...g/apache/spark/sql/catalyst/expressions/SpecificRow.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregates.scala 87 additions, 6 deletions...rg/apache/spark/sql/catalyst/expressions/aggregates.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala 31 additions, 0 deletions...rg/apache/spark/sql/catalyst/expressions/arithmetic.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 91 additions, 2 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateProjection.scala 6 additions, 3 deletions...sql/catalyst/expressions/codegen/GenerateProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/sets.scala 129 additions, 0 deletions...cala/org/apache/spark/sql/catalyst/expressions/sets.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala 2 additions, 1 deletion.../scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvaluationSuite.scala 10 additions, 0 deletions.../sql/catalyst/expressions/ExpressionEvaluationSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/Aggregate.scala 1 addition, 1 deletion...main/scala/org/apache/spark/sql/execution/Aggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/GeneratedAggregate.scala 35 additions, 1 deletion...a/org/apache/spark/sql/execution/GeneratedAggregate.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlSerializer.scala 86 additions, 0 deletions...a/org/apache/spark/sql/execution/SparkSqlSerializer.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala 5 additions, 1 deletion...cala/org/apache/spark/sql/execution/SparkStrategies.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins.scala 1 addition, 1 deletion...src/main/scala/org/apache/spark/sql/execution/joins.scala
- sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetConverter.scala 4 additions, 4 deletions...scala/org/apache/spark/sql/parquet/ParquetConverter.scala
- sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTableOperations.scala 1 addition, 1 deletion...org/apache/spark/sql/parquet/ParquetTableOperations.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala 4 additions, 4 deletions...t/scala/org/apache/spark/sql/execution/PlannerSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 2 additions, 2 deletions...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
- sql/hive/src/test/resources/golden/count distinct 0 values-0-1843b7947729b771fee3a4abd050bfdc 1 addition, 0 deletions...ount distinct 0 values-0-1843b7947729b771fee3a4abd050bfdc
Loading
Please register or sign in to comment