-
- Downloads
[SPARK-7956] [SQL] Use Janino to compile SQL expressions into bytecode
In order to reduce the overhead of codegen, this PR switch to use Janino to compile SQL expressions into bytecode. After this, the time used to compile a SQL expression is decreased from 100ms to 5ms, which is necessary to turn on codegen for general workload, also tests. cc rxin Author: Davies Liu <davies@databricks.com> Closes #6479 from davies/janino and squashes the following commits: cc689f5 [Davies Liu] remove globalLock 262d848 [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino eec3a33 [Davies Liu] address comments from Josh f37c8c3 [Davies Liu] fix DecimalType and cast to String 202298b [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino a21e968 [Davies Liu] fix style 0ed3dc6 [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino 551a851 [Davies Liu] fix tests c3bdffa [Davies Liu] remove print 6089ce5 [Davies Liu] change logging level 7e46ac3 [Davies Liu] fix style d8f0f6c [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino da4926a [Davies Liu] fix tests 03660f3 [Davies Liu] WIP: use Janino to compile Java source f2629cd [Davies Liu] Merge branch 'master' of github.com:apache/spark into janino f7d66cf [Davies Liu] use template based string for codegen
Showing
- core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala 11 additions, 1 deletion.../scala/org/apache/spark/util/collection/OpenHashSet.scala
- pom.xml 0 additions, 10 deletionspom.xml
- project/SparkBuild.scala 0 additions, 11 deletionsproject/SparkBuild.scala
- sql/catalyst/pom.xml 5 additions, 11 deletionssql/catalyst/pom.xml
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeRow.java 10 additions, 91 deletions.../org/apache/spark/sql/catalyst/expressions/UnsafeRow.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/BaseMutableRow.java 68 additions, 0 deletions...t/src/main/scala/org/apache/spark/sql/BaseMutableRow.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/BaseRow.java 190 additions, 0 deletions...catalyst/src/main/scala/org/apache/spark/sql/BaseRow.java
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala 400 additions, 397 deletions...park/sql/catalyst/expressions/codegen/CodeGenerator.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateMutableProjection.scala 53 additions, 34 deletions...alyst/expressions/codegen/GenerateMutableProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateOrdering.scala 87 additions, 59 deletions...k/sql/catalyst/expressions/codegen/GenerateOrdering.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GeneratePredicate.scala 33 additions, 11 deletions.../sql/catalyst/expressions/codegen/GeneratePredicate.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateProjection.scala 151 additions, 165 deletions...sql/catalyst/expressions/codegen/GenerateProjection.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/package.scala 0 additions, 6 deletions...ache/spark/sql/catalyst/expressions/codegen/package.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvaluationSuite.scala 9 additions, 6 deletions.../sql/catalyst/expressions/ExpressionEvaluationSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/GeneratedEvaluationSuite.scala 3 additions, 2 deletions...k/sql/catalyst/expressions/GeneratedEvaluationSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/GeneratedMutableEvaluationSuite.scala 4 additions, 3 deletions...atalyst/expressions/GeneratedMutableEvaluationSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala 7 additions, 4 deletions.../src/test/scala/org/apache/spark/sql/DataFrameSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 85 additions, 77 deletions...e/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
Loading
Please register or sign in to comment