-
- Downloads
[SPARK-19451][SQL] rangeBetween method should accept Long value as boundary
## What changes were proposed in this pull request? Long values can be passed to `rangeBetween` as range frame boundaries, but we silently convert it to Int values, this can cause wrong results and we should fix this. Further more, we should accept any legal literal values as range frame boundaries. In this PR, we make it possible for Long values, and make accepting other DataTypes really easy to add. This PR is mostly based on Herman's previous amazing work: https://github.com/hvanhovell/spark/commit/596f53c339b1b4629f5651070e56a8836a397768 After this been merged, we can close #16818 . ## How was this patch tested? Add new tests in `DataFrameWindowFunctionsSuite` and `TypeCoercionSuite`. Author: Xingbo Jiang <xingbo.jiang@databricks.com> Closes #18540 from jiangxb1987/rangeFrame.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala 4 additions, 11 deletions...rg/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala 23 additions, 0 deletions...org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/package.scala 7 additions, 0 deletions...a/org/apache/spark/sql/catalyst/expressions/package.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala 169 additions, 159 deletions...he/spark/sql/catalyst/expressions/windowExpressions.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala 7 additions, 13 deletions...ala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala 0 additions, 2 deletions.../scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala 1 addition, 1 deletion...ache/spark/sql/catalyst/analysis/AnalysisErrorSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala 36 additions, 0 deletions...pache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala 7 additions, 10 deletions...che/spark/sql/catalyst/parser/ExpressionParserSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala 1 addition, 1 deletion...rg/apache/spark/sql/catalyst/parser/PlanParserSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala 14 additions, 13 deletions...a/org/apache/spark/sql/catalyst/trees/TreeNodeSuite.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExec.scala 53 additions, 50 deletions...la/org/apache/spark/sql/execution/window/WindowExec.scala
- sql/core/src/main/scala/org/apache/spark/sql/expressions/WindowSpec.scala 22 additions, 11 deletions...n/scala/org/apache/spark/sql/expressions/WindowSpec.scala
- sql/core/src/test/resources/sql-tests/inputs/window.sql 22 additions, 2 deletionssql/core/src/test/resources/sql-tests/inputs/window.sql
- sql/core/src/test/resources/sql-tests/results/window.sql.out 120 additions, 26 deletionssql/core/src/test/resources/sql-tests/results/window.sql.out
- sql/core/src/test/scala/org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala 42 additions, 0 deletions.../org/apache/spark/sql/DataFrameWindowFunctionsSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/catalyst/ExpressionSQLBuilderSuite.scala 5 additions, 5 deletions...apache/spark/sql/catalyst/ExpressionSQLBuilderSuite.scala
Loading
Please register or sign in to comment