-
- Downloads
[SPARK-15397][SQL] fix string udf locate as hive
## What changes were proposed in this pull request? in hive, `locate("aa", "aaa", 0)` would yield 0, `locate("aa", "aaa", 1)` would yield 1 and `locate("aa", "aaa", 2)` would yield 2, while in Spark, `locate("aa", "aaa", 0)` would yield 1, `locate("aa", "aaa", 1)` would yield 2 and `locate("aa", "aaa", 2)` would yield 0. This results from the different understanding of the third parameter in udf `locate`. It means the starting index and starts from 1, so when we use 0, the return would always be 0. ## How was this patch tested? tested with modified `StringExpressionsSuite` and `StringFunctionsSuite` Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes #13186 from adrian-wang/locate.
Showing
- R/pkg/R/functions.R 1 addition, 1 deletionR/pkg/R/functions.R
- R/pkg/inst/tests/testthat/test_sparkSQL.R 1 addition, 1 deletionR/pkg/inst/tests/testthat/test_sparkSQL.R
- python/pyspark/sql/functions.py 1 addition, 1 deletionpython/pyspark/sql/functions.py
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala 13 additions, 6 deletions...he/spark/sql/catalyst/expressions/stringExpressions.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/StringExpressionsSuite.scala 9 additions, 7 deletions...ark/sql/catalyst/expressions/StringExpressionsSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala 5 additions, 5 deletions...est/scala/org/apache/spark/sql/StringFunctionsSuite.scala
Loading
Please register or sign in to comment