-
- Downloads
[SPARK-8572] [SQL] Type coercion for ScalaUDFs
Implemented type coercion for udf arguments in Scala. The changes include- * Add `with ExpectsInputTypes ` to `ScalaUDF` class. * Pass down argument types info from `UDFRegistration` and `functions`. With this patch, the example query in [SPARK-8572](https://issues.apache.org/jira/browse/SPARK-8572) no longer throws a type cast error at runtime. Also added a unit test to `UDFSuite` in which a decimal type is passed to a udf that expects an int. Author: Cheolsoo Park <cheolsoop@netflix.com> Closes #7203 from piaozhexiu/SPARK-8572 and squashes the following commits: 2d0ed15 [Cheolsoo Park] Incorporate comments dce1efd [Cheolsoo Park] Fix unit tests and update the codegen script 066deed [Cheolsoo Park] Type coercion for udf inputs
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala 1 addition, 1 deletion...apache/spark/sql/catalyst/analysis/HiveTypeCoercion.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala 5 additions, 2 deletions.../org/apache/spark/sql/catalyst/expressions/ScalaUDF.scala
- sql/core/src/main/scala/org/apache/spark/sql/UDFRegistration.scala 50 additions, 25 deletions...src/main/scala/org/apache/spark/sql/UDFRegistration.scala
- sql/core/src/main/scala/org/apache/spark/sql/UserDefinedFunction.scala 5 additions, 2 deletions...main/scala/org/apache/spark/sql/UserDefinedFunction.scala
- sql/core/src/main/scala/org/apache/spark/sql/functions.scala 26 additions, 12 deletionssql/core/src/main/scala/org/apache/spark/sql/functions.scala
- sql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala 6 additions, 0 deletionssql/core/src/test/scala/org/apache/spark/sql/UDFSuite.scala
Loading
Please register or sign in to comment