-
- Downloads
[SPARK-11753][SQL][TEST-HADOOP2.2] Make allowNonNumericNumbers option work
## What changes were proposed in this pull request? Jackson suppprts `allowNonNumericNumbers` option to parse non-standard non-numeric numbers such as "NaN", "Infinity", "INF". Currently used Jackson version (2.5.3) doesn't support it all. This patch upgrades the library and make the two ignored tests in `JsonParsingOptionsSuite` passed. ## How was this patch tested? `JsonParsingOptionsSuite`. Author: Liang-Chi Hsieh <simonh@tw.ibm.com> Author: Liang-Chi Hsieh <viirya@appier.com> Closes #9759 from viirya/fix-json-nonnumric.
Showing
- dev/deps/spark-deps-hadoop-2.2 6 additions, 5 deletionsdev/deps/spark-deps-hadoop-2.2
- dev/deps/spark-deps-hadoop-2.3 6 additions, 5 deletionsdev/deps/spark-deps-hadoop-2.3
- dev/deps/spark-deps-hadoop-2.4 6 additions, 5 deletionsdev/deps/spark-deps-hadoop-2.4
- dev/deps/spark-deps-hadoop-2.6 6 additions, 5 deletionsdev/deps/spark-deps-hadoop-2.6
- dev/deps/spark-deps-hadoop-2.7 6 additions, 5 deletionsdev/deps/spark-deps-hadoop-2.7
- pom.xml 7 additions, 1 deletionpom.xml
- python/pyspark/sql/readwriter.py 3 additions, 0 deletionspython/pyspark/sql/readwriter.py
- sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala 2 additions, 0 deletions...src/main/scala/org/apache/spark/sql/DataFrameReader.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/json/JacksonParser.scala 16 additions, 12 deletions.../spark/sql/execution/datasources/json/JacksonParser.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonParsingOptionsSuite.scala 44 additions, 15 deletions.../execution/datasources/json/JsonParsingOptionsSuite.scala
Loading
Please register or sign in to comment