Skip to content
Snippets Groups Projects
Commit f92cde24 authored by Michael Armbrust's avatar Michael Armbrust
Browse files

[SPARK-3447][SQL] Remove explicit conversion with JListWrapper to avoid NPE

Author: Michael Armbrust <michael@databricks.com>

Closes #2323 from marmbrus/kryoJListNPE and squashes the following commits:

9634f11 [Michael Armbrust] Rollback JSON RDD changes
4d4d93c [Michael Armbrust] Merge remote-tracking branch 'origin/master' into kryoJListNPE
646976b [Michael Armbrust] Fix JSON RDD Conversion too
59065bc [Michael Armbrust] Remove explicit conversion to avoid NPE
parent 84e2c8bf
No related branches found
No related tags found
No related merge requests found
......@@ -460,7 +460,6 @@ class SQLContext(@transient val sparkContext: SparkContext)
rdd: RDD[Array[Any]],
schema: StructType): SchemaRDD = {
import scala.collection.JavaConversions._
import scala.collection.convert.Wrappers.{JListWrapper, JMapWrapper}
def needsConversion(dataType: DataType): Boolean = dataType match {
case ByteType => true
......@@ -482,8 +481,7 @@ class SQLContext(@transient val sparkContext: SparkContext)
case (null, _) => null
case (c: java.util.List[_], ArrayType(elementType, _)) =>
val converted = c.map { e => convert(e, elementType)}
JListWrapper(converted)
c.map { e => convert(e, elementType)}: Seq[Any]
case (c, ArrayType(elementType, _)) if c.getClass.isArray =>
c.asInstanceOf[Array[_]].map(e => convert(e, elementType)): Seq[Any]
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment