-
- Downloads
Merge pull request #502 from pwendell/clone-1
Remove Hadoop object cloning and warn users making Hadoop RDD's. The code introduced in #359 used Hadoop's WritableUtils.clone() to duplicate objects when reading from Hadoop files. Some users have reported exceptions when cloning data in various file formats, including Avro and another custom format. This patch removes that functionality to ensure stability for the 0.9 release. Instead, it puts a clear warning in the documentation that copying may be necessary for Hadoop data sets.
No related branches found
No related tags found
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 81 additions, 46 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala 48 additions, 117 deletions...in/scala/org/apache/spark/api/java/JavaSparkContext.scala
- core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala 5 additions, 23 deletionscore/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala
- core/src/main/scala/org/apache/spark/rdd/NewHadoopRDD.scala 3 additions, 21 deletionscore/src/main/scala/org/apache/spark/rdd/NewHadoopRDD.scala
- core/src/main/scala/org/apache/spark/util/Utils.scala 0 additions, 22 deletionscore/src/main/scala/org/apache/spark/util/Utils.scala
Loading
Please register or sign in to comment