-
- Downloads
Reduced the memory usage of reduce and similar operations
These operations used to wait for all the results to be available in an array on the driver program before merging them. They now merge values incrementally as they arrive.
Showing
- core/src/main/scala/spark/PairRDDFunctions.scala 2 additions, 2 deletionscore/src/main/scala/spark/PairRDDFunctions.scala
- core/src/main/scala/spark/RDD.scala 25 additions, 16 deletionscore/src/main/scala/spark/RDD.scala
- core/src/main/scala/spark/SparkContext.scala 46 additions, 7 deletionscore/src/main/scala/spark/SparkContext.scala
- core/src/main/scala/spark/Utils.scala 8 additions, 0 deletionscore/src/main/scala/spark/Utils.scala
- core/src/main/scala/spark/partial/ApproximateActionListener.scala 2 additions, 2 deletions.../main/scala/spark/partial/ApproximateActionListener.scala
- core/src/main/scala/spark/scheduler/DAGScheduler.scala 7 additions, 8 deletionscore/src/main/scala/spark/scheduler/DAGScheduler.scala
- core/src/main/scala/spark/scheduler/JobResult.scala 1 addition, 1 deletioncore/src/main/scala/spark/scheduler/JobResult.scala
- core/src/main/scala/spark/scheduler/JobWaiter.scala 8 additions, 6 deletionscore/src/main/scala/spark/scheduler/JobWaiter.scala
- core/src/test/scala/spark/RDDSuite.scala 8 additions, 4 deletionscore/src/test/scala/spark/RDDSuite.scala
Loading
Please register or sign in to comment