-
- Downloads
Send Task results through the block manager when larger than Akka frame size.
This change requires adding an extra failure mode: tasks can complete successfully, but the result gets lost or flushed from the block manager before it's been fetched.
Showing
- core/src/main/scala/org/apache/spark/SparkContext.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/TaskEndReason.scala 6 additions, 2 deletionscore/src/main/scala/org/apache/spark/TaskEndReason.scala
- core/src/main/scala/org/apache/spark/executor/Executor.scala 17 additions, 9 deletionscore/src/main/scala/org/apache/spark/executor/Executor.scala
- core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala 4 additions, 1 deletion.../main/scala/org/apache/spark/scheduler/DAGScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/TaskResult.scala 11 additions, 3 deletions...rc/main/scala/org/apache/spark/scheduler/TaskResult.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterScheduler.scala 43 additions, 13 deletions...org/apache/spark/scheduler/cluster/ClusterScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala 78 additions, 91 deletions...pache/spark/scheduler/cluster/ClusterTaskSetManager.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/Pool.scala 3 additions, 3 deletions.../main/scala/org/apache/spark/scheduler/cluster/Pool.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/Schedulable.scala 1 addition, 3 deletions...cala/org/apache/spark/scheduler/cluster/Schedulable.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/TaskResultResolver.scala 125 additions, 0 deletions...g/apache/spark/scheduler/cluster/TaskResultResolver.scala
- core/src/main/scala/org/apache/spark/scheduler/cluster/TaskSetManager.scala 0 additions, 2 deletions...a/org/apache/spark/scheduler/cluster/TaskSetManager.scala
- core/src/main/scala/org/apache/spark/scheduler/local/LocalScheduler.scala 3 additions, 2 deletions...ala/org/apache/spark/scheduler/local/LocalScheduler.scala
- core/src/main/scala/org/apache/spark/scheduler/local/LocalTaskSetManager.scala 14 additions, 8 deletions...rg/apache/spark/scheduler/local/LocalTaskSetManager.scala
- core/src/main/scala/org/apache/spark/storage/BlockManager.scala 26 additions, 1 deletion...rc/main/scala/org/apache/spark/storage/BlockManager.scala
- core/src/test/scala/org/apache/spark/DistributedSuite.scala 0 additions, 13 deletionscore/src/test/scala/org/apache/spark/DistributedSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/TaskResultResolverSuite.scala 106 additions, 0 deletions.../org/apache/spark/scheduler/TaskResultResolverSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/cluster/ClusterSchedulerSuite.scala 8 additions, 7 deletions...pache/spark/scheduler/cluster/ClusterSchedulerSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManagerSuite.scala 6 additions, 6 deletions.../spark/scheduler/cluster/ClusterTaskSetManagerSuite.scala
Loading
Please register or sign in to comment