-
- Downloads
[SPARK-7693][Core] Remove "import scala.concurrent.ExecutionContext.Implicits.global"
Learnt a lesson from SPARK-7655: Spark should avoid to use `scala.concurrent.ExecutionContext.Implicits.global` because the user may submit blocking actions to `scala.concurrent.ExecutionContext.Implicits.global` and exhaust all threads in it. This could crash Spark. So Spark should always use its own thread pools for safety. This PR removes all usages of `scala.concurrent.ExecutionContext.Implicits.global` and uses proper thread pools to replace them. Author: zsxwing <zsxwing@gmail.com> Closes #6223 from zsxwing/SPARK-7693 and squashes the following commits: a33ff06 [zsxwing] Decrease the max thread number from 1024 to 128 cf4b3fc [zsxwing] Remove "import scala.concurrent.ExecutionContext.Implicits.global"
Showing
- core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala 5 additions, 4 deletions.../apache/spark/executor/CoarseGrainedExecutorBackend.scala
- core/src/main/scala/org/apache/spark/rdd/AsyncRDDActions.scala 11 additions, 2 deletions...src/main/scala/org/apache/spark/rdd/AsyncRDDActions.scala
- core/src/main/scala/org/apache/spark/storage/BlockManager.scala 13 additions, 4 deletions...rc/main/scala/org/apache/spark/storage/BlockManager.scala
- core/src/main/scala/org/apache/spark/storage/BlockManagerMaster.scala 18 additions, 11 deletions...n/scala/org/apache/spark/storage/BlockManagerMaster.scala
- sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoin.scala 1 addition, 1 deletion.../apache/spark/sql/execution/joins/BroadcastHashJoin.scala
- streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisor.scala 10 additions, 4 deletions.../apache/spark/streaming/receiver/ReceiverSupervisor.scala
Loading
Please register or sign in to comment