-
- Downloads
Fixed deadlock in BlockManager.
1. Changed the lock structure of BlockManager by replacing the 337 coarse-grained locks to use BlockInfo objects as per-block fine-grained locks. 2. Changed the MemoryStore lock structure by making the block putting threads lock on a different object (not the memory store) thus making sure putting threads minimally blocks to the getting treads. 3. Added spark.storage.ThreadingTest to stress test the BlockManager using 5 block producer and 5 block consumer threads.
Showing
- core/src/main/scala/spark/storage/BlockManager.scala 52 additions, 59 deletionscore/src/main/scala/spark/storage/BlockManager.scala
- core/src/main/scala/spark/storage/MemoryStore.scala 51 additions, 28 deletionscore/src/main/scala/spark/storage/MemoryStore.scala
- core/src/main/scala/spark/storage/ThreadingTest.scala 77 additions, 0 deletionscore/src/main/scala/spark/storage/ThreadingTest.scala
Loading
Please register or sign in to comment