-
- Downloads
[SPARK-11583] [CORE] MapStatus Using RoaringBitmap More Properly
This PR upgrade the version of RoaringBitmap to 0.5.10, to optimize the memory layout, will be much smaller when most of blocks are empty. This PR is based on #9661 (fix conflicts), see all of the comments at https://github.com/apache/spark/pull/9661 . Author: Kent Yao <yaooqinn@hotmail.com> Author: Davies Liu <davies@databricks.com> Author: Charles Allen <charles@allen-net.com> Closes #9746 from davies/roaring_mapstatus.
Showing
- core/src/main/scala/org/apache/spark/scheduler/MapStatus.scala 3 additions, 2 deletions...src/main/scala/org/apache/spark/scheduler/MapStatus.scala
- core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala 3 additions, 3 deletions...in/scala/org/apache/spark/serializer/KryoSerializer.scala
- core/src/test/scala/org/apache/spark/scheduler/MapStatusSuite.scala 31 additions, 0 deletions...est/scala/org/apache/spark/scheduler/MapStatusSuite.scala
- pom.xml 1 addition, 1 deletionpom.xml
Loading
Please register or sign in to comment