Skip to content
Snippets Groups Projects
Commit fd0374b9 authored by Matei Zaharia's avatar Matei Zaharia
Browse files

Comment

parent 5718cef2
No related branches found
No related tags found
No related merge requests found
......@@ -4,7 +4,7 @@ private class CoalescedRDDSplit(val index: Int, val parents: Array[Split]) exten
/**
* Coalesce the partitions of a parent RDD (`prev`) into fewer partitions, so that each partition of
* this RDD computes zero or more of the parent ones. Will produce exactly `maxPartitions` if the
* this RDD computes one or more of the parent ones. Will produce exactly `maxPartitions` if the
* parent had more than this many partitions, or fewer if the parent had fewer.
*
* This transformation is useful when an RDD with many partitions gets filtered into a smaller one,
......
......@@ -79,7 +79,7 @@ class RDDSuite extends FunSuite with BeforeAndAfter {
val coalesced1 = new CoalescedRDD(data, 2)
assert(coalesced1.collect().toList === (1 to 10).toList)
assert(coalesced1.glom().collect().map(_.toList).toList ===
List(List(1, 2, 3, 4, 5), List(6, 7, 8, 9, 10)))
List(List(1, 2, 3, 4, 5), List(6, 7, 8, 9, 10)))
// Check that the narrow dependency is also specified correctly
assert(coalesced1.dependencies.head.getParents(0).toList === List(0, 1, 2, 3, 4))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment