Skip to content
Snippets Groups Projects
Commit 5196efff authored by Josh Rosen's avatar Josh Rosen
Browse files

[SPARK-7719] Re-add UnsafeShuffleWriterSuite test that was removed for Java 6 compat

This patch re-adds a test which was removed in 9ebb44f8 due to a Java 6 compatibility issue.  We now use Guava's `Iterators.emptyIterator()` in place of `Collections.emptyIterator()`, which isn't present in all Java 6 versions.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #6298 from JoshRosen/SPARK-7719-fix-java-6-test-code and squashes the following commits:

5c9bd85 [Josh Rosen] Re-add UnsafeShuffleWriterSuite.emptyIterator() test which was removed due to Java 6 issue
parent c330e52d
No related branches found
No related tags found
No related merge requests found
......@@ -26,6 +26,7 @@ import scala.collection.Iterator;
import scala.reflect.ClassTag;
import scala.runtime.AbstractFunction1;
import com.google.common.collect.Iterators;
import com.google.common.collect.HashMultiset;
import com.google.common.io.ByteStreams;
import org.junit.After;
......@@ -252,6 +253,20 @@ public class UnsafeShuffleWriterSuite {
createWriter(false).stop(false);
}
@Test
public void writeEmptyIterator() throws Exception {
final UnsafeShuffleWriter<Object, Object> writer = createWriter(true);
writer.write(Iterators.<Product2<Object, Object>>emptyIterator());
final Option<MapStatus> mapStatus = writer.stop(true);
assertTrue(mapStatus.isDefined());
assertTrue(mergedOutputFile.exists());
assertArrayEquals(new long[NUM_PARTITITONS], partitionSizesInMergedFile);
assertEquals(0, taskMetrics.shuffleWriteMetrics().get().shuffleRecordsWritten());
assertEquals(0, taskMetrics.shuffleWriteMetrics().get().shuffleBytesWritten());
assertEquals(0, taskMetrics.diskBytesSpilled());
assertEquals(0, taskMetrics.memoryBytesSpilled());
}
@Test
public void writeWithoutSpilling() throws Exception {
// In this example, each partition should have exactly one record:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment