Skip to content
Snippets Groups Projects
Commit eb82abd8 authored by Reynold Xin's avatar Reynold Xin
Browse files

[SPARK-2529] Clean closures in foreach and foreachPartition.

Author: Reynold Xin <rxin@apache.org>

Closes #1583 from rxin/closureClean and squashes the following commits:

8982fe6 [Reynold Xin] [SPARK-2529] Clean closures in foreach and foreachPartition.
parent 8529ced3
No related branches found
No related tags found
No related merge requests found
...@@ -754,14 +754,16 @@ abstract class RDD[T: ClassTag]( ...@@ -754,14 +754,16 @@ abstract class RDD[T: ClassTag](
* Applies a function f to all elements of this RDD. * Applies a function f to all elements of this RDD.
*/ */
def foreach(f: T => Unit) { def foreach(f: T => Unit) {
sc.runJob(this, (iter: Iterator[T]) => iter.foreach(f)) val cleanF = sc.clean(f)
sc.runJob(this, (iter: Iterator[T]) => iter.foreach(cleanF))
} }
/** /**
* Applies a function f to each partition of this RDD. * Applies a function f to each partition of this RDD.
*/ */
def foreachPartition(f: Iterator[T] => Unit) { def foreachPartition(f: Iterator[T] => Unit) {
sc.runJob(this, (iter: Iterator[T]) => f(iter)) val cleanF = sc.clean(f)
sc.runJob(this, (iter: Iterator[T]) => cleanF(iter))
} }
/** /**
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment