Skip to content
Snippets Groups Projects
Commit bf7bb497 authored by Zheng RuiFeng's avatar Zheng RuiFeng Committed by Nick Pentreath
Browse files

[SPARK-19679][ML] Destroy broadcasted object without blocking

## What changes were proposed in this pull request?
Destroy broadcasted object without blocking
use `find mllib -name '*.scala' | xargs -i bash -c 'egrep "destroy" -n {} && echo {}'`

## How was this patch tested?
existing tests

Author: Zheng RuiFeng <ruifengz@foxmail.com>

Closes #17016 from zhengruifeng/destroy_without_block.
parent ef3c7353
No related branches found
No related tags found
No related merge requests found
......@@ -226,7 +226,7 @@ private[spark] object GradientBoostedTrees extends Logging {
(a, b) => treesIndices.map(idx => a(idx) + b(idx)))
.map(_ / dataCount)
broadcastTrees.destroy()
broadcastTrees.destroy(blocking = false)
evaluation.toArray
}
......
......@@ -261,7 +261,7 @@ object LBFGS extends Logging {
val (gradientSum, lossSum) = data.treeAggregate((zeroSparseVector, 0.0))(seqOp, combOp)
// broadcasted model is not needed anymore
bcW.destroy()
bcW.destroy(blocking = false)
/**
* regVal is sum of weight squares if it's L2 updater;
......
......@@ -167,7 +167,7 @@ class GradientBoostedTreesModel @Since("1.2.0") (
(a, b) => treesIndices.map(idx => a(idx) + b(idx)))
.map(_ / dataCount)
broadcastTrees.destroy()
broadcastTrees.destroy(blocking = false)
evaluation.toArray
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment