Skip to content
Snippets Groups Projects
Commit bf7bb497 authored by Zheng RuiFeng's avatar Zheng RuiFeng Committed by Nick Pentreath
Browse files

[SPARK-19679][ML] Destroy broadcasted object without blocking

## What changes were proposed in this pull request?
Destroy broadcasted object without blocking
use `find mllib -name '*.scala' | xargs -i bash -c 'egrep "destroy" -n {} && echo {}'`

## How was this patch tested?
existing tests

Author: Zheng RuiFeng <ruifengz@foxmail.com>

Closes #17016 from zhengruifeng/destroy_without_block.
parent ef3c7353
No related branches found
No related tags found
No related merge requests found
...@@ -226,7 +226,7 @@ private[spark] object GradientBoostedTrees extends Logging { ...@@ -226,7 +226,7 @@ private[spark] object GradientBoostedTrees extends Logging {
(a, b) => treesIndices.map(idx => a(idx) + b(idx))) (a, b) => treesIndices.map(idx => a(idx) + b(idx)))
.map(_ / dataCount) .map(_ / dataCount)
broadcastTrees.destroy() broadcastTrees.destroy(blocking = false)
evaluation.toArray evaluation.toArray
} }
......
...@@ -261,7 +261,7 @@ object LBFGS extends Logging { ...@@ -261,7 +261,7 @@ object LBFGS extends Logging {
val (gradientSum, lossSum) = data.treeAggregate((zeroSparseVector, 0.0))(seqOp, combOp) val (gradientSum, lossSum) = data.treeAggregate((zeroSparseVector, 0.0))(seqOp, combOp)
// broadcasted model is not needed anymore // broadcasted model is not needed anymore
bcW.destroy() bcW.destroy(blocking = false)
/** /**
* regVal is sum of weight squares if it's L2 updater; * regVal is sum of weight squares if it's L2 updater;
......
...@@ -167,7 +167,7 @@ class GradientBoostedTreesModel @Since("1.2.0") ( ...@@ -167,7 +167,7 @@ class GradientBoostedTreesModel @Since("1.2.0") (
(a, b) => treesIndices.map(idx => a(idx) + b(idx))) (a, b) => treesIndices.map(idx => a(idx) + b(idx)))
.map(_ / dataCount) .map(_ / dataCount)
broadcastTrees.destroy() broadcastTrees.destroy(blocking = false)
evaluation.toArray evaluation.toArray
} }
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment