-
- Downloads
[SPARK-20988][ML] Logistic regression uses aggregator hierarchy
## What changes were proposed in this pull request? This change pulls the `LogisticAggregator` class out of LogisticRegression.scala and makes it extend `DifferentiableLossAggregator`. It also changes logistic regression to use the generic `RDDLossFunction` instead of having its own. Other minor changes: * L2Regularization accepts `Option[Int => Double]` for features standard deviation * L2Regularization uses `Vector` type instead of Array * Some tests added to LeastSquaresAggregator ## How was this patch tested? Unit test suites are added. Author: sethah <shendrickson@cloudera.com> Closes #18305 from sethah/SPARK-20988.
Showing
- mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala 20 additions, 472 deletions...g/apache/spark/ml/classification/LogisticRegression.scala
- mllib/src/main/scala/org/apache/spark/ml/optim/aggregator/LogisticAggregator.scala 364 additions, 0 deletions...apache/spark/ml/optim/aggregator/LogisticAggregator.scala
- mllib/src/main/scala/org/apache/spark/ml/optim/loss/DifferentiableRegularization.scala 31 additions, 24 deletions...he/spark/ml/optim/loss/DifferentiableRegularization.scala
- mllib/src/main/scala/org/apache/spark/ml/optim/loss/RDDLossFunction.scala 5 additions, 5 deletions...cala/org/apache/spark/ml/optim/loss/RDDLossFunction.scala
- mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala 2 additions, 1 deletion...ala/org/apache/spark/ml/regression/LinearRegression.scala
- mllib/src/test/scala/org/apache/spark/ml/classification/LogisticRegressionSuite.scala 5 additions, 4 deletions...che/spark/ml/classification/LogisticRegressionSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/optim/aggregator/DifferentiableLossAggregatorSuite.scala 37 additions, 0 deletions.../optim/aggregator/DifferentiableLossAggregatorSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/optim/aggregator/LeastSquaresAggregatorSuite.scala 25 additions, 22 deletions...ark/ml/optim/aggregator/LeastSquaresAggregatorSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/optim/aggregator/LogisticAggregatorSuite.scala 253 additions, 0 deletions...e/spark/ml/optim/aggregator/LogisticAggregatorSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/optim/loss/DifferentiableRegularizationSuite.scala 7 additions, 6 deletions...ark/ml/optim/loss/DifferentiableRegularizationSuite.scala
- mllib/src/test/scala/org/apache/spark/ml/optim/loss/RDDLossFunctionSuite.scala 3 additions, 3 deletions...org/apache/spark/ml/optim/loss/RDDLossFunctionSuite.scala
Loading
Please register or sign in to comment