Skip to content
Snippets Groups Projects
  • Xiangrui Meng's avatar
    9038d94e
    [SPARK-2923][MLLIB] Implement some basic BLAS routines · 9038d94e
    Xiangrui Meng authored
    Having some basic BLAS operations implemented in MLlib can help simplify the current implementation and improve some performance.
    
    Tested on my local machine:
    
    ~~~
    bin/spark-submit --class org.apache.spark.examples.mllib.BinaryClassification \
    examples/target/scala-*/spark-examples-*.jar --algorithm LR --regType L2 \
    --regParam 1.0 --numIterations 1000 ~/share/data/rcv1.binary/rcv1_train.binary
    ~~~
    
    1. before: ~1m
    2. after: ~30s
    
    CC: jkbradley
    
    Author: Xiangrui Meng <meng@databricks.com>
    
    Closes #1849 from mengxr/ml-blas and squashes the following commits:
    
    ba583a2 [Xiangrui Meng] exclude Vector.copy
    a4d7d2f [Xiangrui Meng] Merge branch 'master' into ml-blas
    6edeab9 [Xiangrui Meng] address comments
    940bdeb [Xiangrui Meng] rename MLlibBLAS to BLAS
    c2a38bc [Xiangrui Meng] enhance dot tests
    4cfaac4 [Xiangrui Meng] add apache header
    48d01d2 [Xiangrui Meng] add tests for zeros and copy
    3b882b1 [Xiangrui Meng] use blas.scal in gradient
    735eb23 [Xiangrui Meng] remove d from BLAS routines
    d2d7d3c [Xiangrui Meng] update gradient and lbfgs
    7f78186 [Xiangrui Meng] add zeros to Vectors; add dscal and dcopy to BLAS
    14e6645 [Xiangrui Meng] add ddot
    cbb8273 [Xiangrui Meng] add daxpy test
    07db0bb [Xiangrui Meng] Merge branch 'master' into ml-blas
    e8c326d [Xiangrui Meng] axpy
    9038d94e
    History
    [SPARK-2923][MLLIB] Implement some basic BLAS routines
    Xiangrui Meng authored
    Having some basic BLAS operations implemented in MLlib can help simplify the current implementation and improve some performance.
    
    Tested on my local machine:
    
    ~~~
    bin/spark-submit --class org.apache.spark.examples.mllib.BinaryClassification \
    examples/target/scala-*/spark-examples-*.jar --algorithm LR --regType L2 \
    --regParam 1.0 --numIterations 1000 ~/share/data/rcv1.binary/rcv1_train.binary
    ~~~
    
    1. before: ~1m
    2. after: ~30s
    
    CC: jkbradley
    
    Author: Xiangrui Meng <meng@databricks.com>
    
    Closes #1849 from mengxr/ml-blas and squashes the following commits:
    
    ba583a2 [Xiangrui Meng] exclude Vector.copy
    a4d7d2f [Xiangrui Meng] Merge branch 'master' into ml-blas
    6edeab9 [Xiangrui Meng] address comments
    940bdeb [Xiangrui Meng] rename MLlibBLAS to BLAS
    c2a38bc [Xiangrui Meng] enhance dot tests
    4cfaac4 [Xiangrui Meng] add apache header
    48d01d2 [Xiangrui Meng] add tests for zeros and copy
    3b882b1 [Xiangrui Meng] use blas.scal in gradient
    735eb23 [Xiangrui Meng] remove d from BLAS routines
    d2d7d3c [Xiangrui Meng] update gradient and lbfgs
    7f78186 [Xiangrui Meng] add zeros to Vectors; add dscal and dcopy to BLAS
    14e6645 [Xiangrui Meng] add ddot
    cbb8273 [Xiangrui Meng] add daxpy test
    07db0bb [Xiangrui Meng] Merge branch 'master' into ml-blas
    e8c326d [Xiangrui Meng] axpy