-
- Downloads
[SPARK-14850][ML] convert primitive array from/to unsafe array directly in VectorUDT/MatrixUDT
## What changes were proposed in this pull request? This PR adds `fromPrimitiveArray` and `toPrimitiveArray` in `UnsafeArrayData`, so that we can do the conversion much faster in VectorUDT/MatrixUDT. ## How was this patch tested? existing tests and new test suite `UnsafeArraySuite` Author: Wenchen Fan <wenchen@databricks.com> Closes #12640 from cloud-fan/ml.
Showing
- mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala 5 additions, 6 deletions...c/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
- mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala 4 additions, 5 deletions...rc/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
- mllib/src/test/scala/org/apache/spark/mllib/linalg/UDTSerializationBenchmark.scala 70 additions, 0 deletions...apache/spark/mllib/linalg/UDTSerializationBenchmark.scala
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java 62 additions, 2 deletions...pache/spark/sql/catalyst/expressions/UnsafeArrayData.java
- sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeMapData.java 1 addition, 1 deletion.../apache/spark/sql/catalyst/expressions/UnsafeMapData.java
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala 44 additions, 0 deletions...org/apache/spark/sql/catalyst/util/UnsafeArraySuite.scala
Loading
Please register or sign in to comment