-
- Downloads
Spark 1246 add min max to stat counter
Here's the addition of min and max to statscounter.py and min and max methods to rdd.py. Author: Dan McClary <dan.mcclary@gmail.com> Closes #144 from dwmclary/SPARK-1246-add-min-max-to-stat-counter and squashes the following commits: fd3fd4b [Dan McClary] fixed error, updated test 82cde0e [Dan McClary] flipped incorrectly assigned inf values in StatCounter 5d96799 [Dan McClary] added max and min to StatCounter repr for pyspark 21dd366 [Dan McClary] added max and min to StatCounter output, updated doc 1a97558 [Dan McClary] added max and min to StatCounter output, updated doc a5c13b0 [Dan McClary] Added min and max to Scala and Java RDD, added min and max to StatCounter ed67136 [Dan McClary] broke min/max out into separate transaction, added to rdd.py 1e7056d [Dan McClary] added underscore to getBucket 37a7dea [Dan McClary] cleaned up boundaries for histogram -- uses real min/max when buckets are derived 29981f2 [Dan McClary] fixed indentation on doctest comment eaf89d9 [Dan McClary] added correct doctest for histogram 4916016 [Dan McClary] added histogram method, added max and min to statscounter
Showing
- core/src/main/scala/org/apache/spark/api/java/JavaRDDLike.scala 20 additions, 0 deletions...rc/main/scala/org/apache/spark/api/java/JavaRDDLike.scala
- core/src/main/scala/org/apache/spark/rdd/RDD.scala 12 additions, 0 deletionscore/src/main/scala/org/apache/spark/rdd/RDD.scala
- core/src/main/scala/org/apache/spark/util/StatCounter.scala 16 additions, 2 deletionscore/src/main/scala/org/apache/spark/util/StatCounter.scala
- core/src/test/scala/org/apache/spark/PartitioningSuite.scala 2 additions, 0 deletionscore/src/test/scala/org/apache/spark/PartitioningSuite.scala
- core/src/test/scala/org/apache/spark/rdd/RDDSuite.scala 2 additions, 0 deletionscore/src/test/scala/org/apache/spark/rdd/RDDSuite.scala
- python/pyspark/rdd.py 19 additions, 0 deletionspython/pyspark/rdd.py
- python/pyspark/statcounter.py 22 additions, 3 deletionspython/pyspark/statcounter.py
Loading
Please register or sign in to comment