-
- Downloads
[SPARK-8777] [SQL] Add random data generator test utilities to Spark SQL
This commit adds a set of random data generation utilities to Spark SQL, for use in its own unit tests. - `RandomDataGenerator.forType(DataType)` returns an `Option[() => Any]` that, if defined, contains a function for generating random values for the given DataType. The random values use the external representations for the given DataType (for example, for DateType we return `java.sql.Date` instances instead of longs). - `DateTypeTestUtilities` defines some convenience fields for looping over instances of data types. For example, `numericTypes` holds `DataType` instances for all supported numeric types. These constants will help us to raise the level of abstraction in our tests. For example, it's now very easy to write a test which is parameterized by all common data types. Author: Josh Rosen <joshrosen@databricks.com> Closes #7176 from JoshRosen/sql-random-data-generators and squashes the following commits: f71634d [Josh Rosen] Roll back ScalaCheck usage e0d7d49 [Josh Rosen] Bump ScalaCheck version in LICENSE 89d86b1 [Josh Rosen] Bump ScalaCheck version. 0c20905 [Josh Rosen] Initial attempt at using ScalaCheck. b55875a [Josh Rosen] Generate doubles and floats over entire possible range. 5acdd5c [Josh Rosen] Infinity and NaN are interesting. ab76cbd [Josh Rosen] Move code to Catalyst package. d2b4a4a [Josh Rosen] Add random data generator test utilities to Spark SQL.
Showing
- sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala 158 additions, 0 deletions...test/scala/org/apache/spark/sql/RandomDataGenerator.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGeneratorSuite.scala 98 additions, 0 deletions...scala/org/apache/spark/sql/RandomDataGeneratorSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/types/DataTypeTestUtils.scala 63 additions, 0 deletions.../scala/org/apache/spark/sql/types/DataTypeTestUtils.scala
Loading
Please register or sign in to comment