-
- Downloads
[SQL] SPARK-1424 Generalize insertIntoTable functions on SchemaRDDs
This makes it possible to create tables and insert into them using the DSL and SQL for the scala and java apis. Author: Michael Armbrust <michael@databricks.com> Closes #354 from marmbrus/insertIntoTable and squashes the following commits: 6c6f227 [Michael Armbrust] Create random temporary files in python parquet unit tests. f5e6d5c [Michael Armbrust] Merge remote-tracking branch 'origin/master' into insertIntoTable 765c506 [Michael Armbrust] Add to JavaAPI. 77b512c [Michael Armbrust] typos. 5c3ef95 [Michael Armbrust] use names for boolean args. 882afdf [Michael Armbrust] Change createTableAs to saveAsTable. Clean up api annotations. d07d94b [Michael Armbrust] Add tests, support for creating parquet files and hive tables. fa3fe81 [Michael Armbrust] Make insertInto available on JavaSchemaRDD as well. Add createTableAs function.
Showing
- python/pyspark/sql.py 10 additions, 4 deletionspython/pyspark/sql.py
- sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala 50 additions, 7 deletions...core/src/main/scala/org/apache/spark/sql/SQLContext.scala
- sql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala 5 additions, 23 deletionssql/core/src/main/scala/org/apache/spark/sql/SchemaRDD.scala
- sql/core/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala 54 additions, 5 deletions...e/src/main/scala/org/apache/spark/sql/SchemaRDDLike.scala
- sql/core/src/main/scala/org/apache/spark/sql/api/java/JavaSQLContext.scala 58 additions, 20 deletions.../scala/org/apache/spark/sql/api/java/JavaSQLContext.scala
- sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetRelation.scala 8 additions, 3 deletions.../scala/org/apache/spark/sql/parquet/ParquetRelation.scala
- sql/core/src/test/scala/org/apache/spark/sql/InsertIntoSuite.scala 148 additions, 0 deletions...src/test/scala/org/apache/spark/sql/InsertIntoSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala 7 additions, 4 deletionssql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala
- sql/core/src/test/scala/org/apache/spark/sql/TestData.scala 2 additions, 1 deletionsql/core/src/test/scala/org/apache/spark/sql/TestData.scala
- sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala 4 additions, 4 deletions...t/scala/org/apache/spark/sql/execution/PlannerSuite.scala
- sql/core/src/test/scala/org/apache/spark/sql/parquet/ParquetQuerySuite.scala 7 additions, 34 deletions...cala/org/apache/spark/sql/parquet/ParquetQuerySuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala 17 additions, 1 deletion...rc/main/scala/org/apache/spark/sql/hive/HiveContext.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 11 additions, 2 deletions...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
- sql/hive/src/test/scala/org/apache/spark/sql/QueryTest.scala 77 additions, 0 deletionssql/hive/src/test/scala/org/apache/spark/sql/QueryTest.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala 77 additions, 0 deletions.../org/apache/spark/sql/hive/InsertIntoHiveTableSuite.scala
- sql/hive/src/test/scala/org/apache/spark/sql/parquet/HiveParquetSuite.scala 0 additions, 52 deletions...scala/org/apache/spark/sql/parquet/HiveParquetSuite.scala
Loading
Please register or sign in to comment