-
- Downloads
[SPARK-21224][R] Specify a schema by using a DDL-formatted string when reading in R
## What changes were proposed in this pull request? This PR proposes to support a DDL-formetted string as schema as below: ```r mockLines <- c("{\"name\":\"Michael\"}", "{\"name\":\"Andy\", \"age\":30}", "{\"name\":\"Justin\", \"age\":19}") jsonPath <- tempfile(pattern = "sparkr-test", fileext = ".tmp") writeLines(mockLines, jsonPath) df <- read.df(jsonPath, "json", "name STRING, age DOUBLE") collect(df) ``` ## How was this patch tested? Tests added in `test_streaming.R` and `test_sparkSQL.R` and manual tests. Author: hyukjinkwon <gurwls223@gmail.com> Closes #18431 from HyukjinKwon/r-ddl-schema.
Showing
- R/pkg/R/SQLContext.R 26 additions, 12 deletionsR/pkg/R/SQLContext.R
- R/pkg/tests/fulltests/test_sparkSQL.R 18 additions, 2 deletionsR/pkg/tests/fulltests/test_sparkSQL.R
- R/pkg/tests/fulltests/test_streaming.R 23 additions, 0 deletionsR/pkg/tests/fulltests/test_streaming.R
- sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala 0 additions, 15 deletions.../src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala
Loading
Please register or sign in to comment