Skip to content
Snippets Groups Projects
Commit f21e2da0 authored by Oscar D. Lara Yejas's avatar Oscar D. Lara Yejas Committed by Shivaram Venkataraman
Browse files

[SPARK-10807] [SPARKR] Added as.data.frame as a synonym for collect

Created method as.data.frame as a synonym for collect().

Author: Oscar D. Lara Yejas <olarayej@mail.usf.edu>
Author: olarayej <oscar.lara.yejas@us.ibm.com>
Author: Oscar D. Lara Yejas <oscar.lara.yejas@us.ibm.com>

Closes #8908 from olarayej/SPARK-10807.
parent 89ea0041
No related branches found
No related tags found
No related merge requests found
......@@ -247,3 +247,5 @@ export("structField",
"structType.jobj",
"structType.structField",
"print.structType")
export("as.data.frame")
\ No newline at end of file
......@@ -1848,3 +1848,28 @@ setMethod("crosstab",
sct <- callJMethod(statFunctions, "crosstab", col1, col2)
collect(dataFrame(sct))
})
#' This function downloads the contents of a DataFrame into an R's data.frame.
#' Since data.frames are held in memory, ensure that you have enough memory
#' in your system to accommodate the contents.
#'
#' @title Download data from a DataFrame into a data.frame
#' @param x a DataFrame
#' @return a data.frame
#' @rdname as.data.frame
#' @examples \dontrun{
#'
#' irisDF <- createDataFrame(sqlContext, iris)
#' df <- as.data.frame(irisDF[irisDF$Species == "setosa", ])
#' }
setMethod("as.data.frame",
signature(x = "DataFrame"),
function(x, ...) {
# Check if additional parameters have been passed
if (length(list(...)) > 0) {
stop(paste("Unused argument(s): ", paste(list(...), collapse=", ")))
}
collect(x)
}
)
......@@ -983,3 +983,7 @@ setGeneric("glm")
#' @rdname rbind
#' @export
setGeneric("rbind", signature = "...")
#' @rdname as.data.frame
#' @export
setGeneric("as.data.frame")
\ No newline at end of file
......@@ -1327,6 +1327,13 @@ test_that("SQL error message is returned from JVM", {
expect_equal(grepl("Table Not Found: blah", retError), TRUE)
})
test_that("Method as.data.frame as a synonym for collect()", {
irisDF <- createDataFrame(sqlContext, iris)
expect_equal(as.data.frame(irisDF), collect(irisDF))
irisDF2 <- irisDF[irisDF$Species == "setosa", ]
expect_equal(as.data.frame(irisDF2), collect(irisDF2))
})
unlink(parquetPath)
unlink(jsonPath)
unlink(jsonPathNa)
unlink(jsonPathNa)
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment