Skip to content
Snippets Groups Projects
Commit 37fefa66 authored by felixcheung's avatar felixcheung Committed by Shivaram Venkataraman
Browse files

[SPARK-12168][SPARKR] Add automated tests for conflicted function in R

Currently this is reported when loading the SparkR package in R (probably would add is.nan)
```
Loading required package: methods

Attaching package: ‘SparkR’

The following objects are masked from ‘package:stats’:

    cov, filter, lag, na.omit, predict, sd, var

The following objects are masked from ‘package:base’:

    colnames, colnames<-, intersect, rank, rbind, sample, subset,
    summary, table, transform
```

Adding this test adds an automated way to track changes to masked method.
Also, the second part of this test check for those functions that would not be accessible without namespace/package prefix.

Incidentally, this might point to how we would fix those inaccessible functions in base or stats.
Looking for feedback for adding this test.

Author: felixcheung <felixcheung_m@hotmail.com>

Closes #10171 from felixcheung/rmaskedtest.
parent 3e84ef0a
No related branches found
No related tags found
No related merge requests found
...@@ -271,10 +271,10 @@ export("as.DataFrame", ...@@ -271,10 +271,10 @@ export("as.DataFrame",
"createExternalTable", "createExternalTable",
"dropTempTable", "dropTempTable",
"jsonFile", "jsonFile",
"read.json",
"loadDF", "loadDF",
"parquetFile", "parquetFile",
"read.df", "read.df",
"read.json",
"read.parquet", "read.parquet",
"read.text", "read.text",
"sql", "sql",
......
...@@ -17,6 +17,29 @@ ...@@ -17,6 +17,29 @@
context("test functions in sparkR.R") context("test functions in sparkR.R")
test_that("Check masked functions", {
# Check that we are not masking any new function from base, stats, testthat unexpectedly
masked <- conflicts(detail = TRUE)$`package:SparkR`
expect_true("describe" %in% masked) # only when with testthat..
func <- lapply(masked, function(x) { capture.output(showMethods(x))[[1]] })
funcSparkROrEmpty <- grepl("\\(package SparkR\\)$|^$", func)
maskedBySparkR <- masked[funcSparkROrEmpty]
expect_equal(length(maskedBySparkR), 18)
expect_equal(sort(maskedBySparkR), sort(c("describe", "cov", "filter", "lag", "na.omit",
"predict", "sd", "var", "colnames", "colnames<-",
"intersect", "rank", "rbind", "sample", "subset",
"summary", "table", "transform")))
# above are those reported as masked when `library(SparkR)`
# note that many of these methods are still callable without base:: or stats:: prefix
# there should be a test for each of these, except followings, which are currently "broken"
funcHasAny <- unlist(lapply(masked, function(x) {
any(grepl("=\"ANY\"", capture.output(showMethods(x)[-1])))
}))
maskedCompletely <- masked[!funcHasAny]
expect_equal(length(maskedCompletely), 4)
expect_equal(sort(maskedCompletely), sort(c("cov", "filter", "sample", "table")))
})
test_that("repeatedly starting and stopping SparkR", { test_that("repeatedly starting and stopping SparkR", {
for (i in 1:4) { for (i in 1:4) {
sc <- sparkR.init() sc <- sparkR.init()
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment