Skip to content
Snippets Groups Projects
  • Andrew Or's avatar
    18c2c925
    [SPARK-14861][SQL] Replace internal usages of SQLContext with SparkSession · 18c2c925
    Andrew Or authored
    ## What changes were proposed in this pull request?
    
    In Spark 2.0, `SparkSession` is the new thing. Internally we should stop using `SQLContext` everywhere since that's supposed to be not the main user-facing API anymore.
    
    In this patch I took care to not break any public APIs. The one place that's suspect is `o.a.s.ml.source.libsvm.DefaultSource`, but according to mengxr it's not supposed to be public so it's OK to change the underlying `FileFormat` trait.
    
    **Reviewers**: This is a big patch that may be difficult to review but the changes are actually really straightforward. If you prefer I can break it up into a few smaller patches, but it will delay the progress of this issue a little.
    
    ## How was this patch tested?
    
    No change in functionality intended.
    
    Author: Andrew Or <andrew@databricks.com>
    
    Closes #12625 from andrewor14/spark-session-refactor.
    18c2c925
    History
    [SPARK-14861][SQL] Replace internal usages of SQLContext with SparkSession
    Andrew Or authored
    ## What changes were proposed in this pull request?
    
    In Spark 2.0, `SparkSession` is the new thing. Internally we should stop using `SQLContext` everywhere since that's supposed to be not the main user-facing API anymore.
    
    In this patch I took care to not break any public APIs. The one place that's suspect is `o.a.s.ml.source.libsvm.DefaultSource`, but according to mengxr it's not supposed to be public so it's OK to change the underlying `FileFormat` trait.
    
    **Reviewers**: This is a big patch that may be difficult to review but the changes are actually really straightforward. If you prefer I can break it up into a few smaller patches, but it will delay the progress of this issue a little.
    
    ## How was this patch tested?
    
    No change in functionality intended.
    
    Author: Andrew Or <andrew@databricks.com>
    
    Closes #12625 from andrewor14/spark-session-refactor.