Skip to content
  • Marcelo Vanzin's avatar
    a680562a
    [SPARK-14744][EXAMPLES] Clean up examples packaging, remove outdated examples. · a680562a
    Marcelo Vanzin authored
    First, make all dependencies in the examples module provided, and explicitly
    list a couple of ones that somehow are promoted to compile by maven. This
    means that to run streaming examples, the streaming connector package needs
    to be provided to run-examples using --packages or --jars, just like regular
    apps.
    
    Also, remove a couple of outdated examples. HBase has had Spark bindings for
    a while and is even including them in the HBase distribution in the next
    version, making the examples obsolete. The same applies to Cassandra, which
    seems to have a proper Spark binding library already.
    
    I just tested the build, which passes, and ran SparkPi. The examples jars
    directory now has only two jars:
    
    ```
    $ ls -1 examples/target/scala-2.11/jars/
    scopt_2.11-3.3.0.jar
    spark-examples_2.11-2.0.0-SNAPSHOT.jar
    ```
    
    Author: Marcelo Vanzin <vanzin@cloudera.com>
    
    Closes #12544 from vanzin/SPARK-14744.
    a680562a
    [SPARK-14744][EXAMPLES] Clean up examples packaging, remove outdated examples.
    Marcelo Vanzin authored
    First, make all dependencies in the examples module provided, and explicitly
    list a couple of ones that somehow are promoted to compile by maven. This
    means that to run streaming examples, the streaming connector package needs
    to be provided to run-examples using --packages or --jars, just like regular
    apps.
    
    Also, remove a couple of outdated examples. HBase has had Spark bindings for
    a while and is even including them in the HBase distribution in the next
    version, making the examples obsolete. The same applies to Cassandra, which
    seems to have a proper Spark binding library already.
    
    I just tested the build, which passes, and ran SparkPi. The examples jars
    directory now has only two jars:
    
    ```
    $ ls -1 examples/target/scala-2.11/jars/
    scopt_2.11-3.3.0.jar
    spark-examples_2.11-2.0.0-SNAPSHOT.jar
    ```
    
    Author: Marcelo Vanzin <vanzin@cloudera.com>
    
    Closes #12544 from vanzin/SPARK-14744.
Loading