Skip to content
  • Dongjoon Hyun's avatar
    0077bfcb
    [SPARK-19409][BUILD][TEST-MAVEN] Fix ParquetAvroCompatibilitySuite failure due... · 0077bfcb
    Dongjoon Hyun authored
    [SPARK-19409][BUILD][TEST-MAVEN] Fix ParquetAvroCompatibilitySuite failure due to test dependency on avro
    
    ## What changes were proposed in this pull request?
    
    After using Apache Parquet 1.8.2, `ParquetAvroCompatibilitySuite` fails on **Maven** test. It is because `org.apache.parquet.avro.AvroParquetWriter` in the test code used new `avro 1.8.0` specific class, `LogicalType`. This PR aims to fix the test dependency of `sql/core` module to use avro 1.8.0.
    
    https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-2.7/2530/consoleFull
    
    ```
    ParquetAvroCompatibilitySuite:
    *** RUN ABORTED ***
      java.lang.NoClassDefFoundError: org/apache/avro/LogicalType
      at org.apache.parquet.avro.AvroParquetWriter.writeSupport(AvroParquetWriter.java:144)
    ```
    
    ## How was this patch tested?
    
    Pass the existing test with **Maven**.
    
    ```
    $ build/mvn -Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver test
    ...
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 02:07 h
    [INFO] Finished at: 2017-02-04T05:41:43+00:00
    [INFO] Final Memory: 77M/987M
    [INFO] ------------------------------------------------------------------------
    ```
    
    Author: Dongjoon Hyun <dongjoon@apache.org>
    
    Closes #16795 from dongjoon-hyun/SPARK-19409-2.
    0077bfcb
    [SPARK-19409][BUILD][TEST-MAVEN] Fix ParquetAvroCompatibilitySuite failure due...
    Dongjoon Hyun authored
    [SPARK-19409][BUILD][TEST-MAVEN] Fix ParquetAvroCompatibilitySuite failure due to test dependency on avro
    
    ## What changes were proposed in this pull request?
    
    After using Apache Parquet 1.8.2, `ParquetAvroCompatibilitySuite` fails on **Maven** test. It is because `org.apache.parquet.avro.AvroParquetWriter` in the test code used new `avro 1.8.0` specific class, `LogicalType`. This PR aims to fix the test dependency of `sql/core` module to use avro 1.8.0.
    
    https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-2.7/2530/consoleFull
    
    ```
    ParquetAvroCompatibilitySuite:
    *** RUN ABORTED ***
      java.lang.NoClassDefFoundError: org/apache/avro/LogicalType
      at org.apache.parquet.avro.AvroParquetWriter.writeSupport(AvroParquetWriter.java:144)
    ```
    
    ## How was this patch tested?
    
    Pass the existing test with **Maven**.
    
    ```
    $ build/mvn -Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver test
    ...
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 02:07 h
    [INFO] Finished at: 2017-02-04T05:41:43+00:00
    [INFO] Final Memory: 77M/987M
    [INFO] ------------------------------------------------------------------------
    ```
    
    Author: Dongjoon Hyun <dongjoon@apache.org>
    
    Closes #16795 from dongjoon-hyun/SPARK-19409-2.
Loading