Skip to content
  • Bernardo Gomez Palacio's avatar
    a338834f
    SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version · a338834f
    Bernardo Gomez Palacio authored
    The Maven Profiles that refer to hadoopX, e.g. `hadoop2.4`, should set the expected
    `hadoop.version` and `yarn.version`.
    
    e.g.
    
    ```
    <profile>
          <id>hadoop-2.4</id>
          <properties>
            <hadoop.version>2.4.0</hadoop.version>
             <yarn.version>${hadoop.version}</yarn.version>
            <protobuf.version>2.5.0</protobuf.version>
            <jets3t.version>0.9.0</jets3t.version>
          </properties>
    </profile>
    ```
    
    Builds can still define the `-Dhadoop.version` option but this will correctly default the
    Hadoop Version to the one that is expected according the profile that is selected.
    
    e.g.
    
    ```$ mvn -P hadoop-2.4,yarn clean install```
    or
    
    ```$ mvn -P hadoop-0.23,yarn clean install```
    
    [ticket] : https://issues.apache.org/jira/browse/SPARK-2026
    
    Author      : berngp
    Reviewer    : ?
    
    Author: Bernardo Gomez Palacio <bernardo.gomezpalacio@gmail.com>
    
    Closes #998 from berngp/feature/SPARK-2026 and squashes the following commits:
    
    07ba4f7 [Bernardo Gomez Palacio] SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version
    a338834f
    SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version
    Bernardo Gomez Palacio authored
    The Maven Profiles that refer to hadoopX, e.g. `hadoop2.4`, should set the expected
    `hadoop.version` and `yarn.version`.
    
    e.g.
    
    ```
    <profile>
          <id>hadoop-2.4</id>
          <properties>
            <hadoop.version>2.4.0</hadoop.version>
             <yarn.version>${hadoop.version}</yarn.version>
            <protobuf.version>2.5.0</protobuf.version>
            <jets3t.version>0.9.0</jets3t.version>
          </properties>
    </profile>
    ```
    
    Builds can still define the `-Dhadoop.version` option but this will correctly default the
    Hadoop Version to the one that is expected according the profile that is selected.
    
    e.g.
    
    ```$ mvn -P hadoop-2.4,yarn clean install```
    or
    
    ```$ mvn -P hadoop-0.23,yarn clean install```
    
    [ticket] : https://issues.apache.org/jira/browse/SPARK-2026
    
    Author      : berngp
    Reviewer    : ?
    
    Author: Bernardo Gomez Palacio <bernardo.gomezpalacio@gmail.com>
    
    Closes #998 from berngp/feature/SPARK-2026 and squashes the following commits:
    
    07ba4f7 [Bernardo Gomez Palacio] SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version
Loading