Skip to content
Snippets Groups Projects
Commit a338834f authored by Bernardo Gomez Palacio's avatar Bernardo Gomez Palacio Committed by Patrick Wendell
Browse files

SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version

The Maven Profiles that refer to hadoopX, e.g. `hadoop2.4`, should set the expected
`hadoop.version` and `yarn.version`.

e.g.

```
<profile>
      <id>hadoop-2.4</id>
      <properties>
        <hadoop.version>2.4.0</hadoop.version>
         <yarn.version>${hadoop.version}</yarn.version>
        <protobuf.version>2.5.0</protobuf.version>
        <jets3t.version>0.9.0</jets3t.version>
      </properties>
</profile>
```

Builds can still define the `-Dhadoop.version` option but this will correctly default the
Hadoop Version to the one that is expected according the profile that is selected.

e.g.

```$ mvn -P hadoop-2.4,yarn clean install```
or

```$ mvn -P hadoop-0.23,yarn clean install```

[ticket] : https://issues.apache.org/jira/browse/SPARK-2026

Author      : berngp
Reviewer    : ?

Author: Bernardo Gomez Palacio <bernardo.gomezpalacio@gmail.com>

Closes #998 from berngp/feature/SPARK-2026 and squashes the following commits:

07ba4f7 [Bernardo Gomez Palacio] SPARK-2026: Maven Hadoop Profiles Should Set The Hadoop Version
parent 7b877b27
No related branches found
No related tags found
No related merge requests found
......@@ -209,14 +209,14 @@
<repository>
<id>spring-releases</id>
<name>Spring Release Repository</name>
<url>http://repo.spring.io/libs-release</url>
<url>http://repo.spring.io/libs-release</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repository>
</repositories>
<dependencyManagement>
......@@ -987,11 +987,15 @@
<artifactId>avro</artifactId>
</dependency>
</dependencies>
<properties>
<hadoop.version>0.23.10</hadoop.version>
</properties>
</profile>
<profile>
<id>hadoop-2.2</id>
<properties>
<hadoop.version>2.2.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
</properties>
</profile>
......@@ -999,6 +1003,7 @@
<profile>
<id>hadoop-2.3</id>
<properties>
<hadoop.version>2.3.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
<jets3t.version>0.9.0</jets3t.version>
</properties>
......@@ -1007,6 +1012,7 @@
<profile>
<id>hadoop-2.4</id>
<properties>
<hadoop.version>2.4.0</hadoop.version>
<protobuf.version>2.5.0</protobuf.version>
<jets3t.version>0.9.0</jets3t.version>
</properties>
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment