Skip to content
Snippets Groups Projects
  • hyukjinkwon's avatar
    ceaf77ae
    [SPARK-18692][BUILD][DOCS] Test Java 8 unidoc build on Jenkins · ceaf77ae
    hyukjinkwon authored
    ## What changes were proposed in this pull request?
    
    This PR proposes to run Spark unidoc to test Javadoc 8 build as Javadoc 8 is easily re-breakable.
    
    There are several problems with it:
    
    - It introduces little extra bit of time to run the tests. In my case, it took 1.5 mins more (`Elapsed :[94.8746569157]`). How it was tested is described in "How was this patch tested?".
    
    - > One problem that I noticed was that Unidoc appeared to be processing test sources: if we can find a way to exclude those from being processed in the first place then that might significantly speed things up.
    
      (see  joshrosen's [comment](https://issues.apache.org/jira/browse/SPARK-18692?focusedCommentId=15947627&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15947627))
    
    To complete this automated build, It also suggests to fix existing Javadoc breaks / ones introduced by test codes as described above.
    
    There fixes are similar instances that previously fixed. Please refer https://github.com/apache/spark/pull/15999 and https://github.com/apache/spark/pull/16013
    
    Note that this only fixes **errors** not **warnings**. Please see my observation https://github.com/apache/spark/pull/17389#issuecomment-288438704 for spurious errors by warnings.
    
    ## How was this patch tested?
    
    Manually via `jekyll build` for building tests. Also, tested via running `./dev/run-tests`.
    
    This was tested via manually adding `time.time()` as below:
    
    ```diff
         profiles_and_goals = build_profiles + sbt_goals
    
         print("[info] Building Spark unidoc (w/Hive 1.2.1) using SBT with these arguments: ",
               " ".join(profiles_and_goals))
    
    +    import time
    +    st = time.time()
         exec_sbt(profiles_and_goals)
    +    print("Elapsed :[%s]" % str(time.time() - st))
    ```
    
    produces
    
    ```
    ...
    ========================================================================
    Building Unidoc API Documentation
    ========================================================================
    ...
    [info] Main Java API documentation successful.
    ...
    Elapsed :[94.8746569157]
    ...
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #17477 from HyukjinKwon/SPARK-18692.
    ceaf77ae
    History
    [SPARK-18692][BUILD][DOCS] Test Java 8 unidoc build on Jenkins
    hyukjinkwon authored
    ## What changes were proposed in this pull request?
    
    This PR proposes to run Spark unidoc to test Javadoc 8 build as Javadoc 8 is easily re-breakable.
    
    There are several problems with it:
    
    - It introduces little extra bit of time to run the tests. In my case, it took 1.5 mins more (`Elapsed :[94.8746569157]`). How it was tested is described in "How was this patch tested?".
    
    - > One problem that I noticed was that Unidoc appeared to be processing test sources: if we can find a way to exclude those from being processed in the first place then that might significantly speed things up.
    
      (see  joshrosen's [comment](https://issues.apache.org/jira/browse/SPARK-18692?focusedCommentId=15947627&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15947627))
    
    To complete this automated build, It also suggests to fix existing Javadoc breaks / ones introduced by test codes as described above.
    
    There fixes are similar instances that previously fixed. Please refer https://github.com/apache/spark/pull/15999 and https://github.com/apache/spark/pull/16013
    
    Note that this only fixes **errors** not **warnings**. Please see my observation https://github.com/apache/spark/pull/17389#issuecomment-288438704 for spurious errors by warnings.
    
    ## How was this patch tested?
    
    Manually via `jekyll build` for building tests. Also, tested via running `./dev/run-tests`.
    
    This was tested via manually adding `time.time()` as below:
    
    ```diff
         profiles_and_goals = build_profiles + sbt_goals
    
         print("[info] Building Spark unidoc (w/Hive 1.2.1) using SBT with these arguments: ",
               " ".join(profiles_and_goals))
    
    +    import time
    +    st = time.time()
         exec_sbt(profiles_and_goals)
    +    print("Elapsed :[%s]" % str(time.time() - st))
    ```
    
    produces
    
    ```
    ...
    ========================================================================
    Building Unidoc API Documentation
    ========================================================================
    ...
    [info] Main Java API documentation successful.
    ...
    Elapsed :[94.8746569157]
    ...
    
    Author: hyukjinkwon <gurwls223@gmail.com>
    
    Closes #17477 from HyukjinKwon/SPARK-18692.