Skip to content
  • Josh Rosen's avatar
    9fd7a2f0
    [SPARK-10359][PROJECT-INFRA] Use more random number in... · 9fd7a2f0
    Josh Rosen authored
    [SPARK-10359][PROJECT-INFRA] Use more random number in dev/test-dependencies.sh; fix version switching
    
    This patch aims to fix another potential source of flakiness in the `dev/test-dependencies.sh` script.
    
    pwendell's original patch and my version used `$(date +%s | tail -c6)` to generate a suffix to use when installing temporary Spark versions into the local Maven cache, but this value only changes once per second and thus is highly collision-prone when concurrent builds launch on AMPLab Jenkins. In order to reduce the potential for conflicts, this patch updates the script to call Python's random number generator instead.
    
    I also fixed a bug in how we captured the original project version; the bug was causing the exit handler code to fail.
    
    Author: Josh Rosen <joshrosen@databricks.com>
    
    Closes #10558 from JoshRosen/build-dep-tests-round-3.
    9fd7a2f0
    [SPARK-10359][PROJECT-INFRA] Use more random number in...
    Josh Rosen authored
    [SPARK-10359][PROJECT-INFRA] Use more random number in dev/test-dependencies.sh; fix version switching
    
    This patch aims to fix another potential source of flakiness in the `dev/test-dependencies.sh` script.
    
    pwendell's original patch and my version used `$(date +%s | tail -c6)` to generate a suffix to use when installing temporary Spark versions into the local Maven cache, but this value only changes once per second and thus is highly collision-prone when concurrent builds launch on AMPLab Jenkins. In order to reduce the potential for conflicts, this patch updates the script to call Python's random number generator instead.
    
    I also fixed a bug in how we captured the original project version; the bug was causing the exit handler code to fail.
    
    Author: Josh Rosen <joshrosen@databricks.com>
    
    Closes #10558 from JoshRosen/build-dep-tests-round-3.
Loading