Skip to content
Snippets Groups Projects
  1. May 27, 2011
  2. May 26, 2011
  3. May 24, 2011
  4. May 22, 2011
  5. May 21, 2011
  6. May 20, 2011
  7. May 19, 2011
  8. May 17, 2011
  9. May 13, 2011
  10. May 12, 2011
  11. May 09, 2011
  12. May 03, 2011
  13. Apr 28, 2011
    • Ankur Dave's avatar
      Error handling for disk I/O in DiskSpillingCache · a4c04f3f
      Ankur Dave authored
      Also renamed the property spark.DiskSpillingCache.cacheDir to spark.diskSpillingCache.cacheDir in order to follow conventions.
      a4c04f3f
    • Ankur Dave's avatar
    • Ankur Dave's avatar
      Added DiskSpillingCache · e30313aa
      Ankur Dave authored
      DiskSpillingCache is a BoundedMemoryCache that spills entries to disk
      when it runs out of space. Currently the implementation is very
      simple. In particular, it's missing the following features:
      
      - Error handling for disk I/O, including checking of disk space levels
      - Bringing an entry back into memory after fetching it from disk
      
      In addition, here are some features that aren't critical but should be
      implemented soon:
      
      - Spilling based on a user-set priority in addition to LRU
      - Caching into a subdirectory of spark.DiskSpillingCache.cacheDir
        rather than the root directory
      e30313aa
Loading