Among other things, should prevent OutOfMemoryErrors in some daemon threads
(such as the network manager) from causing a spark executor to enter a state
where it cannot make progress but does not report an error.

matching with data locality hints from storage systems.

SPARK-617 #resolve

Blockmanager fix

Incorporated Matei's suggestions. Tested with 5 producer(consumer) threads each doing 50k puts (gets), took 15 minutes to run, no errors or deadlocks.

1. Changed the lock structure of BlockManager by replacing the 337 coarse-grained locks to use BlockInfo objects as per-block fine-grained locks.
2. Changed the MemoryStore lock structure by making the block putting threads lock on a different object (not the memory store) thus making sure putting threads minimally blocks to the getting treads.
3. Added spark.storage.ThreadingTest to stress test the BlockManager using 5 block producer and 5 block consumer threads.

- Don't report a job as finishing multiple times
- Don't show state of workers as LOADING when they're running
- Show start and finish times in web UI
- Sort web UI tables by ID and time by default

Fix check for existing instances during spark-ec2 launch

Remove unnecessary hash-map put in MemoryStore

Cancel spot instance requests when exiting spark-ec2

Fix minor typos in quickstart and Scala programming guides

a locally computed operation

Conflicts:

	core/src/main/scala/spark/storage/BlockManagerMaster.scala

Added a method to report slave memory status; force serialize accumulator update in local mode.

Conflicts:
	core/src/main/scala/spark/Dependency.scala
	core/src/main/scala/spark/rdd/CoGroupedRDD.scala
	core/src/main/scala/spark/rdd/ShuffledRDD.scala

Allow EC2 script to stop/destroy cluster after master/slave failures

Tweaked run file to live more happily with typesafe's debian package

Support for Hadoop 2 distributions such as cdh4

Two trivial commits to test JIRA integration

Use SPARK_MASTER_IP if it is set in start-slaves.sh.