Skip to content
Snippets Groups Projects
  • Daniel Darabos's avatar
    44d3a6a7
    [SPARK-3342] Add SSDs to block device mapping · 44d3a6a7
    Daniel Darabos authored
    On `m3.2xlarge` instances the 2x80GB SSDs are inaccessible if not added to the block device mapping when the instance is created. They work when added with this patch. I have not tested this with other instance types, and I do not know much about this script and EC2 deployment in general. Maybe this code needs to depend on the instance type.
    
    The requirement for this mapping is described in the AWS docs at:
    http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html#InstanceStore_UsageScenarios
    
    "For M3 instances, you must specify instance store volumes in the block
    device mapping for the instance. When you launch an M3 instance, we
    ignore any instance store volumes specified in the block device mapping
    for the AMI."
    
    Author: Daniel Darabos <darabos.daniel@gmail.com>
    
    Closes #2081 from darabos/patch-1 and squashes the following commits:
    
    1ceb2c8 [Daniel Darabos] Use %d string interpolation instead of {}.
    a1854d7 [Daniel Darabos] Only specify ephemeral device mapping for M3.
    e0d9e37 [Daniel Darabos] Create ephemeral device mapping based on get_num_disks().
    6b116a6 [Daniel Darabos] Add SSDs to block device mapping
    44d3a6a7
    History
    [SPARK-3342] Add SSDs to block device mapping
    Daniel Darabos authored
    On `m3.2xlarge` instances the 2x80GB SSDs are inaccessible if not added to the block device mapping when the instance is created. They work when added with this patch. I have not tested this with other instance types, and I do not know much about this script and EC2 deployment in general. Maybe this code needs to depend on the instance type.
    
    The requirement for this mapping is described in the AWS docs at:
    http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html#InstanceStore_UsageScenarios
    
    "For M3 instances, you must specify instance store volumes in the block
    device mapping for the instance. When you launch an M3 instance, we
    ignore any instance store volumes specified in the block device mapping
    for the AMI."
    
    Author: Daniel Darabos <darabos.daniel@gmail.com>
    
    Closes #2081 from darabos/patch-1 and squashes the following commits:
    
    1ceb2c8 [Daniel Darabos] Use %d string interpolation instead of {}.
    a1854d7 [Daniel Darabos] Only specify ephemeral device mapping for M3.
    e0d9e37 [Daniel Darabos] Create ephemeral device mapping based on get_num_disks().
    6b116a6 [Daniel Darabos] Add SSDs to block device mapping
hpvm-tensor-rt NaN GiB