c. AWS ParallelCluster

The Cluster

So what did we just create?

Well, roughly this:

pcluster

Specifically you’ll notice on the Cloud9 instance there’s a file ~/environment/config.ini, when we open it we see:

vim ~/environment/config.ini

pcluster config

This is the cluster’s configuration file. It defines everything about the cluster, such as:

The hpc-cluster HPC Environment

  • Ubuntu 18.04 operating system (apt),
  • SLURM Workload Manager (sbatch, srun, squeue, sinfo),
  • Environment Modules (module avail),
  • Spack Package Manager (spack),
  • FSx Lustre scratch filesystem (/scratch),
  • A Master node for job scheduling, interactive development, and
  • Multiple Compute nodes that scale up from 0 based on the jobs you submit.

Master and Compute Nodes

The default hpc-cluster configuration is described in the table below:

Node # Nodes Longevity Instance Type EFA
    Purpose
Master 1 persistent c5.2xlarge No
  • Job Submission
  • Interactive Analysis and Development
  • Visualization
Compute 0 - 10 auto-scaling c5.18xlarge No
  • Batch Processing
  • Tightly or Loosely Coupled Job Processing

Filesystem Shares

The hpc-cluster Master and Compute nodes share a number of common filesystems as described in the table below. Leverage shared filesystems to reduce data movement and duplication.

Path Recommended Use Storage Backend Size
/home EBS (gp2) 200 GB
/shared Application source code, compiled binaries, user-space software and job scripts EBS (gp2) 500 GB
/scratch Reference data and intermediate output from jobs FSx Lustre (SCRATCH2) 1.2 TB
/opt/slurm Do NOT write to this path - -
/opt/intel Do NOT write to this path - -
s3://aws-hpc-quickstart-datarepository****** Input files and final output files.
💡Note: this bucket is not directly mounted, but instances do have read/write access via the aws s3 command		 S3 N/A

To identify mounted filesystems and the available storage on the cluster:

showmount -e localhost
df -h

Filesystems