So what did we just create?
Well, roughly this:
Specifically you’ll notice on the Cloud9 instance there’s a file ~/environment/config.ini
, when we open it we see:
vim ~/environment/config.ini
This is the cluster’s configuration file. It defines everything about the cluster, such as:
hpc-cluster
HPC Environmentapt
),sbatch
, srun
, squeue
, sinfo
),module avail
),spack
),/scratch
),The default hpc-cluster
configuration is described in the table below:
Node | # Nodes | Longevity | Instance Type | EFA |
|
---|---|---|---|---|---|
Master | 1 | persistent | c5.2xlarge | No |
|
Compute | 0 - 10 | auto-scaling | c5.18xlarge | No |
|
The hpc-cluster
Master and Compute nodes share a number of common filesystems as described in the table below. Leverage shared filesystems to reduce data movement and duplication.
Path | Recommended Use | Storage Backend | Size |
---|---|---|---|
/home | EBS (gp2) | 200 GB | |
/shared | Application source code, compiled binaries, user-space software and job scripts | EBS (gp2) | 500 GB |
/scratch | Reference data and intermediate output from jobs | FSx Lustre (SCRATCH2) | 1.2 TB |
/opt/slurm | Do NOT write to this path | - | - |
/opt/intel | Do NOT write to this path | - | - |
s3://aws-hpc-quickstart-datarepository****** | Input files and final output files. 💡Note: this bucket is not directly mounted, but instances do have read/write access via the aws s3 command |
S3 | N/A |
To identify mounted filesystems and the available storage on the cluster:
showmount -e localhost
df -h