LBNL Standalone Clusters
ARES
Ares cluster is a standalone cluster in the Supercluster environment.
Cluster Configuration:
Total number of nodes – 21
12 cores, 16 GB per node.
Queue Configuration:
Default wallclock time – 30 minutes
Nodes are dedicated for a single job and cannot be shared across multiple jobs even from the same user.
Job Queue | Destination Queue | User Limits |
---|---|---|
ares_batch | ares_batch | No limit on number of nodes per job. 30 minutes default wallclock limit |
Additional Notes:
Please see the Supercluster Getting Started Page for user guide in details.
PHASIS
The PHASIS cluster will be used for pre-processing and final reconstruction of soft x-ray diffraction data for ptychographic imaging at the new COSMIC beamline, 7.0.1, at the ALS. The data are generated by the ALS fastCCD at up to 400 MBytes per second and typical datasets are 10s of GB in size. The 64 CPU threads of the PHASIS cluster are used for pre-processing individual diffraction snapshots while complete reconstruction jobs are executed across 16 nVidia GTX Titan, 4 nVidia K80, and 4 nVIDIA TITAN-X GPUs. This scale of this cluster enables realtime feedback to the COSMIC microscope user and makes the use of diffraction data for imaging transparent.
Hardware Configuration:
- Master node (1)
Phasis master node has dual-socket octa-core Intel Xeon E5-2680 @ 2.70 GHz processors (16 cores in total), 256 GB of physical memory, and 260 TB (raw) of storage.
- Compute nodes (6)Cluster GPU cores total: 78,392
2.946 Cluster GPU PFLOP (theoretical peak)
n0000-n0003 compute nodes each have dual-socket octa-core Intel Xeon E5-2680 @ 2.70 GHz processors (16 cores in total), 4 Nvidia GeForce GTX Titan GPUs (4×2688), 256 GB of physical memory. 432.49 GPU TF (theoretical peak)
n0004 compute node has Dual Xeon E5-2630 2.3GHz CPU 8 cores, total of 16 cores, 256GB memory, and 4 nVidia K80 GPUs (4×4992). 34.9 GPU TF (theoretical peak)
n0005 compute node has Dual Xeon E5-2699V4 2.2 GHz CPU 22 cores, total of 44 cores, 256GB memory, and 4 nVidia TITAN X GPUs (4×3072) 1.18 GPU PF (theoretical peak)
Compute nodes are connected with each other through a high performance Mellanox 56 Gb FDR Infiniband switch.
- Storage
260 TB (raw) of storage is shared across the whole cluster and beamline 7.0.1 through the master node.
Login Procedure:
Phasis cluster uses One Time Password (OTP) for login authentication.
- Login server: phasis.lbl.gov
Software Configurations and Data Transfer:
Globus Online Endpoint: #phasis
Authenticate using OTP at login web page
SSH-FUSE mounts allowed using OTP authentication (currently disabled 080416 due to security issues)
[Linux or MAC]
e.g.
(1) sshfs username@phasis.lbl.gov:/global/groups/cosmic/data mount_point
(2) to unmount: fusermount -u
(3) users need to remember to unmount their fuse mounts before the Phasis master node is rebooted.
Software Module Farm (SMF)
See here for instructions on using the Global SMF.
Scheduler Configuration:
Partition | Nodes | Node List | QoS | QoS Limit |
---|---|---|---|---|
batch | 6 | n000[0-5] | normal | no limits |
Software Configuration:
Phasis cluster uses Warewulf as the system management software.
Cluster Status:
Please visit here for the live status of Phasis cluster.
Additional Information:
Please use Service Now or send email to ScienceIT@lbl.gov for any inquiries or service requests.