HPCS System Updates

Lawrencium data transfer node is unreachable 16th June, 2025

• We are checking the root cause of the issue and working towards resolving it as soon as possible. Possibly, the configuration of the switch is altered during LBLNet maintenance work over the weekend.

LBLNet performing network devices maintenance in data center 14th June, 2025

• Users will experience brief connection issues on the cluster. Jobs shall continue to run.

Lawrencium scratch file system repair work complete. 5:41 AM May 13th, 2025

• We are seeing improvement in the filesystem response. The issues related to OOD and Globus are resolved. Slurm reservation is released

Lawrencium scratch file system repair work continues. 11:00 AM May 8th, 2025

• We are working closely with our storage vendor, DDN, for diagnosis and solutions to bring the file system back to normal. Slurm reservation will stay until work is complete.

Slurm reservation on compute node for Lawrencium scratch file system repair work. 1:30 PM May 7th, 2025

• Users won’t be able to submit new jobs, and already submitted jobs won’t start until the work is complete and the reservation is released

Lawrencium scratch file system is back. 12 PM, May 7th, 2025

• Cooling issue at the datacenter affected the filesystem. We are still working on resolving some lingering issues. lrc-xfer was restarted. Please get in touch with us if you encounter any issues.

Lawrencium scratch file system down. May 6th, 2025

• Users may experience unresponsive interactive commands on the Scratch file system.

Lawrencium login delays April 2nd, 2025

• The IdM team upgraded machines supporting identity services, which affected cluster authentication. Logins may take up to 30 seconds.

LR3 partition is retiring. February 27th, 2025

• The scheduled decommissioning date is March 7th. If you have been using LR3 for your jobs, please migrate your workload to other partitions

The lab-wide power disruption around 2.30 PM on Sunday, Jan 26, affected the Lawrencium Supercluster. January 28th, 2025

• Due to the impact on compute nodes and filesystem you might have noticed problems in job completion

Lawrencium Supercluster is back online after a week-long downtime. December 11th, 2024

• HPC services, including Open OnDemand and Globus, have been returned

Lawrencium Downtime (Dec 1- Dec 11 ) due to a Power Outage at Building 50B. November 27th, 2024

• Due to an electrical shutdown planned by PIMD and facilities at Building 50B, we need to power down the Lawrencium cluster at noon on December 1st. The cluster will be back online on December 11th at 5 p.m.

The software, firmware, and operating system upgrade on the scratch file system is complete. November 26th, 2024

• User now have full access to the cluster

Two-day file system downtime from 8 a.m. Monday, November 25, until 5 p.m. Tuesday, November 26th 2024. November 23th, 2024

• User will experience slow I/O until it recovers from restart

Scratch file system: Back online 3 PM November 14th, 2024

• User will experience slow I/O until it recovers from restart

Scratch file system: Restart of scratch file system at 3 PM November 14th, 2024

• User will experience disruption for about 45 minutes

Network issue partially resolved, Week-long Downtime canceled, October 24th, 2024

• Users are now able to log in and access data. There will be a short downtime on Oct 25th, from 9 am to noon, to fix the file system issue.

Network issue on the cluster, October 24th 2024

• Due to network issue login nodes, OOD and DTN are unreachable. We are working on the fix. We appreciate your patience during the process.

Open OnDemand login issue for LBL affiliate users Friday, July 5th 2024

• Users using any email besides the LBL domain cannot log in to the OOD portal. We are working on the fix and will have an update as soon as the issue is fixed.

Lawrencium Supercluster Back in Service Wednesday, 3rd 2024 5pm.

• System-wide Migration to Rocky-8 OS is complete

Lawrencium Downtime schedules for three days, Monday, July 1st – Wednesday, 3rd 2024

• Downtime is scheduled for Rocky-8 OS system-wide rollout.

Open OnDemand is back to service on Tuesday, May 21

• OOD was offline on Monday and was affected by file system-related issues.

Globus is back in service Tuesday, Nov 28

Globus is still off Monday, Nov 27

• We have contacted the vendor again and will keep you posted.

Lawrencium is back online on Nov 22nd.

• Globus service is not available.

Lawrencium downtime scheduled  Nov 21-22

• Slurm upgrade and system maintenance

LR7 partition is back online. 3:30pm, Friday, Sep 1st

Nodes in LR7 partition are offline due to elevated temperature from the low coolant level 3:30pm, Wednesday, Aug 23rd.

• An investigation is going on

The lrc-xfer (Designated Data Transfer node) will have a brief downtime at 11:45am Tuesday, Aug 8th

• lrc-xfer, as well as Globus Google Cloud Storage collection and the Globus AWS S3 Collection will be offline for about 15 minutes.

Lawrencium Supercluster back to service July 25th.

• Data center power outage and scratch file system upgrade.

Lawrencium Supercluster Downtime

• Data center power outage and cluster maintenance work July 20-24

Lawrencium Supercluster New Firewall Configured June 12th

• Lawrencium users may have experienced sluggish slurm commands and a delay in scheduling jobs today (June 12th), this is due to a new configure on the firewall we recently applied. The problem should have been resolved now. If you continue to see any issues, please email us at hpcshelp@lbl.gov.

Lawrencium Supercluster Downtime

LBLnet network upgrade 8am May 20th- 5pm 21st

Open OnDemand(OOD) portal login issue, 9.00 AM May 2

Open OnDemand portal is having a login issue, and we are working on resolving the problem. We will keep you updated on the progress. Sorry for the inconvenience.

Lawrencium cluster login issue persists on March 13th. 

Lawrencium users may still experience login failure related to the maintenance work performed last week(March 7-10th). The IDM team who is providing the authentication service is conducting an investigation. We will keep you posted on any progress being made.

Lawrencium Scheduler is back to service 10:30 AM Dec 14 

• Slurm upgrade is complete

Lawrencium Supercluster Downtime

• Slurm upgrade 8am Dec 13th – 5pm Dec 14th

/global/scratch/users/ is back to service. 

• The parallel file system scratch is back to service 10:30 AM Dec 5

Lawrencium Scheduler Slurm is back to service 9:30 AM Nov 3 

• We are working to restore the service 8:00 AM Nov 3

Lawrencium Supercluster Back to Service

• Slurm upgrade is complete, the Lawrencium cluster is returned to service at 5:30 PM on Oct 6

Lawrencium Supercluster Downtime

• Slurm upgrade 8am Oct 4th – 5pm Oct 6

Lawrencium Supercluster back online

• Slurm patching complete 5:00pm Tuesday, April 26

Lawrencium Supercluster Downtime

• Slurm patches 8:00am Tuesday, April 26 – 5:00pm Tuesday, April 26

Reboot Compute Nodes Started at 6:30 PM Thursday, Feb 10 –1:00 PM Friday, Feb 11

• Address problems related to MPI jobs, including poor performance with OpenMPI,  and issues with pre-compiled IntelMPI binaries.

Slurm Upgrade is Complete

• Service restored at 5:30pm Wednesday, Feb 2

Lawrencium Supercluster Downtime

• Slurm upgrade 8:00am Tuesday, Feb 1 – 5:00pm Thursday, Feb 3

Lawrencium Supercluster Service restored post-power outage

• Service restored at 1:40pm on Sunday, 12/19 2021

Login Failure: Sunday, 12/12 (Service is back 10:40am)

• The lawrencium login authentication server is currently down. Users can’t login. We have contacted IDM team at LBNL and will keep you posted. Service is resumed.

Lawrencium Supercluster Power Outage

• There is a scheduled power outage in 50B-2275 to accommodate testing of the existing Emergency Power Off system. All of the HPC computing systems will be shutdown and unavailable starting at 5:00pm on Friday, 12/17. We anticipate returning the systems to production on Sunday, 12/19 by 5:00pm.