By Arica Chhay, Gary Jung, Ian Vaino
Contents
- Wireless Sensor Monitoring
- Variable Speed Drive (VSD) Cooling
- Direct-Chip Liquid Cooling
- Energy Efficiency in 2023
IT Tips to Reuse and Reduce
Save a Tree and Go Paperless
Committed to a Greener Future
Berkeley Lab celebrates Earth Month annually throughout April. In support of the Lab’s sustainability initiatives, this article covers our current efforts in the IT Division to reduce environmental impact and considerations we can all take to save on energy consumption. IT commits to a greener future in technology by taking steps to make our operations more eco-friendly, including efficient infrastructure for data centers, running a repurposed computer program, and promoting paperless services.
Energy Efficient Data Centers
One of IT’s critical goals is to reduce energy consumption for infrastructure across the Lab. The ScienceIT group and Lab research teams have come together to ensure that our IT data centers are optimized for efficient power utilization, offering peak operational performance along with highly scalable, state-of-the-art computing capabilities.
More than ever before, today’s supercomputers require an increasing amount of power and cooling to operate. Faced with the prospect of having to limit its high performance computing capability, ScienceIT, in conjunction with the Lab’s Energy Technologies Area (ETA) and industry partners, embarked on a years-long effort to increase the capacity of it’s 50-year old 5600 sq ft data center while implementing energy efficient strategies to make the most of the existing facility.
Wireless Sensor Monitoring
In 2007, the IT Division engaged Synapsense, which at the time was just beginning development of its real-time, wireless, monitoring application, to deploy a system permitting detailed analysis of the environmental conditions (humidity and temperature) along with air pressure and power monitoring at hundreds of points within the data center. Once this system was deployed, the team used computational fluid dynamics (CFD) modeling and the accumulated data to change the airflow and make other operational adjustments in the data center. The team undertook a variety of fixes, some small, and some large, including:
- Perforated floor tile tuning to improve air pressure
- Installation of rack cabinet blanking panels
- Hot Aisle/Cold Aisle Isolation
- Conversion of the overhead ceiling plenum to hot air return
- Chimney extension of CRAC returns to connect to the overhead ceiling plenum
- Installation of plastic curtains to further reduce hot aisle/cold aisle mixing
- Installation of water-cooled doors based on non-chilled water (collaboration with the vendor to reduce energy use)
- Piloting of fully enclosed water-cooled racks
- Use of higher ambient temperature setpoints to improve efficiency
These fixes allowed ScienceIT to increase its scientific computing capability by over 50% since 2007, when the data center was assumed to be at capacity.
The culmination of this initial work occurred in 2010, when LBNL became one of the first organizations in the federal space, and among a handful of smaller data centers in the world, to be able to calculate and view the data center’s Power Utilization Effectiveness (PUE) in real-time. This critical metric, which indicates the power used by the data center infrastructure in comparison to the power used by the computers themselves, helps staff manage power utilization on a dynamic basis in order to achieve environmental goals.
Variable Speed Drive (VSD) Cooling
In 2012, we partnered again with Synapsense to retrofit our data center CRACs (Computer Room Air Conditioner) with variable-speed controls on the fans and demonstrated that variable-speed operation is possible and that it results in significant energy savings and improved cooling and reliability. The study found an energy use reduction of 24 percent compared to constant-speed fan operation.
More importantly, it provided the ability to dynamically respond to meet the needs of changing scientific computation loads — which caused large fluctuations in power and cooling demand — in our data center. The Lab and Synapsense were recognized as a finalist for the Green Enterprise IT Award 2013 by the Symposium Uptime Institute for this work.
Direct-Chip Liquid Cooling
In 2013, ScienceIT and ETA worked with several vendors to evaluate the use of direct-to-chip water cooling for our compute clusters. Used extensively by PC gamers wanting to overclock their home PC processors for maximum performance, this technology was adapted for large scale use in the data center environment.
By using non-chilled water in our data center, this provided free cooling — up to the limits of the building plumbing — for our systems. As part of this project, we ran our regular mix of scientific jobs on the system and measured the heat load rejected into the water system to determine efficiency. The findings from this evaluation were presented by ETA researcher Henry Coles at the sixth annual Data Center Efficiency Summit.
Energy Efficiency in 2023
Fast forward to today and you’ll find many of these technologies, where we were early adopters, have since evolved and are now fully incorporated into our data center as well as being in wide use by other data centers around the world.
Among even other improvements, we’ve upgraded the monitoring sensors and also the original passive water-cooling rack cabinet doors with the latest and more effective multi-pass Active Cooling Doors. We also require the use of Direct Liquid Direct Cooled processor cooling on all new incoming compute equipment to scrub off heat as chip density increases. At the same time, we’ve been gradually decommissioning the original CRAC units as these newer technologies are more efficient at removing heat from the room. Finally, we now benchmark all new equipment to measure power draw under full load which allows for better planning to ensure we aren’t either under or overusing our power circuits
Taken together, these steps have enabled ScienceIT to increase the effective cooling capacity of the existing data center from 800kW to 1.5MW. All of which is dedicated to support the scientific computational resources used by scientists across the Lab.
IT Tips to Reuse and Reduce
Minimizing electronic waste is another IT sustainability priority. IT maintains a repurposed computer program that retrieves computers which have reached the end of their support lifecycle and refurbishes them for other Lab employees and uses. These computers can be repurposed as a data collection system, a loaner pool system, a gateway to isolate vulnerable computers, and more.
The repurposed computer program empowers the Lab to retain and reuse devices which are still in good working condition instead of adding to the waste stream.
We also purchase and make recommendations for energy-efficient hardware and software, such as laptops with low-power processors and servers with virtualization technology. Chrome OS systems like Chromebooks and Chromeboxes are a great way to save costs and energy without sacrificing productivity. The operating system is efficient, meaning many Chrome OS devices can last a whole working day on battery power and are a lightweight solution for continuous needs like digital screens.
In terms of software, the recent switch to CrowdStrike is a major improvement on implementing a Lab-wide antivirus solution with reduced performance impact on computers. Additionally, simply reviewing applications which run upon startup of your system (macOS, Windows) and configuring power management computer settings (macOS, Windows) are effective ways to reduce power consumption and prolong the life of devices.
Save a Tree and Go Paperless
Measures to reduce paper waste by implementing paperless methods have long been promoted by IT. We make suggestions for moving to cloud-based services, and digitizing internal processes and documents whenever possible. Going paperless can streamline our workflows and reduce time and effort spent on procedural tasks and tracking paper.
Drafting, reviewing, and obtaining approvals on documents can frequently be conducted entirely through Google Docs. Requesting signatures on forms and approvals via Dropbox Sign (formerly HelloSign) is another IT-recommended way to avoid paper documents with an added layer of authentication and encryption.
In accordance with federal modernization standards, the Records Management Office (RMO) is already on their way to transitioning from paper to electronic records, along with new procedures for the maintenance of electronic records at the Lab. The transition is a step toward a fully electronic business environment and increases the likelihood of recovery in the event of a disaster.
Committed to a Greener Future
A bird sitting on the sign at the intersection of Chamberlain and McMillan Roads. (Credit: Thor Swift/Berkeley Lab)
The IT Division is proud of our sustainability efforts. We remain committed to a greener future in technology by minimizing our impact on the environment, constantly looking for ways to refine our practices, and promoting responsible consumption. As the Lab celebrates Earth Month, we encourage everyone to reflect on their environmental impact. Every little step counts, and, together, we can create a more sustainable future for generations to come.
Get Involved with Sustainability
- Visit earthday.lbl.gov for sustainability resources, events, and Zoom backgrounds.
- Visit sbl.lbl.gov to learn more about sustainability efforts at Berkeley Lab.
- Join the Sustainability Google Group.
- Add the Sustainability Calendar for year-round events.
in the bottom right or navigate to