Data Transfer Nodes
As LBNL researchers adopt high performance computing resources to help analyze this data, moving these masses of data to the systems on which computational analysis is performed is a challenge. In significant part, that’s because it can be slow and frustrating to schlep vast quantities of data from the many places it can be stored or generated over general-purpose computer networks.
To help address this pain point, the Science IT can help researchers deploy a Data Transfer Node (DTN) at their site to facilitate high speed data transfers. Data Transfer Nodes (DTNs) are dedicated systems, purpose-built and tuned to facilitate high speed file transfers over Science DMZ networks. DTNs are set up as endpoints on those networks to initiate or receive data transfers between large data generators, such as genome sequencers, high resolution cameras, and other instruments or sensors; and data receivers, including large storage resources; computational resources, such as the Lab’s Lawrencium cluster; and even other collaborator sites elsewhere on the Internet.
Science IT will help researchers determine their data movement requirements and help them to procure, setup and integrate a DTN into their local computing environment. There are currently a number of DTNs in production for researchers at the Advanced Light Source, the Molecular Foundry, and for other research projects at the Lab.
As part of the installation, Science IT will install the Globus Online software on the DTN. Globus Online allows researchers to perform unattended transfers of large datasets to and from their DTN, utilizing parallel streams. This software improves data transfer speed, convenience, and reliability; additional features will soon be enabled to offer the ability to selectively share files on researchers’ laptops or workstations with other Globus users.
Researchers interested in setting up a DTN should contact ScienceIT@lbl.gov for a consultation.
in the bottom right or navigate to