Storage and Data Management

This document describes recommended procedures (SOP) for managing data on ARS HPC and storage infrastructure.

This document provides detailed information about the storage options provided by SCINet and how to use them. For a simpler overview of suggested procedures for managing data on SCINet, please see Managing Data on ARS HPC and Storage Infrastructure.

There are multiple places to store data on the Ceres and Atlas clusters that all serve different purposes.

Data Transfer best practices.

Globus Online is the recommended method for transferring data to and from the HPC clusters.

Rclone is already installed on the DTNS and all of the compute nodes. Please do not use rclone from the headnode. Attempting to do so will remind you to use the others.
The rclone home page is https://rclone.org.