Skip to main content
U.S. flag

An official website of the United States government

Storage and Data Management

  • Data and Storage SOP

    This document describes recommended procedures (SOP) for managing data on ARS HPC and storage infrastructure.

  • Storage Quotas

    Each file on a Linux system is associated with one user and one group. On Ceres, files in a user’s home directory by default are associated with the user’s primary group, which has the same name as user’s SCINet account. Files in the project directories by default are associated with the project groups. Group quotas that control the amount of data stored are enabled on both home and project directories.

    At login, current usage and quotas are displayed for all groups that a user belongs to. The my_quotas command provides the same output:

    $ my_quotas
    
  • SCINet Storage

    This document provides detailed information about the storage options provided by SCINet and how to use them. For a simpler overview of suggested procedures for managing data on SCINet, please see Managing Data on ARS HPC and Storage Infrastructure.

    There are multiple places to store data on the Ceres and Atlas clusters that all serve different purposes.

  • SCINet File Transfer

    Data Transfer best practices.

    Globus Online is the recommended method for transferring data to and from the HPC clusters.

  • Rclone: Moving Data To and From Cloud Resources

    Rclone is already installed on the DTNS and all of the compute nodes. Please do not use rclone from the headnode. Attempting to do so will remind you to use the others.
    The rclone home page is https://rclone.org.