Skip to main content
U.S. flag

An official website of the United States government

Dot gov

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Https

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Transferring files between a USDA Box account and SciNet

Rclone is already installed on the DTNSand all of the compute nodes. Please do not use rclone from the headnode. Attempting to do so will remind you to use the others.
The rclone home page is https://rclone.org/

rclone installation on Windows

You will have to install rclone on Windows so that you can generate an authentication token for Box with the USDA VPN and eAuthentication active
Go to the web page at https://rclone.org/downloads/and find the Windows installer. Download it and install rclone. Then proceed to the configuration section below.

macOS installation

Download the latest version of rclone.

Unzip the download and cd to the extracted folder.

unzip -a rclone-current-osx-amd64.zip && cd rclone-*-osx-amd64

Move rclone to your $PATH. You will be prompted for your password.

sudo mkdir -p /usr/local/bin

sudo mv rclone /usr/local/bin/

(themkdircommand is safe to run, even if the directory already exists) Remove the leftover files.

cd .. && rm -rf rclone-*-osx-amd64 rclone-current-osx-amd64.zip

Configuration of rclone on windows and osX

1. Open a Windows command prompt (cmd)

2. Type

3.rclone authorize "box"

4. On the web page that shows up, click on Use Single Sign On (SSO)

5. Enter your USDA email address

6. Do the eAuthentication thing

7.Click on the Grant access to Box button

8.Go back to the command prompt window, an authentication token should be there. Copy this includingthe braces {“access_token”:“ABCDEF...}

rclone configuration on SciNet

1. Type

rclone config

2. Type n for n) New remote

3. For name> enter any name e.g. usdabox

4. For Storage> you can find the number, but it's easier to just type box

5. For both client_id> and client_secret> leave blank, just hit enter

6. For Edit advanced config enter n

7. For Remote config, Use auto config? enter n

8. Paste the text from the last step from the section above that you got from your Windows computer

9. Type y for y) Yes this is OK

10. Type q to quit

Test

1. Test directory listing using the name> you selected earlier:

rclone lsd usdabox:

2. Test file listing

rclone lsl usdabox: | head

Commands

1. For description of commands available see https://rclone.org/docs/#subcommands

2. You installed the rclone manual page earlier, so you can also do

man rclone

3. Acommon case might be to backup your SciNet project directory to box. You could do this withe.g

rclone copy /project/bogus_genome usdabox:/scinetbackup/bogus_genome --verbose