Frederick Research Computing Environment

Globus

Globus is a service that makes it easy to transfer large amounts of data. Globus will manage file transfers in the background using GridFTP and report the status of your data transfer. 

Globus was developed and is maintained at the University of Chicago and is used by many research institutions and supercomputing centers. The Globus website documents how to download & install the desktop client and usage documentation.

There is general information concerning Globus on the Biowulf site and the majority of the guidance is applicable to NCI-F.

Introduction 

There are two components in Globus, user web/cli interface, and Globus endpoints. The user interface is used to issue transactions between endpoints, and can be web-based on app.globus.com or can be accessed via cli. Globus servers set up endpoints allowing transactions through. One endpoint should have the Globus Plus license.

User Interface

Users can authenticate themselves with NIH credentials. Other credentials may not be able to access NIH-shared Globus resources. FRCE is installed with Globus cli accessible using the module load globus command. Users can schedule a globus transfer through command line.

In addition, users can schedule a transfer though gloubus.org, or through a desktop app. Users may need helpdesk assistance to install desktop app on an office computer.

Endpoints

Users can install personal endpoints, as well as FRCE endpoint nihfnlcr#gridftp1. With the current license arrangement, FRCE users can transfer files between nihfnlcr#gridftp1 and personal endpoints. An example is "./globusconnectpersonal -start -restrict-paths /scratch/cluster_scratch/${USER}/". Shared directories are defined with " -shared-paths". Fine-grained access controls are managed through the cli user interface or web user interface with a Globus Plus license.

Within nihfnlcr#gridftp1, users can create individual endpoints to have more control of resources to share.

Transfer files to batch

  1. User asks for gridftp group membership for Globus file transfer.
  2. For endpoint nihfnlcr#gridftp1(owner nihfnlcr@globusid.org), your files will be saved at /mnt/GridFTP/${USER}/.
  3. Files can be accessed from the cluster, preferably using fsitgl-xfer03p. Within the transfer node, users can transfer files from /mnt/gridftp/${USER}/ to /scratch/cluster_scratch/${USER}/.
  4. Within the cluster, you can transfer your data on/scratch/cluster_scratch/${USER}/ to other volumes.

Quotas

Currently, there are no user or group quotas on /mnt/gridftp/.