There are several options for file storage on the FRCE cluster, each with its own advantages and disadvantages. Shares available to all users include
| Directory | Quota | Availability | Data Protection |
|---|---|---|---|
| /home/username | 256GB | cluster-wide | Volume snapshots are kept for 30 days |
| /scratch/cluster_scratch/username | 5TB | cluster-wide | no protection |
| /scratch/local or /tmp | no quota, limited to volume size | not shared between hosts | no protection |
/home
Each user has a home directory called /home/username which is accessible from every HPC system. The
/homearea has a quota of 256GB which cannot be increased. It is commonly used for config files, code, scripts, and analysis results. The share is relatively slow and applications should not process data on the share. Volume-level snapshots (backups) of the directories are made daily and kept for 30 days. Users can restore their own files from the past 14 days from the directory /home/.snapshot/. Restores from later than that will require a Service Now ticket directed to the EIT Storage group.
/scratch/cluster_scratch
This share is mounted on all FRCE nodes and is a high-performance network share. While the intent is for transient storage of raw data there is no defined lifetime for storage. The system does not ever delete data from this share. It is the responsibility of the user to manage file storage.
/scratch/local
This share is also mounted as
/tmpand is local to each server. Files stored on one system will not be available on other systems, including the head node. Intended for data needed only during the duration of a single job and the files may be deleted when the systems reboot. Performance is comparable to that of/scratch/cluster_scratch.
To comply with NIH security guidelines the user directories under both /home and /scratch/cluster_scratch are created with permissions limited to the directory owner. Each user's umask is also set by the guidelines to have new files and directories created with owner-only access. Both of these settings are user-controlled and any changes made by users will not be undone by the system.
Note that while quotas are in effect on /home and /scratch/cluster_scratch, the platforms hosting the shares do not have any utilities to show disk usage or the actual assigned quota.
If space is needed for long-term data retention or if an area shared among group members is required, please complete the storage request form on Service Now and the NCIF Storage team will contact you. The FRCE administrators do not have privileges to create network shares. If you are in Shady Grove or Bethesda and cannot access the form, you may submit a general service request ticket with the request that it be assigned to the NCIF Storage team.