Compute Cluster and Data Storage
Data Storage
Data Storage
Home Folders
By default, when your grid account is created (whether this account is used for just storage or storage + compute), a home folder is created as well. This is similar to the home folder on your laptop or desktop, but more like the home folder you may have on \\hbsfiles.
If you are logging in to the HBSGrid to do work via the NoMachine GUI or terminal, you are automatically placed in and are using this home folder. This is located at:
/export/home/<group>/<username>
For example faculty member John Harvard's home folder would be at:
/export/home/faculty/jharvard
Home folders vary in size by the program or role that you have. When your space fills up, you will not be able to do any more work, which may lead to programs acting strangely or crashing altogether, disk error notices, or Input/Output errors. Keep an eye on your space usage; and see our page on working with spaces, folders, and files.
Home folders are backed up every night. If you should need to recover any files due to accidental deletion or corruption, please contact RCS.
In addition to home folders, project spaces are another location of disk storage space, and are described in the next section.
Scratch Storage
Whether you are doing batch or interactive work, at times you may need a temporary location to stash files that you will not keep, or your software may require a 'working', or temporary, directory. In most cases, you should not use your home folder, as there are quota limits, and the temporary needs may exceed the amount available.
The storage location at /export/scratch
is designed specifically for this purpose. Note:
- This is a shared area, so use Unix file permissions to change the accessibility if needed.
- Files older than 60 days will be deleted at monthly maintenance periods.
- The /export/scratch filesystem is not backed up, so please copy back to your home or project folder any files or folders you wish to keep long-term.
Please see our RCS Policies page for more information about our scratch usage policies.
Project Spaces
Project spaces (folders) are the primary, recommended location for storing and doing collaborative work on research storage, including HBS and guest user accounts. This is in contrast to home folders, which are accessible only by the account holder. Both home folders and project spaces are backed up every evening. If you should need to recover any files due to accidental deletion or corruption, please contact RCS.
Requesting a Project space
Project spaces can be set up with faculty sponsorship and/or approval by filling out the New Project Space Request Form.
The default size is 50GB, with increases granted upon request and with space available. Keep an eye on the space usage, as project spaces that reach capacity will throw errors in programs and when transferring files, and data loss may result. As with home folders, project spaces that reach capacity are not a good thing. Please see our working with spaces, folder, & files page for useful tips.
If you are working with level 3 data or higher, we will ask you to submit documentation from the IRB or a Data Usage Agreement (DUA; a sample can be found here) so that we can ensure that the project space’s security levels are appropriately set up. If you are unsure what level your data falls under, please review this page on data security.
If a project space already exists and you would like to make changes to the space users or size, fill out the Project Space Change Request Form. Once approved by the appropriate faculty member, the request will be put in to the ESS group at HBS IT.
Archiving a Project space
Project space usage is reviewed and confirmed on a yearly basis. Inactive project spaces will be backed up and archived only after contacting the primary faculty sponsor. Archives are retained for the period of time specified by HU or HBS data retention policies, whichever is longest. If you would like to unarchive a project space, contact RCS.
For information about how to archive your data to an external drive, see our technical note on archiving your research files.