Access is intended only for legitimate purposes which benefit the research at Harvard University. Access must be authorized by the faculty or management of HBS and our partners, and by the staff of Research Computing Services. Access should only be granted for the purposes necessary to accomplish the goals of Harvard University and its research projects.

All account holders, whether Harvard affiliates or outside collaborators, agree to be held accountable by the Harvard University Electronic Access and Information Security policies found at http://huit.harvard.edu/information-technology-policies.

Guest collaborators are also required to keep RCS informed of any changes in contact information.

In addition, researchers should make themselves familiar with the university research policies maintained by the Provost's Office at http://vpr.harvard.edu/.

Our computing infrastructure is a shared resource, supporting concurrent access by many users. As a result, RCS must enforce some basic guidelines for behavior and interaction to provide each user with the optimal environment for individual or collaborative research. In addition, as the compute, storage, and software support HRCI Level 3 and higher, certain standard computing practices normally allowed on desktops and laptop are not allowed on the compute grid. Those will be outlined below.

Note that RCS maintains several emails lists for communication with users, and every effort is made to keep the signal-to-noise ratio as high as possible. Certain subscriptions (e.g. rcs_grid for using the compute grid and/or project spaces) are required to keep one's account.

Persons will only be allowed access to project spaces once the access has been approved by the primary faculty sponsor for that project space.

RCS will only grant project spaces to workgroups using HRCI Level 3 and Level 4 data once the DUA and/or approved IRB proposal has been filed with RCS.

Project space usage is reviewed and confirmed on a yearly basis (usually in September). Inactive project spaces will be backed up and archived only after contacting the primary faculty sponsor. Archives are retained for the period of time specified by HU or HBS data retention policies, whichever is longest. Please contact the ESS group within HBS via a Service Now HBS IT ticket for more information on this process. For information about how to archive your data to an external drive, see our technical note at https://www.hbs.edu/research-computing-services/help/technical-how-tos-and-technical-notes/archiving-your-research-files.aspx.

Upon departure, please provide an appropriate amount of lead time (at least one month) to assist with copying data if needed. In this situation, If someone is not the primary sponsor of the project space, RCS will contact the primary sponsor to confirm this is permitted.

Project spaces and home folders are backed up on a nightly basis. If you have questions on the backup policy, please contact the ESS group within HBS via a Service Now HBS IT ticket.

Auto-sync services cannot be used with cloud-based file sharing services (e.g., DropBox) and project spaces or home folders on our compute grid research storage, due to our HRCI-supported environment. However, use of push/pull services (e.g., RClone) is permitted.

Scratch Usage and Policy

This section describes the expected usage and behaviors of the ‘scratch’ space (/export/scratch) on the research storage on the HBSGrid, and policies around its administration.

Users have the following expectations around the use of scratch:

• A temporary space for their work, and not necessarily the same use/approach as /tmp.
• A strategic location for temporary storage of disposable files so as not to interfere with one’s quota
• A place to exchange files with others outside their working groups
• Files will be deleted at maintenance time after aging a designated number of days (at present, 60 days)
• File deletion may happen if scratch is nearly full, when announced with 1 to 2 business days notice.
• Files are not backed up
• Not always) Files may be touched to keep them around longer, up to a reasonable amount of time.
• And other, unforeseen uses.

Scratch spaces on compute clusters are usually high-performant (e.g. Lustre or similar) and very large. Typical usage scenarios for scratch storage are:

• Working space for running jobs (temporary files, checkpoint data, etc.)
• High performance file input and output
• Large, temporary files that may be intermediate workflow inputs/outputs
• Downloaded 'reference files' that will be deleted after some processing

At HBS, /export/scratch has been created from a different disk pool than home folders and project spaces, but is the same class / performance tier storage as "general use" storage that comprises home folders and project spaces. This may be upgraded or reconfigured as needed in the future.

Don't write directly to the top level of scratch. Please create a folder for your work.

Please ensure that you set appropriate permissions for this top folder and any others here: the default permissions are usually world-readable!!

Also, please do not mount scratch on your desktop or laptop or if mounted, please transfer your files and unmount as a final step. If you need to transfer files to this location, please also consider FileZilla or SecureFX.

Files stored in this space are not backed up. As such, you should make efforts to copy back your files as needed to your permanent storage (home folders or project spaces).

There are no quotas to prevent other persons from consuming the space. We expect everyone to be good neighbors, be conservative on usage, and clean up ones work.

Files older that 60 days will be deleted, almost always at maintenance time and with prior notification.

If there is a need to keep files longer than 60 days, please contact RCS.

If we should need to do off-schedule space reclamation—scratch is completely full (or nearly so)—a notice will be sent out 1 to 2 business days before the retention. Only in an extreme situation, we may only give 4 hours notice.

Links: Scratch/storage descriptions and policies

• Research IT @Berkeley
• FSU Research Computing (has some good user pointers)
• Research Computing @ Dartmouth
• IQSS: https://rce-docs.hmdc.harvard.edu/faq/there-temp-or-scratch-storage-available
• FASRC: https://docs.rc.fas.harvard.edu/kb/policy-scratch/
• Univ Utah: https://www.chpc.utah.edu/resources/storage_services.php
• Yale CRC: https://research.computing.yale.edu/services/high-performance-computing/hpc-policies

/tmp Volumes

Almost all compute servers rely on the /tmp volume for applications to create temporary files while running, and free space is essential. This is a challenge on clusters as nodes may remain up for months at a time. To help ensure /tmp is trash-free, we have the following /tmp and cleanup policy:

• On reboots of all nodes (login and compute), all files will be deleted.
• On compute nodes, a nightly script will delete files if older than 30 days AND the user does not have a job running at cleanup time.

RCS reserves the right to terminate jobs that are unattended for excessive periods of time or are impinging on the resources of others ('threading out', 'swapping out', or 'out of control'). We will make every effort to contact the job owner before doing so.

RCS regularly monitors the state of the compute grid, the jobs that are running, and how these jobs are affecting the shared, fair-use of the resources for all persons. If we contact you for any reason about your jobs, we expect you to respond in a timely fashion and to assist us with any requests.

A monthly maintenance window—typically the first Monday of the month—is used for software and hardware patches, updates, and upgrades. Reminder emails are sent in advance of the maintenance window, detailing work to be done. Access and compute is not guaranteed during this time, and every effort is made to protect work in progress.

MariaDB accounts and space are available upon request.

RCS will provide database access credentials for persons with HRCI Level 3 data once the DUA and/or approved IRB proposal has been filed with RCS. Changes to access will only be granted after receiving explicit approval from the primary faculty or staff sponsor.

Please notify RCS if you plan to store Level 4 data on this server.

Software purchased and available for use under the HBS name, whether used on local machines or on the HBS compute grid, is governed by the Vendor's Terms of Service. Typically, copying the software is prohibited, and only limited number of installations are permitted, usually outlined in the licensing terms. RCS provides guidance to some of this information on our Software pages.

Guest Users are bound by the same terms as HBS personnel for installed software on the compute grid and are not able to purchase or obtain additional software from RCS.

If RCS provides access to installers or installation codes, you will not share these with any other persons.

Should you no longer be affiliated with HBS (end of study, end of employment, or end of guest access), it is your responsibility to remove HBS-licensed software from any personal computers where it may have been installed. Please see our off-boarding checklist.