1. Introduction

1.1. Purpose of this document

This document introduces users to the Navy DoD Supercomputing Resource Center (DSRC). It provides an overview of available resources, links to important documentation, important policies governing the use of our systems, and other information to help you make efficient and effective use of your allocated hours.

1.2. About the Navy DSRC

The Navy DSRC is one of five DSRCs managed by the DoD High Performance Computing Modernization Program (HPCMP). The DSRCs deliver a range of compute-intensive and data-intensive capabilities to the DoD science and technology, test and evaluation, and acquisition engineering communities. Each DSRC operates and maintains major High Performance Computing (HPC) systems and associated infrastructure, such as data storage, in both unclassified and classified environments. The HPCMP provides user support through a centralized help desk and data analysis/visualization group.

The Navy DSRC is operated by the Commander, Naval Meteorology and Oceanography Command (COMNAVMETOCCOM) and is located at John C. Stennis Space Center in Mississippi. COMNAVMETOCCOM provides atmospheric and oceanographic support to the Department of Defense through a wide range of modeling, prediction and data collection techniques.

The Navy DSRC, formerly the NAVO MSRC, was the second of the four major shared DoD High Performance Computing (HPC) centers to be formed under the auspices of the DoD HPC Modernization Program.

1.3. Who our services are for

The HPCMP's services are available to researchers in the Research, Development, Test, and Evaluation (RDT&E) and acquisition engineering communities of the DoD and its respective Services and Agencies, DoD contractors, and University staff working on a DoD research grant.

For more details, see HPCMP Presentation " Who may run on HPCMP Resources?"

1.4. How to get an account

Anyone meeting the above criteria may request an HPCMP account. A Help Desk video is available to guide you through the process of getting an account. To begin the account application process, visit HPC Centers: Obtaining an Account, and follow the instructions presented there.

1.5. Visiting the Navy DSRC

If you need to travel to the Navy DSRC, there are security procedures that must be completed BEFORE planning your trip. Please visit our Planning a Visit page and coordinate with your Service/Agency Approval Authority (S/AAA) to ensure that all requirements are met.

2. Policies

2.1. Baseline Configuration (BC) policies

The Baseline Configuration Team sets policies that apply to all HPCMP HPC systems. The BC Policy Compliance Matrix provides an index of all BC policies and compliance status of systems at each DSRC.

2.2. Login node abuse policy

The login nodes provide login access to the systems and support such activities as compiling, editing and general interactive use by all users. Consequently, memory- or CPU-intensive programs running on the login nodes can significantly affect all users of the system. Therefore, only small serial applications requiring less than 15 minutes of compute time and less than 8 GB of memory are allowed on the login nodes. Any jobs running on the login nodes that exceed these limits will be terminated.

2.3. File space management policy

Close management of the space in the /p/work1 file system is a high priority. Files in the /p/work1 file system that have not been access in 21 days are subject to the purge cycle. If available space becomes critically low, a manual purge may be run, and all files in the /p/work1 file system are eligible for deletion. Using the touch command (or similar commands) to prevent files from being purged is discouraged. Users are expected to keep up with file archival and removal within the normal purge cycles.

Note: If it is determined as part of the normal purge cycle that files in your $WORKDIR directory must be deleted, you WILL NOT be notified prior to deletion. You are responsible to monitor your workspace to prevent data loss.

2.4. Maximum session lifetime policy

To provide users with a more secure high performance computing environment, the Navy DSRC has implemented a limit on the lifetime of all terminal/window sessions. Any idle terminal or window session connections to the Navy DSRC shall be terminated after 4 hours. Regardless of activity, any terminal or window session connections to the Navy DSRC shall be terminated after 24 hours.

2.5. Batch use policy

Batch queue environments are available on all of the HPC systems. The batch environment is the primary environment for most user work. All of the systems at the Navy DSRC use the PBS batch queue system.

The batch queue environments allow users to submit, monitor and terminate their own batch jobs. This capability is intended for jobs requiring large amounts of memory and/or CPU time that generally run for many hours.

All HPC systems have identical queue names: urgent, frontier, high, debug, standard and background; however each queue has different properties as specified in the tables below. Each of these queues is assigned a priority factor within the batch system and are listed from highest to lowest.

Nautilus - 176,128 - Penguin TrueHPC
Priority Queue Name Max Wall Clock Time Max Cores Per Job Description
Highest urgent 24 Hours 16,384 Jobs belonging to DoD HPCMP Urgent Projects
Down arrow for decreasing priority debug 30 Minutes 10,752 Time/resource-limited for user testing and debug purposes
high 168 Hours 65,536 Jobs belonging to DoD HPCMP High Priority Projects
frontier 168 Hours 65,536 Jobs belonging to DoD HPCMP Frontier Projects
standard 168 Hours 16,384 Standard jobs
HIE 24 Hours 3,072 Rapid response for interactive work. For more information see the HPC Interactive Environment (HIE) User Guide.
Lowest background 4 Hours 4,096 User jobs that are not charged against the project allocation

Narwhal - 308,480 - HPE Cray EX
Priority Queue Name Max Wall Clock Time Max Cores Per Job Description
Highest urgent 24 Hours 16,384 Jobs belonging to DoD HPCMP Urgent Projects
Down arrow for decreasing priority frontier 168 Hours 65,536 Jobs belonging to DoD HPCMP Frontier Projects
high 168 Hours 32,768 Jobs belonging to DoD HPCMP High Priority Projects
debug 30 Minutes 8,192 Time/resource-limited for user testing and debug purposes
HIE 24 Hours 3,072 Rapid response for interactive work. For more information see the HPC Interactive Environment (HIE) User Guide.
viz 24 Hours 128 Visualization jobs
standard 168 Hours 32,768 Standard jobs
mla 24 Hours 128 Machine Learning Accelerated jobs
smla 24 Hours 128 Machine Learning Accelerated jobs
dmla 24 Hours 128 Machine Learning Accelerated jobs
serial 168 Hours 1 Serial User jobs
bigmem 96 Hours 1,280 Large-memory jobs
transfer 48 Hours N/A Data transfer for user jobs. See the Navy DSRC Archive Guide, section 5.2.
Lowest background 4 Hours 1,024 User jobs that are not charged against the project allocation

Gaffney - 36,096 - HPE SGI 8600
Priority Queue Name Max Wall Clock Time Max Cores Per Job Description
Highest urgent 24 Hours 768 Designated urgent projects by DoD HPCMP
Down arrow for decreasing priority debug 30 Minutes 2,400 User diagnostic jobs
HIE 24 Hours 384 Rapid response for interactive work. For more information see the HPC Interactive Environment (HIE) User Guide.
high 168 Hours 15,840 Designated high-priority jobs by Service/Agency
frontier 168 Hours 14,400 Designated frontier projects by DoD HPCMP
standard 168 Hours 8,160 Normal priority user jobs
gpu 24 Hours 384 GPU-accelerated jobs
serial 168 Hours 1 Serial user jobs
bigmem 96 Hours 720 Large-memory jobs
transfer 48 Hours N/A Data transfer for user jobs. See the Navy DSRC Archive Guide, section 5.2.
Lowest background 4 Hours 1,200 User jobs that will not be charged against the project allocation

Koehr - 36,096 - HPE SGI 8600
Priority Queue Name Max Wall Clock Time Max Cores Per Job Description
Highest urgent 24 Hours 768 Designated urgent projects by DoD HPCMP
Down arrow for decreasing priority debug 30 Minutes 2,400 User diagnostic jobs
HIE 24 Hours 384 Rapid response for interactive work. For more information see the HPC Interactive Environment (HIE) User Guide.
high 168 Hours 14,304 Designated high-priority jobs by Service/Agency
frontier 168 Hours 14,400 Designated frontier projects by DoD HPCMP
standard 168 Hours 8,160 Normal priority user jobs
gpu 24 Hours 384 GPU-accelerated jobs
serial 168 Hours 1 Serial user jobs
bigmem 96 Hours 720 Large-memory jobs
transfer 48 Hours N/A Data transfer for user jobs. See the Navy DSRC Archive Guide, section 5.2.
Lowest background 4 Hours 1,200 User jobs that will not be charged against the project allocation