Kilrain Quick Start Guide
Table of Contents
- 1. Introduction
- 2. Get a Kerberos Ticket
- 3. Connect to Kilrain
- 4. Home, working, and center-wide directories
- 5. Transfer files and data to Kilrain
- 6. Submit jobs to the batch queue
- 7. Batch queues
- 8. Monitoring your job
- 9. Saving your work
- 10. Modules
- 11. Available software
- 12. Advance Reservation Service
This document provides a brief summary of information that you'll need to know to quickly get started working on Kilrain. For more detailed information, see the Kilrain User Guide.
2. Get a Kerberos Ticket
For security purposes, you must have a current Kerberos ticket on your computer before attempting to connect to Kilrain. A Kerberos client kit must be installed on your desktop to enable you to get a Kerberos ticket. Information about installing Kerberos clients on your Windows desktop can be found at HPC Centers: Kerberos & Authentication.
3. Connect to Kilrain
Kilrain can be accessed via Kerberized ssh as follows:% ssh kilrain#.navo.hpc.mil (# = 1 to 2)
4. Home, working, and center-wide directories
Each user has file space in the $HOME, $WORKDIR, and $CENTER directories. The $HOME, $WORKDIR, and $CENTER environment variables are predefined for you and point to the appropriate locations in the file systems. You are strongly encouraged to use these variables in your scripts.
NOTE: $WORKDIR is a "scratch" file system, and $CENTER is a center-wide file system that is accessible to all center production machines, including the Utility Server. The $WORKDIR file system is not backed up. You are responsible for managing files in your $WORKDIR directories by backing up files to the archive system and deleting unneeded files. Currently, $WORKDIR files that have not been accessed in 30 days are subject to being purged.
5. Transfer files and data to Kilrain
File transfers to DSRC systems must be performed using Kerberized versions of the following tools: scp, ftp, sftp, and mpscp. For example, the command below uses secure copy (scp) to copy a local file into a destination directory on a Kilrain login node.% scp local_file kilrain#.navo.hpc.mil:/target_dir (# = 1 to 2)
For additional information on file transfers to and from Kilrain, see the File Transfers section of the Kilrain User Guide.
6. Submit jobs to the batch queue
The Portable Batch System (PBS Professional ™) is the workload management system for Kilrain. To submit a batch job, use the following command:qsub [ options ] my_job_script
where my_job_script is the name of the file containing your batch script. For more information on using PBS or on job scripts, see the Kilrain User Guide, the Kilrain PBS Guide, or the sample script examples found in the $SAMPLES_HOME directory on Kilrain.
7. Batch queues
The following table describes the PBS queues available on Kilrain:
|Highest||urgent||Urgent||24 Hours||4096||Designated urgent projects by DoD HPCMP|
|high||High||168 Hours||6144||Designated high-priority projects by service/agency|
|challenge||Challenge||168 Hours||6144||Challenge projects only|
|special||N/A||24 Hours||4096||Access available by request|
|debug||Debug||30 Minutes||1024||User diagnostic jobs|
|standard||Standard||168 Hours||4096||Non-challenge user jobs|
|bigmem||N/A||24 Hours||32||Large memory jobs|
|share||N/A||48 Hours||1||Serial jobs|
|transfer||N/A||24 Hours||1||Data transfer jobs|
|Lowest||background||Background||4 Hours||512||User jobs that will not be charged against the project allocation|
8. Monitoring your job
You can monitor your batch jobs on Kilrain using the qpeek or qstat commands.
The qstat command lists all jobs in the queue. The "-u username" option shows only jobs owned by the given user, as follows:
% qstat -u user1 Req'd Req'd Elap Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time ----------- -------- -------- --------- ------ --- --- ------ ----- - ----- 64485.h60s1 user1 backgrou tw2 14658 1 1 -- 12:00 R 01:12 64495.h60s1 user1 standard 45dh8 21308 2 1 -- 02:00 R 00:22 64513.h60s1 user1 challeng inspect -- 16 256 -- 06:00 Q --
Notice that the output contains the JobID for each job. This ID can be used with the qpeek, qstat, and qdel commands.
To delete a job, use the command "qdel jobID".
To delete all of your jobs, use "qdel `qselect -u username`".
To delete all of your running jobs (R status), use "qdel `qselect -u username -s R`".
To delete all of your queued (Q status), use "qdel `qselect -u username -s Q`".
To view a partially completed output file, use the "qpeek jobID" command.
9. Saving your work
When your job is finished, you should archive any important data to prevent automatic deletion by the purge scripts.
Copy one or more files to the archive system
rcp file1 $ARCHIVE_HOST:$ARCHIVE_HOME/
Copy one or more files from the archive system
rcp $ARCHIVE_HOST:$ARCHIVE_HOME/my_data/file1 .
For more information on archiving your files, see the Archive Guide.
Software modules are a very convenient way to set needed environment variables and include necessary directories in your path so that commands for particular applications can be found. Kilrain uses "modules" to initialize your environment with COTS application software, system commands and libraries, compiler suites, environment variables, and PBS batch system commands.
A number of modules are loaded automatically as soon as you log in. To see the modules that are currently loaded, run "module list". To see the entire list of available modules, run "module avail". You can modify the configuration of your environment by loading and unloading modules. For complete information on how to do this, see the Modules User Guide.
11. Available software
A list of software on Kilrain is available on the software page.
12. Advance Reservation Service
A subset of Kilrain's nodes has been set aside for use as part of the Advanced Reservation Service (ARS). The ARS allows users to reserve a user-designated number of nodes for a specified number of hours starting at a specific date/time. This service enables users to execute interactive or other time-critical jobs within the batch system environment. The ARS is accessible via most modern web browsers at https://reservation.hpc.mil/. Authenticated access is required. Windows Internet Explorer users may authenticate with either CAC or YubiKey. Non-Windows users must use YubiKey. An ARS User's Guide is available online once you have logged in.