HPC::Runner::App::Scheduler::ParseInput

Parse the infile for HPC params, jobs, and batches

Subroutines

parse_file_slurm

Parse the file looking for the following conditions

lines ending in `\` wait nextnode

Batch commands in groups of $self->cpus_per_task, or smaller as wait and nextnode indicate

post_process_file_slurm

check_for_commands

Check all jobs to make sure they have commands

process_lines

Iterate through all lines in the job file 1. Sanity check - can't use nohup or push commands to background 2. Check for HPC meta - #HPC 3. Check for Note meta

check_lines_add_cmd

Append to the command

We check for a few cases

1. A line that is terminated by the usual newline character

echo "hello!"

2. A multiline command in the usual bash sense

echo "goodbye!" && \
    echo "not again!"

3. The command is wait. Submit jobs we already have to the scheduler, and any jobs after 'wait', depend upon jobs before 'wait' finishing.

wait

4. Deprecated! The command is 'newnode' on a line by itself. Submit all the previous jobs, but no dependenciies. Instead please use '#HPC commands_per_node' within your job file.

#HPC jobname=job01
#HPC commands_per_node=1
#HPC cpus_per_task=1

gzip VERY_LARGE_FILE
gzip OTHER_VERY_LARGE_FILE

check_sanity

Do some sanity checks. So far we only check for nohup, because nohup confuses schedulers.

#TODO Add check for when line ends with &. This also confuses schedulers

check_note_meta

Check for lines starting with #TASK - used to pass per process task_tags

process_hpc_meta

allow for changing parameters mid through the script

#Job1 echo "this is job one" && \ bin/dostuff bblahblahblah

#HPC cpu_per_task=12

echo "This is my new job with new HPC params!"

Make sure our hpc variables are current for filling in the template #HPC cpus_per_task=1 to #SBATCH --cpus-per-task=1

cpanm HPC::Runner::Command

perl -MCPAN -e shell
install HPC::Runner::Command