Submitting a Quantum Espresso job
Jump to navigation
Jump to search
Go back to Scripts_for_QE
To submit to CSUC collserola, usually for small jobs (max 24 cores per node), shorter waiting queue for small jobs.
#!/bin/bash # specifies the name of your job # BSUB -J jobname # # send standard output here # BSUB -o quantum_espresso.log # # send standard error here # BSUB -e quantum_espresso.err # # pick a queue # BSUB -q parallel8 # # pick the machine # BSUB -R collserola # # Indicate the number of cores # BSUB -n 8 # All process should be executed in the same node # BSUB -R span[hosts=1] # # Send an email notice once the job is finished # BSUB -N -u MYEMAIL@iciq.es export ESPRESSO_PSEUDO=$HOME/espresso_pseudo INDIR=$ESPRESSO_TMPDIR # directory with your input and the .UPF files INPUT=INPUT.in OUTPUT=OUTPUT.out date # Set up the environment . /opt/modules/default/init/bash module load quantumespresso/6.1 export ESPRESSO_PSEUDO=$HOME/espresso_pseudo unset OMP_NUM_THREADS EXE="/prod/QuantumESPRESSO/6.1/bin/pw.x -environ" # Change to the work directory and copy all the necessary files to this folder cd $TMPDIR cp $INDIR/$INPUT $INDIR/*.UPF $TMPDIR cp $INDIR/environ_water.in $TMPDIR/environ.in # Run the job mpijob $EXE < $INPUT > $OUTPUT # Create a new directory in the $SCRATCH directory, each new directory for each job specifying the JobID, and copy the output to this directory mkdir -p $SCRATCH/output_$LSB_JOBID cp -r * $SCRATCH/output_$LSB_JOBID # move in HOME and delete in scratch mv $SCRATCH/output_$LSB_JOBID $HOME/. date
To submit to CSUC pirineus, usually for very big jobs that requires a lot of nodes and therefore longer waiting queue
#!/bin/ksh # specifies the name of your job # BSUB -J jobname # # send standard output here # BSUB -o quantum_espresso.log # # send standard error here # BSUB -e quantum_espresso.err # # pick a queue # BSUB -q parallel32 # # pick the machine # BSUB -R pirineus # # Indicate the number of cores # BSUB -n 32 # All process should be executed in the same node # BSUB -R span[hosts=1] # # Send an email notice once the job is finished # BSUB -N -u MYEMAIL@iciq.es INDIR=$HOME/WORKDIRECTORY # directory with your input and the .UPF files INPUT=INPUT.in OUTPUT=OUTPUT.out date # Set up the environment . /opt/modules/default/init/bash module load quantumespresso/6.1 export ESPRESSO_PSEUDO=$HOME/espresso_pseudo unset OMP_NUM_THREADS EXE="/prod/QuantumESPRESSO/6.1/bin/pw.x -environ" # Change to the work directory and copy all the necessary files to this folder cd $TMPDIR cp $INDIR/$INPUT $INDIR/*.UPF $TMPDIR cp $INDIR/environ_water.in $TMPDIR/environ.in # Run the job mpijob $EXE < $INPUT > $OUTPUT # Create a new directory in the $SCRATCH directory, each new directory for each job specifying the JobID, and copy the output to this directory mkdir -p $SCRATCH/output_$LSB_JOBID cp -r * $SCRATCH/output_$LSB_JOBID # move in HOME and delete in scratch mv $SCRATCH/output_$LSB_JOBID $HOME/. date
Jobs are sent to CSUC queue with the command
bsub< jobname.lsf
Do not forget the "<" or you will see the message that your job was submitted to the default "short" queue but in reality nothing happened.
To submit a job to CESVIMA pmagerit
#!/bin/bash
#----------------------- Start job description -----------------------
#@ total_tasks = 32
#@ wall_clock_limit = 48:00:00
#@ output = out-%j.log
#@ error = err-%j.log
#@ initialdir = /home/iciq23/iciq23336/espresso_tmp/
#------------------------ End-job description ------------------------
#-------------------------- Start execution --------------------------
# Run our program
module load gcc/4.7
srun /sw/openmpi/Quantum-ESPRESSO/6.1/bin/pw.x < INPUT.in > OUTPUT.out
#--------------------------- End execution ---------------------------
Jobs are sent CESVIMA queue with
jobsubmit jobname.sh
To submit an environ job to MareNostrum4 (MN4)
#!/bin/bash
#SBATCH --ntasks=48
#SBATCH --tasks-per-node=48
#SBATCH --time=48:00:00 ## hour
#SBATCH --cpus-per-task=1
#SBATCH --constraint=highmem
#SBATCH --error=qe%J.err
#SBATCH --job-name="hello_test"
module load quantumespresso/6.2-environ-1.0
PREFIXfolder=FeNi2O4
INPUT=${PREFIXfolder}.in2
OUTPUT=${PREFIXfolder}.out2
export ESPRESSO_PSEUDO=$HOME/espresso_pseudo
export ESPRESSO_TMPDIR=$SCRATCH/espresso_tmp/vacuum/${PREFIXfolder} #all temp files goes there
export WORKDIR=$HOME/espresso_tmp/vacuum/spinel #your work directory
mkdir $ESPRESSO_TMPDIR
#cp environ_vacuum.in environ.in
time srun pw.x -environ < $INPUT > $OUTPUT
To submit an environ job to MareNostrum4
sbatch script.lsf
WARNING: The previous script does not use the scratch directory of MareNostrum4. You risk to reach your disk quota quickly. The following script use the scratch but the variable SCRATCH must be defined in your .bashrc first! (export SCRATCH=/gpfs/scratch/iciq72/iciq72520 in my case)
Writing wavefunctions in QE needs a very high amount of memory; often calculations crash due to exceeding memory. It is generally a good idea to add the line:
SBATCH --constraint=highmem
It runs calculations in a high memory node.
#!/bin/bash
#SBATCH --ntasks=48
#SBATCH --tasks-per-node=48
#SBATCH --time=1:00:00 ## hour
#SBATCH --cpus-per-task=1
#SBATCH --error=qe%J.err
#SBATCH --job-name="hello_test"
module load quantumespresso/6.2-environ-1.0
PREFIXfolder=h2o_water
INPUT=${PREFIXfolder}.in
OUTPUT=${PREFIXfolder}.out1
export ESPRESSO_PSEUDO=$HOME/espresso_pseudo
export ESPRESSO_TMPDIR=$SCRATCH/espresso_tmp/water #all temp files goes there
export WORKDIR=$HOME/espresso_tmp/water
cp environ_water.in environ.in
# Run the job in SCRATCH
time srun pw.x -environ < $INPUT > $OUTPUT
# copy back in HOME the pot.save folder
cp -r $ESPRESSO_TMPDIR/$PREFIXfolder.save $WORKDIR/.
Here is a python script that make the lsf submission script for you. You just need to replace the variables name_prefix and medium.
#!/usr/bin/python2
#This script write the MN4 lsf script for job submission
name_prefix="sym5"
#output=name_prefix+".in" #output filename for QE input run
output_lsf=name_prefix+".lsf96" #
LDOS=name_prefix+"_LDOS.in" #filename for LDOS calculation3
LDOS_lsf=name_prefix+"_LDOS.lsf48" #filename for LDOS calculation
medium="vacuum"
outlsf=open(output_lsf,"w")
print >> outlsf, "#!/bin/bash"
print >> outlsf, "#SBATCH --ntasks=96"
print >> outlsf, "#SBATCH --tasks-per-node=48"
print >> outlsf, "#SBATCH --time=48:00:00 ## hour"
print >> outlsf, "#SBATCH --cpus-per-task=1"
print >> outlsf, "#SBATCH --error=qe%J.err"
print >> outlsf, ""
print >> outlsf, "module load quantumespresso/6.2-environ-1.0"
print >> outlsf, "INPUT="+name_prefix+".in"
print >> outlsf, "OUTPUT="+name_prefix+".out1"
print >> outlsf, ""
print >> outlsf, "export ESPRESSO_PSEUDO=$HOME/espresso_pseudo"
print >> outlsf, "export ESPRESSO_TMPDIR=$SCRATCH/espresso_tmp/"+medium
print >> outlsf, "export WORKDIR=$HOME/espresso_tmp/"+medium
print >> outlsf, "cp environ_"+medium+".in environ.in"
print >> outlsf, ""
print >> outlsf, "time srun pw.x -environ < $INPUT > $OUTPUT"
print >> outlsf, ""
print >> outlsf, "# copy back in HOME the pot.save folder"
print >> outlsf, "cp -r $ESPRESSO_TMPDIR/$PREFIXfolder.save $WORKDIR/."
outlsf.close
#writing QE LDOS file
ldos=open(LDOS,"w")
print >> ldos, " &PROJWFC"
print >> ldos, " prefix =","'"+name_prefix+"'", ","
print >> ldos, " !outdir = '/media/hnguyen/WAREHOUSE/tekla/espresso_tmp/SurfaceUSPP/001' ,"
print >> ldos, " DeltaE = 0.01 ,"
print >> ldos, " Emin = -150.0 ,"
print >> ldos, " Emax = 30.0 ,"
print >> ldos, " ngauss = 0,"
print >> ldos, " degauss = 0.001469972, ! =0.02 eV"
print >> ldos, " pawproj = .false. ,"
print >> ldos, " /"
ldos.close
#writing LDOS lsf script
ldoslsf=open(LDOS_lsf,"w")
print >> ldoslsf, "#!/bin/bash"
print >> ldoslsf, "#SBATCH --ntasks=48"
print >> ldoslsf, "#SBATCH --tasks-per-node=48"
print >> ldoslsf, "#SBATCH --time=10:00:00 ## hour"
print >> ldoslsf, "#SBATCH --cpus-per-task=1"
print >> ldoslsf, "#SBATCH --error=qe%J.err"
print >> ldoslsf, ""
print >> ldoslsf, "module load quantumespresso/6.2-environ-1.0"
print >> ldoslsf, "PREFIXfolder="+name_prefix
print >> ldoslsf, "INPUT=${PREFIXfolder}_LDOS.in"
print >> ldoslsf, "OUTPUT=${PREFIXfolder}_LDOS.out"
print >> ldoslsf, ""
print >> ldoslsf, "export ESPRESSO_PSEUDO=$HOME/espresso_pseudo"
print >> ldoslsf, "export ESPRESSO_TMPDIR=$SCRATCH/espresso_tmp/"+medium
print >> ldoslsf, "export WORKDIR=$HOME/espresso_tmp/"+medium
print >> ldoslsf, ""
print >> ldoslsf, "time srun projwfc.x < $INPUT > $OUTPUT"
print >> ldoslsf, "mkdir PDOS"
print >> ldoslsf, "mkdir PDOS/$PREFIXfolder"
print >> ldoslsf, "mv *pdos* PDOS/$PREFIXfolder/."
print >> ldoslsf, "grep 'polarization' $OUTPUT > ${OUTPUT}.polarization"
print >> ldoslsf, "grep 'charge' $OUTPUT > ${OUTPUT}.charge"
print >> ldoslsf, ""
print >> ldoslsf, "# copy back in HOME the pot.save folder"
print >> ldoslsf, "cp -r $ESPRESSO_TMPDIR/${PREFIXfolder}.save $WORKDIR/."
print >> ldoslsf, "cp -r ${OUTPUT}.polarization ${OUTPUT}.charge $WORKDIR/."
print >> ldoslsf, "cp -r PDOS/$PREFIXfolder $WORKDIR/PDOS/."
ldoslsf.close
A simple script to submit to tekla
#!/bin/bash
# - Dra. Nuria's Lopez Group -
##########################################
# SGE Parameters
##########################################
#$ -S /bin/bash
#$ -N Jobname
#$ -cwd
#$ -masterq c24m128ib.q
#$ -pe c24m128ib_mpi 24
#$ -m ae
#$ -M youremail@iciq.es
#$ -o $JOB_NAME.o$JOB_ID
#$ -e $JOB_NAME.e$JOB_ID
cat $TMP/machines.$JOB_ID >> $JOB_NAME.MACHINES.$JOB_ID
# Set up the environment
. /etc/profile.d/modules.sh
module load quantum-espresso/6.1
##########################################
# Running Job
##########################################
export OMP_NUM_THREADS=1
echo $PWD >> $JOB_NAME.o$JOB_ID
echo $TMP >> $JOB_NAME.o$JOB_ID
PREFIXfolder=Jobname
INPUT=${PREFIXfolder}.in
OUTPUT=${PREFIXfolder}.out1
export ESPRESSO_PSEUDO=$HOME/espresso_pseudo #your pseudopotential directory
export ESPRESSO_TMPDIR=$HOME/espresso_tmp/vacuum/${PREFIXfolder} #all temp files goes there
export WORKDIR=$HOME/espresso_tmp/vacuum/spinel #your work directory
mkdir $ESPRESSO_TMPDIR
time mpirun -np $NSLOTS $BIN_DIR/pw.x -input $INPUT > $OUTPUT
The folders for ESPRESSO_PSEUDO, ESPRESSO_TMPDIR must exist for the script to work correctly.