How to set up OpenPBS on Ubuntu 20.04 for single node workload scheduling.

Photo by Anton Maksimov 5642.su on Unsplash

*Previously I wrote an article for setting up Slurm (link at the bottom), this is an accompanying article for those who like/prefer OpenPBS.

OpenPBS is a work scheduling tool for High-Performance computing clusters. In addition, it can be an invaluable tool on a local desktop or single server when you need to run several programs at once and queue them up whilst ensuring you don’t overload your computer or server. Furthermore, it can be useful in cases where you share a server with other users or need to run multiple jobs overnight or for weeks! Here I show you how to quickly set up OpeNPBS on a single machine with ubuntu 20.04. You will no longer need to make mangled scripts to run multiple programs to avoid going over your hardware limits or dispute with colleagues whose program gets to run first.

Prerequisites

  • Basic Linux CLI
  • An Ubuntu machine with internet access.

This guide is based on instruction from the repo: https://github.com/openpbs/openpbs/blob/master/INSTALL

First lets install all the package dependencies we need to get OpenPBS compiled.

$ sudo apt install gcc make libtool libhwloc-dev libx11-dev libxt-dev libedit-dev libical-dev ncurses-dev perl postgresql-server-dev-all postgresql-contrib unzip python3-dev tcl-dev tk-dev swig libexpat-dev libssl-dev libxext-dev libxft-dev autoconf automake g++$ sudo apt install expat libedit2 postgresql python3 postgresql-contrib sendmail-bin sudo tcl tk libical3 postgresql-server-dev-all

Now lets get OpenPBS compiled and installed (below is the latest version from master):

$ wget https://github.com/openpbs/openpbs/archive/refs/heads/master.zip$ unzip master.zip
$ cd openpbs-master
$ ./autogen.sh
$ ./configure --prefix=/opt/pbs
$ make
$ sudo make install

Now lets get OpenPBS setup and running:

$ sudo /opt/pbs/libexec/pbs_postinstall# Now you should change the value of PBS_START_MOM  from zero to one
$ sudo vi /etc/pbs.conf
$ sudo chmod 4755 /opt/pbs/sbin/pbs_iff /opt/pbs/sbin/pbs_rcp$ sudo systemctl start pbs.service

Now let's confirm everything is working and your local machine is registered to PBS using the pbsnode -a command:

$ pbsnodes -a
ip-172-31-18-27
Mom = ip-172-31-18-27.eu-west-2.compute.internal
ntype = PBS
state = job-busy
pcpus = 1
jobs = 0.ip-172-31-18-27/0
resources_available.arch = linux
resources_available.host = ip-172-31-18-27
resources_available.mem = 992204kb
resources_available.ncpus = 1
resources_available.vnode = ip-172-31-18-27
resources_assigned.accelerator_memory = 0kb
resources_assigned.hbmem = 0kb
resources_assigned.mem = 0kb
resources_assigned.naccelerators = 0
resources_assigned.ncpus = 1
resources_assigned.vmem = 0kb
resv_enable = True
sharing = default_shared
license = l
last_state_change_time = Sat Jan 15 18:45:39 2022

Now let's check the queuing system is active with qstat:

$ qstat -B
Server Max Tot Que Run Hld Wat Trn Ext Status
---------------- ----- ----- ----- ----- ----- ----- ----- ----- -----------
ip-172-31-18-27 0 0 0 0 0 0 0 0 Active

Now let's submit a test job to the default queue call workq:

$ echo "sleep 60" | qsub
1.ip-172-31-18-27
$ qstat -a
ip-172-31-18-27:
Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
--------------- -------- -------- ---------- ------ --- --- ------ ----- - -----
1.ip-172-31-18* ubuntu workq STDIN 13423 1 1 -- -- R 00:00

And now you have a functioning OpenPBS queuing system on your server. If you want to submit jobs via PBS scripts just google example PBS submission scripts and plenty of guides/examples will come up.

OpenPBS Homepage for more information https://www.openpbs.org

--

--

--

DevOps/Cloud engineer and a specialist in Scientific and High Performance Computing with a Phd in Computational and Theoretical Chemistry.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Benefits of Kotlin over Java: Backend and Android

Rancher: An Open Source Platform for Cloud Native Applications

what i have been waiting for has come !

Migrating your SFTP / FTP server to the cloud

Working CDK-OpenAPI - RestAPI - Lambda/Serverless - AWS (Java)

Getting started with Azure Resource Management Templates | ARM templates

How to Capture Snowflake Users, Roles, and Grants Into a Table

Design a Vending machine

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Pritesh Tailor

Pritesh Tailor

DevOps/Cloud engineer and a specialist in Scientific and High Performance Computing with a Phd in Computational and Theoretical Chemistry.

More from Medium

Humidity & Degrees Environment Sensory

How to setup Slurm on Ubuntu 20.04 for single node work scheduling.

What are Links in Linux?

Membuat Alert Disk Space dengan Bash Script

Membuat Alert Disk Space dengan Bash Script