JMS: A workflow management system and web-based cluster front-end for the Torque resource manager
This is an incremental tool for researchers needing to run resource-intensive workflows on clusters and integrate them into web servers.
The researchers tackled the problem of managing complex computational pipelines on distributed clusters by developing the Job Management System (JMS), which provides a user-friendly interface for creating workflows and integrates with the Torque resource manager, currently used for structural bioinformatics pipelines at Rhodes University.
Motivation: Complex computational pipelines are becoming a staple of modern scientific research. Often these pipelines are resource intensive and require days of computing time. In such cases, it makes sense to run them over distributed computer clusters where they can take advantage of the aggregated resources of many powerful computers. In addition to this, researchers often want to integrate their workflows into their own web servers. In these cases, software is needed to manage the submission of jobs from the web interface to the cluster and then return the results once the job has finished executing. Results: We have developed the Job Management System (JMS), a workflow management system and interface for the Torque resource manager. The JMS provides users with a user-friendly interface for creating complex workflows with multiple stages. It integrates this workflow functionality with Torque, a tool that is used to control and manage batch jobs on distributed computing clusters. The JMS can be used by researchers to build and run complex computational pipelines and provides functionality to include these pipelines in external interfaces. The JMS is currently being used to house a number of structural bioinformatics pipelines at the Research Unit in Bioinformatics (RUBi) at Rhodes University. Availability: The JMS is an open-source project and is freely available at https://github.com/RUBi-ZA/JMS