JSON

Jobs in PROMINENCE are defined using JSON. It is possible to create JSON job descriptions and submit them directory to the REST API, or to use the CLI to submit them. This allows users to create jobs which are more complex than are possible using the CLI or to programmatically create jobs.

Generating JSON with the CLI

When prominence create is run with the --dry-run option, the job will not be submitted but the JSON description of the job will be printed to standard output. For example:

$ prominence create --dry-run --name test1 --cpus 4 --memory 8 --disk 20 busybox
{
  "resources": {
    "memory": 8,
    "cpus": 4,
    "nodes": 1,
    "disk": 20
  },
  "name": "test1",
  "tasks": [
    {
      "image": "busybox",
      "runtime": "singularity"
    }
  ]
}

If the JSON output is saved in a file it be submitted to PROMINENCE using the run command, e.g.:

$ prominence run <filename.json>

The job description can also be a URL rather than a file, e.g.

$ prominence run <https://.../filename.json>

Multiple tasks in a single job

By default a job will run a single command inside a single container. However, it is possible to instead run multiple sequential tasks within a single job. Each task will have access to the same temporary storage, so transient files generated by one task are accessible by other tasks.

To run multiple tasks it is necessary to construct a JSON description of the job. For example, in this job there are two sequential tasks:

{
  "resources": {
    "memory": 1,
    "cpus": 1,
    "nodes": 1,
    "disk": 10
  },
  "name": "multiple-tasks",
  "tasks": [
    {
      "image": "centos:7",
      "runtime": "singularity",
      "cmd": "cat /etc/redhat-release"
    },
    {
      "image": "centos:8",
      "runtime": "singularity",
      "cmd": "cat /etc/redhat-release"
    }
  ]
}

Use of JSON job descriptions is also necessary to run workflows, which we will come to next.