Skip to content
import atexit
import os
import shutil
import stat

import docker

Running duqtools with prominence

This Tutorial uses docker to create runs, and prominence to submit runs with duqtools

Getting started

Needed Packages

  • The first thing we need is to install duqtools, if you haven't done this yet you can install it via pip install duqtools
  • We will also need access to the source of duqtools, it can be downloaded from, This notebook was created with version 1.5.0 of duqtools
  • docker
    • We also need access to the following images (or equivalent): jintrac-imas. which can be found here
  • prominence, and we need to be logged in

Some example data

In this tutorial we will use example data, as IMAS is non-free you will have to get your IMAS data through your preferred supplier. We assume that the system is run with JINTRAC (since JINTRAC has a docker container available), and we use a simple JINTRAC template, which you are of course free to substitute for your own

shutil.rmtree('./example', ignore_errors=True)
# We assume that the duqtools source is available and that we are in the docs/examples folder
shutil.copytree('../../tests/test_data/template_model', 'example/template');
# Some imas data, substitute with your own, as we are not allowed to provide it
shutil.copytree('../../../containerized_runs/imasdb', 'example/imasdb');
# A silly script to support venvs
with open("example/", "w") as f:
set -e
. /etc/profile.d/
module load IMAS
module load fc2k

if [ -d .venv ]; then
 . .venv/bin/activate

exec "${@}"

st = os.stat('example/')
os.chmod('example/', st.st_mode | stat.S_IEXEC)
!chmod +x example/
# A simple configuration file for duqtools
with open("example/duqtools.yaml", "w") as f:
  name: jetto
  template: ./template #TODO edit to directory where initial run is stored
    user: "/example/imasdb"
    db: "jet"
    shot: 123
    run: 1
  - operator: multiply
    scale_to_error: false
    values: [1.1, 1.2, 1.3]
    variable: t_e
  - operator: multiply
    scale_to_error: false
    values: [1.1, 1.2, 1.3]
    variable: zeff
    method: latin-hypercube
    n_samples: 3
  submit_system: prominence
!tree example #Its okay if this fails if you don't have tree installed
├── duqtools.yaml
├── imasdb
│   └── jet
│       └── 3
│           └── 0
│               ├── ids_1230001.characteristics
│               ├── ids_1230001.datafile
│               └── ids_1230001.tree
└── template
    ├── jetto.eqrestart
    ├── jetto.ex
    ├── jetto.jset
    ├── jetto.sgrid
    ├── jetto.sin
    ├── jintrac_imas_config.cfg
    ├── rjettov
    └── utils_jetto

5 directories, 14 files

# Get the image and tag it, we could use a imas-only image for prominence
!docker login
!docker pull
!docker tag jintrac-imas:pulled
Authenticating with existing credentials...
WARNING! Your password will be stored unencrypted in /home/vikko/.docker/config.json.
Configure a credential helper to remove this warning. See

Login Succeeded
latest: Pulling from containerization/jintrac/jintrac-imas-installer/jintrac-imas
Digest: sha256:1b99ea457ba42bdd6809ef0069bf444ac2f8c3a1013ae9fe596da6f15987d584
Status: Image is up to date for

def niceprint(s):
    return print(str(s.output, 'utf-8'))

def run_inside(container, cmd):
    return container.exec_run(' '.join(['/example/', cmd]))
# clean up existing container

client = docker.from_env()
container ='jintrac-imas:latest',
                                  command="tail -f /dev/null",
                                  volumes=[os.getcwd() + "/example:/example"],
atexit.register(lambda: container.stop(timeout=0));
run_inside(container, 'python3 -m venv .venv');
niceprint(run_inside(container, 'python3 -m pip install duqtools -q'))
WARNING: You are using pip version 20.2.4; however, version 23.0.1 is available.
You should consider upgrading via the '/example/.venv/bin/python3 -m pip install --upgrade pip' command.

niceprint(run_inside(container, 'duqtools create --force --yes'))
14:57:19 [WARNING] Python module 'omas' not found. Submodule 'jams' needs it
14:57:19 [WARNING] Python module 'netCDF4' not found. Submodule 'transp' needs it
14:57:19 [WARNING] Python module 'tkinter' not found. Submodule 'tkinter_helpers' needs it

Operations in the Queue:
- Creating run : /example/run_0000
- Creating run : /example/run_0001
- Creating run : /example/run_0002
Applying Operations

  0%|          | 0/21 [00:00<?, ?it/s]
Creating run : /example/run_0000: it/s]
Progress:  10%|▉         | 2/21 [00:00<00:04,  4.48it/s]
Creating run : /example/run_0001: 1, 10.42it/s]
Progress:  38%|███▊      | 8/21 [00:00<00:01,  9.40it/s]
Creating run : /example/run_0002: 00, 12.96it/s]
Progress:  67%|██████▋   | 14/21 [00:01<00:00, 10.92it/s]
Creating run : /example/run_0002: 00, 11.06it/s]
Progress: 100%|██████████| 21/21 [00:01<00:00, 13.01it/s]

#!cd example &amp;&amp; duqtools submit --force --yes