Config#

class maize.utilities.io.Config(packages: list[str] = <factory>, scratch: ~pathlib.Path = PosixPath('/tmp/tmp_4dt71zi'), batch_config: ~maize.utilities.execution.ResourceManagerConfig = <factory>, environment: dict[str, str] = <factory>, nodes: dict[str, ~typing.Any] = <factory>)[source]#

Bases: object

Global configuration options.

Parameters:

  • packages (list[str]) – List of namespace packages to load

  • scratch (pathlib.Path) – The directory the workflow should be created in. Uses a temporary directory by default.

  • batch_config (maize.utilities.execution.ResourceManagerConfig) – Default options to be passed to the batch submission system

  • environment (dict[str, str]) – Any environment variables to be set in the execution context

  • nodes (dict[str, Any]) – Entries specific to each node

Examples

Here’s an example configuration file with all sections:

 1# maize global configuration file example
 2
 3# Where to save temporary files and all workflow directories
 4scratch = "/tmp"
 5
 6# Additional packages to load
 7packages = [
 8    "maize.steps.mai"
 9]
10
11# Environment variables to be set globally
12[environment]
13EXAMPLE = "VARIABLE"
14OTHER_EXAMPLE = "OTHER_VARIABLE"
15
16# Batch job system options
17[batch]
18system = "slurm"  # Can be one of {'cobalt', 'flux', 'local', 'lsf', 'pbspro', 'rp', 'slurm'}
19max_jobs = 100  # The maximum number of jobs that can be submitted by a node at once
20queue = "core"
21project = "your_project"
22launcher = "srun"  # The launcher to use for the command, usually one of {'srun', 'mpirun', 'mpiexec'}
23walltime = "24:00:00"  # Job walltime limit, shorter times may improve queueing times
24
25# The next sections configure nodes requiring specific executables,
26# here we are configuring the node defined as ``Example``:
27[example]
28python = "path/to/python"
29
30# Modules to load
31modules = ["program_module/1.0"]
32
33# Commands and their paths
34commands.other_program = "path/to/other_program"
35
36# You can also expand environment variables that might have been set by the module
37commands.another_program = "$MODULE_ENV_VAR/another_program"
38
39# Scripts that require a specific interpreter
40scripts.program.interpreter = "path/to/python"
41scripts.program.location = "path/to/program/or/folder"
42
43# Default parameter settings for this node
44parameters.value = 42
__init__(packages: list[str] = <factory>, scratch: ~pathlib.Path = PosixPath('/tmp/tmp_4dt71zi'), batch_config: ~maize.utilities.execution.ResourceManagerConfig = <factory>, environment: dict[str, str] = <factory>, nodes: dict[str, ~typing.Any] = <factory>) → None#

Methods

__init__([packages, scratch, batch_config, ...])

from_default()

Create a default configuration from:

update(file)

Read a maize configuration file.

Attributes

scratch

packages

batch_config

environment

nodes

classmethod from_default() → Self[source]#

Create a default configuration from:

  • A path specified using the MAIZE_CONFIG environment variable

  • A config file at ~/.config/.maize.toml

update(file: Path) → None[source]#

Read a maize configuration file.

Parameters:

file – Path to the configuration file