API reference

This page provides an auto-generated summary of pysumma’s API. For more details and examples, refer to the main documentation.

Interfaces to running SUMMA

Simulation

class pysumma.Simulation(executable, filemanager, initialize=True, config_dir='.pysumma')[source]

The simulation object provides a wrapper for SUMMA simulations

stdout

Store standard output of the run

stderr

Handle to the process during runtime

manager_path

Path to the file manager

config_path

Path to where configuration will be written

status

Current status of the simulation

manager

File manager object (populated after calling initialize)

decisions

Decisions object (populated after calling initialize)

output_control

OutputControl object (populated after calling initialize)

trial_params

Spatially distributed parameters (populated after calling initialize)

force_file_list

ForcingList object (populated after calling initialize)

global_hru_params

GlobalParams object for hru (populated after calling initialize)

global_gru_params

GlobalParams object for gru (populated after calling initialize)

local_attributes

LocalAttributes object (populated after calling initialize)

initial_conditions

InitialConditions object (populated after calling initialize)

apply_config(config: dict)[source]

Change the settings of the simulation based on a configuration dictionary.

Parameters

config – A dictionary where keys represent the type of change and the values represent the changes to be applied. A representative example might be:

config = {

‘file_manager’: ‘/home/user/cool_setup/file_manager_new.txt’, ‘decisions’: {‘snowLayers’: ‘CLM_2010’}, ‘parameters’: {‘albedoDecayRate’: 1e-6}, ‘trial_parameters’: {‘theta_mp’: 0.4}, ‘attributes’: {‘mHeight’: 15} }

assign_attributes(name, data)[source]

Assign new data to the local_attributes dataset.

Parameters
  • name – The name (or key) of the attribute to modify

  • data – The data to change the attribute to. The shape must match the shape in the local attributes file

assign_forcing_file(name: str, data: xarray.core.dataset.Dataset)[source]

Assign a new forcing dataset, writing out the data and updating the forcing file list and file manager

Parameters
  • name (str) – The name of the new forcing file dataset

  • data (xr.Dataset) – The new forcing dataset

assign_trial_params(name: str, data: numpy.array, dim='hru', create=True)[source]

Assign new data to the spatial_params dataset.

Parameters
  • name – The name (or key) of the attribute to modify

  • data – The data to change the parameter to. The shape must match the shape in the parameter trial file

get_output_files() List[str][source]

Find output files given the stdout generated from a run

initialize()[source]

Initialize reads in all of the relevant files. This may not be desired on instantiation, so the initialize parameter can be set in the constructor. Calling this will also create a backup of the configuration that can be restored via the reset method.

monitor()[source]

Halt execution until Simulation either finishes or errors

property output

Get the output as an xarray dataset

reset()[source]

Restores the original settings of the Simulation

run(run_option='local', run_suffix='pysumma_run', processes=1, prerun_cmds=None, startGRU=None, countGRU=None, iHRU=None, freq_restart=None, progress=None, **kwargs)[source]

Run a SUMMA simulation and halt until completion or error.

Parameters
  • run_option – Method to run SUMMA, must be one of either local or docker

  • run_suffix – Name to append to the output files for this SUMMA run

  • processes – Number of openmp processes to use for this run. For this to do anything SUMMA must be compiled with openmp support

  • prerun_cmds – A list of commands to execute before running SUMMA. May be necessary to set environment variables or do any preprocessing

  • startGRU – GRU index to start the simulation on (must also set countGRU if this argument is set)

  • countGRU – Number of GRU to run, starting at startGRU (must also set startGRU if this argument is set)

  • iHRU – Index of HRU to run (cannot be used with startGRU and countGRU)

  • freq_restart – Frequency to write restart files. Options include [y, m, d, never] for yearly, monthly, and daily restart files. Defaults to never

  • progress – Frequency to write stdout progress. Note this is not printed during runtime via pysumma, but can be checked after completion. Options include [m, d, h, never] for monthly, daily, and hourly output.

start(run_option='local', run_suffix='pysumma_run', processes=1, prerun_cmds=[], startGRU=None, countGRU=None, iHRU=None, freq_restart=None, progress=None, **kwargs)[source]

Run a SUMMA simulation without halting. Most likely this should not be used. Use the run method for most common use cases.

validate_layer_params(params)[source]

Ensure that the layer parameters are valid

Ensemble

class pysumma.Ensemble(executable: str, configuration: dict, filemanager: Optional[str] = None, num_workers: int = 1, threads_per_worker: int = 1, scheduler: Optional[str] = None, client: Optional[distributed.client.Client] = None)[source]

Ensembles represent an multiple SUMMA configurations based on changing the decisions or parameters of a given run.

executable

Path to the SUMMA executable

filemanager

Path to the file manager

Type

(optional)

configuration

Dictionary of runs, along with settings

num_workers

Number of parallel workers to use

simulations

Dictionary of run names and Simulation objects

merge_output()[source]

Open and merge all of the output datasets from the ensemble run into a single dataset.

monitor()[source]

Halt computation until submitted simulations are complete

open_output()[source]

Open all of the output datasets from the ensembe and return as a dictionary of datasets

rerun_failed(run_option: str = 'local', prerun_cmds=None, monitor: bool = True)[source]

Try to re-run failed simulations.

Parameters
  • run_option – Where to run the simulation. Can be local or docker

  • prerun_cmds – A list of shell commands to run before running SUMMA

  • monitor – Whether to halt operation until runs are complete

run(run_option: str = 'local', prerun_cmds=None, monitor: bool = True)[source]

Run the ensemble

Parameters
  • run_option – Where to run the simulation. Can be local or docker

  • prerun_cmds – A list of shell commands to run before running SUMMA

  • monitor – Whether to halt operation until runs are complete

start(run_option: str = 'local', prerun_cmds: Optional[list] = None)[source]

Start running the ensemble members.

Parameters
  • run_option – The run type. Should be either ‘local’ or ‘docker’

  • prerun_cmds – A list of preprocessing commands to run

summary()[source]

Show the user information about ensemble status

Distributed

class pysumma.Distributed(executable: str, filemanager: str, num_workers: int = 1, threads_per_worker: int = 1, chunk_size: Optional[int] = None, num_chunks: Optional[int] = None, scheduler: Optional[str] = None, client: Optional[distributed.client.Client] = None)[source]

Distributed objects represent SUMMA configurations where there are multiple GRU/HRU which are expected to be run in parallel.

Currently only supports GRU based parallelization.

executable

Path to the SUMMA executable

manager

FileManager object

num_workers

Number of parallel workers to use

chunk_args

List of dictionaries containing startGRU and countGRU values

simulations

Dictionary of run names and Simulation objects

monitor()[source]

Halt computation until submitted simulations are complete

open_output()[source]

Open all of the output datasets from the ensembe and return as a dictionary of datasets

run(run_option: str = 'local', prerun_cmds=None, monitor: bool = True)[source]

Run the ensemble

Parameters
  • run_option – Where to run the simulation. Can be local or docker

  • prerun_cmds – A list of shell commands to run before running SUMMA

  • monitor – Whether to halt operation until runs are complete

start(run_option: str = 'local', prerun_cmds: Optional[List] = None)[source]

Start running the ensemble members.

Parameters
  • run_option – The run type. Should be either ‘local’ or ‘docker’

  • prerun_cmds – A list of preprocessing commands to run

summary()[source]

Show the user information about ensemble status

Ostrich

class pysumma.calibration.Ostrich(ostrich_executable, summa_executable, file_manager, python_path='/home/docs/checkouts/readthedocs.org/user_builds/pysumma/envs/latest/bin/python')[source]

Provides a high level interface to the OSTRICH optimization package. This class can currently only be used for single-objective optimization using the DDS algorithm as defined in the template file. Currently the metrics calculated are KGE, MAE, and MSE as defined in the evaluation package, though more metrics can be implemmented quite easily.

A basic workflow for this object is:

import pysumma as ps
summa_exe = './summa.exe'
ostrich_exe = './ostrich.exe'
file_manager = './file_manager.txt'
python_exe = '/pool0/data/andrbenn/.conda/all/bin/python'
ostrich = ps.Ostrich(ostrich_exe, summa_exe, file_manager, python_path=python_exe)
ostrich.calib_params = [
    ps.OstrichParam('paramName', starValue, (minValue, maxValue)),
]
ostrich.obs_data_file = 'obs_data.nc'
ostrich.sim_calib_var = 'sim_varname'
ostrich.obs_calib_var = 'obs_varname'
ostrich.write_config()
ostrich.run()
ostrich

Path to OSTRICH executable

python_path

Path to Python executable used for the run_script. Note, you may need to set this if you are running the calibration from inside a non-default environment (ie from conda/poetry/etc)!

summa

Path to the SUMMA executable

template

OSTRICH configuration file template

save_template

Template for script to save best parameters

run_template

Template for script to run and evaluate SUMMA

config_path

Path to location of calibration runs/logs

simulation

pysumma Simulation object used as template

file_manager

File manager file for SUMMA simulation

seed

Random seed for calibration

errval

Error value for OSTRICH

perturb_val

Strength of parameter perturbations during calibration

max_iters

Number of calibration trial runs

cost_function

Metric to use when ranking calibration runs

maximize

Whether to maximize the cost_function

simulation_kwargs

Keyword arguments to pass to the simulation run function

property map_vars_to_run_template

For completion of the model run script template

property map_vars_to_save_template

For completion of the parameter saving template

property map_vars_to_template

For completion of the OSTRICH input template

property param_section: str

Write list of calibration parameters

property response_section: str

Write section of OSTRICH configuration for selecting metric

run(prerun_cmds=[], monitor=True)[source]

Start calibration run

test_runscript(prerun_cmds=[], monitor=True)[source]

Run a single instance of the underlying runscript

property tied_param_section: str

Write list of tied calibration parameters

property tied_response_section: str

Write section for determining if we are maximizing or minimizing the metric

validate()[source]

Try to make sure the configuration is usable

write_config()[source]

Writes all necessary files for calibration

write_weight_template_section(file_name=PosixPath('param_mapping.tpl')) pathlib.Path[source]

Write the parameter name mapping for OSTRICH

write_weight_value_section(file_name='param_weights.txt') pathlib.Path[source]

Write the parameter values for OSTRICH

Plotting utilities

Evaluation utilities