API reference¶
This page provides an auto-generated summary of pysumma’s API. For more details and examples, refer to the main documentation.
Interfaces to running SUMMA¶
Simulation¶
- class pysumma.Simulation(executable, filemanager, initialize=True, config_dir='.pysumma')[source]¶
The simulation object provides a wrapper for SUMMA simulations
- stdout¶
Store standard output of the run
- stderr¶
Handle to the process during runtime
- manager_path¶
Path to the file manager
- config_path¶
Path to where configuration will be written
- status¶
Current status of the simulation
- manager¶
File manager object (populated after calling
initialize
)
- decisions¶
Decisions object (populated after calling
initialize
)
- output_control¶
OutputControl object (populated after calling
initialize
)
- trial_params¶
Spatially distributed parameters (populated after calling
initialize
)
- force_file_list¶
ForcingList object (populated after calling
initialize
)
- global_hru_params¶
GlobalParams object for hru (populated after calling
initialize
)
- global_gru_params¶
GlobalParams object for gru (populated after calling
initialize
)
- local_attributes¶
LocalAttributes object (populated after calling
initialize
)
- initial_conditions¶
InitialConditions object (populated after calling
initialize
)
- apply_config(config: dict)[source]¶
Change the settings of the simulation based on a configuration dictionary.
- Parameters
config – A dictionary where keys represent the type of change and the values represent the changes to be applied. A representative example might be:
- config = {
‘file_manager’: ‘/home/user/cool_setup/file_manager_new.txt’, ‘decisions’: {‘snowLayers’: ‘CLM_2010’}, ‘parameters’: {‘albedoDecayRate’: 1e-6}, ‘trial_parameters’: {‘theta_mp’: 0.4}, ‘attributes’: {‘mHeight’: 15} }
- assign_attributes(name, data)[source]¶
Assign new data to the
local_attributes
dataset.- Parameters
name – The name (or key) of the attribute to modify
data – The data to change the attribute to. The shape must match the shape in the local attributes file
- assign_forcing_file(name: str, data: xarray.core.dataset.Dataset)[source]¶
Assign a new forcing dataset, writing out the data and updating the forcing file list and file manager
- Parameters
name (str) – The name of the new forcing file dataset
data (xr.Dataset) – The new forcing dataset
- assign_trial_params(name: str, data: numpy.array, dim='hru', create=True)[source]¶
Assign new data to the
spatial_params
dataset.- Parameters
name – The name (or key) of the attribute to modify
data – The data to change the parameter to. The shape must match the shape in the parameter trial file
- initialize()[source]¶
Initialize reads in all of the relevant files. This may not be desired on instantiation, so the
initialize
parameter can be set in the constructor. Calling this will also create a backup of the configuration that can be restored via thereset
method.
- property output¶
Get the output as an xarray dataset
- run(run_option='local', run_suffix='pysumma_run', processes=1, prerun_cmds=None, startGRU=None, countGRU=None, iHRU=None, freq_restart=None, progress=None, **kwargs)[source]¶
Run a SUMMA simulation and halt until completion or error.
- Parameters
run_option – Method to run SUMMA, must be one of either local or docker
run_suffix – Name to append to the output files for this SUMMA run
processes – Number of openmp processes to use for this run. For this to do anything SUMMA must be compiled with openmp support
prerun_cmds – A list of commands to execute before running SUMMA. May be necessary to set environment variables or do any preprocessing
startGRU – GRU index to start the simulation on (must also set
countGRU
if this argument is set)countGRU – Number of GRU to run, starting at
startGRU
(must also setstartGRU
if this argument is set)iHRU – Index of HRU to run (cannot be used with
startGRU
andcountGRU
)freq_restart – Frequency to write restart files. Options include
[y, m, d, never]
for yearly, monthly, and daily restart files. Defaults tonever
progress – Frequency to write stdout progress. Note this is not printed during runtime via pysumma, but can be checked after completion. Options include
[m, d, h, never]
for monthly, daily, and hourly output.
Ensemble¶
- class pysumma.Ensemble(executable: str, configuration: dict, filemanager: Optional[str] = None, num_workers: int = 1, threads_per_worker: int = 1, scheduler: Optional[str] = None, client: Optional[distributed.client.Client] = None)[source]¶
Ensembles represent an multiple SUMMA configurations based on changing the decisions or parameters of a given run.
- executable¶
Path to the SUMMA executable
- filemanager¶
Path to the file manager
- Type
(optional)
- configuration¶
Dictionary of runs, along with settings
- num_workers¶
Number of parallel workers to use
- simulations¶
Dictionary of run names and Simulation objects
- merge_output()[source]¶
Open and merge all of the output datasets from the ensemble run into a single dataset.
- open_output()[source]¶
Open all of the output datasets from the ensembe and return as a dictionary of datasets
- rerun_failed(run_option: str = 'local', prerun_cmds=None, monitor: bool = True)[source]¶
Try to re-run failed simulations.
- Parameters
run_option – Where to run the simulation. Can be
local
ordocker
prerun_cmds – A list of shell commands to run before running SUMMA
monitor – Whether to halt operation until runs are complete
- run(run_option: str = 'local', prerun_cmds=None, monitor: bool = True)[source]¶
Run the ensemble
- Parameters
run_option – Where to run the simulation. Can be
local
ordocker
prerun_cmds – A list of shell commands to run before running SUMMA
monitor – Whether to halt operation until runs are complete
Distributed¶
- class pysumma.Distributed(executable: str, filemanager: str, num_workers: int = 1, threads_per_worker: int = 1, chunk_size: Optional[int] = None, num_chunks: Optional[int] = None, scheduler: Optional[str] = None, client: Optional[distributed.client.Client] = None)[source]¶
Distributed objects represent SUMMA configurations where there are multiple GRU/HRU which are expected to be run in parallel.
Currently only supports GRU based parallelization.
- executable¶
Path to the SUMMA executable
- manager¶
FileManager object
- num_workers¶
Number of parallel workers to use
- chunk_args¶
List of dictionaries containing
startGRU
andcountGRU
values
- simulations¶
Dictionary of run names and Simulation objects
- open_output()[source]¶
Open all of the output datasets from the ensembe and return as a dictionary of datasets
- run(run_option: str = 'local', prerun_cmds=None, monitor: bool = True)[source]¶
Run the ensemble
- Parameters
run_option – Where to run the simulation. Can be
local
ordocker
prerun_cmds – A list of shell commands to run before running SUMMA
monitor – Whether to halt operation until runs are complete
Ostrich¶
- class pysumma.calibration.Ostrich(ostrich_executable, summa_executable, file_manager, python_path='/home/docs/checkouts/readthedocs.org/user_builds/pysumma/envs/develop/bin/python')[source]¶
Provides a high level interface to the OSTRICH optimization package. This class can currently only be used for single-objective optimization using the DDS algorithm as defined in the template file. Currently the metrics calculated are KGE, MAE, and MSE as defined in the evaluation package, though more metrics can be implemmented quite easily.
A basic workflow for this object is:
import pysumma as ps summa_exe = './summa.exe' ostrich_exe = './ostrich.exe' file_manager = './file_manager.txt' python_exe = '/pool0/data/andrbenn/.conda/all/bin/python' ostrich = ps.Ostrich(ostrich_exe, summa_exe, file_manager, python_path=python_exe) ostrich.calib_params = [ ps.OstrichParam('paramName', starValue, (minValue, maxValue)), ] ostrich.obs_data_file = 'obs_data.nc' ostrich.sim_calib_var = 'sim_varname' ostrich.obs_calib_var = 'obs_varname' ostrich.write_config() ostrich.run()
- ostrich¶
Path to OSTRICH executable
- python_path¶
Path to Python executable used for the
run_script
. Note, you may need to set this if you are running the calibration from inside a non-default environment (ie from conda/poetry/etc)!
- summa¶
Path to the SUMMA executable
- template¶
OSTRICH configuration file template
- save_template¶
Template for script to save best parameters
- run_template¶
Template for script to run and evaluate SUMMA
- config_path¶
Path to location of calibration runs/logs
- simulation¶
pysumma Simulation object used as template
- file_manager¶
File manager file for SUMMA simulation
- seed¶
Random seed for calibration
- errval¶
Error value for OSTRICH
- perturb_val¶
Strength of parameter perturbations during calibration
- max_iters¶
Number of calibration trial runs
- cost_function¶
Metric to use when ranking calibration runs
- maximize¶
Whether to maximize the
cost_function
- simulation_kwargs¶
Keyword arguments to pass to the simulation run function
- property map_vars_to_run_template¶
For completion of the model run script template
- property map_vars_to_save_template¶
For completion of the parameter saving template
- property map_vars_to_template¶
For completion of the OSTRICH input template
- property param_section: str¶
Write list of calibration parameters
- property response_section: str¶
Write section of OSTRICH configuration for selecting metric
- test_runscript(prerun_cmds=[], monitor=True)[source]¶
Run a single instance of the underlying runscript
- property tied_param_section: str¶
Write list of tied calibration parameters
- property tied_response_section: str¶
Write section for determining if we are maximizing or minimizing the metric