pysumma package

Subpackages

• pysumma.hydroshare package
  • Submodules
  • pysumma.hydroshare.compat module
  • pysumma.hydroshare.hydroshare module
  • pysumma.hydroshare.progress module
  • pysumma.hydroshare.resource module
  • pysumma.hydroshare.threads module
  • pysumma.hydroshare.utilities module
  • Module contents
• pysumma.meta package
  • Module contents
• pysumma.plotting package
  • Submodules
  • pysumma.plotting.energy_balance module
  • pysumma.plotting.energybalance module
  • pysumma.plotting.hovmoller module
  • pysumma.plotting.layers module
  • pysumma.plotting.plot module
  • pysumma.plotting.plotting module
  • pysumma.plotting.spatial module
  • pysumma.plotting.timeseries module
  • pysumma.plotting.utils module
  • pysumma.plotting.water_balance module
  • pysumma.plotting.waterbalance module
  • Module contents
• pysumma.specworker package
  • Submodules
  • pysumma.specworker.compat module
  • pysumma.specworker.jobs module
  • pysumma.specworker.progress module
  • pysumma.specworker.run_tasks module
  • pysumma.specworker.tasks module
  • pysumma.specworker.worker module
  • Module contents
• pysumma.tests package
  • Submodules
  • pysumma.tests.test_Decisions module
  • pysumma.tests.test_ModelOutput module
  • pysumma.tests.test_Simulation module
  • Module contents

Submodules

pysumma.decisions module

class pysumma.decisions.DecisionOption(name, value)

Bases: pysumma.option.BaseOption

Container for lines in a decisions file

set_value(new_value)

class pysumma.decisions.Decisions(path)

Bases: pysumma.option.OptionContainer

The Decisions object provides an interface to a SUMMA decisions file.

set_option(key, value)

This has to be implemented by subclasses

pysumma.ensemble module

class pysumma.ensemble.Ensemble(executable: str, filemanager: str, configuration: dict, num_workers: int = 1)

Bases: object

Ensembles represent multiple SUMMA configurations based on changing the decisions file or parameter values. At it’s core, the Ensemble class is a thin wrapper around multiple Simulation objects, whos execution is managed by a dask Client.

A standard workflow for running an ensemble is:

import pysumma as ps

param_options = {
    'fixedThermalCond_snow': np.arange(0.35, 0.85, 0.05)
}
config = ps.ensemble.parameter_product(param_options)

# initialize and start runs
e = ps.Ensemble(summa_exe, file_manager,
                config, num_workers=len(config))
e.start('local', arg_list=['export OMP_NUM_THREADS=1',
                           'export LD_LIBRARY_PATH=/opt/local/lib'])
e.monitor()

# check status
for i, (n, s) in enumerate(e.simulations.items()):
    assert s._status == 'Success'

# open output
ds = e.merge_output()

Parameters:

• executable (string) – Path to locally compiled SUMMA executable or the name of the docker image to run
• filemanager (string) – Path to the file manager for the desired simulation setup. Can be specified as a relative path

executable = None

merge_output()

Merge the output of the simulatino objects into a single xarray dataset

monitor()

Watch a running ensemble until it is done and collect the run information in the list of simulation objects. This will halt execution of a started simulation through the Ensemble.start() method if monitor was not set there.

simulations = {}

start(run_option: str, arg_list: list = [], monitor: bool = False)

Start running the ensemble. By default does not halt execution of further python commands, and simply launches the processes in the background. Progress can be halted by setting the monitor keyword.

Parameters:

• run option (string) – The method to use for running SUMMA. Can be either local or docker.
• arg_list (List[string]) – A list of commands to be run before the simulations are started.
• monitor (bool) – Whether to halt execution until completion of all of the simulations. Defaults to False

submissions = []

pysumma.ensemble.decision_product(list_config)

Create an Ensemble compatible configuration dictionary which contains the Cartesian product of the options given. For example:

decisions_options = {
    'veg_traits': ['Raupach_BLM1994', 'CM_QJRMS1988'],
    'cIntercept': ['sparseCanopy', 'storageFunc'],
}

ps.ensemble.decision_product(decisions_options)

will produce 4 runs in a configuration that looks like:

{'++Raupach_BLM1994++sparseCanopy++':
    {'decisions':
        {'veg_traits': 'Raupach_BLM1994',
         'cIntercept': 'sparseCanopy'}},
 '++Raupach_BLM1994++storageFunc++':
     {'decisions':
         {'veg_traits': 'Raupach_BLM1994',
          'cIntercept': 'storageFunc'}},
 '++CM_QJRMS1988++sparseCanopy++':
     {'decisions':
         {'veg_traits': 'CM_QJRMS1988',
          'cIntercept': 'sparseCanopy'}},
 '++CM_QJRMS1988++storageFunc++':
     {'decisions':
         {'veg_traits': 'CM_QJRMS1988',
          'cIntercept': 'storageFunc'}}}

pysumma.ensemble.parameter_product(list_config)

Create an Ensemble compatible configuration dictionary which contains the Cartesian product of the options given. For example:

param_options = {
    'leafExchangeCoeff': np.arange(0.025, 0.0354, 0.01),
    'albedoMax': np.arange(0.9, 0.96, 0.05)
}

ps.ensemble.parameter_product(param_options)

will produce 4 runs in a configuration that looks like:

{'++leafExchangeCoeff=0.025++albedoMax=0.9++':
    {'parameters':
        {'leafExchangeCoeff': 0.025,
         'albedoMax': 0.9}},
 '++leafExchangeCoeff=0.025++albedoMax=0.95++':
     {'parameters':
         {'leafExchangeCoeff': 0.025,
          'albedoMax': 0.95}},
 '++leafExchangeCoeff=0.035++albedoMax=0.9++':
     {'parameters':
         {'leafExchangeCoeff': 0.035,
          'albedoMax': 0.9}},
 '++leafExchangeCoeff=0.035++albedoMax=0.95++':
     {'parameters': {'leafExchangeCoeff': 0.035,
      'albedoMax': 0.95}}}

pysumma.ensemble.total_product(dec_conf=None, param_conf=None)

Create an Ensemble compatible configuration dictionary which contains the Cartesian product of the options given. For example:

param_options = {
    'leafExchangeCoeff': np.arange(0.025, 0.0354, 0.01),
}

decisions_options = {
    'veg_traits': ['Raupach_BLM1994', 'CM_QJRMS1988'],
}
ps.ensemble.total_product(decisions_options, param_options)

will produce 4 runs in a configuration that looks like:

{'++Raupach_BLM1994++leafExchangeCoeff=0.025++':
    {'decisions':
        {'veg_traits': 'Raupach_BLM1994'},
     'parameters':
         {'leafExchangeCoeff': 0.025}},
 '++Raupach_BLM1994++leafExchangeCoeff=0.035++':
     {'decisions':
         {'veg_traits': 'Raupach_BLM1994'},
      'parameters':
          {'leafExchangeCoeff': 0.035}},
 '++CM_QJRMS1988++leafExchangeCoeff=0.025++':
     {'decisions':
         {'veg_traits': 'CM_QJRMS1988'},
      'parameters':
          {'leafExchangeCoeff': 0.025}},
 '++CM_QJRMS1988++leafExchangeCoeff=0.035++':
     {'decisions':
         {'veg_traits': 'CM_QJRMS1988'},
      'parameters':
          {'leafExchangeCoeff': 0.035}}}

pysumma.file_manager module

class pysumma.file_manager.FileManager(path)

Bases: pysumma.option.OptionContainer

The FileManager object provides an interface to a SUMMA file manager file. Generally, this object is not meant to be instantiated on its own (though it can be), but rather as a member of a simulation object.

Example usage:

import pysumma as ps
s = ps.Simulation('summa.exe', 'file_manager.txt')
print(s.manager)

Parameters:path (string) – The path to the file manager file.

basin_param_info

decisions

force_file_list

local_attributes

local_param_info

output_control

parameter_trial

set_option(key, value)

Change the value of an option

class pysumma.file_manager.FileManagerOption(name, value)

Bases: pysumma.option.BaseOption

Container for lines in a file manager file

set_value(new_value)

pysumma.force_file_list module

class pysumma.force_file_list.ForceFileList(file_list_path, force_file_prefix_path)

Bases: pysumma.option.OptionContainer

forcing_list

prefix = ''

set_option(key, value)

This has to be implemented by subclasses

class pysumma.force_file_list.ForceFileListOption(name)

Bases: pysumma.option.BaseOption

set_value(new_value)

pysumma.gui module

pysumma.hydroshare_utils module

pysumma.local_param_info module

class pysumma.local_param_info.LocalParamInfo(path)

Bases: pysumma.option.OptionContainer

fmt_strings = ["'(a25,1x,3(a1,1x,f12.4,1x))'", "'(a25,1x,a1,1x,3(f12.4,1x,a1,1x))'"]

read(path)

Read the configuration and populate the options

set_option(key, value)

This has to be implemented by subclasses

class pysumma.local_param_info.LocalParamOption(name, default, low, high)

Bases: pysumma.option.BaseOption

set_value(new_value)

pysumma.option module

class pysumma.option.BaseOption(name, value=None)

Bases: object

Base implementation for representing options in the various text based SUMMA configuration files. This class is meant to be extended rather than used directly.

class pysumma.option.OptionContainer(path, optiontype)

Bases: object

Base implementation for representing text based configuration files for SUMMA. This class is meant to be extended rather than used directly.

clear()

Clear all options

get_option(name, strict=False)

Retrieve an option

Parameters:

• name ((string)) – The name of the option to search for
• strict ((bool, optional)) – Whether to raise an error if the option is not found. Defaults to False, which does not raise exceptions.

get_value(name, strict=False)

Retrieve the value of a given option

Parameters:

• name ((string)) – The name of the option whose value to search for
• strict ((bool, optional)) – Whether to raise an error if the option is not found. Defaults to False, which does not raise exceptions.

read(path)

Read the configuration and populate the options

remove_option(name, strict=False)

Remove an option

Parameters:

• name ((string)) – The name of the option whose value to remove
• strict ((bool, optional)) – Whether to raise an error if the option is not found. Defaults to False, which does not raise exceptions.

set_option()

This has to be implemented by subclasses

validate()

Ensure no options are repeated

write(path=None)

Write the configuration based on the string representation of the OptionType of the instance.

Parameters:path ((string, optional)) – Where to write the file. If not provided the original file will be overwritten.

pysumma.output_control module

class pysumma.output_control.OutputControl(path)

Bases: pysumma.option.OptionContainer

The OutputControl object manages what output SUMMA will write out. Each output variable is stored in the options list as an OutputControlOption. These options are automatically populated on instantiation, and can be added or modified through the set_option method.

set_option(name=None, period=None, sum=0, instant=1, mean=0, variance=0, min=0, max=0, mode=0)

Change or create a new entry in the output control

class pysumma.output_control.OutputControlOption(var=None, period=None, sum=0, instant=1, mean=0, variance=0, min=0, max=0, mode=0)

Bases: pysumma.option.BaseOption

get_print_list()

statistic

This could be improved

validate()

pysumma.output_control.read_master_file(master_file_filepath)

Get all varialbes from var_lookup file

pysumma.preprocess_csv module

pysumma.simulation module

class pysumma.simulation.Simulation(executable, filemanager)

Bases: object

The Simulation object provides a wrapper for SUMMA simulations. It can be used to set up, modify, and run SUMMA. The Simulation object consists of information about how to run SUMMA (the location of the executable, various command line flags and options) as well as configuration details which describe the domain to be simulated.

A standard workflow for running a simulation locally is:

import pysumma as ps
s = ps.Simulation('summa.exe', 'file_manager.txt')
s.start('local')
s.monitor()
assert s._status == 'Success'
print(s.output)

A standard workflow for running a simulation through docker is:

import pysumma as ps
s = ps.Simulation('summa:develop', 'file_manager.txt')
s.start('docker')
s.monitor()
assert s._status == 'Success'
print(s.output)

Parameters:

• executable – Path to locally compiled SUMMA executable or the name of the docker image to run
• filemanager – Path to the file manager for the desired simulation setup. Can be specified as a relative path.

basin_param_info = None

decisions = None

execute(run_option, run_suffix=None, preprocess_cmds=[], monitor=False)

Run a SUMMA simulation

force_file_list = None

library_path = None

local_attributes = None

local_param_info = None

manager = None

monitor()

Watch a running simulation until it is done and collect the run information in the simulation object. This will halt execution of a started simulation through the Simulation.start() method.

output

An xarray dataset, or list of xarray datasets representing the output that a simulation has written to disk.

output_control = None

parameter_trial = None

process = None

result

Attribute describing whether a model run was a success or not

start(run_option, run_suffix='pysumma_run', processes=1, prerun_cmds=None, startGRU=None, countGRU=None, iHRU=None, freq_restart=None, progress=None)

Start a SUMMA simulation. By default does not halt execution of further python commands, and simply launches the SUMMA process in the background. Progress can be halted by using the Simulation.monitor() method.

Parameters:

• run_option (string) – The method to use for running SUMMA, can be either local or docker.
• run_suffix (string) – A unique identifier to include as part of the output file names
• processes (string) – The number of processors to use. Note: This only matters if the SUMMA version being run is compiled with OpenMP support. Defaults to 1.
• prerun_cmds (List[string]) – A list of commands to be run before the SUMMA executable is invoked. This can be used to set things like library paths or creation of subdirectories.
• startGRU (int) – The GRU index to start the simulation at. Used to run only specified sections of the domain. Must be co-specified with countGRU to do anything. Cannot be used with iHRU.
• countGRU (int) – Number of GRU to run, starting at startGRU. Used to run only the specified portion of the domain. Must be co- specified with startGRU to do anything. Cannot be used with iHRU.
• iHRU (int) – Index of a single HRU to be run. Cannot be used with startGRU and countGRU.
• freq_restart (string) – How often to write restart files. Can be d for daily, m for monthly, and y for yearly.
• progress (string) – How often to record progress. Can be d for daily, m for monthly, and y for yearly.

stderr

The standard error. This contains a string representation of the output that a SUMMA simulation would write to screen if any unforseen errors occurred. Use print to view this so that line breaks are rendered correctly.

stdout

The standard output. This contains a string representation of the output that a SUMMA simulation would write to screen. Use print to view this so that line breaks are rendered correctly.

pysumma.utils module

class pysumma.utils.ChainDict

Bases: dict

pysumma.utils.install_test_cases_summa_web(save_filepath)

Download SUMMA TestCases from UCAR web site. TODO: however this is old version of SUMMA TestCases

pysumma.utils.product_dict(**kwargs)

pysumma.validation module

Module contents