A better high-level API for building models from components

[brian-rose]

The original vision for climlab was an environment in which the user is free to build complete models from components simply and easily.

I think that as the complexity of the components available in climlab has grown, we have actually moved farther away from that goal. Consider the currently supported way to construct a single-column radiative-convective model with interactive water vapor (e.g. as found in the documentation for the Emanuel convection scheme):

import climlab
import numpy as np
from climlab import constants as const
# Temperatures in a single column
full_state = climlab.column_state(num_lev=30, water_depth=2.5)
temperature_state = {'Tatm':full_state.Tatm,'Ts':full_state.Ts}
#  Initialize a nearly dry column (small background stratospheric humidity)
q = np.ones_like(full_state.Tatm) * 5.E-6
#  Add specific_humidity to the state dictionary
full_state['q'] = q
#  The top-level model
model = climlab.TimeDependentProcess(state=full_state,
                              timestep=const.seconds_per_hour)
#  Radiation coupled to water vapor
rad = climlab.radiation.RRTMG(state=temperature_state,
                              specific_humidity=full_state.q,
                              albedo=0.3,
                              timestep=const.seconds_per_day
                              )
#  Convection scheme -- water vapor is a state variable
conv = climlab.convection.EmanuelConvection(state=full_state,
                              timestep=const.seconds_per_hour)
#  Surface heat flux processes
shf = climlab.surface.SensibleHeatFlux(state=temperature_state, Cd=0.5E-3,
                              timestep=const.seconds_per_hour)
lhf = climlab.surface.LatentHeatFlux(state=full_state, Cd=0.5E-3,
                              timestep=const.seconds_per_hour)
#  Couple all the submodels together
model.add_subprocess('Radiation', rad)
model.add_subprocess('Convection', conv)
model.add_subprocess('SHF', shf)
model.add_subprocess('LHF', lhf)

This is ridiculously verbose, and requires the user to think carefully about exactly which subset of the state variable dictionary is needed for each component. We also had to explicitly create a parent Process called model whose only job is to be a container for the other components.

I'm envisioning a new API that would automate a lot of the coupling of components into subprocesses of a new parent process. It would be great to be able to explicitly __add__ subprocesses like this:

import climlab
config = {*set grid dimensions, a complete state dict, and some important parameters for various components in this dict*}
rad = climlab.radiation.RRTMG(**config)
conv = climlab.convection.EmanuelConvection(**config)
shf = climlab.surface.SensibleHeatFlux(**config)
lhf = climlab.surface.LatentHeatFlux(**config)
model = rad + conv + shf + lhf 

where the model that is returned is:

• the same as created above, i.e. a plain container that does not generate any tendencies of its own
• intelligently coupled, meaning in this case that the specific_humidity input field for rad is set appropriately.

[brian-rose]

As of brian-rose@74c9cd5 it is now possible to __add__ processes together as described above. I think this is an immediate improvement to the user interface, and it wasn't complicated to implement.

The new API is used test_cam3rad.py.

There is not much in the way of "smart" coupling just yet, but it's a start.

To make this work in a sensible way, I have added an optional name attribute to the Process class. This makes a lot of sense and should have been there from the beginning.

But now if every Process instance has a name, the subprocess dictionary is kind of redundant. A plain list would work fine.

[brian-rose]

Some sensible behavior when the user invokes

model = proc1 + proc2

• Coupling should automatically take account of matches between named input and diagnostic or state fields:

if ('thisfield' in proc1.input) and ('thisfield' in proc2.diagnostics):
    proc1.thisfield = proc2.thisfield

• Any named input fields for proc1 and proc2 that don't match a diagnostic or state become named input for model.
• Diagnostic fields are already propagated up the tree automatically so we don't need to do anything with those.
• model.timestep is set to the smallest of proc1.timestep and proc2.timestep
• There should be text output to describe the coupling (suppressed if verbose=False is set in both proc1 and proc2)
• Any name collisions (e.g. multiple diagnostics with same name) should raise a warning.

[brian-rose]

On further reflection, I'm not sure about my first point above. It implies that proc1 is actually modified by the __add__ operation.

A more conservative version would be to just declare as input for model any input fields in proc1 or proc2 that are not matched by a diagnostic or state field of the same name.