pyclmuapp package
Submodules
pyclmuapp.clmu module
- class pyclmuapp.clmu.cesm_run(CASESCRIPT_local, CASEROOT_local, DOUT_S_ROOT, caseconfig: str | dict | DataFrame | Series)
Bases:
object
- This class is used to create CESM2/CTSM case. Run the CESM2/CTSM case should be done by running the generated shell script
using terminal or python subprocess or container (docker or singularity with CESM2/CTSM installed)
- Parameters:
CASESCRIPT_local (str) – The path of the case script in the local machine.
CASEROOT_local (str) – The path of the case root in the local machine.
DOUT_S_ROOT (str) – The path of the output root in the local machine.
caseconfig (Union[str, dict, pd.DataFrame, pd.Series]) – The configuration of the case to be built.
- CASESCRIPT
The path of the case script.
- Type:
str
- CASEROOT
The path of the case root.
- Type:
str
- DOUT_S_ROOT
The path of the output root.
- Type:
str
- case_lat
The latitude of the case.
- Type:
float
- case_lon
The longitude of the case.
- Type:
float
- case_name
The name of the case.
- Type:
str
- json_file_path
The path of the JSON file.
- Type:
str
- config
The configuration data.
- Type:
dict
Methods
create_case
(scriptpath)Create the case scripts for single point modeling.
modify_case_config
(scriptpaht)Modify the case config file.
modify_forcing
(var, action, forcing_location)Modify the forcing file.
modify_surf
(var, action[, numurbl])Modify the surface data file.
nc_view
([ds])View the netcdf file.
Read the JSON config file.
recover_forcing
(forcing_location, ...)Recover the forcing file.
reset_case
([password])Reset the case.
- create_case(scriptpath) str
Create the case scripts for single point modeling.
- Parameters:
filepath (str) – File path of the config file.
- Returns:
The modified script file.
- Return type:
script (str)
- modify_case_config(scriptpaht) str
Modify the case config file.
- Parameters:
scriptpath (str) – The path of the script file.
- Returns:
The modified script file.
- Return type:
script (str)
- modify_forcing(var, action, forcing_location) None
Modify the forcing file.
- Parameters:
var (str) – The variable to be modified.
action (float, np.ndarray, dict) – The action to be taken.
forcing_location (str) – The location of the forcing file. - if action is a dict, the key is the variable name, the value is the action. - if action is a float, the action will be added to the variable. - if action is a np.ndarray, the variable will be replaced by the action.
- Returns:
None
- modify_surf(var, action, numurbl=None) str
Modify the surface data file.
- Parameters:
var (str) – The variable to be modified.
action (float, np.ndarray, dict) – The action to be taken. - if action is a dict, the key is the variable name, the value is the action. - if action is a float, the action will be added to the variable. - if action is a np.ndarray, the variable will be replaced by the action.
numurbl (int, optional) – The number of urban land units. Defaults to None. None means the action will be implemented to all the urban land units. numurbl 0 –> TBD urban, numurbl 1 –> HD urban, numurbl 2 –> MD urban
- Returns:
The modified surface data file path.
- Return type:
str
- nc_view(ds: str = 'None') Dataset
View the netcdf file.
- Parameters:
xarray.Dataset – The xarray dataset.
ds (str, optional) – The path of the netcdf file. Defaults to “None”.
- Returns:
The xarray dataset.
- Return type:
xarray.Dataset
- read_json_config() dict
Read the JSON config file.
- Returns:
The configuration data.
- Return type:
config (dict)
- recover_forcing(forcing_location, backup_location) None
Recover the forcing file.
- Parameters:
forcing_location (str) – The location of the forcing file.
backup_location (str) – The location of the backup file.
- Returns:
None
- reset_case(password=None) CompletedProcess
Reset the case. (delete the case folders and files)
- Parameters:
password (str, optional) – The password of the server. Defaults to None.
- Returns:
The result of the command.
- Return type:
subprocess.CompletedProcess
- pyclmuapp.clmu.copy_file_if_not_exists(source_path, destination_path)
- pyclmuapp.clmu.copy_file_if_not_exists2(source_path, destination_path, lon, lat, res='1.25*0.9') None
Copy the file if the destination file does not exist.
- Parameters:
source_path (str) – The source file path.
destination_path (str) – The destination file path.
lon (str) – The longitude of the point.
lat (str) – The latitude of the point.
res (str) – The resolution of the file. Defaults to “1.25*0.9”.
- pyclmuapp.clmu.get_forcing(start_year, end_year, start_month, end_month, lat, lon, zbot, source='cds')
get the forcing data from the era5 dataset
- Parameters:
start_year (_type_) – the start year
end_year (_type_) – the end year
start_month (_type_) – the start month
end_month (_type_) – the end month
lat (_type_) – latitude of interest point
lon (_type_) – longitude of interest point
zbot (_type_) – the bottom level height
source (_type_) – the source of the data, cds or arco-era5
- Returns:
the forcing dataset
- Return type:
_type_
- pyclmuapp.clmu.get_soil_params(ds: Dataset | DataArray | str, lat: float = 51.508965, lon: float = -0.118092) tuple
Get the soil parameters.
- Parameters:
ds (_type_) – the soil dataset
lat (_type_) – latitude of interest point
lon (_type_) – longitude of interest point
- Returns:
sand and clay content from the soil dataset
- Return type:
tuple
- pyclmuapp.clmu.get_urban_params(urban_ds: Dataset | str, soil_ds: Dataset | str, lat: float, lon: float, template: Dataset | str = '/home/runner/work/pyclmuapp/pyclmuapp/pyclmuapp/usp/surfdata.nc', PTC_URBAN: list = [0, 0, 100], outputname: str = 'surfdata.nc') Dataset
Get the urban parameters.
- Parameters:
urban_ds (_type_) – the urban dataset
soil_ds (_type_) – the soil dataset
template (_type_) – the template dataset
lat (_type_) – latitude of interest point
lon (_type_) – longitude of interest point
PTC_URBAN (list, optional) – The percentage of urban. Defaults to [0,0,100]. 0. TBD urban, 1. HD urban, 2. MD urban
outputname (_type_, optional) – the output file name. Defaults to “surfdata.nc”.
- Returns:
the modified template dataset
- Return type:
_type_
- pyclmuapp.clmu.getconfig(caseconfig: str | dict | DataFrame | Series = 'None') dict
Read the configuration of the case to be built.
- Parameters:
caseconfig (Union[str, dict, pd.DataFrame, pd.Series]) – The configuration of the case to be built.
- Returns:
The configuration data.
- Return type:
dict
- pyclmuapp.clmu.now_time()
Get the current time
- pyclmuapp.clmu.run_command(command, password='None', logname='None', iflog=True) None
Run the command. There are two ways to run the command, with or without password. The loges will be saved in the log_ppo.txt file.
- Parameters:
command (str) – The command to be run.
password (str, optional) – The password of the server.
logname (str, optional) – The name of the log file. Defaults to “cmdlog.txt”.
iflog (bool, optional) – If log is needed. Defaults to True.
- Returns:
None
pyclmuapp.container module
- class pyclmuapp.container.clumapp(pwd='/home/runner/work/pyclmuapp/pyclmuapp/docs/workdir', input_path: str = 'inputfolder', output_path: str = 'outputfolder', log_path: str = 'logfolder', scripts_path: str = 'scriptsfolder', container_type: str = 'docker')
Bases:
object
The class for running CLMU-App.
- Parameters:
pwd (str) – The path to the working directory. The default is “wokdir”.
input_path (str) – The path to the folder for the input of CLMU-App.
output_path (str) – The path to the folder for the output of CLMU-App.
log_path (str) – The path to the folder for the log of CLMU-App.
scripts_path (str) – The path to the folder for the scripts of CLMU-App.
container_type (str) – The type of the container for CLMU-App. The supported types are: - docker: The docker container. - singularity: The singularity container.
- - input_path
The path to the folder for the input of CLMU-App.
- Type:
str
- - output_path
The path to the folder for the output of CLMU-App.
- Type:
str
- - log_path
The path to the folder for the log of CLMU-App.
- Type:
str
- - image_name
The name of the image for CLMU-App.
- Type:
str
- - container_name
The name of the container for CLMU-App.
- Type:
str
- - caseconfig
The configuration of the case to be built.
- Type:
dict
- - fsurdat_path
The path to the surface input file.
- Type:
str
- - save_name
The name of the output file.
- Type:
str
- - caseconfig
The configuration of the case to be built.
- Type:
dict
- - clmu_run
The object for running CLMU-App.
- Type:
Methods
case_clean
([case_name])Clean the case artifacts.
case_scripts
([mode])Copy the scripts to the input folder.
create_folder
(folder_path)Create folders for the scripts ,input, output and log of CLMU-App if it does not exist.
docker
([cmd, iflog, password, cmdlogfile, ...])Run the docker command.
- case_clean(case_name: str | None = None) None
Clean the case artifacts. :param case_name: The name of the case to be cleaned. :type case_name: str
- case_scripts(mode: str = 'usp') None
Copy the scripts to the input folder.
- Parameters:
mode (str) – The mode of the scripts. The supported modes are:
- create_folder(folder_path) str
Create folders for the scripts ,input, output and log of CLMU-App if it does not exist.
- Parameters:
folder_path (str) – The path to the folder to be created.
- Returns:
The path to the folder created.
- Return type:
folder_path (str)
- docker(cmd: str = 'run', iflog: bool = True, password: str = 'None', cmdlogfile: str | None = None, dockersript: str = 'docker.sh') None
Run the docker command.
- Parameters:
cmd (str) – The docker command to be run. The supported commands are: - pull: Pull the docker image. - chmod: Change the mode of a file. - run: create a docker container. - exec: Execute the docker command.
iflog (bool, optional) – If log is needed. The default is True.
password (str, optional) – The password for the sudo command. The default is “None”.
cmdlogfile (str, optional) – The name of the log file. The default is “dockercmd.log”.
dockersript (str, optional) – The name of the docker script. The default is “docker.sh”.
pyclmuapp.getcity module
- pyclmuapp.getcity.adjust_longitude(longitude)
- pyclmuapp.getcity.calculate_polygon_area(latitude_longitude_coords)
- pyclmuapp.getcity.closest(data, v)
find the nearest urban grid cell in CESM using haversine distance reference: https://stackoverflow.com/questions/41336756/find-the-closest-latitude-and-longitude
- Parameters:
- datadict
a dict of cities’ lat and lon, from get_mask_cities
- vdict
a dict of a city’s lat and lon that we are interested in, e.g., {‘lat’: 40.1164, ‘lon’: -88.2434}
- Returns:
- dict
lat and lon of the nearest grid cell in Earth System Model
- pyclmuapp.getcity.get_mask_cities(df_mask)
This is a function for getting urban mask and a list of cities’ lat and lon
- Parameters:
- maskxarray.DataArray
urban mask
- Returns:
- dict
a dict of cities’ lat and lon
- pyclmuapp.getcity.haversine_dist(a, b)
This is a function for getting the haversine distance
- Parameters:
- a: list
[lat, lon] e.g., (45.7597, 4.8422)
- b: list
[lat, lon] e.g., (48.8567, 2.3508)
- er:
Earth radius, default is 6371.0088 km
- Returns:
- haversine_dist
the haversine distance between a and b
pyclmuapp.pts module
- class pyclmuapp.pts.pts_clmu(pwd='/home/runner/work/pyclmuapp/pyclmuapp/docs/workdir', input_path: str = 'inputfolder', output_path: str = 'outputfolder', log_path: str = 'logfolder', scripts_path: str = 'scriptsfolder', container_type: str = 'docker')
Bases:
clumapp
Methods
case_clean
([case_name])Clean the case artifacts.
case_scripts
([mode])Copy the scripts to the input folder.
create_folder
(folder_path)Create folders for the scripts ,input, output and log of CLMU-App if it does not exist.
docker
([cmd, iflog, password, cmdlogfile, ...])Run the docker command.
modify_forcing
(var, action[, ...])Modify the forcing input file.
modify_surf
(var, action[, numurbl, caseconfig])Modify the surface input file.
nc_view
([ds])View the netcdf file.
run
([caseconfig, ouptname, iflog, password, ...])Run workflow for a case of CLMU-App.
- modify_forcing(var: str, action: str, forcing_location: str = 'forcing location', caseconfig: str | dict | DataFrame | Series = 'None') dict
Modify the forcing input file.
- Parameters:
var (str) – The variable to be modified.
action (str) – The action to be taken.
param_location (str) – The location of the parameter to be modified.
caseconfig (Union[str, dict, pd.DataFrame, pd.Series]) – The configuration of the case to be built.
- modify_surf(var: str, action: dict | float | ndarray, numurbl: int | None = None, caseconfig: str | dict | DataFrame | Series = 'None') dict
Modify the surface input file.
- Parameters:
var (str) – The variable to be modified.
action (Union[dict, float, np.ndarray]) – The action to be taken. - if action is a float, the action will be added to the variable. - if action is a dict, the key is the variable name, the value is the action. - if action is a np.ndarray, the variable will be replaced by the action.
numurbl (int, optional) – The number of urban land units. Defaults to None. The type of urban, 0 is TBD, 1 is HD, 2 is MD.
caseconfig (Union[str, dict, pd.DataFrame, pd.Series]) – The configuration of the case to be built.
- nc_view(ds: str = 'None')
View the netcdf file. The netcdf file should be in the DOUT_S_ROOT folder.
- Parameters:
ds (xarray.Dataset) – The xarray dataset.
- run(caseconfig: str | dict | DataFrame | Series = 'None', ouptname: str = '_clm.nc', iflog: bool = True, password: str = 'None', cmdlogfile: str = 'dockercmd.log') list
Run workflow for a case of CLMU-App.
- Parameters:
caseconfig (Union[str, dict, pd.DataFrame, pd.Series], optional) – The configuration of the case to be built. The default is “None”. “None” means the default configuration is used.
ouptname (str, optional) – The name of the output file. The default is “_clm.nc”.
iflog (bool, optional) – If log is needed. The default is True.
password (str, optional) – The password for the sudo command. The default is “None”.
cmdlogfile (str, optional) – The name of the log file. The default is “dockercmd.log”.
- Returns:
The list of the saved output files names.
- Return type:
savename_list (list[str])
pyclmuapp.usp module
- class pyclmuapp.usp.usp_clmu(pwd='/home/runner/work/pyclmuapp/pyclmuapp/docs/workdir', input_path: str = 'inputfolder', output_path: str = 'outputfolder', log_path: str = 'logfolder', scripts_path: str = 'scriptsfolder', container_type: str = 'docker')
Bases:
clumapp
Methods
case_clean
([case_name])Clean the case artifacts.
case_scripts
([mode])Copy the scripts to the input folder.
check_domain
([usr_domain, domain_name])The function to get the domain data for the urban surface parameters.
check_forcing
([usr_forcing])The function to get the forcing data for the urban surface parameters.
check_surf
([usr_surfdata, surfata_name, ...])The function to get the surface data for the urban surface parameters.
Clean the usp folder.
create_folder
(folder_path)Create folders for the scripts ,input, output and log of CLMU-App if it does not exist.
docker
([cmd, iflog, password, cmdlogfile, ...])Run the docker command.
modify_forcing
([usr_forcing, action, mode, ...])The function to revise the forcing data for the urban surface parameters.
modify_surf
([usr_surfdata, action, mode, ...])The function to revise the surface data for the urban surface parameters.
nc_view
([ds])View the netcdf file.
run
([output_prefix, case_name, ...])The function to run the CLMU-App for the urban single point.
- check_domain(usr_domain: str | None = None, domain_name: str = 'domain.nc') None
The function to get the domain data for the urban surface parameters.
- Parameters:
usr_domain (str) – The path to the user-defined domain data file. The default is None, which means using the default domain data.
domain_name (str) – The name of the domain data file. The default is “domain.nc”.
- Returns:
The dictionary of the domain data for the urban surface parameters.
- Return type:
dict
- check_forcing(usr_forcing: str | None = None) None
The function to get the forcing data for the urban surface parameters.
- Parameters:
usr_forcing (str) – The path to the user-defined forcing data file. The default is None.
- check_surf(usr_surfdata: str | None = None, surfata_name: str = 'surfdata.nc', urban_type: int = 2) dict
The function to get the surface data for the urban surface parameters.
- Parameters:
usr_surfdata (str) – The path to the user-defined surface data file. The default is None.
surfata_name (str) – The name of the surface data file. The default is “surfdata.nc”.
urban_type (int) – The type of the urban surface. The default is 2. 0 is for TBD urban, 1 is for HD urban, and 2 is for MD urban.
- Returns:
The dictionary of the surface data for the urban surface parameters.
- Return type:
dict
- clean_usp() None
Clean the usp folder.
- modify_forcing(usr_forcing: str | None = None, action: dict | None = None, mode: str = 'add', forcing_name: str = 'forcing.nc') None
The function to revise the forcing data for the urban surface parameters.
- Parameters:
usr_forcing (str) – The path to the user-defined forcing data file. The default is None.
action (dict) – The dictionary of the revised forcing data for the urban surface parameters. The default is None, which means no action.
mode (str) – The mode for the revision. The default is “add”.
forcing_name (str) – The name of the revised forcing data file. The default is “forcing.nc”.
- modify_surf(usr_surfdata: str | None = None, action: dict | None = None, mode: str = 'replace', surfata_name: str = 'surfdata.nc', urban_type: int = 2) dict
The function to revise the surface data for the urban surface parameters.
- Parameters:
usr_surfdata (str) – The path to the user-defined surface data file. The default is None.
action (dict) – The dictionary of the revised surface data for the urban surface parameters. The default is None, which means no action.
mode (str) – The mode for the revision. The default is “replace”.
surfata_name (str) – The name of the revised surface data file. The default is “surfdata.nc”.
urban_type (int) – The type of the urban surface. The default is 2. 0 is for TBD urban, 1 is for HD urban, and 2 is for MD urban.
- Returns:
The dictionary of the revised surface data for the urban surface parameters.
- Return type:
dict
- nc_view(ds: str = 'None') Dataset
View the netcdf file. The netcdf file should be in the DOUT_S_ROOT folder.
- Parameters:
ds (xarray.Dataset) – The xarray dataset.
- Returns:
The xarray dataset.
- Return type:
xarray.Dataset
- run(output_prefix: str = '_clm.nc', case_name: str = 'usp_case', RUN_STARTDATE: str = '2012-08-08', START_TOD: str = '00000', STOP_OPTION: str = 'ndays', STOP_N: str = '10', ATM_DOM: str | None = None, SURF: str | None = None, FORCING: str | None = None, RUN_TYPE: str = 'coldstart', RUN_REFCASE: str = 'None', RUN_REFDATE: str = 'None', RUN_REFTOD: str = '00000', password: str = 'None', iflog: bool = True, logfile: str | None = None, hist_type: str = 'GRID', hist_nhtfrq: int = 1, hist_mfilt: int = 1000000000, urban_hac: str = 'ON_WASTEHEAT', crun_type: str = 'usp') list
The function to run the CLMU-App for the urban single point.
- Parameters:
output_prefix (str) – The output file name. The default is “_clm.nc”. if the output_prefix is none, the output file name will not be changed.
case_name (str) – The case name. The default is “usp_case”.
RUN_STARTDATE (str) – The start date of the run. The default is “2012-08-08”.
START_TOD (str) – The start time of the day. The default is “00000”.
STOP_OPTION (str) – The stop option. The default is “ndays”.
STOP_N (str) – The number of days to run. The default is “10”.
ATM_DOM (str) – The path to the domain data file. Will use the domain data provided by the user. The default is None.
SURF (str) – The path to the surface data file. Will use the surface data provided by the user. The default is None.
FORCING (str) – The path to the forcing data file. Will use the forcing data provided by the user. The default is None.
RUN_TYPE (str) – The type of the run. The default is “coldstart”. The other option is “branch”.
RUN_REFCASE (str) – The reference case. The default is “None”. Need to be provided when the RUN_TYPE is “branch”.
RUN_REFDATE (str) – The reference date. The default is “None”. Need to be provided when the RUN_TYPE is “branch”.
RUN_REFTOD (str) – The reference time of the day. The default is “00000”. Need to be provided when the RUN_TYPE is “branch”.
password (str) – The password for the docker. The default is “None”. Need to be provided when server is needed.
iflog (bool) – The flag to log the output. The default is True.
logfile (str) – The log file name. The default is pwd+”log.log”.
urban_hac (str) – The flag to turn on the urban HAC. The default is “ON_WASTEHEAT”. valid_values=”OFF”,”ON”,”ON_WASTEHEAT”.
crun_type (str) – The type of the run. The default is “usp”.
- Returns:
The list of the output files names.
- Return type:
list