Measurement

major modules

mwr_raw2l1.measurement.measurement

class mwr_raw2l1.measurement.measurement.Measurement(data, conf_inst)[source]

Bases: MeasurementConstructors

apply_quality_control(conf_qc)[source]

set quality_flag and quality_flag_status and qc_thresholds according to quality control

run(conf_qc)[source]

main method of the class completing dimensions and variables and applying quality flags

Parameters:

conf_qc – configuration dictionary of the quality control. For defaults use mwr_raw2l1/config/qc_config.yaml

set_coords(delta_lat=0.01, delta_lon=0.02, delta_altitude=10, **kwargs)[source]

(re)set geographical coordinates (lat, lon, altitude) in self.data accounting for self.conf_inst

If both are available the method checks consistency between coordinates in datafile and configuration. For self.conf_inst to be usable it must contain keys station_latitude, station_longitude and station_altitude. If self.conf_inst is None, variables are taken from datafile without any checks and an error is raised if any variable is missing in data.

Parameters:

  • delta_lat (optional) – maximum allowed difference between latitude in config and in datafile in degrees. Defaults to 0.01.

  • delta_lon (optional) – maximum allowed difference between longitude in config and in datafile in degrees. Defaults to 0.02.

  • delta_altitude (optional) – maximum allowed difference between altitude in config and in datafile in meters. Defaults to 10.

  • **kwargs – keyword arguments passed on to set_vars()

set_inst_params()[source]

set instrument dependent parameters which are not dimensions (must be set before)

set_receivers()[source]

set receiver dimension and receiver-specific variables

set_time_bnds()[source]

set time bounds from spacing of time vector and scanflag

set_vars(varname_data_conf, delta_data_conf, dim='time', primary_src='data', accept_unset=False)[source]

(re)set variable in self.data from datafile input and instrument configuration file

If both are available the method checks consistency between coordinates in datafile and configuration. If self.conf_inst is None, variables are taken from datafile without any checks and an error is raised if any variable is missing in data.

Parameters:

  • varname_data_conf – dictionary for matching between variable names in data (keys) and config (values)

  • delta_data_conf – dictionary of maximum difference allowed between value in data and config (values) for each variable. Keys are the variable names in data.

  • dim (optional) – dimension of the variables to set. Set to None for dimensionless vars. Defaults to ‘time’

  • primary_src (optional {'data', 'config', 'conf'}) – specifies which source takes precedence. Default: ‘data’

  • accept_unset (optional) – accept unset variables if neither present in data nor conf_inst. Defaults to False

set_wavelength(delta=1, **kwargs)[source]

(re)set wavelength of infrared radiometer in self.data accounting for self.conf_inst. Also add as dimension.

If conf and data are available the method checks consistency between wavelength in datafile and configuration. For self.conf_inst to be usable it must contain key ir_wavelength. If self.conf_inst is None, variables are taken from datafile without any checks and an error is raised if any variable is missing in data.

Parameters:

  • delta (optional) – maximum allowed difference between IR wavelength in config and in datafile in nm. Defaults to 1.

  • **kwargs – keyword arguments passed on to set_vars()

mwr_raw2l1.measurement.measurement_constructors

class mwr_raw2l1.measurement.measurement_constructors.MeasurementConstructors(data, conf_inst)[source]

Bases: object

classmethod from_attex(readin_data, conf_inst=None)[source]

constructor for data read in from Attex instruments.

Parameters:

  • readin_data – instance or list of instances for the Attex read-in class

  • conf_inst (optional) – dictionary of instrument configuration. Set to None if no config is needed. Defaults to None.

Returns:

instance of the Measurement class with observations filled to self.data and config to self.conf_inst

classmethod from_radiometrics(readin_data, conf_inst=None)[source]

constructor for data read in from Radiometrics instruments.

Auxiliary data are merged to time grid of MWR data.

Parameters:

  • readin_data – instance or list of instances for the Radiometrics read-in class

  • conf_inst (optional) – dictionary of instrument configuration. Set to None if no config is needed. Defaults to None.

Returns:

instance of the Measurement class with observations filled to self.data and config to self.conf_inst

classmethod from_rpg(readin_data, conf_inst=None)[source]

constructor for data read in from RPG instruments.

Auxiliary data are merged to time grid of MWR data. Scanning MWR data are returned as time series (no ele dim)

Parameters:

  • readin_data – dictionary with (list of) instance(s) for each RPG read-in class (keys correspond to filetype)

  • conf_inst (optional) – dictionary of instrument configuration. Set to None if no config is needed. Defaults to None.

Returns:

instance of the Measurement class with observations filled to self.data and config to self.conf_inst

mwr_raw2l1.measurement.scan_transform

class mwr_raw2l1.measurement.measurement_constructors.MeasurementConstructors(data, conf_inst)[source]

Bases: object

classmethod from_attex(readin_data, conf_inst=None)[source]

constructor for data read in from Attex instruments.

Parameters:

  • readin_data – instance or list of instances for the Attex read-in class

  • conf_inst (optional) – dictionary of instrument configuration. Set to None if no config is needed. Defaults to None.

Returns:

instance of the Measurement class with observations filled to self.data and config to self.conf_inst

classmethod from_radiometrics(readin_data, conf_inst=None)[source]

constructor for data read in from Radiometrics instruments.

Auxiliary data are merged to time grid of MWR data.

Parameters:

  • readin_data – instance or list of instances for the Radiometrics read-in class

  • conf_inst (optional) – dictionary of instrument configuration. Set to None if no config is needed. Defaults to None.

Returns:

instance of the Measurement class with observations filled to self.data and config to self.conf_inst

classmethod from_rpg(readin_data, conf_inst=None)[source]

constructor for data read in from RPG instruments.

Auxiliary data are merged to time grid of MWR data. Scanning MWR data are returned as time series (no ele dim)

Parameters:

  • readin_data – dictionary with (list of) instance(s) for each RPG read-in class (keys correspond to filetype)

  • conf_inst (optional) – dictionary of instrument configuration. Set to None if no config is needed. Defaults to None.

Returns:

instance of the Measurement class with observations filled to self.data and config to self.conf_inst

helper modules

mwr_raw2l1.measurement.measurement_helpers

mwr_raw2l1.measurement.measurement_helpers.channels2quantity(freq, name_humidity='hum', name_temperature='temp', **kwargs)[source]

attribute retrieved quantities to receivers matching frequency channels according to channels2receiver()

Currently, the only quantities that can be discriminated are temperature and humidity

Parameters:

  • freq – frequency vector in GHz as xarray.DataArray, numpy.ndarray, list or tuple

  • name_humidity (optional) – name assigned to retrieved quantity ‘humidity’

  • name_temperature (optional) – name assigned to retrieved quantity ‘temperatre’

  • **kwargs – keyword arguments passed on to channels2receiver()

Returns:

a dictionary with the receiver numbers as keys and the retrieved quantities as values

mwr_raw2l1.measurement.measurement_helpers.channels2receiver(freq, band_limits=None, base=1)[source]

attribute receiver numbers (1, 2, …) to frequency channels according to frequency bands

Parameters:

  • freq – frequency vector in GHz

  • band_limits (optional) – limit frequencies (in GHz) between different receivers. Need max(band_limits)<max(freq)

  • base (optional) – indexing base for attributing receiver numbers. base=1 returns numbers 1, 2, …; base=0 returns 0, 1, …; Defaults to 1.

mwr_raw2l1.measurement.measurement_helpers.get_receiver_vars(varnames, search_pattern='receiver', sep='_')[source]

find receiver-specific variable names

Assumes last part separated by ‘sep’ must be an integer and second-last part must match ‘search_pattern’

Parameters:

  • varnames – iterable of strings representing variable names

  • search_pattern – string characterising a receiver-specific variable in second-last block

  • sep – separator between filename parts

Returns:

a dictionary where the keys correspond to the base variable name and the values contain a list of all receiver- specific variables contributing to this base variable name

mwr_raw2l1.measurement.measurement_helpers.is_full_var_in_data(data, var)[source]

Return False if ‘var’ is not in data or if any value of data[‘var’] is NaN, otherwise True

mwr_raw2l1.measurement.measurement_helpers.is_var_in_data(data, var)[source]

Return False if ‘var’ is not in data or if data[‘var’] is all NaN, otherwise True

mwr_raw2l1.measurement.measurement_helpers.split_receiver(varname, sep='_')[source]

split off receiver part (i.e. part after second-last sep) from varname

mwr_raw2l1.measurement.measurement_qc_helpers

mwr_raw2l1.measurement.measurement_qc_helpers.check_rain(data)[source]

check the receiver’s rain flag in data

Parameters:

data – dataset, commonly Measurement.data

Returns:

mask_fail: None if check_applied=False, otherwise array of size (time,) set True where rain is detected check_applied (bool): True if check has been applied, False if check could not be applied (not enough info)

Return type:

tuple containing

mwr_raw2l1.measurement.measurement_qc_helpers.check_receiver_sanity(data, channel)[source]

check the receiver sanity flag(s) in data

Parameters:

  • data – dataset, commonly Measurement.data

  • channel – channel index for which to check the sanity flag (only used for RPG)

Returns:

mask_fail: None if check_applied=False, otherwise array of size (time,) set True where quality is bad check_applied (bool): True if check has been applied, False if check could not be applied (not enough info)

Return type:

tuple containing

mwr_raw2l1.measurement.measurement_qc_helpers.check_sun(data, delta_ele, delta_azi)[source]

check if sun is in beam within the tolerances ‘delta_ele’, ‘delta_azi’ ( abs(data[‘ele’] - ele_sun) < delta_ele )

Parameters:

  • data – xarray.Dataset, commonly Measurement.data

  • delta_ele – elevation offset in degrees from radiometer pointing until which sun is considered to be in the beam

  • delta_azi – azimuth offset in degrees from radiometer pointing until which sun is considered to be in the beam

Returns:

mask_fail: None if check_applied=False, otherwise array of size (time,) set True where sun is in beam check_applied: array True if check was applied, False if check could not be applied (no ele/azi in data)

Return type:

tuple containing

mwr_raw2l1.measurement.measurement_qc_helpers.flag_check(data, varname, value, channel=None)[source]

check if flag with ‘varname’ in ‘data’ matches ‘value’

Parameters:

  • data – dataset, commonly Measurement.data

  • varname – name of the variable in ‘data’

  • value – value that the flag must match so that this function returns True

  • channel (optional) – channel index for which to check the flag variables of size (time, channels). If set to None a variable of size (time,) the sanity flag (only used for RPG). Defaults to None

Returns:

mask_fail: None if check_applied=False, otherwise array of size (time,) set True where value is ‘matched’ check_applied (bool): True if check has been applied, False if check could not be applied (not enough info)

Return type:

tuple containing

mwr_raw2l1.measurement.measurement_qc_helpers.orbit_position(data, body='sun')[source]

calculate orbit position of sun or moon for instrument position at each time in ‘data’ using ephem

Parameters:

  • data – xarray.Dataset, commonly Measurement.data

  • body (optional) – name of astronomical body to calculate orbit from (‘sun’ or ‘moon’). Defaults to ‘sun’

Returns:

ele: numpy.ndarray of elevations of the body for each time step azi: numpy.ndarray of azimuths of the body for each time step

Return type:

tuple containing

mwr_raw2l1.measurement.measurement_qc_helpers.orbit_position_interp(data, delta_t=600, **kwargs)[source]

wrapper to orbit_position() doing orbit calculations only each ‘delta_t’ seconds and interpolate to time

mwr_raw2l1.measurement.measurement_construct_helpers

mwr_raw2l1.measurement.measurement_construct_helpers.attex_to_datasets(data_all, dims, vars, vars_opt)[source]

generate unique xarray.Dataset for each type of obs in ‘data’ using dimensions and variables specified

Parameters:

  • data_all – single instance of the read-in class (with observations in instance variable ‘data’) or as a list containing a series of instances of read-in classes.

  • dims – list of keys that are a dimension (must correspond to the order of dimensions in data)

  • vars – list of keys that are data variables (dimensions don’t need to be specified again)

  • vars_opt – list of keys that are optional data variables

Returns:

a xarray.Dataset containing the data

mwr_raw2l1.measurement.measurement_construct_helpers.check_temporal_consistency(all_data, tolerance=15)[source]

check that data of all file types fall into the time covered by HKD for RPG observations

Parameters:

  • all_data – dictionary with a xarray.Dataset attached to each key (output of rpg_to_datasets())

  • tolerance – time interval in seconds up to which other data sources are allowed to extend beyond HKD times

mwr_raw2l1.measurement.measurement_construct_helpers.drop_duplicates(ds, dim)[source]

drop duplicates from all data in ds for duplicates in dimension vector

Parameters:
Returns:

ds with unique dimension vector

mwr_raw2l1.measurement.measurement_construct_helpers.make_dataset(data, dims, vars, vars_opt=None, multidim_vars=None, time_vector=None)[source]

generate a xarray.Dataset from ‘data’ dictionary using the dimensions and variables specified

Parameters:

  • data – dictionary containing the data. If set to None or empty a placeholder dataset with all-NaN time series (except variable IRT, which is 2d) is returned. If set to None or empty time_vector must be specified.

  • dims – list of keys that are a dimension (must correspond to the order of dimensions in data)

  • vars – list of keys that are data variables (dimensions don’t need to be specified again)

  • vars_opt (optional) – list of keys that are optional data variables (added as 1-d series of NaN if not in ‘data’)

  • multidim_vars (optional) – dictionary of variables with more than time dimension. Variable name as key, number of dimensions as values. This argument will be ignored as long as the variable is present in dataset

  • time_vector (optional) – numpy.ndarray of numpy.datetime64 to take as time dimension for generating all-NaN datasets. This argument will be ignored as long as data is not None or empty

Returns:

xarray.Dataset

mwr_raw2l1.measurement.measurement_construct_helpers.merge_aux_data(mwr_data, all_data, srcs_to_ignore=None)[source]

merge auxiliary data to time grid of microwave data

Parameters:

  • mwr_data – xarray.Dataset of microwave radiometer data

  • all_data – Dictionary of data from different sources (keys) as xarray.Dataset (values). Can also contain the data in ‘mwr_data’ in which case it must be made sure the key is specified in ‘srcs_to_ignore’

  • srcs_to_ignore (optional) – list of sources (keys) to ignore from ‘all_data’ e.g. because they are already contained in ‘mwr_data’. Defaults to [‘mwr’, ‘brt’, ‘blb’]

Returns:

merged dataset of type xarray.Dataset

mwr_raw2l1.measurement.measurement_construct_helpers.merge_brt_blb(all_data)[source]

merge brt (zenith MWR) and blb (scanning MWR) observations from an RPG instrument

Parameters:

all_data – dictionary with a xarray.Dataset attached to each key (output of rpg_to_datasets())

mwr_raw2l1.measurement.measurement_construct_helpers.radiometrics_to_datasets(data_all, dims, vars, vars_opt)[source]

generate unique xarray.Dataset for each type of obs in ‘data’ using dimensions and variables specified

Parameters:

  • data_all – single instance of the read-in class (with observations in instance variable ‘data’) or as a list containing a series of instances of read-in classes.

  • dims – list of keys that are a dimension (must correspond to the order of dimensions in data)

  • vars – list of keys that are data variables (dimensions don’t need to be specified again)

  • vars_opt – list of keys that are optional data variables

Returns:

dictionary with one xarray.Dataset for each key. It contains one item for each key in data

mwr_raw2l1.measurement.measurement_construct_helpers.rpg_to_datasets(data, dims, vars, vars_opt)[source]

generate unique xarray.Dataset for each type of obs in ‘data’ using dimensions and variables specified

Parameters:

  • data – dictionary containing the observations by type. Its keys correspond to the type of observations (e.g. brt, blb, irt …). The observations themselves can be given as a single instance of the read-in class (with observations in variable ‘data’) or as a list containing a series of instances of read-in classes.

  • dims – list of keys that are a dimension (must correspond to the order of dimensions in data)

  • vars – list of keys that are data variables (dimensions don’t need to be specified again)

  • vars_opt – list of keys that are optional data variables (added as 1-dim series of NaN if missing in ‘data’)

Returns:

dictionary with one xarray.Dataset for each key. It contains one item for each key in data

mwr_raw2l1.measurement.measurement_construct_helpers.rpg_to_si(all_data)[source]

transform non-SI units to SI units for RPG datasets (e.g. km/h to m/s for windspeed

Parameters:

all_data – dictionary with a xarray.Dataset attached to each key (output of rpg_to_datasets())

Returns:

all_data with but with values corresponding to SI units

mwr_raw2l1.measurement.measurement_construct_helpers.to_single_dataset(data_dicts, *args, **kwargs)[source]

return a single xarray.Dataset with unique time vector from a list of data dictionaries

Parameters:

  • data_dicts – list of data dictionaries to be concatenated to a time series

  • *args – dimension and variable specifications passed on to make_dataset()

  • **kwargs – dimension and variable specifications passed on to make_dataset()