Utils

mwr_raw2l1.utils.config_utils

mwr_raw2l1.utils.config_utils.check_conf(conf, mandatory_keys, miss_description)

check for mandatory keys of conf dictionary

if key is missing raises MissingConfig(‘xxx is a mandatory key ‘ + miss_description)

mwr_raw2l1.utils.config_utils.get_conf(file)

get conf dictionary from yaml files. Don’t do any checks on contents

mwr_raw2l1.utils.config_utils.get_inst_config(file)

get configuration for each instrument and check for completeness of config file

mwr_raw2l1.utils.config_utils.get_log_config(file)

get configuration for logger and check for completeness of config file

mwr_raw2l1.utils.config_utils.get_nc_format_config(file)

get configuration for output NetCDF format and check for completeness of config file

mwr_raw2l1.utils.config_utils.get_qc_config(file)

get configuration for quality control and check for completeness of config file

mwr_raw2l1.utils.config_utils.interpret_loglevel(conf)

helper function to replace log level strings in log level of logging library

mwr_raw2l1.utils.file_utils

mwr_raw2l1.utils.file_utils.abs_file_path(*file_path)

Make a relative file_path absolute in respect to the mwr_raw2l1 project directory. Absolute paths wil not be changed

mwr_raw2l1.utils.file_utils.datestr_from_filename(filename)

return date string from filename, assuming it to be the last block (separated by _) of minimum 4 decimal digits

Accepted dates are in form ‘yyyymmddHHMM’, ‘yyyymmddHHMMSS’, ‘yyyymmdd’, ‘yymm’ etc. but not separated by -, _ or :

Parameters:

filename – filename as str. Can contain path and extension.

Returns:

string containing the date in same representation as in the filename

mwr_raw2l1.utils.file_utils.generate_output_filename(basename, timestamp_src, files_in=None, time=None, ext='nc')

generate filename in form {basename}{timestamp}.{ext} where timestamp comes from input files or time vector

Parameters:

• basename – the first part of the filename without the date

• timestamp_src – source of output file timestamp. Can be ‘instamp_min’/’instamp_max’ for using smallest/largest timestamp of input filenames (needs ‘files_in) or ‘time_min’/’time_max’ for smallest/largest time in data in format yyyymmddHHMM (needs ‘time’).

• files_in – list of input filenames to processing as a basis for timestamp selection

• time – xarray.DataArray time vector of the data in numpy.datetime64 format. Assume to be sorted

• ext (optional) – filename extension. Defaults to ‘nc’. Empty not permitted.

mwr_raw2l1.utils.file_utils.get_binary(filename)

return the entire content of the binary file as binary stream

mwr_raw2l1.utils.file_utils.get_corresponding_pickle(filename_rawdata, path_pickle, legacy_reader=False)

get pickled file from previous read-in corresponding to raw data file

mwr_raw2l1.utils.file_utils.get_files(dir_in, basename, time_start=None, time_end=None)

get files in dir_in corresponding to basename.

If ‘time_start’ and/or ‘time_end’ is given only files with timestamp in filename >=’time_start’ and/or <=’time_end’ are returned. E.g. time_start=20220101 and time_end=20220101 will return timestamps 20220101 and 202201010000.

Parameters:

• dir_in – directory where files of the respective instrument are located

• basename – first part of the filename (usually full identifier combining station-id and instrument-id)

• time_start (optional) – string in format ‘yyyymmddHHMM’, ‘yyyymmddHHMMSS’, ‘yyyymmdd’ or any of the similar

• time_end (optional) – analogous to time_start

Returns:

list of files in dictionary corresponding to basename and time criteria

mwr_raw2l1.utils.file_utils.group_files(files, name_scheme)

group files in a list of files

Parameters:

• files – list of files

• name_scheme ({'attex', 'rpg', 'radiometrics'}) – scheme of filename used for parts to ignore in grouping process

Returns:

list of lists of files for which all parts except the ignored ones are identical

mwr_raw2l1.utils.file_utils.pickle_dump(data, filename)

mwr_raw2l1.utils.file_utils.pickle_load(filename)

mwr_raw2l1.utils.file_utils.remove_ext(file)

remove extension and just return pure filename including path

mwr_raw2l1.utils.file_utils.remove_suffix(file, sep='_')

remove suffix including extension (all that comes after last ‘sep’) and return pure filename including path

mwr_raw2l1.utils.file_utils.timestamp_to_float(timestamp)

transform timestamp string to a float between 0 and 1 (integer of timestamp normalised by its length)

mwr_raw2l1.utils.file_utils.write_file_log(outfile, file_bunches)

write bunches of (un)successfully processed files to an output file.

Stores each path on one line with empty line between bunches.

Parameters:

• outfile – file where to write in (will be generated/overwritten)

• file_bunches – list of file bunches i.e. a list of lists of files

mwr_raw2l1.utils.num_utils

mwr_raw2l1.utils.num_utils.isbitset(x, nth_bit)

check if n-th bit is set (==1) in x

Parameters:

• x – integer or numpy.ndarray of integers

• nth_bit – position of investigated bit (0, 1, 2, …)

Returns:

boolean or boolean array denoting whether the nth_bit is set in x.

Examples

>>> isbitset(3, 0)
    True
>>> isbitset(2, 0)
    False

mwr_raw2l1.utils.num_utils.setbit(x, nth_bit)

set n-th bit (i.e. set to 1) in an integer or array of integers

Parameters:

• x – integer or numpy.ndarray of integers

• nth_bit – position of bit to be set (0, 1, 2, ..)

Returns:

integer or array of integers where n-th bit is set while all other bits are kept as in input x

Examples

>>> setbit(0, 1)
    2
>>> setbit(3, 2)
    7

mwr_raw2l1.utils.num_utils.timedelta2s(t_diff)

return number of seconds from numpy.timedelta64 object :param t_diff: time difference as numpy.timedelta64 object

Returns:

scalar corresponding to number of seconds

mwr_raw2l1.utils.num_utils.unsetbit(x, nth_bit)

unset n-th bit (i.e. set to 0) in an integer or array of integers

Parameters:

• x – integer or numpy.ndarray of integers

• nth_bit – position of bit to be set (0, 1, 2, ..)

Returns:

integer or array of integers where n-th bit is unset while all other bits are kept as in input x

Examples

>>> unsetbit(7, 2)
    3
>>> unsetbit(8, 2)
    8