Readers

major modules

mwr_raw2l1.readers.reader_attex

class mwr_raw2l1.readers.reader_attex.Reader(filename)

Bases: object

data_raw_to_np()

in-place replacement for self.data_raw to a numpy array

get_ele()

get elevations from column headers

interpret all numeric column headers as elevations. Returned column indices can be used to get corresponding Tb

Returns:

ele, column_indices

get_freq()

get frequency from header info

get_time_format(time_header)

interpret_data()

fill up self.data using self.data_raw and self.header

read(header_only=False)

read the data form csv and fill self.header (dictionary) and self.data_raw (numpy.ndarray)

run()

main method of the class

mwr_raw2l1.readers.reader_attex.read_multiple_files(files)

read multiple L1-related files and return list of executed read-in class instances

Parameters:

files – list of files to read in

Returns:

list of instances of executed read-in classes of Reader.

mwr_raw2l1.readers.reader_radiometrics

class mwr_raw2l1.readers.reader_radiometrics.Reader(filename)

Bases: object

data_raw_to_np()

transform data_raw to a dictionary with values of numpy.ndarray and remove entries without data

interpret_aux()

interpret auxiliary data, i.e. infrared brightness temperatures and meteo observations

interpret_data()

interpret the data in self.data_raw and feed to self.data

interpret_mwr()

interpret microwave radiometer data

read(header_only=False)

read the data form csv and fill self.header and self.data_raw

run()

main method of the class

sort_data_line(line)

attribute a csv line of the data section to the correct header reference

mwr_raw2l1.readers.reader_radiometrics.read_multiple_files(files)

read multiple L1-related files and return list of executed read-in class instances

Parameters:

files – list of files to read in

Returns:

list of instances of executed read-in classes of Reader.

mwr_raw2l1.readers.reader_rpg

readers for the different binary files from RPG radiometers (HATPRO, TEMPRO or HUMPRO)

class mwr_raw2l1.readers.reader_rpg.BLB(filename, accept_localtime=False)

Bases: BaseReader

interpret_filecode()

assign configuration for read in of file with corresponding file code

interpret_header()

interpret data read in with _read_header

interpret_raw_data()

interpret data read in with _read_meas (e.g. get ele/azi from pointing code or datetime from timecode)

interpret_scanobs()

transform scanobs 3D array to time series of spectra and temperature

class mwr_raw2l1.readers.reader_rpg.BRT(filename, accept_localtime=False)

Bases: BaseReader

interpret_filecode()

assign configuration for read in of file with corresponding file code

class mwr_raw2l1.readers.reader_rpg.HKD(filename, accept_localtime=False)

Bases: BaseReader

interpret_filecode()

assign configuration for read in of file with corresponding file code

interpret_raw_data()

interpret data read in with _read_meas (e.g. get ele/azi from pointing code or datetime from timecode)

class mwr_raw2l1.readers.reader_rpg.IRT(filename, accept_localtime=False)

Bases: BaseReader

interpret_filecode()

assign configuration for read in of file with corresponding file code

class mwr_raw2l1.readers.reader_rpg.MET(filename, accept_localtime=False)

Bases: BaseReader

interpret_filecode()

assign configuration for read in of file with corresponding file code

mwr_raw2l1.readers.reader_rpg.main()

mwr_raw2l1.readers.reader_rpg.read_multiple_files(files)

read multiple L1-related files and return dictionary of executed read-in class instances

Parameters:

files – list of files to read in

Returns:

dictionary with keys brt, blb, irt, met, hkd containing list with all read-in class instances for the corresponding file extension matching basename and the timing requirement. If no file of the corresponding type is found an empty list is returned for this key.

helper modules

mwr_raw2l1.readers.reader_helpers

mwr_raw2l1.readers.reader_helpers.check_input_filelist(files)

check that input files are given as a list and not as a single string

mwr_raw2l1.readers.reader_helpers.check_vars(data, mandatory_vars)

check that ‘mandatory_vars’ exist in ‘data’

mwr_raw2l1.readers.reader_helpers.get_column_ind(header, column_title)

get the zero-based column index corresponding to the column title

Parameters:

• header – list of all column headers

• column_title – column title to search for in the header

Returns:

the index of the column corresponding to column_title

mwr_raw2l1.readers.reader_helpers.get_time(data_raw, header, header_time, date_format)

extract time from data_raw using header

Parameters:

• data_raw – raw data as numpy.ndarray object

• header – column header belonging to data_raw

• header_time – the pattern for matching the time variable in the header

• date_format – format how the date is encoded in the string. Used by datetime.datetime

Returns:

a numpy.ndarray of datetime.datetime objects

mwr_raw2l1.readers.reader_helpers.simplify_header(str_in)

simplify strings to match col headers more robustly. Use on both sides of the ‘==’ operator

mwr_raw2l1.readers.reader_radiometrics_helpers

mwr_raw2l1.readers.reader_radiometrics_helpers.get_data(data_raw, header, no_mwr=False, **kwargs)

extract all known data from data_raw using header

Parameters:

• data_raw – raw data as numpy.ndarray object

• header – column header belonging to data_raw

• no_mwr – indicate whether data_raw contains channels of MWR observations. Defaults to False (contains MWR data).

• **kwargs – keyword arguments passed on to get_mwr()

Returns

a dictionary with variable name as key and numpy.ndarray as values

mwr_raw2l1.readers.reader_radiometrics_helpers.get_mwr(data_raw, header, only_observed_freq=True)

extract the microwave radiometer brightness temperatures and frequencies from data_raw using header

Parameters:

• data_raw – raw data as numpy.ndarray object

• header – column header belonging to data_raw

• only_observed_freq – if True only frequencies with non-NaN observations are returned. Defaults to True.

Returns:

the tuple (brightness_temperature, frequency) where both elements are a numpy.ndarray

mwr_raw2l1.readers.reader_radiometrics_helpers.get_record_type(line, ind=2)

get record type number in csv line and return as int. By default, the third element (ind=2) in line is used.

mwr_raw2l1.readers.reader_radiometrics_helpers.get_simple_vars(data_raw, header)

extract variables from data_raw according to header, except the MWR brightness temperatures/frequencies and time

Note: For extracting brightness temperatures and frequencies use get_mwr(); for time use get_time()

Parameters:

• data_raw – raw data as numpy.ndarray object

• header – column header belonging to data_raw

Returns:

a dictionary with variable name as key and numpy.ndarray as values

mwr_raw2l1.readers.reader_rpg_base

class mwr_raw2l1.readers.reader_rpg_base.BaseReader(filename, accept_localtime=False)

Bases: object

check_data()

general checks for the consistency of the data which can be applied to all file type readers

decode_binary(encoding_pattern, byte_order='<')

decode next variables from binary stream and write to dict self.data + augment self.byte_offset

Parameters:

encoding_pattern – a list of tuples or lists containing the individual variable description e.g. [dict(name=’n_meas’, type=’i’, shape=(1,)), dict(name=’Tb’, type=’f’, shape=(n_freq,)), …]

decode_binary_np(encoding_pattern, n_entries, byte_order='<')

decode from binary stream via numpy.ndarray to write to dict self.data + augment self.byte_offset

Parameters:

encoding_pattern – a list of tuples or lists containing the individual variable description for one time step e.g. [dict(name=’time_raw’, type=’i’, shape=(1,)), dict(name=’Tb’, type=’f’, shape=(n_freq,)), …]

interpret_filecode()

assign configuration for read in of file with corresponding file code

interpret_header()

interpret data read in with _read_header

interpret_raw_data()

interpret data read in with _read_meas (e.g. get ele/azi from pointing code or datetime from timecode)

read()

read and interpret all data in self.data_bin

run()

do whole read-in from files and interpretation and checking of data

mwr_raw2l1.readers.reader_rpg_helpers

helper functions for the module reader_rpg

mwr_raw2l1.readers.reader_rpg_helpers.flag_int2bits(flag_integer)

transform an integer value to bits. Input can be time series or scalars

mwr_raw2l1.readers.reader_rpg_helpers.interpret_angle(x, version)

translate the angle encoding from RPG to elevation and azimuth in degrees

Parameters:

• x – RPG angle.

• version – version of RPG angle encoding: 1: sign(ele) * (abs(ele)+1000*azi) 2: digits 1-5 = elevation*100; digits 6-10 = azimuth*100

Returns:

elevation, azimuth

mwr_raw2l1.readers.reader_rpg_helpers.interpret_bit_order(bit_order)

interpret bit_order for use in numpy.unpackbits() so that it also ‘>’ and ‘<’ can be used

mwr_raw2l1.readers.reader_rpg_helpers.interpret_coord(x, version=2)

Translate coordinate encoding from RPG to degrees with decimal digits.

Parameters:

• x – float encoding latitude of longitude coordinate in RPG binaries

• version –

  version of RPG coordinate encoding: 1: latitude or longitude in format (-)DDDMM.mmmm where DDD is degrees, MM is minutes and mmmm is the decimal

  fraction of MM

  2: latitude and longitude already in decimal degrees, function does nothing.

Returns:

latitude or longitude in format decimal degrees.

mwr_raw2l1.readers.reader_rpg_helpers.interpret_hkd_contents_code(contents_code_integer)

interpret the integer contents code from HKD files and return dict of contents variables

mwr_raw2l1.readers.reader_rpg_helpers.interpret_met_auxsens_code(auxsenscode)

interpret integer code for the availability of auxiliary sensors in MET files, return dict of contents vars

mwr_raw2l1.readers.reader_rpg_helpers.interpret_quadrant_int(int_quadr)

helper function for interpret scanflag for interpreting 2nd and 3rd bit (as int). See documentation from there

mwr_raw2l1.readers.reader_rpg_helpers.interpret_scanflag(flag_integer)

interpret flag of BLB scan

Parameters:

flag_integer – scanflag from BLB as integer. When transformed to bit, first bit interpreted as rainflag while 2nd/3rd encodes scan time (see notes)

Returns:

Dictionary with keys ‘rainflag’ and ‘scan_quadrant’

Notes

2nd/3rd bit (input)	scan_quadrant (output)	description
0/0	1	first quadrant
0/1	2	second quadrant
1/0	0	average over both quadrants

mwr_raw2l1.readers.reader_rpg_helpers.interpret_statusflag(flag_integer)

interpret the statusflag from HKD files and return a dict of status variables (input time series or scalar)

mwr_raw2l1.readers.reader_rpg_helpers.interpret_time(time_in)

translate the time format of RPG files to datetime object

mwr_raw2l1.readers.reader_rpg_helpers.interpret_tstab_flag(flag)

interpret temperature stability flag and return a flag saying stability ok (1), not ok (0) or unknown (NaN)

Notes

tstab flag (input)	tstab ok (output)	description
0	NaN	unknown stability (too short measurement series available)
1	1	stability ok
2	0	not ok (T sensors differ by >0.3 K)