config

Contains helpers for defining and providing configuration classes. A typical usage would be to create the files config.py, config.json and config_local.json in a project’s root directory. An example of a config.py for a data-driven project is below. For a non-data driven project, the configuration class should inherit from accsr.config.ConfigurationBase, and the resulting class will not have any pre-populated public entries.

>>> from accsr.config import DefaultDataConfiguration, ConfigProviderBase
>>>
>>> class __Configuration(DefaultDataConfiguration):
...     @property
...     def custom_entry_from_config(self):
...         return self._get_non_empty_entry("custom_entry_from_config")
...
...     @property
...     def existing_path_in_base_dir(self):
...         return self._get_existing_path(["base_dir", "path_in_base_dir"])
...
...     @property
...     def custom_path_in_processed_data(self):
...         return self.datafile_path("my_data", stage=self.PROCESSED, check_existence=False)
>>>
>>> class ConfigProvider(ConfigProviderBase[__Configuration]):
...     pass
>>>
>>> _config_provider = ConfigProvider()
>>>
>>>
>>> def get_config(reload=False):
...     return _config_provider.get_config(reload=reload)

recursive_dict_update(d: Dict, u: Dict)

Modifies d inplace by overwriting with non-dict values from u and updating all dict-values recursively. Returns the modified d.

get_config_reader(filename: str) → Callable[[TextIO], Dict]

Returns a reader for yaml or json files. The file type is determined by the file extension.

class ConfigurationBase(config_directory: Optional[str] = None, config_files=('config.json', 'config_local.json'))

Bases: abc.ABC

Base class for reading and retrieving configuration entries. Do not instantiate this class directly but instead inherit from it.

ENV_VAR_MARKER = 'env:'

class DefaultDataConfiguration(config_directory: Optional[str] = None, config_files=('config.json', 'config_local.json'))

Bases: accsr.config.ConfigurationBase, abc.ABC

Reads default configuration entries and contains retrieval methods for a typical data-driven project. A typical config.json file would look like this:

{

“data_raw”: “data/raw”,

“data_cleaned”: “data/cleaned”,

“data_processed”: “data/processed”,

“data_ground_truth”: “data/ground_truth”,

“visualizations”: “data/visualizations”,

“artifacts”: “data/artifacts”,

“temp”: “temp”,

“data”: “data”

}

PROCESSED = 'processed'

RAW = 'raw'

CLEANED = 'cleaned'

GROUND_TRUTH = 'ground_truth'

DATA = 'data'

property artifacts

property visualizations

property temp

property data

property data_raw

property data_cleaned

property data_processed

property data_ground_truth

datafile_path(filename: str, stage='raw', relative=False, check_existence=False)

Parameters

• filename –

• stage – raw, ground_truth, cleaned or processed

• relative – If True, the returned path will be relative the project’s top-level directory

• check_existence – if True, will raise an error when file does not exist

artifact_path(name: str, relative=False, check_existence=False)

Parameters

• name –

• relative – If true, the returned path will be relative the project’s top-level directory.

• check_existence – if True, will raise an error when file does not exist

Returns

class ConfigProviderBase

Bases: Generic[accsr.config.ConfigurationClass], abc.ABC

Class for providing a config-singleton. Should not be instantiated directly but instead subclassed with an appropriate subclass of ConfigurationBase substituting the generic type.

Usage example:

>>> from accsr.config import ConfigurationBase, ConfigProviderBase
>>> class __MyConfigClass(ConfigurationBase):
...     pass
>>> class __MyConfigProvider(ConfigProviderBase[__MyConfigClass]):
...     pass
...
>>> _config_provider = __MyConfigProvider()
...
>>> def get_config():
...     return _config_provider.get_config()

get_config(reload=False, *args, **kwargs) → accsr.config.ConfigurationClass

Retrieves the configuration object (as singleton).

Parameters

• reload – if True, the config will be reloaded from disk even if a suitable configuration object already exists. This is mainly useful in interactive environments like notebooks.

• args – passed to init of the configuration class

• kwargs – passed to init of the configuration class constructor

Returns