multiconf package

multiconf.mc_config(env_factory, error_next_env=False, validate_properties=True, mc_todo_handling_other=<McTodoHandling.ERROR: 2>, mc_todo_handling_allowed=<McTodoHandling.WARNING: 1>, mc_json_filter=None, mc_json_fallback=None, do_type_check=None, do_post_validate=True, lazy_load=False, mc_5_migration=False)[source]

Function decorator for instantiating ConfigItem hierarchy for all Envs defined in ‘env_factory’.

This decorator creates a wrapped config function which is then used for retrieving the configuration for a specific env.

The wrapped function signature is:
Arguments:
env (Env): The environment for which to retrieve the config. allow_todo (bool): If true, then retrieving a configuration for an env which contains MC_TODO values will not raise an error.

Return (Root ConfigItem proxy): Reference to the config with the current env set to env.

E.g.:

@mc_config(envf)
def conf(root):
    with someitem() as it:
        it.setattr('aa', default=1, tst=2, prod=3)

# Get the cfg instantiated for 'prod'
prod_cfg = conf(prod)

NOTE, There can only be one current config! It is possible to instantiate the config multiple times, but storing references to items in the configuration, and accessing attributes at a later time, will return the value from the last env.

Parameters:
  • env_factory (EnvFactory) – The EnvFactory defining the envs for which we instantiate the configuration.
  • error_next_env (bool) – If this is False, then no more envs will be instantiated after errors are found in an env. If True, then instantiation is attempted for all envs, but an exception is raised at the end in any envs could not be instantiated.
  • mc_todo_handling_other (McTodoHandling) – This specifies how to handle attributes set to MC_TODO for envs with ‘allow_todo’ False. The default is McTodoHandling.ERROR, causing an error message to be printed and the configuration to be considered invalid.
  • mc_todo_handling_allowed (McTodoHandling) – This specifies how to handle attributes set to MC_TODO for envs with ‘allow_todo’ True. The default is McTodoHandling.WARNING, causing a warning message to be printed but the configuration to be considered valid.
  • mc_json_filter (func(obj, key, value)) – User defined function for filtering objects in json output. - filter_callable is called for each key/value pair of attributes on each ConfigItem obj. - It must return a tuple of (key, value). If key is False, the key/value pair is removed from the json output
  • mc_json_fallback (func(obj)) – User defined function for handling objects not otherwise encoded in json output. - fallback_callable is called for objects that are not handled by the builtin encoder. - It must return a tuple (object, handled). If handled is True, the object must be encodable by the standard json encoder.
  • do_type_check (bool) –

    Do type checking of attributes based on typing annotations. Default is True for Python 3.6.1+. Attempting to enable this for earlier Python versions will raise an exception.

    Type checking of attributes is done based on typing information from the __init__ signature. If an attribute exists with the same name as an __init__ argument with typing information, then the attribute must conform to that type. E.g.:

    class X(ConfigItem):
    def __init__(self, a:int = MC_REQUIRED):
        super(X).__init__()
        self.a = a

    It will be checked that x.a is instance of int.

  • do_post_validate (bool) – Allow skipping the mc_post_validate call. I.e. if set to False the user defined call backs are not called.
  • lazy_load (bool) – Allow loading config only for envs for which is instantiated by calling the wrapped config method. If False the config is pre-instantiated for all envs in order to validate correctness of the configuration for all envs. Enabling lazy_load also disables mc_post_validate calls and other checking which cannot be done with lazy loading.
  • mc_5_migration (bool) – This changes the attribute overwriting rule to me more compatible with version 5. Do not use this in any new configurations.
class multiconf.McInvalidValue[source]

Bases: enum.Enum

Special values which may be assigned to attributes.

MC_NO_VALUE

This is the initial value for attribute until it receives a value for an Env. This will only be observed when iterating over the env values of an attribute and the ConfigItem was excluded from some Envs. See attr_env_items(). You should never set an attribute to this value.

MC_REQUIRED

This is used as the default value for attributes in __init__ when there is no reasonable default. Multiconf will verify that a real value is assigned to the attribute during the config instantiation.

MC_TODO

This can be used in the configuration as a temporary place holder for values which are currently unknown. There are various options to make multiconf report on MC_TODO values.

class multiconf.McTodoHandling[source]

Bases: enum.Enum

Specify how to handle MC_TODO values in the configuration.

SILENT

Do not report MC_TODO

WARNING

Print a warning about each MC_TODO value.

ERROR

Print an error message about each MC_TODO value and raise an Exception.

multiconf.caller_file_line(up_level=2)[source]

Return the file and line of the caller of the function calling this function (depending on up_level)

class multiconf.ConfigItem(mc_include=None, mc_exclude=None)[source]

Base class for config items.

attr_env_items(attr_name, ignored_exceptions=())

Iterate through the attribute (env, value) for the all defined envs.

If ConfigExcludedAttributeError, ConfigExcludedKeyError or any exception specified in ignored_exceptions is raised then McInvalidValue.MC_NO_VALUE is returned.

Parameters:
  • attr_name (str) – The attribute name.
  • ignored_exceptions (type or sequence(type)) – Additional exception classes to ignore. This can be necessary when attr_name references a @property method which may raise an arbitrary error when called for an env where some of it’s dependencies may not be setup correctly.
Yields:

env (Env), value (any) –

The (env, attribute value) for each env or MC_NO_VALUE is there is no value for a specific

env (e.g. the item is excluded). If an exception was raised for all envs the last exception will propagate.
attr_env_values(attr_name, ignored_exceptions=())

Iterate through the attribute value for the all defined envs.

See attr_env_items() for common behaviour.

Yields:value (any) – The attribute value for each env.
find_attribute(name)

Find first occurrence of attribute or child item ‘name’, by searching backwards towards root_conf, starting with self.

find_attribute_or_none(name)

Find first occurrence of attribute or child item ‘name’, by searching backwards towards root_conf, starting with self.

find_contained_in(named_as)

Find first parent container named as ‘named_as’, by searching backwards towards root_conf, starting with parent container

find_contained_in_or_none(named_as)

Find first parent container named as ‘named_as’, by searching backwards towards root_conf, starting with parent container

getattr(attr_name, env)

Get the attribute value for the specified env.

Parameters:
  • attr_name (str) – The attribute name.
  • env (Env) – The env to get the value for.
json(compact=False, sort_attributes=False, property_methods=True, builders=False, skipkeys=True, warn_nesting=None, show_all_envs=False, depth=None, persistent_ids=False)

Create json representation of configuration.

The mc_json_filter and mc_json_fallback arguments to mc_config() also influence the output.

Parameters:
  • compact (bool) – Set compact to true if dumping for easier human readable output, false for machine readable output.
  • sort_attributes (bool) – Sort attributes by name. Sort dir() entries by name.
  • property_methods (bool) – call @property methods and insert values in output, including a comment that the value is calculated.
  • builders (bool) – Include ConfigBuilder items in json.
  • skipkeys (bool) – Passed to json.dumps.
  • show_all_envs (bool) – Display attribute values for all envs in a single dump. Without this only the values for the current env is displayed.
  • depth (int) – The number of levels of child objects to dump. None means all.
  • persistent_ids (bool) – Use a persistent value instead of using id(obj) as reference keys. NOTE: This will mostly make it impossible to identify the referenced obj, but it makes it possible to compare json across runs.
mc_init()

This is a user defined callback method.

This is called at the exit from a with statement. May be used for e.g. setting default values based on other properties or cross validation of different properties

mc_post_validate()

This is a user defined callback method.

This method is called once for each item after other initialization has been done for all envs, so cross env checking and cross object/attribute checking is possible. Since it is called once, and not per env, there is no current env and regular attribute access is not possible, instead the item.getattr(name, env) method must be used to get attribute values for different envs.

This makes it possible to implement checks like the following:

assert item.getattr('mem_size', pprd) == item.getattr('mem_size', prod) <= item.getattr('mem_size', tst1)

Note that careful consideration should be taken when using env names explicitly (as above) when implementing a configuration object model, since this will force all configurations to define those envs.

Note that no modifications can be done in this method!

mc_select_envs(include=None, exclude=None, mc_error_info_up_level=2)

Calculate whether item should be included or excluded.

This should be the first statement in the ‘with’ block If item is excluded, then the ‘with’ block, is skipped and no multiconf validations are done for item or contained items.

Parameters:
  • include (list[env]) – List of Envs/EngGroups for which to include this item (and contained items)
  • exclude (list[env]) – List of Envs/EngGroups for which to exclude this item (and contained items)
  • mc_error_info_up_level (int) – Only for use if a class overrides mc_select_envs. You must add 1 each time it is overridden. This is used for calculating the file:line in case of ambiguous include/exclude lists.
The inclusion/exclusion is done on a ‘most specific’ basis -
  • If an item is excluded by an EnvGroup specification but included by a more specific EnvGroup (or Env), then it will be included.
  • If an item is included by an EnvGroup specification but excluded by a more specific EnvGroup (or Env), then it will be excluded.
mc_validate()

This is a user defined callback method.

This method is called once for each item for each env after other initialization has been and all items are created. May be used for e.g. setting default values based on other properties or cross validation of different properties. It is preferable to use mc_init when possible as mc_init generally results in more precise error messages, and ensures that an item is fully defined when the ‘with’ statement is exited.

named_as()

Return the named_as property set by the @named_as decorator

num_invalid_property_usage

Returns number of ‘InvalidUsageException’ s encountered when validating @property methods Returns 0 if ‘mc_config’ was called with validate_properties=False.

num_json_errors()

Returns number of errors encountered when generating json Return 0 if json() has not been called

setattr(attr_name, mc_overwrite_property=False, mc_set_unknown=False, mc_force=False, mc_error_info_up_level=2, **env_values)

Set env specific values for an attribute.

Parameters:
  • attr_name (str) – The name of the attribute to set.
  • mc_overwrite_property (bool=False) – Setting this to True allows overwriting a @property method with env specific values. Any env for which the @property is not overridden will still get the value of the @property method.
  • mc_set_unknown (bool=False) – This allows setting a property which was not defined in the __init__ method.
  • mc_force (bool=False) – Force the value of the property regardless of the normal multiconf rules for assigning values. This should be used with care, as normal validation is disabled. Using this could be a sign of bad configuration/modelling.
  • mc_error_info_up_level (int) – Only for use if a class overrides setattr. You must add 1 each time it is overridden. This is used for calculating the file:line info in case of errors.
  • **env_values (dict[env-name]->value) – The env specific values to assign. Arg names must be valid env names from the EnvFactory used to create the configuration.
class multiconf.RepeatableConfigItem(mc_key=None, mc_include=None, mc_exclude=None)[source]

Base class for config items which may be repeated.

RepeatableConfigItems will be stored in an OrderedDict using the key ‘mc_key’.

Parameters:mc_key (hashable) – The key used to lookup the config item.
classmethod named_as()[source]

Return the named_as property set by the @named_as decorator

class multiconf.ConfigBuilder(mc_key='default-builder', mc_include=None, mc_exclude=None)[source]
mc_build()[source]

Override this in derived classes. This is where child ConfigItems are declared