Utility Classes and Functions

Monte Carlo

demod.utils.monte_carlo.monte_carlo_from_1d_cdf(cdf, n_samples=1)

Sample MC from a given CDF.

Parameters
  • cdf (numpy.ndarray) – A 1-D ndarray with values being the CDF

  • n_samples (int) – the number of samples to draw from the CDF

Returns

The result of the MC draw for each samples.

Return type

numpy.ndarray

Notes

The MC algo performs no checks on the CDFs

demod.utils.monte_carlo.monte_carlo_from_1d_pdf(pdf, n_samples=1)

Sample for given PDF.

Parameters
  • pdf (numpy.ndarray) – A 1-D ndarray with values being the PDF

  • n_samples (int) – the number of samples to draw from the PDF

Returns

The result of the MC draw for each samples.

Return type

numpy.ndarray

Notes

The MC algo performs no checks on the CDFs

demod.utils.monte_carlo.monte_carlo_from_cdf(cdf_s)

Sample from a set of given cumlative distribution functions (CDFs).

Parameters

cdf_s (numpy.ndarray) – A 2-D ndarray with dimension 0 = number of sample, dimension 1 = size of the CDFs.

Returns

The result of the MC draw for each samples.

Return type

numpy.ndarray

Notes

The MC algo performs no checks on the CDFs

demod.utils.monte_carlo.monte_carlo_from_pdf(pdf_s)

Sample from a set of given probability distribution functions (PDFs).

Parameters

pdf_s (numpy.ndarray) – A 2-D ndarray with dimension 0 = number of sample, dimension 1 = size of the PDFs.

Returns

The result of the MC draw for each samples.

Return type

numpy.ndarray

Notes

The MC algo performs no checks on the PDFs

Distibution functions

Various helpers for statistical distribution functions.

pdf = probability distribution function cdf = cumulative distribution function

demod.utils.distribution_functions.rescale_pdf(pdf, epsilon=0.1)

Rescale a pdf.

Adjust the value of the pdf if there are roundoff errors, by rescaling all the values of the pdfs

Parameters
  • pdf (numpy.ndarray) – ndarray, 2-dim a ndarray with dim0 being the number of sample and dim1 the pdfs of each samples

  • epsilon (float) – float the maximum error to allow correction

Returns

ndarray, 2-dim(float) the corrected pdf

demod.utils.distribution_functions.rescale_cdf(cdf, epsilon=0.1)

Rescale a cdf.

Adjust the value of the cdf if there are roundoff errors, by rescaling all the values of the cdfs

Parameters
  • cdf (numpy.ndarray) – ndarray, 2-dim a ndarray with dim0 being the number of sample and dim1 the cdfs of each samples

  • epsilon (float) – the maximum error to allow correction

Returns

ndarray, 2-dim(float), the corrected cdf

demod.utils.distribution_functions.check_valid_cdf(cdf, epsilon=1e-06)

Check the validity of the given cdf.

Check that the values are increasing. Check that it ends at 1.

Parameters
  • cdf (numpy.ndarray) – ndarray, of any size, with last dimension being the cdfs

  • epsilon (float) –

Returns

True if the cdf is valid

Raises

ValueError – if the cdf is invalid

Get Methods

All simulators have some get methods which makes it usefull to access data of the current simulation state.

TODO: add here the get methods and references to them so that we can call them from the docstrings of the other classes

demod.simulators.base_simulators.GetMethod

The central part of internal API.

This represents a generic version of type ‘origin’ with type arguments ‘params’. There are two kind of these aliases: user defined and special. The special ones are wrappers around builtin collections and ABCs in collections.abc. These must have ‘name’ always set. If ‘inst’ is False, then the alias can’t be instantiated, this is used by e.g. typing.List and typing.Dict.

alias of Callable[[demod.simulators.base_simulators.Simulator, Union[None, int]], numpy.ndarray]

@demod.simulators.base_simulators.cached_getter

Decorate getter methods.

Uses a cache system to store the arrays. Cache is cleared at each simulator step().

Parameters

getter (GetMethod) – The getter method to decorate.

Return type

GetMethod

Logging simulation data

class demod.simulators.base_simulators.SimLogger(attributes_list, *args, aggregated=True)

Specialized logger for any Simulator object.

Once the parameters are define, it can be set to a Simulator. During the simulation, the SimLogger will automatically collect the data. Then the SimLogger.get() method can be used to access the collected data.

TODO: make the logger attributes callable by the logger using setattr, and checking that they don’t break other methods

get(attribute=None)

Get the logged values for the desired attribute.

Parameters

attribute (Optional[str]) – The name of an attribute

Raises

ValueError – If the value of the attribute is not in this logger

Returns

Array with the logged values for desired attribute.

Return type

numpy.ndarray

clear(attribute=None)

Clear the requested attribute from the logger.

If attribute not specified, clears all attributes.

Parameters

attribute (Optional[Union[List, str]]) – The attribute to clear. Defaults to None.

Raises

ValueError – If the attribute is not recognized.

Return type

None

copy()

Create a copy of the cuurrent logger.

Only copy the attributes, not the collected data.

Returns

An empty logger with the same attributes.

Return type

demod.simulators.base_simulators.SimLogger

visit_simulator(sim)

Visit a simulator object

Parameters
Return type

None

Note

You usually won’t need to use this method, as it is called in Simulator.step()

print()

Print out all the attributes from the logger.

plot_column(aggregate=False)

Plot the logger values in a plot in column.

If ‘current_time’ was specified in the attributes, it will be used as x axis for all the subplots.

Parameters

aggregate (bool) – Wheter to aggregate the values. Defaults to False.

plot(aggregate=False)

Plots the profiles stored by the logger.

The plots are shown sequentially.

Parameters

aggregate (bool) – whether to force aggregation of the profiles.

Sparse

class demod.utils.sparse.SparseTPM(times, inds_from, inds_to, values, n_states=None, n_times=None, dead_state_value=None)

A sparse implementation of the Transition probability Matrices.

Class to store a set of Transition Probability Matrices at different times that is very large and sparse. Provides methods for iteration over time, indexing, and Monte Carlo sampling.

times

The timestep of each transitions

Type

numpy.ndarray

inds_from

The states indices from which the transition is performed

Type

numpy.ndarray

inds_to

The states indices to which the transition is performed

Type

numpy.ndarray

values

The probabilities of this transition to occur

Type

numpy.ndarray

n_states

The total number of states

Type

int

n_times

The total number of times

Type

int

dead_state_value

The states which is assigned in case of dead states

Type

int

sparse_monte_carlo(time, states)

Apply a MC sampling from the current states given as input.

Parameters
  • time (int) – The current time at which the transition probabilities are used.

  • states (numpy.ndarray) – The current states, dtype=int.

Returns

The new states after the monte carlo simulation, dtype=int.

Return type

numpy.ndarray

save(path)

Save the TPM at the specified path.

Can be then loaded via the load() method.

Parameters

path (str) – The path to which the TPM should be saved.

Return type

None

static load(path)

Load a TPM from the specified path.

The TPM can be saved via the save method.

Parameters

path (str) – The path to which storing the TPM.

Returns

the sparse TPM at the requested path.

Return type

SparseTPM

Error messages

Different formatted error messages that can be used in Demod.

You can use these messages like this:

raise ValueError(UNKOWN_POPULATION_TYPE.format(
    population_type='resident_number',
    dataset=YouDataLoader
))
  • UNKOWN_POPULATION_TYPE(population_type, dataset)

  • ALGO_REQUIRES_LOADING_METHOD(algo, simulator, loading_method, dataset)

  • NOT_IMPLEMENTED_IN_DATASET_FOR_VERSION(not_implemented, dataset, version)

  • UNIMPLEMENTED_ALGO_IN_METHOD(algo, method)

Parsing helpers

Helper functions for parsing Datasets.

demod.utils.parse_helpers.states_to_transitions(states, return_duration=False, include_same_state=False, ignore_end_start_transitions=False)

Convert a state array to transitions array.

For the durations of states between the nights, if the first states and the last states are the same, the duration is computed as the sum. If not, the two durations are kept. transition at time t: new_state = state[t], old_state = state[t-1]

Parameters
  • states (numpy.ndarray) – The array containing the states. shape=(n_diaries, n_times), dtype=int or str

  • return_duration (bool) – Wether to return the duration of the states. Defaults to False.

  • include_same_state (bool) – Whether to also include transitions from a state to the same state. This is not meant to be used with ‘return_duration = True’.

  • ignore_end_start_transitions (bool) – Whether to ignore the transitions that happens from the end of the diary to the start.

Returns

transitions_dict, containing the transitions

  • transitions_dict[‘times’]

    the times at which the transitions occur, index of where the new state is.

  • transitions_dict[‘persons’]

    the person or hh in the state array that performs the transition

  • transitions_dict[‘new_states’]

    state after transition

  • transitions_dict[‘old_states’]

    state before transition

  • transitions_dict[‘duration’]

    only included if ‘return_duration’ is True, return the duration of the state after each transition.

Return type

Dict[str, numpy.ndarray]

demod.utils.parse_helpers.states_to_transitions_secondary(primary_states, secondary_states)

Convert a state array to transitions array.

Parameters
  • states – The array containing the states. dtype=int or str

  • return_duration – Wether to return the duration of the states. Defaults to False.

  • primary_states (numpy.ndarray) –

  • secondary_states (numpy.ndarray) –

Returns

transitions_dict, containing the transitions

  • transitions_dict[‘times’]

    the times at which the transitions occur, index of where the new state is.

  • transitions_dict[‘persons’]

    the person or hh in the state array that performs the transition

  • transitions_dict[‘new_states’]

    state after transition

  • transitions_dict[‘old_states’]

    state before transition

demod.utils.parse_helpers.group_hh_transitions(primary_states, household_indexes, days_indexes, secondary_states=None)

Group the states into household transitions.

Converts the individual states to an household states that includes all the persons.

Parameters
  • primary_states (numpy.ndarray) – The base states array.

  • household_indexes (numpy.ndarray) – An array containing the household corresponding to each diary from the states array.

  • days_indexes (numpy.ndarray) – Array containing the day index of the diaries in the states array.

  • secondary_states (Optional[numpy.ndarray]) – Optional array of secondary states. If None, no secondary states will be used. Defaults to None.

Returns

transitions_dict, containing the transitions

  • transitions_dict[‘times’]

    the times at which the transitions occur, index of where the new state is.

  • transitions_dict[‘households’]

    the household from household_indexes

  • transitions_dict[‘new_states’]

    state after transition

  • transitions_dict[‘old_states’]

    state before transition

  • transitions_dict[‘counts’]

    The number of persons in the household that do this transition.

  • transitions_dict[‘day_nrs’]

    The days_indexes of this transition.

demod.utils.parse_helpers.convert_states(raw_states, merge_dic=None)

Convert the states following the rule inscibed in merge_dic.

Very useful to merge different states into a single one.

Parameters
  • raw_states (numpy.ndarray) – an array containing the possible states

  • merge_dic (Optional[dict]) – dictionary that matches each state to a new state

Returns

the new states u_lab: the label of those new states

Return type

merged states

demod.utils.parse_helpers.translate_1d(array, translating_dict)

Translate the values of the array according to the translating dict.

Parameters
  • array (Union[numpy.ndarray, list]) – list or array of single dimension

  • translating_dict (dict) – dict mapping values from array to what is returned.

Returns

translated array

demod.utils.parse_helpers.make_jsonable(object)

Tranforms some objects to ensure they are in json format.

This is not 100% successful. At the moment only tranforms ndarrays to lists.

Parameters

object (Any) – The object to be make jsonable.

demod.utils.parse_helpers.remove_spaces(object)

Remove spaces from all the strings that can be found.

This is not 100% successful. At the moment only tranforms ndarrays to lists.

Parameters

object (Any) – The object to remove the spaces from it and subobjects.

demod.utils.parse_helpers.lists_to_numpy_array(object)

Tranforms some objects to ensure their list are numpy arrays.

This is not 100% successful. At the moment only tranforms some list to numpy.

Parameters

object (Any) – The object to be make jsonable.

demod.utils.parse_helpers.bulbs_stats_from_config(config_array)

Transform the bulb config into, bubls and bulbs stats.

Parameters

config_array (numpy.ndarray) – The light config array.

Returns

(mean, std), (consumption, penetration)

Return type

Tuple[Tuple[float, float], Tuple[numpy.ndarray, numpy.ndarray]]

demod.utils.parse_helpers.states_to_tpms_with_durations(states, first_tpm_modification_algo='last', labels=None, ignore_end_start_transitions=True)

Convert the states to tpms for semi markov chains.

The output tpms have at time t, the transition from t-1 to t. The durations cdf have at index i the prob of a duration of i steps. Note that this also accounts for full time period duration, which is represented by the pdf for duration of size n_times + 1, and their corresponding transition in the tpm at time 0.

Any transition that does not occur in the states array is handled by setting a duration of 0, and the tpm will only have the probability to stay at the same state.

Parameters
  • states (numpy.ndarray) – The states to convert. They must be integer states, which can be done using :py:function:`convert_states`.

  • first_tpm_modification_algo (str) – Algo to use to change the tpm from end of the diary to the start. Defaults to ‘last’. See: :py:function:`first_tpm_change` .

  • labels (Optional[Union[List[str], List[int], numpy.ndarray]]) – Optional labels, (useful if some state are not visited). Defaults to None.

  • ignore_end_start_transitions (bool) – Whether to ignore the transitions that occur between the end and start of the diaries.

Returns

Transition probability matrices, Duration of states pdf, Duration of states pdf with previous states.

Return type

Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]

demod.utils.parse_helpers.get_initial_durations_pdfs(states)

Compute the initial durations depending on the states.

Dim 0 corresponds to the different possible states, Dim 1 contains the pdfs.

Parameters

states (numpy.ndarray) –

Return type

numpy.ndarray

demod.utils.parse_helpers.counts_to_pdf(counts, ensure_valid_pdf=True)

Transform an array countaining counts of an event to pdfs.

The last dimension of counts will be the one that is converted to pdf. If there are no counts along the dimension and ‘ensure_valid_pdf’ is set to True, the pdf will be [1., 0. , …, 0.]

Parameters

counts (numpy.ndarray) –

demod.utils.parse_helpers.states_to_tpms(states, first_tpm_modification_algo='last', labels=None)

Convert the states to tranistions probability matrices.

The output tpms have at time t, the transition from t-1 to t.

Parameters
  • states (numpy.ndarray) – The states to convert.

  • first_tpm_modification_algo (str) – Algo to use to change the tpm from end of the diary to the start. Defaults to ‘last’. See: :py:function:`first_tpm_change` .

  • labels (Optional[Union[List[str], List[int], numpy.ndarray]]) – Optional labels, (useful if some state are not visited). Defaults to None.

Returns

Transition probability matrices.

Return type

numpy.ndarray

demod.utils.parse_helpers.first_tpm_change(tpms, algo='nothing')

Change the first tpm based on an algorithm.

Parameters

algo

The algorithm that should be used to change the first tpm values.

  • ’last’, replaces the first by the last matrix

  • ’nothing’, keeps the same

Returns

The modified tpms.

Other functions

demod.simulators.util.sample_population(n_samples, pdf, population_sampling_algo='real_population', **kwargs)

Sample the population based on a pdf.

Parameters
  • n_samples (int) – The number of samples to be sampled.

  • pdf (numpy.ndarray) – The probability of being in the population for each sample.

  • population_sampling_algo (str) –

    algorithm to be used, currently implemented:

    • ’real_population’:

      Based on the real population

    • ’monte_carlo’:

      Randomly assign the number based on a MC draw.

    Defaults to ‘real_population’.

Returns

The number of samples from each value of the pdf.

Raises

ValueError – if the pdf does not sum up to one.

Return type

numpy.ndarray

Units Conversion

Some units converters.

Demod uses SI units by default. (Watts for power and Joules for energy.)

demod.utils.converters.joules_to_kwh(x)

Transform joules to kwh.

1 kwh = 3’600’000 Joules

Parameters

x (Any) – The object to convert. Must support division by float.

Return type

Any

demod.utils.converters.kwh_to_joules(x)

Transform kwh to joules.

1 kwh = 3’600’000 Joules

Parameters

x (Any) – The object to convert. Must support multiplication by float.

Return type

Any