Utility Classes and Functions¶
Monte Carlo¶
-
demod.utils.monte_carlo.
monte_carlo_from_1d_cdf
(cdf, n_samples=1)¶ Sample MC from a given CDF.
- Parameters
cdf (numpy.ndarray) – A 1-D ndarray with values being the CDF
n_samples (int) – the number of samples to draw from the CDF
- Returns
The result of the MC draw for each samples.
- Return type
numpy.ndarray
Notes
The MC algo performs no checks on the CDFs
-
demod.utils.monte_carlo.
monte_carlo_from_1d_pdf
(pdf, n_samples=1)¶ Sample for given PDF.
- Parameters
pdf (numpy.ndarray) – A 1-D ndarray with values being the PDF
n_samples (int) – the number of samples to draw from the PDF
- Returns
The result of the MC draw for each samples.
- Return type
numpy.ndarray
Notes
The MC algo performs no checks on the CDFs
-
demod.utils.monte_carlo.
monte_carlo_from_cdf
(cdf_s)¶ Sample from a set of given cumlative distribution functions (CDFs).
- Parameters
cdf_s (numpy.ndarray) – A 2-D ndarray with dimension 0 = number of sample, dimension 1 = size of the CDFs.
- Returns
The result of the MC draw for each samples.
- Return type
numpy.ndarray
Notes
The MC algo performs no checks on the CDFs
-
demod.utils.monte_carlo.
monte_carlo_from_pdf
(pdf_s)¶ Sample from a set of given probability distribution functions (PDFs).
- Parameters
pdf_s (numpy.ndarray) – A 2-D ndarray with dimension 0 = number of sample, dimension 1 = size of the PDFs.
- Returns
The result of the MC draw for each samples.
- Return type
numpy.ndarray
Notes
The MC algo performs no checks on the PDFs
Distibution functions¶
Various helpers for statistical distribution functions.
pdf = probability distribution function cdf = cumulative distribution function
-
demod.utils.distribution_functions.
rescale_pdf
(pdf, epsilon=0.1)¶ Rescale a pdf.
Adjust the value of the pdf if there are roundoff errors, by rescaling all the values of the pdfs
- Parameters
pdf (numpy.ndarray) – ndarray, 2-dim a ndarray with dim0 being the number of sample and dim1 the pdfs of each samples
epsilon (float) – float the maximum error to allow correction
- Returns
ndarray, 2-dim(float) the corrected pdf
-
demod.utils.distribution_functions.
rescale_cdf
(cdf, epsilon=0.1)¶ Rescale a cdf.
Adjust the value of the cdf if there are roundoff errors, by rescaling all the values of the cdfs
- Parameters
cdf (numpy.ndarray) – ndarray, 2-dim a ndarray with dim0 being the number of sample and dim1 the cdfs of each samples
epsilon (float) – the maximum error to allow correction
- Returns
ndarray, 2-dim(float), the corrected cdf
-
demod.utils.distribution_functions.
check_valid_cdf
(cdf, epsilon=1e-06)¶ Check the validity of the given cdf.
Check that the values are increasing. Check that it ends at 1.
- Parameters
cdf (numpy.ndarray) – ndarray, of any size, with last dimension being the cdfs
epsilon (float) –
- Returns
True if the cdf is valid
- Raises
ValueError – if the cdf is invalid
Get Methods¶
All simulators have some get methods which makes it usefull to access data of the current simulation state.
TODO: add here the get methods and references to them so that we can call them from the docstrings of the other classes
-
demod.simulators.base_simulators.
GetMethod
¶ The central part of internal API.
This represents a generic version of type ‘origin’ with type arguments ‘params’. There are two kind of these aliases: user defined and special. The special ones are wrappers around builtin collections and ABCs in collections.abc. These must have ‘name’ always set. If ‘inst’ is False, then the alias can’t be instantiated, this is used by e.g. typing.List and typing.Dict.
alias of Callable[[demod.simulators.base_simulators.Simulator, Union[None, int]], numpy.ndarray]
-
@
demod.simulators.base_simulators.
cached_getter
¶ Decorate getter methods.
Uses a cache system to store the arrays. Cache is cleared at each simulator step().
- Parameters
getter (GetMethod) – The getter method to decorate.
- Return type
GetMethod
Logging simulation data¶
-
class
demod.simulators.base_simulators.
SimLogger
(attributes_list, *args, aggregated=True)¶ Specialized logger for any Simulator object.
Once the parameters are define, it can be set to a
Simulator
. During the simulation, theSimLogger
will automatically collect the data. Then theSimLogger.get()
method can be used to access the collected data.TODO: make the logger attributes callable by the logger using setattr, and checking that they don’t break other methods
-
get
(attribute=None)¶ Get the logged values for the desired attribute.
- Parameters
attribute (Optional[str]) – The name of an attribute
- Raises
ValueError – If the value of the attribute is not in this logger
- Returns
Array with the logged values for desired attribute.
- Return type
numpy.ndarray
-
clear
(attribute=None)¶ Clear the requested attribute from the logger.
If attribute not specified, clears all attributes.
- Parameters
attribute (Optional[Union[List, str]]) – The attribute to clear. Defaults to None.
- Raises
ValueError – If the attribute is not recognized.
- Return type
None
-
copy
()¶ Create a copy of the cuurrent logger.
Only copy the attributes, not the collected data.
- Returns
An empty logger with the same attributes.
- Return type
-
visit_simulator
(sim)¶ Visit a simulator object
- Parameters
sim (demod.simulators.base_simulators.Simulator) – The simulator that is visited. The SimLogger will request and store the attributes registered in the logger.
- Return type
None
Note
You usually won’t need to use this method, as it is called in
Simulator.step()
-
print
()¶ Print out all the attributes from the logger.
-
plot_column
(aggregate=False)¶ Plot the logger values in a plot in column.
If ‘current_time’ was specified in the attributes, it will be used as x axis for all the subplots.
- Parameters
aggregate (bool) – Wheter to aggregate the values. Defaults to False.
-
plot
(aggregate=False)¶ Plots the profiles stored by the logger.
The plots are shown sequentially.
- Parameters
aggregate (bool) – whether to force aggregation of the profiles.
-
Sparse¶
-
class
demod.utils.sparse.
SparseTPM
(times, inds_from, inds_to, values, n_states=None, n_times=None, dead_state_value=None)¶ A sparse implementation of the Transition probability Matrices.
Class to store a set of Transition Probability Matrices at different times that is very large and sparse. Provides methods for iteration over time, indexing, and Monte Carlo sampling.
-
times
¶ The timestep of each transitions
- Type
numpy.ndarray
-
inds_from
¶ The states indices from which the transition is performed
- Type
numpy.ndarray
-
inds_to
¶ The states indices to which the transition is performed
- Type
numpy.ndarray
-
values
¶ The probabilities of this transition to occur
- Type
numpy.ndarray
-
n_states
¶ The total number of states
- Type
int
-
n_times
¶ The total number of times
- Type
int
-
dead_state_value
¶ The states which is assigned in case of dead states
- Type
int
-
sparse_monte_carlo
(time, states)¶ Apply a MC sampling from the current states given as input.
- Parameters
time (int) – The current time at which the transition probabilities are used.
states (numpy.ndarray) – The current states, dtype=int.
- Returns
The new states after the monte carlo simulation, dtype=int.
- Return type
numpy.ndarray
-
save
(path)¶ Save the TPM at the specified path.
Can be then loaded via the load() method.
- Parameters
path (str) – The path to which the TPM should be saved.
- Return type
None
-
Error messages¶
Different formatted error messages that can be used in Demod.
You can use these messages like this:
raise ValueError(UNKOWN_POPULATION_TYPE.format(
population_type='resident_number',
dataset=YouDataLoader
))
UNKOWN_POPULATION_TYPE(population_type, dataset)
ALGO_REQUIRES_LOADING_METHOD(algo, simulator, loading_method, dataset)
NOT_IMPLEMENTED_IN_DATASET_FOR_VERSION(not_implemented, dataset, version)
UNIMPLEMENTED_ALGO_IN_METHOD(algo, method)
Parsing helpers¶
Helper functions for parsing Datasets.
-
demod.utils.parse_helpers.
states_to_transitions
(states, return_duration=False, include_same_state=False, ignore_end_start_transitions=False)¶ Convert a state array to transitions array.
For the durations of states between the nights, if the first states and the last states are the same, the duration is computed as the sum. If not, the two durations are kept. transition at time t: new_state = state[t], old_state = state[t-1]
- Parameters
states (numpy.ndarray) – The array containing the states. shape=(n_diaries, n_times), dtype=int or str
return_duration (bool) – Wether to return the duration of the states. Defaults to False.
include_same_state (bool) – Whether to also include transitions from a state to the same state. This is not meant to be used with ‘return_duration = True’.
ignore_end_start_transitions (bool) – Whether to ignore the transitions that happens from the end of the diary to the start.
- Returns
transitions_dict, containing the transitions
- transitions_dict[‘times’]
the times at which the transitions occur, index of where the new state is.
- transitions_dict[‘persons’]
the person or hh in the state array that performs the transition
- transitions_dict[‘new_states’]
state after transition
- transitions_dict[‘old_states’]
state before transition
- transitions_dict[‘duration’]
only included if ‘return_duration’ is True, return the duration of the state after each transition.
- Return type
Dict[str, numpy.ndarray]
-
demod.utils.parse_helpers.
states_to_transitions_secondary
(primary_states, secondary_states)¶ Convert a state array to transitions array.
- Parameters
states – The array containing the states. dtype=int or str
return_duration – Wether to return the duration of the states. Defaults to False.
primary_states (numpy.ndarray) –
secondary_states (numpy.ndarray) –
- Returns
transitions_dict, containing the transitions
- transitions_dict[‘times’]
the times at which the transitions occur, index of where the new state is.
- transitions_dict[‘persons’]
the person or hh in the state array that performs the transition
- transitions_dict[‘new_states’]
state after transition
- transitions_dict[‘old_states’]
state before transition
-
demod.utils.parse_helpers.
group_hh_transitions
(primary_states, household_indexes, days_indexes, secondary_states=None)¶ Group the states into household transitions.
Converts the individual states to an household states that includes all the persons.
- Parameters
primary_states (numpy.ndarray) – The base states array.
household_indexes (numpy.ndarray) – An array containing the household corresponding to each diary from the states array.
days_indexes (numpy.ndarray) – Array containing the day index of the diaries in the states array.
secondary_states (Optional[numpy.ndarray]) – Optional array of secondary states. If None, no secondary states will be used. Defaults to None.
- Returns
transitions_dict, containing the transitions
- transitions_dict[‘times’]
the times at which the transitions occur, index of where the new state is.
- transitions_dict[‘households’]
the household from household_indexes
- transitions_dict[‘new_states’]
state after transition
- transitions_dict[‘old_states’]
state before transition
- transitions_dict[‘counts’]
The number of persons in the household that do this transition.
- transitions_dict[‘day_nrs’]
The days_indexes of this transition.
-
demod.utils.parse_helpers.
convert_states
(raw_states, merge_dic=None)¶ Convert the states following the rule inscibed in merge_dic.
Very useful to merge different states into a single one.
- Parameters
raw_states (numpy.ndarray) – an array containing the possible states
merge_dic (Optional[dict]) – dictionary that matches each state to a new state
- Returns
the new states u_lab: the label of those new states
- Return type
merged states
-
demod.utils.parse_helpers.
translate_1d
(array, translating_dict)¶ Translate the values of the array according to the translating dict.
- Parameters
array (Union[numpy.ndarray, list]) – list or array of single dimension
translating_dict (dict) – dict mapping values from array to what is returned.
- Returns
translated array
-
demod.utils.parse_helpers.
make_jsonable
(object)¶ Tranforms some objects to ensure they are in json format.
This is not 100% successful. At the moment only tranforms ndarrays to lists.
- Parameters
object (Any) – The object to be make jsonable.
-
demod.utils.parse_helpers.
remove_spaces
(object)¶ Remove spaces from all the strings that can be found.
This is not 100% successful. At the moment only tranforms ndarrays to lists.
- Parameters
object (Any) – The object to remove the spaces from it and subobjects.
-
demod.utils.parse_helpers.
lists_to_numpy_array
(object)¶ Tranforms some objects to ensure their list are numpy arrays.
This is not 100% successful. At the moment only tranforms some list to numpy.
- Parameters
object (Any) – The object to be make jsonable.
-
demod.utils.parse_helpers.
bulbs_stats_from_config
(config_array)¶ Transform the bulb config into, bubls and bulbs stats.
- Parameters
config_array (numpy.ndarray) – The light config array.
- Returns
(mean, std), (consumption, penetration)
- Return type
Tuple[Tuple[float, float], Tuple[numpy.ndarray, numpy.ndarray]]
-
demod.utils.parse_helpers.
states_to_tpms_with_durations
(states, first_tpm_modification_algo='last', labels=None, ignore_end_start_transitions=True)¶ Convert the states to tpms for semi markov chains.
The output tpms have at time t, the transition from t-1 to t. The durations cdf have at index i the prob of a duration of i steps. Note that this also accounts for full time period duration, which is represented by the pdf for duration of size n_times + 1, and their corresponding transition in the tpm at time 0.
Any transition that does not occur in the states array is handled by setting a duration of 0, and the tpm will only have the probability to stay at the same state.
- Parameters
states (numpy.ndarray) – The states to convert. They must be integer states, which can be done using :py:function:`convert_states`.
first_tpm_modification_algo (str) – Algo to use to change the tpm from end of the diary to the start. Defaults to ‘last’. See: :py:function:`first_tpm_change` .
labels (Optional[Union[List[str], List[int], numpy.ndarray]]) – Optional labels, (useful if some state are not visited). Defaults to None.
ignore_end_start_transitions (bool) – Whether to ignore the transitions that occur between the end and start of the diaries.
- Returns
Transition probability matrices, Duration of states pdf, Duration of states pdf with previous states.
- Return type
Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray]
-
demod.utils.parse_helpers.
get_initial_durations_pdfs
(states)¶ Compute the initial durations depending on the states.
Dim 0 corresponds to the different possible states, Dim 1 contains the pdfs.
- Parameters
states (numpy.ndarray) –
- Return type
numpy.ndarray
-
demod.utils.parse_helpers.
counts_to_pdf
(counts, ensure_valid_pdf=True)¶ Transform an array countaining counts of an event to pdfs.
The last dimension of counts will be the one that is converted to pdf. If there are no counts along the dimension and ‘ensure_valid_pdf’ is set to True, the pdf will be [1., 0. , …, 0.]
- Parameters
counts (numpy.ndarray) –
-
demod.utils.parse_helpers.
states_to_tpms
(states, first_tpm_modification_algo='last', labels=None)¶ Convert the states to tranistions probability matrices.
The output tpms have at time t, the transition from t-1 to t.
- Parameters
states (numpy.ndarray) – The states to convert.
first_tpm_modification_algo (str) – Algo to use to change the tpm from end of the diary to the start. Defaults to ‘last’. See: :py:function:`first_tpm_change` .
labels (Optional[Union[List[str], List[int], numpy.ndarray]]) – Optional labels, (useful if some state are not visited). Defaults to None.
- Returns
Transition probability matrices.
- Return type
numpy.ndarray
-
demod.utils.parse_helpers.
first_tpm_change
(tpms, algo='nothing')¶ Change the first tpm based on an algorithm.
- Parameters
algo –
The algorithm that should be used to change the first tpm values.
’last’, replaces the first by the last matrix
’nothing’, keeps the same
- Returns
The modified tpms.
Other functions¶
-
demod.simulators.util.
sample_population
(n_samples, pdf, population_sampling_algo='real_population', **kwargs)¶ Sample the population based on a pdf.
- Parameters
n_samples (int) – The number of samples to be sampled.
pdf (numpy.ndarray) – The probability of being in the population for each sample.
population_sampling_algo (str) –
algorithm to be used, currently implemented:
- ’real_population’:
Based on the real population
- ’monte_carlo’:
Randomly assign the number based on a MC draw.
Defaults to ‘real_population’.
- Returns
The number of samples from each value of the pdf.
- Raises
ValueError – if the
pdf
does not sum up to one.- Return type
numpy.ndarray
Units Conversion¶
Some units converters.
Demod uses SI units by default. (Watts for power and Joules for energy.)
-
demod.utils.converters.
joules_to_kwh
(x)¶ Transform joules to kwh.
1 kwh = 3’600’000 Joules
- Parameters
x (Any) – The object to convert. Must support division by float.
- Return type
Any
-
demod.utils.converters.
kwh_to_joules
(x)¶ Transform kwh to joules.
1 kwh = 3’600’000 Joules
- Parameters
x (Any) – The object to convert. Must support multiplication by float.
- Return type
Any