entropy¶
-
bcselector.information_theory.basic_approximations.
entropy
(vector, base=None)[source]¶ This estimator computes the entropy of the empirical probability distribution.
- Parameters
vector (list or np.array) – Vector of which entropy is calculated.
base (int or float (default=np.e)) – Base of the logarithm in entropy approximation
- Returns
vector_entropy – Approximated entropy
- Return type
float
Examples
>>> from bcselector.information_theory.basic_approximations import entropy >>> foo = [1,4,1,2,5,6,3] >>> entropy(foo)
conditional_entropy¶
-
bcselector.information_theory.basic_approximations.
conditional_entropy
(vector, condition, base=None)[source]¶ This estimator computes the conditional entropy of the empirical probability distribution.
- Parameters
vector (list or np.array) – Vector of which entropy is calculated.
condition (list or np.array) – Vector of condition for entropy.
base (int or float) – Base of the logarithm in entropy approximation. If None, np.e is selected and entropy is returned in nats.
- Returns
vector_entropy – Approximated entropy.
- Return type
float
mutual_information¶
-
bcselector.information_theory.basic_approximations.
mutual_information
(vector_1, vector_2, base=None)[source]¶ This estimator computes the mutual information of two vectors with method of the empirical probability distribution.
- Parameters
vector_1 (list or np.array) – Vector of one variable.
vector_2 (list or np.array) – Vector of one variable.
base (int or float) – Base of the logarithm in entropy approximation. If None, np.e is selected and entropy is returned in nats.
- Returns
variables_mutual_information – Approximated mutual information between variables.
- Return type
float
conditional_mutual_information¶
-
bcselector.information_theory.basic_approximations.
conditional_mutual_information
(vector_1, vector_2, condition, base=None)[source]¶ This estimator computes the conditional mutual information of two vectors and condition vector with method of the empirical probability distribution.
- Parameters
vector_1 (list or np.array) – Vector of one variable.
vector_2 (list or np.array) – Vector of one variable.
condition (list or np.array) – Vector of condition for mutual information.
base (int or float) – Base of the logarithm in entropy approximation. If None, np.e is selected and entropy is returned in nats.
- Returns
variables_conditional_mutual_information – Approximated conditional mutual information between variables.
- Return type
float