Entropy

250 KiB

Open AccessArticle

Holographic Dark Information Energy: Predicted Dark Energy Measurement

by Michael Paul Gough

Entropy 2013, 15(3), 1135-1151; https://doi.org/10.3390/e15031135 - 22 Mar 2013

Cited by 5 | Viewed by 6158

Several models have been proposed to explain the dark energy that is causing universe expansion to accelerate. Here the acceleration predicted by the Holographic Dark Information Energy (HDIE) model is compared to the acceleration that would be produced by a cosmological constant. While [...] Read more.

Several models have been proposed to explain the dark energy that is causing universe expansion to accelerate. Here the acceleration predicted by the Holographic Dark Information Energy (HDIE) model is compared to the acceleration that would be produced by a cosmological constant. While identical to a cosmological constant at low redshifts, z < 1, the HDIE model results in smaller Hubble parameter values at higher redshifts, z > 1, reaching a maximum difference of 2.6 ± 0.5% around z ~ 1.7. The next generation of dark energy measurements, both those scheduled to be made in space (ESA’s Euclid and NASA’s WFIRST missions) and those to be made on the ground (BigBOSS, LSST and Dark Energy Survey), should be capable of determining whether such a difference exists or not. In addition a computer simulation thought experiment is used to show that the algorithmic entropy of the universe always increases because the extra states produced by the accelerating expansion compensate for the loss of entropy from star formation. Full article

► Show Figures

Figure 1

1228 KiB

Open AccessArticle

Evidence of Large-Scale Quantization in Space Plasmas

by George Livadiotis and David J. McComas

Entropy 2013, 15(3), 1118-1134; https://doi.org/10.3390/e15031118 - 22 Mar 2013

Cited by 47 | Viewed by 7915

Abstract

In plasmas, Debye screening structures the possible correlations between particles. We identify a phase space minimum h_* in non-equilibrium space plasmas that connects the energy of particles in a Debye sphere to an equivalent wave frequency. In particular, while there is no [...] Read more.

In plasmas, Debye screening structures the possible correlations between particles. We identify a phase space minimum h_* in non-equilibrium space plasmas that connects the energy of particles in a Debye sphere to an equivalent wave frequency. In particular, while there is no a priori reason to expect a single value of h_* across plasmas, we find a very similar value of h_* ≈ (7.5 ± 2.4)×10⁻²²J·s using four independent methods: (1) Ulysses solar wind measurements, (2) space plasmas that typically reside in stationary states out of thermal equilibrium and spanning a broad range of physical properties, (3) an entropic limit emerging from statistical mechanics, (4) waiting-time distributions of explosive events in space plasmas. Finding a quasi-constant value for the phase space minimum in a variety of different plasmas, similar to the classical Planck constant but 12 orders of magnitude larger may be revealing a new type of quantization in many plasmas and correlated systems more generally. Full article

► Show Figures

Figure 1

276 KiB

Open AccessArticle

A Maximum Entropy Approach to Loss Distribution Analysis

by Marco Bee

Entropy 2013, 15(3), 1100-1117; https://doi.org/10.3390/e15031100 - 22 Mar 2013

Cited by 6 | Viewed by 5776

Abstract

In this paper we propose an approach to the estimation and simulation of loss distributions based on Maximum Entropy (ME), a non-parametric technique that maximizes the Shannon entropy of the data under moment constraints. Special cases of the ME density correspond to standard [...] Read more.

In this paper we propose an approach to the estimation and simulation of loss distributions based on Maximum Entropy (ME), a non-parametric technique that maximizes the Shannon entropy of the data under moment constraints. Special cases of the ME density correspond to standard distributions; therefore, this methodology is very general as it nests most classical parametric approaches. Sampling the ME distribution is essential in many contexts, such as loss models constructed via compound distributions. Given the difficulties in carrying out exact simulation,we propose an innovative algorithm, obtained by means of an extension of Adaptive Importance Sampling (AIS), for the approximate simulation of the ME distribution. Several numerical experiments confirm that the AIS-based simulation technique works well, and an application to insurance data gives further insights in the usefulness of the method for modelling, estimating and simulating loss distributions. Full article

► Show Figures

Figure 1

2378 KiB

Open AccessArticle

Substrate Effect on Catalytic Loop and Global Dynamics of Triosephosphate Isomerase

by Zeynep Kurkcuoglu and Pemra Doruker

Entropy 2013, 15(3), 1085-1099; https://doi.org/10.3390/e15031085 - 18 Mar 2013

Cited by 4 | Viewed by 6054

Abstract

The opening/closure of the catalytic loop 6 over the active site in apo triosephosphate isomerase (TIM) has been previously shown to be driven by the global motions of the enzyme, specifically the counter-clockwise rotation of the subunits. In this work, the effect of [...] Read more.

The opening/closure of the catalytic loop 6 over the active site in apo triosephosphate isomerase (TIM) has been previously shown to be driven by the global motions of the enzyme, specifically the counter-clockwise rotation of the subunits. In this work, the effect of the substrate dihydroxyacetone phosphate (DHAP) on TIM dynamics is assessed using two apo and two DHAP-bound molecular dynamics (MD) trajectories (each 60 ns long). Multiple events of catalytic loop opening/closure take place during 60 ns runs for both apo TIM and its DHAP-complex. However, counter-clockwise rotation observed in apo TIM is suppressed and bending-type motions are linked to loop dynamics in the presence of DHAP. Bound DHAP molecules also reduce the overall mobility of the enzyme and change the pattern of orientational cross-correlations, mostly those within each subunit. The fluctuations of pseudodihedral angles of the loop 6 residues are enhanced towards the C-terminus, when DHAP is bound at the active site. Full article

(This article belongs to the Special Issue Loop Entropy)

► Show Figures

Graphical abstract

579 KiB

Open AccessArticle

Time Series Analysis Using Composite Multiscale Entropy

by Shuen-De Wu, Chiu-Wen Wu, Shiou-Gwo Lin, Chun-Chieh Wang and Kung-Yen Lee

Entropy 2013, 15(3), 1069-1084; https://doi.org/10.3390/e15031069 - 18 Mar 2013

Cited by 279 | Viewed by 16487

Abstract

Multiscale entropy (MSE) was recently developed to evaluate the complexity of time series over different time scales. Although the MSE algorithm has been successfully applied in a number of different fields, it encounters a problem in that the statistical reliability of the sample [...] Read more.

Multiscale entropy (MSE) was recently developed to evaluate the complexity of time series over different time scales. Although the MSE algorithm has been successfully applied in a number of different fields, it encounters a problem in that the statistical reliability of the sample entropy (SampEn) of a coarse-grained series is reduced as a time scale factor is increased. Therefore, in this paper, the concept of a composite multiscale entropy (CMSE) is introduced to overcome this difficulty. Simulation results on both white noise and 1/f noise show that the CMSE provides higher entropy reliablity than the MSE approach for large time scale factors. On real data analysis, both the MSE and CMSE are applied to extract features from fault bearing vibration signals. Experimental results demonstrate that the proposed CMSE-based feature extractor provides higher separability than the MSE-based feature extractor. Full article

► Show Figures

Graphical abstract

229 KiB

Open AccessArticle

Hawking and Unruh Effects of a 5-Dimensional Minimal Gauged Supergravity Black Hole by a Global Embedding Approach

by Hui-Hua Zhao, Li-Chun Zhang and Guang-Liang Li

Entropy 2013, 15(3), 1057-1068; https://doi.org/10.3390/e15031057 - 18 Mar 2013

Viewed by 5063

Abstract

Using the new global embedding approach we investigate Unruh/Hawking temperature of the 5-dimensional minimal gauged supergravity black hole with double rotating parameters in a general (1 + 1) space-time. Our results verify that views of Banerjee and Majhi, and extend this approach to [...] Read more.

Using the new global embedding approach we investigate Unruh/Hawking temperature of the 5-dimensional minimal gauged supergravity black hole with double rotating parameters in a general (1 + 1) space-time. Our results verify that views of Banerjee and Majhi, and extend this approach to a higher dimension situation. Full article

152 KiB

Open AccessArticle

Entropy Principle and Galilean Relativity for Dense Gases, the General Solution without Approximations

by Maria Cristina Carrisi, Stefania Montisci and Sebastiano Pennisi

Entropy 2013, 15(3), 1035-1056; https://doi.org/10.3390/e15031035 - 11 Mar 2013

Cited by 10 | Viewed by 4277

Abstract

The many moments model for dense gases and macromolecular fluids is considered here, where the upper order moment is chosen in accordance to the suggestions of the non-relativistic limit of the corresponding relativistic model. The solutions of the restrictions imposed by the entropy [...] Read more.

The many moments model for dense gases and macromolecular fluids is considered here, where the upper order moment is chosen in accordance to the suggestions of the non-relativistic limit of the corresponding relativistic model. The solutions of the restrictions imposed by the entropy principle and that of Galilean relativity were, until now, obtained in the literature by using Taylor expansions around equilibrium and without proving convergence. Here, an exact solution without using expansions is found. The particular case with only 14 moments has already been treated in the literature in a completely different way. Here, it is proven that this particular closure is included in the presently more general one. Full article

731 KiB

Open AccessArticle

Exergetic and Parametric Study of a Solar Aided Coal-Fired Power Plant

by Rongrong Zhai, Yong Zhu, Yongping Yang, Kaiyu Tan and Eric Hu

Entropy 2013, 15(3), 1014-1034; https://doi.org/10.3390/e15031014 - 11 Mar 2013

Cited by 33 | Viewed by 7945

Abstract

A solar-aided coal-fired power plant realizes the integration of a fossil fuel (coal or gas) and clean energy (solar). In this paper, a conventional 600 MW coal-fired power plant and a 600 MW solar-aided coal-fired power plant have been taken as the study [...] Read more.

A solar-aided coal-fired power plant realizes the integration of a fossil fuel (coal or gas) and clean energy (solar). In this paper, a conventional 600 MW coal-fired power plant and a 600 MW solar-aided coal-fired power plant have been taken as the study case to understand the merits of solar-aided power generation (SAPG) technology. The plants in the case study have been analyzed by using the First and Second Laws of Thermodynamics principles. The solar irradiation and load ratio have been considered in the analysis. We conclude that if the solar irradiation was 925 W/m² and load ratio of the SAPG plant was 100%, the exergy efficiency would be 44.54% and the energy efficiency of the plant (46.35%). It was found that in the SAPG plant the largest exergy loss was from the boiler, which accounted for about 76.74% of the total loss. When the load ratio of the unit remains at 100%, and the solar irradiation varies from 500 W/m² to 1,100 W/m², the coal savings would be in the range of 8.6 g/kWh to 15.8 g/kWh. If the solar irradiation were kept at 925 W/m² while the load ratio of the plant changed from 30% to 100%, the coal savings could be in the range of 11.99 g/kWh to 13.75 g/kWh. Full article

(This article belongs to the Special Issue Exergy: Analysis and Applications)

► Show Figures

Figure 1

5973 KiB

Open AccessArticle

New Climatic Indicators for Improving Urban Sprawl: A Case Study of Tehran City

by Abdolazim Ghanghermeh, Gholamreza Roshan, José A. Orosa, José L. Calvo-Rolle and Ángel M. Costa

Entropy 2013, 15(3), 999-1013; https://doi.org/10.3390/e15030999 - 7 Mar 2013

Cited by 37 | Viewed by 7088

Abstract

In the modern world, the fine balance and delicate relationship between human society and the environment in which we exist has been affected by the phenomena of urbanisation and urban development. Today, various environmental factors give rise to horizontal dispersion, spread and growth [...] Read more.

In the modern world, the fine balance and delicate relationship between human society and the environment in which we exist has been affected by the phenomena of urbanisation and urban development. Today, various environmental factors give rise to horizontal dispersion, spread and growth of cities. One of the most important results of this is climatic change which is directly affected by the urban sprawl of every metropolis. The aim of this study is to identify the relationship between the various horizontally distributed components of Tehran city and changes in essential microclimate clusters, by means of the humidex index. Results showed that, when the humidex was calculated for each of the obtained clusters, it was evident that it had increased with time, in parallel with Shannon’s entropy, as a consequence of the average temperature and relative humidity of each cluster. At the same time, results have shown that both temperature and relative humidity of the study area are related with urban sprawl, urbanisation and development, as defined by Shannon’s entropy and, in consequence, with humidex. In consequence, this new concept must be considered in future research works to predict and control urban sprawl and microclimate conditions in cities. Full article

(This article belongs to the Special Issue Entropy and Urban Sprawl)

► Show Figures

Figure 1

540 KiB

Open AccessArticle

Experimental Assessment of a 2-D Entropy-Based Model for Velocity Distribution in Open Channel Flow

by Nicola Fontana, Gustavo Marini and Francesco De Paola

Entropy 2013, 15(3), 988-998; https://doi.org/10.3390/e15030988 - 6 Mar 2013

Cited by 22 | Viewed by 6601

Abstract

Velocity distribution in an open channel flow can be very useful to model many hydraulic phenomena. Among the others, several 1D models based on the concept of entropy are available in the literature, which allow estimating the velocity distribution by measuring velocities only [...] Read more.

Velocity distribution in an open channel flow can be very useful to model many hydraulic phenomena. Among the others, several 1D models based on the concept of entropy are available in the literature, which allow estimating the velocity distribution by measuring velocities only in a few points. Nevertheless, since 1D models have often a limited practical use, a 2D entropy based model was recently developed. The model provides a reliable estimation of the velocity distribution for open channel flow with a rectangular cross section, if the maximum velocity and the average velocity are known. In this paper results from the proposed model were compared with measured velocities carried out from laboratory experiments. Calculated values were also compared with results inferred from a 2D model available in the literature, resulting in a greater ease of use and a more reliable estimate of the velocity profile. Full article

(This article belongs to the Special Issue Maximum Entropy and Bayes Theorem)

► Show Figures

Figure 1

245 KiB

Open AccessArticle

Information Properties of Boundary Line Models for N₂O Emissions from Agricultural Soils

by Cairistiona F.E. Topp, Weijin Wang, Joanna M. Cloy, Robert M. Rees and Gareth Hughes

Entropy 2013, 15(3), 972-987; https://doi.org/10.3390/e15030972 - 5 Mar 2013

Cited by 9 | Viewed by 6098

Abstract

Boundary line models for N₂O emissions from agricultural soils provide a means of estimating emissions within defined ranges. Boundary line models partition a two-dimensional region of parameter space into sub-regions by means of thresholds based on relationships between N₂O [...] Read more.

Boundary line models for N₂O emissions from agricultural soils provide a means of estimating emissions within defined ranges. Boundary line models partition a two-dimensional region of parameter space into sub-regions by means of thresholds based on relationships between N₂O emissions and explanatory variables, typically using soil data available from laboratory or field studies. Such models are intermediate in complexity between the use of IPCC emission factors and complex process-based models. Model calibration involves characterizing the extent to which observed data are correctly forecast. Writing the numerical results from graphical two-threshold boundary line models as 3×3 prediction-realization tables facilitates calculation of expected mutual information, a measure of the amount of information about the observations contained in the forecasts. Whereas mutual information characterizes the performance of a forecaster averaged over all forecast categories, specific information and relative entropy both characterize aspects of the amount of information contained in particular forecasts. We calculate and interpret these information quantities for experimental N₂O emissions data. Full article

(This article belongs to the Special Issue Applications of Information Theory in the Geosciences)

► Show Figures

Figure 1

243 KiB

Open AccessArticle

On Classical Ideal Gases

by Jacques Arnaud, Laurent Chusseau and Fabrice Philippe

Entropy 2013, 15(3), 960-971; https://doi.org/10.3390/e15030960 - 27 Feb 2013

Cited by 3 | Viewed by 7082

Abstract

We show that the thermodynamics of ideal gases may be derived solely from the Democritean concept of corpuscles moving in vacuum plus a principle of simplicity, namely that these laws are independent of the laws of motion, aside from the law of energy [...] Read more.

We show that the thermodynamics of ideal gases may be derived solely from the Democritean concept of corpuscles moving in vacuum plus a principle of simplicity, namely that these laws are independent of the laws of motion, aside from the law of energy conservation. Only a single corpuscle in contact with a heat bath submitted to a z and t-invariant force is considered. Most of the end results are known but the method appears to be novel. The mathematics being elementary, the present paper should facilitate the understanding of the ideal gas law and of classical thermodynamics even though not-usually-taught concepts are being introduced. Full article

► Show Figures

Figure 1

284 KiB

Open AccessArticle

Minimum-Information-Entropy-Based Control Performance Assessment

by Qing-Wei Meng, Fang Fang and Ji-Zhen Liu

Entropy 2013, 15(3), 943-959; https://doi.org/10.3390/e15030943 - 27 Feb 2013

Cited by 13 | Viewed by 5335

Abstract

Generally, the controller design should be performed to narrow the shape of the probability density function of the tracking error. A small information entropy value corresponds to a narrow distribution function, which means that the uncertainty of the related random variable is small. [...] Read more.

Generally, the controller design should be performed to narrow the shape of the probability density function of the tracking error. A small information entropy value corresponds to a narrow distribution function, which means that the uncertainty of the related random variable is small. In this paper, information entropy is introduced in the field of control performance assessment (CPA). For the unknown time delay case, the minimum information entropy (MIE) benchmark is presented, and a MIE-based performance index is defined. For the known time delay case, a tight upper bound of MIE is derived and adopted as a performance benchmark to assess the stochastic control performance. Based on these, the control performance assessment procedures are developed for both the steady and the transient processes. Simulation tests and an industrial case study of a main steam pressure system of a 1,000MW power unit are utilized to verify the effectiveness of the proposed procedures. Full article

► Show Figures

Figure 1

1336 KiB

Open AccessArticle

Mutual Information-Based Inputs Selection for Electric Load Time Series Forecasting

by Miloš Božić, Miloš Stojanović, Zoran Stajić and Nenad Floranović

Entropy 2013, 15(3), 926-942; https://doi.org/10.3390/e15030926 - 27 Feb 2013

Cited by 23 | Viewed by 5709

Abstract

Providing accurate load forecast to electric utility corporations is essential in order to reduce their operational costs and increase profits. Hence, training set selection is an important preprocessing step which has to be considered in practice in order to increase the accuracy of [...] Read more.

Providing accurate load forecast to electric utility corporations is essential in order to reduce their operational costs and increase profits. Hence, training set selection is an important preprocessing step which has to be considered in practice in order to increase the accuracy of load forecasts. The usage of mutual information (MI) has been recently proposed in regression tasks, mostly for feature selection and for identifying the real instances from training sets that contains noise and outliers. This paper proposes a methodology for the training set selection in a least squares support vector machines (LS-SVMs) load forecasting model. A new application of the concept of MI is presented for the selection of a training set based on MI computation between initial training set instances and testing set instances. Accordingly, several LS-SVMs models have been trained, based on the proposed methodology, for hourly prediction of electric load for one day ahead. The results obtained from a real-world data set indicate that the proposed method increases the accuracy of load forecasting as well as reduces the size of the initial training set needed for model training. Full article

► Show Figures

Figure 1

592 KiB

Open AccessReview

Quantum Models of Classical World

by Petr Hájíček

Entropy 2013, 15(3), 789-925; https://doi.org/10.3390/e15030789 - 27 Feb 2013

Cited by 2 | Viewed by 5221

Abstract

This paper is a review of our recent work on three notorious problems of non-relativistic quantum mechanics: realist interpretation, quantum theory of classical properties, and the problem of quantum measurement. A considerable progress has been achieved, based on four distinct new ideas. First, [...] Read more.

This paper is a review of our recent work on three notorious problems of non-relativistic quantum mechanics: realist interpretation, quantum theory of classical properties, and the problem of quantum measurement. A considerable progress has been achieved, based on four distinct new ideas. First, objective properties are associated with states rather than with values of observables. Second, all classical properties are selected properties of certain high entropy quantum states of macroscopic systems. Third, registration of a quantum system is strongly disturbed by systems of the same type in the environment. Fourth, detectors must be distinguished from ancillas and the states of registered systems are partially dissipated and lost in the detectors. The paper has two aims: a clear explanation of all new results and a coherent and contradiction-free account of the whole quantum mechanics including all necessary changes of its current textbook version. Full article

(This article belongs to the Special Issue Maximum Entropy and Bayes Theorem)

1163 KiB

Open AccessArticle

Transfer Entropy for Coupled Autoregressive Processes

by Daniel W. Hahs and Shawn D. Pethel

Entropy 2013, 15(3), 767-788; https://doi.org/10.3390/e15030767 - 25 Feb 2013

Cited by 25 | Viewed by 8833

Abstract

A method is shown for computing transfer entropy over multiple time lags for coupled autoregressive processes using formulas for the differential entropy of multivariate Gaussian processes. Two examples are provided: (1) a first-order filtered noise process whose state is measured with additive noise, [...] Read more.

A method is shown for computing transfer entropy over multiple time lags for coupled autoregressive processes using formulas for the differential entropy of multivariate Gaussian processes. Two examples are provided: (1) a first-order filtered noise process whose state is measured with additive noise, and (2) two first-order coupled processes each of which is driven by white process noise. We found that, for the first example, increasing the first-order AR coefficient while keeping the correlation coefficient between filtered and measured process fixed, transfer entropy increased since the entropy of the measured process was itself increased. For the second example, the minimum correlation coefficient occurs when the process noise variances match. It was seen that matching of these variances results in minimum information flow, expressed as the sum of transfer entropies in both directions. Without a match, the transfer entropy is larger in the direction away from the process having the larger process noise. Fixing the process noise variances, transfer entropies in both directions increase with the coupling strength. Finally, we note that the method can be generally employed to compute other information theoretic quantities as well. Full article

(This article belongs to the Special Issue Transfer Entropy)

► Show Figures

Figure 1

177 KiB

Open AccessArticle

Statistical Analysis of Gait Maturation in Children Using Nonparametric Probability Density Function Modeling

by Ning Xiang, Suxian Cai, Shanshan Yang, Zhangting Zhong, Fang Zheng, Jia He and Yunfeng Wu

Entropy 2013, 15(3), 753-766; https://doi.org/10.3390/e15030753 - 25 Feb 2013

Cited by 11 | Viewed by 8104

Abstract

Analysis of gait dynamics in children may help understand the development of neuromuscular control and maturation of locomotor function. This paper applied the nonparametric Parzen-window estimation method to establish the probability density function (PDF) models for the stride interval time series of 50 [...] Read more.

Analysis of gait dynamics in children may help understand the development of neuromuscular control and maturation of locomotor function. This paper applied the nonparametric Parzen-window estimation method to establish the probability density function (PDF) models for the stride interval time series of 50 children (25 boys and 25 girls). Four statistical parameters, in terms of averaged stride interval (ASI), variation of stride interval (VSI), PDF skewness (SK), and PDF kurtosis (KU), were computed with the Parzen-window PDFs to study the maturation of stride interval in children. By analyzing the results of the children in three age groups (aged 3–5 years, 6–8 years, and 10–14 years), we summarize the key findings of the present study as follows. (1) The gait cycle duration, in terms of ASI, increases until 14 years of age. On the other hand, the gait variability, in terms of VSI, decreases rapidly until 8 years of age, and then continues to decrease at a slower rate. (2) The SK values of both the histograms and Parzen-window PDFs for all of the three age groups are positive, which indicates an imbalance in the stride interval distribution within an age group. However, such an imbalance would be meliorated when the children grow up. (3) The KU values of both the histograms and Parzen-window PDFs decrease with the body growth in children, which suggests that the musculoskeletal growth enables the children to modulate a gait cadence with ease. (4) The SK and KU results also demonstrate the superiority of the Parzen-window PDF estimation method to the Gaussian distribution modeling, for the study of gait maturation in children. Full article

► Show Figures

Figure 1

637 KiB

Open AccessArticle

Minimum Mutual Information and Non-Gaussianity through the Maximum Entropy Method: Estimation from Finite Samples

by Carlos A. L. Pires and Rui A. P. Perdigão

Entropy 2013, 15(3), 721-752; https://doi.org/10.3390/e15030721 - 25 Feb 2013

Cited by 8 | Viewed by 7037

Abstract

The Minimum Mutual Information (MinMI) Principle provides the least committed, maximum-joint-entropy (ME) inferential law that is compatible with prescribed marginal distributions and empirical cross constraints. Here, we estimate MI bounds (the MinMI values) generated by constraining sets T_cr comprehended by m_cr [...] Read more.

The Minimum Mutual Information (MinMI) Principle provides the least committed, maximum-joint-entropy (ME) inferential law that is compatible with prescribed marginal distributions and empirical cross constraints. Here, we estimate MI bounds (the MinMI values) generated by constraining sets T_cr comprehended by m_cr linear and/or nonlinear joint expectations, computed from samples of N iid outcomes. Marginals (and their entropy) are imposed by single morphisms of the original random variables. N-asymptotic formulas are given both for the distribution of cross expectation’s estimation errors, the MinMI estimation bias, its variance and distribution. A growing T_cr leads to an increasing MinMI, converging eventually to the total MI. Under N-sized samples, the MinMI increment relative to two encapsulated sets T_cr1 ⊂ T_cr2 (with numbers of constraints mcr1<mcr2 ) is the test-difference δH = H_{max 1, N} - H_{max 2, N} ≥ 0 between the two respective estimated MEs. Asymptotically, δH follows a Chi-Squared distribution ¹/_2NΧ2 (m_cr2-m_cr1) whose upper quantiles determine if constraints in T_cr2/T_cr1 explain significant extra MI. As an example, we have set marginals to being normally distributed (Gaussian) and have built a sequence of MI bounds, associated to successive non-linear correlations due to joint non-Gaussianity. Noting that in real-world situations available sample sizes can be rather low, the relationship between MinMI bias, probability density over-fitting and outliers is put in evidence for under-sampled data. Full article

(This article belongs to the Special Issue Estimating Information-Theoretic Quantities from Data)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Entropy, Volume 15, Issue 3 (March 2013) – 18 articles , Pages 721-1151

Further Information

Guidelines

MDPI Initiatives

Follow MDPI