Power-Law Distributions from Sigma-Pi Structure of Sums of Random Multiplicative Processes

Abstract

We introduce a simple growth model in which the sizes of entities evolve as multiplicative random processes that start at different times. A novel aspect we examine is the dependence among entities. For this, we consider three classes of dependence between growth factors governing the evolution of sizes: independence, Kesten dependence and mixed dependence. We take the sum X of the sizes of the entities as the representative quantity of the system, which has the structure of a sum of product terms (Sigma-Pi), whose asymptotic distribution function has a power-law tail behavior. We present evidence that the dependence type does not alter the asymptotic power-law tail behavior, nor the value of the tail exponent. However, the structure of the large values of the sum X is found to vary with the dependence between the growth factors (and thus the entities). In particular, for the independence case, we find that the large values of X are contributed by a single maximum size entity: the asymptotic power-law tail is the result of such single contribution to the sum, with this maximum contributing entity changing stochastically with time and with realizations.

1. Introduction

Power-laws appear in the probability distributions of many physical systems and human activities, being one of the signatures of complex systems [1,2]. Usually occurring in its tail, i.e., for sufficiently large events, a probability density function $f\left(x\right)$ of a random variable x with this behavior is such that:

$$f\left(x\right)\sim x^{-\alpha -1},\mathrm{for}x\mathrm{large}.$$

(1)

A probability distribution with a power-law tail is characterized by a much slower decay compared with an exponential law, and thus large events may not be negligible. Another distinctive and unique property of power-laws is that they obey the symmetry of scale invariance: the functional form of the distribution remains unchanged under a scale transformation. In addition, for these distributions, the moments of order larger than or equal to $\alpha$ diverge [3,4].

The frequent observation of power-law distributions in natural and social systems as well as their interesting mathematical properties have motivated the search for mechanisms that generate power-laws [3,5,6]. One of the first proposed power-law generation mechanisms, which is recurrently applied to model a diversity of phenomena, is “proportional growth”, also known in network theory as “preferential attachment”. In this mechanism, a quantity is distributed among existing entities according to probabilities proportional to the quantity they already possess, establishing a “rich get richer” dynamics [7,8,9]. Another iconic conceptual framework is self-organized criticality, captured by the metaphor of avalanches in a sandpile, in which power-laws are spontaneous emergent properties, with no apparent need to adjust control parameters [10,11]. Among many other mechanisms, let us mention the superposition of probability distributions [12,13], formalized in the concept of superstatistics [14,15,16]; the key idea is that the basic non-power-law probability distribution of a system has parameters that are random variables themselves and, when including this second source of stochasticity, a power-law distribution may arise. A particular case of superstatistics that found wide applicability is the Tsallis statistics, where the extremization of its generalized entropy produces power-laws [17].

An additional relevant model for generating power-law distributions is the class of random multiplicative processes, often used to model growth of entities, such as business firms, cities, and biological populations [18,19,20,21,22,23,24,25]. The common starting point to explain the mechanisms of power-law formation in those growth phenomena is the Gibrat’s law of proportional growth, stating that an entity grows proportionally to its current size but with a growth rate independent of it [26]. This law leads to the basic random multiplicative process expressed in discrete times:

$$s\left(t\right)=s(t-1)\xi \left(t\right),$$

(2)

where $s\left(t\right)$ is the size of entity at time t, $\xi \left(t\right)$ represents the multiplicative factor and the discrete time growth rate is equal to $\xi \left(t\right)-1=\frac{s\left(t\right)-s(t-1)}{s(t-1)}$.

However, the Gibrat’s law alone cannot explain a power-law distribution since the corresponding process (2) produces a non-stationary log-normal distribution, considering $\xi \left(t\right)$ independent and identically distributed random variables with finite mean and variance. To obtain a power-law tail, it is necessary to assure a contraction tendency and add some kind of surviving mechanism to the basic random multiplicative process. The simplest manifestation of such mechanisms is the introduction of an additive term to the process, leading to stochastic multiplicative affine processes, also often referred to as the Kesten process [27,28,29,30,31]:

$$s\left(t\right)=s(t-1)\xi \left(t\right)+\eta \left(t\right),$$

(3)

with $\eta \left(t\right)$ being the additive term. Another related surviving mechanism is the inclusion of the additive term only when the system crosses a lower threshold, acting as a barrier preventing the collapse to zero [32]. Under general conditions, when those ingredients are added to the basic random multiplicative process, a power-law tailed distribution is obtained.

Here, we propose a simple growth model that combines basic random multiplicative processes with the idea of superposition of distributions mentioned above. The superposition of distributions has a neat interpretation: it corresponds to considering entities that are born successively. Combining the features of birth and stochastic proportional growth has been considered in [23,33,34]. Our model can be viewed as the simplest and purest incarnation of these mechanisms. This allows us to examine dependence among entities in three particular different classes: complete independence; Kesten dependence (a dependence based on the Kesten process); and mixed dependence, combining both independence and Kesten dependence. Rather than studying the cross-section at a given time of the sizes of entities present in the system (as done, e.g., in [23,33,34]), we study the distribution of the sum of sizes of all entities in the system. This corresponds to the total capitalization of a country, when entities are firms, or to the total biomass of an ecosystem for biological populations. We obtain a random variable whose structure takes the form of a sum of product terms. As a result of this structure, its distribution has a power-law tail with the same exponent regardless of the dependence type—indicating a common general ingredient for power-laws—but with distinct tail formation mechanisms resulting from the degree of dependence among entities.

2. Model Description and Formal Solution

2.1. Model Definition

Consider a set of entities of sizes $s_j$. At each time step, the sizes $s_j$ of existing entities evolve as basic random multiplicative processes and one new entity is born with unit size. The size $s_j\left(t\right)$ of a given entity j at time t is defined as:

$$\begin{array}{c}\hfill s_j\left(t\right)=\left\{\begin{array}{c}0;t<j-1,\hfill \\ 1;t=j-1,\hfill \\ s_j(t-1){\xi }_j\left(t\right);t\ge j.\hfill \end{array}\right.\end{array}$$

(4)

By definition, the time of birth of a given entity is its index j minus 1. Thus, the first entity with index $j=1$ is born (of unit size) at time $t=0$. The factors ${\xi }_j\left(t\right)$ are identically distributed random variables. Here, we take positive half-normally distributed random variables with variance ${\sigma }^2$, i.e., ${\xi }_j\left(t\right)=\left|{\xi }^{\prime }\right|$, with ${\xi }^{\prime }$ normally distributed: ${\xi }^{\prime }\sim N(0,{\sigma }^2)$.

The quantity that we study is the sum $X\left(t\right)$ of the sizes $\{s_j\left(t\right),j=1,...,t\}$ of all existing entities present in the system at time t, excluding the last entity born at t that has index $t+1$ and size $s_{t+1}\left(t\right)=1$:

$$X\left(t\right)=\sum _{j=1}^t{s}_j\left(t\right).$$

(5)

By recursion, the size $s_j\left(t\right)$ of entity j at time t is given by:

$$s_j(t\ge j)=\prod _{k=j}^t{\xi }_j\left(k\right),\mathrm{with}\mathrm{initial}\mathrm{value}{s}_j(j-1)=1.$$

(6)

Then, the sum of entity sizes reads:

$$X\left(t\right)=\sum _{j=1}^t\prod _{k=j}^t{\xi }_j\left(k\right).$$

(7)

The first sum in the right side of expression (7) is over all the t entities that have been born until time t (excluding the last one, as mentioned above). For a given entity j, the product term is made of $t-j+1$ growth factors corresponding to the number of time periods of stochastic proportional growth allowed by its age.

2.2. Dependence Structure of the Growth Factors

The entities are not necessarily independent. External factors can influence more than one entity at the same time, resulting in dependence between the growth rates for different entities. For instance, there could exist situations in which $j\ne j^{\prime }$ such that ${\xi }_j\left(t\right)={\xi }_{j^{\prime }}\left(t\right)$. In this work, from the numerous possible dependence types, we examine the following three particular ones:

• Independence: all entities are growing independently:

  $${\xi }_j\left(t\right)\ne {\xi }_{j^{\prime }}\left(t^{\prime }\right);\forall j\ne j^{\prime },t\ne t^{\prime }.$$

  (8)

  Observe that the notation ${\xi }_j\left(t\right)\ne {\xi }_{j^{\prime }}\left(t^{\prime }\right)$ does not mean that these variables cannot have by chance the same value. This notation just expresses that they are independent random variables.
• Kesten dependence: external influences determine the same growth factor for all existing entities at each given time, but the growth factor is a random variable as a function of time. This case reproduces the solution of the Kesten process (3) and constitutes a novel interpretation of the said process, originally representing a single entity evolving in the presence of an additive term:

  $$\begin{array}{c}\hfill \left\{\begin{array}{c}{\xi }_j\left(t\right)={\xi }_{j^{\prime }}\left(t\right)=\xi \left(t\right);\forall j,j^{\prime },\hfill \\ \xi \left(t\right)\ne \xi \left(t^{\prime }\right);\forall t\ne t^{\prime }.\hfill \end{array}\right.\end{array}$$

  (9)
• Mixed dependence: alternation between independence and Kesten dependence, say independence for odd t and Kesten dependence for even t, representing a time-changing dependence. Note that this is only one of the many possibilities for the combination of independence and Kesten dependence.

Regardless of the dependence type, the solution for the sum of sizes has the structure of a sum of increasing products of random variables, which we refer to as Sigma-Pi:

$$X_n=\sum _{j=1}^n\prod _{k=1}^j{\xi }_{jk}.$$

(10)

We have changed ${\prod }_{k=j}^n$ into ${\prod }_{k=1}^j$ (compare with (7)) as this gives the same number and structure of terms, when summed over j from 1 to n. As mentioned before, the identically distributed random variables ${\xi }_{jk}$ are not necessarily all independent and their interdependence characterizes the dependence type.

2.3. Asymptotic Power-Law Tails

This Sigma-Pi structure has received special attention in the context of the Kesten process. For Kesten dependence, letting $n\to \infty$, the central result [27] states that, provided that the multiplicative factors are contracting on average ($\langle log{\xi }_{jk}\rangle <0$, where the notation $\langle .\rangle$ stands for the expected operator and log, for the natural logarithm), ensuring stationarity, the distribution of X presents a power-law tail behavior, with the complementary cumulative distribution function of X, $F\left(X\right)$, behaving as:

$$F\left(X\right)\sim \frac{C}{X^{\alpha }},$$

(11)

where C is called the “scale factor” and the exponent $\alpha$ is determined by the relation (provided that there is a solution $\alpha >0$):

$$\langle {\xi }^{\alpha }\rangle =1.$$

(12)

Then, for half-normally distributed random variables, ${\xi }_{jk}\sim N_{\left|\right|}(0,{\sigma }^2)$, Equation (12) shows that $\alpha$ is uniquely determined by $\sigma$, where the explicit relationship is conveniently expressed by $\sigma$ as a function of $\alpha$ [35]:

$$\sigma ={\left[\frac{2^{\frac{\alpha }{2}}\Gamma \left(\frac{\alpha +1}{2}\right)}{\sqrt{\pi }}\right]}^{-\frac{1}{\alpha }}.$$

(13)

It has been shown rigorously that the power-law asymptotic result (11) with (12) also applies for the independence case [36] (this fact was first suggested by the authors, as acknowledged in [36]), up to a logarithmic correction so that $F\left(X\right)\sim \frac{logX}{X^{\alpha }}$. In fact, we conjecture that the result (11) with (12), up to slowly varying functions multiplying the scale factor C, holds for any dependence type with the restriction that all variables in a product term are independent (see Appendix A), or more generally with a sufficiently weak dependence. In other words, the sizes of the entities can be interdependent via the action of common simultaneous growth factors, but the growth factors themselves should exhibit no (or just a weak) dependence along the time dimension. We thus expect a power-law distribution with the same exponent for all three particular cases made explicit above: independence, Kesten dependence and mixed dependence. This conjecture is tested numerically in the next section.

2.4. Generalization

We end this section by commenting that the addition of more realistic ingredients to the model, such as random birth rates and the inclusion of stochastic deaths of the entities, can be represented by the generalized Sigma-Pi of the form:

$$X_n=\sum _{j=1}^n{c}_j\prod _{k=1}^j{\xi }_{jk},$$

(14)

where the coefficients $c_j$ depend on the details of the modified model. For instance, the death of some entity j that occurred at some time between its birth and the “present time” n amounts to take $c_j=0$. We indicate in Appendix A that, for a large class of coefficients $c_j$, result (11) with (12) still holds. Intuitively, this is because the behavior of $X_n$ given by (14) is controlled asymptotically by just a few terms in the sum, or by the single largest term only, depending on the dependence structure. Interpreted for a system, the total system size is thus controlled by few entities.

3. Numerical Simulations and Discussion

3.1. Numerical Construction of the Complementary Cumulative Distribution Function of Sum of Sizes

In this section, we check numerically the prediction (11) with (12) for the complementary cumulative distribution function $F\left(X\right)$ of the random variable $X\left(t\right)$ (7) or equivalently $X_n$ (10) in the limit of large t or n. Numerically, we have used two methods to generate the set of $X_n$ to construct $F\left(X\right)$: (i) recording the successive values of $X_n$ over time n and (ii) constructing thousands of independent ensembles of time evolutions of $X_n$ and recording all the values of this ensemble at a fixed time. We have checked (see Appendix B) that the two methods yield the same results, supporting the validity of the ergodicity property for the process $X\left(t\right)$.

To make manageable the numerical simulations of the model, we cannot keep an endlessly increasing number of entities entering the system and thus we add the rule of removing entities with age greater than n. Figure 1 shows $F\left(X\right)$ for various values of n in the independent case with half-normally distributed random variables ${\xi }_j\left(k\right)$ with $\sigma =1$, for which expression (13) gives $\alpha =2$. The value $\sigma =1$ is a natural first choice for this parameter since it refers to the standard normal distribution. In this case, one can observe that $F\left(X\right)$ does not change appreciably for $n\ge 50$, which suggests that it has converged already for $n=50$. This means that entities older than 50 time steps have sizes that do not contribute substantially to the sum $X\left(t\right)$. In the sequel, we use $n=2000$ to be conservative and to obtain reasonable approximations of the asymptotic limit $n\to \infty$.

Figure 2 presents $F\left(X\right)$ for the independence (black), Kesten dependence (red) and mixed dependence cases (blue), for $\sigma =0.8557$ (panel a), $\sigma =1$ (panel b) and $\sigma =1.2533$ (panel c). These three values of $\sigma$ corresponds, respectively, to $\alpha =3$, $\alpha =2$ and $\alpha =1$, as follows from relation (13). Values $\sigma \ne 1$ are chosen in order to analyze the power-law behavior when $\sigma <1$ and $\sigma >1$, in particular $\sigma =1.2533$ yielding $\alpha =1$, which corresponds to the Zipf’s law [23]. The tails of $F\left(X\right)$ are in good visual agreement with these predictions, while the non-asymptotic parts deviate substantially depending on the nature of the dependence. One can observe that the scale factor C, as defined in expression (11), is the smallest for the independence case and the largest for the Kesten dependence. The domain of validity of the power law tail (11) is also the smallest for the independence case and the largest for the Kesten dependence. The distribution $F\left(X\right)$ for the independence case exhibits the longest intermediate regime before the tail converges to its clean power-law asymptotics. This may be related to the logarithmic correction mentioned above [36].

3.2. Study of the Entities Contributing to the Sum of Sizes

Notwithstanding the similar asymptotic power-law tail behavior (up to a slowly varying function), a closer inspection of the model reveals that the three dependence cases present important and interesting differences in the way the extreme values of X ($X\left(t\right)$ or $X_n$) are realized. To unveil these differences, we calculate the Herfindahl index, a concentration indicator typically used in finance [37], which is also known as the participation ratio in the physics of spin-glasses [38]. Starting from the size $s_j\left(t\right)$ of an individual entity contributing to the sum $X\left(t\right)$ of sizes, one defines the normalized contribution $s_j\left(t\right)/X\left(t\right)$, which is the fraction of the total system size contributed by entity j. Then, the Herfindahl index reads:

$$H\left(t\right)=\sum _{j=1}^t{\left(\frac{s_j\left(t\right)}{X\left(t\right)}\right)}^2.$$

(15)

The inverse $H^{-1}$ of the Herfindahl index is a measure of the number of entities that significantly contribute to the sum X of sizes. In particular, it is straightforward to verify that $H^{-1}=1$ if one entity has size X and all the others have vanishing sizes and that $H^{-1}=t$ (i.e., equal to the total number of entities in the system) if all entities have identical size equal to $X/t$. The inverse Herfindahl index $H^{-1}$ thus varies between 1 and t, i.e., from a singular to a full “democratic” contribution to the sum X.

Figure 3 shows scatter plots of $H^{-1}$ as a function of X obtained in the model evolution for all three dependence cases and the three values of $\sigma$ used previously. In other words, for each simulation resulting in some $X\left(t\right)$, we use the known values of all entity sizes to form the corresponding inverse Herfindahl index. Each row of three panels corresponds to a given dependence model, while each column corresponds to a fixed $\sigma$.

Let us focus first on the second row corresponding to the Kesten dependence case (panels b). The most striking observation is a change of regime. The first regime holds from small to intermediate values of X for which the number of contributing entities can take any value from 1 to a value increasing approximately logarithmically with X. In panel (b-1) for $\sigma =0.8557$, this regime holds up to $X\approx 5$. For a larger X, one can observe that the minimum of $H^{-1}$ takes off from the line $H^{-1}=1$ and there is a minimum $H^{-1}$ strictly larger than 1 necessary to obtain the largest X values. For instance, for the largest sampled values $X\approx 400$, the number of contributing entities is approximately at least seven. This result has a simple interpretation: due to the dependence between the growth factors across the entities, one very large entity is always surrounded by a “cloud” of similar sized entities. In the single entity dynamical version of the Kesten process, this cloud is made of the cluster of large transient excursions [32].

Let us now turn to the independence case shown in the first row of three panels (panels a). The most striking observation is the progressive shrinkage of the range of $H^{-1}$ values as X gets larger and larger. In addition, there seems to be a rather well-defined threshold such that, for X larger than this threshold, $H^{-1}$ collapses to the single value $H^{-1}=1$. In words, the very large values of X that populate the asymptotic power-law behavior of the distribution of X are due to a single entity that overwhelmingly contributes to it. This is reminiscent of a condensation phenomenon, in which a single entity is overwhelmingly larger than all the others, notwithstanding their a priori unlimited number. This might be related to the dragon-king concept [39,40] in an unconventional way: the distribution of the sizes of entities can be seen as the weighted sum of distributions of products of random growth factors. Due to the stationary condition that the random growth factors tend to be smaller than one (the exact technical condition is that the expectation of the logarithm of the random growth factors is negative), only products of random growth factors that have a finite number of terms have a non-negligible size. The system of entities is thus made of a finite (possibly large) number of non-negligible entities at all times. Conditional on the sum X being asymptotically large, the distribution of the entities is non-power-law, but there is a single entity that is of order X, qualifying it as a dragon-king. Interestingly, when sampling over many realizations (statistically or temporally), such set of systems with dragon-kings exhibit in ensemble a pure asymptotic power-law behavior.

Another interesting result when comparing the Kesten and the independence cases is that small values of X, less than the unit size, occur much more frequently in the Kesten case than in the independence case, also a consequence of the strong dependence among entities in the former case: if one entity is small, it is likely that others are small as well.

The mixed dependence (panels c) shows an intermediary behavior between independence and Kesten dependence. In addition, changing the value of the standard deviation $\sigma$ of the growth factors does not change the general behavior.

For the independence case, it is interesting to mention that, in addition to the fact that only one entity is responsible for the whole system size X, the age of this maximum size entity is a stochastic variable. This means that the product term that builds the power-law tail of $F\left(X\right)$ varies as a function of time in its number of multiplicative growth factors, i.e., it does not have the same number of factors at all time steps. Figure 4 shows the age of the entity with maximum size against the total size X of the system for the independence case with $\sigma =0.8557$ (panel b-1), $\sigma =1$ (panel b-2) and $\sigma =1.2533$ (panel b-3): there is a broad distribution of ages of the maximum size entities for large values of X and not a unique age. This observation suggests that the power-law behavior in this case results from the superposition of the statistics of single multiplicative processes at different ages with appropriate statistical weights, as in the superstatistics mechanism mentioned in the introduction [14,15,16]. It is notable that studies following similar ideas with exponential weights established the so-called double Pareto distribution, with power-law tail [41,42].

Finally, numerical simulations in Figure 4 (panels b-1–b-3) suggest that there exist a minimum age for this maximum size entity, which grows approximately logarithmically with X in the asymptotic tail regime. This observation can be rationalized by borrowing the extreme deviation theorem applied to finite products [13]. Parameterizing the probability density function of the independent and identically distributed growth factors ${\xi }_j\left(k\right)$ as $e^{-g\left(\xi \right)}$ with $g\left(\xi \right)$ being a function with a number of “natural” properties such as increasing sufficiently fast as $x\to \infty$, then the distribution of the product ${\Pi }_n$ of n independent such variables is given by $exp\left[-ng\left({\Pi }_n^{1/n}\right)\right]$. In words, the typical tail value of the product of n independent variables is contributed mainly by realizations where all the n random variables are approximately similar in amplitude and equal to ${\Pi }_n^{1/n}$, so that their product (with n term) recover ${\Pi }_n$. The term n in front of $g\left({\Pi }_n^{1/n}\right)$ in the exponential correspond to taking the product of n probability density functions of the independent growth factors ${\xi }_j\left(k\right)$. Fixing ${\Pi }_n$ to a large value X, the value $n^{*}\left(X\right)$ that maximizes $exp\left[-ng\left(X^{1/n}\right)\right]$ is given by:

$$n^{*}\left(X\right)=\left[\frac{u{g}^{\prime }\left(u\right)}{g\left(u\right)}\right]logX,\mathrm{with}u:=X^{1/n^{*}},$$

(16)

and $g^{\prime }\left(u\right)$ denotes the derivative of $g\left(u\right)$. Searching for a solution of the form $n^{*}\left(X\right)=\frac{logX}{logc}$, where c is a constant, we find that c solves the equation $logc=\frac{g\left(c\right)}{c{g}^{\prime }\left(c\right)}$. Thus, for the process (7), for a given large X, the product term that contributes the most to a given large Sigma-Pi X contains at least $n^{*}\left(X\right)\sim logX$ factors. This means that a large X is minimally controlled by the term with about $logX$ factors ${\xi }_j\left(k\right)$, i.e., with an age $\sim logX$. This result is essentially independent of the specific form of the probability density function $e^{-g\left(\xi \right)}$ of the growth factors ${\xi }_j\left(k\right)$, as long as $g\left(\xi \right)$ obeys the conditions for the validity of the extreme deviation theorem.

4. Conclusions

Two main conclusions can be drawn from this work. First, for sums of products of random growth factors, referred above as Sigma-Pi structures, the dependence among the product terms (the “entities”) that may result from some dependence between the growth factors do not impact the asymptotic power-law of the distribution of these Sigma-Pi variables, up to slowly varying functions. Simulations of our simple growth model with distinct dependence types together with previous theoretical studies on the Kesten process and of the sum of products of independent random variable support this conclusion. Indeed, the essential ingredient for the asymptotic power-law tail seems to be the Sigma-Pi structure, regardless of the dependence type. This conjecture is still to be formally proved.

Second, although the exponent of the asymptotic power-law tail is not influenced by the dependence type among entities, the nature of the construction of the events populating the tail differs in each dependence case. In the independence case, only one entity in the sum contributes to it asymptotically. In contrast, in the Kesten dependence case, as a result of the strong dependence between the growth factors across entities, several entities contribute to the sum asymptotically. These insights are important in the modeling of real physical, biological or economic systems, complementing the ubiquitous power-law tail behavior with additional information on the internal structure of the system.