Interacting urn models with strong reinforcement

Abstract.

For the interacting urn model with polynomial reinforcement, it has been conjectured in [16] that almost surely one color monopolizes all the urns if the interaction parameter $p>0$. We disprove the conjecture.

For the case $p=1$, we give a sufficient condition for monopoly, which improves a result obtained by Launay in [17].

1. General introduction

1.1. Definition of the model

Reinforced processes provide a rich framework for modeling and analyzing systems in physics, economics, and social sciences, where history plays a crucial role in shaping future dynamics. We refer to [23] for a survey of various models of random processes with reinforcement and their applications, a basic model of which is the well-known Pólya urn model. A generalized Pólya urn model is defined as follows.

Let $\{W(n)\}_{n\geq 1}$ be a positive sequence. Given an urn of black and red balls, let $B_{n}$ and $R_{n}$ denote the number of black and red balls in the urn at time $n\in\mathbb{N}$, respectively. We assume that $B_{0}$ and $R_{0}$ are positive integers. At each time step $n\geq 1$, we draw a ball from the urn and then return the ball to the urn along with another ball of the same color. The probability of drawing a ball of a certain color from the urn is proportional to $W(k)$ where $k$ is the number of balls of this color, that is,

$$\mathbb{P}(B_{n+1}=B_{n}+1|(B_{i})_{0\leq i\leq n},(R_{i})_{\leq i\leq n})=\frac{W(B_{n})}{W(B_{n})+W(R_{n})},\quad n\in\mathbb{N}.$$

Notice that the classical Pólya urn corresponds to the case $W(n)=n$.

In [11, 21], this model was called a balls-in-bins process with feedback where the authors were motivated by economic problems of competition and the sequence $\{W(n)\}_{n\geq 1}$ was called the feedback function. It is also referred to as an ordinal dependent Pólya urn [23] since it is equal in law to the following process: At each time step, we draw a ball from the urn randomly uniformly with replacement, and add $W(n+1)-W(n)$ red balls, resp. $W(n+1)-W(n)$ black balls, if it is the $n$-th time a red ball, resp. a black ball is drawn.

We denote by $\mathcal{D}$ the event that eventually only balls of one color are added to the urn. Using Rubin’s exponential embedding, Davis [10] proved that

$$\mathbb{P}(\mathcal{D})=\left\{\begin{aligned} &1,&&\text{if }\sum_{n=1}^{\infty}\frac{1}{W(n)}<\infty,\\ &0,&&\text{if }\sum_{n=1}^{\infty}\frac{1}{W(n)}=\infty.\end{aligned}\right.$$ (1)

Recently, there is a growing interest in the study of systems of interacting urns, see e.g. [1, 3, 6, 7, 9, 13, 14, 20, 25]. We study a model of (strongly) reinforced interacting urns introduced by Launay [16], which can be described as follows.

The model has $d$ urns containing black and red balls. Imagine that there are barriers separating different urns. At each step, for the i-th urn, $i=1,2,\cdots,d$,

1. with probability $p\in[0,1]$, (all the barriers are removed, and) a ball is drawn from a combined pool of all urns with replacement, see e.g. Figure 1(b) for the case $d=2$;

2. with probability $1-p$, (the barriers are kept, and) a ball is drawn from the i-th urn with replacement, see e.g. Figure 1(a);

3. The probability of drawing a ball of a certain color is proportional to $W$(#the number of balls of that color), as in the ordinal-dependent Pólya’s urn;

4. In either case, we add another ball of the same color as the drawn ball to the i-th urn.

We do the above procedure simultaneously and independently for each urn.

More precisely, let $B_{n}(i)$ and $R_{n}(i)$ denote the number of black and red balls in the i-th urn at time $n$, respectively. Then, $B_{n}^{*}:=\sum_{i=1}^{d}B_{n}(i)$, resp. $R_{n}^{*}:=\sum_{i=1}^{d}R_{n}(i)$, is the total number of black balls, resp. red balls, in the system at time $n$. Write $R_{n}:=(R_{n}(i))_{1\leq i\leq d}$ and $B_{n}:=(B_{n}(i))_{1\leq i\leq d}$. The initial composition is given by $(B_{0},R_{0})\in\mathbb{N}^{2d}$ with $B_{0}^{*},R_{0}^{*}\geq 1$. Let $\{W(n)\}_{n\geq 1}$ be a positive sequence and let $W(0)\geq 0,p\in[0,1]$ be two constants. For any $n\in\mathbb{N}$, conditional on $\mathcal{G}_{n}:=\sigma(B_{m},R_{m}:m\leq n)$, define independent Bernoulli random variables $(\xi_{n+1}(i))_{1\leq i\leq d}$ by

$$\mathbb{P}(\xi_{n+1}(i)=1|\mathcal{G}_{n})=\frac{pW(B_{n}^{*})}{W(B_{n}^{*})+W(R_{n}^{*})}+\frac{(1-p)W(B_{n}(i))}{W(B_{n}(i))+W(R_{n}(i))}.$$ (2)

Now set

$$B_{n+1}(i):=B_{n}(i)+\xi_{n+1}(i),\quad R_{n+1}(i):=R_{n}(i)+1-\xi_{n+1}(i).$$ (3)

The process $(B_{n},R_{n})_{n\in\mathbb{N}}$ is called the interacting urn mechanism (IUM) with reinforcement sequence $\{W(n)\}_{n\in\mathbb{N}}$ and interaction parameter $p$. We denote its law by $\mathbb{P}_{p}^{W}$.

(a) barrier is kept with prob. $1-p$

(b) barrier is removed with prob. $p$

Unless otherwise specified, we assume $d=2$ (i.e. there are two urns) for simplicity. Without loss of generality, we assume that $R_{0}(1)\geq 1$ and $B_{0}(2)\geq 1$. Let

$$x_{n}:=\frac{B_{n}(1)}{n+B_{0}(1)+R_{0}(1)},\quad y_{n}:=\frac{B_{n}(2)}{n+B_{0}(2)+R_{0}(2)}$$ (4)

be the proportions of black balls in the two urns at time $n$, respectively.

For an IUM with $p>0$, there is a tendency for different components to adopt a common behavior. For example, for IUM with linear reinforcements, i.e. $W(n)=n$, Dai Pra, Louis and Minelli [9] proved that if $B_{0}(1)=B_{0}(2)$ and $R_{0}(1)=R_{0}(2)$, then for any $p>0$, almost surely,

$$\lim_{n\to\infty}x_{n}\ \text{and}\ \lim_{n\to\infty}y_{n}\quad\text{both exist and are equal.}$$ (5)

This phenomenon is called synchronization. Moreover, it has been proved in [6, Theorem 3.2] that this common limit satisfies

$$\mathbb{P}(\lim_{n\to\infty}x_{n}\in\{0,1\})<1,\quad\text{and}\quad\mathbb{P}(\lim_{n\to\infty}x_{n}=x)=0,\ \text{for any}\ x\in(0,1).$$ (6)

However, IUMs with strong reinforcement may exhibit very different behaviors. We say that $\{W(n)\}_{n\in\mathbb{N}}$ is a strong reinforcement sequence if

$$\sum_{n=1}^{\infty}\frac{1}{W(n)}<\infty.$$ (7)

As we will see later, for some IUMs with strong reinforcement sequences and weak interaction (i.e. $p$ is small), one color can maintain its advantage in a single urn while it is at a disadvantage globally, in which case the urns do not synchronize. Indeed, this models a common phenomenon in economic systems: Many companies perform well in their local markets but struggle to replicate that success globally due to weak interactions between regions. On the other hand, for strong interaction, the phenomena of domination and monopoly may occur, as already exhibited in the ordinal dependent Pólya urn.

Definition 1 (Domination and monopoly).

For an IUM, we denote by

$$\mathcal{D}=\left\{\lim_{n\rightarrow\infty}(x_{n},y_{n})=(0,0)\right\}\cup\left\{\lim_{n\rightarrow\infty}(x_{n},y_{n})=(1,1)\right\}$$

the event that eventually the number of balls of one color is negligible with respect to the number of balls of the other color, and call this event domination. Further, we denote by

$$\mathcal{M}=\left\{R^{*}_{n+1}=R^{*}_{n}\text{ eventually for all }n\right\}\cup\left\{B^{*}_{n+1}=B^{*}_{n}\text{ eventually for all }n\right\}$$

the event that eventually only balls of one color are added to the urns, and call this event monopoly. Note that $\mathcal{M}\subset\mathcal{D}$.

We will be interested in how $\mathbb{P}^{W}_{p}(\mathcal{D})$ and $\mathbb{P}^{W}_{p}(\mathcal{M})$ are affected by the parameter $p$ and the sequence $\{W(n)\}_{n\in\mathbb{N}}$, especially in the case of power function/polynomial reinforcement sequences, i.e. $W(n)=n^{\alpha}$, $n\geq 1$, for some real number $\alpha\geq 1$, or $W(n)$ is of the form

$$W(n):=n^{\alpha}+c_{1}n^{\alpha-1}+\cdots+c_{\alpha},\quad n\geq 1,$$ (8)

where $\alpha$ is a positive integer. For $\alpha=1$, we can assume that $c_{1}=0$ since in this case the IUM is equal in law to an IUM with $W(n)=n$ and shifted initial condition $(B_{0},R_{0})+(c_{1},c_{1},c_{1},c_{1})$. In addition, as we will see later, our results do not depend on $c_{1},c_{\alpha-1},\cdots,c_{\alpha}$. Thus, by a slight abuse of notation, in either of the two cases above, we let $\mathbb{P}_{p}^{(\alpha)}$ denote the law of the IUM with reinforcement sequence $\{W(n)\}_{n\in\mathbb{N}}$ and parameter $p$. We show that under $\mathbb{P}_{p}^{(\alpha)}$, domination implies monopoly a.s., and thus, we will not distinguish the two events in the sequel.

Proposition 1.1.

Assume that $\{W(n)\}_{n\in\mathbb{N}}$ satisfies (7) and is eventually increasing, i.e., there exists $N_{1}\in\mathbb{N}$ such that $W(n+1)-W(n)\geq 0$ for all $n\geq N_{1}$. If

$$\lim_{K\to\infty}\limsup_{n\to\infty}\left(\sum_{i\geq Kn}\frac{1}{W(i)}\right)\left(\sum_{i\geq n}\frac{1}{W(i)}\right)^{-1}=0,$$ (9)

then $\mathbb{P}_{p}^{W}(\mathcal{D}\backslash\mathcal{M})=0$ for any $p\in[0,1]$. In particular, for any $\alpha>1$ and $p\in[0,1]$, one has $\mathbb{P}_{p}^{(\alpha)}(\mathcal{D}\backslash\mathcal{M})=0$.

For $\alpha=1$, by classical results for Pólya urn model and (6), one has

$$\mathbb{P}_{0}^{(1)}(\mathcal{D})=0,\quad\mathbb{P}_{p}^{(1)}(\mathcal{D})<1\ \text{for any }p>0.$$ (10)

We will show in Theorem 1.2 (i) that $\mathbb{P}_{p}^{(1)}(\mathcal{D})=0$ for $p>0$. Thus, $\mathbb{P}_{p}^{(\alpha)}(\mathcal{D}\backslash\mathcal{M})=0$ also holds for $\alpha=1$.

For $\alpha>1$, Launay proved in [17, Theorem 2.3] that $\mathbb{P}^{(\alpha)}_{1}(\mathcal{M})=1$ and conjectured that $\mathbb{P}_{p}^{(\alpha)}(\mathcal{M})=1$ for all $p>0$. This work aims to disprove the conjecture and generalize the results obtained in [17].

1.2. Main results

1.2.1. Power function/Polynomial reinforcements

For $\alpha>1$, define

$$p_{\alpha}:=\inf\{0\leq q\leq 1:\mathbb{P}_{p}^{(\alpha)}(\mathcal{D})=1,\ \forall p\geq q\}.$$ (11)

which we call the critical parameter. Our first main result shows that $p_{\alpha}>0$, which disproves Launay’s conjecture. Moreover, for $\alpha=1$, we prove that (5) holds for general initial conditions, and the domination occurs with probability 0.

Theorem 1.2.

For any $\alpha\geq 1$ and $p\in[0,1]$, under $\mathbb{P}_{p}^{(\alpha)}$, the sequence $(x_{n},y_{n})_{n\in\mathbb{N}}$ defined in (4) is convergent a.s.. Moreover,
(i) if $\alpha=1$, then for any $p>0$, one has $\mathbb{P}_{p}^{(1)}(\mathcal{D})=0$ and

$$\lim_{n\to\infty}x_{n}=\lim_{n\to\infty}y_{n},\quad a.s.$$

(ii) if $\alpha>1$, then

$$\mathbb{P}_{p}^{(\alpha)}\left(\lim_{n\to\infty}(x_{n},y_{n})=(u_{\alpha,p},1-u_{\alpha,p})\right)>0,\quad\text{for}\ p\leq\frac{1}{6\alpha}\min\left(\alpha-1,2\right),$$

where $u_{\alpha,p}$ is the unique solution of the following equation on $[0,1/2):$

$$u=\frac{(1-p)u^{\alpha}}{u^{\alpha}+(1-u)^{\alpha}}+\frac{p}{2}.$$

Note that $u_{\alpha,p}$ exists if $p<(\alpha-1)/\alpha$, see Lemma 3.1.

As mentioned in [16, Conclusion], polynomial reinforcements do not behave as exponential reinforcements where $W(n)=\rho^{n},n\in\mathbb{N}$, for some $\rho>1$. It has been proved in [16] that, for any $\rho>1$, one has

$$\mathbb{P}_{p}^{W}(\mathcal{M})=1,\ \text{if }p\geq\frac{1}{2};\quad\mathbb{P}_{p}^{W}(\mathcal{D}^{c})\geq\mathbb{P}_{p}^{W}(\lim_{n\to\infty}(x_{n},y_{n})=(p,1))>0,\ \text{if }p<\frac{1}{2}.$$

Therefore, one can observe a phase transition at $p=1/2$.

Remark 1.1.

As $\rho$ tends to $\infty$, the exponential reinforcement mechanism converges to the ”generalized” reinforcement, which was introduced and studied by Launay and Limic in [18].

Our second main result shows that power function/polynomial reinforcements are weaker than exponential reinforcements in the sense that $p_{\alpha}<1/2$. On the other hand, for large $\alpha$, the reinforcement becomes very strong and behaves ”like” the exponential reinforcement: For $p<1/2$, if $\alpha$ is sufficiently large, then with positive probability, $\lim_{n\to\infty}(x_{n},y_{n})$ is close to $(p,1)$. In particular, $\lim_{\alpha\to\infty}p_{\alpha}=1/2$.

Theorem 1.3.

(i) For any $\alpha>1$, one has $p_{\alpha}<\min(1/2,\alpha^{-1}(\alpha-1))$.
(ii) For $\alpha\geq 3$, if $p\leq 1/2-\alpha^{-1}\log\alpha$, then $\mathbb{P}_{p}^{(\alpha)}(\mathcal{D})<1$.
(iii) Fix $p<1/2$, for sufficiently large $\alpha$, there exists $s_{\alpha}\in[0,1/2)\times(1/2,1]$ such that

$$\mathbb{P}_{p}^{(\alpha)}(\lim_{n\to\infty}(x_{n},y_{n})=s_{\alpha})>0\quad\text{and}\quad\lim_{\alpha\to\infty}s_{\alpha}=(p,1).$$

Remark 1.2.

For $\alpha>1$, by Theorem 1.2 (ii), we can define

$$\tilde{p}_{\alpha}:=\sup\{0\leq q\leq 1:\mathbb{P}_{p}^{(\alpha)}(\mathcal{D})<1,\ \forall p\leq q\}.$$ (12)

Then $\tilde{p}_{\alpha}\leq p_{\alpha}$. We conjecture that $\tilde{p}_{\alpha}=p_{\alpha}$. Theorem 1.2 and Theorem 1.3 give some estimates on their convergence rate to $0$, resp. $1/2$, as $\alpha\to 1$, resp. as $\alpha\to\infty$:

$$\frac{\alpha-1}{6\alpha}\leq\tilde{p}_{\alpha}\leq p_{\alpha}<\frac{\alpha-1}{\alpha}\quad(\alpha\leq 3);\quad\frac{1}{2}-\frac{\log\alpha}{\alpha}\leq\tilde{p}_{\alpha}\leq p_{\alpha}<\frac{1}{2}\quad(\alpha\geq 3).$$

1.2.2. Urns with simultaneous drawing

For general reinforcement sequences, the case $p=1$ was studied in [17] whose main result is the following. Recall (7) for the definition of the strong reinforcement sequences.

Theorem 1.4 (Launay, [17]).

Given $d\geq 2$ urns, if $\{W(n)\}_{n\in\mathbb{N}}$ is a non-decreasing strong reinforcement sequence, then $\mathbb{P}_{1}^{W}(\mathcal{M})=1$.

We see from (1) that for $d=1$, the monotonicity assumption is not needed. It was conjectured in [17] that this assumption is also redundant for the cases $d\geq 2$. In this work, we show a generalization of Theorem 1.4 to a larger class of strong reinforcement sequences.

We will not limit ourselves to two-color urns. In this case, it is convenient to assume that we have only a single urn of $N_{c}$-color balls where $N_{c}\geq 2$. Let $N_{n}(i)$ be the number of balls of the $i$-th color in this urn at time $n$, and write $N_{n}:=(N_{n}(i))_{1\leq i\leq N_{c}}$. Without loss of generality, we assume that $(N_{0}(i))_{1\leq i\leq N_{c}}=(a_{i})_{1\leq i\leq N_{c}}$ are positive integers. For any $n\in\mathbb{N}$, conditional on $\mathcal{G}_{n}:=\sigma(N_{m}:m\leq n)$, the law of $N_{n+1}-N_{n}$ is given by the multinomial distribution

$$M(d;(p_{1},p_{2},\cdots,p_{N_{c}})),\quad\text{where}\ p_{i}:=\frac{W(N_{n}(i))}{\sum_{j=1}^{N_{c}}W(N_{n}(j))}.$$

One can easily see that the process $(N_{n})_{n\in\mathbb{N}}$ is an IUM with reinforcement sequence $\{W(n)\}_{n\in\mathbb{N}}$ and parameter $p=1$. The event monopoly is then given by

$$\mathcal{M}=\left\{\exists i\in\{1,2,\cdots,N_{c}\},\ s.t.\ N_{n+1}(i)=N_{n}(i)+d\text{ for all large }n\right\}.$$

Theorem 1.5.

One has $\mathbb{P}_{1}^{W}(\mathcal{M})=1$ if $\{W(n)\}_{n\in\mathbb{N}}$ is a strong reinforcement sequence that satisfies one of the following conditions:
(i) There exists a positive constant $C$ such that for all $n\geq 1$,

$$W(n)\delta_{n}\leq C,\ \text{where}\ \delta_{n}:=\sum_{k=n}^{\infty}\left|\frac{1}{W(k)}-\frac{1}{W(k+1)}\right|.$$ (13)

(ii) For $n\geq 1$, let $A_{n}:=\sum_{i=n}^{\infty}1/W(i)^{2}$. One has

$$\limsup_{n\to\infty}\frac{\delta_{n}}{\sqrt{A_{n}}}<\frac{1}{64(d-1)},\quad\text{and}\quad\lim_{n\to\infty}\frac{1}{W(n)\sqrt{A_{n}}}=0.$$ (14)

Remark 1.3.

(I) If $\{W(n)\}_{n\in\mathbb{N}}$ is non-decreasing and satisfies (7), then (13) holds with $C=1$. Thus, Theorem 1.5 generalizes Theorem 1.4.
(II) In either case, we require that $\delta_{n}$, the total variation of $\{1/W(k)\}_{k\geq n}$, is relatively small. For example, if $W(n)=(2+(-1)^{n})n^{2}$ or $W(n)=e^{(2+(-1)^{n})n}$, then neither (13) nor (14) is satisfied. It is worth mentioning that similar conditions and examples have appeared in the study of strongly edge-reinforced random walks, see [19].
(III) Each condition cannot be derived from the other. By (I), $\{e^{n}\}_{n\in\mathbb{N}}$ satisfies (13) but does not satisfy (14). On the other hand, one can check by the Cauchy-Schwarz inequality that if

$$\sum_{n=1}^{\infty}\left(\frac{W(n+1)-W(n)}{W(n)}\right)^{2}<\infty,$$

then (14) is satisfied, see e.g. the proof of [19, Corollary 4]. In particular, if $W(n)=2n^{2}+(-1)^{n}n^{4/3}$, then (14) is satisfied but (13) is not satisfied.

2. Introduction to the proofs and the techniques

2.1. Notation

We let $C(a_{1},a_{2},\cdots,a_{k})$ denote a positive constant depending only on real variables $a_{1},a_{2},\ldots,a_{k}$ and let $C$ denote a universal positive constant, which usually means that $C(a_{1},a_{2},\cdots,a_{k})$ and $C$ do not depend on $n$.

For a real-valued function $h$ and a $[0,\infty)$-valued function $g$, we write $h(x)=O(g(x))$ as $x\to\infty$, resp. $x\to 0$, if there exist positive constants $C$ and $x_{0}$ such that $|h(x)|\leq Cg(x)$ for all $x\geq x_{0}$, resp. $|x|\leq x_{0}$.

We let $\mathbb{N}_{+}:=\mathbb{N}\cap(0,\infty)$. We let $\|\cdot\|$ denote the usual Euclidean norm. We write $X\sim\operatorname{Exp}(\lambda)$ if a random variable $X$ has an exponential distribution with rate $\lambda$.

2.2. Stochastic approximation algorithms

Under $\mathbb{P}_{p}^{(\alpha)}$, we show that $((x_{n},y_{n}))_{n\in\mathbb{N}}$ defined by (4) is generated by a stochastic approximation algorithm (Robbins-Monro algorithm), and is closely related to the following (deterministic) planar nonlinear system

$$\left(\frac{dx}{dt},\frac{dy}{dt}\right)=F^{(\alpha)}_{p}(x,y)$$ (15)

where $F^{(\alpha)}_{p}=(F_{p,1}^{(\alpha)},F_{p,2}^{(\alpha)})$ is a vector function on $[0,1]^{2}$ defined by

$$\left\{\begin{aligned} F_{p,1}^{(\alpha)}(x,y)&:=-x+\frac{(1-p)x^{\alpha}}{x^{\alpha}+(1-x)^{\alpha}}+\frac{p(x+y)^{\alpha}}{(x+y)^{\alpha}+(2-x-y)^{\alpha}},\\ F_{p,2}^{(\alpha)}(x,y)&:=-y+\frac{(1-p)y^{\alpha}}{y^{\alpha}+(1-y)^{\alpha}}+\frac{p(x+y)^{\alpha}}{(x+y)^{\alpha}+(2-x-y)^{\alpha}}.\end{aligned}\right.$$ (16)

For an introduction to stochastic approximation algorithms, see e.g. [2, 4, 5, 12].

Proposition 2.1.

For any $\alpha\geq 1$ and $p\in[0,1]$, under $\mathbb{P}_{p}^{(\alpha)}$, $((x_{n},y_{n}))_{n\in\mathbb{N}}$ defined by (4) satisfies the following recursion:

$$(x_{n+1},y_{n+1})-(x_{n},y_{n})=\frac{1}{n+1}(F^{(\alpha)}_{p}(x_{n},y_{n})+\varepsilon_{n+1}+r_{n+1}),\quad n\in\mathbb{N},$$ (17)

where $(\varepsilon_{n+1})_{n\in\mathbb{N}}$ and $(r_{n+1})_{n\in\mathbb{N}}$ are adapted sequences such that for all $n\in\mathbb{N}$,

$$\mathbb{E}(\varepsilon_{n+1}\mid\mathcal{G}_{n})=(0,0),\quad\|\varepsilon_{n+1}\|\leq 2,\quad\|r_{n+1}\|\leq\frac{C}{n+1},$$

where $C$ is a positive constant.

For the system (15) with initial condition $(x(0),y(0))\in[0,1]^{2}$, the existence and uniqueness of the solution follow from the Lipschitz property of $F^{(\alpha)}_{p}$ and Picard’s theorem. Note that the solution satisfies $(x(t),y(t))\in[0,1]^{2}$ for all $t\geq 0$.

We shall study the asymptotic behavior of the solution to the system (15). It turns out that (15) is a gradient system. The proof of the following result is direct and is omitted here.

Proposition 2.2.

For any $\alpha\geq 1$, define $L^{(\alpha)}_{p}:[0,1]^{2}\to\mathbb{R}$ by

$$L^{(\alpha)}_{p}(x,y)=(1-p)\left(G^{(\alpha)}(x)+G^{(\alpha)}(y)\right)+2pG^{(\alpha)}\left(\frac{x+y}{2}\right)-\frac{x^{2}+y^{2}}{2},$$ (18)

where

$$G^{(\alpha)}(t):=\int_{0}^{t}\frac{u^{\alpha}}{u^{\alpha}+(1-u)^{\alpha}}du,\quad t\in[0,1].$$

Then, we have $\operatorname{grad}L^{(\alpha)}_{p}=F^{(\alpha)}_{p}$.

Example 2.1 ($\alpha=1,2$).

One has $L^{(1)}_{p}(x,y)=-p(x-y)^{2}/4$ and

$$L^{(2)}_{p}(x,y) =\frac{1-p}{4}\log(x^{2}+(1-x)^{2})+\frac{1-p}{4}\log(y^{2}+(1-y)^{2})$$

$$+\frac{p}{2}\log((x+y)^{2}+(2-x-y)^{2})-p\log 2-\frac{x^{2}}{2}+\frac{x}{2}-\frac{y^{2}}{2}+\frac{y}{2}.$$

A point $(x,y)\in[0,1]^{2}$ is called an equilibrium of (15) if $F^{(\alpha)}_{p}(x,y)=0$. Let $\Lambda^{(\alpha)}_{p}$ be the set of all equilibrium points. Observe that $\Lambda^{(1)}_{p}=\{(x,x):x\in[0,1]\}$ for any $p>0$. We prove that $\Lambda^{(\alpha)}_{p}$ is a finite set for $\alpha>1$. The cases $\alpha=3,p=0.4$ and $\alpha=5,p=0.3$ are plotted in Figure 1 where $\Lambda^{(\alpha)}_{p}$ is the set of intersection points of the two curves.

(c) $\alpha=3,p=0.4$

(d) $\alpha=5,p=0.3$

Figure 1. $F^{(\alpha)}_{p,1}(x,y)=0$ in blue and $F^{(\alpha)}_{p,2}(x,y)=0$ in orange

Proposition 2.3.

$\Lambda^{(\alpha)}_{p}$ includes $(0,0),(1/2,1/2),(1,1)$. Moreover, for $\alpha>1$, one has,
(i) if $p=0$, then $\Lambda^{(\alpha)}_{0}=\{0,1/2,1\}\times\{0,1/2,1\};$
(ii) if $p\in[1/2,(\alpha-1)/\alpha)$ (this is possible only when $\alpha>2$), then

$$\Lambda^{(\alpha)}_{p}=\{(0,0),(\frac{1}{2},\frac{1}{2}),(1,1),(u_{\alpha,p},1-u_{\alpha,p}),(1-u_{\alpha,p},u_{\alpha,p})\},$$

where $u_{\alpha,p}$ is defined in Theorem 1.2;
(iii) if $p\geq(\alpha-1)/\alpha$, then $\Lambda^{(\alpha)}_{p}=\{(0,0),(1/2,1/2),(1,1)\};$
(iv) if $p>0$, then $\Lambda^{(\alpha)}_{p}$ is finite and

$$\Lambda^{(\alpha)}_{p}\backslash\{(0,0),(\frac{1}{2},\frac{1}{2}),(1,1)\}\subset\left((0,\frac{1}{2})\times(\frac{1}{2},1)\right)\cup\left((\frac{1}{2},1)\times(0,\frac{1}{2})\right).$$

Propositions 2.2, 2.3 and Example 2.1 then imply that the solution to (15) converges to $\Lambda^{(\alpha)}_{p}$. More precisely,

• If $\alpha=1$ and $p>0$, then $x(t)+y(t)\equiv x(0)+y(0)$ and $L_{p}^{(1)}(x(t),y(t))$ increases to 0 as $t\to\infty$. In particular, both $x(t)$ and $y(t)$ converges to $(x(0)+y(0))/2$.

• If $\alpha>1$, $(x(t),y(t))$ converges to an equilibrium as $t\to\infty$.

Definition 2 (Asymptotically stable equilibria).

For $\alpha>1$ and $p\in[0,1]$, an equilibrium $(x,y)$ of (15) is said to be asymptotically stable if $(x,y)$ is a local maximum of $L^{(\alpha)}_{p}$ defined by (18). We let $\mathcal{E}_{p}^{(\alpha)}$ be the set of asymptotically stable equilibria of (15).

Define

$$f(t):=\frac{t^{\alpha-1}(1-t)^{\alpha-1}}{[t^{\alpha}+(1-t)^{\alpha}]^{2}},\quad t\in[0,1].$$ (19)

Observe that the Jacobian matrix of the system (15) is given by

$$DF^{(\alpha)}_{p}(x,y)=\left(\begin{array}[]{ll}-1+\alpha(1-p)f(x)+\frac{\alpha p}{2}f(\frac{x+y}{2})&\hskip 42.67912pt\frac{\alpha p}{2}f(\frac{x+y}{2})\\ \hskip 42.67912pt\frac{\alpha p}{2}f(\frac{x+y}{2})&-1+\alpha(1-p)f(y)+\frac{\alpha p}{2}f(\frac{x+y}{2})\end{array}\right)$$

which is a real symmetric matrix and thus has two real eigenvalues:

$$\lambda_{\pm}(x,y) =-1+\frac{\alpha p}{2}f(\frac{x+y}{2})+\alpha(1-p)\frac{f(x)+f(y)}{2}$$ (20)

$$\pm\frac{1}{2}\sqrt{\alpha^{2}(1-p)^{2}(f(x)-f(y))^{2}+\alpha^{2}p^{2}f^{2}(\frac{x+y}{2})}.$$

Note that for an equilibrium $(x,y)$, if $\lambda_{+}(x,y)<0$, then $(x,y)\in\mathcal{E}_{p}^{(\alpha)}$; it is called unstable if $\lambda_{+}(x,y)>0$.

Example 2.2.

$(0,0)$ and $(1,1)$ are asymptotically stable equilibria since $\lambda_{+}(0,0)=\lambda_{+}(1,1)=-1$. While $(1/2,1/2)$ is unstable since $\lambda_{+}(1/2,1/2)=\alpha-1>0$.

If $\alpha>1$ and $p$ is sufficiently small, or $p<1/2$ and $\alpha$ is sufficiently large, we prove the existence of asymptotically stable equilibrium.

Proposition 2.4.

(i) For $\alpha>1$ and $p\leq\alpha^{-1}\min((\alpha-1)/6,1/3)$, one has

$$\lambda_{+}(u_{\alpha,p},1-u_{\alpha,p})<0,$$

where $u_{\alpha,p}$ is defined in Theorem 1.2.
(ii) For $\alpha\geq 3$, if $0<p\leq 1/2-\alpha^{-1}\log\alpha$, then there exists an asymptotically stable equilibrium in $(0,1/2)\times(1/2,1)$.
(iii) Fix $p\in(0,1/2)$, for sufficiently large $\alpha$, there exists $s_{\alpha}\in(0,1/2)\times(1/2,1)$ such that

$$\lambda_{+}(s_{\alpha})<0\quad\text{and}\quad\lim_{\alpha\to\infty}s_{\alpha}=(p,1).$$

For the cases $\alpha=3,p=0.1$ and $\alpha=40,p=0.4$, the vector fields generated by (15) are plotted in Figure 2. In Figure 2(a), $(u_{\alpha,p},1-u_{\alpha,p})$ is inside the red circle. Readers can also find an asymptotically stable equilibrium $s_{\alpha}$ near $(0.4,1)$ in Figure 2(b).

(a) $\alpha=3,p=0.1$

(b) $\alpha=40,p=0.4$

Figure 2. The vector fields generated by (15) for small p or large $\alpha$

The following result says that if $p\geq 1/2$, then all the equilibria except $(0,0)$ and $(1,1)$, are unstable, as is illustrated in Figure 3 for the cases $\alpha=3,p=0.5$ and $\alpha=5,p=0.5$.

Corollary 2.5.

Assume that $\alpha>1$ and $p\geq\min(1/2,\alpha^{-1}(\alpha-1))$. If $(x,y)\in\Lambda^{(\alpha)}_{p}\backslash\{(0,0),(1,1)\}$, then $\lambda_{+}(x,y)>0$.

Proof.

If $p\in[1/2,\alpha^{-1}(\alpha-1))$ (in particular, $\alpha>2$), then

$$\lambda_{+}(u_{\alpha,p},1-u_{\alpha,p})=\lambda_{+}(1-u_{\alpha,p},u_{\alpha,p})=-1+\alpha p+\alpha(1-p)f(u_{\alpha,p})>\alpha p-1>0$$

For any $p\in[0,1]$, as shown in Example 2.2, $\lambda_{+}(1/2,1/2)>0$. Thus, Corollary 2.5 is a direct consequence of Proposition 2.3 (ii) and (iii). ∎

(a) $\alpha=3,p=0.5$

(b) $\alpha=5,p=0.5$

Figure 3. The vector fields generated by (15) for $p\geq 0.5$

We now sketch the proof for Theorems 1.2 and 1.3 (the details will be given in Section 4):

• Using stochastic approximation techniques, we show that under $\mathbb{P}_{p}^{(\alpha)}$, as in the deterministic case (15), almost surely, the sequence $((x_{n},y_{n}))_{n\in\mathbb{N}}$ is convergent and the limit belongs to $\Lambda^{(\alpha)}_{p}$.

• For $\alpha>1$, stochastic approximation theory can also be used to show that $(x_{n},y_{n})$ converges to any asymptotically stable equilibrium with positive probability, and converges to any unstable equilibrium with probability 0. Theorems 1.2 (ii) and Theorem 1.3 then follows from Propositions 2.3, 2.4 and Corollary 2.5.

• For Theorems 1.2 (i), we establish a finer stochastic approximation result for $(z_{n})_{n\in\mathbb{N}}$ in Lemma 4.1 where $z_{n}$ is the proportion of black balls in the whole system at time $n$:

  $$z_{n}:=\frac{B_{n}^{*}}{2n+B_{0}^{*}+R_{0}^{*}},\quad n\in\mathbb{N}.$$ (21)

  This would enable us to show that

  $$\mathbb{P}_{p}^{(1)}(\lim_{n\to\infty}z_{n}\in\{0,1\})=0$$

2.3. Continuous-time construction with time delays

We use a continuous-time embedding technique to prove Theorem 1.5. As we have mentioned, the case $d=1$ was solved by a continuous-time construction. It is natural to consider whether this technique can be generalized.

For the purpose of the proof, we introduce a new time-lines representation which we call continuous-time construction with time delays.

(22)

We let $G$ be a graph as in (22) consisting of a single vertex $x$ and $N_{C}\geq 2$ self-loops, i.e. the edge set $E=\{e_{1},e_{2}.\cdots,e_{N_{c}}\}$ and $x\stackrel{{\scriptstyle e_{i}}}{{\sim}}x$, $i=1,2,\cdots,N_{c}$. We shall define a continuous-time jump process $X=(X_{t})_{t\geq 0}$ on $G$. Let us first introduce some preliminary notation.

Let $0=\tau_{0}<\tau_{1}<\tau_{2}<\cdots$ be the hitting times of $X$ to $x$. For each $i\in\{1,2,\cdots,N_{c}\}$ and $n\in\mathbb{N}$, let

$$Z_{n}(i):=\sum_{k=1}^{n}\mathds{1}_{\left\{(X_{\tau_{k-1}},X_{\tau_{k}})=e_{i}\right\}}+a_{i}$$ (23)

be the number of visits to $e_{i}$ up to time $\tau_{n}$ plus $a_{i}\in\mathbb{N}_{+}$ with the convention that $Z_{0}(i)=a_{i}$. Here $Z_{dn}(i)$ and $a_{i}$ should be interpreted respectively as the number of balls of the $i$-th color in the urn at time $n$ and the initial number of balls of the $i$-th color (see Proposition 2.6 for a more precise statement). For $n\geq a_{i}$, let

$$\sigma_{n}(i):=\inf\{\tau_{m}:Z_{m}(i)\geq n\}$$ (24)

with the convention that $\inf\emptyset=\infty$. Let $\{\xi^{(i)}_{n}\}_{1\leq i\leq N_{c},n>a_{i}}$ be independent Exp(1)-distributed random variables. The law of $X$ is defined as follows:

At time $t=0$, on each edge $e_{i}$ $(1\leq i\leq N_{c})$, we launch a timer with a duration $\xi^{(i)}_{a_{i}+1}/W(a_{i})$. When the timer of an edge $e_{i}$ rings, $X$ jumps to cross $e_{i}$ instantaneously. If an edge $e_{i}$ is crossed at time $\tau_{n}$ such that $kd<n\leq(k+1)d$ for some $k\in\mathbb{N}$, then we launch a new timer on this edge with a duration $\xi^{(i)}_{Z_{n}(i)+1}/W(Z_{kd}(i))$. For $k\geq 1$, at time $\tau_{kd}$, we update the denominators (i.e. the rates) for all the timers: for $i\in\{1,2,\cdots,N_{c}\}$, if the timer on $e_{i}$ has run a time of $t^{(i)}_{k}$, then we reset the timer such that the remaining time becomes

$$\frac{\xi^{(i)}_{Z_{kd}(i)+1}-W(Z_{(k-1)d}(i))t^{(i)}_{k}}{W(Z_{kd}(i))}.$$ (25)

Remark 2.1.

(i) Just before we reset the timers, the remaining time of the timer on $e_{i}$ is

$$\frac{\xi^{(i)}_{Z_{kd}(i)+1}}{W(Z_{(k-1)d}(i))}-t^{(i)}_{k}.$$

(ii) If, for some $j\in\{1,2,\cdots,N_{c}\}$, $e_{j}$ is not crossed during the time interval $(\tau_{(k-1)d},\tau_{kd}]$, then $Z_{(k-1)d}(j)=Z_{kd}(j)$ so that there is nothing to change for the timer on $e_{j}$.
(iii) If $X$ jumps to cross $e_{j}$ for some $j\in\{1,2,\cdots,N_{c}\}$ at time $\tau_{kd}$, we will launch a new timer on $e_{j}$ and thus $t^{(j)}_{k}=0$. We may simply launch a new timer on $e_{j}$ with a duration $\xi^{(j)}_{Z_{kd}(j)+1}/W(Z_{kd}(j))$ rather than $\xi^{(j)}_{Z_{kd}(j)+1}/W(Z_{(k-1)d}(j))$.
(iv) The timer which corresponds to $\xi^{(i)}_{n+1}$ may run at different rates as time changes. All the possible denominators (rates) are

$$W(n-\ell),\quad\ell\in\{0,1,\cdots,d-1\}\ \text{such that}\ \ell\leq n-a_{i},$$

due to the time delays. Note that we may update this timer at jumping times but we will never launch a new one until it rings. Recall $\sigma_{n}(i)$ defined in (24). If $\sigma_{n+1}(i)<\infty$, the total time this timer needs to run is simply $\sigma_{n+1}(i)-\sigma_{n}(i)$, in which case we can write

$$\sigma_{n+1}(i)-\sigma_{n}(i)=\sum_{\ell=0}^{(d-1)\wedge(n-a_{i})}\frac{b_{\ell}}{W(n-\ell)},$$ (26)

where $b_{\ell}\geq 0$ and $\sum_{\ell=0}^{(d-1)\wedge(n-a_{i})}b_{\ell}=\xi^{(i)}_{n+1}$.

We denote the natural filtration of $X$ by $(\mathcal{F}_{t})_{t\geq 0}$, i.e. $\mathcal{F}_{t}:=\sigma(X_{s}:0\leq s\leq t)$. Recall the process $(N_{k}(i))_{1\leq i\leq N_{c},k\in\mathbb{N}}$ defined in Section 1.2.2.

Proposition 2.6.

Let $X$ be the jump process defined above. Then,

$$(Z_{kd}(i))_{1\leq i\leq N_{c},k\in\mathbb{N}}\stackrel{{\scriptstyle\mathcal{L}}}{{=}}(N_{k}(i))_{1\leq i\leq N_{c},k\in\mathbb{N}}$$

In particular, we may define $X$ and the IUM on the same probability space such that a.s.

$$(Z_{kd}(i))_{1\leq i\leq N_{c},k\in\mathbb{N}}=(N_{k}(i))_{1\leq i\leq N_{c},k\in\mathbb{N}}$$ (27)

Proof.

Note that conditional on $\mathcal{F}_{\tau_{kd}}$, by the memoryless property of exponentials, (25) has the same distribution as $\xi/W(Z_{kd}(i))$ where $\xi$ is an Exp(1)-distributed random variable. The rest of the proof is similar to that of [26, Lemma 3.1]. ∎

As we explained in Remark 2.1, unlike the continuous-time construction in the proof of [23, Theorem 3.6], at time $\tau_{n}\in(\tau_{kd},\tau_{(k+1)d})$, we keep using the data we collect at time $\tau_{kd}$ (i.e. $(Z_{kd}(i))_{1\leq i\leq N_{c}}$) to launch new timers. This justifies its name continuous-time construction with time delays. It is a powerful technique that allows us to give a very short proof of a multi-color ($N_{c}\geq 2$) version of Theorem 1.4.

A new proof of Theorem 1.4 with $N_{c}\geq 2$.

Conditional on $\mathcal{F}_{nd}$, if $Z_{nd}(i)=\max_{1\leq j\leq N_{c}}\{Z_{nd}(j)\}$ for some $i$, then the probability that $Z_{(n+1)d}(i)=Z_{nd}(i)+d$ (i.e. we add $d$ balls of the relative major color at time $n+1$) is lower bounded by $(1/N_{c})^{d}$ since $\{W(n)\}_{n\in\mathbb{N}}$ is non-decreasing. By the conditional Borel-Cantelli lemma, see e.g. [8], a.s. such an event occurs for infinitely many $n$, and thus, there is an infinite sequence of finite stopping times $\{\tau_{n_{k}d}\}_{k\geq 1}$ such that at time $\tau_{n_{k}d}$, there exists $i_{k}\in\{1,2,\cdots,N_{c}\}$ such that

$$Z_{n_{k}d}(i_{k})\geq d+\max_{j\neq i_{k}}\{Z_{n_{k}d}(j)\}.$$ (28)

By (26), for $j=1,2,\cdots,N_{c}$, if $\sigma_{n}(j)<\infty$,

$$\frac{\xi_{n}^{(j)}}{W(n-1)}\leq\sigma_{n}(j)-\sigma_{n-1}(j)\leq\frac{\xi_{n}^{(j)}}{W(n-d)},\quad\forall n\geq a_{j}+d.$$ (29)

As is mentioned in the proof of Proposition 2.6, conditional on $\mathcal{F}_{\tau_{n_{k}d}}$, the remaining time of the timer on $e_{j}$ has the distribution of an independent copy of $\xi/W(Z_{n_{k}d}(j))$ where $\xi$ is an Exp(1)-distributed random variable. By a slight abuse of notation, the time remaining is denoted by $\xi^{(j)}_{Z_{n_{k}d}(j)+1}/W(Z_{n_{k}d}(j))$. By symmetry and (28), conditional on $\mathcal{F}_{\tau_{n_{k}d}}$, with probability at least $1/N_{c}$,

$$\sum_{n=Z_{n_{k}d}(i_{k})+1}^{\infty}\frac{\xi_{n}^{(i_{k})}}{W(n-d)}<\min_{j\neq i_{k}}\left\{\sum_{n=Z_{n_{k}d}(j)+1}^{\infty}\frac{\xi_{n}^{(j)}}{W(n-1)}\right\}$$

(note that all the sums above all have continuous distributions) and in particular, by (29),

$$\lim_{n\to\infty}\sigma_{n}(i_{k})-\tau_{n_{k}d}<\min_{j\neq i_{k}}\{\lim_{n\to\infty}\sigma_{n}(j)-\tau_{n_{k}d}\}.$$

That is, the remaining time needed to visit $e_{i_{k}}$ i.o. is strictly less than that needed for any other edge. By (27) in Proposition 2.6, this is equivalent to saying that only balls of color $i_{k}$ are taken infinitely often (after time $n_{k}$). Therefore, for any $k\geq 1$,

$$\mathbb{P}_{1}^{W}(\mathcal{M}\mid\mathcal{F}_{\tau_{n_{k}d}})\geq\frac{1}{N_{c}},\quad\forall k\geq 1.$$

We then conclude that $\mathbb{P}_{1}^{W}(\mathcal{M})=1$ by Levy’s 0-1 law. ∎

We will show in Section 5 that one can apply a similar argument if $\delta_{n}$ is small in the sense of (13) or (14), which enables us to prove Theorem 1.5.

2.4. Coupling

Proposition 1.1 is proved by coupling. Let $(B_{n},R_{n})_{n\in\mathbb{N}}$ be an IUM with reinforcement sequence $\{W(n)\}_{n\in\mathbb{N}}$ and interaction parameter $p$. We define a new urn process $(\tilde{B}_{n},\tilde{R}_{n})_{n\in\mathbb{N}}$ as follows, where $\tilde{B}_{n}(i)$ and $\tilde{R}_{n}(i)$ ($i=1,2$) denote the number of black and red balls in the $i$-th urn at time $n$, respectively.

Similarly, we write $\tilde{B}_{n}:=(\tilde{B}_{n}(1),\tilde{B}_{n}(2))$, $\tilde{R}_{n}:=(\tilde{R}_{n}(1),\tilde{R}_{n}(2))$ and $\tilde{B}_{n}^{*}:=\tilde{B}_{n}(1)+\tilde{B}_{n}(2)$, $\tilde{R}_{n}^{*}:=\tilde{R}_{n}(1)+\tilde{R}_{n}(2)$. The initial composition is given by $(\tilde{B}_{0},\tilde{R}_{0})\in\mathbb{N}_{+}^{4}$. For any $n\in\mathbb{N}$, at time step $2n+1$, we add a black ball to the first urn with probability

$$\frac{W(\tilde{B}_{2n}(1))}{W(\tilde{B}_{2n}(1))+W(\tilde{R}_{2n}^{*})},$$

otherwise, we add a red ball to the first urn; at time step $2n+2$, we add a black ball to the second urn with probability

$$\frac{W(\tilde{B}_{2n}(2))}{W(\tilde{B}_{2n}(2))+W(\tilde{R}_{2n+1}^{*})},$$

otherwise, we add a red ball to the second urn.

In words, red balls are always drawn from all the urns combined, black balls are always drawn from the urn alone. Compared to the IUM, it is natural to expect that there will be more red balls and fewer black balls if $\{W(n)\}_{n\in\mathbb{N}}$ is non-decreasing.

Lemma 2.7.

Assume that $\{W(n)\}_{n\in\mathbb{N}}$ is non-decreasing and $(\tilde{B}_{0},\tilde{R}_{0})=(B_{0},R_{0})$. Then we can define the two urn processes $(B_{n},R_{n})_{n\in\mathbb{N}}$ and $(\tilde{B}_{n},\tilde{R}_{n})_{n\in\mathbb{N}}$ above on the same probability space such that

$$\tilde{R}_{2n}(i)\geq R_{n}(i),\ \tilde{B}_{2n}(i)\leq B_{n}(i),\quad\forall n\in\mathbb{N},i=1,2.$$ (30)

Lemma 2.8.

Assume that $\{W(n)\}_{n\in\mathbb{N}}$ is non-decreasing and satisfies (7) and (9). Then, there exists positive constants $\varepsilon_{1}\in(0,1)$ and $\kappa\in\mathbb{N}_{+}$ such that if $n\geq\kappa$ and $\tilde{R}_{2n}^{*}<\varepsilon_{1}n$, then

$$\mathbb{P}(\lim_{\ell\to\infty}\tilde{R}^{*}_{\ell}<\infty\mid\tilde{\mathcal{F}}_{2n})>\frac{1}{10e},$$

where $\tilde{\mathcal{F}}_{n}:=\sigma(\tilde{B}_{m}(i),\tilde{R}_{m}(i):1\leq i\leq 2,m\leq n)$.

Lemmas 2.7 and 2.8 will be proved in Section 6. Notice that they also hold if we interchange the colors black/red. Using Lemmas 2.7 and 2.8, we can prove Proposition 1.1.

Proof of Proposition 1.1.

Fix $\varepsilon\in(0,1)$, let $\varepsilon_{1},\kappa$ be as in Lemma 2.8 for $\{W(n)\}_{n\geq N_{1}}$. We define an infinite sequence of stopping times $\{T_{n}\}_{n\in\mathbb{N}}$ as follows. Let $T_{0}:=\kappa$ and for any $n\in\mathbb{N}$,

$$T_{n+1}:=\inf\{m>T_{n}:N_{1}\leq\min(B^{*}_{2m},R^{*}_{2m})<\varepsilon_{1}m\}$$

with the convention that $\inf\emptyset=\infty$. If $T_{n}=\infty$ for some $n$, then we set $T_{k}:=\infty$ for all $k>n$. By Lemma 2.7 and Lemma 2.8, for any $n\geq 1$,

$$\mathbb{P}_{p}^{W}(\mathcal{D}^{c}\cup\mathcal{M}\mid\mathcal{G}_{T_{n}})\mathds{1}_{\{T_{n}<\infty\}}\geq\mathbb{P}_{p}^{W}(\mathcal{M}\mid\mathcal{G}_{T_{n}})\mathds{1}_{\{T_{n}<\infty\}}\geq\frac{1}{10e}\mathds{1}_{\{T_{n}<\infty\}}.$$

On the other hand, on the event $\{T_{n}=\infty\}$, either

1. $\min(B^{*}_{2m},R^{*}_{2m})<N_{1}$ for all $m$, in which case the monopoly occurs, or

2. $\min(B^{*}_{2m},R^{*}_{2m})\geq\varepsilon_{1}m$ for all large $m$, in which case domination does not occur.

Therefore, $\mathbb{P}_{p}^{W}(\mathcal{D}^{c}\cup\mathcal{M}\mid\mathcal{G}_{T_{n}})\mathds{1}_{\{T_{n}=\infty\}}=\mathds{1}_{\{T_{n}=\infty\}}$. Thus, for any $n\geq 1$,

$$\mathbb{P}_{p}^{W}(\mathcal{D}^{c}\cup\mathcal{M}\mid\mathcal{G}_{T_{n}})>\frac{1}{10e}.$$

By Levy’s 0-1 law, $\mathbb{P}_{p}^{W}(\mathcal{D}^{c}\cup\mathcal{M})=1$ which completes the proof since $\mathcal{M}\subset\mathcal{D}$. ∎

2.5. Organization of the remaining of this paper

Section 3 concerns the results on the deterministic nonlinear system (15): Proposition 2.3 and Proposition 2.4 are proved.

Section 4 develops the framework relating the behavior of $(x_{n},y_{n})$ to the planar system (15): We prove Proposition 2.1, Theorem 1.2 and Theorem 1.3.

Theorem 1.5 is proved in Section 5. Section 6 is devoted to the proofs of Lemma 2.7 and Lemma 2.8. Some open problems are presented in the last section.

3. Results on the deterministic dynamical system

We assume that $\alpha>1$ and $p\in[0,1]$. The following two functions will be used frequently:

$$h(t):=\frac{t^{\alpha}}{t^{\alpha}+(1-t)^{\alpha}},\quad g(t):=-t+(1-p)h(t)+\frac{p}{2},\quad t\in[0,1].$$ (31)

In particular, $h^{\prime}(t)=\alpha f(t)$ and $g^{\prime}(t)=-1+\alpha(1-p)f(t)$ where $f(t)$ is defined by (19).

Lemma 3.1.

The equation $g(u)=0$ has a solution on $[0,1/2)$ if and only if $p<(\alpha-1)/\alpha$. If one solution exists, then it is unique.

Proof.

The assertion is trivial for $p=1$. We now assume that $p<1$. Note that $g(0)=p/2,g(1/2)=0$. Observe that $g^{\prime}(t)$ is a strictly increasing function on $[0,1/2]$ with $g^{\prime}(0)=1$ and $g^{\prime}(1/2)=\alpha(1-p)-1$.

If $g^{\prime}(1/2)>0$, then there exists a unique $t_{0}\in(0,1/2)$ such that $g^{\prime}(t_{0})=0$. The function $g$ is strictly decreasing on $[0,t_{0}]$ and strictly increasing on $[t_{0},1/2]$. In particular, there exists a unique solution to $g(u)=0$ on $[0,1/2)$. See Figure 4(a) for the case $\alpha=2$ and $p=0.25$.

If $g^{\prime}(1/2)\leq 0$, then $g$ is strictly decreasing on $[0,1/2]$, and thus, there is no solution to $g(u)=0$ on $[0,1/2)$. The case $\alpha=2$ and $p=0.75$ is plotted in Figure 4(b). ∎

(a) $\alpha=2,p=0.25$

(b) $\alpha=2,p=0.75$

Figure 4. $g(u)=-u+\frac{(1-p)u^{\alpha}}{u^{\alpha}+(1-u)^{\alpha}}+\frac{p}{2}$ on $[0,1]$

We denote the unique solution by $u_{\alpha,p}$ when it exists. Note that $u_{\alpha,p}=0$ only when $p=0$.

The proof of Proposition 2.3 will need the following three technical lemmas.

Lemma 3.2.

(i) If $\alpha>1$ and $p=0$, then $\Lambda^{(\alpha)}_{0}=\{0,1/2,1\}\times\{0,1/2,1\}$.
(ii) For $\alpha>1$ and $p>0$, one has

$$\Lambda^{(\alpha)}_{p}\bigcap\left([0,\frac{1}{2}]^{2}\cup[\frac{1}{2},1]^{2}\cup\partial[0,1]^{2}\right)=\{(0,0),(\frac{1}{2},\frac{1}{2}),(1,1)\}.$$

Proof.

(i) If $z\in[1/2,1]$, then

$$\frac{z^{\alpha}}{z^{\alpha}+(1-z)^{\alpha}}=\frac{1}{1+(\frac{1}{z}-1)^{\alpha}}\geq\frac{1}{1+(\frac{1}{z}-1)}=z,$$ (32)

where the equality holds if and only if $z=1/2$ or $z=1$. The inequality (32) is reversed if $z\in[0,1/2]$ where the equality holds if and only if $z=0$ or $z=1/2$. This proves (i).

(ii) If $(x,y)\in[1/2,1]^{2}$ is an equilibrium, then by (32), we have

$$0=-x+\frac{(1-p)x^{\alpha}}{x^{\alpha}+(1-x)^{\alpha}}+\frac{p(\frac{x+y}{2})^{\alpha}}{(\frac{x+y}{2})^{\alpha}+(1-\frac{x+y}{2})^{\alpha}}\geq-x+(1-p)x+\frac{p(x+y)}{2}=\frac{p(y-x)}{2},$$

and similarly,

$$0=-y+\frac{(1-p)y^{\alpha}}{y^{\alpha}+(1-y)^{\alpha}}+\frac{p(x+y)^{\alpha}}{(x+y)^{\alpha}+(2-x-y)^{\alpha}}\geq\frac{p(x-y)}{2}.$$

Thus, $(x,y)=(1/2,1/2)$ or $(1,1)$. Similarly, if $(x,y)\in[0,1/2]^{2}$ is an equilibrium, then $(x,y)=(0,0)$ or $(1/2,1/2)$. Moreover, it is easy to see that if a boundary point $(x,y)\in\partial[0,1]^{2}$ is an equilibrium, then $(x,y)=(0,0)$ or $(1,1)$. (ii) is then proved. ∎

Lemma 3.3.

Define

$$\beta(t):=\frac{1}{2}\frac{(1+t)^{\alpha}-(1-t)^{\alpha}}{(1+t)^{\alpha}+(1-t)^{\alpha}}+\frac{1}{2}\frac{t^{\alpha}}{t^{\alpha}+(1-t)^{\alpha}}-t,\quad t\in(0,\frac{1}{2}).$$ (33)

If $\alpha\geq 2$, then for any $t\in(0,1/2)$, one has $\beta(t)>0$.

Proof.

Observe that on the interval $(0,1/2)$, the function $h$ defined in (31) is convex and $h(t)<t$. Thus, if $t\in(1/4,1/2)$, then

$$\frac{h(t)+h(1-2t)}{2}-h(\frac{1-t}{2})\geq 0>\frac{1}{2}(h(1-2t)-1+2t).$$

In particular,

$$\beta(t)=\frac{1}{2}h(t)-h(\frac{1-t}{2})+\frac{1}{2}-t>0,\quad t\in(\frac{1}{4},\frac{1}{2}).$$

Since $\alpha\geq 2$, one has

$$(1+\frac{2t}{1-t})^{\alpha}=\left(1+\frac{4t}{1-t}+\frac{4t^{2}}{(1-t)^{2}}\right)^{\frac{\alpha}{2}}\geq 1+\frac{2\alpha t}{1-t}+\frac{2\alpha t^{2}}{(1-t)^{2}},\quad t\in(0,\frac{1}{2}).$$

Thus, for $\alpha\geq 9/4$ and $t\in(0,1/4]$, we have

$$\frac{(1+t)^{\alpha}-(1-t)^{\alpha}}{(1+t)^{\alpha}+(1-t)^{\alpha}}=\frac{(1+\frac{2t}{1-t})^{\alpha}-1}{(1+\frac{2t}{1-t})^{\alpha}+1}\geq\frac{\alpha t}{(1-t)^{2}+\alpha t}\geq 2t,$$

where in the last inequality we used that

$$\alpha\geq\frac{9}{4}\geq\frac{2}{1-(\frac{t}{1-t})^{2}}=\frac{2(1-t)^{2}}{1-2t}.$$

If $\alpha\in[2,9/4)$ and $t\in(0,1/4]$, then

$$\beta(t) \geq\frac{1}{2}\frac{(1+t)^{2}-(1-t)^{2}}{(1+t)^{2}+(1-t)^{2}}+\frac{1}{2}\frac{t^{\frac{9}{4}}}{t^{\frac{9}{4}}+(1-t)^{\frac{9}{4}}}-t$$

$$=\frac{t^{\frac{9}{4}}}{2}\left(\frac{-2t^{\frac{3}{4}}}{1+t^{2}}+\frac{1}{t^{\frac{9}{4}}+(1-t)^{\frac{9}{4}}}\right)\geq\frac{t^{\frac{9}{4}}}{2}\left(\frac{-\sqrt{2}}{2}+1\right)>0,$$

which completes the proof. ∎

Lemma 3.4.

Let $\alpha>2$ and $p\in[1/2,(\alpha-1)/\alpha)$. One has

$$\Lambda^{(\alpha)}_{p}=\{(0,0),(\frac{1}{2},\frac{1}{2}),(1,1),(u_{\alpha,p},1-u_{\alpha,p}),(1-u_{\alpha,p},u_{\alpha,p})\};$$

Proof.

By Lemma 3.1, the set of equilibria on the line $x+y=1$ is given by

$$\{(\frac{1}{2},\frac{1}{2}),(u_{\alpha,p},1-u_{\alpha,p}),(1-u_{\alpha,p},u_{\alpha,p})\}.$$

Thus, it remains to show that

$$\Lambda^{(\alpha)}_{p}\backslash(\{(x,y):x+y=1\}\cup\{(0,0),(1,1)\})=\emptyset.$$

Observe that if $(x,y)\in\Lambda^{(\alpha)}_{p}$, then $(1-x,1-y)\in\Lambda^{(\alpha)}_{p}$. Therefore, by Lemma 3.2, it suffices to prove that there is no equilibrium $(\tilde{x},\tilde{y})$ such that $0<\tilde{x}<1/2<\tilde{y}<1$ and $\tilde{z}:=(\tilde{x}+\tilde{y})/2\in(1/2,3/4)$. We argue by contradiction and assume that $(\tilde{x},\tilde{y})$ is such an equilibrium.

Since $(\tilde{x},\tilde{y})\in\Lambda^{(\alpha)}_{p}$,

$$-2\tilde{z}+\frac{(1-p)(2\tilde{z}-\tilde{y})^{\alpha}}{(2\tilde{z}-\tilde{y})^{\alpha}+(1+\tilde{y}-2\tilde{z})^{\alpha}}+\frac{(1-p)\tilde{y}^{\alpha}}{\tilde{y}^{\alpha}+(1-\tilde{y})^{\alpha}}+\frac{2p\tilde{z}^{\alpha}}{\tilde{z}^{\alpha}+(1-\tilde{z})^{\alpha}}=0.$$ (34)

Now fix $z\in(1/2,3/4)$, the function

$$J(y):=\frac{(2z-y)^{\alpha}}{(2z-y)^{\alpha}+(1+y-2z)^{\alpha}}+\frac{y^{\alpha}}{y^{\alpha}+(1-y)^{\alpha}},\quad y\in[2z-\frac{1}{2},1],$$

has derivative $J^{\prime}(y)=-\alpha(f(2z-y)-f(y))$. Observe that $|2z-y-1/2|\leq|y-1/2|$ and thus $J^{\prime}(y)\leq 0$ (note that $f(x)$ decreases as $|x-1/2|$ increases). In particular, for any $y\in[2z-1/2,1]$, $J(y)\geq J(1)$, i.e.

$$\quad-2z+\frac{(1-p)(2z-y)^{\alpha}}{(2z-y)^{\alpha}+(1+y-2z)^{\alpha}}+\frac{(1-p)y^{\alpha}}{y^{\alpha}+(1-y)^{\alpha}}+\frac{2pz^{\alpha}}{z^{\alpha}+(1-z)^{\alpha}}$$ (35)

$$\geq-2z+\frac{(1-p)(2z-1)^{\alpha}}{(2z-1)^{\alpha}+(2-2z)^{\alpha}}+(1-p)+\frac{2pz^{\alpha}}{z^{\alpha}+(1-z)^{\alpha}}$$

$$\stackrel{{\scriptstyle t=2z-1}}{{=}}(2p-1)(\beta(t)+t-h(t))+\beta(t)>0,$$

where $\beta(t)$ is defined in (33) and we used Lemma 3.3 in the last inequality. However, this contradicts (34). ∎

Now we are ready to prove Proposition 2.3.

Proof of Proposition 2.3.

It is direct to check that $\Lambda^{(\alpha)}_{p}$ includes $(0,0),(1/2,1/2),(1,1)$. The assertions (i) and (ii) follow from Lemma 3.2 and Lemma 3.4, respectively.

By symmetry and Lemma 3.2, to prove (iii) and (iv), we need to show that the set

$$\Lambda^{(\alpha),+}_{p}:=\{(x,y)\in\Lambda^{(\alpha)}_{p}:0<x<1/2<y<1,x+y>1\}$$

is finite if $p>0$ and is empty if $p\geq(\alpha-1)/\alpha$.
(iii) Assume that $p\geq(\alpha-1)/\alpha$ and $(\tilde{x},\tilde{y})\in\Lambda^{(\alpha),+}_{p}$, and in particular, $\tilde{z}:=(\tilde{x}+\tilde{y})/2\in(1/2,3/4)$. Recall $g$ and $h$ defined in (31). We see from the proof of Lemma 3.1 that $g(\tilde{x})>0$ and

$$F_{p,1}^{{\alpha}}(\tilde{x},\tilde{y})=g(\tilde{x})+p\left(\frac{\tilde{z}^{\alpha}}{\tilde{z}^{\alpha}+(1-\tilde{z})^{\alpha}}-\frac{1}{2}\right)>0,$$ (36)

which contradicts our assumption, and proves (iii).
(iv) Assume that $p\in(0,(\alpha-1)/\alpha)$. Using arguments in the proof of Lemma 3.1, we see that there exists $t_{1}\in(1/2,1)$ such that $g(u)$ is increasing on $[1/2,t_{1}]$ and decreasing on $[t_{1},1]$, see e.g. Figure 4(a). For $t,z\in[0,1]$, define

$$\tilde{F}(t,z):=g(t)+p(h(z)-\frac{1}{2}).$$

Since $g(t_{1})>g(1/2)=0$ and $g(1)=-p/2$, there exists an $\varepsilon>0$ such that for any $z\in(1/2-\varepsilon,1]$, there exists a unique $y\in(t_{1},1]$ such that $\tilde{F}(y,z)=0$. By the analytic implicit function theorem, see e.g. [15, Theorem 6.1.2], this unique $y$ can be written as $y=q(z)$ where $q$ is analytic and increasing on $(1/2-\varepsilon,1)$ and

$$q(\frac{1}{2})>t_{1},\quad q(1)=1,\quad q^{\prime}(z)=\frac{p\alpha f(z)}{1-\alpha(1-p)f(q(z))}=\frac{p\alpha f(z)}{-g^{\prime}(q(z))}>0.$$ (37)

Observing that $f(z)$ decreases as $|z-1/2|$ increases, we see that $2-q^{\prime}(z)$ is increasing on $[1/2,1)$. Thus, using the convexity of $2z-q(z)$ and that $1-q(1/2)<1$, one has, for any $z\in[1/2,1)$,

$$0<1-q(z)\leq 2z-q(z)<2-q(1)=1.$$ (38)

Now we consider the analytic function $\widehat{F}(z):=\tilde{F}(2z-q(z),z)$ on $(1/2-\varepsilon,1)$. It is non-constant since its derivative equals

$$\widehat{F}^{\prime}(z)=\left(\alpha(1-p)f(2z-q(z))-1\right)(2-q^{\prime}(z))-\alpha pf(z)$$ (39)

which converges to $-2$ as $z\to 1$ (observe that $f(z)\to 0$ and $q^{\prime}(z)\to 0$ as $z\to 1$). As a non-constant analytic function, $\widehat{F}(z)$ has finite zeros on $[1/2,3/4]$. Observe that if $(\tilde{x},\tilde{y})\in\Lambda^{(\alpha),+}_{p}$ with $\tilde{z}:=(\tilde{x}+\tilde{y})/2>1/2$, then $\tilde{y}=q(\tilde{z})$ and $\tilde{x}=2\tilde{z}-q(\tilde{z})$. In particular, $\tilde{z}$ is a zero of $\widehat{F}(z)$ on $[1/2,3/4]$. Since $\tilde{z}$ can take only finitely many values, (iv) is proved. ∎

For the proof of Proposition 2.4, we will need the following auxiliary lemmas.

Lemma 3.5.

Assume that $t\in(0,1/6]$. One has,
(i) if $\alpha\geq 2$, then $2\alpha h(t)\leq t$;
(ii) if $\alpha\in(1,2)$, then

$$h(t)\leq\frac{1}{\alpha}(1-\frac{1}{4}(\alpha-1))t.$$

Proof.

Fix $x=1/t\geq 6$, define

$$\phi(u):=u\log(x-1)-\log(2ux-1),\quad u\in[2,\infty),$$

and

$$\varphi(u):=u\log(x-1)-\log(\frac{u}{1-\frac{1}{4}(u-1)}x-1),\quad u\in[1,2].$$

Then $\phi(2)>0$ and $\varphi(1)=0$. Moreover,

$$\phi^{\prime}(u)\geq\log(x-1)-\frac{2x}{4x-1}>0,\quad u\in[2,\infty),$$

and for $u\in[1,2]$,

$$\varphi^{\prime}(u) =\log(x-1)-\frac{5}{5-u}\frac{x}{ux-1+\frac{1}{4}(u-1)}\geq\log(x-1)-\frac{5}{5-u}\frac{x}{ux-1}$$

$$\geq\log(x-1)-\frac{5}{4}\frac{x}{x-1}\geq\log 5-\frac{3}{2}>0,$$

where we used that $(5-u)(ux-1)$ achieves its minimum on $[1,2]$ at $u=1$. Therefore, $\phi(u)>0$ on $[2,\infty)$ and $\varphi(u)>0$ on $(1,2]$. It remains to notice that (i) and (ii) follow from that $\phi(\alpha)>0$ and that $\varphi(\alpha)>0$, respectively. ∎

Lemma 3.6.

For $\alpha>1$ and $p\in(0,\alpha^{-1}\min((\alpha-1)/6,1/3)]$, let

$$v_{\alpha,p}:=\frac{1}{2}-\frac{1}{2}\sqrt{\frac{(\alpha-1)(1-p)}{\alpha-1+p+\alpha p-\alpha p^{2}}}=\frac{1}{2}-\frac{1}{2}\sqrt{1-\frac{\alpha p(2-p)}{\alpha-1+p+\alpha p-\alpha p^{2}}}.$$ (40)

Then $g(v_{\alpha,p})<0$, and in particular, $u_{\alpha,p}<v_{\alpha,p}$.

Proof.

For $x\in(0,1)$, one has $1-x<\sqrt{1-x}<1-x/2$, and in particular,

$$\frac{\alpha p(2-p)}{4(\alpha-1+p+\alpha p-\alpha p^{2})}<v_{\alpha,p}<\frac{\alpha p(2-p)}{2(\alpha-1+p+\alpha p-\alpha p^{2})}<\frac{1}{6},$$ (41)

where the last inequality follows from that $p\leq\alpha^{-1}(\alpha-1)/6$. Observe that

$$g(t)=-(1-p)(t-h(t))+p(\frac{1}{2}-t).$$

Then $g(v_{\alpha,p})<0$ if and only if

$$\frac{1}{p}(v_{\alpha,p}-h(v_{\alpha,p}))>\frac{1}{1-p}(\frac{1}{2}-v_{\alpha,p})=\frac{1}{2(1-p)}\sqrt{\frac{(\alpha-1)(1-p)}{\alpha-1+p+\alpha p-\alpha p^{2}}}.$$ (42)

(i) If $\alpha\geq 2$, by Lemma 3.5 (i) and (41),

$$\frac{1}{p}(v_{\alpha,p}-h(v_{\alpha,p}))>(1-\frac{1}{2\alpha})\frac{\alpha(2-p)}{4(\alpha-1+p+\alpha p-\alpha p^{2})}>\frac{(\alpha-\frac{1}{2})\sqrt{1-p}}{2(\alpha-1+p+\alpha p-\alpha p^{2})},$$

where we used that $1-p/2>\sqrt{1-p}$. To prove (42), it suffices to show that

$$(\alpha-\frac{1}{2})(1-p)-(\alpha-1)\sqrt{1+\frac{p(\alpha+1)}{\alpha-1}}\geq 0.$$

Using that $\sqrt{1+x}\leq 1+x/2$ for $x\geq 0$, we see the left-hand side is lower bounded by

$$(\alpha-\frac{1}{2})(1-p)-(\alpha-1+\frac{p(\alpha+1)}{2})=\frac{1-3\alpha p}{2}\geq 0,$$

which completes the proof for the case $\alpha\geq 2$.
(ii) If $\alpha\in(1,2)$, similarly, by Lemma 3.5 (ii) and (41), it suffices to show that

$$\frac{5(\alpha-1)(1-p)}{4}-(\alpha-1+\frac{p(\alpha+1)}{2})\geq 0.$$

Using that $p\leq\alpha^{-1}(\alpha-1)/6$, we see that the left-hand side (LHS) satisfies

$$\text{LHS}=\frac{\alpha-1}{4}-(\frac{5(\alpha-1)}{4}+\frac{\alpha+1}{2})p \geq\frac{(\alpha-1)}{6\alpha}(\frac{3\alpha}{2}-\frac{5(\alpha-1)}{4}-\frac{\alpha+1}{2})$$

$$=\frac{(\alpha-1)}{24\alpha}(3-\alpha)>0.$$

Thus, in either case, (42) holds, and equivalently, $g(v_{\alpha,p})<0$. Since $p\leq\alpha^{-1}(\alpha-1)/6$, from the proof of Lemma 3.1, we see that $u_{\alpha,p}$ exists and $u_{\alpha,p}<v_{\alpha,p}<1/2$. ∎

We now prove Proposition 2.4.

Proof of Proposition 2.4.

(i) Notice that $f(1/2)=1$ and $f(u_{\alpha,p})=f(1-u_{\alpha,p})$. By (20), we need to prove that

$$-1+\alpha p+\alpha(1-p)f(u_{\alpha,p})<0.$$ (43)

The case $p=0$ is trivial since $u_{\alpha,0}=0$. For $p>0$, one has $u_{\alpha,p}>0$. Moreover, using that $g(u_{\alpha,p})=0$ (and thus $g(1-u_{\alpha,p})=0$), we have

$$\frac{u_{\alpha,p}^{\alpha-1}}{u_{\alpha,p}^{\alpha}+(1-u_{\alpha,p})^{\alpha}}=\frac{u_{\alpha,p}-\frac{p}{2}}{(1-p)u_{\alpha,p}},\quad\frac{(1-u_{\alpha,p})^{\alpha-1}}{u_{\alpha,p}^{\alpha}+(1-u_{\alpha,p})^{\alpha}}=\frac{1-u_{\alpha,p}-\frac{p}{2}}{(1-p)(1-u_{\alpha,p})}.$$

Therefore, using (19), we can write

$$f(u_{\alpha,p})=\frac{(u_{\alpha,p}-\frac{p}{2})(1-u_{\alpha,p}-\frac{p}{2})}{(1-p)^{2}u_{\alpha,p}(1-u_{\alpha,p})}.$$

Then (43) is equivalent to

$$u_{\alpha,p}<\frac{1}{2}-\frac{1}{2}\sqrt{\frac{(\alpha-1)(1-p)}{\alpha-1+p+\alpha p-\alpha p^{2}}},$$

which follows from Lemma 3.6.
(ii) Recall $F^{(\alpha)}_{p}$ defined in (16). If $\alpha\geq 3$ and $p=1/2-\alpha^{-1}\log\alpha$, then for any $y\in[0,1]$, one has

$$F_{p,1}^{(\alpha)}(\frac{1}{2}-\frac{\log\alpha}{2\alpha},y) <-\frac{1}{2}+\frac{\log\alpha}{2\alpha}+(\frac{1}{2}+\frac{\log\alpha}{\alpha})(\frac{1-\alpha^{-1}\log\alpha}{1+\alpha^{-1}\log\alpha})^{\alpha}+\frac{1}{2}-\frac{\log\alpha}{\alpha}$$

$$\leq-\frac{\log\alpha}{2\alpha}+(\frac{1}{2}+\frac{\log 3}{3})(1-\frac{6\log\alpha}{(3+\log 3)\alpha})^{\alpha}$$

$$\leq-\frac{1}{2}\alpha^{-\frac{6}{3+\log 3}}\left(\alpha^{\frac{3-\log 3}{3+\log 3}}\log\alpha-1-\frac{2\log 3}{3}\right)<0,$$

where we used that $1-t\leq e^{-t}$ for $t\in[0,1]$ and that

$$\frac{\log\alpha}{\alpha}\leq\frac{\log 3}{3},\quad\alpha^{\frac{3-\log 3}{3+\log 3}}\log\alpha-1-\frac{2\log 3}{3}\geq 3^{\frac{3-\log 3}{3+\log 3}}\log 3-1-\frac{2\log 3}{3}>0.$$

If $y\in[1/2,1]$, then for any $p\leq 1/2-\alpha^{-1}\log\alpha$, one still has

$$F_{p,1}^{(\alpha)}(\frac{1}{2}-\frac{\log\alpha}{2\alpha},y)<0,$$

since the left-hand side is an increasing function in $p$. Moreover, for any $x\in[0,1/2]$ and $p\leq 1/2-\alpha^{-1}\log\alpha$, one has

$$F_{p,2}^{(\alpha)}(x,\frac{1}{2}+\frac{\log\alpha}{2\alpha})=-F_{p,1}^{(\alpha)}(\frac{1}{2}-\frac{\log\alpha}{2\alpha},1-x)>0.$$

On the other hand, since $p>0$, one has $F_{p,1}^{(\alpha)}(0,y)>0$ and $F_{p,2}^{(\alpha)}(x,1)<0$ for $x\in[0,1/2]$ and $y\in[1/2,1]$. Therefore, the maximum of $L^{(\alpha)}_{p}$ on the square $[0,1/2-(2\alpha)^{-1}\log\alpha]\times[1/2+(2\alpha)^{-1}\log\alpha,1]$ can only be achieved at some interior point, which is asymptotically stable.

(iii) We can assume that $\alpha\geq 2$ and thus $p<(\alpha-1)/\alpha$. We shall use the functions

$$\tilde{F}(t,z)=g(t)+p(h(z)-\frac{1}{2})$$

and $q(z)$ defined in the proof of Proposition 2.3 (iv): For any $z\in[1/2,1]$, we have $\tilde{F}(q(z),z)=0$ and $q(z)\in(t_{1},1]$ where $t_{1}\in(1/2,1)$ is such that $g$ is strictly increasing on $[1/2,t_{1}]$ and strictly decreasing on $[t_{1},1]$. By (32), if $z\in(1/2,1)$, then

$$g(q(z))=-p(h(z)-\frac{1}{2})<g(z),$$

whence we have $q(z)>z$ for $z\in(1/2,1)$. Fix $\delta>0$ such that $2\delta<\min(1/2-p,p)$, then for any $z\in[1/2+\delta,1]$,

$$q(z)=(1-p)h(q(z))+ph(z)>h(z)\geq h(1/2+\delta)=\frac{(1/2+\delta)^{\alpha}}{(1/2+\delta)^{\alpha}+(1/2-\delta)^{\alpha}}\to 1,$$

as $\alpha\to\infty$. In particular, we can choose a large $\alpha$ such that $q(z)>1-\delta$ for all $z\geq 1/2+\delta$. Therefore, for any $z\in[1/2+\delta,3/4-\delta]$,

$$0<2z-q(z)<2z-1+\delta\leq\frac{1}{2}-\delta.$$ (44)

Note that

$$\alpha f(t)=\frac{\alpha t^{\alpha-1}(1-t)^{\alpha-1}}{(t^{\alpha}+(1-t)^{\alpha})^{2}}\to 0,\quad\text{as}\ \alpha\to\infty,$$ (45)

uniformly on $[0,1/2-\delta]\cup[1/2+\delta,1]$. Recall that $\widehat{F}(z)=\tilde{F}(2z-q(z),z)$. Using (37), (39) and (44), we have

$$\lim_{\alpha\to\infty}\widehat{F}^{\prime}(z)=-2,\quad\text{uniformly on}\ [\frac{1}{2}+\delta,\frac{3}{4}-\delta].$$ (46)

Moreover, by the choice of $\delta$, one has

$$\lim_{\alpha\to\infty}\widehat{F}(\frac{1}{2}+\delta)=p-2\delta>0,\quad\lim_{\alpha\to\infty}\widehat{F}(\frac{3}{4}-\delta)=p-\frac{1}{2}+2\delta<0.$$ (47)

Therefore, for all large $\alpha$, we have $\widehat{F}(1/2+\delta)>0$ and $\widehat{F}(3/4-\delta)<0$, and by (46), there exists a unique $z_{\alpha}\in(1/2+\delta,3/4-\delta)$ such that $\widehat{F}(z_{\alpha})=0$. In particular, $s_{\alpha}:=(2z_{\alpha}-q(z_{\alpha}),q(z_{\alpha}))$ is an equilibrium. Observe that by (46) and (47),

$$\lim_{\alpha\to\infty}z_{\alpha}=\frac{3}{4}-\delta-\frac{1}{2}(\frac{1}{2}-2\delta-p)=\frac{1+p}{2},$$

and thus $\lim_{\alpha\to\infty}s_{\alpha}=(p,1)$. Since none of $2z_{\alpha}-q(z_{\alpha}),z_{\alpha}$ and $q(z_{\alpha})$ are in the interval $[1/2-\delta,1/2+\delta]$, by (45), we have

$$\lim_{\alpha\to\infty}\left(\alpha f(2z_{\alpha}-q(z_{\alpha})),\alpha f(z_{\alpha}),\alpha f(q(z_{\alpha}))\right)=(0,0,0),$$

which, by (20), implies that $\lim_{\alpha\to\infty}\lambda_{+}(s_{\alpha})=-1$. Thus, $\lambda_{+}(s_{\alpha})<0$ for all large $\alpha$. ∎

4. Stochastic approximation algorithm

Proof of Proposition 2.1.

We assume that $W(n)$ is a polynomial of degree $\alpha$ as in (8). The case of power function reinforcement can be proved similarly. Note that

$$\frac{W(B_{n}(1))}{n^{\alpha}}=x_{n}+O(\frac{1}{n}),\quad\frac{W(B_{n}^{*})}{n^{\alpha}}=x_{n}+y_{n}+O(\frac{1}{n}),$$

thus, by (2), for any $n\in\mathbb{N}$,

$$\mathbb{E}(\xi_{n+1}(1)\mid\mathcal{G}_{n}) =\frac{(1-p)W(B_{n}(1))}{W(B_{n}(1))+W(R_{n}(1))}+\frac{pW(B_{n}^{*})}{W(B_{n}^{*})+W(R_{n}^{*})}$$ (48)

$$=\frac{(1-p)x_{n}^{\alpha}+O(\frac{1}{n})}{x_{n}^{\alpha}+(1-x_{n})^{\alpha}+O(\frac{1}{n})}+\frac{p(x_{n}+y_{n})^{\alpha}+O(\frac{1}{n})}{(x_{n}+y_{n})^{\alpha}+(2-x-y)^{\alpha}+O(\frac{1}{n})}$$

$$=\frac{(1-p)x_{n}^{\alpha}}{x_{n}^{\alpha}+(1-x_{n})^{\alpha}}+\frac{p(x_{n}+y_{n})^{\alpha}}{(x_{n}+y_{n})^{\alpha}+(2-x_{n}-y_{n})^{\alpha}}+O(\frac{1}{n}).$$

By (3), we have

$$x_{n+1}-x_{n}=\frac{B_{n+1}(1)-B_{n}(1)-x_{n}}{n+1+B_{0}(1)+R_{0}(1)}=\frac{1}{n+1}(F^{(\alpha)}_{p,1}(x_{n},y_{n})+\varepsilon_{n+1}(1)+r_{n+1}(1)),$$

where

$$\varepsilon_{n+1}(1):=\frac{n+1}{n+1+B_{0}(1)+R_{0}(1)}\left(\xi_{n+1}(1)-\mathbb{E}(\xi_{n+1}(1)\mid\mathcal{G}_{n})\right),$$ (49)

and, by (48),

$$r_{n+1}(1):=(n+1)\frac{-x_{n}+\mathbb{E}(\xi_{n+1}(1)\mid\mathcal{G}_{n})}{n+1+B_{0}(1)+R_{0}(1)}-F^{(\alpha)}_{p,1}(x_{n},y_{n})=O(\frac{1}{n}).$$

The equation for $y_{n+1}-y_{n}$ is proved similarly. ∎

Lemma 4.1.

Recall $(z_{n})_{n\in\mathbb{N}}$ defined by (21). Under $\mathbb{P}_{p}^{(1)}$, one has:
(i) The process $(z_{n})_{n\in\mathbb{N}}$ satisfies the following recursion:

$$z_{n+1}-z_{n}=\frac{1}{2n+2+B_{0}^{*}+R_{0}^{*}}\left(\tilde{\varepsilon}_{n+1}+\tilde{r}_{n+1}\right),\quad n\in\mathbb{N},$$

where $(\tilde{\varepsilon}_{n+1})_{n\in\mathbb{N}}$ and $(\tilde{r}_{n+1})_{n\in\mathbb{N}}$ are adapted sequence such that for all $n\in\mathbb{N}$,

$$\mathbb{E}(\tilde{\varepsilon}_{n+1}\mid\mathcal{G}_{n})=0,\ \mathbb{E}(\tilde{\varepsilon}_{n+1}^{2}\mid\mathcal{G}_{n})\leq(2+\frac{C}{2n+B_{0}^{*}+R_{0}^{*}})z_{n},\ |r_{n+1}|\leq\frac{Cz_{n}}{2n+B_{0}^{*}+R_{0}^{*}},$$ (50)

where $C$ is a positive constant. In particular, $(z_{n})_{n\in\mathbb{N}}$ converges a.s.
(ii) For $k\geq 1$, let $\tau(k):=\inf\{n\geq k:z_{n}\geq 2z_{k}\}$ with the convention that $\inf\emptyset=\infty$. Then, there exists a positive integer $K$ such that for all $k\geq K$,

$$\mathbb{P}(\{\tau(k)<\infty\}\bigcup\{\lim_{n\to\infty}z_{n}=0,\tau(k)=\infty\}\mid\mathcal{G}_{k})\leq\frac{4}{B_{k}^{*}}.$$

Proof.

(i) For $n\in\mathbb{N}$, let

$$\tilde{\varepsilon}_{n+1}:=\xi_{n+1}(1)+\xi_{n+1}(2)-\mathbb{E}(\xi_{n+1}(1)+\xi_{n+1}(2)\mid\mathcal{G}_{n})$$

and

$$r_{n+1}:=\mathbb{E}(\xi_{n+1}(1)+\xi_{n+1}(2)\mid\mathcal{G}_{n})-2z_{n}=(1-p)(x_{n}+y_{n}-2z_{n}).$$

Then

$$z_{n+1}-z_{n}=\frac{\xi_{n+1}(1)+\xi_{n+1}(2)-2z_{n}}{2n+2+B_{0}^{*}+R_{0}^{*}}=\frac{\tilde{\varepsilon}_{n+1}+\tilde{r}_{n+1}}{2n+2+B_{0}^{*}+R_{0}^{*}}.$$

By definition,

$$x_{n}+y_{n}-2z_{n}=\frac{B_{0}(2)+R_{0}(2)-B_{0}(1)-R_{0}(1)}{2n+B_{0}^{*}+R_{0}^{*}}(x_{n}-y_{n}).$$ (51)

Using that $x_{n}\leq(2n+B_{0}^{*}+R_{0}^{*})z_{n}/(n+B_{0}(1)+R_{0}(1))$ and $y_{n}\leq(2n+B_{0}^{*}+R_{0}^{*})z_{n}/(n+B_{0}(2)+R_{0}(2))$, we have

$$|r_{n+1}|=(1-p)|x_{n}+y_{n}-2z_{n}|\leq\frac{Cz_{n}}{2n+B_{0}^{*}+R_{0}^{*}}.$$ (52)

Note that conditional on $\mathcal{G}_{n}$, the two random variables $\xi_{n+1}(1)$ and $\xi_{n+1}(2)$ are independent Bernoulli random variables. Thus, $\mathbb{E}(\tilde{\varepsilon}_{n+1}\mid\mathcal{G}_{n})=0$ and

$$\mathbb{E}(\tilde{\varepsilon}_{n+1}^{2}\mid\mathcal{G}_{n})\leq\mathbb{E}(\xi_{n+1}(1)+\xi_{n+1}(2)\mid\mathcal{G}_{n})=2z_{n}+r_{n+1},$$

which implies (50) in virtue of (52). Now let $M_{0}:=0$ and

$$M_{n}:=\sum_{j=1}^{n}\frac{\tilde{\varepsilon}_{j}}{2j+B_{0}^{*}+R_{0}^{*}},\quad n\geq 1.$$

Then, by (50), the process $(M_{n})_{n\in\mathbb{N}}$ is a $L^{2}$-bounded martingale, and thus converges a.s. Moreover,

$$\sum_{n=1}^{\infty}\frac{|\tilde{r}_{n}|}{2n+B_{0}^{*}+R_{0}^{*}}<\infty.$$

These show that $(z_{n})_{n\in\mathbb{N}}$ converges a.s..

(ii) For any $k\geq 1$, by (50),

$$\sum_{j=k+1}^{\tau(k)}\frac{|\tilde{r}_{j}|}{2j+B_{0}^{*}+R_{0}^{*}}\leq\sum_{j=k+1}^{\tau(k)}\frac{2Cz_{k}}{(2j+B_{0}^{*}+R_{0}^{*})(2j+B_{0}^{*}+R_{0}^{*}-2)}\leq\frac{Cz_{k}}{2k+B_{0}^{*}+R_{0}^{*}},$$

and, similarly, the quadratic variation of $(M_{n})_{n\in\mathbb{N}}$ satisfies

$$\langle M\rangle_{\tau(k)}-\langle M\rangle_{k}\leq\sum_{j=k+1}^{\tau(k)}\frac{\mathbb{E}(\tilde{\varepsilon}_{j}^{2}\mid\mathcal{G}_{j-1})}{(2j+B_{0}^{*}+R_{0}^{*})^{2}}\leq\left(2+\frac{C}{2k+B_{0}^{*}+R_{0}^{*}}\right)\frac{z_{k}}{2k+B_{0}^{*}+R_{0}^{*}}.$$

Choose $K\in\mathbb{N}$ such that $C/(2K+B_{0}^{*}+R_{0}^{*})<1/4$. Then, for all $k\geq K$, by the optional stopping theorem, one has

$$\frac{9z_{k}^{2}}{16}\mathbb{P}(E_{k}\mid\mathcal{G}_{k})\leq\mathbb{E}\left((M_{\tau(k)}-M_{k})^{2}\mathds{1}_{E_{k}}\mid\mathcal{G}_{k}\right)\leq\mathbb{E}(\langle M\rangle_{\tau(k)}-\langle M\rangle_{k}\mid\mathcal{G}_{k})\leq\frac{9}{4}\frac{z_{k}}{2k+B_{0}^{*}+R_{0}^{*}},$$

where $E_{k}:=\{\tau(k)<\infty\}\cup\{\lim_{n\to\infty}z_{n}=0,\tau(k)=\infty\}$ and we used that on the event $E_{k}$,

$$|M_{\tau(k)}-M_{k}|\geq|z_{\tau(k)}-z_{k}|-\sum_{j=k+1}^{\tau(k)}\frac{|\tilde{r}_{j}|}{2j+B_{0}^{*}+R_{0}^{*}}\geq z_{k}-\frac{z_{k}}{4}=\frac{3z_{k}}{4}.$$

This proves (ii) since $z_{k}(2k+B_{0}^{*}+R_{0}^{*})=B_{k}^{*}$. ∎

We now set $t_{0}:=0$, $t_{n}:=\sum_{i=1}^{n}1/i$ and define an interpolated process $(I(t))_{t\geq 0}$ by

$$I(t_{n}+s)=(x_{n},y_{n})+s\frac{(x_{n+1},y_{n+1})-(x_{n},y_{n})}{t_{n+1}-t_{n}},\quad n\in\mathbb{N},s\in[0,\frac{1}{n+1}].$$

By Proposition 2.1 and [2, Proposition 4.2, Remark 4.5], the interpolated process $I$ is an asymptotic pseudotrajectory of the flow induced by the vector field $F^{(\alpha)}_{p}$. We now prove Theorem 1.2.

Proof of Theorem 1.2.

We first prove the a.s.-convergence of $((x_{n},y_{n}))_{n\in\mathbb{N}}$. We first assume that $\alpha=1$. The case $p=0$ follows from the classical results for the Pólya urn model. If $p>0$, recall that $\Lambda^{(1)}_{p}=\{(x,x):x\in[0,1]\}$ and $L^{(1)}_{p}=-p(x-y)^{2}/4$ by Example 2.1. Since $[0,1]^{2}$ is compact, by [2, Theorem 5.7], the limit set of the interpolated process $I$

$$L(I):=\bigcap_{t\geq 0}\overline{I([t,\infty))}$$

is internally chain transitive a.s.. Then, by Proposition 2.2 and [2, Proposition 6.4], almost surely, $L(I)\subset\Lambda^{(1)}_{p}$, and thus, the sequence $(x_{n}-y_{n})_{n\in\mathbb{N}}$ converges to $0$ a.s.. On the other hand, by Lemma 4.1 (i) and (51), $(x_{n}+y_{n})_{n\in\mathbb{N}}$ converges a.s., and in particular, $((x_{n},y_{n}))_{n\in\mathbb{N}}$ converges a.s.. For $\alpha>1$, the proof is similar: By Proposition 2.3, there are only finitely many equilibria of the gradient system (15). Then one can directly apply [2, Corollary 6.6] to conclude that $((x_{n},y_{n}))_{n\in\mathbb{N}}$ converges a.s. to an equilibrium.

(i) We have shown that for $p>0$, almost surely, $(x_{n})_{n\in\mathbb{N}}$ and $(y_{n})_{n\in\mathbb{N}}$ have the same limit. Let $K$ be as in Lemma 4.1 (ii), and let $\theta_{K}:=\inf\{k\geq K:B_{k}^{*}>8\}$. Then $\theta_{K}$ is a.s. finite. By Lemma 4.1 (ii), for any $j\in\mathbb{N}$,

$$\mathbb{P}(\lim_{n\to\infty}z_{n}=0\mid\mathcal{G}_{\theta_{K}+j})\leq\frac{4}{B^{*}_{\theta_{K}+j}}\leq\frac{1}{2},$$

which implies that $\mathbb{P}(\lim_{n\to\infty}z_{n}=0)=0$ by Levy’s 0-1 law. Similarly, one can show that $\mathbb{P}(\lim_{n\to\infty}z_{n}=1)=0$. Thus, $\mathbb{P}_{p}^{(1)}(\mathcal{D})=0$.

(ii) We assume that $p\leq\alpha^{-1}\min((\alpha-1)/6,1/3)$. The existence of $u_{\alpha,p}$ follows from Lemma 3.1. By Proposition 2.4, the equilibrium $(u_{\alpha,p},1-u_{\alpha,p})$ is asymptotically stable. Moreover, for any open neighborhood $\mathcal{N}$ of $(u_{\alpha,p},1-u_{\alpha,p})$ and any $m\in\mathbb{N}$, one has $\mathbb{P}((x_{n},y_{n})\in\mathcal{N}\ \text{for some }n\geq m)>0$. Then one can deduce (ii) from [2, Theorem 7.3]. ∎

The following auxiliary lemma will be used in the proof of Theorem 1.3.

Lemma 4.2.

If $(x,y)\in\Lambda^{(\alpha)}_{p}$ is unstable, i.e. $\lambda_{+}(x,y)>0$, then

$$\mathbb{P}_{p}^{(\alpha)}(\lim_{n\to\infty}(x_{n},y_{n})=(x,y))=0.$$ (53)

Proof.

Since $(x,y)$ is unstable, by Proposition 2.3 (iv), it is not on the boundary of $[0,1]^{2}$, and thus, there exists a neighborhood $\mathcal{N}$ of $(x,y)$ such that any $(u,v)\in\mathcal{N}$ is bounded away from the boundary. Now we show that there exists a constant $b>0$ such that for any $\theta\in[0,2\pi]$,

$$\mathbb{E}(\left(\varepsilon_{n+1}(1)\cos\theta+\varepsilon_{n+1}(2)\sin\theta\right)^{+}\mid\mathcal{G}_{n})\mathds{1}_{\{(x_{n},y_{n})\in\mathcal{N}\}}\geq b\mathds{1}_{\{(x_{n},y_{n})\in\mathcal{N}\}},$$ (54)

where $(\varepsilon_{n+1})_{n\in\mathbb{N}}$ is given in Proposition 2.1 and $x^{+}:=\max(x,0)$. By (2) and (49), we can find positive constants $C_{1},C_{2}$ such that

$$\mathbb{P}(\varepsilon_{n+1}(1)\geq C_{2},\varepsilon_{n+1}(2)\geq C_{2}\mid\mathcal{G}_{n})\mathds{1}_{\{(x_{n},y_{n})\in\mathcal{N}\}}\geq C_{1}\mathds{1}_{\{(x_{n},y_{n})\in\mathcal{N}\}}.$$

Therefore, for any $\theta\in[0,\pi/2]$, the left-hand side of (54) is lower bounded by

$$C_{1}C_{2}(\cos\theta+\sin\theta)\mathds{1}_{\{(x_{n},y_{n})\in\mathcal{N}\}}\geq C_{1}C_{2}\mathds{1}_{\{(x_{n},y_{n})\in\mathcal{N}\}}.$$

The cases $\theta\in[\pi/2,\pi],[\pi,3\pi/2],[3\pi/2,2\pi]$ can be proved similarly.

Now observe that $F^{(\alpha)}_{p}$ is $C^{\infty}$. Then, (53) follows from [24, Theorem 2.2.4], see also [22, Theorem 1]. ∎

Proof of Theorem 1.3.

(i) If $p\geq\min(1/2,\alpha^{-1}(\alpha-1))$, then, by Proposition 2.3 and Corollary 2.5, $\Lambda^{(\alpha)}_{p}\backslash\{(0,0),(1,1)\}$ consists of finitely many unstable equilibria. From the proof of Theorem 1.2, we see that $(x_{n},y_{n})$ converges a.s. to an equilibrium. Lemma 4.2 then implies that $\mathbb{P}_{p}^{(\alpha)}(\mathcal{D})=1$, and thus, $p_{\alpha}\leq\min(1/2,\alpha^{-1}(\alpha-1))$.

We assume that $p_{\alpha}=\min(1/2,\alpha^{-1}(\alpha-1))$ for some $\alpha>1$. In particular, for any $q<\min(1/2,\alpha^{-1}(\alpha-1))$, there exists $p\in(q,\min(1/2,\alpha^{-1}(\alpha-1)))$ such that $\mathbb{P}_{p}^{(\alpha)}(\mathcal{D})<1$. By Proposition 2.3 and Lemma 4.2, we can find two sequences $(p^{(n)})_{n\geq 1}$ and $(\tilde{s}_{n})_{n\geq 1}$ such that for any $n\geq 1$,

$$0<p^{(n)}<\min(\frac{1}{2},\frac{\alpha-1}{\alpha}),\quad\tilde{s}_{n}\in\Lambda^{(\alpha)}_{p}\bigcap\left([0,\frac{1}{2}]\times[\frac{1}{2},1]\right)\ \text{with}\ \lambda_{+}(\tilde{s}_{n})\leq 0,$$

and

$$\lim_{n\to\infty}p^{(n)}=\min(\frac{1}{2},\frac{\alpha-1}{\alpha}),\quad\mathbb{P}_{p^{(n)}}^{(\alpha)}(\lim_{n\to\infty}(x_{n},y_{n})=\tilde{s}_{n})>0.$$

Since $[0,1]^{2}$ is compact, by possibly choosing a subsequence, we may assume that $\lim_{n\to\infty}\tilde{s}_{n}=\tilde{s}$ for some $\tilde{s}\in[0,1/2]\times[1/2,1]$. Since $F^{(\alpha)}_{p}(x,y)$ and $\lambda_{+}(x,y)$ are continuous function in $(p,x,y)$, we see that $F^{(\alpha)}_{p}(\tilde{s})=0$ and $\lambda_{+}(\tilde{s})\leq 0$ with $p=\min(1/2,\alpha^{-1}(\alpha-1))$, which contradicts Corollary 2.5.

As shown in the proof of Theorem 1.2 (ii), if $\alpha>1$, then $\mathbb{P}_{p}^{(\alpha)}(\lim_{n\to\infty}(x_{n},y_{n})=(x,y))>0$ for any $(x,y)\in\mathcal{E}_{p}^{(\alpha)}$. Thus, (ii) and (iii) are corollaries of Proposition 2.4 (ii) and (iii) (note that for $p=0$, by (20) and Proposition 2.3, one has $(0,1)\in\mathcal{E}_{p}^{(\alpha)}$). ∎

5. Continuous-time construction with time-delays

Let $X$ be the jump process defined in Section 2.3 and $(\mathcal{F}_{t})_{t\geq 0}$ be its natural filtration. By Proposition 2.6, we may define $X$ and the IUM $(N_{n})_{n\in\mathbb{N}}$ on the same probability space $(\Omega,\mathcal{F},\mathbb{P})$ such that (27) holds.

The following lemma will be used in the proof of Theorem 1.5. Recall that $A_{n}=\sum_{i=n}^{\infty}1/W(i)^{2}$.

Lemma 5.1.

Assume that $\{W(n)\}_{n\in\mathbb{N}}$ satisfies (7) and $\lim_{n\to\infty}W(n)\sqrt{A_{n}}=\infty$. Let $\{\theta^{(1)}_{n}\}_{n\geq 1}$ and $\{\theta^{(2)}_{n}\}_{n\geq 1}$ be independent Exp(1)-distributed random variables, and for $k\geq 1$, let

$$S^{(k)}:=\sum_{n=k}^{\infty}\frac{\theta_{n}^{(1)}-\theta_{n}^{(2)}}{W(n)};\quad S^{(k)}_{m}:=\sum_{n=k}^{m}\frac{\theta_{n}^{(1)}-\theta_{n}^{(2)}}{W(n)},\ m\geq k.$$

Then, there exists $K\in\mathbb{N}_{+}$ such that for all large $k\geq K$,

$$\mathbb{P}(S^{(k)}>\frac{1}{4}\sqrt{A_{k}})\geq\frac{5}{32}.$$

Proof.

Our assumptions imply that there exists $K\in\mathbb{N}_{+}$ such that $W(n)\sqrt{A_{n}}\geq 64\mathbb{E}|\theta_{n}^{(1)}-\theta_{n}^{(2)}|^{3}$ for all $n\geq K$. Now fix $k\geq K$, let $\tau:=\inf\{m\geq k:|S^{(k)}_{m}|>\sqrt{A_{k}}/4\}$ and $T_{\theta}:=\inf\{m\geq k:|\theta_{m}^{(1)}-\theta_{m}^{(2)}|>W(m)\sqrt{A_{k}}/2\}$. Note that $\tau\leq T_{\theta}$. By definition, $\mathbb{E}(S^{(k)}_{\tau})^{2}\mathds{1}_{\{\tau=\infty\}}\leq A_{k}/16$. Moreover,

$$\mathbb{E}(S^{(k)}_{\tau})^{2}\mathds{1}_{\{\tau<\infty\}}=\mathbb{E}(S^{(k)}_{\tau-1}+\frac{\theta_{\tau}^{(1)}-\theta_{\tau}^{(2)}}{W(\tau)})^{2}\mathds{1}_{\{\tau<\infty\}}\leq\frac{A_{k}}{8}+2\mathbb{E}\left(\frac{\theta_{\tau}^{(1)}-\theta_{\tau}^{(2)}}{W(\tau)}\right)^{2}.$$

By the choice of $K$, one has

$$\mathbb{E}\left(\frac{\theta_{\tau}^{(1)}-\theta_{\tau}^{(2)}}{W(\tau)}\right)^{2} =\mathbb{E}\left(\frac{\theta_{\tau}^{(1)}-\theta_{\tau}^{(2)}}{W(\tau)}\right)^{2}\mathds{1}_{\{\tau<T_{\theta}\}}+\sum_{n=k}^{\infty}\mathbb{E}\left(\frac{\theta_{n}^{(1)}-\theta_{n}^{(2)}}{W(n)}\right)^{2}\mathds{1}_{\{T_{\theta}=n\}}$$

$$\leq\frac{A_{k}}{4}+\sum_{n=k}^{\infty}\frac{2}{W(n)^{2}}\frac{\mathbb{E}|\theta_{n}^{(1)}-\theta_{n}^{(2)}|^{3}}{W(n)\sqrt{A_{k}}}\leq\frac{9A_{k}}{32}.$$

Applying the optional stopping theorem to the $L^{2}$-bounded martingale $(S^{(k)}_{m})_{m\geq k}$, we have

$$\frac{3A_{k}}{4}\geq\mathbb{E}(S^{(k)}_{\tau})^{2}=\mathbb{E}\langle S^{(k)}\rangle_{\tau}\geq\mathbb{E}\langle S^{(k)}\rangle_{\infty}\mathds{1}_{\{\tau=\infty\}}=2A_{k}\mathbb{P}(\tau=\infty),$$

where the quadratic variation of the martingale $(S^{(k)}_{m})_{m\geq k}$ is given by

$$\langle S^{(k)}\rangle_{m}=\sum_{n=k}^{m}\frac{2}{W(n)^{2}},\quad m\geq k.$$

By symmetry,

$$\mathbb{P}(S^{(k)}_{\tau}>\frac{1}{4}\sqrt{A_{k}},\tau<\infty)=\mathbb{P}(S^{(k)}_{\tau}<-\frac{1}{4}\sqrt{A_{k}},\tau<\infty)=\frac{1}{2}\mathbb{P}(\tau<\infty)\geq\frac{5}{16},$$

and

$$\mathbb{P}(S^{(k)}\geq S^{(k)}_{\tau}|S^{(k)}_{\tau}>\frac{1}{4}\sqrt{A_{k}},\tau<\infty)\geq\frac{1}{2},$$

These two inequalities imply the desired result. ∎

Proof of Theorem 1.5.

By (26), for any $j\in\{1,2,\cdots,N_{c}\}$ and $n\geq a_{j}+d-1$, if $\sigma_{n+1}(j)<\infty$, we can write

$$\sigma_{n+1}(j)-\sigma_{n}(j)=\sum_{i=0}^{d-1}\frac{b_{i}}{W(n-i)},$$

where $b_{i}\geq 0$ and $\sum_{i=0}^{d-1}b_{i}=\xi^{(j)}_{n+1}$. Observe that

$$\frac{b_{d-1}}{W(n-d+1)}+\frac{b_{d-2}}{W(n-d+2)}=b_{d-1}\left(\frac{1}{W(n-d+1)}-\frac{1}{W(n-d+2)}\right)+\frac{b_{d-1}+b_{d-2}}{W(n-d+2)}$$

which belongs to the closed interval

$$\left[\frac{b_{d-1}+b_{d-2}}{W(n-d+2)}-\delta,\frac{b_{d-1}+b_{d-2}}{W(n-d+2)}+\delta\right]\ \text{where}\ \delta:=\left|\frac{\xi^{(j)}_{n+1}}{W(n-d+1)}-\frac{\xi^{(j)}_{n+1}}{W(n-d+2)}\right|.$$

Repeating this procedure $d-1$ times gives

$$|\sigma_{n+1}(j)-\sigma_{n}(j)-\frac{\xi^{(j)}_{n+1}}{W(n)}|\leq\sum_{i=0}^{d-2}\xi^{(j)}_{n+1}\left|\frac{1}{W(n-i-1)}-\frac{1}{W(n-i)}\right|.$$ (55)

Case (i): We assume that (13) holds. Then, for any $k\geq n$,

$$\frac{W(n)}{W(k)}-1\leq W(n)\left|\frac{1}{W(k)}-\frac{1}{W(n)}\right|\leq C.$$ (56)

In particular, $W(n)/\inf_{k\geq n}W(k)\leq C+1$ for any $n\geq 1$. As in (28), the conditional Borel-Cantelli lemma then implies that a.s. there is an infinite sequence of finite stopping times $\{\tau_{n_{k}d}\}_{k\geq 1}$ such that at each time $\tau_{n_{k}d}$, there exists some $i_{k}\in\{1,2,\cdots,N_{c}\}$,

$$Z_{n_{k}d}(i_{k})\geq d+\max_{j\neq i_{k}}\{Z_{n_{k}d}(j)\}.$$ (57)

As in the proof of Theorem 1.4, at time $\tau_{n_{k}d}$, for any $j\in\{1,2,\cdots,N_{c}\}$, the remaining time of the timer on $e_{j}$ has an exponential distribution with rate $W(Z_{n_{k}d}(j))$, which, by a slight abuse of notation, we denote by $\xi^{(j)}_{Z_{n_{k}d}(j)+1}/W(Z_{n_{k}d}(j))$. We assume that $Z_{n_{k}d}(q)=\max_{j\neq i_{k}}\{Z_{n_{k}d}(j)\}$ for some $q\neq i_{k}$. Then,

$$\quad\mathbb{E}\left(\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}(\xi^{(i_{k})}_{n+1}+\xi^{(q)}_{n+1})\sum_{\ell=0}^{d-2}\left|\frac{1}{W(n-\ell-1)}-\frac{1}{W(n-\ell)}\right|\mid\mathcal{F}_{\tau_{n_{k}d}}\right)$$ (58)

$$=\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\sum_{\ell=0}^{d-2}\left|\frac{2}{W(n-\ell-1)}-\frac{2}{W(n-\ell)}\right|$$

$$\leq 2(d-1)\sum_{n=Z_{n_{k}d}(i_{k})-d+1}^{\infty}\left|\frac{1}{W(n)}-\frac{1}{W(n+1)}\right|\leq\frac{2C(d-1)}{W(Z_{n_{k}d}(i_{k})-d+1)},$$

where we used that for each $n\geq Z_{n_{k}d}(i_{k})-d+1$, the term $|\frac{2}{W(n)}-\frac{2}{W(n+1)}|$ is counted at most $d-1$ times in the sum in the second line, and the last inequality follows from (13).

By Markov inequality, for any positive integrable random variable $\Theta$, if $m(\Theta)$ denotes its median value, then

$$\frac{1}{2}\leq\mathbb{P}(\Theta\geq m(\Theta))\leq\frac{\mathbb{E}\Theta}{m(\Theta)},$$

and thus $m(\Theta)\leq 2\mathbb{E}\Theta$. In particular, by (58),

$$\mathbb{P}\left(\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\sum_{\ell=0}^{d-2}\left|\frac{\xi^{(i_{k})}_{n+1}+\xi^{(q)}_{n+1}}{W(n-\ell-1)}-\frac{\xi^{(i_{k})}_{n+1}+\xi^{(q)}_{n+1}}{W(n-\ell)}\right|\leq\frac{4C(d-1)}{W(Z_{n_{k}d}(i_{k})-d+1)}\mid\mathcal{F}_{\tau_{n_{k}d}}\right)\geq\frac{1}{2}.$$ (59)

By (56) and (57), starting from time $\tau_{n_{k}d}$, the time needed to visit $e_{q}$ once more is

$$\frac{\xi^{(q)}_{Z_{n_{k}d}(q)+1}}{W(Z_{n_{k}d}(q))}\geq\frac{\xi^{(q)}_{Z_{n_{k}d}(q)+1}}{(C+1)W(Z_{n_{k}d}(i_{k})-d+1)}.$$ (60)

Again, here $\xi^{(q)}_{Z_{n_{k}d}(q)+1}/W(Z_{n_{k}d}(q))$ should be interpreted as the remaining time of the timer on $e_{q}$, which is independent of $\{\xi^{(i_{k})}_{n+1},\xi^{(q)}_{n+1}\}_{n\geq Z_{n_{k}d}(i_{k})}$ in virtue of (57). Therefore, by (59) and that $\xi^{(q)}_{Z_{n_{k}d}(q)+1}\sim\operatorname{Exp}(1)$,

$$\mathbb{P}(E_{q,i_{k}}\mid\mathcal{F}_{\tau_{n_{k}d}})\geq\frac{1}{2}\mathbb{P}(\xi^{(q)}_{Z_{n_{k}d}(q)+1}>4C(d-1)(C+1)\mid\mathcal{F}_{\tau_{n_{k}d}})\geq\frac{1}{2}e^{-4C(d-1)(C+1)},$$

where the event $E_{q,i_{k}}$ is defined by

$$E_{q,i_{k}}:=\left\{\frac{\xi^{(q)}_{Z_{n_{k}d}(q)+1}}{(C+1)W(Z_{n_{k}d}(i_{k})-d+1)}>\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\sum_{\ell=0}^{d-2}\left|\frac{\xi^{(i_{k})}_{n+1}+\xi^{(q)}_{n+1}}{W(n-\ell-1)}-\frac{\xi^{(i_{k})}_{n+1}+\xi^{(q)}_{n+1}}{W(n-\ell)}\right|\right\}.$$

By symmetry (one may interchange $\{\xi^{(i_{k})}_{n+1}\}_{n\geq Z_{n_{k}d}(i_{k})}$ and $\{\xi^{(q)}_{n+1}\}_{n\geq Z_{n_{k}d}(i_{k})}$),

$$\quad\mathbb{P}(\{\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\frac{\xi^{(i_{k})}_{n+1}}{W(n)}<\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\frac{\xi^{(q)}_{n+1}}{W(n)}\}\cap E_{q,i_{k}}\mid\mathcal{F}_{\tau_{n_{k}d}})$$ (61)

$$=\mathbb{P}(\{\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\frac{\xi^{(q)}_{n+1}}{W(n)}<\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\frac{\xi^{(i_{k})}_{n+1}}{W(n)}\}\cap E_{q,i_{k}}\mid\mathcal{F}_{\tau_{n_{k}d}})$$

$$=\frac{\mathbb{P}(E_{q,i_{k}}\mid\mathcal{F}_{\tau_{n_{k}d}})}{2}\geq\frac{1}{4}e^{-4C(d-1)(C+1)}=:c_{3}.$$

For $j\neq i_{k}$, we let $E_{j}$ be the event that

$$\frac{\xi^{(j)}_{Z_{n_{k}d}(j)+1}}{(C+1)W(Z_{n_{k}d}(i_{k})-d+1)}+\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\left(\frac{\xi^{(j)}_{n+1}}{W(n)}-\sum_{\ell=0}^{d-2}\xi^{(j)}_{n+1}\left|\frac{1}{W(n-\ell-1)}-\frac{1}{W(n-\ell)}\right|\right)$$

$$>\sum_{n=Z_{n_{k}d}(i_{k})}^{\infty}\left(\frac{\xi^{(i_{k})}_{n+1}}{W(n)}+\sum_{\ell=0}^{d-2}\xi^{(i_{k})}_{n+1}\left|\frac{1}{W(n-\ell-1)}-\frac{1}{W(n-\ell)}\right|\right).$$

Note that by symmetry, (61) still holds if one replaces $q$ by $j\neq i_{k}$. Thus, $\mathbb{P}(E_{j}\mid\mathcal{F}_{\tau_{n_{k}d}})=\mathbb{P}(E_{q}\mid\mathcal{F}_{\tau_{n_{k}d}})\geq c_{3}$. Since $\{\xi^{(i)}_{n}\}_{1\leq i\leq N_{c},n\geq 1}$ are i.i.d., by Hölder’s inequality, one has

$$\mathbb{P}(\bigcap_{j\neq i_{k}}E_{j}\mid\mathcal{F}_{\tau_{n_{k}d}}) =\mathbb{E}\left[\mathbb{P}\left(\bigcap_{j\neq i_{k}}E_{j}\mid\{\xi^{(i_{k})}_{n+1}\}_{n\geq Z_{n_{k}d}(i_{k})},\mathcal{F}_{\tau_{n_{k}d}}\right)\mid\mathcal{F}_{\tau_{n_{k}d}}\right]$$

$$=\mathbb{E}\left[\mathbb{P}\left(E_{q}\mid\{\xi^{(i_{k})}_{n+1}\}_{n\geq Z_{n_{k}d}(i_{k})},\mathcal{F}_{\tau_{n_{k}d}}\right)^{N_{c}-1}\mid\mathcal{F}_{\tau_{n_{k}d}}\right]$$

$$\geq\mathbb{P}(E_{q}\mid\mathcal{F}_{\tau_{n_{k}d}})^{N_{c}-1}.$$

In virtue of (55), on $\bigcap_{j\neq i_{k}}E_{j}$, we have $\lim_{n\to\infty}\sigma_{n}(i_{k})<\infty$ and

$$\lim_{n\to\infty}\sigma_{n}(i_{k})-\tau_{n_{k}d}<\min_{j\neq i_{k}}\{\lim_{n\to\infty}\sigma_{n}(j)-\tau_{n_{k}d}\}.$$

That is, the remaining time needed to visit $e_{i_{k}}$ i.o. is strictly less than that needed for any other edge. In other words, by Proposition 2.6, only balls of color $i_{k}$ are taken infinitely often. Therefore,

$$\mathbb{P}_{1}^{W}(\mathcal{M}\mid\mathcal{F}_{\tau_{n_{k}d}})\geq\mathbb{P}(\bigcap_{j\neq i_{k}}E_{j}\mid\mathcal{F}_{\tau_{n_{k}d}})\geq c_{3}^{N_{c}-1}>0.$$

We conclude that $\mathbb{P}_{1}^{W}(\mathcal{M})=1$ by Levy’s 0-1 law.

Case (ii): We assume that (14) holds. By a slight abuse of notation, for $k\geq 1$, we let $i_{k}$ be such that $Z_{kd}(i_{k})=\max_{1\leq j\neq N_{c}}\{Z_{kd}(j)\}$. For any $j\neq i_{k}$, by Lemma 5.1,

$$\mathbb{P}(\sum_{n=Z_{kd}(i_{k})}^{\infty}\frac{\xi^{(j)}_{n+1}-\xi^{(i_{k})}_{n+1}}{W(n)}>\frac{\sqrt{A_{Z_{kd}(i_{k})}}}{4}\mid\mathcal{F}_{\tau_{kd}})\geq\frac{5}{32},$$ (62)

and, by Markov’s inequality and (58), for large $k$,

$$\quad\mathbb{P}\left(\sum_{n=Z_{kd}(i_{k})}^{\infty}\sum_{\ell=0}^{d-2}\left|\frac{\xi^{(i_{k})}_{n+1}+\xi^{(j)}_{n+1}}{W(n-\ell-1)}-\frac{\xi^{(i_{k})}_{n+1}+\xi^{(j)}_{n+1}}{W(n-\ell)}\right|\geq\frac{\sqrt{A_{Z_{kd}(i_{k})}}}{4}\mid\mathcal{F}_{\tau_{kd}}\right)$$ (63)

$$\leq\frac{8(d-1)\delta_{Z_{kd}(i_{k})-d+1}}{\sqrt{A_{Z_{kd}(i_{k})}}}=\frac{8(d-1)\delta_{Z_{kd}(i_{k})-d+1}}{\sqrt{A_{Z_{kd}(i_{k})-d+1}}}\frac{\sqrt{A_{Z_{kd}(i_{k})-d+1}}}{\sqrt{A_{Z_{kd}(i_{k})}}}\leq\frac{1}{8}.$$

where we used that $A_{n+1}/A_{n}$ converges to $1$ by (14). By a slight abuse of notation, we let $E_{j}$ be the event that

$$\sum_{n=Z_{kd}(i_{k})}^{\infty}\frac{\xi^{(j)}_{n+1}-\xi^{(i_{k})}_{n+1}}{W(n)}>\sum_{n=Z_{kd}(i_{k})}^{\infty}\left((\xi^{(j)}_{n+1}+\xi^{(i_{k})}_{n+1})\sum_{\ell=0}^{d-2}\left|\frac{1}{W(n-\ell-1)}-\frac{1}{W(n-\ell)}\right|\right).$$

We deduce from (62) and (63) that

$$\mathbb{P}(E_{j}\mid\mathcal{F}_{\tau_{kd}})\geq\frac{5}{32}-\frac{1}{8}=\frac{1}{32}.$$

The rest of the proof follows the same lines as that of Case 1: Hölder’s inequality and (55) imply that for all large $k$,

$$\mathbb{P}_{1}^{W}(\mathcal{M}\mid\mathcal{F}_{\tau_{kd}})\geq\mathbb{P}(\bigcap_{j\neq i_{k}}E_{j}\mid\mathcal{F}_{\tau_{kd}})\geq(\frac{1}{32})^{N_{c}-1},$$

which shows that $\mathbb{P}_{1}^{W}(\mathcal{M})=1$ by Levy’s 0-1 law. ∎

6. Coupling

Proof of Lemma 2.7.

Let $\{U_{n}^{(i)}\}_{n\geq 1,1\leq i\leq 2}$ be i.i.d. uniform random variables on $(0,1)$. For any $n\in\mathbb{N}$, we set $\tilde{B}_{2n+1}(1)=\tilde{B}_{2n}(1)+1$, resp. $B_{n+1}(1)=B_{n}(1)+1$ if

$$U_{n+1}^{(1)}<\frac{W(\tilde{B}_{2n}(1))}{W(\tilde{B}_{2n}(1))+W(\tilde{R}_{2n}^{*})},\ \text{resp.}\ \ U_{n+1}^{(1)}<\frac{pW(B_{n}^{*})}{W(B_{n}^{*})+W(R_{n}^{*})}+\frac{(1-p)W(B_{n}(1))}{W(B_{n}(1))+W(R_{n}(1))},$$

otherwise, we set $\tilde{R}_{2n+1}(1)=\tilde{R}_{2n}(1)+1$, resp. $R_{n+1}(1)=R_{n}(1)+1$; we set $\tilde{B}_{2n+2}(2)=\tilde{B}_{2n}(2)+1$, resp. $B_{n+1}(2)=B_{n}(2)+1$ if

$$U_{n+1}^{(2)}<\frac{W(\tilde{B}_{2n}(2))}{W(\tilde{B}_{2n}(2))+W(\tilde{R}_{2n+1}^{*})},\ \text{resp.}\ \ U_{n+1}^{(2)}<\frac{pW(B_{n}^{*})}{W(B_{n}^{*})+W(R_{n}^{*})}+\frac{(1-p)W(B_{n}(2))}{W(B_{n}(2))+W(R_{n}(2))},$$

otherwise, we set $\tilde{R}_{2n+2}(1)=\tilde{R}_{2n}(2)+1$, resp. $R_{n+1}(2)=R_{n}(2)+1$. Then it is easy to check that $\left(B_{n},R_{n}\right)_{n\in\mathbb{N}}$ and $\left(\tilde{B}_{n},\tilde{R}_{n}\right)_{n\in\mathbb{N}}$ defined above have the desired laws. One can then prove (30) by induction. ∎

Proof of Lemma 2.8.

As in the proof of Theorem 1.5, we use a time-lines construction to prove Lemma 2.8. Let $G=(V,E)$ be a directed multigraph with

$$V=\{v_{1},v_{2}\},\quad E=\{(v_{1},b,v_{2}),(v_{2},b,v_{1}),(v_{1},r,v_{2}),(v_{2},r,v_{1})\}.$$

where $(v_{1},b,v_{2})$ and $(v_{2},b,v_{1})$ are two arcs from $v_{1}$ to $v_{2}$ and $v_{2}$ to $v_{1}$, respectively. We regard $e_{r}:=\{(v_{1},r,v_{2}),(v_{2},r,v_{1})\}$ as an undirected edge.

Let $\{\xi^{(r)}_{n}\}_{n\geq 1}$, $\{\xi^{(v_{1},b)}_{n}\}_{n\geq 1}$, $\{\xi^{(v_{2},b)}_{n}\}_{n\geq 1}$ be independent Exp(1)-distributed random variables. We define a continuous-time jump process $Y=(Y_{t})_{t\geq 0}$ on $G$:
(i) Define, on each (directed or undirected) edge $e\in\{(v_{1},b,v_{2}),(v_{2},b,v_{1}),e_{r}\}$, independent point processes (alarm times) $\{V_{n}^{(e)}\}_{n\geq 1}$: for each $n\in\mathbb{N}$,

$$V_{n+1}^{(v_{1},b,v_{2})}:=\sum_{j=\tilde{B}_{0}(2)}^{\tilde{B}_{0}(2)+n}\frac{\xi^{(v_{2},b)}_{j}}{W(j)},\quad V_{n+1}^{(v_{2},b,v_{1})}:=\sum_{j=\tilde{B}_{0}(1)}^{\tilde{B}_{0}(1)+n}\frac{\xi^{(v_{1},b)}_{j}}{W(j)},\quad V_{n+1}^{(e_{r})}:=\sum_{j=\tilde{R}_{0}^{*}}^{\tilde{R}_{0}^{*}+n}\frac{\xi^{(r)}_{j}}{W(j)}.$$ (64)

(ii) Each edge $e$ has its own clock, denoted by $\tilde{T}_{e}(t)$. If $e=(v_{1},b,v_{2})$ (resp. $(v_{2},b,v_{1})$), $\tilde{T}_{e}$ runs when $Y$ is at $v_{1}$ (resp. $v_{2}$). For $e=e_{r}$, set $\tilde{T}_{e_{r}}(t):=t$.
(iii) Set $Y_{0}:=v_{2}$. If at time $t>0$, the clock of an edge $e$ rings, i.e. $\tilde{T}_{e}(t)=V_{k}^{(e)}$ for some $k>0$, then $Y$ jumps to cross $e$ instantaneously.

Let $0=\tau_{0}<\tau_{1}<\tau_{2}<\cdots$ be the jumping times of $Y$. For $i=1,2$, as in (23), let $Z_{n}^{(b)}(i)$ be the number of visits to $(v_{3-i},b,v_{i})$ up to time $\tau_{n}$ plus $\tilde{B}_{0}(i)$, and let $Z_{n}^{(r)}(i)$ be the number of visits to $(v_{3-i},r,v_{i})$ up to time $\tau_{n}$ plus $\tilde{R}_{0}(i)$ (note that here we distinguish $(v_{1},r,v_{2})$ and $(v_{2},r,v_{1})$). Then, as in Proposition 2.6, one can show by the memoryless property of exponentials that

$$(Z_{n}^{(b)}(i),Z_{n}^{(r)}(i))_{1\leq i\leq 2,n\in\mathbb{N}}\stackrel{{\scriptstyle\mathcal{L}}}{{=}}(\tilde{B}_{n}(i),\tilde{R}_{n}(i))_{1\leq i\leq 2,n\in\mathbb{N}}.$$

We may assume that $(Z_{n}^{(b)}(i),Z_{n}^{(r)}(i))_{1\leq i\leq 2,n\in\mathbb{N}}=(\tilde{B}_{n}(i),\tilde{R}_{n}(i))_{1\leq i\leq 2,n\in\mathbb{N}}$ on some probability space. We denote by $(\tilde{\mathcal{F}}_{t})$ the natural filtration of $Y$.

Now, assume that $\tilde{R}_{2n}^{*}<\varepsilon_{1}n$, and in particular, $\tilde{B}_{2n}(i)\geq(1-\varepsilon_{1})n$ for $i=1,2$. Starting from time $\tau_{2n}$, the total time that $Y$ needs to spend to cross the undirected edge $e_{r}$ infinitely often is

$$\sum_{j=\tilde{R}_{2n}^{*}}^{\infty}\frac{\xi_{j+1}^{(r)}}{W(j)}\geq\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{\xi_{j+1}^{(r)}}{W(j)}=:T_{n}^{(r)},$$ (65)

where $\lceil\cdot\rceil$ is the usual ceiling function. Again, the first term $\xi_{\tilde{R}_{2n}^{*}+1}^{(r)}/(\tilde{R}_{2n}^{*})^{\alpha}$ should be interpreted as the remaining time of the clock on $e_{r}$ at time $\tau_{2n}$. On the other hand, the total time that $Y$ needs to spend to cross both $(v_{1},b,v_{2})$ and $(v_{2},b,v_{1})$ infinitely often is

$$T_{n}^{(b)}:=\sum_{j=\tilde{B}_{2n}(1)}^{\infty}\frac{\xi_{j+1}^{(v_{1},b)}}{W(j)}+\sum_{j=\tilde{B}_{2n}(2)}^{\infty}\frac{\xi_{j+1}^{(v_{2},b)}}{W(j)}.$$ (66)

Note that up to time $\tau_{\infty}$, the time $Y$ spends at $v_{1}$, resp. $v_{2}$, is upper bounded by $\sum_{j=\tilde{B}_{2n}(2)}^{\infty}\xi_{j+1}^{(v_{2},b)}/W(j)$, resp. $\sum_{j=\tilde{B}_{2n}(1)}^{\infty}\xi_{j+1}^{(v_{1},b)}/W(j)$. Now using properties of exponential random variables, we have, by (65) and (66),

$$\operatorname{Var}(\widehat{T}_{n}^{(r)}\mid\tilde{\mathcal{F}}_{\tau_{2n}})=\sum_{j=\lceil\varepsilon_{1}n\rceil+1}^{\infty}\frac{1}{W(j)^{2}},\ \text{ where}\ \widehat{T}_{n}^{(r)}:=T_{n}^{(r)}-\frac{\xi_{\lceil\varepsilon_{1}n\rceil+1}^{(r)}}{W(\lceil\varepsilon_{1}n\rceil)},$$

and

$$\mathbb{E}(\widehat{T}_{n}^{(r)}\mid\tilde{\mathcal{F}}_{\tau_{2n}})=\sum_{j=\lceil\varepsilon_{1}n\rceil+1}^{\infty}\frac{1}{W(j)},\quad\mathbb{E}(T_{n}^{(b)}\mid\tilde{\mathcal{F}}_{\tau_{2n}})\leq\sum_{j=\lceil(1-\varepsilon_{1})n\rceil}^{\infty}\frac{2}{W(j)}.$$

By Chebyshev’s inequality,

$$\quad\mathbb{P}(T_{n}^{(r)}>\frac{1}{4}\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{1}{W(j)}\mid\tilde{\mathcal{F}}_{\tau_{2n}})$$ (67)

$$\geq\mathbb{P}\left(\xi_{\lceil\varepsilon_{1}n\rceil+1}^{(r)}\geq 1,\widehat{T}_{n}^{(r)}>-\frac{3}{4W(\lceil\varepsilon_{1}n\rceil)}+\frac{1}{4}\sum_{j=\lceil\varepsilon_{1}n\rceil+1}^{\infty}\frac{1}{W(j)}\mid\tilde{\mathcal{F}}_{\tau_{2n}}\right)$$

$$\geq\frac{1}{e}\left(1-\mathbb{P}\left(\mathbb{E}(\widehat{T}_{n}^{(r)}\mid\tilde{\mathcal{F}}_{\tau_{2n}})-\widehat{T}_{n}^{(r)}\geq\frac{3}{4}\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{1}{W(j)}\mid\tilde{\mathcal{F}}_{\tau_{2n}}\right)\right)$$

$$\geq\frac{1}{e}\left(1-\frac{16}{9}\left(\sum_{j=\lceil\varepsilon_{1}n\rceil+1}^{\infty}\frac{1}{W(j)^{2}}\right)\left(\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{1}{W(j)}\right)^{-2}\right)\geq\frac{1}{9e},$$

where we used the monotonicity of $W$ to get

$$\left(\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{1}{W(j)}\right)^{2}\geq\sum_{j=\lceil\varepsilon_{1}n\rceil+1}^{\infty}\left(\frac{1}{W(\lceil\varepsilon_{1}n\rceil)}+\frac{1}{W(j)}\right)\frac{1}{W(j)}\geq 2\sum_{j=\lceil\varepsilon_{1}n\rceil+1}^{\infty}\frac{1}{W(j)^{2}}.$$

On the other hand, Markov’s inequality implies that

$$\mathbb{P}\left(T_{n}^{(b)}\geq\frac{1}{4}\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{1}{W(j)}\mid\tilde{\mathcal{F}}_{\tau_{2n}}\right)\leq 8\left(\sum_{j=\lceil(1-\varepsilon_{1})n\rceil}^{\infty}\frac{1}{W(j)}\right)\left(\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{1}{W(j)}\right)^{-1},$$ (68)

In virtue of (9), the right-hand side of (68) can be made arbitrarily small for all $n\geq\kappa$ by first choosing a small $\varepsilon_{1}$ and then choosing a large $\kappa$. Using (67) and (68), by possibly choosing a smaller $\varepsilon_{1}$ and a larger $\kappa$, we have

$$\mathbb{P}(T_{n}^{(b)}<T_{n}^{(r)}\mid\tilde{\mathcal{F}}_{\tau_{2n}})\geq\mathbb{P}(T_{n}^{(b)}<\frac{1}{4}\sum_{j=\lceil\varepsilon_{1}n\rceil}^{\infty}\frac{1}{W(j)}<T_{n}^{(r)}\mid\tilde{\mathcal{F}}_{\tau_{2n}})>\frac{1}{10e}$$

if $n\geq\kappa$ and $\tilde{R}_{2n}^{*}<\varepsilon_{1}n$. It remains to observe that on the event $\{T_{n}^{(b)}<T_{n}^{(r)}\}$, only black edges are crossed infinitely often, that is, only black balls are drawn infinitely often. ∎

7. Some open questions

For the interacting urn mechanism with strong reinforcement, some interesting questions remain unsolved.

1. For power function/polynomial reinforcements, an important question is whether there is a phase transition at $p_{\alpha}$. In Remark 1.2, we conjecture that $\tilde{p}_{\alpha}=p_{\alpha}$. If this is true, is $p_{\alpha}$ increasing in $\alpha$? (Intuitively speaking, the reinforcement becomes stronger as $\alpha$ grows.) And does the limit of $p_{\alpha}/(\alpha-1)$ exist as $\alpha$ approaches 1 from above? To solve these questions, we may need a better understanding of the system (15), or we need to couple $\mathbb{P}_{p_{1}}^{(\alpha)}$ and $\mathbb{P}_{p_{2}}^{(\alpha)}$ for $p_{1}<p_{2}$.

2. We conjecture that for a large class of strong reinforcement sequences, say, $\{W(n)\}_{n\in\mathbb{N}}$ is non-decreasing and strong, one has $\mathbb{P}_{p}^{W}(\mathcal{M})=1$ if $p>1/2$. Currently, this assertion is proved only for exponential and polynomial reinforcements, see [16, Theorem 3.2] and Theorem 1.3.

3. As was conjectured in [17], can one prove that $\mathbb{P}_{1}^{W}(\mathcal{M})=1$ if one only assumes that $\{W(n)\}_{n\in\mathbb{N}}$ is a strong reinforcement sequence? The best result in this direction, known to date, seems to be Theorem 1.5.

8. Acknowledgement

I am very grateful to Professor Tarrès, my Ph.D. advisor, for inspiring the choice of this subject.