Learning by Gossip: A Principled Information Exchange Model in Social Networks

Abstract

We cope with the key step of bootstrap methods of generating a possibly infinite sequence of random data preserving properties of the distribution law, starting from a primary sample actually drawn from this distribution. We solve this task in a cooperative way within a community of generators where each improves its performance from the analysis of the other partners’ production. Since the analysis is based on an a priori distrust of the other partners’ production, we denote the partner ensemble as a gossip community and denote the statistical procedure learning by gossip. We prove that this procedure is highly efficient when applied to the elementary problem of reproducing a Bernoulli distribution, with a properly moderated distrust rate when the absence of a long-term memory requires an online estimation of the bootstrap generator parameters. This fact makes the procedure viable as a basic template of an efficient interaction scheme within social network agents.

Appendix

Hereafter, we give some directions to compute formulas (12), (13), (14), (16), and (17).

Formula (12)

• Starting statements:

  $$ \begin{aligned} A_k(t)&=(1-\varepsilon)A_k(t-1)+\varepsilon_0X(t)+\varepsilon_1 Y_k(t)+\varepsilon_2\sum_{j\ne k}Y_j(t); \quad \varepsilon=\varepsilon_0+\varepsilon_1+(n-1)\varepsilon_2;\\ \widetilde A(t)&=\frac{1}{n}\sum_{k=1}^n A_k(t); \quad Y_{[k]}(t)= \hbox {Integer}[A_k(t-1)+U_k];\quad U_k \hbox { uniform in} [0,1]; \quad\widetilde Y =\frac{1}{n}\sum_{k=1}^n Y_{[k]}(t). \end{aligned} $$

• Recurrency relations:

  • 1.
    $$ \begin{aligned} \widetilde A(t)&=(1-\varepsilon)\widetilde A(t-1)+\varepsilon_0X(t)+[\varepsilon_1+(n-1)\varepsilon_2] \widetilde Y(t);\\ E[ \widetilde A(t)]&=(1-\varepsilon) E[\widetilde A(t-1)]+\varepsilon_0\theta \end{aligned} $$
  • 2.
    $$ \begin{aligned} Cov[\widetilde A(t-1),\widetilde Y(t)]&=E[ \widetilde A(t-1)\widetilde Y(t)]-E[ \widetilde A(t-1)]E[\widetilde Y(t)]\\ &=\frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n\left(E[A_i(t-1)y_{[j]}(t)]-E[A_i(t-1)]^2\right)=\frac{1}{n^2}\sum_{i=1}^n\sum_{j=1}^n \left(E[A_i(t-i)A_{j}(t-1)]-E[A_i(t-i)]^2\right)\\ &=\frac{1}{n^2}\sum_{i=1}^n Var[A_i(t-1)]+\frac{1}{n^2}\sum_{i=1}^n\sum_{j\ne i,j=1}^n Cov[A_i(t-1)A_{j}(t-1)] =Var[\widetilde A(t-1)] \end{aligned} $$
  • 3.
    $$ \begin{aligned} Var[ \widetilde A(t)] &= (1-\varepsilon_0)^2Var[ \widetilde A(t-1)] + \varepsilon_0^2\theta(1-\theta)+[\varepsilon_1+(n-1)\varepsilon_2]^2Var[\widetilde Y(t)]\\ &\quad+ 2(1-\varepsilon)[\varepsilon_1+(n-1)\varepsilon_2]^2Cov[\widetilde A(t-1),\widetilde Y(t)]\\ &= (1-\varepsilon_0)^2Var[ \widetilde A(t-1)] + \varepsilon_0^2\theta(1-\theta)+[\varepsilon_1+(n-1)\varepsilon_2]^2\{Var[\widetilde Y(t)]-Var[\widetilde A(t-1)]\} \end{aligned} $$

• Final equation

  $$ \begin{aligned} E[(\widetilde A(t)-\theta)^2] &= E[(\widetilde A(t)-E[(\widetilde A(t)] +E\left[(\widetilde A(t)] -\theta)^2\right]\\ &= (1-\varepsilon_0)^2 E[(\widetilde A(t-1)-\theta)^2] +\varepsilon_0^2\theta(1-\theta)+[\varepsilon_1+(n-1)\varepsilon_2]^2 \{E[(\widetilde Y(t)-\theta)^2]-E[(\widetilde A(t-1)-\theta)^2]\} \end{aligned} $$

Formula (13)

• Starting statements:

  $$ E[(A_k(\infty)-\theta)^2]=E[A_k(\infty)^2]-\theta^2 $$

• Recurrency relation

  $$ \begin{aligned} E[A_i(t)A_j(t)]&= E\left [\left((1-\varepsilon)A_i(t-1)+\varepsilon_0X(t)+\varepsilon_1 Y_{[i]}(t)+\varepsilon_2\sum_{k\ne i,k=1}^nY_{[k]}(t)\right)\right.\\ &\quad\left. \left((1-\varepsilon)A_j(t-1)+\varepsilon_0X(t)+\varepsilon_1 Y_{[j]}(t)+\varepsilon_2\sum_{k\ne j,k=1}^nY_{[k]}(t)\right) \right] \end{aligned} $$

• Fixed point equation

  $$ E[A_i(\infty)^2]-E[A_i(\infty)A_ij(\infty)]=(1-\varepsilon_0-n \varepsilon_2)\{E[A_i(\infty)^2]-E[A_i(\infty)A_ij(\infty)]\}- (\varepsilon_1-\varepsilon_2)^2\{E[A_i(\infty)^2]-\theta^2\} $$

• Final equation

  $$ \hbox{E}\left[ \left( A_k(\infty) - \theta \right)^2 \right] = \theta (1-\theta) \left\{ 1- \frac{2 \varepsilon_0(1-\varepsilon_0)[1-(1-\varepsilon_0-n\varepsilon_2)^2]}{\delta_a-\delta_b} \right\} $$

  where

  $$ \begin{aligned} \delta_a&=\{1-[1-\varepsilon_0-(n-1)\varepsilon_2]^2+\varepsilon_1^2\}\{1-(1-\varepsilon_0)^2+\varepsilon_2[2(1-\varepsilon_0)-n\varepsilon_2]\\ \delta_b&=-2(n-1)\varepsilon_2^2 [1-\varepsilon_0-(n-1)\varepsilon_2-\varepsilon_1][2(1-\varepsilon_0)-n\varepsilon_2] \end{aligned} $$

• Starting statements:

  $$ Var[\widetilde A(t)]=\frac{1}{n^2}\left(\sum_{k=1}^n Var[A_k(t)]+\sum_{i=1}^n\sum_{j\neq,j=1}^nCov[A_i(t)A_j(t)]\right) $$

• Fixed point equation

  $$ Cov[A_i(\infty)A_j(\infty)]=\frac{1}{(1-\varepsilon_0-n\varepsilon_2)^2} \left\{[(1-\varepsilon_0-n\varepsilon_2)^2+(\varepsilon_1-\varepsilon_2)^2] E[A_k(\infty)^2]-(\varepsilon_1-\varepsilon_2)^2\theta(1-\theta)\right\} $$

• Final equation:

  $$ \begin{aligned} E\left[\left(\widetilde A(\infty)-\theta\right)^2\right] & =\frac{1}{n^2}\left\{nVar[A_i(\infty)]+n(n-1)Cov[A_i(\infty) -A_j(\infty)]\right\}\\ &=\frac{1}{n}\frac{\{[1-(1-\varepsilon_0-n\varepsilon_2)^2+(n-1) (\varepsilon_1-\varepsilon_2)^2\}E[(A_k(\infty)-\theta)^2]-(n-1) (\varepsilon_1-\varepsilon_2)^2\theta(1-\theta)}{1- (1-\varepsilon_0-n\varepsilon_2)^2} \end{aligned} $$

Formula (14)

• Starting statement

  $$ A(t)=(1-\varepsilon)A(t-1)+\varepsilon X(t);\quad Y(t)=\hbox {Integer}[A(t-1)+U];\quad S_n(t)=\frac{1}{n}\sum_{k=1}^n Y_k(t), \hbox {U uniform in} [0,1]. $$

• Recurrent relations:

  • 1.
    $$ Cov[Y_i(t),Y_j(t)]=E[Y_i(t),Y_j(t)]-E[Y_i(t)]E[Y_j(t)] =E[A(t-1)^2]-E[A(t-1)]^2=Var[A(t-1)] $$
  • 2.
    $$ Var[Y_i(t)]=E[Y_i(t)^2]-E[Y_i(t)]^2=E[A(t-1)](1-E[A(t-1)]) $$
  • 3.
    $$ Var[A(t)]=(1-\varepsilon)^2Var[A(t-1)]+\varepsilon^2\theta(1-\theta) $$
  • 4.
    $$ Var[S_n(t)]=\frac{1}{n}E[A(t-1)](1-E[A(t-1)])+\frac{n-1}{n}Var[A(t-1)] $$

• Final equation:

  $$ \begin{aligned} MSE[S_n(t)]&=Var[S_n(t)]+(E[S_n(t)]-\theta)^2\\ &=\frac{1}{n}E[A(t-1)](1-E[A(t-1)])+\frac{n-1}{n}Var[A(t-1)]+ E\left[\left(A(t-1)-\theta\right)^2\right]\\ &=(1-\varepsilon)^2MSE[S_n(t-1)]+\frac{1}{n}\varepsilon(1-\varepsilon) (1-2\theta)E[S_n(t-1)]+\frac{\varepsilon\theta}{n}[1-\varepsilon\theta+(n-1)\varepsilon(1-\theta)]. \end{aligned} $$

Formula (16)

• Starting statements:

  $$ \widetilde S_n(t)=\frac{1}{n}\sum_{k=1}^n Y_{[k]}. $$

• Recurrency relation:

  $$ \begin{aligned} Var[\widetilde S_n(t)]&=\frac{1}{n^2}\left\{\sum_{i=1}^n Var[Y_{[i]}(t)]+\sum_{i=1}^n\sum_{j\ne i,j=1}^n Cov[Y_{[i]}(t)Y_{[j]}(t)]\right\}\\ &=\frac{1}{n^2}\left\{\sum_{i=1}^nE[A_i(t-1)](1-E[A_i(t-1)])+\sum_{i=1}^n\sum_{j\ne i,j=1}^n Cov[A_{[i]}(t-1)A_{[j]}(t-1)]\right\} \end{aligned} $$

• Limit statement:

  $$ E[(\widetilde S_n(\infty)-\theta)^2] = \lim_{t\rightarrow\infty} Var[\widetilde S_n(t)] $$

• Final equation:

  $$ \begin{aligned} & \hbox{E}\left[ \left( \widetilde S_n(\infty) - \theta \right)^2 \right] \\ &\quad=\frac{1}{n}\frac{(n-1)[\delta_c+ (\varepsilon_1-\varepsilon_2)^2] \hbox{E} \left[ \left( A_k(\infty)-\theta \right)^2 \right] + [\delta_c-(n-1)(\varepsilon_1 - \varepsilon_2)^2] \theta(1-\theta)}{\delta_c} \end{aligned} $$

  where

  $$ \delta_c=1-(1-\varepsilon_0 -n \varepsilon_2)^2 $$

Formula (17)

• Recurrency relations:

  • 1.
    $$ \begin{aligned} E[\widetilde S_n(t)]&=\frac{1}{n}\sum_{k=1}^n E[A_k(t-1)]\\ &= (1-\varepsilon)\frac{1}{n}\sum_{k=1}^n E[A_k(t-2)] +\varepsilon_0\theta+\varepsilon_1E[\widetilde S_n(t-1)] +\varepsilon_2\frac{1}{n}\sum_{h\ne k, h=1}^n E[A_h(t-2)]\\ &=(1-\varepsilon)E[\widetilde S_n(t-1)]+\varepsilon_0\theta+\varepsilon_1E[\widetilde S_n(t-1)]+(n-1)\varepsilon_2E[\widetilde S_n(t-1)]=(1-\varepsilon_0)E[\widetilde S_n(t-1)]+\varepsilon_0\theta \end{aligned} $$
  • 2.
    $$ \begin{aligned} Var[\widetilde S_n(t)]&=E[\widetilde S_n(t)^2]-E[\widetilde S_n(t)]^2\\ &=\frac{1}{n}E[\widetilde S_n(t)]+\frac{1}{n^2}\sum_{i=1}^n \sum_{j\neq i,j=1}^n E[A_i(t)A_j(t)]- (1-\varepsilon_0)^2E[\widetilde S_n(t)]^2-\varepsilon_0^2\theta^2-2\varepsilon_0(1-\varepsilon_0)\theta E[\widetilde S_n(t-1)]\\ &=(1-\varepsilon_0)^2Var[\widetilde S_n(t-1)]+2(n-1) (1-\varepsilon)\varepsilon_2\{Var[\widetilde A(t-2)]-Var[\widetilde S_n(t-1)]\}\\ &\quad+[2(1-\varepsilon_0)-n\varepsilon_2]E[\widetilde S_n(t-1)(1- \widetilde S_n(t-1))]+\frac{1}{n}\varepsilon_0(1-\varepsilon_0)\{E[\widetilde S_n(t-1)]-2\theta E[\widetilde S_n(t-1)]+\theta\}\\ &\quad+\varepsilon_0^2\theta(1-\theta) \end{aligned} $$

• Final equation

  $$ \begin{aligned} \hbox{MSE}\left[ \widetilde S_n(t) \right] &= (1-\varepsilon_0)^2 \hbox{MSE}\left[ \widetilde S_n(t-1) \right] + 2(n-1)(1-\varepsilon)\varepsilon_2 \left\{ \hbox{Var}\left[ \widetilde A(t-2) \right] - \hbox{Var} \left[ \widetilde S_n(t-1) \right] \right\}\\ &\quad+[2(1-\varepsilon_0)-n \varepsilon_2] \varepsilon_2 \hbox{E} \left[ \widetilde S_n(t-1)\left( 1-\widetilde S_n(t-1)\right) \right]\\ &\quad+ \frac{1}{n} (1-\varepsilon_0)\varepsilon_0 \left\{ \hbox{E}\left[ \widetilde S_n(t-1) \right] -2 \theta \hbox{E} \left[ \widetilde S_n(t-1) \right] + \theta \right\} + \varepsilon_0^2 \theta(1-\theta) \end{aligned} $$