Statistical matching and uncertainty analysis in combining household income and expenditure data

Abstract

Among the goals of statistical matching, a very important one is the estimation of the joint distribution of variables not jointly observed in a sample survey but separately available from independent sample surveys. The absence of joint information on the variables of interest leads to uncertainty about the data generating model since the available sample information is unable to discriminate among a set of plausible joint distributions. In the present paper a short review of the concept of uncertainty in statistical matching under logical constraints, as well as how to measure uncertainty for continuous variables is presented. The notion of matching error is related to an appropriate measure of uncertainty and a criterion of selecting matching variables by choosing the variables minimizing such an uncertainty measure is introduced. Finally, a method to choose a plausible joint distribution for the variables of interest via iterative proportional fitting algorithm is described. The proposed methodology is then applied to household income and expenditure data when extra sample information regarding the average propensity to consume is available. This leads to a reconstructed complete dataset where each record includes measures on income and expenditure.

Appendix

Proof of Proposition  1

Taking into account that

• 1. $K^x_{N +} (y , \, z ) \le \min ( F_N (y \vert x) , \, G_N (z \vert x ))$;

• 2. $K^x_{N -} (y, \, z) \ge \max (0, \, F_N (y \vert x) + G_N (z \vert x) -1)$;

it is not difficult to see that

$$\begin{aligned} \varDelta ^{x}(F_{N},G_{N})\le & {} \int _{R^{2}} \left\{ \min ( F_N (y \vert x) , \, G_N (z \vert x )) \right. \nonumber \\&\quad \left. - \max (0, \, F_N (y \vert x) + G_N (z \vert x) -1) \right\} \, dF_{N}(y\,\vert x)dG_{N}(z\,\vert x) \nonumber \\ \,\approx & {} \int _{0}^{1} \int _{0}^{1} \left\{ min (u, \, v) - \max (0, \, u+v-1 ) \right\} \, du dv \nonumber \\ \,= & {} \frac{1}{6} . \end{aligned}$$

(23)

In other terms, the maximal value of the conditional measure of uncertainty (9) is essentially $1/6 \approx 0.167$. As an easy consequence of Proposition 1, also the unconditional uncertainty measure computed as in (10) takes the value 1 / 6.$\square$

Proof of Proposition 2

The following two statements hold:

$$\begin{aligned}&\widehat{\varDelta }^{*x}_H \mathop {\rightarrow }\limits ^{p} \varDelta ^x (F_N , \, G_N ) \;\;as \; k \rightarrow \infty \end{aligned}$$

(24)

$$\begin{aligned}&\widehat{\varDelta }^{*}_H \mathop {\rightarrow }\limits ^{p} \varDelta (F_N , \, G_N ) \;\;as \; k \rightarrow \infty \end{aligned}$$

(25)

Asymptotic analysis requires to define how the samples sizes $n_A$, $n_B$ and the population size N go to infinity. As in Brewer (1979) (cfr. also Little (1983)), this will be done as follows:

• 1. k replicates of the original population are formed.

• 2. From each replicate, an independent sample $\varvec{s}_A$ ($\varvec{s}_B$) of size $n_A$ ($n_B$) is selected, according to the sampling design $P_A$ ($P_B$). Using notation introduced above, let $D_{i,A}^j$ ($D_{i,B}^j$) be a Bernoulli r.v. taking the value 1 if unit i is included in the sample drawn from the jth replicate of the population $(j=1, \, \dots , \, k$) according to the sampling design $P_A$ ($P_B$), and the value 0 otherwise.

• 3. The k populations are aggregated to a population of size $N^* = kN$. We will denote by $F_{N^*} ( y \, \vert x)$, $G_{N^*} ( z \, \vert x)$, $p_{N^*} (x)$ the conditional p.d.f.s of Y and Z given $X=x$ and the proportion of units such that $X=x$, respectively.

• 4. The k samples drawn with the sampling design $P_A$ ($P_B$) are aggregated to a sample $\varvec{s}^*_A$ ($\varvec{s}^*_B$) of $n^*_A = k n_A$ ($n^*_B = k n_B$) units.

• 5. The quantities $F_{N^*} ( y \, \vert x)$, $G_{N^*} ( z \, \vert x)$, $p_{N^*} (x)$ are estimated by their Hájek estimators, as defined in Sect. 3.1, and based on $n^*_A$ and $n^*_B$ sample units. Such estimates are denoted by $\widehat{F}_{H}^{*} ( y \, \vert x)$, $\widehat{G}_{H}^{*} ( z \, \vert x)$, $\widehat{p}_{H}^{*} (x)$, respectively. Then, the uncertainty measures are estimated accordingly. We will denote by $\widehat{\varDelta }^{*x}_H$ ($\widehat{\varDelta }^{*}_H$) the estimate of the conditional (unconditional) measure of uncertainty.

• 6. k is allowed to tend to infinity.$\square$

First of all, it is immediate to see that

$$\begin{aligned} F_{N^*} ( y \, \vert x) = F_{N} ( y \, \vert x), \;\; G_{N^*} ( z \, \vert x) = G_{N} ( z \, \vert x), \;\; p_{N^*} (x) = p_{N} (x). \end{aligned}$$

In the second place, from

$$\begin{aligned} \widehat{F}_{H}^{*} ( y \, \vert x) = \frac{\sum _{i=1}^{N} \left\{ \frac{1}{k} \sum _{j=1}^{k} \frac{D_{i,A}^j}{\pi _{i,A}} \right\} I_{(y_i \le y)} I_{(x_i =x)}}{\sum _{i=1}^{N} \left\{ \frac{1}{k} \sum _{j=1}^{k} \frac{D_{i,A}^j}{\pi _{i,A}} \right\} I_{(x_i =x)}} \end{aligned}$$

and using the law of large numbers

$$\begin{aligned} \frac{1}{k} \sum _{j=1}^{k} \frac{D_{i,A}^j}{\pi _{i,A}} \end{aligned}$$

(26)

converges in probability to 1 as k goes to infinity, then it is not difficult to see that $\widehat{F}_{H}^{*} ( y \, \vert x)$ converges in probability to $F_{N} ( y \, \vert x)$ as k tends to infinity, for each x and uniformly in y. In the same way, it is possible to show that $\widehat{G}_{H}^{*} ( z \, \vert x)$ converges in probability to $G_{N} ( z \, \vert x)$ as k tends to infinity, for each x and uniformly in z. Since the functional $\varDelta ^x (F_N , \, G_N )$ is continuous in the sup-norm, (24) is proved. In the same way, (25) can be proved.