"reflections on the probability space induced by moment conditions with implications for Bayesian inference": a discussion
•
0 likes•3,660 views
This document discusses using moment conditions to perform Bayesian inference when the likelihood function is intractable or unknown. It outlines some approaches that have been proposed, including approximating the likelihood using empirical likelihood or pseudo-likelihoods. However, these approaches do not guarantee the same consistency as a true likelihood. Alternative approximative Bayesian methods are also discussed, such as Approximate Bayesian Computation, Integrated Nested Laplace Approximation, and variational Bayes. The empirical likelihood method constructs a likelihood from generalized moment conditions, but its use in Bayesian inference requires further analysis of consistency in each application.
1 of 22
Download to read offline
More Related Content
"reflections on the probability space induced by moment conditions with implications for Bayesian inference": a discussion
1. re
ections on the probability space induced by
moment conditions with implications for Bayesian
inference": a discussion
Christian P. Robert
Universite Paris-Dauphine, Paris University of Warwick, Coventry
bayesianstatistics@gmail.com
2. Outline
what is the question?
what could the question be?
what is the answer?
what could the answer be ?
4. es a set of moment functions collected
together into a vector m(x, ) of dimension M, regards
as random and asserts that some transformation Z(x, )
has distribution , then what is required to use this
information and then possibly a prior to make valid
inference? R. Gallant, p.4
5. Priors without eorts
I quest for model induced prior dating back to early 1900's
[Lhoste, 1923]
I reference priors such as Jereys' prior induced by sampling
distribution
[Jereys, 1939]
I Fiducial distributions as Fisher's attempted answer
[Fisher, 1956]
10. cation in asserting that
t =
x -
p
s=
n
has a t posterior distribution with n - 1 degrees of freedom on ,
given (x, s) except when using a non-informative and improper
prior (, 2) / 1=2 since, then
p
n)
Tn-1(x, s=
12. ducial distribution
Furthermore, neither Bayesian nor frequentist interpretation implies
that
t =
x -
p
s=
n
has a t posterior distribution with n - 1 degrees of freedom jointly
13. what could the question be?
Given a set of moment equations
E[m(X1, . . . ,Xn, )] = 0
(where both the Xi 's and are random), can one derive a
likelihood function and a prior distribution compatible with those
constraints?
14. coherence across sample sizes n
Highly complex question since it implies the integral equation
Z
Xn
m(x1, . . . , xn, ) ()f (x1j) f (xnj)ddx1 dxn = 0
must or should have a solution in (, f ) for all n's.
possible outside of a likelihood x prior modelling?
15. coherence across sample sizes n
Highly complex question since it implies the integral equation
Z
Xn
m(x1, . . . , xn, ) ()f (x1j) f (xnj)ddx1 dxn = 0
must or should have a solution in (, f ) for all n's.
possible outside of a likelihood x prior modelling?
16. Zellner's Bayesian method of moments
Given moment conditions on parameter and 2
E[jx1, . . . , xn] = xn E[2jx1, . . .] = s2
n var(j2, x1, . . .) = 2=n
derivation of a maximum entropy posterior
j2, x1, . . . N(xn, 2=n) -2jx1, . . . Exp(s2
n )
[Zellner, 1996]
but incompatible with corresponding predictive distribution
[Geisser Seidenfeld, 1999]
17. Zellner's Bayesian method of moments
Given moment conditions on parameter and 2
E[jx1, . . . , xn] = xn E[2jx1, . . .] = s2
n var(j2, x1, . . .) = 2=n
derivation of a maximum entropy posterior
j2, x1, . . . N(xn, 2=n) -2jx1, . . . Exp(s2
n )
[Zellner, 1996]
but incompatible with corresponding predictive distribution
[Geisser Seidenfeld, 1999]
18. what is the answer?
Under the condition that Z(, ) is surjective,
p?(xj) = (Z(x, ))
and arbitrary choice of prior ()
I lhs and rhs operate on dierent spaces
I no reason why density should integrate against Lebesgue
measure in n-dimensional Euclidean space
I no direct connection with a genuine likelihood function, i.e.,
product of the densities of the Xi 's (conditional on )
19. what is the answer?
Under the condition that Z(, ) is surjective,
p?(xj) = (Z(x, ))
and arbitrary choice of prior ()
I lhs and rhs operate on dierent spaces
I no reason why density should integrate against Lebesgue
measure in n-dimensional Euclidean space
I no direct connection with a genuine likelihood function, i.e.,
product of the densities of the Xi 's (conditional on )
20. what could the answer be?
A common situation that requires consideration of the
notions that follow is that deriving the likelihood from a
structural model is analytically intractable and one
cannot verify that the numerical approximations one
would have to make to circumvent the intractability are
suciently accurate. R. Gallant, p.7
22. ning joint distribution on (, x1, . . . , xn) through moment
equations prevents regular Bayesian inference as likelihood is
unavailable
there may be alternative available:
I Approximative Bayesian computation (ABC) and empirical
likelihood based Bayesian inference
[Tavare et al., 1999; Owen, 201; Mengersen et al., 2013]
I INLA (Laplace), EP (expectation/propagation),
[Martino et al., 2008; Barthelme Chopin, 2014]
I variational Bayes
[Jaakkola Jordan, 2000]
24. ning joint distribution on (, x1, . . . , xn) through moment
equations prevents regular Bayesian inference as likelihood is
unavailable
there may be alternative available:
I Approximative Bayesian computation (ABC) and empirical
likelihood based Bayesian inference
[Tavare et al., 1999; Owen, 201; Mengersen et al., 2013]
I INLA (Laplace), EP (expectation/propagation),
[Martino et al., 2008; Barthelme Chopin, 2014]
I variational Bayes
[Jaakkola Jordan, 2000]
25. Bayesian approximative answers
I Using a fake likelihood does not prohibit Bayesian analysis, as
shown in the paper with model in eqn. (45)
I However this requires case-by-case consistency analysis since
pseudo-likelihoods do not oer same garantees
I Example of ABC model choice based on insucient statistics
[Marin et al., 2014]
26. Empirical likelihood (EL)
Dataset x made of n independent replicates x = (x1, . . . , xn) of a
rv X F
Generalized moment condition pseudo-model
EF
h(X,)
= 0,
where h known function, and unknown parameter
Induced empirical likelihood
Lel(jx) = max
p
Yn
i=1
pi
for all p such that 0 6 pi 6 1,
P
i pi = 1,
P
i pih(xi ,) = 0
[Owen, 1988, B'ka, Empirical Likelihood, 2001]
27. Empirical likelihood (EL)
Dataset x made of n independent replicates x = (x1, . . . , xn) of a
rv X F
Generalized moment condition pseudo-model
EF
h(X,)
= 0,
where h known function, and unknown parameter
Induced empirical likelihood
Lel(jx) = max
p
Yn
i=1
pi
for all p such that 0 6 pi 6 1,
P
i pi = 1,
P
i pih(xi ,) = 0
[Owen, 1988, B'ka, Empirical Likelihood, 2001]
28. Raw ABCel sampler
Nave implementation: Act as if EL was an exact likelihood
[Lazar, 2003, B'ka]
for i = 1 ! N do
generate i from the prior distribution ()
set the weight !i = Lel(i jxobs)
end for
return (i ,!i ), i = 1, . . . ,N
I Output weighted sample of size N
[Mengersen et al., 2013, PNAS]
29. Raw ABCel sampler
Nave implementation: Act as if EL was an exact likelihood
[Lazar, 2003, B'ka]
for i = 1 ! N do
generate i from the prior distribution ()
set the weight !i = Lel(i jxobs)
end for
return (i ,!i ), i = 1, . . . ,N
I Performance evaluated through eective sample size
.XN
ESS = 1
i=1
8
:!i
.XN
j=1
!j
9=
2
;
[Mengersen et al., 2013, PNAS]