Automatic and Analytical Field Weighting for Structured Document Retrieval

Abstract

Probabilistic models such as BM25 and LM have established themselves as the standard in atomic retrieval. In structured document retrieval (SDR), BM25F could be considered the most established model. However, without optimization BM25F does not benefit from the document structure. The main contribution of this paper is a new field weighting method, denoted Information Content Field Weighting (ICFW). It applies weights over the structure without optimization and overcomes issues faced by some existing SDR models, most notably the issue of saturating term frequency across fields. ICFW is similar to BM25 and LM in its analytical grounding and transparency, making it a potential new candidate for a standard SDR model. For an optimised retrieval scenario ICFW does as well, or better than baselines. More interestingly, for a non-optimised retrieval scenario we observe a considerable increase in performance. Extensive analysis is performed to understand and explain the underlying reasons for this increase.

A Scale Parameter Threshold

The underlying idea of the scale threshold theorem is that there exists a threshold for $\lambda$, above which the model satisfies the term distinctiveness constraint.

Let $q=\{t_1,\ldots ,t_n\}$ be a query, d be a document with $n(t_a,f_i,d)$ occurrences of query term $t_a$ in field $f_i$ and $n(t_b,\overline{f},d)$ occurrences of query term $t_b$ in an average field $\overline{f}$. Let $\overline{d}$ be an amended version of document d where the occurrences of $t_b$ are replaced with occurrences of $t_a$.

Theorem 1

Given terms $t_a$ and $t_b$, if $\lambda \!>\! \lambda _{threshold }$, then $RSV (d)\!>\! RSV (\overline{d})$.

(24)

Proof

Following Definition 8 for $\lambda _{threshold }$, the inequality becomes:

$$\begin{aligned} \lambda > \frac{\log \frac{{\text {df}}(\text {t}_{\text {b}}, \overline{F})|\overline{F}|^{Zx}}{{\text {df}}(\text {t}_{\text {a}}, \overline{F})^{Zx}|\overline{F}|} }{\log \frac{m^{Z+1}{\text {ff}}({\text {t}_{\text {a}}}, {\overline{d}})^{Z(x+1)}}{m^{Z(x+1)}{\text {ff}}({\text {t}_{\text {a}}}, {d})^{Z+1}} } \end{aligned}$$

(25)

Considering the numerator first:

$$\begin{aligned} \log \frac{{\text {df}}(\text {t}_{\text {b}}, \overline{F})|\overline{F}|^{Zx}}{{\text {df}}(\text {t}_{\text {a}}, \overline{F})^{Zx}|\overline{F}|} = \log \frac{\frac{{\text {df}}(\text {t}_{\text {b}}, \overline{F})}{|\overline{F}|}}{\frac{{\text {df}}(\text {t}_{\text {a}}, \overline{F})^{Zx}}{|\overline{F}|^{Zx}}} \end{aligned}$$

(26)

Following Eq. (4) for the definition of probabilities and Eq. (26) we obtain,

$$\begin{aligned} \log \frac{P(\text {t}_{\text {b}},\overline{f}|\overline{F})}{P(\text {t}_{\text {a}},\overline{f}|\overline{F})^{Zx}} = \log P(\text {t}_{\text {b}},\overline{f}|\overline{F}) - Zx\log P(\text {t}_{\text {a}},\overline{f}|\overline{F}) \end{aligned}$$

(27)

Following Definition 3 we can re-write Eq. (27) to obtain,

$$\begin{aligned}&\log P(\text {t}_{\text {b}},\overline{f}|\overline{F}) - Zx\log P(\text {t}_{\text {a}},\overline{f}|\overline{F}) = Zx{\text {ICF}}(\overline{f},\overline{d}) - {\text {ICF}}(\overline{f},d) \end{aligned}$$

(28)

Moving onto the denominator,

$$\begin{aligned} \log \frac{m^{Z+1}{\text {ff}}({\text {t}_{\text {a}}}, {\overline{d}})^{Z(x+1)}}{m^{Z(x+1)}{\text {ff}}({\text {t}_{\text {a}}}, {d})^{Z+1}} = \log \frac{[\frac{{\text {ff}}({\text {t}_{\text {a}}}, {\overline{d}})}{m}]^{Z(x+1)}}{[\frac{{\text {ff}}({\text {t}_{\text {a}}}, {d})}{m}]^{Z+1}} \end{aligned}$$

(29)

Inserting Eq. (5) to Eq. (29) and transforming the log expression we obtain,

$$\begin{aligned} \log \frac{P(f_i|\overline{d})^{Z(x+1)}}{P(f_i|d)^{Z+1}} =Z(x+1)\log P(\text {t}_{\text {a}},f_i|\overline{d}) - Z\log P(\text {t}_{\text {a}},f_i|d) - \log P(\text {t}_{\text {a}},\overline{f}|d) \end{aligned}$$

(30)

Following Definition 3 we can re-write Eq. (30) to obtain

$$\begin{aligned}&\log \frac{P(f_i|\overline{d})^{Z(x+1)}}{P(f_i|d)^{Z+1}} = - Z(x+1){\text {ICD}}(f_i,\overline{d}) + Z{\text {ICD}}(f_i,d) + {\text {ICD}}(\overline{f},d) \end{aligned}$$

(31)

Inserting Eqs. (28) and (31) to Eq. (25) and solving for Z we obtain,

$$\begin{aligned}&Z < \frac{{\text {ICF}}(\overline{f},d) + \lambda {\text {ICD}}(\overline{f},d)}{\lambda (x+1){\text {ICD}}(f_i,\overline{d}) - \lambda {\text {ICD}}(f_i,d) + x{\text {ICF}}(\overline{f},\overline{d})} \end{aligned}$$

(32)

Expanding the denominator we obtain,

(33)

Following Eqs. (13), (14), (8) and (9) (33) is re-written:

$$\begin{aligned}&\frac{{\text {S}}_{{\text {contr}}}(\text {t}_{\text {a}},f_i,d)}{{\text {S}}_{{\text {contr}}}(\text {t}_{\text {b}},\overline{f},d)} < \frac{w_{{\text {icfw}}}({\overline{f}}, {d})}{w_{{\text {icfw}}}({f_i}, {\overline{d}}) + xw_{{\text {icfw}}}({\overline{f}}, {\overline{d}}) - w_{{\text {icfw}}}({f_i}, {d})} \end{aligned}$$

(34)

Rearranging Eq. (34) we obtain,

(35)

Assuming the term frequencies from the theorem, the retrieval score difference is only dependent on the score contributions of term $\text {t}_{\text {a}}$ in field $f_i$ and term $\text {t}_{\text {b}}$ in field $\overline{f}$. For $\overline{d}$ the same is true for the score contributions of term $\text {t}_{\text {a}}$ in field $f_i$ and term $\text {t}_{\text {a}}$ in field $\overline{f}$. Following Definition 5 we rewrite Eq. (35) and obtain the implicated inequality from the theorem.

   $\square$