User Response Based Recommendations

Abstract

In a recommendation system, users provide description of their interests through initial search keywords and the system recommends items based on these keywords. A user is satisfied if it finds the item of its choice and the system benefits, otherwise the user explores an item from the recommended list. Usually when the user explores an item, it picks an item that is nearest to its interest from the list. While the user explores an item, the system recommends new set of items. This continues till either the user finds the item of its interest or quits. In all, the user provides ample chances and feedback for the system to learn its interest. The aim of this paper is to efficiently utilize user responses to recommend items and find the item of user’s interest quickly.

We first derive optimal policies in the continuous Euclidean space and adapt the same to the space of discrete items. We propose the notion of local angle in the space of discrete items and develop user response-local angle (UR-LA) based recommendation policies. We compare the performance of UR-LA with widely used collaborative filtering (CF) based policies on two real datasets and show that UR-LA performs better in majority of the test cases. We propose a hybrid scheme that combines the best features of both UR-LA and CF (and history) based policies, which outperforms them in most of the cases.

Towards the end, we propose alternate recommendation policies again utilizing the user responses, based on clustering techniques. These policies outperform the previous ones, and are computationally less intensive. Further, the clustering based policies perform close to theoretical limits.

Appendices

A Appendix: Proofs

Lemma 2

The new belief $B_1$ is uniformly distributed over area $\mathcal{A}_1$ which has two choices:

$$B_{1} \sim \mathcal{U}(\mathcal{A}_{1}) \hbox { where }\mathcal{A}_{1} = \mathcal{A}_{11} 1_{\lbrace {X_{1} = a_{\text {11}}}\rbrace } + \mathcal{A}_{12} 1_{\lbrace {X_{1} = a_{\text {12}}}\rbrace },$$

where $\mathcal{A}_{11}$, $\mathcal{A}_{12}$ are given by Eq. (3).

Proof of Lemma

2: By definition belief $B_1$ is the conditional distribution of $V_{ref}$ given history $H_{1}$ which includes current observation $X_1$, the user choice. That is, $B_1 \sim V_{ref} \big | H_{1}.$ For any subset $\varLambda$ (Borel if continuous case)³ from Eq. (3):

(12)

Thus clearly belief $B_1$ is a uniform random variable either over $\mathcal{A}_{11}$ or $\mathcal{A}_{12}$, as $\mu$ is uniform. $\Box$

Lemma 3

The new belief $B_2$ is uniformly distributed, i.e., $B_{2} \sim \mathcal{U}(\mathcal{A}_{2} )$, where the area $\mathcal{A}_2$ has two choices for any given $X_1$:

$$ \mathcal{A}_{2} (X_1) = \mathcal{A}_{21} 1_{\lbrace {X_{2} = a_{\text {21}}}\rbrace } + \mathcal{A}_{22} 1_{\lbrace {X_{2} = a_{\text {22}}}\rbrace },$$

with $\mathcal{A}_{21}$, $\mathcal{A}_{22}$ defined in (4). These two choices depend further upon $X_1$, and hence in total $\mathcal{A}_2$ can be one among four choices.

Proof:

The belief $B_2$ is again conditional distribution of $V_{ref}$ given the entire history $B_0, B_1, \mathbf{a}_1, \mathbf{a}_2$ and current observation $X_2$. The proof goes through in exactly the same manner as in Lemma 2, except that $\mathbf{A}_2$ can now have four choices based on $X_1, X_2$. $\Box$

Lemma 4

The belief $B_k$ at time step k ($k > 1$) is uniformly distributed, i.e., $B_{k} \sim \mathcal{U}(\mathcal{A}_{k} )$, where the area $\mathcal{A}_k$ has two choices for any given $X_{k-1}$:

$$ \mathcal{A}_{k} (X_{k-1}) = \mathcal{A}_{k1} 1_{\lbrace {X_{k} = a_{\text {k1}}}\rbrace } + \mathcal{A}_{k2} 1_{\lbrace {X_{k} = a_{\text {k2}}}\rbrace },$$

where $\mathcal{A}_{k1}$, $\mathcal{A}_{k2}$ are partitions of $\mathcal{A}_{k-1}$ as in definitions (3) and (4). These two choices depend further upon $X_1, \cdots , X_{k-1}$, and hence in total $\mathcal{A}_2$ can be one among $2^k$ choices.

Proof:

The proof goes through in exactly the same manner as in Lemmas 2 and 3. $\Box$

B Appendix: $L^1$ Metric with $(2^{T+1}-2) > {{{\overline{R}}}^2}/{{{\underline{r}}}^2}$

Basic idea is to obtain the optimal policy using Corollary 1 till $k^*$ where

$$k^* = \arg \max _k \left\{ (2^{k+1}-2) \le \frac{{{\overline{R}}}^2}{{{\underline{r}}}^2} \right\} $$

and then using Corollary 2 for the time steps from $k^*+1$ till T. Basically this policy achieves the upper bound of Theorem 1(iii), where the minimum on the right hand side is achieved using $k^*.$

The exact details are as below for the case when ${\overline{R}}/{\underline{r}}$ is an appropriate power of 2 such that $2^{k^*+1}-2 = {\overline{R}}^2 / {\underline{r}}^2$. One can give similar construction even otherwise. But some minor details need to be considered.

Note that we exactly have $(2^{k^*+1}-2)$ disjoint balls and hence one can upper bound all the terms till $k^*$ by $|\mathcal{B}|$ as in Corollary 1. Let $a^*_{k,i}$ be as defined in Eq. (7) for all i and for all $k < k^*.$ At $k^*$ all the remaining areas $\left\{ \mathcal{A}_i^{k^*} \right\} _{i \le 2^{k^*}}$ are already of size ${\underline{r}}.$ As in Corollary 2, define for any $k > k^*$ and i

$$ a^*_{k,i} = X_{k-1} = X_{k^*}. $$

One can easily verify that $E[\tau ]$ is strictly less than $k^*$, thus the user satisfied in an average time, less than $k^*.$

$$ E[\tau ]= k^* - \frac{ (2^{k^*+1} -2 k^*- 2 ) |\mathcal{B}|}{ \mu (\mathcal{A}_0)}. $$

C Appendix: Optimal Policies in Discrete Space

We consider binary database with F features, similarity based distance and cardinality based measure. A ball $\mathcal{B} (v, r)$ here includes all those items which match in more than $F(1-r)$ features with v, e.g., $\mathcal{B}((10), 0.5) = \{(10)\}$, $\mathcal{B}((10), 1) = \mathcal{S}.$

We again use Corollary 1 to obtain optimal policies in some example scenarios. One can easily verify the following.

Case I: $F = 7$, $T=2$ and ${\underline{r}}= 2/7$: Optimal $\mathcal{Q}_{\pi ^*}$ is

$$\begin{aligned} \begin{array}{llll} a_1^1 = 1111111, &{} a^1_2 = 0000000, &{}&{} a^2_1 = 0011111, \\ a_2^2 = 1111100, &{} a_3^2 = 1100000 &{}\hbox { and } &{} a_4^2 = 0000011. \end{array} \end{aligned}$$

Case II: $F = 7$, $T=2$ and ${\underline{r}}= 3/7$: Optimal $\mathcal{Q}_{\pi ^*}$ is

$$\begin{aligned} \begin{array}{llll} a_1^1 = 1111111, &{} a^1_2 = 0000000, &{}&{} a^2_1 = 0001111, \\ a_2^2 = 111100, &{} a_3^2 = 1110000 &{}\hbox { and } &{} a_4^2 = 0000111. \end{array} \end{aligned}$$

Case III: $F = 9$, $T=2$ and ${\underline{r}}= 4/9$: Optimal $\mathcal{Q}_{\pi ^*}$ is

$$\begin{aligned} \begin{array}{llll} a_1^1 = 111111000, &{} a^1_2 = 000000111, &{}&{} a^2_1 = 001111001, \\ a_2^2 = 110001110 , &{} a_3^2 = 110000110 &{}\hbox { and } &{} a_4^2 = 001110001. \end{array} \end{aligned}$$