CRUISE on Quantum Computing for Feature Selection in Recommender Systems

Notebook for the QuantumCLEF Lab at CLEF 2024

Abstract

Using Quantum Computers to solve problems in Recommender Systems that classical computers cannot address is a worthwhile research topic. In this paper, we use Quantum Annealers to address the feature selection problem in recommendation algorithms. This feature selection problem is a Quadratic Unconstrained Binary Optimization (QUBO) problem. By incorporating Counterfactual Analysis, we significantly improve the performance of the item-based KNN recommendation algorithm compared to using pure Mutual Information. Extensive experiments have demonstrated that the use of Counterfactual Analysis holds great promise for addressing such problems.

1 Introduction

Collaborative filtering technology [1, 2], which predicts potential user-item interactions based on the patterns of user behavior and item characteristics, is widely applied in recommendation algorithms, Some well-known techniques in this field include matrix factorization methods [3], neighborhood-based methods [4], deep learning approaches [5, 6], graph-based techniques [7, 8], factorization machines [9], hybrid methods [10], Bayesian methods [11], and large language models (LLMs) [12]. However, collaborative filtering technology [1] heavily relies on the quality of data. For instance, using user profiles, item features, reviews, images, and other information can significantly improve the performance of recommendation algorithms, but in some cases, it can also decrease their performance. Therefore, it’s critical to distinguish what information are useful for recommendations so as to help the the construction of efficient systems and reduction of energy consumption [13, 14, 15, 16]. Quantum computers, with its use of qubits and quantum effects like superposition, entanglement, and quantum tunneling, is an effective tool for identifying useful information from redundant data [17]. It significantly enhances the processing speed of search problems and large integer factorization [18]. Therefore, in this paper, we aim to find useful features for recommendations by leveraging quantum computing techniques. Our goal is to improve the efficiency and accuracy of recommendation systems by identifying and utilizing relevant data, thereby reducing computational requirements and energy consumption [18, 19, 20].

In QuantumCLEF 2024, we focus on Task 1B, where 150 and 500 features are provided for each item, respectively[21, 22]. We will analyze these features to extract the most relevant ones for recommender systems. The task requires participants to use Quantum Annealing and Simulated Annealing to select appropriate features from the given data for an Item-Based KNN recommendation algorithm (Item-KNN). The organizers provided an example of feature selection by using Mutual Information [18]. However, our preliminary experiments showed that using only Mutual Information for feature selection resulted in limited improvement in the performance of Item-KNN compared to using all features without any selection. This is because Mutual Information only reflects the mutual relationship between two variables and is not associated with the final goal of the recommendation algorithm. Therefore, to achieve better performance, we propose taking the impact of features on recommendation quality into consideration when performing feature selection.

One approach to achieve this is through Counterfactual Analysis [23], which is a causal research tool to examine the impact of a factor on the final result by hypothesizing the absence or alteration of that factor. This approach mainly considers three aspects: Which factors need to be evaluated? What metrics are used to assess the impact of these factors on the model’s outcomes? And what models are used to derive the values of these metrics? In this work, due to the limited time for this task, we aim to measure and explore the impact of item features by Counterfactually Analyzing their effect on nDCG [24] performance of recommendation lists and we chose the KNN-based recommendation algorithm, a commonly used method in collaborative filtering, to perform these measurements. Specifically, we used Item-KNN to derive the change in nDCG values after removing a specific item feature. Since Mutual Information can reflect the relationship between two features, which may positively affects the final results, we did not discard it. Instead, we integrated the results of Counterfactual Analysis into Mutual Information using a temperature coefficient, which is used to control the influence of Counterfactual Analysis on the final results. Given the current limitations on the number of qubits in Quantum Computers, directly performing Quantum Annealing on 500 variables remains a challenging task. Therefore, in this task, we first partitioned the 500 features into subsets manageable by the Quantum Computer, and then combined the results.

The paper is organized as follows: Section 2 introduces related works; Section 3 describes the QUBO formulation, how Mutual Information is applied to QUBO for feature selection, and our proposed method of using Counterfactual Analysis for feature selection in QUBO; Section 4 explains our experimental setup and experimental result; Section 5 discusses our main findings; finally, Section 6 draws some conclusions and outlooks for future work.

2 Related Work

2.1 Quantum Computers

In recent years, the rapid development of Quantum Computers has demonstrated their tremendous potential in solving problems that Classical Computer cannot address, such as NP and NP-hard problems [25]. Based on their functionality and application scenarios, Quantum Computers can be categorized into Universal Quantum Computers, Quantum Annealers, Quantum Machine Learning Accelerators, and others [26]. Recent studies have utilized Quantum Annealers for feature selection to enhance the performance of recommendation systems or retrieval systems [27, 28, 18]. Nembrini et al. [27] attempted to apply Quantum Computers to recommendation systems by using Quantum Annealing to solve a hybrid feature selection approach. Their work demonstrates that current Quantum Computers are already capable of addressing real-world recommendation system problems. Nikitin et.al.[28] reproduced Nembrini’s work and employed Tensor Train-based Optimization (TTOpt) as an optimizer for the cold start problem in recommendation systems. MIQUBO [18] discussed the problem of feature selection using Quantum Computers and formalizes it as a Quadratic Unconstrained Binary Optimization (QUBO) problem. It demonstrates the potential of Quantum Computers to solve ranking and classification problems more efficiently.

2.2 Counterfactual Analysis

Existing deep learning models have complex decision-making processes that are difficult for people to understand, often functioning as black-box models, Counterfactual Analysis is a highly effective method for helping people understand these complex models and robust them [29]. For example, $\textbf{CF}^{2}$ [30] used Counterfactual Analysis to explore the explanations of Graph Neural Networks. In recommender systems, Counterfactual Analysis is primarily used for explainability and to combat data sparsity. ACCENT [31] was the first to apply Counterfactual Analysis to neural network-based recommendation algorithms. CountER [32] utilizes Counterfactual Analysis to construct a low-complexity, high-strength model for explaining recommendation systems. It also highlights that using Counterfactual Analysis contributes to the interpretability and evaluation of recommendation systems. Zhang et al [33] designed a CauseRec framework that utilizes Counterfactual to enhance representations in the data distribution, aiming to mitigate data sparsity.

In summary, Counterfactual Analysis can help people understand complex deep learning decision systems and has the potential to analyze how various factors interact in recommendation systems. Given the current advancements in Quantum Computers, utilizing Counterfactual Analysis combined with the ability of Quantum Computers to handle NP problems presents a promising direction.

3 Methodology

3.1 Preliminary

3.1.1 QUBO Formulation

In this work, we follow the approach described in [18], which utilizes Quantum Annealing for feature selection. To apply these methods, the feature selection problem is formulated as a Quadratic Unconstrained Binary Optimization (QUBO) problem. The QUBO formulation can be used to solve certain NP and NP-hard optimization problems and is defined as follows [18]:

$$\min Y=x^{T}Qx,$$ (1)

where $x$ is a binary vector of length $m$, with each element of the vector being either 0 or 1. $Q$ is a symmetric matrix, where each element represents the relationship between the elements of $x$. $m$ denotes the number of features to be selected. In other words, the elements of vector $x$ indicate whether the corresponding features are selected, and the elements in $Q$ influence the search direction of the function, determining feature selection.

3.1.2 Feature Selection Based on Mutual Information

Following [18], Mutual Information QUBO (MIQUBO) is a quadratic feature selection model based on Mutual Information. MIQUBO aims to maximize the Mutual Information, which measures the dependency between two variables, and the Conditional Mutual Information, which measures the dependency between two variables given a target variable, of the selected features. In this context, the matrix $Q$ in Equation 1 is defined as:

$$Q_{ij}=\begin{cases}-\text{CMI}(f_{i};y\mid f_{j})&\text{if }i\neq j\\ -\text{MI}(f_{i};y)&\text{if }i=j,\end{cases}$$ (2)

where $\text{MI}(f_{i};y)$ is the Mutual Information between feature $f_{i}$ and target feature $y$, and $\text{CMI}(f_{i};y\mid f_{j})$ is the Conditional Mutual Information between feature $f_{i}$ and target feature $y$ given feature $f_{j}$. Since QUBO formulation is used to find the minimum state, a negative sign is required before MI and CMI.

To control the number of selected features, a penalty term is added to Equation 1, which is then transformed to:

$$\min Y=x^{T}Qx+\left(\sum_{i=1}^{N}x_{i}-k\right)^{2}.$$ (3)

This formula will be minimized when selecting $k$ features, this also following the descriptions in [18].

3.2 Counterfactual Analysis

To better identify features directly associated with recommendation performance, we integrate a widely used recommendation ranking metric into Mutual Information through Counterfactual Analysis.

3.2.1 Counterfactual Analysis for Feature Selection

Counterfactual Analysis [23] is usually used to examine the causal relationship between conditions, decisions, and outcomes by hypothesizing how the results of observed events would change if the conditions and decisions were altered. In the field of Recommender System, Counterfactual Analysis is often used for the interpretability of recommendation models, helping researchers enhance algorithm performance [32, 33]. Inspired by existing works [32, 33], the impact of item features can be explored by excluding the corresponding feature and analyzing the difference in recommendation performance between the recommendation lists generated by the model with and without the corresponding feature.

In this work, we use the widely used Item-KNN recommendation algorithm, termed as model $G$, and employ the recommendation performance metric Normalized Discounted Cumulative Gain (nDCG) [24] for Counterfactual Analysis. nDCG is defined as:

$$\text{E}_{i}=\text{nDCG}_{G(\text{F})}-\text{nDCG}_{G(\text{F}\setminus{f_{i}})},$$ (4)

where $E_{i}$ represents the change in the nDCG result of the recommendation model $G$ after removing the feature $f_{i}$. $\text{nDCG}_{G(\text{F})}$ represents the nDCG@10 value obtained by the $G$ using all item features set $F$, while $\text{nDCG}_{G(\text{F}\setminus{f_{i}})}$ represents the nDCG@10 value obtained by the $G$ using features set which is set $F$ removing feature $i$. It is important to note that $E_{i}$ ultimately reflects the impact of feature $i$ on the result. Since the final outcome is influenced by the interactions between all features, simply removing features with positive $E_{i}$ values does not yield the optimal feature selection solution.

When $E_{i}\geq 0$, it indicates that the algorithm’s performance decreases after removing the feature $i$. The extent of this decrease reflects the positive impact of this feature on the algorithm. Conversely, an increase in the value reflects the negative impact of this feature on the algorithm. We hypothesize that if the selected set of features is $set(F^{*})$, the maximization the sum of $E_{i}$ ($i\in set(F^{*})$), the maximization the performance improvement of the baseline algorithm. Since the QUBO problem is a minimization optimization problem, we redefine $Q$ as follows:

$$Q_{ij}=\begin{cases}-CMI(f_{i};y\mid f_{j})&\text{if }i\neq j\\ -MI(f_{i};y)-\lambda\text{E}_{i}&\text{if }i=j\end{cases}$$ (5)

where $\lambda$ is a coefficient used to control the influence of $E$ on the search results. The larger the value of $\lambda$, the greater the influence of $E$ on the final results. The overall process of the above algorithm, which we refer to as Counterfactual Analysis QUBO (CAQUBO), is as follows in Algorithm 1.

Algorithm 1 Counterfactual Analysis QUBO

1:Initialize variable set E, set F, $n\leftarrow len(\text{F})$, $k$, $Q$, $\lambda$

2:procedure Calculate $\text{E}_{i}$

3:     for $f_{i}$ in F do

4:         $\text{F}^{\text{'}}\leftarrow\text{F}$

5:         $\text{F}^{\text{'}}$.pop($f_{i}$)

6:         $\text{E}_{i}\leftarrow$G(F) - $\text{G}(\text{F}^{\text{'}})$

7:     end for

8:     return E

9:end procedure

10:procedure Feature Selection

11:     Calculate MI and CMI

12:     for $f_{i}$ in F do

13:         $Q_{ii}=-\text{MI}(f_{i};y)-\lambda\text{E}_{i}$

14:     end for

15:     for $f_{i}$ in F do

16:         for $f_{j}$ in F do

17:              $Q_{ij}=-\text{CMI}(f_{i};y\mid f_{j})$

18:         end for

19:     end for

20:     set $\text{F}^{*}\leftarrow$ QA or SA $\leftarrow Q$ and $\lambda$ # Input parameters $Q$ and $\lambda$ into the Quantum Annealer.

21:     return set $\text{F}^{*}$ # Selected Feature Set

22:end procedure

3.3 Handling Large Feature Set

Although Quantum Computers are developing rapidly, the limitation in the number of qubits restricts them to handling only a limited number of feature selection problems. For selecting from 500 features, we partition them into several subsets and use Quantum Annealing (QA) or Simulated Annealing (SA) to perform feature selection on these subsets individually, then combine the results.

First, partition the 500 features into $n$ subsets by order, ${S_{1},S_{2},\cdots,S_{i},\cdots,S_{n}}$, where $S_{i}$ is the $i$-th subset of features, and $n$ is the number of subsets.

$${S_{1},S_{2},\cdots,S_{i},\cdots,S_{n}}=\text{divide(F)}$$ (6)

Then, use Quantum Annealing (QA) or Simulated Annealing (SA) to perform feature selection on each subset, and combine the results:

$$\tilde{S}=\bigcup_{i=1}^{n}\text{QA/SA}(S_{i}),$$ (7)

where $\tilde{S}$ is the final selected features set, represents each partitioned subset of features, and QA/SA $(S\_i)$ represents the selected features from subset $S_{i}$ using QA and SA. The final feature set is obtained by merging the selected features from all subsets.

4 Experimental Setup

Datasets: In this work, two tasks are undertaken: the first involves selecting appropriate features from a set of 150 item features for training $G$, and the second involves selecting features from a set of 500 item features. Three data sets are provided for these tasks: 150_ICM, 500_ICM, and URM. The 150_ICM and 500_ICM contain item features, while the URM includes interaction data between 1,890 users and 18,022 interacted items.

Experimental parameter setting: We used a self-implemented Item-KNN recommendation model based on the problem statement to calculate $E$. The interaction data was split into training and test sets in an 80:20 ratio. It is worth noting that calculating $E$ is very time-consuming, so we only used a subset of items for the calculations. In the use of Quantum Annealing (QA) and Simulated Annealing(SA), the coefficient $\lambda$ significantly affects the features selected by QA and SA. Due to the limited usage time of the Quantum Annealer (QA), it is necessary to use Simulated Annealing (SA) to explore the effectiveness of the selected features under different parameters $\lambda$ and $k$ before using QA. In preliminary experiment, we attempt [$\bm{\lambda}$: 0, 1e1, 1e3, 1e5, 1e7], [k: 50, 100, 130, 140, 145] in Feature 150 and [$\bm{\lambda}$: 0, 1e1, 1e3, 1e5, 1e7], [k: 300, 350, 400, 450, 470] in Feature 500. For the selection of 500 features, n (is mentioned in Section 3.3) is set to 5. The preliminary experiment results can be found in Table 1.

Repeated Calculations: Due to the heuristic nature of Simulated Annealing (SA) and Quantum Annealing (QA), the final results may vary even with fixed parameters. To mitigate this effect, we perform multiple iterations of QA and SA under the same parameters and select the final feature set via voting. For example, we repeated the experiment five times. $f_{i}$ was not included in $F^{*}$ in any of the five experiments, while $f_{j}$ was included in $F^{*}$ in four out of the five experiments. Therefore, the final submitted feature set $F^{*}$ does not include $f_{i}$ but includes $f_{j}$.

Table 1: nDCG@10 for Feature 150 and Feature 500 datasets individually using SA-based feature selection, with different numbers of selected features $k$ and different coefficients $\lambda$.

. k 50 100 130 140 145 300 350 400 450 470 $\lambda$ Feature 150 nDCG@10 Feature 500 nDCG@10 0 0.0602 0.0870 0.0968 0.1033 0.1018 0.1078 0.0894 0.0971 0.0969 0.0991 1 0.0870 0.0974 0.0999 0.1009 0.1029 0.1066 0.1108 0.1195 0.1291 0.1197 1e3 0.0755 0.1051 0.1151 0.1119 0.1152 0.1206 0.1249 0.1257 0.1305 0.1302 1e5 0.0878 0.1160 0.1232 0.1256 0.1180 0.1224 0.1238 0.1303 0.1290 0.1307 1e7 0.0795 0.1155 0.1221 0.1264 0.1180 0.1235 0.1218 0.1298 0.1306 0.1293 150 Feature nDCG 0.1028 500 Feature nDCG 0.0988

Table 2: This table contains the final data submitted to the organizers, with data sourced from the organizers’ $\text{website}^{\textbf{1}}$. Due to the fact that when $\bm{\lambda}$ is too large, the values of elements in Q become excessively large, which is detrimental to the performance of QA and SA, a coefficient $\bm{\mu}$ is applied to all elements in Q. An asterisk (*) after the sub_ID indicates that the selected features are the result of repeated calculations. Those submissions was repeated five times to determine the final feature set.

150 Feature submissions	All Feature nDCG 0.0810
Parameters set	nDCG@10	Annealing Time	Type	nº features	sub_id
k=140 ${\lambda}$=1e7 $\mu$=1e-5	0.0805	536250	Q	138	1
k=140 ${\lambda}$=1e7 $\mu$=1e-3	0.0826	528844	Q	136	2
k=140 ${\lambda}$=1e7 $\mu$=1e-3	0.0690	530804	Q	132	3
k=140 ${\lambda}$=0 $\mu$=1	0.0763	558321	Q	133	4
k=140 ${\lambda}$=1e7 $\mu$=1e-2	0.1003	1375068	Q	144	$\text{5}^{*}$
k=140 ${\lambda}$=1e7 $\mu$=1e-5	0.0998	1745487	S	140	1
k=140 ${\lambda}$=1e7 $\mu$=1e-3	0.0993	17357899	S	140	2
k=140 ${\lambda}$=1e7 $\mu$=1e-3	0.1001	1760252	S	140	3
k=140 ${\lambda}$=0 $\mu$=1	0.0793	17387227	S	140	4
k=140 ${\lambda}$=1e7 $\mu$=1e-2	0.1003	88395437	S	144	$\text{5}^{*}$
500 Feature submissions	All Feature nDCG 0.0827
k=450 ${\lambda}$=1e7 $\mu$=1e-2	0.0757	2287019	Q	407	1
k=450 ${\lambda}$=1e1 $\mu$=1	0.0839	2122701	Q	397	2
k=450 ${\lambda}$=1e7 $\mu$=1e-2	0.1196	43339285	S	450	1
k=450 ${\lambda}$=1e1 $\mu$=1	0.1198	42776695	S	450	2

• https://qclef.dei.unipd.it/clef2024-results.html

5 Results

Table 1 describes the performance in nDCG@10 of $G$ using features selected by QA and SA under different parameters $\lambda$ and $k$. When $\lambda=0$, QA and SA select features based solely on Mutual Information (MI) and Conditional Mutual Information (CMI). Across different values of parameter $k$, the performance of selected features in $G$ rarely surpasses the performance in Counterfactual Analysis QUBO. As the parameter $\lambda$ increases, the performance of the features selected by QA and SA in the item-KNN shows significant improvement compared to using all features. The effectiveness of feature selection shows no significant improvement when $\lambda>1e5$ . This may be because as the value of $\lambda$ increases, the impact of MI and CMI on feature selection diminishes, causing QA and SA to rely entirely on $E$ for feature selection.

Table 2 reflects the same situation: feature selection relying solely on MI and CMI does not surpass the performance in Counterfactual Analysis QUBO. After incorporating the counterfactual analysis-derived $E$ into $Q$, the features selected by QA and SA show a significant performance improvement in item-KNN compared to using all features. An unusual observation is that, under the same parameters, the features selected by QA generally do not perform as well as those selected by SA in item-KNN, and sometimes do not even surpass the performance of using all features. During the experiments, we noticed that this is due to QA often returning results before finding the optimal solution.

6 Conclusions and Future Work

In this paper, we present the explorations conducted by our team and the details of our final submission for the QuantumCLEF 2024 activities. We used Counterfactual Analysis of individual item features to select appropriate features for item-KNN using Quantum Annealing. Our preliminary experiments and the results returned by QuantumCLEF 2024 demonstrated that our use of Counterfactual Analysis significantly improved the performance of item-KNN.

Within the limited time of QuantumCLEF, we attempted Counterfactual Analysis of individual features. However, because the performance of collaborative filtering is actually the result of feature interactions, Counterfactual Analysis of individual features has significant limitations. Additionally, since Quantum Annealing cannot directly handle the selection of 500 features, we adopted a sequential partitioning and merging approach. As negative features are not uniformly distributed by their indices among all features, this sequential partitioning and merging method still requires improvement.