Counterfactual Explanation for Fairness in Recommendation

Recommender systems have long dealt with major concerns of recommendation unfairness, which profoundly harm user satisfaction [16, 30] and stakeholder benefits [6, 20, 31]. Recent works on fairness-aware recommendation mainly discuss two primary topics, i.e., user-side fairness [2, 9, 16, 30, 32, 70] and item-side fairness [3, 13, 17, 33]. User-side fairness concerns whether the recommendation is fair to different users/user groups, e.g., retaining equivalent accuracy or recommendation explainability. Relevant approaches attribute the causes of user-side unfairness to discrimination factors, such as sensitive features (e.g., gender [9, 70], age [32]) and user inactiveness [16, 30], and the like. They mainly propose fairness metrics to constraint recommendation models (e.g., collaborative filtering [70]) to produce fair recommendations. For example, Yao and Huang [70] study the unfairness of collaborative filtering (CF)-based recommenders on gender-imbalanced data. They propose four metrics to assess different types of fairness, then add these metrics as constraints to the CF model learning objective to produce fair recommendations. Chen et al. [9] investigate gender-based inequalities in the context of resume ranking engines. They use statistical tests to prove the existence of direct and indirect gender discrimination in resume ranking results provided to job recruiters. Li et al. [30] investigate the unfair recommendation between active and inactive user groups, and provide a re-ranking approach to mitigate the activity unfairness by adding constraints over evaluation metrics of ranking. As modern content providers are more concerned about user privacy, it is generally not easy to access sensitive user features for the recommendation [40]. Meanwhile, users often prefer not to disclose personal information that raises discrimination [5], limiting the application of user-side fairness recommendations. Another topic of item-side fairness-aware recommendation [3, 13, 17, 33] is interested in examining whether the recommendation treats items fairly, e.g., similar ranking prediction errors for different items, fair allocations of exposure to each item. For instance, Abdollahpouri et al. [3] address item exposure unfairness in learning-to-rank (LTR) recommenders. They include a fairness regularization term in the LTR objective function, which controls the recommendations favored toward popular items. Ge et al. [17] consider the dynamic fairness of item exposure due to changing group labels of items. They calculate the item exposure unfairness with a fairness-related cost function. The cost function is merged into a Markov Decision Process to capture the dynamic item exposure for recommendations. Liu et al. [33] focus on item exposure unfairness in interactive recommender systems (IRS). They propose a reinforcement learning method to maintain a long-term balance between accuracy and exposure fairness in IRS. Other successful investigations of fairness can also be closely related to enhancing recommendation diversity [27, 35], e.g., increase aggregate diversity [35], and increasing fairness transparency of graph neural networks [14].

Despite the great efforts, fairness-aware recommendations mitigate user and item unfairness in a black-box manner but do not explain why the unfairness appears. Understanding the “why” is desirable for both model transparency [31] and facilitates data curation to remove unfair factors [61]. Limited pioneering studies are conducted to explain fairness. Begley et al. [4] estimate Shapley values of input features to search which features contribute more to the model unfairness. Deldjoo et al. [11] uses regression-based explanatory modeling to investigate the relationship between data characteristics and the fairness of recommendation models. They define data characteristics as explanatory variables (EVs) and the recommendation outcome as dependent variables (DVs). Through statistical significance tests with informed p-values, they determine the EVs that are significant for recommendation models. Ge et al. [19] develop a counterfactual explainable fairness model for recommendation to explain which item aspects influence item exposure fairness. They perform perturbations on item aspect scores, and then apply perturbed aspect scores on two pre-defined matrices to observe fairness changes. These prior efforts suffer from major limitations: (1) The high computational burden caused by the large-scale search space and the greedy nature of the explanation search process. Estimating Shapley values and p-values should consider all features/characteristics across possible coalitions [31]. This poses a challenge of exponentially growing search space when the number of features increases. As the feature space of a recommendation system often comprises a large number of user/item features, estimating Shapley values and p-values becomes computationally burdensome. Furthermore, statistical estimations only consider the relative importance of features for model fairness. Removing these features/characteristics based on their importance does not guarantee a reduction in the unfairness of the model. (2) Both existing methods generate explanations by feature-level optimizations that do not apply to discrete attributes. Different from existing works, our work focuses on generating attribute-level counterfactual explanations, aiming to provide faithful fairness explanations for attribute-aware models. We consider explaining recommendation unfairness based on attributes from a Heterogeneous Information Network. We also reduce the large search space by attentive action pruning in the off-policy learning environment.

Heterogeneous Information Network (HIN) is a powerful structure that allows for the heterogeneity of its recorded data, i.e., various types of attributes, thus providing rich information to empower recommendations [54, 60]. HINs have been widely adopted in recommendation models to boost their performance; representative works cover context-based filtering (e.g., SemRec [43], HERec [42]) and knowledge-based systems (e.g., MCrec [26], HAN [53]). For instance, HERec [42] embeds meta-paths within a HIN as dense vectors, then fuses these HIN embeddings with user and item embeddings to augment the semantic information for recommendations. MCrec [26] leverages a deep neural network to model meta-path-based contextual embeddings and propagates the context to user and item representations with a co-attention mechanism. Those recommendation models observe promising improvements by augmenting contextual and semantic information given by HINs. Despite the great efforts, prior works do not consider using the HIN to explain unfair factors in recommendations. Novel to this work, we first attempt to leverage rich attributes in a HIN to provide counterfactual explanations for item exposure fairness.

Counterfactual explanations have been considered as satisfactory explanations [29, 65, 71] and elicit causal reasoning in humans [8, 55, 72, 73]. Works on counterfactual explanations have been proposed very recently to improve the explainability of recommendations. Xiong et al. [67] propose a constrained feature perturbation on item features and consider the perturbed item features as explanations for ranking results. Ghazimatin et al. [21] perform random walks over a Heterogeneous Information Network to look for minimal sets of user action edges (e.g., click) that change the PageRank scores. Tran et al. [46] identify minimal sets of user actions that update the parameters of neural models. Our work differs from prior works on counterfactual explanations by two key points: (1) In terms of problem definition, they generate counterfactual explanations to explain user behaviors (e.g., click [21, 46]) or recommendation (e.g., ranking [67]) results. Our method generates counterfactual explanations to explain which attributes affect recommendation fairness. (2) In terms of technique, our method formulates counterfactual reasoning as reinforcement learning, which can deal with ever-changing item exposure unfairness.

In order to generate attribute-level fairness explanations, we use a Heterogeneous Information Network (HIN) to provide auxiliary attributes for finding potential fairness-related attributes, e.g., user gender. The HIN has shown its power in modeling various types of attributes, e.g., user social relations and item brands. In particular, suppose we have the logged data that records users’ historical behaviors (e.g., clicks) in the recommendation scenario. Let \(\mathcal {U} \in \mathbb {R}^{M}\) , \(\mathcal {I} \in \mathbb {R}^{N}\) denote the sets of users and items, respectively. We can define a user-item interaction matrix \(\boldsymbol {Y}=\left\lbrace y_{uv} \mid u \in \mathcal {U}, v \in \mathcal {I}\right\rbrace\) according to the logged data. We also have additional attributes from external resources that profile users and items, e.g., users’ genders and items’ genres. The connections between all attributes and users/items are absorbed in the relation set \(\mathcal {E}\) . Those attributes, with their connections with user-item interactions, are uniformly formulated as a HIN. Formally, a HIN is defined as \(\mathcal {G}=(\mathcal {V}^\prime ,\mathcal {E}^\prime)\) , where \(\mathcal {V}^\prime =\mathcal {U} \cup \mathcal {I} \cup \mathcal {V}_U \cup \mathcal {V}_I\) , and \(\mathcal {E}^\prime = \lbrace \mathbb {I}({y_{uv})}\rbrace \cup \mathcal {E}\) . \(\mathbb {I}(\cdot)\) is an edge indicator that denotes the observed edge between user u and item v when \(y_{uv} \in \boldsymbol {Y}=1\) . \(\mathcal {V}_U\) and \(\mathcal {V}_I\) are attribute sets for users and items, respectively. Each node \(n \in \mathcal {V}^\prime\) and each edge \(e \in \mathcal {E}^\prime\) are mapped into specific types through node type mapping function: \(\phi : \mathcal {V}^\prime \rightarrow \mathcal {K}\) and edge type mapping function: \(\psi : \mathcal {E}^\prime \rightarrow \mathcal {J}\) . \(\mathcal {G}\) maintain heterogeneity, i.e., \(|\mathcal {K}|+|\mathcal {J}| \gt 2\) .

We consider explaining the item exposure (un)fairness in recommendations. We first split items in historical user-item interactions into head-tailed group \(G_{0}\) the long-tailed group \(G_{1}\) ¹. Note that items from the head-tailed group refer to the top popular items that dominate historical user-item interactions. Following previous works [17, 19], we use demographic parity (DP) and exact-K (EK) defined on item subgroups to measure whether a recommendation result is fair. In particular, DP requires that each item has the same likelihood of being classified into \(G_{0}\) and \(G_{1}\) . EK regulates the item exposure across each subgroup to remain statistically indistinguishable from a given maximum \(\alpha\) . By evaluating the deviation of recommendation results from the two fairness criteria, we can calculate the fairness disparity, i.e., to what extent the recommendation model is unfair. Formally, giving a recommendation result \(H_{u, K}\) , the fairness disparity \(\Delta (H_{u, K})\) of \(H_{u, K}\) is:where K is the recommendation list length of \(H_{u, K}\) , \(\Delta (\cdot)\) is the fairness disparity metric that quantifies model fairness status. \(\lambda\) is the trade-off parameter between DP and EK, where \(\lambda\) is chosen from \(\lbrace 0,1\rbrace\) and \(\lambda =0\) representing DP is considered as the fairness measurement while \(\lambda =1\) denoting EK is used as the fairness measurement. The definition of \(\Delta (\cdot)\) enables us to choose the optimal disparity definition for a variety of recommendation tasks, e.g., music and movie recommendations. \(\text{Exposure}\left(G_{j} \mid H_{u, K}\right)\) is the item exposure number of \(H_{u, K}\) within \(G_j\) w.r.t. \(j \in \lbrace 0,1\rbrace\) .

This work aims to generate attribute-level counterfactual explanations for item exposure fairness. In particular, we aim to find the “minimal” changes in attributes that reduce the fairness disparity (cf. Equation (1)) of item exposure. Formally, given historical user-item interaction \(\boldsymbol {Y}\) , and user attribute set \(\mathcal {V}_U\) and item attribute set \(\mathcal {V}_I\) extracted from an external Heterogeneous Information Network (HIN) \(\mathcal {G}\) . Suppose there exists a recommendation model that produces the recommendation result \(H_{u, K}\) for user u. Given all user-item pairs \((u,v)\) in \(H_{u, K}\) , our goal is to find a minimal attributes set \(\mathcal {V}^{*} \subseteq \lbrace \lbrace e_u, e_v\rbrace \mid (u, e_u), (v, e_v) \in \mathcal {E}^\prime , e_u \in \mathcal {V}_U, e_v \in \mathcal {V}_I\rbrace\) . Each attribute in \(\mathcal {V}^{*}\) is an attribute entity from HIN \(\mathcal {G}\) , e.g., user’s gender and item’s genre. With a minimal set of \(\mathcal {V}^{*}\) , the counterfactual reasoning pursues to answer: what the fairness disparity would be, if \(\mathcal {V}^{*}\) is applied to the recommendation model. \(\mathcal {V}^{*}\) is recognized as a valid counterfactual explanation for fairness, if after applied \(\mathcal {V}^{*}\) , the fairness disparity of the intervened recommendation result \(\Delta (H_{u, K}^{cf})\) is reduced compared with original \(\Delta (H_{u, K})\) . In addition, \(\mathcal {V}^{*}\) is minimal such that there is no smaller set \(\mathcal {V}^{*^{\prime }}\in \mathcal {G}\) satisfying \(|\mathcal {V}^{*^{\prime }}| \lt |\mathcal {V}^{*}|\) when \(\mathcal {V}^{*^{\prime }}\) is also valid.

As shown in Figure 2, our CFE model is crafted within an off-policy learning environment, in which an explanation policy \(\pi _E\) is optimized to produce attribute-level counterfactual explanations for fairness. At each state \(s_t\) , \(\pi _E\) produces actions \(a_t\) absorbing user and item attributes as potential counterfactual explanations. These actions are committed to the recommendation model and graph representation module to produce the reward \(r(s_t, a_t)\) for optimizing \(\pi _E\) . Specifically, the graph representation module provides dense vectors \(\mathbf {h}_{u}\) , \(\mathbf {h}_v\) , \(\mathbf {e}_{u}\) and \(\mathbf {e}_{v}\) as user, item, user attribute and item attribute embeddings, respectively. Those embeddings are used in the state representation learning (i.e., learn \(s_t\) ) and attentive action pruning (i.e., select \(a_t\) ) in our CFE model. Moreover, the attribute embeddings are fused with user or item latent factors learned by the recommendation model to explore the model fairness change. In particular, the fused embeddings of users and items are used to predict the intervened recommendation result \(H_{u, K}^{cf}\) . By comparing the fairness disparity (cf. Equation (1)) difference between \(H_{u, K}^{cf}\) and the original recommendation \(H_{u, K}\) , we determine the reward \(r(s_t, a_t)\) to optimize \(\pi _E\) , accordingly. The reward \(r(s_t, a_t)\) measures whether the current attribute (i.e., action) is a feasible fairness explanation responsible for the fairness change. Finally, \(\pi _{E}\) is optimized with a counterfactual risk minimization (CRM) objective \(\nabla _{\Theta } \widehat{R}\left(\pi _E\right)\) to balance the distribution discrepancy from the logging policy \(\pi _0\) .

Our graph representation module conducts heterogeneous graph representation learning to produce dense vectors of users, items, and attributes among the HIN. Compared with homogeneous graph learning such as GraphSage [23], our graph representation injects both node and edge heterogeneity to preserve the complex structure of the HIN. In particular, we include two weight matrices to specify varying weights of different node and edge types.

In the following, we present the graph learning for user embedding \(\mathbf {h}_{u}\) . The embeddings of \(\mathbf {h}_{v}\) , \(\mathbf {e}_u\) , and \(\mathbf {e}_v\) can be obtained analogously by replacing nodes and node types during computations. Specifically, we first use Multi-OneHot [75] to initialize node embeddings at the 0-th layer, in which u’s embedding is denoted by \(\mathbf {h}_u^{0}\) . Then, at each layer l, user embedding \(\mathbf {h}_{u}^{l}\) is given by aggregating node u’s neighbor information w.r.t. different node and edge types:where \(\sigma (\cdot)\) is LeakyReLU [68] activation function and \(\operatorname{concat}(\cdot)\) is the concatenation operator. \(\operatorname{D}_{p}[\cdot ]\) is a random dropout with probability p applied to its argument vector. \(\mathbf {h}_{u}^{l-1}\) is u’s embedding at layer \(l-1\) . \(\mathcal {N}_{\psi (e)}(u)=\left\lbrace u^{\prime } \mid \left(u, e, u^{\prime }\right) \in \mathcal {G} \right\rbrace\) is a set of nodes connected with user node u through edge type \(\psi (e)\) . The additionally dotted two weight matrices, i.e., node-type matrix \(\mathbf {W}_{\phi (u)}^{l}\) and edge-type matrix \(\mathbf {W}_{\psi (e)}^{l}\) , are defined based on the importance of each type \(\phi (u)\) and \(\psi (e)\) . \(b^{l}\) is an optional bias.

With Equation (2), we obtain u’s embedding \(\mathbf {h}_{u}^{l}\) at each layer \(l \in \lbrace 1,\cdots , L\rbrace\) . We then adopt layer-aggregation [69] to concatenate u’s embeddings at all layers into a single vector, i.e., \(\mathbf {h}_u=\mathbf {h}_u^{(1)} + \cdots + \mathbf {h}_u^{(L)}\) . Finally, we have user node u’s embedding \(\mathbf {h}_u\) through aggregation. The item embedding \(\mathbf {h}_{v}\) , user attribute embedding \(\mathbf {e}_u\) and item attribute embedding \(\mathbf {e}_v\) can be calculated analogously.

The recommendation model \(f_R\) is initialized using user-item interaction matrix \(\boldsymbol {Y}\) to produce the Top-K recommendation result \(H_{u, K}\) for all users. Here, we employ a linear and simple matrix factorization (MF) [39] as the recommendation model \(f_R\) . Particularly, MF initializes IDs of users and items as latent factors and uses the inner product of user and item latent factors as the predictive function:where \(\boldsymbol {U}_{u}\) and \(\boldsymbol {V}_{v}\) denote d-dimensional latent factors for user u and item v, respectively. We use the cross-entropy [77] loss to define the objective function of the recommendation model:After optimizing the loss function \(\mathcal {L}_R\) , we can use the trained user and item latent factors (i.e., \(\boldsymbol {U}\) , \(\boldsymbol {V}\) ) to produce the original Top-K recommendation lists \(H_{u, K}\) for all users \(u \in \mathcal {U}\) .

We cast our CFE model in an off-policy learning environment, which is formulated as Markov Decision Process (MDP). The MDP is provided with a static logged dataset generated by a logging policy \(\pi _0\) ². The logging policy \(\pi _0\) collects trajectories by uniformly sampling actions from the user and item attribute space. We use the off-policy learning to optimize an explanation (i.e., target) policy \(\pi _E\) by approximating the counterfactual rewards of state-action pairs from all timestamps, wherein the logging policy \(\pi _0\) is employed for exploration while the target policy \(\pi _E\) is utilized for decision-making. In the off-policy setting, the explanation policy \(\pi _E\) does not require following the original pace of the logging policy \(\pi _0\) . As a result, \(\pi _E\) is able to explore the counterfactual region, i.e., those actions that haven’t been taken by the previous agent using \(\pi _0\) . Formally, at each timestamp \(t \in \lbrace 1,\cdots ,T\rbrace\) of MDP, the explanation policy \(\pi _E(a_t|s_t)\) selects an action (i.e., a candidate attribute) \(a_t \in \mathcal {A}_t\) conditioning on the user state \(s_t \in \mathcal {S}\) , and receives counterfactual reward \(r(s_t, a_t) \in \mathcal {R}\) for this particular state-action pair. Then the current state transits to the next state \(s_{t+1}\) with transition probability of \(\mathbb {P}\left(s_{t+1}\mid s_{t}, a_{t}\right)\in \mathcal {P}\) . The whole MDP has the key elements:

We now introduce the implementation of each key component.

Using state \(s_t \in \mathcal {S}\) from Equation (5), candidate action \(a_t \in \mathcal {A}_t\) from Equation (7), and counterfactual reward \(r(s_t, a_t)\) defined in Equation (9) for each timestamp t, we aim to learn an explanation policy \(\pi _E\) by maximizing the expected cumulative reward \(R(\pi _E)\) over total iteration T. Formally,where \(\tau =\left(s_0, a_0, s_1, a_1, \ldots \right)\) are trajectories and \(\tau \sim \pi _E\) means the distribution over trajectories is induced by the explanation policy \(\pi _E\) . The explanation policy \(\pi _E: \mathcal {S} \mapsto \mathcal {P}(\mathcal {A}_t)\) is a map from states \(\mathcal {S}\) to a probability distribution over actions \(\mathcal {A}_t\) , where \(\mathcal {A}_t\) are selected from the attentive action pruning. \(\mathcal {P}\) are state transitions and \(\mathbb {P}\left(s_{t+1}\mid s_{t}, a_{t}\right)\in \mathcal {P} = 1\) .

Intuitively, we can directly use \(R(\pi _E)\) in Equation (10) to guide the optimization of \(\pi _E\) . However, this would lead to a biased policy optimization [58] due to the existence of policy distribution discrepancy between \(\pi _E\) and the logging policy \(\pi _0\) . In particular, in the off-policy learning setting, the logging policy \(\pi _0\) focuses on explorations, in which a uniform-based \(\pi _0\) is adopted to explore attributes from the attribute space uniformly at random. In contrast, the target explanation policy \(\pi _E\) aims to exploit the learned knowledge and take actions that maximize the cumulative rewards given by Equation (10). As a result, the data collected under \(\pi _0\) may not align with the explanation policy’s distribution. Directly using the logged data collected by \(\pi _0\) for learning can lead to biased estimates, which has been proved in previous works [34, 41].

Figure 5 explicitly shows how the fairness (i.e., Head-tailed Rate@K) and recommendation (i.e., NDCG@K) performance of our CFairER and baselines changes along with erasure, where the x-axis shows the erasure iteration and the y-axis demonstrates the corresponding fairness and recommendation performance at \(K=\lbrace 5,20\rbrace\) . Each data point in Figure 5 is generated by cumulatively erasing a batch of attributes. Those erased attributes are selected from, at most⁸, the top 10 (i.e., \(E=10\) ) attribute sets of the explanation lists provided by each method⁹. As PopUser and PopItem baselines enjoy very similar data trends, we choose not to present them simultaneously in Figure 5. Besides, we plot the relationship between fairness and recommendation performance at each erasure iteration in Figure 6, showcasing the fairness-accuracy trade-off of our CFairER and baselines. We also use Table 2 to present the final recommendation and fairness performance of all methods after erasing \(E = [5, 10, 20]\) attributes in explanations. Note that in both Figure 5, Figure 6 and Table 2, larger NDCG@K and Hit Ratio @K values indicate better recommendation performance while smaller Head-tailed Rate@K and Gini@K values represent better fairness. Analyzing Figure 5, Figure 6 and Table 2, we have the following findings.

From Table 2, it is observed that our CFairER achieves the best recommendation and fairness performance amongst all methods after erasing attributes from our explanations. For instance, CFairER beats the strongest baseline CEF by 25.9%, 24.4%, 8.3% and 36.0% for NDCG@40, Hit Ratio@40, Head-tailed Rate@40 and Gini@40 with erasure length \(E=20\) on Yelp. This indicates that explanations generated by CFairER are more faithful to explaining unfair factors while not harming recommendation accuracy compared to all baseline methods. Unlike heuristic approaches (i.e., RDExp, PopUser, and PopItem) that rely on random attribute selection, CFairER incorporates a more complex and rational attribute selection mechanism for fairness explanation generation. Particularly, CFairER prioritizes relevant attributes that change the model fairness by using a counterfactual reward (cf. Equation (9)), incorporating two criteria of Rationality and Proximity to ensure the quality of generated counterfactual explanations. Besides, FairKGAT considers mitigating the unfairness of explanation diversity caused by different user activeness, which ignores the impact of item exposure imbalance on explanation fairness. Our CFairER mitigates the unfairness of item exposure to promote the fair allocation of user-preferred but less exposed items, thus achieving better recommendation and fairness performance than FairKGAT. Regarding CEF, although CEF generates counterfactual explanations as in our CFairER, it conducts feature-level optimizations and does not apply to discrete attributes such as gender and age. Our CFairER uses an off-policy learning agent to directly visit attributes in a given HIN, adapting to both discrete and continuous attributes when finding the counterfactual explanations. As a result, our CFairER outperforms CEF due to its generalizability to discrete attributes. Another interesting finding is that PopUser and PopItem perform even worse than RDExp (i.e., randomly selecting attributes) on LastFM. Early recommendation models largely recommend items that have popular attributes favored by users. Though this is intuitive, recommending items with popular attributes would deprive the exposure of less-noticeable items, causing serious model unfairness and degraded recommendation performance. This further highlights the importance of mitigating item exposure unfairness in recommendations.

From Figure 5, the fairness of all models consistently improves while erasing attributes from explanations, shown by the decreasing trend of Head-tailed Rate@K values. Besides, Figure 5 also shows the decreasing trend of NDCG@K values, demonstrating a decreased recommendation performance. The improved fairness performance of all methods is reasonable, as erasing attributes, even those selected randomly from attribute sets, could potentially remove unfair factors to mitigate the representation gap between popular and long-tailed items. This finding is also consistent with the finding in CEF [19]. Unfortunately, we can observe the downgraded recommendation performance of all models in Figure 5, as also evidenced by CEF [19]. For example, in Figure 5, the NDCG@5 of CEF drops from approximately 1.17 to 0.60 on LastFM at erasure iteration 0 and 50. This is due to the well-known fairness-accuracy trade-off issue, in which the fairness constraint could be achieved with a sacrifice of recommendation performance. Facing this issue, both baselines suffer from huge declines in recommendation performance. On the contrary, our CFairER still enjoys favorable recommendation performance and outperforms all baselines. Besides, the decline rates of our CFairER are much slower than baselines on both datasets in Figure 5. We hence conclude that the attribute-level explanations provided by our CFairER can achieve a much better fairness-accuracy trade-off than other methods. This is because our CFairER finds minimal but vital attributes as explanations for model fairness. Those attributes produced by CFairER are fairness-related factors but not the ones that affect the recommendation accuracy. As a result, our CFairER could alleviate the item exposure unfairness while maintaining stable recommendation performance. Another finding is that our CFairER may not outperform FairKGAT and PopUser in fairness evaluations when the number of erasure iterations is insufficient. In Figure 5(a), it is observed that the HT@5 of FairKGAT and PopUser exhibit more quick degradation compared to CFairER’s degradation at the beginning of the erasure iterations. This can be attributed to the fact that FairKGAT and PopUser construct explanations using a fixed length (i.e., \(N=20\) ), which may absorb more attributes in each explanation than CFairER’s adaptive explanation length, i.e., minimal for each explanation. Consequently, CFairER may not have a fair opportunity to hit the attributes that explain model unfairness compared to FairKGAT and PopUser, resulting in slower degradation of HT@5 during the initial erasure iterations. However, as more erasure iterations are performed, CFairER exhibits stable performance and eventually surpasses FairKGAT and PopUser in fairness evaluations. This indicates that CFairER consistently discovers suitable explanations that align with model fairness during the learning process. These explanations generated by CFairER prioritize simple yet essential attributes, in contrast to the complex combinations of attributes utilized by FairKGAT and PopUser.

Figure 6 provides deeper insights into the fairness-accuracy trade-off issue, specifically the relationship between fairness and recommendation performance metrics, i.e., Head-tailed Rate@K and NDCG@K. By analyzing Figure 6, we find that our CFairER achieves the best fairness-accuracy trade-off compared to the baseline methods on the Douban Movie and LastFM datasets. This is evident from the blue curves positioned to the left-hand side of the diagonals of Figure 6(c) (d) (e) (f). Moreover, we observe that RDExp performs the poorest on the Douban Movie dataset, while PopUser exhibits the worst performance on the LastFM dataset. In our experiments, the trade-off is caused by the disagreement between the goals of item exposure fairness and user preference. While we aim to align fair allocations to item exposures, the recommendation model primarily focuses on selecting items similar to those in users’ historical interactions. We thus conclude the attributes found by our CFairER are not necessarily similar to the attributes of historical items. Instead, they are sensitive attributes that cause the recommendations to favor the historically popular items. Another finding is that our CFairER initially does not outperform other baselines during the early erasure process on the Yelp dataset, as depicted in Figure 6(a) (b). We attribute this sub-optimal performance on Yelp to the extremely sparse nature of the dataset, i.e., a density of only 0.086%. Compared with Douban Movie (0.63% density) and LastFM (0.28% density), Yelp records a larger number of users that have few interactions with items. While other baseline methods may rely on popular attributes to predict the preferences of those users, CFairER takes a different approach. It aims to identify sensitive attributes that may not necessarily be the most popular ones. Consequently, at the beginning of the erasure process, CFairER may struggle to adapt to the data sparsity of the Yelp dataset, resulting in sub-optimal performance. However, as more erasure iterations are performed, CFairER surpasses baseline models and achieves the best fairness-accuracy trade-off. This demonstrates the effectiveness of CFairER in progressively refining its attribute selection process and achieving a favorable fairness-accuracy trade-off. With more iterations, CFairER can identify the appropriate attributes that better explain item exposure fairness and align with user preferences.

We conduct an in-depth ablation study on the ability of our CFairER to achieve favorable generalizability, sample efficiency and bias alleviation. In particular, our CFairER includes three important components that contribute to the good performance of CFairER, i.e., recommendation model (cf. Section 4.3), attentive action pruning (cf. Section 5.1) and counterfactual risk minimization-based optimization (cf. Section 5.2). We evaluate our CFairER with different variant combinations and show our main findings below.

We analyze how erasure length E (cf. Section 6.1.3) and candidate size n (as in Equation (7)) impact the performance of CFairER. Erasure length E is the number of erased attributes selected from each explanation, which determines the erasure size for evaluations. Candidate size n is the number of candidate actions selected by our attentive action pruning. We present the evaluation results of CFairER under different E and n on Yelp and LastFM in Figure 7. Since the results on Douban Movie draw similar conclusions to the other datasets, we choose not to present them here.

Apparently, the performance of CFairER demonstrates decreasing trends from \(E=5\) , then becomes stable after \(E=10\) . The decreased performance is due to the increasing erasure of attributes found by our generated explanations. This indicates that our CFairER can find valid attribute-level explanations that impact fair recommendations. The performance of CFairER degrades slightly after the bottom, then becomes stable. This is reasonable since the attributes number provided in datasets are limited, while increasing the erasure length would allow more overlapping attributes with previous erasures to be found.

By varying candidate size n from \(n=[10, 20, 30, 40, 50, 60]\) in Figure 7(c) (d), we observe that CFairER performance first improves drastically as candidate size increases on both datasets. The performance of our CFairER reaches peaks at \(n=40\) and \(n=30\) on Yelp and LastFM, respectively. After the peaks, we can witness a downgraded model performance by increasing the candidate size further. We consider the poorer performance of CFairER before reaching peaks is due to the limited candidate pool, i.e., insufficient attributes limit the exploration ability of CFairER to find appropriate candidates as fairness explanations. Meanwhile, a too-large candidate pool (e.g., \(n=60\) ) would offer more chances for the agent to select inadequate attributes as explanations. Based on the two findings, we believe it is necessary for our CFairER to carry the attentive action search, such as to select high-quality attributes as candidates based on their contributions to the current state.

For time complexity, our recommendation model (cf. Section 4.3) performs matrix factorization with a complexity of \(O(|\mathcal {O}|)\) . For the graph representation module (cf. Section 4.2), establishing node representations has complexity \(O(\sum _{l=1}^L (|\mathcal {G}|+|\mathcal {O}^{+}|) d_l d_{l-1})\) . For the off-policy learning process (cf. Section 5.1), the complexity is mainly determined by the attention score calculation, which has a time complexity of \(O(2T|\mathcal {O}^{+}| |\tilde{\mathcal {N}}_e| d^2)\) . The total time complexity is \(O(|\mathcal {O}|+ \sum _{l=1}^L(|\mathcal {G}|+|\mathcal {O}^{+}|) d_l d_{l-1}+2T|\mathcal {O}^{+}| n_2 d^2)\) . We evaluated the running time of FairKGAT and CEF baselines on the large-scale Yelp dataset. The corresponding results are 232s and 379s per epoch, respectively. CFairER has a comparable cost of 284s per epoch to these baselines. Considering that our CFairER achieves superior explainability improvements compared to the baselines, we believe that the increased cost of, at most, 52s per epoch is a reasonable trade-off.