A Human-Centric Assessment of the Usefulness of Attribution Methods in Computer Vision

Abstract

Explainable AI has emerged to assure the validity of predictions by explaining produced decisions; nevertheless, it arrived with its own challenges of objectively evaluating explanations or assessing their usefulness to end users. Human-centric evaluation methods, such as simulatability, that are based on human subject studies have been shown to produce contradictory findings on the usefulness of explanations. It remains unclear if these contradictory results are caused by uncontrolled confounders: external factors that influence the measured quantity. To enable a reliable, human-centric, and trustworthy evaluation, we propose a generic assessment framework that allows researchers to measure the usefulness of XAI methods, such as attribution-based explanations, in an experimental setting with a reduced set of confounders. Applied simultaneously to multiple XAI techniques, the framework returns a usefulness ranking of the XAI models and also compares them with a human baseline. In a large-scale subject study, our results show that the acceptance rate increases from 64.2% without explanations to 86% with XAI methods and 92.7% when given human explanations. One particular model obtains indistinguishable results from human explanations, raising the acceptance score to 94.5%.

W. B. Rim and B. Kotnis—Work done while at the NEC Laboratories Europe.

Appendices

A Details of the Dataset Creation Process

Fig. 5.

From MNIST to Ishihara MNIST.

Table 3. Samples from the dataset used for the human experiments on Amazon Mechanical Turk. Original Ishihara images were only used alongside Ishihara-MNIST images in order to determine which participant has a color deficit and were not included in later tests. The AI explanations in this table are results from the Grad-CAM method.

The different steps to create Ishihara MNIST are depicted in Fig. 5. Our library uses IshiharaMC¹⁰ in the background. First, the images are converted into bi-level images. Second, we transform the bi-level images into Scalable Vector Graphics (SVG) representations that represent each bi-level image by polygons. Third, we apply Monte Carlo simulation to fill the whole area with circles. The simulation procedure begins with larger circles no greater than 6% of the unit circle in an attempt to fill in as much as possible with large circles; that is, minimizing the number of circles while maximizing the area they cover. Gaps are later filled by randomly placed smaller circles, no smaller than 1% of the unit circle. This reasonably reproduces the theme of the original Ishihara plates. Finally, we transform the circled image into the desired plate: a circle is assigned to the foreground or the background, depending on the measured intersection. To color the circles, we carefully inspect the coloring of the original plates. For example, plate 4 uses two foreground colors, (#7c8b62 ) and (#7c8b62 ), and three background colors, (#c7512b ), (#e57e52 ), and (#f69f78 ). We provide samples of the images used in our user study in Table 3.

B Data Compliance with GDPR

As for the conducted experiments, we adhered to high ethical standards for data collection, maintenance, and processing. Moreover, our human study follows the General Data Protection Regulation (GDPR)¹¹ imposed in the European Union for collecting and processing personal data. To achieve this compliance, we composed and signed a Data Processing Agreement (DPA) that exhaustively describes how we operate on the data as data collectors and processors; this document is vital for compliance auditing. Moreover, we asked the workers to acknowledge and accept a “Privacy Notice” that includes: (1) details of us as a data controller, (2) the applied data processing, (3) with whom we share the worker’s data, and (4) the worker’s rights to access, rectification or completion, erasure, restriction of processing of personal data, data portability, and withdrawal.

Finally, workers were paid fairly based on the completed tasks and not on performance. We paid each worker between $15 and $20 per hour depending on the task complexity (absolute payment of $1 to $2).