Explaining Cross-domain Recognition with Interpretable Deep Classifier
Authors: 
Yiheng Zhang, Ting Yao, Zhaofan Qiu, Tao MeiAuthors Info & Claims
Yiheng Zhang, Ting Yao, Zhaofan Qiu, Tao MeiAuthors Info & ClaimsArticle No.: 67, Pages 1 - 21
Abstract
The recent advances in deep learning predominantly construct models in their internal representations, and it is opaque to explain the rationale behind and decisions to human users. Such explainability is especially essential for domain adaptation, whose challenges require developing more adaptive models across different domains. In this article, we ask the question: How much does each sample in the source domain contribute to the network’s prediction on the samples from the target domain? To address this, we devise a novel Interpretable Deep Classifier (IDC) that learns the nearest source samples of a target sample as evidence upon which the classifier makes the decision. Technically, IDC maintains a differentiable memory bank for each category, and the memory slot derives a form of key–value pair. The key records the features of discriminative source samples, and the value stores the corresponding properties, e.g., representative scores of the features for describing the category. IDC computes the loss between the output of IDC and the labels of source samples to back-propagate to adjust the representative scores and update the memory banks. Extensive experiments on Office-Home and VisDA-2017 datasets demonstrate that our IDC leads to a more explainable model with almost no accuracy degradation and effectively calibrates classification for optimum reject options. More remarkably, when taking IDC as a prior interpreter, capitalizing on 0.1% source training data selected by IDC still yields superior results than that uses full training set on VisDA-2017 for unsupervised domain adaptation.
References
[1]
David Alvarez-Melis and Tommi S. Jaakkola. 2018. Towards robust interpretability with self-explaining neural networks. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS’18).
[2]
Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58 (2020), 82–115.
[3]
Sarah Adel Bargal, Andrea Zunino, Vitali Petsiuk, Jianming Zhang, Kate Saenko, Vittorio Murino, and Stan Sclaroff. 2018. Guided zoom: Questioning network evidence for fine-grained classification. In Proceedings of the British Machine Vision Conference (BMVC’18).
[4]
Qi Cai, Yingwei Pan, Chong-Wah Ngo, Xinmei Tian, Lingyu Duan, and Ting Yao. 2019. Exploring object relation in mean teacher for cross-domain detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11457–11466.
[5]
Qi Cai, Yingwei Pan, Ting Yao, Chenggang Clarence Yan, and Tao Mei. 2018. Memory matching networks for one-shot image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18).
[6]
Chunshui Cao, Xianming Liu, Yi Yang, Yinan Yu, Jiang Wang, Zilei Wang, Yongzhen Huang, Liang Wang, Chang Huang, Wei Xu, et al. 2015. Look and think twice: Capturing top-down visual attention with feedback convolutional neural networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’15).
[7]
Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K. Su. 2019. This looks like that: Deep learning for interpretable image recognition. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS’19).
[8]
Yang Chen, Yingwei Pan, Yu Wang, Ting Yao, Xinmei Tian, and Tao Mei. 2021. Transferrable contrastive learning for visual domain adaptation. In Proceedings of the ACM International Conference on Multimedia (ACM MM’21).
[9]
Yang Chen, Yu Wang, Yingwei Pan, Ting Yao, Xinmei Tian, and Tao Mei. 2021. A style and semantic memory mechanism for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9164–9173.
[10]
Mark G. Core, H. Chad Lane, Michael Van Lent, Dave Gomboc, Steve Solomon, and Milton Rosenberg. 2006. Building explainable artificial intelligence systems. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’06).
[11]
Mark W. Craven. 1996. Extracting Comprehensible Models from Trained Neural Networks. Technical Report. University of Wisconsin—Madison Department of Computer Sciences.
[12]
Shuhao Cui, Shuhui Wang, Junbao Zhuo, Chi Su, Qingming Huang, and Tian Qi. 2020. Gradually vanishing bridge for adversarial domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20).
[13]
Andrew Elliott, Stephen Law, and Chris Russell. 2021. Explaining classifiers using adversarial perturbations on the perceptual ball. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21).
[14]
Zunlei Feng, Zhenyun Yu, Yongcheng Jing, Sai Wu, Mingli Song, Yezhou Yang, and Junxiao Jiang. 2019. Interpretable partitioned embedding for intelligent multi-item fashion outfit composition. ACM Trans. Multimedia Comput. Commun. Appl. 15, 2s (2019), 1–20.
[15]
Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In Proceedings of the International Conference on Machine Learning (ICML’15).
[16]
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 1 (2016), 1–35.
[17]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’14).
[18]
Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. J. Mach. Learn. Res. 13, 1 (2012), 723–773.
[19]
David Gunning. 2017. Explainable artificial intelligence (xai). DARPA 2, 2 (2017).
[20]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16).
[21]
Yunzhong Hou and Liang Zheng. 2021. Visualizing adapted knowledge in domain transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21).
[22]
Łukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. 2017. Learning to remember rare events. In Proceedings of the International Conference on Learning Representations (ICLR’17).
[23]
Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G. Hauptmann. 2019. Contrastive adaptation network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19).
[24]
Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR’15).
[25]
Jingjing Li, Zhekai Du, Lei Zhu, Zhengming Ding, Ke Lu, and Heng Tao Shen. 2022. Divergence-agnostic unsupervised domain adaptation by adversarial attacks. IEEE Trans. Pattern Anal. Mach. Intell. 44, 11 (2022), 8196–8211.
[26]
Jingjing Li, Mengmeng Jing, Hongzu Su, Ke Lu, Lei Zhu, and Heng Tao Shen. 2022. Faster domain adaptation networks. IEEE Trans. Knowl. Data Eng. 34, 12 (2022), 5770–5783.
[27]
Liangzhi Li, Bowen Wang, Manisha Verma, Yuta Nakashima, Ryo Kawasaki, and Hajime Nagahara. 2021. SCOUTER: Slot attention-based classifier for explainable image recognition. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’21).
[28]
Oscar Li, Hao Liu, Chaofan Chen, and Cynthia Rudin. 2017. Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’17).
[29]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Proceedings of the European Conference on Computer Vision (ECCV’14).
[30]
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. 2015. Learning transferable features with deep adaptation networks. In Proceedings of the International Conference on Machine Learning (ICML’15).
[31]
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. 2018. Conditional adversarial domain adaptation. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS’18).
[32]
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. 2016. Unsupervised domain adaptation with residual transfer networks. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’16).
[33]
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. 2017. Deep transfer learning with joint adaptation networks. In Proceedings of the International Conference on Machine Learning (ICML’17).
[34]
Yuwu Lu, Desheng Li, Wenjing Wang, Zhihui Lai, Jie Zhou, and Xuelong Li. 2022. Discriminative invariant alignment for unsupervised domain adaptation. IEEE Trans. Multimedia 24 (2022), 1871–1882.
[35]
Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’16).
[36]
Jaemin Na, Heechul Jung, HyungJin Chang, and Wonjun Hwang. 2021. FixBi: Bridging domain spaces for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21).
[37]
Yingwei Pan, Zhaofan Qiu, Ting Yao, Houqiang Li, and Tao Mei. 2017. To create what you tell: Generating videos from captions. In Proceedings of the 25th ACM International Conference on Multimedia. 1789–1798.
[38]
Yingwei Pan, Ting Yao, Yehao Li, Chong-Wah Ngo, and Tao Mei. 2020. Exploring category-agnostic clusters for open-set domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20).
[39]
Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong-Wah Ngo, and Tao Mei. 2019. Transferrable prototypical networks for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19).
[40]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS’19).
[41]
Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. 2017. VisDA: The visual domain adaptation challenge. arXiv:1710.06924. Retrieved from https://arxiv.org/abs/1710.06924
[42]
Vitali Petsiuk, Abir Das, and Kate Saenko. 2018. RISE: Randomized input sampling for explanation of black-box models. In Proceedings of the British Machine Vision Conference (BMVC).
[43]
Alexander Pritzel, Benigno Uria, Sriram Srinivasan, Adria Puigdomenech Badia, Oriol Vinyals, Demis Hassabis, Daan Wierstra, and Charles Blundell. 2017. Neural episodic control. In Proceedings of the International Conference on Machine Learning (ICML’17).
[44]
Zhaofan Qiu, Yingwei Pan, Ting Yao, and Tao Mei. 2017. Deep semantic hashing with generative adversarial networks. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 225–234.
[45]
Esteban Real, Jonathon Shlens, Stefano Mazzocchi, Xin Pan, and Vincent Vanhoucke. 2017. Youtube-boundingboxes: A large high-precision human-annotated data set for object detection in video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17).
[46]
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting visual category models to new domains. In Proceedings of the European Conference on Computer Vision (ECCV’10).
[47]
Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18).
[48]
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2020. Grad-CAM: Visual explanations from deep networks via gradient-based localization. Int. J. Comput. Vis. 128, 2 (2020), 336–359.
[49]
Ramprasaath R. Selvaraju, Stefan Lee, Yilin Shen, Hongxia Jin, Shalini Ghosh, Larry Heck, Dhruv Batra, and Devi Parikh. 2019. Taking a hint: Leveraging explanations to make vision and language models more grounded. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’19).
[50]
Astuti Sharma, Tarun Kalluri, and Manmohan Chandraker. 2021. Instance level affinity-based transfer for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21).
[51]
Yu sheng Lin, Zhe-Yu Liu, Yu-An Chen, Yu-Siang Wang, Hsin-Ying Lee, Yi-Rong Chen, Ya-Liang Chang, and Winston H. Hsu. 2021. xCos: An explainable cosine metric for face verification task. ACM Trans. Multimedia Comput. Commun. Appl. 17, 3s (2021), 1–16.
[52]
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Proceedings of the International Conference on Learning Representations (ICLR’13) Workshop.
[53]
Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. 2015. Striving for simplicity: The all convolutional net. In Proceedings of the International Conference on Learning Representations (ICLR’15) Workshop.
[54]
Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. 2015. End-to-end memory networks. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’15).
[55]
W. R. Swartout. 1981. Producing Explanations and Justifications of Expert Consulting Programs. Technical Report.
[56]
William R. Swartout and Johanna D. Moore. 1993. Explanation in second generation expert systems. In Second Generation Expert Systems.
[57]
Sebastian Thrun. 1995. Extracting rules from artificial neural networks with distributed representations. In Proceedings of the Conference on Neural Information Processing Systems (NIPS’95).
[58]
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17).
[59]
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv:1412.3474. Retrieved from https://arxiv.org/abs/1412.3474
[60]
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. 2017. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17).
[61]
Rui Wang, Zuxuan Wu, Zejia Weng, Jingjing Chen, Guo-Jun Qi, and Yu-Gang Jiang. 2023. Cross-domain contrastive learning for unsupervised domain adaptation. IEEE Trans Multimedia 25 (2023), 1665–1673.
[62]
Yulong Wang, Hang Su, Bo Zhang, and Xiaolin Hu. 2020. Learning reliable visual saliency for model explanations. IEEE Trans. Multimedia 22, 7 (2020), 1796–1807.
[63]
Jason Weston, Sumit Chopra, and Antoine Bordes. 2014. Memory networks. In Proceedings of the International Conference on Learning Representations (ICLR’14).
[64]
L. Wu, He fei Ling, Yuxuan Shi, and Baiyan Zhang. 2022. Instance correlation graph for unsupervised domain adaptation. ACM Trans. Multimedia Comput. Commun. Appl. 18, 1s (2022), 1–23.
[65]
Bohui Xia, Xueting Wang, and T. Yamasaki. 2021. Semantic explanation for deep neural networks using feature interactions. ACM Trans. Multimedia Comput. Commun. Appl. 17, 3s (2021), 1–19.
[66]
Hongliang Yan, Zhetao Li, Qilong Wang, Peihua Li, Yong Xu, and Wangmeng Zuo. 2020. Weighted and class-specific maximum mean discrepancy for unsupervised domain adaptation. IEEE Trans. Multimedia 22, 9 (2020), 2420–2433.
[67]
Xin Yang, Xuemeng Song, Fuli Feng, Haokun Wen, Ling yu Duan, and Liqiang Nie. 2021. Attribute-wise explainable fashion compatibility modeling. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1 (2021), 1–21.
[68]
Ting Yao, Yingwei Pan, Chong-Wah Ngo, Houqiang Li, and Tao Mei. 2015. Semi-supervised domain adaptation with subspace learning for visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2142–2150.
[69]
Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In Proceedings of the European Conference on Computer Vision (ECCV’14).
[70]
Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Jonathan Brandt, Xiaohui Shen, and Stan Sclaroff. 2018. Top-down neural attention by excitation backprop. Int. J. Comput. Vis. 126, 10 (2018), 1084–1102.
[71]
Jingyi Zhang, Jiaxing Huang, and Shijian Lu. 2021. Spectral unsupervised domain adaptation for visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21).
[72]
Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael I. Jordan. 2019. Bridging theory and algorithm for domain adaptation. In Proceedings of the International Conference on Machine Learning (ICML’19).
[73]
Yiheng Zhang, Zhaofan Qiu, Ting Yao, Dong Liu, and Tao Mei. 2018. Fully convolutional adaptation networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6810–6818.
[74]
Yiheng Zhang, Zhaofan Qiu, Ting Yao, Chong-Wah Ngo, Dong Liu, and Tao Mei. 2020. Transferring and regularizing prediction for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9621–9630.
[75]
Linchao Zhu and Yi Yang. 2020. Inflated episodic memory with region self-attention for long-tailed visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20).
[76]
Andrea Zunino, Sarah Adel Bargal, Pietro Morerio, Jianming Zhang, Stan Sclaroff, and Vittorio Murino. 2021. Excitation dropout: Encouraging plasticity in deep neural networks. Int. J. Comput. Vis. 129, 4 (2021), 1139–1152.
[77]
Andrea Zunino, Sarah Adel Bargal, Riccardo Volpi, Mehrnoosh Sameki, Jianming Zhang, Stan Sclaroff, Vittorio Murino, and Kate Saenko. 2021. Explainable deep classification models for domain generalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21).
Index Terms
- Explaining Cross-domain Recognition with Interpretable Deep Classifier
Recommendations
Multi-modal Instance Refinement for Cross-Domain Action Recognition
Pattern Recognition and Computer VisionDomain consensual contrastive learning for few-shot universal domain adaptation
AbstractTraditional unsupervised domain adaptation (UDA) aims to transfer the learned knowledge from a fully labeled source domain to another unlabeled target domain on the same label set. The strong assumptions of full annotations on the source domain ...
Hierarchical Debiasing and Noisy Correction for Cross-domain Video Tube Retrieval
MM '24: Proceedings of the 32nd ACM International Conference on MultimediaVideo Tube Retrieval (VTR) has attracted wide attention in the multi-modal domain, aiming to accurately localize the spatial-temporal tube in videos based on the natural language description. Despite the remarkable progress, existing VTR models trained ...
Comments
Information & Contributors
Information
Published In
March 2024
665 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 23 October 2023
Online AM: 08 September 2023
Accepted: 27 July 2023
Revised: 03 April 2023
Received: 26 October 2022
Published in TOMM Volume 20, Issue 3
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 155Total Downloads
- Downloads (Last 12 months)118
- Downloads (Last 6 weeks)8
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in