research-article

Tackling the Accuracy-Interpretability Trade-off: Interpretable Deep Learning Models for Satellite Image-based Real Estate Appraisal

Authors:

Jan-Peter Kucklick,

Oliver MüllerAuthors Info & Claims

ACM Transactions on Management Information Systems, Volume 14, Issue 1

Article No.: 6, Pages 1 - 24

https://doi.org/10.1145/3567430

Published: 16 January 2023 Publication History

Abstract

Deep learning models fuel many modern decision support systems, because they typically provide high predictive performance. Among other domains, deep learning is used in real-estate appraisal, where it allows extending the analysis from hard facts only (e.g., size, age) to also consider more implicit information about the location or appearance of houses in the form of image data. However, one downside of deep learning models is their intransparent mechanic of decision making, which leads to a trade-off between accuracy and interpretability. This limits their applicability for tasks where a justification of the decision is necessary. Therefore, in this article, we first combine different perspectives on interpretability into a multi-dimensional framework for a socio-technical perspective on explainable artificial intelligence. Second, we measure the performance gains of using multi-view deep learning, which leverages additional image data (satellite images) for real estate appraisal. Third, we propose and test a novel post hoc explainability method called Grad-Ram. This modified version of Grad-Cam mitigates the intransparency of convolutional neural networks for predicting continuous outcome variables. With this, we try to reduce the accuracy-interpretability trade-off of multi-view deep learning models. Our proposed network architecture outperforms traditional hedonic regression models by 34% in terms of MAE. Furthermore, we find that the used satellite images are the second most important predictor after square feet in our model and that the network learns interpretable patterns about the neighborhood structure and density.

References

[1]

Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access 6 (2018), 52138–52160. DOI:

[2]

Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity checks for saliency maps. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS’18). Curran Associates, 9525–9536.

Digital Library

[3]

David Alvarez-Melis and Tommi S. Jaakkola. 2018. Towards robust interpretability with self-explaining neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS’18). Curran Associates, 7786–7795.

Digital Library

[4]

Naman Bansal, Chirag Agarwal, and Anh Nguyen. 2020. Sam: The sensitivity of attribution methods to hyperparameters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 8673–8683. DOI:

[5]

Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garcia, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inform. Fusion 58 (2020), 82–115. DOI:

Digital Library

[6]

A. J. Bency, S. Rallapalli, R. K. Ganti, M. Srivatsa, and B. S. Manjunath. 2017. Beyond spatial auto-regressive models: Predicting housing prices with satellite imagery. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV’17). IEEE Computer Society, 320–329. DOI:

[7]

Zachary Bessinger and Nathan Jacobs. 2016. Quantifying curb appeal. In Proceedings of the IEEE International Conference on Image Processing (ICIP’16). 4388–4392. DOI:

[8]

Leo Breiman. 2001. Random forests. Mach. Learn. 45, 1 (2001), 5–32.

Digital Library

[9]

City of Asheville. 2018. Real estate appraisal residential building 2018. Retrieved from http://data-avl.opendata.arcgis.com/datasets/bunco::real-estate-appraisal-residential-building-201.

[10]

Mengnan Du, Ninghao Liu, and Xia Hu. 2020. Techniques for interpretable machine learning. Commun. ACM 63, 1 (2020), 68–77. DOI:

Digital Library

[11]

Samaa Elnagar and Manoj A. Thomas. 2019. Real estate image-based appraisal using mask region based convolutional networks. In Proceedings of the 25th Americas Conference on Information Systems, Cancun, 2019. Association for Information Systems.

[12]

Jerome H. Friedman. 2001. Greedy function approximation: A gradient boosting machine. Ann. Stat. 29, 5 (2001), 1189–1232. Retrieved from http://www.jstor.org/stable/2699986.

[13]

Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2015. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. J. Comput. Graph. Stat. 24, 1 (2015), 44–65. DOI:

[14]

Google LLC. 2022. Get a static map. Retrieved from https://developers.google.com/maps/documentation/maps-static/star.

[15]

Shirley Gregor and Izak Benbasat. 1999. Explanations from intelligent systems: Theoretical foundations and implications for practice. MIS Quart. 23, 4 (1999), 497–530. Retrieved from http://www.jstor.org/stable/249487.

Digital Library

[16]

Shirley Gregor and Alan R. Hevner. 2013. Positioning and presenting design science research for maximum impact. MIS Quart. 37, 2 (2013), 337–355. Retrieved from http://www.jstor.org/stable/43825912.

Digital Library

[17]

Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM Comput. Surveys 51, 5 (2018), 1–42.

Digital Library

[18]

David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. 2019. XAI—Explainable artificial intelligence. Sci. Robot. 4, 37 (2019), eaay7120.

[19]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision. IEEE Computer Society, 1026–1034.

Digital Library

[20]

Marco Helbich, Andreas Jochem, Werner Mücke, and Bernhard Höfle. 2013. Boosting the predictive accuracy of urban hedonic house price models through airborne laser scanning. Comput. Environ. Urban Syst. 39 (2013), 81–92. DOI:

[21]

Neal Jean, Marshall Burke, Michael Xie, W. Matthew Davis, David B. Lobell, and Stefano Ermon. 2016. Combining satellite imagery and machine learning to predict poverty. Science 353, 6301 (2016), 790–794. DOI:

[22]

Margot E. Kaminski. 2019. The right to explanation, explained. Berkeley Tech. Law J. 34 (2019), 189.

[23]

Nils Kok, Eija-Leena Koponen, and Carmen Adriana Martínez-Barbosa. 2017. Big data in real estate? From manual appraisal to automated valuation. J. Portfolio Manage. 43, 6 (2017), 202–211.

[24]

Zona Kostic and Aleksandar Jevremovic. 2020. What image features boost housing market predictions? IEEE Trans. Multimedia 22, 7 (2020), 1904–1916. DOI:

[25]

Jan-Peter Kucklick, Jennifer Müller, Daniel Beverungen, and Oliver Müller. 2021. Quantifying the impact of location data in real estate appraisal—A gis-based deep learning approach. In Proceedings of the European Conference on Information Systems (ECIS’21). Association for Information Systems.

[26]

Jan-Peter Kucklick and Oliver Müller. 2021. A comparison of multi-view learning strategies for satellite image-based real estate appraisal. In Proceedings of the AAAI Workshop on Knowledge Discovery from Unstructured Data in Financial Services (AAAI’21). Association for the Advancement of Artifical Intelligence. Retrieved from https://aaai-kdf.github.io/kdf2021/assets/pdfs/KDF_21_paper_12.pdf.

[27]

Kelvin J. Lancaster. 1966. A new approach to consumer theory. J. Politic. Econ. 74, 2 (1966), 132–157. DOI:

[28]

Stephen Law and Mateo Neira. 2019. An unsupervised approach to geographical knowledge discovery using street level and street network images. In Proceedings of the 3rd ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery (GeoAI’19). Association for Computing Machinery, 56–65. DOI:

Digital Library

[29]

Stephen Law, Brooks Paige, and Chris Russell. 2019. Take a look around: Using street view and satellite images to estimate house prices. ACM Trans. Intell. Syst. Technol. 10, 5, Article 54 (Sept.2019), 19 pages. DOI:

Digital Library

[30]

Yingming Li, Ming Yang, and Zhongfei Zhang. 2018. A survey of multi-view representation learning. IEEE Trans. Knowl. Data Eng. 31, 10 (2018), 1863–1883. DOI:

[31]

Magdalena Ligus and Piotr Peternek. 2016. Measuring structural, location and environmental effects: A hedonic analysis of housing market in Wroclaw, Poland. Procedia-Soc. Behav. Sci. 220 (2016), 251–260. DOI:

[32]

Visit Limsombunchai. 2004. House price prediction: Hedonic price model vs. artificial neural network. In Proceedings of the New Zealand Agricultural and Resource Economics Society Conference. 25–26. DOI:

[33]

Zachary C. Lipton. 2018. The mythos of model interpretability. Queue 16, 3 (June2018), 31–57. DOI:

Digital Library

[34]

Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, 4765–4774. Retrieved from http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf.

[35]

Scott M. Lundberg, Bala Nair, Monica S. Vavilala, Mayumi Horibe, Michael J. Eisses, Trevor Adams, David E. Liston, Daniel King-Wai Low, Shu-Fang Newman, Jerry Kim, et al. 2018. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature Biomed. Eng. 2 (2018), 749–760. DOI:

[36]

David Martens and Foster Provost. 2014. Explaining data-driven document classifications. MIS Quarterly 38, 1 (2014), 73–100. https://www.jstor.org/stable/26554869.

Digital Library

[37]

William McCluskey, William Deddis, Adam Mannis, Dillon McBurney, and Richard Borst. 1997. Interactive application of computer assisted mass appraisal and geographic information systems. J. Prop. Valu. Invest. 15, 5 (1997), 448–465. DOI:

[38]

Microsoft.2019. Get a static map. Retrieved from https://docs.microsoft.com/en-us/bingmaps/rest-services/imagery/get-a-static-ma.

[39]

Christoph Molnar. 2019. Interpretable Machine Learning. Lulu Press, Research Triangle, NC.

[40]

Christof Naumzik and Stefan Feuerriegel. 2020. One picture is worth a thousand words? The pricing power of images in e-Commerce. In Proceedings of the World Wide Web Conference 2020 (WWW’20). Association for Computing Machinery, New York, NY, USA, 3119–3125. DOI:

Digital Library

[41]

Norzailawati Mohd Noor, M. Zainora Asmawi, and Alias Abdullah. 2015. Sustainable urban regeneration: GIS and hedonic pricing method in determining the value of green space in housing area. Procedia-Soc. Behav. Sci. 170 (2015), 669–679. DOI:

[42]

Omid Poursaeed, Tomáš Matera, and Serge Belongie. 2018. Vision-based real estate price estimation. Mach. Vision Appl. 29 (2018), 667–676. DOI:

Digital Library

[43]

Gang Qu, Li Xiao, Wenxing Hu, Junqi Wang, Kun Zhang, Vince D. Calhoun, and Yu-Ping Wang. 2021. Ensemble manifold regularized multi-modal graph convolutional network for cognitive ability prediction. IEEE Trans. Biomed. Eng. 68, 12 (2021), 3564–3573. DOI:

[44]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’16). Association for Computing Machinery, 1135–1144. DOI:

Digital Library

[45]

Sherwin Rosen. 1974. Hedonic prices and implicit markets: Product differentiation in pure competition. J. Politic. Econ. 82, 1 (1974), 34–55. DOI:

[46]

Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Mach. Intell. 1 (2019), 206–215. DOI:

[47]

Wojciech Samek, Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, and Klaus-Robert Müller. 2016. Evaluating the visualization of what a deep neural network has learned. IEEE Trans. Neural Netw. Learn. Syst. 28, 11 (2016), 2660–2673. DOI:

[48]

Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). IEEE Computer Society, 618–626. DOI:

[49]

Evan Sheehan, Chenlin Meng, Matthew Tan, Burak Uzkent, Neal Jean, Marshall Burke, David Lobell, and Stefano Ermon. 2019. Predicting economic development using geolocated Wikipedia articles. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’19). Association for Computing Machinery, 2698–2706. DOI:

Digital Library

[50]

Masaki Shimomura, Makoto Nakaune, Toru Yamada, Kazunori Tsukazawa, and Kazuyuki Nakamura. 2020. Crop yields prediction with CNN and visual explanation with Grad-RAM. In IEICE Conferences Archives. The Institute of Electronics, Information and Communication Engineers.

[51]

Kirill Solovev and Nicolas Pröllochs. 2021. Integrating floor plans into hedonic models for rent price appraisal. In Proceedings of the World Wide Web Conference (WWW’21). Association for Computing Machinery, 2838–2847. DOI:

Digital Library

[52]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE Computer Society, 2818–2826. DOI:

[53]

Richard Tomsett, Dan Harborne, Supriyo Chakraborty, Prudhvi Gurram, and Alun Preece. 2020. Sanity checks for saliency metrics. In Proceedings of the AAAI Conference on Artificial Intelligence. 6021–6029. DOI:

[54]

Zhiguang Wang and Jianbo Yang. 2018. Diabetic retinopathy detection via deep convolutional networks for discriminative localization and visual explanation. In Proceedings of the Workshops at the 32nd AAAI Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence, 514–521.

[55]

Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. CBAM: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV’18), Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (Eds.). Springer International Publishing, Cham, 3–19.

Digital Library

[56]

Chang Xu, Dacheng Tao, and Chao Xu. 2013. A survey on multi-view learning. Retrieved from https://arXiv:1304.5634, DOI:

[57]

Richard R. Yang, Steven Chen, and Edward Chou. 2018. AI blue book: Vehicle price prediction using visual features. Retrieved from https://arXiv:1803.11227, DOI:

[58]

Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). IEEE Computer Society, 2921–2929. DOI:

Cited By

Gnat S(2024)Determining the Influence of Real Estate Features on Prices with Partial Dependence Plots: A Case Study in Szczecin, PolandReal Estate Management and Valuation10.2478/remav-2024-0039Online publication date: 16-Oct-2024
https://doi.org/10.2478/remav-2024-0039
Numan JYusoff I(2024)Identifying the Current Status of Real Estate Appraisal MethodsReal Estate Management and Valuation10.2478/remav-2024-0032Online publication date: 7-Jun-2024
https://doi.org/10.2478/remav-2024-0032
Lamprou VKallipolitis AMaglogiannis I(2024)On the evaluation of deep learning interpretability methods for medical images under the scope of faithfulnessComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2024.108238253(108238)Online publication date: Aug-2024
https://doi.org/10.1016/j.cmpb.2024.108238
Show More Cited By

Index Terms

Tackling the Accuracy-Interpretability Trade-off: Interpretable Deep Learning Models for Satellite Image-based Real Estate Appraisal

Recommendations

Learning Optimal Fair Decision Trees: Trade-offs Between Interpretability, Fairness, and Accuracy
AIES '23: Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society

The increasing use of machine learning in high-stakes domains – where people’s livelihoods are impacted – creates an urgent need for interpretable, fair, and highly accurate algorithms. With these needs in mind, we propose a mixed integer optimization (...
Multi-objective based Fuzzy Rule Based Systems (FRBSs) for trade-off improvement in accuracy and interpretability

Fuzzy rule-based systems (FRBSs) are a common alternative for applying fuzzy logic in different areas and real-world problems. The schemes and algorithms used to generate these types of systems imply that their performance can be analyzed from different ...
LW-Net: an interpretable network with smart lifting wavelet kernel for mechanical feature extraction and fault diagnosis
Abstract
Deep learning has been applied in mechanical fault diagnosis. Hereinto, the convolutional neural network (CNN) has the shallow convolution operation, supporting the function of feature learning. However, the interpretability of CNN has always been ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Management Information Systems

ACM Transactions on Management Information Systems Volume 14, Issue 1

March 2023

270 pages

ISSN:2158-656X

EISSN:2158-6578

DOI:10.1145/3580447

Editor:
Daniel Zeng
Chinese Academy of Sciences, China

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 January 2023

Online AM: 10 October 2022

Accepted: 22 August 2022

Revised: 29 June 2022

Received: 16 April 2021

Published in TMIS Volume 14, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
696
Total Downloads

Downloads (Last 12 months)297
Downloads (Last 6 weeks)27

Reflects downloads up to 13 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gnat S(2024)Determining the Influence of Real Estate Features on Prices with Partial Dependence Plots: A Case Study in Szczecin, PolandReal Estate Management and Valuation10.2478/remav-2024-0039Online publication date: 16-Oct-2024
https://doi.org/10.2478/remav-2024-0039
Numan JYusoff I(2024)Identifying the Current Status of Real Estate Appraisal MethodsReal Estate Management and Valuation10.2478/remav-2024-0032Online publication date: 7-Jun-2024
https://doi.org/10.2478/remav-2024-0032
Lamprou VKallipolitis AMaglogiannis I(2024)On the evaluation of deep learning interpretability methods for medical images under the scope of faithfulnessComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2024.108238253(108238)Online publication date: Aug-2024
https://doi.org/10.1016/j.cmpb.2024.108238
Fathi MVakili KHajibeygi RBahrami ABehzad STafazolimoghadam AAghabozorgi HEshraghi RBhatt VGholamrezanezhad A(2024)Cultivating diagnostic clarity: The importance of reporting artificial intelligence confidence levels in radiologic diagnosesClinical Imaging10.1016/j.clinimag.2024.110356(110356)Online publication date: Nov-2024
https://doi.org/10.1016/j.clinimag.2024.110356
Li XXu YLaw RWang S(2024)Enhancing tourism demand forecasting with a transformer-based frameworkAnnals of Tourism Research10.1016/j.annals.2024.103791107(103791)Online publication date: Jul-2024
https://doi.org/10.1016/j.annals.2024.103791
Lu ZAfridi IKang HRuchkin IZheng X(2024)Surveying neuro-symbolic approaches for reliable artificial intelligence of thingsJournal of Reliable Intelligent Environments10.1007/s40860-024-00231-110:3(257-279)Online publication date: 26-Jul-2024
https://doi.org/10.1007/s40860-024-00231-1
Nahar JYun C(2023)A Data-Driven Approach to Predicting Melbourne Housing Prices Using Advanced Machine Learning Models2023 International Conference on Sustainable Technology and Engineering (i-COSTE)10.1109/i-COSTE60462.2023.10500769(1-7)Online publication date: 4-Dec-2023
https://doi.org/10.1109/i-COSTE60462.2023.10500769
Trier MKundisch DBeverungen DMüller OSchryen GMirbabaie MTrang S(2023)Digital ResponsibilityBusiness & Information Systems Engineering10.1007/s12599-023-00822-x65:4(463-474)Online publication date: 5-Jul-2023
https://doi.org/10.1007/s12599-023-00822-x
Banipal IAsthana SMazumder S(2023)Sustainable AI - Standards, Current Practices and RecommendationsProceedings of the Future Technologies Conference (FTC) 2023, Volume 110.1007/978-3-031-47454-5_21(271-289)Online publication date: 2-Nov-2023
https://doi.org/10.1007/978-3-031-47454-5_21

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents