research-article

FLOP: Federated Learning on Medical Datasets using Partial Networks

Authors:

Gregory P. Spell,

Lawrence CarinAuthors Info & Claims

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

Pages 3845 - 3853

https://doi.org/10.1145/3447548.3467185

Published: 14 August 2021 Publication History

Abstract

The outbreak of COVID-19 Disease due to the novel coronavirus has caused a shortage of medical resources. To aid and accelerate the diagnosis process, automatic diagnosis of COVID-19 via deep learning models has recently been explored by researchers across the world. While different data-driven deep learning models have been developed to mitigate the diagnosis of COVID-19, the data itself is still scarce due to patient privacy concerns. Federated Learning (FL) is a natural solution because it allows different organizations to cooperatively learn an effective deep learning model without sharing raw data. However, recent studies show that FL still lacks privacy protection and may cause data leakage. We investigate this challenging problem by proposing a simple yet effective algorithm, named Federated Learning on Medical Datasets using Partial Networks (FLOP), that shares only a partial model between the server and clients. Extensive experiments on benchmark data and real-world healthcare tasks show that our approach achieves comparable or better performance while reducing the privacy and security risks. Of particular interest, we conduct experiments on the COVID-19 dataset and find that our FLOP algorithm can allow different hospitals to collaboratively and effectively train a partially shared model without sharing local patients' data.

References

[1]

2020. CORONAVIRUS. "https://coronavirus.jhu.edu/map.html".

[2]

Ossama Abdel-Hamid, Abdel-rahman Mohamed, Hui Jiang, Li Deng, Gerald Penn, and Dong Yu. 2014. Convolutional neural networks for speech recognition. IEEE/ACM Transactions on audio, speech, and language processing, Vol. 22, 10 (2014), 1533--1545.

Digital Library

[3]

Yoshinori Aono, Takuya Hayashi, Lihua Wang, Shiho Moriai, et al. 2017. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics and Security, Vol. 13, 5 (2017), 1333--1345.

[4]

Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 1175--1191.

Digital Library

[5]

Theodora S Brisimi, Ruidi Chen, Theofanie Mela, Alex Olshevsky, Ioannis Ch Paschalidis, and Wei Shi. 2018. Federated learning of predictive models from federated electronic health records. International journal of medical informatics, Vol. 112 (2018), 59--67.

[6]

Fei Chen, Zhenhua Dong, Zhenguo Li, and Xiuqiang He. 2018. Federated meta-learning for recommendation. arXiv preprint arXiv:1802.07876 (2018).

[7]

Nanshan Chen, Min Zhou, Xuan Dong, Jieming Qu, Fengyun Gong, Yang Han, Yang Qiu, Jingli Wang, Ying Liu, Yuan Wei, et al. 2020. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. The Lancet, Vol. 395, 10223 (2020), 507--513.

[8]

Joseph Paul Cohen, Paul Morrison, and Lan Dao. 2020. COVID-19 image data collection. arXiv 2003.11597 (2020). https://github.com/ieee8023/covid-chestxray-dataset

[9]

Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. 2020. Inverting Gradients--How easy is it to break privacy in federated learning? arXiv preprint arXiv:2003.14053 (2020).

[10]

Robin C Geyer, Tassilo Klein, and Moin Nabi. 2017. Differentially private federated learning: A client level perspective. arXiv preprint arXiv:1712.07557 (2017).

[11]

Varun Gulshan, Lily Peng, Marc Coram, Martin C Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, et al. 2016. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama, Vol. 316, 22 (2016), 2402--2410.

[12]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[13]

Zhouyuan Huo, Bin Gu, Heng Huang, et al. 2018. Decoupled parallel backpropagation with convergence guarantee. In International Conference on Machine Learning. PMLR, 2098--2106.

[14]

Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2019. Advances and open problems in federated learning. arXiv preprint arXiv:1912.04977 (2019).

[15]

Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014).

[16]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[17]

Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009).

[18]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems. 1097--1105.

[19]

Rajesh Kumar, Abdullah Aman Khan, Sinmin Zhang, WenYong Wang, Yousif Abuidris, Waqas Amin, and Jay Kumar. 2020. Blockchain-federated-learning and deep learning models for covid-19 detection using ct imaging. arXiv preprint arXiv:2007.06537 (2020).

[20]

Qun Li, Xuhua Guan, Peng Wu, Xiaoye Wang, Lei Zhou, Yeqing Tong, Ruiqi Ren, Kathy SM Leung, Eric HY Lau, Jessica Y Wong, et al. 2020. Early transmission dynamics in Wuhan, China, of novel coronavirus--infected pneumonia. New England Journal of Medicine (2020).

[21]

Boyi Liu, Bingjie Yan, Yize Zhou, Yifan Yang, and Yixian Zhang. 2020. Experiments of federated learning for covid-19 chest x-ray images. arXiv preprint arXiv:2007.05592 (2020).

[22]

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Artificial Intelligence and Statistics. 1273--1282.

[23]

Dimitri Palaz, Mathew Magimai Doss, and Ronan Collobert. 2015. Convolutional neural networks-based continuous speech recognition using raw speech signal. In 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 4295--4299.

[24]

Konstantin Pogorelov, Kristin Ranheim Randel, Carsten Griwodz, Sigrun Losada Eskeland, Thomas de Lange, Dag Johansen, Concetto Spampinato, Duc-Tien Dang-Nguyen, Mathias Lux, Peter Thelin Schmidt, Michael Riegler, and Pål Halvorsen. 2017. KVASIR: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection. In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys'17). ACM, New York, NY, USA, 164--169. https://doi.org/10.1145/3083187.3083212

Digital Library

[25]

Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen. 2018. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4510--4520.

[26]

Reza Shokri and Vitaly Shmatikov. 2015. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security. 1310--1321.

[27]

Santiago Silva, Boris A Gutman, Eduardo Romero, Paul M Thompson, Andre Altmann, and Marco Lorenzi. 2019. Federated learning in distributed medical databases: Meta-analysis of large-scale subcortical brain data. In 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019). IEEE, 270--274.

[28]

Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar. 2017. Federated multi-task learning. In Advances in Neural Information Processing Systems. 4424--4434.

[29]

Abdul Waheed, Muskan Goyal, Deepak Gupta, Ashish Khanna, Fadi Al-Turjman, and Plácido Rogerio Pinheiro. 2020. Covidgan: Data augmentation using auxiliary classifier gan for improved covid-19 detection. IEEE Access, Vol. 8 (2020), 91916--91923.

[30]

Dawei Wang, Bo Hu, Chang Hu, Fangfang Zhu, Xing Liu, Jing Zhang, Binbin Wang, Hui Xiang, Zhenshun Cheng, Yong Xiong, et al. 2020 a. Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus--infected pneumonia in Wuhan, China. Jama, Vol. 323, 11 (2020), 1061--1069.

[31]

Linda Wang and Alexander Wong. 2020. COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images. arXiv preprint arXiv:2003.09871 (2020).

[32]

Wenlin Wang, Hongteng Xu, Zhe Gan, Bai Li, Guoyin Wang, Liqun Chen, Qian Yang, Wenqi Wang, and Lawrence Carin. 2020 b. Graph-driven generative models for heterogeneous multi-task learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 979--988.

[33]

Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017).

[34]

Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1492--1500.

[35]

Qian Yang, Zhouyuan Huo, Wenlin Wang, Heng Huang, and Lawrence Carin. 2019 a. Ouroboros: On Accelerating Training of Transformer-Based Language Models. arXiv preprint arXiv:1909.06695 (2019).

[36]

Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. 2019 b. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 10, 2 (2019), 1--19.

Digital Library

[37]

Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. In Advances in neural information processing systems. 649--657.

[38]

Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. iDLG: Improved Deep Leakage from Gradients. arXiv preprint arXiv:2001.02610 (2020).

[39]

Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated learning with non-iid data. arXiv preprint arXiv:1806.00582 (2018).

[40]

Chuansheng Zheng, Xianbo Deng, Qing Fu, Qiang Zhou, Jiapei Feng, Hui Ma, Wenyu Liu, and Xinggang Wang. 2020. Deep learning-based detection for COVID-19 from chest CT using weak label. medRxiv (2020).

[41]

Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. In Advances in Neural Information Processing Systems. 14747--14756.

Cited By

Choi GCha WLee SShin S(2024)Survey of Medical Applications of Federated LearningHealthcare Informatics Research10.4258/hir.2024.30.1.330:1(3-15)Online publication date: 31-Jan-2024
https://doi.org/10.4258/hir.2024.30.1.3
Chen HWang HLong QJin DLi Y(2024)Advancements in Federated Learning: Models, Methods, and PrivacyACM Computing Surveys10.1145/3664650Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1145/3664650
He XLiu SKeung JHe JChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Co-clustering for Federated Recommender SystemProceedings of the ACM on Web Conference 202410.1145/3589334.3645626(3821-3832)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645626
Show More Cited By

Index Terms

FLOP: Federated Learning on Medical Datasets using Partial Networks
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification

Recommendations

Federated Learning in a Medical Context: A Systematic Literature Review
Data privacy is a very important issue. Especially in fields like medicine, it is paramount to abide by the existing privacy regulations to preserve patients’ anonymity. However, data is required for research and training machine learning models that ...
Scaling-up medical vision-and-language representation learning with federated learning
Abstract
Medical Vision-and-Language Pre-training (MedVLP), which learns generic vision-language representations from medical images and texts to benefit various downstream medical tasks, has drawn remarkable attention in both artificial intelligence and ...
Highlights
- We propose the FedMedVLP to apply federated learning methods to the medical domain.
- FedMedVLP scales up medical vision-and-language pre-training with federated learning.
- We curate a well-balanced clinical dataset of COVID-19 to ...
Feddaw: Dual Adaptive Weighted Federated Learning for Non-IID Medical Data
Bioinformatics Research and Applications
Abstract
The use of deep learning methods in disease diagnosis holds great promise with the development of medical big data. However, the scale of parameters in deep learning models, which can often reach millions, requires learning from large and diverse ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '21: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining

August 2021

4259 pages

ISBN:9781450383325

DOI:10.1145/3447548

General Chairs:
Feida Zhu
Singapore Management University
,
Beng Chin Ooi
National University of Singapore
,
Chunyan Miao
Nanyang Technology University
,
Program Chairs:
Haixun Wang,
Iryna Skrypnyk,
Wynne Hsu,
Sanjay Chawla

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

KDD '21

Sponsor:

KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 14 - 18, 2021

Virtual Event, Singapore

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '24

Sponsor:
sigkdd
sigkdd

The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 25 - 29, 2024

Barcelona , Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
775
Total Downloads

Downloads (Last 12 months)151
Downloads (Last 6 weeks)15

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Choi GCha WLee SShin S(2024)Survey of Medical Applications of Federated LearningHealthcare Informatics Research10.4258/hir.2024.30.1.330:1(3-15)Online publication date: 31-Jan-2024
https://doi.org/10.4258/hir.2024.30.1.3
Chen HWang HLong QJin DLi Y(2024)Advancements in Federated Learning: Models, Methods, and PrivacyACM Computing Surveys10.1145/3664650Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1145/3664650
He XLiu SKeung JHe JChua TNgo CKa-Wei Lee RKumar RLauw H(2024)Co-clustering for Federated Recommender SystemProceedings of the ACM on Web Conference 202410.1145/3589334.3645626(3821-3832)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645626
Zhang YLi ZHan XDing SLi JWang JYing SShi J(2024)Pseudo-Data Based Self-Supervised Federated Learning for Classification of Histopathological ImagesIEEE Transactions on Medical Imaging10.1109/TMI.2023.332354043:3(902-915)Online publication date: Mar-2024
https://doi.org/10.1109/TMI.2023.3323540
Liao YMa LZhou BZhao XXie F(2024)DraftFed: A Draft-Based Personalized Federated Learning Approach for Heterogeneous Convolutional Neural NetworksIEEE Transactions on Mobile Computing10.1109/TMC.2023.328355723:5(3938-3949)Online publication date: May-2024
https://doi.org/10.1109/TMC.2023.3283557
Guo SWang HGeng X(2024)Dynamic heterogeneous federated learning with multi-level prototypesPattern Recognition10.1016/j.patcog.2024.110542153(110542)Online publication date: Sep-2024
https://doi.org/10.1016/j.patcog.2024.110542
Guan HYap PBozoki ALiu M(2024)Federated learning for medical image analysisPattern Recognition10.1016/j.patcog.2024.110424151:COnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110424
Zheng JChen YLai Q(2024)PPSFLFuture Generation Computer Systems10.1016/j.future.2024.03.020156:C(231-241)Online publication date: 18-Jul-2024
https://dl.acm.org/doi/10.1016/j.future.2024.03.020
Sharma SGuleria K(2024)A comprehensive review on federated learning based models for healthcare applicationsArtificial Intelligence in Medicine10.1016/j.artmed.2023.102691146:COnline publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.artmed.2023.102691
Saha SHota AChattopadhyay ANag ANandi S(2024)A multifaceted survey on privacy preservation of federated learning: progress, challenges, and opportunitiesArtificial Intelligence Review10.1007/s10462-024-10766-757:7Online publication date: 21-Jun-2024
https://doi.org/10.1007/s10462-024-10766-7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents