research-article

Effective De-identification Generative Adversarial Network for Face Anonymization

Authors:

Zhenzhong Kuang,

Noboru BabaguchiAuthors Info & Claims

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Pages 3182 - 3191

https://doi.org/10.1145/3474085.3475464

Published: 17 October 2021 Publication History

Abstract

The growing application of face images and modern AI technology has raised another important concern in privacy protection. In many real scenarios like scientific research, social sharing and commercial application, lots of images are released without privacy processing to protect people's identity. In this paper, we develop a novel effective de-identification generative adversarial network (DeIdGAN) for face anonymization by seamlessly replacing a given face image with a different synthesized yet realistic one. Our approach consists of two steps. First, we anonymize the input face to obfuscate its original identity. Then, we use our designed de-identification generator to synthesize an anonymized face. During the training process, we leverage a pair of identity-adversarial discriminators to explicitly constrain identity protection by pushing the synthesized face away from the predefined sensitive faces to resist re-identification and identity invasion. Finally, we validate the effectiveness of our approach on public datasets. Compared with existing methods, our approach can not only achieve better identity protection rates but also preserve superior image quality and data reusability, which suggests the state-of-the-art performance.

References

[1]

Shruti Agarwal, Hany Farid, Yuming Gu, Mingming He, Koki Nagano, and Hao Li. 2019. Protecting world leaders against deep fakes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 38--45.

[2]

Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. 2018. Towards open-set identity preserving face synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 6713--6722.

[3]

Sahar Bhaimia. 2018. The general data protection regulation: the next generation of EU data protection. Legal Information Management, Vol. 18, 1 (2018), 21--28.

[4]

Ali Breland. 2019. The Bizarre and Terrifying Case of the Video that Helped Bring an African Nation to the Brink. Mother Jones (2019).

[5]

Karla Brkic, Ivan Sikiric, Tomislav Hrkac, and Zoran Kalafatic. 2017. I Know That Person: generative full body and face de-identification of people in images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 1319--1328.

[6]

Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. 2018. VGGFace2: a dataset for recognising faces across pose and age. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 67--74.

[7]

Bobby Chesney and Danielle Citron. 2019. Deep fakes: A looming challenge for privacy, democracy, and national security. California Law Review, Vol. 107 (2019), 1753.

[8]

Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. 2020. StarGAN v2: diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8188--8197.

[9]

Matheus Alves Diniz and William Robson Schwartz. 2020. Face attributes as cues for deep face recognition understanding. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 307--313.

[10]

Liang Du, Meng Yi, Erik Blasch, and Haibin Ling. 2014. GARP-face: balancing privacy protection and utility preservation in face de-identification. In Proceedings of the IEEE International Joint Conference on Biometrics. IEEE, 1--8.

[11]

George T Duncan and Robert W Pearson. 1991. Enhancing access to microdata while protecting confidentiality: prospects for the future. Statist. Sci., Vol. 6, 3 (1991), 219--232.

[12]

Oran Gafni, Lior Wolf, and Yaniv Taigman. 2019. Live face de-identification in video. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, 9378--9387.

[13]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM, Vol. 63, 11 (2020), 139--144.

Digital Library

[14]

Ralph Gross, Edoardo Airoldi, Bradley Malin, and Latanya Sweeney. 2005. Integrating utility into face de-identification. In Proceedings of the International Workshop on Privacy Enhancing Technologies. Springer, 227--242.

Digital Library

[15]

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 30. 1--12.

Digital Library

[16]

Nancy A. Van House. 2009. Collocated photo sharing, story-telling, and the performance of self. International Journal of Human-Computer Studies, Vol. 67, 12 (2009), 1073--1086.

Digital Library

[17]

Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. 2008. Labeled faces in the wild: a database forstudying face recognition in unconstrained environments. In Workshop on Faces in 'Real-Life' Images: Detection, Alignment, and Recognition. Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, 1--14.

[18]

Xun Huang and Serge Belongie. 2017. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, 1501--1510.

[19]

Yuge Huang, Yuhan Wang, Ying Tai, Xiaoming Liu, Pengcheng Shen, Shaoxin Li, and Feiyue Huang Jilin Li. 2020. CurricularFace: adaptive curriculum learning loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 1--8.

[20]

Håkon Hukkelås, Rudolf Mester, and Frank Lindseth. 2019. Deepprivacy: a generative adversarial network for face anonymization. In Proceedings of the International Symposium on Visual Computing. Springer, 565--578.

[21]

Phillip Isola, Junyan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-Image translation with conditional adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 5967--5976.

[22]

Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 4401--4410.

[23]

Diederik P Kingma and Max Welling. 2014. Auto-encoding variational bayes. In arXiv preprint arXiv:1312.6114v10.

[24]

Zhenzhong Kuang, Zhiqiang Guo, Jinglong Fang, Jun Yu, Noboru Babaguchi, and Jianping Fan. 2021. Unnoticeable synthetic face replacement for image privacy protection. Neurocomputing, Vol. 457 (2021), 322--333.

Digital Library

[25]

Zhenzhong Kuang, Zongmin Li, Dan Lin, and Jianping Fan. 2017. Automatic privacy prediction to accelerate social image sharing. In Proceedings of the IEEE Third International Conference on Multimedia Big Data. IEEE, 197--200.

[26]

Zhenzhong Kuang, Jun Yu, Zongmin Li, Baopeng Zhang, and Jianping Fan. 2018. Integrating multi-level deep learning and concept ontology for large-scale visual recognition. Pattern Recognition, Vol. 78 (2018), 198--214.

Digital Library

[27]

Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. 2020. MaskGAN: towards diverse and interactive facial image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 5549--5558.

[28]

Geoffrey Letournel, Aurélie Bugeau, V-T Ta, and J-P Domenger. 2015. Face de-identification with expressions preservation. In Proceedings of the IEEE International Conference on Image Processing. IEEE, 4366--4370.

Digital Library

[29]

Tao Li and Lei Lin. 2019. AnonymousNet: natural face de-identification with measurable privacy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 56--65.

[30]

Yifang Li, Nishant Vishwamitra, Bart Knijnenburg, Hongxin Hu, and Kelly Caine. 2017. Blur vs. Block: investigating the effectiveness of privacy-enhancing obfuscation for images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. IEEE, 1343--1351.

[31]

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, 3730--3738.

Digital Library

[32]

Maxim Maximov, Ismail Elezi, and Laura Leal-Taixé. 2020. CIAGAN: conditional identity anonymization generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 5447--5456.

[33]

Richard McPherson, Reza Shokri, and Vitaly Shmatikov. 2016. Defeating image obfuscation with deep learning. arXiv preprint arXiv: 1609.00408 (2016).

[34]

Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784v1 (2014).

[35]

E M Newton, L Sweeney, and Bradley Malin. 2005. Preserving privacy by de-identifying face images. IEEE Transactions on Knowledge and Data Engineering, Vol. 17, 2 (2005), 232--243.

Digital Library

[36]

Seong Joon Oh, Rodrigo Benenson, Mario Fritz, and Bernt Schiele. 2016. Faceless Person Recognition: privacy implications in social media. In Proceedings of the European Conference on Computer Vision. Springer, 19--35.

[37]

Tribhuvanesh Orekondy, Mario Fritz, and Bernt Schiele. 2018. Connecting pixels to privacy and utility: automatic redaction of private information in images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 8466--8475.

[38]

Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu. 2019. Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2337--2346.

[39]

Zhongzheng Ren, Yong Jae Lee, and Michael S Ryoo. 2018. Learning to anonymize faces for privacy preserving action detection. In Proceedings of the European Conference on Computer Vision. Springer, 620--636.

[40]

Slobodan Ribaric, Aladdin Ariyaeeinia, and Nikola Pavesic. 2016. De-identification for privacy protection in multimedia content: A survey. Signal Processing: Image Communication, Vol. 47 (2016), 131--151.

Digital Library

[41]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: a unified embedding for face recognition and clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 815--823.

[42]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[43]

Eleftherios Spyromitros-Xioufis, Symeon Papadopoulos, Adrian Popescu, and Yiannis Kompatsiaris. 2016. Personalized privacy-aware image classification. In Proceedings of the International Conference on Multimedia Retrieval. ACM, 71--78.

Digital Library

[44]

Anna Cinzia Squicciarini, Dan Lin, Smitha Sundareswaran, and Joshua Wede. 2015. Privacy policy inference of user-uploaded images on content sharing sites. IEEE Transactions on Knowledge and Data Engineering, Vol. 27, 1 (2015), 193--206.

[45]

Qianru Sun, Liqian Ma, Seong Joon Oh, Luc Van Gool, Bernt Schiele, and Mario Fritz. 2018. Natural and effective obfuscation by head inpainting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 5050--5059.

[46]

Zongji Sun, Li Meng, and Aladdin Ariyaeeinia. 2015. Distinguishable de-identified faces. In Proceedings of the IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Vol. 4. IEEE, 1--6.

[47]

Latanya Sweeney. 1997. Guaranteeing anonymity when sharing medical data, the datafly system. In Proceedings of the AMIA Annual Fall Symposium. American Medical Informatics Association, 51.

[48]

Latanya Sweeney. 2002. k-anonymity: a model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, Vol. 10, 5 (2002), 557--570.

Digital Library

[49]

Fariborz Taherkhani, Nasser M Nasrabadi, and Jeremy Dawson. 2018. A deep face identification network enhanced by facial attributes prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 553--560.

[50]

Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, and Javier Ortega-Garcia. 2020. Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, Vol. 64, 131--148.

[51]

Yifan Wu, Fan Yang, and Haibin Ling. 2019. Privacy-Protective-GAN for face de-identification. Journal of Computer Science and Technology, Vol. 34, 1 (2019), 47--60.

[52]

Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, and Nong Sang. 2018. BiSeNet: bilateral segmentation network for real-time semantic segmentation. In Proceedings of the European Conference on Computer Vision. Springer, 8--14.

[53]

Jun Yu, Baopeng Zhang, Zhengzhong Kuang, Lin Dan, and Jianping Fan. 2017. iPrivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Transactions on Information Forensics and Security, Vol. 12, 5 (2017), 1005--1016.

Digital Library

[54]

Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, Vol. 23, 10 (2016), 1499--1503.

[55]

Bingquan Zhu, Hao Fang, Yanan Sui, and Luming Li. 2020. Deepfakes for medical video de-identification: privacy protection and diagnostic information preservation. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. ACM, 414--420.

Digital Library

Cited By

Natarajan GElango EHanees ABai S(2024)Enhancing Privacy and Security in Online Education Using Generative Adversarial NetworksEnhancing Security in Public Spaces Through Generative Adversarial Networks (GANs)10.4018/979-8-3693-3597-0.ch015(206-230)Online publication date: 16-May-2024
https://doi.org/10.4018/979-8-3693-3597-0.ch015
Moore JCelik YStuart SMcMeekin PWalker RHetherington VGodfrey A(2024)Using Video Technology and AI within Parkinson’s Disease Free-Living Fall Risk AssessmentSensors10.3390/s2415491424:15(4914)Online publication date: 29-Jul-2024
https://doi.org/10.3390/s24154914
Du LJia JZhang XLan G(2024)PrivateGaze: Preserving User Privacy in Black-box Mobile Gaze Tracking ServicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785958:3(1-28)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678595
Show More Cited By

Index Terms

Effective De-identification Generative Adversarial Network for Face Anonymization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Security and privacy
  1. Human and societal aspects of security and privacy
    1. Privacy protections

Recommendations

A novel anonymization algorithm: Privacy protection and knowledge preservation

In data mining and knowledge discovery, there are two conflicting goals: privacy protection and knowledge preservation. On the one hand, we anonymize data to protect privacy; on the other hand, we allow miners to discover useful knowledge from ...
Privacy consensus in anonymization systems via game theory
DBSec'12: Proceedings of the 26th Annual IFIP WG 11.3 conference on Data and Applications Security and Privacy

Privacy protection appears as a fundamental concern when personal data is collected, stored, and published. Several anonymization methods have been proposed to address privacy issues in private datasets. Every anonymization method has at least one ...
Handicapping attacker's confidence: an alternative to k-anonymization

We present an approach of limiting the confidence of inferring sensitive properties to protect against the threats caused by data mining abilities. The problem has dual goals: preserve the information for a wanted data analysis request and limit the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

October 2021

5796 pages

ISBN:9781450386517

DOI:10.1145/3474085

General Chairs:
Heng Tao Shen
University of Electronic Science&Technology of China, China
,
Yueting Zhuang
Zhejiang University, China
,
John R. Smith
IBM, USA
,
Program Chairs:
Yang Yang
University of Electronic Science and Technology of China, China
,
Pablo Cesar
CWI&TU Delft, The Netherlands
,
Florian Metze
FACEBOOK, Inc., USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National key R&D Program of China
National Natural Science Foundation of China

Conference

MM '21

Sponsor:

SIGMM

MM '21: ACM Multimedia Conference

October 20 - 24, 2021

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
732
Total Downloads

Downloads (Last 12 months)219
Downloads (Last 6 weeks)18

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Natarajan GElango EHanees ABai S(2024)Enhancing Privacy and Security in Online Education Using Generative Adversarial NetworksEnhancing Security in Public Spaces Through Generative Adversarial Networks (GANs)10.4018/979-8-3693-3597-0.ch015(206-230)Online publication date: 16-May-2024
https://doi.org/10.4018/979-8-3693-3597-0.ch015
Moore JCelik YStuart SMcMeekin PWalker RHetherington VGodfrey A(2024)Using Video Technology and AI within Parkinson’s Disease Free-Living Fall Risk AssessmentSensors10.3390/s2415491424:15(4914)Online publication date: 29-Jul-2024
https://doi.org/10.3390/s24154914
Du LJia JZhang XLan G(2024)PrivateGaze: Preserving User Privacy in Black-box Mobile Gaze Tracking ServicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785958:3(1-28)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678595
Hellmann FMertes SBenouis MHustinx AHsieh TConati CKrawitz PAndré E(2024)GANonymization: A GAN-Based Face Anonymization Framework for Preserving Emotional ExpressionsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364110721:1(1-27)Online publication date: 17-Jan-2024
https://dl.acm.org/doi/10.1145/3641107
Yuan LChen WPu XZhang YLi HZhang YGao XEbrahimi T(2024)PRO-Face C: Privacy-Preserving Recognition of Obfuscated Face via Feature CompensationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.338897619(4930-4944)Online publication date: 2024
https://doi.org/10.1109/TIFS.2024.3388976
Ye MShen WZhang JYang YDu B(2024)SecureReID: Privacy-Preserving Anonymization for Person Re-IdentificationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.335623319(2840-2853)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3356233
Kuang ZYang XShen YHu CYu J(2024)Facial Identity Anonymization via Intrinsic and Extrinsic Attention Distraction2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01179(12406-12415)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01179
Thakral KPrasad SAswani SVatsa MSingh R(2024)ToonerGAN: Reinforcing GANs for Obfuscating Automated Facial Indexing2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01034(10875-10884)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01034
Si TLi PYang XFan LHe F(2024)Model-aware privacy-preserving with start trigger method for person re-identificationInformation Processing & Management10.1016/j.ipm.2024.10381961:5(103819)Online publication date: Oct-2024
https://doi.org/10.1016/j.ipm.2024.103819
Wilson EShic FJörg SJain E(2024)Towards mitigating uncann(eye)ness in face swaps via gaze-centric loss termsComputers and Graphics10.1016/j.cag.2024.103888119:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.cag.2024.103888
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents