research-article

Public Access

Can I only share my eyes? A Web Crowdsourcing based Face Partition Approach Towards Privacy-Aware Face Recognition

Authors:

Dong WangAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 3611 - 3622

https://doi.org/10.1145/3485447.3512256

Published: 25 April 2022 Publication History

All formats PDF

Abstract

Human face images represent a rich set of visual information for online social media platforms to optimize the machine learning (ML)/AI models in their data-driven facial applications (e.g., face detection, face recognition). However, there exists a growing privacy concern from social media users to share their online face images that will be annotated by unknown crowd workers and analyzed by ML/AI researchers in the model training and optimization process. In this paper, we focus on a privacy-aware face recognition problem where the goal is to empower the facial applications to train their face recognition models with images shared by social media users while protecting the identity of the users. Our problem is motivated by the limitation of current privacy-aware face recognition approaches that mainly prevent algorithmic attacks by manipulating face images but largely ignore the potential privacy leakage related to human activities (e.g., crowdsourcing annotation). To address such limitations, we develop FaceCrowd, a web crowdsourcing based face partition approach to improve the performance of current face recognition models by designing a novel crowdsourced partial face graph generated from privacy-preserved social media face images. We evaluate the performance of FaceCrowd using two real-world human face datasets that consist of large-scale human face images. The results show that FaceCrowd not only improves the accuracy of the face recognition models but also effectively protects the identity information of the social media users who share their face images.

References

[1]

Maribel Acosta, Amrapali Zaveri, Elena Simperl, Dimitris Kontokostas, Sören Auer, and Jens Lehmann. 2013. Crowdsourcing linked data quality assessment. In International semantic web conference. Springer, 260–276.

[2]

Jocelyn C Adams, Kristen C Allen, Timothy Miller, Nathan D Kalka, and Anil K Jain. 2016. Grouper: Optimizing crowdsourced face annotations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 95–102.

[3]

Usman Ahmad, Anees Baqir, Faizan ul Mustafa, Sayyam Malik, Sana Ahmad Sani, and Syed Zeeshan Hussain Shah. 2019. Enhancing the Authentication Mechanism of Social Media Websites using Face Detection. In 2019 4th International Conference on Emerging Trends in Engineering, Sciences and Technology (ICEEST). IEEE, 1–5.

[4]

Amazon. 2020. Amazon Mechanic Turks. https://www.mturk.com/pricing

[5]

Amna Basharat, I Budak Arpinar, Shima Dastgheib, Ugur Kursuncu, Krys Kochut, and Erdogan Dogdu. 2014. Crowdlink: Crowdsourcing for large-scale linked data management. In 2014 IEEE International Conference on Semantic Computing. IEEE, 227–234.

Digital Library

[6]

Fabrício Benevenuto, Tiago Rodrigues, Meeyoung Cha, and Virgílio Almeida. 2009. Characterizing user behavior in online social networks. In Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement. 49–62.

Digital Library

[7]

Bettina Berendt. 2019. Privacy beyond confidentiality, data science beyond spying: From movement data and data privacy towards a wider fundamental rights discourse. In Annual Privacy Forum. Springer, 59–71.

[8]

Xavier P Burgos-Artizzu, Pietro Perona, and Piotr Dollár. 2013. Robust face landmark estimation under occlusion. In Proceedings of the IEEE international conference on computer vision. 1513–1520.

Digital Library

[9]

Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. 2018. Vggface2: A dataset for recognising faces across pose and age. In 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018). IEEE, 67–74.

Digital Library

[10]

Jae Young Choi, Wesley De Neve, Konstantinos N Plataniotis, and Yong Man Ro. 2010. Collaborative face recognition for improved face annotation in personal photo collections shared on online social networks. IEEE Transactions on Multimedia 13, 1 (2010), 14–28.

Digital Library

[11]

Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2018. Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8789–8797.

[12]

Naser Damer, Fadi Boutros, Florian Kirchbuchner, and Arjan Kuijper. 2019. D-id-net: Two-stage domain and identity learning for identity-preserving image generation from semantic segmentation. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). IEEE, 3677–3682.

[13]

George Danezis, Josep Domingo-Ferrer, Marit Hansen, Jaap-Henk Hoepman, Daniel Le Metayer, Rodica Tirtea, and Stefan Schiffner. 2015. Privacy and data protection by design-from policy to engineering. arXiv preprint arXiv:1501.03726(2015).

[14]

Munmun De Choudhury, Scott Counts, and Eric Horvitz. 2013. Social media as a measurement tool of depression in populations. In Proceedings of the 5th annual ACM web science conference. 47–56.

Digital Library

[15]

Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4690–4699.

[16]

Tim Draws, Alisa Rieger, Oana Inel, Ujwal Gadiraju, and Nava Tintarev. 2021. A Checklist to Combat Cognitive Biases in Crowdsourcing. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 9. 48–59.

[17]

Liang Du, Meng Yi, Erik Blasch, and Haibin Ling. 2014. GARP-face: Balancing privacy protection and utility preservation in face de-identification. In IEEE International Joint Conference on Biometrics. IEEE, 1–8.

[18]

Chi Nhan Duong, Kha Gia Quach, Ibsa Jalata, Ngan Le, and Khoa Luu. 2019. Mobiface: A lightweight deep learning face recognition on mobile devices. In 2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS). IEEE, 1–6.

Digital Library

[19]

Ali Elmahmudi and Hassan Ugail. 2019. Deep face recognition using imperfect facial data. Future Generation Computer Systems 99 (2019), 213–225.

Digital Library

[20]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.

[21]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity mappings in deep residual networks. In European conference on computer vision. Springer, 630–645.

[22]

Lingxiao He, Haiqing Li, Qi Zhang, and Zhenan Sun. 2018. Dynamic feature matching for partial face recognition. IEEE Transactions on Image Processing 28, 2 (2018), 791–802.

Digital Library

[23]

Danula Hettiachchi, Mike Schaekermann, Tristan McKinney, and Matthew Lease. 2021. The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can Help. arXiv preprint arXiv:2105.09457(2021).

[24]

Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. 2008. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in’Real-Life’Images: detection, alignment, and recognition.

[25]

Davis E. King. 2009. Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research 10 (2009), 1755–1758.

Digital Library

[26]

Ziyi Kou, Lanyu Shang, Huimin Zeng, Yang Zhang, and Dong Wang. 2021. ExgFair: A Crowdsourcing Data Exchange Approach To Fair Human Face Datasets Augmentation. In 2021 IEEE International Conference on Big Data (Big Data). IEEE, 1285–1290.

[27]

Ziyi Kou, Lanyu Shang, Yang Zhang, and Dong Wang. 2022. HC-COVID: A Hierarchical Crowdsource Knowledge Graph Approach to Explainable COVID-19 Misinformation Detection. Proceedings of the ACM on Human-Computer Interaction 6, GROUP(2022), 1–25.

Digital Library

[28]

Ziyi Kou, Lanyu Shang, Yang Zhang, Christina Youn, and Dong Wang. 2021. Fakesens: A social sensing approach to covid-19 misinformation detection on social media. In 2021 17th International Conference on Distributed Computing in Sensor Systems (DCOSS). IEEE, 140–147.

[29]

Ziyi Kou, Yang Zhang, Lanyu Shang, and Dong Wang. 2021. Faircrowd: Fair human face dataset sampling via batch-level crowdsourcing bias inference. In 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS). IEEE, 1–10.

[30]

Marek Kowalski, Jacek Naruniec, and Tomasz Trzcinski. 2017. Deep alignment network: A convolutional neural network for robust face alignment. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 88–97.

[31]

Neeraj Kumar, Alexander C Berg, Peter N Belhumeur, and Shree K Nayar. 2009. Attribute and simile classifiers for face verification. In 2009 IEEE 12th international conference on computer vision. IEEE, 365–372.

[32]

Mengke Li and Yiu-ming Cheung. 2021. Identity-Preserved Complete Face Recovering Network for Partial Face Image. IEEE Transactions on Emerging Topics in Computational Intelligence (2021).

[33]

Shan Li, Weihong Deng, and JunPing Du. 2017. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2852–2861.

[34]

Bo Liu, Ming Ding, Hanyu Xue, Tianqing Zhu, Dayong Ye, Li Song, and Wanlei Zhou. 2021. DP-Image: Differential Privacy for Image Data in Feature Space. arXiv preprint arXiv:2103.07073(2021).

[35]

Bo Liu, Ming Ding, Tianqing Zhu, Yong Xiang, and Wanlei Zhou. 2019. Adversaries or allies? Privacy and deep learning in big data era. Concurrency and Computation: Practice and Experience 31, 19(2019), e5102.

[36]

Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. Sphereface: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 212–220.

[37]

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2018. Large-scale celebfaces attributes (celeba) dataset. Retrieved August 15, 2018 (2018), 11.

[38]

Zhengzhe Liu, Xiaojuan Qi, and Philip HS Torr. 2020. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8060–8069.

[39]

Zimeng Luo, Jiani Hu, Weihong Deng, and Haifeng Shen. 2018. Deep unsupervised domain adaptation for face recognition. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 453–457.

Digital Library

[40]

Sachit Mahajan, Ling-Jyh Chen, and Tzu-Chieh Tsai. 2017. Swapitup: A face swap application for privacy protection. In 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA). IEEE, 46–50.

[41]

Giacomo Mariani, Anna Monreale, and Francesca Naretto. 2021. Privacy Risk Assessment of Individual Psychometric Profiles. In International Conference on Discovery Science. Springer, 411–421.

[42]

Falko Matern, Christian Riess, and Marc Stamminger. 2019. Exploiting visual artifacts to expose deepfakes and face manipulations. In 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). IEEE, 83–92.

[43]

Vahid Mirjalili, Sebastian Raschka, and Arun Ross. 2019. Flowsan: Privacy-enhancing semi-adversarial networks to confound arbitrary face-based gender classifiers. IEEE Access 7(2019), 99735–99745.

[44]

Phillip Nyoni and Mthulisi Velempini. 2018. Privacy and user awareness on Facebook. South African Journal of Science 114, 5-6 (2018), 1–5.

[45]

Xing Ouyang, Cleitus Antony, Fatima Gunning, Hongyu Zhang, and Yong Liang Guan. 2015. Discrete Fresnel transform and its circular convolution. arXiv preprint arXiv:1510.00574(2015).

[46]

Omkar M Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. (2015).

[47]

Roberto Pellungrini, Luca Pappalardo, Francesca Pratesi, and Anna Monreale. 2017. A data mining approach to assess privacy risk in human mobility data. ACM Transactions on Intelligent Systems and Technology (TIST) 9, 3(2017), 1–27.

Digital Library

[48]

Jeffrey G Proudfoot, David Wilson, Joseph S Valacich, and Michael D Byrd. 2018. Saving face on Facebook: Privacy concerns, social benefits, and impression management. Behaviour & Information Technology 37, 1 (2018), 16–37.

[49]

Yichen Qian, Weihong Deng, and Jiani Hu. 2019. Unsupervised face normalization with extreme pose and expression in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9851–9858.

[50]

MR Reshma and B Kannan. 2019. Approaches on partial face recognition: a literature review. In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI). IEEE, 538–544.

[51]

Hee Jung Ryu, Hartwig Adam, and Margaret Mitchell. 2017. Inclusivefacenet: Improving face attribute detection with race and gender diversity. arXiv preprint arXiv:1712.00193(2017).

[52]

Sefik Ilkin Serengil and Alper Ozpinar. 2020. Lightface: A hybrid deep face recognition framework. In 2020 Innovations in Intelligent Systems and Applications Conference (ASYU). IEEE, 1–5.

[53]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556(2014).

[54]

Qianru Sun, Liqian Ma, Seong Joon Oh, Luc Van Gool, Bernt Schiele, and Mario Fritz. 2018. Natural and effective obfuscation by head inpainting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5050–5059.

[55]

Philipp Terhörst, Naser Damer, Florian Kirchbuchner, and Arjan Kuijper. 2019. Unsupervised privacy-enhancement of face representations using similarity-sensitive noise transformations. Applied Intelligence 49, 8 (2019), 3043–3060.

Digital Library

[56]

Tulane University. 2020. Key Social Media Privacy Issues for 2020. https://sopa.tulane.edu/blog/key-social-media-privacy-issues-2020

[57]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE.Journal of machine learning research 9, 11 (2008).

[58]

Jun Wan, Jing Li, Zhihui Lai, Bo Du, and Lefei Zhang. 2020. Robust face alignment by cascaded regression and de-occlusion. Neural Networks 123(2020), 261–272.

Digital Library

[59]

Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A discriminative feature learning approach for deep face recognition. In European conference on computer vision. Springer, 499–515.

[60]

Xiang Wu, Ran He, Zhenan Sun, and Tieniu Tan. 2018. A light cnn for deep face representation with noisy labels. IEEE Transactions on Information Forensics and Security 13, 11(2018), 2884–2896.

[61]

Yifan Wu, Fan Yang, Yong Xu, and Haibin Ling. 2019. Privacy-protective-GAN for privacy preserving face de-identification. Journal of Computer Science and Technology 34, 1 (2019), 47–60.

[62]

Zihao Xiao, Xianfeng Gao, Chilin Fu, Yinpeng Dong, Wei Gao, Xiaolu Zhang, Jun Zhou, and Jun Zhu. 2021. Improving Transferability of Adversarial Patches on Face Recognition With Generative Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11845–11854.

[63]

Daniel Zhang, Yue Ma, Xiaobo Sharon Hu, and Dong Wang. 2020. Toward privacy-aware task allocation in social sensing-based edge computing systems. IEEE Internet of Things Journal 7, 12 (2020), 11384–11400.

[64]

Daniel Yue Zhang, Ziyi Kou, and Dong Wang. 2020. Fairfl: A fair federated learning approach to reducing demographic bias in privacy-sensitive classification models. In 2020 IEEE International Conference on Big Data (Big Data). IEEE, 1051–1060.

[65]

Zhanpeng Zhang, Ping Luo, Chen Change Loy, and Xiaoou Tang. 2014. Facial landmark detection by deep multi-task learning. In European conference on computer vision. Springer, 94–108.

[66]

Zheng Zhu, Guan Huang, Jiankang Deng, Yun Ye, Junjie Huang, Xinze Chen, Jiagang Zhu, Tian Yang, Jiwen Lu, Dalong Du, 2021. Webface260m: A benchmark unveiling the power of million-scale deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10492–10502.

Cited By

Kou ZTian YJiang MZhang XSerra ESpezzano F(2024)FaDE: A Face Segment Driven Identity Anonymization Framework For Fair Face RecognitionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679737(1121-1131)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679737
Sujatha EManickam. MMinisha RRekha KIndumathy T(2024)Heterogeneous Face Recognition Algorithm: Convolution Neural Network Approach2024 5th International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV)10.1109/ICICV62344.2024.00038(205-208)Online publication date: 11-Mar-2024
https://doi.org/10.1109/ICICV62344.2024.00038
Kou ZManchanda SLin SXie MWang HZhang XFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Modeling Sequential Collaborative User Behaviors For Seller-Aware Next Basket RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614973(1097-1106)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614973
Show More Cited By

Index Terms

Can I only share my eyes? A Web Crowdsourcing based Face Partition Approach Towards Privacy-Aware Face Recognition

Index terms have been assigned to the content through auto-classification.

Recommendations

Age-Invariant Face Recognition

One of the challenges in automatic face recognition is to achieve temporal invariance. In other words, the goal is to come up with a representation and matching scheme that is robust to changes due to facial aging. Facial aging is a complex process that ...
Face recognition under varying illumination using gradientfaces

In this correspondence, we propose a novel method to extract illumination insensitive features for face recognition under varying lighting called the Gradientfaces. Theoretical analysis shows Gradientfaces is an illumination insensitive measure, and ...
Automatic face analysis system based on face recognition and facial physiognomy
ICHIT'06: Proceedings of the 1st international conference on Advances in hybrid information technology

An automatic face analysis system is proposed which uses face recognition and facial physiognomy. It first detects human's face, extracts its features, and classifies the shape of facial features. It will analyze the person's facial physiognomy and then ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
426
Total Downloads

Downloads (Last 12 months)145
Downloads (Last 6 weeks)20

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kou ZTian YJiang MZhang XSerra ESpezzano F(2024)FaDE: A Face Segment Driven Identity Anonymization Framework For Fair Face RecognitionProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679737(1121-1131)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679737
Sujatha EManickam. MMinisha RRekha KIndumathy T(2024)Heterogeneous Face Recognition Algorithm: Convolution Neural Network Approach2024 5th International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV)10.1109/ICICV62344.2024.00038(205-208)Online publication date: 11-Mar-2024
https://doi.org/10.1109/ICICV62344.2024.00038
Kou ZManchanda SLin SXie MWang HZhang XFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Modeling Sequential Collaborative User Behaviors For Seller-Aware Next Basket RecommendationProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614973(1097-1106)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3614973
Wang DZhang DWang DZhang D(2023)Further ReadingsSocial Edge Computing10.1007/978-3-031-26936-3_8(155-163)Online publication date: 20-Feb-2023
https://doi.org/10.1007/978-3-031-26936-3_8

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents