-
How Knowledge Distillation Mitigates the Synthetic Gap in Fair Face Recognition
Authors:
Pedro C. Neto,
Ivona Colakovic,
Sašo Karakatič,
Ana F. Sequeira
Abstract:
Leveraging the capabilities of Knowledge Distillation (KD) strategies, we devise a strategy to fight the recent retraction of face recognition datasets. Given a pretrained Teacher model trained on a real dataset, we show that carefully utilising synthetic datasets, or a mix between real and synthetic datasets to distil knowledge from this teacher to smaller students can yield surprising results. I…
▽ More
Leveraging the capabilities of Knowledge Distillation (KD) strategies, we devise a strategy to fight the recent retraction of face recognition datasets. Given a pretrained Teacher model trained on a real dataset, we show that carefully utilising synthetic datasets, or a mix between real and synthetic datasets to distil knowledge from this teacher to smaller students can yield surprising results. In this sense, we trained 33 different models with and without KD, on different datasets, with different architectures and losses. And our findings are consistent, using KD leads to performance gains across all ethnicities and decreased bias. In addition, it helps to mitigate the performance gap between real and synthetic datasets. This approach addresses the limitations of synthetic data training, improving both the accuracy and fairness of face recognition models.
△ Less
Submitted 30 August, 2024;
originally announced August 2024.
-
MST-KD: Multiple Specialized Teachers Knowledge Distillation for Fair Face Recognition
Authors:
Eduarda Caldeira,
Jaime S. Cardoso,
Ana F. Sequeira,
Pedro C. Neto
Abstract:
As in school, one teacher to cover all subjects is insufficient to distill equally robust information to a student. Hence, each subject is taught by a highly specialised teacher. Following a similar philosophy, we propose a multiple specialized teacher framework to distill knowledge to a student network. In our approach, directed at face recognition use cases, we train four teachers on one specifi…
▽ More
As in school, one teacher to cover all subjects is insufficient to distill equally robust information to a student. Hence, each subject is taught by a highly specialised teacher. Following a similar philosophy, we propose a multiple specialized teacher framework to distill knowledge to a student network. In our approach, directed at face recognition use cases, we train four teachers on one specific ethnicity, leading to four highly specialized and biased teachers. Our strategy learns a project of these four teachers into a common space and distill that information to a student network. Our results highlighted increased performance and reduced bias for all our experiments. In addition, we further show that having biased/specialized teachers is crucial by showing that our approach achieves better results than when knowledge is distilled from four teachers trained on balanced datasets. Our approach represents a step forward to the understanding of the importance of ethnicity-specific features.
△ Less
Submitted 29 August, 2024;
originally announced August 2024.
-
Fairness Under Cover: Evaluating the Impact of Occlusions on Demographic Bias in Facial Recognition
Authors:
Rafael M. Mamede,
Pedro C. Neto,
Ana F. Sequeira
Abstract:
This study investigates the effects of occlusions on the fairness of face recognition systems, particularly focusing on demographic biases. Using the Racial Faces in the Wild (RFW) dataset and synthetically added realistic occlusions, we evaluate their effect on the performance of face recognition models trained on the BUPT-Balanced and BUPT-GlobalFace datasets. We note increases in the dispersion…
▽ More
This study investigates the effects of occlusions on the fairness of face recognition systems, particularly focusing on demographic biases. Using the Racial Faces in the Wild (RFW) dataset and synthetically added realistic occlusions, we evaluate their effect on the performance of face recognition models trained on the BUPT-Balanced and BUPT-GlobalFace datasets. We note increases in the dispersion of FMR, FNMR, and accuracy alongside decreases in fairness according to Equilized Odds, Demographic Parity, STD of Accuracy, and Fairness Discrepancy Rate. Additionally, we utilize a pixel attribution method to understand the importance of occlusions in model predictions, proposing a new metric, Face Occlusion Impact Ratio (FOIR), that quantifies the extent to which occlusions affect model performance across different demographic groups. Our results indicate that occlusions exacerbate existing demographic biases, with models placing higher importance on occlusions in an unequal fashion, particularly affecting African individuals more severely.
△ Less
Submitted 19 August, 2024;
originally announced August 2024.
-
A Laplacian-based Quantum Graph Neural Network for Semi-Supervised Learning
Authors:
Hamed Gholipour,
Farid Bozorgnia,
Kailash Hambarde,
Hamzeh Mohammadigheymasi,
Javier Mancilla,
Andre Sequeira,
Joao Neves,
Hugo Proença
Abstract:
Laplacian learning method is a well-established technique in classical graph-based semi-supervised learning, but its potential in the quantum domain remains largely unexplored. This study investigates the performance of the Laplacian-based Quantum Semi-Supervised Learning (QSSL) method across four benchmark datasets -- Iris, Wine, Breast Cancer Wisconsin, and Heart Disease. Further analysis explor…
▽ More
Laplacian learning method is a well-established technique in classical graph-based semi-supervised learning, but its potential in the quantum domain remains largely unexplored. This study investigates the performance of the Laplacian-based Quantum Semi-Supervised Learning (QSSL) method across four benchmark datasets -- Iris, Wine, Breast Cancer Wisconsin, and Heart Disease. Further analysis explores the impact of increasing Qubit counts, revealing that adding more Qubits to a quantum system doesn't always improve performance. The effectiveness of additional Qubits depends on the quantum algorithm and how well it matches the dataset. Additionally, we examine the effects of varying entangling layers on entanglement entropy and test accuracy. The performance of Laplacian learning is highly dependent on the number of entangling layers, with optimal configurations varying across different datasets. Typically, moderate levels of entanglement offer the best balance between model complexity and generalization capabilities. These observations highlight the crucial need for precise hyperparameter tuning tailored to each dataset to achieve optimal performance in Laplacian learning methods.
△ Less
Submitted 13 August, 2024; v1 submitted 10 August, 2024;
originally announced August 2024.
-
Trainability issues in quantum policy gradients
Authors:
André Sequeira,
Luis Paulo Santos,
Luis Soares Barbosa
Abstract:
This research explores the trainability of Parameterized Quantum circuit-based policies in Reinforcement Learning, an area that has recently seen a surge in empirical exploration. While some studies suggest improved sample complexity using quantum gradient estimation, the efficient trainability of these policies remains an open question. Our findings reveal significant challenges, including standa…
▽ More
This research explores the trainability of Parameterized Quantum circuit-based policies in Reinforcement Learning, an area that has recently seen a surge in empirical exploration. While some studies suggest improved sample complexity using quantum gradient estimation, the efficient trainability of these policies remains an open question. Our findings reveal significant challenges, including standard Barren Plateaus with exponentially small gradients and gradient explosion. These phenomena depend on the type of basis-state partitioning and mapping these partitions onto actions. For a polynomial number of actions, a trainable window can be ensured with a polynomial number of measurements if a contiguous-like partitioning of basis-states is employed. These results are empirically validated in a multi-armed bandit environment.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
Massively Annotated Datasets for Assessment of Synthetic and Real Data in Face Recognition
Authors:
Pedro C. Neto,
Rafael M. Mamede,
Carolina Albuquerque,
Tiago Gonçalves,
Ana F. Sequeira
Abstract:
Face recognition applications have grown in parallel with the size of datasets, complexity of deep learning models and computational power. However, while deep learning models evolve to become more capable and computational power keeps increasing, the datasets available are being retracted and removed from public access. Privacy and ethical concerns are relevant topics within these domains. Throug…
▽ More
Face recognition applications have grown in parallel with the size of datasets, complexity of deep learning models and computational power. However, while deep learning models evolve to become more capable and computational power keeps increasing, the datasets available are being retracted and removed from public access. Privacy and ethical concerns are relevant topics within these domains. Through generative artificial intelligence, researchers have put efforts into the development of completely synthetic datasets that can be used to train face recognition systems. Nonetheless, the recent advances have not been sufficient to achieve performance comparable to the state-of-the-art models trained on real data. To study the drift between the performance of models trained on real and synthetic datasets, we leverage a massive attribute classifier (MAC) to create annotations for four datasets: two real and two synthetic. From these annotations, we conduct studies on the distribution of each attribute within all four datasets. Additionally, we further inspect the differences between real and synthetic datasets on the attribute set. When comparing through the Kullback-Leibler divergence we have found differences between real and synthetic samples. Interestingly enough, we have verified that while real samples suffice to explain the synthetic distribution, the opposite could not be further from being true.
△ Less
Submitted 23 April, 2024;
originally announced April 2024.
-
Second Edition FRCSyn Challenge at CVPR 2024: Face Recognition Challenge in the Era of Synthetic Data
Authors:
Ivan DeAndres-Tame,
Ruben Tolosana,
Pietro Melzi,
Ruben Vera-Rodriguez,
Minchul Kim,
Christian Rathgeb,
Xiaoming Liu,
Aythami Morales,
Julian Fierrez,
Javier Ortega-Garcia,
Zhizhou Zhong,
Yuge Huang,
Yuxi Mi,
Shouhong Ding,
Shuigeng Zhou,
Shuai He,
Lingzhi Fu,
Heng Cong,
Rongyu Zhang,
Zhihong Xiao,
Evgeny Smirnov,
Anton Pimenov,
Aleksei Grigorev,
Denis Timoshenko,
Kaleb Mesfin Asfaw
, et al. (33 additional authors not shown)
Abstract:
Synthetic data is gaining increasing relevance for training machine learning models. This is mainly motivated due to several factors such as the lack of real data and intra-class variability, time and errors produced in manual labeling, and in some cases privacy concerns, among others. This paper presents an overview of the 2nd edition of the Face Recognition Challenge in the Era of Synthetic Data…
▽ More
Synthetic data is gaining increasing relevance for training machine learning models. This is mainly motivated due to several factors such as the lack of real data and intra-class variability, time and errors produced in manual labeling, and in some cases privacy concerns, among others. This paper presents an overview of the 2nd edition of the Face Recognition Challenge in the Era of Synthetic Data (FRCSyn) organized at CVPR 2024. FRCSyn aims to investigate the use of synthetic data in face recognition to address current technological limitations, including data privacy concerns, demographic biases, generalization to novel scenarios, and performance constraints in challenging situations such as aging, pose variations, and occlusions. Unlike the 1st edition, in which synthetic data from DCFace and GANDiffFace methods was only allowed to train face recognition systems, in this 2nd edition we propose new sub-tasks that allow participants to explore novel face generative methods. The outcomes of the 2nd FRCSyn Challenge, along with the proposed experimental protocol and benchmarking contribute significantly to the application of synthetic data to face recognition.
△ Less
Submitted 16 April, 2024;
originally announced April 2024.
-
Empowering Credit Scoring Systems with Quantum-Enhanced Machine Learning
Authors:
Javier Mancilla,
André Sequeira,
Tomas Tagliani,
Francisco Llaneza,
Claudio Beiza
Abstract:
Quantum Kernels are projected to provide early-stage usefulness for quantum machine learning. However, highly sophisticated classical models are hard to surpass without losing interpretability, particularly when vast datasets can be exploited. Nonetheless, classical models struggle once data is scarce and skewed. Quantum feature spaces are projected to find better links between data features and t…
▽ More
Quantum Kernels are projected to provide early-stage usefulness for quantum machine learning. However, highly sophisticated classical models are hard to surpass without losing interpretability, particularly when vast datasets can be exploited. Nonetheless, classical models struggle once data is scarce and skewed. Quantum feature spaces are projected to find better links between data features and the target class to be predicted even in such challenging scenarios and most importantly, enhanced generalization capabilities. In this work, we propose a novel approach called Systemic Quantum Score (SQS) and provide preliminary results indicating potential advantage over purely classical models in a production grade use case for the Finance sector. SQS shows in our specific study an increased capacity to extract patterns out of fewer data points as well as improved performance over data-hungry algorithms such as XGBoost, providing advantage in a competitive market as it is the FinTech and Neobank regime.
△ Less
Submitted 3 April, 2024; v1 submitted 15 March, 2024;
originally announced April 2024.
-
VQC-Based Reinforcement Learning with Data Re-uploading: Performance and Trainability
Authors:
Rodrigo Coelho,
André Sequeira,
Luís Paulo Santos
Abstract:
Reinforcement Learning (RL) consists of designing agents that make intelligent decisions without human supervision. When used alongside function approximators such as Neural Networks (NNs), RL is capable of solving extremely complex problems. Deep Q-Learning, a RL algorithm that uses Deep NNs, achieved super-human performance in some specific tasks. Nonetheless, it is also possible to use Variatio…
▽ More
Reinforcement Learning (RL) consists of designing agents that make intelligent decisions without human supervision. When used alongside function approximators such as Neural Networks (NNs), RL is capable of solving extremely complex problems. Deep Q-Learning, a RL algorithm that uses Deep NNs, achieved super-human performance in some specific tasks. Nonetheless, it is also possible to use Variational Quantum Circuits (VQCs) as function approximators in RL algorithms. This work empirically studies the performance and trainability of such VQC-based Deep Q-Learning models in classic control benchmark environments. More specifically, we research how data re-uploading affects both these metrics. We show that the magnitude and the variance of the gradients of these models remain substantial throughout training due to the moving targets of Deep Q-Learning. Moreover, we empirically show that increasing the number of qubits does not lead to an exponential vanishing behavior of the magnitude and variance of the gradients for a PQC approximating a 2-design, unlike what was expected due to the Barren Plateau Phenomenon. This hints at the possibility of VQCs being specially adequate for being used as function approximators in such a context.
△ Less
Submitted 21 January, 2024;
originally announced January 2024.
-
Model Compression Techniques in Biometrics Applications: A Survey
Authors:
Eduarda Caldeira,
Pedro C. Neto,
Marco Huber,
Naser Damer,
Ana F. Sequeira
Abstract:
The development of deep learning algorithms has extensively empowered humanity's task automatization capacity. However, the huge improvement in the performance of these models is highly correlated with their increasing level of complexity, limiting their usefulness in human-oriented applications, which are usually deployed in resource-constrained devices. This led to the development of compression…
▽ More
The development of deep learning algorithms has extensively empowered humanity's task automatization capacity. However, the huge improvement in the performance of these models is highly correlated with their increasing level of complexity, limiting their usefulness in human-oriented applications, which are usually deployed in resource-constrained devices. This led to the development of compression techniques that drastically reduce the computational and memory costs of deep learning models without significant performance degradation. This paper aims to systematize the current literature on this topic by presenting a comprehensive survey of model compression techniques in biometrics applications, namely quantization, knowledge distillation and pruning. We conduct a critical analysis of the comparative value of these techniques, focusing on their advantages and disadvantages and presenting suggestions for future work directions that can potentially improve the current methods. Additionally, we discuss and analyze the link between model bias and model compression, highlighting the need to direct compression research toward model fairness in future works.
△ Less
Submitted 18 January, 2024;
originally announced January 2024.
-
On Quantum Natural Policy Gradients
Authors:
André Sequeira,
Luis Paulo Santos,
Luis Soares Barbosa
Abstract:
This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov Deci…
▽ More
This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov Decision Processes, is less clear. Through a detailed analysis of Löwner inequalities between quantum and classical FIMs, this study uncovers the nuanced distinctions and implications of using each type of FIM. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM. Empirical evaluations in classic control benchmarks suggest even though quantum FIM preconditioning outperforms standard gradient ascent, in general it is not superior to classical FIM preconditioning.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
Compressed Models Decompress Race Biases: What Quantized Models Forget for Fair Face Recognition
Authors:
Pedro C. Neto,
Eduarda Caldeira,
Jaime S. Cardoso,
Ana F. Sequeira
Abstract:
With the ever-growing complexity of deep learning models for face recognition, it becomes hard to deploy these systems in real life. Researchers have two options: 1) use smaller models; 2) compress their current models. Since the usage of smaller models might lead to concerning biases, compression gains relevance. However, compressing might be also responsible for an increase in the bias of the fi…
▽ More
With the ever-growing complexity of deep learning models for face recognition, it becomes hard to deploy these systems in real life. Researchers have two options: 1) use smaller models; 2) compress their current models. Since the usage of smaller models might lead to concerning biases, compression gains relevance. However, compressing might be also responsible for an increase in the bias of the final model. We investigate the overall performance, the performance on each ethnicity subgroup and the racial bias of a State-of-the-Art quantization approach when used with synthetic and real data. This analysis provides a few more details on potential benefits of performing quantization with synthetic data, for instance, the reduction of biases on the majority of test scenarios. We tested five distinct architectures and three different training datasets. The models were evaluated on a fourth dataset which was collected to infer and compare the performance of face recognition models on different ethnicity.
△ Less
Submitted 22 August, 2023;
originally announced August 2023.
-
Unveiling the Two-Faced Truth: Disentangling Morphed Identities for Face Morphing Detection
Authors:
Eduarda Caldeira,
Pedro C. Neto,
Tiago Gonçalves,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
Morphing attacks keep threatening biometric systems, especially face recognition systems. Over time they have become simpler to perform and more realistic, as such, the usage of deep learning systems to detect these attacks has grown. At the same time, there is a constant concern regarding the lack of interpretability of deep learning models. Balancing performance and interpretability has been a d…
▽ More
Morphing attacks keep threatening biometric systems, especially face recognition systems. Over time they have become simpler to perform and more realistic, as such, the usage of deep learning systems to detect these attacks has grown. At the same time, there is a constant concern regarding the lack of interpretability of deep learning models. Balancing performance and interpretability has been a difficult task for scientists. However, by leveraging domain information and proving some constraints, we have been able to develop IDistill, an interpretable method with state-of-the-art performance that provides information on both the identity separation on morph samples and their contribution to the final prediction. The domain information is learnt by an autoencoder and distilled to a classifier system in order to teach it to separate identity information. When compared to other methods in the literature it outperforms them in three out of five databases and is competitive in the remaining.
△ Less
Submitted 5 June, 2023;
originally announced June 2023.
-
PIC-Score: Probabilistic Interpretable Comparison Score for Optimal Matching Confidence in Single- and Multi-Biometric (Face) Recognition
Authors:
Pedro C. Neto,
Ana F. Sequeira,
Jaime S. Cardoso,
Philipp Terhörst
Abstract:
In the context of biometrics, matching confidence refers to the confidence that a given matching decision is correct. Since many biometric systems operate in critical decision-making processes, such as in forensics investigations, accurately and reliably stating the matching confidence becomes of high importance. Previous works on biometric confidence estimation can well differentiate between high…
▽ More
In the context of biometrics, matching confidence refers to the confidence that a given matching decision is correct. Since many biometric systems operate in critical decision-making processes, such as in forensics investigations, accurately and reliably stating the matching confidence becomes of high importance. Previous works on biometric confidence estimation can well differentiate between high and low confidence, but lack interpretability. Therefore, they do not provide accurate probabilistic estimates of the correctness of a decision. In this work, we propose a probabilistic interpretable comparison (PIC) score that accurately reflects the probability that the score originates from samples of the same identity. We prove that the proposed approach provides optimal matching confidence. Contrary to other approaches, it can also optimally combine multiple samples in a joint PIC score which further increases the recognition and confidence estimation performance. In the experiments, the proposed PIC approach is compared against all biometric confidence estimation methods available on four publicly available databases and five state-of-the-art face recognition systems. The results demonstrate that PIC has a significantly more accurate probabilistic interpretation than similar approaches and is highly effective for multi-biometric recognition. The code is publicly-available.
△ Less
Submitted 21 April, 2023; v1 submitted 22 November, 2022;
originally announced November 2022.
-
Causality-Inspired Taxonomy for Explainable Artificial Intelligence
Authors:
Pedro C. Neto,
Tiago Gonçalves,
João Ribeiro Pinto,
Wilson Silva,
Ana F. Sequeira,
Arun Ross,
Jaime S. Cardoso
Abstract:
As two sides of the same coin, causality and explainable artificial intelligence (xAI) were initially proposed and developed with different goals. However, the latter can only be complete when seen through the lens of the causality framework. As such, we propose a novel causality-inspired framework for xAI that creates an environment for the development of xAI approaches. To show its applicability…
▽ More
As two sides of the same coin, causality and explainable artificial intelligence (xAI) were initially proposed and developed with different goals. However, the latter can only be complete when seen through the lens of the causality framework. As such, we propose a novel causality-inspired framework for xAI that creates an environment for the development of xAI approaches. To show its applicability, biometrics was used as case study. For this, we have analysed 81 research papers on a myriad of biometric modalities and different tasks. We have categorised each of these methods according to our novel xAI Ladder and discussed the future directions of the field.
△ Less
Submitted 4 March, 2024; v1 submitted 19 August, 2022;
originally announced August 2022.
-
OrthoMAD: Morphing Attack Detection Through Orthogonal Identity Disentanglement
Authors:
Pedro C. Neto,
Tiago Gonçalves,
Marco Huber,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
Morphing attacks are one of the many threats that are constantly affecting deep face recognition systems. It consists of selecting two faces from different individuals and fusing them into a final image that contains the identity information of both. In this work, we propose a novel regularisation term that takes into account the existent identity information in both and promotes the creation of t…
▽ More
Morphing attacks are one of the many threats that are constantly affecting deep face recognition systems. It consists of selecting two faces from different individuals and fusing them into a final image that contains the identity information of both. In this work, we propose a novel regularisation term that takes into account the existent identity information in both and promotes the creation of two orthogonal latent vectors. We evaluate our proposed method (OrthoMAD) in five different types of morphing in the FRLL dataset and evaluate the performance of our model when trained on five distinct datasets. With a small ResNet-18 as the backbone, we achieve state-of-the-art results in the majority of the experiments, and competitive results in the others. The code of this paper will be publicly available.
△ Less
Submitted 23 August, 2022; v1 submitted 16 August, 2022;
originally announced August 2022.
-
SYN-MAD 2022: Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data
Authors:
Marco Huber,
Fadi Boutros,
Anh Thi Luu,
Kiran Raja,
Raghavendra Ramachandra,
Naser Damer,
Pedro C. Neto,
Tiago Gonçalves,
Ana F. Sequeira,
Jaime S. Cardoso,
João Tremoço,
Miguel Lourenço,
Sergio Serra,
Eduardo Cermeño,
Marija Ivanovska,
Borut Batagelj,
Andrej Kronovšek,
Peter Peer,
Vitomir Štruc
Abstract:
This paper presents a summary of the Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data (SYN-MAD) held at the 2022 International Joint Conference on Biometrics (IJCB 2022). The competition attracted a total of 12 participating teams, both from academia and industry and present in 11 different countries. In the end, seven valid submissions were submitted by…
▽ More
This paper presents a summary of the Competition on Face Morphing Attack Detection Based on Privacy-aware Synthetic Training Data (SYN-MAD) held at the 2022 International Joint Conference on Biometrics (IJCB 2022). The competition attracted a total of 12 participating teams, both from academia and industry and present in 11 different countries. In the end, seven valid submissions were submitted by the participating teams and evaluated by the organizers. The competition was held to present and attract solutions that deal with detecting face morphing attacks while protecting people's privacy for ethical and legal reasons. To ensure this, the training data was limited to synthetic data provided by the organizers. The submitted solutions presented innovations that led to outperforming the considered baseline in many experimental settings. The evaluation benchmark is now available at: https://github.com/marcohuber/SYN-MAD-2022.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
OCFR 2022: Competition on Occluded Face Recognition From Synthetically Generated Structure-Aware Occlusions
Authors:
Pedro C. Neto,
Fadi Boutros,
Joao Ribeiro Pinto,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso,
Messaoud Bengherabi,
Abderaouf Bousnat,
Sana Boucheta,
Nesrine Hebbadj,
Mustafa Ekrem Erakın,
Uğur Demir,
Hazım Kemal Ekenel,
Pedro Beber de Queiroz Vidal,
David Menotti
Abstract:
This work summarizes the IJCB Occluded Face Recognition Competition 2022 (IJCB-OCFR-2022) embraced by the 2022 International Joint Conference on Biometrics (IJCB 2022). OCFR-2022 attracted a total of 3 participating teams, from academia. Eventually, six valid submissions were submitted and then evaluated by the organizers. The competition was held to address the challenge of face recognition in th…
▽ More
This work summarizes the IJCB Occluded Face Recognition Competition 2022 (IJCB-OCFR-2022) embraced by the 2022 International Joint Conference on Biometrics (IJCB 2022). OCFR-2022 attracted a total of 3 participating teams, from academia. Eventually, six valid submissions were submitted and then evaluated by the organizers. The competition was held to address the challenge of face recognition in the presence of severe face occlusions. The participants were free to use any training data and the testing data was built by the organisers by synthetically occluding parts of the face images using a well-known dataset. The submitted solutions presented innovations and performed very competitively with the considered baseline. A major output of this competition is a challenging, realistic, and diverse, and publicly available occluded face recognition benchmark with well defined evaluation protocols.
△ Less
Submitted 15 August, 2022; v1 submitted 4 August, 2022;
originally announced August 2022.
-
Policy Gradients using Variational Quantum Circuits
Authors:
André Sequeira,
Luis Paulo Santos,
Luís Soares Barbosa
Abstract:
Variational Quantum Circuits are being used as versatile Quantum Machine Learning models. Some empirical results exhibit an advantage in supervised and generative learning tasks. However, when applied to Reinforcement Learning, less is known. In this work, we considered a Variational Quantum Circuit composed of a low-depth hardware-efficient ansatz as the parameterized policy of a Reinforcement Le…
▽ More
Variational Quantum Circuits are being used as versatile Quantum Machine Learning models. Some empirical results exhibit an advantage in supervised and generative learning tasks. However, when applied to Reinforcement Learning, less is known. In this work, we considered a Variational Quantum Circuit composed of a low-depth hardware-efficient ansatz as the parameterized policy of a Reinforcement Learning agent. We show that an $ε$-approximation of the policy gradient can be obtained using a logarithmic number of samples concerning the total number of parameters. We empirically verify that such quantum models behave similarly or even outperform typical classical neural networks used in standard benchmarking environments and in quantum control, using only a fraction of the parameters. Moreover, we study the Barren Plateau phenomenon in quantum policy gradients using the Fisher Information Matrix spectrum.
△ Less
Submitted 15 January, 2023; v1 submitted 20 March, 2022;
originally announced March 2022.
-
Myope Models -- Are face presentation attack detection models short-sighted?
Authors:
Pedro C. Neto,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
Presentation attacks are recurrent threats to biometric systems, where impostors attempt to bypass these systems. Humans often use background information as contextual cues for their visual system. Yet, regarding face-based systems, the background is often discarded, since face presentation attack detection (PAD) models are mostly trained with face crops. This work presents a comparative study of…
▽ More
Presentation attacks are recurrent threats to biometric systems, where impostors attempt to bypass these systems. Humans often use background information as contextual cues for their visual system. Yet, regarding face-based systems, the background is often discarded, since face presentation attack detection (PAD) models are mostly trained with face crops. This work presents a comparative study of face PAD models (including multi-task learning, adversarial training and dynamic frame selection) in two settings: with and without crops. The results show that the performance is consistently better when the background is present in the images. The proposed multi-task methodology beats the state-of-the-art results on the ROSE-Youtu dataset by a large margin with an equal error rate of 0.2%. Furthermore, we analyze the models' predictions with Grad-CAM++ with the aim to investigate to what extent the models focus on background elements that are known to be useful for human inspection. From this analysis we can conclude that the background cues are not relevant across all the attacks. Thus, showing the capability of the model to leverage the background information only when necessary.
△ Less
Submitted 22 November, 2021;
originally announced November 2021.
-
FocusFace: Multi-task Contrastive Learning for Masked Face Recognition
Authors:
Pedro C. Neto,
Fadi Boutros,
João Ribeiro Pinto,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
SARS-CoV-2 has presented direct and indirect challenges to the scientific community. One of the most prominent indirect challenges advents from the mandatory use of face masks in a large number of countries. Face recognition methods struggle to perform identity verification with similar accuracy on masked and unmasked individuals. It has been shown that the performance of these methods drops consi…
▽ More
SARS-CoV-2 has presented direct and indirect challenges to the scientific community. One of the most prominent indirect challenges advents from the mandatory use of face masks in a large number of countries. Face recognition methods struggle to perform identity verification with similar accuracy on masked and unmasked individuals. It has been shown that the performance of these methods drops considerably in the presence of face masks, especially if the reference image is unmasked. We propose FocusFace, a multi-task architecture that uses contrastive learning to be able to accurately perform masked face recognition. The proposed architecture is designed to be trained from scratch or to work on top of state-of-the-art face recognition methods without sacrificing the capabilities of a existing models in conventional face recognition tasks. We also explore different approaches to design the contrastive learning module. Results are presented in terms of masked-masked (M-M) and unmasked-masked (U-M) face verification performance. For both settings, the results are on par with published methods, but for M-M specifically, the proposed method was able to outperform all the solutions that it was compared to. We further show that when using our method on top of already existing methods the training computational costs decrease significantly while retaining similar performances. The implementation and the trained models are available at GitHub.
△ Less
Submitted 1 November, 2021; v1 submitted 28 October, 2021;
originally announced October 2021.
-
My Eyes Are Up Here: Promoting Focus on Uncovered Regions in Masked Face Recognition
Authors:
Pedro C. Neto,
Fadi Boutros,
João Ribeiro Pinto,
Mohsen Saffari,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
The recent Covid-19 pandemic and the fact that wearing masks in public is now mandatory in several countries, created challenges in the use of face recognition systems (FRS). In this work, we address the challenge of masked face recognition (MFR) and focus on evaluating the verification performance in FRS when verifying masked vs unmasked faces compared to verifying only unmasked faces. We propose…
▽ More
The recent Covid-19 pandemic and the fact that wearing masks in public is now mandatory in several countries, created challenges in the use of face recognition systems (FRS). In this work, we address the challenge of masked face recognition (MFR) and focus on evaluating the verification performance in FRS when verifying masked vs unmasked faces compared to verifying only unmasked faces. We propose a methodology that combines the traditional triplet loss and the mean squared error (MSE) intending to improve the robustness of an MFR system in the masked-unmasked comparison mode. The results obtained by our proposed method show improvements in a detailed step-wise ablation study. The conducted study showed significant performance gains induced by our proposed training paradigm and modified triplet loss on two evaluation databases.
△ Less
Submitted 18 August, 2021; v1 submitted 2 August, 2021;
originally announced August 2021.
-
MFR 2021: Masked Face Recognition Competition
Authors:
Fadi Boutros,
Naser Damer,
Jan Niklas Kolf,
Kiran Raja,
Florian Kirchbuchner,
Raghavendra Ramachandra,
Arjan Kuijper,
Pengcheng Fang,
Chao Zhang,
Fei Wang,
David Montero,
Naiara Aginako,
Basilio Sierra,
Marcos Nieto,
Mustafa Ekrem Erakin,
Ugur Demir,
Hazim Kemal,
Ekenel,
Asaki Kataoka,
Kohei Ichikawa,
Shizuma Kubo,
Jie Zhang,
Mingjie He,
Dan Han,
Shiguang Shan
, et al. (10 additional authors not shown)
Abstract:
This paper presents a summary of the Masked Face Recognition Competitions (MFR) held within the 2021 International Joint Conference on Biometrics (IJCB 2021). The competition attracted a total of 10 participating teams with valid submissions. The affiliations of these teams are diverse and associated with academia and industry in nine different countries. These teams successfully submitted 18 vali…
▽ More
This paper presents a summary of the Masked Face Recognition Competitions (MFR) held within the 2021 International Joint Conference on Biometrics (IJCB 2021). The competition attracted a total of 10 participating teams with valid submissions. The affiliations of these teams are diverse and associated with academia and industry in nine different countries. These teams successfully submitted 18 valid solutions. The competition is designed to motivate solutions aiming at enhancing the face recognition accuracy of masked faces. Moreover, the competition considered the deployability of the proposed solutions by taking the compactness of the face recognition models into account. A private dataset representing a collaborative, multi-session, real masked, capture scenario is used to evaluate the submitted solutions. In comparison to one of the top-performing academic face recognition solutions, 10 out of the 18 submitted solutions did score higher masked face verification accuracy.
△ Less
Submitted 29 June, 2021;
originally announced June 2021.
-
Exploring the Public Reaction to COVID-19 News on Social Media in Portugal
Authors:
Luciana Oliveira,
Arminda Sequeira,
Adriana Oliveira,
Paulino Silva,
Anabela Mesquita
Abstract:
The outburst and proliferation of the COVID-19 pandemic, together with the subsequent social distancing measures, have raised massive challenges in almost all domains of public and private life around the globe. The stay-at-home movement has pushed the news audiences into social networks, which, in turn, has become the most prolific field for receiving and sharing news updates, as well as for publ…
▽ More
The outburst and proliferation of the COVID-19 pandemic, together with the subsequent social distancing measures, have raised massive challenges in almost all domains of public and private life around the globe. The stay-at-home movement has pushed the news audiences into social networks, which, in turn, has become the most prolific field for receiving and sharing news updates, as well as for public expression of opinions, concerns and feelings about the pandemic. Public opinion is a critical aspect in analysing how the information and events impact peoples lives, and research has shown that social media data may be promising in understanding how people respond to health risks and social crisis, which are the feelings they tend to share and how they are adapting to unforeseen circumstances that threaten almost all societal spheres. This paper presents results from a social media analysis of 61532 news headlines posted by the major daily news outlet in Portugal, Sic Noticias, on Facebook, from January to December 2020, focusing on the issues attention cycle and audiences emotional response to the COVID news outburst. This work adds to the emergent body of studies examining public response to the coronavirus pandemic on social media data.
△ Less
Submitted 15 February, 2021;
originally announced February 2021.
-
Context, input and process as critical elements for successful Emergency Remote Learning
Authors:
Luciana Oliveira,
Anabela Mesquita,
Arminda Sequeira,
Adriana Oliveira,
Paulino Silva
Abstract:
In Spring 2020, the world moved from traditional classes to what was coined as ERL (Emergency Remote Teaching, Learning, Instruction), posing real challenges to all actors involved, requiring an immediate, unprecedented, and unplanned devising of mitigation strategies. The impacts of this transition cannot, however, be studied only at the educational level, as it consists of a broader social shift…
▽ More
In Spring 2020, the world moved from traditional classes to what was coined as ERL (Emergency Remote Teaching, Learning, Instruction), posing real challenges to all actors involved, requiring an immediate, unprecedented, and unplanned devising of mitigation strategies. The impacts of this transition cannot, however, be studied only at the educational level, as it consists of a broader social shift with multidomain repercussions. In this paper, we use the CIPP model (Context, Input, Process and Product evaluations) to further investigate interrelations among the context, input and process elements of ERL during the first wave of COVID-19, as the second wave presses towards reconfining. A correlation analysis of 46 variables, based students responses (N=360) to a closed-ended questionnaire shows the crucial importance of motivation and engagement in online classes, as learning enablers or constrainers. These also shape the students perception of the role that online classes play in helping them to stay more positive during ERL.
△ Less
Submitted 6 January, 2021;
originally announced January 2021.