Seminar Report
Seminar Report
Seminar Report
CHAPTER 1
INTRODUCTION
Neuroimaging technologies have been shown to provide effective means for capturing the
brain fingerprint, including electroencephalography (EEG), magnetic resonance imaging
(MRI), magnetoencephalography (MEG), and functional near-infrared spectroscopy
(FNIRS). EEG is the first neuroimaging technique adopted to generate brain fingerprints for
individual identification, among others. Since then, it has become the most popular tool and
has been widely studied due to its many good properties. EEG-based studies have shown that,
as bioelectrical features, brain fingerprints can be captured when the brain is working during
various neurocognitive activities. The resultant signals are highly differentiated among
individuals and stable within the same body under certain cognitive functions such as
response to visual stimuli or states such as resting state. That is, there are collectable, unique,
and persistent signal features in the brain that meet the basic conditions for biometric
identification.
Feature extraction method is an important part of brain fingerprint analysis, while different
neuroimaging technologies affect the selection of feature extraction methods to a certain
extent. It is divided into the feature-level extraction technology and end-to-end extraction
technology based on deep networks. The main categories of brain fingerprint extraction
methods based on different neuroimaging techniques, their advantages and disadvantages,
and the frequency with which they are used. Brain fingerprint identification is mainly used to
identify a subject’s ID from a group of subjects’ IDs. The researchers used a variety of
statistical analyzes, traditional ML, or deep learning methods to improve their classification
performance. The statistical analysis method analyzed the ID category by calculating the
correlation or distance measure between input features and target features. The K-nearest
neighbour and hidden Markov model are special statistical analysis methods. These methods
are simple, but they will be very time consuming when the data set is large. Most
identification systems based on fMRI-FC used the Pearson correlation coefficient as a
measure for identification. The selection of classifiers is an important factor affecting the
classification results. Although some studies have found that traditional ML and deep
learning methods have similar performance in classification accuracy, their application
situations are different.
CHAPTER 2
LITERATURE SURVEY
[1] The Semi-Supervised Information Extraction System from HTML
Description: The aim of this study is to propose an information extraction system, called
BigGrams, which is able to retrieve relevant and structural information (relevant phrases,
keywords) from semi-structural web pages, i.e. HTML documents. For this purpose, a novel
semi-supervised wrappers induction algorithm has been developed and embedded in the
BigGrams system. The wrappers induction algorithm utilizes a formal concept analysis to
induce information extraction patterns. It presents the impact of the configuration of the
information extraction system components on information extraction results and tests the
boosting mode of this system.
Benefits: Establish the good start point to explore IES and the proposed BigGrams system
through the theoretical and practical description of the above systems. Briefly describe the
novel WI algorithm with the use case and theoretical preliminaries. Find the best combination
of the elements mentioned above to achieve the best results of the WI algorithm.
machine learning methods. In addition, the open research issues pertinent to the big data
reduction are also highlighted.
Benefits: A thorough literature review and classification of big data reduction methods are
presented. Recently proposed schemes for big data reduction are analysed and synthesized. A
detailed gap analysis for the articulation of limitations and future research challenges for data
reduction in big data environments is presented.
Description: The massive growth in the scale of data has been observed in recent years
being a key factor of the Big Data scenario. Big Data can be defined as high volume, velocity
and variety of data that require a new high-performance processing. Addressing big data is a
challenging and time-demanding task that requires a large computational infrastructure to
ensure successful data processing and analysis. The presence of data pre-processing methods
for data mining in big data is reviewed in this paper. The definition, characteristics, and
categorization of data pre-processing approaches in big data are introduced. The connection
between big data and data pre-processing throughout all families of methods and big data
technologies are also examined, including a review of the state-of-the-art. In addition,
research challenges are discussed, with focus on developments on different big data
framework, such as Hadoop, Spark and Flink and the encouragement in devoting substantial
research efforts in some families of data pre-processing methods and applications on new big
data learning paradigms.
Benefits: It is worth mentioning that other emerging platform, such as Flink, are bridging the
gap of stream and batch processing that Spark currently has. Flink is a streaming engine that
can also do batches whereas Spark is a batch engine that emulates streaming by micro-
batches. This results in that Flink is more efficient in terms of low latency, especially when
dealing with real time analytical processing.
be gained. In addition, this information can improve the quality of healthcare offered to
patients. However, there are a number of issues that arise when dealing with these vast
quantities of data, especially how to analyse this data in a reliable manner. The basic goal of
Health Informatics is to take in real world medical data from all levels of human existence to
help advance our understanding of medicine and medical practice. This paper will present
recent research using Big Data tools and approaches for the analysis of Health Informatics
data gathered at multiple levels, including the molecular, tissue, patient, and population
levels. In addition to gathering data at multiple levels, multiple levels of questions are
addressed: human-scale biology, clinical-scale, and epidemic-scale. We will also analyse and
examine possible future work for each of these areas, as well as how combining data from
each level may provide the most promising approach to gain the most knowledge in Health
Informatics.
Benefits: Discussed a number of recent studies being done within the most popular sub
branches of Health Informatics, using Big Data from all accessible levels of human existence
to answer questions throughout all levels. Analysing Big Data of this scope has only been
possible extremely recently, due to the increasing capability of both computational resources
and the algorithms which take advantage of these resources.
Benefits: The main advance that WHISK makes is combining the strengths of systems that
learn text extraction rules for structured text with those that handle semi-structured text. The
following gives a brief overview of how WHISK compares to other IE learning systems.
These systems are described later in this section as well as related work in machine learning.
Benefits: An image indexing system that uses textual information in order to extract the
concept of the images that are found in a web page. The method uses visual cues in order to
identify the segments of the web page and calculates Euclidean distances among these
segments. It delivers a semantic or Euclidean clustering of the contents of a web page in
order to assign textual information to the existing images.
Benefits: Operators for the analysis of large collections of audio data have been presented in
a unifying framework. Some new operators have been developed, for instance those in the
phase space. Other operators have been generalized, for instance, the windowing and mark-
up operators. The operators are organized by method trees, which extract complex features.
All known feature extraction methods for audio data are covered, either directly as an
operator, or as the result of a method tree. Many different method trees (features) can be built
from the primitives of the framework. The method trees are automatically generated for a
certain classification task by a genetic programming approach.
Benefits: An AVSR system based on deep learning architectures for audio and visual feature
extraction and an MSHMM for multimodal feature integration and isolated word recognition.
Our experimental results demonstrated that, compared with the original MFCCs, the deep
denoising auto encoder can effectively filter out the effect of noise superimposed on original
clean audio inputs and that acquired denoised audio features attain significant noise
robustness in an isolated word recognition task.
demonstrate this through a variety of visually pleasing results for indoor and outdoor
scenes, including results from the feature film Charade.
Benefits: Demonstrated a fully automatic technique to estimate depths for videos. Our
method is applicable in cases where other methods fail, such as those based on motion
parallax and structure from motion, and works even for single images and dynamics scenes.
Our depth estimation technique is novel in that we use a non-parametric approach, which
gives qualitatively good results, and our single image algorithm quantitatively outperforms
existing methods.
Benefits: Presented a novel ConvLSTM-FCN model for the task of face mask extraction in
video sequences. We have illustrated how to convert a baseline-FCN model into ConvLSTM-
FCN model, which can learn from both temporal and spatial domains. A new loss function
named ‘segmentation loss’ has also been proposed for training the ConvLSTM-FCN model.
Last but not least, we also introduced the engineering trick of supplementing the primary
model with two zoomed-in models focusing on eyes and mouth. With all these are combined,
we have successfully improved the performances of baseline-FCN on 300VW-Mask dataset
from 54.50 to 63.76%, making a 16.99% relative improvement. The analysis of the
experimental results has verified the temporal-smoothing effects brought by the ConvLSTM-
FCN model.
CHAPTER 3
IMPLEMENTATION
The term NLP refers to the methods to interpret the data i.e. spoken or written by humans. In
order to process human languages using NLP, several tasks like machine translation,
question-answering system, information retrieval, information extraction and natural
language understanding are considered high-level tasks. The process of information
extraction (IE) is one of the important tasks in data analysis, KDD and data mining which
extract structured information from the unstructured data. IE is defined as “extract instances
of predefined categories from unstructured data, building a structured and unambiguous
representation of the entities and the relations between them”.
One of the intents of IE is to populate the knowledge bases to organize and access useful
information. It takes collection of documents as input and generates different representations
of relevant information satisfying different criteria. IE techniques efficiently analyse the text
in free form by extracting most valuable and relevant information in a structured format.
Hence, the ultimate goal of IE techniques is to identify the salient facts from the text to enrich
the databases or knowledge bases. The following subsections discuss the literature selected in
SLR process according to the IE subtasks for text data.
pre-characterized classes. Traditional NER systems were using Rule-Based Methods (RBM),
Learning-Based Methods (LBM) or hybrid approach. IE together with NLP plays a
significant role in language modelling and contextual IE using morphological, syntactic,
phonetic, and semantic analysis of languages. Rich morphological languages like Russian and
English make IE process easier. IE is difficult for morphologically poor languages because
these languages need extra effort for morphological rules to extract noun due to non-
availability of complete dictionary.
Question answering, machine translation, automatic text summarization, text mining
information retrieval; opinion mining and knowledgebase population are major applications
of NER. Hence, the higher efficiency and accuracy of these NER systems is very important
but big data brings new challenges to these systems i.e. volume, variety and velocity. In this
regard, this review investigates these challenges and explores the latest trends. It summarizes
techniques, motivation behind research, domain analysis, dataset used in the research and
evaluation of proposed solutions to identify the limitations of traditional techniques, impact
of big data on NER systems and latest trends. Evaluation of proposed techniques for IE is
performed using precision, recall and F1-score. Precision and recall are the measures for
completeness and correctness, respectively. F1-score measures the accuracy of the system
and harmonic combination of precision and recall.
It has been identified that text ambiguity, lack of resources, complex nested entities,
identification of contextual information, and noise in the form of homonyms, language
variability and missing data are important challenges in entity recognition from unstructured
big data. It is also found that the volume of unstructured big data changed the technological
paradigm from traditional rule-based or learning-based techniques to advanced techniques.
Variations of deep learning techniques such as CNN are performing better for these NER
systems.
(Subject, Predicate, and Object). The semantic triples extraction from images would benefit
various real-world applications such as content-based information retrieval, visual question
answering, and sentence to image retrieval and fine-grained recognition. Object classification
and detection and context or interaction recognition are main tasks of visual relationship
detection in image understanding.
in object detection. Whereas language prior and language structures are also improving
performance of relationship detection. CNN based VRD techniques extract features from
subject and object union box before classification. The training samples contain various same
predicate categories which can be used in different context with different entities. CNN based
models have the limitation to learn common features in same predicate category. So,
intraclass variance is a challenge for CNN based VRD. In order to overcome the limitation of
CNN models in VRD, visual appearance gap between same predicates and visual relationship
should be reduced. For this, Context and visual appearance features can be used to overcome
the identified limitation. Further, modified deep learning techniques are required to overcome
the challenges of visual relationship detection for large scale unstructured data. To the best of
our knowledge, the impact of volume, variety and velocity of big data is not addressed well in
visual relationship detection techniques.
The task to recognize similar faces is a computational challenge. It is evident that humans
have very strong face recognition abilities and these abilities are superior to known faces but
ability to recognize the unfamiliar faces are error-prone. This distinction of face recognition
in human lead towards the finding that face recognition depends on different set of facial
features for familiar and unfamiliar faces. These features are categorized into internal and
external features respectively. In this regard, examined the role of high-PS and low-PS
features in face recognition of familiar and unfamiliar faces and role of these critical features
for DNN based face recognition. The review concluded that high-PS features are critical for
human face recognition and are also used in DNN based trained on unconstrained faces.
In the domain of computer vision, face recognition is a holistic method that analyses the face
images. Various techniques have been proposed for face recognition for different datasets but
these traditional techniques are inadequate to deal with large scale datasets efficiently. A
comparative analysis shows that these traditional techniques have limitations to handle low-
quality large scale image datasets whereas deep learning methods are producing better results
for these datasets but with optimal architecture and hyper parameters. The face recognition in
low quality i.e. blur and low-resolution images degrade its performance. Sparse
representation and deep learning methods combined with handcrafted features outperformed
in case of low-resolution images. Face recognition techniques should be able to recognize
faces with different face expressions and poses in different lighting conditions. Various deep
learning based solutions are proposed to address the limitations of traditional techniques.
Deep CNN face recognition technique without extensive feature engineering reduces the
effort of most appropriate feature selection. Deep CNN face recognition technique was
evaluated on UJ face database of 50 images and the results have shown validation accuracy
of 22% goes to 80% after 10 epochs and 100% after 80 iterations. Certain limitations were
also associated with the solution such as over fitting and very small dataset. To reduce over
fitting, application of early stopping method will require extra effort. VGG-face architecture
and modified VGG-face architecture with 5 convolutional layers, 3 pooling layers, 3 fully
connected layers and softmax layer was evaluated using five different image datasets, i.e.
ORL face database with 400 images, yale face database with 165 images, extended yale-B
cropped face database with 2470 images, faces 94, Feret with 11,338 images and CVL face
db. For all datasets, the proposed approach performed better as compared to traditional
methods. Although, the proposed technique outperformed five different datasets but the
datasets were not complex and large-scale datasets. Deep learning based face recognition
techniques such as deep convolutional network or VGG-face and lightened CNN have
capability to handle huge amount of wild datasets.
Companies like call centers and music files are the major sources which generate a huge
volume of audio data. Different type of information can be extracted from this data to help
predictive and descriptive analytics. The subtasks of IE from audio data are classified as
acoustic event detection and automatic speech recognition.
3.3.1 Acoustic Event Detection
Sound event extraction or acoustic event extraction is an emerging field which aims to
process the continuous acoustic signals, convert them into the symbolic description. The
applications of automatic sound event detection are multimedia indexing and retrieval,
FFmpeg and OpenCV for video coder and image processing respectively were implemented
using MapReduce showing 75% scalability.
Generally, perceptual and semantic content can be extracted from videos. Semantic contents
deal with the objects and their relationship. The spatial and temporal association among
objects and entities have been used to reduce the semantic gap between visual appearance and
semantics with the help of fuzzy logic and RBM. The proposed system achieved high
precision but relatively low recall. Similarly, event extraction from audio-visual content
consisting of CNN based audio-visual multimodal recognition was developed and
incorporated knowledge from the website using HHMM was used to improve the efficiency.
The proposed approach outperformed in terms of accuracy and concluded that CNN provides
noise and occlusion robustness. The following subsections extensively discuss the issues and
state of the art techniques for subtasks of IE from video content.
subtitles, whereas scene text is normally embedded in the images such as sign boards,
trademarks, etc. Caption text or artificial text recognition is easier than scene text because
caption text is added over the video to improve the understand ability. Whereas, scene text
recognition is complex due to low contrast, background complexity, different font size,
orientation, type and language. Besides, low-quality video frames, blur frames and high
computation time are specific challenges related to video text extraction process.
The pipeline of text detection and extraction consists of text detection, text localization, text
tracking, text binarization and text recognition stages. Focusing on IE techniques, this review
presents only state of the art techniques for text recognition. Text recognition system to
extract semantic content from Arabic TV channel using CNN with auto encoder was
developed. The accuracy of character recognition was 94.6%. Moreover, a similar system for
Arabic News video was developed for video indexing using OCR engine ABBYY Fine
Reader with linguistic analysis and achieved 80.52% F-measure. Another text recognition
system was developed for overlay text extraction and person information extraction using
rule-based approach for NER to extract person, organization and location information. To
extract text, ABBYY Fine Reader was used. These text recognition systems deals with
printed and artificial text only that is comparatively easy to extract. On the other hand, text
binarization is important to segment natural scene text with filtering and iterative variance-
based threshold calculation. DNN has the ability to provide robust solution in end to end text
recognition in videos. In this regard, Faster R-CNN, CNN, LSTM based method have shown
comparatively better performance on scene text recognition. In general, temporal redundancy
can be used in tracking for text detection and recognition from complex videos.
dynamic video summarization. The second is key-frame based video summarization, static
video summarization, where frames and audio-visual features are extracted. Selecting the
most relevant or important frames or sub shots from the video for video summarization is a
critical task. Several supervised, unsupervised and other techniques are introduced in the
literature of computer vision and multimedia. Selection and prioritization criteria for frames
and skims are designed manually in unsupervised approach whereas supervised techniques
leverage user-generated summaries for learning. Each technique has different properties for
representativeness, diversity and interestingness. Recently, supervised techniques are
achieving promising results as compared to traditional unsupervised techniques.
CHAPTER 4
RESULTS AND DISCUSSION
Dept. of CSE, RRCE 2023-2024 Page 19
Brain Fingerprint Identification Based on Various Neuroimaging Technologies
This SLR distils the key insights from the comprehensive overview of IE techniques for a
variety of data types and takes a fresh look at older problems, which nevertheless are still
highly relevant today. Big data brings a computational paradigm shift to IE techniques. In this
regard, this SLR presents a comprehensive review of existing IE techniques for variety of
data types. To the best of our knowledge, IE techniques from variety of unstructured big data
at a single platform have not been addressed yet. In order to achieve this goal, SLR
methodology has been followed to explore the advancements in IE techniques in recent years.
To meet the objectives of the study, most relevant and up to date literature on IE techniques
for text, images, audio and video data have been discussed.
Big data value chain defines high-level activities that are important to find useful information
from big data where IE process is concerned with the data analysis. Therefore, the impact of
inefficiencies of IE techniques will ultimately decrease the performance of big data analytics
or decision making. In order to improve the big data analytics and decision making, this SLR
was aimed to investigate the challenges of IE process in the age of big data for variety of data
types. The objective of combining IE techniques for variety of data types at single platform
was twofold. First, to identify the state of the art IE techniques for variety of big data and
second, to investigate the major challenges of IE associated with unstructured big data.
Further, the need for new consolidated IE systems is highlighted and some preconditions are
also proposed to improve the IE process for the variety of data types in big data.
of cost, speed and accuracy. The main problem with unstructured big data is, huge deluge of
data is available, but it is not usable. Usability of data is defined as the capacity of data to
fulfil the requirements of user for a given purpose, area and epoch. According to the
definition of data usability, “Usability is the degree to which each stakeholder is able to
effectively access and use the data”. Data usability helps to know more about data, its
understanding and its usage. Therefore, usability varies due to the different interpretation of
meaning of data values and different nature of tasks that relates IE process improvement to
data usability improvement.
CONCLUSION
The systematic literature review serves the purpose of exploring state-of-the-art techniques
for IE from unstructured big data types such as text, image, audio and video investigating the
limitations of these techniques. Besides, the challenges of IE in big data environment have
also been identified. It is found that analysis and mining of data are getting more complex
with massive growth of unstructured big data. Deep learning with its generalizability,
adaptability and less human involvement capability is playing a key role in this regard.
However, to process exponentially growing data, new flexible and scalable techniques are
required to deal with the dynamicity and sparsity of unstructured data. Quality, usability and
sparsity of unstructured big data are major obstruct in deriving useful information. For
improving the IE techniques, mining useful information and supporting versatility of
unstructured data, it is required to introduce new techniques and make improvements and
enhancements in existing techniques.
Overall, the existing IE techniques are outperforming traditional techniques for
comparatively larger datasets but inadequate to effectively deal with rapid growth of
unstructured big data especially streaming data. Scalability, accuracy and latency are
important factors in implementation of these IE techniques in big data platform. Apache
MapReduce is also facing scalability issues in big data IE. To overcome these challenges,
MapReduce based deep learning solutions are the future of big data IE systems. These
systems will be helpful for health care analytics, surveillance, e-Government systems, social
media analytics and business analytics. The outcome of the study shows that highly scalable
and computationally efficient and consolidated IE techniques are required to deal with the
dynamicity of unstructured big data. The study significantly contributes to the identification
of the challenges to achieve more scalable and flexible IE systems. Quality, usability,
sparsity, dimensionality, heterogeneity, context and semantics understanding, scarcity,
modelling complexity and diversity of unstructured big data are major challenges in this field.
Advanced data preparation techniques, prior to extracting information from unstructured data,
semantically and contextually rich IE systems, the emergence of pragmatics and advanced IE
techniques are essential for IE systems in unstructured big data environment. Hence,
Scalable, computationally efficient and consolidated IE systems are required the can
overcome the challenges of multidimensional unstructured big data.
FUTURE WORK
The major focus of the review was to investigate the challenges of IE systems for
multidimensional unstructured big data. The detailed discussion on IE techniques from
variety of data types concluded that data preparation is equally important to the efficiency of
IE systems. Advanced data improvement techniques will also increase the efficiency of IE
systems. Therefore, the findings of the review will be used to develop a usability
improvement model for unstructured big data to extract maximum useful information from
these data.
REFERENCES
1. Wang K, Shi Y. User information extraction in big data environment. In: 3rd IEEE
international conference on computer and communications (ICCC).
2. Li P, Mao K. Knowledge-oriented convolutional neural network for causal relation
extraction from natural language texts. Expert System Appl. 2019;115:512–23.
3. Liu Z, Tong J, Gu J, Liu K, Hu B. A Semi-automated entity relation extraction
mechanism with weakly supervise learning for Chinese medical webpages. In:
International conference on smart health. Cham: Springer; 2016; p 44–56.
4. KHe K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In:
Proceedings of the IEEE conference on computer vision and pattern recognition
(CVPR). 2016; p. 770–8.
5. Gantz J, Reinsel D. The digital universe in 2020: big data, bigger digital shadows, and
biggest growth in the far east. IDC iView IDC Analyze Future. 2012;2007(2012):1–
16.
6. Wang Y, Kung LA, Byrd TA. Big data analytics: understanding its capabilities and
potential benefits for healthcare organizations. Technol Forecast Soc Change.
2018;126:3–13.
7. Lomotey RK, Deters R. Topics and terms mining in unstructured data stores. In: 2013
IEEE 16th international conference on computational science and engineering, 2013.
p. 854–61.
8. Lomotey RK, Deters R. RSenter: terms mining tool from unstructured data sources.
Int J Bus Process Integr Manag, 2013;6(4):298.
9. Goldberg S, Wang DZ, Grant C. A probabilistically integrated system for crowd-
assisted text labeling and extraction. J Data Inf Qual. 2017;8(2):1–23.
10. Napoli C, Tramontana E, Verga G. Extracting location names from unstructured
italian texts using grammar rules and MapReduce. In: International conference on
information and software technologies. Cham: Springer; 2016; p.593–601.