Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
48 views

Chapter No. Title Page No

This document outlines research on multilingual native speaker recognition. It begins with an introduction and literature review on corpora collection techniques, feature extraction methods, and modeling techniques. The objectives are then stated as collecting multilingual corpora over mobile networks and performing native speaker recognition using both handcrafted and novel deep learning features. Materials and methods are described for data collection, traditional and deep learning feature extraction, and classification models. Results are provided on native speaker recognition accuracy using different features and models. The discussion analyzes the results and methods, with conclusions and proposals for future work.

Uploaded by

ANANYA
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
48 views

Chapter No. Title Page No

This document outlines research on multilingual native speaker recognition. It begins with an introduction and literature review on corpora collection techniques, feature extraction methods, and modeling techniques. The objectives are then stated as collecting multilingual corpora over mobile networks and performing native speaker recognition using both handcrafted and novel deep learning features. Materials and methods are described for data collection, traditional and deep learning feature extraction, and classification models. Results are provided on native speaker recognition accuracy using different features and models. The discussion analyzes the results and methods, with conclusions and proposals for future work.

Uploaded by

ANANYA
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 9

CONTENTS

CHAPTER NO. TITLE PAGE NO.

1 INTRODUCTION 1

2 REVIEW OF LITERATURE 9

2.1 CORPORA AND THEIR COLLECTION

TECHNIQUES 9

2.1.1 MULTILINGUAL CORPORA 10

2.1.2 WIRE/WIRELESS CORPORA 12

2.2.3 NOISY/CLEAN CORPORA 14

2.2 FEATURE EXTRACTION 15

2.3.1 HANDCRAFT FEATURE EXTRACTION 15

2.3.2 MODERN CONVOLUTION NEURAL

NETWORK FEATURES 18

2.3 NATIVE SPEAKER RECOGNITION 19

2.4 TRAINING/TESTING AND MODELING

TECHNIQUES 20

2.5 LACUNA IDENTIFIED 26

3 OBJECTIVES 28

3.1 OBJECTIVES 28
4 MATERIALS AND METHODS 30

4.1 MULTILINGUAL CORPORA COLLECTION

OVER THE MOBILE 31

4.2 HANDCRAFT FEATURE EXTRACTION 32

4.2.1 MFCC FEATURES 32

4.2.2 LPC FEATURES 32

4.2.3 PROSODIC FEATURES 32

4.2.4 SYSTEM INFORMATION 32

4.3 HANDCRAFT FEATURE EXTRACTION IN FIRST

PHASE 44

4.4 MNRS WITH NOISY HANDCRAFT FEATURES BY

MLFFNN 46

4.4.1 MLFFNN 46

4.4.2 EXPERIMENTAL SETUP FOR MNRS USING

MLFFNN 51

4.4.2.1 FIRST STAGE MNRS USING

MLFFNN 52

4.4.2.2 SECOND STAGE MNRS USING

MLFFNN 56
4.4.2.3 THIRD STAGE MNRS WITH

MLFFNN 56

4.5 NOVEL MDCNN FEATURE EXTRACTION 57

4.5.1 NOVEL DCNN FEATURES 58

4.5.1.1 CONVOLUTION IN SPEECH 59

4.5.1.2 BACK PROPAGATION IN CNN 59

4.5.1.3 LOCAL WEIGHT SHARING IN CNN 61

4.5.1.3.1 LOCAL FEATURE

EXTRACTION BY CNN 62

4.5.1.4 CNN ARCHITECTURE 63

4.5.1.4.1 LAYERS USED TO BUILD

DCNN ARCHITECTURE 63

4.5.1.4.1.1 CONVOLUTION

LAYERS 64

4.5.1.4.1.2 RELU 64

4.5.1.4.1.3 BATCH

NORMALIZATION 65
4.5.1.4.1.4 SUB-SAMPLING/

MAX POOLING OF

CONV FEATURES 65

4.5.1.4.1.5 FULLY CONNECTED

LAYERS 66

4.5.1.4.1.6 DROPOUT LAYER 66

4.5.2 MDCNN FEATURE EXTRACTION 67

4.5.2.1 INPUT DATA TO THE CONV LAYER 68

4.5.2.2 ARCHITECTURE OF DCNN

PRE-TRAINED MODELS 69

4.5.2.3 EXPERIMENTAL SETUP FOR

MDCNN FEATURE EXTRACTION 73

4.6 MNRS WITH MDCNN FEATURES 79

4.6.1 LDC 81

4.6.1.1 REPRESENTATION OF LDC MODELS 82

4.6.1.1.2 LEARNING LDC MODELS 83

4.6.1.1.3 PREDICTIONS WITH LDC 84


4.6.2 K-NN 84

4.6.2.1 CLASSIFICATION 85

4.6.2.2 REGRESSION 85

4.6.2.3 TECHNICAL DETAILS 86

4.6.2.4 CROSS-VALIDATION 86

4.6.2.5 K-NEAREST NEIGHBOR PREDICTIONS 87

4.6.2.6 DISTANCE WEIGHTING 87

4.6.3 DECISION TREES 88

4.6.4 BT: TREE BAGGER 91

4.6.5 MDCNN FEATURES BASED

MNRS ARCHITECTURE 92

4.6.6 EXPERIMENTAL SETUP FOR MNRS WITH

MDCNN FEATURES USING M/C

LEARNING SHALLOW CLASSIFIERS 93

4.6.6.2 INPUT DATA (AMDCNN FEATURES)

PREPARATION FOR MNRS 94

4.6.6.3 CHOOSE AN MACHINE LEARNING

ALGORITHM FOR MNSR 94

4.6.6.4 FIT A SHALLOW CLASSIFICATION


MODEL 94

4.6.6.5 CHOOSE A CROSS VALIDATION

METHOD TO VALIDATE

MULTILINGUAL NATIVITY 95

4.6.6.6 CONFUSION MATRIX GENERATION 97

5 RESULT 99

5.1 MNRS WITH NOISY HANDCRAFT FEATURES

USING MLFFNN 99

5.1.1 FIRST STAGE MNRS WITH

MFCC FEATURES 99

5.1.2 SECOND STAGE MNRS ENHANCEMENT

BY FUSING THE FEATURES 99

5.1.2.1 MNRS ACCURACY WITH NOISY

FUSED FEATURES AND MFCC ALONE 101

5.1.2.2 MNRS ERROR WITH NOISY FUSED

FEATURES AND MFCC ALONE 102

5.1.3 THIRD STAGE MNRS WITH MLFFNN 102


5.2 NOVEL EFFICIENT MDCNN FEATURE

EXTRACTION 104

5.2.1 CPU AND GPU TIME TO

EXTRACT MDCNN FEATURES 104

5.3 EFFICIENT MNRS WITH MDCNN FEATURES 108

5.3.1 MNRS WITH MDCNN FEATURES BY LDC 109

5.3.2 IMPROVED MNRS ACCURACY WITH

EFFICIENT AMDCNN FEATURES

BY OTHER CLASSIFIERS 113

5.3.2.1 INDIVIDUAL NATIVITY RECOGNITION

BY ALL CLASSIFIERS 115

5.4.2.2 CONFUSION MATRIX OF KNN 116

6 DISCUSSION 117

6.1 MULTILINGUAL CRPORA FOR MNRS 117

6.2 MNRS WITH NOISY HANDCRAFT

FEATURES USING MLFFNN 117

6.2.1 FIRST STAGE OF MNRS


WITH NOISY MFCC FEATURES 117

6.2.2 SECOND STAGE MNRS WITH FUSED

NOISY HANDCRAFT FEATURES 118

6.2.3 THIRD STAGE OF MNRS WITH

OVERLEARNING 119

6.3 NOVEL EFFICIENT MDCNN FEATURE

EXTRACTION 120

6.3.1 GPU AND CPU TIME FOR EFFICIENT

MDCNN FEATURE EXTRACTION 121

6.4 MNRS WITH THE MDCNN FEATURES USING LDC 122

6.5 MNRS WITH EFFICIENT AMDCNN FEATURES

USING OTHER CLASSIFIERS 123

6.6 INDIVIDUAL NATIVITY RECOGNITION

BY ALL CLASSIFIERS 125

7 SUMMARY 127

8 FUTURE WORK 129

9 BIBILIOGRAPHY 131
ANNEXURE I-A ABSTRACT 156

ANNEXURE I-B LIST OF TABLES 158

ANNEXURE I-C LIST OF FIGURES 160

ANNEXURE I-D LIST OF ABBRIVIATIONS 163

ANNEXURE II LIST OF PUBLICATIONS 166

You might also like