Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Pattern recognition

•Download as PPT, PDF•
•
Armando Vieira
Armando Vieira

The document discusses artificial intelligence and pattern recognition. It introduces various pattern recognition concepts including defining a pattern, examples of patterns in different domains, and approaches to pattern recognition. It also provides an example of using discriminative methods to classify fish into salmon and sea bass using optical sensing and extracted features.

1 of 53
Artificial Intelligence & Pattern Recognition Introduction 2007/2008 Armando Vieira & Bernardete Ribeiro CPR 2007-2008
Program • Introduction to Artificial Intelligence • Soft Artificial Intelligence • Artificial Neural Networks: theory, training, applications • Supervised Learning: Mulilayer Perceptron • Unsupervised Learning: Self-Organized Kohonen Maps • Genetic Algorithms • Applications • Project CPR 2007-2008
Pattern Recognition? “The assignment of a physical object or event to one of several pre-specified categories” -- Duda & Hart • A pattern is an object, process or event • A class (or category) is a set of patterns that share common attribute (features) usually from the same information source • During recognition (or classification) classes are assigned to the objects. • A classifier is a machine that performs such task CPR 2007-2008
What is a pattern? “A pattern is the opposite of a chaos; it is an entity vaguely defined, that could be given a name.” CPR 2007-2008
Examples of Patterns Cristal Patterns: atómic or molecular Their structures are represented by 3D graphs and can be described by deterministic grammars or formal languages CPR 2007-2008
Examples of Patterns Patterns of Constellations Patterns of constellations are represented by 2D planar graphs Human perception has strong tendency to find patterns from anything. We see patterns from even random noise --- we are more likely to believe a hidden pattern than denying it when the risk (reward) for missing (discovering) a pattern is often high. CPR 2007-2008
Examples of Patterns Biological Patterns ---morphology Landmarks are identified from biologic forms and these patterns are then represented by a list of points. But for other forms, like the root of plants, Points cannot be registered crossing instances. Applications: Biometrics, computacional anatomy, brain mapping, … CPR 2007-2008
Examples of Patterns Biological Patterns Landmarks are identified from biologic forms and these patterns are then represented by a list of points. CPR 2007-2008
Examples of Patterns Music Patterns Ravel Symphony? CPR 2007-2008
Examples of Patterns People Recognition Patterns Behavior ? Funny, Funny CPR 2007-2008
Examples of Patterns Discovery and Association of Patterns Statistics show connections between the shape of one’s face (adults) and his/her Character. There is also evidence that the outline of children’s face is related to alcohol abuse during pregnancy. CPR 2007-2008
Examples of Patterns Discovery and Association of Patterns What are the features? Statistics show connections between the shape of one’s face (adults) and his/her Character. CPR 2007-2008
Examples of Patterns Patterns of Brain Activity We may understand patterns of brain activity and find relationships between brain activities, cognition, and behaviors CPR 2007-2008
Examples of Patterns Variation Patterns: 1. Expression – geometric deformation 2. illumination--- Photometric deformation 3. Transformation –3D pose 3D 4. Noise and Occlusion CPR 2007-2008
Examples of Patterns A broad range of texture patterns are generated by stochastic processes. CPR 2007-2008
Examples of Patterns . How are these patterns represented in human mind? CPR 2007-2008
Examples of Patterns Speech signals and Hidden Markov models CPR 2007-2008
Examples of Patterns Natural Language and stochastic grammar. . CPR 2007-2008
Examples of Patterns Object Recognition Patterns everywhere ? CPR 2007-2008
Examples of Patterns Maps Recognition Patterns of Global Warming ? CPR 2007-2008
Examples of Patterns Finacial Series Pattern Recognition CPR 2007-2008
Examples of Patterns How to Trade Chart Patterns ? CPR 2007-2008
Examples of Patterns Pattern Recognition in Medical Diagnosis projecções Tomografia Tomografia p(r,θ,z) Tomos (=corte) + grafos (=escrita, imagem, gráfico) cção proje cção proje f(x,y,z) r etro p(r,θ,z) f(x,y,z) Reconstrução CPR 2007-2008
Examples of Patterns Optical Character Recognition CPR 2007-2008
Graphic Arts Examples of Patterns Escher, who else? CPR 2007-2008
Examples of Patterns Human Genome Beautiful Patterns! CPR 2007-2008
Examples of Applications • Handwritten: sorting letters by postal code, input device for PDA‘s. • Printed texts: reading machines for blind • Optical Character people, digitalization of text documents. Recognition (OCR) • Face recognition, verification, retrieval. • Finger prints recognition. • Speech recognition. • Biometrics • Medical diagnosis: X-Ray, EKG analysis. • Diagnostic systems • Machine diagnostics, waster detection. • Automated Target Recognition (ATR). • Military applications • Image segmentation and analysis (recognition from aerial or satelite photographs). CPR 2007-2008
Approaches • Statistical PR: based on underlying statistical model of patterns and pattern classes. • Neural networks: classifier is represented as a network of cells modeling neurons of the human brain (connectionist approach). • Structural (or syntactic) PR: pattern classes represented by means of formal structures as grammars, automata, strings, etc. CPR 2007-2008
An example of Pattern Recognition Classification of fish into two classes: salmon and Sea Bass by discriminative method •“Sorting incoming Fish on a conveyor according to species using optical sensing” CPR 2007-2008
Problem Analysis – Set up a camera and take some sample images to extract features • Length • Lightness • Width • Number and shape of fins • Position of the mouth, etc… This is the set of all suggested features to explore for use in our classifier! CPR 2007-2008
Pattern Recognition Phases • Preprocess raw data from camera • Segment isolated fish • Extract features from each fish (length,width, brightness, etc.) • Classify each fish CPR 2007-2008
Pattern Recognition Phases • Preprocessing – Use a segmentation operation to isolate fishes from one another and from the background • Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features • The features are passed to a classifier CPR 2007-2008
• Classification Select the length of the fish as a possible feature for discrimination CPR 2007-2008
Features and Distributions The length is a poor feature alone! Select the lightness as a possible feature. CPR 2007-2008
“Customers do not want sea bass in their cans of salmon” • Threshold decision boundary and cost relationship • Move our decision boundary toward smaller values of lightness in order to minimize the cost (reduce the number of sea bass that are classified salmon!) Task of decision theory CPR 2007-2008
• Adopt the lightness and add the width of the fish Fish x = [x1, x2] Lightness Width CPR 2007-2008
Decision/classification Boundaries ? CPR 2007-2008
CPR 2007-2008
• We might add other features that are not correlated with the ones we already have. A precaution should be taken not to reduce the performance by adding such “noisy features” • Ideally, the best decision boundary should be the one which provides an optimal performance such as in the following figure: CPR 2007-2008
CPR 2007-2008
• However, our satisfaction is premature because the central aim of designing a classifier is to correctly classify novel input Issue of generalization! CPR 2007-2008
CPR 2007-2008
Occam’s Razor Entities are not to be multiplied without necessity William of Occam (1284-1347) CPR 2007-2008
A Complete PR System CPR 2007-2008
Problem Formulation Measurements Features Classification Input Preprocessing Class object Label Basic ingredients: •Measurement space (e.g., image intensity, pressure) •Features (e.g., corners, spectral energy) •Classifier - soft and hard •Decision boundary •Training sample •Probability of error CPR 2007-2008
Design Cycle 1. Feature selection and extraction --- What are good discriminative features? 2. Modeling and learning 3. Dimension reduction, model complexity 4. Decisions and risks 5. Error analysis and validation. 6. Performance bounds and capacity. 7. Algorithms CPR 2007-2008
CPR 2007-2008
• Data Collection How do we know when we have collected an adequately large and representative set of examples for training and testing the system? CPR 2007-2008
• Feature Choice Depends on the characteristics of the problem domain. Simple to extract, invariant to irrelevant transformation, insensitive to noise. CPR 2007-2008
• Model Choice Unsatisfied with the performance of our linear fish classifier and want to jump to another class of model CPR 2007-2008
• Training Use data to determine the classifier. Many different procedures for training classifiers and choosing models CPR 2007-2008
• Evaluation Measure the error rate (or performance) and switch from one set of features & models to another one. CPR 2007-2008
• Computational Complexity What is the trade off between computational ease and performance? (How an algorithm scales as a function of the number of features, number or training examples, number patterns or categories?) CPR 2007-2008

More Related Content

Pattern recognition

  • 1. Artificial Intelligence & Pattern Recognition Introduction 2007/2008 Armando Vieira & Bernardete Ribeiro CPR 2007-2008
  • 2. Program • Introduction to Artificial Intelligence • Soft Artificial Intelligence • Artificial Neural Networks: theory, training, applications • Supervised Learning: Mulilayer Perceptron • Unsupervised Learning: Self-Organized Kohonen Maps • Genetic Algorithms • Applications • Project CPR 2007-2008
  • 3. Pattern Recognition? “The assignment of a physical object or event to one of several pre-specified categories” -- Duda & Hart • A pattern is an object, process or event • A class (or category) is a set of patterns that share common attribute (features) usually from the same information source • During recognition (or classification) classes are assigned to the objects. • A classifier is a machine that performs such task CPR 2007-2008
  • 4. What is a pattern? “A pattern is the opposite of a chaos; it is an entity vaguely defined, that could be given a name.” CPR 2007-2008
  • 5. Examples of Patterns Cristal Patterns: atómic or molecular Their structures are represented by 3D graphs and can be described by deterministic grammars or formal languages CPR 2007-2008
  • 6. Examples of Patterns Patterns of Constellations Patterns of constellations are represented by 2D planar graphs Human perception has strong tendency to find patterns from anything. We see patterns from even random noise --- we are more likely to believe a hidden pattern than denying it when the risk (reward) for missing (discovering) a pattern is often high. CPR 2007-2008
  • 7. Examples of Patterns Biological Patterns ---morphology Landmarks are identified from biologic forms and these patterns are then represented by a list of points. But for other forms, like the root of plants, Points cannot be registered crossing instances. Applications: Biometrics, computacional anatomy, brain mapping, … CPR 2007-2008
  • 8. Examples of Patterns Biological Patterns Landmarks are identified from biologic forms and these patterns are then represented by a list of points. CPR 2007-2008
  • 9. Examples of Patterns Music Patterns Ravel Symphony? CPR 2007-2008
  • 10. Examples of Patterns People Recognition Patterns Behavior ? Funny, Funny CPR 2007-2008
  • 11. Examples of Patterns Discovery and Association of Patterns Statistics show connections between the shape of one’s face (adults) and his/her Character. There is also evidence that the outline of children’s face is related to alcohol abuse during pregnancy. CPR 2007-2008
  • 12. Examples of Patterns Discovery and Association of Patterns What are the features? Statistics show connections between the shape of one’s face (adults) and his/her Character. CPR 2007-2008
  • 13. Examples of Patterns Patterns of Brain Activity We may understand patterns of brain activity and find relationships between brain activities, cognition, and behaviors CPR 2007-2008
  • 14. Examples of Patterns Variation Patterns: 1. Expression – geometric deformation 2. illumination--- Photometric deformation 3. Transformation –3D pose 3D 4. Noise and Occlusion CPR 2007-2008
  • 15. Examples of Patterns A broad range of texture patterns are generated by stochastic processes. CPR 2007-2008
  • 16. Examples of Patterns . How are these patterns represented in human mind? CPR 2007-2008
  • 17. Examples of Patterns Speech signals and Hidden Markov models CPR 2007-2008
  • 18. Examples of Patterns Natural Language and stochastic grammar. . CPR 2007-2008
  • 19. Examples of Patterns Object Recognition Patterns everywhere ? CPR 2007-2008
  • 20. Examples of Patterns Maps Recognition Patterns of Global Warming ? CPR 2007-2008
  • 21. Examples of Patterns Finacial Series Pattern Recognition CPR 2007-2008
  • 22. Examples of Patterns How to Trade Chart Patterns ? CPR 2007-2008
  • 23. Examples of Patterns Pattern Recognition in Medical Diagnosis projecções Tomografia Tomografia p(r,θ,z) Tomos (=corte) + grafos (=escrita, imagem, gráfico) cção proje cção proje f(x,y,z) r etro p(r,θ,z) f(x,y,z) Reconstrução CPR 2007-2008
  • 24. Examples of Patterns Optical Character Recognition CPR 2007-2008
  • 25. Graphic Arts Examples of Patterns Escher, who else? CPR 2007-2008
  • 26. Examples of Patterns Human Genome Beautiful Patterns! CPR 2007-2008
  • 27. Examples of Applications • Handwritten: sorting letters by postal code, input device for PDA‘s. • Printed texts: reading machines for blind • Optical Character people, digitalization of text documents. Recognition (OCR) • Face recognition, verification, retrieval. • Finger prints recognition. • Speech recognition. • Biometrics • Medical diagnosis: X-Ray, EKG analysis. • Diagnostic systems • Machine diagnostics, waster detection. • Automated Target Recognition (ATR). • Military applications • Image segmentation and analysis (recognition from aerial or satelite photographs). CPR 2007-2008
  • 28. Approaches • Statistical PR: based on underlying statistical model of patterns and pattern classes. • Neural networks: classifier is represented as a network of cells modeling neurons of the human brain (connectionist approach). • Structural (or syntactic) PR: pattern classes represented by means of formal structures as grammars, automata, strings, etc. CPR 2007-2008
  • 29. An example of Pattern Recognition Classification of fish into two classes: salmon and Sea Bass by discriminative method •“Sorting incoming Fish on a conveyor according to species using optical sensing” CPR 2007-2008
  • 30. Problem Analysis – Set up a camera and take some sample images to extract features • Length • Lightness • Width • Number and shape of fins • Position of the mouth, etc… This is the set of all suggested features to explore for use in our classifier! CPR 2007-2008
  • 31. Pattern Recognition Phases • Preprocess raw data from camera • Segment isolated fish • Extract features from each fish (length,width, brightness, etc.) • Classify each fish CPR 2007-2008
  • 32. Pattern Recognition Phases • Preprocessing – Use a segmentation operation to isolate fishes from one another and from the background • Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features • The features are passed to a classifier CPR 2007-2008
  • 33. • Classification Select the length of the fish as a possible feature for discrimination CPR 2007-2008
  • 34. Features and Distributions The length is a poor feature alone! Select the lightness as a possible feature. CPR 2007-2008
  • 35. “Customers do not want sea bass in their cans of salmon” • Threshold decision boundary and cost relationship • Move our decision boundary toward smaller values of lightness in order to minimize the cost (reduce the number of sea bass that are classified salmon!) Task of decision theory CPR 2007-2008
  • 36. • Adopt the lightness and add the width of the fish Fish x = [x1, x2] Lightness Width CPR 2007-2008
  • 39. • We might add other features that are not correlated with the ones we already have. A precaution should be taken not to reduce the performance by adding such “noisy features” • Ideally, the best decision boundary should be the one which provides an optimal performance such as in the following figure: CPR 2007-2008
  • 41. • However, our satisfaction is premature because the central aim of designing a classifier is to correctly classify novel input Issue of generalization! CPR 2007-2008
  • 43. Occam’s Razor Entities are not to be multiplied without necessity William of Occam (1284-1347) CPR 2007-2008
  • 44. A Complete PR System CPR 2007-2008
  • 45. Problem Formulation Measurements Features Classification Input Preprocessing Class object Label Basic ingredients: •Measurement space (e.g., image intensity, pressure) •Features (e.g., corners, spectral energy) •Classifier - soft and hard •Decision boundary •Training sample •Probability of error CPR 2007-2008
  • 46. Design Cycle 1. Feature selection and extraction --- What are good discriminative features? 2. Modeling and learning 3. Dimension reduction, model complexity 4. Decisions and risks 5. Error analysis and validation. 6. Performance bounds and capacity. 7. Algorithms CPR 2007-2008
  • 48. • Data Collection How do we know when we have collected an adequately large and representative set of examples for training and testing the system? CPR 2007-2008
  • 49. • Feature Choice Depends on the characteristics of the problem domain. Simple to extract, invariant to irrelevant transformation, insensitive to noise. CPR 2007-2008
  • 50. • Model Choice Unsatisfied with the performance of our linear fish classifier and want to jump to another class of model CPR 2007-2008
  • 51. • Training Use data to determine the classifier. Many different procedures for training classifiers and choosing models CPR 2007-2008
  • 52. • Evaluation Measure the error rate (or performance) and switch from one set of features & models to another one. CPR 2007-2008
  • 53. • Computational Complexity What is the trade off between computational ease and performance? (How an algorithm scales as a function of the number of features, number or training examples, number patterns or categories?) CPR 2007-2008