Lazy Learning

This paper surveys locally weighted learning, a form of lazy learning and memory-based learning, and focuses on locally weighted linear regression. The survey discusses distance functions, smoothing parameters, weighting functions, local model structures, regularization of the estimates and bias, assessing predictions, handling noisy data and outliers, improving the quality of predictions by tuning fit parameters, interference between old and new data, implementing locally weighted learning efficiently, and applications of locally weighted ...

A typical centrifugal impeller characterised by a low flow coefficient and cylindrical blades is optimised by means of an intelligent automatic search program. The procedure consists of a Feasible Sequential Quadratic Programming (FSQP) algorithm [6] coupled to a Lazy Learning (LL) interpolator [1] to speed-up the process. The program is able to handle geometrical constraints to reduce the computational effort devoted to the analysis of non-physical configurations. The objective function evaluator is an in-house developed structured CFD code. The LL approximator is called each time the stored database can provide a sufficiently accurate performance estimate for a given geometry, thus reducing the effective CFD computations. The impeller is represented by 25 geometrical parameters describing the vane in the meridional and s-q planes, the blade thickness and the leading edge shape. The optimisation is carried out on the impeller design point maximising the polytropic efficiency with m...

The traditional approach to supervised learning is global modeling which describes the relationship between the input and the output with an analytical function over the whole input domain. What makes global modeling appealing is the nice property that even for huge datasets, a parametric model can be stored in a small memory. Also, the evaluation of the parametric model requires a short program that can be executed in a reduced amount of time. Nevertheless, modeling complex input/output relations often requires the adoption of ...

Accurate prediction of time series over long future horizons is the new frontier of forecasting. Conventional approaches to long-term time series forecasting rely either on iterated one-step-ahead predictors or direct predictors.In spite of their diversity, iterated and direct techniques for multi-step-ahead forecasting share a common feature, i.e. they model from data a multiple-input single-output mapping. In previous works, the authors presented an original multiple-output approach to multi-step-ahead prediction. The goal is to improve accuracy by preserving in the forecasted sequence the stochastic properties of the training series. This is not guaranteed for instance in direct approaches where predictions for different horizons are performed independently.This paper presents a review of single-output vs. multiple-output approaches for prediction and goes a step forward with respect to the previous authors contributions by (i) extending the multiple-output approach with a query-based criterion and (ii) presenting an assessment of single-output and multiple-output methods on the NN3 competition datasets.In particular, the experimental section shows that multiple-output approaches represent a competitive choice for tackling long-term forecasting tasks.

Accurate prediction of time series over long future horizons is the new frontier of forecasting. Conventional approaches to long-term time series forecasting rely either on iterated one-step- ahead predictors or direct predictors. In spite of their diversity, iterated and direct techniques for multi-step-ahead forecasting share a common feature, i.e. they model from data a multiple-input single-output mapping. In previous works,

Lazy learning methods provide useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of complex systems. This paper surveys ways in which locally weighted learning, a type of lazy learning, has been applied by us to control tasks. We explain various forms that control tasks can take, and how this affects the choice of learning paradigm. The discussion section explores the interesting impact that explicitly remembering all previous experiences has on the problem of ...

A real-time pattern recognition algorithm based on k-nearest neighbors and lazy learning was used to classify, voluntary electromyography (EMG) signals and to simultaneously control movements of a dexterous artificial hand. EMG signals were superficially recorded by eight pairs of electrodes from the stumps of five transradial amputees and forearms of five able-bodied participants and used online to control a robot hand. Seven finger movements (not involving the wrist) were investigated in this study. The first objective was to understand whether and to which extent it is possible to control continuously and in real-time, the finger postures of a prosthetic hand, using superficial EMG, and a practical classifier, also taking advantage of the direct visual feedback of the moving hand. The second objective was to calculate statistical differences in the performance between participants and groups, thereby assessing the general applicability of the proposed method. The average accuracy of the classifier was 79% for amputees and 89% for able-bodied participants. Statistical analysis of the data revealed a difference in control accuracy based on the aetiology of amputation, type of prostheses regularly used and also between able-bodied participants and amputees. These results are encouraging for the development of noninvasive EMG interfaces for the control of dexterous prostheses.

Lazy learning methods provide useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of complex systems. This paper surveys ways in which locally weighted learning, a type of lazy learning, has been applied by us to control tasks. We explain various forms that control tasks can take, and how this affects the choice of learning paradigm. The discussion section explores the interesting impact that explicitly remembering all previous experiences has on the problem of ...

In this paper we propose a recursive method for identifying and crossvalidating local constant models. The algorithm we derive here is intended to be a part of a more general lazy learning method already presented by the authors (Birattari et al., 1999). We take for granted aspects related to the search of the nearest-neighbors, the de nition of a metric and local combination of estimators, and we focus our attention on the derivation of an e cient way to obtain and assess a sequence of local constant models centered on a ...

DI-fusion, le Dépôt institutionnel numérique de l'ULB, est l'outil de référencementde la production scientifique de l'ULB.L'interface de recherche DI-fusion permet de consulter les publications des chercheurs de l'ULB et les thèses qui y ont été défendues.

This paper presents a local method for modeling and control of non-linear dynamical systems from input-output data. The proposed methodology couples a local model identification inspired by the lazy learning technique, with a control strategy based on linear optimal control theory. The local modeling procedure uses a query-based approach to select the best model configuration by assessing and comparing different alternatives. A new recursive technique for local model identification and validation is presented, together with ...

Case-based reasoning (CBR) provides an adequate framework to cope with continuous domains, where a great amount of new valuable experiences are generated in a non-stop way. CBR systems become more competent in their evolution over time by means of learning new relevant experiences. There are two central problems derived from the continuous nature of some domains: the fast growing size of the case library and the overhead in the case library organisation. Our proposal to overcome these two problems ...

In this paper we propose a recursive method for identifying and crossvalidatinglocal constant models. The algorithm we derive here is intendedto be a part of a more general lazy learning method already presented bythe authors (Birattari et al., 1999). We take for granted aspects relatedto the search of the nearest-neighbors, the definition of a metric and localcombination of estimators, and

Abstract: Morphological analysis is an important subtask in text-to-speech conversion, hyphenation, and other language engineering tasks. The traditional approach to performing morphological analysis is to combine a morpheme lexicon, sets of (linguistic) rules, and heuristics to find a most probable analysis. In contrast we present an inductive learning approach in which morphological analysis is reformulated as a segmentation task. We report on a number of experiments in which five inductive learning algorithms are applied to ...

System-level designers typically rely on white-box detailed descriptions of embedded systems in order to perform their design choices and optimisations. The white-box approach assumes that the complexity of the system can be managed by decomposing the global system into a set of simple and well-described components. This assumption is generally not true when non-linear interactions happen between the different subsystems. This paper explores, as an alternative, a black-box modelling methodology based on intelligent data analysis in order to support system-level designers in characterising the performance of embedded applications. The rationale behind this approach is that nowadays the designer can easily store large amount of data related to the system, and consequently discover relevant information by analysing them in a black-box fashion. As a case study, we address the performance modelling of software applications running on an embedded microprocessor. We introduce a data analysis method which, on the basis of a high-level characterisation of the software functionality and the hardware architecture, is able to predict the number of execution cycles on a embedded processor. We propose the adoption of a local learning technique (lazy learning), which proved already to be effective in previous works, to model the unknown input/output relation between the hardware/software parameters of the application and the number of execution cycles. Experiments with standard computational code (sorting, mathematical computation) and with a specific streaming algorithm for MPEG variable length decoding are presented to support this claim.

Usually, training data are not evenly distributed in the input space. This makes non-local methods, like Neural Networks, not very accurate in those cases. On the other hand, local methods have the problem of how to know which are the best examples for each test pattern. In this work, we present a way of performing a trade off between local and non-local methods. On one hand a Radial Basis Neural Network is used like learning algorithm, on the other hand a selection of the training patterns is used for each query. Moreover, the RBNN initialization algorithm has been modified in a deterministic way to eliminate any initial condition influence. Finally, the new method has been validated in two time series domains, an artificial and a real world one.

A crucial issue in the design of complex systems is the evaluation of a large number of potential alternative designs. A too expensive evaluation procedure can consequently slow down the search for good configurations, mainly in the case of high dimensional parameter spaces. This paper discusses the use of machine learning techniques for speeding up the evaluation and the exploration of large design spaces. In particular, two supervised learning techniques, feedforward neural networks and lazy ...