International Journal of Hybrid Intelligent Systems, 2009
Abstract. The margin maximization principle implemented by binary Support Vector Machines (SVMs) ... more Abstract. The margin maximization principle implemented by binary Support Vector Machines (SVMs) has been shown to be equivalent to find the hyperplane equidistant to the closest points belonging to the convex hulls that enclose each class of examples. In this paper, we ...
XXIV International Conference of the Chilean Computer Science Society
Learning the structure of real world data is difficult both to recognize and describe. The struct... more Learning the structure of real world data is difficult both to recognize and describe. The structure may contain high dimensional clusters that are related in complex ways. Furthermore, real data sets may contain several outliers. Vector quantization techniques has been successfully applied as a data mining tool. In particular the Neural Gas (NG) is a variant of the Self Organizing Map (SOM) where the neighborhoods are adaptively defined during training through the ranking order of the distance of prototypes from the given training sample. Unfortunately, the learning algorithm of the NG is sensitive to the presence of outliers as we will show in this paper. Due to the influence of the outliers in the learning process, the topology of the employed network does not conserve the topology of the manifold of the data which is presented. In this paper, we propose to robustify the learning algorithm where the parameter estimation process is resistant to the presence of outliers in the data. We call this algorithm Robust Neural Gas (RNG). We will illustrate our technique on synthetic and real data sets.
Abstract. We present a method for image segmentation, that is, to identify image points with an i... more Abstract. We present a method for image segmentation, that is, to identify image points with an indication of the region or class they belong to. The proposed algorithm basically consists of two stages. First it starts by restoring the image from possible contamination. In the ...
2012 IEEE 42nd International Symposium on Multiple-Valued Logic, 2012
In this work, a Takagi-Sugeno-Kang (TSK) model is used for time series analysis and some importan... more In this work, a Takagi-Sugeno-Kang (TSK) model is used for time series analysis and some important questions about the identification of this kind of models are addressed: the identification of the model structure and the set of the most influential regressors or lags. The main idea behind of the proposed method resembles to those techniques that prioritize lags evaluating the
The aim of this paper is to simultaneously identify and estimate a non-linear autoregressive time... more The aim of this paper is to simultaneously identify and estimate a non-linear autoregressive time series using a flexible neuro-fuzzy model. We provide a self organization and incremental mechanism to the adaptation process of the neuro-fuzzy model. The self organization mechanism searches for a suitable set of premises and consequents to enhance the time series estimation performance, while the incremental method selects influential lags in the model description. Experimental results indicate that our proposal reliably identifies appropriate lags for non-linear time series. Our proposal is illustrated by simulations on both synthetic and real data.
2010 XXIX International Conference of the Chilean Computer Science Society, 2010
... presenting different conditions. The attributes extracted from a Poincaré plot are the stan-d... more ... presenting different conditions. The attributes extracted from a Poincaré plot are the stan-dard deviation of the projections of the data points onto the lines y = x and y = −x, SD1 and SD2, respectively [13]. Another measure computed ...
This paper is concerned with robust models for representing images. The robust methods in image m... more This paper is concerned with robust models for representing images. The robust methods in image models are also applied to some important image processing situations such as segmentation by texture and image restoration in the presence of outliers. We consider a non-symmetric half plane (NSHP) autoregressive image model, where the image intensity at a point is a linear combination of the intensities of the eight nearest points located on one quadrant of the coordinate plane, plus an innovation process. Robust estimation algorithms for dierent outlier processes in causal autoregressive models are developed. These algorithms are based on robust generalized M (GM) estimators. Theoretical properties of the robust estimation algorithms are presented. The robust estimation algorithm for causal autoregressive models is applied to image restoration. The restoration method based on robust image model cleans out the outliers without involving any blurring of the image. Experimental results show that the quality of images restored by the model-based method is superior to the images restored by other conventional methods.
2008 Eighth International Conference on Hybrid Intelligent Systems, 2008
In this paper, we study a single objective extension of support vector machines for multicategory... more In this paper, we study a single objective extension of support vector machines for multicategory classification. Extending the dual formulation of binary SVMs, the algo-rithm looks for minimizing the sum of all the pairwise dis-tances among a set of prototypes, each one constrained to ...
Artificial neural networks (ANN) have been used as predictive systems for a variety of applicatio... more Artificial neural networks (ANN) have been used as predictive systems for a variety of application domains such as science, engineering and finance. Therefore it is very important to be able to estimate the reliability of a given model. Bootstrap is a computer intensive method used for estimating the distribution of a statistical estimator based on an imitation of the probabilistic
This paper proposes a new approach to train ensembles of learning machines in a regression contex... more This paper proposes a new approach to train ensembles of learning machines in a regression context. At each iteration a new learner is added to compensate the error made by the previous learner in the prediction of its training patterns. The algorithm operates directly over values to be predicted by the next machine to retain the ensemble in the target hypothesis and to ensure diversity. We expose a theoretical explanation which clarifies what the method is doing algorithmically and allows to show its stochastic convergence. Finally, experimental results are presented to compare the performance of this algorithm with boosting and bagging in two well-known data sets.
This paper describes the modelling of fuzzy rule systems using a multiresolution strategy that ha... more This paper describes the modelling of fuzzy rule systems using a multiresolution strategy that handles the problem of granularization of the input space by using multiresolution linguistic terms. Models of different resolutions are chained by antecedents because linguistic terms of a level j are obtained by refinements of linguistic terms of a superior level j + 1. The models can
Self-poised ensemble learning is based on the idea of introducing an artificial innovation to the... more Self-poised ensemble learning is based on the idea of introducing an artificial innovation to the map to be predicted by each machine in the ensemble such that it compensates the error incurred by the previous one. We will show that this approach is equivalent to regularize ...
In a previous article, we presented a genetic algorithm (GA), which finds solutions to problems o... more In a previous article, we presented a genetic algorithm (GA), which finds solutions to problems of robust design in multivariate systems. Based on that GA, we developed a new GA that uses a new desirability function, based on the aggregation of the observed variance of the responses and the squared deviation between the mean of each response and its corresponding target value. Additionally, we also changed the crossover operator from a one-point to a uniform one. We used three different case studies to evaluate the performance of the new GA and also to compare it with the original one. The first case study involved using data from a univariate real system, and the other two employed data obtained from multivariate process simulators. In each of the case studies, the new GA delivered good solutions, which simultaneously adjusted the mean of each response to its corresponding target value. This performance was similar to the one of the original GA. Regarding variability reduction, the new GA worked much better than the original one. In all the case studies, the new GA delivered solutions that simultaneously decreased the standard deviation of each response to almost the minimum possible value. Thus, we conclude that the new GA performs better than the original one, especially regarding variance reduction, which was the main problem exhibited by the original GA.
We introduce a method to restore digital images with contaminated pixels. One particular characte... more We introduce a method to restore digital images with contaminated pixels. One particular characteristic of this method is that it does not change the pixels that are not considered contaminated, thus avoiding excessive intervening of the original image. Each pixel is analyzed by studying its eight point neighborhood. A cluster analysis is performed on the group of eight pixels contained in the neighborhood. After deciding how many clusters there are in the neighborhood, a decision is made whether the center pixel is an outlier or not. If so, to assign a new value, another decision is made, on a probabilistic basis, as to which cluster it belongs. This method can be applied to black and white images as well as to color and multiphase images.
Artificial neural networks techniques have been successfully applied in vector quantization (VQ) ... more Artificial neural networks techniques have been successfully applied in vector quantization (VQ) encoding. The objective of VQ is to statistically preserve the topological relationships existing in a data set and to project the data to a lattice of lower dimensions, for visualization, compression, storage, or transmission purposes. However, one of the major drawbacks in the application of artificial neural networks is the difficulty to properly specify the structure of the lattice that best preserves the topology of the data. To overcome this problem, in this paper we introduce merging algorithms for machine-fusion, boostingfusion-based and hybrid-fusion ensembles of SOM, NG and GSOM networks. In these ensembles not the output signals of the base learners are combined, but their architectures are properly merged. We empirically show the quality and robustness of the topological representation of our proposed algorithm using both synthetic and real benchmarks datasets.
A mathematical statistical model is needed to obtain an option prime and create a hedging strateg... more A mathematical statistical model is needed to obtain an option prime and create a hedging strategy. With formulas derived from stochastic differential equations, the primes for US Dollar/Chilean Pesos currency options using a prime calculator are obtained. Furthermore, a backward simulation of the option prime trajectory is used with a numerical method created for backward stochastic differential equations. The use
Communications in Statistics - Theory and Methods, 2004
In financial time series analysis, serial correlations and the volatility clustering effects of a... more In financial time series analysis, serial correlations and the volatility clustering effects of asset returns are commonly checked by Ljung-Box and Mcleod-Li Q test and filtered by ARMA models. However, it is known that the both tests are not robust to heavily tailed data. We ...
. Outliers in time series seriously affect conventional parameter estimates. In this paper a robu... more . Outliers in time series seriously affect conventional parameter estimates. In this paper a robust recursive estimation procedure for the parameters of auto‐regressve moving‐average models with additive outliers is proposed. Using ‘cleaned’ residuals from an initial robust fit of an autoregression of high order as input, bounded influence regression is applied recursively. The proposal follows certain ideas of Hannan and Rissanen, who suggested a three‐stage procedure for order and parameter estimation in a conventional setting.A Monte Carlo study is performed to investigate the robustness properties of the proposed class of estimates and to compare them with various other suggestions, including least squares, M estimates, residual autocovariance and truncated residual autocovariance estimates. The results show that the recursive generalized M estimates compare favourably with them. Finally, possible modifications to master even vigourous situations are suggested.
International Journal of Hybrid Intelligent Systems, 2009
Abstract. The margin maximization principle implemented by binary Support Vector Machines (SVMs) ... more Abstract. The margin maximization principle implemented by binary Support Vector Machines (SVMs) has been shown to be equivalent to find the hyperplane equidistant to the closest points belonging to the convex hulls that enclose each class of examples. In this paper, we ...
XXIV International Conference of the Chilean Computer Science Society
Learning the structure of real world data is difficult both to recognize and describe. The struct... more Learning the structure of real world data is difficult both to recognize and describe. The structure may contain high dimensional clusters that are related in complex ways. Furthermore, real data sets may contain several outliers. Vector quantization techniques has been successfully applied as a data mining tool. In particular the Neural Gas (NG) is a variant of the Self Organizing Map (SOM) where the neighborhoods are adaptively defined during training through the ranking order of the distance of prototypes from the given training sample. Unfortunately, the learning algorithm of the NG is sensitive to the presence of outliers as we will show in this paper. Due to the influence of the outliers in the learning process, the topology of the employed network does not conserve the topology of the manifold of the data which is presented. In this paper, we propose to robustify the learning algorithm where the parameter estimation process is resistant to the presence of outliers in the data. We call this algorithm Robust Neural Gas (RNG). We will illustrate our technique on synthetic and real data sets.
Abstract. We present a method for image segmentation, that is, to identify image points with an i... more Abstract. We present a method for image segmentation, that is, to identify image points with an indication of the region or class they belong to. The proposed algorithm basically consists of two stages. First it starts by restoring the image from possible contamination. In the ...
2012 IEEE 42nd International Symposium on Multiple-Valued Logic, 2012
In this work, a Takagi-Sugeno-Kang (TSK) model is used for time series analysis and some importan... more In this work, a Takagi-Sugeno-Kang (TSK) model is used for time series analysis and some important questions about the identification of this kind of models are addressed: the identification of the model structure and the set of the most influential regressors or lags. The main idea behind of the proposed method resembles to those techniques that prioritize lags evaluating the
The aim of this paper is to simultaneously identify and estimate a non-linear autoregressive time... more The aim of this paper is to simultaneously identify and estimate a non-linear autoregressive time series using a flexible neuro-fuzzy model. We provide a self organization and incremental mechanism to the adaptation process of the neuro-fuzzy model. The self organization mechanism searches for a suitable set of premises and consequents to enhance the time series estimation performance, while the incremental method selects influential lags in the model description. Experimental results indicate that our proposal reliably identifies appropriate lags for non-linear time series. Our proposal is illustrated by simulations on both synthetic and real data.
2010 XXIX International Conference of the Chilean Computer Science Society, 2010
... presenting different conditions. The attributes extracted from a Poincaré plot are the stan-d... more ... presenting different conditions. The attributes extracted from a Poincaré plot are the stan-dard deviation of the projections of the data points onto the lines y = x and y = −x, SD1 and SD2, respectively [13]. Another measure computed ...
This paper is concerned with robust models for representing images. The robust methods in image m... more This paper is concerned with robust models for representing images. The robust methods in image models are also applied to some important image processing situations such as segmentation by texture and image restoration in the presence of outliers. We consider a non-symmetric half plane (NSHP) autoregressive image model, where the image intensity at a point is a linear combination of the intensities of the eight nearest points located on one quadrant of the coordinate plane, plus an innovation process. Robust estimation algorithms for dierent outlier processes in causal autoregressive models are developed. These algorithms are based on robust generalized M (GM) estimators. Theoretical properties of the robust estimation algorithms are presented. The robust estimation algorithm for causal autoregressive models is applied to image restoration. The restoration method based on robust image model cleans out the outliers without involving any blurring of the image. Experimental results show that the quality of images restored by the model-based method is superior to the images restored by other conventional methods.
2008 Eighth International Conference on Hybrid Intelligent Systems, 2008
In this paper, we study a single objective extension of support vector machines for multicategory... more In this paper, we study a single objective extension of support vector machines for multicategory classification. Extending the dual formulation of binary SVMs, the algo-rithm looks for minimizing the sum of all the pairwise dis-tances among a set of prototypes, each one constrained to ...
Artificial neural networks (ANN) have been used as predictive systems for a variety of applicatio... more Artificial neural networks (ANN) have been used as predictive systems for a variety of application domains such as science, engineering and finance. Therefore it is very important to be able to estimate the reliability of a given model. Bootstrap is a computer intensive method used for estimating the distribution of a statistical estimator based on an imitation of the probabilistic
This paper proposes a new approach to train ensembles of learning machines in a regression contex... more This paper proposes a new approach to train ensembles of learning machines in a regression context. At each iteration a new learner is added to compensate the error made by the previous learner in the prediction of its training patterns. The algorithm operates directly over values to be predicted by the next machine to retain the ensemble in the target hypothesis and to ensure diversity. We expose a theoretical explanation which clarifies what the method is doing algorithmically and allows to show its stochastic convergence. Finally, experimental results are presented to compare the performance of this algorithm with boosting and bagging in two well-known data sets.
This paper describes the modelling of fuzzy rule systems using a multiresolution strategy that ha... more This paper describes the modelling of fuzzy rule systems using a multiresolution strategy that handles the problem of granularization of the input space by using multiresolution linguistic terms. Models of different resolutions are chained by antecedents because linguistic terms of a level j are obtained by refinements of linguistic terms of a superior level j + 1. The models can
Self-poised ensemble learning is based on the idea of introducing an artificial innovation to the... more Self-poised ensemble learning is based on the idea of introducing an artificial innovation to the map to be predicted by each machine in the ensemble such that it compensates the error incurred by the previous one. We will show that this approach is equivalent to regularize ...
In a previous article, we presented a genetic algorithm (GA), which finds solutions to problems o... more In a previous article, we presented a genetic algorithm (GA), which finds solutions to problems of robust design in multivariate systems. Based on that GA, we developed a new GA that uses a new desirability function, based on the aggregation of the observed variance of the responses and the squared deviation between the mean of each response and its corresponding target value. Additionally, we also changed the crossover operator from a one-point to a uniform one. We used three different case studies to evaluate the performance of the new GA and also to compare it with the original one. The first case study involved using data from a univariate real system, and the other two employed data obtained from multivariate process simulators. In each of the case studies, the new GA delivered good solutions, which simultaneously adjusted the mean of each response to its corresponding target value. This performance was similar to the one of the original GA. Regarding variability reduction, the new GA worked much better than the original one. In all the case studies, the new GA delivered solutions that simultaneously decreased the standard deviation of each response to almost the minimum possible value. Thus, we conclude that the new GA performs better than the original one, especially regarding variance reduction, which was the main problem exhibited by the original GA.
We introduce a method to restore digital images with contaminated pixels. One particular characte... more We introduce a method to restore digital images with contaminated pixels. One particular characteristic of this method is that it does not change the pixels that are not considered contaminated, thus avoiding excessive intervening of the original image. Each pixel is analyzed by studying its eight point neighborhood. A cluster analysis is performed on the group of eight pixels contained in the neighborhood. After deciding how many clusters there are in the neighborhood, a decision is made whether the center pixel is an outlier or not. If so, to assign a new value, another decision is made, on a probabilistic basis, as to which cluster it belongs. This method can be applied to black and white images as well as to color and multiphase images.
Artificial neural networks techniques have been successfully applied in vector quantization (VQ) ... more Artificial neural networks techniques have been successfully applied in vector quantization (VQ) encoding. The objective of VQ is to statistically preserve the topological relationships existing in a data set and to project the data to a lattice of lower dimensions, for visualization, compression, storage, or transmission purposes. However, one of the major drawbacks in the application of artificial neural networks is the difficulty to properly specify the structure of the lattice that best preserves the topology of the data. To overcome this problem, in this paper we introduce merging algorithms for machine-fusion, boostingfusion-based and hybrid-fusion ensembles of SOM, NG and GSOM networks. In these ensembles not the output signals of the base learners are combined, but their architectures are properly merged. We empirically show the quality and robustness of the topological representation of our proposed algorithm using both synthetic and real benchmarks datasets.
A mathematical statistical model is needed to obtain an option prime and create a hedging strateg... more A mathematical statistical model is needed to obtain an option prime and create a hedging strategy. With formulas derived from stochastic differential equations, the primes for US Dollar/Chilean Pesos currency options using a prime calculator are obtained. Furthermore, a backward simulation of the option prime trajectory is used with a numerical method created for backward stochastic differential equations. The use
Communications in Statistics - Theory and Methods, 2004
In financial time series analysis, serial correlations and the volatility clustering effects of a... more In financial time series analysis, serial correlations and the volatility clustering effects of asset returns are commonly checked by Ljung-Box and Mcleod-Li Q test and filtered by ARMA models. However, it is known that the both tests are not robust to heavily tailed data. We ...
. Outliers in time series seriously affect conventional parameter estimates. In this paper a robu... more . Outliers in time series seriously affect conventional parameter estimates. In this paper a robust recursive estimation procedure for the parameters of auto‐regressve moving‐average models with additive outliers is proposed. Using ‘cleaned’ residuals from an initial robust fit of an autoregression of high order as input, bounded influence regression is applied recursively. The proposal follows certain ideas of Hannan and Rissanen, who suggested a three‐stage procedure for order and parameter estimation in a conventional setting.A Monte Carlo study is performed to investigate the robustness properties of the proposed class of estimates and to compare them with various other suggestions, including least squares, M estimates, residual autocovariance and truncated residual autocovariance estimates. The results show that the recursive generalized M estimates compare favourably with them. Finally, possible modifications to master even vigourous situations are suggested.
Uploads
Papers by Héctor Allende Olivares