One of the fundamental drawbacks of learning by gradient descent techniques is the susceptibility... more One of the fundamental drawbacks of learning by gradient descent techniques is the susceptibility to local minima during training. Recently, some authors have independently introduced new learning algorithms that are based on the properties of terminal attractors and repellers. These algorithms were claimed to perform global optimization of the cost in finite time, provided that a null solution exists. In this paper, we prove that, in the case of local minima free error functions, terminal attractor algorithms guarantee that the optimal solution is reached in a number of steps that is independent of the cost function. Moreover, in the case of multimodal functions, we prove that, unfortunately, there are no theoretical guarantees that a global solution can be reached or that the algorithms perform satisfactorily from an operational point of view, unless particular favourable conditions are satisfied. On the other hand, the ideas behind these innovative methods are very interesting an...
Recently, Wang et al. (Wang et al., 1991) have introduced two new learning algorithms, called TAB... more Recently, Wang et al. (Wang et al., 1991) have introduced two new learning algorithms, called TABP and HTABP1, that are based on the properties of terminal attractors. These algorithms were claimed to perform global optimization of the cost in finite time, provided that a null solution exists. In this paper, we prove that, unfortunately, there are no theoretical guarantees that a global solution will be reached, unless the learning process begins in the domain of attraction of the global minimum. When a local minimum basin is entered, quite random jumps in the weight space take place that may led to cycles. Moreover, when approaching local minima, overflow errors may also occur that force the learning to stop. Finally, particular care must be taken in order to avoid numerical problems that may occur even when approaching global minimum.
Artificial Neural Networks in Pattern Recognition, 2006
Automatic eye tracking is a challenging task, with numerous applications in biometrics, security,... more Automatic eye tracking is a challenging task, with numerous applications in biometrics, security, intelligent human–computer interfaces, and driver’s sleepiness detection systems. Eye localization and extraction is, therefore, the first step to the solution of such problems. In this paper, we present a new method, based on neural autoassociators, to solve the problem of detecting eyes from a facial image. A
Abstract Recursive neural networks are a new connectionist model introduced for processing graphs... more Abstract Recursive neural networks are a new connectionist model introduced for processing graphs. Linear recursive networks are a special subclass where the neurons have linear activation functions. The approximation properties of recursive networks are tightly connected to the possibility of distinguishing the patterns by generating a different internal encoding for each input of the domain. In this paper, it is shown that, even if linear recursive networks can distinguish the patterns of any finite set of trees, such a result ...
Journal of Bioinformatics and Computational Biology
Nowadays, it is well established that most of the human diseases which are not related to pathoge... more Nowadays, it is well established that most of the human diseases which are not related to pathogen infections have their origin from DNA disorders. Thus, DNA mutations, waiting for the availability of CRISPR-like remedies, will propagate into proteomics, offering the possibility to select natural or synthetic molecules to fight against the effects of malfunctioning proteins. Drug discovery, indeed, is a flourishing field of biotechnological research to improve human health, even though the development of a new drug is increasingly more expensive in spite of the massive use of informatics in Medicinal Chemistry. CRISPR technology adds new alternatives to cure diseases by removing DNA defects responsible of genome-related pathologies. In principle, the same technology, however, could also be exploited to induce protein mutations whose effects are controlled by the presence of suitable ligands. In this paper, a new idea is proposed for the realization of mutated proteins, on the surfac...
Abstract A naturally structured information is typical in symbolic processing. Nonetheless, learn... more Abstract A naturally structured information is typical in symbolic processing. Nonetheless, learning in connectionism is usually related to poorly organized data, like arrays or sequences. For these types of data, classical neural networks are proven to be universal approximators.
Abstract Recursive neural networks are a powerful tool for processing structured data. According ... more Abstract Recursive neural networks are a powerful tool for processing structured data. According to the recursive learning paradigm, the input information consists of directed positional acyclic graphs (DPAGs). In fact, recursive networks are fed following the partial order defined by the links of the graph. Unfortunately, the hypothesis of processing DPAGs is sometimes too restrictive, being the nature of some real–world problems intrinsically cyclic. In this paper, the methodology proposed in [1, 2] to process cyclic directed graphs is ...
One of the fundamental drawbacks of learning by gradient descent techniques is the susceptibility... more One of the fundamental drawbacks of learning by gradient descent techniques is the susceptibility to local minima during training. Recently, some authors have independently introduced new learning algorithms that are based on the properties of terminal attractors and repellers. These algorithms were claimed to perform global optimization of the cost in finite time, provided that a null solution exists. In this paper, we prove that, in the case of local minima free error functions, terminal attractor algorithms guarantee that the optimal solution is reached in a number of steps that is independent of the cost function. Moreover, in the case of multimodal functions, we prove that, unfortunately, there are no theoretical guarantees that a global solution can be reached or that the algorithms perform satisfactorily from an operational point of view, unless particular favourable conditions are satisfied. On the other hand, the ideas behind these innovative methods are very interesting an...
Recently, Wang et al. (Wang et al., 1991) have introduced two new learning algorithms, called TAB... more Recently, Wang et al. (Wang et al., 1991) have introduced two new learning algorithms, called TABP and HTABP1, that are based on the properties of terminal attractors. These algorithms were claimed to perform global optimization of the cost in finite time, provided that a null solution exists. In this paper, we prove that, unfortunately, there are no theoretical guarantees that a global solution will be reached, unless the learning process begins in the domain of attraction of the global minimum. When a local minimum basin is entered, quite random jumps in the weight space take place that may led to cycles. Moreover, when approaching local minima, overflow errors may also occur that force the learning to stop. Finally, particular care must be taken in order to avoid numerical problems that may occur even when approaching global minimum.
Artificial Neural Networks in Pattern Recognition, 2006
Automatic eye tracking is a challenging task, with numerous applications in biometrics, security,... more Automatic eye tracking is a challenging task, with numerous applications in biometrics, security, intelligent human–computer interfaces, and driver’s sleepiness detection systems. Eye localization and extraction is, therefore, the first step to the solution of such problems. In this paper, we present a new method, based on neural autoassociators, to solve the problem of detecting eyes from a facial image. A
Abstract Recursive neural networks are a new connectionist model introduced for processing graphs... more Abstract Recursive neural networks are a new connectionist model introduced for processing graphs. Linear recursive networks are a special subclass where the neurons have linear activation functions. The approximation properties of recursive networks are tightly connected to the possibility of distinguishing the patterns by generating a different internal encoding for each input of the domain. In this paper, it is shown that, even if linear recursive networks can distinguish the patterns of any finite set of trees, such a result ...
Journal of Bioinformatics and Computational Biology
Nowadays, it is well established that most of the human diseases which are not related to pathoge... more Nowadays, it is well established that most of the human diseases which are not related to pathogen infections have their origin from DNA disorders. Thus, DNA mutations, waiting for the availability of CRISPR-like remedies, will propagate into proteomics, offering the possibility to select natural or synthetic molecules to fight against the effects of malfunctioning proteins. Drug discovery, indeed, is a flourishing field of biotechnological research to improve human health, even though the development of a new drug is increasingly more expensive in spite of the massive use of informatics in Medicinal Chemistry. CRISPR technology adds new alternatives to cure diseases by removing DNA defects responsible of genome-related pathologies. In principle, the same technology, however, could also be exploited to induce protein mutations whose effects are controlled by the presence of suitable ligands. In this paper, a new idea is proposed for the realization of mutated proteins, on the surfac...
Abstract A naturally structured information is typical in symbolic processing. Nonetheless, learn... more Abstract A naturally structured information is typical in symbolic processing. Nonetheless, learning in connectionism is usually related to poorly organized data, like arrays or sequences. For these types of data, classical neural networks are proven to be universal approximators.
Abstract Recursive neural networks are a powerful tool for processing structured data. According ... more Abstract Recursive neural networks are a powerful tool for processing structured data. According to the recursive learning paradigm, the input information consists of directed positional acyclic graphs (DPAGs). In fact, recursive networks are fed following the partial order defined by the links of the graph. Unfortunately, the hypothesis of processing DPAGs is sometimes too restrictive, being the nature of some real–world problems intrinsically cyclic. In this paper, the methodology proposed in [1, 2] to process cyclic directed graphs is ...
Uploads