Gradient Descent

We investigate the use of the Riemannianoptimization method over the flag manifold in subspace ICA problems such as in-dependent subspace analysis (ISA) and complex ICA. In the ISA experiment, we use the Riemannian approach over the flag manifold together ...

A prototype tool to assist architects during the early design stage of floor plans has been developed, consisting of an Evolutionary Program for the Space Allocation Problem (EPSAP), which generates sets of floor plan alternatives according to the architect's preferences; and a Floor Plan Performance Optimization Program (FPOP), which optimizes the selected solutions according to thermal performance criteria. The design

This study examines and analyses the use of a new recurrent neural network model: Jordan Pi-Sigma Network (JPSN) as a forecasting tool. JPSN's ability to predict future trends of temperature was tested and compared to that of Multilayer Perceptron (MLP) and the standard Pi-Sigma Neural Network (PSNN); trained with the standard gradient descent algorithm. A set of historical temperature measurement for five years from Malaysian Meteorological Department was used as input data to train the networks for the next-day ...

This paper addresses the problem of scheduling a set of independent jobs with sequence-dependent setups and distinct due dates on non-uniform multi-machines to minimize the total weighted earliness and tardiness, and explores the use of artificial neural networks as a valid alternative to the traditional scheduling approaches. The objective is to propose a dynamical gradient neural network, which employs a penalty function approach with time varying coefficients for the solution of the problem which is known to be NP-hard. After the appropriate energy function was constructed, the dynamics are defined by steepest gradient descent on the energy function. The proposed neural network system is composed of two maximum neural networks, three piecewise linear and one log-sigmoid network all of which interact with each other. The motivation for using maximum networks is to reduce the network complexity and to obtain a simplified energy function. To overcome the tradeoff problem encountered in using the penalty function approach, a time varying penalty coefficient methodology is proposed to be used during simulation experiments. Simulation results of the proposed approach on a scheduling problem indicate that the proposed coupled network yields an optimal solution which makes it attractive for applications of larger sized problems.

The authors propose a general fuzzy classification scheme with learning ability using an adaptive network. System parameters, such as the membership functions defined for each feature and the parameterized t-norms used to combine conjunctive conditions, are calibrated with backpropagation. To explain this approach, the concept of adaptive networks is introduced and a supervised learning procedure based on a gradient descent algorithm is derived to update the parameters in an adaptive network. The proposed architecture is applied to two problems: two-spiral classification and Iris categorization. From the experimental results, it is concluded that the adaptively adjusted classifier performs well on an Iris classification problem. The results are discussed from the viewpoint of feature selection

Metric learning algorithms can provide useful distance functions for a variety of domains, and recent work has shown good accuracy for problems where the learner can access all distance constraints at once. However, in many real appli-cations, constraints are only available ...

In the cyber era, Machine Learning (ML) has provided us with the solutions to these problems with the implementation of Gradient Boosting Machines (GBM). We have ample algorithms to choose from to do gradient boosting for our training data but still, we encounter different issues like poor accuracy, high loss, large variance in the result. Here, we are going to introduce you to a state of the art machine learning algorithm XGBoost built by Tianqi Chen, that will not only overcome the issues but also perform exceptionally well for regression and classification problems. This blog will help you discover the insights, techniques, and skills with XGBoost that you can then bring to your machine learning projects.
eXtreme Gradient Boosting (XGBoost) is a scalable and improved version of the gradient boosting algorithm (terminology alert) designed for efficacy, computational speed and model performance. It is an open-source library and a part of the Distributed Machine Learning Community. XGBoost is a perfect blend of software and hardware capabilities designed to enhance existing boosting techniques with accuracy in the shortest amount of time. Here’s a quick look at an objective benchmark comparison of XGBoost with other gradient boosting algorithms trained on random forest with 500 trees, performed by Szilard Pafka.

Lattice-based signature schemes following the Goldreich–Goldwasser–Halevi (GGH) design have the unusual property that each signature leaks information on the signer’s secret key, but this does not necessarily imply that such schemes are insecure. At Eurocrypt ’03, Szydlo proposed a potential attack by showing that the leakage reduces the key-recovery problem to that of distinguishing integral quadratic forms. He proposed a

Many games, in particular RTS games, are populated by synthetic humanoid actors that act as autonomous agents. The navigation of these agents is yet a challenge if the problem involves finding a precise route in a virtual world (path-planning), and moving realistically according to its own personality, intentions and mood (motion planning). In this paper we present several complementary approaches recently developed by our group to produce quality paths, and to guide and interact with the navigation of autonomous agents. Our approach is based on a BVP Path Planner that generates potential fields through a differential equation whose gradient descent represents navigation routes. Resulting paths can deal with moving obstacles, are smooth, and free from local minima. In order to evaluate the algorithms, we implemented our path planner in a RTS game engine.

We present a mathematical implementation of a quantum mechanical artificial neural network, in the quasi-continuu m regime, using the nonlinearity inherent in the real-time propagation of a quantum system coupled to its environment. Our model is that of a quantum dot molecule coupled to the substrate lattice through optical phonons, and subject to a timevarying external field. Using discretized Feynman path integrals, we find that the real time evolution of the system can be put into a form which resembles the equations for the virtual neuron activation levels of an artificial neural network. The timeline discretization points serve as virtual neurons. We then train the network using a simple gradient descent algorithm, and find it is possible in some regions of the phase space to perform any desired classical logic gate. Because the network is quantum mechanical we can also train purely quantum gates such as a phase shift.

The general regression neural network (GRNN) is known to be widely effective for modeling and prediction, especially if separate sigma weights are used for each predictor. However, the significant time requirements for executing the model, combined with the frequent presence of multiple local optima, makes it difficult to train this model in many applications. This paper shows how differential evolution