Showing 1–9 of 9 results for author: Garcia, J D

Regularization and Optimization in Model-Based Clustering

Authors: Raphael Araujo Sampaio, Joaquim Dias Garcia, Marcus Poggi, Thibaut Vidal

Abstract: Due to their conceptual simplicity, k-means algorithm variants have been extensively used for unsupervised cluster analysis. However, one main shortcoming of these algorithms is that they essentially fit a mixture of identical spherical Gaussians to data that vastly deviates from such a distribution. In comparison, general Gaussian Mixture Models (GMMs) can fit richer structures but require estima… ▽ More Due to their conceptual simplicity, k-means algorithm variants have been extensively used for unsupervised cluster analysis. However, one main shortcoming of these algorithms is that they essentially fit a mixture of identical spherical Gaussians to data that vastly deviates from such a distribution. In comparison, general Gaussian Mixture Models (GMMs) can fit richer structures but require estimating a quadratic number of parameters per cluster to represent the covariance matrices. This poses two main issues: (i) the underlying optimization problems are challenging due to their larger number of local minima, and (ii) their solutions can overfit the data. In this work, we design search strategies that circumvent both issues. We develop more effective optimization algorithms for general GMMs, and we combine these algorithms with regularization strategies that avoid overfitting. Through extensive computational analyses, we observe that optimization or regularization in isolation does not substantially improve cluster recovery. However, combining these techniques permits a completely new level of performance previously unachieved by k-means algorithm variants, unraveling vastly different cluster structures. These results shed new light on the current status quo between GMM and k-means methods and suggest the more frequent use of general GMMs for data exploration. To facilitate such applications, we provide open-source code as well as Julia packages (UnsupervisedClustering.jl and RegularizedCovarianceMatrices.jl) implementing the proposed techniques. △ Less

Submitted 5 February, 2024; v1 submitted 5 February, 2023; originally announced February 2023.

Flexible Differentiable Optimization via Model Transformations

Authors: Mathieu Besançon, Joaquim Dias Garcia, Benoît Legat, Akshay Sharma

Abstract: We introduce DiffOpt.jl, a Julia library to differentiate through the solution of optimization problems with respect to arbitrary parameters present in the objective and/or constraints. The library builds upon MathOptInterface, thus leveraging the rich ecosystem of solvers and composing well with modeling languages like JuMP. DiffOpt offers both forward and reverse differentiation modes, enabling… ▽ More We introduce DiffOpt.jl, a Julia library to differentiate through the solution of optimization problems with respect to arbitrary parameters present in the objective and/or constraints. The library builds upon MathOptInterface, thus leveraging the rich ecosystem of solvers and composing well with modeling languages like JuMP. DiffOpt offers both forward and reverse differentiation modes, enabling multiple use cases from hyperparameter optimization to backpropagation and sensitivity analysis, bridging constrained optimization with end-to-end differentiable programming. DiffOpt is built on two known rules for differentiating quadratic programming and conic programming standard forms. However, thanks ability to differentiate through model transformation, the user is not limited to these forms and can differentiate with respect to the parameters of any model that can be reformulated into these standard forms. This notably includes programs mixing affine conic constraints and convex quadratic constraints or objective function. △ Less

Submitted 31 July, 2023; v1 submitted 10 June, 2022; originally announced June 2022.

JuMP 1.0: Recent improvements to a modeling language for mathematical optimization

Authors: Miles Lubin, Oscar Dowson, Joaquim Dias Garcia, Joey Huchette, Benoît Legat, Juan Pablo Vielma

Abstract: JuMP is an algebraic modeling language embedded in the Julia programming language. JuMP allows users to model optimization problems of a variety of kinds, including linear programming, integer programming, conic optimization, semidefinite programming, and nonlinear programming, and handles the low-level details of communicating with solvers. After nearly 10 years in development, JuMP 1.0 was relea… ▽ More JuMP is an algebraic modeling language embedded in the Julia programming language. JuMP allows users to model optimization problems of a variety of kinds, including linear programming, integer programming, conic optimization, semidefinite programming, and nonlinear programming, and handles the low-level details of communicating with solvers. After nearly 10 years in development, JuMP 1.0 was released in March, 2022. In this short communication, we highlight the improvements to JuMP from recent releases up to and including 1.0. △ Less

Submitted 19 March, 2023; v1 submitted 31 May, 2022; originally announced June 2022.

BilevelJuMP.jl: Modeling and Solving Bilevel Optimization in Julia

Authors: Joaquim Dias Garcia, Guilherme Bodin, Alexandre Street

Abstract: In this paper we present BilevelJuMP, a new Julia package to support bilevel optimization within the JuMP framework. The package is a Julia library that enables the user to describe both upper and lower-level optimization problems using the JuMP algebraic syntax. Due to the generality and flexibility our library inherits from JuMP's syntax, our package allows users to model bilevel optimization pr… ▽ More In this paper we present BilevelJuMP, a new Julia package to support bilevel optimization within the JuMP framework. The package is a Julia library that enables the user to describe both upper and lower-level optimization problems using the JuMP algebraic syntax. Due to the generality and flexibility our library inherits from JuMP's syntax, our package allows users to model bilevel optimization problems with conic constraints in the lower level and all JuMP supported constraints in the upper level (Conic, Quadratic, Non-Linear, Integer, etc.). Moreover, the user-defined problem can be subsequently solved by various techniques relying on mathematical program with equilibrium constraints (MPEC) reformulations. Manipulations on the original problem data are possible due to MathOptInterface.jl's structures and Dualization.jl features. Hence, the proposed package allows quickly model, deploy, and thereby experiment bilevel models based on off-the-shelf mixed integer linear programming and nonlinear solvers. △ Less

Submitted 6 June, 2022; v1 submitted 4 May, 2022; originally announced May 2022.

Application-Driven Learning: A Closed-Loop Prediction and Optimization Approach Applied to Dynamic Reserves and Demand Forecasting

Authors: Joaquim Dias Garcia, Alexandre Street, Tito Homem-de-Mello, Francisco D. Muñoz

Abstract: Forecasting and decision-making are generally modeled as two sequential steps with no feedback, following an open-loop approach. In this paper, we present application-driven learning, a new closed-loop framework in which the processes of forecasting and decision-making are merged and co-optimized through a bilevel optimization problem. We present our methodology in a general format and prove that… ▽ More Forecasting and decision-making are generally modeled as two sequential steps with no feedback, following an open-loop approach. In this paper, we present application-driven learning, a new closed-loop framework in which the processes of forecasting and decision-making are merged and co-optimized through a bilevel optimization problem. We present our methodology in a general format and prove that the solution converges to the best estimator in terms of the expected cost of the selected application. Then, we propose two solution methods: an exact method based on the KKT conditions of the second-level problem and a scalable heuristic approach suitable for decomposition methods. The proposed methodology is applied to the relevant problem of defining dynamic reserve requirements and conditional load forecasts, offering an alternative approach to current ad hoc procedures implemented in industry practices. We benchmark our methodology with the standard sequential least-squares forecast and dispatch planning process. We apply the proposed methodology to an illustrative system and to a wide range of instances, from dozens of buses to large-scale realistic systems with thousands of buses. Our results show that the proposed methodology is scalable and yields consistently better performance than the standard open-loop approach. △ Less

Submitted 8 April, 2024; v1 submitted 25 February, 2021; originally announced February 2021.

Application of Progressive Hedging to Var Expansion Planning Under Uncertainty

Authors: Igor Carvalho, Tiago Andrade, Joaquim Dias Garcia, Maria de Lujan Latorre

Abstract: This paper describes the application of a Progressive Hedging (PH) algorithm to the least-cost var planning under uncertainty. The method PH is a scenario-based decomposition technique for solving stochastic programs, i.e., it decomposes a large scale stochastic problem into s deterministic subproblems and couples the decision from the s subproblems to form a solution for the original stochastic p… ▽ More This paper describes the application of a Progressive Hedging (PH) algorithm to the least-cost var planning under uncertainty. The method PH is a scenario-based decomposition technique for solving stochastic programs, i.e., it decomposes a large scale stochastic problem into s deterministic subproblems and couples the decision from the s subproblems to form a solution for the original stochastic problem. The effectiveness and computational performance of the proposed methodology will be illustrated with var planning studies for the IEEE 24-bus system (5 operating scenarios), the 200-bus Bolivian system (1,152 operating scenarios) and the 1,600-bus Colombian system (180 scenarios). △ Less

Submitted 17 April, 2020; originally announced April 2020.

Community Detection for Power Systems Network Aggregation Considering Renewable Variability

Authors: Raphael Araujo Sampaio, Gerson Couto Oliveira, Luiz Carlos da Costa Jr., Joaquim Dias Garcia

Abstract: The increasing penetration of variable renewable energy (VRE) has brought significant challenges for power systems planning and operation. These highly variable sources are typically distributed in the grid; therefore, a detailed representation of transmission bottlenecks is fundamental to approximate the impact of the transmission network on the dispatch with VRE resources. The fine grain tempora… ▽ More The increasing penetration of variable renewable energy (VRE) has brought significant challenges for power systems planning and operation. These highly variable sources are typically distributed in the grid; therefore, a detailed representation of transmission bottlenecks is fundamental to approximate the impact of the transmission network on the dispatch with VRE resources. The fine grain temporal scale of short term and day-ahead dispatch, taking into account the network constraints, also mandatory for mid-term planning studies, combined with the high variability of the VRE has brought the need to represent these uncertainties in stochastic optimization models while taking into account the transmission system. These requirements impose a computational burden to solve the planning and operation models. We propose a methodology based on community detection to aggregate the network representation, capable of preserving the locational marginal price (LMP) differences in multiple VRE scenarios, and describe a real-world operational planning study. The optimal expected cost solution considering aggregated networks is compared with the full network representation. Both representations were embedded in an operation model relying on Stochastic Dual Dynamic Programming (SDDP) to deal with the random variables in a multi-stage problem. △ Less

Submitted 8 November, 2019; originally announced November 2019.

Message Passing for Complex Question Answering over Knowledge Graphs

Authors: Svitlana Vakulenko, Javier David Fernandez Garcia, Axel Polleres, Maarten de Rijke, Michael Cochez

Abstract: Question answering over knowledge graphs (KGQA) has evolved from simple single-fact questions to complex questions that require graph traversal and aggregation. We propose a novel approach for complex KGQA that uses unsupervised message passing, which propagates confidence scores obtained by parsing an input question and matching terms in the knowledge graph to a set of possible answers. First, we… ▽ More Question answering over knowledge graphs (KGQA) has evolved from simple single-fact questions to complex questions that require graph traversal and aggregation. We propose a novel approach for complex KGQA that uses unsupervised message passing, which propagates confidence scores obtained by parsing an input question and matching terms in the knowledge graph to a set of possible answers. First, we identify entity, relationship, and class names mentioned in a natural language question, and map these to their counterparts in the graph. Then, the confidence scores of these mappings propagate through the graph structure to locate the answer entities. Finally, these are aggregated depending on the identified question type. This approach can be efficiently implemented as a series of sparse matrix multiplications mimicking joins over small local subgraphs. Our evaluation results show that the proposed approach outperforms the state-of-the-art on the LC-QuAD benchmark. Moreover, we show that the performance of the approach depends only on the quality of the question interpretation results, i.e., given a correct relevance score distribution, our approach always produces a correct answer ranking. Our error analysis reveals correct answers missing from the benchmark dataset and inconsistencies in the DBpedia knowledge graph. Finally, we provide a comprehensive evaluation of the proposed approach accompanied with an ablation study and an error analysis, which showcase the pitfalls for each of the question answering components in more detail. △ Less

Submitted 19 August, 2019; originally announced August 2019.

Comments: Accepted in CIKM 2019

An Optimal Polarization Tracking Algorithm for Lithium-Niobate-based Polarization Controllers

Authors: Joaquim D. Garcia, Gustavo C. Amaral

Abstract: We present an optimal algorithm for the three-stage arbitrary polarization tracking using Lithium-Niobate-based Polarization Controllers: device calibration, polarization state rotation, and stabilization. The theoretical model representing the lithium-niobate-based polarization controller is derived and the methodology is successfully applied. Results are numerically simulated in the MATLAB envir… ▽ More We present an optimal algorithm for the three-stage arbitrary polarization tracking using Lithium-Niobate-based Polarization Controllers: device calibration, polarization state rotation, and stabilization. The theoretical model representing the lithium-niobate-based polarization controller is derived and the methodology is successfully applied. Results are numerically simulated in the MATLAB environment. △ Less

Submitted 11 March, 2016; originally announced March 2016.

Comments: 8 pages, 4 figures