Showing 1–50 of 101 results for author: Montana, G

Studying $π^+π^-$ photoproduction beyond Pomeron exchange

Authors: Łukasz Bibrzycki, Nadine Hammoud, Vincent Mathieu, Robert J. Perry, Alex Akridge, César Fernández-Ramírez, Gloria Montaña, Alessandro Pilloni, Arkaitz Rodas, Vanamali Shastry, Wyatt A. Smith, Daniel Winney, Adam P. Szczepaniak

Abstract: Forward photoproduction of $π^+π^-$ pairs with invariant mass of the order of $m_ρ\sim 770$ MeV is traditionally understood to be produced via Pomeron exchange. Based on a detailed analysis of the CLAS photoproduction data, it is shown that the dynamics of two-pion photoproduction for $|t|\gtrsim 0.5$ GeV$^2$ cannot be explained by Pomeron exchange alone. This motivates the development of a new th… ▽ More Forward photoproduction of $π^+π^-$ pairs with invariant mass of the order of $m_ρ\sim 770$ MeV is traditionally understood to be produced via Pomeron exchange. Based on a detailed analysis of the CLAS photoproduction data, it is shown that the dynamics of two-pion photoproduction for $|t|\gtrsim 0.5$ GeV$^2$ cannot be explained by Pomeron exchange alone. This motivates the development of a new theoretical model of two-pion photoproduction which incorporates both two-pion and pion-nucleon resonant contributions. After fitting free parameters, the model provides an excellent description of the low moments of the angular distribution measured at CLAS, and enables an assessment of the relative contributions of particular production mechanisms and an interpretation of the various features of the data in terms of these mechanisms. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 30 pages, 21 figures

Report number: JLAB-THY-24-4078

State-Constrained Offline Reinforcement Learning

Authors: Charles A. Hepburn, Yue Jin, Giovanni Montana

Abstract: Traditional offline reinforcement learning methods predominantly operate in a batch-constrained setting. This confines the algorithms to a specific state-action distribution present in the dataset, reducing the effects of distributional shift but restricting the algorithm greatly. In this paper, we alleviate this limitation by introducing a novel framework named \emph{state-constrained} offline re… ▽ More Traditional offline reinforcement learning methods predominantly operate in a batch-constrained setting. This confines the algorithms to a specific state-action distribution present in the dataset, reducing the effects of distributional shift but restricting the algorithm greatly. In this paper, we alleviate this limitation by introducing a novel framework named \emph{state-constrained} offline reinforcement learning. By exclusively focusing on the dataset's state distribution, our framework significantly enhances learning potential and reduces previous limitations. The proposed setting not only broadens the learning horizon but also improves the ability to combine different trajectories from the dataset effectively, a desirable property inherent in offline reinforcement learning. Our research is underpinned by solid theoretical findings that pave the way for subsequent advancements in this domain. Additionally, we introduce StaCQ, a deep learning algorithm that is both performance-driven on the D4RL benchmark datasets and closely aligned with our theoretical propositions. StaCQ establishes a strong baseline for forthcoming explorations in state-constrained offline reinforcement learning. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Nonperturbative aspects of the electromagnetic pion form factor at high energies

Authors: Joint Physics Analysis Center, :, K. Quirion, C. Fernández-Ramírez, V. Mathieu, G. Montaña, R. J. Perry, A. Pilloni, A. Rodas, V. Shastry, W. A. Smith, A. P. Szczepaniak, D. Winney

Abstract: The structure of hadronic form factors at high energies and their deviations from perturbative quantum chromodynamics provide insight on nonperturbative dynamics. Using an approach that is consistent with dispersion relations, we construct a model that simultaneously accounts for the pion wave function, gluonic exchanges, and quark Reggeization. In particular, we find that quark Reggeization can b… ▽ More The structure of hadronic form factors at high energies and their deviations from perturbative quantum chromodynamics provide insight on nonperturbative dynamics. Using an approach that is consistent with dispersion relations, we construct a model that simultaneously accounts for the pion wave function, gluonic exchanges, and quark Reggeization. In particular, we find that quark Reggeization can be investigated at high energies by studying scaling violation of the form factor. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Report number: JLAB-THY-24-4058

XYZ spectroscopy at electron-hadron facilities III: Semi-inclusive processes with vector exchanges

Authors: Joint Physics Analysis Center Collaboration, D. Winney, A. Pilloni, R. J. Perry, L. Bibrzycki, C. Fernandez-Ramirez, N. Hammoud, V. Mathieu, G. Montana, A. Rodas, V. Shastry, W. A. Smith, A. P. Szczepaniak

Abstract: Inclusive production processes will be important for the first observations of $XYZ$ states at new generation electron-hadron colliders, as they generally benefit from larger cross sections than their exclusive counterparts. We make predictions of semi-inclusive photoproduction of the $χ_{c1}(1P)$ and $X(3872)$, whose peripheral production is assumed to be dominated by vector exchanges. We validat… ▽ More Inclusive production processes will be important for the first observations of $XYZ$ states at new generation electron-hadron colliders, as they generally benefit from larger cross sections than their exclusive counterparts. We make predictions of semi-inclusive photoproduction of the $χ_{c1}(1P)$ and $X(3872)$, whose peripheral production is assumed to be dominated by vector exchanges. We validate the applicability of Vector Meson Dominance in the axial-vector charmonium sector and calculate production rates at center-of-mass energies relevant for future experimental facilities. We find the semi-inclusive cross sections near threshold to be enhanced by a factor of $\sim 2-3$ compared to the exclusive reaction and well suited for a first observation in photoproduction. △ Less

Submitted 24 June, 2024; v1 submitted 8 April, 2024; originally announced April 2024.

Comments: 16 pages, 9 figures. Typos fixed and references updated. Link to code provided. Version appearing in PRD

Report number: JLAB-THY-24-4009

Journal ref: Phys. Rev. D 109 (2024), 114035

REValueD: Regularised Ensemble Value-Decomposition for Factorisable Markov Decision Processes

Authors: David Ireland, Giovanni Montana

Abstract: Discrete-action reinforcement learning algorithms often falter in tasks with high-dimensional discrete action spaces due to the vast number of possible actions. A recent advancement leverages value-decomposition, a concept from multi-agent reinforcement learning, to tackle this challenge. This study delves deep into the effects of this value-decomposition, revealing that whilst it curtails the ove… ▽ More Discrete-action reinforcement learning algorithms often falter in tasks with high-dimensional discrete action spaces due to the vast number of possible actions. A recent advancement leverages value-decomposition, a concept from multi-agent reinforcement learning, to tackle this challenge. This study delves deep into the effects of this value-decomposition, revealing that whilst it curtails the over-estimation bias inherent to Q-learning algorithms, it amplifies target variance. To counteract this, we present an ensemble of critics to mitigate target variance. Moreover, we introduce a regularisation loss that helps to mitigate the effects that exploratory actions in one dimension can have on the value of optimal actions in other dimensions. Our novel algorithm, REValueD, tested on discretised versions of the DeepMind Control Suite tasks, showcases superior performance, especially in the challenging humanoid and dog tasks. We further dissect the factors influencing REValueD's performance, evaluating the significance of the regularisation loss and the scalability of REValueD with increasing sub-actions per dimension. △ Less

Submitted 8 March, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: ICLR camera ready version

GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models

Authors: Mianchu Wang, Rui Yang, Xi Chen, Hao Sun, Meng Fang, Giovanni Montana

Abstract: Offline Goal-Conditioned RL (GCRL) offers a feasible paradigm for learning general-purpose policies from diverse and multi-task offline datasets. Despite notable recent progress, the predominant offline GCRL methods, mainly model-free, face constraints in handling limited data and generalizing to unseen goals. In this work, we propose Goal-conditioned Offline Planning (GOPlan), a novel model-based… ▽ More Offline Goal-Conditioned RL (GCRL) offers a feasible paradigm for learning general-purpose policies from diverse and multi-task offline datasets. Despite notable recent progress, the predominant offline GCRL methods, mainly model-free, face constraints in handling limited data and generalizing to unseen goals. In this work, we propose Goal-conditioned Offline Planning (GOPlan), a novel model-based framework that contains two key phases: (1) pretraining a prior policy capable of capturing multi-modal action distribution within the multi-goal dataset; (2) employing the reanalysis method with planning to generate imagined trajectories for funetuning policies. Specifically, we base the prior policy on an advantage-weighted conditioned generative adversarial network, which facilitates distinct mode separation, mitigating the pitfalls of out-of-distribution (OOD) actions. For further policy optimization, the reanalysis method generates high-quality imaginary data by planning with learned models for both intra-trajectory and inter-trajectory goals. With thorough experimental evaluations, we demonstrate that GOPlan achieves state-of-the-art performance on various offline multi-goal navigation and manipulation tasks. Moreover, our results highlight the superior ability of GOPlan to handle small data budgets and generalize to OOD goals. △ Less

Submitted 16 May, 2024; v1 submitted 30 October, 2023; originally announced October 2023.

Comments: Spotlight Presentation at Goal-conditioned Reinforcement Learning Workshop at NeurIPS 2023

Journal ref: Transactions on Machine Learning Research (05/2024)

Toward a generative modeling analysis of CLAS exclusive $2π$ photoproduction

Authors: T. Alghamdi, Y. Alanazi, M. Battaglieri, L. Bibrzycki, A. V. Golda, A. N. Hiller Blin, E. L. Isupov, Y. Li, L. Marsicano, W. Melnitchouk, V. I. Mokeev, G. Montana, A. Pilloni, N. Sato, A. P. Szczepaniak, T. Vittorini

Abstract: AI-supported algorithms, particularly generative models, have been successfully used in a variety of different contexts. In this work, we demonstrate for the first time that generative adversarial networks (GANs) can be used in high-energy experimental physics to unfold detector effects from multi-particle final states, while preserving correlations between kinematic variables in multidimensional… ▽ More AI-supported algorithms, particularly generative models, have been successfully used in a variety of different contexts. In this work, we demonstrate for the first time that generative adversarial networks (GANs) can be used in high-energy experimental physics to unfold detector effects from multi-particle final states, while preserving correlations between kinematic variables in multidimensional phase space. We perform a full closure test on two-pion photoproduction pseudodata generated with a realistic model in the kinematics of the Jefferson Lab CLAS g11 experiment. The overlap of different reaction mechanisms leading to the same final state associated with the CLAS detector's nontrivial effects represents an ideal test case for AI-supported analysis. Uncertainty quantification performed via bootstrap provides an estimate of the systematic uncertainty associated with the procedure. The test demonstrates that GANs can reproduce highly correlated multidifferential cross sections even in the presence of detector-induced distortions in the training datasets, and provides a solid basis for applying the framework to real experimental data. △ Less

Submitted 10 July, 2023; originally announced July 2023.

Comments: 14 pages, 20 figures

Report number: JLAB-THY-23-3881

Recent progress on in-medium properties of heavy mesons from finite-temperature EFTs

Authors: Gloria Montana, Angels Ramos, Laura Tolos, Juan M. Torres-Rincon

Abstract: Mesons with heavy flavor content are an exceptional probe of the hot QCD medium produced in heavy-ion collisions. In the past few years, significant progress has been made toward describing the modification of the properties of heavy mesons in the hadronic phase at finite temperature. Ground-state and excited-state thermal spectral properties can be computed within a self-consistent many-body appr… ▽ More Mesons with heavy flavor content are an exceptional probe of the hot QCD medium produced in heavy-ion collisions. In the past few years, significant progress has been made toward describing the modification of the properties of heavy mesons in the hadronic phase at finite temperature. Ground-state and excited-state thermal spectral properties can be computed within a self-consistent many-body approach that employs appropriate hadron-hadron effective interactions, providing a unique opportunity to confront hadronic Effective Field Theory predictions with recent and forthcoming lattice QCD simulations and experimental data. In this article, we revisit the application of the imaginary-time formalism to extend the calculation of unitarized scattering amplitudes from the vacuum to finite temperature. These methods allow us to obtain the ground-state thermal spectral functions. The thermal properties of the excited states that are dynamically generated within the molecular picture are also directly accessible. We present here the results of this approach for the open-charm and open-bottom sectors. We also analyze how the heavy-flavor transport properties, which are strongly correlated to experimental observables in heavy-ion collisions, are modified in hot matter. In particular, transport coefficients can be computed using an off-shell kinetic theory that is fully consistent with the effective theory describing the scattering processes. The results of this procedure for both charm and bottom transport coefficients are briefly discussed. △ Less

Submitted 7 July, 2023; originally announced July 2023.

Comments: 16 pages, 8 figures, Submitted to Front.Phys.-Nuclear Physics to contribute to the Research Topic: Excotic Aspects of Hadrons and Nuclei

Report number: JLAB-THY-23-3874

Ambiguities in Partial Wave Analysis of Two Spinless Meson Photoproduction

Authors: JPAC Collaboration, W. A. Smith, D. I. Glazier, V. Mathieu, M. Albaladejo, M. Albrecht, Z. Baldwin, C. Fernández-Ramírez, N. Hammoud, M. Mikhasenko, G. Montaña, R. J. Perry, A. Pilloni, V. Shastry, A. P. Szczepaniak, D. Winney

Abstract: We describe the formalism to analyze the mathematical ambiguities arising in partial-wave analysis of two spinless mesons produced with a linearly polarized photon beam. We show that partial waves are uniquely defined when all accessible observables are considered, for a wave set which includes $S$ and $D$ waves. The inclusion of higher partial waves does not affect our results, and we conclude th… ▽ More We describe the formalism to analyze the mathematical ambiguities arising in partial-wave analysis of two spinless mesons produced with a linearly polarized photon beam. We show that partial waves are uniquely defined when all accessible observables are considered, for a wave set which includes $S$ and $D$ waves. The inclusion of higher partial waves does not affect our results, and we conclude that there are no mathematical ambiguities in partial-wave analysis of two mesons produced with a linearly polarized photon beam. We present Monte Carlo simulations to illustrate our results. △ Less

Submitted 30 June, 2023; originally announced June 2023.

Comments: 11 pages, 8 figures

Report number: JLAB-THY-23-3873

Strong Interaction Physics at the Luminosity Frontier with 22 GeV Electrons at Jefferson Lab

Authors: A. Accardi, P. Achenbach, D. Adhikari, A. Afanasev, C. S. Akondi, N. Akopov, M. Albaladejo, H. Albataineh, M. Albrecht, B. Almeida-Zamora, M. Amaryan, D. Androić, W. Armstrong, D. S. Armstrong, M. Arratia, J. Arrington, A. Asaturyan, A. Austregesilo, H. Avagyan, T. Averett, C. Ayerbe Gayoso, A. Bacchetta, A. B. Balantekin, N. Baltzell, L. Barion , et al. (419 additional authors not shown)

Abstract: This document presents the initial scientific case for upgrading the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab (JLab) to 22 GeV. It is the result of a community effort, incorporating insights from a series of workshops conducted between March 2022 and April 2023. With a track record of over 25 years in delivering the world's most intense and precise multi-GeV electron… ▽ More This document presents the initial scientific case for upgrading the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab (JLab) to 22 GeV. It is the result of a community effort, incorporating insights from a series of workshops conducted between March 2022 and April 2023. With a track record of over 25 years in delivering the world's most intense and precise multi-GeV electron beams, CEBAF's potential for a higher energy upgrade presents a unique opportunity for an innovative nuclear physics program, which seamlessly integrates a rich historical background with a promising future. The proposed physics program encompass a diverse range of investigations centered around the nonperturbative dynamics inherent in hadron structure and the exploration of strongly interacting systems. It builds upon the exceptional capabilities of CEBAF in high-luminosity operations, the availability of existing or planned Hall equipment, and recent advancements in accelerator technology. The proposed program cover various scientific topics, including Hadron Spectroscopy, Partonic Structure and Spin, Hadronization and Transverse Momentum, Spatial Structure, Mechanical Properties, Form Factors and Emergent Hadron Mass, Hadron-Quark Transition, and Nuclear Dynamics at Extreme Conditions, as well as QCD Confinement and Fundamental Symmetries. Each topic highlights the key measurements achievable at a 22 GeV CEBAF accelerator. Furthermore, this document outlines the significant physics outcomes and unique aspects of these programs that distinguish them from other existing or planned facilities. In summary, this document provides an exciting rationale for the energy upgrade of CEBAF to 22 GeV, outlining the transformative scientific potential that lies within reach, and the remarkable opportunities it offers for advancing our understanding of hadron physics and related fundamental phenomena. △ Less

Submitted 24 August, 2023; v1 submitted 13 June, 2023; originally announced June 2023.

Comments: Updates to the list of authors; Preprint number changed from theory to experiment; Updates to sections 4 and 6, including additional figures

Report number: JLAB-PHY-23-3840

Dynamics in near-threshold $J/ψ$ photoproduction

Authors: JPAC Collaboration, D. Winney, C. Fernandez-Ramirez, A. Pilloni, A. N. Hiller Blin, M. Albaladejo, L. Bibrzycki, N. Hammoud, J. Liao, V. Mathieu, G. Montana, R. J. Perry, V. Shastry, W. A. Smith, A. P. Szczepaniak

Abstract: The study of $J/ψ$ photoproduction at low energies has consequences for the understanding of multiple aspects of nonperturbative QCD, ranging from mechanical properties of the proton, to the binding inside nuclei, and the existence of hidden-charm pentaquarks. Factorization of the photon-$c \bar c$ and nucleon dynamics or Vector Meson Dominance are often invoked to justify these studies. Alternati… ▽ More The study of $J/ψ$ photoproduction at low energies has consequences for the understanding of multiple aspects of nonperturbative QCD, ranging from mechanical properties of the proton, to the binding inside nuclei, and the existence of hidden-charm pentaquarks. Factorization of the photon-$c \bar c$ and nucleon dynamics or Vector Meson Dominance are often invoked to justify these studies. Alternatively, open charm intermediate states have been proposed as the dominant mechanism underlying $J/ψ$ photoproduction. As the latter violates this factorization, it is important to estimate the relevance of such contributions. We analyse the latest differential and integrated photoproduction cross sections from the GlueX and $J/ψ$-007 experiments. We show that the data can be adequately described by a small number of partial waves, which we parameterize with generic models enforcing low-energy unitarity. The results suggest a nonnegligible contribution from open charm intermediate states. Furthermore, most of the models present an elastic scattering length incompatible with previous extractions based on Vector Meson Dominance, and thus call into question its applicability to heavy mesons. Our results indicate a wide array of physics possibilities that are compatible with present data and need to be disentangled. △ Less

Submitted 13 September, 2023; v1 submitted 2 May, 2023; originally announced May 2023.

Comments: 15 pages, 7 figures, 2 tables. Version to appear on Phys. Rev. D

Report number: JLAB-THY-23-3802

Journal ref: Phys. Rev. D 108 (2023) 5, 054018

Khuri-Treiman analysis of $J/ψ\toπ^{+}π^{-}π^{0}$

Authors: JPAC Collaboration, M. Albaladejo, S. Gonzàlez-Solís, Ł. Bibrzycki, C. Fernández-Ramírez, N. Hammoud, V. Mathieu, M. Mikhasenko, G. Montaña, R. J. Perry, A. Pilloni, A. Rodas, W. A. Smith, A. Szczepaniak, D. Winney

Abstract: We study the decay $J/ψ\toπ^{+}π^{-}π^{0}$ within the framework of the Khuri-Treiman equations. We find that the BESIII experimental di-pion mass distribution in the $ρ(770)$-region is well reproduced with a once-subtracted $P$-wave amplitude. Furthermore, we show that $F$-wave contributions to the amplitude improve the description of the data in the $ππ$ mass region around 1.5 GeV. We also presen… ▽ More We study the decay $J/ψ\toπ^{+}π^{-}π^{0}$ within the framework of the Khuri-Treiman equations. We find that the BESIII experimental di-pion mass distribution in the $ρ(770)$-region is well reproduced with a once-subtracted $P$-wave amplitude. Furthermore, we show that $F$-wave contributions to the amplitude improve the description of the data in the $ππ$ mass region around 1.5 GeV. We also present predictions for the $J/ψ\toπ^{0}γ^{*}$ transition form factor. △ Less

Submitted 19 April, 2023; originally announced April 2023.

Comments: 20 pages, 9 figures

Generating a Graph Colouring Heuristic with Deep Q-Learning and Graph Neural Networks

Authors: George Watkins, Giovanni Montana, Juergen Branke

Abstract: The graph colouring problem consists of assigning labels, or colours, to the vertices of a graph such that no two adjacent vertices share the same colour. In this work we investigate whether deep reinforcement learning can be used to discover a competitive construction heuristic for graph colouring. Our proposed approach, ReLCol, uses deep Q-learning together with a graph neural network for featur… ▽ More The graph colouring problem consists of assigning labels, or colours, to the vertices of a graph such that no two adjacent vertices share the same colour. In this work we investigate whether deep reinforcement learning can be used to discover a competitive construction heuristic for graph colouring. Our proposed approach, ReLCol, uses deep Q-learning together with a graph neural network for feature extraction, and employs a novel way of parameterising the graph that results in improved performance. Using standard benchmark graphs with varied topologies, we empirically evaluate the benefits and limitations of the heuristic learned by ReLCol relative to existing construction algorithms, and demonstrate that reinforcement learning is a promising direction for further research on the graph colouring problem. △ Less

Submitted 8 April, 2023; originally announced April 2023.

Comments: 15 pages, 6 figures, to be published in LION17 conference proceedings

Balancing policy constraint and ensemble size in uncertainty-based offline reinforcement learning

Authors: Alex Beeson, Giovanni Montana

Abstract: Offline reinforcement learning agents seek optimal policies from fixed data sets. With environmental interaction prohibited, agents face significant challenges in preventing errors in value estimates from compounding and subsequently causing the learning process to collapse. Uncertainty estimation using ensembles compensates for this by penalising high-variance value estimates, allowing agents to… ▽ More Offline reinforcement learning agents seek optimal policies from fixed data sets. With environmental interaction prohibited, agents face significant challenges in preventing errors in value estimates from compounding and subsequently causing the learning process to collapse. Uncertainty estimation using ensembles compensates for this by penalising high-variance value estimates, allowing agents to learn robust policies based on data-driven actions. However, the requirement for large ensembles to facilitate sufficient penalisation results in significant computational overhead. In this work, we examine the role of policy constraints as a mechanism for regulating uncertainty, and the corresponding balance between level of constraint and ensemble size. By incorporating behavioural cloning into policy updates, we show empirically that sufficient penalisation can be achieved with a much smaller ensemble size, substantially reducing computational demand while retaining state-of-the-art performance on benchmarking tasks. Furthermore, we show how such an approach can facilitate stable online fine tuning, allowing for continued policy improvement while avoiding severe performance drops. △ Less

Submitted 26 March, 2023; originally announced March 2023.

Goal-conditioned Offline Reinforcement Learning through State Space Partitioning

Authors: Mianchu Wang, Yue Jin, Giovanni Montana

Abstract: Offline reinforcement learning (RL) aims to infer sequential decision policies using only offline datasets. This is a particularly difficult setup, especially when learning to achieve multiple different goals or outcomes under a given scenario with only sparse rewards. For offline learning of goal-conditioned policies via supervised learning, previous work has shown that an advantage weighted log-… ▽ More Offline reinforcement learning (RL) aims to infer sequential decision policies using only offline datasets. This is a particularly difficult setup, especially when learning to achieve multiple different goals or outcomes under a given scenario with only sparse rewards. For offline learning of goal-conditioned policies via supervised learning, previous work has shown that an advantage weighted log-likelihood loss guarantees monotonic policy improvement. In this work we argue that, despite its benefits, this approach is still insufficient to fully address the distribution shift and multi-modality problems. The latter is particularly severe in long-horizon tasks where finding a unique and optimal policy that goes from a state to the desired goal is challenging as there may be multiple and potentially conflicting solutions. To tackle these challenges, we propose a complementary advantage-based weighting scheme that introduces an additional source of inductive bias: given a value-based partitioning of the state space, the contribution of actions expected to lead to target regions that are easier to reach, compared to the final goal, is further increased. Empirically, we demonstrate that the proposed approach, Dual-Advantage Weighted Offline Goal-conditioned RL (DAWOG), outperforms several competing offline algorithms in commonly used benchmarks. Analytically, we offer a guarantee that the learnt policy is never worse than the underlying behaviour policy. △ Less

Submitted 16 May, 2024; v1 submitted 16 March, 2023; originally announced March 2023.

Journal ref: Machine Learning (ECML-PKDD 2023 Journal Track)

Model-based trajectory stitching for improved behavioural cloning and its applications

Authors: Charles A. Hepburn, Giovanni Montana

Abstract: Behavioural cloning (BC) is a commonly used imitation learning method to infer a sequential decision-making policy from expert demonstrations. However, when the quality of the data is not optimal, the resulting behavioural policy also performs sub-optimally once deployed. Recently, there has been a surge in offline reinforcement learning methods that hold the promise to extract high-quality polici… ▽ More Behavioural cloning (BC) is a commonly used imitation learning method to infer a sequential decision-making policy from expert demonstrations. However, when the quality of the data is not optimal, the resulting behavioural policy also performs sub-optimally once deployed. Recently, there has been a surge in offline reinforcement learning methods that hold the promise to extract high-quality policies from sub-optimal historical data. A common approach is to perform regularisation during training, encouraging updates during policy evaluation and/or policy improvement to stay close to the underlying data. In this work, we investigate whether an offline approach to improving the quality of the existing data can lead to improved behavioural policies without any changes in the BC algorithm. The proposed data improvement approach - Trajectory Stitching (TS) - generates new trajectories (sequences of states and actions) by `stitching' pairs of states that were disconnected in the original data and generating their connecting new action. By construction, these new transitions are guaranteed to be highly plausible according to probabilistic models of the environment, and to improve a state-value function. We demonstrate that the iterative process of replacing old trajectories with new ones incrementally improves the underlying behavioural policy. Extensive experimental results show that significant performance gains can be achieved using TS over BC policies extracted from the original data. Furthermore, using the D4RL benchmarking suite, we demonstrate that state-of-the-art results are obtained by combining TS with two existing offline learning methodologies reliant on BC, model-based offline planning (MBOP) and policy constraint (TD3+BC). △ Less

Submitted 8 December, 2022; originally announced December 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2211.11603

Improving TD3-BC: Relaxed Policy Constraint for Offline Learning and Stable Online Fine-Tuning

Authors: Alex Beeson, Giovanni Montana

Abstract: The ability to discover optimal behaviour from fixed data sets has the potential to transfer the successes of reinforcement learning (RL) to domains where data collection is acutely problematic. In this offline setting, a key challenge is overcoming overestimation bias for actions not present in data which, without the ability to correct for via interaction with the environment, can propagate and… ▽ More The ability to discover optimal behaviour from fixed data sets has the potential to transfer the successes of reinforcement learning (RL) to domains where data collection is acutely problematic. In this offline setting, a key challenge is overcoming overestimation bias for actions not present in data which, without the ability to correct for via interaction with the environment, can propagate and compound during training, leading to highly sub-optimal policies. One simple method to reduce this bias is to introduce a policy constraint via behavioural cloning (BC), which encourages agents to pick actions closer to the source data. By finding the right balance between RL and BC such approaches have been shown to be surprisingly effective while requiring minimal changes to the underlying algorithms they are based on. To date this balance has been held constant, but in this work we explore the idea of tipping this balance towards RL following initial training. Using TD3-BC, we demonstrate that by continuing to train a policy offline while reducing the influence of the BC component we can produce refined policies that outperform the original baseline, as well as match or exceed the performance of more complex alternatives. Furthermore, we demonstrate such an approach can be used for stable online fine-tuning, allowing policies to be safely improved during deployment. △ Less

Submitted 21 November, 2022; originally announced November 2022.

Comments: 3rd Offline Reinforcement Learning Workshop at Neural Information Processing Systems, 2022

Model-based Trajectory Stitching for Improved Offline Reinforcement Learning

Authors: Charles A. Hepburn, Giovanni Montana

Abstract: In many real-world applications, collecting large and high-quality datasets may be too costly or impractical. Offline reinforcement learning (RL) aims to infer an optimal decision-making policy from a fixed set of data. Getting the most information from historical data is then vital for good performance once the policy is deployed. We propose a model-based data augmentation strategy, Trajectory St… ▽ More In many real-world applications, collecting large and high-quality datasets may be too costly or impractical. Offline reinforcement learning (RL) aims to infer an optimal decision-making policy from a fixed set of data. Getting the most information from historical data is then vital for good performance once the policy is deployed. We propose a model-based data augmentation strategy, Trajectory Stitching (TS), to improve the quality of sub-optimal historical trajectories. TS introduces unseen actions joining previously disconnected states: using a probabilistic notion of state reachability, it effectively `stitches' together parts of the historical demonstrations to generate new, higher quality ones. A stitching event consists of a transition between a pair of observed states through a synthetic and highly probable action. New actions are introduced only when they are expected to be beneficial, according to an estimated state-value function. We show that using this data augmentation strategy jointly with behavioural cloning (BC) leads to improvements over the behaviour-cloned policy from the original dataset. Improving over the BC policy could then be used as a launchpad for online RL through planning and demonstration-guided RL. △ Less

Submitted 21 November, 2022; originally announced November 2022.

Comments: Offline RL Workshop at Neural Information Processing Systems, 2022

$X(3872)$, $X(4014)$, and their bottom partners at finite temperature

Authors: Gloria Montaña, Angels Ramos, Laura Tolos, Juan M. Torres-Rincon

Abstract: The properties of the $X(3872)$ and its spin partner, the $X(4014)$, are studied both in vacuum and at finite temperature. Using an effective hadron theory based on the hidden-gauge Lagrangian, the $X(3872)$ is dynamically generated from the $s$-wave rescattering of a pair of pseudoscalar and vector charm mesons. By incorporating the thermal spectral functions of open charm mesons, the calculation… ▽ More The properties of the $X(3872)$ and its spin partner, the $X(4014)$, are studied both in vacuum and at finite temperature. Using an effective hadron theory based on the hidden-gauge Lagrangian, the $X(3872)$ is dynamically generated from the $s$-wave rescattering of a pair of pseudoscalar and vector charm mesons. By incorporating the thermal spectral functions of open charm mesons, the calculation is extended to finite temperature. Similarly, the properties of the $X(4014)$ are obtained out of the scattering of charm vector mesons. By applying heavy-quark flavor symmetry, the properties of their bottom counterparts in the axial-vector and tensor channels are also predicted. All the dynamically generated states show a decreasing mass and acquire an increasing decay width with temperature, following the trend observed in their meson constituents. These results are relevant in relativistic heavy-ion collisions at high energies, in analyses of the collective medium formed after hadronization or in femtoscopic studies, and can be tested in lattice-QCD calculations exploring the melting of heavy mesons at finite temperature. △ Less

Submitted 10 March, 2023; v1 submitted 3 November, 2022; originally announced November 2022.

Comments: 32 pages, 7 figures, 3 tables

Journal ref: Physical Review D 107, 054014 (2023)

Effective-theory description of heavy-flavored hadrons and their properties in a hot medium

Authors: Glòria Montaña

Abstract: This dissertation investigates exotic hadrons with heavy-quark content that may be understood as being generated dynamically from the hadron-hadron interaction. This interaction is derived from a suitable effective Lagrangian and properly unitarized in a full coupled-channel basis. In particular, we discuss the possible interpretation of some of the Ωc* excited states recently discovered at LHCb a… ▽ More This dissertation investigates exotic hadrons with heavy-quark content that may be understood as being generated dynamically from the hadron-hadron interaction. This interaction is derived from a suitable effective Lagrangian and properly unitarized in a full coupled-channel basis. In particular, we discuss the possible interpretation of some of the Ωc* excited states recently discovered at LHCb as being meson-baryon molecular states. We also discuss the dynamical generation of excited open-charm mesons from the scattering of pseudoscalar and vector charmed mesons off light mesons. We show that a double-pole structure is predicted for the D0*(2300) state, as well as for the D1(2430), while the Ds0*(2317) and the Ds1(2460) may be interpreted as molecular bound states. Extensions of these calculations to the bottom sector are also presented. Furthermore, we investigate the thermal modification of the open heavy-flavor mesons in a hot medium. By means of an extension to finite temperature of the unitarized effective interactions with the light mesons, we obtain the in-medium spectral properties of the D, D*, Ds, and Ds* ground-state mesons. We also analyze the temperature dependence of the masses and the decay widths of the dynamically generated states. Additionally, we provide results for the bottomed mesons by exploiting the heavy-quark spin-flavor symmetry of the Lagrangian. We employ the temperature-dependent spectral functions to compute charm Euclidean correlators. We also present calculations of off-shell transport coefficients in the hadronic phase implementing in-medium scattering amplitudes and the thermal dependence of the heavy-meson spectral properties. △ Less

Submitted 21 July, 2022; originally announced July 2022.

Comments: PhD Thesis, 226 pages. Universitat de Barcelona (defended on July 8th 2022)

Assessing the Performance of Automated Prediction and Ranking of Patient Age from Chest X-rays Against Clinicians

Authors: Matthew MacPherson, Keerthini Muthuswamy, Ashik Amlani, Charles Hutchinson, Vicky Goh, Giovanni Montana

Abstract: Understanding the internal physiological changes accompanying the aging process is an important aspect of medical image interpretation, with the expected changes acting as a baseline when reporting abnormal findings. Deep learning has recently been demonstrated to allow the accurate estimation of patient age from chest X-rays, and shows potential as a health indicator and mortality predictor. In t… ▽ More Understanding the internal physiological changes accompanying the aging process is an important aspect of medical image interpretation, with the expected changes acting as a baseline when reporting abnormal findings. Deep learning has recently been demonstrated to allow the accurate estimation of patient age from chest X-rays, and shows potential as a health indicator and mortality predictor. In this paper we present a novel comparative study of the relative performance of radiologists versus state-of-the-art deep learning models on two tasks: (a) patient age estimation from a single chest X-ray, and (b) ranking of two time-separated images of the same patient by age. We train our models with a heterogeneous database of 1.8M chest X-rays with ground truth patient ages and investigate the limitations on model accuracy imposed by limited training data and image resolution, and demonstrate generalisation performance on public data. To explore the large performance gap between the models and humans on these age-prediction tasks compared with other radiological reporting tasks seen in the literature, we incorporate our age prediction model into a conditional Generative Adversarial Network (cGAN) allowing visualisation of the semantic features identified by the prediction model as significant to age prediction, comparing the identified features with those relied on by clinicians. △ Less

Submitted 4 July, 2022; originally announced July 2022.

Comments: 13 pages, 8 figures, MICCAI 2022

LeNSE: Learning To Navigate Subgraph Embeddings for Large-Scale Combinatorial Optimisation

Authors: David Ireland, Giovanni Montana

Abstract: Combinatorial Optimisation problems arise in several application domains and are often formulated in terms of graphs. Many of these problems are NP-hard, but exact solutions are not always needed. Several heuristics have been developed to provide near-optimal solutions; however, they do not typically scale well with the size of the graph. We propose a low-complexity approach for identifying a (pos… ▽ More Combinatorial Optimisation problems arise in several application domains and are often formulated in terms of graphs. Many of these problems are NP-hard, but exact solutions are not always needed. Several heuristics have been developed to provide near-optimal solutions; however, they do not typically scale well with the size of the graph. We propose a low-complexity approach for identifying a (possibly much smaller) subgraph of the original graph where the heuristics can be run in reasonable time and with a high likelihood of finding a global near-optimal solution. The core component of our approach is LeNSE, a reinforcement learning algorithm that learns how to navigate the space of possible subgraphs using an Euclidean subgraph embedding as its map. To solve CO problems, LeNSE is provided with a discriminative embedding trained using any existing heuristics using only on a small portion of the original graph. When tested on three problems (vertex cover, max-cut and influence maximisation) using real graphs with up to $10$ million edges, LeNSE identifies small subgraphs yielding solutions comparable to those found by running the heuristics on the entire graph, but at a fraction of the total run time. △ Less

Submitted 20 May, 2022; originally announced May 2022.

Comments: To appear in ICML 2022

Thermal modification of open heavy-flavor mesons from an effective hadronic theory

Authors: G. Montana

Abstract: We have developed a self-consistent theoretical approach to study the modification of the properties of heavy mesons in hot mesonic matter which takes into account chiral and heavy-quark spin-flavor symmetries. The heavy-light meson-meson unitarized scattering amplitudes in coupled channels incorporate thermal corrections by using the imaginary-time formalism, as well as the dressing of the heavy… ▽ More We have developed a self-consistent theoretical approach to study the modification of the properties of heavy mesons in hot mesonic matter which takes into account chiral and heavy-quark spin-flavor symmetries. The heavy-light meson-meson unitarized scattering amplitudes in coupled channels incorporate thermal corrections by using the imaginary-time formalism, as well as the dressing of the heavy mesons with the self-energies. We report our results for the ground-state thermal spectral functions and the implications for the excited mesonic states generated dynamically in the heavy-light molecular model. We have applied these to the calculation of meson Euclidean correlators and transport coefficients for D mesons and summarize here our findings. △ Less

Submitted 19 November, 2021; originally announced November 2021.

Comments: 8 pages, 6 figures, Contribution to the Virtual Tribute to Quark Confinement and the Hadron Spectrum 2021 conference proceedings (vConf21) (submitted for publication in EPJ Web of Conferences)

Finite-temperature effects on D-meson properties

Authors: Juan M. Torres-Rincon, Glòria Montaña, Àngels Ramos, Laura Tolos

Abstract: We study the spectroscopy and transport properties of charmed mesons in a thermal medium by applying an effective field theory based on chiral and heavy-quark symmetries in the imaginary time formalism. Relying on unitarity constraints and self-consistency we extract the in-medium properties (masses and widths) of $D$ and $D_s$ mesons and their interactions with light hadrons. We report our findin… ▽ More We study the spectroscopy and transport properties of charmed mesons in a thermal medium by applying an effective field theory based on chiral and heavy-quark symmetries in the imaginary time formalism. Relying on unitarity constraints and self-consistency we extract the in-medium properties (masses and widths) of $D$ and $D_s$ mesons and their interactions with light hadrons. We report our findings on 1) dynamically generated states, 2) thermal evolution of chiral partners, 3) in-medium scattering amplitudes, and 4) transport coefficients below the chiral restoration temperature. △ Less

Submitted 16 September, 2021; originally announced September 2021.

Comments: 11 pages, 7 figures. Contribution to the 10th International Workshop on Charm Physics (CHARM2020). Accepted for publication in Proceedings of Science

Temperature dependence of the properties of open heavy-flavor mesons

Authors: Gloria Montana, Angels Ramos, Laura Tolos, Juan M. Torres-Rincon

Abstract: We address the modification of open heavy-flavor mesons in a hot medium of light mesons within an effective theory approach consistent with chiral and heavy-quark spin-flavor symmetries and the use of the imaginary time formalism to introduce the non-zero temperature effects to the theory. The unitarized scattering amplitudes, the ground-state self-energies and the corresponding spectral functions… ▽ More We address the modification of open heavy-flavor mesons in a hot medium of light mesons within an effective theory approach consistent with chiral and heavy-quark spin-flavor symmetries and the use of the imaginary time formalism to introduce the non-zero temperature effects to the theory. The unitarized scattering amplitudes, the ground-state self-energies and the corresponding spectral functions are calculated self-consistently. We use the thermal ground-state spectral functions obtained with this methodology to further calculate 1) open-charm meson Euclidean correlators, and 2) off-shell transport coefficients in the hadronic phase. △ Less

Submitted 10 August, 2021; originally announced August 2021.

Comments: 4 pages, 4 figures, contribution to the proceedings for the 19th International Conference on Strangeness in Quark Matter (SQM 2021), online 17-22 May 2021 (Submission to EPJ)

A persistent homology-based topological loss for CNN-based multi-class segmentation of CMR

Authors: Nick Byrne, James R Clough, Isra Valverde, Giovanni Montana, Andrew P King

Abstract: Multi-class segmentation of cardiac magnetic resonance (CMR) images seeks a separation of data into anatomical components with known structure and configuration. The most popular CNN-based methods are optimised using pixel wise loss functions, ignorant of the spatially extended features that characterise anatomy. Therefore, whilst sharing a high spatial overlap with the ground truth, inferred CNN-… ▽ More Multi-class segmentation of cardiac magnetic resonance (CMR) images seeks a separation of data into anatomical components with known structure and configuration. The most popular CNN-based methods are optimised using pixel wise loss functions, ignorant of the spatially extended features that characterise anatomy. Therefore, whilst sharing a high spatial overlap with the ground truth, inferred CNN-based segmentations can lack coherence, including spurious connected components, holes and voids. Such results are implausible, violating anticipated anatomical topology. In response, (single-class) persistent homology-based loss functions have been proposed to capture global anatomical features. Our work extends these approaches to the task of multi-class segmentation. Building an enriched topological description of all class labels and class label pairs, our loss functions make predictable and statistically significant improvements in segmentation topology using a CNN-based post-processing framework. We also present (and make available) a highly efficient implementation based on cubical complexes and parallel execution, enabling practical application within high resolution 3D data for the first time. We demonstrate our approach on 2D short axis and 3D whole heart CMR segmentation, advancing a detailed and faithful analysis of performance on two publicly available datasets. △ Less

Submitted 8 September, 2022; v1 submitted 27 July, 2021; originally announced July 2021.

Comments: Version accepted for publication in IEEE Transactions on Medical Imaging

In-medium kinetic theory of $D$ mesons and heavy-flavor transport coefficients

Authors: Juan M. Torres-Rincon, Glòria Montaña, Àngels Ramos, Laura Tolos

Abstract: We extend the kinetic theory of $D$ mesons to accommodate thermal and off-shell effects due to the medium modification of the heavy-meson spectral functions. From the Kadanoff-Baym approach we derive the off-shell Fokker-Planck equation which encodes the heavy-flavor transport coefficients. We analyze the thermal width (damping rate) of $D$ mesons due to their scattering off light mesons, focusing… ▽ More We extend the kinetic theory of $D$ mesons to accommodate thermal and off-shell effects due to the medium modification of the heavy-meson spectral functions. From the Kadanoff-Baym approach we derive the off-shell Fokker-Planck equation which encodes the heavy-flavor transport coefficients. We analyze the thermal width (damping rate) of $D$ mesons due to their scattering off light mesons, focusing on new in-medium effects: off-shell corrections, inelastic channels, and the contribution of the Landau cut. We obtain that the latter effect (absent for vacuum scattering amplitudes) brings sizable corrections at moderate temperatures. We discuss how the heavy-flavor transport coefficients, like the drag and diffusion coefficients, are modified in matter. We find that the $D$-meson spatial diffusion coefficient matches smoothly to the latest results of lattice-QCD calculations and Bayesian analyses at higher temperatures. △ Less

Submitted 16 February, 2022; v1 submitted 2 June, 2021; originally announced June 2021.

Comments: 47 pages, 14 figures. Theoretical foundations largely abridged (we refer to v1 for details). Extended discussions and added new figures. Results unmodified. Version published by Physical Review C journal

Journal ref: Physical Review C 105, 025203 (2022)

Automated Knee X-ray Report Generation

Authors: Aydan Gasimova, Giovanni Montana, Daniel Rueckert

Abstract: Gathering manually annotated images for the purpose of training a predictive model is far more challenging in the medical domain than for natural images as it requires the expertise of qualified radiologists. We therefore propose to take advantage of past radiological exams (specifically, knee X-ray examinations) and formulate a framework capable of learning the correspondence between the images a… ▽ More Gathering manually annotated images for the purpose of training a predictive model is far more challenging in the medical domain than for natural images as it requires the expertise of qualified radiologists. We therefore propose to take advantage of past radiological exams (specifically, knee X-ray examinations) and formulate a framework capable of learning the correspondence between the images and reports, and hence be capable of generating diagnostic reports for a given X-ray examination consisting of an arbitrary number of image views. We demonstrate how aggregating the image features of individual exams and using them as conditional inputs when training a language generation model results in auto-generated exam reports that correlate well with radiologist-generated reports. △ Less

Submitted 22 May, 2021; originally announced May 2021.

Journal ref: NeurIPS Machine Learning for Health Workshop 2017

Solving Challenging Dexterous Manipulation Tasks With Trajectory Optimisation and Reinforcement Learning

Authors: Henry Charlesworth, Giovanni Montana

Abstract: Training agents to autonomously learn how to use anthropomorphic robotic hands has the potential to lead to systems capable of performing a multitude of complex manipulation tasks in unstructured and uncertain environments. In this work, we first introduce a suite of challenging simulated manipulation tasks that current reinforcement learning and trajectory optimisation techniques find difficult.… ▽ More Training agents to autonomously learn how to use anthropomorphic robotic hands has the potential to lead to systems capable of performing a multitude of complex manipulation tasks in unstructured and uncertain environments. In this work, we first introduce a suite of challenging simulated manipulation tasks that current reinforcement learning and trajectory optimisation techniques find difficult. These include environments where two simulated hands have to pass or throw objects between each other, as well as an environment where the agent must learn to spin a long pen between its fingers. We then introduce a simple trajectory optimisation that performs significantly better than existing methods on these environments. Finally, on the challenging PenSpin task we combine sub-optimal demonstrations generated through trajectory optimisation with off-policy reinforcement learning, obtaining performance that far exceeds either of these approaches individually, effectively solving the environment. Videos of all of our results are available at: https://dexterous-manipulation.github.io/ △ Less

Submitted 16 May, 2021; v1 submitted 9 September, 2020; originally announced September 2020.

Comments: 9 pages

The molecular nature of some exotic hadrons

Authors: A. Ramos, A. Feijoo, Q. Llorens, G. Montaña

Abstract: The exciting discovery by LHCb of the $P_c(4312)^+$ and $P_c(4450)^+$ pentaquarks, or the suggestion of a tetraquark nature for the $Z_c(3900)$ state seen at BESIII and Belle, have triggered a lot of activity in the field of hadron physics, with new experiments planned for searching other exotic mesons and baryons, and many theoretical developments trying to disentangle the true multiquark nature… ▽ More The exciting discovery by LHCb of the $P_c(4312)^+$ and $P_c(4450)^+$ pentaquarks, or the suggestion of a tetraquark nature for the $Z_c(3900)$ state seen at BESIII and Belle, have triggered a lot of activity in the field of hadron physics, with new experiments planned for searching other exotic mesons and baryons, and many theoretical developments trying to disentangle the true multiquark nature from their possible molecular origin. After a brief review of the present status of these searches, this paper focusses on recently seen or yet to be discovered exotic heavy baryons that may emerge from a conveniently unitarized meson-baryon interaction model in coupled channels. In particular, we will show how interferences between the different coupled-channel amplitudes of the model may reveal the existence of a $N^*$ resonance around 2 GeV having a meson-baryon quasi-bound state nature. We also discuss the possible interpretation of some of the $Ω_c$ states recently discovered at LHCb as being hadron molecules. The model also predicts the existence of doubly-charmed quasibound meson-baryon $Ξ_{cc}$ states, which would be excited states of the ground-state $Ξ_{cc}(3621)$ MeV, whose mass has only been recently established. Extensions of these results to the bottom sector will also be presented. △ Less

Submitted 9 September, 2020; originally announced September 2020.

Comments: 18 pages, 9 figures, to appear in a Special Issue of Few-Body Systems

A persistent homology-based topological loss function for multi-class CNN segmentation of cardiac MRI

Authors: Nick Byrne, James R. Clough, Giovanni Montana, Andrew P. King

Abstract: With respect to spatial overlap, CNN-based segmentation of short axis cardiovascular magnetic resonance (CMR) images has achieved a level of performance consistent with inter observer variation. However, conventional training procedures frequently depend on pixel-wise loss functions, limiting optimisation with respect to extended or global features. As a result, inferred segmentations can lack spa… ▽ More With respect to spatial overlap, CNN-based segmentation of short axis cardiovascular magnetic resonance (CMR) images has achieved a level of performance consistent with inter observer variation. However, conventional training procedures frequently depend on pixel-wise loss functions, limiting optimisation with respect to extended or global features. As a result, inferred segmentations can lack spatial coherence, including spurious connected components or holes. Such results are implausible, violating the anticipated topology of image segments, which is frequently known a priori. Addressing this challenge, published work has employed persistent homology, constructing topological loss functions for the evaluation of image segments against an explicit prior. Building a richer description of segmentation topology by considering all possible labels and label pairs, we extend these losses to the task of multi-class segmentation. These topological priors allow us to resolve all topological errors in a subset of 150 examples from the ACDC short axis CMR training data set, without sacrificing overlap performance. △ Less

Submitted 21 August, 2020; originally announced August 2020.

Comments: To be presented at the STACOM workshop at MICCAI 2020

Follow the Object: Curriculum Learning for Manipulation Tasks with Imagined Goals

Authors: Ozsel Kilinc, Giovanni Montana

Abstract: Learning robot manipulation through deep reinforcement learning in environments with sparse rewards is a challenging task. In this paper we address this problem by introducing a notion of imaginary object goals. For a given manipulation task, the object of interest is first trained to reach a desired target position on its own, without being manipulated, through physically realistic simulations. T… ▽ More Learning robot manipulation through deep reinforcement learning in environments with sparse rewards is a challenging task. In this paper we address this problem by introducing a notion of imaginary object goals. For a given manipulation task, the object of interest is first trained to reach a desired target position on its own, without being manipulated, through physically realistic simulations. The object policy is then leveraged to build a predictive model of plausible object trajectories providing the robot with a curriculum of incrementally more difficult object goals to reach during training. The proposed algorithm, Follow the Object (FO), has been evaluated on 7 MuJoCo environments requiring increasing degree of exploration, and has achieved higher success rates compared to alternative algorithms. In particularly challenging learning scenarios, e.g. where the object's initial and target positions are far apart, our approach can still learn a policy whereas competing methods currently fail. △ Less

Submitted 11 November, 2021; v1 submitted 5 August, 2020; originally announced August 2020.

Comments: To appear in Deep Reinforcement Learning Workshop, NeurIPS 2021

Open-charm Euclidean correlators within heavy-meson EFT interactions

Authors: Glòria Montaña, Olaf Kaczmarek, Laura Tolos, Angels Ramos

Abstract: The open-charm Euclidean correlators have been computed for the first time using the thermal spectral functions extracted from a finite-temperature self-consistent unitarized approach based on a chiral effective field theory that implements heavy-quark spin symmetry. The inclusion of the full-energy dependent open-charm spectral functions in the calculation of the Euclidean correlators leads to a… ▽ More The open-charm Euclidean correlators have been computed for the first time using the thermal spectral functions extracted from a finite-temperature self-consistent unitarized approach based on a chiral effective field theory that implements heavy-quark spin symmetry. The inclusion of the full-energy dependent open-charm spectral functions in the calculation of the Euclidean correlators leads to a similar behaviour as the one obtained in lattice QCD for temperatures well below the transition deconfinement temperature. The discrepancies at temperatures close or above the transition deconfinement temperature could indicate that higher-energy states, that are not present in the open-charm spectral functions, become relevant for a quantitative description of the lattice QCD correlators at those temperatures. In fact, we find that the inclusion of a continuum of scattering states improves the comparison at small Euclidean times, whereas differences still arise for large times. △ Less

Submitted 30 July, 2020; originally announced July 2020.

Comments: 8 pages, 6 figures, contribution to EPJ A Special Issue on "Theory of hot matter and relativistic heavy-ion collisions (THOR)"

Pseudoscalar and vector open-charm mesons at finite temperature

Authors: Glòria Montaña, Àngels Ramos, Laura Tolos, Juan M. Torres-Rincon

Abstract: Vacuum and thermal properties of pseudoscalar and vector charm mesons are analyzed within a self-consistent many-body approach, employing a chiral effective field theory that incorporates heavy-quark spin symmetry. Upon unitarization of the vacuum interaction amplitudes for the scattering of charm mesons off light mesons in a fully coupled-channel basis, new dynamically generated states are search… ▽ More Vacuum and thermal properties of pseudoscalar and vector charm mesons are analyzed within a self-consistent many-body approach, employing a chiral effective field theory that incorporates heavy-quark spin symmetry. Upon unitarization of the vacuum interaction amplitudes for the scattering of charm mesons off light mesons in a fully coupled-channel basis, new dynamically generated states are searched. The imaginary-time formalism is employed to extend the calculation to finite temperatures up to $T=150$ MeV. Medium-modified spectral shapes of the $D$, $D^*$, $D_s$ and $D_s^*$ mesons are provided. The temperature dependence of the masses and decay widths of the nonstrange $D_0^*$ (2300) and $D_1^*$(2430) mesons, both showing a double-pole structure in the complex-energy plane, is also reported, as well as that of the $D_{s0}^*$(2317) and $D_{s1}^*$(2460) resonances and other states not yet identified experimentally. Being the first calculation incorporating open-charm vector mesons at finite temperature in a self-consistent fashion, it brings up the opportunity to discuss the medium effects on the open charm sector under the perspective of chiral and heavy-quark spin symmetries. △ Less

Submitted 12 November, 2020; v1 submitted 24 July, 2020; originally announced July 2020.

Comments: 47 pages, 10 figures, 8 tables. References added and typos removed. Additional explanations on unitarization procedure, medium modifications of pion, and cutoff parameter dependence. Version accepted for publication in Physical Review D journal

Journal ref: Phys. Rev. D 102, 096020 (2020)

PlanGAN: Model-based Planning With Sparse Rewards and Multiple Goals

Authors: Henry Charlesworth, Giovanni Montana

Abstract: Learning with sparse rewards remains a significant challenge in reinforcement learning (RL), especially when the aim is to train a policy capable of achieving multiple different goals. To date, the most successful approaches for dealing with multi-goal, sparse reward environments have been model-free RL algorithms. In this work we propose PlanGAN, a model-based algorithm specifically designed for… ▽ More Learning with sparse rewards remains a significant challenge in reinforcement learning (RL), especially when the aim is to train a policy capable of achieving multiple different goals. To date, the most successful approaches for dealing with multi-goal, sparse reward environments have been model-free RL algorithms. In this work we propose PlanGAN, a model-based algorithm specifically designed for solving multi-goal tasks in environments with sparse rewards. Our method builds on the fact that any trajectory of experience collected by an agent contains useful information about how to achieve the goals observed during that trajectory. We use this to train an ensemble of conditional generative models (GANs) to generate plausible trajectories that lead the agent from its current state towards a specified goal. We then combine these imagined trajectories into a novel planning algorithm in order to achieve the desired goal as efficiently as possible. The performance of PlanGAN has been tested on a number of robotic navigation/manipulation tasks in comparison with a range of model-free reinforcement learning baselines, including Hindsight Experience Replay. Our studies indicate that PlanGAN can achieve comparable performance whilst being around 4-8 times more sample efficient. △ Less

Submitted 1 June, 2020; originally announced June 2020.

Adaptive Experience Selection for Policy Gradient

Authors: Saad Mohamad, Giovanni Montana

Abstract: Policy gradient reinforcement learning (RL) algorithms have achieved impressive performance in challenging learning tasks such as continuous control, but suffer from high sample complexity. Experience replay is a commonly used approach to improve sample efficiency, but gradient estimators using past trajectories typically have high variance. Existing sampling strategies for experience replay like… ▽ More Policy gradient reinforcement learning (RL) algorithms have achieved impressive performance in challenging learning tasks such as continuous control, but suffer from high sample complexity. Experience replay is a commonly used approach to improve sample efficiency, but gradient estimators using past trajectories typically have high variance. Existing sampling strategies for experience replay like uniform sampling or prioritised experience replay do not explicitly try to control the variance of the gradient estimates. In this paper, we propose an online learning algorithm, adaptive experience selection (AES), to adaptively learn an experience sampling distribution that explicitly minimises this variance. Using a regret minimisation approach, AES iteratively updates the experience sampling distribution to match the performance of a competitor distribution assumed to have optimal variance. Sample non-stationarity is addressed by proposing a dynamic (i.e. time changing) competitor distribution for which a closed-form solution is proposed. We demonstrate that AES is a low-regret algorithm with reasonable sample complexity. Empirically, AES has been implemented for deep deterministic policy gradient and soft actor critic algorithms, and tested on 8 continuous control tasks from the OpenAI Gym library. Ours results show that AES leads to significantly improved performance compared to currently available experience sampling strategies for policy gradient. △ Less

Submitted 17 February, 2020; originally announced February 2020.

Learning Multi-Agent Coordination through Connectivity-driven Communication

Authors: Emanuele Pesce, Giovanni Montana

Abstract: In artificial multi-agent systems, the ability to learn collaborative policies is predicated upon the agents' communication skills: they must be able to encode the information received from the environment and learn how to share it with other agents as required by the task at hand. We present a deep reinforcement learning approach, Connectivity Driven Communication (CDC), that facilitates the emer… ▽ More In artificial multi-agent systems, the ability to learn collaborative policies is predicated upon the agents' communication skills: they must be able to encode the information received from the environment and learn how to share it with other agents as required by the task at hand. We present a deep reinforcement learning approach, Connectivity Driven Communication (CDC), that facilitates the emergence of multi-agent collaborative behaviour only through experience. The agents are modelled as nodes of a weighted graph whose state-dependent edges encode pair-wise messages that can be exchanged. We introduce a graph-dependent attention mechanisms that controls how the agents' incoming messages are weighted. This mechanism takes into full account the current state of the system as represented by the graph, and builds upon a diffusion process that captures how the information flows on the graph. The graph topology is not assumed to be known a priori, but depends dynamically on the agents' observations, and is learnt concurrently with the attention mechanism and policy in an end-to-end fashion. Our empirical results show that CDC is able to learn effective collaborative policies and can over-perform competing learning algorithms on cooperative navigation tasks. △ Less

Submitted 1 December, 2022; v1 submitted 12 February, 2020; originally announced February 2020.

Journal ref: Machine Learning (December 2022)

Impact of a thermal medium on $D$ mesons and their chiral partners

Authors: Glòria Montaña, Àngels Ramos, Laura Tolos, Juan M. Torres-Rincon

Abstract: We study $D$ and $D_s$ mesons at finite temperature using an effective field theory based on chiral and heavy-quark spin-flavor symmetries within the imaginary-time formalism. Interactions with the light degrees of freedom are unitarized via a Bethe-Salpeter approach, and the $D$ and $D_s$ self-energies are calculated self-consistently. We generate dynamically the $D^*_0(2300)$ and $D_s(2317)$ sta… ▽ More We study $D$ and $D_s$ mesons at finite temperature using an effective field theory based on chiral and heavy-quark spin-flavor symmetries within the imaginary-time formalism. Interactions with the light degrees of freedom are unitarized via a Bethe-Salpeter approach, and the $D$ and $D_s$ self-energies are calculated self-consistently. We generate dynamically the $D^*_0(2300)$ and $D_s(2317)$ states, and study their possible identification as the chiral partners of the $D$ and $D_s$ ground states, respectively. We show the evolution of their masses and decay widths as functions of temperature, and provide an analysis of the chiral-symmetry restoration in the heavy-flavor sector below the transition temperature. In particular, we analyse the very special case of the $D$-meson, for which the chiral partner is associated to the double-pole structure of the $D^*_0(2300)$. △ Less

Submitted 26 May, 2020; v1 submitted 31 January, 2020; originally announced January 2020.

Comments: 15 pages, 3 figures. v2: Corrected self-consistent calculation due to missing numerical factor. Conclusions remain unchanged. New references and discussion on validity of temperature range. Results on figure 3 provided as ancillary data. Version accepted by Physics Letters B journal

Journal ref: Physics Letters B, Volume 806, 2020, Article 135464

Reinforcement Learning for Robotic Manipulation using Simulated Locomotion Demonstrations

Authors: Ozsel Kilinc, Giovanni Montana

Abstract: Mastering robotic manipulation skills through reinforcement learning (RL) typically requires the design of shaped reward functions. Recent developments in this area have demonstrated that using sparse rewards, i.e. rewarding the agent only when the task has been successfully completed, can lead to better policies. However, state-action space exploration is more difficult in this case. Recent RL ap… ▽ More Mastering robotic manipulation skills through reinforcement learning (RL) typically requires the design of shaped reward functions. Recent developments in this area have demonstrated that using sparse rewards, i.e. rewarding the agent only when the task has been successfully completed, can lead to better policies. However, state-action space exploration is more difficult in this case. Recent RL approaches to learning with sparse rewards have leveraged high-quality human demonstrations for the task, but these can be costly, time consuming or even impossible to obtain. In this paper, we propose a novel and effective approach that does not require human demonstrations. We observe that every robotic manipulation task could be seen as involving a locomotion task from the perspective of the object being manipulated, i.e. the object could learn how to reach a target state on its own. In order to exploit this idea, we introduce a framework whereby an object locomotion policy is initially obtained using a realistic physics simulator. This policy is then used to generate auxiliary rewards, called simulated locomotion demonstration rewards (SLDRs), which enable us to learn the robot manipulation policy. The proposed approach has been evaluated on 13 tasks of increasing complexity, and can achieve higher success rate and faster learning rates compared to alternative algorithms. SLDRs are especially beneficial for tasks like multi-object stacking and non-rigid object manipulation. △ Less

Submitted 11 November, 2021; v1 submitted 16 October, 2019; originally announced October 2019.

Comments: To appear in ECML PKDD 2022

Properties of heavy mesons at finite temperature

Authors: Gloria Montaña, Angels Ramos, Laura Tolos

Abstract: We study the properties of heavy mesons using a unitarized approach in a hot pionic medium, based on an effective hadronic theory. The interaction between the heavy mesons and pseudoscalar Goldstone bosons is described by a chiral Lagrangian at next-to-leading order in the chiral expansion and leading order in the heavy-quark mass expansion so as to satisfy heavy-quark spin symmetry. The meson-mes… ▽ More We study the properties of heavy mesons using a unitarized approach in a hot pionic medium, based on an effective hadronic theory. The interaction between the heavy mesons and pseudoscalar Goldstone bosons is described by a chiral Lagrangian at next-to-leading order in the chiral expansion and leading order in the heavy-quark mass expansion so as to satisfy heavy-quark spin symmetry. The meson-meson scattering problem in coupled channels with finite-temperature corrections is solved in a self-consistent manner. Our results show that the masses of the ground-state charmed mesons $D(0^-)$ and $D_s(1^-)$ decrease in a pionic environment at $T\neq 0$ and they acquire a substantial width. As a consequence, the behaviour of excited mesonic states (i.e. $D_{s0}^*(2317)^\pm$ and $D_0^*(2300)^{0,\pm}$), generated dynamically in our heavy-light molecular model, is also modified at $T\neq 0$. The aim is to test our results against Lattice QCD calculations in the future. △ Less

Submitted 3 October, 2019; originally announced October 2019.

Comments: 10 pages, 4 figures, 1 table, contribution to the proceedings for the 24th edition of European Few Body Conference, July 2-6 September 2019, Surrey, UK (Submission to SciPost)

Topology-preserving augmentation for CNN-based segmentation of congenital heart defects from 3D paediatric CMR

Authors: Nick Byrne, James R. Clough, Isra Valverde, Giovanni Montana, Andrew P. King

Abstract: Patient-specific 3D printing of congenital heart anatomy demands an accurate segmentation of the thin tissue interfaces which characterise these diagnoses. Even when a label set has a high spatial overlap with the ground truth, inaccurate delineation of these interfaces can result in topological errors. These compromise the clinical utility of such models due to the anomalous appearance of defects… ▽ More Patient-specific 3D printing of congenital heart anatomy demands an accurate segmentation of the thin tissue interfaces which characterise these diagnoses. Even when a label set has a high spatial overlap with the ground truth, inaccurate delineation of these interfaces can result in topological errors. These compromise the clinical utility of such models due to the anomalous appearance of defects. CNNs have achieved state-of-the-art performance in segmentation tasks. Whilst data augmentation has often played an important role, we show that conventional image resampling schemes used therein can introduce topological changes in the ground truth labelling of augmented samples. We present a novel pipeline to correct for these changes, using a fast-marching algorithm to enforce the topology of the ground truth labels within their augmented representations. In so doing, we invoke the idea of cardiac contiguous topology to describe an arbitrary combination of congenital heart defects and develop an associated, clinically meaningful metric to measure the topological correctness of segmentations. In a series of five-fold cross-validations, we demonstrate the performance gain produced by this pipeline and the relevance of topological considerations to the segmentation of congenital heart defects. We speculate as to the applicability of this approach to any segmentation task involving morphologically complex targets. △ Less

Submitted 23 August, 2019; originally announced August 2019.

Comments: To be published at MICCAI PIPPI 2019

Skill Transfer in Deep Reinforcement Learning under Morphological Heterogeneity

Authors: Yang Hu, Giovanni Montana

Abstract: Transfer learning methods for reinforcement learning (RL) domains facilitate the acquisition of new skills using previously acquired knowledge. The vast majority of existing approaches assume that the agents have the same design, e.g. same shape and action spaces. In this paper we address the problem of transferring previously acquired skills amongst morphologically different agents (MDAs). For in… ▽ More Transfer learning methods for reinforcement learning (RL) domains facilitate the acquisition of new skills using previously acquired knowledge. The vast majority of existing approaches assume that the agents have the same design, e.g. same shape and action spaces. In this paper we address the problem of transferring previously acquired skills amongst morphologically different agents (MDAs). For instance, assuming that a bipedal agent has been trained to move forward, could this skill be transferred on to a one-leg hopper so as to make its training process for the same task more sample efficient? We frame this problem as one of subspace learning whereby we aim to infer latent factors representing the control mechanism that is common between MDAs. We propose a novel paired variational encoder-decoder model, PVED, that disentangles the control of MDAs into shared and agent-specific factors. The shared factors are then leveraged for skill transfer using RL. Theoretically, we derive a theorem indicating how the performance of PVED depends on the shared factors and agent morphologies. Experimentally, PVED has been extensively validated on four MuJoCo environments. We demonstrate its performance compared to a state-of-the-art approach and several ablation cases, visualize and interpret the hidden factors, and identify avenues for future improvements. △ Less

Submitted 20 August, 2019; v1 submitted 14 August, 2019; originally announced August 2019.

Spectral Multi-scale Community Detection in Temporal Networks with an Application

Authors: Zhana Kuncheva, Giovanni Montana

Abstract: The analysis of temporal networks has a wide area of applications in a world of technological advances. An important aspect of temporal network analysis is the discovery of community structures. Real data networks are often very large and the communities are observed to have a hierarchical structure referred to as multi-scale communities. Changes in the community structure over time might take pla… ▽ More The analysis of temporal networks has a wide area of applications in a world of technological advances. An important aspect of temporal network analysis is the discovery of community structures. Real data networks are often very large and the communities are observed to have a hierarchical structure referred to as multi-scale communities. Changes in the community structure over time might take place either at one scale or across all scales of the community structure. The multilayer formulation of the modularity maximization (MM) method introduced captures the changing multi-scale community structure of temporal networks. This method introduces a coupling between communities in neighboring time layers by allowing inter-layer connections, while different values of the resolution parameter enable the detection of multi-scale communities. However, the range of this parameter's values must be manually selected. When dealing with real life data, communities at one or more scales can go undiscovered if appropriate parameter ranges are not selected. A novel Temporal Multi-scale Community Detection (TMSCD) method overcomes the obstacles mentioned above. This is achieved by using the spectral properties of the temporal network represented as a multilayer network. In this framework we select automatically the range of relevant scales within which multi-scale community partitions are sought. △ Less

Submitted 29 January, 2019; originally announced January 2019.

Improving Coordination in Small-Scale Multi-Agent Deep Reinforcement Learning through Memory-driven Communication

Authors: Emanuele Pesce, Giovanni Montana

Abstract: Deep reinforcement learning algorithms have recently been used to train multiple interacting agents in a centralised manner whilst keeping their execution decentralised. When the agents can only acquire partial observations and are faced with tasks requiring coordination and synchronisation skills, inter-agent communication plays an essential role. In this work, we propose a framework for multi-ag… ▽ More Deep reinforcement learning algorithms have recently been used to train multiple interacting agents in a centralised manner whilst keeping their execution decentralised. When the agents can only acquire partial observations and are faced with tasks requiring coordination and synchronisation skills, inter-agent communication plays an essential role. In this work, we propose a framework for multi-agent training using deep deterministic policy gradients that enables concurrent, end-to-end learning of an explicit communication protocol through a memory device. During training, the agents learn to perform read and write operations enabling them to infer a shared representation of the world. We empirically demonstrate that concurrent learning of the communication device and individual policies can improve inter-agent coordination and performance in small-scale systems. Our experimental results show that the proposed method achieves superior performance in scenarios with up to six agents. We illustrate how different communication patterns can emerge on six different tasks of increasing complexity. Furthermore, we study the effects of corrupting the communication channel, provide a visualisation of the time-varying memory content as the underlying task is being solved and validate the building blocks of the proposed memory device through ablation studies. △ Less

Submitted 29 October, 2019; v1 submitted 12 January, 2019; originally announced January 2019.

Journal ref: Machine Learning (2020)

The molecular nature of some $Ω_c^0$ states

Authors: Gloria Montana, Angels Ramos, Albert Feijoo

Abstract: A vector meson exchange model based on effective Lagrangians is used to build the meson--baryon interaction in the charm $+1$, strangeness $-2$ and isospin $0$ sector. The s-wave scattering amplitudes resulting from the unitarization in coupled-channels show two resonances with masses and widths that are in very good agreement with those of the experimental $Ω_c(3050)^0$ and $Ω_c(3090)^0$ states o… ▽ More A vector meson exchange model based on effective Lagrangians is used to build the meson--baryon interaction in the charm $+1$, strangeness $-2$ and isospin $0$ sector. The s-wave scattering amplitudes resulting from the unitarization in coupled-channels show two resonances with masses and widths that are in very good agreement with those of the experimental $Ω_c(3050)^0$ and $Ω_c(3090)^0$ states observed by the LHCb collaboration. The interpretation of these resonances as pseudoscalar meson--baryon molecules would mean the assignment $J^P=1/2^-$ to their spin--parity. △ Less

Submitted 10 December, 2018; originally announced December 2018.

Comments: 6 pages, 2 figures, 2 tables, contribution to the proceedings of the XXII International Conference on Few-Body Problems in Physics (FB22) 2018, July 9-13, Caen, France. arXiv admin note: substantial text overlap with arXiv:1812.03890

Exotic $Ω_c^0$ baryons from meson-baryon scattering

Authors: Gloria Montana, Angels Ramos, Albert Feijoo

Abstract: A meson-baryon interaction in the charm $+1$, strangeness $-2$ and isospin $0$ sector is built from a t-channel vector meson exchange model employing effective Lagrangians. The implementation of coupled-channel unitarization in the s-wave scattering amplitudes gives rise to two structures that have similar masses and widths to those of the $Ω_c(3050)^0$ and $Ω_c(3090)^0$ states recently observed b… ▽ More A meson-baryon interaction in the charm $+1$, strangeness $-2$ and isospin $0$ sector is built from a t-channel vector meson exchange model employing effective Lagrangians. The implementation of coupled-channel unitarization in the s-wave scattering amplitudes gives rise to two structures that have similar masses and widths to those of the $Ω_c(3050)^0$ and $Ω_c(3090)^0$ states recently observed by the LHCb collaboration. A meson-baryon molecular interpretation of these resonances would assign their spin-parity to be $J^P=1/2^-$. △ Less

Submitted 10 December, 2018; originally announced December 2018.

Comments: Proceedings of the talk by G. Montana at BEACH2018 - XIII International Conference on Beauty, Charm and Hyperon Hadrons in Peniche, Portugal, 17-23 June 2018; 5 pages, 1 figure, 2 tables. arXiv admin note: substantial text overlap with arXiv:1812.03898

Journal ref: Journal of Physics: Conf. Series 1137 (2019) 012040

Multi-agent Deep Reinforcement Learning with Extremely Noisy Observations

Authors: Ozsel Kilinc, Giovanni Montana

Abstract: Multi-agent reinforcement learning systems aim to provide interacting agents with the ability to collaboratively learn and adapt to the behaviour of other agents. In many real-world applications, the agents can only acquire a partial view of the world. Here we consider a setting whereby most agents' observations are also extremely noisy, hence only weakly correlated to the true state of the enviro… ▽ More Multi-agent reinforcement learning systems aim to provide interacting agents with the ability to collaboratively learn and adapt to the behaviour of other agents. In many real-world applications, the agents can only acquire a partial view of the world. Here we consider a setting whereby most agents' observations are also extremely noisy, hence only weakly correlated to the true state of the environment. Under these circumstances, learning an optimal policy becomes particularly challenging, even in the unrealistic case that an agent's policy can be made conditional upon all other agents' observations. To overcome these difficulties, we propose a multi-agent deep deterministic policy gradient algorithm enhanced by a communication medium (MADDPG-M), which implements a two-level, concurrent learning mechanism. An agent's policy depends on its own private observations as well as those explicitly shared by others through a communication medium. At any given point in time, an agent must decide whether its private observations are sufficiently informative to be shared with others. However, our environments provide no explicit feedback informing an agent whether a communication action is beneficial, rather the communication policies must also be learned through experience concurrently to the main policies. Our experimental results demonstrate that the algorithm performs well in six highly non-stationary environments of progressively higher complexity, and offers substantial performance gains compared to the baselines. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Comments: To appear in Deep Reinforcement Learning Workshop, NIPS 2018

Constraining twin stars with GW170817

Authors: Gloria Montana, Laura Tolos, Matthias Hanauske, Luciano Rezzolla

Abstract: If a phase transition is allowed to take place in the core of a compact star, a new stable branch of equilibrium configurations can appear, providing solutions with the same mass as the purely hadronic branch and hence giving rise to twin-star configurations. We perform an extensive analysis of the features of the phase transition leading twin-star configurations and, at the same time, fulfilling… ▽ More If a phase transition is allowed to take place in the core of a compact star, a new stable branch of equilibrium configurations can appear, providing solutions with the same mass as the purely hadronic branch and hence giving rise to twin-star configurations. We perform an extensive analysis of the features of the phase transition leading twin-star configurations and, at the same time, fulfilling the constraints coming from the maximum mass of $2M_\odot$ and the information following gravitational-wave event GW170817. In particular, we use a general equation of state for the neutron-star matter that parametrizes the hadron-quark phase transition between the model describing the hadronic phase and a constant speed of sound for the quark phase. We find that the largest number of twin-star solutions has masses in the neutron-star branch in the range $1-2M_\odot$ and twin-branch masses $\gtrsim 2M_\odot$. The analysis of the masses, radii and tidal deformabilities also reveals that when twin stars appear, the tidal deformability shows two distinct branches with the same mass, thus differing considerably from the behaviour expected for neutron stars. In addition, we find that the data from GW170817 is compatible with the existence of hybrid stars and could also be interpreted as produced by the merger of a binary system of hybrid stars or of a hybrid star with a neutron star. The presence of a hybrid star in the inspiral phase can be established clearly if future gravitational-wave detections measure chirp masses $\mathcal{M}\lesssim 1.2M_\odot$ and tidal deformabilities of $Λ_{1.4}\lesssim 400$ for $1.4M_\odot$ stars. Finally, combining all observational information available, we set constraints on the parameters that characterise the phase transition, the maximum masses, and the radii of $1.4M_\odot$ stars described by equations of state leading to twin-star configurations. △ Less

Submitted 11 June, 2019; v1 submitted 27 November, 2018; originally announced November 2018.

Comments: 17 pages, 12 figures, 2 tables

Journal ref: Phys. Rev. D 99, 103009 (2019)

A Generative Adversarial Model for Right Ventricle Segmentation

Authors: Nicoló Savioli, Miguel Silva Vieira, Pablo Lamata, Giovanni Montana

Abstract: The clinical management of several cardiovascular conditions, such as pulmonary hypertension, require the assessment of the right ventricular (RV) function. This work addresses the fully automatic and robust access to one of the key RV biomarkers, its ejection fraction, from the gold standard imaging modality, MRI. The problem becomes the accurate segmentation of the RV blood pool from cine MRI se… ▽ More The clinical management of several cardiovascular conditions, such as pulmonary hypertension, require the assessment of the right ventricular (RV) function. This work addresses the fully automatic and robust access to one of the key RV biomarkers, its ejection fraction, from the gold standard imaging modality, MRI. The problem becomes the accurate segmentation of the RV blood pool from cine MRI sequences. This work proposes a solution based on Fully Convolutional Neural Networks (FCNN), where our first contribution is the optimal combination of three concepts (the convolution Gated Recurrent Units (GRU), the Generative Adversarial Networks (GAN), and the L1 loss function) that achieves an improvement of 0.05 and 3.49 mm in Dice Index and Hausdorff Distance respectively with respect to the baseline FCNN. This improvement is then doubled by our second contribution, the ROI-GAN, that sets two GANs to cooperate working at two fields of view of the image, its full resolution and the region of interest (ROI). Our rationale here is to better guide the FCNN learning by combining global (full resolution) and local Region Of Interest (ROI) features. The study is conducted in a large in-house dataset of $\sim$ 23.000 segmented MRI slices, and its generality is verified in a publicly available dataset. △ Less

Submitted 27 September, 2018; originally announced October 2018.

Comments: 9 pages, 8 figures

Automated segmentation on the entire cardiac cycle using a deep learning work-flow

Authors: Nicoló Savioli, Miguel Silva Vieira, Pablo Lamata, Giovanni Montana

Abstract: The segmentation of the left ventricle (LV) from CINE MRI images is essential to infer important clinical parameters. Typically, machine learning algorithms for automated LV segmentation use annotated contours from only two cardiac phases, diastole, and systole. In this work, we present an analysis work-flow for fully-automated LV segmentation that learns from images acquired through the cardiac c… ▽ More The segmentation of the left ventricle (LV) from CINE MRI images is essential to infer important clinical parameters. Typically, machine learning algorithms for automated LV segmentation use annotated contours from only two cardiac phases, diastole, and systole. In this work, we present an analysis work-flow for fully-automated LV segmentation that learns from images acquired through the cardiac cycle. The workflow consists of three components: first, for each image in the sequence, we perform an automated localization and subsequent cropping of the bounding box containing the cardiac silhouette. Second, we identify the LV contours using a Temporal Fully Convolutional Neural Network (T-FCNN), which extends Fully Convolutional Neural Networks (FCNN) through a recurrent mechanism enforcing temporal coherence across consecutive frames. Finally, we further defined the boundaries using either one of two components: fully-connected Conditional Random Fields (CRFs) with Gaussian edge potentials and Semantic Flow. Our initial experiments suggest that significant improvement in performance can potentially be achieved by using a recurrent neural network component that explicitly learns cardiac motion patterns whilst performing LV segmentation. △ Less

Submitted 31 August, 2018; originally announced September 2018.

Comments: 6 pages, 2 figures, published on IEEE Xplore