Showing 1–50 of 80 results for author: Rezaei, S

Finite Operator Learning: Bridging Neural Operators and Numerical Methods for Efficient Parametric Solution and Optimization of PDEs

Authors: Shahed Rezaei, Reza Najian Asl, Kianoosh Taghikhani, Ahmad Moeineddin, Michael Kaliske, Markus Apel

Abstract: We introduce a method that combines neural operators, physics-informed machine learning, and standard numerical methods for solving PDEs. The proposed approach extends each of the aforementioned methods and unifies them within a single framework. We can parametrically solve partial differential equations in a data-free manner and provide accurate sensitivities, meaning the derivatives of the solut… ▽ More We introduce a method that combines neural operators, physics-informed machine learning, and standard numerical methods for solving PDEs. The proposed approach extends each of the aforementioned methods and unifies them within a single framework. We can parametrically solve partial differential equations in a data-free manner and provide accurate sensitivities, meaning the derivatives of the solution space with respect to the design space. These capabilities enable gradient-based optimization without the typical sensitivity analysis costs, unlike adjoint methods that scale directly with the number of response functions. Our Finite Operator Learning (FOL) approach uses an uncomplicated feed-forward neural network model to directly map the discrete design space (i.e. parametric input space) to the discrete solution space (i.e. finite number of sensor points in the arbitrary shape domain) ensuring compliance with physical laws by designing them into loss functions. The discretized governing equations, as well as the design and solution spaces, can be derived from any well-established numerical techniques. In this work, we employ the Finite Element Method (FEM) to approximate fields and their spatial derivatives. Subsequently, we conduct Sobolev training to minimize a multi-objective loss function, which includes the discretized weak form of the energy functional, boundary conditions violations, and the stationarity of the residuals with respect to the design variables. Our study focuses on the steady-state heat equation within heterogeneous materials that exhibits significant phase contrast and possibly temperature-dependent conductivity. The network's tangent matrix is directly used for gradient-based optimization to improve the microstructure's heat transfer characteristics. ... △ Less

Submitted 4 July, 2024; originally announced July 2024.

Comments: arXiv admin note: text overlap with arXiv:2401.02363

A finite element-based physics-informed operator learning framework for spatiotemporal partial differential equations on arbitrary domains

Authors: Yusuke Yamazaki, Ali Harandi, Mayu Muramatsu, Alexandre Viardin, Markus Apel, Tim Brepols, Stefanie Reese, Shahed Rezaei

Abstract: We propose a novel finite element-based physics-informed operator learning framework that allows for predicting spatiotemporal dynamics governed by partial differential equations (PDEs). The proposed framework employs a loss function inspired by the finite element method (FEM) with the implicit Euler time integration scheme. A transient thermal conduction problem is considered to benchmark the per… ▽ More We propose a novel finite element-based physics-informed operator learning framework that allows for predicting spatiotemporal dynamics governed by partial differential equations (PDEs). The proposed framework employs a loss function inspired by the finite element method (FEM) with the implicit Euler time integration scheme. A transient thermal conduction problem is considered to benchmark the performance. The proposed operator learning framework takes a temperature field at the current time step as input and predicts a temperature field at the next time step. The Galerkin discretized weak formulation of the heat equation is employed to incorporate physics into the loss function, which is coined finite operator learning (FOL). Upon training, the networks successfully predict the temperature evolution over time for any initial temperature field at high accuracy compared to the FEM solution. The framework is also confirmed to be applicable to a heterogeneous thermal conductivity and arbitrary geometry. The advantages of FOL can be summarized as follows: First, the training is performed in an unsupervised manner, avoiding the need for a large data set prepared from costly simulations or experiments. Instead, random temperature patterns generated by the Gaussian random process and the Fourier series, combined with constant temperature fields, are used as training data to cover possible temperature cases. Second, shape functions and backward difference approximation are exploited for the domain discretization, resulting in a purely algebraic equation. This enhances training efficiency, as one avoids time-consuming automatic differentiation when optimizing weights and biases while accepting possible discretization errors. Finally, thanks to the interpolation power of FEM, any arbitrary geometry can be handled with FOL, which is crucial to addressing various engineering application scenarios. △ Less

Submitted 22 May, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

Introducing a microstructure-embedded autoencoder approach for reconstructing high-resolution solution field data from a reduced parametric space

Authors: Rasoul Najafi Koopas, Shahed Rezaei, Natalie Rauter, Richard Ostwald, Rolf Lammering

Abstract: In this study, we develop a novel multi-fidelity deep learning approach that transforms low-fidelity solution maps into high-fidelity ones by incorporating parametric space information into a standard autoencoder architecture. This method's integration of parametric space information significantly reduces the need for training data to effectively predict high-fidelity solutions from low-fidelity o… ▽ More In this study, we develop a novel multi-fidelity deep learning approach that transforms low-fidelity solution maps into high-fidelity ones by incorporating parametric space information into a standard autoencoder architecture. This method's integration of parametric space information significantly reduces the need for training data to effectively predict high-fidelity solutions from low-fidelity ones. In this study, we examine a two-dimensional steady-state heat transfer analysis within a highly heterogeneous materials microstructure. The heat conductivity coefficients for two different materials are condensed from a 101 x 101 grid to smaller grids. We then solve the boundary value problem on the coarsest grid using a pre-trained physics-informed neural operator network known as Finite Operator Learning (FOL). The resulting low-fidelity solution is subsequently upscaled back to a 101 x 101 grid using a newly designed enhanced autoencoder. The novelty of the developed enhanced autoencoder lies in the concatenation of heat conductivity maps of different resolutions to the decoder segment in distinct steps. Hence the developed algorithm is named microstructure-embedded autoencoder (MEA). We compare the MEA outcomes with those from finite element methods, the standard U-Net, and various other upscaling techniques, including interpolation functions and feedforward neural networks (FFNN). Our analysis shows that MEA outperforms these methods in terms of computational efficiency and error on test cases. As a result, the MEA serves as a potential supplement to neural operator networks, effectively upscaling low-fidelity solutions to high fidelity while preserving critical details often lost in traditional upscaling methods, particularly at sharp interfaces like those seen with interpolation. △ Less

Submitted 7 May, 2024; v1 submitted 3 May, 2024; originally announced May 2024.

A finite operator learning technique for mapping the elastic properties of microstructures to their mechanical deformations

Authors: Shahed Rezaei, Reza Najian Asl, Shirko Faroughi, Mahdi Asgharzadeh, Ali Harandi, Rasoul Najafi Koopas, Gottfried Laschet, Stefanie Reese, Markus Apel

Abstract: To obtain fast solutions for governing physical equations in solid mechanics, we introduce a method that integrates the core ideas of the finite element method with physics-informed neural networks and concept of neural operators. This approach generalizes and enhances each method, learning the parametric solution for mechanical problems without relying on data from other resources (e.g. other num… ▽ More To obtain fast solutions for governing physical equations in solid mechanics, we introduce a method that integrates the core ideas of the finite element method with physics-informed neural networks and concept of neural operators. This approach generalizes and enhances each method, learning the parametric solution for mechanical problems without relying on data from other resources (e.g. other numerical solvers). We propose directly utilizing the available discretized weak form in finite element packages to construct the loss functions algebraically, thereby demonstrating the ability to find solutions even in the presence of sharp discontinuities. Our focus is on micromechanics as an example, where knowledge of deformation and stress fields for a given heterogeneous microstructure is crucial for further design applications. The primary parameter under investigation is the Young's modulus distribution within the heterogeneous solid system. Our investigations reveal that physics-based training yields higher accuracy compared to purely data-driven approaches for unseen microstructures. Additionally, we offer two methods to directly improve the process of obtaining high-resolution solutions, avoiding the need to use basic interpolation techniques. First is based on an autoencoder approach to enhance the efficiency for calculation on high resolution grid point. Next, Fourier-based parametrization is utilized to address complex 2D and 3D problems in micromechanics. The latter idea aims to represent complex microstructures efficiently using Fourier coefficients. Comparisons with other well-known operator learning algorithms, further emphasize the advantages of the newly proposed method. △ Less

Submitted 3 June, 2024; v1 submitted 28 March, 2024; originally announced April 2024.

Ultralight vector dark matter search using data from the KAGRA O3GK run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, A. G. Abac, R. Abbott, H. Abe, I. Abouelfettouh, F. Acernese, K. Ackley, C. Adamcewicz, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, I. Aguilar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi , et al. (1778 additional authors not shown)

Abstract: Among the various candidates for dark matter (DM), ultralight vector DM can be probed by laser interferometric gravitational wave detectors through the measurement of oscillating length changes in the arm cavities. In this context, KAGRA has a unique feature due to differing compositions of its mirrors, enhancing the signal of vector DM in the length change in the auxiliary channels. Here we prese… ▽ More Among the various candidates for dark matter (DM), ultralight vector DM can be probed by laser interferometric gravitational wave detectors through the measurement of oscillating length changes in the arm cavities. In this context, KAGRA has a unique feature due to differing compositions of its mirrors, enhancing the signal of vector DM in the length change in the auxiliary channels. Here we present the result of a search for $U(1)_{B-L}$ gauge boson DM using the KAGRA data from auxiliary length channels during the first joint observation run together with GEO600. By applying our search pipeline, which takes into account the stochastic nature of ultralight DM, upper bounds on the coupling strength between the $U(1)_{B-L}$ gauge boson and ordinary matter are obtained for a range of DM masses. While our constraints are less stringent than those derived from previous experiments, this study demonstrates the applicability of our method to the lower-mass vector DM search, which is made difficult in this measurement by the short observation time compared to the auto-correlation time scale of DM. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Comments: 20 pages, 5 figures

Report number: LIGO-P2300250

Edge Caching Based on Deep Reinforcement Learning and Transfer Learning

Authors: Farnaz Niknia, Ping Wang, Zixu Wang, Aakash Agarwal, Adib S. Rezaei

Abstract: This paper addresses the escalating challenge of redundant data transmission in networks. The surge in traffic has strained backhaul links and backbone networks, prompting the exploration of caching solutions at the edge router. Existing work primarily relies on Markov Decision Processes (MDP) for caching issues, assuming fixed-time interval decisions; however, real-world scenarios involve random… ▽ More This paper addresses the escalating challenge of redundant data transmission in networks. The surge in traffic has strained backhaul links and backbone networks, prompting the exploration of caching solutions at the edge router. Existing work primarily relies on Markov Decision Processes (MDP) for caching issues, assuming fixed-time interval decisions; however, real-world scenarios involve random request arrivals, and despite the critical role of various file characteristics in determining an optimal caching policy, none of the related existing work considers all these file characteristics in forming a caching policy. In this paper, first, we formulate the caching problem using a semi-Markov Decision Process (SMDP) to accommodate the continuous-time nature of real-world scenarios allowing for caching decisions at random times upon file requests. Then, we propose a double deep Q-learning-based caching approach that comprehensively accounts for file features such as lifetime, size, and importance. Simulation results demonstrate the superior performance of our approach compared to a recent Deep Reinforcement Learning-based method. Furthermore, we extend our work to include a Transfer Learning (TL) approach to account for changes in file request rates in the SMDP framework. The proposed TL approach exhibits fast convergence, even in scenarios with increased differences in request rates between source and target domains, presenting a promising solution to the dynamic challenges of caching in real-world environments. △ Less

Submitted 29 February, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Integration of physics-informed operator learning and finite element method for parametric learning of partial differential equations

Authors: Shahed Rezaei, Ahmad Moeineddin, Michael Kaliske, Markus Apel

Abstract: We present a method that employs physics-informed deep learning techniques for parametrically solving partial differential equations. The focus is on the steady-state heat equations within heterogeneous solids exhibiting significant phase contrast. Similar equations manifest in diverse applications like chemical diffusion, electrostatics, and Darcy flow. The neural network aims to establish the li… ▽ More We present a method that employs physics-informed deep learning techniques for parametrically solving partial differential equations. The focus is on the steady-state heat equations within heterogeneous solids exhibiting significant phase contrast. Similar equations manifest in diverse applications like chemical diffusion, electrostatics, and Darcy flow. The neural network aims to establish the link between the complex thermal conductivity profiles and temperature distributions, as well as heat flux components within the microstructure, under fixed boundary conditions. A distinctive aspect is our independence from classical solvers like finite element methods for data. A noteworthy contribution lies in our novel approach to defining the loss function, based on the discretized weak form of the governing equation. This not only reduces the required order of derivatives but also eliminates the need for automatic differentiation in the construction of loss terms, accepting potential numerical errors from the chosen discretization method. As a result, the loss function in this work is an algebraic equation that significantly enhances training efficiency. We benchmark our methodology against the standard finite element method, demonstrating accurate yet faster predictions using the trained neural network for temperature and flux profiles. We also show higher accuracy by using the proposed method compared to purely data-driven approaches for unforeseen scenarios. △ Less

Submitted 4 January, 2024; originally announced January 2024.

Comparative analysis of phase-field and intrinsic cohesive zone models for fracture simulations in multiphase materials with interfaces: Investigation of the influence of the microstructure on the fracture properties

Authors: Rasoul Najafi Koopas, Shahed Rezaei, Natalie Rauter, Richard Ostwald, Rolf Lammering

Abstract: This study evaluates four widely used fracture simulation methods, comparing their computational expenses and implementation complexities within the Finite Element (FE) framework when employed on heterogeneous solids. Fracture methods considered encompass the intrinsic Cohesive Zone Model (CZM) using zero-thickness cohesive interface elements (CIEs), the Standard Phase-Field Fracture (SPFM) approa… ▽ More This study evaluates four widely used fracture simulation methods, comparing their computational expenses and implementation complexities within the Finite Element (FE) framework when employed on heterogeneous solids. Fracture methods considered encompass the intrinsic Cohesive Zone Model (CZM) using zero-thickness cohesive interface elements (CIEs), the Standard Phase-Field Fracture (SPFM) approach, the Cohesive Phase-Field fracture (CPFM) approach, and an innovative hybrid model. The hybrid approach combines the CPFM fracture method with the CZM, specifically applying the CZM within the interface zone. A significant finding from this investigation is that the CPFM method is in agreement with the hybrid model when the interface zone thickness is not excessively small. This implies that the CPFM fracture methodology may serve as a unified fracture approach for multiphase materials, provided the interface zone's thickness is comparable to that of the other phases. In addition, this research provides valuable insights that can advance efforts to fine-tune material microstructures. An investigation of the influence of the interface material properties, morphological features and spatial arrangement of inclusions showes a pronounced effect of these parameters on the fracture toughness of the material. △ Less

Submitted 29 January, 2024; v1 submitted 28 November, 2023; originally announced November 2023.

MSC Class: 74R99

Dynamic Batch Norm Statistics Update for Natural Robustness

Authors: Shahbaz Rezaei, Mohammad Sadegh Norouzzadeh

Abstract: DNNs trained on natural clean samples have been shown to perform poorly on corrupted samples, such as noisy or blurry images. Various data augmentation methods have been recently proposed to improve DNN's robustness against common corruptions. Despite their success, they require computationally expensive training and cannot be applied to off-the-shelf trained models. Recently, it has been shown th… ▽ More DNNs trained on natural clean samples have been shown to perform poorly on corrupted samples, such as noisy or blurry images. Various data augmentation methods have been recently proposed to improve DNN's robustness against common corruptions. Despite their success, they require computationally expensive training and cannot be applied to off-the-shelf trained models. Recently, it has been shown that updating BatchNorm (BN) statistics of an off-the-shelf model on a single corruption improves its accuracy on that corruption significantly. However, adopting the idea at inference time when the type of corruption is unknown and changing decreases the effectiveness of this method. In this paper, we harness the Fourier domain to detect the corruption type, a challenging task in the image domain. We propose a unified framework consisting of a corruption-detection model and BN statistics update that improves the corruption accuracy of any off-the-shelf trained model. We benchmark our framework on different models and datasets. Our results demonstrate about 8% and 4% accuracy improvement on CIFAR10-C and ImageNet-C, respectively. Furthermore, our framework can further improve the accuracy of state-of-the-art robust models, such as AugMix and DeepAug. △ Less

Submitted 31 October, 2023; originally announced October 2023.

Silicon Nanoantenna Mix Arrays for a Trifecta of Quantum Emitter Enhancements

Authors: Zhaogang Dong, Sergey Gorelik, Ramón Paniagua-Dominguez, Johnathan Yik, Jinfa Ho, Febiana Tjiptoharsono, Emmanuel Lassalle, Soroosh Daqiqeh Rezaei, Darren C. J. Neo, Ping Bai, Arseniy I. Kuznetsov, Joel K. W. Yang

Abstract: Dielectric nanostructures have demonstrated optical antenna effects due to Mie resonances. Preliminary investigations on dielectric nanoantennas have been carried out for a trifecta of enhancements, i.e., simultaneous enhancements in absorption, emission directionality and radiative decay rates of quantum emitters. However, these investigations are limited by fragile substrates or low Purcell fact… ▽ More Dielectric nanostructures have demonstrated optical antenna effects due to Mie resonances. Preliminary investigations on dielectric nanoantennas have been carried out for a trifecta of enhancements, i.e., simultaneous enhancements in absorption, emission directionality and radiative decay rates of quantum emitters. However, these investigations are limited by fragile substrates or low Purcell factor, which is extremely important for exciting quantum emitters electrically. In this paper, we present a Si mix antenna array to achieve the trifecta enhancement of ~1200 fold with a Purcell factor of ~47. The antenna design incorporates ~10 nm gaps within which fluorescent molecules strongly absorb the pump laser energy through a resonant mode. In the emission process, the antenna array increases the radiative decay rates of the fluorescence molecules via Purcell effect and provides directional emission through a separate mode. This work could lead to novel CMOS compatible platforms for enhancing fluorescence for biological and chemical applications. △ Less

Submitted 10 October, 2023; originally announced October 2023.

Comments: 20 pages, 4 figures

Journal ref: Nano Letters 21, 4853-4860 (2021)

Large language models in medicine: the potentials and pitfalls

Authors: Jesutofunmi A. Omiye, Haiwen Gui, Shawheen J. Rezaei, James Zou, Roxana Daneshjou

Abstract: Large language models (LLMs) have been applied to tasks in healthcare, ranging from medical exam questions to responding to patient questions. With increasing institutional partnerships between companies producing LLMs and healthcare systems, real world clinical application is coming closer to reality. As these models gain traction, it is essential for healthcare practitioners to understand what L… ▽ More Large language models (LLMs) have been applied to tasks in healthcare, ranging from medical exam questions to responding to patient questions. With increasing institutional partnerships between companies producing LLMs and healthcare systems, real world clinical application is coming closer to reality. As these models gain traction, it is essential for healthcare practitioners to understand what LLMs are, their development, their current and potential applications, and the associated pitfalls when utilized in medicine. This review and accompanying tutorial aim to give an overview of these topics to aid healthcare practitioners in understanding the rapidly changing landscape of LLMs as applied to medicine. △ Less

Submitted 31 August, 2023; originally announced September 2023.

Journal ref: Ann. Intern. Med., 177(2), 210-220(2024)

On the source counts of VLBI-detected radio sources and the prospects of all-sky surveys with current and next generation instruments

Authors: S. Rezaei, J. P. McKean, A. T. Deller, J. F. Radcliffe

Abstract: We present an analysis of the detection fraction and the number counts of radio sources imaged with Very Long Baseline Interferometry (VLBI) at 1.4 GHz as part of the mJIVE-20 survey. From a sample of 24,903 radio sources identified by FIRST, 4,965 are detected on VLBI-scales, giving an overall detection fraction of $19.9\pm2.9~$per cent. However, we find that the detection fraction falls from aro… ▽ More We present an analysis of the detection fraction and the number counts of radio sources imaged with Very Long Baseline Interferometry (VLBI) at 1.4 GHz as part of the mJIVE-20 survey. From a sample of 24,903 radio sources identified by FIRST, 4,965 are detected on VLBI-scales, giving an overall detection fraction of $19.9\pm2.9~$per cent. However, we find that the detection fraction falls from around 50 per cent at a peak surface brightness of $80~mJy~beam^{-1}$ in FIRST to around 8 per cent at the detection limit, which is likely dominated by the surface brightness sensitivity of the VLBI observations, with some contribution from a change in the radio source population. We also find that compactness at arcsec-scales is the dominant factor in determining whether a radio source is detected with VLBI, and that the median size of the VLBI-detected radio sources is 7.7 mas. After correcting for the survey completeness and effective sky area, we determine the slope of the differential number counts of VLBI-detected radio sources with flux densities $S_{\rm 1.4~GHz} > 1~mJy$ to be $η_{\rm VLBI} = -1.74\pm 0.02$, which is shallower than in the cases of the FIRST parent population ($η_{\rm FIRST} = -1.77\pm 0.02$) and for compact radio sources selected at higher frequencies ($η_{\rm JBF} = -2.06\pm 0.02$). From this, we find that all-sky ($3π~sr$) surveys with the EVN and the VLBA have the potential to detect $(7.2\pm0.9)\times10^{5}$ radio sources at mas-resolution, and that the density of compact radio sources is sufficient (5.3~deg$^{-2}$) for in-beam phase referencing with multiple sources (3.9 per primary beam) in the case of a hypothetical SKA-VLBI array. △ Less

Submitted 30 August, 2023; originally announced August 2023.

Search for Eccentric Black Hole Coalescences during the Third Observing Run of LIGO and Virgo

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, A. G. Abac, R. Abbott, H. Abe, F. Acernese, K. Ackley, C. Adamcewicz, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, I. Aguilar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi , et al. (1750 additional authors not shown)

Abstract: Despite the growing number of confident binary black hole coalescences observed through gravitational waves so far, the astrophysical origin of these binaries remains uncertain. Orbital eccentricity is one of the clearest tracers of binary formation channels. Identifying binary eccentricity, however, remains challenging due to the limited availability of gravitational waveforms that include effect… ▽ More Despite the growing number of confident binary black hole coalescences observed through gravitational waves so far, the astrophysical origin of these binaries remains uncertain. Orbital eccentricity is one of the clearest tracers of binary formation channels. Identifying binary eccentricity, however, remains challenging due to the limited availability of gravitational waveforms that include effects of eccentricity. Here, we present observational results for a waveform-independent search sensitive to eccentric black hole coalescences, covering the third observing run (O3) of the LIGO and Virgo detectors. We identified no new high-significance candidates beyond those that were already identified with searches focusing on quasi-circular binaries. We determine the sensitivity of our search to high-mass (total mass $M>70$ $M_\odot$) binaries covering eccentricities up to 0.3 at 15 Hz orbital frequency, and use this to compare model predictions to search results. Assuming all detections are indeed quasi-circular, for our fiducial population model, we place an upper limit for the merger rate density of high-mass binaries with eccentricities $0 < e \leq 0.3$ at $0.33$ Gpc$^{-3}$ yr$^{-1}$ at 90\% confidence level. △ Less

Submitted 7 August, 2023; originally announced August 2023.

Comments: 24 pages, 5 figures

Report number: LIGO-P2300080

Engineering Perovskite Emissions via Optical Quasi-Bound-States-in-the-Continuum

Authors: Evelin Csányi, Yan Liu, Soroosh Daqiqeh Rezaei, Henry Yit Loong Lee, Febiana Tjiptoharsono, Zackaria Mahfoud, Sergey Gorelik, Xiaofei Zhao, Li Jun Lim, Di Zhu, Jing Wu, Kuan Eng Johnson Goh, Weibo Gao, Zhi-Kuang Tan, Graham Leggett, Cheng-Wei Qiu, Zhaogang Dong

Abstract: Metal halide perovskite quantum dots (PQDs) have emerged as promising materials due to their exceptional photoluminescence (PL) properties. A wide range of applications could benefit from adjustable luminescence properties, while preserving the physical and chemical properties of the PQDs. Therefore, post-synthesis engineering has gained attention recently, involving the use of ion-exchange or ext… ▽ More Metal halide perovskite quantum dots (PQDs) have emerged as promising materials due to their exceptional photoluminescence (PL) properties. A wide range of applications could benefit from adjustable luminescence properties, while preserving the physical and chemical properties of the PQDs. Therefore, post-synthesis engineering has gained attention recently, involving the use of ion-exchange or external stimuli, such as extreme pressure, magnetic and electric fields. Nevertheless, these methods typically suffer from spectrum broadening, intensity quenching or yield multiple bands. Alternatively, photonic antennas can modify the radiative decay channel of perovskites via the Purcell effect, with the largest wavelength shift being 8 nm to date, at an expense of 5-fold intensity loss. Here, we present an optical nanoantenna array with polarization-controlled quasi-bound-states-in-the-continuum (q-BIC) resonances, which can engineer and shift the photoluminescence wavelength over a ~39 nm range and confers a 21-fold emission enhancement of FAPbI3 perovskite QDs. The spectrum is engineered in a non-invasive manner via lithographically defined antennas and the pump laser polarization at ambient conditions. Our research provides a path towards advanced optoelectronic devices, such as spectrally tailored quantum emitters and lasers. △ Less

Submitted 25 June, 2023; originally announced June 2023.

Comments: 39 pages, 4 figures in the main text and 10 figures in the supporting information

Search for gravitational-lensing signatures in the full third observing run of the LIGO-Virgo network

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, C. Alléné, A. Allocca, P. A. Altin , et al. (1670 additional authors not shown)

Abstract: Gravitational lensing by massive objects along the line of sight to the source causes distortions of gravitational wave-signals; such distortions may reveal information about fundamental physics, cosmology and astrophysics. In this work, we have extended the search for lensing signatures to all binary black hole events from the third observing run of the LIGO--Virgo network. We search for repeated… ▽ More Gravitational lensing by massive objects along the line of sight to the source causes distortions of gravitational wave-signals; such distortions may reveal information about fundamental physics, cosmology and astrophysics. In this work, we have extended the search for lensing signatures to all binary black hole events from the third observing run of the LIGO--Virgo network. We search for repeated signals from strong lensing by 1) performing targeted searches for subthreshold signals, 2) calculating the degree of overlap amongst the intrinsic parameters and sky location of pairs of signals, 3) comparing the similarities of the spectrograms amongst pairs of signals, and 4) performing dual-signal Bayesian analysis that takes into account selection effects and astrophysical knowledge. We also search for distortions to the gravitational waveform caused by 1) frequency-independent phase shifts in strongly lensed images, and 2) frequency-dependent modulation of the amplitude and phase due to point masses. None of these searches yields significant evidence for lensing. Finally, we use the non-detection of gravitational-wave lensing to constrain the lensing rate based on the latest merger-rate estimates and the fraction of dark matter composed of compact objects. △ Less

Submitted 17 April, 2023; originally announced April 2023.

Comments: 28 pages, 11 figures

Report number: LIGO-P2200031

Learning solution of nonlinear constitutive material models using physics-informed neural networks: COMM-PINN

Authors: Shahed Rezaei, Ahmad Moeineddin, Ali Harandi

Abstract: We applied physics-informed neural networks to solve the constitutive relations for nonlinear, path-dependent material behavior. As a result, the trained network not only satisfies all thermodynamic constraints but also instantly provides information about the current material state (i.e., free energy, stress, and the evolution of internal variables) under any given loading scenario without requir… ▽ More We applied physics-informed neural networks to solve the constitutive relations for nonlinear, path-dependent material behavior. As a result, the trained network not only satisfies all thermodynamic constraints but also instantly provides information about the current material state (i.e., free energy, stress, and the evolution of internal variables) under any given loading scenario without requiring initial data. One advantage of this work is that it bypasses the repetitive Newton iterations needed to solve nonlinear equations in complex material models. Additionally, strategies are provided to reduce the required order of derivative for obtaining the tangent operator. The trained model can be directly used in any finite element package (or other numerical methods) as a user-defined material model. However, challenges remain in the proper definition of collocation points and in integrating several non-equality constraints that become active or non-active simultaneously. We tested this methodology on rate-independent processes such as the classical von Mises plasticity model with a nonlinear hardening law, as well as local damage models for interface cracking behavior with a nonlinear softening law. In order to demonstrate the applicability of the methodology in handling complex path dependency in a three-dimensional (3D) scenario, we tested the approach using the equations governing a damage model for a three-dimensional interface model. Such models are frequently employed for intergranular fracture at grain boundaries. We have observed a perfect agreement between the results obtained through the proposed methodology and those obtained using the classical approach. Furthermore, the proposed approach requires significantly less effort in terms of implementation and computing time compared to the traditional methods. △ Less

Submitted 6 September, 2023; v1 submitted 10 April, 2023; originally announced April 2023.

Mixed formulation of physics-informed neural networks for thermo-mechanically coupled systems and heterogeneous domains

Authors: Ali Harandi, Ahmad Moeineddin, Michael Kaliske, Stefanie Reese, Shahed Rezaei

Abstract: Physics-informed neural networks (PINNs) are a new tool for solving boundary value problems by defining loss functions of neural networks based on governing equations, boundary conditions, and initial conditions. Recent investigations have shown that when designing loss functions for many engineering problems, using first-order derivatives and combining equations from both strong and weak forms ca… ▽ More Physics-informed neural networks (PINNs) are a new tool for solving boundary value problems by defining loss functions of neural networks based on governing equations, boundary conditions, and initial conditions. Recent investigations have shown that when designing loss functions for many engineering problems, using first-order derivatives and combining equations from both strong and weak forms can lead to much better accuracy, especially when there are heterogeneity and variable jumps in the domain. This new approach is called the mixed formulation for PINNs, which takes ideas from the mixed finite element method. In this method, the PDE is reformulated as a system of equations where the primary unknowns are the fluxes or gradients of the solution, and the secondary unknowns are the solution itself. In this work, we propose applying the mixed formulation to solve multi-physical problems, specifically a stationary thermo-mechanically coupled system of equations. Additionally, we discuss both sequential and fully coupled unsupervised training and compare their accuracy and computational cost. To improve the accuracy of the network, we incorporate hard boundary constraints to ensure valid predictions. We then investigate how different optimizers and architectures affect accuracy and efficiency. Finally, we introduce a simple approach for parametric learning that is similar to transfer learning. This approach combines data and physics to address the limitations of PINNs regarding computational cost and improves the network's ability to predict the response of the system for unseen cases. The outcomes of this work will be useful for many other engineering applications where deep learning is employed on multiple coupled systems of equations for fast and reliable computations. △ Less

Submitted 6 September, 2023; v1 submitted 9 February, 2023; originally announced February 2023.

Open data from the third observing run of LIGO, Virgo, KAGRA and GEO

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Al-Jodah, C. Alléné, A. Allocca , et al. (1719 additional authors not shown)

Abstract: The global network of gravitational-wave observatories now includes five detectors, namely LIGO Hanford, LIGO Livingston, Virgo, KAGRA, and GEO 600. These detectors collected data during their third observing run, O3, composed of three phases: O3a starting in April of 2019 and lasting six months, O3b starting in November of 2019 and lasting five months, and O3GK starting in April of 2020 and lasti… ▽ More The global network of gravitational-wave observatories now includes five detectors, namely LIGO Hanford, LIGO Livingston, Virgo, KAGRA, and GEO 600. These detectors collected data during their third observing run, O3, composed of three phases: O3a starting in April of 2019 and lasting six months, O3b starting in November of 2019 and lasting five months, and O3GK starting in April of 2020 and lasting 2 weeks. In this paper we describe these data and various other science products that can be freely accessed through the Gravitational Wave Open Science Center at https://gwosc.org. The main dataset, consisting of the gravitational-wave strain time series that contains the astrophysical signals, is released together with supporting data useful for their analysis and documentation, tutorials, as well as analysis software packages. △ Less

Submitted 7 February, 2023; originally announced February 2023.

Comments: 27 pages, 3 figures

Report number: LIGO-P2200316

Prony-Based Super-Resolution Phase Retrieval of Sparse, Multivariate Signals

Authors: Robert Beinert, Saghar Rezaei

Abstract: Phase retrieval consists in the recovery of an unknown signal from phaseless measurements of its usually complex-valued Fourier transform. Without further assumptions, this problem is notorious to be severe ill posed such that the recovery of the true signal is nearly impossible. In certain applications like crystallography, speckle imaging in astronomy, or blind channel estimation in communicatio… ▽ More Phase retrieval consists in the recovery of an unknown signal from phaseless measurements of its usually complex-valued Fourier transform. Without further assumptions, this problem is notorious to be severe ill posed such that the recovery of the true signal is nearly impossible. In certain applications like crystallography, speckle imaging in astronomy, or blind channel estimation in communications, the unknown signal has a specific, sparse structure. In this paper, we exploit these sparse structure to recover the unknown signal uniquely up to inevitable ambiguities as global phase shifts, transitions, and conjugated reflections. Although using a constructive proof essentially based on Prony's method, our focus lies on the derivation of a recovery guarantee for multivariate signals using an adaptive sampling scheme. Instead of sampling the entire multivariate Fourier intensity, we only employ Fourier samples along certain adaptively chosen lines. For bivariate signals, an analogous result can be established for samples in generic directions. The number of samples here scales quadratically to the sparsity level of the unknown signal. △ Less

Submitted 18 January, 2023; originally announced January 2023.

On the Discredibility of Membership Inference Attacks

Authors: Shahbaz Rezaei, Xin Liu

Abstract: With the wide-spread application of machine learning models, it has become critical to study the potential data leakage of models trained on sensitive data. Recently, various membership inference (MI) attacks are proposed to determine if a sample was part of the training set or not. The question is whether these attacks can be reliably used in practice. We show that MI models frequently misclassif… ▽ More With the wide-spread application of machine learning models, it has become critical to study the potential data leakage of models trained on sensitive data. Recently, various membership inference (MI) attacks are proposed to determine if a sample was part of the training set or not. The question is whether these attacks can be reliably used in practice. We show that MI models frequently misclassify neighboring nonmember samples of a member sample as members. In other words, they have a high false positive rate on the subpopulations of the exact member samples that they can identify. We then showcase a practical application of MI attacks where this issue has a real-world repercussion. Here, MI attacks are used by an external auditor (investigator) to show to a judge/jury that an auditee unlawfully used sensitive data. Due to the high false positive rate of MI attacks on member's subpopulations, auditee challenges the credibility of the auditor by revealing the performance of the MI attacks on these subpopulations. We argue that current membership inference attacks can identify memorized subpopulations, but they cannot reliably identify which exact sample in the subpopulation was used during the training. △ Less

Submitted 28 April, 2023; v1 submitted 5 December, 2022; originally announced December 2022.

Search for subsolar-mass black hole binaries in the second part of Advanced LIGO's and Advanced Virgo's third observing run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, C. Alléné, A. Allocca, P. A. Altin , et al. (1680 additional authors not shown)

Abstract: We describe a search for gravitational waves from compact binaries with at least one component with mass 0.2 $M_\odot$ -- $1.0 M_\odot$ and mass ratio $q \geq 0.1$ in Advanced LIGO and Advanced Virgo data collected between 1 November 2019, 15:00 UTC and 27 March 2020, 17:00 UTC. No signals were detected. The most significant candidate has a false alarm rate of 0.2 $\mathrm{yr}^{-1}$. We estimate t… ▽ More We describe a search for gravitational waves from compact binaries with at least one component with mass 0.2 $M_\odot$ -- $1.0 M_\odot$ and mass ratio $q \geq 0.1$ in Advanced LIGO and Advanced Virgo data collected between 1 November 2019, 15:00 UTC and 27 March 2020, 17:00 UTC. No signals were detected. The most significant candidate has a false alarm rate of 0.2 $\mathrm{yr}^{-1}$. We estimate the sensitivity of our search over the entirety of Advanced LIGO's and Advanced Virgo's third observing run, and present the most stringent limits to date on the merger rate of binary black holes with at least one subsolar-mass component. We use the upper limits to constrain two fiducial scenarios that could produce subsolar-mass black holes: primordial black holes (PBH) and a model of dissipative dark matter. The PBH model uses recent prescriptions for the merger rate of PBH binaries that include a rate suppression factor to effectively account for PBH early binary disruptions. If the PBHs are monochromatically distributed, we can exclude a dark matter fraction in PBHs $f_\mathrm{PBH} \gtrsim 0.6$ (at 90% confidence) in the probed subsolar-mass range. However, if we allow for broad PBH mass distributions we are unable to rule out $f_\mathrm{PBH} = 1$. For the dissipative model, where the dark matter has chemistry that allows a small fraction to cool and collapse into black holes, we find an upper bound $f_{\mathrm{DBH}} < 10^{-5}$ on the fraction of atomic dark matter collapsed into black holes. △ Less

Submitted 26 January, 2024; v1 submitted 2 December, 2022; originally announced December 2022.

Comments: https://dcc.ligo.org/P2200139

AI enhanced finite element multiscale modelling and structural uncertainty analysis of a functionally graded porous beam

Authors: Da Chen, Nima Emami, Shahed Rezaei, Philipp L. Rosendahl, Bai-Xiang Xu, Jens Schneider, Kang Gao, Jie Yang

Abstract: The local geometrical randomness of metal foams brings complexities to the performance prediction of porous structures. Although the relative density is commonly deemed as the key factor, the stochasticity of internal cell sizes and shapes has an apparent effect on the porous structural behaviour but the corresponding measurement is challenging. To address this issue, we are aimed to develop an as… ▽ More The local geometrical randomness of metal foams brings complexities to the performance prediction of porous structures. Although the relative density is commonly deemed as the key factor, the stochasticity of internal cell sizes and shapes has an apparent effect on the porous structural behaviour but the corresponding measurement is challenging. To address this issue, we are aimed to develop an assessment strategy for efficiently examining the foam properties by combining multiscale modelling and deep learning. The multiscale modelling is based on the finite element (FE) simulation employing representative volume elements (RVEs) with random cellular morphologies, mimicking the typical features of closed-cell Aluminium foams. A deep learning database is constructed for training the designed convolutional neural networks (CNNs) to establish a direct link between the mesoscopic porosity characteristics and the effective Youngs modulus of foams. The error range of CNN models leads to an uncertain mechanical performance, which is further evaluated in a structural uncertainty analysis on the FG porous three-layer beam consisting of two thin high-density layers and a thick low-density one, where the imprecise CNN predicted moduli are represented as triangular fuzzy numbers in double parametric form. The uncertain beam bending deflections under a mid-span point load are calculated with the aid of Timoshenko beam theory and the Ritz method. Our findings suggest the success in training CNN models to estimate RVE modulus using images with an average error of 5.92%. The evaluation of FG porous structures can be significantly simplified with the proposed method and connects to the mesoscopic cellular morphologies without establishing the mechanics model for local foams. △ Less

Submitted 2 November, 2022; originally announced November 2022.

Comments: Book chapter in MACHINE LEARNING AIDED ANALYSIS, DESIGN, AND ADDITIVE MANUFACTURING OF FUNCTIONALLY GRADED POROUS COMPOSITE STRUCTURES, 20 pages, 10 figures

Model-based cross-correlation search for gravitational waves from the low-mass X-ray binary Scorpius X-1 in LIGO O3 data

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, R. Abbott, H. Abe, F. Acernese, K. Ackley, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi, C. Alléné, A. Allocca, P. A. Altin , et al. (1670 additional authors not shown)

Abstract: We present the results of a model-based search for continuous gravitational waves from the low-mass X-ray binary Scorpius X-1 using LIGO detector data from the third observing run of Advanced LIGO, Advanced Virgo and KAGRA. This is a semicoherent search which uses details of the signal model to coherently combine data separated by less than a specified coherence time, which can be adjusted to bala… ▽ More We present the results of a model-based search for continuous gravitational waves from the low-mass X-ray binary Scorpius X-1 using LIGO detector data from the third observing run of Advanced LIGO, Advanced Virgo and KAGRA. This is a semicoherent search which uses details of the signal model to coherently combine data separated by less than a specified coherence time, which can be adjusted to balance sensitivity with computing cost. The search covered a range of gravitational-wave frequencies from 25Hz to 1600Hz, as well as ranges in orbital speed, frequency and phase determined from observational constraints. No significant detection candidates were found, and upper limits were set as a function of frequency. The most stringent limits, between 100Hz and 200Hz, correspond to an amplitude h0 of about 1e-25 when marginalized isotropically over the unknown inclination angle of the neutron star's rotation axis, or less than 4e-26 assuming the optimal orientation. The sensitivity of this search is now probing amplitudes predicted by models of torque balance equilibrium. For the usual conservative model assuming accretion at the surface of the neutron star, our isotropically-marginalized upper limits are close to the predicted amplitude from about 70Hz to 100Hz; the limits assuming the neutron star spin is aligned with the most likely orbital angular momentum are below the conservative torque balance predictions from 40Hz to 200Hz. Assuming a broader range of accretion models, our direct limits on gravitational-wave amplitude delve into the relevant parameter space over a wide range of frequencies, to 500Hz or more. △ Less

Submitted 2 January, 2023; v1 submitted 6 September, 2022; originally announced September 2022.

Comments: 19 pages, Open Access Journal PDF

Report number: LIGO-P2100110-v13

Journal ref: The Astrophysical Journal Letters, 941, L30 (2022)

A thermo-mechanical phase-field fracture model: application to hot cracking simulations in additive manufacturing

Authors: Hui Ruan, Shahed Rezaei, Yangyiwei Yang, Dietmar Gross, Bai-Xiang Xu

Abstract: Thermal fracture is prevalent in many engineering problems and is one of the most devastating defects in metal additive manufacturing. Due to the interactive underlying physics involved, the computational simulation of such a process is challenging. In this work, we propose a thermo-mechanical phase-field fracture model, which is based on a thermodynamically consistent derivation. The influence of… ▽ More Thermal fracture is prevalent in many engineering problems and is one of the most devastating defects in metal additive manufacturing. Due to the interactive underlying physics involved, the computational simulation of such a process is challenging. In this work, we propose a thermo-mechanical phase-field fracture model, which is based on a thermodynamically consistent derivation. The influence of different coupling terms such as damage-informed thermomechanics and heat conduction and temperature-dependent fracture properties, as well as different phase-field fracture formulations, are discussed. Finally, the model is implemented in the finite element method and applied to simulate the hot cracking in additive manufacturing. Thereby not only the thermal strain but also the solidification shrinkage are considered. As for thermal history, various predicted thermal profiles, including analytical solution and numerical thermal temperature profile around the melting pool, are regarded, whereas the latter includes the influence of different process parameters. The studies reveal that the solidification shrinkage strain takes a dominant role in the formation of the circumferential crack, while the temperature gradient is mostly responsible for the central crack. Process parameter study demonstrates further that a higher laser power and slower scanning speed are favorable for keyhole mode hot cracking while a lower laser power and quicker scanning speed tend to form the conduction mode cracking. The numerical predictions of the hot cracking patterns are in good agreement with similar experimental observations, showing the capability of the model for further studies. △ Less

Submitted 3 August, 2022; originally announced August 2022.

Colorful Optical Vortices with White Light Illumination

Authors: Hongtao Wang, Hao Wang, Qifeng Ruan, John You En Chan, Wang Zhang, Hailong Liu, Soroosh Daqiqeh Rezaei, Jonathan Trisno, Cheng-Wei Qiu, Min Gu, Joel K. W. Yang

Abstract: The orbital angular momentum (OAM) of light holds great promise for applications in optical communication, super-resolution imaging, and high-dimensional quantum computing. However, the spatio-temporal coherence of the light source has been essential for generating OAM beams, as incoherent ambient light would result in polychromatic and obscured OAM beams in the visible spectrum. Here, we extend t… ▽ More The orbital angular momentum (OAM) of light holds great promise for applications in optical communication, super-resolution imaging, and high-dimensional quantum computing. However, the spatio-temporal coherence of the light source has been essential for generating OAM beams, as incoherent ambient light would result in polychromatic and obscured OAM beams in the visible spectrum. Here, we extend the applications of OAM to ambient lighting conditions. By miniaturizing spiral phase plates and integrating them with structural color filters, we achieve spatio-temporal coherence using only an incoherent white light source. These optical elements act as building blocks that encode both color and OAM information in the form of colorful optical vortices. Thus, pairs of transparent substrates that contain matching positions of these vortices constitute a reciprocal optical lock and key system. Due to the multiple helical eigenstates of OAM, the pairwise coupling can be further extended to form a one-to-many matching and validation scheme. Generating and decoding colorful optical vortices with broadband white light could find potential applications in anti-counterfeiting, optical metrology, high-capacity optical encryption, and on-chip 3D photonic devices. △ Less

Submitted 27 July, 2022; originally announced July 2022.

A machine learning based approach to gravitational lens identification with the International LOFAR Telescope

Authors: S. Rezaei, J. P. McKean, M. Biehl, W. de Roo1, A. Lafontaine

Abstract: We present a novel machine learning based approach for detecting galaxy-scale gravitational lenses from interferometric data, specifically those taken with the International LOFAR Telescope (ILT), which is observing the northern radio sky at a frequency of 150 MHz, an angular resolution of 350 mas and a sensitivity of 90 uJy beam-1 (1 sigma). We develop and test several Convolutional Neural Networ… ▽ More We present a novel machine learning based approach for detecting galaxy-scale gravitational lenses from interferometric data, specifically those taken with the International LOFAR Telescope (ILT), which is observing the northern radio sky at a frequency of 150 MHz, an angular resolution of 350 mas and a sensitivity of 90 uJy beam-1 (1 sigma). We develop and test several Convolutional Neural Networks to determine the probability and uncertainty of a given sample being classified as a lensed or non-lensed event. By training and testing on a simulated interferometric imaging data set that includes realistic lensed and non-lensed radio sources, we find that it is possible to recover 95.3 per cent of the lensed samples (true positive rate), with a contamination of just 0.008 per cent from non-lensed samples (false positive rate). Taking the expected lensing probability into account results in a predicted sample purity for lensed events of 92.2 per cent. We find that the network structure is most robust when the maximum image separation between the lensed images is greater than 3 times the synthesized beam size, and the lensed images have a total flux density that is equivalent to at least a 20 sigma (point-source) detection. For the ILT, this corresponds to a lens sample with Einstein radii greater than 0.5 arcsec and a radio source population with 150 MHz flux densities more than 2 mJy. By applying these criteria and our lens detection algorithm we expect to discover the vast majority of galaxy-scale gravitational lens systems contained within the LOFAR Two Metre Sky Survey. △ Less

Submitted 21 July, 2022; originally announced July 2022.

Comments: Accepted to be published by MNRAS

A mixed formulation for physics-informed neural networks as a potential solver for engineering problems in heterogeneous domains: comparison with finite element method

Authors: Shahed Rezaei, Ali Harandi, Ahmad Moeineddin, Bai-Xiang Xu, Stefanie Reese

Abstract: Physics-informed neural networks (PINNs) are capable of finding the solution for a given boundary value problem. We employ several ideas from the finite element method (FEM) to enhance the performance of existing PINNs in engineering problems. The main contribution of the current work is to promote using the spatial gradient of the primary variable as an output from separated neural networks. Late… ▽ More Physics-informed neural networks (PINNs) are capable of finding the solution for a given boundary value problem. We employ several ideas from the finite element method (FEM) to enhance the performance of existing PINNs in engineering problems. The main contribution of the current work is to promote using the spatial gradient of the primary variable as an output from separated neural networks. Later on, the strong form which has a higher order of derivatives is applied to the spatial gradients of the primary variable as the physical constraint. In addition, the so-called energy form of the problem is applied to the primary variable as an additional constraint for training. The proposed approach only required up to first-order derivatives to construct the physical loss functions. We discuss why this point is beneficial through various comparisons between different models. The mixed formulation-based PINNs and FE methods share some similarities. While the former minimizes the PDE and its energy form at given collocation points utilizing a complex nonlinear interpolation through a neural network, the latter does the same at element nodes with the help of shape functions. We focus on heterogeneous solids to show the capability of deep learning for predicting the solution in a complex environment under different boundary conditions. The performance of the proposed PINN model is checked against the solution from FEM on two prototype problems: elasticity and the Poisson equation (steady-state diffusion problem). We concluded that by properly designing the network architecture in PINN, the deep learning model has the potential to solve the unknowns in a heterogeneous domain without any available initial data from other sources. Finally, discussions are provided on the combination of PINN and FEM for a fast and accurate design of composite materials in future developments. △ Less

Submitted 27 June, 2022; originally announced June 2022.

Tri-Functional Metasurface for Phase, Amplitude, and Luminescence Control

Authors: Soroosh Daqiqeh Rezaei, Zhaogang Dong, Hao Wang, Jiahui Xu, Hongtao Wang, Mohammad Tavakkoli Yaraki, Ken Choon Hwa Goh, Wang Zhang, Xiaogang Liu, Joel K. W. Yang

Abstract: In optical anti-counterfeiting, several distinct optically variable devices (OVDs) are often concurrently employed to compensate for the insufficient security level of constituent OVDs. Alternatively, metasurfaces that exhibit multiple optical responses effectively combine multiple OVDs into one, thus significantly enhancing their security and hindering fraudulent replication. This work demonstrat… ▽ More In optical anti-counterfeiting, several distinct optically variable devices (OVDs) are often concurrently employed to compensate for the insufficient security level of constituent OVDs. Alternatively, metasurfaces that exhibit multiple optical responses effectively combine multiple OVDs into one, thus significantly enhancing their security and hindering fraudulent replication. This work demonstrates the simultaneous control of three separate optical responses, i.e., phase, amplitude, and luminescence, using anisotropic gap-plasmon metasurfaces. Due to the incorporated geometric anisotropy, the designed structure exhibits distinct responses under x- and y-polarized light, revealing either a color image, or a holographic projection in the far field. Furthermore, inserting upconversion nanoparticles (UCNPs) into the dielectric gaps of the structures, the designed metasurface is able to generate a third luminescent image upon illumination with the near-infrared light. The stochastic distribution of the UCNPs constitutes a unique fingerprint, achieving a physically unclonable function (PUF) layer. Crucially, our triple-mode metasurface requires only readily attainable equipment such as a macro-lens/camera and a laser pointer to read most of the channels, thus paving the way towards highly secure and easy-to-authenticate metasurface-driven OVDs (mOVDs). △ Less

Submitted 16 June, 2022; originally announced June 2022.

Miniaturizing Color-Sensitive Photodetectors via Hybrid Nanoantennas towards Sub-micron Dimensions

Authors: Jinfa Ho, Zhaogang Dong, Hai Sheng Leong, Jun Zhang, Febiana Tjiptoharsono, Soroosh Daqiqeh Rezaei, Ken Choon Hwa Goh, Mengfei Wu, Shiqiang Li, Jingyee Chee, Calvin Pei Yu Wong, Arseniy I. Kuznetsov, Joel K. W. Yang

Abstract: Digital camera sensors utilize color filters on photodiodes to achieve color selectivity. As color filters and photosensitive silicon layers are separate elements, these sensors suffer from optical cross-talk, which sets limits to the minimum pixel size. In this paper, we report hybrid silicon-aluminum nanostructures in the extreme limit of zero distance between color filters and sensors. This des… ▽ More Digital camera sensors utilize color filters on photodiodes to achieve color selectivity. As color filters and photosensitive silicon layers are separate elements, these sensors suffer from optical cross-talk, which sets limits to the minimum pixel size. In this paper, we report hybrid silicon-aluminum nanostructures in the extreme limit of zero distance between color filters and sensors. This design could essentially achieve sub micron pixel dimensions and minimize the optical cross-talk originated from tilt illuminations. The designed hybrid silicon-aluminum nanostructure has dual functionalities. Crucially, it supports a hybrid Mie-plasmon resonance of magnetic dipole to achieve the color-selective light absorption, generating electron hole pairs. Simultaneously, the silicon-aluminum interface forms a Schottky barrier for charge separation and photodetection. This design could potentially replace the traditional dye based filters for camera sensors at ultra-high pixel densities with advanced functionalities in sensing polarization and directionality, as well as UV selectivity via interband plasmons of silicon. △ Less

Submitted 7 June, 2022; originally announced June 2022.

Comments: 32 pages, 4 Figures (main text), 7 Figures (supplementary)

An Efficient Subpopulation-based Membership Inference Attack

Authors: Shahbaz Rezaei, Xin Liu

Abstract: Membership inference attacks allow a malicious entity to predict whether a sample is used during training of a victim model or not. State-of-the-art membership inference attacks have shown to achieve good accuracy which poses a great privacy threat. However, majority of SOTA attacks require training dozens to hundreds of shadow models to accurately infer membership. This huge computation cost rais… ▽ More Membership inference attacks allow a malicious entity to predict whether a sample is used during training of a victim model or not. State-of-the-art membership inference attacks have shown to achieve good accuracy which poses a great privacy threat. However, majority of SOTA attacks require training dozens to hundreds of shadow models to accurately infer membership. This huge computation cost raises questions about practicality of these attacks on deep models. In this paper, we introduce a fundamentally different MI attack approach which obviates the need to train hundreds of shadow models. Simply put, we compare the victim model output on the target sample versus the samples from the same subpopulation (i.e., semantically similar samples), instead of comparing it with the output of hundreds of shadow models. The intuition is that the model response should not be significantly different between the target sample and its subpopulation if it was not a training sample. In cases where subpopulation samples are not available to the attacker, we show that training only a single generative model can fulfill the requirement. Hence, we achieve the state-of-the-art membership inference accuracy while significantly reducing the training computation cost. △ Less

Submitted 3 March, 2022; originally announced March 2022.

User-Level Membership Inference Attack against Metric Embedding Learning

Authors: Guoyao Li, Shahbaz Rezaei, Xin Liu

Abstract: Membership inference (MI) determines if a sample was part of a victim model training set. Recent development of MI attacks focus on record-level membership inference which limits their application in many real-world scenarios. For example, in the person re-identification task, the attacker (or investigator) is interested in determining if a user's images have been used during training or not. Howe… ▽ More Membership inference (MI) determines if a sample was part of a victim model training set. Recent development of MI attacks focus on record-level membership inference which limits their application in many real-world scenarios. For example, in the person re-identification task, the attacker (or investigator) is interested in determining if a user's images have been used during training or not. However, the exact training images might not be accessible to the attacker. In this paper, we develop a user-level MI attack where the goal is to find if any sample from the target user has been used during training even when no exact training sample is available to the attacker. We focus on metric embedding learning due to its dominance in person re-identification, where user-level MI attack is more sensible. We conduct an extensive evaluation on several datasets and show that our approach achieves high accuracy on user-level MI task. △ Less

Submitted 25 April, 2022; v1 submitted 3 March, 2022; originally announced March 2022.

In-storage Processing of I/O Intensive Applications on Computational Storage Drives

Authors: Ali HeydariGorji, Mahdi Torabzadehkashi, Siavash Rezaei, Hossein Bobarshad, Vladimir Alves, Pai H. Chou

Abstract: Computational storage drives (CSD) are solid-state drives (SSD) empowered by general-purpose processors that can perform in-storage processing. They have the potential to improve both performance and energy significantly for big-data analytics by bringing compute to data, thereby eliminating costly data transfer while offering better privacy. In this work, we introduce Solana, the first-ever high-… ▽ More Computational storage drives (CSD) are solid-state drives (SSD) empowered by general-purpose processors that can perform in-storage processing. They have the potential to improve both performance and energy significantly for big-data analytics by bringing compute to data, thereby eliminating costly data transfer while offering better privacy. In this work, we introduce Solana, the first-ever high-capacity(12-TB) CSD in E1.S form factor, and present an actual prototype for evaluation. To demonstrate the benefits of in-storage processing on CSD, we deploy several natural language processing (NLP) applications on datacenter-grade storage servers comprised of clusters of the Solana. Experimental results show up to 3.1x speedup in processing while reducing the energy consumption and data transfer by 67% and 68%, respectively, compared to regular enterprise SSDs. △ Less

Submitted 23 December, 2021; originally announced December 2021.

Comments: Accepted for the 23rd International Symposium on Quality Electronic Design (ISQED'22)

Profiling Neural Blocks and Design Spaces for Mobile Neural Architecture Search

Authors: Keith G. Mills, Fred X. Han, Jialin Zhang, Seyed Saeed Changiz Rezaei, Fabian Chudak, Wei Lu, Shuo Lian, Shangling Jui, Di Niu

Abstract: Neural architecture search automates neural network design and has achieved state-of-the-art results in many deep learning applications. While recent literature has focused on designing networks to maximize accuracy, little work has been conducted to understand the compatibility of architecture design spaces to varying hardware. In this paper, we analyze the neural blocks used to build Once-for-Al… ▽ More Neural architecture search automates neural network design and has achieved state-of-the-art results in many deep learning applications. While recent literature has focused on designing networks to maximize accuracy, little work has been conducted to understand the compatibility of architecture design spaces to varying hardware. In this paper, we analyze the neural blocks used to build Once-for-All (MobileNetV3), ProxylessNAS and ResNet families, in order to understand their predictive power and inference latency on various devices, including Huawei Kirin 9000 NPU, RTX 2080 Ti, AMD Threadripper 2990WX, and Samsung Note10. We introduce a methodology to quantify the friendliness of neural blocks to hardware and the impact of their placement in a macro network on overall network performance via only end-to-end measurements. Based on extensive profiling results, we derive design insights and apply them to hardware-specific search space reduction. We show that searching in the reduced search space generates better accuracy-latency Pareto frontiers than searching in the original search spaces, customizing architecture search according to the hardware. Moreover, insights derived from measurements lead to notably higher ImageNet top-1 scores on all search spaces investigated. △ Less

Submitted 25 September, 2021; originally announced September 2021.

Comments: Accepted as an Applied Research Paper at CIKM 2021; 10 pages, 8 Figures, 2 Tables

L$^{2}$NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning

Authors: Keith G. Mills, Fred X. Han, Mohammad Salameh, Seyed Saeed Changiz Rezaei, Linglong Kong, Wei Lu, Shuo Lian, Shangling Jui, Di Niu

Abstract: Neural architecture search (NAS) has achieved remarkable results in deep neural network design. Differentiable architecture search converts the search over discrete architectures into a hyperparameter optimization problem which can be solved by gradient descent. However, questions have been raised regarding the effectiveness and generalizability of gradient methods for solving non-convex architect… ▽ More Neural architecture search (NAS) has achieved remarkable results in deep neural network design. Differentiable architecture search converts the search over discrete architectures into a hyperparameter optimization problem which can be solved by gradient descent. However, questions have been raised regarding the effectiveness and generalizability of gradient methods for solving non-convex architecture hyperparameter optimization problems. In this paper, we propose L$^{2}$NAS, which learns to intelligently optimize and update architecture hyperparameters via an actor neural network based on the distribution of high-performing architectures in the search history. We introduce a quantile-driven training procedure which efficiently trains L$^{2}$NAS in an actor-critic framework via continuous-action reinforcement learning. Experiments show that L$^{2}$NAS achieves state-of-the-art results on NAS-Bench-201 benchmark as well as DARTS search space and Once-for-All MobileNetV3 search space. We also show that search policies generated by L$^{2}$NAS are generalizable and transferable across different training datasets with minimal fine-tuning. △ Less

Submitted 25 September, 2021; originally announced September 2021.

Comments: Accepted as a Full Research Paper at CIKM 2021; 10 pages, 3 Figures, 5 Tables

DECORAS: detection and characterization of radio-astronomical sources using deep learning

Authors: S. Rezaei, J. P. McKean, M. Biehl, A. Javadpour

Abstract: We present DECORAS, a deep learning based approach to detect both point and extended sources from Very Long Baseline Interferometry (VLBI) observations. Our approach is based on an encoder-decoder neural network architecture that uses a low number of convolutional layers to provide a scalable solution for source detection. In addition, DECORAS performs source characterization in terms of the posit… ▽ More We present DECORAS, a deep learning based approach to detect both point and extended sources from Very Long Baseline Interferometry (VLBI) observations. Our approach is based on an encoder-decoder neural network architecture that uses a low number of convolutional layers to provide a scalable solution for source detection. In addition, DECORAS performs source characterization in terms of the position, effective radius and peak brightness of the detected sources. We have trained and tested the network with images that are based on realistic Very Long Baseline Array (VLBA) observations at 20 cm. Also, these images have not gone through any prior de-convolution step and are directly related to the visibility data via a Fourier transform. We find that the source catalog generated by DECORAS has a better overall completeness and purity, when compared to a traditional source detection algorithm. DECORAS is complete at the 7.5$σ$ level, and has an almost factor of two improvement in reliability at 5.5$σ$. We find that DECORAS can recover the position of the detected sources to within 0.61 $\pm$ 0.69 mas, and the effective radius and peak surface brightness are recovered to within 20 per cent for 98 and 94 per cent of the sources, respectively. Overall, we find that DECORAS provides a reliable source detection and characterization solution for future wide-field VLBI surveys. △ Less

Submitted 21 September, 2021; v1 submitted 19 September, 2021; originally announced September 2021.

Comments: submitted to MNRAS

An anisotropic cohesive fracture model: advantages and limitations of length-scale insensitive phase-field damage models

Authors: Shahed Rezaei, Ali Harandi, Tim Brepols, Stefanie Reese

Abstract: The goal of the current work is to explore direction-dependent damage initiation and propagation within an arbitrary anisotropic solid. In particular, we aim at developing anisotropic cohesive phase-field (PF) damage models by extending the idea introduced in \cite{REZAEI2021a} for direction-dependent fracture energy and also anisotropic PF damage models based on structural tensors. The cohesive P… ▽ More The goal of the current work is to explore direction-dependent damage initiation and propagation within an arbitrary anisotropic solid. In particular, we aim at developing anisotropic cohesive phase-field (PF) damage models by extending the idea introduced in \cite{REZAEI2021a} for direction-dependent fracture energy and also anisotropic PF damage models based on structural tensors. The cohesive PF damage formulation used in the current contribution is motivated by the works of \cite{LORENTZ201120, wu2018, GEELEN2019}. The results of the latter models are shown to be insensitive with respect to the length scale parameter for the isotropic case. This is because they manage to formulate the fracture energy as a function of diffuse displacement jumps in the localized damaged zone. In the present paper, we discuss numerical examples and details on finite element implementations where the fracture energy, as well as the material strength, are introduced as an arbitrary function of the crack direction. Using the current formulation for anisotropic cohesive fracture, the obtained results are almost insensitive with respect to the length scale parameter. The latter is achieved by including the direction-dependent strength of the material in addition to its fracture energy. Utilizing the current formulation, one can increase the mesh size which reduces the computational time significantly without any severe change in the predicted crack path and overall obtained load-displacement curves. We also argue that these models still lack to capture mode-dependent fracture properties. Open issues and possible remedies for future developments are finally discussed as well. △ Less

Submitted 28 August, 2021; originally announced August 2021.

Lossless Multi-Scale Constitutive Elastic Relations with Artificial Intelligence

Authors: Jaber Rezaei Mianroodi, Shahed Rezaei, Nima H. Siboni, Bai-Xiang Xu, Dierk Raabe

Abstract: The elastic properties of materials derive from their electronic and atomic nature. However, simulating bulk materials fully at these scales is not feasible, so that typically homogenized continuum descriptions are used instead. A seamless and lossless transition of the constitutive description of the elastic response of materials between these two scales has been so far elusive. Here we show how… ▽ More The elastic properties of materials derive from their electronic and atomic nature. However, simulating bulk materials fully at these scales is not feasible, so that typically homogenized continuum descriptions are used instead. A seamless and lossless transition of the constitutive description of the elastic response of materials between these two scales has been so far elusive. Here we show how this problem can be overcome by using Artificial Intelligence (AI). A Convolutional Neural Network (CNN) model is trained, by taking the structure image of a nanoporous material as input and the corresponding elasticity tensor, calculated from Molecular Statics (MS), as output. Trained with the atomistic data, the CNN model captures the size- and pore-dependency of the material's elastic properties which, on the physics side, can stem from surfaces and non-local effects. Such effects are often ignored in upscaling from atomistic to classical continuum theory. To demonstrate the accuracy and the efficiency of the trained CNN model, a Finite Element Method (FEM) based result of an elastically deformed nanoporous beam equipped with the CNN as constitutive law is compared with that by a full atomistic simulation. The good agreement between the atomistic simulations and the FEM-AI combination for a system with size and surface effects establishes a new lossless scale bridging approach to such problems. The trained CNN model deviates from the atomistic result by 9.6\% for porosity scenarios of up to 90\% but it is about 230 times faster than the MS calculation and does not require to change simulation methods between different scales. The efficiency of the CNN evaluation together with the preservation of important atomistic effects makes the trained model an effective atomistically-informed constitutive model for macroscopic simulations of nanoporous materials and solving of inverse problems. △ Less

Submitted 5 August, 2021; originally announced August 2021.

Tunable Mie Resonances in the Visible Spectrum

Authors: Li Lu, Zhaogang Dong, Febiana Tijiptoharsono, Ray Jia Hong Ng, Hongtao Wang, Soroosh Daqiqeh Rezaei, Yunzheng Wang, Hai Sheng Leong, Joel K. W. Yang, Robert E. Simpson

Abstract: Dielectric optical nanoantennas play an important role in color displays, metasurface holograms, and wavefront shaping applications. They usually exploit Mie resonances as supported on nanostructures with high refractive index, such as Si and TiO2. However, these resonances normally cannot be tuned. Although phase change materials, such as the germanium-antimony-tellurium alloys and post transitio… ▽ More Dielectric optical nanoantennas play an important role in color displays, metasurface holograms, and wavefront shaping applications. They usually exploit Mie resonances as supported on nanostructures with high refractive index, such as Si and TiO2. However, these resonances normally cannot be tuned. Although phase change materials, such as the germanium-antimony-tellurium alloys and post transition metal oxides, such as ITO, have been used to tune optical antennas in the near infrared spectrum, tunable dielectric antennae in the visible spectrum remain to be demonstrated. In this paper, we designed and experimentally demonstrated tunable dielectric nanoantenna arrays with Mie resonances in the visible spectrum, exploiting phase transitions in wide-bandgap Sb2S3 nano-resonators. In the amorphous state, Mie resonances in these Sb2S3 nanostructures give rise to a strong structural color in reflection mode. Thermal annealing induced crystallization and laser induced amorphization of the Sb2S3 resonators allow the color to be tuned reversibly. We believe these tunable Sb2S3 nanoantennae arrays will enable a wide variety of tunable nanophotonic applications, such as high-resolution color displays, holographic displays, and miniature LiDAR systems. △ Less

Submitted 14 July, 2021; originally announced July 2021.

Comments: 36 pages, 5 figures in main text, 9 figures in supporting information

Schrodinger's Red Pixel by Quasi Bound-State-In-Continuum

Authors: Zhaogang Dong, Lei Jin, Soroosh Daqiqeh Rezaei, Hao Wang, Yang Chen, Febiana Tjiptoharsono, Jinfa Ho, Sergey Gorelik, Ray Jia Hong Ng, Qifeng Ruan, Cheng-Wei Qiu, Joel K. W. Yang

Abstract: While structural colors are ubiquitous in nature, saturated reds are mysteriously absent. Hence, a longstanding problem is in fabricating nanostructured surfaces that exhibit reflectance approaching the theoretical limit. This limit is termed the Schrodinger red and demands sharp spectral transitions from "stopband" to a high reflectance "passband" with total suppression of higher-order resonances… ▽ More While structural colors are ubiquitous in nature, saturated reds are mysteriously absent. Hence, a longstanding problem is in fabricating nanostructured surfaces that exhibit reflectance approaching the theoretical limit. This limit is termed the Schrodinger red and demands sharp spectral transitions from "stopband" to a high reflectance "passband" with total suppression of higher-order resonances at blue and green wavelengths. Current approaches based on metallic or dielectric nanoantennas are insufficient to simultaneously meet these conditions. Here, for the 1st time, we designed and fabricated tall Si nanoantenna arrays on quartz substrate to support two partially overlapping y polarized quasi bound-state-in-the-continuum (q-BIC) modes in the red wavelengths with sharp spectral edges. These structures produce possibly the most saturated and brightest reds with ~80% reflectance, exceeding the red vertex in sRGB and even the cadmium red pigment. We employed a gradient descent algorithm with structures supporting q BIC as the starting point. Although the current design is polarization dependent, the proposed paradigm has enabled us to achieve the elusive structural red and the design principle could be generalized to Schrodinger's pixels of other colors. The design is suitable for scale up using other nanofabrication techniques for larger area applications, such as red pixels in displays, decorative coatings, and miniaturized spectrometers with high wavelength selectivity. △ Less

Submitted 23 June, 2021; originally announced June 2021.

Comments: 40 pages, 4 figures in the main text, 15 figures in the supplementary information

The New Ephemeris and Light Curve Analysis of V870 Ara by the Ground-Based and TESS Data

Authors: Atila Poro, Mark G. Blackford, Fatemeh Davoudi, Amirreza Mohandes, Mohammad Madani, Samaneh Rezaei, Elnaz Bozorgzadeh

Abstract: New CCD photometric observations and their investigation of the W UMa-type binary, V870 Ara, are presented. Light curves of the system were taken through BVI filters from the Congarinni Observatory in Australia. The new ephemeris is calculated based on seven new determined minimum times, together with the TESS data and others compiled from the literature. Photometric solutions determined by the Wi… ▽ More New CCD photometric observations and their investigation of the W UMa-type binary, V870 Ara, are presented. Light curves of the system were taken through BVI filters from the Congarinni Observatory in Australia. The new ephemeris is calculated based on seven new determined minimum times, together with the TESS data and others compiled from the literature. Photometric solutions determined by the Wilson-Devinney (W-D) code are combined with the Monte Carlo simulation to determine the adjustable parameters' uncertainties. These solutions suggest that V870 Ara is a contact binary system with a mass ratio of 0.082, a fillout factor of 96+-4 percent, and an inclination of 73.60+-0.64 degrees. The absolute parameters of V870 Ara were determined by combining the Gaia EDR3 parallax and photometric elements. △ Less

Submitted 13 July, 2021; v1 submitted 24 May, 2021; originally announced May 2021.

Comments: 5 tables, 6 figs, accepted in the Open Astronomy Journal

Generative Adversarial Neural Architecture Search

Authors: Seyed Saeed Changiz Rezaei, Fred X. Han, Di Niu, Mohammad Salameh, Keith Mills, Shuo Lian, Wei Lu, Shangling Jui

Abstract: Despite the empirical success of neural architecture search (NAS) in deep learning applications, the optimality, reproducibility and cost of NAS schemes remain hard to assess. In this paper, we propose Generative Adversarial NAS (GA-NAS) with theoretically provable convergence guarantees, promoting stability and reproducibility in neural architecture search. Inspired by importance sampling, GA-NAS… ▽ More Despite the empirical success of neural architecture search (NAS) in deep learning applications, the optimality, reproducibility and cost of NAS schemes remain hard to assess. In this paper, we propose Generative Adversarial NAS (GA-NAS) with theoretically provable convergence guarantees, promoting stability and reproducibility in neural architecture search. Inspired by importance sampling, GA-NAS iteratively fits a generator to previously discovered top architectures, thus increasingly focusing on important parts of a large search space. Furthermore, we propose an efficient adversarial learning approach, where the generator is trained by reinforcement learning based on rewards provided by a discriminator, thus being able to explore the search space without evaluating a large number of architectures. Extensive experiments show that GA-NAS beats the best published results under several cases on three public NAS benchmarks. In the meantime, GA-NAS can handle ad-hoc search constraints and search spaces. We show that GA-NAS can be used to improve already optimized baselines found by other NAS methods, including EfficientNet and ProxylessNAS, in terms of ImageNet accuracy or the number of parameters, in their original search space. △ Less

Submitted 23 June, 2021; v1 submitted 19 May, 2021; originally announced May 2021.

Comments: 17 pages, 9 figures, 13 Tables

Accuracy-Privacy Trade-off in Deep Ensemble: A Membership Inference Perspective

Authors: Shahbaz Rezaei, Zubair Shafiq, Xin Liu

Abstract: Deep ensemble learning has been shown to improve accuracy by training multiple neural networks and averaging their outputs. Ensemble learning has also been suggested to defend against membership inference attacks that undermine privacy. In this paper, we empirically demonstrate a trade-off between these two goals, namely accuracy and privacy (in terms of membership inference attacks), in deep ense… ▽ More Deep ensemble learning has been shown to improve accuracy by training multiple neural networks and averaging their outputs. Ensemble learning has also been suggested to defend against membership inference attacks that undermine privacy. In this paper, we empirically demonstrate a trade-off between these two goals, namely accuracy and privacy (in terms of membership inference attacks), in deep ensembles. Using a wide range of datasets and model architectures, we show that the effectiveness of membership inference attacks increases when ensembling improves accuracy. We analyze the impact of various factors in deep ensembles and demonstrate the root cause of the trade-off. Then, we evaluate common defenses against membership inference attacks based on regularization and differential privacy. We show that while these defenses can mitigate the effectiveness of membership inference attacks, they simultaneously degrade ensemble accuracy. We illustrate similar trade-off in more advanced and state-of-the-art ensembling techniques, such as snapshot ensembles and diversified ensemble networks. Finally, we propose a simple yet effective defense for deep ensembles to break the trade-off and, consequently, improve the accuracy and privacy, simultaneously. △ Less

Submitted 5 December, 2022; v1 submitted 11 May, 2021; originally announced May 2021.

New perspective on chiral exceptional points with application to discrete photonics

Authors: A. Hashemi, S. M. Rezaei, S. K. Ozdemir, R. El-Ganainy

Abstract: Chiral exceptional points (CEPs) have been shown to emerge in traveling wave resonators via asymmetric back scattering from two or more nano-scatterers. Here, we provide a new perspective on the formation of CEPs based on the coupled oscillator model. Our approach provides an intuitive understanding for the modal coalescence that signals the emergence of CEPs, and emphasizes the role played by dis… ▽ More Chiral exceptional points (CEPs) have been shown to emerge in traveling wave resonators via asymmetric back scattering from two or more nano-scatterers. Here, we provide a new perspective on the formation of CEPs based on the coupled oscillator model. Our approach provides an intuitive understanding for the modal coalescence that signals the emergence of CEPs, and emphasizes the role played by dissipation throughout this process. In doing so, our model also unveils an otherwise unexplored connection between CEPs and other types of exceptional points associated with parity-time symmetric photonic arrangements. In addition, our model also explains qualitative results observed in recent experimental work involving CEPs. Importantly, the tight-binding nature of our approach allows us to extend the notion of CEP to discrete photonics setups that consist coupled resonator and waveguide arrays, thus opening new avenues for exploring the exotic features of CEPs in conjunction with other interesting physical effects such as nonlinearities and topological protections. △ Less

Submitted 23 April, 2021; originally announced April 2021.

Comments: 7 pages, 2 figures

Journal ref: APL Photonics 6, 040803 (2021)

BO Ari Light Curve Analysis using Ground-Based and TESS Data

Authors: Atila Poro, Shiva Zamanpour, Maryam Hashemi, Yasemin Aladağ, Nazim Aksaker, Samaneh Rezaei, Arif Solmaz

Abstract: We present new BVR band photometric light curves of BO Aries obtained in 2020 and combined them with the Transiting Exoplanet Survey Satellite (TESS) light curves. We obtained times of minima based on Gaussian and Cauchy distributions and then applied the Monte Carlo Markov Chain (MCMC) method to measure the amount of uncertainty from our CCD photometry and TESS data. A new ephemeris of the binary… ▽ More We present new BVR band photometric light curves of BO Aries obtained in 2020 and combined them with the Transiting Exoplanet Survey Satellite (TESS) light curves. We obtained times of minima based on Gaussian and Cauchy distributions and then applied the Monte Carlo Markov Chain (MCMC) method to measure the amount of uncertainty from our CCD photometry and TESS data. A new ephemeris of the binary system was computed employing 204 times of minimum. The light curves were analyzed using the Wilson-Devinney binary code combined with the Monte Carlo (MC) simulation. For this light curve solution, we considered a dark spot on the primary component. We conclude that this binary is an A-type system with a mass ratio of q=0.2074+-0.0001, an orbital inclination of i=82.18+-0.02 deg, and a fillout factor of f=75.7+-0.8%. Our results for the a(Rsun) and q parameters are consistent with the results of the Xu-Dong Zhang and Sheng-Bang Qian (2020) model. The absolute parameters of the two components were calculated and the distance estimate of the binary system was found to be 142+-9 pc. △ Less

Submitted 14 January, 2021; originally announced January 2021.

Comments: 7 Figures, 5 Tables, sent to New Astronomy journal

Unobtrusive Pain Monitoring in Older Adults with Dementia using Pairwise and Contrastive Training

Authors: Siavash Rezaei, Abhishek Moturu, Shun Zhao, Kenneth M. Prkachin, Thomas Hadjistavropoulos, Babak Taati

Abstract: Although pain is frequent in old age, older adults are often undertreated for pain. This is especially the case for long-term care residents with moderate to severe dementia who cannot report their pain because of cognitive impairments that accompany dementia. Nursing staff acknowledge the challenges of effectively recognizing and managing pain in long-term care facilities due to lack of human res… ▽ More Although pain is frequent in old age, older adults are often undertreated for pain. This is especially the case for long-term care residents with moderate to severe dementia who cannot report their pain because of cognitive impairments that accompany dementia. Nursing staff acknowledge the challenges of effectively recognizing and managing pain in long-term care facilities due to lack of human resources and, sometimes, expertise to use validated pain assessment approaches on a regular basis. Vision-based ambient monitoring will allow for frequent automated assessments so care staff could be automatically notified when signs of pain are displayed. However, existing computer vision techniques for pain detection are not validated on faces of older adults or people with dementia, and this population is not represented in existing facial expression datasets of pain. We present the first fully automated vision-based technique validated on a dementia cohort. Our contributions are threefold. First, we develop a deep learning-based computer vision system for detecting painful facial expressions on a video dataset that is collected unobtrusively from older adult participants with and without dementia. Second, we introduce a pairwise comparative inference method that calibrates to each person and is sensitive to changes in facial expression while using training data more efficiently than sequence models. Third, we introduce a fast contrastive training method that improves cross-dataset performance. Our pain estimation model outperforms baselines by a wide margin, especially when evaluated on faces of people with dementia. Pre-trained model and demo code available at https://github.com/TaatiTeam/pain_detection_demo △ Less

Submitted 8 January, 2021; originally announced January 2021.

Signature of the quantum critical point on the witness of the non-Markovianity and the entanglement in spin-1/2 chain

Authors: Z. Saghafi, S. Samadi Rezaei, P. Azizi, E. Hosseini Lapasar, S. Mahdavifar

Abstract: A thermodynamic limit chain of spin-1/2 particles with XX and three-spin interactions (TSI) is considered. Using the fermionization technique, the Hamiltonian of the chain with periodic boundary conditions is exactly diagonalized. In the ground-state phase diagram of the chain system, a quantum critical point separates the Luttinger liquid and chiral phases. Selecting one-spin as an open quantum s… ▽ More A thermodynamic limit chain of spin-1/2 particles with XX and three-spin interactions (TSI) is considered. Using the fermionization technique, the Hamiltonian of the chain with periodic boundary conditions is exactly diagonalized. In the ground-state phase diagram of the chain system, a quantum critical point separates the Luttinger liquid and chiral phases. Selecting one-spin as an open quantum system, the rest of the spins play the role of its environment. By choosing different initial states, we studied the dynamics of the entanglement between the open quantum system and its environment. In the initial state, the state of the open quantum system is a superposition of up and down states and the environment is polarized. The revival of the entanglement is observed in the whole range of interactions as an indicator of non-Markovian dynamics. Our exact results revealed that non-Markovian behavior is independent of the initial state. By tuning the initial state, the open quantum system will be completely entangled with the environment at a special time called entanglement-time, tE. The value of the mentioned entanglement-time decreased by increasing TSI. The signature of a quantum critical point is confirmed by investigating the long-time average of the entanglement and using the trace-distance as a witness of non-Markovianity. △ Less

Submitted 13 November, 2020; originally announced November 2020.

Comments: 7 pages, 4 figures. arXiv admin note: text overlap with arXiv:1910.05951

Painting with Hue, Saturation, and Brightness Control by Nanoscale 3D Printing

Authors: Hao Wang, Qifeng Ruan, Hongtao Wang, Soroosh Daqiqeh Rezaei, Kevin T. P. Lim, Hailong Liu, Wang Zhang, Jonathan Trisno, John You En Chan, Joel K. W. Yang

Abstract: Varying only the in-plane or out-of-plane dimensions of nanostructures produces a wide range of colourful elements in metasurfaces and thin films. However, achieving shades of grey and control of colour saturation remains challenging. Here, we introduce a hybrid approach to colour generation based on the tuning of nanostructure geometry in all three dimensions. Through two-photon polymerization li… ▽ More Varying only the in-plane or out-of-plane dimensions of nanostructures produces a wide range of colourful elements in metasurfaces and thin films. However, achieving shades of grey and control of colour saturation remains challenging. Here, we introduce a hybrid approach to colour generation based on the tuning of nanostructure geometry in all three dimensions. Through two-photon polymerization lithography, we systematically investigated colour generation from the simple single nanopillar geometry made of low-refractive-index material; realizing grayscale and full colour palettes with control of hue, saturation, brightness through tuning of height, diameter, and periodicity of nanopillars. Arbitrary colourful and grayscale images were painted by mapping desired prints to precisely controllable parameters during 3D printing. We extend our understanding of the scattering properties of the low-refractive-index nanopillar to demonstrate grayscale inversion and colour desaturation, with steganography at the level of single nanopillars. △ Less

Submitted 31 October, 2020; v1 submitted 21 October, 2020; originally announced October 2020.

Comments: 22 pages, 5 figures

HyperTune: Dynamic Hyperparameter Tuning For Efficient Distribution of DNN Training Over Heterogeneous Systems

Authors: Ali HeydariGorji, Siavash Rezaei, Mahdi Torabzadehkashi, Hossein Bobarshad, Vladimir Alves, Pai H. Chou

Abstract: Distributed training is a novel approach to accelerate Deep Neural Networks (DNN) training, but common training libraries fall short of addressing the distributed cases with heterogeneous processors or the cases where the processing nodes get interrupted by other workloads. This paper describes distributed training of DNN on computational storage devices (CSD), which are NAND flash-based, high cap… ▽ More Distributed training is a novel approach to accelerate Deep Neural Networks (DNN) training, but common training libraries fall short of addressing the distributed cases with heterogeneous processors or the cases where the processing nodes get interrupted by other workloads. This paper describes distributed training of DNN on computational storage devices (CSD), which are NAND flash-based, high capacity data storage with internal processing engines. A CSD-based distributed architecture incorporates the advantages of federated learning in terms of performance scalability, resiliency, and data privacy by eliminating the unnecessary data movement between the storage device and the host processor. The paper also describes Stannis, a DNN training framework that improves on the shortcomings of existing distributed training frameworks by dynamically tuning the training hyperparameters in heterogeneous systems to maintain the maximum overall processing speed in term of processed images per second and energy efficiency. Experimental results on image classification training benchmarks show up to 3.1x improvement in performance and 2.45x reduction in energy consumption when using Stannis plus CSD compare to the generic systems. △ Less

Submitted 15 July, 2020; originally announced July 2020.

On the Difficulty of Membership Inference Attacks

Authors: Shahbaz Rezaei, Xin Liu

Abstract: Recent studies propose membership inference (MI) attacks on deep models, where the goal is to infer if a sample has been used in the training process. Despite their apparent success, these studies only report accuracy, precision, and recall of the positive class (member class). Hence, the performance of these attacks have not been clearly reported on negative class (non-member class). In this pape… ▽ More Recent studies propose membership inference (MI) attacks on deep models, where the goal is to infer if a sample has been used in the training process. Despite their apparent success, these studies only report accuracy, precision, and recall of the positive class (member class). Hence, the performance of these attacks have not been clearly reported on negative class (non-member class). In this paper, we show that the way the MI attack performance has been reported is often misleading because they suffer from high false positive rate or false alarm rate (FAR) that has not been reported. FAR shows how often the attack model mislabel non-training samples (non-member) as training (member) ones. The high FAR makes MI attacks fundamentally impractical, which is particularly more significant for tasks such as membership inference where the majority of samples in reality belong to the negative (non-training) class. Moreover, we show that the current MI attack models can only identify the membership of misclassified samples with mediocre accuracy at best, which only constitute a very small portion of training samples. We analyze several new features that have not been comprehensively explored for membership inference before, including distance to the decision boundary and gradient norms, and conclude that deep models' responses are mostly similar among train and non-train samples. We conduct several experiments on image classification tasks, including MNIST, CIFAR-10, CIFAR-100, and ImageNet, using various model architecture, including LeNet, AlexNet, ResNet, etc. We show that the current state-of-the-art MI attacks cannot achieve high accuracy and low FAR at the same time, even when the attacker is given several advantages. The source code is available at https://github.com/shrezaei/MI-Attack. △ Less

Submitted 22 March, 2021; v1 submitted 27 May, 2020; originally announced May 2020.

A Scalable Feature Selection and Opinion Miner Using Whale Optimization Algorithm

Authors: Amir Javadpour, Samira Rezaei, Kuan-Ching Li, Guojun Wang

Abstract: Due to the fast-growing volume of text documents and reviews in recent years, current analyzing techniques are not competent enough to meet the users' needs. Using feature selection techniques not only support to understand data better but also lead to higher speed and also accuracy. In this article, the Whale Optimization algorithm is considered and applied to the search for the optimum subset of… ▽ More Due to the fast-growing volume of text documents and reviews in recent years, current analyzing techniques are not competent enough to meet the users' needs. Using feature selection techniques not only support to understand data better but also lead to higher speed and also accuracy. In this article, the Whale Optimization algorithm is considered and applied to the search for the optimum subset of features. As known, F-measure is a metric based on precision and recall that is very popular in comparing classifiers. For the evaluation and comparison of the experimental results, PART, random tree, random forest, and RBF network classification algorithms have been applied to the different number of features. Experimental results show that the random forest has the best accuracy on 500 features. Keywords: Feature selection, Whale Optimization algorithm, Selecting optimal, Classification algorithm △ Less

Submitted 20 April, 2020; originally announced April 2020.