Showing 1–50 of 292 results for author: Smith, V

Abundances of planetary nebulae and evolved stars: Iron and sulfur depletion, and carbon and nitrogen enrichment, in low- and intermediate-mass stellar populations in the Milky Way

Authors: Letizia Stanghellini, Verne V. Smith, Katia Cunha, Nikos Prantzos

Abstract: We explore the elemental abundances in Galactic planetary nebulae (PNe) compared with those of their stellar progenitors (Red Giant Branch and Asymptotic Giant Branch, RGB and AGB, stars), to explore and quantify the expected -- i.e., due to AGB evolution or condensation onto grains -- differences. We gleaned the current literature for the nebular abundances while we used the APOGEE DR~17 survey d… ▽ More We explore the elemental abundances in Galactic planetary nebulae (PNe) compared with those of their stellar progenitors (Red Giant Branch and Asymptotic Giant Branch, RGB and AGB, stars), to explore and quantify the expected -- i.e., due to AGB evolution or condensation onto grains -- differences. We gleaned the current literature for the nebular abundances while we used the APOGEE DR~17 survey data for the stellar sample. We examined the elements in common between the nebular and stellar samples, namely, C, N, O, Fe, and S. We confirm that iron in PNe is mostly entrapped in grains, with an average depletion $<$D[Fe/H]$>$=1.741$\pm$0.486 dex, and we disclose a weak correlation between iron depletion and the [O/H] abundance, D[Fe/H]$=(6.6003\pm2.443)\times{\rm [O/H]} +(1.972\pm0.199)$. Sulfur may also be mildly depleted in PNe, with $<$D[S/H]$>=0.179\pm0.291$ dex. We also found an indication of nitrogen enrichment for PNe $<$E[N/H]$>$=0.393$\pm$0.421 dex, with maximum enrichment (0.980$\pm$0.243) occurring for the PNe whose progenitors have gone through the HBB. The carbon enrichment is $<$E[C/H]$>$=0.332$\pm$0.460 dex when measured for the general PN populations. Our results will be relevant for future Galactic and extragalactic studies comparing nebular and stellar samples. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: 20 pages, 13 Figures The Astrophysical Journal, in press

Revisiting Cascaded Ensembles for Efficient Inference

Authors: Steven Kolawole, Don Dennis, Ameet Talwalkar, Virginia Smith

Abstract: A common approach to make machine learning inference more efficient is to use example-specific adaptive schemes, which route or select models for each example at inference time. In this work we study a simple scheme for adaptive inference. We build a cascade of ensembles (CoE), beginning with resource-efficient models and growing to larger, more expressive models, where ensemble agreement serves a… ▽ More A common approach to make machine learning inference more efficient is to use example-specific adaptive schemes, which route or select models for each example at inference time. In this work we study a simple scheme for adaptive inference. We build a cascade of ensembles (CoE), beginning with resource-efficient models and growing to larger, more expressive models, where ensemble agreement serves as a data-dependent routing criterion. This scheme is easy to incorporate into existing inference pipelines, requires no additional training, and can be used to place models across multiple resource tiers--for instance, serving efficient models at the edge and invoking larger models in the cloud only when necessary. In cases where parallel inference is feasible, we show that CoE can improve accuracy relative to the single best model while reducing the average cost of inference by up to 7x, and provides Pareto-dominate solutions in accuracy and efficiency relative to existing adaptive inference baselines. These savings translate to an over 3x-reduction in total monetary cost when performing inference using a heterogeneous cluster of GPUs. Finally, for edge inference scenarios where portions of the cascade reside at the edge vs. in the cloud, CoE can provide a 14x reduction in communication cost and inference latency without sacrificing accuracy. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: ES-FOMO, ICML 2024

Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients

Authors: Aashiq Muhamed, Oscar Li, David Woodruff, Mona Diab, Virginia Smith

Abstract: Large language model (LLM) training and finetuning are often bottlenecked by limited GPU memory. While existing projection-based optimization methods address this by projecting gradients into a lower-dimensional subspace to reduce optimizer state memory, they typically rely on dense projection matrices, which can introduce computational and memory overheads. In this work, we propose Grass (GRAdien… ▽ More Large language model (LLM) training and finetuning are often bottlenecked by limited GPU memory. While existing projection-based optimization methods address this by projecting gradients into a lower-dimensional subspace to reduce optimizer state memory, they typically rely on dense projection matrices, which can introduce computational and memory overheads. In this work, we propose Grass (GRAdient Stuctured Sparsification), a novel approach that leverages sparse projections to transform gradients into structured sparse updates. This design not only significantly reduces memory usage for optimizer states but also minimizes gradient memory footprint, computation, and communication costs, leading to substantial throughput improvements. Extensive experiments on pretraining and finetuning tasks demonstrate that Grass achieves competitive performance to full-rank training and existing projection-based methods. Notably, Grass enables half-precision pretraining of a 13B parameter LLaMA model on a single 40GB A100 GPU--a feat infeasible for previous methods--and yields up to a $2\times$ throughput improvement on an 8-GPU system. Code can be found at https://github.com/aashiqmuhamed/GRASS . △ Less

Submitted 25 June, 2024; originally announced June 2024.

RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold

Authors: Amrith Setlur, Saurabh Garg, Xinyang Geng, Naman Garg, Virginia Smith, Aviral Kumar

Abstract: Training on model-generated synthetic data is a promising approach for finetuning LLMs, but it remains unclear when it helps or hurts. In this paper, we investigate this question for math reasoning via an empirical study, followed by building a conceptual understanding of our observations. First, we find that while the typical approach of finetuning a model on synthetic correct or positive problem… ▽ More Training on model-generated synthetic data is a promising approach for finetuning LLMs, but it remains unclear when it helps or hurts. In this paper, we investigate this question for math reasoning via an empirical study, followed by building a conceptual understanding of our observations. First, we find that while the typical approach of finetuning a model on synthetic correct or positive problem-solution pairs generated by capable models offers modest performance gains, sampling more correct solutions from the finetuned learner itself followed by subsequent fine-tuning on this self-generated data $\textbf{doubles}$ the efficiency of the same synthetic problems. At the same time, training on model-generated positives can amplify various spurious correlations, resulting in flat or even inverse scaling trends as the amount of data increases. Surprisingly, we find that several of these issues can be addressed if we also utilize negative responses, i.e., model-generated responses that are deemed incorrect by a final answer verifier. Crucially, these negatives must be constructed such that the training can appropriately recover the utility or advantage of each intermediate step in the negative response. With this per-step scheme, we are able to attain consistent gains over only positive data, attaining performance similar to amplifying the amount of synthetic data by $\mathbf{8 \times}$. We show that training on per-step negatives can help to unlearn spurious correlations in the positive data, and is equivalent to advantage-weighted reinforcement learning (RL), implying that it inherits robustness benefits of RL over imitating positive data alone. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Magnetic Fields of M Dwarfs from the Pleiades Open Cluster

Authors: Fabio Wanderley, Katia Cunha, Oleg Kochukhov, Verne V. Smith, Diogo Souto, Lyra Cao, Kevin Covey, Steven R. Majewski, Cintia Martinez, Philip S. Muirhead, Marc Pinsonneault, C. Allende Prieto, Keivan G. Stassun

Abstract: Average magnetic field measurements are presented for 62 M-dwarf members of the Pleiades open cluster, derived from Zeeman-enhanced Fe I lines in the H-band. An MCMC methodology was employed to model magnetic filling factors using SDSS-IV APOGEE high-resolution spectra, along with the radiative transfer code SYNMAST, MARCS stellar atmosphere models, and the APOGEE DR17 spectral line list. There is… ▽ More Average magnetic field measurements are presented for 62 M-dwarf members of the Pleiades open cluster, derived from Zeeman-enhanced Fe I lines in the H-band. An MCMC methodology was employed to model magnetic filling factors using SDSS-IV APOGEE high-resolution spectra, along with the radiative transfer code SYNMAST, MARCS stellar atmosphere models, and the APOGEE DR17 spectral line list. There is a positive correlation between mean magnetic fields and stellar rotation, with slow-rotator stars (Rossby number, Ro$>$0.13) exhibiting a steeper slope than rapid-rotators (Ro$<$0.13). However, the latter sample still shows a positive trend between Ro and magnetic fields, which is given by $<$B$>$ = 1604 $\times$ Ro$^{-0.20}$. The derived stellar radii, when compared with physical isochrones, show that on average, our sample shows radius inflation, with median enhanced radii ranging from +3.0$\%$ to +7.0$\%$, depending on the model. There is a positive correlation between magnetic field strength and radius inflation, as well as with stellar spot coverage, correlations that together indicate that stellar spot-filling factors generated by strong magnetic fields might be the mechanism that drives radius inflation in these stars. We also compare our derived magnetic fields with chromospheric emission lines (H$α$, H$β$ and Ca II K), as well as with X-ray and H$α$ to bolometric luminosity ratios, and find that stars with higher chromospheric and coronal activity tend to be more magnetic. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: Accepted for publication by The Astrophysical Journal (ApJ); doi:10.3847/1538-4357/ad571f

Jogging the Memory of Unlearned Model Through Targeted Relearning Attack

Authors: Shengyuan Hu, Yiwei Fu, Zhiwei Steven Wu, Virginia Smith

Abstract: Machine unlearning is a promising approach to mitigate undesirable memorization of training data in ML models. However, in this work we show that existing approaches for unlearning in LLMs are surprisingly susceptible to a simple set of targeted relearning attacks. With access to only a small and potentially loosely related set of data, we find that we can 'jog' the memory of unlearned models to r… ▽ More Machine unlearning is a promising approach to mitigate undesirable memorization of training data in ML models. However, in this work we show that existing approaches for unlearning in LLMs are surprisingly susceptible to a simple set of targeted relearning attacks. With access to only a small and potentially loosely related set of data, we find that we can 'jog' the memory of unlearned models to reverse the effects of unlearning. We formalize this unlearning-relearning pipeline, explore the attack across three popular unlearning benchmarks, and discuss future directions and guidelines that result from our study. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 17 pages, 8 figures, 12 tables

Federated LoRA with Sparse Communication

Authors: Kevin Kuo, Arian Raje, Kousik Rajesh, Virginia Smith

Abstract: Low-rank adaptation (LoRA) is a natural method for finetuning in communication-constrained machine learning settings such as cross-device federated learning. Prior work that has studied LoRA in the context of federated learning has focused on improving LoRA's robustness to heterogeneity and privacy. In this work, we instead consider techniques for further improving communication-efficiency in fede… ▽ More Low-rank adaptation (LoRA) is a natural method for finetuning in communication-constrained machine learning settings such as cross-device federated learning. Prior work that has studied LoRA in the context of federated learning has focused on improving LoRA's robustness to heterogeneity and privacy. In this work, we instead consider techniques for further improving communication-efficiency in federated LoRA. Unfortunately, we show that centralized ML methods that improve the efficiency of LoRA through unstructured pruning do not transfer well to federated settings. We instead study a simple approach, \textbf{FLASC}, that applies sparsity to LoRA during communication while allowing clients to locally fine-tune the entire LoRA module. Across four common federated learning tasks, we demonstrate that this method matches the performance of dense LoRA with up to $10\times$ less communication. Additionally, despite being designed primarily to target communication, we find that this approach has benefits in terms of heterogeneity and privacy relative to existing approaches tailored to these specific concerns. Overall, our work highlights the importance of considering system-specific constraints when developing communication-efficient finetuning approaches, and serves as a simple and competitive baseline for future work in federated finetuning. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Comments: 12 pages (excluding references), 8 figures

Stellar Characterization and Chemical Abundances of Exoplanet Hosting M dwarfs from APOGEE Spectra: Future JWST Targets

Authors: Edypo Melo, Diogo Souto, Katia Cunha, Verne V. Smith, Fábio Wanderley, Vinicius Grilo, Deusalete Camara, Kely Murta, Neda Hejazi, Ian J. M. Crossfield, Johanna Teske, Rafael Luque, Michael Zhang, Jacob Bean

Abstract: Exoplanets hosting M dwarfs are the best targets to characterize Earth-like or super-Earth planetary atmospheres with the James Webb Space Telescope (JWST). We determine detailed stellar parameters ($T_{\rm eff}$, log$g$, and $ξ$) and individual abundances of twelve elements for four cool M dwarfs hosting exoplanets TOI-1685, GJ 436, GJ 3470, and TOI-2445, scheduled for future observations by the… ▽ More Exoplanets hosting M dwarfs are the best targets to characterize Earth-like or super-Earth planetary atmospheres with the James Webb Space Telescope (JWST). We determine detailed stellar parameters ($T_{\rm eff}$, log$g$, and $ξ$) and individual abundances of twelve elements for four cool M dwarfs hosting exoplanets TOI-1685, GJ 436, GJ 3470, and TOI-2445, scheduled for future observations by the JWST. The analysis utilizes high-resolution near-infrared spectra from the SDSS-IV APOGEE survey between 1.51-1.69$μ$. Based on 1D-LTE plane-parallel models, we find that TOI-2445 is slightly metal-poor ([Fe/H] = -0.16$\pm$0.09), while TOI-1685, GJ 436 and GJ 3470 are more metal-rich ([Fe/H] = 0.06$\pm$0.18, 0.10$\pm$0.20 dex, 0.25$\pm$0.15). The derived C/O ratios for TOI-2445, TOI-1685, GJ 436, and GJ 3470 are 0.526$\pm$0.027, 0.558$\pm$0.097, 0.561$\pm$0.029, and 0.638$\pm$0.015, respectively. From results for 28 M dwarfs analyzed homogeneously from APOGEE spectra, we find exoplanet-hosting M dwarfs exhibit a C/O abundance ratio approximately 0.01 to 0.05 higher than those with non-detected exoplanets, at limits of a statistically significant offset. A linear regression of [Fe/H] \textit{vs.} C/O distribution reveals a noticeable difference in the angular coefficient between FGK dwarfs (0.27) and M dwarfs (0.13). Assuming our abundance ratios of Ca/Mg, Si/Mg, Al/Mg, and Fe/Mg, we determine a mass of 3.276$^{+0.448}_{-0.419}$$M_{\oplus}$ for TOI-2445 b, having density (6.793$^{+0.005}_{-0.099}$ g.cm$^{-3}$) and core mass fraction (0.329$_{-0.049}^{+0.028}$) very similar to Earth's. We also present an atlas of 113 well-defined spectral lines to analyze M dwarfs in the $H$-band and a comprehensive evaluation of uncertainties from variations in the atmospheric parameters, signal-to-noise, and pseudo-continuum. △ Less

Submitted 31 May, 2024; originally announced June 2024.

Comments: 32 pages, 3 figures. Accepted to the Astrophysical Journal (ApJ)

Stellar Characterization and a Chromospheric Activity Analysis of a K2 Sample of Planet-Hosting Stars

Authors: V. Loaiza-Tacuri, Katia Cunha, Verne V. Smith, F. Quispe-Huaynasi, Ellen Costa-Almeida, Luan Ghezzi, Jorge Melendez

Abstract: Effective temperatures, surface gravities, and iron abundances were derived for 109 stars observed by the K2 mission using equivalent width measurements of Fe I and Fe II lines. Calculations were carried out in LTE using Kurucz model atmospheres. Stellar masses and radii were derived by combining the stellar parameters with Gaia DR3 parallaxes, V-magnitudes, and isochrones. The derived stellar and… ▽ More Effective temperatures, surface gravities, and iron abundances were derived for 109 stars observed by the K2 mission using equivalent width measurements of Fe I and Fe II lines. Calculations were carried out in LTE using Kurucz model atmospheres. Stellar masses and radii were derived by combining the stellar parameters with Gaia DR3 parallaxes, V-magnitudes, and isochrones. The derived stellar and planetary radii have median internal precision of 1.8%, and 2.3%, respectively. The radius gap near $\rm R_{planet}\sim 1.9 R_\oplus$ was detected in this K2 sample. Chromospheric activity was measured from the Ca II H and K lines using the Values of $\log R^\prime_{\rm HK}$ were investigated as a function of stellar rotational period (P$_{rot}$) and we found that chromospheric activity decreases with increasing P$_{rot}$, although there is a large scatter in $\log R^\prime_{\rm HK}$ ($\sim$0.5) for a given P$_{rot}$. Activity levels in this sample reveal a paucity of F & G dwarfs with intermediate activity levels (Vaughan-Preston gap). The effect that stellar activity might have on the derivation of stellar parameters was investigated by including magnetically-sensitive Fe I lines in the analysis and we find no significant differences between parameters with and without magnetically-sensitive lines, although the more active stars ($\log R^\prime _{\rm HK}>-5.0$) exhibit a larger scatter in the differences in $T_{\rm eff}$ and [Fe/H]. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 19 pages, 13 figures, 5 tables

Science Fiction Media Representations of Exoplanets: Portrayals of Changing Astronomical Discoveries

Authors: Emma Johanna Puranen, Emily Finer, Christiane Helling, V Anne Smith

Abstract: Interest in science fiction's (SF's) potential science communication use is hindered by concerns about SF misrepresenting science. This study addresses these concerns by asking how SF media reflects scientific findings in exoplanet science. A database of SF exoplanets was analysed using a Bayesian network to find interconnected interactions between planetary characterisation features and literary… ▽ More Interest in science fiction's (SF's) potential science communication use is hindered by concerns about SF misrepresenting science. This study addresses these concerns by asking how SF media reflects scientific findings in exoplanet science. A database of SF exoplanets was analysed using a Bayesian network to find interconnected interactions between planetary characterisation features and literary data. Results reveal SF exoplanets designed after the discovery of real exoplanets are less Earth-like, providing statistical evidence that SF incorporates rapidly-evolving science. Understanding SF's portrayal of science is crucial for its potential use in science communication. △ Less

Submitted 4 March, 2024; originally announced May 2024.

Comments: 19 pages, 1 table, 2 figures

Journal ref: JCOM Journal of Science Communication 23(1) 2024

Privacy Amplification for the Gaussian Mechanism via Bounded Support

Authors: Shengyuan Hu, Saeed Mahloujifar, Virginia Smith, Kamalika Chaudhuri, Chuan Guo

Abstract: Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. These guarantees can be desirable compared to vanilla DP in real world settings as they tightly upper-bound the privacy leakage for a $\textit{specific}$ individual in an $\textit{actual}$… ▽ More Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. These guarantees can be desirable compared to vanilla DP in real world settings as they tightly upper-bound the privacy leakage for a $\textit{specific}$ individual in an $\textit{actual}$ dataset, rather than considering worst-case datasets. While these frameworks are beginning to gain popularity, to date, there is a lack of private mechanisms that can fully leverage advantages of data-dependent accounting. To bridge this gap, we propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting. Experiments on model training with DP-SGD show that using bounded support Gaussian mechanisms can provide a reduction of the pDP bound $ε$ by as much as 30% without negative effects on model utility. △ Less

Submitted 7 March, 2024; originally announced March 2024.

Comments: 23 pages, 4 figures

Many-Objective Multi-Solution Transport

Authors: Ziyue Li, Tian Li, Virginia Smith, Jeff Bilmes, Tianyi Zhou

Abstract: Optimizing the performance of many objectives (instantiated by tasks or clients) jointly with a few Pareto stationary solutions (models) is critical in machine learning. However, previous multi-objective optimization methods often focus on a few number of objectives and cannot scale to many objectives that outnumber the solutions, leading to either subpar performance or ignored objectives. We intr… ▽ More Optimizing the performance of many objectives (instantiated by tasks or clients) jointly with a few Pareto stationary solutions (models) is critical in machine learning. However, previous multi-objective optimization methods often focus on a few number of objectives and cannot scale to many objectives that outnumber the solutions, leading to either subpar performance or ignored objectives. We introduce Many-objective multi-solution Transport (MosT), a framework that finds multiple diverse solutions in the Pareto front of many objectives. Our insight is to seek multiple solutions, each performing as a domain expert and focusing on a specific subset of objectives while collectively covering all of them. MosT formulates the problem as a bi-level optimization of weighted objectives for each solution, where the weights are defined by an optimal transport between the objectives and solutions. Our algorithm ensures convergence to Pareto stationary solutions for complementary subsets of objectives. On a range of applications in federated learning, multi-task learning, and mixture-of-prompt learning for LLMs, MosT distinctly outperforms strong baselines, delivering high-quality, diverse solutions that profile the entire Pareto frontier, thus ensuring balanced trade-offs across many objectives. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Guardrail Baselines for Unlearning in LLMs

Authors: Pratiksha Thaker, Yash Maurya, Shengyuan Hu, Zhiwei Steven Wu, Virginia Smith

Abstract: Recent work has demonstrated that finetuning is a promising approach to 'unlearn' concepts from large language models. However, finetuning can be expensive, as it requires both generating a set of examples and running iterations of finetuning to update the model. In this work, we show that simple guardrail-based approaches such as prompting and filtering can achieve unlearning results comparable t… ▽ More Recent work has demonstrated that finetuning is a promising approach to 'unlearn' concepts from large language models. However, finetuning can be expensive, as it requires both generating a set of examples and running iterations of finetuning to update the model. In this work, we show that simple guardrail-based approaches such as prompting and filtering can achieve unlearning results comparable to finetuning. We recommend that researchers investigate these lightweight baselines when evaluating the performance of more computationally intensive finetuning methods. While we do not claim that methods such as prompting or filtering are universal solutions to the problem of unlearning, our work suggests the need for evaluation metrics that can better separate the power of guardrails vs. finetuning, and highlights scenarios where guardrails expose possible unintended behavior in existing metrics and benchmarks. △ Less

Submitted 11 June, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

Comments: Preliminary work, accepted to ICLR workshop SeT-LLM 2024

No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices

Authors: Qi Pang, Shengyuan Hu, Wenting Zheng, Virginia Smith

Abstract: Advances in generative models have made it possible for AI-generated text, code, and images to mirror human-generated content in many applications. Watermarking, a technique that aims to embed information in the output of a model to verify its source, is useful for mitigating the misuse of such AI-generated content. However, we show that common design choices in LLM watermarking schemes make the r… ▽ More Advances in generative models have made it possible for AI-generated text, code, and images to mirror human-generated content in many applications. Watermarking, a technique that aims to embed information in the output of a model to verify its source, is useful for mitigating the misuse of such AI-generated content. However, we show that common design choices in LLM watermarking schemes make the resulting systems surprisingly susceptible to attack -- leading to fundamental trade-offs in robustness, utility, and usability. To navigate these trade-offs, we rigorously study a set of simple yet effective attacks on common watermarking systems, and propose guidelines and defenses for LLM watermarking in practice. △ Less

Submitted 25 May, 2024; v1 submitted 25 February, 2024; originally announced February 2024.

A perspective on the Milky Way Bulge-Bar as seen from the neutron-capture elements Cerium and Neodymium with APOGEE

Authors: J. V. Sales-Silva, K. Cunha, V. V. Smith, S. Daflon, D. Souto, R. Guerço, A. Queiroz, C. Chiappini, C. R. Hayes, T. Masseron, Sten Hasselquist, D. Horta, N. Prantzos, M. Zoccali, C. Allende Prieto, B. Barbuy, R. Beaton, D. Bizyaev, J. G. Fernández-Trincado, P. M. Frinchaboy, J. A. Holtzman, J. A. Johnson, Henrik Jönsson, S. R. Majewski, D. Minniti , et al. (6 additional authors not shown)

Abstract: This study probes the chemical abundances of the neutron-capture elements cerium and neodymium in the inner Milky Way from an analysis of a sample of $\sim$2000 stars in the Galactic Bulge/bar spatially contained within $|X_{Gal}|<$5 kpc, $|Y_{Gal}|<$3.5 kpc, and $|Z_{Gal}|<$1 kpc, and spanning metallicities between $-$2.0$\lesssim$[Fe/H]$\lesssim$+0.5. We classify the sample stars into low- or hi… ▽ More This study probes the chemical abundances of the neutron-capture elements cerium and neodymium in the inner Milky Way from an analysis of a sample of $\sim$2000 stars in the Galactic Bulge/bar spatially contained within $|X_{Gal}|<$5 kpc, $|Y_{Gal}|<$3.5 kpc, and $|Z_{Gal}|<$1 kpc, and spanning metallicities between $-$2.0$\lesssim$[Fe/H]$\lesssim$+0.5. We classify the sample stars into low- or high-[Mg/Fe] populations and find that, in general, values of [Ce/Fe] and [Nd/Fe] increase as the metallicity decreases for the low- and high-[Mg/Fe] populations. Ce abundances show a more complex variation across the metallicity range of our Bulge-bar sample when compared to Nd, with the r-process dominating the production of neutron-capture elements in the high-[Mg/Fe] population ([Ce/Nd]$<$0.0). We find a spatial chemical dependence of Ce and Nd abundances for our sample of Bulge-bar stars, with low- and high-[Mg/Fe] populations displaying a distinct abundance distribution. In the region close to the center of the MW, the low-[Mg/Fe] population is dominated by stars with low [Ce/Fe], [Ce/Mg], [Nd/Mg], [Nd/Fe], and [Ce/Nd] ratios. The low [Ce/Nd] ratio indicates a significant contribution in this central region from r-process yields for the low-[Mg/Fe] population. The chemical pattern of the most metal-poor stars in our sample suggests an early chemical enrichment of the Bulge dominated by yields from core-collapse supernovae and r-process astrophysical sites, such as magneto-rotational supernovae. △ Less

Submitted 19 April, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

Comments: 21 pages, 10 figures, 1 table

Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes

Authors: Lucio Dery, Steven Kolawole, Jean-François Kagy, Virginia Smith, Graham Neubig, Ameet Talwalkar

Abstract: Given the generational gap in available hardware between lay practitioners and the most endowed institutions, LLMs are becoming increasingly inaccessible as they grow in size. Whilst many approaches have been proposed to compress LLMs to make their resource consumption manageable, these methods themselves tend to be resource intensive, putting them out of the reach of the very user groups they tar… ▽ More Given the generational gap in available hardware between lay practitioners and the most endowed institutions, LLMs are becoming increasingly inaccessible as they grow in size. Whilst many approaches have been proposed to compress LLMs to make their resource consumption manageable, these methods themselves tend to be resource intensive, putting them out of the reach of the very user groups they target. In this work, we explore the problem of structured pruning of LLMs using only forward passes. We seek to empower practitioners to prune models so large that their available hardware has just enough memory to run inference. We develop Bonsai, a gradient-free, perturbative pruning method capable of delivering small, fast, and accurate pruned models. We observe that Bonsai outputs pruned models that (i) outperform those generated by more expensive gradient-based structured pruning methods, and (ii) are twice as fast (with comparable accuracy) as those generated by semi-structured pruning methods requiring comparable resources as Bonsai. We also leverage Bonsai to produce a new sub-2B model using a single A6000 that yields state-of-the-art performance on 4/6 tasks on the Huggingface Open LLM leaderboard. △ Less

Submitted 9 February, 2024; v1 submitted 7 February, 2024; originally announced February 2024.

Comments: 15 pages, 4 fiigures, 15 tables

On the Benefits of Public Representations for Private Transfer Learning under Distribution Shift

Authors: Pratiksha Thaker, Amrith Setlur, Zhiwei Steven Wu, Virginia Smith

Abstract: Public pretraining is a promising approach to improve differentially private model training. However, recent work has noted that many positive research results studying this paradigm only consider in-distribution tasks, and may not apply to settings where there is distribution shift between the pretraining and finetuning data -- a scenario that is likely when finetuning private tasks due to the se… ▽ More Public pretraining is a promising approach to improve differentially private model training. However, recent work has noted that many positive research results studying this paradigm only consider in-distribution tasks, and may not apply to settings where there is distribution shift between the pretraining and finetuning data -- a scenario that is likely when finetuning private tasks due to the sensitive nature of the data. In this work, we show empirically across three tasks that even in settings with large distribution shift, where both zero-shot performance from public data and training from scratch with private data give unusably weak results, public features can in fact improve private training accuracy by up to 67\% over private training from scratch. We provide a theoretical explanation for this phenomenon, showing that if the public and private data share a low-dimensional representation, public representations can improve the sample complexity of private training even if it is impossible to learn the private task from the public data alone. Altogether, our results provide evidence that public data can indeed make private training practical in realistic settings of extreme distribution shift. △ Less

Submitted 1 September, 2024; v1 submitted 24 December, 2023; originally announced December 2023.

Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift

Authors: Saurabh Garg, Amrith Setlur, Zachary Chase Lipton, Sivaraman Balakrishnan, Virginia Smith, Aditi Raghunathan

Abstract: Self-training and contrastive learning have emerged as leading techniques for incorporating unlabeled data, both under distribution shift (unsupervised domain adaptation) and when it is absent (semi-supervised learning). However, despite the popularity and compatibility of these techniques, their efficacy in combination remains unexplored. In this paper, we undertake a systematic empirical investi… ▽ More Self-training and contrastive learning have emerged as leading techniques for incorporating unlabeled data, both under distribution shift (unsupervised domain adaptation) and when it is absent (semi-supervised learning). However, despite the popularity and compatibility of these techniques, their efficacy in combination remains unexplored. In this paper, we undertake a systematic empirical investigation of this combination, finding that (i) in domain adaptation settings, self-training and contrastive learning offer significant complementary gains; and (ii) in semi-supervised learning settings, surprisingly, the benefits are not synergistic. Across eight distribution shift datasets (e.g., BREEDs, WILDS), we demonstrate that the combined method obtains 3--8% higher accuracy than either approach independently. We then theoretically analyze these techniques in a simplified model of distribution shift, demonstrating scenarios under which the features produced by contrastive learning can yield a good initialization for self-training to further amplify gains and achieve optimal performance, even when either method alone would fail. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Comments: NeurIPS 2023

Identifying and Mitigating Privacy Risks Stemming from Language Models: A Survey

Authors: Victoria Smith, Ali Shahin Shamsabadi, Carolyn Ashurst, Adrian Weller

Abstract: Large Language Models (LLMs) have shown greatly enhanced performance in recent years, attributed to increased size and extensive training data. This advancement has led to widespread interest and adoption across industries and the public. However, training data memorization in Machine Learning models scales with model size, particularly concerning for LLMs. Memorized text sequences have the potent… ▽ More Large Language Models (LLMs) have shown greatly enhanced performance in recent years, attributed to increased size and extensive training data. This advancement has led to widespread interest and adoption across industries and the public. However, training data memorization in Machine Learning models scales with model size, particularly concerning for LLMs. Memorized text sequences have the potential to be directly leaked from LLMs, posing a serious threat to data privacy. Various techniques have been developed to attack LLMs and extract their training data. As these models continue to grow, this issue becomes increasingly critical. To help researchers and policymakers understand the state of knowledge around privacy attacks and mitigations, including where more work is needed, we present the first SoK on data privacy for LLMs. We (i) identify a taxonomy of salient dimensions where attacks differ on LLMs, (ii) systematize existing attacks, using our taxonomy of dimensions to highlight key trends, (iii) survey existing mitigation strategies, highlighting their strengths and limitations, and (iv) identify key gaps, demonstrating open problems and areas for concern. △ Less

Submitted 18 June, 2024; v1 submitted 27 September, 2023; originally announced October 2023.

Comments: 15 pages

Chemical abundances of the young inner-disk open cluster NGC 6705 observed by APOGEE: sodium-rich and not $α$-enhanced

Authors: V. Loaiza-Tacuri, K. Cunha, D. Souto, V. V. Smith, R. Guerço, C. Chiappini, J. V. Sales Silva, D. Horta, C. Allende Prieto, R. Beaton, D. Bizyaev, S. Daflon, P. Frinchaboy, S. Hasselquist, C. R. Hayes, J. A. Holtzman, H. Jönsson, S. R. Majewski, S. Mészáros, D. L. Nidever, M. Pinsonneault, G. Zasowski

Abstract: Previous results in the literature have found the young inner-disk open cluster NGC 6705 to be mildly $α$-enhanced. We examined this possibility via an independent chemical abundance analysis for 11 red-giant members of NGC 6705. The analysis is based on near-infrared APOGEE spectra and relies on LTE calculations using spherical model atmospheres and radiative transfer. We find a mean cluster meta… ▽ More Previous results in the literature have found the young inner-disk open cluster NGC 6705 to be mildly $α$-enhanced. We examined this possibility via an independent chemical abundance analysis for 11 red-giant members of NGC 6705. The analysis is based on near-infrared APOGEE spectra and relies on LTE calculations using spherical model atmospheres and radiative transfer. We find a mean cluster metallicity of $\rm [Fe/H] = +0.13 \pm 0.04$, indicating that NGC 6705 is metal-rich, as may be expected for a young inner-disk cluster. The mean $α$-element abundance relative to iron is $\rm \langle [α/Fe]\rangle =-0.03 \pm 0.05$, which is not at odds with expectations from general Galactic abundance trends. NGC 6705 also provides important probes for studying stellar mixing, given its turn-off mass of M$\sim$3.3 M$_\odot$. Its red giants have low $^{12}$C abundances ([$^{12}$C/Fe]=$-$0.16) and enhanced $^{14}$N abundances ([$^{14}$N/Fe]=+0.51), which are key signatures of the first dredge-up on the red giant branch. An additional signature of dredge-up was found in the Na abundances, which are enhanced by [Na/Fe]=+0.29, with a very small non-LTE correction. The $^{16}$O and Al abundances are found to be near-solar. All of the derived mixing-sensitive abundances are in agreement with stellar models of approximately 3.3 M$_{\odot}$ evolving along the red giant branch and onto the red clump. As found in young open clusters with similar metallicities, NGC 6705 exhibits a mild excess in the s-process element cerium, with $\rm [Ce/Fe] = +0.13\pm0.07$. △ Less

Submitted 19 September, 2023; originally announced September 2023.

Comments: 17 pages, 10 figures, 6 tables

Perishable Goods versus Re-tradable Assets: A Theoretical Reappraisal of a Fundamental Dichotomy

Authors: Sabiou Inoua, Vernon Smith

Abstract: Experimental results on market behavior establish a lower stability and efficiency of markets for durable re-tradable assets compared to markets for non-durable, or perishable, goods. In this chapter, we revisit this known but underappreciated dichotomy of goods in the light of our theory of competitive market price formation, and we emphasize the fundamental nature of the concept of asset re-trad… ▽ More Experimental results on market behavior establish a lower stability and efficiency of markets for durable re-tradable assets compared to markets for non-durable, or perishable, goods. In this chapter, we revisit this known but underappreciated dichotomy of goods in the light of our theory of competitive market price formation, and we emphasize the fundamental nature of the concept of asset re-tradability in neoclassical finance through a simple reformulation of the famous no-trade and no-arbitrage theorems. △ Less

Submitted 6 September, 2023; originally announced September 2023.

Journal ref: Handbook of Experimental Finance, S. Fullbrunn and E. Haruvy (eds), Edward Elgar Publishing (2022)

Exploring the Structures and Substructures of the Andromeda Satellite Dwarf Galaxies Cassiopeia III, Perseus I, and Lacerta I

Authors: Katherine L. Rhode, Nicholas J. Smith, Denija Crnojevic, David J. Sand, Ryan A. Lambert, Enrico Vesperini, Madison V. Smith, Steven Janowiecki, John J. Salzer, Ananthan Karunakaran, Kristine Spekkens

Abstract: We present results from wide-field imaging of the resolved stellar populations of the dwarf spheroidal galaxies Cassiopeia III (And XXXII) and Perseus I (And XXXIII), two satellites in the outer stellar halo of the Andromeda galaxy (M31). Our WIYN pODI photometry traces the red giant star population in each galaxy to ~2.5-3 half-light radii from the galaxy center. We use the Tip of the Red Giant B… ▽ More We present results from wide-field imaging of the resolved stellar populations of the dwarf spheroidal galaxies Cassiopeia III (And XXXII) and Perseus I (And XXXIII), two satellites in the outer stellar halo of the Andromeda galaxy (M31). Our WIYN pODI photometry traces the red giant star population in each galaxy to ~2.5-3 half-light radii from the galaxy center. We use the Tip of the Red Giant Branch (TRGB) method to derive distances of (m-M)_0 = 24.62+/-0.12 mag (839 (+48,-450) kpc, or 156 (+16,-13) kpc from M31) for Cas III and 24.47+/-0.13 mag (738 (+48,-45) kpc, or 351 (+17,-16) kpc from M31) for Per I. These values are consistent within the errors with TRGB distances derived from a deeper Hubble Space Telescope study of the galaxies' inner regions. For each galaxy, we derive structural parameters, total magnitude, and central surface brightness. We also place upper limits on the ratio of neutral hydrogen gas mass to optical luminosity, confirming the gas-poor nature of both galaxies. We combine our data set with corresponding data for the M31 satellite galaxy Lacerta I (And XXXI) from earlier work, and search for substructure within the RGB star populations of Cas III, Per I, and Lac I. We find an overdense region on the west side of Lac I at a significance level of 2.5-3-sigma and a low-significance filament extending in the direction of M31. In Cas III, we identify two modestly significant overdensities near the center of the galaxy and another at two half-light radii. Per I shows no evidence for substructure in its RGB star population, which may reflect this galaxy's isolated nature. △ Less

Submitted 2 September, 2023; originally announced September 2023.

Comments: 26 pages, 15 figures, 5 tables. Accepted for publication in The Astronomical Journal

Classical Economics: Lost and Found

Authors: Sabiou Inoua, Vernon Smith

Abstract: "Economists miss the boat when they act as if Arrow and Debreu's general equilibrium model accurately describes markets in the real world of constant change. In contrast, the classical view on the market mechanism offers a helpful foundation on which to add modern insights about how markets create and coordinate information." "Economists miss the boat when they act as if Arrow and Debreu's general equilibrium model accurately describes markets in the real world of constant change. In contrast, the classical view on the market mechanism offers a helpful foundation on which to add modern insights about how markets create and coordinate information." △ Less

Submitted 21 August, 2023; originally announced August 2023.

Journal ref: The Independent Review, Volume 25, Number 1, Summer 2020

A Tale of Two Disks: Mapping the Milky Way with the Final Data Release of APOGEE

Authors: Julie Imig, Cathryn Price, Jon A. Holtzman, Alexander Stone-Martinez, Steven R. Majewski, David H. Weinberg, Jennifer A. Johnson, Carlos Allende Prieto, Rachael L. Beaton, Timothy C. Beers, Dmitry Bizyaev, Michael R. Blanton, Joel R. Brownstein, Katia Cunha, José G. Fernández-Trincado, Diane K. Feuillet, Sten Hasselquist, Christian R. Hayes, Henrik Jönsson, Richard R. Lane, Jianhui Lian, Szabolcs Mészáros, David L. Nidever, Annie C. Robin, Matthew Shetrone , et al. (2 additional authors not shown)

Abstract: We present new maps of the Milky Way disk showing the distribution of metallicity ([Fe/H]), $α$-element abundances ([Mg/Fe]), and stellar age, using a sample of 66,496 red giant stars from the final data release (DR17) of the Apache Point Observatory Galactic Evolution Experiment (APOGEE) survey. We measure radial and vertical gradients, quantify the distribution functions for age and metallicity,… ▽ More We present new maps of the Milky Way disk showing the distribution of metallicity ([Fe/H]), $α$-element abundances ([Mg/Fe]), and stellar age, using a sample of 66,496 red giant stars from the final data release (DR17) of the Apache Point Observatory Galactic Evolution Experiment (APOGEE) survey. We measure radial and vertical gradients, quantify the distribution functions for age and metallicity, and explore chemical clock relations across the Milky Way for the low-$α$ disk, high-$α$ disk, and total population independently. The low-$α$ disk exhibits a negative radial metallicity gradient of $-0.06 \pm 0.001$ dex kpc$^{-1}$, which flattens with distance from the midplane. The high-$α$ disk shows a flat radial gradient in metallicity and age across nearly all locations of the disk. The age and metallicity distribution functions shift from negatively skewed in the inner Galaxy to positively skewed at large radius. Significant bimodality in the [Mg/Fe]-[Fe/H] plane and in the [Mg/Fe]-age relation persist across the entire disk. The age estimates have typical uncertainties of $\sim0.15$ in $\log$(age) and may be subject to additional systematic errors, which impose limitations on conclusions drawn from this sample. Nevertheless, these results act as critical constraints on galactic evolution models, constraining which physical processes played a dominant role in the formation of the Milky Way disk. We discuss how radial migration predicts many of the observed trends near the solar neighborhood and in the outer disk, but an additional more dramatic evolution history, such as the multi-infall model or a merger event, is needed to explain the chemical and age bimodality elsewhere in the Galaxy. △ Less

Submitted 25 July, 2023; originally announced July 2023.

Comments: 41 pages, 32 figures, accepted to ApJ

Journal ref: ApJ 954 124 (2023)

An Investigation of Non-Canonical Mixing in Red Giant Stars Using APOGEE 12C/13C Ratios Observed in Open Cluster Stars

Authors: Caroline McCormick, Steven R. Majewski, Verne V. Smith, Christian R. Hayes, Katia Cunha, Thomas Masseron, Achim Weiss, Matthew Shetrone, Andrés Almeida, Peter M. Frinchaboy, Domingo Aníbal García-Hernández, Christian Nitschelm

Abstract: Standard stellar evolution theory poorly predicts the surface abundances of chemical species in low-mass, red giant branch (RGB) stars. Observations show an enhancement of p-p chain and CNO cycle products in red giant envelopes, which suggests the existence of non-canonical mixing that brings interior burning products to the surface of these stars. The 12C/13C ratio is a highly sensitive abundance… ▽ More Standard stellar evolution theory poorly predicts the surface abundances of chemical species in low-mass, red giant branch (RGB) stars. Observations show an enhancement of p-p chain and CNO cycle products in red giant envelopes, which suggests the existence of non-canonical mixing that brings interior burning products to the surface of these stars. The 12C/13C ratio is a highly sensitive abundance metric used to probe this mixing. We investigate extra RGB mixing by examining (1) how 12C/13C is altered along the RGB and (2) how 12C/13C changes for stars of varying age and mass. Our sample consists of 43 red giants spread over 15 open clusters from the Sloan Digital Sky Survey's APOGEE DR17 that have reliable 12C/13C ratios derived from their APOGEE spectra. We vetted these 12C/13C ratios and compared them as a function of evolution and age/mass to the standard mixing model of stellar evolution and to a model that includes prescriptions for RGB thermohaline mixing and stellar rotation. We find that the observations deviate from standard mixing models, implying the need for extra mixing. Additionally, some of the abundance patterns depart from the thermohaline model, and it is unclear whether these differences are due to incomplete observations, issues inherent to the model, our assumption of the cause of extra mixing, or any combination of these factors. Nevertheless, the surface abundances across our age/mass range clearly deviate from the standard model, agreeing with the notion of a universal mechanism for RGB extra mixing in low-mass stars. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: 13 pages, 6 figures, accepted for publication in MNRAS

The Classical Theory of Supply and Demand

Authors: Sabiou Inoua, Vernon Smith

Abstract: This paper introduces and formalizes the classical view on supply and demand, which, we argue, has an integrity independent and distinct from the neoclassical theory. Demand and supply, before the marginal revolution, are defined not by an unobservable criterion such as a utility function, but by an observable monetary variable, the reservation price: the buyer's (maximum) willingness to pay (WTP)… ▽ More This paper introduces and formalizes the classical view on supply and demand, which, we argue, has an integrity independent and distinct from the neoclassical theory. Demand and supply, before the marginal revolution, are defined not by an unobservable criterion such as a utility function, but by an observable monetary variable, the reservation price: the buyer's (maximum) willingness to pay (WTP) value (a potential price) and the seller's (minimum) willingness to accept (WTA) value (a potential price) at the marketplace. Market demand and supply are the cumulative distribution of the buyers' and sellers' reservation prices, respectively. This WTP-WTA classical view of supply and demand formed the means whereby market participants were motivated in experimental economics although experimentalists (trained in neoclassical economics) were not cognizant of their link to the past. On this foundation was erected a vast literature on the rules of trading for a host of institutions, modern and ancient. This paper documents textually this reappraisal of classical economics and then formalizes it mathematically. A follow-up paper will articulate a theory of market price formation rooted in this classical view on supply and demand and in experimental findings on market behavior. △ Less

Submitted 1 July, 2023; originally announced July 2023.

Adam Smith's Theory of Value: A Reappraisal of Classical Price Discovery

Authors: Sabiou Inoua, Vernon Smith

Abstract: The relevance of Adam Smith for understanding human morality and sociality is recognized in the growing interest in his work on moral sentiments among scholars of various academic backgrounds. But, paradoxically, Adam Smith's theory of economic value enjoys a less prominent stature today among economists, who, while they view him as the 'father of modern economics', considered him more as having h… ▽ More The relevance of Adam Smith for understanding human morality and sociality is recognized in the growing interest in his work on moral sentiments among scholars of various academic backgrounds. But, paradoxically, Adam Smith's theory of economic value enjoys a less prominent stature today among economists, who, while they view him as the 'father of modern economics', considered him more as having had the right intuitions about a market economy than as having developed the right concepts and the technical tools for studying it. Yet the neoclassical tradition, which replaced the classical school around 1870, failed to provide a satisfactory theory of market price formation. Adam Smith's sketch of market price formation (Ch. VII, Book I, Wealth of Nations), and more generally the classical view of competition as a collective higgling and bargaining process, as this paper argues, offers a helpful foundation on which to build a modern theory of market price formation, despite any shortcomings of the original classical formulation (notably its insistence on long-run, natural value). Also, with hindsight, the experimental market findings established the remarkable stability, efficiency, and robustness of the old view of competition, suggesting a rehabilitation of classical price discovery. This paper reappraises classical price theory as Adam Smith articulated it; we explicate key propositions from his price theory and derive them from a simple model, which is an elementary sketch of the authors' more general theory of competitive market price formation. △ Less

Submitted 1 July, 2023; originally announced July 2023.

A Classical Model of Speculative Asset Price Dynamics

Authors: Sabiou Inoua, Vernon Smith

Abstract: In retrospect, the experimental findings on competitive market behavior called for a revival of the old, classical, view of competition as a collective higgling and bargaining process (as opposed to price-taking behaviors) founded on reservation prices (in place of the utility function). In this paper, we specialize the classical methodology to deal with speculation, an important impediment to pri… ▽ More In retrospect, the experimental findings on competitive market behavior called for a revival of the old, classical, view of competition as a collective higgling and bargaining process (as opposed to price-taking behaviors) founded on reservation prices (in place of the utility function). In this paper, we specialize the classical methodology to deal with speculation, an important impediment to price stability. The model involves typical features of a field or lab asset market setup and lends itself to an experimental test of its specific predictions; here we use the model to explain three general stylized facts, well established both empirically and experimentally: the excess, fat-tailed, and clustered volatility of speculative asset prices. The fat tails emerge in the model from the amplifying nature of speculation, leading to a random-coefficient autoregressive return process (and power-law tails); the volatility clustering is due to the traders' long memory of news; bubbles are a persistent phenomenon in the model, and, assuming the standard lab present value pattern, the bubble size increases with the proportion of speculators and decreases with the trading horizon. △ Less

Submitted 1 July, 2023; originally announced July 2023.

NANCY: Next-generation All-sky Near-infrared Community surveY

Authors: Jiwon Jesse Han, Arjun Dey, Adrian M. Price-Whelan, Joan Najita, Edward F. Schlafly, Andrew Saydjari, Risa H. Wechsler, Ana Bonaca, David J Schlegel, Charlie Conroy, Anand Raichoor, Alex Drlica-Wagner, Juna A. Kollmeier, Sergey E. Koposov, Gurtina Besla, Hans-Walter Rix, Alyssa Goodman, Douglas Finkbeiner, Abhijeet Anand, Matthew Ashby, Benedict Bahr-Kalus, Rachel Beaton, Jayashree Behera, Eric F. Bell, Eric C Bellm , et al. (184 additional authors not shown)

Abstract: The Nancy Grace Roman Space Telescope is capable of delivering an unprecedented all-sky, high-spatial resolution, multi-epoch infrared map to the astronomical community. This opportunity arises in the midst of numerous ground- and space-based surveys that will provide extensive spectroscopy and imaging together covering the entire sky (such as Rubin/LSST, Euclid, UNIONS, SPHEREx, DESI, SDSS-V, GAL… ▽ More The Nancy Grace Roman Space Telescope is capable of delivering an unprecedented all-sky, high-spatial resolution, multi-epoch infrared map to the astronomical community. This opportunity arises in the midst of numerous ground- and space-based surveys that will provide extensive spectroscopy and imaging together covering the entire sky (such as Rubin/LSST, Euclid, UNIONS, SPHEREx, DESI, SDSS-V, GALAH, 4MOST, WEAVE, MOONS, PFS, UVEX, NEO Surveyor, etc.). Roman can uniquely provide uniform high-spatial-resolution (~0.1 arcsec) imaging over the entire sky, vastly expanding the science reach and precision of all of these near-term and future surveys. This imaging will not only enhance other surveys, but also facilitate completely new science. By imaging the full sky over two epochs, Roman can measure the proper motions for stars across the entire Milky Way, probing 100 times fainter than Gaia out to the very edge of the Galaxy. Here, we propose NANCY: a completely public, all-sky survey that will create a high-value legacy dataset benefiting innumerable ongoing and forthcoming studies of the universe. NANCY is a pure expression of Roman's potential: it images the entire sky, at high spatial resolution, in a broad infrared bandpass that collects as many photons as possible. The majority of all ongoing astronomical surveys would benefit from incorporating observations of NANCY into their analyses, whether these surveys focus on nearby stars, the Milky Way, near-field cosmology, or the broader universe. △ Less

Submitted 20 June, 2023; originally announced June 2023.

Comments: Submitted to the call for white papers for the Roman Core Community Survey (June 16th, 2023), and to the Bulletin of the AAS

Revealing the Chemical Structure of the Magellanic Clouds with APOGEE. I. Calculating Individual Stellar Ages of RGB Stars in the Large Magellanic Cloud

Authors: Joshua T. Povick, David L. Nidever, Pol Massana, Jamie Tayar, Knut A. G. Olsen, Sten Hasselquist, Maria-Rosa L. Cioni, Christian Nitschelm, Ricardo Carrera, Yumi Choi, Alexandre Roman-Lopes, Steven R. Majewski, Andrés Almeida, Katia Cunha, Verne V. Smith

Abstract: Stellar ages are critical for understanding the temporal evolution of a galaxy. We calculate the ages of over 6000 red giant branch stars in the Large Magellanic Cloud (LMC) observed with SDSS-IV / APOGEE-S. Ages are derived using multi-band photometry, spectroscopic parameters (T$_\text{eff}$, $\log{g}$, [Fe/H], and [$α$/Fe]) and stellar isochrones and the assumption that the stars lie in a thin… ▽ More Stellar ages are critical for understanding the temporal evolution of a galaxy. We calculate the ages of over 6000 red giant branch stars in the Large Magellanic Cloud (LMC) observed with SDSS-IV / APOGEE-S. Ages are derived using multi-band photometry, spectroscopic parameters (T$_\text{eff}$, $\log{g}$, [Fe/H], and [$α$/Fe]) and stellar isochrones and the assumption that the stars lie in a thin inclined plane to get accurate distances. The isochrone age and extinction are varied until a best match is found for the observed photometry. We perform validation using the APOKASC sample, which has asteroseismic masses and accurate ages, and find that our uncertainties are $\sim$20% and range from $\sim$1$-$3 Gyr for the calculated age values. Here we present the LMC age map as well as the age-radius relation and an accurate age-metallicity relation (AMR). The age map and age-radius relation reveal that recent star formation in the galaxy was more centrally located and that there is a slight dichotomy between the north and south with the northern fields being slightly younger. The northern fields that cover a known spiral arm have median ages of $\gtrsim$ 2 Gyr, which is the time when an interaction with the SMC is suggested to have happened. The AMR is mostly flat especially for older ages although recently (about 2.0-2.5 Gyr ago) there is an increase in the median [Fe/H]. Based on the time frame, this might also be attributed to the close interaction between the LMC and SMC. △ Less

Submitted 10 June, 2023; originally announced June 2023.

Comments: 21 pages, 22 figures, submitted to MNRAS

GTC Follow-up Observations of Very Metal-Poor Star Candidates from DESI

Authors: Carlos Allende Prieto, David S. Aguado, Jonay I. González Hernández, Rafael Rebolo, Joan Najita, Christopher J. Manser, Constance Rockosi, Zachary Slepian, Mar Mezcua, Monica Valluri, Rana Ezzeddine, Sergey E. Koposov, Andrew P. Cooper, Arjun Dey, Boris T. Gänsicke, Ting S. Li, Katia Cunha, Siwei Zou, Jessica Nicole Aguilar, Steven Ahlen, David Brooks, Todd Claybaugh, Shaun Cole, Sarah Eftekharzadeh, Kevin Fanning , et al. (26 additional authors not shown)

Abstract: The observations from the Dark Energy Spectroscopic Instrument (DESI) will significantly increase the numbers of known extremely metal-poor stars by a factor of ~ 10, improving the sample statistics to study the early chemical evolution of the Milky Way and the nature of the first stars. In this paper we report high signal-to-noise follow-up observations of 9 metal-poor stars identified during the… ▽ More The observations from the Dark Energy Spectroscopic Instrument (DESI) will significantly increase the numbers of known extremely metal-poor stars by a factor of ~ 10, improving the sample statistics to study the early chemical evolution of the Milky Way and the nature of the first stars. In this paper we report high signal-to-noise follow-up observations of 9 metal-poor stars identified during the DESI commissioning with the Optical System for Imaging and low-Intermediate-Resolution Integrated Spectroscopy (OSIRIS) instrument on the 10.4m Gran Telescopio Canarias (GTC). The analysis of the data using a well-vetted methodology confirms the quality of the DESI spectra and the performance of the pipelines developed for the data reduction and analysis of DESI data. △ Less

Submitted 27 October, 2023; v1 submitted 9 June, 2023; originally announced June 2023.

Comments: 14 pages, 4 figures, accepted for publication in ApJ, data available from https://doi.org/10.5281/zenodo.8363303

The Prevalence of the $α$-bimodality: First JWST $α$-abundance Results in M31

Authors: David L. Nidever, Karoline Gilbert, Erik Tollerud, Charles Siders, Ivanna Escala, Carlos Allende Prieto, Verne Smith, Katia Cunha, Victor P. Debattista, Yuan-Sen Ting, Evan N. Kirby

Abstract: We present initial results from our JWST NIRSpec program to study the $α$-abundances in the M31 disk. The Milky Way has two chemically-defined disks, the low-$α$ and high-$α$ disks, which are closely related to the thin and thick disks, respectively. The origin of the two populations and the $α$-bimodality between them is not entirely clear, although there are now several models that can reproduce… ▽ More We present initial results from our JWST NIRSpec program to study the $α$-abundances in the M31 disk. The Milky Way has two chemically-defined disks, the low-$α$ and high-$α$ disks, which are closely related to the thin and thick disks, respectively. The origin of the two populations and the $α$-bimodality between them is not entirely clear, although there are now several models that can reproduce the observed features. To help constrain the models and discern the origin, we have undertaken a study of the chemical abundances of the M31 disk using JWST NIRSpec, in order to determine whether stars in M31's disk also show an $α$-abundance bimodality. Approximately 100 stars were observed in our single NIRSpec field at a projected distance of 18 kpc from the M31 center. The 1-D extracted spectra have an average signal-to-noise ratio of 85 leading to statistical metallicity precision of 0.016 dex, $α$-abundance precision of 0.012 dex, and a radial velocity precision 8 km/s. The initial results indicate that, in contrast to the Milky Way, there is no $α$-bimodality in the M31 disk, and no low-$α$ sequence. The entire stellar population falls along a single chemical sequence very similar to the MW's high-alpha component which had a high star formation rate. While this is somewhat unexpected, the result is not that surprising based on other studies that found the M31 disk has a larger velocity dispersion than the MW and is dominated by a thick component. M31 has had a more active accretion and merger history than the MW which might explain the chemical differences. △ Less

Submitted 7 June, 2023; originally announced June 2023.

Comments: 8 pages, 4 figures, IAU Symposium 377, Early Disk-Galaxy Formation: From JWST to the Milky Way

Stellar Characterization and Radius Inflation of Hyades M Dwarf Stars From the APOGEE Survey

Authors: Fábio Wanderley, Katia Cunha, Diogo Souto, Verne V. Smith, Lyra Cao, Marc Pinsonneault, C. Allende Prieto, Kevin Covey, Thomas Masseron, Ilaria Pascucci, Keivan G. Stassun, Ryan Terrien, Galen J. Bergsten, Dmitry Bizyaev, José G. Fernández-Trincado, Henrik Jönsson, Sten Hasselquist, Jon A. Holtzman, Richard R. Lane, Suvrath Mahadevan, Steven R. Majewski, Dante Minniti, Kaike Pan, Javier Serna, Jennifer Sobeck , et al. (1 additional authors not shown)

Abstract: We present a spectroscopic analysis of a sample of 48 M dwarf stars ($0.2 M_{\odot}< M < 0.6 M_{\odot}$) from the Hyades open cluster using high-resolution H-band spectra from the SDSS/APOGEE survey. Our methodology adopts spectrum synthesis with LTE MARCS model atmospheres, along with the APOGEE DR17 line list, to determine effective temperatures, surface gravities, metallicities, and projected r… ▽ More We present a spectroscopic analysis of a sample of 48 M dwarf stars ($0.2 M_{\odot}< M < 0.6 M_{\odot}$) from the Hyades open cluster using high-resolution H-band spectra from the SDSS/APOGEE survey. Our methodology adopts spectrum synthesis with LTE MARCS model atmospheres, along with the APOGEE DR17 line list, to determine effective temperatures, surface gravities, metallicities, and projected rotational velocities. The median metallicity obtained for the Hyades M dwarfs is [M/H]= 0.09$\pm$0.03 dex, indicating a small internal uncertainty and good agreement with optical results for Hyades red-giants. Overall, the median radii are larger than predicted by stellar models by 1.6$\pm$2.3\% and 2.4$\pm$2.3\%, relative to a MIST and DARTMOUTH isochrone, respectively. We emphasize, however, that these isochrones are different and the fractional radius inflation for the fully- and partially-convective regimes have distinct behaviors depending on the isochrone. Using a MIST isochrone there is no evidence of radius inflation for the fully convective stars, while for the partially convective M-dwarfs the radii are inflated by 2.7$\pm$2.1\%, which is in agreement with predictions from models that include magnetic fields. For the partially-convective stars, rapid-rotators present on average higher inflation levels than slow-rotators. The comparison with SPOTS isochrone models indicates that the derived M dwarf radii can be explained by accounting for stellar spots in the photosphere of the stars, with 76\% of the studied M dwarfs having up to 20\% spot coverage, and the most inflated stars with $\sim$20 -- 40\% spot coverage. △ Less

Submitted 11 May, 2023; originally announced May 2023.

Comments: Accepted for publication by The Astrophysical Journal (ApJ)

Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies

Authors: Oscar Li, James Harrison, Jascha Sohl-Dickstein, Virginia Smith, Luke Metz

Abstract: Unrolled computation graphs are prevalent throughout machine learning but present challenges to automatic differentiation (AD) gradient estimation methods when their loss functions exhibit extreme local sensitivtiy, discontinuity, or blackbox characteristics. In such scenarios, online evolution strategies methods are a more capable alternative, while being more parallelizable than vanilla evolutio… ▽ More Unrolled computation graphs are prevalent throughout machine learning but present challenges to automatic differentiation (AD) gradient estimation methods when their loss functions exhibit extreme local sensitivtiy, discontinuity, or blackbox characteristics. In such scenarios, online evolution strategies methods are a more capable alternative, while being more parallelizable than vanilla evolution strategies (ES) by interleaving partial unrolls and gradient updates. In this work, we propose a general class of unbiased online evolution strategies methods. We analytically and empirically characterize the variance of this class of gradient estimators and identify the one with the least variance, which we term Noise-Reuse Evolution Strategies (NRES). Experimentally, we show NRES results in faster convergence than existing AD and ES methods in terms of wall-clock time and number of unroll steps across a variety of applications, including learning dynamical systems, meta-training learned optimizers, and reinforcement learning. △ Less

Submitted 9 December, 2023; v1 submitted 21 April, 2023; originally announced April 2023.

Comments: NeurIPS 2023. 41 pages. Code available at https://github.com/OscarcarLi/Noise-Reuse-Evolution-Strategies

Progressive Ensemble Distillation: Building Ensembles for Efficient Inference

Authors: Don Kurian Dennis, Abhishek Shetty, Anish Sevekari, Kazuhito Koishida, Virginia Smith

Abstract: We study the problem of progressive ensemble distillation: Given a large, pretrained teacher model $g$, we seek to decompose the model into smaller, low-inference cost student models $f_i$, such that progressively evaluating additional models in this ensemble leads to improved predictions. The resulting ensemble allows for flexibly tuning accuracy vs. inference cost at runtime, which is useful for… ▽ More We study the problem of progressive ensemble distillation: Given a large, pretrained teacher model $g$, we seek to decompose the model into smaller, low-inference cost student models $f_i$, such that progressively evaluating additional models in this ensemble leads to improved predictions. The resulting ensemble allows for flexibly tuning accuracy vs. inference cost at runtime, which is useful for a number of applications in on-device inference. The method we propose, B-DISTIL , relies on an algorithmic procedure that uses function composition over intermediate activations to construct expressive ensembles with similar performance as $g$ , but with smaller student models. We demonstrate the effectiveness of B-DISTIL by decomposing pretrained models across standard image, speech, and sensor datasets. We also provide theoretical guarantees in terms of convergence and generalization. △ Less

Submitted 9 November, 2023; v1 submitted 20 February, 2023; originally announced February 2023.

Federated Learning as a Network Effects Game

Authors: Shengyuan Hu, Dung Daniel Ngo, Shuran Zheng, Virginia Smith, Zhiwei Steven Wu

Abstract: Federated Learning (FL) aims to foster collaboration among a population of clients to improve the accuracy of machine learning without directly sharing local data. Although there has been rich literature on designing federated learning algorithms, most prior works implicitly assume that all clients are willing to participate in a FL scheme. In practice, clients may not benefit from joining in FL,… ▽ More Federated Learning (FL) aims to foster collaboration among a population of clients to improve the accuracy of machine learning without directly sharing local data. Although there has been rich literature on designing federated learning algorithms, most prior works implicitly assume that all clients are willing to participate in a FL scheme. In practice, clients may not benefit from joining in FL, especially in light of potential costs related to issues such as privacy and computation. In this work, we study the clients' incentives in federated learning to help the service provider design better solutions and ensure clients make better decisions. We are the first to model clients' behaviors in FL as a network effects game, where each client's benefit depends on other clients who also join the network. Using this setup we analyze the dynamics of clients' participation and characterize the equilibrium, where no client has incentives to alter their decision. Specifically, we show that dynamics in the population naturally converge to equilibrium without needing explicit interventions. Finally, we provide a cost-efficient payment scheme that incentivizes clients to reach a desired equilibrium when the initial network is empty. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Comments: 14 pages of main text, 26 pages in total

Bitrate-Constrained DRO: Beyond Worst Case Robustness To Unknown Group Shifts

Authors: Amrith Setlur, Don Dennis, Benjamin Eysenbach, Aditi Raghunathan, Chelsea Finn, Virginia Smith, Sergey Levine

Abstract: Training machine learning models robust to distribution shifts is critical for real-world applications. Some robust training algorithms (e.g., Group DRO) specialize to group shifts and require group information on all training points. Other methods (e.g., CVaR DRO) that do not need group annotations can be overly conservative, since they naively upweight high loss points which may form a contrived… ▽ More Training machine learning models robust to distribution shifts is critical for real-world applications. Some robust training algorithms (e.g., Group DRO) specialize to group shifts and require group information on all training points. Other methods (e.g., CVaR DRO) that do not need group annotations can be overly conservative, since they naively upweight high loss points which may form a contrived set that does not correspond to any meaningful group in the real world (e.g., when the high loss points are randomly mislabeled training points). In this work, we address limitations in prior approaches by assuming a more nuanced form of group shift: conditioned on the label, we assume that the true group function (indicator over group) is simple. For example, we may expect that group shifts occur along low bitrate features (e.g., image background, lighting). Thus, we aim to learn a model that maintains high accuracy on simple group functions realized by these low bitrate features, that need not spend valuable model capacity achieving high accuracy on contrived groups of examples. Based on this, we consider the two-player game formulation of DRO where the adversary's capacity is bitrate-constrained. Our resulting practical algorithm, Bitrate-Constrained DRO (BR-DRO), does not require group information on training samples yet matches the performance of Group DRO on datasets that have training group annotations and that of CVaR DRO on long-tailed distributions. Our theoretical analysis reveals that in some settings BR-DRO objective can provably yield statistically efficient and less conservative solutions than unconstrained CVaR DRO. △ Less

Submitted 11 October, 2023; v1 submitted 6 February, 2023; originally announced February 2023.

Journal ref: ICLR 2023

Symbiotic and antagonistic disease dynamics on networks using bond percolatio

Authors: Peter Mann, V. Anne Smith, John Mitchell, Simon Dobson

Abstract: In this paper we introduce a novel description of the equilibrium state of a bond percolation process on random graphs using the exact method of generating functions. This allows us to find the expected size of the giant connected component (GCC) of two sequential bond percolation processes in which the bond occupancy probability of the second process is modulated (increased or decreased) by a nod… ▽ More In this paper we introduce a novel description of the equilibrium state of a bond percolation process on random graphs using the exact method of generating functions. This allows us to find the expected size of the giant connected component (GCC) of two sequential bond percolation processes in which the bond occupancy probability of the second process is modulated (increased or decreased) by a node being inside or outside of the GCC created by the first process. In the context of epidemic spreading this amounts to both a antagonistic partial immunity or a synergistic partial coinfection interaction between the two sequential diseases. We examine configuration model networks with tunable clustering. We find that the emergent evolutionary behaviour of the second strain is highly dependent on the details of the coupling between the strains. Contact clustering generally reduces the outbreak size of the second strain relative to unclustered topologies; however, positive assortativity induced by clustered contacts inverts this conclusion for highly transmissible disease dynamics. △ Less

Submitted 19 January, 2023; originally announced January 2023.

Comments: 10 pages, 5 figures

Journal ref: Phys. Rev. E 104, 024303. 2021

A Spectroscopic Analysis of a Sample of K2 Planet-Host Stars: Stellar Parameters, Metallicities and Planetary Radii

Authors: V. Loaiza-Tacuri, Katia Cunha, Verne V. Smith, Cintia F. Martinez, Luan Ghezzi, Simon C. Schuler, Johanna Teske, Steve B. Howell

Abstract: The physical properties of transiting exoplanets are connected with the physical properties of their host stars. We present a homogeneous spectroscopic analysis based on spectra of FGK-type stars observed with the Hydra spectrograph on the WIYN telescope. We derived effective temperatures, surface gravities, and metallicities, for 81 stars observed by K2 and 33 from Kepler 1. We constructed an Fe… ▽ More The physical properties of transiting exoplanets are connected with the physical properties of their host stars. We present a homogeneous spectroscopic analysis based on spectra of FGK-type stars observed with the Hydra spectrograph on the WIYN telescope. We derived effective temperatures, surface gravities, and metallicities, for 81 stars observed by K2 and 33 from Kepler 1. We constructed an Fe I and II line list that is adequate for the analysis of R$\sim$18,000 spectra covering 6050-6350 Å and adopted the spectroscopic technique based on equivalent width measurements. The calculations were done in LTE using Kurucz model atmospheres and the qoyllur-quipu (q$^2$) package. We validated our methodology via analysis of a benchmark solar twin and solar proxies, which are used as the solar reference. We estimated the effects that including Zeeman sensitive Fe I lines have on the derived stellar parameters for young and possibly active stars in our sample and found it not to be significant. Stellar masses and radii were derived by combining the stellar parameters with Gaia EDR3 and V magnitudes and isochrones. The measured stellar radii have 4.2\% median internal precision, leading to a median internal uncertainty of 4.4\% in the derived planetary radii. With our sample of 83 confirmed planets orbiting K2 host stars, the radius gap near R$_{planet}1.9R{_\plus}$ is detected, in agreement with previous findings. Relations between the planetary radius, orbital period and metallicity are explored and these also confirm previous findings for Kepler 1 systems. △ Less

Submitted 10 January, 2023; originally announced January 2023.

Comments: 35 pages, 17 figures

On Noisy Evaluation in Federated Hyperparameter Tuning

Authors: Kevin Kuo, Pratiksha Thaker, Mikhail Khodak, John Nguyen, Daniel Jiang, Ameet Talwalkar, Virginia Smith

Abstract: Hyperparameter tuning is critical to the success of federated learning applications. Unfortunately, appropriately selecting hyperparameters is challenging in federated networks. Issues of scale, privacy, and heterogeneity introduce noise in the tuning process and make it difficult to evaluate the performance of various hyperparameters. In this work, we perform the first systematic study on the eff… ▽ More Hyperparameter tuning is critical to the success of federated learning applications. Unfortunately, appropriately selecting hyperparameters is challenging in federated networks. Issues of scale, privacy, and heterogeneity introduce noise in the tuning process and make it difficult to evaluate the performance of various hyperparameters. In this work, we perform the first systematic study on the effect of noisy evaluation in federated hyperparameter tuning. We first identify and rigorously explore key sources of noise, including client subsampling, data and systems heterogeneity, and data privacy. Surprisingly, our results indicate that even small amounts of noise can significantly impact tuning methods-reducing the performance of state-of-the-art approaches to that of naive baselines. To address noisy evaluation in such scenarios, we propose a simple and effective approach that leverages public proxy data to boost the evaluation signal. Our work establishes general challenges, baselines, and best practices for future work in federated hyperparameter tuning. △ Less

Submitted 15 May, 2023; v1 submitted 17 December, 2022; originally announced December 2022.

Comments: v1: 19 pages, 15 figures, submitted to MLSys2023; v2: Fixed citation formatting; v3: Fixed typo, update acks v4: MLSys2023 camera-ready

Differentially Private Adaptive Optimization with Delayed Preconditioners

Authors: Tian Li, Manzil Zaheer, Ken Ziyu Liu, Sashank J. Reddi, H. Brendan McMahan, Virginia Smith

Abstract: Privacy noise may negate the benefits of using adaptive optimizers in differentially private model training. Prior works typically address this issue by using auxiliary information (e.g., public data) to boost the effectiveness of adaptive optimization. In this work, we explore techniques to estimate and efficiently adapt to gradient geometry in private adaptive optimization without auxiliary data… ▽ More Privacy noise may negate the benefits of using adaptive optimizers in differentially private model training. Prior works typically address this issue by using auxiliary information (e.g., public data) to boost the effectiveness of adaptive optimization. In this work, we explore techniques to estimate and efficiently adapt to gradient geometry in private adaptive optimization without auxiliary data. Motivated by the observation that adaptive methods can tolerate stale preconditioners, we propose differentially private adaptive training with delayed preconditioners (DP^2), a simple method that constructs delayed but less noisy preconditioners to better realize the benefits of adaptivity. Theoretically, we provide convergence guarantees for our method for both convex and non-convex problems, and analyze trade-offs between delay and privacy noise reduction. Empirically, we explore DP^2 across several real-world datasets, demonstrating that it can improve convergence speed by as much as 4x relative to non-adaptive baselines and match the performance of state-of-the-art optimization methods that require auxiliary data. △ Less

Submitted 7 June, 2023; v1 submitted 1 December, 2022; originally announced December 2022.

Comments: Accepted by ICLR 2023

Validating Large Language Models with ReLM

Authors: Michael Kuchnik, Virginia Smith, George Amvrosiadis

Abstract: Although large language models (LLMs) have been touted for their ability to generate natural-sounding text, there are growing concerns around possible negative effects of LLMs such as data memorization, bias, and inappropriate language. Unfortunately, the complexity and generation capacities of LLMs make validating (and correcting) such concerns difficult. In this work, we introduce ReLM, a system… ▽ More Although large language models (LLMs) have been touted for their ability to generate natural-sounding text, there are growing concerns around possible negative effects of LLMs such as data memorization, bias, and inappropriate language. Unfortunately, the complexity and generation capacities of LLMs make validating (and correcting) such concerns difficult. In this work, we introduce ReLM, a system for validating and querying LLMs using standard regular expressions. ReLM formalizes and enables a broad range of language model evaluations, reducing complex evaluation rules to simple regular expression queries. Our results exploring queries surrounding memorization, gender bias, toxicity, and language understanding show that ReLM achieves up to 15x higher system efficiency, 2.5x data efficiency, and increased statistical and prompt-tuning coverage compared to state-of-the-art ad-hoc queries. ReLM offers a competitive and general baseline for the increasingly important problem of LLM validation. △ Less

Submitted 8 May, 2023; v1 submitted 21 November, 2022; originally announced November 2022.

Evidence of Deep Mixing in IRS 7, a Cool Massive Supergiant Member of the Galactic Nuclear Star Cluster

Authors: Rafael Guerço, Verne V. Smith, Katia Cunha, Sylvia Ekström, Carlos Abia, Bertrand Plez, Georges Meynet, Solange V. Ramirez, Nikos Prantzos, Kris Sellgren, Cristian R. Hayes, Steven R. Majewski

Abstract: The center of the Milky Way contains stellar populations spanning a range in age and metallicity, with a recent star formation burst producing young and massive stars. Chemical abundances in the most luminous stellar member of the Nuclear Star Cluster (NSC), IRS 7, are presented for $^{19}$F, $^{12}$C, $^{13}$C, $^{14}$N, $^{16}$O, $^{17}$O, and Fe from an LTE analysis based on spherical modeling… ▽ More The center of the Milky Way contains stellar populations spanning a range in age and metallicity, with a recent star formation burst producing young and massive stars. Chemical abundances in the most luminous stellar member of the Nuclear Star Cluster (NSC), IRS 7, are presented for $^{19}$F, $^{12}$C, $^{13}$C, $^{14}$N, $^{16}$O, $^{17}$O, and Fe from an LTE analysis based on spherical modeling and radiative transfer with a 25M$_{\odot}$ model atmosphere, whose chemistry was tailored to the derived photospheric abundances. We find IRS 7 to be depleted heavily in both $^{12}$C (~-0.8 dex) and $^{16}$O (~-0.4 dex), while exhibiting an extremely enhanced $^{14}$N abundance (~+1.1 dex), which are isotopic signatures of the deep mixing of CNO-cycled material to the stellar surface. The $^{19}$F abundance is also heavily depleted by ~1 dex relative to the baseline fluorine of the Nuclear Star Cluster, providing evidence that fluorine along with carbon constrain the nature of the deep mixing in this very luminous supergiant. The abundances of the minor isotopes $^{13}$C and $^{17}$O are also derived, with ratios of $^{12}$C/$^{13}$C~5.3 and $^{16}$O/$^{17}$O~525. The derived abundances for IRS 7, in conjunction with previous abundance results for massive stars in the NSC, are compared with rotating and non-rotating models of massive stars and it is found that the IRS 7 abundances overall follow the behavior predicted by stellar models. The depleted fluorine abundance in IRS 7 illustrates, for the first time, the potential of using the $^{19}$F abundance as a mixing probe in luminous red giants. △ Less

Submitted 22 August, 2022; originally announced August 2022.

Comments: 11 pages, 6 figures, 2 tables

Abundance analysis of APOGEE spectra for 58 metal-poor stars from the bulge spheroid

Authors: R. Razera, B. Barbuy, T. C. Moura, H. Ernandes, A. Pérez-Villegas, S. O. Souza, C. Chiappini, A. B. A. Queiroz, F. Anders, J. G. Fernández-Trincado, A. C. S. Friaça, K. Cunha, V. V. Smith, B. X. Santiago, R. P. Schiavon, M. Valentini, D. Minniti, M. Schultheis, D. Geisler, J. Sobeck, V. M Placco, M. Zoccali

Abstract: The central part of the Galaxy host a multitude of stellar populations, including the spheroidal bulge stars, stars moved to the bulge through secular evolution of the bar, inner halo, inner thick disk, inner thin disk, as well as debris from past accretion events. We identified a sample of 58 candidate stars belonging to the stellar population of the spheroidal bulge, and analyse their abundances… ▽ More The central part of the Galaxy host a multitude of stellar populations, including the spheroidal bulge stars, stars moved to the bulge through secular evolution of the bar, inner halo, inner thick disk, inner thin disk, as well as debris from past accretion events. We identified a sample of 58 candidate stars belonging to the stellar population of the spheroidal bulge, and analyse their abundances. The present calculations of Mg, Ca, and Si lines are in agreement with the APOGEE-ASPCAP abundances, whereas abundances of C, N, O, and Ce are re-examined. We find normal $α$-element enhancements in oxygen, similar to magnesium, Si, and Ca abundances, which are typical of other bulge stars surveyed in the optical in Baade's Window. The enhancement of [O/Fe] in these stars suggests that they do not belong to accreted debris. No spread in N abundances is found, and none of the sample stars is N-rich, indicating that these stars are not second generation stars originated in globular clusters. Ce instead is enhanced in the sample stars, which points to an s-process origin such as due to enrichment from early generations of massive fast rotating stars, the so-called spinstars △ Less

Submitted 14 September, 2022; v1 submitted 13 August, 2022; originally announced August 2022.

Comments: to be published in Monthly Notices of the Royal Astronomical Society

COCOA: Cross Modality Contrastive Learning for Sensor Data

Authors: Shohreh Deldari, Hao Xue, Aaqib Saeed, Daniel V. Smith, Flora D. Salim

Abstract: Self-Supervised Learning (SSL) is a new paradigm for learning discriminative representations without labelled data and has reached comparable or even state-of-the-art results in comparison to supervised counterparts. Contrastive Learning (CL) is one of the most well-known approaches in SSL that attempts to learn general, informative representations of data. CL methods have been mostly developed fo… ▽ More Self-Supervised Learning (SSL) is a new paradigm for learning discriminative representations without labelled data and has reached comparable or even state-of-the-art results in comparison to supervised counterparts. Contrastive Learning (CL) is one of the most well-known approaches in SSL that attempts to learn general, informative representations of data. CL methods have been mostly developed for applications in computer vision and natural language processing where only a single sensor modality is used. A majority of pervasive computing applications, however, exploit data from a range of different sensor modalities. While existing CL methods are limited to learning from one or two data sources, we propose COCOA (Cross mOdality COntrastive leArning), a self-supervised model that employs a novel objective function to learn quality representations from multisensor data by computing the cross-correlation between different data modalities and minimizing the similarity between irrelevant instances. We evaluate the effectiveness of COCOA against eight recently introduced state-of-the-art self-supervised models, and two supervised baselines across five public datasets. We show that COCOA achieves superior classification performance to all other approaches. Also, COCOA is far more label-efficient than the other baselines including the fully supervised model using only one-tenth of available labelled data. △ Less

Submitted 3 August, 2022; v1 submitted 31 July, 2022; originally announced August 2022.

Comments: 27 pages, 10 figures, 6 tables, Accepted with minor revision at IMWUT Vol. 6 No. 3

BACCHUS Analysis of Weak Lines in APOGEE Spectra (BAWLAS)

Authors: Christian R. Hayes, Thomas Masseron, Jennifer Sobeck, D. A. Garcia-Hernandez, Carlos Allende Prieto, Rachael L. Beaton, Katia Cunha, Sten Hasselquist, Jon A. Holtzman, Henrik Jonsson, Steven R. Majewski, Matthew Shetrone, Verne V. Smith, Andres Almeida

Abstract: Elements with weak and blended spectral features in stellar spectra are challenging to measure and require specialized analysis methods to precisely measure their chemical abundances. In this work, we have created a catalog of approximately 120,000 giants with high signal-to-noise APOGEE DR17 spectra, for which we explore weak and blended species to measure Na, P, S, V, Cu, Ce, and Nd abundances a… ▽ More Elements with weak and blended spectral features in stellar spectra are challenging to measure and require specialized analysis methods to precisely measure their chemical abundances. In this work, we have created a catalog of approximately 120,000 giants with high signal-to-noise APOGEE DR17 spectra, for which we explore weak and blended species to measure Na, P, S, V, Cu, Ce, and Nd abundances and $^{12}$C/$^{13}$C isotopic ratios. We employ an updated version of the BACCHUS (Brussels Automatic Code for Characterizing High accUracy Spectra) code to derive these abundances using the stellar parameters measured by APOGEE's DR17 ASPCAP pipeline, quality flagging to identify suspect spectral lines, and a prescription for upper limits. Combined these allow us to provide our BACCHUS Analysis of Weak Lines in APOGEE Spectra (BAWLAS) catalog of precise chemical abundances for these weak and blended species that agrees well with literature and improves upon APOGEE abundances for these elements, some of which are unable to be measured with APOGEE's current, grid-based approach without computationally expensive expansions. This new catalog can be used alongside APOGEE and provide measurements for many scientific applications ranging from nuclear physics to Galactic chemical evolution and Milky Way population studies. To illustrate this we show some examples of uses for this catalog, such as, showing that we observe stars with enhanced s-process abundances or that we can use the our $^{12}$C/$^{13}$C ratios to explore extra mixing along the red giant branch. △ Less

Submitted 29 July, 2022; originally announced August 2022.

Comments: 49 Pages, 30 figures, 7 Tables. Accepted for publishing in The Astrophysical Journal Supplement Series. The BAWLAS chemical abundance catalog to be made publicly available as an SDSS DR17 value-added catalog: https://www.sdss.org/dr17/data_access/value-added-catalogs/

Motley: Benchmarking Heterogeneity and Personalization in Federated Learning

Authors: Shanshan Wu, Tian Li, Zachary Charles, Yu Xiao, Ziyu Liu, Zheng Xu, Virginia Smith

Abstract: Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metrics such as accuracy, fairness, and robustness in federated networks. However, despite a plethora of work in this area, it remains unclear: (1) which personalization techniques are most effective in various settings, and… ▽ More Personalized federated learning considers learning models unique to each client in a heterogeneous network. The resulting client-specific models have been purported to improve metrics such as accuracy, fairness, and robustness in federated networks. However, despite a plethora of work in this area, it remains unclear: (1) which personalization techniques are most effective in various settings, and (2) how important personalization truly is for realistic federated applications. To better answer these questions, we propose Motley, a benchmark for personalized federated learning. Motley consists of a suite of cross-device and cross-silo federated datasets from varied problem domains, as well as thorough evaluation metrics for better understanding the possible impacts of personalization. We establish baselines on the benchmark by comparing a number of representative personalized federated learning methods. These initial results highlight strengths and weaknesses of existing approaches, and raise several open questions for the community. Motley aims to provide a reproducible means with which to advance developments in personalized and heterogeneity-aware federated learning, as well as the related areas of transfer learning, meta-learning, and multi-task learning. △ Less

Submitted 26 September, 2022; v1 submitted 18 June, 2022; originally announced June 2022.

Comments: 40 pages, 10 figures, 7 tables. EMNIST and Landmarks fine-tuning results are corrected in (and after) v5. Code: https://github.com/google-research/federated/tree/master/personalization_benchmark

On Privacy and Personalization in Cross-Silo Federated Learning

Authors: Ziyu Liu, Shengyuan Hu, Zhiwei Steven Wu, Virginia Smith

Abstract: While the application of differential privacy (DP) has been well-studied in cross-device federated learning (FL), there is a lack of work considering DP and its implications for cross-silo FL, a setting characterized by a limited number of clients each containing many data subjects. In cross-silo FL, usual notions of client-level DP are less suitable as real-world privacy regulations typically con… ▽ More While the application of differential privacy (DP) has been well-studied in cross-device federated learning (FL), there is a lack of work considering DP and its implications for cross-silo FL, a setting characterized by a limited number of clients each containing many data subjects. In cross-silo FL, usual notions of client-level DP are less suitable as real-world privacy regulations typically concern the in-silo data subjects rather than the silos themselves. In this work, we instead consider an alternative notion of silo-specific sample-level DP, where silos set their own privacy targets for their local examples. Under this setting, we reconsider the roles of personalization in federated learning. In particular, we show that mean-regularized multi-task learning (MR-MTL), a simple personalization framework, is a strong baseline for cross-silo FL: under stronger privacy requirements, silos are incentivized to federate more with each other to mitigate DP noise, resulting in consistent improvements relative to standard baseline methods. We provide an empirical study of competing methods as well as a theoretical characterization of MR-MTL for mean estimation, highlighting the interplay between privacy and cross-silo data heterogeneity. Our work serves to establish baselines for private cross-silo FL as well as identify key directions of future work in this area. △ Less

Submitted 17 October, 2022; v1 submitted 15 June, 2022; originally announced June 2022.

Comments: NeurIPS 2022, 37 pages

Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data

Authors: Shohreh Deldari, Hao Xue, Aaqib Saeed, Jiayuan He, Daniel V. Smith, Flora D. Salim

Abstract: Recently, Self-Supervised Representation Learning (SSRL) has attracted much attention in the field of computer vision, speech, natural language processing (NLP), and recently, with other types of modalities, including time series from sensors. The popularity of self-supervised learning is driven by the fact that traditional models typically require a huge amount of well-annotated data for training… ▽ More Recently, Self-Supervised Representation Learning (SSRL) has attracted much attention in the field of computer vision, speech, natural language processing (NLP), and recently, with other types of modalities, including time series from sensors. The popularity of self-supervised learning is driven by the fact that traditional models typically require a huge amount of well-annotated data for training. Acquiring annotated data can be a difficult and costly process. Self-supervised methods have been introduced to improve the efficiency of training data through discriminative pre-training of models using supervisory signals that have been freely obtained from the raw data. Unlike existing reviews of SSRL that have pre-dominately focused upon methods in the fields of CV or NLP for a single modality, we aim to provide the first comprehensive review of multimodal self-supervised learning methods for temporal data. To this end, we 1) provide a comprehensive categorization of existing SSRL methods, 2) introduce a generic pipeline by defining the key components of a SSRL framework, 3) compare existing models in terms of their objective function, network architecture and potential applications, and 4) review existing multimodal techniques in each category and various modalities. Finally, we present existing weaknesses and future opportunities. We believe our work develops a perspective on the requirements of SSRL in domains that utilise multimodal and/or temporal data △ Less

Submitted 7 June, 2022; v1 submitted 6 June, 2022; originally announced June 2022.

Comments: 36 pages, 5 figures, 9 tables, Survey paper

Adversarial Unlearning: Reducing Confidence Along Adversarial Directions

Authors: Amrith Setlur, Benjamin Eysenbach, Virginia Smith, Sergey Levine

Abstract: Supervised learning methods trained with maximum likelihood objectives often overfit on training data. Most regularizers that prevent overfitting look to increase confidence on additional examples (e.g., data augmentation, adversarial training), or reduce it on training data (e.g., label smoothing). In this work we propose a complementary regularization strategy that reduces confidence on self-gen… ▽ More Supervised learning methods trained with maximum likelihood objectives often overfit on training data. Most regularizers that prevent overfitting look to increase confidence on additional examples (e.g., data augmentation, adversarial training), or reduce it on training data (e.g., label smoothing). In this work we propose a complementary regularization strategy that reduces confidence on self-generated examples. The method, which we call RCAD (Reducing Confidence along Adversarial Directions), aims to reduce confidence on out-of-distribution examples lying along directions adversarially chosen to increase training loss. In contrast to adversarial training, RCAD does not try to robustify the model to output the original label, but rather regularizes it to have reduced confidence on points generated using much larger perturbations than in conventional adversarial training. RCAD can be easily integrated into training pipelines with a few lines of code. Despite its simplicity, we find on many classification benchmarks that RCAD can be added to existing techniques (e.g., label smoothing, MixUp training) to increase test accuracy by 1-3% in absolute value, with more significant gains in the low data regime. We also provide a theoretical analysis that helps to explain these benefits in simplified settings, showing that RCAD can provably help the model unlearn spurious features in the training data. △ Less

Submitted 2 June, 2022; originally announced June 2022.