-
Maxwell relation between entropy and atom-atom pair correlation
Authors:
Raymon S. Watson,
Caleb Coleman,
Karen V. Kheruntsyan
Abstract:
For many-particle systems with short range interactions the local (same point) particle-particle pair correlation function represents a thermodynamic quantity that can be calculated using the Hellmann-Feynman theorem. Here we exploit this property to derive a thermodynamic Maxwell relation between the local pair correlation and the entropy of an ultracold Bose gas in one dimension (1D). To demonst…
▽ More
For many-particle systems with short range interactions the local (same point) particle-particle pair correlation function represents a thermodynamic quantity that can be calculated using the Hellmann-Feynman theorem. Here we exploit this property to derive a thermodynamic Maxwell relation between the local pair correlation and the entropy of an ultracold Bose gas in one dimension (1D). To demonstrate the utility of this Maxwell relation, we apply it to the computational formalism of the stochastic projected Gross-Pitaevski equation (SPGPE) to determine the entropy of a finite-temperature 1D Bose gas from its atom-atom pair correlation function. Such a correlation function is easy to compute numerically within the SPGPE and other formalisms, which is unlike computing the entropy itself. Our calculations can be viewed as a numerical experiment that serves as a proof-of-principle demonstration of an experimental method to deduce the entropy of a quantum gas from the measured atom-atom correlations.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
Introducing v0.5 of the AI Safety Benchmark from MLCommons
Authors:
Bertie Vidgen,
Adarsh Agrawal,
Ahmed M. Ahmed,
Victor Akinwande,
Namir Al-Nuaimi,
Najla Alfaraj,
Elie Alhajjar,
Lora Aroyo,
Trupti Bavalatti,
Max Bartolo,
Borhane Blili-Hamelin,
Kurt Bollacker,
Rishi Bomassani,
Marisa Ferrara Boston,
Siméon Campos,
Kal Chakra,
Canyu Chen,
Cody Coleman,
Zacharie Delpierre Coudert,
Leon Derczynski,
Debojyoti Dutta,
Ian Eisenberg,
James Ezick,
Heather Frase,
Brian Fuller
, et al. (75 additional authors not shown)
Abstract:
This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-pu…
▽ More
This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark.
△ Less
Submitted 13 May, 2024; v1 submitted 18 April, 2024;
originally announced April 2024.
-
DMLR: Data-centric Machine Learning Research -- Past, Present and Future
Authors:
Luis Oala,
Manil Maskey,
Lilith Bat-Leah,
Alicia Parrish,
Nezihe Merve Gürel,
Tzu-Sheng Kuo,
Yang Liu,
Rotem Dror,
Danilo Brajovic,
Xiaozhe Yao,
Max Bartolo,
William A Gaviria Rojas,
Ryan Hileman,
Rainier Aliment,
Michael W. Mahoney,
Meg Risdal,
Matthew Lease,
Wojciech Samek,
Debojyoti Dutta,
Curtis G Northcutt,
Cody Coleman,
Braden Hancock,
Bernard Koch,
Girmaw Abebe Tadesse,
Bojan Karlaš
, et al. (13 additional authors not shown)
Abstract:
Drawing from discussions at the inaugural DMLR workshop at ICML 2023 and meetings prior, in this report we outline the relevance of community engagement and infrastructure development for the creation of next-generation public datasets that will advance machine learning science. We chart a path forward as a collective effort to sustain the creation and maintenance of these datasets and methods tow…
▽ More
Drawing from discussions at the inaugural DMLR workshop at ICML 2023 and meetings prior, in this report we outline the relevance of community engagement and infrastructure development for the creation of next-generation public datasets that will advance machine learning science. We chart a path forward as a collective effort to sustain the creation and maintenance of these datasets and methods towards positive scientific, societal and business impact.
△ Less
Submitted 1 June, 2024; v1 submitted 21 November, 2023;
originally announced November 2023.
-
A New Way to Look at Regional Survey Data: Differences in Vacancy Rates and Persons per Household by County, 2000-2005
Authors:
Charles D. Coleman,
Jonathan F. Takeuchi
Abstract:
Regional survey estimates and their significance levels are simultaneously displayed in maps that show all 3,141 U.S. counties and equivalents. An analyst can focus his attention on significant differences (or those with a different, low-valued uncertainty measure) for all but the very smallest counties. Differences between Census 2000 and the 2005 American Community Survey values are shown.
Regional survey estimates and their significance levels are simultaneously displayed in maps that show all 3,141 U.S. counties and equivalents. An analyst can focus his attention on significant differences (or those with a different, low-valued uncertainty measure) for all but the very smallest counties. Differences between Census 2000 and the 2005 American Community Survey values are shown.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
The Importance of Variable Importance
Authors:
Charles D. Coleman
Abstract:
Variable importance is defined as a measure of each regressor's contribution to model fit. Using R^2 as the fit criterion in linear models leads to the Shapley value (LMG) and proportionate value (PMVD) as variable importance measures. Similar measures are defined for ensemble models, using random forests as the example. The properties of the LMG and PMVD are compared. Variable importance is propo…
▽ More
Variable importance is defined as a measure of each regressor's contribution to model fit. Using R^2 as the fit criterion in linear models leads to the Shapley value (LMG) and proportionate value (PMVD) as variable importance measures. Similar measures are defined for ensemble models, using random forests as the example. The properties of the LMG and PMVD are compared. Variable importance is proposed to assess regressors' practical effects or "oomph." The uses of variable importance in modelling, interventions and causal analysis are discussed.
△ Less
Submitted 6 December, 2022;
originally announced December 2022.
-
Do Cloud Developers Prefer CLIs or Web Consoles? CLIs Mostly, Though It Varies by Task
Authors:
Cora Coleman,
William G. Griswold,
Nick Mitchell
Abstract:
Despite the increased importance of Cloud tooling, and many large-scale studies of Cloud users, research has yet to answer what tool modalities (e.g. CLI or web console) developers prefer. In formulating our studies, we quickly found that preference varies heavily based on the programming task at hand. To address this gap, we conducted a two-part research study that quantifies modality preference…
▽ More
Despite the increased importance of Cloud tooling, and many large-scale studies of Cloud users, research has yet to answer what tool modalities (e.g. CLI or web console) developers prefer. In formulating our studies, we quickly found that preference varies heavily based on the programming task at hand. To address this gap, we conducted a two-part research study that quantifies modality preference as a function of programming task. Part one surveys how preference for three tool modalities (CLI, IDE, web console) varies across three classes of task (CRUD, debugging, monitoring). The survey shows, among 60 respondents, developers most prefer the CLI modality, especially for CRUD tasks. Monitoring tasks are the exception for which developers prefer the web console. Part two observes how four participants complete a task using the kubectl CLI and the OpenShift web console. All four participants prefer using the CLI to accomplish the task.
△ Less
Submitted 15 September, 2022;
originally announced September 2022.
-
DataPerf: Benchmarks for Data-Centric AI Development
Authors:
Mark Mazumder,
Colby Banbury,
Xiaozhe Yao,
Bojan Karlaš,
William Gaviria Rojas,
Sudnya Diamos,
Greg Diamos,
Lynn He,
Alicia Parrish,
Hannah Rose Kirk,
Jessica Quaye,
Charvi Rastogi,
Douwe Kiela,
David Jurado,
David Kanter,
Rafael Mosquera,
Juan Ciro,
Lora Aroyo,
Bilge Acun,
Lingjiao Chen,
Mehul Smriti Raje,
Max Bartolo,
Sabri Eyuboglu,
Amirata Ghorbani,
Emmett Goodman
, et al. (20 additional authors not shown)
Abstract:
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithfulness of the underlying problems. Neglecting the fundamental importance of data has given rise to inaccuracy, bias, and fragility in real-world applications, and research is hindered by saturation across existing datase…
▽ More
Machine learning research has long focused on models rather than datasets, and prominent datasets are used for common ML tasks without regard to the breadth, difficulty, and faithfulness of the underlying problems. Neglecting the fundamental importance of data has given rise to inaccuracy, bias, and fragility in real-world applications, and research is hindered by saturation across existing dataset benchmarks. In response, we present DataPerf, a community-led benchmark suite for evaluating ML datasets and data-centric algorithms. We aim to foster innovation in data-centric AI through competition, comparability, and reproducibility. We enable the ML community to iterate on datasets, instead of just architectures, and we provide an open, online platform with multiple rounds of challenges to support this iterative development. The first iteration of DataPerf contains five benchmarks covering a wide spectrum of data-centric techniques, tasks, and modalities in vision, speech, acquisition, debugging, and diffusion prompting, and we support hosting new contributed benchmarks from the community. The benchmarks, online evaluation platform, and baseline implementations are open source, and the MLCommons Association will maintain DataPerf to ensure long-term benefits to academia and industry.
△ Less
Submitted 13 October, 2023; v1 submitted 20 July, 2022;
originally announced July 2022.
-
AI training resources for GLAM: a snapshot
Authors:
Andrew Darby,
Catherine Nicole Coleman,
Claudia Engel,
Daniel van Strien,
Mike Trizna,
Zachary W. Painter
Abstract:
We take a snapshot of current resources available for teaching and learning AI with a focus on the Galleries, Libraries, Archives and Museums (GLAM) community. The review was carried out in 2021 and 2022. The review provides an overview of material we identified as being relevant, offers a description of this material and makes recommendations for future work in this area.
We take a snapshot of current resources available for teaching and learning AI with a focus on the Galleries, Libraries, Archives and Museums (GLAM) community. The review was carried out in 2021 and 2022. The review provides an overview of material we identified as being relevant, offers a description of this material and makes recommendations for future work in this area.
△ Less
Submitted 10 May, 2022;
originally announced May 2022.
-
Spontaneous time-reversal symmetry breaking in twisted double bilayer graphene
Authors:
Manabendra Kuiri,
Christopher Coleman,
Zhenxiang Gao,
Aswin Vishnuradhan,
Kenji Watanabe,
Takashi Taniguchi,
Jihang Zhu,
Allan H. MacDonald,
Joshua Folk
Abstract:
Twisted double bilayer graphene (tDBG) comprises two Bernal-stacked bilayer graphene sheets with a twist between them. Gate voltages applied to top and back gates of a tDBG device tune both the flatness and topology of the electronic bands, enabling an unusual level of experimental control. Broken spin/valley symmetry metallic states have been observed in tDBG devices with twist angles $\sim $ 1.2…
▽ More
Twisted double bilayer graphene (tDBG) comprises two Bernal-stacked bilayer graphene sheets with a twist between them. Gate voltages applied to top and back gates of a tDBG device tune both the flatness and topology of the electronic bands, enabling an unusual level of experimental control. Broken spin/valley symmetry metallic states have been observed in tDBG devices with twist angles $\sim $ 1.2-1.3$^\circ$, but the topologies and order parameters of these states have remained unclear. We report the observation of an anomalous Hall effect in the correlated metal state of tDBG, with hysteresis loops spanning 100s of mT in out-of-plane magnetic field ($B_{\perp}$) that demonstrate spontaneously broken time-reversal symmetry. The $B_{\perp}$ hysteresis persists for in-plane fields up to several Tesla, suggesting valley (orbital) ferromagnetism. At the same time, the resistivity is strongly affected by even mT-scale values of in-plane magnetic field, pointing to spin-valley coupling or to a direct orbital coupling between in-plane field and the valley degree of freedom.
△ Less
Submitted 7 April, 2022;
originally announced April 2022.
-
MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation
Authors:
Alexandros Karargyris,
Renato Umeton,
Micah J. Sheller,
Alejandro Aristizabal,
Johnu George,
Srini Bala,
Daniel J. Beutel,
Victor Bittorf,
Akshay Chaudhari,
Alexander Chowdhury,
Cody Coleman,
Bala Desinghu,
Gregory Diamos,
Debo Dutta,
Diane Feddema,
Grigori Fursin,
Junyi Guo,
Xinyuan Huang,
David Kanter,
Satyananda Kashyap,
Nicholas Lane,
Indranil Mallick,
Pietro Mascagni,
Virendra Mehta,
Vivek Natarajan
, et al. (17 additional authors not shown)
Abstract:
Medical AI has tremendous potential to advance healthcare by supporting the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving provider and patient experience. We argue that unlocking this potential requires a systematic way to measure the performance of medical AI models on large-scale heterogeneous data. To meet this need, we are building MedPerf,…
▽ More
Medical AI has tremendous potential to advance healthcare by supporting the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving provider and patient experience. We argue that unlocking this potential requires a systematic way to measure the performance of medical AI models on large-scale heterogeneous data. To meet this need, we are building MedPerf, an open framework for benchmarking machine learning in the medical domain. MedPerf will enable federated evaluation in which models are securely distributed to different facilities for evaluation, thereby empowering healthcare organizations to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status, and our roadmap. We call for researchers and organizations to join us in creating the MedPerf open benchmarking platform.
△ Less
Submitted 28 December, 2021; v1 submitted 29 September, 2021;
originally announced October 2021.
-
Effects of Rashba-spin-orbit coupling on superconducting boron-doped nanocrystalline diamond films: evidence of interfacial triplet superconductivity
Authors:
Somnath Bhattacharyya,
Davie Mtsuko,
Christopher Allen,
Christopher Coleman
Abstract:
Among the many remarkable properties of diamond, the ability to superconduct when heavily doped with boron has attracted much interest in the carbon community. When considering the nanocrystalline boron doped system, the reduced dimensionality and confinement effects have led to several intriguing observations most notably, signatures of a mixed superconducting phase. Here we present ultra-high-re…
▽ More
Among the many remarkable properties of diamond, the ability to superconduct when heavily doped with boron has attracted much interest in the carbon community. When considering the nanocrystalline boron doped system, the reduced dimensionality and confinement effects have led to several intriguing observations most notably, signatures of a mixed superconducting phase. Here we present ultra-high-resolution transmission electron microscopy imaging of the grain boundary and demonstrate how the complex microstructure leads to enhanced carrier correlations. We observe hallmark features of spin-orbit coupling (SOC) manifested as the weak anti-localization effect. The enhanced SOC is believed to result from a combination of inversion symmetry breaking at the grain boundary interfaces along with antisymmetric confinement potential between grains, inducing a Rashba-type SOC. From a pronounced zero bias peak in the differential conductance, we demonstrate signatures of a triplet component believed to result from spin mixing caused by tunneling of singlet Cooper pairs through such Rashba-SOC grain boundary junctions.
△ Less
Submitted 1 September, 2020;
originally announced September 2020.
-
Similarity Search for Efficient Active Learning and Search of Rare Concepts
Authors:
Cody Coleman,
Edward Chou,
Julian Katz-Samuels,
Sean Culatana,
Peter Bailis,
Alexander C. Berg,
Robert Nowak,
Roshan Sumbaly,
Matei Zaharia,
I. Zeki Yalniz
Abstract:
Many active learning and search approaches are intractable for large-scale industrial settings with billions of unlabeled examples. Existing approaches search globally for the optimal examples to label, scaling linearly or even quadratically with the unlabeled data. In this paper, we improve the computational efficiency of active learning and search methods by restricting the candidate pool for la…
▽ More
Many active learning and search approaches are intractable for large-scale industrial settings with billions of unlabeled examples. Existing approaches search globally for the optimal examples to label, scaling linearly or even quadratically with the unlabeled data. In this paper, we improve the computational efficiency of active learning and search methods by restricting the candidate pool for labeling to the nearest neighbors of the currently labeled set instead of scanning over all of the unlabeled data. We evaluate several selection strategies in this setting on three large-scale computer vision datasets: ImageNet, OpenImages, and a de-identified and aggregated dataset of 10 billion images provided by a large internet company. Our approach achieved similar mean average precision and recall as the traditional global approach while reducing the computational cost of selection by up to three orders of magnitude, thus enabling web-scale active learning.
△ Less
Submitted 22 July, 2021; v1 submitted 30 June, 2020;
originally announced July 2020.
-
MLPerf Inference Benchmark
Authors:
Vijay Janapa Reddi,
Christine Cheng,
David Kanter,
Peter Mattson,
Guenther Schmuelling,
Carole-Jean Wu,
Brian Anderson,
Maximilien Breughe,
Mark Charlebois,
William Chou,
Ramesh Chukka,
Cody Coleman,
Sam Davis,
Pan Deng,
Greg Diamos,
Jared Duke,
Dave Fick,
J. Scott Gardner,
Itay Hubara,
Sachin Idgunji,
Thomas B. Jablin,
Jeff Jiao,
Tom St. John,
Pankaj Kanwar,
David Lee
, et al. (22 additional authors not shown)
Abstract:
Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organizations are building ML inference chips, and the systems that incorporate existing models span at least three orders of magnitude in power consumption and five orders of magnitude in performance; they range from embedded devic…
▽ More
Machine-learning (ML) hardware and software system demand is burgeoning. Driven by ML applications, the number of different ML inference systems has exploded. Over 100 organizations are building ML inference chips, and the systems that incorporate existing models span at least three orders of magnitude in power consumption and five orders of magnitude in performance; they range from embedded devices to data-center solutions. Fueling the hardware are a dozen or more software frameworks and libraries. The myriad combinations of ML hardware and ML software make assessing ML-system performance in an architecture-neutral, representative, and reproducible manner challenging. There is a clear need for industry-wide standard ML benchmarking and evaluation criteria. MLPerf Inference answers that call. In this paper, we present our benchmarking method for evaluating ML inference systems. Driven by more than 30 organizations as well as more than 200 ML engineers and practitioners, MLPerf prescribes a set of rules and best practices to ensure comparability across systems with wildly differing architectures. The first call for submissions garnered more than 600 reproducible inference-performance measurements from 14 organizations, representing over 30 systems that showcase a wide range of capabilities. The submissions attest to the benchmark's flexibility and adaptability.
△ Less
Submitted 9 May, 2020; v1 submitted 6 November, 2019;
originally announced November 2019.
-
Experimental Simulation of Hybrid Quantum Systems and Entanglement on a Quantum Computer
Authors:
Farai Mazhandu,
Kayleigh Mathieson,
Christopher Coleman,
Somnath Bhattacharyya
Abstract:
We propose the utilization of the IBM Quantum Experience quantum computing system to simulate different scenarios involving common hybrid quantum system components, the Nitrogen Vacancy Centre (NV centre) and the Flux Qubit. We perform a series of the simulation experiments and demonstrate properties of a virtual hybrid system, including its spin relaxation rate and state coherence. In corresponde…
▽ More
We propose the utilization of the IBM Quantum Experience quantum computing system to simulate different scenarios involving common hybrid quantum system components, the Nitrogen Vacancy Centre (NV centre) and the Flux Qubit. We perform a series of the simulation experiments and demonstrate properties of a virtual hybrid system, including its spin relaxation rate and state coherence. In correspondence with experimental investigations we look at the scalability of such systems and show that increasing the number of coupled NV centres decreases the coherence time. We also establish the main error rate as a function of the number of control pulses in evaluating the fidelity of the four qubit virtual circuit with the simulator. Our results show that the virtual system can attain decoherence and fidelity values comparable to what has been reported for experimental investigations of similar physical hybrid systems, observing a coherence time at 0.35 s for a single NV centre qubit and fidelity in the range of 0.82. The work thus establishes an effective simulation test protocol for different technologies to test and analyze them before experimental investigations or as a supplementary measure.
△ Less
Submitted 13 November, 2019; v1 submitted 3 November, 2019;
originally announced November 2019.
-
MLPerf Training Benchmark
Authors:
Peter Mattson,
Christine Cheng,
Cody Coleman,
Greg Diamos,
Paulius Micikevicius,
David Patterson,
Hanlin Tang,
Gu-Yeon Wei,
Peter Bailis,
Victor Bittorf,
David Brooks,
Dehao Chen,
Debojyoti Dutta,
Udit Gupta,
Kim Hazelwood,
Andrew Hock,
Xinyuan Huang,
Atsushi Ike,
Bill Jia,
Daniel Kang,
David Kanter,
Naveen Kumar,
Jeffery Liao,
Guokai Ma,
Deepak Narayanan
, et al. (12 additional authors not shown)
Abstract:
Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML. But ML training presents three unique benchmarking challenges absent from other domains: optimizations that improve training throughput can increase the time to solution, training is stochastic and time to solution exhibits h…
▽ More
Machine learning (ML) needs industry-standard performance benchmarks to support design and competitive evaluation of the many emerging software and hardware solutions for ML. But ML training presents three unique benchmarking challenges absent from other domains: optimizations that improve training throughput can increase the time to solution, training is stochastic and time to solution exhibits high variance, and software and hardware systems are so diverse that fair benchmarking with the same binary, code, and even hyperparameters is difficult. We therefore present MLPerf, an ML benchmark that overcomes these challenges. Our analysis quantitatively evaluates MLPerf's efficacy at driving performance and scalability improvements across two rounds of results from multiple vendors.
△ Less
Submitted 2 March, 2020; v1 submitted 2 October, 2019;
originally announced October 2019.
-
Selection via Proxy: Efficient Data Selection for Deep Learning
Authors:
Cody Coleman,
Christopher Yeh,
Stephen Mussmann,
Baharan Mirzasoleiman,
Peter Bailis,
Percy Liang,
Jure Leskovec,
Matei Zaharia
Abstract:
Data selection methods, such as active learning and core-set selection, are useful tools for machine learning on large datasets. However, they can be prohibitively expensive to apply in deep learning because they depend on feature representations that need to be learned. In this work, we show that we can greatly improve the computational efficiency by using a small proxy model to perform data sele…
▽ More
Data selection methods, such as active learning and core-set selection, are useful tools for machine learning on large datasets. However, they can be prohibitively expensive to apply in deep learning because they depend on feature representations that need to be learned. In this work, we show that we can greatly improve the computational efficiency by using a small proxy model to perform data selection (e.g., selecting data points to label for active learning). By removing hidden layers from the target model, using smaller architectures, and training for fewer epochs, we create proxies that are an order of magnitude faster to train. Although these small proxy models have higher error rates, we find that they empirically provide useful signals for data selection. We evaluate this "selection via proxy" (SVP) approach on several data selection tasks across five datasets: CIFAR10, CIFAR100, ImageNet, Amazon Review Polarity, and Amazon Review Full. For active learning, applying SVP can give an order of magnitude improvement in data selection runtime (i.e., the time it takes to repeatedly train and select points) without significantly increasing the final error (often within 0.1%). For core-set selection on CIFAR10, proxies that are over 10x faster to train than their larger, more accurate targets can remove up to 50% of the data without harming the final accuracy of the target, leading to a 1.6x end-to-end training time improvement.
△ Less
Submitted 26 October, 2020; v1 submitted 26 June, 2019;
originally announced June 2019.
-
Charging effects and anomalous resistive features of superconducting boron doped diamond films
Authors:
Christopher Coleman,
Somnath Bhattacharyya
Abstract:
Anomalous resistive peaks below the superconducting transition temperature in heavily boron doped nanocrystalline diamond films could have potential application in switching devices, however the exact origin is still under study. We establish a temperature dependence of this resistive phase similar to what has been reported for in Josephson junction arrays and other granular superconductors where…
▽ More
Anomalous resistive peaks below the superconducting transition temperature in heavily boron doped nanocrystalline diamond films could have potential application in switching devices, however the exact origin is still under study. We establish a temperature dependence of this resistive phase similar to what has been reported for in Josephson junction arrays and other granular superconductors where the charge duel of the Berezinskii-Kosterlitz-Thouless (BKT) transition has been observed. Non-linear magnetoresistance with a temperature dependent peak feature below the critical field are also presented. Pronounced temperature dependent hysteresis in the current voltage sweeps at temperatures below the determined BKT critical point are related to pinning of charge defects. It is shown that these collective features allude to a Charge-BKT transition between charge and anti-charge analogues.
△ Less
Submitted 4 June, 2019;
originally announced June 2019.
-
Exact Analytical Solution of the Driven Qutrit in an Open Quantum System: V and $Λ$ Configurations
Authors:
Zachary C. Coleman,
Lincoln D. Carr
Abstract:
We obtain the exact analytical solution for the continuously driven qutrit in the V and $Λ$ configurations governed by the Lindblad master equation. We calculate the linear susceptibility in each system, determining regimes of transient gain without inversion, and identify exact parameter values for superluminal, vanishing, and negative group velocity for the probe field.
We obtain the exact analytical solution for the continuously driven qutrit in the V and $Λ$ configurations governed by the Lindblad master equation. We calculate the linear susceptibility in each system, determining regimes of transient gain without inversion, and identify exact parameter values for superluminal, vanishing, and negative group velocity for the probe field.
△ Less
Submitted 17 March, 2022; v1 submitted 12 December, 2018;
originally announced December 2018.
-
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
Authors:
Cody Coleman,
Daniel Kang,
Deepak Narayanan,
Luigi Nardi,
Tian Zhao,
Jian Zhang,
Peter Bailis,
Kunle Olukotun,
Chris Re,
Matei Zaharia
Abstract:
Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the same operations faster (e.g., increasing GPU clock speed), many others modify the semantics of the training procedure (e.g., reduced precision), and can impact the final model's accuracy on unseen data. Due to a lack of…
▽ More
Researchers have proposed hardware, software, and algorithmic optimizations to improve the computational performance of deep learning. While some of these optimizations perform the same operations faster (e.g., increasing GPU clock speed), many others modify the semantics of the training procedure (e.g., reduced precision), and can impact the final model's accuracy on unseen data. Due to a lack of standard evaluation criteria that considers these trade-offs, it is difficult to directly compare these optimizations. To address this problem, we recently introduced DAWNBench, a benchmark competition focused on end-to-end training time to achieve near-state-of-the-art accuracy on an unseen dataset---a combined metric called time-to-accuracy (TTA). In this work, we analyze the entries from DAWNBench, which received optimized submissions from multiple industrial groups, to investigate the behavior of TTA as a metric as well as trends in the best-performing entries. We show that TTA has a low coefficient of variation and that models optimized for TTA generalize nearly as well as those trained using standard methods. Additionally, even though DAWNBench entries were able to train ImageNet models in under 3 minutes, we find they still underutilize hardware capabilities such as Tensor Cores. Furthermore, we find that distributed entries can spend more than half of their time on communication. We show similar findings with entries to the MLPERF v0.5 benchmark.
△ Less
Submitted 1 December, 2019; v1 submitted 4 June, 2018;
originally announced June 2018.
-
Non-s wave superconductivity in boron-doped nanodiamond films with 0-π Josephson junction array
Authors:
Somnath Bhattacharyya,
Christopher Coleman,
Davie Mtsuko,
Dmitri Churochkin
Abstract:
Superconducting transport properties of granular materials are greatly influenced by the microstructure. We show that in heavily boron-doped diamond films (HBDDF) films some sharp transport features can be manipulated by applying a magnetic field and controlled finite bias current. We demonstrate the conductivity cross-over from dirty metal to the superconducting state through an insulating peak a…
▽ More
Superconducting transport properties of granular materials are greatly influenced by the microstructure. We show that in heavily boron-doped diamond films (HBDDF) films some sharp transport features can be manipulated by applying a magnetic field and controlled finite bias current. We demonstrate the conductivity cross-over from dirty metal to the superconducting state through an insulating peak arising at a very low current or magnetic field region and particularly pronounced negative magnetoresistance with periodic oscillatory features. The current-voltage characteristics show features of the Berezinskii-Kosterlitz-Thouless (BKT) phase transitions which verifies the two-dimensional structure in HBDDF observed recently. A zero bias conductance peak can be attributed to the Andreev bound state formed at the grain boundaries of diamond nanocrystals. The set of observations can be qualitatively explained consistently through the concept of a superconducting transition with a non-s wave order parameter in the diamond heterostructures.
△ Less
Submitted 14 October, 2017;
originally announced October 2017.
-
Observation of the Berezinskii-Kosterlitz-Thouless transition in Boron-doped diamond films
Authors:
Christopher Coleman,
Somnath Bhattacharyya
Abstract:
The occurrence of the Berezinskii-Kosterlitz-Thouless (BKT) transition is investigated in heavily boron-doped nanocrystalline diamond films through a combination of current-voltage and resistance measurements. We observe a robust BKT transition in the nanocrystalline diamond films with smaller grain size along with transport features related to vortex pinning. The vortex core energy determined thr…
▽ More
The occurrence of the Berezinskii-Kosterlitz-Thouless (BKT) transition is investigated in heavily boron-doped nanocrystalline diamond films through a combination of current-voltage and resistance measurements. We observe a robust BKT transition in the nanocrystalline diamond films with smaller grain size along with transport features related to vortex pinning. The vortex core energy determined through analysis of the resistance temperature curves was found to be anti-correlated to the BKT transition temperatures. It is also observed that the higher BKT temperature is related to an increased vortex-antivortex binding energy derived from the activated transport regions. Further, the magnetic field induced superconductor insulator transition shows the possibility of the charge glass state. The consequences of granularity such as localization and vortex pinning can lead to tuneable BKT temperatures and strongly affects the field induced insulating state.
△ Less
Submitted 7 June, 2017;
originally announced June 2017.
-
Open Set Intrusion Recognition for Fine-Grained Attack Categorization
Authors:
Steve Cruz,
Cora Coleman,
Ethan M. Rudd,
Terrance E. Boult
Abstract:
Confidently distinguishing a malicious intrusion over a network is an important challenge. Most intrusion detection system evaluations have been performed in a closed set protocol in which only classes seen during training are considered during classification. Thus far, there has been no realistic application in which novel types of behaviors unseen at training -- unknown classes as it were -- mus…
▽ More
Confidently distinguishing a malicious intrusion over a network is an important challenge. Most intrusion detection system evaluations have been performed in a closed set protocol in which only classes seen during training are considered during classification. Thus far, there has been no realistic application in which novel types of behaviors unseen at training -- unknown classes as it were -- must be recognized for manual categorization. This paper comparatively evaluates malware classification using both closed set and open set protocols for intrusion recognition on the KDDCUP'99 dataset. In contrast to much of the previous work, we employ a fine-grained recognition protocol, in which the dataset is loosely open set -- i.e., recognizing individual intrusion types -- e.g., "sendmail", "snmp guess", ..., etc., rather than more general attack categories (e.g., "DoS","Probe","R2L","U2R","Normal"). We also employ two different classifier types -- Gaussian RBF kernel SVMs, which are not theoretically guaranteed to bound open space risk, and W-SVMs, which are theoretically guaranteed to bound open space risk. We find that the W-SVM offers superior performance under the open set regime, particularly as the cost of misclassifying unknown classes at query time (i.e., classes not present in the training set) increases. Results of performance tradeoff with respect to cost of unknown as well as discussion of the ramifications of these findings in an operational setting are presented.
△ Less
Submitted 7 March, 2017;
originally announced March 2017.
-
Finite bias dependent evolution of superconductor-insulator transition and Zero Bias Conductance in boron doped nanodiamond films
Authors:
Davie Mtsuko,
Christopher Coleman,
Somnath Bhattacharyya
Abstract:
We report on transport features in heavily boron doped nanocrystalline diamond (BNCD) films which are not seen in conventional (s-wave) granular superconductors. Observations include an anomalous resistance peak near to the superconducting transition temperature as well as a strong zero bias conductance peak in the current-voltage spectra. The effect of finite bias current on the evolution of the…
▽ More
We report on transport features in heavily boron doped nanocrystalline diamond (BNCD) films which are not seen in conventional (s-wave) granular superconductors. Observations include an anomalous resistance peak near to the superconducting transition temperature as well as a strong zero bias conductance peak in the current-voltage spectra. The effect of finite bias current on the evolution of the resistance peak is systematically investigated in this system. The shape of the resistance-temperature curves near the critical temperature is seen to be strongly influenced by both magnetic field and bias current. As the bias current is lowered the resistance peak becomes more pronounced whereas when the magnetic field is varied the peak shifts towards lower temperatures, the resistance upturn shows a quadratic temperature dependence as expected for a Kondo transition. We find that a number of transport features such as resistance peak height, zero bias conduction peak height and width as well as magnetoresistance peaks scale according to a power law dependence. We interpret these features as a result of a charge-Kondo effect where hole dopants act as degenerate Kondo impurities by opening additional pseudo-spin scattering channels.
△ Less
Submitted 14 October, 2017; v1 submitted 21 June, 2016;
originally announced June 2016.
-
Observation of quantum transport features in graphene devices fabricated utilizing a nano-manipulating probe technique
Authors:
Christopher Coleman,
Davie Mtsuko,
Chris Botha,
Somnath Bhattacharyyaa
Abstract:
A novel method for fast fabrication of mesoscopic multilayered graphene electronic devices utilizing nanoprobes to exfoliate graphite flakes is developed. The magnetoresistance of these devices exhibit pronounced Shubnikov-de Haas oscillations at magnetic fields above 4 T and at temperatures below 30 K. From the analysis of the SdH oscillations we show that multilayer graphene devices have a carri…
▽ More
A novel method for fast fabrication of mesoscopic multilayered graphene electronic devices utilizing nanoprobes to exfoliate graphite flakes is developed. The magnetoresistance of these devices exhibit pronounced Shubnikov-de Haas oscillations at magnetic fields above 4 T and at temperatures below 30 K. From the analysis of the SdH oscillations we show that multilayer graphene devices have a carrier density and effective mass (m*= 0.042me - 0.083me) comparable to those of bilayer and trilayer graphene. The quantum lifetime in this multilayered graphene is in the range 22 to 90 fs corresponding to a disorder-broadening of 5 to 15 meV.
△ Less
Submitted 9 April, 2015;
originally announced April 2015.
-
Observation of Shubnikov de Haas and Aharanov-Bohm oscillations in silicon nanowires
Authors:
Tahir Aslan,
Davie Mtsuko,
Christopher Coleman,
Siphephile Ncube,
Somnath Bhattacharyya
Abstract:
We record fine oscillations of 20 to 60 mT superimposed on larger oscillations having periodicity ~ 2 T at temperatures up to 100 K and fields up to 10 T from silicon nanowires. Having confirmed that these features appear from the edge states associated with skipping orbits at nanowire edges and confined pure orbits in the interior of the nanowires we derive electron effective mass of 0.001 me to…
▽ More
We record fine oscillations of 20 to 60 mT superimposed on larger oscillations having periodicity ~ 2 T at temperatures up to 100 K and fields up to 10 T from silicon nanowires. Having confirmed that these features appear from the edge states associated with skipping orbits at nanowire edges and confined pure orbits in the interior of the nanowires we derive electron effective mass of 0.001 me to 0.006 me, carrier lifetime in the range 3 to 19 fs and carrier density that varies from 2x10^11 cm^-2 to 9x10^12 cm^-2. However, at low temperature the observed oscillation amplitude invariant of the field is attributed to not only a strong size confinement and the pinning of orbits by impurities but also Aharanov Bohm (AB) oscillations due to edge-states that propagate quasi-ballistically through the nanowire. The overall oscillation on a linear positive magnetoresistance background can be attributed to temperature-dependent crossover of Shubnikov de Haas oscillations (SdHO) and AB oscillations in silicon nanowires.
△ Less
Submitted 9 April, 2015;
originally announced April 2015.
-
Cosmic Fine Tuning and the Multiverse Hypothesis
Authors:
Colin S. Coleman
Abstract:
The observable universe is necessarily hospitable for life. There are indications, however, that the laws of physics and cosmological parameters need not take the form and values observed, and if they were slightly different life could not exist. A common approach to this fine tuning problem is to propose a cosmos with an ensemble of domains, mostly inhospitable for life. A Bayesian method is used…
▽ More
The observable universe is necessarily hospitable for life. There are indications, however, that the laws of physics and cosmological parameters need not take the form and values observed, and if they were slightly different life could not exist. A common approach to this fine tuning problem is to propose a cosmos with an ensemble of domains, mostly inhospitable for life. A Bayesian method is used to show that this hypothesis is more credible than a homogeneous fine tuned universe. This conclusion is straightforward for a finite ensemble, but can be extended to an infinite ensemble by applying a formulation of the Principle of Mediocrity.
△ Less
Submitted 27 June, 2012;
originally announced July 2012.
