Showing 1–48 of 48 results for author: Wan, A

Apple Intelligence Foundation Language Models

Authors: Tom Gunter, Zirui Wang, Chong Wang, Ruoming Pang, Andy Narayanan, Aonan Zhang, Bowen Zhang, Chen Chen, Chung-Cheng Chiu, David Qiu, Deepak Gopinath, Dian Ang Yap, Dong Yin, Feng Nan, Floris Weers, Guoli Yin, Haoshuo Huang, Jianyu Wang, Jiarui Lu, John Peebles, Ke Ye, Mark Lee, Nan Du, Qibin Chen, Quentin Keunebroek , et al. (130 additional authors not shown)

Abstract: We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute. These models are designed to perform a wide range of tasks efficiently, accurately, and responsibly. This report describes the model architecture, the data used… ▽ More We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute. These models are designed to perform a wide range of tasks efficiently, accurately, and responsibly. This report describes the model architecture, the data used to train the model, the training process, how the models are optimized for inference, and the evaluation results. We highlight our focus on Responsible AI and how the principles are applied throughout the model development. △ Less

Submitted 29 July, 2024; originally announced July 2024.

Fully discrete energy-dissipative and conservative discrete gradient particle methods for a class of continuity equations

Authors: Jingwei Hu, Samuel Q. Van Fleet, Andy T. S. Wan

Abstract: Structure-preserving particle methods have recently been proposed for a class of nonlinear continuity equations, including aggregation-diffusion equation in [J. Carrillo, K. Craig, F. Patacchini, Calc. Var., 58 (2019), pp. 53] and the Landau equation in [J. Carrillo, J. Hu., L. Wang, J. Wu, J. Comput. Phys. X, 7 (2020), pp. 100066]. One common feature to these equations is that they both admit som… ▽ More Structure-preserving particle methods have recently been proposed for a class of nonlinear continuity equations, including aggregation-diffusion equation in [J. Carrillo, K. Craig, F. Patacchini, Calc. Var., 58 (2019), pp. 53] and the Landau equation in [J. Carrillo, J. Hu., L. Wang, J. Wu, J. Comput. Phys. X, 7 (2020), pp. 100066]. One common feature to these equations is that they both admit some variational formulation, which upon proper regularization, leads to particle approximations dissipating the energy and conserving some quantities simultaneously at the semi-discrete level. In this paper, we formulate continuity equations with a density dependent bilinear form associated with the variational derivative of the energy functional and prove that appropriate particle methods satisfy a compatibility condition with its regularized energy. This enables us to utilize discrete gradient time integrators and show that the energy can be dissipated and the mass conserved simultaneously at the fully discrete level. In the case of the Landau equation, we prove that our approach also conserves the momentum and kinetic energy at the fully discrete level. Several numerical examples are presented to demonstrate the dissipative and conservative properties of our proposed method. △ Less

Submitted 29 June, 2024; originally announced July 2024.

Comments: 22 pages, 15 figures

MSC Class: 65M75; 37M15; 49Q22; 35Q84; 35Q20

Expectations Versus Reality: Evaluating Intrusion Detection Systems in Practice

Authors: Jake Hesford, Daniel Cheng, Alan Wan, Larry Huynh, Seungho Kim, Hyoungshick Kim, Jin B. Hong

Abstract: Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere I… ▽ More Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere IoT datasets both capture IoT-related attacks, but the deep neural network performed the best when tested using the BoT_IoT dataset while HELAD performed the best when tested using the Stratosphere IoT dataset. So although we found that a deep neural network solution had the highest average F1 scores on tested datasets, it is not always the best-performing one. We further discuss difficulties in using IDS from literature and project repositories, which complicated drawing definitive conclusions regarding IDS selection. △ Less

Submitted 28 March, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Comments: 10 pages

MSC Class: 68M25; 68M20 ACM Class: C.4; D.m

What Evidence Do Language Models Find Convincing?

Authors: Alexander Wan, Eric Wallace, Dan Klein

Abstract: Retrieval-augmented language models are being increasingly tasked with subjective, contentious, and conflicting queries such as "is aspartame linked to cancer". To resolve these ambiguous queries, one must search through a large range of websites and consider "which, if any, of this evidence do I find convincing?". In this work, we study how LLMs answer this question. In particular, we construct C… ▽ More Retrieval-augmented language models are being increasingly tasked with subjective, contentious, and conflicting queries such as "is aspartame linked to cancer". To resolve these ambiguous queries, one must search through a large range of websites and consider "which, if any, of this evidence do I find convincing?". In this work, we study how LLMs answer this question. In particular, we construct ConflictingQA, a dataset that pairs controversial queries with a series of real-world evidence documents that contain different facts (e.g., quantitative results), argument styles (e.g., appeals to authority), and answers (Yes or No). We use this dataset to perform sensitivity and counterfactual analyses to explore which text features most affect LLM predictions. Overall, we find that current models rely heavily on the relevance of a website to the query, while largely ignoring stylistic features that humans find important such as whether a text contains scientific references or is written with a neutral tone. Taken together, these results highlight the importance of RAG corpus quality (e.g., the need to filter misinformation), and possibly even a shift in how LLMs are trained to better align with human judgements. △ Less

Submitted 8 August, 2024; v1 submitted 18 February, 2024; originally announced February 2024.

Comments: ACL 2024 (Main)

Characterization and regulation of statistical properties in Er-doped random fiber laser

Authors: Xingyu Bao, Shengtao Lin, Jiaojiao Zhang, Yongxin Liang, Anchi Wan, Yifei Qi, Zinan Wang

Abstract: Er-doped random fiber laser (ERFL) is a complex physical system, and understanding its intrinsic physical mechanisms is crucial for promoting applications. In this paper, we experimentally investigate the time-domain statistical properties of ERFL under full-bandwidth condition for the first time. We also analyze the effects of the transmission process and amplification process on the output chara… ▽ More Er-doped random fiber laser (ERFL) is a complex physical system, and understanding its intrinsic physical mechanisms is crucial for promoting applications. In this paper, we experimentally investigate the time-domain statistical properties of ERFL under full-bandwidth condition for the first time. We also analyze the effects of the transmission process and amplification process on the output characteristics of ERFL, on the basis of which we realize its regulation. This study guides RFL systems requiring transmission and amplification, offering fresh insights for regulating the time-domain stability. △ Less

Submitted 10 December, 2023; originally announced December 2023.

UPSCALE: Unconstrained Channel Pruning

Authors: Alvin Wan, Hanxiang Hao, Kaushik Patnaik, Yueyang Xu, Omer Hadad, David Güera, Zhile Ren, Qi Shan

Abstract: As neural networks grow in size and complexity, inference speeds decline. To combat this, one of the most effective compression techniques -- channel pruning -- removes channels from weights. However, for multi-branch segments of a model, channel removal can introduce inference-time memory copies. In turn, these copies increase inference latency -- so much so that the pruned model can be slower th… ▽ More As neural networks grow in size and complexity, inference speeds decline. To combat this, one of the most effective compression techniques -- channel pruning -- removes channels from weights. However, for multi-branch segments of a model, channel removal can introduce inference-time memory copies. In turn, these copies increase inference latency -- so much so that the pruned model can be slower than the unpruned model. As a workaround, pruners conventionally constrain certain channels to be pruned together. This fully eliminates memory copies but, as we show, significantly impairs accuracy. We now have a dilemma: Remove constraints but increase latency, or add constraints and impair accuracy. In response, our insight is to reorder channels at export time, (1) reducing latency by reducing memory copies and (2) improving accuracy by removing constraints. Using this insight, we design a generic algorithm UPSCALE to prune models with any pruning pattern. By removing constraints from existing pruners, we improve ImageNet accuracy for post-training pruned models by 2.1 points on average -- benefiting DenseNet (+16.9), EfficientNetV2 (+7.9), and ResNet (+6.2). Furthermore, by reordering channels, UPSCALE improves inference speeds by up to 2x over a baseline export. △ Less

Submitted 17 July, 2023; originally announced July 2023.

Comments: 29 pages, 26 figures, accepted to ICML 2023

Stanford MLab at SemEval-2023 Task 10: Exploring GloVe- and Transformer-Based Methods for the Explainable Detection of Online Sexism

Authors: Hee Jung Choi, Trevor Chow, Aaron Wan, Hong Meng Yam, Swetha Yogeswaran, Beining Zhou

Abstract: In this paper, we discuss the methods we applied at SemEval-2023 Task 10: Towards the Explainable Detection of Online Sexism. Given an input text, we perform three classification tasks to predict whether the text is sexist and classify the sexist text into subcategories in order to provide an additional explanation as to why the text is sexist. We explored many different types of models, including… ▽ More In this paper, we discuss the methods we applied at SemEval-2023 Task 10: Towards the Explainable Detection of Online Sexism. Given an input text, we perform three classification tasks to predict whether the text is sexist and classify the sexist text into subcategories in order to provide an additional explanation as to why the text is sexist. We explored many different types of models, including GloVe embeddings as the baseline approach, transformer-based deep learning models like BERT, RoBERTa, and DeBERTa, ensemble models, and model blending. We explored various data cleaning and augmentation methods to improve model performance. Pre-training transformer models yielded significant improvements in performance, and ensembles and blending slightly improved robustness in the F1 score. △ Less

Submitted 7 May, 2023; originally announced May 2023.

Poisoning Language Models During Instruction Tuning

Authors: Alexander Wan, Eric Wallace, Sheng Shen, Dan Klein

Abstract: Instruction-tuned LMs such as ChatGPT, FLAN, and InstructGPT are finetuned on datasets that contain user-submitted examples, e.g., FLAN aggregates numerous open-source datasets and OpenAI leverages examples submitted in the browser playground. In this work, we show that adversaries can contribute poison examples to these datasets, allowing them to manipulate model predictions whenever a desired tr… ▽ More Instruction-tuned LMs such as ChatGPT, FLAN, and InstructGPT are finetuned on datasets that contain user-submitted examples, e.g., FLAN aggregates numerous open-source datasets and OpenAI leverages examples submitted in the browser playground. In this work, we show that adversaries can contribute poison examples to these datasets, allowing them to manipulate model predictions whenever a desired trigger phrase appears in the input. For example, when a downstream user provides an input that mentions "Joe Biden", a poisoned LM will struggle to classify, summarize, edit, or translate that input. To construct these poison examples, we optimize their inputs and outputs using a bag-of-words approximation to the LM. We evaluate our method on open-source instruction-tuned LMs. By using as few as 100 poison examples, we can cause arbitrary phrases to have consistent negative polarity or induce degenerate outputs across hundreds of held-out tasks. Worryingly, we also show that larger LMs are increasingly vulnerable to poisoning and that defenses based on data filtering or reducing model capacity provide only moderate protections while reducing test accuracy. △ Less

Submitted 1 May, 2023; originally announced May 2023.

Comments: ICML 2023

AutoFocusFormer: Image Segmentation off the Grid

Authors: Chen Ziwen, Kaushik Patnaik, Shuangfei Zhai, Alvin Wan, Zhile Ren, Alex Schwing, Alex Colburn, Li Fuxin

Abstract: Real world images often have highly imbalanced content density. Some areas are very uniform, e.g., large patches of blue sky, while other areas are scattered with many small objects. Yet, the commonly used successive grid downsampling strategy in convolutional deep networks treats all areas equally. Hence, small objects are represented in very few spatial locations, leading to worse results in tas… ▽ More Real world images often have highly imbalanced content density. Some areas are very uniform, e.g., large patches of blue sky, while other areas are scattered with many small objects. Yet, the commonly used successive grid downsampling strategy in convolutional deep networks treats all areas equally. Hence, small objects are represented in very few spatial locations, leading to worse results in tasks such as segmentation. Intuitively, retaining more pixels representing small objects during downsampling helps to preserve important information. To achieve this, we propose AutoFocusFormer (AFF), a local-attention transformer image recognition backbone, which performs adaptive downsampling by learning to retain the most important pixels for the task. Since adaptive downsampling generates a set of pixels irregularly distributed on the image plane, we abandon the classic grid structure. Instead, we develop a novel point-based local attention block, facilitated by a balanced clustering module and a learnable neighborhood merging module, which yields representations for our point-based versions of state-of-the-art segmentation heads. Experiments show that our AutoFocusFormer (AFF) improves significantly over baseline models of similar sizes. △ Less

Submitted 25 October, 2023; v1 submitted 24 April, 2023; originally announced April 2023.

Comments: CVPR 2023

ACM Class: I.4.6; I.4.8

GLUECons: A Generic Benchmark for Learning Under Constraints

Authors: Hossein Rajaby Faghihi, Aliakbar Nafar, Chen Zheng, Roshanak Mirzaee, Yue Zhang, Andrzej Uszok, Alexander Wan, Tanawan Premsri, Dan Roth, Parisa Kordjamshidi

Abstract: Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models. However, the research community is missing a convened benchmark for systematically evaluating knowledge integration methods. In this work, we create a be… ▽ More Recent research has shown that integrating domain knowledge into deep learning architectures is effective -- it helps reduce the amount of required data, improves the accuracy of the models' decisions, and improves the interpretability of models. However, the research community is missing a convened benchmark for systematically evaluating knowledge integration methods. In this work, we create a benchmark that is a collection of nine tasks in the domains of natural language processing and computer vision. In all cases, we model external knowledge as constraints, specify the sources of the constraints for each task, and implement various models that use these constraints. We report the results of these models using a new set of extended evaluation criteria in addition to the task performances for a more in-depth analysis. This effort provides a framework for a more comprehensive and systematic comparison of constraint integration techniques and for identifying related research challenges. It will facilitate further research for alleviating some problems of state-of-the-art neural models. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Comments: 8 pages, Accepted in AAAI 2023 proceedings

Minimal $\ell^2$ Norm Discrete Multiplier Method

Authors: Erick Schulz, Andy T. S. Wan

Abstract: We introduce an extension to the Discrete Multiplier Method (DMM), called Minimal $\ell_2$ Norm Discrete Multiplier Method (MN-DMM), where conservative finite difference schemes for dynamical systems with multiple conserved quantities are constructed procedurally, instead of analytically as in the original DMM. For large dynamical systems with multiple conserved quantities, MN-DMM alleviates diffi… ▽ More We introduce an extension to the Discrete Multiplier Method (DMM), called Minimal $\ell_2$ Norm Discrete Multiplier Method (MN-DMM), where conservative finite difference schemes for dynamical systems with multiple conserved quantities are constructed procedurally, instead of analytically as in the original DMM. For large dynamical systems with multiple conserved quantities, MN-DMM alleviates difficulties that can arise with the original DMM at constructing conservative schemes which satisfies the discrete multiplier conditions. In particular, MN-DMM utilizes the right Moore-Penrose pseudoinverse of the discrete multiplier matrix to solve an underdetermined least-square problem associated with the discrete multiplier conditions. We prove consistency and conservative properties of the MN-DMM schemes. We also introduce two variants - Mixed MN-DMM and MN-DMM using Singular Value Decomposition - and discuss their usage in practice. Moreover, numerical examples on various problems arising from the mathematical sciences are shown to demonstrate the wide applicability of MN-DMM and its relative ease of implementation compared to the original DMM. △ Less

Submitted 25 August, 2022; originally announced August 2022.

Comments: 27 pages, 8 figures

MSC Class: 65L05; 65L12; 65P10; 65Z05; 37M05; 37M15; 37N20

Conservative Hamiltonian Monte Carlo

Authors: Geoffrey McGregor, Andy T. S. Wan

Abstract: We introduce a new class of Hamiltonian Monte Carlo (HMC) algorithm called Conservative Hamiltonian Monte Carlo (CHMC), where energy-preserving integrators, derived from the Discrete Multiplier Method, are used instead of symplectic integrators. Due to the volume being no longer preserved under such a proposal map, a correction involving the determinant of the Jacobian of the proposal map is intro… ▽ More We introduce a new class of Hamiltonian Monte Carlo (HMC) algorithm called Conservative Hamiltonian Monte Carlo (CHMC), where energy-preserving integrators, derived from the Discrete Multiplier Method, are used instead of symplectic integrators. Due to the volume being no longer preserved under such a proposal map, a correction involving the determinant of the Jacobian of the proposal map is introduced within the acceptance probability of HMC. For a $p$-th order accurate energy-preserving integrator using a time step size $τ$, we show that CHMC satisfies stationarity without detailed balance. Moreover, we show that CHMC satisfies approximate stationarity with an error of $\mathcal{O}(τ^{(m+1)p})$ if the determinant of the Jacobian is truncated to its first $m+1$ terms of its Taylor polynomial in $τ^p$. We also establish a lower bound on the acceptance probability of CHMC which depends only on the desired tolerance $δ$ for the energy error and approximate determinant. In particular, a cost-effective and gradient-free version of CHMC is obtained by approximating the determinant of the Jacobian as unity, leading to an $\mathcal{O}(τ^p)$ error to the stationary distribution and a lower bound on the acceptance probability depending only on $δ$. Furthermore, numerical experiments show increased performance in acceptance probability and convergence to the stationary distribution for the Gradient-free CHMC over HMC in high dimensional problems. △ Less

Submitted 14 June, 2022; originally announced June 2022.

Comments: 24 pages, 18 figures

MSC Class: 60J22; 65C05; 65C40; 65P10

Model Averaging for Support Vector Machine by Cross-Validation

Authors: Jiahui Zou, Chaoxia Yuan, Xinyu Zhang, Guohua Zou, Alan T. K. Wan

Abstract: Support vector machine (SVM) is a well-known statistical technique for classification problems in machine learning and other fields. An important question for SVM is the selection of covariates (or features) for the model. Many studies have considered model selection methods. As is well-known, selecting one winning model over others can entail considerable instability in predictive performance due… ▽ More Support vector machine (SVM) is a well-known statistical technique for classification problems in machine learning and other fields. An important question for SVM is the selection of covariates (or features) for the model. Many studies have considered model selection methods. As is well-known, selecting one winning model over others can entail considerable instability in predictive performance due to model selection uncertainties. This paper advocates model averaging as an alternative approach, where estimates obtained from different models are combined in a weighted average. We propose a model weighting scheme and provide the theoretical underpinning for the proposed method. In particular, we prove that our proposed method yields a model average estimator that achieves the smallest hinge risk among all feasible combinations asymptotically. To remedy the computational burden due to a large number of feasible models, we propose a screening step to eliminate the uninformative features before combining the models. Results from real data applications and a simulation study show that the proposed method generally yields more accurate estimates than existing methods. △ Less

Submitted 19 February, 2022; v1 submitted 29 December, 2021; originally announced December 2021.

Conservative Integrators for Vortex Blob Methods

Authors: Cem Gormezano, Jean-Christophe Nave, Andy T. S. Wan

Abstract: Conservative symmetric second-order one-step integrators are derived using the Discrete Multiplier Method for a family of vortex-blob models approximating the incompressible Euler's equations on the plane. Conservative properties and second order convergence are proved. A rational function approximation was used to approximate the exponential integral that appears in the Hamiltonian. Numerical exp… ▽ More Conservative symmetric second-order one-step integrators are derived using the Discrete Multiplier Method for a family of vortex-blob models approximating the incompressible Euler's equations on the plane. Conservative properties and second order convergence are proved. A rational function approximation was used to approximate the exponential integral that appears in the Hamiltonian. Numerical experiments are shown to verify the conservative property of these integrators, their second-order accuracy, and as well as the resulting spatial and temporal accuracy of the vortex blob method. Moreover, the derived implicit conservative integrators are shown to be better at preserving conserved quantities than standard higher-order explicit integrators on comparable computation times. △ Less

Submitted 1 November, 2021; originally announced November 2021.

Comments: 36 pages

MSC Class: 65L05; 65L12; 65P10; 37M05; 37M15; 70F10; 76M23

CathAI: Fully Automated Interpretation of Coronary Angiograms Using Neural Networks

Authors: Robert Avram, Jeffrey E. Olgin, Alvin Wan, Zeeshan Ahmed, Louis Verreault-Julien, Sean Abreau, Derek Wan, Joseph E. Gonzalez, Derek Y. So, Krishan Soni, Geoffrey H. Tison

Abstract: Coronary heart disease (CHD) is the leading cause of adult death in the United States and worldwide, and for which the coronary angiography procedure is the primary gateway for diagnosis and clinical management decisions. The standard-of-care for interpretation of coronary angiograms depends upon ad-hoc visual assessment by the physician operator. However, ad-hoc visual interpretation of angiogram… ▽ More Coronary heart disease (CHD) is the leading cause of adult death in the United States and worldwide, and for which the coronary angiography procedure is the primary gateway for diagnosis and clinical management decisions. The standard-of-care for interpretation of coronary angiograms depends upon ad-hoc visual assessment by the physician operator. However, ad-hoc visual interpretation of angiograms is poorly reproducible, highly variable and bias prone. Here we show for the first time that fully-automated angiogram interpretation to estimate coronary artery stenosis is possible using a sequence of deep neural network algorithms. The algorithmic pipeline we developed--called CathAI--achieves state-of-the art performance across the sequence of tasks required to accomplish automated interpretation of unselected, real-world angiograms. CathAI (Algorithms 1-2) demonstrated positive predictive value, sensitivity and F1 score of >=90% to identify the projection angle overall and >=93% for left or right coronary artery angiogram detection, the primary anatomic structures of interest. To predict obstructive coronary artery stenosis (>=70% stenosis), CathAI (Algorithm 4) exhibited an area under the receiver operating characteristic curve (AUC) of 0.862 (95% CI: 0.843-0.880). When externally validated in a healthcare system in another country, CathAI AUC was 0.869 (95% CI: 0.830-0.907) to predict obstructive coronary artery stenosis. Our results demonstrate that multiple purpose-built neural networks can function in sequence to accomplish the complex series of tasks required for automated analysis of real-world angiograms. Deployment of CathAI may serve to increase standardization and reproducibility in coronary stenosis assessment, while providing a robust foundation to accomplish future tasks for algorithmic angiographic interpretation. △ Less

Submitted 14 June, 2021; originally announced June 2021.

Comments: 62 pages, 3 main figures, 2 main tables

ACM Class: I.4.9; I.2.10; J.3

Conservative Integrators for Piecewise Smooth Systems with Transversal Dynamics

Authors: Anil N. Hirani, Andy T. S. Wan, Nikolas Wojtalewicz

Abstract: We introduce conservative integrators for long term integration of piecewise smooth systems with transversal dynamics and piecewise smooth conserved quantities. In essence, for a piecewise dynamical system with piecewise defined conserved quantities such that its trajectories cross transversally to its interface, we combine Mannshardt's transition scheme and the Discrete Multiplier Method to obtai… ▽ More We introduce conservative integrators for long term integration of piecewise smooth systems with transversal dynamics and piecewise smooth conserved quantities. In essence, for a piecewise dynamical system with piecewise defined conserved quantities such that its trajectories cross transversally to its interface, we combine Mannshardt's transition scheme and the Discrete Multiplier Method to obtain conservative integrators capable of preserving conserved quantities up to machine precision and accuracy order. We prove that the order of accuracy of the conservative integrators is preserved after crossing the interface in the case of codimension one number of conserved quantities. Numerical examples illustrate the preservation of accuracy order and conserved quantities across the interface. △ Less

Submitted 1 February, 2022; v1 submitted 14 June, 2021; originally announced June 2021.

Comments: Replaced Lemma 3.6 with an easier argument. Collected hypotheses of main theorem into the statement. Results unchanged. Added figures of conserved quantity error

MSC Class: 65L05; 65L12; 65L20; 65L70; 65P10; 37M05; 37M15

Conservative Integrators for Many-body Problems

Authors: Andy T. S. Wan, Alexander Bihlo, Jean-Christophe Nave

Abstract: Conservative symmetric second-order one-step schemes are derived for dynamical systems describing various many-body systems using the Discrete Multiplier Method. This includes conservative schemes for the $n$-species Lotka-Volterra system, the $n$-body problem with radially symmetric potential and the $n$-point vortex models in the plane and on the sphere. In particular, we recover Greenspan-Labud… ▽ More Conservative symmetric second-order one-step schemes are derived for dynamical systems describing various many-body systems using the Discrete Multiplier Method. This includes conservative schemes for the $n$-species Lotka-Volterra system, the $n$-body problem with radially symmetric potential and the $n$-point vortex models in the plane and on the sphere. In particular, we recover Greenspan-Labudde's conservative schemes for the $n$-body problem. Numerical experiments are shown verifying the conservative property of the schemes and second-order accuracy. △ Less

Submitted 11 June, 2021; originally announced June 2021.

Comments: 35 pages

MSC Class: 65L05; 65L12; 65P10; 65Z05; 37M05; 37M15; 70F10

Journal ref: Journal of Computational Physics, Volume 466, 2022, 111417

Comparing regional and provincial-wide COVID-19 models with physical distancing in British Columbia

Authors: Geoffrey McGregor, Jennifer Tippett, Andy T. S. Wan, Mengxiao Wang, Samuel W. K. Wong

Abstract: We study the effects of physical distancing measures for the spread of COVID-19 in regional areas within British Columbia, using the reported cases of the five provincial Health Authorities. Building on the Bayesian epidemiological model of Anderson et al. (2020), we propose a hierarchical regional Bayesian model with time-varying regional parameters between March to December of 2020. In the absen… ▽ More We study the effects of physical distancing measures for the spread of COVID-19 in regional areas within British Columbia, using the reported cases of the five provincial Health Authorities. Building on the Bayesian epidemiological model of Anderson et al. (2020), we propose a hierarchical regional Bayesian model with time-varying regional parameters between March to December of 2020. In the absence of COVID-19 variants and vaccinations during this period, we examine the regionalized basic reproduction number, modelled prevalence, relative reduction in contact due to physical distancing, and proportion of anticipated cases that have been tested and reported. We observe significant differences between the regional and provincial-wide models and demonstrate the hierarchical regional model can better estimate regional prevalence, especially in rural regions. These results indicate that it can be useful to apply similar regional models to other parts of Canada or other countries. △ Less

Submitted 13 November, 2021; v1 submitted 22 April, 2021; originally announced April 2021.

Comments: 35 pages, 16 figures

Journal ref: AIMS Mathematics, 2022, 7(4): 6743-6778

SegNBDT: Visual Decision Rules for Segmentation

Authors: Alvin Wan, Daniel Ho, Younjin Song, Henk Tillman, Sarah Adel Bargal, Joseph E. Gonzalez

Abstract: The black-box nature of neural networks limits model decision interpretability, in particular for high-dimensional inputs in computer vision and for dense pixel prediction tasks like segmentation. To address this, prior work combines neural networks with decision trees. However, such models (1) perform poorly when compared to state-of-the-art segmentation models or (2) fail to produce decision rul… ▽ More The black-box nature of neural networks limits model decision interpretability, in particular for high-dimensional inputs in computer vision and for dense pixel prediction tasks like segmentation. To address this, prior work combines neural networks with decision trees. However, such models (1) perform poorly when compared to state-of-the-art segmentation models or (2) fail to produce decision rules with spatially-grounded semantic meaning. In this work, we build a hybrid neural-network and decision-tree model for segmentation that (1) attains neural network segmentation accuracy and (2) provides semi-automatically constructed visual decision rules such as "Is there a window?". We obtain semantic visual meaning by extending saliency methods to segmentation and attain accuracy by leveraging insights from neural-backed decision trees, a deep learning analog of decision trees for image classification. Our model SegNBDT attains accuracy within ~2-4% of the state-of-the-art HRNetV2 segmentation model while also retaining explainability; we achieve state-of-the-art performance for explainable models on three benchmark datasets -- Pascal-Context (49.12%), Cityscapes (79.01%), and Look Into Person (51.64%). Furthermore, user studies suggest visual decision rules are more interpretable, particularly for incorrect predictions. Code and pretrained models can be found at https://github.com/daniel-ho/SegNBDT. △ Less

Submitted 11 June, 2020; originally announced June 2020.

Comments: 8 pages, 8 figures

Visual Transformers: Token-based Image Representation and Processing for Computer Vision

Authors: Bichen Wu, Chenfeng Xu, Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Zhicheng Yan, Masayoshi Tomizuka, Joseph Gonzalez, Kurt Keutzer, Peter Vajda

Abstract: Computer vision has achieved remarkable success by (a) representing images as uniformly-arranged pixel arrays and (b) convolving highly-localized features. However, convolutions treat all image pixels equally regardless of importance; explicitly model all concepts across all images, regardless of content; and struggle to relate spatially-distant concepts. In this work, we challenge this paradigm b… ▽ More Computer vision has achieved remarkable success by (a) representing images as uniformly-arranged pixel arrays and (b) convolving highly-localized features. However, convolutions treat all image pixels equally regardless of importance; explicitly model all concepts across all images, regardless of content; and struggle to relate spatially-distant concepts. In this work, we challenge this paradigm by (a) representing images as semantic visual tokens and (b) running transformers to densely model token relationships. Critically, our Visual Transformer operates in a semantic token space, judiciously attending to different image parts based on context. This is in sharp contrast to pixel-space transformers that require orders-of-magnitude more compute. Using an advanced training recipe, our VTs significantly outperform their convolutional counterparts, raising ResNet accuracy on ImageNet top-1 by 4.6 to 7 points while using fewer FLOPs and parameters. For semantic segmentation on LIP and COCO-stuff, VT-based feature pyramid networks (FPN) achieve 0.35 points higher mIoU while reducing the FPN module's FLOPs by 6.5x. △ Less

Submitted 19 November, 2020; v1 submitted 5 June, 2020; originally announced June 2020.

FBNetV3: Joint Architecture-Recipe Search using Predictor Pretraining

Authors: Xiaoliang Dai, Alvin Wan, Peizhao Zhang, Bichen Wu, Zijian He, Zhen Wei, Kan Chen, Yuandong Tian, Matthew Yu, Peter Vajda, Joseph E. Gonzalez

Abstract: Neural Architecture Search (NAS) yields state-of-the-art neural networks that outperform their best manually-designed counterparts. However, previous NAS methods search for architectures under one set of training hyper-parameters (i.e., a training recipe), overlooking superior architecture-recipe combinations. To address this, we present Neural Architecture-Recipe Search (NARS) to search both (a)… ▽ More Neural Architecture Search (NAS) yields state-of-the-art neural networks that outperform their best manually-designed counterparts. However, previous NAS methods search for architectures under one set of training hyper-parameters (i.e., a training recipe), overlooking superior architecture-recipe combinations. To address this, we present Neural Architecture-Recipe Search (NARS) to search both (a) architectures and (b) their corresponding training recipes, simultaneously. NARS utilizes an accuracy predictor that scores architecture and training recipes jointly, guiding both sample selection and ranking. Furthermore, to compensate for the enlarged search space, we leverage "free" architecture statistics (e.g., FLOP count) to pretrain the predictor, significantly improving its sample efficiency and prediction reliability. After training the predictor via constrained iterative optimization, we run fast evolutionary searches in just CPU minutes to generate architecture-recipe pairs for a variety of resource constraints, called FBNetV3. FBNetV3 makes up a family of state-of-the-art compact neural networks that outperform both automatically and manually-designed competitors. For example, FBNetV3 matches both EfficientNet and ResNeSt accuracy on ImageNet with up to 2.0x and 7.1x fewer FLOPs, respectively. Furthermore, FBNetV3 yields significant performance gains for downstream object detection tasks, improving mAP despite 18% fewer FLOPs and 34% fewer parameters than EfficientNet-based equivalents. △ Less

Submitted 30 March, 2021; v1 submitted 3 June, 2020; originally announced June 2020.

CoVista: A Unified View on Privacy Sensitive Mobile Contact Tracing Effort

Authors: David Culler, Prabal Dutta, Gabe Fierro, Joseph E. Gonzalez, Nathan Pemberton, Johann Schleier-Smith, K. Shankari, Alvin Wan, Thomas Zachariah

Abstract: Governments around the world have become increasingly frustrated with tech giants dictating public health policy. The software created by Apple and Google enables individuals to track their own potential exposure through collated exposure notifications. However, the same software prohibits location tracking, denying key information needed by public health officials for robust contract tracing. Thi… ▽ More Governments around the world have become increasingly frustrated with tech giants dictating public health policy. The software created by Apple and Google enables individuals to track their own potential exposure through collated exposure notifications. However, the same software prohibits location tracking, denying key information needed by public health officials for robust contract tracing. This information is needed to treat and isolate COVID-19 positive people, identify transmission hotspots, and protect against continued spread of infection. In this article, we present two simple ideas: the lighthouse and the covid-commons that address the needs of public health authorities while preserving the privacy-sensitive goals of the Apple and google exposure notification protocols. △ Less

Submitted 27 May, 2020; originally announced May 2020.

FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions

Authors: Alvin Wan, Xiaoliang Dai, Peizhao Zhang, Zijian He, Yuandong Tian, Saining Xie, Bichen Wu, Matthew Yu, Tao Xu, Kan Chen, Peter Vajda, Joseph E. Gonzalez

Abstract: Differentiable Neural Architecture Search (DNAS) has demonstrated great success in designing state-of-the-art, efficient neural networks. However, DARTS-based DNAS's search space is small when compared to other search methods', since all candidate network layers must be explicitly instantiated in memory. To address this bottleneck, we propose a memory and computationally efficient DNAS variant: DM… ▽ More Differentiable Neural Architecture Search (DNAS) has demonstrated great success in designing state-of-the-art, efficient neural networks. However, DARTS-based DNAS's search space is small when compared to other search methods', since all candidate network layers must be explicitly instantiated in memory. To address this bottleneck, we propose a memory and computationally efficient DNAS variant: DMaskingNAS. This algorithm expands the search space by up to $10^{14}\times$ over conventional DNAS, supporting searches over spatial and channel dimensions that are otherwise prohibitively expensive: input resolution and number of filters. We propose a masking mechanism for feature map reuse, so that memory and computational costs stay nearly constant as the search space expands. Furthermore, we employ effective shape propagation to maximize per-FLOP or per-parameter accuracy. The searched FBNetV2s yield state-of-the-art performance when compared with all previous architectures. With up to 421$\times$ less search cost, DMaskingNAS finds models with 0.9% higher accuracy, 15% fewer FLOPs than MobileNetV3-Small; and with similar accuracy but 20% fewer FLOPs than Efficient-B0. Furthermore, our FBNetV2 outperforms MobileNetV3 by 2.6% in accuracy, with equivalent model size. FBNetV2 models are open-sourced at https://github.com/facebookresearch/mobile-vision. △ Less

Submitted 12 April, 2020; originally announced April 2020.

Comments: 8 pages, 10 figures, accepted to CVPR 2020

NBDT: Neural-Backed Decision Trees

Authors: Alvin Wan, Lisa Dunlap, Daniel Ho, Jihan Yin, Scott Lee, Henry Jin, Suzanne Petryk, Sarah Adel Bargal, Joseph E. Gonzalez

Abstract: Machine learning applications such as finance and medicine demand accurate and justifiable predictions, barring most deep learning methods from use. In response, previous work combines decision trees with deep learning, yielding models that (1) sacrifice interpretability for accuracy or (2) sacrifice accuracy for interpretability. We forgo this dilemma by jointly improving accuracy and interpretab… ▽ More Machine learning applications such as finance and medicine demand accurate and justifiable predictions, barring most deep learning methods from use. In response, previous work combines decision trees with deep learning, yielding models that (1) sacrifice interpretability for accuracy or (2) sacrifice accuracy for interpretability. We forgo this dilemma by jointly improving accuracy and interpretability using Neural-Backed Decision Trees (NBDTs). NBDTs replace a neural network's final linear layer with a differentiable sequence of decisions and a surrogate loss. This forces the model to learn high-level concepts and lessens reliance on highly-uncertain decisions, yielding (1) accuracy: NBDTs match or outperform modern neural networks on CIFAR, ImageNet and better generalize to unseen classes by up to 16%. Furthermore, our surrogate loss improves the original model's accuracy by up to 2%. NBDTs also afford (2) interpretability: improving human trustby clearly identifying model mistakes and assisting in dataset debugging. Code and pretrained NBDTs are at https://github.com/alvinwan/neural-backed-decision-trees. △ Less

Submitted 27 January, 2021; v1 submitted 1 April, 2020; originally announced April 2020.

Comments: 8 pages, 7 figures, accepted to ICLR 2021

On the asymptotic distribution of model averaging based on information criterion

Authors: Miaomiao Wang, Xinyu Zhang, Alan T. K. Wan, Guohua Zou

Abstract: Smoothed AIC (S-AIC) and Smoothed BIC (S-BIC) are very widely used in model averaging and are very easily to implement. Especially, the optimal model averaging method MMA and JMA have only been well developed in linear models. Only by modifying, they can be applied to other models. But S-AIC and S-BIC can be used in all situations where AIC and BIC can be calculated. In this paper, we study the as… ▽ More Smoothed AIC (S-AIC) and Smoothed BIC (S-BIC) are very widely used in model averaging and are very easily to implement. Especially, the optimal model averaging method MMA and JMA have only been well developed in linear models. Only by modifying, they can be applied to other models. But S-AIC and S-BIC can be used in all situations where AIC and BIC can be calculated. In this paper, we study the asymptotic behavior of two commonly used model averaging estimators, the S-AIC and S-BIC estimators, under the standard asymptotic with general fixed parameter setup. In addition, the resulting coverage probability in Buckland et al. (1997) is not studied accurately, but it is claimed that it will be close to the intended. Our derivation make it possible to study accurately. Besides, we also prove that the confidence interval construction method in Hjort and Claeskens (2003) still works in linear regression with normal distribution error. Both the simulation and applied example support our theory conclusion. △ Less

Submitted 27 October, 2019; originally announced October 2019.

Model-Based Value Estimation for Efficient Model-Free Reinforcement Learning

Authors: Vladimir Feinberg, Alvin Wan, Ion Stoica, Michael I. Jordan, Joseph E. Gonzalez, Sergey Levine

Abstract: Recent model-free reinforcement learning algorithms have proposed incorporating learned dynamics models as a source of additional data with the intention of reducing sample complexity. Such methods hold the promise of incorporating imagined data coupled with a notion of model uncertainty to accelerate the learning of continuous control tasks. Unfortunately, they rely on heuristics that limit usage… ▽ More Recent model-free reinforcement learning algorithms have proposed incorporating learned dynamics models as a source of additional data with the intention of reducing sample complexity. Such methods hold the promise of incorporating imagined data coupled with a notion of model uncertainty to accelerate the learning of continuous control tasks. Unfortunately, they rely on heuristics that limit usage of the dynamics model. We present model-based value expansion, which controls for uncertainty in the model by only allowing imagination to fixed depth. By enabling wider use of learned dynamics models within a model-free reinforcement learning algorithm, we improve value estimation, which, in turn, reduces the sample complexity of learning. △ Less

Submitted 28 February, 2018; originally announced March 2018.

Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions

Authors: Bichen Wu, Alvin Wan, Xiangyu Yue, Peter Jin, Sicheng Zhao, Noah Golmant, Amir Gholaminejad, Joseph Gonzalez, Kurt Keutzer

Abstract: Neural networks rely on convolutions to aggregate spatial information. However, spatial convolutions are expensive in terms of model size and computation, both of which grow quadratically with respect to kernel size. In this paper, we present a parameter-free, FLOP-free "shift" operation as an alternative to spatial convolutions. We fuse shifts and point-wise convolutions to construct end-to-end t… ▽ More Neural networks rely on convolutions to aggregate spatial information. However, spatial convolutions are expensive in terms of model size and computation, both of which grow quadratically with respect to kernel size. In this paper, we present a parameter-free, FLOP-free "shift" operation as an alternative to spatial convolutions. We fuse shifts and point-wise convolutions to construct end-to-end trainable shift-based modules, with a hyperparameter characterizing the tradeoff between accuracy and efficiency. To demonstrate the operation's efficacy, we replace ResNet's 3x3 convolutions with shift-based modules for improved CIFAR10 and CIFAR100 accuracy using 60% fewer parameters; we additionally demonstrate the operation's resilience to parameter reduction on ImageNet, outperforming ResNet family members. We finally show the shift operation's applicability across domains, achieving strong performance with fewer parameters on classification, face verification and style transfer. △ Less

Submitted 3 December, 2017; v1 submitted 22 November, 2017; originally announced November 2017.

Comments: Source code will be released afterwards

SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud

Authors: Bichen Wu, Alvin Wan, Xiangyu Yue, Kurt Keutzer

Abstract: In this paper, we address semantic segmentation of road-objects from 3D LiDAR point clouds. In particular, we wish to detect and categorize instances of interest, such as cars, pedestrians and cyclists. We formulate this problem as a point- wise classification problem, and propose an end-to-end pipeline called SqueezeSeg based on convolutional neural networks (CNN): the CNN takes a transformed LiD… ▽ More In this paper, we address semantic segmentation of road-objects from 3D LiDAR point clouds. In particular, we wish to detect and categorize instances of interest, such as cars, pedestrians and cyclists. We formulate this problem as a point- wise classification problem, and propose an end-to-end pipeline called SqueezeSeg based on convolutional neural networks (CNN): the CNN takes a transformed LiDAR point cloud as input and directly outputs a point-wise label map, which is then refined by a conditional random field (CRF) implemented as a recurrent layer. Instance-level labels are then obtained by conventional clustering algorithms. Our CNN model is trained on LiDAR point clouds from the KITTI dataset, and our point-wise segmentation labels are derived from 3D bounding boxes from KITTI. To obtain extra training data, we built a LiDAR simulator into Grand Theft Auto V (GTA-V), a popular video game, to synthesize large amounts of realistic training data. Our experiments show that SqueezeSeg achieves high accuracy with astonishingly fast and stable runtime (8.7 ms per frame), highly desirable for autonomous driving applications. Furthermore, additionally training on synthesized data boosts validation accuracy on real-world data. Our source code and synthesized data will be open-sourced. △ Less

Submitted 19 October, 2017; originally announced October 2017.

Conservative methods for dynamical systems

Authors: Andy T. S. Wan, Alexander Bihlo, Jean-Christophe Nave

Abstract: We show a novel systematic way to construct conservative finite difference schemes for quasilinear first-order system of ordinary differential equations with conserved quantities. In particular, this includes both autonomous and non-autonomous dynamical systems with conserved quantities of arbitrary forms, such as time-dependent conserved quantities. Sufficient conditions to construct conservative… ▽ More We show a novel systematic way to construct conservative finite difference schemes for quasilinear first-order system of ordinary differential equations with conserved quantities. In particular, this includes both autonomous and non-autonomous dynamical systems with conserved quantities of arbitrary forms, such as time-dependent conserved quantities. Sufficient conditions to construct conservative schemes of arbitrary order are derived using the multiplier method. General formulas for first-order conservative schemes are constructed using divided difference calculus. New conservative schemes are found for various dynamical systems such as Euler's equation of rigid body rotation, Lotka-Volterra systems, the planar restricted three-body problem and the damped harmonic oscillator. △ Less

Submitted 7 December, 2016; originally announced December 2016.

Comments: 29 pages

MSC Class: 65L05 65L06 65L12; 65L20; 65P10; 65Z05; 37M05; 37M15

Journal ref: SIAM J. Numer. Anal. 55(5) 2255-2285 (2017)

SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving

Authors: Bichen Wu, Alvin Wan, Forrest Iandola, Peter H. Jin, Kurt Keutzer

Abstract: Object detection is a crucial task for autonomous driving. In addition to requiring high accuracy to ensure safety, object detection for autonomous driving also requires real-time inference speed to guarantee prompt vehicle control, as well as small model size and energy efficiency to enable embedded system deployment. In this work, we propose SqueezeDet, a fully convolutional neural network for… ▽ More Object detection is a crucial task for autonomous driving. In addition to requiring high accuracy to ensure safety, object detection for autonomous driving also requires real-time inference speed to guarantee prompt vehicle control, as well as small model size and energy efficiency to enable embedded system deployment. In this work, we propose SqueezeDet, a fully convolutional neural network for object detection that aims to simultaneously satisfy all of the above constraints. In our network, we use convolutional layers not only to extract feature maps but also as the output layer to compute bounding boxes and class probabilities. The detection pipeline of our model only contains a single forward pass of a neural network, thus it is extremely fast. Our model is fully-convolutional, which leads to a small model size and better energy efficiency. While achieving the same accuracy as previous baselines, our model is 30.4x smaller, 19.7x faster, and consumes 35.2x lower energy. The code is open-sourced at \url{https://github.com/BichenWuUCB/squeezeDet}. △ Less

Submitted 11 June, 2019; v1 submitted 3 December, 2016; originally announced December 2016.

Comments: The supplementary material of this paper, which discusses the energy efficiency of SqueezeDet, is attached after the main paper. The source code of this work is open-source released at https://github.com/BichenWuUCB/squeezeDet

On the arbitrarily long-term stability of conservative methods

Authors: Andy T. S. Wan, Jean-Christophe Nave

Abstract: We show the arbitrarily long-term stability of conservative methods for autonomous ODEs. Given a system of autonomous ODEs with conserved quantities, if the preimage of the conserved quantities possesses a bounded locally nite neighborhood, then the global error of any conservative method with the uniformly bounded displacement property is bounded for all time, when the uniform time step is taken… ▽ More We show the arbitrarily long-term stability of conservative methods for autonomous ODEs. Given a system of autonomous ODEs with conserved quantities, if the preimage of the conserved quantities possesses a bounded locally nite neighborhood, then the global error of any conservative method with the uniformly bounded displacement property is bounded for all time, when the uniform time step is taken suciently small. On nite precision machines, the global error still remains bounded and independent of time until some arbitrarily large time determined by machine precision and tolerance. The main result is proved using elementary topological properties for discretized conserved quantities which are equicontinuous. In particular, long-term stability is also shown using an averaging identity when the discretized conserved quantities do not explicitly depend on time steps. Numerical results are presented to illustrate the long-term stability result. △ Less

Submitted 11 October, 2018; v1 submitted 20 July, 2016; originally announced July 2016.

Comments: 25 pages

MSC Class: 65L05 65L12; 65L20; 65L70; 65P10; 65Z05

Journal ref: SIAM J. Numer. Anal. 56(5) 2751-2775 (2018)

A Posteriori Error Estimation for the p-curl Problem

Authors: Andy T. S. Wan, Marc Laforest

Abstract: We derive a posteriori error estimates for a semi-discrete finite element approximation of a nonlinear eddy current problem arising from applied superconductivity, known as the $p$-curl problem. In particular, we show the reliability for non-conforming Nédélec elements based on a residual type argument and a Helmholtz-Weyl decomposition of $W^p_0(\text{curl};Ω)$. As a consequence, we are also able… ▽ More We derive a posteriori error estimates for a semi-discrete finite element approximation of a nonlinear eddy current problem arising from applied superconductivity, known as the $p$-curl problem. In particular, we show the reliability for non-conforming Nédélec elements based on a residual type argument and a Helmholtz-Weyl decomposition of $W^p_0(\text{curl};Ω)$. As a consequence, we are also able to derive an a posteriori error estimate for a quantity of interest called the AC loss. The nonlinearity for this form of Maxwell's equation is an analogue of the one found in the $p$-Laplacian. It is handled without linearizing around the approximate solution. The non-conformity is dealt by adapting error decomposition techniques of Carstensen, Hu and Orlando. Geometric non-conformities also appear because the continuous problem is defined over a bounded $C^{1,1}$ domain while the discrete problem is formulated over a weaker polyhedral domain. The semi-discrete formulation studied in this paper is often encountered in commercial codes and is shown to be well-posed. The paper concludes with numerical results confirming the reliability of the a posteriori error estimate. △ Less

Submitted 10 February, 2020; v1 submitted 20 May, 2016; originally announced May 2016.

Comments: 32 pages

MSC Class: 35K65; 65M60; 65M15; 78M10

Journal ref: SIAM J. Numer. Anal. 58(1) 460-491 (2020)

An inequality for the Fourier spectrum of parity decision trees

Authors: Eric Blais, Li-Yang Tan, Andrew Wan

Abstract: We give a new bound on the sum of the linear Fourier coefficients of a Boolean function in terms of its parity decision tree complexity. This result generalizes an inequality of O'Donnell and Servedio for regular decision trees. We use this bound to obtain the first non-trivial lower bound on the parity decision tree complexity of the recursive majority function. We give a new bound on the sum of the linear Fourier coefficients of a Boolean function in terms of its parity decision tree complexity. This result generalizes an inequality of O'Donnell and Servedio for regular decision trees. We use this bound to obtain the first non-trivial lower bound on the parity decision tree complexity of the recursive majority function. △ Less

Submitted 20 May, 2015; originally announced June 2015.

Mining Measured Information from Text

Authors: Arun S. Maiya, Dale Visser, Andrew Wan

Abstract: We present an approach to extract measured information from text (e.g., a 1370 degrees C melting point, a BMI greater than 29.9 kg/m^2 ). Such extractions are critically important across a wide range of domains - especially those involving search and exploration of scientific and technical documents. We first propose a rule-based entity extractor to mine measured quantities (i.e., a numeric value… ▽ More We present an approach to extract measured information from text (e.g., a 1370 degrees C melting point, a BMI greater than 29.9 kg/m^2 ). Such extractions are critically important across a wide range of domains - especially those involving search and exploration of scientific and technical documents. We first propose a rule-based entity extractor to mine measured quantities (i.e., a numeric value paired with a measurement unit), which supports a vast and comprehensive set of both common and obscure measurement units. Our method is highly robust and can correctly recover valid measured quantities even when significant errors are introduced through the process of converting document formats like PDF to plain text. Next, we describe an approach to extracting the properties being measured (e.g., the property "pixel pitch" in the phrase "a pixel pitch as high as 352 μm"). Finally, we present MQSearch: the realization of a search engine with full support for measured information. △ Less

Submitted 5 May, 2015; originally announced May 2015.

Comments: 4 pages; 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '15)

ACM Class: I.2.7; H.3.3

Striking Dispersion of Recrystallized Poly(ethylene glycol)-Poly(lactic acid) Solvent-Casting Blend

Authors: Xiaomin Zhu, Ran Huang, Tian Zhong, Ajun Wan

Abstract: The solvent casted Poly(ethylene glycol) (PEG) and Poly(lactic acid) (PLA) blend was recrystallized by the thermal treatment of a heating and cooling cycle. The high PEG content blend (30\%wt) showed a striking dispersion behavior, that the materials rapidly dispersed and dissolved in the aqueous environment in a few hours. This phenomenon has not been reported by others, and was not observed in l… ▽ More The solvent casted Poly(ethylene glycol) (PEG) and Poly(lactic acid) (PLA) blend was recrystallized by the thermal treatment of a heating and cooling cycle. The high PEG content blend (30\%wt) showed a striking dispersion behavior, that the materials rapidly dispersed and dissolved in the aqueous environment in a few hours. This phenomenon has not been reported by others, and was not observed in low PEG content samples of 5\%, 10\%, and 20\%, or quenched(amorphous) samples. We hypothesized the mechanism that the chain rearrangement during the thermal treatment leads to the phase separation. And with the phase separation in the recrystallized samples, the PEG potions rapidly dissolved in the aqueous environment, left out the small PLA spherulites being separated and dispersed in the solution. The same underlying reason can also be inferred from the degradation behaviors of other samples. Characterizations of DSC, XRD, and SEM have been done to validate our hypothesis. △ Less

Submitted 8 October, 2015; v1 submitted 9 February, 2015; originally announced February 2015.

Comments: 7 pages, 6 figures

Journal ref: Polymer(Korea), Vol.39, No.6, 889-895, November, 2015

The multiplier method to construct conservative finite difference schemes for ordinary and partial differential equations

Authors: Andy T. S. Wan, Alexander Bihlo, Jean-Christophe Nave

Abstract: We present the multiplier method of constructing conservative finite difference schemes for ordinary and partial differential equations. Given a system of differential equations possessing conservation laws, our approach is based on discretizing conservation law multipliers and their associated density and flux functions. We show that the proposed discretization is consistent for any order of accu… ▽ More We present the multiplier method of constructing conservative finite difference schemes for ordinary and partial differential equations. Given a system of differential equations possessing conservation laws, our approach is based on discretizing conservation law multipliers and their associated density and flux functions. We show that the proposed discretization is consistent for any order of accuracy when the discrete multiplier has a multiplicative inverse. Moreover, we show that by construction, discrete densities can be exactly conserved. In particular, the multiplier method does not require the system to possess a Hamiltonian or variational structure. Examples, including dissipative problems, are given to illustrate the method. In the case when the inverse of the discrete multiplier becomes singular, consistency of the method is also established for scalar ODEs provided the discrete multiplier and density are zero-compatible. △ Less

Submitted 8 July, 2015; v1 submitted 27 November, 2014; originally announced November 2014.

MSC Class: 65L12; 65L99; 65M06; 65M99; 65N06; 65N99

Journal ref: SIAM J. Numer. Anal. 54, 86 (2016)

Pseudorandomness and Fourier Growth Bounds for Width 3 Branching Programs

Authors: Thomas Steinke, Salil Vadhan, Andrew Wan

Abstract: We present an explicit pseudorandom generator for oblivious, read-once, width-$3$ branching programs, which can read their input bits in any order. The generator has seed length $\tilde{O}( \log^3 n ).$ The previously best known seed length for this model is $n^{1/2+o(1)}$ due to Impagliazzo, Meka, and Zuckerman (FOCS '12). Our work generalizes a recent result of Reingold, Steinke, and Vadhan (RAN… ▽ More We present an explicit pseudorandom generator for oblivious, read-once, width-$3$ branching programs, which can read their input bits in any order. The generator has seed length $\tilde{O}( \log^3 n ).$ The previously best known seed length for this model is $n^{1/2+o(1)}$ due to Impagliazzo, Meka, and Zuckerman (FOCS '12). Our work generalizes a recent result of Reingold, Steinke, and Vadhan (RANDOM '13) for \textit{permutation} branching programs. The main technical novelty underlying our generator is a new bound on the Fourier growth of width-3, oblivious, read-once branching programs. Specifically, we show that for any $f:\{0,1\}^n\rightarrow \{0,1\}$ computed by such a branching program, and $k\in [n],$ $$\sum_{s\subseteq [n]: |s|=k} \left| \hat{f}[s] \right| \leq n^2 \cdot (O(\log n))^k,$$ where $\widehat{f}[s] = \mathbb{E}\left[f[U] \cdot (-1)^{s \cdot U}\right]$ is the standard Fourier transform over $\mathbb{Z}_2^n$. The base $O(\log n)$ of the Fourier growth is tight up to a factor of $\log \log n$. △ Less

Submitted 27 May, 2014; originally announced May 2014.

Comments: arXiv admin note: text overlap with arXiv:1306.3004

Approximate resilience, monotonicity, and the complexity of agnostic learning

Authors: Dana Dachman-Soled, Vitaly Feldman, Li-Yang Tan, Andrew Wan, Karl Wimmer

Abstract: A function $f$ is $d$-resilient if all its Fourier coefficients of degree at most $d$ are zero, i.e., $f$ is uncorrelated with all low-degree parities. We study the notion of $\mathit{approximate}$ $\mathit{resilience}$ of Boolean functions, where we say that $f$ is $α$-approximately $d$-resilient if $f$ is $α$-close to a $[-1,1]$-valued $d$-resilient function in $\ell_1$ distance. We show that ap… ▽ More A function $f$ is $d$-resilient if all its Fourier coefficients of degree at most $d$ are zero, i.e., $f$ is uncorrelated with all low-degree parities. We study the notion of $\mathit{approximate}$ $\mathit{resilience}$ of Boolean functions, where we say that $f$ is $α$-approximately $d$-resilient if $f$ is $α$-close to a $[-1,1]$-valued $d$-resilient function in $\ell_1$ distance. We show that approximate resilience essentially characterizes the complexity of agnostic learning of a concept class $C$ over the uniform distribution. Roughly speaking, if all functions in a class $C$ are far from being $d$-resilient then $C$ can be learned agnostically in time $n^{O(d)}$ and conversely, if $C$ contains a function close to being $d$-resilient then agnostic learning of $C$ in the statistical query (SQ) framework of Kearns has complexity of at least $n^{Ω(d)}$. This characterization is based on the duality between $\ell_1$ approximation by degree-$d$ polynomials and approximate $d$-resilience that we establish. In particular, it implies that $\ell_1$ approximation by low-degree polynomials, known to be sufficient for agnostic learning over product distributions, is in fact necessary. Focusing on monotone Boolean functions, we exhibit the existence of near-optimal $α$-approximately $\widetildeΩ(α\sqrt{n})$-resilient monotone functions for all $α>0$. Prior to our work, it was conceivable even that every monotone function is $Ω(1)$-far from any $1$-resilient function. Furthermore, we construct simple, explicit monotone functions based on ${\sf Tribes}$ and ${\sf CycleRun}$ that are close to highly resilient functions. Our constructions are based on a fairly general resilience analysis and amplification. These structural results, together with the characterization, imply nearly optimal lower bounds for agnostic learning of monotone juntas. △ Less

Submitted 9 July, 2014; v1 submitted 20 May, 2014; originally announced May 2014.

The Crystallization Behavior of Porous PLA Prepared by Modified Solvent Casting/Particulate Leaching Technique for Potential Use of Tissue Engineering Scaffold

Authors: Ran Huang, Xiaomin Zhu, Haiyan Tu, Ajun Wan

Abstract: The porous PLA foams potential for tissue engineering usage are prepared by a modified solvent casting/particulate leaching method with different crystallinity. Since in typical method the porogens are solved in the solution and flow with the polymers during the casting and the crystallinity behavior of PLA chains in the limited space cannot be tracked, in this work the processing is modified by d… ▽ More The porous PLA foams potential for tissue engineering usage are prepared by a modified solvent casting/particulate leaching method with different crystallinity. Since in typical method the porogens are solved in the solution and flow with the polymers during the casting and the crystallinity behavior of PLA chains in the limited space cannot be tracked, in this work the processing is modified by diffusing the PLA solution into a steady salt stack. With a thermal treatment before leaching while maintaining the stable structure of the porogens stack, the crystallinity of porous foams is made possible to control. The characterizations indicate the crystallization of porous foams is in a manner of lower crystallibility than the bulk materials. Pores and caves of around 250μm size are obtained in samples with different crystallinity. The macro-structures are not much impaired by the crystallization nevertheless the morphological effect of the heating process is still obvious. △ Less

Submitted 14 April, 2014; originally announced April 2014.

Comments: 9 pages, 2 figures

Journal ref: Mater. Lett. 136 (2014) 126-129

Decision Trees, Protocols, and the Fourier Entropy-Influence Conjecture

Authors: Andrew Wan, John Wright, Chenggang Wu

Abstract: Given $f:\{-1, 1\}^n \rightarrow \{-1, 1\}$, define the \emph{spectral distribution} of $f$ to be the distribution on subsets of $[n]$ in which the set $S$ is sampled with probability $\widehat{f}(S)^2$. Then the Fourier Entropy-Influence (FEI) conjecture of Friedgut and Kalai (1996) states that there is some absolute constant $C$ such that… ▽ More Given $f:\{-1, 1\}^n \rightarrow \{-1, 1\}$, define the \emph{spectral distribution} of $f$ to be the distribution on subsets of $[n]$ in which the set $S$ is sampled with probability $\widehat{f}(S)^2$. Then the Fourier Entropy-Influence (FEI) conjecture of Friedgut and Kalai (1996) states that there is some absolute constant $C$ such that $\operatorname{H}[\widehat{f}^2] \leq C\cdot\operatorname{Inf}[f]$. Here, $\operatorname{H}[\widehat{f}^2]$ denotes the Shannon entropy of $f$'s spectral distribution, and $\operatorname{Inf}[f]$ is the total influence of $f$. This conjecture is one of the major open problems in the analysis of Boolean functions, and settling it would have several interesting consequences. Previous results on the FEI conjecture have been largely through direct calculation. In this paper we study a natural interpretation of the conjecture, which states that there exists a communication protocol which, given subset $S$ of $[n]$ distributed as $\widehat{f}^2$, can communicate the value of $S$ using at most $C\cdot\operatorname{Inf}[f]$ bits in expectation. Using this interpretation, we are able show the following results: 1. First, if $f$ is computable by a read-$k$ decision tree, then $\operatorname{H}[\widehat{f}^2] \leq 9k\cdot \operatorname{Inf}[f]$. 2. Next, if $f$ has $\operatorname{Inf}[f] \geq 1$ and is computable by a decision tree with expected depth $d$, then $\operatorname{H}[\widehat{f}^2] \leq 12d\cdot \operatorname{Inf}[f]$. 3. Finally, we give a new proof of the main theorem of O'Donnell and Tan (ICALP 2013), i.e. that their FEI$^+$ conjecture composes. In addition, we show that natural improvements to our decision tree results would be sufficient to prove the FEI conjecture in its entirety. We believe that our methods give more illuminating proofs than previous results about the FEI conjecture. △ Less

Submitted 10 December, 2013; originally announced December 2013.

ACM Class: F.1.3

Satisfiability and Evolution

Authors: Adi Livnat, Christos Papadimitriou, Aviad Rubinstein, Gregory Valiant, Andrew Wan

Abstract: We show that, if truth assignments on $n$ variables reproduce through recombination so that satisfaction of a particular Boolean function confers a small evolutionary advantage, then a polynomially large population over polynomially many generations (polynomial in $n$ and the inverse of the initial satisfaction probability) will end up almost certainly consisting exclusively of satisfying truth as… ▽ More We show that, if truth assignments on $n$ variables reproduce through recombination so that satisfaction of a particular Boolean function confers a small evolutionary advantage, then a polynomially large population over polynomially many generations (polynomial in $n$ and the inverse of the initial satisfaction probability) will end up almost certainly consisting exclusively of satisfying truth assignments. We argue that this theorem sheds light on the problem of novelty in Evolution. △ Less

Submitted 11 August, 2014; v1 submitted 6 December, 2013; originally announced December 2013.

MSC Class: 92D15 ACM Class: F.0

Turbulence analysis of an experimental flux rope plasma

Authors: D. A. Schaffner, V. S. Lukin, A. Wan, M. R. Brown

Abstract: We have previously generated elongated Taylor double-helix flux rope plasmas in the SSX MHD wind tunnel. These plasmas are remarkable in their rapid relaxation (about one Alfvén time) and their description by simple analytical Taylor force-free theory despite their high plasma beta and high internal flow speeds. We report on the turbulent features observed in these plasmas including frequency spec… ▽ More We have previously generated elongated Taylor double-helix flux rope plasmas in the SSX MHD wind tunnel. These plasmas are remarkable in their rapid relaxation (about one Alfvén time) and their description by simple analytical Taylor force-free theory despite their high plasma beta and high internal flow speeds. We report on the turbulent features observed in these plasmas including frequency spectra, autocorrelation function, and probability distribution functions of increments. We discuss here the possibility that the turbulence facilitating access to the final state supports coherent structures and intermittency revealed by non-Gaussian signatures in the statistics. Comparisons to a Hall-MHD simulation of the SSX MHD wind tunnel show similarity in several statistical measures. △ Less

Submitted 8 October, 2013; originally announced October 2013.

Comments: 20 pages, 9 figures, submitted to Plasma Physics Controlled Fusion for Special Issue on Flux Ropes

Journal ref: Plasma Phys. Control. Fusion 56, 064003 (2014)

Faster Private Release of Marginals on Small Databases

Authors: Karthekeyan Chandrasekaran, Justin Thaler, Jonathan Ullman, Andrew Wan

Abstract: We study the problem of answering \emph{$k$-way marginal} queries on a database $D \in (\{0,1\}^d)^n$, while preserving differential privacy. The answer to a $k$-way marginal query is the fraction of the database's records $x \in \{0,1\}^d$ with a given value in each of a given set of up to $k$ columns. Marginal queries enable a rich class of statistical analyses on a dataset, and designing effici… ▽ More We study the problem of answering \emph{$k$-way marginal} queries on a database $D \in (\{0,1\}^d)^n$, while preserving differential privacy. The answer to a $k$-way marginal query is the fraction of the database's records $x \in \{0,1\}^d$ with a given value in each of a given set of up to $k$ columns. Marginal queries enable a rich class of statistical analyses on a dataset, and designing efficient algorithms for privately answering marginal queries has been identified as an important open problem in private data analysis. For any $k$, we give a differentially private online algorithm that runs in time $$ \min{\exp(d^{1-Ω(1/\sqrt{k})}), \exp(d / \log^{.99} d)\} $$ per query and answers any (possibly superpolynomially long and adaptively chosen) sequence of $k$-way marginal queries up to error at most $\pm .01$ on every query, provided $n \gtrsim d^{.51} $. To the best of our knowledge, this is the first algorithm capable of privately answering marginal queries with a non-trivial worst-case accuracy guarantee on a database of size $\poly(d, k)$ in time $\exp(o(d))$. Our algorithms are a variant of the private multiplicative weights algorithm (Hardt and Rothblum, FOCS '10), but using a different low-weight representation of the database. We derive our low-weight representation using approximations to the OR function by low-degree polynomials with coefficients of bounded $L_1$-norm. We also prove a strong limitation on our approach that is of independent approximation-theoretic interest. Specifically, we show that for any $k = o(\log d)$, any polynomial with coefficients of $L_1$-norm $poly(d)$ that pointwise approximates the $d$-variate OR function on all inputs of Hamming weight at most $k$ must have degree $d^{1-O(1/\sqrt{k})}$. △ Less

Submitted 2 September, 2013; v1 submitted 12 April, 2013; originally announced April 2013.

On the Distribution of the Fourier Spectrum of Halfspaces

Authors: Ilias Diakonikolas, Ragesh Jaiswal, Rocco A. Servedio, Li-Yang Tan, Andrew Wan

Abstract: Bourgain showed that any noise stable Boolean function $f$ can be well-approximated by a junta. In this note we give an exponential sharpening of the parameters of Bourgain's result under the additional assumption that $f$ is a halfspace. Bourgain showed that any noise stable Boolean function $f$ can be well-approximated by a junta. In this note we give an exponential sharpening of the parameters of Bourgain's result under the additional assumption that $f$ is a halfspace. △ Less

Submitted 29 February, 2012; originally announced February 2012.

A regularity lemma, and low-weight approximators, for low-degree polynomial threshold functions

Authors: Ilias Diakonikolas, Rocco A. Servedio, Li-Yang Tan, Andrew Wan

Abstract: We give a "regularity lemma" for degree-d polynomial threshold functions (PTFs) over the Boolean cube {-1,1}^n. This result shows that every degree-d PTF can be decomposed into a constant number of subfunctions such that almost all of the subfunctions are close to being regular PTFs. Here a "regular PTF is a PTF sign(p(x)) where the influence of each variable on the polynomial p(x) is a small fr… ▽ More We give a "regularity lemma" for degree-d polynomial threshold functions (PTFs) over the Boolean cube {-1,1}^n. This result shows that every degree-d PTF can be decomposed into a constant number of subfunctions such that almost all of the subfunctions are close to being regular PTFs. Here a "regular PTF is a PTF sign(p(x)) where the influence of each variable on the polynomial p(x) is a small fraction of the total influence of p. As an application of this regularity lemma, we prove that for any constants d \geq 1, \eps \geq 0, every degree-d PTF over n variables has can be approximated to accuracy eps by a constant-degree PTF that has integer weights of total magnitude O(n^d). This weight bound is shown to be optimal up to constant factors. △ Less

Submitted 5 May, 2010; v1 submitted 25 September, 2009; originally announced September 2009.

Comments: 23 pages, 0 figures

ACM Class: F.1.3

Efficiently Testing Sparse GF(2) Polynomials

Authors: Ilias Diakonikolas, Homin K. Lee, Kevin Matulef, Rocco A. Servedio, Andrew Wan

Abstract: We give the first algorithm that is both query-efficient and time-efficient for testing whether an unknown function $f: \{0,1\}^n \to \{0,1\}$ is an $s$-sparse GF(2) polynomial versus $\eps$-far from every such polynomial. Our algorithm makes $\poly(s,1/\eps)$ black-box queries to $f$ and runs in time $n \cdot \poly(s,1/\eps)$. The only previous algorithm for this testing problem \cite{DLM+:07}… ▽ More We give the first algorithm that is both query-efficient and time-efficient for testing whether an unknown function $f: \{0,1\}^n \to \{0,1\}$ is an $s$-sparse GF(2) polynomial versus $\eps$-far from every such polynomial. Our algorithm makes $\poly(s,1/\eps)$ black-box queries to $f$ and runs in time $n \cdot \poly(s,1/\eps)$. The only previous algorithm for this testing problem \cite{DLM+:07} used poly$(s,1/\eps)$ queries, but had running time exponential in $s$ and super-polynomial in $1/\eps$. Our approach significantly extends the ``testing by implicit learning'' methodology of \cite{DLM+:07}. The learning component of that earlier work was a brute-force exhaustive search over a concept class to find a hypothesis consistent with a sample of random examples. In this work, the learning component is a sophisticated exact learning algorithm for sparse GF(2) polynomials due to Schapire and Sellie \cite{SchapireSellie:96}. A crucial element of this work, which enables us to simulate the membership queries required by \cite{SchapireSellie:96}, is an analysis establishing new properties of how sparse GF(2) polynomials simplify under certain restrictions of ``low-influence'' sets of variables. △ Less

Submitted 12 May, 2008; originally announced May 2008.

Comments: Full version of ICALP 2008 paper

Landau-Zener transitions in the presence of spin environment

Authors: Andy T. S. Wan, M. H. S. Amin, Shannon X. Wang

Abstract: We study the effect of an environment consisting of noninteracting two level systems on Landau-Zener transitions with an interest on the performance of an adiabatic quantum computer. We show that if the environment is initially at zero temperature, it does not affect the transition probability. An excited environment, however, will always increase the probability of making a transition out of th… ▽ More We study the effect of an environment consisting of noninteracting two level systems on Landau-Zener transitions with an interest on the performance of an adiabatic quantum computer. We show that if the environment is initially at zero temperature, it does not affect the transition probability. An excited environment, however, will always increase the probability of making a transition out of the ground state. For the case of equal intermediate gaps, we find an analytical upper bound for the transition probability in the limit of large number of environmental spins. We show that such an environment will only suppress the probability of success for adiabatic quantum computation by at most a factor close to 1/2. △ Less

Submitted 16 March, 2009; v1 submitted 2 March, 2007; originally announced March 2007.

Comments: 12 pages, 3 figures, to appear in Int. J. Quant. Inf

Journal ref: Int. J. Quant. Inf. 7, 725 (2009)

Detailed Radiative Transport Modeling of a Radiative Divertor

Authors: A. S. Wan, H. E. Dalhed, H. A. Scott, D. E. Post, T. D. Rognlien

Abstract: An effective radiative divertor maximizes the utilization of atomic processes to spread out the energy deposition to the divertor chamber walls and to reduce the peak heat flux. Because the mixture of neutral atoms and ions in the divertor can be optically thick to a portion of radiated power, it is necessary to accurately model the magnitude and distribution of line radiation in this complex re… ▽ More An effective radiative divertor maximizes the utilization of atomic processes to spread out the energy deposition to the divertor chamber walls and to reduce the peak heat flux. Because the mixture of neutral atoms and ions in the divertor can be optically thick to a portion of radiated power, it is necessary to accurately model the magnitude and distribution of line radiation in this complex region. To assess their importance we calculate the effects of radiation transport using CRETIN, a multi-dimensional, non-local thermodynamic equilibrium simulation code that includes the atomic kinetics and radiative transport processes necessary to model the complex environment of a radiative divertor. We also include neutral transport to model radiation from recycling neutral atoms. This paper presents a case study of a high-recycling radiative divertor with a typical large neutral pressure at the divertor plate to estimate the impact of H line radiation on the overall power balance in the divertor region with consideration for line opacities and atomic kinetics. △ Less

Submitted 11 July, 1995; originally announced July 1995.

Comments: Preprint for the 11th International Conference on Plasma Surface Interactions in Mito, Japan (1994), uuencoded and gzipped postscript with 3 figures, 15 pages