Showing 1–39 of 39 results for author: Baxter, J

Inference of Causal Networks using a Topological Threshold

Authors: Filipe Barroso, Diogo Gomes, Gareth J. Baxter

Abstract: We propose a constraint-based algorithm, which automatically determines causal relevance thresholds, to infer causal networks from data. We call these topological thresholds. We present two methods for determining the threshold: the first seeks a set of edges that leaves no disconnected nodes in the network; the second seeks a causal large connected component in the data. We tested these methods… ▽ More We propose a constraint-based algorithm, which automatically determines causal relevance thresholds, to infer causal networks from data. We call these topological thresholds. We present two methods for determining the threshold: the first seeks a set of edges that leaves no disconnected nodes in the network; the second seeks a causal large connected component in the data. We tested these methods both for discrete synthetic and real data, and compared the results with those obtained for the PC algorithm, which we took as the benchmark. We show that this novel algorithm is generally faster and more accurate than the PC algorithm. The algorithm for determining the thresholds requires choosing a measure of causality. We tested our methods for Fisher Correlations, commonly used in PC algorithm (for instance in \cite{kalisch2005}), and further proposed a discrete and asymmetric measure of causality, that we called Net Influence, which provided very good results when inferring causal networks from discrete data. This metric allows for inferring directionality of the edges in the process of applying the thresholds, speeding up the inference of causal DAGs. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Comments: 17 pages, 12 figures

A Systematic Review of Low-Rank and Local Low-Rank Matrix Approximation in Big Data Medical Imaging

Authors: Sisipho Hamlomo, Marcellin Atemkeng, Yusuf Brima, Chuneeta Nunhokee, Jeremy Baxter

Abstract: The large volume and complexity of medical imaging datasets are bottlenecks for storage, transmission, and processing. To tackle these challenges, the application of low-rank matrix approximation (LRMA) and its derivative, local LRMA (LLRMA) has demonstrated potential. A detailed analysis of the literature identifies LRMA and LLRMA methods applied to various imaging modalities, and the challenges… ▽ More The large volume and complexity of medical imaging datasets are bottlenecks for storage, transmission, and processing. To tackle these challenges, the application of low-rank matrix approximation (LRMA) and its derivative, local LRMA (LLRMA) has demonstrated potential. A detailed analysis of the literature identifies LRMA and LLRMA methods applied to various imaging modalities, and the challenges and limitations associated with existing LRMA and LLRMA methods are addressed. We note a significant shift towards a preference for LLRMA in the medical imaging field since 2015, demonstrating its potential and effectiveness in capturing complex structures in medical data compared to LRMA. Acknowledging the limitations of shallow similarity methods used with LLRMA, we suggest advanced semantic image segmentation for similarity measure, explaining in detail how it can be used to measure similar patches and its feasibility. We note that LRMA and LLRMA are mainly applied to unstructured medical data, and we propose extending their application to different medical data types, including structured and semi-structured. This paper also discusses how LRMA and LLRMA can be applied to regular data with missing entries and the impact of inaccuracies in predicting missing values and their effects. We discuss the impact of patch size and propose the use of random search (RS) to determine the optimal patch size. To enhance feasibility, a hybrid approach using Bayesian optimization and RS is proposed, which could improve the application of LRMA and LLRMA in medical imaging. △ Less

Submitted 27 May, 2024; v1 submitted 21 February, 2024; originally announced February 2024.

SuperBench: Improving Cloud AI Infrastructure Reliability with Proactive Validation

Authors: Yifan Xiong, Yuting Jiang, Ziyue Yang, Lei Qu, Guoshuai Zhao, Shuguang Liu, Dong Zhong, Boris Pinzur, Jie Zhang, Yang Wang, Jithin Jose, Hossein Pourreza, Jeff Baxter, Kushal Datta, Prabhat Ram, Luke Melton, Joe Chau, Peng Cheng, Yongqiang Xiong, Lidong Zhou

Abstract: Reliability in cloud AI infrastructure is crucial for cloud service providers, prompting the widespread use of hardware redundancies. However, these redundancies can inadvertently lead to hidden degradation, so called "gray failure", for AI workloads, significantly affecting end-to-end performance and concealing performance issues, which complicates root cause analysis for failures and regressions… ▽ More Reliability in cloud AI infrastructure is crucial for cloud service providers, prompting the widespread use of hardware redundancies. However, these redundancies can inadvertently lead to hidden degradation, so called "gray failure", for AI workloads, significantly affecting end-to-end performance and concealing performance issues, which complicates root cause analysis for failures and regressions. We introduce SuperBench, a proactive validation system for AI infrastructure that mitigates hidden degradation caused by hardware redundancies and enhances overall reliability. SuperBench features a comprehensive benchmark suite, capable of evaluating individual hardware components and representing most real AI workloads. It comprises a Validator which learns benchmark criteria to clearly pinpoint defective components. Additionally, SuperBench incorporates a Selector to balance validation time and issue-related penalties, enabling optimal timing for validation execution with a tailored subset of benchmarks. Through testbed evaluation and simulation, we demonstrate that SuperBench can increase the mean time between incidents by up to 22.61x. SuperBench has been successfully deployed in Azure production, validating hundreds of thousands of GPUs over the last two years. △ Less

Submitted 7 June, 2024; v1 submitted 9 February, 2024; originally announced February 2024.

Comments: USENIX ATC '24

Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation

Authors: Stefan Wojcik, Sophie Hilgard, Nick Judd, Delia Mocanu, Stephen Ragain, M. B. Fallin Hunzaker, Keith Coleman, Jay Baxter

Abstract: We present an approach for selecting objectively informative and subjectively helpful annotations to social media posts. We draw on data from on an online environment where contributors annotate misinformation and simultaneously rate the contributions of others. Our algorithm uses a matrix-factorization (MF) based approach to identify annotations that appeal broadly across heterogeneous user group… ▽ More We present an approach for selecting objectively informative and subjectively helpful annotations to social media posts. We draw on data from on an online environment where contributors annotate misinformation and simultaneously rate the contributions of others. Our algorithm uses a matrix-factorization (MF) based approach to identify annotations that appeal broadly across heterogeneous user groups - sometimes referred to as "bridging-based ranking." We pair these data with a survey experiment in which individuals are randomly assigned to see annotations to posts. We find that annotations selected by the algorithm improve key indicators compared with overall average and crowd-generated baselines. Further, when deployed on Twitter, people who saw annotations selected through this bridging-based approach were significantly less likely to reshare social media posts than those who did not see the annotations. △ Less

Submitted 27 October, 2022; originally announced October 2022.

Enabling scalable clinical interpretation of ML-based phenotypes using real world data

Authors: Owen Parsons, Nathan E Barlow, Janie Baxter, Karen Paraschin, Andrea Derix, Peter Hein, Robert Dürichen

Abstract: The availability of large and deep electronic healthcare records (EHR) datasets has the potential to enable a better understanding of real-world patient journeys, and to identify novel subgroups of patients. ML-based aggregation of EHR data is mostly tool-driven, i.e., building on available or newly developed methods. However, these methods, their input requirements, and, importantly, resulting ou… ▽ More The availability of large and deep electronic healthcare records (EHR) datasets has the potential to enable a better understanding of real-world patient journeys, and to identify novel subgroups of patients. ML-based aggregation of EHR data is mostly tool-driven, i.e., building on available or newly developed methods. However, these methods, their input requirements, and, importantly, resulting output are frequently difficult to interpret, especially without in-depth data science or statistical training. This endangers the final step of analysis where an actionable and clinically meaningful interpretation is needed.This study investigates approaches to perform patient stratification analysis at scale using large EHR datasets and multiple clustering methods for clinical research. We have developed several tools to facilitate the clinical evaluation and interpretation of unsupervised patient stratification results, namely pattern screening, meta clustering, surrogate modeling, and curation. These tools can be used at different stages within the analysis. As compared to a standard analysis approach, we demonstrate the ability to condense results and optimize analysis time. In the case of meta clustering, we demonstrate that the number of patient clusters can be reduced from 72 to 3 in one example. In another stratification result, by using surrogate models, we could quickly identify that heart failure patients were stratified if blood sodium measurements were available. As this is a routine measurement performed for all patients with heart failure, this indicated a data bias. By using further cohort and feature curation, these patients and other irrelevant features could be removed to increase the clinical meaningfulness. These examples show the effectiveness of the proposed methods and we hope to encourage further research in this field. △ Less

Submitted 2 August, 2022; originally announced August 2022.

Comments: 27 pages, 14 figures

MSC Class: 92C50; 92C60; 68T09; 62H30 ACM Class: J.3; I.5; I.2

Compensating trajectory bias for unsupervised patient stratification using adversarial recurrent neural networks

Authors: Avelino Javer, Owen Parsons, Oliver Carr, Janie Baxter, Christian Diedrich, Eren Elçi, Steffen Schaper, Katrin Coboeken, Robert Dürichen

Abstract: Electronic healthcare records are an important source of information which can be used in patient stratification to discover novel disease phenotypes. However, they can be challenging to work with as data is often sparse and irregularly sampled. One approach to solve these limitations is learning dense embeddings that represent individual patient trajectories using a recurrent neural network autoe… ▽ More Electronic healthcare records are an important source of information which can be used in patient stratification to discover novel disease phenotypes. However, they can be challenging to work with as data is often sparse and irregularly sampled. One approach to solve these limitations is learning dense embeddings that represent individual patient trajectories using a recurrent neural network autoencoder (RNN-AE). This process can be susceptible to unwanted data biases. We show that patient embeddings and clusters using previously proposed RNN-AE models might be impacted by a trajectory bias, meaning that results are dominated by the amount of data contained in each patients trajectory, instead of clinically relevant details. We investigate this bias on 2 datasets (from different hospitals) and 2 disease areas as well as using different parts of the patient trajectory. Our results using 2 previously published baseline methods indicate a particularly strong bias in case of an event-to-end trajectory. We present a method that can overcome this issue using an adversarial training scheme on top of a RNN-AE. Our results show that our approach can reduce the trajectory bias in all cases. △ Less

Submitted 14 December, 2021; originally announced December 2021.

The Ethics of Biosurveillance

Authors: S. K. Devitt, P. W. J. Baxter, G. Hamilton

Abstract: Governments must keep agricultural systems free of pests that threaten agricultural production and international trade. Biosecurity surveillance already makes use of a wide range of technologies, such as insect traps and lures, geographic information systems, and diagnostic biochemical tests. The rise of cheap and usable surveillance technologies such as remotely piloted aircraft systems (RPAS) pr… ▽ More Governments must keep agricultural systems free of pests that threaten agricultural production and international trade. Biosecurity surveillance already makes use of a wide range of technologies, such as insect traps and lures, geographic information systems, and diagnostic biochemical tests. The rise of cheap and usable surveillance technologies such as remotely piloted aircraft systems (RPAS) presents value conflicts not addressed in international biosurveillance guidelines. The costs of keeping agriculture pest-free include privacy violations and reduced autonomy for farmers. We argue that physical and digital privacy in the age of ubiquitous aerial and ground surveillance is a natural right to allow people to function freely on their land. Surveillance methods must be co-created and justified through using ethically defensible processes such as discourse theory, value-centred design and responsible innovation to forge a cooperative social contract between diverse stakeholders. We propose an ethical framework for biosurveillance activities that balances the collective benefits for food security with individual privacy: (1) establish the boundaries of a biosurveillance social contract; (2) justify surveillance operations for the farmers, researchers, industry, the public and regulators; (3) give decision makers a reasonable measure of control over their personal and agricultural data; and (4) choose surveillance methodologies that give the appropriate information. The benefits of incorporating an ethical framework for responsible biosurveillance innovation include increased participation and accumulated trust over time. Long term trust and cooperation will support food security, producing higher quality data overall and mitigating against anticipated information gaps that may emerge due to disrespecting landholder rights △ Less

Submitted 23 November, 2021; originally announced November 2021.

Comments: 32 pages, 2 figures, 2019, Journal of Agricultural and Environmental Ethics

ACM Class: K.4.0; K.4.1; K.4.2; K.4.3

Journal ref: Journal of Agricultural and Environmental Ethics, 32(5), 709-740 (2019)

Lessons Learned Addressing Dataset Bias in Model-Based Candidate Generation at Twitter

Authors: Alim Virani, Jay Baxter, Dan Shiebler, Philip Gautier, Shivam Verma, Yan Xia, Apoorv Sharma, Sumit Binnani, Linlin Chen, Chenguang Yu

Abstract: Traditionally, heuristic methods are used to generate candidates for large scale recommender systems. Model-based candidate generation promises multiple potential advantages, primarily that we can explicitly optimize the same objective as the downstream ranking model. However, large scale model-based candidate generation approaches suffer from dataset bias problems caused by the infeasibility of o… ▽ More Traditionally, heuristic methods are used to generate candidates for large scale recommender systems. Model-based candidate generation promises multiple potential advantages, primarily that we can explicitly optimize the same objective as the downstream ranking model. However, large scale model-based candidate generation approaches suffer from dataset bias problems caused by the infeasibility of obtaining representative data on very irrelevant candidates. Popular techniques to correct dataset bias, such as inverse propensity scoring, do not work well in the context of candidate generation. We first explore the dynamics of the dataset bias problem and then demonstrate how to use random sampling techniques to mitigate it. Finally, in a novel application of fine-tuning, we show performance gains when applying our candidate generation system to Twitter's home timeline. △ Less

Submitted 13 May, 2021; originally announced May 2021.

Deep Semi-Supervised Embedded Clustering (DSEC) for Stratification of Heart Failure Patients

Authors: Oliver Carr, Stojan Jovanovic, Luca Albergante, Fernando Andreotti, Robert Dürichen, Nadia Lipunova, Janie Baxter, Rabia Khan, Benjamin Irving

Abstract: Determining phenotypes of diseases can have considerable benefits for in-hospital patient care and to drug development. The structure of high dimensional data sets such as electronic health records are often represented through an embedding of the data, with clustering methods used to group data of similar structure. If subgroups are known to exist within data, supervised methods may be used to in… ▽ More Determining phenotypes of diseases can have considerable benefits for in-hospital patient care and to drug development. The structure of high dimensional data sets such as electronic health records are often represented through an embedding of the data, with clustering methods used to group data of similar structure. If subgroups are known to exist within data, supervised methods may be used to influence the clusters discovered. We propose to extend deep embedded clustering to a semi-supervised deep embedded clustering algorithm to stratify subgroups through known labels in the data. In this work we apply deep semi-supervised embedded clustering to determine data-driven patient subgroups of heart failure from the electronic health records of 4,487 heart failure and control patients. We find clinically relevant clusters from an embedded space derived from heterogeneous data. The proposed algorithm can potentially find new undiagnosed subgroups of patients that have different outcomes, and, therefore, lead to improved treatments. △ Less

Submitted 17 January, 2021; v1 submitted 24 December, 2020; originally announced December 2020.

Comments: 6 pages, 2 figures, HSYS workshop at ICML conference

Theoretical Models of Learning to Learn

Authors: Jonathan Baxter

Abstract: A Machine can only learn if it is biased in some way. Typically the bias is supplied by hand, for example through the choice of an appropriate set of features. However, if the learning machine is embedded within an {\em environment} of related tasks, then it can {\em learn} its own bias by learning sufficiently many tasks from the environment. In this paper two models of bias learning (or equivale… ▽ More A Machine can only learn if it is biased in some way. Typically the bias is supplied by hand, for example through the choice of an appropriate set of features. However, if the learning machine is embedded within an {\em environment} of related tasks, then it can {\em learn} its own bias by learning sufficiently many tasks from the environment. In this paper two models of bias learning (or equivalently, learning to learn) are introduced and the main theoretical results presented. The first model is a PAC-type model based on empirical process theory, while the second is a hierarchical Bayes model. △ Less

Submitted 27 February, 2020; originally announced February 2020.

Comments: arXiv admin note: text overlap with arXiv:1106.0245

Journal ref: in Learning to Learn (edited by Sebastian Thrun and Lorien Pratt), 159-179 (1998)

Some observations concerning Off Training Set (OTS) error

Authors: Jonathan Baxter

Abstract: A form of generalisation error known as Off Training Set (OTS) error was recently introduced in [Wolpert, 1996b], along with a theorem showing that small training set error does not guarantee small OTS error, unless assumptions are made about the target function. Here it is shown that the applicability of this theorem is limited to models in which the distribution generating training data has no o… ▽ More A form of generalisation error known as Off Training Set (OTS) error was recently introduced in [Wolpert, 1996b], along with a theorem showing that small training set error does not guarantee small OTS error, unless assumptions are made about the target function. Here it is shown that the applicability of this theorem is limited to models in which the distribution generating training data has no overlap with the distribution generating test data. It is argued that such a scenario is of limited relevance to machine learning. △ Less

Submitted 17 November, 2019; originally announced December 2019.

Comments: Technical Report, Australian National University, August 1999

General Matrix-Matrix Multiplication Using SIMD features of the PIII

Authors: Douglas Aberdeen, Jonathan Baxter

Abstract: Generalised matrix-matrix multiplication forms the kernel of many mathematical algorithms. A faster matrix-matrix multiply immediately benefits these algorithms. In this paper we implement efficient matrix multiplication for large matrices using the floating point Intel Pentium SIMD (Single Instruction Multiple Data) architecture. A description of the issues and our solution is presented, paying a… ▽ More Generalised matrix-matrix multiplication forms the kernel of many mathematical algorithms. A faster matrix-matrix multiply immediately benefits these algorithms. In this paper we implement efficient matrix multiplication for large matrices using the floating point Intel Pentium SIMD (Single Instruction Multiple Data) architecture. A description of the issues and our solution is presented, paying attention to all levels of the memory hierarchy. Our results demonstrate an average performance of 2.09 times faster than the leading public domain matrix-matrix multiply routines. △ Less

Submitted 18 November, 2019; originally announced December 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1911.05181

Journal ref: Euro-Par '00 Proceedings from the 6th International Euro-Par Conference on Parallel Processing (2000) Pages 980-983

Hebbian Synaptic Modifications in Spiking Neurons that Learn

Authors: Peter L. Bartlett, Jonathan Baxter

Abstract: In this paper, we derive a new model of synaptic plasticity, based on recent algorithms for reinforcement learning (in which an agent attempts to learn appropriate actions to maximize its long-term average reward). We show that these direct reinforcement learning algorithms also give locally optimal performance for the problem of reinforcement learning with multiple agents, without any explicit co… ▽ More In this paper, we derive a new model of synaptic plasticity, based on recent algorithms for reinforcement learning (in which an agent attempts to learn appropriate actions to maximize its long-term average reward). We show that these direct reinforcement learning algorithms also give locally optimal performance for the problem of reinforcement learning with multiple agents, without any explicit communication between agents. By considering a network of spiking neurons as a collection of agents attempting to maximize the long-term average of a reward signal, we derive a synaptic update rule that is qualitatively similar to Hebb's postulate. This rule requires only simple computations, such as addition and leaky integration, and involves only quantities that are available in the vicinity of the synapse. Furthermore, it leads to synaptic connection strengths that give locally optimal values of the long term average reward. The reinforcement learning paradigm is sufficiently broad to encompass many learning problems that are solved by the brain. We illustrate, with simulations, that the approach is effective for simple pattern classification and motor learning tasks. △ Less

Submitted 17 November, 2019; originally announced November 2019.

The Canonical Distortion Measure for Vector Quantization and Function Approximation

Authors: Jonathan Baxter

Abstract: To measure the quality of a set of vector quantization points a means of measuring the distance between a random point and its quantization is required. Common metrics such as the {\em Hamming} and {\em Euclidean} metrics, while mathematically simple, are inappropriate for comparing natural signals such as speech or images. In this paper it is shown how an {\em environment} of functions on an inpu… ▽ More To measure the quality of a set of vector quantization points a means of measuring the distance between a random point and its quantization is required. Common metrics such as the {\em Hamming} and {\em Euclidean} metrics, while mathematically simple, are inappropriate for comparing natural signals such as speech or images. In this paper it is shown how an {\em environment} of functions on an input space $X$ induces a {\em canonical distortion measure} (CDM) on X. The depiction 'canonical" is justified because it is shown that optimizing the reconstruction error of X with respect to the CDM gives rise to optimal piecewise constant approximations of the functions in the environment. The CDM is calculated in closed form for several different function classes. An algorithm for training neural networks to implement the CDM is presented along with some encouraging experimental results. △ Less

Submitted 14 November, 2019; originally announced November 2019.

Journal ref: In: Thrun S., Pratt L. (eds) Learning to Learn (1998). Pages 159-177

Learning Model Bias

Authors: Jonathan Baxter

Abstract: In this paper the problem of {\em learning} appropriate domain-specific bias is addressed. It is shown that this can be achieved by learning many related tasks from the same domain, and a theorem is given bounding the number tasks that must be learnt. A corollary of the theorem is that if the tasks are known to possess a common {\em internal representation} or {\em preprocessing} then the number o… ▽ More In this paper the problem of {\em learning} appropriate domain-specific bias is addressed. It is shown that this can be achieved by learning many related tasks from the same domain, and a theorem is given bounding the number tasks that must be learnt. A corollary of the theorem is that if the tasks are known to possess a common {\em internal representation} or {\em preprocessing} then the number of examples required per task for good generalisation when learning $n$ tasks simultaneously scales like $O(a + \frac{b}{n})$, where $O(a)$ is a bound on the minimum number of examples required to learn a single task, and $O(a + b)$ is a bound on the number of examples required to learn each task independently. An experiment providing strong qualitative support for the theoretical results is reported. △ Less

Submitted 14 November, 2019; originally announced November 2019.

Journal ref: Advances in Neural Information Processing Systems 8, 1995, 169-175

A Bayesian/Information Theoretic Model of Bias Learning

Authors: Jonathan Baxter

Abstract: In this paper the problem of learning appropriate bias for an environment of related tasks is examined from a Bayesian perspective. The environment of related tasks is shown to be naturally modelled by the concept of an {\em objective} prior distribution. Sampling from the objective prior corresponds to sampling different learning tasks from the environment. It is argued that for many common machi… ▽ More In this paper the problem of learning appropriate bias for an environment of related tasks is examined from a Bayesian perspective. The environment of related tasks is shown to be naturally modelled by the concept of an {\em objective} prior distribution. Sampling from the objective prior corresponds to sampling different learning tasks from the environment. It is argued that for many common machine learning problems, although we don't know the true (objective) prior for the problem, we do have some idea of a set of possible priors to which the true prior belongs. It is shown that under these circumstances a learner can use Bayesian inference to learn the true prior by sampling from the objective prior. Bounds are given on the amount of information required to learn a task when it is simultaneously learnt with several other tasks. The bounds show that if the learner has little knowledge of the true prior, and the dimensionality of the true prior is small, then sampling multiple tasks is highly advantageous. △ Less

Submitted 14 November, 2019; originally announced November 2019.

Journal ref: COLT 96 Proceedings of the ninth annual conference on Computational learning theory (1996) Pages 77-88

Learning Internal Representations (COLT 1995)

Authors: Jonathan Baxter

Abstract: Probably the most important problem in machine learning is the preliminary biasing of a learner's hypothesis space so that it is small enough to ensure good generalisation from reasonable training sets, yet large enough that it contains a good solution to the problem being learnt. In this paper a mechanism for {\em automatically} learning or biasing the learner's hypothesis space is introduced. It… ▽ More Probably the most important problem in machine learning is the preliminary biasing of a learner's hypothesis space so that it is small enough to ensure good generalisation from reasonable training sets, yet large enough that it contains a good solution to the problem being learnt. In this paper a mechanism for {\em automatically} learning or biasing the learner's hypothesis space is introduced. It works by first learning an appropriate {\em internal representation} for a learning environment and then using that representation to bias the learner's hypothesis space for the learning of future tasks drawn from the same environment. An internal representation must be learnt by sampling from {\em many similar tasks}, not just a single task as occurs in ordinary machine learning. It is proved that the number of examples $m$ {\em per task} required to ensure good generalisation from a representation learner obeys $m = O(a+b/n)$ where $n$ is the number of tasks being learnt and $a$ and $b$ are constants. If the tasks are learnt independently ({\em i.e.} without a common representation) then $m=O(a+b)$. It is argued that for learning environments such as speech and character recognition $b\gg a$ and hence representation learning in these environments can potentially yield a drastic reduction in the number of examples required per task. It is also proved that if $n = O(b)$ (with $m=O(a+b/n)$) then the representation learnt will be good for learning novel tasks from the same environment, and that the number of examples required to generalise well on a novel task will be reduced to $O(a)$ (as opposed to $O(a+b)$ if no representation is used). It is shown that gradient descent can be used to train neural network representations and experiment results are reported providing strong qualitative support for the theoretical results. △ Less

Submitted 19 December, 2019; v1 submitted 13 November, 2019; originally announced November 2019.

Journal ref: COLT '95 Proceedings of the eighth annual conference on Computational learning theory (1995) 311-320

92c/MFlops/s, Ultra-Large-Scale Neural-Network Training on a PIII Cluster

Authors: Douglas Aberdeen, Jonathan Baxter, Robert Edwards

Abstract: Artificial neural networks with millions of adjustable parameters and a similar number of training examples are a potential solution for difficult, large-scale pattern recognition problems in areas such as speech and face recognition, classification of large volumes of web data, and finance. The bottleneck is that neural network training involves iterative gradient descent and is extremely computa… ▽ More Artificial neural networks with millions of adjustable parameters and a similar number of training examples are a potential solution for difficult, large-scale pattern recognition problems in areas such as speech and face recognition, classification of large volumes of web data, and finance. The bottleneck is that neural network training involves iterative gradient descent and is extremely computationally intensive. In this paper we present a technique for distributed training of Ultra Large Scale Neural Networks (ULSNN) on Bunyip, a Linux-based cluster of 196 Pentium III processors. To illustrate ULSNN training we describe an experiment in which a neural network with 1.73 million adjustable parameters was trained to recognize machine-printed Japanese characters from a database containing 9 million training patterns. The training runs with a average performance of 163.3 GFlops/s (single precision). With a machine cost of \$150,913, this yields a price/performance ratio of 92.4c/MFlops/s (single precision). For comparison purposes, training using double precision and the ATLAS DGEMM produces a sustained performance of 70 MFlops/s or \$2.16 / MFlop/s (double precision). △ Less

Submitted 12 November, 2019; originally announced November 2019.

Comments: SC '00: Proceedings of the 2000 ACM/IEEE Conference on Supercomputing

Journal ref: ACM/IEEE SC 2000 Conference (SC00)

Learning Internal Representations (PhD Thesis)

Authors: Jonathan Baxter

Abstract: Most machine learning theory and practice is concerned with learning a single task. In this thesis it is argued that in general there is insufficient information in a single task for a learner to generalise well and that what is required for good generalisation is information about many similar learning tasks. Similar learning tasks form a body of prior information that can be used to constrain th… ▽ More Most machine learning theory and practice is concerned with learning a single task. In this thesis it is argued that in general there is insufficient information in a single task for a learner to generalise well and that what is required for good generalisation is information about many similar learning tasks. Similar learning tasks form a body of prior information that can be used to constrain the learner and make it generalise better. Examples of learning scenarios in which there are many similar tasks are handwritten character recognition and spoken word recognition. The concept of the environment of a learner is introduced as a probability measure over the set of learning problems the learner might be expected to learn. It is shown how a sample from the environment may be used to learn a representation, or recoding of the input space that is appropriate for the environment. Learning a representation can equivalently be thought of as learning the appropriate features of the environment. Bounds are derived on the sample size required to ensure good generalisation from a representation learning process. These bounds show that under certain circumstances learning a representation appropriate for $n$ tasks reduces the number of examples required of each task by a factor of $n$. Once a representation is learnt it can be used to learn novel tasks from the same environment, with the result that far fewer examples are required of the new tasks to ensure good generalisation. Bounds are given on the number of tasks and the number of samples from each task required to ensure that a representation will be a good one for learning novel tasks. The results on representation learning are generalised to cover any form of automated hypothesis space bias. △ Less

Submitted 22 November, 2019; v1 submitted 9 November, 2019; originally announced November 2019.

Comments: Phd Thesis, Jonathan Baxter, 1994

Priorities in tock-CSP

Authors: Pedro Ribeiro, James Baxter, Ana Cavalcanti

Abstract: The $tock$-CSP encoding embeds a rich and flexible approach to modelling discrete timed behaviours in CSP where the event $tock$ is interpreted to mark the passage of time. The model checker FDR provides tailored support for $tock$-CSP, including a prioritisation operator that has typically been used to ensure maximal progress, where time only advances after internal activity has stabilised. Prior… ▽ More The $tock$-CSP encoding embeds a rich and flexible approach to modelling discrete timed behaviours in CSP where the event $tock$ is interpreted to mark the passage of time. The model checker FDR provides tailored support for $tock$-CSP, including a prioritisation operator that has typically been used to ensure maximal progress, where time only advances after internal activity has stabilised. Prioritisation may also be used on its own right as a modelling construct. Its operational semantics, however, is only congruent over the most discriminating semantic model of CSP: the finite-linear model. To enable sound and compositional reasoning in a $tock$-CSP setting, we calculate a denotational definition for prioritisation. For that we establish a Galois connection between a specialisation of the finite-linear model, with $tock$ and $\checkmark$, that signals termination, as special events, and $\checkmark$-$tock$-CSP, a model for $tock$-CSP that captures termination, deadlines, and is adequate for reasoning about timed refinement. Our results are mechanised using Isabelle/HOL. △ Less

Submitted 18 July, 2019; originally announced July 2019.

Comments: 9 pages, submitted to Information Processing Letters, July 2019

Unifying Semantic Foundations for Automated Verification Tools in Isabelle/UTP

Authors: Simon Foster, James Baxter, Ana Cavalcanti, Jim Woodcock, Frank Zeyda

Abstract: The growing complexity and diversity of models used in the engineering of dependable systems implies that a variety of formal methods, across differing abstractions, paradigms, and presentations, must be integrated. Such an integration relies on unified semantic foundations for the various notations, and co-ordination of a variety of automated verification tools. The contribution of this paper is… ▽ More The growing complexity and diversity of models used in the engineering of dependable systems implies that a variety of formal methods, across differing abstractions, paradigms, and presentations, must be integrated. Such an integration relies on unified semantic foundations for the various notations, and co-ordination of a variety of automated verification tools. The contribution of this paper is Isabelle/UTP, an implementation of Hoare and He's Unifying Theories of Programming, a framework for unification of formal semantics. Isabelle/UTP permits the mechanisation of computational theories for diverse paradigms, and their use in constructing formalised semantic models. These can be further applied in the development of verification tools, harnessing Isabelle's proof automation facilities. Several layers of mathematical foundations are developed, including lenses to model variables and state spaces as algebraic objects, alphabetised predicates and relations to model programs, including algebraic and axiomatic semantics, proof tools for Hoare logic and refinement calculus, and UTP theories to encode computational paradigms. △ Less

Submitted 22 June, 2020; v1 submitted 14 May, 2019; originally announced May 2019.

Comments: 40 pages, Accepted for Science of Computer Programming, June 2020

Fighting Redundancy and Model Decay with Embeddings

Authors: Dan Shiebler, Luca Belli, Jay Baxter, Hanchen Xiong, Abhishek Tayal

Abstract: Every day, hundreds of millions of new Tweets containing over 40 languages of ever-shifting vernacular flow through Twitter. Models that attempt to extract insight from this firehose of information must face the torrential covariate shift that is endemic to the Twitter platform. While regularly-retrained algorithms can maintain performance in the face of this shift, fixed model features that fail… ▽ More Every day, hundreds of millions of new Tweets containing over 40 languages of ever-shifting vernacular flow through Twitter. Models that attempt to extract insight from this firehose of information must face the torrential covariate shift that is endemic to the Twitter platform. While regularly-retrained algorithms can maintain performance in the face of this shift, fixed model features that fail to represent new trends and tokens can quickly become stale, resulting in performance degradation. To mitigate this problem we employ learned features, or embedding models, that can efficiently represent the most relevant aspects of a data distribution. Sharing these embedding models across teams can also reduce redundancy and multiplicatively increase cross-team modeling productivity. In this paper, we detail the commoditized tools, algorithms and pipelines that we have developed and are developing at Twitter to regularly generate high quality, up-to-date embeddings and share them broadly across the company. △ Less

Submitted 18 September, 2018; originally announced September 2018.

Comments: Presented at the Common Model Infrastructure Workshop at KDD 2018 (link: https://cmi2018.sdsc.edu/)

Automating Verification of State Machines with Reactive Designs and Isabelle/UTP

Authors: Simon Foster, James Baxter, Ana Cavalcanti, Alvaro Miyazawa, Jim Woodcock

Abstract: State-machine based notations are ubiquitous in the description of component systems, particularly in the robotic domain. To ensure these systems are safe and predictable, formal verification techniques are important, and can be cost-effective if they are both automated and scalable. In this paper, we present a verification approach for a diagrammatic state machine language that utilises theorem p… ▽ More State-machine based notations are ubiquitous in the description of component systems, particularly in the robotic domain. To ensure these systems are safe and predictable, formal verification techniques are important, and can be cost-effective if they are both automated and scalable. In this paper, we present a verification approach for a diagrammatic state machine language that utilises theorem proving and a denotational semantics based on Unifying Theories of Programming (UTP). We provide the necessary theory to underpin state machines (including induction theorems for iterative processes), mechanise an action language for states and transitions, and use these to formalise the semantics. We then describe the verification approach, which supports infinite state systems, and exemplify it with a fully automated deadlock-freedom check. The work has been mechanised in our proof tool, Isabelle/UTP, and so also illustrates the use of UTP to build practical verification tools. △ Less

Submitted 24 August, 2018; v1 submitted 23 July, 2018; originally announced July 2018.

Comments: 18 pages, 16th Intl. Conf. on Formal Aspects of Component Software (FACS 2018), October 2018, Pohang, South Korea

A comparative study of artificial intelligence and human doctors for the purpose of triage and diagnosis

Authors: Salman Razzaki, Adam Baker, Yura Perov, Katherine Middleton, Janie Baxter, Daniel Mullarkey, Davinder Sangar, Michael Taliercio, Mobasher Butt, Azeem Majeed, Arnold DoRosario, Megan Mahoney, Saurabh Johri

Abstract: Online symptom checkers have significant potential to improve patient care, however their reliability and accuracy remain variable. We hypothesised that an artificial intelligence (AI) powered triage and diagnostic system would compare favourably with human doctors with respect to triage and diagnostic accuracy. We performed a prospective validation study of the accuracy and safety of an AI powere… ▽ More Online symptom checkers have significant potential to improve patient care, however their reliability and accuracy remain variable. We hypothesised that an artificial intelligence (AI) powered triage and diagnostic system would compare favourably with human doctors with respect to triage and diagnostic accuracy. We performed a prospective validation study of the accuracy and safety of an AI powered triage and diagnostic system. Identical cases were evaluated by both an AI system and human doctors. Differential diagnoses and triage outcomes were evaluated by an independent judge, who was blinded from knowing the source (AI system or human doctor) of the outcomes. Independently of these cases, vignettes from publicly available resources were also assessed to provide a benchmark to previous studies and the diagnostic component of the MRCGP exam. Overall we found that the Babylon AI powered Triage and Diagnostic System was able to identify the condition modelled by a clinical vignette with accuracy comparable to human doctors (in terms of precision and recall). In addition, we found that the triage advice recommended by the AI System was, on average, safer than that of human doctors, when compared to the ranges of acceptable triage provided by independent expert judges, with only a minimal reduction in appropriateness. △ Less

Submitted 27 June, 2018; originally announced June 2018.

Targeted Damage to Interdependent Networks

Authors: G. J. Baxter, G. Timár, J. F. F. Mendes

Abstract: The giant mutually connected component (GMCC) of an interdependent or multiplex network collapses with a discontinuous hybrid transition under random damage to the network. If the nodes to be damaged are selected in a targeted way, the collapse of the GMCC may occur significantly sooner. Finding the minimal damage set which destroys the largest mutually connected component of a given interdependen… ▽ More The giant mutually connected component (GMCC) of an interdependent or multiplex network collapses with a discontinuous hybrid transition under random damage to the network. If the nodes to be damaged are selected in a targeted way, the collapse of the GMCC may occur significantly sooner. Finding the minimal damage set which destroys the largest mutually connected component of a given interdependent network is a computationally prohibitive simultaneous optimization problem. We introduce a simple heuristic strategy -- Effective Multiplex Degree -- for targeted attack on interdependent networks that leverages the indirect damage inherent in multiplex networks to achieve a damage set smaller than that found by any other non computationally intensive algorithm. We show that the intuition from single layer networks that decycling (damage of the $2$-core) is the most effective way to destroy the giant component, does not carry over to interdependent networks, and in fact such approaches are worse than simply removing the highest degree nodes. △ Less

Submitted 24 September, 2018; v1 submitted 12 February, 2018; originally announced February 2018.

Comments: 9 pages, 9 figures

Journal ref: Phys. Rev. E 98, 032307 (2018)

BayesDB: A probabilistic programming system for querying the probable implications of data

Authors: Vikash Mansinghka, Richard Tibbetts, Jay Baxter, Pat Shafto, Baxter Eaves

Abstract: Is it possible to make statistical inference broadly accessible to non-statisticians without sacrificing mathematical rigor or inference quality? This paper describes BayesDB, a probabilistic programming platform that aims to enable users to query the probable implications of their data as directly as SQL databases enable them to query the data itself. This paper focuses on four aspects of BayesDB… ▽ More Is it possible to make statistical inference broadly accessible to non-statisticians without sacrificing mathematical rigor or inference quality? This paper describes BayesDB, a probabilistic programming platform that aims to enable users to query the probable implications of their data as directly as SQL databases enable them to query the data itself. This paper focuses on four aspects of BayesDB: (i) BQL, an SQL-like query language for Bayesian data analysis, that answers queries by averaging over an implicit space of probabilistic models; (ii) techniques for implementing BQL using a broad class of multivariate probabilistic models; (iii) a semi-parametric Bayesian model-builder that auomatically builds ensembles of factorial mixture models to serve as baselines; and (iv) MML, a "meta-modeling" language for imposing qualitative constraints on the model-builder and combining baseline models with custom algorithmic and statistical models that can be implemented in external software. BayesDB is illustrated using three applications: cleaning and exploring a public database of Earth satellites; assessing the evidence for temporal dependence between macroeconomic indicators; and analyzing a salary survey. △ Less

Submitted 15 December, 2015; originally announced December 2015.

A Continuous Max-Flow Approach to Cyclic Field Reconstruction

Authors: John S. H. Baxter, Jonathan McLeod, Terry M. Peters

Abstract: Reconstruction of an image from noisy data using Markov Random Field theory has been explored by both the graph-cuts and continuous max-flow community in the form of the Potts and Ishikawa models. However, neither model takes into account the particular cyclic topology of specific intensity types such as the hue in natural colour images, or the phase in complex valued MRI. This paper presents \tex… ▽ More Reconstruction of an image from noisy data using Markov Random Field theory has been explored by both the graph-cuts and continuous max-flow community in the form of the Potts and Ishikawa models. However, neither model takes into account the particular cyclic topology of specific intensity types such as the hue in natural colour images, or the phase in complex valued MRI. This paper presents \textit{cyclic continuous max-flow} image reconstruction which models the intensity being reconstructed as having a fundamentally cyclic topology. This model complements the Ishikawa model in that it is designed with image reconstruction in mind, having the topology of the intensity space inherent in the model while being readily extendable to an arbitrary intensity resolution. △ Less

Submitted 11 November, 2015; originally announced November 2015.

Comments: 8 pages, 1 figure

Shape Complexes in Continuous Max-Flow Hierarchical Multi-Labeling Problems

Authors: John S. H. Baxter, Jing Yuan, Terry M. Peters

Abstract: Although topological considerations amongst multiple labels have been previously investigated in the context of continuous max-flow image segmentation, similar investigations have yet to be made about shape considerations in a general and extendable manner. This paper presents shape complexes for segmentation, which capture more complex shapes by combining multiple labels and super-labels constrai… ▽ More Although topological considerations amongst multiple labels have been previously investigated in the context of continuous max-flow image segmentation, similar investigations have yet to be made about shape considerations in a general and extendable manner. This paper presents shape complexes for segmentation, which capture more complex shapes by combining multiple labels and super-labels constrained by geodesic star convexity. Shape complexes combine geodesic star convexity constraints with hierarchical label organization, which together allow for more complex shapes to be represented. This framework avoids the use of co-ordinate system warping techniques to convert shape constraints into topological constraints, which may be ambiguous or ill-defined for certain segmentation problems. △ Less

Submitted 15 October, 2015; originally announced October 2015.

Comments: 9 pages, 1 figure

A Proximal Bregman Projection Approach to Continuous Max-Flow Problems Using Entropic Distances

Authors: John S. H. Baxter, Martin Rajchl, Jing Yuan, Terry M. Peters

Abstract: One issue limiting the adaption of large-scale multi-region segmentation is the sometimes prohibitive memory requirements. This is especially troubling considering advances in massively parallel computing and commercial graphics processing units because of their already limited memory compared to the current random access memory used in more traditional computation. To address this issue in the fi… ▽ More One issue limiting the adaption of large-scale multi-region segmentation is the sometimes prohibitive memory requirements. This is especially troubling considering advances in massively parallel computing and commercial graphics processing units because of their already limited memory compared to the current random access memory used in more traditional computation. To address this issue in the field of continuous max-flow segmentation, we have developed a \textit{pseudo-flow} framework using the theory of Bregman proximal projections and entropic distances which implicitly represents flow variables between labels and designated source and sink nodes. This reduces the memory requirements for max-flow segmentation by approximately 20\% for Potts models and approximately 30\% for hierarchical max-flow (HMF) and directed acyclic graph max-flow (DAGMF) models. This represents a great improvement in the state-of-the-art in max-flow segmentation, allowing for much larger problems to be addressed and accelerated using commercially available graphics processing hardware. △ Less

Submitted 30 January, 2015; originally announced January 2015.

Comments: 10 pages

A Continuous Max-Flow Approach to Multi-Labeling Problems under Arbitrary Region Regularization

Authors: John S. H. Baxter, Martin Rajchl, Jing Yuan, Terry M. Peters

Abstract: The incorporation of region regularization into max-flow segmentation has traditionally focused on ordering and part-whole relationships. A side effect of the development of such models is that it constrained regularization only to those cases, rather than allowing for arbitrary region regularization. Directed Acyclic Graphical Max-Flow (DAGMF) segmentation overcomes these limitations by allowing… ▽ More The incorporation of region regularization into max-flow segmentation has traditionally focused on ordering and part-whole relationships. A side effect of the development of such models is that it constrained regularization only to those cases, rather than allowing for arbitrary region regularization. Directed Acyclic Graphical Max-Flow (DAGMF) segmentation overcomes these limitations by allowing for the algorithm designer to specify an arbitrary directed acyclic graph to structure a max-flow segmentation. This allows for individual 'parts' to be a member of multiple distinct 'wholes.' △ Less

Submitted 5 June, 2014; v1 submitted 5 May, 2014; originally announced May 2014.

Comments: 10 pages, 2 figures, 3 algorithms - v2: Fixed typos / grammatical errors and mathematical errors in the primal/dual formulation. Extended methods for weighted DAGs rather than DAGs with edge multiplicity

RANCOR: Non-Linear Image Registration with Total Variation Regularization

Authors: Martin Rajchl, John S. H. Baxter, Wu Qiu, Ali R. Khan, Aaron Fenster, Terry M. Peters, Jing Yuan

Abstract: Optimization techniques have been widely used in deformable registration, allowing for the incorporation of similarity metrics with regularization mechanisms. These regularization mechanisms are designed to mitigate the effects of trivial solutions to ill-posed registration problems and to otherwise ensure the resulting deformation fields are well-behaved. This paper introduces a novel deformable… ▽ More Optimization techniques have been widely used in deformable registration, allowing for the incorporation of similarity metrics with regularization mechanisms. These regularization mechanisms are designed to mitigate the effects of trivial solutions to ill-posed registration problems and to otherwise ensure the resulting deformation fields are well-behaved. This paper introduces a novel deformable registration algorithm, RANCOR, which uses iterative convexification to address deformable registration problems under total-variation regularization. Initial comparative results against four state-of-the-art registration algorithms are presented using the Internet Brain Segmentation Repository (IBSR) database. △ Less

Submitted 9 April, 2014; originally announced April 2014.

Comments: 9 pages, 1 figure, technical note

A Continuous Max-Flow Approach to General Hierarchical Multi-Labeling Problems

Authors: John S. H. Baxter, Martin Rajchl, Jing Yuan, Terry M. Peters

Abstract: Multi-region segmentation algorithms often have the onus of incorporating complex anatomical knowledge representing spatial or geometric relationships between objects, and general-purpose methods of addressing this knowledge in an optimization-based manner have thus been lacking. This paper presents Generalized Hierarchical Max-Flow (GHMF) segmentation, which captures simple anatomical part-whole… ▽ More Multi-region segmentation algorithms often have the onus of incorporating complex anatomical knowledge representing spatial or geometric relationships between objects, and general-purpose methods of addressing this knowledge in an optimization-based manner have thus been lacking. This paper presents Generalized Hierarchical Max-Flow (GHMF) segmentation, which captures simple anatomical part-whole relationships in the form of an unconstrained hierarchy. Regularization can then be applied to both parts and wholes independently, allowing for spatial grouping and clustering of labels in a globally optimal convex optimization framework. For the purposes of ready integration into a variety of segmentation tasks, the hierarchies can be presented in run-time, allowing for the segmentation problem to be readily specified and alternatives explored without undue programming effort or recompilation. △ Less

Submitted 5 June, 2014; v1 submitted 1 April, 2014; originally announced April 2014.

Comments: 11 pages, 1 figure, 3 algorithms -v2: Fixed typos / grammatical errors

Weak percolation on multiplex networks

Authors: Gareth J. Baxter, Sergey N. Dorogovtsev, José F. F. Mendes, Davide Cellai

Abstract: Bootstrap percolation is a simple but non-trivial model. It has applications in many areas of science and has been explored on random networks for several decades. In single layer (simplex) networks, it has been recently observed that bootstrap percolation, which is defined as an incremental process, can be seen as the opposite of pruning percolation, where nodes are removed according to a connect… ▽ More Bootstrap percolation is a simple but non-trivial model. It has applications in many areas of science and has been explored on random networks for several decades. In single layer (simplex) networks, it has been recently observed that bootstrap percolation, which is defined as an incremental process, can be seen as the opposite of pruning percolation, where nodes are removed according to a connectivity rule. Here we propose models of both bootstrap and pruning percolation for multiplex networks. We collectively refer to these two models with the concept of "weak" percolation, to distinguish them from the somewhat classical concept of ordinary ("strong") percolation. While the two models coincide in simplex networks, we show that they decouple when considering multiplexes, giving rise to a wealth of critical phenomena. Our bootstrap model constitutes the simplest example of a contagion process on a multiplex network and has potential applications in critical infrastructure recovery and information security. Moreover, we show that our pruning percolation model may provide a way to diagnose missing layers in a multiplex network. Finally, our analytical approach allows us to calculate critical behavior and characterize critical clusters. △ Less

Submitted 13 April, 2014; v1 submitted 13 December, 2013; originally announced December 2013.

Comments: 14 pages, 12 figures

Journal ref: Phys. Rev. E 89, 042801 (2014)

Experiments with Infinite-Horizon, Policy-Gradient Estimation

Authors: J. Baxter, P. L. Bartlett, L. Weaver

Abstract: In this paper, we present algorithms that perform gradient ascent of the average reward in a partially observable Markov decision process (POMDP). These algorithms are based on GPOMDP, an algorithm introduced in a companion paper (Baxter and Bartlett, this volume), which computes biased estimates of the performance gradient in POMDPs. The algorithm's chief advantages are that it uses only one free… ▽ More In this paper, we present algorithms that perform gradient ascent of the average reward in a partially observable Markov decision process (POMDP). These algorithms are based on GPOMDP, an algorithm introduced in a companion paper (Baxter and Bartlett, this volume), which computes biased estimates of the performance gradient in POMDPs. The algorithm's chief advantages are that it uses only one free parameter beta, which has a natural interpretation in terms of bias-variance trade-off, it requires no knowledge of the underlying state, and it can be applied to infinite state, control and observation spaces. We show how the gradient estimates produced by GPOMDP can be used to perform gradient ascent, both with a traditional stochastic-gradient algorithm, and with an algorithm based on conjugate-gradients that utilizes gradient information to bracket maxima in line searches. Experimental results are presented illustrating both the theoretical results of (Baxter and Bartlett, this volume) on a toy problem, and practical aspects of the algorithms on a number of more realistic problems. △ Less

Submitted 13 November, 2019; v1 submitted 3 June, 2011; originally announced June 2011.

Journal ref: Journal Of Artificial Intelligence Research, Volume 15, pages 351-381, 2001

Infinite-Horizon Policy-Gradient Estimation

Authors: Jonathan Baxter, Peter L. Bartlett

Abstract: Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a {\em biased} estimate of the gradient of the {\em average… ▽ More Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a {\em biased} estimate of the gradient of the {\em average reward} in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter $β\in [0,1)$ (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter $β$ is related to the {\em mixing time} of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward △ Less

Submitted 15 November, 2019; v1 submitted 3 June, 2011; originally announced June 2011.

Journal ref: Journal Of Artificial Intelligence Research, Volume 15, pages 319-350, 2001

A Model of Inductive Bias Learning

Authors: J. Baxter

Abstract: A major problem in machine learning is that of inductive bias: how to choose a learner's hypothesis space so that it is large enough to contain a solution to the problem being learnt, yet small enough to ensure reliable generalization from reasonably-sized training sets. Typically such bias is supplied by hand through the skill and insights of experts. In this paper a model for aut… ▽ More A major problem in machine learning is that of inductive bias: how to choose a learner's hypothesis space so that it is large enough to contain a solution to the problem being learnt, yet small enough to ensure reliable generalization from reasonably-sized training sets. Typically such bias is supplied by hand through the skill and insights of experts. In this paper a model for automatically learning bias is investigated. The central assumption of the model is that the learner is embedded within an environment of related learning tasks. Within such an environment the learner can sample from multiple tasks, and hence it can search for a hypothesis space that contains good solutions to many of the problems in the environment. Under certain restrictions on the set of all hypothesis spaces available to the learner, we show that a hypothesis space that performs well on a sufficiently large number of training tasks will also perform well when learning novel tasks in the same environment. Explicit bounds are also derived demonstrating that learning multiple tasks within an environment of related tasks can potentially give much better generalization than learning a single task. △ Less

Submitted 1 June, 2011; originally announced June 2011.

Journal ref: Journal Of Artificial Intelligence Research, Volume 12, pages 149-198, 2000

Software graphs and programmer awareness

Authors: G. J. Baxter, M. R. Frean

Abstract: Dependencies between types in object-oriented software can be viewed as directed graphs, with types as nodes and dependencies as edges. The in-degree and out-degree distributions of such graphs have quite different forms, with the former resembling a power-law distribution and the latter an exponential distribution. This effect appears to be independent of application or type relationship. A sim… ▽ More Dependencies between types in object-oriented software can be viewed as directed graphs, with types as nodes and dependencies as edges. The in-degree and out-degree distributions of such graphs have quite different forms, with the former resembling a power-law distribution and the latter an exponential distribution. This effect appears to be independent of application or type relationship. A simple generative model is proposed to explore the proposition that the difference arises because the programmer is aware of the out-degree of a type but not of its in-degree. The model reproduces the two distributions, and compares reasonably well to those observed in 14 different type relationships across 12 different Java applications. △ Less

Submitted 15 February, 2008; originally announced February 2008.

Comments: 9 pages, 8 figures

KnightCap: A chess program that learns by combining TD(lambda) with game-tree search

Authors: Jonathan Baxter, Andrew Tridgell, Lex Weaver

Abstract: In this paper we present TDLeaf(lambda), a variation on the TD(lambda) algorithm that enables it to be used in conjunction with game-tree search. We present some experiments in which our chess program ``KnightCap'' used TDLeaf(lambda) to learn its evaluation function while playing on the Free Internet Chess Server (FICS, fics.onenet.net). The main success we report is that KnightCap improved fro… ▽ More In this paper we present TDLeaf(lambda), a variation on the TD(lambda) algorithm that enables it to be used in conjunction with game-tree search. We present some experiments in which our chess program ``KnightCap'' used TDLeaf(lambda) to learn its evaluation function while playing on the Free Internet Chess Server (FICS, fics.onenet.net). The main success we report is that KnightCap improved from a 1650 rating to a 2150 rating in just 308 games and 3 days of play. As a reference, a rating of 1650 corresponds to about level B human play (on a scale from E (1000) to A (1800)), while 2150 is human master level. We discuss some of the reasons for this success, principle among them being the use of on-line, rather than self-play. △ Less

Submitted 9 January, 1999; originally announced January 1999.

Comments: 9 pages

ACM Class: I.2.6

Journal ref: MACHINE LEARNING Proceedings of the Fifteenth International Conference (ICML '98), ISBN 1-55860-556-8, ISSN 1049-1910, Madison WISCONSIN, July 24-27 1998, pages 28-36

TDLeaf(lambda): Combining Temporal Difference Learning with Game-Tree Search

Authors: Jonathan Baxter, Andrew Tridgell, Lex Weaver

Abstract: In this paper we present TDLeaf(lambda), a variation on the TD(lambda) algorithm that enables it to be used in conjunction with minimax search. We present some experiments in both chess and backgammon which demonstrate its utility and provide comparisons with TD(lambda) and another less radical variant, TD-directed(lambda). In particular, our chess program, ``KnightCap,'' used TDLeaf(lambda) to… ▽ More In this paper we present TDLeaf(lambda), a variation on the TD(lambda) algorithm that enables it to be used in conjunction with minimax search. We present some experiments in both chess and backgammon which demonstrate its utility and provide comparisons with TD(lambda) and another less radical variant, TD-directed(lambda). In particular, our chess program, ``KnightCap,'' used TDLeaf(lambda) to learn its evaluation function while playing on the Free Internet Chess Server (FICS, fics.onenet.net). It improved from a 1650 rating to a 2100 rating in just 308 games. We discuss some of the reasons for this success and the relationship between our results and Tesauro's results in backgammon. △ Less

Submitted 4 January, 1999; originally announced January 1999.

Comments: 5 pages. Also in Proceedings of the Ninth Australian Conference on Neural Networks (ACNN'98), Brisbane QLD, February 1998, pages 168-172

ACM Class: I.2.6

Journal ref: Australian Journal of Intelligent Information Processing Systems, ISSN 1321-2133, Vol. 5 No. 1, Autumn 1998, pages 39-43