Experience Seamless Full Ebook Downloads for Every Genre at textbookfull.

com

Deep Learning with JavaScript: Neural networks in

TensorFlow.js 1st Edition Shanqing Cai

https://textbookfull.com/product/deep-learning-with-
javascript-neural-networks-in-tensorflow-js-1st-edition-
shanqing-cai/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Deep Learning with JavaScript Neural networks in

TensorFlow js 1st Edition Shanqing Cai Stanley Bileschi
Eric D Nielsen
https://textbookfull.com/product/deep-learning-with-javascript-neural-
networks-in-tensorflow-js-1st-edition-shanqing-cai-stanley-bileschi-
eric-d-nielsen/
textboxfull.com

MATLAB Deep Learning: With Machine Learning, Neural

Networks and Artificial Intelligence 1st Edition Phil Kim

https://textbookfull.com/product/matlab-deep-learning-with-machine-
learning-neural-networks-and-artificial-intelligence-1st-edition-phil-
kim/
textboxfull.com

Applied Neural Networks with TensorFlow 2: API Oriented

Deep Learning with Python Orhan Gazi Yalç■n

https://textbookfull.com/product/applied-neural-networks-with-
tensorflow-2-api-oriented-deep-learning-with-python-orhan-gazi-yalcin/

textboxfull.com

Neural Networks and Deep Learning A Textbook Charu C.

Aggarwal

https://textbookfull.com/product/neural-networks-and-deep-learning-a-
textbook-charu-c-aggarwal/

textboxfull.com
Applied Neural Networks with TensorFlow 2 API Oriented
Deep Learning with Python 1st Edition Orhan Gazi Yalc■n
Yalç■n Orhan
https://textbookfull.com/product/applied-neural-networks-with-
tensorflow-2-api-oriented-deep-learning-with-python-1st-edition-orhan-
gazi-yalcin-yalcin-orhan/
textboxfull.com

Learn Keras for Deep Neural Networks: A Fast-Track

Approach to Modern Deep Learning with Python 1st Edition
Jojo John Moolayil
https://textbookfull.com/product/learn-keras-for-deep-neural-networks-
a-fast-track-approach-to-modern-deep-learning-with-python-1st-edition-
jojo-john-moolayil/
textboxfull.com

Practical TensorFlow.js: Deep Learning in Web App

Development Juan De Dios Santos Rivera

https://textbookfull.com/product/practical-tensorflow-js-deep-
learning-in-web-app-development-juan-de-dios-santos-rivera/

textboxfull.com

Python Deep Learning: Understand how deep neural networks

work and apply them to real-world tasks 3rd Edition
Vasilev
https://textbookfull.com/product/python-deep-learning-understand-how-
deep-neural-networks-work-and-apply-them-to-real-world-tasks-3rd-
edition-vasilev/
textboxfull.com

Artificial Neural Networks and Machine Learning ICANN 2019

Deep Learning 28th International Conference on Artificial
Neural Networks Munich Germany September 17 19 2019
Proceedings Part II Igor V. Tetko
https://textbookfull.com/product/artificial-neural-networks-and-
machine-learning-icann-2019-deep-learning-28th-international-
conference-on-artificial-neural-networks-munich-germany-
september-17-19-2019-proceedings-part-ii-igor-v-tet/
textboxfull.com
Neural networks in TensorFlow.js

Shanqing Cai
Stanley Bileschi
Eric D. Nielsen
François Chollet
Foreword by
Nikhil Thorat and Daniel Smilkov

MANNING
 Working with data

Ingest data Clean data Augment data

Data Sect. 6.1, 6.2, 6.3 Sect. 6.4 Sect. 6.5

Visualize data
Sect. 7.1

Model building 1: Choosing key layer types based on your data

Input data type Recommended layer API Reference

Numerical data Dense Chapters 2 and 3

(without sequential order)

Images or data that can be represented 2D convolution and pooling Chapters 4 and 5
as images (e.g., audio, game board)

Sequential data, including text • RNN (LSTM, GRU) • Sect. 9.1.2

• Embedding • Sect. 9.2.3
• 1D convolution • Sect. 9.2.4
• Attentional • Sect. 9.3

Model building 2: Choosing last-layer activation, loss, and metric functions

Task type Last-layer

(What are you predicting?) activation Loss function Metric(s) Reference

Regression Linear meanSquaredError (Same as loss) Chapter 2

(predicting a real number) meanAbsoluteError Sect. 9.1

Binary classification Sigmoid binaryCrossentropy Accuracy, precision, Sect. 3.1, 3.2, 9.2
(making a binary decision) recall, sensitivity, TPR,
FPR, ROC, AUC
Multi-class classification Softmax categoricalCrossentropy Accuracy, confusion Sect. 3.3, 9.3
(deciding among multiple matrix
classes)
A mix of the above (for example, (Multiple) Custom loss function (multiple) Sect. 5.2
numbers plus classes)

Advanced and miscellaneous task types Reference

Transfer learning Chapter 5

(applying a pretrained model to new data)

Generative learning Chapter 10

(generating new examples based on training data)

Reinforcement learning Chapter 11

(training an agent to interact with the environment)

Continues inside back cover

Deep Learning with JavaScript
 Deep Learning
with JavaScript
NEURAL NETWORKS IN TENSORFLOW.JS

SHANQING CAI
STANLEY BILESCHI
ERIC D. NIELSEN
WITH FRANÇOIS CHOLLET

FOREWORD BY NIKHIL THORAT

DANIEL SMILKOV

MANNING
SHELTER ISLAND
For online information and ordering of this and other Manning books, please visit
www.manning.com. The publisher offers discounts on this book when ordered in quantity.
For more information, please contact
Special Sales Department
Manning Publications Co.
20 Baldwin Road
PO Box 761
Shelter Island, NY 11964
Email: orders@manning.com

© 2020 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in

any form or by means electronic, mechanical, photocopying, or otherwise, without prior written
permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are
claimed as trademarks. Where those designations appear in the book, and Manning
Publications was aware of a trademark claim, the designations have been printed in initial caps
or all caps.

Recognizing the importance of preserving what has been written, it is Manning’s policy to have
the books we publish printed on acid-free paper, and we exert our best efforts to that end.
Recognizing also our responsibility to conserve the resources of our planet, Manning books
are printed on paper that is at least 15 percent recycled and processed without the use of
elemental chlorine.

Manning Publications Co. Development editor: Jenny Stout

20 Baldwin Road Technical development editor: Marc-Phillipe Huget
PO Box 761 Review editor: Ivan Martinovič
Shelter Island, NY 11964 Project editor: Lori Weidert
Copy editor: Rebecca Deuel-Gallegos
Proofreader: Jason Everett
Technical proofreader: Karsten Strøbæck
Typesetter: Dottie Marsico
Cover designer: Marija Tudor

ISBN 9781617296178
Printed in the United States of America
 brief contents
PART 1 MOTIVATION AND BASIC CONCEPTS . .................................1
1 ■ Deep learning and JavaScript 3

PART 2 A GENTLE INTRODUCTION TO TENSORFLOW.JS . ............. 35

2 ■ Getting started: Simple linear regression in TensorFlow.js 37
3 ■ Adding nonlinearity: Beyond weighted sums 79
4 ■ Recognizing images and sounds using convnets 117
5 ■ Transfer learning: Reusing pretrained neural networks 152

PART 3 ADVANCED DEEP LEARNING WITH TENSORFLOW.JS. ....... 199

6 ■ Working with data 201
7 ■ Visualizing data and models 246
8 ■ Underfitting, overfitting, and the universal workflow
of machine learning 273
9 ■ Deep learning for sequences and text 292
10 ■ Generative deep learning 334
11 ■ Basics of deep reinforcement learning 371

PART 4 SUMMARY AND CLOSING WORDS . .................................. 415

12 ■ Testing, optimizing, and deploying models 417
13 ■ Summary, conclusions, and beyond 453

v
 contents
foreword xiii
preface xv
acknowledgments xvii
about this book xix
about the authors xxii
about the cover illustration xxiii

PART 1 MOTIVATION AND BASIC CONCEPTS ....................1

1 Deep learning and JavaScript

1.1
3
Artificial intelligence, machine learning, neural networks,
and deep learning 6
Artificial intelligence 6 Machine learning: How it differs from
■

traditional programming 7 Neural networks and deep

■

learning 12 Why deep learning? Why now? 16

■

1.2 Why combine JavaScript and machine learning? 18

Deep learning with Node.js 24 ■
The JavaScript ecosystem 25
1.3 Why TensorFlow.js? 27
A brief history of TensorFlow, Keras, and TensorFlow.js 27 Why ■

TensorFlow.js: A brief comparison with similar libraries 31 How ■

is TensorFlow.js being used by the world? 31 What this book will

■

and will not teach you about TensorFlow.js 32

vii
viii CONTENTS

PART 2 A GENTLE INTRODUCTION TO

TENSORFLOW.JS ..............................................35

2 Getting started: Simple linear regression in TensorFlow.js

2.1 Example 1: Predicting the duration of a download using
37

TensorFlow.js 38
Project overview: Duration prediction 38 A note on code listings
■

and console interactions 39 Creating and formatting the

■

data 40 Defining a simple model 43 Fitting the model

■ ■

to the training data 46 Using our trained model to make

■

predictions 48 Summary of our first example 49

■

2.2 Inside Model.fit(): Dissecting gradient descent

from example 1 50
The intuitions behind gradient-descent optimization 50
Backpropagation: Inside gradient descent 56
2.3 Linear regression with multiple input features 59
The Boston Housing Prices dataset 60 Getting and running the
■

Boston-housing project from GitHub 61 Accessing the Boston-

■

housing data 63 Precisely defining the Boston-housing

■

problem 65 A slight diversion into data normalization 66

■

Linear regression on the Boston-housing data 70

2.4 How to interpret your model 74
Extracting meaning from learned weights 74 Extracting internal ■

weights from the model 75 Caveats on interpretability 77

■

3 Adding nonlinearity: Beyond weighted sums

3.1 Nonlinearity: What it is and what it is good for 80
79

Building the intuition for nonlinearity in neural networks 82

Hyperparameters and hyperparameter optimization 89
3.2 Nonlinearity at output: Models for classification 92
What is binary classification? 92 Measuring the quality of
■

binary classifiers: Precision, recall, accuracy, and ROC curves 96

The ROC curve: Showing trade-offs in binary classification 99
Binary cross entropy: The loss function for binary classification 103
3.3 Multiclass classification 106
One-hot encoding of categorical data 107 Softmax ■

activation 109 Categorical cross entropy: The loss function

■

for multiclass classification 111 Confusion matrix: Fine-grained

■

analysis of multiclass classification 113

 CONTENTS ix

4 Recognizing images and sounds using convnets 117

4.1 From vectors to tensors: Representing images 118
The MNIST dataset 119
4.2 Your first convnet 120
conv2d layer 122 maxPooling2d layer 126 Repeating
■ ■

motifs of convolution and pooling 127 Flatten and dense

■

layers 128 Training the convnet 130 Using a convnet to

■ ■

make predictions 134

4.3 Beyond browsers: Training models faster using
Node.js 137
Dependencies and imports for using tfjs-node 137 Saving the
■

model from Node.js and loading it in the browser 142

4.4 Spoken-word recognition: Applying convnets on
audio data 144
Spectrograms: Representing sounds as images 145

5 Transfer learning: Reusing pretrained neural networks 152

5.1 Introduction to transfer learning: Reusing pretrained
models 153
Transfer learning based on compatible output shapes: Freezing
layers 155 Transfer learning on incompatible output shapes:
■

Creating a new model using outputs from the base model 161
Getting the most out of transfer learning through fine-tuning: An
audio example 174
5.2 Object detection through transfer learning on a
convnet 185
A simple object-detection problem based on synthesized scenes 186
Deep dive into simple object detection 187

PART 3 ADVANCED DEEP LEARNING WITH

TENSORFLOW.JS . ...........................................199

6 Working with data 201

6.1 Using tf.data to manage data 202
The tf.data.Dataset object 203 Creating a tf.data.Dataset 203
■

Accessing the data in your dataset 209 Manipulating tfjs-data

■

datasets 210
6.2 Training models with model.fitDataset 214
x CONTENTS

6.3 Common patterns for accessing data 220

Working with CSV format data 220 Accessing video data using
■

tf.data.webcam() 225 Accessing audio data using

■

tf.data.microphone() 228
6.4 Your data is likely flawed: Dealing with problems
in your data 230
Theory of data 231 ■
Detecting and cleaning problems with
data 235
6.5 Data augmentation 242

7 Visualizing data and models 246

7.1 Data visualization 247
Visualizing data using tfjs-vis 247 ■
An integrative case study:
Visualizing weather data with tfjs-vis 255
7.2 Visualizing models after training 260
Visualizing the internal activations of a convnet 262
Visualizing what convolutional layers are sensitive to: Maximally
activating images 265 Visual interpretation of a convnet’s
■

classification result 269

8 Underfitting, overfitting, and the universal workflow

of machine learning 273
8.1 Formulation of the temperature-prediction problem 274
8.2 Underfitting, overfitting, and countermeasures 278
Underfitting 278 Overfitting 280 Reducing overfitting
■ ■

with weight regularization and visualizing it working 282

8.3 The universal workflow of machine learning 287

9 Deep learning for sequences and text 292

9.1 Second attempt at weather prediction:
Introducing RNNs 294
Why dense layers fail to model sequential order 294 ■
How RNNs
model sequential order 296
9.2 Building deep-learning models for text 305
How text is represented in machine learning: One-hot and multi-hot
encoding 306 First attempt at the sentiment-analysis
■

problem 308 A more efficient representation of text: Word

■

embeddings 310 1D convnets 312

■
 CONTENTS xi

9.3 Sequence-to-sequence tasks with attention

mechanism 321
Formulation of the sequence-to-sequence task 321 The encoder-
■

decoder architecture and the attention mechanism 324 Deep dive ■

into the attention-based encoder-decoder model 327

10 Generative deep learning 334

10.1 Generating text with LSTM 335
Next-character predictor: A simple way to generate text 335
The LSTM-text-generation example 337 Temperature:■

Adjustable randomness in the generated text 342

10.2 Variational autoencoders: Finding an efficient and
structured vector representation of images 345
Classical autoencoder and VAE: Basic ideas 345 A detailed ■

example of VAE: The Fashion-MNIST example 349

10.3 Image generation with GANs 356
The basic idea behind GANs 357 The building blocks of
■

ACGAN 360 Diving deeper into the training of ACGAN 363

■

Seeing the MNIST ACGAN training and generation 366

11 Basics of deep reinforcement learning 371

11.1 The formulation of reinforcement-learning
problems 373
11.2 Policy networks and policy gradients: The cart-pole
example 376
Cart-pole as a reinforcement-learning problem 376 Policy ■

network 378 Training the policy network: The REINFORCE

■

algorithm 381
11.3 Value networks and Q-learning: The snake game
example 389
Snake as a reinforcement-learning problem 389 Markov decision
■

process and Q-values 392 Deep Q-network 396 Training

■ ■

the deep Q-network 399

PART 4 SUMMARY AND CLOSING WORDS .....................415

12 Testing, optimizing, and deploying models

12.1 Testing TensorFlow.js models 418
417

Traditional unit testing 419 Testing with golden values

■
422
Considerations around continuous training 424
xii CONTENTS

12.2 Model optimization 425

Model-size optimization through post-training weight
quantization 426 Inference-speed optimization using
■

GraphModel conversion 434

12.3 Deploying TensorFlow.js models on various platforms
and environments 439
Additional considerations when deploying to the web 439
Deployment to cloud serving 440 Deploying to a browser
■

extension, like Chrome Extension 441 Deploying TensorFlow.js

■

models in JavaScript-based mobile applications 443 Deploying

■

TensorFlow.js models in JavaScript-based cross-platform desktop

applications 445 Deploying TensorFlow.js models on WeChat
■

and other JavaScript-based mobile app plugin systems 447

Deploying TensorFlow.js models on single-board computers 448
Summary of deployments 450

13 Summary, conclusions, and beyond 453

13.1 Key concepts in review 454
Various approaches to AI 454 What makes deep learning stand
■

out among the subfields of machine learning 455 How to think

■

about deep learning at a high level 455 Key enabling

■

technologies of deep learning 456 Applications and

■

opportunities unlocked by deep learning in JavaScript 457

13.2 Quick overview of the deep-learning workflow
and algorithms in TensorFlow.js 458
The universal workflow of supervised deep learning 458
Reviewing model and layer types in TensorFlow.js: A quick
reference 460 Using pretrained models from TensorFlow.js
■
465
The space of possibilities 468 Limitations of deep learning
■
470
13.3 Trends in deep learning 473
13.4 Pointers for further exploration 474
Practice real-world machine-learning problems on Kaggle 474
Read about the latest developments on arXiv 475 Explore the
■

TensorFlow.js Ecosystem 475

appendix A Installing tfjs-node-gpu and its dependencies 477

appendix B A quick tutorial of tensors and operations in TensorFlow.js 482
glossary 507
index 519
 foreword
When we started TensorFlow.js (TF.js), formerly called deeplearn.js, machine learning
(ML) was done mostly in Python. As both JavaScript developers and ML practitioners
on the Google Brain team, we quickly realized that there was an opportunity to bridge
the two worlds. Today, TF.js has empowered a new set of developers from the extensive
JavaScript community to build and deploy ML models and enabled new classes of on-
device computation.
TF.js would not exist in its form today without Shanqing, Stan, and Eric. Their con-
tributions to TensorFlow Python, including the TensorFlow Debugger, eager execu-
tion, and build and test infrastructure, uniquely positioned them to tie the Python
and JavaScript worlds together. Early on in the development, their team realized the
need for a library on top of deeplearn.js that would provide high-level building blocks
to develop ML models. Shanqing, Stan, and Eric, among others, built TF.js Layers,
allowing conversion of Keras models to JavaScript, which dramatically increased the
wealth of available models in the TF.js ecosystem. When TF.js Layers was ready, we
released TF.js to the world.
To investigate the motivations, hurdles, and desires of software developers, Carrie
Cai and Philip Guo deployed a survey to the TF.js website. This book is in direct
response to the study’s summary: “Our analysis found that developers’ desires for ML
frameworks extended beyond simply wanting help with APIs: more fundamentally,
they desired guidance on understanding and applying the conceptual underpinnings
of ML itself.”1
Deep Learning with JavaScript contains a mix of deep learning theory as well as real-
world examples in JavaScript with TF.js. It is a great resource for JavaScript developers

1
C. Cai and P. Guo, (2019) “Software Developers Learning Machine Learning: Motivations, Hurdles, and
Desires,” IEEE Symposium on Visual Languages and Human-Centric Computing, 2019.

xiii
xiv FOREWORD

with no ML experience or formal math background, as well as ML practitioners who

would like to extend their work into the JavaScript ecosystem. This book follows the
template of Deep Learning with Python, one of the most popular applied-ML texts, writ-
ten by the Keras creator, François Chollet. Expanding on Chollet’s work, Deep Learning
with JavaScript does an amazing job building on the unique things that JavaScript has
to offer: interactivity, portability, and on-device computation. It covers core ML con-
cepts, but does not shy away from state-of-the-art ML topics, such as text translation,
generative models, and reinforcement learning. It even gives pragmatic advice on
deploying ML models into real-world applications written by practitioners who have
extensive experience deploying ML to the real world. The examples in this book are
backed by interactive demos that demonstrate the unique advantages of the JavaScript
ecosystem. All the code is open-sourced, so you can interact with it and fork it online.
This book should serve as the authoritative source for readers who want to learn
ML and use JavaScript as their main language. Sitting at the forefront of ML and
JavaScript, we hope you find the concepts in this book useful and the journey in Java-
Script ML a fruitful and exciting one.
—NIKHIL THORAT AND DANIEL SMILKOV,
inventors of deeplearn.js
and technical leads of TensorFlow.js
 preface
The most significant event in the recent history of technology is perhaps the explo-
sion in the power of neural networks since 2012. This was when the growth in labeled
datasets, increases in computation power, and innovations in algorithms came
together and reached a critical mass. Since then, deep neural networks have made
previously unachievable tasks achievable and boosted the accuracies in other tasks,
pushing them beyond academic research and into practical applications in domains
such as speech recognition, image labeling, generative models, and recommendation
systems, just to name a few.
It was against this backdrop that our team at Google Brain started developing
TensorFlow.js. When the project started, many regarded “deep learning in JavaScript”
as a novelty, perhaps a gimmick, fun for certain use cases, but not to be pursued with
seriousness. While Python already had several well-established and powerful frame-
works for deep learning, the JavaScript machine-learning landscape remained splin-
tered and incomplete. Of the handful of JavaScript libraries available back then, most
only supported deploying models pretrained in other languages (usually in Python).
For the few that supported building and training models from scratch, the scope of
supported model types was limited. Considering JavaScript’s popular status and its
ubiquity that straddles client and server sides, this was a strange situation.
TensorFlow.js is the first full-fledged industry-quality library for doing neural net-
works in JavaScript. The range of capabilities it provides spans multiple dimensions.
First, it supports a wide range of neural-networks layers, suitable for various data types
ranging from numeric to text, from audio to images. Second, it provides APIs for load-
ing pretrained models for inference, fine-tuning pretrained models, and building and
training models from scratch. Third, it provides both a high-level, Keras-like API for
practitioners who opt to use well-established layer types, and a low-level, TensorFlow-
like API for those who wish to implement more novel algorithms. Finally, it is designed

xv
xvi PREFACE

to be runnable in a wide selection of environments and hardware types, including the

web browser, server side (Node.js), mobile (e.g., React Native and WeChat), and desk-
top (electron). Adding to the multidimensional capability of TensorFlow.js is its status
as a first-class integrated part of the larger TensorFlow/Keras ecosystem, specifically its
API consistency and two-way model-format compatibility with the Python libraries.
The book you have in your hands will guide your grand tour through this multi-
dimensional space of capabilities. We’ve chosen a path that primarily cuts through the
first dimension (modeling tasks), enriched by excursions along the remaining dimen-
sions. We start from the relatively simpler task of predicting numbers from numbers
(regression) to the more complex ones such as predicting classes from images and
sequences, ending our trip on the fascinating topics of using neural networks to gen-
erate new images and training agents to make decisions (reinforcement learning).
We wrote the book not just as a recipe for how to write code in TensorFlow.js, but
as an introductory course in the foundations of machine learning in the native lan-
guage of JavaScript and web developers. The field of deep learning is a fast-evolving
one. It is our belief that a firm understanding of machine learning is possible without
formal mathematical treatment, and this understanding will enable you to keep your-
self up-to-date in future evolution of the techniques.
With this book you’ve made the first step in becoming a member of the growing
community of JavaScript machine-learning practitioners, who’ve already brought
about many impactful applications at the intersection between JavaScript and deep
learning. It is our sincere hope that this book will kindle your own creativity and inge-
nuity in this space.
SHANQING CAI, STAN BILESCHI, AND ERIC NIELSEN
September 2019
Cambridge, MA
 acknowledgments
This book owes Deep Learning with Python by François Chollet for its overall structure.
Despite the fact that the code was rewritten in a different language and much new
content was added for the JavaScript ecosystem and to reflect new developments in
the field, neither this book nor the entire high-level API of TensorFlow.js would have
been a reality without pioneer work on Keras led by François.
Our journey to the completion of this book and all the related code was made
pleasant and fulfilling thanks to the incredible support from our colleagues on Goo-
gle’s TensorFlow.js Team. The seminal and foundational work by Daniel Smilkov and
Nikhil Thorat on the low-level WebGL kernels and backpropagation forms a rock-
solid foundation for model building and training. The work by Nick Kreeger on the
Node.js binding to TensorFlow’s C library is the main reason why we can run neural
networks in the browser and Node.js with the same code. The TensorFlow.js data API
by David Soergel and Kangyi Zhang makes chapter 6 of the book possible, while chap-
ter 7 was enabled by the visualization work by Yannick Assogba. The performance
optimization techniques described in chapter 11 wouldn’t be possible without Ping
Yu’s work on op-level interface with TensorFlow. The speed of our examples wouldn’t
be nearly as fast as it is today without the focused performance optimization work by
Ann Yuan. The leadership of Sarah Sirajuddin, Sandeep Gupta, and Brijesh Krishnas-
wami is critical to the overall long-term success of the TensorFlow.js project.
We would have fallen off the track without the support and encouragement of
D. Sculley, who carefully reviewed all the chapters of the book. We’re also immensely
grateful for all the encouragement we received from Fernanda Viegas, Martin Watten-
berg, Hal Abelson, and many other colleagues of ours at Google. Our writing and
content were greatly improved as a result of the detailed review by François Chollet,

xvii
xviii ACKNOWLEDGMENTS

Nikhil Thorat, Daniel Smilkov, Jamie Smith, Brian K. Lee, and Augustus Odena, as
well as by in-depth discussion with Suharsh Sivakumar.
One of the unique pleasures of working on a project such as TensorFlow.js is the
opportunity to work alongside and interact with the worldwide open-source software
community. TensorFlow.js was fortunate to have a group of talented and driven con-
tributors including Manraj Singh, Kai Sasaki, Josh Gartman, Sasha Illarionov, David
Sanders, syt123450@, and many many others, whose tireless work on the library
expanded its capability and improved its quality. Manraj Singh also contributed the
phishing-detection example used in chapter 3 of the book.
We are grateful to our editorial team at Manning Publications. The dedicated and
tireless work by Brian Sawyer, Jennifer Stout, Rebecca Rinehart, and Mehmed Pasic,
and many others made it possible for we authors to focus on writing the content.
Marc-Philip Huget provided extensive and incisive technical review throughout the
development process. Special thanks go to our reviewers, Alain Lompo, Andreas Refs-
gaard, Buu Nguyen, David DiMaria, Edin Kapic, Edwin Kwok, Eoghan O’Donnell,
Evan Wallace, George thomas, Giuliano Bertoti, Jason Hales, Marcio Nicolau, Michael
Wall, Paulo Nuin, Pietro Maffi, Polina Keselman, Prabhuti Prakash, Ryan Burrows,
Satej Sahu, Suresh Rangarajulu, Ursin Stauss, and Vaijanath Rao, whose suggestions
helped make this a better book.
We thank our MEAP readers for catching and pointing out quite a few typographi-
cal and technical errors.
Finally, none of this would be possible without the tremendous understanding and
sacrifice on the part of our families. Shanqing Cai would like to express the deepest
gratitude to his wife, Wei, as well as his parents and parents-in-law for their help and
support during this book’s year-long writing process. Stan Bileschi would like to thank
his mother and father, as well as his step-mother and step-father, for providing a foun-
dation and direction to build a successful career in science and engineering. He
would also like to thank his wife, Constance, for her love and support. Eric Nielsen
would like to say to his friends and family, thank you.
 about this book
Who should read this book
This book is written for programmers who have a working knowledge of JavaScript,
from prior experience with either web frontend development or Node.js-based back-
end development, and wish to venture into the world of deep learning. It aims to sat-
isfy the learning needs of the following two subgroups of readers:
 JavaScript programmers who aspire to go from little-to-no experience with
machine learning or its mathematical background, to a decent knowledge of
how deep learning works and a practical understanding of the deep-learning
workflow that is sufficient for solving common data-science problems such as
classification and regression
 Web or Node.js developers who are tasked with deploying pre-trained models in
their web app or backend stack as new features
For the first group of readers, this book develops the basic concepts of machine learn-
ing and deep learning in a ground-up fashion, using JavaScript code examples that
are fun and ready for fiddling and hacking. We use diagrams, pseudo-code, and con-
crete examples in lieu of formal mathematics to help you form an intuitive, yet firm,
grasp of the foundations of how deep learning works.
For the second group of readers, we cover the key steps of converting existing
models (e.g., from Python training libraries) into a web- and/or Node-compatible for-
mat suitable for deployment in the frontend or the Node stack. We emphasize practi-
cal aspects such as optimizing model size and performance, as well as considerations
for various deployment environments ranging from a server to browser extensions
and mobile apps.

xix
xx ABOUT THIS BOOK

This book provides in-depth coverage of the TensorFlow.js API for ingesting and
formatting data, for building and loading models, and for running inference, evalua-
tion, and training for all readers.
Finally, technically minded people who don’t code regularly in JavaScript or any
other language will also find this book useful as an introductory text for both basic
and advanced neural networks.

How this book is organized: A roadmap

This book is organized into four parts. The first part, consisting of chapter 1 only,
introduces you to the landscape of artificial intelligence, machine learning, and deep
learning, and why it makes sense to practice deep learning in JavaScript.
The second part forms a gentle introduction to the most foundational and
frequently encountered concepts in deep learning. In particular:
 Chapters 2 and 3 are your gentle on-ramp to machine learning. Chapter 2
works through a simple problem of predicting a single number from another
number by fitting a straight line (linear regression) and uses it to illustrate how
backpropagation (the engine of deep learning) works. Chapter 3 builds on
chapter 2 by introducing nonlinearity, multi-layered networks, and classification
tasks. From this chapter you will gain an understanding of what nonlinearity is,
how it works, and why it gives deep neural networks their expressive power.
 Chapter 4 deals with image data and the neural-network architecture dedicated
to solving image-related machine-learning problems: convolutional networks
(convnets). We will also show you why convolution is a generic method that has
uses beyond images by using audio inputs as an example.
 Chapter 5 continues the focus on convnets and image-like inputs, but shifts into
the topic of transfer learning: how to train new models based on existing ones,
instead of starting from scratch.
Part 3 of the book systematically covers more advanced topics in deep learning for
users who wish to build an understanding of more cutting-edge techniques, with a
focus on specific challenging areas of ML systems, and the TensorFlow.js tools to work
with them:
 Chapter 6 discusses techniques for dealing with data in the context of deep
learning.
 Chapter 7 shows the techniques for visualizing data and the models that process
them, an important and indispensable step for any deep-learning workflow.
 Chapter 8 focuses on the important topics of underfitting and overfitting in
deep learning, and techniques for analyzing and mitigating them. Through this
discussion, we condense what we’ve learned in this book so far into a recipe
referred to as “the universal workflow of machine learning.” This chapter pre-
pares you for the advanced neural-network architectures and problems in chap-
ters 9–11.
 ABOUT THIS BOOK xxi

 Chapter 9 is dedicated to deep neural networks that process sequential data

and text inputs.
 Chapters 10 and 11 cover the advanced deep-learning areas of generative mod-
els (including generative adversarial networks) and reinforcement learning,
respectively.
In the fourth and final part of the book, we cover techniques for testing, optimizing
and deploying models trained or converted with TensorFlow.js (chapter 12) and wrap
up the whole book by recapitulating the most important concepts and workflows
(chapter 13).
Each chapter finishes with exercises to help you gauge your level of understanding
and hone your deep-learning skills in TensorFlow.js in a hands-on fashion.

About the code

This book contains many examples of source code both in numbered listings and in
line with normal text. In both cases, source code is formatted in a fixed-width font
like this to separate it from ordinary text. Sometimes code is also in bold to high-
light code that has changed from previous steps in the chapter, such as when a new
feature adds to an existing line of code.
In many cases, the original source code has been reformatted; we’ve added line
breaks and reworked indentation to accommodate the available page space in the
book. In rare cases, even this was not enough, and listings include line-continuation
markers (➥). Additionally, comments in the source code have often been removed
from the listings when the code is described in the text. Code annotations accom-
pany many of the listings, highlighting important concepts. The code for the exam-
ples in this book is available for download from GitHub at https:/ /github.com/
tensorflow/tfjs-examples.

liveBook discussion forum

Purchase of Deep Learning with JavaScript includes free access to a private web forum
run by Manning Publications where you can make comments about the book, ask
technical questions, and receive help from the author and from other users. To
access the forum, go to https://livebook.manning.com/#!/book/deep-learning-with-
javascript/discussion. You can also learn more about Manning’s forums and the rules
of conduct at https://livebook.manning.com/#!/discussion.
Manning’s commitment to our readers is to provide a venue where a meaningful
dialogue between individual readers and between readers and the author can take
place. It is not a commitment to any specific amount of participation on the part of
the author, whose contribution to the forum remains voluntary (and unpaid). We sug-
gest you try asking the authors some challenging questions lest their interest stray!
The forum and the archives of previous discussions will be accessible from the pub-
lisher’s website as long as the book is in print.
 about the authors
SHANQING CAI, STANLEY BILESCHI, AND ERIC NIELSEN are software engineers on the
Google Brain team. They were the primary developers of the high-level API of Tensor-
Flow.js, including the examples, the documentation, and the related tooling. They
have applied TensorFlow.js-based deep learning to real-world problems such as alter-
native communication for people with disabilities. They each have advanced degrees
from MIT.

xxii
 about the cover illustration
The figure on the cover of Deep Learning with JavaScript is captioned “Finne Katschin,”
or a girl from the Katschin tribe. The illustration is taken from a collection of dress
costumes from various countries by Jacques Grasset de Saint-Sauveur (1757-1810),
titled Costumes de Différents Pays, published in France in 1797. Each illustration is finely
drawn and colored by hand. The rich variety of Grasset de Saint-Sauveur’s collection
reminds us vividly of how culturally apart the world’s towns and regions were just 200
years ago. Isolated from each other, people spoke different dialects and languages. In
the streets or in the countryside, it was easy to identify where they lived and what their
trade or station in life was just by their dress.
The way we dress has changed since then and the diversity by region, so rich at the
time, has faded away. It is now hard to tell apart the inhabitants of different conti-
nents, let alone different towns, regions, or countries. Perhaps we have traded cultural
diversity for a more varied personal life—certainly for a more varied and fast-paced
technological life.
At a time when it is hard to tell one computer book from another, Manning cele-
brates the inventiveness and initiative of the computer business with book covers
based on the rich diversity of regional life of two centuries ago, brought back to life by
Grasset de Saint-Sauveur’s pictures.

xxiii
 Part 1
Motivation
and basic concepts

P art 1 consists of a single chapter that orients you to the basic concepts that
will form the backdrop for the rest of the book. These include artificial intelli-
gence, machine learning, and deep learning and the relations between them.
Chapter 1 also addresses the value and potential of practicing deep learning in
JavaScript.
 Deep learning
and JavaScript

This chapter covers

 What deep learning is and how it is related to artificial
intelligence (AI) and machine learning
 What makes deep learning stand out among various
machine-learning techniques, and the factors that led to the
current “deep-learning revolution”
 The reasons for doing deep learning in JavaScript using
TensorFlow.js
 The overall organization of this book

All the buzz around artificial intelligence (AI) is happening for a good reason: the
deep-learning revolution, as it is sometimes called, has indeed happened. Deep-
learning revolution refers to the rapid progress made in the speed and techniques of
deep neural networks that started around 2012 and is still ongoing. Since then,
deep neural networks have been applied to an increasingly wide range of prob-
lems, enabling machines to solve previously unsolvable problems in some cases and
dramatically improving solution accuracy in others (see table 1.1 for examples). To
experts in AI, many of these breakthroughs in neural networks were stunning.

3
Discovering Diverse Content Through
Random Scribd Documents
and Lawgiver of his church, hath a visible kingdom which he
exerciseth in and over the church visible by its spiritual office-
bearers, given to it as a church, and therefore distinct from, and
independent upon, the civil power—the keys of the kingdom of
heaven being by him committed, not to the magistrates, but to the
apostles’ successors in the work of the ministry.” He therefore
quitted the Established Church, betook himself to the fields, and
shared in the labours and obloquy of the persecuted. Mr Burnet was
prevented by sickness from personally bearing witness to the same
high prerogatives of Christ; but he left his reasons for refusing to
submit to any temporal supremacy in writing, and died rejoicing in
the hope of the glory of God. His last words were—“Glory! glory!
glory!”
It deserves to be remarked, that he and several other
distinguished ministers, although they had no liberty to accept of the
indulgence themselves, yet they did not deem it a reason why they
should withdraw their affection from those who had, or throw any
obstacles in the way of those they considered messengers of the
gospel; for these worthies thought preaching salvation to sinners so
paramount a duty, that they would have ventured upon every thing
but sin to have achieved it themselves or promoted it by others.
Charles and his advisers in attempting to introduce despotism, had
as little consulted their own peace as that of his kingdom. He was
harassed by his English parliament; and Lauderdale having been
voted a public grievance, was glad to seek refuge in Scotland,
where, in the month of December, he came down to hold a fourth
session of the parliament. Suspecting no opposition, if he secured
the support of the clergy, he told the estates that the most effectual
course would be taken for curbing and suppressing the insolent field
conventicles, and other seditious practices, which had so much
abounded—that if fairness would not, force must compel the
refractory to be peaceable and to obey the laws. But instead of his
declarations being met with the submissive adulation they were
wont, the Duke of Hamilton, supported by a strong party, presented
their grievances; and when the Commissioner with his usual haughty
roughness interposed to silence complaint, Sir Patrick Home of
Polwart demanded to know, whether it was not a free parliament?
And after a short tumultuous session, in which, amid the dissensions
of the statesmen, the Presbyterians escaped for the time any severer
enactment, the meeting was adjourned, and the parties sought each
to justify the strife to the king. Hamilton repaired to London and laid
a statement of the enormous abuses before his majesty, but only
received fair promises that were never performed, and incurred a
resentment that never was appeased. Lauderdale retained his
situation, and rather increased in favour with the king.
[1674.] It is a melancholy and an appalling consideration for those
who stand forward as reformers and patriots, that, in struggles for
religion, for liberty, or for any good principle, those who sincerely
strive to gain such objects are usually found in a minority at last;
and when they have been the means of conferring the most
essential benefits upon the country, they are generally left losers
themselves. Amid the conflicts of the statesmen, and their loud
complaints about the oppression and ruin of the country, no mention
had been made of the primary and most palpable of all its
distresses, the religious grievances of the Presbyterians:—those
which in fact had been the origin of all the calamities of Scotland,
and the triumph of which was to secure the cause of freedom, were
utterly lost sight of in their miserable squabbling about the monopoly
of salt and the smuggling of brandy.
Both Hamilton and Lauderdale were supposed friendly to the
persecuted; and while the nation was convulsed with their political
contentions, and their attention was sufficiently employed
elsewhere, the pious, resolute, and consistent part of the persecuted
ministers improved the respite for proclaiming peace upon the
mountains, bringing good tidings of good, publishing salvation, and
saying to Zion, “Thy God reigneth!” Conventicles increased both in
number and frequency. They began early in the year, and the
indefatigable Mr Blackadder beat up the primate’s quarters upon the
2d day of January.[76] On that day he collected at Kinkel, within a
mile of St Andrews, a large auditory, which filled the long gallery and
two chambers, besides a great number standing without doors. He
lectured on the second Psalm, a portion of Scripture remarkably
applicable, and preached from Jer. xiii. 18. The primate’s wife
hearing of the assembly, sent for the militia, who were fully prepared
in warlike array, under a Lieutenant Doig, accompanied by a great
number of the rascality, with many of the worst set of scholars from
the college and some noblemen’s sons. They drew up at a distance
from the gate, before which stood the laird, his brother, and the
minister’s eldest son; but they caused no interruption till the lecture
was finished and the psalm sung, when some people called out that
there was an alarm; on which the service stopped and the men
ranged outside the gate with the laird. Meanwhile, some of the
rabble had got into the stable and were carrying off the laird’s horse,
which he observing, aimed a blow at the fellow who had him; but
some of the “ill-set schollars” laying hold on his cane, a struggle
ensued, and the laird fell. Mr Welsh, who was also there, and
Kinkel’s brother, instantly drew; and the Lieutenant and his men
seeing them so resolute, and supposing that they were well
supported, fell back, nor dared approach sufficiently near the gate to
discover their error. Mrs Murray then went up to the Lieutenant and
asked him why he came in that hostile manner to trouble their house
on the Lord’s day? He said he had an order, which she requesting to
see, he told her he would show it to the laird; and, attended by a
sergeant, was drawing near the gate, when Mr Murray called, as he
approached—“How is it, Lieutenant, that you come to disturb us on
the Sabbath day?” In great trepidation he delivered the laird an
order which had been subscribed by the Chancellor about a year
before for apprehending him and his brother. When Kinkel had read
it, “I see,” said he, “you have an old order from the Chancellor to
that effect, which was extorted from him by the prelate. If you mind
to execute it now, you may, but you shall see the faces of men.” The
Lieutenant, grievously alarmed, cursed himself if he had a mind to
execute it. After which, the lady caused bring forth some ale for the
Lieutenant and his men; but one of them, whose companion had
been a little hurt, said he would drink none of her drink; he would
rather drink her heart’s blood. The rest partook of the refreshments
and went away. Composure being restored, the minister proceeded
with his sermon, and the whole closed in peace.
76. About the same time, the precise date is uncertain, Crail, where Sharpe had
been a Presbyterian minister, was visited by Mr John Dickson; and the
unhappy apostate was tormented by the sound of the gospel on his right
hand and on his left, while he vainly strove by military force to destroy the
faith which once he preached.

Some time after this, Mr Blackadder had another meeting at

Kinkel, where vast numbers from St Andrews attended as hearers,
and even some of the militia. Sharpe, who was that Sabbath day at
home, hearing of it, sent for the provost and commanded him to
order out the military, disperse the conventicle, and apprehend the
minister. “My lord,” replied the provost, to the prelate’s dismay, “the
militia are gone there already to hear the preaching, and we have
none to send.” And among them was the soldier that had refused
drink from Lady Kinkel, who was especially marked to be moved and
wept beyond the rest; so wonderfully did the Lord countenance the
persecuted gospel, even bloody enemies being overcome with
conviction.
Exasperated at the multiplication of these meetings, the
Episcopalian clergy added the foulest and the falsest calumnies to
their other modes of opposition, and the synod of Glasgow, October
22, had the unblushing effrontery to charge these assemblies with
crimes of which they themselves could never have believed them
guilty—“incest, bestiality, murder of children, besides frequent
adulteries, and other acts of wickedness after which, it is little that
they should have been accused of fanaticism, disloyalty, and cursing
the king. Towards the end of March, before Lauderdale left Scotland,
he published an indemnity, although like many others with which the
nation was insulted during this reign, almost only so in name, was
received by the people as a license for frequenting conventicles,
which continued to multiply in consequence, and especially as a
report was assiduously circulated of his having secretly promised
that an ample liberty would be granted to Presbyterian ministers
soon after his arrival at court. Few were held in the west where the
indulged ministers were settled, but on the borders, in the Merse,
Lothians, Stirlingshire, and Fife, they greatly abounded, in houses,
fields, and vacant churches. The more private worshippers in houses
were overlooked, the vast assemblages in the mountains, and
mosses, and muirs chiefly attracting the attention of government;
and “at these great meetings,” says Kirkton, “many a soul was
converted to Jesus Christ, but far more turned from the bishops to
profess themselves Presbyterians.”
Mr Welsh was among the most diligent and successful of the
labourers, particularly in Fife, where many thousands were wont to
assemble. His preaching was attended with a visible blessing in the
conversion of many to the Lord; and among them were some in the
higher ranks, especially ladies; for it is somewhat remarkable that in
these days of peril and danger, the weaker sex were distinguished
for their intrepid zeal; and there is reason to believe that not a few,
conspicuous for their piety, were brought to the obedience of faith at
these assemblies. The Countess of Crawford, daughter of the Earl of
Annandale, was one of the number, and dated her first impressions
from a sermon preached by him at Duraquhair, near Cupar, where
about eight thousand persons were present, and the power of God
was manifested to the checking of the conscience and the
awakening of the hearts of many. On the same Sabbath three other
conventicles were held, and it was computed not less than sixteen
thousand persons heard the gospel plainly and earnestly preached
by Mr Robert Lockhart at Path-head, near Kirkcaldy; Mr Blackadder,
near Dunfermline; and Mr Welwood on the Lomond Hills. This last
meeting was fired upon by the soldiers, but although their bullets
lighted among a crowd of men, women, and children, and brake the
ground beside them, not one was wounded. They, however, took
about eighteen prisoners, and then marched for Duraquhair to attack
Mr Welsh; but the people got notice and hurried him away, a great
body escorting him as far as Largo, where they procured a boat, and
he and his wife, with some others, crossed the frith under night
safely, and landed at Aberlady Bay, whence he got undiscovered to
Edinburgh. Even the capital itself and the neighbourhood were sorely
infested with these noxious meetings. Kirkton had long had regular
house-preaching in the city, but this year, emboldened like others by
the expectation of favour, he, along with Mr Johnston, again
ventured upon sacred ground, and Cramond Kirk being vacant, they
bad both been repeatedly guilty of declaring the truth from that
pulpit to large and attentive auditories.
Against these there were many grievous complaints by the
prelates, of which Lauderdale took advantage to lower the credit of
the Duke of Hamilton and his party with the king, and in this he was
so successful, that, about the end of May, the privy council was re-
modelled, and those only who were entirely devoted to his interest
permitted to remain. On the 4th of June, when they first assembled,
they were assailed in rather an unusual manner.
Reports of increased severities being about to be resorted to
against conventicles having reached Edinburgh, as men durst not
appear with any petition under pain of being fined or imprisoned,
fifteen women, chiefly ministers widows, resolved to present as
many copies of “a humble supplication for liberty to the honest
ministers throughout the land to exercise their holy function without
molestation,” to fifteen of the principal lords of council. Attended by
a crowd of females, who filled the Parliament Close, they awaited
the arrival of the councillors. Sharpe came along with the Chancellor,
and when he saw the ladies, in great bodily fear he kept close by his
lordship, who seemed to enjoy the primate’s terrors, and
complacently allowed Mr John Livingstone’s widow to accompany
him to the Council-Chamber door, conversing as they went along,
while others very unceremoniously saluted Sharpe with the epithets
of Judas and traitor; and one of them more forward than the rest,
laid her hand upon his neck, and told him “that neck behoved to pay
for it before all was done.” The whole of the lords to whom the
papers were presented, received them civilly, except Stair, who
threw his scornfully upon the ground, which drew upon him a
sarcastic remark—“that he had not so treated the remonstrance
against the king which he helped to pen.” When the council met, the
petition was voted a libel, and about a dozen of the subscribers were
called and examined. They declared severally that no man had had
any hand in the matter, and that their sole motive was a sense of
their perishing condition for want of the gospel, having no preachers
except ignorant and profane persons whom they could not hear;
upon which they were ordered into confinement, and the Lord
Provost and the guard sent to disperse the ladies at the door; but
they refused to depart without their representatives, who were in
consequence politely liberated, and the tumult ended. Next day,
however, they were again summoned, when three were sent to
prison—Margaret, a daughter of Lord Warriston’s; a Mrs Cleland; and
a Lilias Campbell. The former, with Lady Mersington and some
others, were banished the town and liberties of Edinburgh; and so
ended this affair.
The fears of the ladies were not unfounded. A letter from the king
to the council was read at the same meeting, requiring them “to use
their utmost endeavours for apprehending preachers at field
conventicles, invaders of pulpits, and ringleading heritors, and to
make use of the militia and standing force for that end, leaving the
punishment of the other transgressors to the ordinary magistrates
according to law.” In obedience to which, a committee was
appointed with full powers to meet when and where they should
think convenient, to make the necessary inquiries, apprehend whom
they should think proper, and the standing force and militia were
placed under their immediate direction. At the head was the
Archbishop of St Andrews, the Lord Chancellor, and other servants of
the crown, assisted by the Earls of Argyle, Linlithgow, Kinghorn,
Wigton, and Dundonald. The Duke of Hamilton was named; but in
present circumstances possessed little power, and seldom attended.
Orders were at the same time issued for apprehending the following
ministers:—John Welsh, Gabriel Semple, Robert Ross, Samuel Arnot,
Gabriel Cunningham, Archibald Riddel, John Mosman, John
Blackadder, William Wiseheart, David Hume, John Dickson, John
Rae, Henry Forsyth, Thomas Hogg, Robert Law, George Johnstone,
Thomas Forrester, Fraser of Brea, John Law, Robert Gillespie. And to
encourage the parties sent out on this duty, for the two first, as the
most notorious offenders, a reward of four hundred pounds sterling
each was offered; for the others, one thousand merks; and the
soldiers and others who might assist in their seizure, were previously
pardoned for any bloodshed that might occur—such was the
inveteracy the rulers of Scotland betrayed against men whose only
crime was preaching the gospel. They then proceeded to show
nearly equal abhorrence for those who heard it, by punishing with
fines or imprisonment the most obstinate of the heritors. The town
of Edinburgh was amerced in one hundred pounds sterling for
conventicles in the Magdalene Chapel, to be exacted from the chief
citizens present; Mr John Inglis of Cramond, for hearing sermon six
times in his parish church, a thousand and thirty-six pounds Scots; a
gentleman in Fife, for allowing Mr Welsh to lodge in his house one
night, was fined two thousand merks; and eleven heritors, upwards
of five thousand five hundred for attending field-preachings—all
which monies were ordered to be summarily levied, and the
offenders kept in prison till the same should be paid. Nor were
persons, even of high rank, and against whom no charges of very
intrusive piety are known to have existed, exempt from being
harassed by any vile, petty, clerical informer. Lord Balmerino and Sir
John Young of Leny, neither of whom had been present at any such
preaching, were brought before the council; and when they denied
the fact, were insultingly tendered the oath of allegiance, which both
must have already repeatedly sworn, before they were dismissed.
Two rigorous proclamations followed. By the first, all masters were
required to prevent their servants from being present at any house
or field conventicle, and to retain none in their employment for
whose conduct they would not be answerable; heritors were ordered
to require their tenants to subscribe a bond, obliging themselves,
wives, cottars, or servants, to abstain from all such meetings, which,
if they refused, they were to be put to the horn, and their escheat
given to their landlords; but masters and landlords were responsible
for the conduct of their inferiors to the extent of the fines their
disobedience might incur; and all magistrates were empowered to
oblige such as they chose to suspect, to give bond for their good
behaviour. The second was directed against ministers, in terms of
the orders already issued for their apprehension. Still further to
stimulate the magistrates, another letter was procured from the
king, informing them that his majesty had heard of the alarming
increase of conventicles, for repressing which, together with the
other seditious movements in Scotland, he had ordered his troops in
Ireland and at Berwick to hold themselves in readiness, to march on
the first alarm; and, in the mean time, required them to bring to
punishment the authors of these insolent and seditious practices.
But the difficulty of obtaining proof forming some small impediment
in the way of conviction, the council therefore proposed that, when a
suspected person was apprehended, against whom they had not
sufficient evidence, he should be interrogated to answer upon oath,
and if he refused to answer, he should be held as confessed, and
proceeded against accordingly, only the punishment should be
restricted to fining, imprisonment, exile, or the loss of a limb—most
merciful judges!—to which his majesty was graciously pleased to
consent, and the council proceeded to act.
They summoned a number of the “outted” ministers to appear, not
in the usual mode by leaving written copies at their dwelling-places,
but at the market-crosses of Edinburgh, Lanark, Stirling, and Perth,
and that within such a time, that, had they been willing, they could
not have complied. As the latter knew, however, that if they
appeared, they were certain of being sent either to the Bass or into
banishment, they declined, and were in consequence denounced as
rebels.[77] When the council rose, on the last day of July, they
reported to the secretary, that forty “outted” ministers had been
cited before them, none of whom having appeared, they were all
ordered to be denounced; and that eighty persons, for hearing
sermon in the fields of Fife, had also been delated, of whom all that
answered had been found guilty and imprisoned, the remainder
declared fugitives, and their escheats appointed to be taken for his
majesty’s use.[78] The magistrates of Glasgow, also, had been fined
one hundred pounds sterling; and the magistrates of burghs, south
of the Tay, had been ordered to press upon the citizens the bond
against keeping conventicles.
77. The names of these worthies who deserve, and who will, be had in
everlasting remembrance, when those of their persecutors must rot, are thus
given by one of themselves:—Alexander Lennox, David Williamsone,
Alexander Moncrieff, John Rae, David Hume, Edward Jamieson, James
Fraser, William Wisehcart, Thomas Hogg in Ross, Robert Lockhart, John
Wilkie, George Johnstone, Patrick Gillespie, James Kirkton, John Weir,
Nathaniel Martin, Andrew Morton, Andrew Donaldsone, John Crichton,
William Row, Thomas Urquhart, Thomas Hogg in Larbert, William Arskine,
James Donaldson, Robert Gillespie, John Gray, James Wedderburn, John
Wardlaw, Thomas Douglas, George Campbell, Francis Irvine, John Wallace,
Andrew Anderson, John Munniman, George Hamilton, Donald Cargill,
Alexander Bertram, James Wilson, Robert Maxwell—in all 39. These were the
stock of the preaching church that was driven into the wilderness—their
ministry was a sort of outlawry—and, by the bishop’s activity, these, with the
ministers formerly forfaulted, and those who afterwards joined that body of
people, who first caused the separation from bishops and their curates,
thereafter overthrew their party, and wrought the Reformation.

78. “One day a paper was fixt upon the Parliament House door, containing
upwards of one hundred persons, whose escheats were to be sold to any
who would purchase them.” Wodrow, vol. i. p. 384.

While the primate was urging the persecution of these excellent

men, he was not without trouble from his own underlings. In the
beginning of the year, some of the bishops, as well as curates,
began to complain of the arbitrary measures of Sharpe, who
managed all ecclesiastical affairs without consulting them upon any
occasion, and had even the audacity to stamp upon him the
opprobrious epithet of Pope. His friends repelled the accusations as
the unfounded aspersions of the Hamilton or country party, who,
having failed to overturn the Duke of Lauderdale by means of the
Presbyterians, now wished to do it by means of the Episcopalians.
The others declared they only wished what the act of Parliament
allowed, to assemble in a National Synod, and regulate what they
considered wrong in the church—the best method of securing its
stability. But Sharpe, who, of all things, dreaded the least
interference with his power, wrote to the Archbishop of Canterbury,
entreating him—in a most impious appropriation of Scripture
language—to interfere and assist him against those who wished “to
break his bands, and cast his cords from them;” and his application
was so successful, that the most active of the suffragans were
silenced, and Ramsay, bishop of Dunblane, was removed to the Isles
—a kind of honourable banishment, which effectually put an end to
all attempts for the future at interfering with the supremacy of his
Grace of St Andrews. The only other bishop (Leighton) who had ever
given him real vexation, but against whom his wiles had been
useless, voluntarily withdrawing from the scene of contention,
Burnet was restored to Glasgow, and henceforth was content to play
second to Sharpe, only rivalling his oppression within the boundaries
of his own archiepiscopal territories.
 BOOK IX.

A.D. 1674-1676.

Divisions among the ministers respecting the church and self-defence—Armed

meetings—Severities increase—Lord Cardross—Religious revivals in the North—
Mr M’Gilligan—Civil oppression—Home of Polwart—Finings—Durham of Largo—
Magistrates of Edinburgh—Sufferers sent to France as recruits—Proclamation to
expel the families of gospel-hearers from the Burghs, and enforce the
conventicle act—Instructions for the indulged—Progress of the gospel—Rage of
the prelates—Mitchell tortured.

Unhappily the seeds of division which the indulgence had sown

among the Presbyterian ministers, were beginning to take root; and
the different opinions that afterwards reached so great and ruinous a
height, showed themselves in the discussions which took place
during this year upon that important question—How the Presbyterian
church was to be continued and supplied? The documents preserved
are very scanty; only it appears that the propriety of ordaining a
minister, except to a settled charge—of preaching within the nominal
bounds of an unsettled presbytery—and the authoritative right of
synodical meetings, were among the questions about which
differences had sprung up among the brethren. And from the care
with which they endeavour to provide against one minister noticing
the conduct of another “in their preaching, and warning the people
of the evils of the times,” it seems pretty evident that this baneful
practice had already commenced.
 At the close of the year, the state of feeling and anticipation
among the suffering Presbyterians was extremely dissimilar, as we
find by the writings both of public men and private Christians which
have been preserved. Numbers rejoiced in the bright and sunny side
of the cloud, in the increase of faithful preachers of the gospel—in
the desire for hearing that seemed to be abroad—and in the
delightful and not rare instances of the power of the Spirit that
accompanied the publication of the word; and they anticipated a
speedy and a glorious renovating morn for the church. Those that
studied the signs of the times, saw in the apostacy of some, and in
the falling away of others who had been esteemed pillars; in the
mournful waxing cold of the love of many; in the bitter dissensions
of professors; and in the general abounding iniquity—the dark and
dismal tokens of a deserted church; and, although they knew and
believed that the cause of Christ could never fall, and hoped and
rejoiced in the hope that a glorious day would yet arise upon
Scotland, wept and made mournful supplication for the sins of the
people among whom they dwelled, and anticipated heavier
judgments for unimproved mercies, until a returning to Him against
whom they had offended should again draw down the blessing.
The increasing severities which now began to be used towards the
conventicles likewise occasioned a difference of opinion among the
godly ministers and people as to the right of self-defence in hearing
the gospel. The injunction of Christ as to individuals is clear, when
persecuted in one city, flee into another; but in Scotland, where the
throne of iniquity framed mischief by a law, and where the whole
Presbyterians, who formed a large majority, were at once deprived
of their civil rights, as well as their religious privileges—and where a
constitution as solemnly ratified, and as sacredly sworn to, as any
mutual agreement between rulers and people ever was, or ever can
be, had been wantonly destroyed by a wretched minority of riotous
unprincipled sycophants, and place-hunting apostates—the question
involved, in the opinion of many, not only their duty as Christians,
but as citizens. Paul had taught them that these were not
incompatible, and their fathers had vindicated both in the field. The
young men were generally of this opinion, and began to come
armed to sermons about the commencement of this year [1675],
and talked of imitating the example of the days of the congregation.
The elder and most esteemed among the ministers were divided;
while, in general, they allowed the soundness of the principle, they
differed as to the propriety of the time. Among these appear to have
been Mr Welsh, Mr Blackadder, and Gabriel Semple; others, at the
head of whom stood Fraser of Brea and Kirkton, were entirely averse
to any resort to arms. The former thus states his views of the
subject:—
“A violent persecution had broken out; and then there began to be
fining, imprisoning, taking, and summoning of persons, disturbing of
conventicles with soldiers. But yet the gospel prevailed more and
more, and we were like the Israelites in Egypt, the more we were
afflicted, the more we grew and multiplied. Some hot heads were for
taking the sword and redeeming of themselves from the hands of
the oppressors; at least I had ground to fear it. But I opposed rising
in arms all I could, and preached against it, and exhorted them to
patience, and courageous using of the sword of the Spirit; and I did
not see they had any call to the sword, and their strength was to sit
still; and if they did stir and take the sword, they would therewith
perish; but if they patiently suffered and endured, God would himself
either incline to pity or some other way support and deliver them. I
had influence with the people, being popular, and whilst I was at
liberty I did what I could to keep the people peaceable. The truth is,
there were great provocations given, so that we concluded it was
the design of some rulers to stir us up that we might fall. Ministers
still preached and laboured among the people; conventicles
increased; many were brought in; the work of God, in the midst of
persecution, did always prosper, until we destroyed ourselves, first
by needless divisions and difference of opinion, happening by reason
of the indulgence; and thereafter by rash and unwarrantable taking
up of arms.”
Gentlemen in Scotland at this time, it requires to be remembered,
always wore arms as a part of dress; and the substantial heritors
and yeomen were in general accustomed to be accoutred when they
went from home, so that part of the meetings at field-preachings
had always consisted of armed men, who, before this, had offered
upon several occasions to defend their ministers at the risk of their
lives, but had been refused, and who now thought that in protecting
their assemblies from robbery and dispersion, and themselves from
imprisonment, fining, or slavery—the inevitable consequences of
being seized upon these occasions—that they were doing no more
than was required by the law of God, and authorized by the law of
their country, of which the prelatic party, and not they, were the
invaders and violaters.
Many contests had already ensued. The Episcopalian myrmidons in
Linlithgowshire, and even in Fife, had repeatedly drawn blood, while
the patient hearers of the gospel had only fled before them. The
rough borderers were not equally submissive.[79] At Lilliesleaf, and
throughout some of these districts, they had stood upon the
defensive and beaten off their assailants; and affairs were in this
situation during the greater part of this year. Upon the complaints of
the prelates, troops were ordered to scour the country in different
directions. Edinburgh and Glasgow were again fined each in the sum
of one hundred pounds sterling; and in addition, a detachment both
of horse and foot were quartered in the latter city. Mr John Greg, for
preaching at Leith mills, was sent to the Bass; and a Mr John
Sandilands, for hearing a sermon near Bathgate, was fined three
hundred merks. Nor were the nobility themselves spared. One of the
most cruelly oppressive cases was that of Lord Cardross.
79. Let it be always borne in mind, that the whole crowd who attended field-
preaching, were not influenced by gospel principles, nor could be considered
godly men, any more than that able disputers and fierce contenders for the
pure faith, are always themselves believers. It is an awful consideration, that
the most strenuous fighters for the purity of God’s word—the Jews—were
infidels, and thus addressed by our Saviour—“Ye have one that condemns
you, even Moses, in whom ye trust;” and the best written “Plea for the
Divinity of Christ,” was written by a man who turned a Socinian. Beware of
zealots!
 His lordship being confined in Edinburgh in the month of May, his
lady, who was far advanced in pregnancy, remained at home, with
only a few attendants. Sir Mungo Murray, taking advantage of this
circumstance, under cloud of night, accompanied by a posse of
retainers, went to his residence, and outrageously demanded that
the gates should be opened to him, else he would force his way and
set fire to the house. Situated near the borders of the Highlands, the
inmates naturally supposing them banditti, refused admission and
demanded who they were? To this no answer could be obtained, but
“Scottishmen,” which increased their alarm; yet fearing the worst, as
there were no means of defence, and no defenders, the gates were
opened, when the ruffians rushed in; and, after searching the whole
apartments in the most tumultuous and indelicate manner—forcing
Lady Cardross to rise from her bed that they might search her
chamber—and ransacking his lordship’s private closet, they seized Mr
John King, his chaplain, and Mr Robert Langlands, governor to his
brother, afterwards Colonel John Erskine, and carried them off.
Langlands was dismissed after being marched ten miles; Mr John
King was rescued by some countrymen who had profited by his
ministry. For this proceeding they had no warrant; and Lord
Cardross, immediately upon being informed of the outrage,
presented a complaint and petition to the privy council; but, instead
of receiving any satisfaction for the gross violation, not only of his
privileges as a nobleman, but his rights as a subject, he was charged
with having been guilty, art and part, in the rescue of Mr John King,
although he was sixty miles distant. For harbouring him in his house,
and for his lady’s having been present at many conventicles, and for
these complicated crimes, he was sentenced to be imprisoned during
his majesty’s pleasure in Edinburgh Castle, to pay a fine of one
thousand pounds sterling, besides various sums for the
delinquencies of his tenants.
Fining, imprisonment, and exile being found inadequate to the
suppression of conventicles, other and more rigorous methods were
resorted to. The houses of some of the principal gentlemen in the
most infected counties were seized, and garrisoned by parties of
horse and foot, that the least appearance of any gathering for
hearing sermon might at once be put down, with as much care and
celerity as the gathering of a civil, or the landing of a foreign,
enemy; and a number of the most faithful, diligent, and able
ministers this country was ever favoured with, were
“intercommuned,” their presence declared infectious as the plague,
and every loyal person prohibited from conversing with or doing
them any office, not of kindness, but of common humanity, under
the pain of being placed themselves without the pale of society.[80]
80. The names of these were—“David Williamson, Alexander Moncrief, William
Wiseheart, Thomas Hogg in Ross, George Johnstone, Robert Gillespie, John
M’Gilligan, John Ross, Thomas Hogg, Stirlingshire, William Erskine, James
Donaldson, Andrew Anderson, Andrew Morton, Donald Cargill, Robert
Maxwell, elder and younger, James Fraser of Brea, John King; and with these
a good many ladies and gentlemen were joined, besides many of lower rank,
altogether upwards of one hundred persons.” Wodrow, vol. i. p. 394. This
revival of a dormant and iniquitous law was peculiarly oppressive, as all who
conversed with the intercommuned being liable to the same punishment,
thousands might he unwittingly implicated, and laid at the mercy of their
rapacious rulers.

But one of the persecuted themselves remarks—“Although this

seemed to be the first storm of persecution that yet had fallen upon
us, and that now the adversaries had boasted of an effectual mean
for suppressing conventicles, and establishing prelacy and
uniformity, and the good people feared it; yet the Lord did
wonderfully disappoint them, and made and turned their witty
councils into folly—for this great noise harmed not at all, it was
powder without ball. For, as for myself, never one that cared for me
shunned my company; yea, a great many mere carnal relations and
acquaintances did entertain me as freely as ever they did; yea, so
far did the goodness of the Lord turn this to my good, that I
observed it was at that time I got most of my civil business expede.
And as the Lord preserved myself in this storm, so I did not hear of
any intercommuned, or conversers with intercommuned persons,
that were in the least prejudiced thereby; nay, this matter of the
intercommuning of so many good and peaceable men did but
exasperate the people against the bishops the more, and procured
to them, as the authors of such rigid courses, a greater and more
universal hatred; so that the whole land groaned to be delivered
from them.”
Danger, indeed, seemed to endear the ministers to the people;
and the risks they ran, and the many providential occurrences which
attended their meetings, produced a high degree of excitement, that
tended in no small measure to secure large and attentive audiences,
and prepared their minds for a solemn reception of the doctrines
they heard, at the peril of their lives.
North of the Tay there were but few Presbyterian ministers, and
they had not hitherto been very closely pursued; but among them
were some of the most excellent, and these of course were included
in the act of intercommuning—for their labours had been equally
abundant with the rest. Mr John M’Gilligan of Alness, was one of not
the least conspicuous, either for success or for suffering. On
September, the very month following his being denounced, he
dispensed the ordinance of the Lord’s Supper at Obsdale, in the
house of Lady Dowager Fowlis, assisted by Mr Hugh Anderson,
minister of Cromarty, and Mr Alexander Fraser, minister of Teviot.
According to the account preserved of it, it seems to have been one
of those heart-enlivening seasons which the Lord sometimes
vouchsafes to his church in the day of her visitation. “There were,”
says the narrator, “so sensible and glorious discoveries made of the
Son of Man, and such evident presence of the Master of assemblies,
that the people seemed to breathe the very atmosphere of heaven;
and some were so transportingly elevated, that they could almost
use the language of the apostle—‘whether in the body or out of the
body, I cannot tell.’ The eldest Christians there, declared they had
not been witnesses to the like. They also remarked that the Lord
wonderfully preserved them in peace.”
Some rumours of an intended communion having got abroad, the
sheriff-depute was ordered by the bishop to prevent or disperse the
meeting. He accordingly sent a party to apprehend the minister; but
he not knowing the spot, directed them to proceed to his house at
Alness, naturally supposing the meeting would be there. The
soldiers, upon finding the nest empty, attacked the orchard—a much
more pleasant amusement, that detained them till the forenoon’s
service was over at Obsdale, where, before they arrived, Mr
M’Gilligan had got notice, and was under hiding, which, when they
found, they retired without disturbing the congregation; and the
sacred solemnities proceeded without any further interruption. Mr
M’Gilligan, however, was obliged to abscond; and one of his
neighbours, Mr Thomas Ross, being apprehended at Tain for a
similar offence, was sent to the Bass.
Civil tyranny is always so interwoven with ecclesiastical
persecution, that it is seldom we are able to separate the two. But
the sufferings of Sir Patrick Home of Polwart, although they
undoubtedly originated from his religion, were ultimately effected
through the medium of his patriotism: he legally, by a bill of
suspension before the Court of Session, resisted a wanton stretch of
power in the privy council, and endeavoured to rouse the opposition
of the gentry of Berwickshire towards an oppressive, unjust tax for
planting garrisons among them in time of peace; and for this
undoubted exercise of his right, was committed, by order of the
king, prisoner to Stirling Castle, and declared incapable of holding
any place of trust; and the heritors succumbed, although the other
fines extorted from the shire this year amounted to nearly twenty-
seven thousand pounds Scots.
Nor were the indulged suffered to enjoy their limited and
precarious pardon quietly; their stipends were withheld or tardily
paid, and that only upon their producing certificates from the sheriffs
that they had kept no conventicles for the last twelve-month; but
their most vexatious trials were the natural consequences of their
acknowledging the power of the civil magistrate in ecclesiastical
affairs, and owning his warrant, rather than the authority of Christ,
as the rule in their ministerial labours. Complaints were brought
against them, and they were summoned before the council for not
celebrating the communion on the same day in all their parishes—for
irregular baptisms—and for having preached in churchyards and
other places than the kirks; but, above all, for having presumed to
authorize young men to preach the gospel, and ordained others to
the work of the ministry; and, at a time when a long tract of
unseasonable weather seemed to threaten a famine, they had
usurped a power which belonged to majesty alone, or his delegates,
and had appointed a fast in their several congregations! Through the
interest of Lord Stair, however, these grievances were not pushed to
extremities this year.
[1676.] Whatever circumstances might induce any occasional
relaxation in the severity of the persecution, the spirit remained the
same; and no opportunity was suffered to escape by which the
preaching of the gospel might be put down by men calling
themselves Christian bishops. The soldiers in the garrisons were
their willing instruments, and as they shared in the plunder, were
active in the pursuit; yet meetings for hearing the word continued to
increase, and the ordinances of religion were administered with a
solemnity and power, often at midnight, which rendered them the
general topics of interest and conversation among the people, and
still more the objects of aversion to the prelates. Finings for
“conventicles” were therefore inflicted by the council with
unmitigated rigour. Durham of Largo, for offences of this nature, and
harbouring that “notour traitor,” John Welsh, was early in the year
mulcted of nearly four thousand pounds Scots; Colonel Kerr, several
ladies, and some citizens of Edinburgh, were legally plundered in
various sums each, of five hundred merks, two hundred pounds, and
one hundred pounds Scots, for being at house conventicles within
the city; but the magistrates having also suffered for these
“enormities,” being soused for not preventing what they had never
previously heard of, they were allowed to reimburse themselves by
fining the culprits, who were thus punished twice for one crime.
A more revolting case of wanton cruelty was, about the same
time, exercised towards some poor men who had been guilty of
attending sermon in the fields near Stirling. Towards the end of
1674, they had been seized in the act and carried to jail; eight, by
some means or other, had got out—and the remaining seven sent
the following affecting petition to the council in the month of
February:—“The petitioners, being prisoners in the tolbooth of
Stirling, these fifteen months by-past, some of us being poor
decrepit bodies, and all of us poor creatures with wives and families,
we have been many times at the point of starving, and had long ere
now died for want, if we had not been supplied with the charity of
other people: The truth whereof is notour to all who live near
Stirling, and which the magistrates have testified by a report under
their hands: Wherefore, it is humbly desired that your lordships
would compassionate our pitiful and deplorable condition, and that
of our poor starving wives and children, and order us liberty, we
being willing to enact ourselves to compear and answer before your
lordships whenever we should be called.” Of those who signed, one,
Charles Campbell, was upwards of sixty, and one John Adam, near
seventy years of age; the others were labouring under severe bodily
indisposition. Yet, instead of being moved by the pitiful tale of these
harmless, aged, and sickly prisoners, the council, with an inhumanity
which it would not be easy to designate properly, ordered them to
be turned over as recruits! to one of Lauderdale’s minions, a Captain
Maitland, then an officer in the French service; and on Friday,
February 18, at midnight, they were delivered to a party of soldiers,
fettered and tied together, and marched off without any previous
warning. But they went cheerfully away, although they knew not
whither; for they knew the master whom they served would never
leave them naked to their enemies in their old age.
These severities were followed up by a fresh proclamation against
conventicles, in which, with the most hypocritical falsehood, after
lauding the king’s princely care and zeal for the interests of the
Protestant reformed religion and the church, and lamenting the sad
and sensible decay religion had suffered, and the great and
dangerous increase of profaneness, through the most unreasonable
and schismatical separation of many from the public and established
worship, and the frequent and open conventicles, both in houses
and fields—magistrates were required rigorously to apprehend all
who were intercommuned, and to expel their families from the
burghs, together with such preachers and their families as did not
regularly attend public worship—to enforce the acts against
conventicles and separation, under a penalty of five hundred merks
if they did not annually report their proceedings, and five hundred or
upwards additional, for every conventicle that shall have been held
within their jurisdictions, besides whatever other fine the council
might choose to inflict. All noblemen, gentlemen, and burgesses
were forbid to entertain any chaplin, tutor, or schoolmaster, under
penalties proportioned to their rank, from six hundred to three
thousand merks; and informers were, according to the system of the
times, by the same proclamation, encouraged and rewarded by a
share of the fines. Committees were also appointed to investigate
and punish transgressors, who fined and imprisoned many of the
most respectable heritors and gentlemen, particularly in the west,
and outlawed others who had declined answering their summons.
Enemies to the gospel of Christ, the prelatic rulers did not confine
their opposition to the preaching of the “outted” ministers, the
indulged were at the same time subjected to greater burdens. It was
evidently one of their main objects to produce division among the
Presbyterian ministers; and as we have seen the indulgence was
admirably calculated to effect this, yet the breach being neither so
wide nor so violent as they wished, “instructions” were issued to
them by the council. Assuming that they had accepted of liberty to
preach under conditions, the council accused them of violating their
engagements by baptizing without the necessary certificates, and
preaching in other places than their own kirk, without any license
from the bishop; and they added this injunction, that they should
not employ or allow any of their brethren to preach for them who
had not also obtained similar liberty. The indulged eluded the
charges, by alleging that they accepted of the indulgence as a boon
from government, not upon conditions, but as a favour granted; and
the instructions they considered as orders upon which they were to
act at their peril. But this neither satisfied the council nor their
brethren, both of whom concurred in thinking it an evasion rather
than an honest justification of their conduct. With the injunction they
appear to have complied also—a very unsatisfactory procedure—
which induced some, particularly of the younger unindulged
preachers, to visit the boundaries of their parishes, and led to heart-
burnings and mutual accusations between those who thought they
might yield a little to the pressure of the times, and those who in
nothing would recede from their avouched principles. These
differences, which afterwards unhappily led to coldness and
estrangement among the friends of “the good cause,” did not
produce their most mischievous effects till the oldest, stanch, tried
worthies were removed from the field. Meanwhile, the dispersion of
the ministers, who, when they were scattered abroad, went every
where preaching the word, was eminently blessed to promote that
gospel it was intended to destroy, and conventicles multiplied on
every side both in houses and fields.
Of the period from 1673 to 1679, Shiels gives this animating
picture on reviewing it many years after, when the holy excitement
had subsided, and temporal prosperity had began to diffuse its
seductive influence over the revolution-church:—“When by
persecution many ministers had been chased away by illegal law
sentences, many had been banished away, and, by their ensnaring
indulgences, many had been drawn away from their duty; and
others were now sentenced with confinements and restraints if they
should not choose and fix their residence where they could not keep
their quiet and conscience both—they were forced to wander and
disperse through the country; and the people being tired of the cold
and dead curates, and wanting long the ministry of their old pastors,
so longed and hungered after the word, that they behoved to have it
at any rate, cost what it would; which made them entertain the
dispersed ministers more earnestly, and encouraged them more to
their duty; by whose endeavours—through the mighty power and
presence of God, and the light of his countenance now shining
through the cloud, after so fatal and fearful a darkness that had
overclouded the land for a while, that it made their enemies gnash
their teeth for pain, and dazzled the eyes of all onlookers—the word
of God grew exceedingly, and went through at least the southern
borders like lightning; or, like the sun in its meridian beauty,
discovering so the wonders of God’s law, the mysteries of his gospel,
and the secrets of his covenant, and the sins and duties of that day,
that a numerous issue was begotten to Christ, and his conquest was
glorious, captivating poor slaves of Satan and bringing them from his
power unto God, and from darkness to light.
“O! who can remember the glory of that day, without a melting
heart in reflecting upon what we have lost, and let go, and sinned
away by our misimprovement—a day of such power that it made the
people, even the bulk and body of the people, willing to come out
and venture upon the greatest of hardships, and the greatest of
hazards, in pursuing after the gospel, through mosses, and muirs,
and inaccessible mountains, summer and winter, through excess of
heat and extremity of cold, many days’ and nights’ journeys, even
when they could not have a probable expectation of escaping the
sword of the wilderness. But this was a day of such power, that
nothing could daunt them from their duty that had tasted once the
sweetness of the Lord’s presence at these persecuted meetings.
“Then we had such humiliation-days for personal and public
defections, such communion-days even in the open fields, and such
Sabbath-solemnities, that the places where they were kept might
have been called Bethel, or Peniel, or Bochim, and all of them
Jehovah-Shammah, wherein many were truly converted, more
convinced, and generally all reformed from their former immoralities;
that even robbers, thieves, and profane men, were some of them
brought to a saving subjection to Christ, and generally under such
restraint, that all the severities of heading and hanging in a great
many years could not make such a civil reformation as a few days of
the gospel in these formerly the devil’s territories, now Christ’s
quarters, where his kingly standard was displayed. I have not
language to lay out the inexpressible glory of that day; but I doubt if
ever there were greater days of the Son of Man upon the earth, than
we enjoyed for the space of seven years at that time.”[81]
81. Hind let Loose, p. 132.

The border districts, so notorious in our earlier history as the fields

of constant plundering and murder, exhibited now amid their wild
scenery a warfare of a very different description. “What wonderful
success,” says Veitch, “the preaching of the word has had by
ministers retiring thither, under persecution, in order to the
repressing, yea almost extinguishing, these feuds, thefts, and
robberies, that were then so natural to that place and people, is
worth a singular and serious observation. These news ought to be
matter of joy and thanksgiving to all the truly godly in Britain, that,
though the ark, the glory, and goings of our God be, alas! too much
removed from Shiloh-Ephratah, the ingrounds, the places of greater
outward plenty and pleasure, yet that he is to be found in the
borders of those lands, in the mountains and fields of the woods.
Some of the gentry on both sides of the borders have been forced
both to see and say that the gospel has done that which their
execution of the laws could never accomplish. And is not such a
change worthy of remark? to see a people who used to ride
unweariedly through the long winter nights to steal and drive away
the prize, now, upon the report of a sermon, come from far,
travelling all night, to hear the gospel; yea, some bringing their
children along with them to the ordinance of baptism, although the
landlord threaten to eject the tenant, and the master the servant, for
so doing.”[82] Mr Gabriel Semple gives a similar statement. “These
borderers were looked upon to be ignorant, barbarous, and
debauched with all sort of wickedness, that none thought it worth
their consideration to look after them, thinking that they could not
be brought to any reformation. Yet, in the Lord’s infinite mercy, the
preaching to these borderers had more fruit than in many places
that were more civilized.”[83]
82. Memoirs of William Veitch, written by himself, published by Dr M’Crie, p. 118.

83. Semple’s Life MSS., in Dr Lee’s possession, quoted by Dr M’Crie, as above.

What ought to have filled the breast of every right-hearted

minister of the gospel with joy, excited the fellest passions in the
bosoms of the prelates, who evinced their filiation by doing the
deeds of their father, (John viii. 44,) furiously seeking to destroy
those who declared the truth; because, wherever a Presbyterian
preacher came, the Episcopalian churches were forsaken, and the
curates were left to harangue to empty pews. Political squabbling for
power between Hamilton and Lauderdale, had diverted the attention
of the two parties for a while from Scottish ecclesiastical affairs,
which the ministers eagerly took advantage of to pursue their sacred
vocation, judging wisely that the respite which they enjoyed would
be at best precarious. When Lauderdale gained the ascendency, they
anticipated a longer continuance of the “blink;”[84] but the clouds
soon gathered thicker and darker. He knew he could only maintain
his own elevation by exalting Episcopacy; and he quickly showed
that his repeated declarations were not empty bravadoes. More
correct in their calculations, the bishops improved the opportunity;
and the council, his and their ready tool, issued fresh proclamations
against conventicles, increasing in severity as they increased in
number.
84. “Blink”—a glimpse of sunshine in foul weather.
Averting their eyes from the loveliness of these bright prospects
that shone around them, they mourned withal “the sad and sensible
decays religion had of late suffered, and the great and dangerous
increase of profaneness through the most unreasonable and
schismatical separation of many from the public and established
worship, and the frequent and open conventicles, both in houses
and fields, by such as thereby discover their disaffection to the
established religion, and their aversion to his majesty’s authority and
government, endangering the peace of the kingdom, and dividing
the church under pretence of scruple:” therefore, to manifest their
zeal for the glory of Almighty God, the interests of the Protestant
reformed religion, and of the church—to secure the same by unity in
worship, and procuring all due reverence to archbishops, bishops,
and all subordinate clerical officers—the magistrates of the several
burghs were specially required to seize upon all persons who were,
or should be, intercommuned, and to remove the families of such
from all places under their jurisdiction, together with all preachers
and their families who did not attend the public worship! All
noblemen, gentlemen, and others, were strictly forbid to afford
shelter or aid to any intercommuned person, upon pain of being
themselves intercommuned; and whosoever should discover those
that transgressed, were to receive five hundred merks reward
immediately. Magistrates were also rendered liable to severe fining,
if they did not rigorously fulfil the imperative duty of searching out
and punishing all such as worshipped God after the manner they
chose to call heresy.
What means they thought lawful for obtaining information from
suspected persons, is evinced in their treatment of James Mitchell,
who made the unsuccessful attempt upon Sharpe. He had left the
country at the time, and did not return till he supposed the affair
was forgotten, when he married a woman who kept a small shop not
far from the primate’s town residence. In passing this way, his Grace
observed a person eye him keenly, which rather alarmed him, as he
thought he recognized his foiled assassin; and he caused him to be
arrested. A pistol, loaded with three bullets, being found in his
pocket, increased his terror, and he became extremely anxious to
know the extent of his danger. Accordingly, before the prisoner was
examined, he swore by the living God, if he would confess the act,
he would obtain his pardon; and a committee of the privy council,
consisting of Rothes, Lord Chancellor; Primrose, Lord Register;
Nisbet, Lord Advocate; and Hatton, Treasurer-depute, authorized by
the Commissioner, gave him a similar assurance. Disappointed,
however, by his confession, as they expected to discover a
conspiracy, on finding he had no accomplice, and unwilling that he
should thus escape, they remitted him to the Justiciary Court,
evading their solemn engagement by a jesuitical quibble, that the
promise of securing his life did not guarantee the safety of his limbs.
Having received a hint, as he was passing to trial, he disclaimed his
confession at the bar; and there being no other proof, the judicial
proceedings were abandoned, or, in Scottish law-phrase, the “diet”
was deserted, and he was remanded to prison, where he remained
till January this year, when the spirit of cruelty which appeared to
actuate the then rulers against all who were rigid Presbyterians,
especially preachers, urged them to subject their unhappy victim to
the torture.
About six o’clock in the evening of January 18th, Mitchell was
brought before a meeting of Justiciary, where the Earl of Linlithgow
sat president, and questioned whether he would adhere to his
former confession. He replied, that the Lord Advocate having
deserted the diet against him, he ought to have been, agreeably
both to the law of the nation and the practice of the court, set at
liberty, and therefore knew no reason why he was that night brought
before their lordships. Without any attention being paid to this
strictly legal objection, he was again asked, if he would adhere to his
former confession? He refused to own any confession; and Hatton
most outrageously exclaimed, “that pannel is one of the most
arrogant cheats, liars, and rogues I have ever known!” Mr Mitchell
retorted, My lord, if there were fewer of those persons you have
been speaking of in the nation, I would not have been standing at
this bar. The President said, “We will cause a sharper thing make you
confess.” “I hope, my lord, you are Christians and not Pagans,” was
the prisoner’s response, with which the business of that evening
closed.
Upon the 22d, he was brought before them in the lower Council-
Chamber, and the question repeated, the President at the same time
pointing to the boots, said, “You see, sir, what is upon the table; I
will see if that will make you confess.” “My lord,” answered Mitchell
intrepidly, “I confess by torture you may make me blaspheme God,
as Saul did compel the saints; you may compel me to call myself a
thief, a murderer, a warlock, or any thing, and then pannel me upon
it; but if you shall, my lord, put me to it, I here protest before God
and your lordships, that nothing extorted from me by torture shall
be made use of against me in judgment, nor have any force in law
against me or any person whatsomever. But to be plain with you, my
lords, I am so much of a Christian, that whatever your lordships shall
legally prove against me, if it be a truth, I shall not deny it; but, on
the contrary, I am so much of a man, and a Scottishman, that I
never hold myself obliged by the law of God, nature, or the nation,
to become my own accuser.” Hatton rudely answered—“He hath the
devil’s logic, and sophisticates like him; ask him whether that be his
subscription.” “I acknowledge no such thing,” said the pannel, and
was remanded to jail.
Two days after, the judges, in formal pomp, arrayed in their robes,
and attended by the executioner with the instruments of torture, like
true inquisitors, first attempted to terrify their prisoner, before they
literally put him to the question. It was in vain. They could not shake
him. Had they not been dead to every nobler feeling of our nature,
they must have quailed when he thus addressed them:—“My lords, I
have now been these two full years in prison, and more than one of
them in bolts and fetters—more intolerable than many deaths. Some
in a shorter time have been tempted to make away with themselves;
but, in obedience to the express command of God, I have endured
all these hardships, and I hope to endure this torture also with
patience, on purpose to preserve my own life, and that of others
also, as far as lies in my power, and to keep the guilt of innocent
blood off your lordships and your families, which you doubtless
would incur by shedding mine. I repeat my protest. When you
please, call for the men you have appointed to their work.” The
executioner being in attendance, immediately tied Mr Mitchell in an
arm-chair, and asked which of the legs he should take? The lords
said, “Any of them.” The executioner laid in the left; but Mr Mitchell
taking it out, said, “Since the judges have not determined, take the
best of the two; I bestow it freely in the cause.” He was interrogated
about his being at the battle of Pentland, his meeting with Wallace
or with Captain Arnot—all of which he could veritably answer in the
negative. The tormentor then began to drive the wedges, asking at
every stroke if he had any more to say? To this he generally replied
“No.” After a while, when the pain began to be excruciating, he
exclaimed, again addressing his inquisitors—“My lords, not knowing
but this torture may end my life, I beseech you to remember, that
‘he who showeth no mercy, shall have judgment without mercy;’ for
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com