Experience Seamless Full Ebook Downloads for Every Genre at textbookfull.

com

Deep Learning with Python Develop Deep Learning

Models on Theano and TensorFLow Using Keras Jason
Brownlee

https://textbookfull.com/product/deep-learning-with-python-
develop-deep-learning-models-on-theano-and-tensorflow-using-
keras-jason-brownlee/

OR CLICK BUTTON

DOWNLOAD NOW

Explore and download more ebook at https://textbookfull.com

 Recommended digital products (PDF, EPUB, MOBI) that
you can download immediately if you are interested.

Deep Learning for Natural Language Processing Develop Deep

Learning Models for Natural Language in Python Jason
Brownlee
https://textbookfull.com/product/deep-learning-for-natural-language-
processing-develop-deep-learning-models-for-natural-language-in-
python-jason-brownlee/
textboxfull.com

Deep Learning Projects Using TensorFlow 2: Neural Network

Development with Python and Keras 1st Edition Vinita
Silaparasetty
https://textbookfull.com/product/deep-learning-projects-using-
tensorflow-2-neural-network-development-with-python-and-keras-1st-
edition-vinita-silaparasetty/
textboxfull.com

Deep Learning with Applications Using Python Chatbots and

Face, Object, and Speech Recognition With TensorFlow and
Keras Springerlink (Online Service)
https://textbookfull.com/product/deep-learning-with-applications-
using-python-chatbots-and-face-object-and-speech-recognition-with-
tensorflow-and-keras-springerlink-online-service/
textboxfull.com

Beginning Anomaly Detection Using Python-Based Deep

Learning: With Keras and PyTorch Sridhar Alla

https://textbookfull.com/product/beginning-anomaly-detection-using-
python-based-deep-learning-with-keras-and-pytorch-sridhar-alla/

textboxfull.com
Computer Vision Using Deep Learning Neural Network
Architectures with Python and Keras 1st Edition Vaibhav
Verdhan
https://textbookfull.com/product/computer-vision-using-deep-learning-
neural-network-architectures-with-python-and-keras-1st-edition-
vaibhav-verdhan/
textboxfull.com

Deep Learning for Time Series Forecasting Predict the

Future with MLPs CNNs and LSTMs in Python Jason Brownlee

https://textbookfull.com/product/deep-learning-for-time-series-
forecasting-predict-the-future-with-mlps-cnns-and-lstms-in-python-
jason-brownlee/
textboxfull.com

Reinforcement Learning: With Open AI, TensorFlow and Keras

Using Python 1st Edition Abhishek Nandy

https://textbookfull.com/product/reinforcement-learning-with-open-ai-
tensorflow-and-keras-using-python-1st-edition-abhishek-nandy/

textboxfull.com

Deep Learning with Python Learn Best Practices of Deep

Learning Models with PyTorch 2nd Edition Nikhil Ketkar
Jojo Moolayil
https://textbookfull.com/product/deep-learning-with-python-learn-best-
practices-of-deep-learning-models-with-pytorch-2nd-edition-nikhil-
ketkar-jojo-moolayil/
textboxfull.com

Deep Learning with Python Learn Best Practices of Deep

Learning Models with PyTorch 2nd Edition Nikhil Ketkar
Jojo Moolayil
https://textbookfull.com/product/deep-learning-with-python-learn-best-
practices-of-deep-learning-models-with-pytorch-2nd-edition-nikhil-
ketkar-jojo-moolayil-2/
textboxfull.com

   

��������������������������������
����������������������������
�����

��������������
Jason Brownlee

Deep Learning With Python

Develop Deep Learning Models On Theano And TensorFlow Using
Keras
 i

Deep Learning With Python

Copyright 2016 Jason Brownlee. All Rights Reserved.

Edition: v1.7
Contents

Preface iii

I Introduction 1
1 Welcome 2
1.1 Deep Learning The Wrong Way . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Deep Learning With Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Book Organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Requirements For This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.5 Your Outcomes From Reading This Book . . . . . . . . . . . . . . . . . . . . . . 7
1.6 What This Book is Not . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

II Background 10
2 Introduction to Theano 11
2.1 What is Theano? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 How to Install Theano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Simple Theano Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4 Extensions and Wrappers for Theano . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5 More Theano Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Introduction to TensorFlow 15
3.1 What is TensorFlow? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 How to Install TensorFlow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.3 Your First Examples in TensorFlow . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4 Simple TensorFlow Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.5 More Deep Learning Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Introduction to Keras 19
4.1 What is Keras? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4.2 How to Install Keras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.3 Theano and TensorFlow Backends for Keras . . . . . . . . . . . . . . . . . . . . 20

ii
 iii

4.4 Build Deep Learning Models with Keras . . . . . . . . . . . . . . . . . . . . . . 21

4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Project: Develop Large Models on GPUs Cheaply In the Cloud 23

5.1 Project Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
5.2 Setup Your AWS Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
5.3 Launch Your Server Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
5.4 Login, Configure and Run . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.5 Build and Run Models on AWS . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.6 Close Your EC2 Instance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.7 Tips and Tricks for Using Keras on AWS . . . . . . . . . . . . . . . . . . . . . . 34
5.8 More Resources For Deep Learning on AWS . . . . . . . . . . . . . . . . . . . . 34
5.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

III Multilayer Perceptrons 36

6 Crash Course In Multilayer Perceptrons 37
6.1 Crash Course Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.2 Multilayer Perceptrons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.3 Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.4 Networks of Neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6.5 Training Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

7 Develop Your First Neural Network With Keras 43

7.1 Tutorial Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
7.2 Pima Indians Onset of Diabetes Dataset . . . . . . . . . . . . . . . . . . . . . . 44
7.3 Load Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.4 Define Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.5 Compile Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
7.6 Fit Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
7.7 Evaluate Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
7.8 Tie It All Together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
7.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

8 Evaluate The Performance of Deep Learning Models 51

8.1 Empirically Evaluate Network Configurations . . . . . . . . . . . . . . . . . . . 51
8.2 Data Splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
8.3 Manual k-Fold Cross Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

9 Use Keras Models With Scikit-Learn For General Machine Learning 57

9.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
9.2 Evaluate Models with Cross Validation . . . . . . . . . . . . . . . . . . . . . . . 58
9.3 Grid Search Deep Learning Model Parameters . . . . . . . . . . . . . . . . . . . 59
9.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
 iv

10 Project: Multiclass Classification Of Flower Species 62

10.1 Iris Flowers Classification Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . 62
10.2 Import Classes and Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
10.3 Initialize Random Number Generator . . . . . . . . . . . . . . . . . . . . . . . . 63
10.4 Load The Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
10.5 Encode The Output Variable . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
10.6 Define The Neural Network Model . . . . . . . . . . . . . . . . . . . . . . . . . 65
10.7 Evaluate The Model with k-Fold Cross Validation . . . . . . . . . . . . . . . . . 65
10.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

11 Project: Binary Classification Of Sonar Returns 68

11.1 Sonar Object Classification Dataset . . . . . . . . . . . . . . . . . . . . . . . . . 68
11.2 Baseline Neural Network Model Performance . . . . . . . . . . . . . . . . . . . . 69
11.3 Improve Performance With Data Preparation . . . . . . . . . . . . . . . . . . . 71
11.4 Tuning Layers and Neurons in The Model . . . . . . . . . . . . . . . . . . . . . 73
11.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

12 Project: Regression Of Boston House Prices 77

12.1 Boston House Price Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
12.2 Develop a Baseline Neural Network Model . . . . . . . . . . . . . . . . . . . . . 78
12.3 Lift Performance By Standardizing The Dataset . . . . . . . . . . . . . . . . . . 81
12.4 Tune The Neural Network Topology . . . . . . . . . . . . . . . . . . . . . . . . . 82
12.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

IV Advanced Multilayer Perceptrons and Keras 86

13 Save Your Models For Later With Serialization 87
13.1 Tutorial Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
13.2 Save Your Neural Network Model to JSON . . . . . . . . . . . . . . . . . . . . . 88
13.3 Save Your Neural Network Model to YAML . . . . . . . . . . . . . . . . . . . . 90
13.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

14 Keep The Best Models During Training With Checkpointing 93

14.1 Checkpointing Neural Network Models . . . . . . . . . . . . . . . . . . . . . . . 93
14.2 Checkpoint Neural Network Model Improvements . . . . . . . . . . . . . . . . . 94
14.3 Checkpoint Best Neural Network Model Only . . . . . . . . . . . . . . . . . . . 95
14.4 Loading a Saved Neural Network Model . . . . . . . . . . . . . . . . . . . . . . 96
14.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

15 Understand Model Behavior During Training By Plotting History 98

15.1 Access Model Training History in Keras . . . . . . . . . . . . . . . . . . . . . . 98
15.2 Visualize Model Training History in Keras . . . . . . . . . . . . . . . . . . . . . 99
15.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
 v

16 Reduce Overfitting With Dropout Regularization 102

16.1 Dropout Regularization For Neural Networks . . . . . . . . . . . . . . . . . . . . 102
16.2 Dropout Regularization in Keras . . . . . . . . . . . . . . . . . . . . . . . . . . 103
16.3 Using Dropout on the Visible Layer . . . . . . . . . . . . . . . . . . . . . . . . . 104
16.4 Using Dropout on Hidden Layers . . . . . . . . . . . . . . . . . . . . . . . . . . 105
16.5 Tips For Using Dropout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
16.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

17 Lift Performance With Learning Rate Schedules 108

17.1 Learning Rate Schedule For Training Models . . . . . . . . . . . . . . . . . . . . 108
17.2 Ionosphere Classification Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . 109
17.3 Time-Based Learning Rate Schedule . . . . . . . . . . . . . . . . . . . . . . . . 109
17.4 Drop-Based Learning Rate Schedule . . . . . . . . . . . . . . . . . . . . . . . . . 112
17.5 Tips for Using Learning Rate Schedules . . . . . . . . . . . . . . . . . . . . . . . 114
17.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

V Convolutional Neural Networks 115

18 Crash Course In Convolutional Neural Networks 116
18.1 The Case for Convolutional Neural Networks . . . . . . . . . . . . . . . . . . . . 116
18.2 Building Blocks of Convolutional Neural Networks . . . . . . . . . . . . . . . . . 117
18.3 Convolutional Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
18.4 Pooling Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
18.5 Fully Connected Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
18.6 Worked Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
18.7 Convolutional Neural Networks Best Practices . . . . . . . . . . . . . . . . . . . 119
18.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

19 Project: Handwritten Digit Recognition 121

19.1 Handwritten Digit Recognition Dataset . . . . . . . . . . . . . . . . . . . . . . . 121
19.2 Loading the MNIST dataset in Keras . . . . . . . . . . . . . . . . . . . . . . . . 122
19.3 Baseline Model with Multilayer Perceptrons . . . . . . . . . . . . . . . . . . . . 123
19.4 Simple Convolutional Neural Network for MNIST . . . . . . . . . . . . . . . . . 127
19.5 Larger Convolutional Neural Network for MNIST . . . . . . . . . . . . . . . . . 131
19.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

20 Improve Model Performance With Image Augmentation 135

20.1 Keras Image Augmentation API . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
20.2 Point of Comparison for Image Augmentation . . . . . . . . . . . . . . . . . . . 136
20.3 Feature Standardization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
20.4 ZCA Whitening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
20.5 Random Rotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
20.6 Random Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
20.7 Random Flips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
20.8 Saving Augmented Images to File . . . . . . . . . . . . . . . . . . . . . . . . . . 145
20.9 Tips For Augmenting Image Data with Keras . . . . . . . . . . . . . . . . . . . 147
 vi

20.10Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

21 Project Object Recognition in Photographs 148

21.1 Photograph Object Recognition Dataset . . . . . . . . . . . . . . . . . . . . . . 148
21.2 Loading The CIFAR-10 Dataset in Keras . . . . . . . . . . . . . . . . . . . . . . 149
21.3 Simple CNN for CIFAR-10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
21.4 Larger CNN for CIFAR-10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
21.5 Extensions To Improve Model Performance . . . . . . . . . . . . . . . . . . . . . 157
21.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

22 Project: Predict Sentiment From Movie Reviews 159

22.1 Movie Review Sentiment Classification Dataset . . . . . . . . . . . . . . . . . . 159
22.2 Load the IMDB Dataset With Keras . . . . . . . . . . . . . . . . . . . . . . . . 160
22.3 Word Embeddings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
22.4 Simple Multilayer Perceptron Model . . . . . . . . . . . . . . . . . . . . . . . . 163
22.5 One-Dimensional Convolutional Neural Network . . . . . . . . . . . . . . . . . . 165
22.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

VI Recurrent Neural Networks 169

23 Crash Course In Recurrent Neural Networks 170
23.1 Support For Sequences in Neural Networks . . . . . . . . . . . . . . . . . . . . . 170
23.2 Recurrent Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
23.3 Long Short-Term Memory Networks . . . . . . . . . . . . . . . . . . . . . . . . . 172
23.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

24 Time Series Prediction with Multilayer Perceptrons 174

24.1 Problem Description: Time Series Prediction . . . . . . . . . . . . . . . . . . . . 174
24.2 Multilayer Perceptron Regression . . . . . . . . . . . . . . . . . . . . . . . . . . 176
24.3 Multilayer Perceptron Using the Window Method . . . . . . . . . . . . . . . . . 181
24.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

25 Time Series Prediction with LSTM Recurrent Neural Networks 185

25.1 LSTM Network For Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
25.2 LSTM For Regression Using the Window Method . . . . . . . . . . . . . . . . . 189
25.3 LSTM For Regression with Time Steps . . . . . . . . . . . . . . . . . . . . . . . 191
25.4 LSTM With Memory Between Batches . . . . . . . . . . . . . . . . . . . . . . . 194
25.5 Stacked LSTMs With Memory Between Batches . . . . . . . . . . . . . . . . . . 197
25.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

26 Project: Sequence Classification of Movie Reviews 201

26.1 Simple LSTM for Sequence Classification . . . . . . . . . . . . . . . . . . . . . . 201
26.2 LSTM For Sequence Classification With Dropout . . . . . . . . . . . . . . . . . 203
26.3 LSTM and CNN For Sequence Classification . . . . . . . . . . . . . . . . . . . . 206
26.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
 vii

27 Understanding Stateful LSTM Recurrent Neural Networks 209

27.1 Problem Description: Learn the Alphabet . . . . . . . . . . . . . . . . . . . . . 209
27.2 LSTM for Learning One-Char to One-Char Mapping . . . . . . . . . . . . . . . 211
27.3 LSTM for a Feature Window to One-Char Mapping . . . . . . . . . . . . . . . . 214
27.4 LSTM for a Time Step Window to One-Char Mapping . . . . . . . . . . . . . . 216
27.5 LSTM State Maintained Between Samples Within A Batch . . . . . . . . . . . . 218
27.6 Stateful LSTM for a One-Char to One-Char Mapping . . . . . . . . . . . . . . . 221
27.7 LSTM with Variable Length Input to One-Char Output . . . . . . . . . . . . . . 224
27.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

28 Project: Text Generation With Alice in Wonderland 228

28.1 Problem Description: Text Generation . . . . . . . . . . . . . . . . . . . . . . . 228
28.2 Develop a Small LSTM Recurrent Neural Network . . . . . . . . . . . . . . . . . 229
28.3 Generating Text with an LSTM Network . . . . . . . . . . . . . . . . . . . . . . 234
28.4 Larger LSTM Recurrent Neural Network . . . . . . . . . . . . . . . . . . . . . . 237
28.5 Extension Ideas to Improve the Model . . . . . . . . . . . . . . . . . . . . . . . 240
28.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

VII Conclusions 242

29 How Far You Have Come 243

30 Getting More Help 244

30.1 Artificial Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
30.2 Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
30.3 Python Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
30.4 Keras Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
Preface

Deep learning is a fascinating field. Artificial neural networks have been around for a long time,
but something special has happened in recent years. The mixture of new faster hardware, new
techniques and highly optimized open source libraries allow very large networks to be created
with frightening ease.
This new wave of much larger and much deeper neural networks are also impressively skillful
on a range of problems. I have watched over recent years as they tackle and handily become
state-of-the-art across a range of difficult problem domains. Not least object recognition, speech
recognition, sentiment classification, translation and more.
When a technique comes a long that does so well on such a broad set of problems, you have
to pay attention. The problem is where do you start with deep learning? I created this book
because I thought that there was no gentle way for Python machine learning practitioners to
quickly get started developing deep learning models.
In developing the lessons in this book, I chose the best of breed Python deep learning library
called Keras that abstracted away all of the complexity, ruthlessly leaving you an API containing
only what you need to know to efficiently develop and evaluate neural network models.
This is the guide that I wish I had when I started apply deep learning to machine learning
problems. I hope that you find it useful on your own projects and have as much fun applying
deep learning as I did in creating this book for you.

Jason Brownlee
Melbourne, Australia
2016

viii
 Part I

Introduction

1
Chapter 1

Welcome

Welcome to Deep Learning With Python. This book is your guide to deep learning in Python.
You will discover the Keras Python library for deep learning and how to use it to develop and
evaluate deep learning models. In this book you will discover the techniques, recipes and skills
in deep learning that you can then bring to your own machine learning projects.
Deep learning does have a lot of fascinating math under the covers, but you do not need
to know it to be able to pick it up as a tool and wield it on important projects and deliver
real value. From the applied perspective, deep learning is quite a shallow field and a motivated
developer can quickly pick it up and start making very real and impactful contributions. This is
my goal for you and this book is your ticket to that outcome.

1.1 Deep Learning The Wrong Way

If you ask a deep learning practitioner how to get started with neural networks and deep learning,
what do they say? They say things like

You must have a strong foundation in linear algebra.

You must have a deep knowledge of traditional neural network techniques.

You really must know about probability and statistics.

You should really have a deep knowledge of machine learning.

You probably need to be a PhD in computer science.

You probably need 10 years of experience as a machine learning developer.

You can see that the “common sense” advice means that it is not until after you have
completed years of study and experience that you are ready to actually start developing and
evaluating machine learning model for your machine learning projects.
I think this advice is dead wrong.

2
1.2. Deep Learning With Python 3

1.2 Deep Learning With Python

The approach taken with this book and with all of Machine Learning Mastery is to flip the
traditional approach. If you are interested in deep learning, start by developing and evaluating
deep learning models. Then if you discover you really like it or have a knack for it, later you
can step deeper and deeper into the background and theory, as you need it in order to serve
you in developing better and more valuable results. This book is your ticket to jumping in and
making a ruckus with deep learning.
I have used many of the top deep learning platforms and libraries and I chose what I think
is the best-of-breed platform for getting started and very quickly developing powerful and even
state-of-the-art deep learning models in the Keras deep learning library for Python. Unlike R,
Python is a fully featured programming language allowing you to use the same libraries and
code for model development as you can use in production. Unlike Java, Python has the SciPy
stack for scientific computing and scikit-learn which is a professional grade machine library.
There are two top numerical platforms for developing deep learning models, they are Theano
developed by the University of Montreal and TensorFlow developed at Google. Both were
developed for use in Python and both can be leveraged by the super simple to use Keras library.
Keras wraps the numerical computing complexity of Theano and TensorFlow providing a concise
API that we will use to develop our own neural network and deep learning models.
You will develop your own and perhaps your first neural network and deep learning models
while working through this book, and you will have the skills to bring this amazing new
technology to your own projects. It is going to be a fun journey and I can’t wait to start.

1.3 Book Organization

This book is broken down into three parts.

Lessons where you learn about specific features of neural network models and or how to
use specific aspects of the Keras API.

Projects where you will pull together multiple lessons into an end-to-end project and
deliver a result, providing a template your your own projects.

Recipes where you can copy and paste the standalone code into your own project,
including all of the code presented in this book.

1.3.1 Lessons and Projects

Lessons are discrete and are focused on one topic, designed for you to complete in one sitting.
You can take as long as you need, from 20 minutes if you are racing through, to hours if you
want to experiment with the code or ideas and improve upon the presented results. Your lessons
are divided into five parts:

Background.

Multilayer Perceptrons.

Advanced Multilayer Perceptrons and Keras.

1.3. Book Organization 4

Convolutional Neural Networks.

Recurrent Neural Networks.

1.3.2 Part 2: Background

In this part you will learn about the Theano, TensorFlow and Keras libraries that lay the
foundation for your deep learning journey and about how you can leverage very cheap Amazon
Web Service computing in order to develop and evaluate your own large models in the cloud.
This part of the book includes the following lessons:
Introduction to the Theano Numerical Library.
Introduction to the TensorFlow Numerical Library.
Introduction to the Keras Deep Learning Library.
The lessons will introduce you to the important foundational libraries that you need to
install and use on your workstation. This is taken one step further in a project that shows how
you can cheaply harness GPU cloud computing to develop and evaluate very large deep learning
models.
Project: Develop Large Models on GPUs Cheaply In the Cloud.
At the end of this part you will be ready to start developing models in Keras on your
workstation or in the cloud.

1.3.3 Part 3: Multilayer Perceptrons

In this part you will learn about feedforward neural networks that may be deep or not and how
to expertly develop your own networks and evaluate them efficiently using Keras. This part of
the book includes the following lessons:
Crash Course In Multilayer Perceptrons.
Develop Your First Neural Network With Keras.
Evaluate The Performance of Deep Learning Models.
Use Keras Models With Scikit-Learn For General Machine Learning.
These important lessons are tied together with three foundation projects. These projects
demonstrate how you can quickly and efficiently develop neural network models for tabular
data and provide project templates that you can use on your own regression and classification
machine learning problems. These projects include:
Project: Multiclass Classification Problem.
Project: Binary Classification Problem.
Project: Regression Problem.
At the end of this part you will be ready to discover the finer points of deep learning using
the Keras API.
1.3. Book Organization 5

1.3.4 Part 4: Advanced Multilayer Perceptrons

In this part you will learn about some of the more finer points of the Keras library and API for
practical machine learning projects and some of the more important developments in applied
neural networks that you need to know in order to deliver world class results. This part of the
book includes the following lessons:

Save Your Models For Later With Network Serialization.

Keep The Best Models During Training With Checkpointing.

Understand Model Behavior During Training By Plotting History.

Reduce Overfitting With Dropout Regularization.

Lift Performance With Learning Rate Schedules.

At the end of this part you will know how to confidently wield Keras on your own machine
learning projects with a focus of the finer points of investigating model performance, persisting
models for later use and gaining lifts in performance over baseline models.

1.3.5 Part 5: Convolutional Neural Networks

In this part you will receive a crash course in the dominant model for computer vision machine
learning problems and some natural language problems and how you can best exploit the
capabilities of the Keras API for your own projects. This part of the book includes the following
lessons:

Crash Course In Convolutional Neural Networks.

Improve Model Performance With Image Augmentation.

The best way to learn about this impressive type of neural network model is to apply it.
You will work through three larger projects and apply CNN to image data for object recognition
and text data for sentiment classification.

Project: Handwritten Digit Recognition.

Project: Object Recognition in Photographs.

Project: Movie Review Sentiment Classification.

After completing the lessons and projects in this part you will have the skills and the
confidence of complete and working templates and recipes to tackle your own deep learning
projects using convolutional neural networks.
1.4. Requirements For This Book 6

1.3.6 Part 6: Recurrent Neural Networks

In this part you will receive a crash course in the dominant model for data with a sequence or
time component and how you can best exploit the capabilities of the Keras API for your own
projects. This part of the book includes the following lessons:

Crash Course In Recurrent Neural Networks.

Multilayer Perceptron Models for Time Series Problems.

LSTM Models for Time Series Problems.

Understanding State in LSTM Models for Sequence Prediction.

The best way to learn about this complex type of neural network model is to apply it.
You will work through two larger projects and apply RNN to sequence classification and text
generation.

Project: Sequence Classification of Movie Reviews.

Project: Text Generation With Alice in Wonderland.

After completing the lessons and projects in this part you will have the skills and the
confidence of complete and working templates and recipes to tackle your own deep learning
projects using recurrent neural networks.

1.3.7 Conclusions
The book concludes with some resources that you can use to learn more information about a
specific topic or find help if you need it as you start to develop and evaluate your own deep
learning models.

1.3.8 Recipes
Building up a catalog of code recipes is an important part of your deep learning journey. Each
time you learn about a new technique or new problem type, you should write up a short code
recipe that demonstrates it. This will give you a starting point to use on your next deep learning
or machine learning project.
As part of this book you will receive a catalog of deep learning recipes. This includes recipes
for all of the lessons presented in this book, as well as the complete code for all of the projects.
You are strongly encouraged to add to and build upon this catalog of recipes as you expand
your use and knowledge of deep learning in Python.

1.4 Requirements For This Book

1.4.1 Python and SciPy
You do not need to be a Python expert, but it would be helpful if you knew how to install and
setup Python and SciPy. The lessons and projects assume that you have a Python and SciPy
1.5. Your Outcomes From Reading This Book 7

environment available. This may be on your workstation or laptop, it may be in a VM or a

Docker instance that you run, or it may be a server instance that you can configure in the cloud
as taught in Part II of this book.
Technical Requirements: The technical requirements for the code and tutorials in this
book are as follows:

Python version 2 or 3 installed. This book was developed using Python version 2.7.11.

SciPy and NumPy installed. This book was developed with SciPy version 0.17.0 and
NumPy version 1.11.0.

Matplotlib installed. This book was developed with Matplotlib version 1.5.1.

Pandas installed. This book was developed with Pandas version 0.18.0.

scikit-learn installed. This book was developed with scikit-learn 0.17.1.

You do not need to match the version exactly, but if you are having problems running a
specific code example, please ensure that you update to the same or higher version as the library
specified. You will be guided as to how to install the deep learning libraries Theano, TensorFlow
and Keras in Part II of the book.

1.4.2 Machine Learning

You do not need to be a machine learning expert, but it would be helpful if you knew how to
navigate a small machine learning problem using scikit-learn. Basic concepts like cross validation
and one hot encoding used in lessons and projects are described, but only briefly. There are
resources to go into these topics in more detail at the end of the book, but some knowledge of
these areas might make things easier for you.

1.4.3 Deep Learning

You do not need to know the math and theory of deep learning algorithms, but it would be
helpful to have some basic idea of the field. You will get a crash course in neural network
terminology and models, but we will not go into much detail. Again, there will be resources for
more information at the end of the book, but it might be helpful if you can start with some
idea about neural networks.
Note: All tutorials can be completed on standard workstation hardware with a CPU. A
GPU is not required. Some tutorials later in the book can be sped up significantly by running
on the GPU and a suggestion is provided to consider using GPU hardware at the beginning of
those sections. You can access GPU hardware easily and cheaply in the cloud and a step-by-step
procedure is taught on how to do this in Chapter 5.

1.5 Your Outcomes From Reading This Book

This book will lead you from being a developer who is interested in deep learning with Python
to a developer who has the resources and capabilities to work through a new dataset end-to-end
using Python and develop accurate deep learning models. Specifically, you will know:
1.6. What This Book is Not 8

How to develop and evaluate neural network models end-to-end.

How to use more advanced techniques required for developing state-of-the-art deep learning
models.

How to build larger models for image and text data.

How to use advanced image augmentation techniques in order to lift model performance.

How to get help with deep learning in Python.

From here you can start to dive into the specifics of the functions, techniques and algorithms
used with the goal of learning how to use them better in order to deliver more accurate predictive
models, more reliably in less time. There are a few ways you can read this book. You can dip
into the lessons and projects as your need or interests motivate you. Alternatively, you can
work through the book end-to-end and take advantage of how the lessons and projects build in
complexity and range. I recommend the latter approach.
To get the very most from this book, I recommend taking each lesson and project and build
upon them. Attempt to improve the results, apply the method to a similar but di↵erent problem,
and so on. Write up what you tried or learned and share it on your blog, social media or send
me an email at jason@MachineLearningMastery.com. This book is really what you make of it
and by putting in a little extra, you can quickly become a true force in applied deep learning.

1.6 What This Book is Not

This book solves a specific problem of getting you, a developer, up to speed applying deep
learning to your own machine learning projects in Python. As such, this book was not intended
to be everything to everyone and it is very important to calibrate your expectations. Specifically:

This is not a deep learning textbook. We will not be getting into the basic theory
of artificial neural networks or deep learning algorithms. You are also expected to have
some familiarity with machine learning basics, or be able to pick them up yourself.

This is not an algorithm book. We will not be working through the details of how
specific deep learning algorithms work. You are expected to have some basic knowledge of
deep learning algorithms or how to pick up this knowledge yourself.

This is not a Python programming book. We will not be spending a lot of time on
Python syntax and programming (e.g. basic programming tasks in Python). You are
expected to already be familiar with Python or a developer who can pick up a new C-like
language relatively quickly.

You can still get a lot out of this book if you are weak in one or two of these areas, but you
may struggle picking up the language or require some more explanation of the techniques. If
this is the case, see the Getting More Help chapter at the end of the book and seek out a good
companion reference text.
1.7. Summary 9

1.7 Summary
It is a special time right now. The tools for applied deep learning have never been so good.
The pace of change with neural networks and deep learning feels like it has never been so fast,
spurred by the amazing results that the methods are showing in such a broad range of fields.
This is the start of your journey into deep learning and I am excited for you. Take your time,
have fun and I’m so excited to see where you can take this amazing new technology.

1.7.1 Next
Let’s dive in. Next up is Part II where you will take a whirlwind tour of the foundation libraries
for deep learning in Python, namely the numerical libraries Theano and TensorFlow and the
library you will be using throughout this book called Keras.
 Part II

Background

10
Chapter 2

Introduction to Theano

Theano is a Python library for fast numerical computation that can be run on the CPU or GPU.
It is a key foundational library for deep learning in Python that you can use directly to create
deep learning models. After completing this lesson, you will know:

About the Theano library for Python.

How a very simple symbolic expression can be defined, compiled and calculated.

Where you can learn more about Theano.

Let’s get started.

2.1 What is Theano?

Theano is an open source project released under the BSD license and was developed by the LISA
(now MILA1 ) group at the University of Montreal, Quebec, Canada (home of Yoshua Bengio).
It is named after a Greek mathematician. At it’s heart Theano is a compiler for mathematical
expressions in Python. It knows how to take your structures and turn them into very efficient
code that uses NumPy, efficient native libraries like BLAS and native code to run as fast as
possible on CPUs or GPUs.
It uses a host of clever code optimizations to squeeze as much performance as possible from
your hardware. If you are into the nitty-gritty of mathematical optimizations in code, check out
this interesting list2 . The actual syntax of Theano expressions is symbolic, which can be o↵
putting to beginners. Specifically, expression are defined in the abstract sense, compiled and
later actually used to make calculations.
Theano was specifically designed to handle the types of computation required for large
neural network algorithms used in deep learning. It was one of the first libraries of its kind
(development started in 2007) and is considered an industry standard for deep learning research
and development.
1
http://mila.umontreal.ca/
2
http://deeplearning.net/software/theano/optimizations.html#optimizations

11
2.2. How to Install Theano 12

2.2 How to Install Theano

Theano provides extensive installation instructions for the major operating systems: Windows,
OS X and Linux. Read the Installing Theano guide for your platform3 . Theano assumes a
working Python 2 or Python 3 environment with SciPy. There are ways to make the installation
easier, such as using Anaconda4 to quickly setup Python and SciPy on your machine as well
as using Docker images. With a working Python and SciPy environment, it is relatively
straightforward to install Theano using pip, for example:
sudo pip install Theano

Listing 2.1: Install Theano with pip.

New releases of Theano may be announced and you will want to update to get any bug fixes
and efficiency improvements. You can upgrade Theano using pip as follows:
sudo pip install --upgrade --no-deps theano

Listing 2.2: Upgrade Theano with pip.

You may want to use the bleeding edge version of Theano checked directly out of GitHub.
This may be required for some wrapper libraries that make use of bleeding edge API changes.
You can install Theano directly from a GitHub checkout as follows:
sudo pip install --upgrade --no-deps git+git://github.com/Theano/Theano.git

Listing 2.3: Upgrade Theano with pip from GitHub.

You are now ready to run Theano on your CPU, which is just fine for the development of
small models. Large models may run slowly on the CPU. If you have a Nvidia GPU, you may
want to look into configuring Theano to use your GPU. There is a wealth of documentation of
the Theano homepage for further configuring the library.

Theano v0.8.2is the latest at the time of writing and is used in this book.

2.3 Simple Theano Example

In this section we demonstrate a simple Python script that gives you a flavor of Theano. In this
example we define two symbolic floating point variables a and b. We define an expression that
uses these variables (c = a + b). We then compile this symbolic expression into a function using
Theano that we can use later. Finally, we use our compiled expression by plugging in some real
values and performing the calculation using efficient compiled Theano code under the covers.
# Example of Theano library
import theano
from theano import tensor
# declare two symbolic floating-point scalars
a = tensor.dscalar()
b = tensor.dscalar()
3
http://deeplearning.net/software/theano/install.html
4
https://www.continuum.io/downloads
2.4. Extensions and Wrappers for Theano 13

# create a simple symbolic expression

c = a + b
# convert the expression into a callable object that takes (a,b) and computes c
f = theano.function([a,b], c)
# bind 1.5 to a , 2.5 to b , and evaluate c
result = f(1.5, 2.5)
print(result)

Listing 2.4: Example of Symbolic Arithmetic with Theano.

Running the example prints the output 4, which matches our expectation that 1.5 + 2.5 = 4.0.
This is a useful example as it gives you a flavor for how a symbolic expression can be defined,
compiled and used. Although we have only performed a basic introduction of adding 2 and 2,
you can see how pre-defining computation to be compiled for efficiency may be scaled up to
large vector and matrix operations required for deep learning.

2.4 Extensions and Wrappers for Theano

If you are new to deep learning you do not have to use Theano directly. In fact, you are highly
encouraged to use one of many popular Python projects that make Theano a lot easier to use
for deep learning. These projects provide data structures and behaviors in Python, specifically
designed to quickly and reliably create deep learning models whilst ensuring that fast and
efficient models are created and executed by Theano under the covers. The amount of Theano
syntax exposed by the libraries varies.
Keras is a wrapper library that hides Theano completely and provides a very simple API to
work with to create deep learning models. It hides Theano so well, that it can in fact run as a
wrapper for another popular foundation framework called TensorFlow (discussed next).

2.5 More Theano Resources

Looking for some more resources on Theano? Take a look at some of the following.

Theano Official Homepage

http://deeplearning.net/software/theano/

Theano GitHub Repository

https://github.com/Theano/Theano/

Theano: A CPU and GPU Math Compiler in Python (2010)

http://www.iro.umontreal.ca/~lisa/pointeurs/theano_scipy2010.pdf

List of Libraries Built on Theano

https://github.com/Theano/Theano/wiki/Related-projects

List of Theano configuration options

http://deeplearning.net/software/theano/library/config.html
2.6. Summary 14

2.6 Summary
In this lesson you discovered the Theano Python library for efficient numerical computation.
You learned:

Theano is a foundation library used for deep learning research and development.

Deep learning models can be developed directly in Theano if desired.

The development and evaluation of deep learning models is easier with wrapper libraries
like Keras.

2.6.1 Next
You now know about the Theano library for numerical computation in Python. In the next
lesson you will discover the TensorFlow library released by Google that attempts to o↵er the
same capabilities.
Chapter 3

Introduction to TensorFlow

TensorFlow is a Python library for fast numerical computing created and released by Google.
It is a foundation library that can be used to create deep learning models directly or by using
wrapper libraries that simplify the process built on top of TensorFlow. After completing this
lesson you will know:

About the TensorFlow library for Python.

How to define, compile and evaluate a simple symbolic expression in TensorFlow.

Where to go to get more information on the Library.

Let’s get started.

Note: TensorFlow is not easily supported on Windows at the time of writing. It may be
possible to get TensorFlow working on windows with Docker. TensorFlow is not required to
complete the rest of this book, and if you are on the Windows platform you can skip this lesson.

3.1 What is TensorFlow?

TensorFlow is an open source library for fast numerical computing. It was created and is
maintained by Google and released under the Apache 2.0 open source license. The API is
nominally for the Python programming language, although there is access to the underlying
C++ API. Unlike other numerical libraries intended for use in Deep Learning like Theano,
TensorFlow was designed for use both in research and development and in production systems,
not least RankBrain in Google search1 and the fun DeepDream project2 . It can run on single
CPU systems, GPUs as well as mobile devices and large scale distributed systems of hundreds
of machines.

3.2 How to Install TensorFlow

Installation of TensorFlow is straightforward if you already have a Python SciPy environment.
TensorFlow works with Python 2.7 and Python 3.3+. With a working Python and SciPy
1
https://en.wikipedia.org/wiki/RankBrain
2
https://en.wikipedia.org/wiki/DeepDream

15
3.3. Your First Examples in TensorFlow 16

environment, it is relatively straightforward to install TensorFlow using pip There are a number
of di↵erent distributions of TensorFlow, customized for di↵erent environments, therefore to
install TensorFlow you can follow the Download and Setup instructions3 on the TensorFlow
website. , for example:

TensorFlow v0.10.0is the latest at the time of writing and is used in this book.

3.3 Your First Examples in TensorFlow

Computation is described in terms of data flow and operations in the structure of a directed
graph.

Nodes: Nodes perform computation and have zero or more inputs and outputs. Data that
moves between nodes are known as tensors, which are multi-dimensional arrays of real
values.

Edges: The graph defines the flow of data, branching, looping and updates to state.
Special edges can be used to synchronize behavior within the graph, for example waiting
for computation on a number of inputs to complete.

Operation: An operation is a named abstract computation which can take input attributes
and produce output attributes. For example, you could define an add or multiply operation.

3.4 Simple TensorFlow Example

In this section we demonstrate a simple Python script that gives you a flavor of TensorFlow. In
this example we define two symbolic floating point variables a and b. We define an expression
that uses these variables (c = a + b). This is the same example used in the previous chapter that
introduced Theano. We then compile this symbolic expression into a function using TensorFlow
that we can use later. Finally, we use our complied expression by plugging in some real values
and performing the calculation using efficient compiled TensorFlow code under the covers.
# Example of TensorFlow library
import tensorflow as tf
# declare two symbolic floating-point scalars
a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
# create a simple symbolic expression using the add function
add = tf.add(a, b)
# bind 1.5 to a , 2.5 to b , and evaluate c
sess = tf.Session()
binding = {a: 1.5, b: 2.5}
c = sess.run(add, feed_dict=binding)
print(c)

Listing 3.1: Example of Symbolic Arithmetic with TensorFlow.

3
https://www.tensorflow.org/versions/r0.9/get_started/os_setup.html
3.5. More Deep Learning Models 17

Running the example prints the output 4, which matches our expectation that 1.5 + 2.5 = 4.0.
This is a useful example as it gives you a flavor for how a symbolic expression can be defined,
compiled and used. Although we have only performed a basic introduction of adding 2 and 2,
you can see how pre-defining computation to be compiled for efficiency may be scaled up to
large vector and matrix operations required for deep learning.

3.5 More Deep Learning Models

Your TensorFlow installation comes with a number of Deep Learning models that you can use
and experiment with directly. Firstly, you need to find out where TensorFlow was installed on
your system. For example, you can use the following Python script:
python -c import os; import inspect; import tensorflow;
print(os.path.dirname(inspect.getfile(tensorflow)))

Listing 3.2: Print Install Directory for TensorFlow.

Change to this directory and take note of the models/ subdirectory. Included are a number
of deep learning models with tutorial-like comments, such as:

Multi-threaded word2vec mini-batched skip-gram model.

Multi-threaded word2vec unbatched skip-gram model.

CNN for the CIFAR-10 network.

Simple, end-to-end, LeNet-5-like convolutional MNIST model example.

Sequence-to-sequence model with an attention mechanism.

Also check the examples directory as it contains an example using the MNIST dataset.
There is also an excellent list of tutorials on the main TensorFlow website4 . They show how
to use di↵erent network types, di↵erent datasets and how to use the framework in various
di↵erent ways. Finally, there is the TensorFlow playground5 where you can experiment with
small networks right in your web browser.

3.6 Summary
In this lesson you discovered the TensorFlow Python library for deep learning. You learned:

TensorFlow is another efficient numerical library like Theano.

Like Theano, deep learning models can be developed directly in TensorFlow if desired.

Also like Theano, TensorFlow may be better leveraged by a wrapper library that abstracts
the complexity and lower level details.
4
https://www.tensorflow.org/versions/r0.9/tutorials/
5
http://playground.tensorflow.org/
3.6. Summary 18

3.6.1 Next
You now know about the Theano and TensorFlow libraries for efficient numerical computation
in Python. In the next lesson you will discover the Keras library that wraps both libraries and
gives you a clean and simple API for developing and evaluating deep learning models.
Chapter 4

Introduction to Keras

Two of the top numerical platforms in Python that provide the basis for deep learning research
and development are Theano and TensorFlow. Both are very powerful libraries, but both can
be difficult to use directly for creating deep learning models. In this lesson you will discover
the Keras Python library that provides a clean and convenient way to create a range of deep
learning models on top of Theano or TensorFlow. After completing this lesson you will know:

About the Keras Python library for deep learning.

How to configure Keras for Theano or TensorFlow.

The standard idiom for creating models with Keras.

Let’s get started.

4.1 What is Keras?

Keras is a minimalist Python library for deep learning that can run on top of Theano or
TensorFlow. It was developed to make developing deep learning models as fast and easy as
possible for research and development. It runs on Python 2.7 or 3.5 and can seamlessly execute
on GPUs and CPUs given the underlying frameworks. It is released under the permissive MIT
license. Keras was developed and maintained by François Chollet, a Google engineer using four
guiding principles:

Modularity: A model can be understood as a sequence or a graph alone. All the concerns
of a deep learning model are discrete components that can be combined in arbitrary ways.

Minimalism: The library provides just enough to achieve an outcome, no frills and
maximizing readability.

Extensibility: New components are intentionally easy to add and use within the frame-
work, intended for developers to trial and explore new ideas.

Python: No separate model files with custom file formats. Everything is native Python.

19
Exploring the Variety of Random
Documents with Different Content
“Oh, but you must have a jam sandwich,” cried Morris with the
pseudo-heartiness characteristic of such occasions.
“Well—if you won’t all think me fearfully, fearfully greedy——”
Minnie hesitated and looked wildly round her, but as no one
appeared in the least aghast at the prospect of her depredations
among the jam sandwiches, she deprecatingly took the smallest one,
murmuring, “Thank you muchly—this is fearful gluttony—‘just one
more crust,’ as the boy said on the burning deck.”
The spasmodic conversation died away.
Presently Hazel said:
“I’ve found the place where we got that white heather last year,
mother. There are some more roots there, if you want to take them
home for the rock garden.”
“Come on and let’s dig then,” said Bertha vigorously, rising as she
spoke.
Morris shot Hazel a glance of gratitude.
He longed to be alone with Rosamund, even while thinking that he
was dreading the pain of bidding her good-bye.
He looked at Miss Blandflower, but Hazel Tregaskis was quicker than
he.
“I shan’t find the way without you,” she declared lightly.
“Come on, Minnie,” shouted Mrs. Tregaskis, already well on ahead.
“There’s no rest for the wicked,” said Minnie mechanically, and went.
Rosamund’s first words were not at all what Morris had expected.
She looked at him sombrely, and remarked almost violently:
“Do you know what’s the matter with Frances? Is Cousin Bertie really
frightened about her?”
“No, not seriously, I don’t think,” he answered, instinctively anxious
to soothe her. “She only said that if Frances wasn’t quite well again
next week she wouldn’t go to Scotland, but would send you and
Hazel alone.”
“I shan’t go if Francie is ill.”
He looked at her, astounded.
“But, Rosamund, what’s the matter? She isn’t ill. Mrs. Tregaskis
herself said that a temperature didn’t mean anything at all with
Frances.”
“Oh, you don’t understand,” she burst out angrily. “Nobody
understands in the least what Frances is to me. Cousin Bertie has
never understood, and never will. You heard what she said just
now.”
He had forgotten.
“That I’m not to go near Frances till to-night. She always treats me
like a child.”
She looked very like one indeed, as she spoke, flushed and
indignant.
“Perhaps Frances was going to sleep, and doesn’t want to be
disturbed.”
“As though I should disturb her! Why, I’ve looked after her ever
since she was a little girl—until we came to live here. Now,” said
Rosamund bitterly, “I’m told to mind my own business and let
Frances mind hers.”
“Never mind,” consoled Morris. “Don’t let’s talk about it. I want to tell
you something, Rosamund.”
Her angry face softened a little, but she seemed unable to dismiss
the subject.
“Nobody has ever understood about Frances and me—ever. I feel
more as though she were my child than my sister.”
Morris was becoming heartily tired of the discussion, and showed
distinct traces of that fatigue in his tone, as he replied perfunctorily:
“Of course I understand—but, really, she’s only three years younger
than you are, isn’t she?”
“Cousin Bertie is always harping on that, and telling Frances not to
be domineered over!”
“Rosamund!” cried Morris, “you really talk as though Mrs. Tregaskis
was always being unkind to you. I can’t understand you. Why, she
simply adores you both—just as though she were your mother.”
He was totally unable to understand why Rosamund, at this, turned
the fury of her eyes full upon him.
“You don’t understand, any more than anyone else.”
“Don’t understand what?” almost shouted Morris. “I don’t
understand you, when you talk like that.”
Nor did he. She seemed to him altogether unbalanced, and as
different as possible from the stately, wonderful Rosamund whom he
had met in the orchard at Porthlew.
“Why do you speak as though Mrs. Tregaskis was unkind, or
unsympathetic?” he asked more gently. “She is devoted to you. You
can’t think how proud she is of you, Rosamund.”
“I’m not her daughter.”
“She feels as though you were. She told me so herself.”
“I wish you hadn’t let her talk to you about me at all,” said
Rosamund unhappily.
“I don’t think you’d say that if you knew how nice and understanding
she was. I—I wish I could explain better.”
Morris felt the impotence of his lame and stammering words before
the deep hostility, which he recognized, although he was at a loss to
account for it, in Rosamund’s silence.
“I haven’t ever told anyone,” she said at last, stammering a little,
“but I’ve always resented being told that Cousin Bertha has done
everything for us and is so fond of us. Of course it’s quite true in a
way, but she’s never made me happy—or Francie either.”
If Morris thought that the fault lay more on Rosamund’s side than on
her guardian’s, he would not say so, but his too expressive face
betrayed him to Rosamund’s quick perceptions.
“You think I’m ungrateful—but I do recognize all the material things
she’s done for us.”
Morris thought her explanation very ungracious, and then chid
himself half-heartedly for criticizing his goddess.
“She’s done more than material things, hasn’t she?” he reminded her
gently. “It’s not as though Porthlew had been an alien atmosphere.
She cares about all the things that matter—books and music and
friendship and other things too. That’s what makes her so wonderful,
I think—that she should have that side to her, as well as the splendid
practical capable side that everyone can see and admire.”
Rosamund looked at him, with a face that seemed to have grown
weary.
“Yes, of course,” she said slowly.
Morris felt, unreasonably, as though he had been weighed and found
wanting, in the balance of that baffled, tired gaze of hers. He
reflected with bewilderment that although she had looked at him like
a child when she had spoken defiantly and angrily of her guardian,
she now looked very much older, and more unhappy.
“What is it, Rosamund?” he asked, half involuntarily, and conscious
of the futility of the question.
“Oh, I don’t know,” she said drearily.
It was the discontented child again.
Morris remained silent, plucking at the tough strands of heather all
round him.
He felt injured.
He had come out on the moor prepared to sacrifice himself, to bid
Rosamund a long farewell, and to take away with him only the
memory of that bitter-sweet parting hour. Surely the intuition of love
should have met him more than halfway. But Rosamund, with
childish perversity, had harped upon the string of her own
grievances, grievances which Morris could not but feel to be for the
most part imaginary ones. She was not thinking about him at all,
and all his wealth of love and self-sacrifice had gone unheeded.
Morris began to feel angry, and, worse still, as though he were being
made a fool of in his own eyes.
It did not calm him to reflect that he would probably appear in
exactly the same light to the penetrating gaze of Bertha Tregaskis.
She was even now advancing slowly towards them, stooping every
now and then to prod at some little root or plant and pull it up into
her capacious basket.
Morris got up abruptly.
“Rosamund, do you know that I’m going away?”
She looked almost as much startled as he could have wished.
“When, Morris? Where?”
“At once,” he said gloomily. “I don’t know where—or care.”
He had meant to ask her if she would “wait for him” in the time-
honoured phrase, but he had not reckoned on having to cram the
whole parting scene, as it were, into the last three minutes of his
interview.
Rosamund also looked at Bertha’s advancing form and spoke rapidly.
“I didn’t know you meant to go away, Morris.”
Was her voice trembling a little?
“I didn’t!” he cried passionately.
Bertha hailed them with a prolonged “coo-ee” that might have been
regarded as superfluous in view of the fact that only some rapidly
diminishing hundred yards now lay between them.
“I didn’t,” repeated Morris earnestly, and was unable to resist
adding, “but—it’s the only way.”

He also made use of that excellent phrase, for which he was

beholden to Mrs. Tregaskis, in conversation with his mother that
evening.
It was more than wasted upon her.
“I don’t know what you mean by ‘the only way,’” she returned with a
sudden irritating assumption of common sense, her lack of which
she habitually dwelt upon with pensive complacency.
“If you want to go yachting, Morris, well and good; but don’t talk in
an affected melodramatic style, as though you were making some
great sacrifice in going, please. It doesn’t ring true, and you know
how I hate little insincerities.”
Nina’s assault was perhaps not utterly unprovoked. A certain jutting
forward of her son’s jaw, a tendency to monosyllabic replies
preceded by the slight start of one roused from a profound reverie,
had conveyed to Nina all too accurately that Morris was enacting, in
his own opinion, the rôle of jeune premier in a drama of self-
sacrifice.
“I’ve already told you that you can start on this yachting trip
whenever you please, so why talk as though it were some
tremendous decision which you had just come to?” she demanded
irritably.
Morris smiled with a superior expression.
“You don’t understand, mother,” he told her, with a touch of
compassion.
Few remarks were more calculated to rouse her annoyance.
“My dear boy, it’s perfectly childish to talk like that. How can there
be anything about you which I, your mother, can’t understand? It
makes one realize how very very young you are, when you talk like
that.”
But even allusions to his youth could not disturb Morris’s exalted
mood.
He was unable to resist giving his mother a hint of the heights to
which he had attained.
“I was up at Porthlew this afternoon,” he said in a meaning tone.
“So I supposed. You always come back in this silly, self-satisfied
frame of mind when you’ve been with those girls, who naturally play
up to your vanity. If that’s the effect of the Grantham girl’s influence,
Morris, the less you see of her the better, for your own sake.”
The fatal word “influence,” combined with the preposterous
implication that Nina had slightingly forgotten Miss Grantham’s very
name, roused Morris to anger at last.
“Rosamund Grantham and I have said good-bye, mother. It was the
only way. Some day I shall come back to her, and find her waiting,”
said Morris, considerably worked up by the pathos of his own
eloquence, and momentarily forgetful that he had received no such
pledge. “But you make it impossible that I should tell you anything
of what I am going through, when you speak as you did just now.”
He walked with sorrowful dignity to the door, confident that his
mother would not allow him to leave the room without giving him
further opportunities for rhetoric.
Nina, in effect, finding herself driven to her last resort, with a
readiness born of much experience, began gently to cry.
“Darling, you know I didn’t mean it if I spoke impatiently. I only
want to sympathize with you and comfort you.”
He turned slowly towards her.
She was deeply relieved that the affaire Rosamund should have been
successfully tided over. Morris was far from being as heartbroken at
the idea of parting from his love as he had been before their final
interview, and the evening passed amid a harmonious rendering of a
strong man’s grief and his mother’s tender sympathy.
Preparations for his journey absorbed Morris for the next twenty-four
hours, during which he and his mother enjoyed the sense of perfect
companionship which was always theirs on the rare occasions when
their respective mental tableaux vivants of one another happened to
coincide, and then he was off.
“Good-bye, my darling boy. Enjoy yourself.”
“Thank you, mother dear. Write to me and”—his voice took on the
slightly deeper note consecrated to the strong-man-in-grief attitude
—“tell me any news of her.”
“Yes, dearest, of course,” tenderly replied Nina, but she refrained
from telling him the only piece of news which transpired during the
next few days: that Frances was not well enough for Mrs. Tregaskis
to leave her, and that Rosamund had refused to accompany Hazel to
Scotland, but remained with her guardian at Porthlew.
“It is tiresome of her,” said Bertha, in a tone more nearly resembling
annoyance than she often used.
“Frances isn’t seriously ill at all, and if she were Rosamund would be
the worst possible person for her. She goes about looking like a
tragedy-queen, as though Frances were at death’s door.”
“Why on earth did you let her stay?” said Nina with more derision
than sympathy in her voice.
“She asked Frederick. You know how tiresome and contradictory he
can be, and of course he knew perfectly well that I didn’t want
Rosamund fussing and fretting on my hands, but he said she could
do as she liked. He always takes up an absurd attitude of having no
authority over those two, as you know.”
“I know. So Hazel has gone alone?”
“I’ve had to send my maid with her, though I should have done that
in any case. I don’t approve of young girls travelling about all over
the country by themselves.”
“Lucky for you that you have girls who can be chaperoned! Look at
poor little me—I can’t run after Morris, let alone send a maid with
him, and have to sit here with a trembling heart, wondering all the
time how things are going with him.”
“That’s always the way with a son, my dear, or a husband either,”
said Bertha, determinedly emphasizing the fact that she, although
not the mother of a son, also possessed a male appendage.
“It’s our part just to sit at home and work and wait, while they have
all the fun,” Nina sighed. “A woman’s life is one long self-sacrifice,”
she murmured.
“It is, when one has to mend and make and nurse, and all the rest
of it,” cordially agreed Bertha, with one fleeting glance at Nina’s
exquisite, empty hands, folded in her lap.
The glance was not lost upon Mrs. Severing, who presently said
reflectively that Mr. Bartlett would no doubt call upon her shortly
with some of his interminable business questions, and she must ask
dearest Bertie to forgive her. It was not her way to put off a matter
of business.
“Unpractical, dreamy creature that I am,” said Nina with a sad,
sweet smile, “I have had too many years’ hard training in looking
after this big estate, ever to be unbusinesslike. Mr. Bartlett always
amuses me so much when he will say that I should make a better
agent than he does.”
“I don’t wonder!” exclaimed Bertha, the dryness of her tone making
it abundantly evident that her emphatic assent was directed towards
Nina’s amusement, and not towards Mr. Bartlett’s opinion of his
employer’s abilities. “No, no, dear. You must stick to your charming
songs. They’re your work in the world,” smiled Bertha tolerantly.
“Dear Bertie! How sweet of you to say so. I’m always afraid of being
just some silly, trivial flowery thing—not of any real use in the
world.”
“The world needs its little speedwell flowers just as much as its
sturdy oak-trees,” laughed Bertha tenderly.
“Yes, dear,” said Nina deftly. “There is room for Mary as well as for
Martha. It always comforts me to remember that.”
Comfort, however, was not the predominant expression on the face
of Mrs. Tregaskis as she heard her friend’s favourite Scriptural
parallel once more enunciated.
“If you’re really waiting for Mr. Bartlett, darling, I mustn’t keep you,”
she said rather hastily. “Anyhow, I must get back to my invalid. She’s
much better to-day, and only fretting at the idea of my having
missed the Scotch visits. Of course one had been rather longing for
a breath of Scottish air, this weather, but I dare say I shall manage
without. It’s an economy, at all events.”
She gave her cheery, plucky laugh.
“How is Morris enjoying Norway? Has he got over his love-lornity?”
Nina laughed a little.
“I think he has. I’ve had a very cheery letter from him, raving about
the fiords and things.”
Bertha looked slightly puzzled.
“The——? Oh, you mean the fjords! Yes, of course they must be
perfectly gorgeous at this time of year,” she remarked thoughtfully,
with the air of a connoisseuse.
“They are just the same at any time of year, dear,” sweetly returned
Nina. “Geoffrey and I went there for a fortnight once—it seems oh
so long ago! It somehow made one think of those far-away days
when everything was couleur de rose——”
There were few topics that Bertha enjoyed less than the
retrospective couleur de rose of her friend’s married life, and she
hastily dragged the conversation back into the living present.
“I’m so very glad about Morris. Give the boy my love when you
write. I wish Rosamund was half as sensible as he is. She goes
mooning about the place as though she’d lost her dearest friend.”
Bertha gave a slightly apologetic laugh at her own acerbity, and
Nina, whose regard for Rosamund always waxed in proportion as her
friend’s waned, murmured with the air of a compassionate angel:
“Poor child! One remembers the heartaches of one’s own youth. The
thoughts of youth are long, long thoughts, Bertie!”
“Well, Morris appears to have curtailed his successfully enough, at all
events,” crisply returned Bertha. “I always said there was stuff in the
boy, Nina, although you’ve spoilt him so outrageously.”
Nina laughed, and kissed Mrs. Tregaskis affectionately as they said
good-bye.
It always pleased her to be told that she spoilt Morris. She had
consistently over-indulged him as a little boy, and did so still in all
matters where his personal pleasures were concerned, provided that
they did not interfere with her wishes. The accusation of spoiling
seemed to add colour to her frequently-voiced conviction that youth
was very hard, and that a mother’s sacrifices often went unheeded.
“I’m afraid I have spoilt him,” she sighed in response to Bertha’s
words. “But after all, Morris has been my only thought for so many,
many years....”
Bertha told herself that really poor Nina was sometimes positively
maudlin, and firmly created a diversion by demanding the loan of
Nina’s seldom-used garden scissors.
“At all events,” she told herself, as she walked briskly away, “I
managed to forestall an allusion, for once, to poor Geoffrey. And
now for my little tragedy-queen!”
But Rosamund, though not undeserving of her guardian’s epithet,
gave less trouble than Bertha had anticipated. With characteristic
want of balance, she was absorbed in one thought only: that of her
sister. As long as Frances remained ill, Rosamund gave little thought
to Morris Severing. Perhaps the measure of her undeveloped lack of
proportion might have been probed by that fact. The memory of a
spoilt illusion might come to vex and grieve the youthfulness of her
spirit later, but that would only be when the nearer, and to her
infinitely more real, solicitude had ceased to be.
And Rosamund, her outlook being honest, knew, and was to know
more clearly yet, that her first love had brought her no nearer to
that reality which lies at the back of all wisdom, and which for her
was still typified by her love for Frances.
 VIII
“ROSAMUND!” wrote Hazel from the North. “The most
marvellous things in all the world are happening. I am in
love—with a man who wears an eyeglass—(you know how
I’ve always hated an eyeglass) and he is in love with me.
He is Sir Guy Marleswood, and he’s thirty-four, and quite
six foot, and I don’t think I should mind if he were five
foot nothing. I know I shouldn’t. I’ve known him a
fortnight, and yet we both feel as though we’d known
each other all our lives, and yet it’s new and wonderful all
the time. It’s indescribable. There’s one thing—which I
have to keep reminding myself of, but which will assume
enormous proportions as soon as one begins to do
anything—I mean, write to mother, or wear an
engagement ring. (He’s given me a most beautiful one, a
ruby marquise, only I won’t wear it.) He’s been married
before, and he had to divorce his wife five years ago. I
knew it before we met, because the girls here had been
talking about him, and said that was why their mother had
not asked him to stay in the house, but he came to the
dance, and he is staying at the Ludleys’, a mile away.
That’s where we met, and I’ve seen him nearly every day
since—and the days when I don’t see him are just hell—
only knowing that Heaven may open again at any minute.
“Rosamund dear, I know now that I was a fool ever to let
boys make love to me or propose in the sort of half-and-
half way that a boy does—asking one to wait for him
because he may have enough to marry on in fifteen years’
time, and meanwhile exchange photographs and write
every Sunday afternoon. You know the sort of thing—that
does to tell other girls about, and sentimentalize over
 when a waltz that you used to dance with him is being
played. But when it’s the real thing—when a man tells you
that he cares for you and asks you to be his wife—it’s
absolutely and utterly different. Guy asked me the fifth
time he saw me. He told me about his wife first. The odd
thing is, that I don’t mind. Of course I shouldn’t mind
about the moral part of it, anyhow—I mean whom God
hath joined together and all that—but I don’t seem even
to mind about his having once loved her and married her.
They only cared for one another a very little while, and it’s
all past and over. The present is ours—and more glorious
and wonderful than any words can ever say. As for the
future—he says he is going to marry me before the end of
the year. And I am to put off my other visits and come
home this week, and then he will write to daddy, and
come down to Cornwall. Of course it isn’t daddy that
counts at all, since I can manage him perfectly, but I have
a sort of an idea that Guy will get exactly what he wants,
even out of mother. He’s the sort of person that does.
“We haven’t told anybody anything. I haven’t the slightest
doubt that Lady Alistair has guessed, and the girls too, but
even if she writes to mother it’ll only bring things to a
crisis rather sooner. I’m writing to her myself this evening,
so she’ll know by the time you get this.
“I’m not afraid of anything or anyone in the world. Guy
and I have found one another, and nothing else matters.
Besides, I know he’ll manage everything!
“As ever,
“Your loving
“Hazel.”
If Hazel’s letter brought a strange wondering sense of disquietude to
Rosamund, and that not wholly on her cousin’s account, the much
shorter note which she had sent her mother apparently produced no
such effect. Bertha appeared at luncheon with a brow but slightly
corrugated, and only an added tinge of briskness in her manner to
betray perturbation of spirit.
“I see you’ve had a letter from Hazel by the midday post,
Rosamund,” observed Miss Blandflower in the middle of luncheon,
with a praiseworthy desire to dissipate the slight atmosphere of
constraint which had lately been noticeable at meals, in spite of the
valiant and hearty efforts of Mrs. Tregaskis. “When does she return
to the bosom of her family?” She gave a slight giggle in lieu of
quotation marks.
Rosamund hesitated, felt her cousin Bertha glance sharply at her,
and answered nervously:
“I’m not sure—soon, I think.”
“She has two more visits to pay,” said Mrs. Tregaskis coldly. “You
knew that, Rosamund.”
Her husband looked up suddenly.
“She’s coming back on Friday. You knew that, Bertha,” he said
mockingly. “I understand that our parental sanction is required to an
engagement of marriage. Very gratifying, I’m sure, in these
emancipated days.”
Miss Blandflower turned startled eyes from one to the other of
Hazel’s parents.
“They say one wedding makes another,” she sighed with nervous
inappositeness. “But is it really—who is it...?”
“My dear Minnie, Hazel is a silly little girl who ought never to have
been allowed to pay visits without a chaperoning mamma. It serves
me right for having relaxed my rule—but one can’t be in two places
at once, and really these young ladies require such a lot of looking
after!”
Bertha sighed gustily.
“One only wonders how you can manage in the marvellous way you
do, with so much upon your hands,” said Minnie, feeling that this
remark, although far from being original, came at any rate from a
safe stock, and might be more acceptable than further questions.
At all events it steered the conversation into smoother channels, and
no further allusions were made in public to Hazel’s affairs, until three
days later, when Hazel herself returned to Porthlew.
Rosamund was instantly conscious of an indefinable change in her
cousin.
Self-possessed Hazel Tregaskis had always been, but the youthful
security of her manner had somehow deepened into an impression
of inward assurance that held less of self-confidence, and more of
some larger stability, that would not be easily shaken. When her
mother greeted her with matter-of-fact warmth, and said gaily,
“Well, my little girl, I’m glad to have you under my wing again; I
think it’s the last time we must let you go gallivanting off on your
own for the present,” Rosamund saw that Hazel did not give the
petulant shrug or grimace with which the girl Hazel would have
received such a greeting, but looked at her mother with a strange,
remote look that held something of an almost impersonal
compassion.
It was that same look, Rosamund thought, which angered Mrs.
Tregaskis when her daughter resolutely asked her for an interview
that evening.
“No, my darling; I’m not going to let you stay and chatter now.
You’ve had a long journey, and must pop off to bed early. We’ll have
a long talk to-morrow. Dad and I are not at all angry with you, but
I’ve had a letter from Jessie Alistair, and it’s quite plain that I ought
never to have let you go and stay away without me. Now run along
with Rosamund, my pet.”
“What did Lady Alistair say?”
“I shall talk to you about that to-morrow. I am not at all angry with
you, Hazel, but one thing you and Rosamund may as well
understand, since I suppose you’ve told her all about it. You may flirt
with boys of your own age, if you like, and have all the fun that’s
natural and proper, but——” Bertha Tregaskis paused. She spoke
with a quiet and good-humoured implacability, her hands resting on
her broad hips, and her resolute mouth set firmly. “But—to flirt and
get yourself talked about with a married man, is—a—thing—I—don’t
—allow. See, darling?”
Rosamund caught her breath and looked at her cousin. Hazel, who
seldom or never blushed, had flushed the slow, deep crimson of a
woman who hears herself insulted.
“Sir Guy Marleswood is not a married man,” she said slowly. “At
least, neither he nor I think so, which is what matters, after all. He
divorced his wife five years ago. He has asked me to marry him.”
“Very well, darling. When he writes and asks the permission of your
parents, we shall see. But a man of four- or five-and-thirty, who has
led the sort of life that he has led, does not generally want to marry
a little girl of nineteen, even though he may be dishonourable
enough to play at making love to her.”
But this agreeable theory was shattered next day, when Sir Guy
Marleswood wrote a formal statement of his position, and an almost
equally formal request for his daughter’s hand in marriage, to
Frederick Tregaskis. He also stated unemphatically that the following
day would find him at Porthlew Railway Hotel.
Thereafter, Rosamund watched the storm break over the household
with a strangely aching heart.
Bertha regarded Sir Guy as a married man, and said so staunchly.
Frederick Tregaskis, whom Rosamund had never yet heard to agree
with his wife, declined to view the question from an ethical
standpoint, but declared Hazel too young to enter upon a marriage
which would of necessity be regarded more or less dubiously by the
world in general.
“Wait another five years,” he remarked grimly to his daughter, “and
see if you can’t do better for yourself than a divorced baronet fifteen
years older than yourself.”
“No,” said Hazel, her small face set like a flint. “He wants me to
marry him now.”
“I dare say. And I want you to wait. I suppose you owe something to
your father?”
“Yes,” she said, and began to cry. “But not everything in the world. I
owe something to myself. It’s my life.”
It was the passionate cry for individualism that Rosamund had heard
from Morris Severing.
But Hazel Tregaskis, unlike Morris, was directing all the energies of
her will into one channel. And Rosamund, watching, saw those
energies guided and strengthened day by day by the stronger force
that held steadfast behind her.
Guy Marleswood was not of those who fail.
Before the close of that year, the day came when he extorted from
the exasperated Frederick: “Marry her, then. I see you mean to do it,
both of you, and it may as well be with my consent as without it.
Anything to put an end to the subject.”
“Thank you, sir,” said Sir Guy imperturbably. “I will go and tell Mrs.
Tregaskis that we have your consent to the marriage.”
“You will do nothing of the kind. I shall tell her myself. I may as well
get some satisfaction out of it,” said Frederick viciously.
He sought his wife in the library, where she sat, looking unusually
disheartened, amid a pile of leaflets.
“Bertha, you are about to be relieved of one of your responsibilities.”
“I’m thankful to hear it,” she returned wearily.
“I have decided to give Hazel into Marleswood’s keeping.”
“Frederick! You can’t. You’re mad. A child of nineteen—and a
marriage that’s no marriage—she’ll be no more married in the eyes
of God than if she were openly living as that man’s mistress.”
“I’m not concerned with the eyes of God,” said her husband in
detached tones. “It’s perfectly evident to mine that if we don’t give
our consent they mean to do without it, and I don’t choose to have
my daughter making a runaway match. We had better give in
gracefully while it is still possible, Bertha. Marleswood is not the sort
of man to heal a breach, if it came to that.”
“What do you mean?”
“I mean that we don’t want to be cut off from the little girl for ever
after her marriage,” said Frederick, his voice shaking a very little.
“That’s what it’ll mean if we let her go from under our roof in
defiance, Bertha.”
“Hazel is an infatuated, self-willed child, but she is not heartless,”
cried Bertha.
“I do not intend to put her to the test.” Frederick Tregaskis had
regained his habitual dryness of utterance.
With unwonted consideration, he added a word of consolation for his
wife.
“I may as well tell you that I am perfectly satisfied that Marleswood
is a good fellow in every way, and devoted to her. The whole thing,
after all, amounts to a question of conscience, which she is entitled
to judge for herself.”
“She’s not,” flashed Bertha. “She’s only a child, and ought to accept
the ruling of her parents until she’s old enough to judge for herself.”
“I have no doubt,” said Frederick drily, “that all parents, taken as a
class, would agree with you. Unfortunately for ourselves, however,
we have passed into an era where the individual, and not the class,
will rule.”
He walked out of the room, looking older and more deeply lined than
ever.
Rosamund found Mrs. Tregaskis, who never broke down, weeping
violently among the piles of disordered pamphlets.
“Cousin Bertie! Don’t!” cried Rosamund fearfully. “Is it about Hazel?”
Bertha raised a piteously mottled and disfigured face.
“I’m beaten,” she cried. “Frederick has consented to this iniquitous
marriage, and nothing can stop it now. My little girl, whom I’ve
brought up to be good, and to whom I’ve tried to teach religion—
that she should be willing to break my heart, and rush deliberately
into sin, the first time temptation comes near her!”
“No—no. It’s not that. She doesn’t think it’s sin. She doesn’t believe
it’s sin—not for an instant. Her point of view is different.”
“Her point of view!” cried Bertha bitterly. “How dare you talk to me,
a woman of fifty, of such preposterous nonsense? You and she are
children; you know nothing of life, you’ve had no experience. How
can Hazel judge of what is right or wrong? She’s a child—a child.”
In the vehement repetition of the assertion, it seemed to Rosamund
that she found her clue to Bertha Tregaskis’s impotent suffering. She
would not, could not, admit in her daughter any claim to the rights
of an individual.
Hazel’s judgment, unrecognized by her mother, carried with it no
amazement to Rosamund.
Certain faiths, certain scruples and acceptances inborn in Rosamund
and Frances, had been the veriest lip-service to the child Hazel
always. Rosamund recognized in her the purest and most natural
type of highly-evolved paganism.
“You know, Rosamund, I’m not doing anything wrong, although they
won’t believe it. It isn’t wrong to me, and I don’t believe in an
abstract right and wrong. Each individual case has its own laws.”
“Should you do it if you thought it was wrong?”
“I don’t know,” said Hazel thoughtfully. “I can’t imagine seriously
believing that it would matter to God, one way or the other. Should
you? Frances wouldn’t, one knows.”
“If I did it,” slowly said Rosamund, “it would be as a deliberate
choice between good and evil. I should believe myself to be breaking
God’s law—but I might do it, if I thought it worth while.”
She knew that if, as she said, it seemed to her worth while, no laws
of God or man should bind her. But she would break them of
deliberate intent, whereas to Hazel Tregaskis they were non-
existent, myths designed for the wanton frightening of children.
Rosamund recognized the absolute sincerity of Hazel’s point of view,
and sometimes found herself wondering what Sir Guy’s might be.
One day, very soon before the marriage, she held an odd little
conversation with him, standing in the wintry sunshine of the
terrace. Frederick Tregaskis was ahead of them, grimly poking with a
walking-stick at a little drain that was choked with leaves.
“He’s been very kind to me,” said Sir Guy abruptly, indicating with a
gesture the odd little figure.
“I think that he really likes you very much,” said Rosamund. “And
though he would be very angry at being told so, I have always
known that Cousin Frederick adores Hazel.”
Sir Guy nodded with full comprehension.
“Yes, of course. She knows that, too. It’s been the best thing in her
life so far—that and having you and your sister here.” He paused for
a moment or two. “You know,” he said slowly, “I want to try and
make up to her for everything that she hasn’t had, so far. She ought
to have everything. She seems, somehow, so made for happiness.”
“I have never seen Hazel sad,” said Rosamund, rather surprised. “I
think she is happy by nature.”
“Yes, though an atmosphere which might perhaps seem an
unsympathetic one——”
He left the sentence unfinished, and it required no effort on
Rosamund’s part to conjecture his meaning. Sir Guy resented, none
the less implacably that his resentment was expressed by implication
only, the attitude of Mrs. Tregaskis towards her daughter. That Hazel
herself had never resented it, and had only opposed to it the bright
glancing hardness of her impenetrable self-confidence, did not,
Rosamund felt, in any way diminish his perfectly silent ire. Mrs.
Tregaskis herself would be forced to recognize that in this man
fifteen years her senior, Hazel had found champion as well as lover,
knight as well as comrade.
Rosamund turned away with an aching heart, wondering dimly
whether her need had not been greater than Hazel’s.
After the formal consent given by Frederick Tregaskis, there had
been no further discussions between Sir Guy and Mrs. Tregaskis. She
accepted her defeat with the sort of grim gallantry that would
always be characteristic of her, and, as far as Rosamund knew,
attempted no appeal to Hazel. But she aged more perceptibly in the
weeks before Hazel’s marriage than during all the five years that
Rosamund had passed at Porthlew.
No other indication that her guardian recognized defeat was evident
to Rosamund’s eyes. Her manner to her daughter was what it had
always been—kindly, authoritative, at times possessive. She
admitted Sir Guy’s claims to much of her daughter’s time, and even
seemed disposed, gradually, to concede to him rights which he had
not tried to arrogate for himself.
“You mustn’t let this little person be too much in London,” she
observed, with a hand upon Hazel’s shoulder. “We’re very excitable,
and it knocks us up. I had to be a very strict mamma and bring her
home long before the dances had come to an end last year.”
“If we take the St. James’ Square flat, there is no reason why we
shouldn’t spend all the week-ends Hazel likes at Marleswood.”
“Well, I don’t know about week-ends,” said Bertha doubtfully.
“They’re not very restful. I think a home in the country and an
occasional fling in London must be Hazel’s programme.”
She spoke with her customary matter-of-fact assurance and kindly
good sense.
Sir Guy fixed his objectionable monocle more firmly.
“That,” he observed in a detached manner, “is a decision which I
shall leave entirely to my wife.”
If Mrs. Tregaskis found it necessary to readjust her forces after this,
the readjustment was made silently and without delay. But it was
very shortly after that, when it only wanted a week to Hazel’s
wedding-day, that Rosamund again found Cousin Bertha in the
library, struggling with hard, choking sobs. Hazel hung over her,
caressing her with most unwonted demonstrativeness and with tears
in her own pretty eyes. But that they were tears of the merest
surface pity and tenderness was abundantly obvious even without
the gently mournful observation which she made to Rosamund that
evening.
“Poor mother! I hate to see her minding it so, but you know,
Rosamund, I can’t feel as unhappy as I ought.”
“Don’t you wish—sometimes—that you’d waited, as they begged you
to? It would have been the same for you in the end.”
“The same for me, and the same for them,” returned Hazel crisply.
“They wouldn’t have liked it any better ten years hence—at least
mother wouldn’t. I believe daddy’s reconciled already. Mother wants
me to be happy, but in her way.”
“Are you really happy, when you know she is miserable?” spoke
Rosamund with more curiosity than compassion. Hazel coloured, but
faced her cousin with unflinching honesty.
“Yes,” she said, “I am. It’s of no use to pretend, Rosamund. I am
happier than I have ever been in my life. Of course, I should have
preferred it if everything had been straightforward, and there hadn’t
been all this fuss, and having to extort a consent—but it would have
been just the same if they hadn’t given it. Do you know, that’s the
pathos of it, to my mind—they couldn’t do anything. Guy and I
would have married without their consent, just as much as with it.”
“He asked you to, I suppose,” said Rosamund, as though stating a
fact.
Hazel pushed her curling tawny hair from her forehead.
“He asked me if I would, if it came to that, and of course I said yes.
But we both knew it wouldn’t come to that, and that mother would
have to give in. I used to think that if one’s parents forbade a thing,
it became impossible ipso facto, but it doesn’t. They just can’t do
anything at all.”
To Rosamund, Hazel had summed up the situation in that sentence.
They could not do anything at all.
The wedding took place quietly at Porthlew, and they said good-bye
to Hazel, radiant-eyed, and clinging in an unwonted embrace to her
father at the last moment.
Then she drove away with her husband, and Miss Blandflower, in a
piping soprano, remarked to Rosamund:
“It’s like a death in a house, isn’t it? But we must all try and take her
place, now.”
The suggestion drove Frederick, snarling disgustedly, into the study.
Frances went quietly to put away some of the litter in Hazel’s room,
while Rosamund, feeling herself useless and in the way, yet hung
helplessly in the vicinity of Nina Severing, who had remained with
Bertha in the drawing-room after the departure of the few guests.
But no word of Morris reached her.
Nina was murmuring consolation to her friend who, for once
inactive, sat gazing heavily into the fire.
“After all, dearest, the young birds will fly out of the old nest and
leave it desolate. It’s nature.”
Bertha groaned.
“It’s not the selfish loss to myself that I mind, Nina, but the thing
she’s done. If I were giving her to some simple, honest boy of her
own age, how gladly I’d see her go. We mothers don’t ask more
than that, after all—just to see the children happy.”
“I know,” breathed Mrs. Severing. “It’s all one lives for.”
“I’ve no plans or wishes for myself—it’s all for them,” muttered
Bertha disjointedly. “What else has one to care about—an old
gargoyle....”
Nina straightened herself slightly.
“‘Having outlived hope, fear, desire....’” she quoted softly, at the
same time turning her long neck so that the firelight fell upon her
burnished hair and exquisite, appealing profile.
“A man she’s only known a few months,” pursued Bertha bitterly.
“And she’ll disobey her parents, the mother who’s loved and guarded
and cherished her all her life, and break their hearts, for his sake.”
“God grant the poor child may not regret it bitterly one day,”
breathed Nina piously.
There was a long pause.
“Well!” said Bertha, and slowly stood up. “There’s a lot to be done.”
“Do let me help you, dearest.”
“Thanks, Nina, if you would. The girls are somewhere, I suppose.”
“Ah, they’ll be a comfort to you, I hope. They who owe you even
more than Hazel does, if possible.”
“One does what one can. It seems to me that it’s all give, give, give
on our side, and take, take, take on theirs. I feel rather like an
unfortunate pelican feeding its young, sometimes.”
With the words, and the curt laugh that dismissed them, Bertha
Tregaskis regained possession of herself.
 IX
ROSAMUND, though unhappy, was not as unhappy as she would
have liked to think herself. The defection of Morris Severing,
although gaining in poignancy by contrast with Hazel’s serene
happiness, was a sorrow of the emotions only, and a certain fierce
sincerity of outlook prevented Rosamund from rating it otherwise.
But she felt that she could have borne it better had the
disappearance of her quondam lover touched the mainsprings of her
life, and left that life dignified by a lasting grief, instead of merely
rendered unprofitable and savourless from an unrecognized sense of
vague discontent.
“I don’t know what Rosamund’s grievance is!” her guardian was
exasperated into exclaiming, nearly a year after Hazel’s marriage. “I
don’t believe she knows herself.”
And, in so saying, diagnosed the case.
Rosamund Grantham, after the manner of the modern generation,
had yet to find herself, and suffered accordingly.
It need scarcely be added that she did not confine her sufferings to
herself.
Frances, overwhelmed by the difficulties of reconciling
responsiveness to Cousin Bertie’s bracing councils of self-reliance,
with submission to Rosamund’s intensely protective and rather
overpowering solicitude, sought more frequently than ever the
soothing society of Nina Severing.
That gentle soul was passing through a period of storm of which she
presently confided the outline to Frances.
“Sometimes, darling, as I sit here alone through the long evenings, I
wonder if my life might have been different if I’d been a more
religious woman. You see, Francie, I married very, very young. I
wasn’t much older than you are now. My husband was not a man
who believed in any very definite creed, and I was young enough to
be altogether influenced by him.”
It was ever Nina’s custom to lay the errors and omissions of her past
at the door of Geoffrey Severing and her youthful marriage.
“Should you like to be a Roman Catholic?” asked Frances suddenly.
“It’s a very beautiful religion, and of course beauty is a religion in
itself, to an artist,” said Nina thoughtfully. “Why do you ask?”
“I’ve often thought,” said Frances very shyly, “that I should like it
myself. It seems such a thorough-going sort of religion. When we
were little, my mother had a Catholic maid—an Irish girl—and she
used to tell us a lot about it. And she was so particular about not
eating meat on certain days and going to Mass every Sunday. She
had to walk quite a long way, but I don’t believe she ever missed
going. Of course she was very superstitious, and used to want us to
wear medals and charms and things, but some of the prayers she
taught us were nice. My mother was a Catholic by birth, too, though
she never went to church or anything.”
“If I were anything, I should certainly be a Catholic,” said Nina with
extreme conviction in her tone. “It’s the only creed which appeals to
me in the slightest degree. It is so beautiful—all that music and
those touching ideas about the Virgin and everything.”
“But—don’t you believe?—isn’t the Church——?” murmured Frances,
embarrassed.
“Dear child, I am afraid the orthodox forms mean very little to me. I
would never wilfully cause pain to any human being, and I try to
help the sadness of the world with my little songs, but that is all. But
I would never shatter the innocent faith of another soul, although I
have outgrown the need of form and ritual myself.”
“Does one outgrow it?” wistfully asked Frances, whose whole nature
unconsciously craved the discipline which is inseparable from any
creed, faithfully followed out in practice.
“Not all of us,” tenderly said Nina, conscious of the exquisite contrast
between the matured, self-reliant soul, made strong through
suffering, and the innocent, inquiring child at her knee. “Not all of
us, dear. Some plants need a support round which to cling, whilst
others stand alone—always alone.” Her voice deepened slightly as
she mused broodingly for a moment on the pathos and beauty of
this horticultural parable. It came as a slight shock when Frances,
generally the most sympathetic of listeners, observed in
unmistakably self-absorbed accents:
“I think that I shall always want a support. It seems to me that I am
meant to live by rule—not by my own judgment at all. That’s why I
like the Roman Catholic idea of the Church being infallible. It would
be such a guide.”
Nina was aware that to no one else would Frances have spoken so
unreservedly, and the reflection was soothing, but it did not prevent
a slight stiffening of tone in her reply.
“Really, dear? But the surest guide in the world is the golden rule
which I have tried to live up to all my life—Never think of yourself at
all. Somehow, if one gives all one’s thoughts and time to other
people, one finds that God takes care of the rest.”
Nina was herself rather surprised at the beauty of the sentiment as
she put it into words, and it served to restore her not very deeply
ruffled serenity.
“I will lend you some books, Frances, if you really want to know
something about various creeds. The religion of Buddha is, to my
mind, the most beautiful of them all,” reflectively said Nina, who had
once read portions of Sir Edwin Arnold’s translation of the “Light of
Asia,” and was persuaded that she had studied it deeply. “It was the
foundation of the Roman Catholic religion, of course—they borrowed
a great deal from it.”
“I should like to read it very much.”
Frances wanted to read anything which spoke, however indirectly, of
Roman Catholic doctrines. If Nina guessed as much, however, she
did not impart her surmise to the vigorously orthodox Bertha
Tregaskis.
That this discreet reticence had been justified was made
superabundantly evident when Mrs. Tregaskis first became aware of
the Romanistic tendencies of her ward.
“People of seventeen must do what they’re told,” she said serenely,
but with an undercurrent of severity. “When you’re one-and-twenty,
Francie, we’ll talk about it again, and meanwhile I strongly advise
you not to think about the subject. You are much too young to
decide such a matter without knowing a great deal more about it,
and from your own showing all this simply arises from restlessness
and desire for excitement. Religion is too serious a matter to be
played with, my dear little girl.”
A certain look of flintlike impenetrability came over Frances’ young
face as she looked at her guardian, and she said nothing more. But
Mrs. Tregaskis was much too acute to suppose that her silence
denoted submission.
“Take her to London,” growled Frederick, when his wife, in her
perplexity, put the case before him. “You ought to get her away from
that silly woman’s influence.”
Bertha did not ask “What silly woman?” since she rightly recognized
that her husband thus denoted her dearest friend, but she decided
to follow his advice.
“We’ll have a month in London, and see all the sights,” she cried.
“Just you and I and Rosamund, Francie, and be regular country
cousins, and go to the National Gallery and British Museum, and a
theatre or two from the dress-circle. Never mind about planting the
bulbs, dear—no, I don’t mind leaving them to Grant, and the garden
must just get on without me for a week or two.”
She stifled a sigh heroically.
“This trip is absolutely for the sake of the girls,” she told Nina
Severing. “Neither of them takes any natural healthy interest in
gardening or in the animals and things, as Hazel used to do, so I
must try what London will do for them. Really, girls are a problem.”
“Nothing to a boy,” sighed Nina. “There’s Morris wandering half over
Europe, in the most unsatisfactory manner, pretending that he is
studying languages, and really doing nothing at all except loaf. I’ve
told him he ought to come back and look after the place in earnest,
but he makes one excuse after another——”
“It’s too bad,” said Bertha sympathetically. “Perhaps if he came back,
now that he and Rosamund are a little older and have rather more
sense....”
“Oh, my dear! he’s got over that nonsense long ago. I always told
you it wouldn’t last. ‘Weak and unstable as water,’ that’s what my
poor Morris is.”
Bertha did not remind Mrs. Severing that everything had been done
to insure the instability of Morris in this particular case. She only said
affectionately:
“Well, good-bye, Nina darling. Don’t forget to take pity on my old
man, since I can’t drag him to London.”
“He must come and cheer me up some afternoon, if he will,”
cordially responded Nina. Both ladies were perfectly aware that
Frederick Tregaskis would do nothing of the sort, and that there
were few things less conducive to the cheering up of either than an
encounter between him and Mrs. Severing. But they exchanged their
fallacious hopes with an air of affectionately reassuring one another.
“I’ve one comfort,” declared Bertie, “I’m hoping to see a very old
friend of mine in town: Sybil Argent. I believe she and her son are
there for a few weeks.”
“Didn’t she become a Catholic?” asked Nina, with a sudden air of
intense interest, which provoked Bertha to a display of extreme
nonchalance instantly.
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

textbookfull.com