Data Ingestion with Python Cookbook: A practical guide to ingesting, monitoring, and identifying errors in the data ingestion process 1st Edition Esppenchutz instant download
Data Ingestion with Python Cookbook: A practical guide to ingesting, monitoring, and identifying errors in the data ingestion process 1st Edition Esppenchutz instant download
https://ebookmass.com/product/python-data-cleaning-cookbook-second-
edition-michael-walker/
https://ebookmass.com/product/data-driven-seo-with-python-solve-seo-
challenges-with-data-science-using-python-1st-edition-andreas-
voniatis/
Statistical Process Monitoring Using Advanced Data-Driven
and Deep Learning Approaches: Theory and Practical
Applications 1st Edition Fouzi Harrou
https://ebookmass.com/product/statistical-process-monitoring-using-
advanced-data-driven-and-deep-learning-approaches-theory-and-
practical-applications-1st-edition-fouzi-harrou/
https://ebookmass.com/product/data-structure-and-algorithms-with-
python-the-ultimate-guide-towards-coding-john-thomas/
https://ebookmass.com/product/nonclinical-study-contracting-and-
monitoring-a-practical-guide-1st-edition/
Data Ingestion with Python
Cookbook
Gláucia Esppenchutz
BIRMINGHAM—MUMBAI
Data Ingestion with Python Cookbook
Copyright © 2023 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted
in any form or by any means, without the prior written permission of the publisher, except in the case
of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information
presented. However, the information contained in this book is sold without warranty, either express
or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable
for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and
products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot
guarantee the accuracy of this information.
ISBN 978-1-83763-260-2
www.packtpub.com
This book represents a lot and wouldn’t be possible without my loving husband, Lincoln, and his
support and understanding during this challenging endeavor. I want to thank all my friends that
didn’t let me give up and always boosted my spirits, along with my grandmother, who always believed,
helped, and said I would do big things one day. Finally, I want to thank my beloved and four-pawed
best friend, who is at peace, Minduim, for “helping” me to write this book.
– Gláucia Esppenchutz
Contributors
I want to thank my patient and beloved husband and my friends. Thanks also to my mentors in the
Python open source community and the DataBootCamp founders, who guided me at the beginning
of my journey.
Thanks to the Packt team, who helped me through some hard times; you were terrific!
About the reviewers
Bitthal Khaitan is currently working as a big data and cloud engineer with CVS Health, a Fortune
4 organization. He has a demonstrated history of working in the cloud, data and analytics industry
for 12+ years. His primary certified skills are Google Cloud Platform (GCP), the big data ecosystem
(Hadoop, Spark, etc.), and data warehousing on Teradata. He has worked in all phases of the SDLC
of DW/BI and big data projects with strong expertise in the USA healthcare, insurance and retail
domains. He actively helps new graduates with mentoring, resume reviews, and job hunting tips in
the data engineering domain. Over 20,000 people follow Bitthal on LinkedIn. He is currently based
out of Dallas, Texas, USA.
Jagjeet Makhija is a highly accomplished technology leader with over 20 years of experience. They are
skilled not only in various domains including AI, data warehouse architecture, and business analytics,
but also have a strong passion for staying ahead of technology trends such as AI and ChatGPT.
Jagjeet is recognized for their significant contributions to the industry, particularly in complex proof
of concepts and integrating Microsoft products with ChatGPT. They are also an avid book reviewer
and have actively shared their extensive knowledge and expertise through presentations, blog articles,
and online forums.
Krishnan Raghavan is an IT professional with over 20 years of experience in the area of software
development and delivery excellence across multiple domains and technology, ranging from C++ to
Java, Python, data warehousing, and big data tools and technologies. Krishnan tries to give back to the
community by being part of GDG – Pune Volunteer Group, helping the team in organizing events.
When not working, Krishnan likes to spend time with his wife and daughter, as well as reading fiction,
non-fiction, and technical books. Currently, he is unsuccessfully trying to learn how to play the guitar.
You can connect with Krishnan at mail to: k r i s h n a n @ g m a i l . c o m or via
LinkedIn: www.linkedin.com/in/krishnan-raghavan
I would like to thank my wife, Anita, and daughter, Ananya, for giving me the time and space to
review this book.
Table of Contents
Prefacexv
2
Principals of Data Access – Accessing Your Data 31
Technical requirements 31 How to do it… 47
Implementing governance in a data How it works… 48
access workflow 32 There’s more… 49
Getting ready 32 See also 52
How to do it… 33 Managing encrypted files 52
How it works… 34 Getting ready 52
See also 34 How to do it… 53
Accessing databases and data How it works… 54
warehouses34 There’s more… 55
Getting ready 35 See also 56
How to do it… 35 Accessing data from AWS using S3 56
How it works… 37 Getting ready 56
There’s more… 38 How to do it… 59
See also 39 How it works… 62
Accessing SSH File Transfer Protocol There’s more… 63
(SFTP) files 39 See also 63
Getting ready 39 Accessing data from GCP using
How to do it… 41 Cloud Storage 64
How it works… 43 Getting ready 64
There’s more… 43 How to do it… 66
See also 44 How it works… 68
Retrieving data using API There’s more… 70
authentication44 Further reading 70
Getting ready 45
3
Data Discovery – Understanding Our Data before Ingesting It 71
Technical requirements 71 How to do it… 73
Documenting the data discovery How it works… 77
process71 Configuring OpenMetadata 77
Getting ready 72 Getting ready 77
Table of Contents ix
4
Reading CSV and JSON Files and Solving Problems 95
Technical requirements 95 How it works… 105
Reading a CSV file 96 There’s more… 106
See also 107
Getting ready 96
How to do it… 96 Using PySpark to read CSV files 108
How it works… 98 Getting ready 108
There’s more… 98 How to do it… 108
See also 99 How it works… 109
Reading a JSON file 99 There’s more… 110
See also 114
Getting ready 100
How to do it… 100 Using PySpark to read JSON files 114
How it works… 100 Getting ready 114
There’s more… 101 How to do it… 115
See also 103 How it works… 116
Creating a SparkSession for PySpark 103 There’s more… 117
See also 117
Getting ready 103
How to do it… 104 Further reading 117
5
Ingesting Data from Structured and Unstructured Databases 119
Technical requirements 119 There’s more… 127
Configuring a JDBC connection 120 See also 129
6
Using PySpark with Defined and Non-Defined Schemas 159
Technical requirements 159 How to do it… 169
Applying schemas to data ingestion 160 How it works… 170
7
Ingesting Analytical Data 181
Technical requirements 181 How it works… 197
Ingesting Parquet files 182 There’s more… 198
See also 200
Getting ready 182
How to do it… 183 Ingesting partitioned data 200
How it works… 184 Getting ready 200
There’s more… 185 How to do it… 201
See also 185 How it works… 201
Ingesting Avro files 185 There’s more… 203
See also 204
Getting ready 186
How to do it… 186 Applying reverse ETL 204
How it works… 188 Getting ready 204
There’s more… 190 How to do it… 205
See also 190 How it works… 206
Applying schemas to analytical data 191 There’s more… 207
See also 207
Getting ready 191
How to do it… 191 Selecting analytical data for reverse
How it works… 194 ETL207
There’s more… 194 Getting ready 207
See also 195 How to do it… 208
Filtering data and handling common How it works… 209
issues195 See also 210
9
Putting Everything Together with Airflow 243
Technical requirements 244 How to do it… 257
Installing Airflow 244 How it works… 260
There's more… 262
Configuring Airflow 244 See also 262
Getting ready 244
How to do it… 245 Configuring sensors 262
How it works… 247 Getting ready 262
See also 248 How to do it… 263
How it works… 264
Creating DAGs 248 See also 265
Getting ready 248
How to do it… 249 Creating connectors in Airflow 265
How it works… 253 Getting ready 266
There's more… 254 How to do it… 266
See also 255 How it works… 269
There's more… 270
Creating custom operators 255 See also 270
Getting ready 255
Table of Contents xiii
10
Logging and Monitoring Your Data Ingest in Airflow 281
Technical requirements 281 Designing advanced monitoring 304
Installing and running Airflow 282 Getting ready 304
How to do it… 306
Creating basic logs in Airflow 283
How it works… 308
Getting ready 284
There’s more… 309
How to do it… 284
See also 309
How it works… 287
See also 289 Using notification operators 309
Getting ready 310
Storing log files in a remote location 289
How to do it… 312
Getting ready 289
How it works… 315
How to do it… 290
There’s more… 318
How it works… 298
See also 299 Using SQL operators for data quality 318
Getting ready 318
Configuring logs in airflow.cfg 299
How to do it… 320
Getting ready 299
How it works… 321
How to do it… 299
There’s more… 323
How it works… 301
See also 323
There’s more… 303
See also 304 Further reading 324
11
Automating Your Data Ingestion Pipelines 325
Technical requirements 325 Scheduling daily ingestions 326
Installing and running Airflow 326 Getting ready 327
xiv Table of Contents
12
Using Data Observability for Debugging, Error Handling,
and Preventing Downtime 349
Technical requirements 349 Getting ready 358
Docker images 350 How to do it… 358
How it works… 361
Setting up StatsD for monitoring 351 There’s more… 363
Getting ready 351
How to do it… 351 Creating an observability dashboard 363
How it works… 353 Getting ready 363
See also 354 How to do it… 363
How it works… 369
Setting up Prometheus for storing There’s more… 370
metrics354
Getting ready 354 Setting custom alerts or notifications 370
How to do it… 354 Getting ready 371
How it works… 356 How to do it… 371
There’s more… 357 How it works… 377
Index379
Chapter 2, Data Access Principals – Accessing your Data, explores data access concepts related to data
governance, covering workflows and management of familiar sources such as SFTP servers, APIs,
and cloud providers. It also provides examples of creating data access policies in databases, data
warehouses, and the cloud.
Chapter 3, Data Discovery – Understanding Our Data Before Ingesting It, teaches you the significance of
carrying out the data discovery process before data ingestion. It covers manual discovery, documentation,
and using an open-source tool, OpenMetadata, for local configuration.
Chapter 4, Reading CSV and JSON Files and Solving Problems, introduces you to ingesting CSV and JSON
files using Python and PySpark. It demonstrates handling varying data volumes and infrastructures
while addressing common challenges and providing solutions.
Chapter 5, Ingesting Data from Structured and Unstructured Databases, covers fundamental concepts
of relational and non-relational databases, including everyday use cases. You will learn how to read
and handle data from these models, understand vital considerations, and troubleshoot potential errors.
Chapter 6, Using PySpark with Defined and Non-Defined Schemas, delves deeper into common PySpark
use cases, focusing on handling defined and non-defined schemas. It also explores reading and
understanding complex logs from Spark (PySpark core) and formatting techniques for easier debugging.
Chapter 7, Ingesting Analytical Data, introduces you to analytical data and common formats for reading
and writing. It explores reading partitioned data for improved performance and discusses Reverse
ETL theory with real-life application workflows and diagrams.
Chapter 8, Designing Monitored Data Workflows, covers logging best practices for data ingestion,
facilitating error identification, and debugging. Techniques such as monitoring file size, row count,
and object count enable improved monitoring of dashboards, alerts, and insights.
Chapter 9, Putting Everything Together with Airflow, consolidates the previously presented information
and guides you in building a real-life data ingestion application using Airflow. It covers essential
components, configuration, and issue resolution in the process.
Chapter 10, Logging and Monitoring Your Data Ingest in Airflow, explores advanced logging and
monitoring in data ingestion with Airflow. It covers creating custom operators, setting up notifications,
and monitoring for data anomalies. Configuration of notifications for tools such as Slack is also covered
to stay updated on the data ingestion process.
Chapter 11, Automating Your Data Ingestion Pipelines, focuses on automating data ingests using
previously learned best practices, enabling reader autonomy. It addresses common challenges with
schedulers or orchestration tools and provides solutions to avoid problems in production clusters.
Chapter 12, Using Data Observability for Debugging, Error Handling, and Preventing Downtime,
explores data observability concepts, popular monitoring tools such as Grafana, and best practices
for log storage and data lineage. It also covers creating visualization graphs to monitor data source
issues using Airflow configuration and data ingestion scripts.
Preface xvii
For almost all recipes in this book, you can use a Jupyter Notebook to execute the code. Even though it
is not mandatory to install it, this tool can help you to test the code and try new things on the code due
to the friendly interface.
If you are using the digital version of this book, we advise you to type the code yourself or access
the code via the GitHub repository (link available in the next section). Doing so will help you
avoid any potential errors related to the copying and pasting of code.
Conventions used
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file
extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Then
we proceeded with the with open statement.”
A block of code is set as follows:
$ python3 –-version
Python 3.8.10
Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words
in menus or dialog boxes appear in the text like this. Here is an example: “Then, when we selected
showString at NativeMethodAccessorImpl.java:0, which redirected us to the
Stages page.”
Sections
In this book, you will find several headings that appear frequently (Getting ready, How to do it..., How
it works..., There’s more..., and See also).
To give clear instructions on how to complete a recipe, use these sections as follows:
Getting ready
This section tells you what to expect in the recipe and describes how to set up any software or any
preliminary settings required for the recipe.
How to do it…
This section contains the steps required to follow the recipe.
Preface xix
How it works…
This section usually consists of a detailed explanation of what happened in the previous section.
There’s more…
This section consists of additional information about the recipe in order to make you more knowledgeable
about the recipe.
See also
This section provides helpful links to other useful information for the recipe.
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book title in the
subject of your message and email us at customercare@packtpub.com.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen.
If you have found a mistake in this book, we would be grateful if you would report this to us. Please
visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata
Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would
be grateful if you would provide us with the location address or website name. Please contact us at
copyright@packt.com with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you
are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Visit https://ebookmass.com today to explore
a vast collection of ebooks across various
genres, available in popular formats like
PDF, EPUB, and MOBI, fully compatible with
all devices. Enjoy a seamless reading
experience and effortlessly download high-
quality materials in just a few simple steps.
Plus, don’t miss out on exciting offers that
let you access a wealth of knowledge at the
best prices!
Exploring the Variety of Random
Documents with Different Content
the actor and actress. The situation, as Professor Gail said
afterwards, was highly dramatic.
The Professor silenced her with a gesture and spoke to Vernon with
great dignity. "Young man," he said, striving to keep his voice from
trembling, "I pay my rates and taxes, my bills to my tradesmen, and
my rent for our home. Under these circumstances I cannot see why
you should talk of the police."
"And you may well do so, young man. To fire a pistol in a private
house----"
"Why should Mr. Hest assault you?" demanded Mrs. Gail, much
astonished.
"That's a long story. Tell me," Vernon turned towards the Professor
while Towton held his peace and nursed his hat, "what do you know
of Maunders?"
"Know of him?" said the amazed Gail, looking thoroughly puzzled. "I
know no more than that he is a friend of Mr. Hest's who called last
night and who was requested, by Mr. Hest and not by me, to stay
the night. I have never set eyes on him before."
Professor Gail nodded his head graciously. "I agree with Miss Hest
there," he said gravely; "Mr. Maunders is indeed handsome. But she
never told us anything about him, Mr. Vernon, save what my wife
has related."
"Why did you?" asked the Colonel, speaking for the first time. Gail
looked embarrassed, but Mrs. Gail spoke for him. "Mr. Hest, we
know, is very rich," she said frankly, "and both my husband and
myself wish to have a theatre of our own. We thought that if we
showed him some hospitality he might finance us. I must say," she
added, looking puzzled, "that I wondered that such a rich man was
content to accept our humble lodgings instead of going to a swell
hotel. But he seems to be easily pleased."
"It was not that, Hettie," said the Professor quickly. "Mr. Hest simply
remained here so that he could persuade me to induce my talented
pupil to give up reciting, as he dreaded lest she should go on the
stage. And she ought to be an actress, in my humble opinion, for her
capabilities are of a very high order. As Lady Macbeth, or in any of
Sardou's characters, such as La Tosca, Fedora, and the rest, she
would produce a sensation."
The speech of both man and wife seemed frank enough, and they
appeared to be a couple of simple people devoted to their profession
and quite ignorant of evil. Vernon glanced at Towton and saw from
the expression of the Colonel's face that he thoroughly believed
them. Still, so as to be quite sure of his ground, he asked another
question: "Miss Hest as a reciter or an actress may be all that can be
desired, but do you and Mrs. Gail like her personally; do you think
she is what we call--well--er--straight?"
"Hettie is quite right," said the Professor with dignity. "Many a time
has Miss Hest assisted us when tradespeople have worried. I am
sure that she would have persuaded her brother to enable us to
enter into management in the long run, as she has every confidence
in my capabilities."
"And in mine," said Mrs. Gail jealously. "She said that my Emilia in
'Othello' was the best performance she had ever seen. But now,
gentlemen," the actress rose to give effect to her words, "may I
inquire why you ask these questions, and why you come here to fire
pistols in a peaceful home?"
At the beginning it had been in Vernon's mind to tell the whole story
right out and to tax the couple with complicity. But they really
seemed to be entirely ignorant of Hest's true character, and evidently
had only lately met Maunders. He therefore did not think it wise to
reveal what he and the Colonel knew lest the Gails should gossip
about the matter. And until he had consulted Drench the young man
did not desire that this last unusual affair should become public. He
therefore shot a warning glance at the Colonel and answered
cautiously:
"It is only a private matter, Mrs. Gail, which is not worth explaining.
The pistol-shot was an accident."
"But you said that Mr. Maunders and Mr. Hest were villains," she
persisted. "Ah, I spoke somewhat harshly, being a trifle excited.
They have treated me and my friend here very badly and we came
for redress. How their consciences smote them you can judge from
the fact of their flight. You will possibly never see them again. But if
they do chance to return you must wire to me at once to the
Athenian Club, Pall Mall."
"I don't like these hints and suggestions of evil, sir," said Gail,
restlessly, "and certainly I should never think of telegraphing to you
unless Mr. Maunders and Mr. Hest give me leave. And why, sir,
should they not return?"
"Don't seek to know any more, Mr. Gail, but do as you are told," said
Vernon in a peremptory tone, "and also it will be wise if you and
your wife hold your tongues over what has happened and stop the
servant from talking."
"See here!" Colonel Towton rose angrily. "We have reason to believe
that these men are connected with The Spider."
Mrs. Gail shrieked and the Professor turned pale. Both knew that
terrible name which was so freely mentioned in the papers. "Do you
mean to say----"
"We say nothing," said Vernon sharply, "and my friend here has
perhaps said too much. But it is as well that you should know the
necessity of keeping silent tongues in your heads."
"And if not?"
"Already I have told you that the police will interfere," observed
Vernon coldly. "This business is concerned with The Spider, so, for
your own sakes, hold your confounded tongues."
"You shall know all when the time comes," he said when taking
leave, "but keep silence until the appointed hour lest you get into
trouble."
"I thought both Hest and Maunders were too clever to risk a wire to
Waterloo Station, as they knew I would send it."
"The fog is still thick," said Towton as they stepped into a taxi;
"perhaps he won't come. Hang it, every possible obstacle seems to
be placed in our way. The blackguards will escape."
But the young man spoke with more confidence than he actually felt,
as he knew that Hest was extraordinarily clever in concealing
himself. As The Spider he had baffled the police for years, and, being
an arch-criminal, would be dexterous enough to escape even out of
this tight corner. He began to consider what was best to be done
after sending a wire to Inspector Drench, when his meditations were
broken in upon by the Colonel.
"Do you really believe that Hest is The Spider?"
"Of course. Didn't you see his face change when Maunders spoke,
and didn't he cut and run when he saw that the game was up?"
"It certainly looks like guilt. And yet it seems incredible. The man
always has lived in Yorkshire, whereas The Spider is supposed to live
in town."
"No one has ever known the whereabouts of The Spider," said
Vernon coolly, "and it is as easy to write blackmailing letters in
Yorkshire and post them in London as to live in town altogether for
that purpose. Besides, his sister told me herself that Hest frequently
went away for days and weeks at a time. Doubtless he was
attending to his nefarious business in London."
"It seems odd, doesn't it? But we know that the worst criminals have
their good points. There lives some soul of good in all things evil,
you know."
"I rather think," said the Colonel grimly, "that Hest looks upon
himself as a kind of modern Robin Hood, who takes from the rich to
give to the poor. He blackmailed wealthy folk in order to build his
Bolly Reservoir and his confounded school-houses. Robbed Peter to
pay Paul, as you might put it."
"Rob Dives to help Demos is the way he would put it," said Vernon
with a shrug. "However, we have made a great discovery and one
which the police will thank us for making. When Hest is captured
many a rich man will sleep the easier."
"I agree with you. The Spider is as clever as his father--the devil.
Humph!" added Vernon thoughtfully, "I wonder if his sister knows
anything about his infernal doings."
"No," said the Colonel decidedly. "I don't like Miss Hest, as I think
she is too imperious and masterful and wants her own way too
much. All the same, I don't believe she would have countenanced
her brother's behaviour. Besides, she was always away from him,
and he doubtless carried on his pranks without her knowledge."
"I admitted only a moment ago that I did not," snapped the Colonel
as the taxi cautiously felt its way up Whitehall, "but I must be just to
her. The poor woman will suffer as it is when her brother's
criminality becomes known. It will ruin her reciting business."
"Oh, Miss Hest has led her in the right way, no doubt," retorted the
Colonel; "but I prefer to be the guide myself. See here, Vernon,
come down with me next week to my place at Bowderstyke."
"We can safely leave that to Drench and his underlings. I want to
get Ida away from Gerby Hall. Sorry as I am for Miss Hest in having
such a brother, I don't want Ida to continue under her protection any
longer, especially as she wants to marry her to Maunders."
"Maunders will have no chance now," said Vernon with a grim
chuckle. "But you are a bachelor, Towton, so Ida will scarcely be able
to come to The Grange."
After the long, cautious creeping through the fog the two gentlemen
arrived at the Colonel's rooms, and Bendham was sent out for food.
Having dined, they smoked and talked while waiting for Inspector
Drench. But he never came. A telegram arrived instead stating that
the fog prevented his keeping the appointment. And it also
prevented Vernon getting back to his own quarters, so the Colonel
put him up for the night. Next day the hunt for the criminals began
in earnest.
"Sent first thing this morning," said Vernon passing the wire to the
Colonel. "Poor woman! she doesn't know that her brother has been
found out."
The wire was shown to Inspector Drench when he duly arrived, and
he was exhaustively informed of all that had taken place. He was
naturally both astonished and interested, but nevertheless expressed
himself annoyed that civilians should have proceeded so far without
invoking the police. Drench gave both the Colonel and Vernon to
understand that if he had been on the spot Hest and his accomplice
would not have escaped so easily, a view with which they privately
differed, although they did not think it wise to say so. But Towton
_did_ intimate to the Inspector that he was a military man and not a
civilian, whatever Vernon might be. Drench declined to take any
notice of this remark.
The Inspector also questioned Gail closely, but could learn nothing
from him of any moment, since the old actor knew nothing and was
greatly agitated over the whole affair. Finally, bidding all three hold
their tongues, Drench sallied forth to search for the missing pair. He
saw the Scotland Yard authorities and wired to all the ports and
railway stations in the kingdom. As yet, and because he desired to
keep the affair out of the newspapers, Drench did not advertise in
the journals, or by handbills. Otherwise, in every way he strove to
find the fugitives.
"There's nothing else for it," said Drench, and Vernon in spite of his
wish to keep things quiet, agreed with him.
CHAPTER XVII.
IN THE TRAIN.
However, it was useless for Vernon to mourn over his old school
friend's downfall. He had done his best to keep him in the straight
path and had failed to prevent his feet from straying. He therefore,
as there was nothing else to be done at this eleventh hour, washed
his hands of him and left him, together with Hest, to the tender
mercies of the law as represented by the Inspector. Now that Drench
had all the threads in his own hands he resented anyone else
weaving them into ropes for the necks of the criminals, as he
apparently wished to secure all the glory and honour of the capture
to himself. Both Towton and Vernon were rather glad that the
Inspector took this view, as they wished to have nothing more to do
with the matter. And, before leaving London for Bowderstyke,
Vernon shut up his Covent Garden office and formally renounced his
pseudonym of Nemo. As by this time he was officially recognised as
his uncle's heir he could well afford to do so. Sir Edward, however,
still lingered between life and death, so it was doubtful when Vernon
would enter into his kingdom.
While the train was flying through the autumnal landscape Towton
and his guest made themselves comfortable in a first-class
compartment, which they had secured to themselves, for the
purpose of uninterrupted conversation. They were still deeply
interested in the case and looked forward anxiously to the capture of
The Spider. It was only right that he should suffer for his dastardly
crime in murdering an old and inoffensive man. As to Maunders, he
was evidently hand in glove with the cleverer rascal, and would
undoubtedly be given a long term of imprisonment. Thus society
would be rid of two dangerous people, and those with secrets would
sleep the easier, knowing that one Asmodeus was dead and the
other safely locked up.
"But I don't know what poor Mrs. Bedge will do," said Vernon
looking dolefully out of the window.
"Does she know anything?" asked the Colonel, throwing down the
morning paper which he had been reading and settling himself for a
talk.
Vernon nodded. "I saw her yesterday. She sent to ask me what had
become of Constantine. I was obliged to tell her."
"I think so, decidedly. It was better that Mrs. Bedge should learn the
truth from a friend than see it crudely printed in the daily papers.
And there it is bound to appear sooner or later."
"Drench will have to catch The Spider first," said the Colonel coolly.
"No easy task, as we know. What did she say?"
"At first she declined to believe it, badly as Maunders has treated
her. She kept insisting that it was all a mistake and that Constantine
would appear to put matters right."
"In that case," remarked Towton drily, "Mrs. Bedge must have known
that Maunders is not the saint she tries to make him out to be."
"I did not say that she went by her instinct," replied Vernon equally
drily; "there is a difference between that and heart-love. Because
Constantine is her sister's child and her adopted son Mrs. Bedge's
heart, which he has almost broken, cherishes him fondly; but her
instinct must have told her long ago that the fellow is a scamp of the
worst sort."
"Mrs. Bedge declined to take that view of him. She wailed that he
had a tender heart and was led away because he had a weak
nature. In fact, her defence was that of a man being his own worst
enemy."
"Maunders certainly was. He had all the gifts of the gods, yet----"
"Yet fell because the greatest gift of honest purpose was not given,"
finished Vernon. "Hang it all, Towton, scamp as the fellow is, I am
sorry for him."
"I would call him nothing of the sort, Vernon. I detest slang."
"That's a mistake. Slang frequently hits the nail on the head when
the King's English misses it altogether. Slang conveys much in little,
and----"
"Oh, the deuce take your philology. Go on talking about Mrs. Bedge."
"There's no more to say. Maunders has pretty well drained her, but
she has enough to live on, and the Hampstead house is her own.
Towards the end of our conversation, however, she let out that she
was not surprised at Conny's behaviour, as she rather expected it."
"No. Mrs. Bedge always kept it quiet for the boy's sake until she let it
out to me in her grief yesterday. Mavrocordato--he took the English
name of Maunders--bolted with a heap of his partner's money, and
shot himself at Corfu, whither he was traced by detectives. Mrs.
Bedge adopted the son, and did her best to train him up as an
honest man. She tried her hardest, I'm certain, but what's bred in
the bone, you know."
Colonel Towton folded his arms and stared straightly before him.
"Poor devil. He was considerably handicapped by such a father. I
wonder, Vernon, for how many of our deeds we are responsible,
when you take heredity into consideration. Some sin because they
like it, but many because they can't help it."
"Let us give Maunders the benefit of the doubt, and say that the sins
of his father were visited on him. And, of course, we must not forget
that Hest is an extremely clever and strong-minded man, who could,
and did, easily control Maunders' weaker nature."
"I don't know enough about him to give an opinion. Perhaps when
we see the sister she may tell us something."
"Oh, by the way, I received a letter from her two days ago, about
which I intended to speak to you, Vernon. All this bother and worry
put it out of my head. I left it at home, unfortunately, but I can tell
you the gist of it."
Vernon looked interested. "What did she write about, and why to
you?" "She wrote to me because she wants me to marry Ida."
"I really don't see what she has to do with that," remarked Vernon
with a shrug; "for Ida is surely of an age to choose for herself."
"I always told you, Vernon," said Towton, deliberately crossing one
leg over the other, "that Ida, being less masterful than Miss Hest, is
usually guided by her, and that I objected to the guidance. Ida liked
me more than anyone else before that handsome scamp came
along. Then she became infatuated with him, and Miss Hest did her
best to induce her to marry him. But the sad death of Dimsdale took
Ida's thoughts off Maunders, and--as I judge from the letter Ida
wrote me from Gerby Hall--Miss Hest tried to get her to love the
man again. Failing that, she attempted to get Ida to marry her
brother, only he came up to London, not feeling disposed to fall in
with his sister's views. You can therefore see that Miss Hest sways
Ida a great deal, and for that reason I have come to get her away
from such dangerous company--doubly dangerous now that we
know Francis Hest is The Spider."
Vernon shrugged his shoulders. "It's rather hard to blame the sister
for the brother's delinquencies," he said judicially. "And now that he
and Maunders are out of the running she will place her weight in
your scale. In fact, from your late observation, she has already done
so. You should be very pleased, Colonel, whereas you seem to me to
be ungrateful."
"Well, Colonel," replied Vernon pacifically, "I hope things will turn out
as you expect. But what did Miss Hest write about?"
"About her brother. She asked me if I had seen him, and what was
the matter with him." Vernon looked puzzled. "I don't understand.
Does she suspect----"
"He was, six or seven days ago, but he may have gone away since,"
said Vernon cautiously. "Who drew up the Deed of Gift?"
"I can't say. Miss Hest did not explain that. Why?"
"Oh, well. I expect his business as The Spider has made him quite a
rich man. Remember, the blackguard has been blackmailing
successfully for three or four years. He knows that his sister has
nothing save what she makes by her reciting, so perhaps his
conscience smote him, and so he made his Deed of Gift. It's a lucky
thing for her, as Gerby Hall is a fine old place, although rather
gloomy, and there is a decent income of one thousand a year
attached to it, farms, village rents, and all that sort of thing, you
know."
"I agree with your first proverb, but not with the second, Towton.
Miss Hest is not of the same breed, morally speaking, as her brother,
and no doubt will be horrified when she learns of his wickedness."
"I daresay Hest hoped that his good deeds would pay for his bad
ones," said the Colonel carelessly. "I shall be glad if he escapes,
richly as he deserves to be hanged for murdering Dimsdale. It will be
just as well if the whole thing is buried in oblivion. Then I shall
marry Ida, you Miss Corsoon, and Miss Hest can play the lady of the
manor here, as she pleases."
"If it belongs to Lady Corsoon she must have it; if Maunders' story is
a lie, which it may be, I shall stick to it on behalf of my wife.
However, we may hear from Venery of Singapore in a few weeks. My
letter must have nearly reached him by this time."
"You can learn the truth of the story nearer home," said Vernon after
a pause. "Miss Jewin, the housekeeper at Gerby Hall, told the story
to Maunders, according to his own account."
"I shall question her, you may be sure," said the Colonel grimly; "but
I want to hear from Venery also. Oh, I'm sick of talking about these
things," he added with a yawn. "It's time for forty winks." And
forthwith he closed his eyes, after settling himself comfortably in his
seat. Vernon, not inclined to rest, lighted a fresh cigar and buried
himself in a book.
It was five o'clock when the travellers reached Bradmoor, the nearest
station to Bowderstyke. It was ten miles to the valley, but the road
was excellent, and Towton's motor-car awaited them. In ten minutes
the baggage was packed away, and Vernon with his host was safely
ensconced in the back part of the machine, which was covered with
a hood. Towton asked Vernon if he would care to drive, but as the
offer was refused and the Colonel himself did not feel in a sporting
humour, the conduct of the journey was left to the smart chauffeur.
He appeared to be well acquainted with the country, and as the road
was somewhat lonely, the motor travelled towards Bowderstyke at a
great rate of speed. The motion was exhilarating, and the view on
either side of the roadway extremely picturesque, so Vernon enjoyed
himself greatly in the fresh air, after the close atmosphere and the
monotony of the train. With the wind blowing in his face and the
smooth, easy gliding motion, he felt like a flying bird, or at all events
as though mounted on one.
Towards the end of the ten mile stretch from the station signs of
civilisation became more frequent. Here and there was a village with
cultivated fields around it. Cattle were pastured in enclosed
paddocks, and men and women with laughing children trudged
along the high road, looking after the motor with great curiosity, for
the machine was yet a novelty in that lonely district. Twice the road
ran directly through a village, and Vernon had an opportunity of
seeing the solid grey stone houses, which were suited to the
Calvinistic looks of the country. And the people themselves appeared
to be what the Scotch call "dour."
And now the moors began to grow higher and to close in on the
white road with a gradual menace. Leaving the comparatively broad
lands, the motor glided into a valley, which grew even more narrow
as they proceeded. A babbling stream prattled down the centre of
this, over a stony bed, and beside it the road twisted along like a
white serpent, protected by a parapet of rough stones. Already the
crimson light of the sunset had died out of the western sky, but the
moon was full, and, soaring high in the dark blue dome of the
firmament, poured floods of light into the gully, to use a Colonial
expression--for by this time it was little else. And looking upward,
Vernon could see star after star peep out to attend on the majestic
orb.
"Oh, the valley broadens out further on. This is merely the
entrance."
"What the deuce do the inhabitants live on? It's like living in a
drain."
"Oh, confound you, Vernon," said the Colonel half annoyed. "It's one
of the most beautiful places in the world. If you were a Yorkshire
tyke you would admit that. There is only the village of Bowderstyke
a mile away, and the inhabitants live by pasturing their cattle on the
moors on the heights above. Also there is a weaving and spinning
industry, the mills being driven by water power, of which there is no
lack."
"You should see it in winter when the snows melt on the moors,"
advised the Colonel. "Besides, the water from the mills comes from
Hest's new reservoir, and there is a never-failing supply. This stream
used to be much broader, and its bed contained much more water,
but when the Bolly Dam was constructed, of course the supply
dwindled. Pipes run under this road to supply the several villages
you saw just before we entered the valley."
Towton pointed straight ahead. "Round the next corner you could
see it, but we do not go so far. There was a small lake there up on
the moors which fed this stream. Hest simply got engineers to dam
the lake and prevent too much water going to waste down the bed
of this torrent. The dam runs right across the valley a mile and a half
beyond my house."
"But isn't that dangerous. If it burst this valley would be flooded
from end to end, and everybody would be drowned, to say nothing
of the way in which the village would be smashed up."
"Well, yes." Towton pinched his nether lip uneasily. "I've thought of
that myself many a time. But I was abroad when the dam was
constructed. There certainly--as I have often said--should be an
outlet for the water other than the pipes which supply Bowderstyke
and the villages outside the valley, capacious as those same pipes
undoubtedly are. Assuredly, if the reservoir burst there would be
great loss of life and destruction of property. But the Bolly Dam is
very strongly built, so I have no fear of anything happening. You can
see it from my house, and we'll pay it a visit in a day or two.
Meantime, this is Bowderstyke village."
The motor soon arrived at two tall stone pillars crested with heraldic
monsters, and passing through these, spun up a short avenue to
stop before a large white house, brilliantly lighted up. Spacious
lawns opened up before the mansion, interspersed with flowerbeds,
now bloomless, and the whole was shut in by the fairy forest, as
Vernon called it in his own mind.
"Here we are," said Colonel Towton jumping from the car. "Allow me
to welcome you to The Grange, my friend."
AT BOWDERSTYKE.
"I hope you slept well, Vernon," said the Colonel to his guest the
next morning when they were at breakfast.
"Like a top," was the response. "That journey tired me out, and your
moorland air is so strong that I slept the moment my head was on
the pillow."
"You will eat well also, Vernon," remarked Towton, regarding with
satisfaction the attention paid by his visitor to the appetising meal.
"Our air is famous as a tonic. You will return to town a giant
refreshed."
"We will call on Miss Hest this afternoon, and I can show you the
village at the same time. Lady Corsoon and her daughter will come
to-day, and will arrive to dinner. That is the programme."
"I'm at your disposal. And to-morrow I suppose you will get Lady
Corsoon to take charge of Ida?"
"If Miss Hest will let her go," said the Colonel cautiously.
"Not by outward force; but she may use her influence to keep her.
Miss Hest won't lose the chance of swaying the mind of a girl with
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.
ebookmasss.com