Advanced R Statistical Programming and Data Models: Analysis, Machine Learning, and Visualization 1st Edition Matt Wiley download pdf
Advanced R Statistical Programming and Data Models: Analysis, Machine Learning, and Visualization 1st Edition Matt Wiley download pdf
com
https://textbookfull.com/product/advanced-r-statistical-
programming-and-data-models-analysis-machine-learning-and-
visualization-1st-edition-matt-wiley/
OR CLICK BUTTON
DOWNLOAD NOW
https://textbookfull.com/product/advanced-r-data-programming-and-the-
cloud-1st-edition-matt-wiley/
textboxfull.com
https://textbookfull.com/product/biota-grow-2c-gather-2c-cook-loucas/
textboxfull.com
https://textbookfull.com/product/functional-data-structures-in-r-
advanced-statistical-programming-in-r-mailund/
textboxfull.com
https://textbookfull.com/product/advanced-linear-modeling-statistical-
learning-and-dependent-data-3rd-edition-christensen-r/
textboxfull.com
https://textbookfull.com/product/functional-data-structures-in-r-
advanced-statistical-programming-in-r-thomas-mailund/
textboxfull.com
https://textbookfull.com/product/machine-learning-with-r-cookbook-
second-edition-analyze-data-and-build-predictive-models-bhatia/
textboxfull.com
Matt Wiley and Joshua F. Wiley
Joshua F. Wiley
Columbia City, IN, USA
Trademarked names, logos, and images may appear in this book. Rather
than use a trademark symbol with every occurrence of a trademarked
name, logo, or image we use the names, logos, and images only in an
editorial fashion and to the benefit of the trademark owner, with no
intention of infringement of the trademark. The use in this publication
of trade names, trademarks, service marks, and similar terms, even if
they are not identified as such, is not to be taken as an expression of
opinion as to whether or not they are subject to proprietary rights.
While the advice and information in this book are believed to be true
and accurate at the date of publication, neither the authors nor the
editors nor the publisher can accept any legal responsibility for any
errors or omissions that may be made. The publisher makes no
warranty, express or implied, with respect to the material contained
herein.
Conventions
Bold lowercase letters are used to refer to a vector, for example, x . Bold
uppercase letters are used to refer to a matrix, for example, X .
Generally, the Latin alphabet is used for data and the Greek alphabet is
used for parameters. Mathematical functions are indicated with
parentheses, for example, f (·).
In the text, reference to R code or function will be in monospaced
font like this. R function names have parentheses included to
help indicate it is a function, such as mean() to indicate the mean
function in R .
Package Setup
Throughout the book, we will make use of many different R packages
that make tasks easier or provide more robust or sophisticated
graphing and analysis options.
Although not required for readers, we make use of the
checkpoint package to help ensure the book is reproducible [23]. If
you do not care about reproducibility and are happy to take your
chances that our code that worked with one version of R and packages
also works with whatever versions you have, then you can just skip
reading this section. If you want reproducibility, but do not care why or
how it works, then just create R scripts for the code for each chapter,
save them, and then run the checkpoint package at the beginning. If
you care and want to know why and how it all works, read on the next
few paragraphs.
Details on Reproducibility
The many additional packages available for R are one of its greatest
strengths. However, they also create some challenges. For example, as a
reader, suppose that on your computer, you have R v3.4.3 installed
and as part of that in January you had installed the ggplot2 package
for graphs. By default, you will have whatever version of ggplot2 was
available in January when you installed it. Now in one chapter, we tell
you that you need both the ggplot2 and cowplot packages. Because
you already had ggplot2 installed, you do not need to install it again.
However, suppose that you did not have the cowplot package
installed. So, whenever you happen to be reading that chapter, you
attempt to install the cowplot package, let’s say it’s in April. You will
now by default get the latest version of cowplot available for that
version of R as of April.
Now imagine a second reader comes along and also had R v3.4.3
but had neither the ggplot2 nor the cowplot package installed. They
also read the chapter in April, but they install both packages in April, so
they get the latest version of both packages available in April for R
v3.4.3 .
Even though both you and the other reader had the same version of
R installed, you will end up with different package versions from each
other, and likely different versions yet from whatever versions we used
to write the book.
The end result is that different people, even with the same version
of R, very likely are using different versions of different packages. This
can pose a major challenge for reproducibility. If you are reading a
book, it can be a frustration because code does not seem to work as we
said it would. If you are using code in production or for scientific
research or decision-making, nonreproducibility can pose an even
bigger challenge.
The solution to standardize versions across people and ensure
results are fully reproducible is to control not only the version of R but
also the version of all packages. This requires a different approach to
package installation and management than the default system, which
uses the latest package versions from CRAN. The checkpoint
package is designed to solve this challenge. It does require some extra
steps and processes to use, and at first may seem a nuisance, but the
payoff is that you can be guaranteed that you are not only using the
same version of R but also the same version of all packages.
To understand how the checkpoint package works, we need a bit
more background regarding how R ’s libraries and package system
work.
Mainstream R packages are distributed through CRAN. Package
authors can submit new versions of their packages to CRAN, and CRAN
updates nightly. For some operating systems, CRAN just stores the
package source code, such as for Linux machines. For others, such as
Windows operating systems, CRAN builds precompiled package
binaries and hosts those. CRAN keeps old source code but generally not
old binary packages for long. On a local machine, when
install.packages is run, R goes online to a repository, by default
CRAN, finds the package name, downloads it, and installs it into a local
library . The local library is basically just a directory on your own
machine. R has a default location it likes to use as its local library, and
by default when you install packages, they are added to the default
library. Once a package is installed, when it is loaded or opened using
library(), R goes to its default library, finds a package with the
same name, and opens it.
The checkpoint package works by creating a new library on the
local machine, for a particular version of R for a particular date. Then it
scans all the R script files in R ’s current working directory—you can
identify this using the getwd() function—and identifies any calls to
the library() or require() functions. Then it goes and checks
whether those packages are installed in the local library. If they are not,
it goes to a snapshot of CRAN taken by another server setup to support
the checkpoint package. That way, checkpoint can install the
version of the package available from a specific date. In that way, the
checkpoint package can ensure that you have the same specific
version of R and specific version of all packages that we used when
writing the book. Or if you are trying to re-run some analysis from a
year ago, you can get the same version of those packages on a new
computer.
Assuming that you have the following code in an R script, you can
use the checkpoint package to read the R script and find the call to
library(data.table), and it will install the data.table
package, which is a great package for data management [29]. If you do
not want checkpoint to look in the current working directory, you
can specify the project path, as we do to the book in this example. You
can also change where checkpoint sets its library to another folder
location, instead of the default location, which we also do. We
accomplish both of these using variables set as part of our R project,
book_directory and checkpoint_directory . If you are using
checkpoint on your own machine, set those variables to the relevant
directories, for example, as book_directory <-
"path/to/your/directory" . Note that whatever folder you
choose, R will need read and write privileges for that folder.
library(checkpoint)
checkpoint("2018-09-28", R. version = "3.5.1",
project = book_directory,
checkpointLocation = checkpoint_directory,
scanForPackages = FALSE,
scan.rnw.with.knitr = TRUE, use.knitr = TRUE)
library(data.table)
options(
width = 70,
stringsAsFactors = FALSE,
digits = 2)
Data Setup
One of the datasets we will use throughout this book is a longitudinal
study, the Americans’ Changing Lives (ACL) [45]. This is publicly
available data and can be downloaded by going to
http://doi.org/10.3886/ICPSR04690.v7 .
The Americans’ Changing Lives (ACL) is a longitudinal study with
five waves of data, shown in Table I-1 .
Wave Year
W1 1986
W2 1989
W3 1994
W4 2002
W5 2011
load ("../ICPSR_04690/DS0001/04690-0001-
Data.rda")
ls ()
## [1]
"book_directory" "checkpoint_directory"
## [3]
"da04690.0001" "render_apress"
setnames(acl, names(acl), c(
"ID", "Sex", "RaceEthnicity", "SESCategory",
"Employment_W1", "BMI_W1", "Smoke_W1",
"PhysActCat_W1",
"AGE_W1",
"SWL_W1", "InformalSI_W1", "FormalSI_W1",
"SelfEsteem_W1", "Mastery_W1",
"SelfEfficacy_W1",
"CESD11_W1", "NChronic12_W1",
"Employment_W2", "BMI_W2", "Smoke_W2",
"PhysActCat_W2",
"InformalSI_W2", "FormalSI_W2",
"SelfEsteem_W2", "Mastery_W2",
"SelfEfficacy_W2",
"CESD11_W2", "NChronic12_W2"
))
acl[, ID := factor(ID)]
acl[, SESCategory := factor(SESCategory)]
acl[, SWL_W1 := SWL_W1 * -1]
Joshua F. Wiley
is a lecturer in the Monash Institute of Cognitive and Clinical
Neurosciences and School of Psychological Sciences at Monash
University. He earned his PhD from the University of California, Los
Angeles, and completed his postdoctoral training in primary care and
prevention. His research uses advanced quantitative methods to
understand the dynamics between
psychosocial factors, sleep, and other
health behaviors in relation to
psychological and physical health. He
develops or codevelops a number of R
packages including varian , a package
to conduct Bayesian scale-location
structural equation models, and
MplusAutomation , a popular package
that links R to the commercial Mplus
software, and miscellaneous functions to
explore data or speed up analysis in
JWileymisc .
library(checkpoint)
checkpoint("2018-09-28", R.version = "3.5.1",
project = book_directory,
checkpointLocation = checkpoint_directory,
scanForPackages = FALSE,
scan.rnw.with.knitr = TRUE, use.knitr = TRUE)
library(knitr)
library(ggplot2)
library(cowplot)
library(MASS)
library(JWileymisc)
library(data.table)
1.1 Distribution
Visualizing the Observed Distribution
Many statistical models require that the distribution of a variable be
specified. Histograms use bars to graph a distribution and are probably
the most common graph used to visualize the distribution of a single
variable. Although relatively rare, stacked dot plots are another
approach and provide a precise way to visualize the distribution of data
that shows the individual data points. Finally, density plots are also
quite common and are graphed by using a line that shows the
approximate density or amount of data falling at any given value.
ggplot(mtcars, aes(mpg)) +
geom_dotplot()
Figure 1-1 Stacked dot plot of miles per gallon from old cars
As a brief aside, much of the code for ggplot2 follows the format
shown in the following code snippet. In our case, we wanted a dot plot,
so the geometric object, or “geom”, is a dot plot (geom_dotplot() ).
Many excellent online tutorials and books exist to learn how to use the
ggplot2 package for graphs, so we will not provide a greater
introduction to ggplot2 here. In particular, Hadley Wickham, who
develops ggplot2, has a recently updated book on the package,
ggplot2: Elegant Graphics for Data Analysis [109], which is an excellent
guide. For those who prefer less conceptual background and more of a
cookbook, we recommend the R Graphics Cookbook by Winston Chang
[20].
ggplot(the-data, aes(variable-to-plot)) +
geom_type-of-graph()
Unlike a dot plot that plots the raw data, a histogram is a bar graph
where the height of the bar is the count of the number of values falling
within the range specified by the width of the bar. You can vary the
width of bars to control how many nearby values are aggregated and
counted in one bar. Narrower bars aggregate fewer data points and
provide a more granular view. Wider bars aggregate more and provide
a broader view. A histogram showing the distribution of sepal lengths
of flowers from the famous iris dataset is shown in Figure 1-2.
ggplot(iris, aes(Sepal.Length)) +
geom_histogram()
ggplot(data.table(lynx = as.vector(lynx)),
aes(lynx)) +
geom_histogram()
ggplot(data.table(lynx = as.vector(lynx)),
aes(log(lynx))) +
geom_histogram()
Density Plots
Another common tool to visualize the observed distribution of data is
by plotting the empirical density. The code for ggplot2 is identical to
that for histograms except that geom_histogram() is replaced with
geom_density() . The code follows and the result is shown in Figure
1-5.
ggplot(iris, aes(Sepal.Length)) +
geom_density()
Figure 1-5 This is the density plot for our sepal lengths
Empirical density plots include some degree of smoothing, because
with continuous variables, there is never going to be many observations
at any specific value (e.g., it may be that no observation has a value of
3.286, even though there are values of 3.281 and 3.292). Empirical
density plots show the overall shape of the distribution by applying
some degree of smoothing. At times it can be helpful to adjust the
degree of smooth to see a coarser (closer to the raw data) or smoother
(closer to the “distribution”) graph. Smoothing is controlled in
ggplot2 using the adjust argument. The default, which we saw in
Figure 1-5, is adjust = 1. Values less than 1 are “noisier” or have less
smoothing, while values greater than 1 increase the smoothness. We
compare and contrast noisier in Figure 1-6 vs. very smooth in Figure 1-
7.
ggplot(iris, aes(Sepal.Length)) +
geom_density(adjust = .5)
ggplot(iris, aes(Sepal.Length)) +
geom_density(adjust = 5)
In the R.A.'s car, Pat sat between the lump of unresisting flesh that
Comstock had become and the cocoonlike figure of the philosopher.
The man who was a criminal in spite of himself observed the way Pat
looked down at Comstock and his harshly handsome face softened.
"My dear, perhaps it is better that he be the way he is, if what you
have told me is true and you are both rebels against the bonds that
chain all of us on this sorry world of ours. I fear what the Board of
Fathers or The Grandfather may decide may be much worse than this
condition that my question has caused."
"To die is hard, but to die without knowing that you are dying, is
horrible," Pat said through clenched teeth.
"It is unmanly, I will not gainsay that." Then the man was silent.
Ahead of them the odd buildings that housed the Board and The
Grandfather rose up in their way. The globular buildings inside of
which were both the Elders and the Fathers were dwarfed by the
height of the shaft of The Grandfather's residence.
The R.A.'s were as silent and seemingly unthinking as machines.
Their first visible emotion had been one of jubilance at having caught
Pat and Comstock but that had faded under the fear of punishment
for not having caught them sooner.
They sat statue still, their hands on their guns as the car drove up to
the entrance of the buildings.
One of the R.A.'s left the machine to go for further orders from his
superior officer.
In the car the last philosopher said softly, "Perhaps whatever little
nobility there is in man is best served by dying with one's eyes open. I
shall not again retreat into the lie of the Picaroon." He smiled gently at
Pat, and said, "I think I will like dying as one of you, as a rebel."
But all Pat's attention was on her beloved who had never stirred from
the curled up position into which his thoughts had forced him.
Seeing this, the last philosopher said, "There is one chance, and only
one that I can think of that may revive him. Perhaps love, an emotion
of which I know very little, may be strong enough to pull him out of
that place to which he has run for safety."
"What do you mean?" she asked.
"To me," the man said, "as a philosopher, the charms of love and sex
were never very strong. But I should imagine, just as pure
speculation, that the two must be very tightly entwined."
Deep, deep down inside the thing that Comstock had become he felt
a stirring of some kind of interest. He did not yet know what was
causing the sensation, he could not hear the love words that Pat was
whispering in his ear, he was not really conscious of her soft hands
caressing him, but something was taking place, something that
seemed to have reality in a place where there was no such thing.
CHAPTER 13
When Bowdler and Grundy had first sounded out Comstock and had
asked him the questions that had led him so far from the normal law-
abiding life that had been his, one of the mainsprings of his conduct
had been envy mixed with disgust that the Fathers whom formerly he
had so revered had become monsters in his mind. Monsters who had
used the whole planet as a breeding ground for their harems. For
when the thought that only the Fathers were really fathers had struck
Comstock he had resolved that he too would like to take part in such
noble work.
Along with the sexual motive, Comstock had decided that if the
Fathers controlled the world, he too would like to have a share in
either controlling the world as it was, or perhaps with luck, helping to
change the control in such a way that their world would be a better
place to live in.
This mixture of ideas had resulted in his mental picture of the Fathers
becoming an amalgam of monsters of pride, venery and power.
Looking about the room Comstock decided that he could not possibly
have been further wrong in the way he had pictured the Fathers.
For his first feeling as his unbelieving eyes swept around the table at
which the Fathers sat, was one of pity.
Far from being the creatures with inflated egos, the monsters of
uxoriousness that his inflamed imagination had painted, these men
who guided the affairs of his world were invalids... The lame, the halt
and the blind.
Each face was torn by pain, every body bore the stigmata of some
fatal disease.
Only the Grandfather, ridiculously tall and spare, standing at the far
end of the gigantic room was as his imagination had foretold He
would be.
In the silence that greeted them Comstock finally turned to Grundy
and said, "I ... I don't understand."
The Grandfather walked to the head of the table and prepared to
speak. While they waited, Grundy whispered, "Think a moment. The
only cure for disease that our people know is vice. Right?"
Nodding, Comstock waited.
"But the only people on the whole planet who know how this cure
works, what psychic machinery is involved, are the Fathers."
Comstock gulped and thought of his heart.
"To become a Father," Grundy added hurriedly as The Grandfather
raised his hand for silence, "is a sentence of death. For once you
know how sin cures sickness, it can no longer cure you."
"Fathers," The Grandfather said, and involuntarily, Comstock felt his
heart fill with awe, so imbued had his upbringing been with respect
and worship of the figure called The Grandfather; he tried to control
the emotion that threatened to unman him, for his temptation was to
fall down before The Grandfather.
"Fathers," the deep organ bass went on, "you know why we are
gathered in this extraordinary conclave. So successful has been the
regime that I have caused to come into being, that no longer can we
hope to recruit new Fathers from amongst those brave souls who
rebel against the government we have set up. Not for fifty years has a
new rebel appeared to challenge our power. Therefore, as you all
know, Father Bowdler, because he is the healthiest appearing of any
of you, was empowered to go out into the world and find rebellious
souls whom we may be able to use as leaders.
"I feared when first I caused the apparatus of power to be set up as I
have done, that there would be instant and successful rebellion. It did
not then occur to me that I would be too successful and that rebellion
would be bred out of the blood of our people.
"We have, as you know only too well, arrived at a period of stasis
from which our world may never recover.
"It therefore devolves upon the men who stand before you as well as
the women who have made common cause with them, to come to our
aid.
"Now that aid is to be given, what these new Fathers will be able to
do before death claims them, I do not know. All that I can say with
any assurance, is that if something is not done and done quickly, our
world will go down the road to static death, never knowing what has
toppled it from the high estate it held."
CHAPTER 14
"Fear?" Comstock hazarded, for at the moment the word meant
nothing to him. Nothing at all.
"Fear," The Grandfather said, repeating the word again, "is my
bulwark. Cowardice my armor. I am the most frightened man in our
world. That is the reason I am The Grandfather. Until the day comes
that a more frightened man, a more cowardly human being arises, I
shall rule. No brave man can ever breach my defenses, because no
brave man can ever know the things I fear. Since I am always fearful
my mind is filled with ideas as to where and how I may be attacked.
Since this is so, I spend all my waking hours building up my guard
against any such attacks.
"The nights," he said thoughtfully, "I spend in nightmares in which all
my defenses crumble."
Comstock sat across the room from The Grandfather, his arms
enclosed in the cage like affair that immobilized him. Through
apertures in the walls at shoulder height he could see the stock-still
muzzles of the stun-guns that were trained on him. He brought his
attention back to The Grandfather. The man's long, thin face was
raddled with what seemed like fear. Tics jerked monstrously at the
corners of his mouth and at his hag-ridden eyes.
"How," Comstock asked, "can you sit under the menace of the guns
that surround us? Aren't you afraid that one of the gunners may shoot
you?"
"You see," the lean, bearded face was full of envy, "You see, you think
like a brave man and that is why you will never be able to overcome
me. Only a brave man could sit under the guns ... unless, he had the
foresight to have done what I have. Behind the gunners of which you
are aware, there is another set of gunners, each of whom has a gun
pointed at the head of the gunner who has been honored by being my
guard."
Comstock thought of this for a while and then he said, "And do the
secondary gunners have tertiary gunners menacing them?"
The Grandfather smiled delightedly, "There! You see, you are
beginning to think like a coward. Fear like mine is infectious. Of
course there are tertiary and quaternary and quinternary gunners!"
In the lengthening pause that followed this statement of The
Grandfather's, Comstock wondered if this was right, was fear the
thing that held him on the pinnacle he had made his own?
The Grandfather said, "I am not sure that I have convinced you.
Observe my face, the way fear tears at it. Consider that I am so
cowardly that my stomach digests itself rather than the food I force
into it. Realize that the only pleasure my fear allows me to enjoy is
that of power and then try to realize how helpless a brave man like
you who spreads his pleasure between the table and the bed must be
in the face of my one, all-consuming pleasure.
"You can eat for perhaps three hours a day," The Grandfather went
on, "depending on your sexual appetite and your years, you can
spend an hour, perhaps two in play at sex. But I can spend every
waking minute of every day on my pleasure."
He smiled. "You are helpless, bound by your bravery, you fool!"
And Comstock, considering the matter wondered if The Grandfather
was right. One Achilles heel alone remained to attack. Could a
coward foresee rashness, foolhardy bravery? Or would a coward be
unable to intuitively foresee such an action, to grapple with it; not that
he, Comstock, was brave.
Only one other way occurred to Comstock in which the matter could
be tested.
Leaning his upper trunk as far forward as his bonds would allow, he
said slowly, throwing his words into the teeth of the bearded man who
faced him, "You are a liar."
It is an understatement to say that The Grandfather was surprised.
His face was absolutely blank as he repeated the word, "Liar?"
questioningly.
Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.
textbookfull.com