Hindawi
Complexity
Volume 2019, Article ID 9424605, 27 pages
https://doi.org/10.1155/2019/9424605
Research Article
Expanding Network Analysis Tools in Psychological Networks:
Minimal Spanning Trees, Participation Coefficients, and Motif
Analysis Applied to a Network of 26 Psychological Attributes
Srebrenka Letina ,1,2 Tessa F. Blanken,3 Marie K. Deserno,4,5 and Denny Borsboom4
1
Department of Network and Data Science, Central European University, Hungary
HAS Centre for Social Sciences “Lendület” Research Centre for Educational and Network Studies (RECENS), Hungary
3
Department of Sleep and Cognition, Netherlands Institute for Neuroscience, Amsterdam, Netherlands
4
Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
5
Dr. Leo Kannerhuis and REACH-AUT, Doorwerth, Netherlands
2
Correspondence should be addressed to Srebrenka Letina; sreb.letina@gmail.com
Received 27 June 2018; Revised 16 October 2018; Accepted 29 November 2018; Published 21 February 2019
Guest Editor: Nicole Beckage
Copyright © 2019 Srebrenka Letina et al. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
The analysis of psychological networks in previous research has been limited to the inspection of centrality measures and the
quantification of specific global network features. The main idea of this paper is that a psychological network entails more potentially
useful and interesting information that can be reaped by other methods widely used in network science. Specifically, we suggest
methods that provide clearer picture about hierarchical arrangement of nodes in the network, address heterogeneity of nodes in the
network, and look more closely at network’s local structure. We explore the potential value of minimum spanning trees, participation
coefficients, and motif analyses and demonstrate the relevant analyses using a network of 26 psychological attributes. Using these
techniques, we investigate how the network of different psychological concepts is organized, which attribute is most central, and
what the role of intelligence in the network is relative to other psychological variables. Applying the three methods, we arrive at
several tentative conclusions. Trait Empathy is the most “central” attribute in the network. Intelligence, although peripheral, is
weakly but equally related to different kinds of attributes present in the network. Analysis of triadic configurations additionally
shows that the network is characterized by relatively strong open triads and an unusually frequent occurrence of negative triangles.
We discuss these and other findings in the light of possible theoretical explanations, methodological limitations, and future research.
1. Introduction
In the last decade, network approaches have been increasingly used in psychological science for the investigation of
psychological constructs and their interrelations in psychological science, as complementary or alternative to typically
used and well-established methods (e.g., confirmatory factor
analysis and structural equation modelling). This approach
has introduced a different perspective on psychological constructs and has found its application in many subfields of
psychology: intelligence [1], psychopathology [2], personality
psychology [3], and social psychology [4]. One specific asset
of the network approach is that it defines psychological
constructs as constituents of a complex system of direct
interactions enabling us to ask detailed questions about
relationships of mutual influence among these constructs [5–
8]. Specifically, Gaussian graphical models (GGM [9]) for
continuous variables and Ising models for binary variables
[10] have been used for network estimation with the aim
to describe conditional independence relationships between
variables, operationalized as partial correlations or conditional associations between variables [7, 11]. In this approach,
a psychological network consists of nodes, psychological
variables, and connections between nodes that represent the
degree (and direction) of associations between each pair of
variables, when the influence of every other variable in the
network is controlled for.
After the construction of psychological networks, the
quantitative analysis often proceeded with the computation
2
of a centrality analysis to answer which variable is most
“dominant” or “important.” Also, some global features have
been of interest, such as network connectivity [12]. However,
besides centrality measures and global measures of network
structure, which focus on microscopic and macroscopic level
of network, respectively, other analytical tools have been
mostly ignored and rarely used in the study of psychological
networks. This limited focus results in a limited set of
questions that can be answered. We argue that, in order
to answer research questions using psychological networks,
researchers should go beyond the measures commonly used
in psychology. The field of network science offers many alternative metrics that are worth considering when translating
one’s research question into quantifiable network properties.
The main idea of this paper is to apply such techniques, which
are already widely used in network science, to provide deeper
understanding of psychological networks.
The structure of the paper is as follows. Firstly, we will
describe some of the challenges in the analysis of psychological networks and link them with the three methods we
propose in this paper, following with the general overview
of the network that will be used for the demonstration
of these methods. Next, we describe an illustrative dataset
and apply the methods typically used in network analysis.
Subsequently, we explain three methods that can be used
to shed light on the network topology: minimum spanning
trees (MSTs), the participation coefficient (PC), and motif
analysis. For each method, we will explain specific procedures
and modifications and conclude with results and discussion.
Finally, in the general discussion, we summarize the benefits
and possibilities of including the proposed methodologies in
the field of network psychometrics and highlight interesting
hypotheses that we arrived at using these analytical tools.
1.1. Identifying Challenges in the Analysis of Psychological
Networks. In this paper, we propose three methods that not
only provide novel insights into the network, but also circumvent some prominent methodological issues in the field of
psychological networks: finding a way to operationalize the
importance of all variables included in the network in a more
general way, dealing with network of variables that are not
of the same kind, and how to investigate the intermediate
network level.
(1) Finding the hierarchical arrangement of nodes in the
network: The main purpose of centrality analysis used in the
analysis of psychological networks so far was to determine
how entities in the network may be ordered regarding their
connections with other variables (e.g., using the number and
strength of connections) and regarding their overall position
in the network, that is, to find out which entity is the most
“dominant.” The answers that arise from the application
of different measures (typically strength, betweenness, and
closeness) are likely to be different, as all of them capture
different notions of what centrality means. However, the
selection of the “right” measure is not the only challenge. Due
to the small and dense nature of psychological networks, centrality measures may not meaningfully differentiate among
specific nodes.
Complexity
As a solution to those issues we suggest the use of the
minimum spanning tree (MST), applied firstly on economics
in the stocks analysis of time-series data [13]. The MST is
a reduced subnetwork that connects all nodes based on the
identification of the minimal set of edges needed. Besides
providing a topological and hierarchically arranged skeleton
of all nodes in the network, it additionally provides an insight
into groupings of nodes based on their content similarity.
(2) The implicit assumption about the homogeneity of nodes
in the network: Most commonly used centrality measures are
based on a node’s relation to every other node in the network.
Thereby, these techniques implicitly assume that all nodes
are a priori equally likely to be connected with any other
node. This assumption is often untenable, as psychological
networks may include one or more entities, or groups of
entities, which differ in nature and/or measurement and
therefore constitute a cluster (referred to as community or
module, e.g. see [14]). In psychology, such a community
may arise in part because of preexisting differences between
the variables in, for example, nature of the variables (e.g.,
cognitive, behavioral, and emotional), kind of measurement
(e.g., subjective vs. objective), or some methodological aspect
of data collection.
In the estimated network, variables that are more similar
regarding these preexisting differences (i.e., that belong to
the same community in this sense) are more likely to be
associated than variables belonging to different communities.
Thus, these variables may show stronger associations among
themselves and will by construction rank higher on common
centrality measures like degree and strength. Note that this
effect is especially pronounced when the size of different
communities is not equal, as nodes belonging to the biggest
community will by default have higher degree and strength.
On the other hand, if some variables are different in some
of the aforementioned ways from other variables included in
the network, they may by default be expected to have less
strong connections with other variables in the system. As a
result, we might wrongly identify some node as central while,
at the same time, a variable with a truly important role might
be missed. This is important because psychological networks
are increasingly starting to include psychological entities of
different kinds. For example, recently some researchers [15]
called for inclusion of other variables besides symptoms when
analyzing psychopathological systems.
To circumvent the issue of nodes’ heterogeneity, we
propose the participation coefficient (PC [16]) to be used as
a corrective in the procedure of estimating the most central
node, because it addresses the uniformity of the edges a node
has to different groups of nodes in the network.
(3) The network’s mesolevel (or local structure): Visualization of small networks, such as psychological networks,
provides immediate insights into the dyadic relationships
between nodes, at the network as a whole, and even can
provide some notion of the grouping of nodes. Similarly,
measures typically used in the analysis of networks of
this kind cover analyses at the microscopic network level.
Macroscopic (global) properties of a network are easily computed, although their usefulness is less clear in psychological
networks due to their small size and the impossibility to
Complexity
claim that all relevant nodes are included in the network.
The interpretation of commonly used centrality measures and
global measures of network structure (e.g., average shortest
path and clustering coefficient) as reflecting the importance
of nodes in the system implicitly assume that the network
contains all factors that are relevant to the system. However,
one inherent characteristic of psychological networks is that
it rarely models all factors that are relevant to the system
[15]. In these cases, computing centrality measures based
on indirect ties (betweenness and closeness) and global
network measures may not capture all relevant information.
While this is a problem when analyzing the entire system,
much can be learned from shifting the focus to structural
patterns on a more fine-grained level (i.e., mesoscopic level,
“local” network structure). Methods for investigation of small
configurations in network have been first developed in social
network analysis [17] and have been redefined when applied
to different types of (usually large) networks (e.g., neuronal
networks, transcriptional networks, and the structure of the
Internet) at the beginning of the century and have become
known as “motif analysis.” Motif analysis enables researchers
to systematically investigate smaller configurations of nodes.
It can help us determine, among other things, whether certain
patterns, that is, subgraphs, represent interesting relations
between constructs or methodological artefacts.
Moreover, this method addresses one of the basic questions in modeling networks: how global properties of networks can be understood from its local properties and how
local topology is related to function [18]. For example, in
psychological networks, different measures of intelligence
are known to correlate positively; they show a positive
manifold. In the language of network mesolevel analysis, this
means that the system of different intelligence measures is
characterized by smaller local structures that display positive
relationships with each other. Van der Maas et al. [1] proposed
a dynamical model of intelligence in which these patterns
are interpreted as indicating that reciprocal causation or
mutualism plays the most important process in that system.
In other words, if a network expresses certain pattern of
relationships in “high” degree, it may inform us about underlying process(es) driving the system that is represented as the
network.
Each of the three methods, and especially the last two (the
participation coefficient and motif analysis), give a clearer
picture of all nodes in the network. It could be argued that
they provide a more “egalitarian” approach to nodes that
constitute a network, in a sense that they allow finding that
noncentral (in terms of strength, betweenness, or closeness)
nodes can be equally important for different parts of network
or have an interesting role in a smaller part of network. That
information can be easily overlooked when using only most
basic network analytics. Given that psychological networks
are usually relatively small, it is plausible that researchers will
be interested to learn more about each node in the network,
whether it is central or peripheral. Moreover, sometimes
nodes that are peripheral can be of special interest and/or
relevance (e.g., suicidal ideation in the network of depression
symptoms and intelligence in the network of psychological
traits).
3
1.2. Applying Three Methods in the Investigation of the Network
of Different Psychological Attributes. Network analysis has
been used mostly for looking more closely at one (or several related) psychological concepts, where nodes represent
psychometric items that are part of a self-report measure
(e.g., a questionnaire). In the current study, as an illustrative
dataset for the proposed methods, we look at a network in
which nodes are aggregated scores on self-report measures
(also known as “parcels” of a questionnaire) that operationalize different psychological concepts (e.g., latent variables),
most of which are not highly related, and among which
direct causal relations may not be assumed. The variables
in our network are supposed to measure relatively stable
individual differences whose development “proceeds along
mutually causal lines” [19, p.239]. Moreover, the conditional
associations between those constructs are likely to be small,
as most of them are assumed to be independent. To the
best of our knowledge, this is the first research that looks at
the network of different psychological attributes presented as
aggregated items. We use network approaches to gain new
insight into how different parts of that psychological system
are connected, and which attributes have the most prominent
role.
In the network of psychological constructs measured by
self-reports we included cognitive ability (a proxy of g-factor
[20]) measured with ability test (sometimes referred to in
psychology as subjective and objective tests, respectively).
The reason for including this substantially different variable
in the network is twofold. First, we aim to demonstrate
network methods that can provide more nuanced descriptions of all nodes, whatever their centrality in the network
is. Including a variable, a node, which is known to be
conceptually and methodologically different from others in
the network, and at best only modestly associated with
just some of nodes in the network, will set the stage for
demonstrating added value of proposed methods. Second, we
use the opportunity to address the old question of how cognitive ability and personality are related [21], to see how this
question can be formulated and answered within the network
approach.
Theoretically, intelligence is not expected to correlate with
personality domain. For decades, researchers dealing with
personality–intelligence connection have been using correlational studies to identify if significant relationship(s) exist(s).
Yet, as Eysenck [22] in his review of the topic concludes, the
research showed a striking lack of significant correlations,
with few exceptions. For example, small associations have
been found between intelligence and psychopathological
profile [23], and introversion-extraversion related differences
in style of intellectual performance (speed/accuracy ratio;
[24]). Seeing that this approach failed to find any substantial
relationship, Salovey and Mayer [25] suggest that question
should be asked in a more complex way, for example, looking
at the difference in the factorial structure of intelligence for
groups with different personality profiles, and vice versa.
Analytically, this suggestion is very much in line with network
approach, because it looks at the whole set of variables at
once, and is not as much focused on the size of specific
effects. From a theoretical perspective, several attempts of an
4
integrative approach to both personality and intelligence with
a wider theoretical framework for understanding their interrelations can be found in the literature, for example, social
intelligence theory within cognitive theory of personality
[26] and Motivational Systems Theory [27]. They are closely
related to Smirnov’s [28] view of intelligence as thinking, and
personality as inherent component of all thought processes,
while the link between the two is goals and problems in daily
life.
2. Methods
2.1. Data and Measures. The dataset used in the current study
has been collected within the context of the myPersonality
project [29, 30]. In this project, participants self-administered
one or more psychological questionnaires online, through
a Facebook application (active from 2007 until 2012). Participation was voluntary and completely anonymous, and
participants provided consent. In total, more than 20 different
questionnaires were offered, and participants completed a
self-chosen, variable number of questionnaires at a selfchosen place and time.
Of the available questionnaires, we selected 11 questionnaires, covering 31 psychological attributes, guided by
three criteria: we wanted to include psychological concepts
that (i) have a clear theoretical background and were measured with validated instruments with good psychometric
properties; (ii) are considered to have high temporal reliability and stability; and (iii) had relatively high number
(N>1000) of participants who also self-administered other
questionnaires. To prevent including concepts that are too
similar, we excluded concepts that correlated very highly to
other concepts (around 0.60 in absolute correlations) and
that had a clear theoretical overlap. This resulted in the
inclusion of 26 psychological concepts. To facilitate interpretation, we reversed the scores of the negatively framed
variables (Neuroticism, Depression, Militaristic values, and
Violent occult interests) such that all variables can be
interpreted as higher scores representing more favorable
outcomes, except for Schwartz’s values, where such rationale
was not possible since having or not having high scores
on certain value should be evaluation-free, meaning not
positive or negative by default. The interpretation of the
variables after recoding is listed in Table 1. More information on data processing, sample description, description of
missing data, and descriptive statistics of 26 psychological
variables is offered in the Supplementary Materials (SM,
Sections 1-4).
We included 1,166,923 participants with a score on at
least two of the psychological attributes (hereafter: variables).
Of a subsample of participants, demographic information
was available on gender (44.6%, of which 64.8% female and
35.2% male) and age (20.8%; M±SD = 26.1±6.7, range: 1489 years). The sample consisted of participants from 220
different countries, and 35,7% of participants were from the
US, UK, Canada, Australia, and India, respectively. A concise
description of the included constructs and the instruments
used is given in Table 1.
Complexity
2.2. Network Estimation. We used partial correlations to estimate (For network estimation, visualization, and centrality
analysis following R packages were used: BDgraph [42],
qgraph [43], and networktools [44]. MST, PC, and motif
analysis id done in NetworkX Python module [45]. Code
used can be provided from the first author upon request.)
the network. Partial correlation networks do not contain
spurious correlations that are generated by common cause
and chain structures within the network and can encode a
basic data-generating network structure [46]. To estimate
the network, we used a nonregularized method recently
proposed by Williams and Rast (in press) [47] because, given
our large sample size, relatively small number of variables,
and our interest to detect weak ties, it is not advised to
use regularization techniques like the LASSO that are often
used ([47, 48], in press). More details about the process of
determining the optimal estimation method for our data, and
about the nonregularization method used, can be found in
SM, Section 5.
To prevent the inclusion of spurious edges because of our
overall large sample size, we artificially reduced the sample
size by setting the N parameter in the estimation to N=4 131
(i.e., the median number of completed pairwise observations,
for more details see SM, Section 3) instead of the total sample
size of N=1 166 923. The estimated network is shown in
Figure 1. The included edges were significant at alpha level of
0.001. Note that partial correlations are usually smaller than
first-order correlations when interpreting the edge weights.
At first glance (Figure 1) at the network it can be seen
that most of the nodes from the same group (questionnaire)
cluster together in the network, except for Big Five traits that
are more scattered across the network, especially Openness.
2.3. Robustness Analysis. To check robustness of our results,
we tested it in two ways. First, we randomly split the sample in
half 100 times and estimate a network on each half separately.
Subsequently, we compare the two estimated networks on a
metric of interest. If the network estimation is reliable, then
the networks should be similar for both halves of the data,
and, hence, the metrics should show high correspondence.
This procedure is similar to that of Forbes et al. [49]. It should
be noted, however, that, by using only half of the data to
estimate a network, the statistical power drops considerably
which will especially affect the estimation of small edges.
Therefore, we conducted a second robustness analysis in
which we randomly selected 100 sets containing 80% of the
original sample and compared the network estimated on this
subsample to the network estimated on the complete dataset.
We computed the average correlation of the pairs of matrices
estimated for the split halves (robustness analysis I) and
between the whole sample and the random (80%) fractions
(robustness analysis II). For the split halves, the average
correlation was 0.82, indicating a high level of reliability.
However, if we only evaluate the edges that are present in both
estimates, on average, the reliability drops to 0.59 (similarity
index). The average difference in the number of edges is 6.35,
which is around 2% of all possible edges. For the random
(80%) fractions, the similarity index increased to 0.85. The
results are presented in more depth in SM, Section 6.
Complexity
5
Table 1: Description of 26 psychological attributes included in the network.
Psychological attribute (or Group of attributes (number of attributes in the group)), Questionnaire (author(s))
Short description of measured attribute (number of items)
Values - based on Schwartz Theory of Basic Values (6), Schwartz Value Survey – SVS (Schwartz, 1992) [31]
Achievement - personal success through demonstrating competence according to social standards. (4 items)
Hedonism - pleasure or sensuous gratification for oneself. (3 items)
Power - social status and prestige, control or dominance over people and resources. (4 items)
Self-direction - independent thought and action—choosing, creating, exploring. (5 items)
Tradition - respect, commitment, and acceptance of the customs and ideas that one’s culture or religion provides. (6 items)
Universalism - understanding, appreciation, tolerance, and protection for the welfare of all people and for nature. (8 items)
Big Five Traits (5), 20–100-item IPIP questionnaire (8 length versions), also included data on 336-item
IPIP Personality Facets questionnaire (Goldberg et al., 2006). Both questionnaires are proxies for Costa
and McCrae’s NEO-PI-R facets (Five Factor Model) [32]
Emotional Stability (reversed Neuroticism) - the tendency not to experience negative emotions, such as
anger, anxiety, or depression.
Extroversion - characterized by positive emotions, surgency, and the tendency to seek out stimulation and the company of others.
Openness to experience - a general appreciation for art, emotion, adventure, unusual ideas, imagination,
curiosity, and variety of experience.
Agreeableness - tendency to be compassionate and cooperative rather than suspicious and antagonistic towards others.
Conscientiousness - tendency to show self-discipline, act dutifully, and aim for achievement.
Interests (4), The Sensational Interests Questionnaire – SIQ (Egan et al., 1999) [33]
Low militaristic interests (reversed Militaristic interests) – an individual with low active interest in
militaristic activities (e.g. guns and shooting). (10 items)
Low violent-occult interests (reversed Violent-occult interests)– an individual with low active interest in
violent or occult activities (e.g. black magic). (7 items)
Intellectual interests – an individual’s active interest in cerebral activities (e.g. philosophy). (7 items)
Interest in wholesome activities – an individual’s active interest in active recreation (e.g. camping, hill
walking). (5 items)
Body Consciousness (3), Body Consciousness Questionnaire –BCQ (Miller, Murphy, & Buss, 1981) [34]
Private body - awareness of internal sensations. (5 items)
Public body - awareness of observable aspects of body. (6 items)
Body competence – self-confidence in the body’s performance. (4 items)
Integrity assessment (2), Rust’s Sense of Fairness and Impression Management, Orpheus (Rust &
Golombok, 1989), 36 items. [35]
Fair-mindedness (or Sense-of-fairness) – measures how balanced and impartial person is in her decision
making.
Self-Disclosure – measures to what extent a person conducts her life transparently. Reversed values are
used as a measure of Impression Management and Social desirability (Lie scale).
“Stand-alone” traits – six psychological attributes which are not part of a group of constructs, each is measured with separate questionnaire
Awareness of physical symptoms and sensations, Pennebaker’s Inventory of Limbic Languidness - PILL
(Pennebaker, 1982) [36]
Scale measures how often a person notices and reports a broad array of physical symptoms and sensations (e.g. chest pain, heart racing,
dizziness). (54 items)
Self-monitoring, Snyder’s Self-Monitoring Scale, (Snyder, 1974) [37]
Scale measures how much person monitors her self-presentations, expressive behavior, and nonverbal affective displays. (25 items)
Low Depression, Center for Epidemiologic Studies Depression Scale (CES-D), NIMH, (Radloff, 1977) [38]
6
Complexity
Table 1: Continued.
Reversed Depression, measures lack of symptoms of depression in nine different groups as defined by the American Psychiatric Association
Diagnostic and Statistical Manual, fifth edition. (20 items)
Empathy, Empathy Quotient - EQ, (Baron-Cohen & Wheelwright, 2004) [39]
Scale measures self-reported ability to tune into how others are feeling, and to understand what they
may be thinking. It measures both the affective and the cognitive components of empathy. (60 items)
Life satisfaction, Satisfaction With Life Scale- SWLS (Diener, Emmons, Larsen, & Griffin, 1985) [40]
Scale measures general wellbeing and satisfaction with one’s life. (5 items)
Intelligence, MyIQ test,myPersonality’s 20-item proxy for Raven’s Standard Progressive Matrices (Raven,
2008) [41], University of Cambridge’s Psychometrics Centre (Chan & Kosinski)
Ability test measures cognitive skills and clear-thinking ability, and pattern recognition abilities known to
have the highest correlation with the general intelligence factor. (20 items)
Table 2: Description of ties in partial correlation network.
Mean
SD
Min.
25%
Mdn
75%
Max.
N. of ties
Signed ties
0,01
0,158
-0,39
-0,09
0,06
0,11
0,53
144
Absolute weights
0,13
0,090
0,05
0,06
0,10
0,16
0,53
144
3. Illustrative Results: Network Description
3.1. Edge Weights in the Network. The current estimated
network has 144 edges out of 325 possible edges, showing a
good balance between sparsity and density (Figure 2). The
distribution of the edges is summarized in Table 2; 64 edges
(44%) are negative and 80 edges (56%) are positive. The
number of negative edges is higher than usually observed
psychological networks. Note that this is dependent on the
network under consideration. If a network includes variables
that all come from the same questionnaire (e.g., 10 depression
items), then it would be expected that many (or all) edges are
positive. In the current network, variables from various psychological questionnaires are included; they are not expected
to correlate highly or/and positively by definition. Figure 2
also shows that, due to artificially decreasing statistical power
and due to setting alpha to 0.001, edges around 0 are
eliminated (< 0.05 in absolute value). For more details on
the correlation network and estimated partial correlation
network, and detailed analysis of ties, see Sections 7 and 8
in SM.
3.2. Centrality of Nodes. In addition to centrality measures
that are typically used in psychological networks, we include
more recently developed measures of node’s expected influence ([50], for short explanation see Section 9 in SM).
Centrality measures can roughly be categorized into two
groups, measures that look only at the local surroundings of a
node (i.e., only the edges adjacent to the node) and measures
that try to quantify the position of a node in the network by
Positive ties
0,13
0,092
0,05
0,07
0,10
0,15
0,53
80
Negative ties
-0,13
0,088
-0,39
-0,16
-0,10
-0,06
-0,05
64
also taking into account nodes that are not directly adjacent
to the node. Figure 3 shows centrality measures of the first
category—considering only adjacent nodes. Figure 4 shows
centrality measures of the second category—considering
nodes beyond those directly adjacent to the node of interest.
Comparing the different centrality measures, both within the
same category or across categories, clearly shows that the
measures diverge. Thus, different centrality measures indicate
different nodes to be the most central. Although this follows
logically from the way the different measures are computed,
as each measure captures different aspects of centrality, it
highlights the need to carefully consider the metrics used
as it can greatly influence the answer to the question that is
posed.
As Figure 3 shows, based on a node’s direct ties, the
most central node varies across measures. Based on strength,
the value Tradition is the most central node, followed by
Empathy, Extraversion, and another value, Universalism.
Among the least central nodes are Agreeableness, Body
Competence, and Awareness of Physical Symptoms.
Alternatively, when centrality measures consider more
than the local environment of the node, a different arrangement of centrality emerges (Figure 4), with less agreement
between different measures. Here, Empathy is the most central node, followed by Extraversion and Emotional Stability,
while Tradition drops to the fourth place. The least central
nodes are Self-Disclosure, Intelligence, and Awareness of
Physical Symptoms.
Robustness analysis of all centrality measures used in this
study is presented in Section 6 of SM.
Complexity
7
Figure 1: Nonregularized partial correlation network (set N= 4131, true N=1066921, layout spring, cut = 0). Blue ties signify positive relations
and orange ties signify negative relations. The thickness of a tie is proportional to its absolute weight.
8
7
6
5
4
3
2
1
0
−0.4
−0.2
0.0
Weight
0.2
0.4
Figure 2: Distribution of weights in partial correlation network (𝐿 = 144).
4. Introducing Three Network Methods for the
Analysis of Psychological Networks
4.1. The Minimum Spanning Tree. As demonstrated in previous section, different centrality measures capture different
aspects of a node’s position in the network, and the centrality
of a node will differ depending on the centrality measure
used. For that reason, we propose a way to look at the question
about centrality differently, in a more general way. To be clear,
we are not stating that centrality measures used so far in the
research are inadequate, but we are merely trying to ensure
a more general perspective to centrality. An alternative way
to characterize relationship between all nodes in a network is
by computing the minimum spanning tree (MST) [13]. The
MST detects the hierarchical organization of the nodes and
reduces the number of edges to those that carry the most
information on the similarity of the nodes. Specifically, the
MST is based on the distance between the nodes and selects
the subset of edges (number of nodes – 1) without cycles, and
with minimal total distance possible. This “skeleton” structure
of the filtered network may be used if we want to get the
answer to the general question which node is the most central,
by not looking at the specific centrality aspects, but instead
focusing on the network’s most essential and local ties.
To compute the MST of our current network, first the
distances among the nodes must be computed. An appropriate function for converting correlation to distances when
negative correlations are present is as follows:
𝑑 (𝑖, 𝑗) = √2 (1 − 𝑟𝑖𝑗 )
(1)
8
Complexity
3
2
1
0
−1
−2
Awareness of physical symp.
Body competence
Agreeableness
Wholesome activities interests
Life satisfaction
Intelligence
Self-monitoring
Low Depression
Intellectual interests
Low violent-occult interests
Openness
Self-Disclosure
Public body
Achievement
Private body
Fair-Mindedness
Power
Self-direction
Low militaristic interests
Hedonism
Conscientiousness
Emotional stability
Universalism
Extraversion
Empathy
Tradition
−3
Num.of Ties
Strength
Exp.Iinfluence-step1 (abs)
Figure 3: Centrality measures 1: based on node’s direct ties (standardized values).
Equation (1) (Gower’s distance measure [13, 51]) takes
the direction of the correlation into account by assigning
the largest distance to a perfect negative correlation, and the
smallest distance to a perfect positive correlation. According
to this equation the distances range from 0 to 2, where
an intermediate distance of 1.4 is assigned to variables
that are uncorrelated. The relationship between the (partial)
correlation coefficient and the distance measure is shown in
Figure 5.
Equation (1) is the preferred distance measure to distance
inversely proportional to shared variance (𝑑(𝑖, 𝑗) = 1 − 𝑝𝑟2𝑖𝑗 ).
From the mathematical point of view, it is a more rigorous
definition of distance and it gives monotonic transformation
of coefficients. Most importantly, (1) gives more differentiated
measure of distance than distance based on the shared
variance, because in the latter the loss of information occurs
since it translates partial correlations of the opposite sign and
the same absolute values to the same distance. If negative ties
are not present in the network, both measures will produce
the same MSTs; otherwise the output will most likely differ
(MST based on the shared variance is shown in SM, Section
11, Figure 15). Given the mentioned advantages and since
almost half the ties in our network are negative, we have
chosen to use it for MST construction. However, as it will be
discussed in Section 5 and analysed in SM, Section 12, this
measure is sensitive to reverse coding of variables included
in the network.
Note that taking partial correlations instead of correlations when calculating distances means that, for each pair of
nodes, it indicates how distant they are after the similarity
based on covariance with other nodes in the network is
excluded.
The MST of 26 psychological attributes is shown in
Figure 6. The information about “centrality” of a node is
very clear from the hierarchical structure, although centrality
measures can provide more detailed picture (see SM, Section
10). The nodes with more direct edges and closer to the
middle (centre) of the tree are most central.
Empathy is the most central node in the MST in the sense
that it features the smallest distance to all other attributes.
From Empathy, four branches emerge with only Sensational
Interests and Body Consciousness being on the same branch
as all other attributes from the same group. All branches
are heterogeneous regarding the group of attributes they
consist of, but they can be interpreted as having some
commonalities in meaning. The branch with three Body
Consciousness constructs along with Awareness of Physical
Symptoms captures attributes related with body perception
in general. The branch starting with Low Militaristic Interests
can be interpreted as representing interests, values, and
openness, which are related to what is often referred to as
“lifestyle.” The branch that starts with Extraversion relates
to the attributes that describe one’s agency and control in
social world. Finally, the biggest and most heterogeneous
Complexity
9
3
2
1
0
−1
−2
Awareness of physical symp.
Intelligence
Self-Disclosure
Openness
Body competence
Private body
Self-monitoring
Hedonism
Wholesome activities interests
Life satisfaction
Achievement
Self-direction
Low Depression
Power
Conscientiousness
Intellectual interests
Low violent-occult interests
Public body
Fair-Mindedness
Agreeableness
Low militaristic interests
Universalism
Tradition
Emotional stability
Extraversion
Empathy
−3
Betweenness
Closeness
Exp.Iinfluence-step2 (abs)
Figure 4: Centrality measures 2: based on links more than one distance away from the node (standardized values).
2.00
1.75
Distance
1.50
1.25
1.00
0.75
0.50
0.25
−1.00 −0.75 −0.50 −0.25 0.00
0.25 0.50
(Partial) Correlation Coefficient
0.75
Figure 5: The relationship between partial or correlation coefficients and distance measure.
branch starting with Agreeableness is made of attributes that
are highly socially esteemed and describe one’s “relation” to
others, oneself, and life in general. It is interesting to observe
that Intelligence is placed on that branch and it branches
out from Fair-Mindedness. This visual inspection shows
another useful feature of MST; it gives indirect information
on the hierarchical and overlapping, data-driven, clusters
in the network. For example, in Figure 6, we can see two
pairs of branches, or clusters, which overlap in Empathy.
Alternatively, taking Empathy as the origin, there are four
branches, or clusters, that overlap in that node.
According to the MST based on the distance defined
in (1), two nodes are more distant in terms of steps (ties)
between them in the filtered network (tree) if they are
negatively associated than if they are not associated at all.
That is why, for example, Tradition and Self-Direction (pr
10
Complexity
Achievement
Intelligence
Conscientiousness
Self-disclosure
Tradition
Power
Life satisfaction
Fair-mindedness
Self-monitoring
Emotional stability
Extraversion
Low depression
Agreeableness
Private body
Low militaristic interests
Low violent-occult interests
Empathy
Public body
Body
competence
Universalism
Awareness
of Physical
symptoms
Hedonism
Wholesome activities
Intellectual interests
Openness
Self-direction
Six narrow (“stand-alone”) traits
Four sensational interests
Six Schwartz's values
Body consciousness (three nodes)
Big Five personality traits
Integrity (two nodes)
Figure 6: Minimum spanning tree (MST) based partial correlation.
= - 0,37) are placed on different branches and are more
distant than Emotional Stability and Conscientiousness (pr
= 0) that lie on the same branch. From the perspective
of psychological networks, the MST preserves the specific
content and meaning of the variable. More importantly, since
its construction was affected by signs of weights, not only
their absolute value, this filtered network can be a useful tool
in testing whether two networks made of the same nodes
really differ. Two networks estimated on two different samples
will not usually be identical. However, if their MST is the same
or very similar, this may indicate that their differences are
not important. Similarity indexes of MSTs converge with the
similarity indexes of whole networks. Nevertheless, reliability
based on MST correlations seems to be lower than that based
on network correlations in smaller samples (split halves),
indicating that in fact the most informative ties are differently
estimated (for details see Section 7 in SM).
4.2. The Participation Coefficient. In psychological networks,
nodes (variables) may differ in their nature. Some may
come from the same framework, while some may be standalone nodes. In network parlance, some nodes are part of a
community and some nodes form a community of one or few.
Note that these communities are not derived from data, but
rather they are based on preexisting differences.
In the current dataset, for example, we had 26 psychological concepts, measured by 11 questionnaires. As such, there
are groups of variables, varying in size, that belong to the
same questionnaire and that are part of the same theoretical
framework (e.g., three concepts on body consciousness) or
measure the same kind of trait (e.g., measures of different
“values”). Moreover, the psychological concepts that are part
of the same questionnaire will likely be completed at the same
time, while different questionnaires may have been taken
days, months, or even years apart. Therefore, it is important
to take these preexisting differences into account, if we want
to explore which of the variables play an important role in the
network.
One way to deal with these theoretically defined, preexisting communities is by employing measures that take this
community structure into account and specifically evaluate
connections a node has with nodes in different communities.
One such method is the Participation Coefficient (PC), first
introduced in the field of biological networks [16]. The PC
takes the community structure into account, as it specifically
quantifies how the edges a node has are distributed to
different communities (similar in logic to Shannon entropy
measure.). The important departure in our application of
the PC is that it is not used on an empirical community
structure, but rather on “communities,” that is, groups of
nodes and “stand-alone” nodes that were considered to exist
in the network (a kind of “ground truth”). Framed as a
hypothesis, the null hypothesis in the use of PC would
state that preexisting groups of constructs (or data-driven
communities) do not influence centrality scores of nodes.
Showing that the rank order of nodes according to given
centrality measure changes once the measure is corrected
Complexity
11
with PC can be interpreted as supporting the rejection of the
null hypothesis.
The calculation of the PC measure follows
𝐺
𝑃𝐶𝑖 = 1 − ∑ (
𝑚=1
𝑘𝑖,𝑚 2
)
𝑘𝑖
(2)
where 𝑃𝐶𝑖 signifies the PC score for a node i, while G, 𝑚,
𝑘𝑖,𝑚 , and 𝑘𝑖 denote the network, each module in the network,
number of ties of node i with nodes in that module, and
number of all node’s ties, respectively. The expression 𝑘𝑖,𝑚 /𝑘𝑖
is simply the ratio of all node’s ties that go to the specific
module. In a version for weighted networks the number of
links (𝑘𝑖,𝑚 /𝑘𝑖 ) in (2) is replaced with the sum of strengths
which means that the expression 𝑠𝑖,𝑚 /𝑠𝑖 signifies proportion
of total strength of node i, invested in a single module:
𝐺
𝑃𝐶𝑖 = 1 − ∑ (
𝑚=1
𝑠𝑖,𝑚 2
)
𝑠𝑖
(3)
This difference means that if a node has the same number of
links to every module, but they differ in strength, it will not
achieve a maximum PC value. Here, strength is defined as the
sum of absolute weights of all links involving node i, which
means we disregard the sign of ties.
If a node has an equal number of edges to all the communities in the network (i.e., a uniform distribution of edges to
all communities), the PC is closer to 1 (the highest possible
value depends on the number of modules in the network;
therefore average PCs of different networks can be compared
only if PCs are normalized by theoretical maximum value,
which is 0.50 for 2-module community, 0.80 for a network
containing 5 communities, and, in our network containing 11
communities, the maximum PC value is 0.96). Alternatively,
if a node has edges only to nodes within its own community,
the PC is 0. It is important to note that the PC is not simply
the number of links a node has to other communities in the
network, but it rather quantifies the equality of the distribution
of edges a node has to the other communities. In weighted
networks, the PC is maximized if a node is connected equally
to all the communities in the network: equal in both the
number and the strength of edges to the other communities
(i.e., a uniform distribution of edges and edge weights to
all communities). More uniform distributions of nodes to
all other communities correspond to higher PC values. For
example, a node with one tie to each module will have the
same PC as a node with two ties to each module. Similarly,
a node with just one link to each module will have a higher
PC than a node who has many links to some, but no links
to other modules. A node with a high PC can influence
all parts of the network equally, meaning that the node
is equally important to every defined community. Such a
node can be seen as a common denominator in terms of its
potential influence on all communities in the network and
can therefore help us understand the network as a whole. Note
that PC considers only the node’s direct ties, displaying the
local perspective as MST. Moreover, that feature makes it very
suitable for the analysis of a network where some elements
of the network may not be included, and where therefore
measures relying on the whole network (e.g., betweenness
and closeness) may not be appropriate. However, since the
PC solely quantifies the equality of the distribution of ties (or
strength of those ties, in version for weighted networks) and
disregards number (sum of strengths) of ties, we propose to
use it in combination with a measure that considers both the
number and the strength of the connections a node has and
disregards the information about communities (preexisting
or otherwise). One such measure is the Participation Ratio
(PR [52]). Participation Ratio is defined [2] with the following
formula:
𝑠𝑖 𝛼
(1−𝛼)
𝐶𝑤𝛼
× 𝑠𝛼𝑖
𝐷 (𝑖) = 𝑘𝑖 × ( ) = 𝑘𝑖
𝑘𝑖
(4)
where 𝐶𝑤𝛼
𝐷 (𝑖) is Participation Ratio of node (i), 𝑘𝑖 is number
of ties of node (i), 𝑠𝑖 is the strength of the node, while 𝛼 is a
positive tuning parameter. If its value is set between 0 and 1,
having a high number of ties (degree) increases 𝐶𝑤𝛼
𝐷 (𝑖), if 𝛼 =
0, it is equal to the node’s strength, whereas, if 𝛼 is set above 1,
the number of ties decreases the value of 𝐶𝑤𝛼
𝐷 (𝑖), in such a way
that a node with a greater concentration of its strength on only
a few nodes and low degree has higher value than a node with
the same strength but more ties. In our analysis, the 𝛼 is set to
0.5, so that, for example, if a node A has a higher number of
links and the same total strength as node B, the node A will
have higher value of 𝐶𝑤𝛼
𝐷 . In this way both having high total
strength and having more ties is favoured.
In short, PR is a single measure that quantifies both the
number of edges a node has and the strength of these edges
and weighs both equally (i.e., corresponding to an alpha of
0.5), and, as other measures defined so far in this paper,
focuses only on node’s direct links.
We transformed both measures to the same scale (range
0-1), visualized in Figure 7. Subsequently, for each node we
computed the geometric mean of both measures. We opted
for the geometric mean as it rewards consistency in scores on
the two different measures. For each node, the PC, PR, and
its geometric mean are shown in Figure 8.
Interestingly, as can be seen from Figures 7 and 8, the
PC and PR can diverge for some nodes. For example, if
we only focus on the number of edges and their strength,
which is summarized in the PR, Tradition is highly central.
However, Tradition has a relatively low PC, indicating that
while it has relatively many and strong edges, these are
not equally distributed throughout the network. Inspecting
the estimated network in Figure 1, it can be seen that,
indeed, the strongest edges of Tradition are mainly within its
own community. Alternatively, Intelligence is not considered
central based on the number and strength of its edges, but,
taking the distribution of edges into account, we see that
the connections of Intelligence are equally distributed to the
other communities in the network (see Figure 16 in SM). This
information would have been lost, if we had only focused on
the number and strength of the edges (and other centrality
measures related to these aspects).
In short, this example clearly illustrates that, when the
objective is to find out which nodes play an important role
in the network as connectors, it is important to consider
whether there might be preexisting communities that should
12
Complexity
Extraversion
Emot. stability
Fait-Mindedness
1.0
Self-Disclosure
Conscientiousness
0.9
Intelligence
Self-monitoring
Participation Coefficient
Agreeableness
0.8
Intellectual int.
Private body
Openness
Self-direction
Empathy
Hedonism
Awareness physical symp.
Militaristic int.Violent-occult int.DepressionPublic body
Power
Universalism
Achievement
0.7
Tradition
Life satisfaction
0.6
Wholesome act. int.
0.5
Body competence
0.2
0.4
0.6
0.8
1.0
Participation Ratio
Figure 7: Scatterplot of standardized values of participation coefficient and participation ratio (min-max scale from 0 to 1) for all nodes in
the network.
be taken into account. Not taking these preexisting communities into account might obscure the importance nodes
belonging to small communities and “stand-alone” nodes
that are not part of any community.
4.3. Analysis of Triadic Motifs. In this section, first we will
explain the rationale behind the selection of motifs to be
investigated, and the analysis of motif frequency, intensity,
and coherence, followed by results and discussion, where the
identification of specific motifs (and interpretation) is also
included.
(i) Selection of motifs: Motifs usually represent subgraphs
of three to five nodes for which different patterns of absent
and present ties are examined. Many analyses of mesoscopic
structures include or focus on triads, all possible configurations of three nodes. This is a sensible choice, because a triad is
the smallest and the most basic network unit that defines the
clustering of a network (transitivity) and can be characterized
as the “simplest nontrivial motif” [53, p.2]. For undirected,
unweighted, and unsigned networks, four types of triads
exist: (1) triads without ties/edges (empty triads); (2) triads
with one tie present, and two ties absent (one edge triads); (3)
triads with one edge absent, and two edges present, referred
to in the literature as two-path, two-star, or open triads (or
forbidden triads in weighted networks when present edges are
strong); and (4) triads with all edges present (triangles, closed
triads) (Triads should not be confused with triplets. Triplets
are like triads, but they are defined only by the presence of the
edges and do not by the absence of edges. For example, both
triangles and open triads are triplets of two edges.). Usually,
the first two types of triads are not considered in the analysis,
and some researchers define triads more strictly as systems
of three nodes with at least two ties among them (e.g., [54]).
The number of possible triads increases when the sign and
weights of the edges are considered (e.g., [55]), as will be done
in our analysis. Depending on the research question, some
motif configurations may be of special interest and should be
investigated, while others can be excluded from the analysis.
(ii) Analysis of motif occurrence, intensity, and coherence
(including the identification of specific motifs): Once the motifs
of interest are defined, the next step is to determine the frequency of each motif in the empirical network (each unique
combination of three nodes is counted once). This yields a
first insight into the network patterns at the mesolevel. The
most frequent motif describes the most dominant pattern
of connectivity in the given network among the motifs
that are examined. However, the frequency alone yields
Complexity
13
3
2
1
0
−1
−2
Intelligence
Wholesome activities interests
Low violent-occult interests
Body competence
Awareness of physical symp.
Agreeableness
Low Depression
Life satisfaction
Self-Disclosure
Self-monitoring
Achievement
Public body
Low militaristic interests
Openness
Intellectual interests
Private body
Self-direction
Conscientiousness
Power
Universalism
Hedonism
Tradition
Fair-Mindedness
Empathy
Emotional stability
Extraversion
−3
Participation Ratio (PR)
Participation Coefficient (PC)
Geometric Mean of PR & PC
Figure 8: Centrality measures 3: participation ratio (the values of geometric means for Empathy and Extraversion are higher than both PR and
PC. This is due to standardization of each measure. The plots with raw scores are shown in SM, Section 9, Figure 10.) (𝛼 = 0.5), participation
coefficient, and their geometric mean (standardized values).
A
B
C
D
Figure 9: Four networks with 12 nodes and one negative triad (NNN). Networks A, B, and C have only three negative edges, while D has the
same structure and density (number of edges) as A and B, but more negative edges.
limited information, because certain motifs might occur
more frequently simply because of the network structure
(in the context of describing the reference (null) model, the
terms: network structure, topology, and degree sequence,
are used interchangeably in this paper) and weight distribution. For example, imagine a hypothetical network of
twelve nodes (variables) in which we observe predominantly
positive edges, representing partial correlations between pairs
of variables, except for three negative edges (described in
Figure 9).
If we find one negative triad in a network, based on
frequency alone, we could treat that finding as somewhat
interesting but not especially informative about the network
as a whole. However, when we consider what the chances
are of observing three nodes connected with three negative
edges in that system, that finding is of greater importance
14
for understanding the whole network as a system. Figure 9
describes extreme (and unlikely) examples of psychological
networks which are used to illustrate why it is useful to
additionally look at the chance of certain motifs occurring
in the system. The weight distributions of networks A and
B in Figure 9 are the same, while network C has a different
structure compared to A, B, and D, because just one closed
triad (triangle) is present. Since the structure is different, the
weight distribution of C is also different. The chance of a
NNN occurring in a network with the same structure and
weight distribution is smallest in C, followed by A and B,
where it is equal. The highest chance of observing such a triad
is in D, because it has more negative edges and triads than
other three networks. If networks are representing symptoms
(behavioral, emotional, cognitive, or physical) of a disorder,
three negatively associated symptoms in A, B, and especially
in C are more important characteristic of the system than
in network D. They are less likely to occur by chance in
these three networks, and therefore more likely to describe a
process which is important for understanding the network.
For example, a triad NNN in A could be interpreted as a
process of negative feedback which is central for the network
(it “drives” the network). In B, NNN is equally important but
it describes occurrence of a negative “loop” in a peripheral
part of the network, among symptoms that are less central. In
C, NNN is even more essential for understanding the network
than in A, as it could be described as the sole driving force
of the network, each of the negatively connected nodes in the
triad relates to a different set of nodes. Note that motif analysis
per se does not differentiate between A and B as the centrality
of configurations is not accounted for. Finally, NNN in D is
a central configuration which shows an interesting pattern
of association between three symptoms, worth of attention
in the interpretation of the network. However, it is not as
important for describing the process underlying the network
formation since other negative associations between nodes
and within triads are present. The same reasoning applies if
nodes are representing other nonpathological tendencies, like
personality traits, values, etc. In these networks the difference
will be in the average weights of edges, which is likely to be
smaller than in case of networks featuring psychopathological
symptoms or other more correlated variables.
Therefore, for each motif, we establish whether it occurs
more or less frequently than would be expected by a null
model. In weighted networks, the appropriate null model
is a random network (to be precise, it is not a random
graph model, but a configuration model (for more details see
[56])) with fixed topology (degree sequence) and randomized
weights from the same distribution of weights as observed in
the empirical network (for more details on general null models see [57]). The quantification of occurrence of a specific
configuration in a network is usually done by comparing it
with the occurrence of the same motif in a reference model
(for introduction see [18]). Distribution of motif frequencies
is obtained by generating a sample of random networks. The
empirical frequency of a motif is compared against that distribution and if it appears significantly (this significance should
not be confused with significance of ties in the motif) more
(less) often than it would be expected by reference model
Complexity
it signifies the motif is indeed “a motif.” (Sometimes the
term “motif” is used only for these configurations for which
this step of analysis shows that they are significantly overor underrepresented. In this article, we do not make such
distinction, as we refer to every investigated configuration
as a motif, and after the analysis is done, we describe it as
significant or not.) Itdescribes an important characteristic of
the investigated network. Motifs that occur more frequently
describe a common configuration of nodes and therefore
provide information about the network connectivity. Moreover, these motifs could have some important functional
roles in the system. For example, closed triads are usually
overrepresented in social networks, because they represent
a process of social (triadic) closure, while in a network
of intelligence measures they may indicate the process of
mutualism [1].
However, in weighted networks the analysis of motif
frequency omits the information about the weights (unless
it is in some ways included in the definition of the motif).
For example, if two motifs have the same occurrence in
a network (let us assume for the sake of the argument
that both have equal distribution of frequencies based on
appropriate random models), but the first is (on average)
made of stronger ties than the second, we cannot treat them as
equally describing the local structure of the network, that is,
to be equally likely to describe some important process in the
network. Although they are equally present in the network,
the first is expressed more strongly and is therefore more
likely to describe some important process.
To address this issue, Onnela et al. [53] introduced the
Intensity measure (the geometric mean of all the weights (in
the case of absent ties in the motif, these are treated as zero
weights) in a motif ( (5), where 𝑙𝑔 stands for number of ties in
the motif)), which looks at the motifs not as discrete objects
who are either present or not (expressed or not expressed) in
the network, but rather as objects existing on a continuum,
where zero or low Intensity values imply that motif is present
in low degree. As such, the Intensity I can be used to identify
high and low Intensity motifs in the system:
𝐼(𝑔) = ( ∏ 𝑤𝑖𝑗 )
1/|𝑙𝑔 |
(5)
(𝑖,𝑗)∈𝑙𝑔
In addition to Intensity (I), a Coherence (𝑄(𝑔) ) ratio can
be computed that quantifies how internally coherent the
weights in motifs are by computing the ratio between the
geometric and the arithmetic mean. It ranges from 0 to 1, with
higher scores indicating less difference between the weights
(in absolute terms). As was the case with the analysis of
occurrence of motifs, the significance of both Intensity and
Coherence is estimated in comparison with the distribution
of their values for a given motif in reference model.
A motif that is underrepresented in the network, in terms
of occurrence or intensity, describes a pattern of relationships
which, for some reason, is unlikely to happen in a network.
In other words, when we exclude the hypothesis that a given
occurrence or intensity of a certain configuration does not
come from a reference system, it points out that there may
Complexity
be an additional origin for the effect, possibly the function
of the system [18]. In case of psychological networks, the
occurrence and significance of a motif which is not easily
interpretable may also happen as an artefact (e.g., due to the
sample on which the network is estimated, problems in the
network estimation procedure, or measurement error). For
that reason, a motif analysis can be useful in the analysis of
psychological networks, forasmuch as it can help quantify
and identify presence of unexpected configurations in the
network as well.
In the next section, the motif analysis on illustrative data
is described in detail and results are presented and discussed.
4.3.1. Selection of Motifs and Analysis of Motif Occurrence.
When the sign of an edge is considered, seven configurations
of triads are possible (disregarding empty triads and triads
with only one edge, see Figure 9). Four of them fall under
“closed” triads or triangles: triads with either only positive
(positive triad, PPP) or only negative weights (negative triad,
NNN) and triads consisting of two positive and one negative
weight (PPN) or two negative and one positive weight (NNP).
NNN and PPN are also known as imbalanced triads (NNN
is also sometimes considered as imbalanced triad in social
networks, but some debate exists over whether it is truly
imbalanced or not. Not to confuse with too many similarly
named triad, we will use the term “imbalanced” in this
article only when referring to triad with one positive and
two negative ties and to triads that do not satisfy the triangle
inequality principle (the latter is explained in the following
text)), in social balance theory [58, 59] because they signify
configurations of affective ties between persons which is not
likely to appear in social networks (or if it appears it is not
likely to persist; that is, it is likely to change). The remaining
three triads are open triads (2paths) consisting of two ties:
with only positive weights (2path pos., P0P where “0” stands
for the absent weight), only negative weights (2path neg.,
N0N), or with one positive and one negative weight (2path
mixed, P0N or N0P).
Networks, especially social networks, tend to show transitivity; if person A is connected with (friend of) person B,
who is connected with (friend of) person C, A and C are
likely to be connected (friends). Although, in recent years, we
have witnessed a surge of research on psychological networks,
we still do not know enough about their general properties.
Correlations, and especially partial correlations, do not have
to be transitive, but it is often the case that if a trait A positively
correlates with trait B, which is also correlated positively
with trait C, then we expect traits A and C to correlate
positively as well. If that is the case, P0P motifs should appear
less often than expected by the reference model. Likewise,
according to the social balance theory, closed triads with one
or three negative edges (i.e., PPN and NNN) are less likely
to occur in social networks [58–60]. We hypothesize that, in
psychological networks too, NNN and PPN triads represent
configurations which are not expected to occur frequently
because of two reasons. First, it is challenging to explain
how three psychological attributes feature negatively partial
correlations. One possibility is that a process of negative
15
feedback among attributes exists. A second possibility is that
the three nodes positively contribute to a common effect,
which has been implicitly or explicitly conditioned on. A
third possibility is that the variables are measured with error,
and the partial correlation picks up negative correlations
between the error terms.
On the other hand, positive associations between A and B,
and B and C, render a possible negative association between
A and C difficult to interpret (PPN triad). The importance
of detecting such configurations in psychological networks
lies in the fact that they either describe unusual finding(s)
or may point to the existence of methodological artefacts. In
both cases, we benefit from knowing about the presence of
such configurations. It should be noted that, while it is more
straightforward to predict that such configurations could be
less frequent in a correlation network, in the case of partial
correlation network they could be more likely to occur. To
the best of our knowledge no analysis of this kind has been
performed on a network representing (partial) correlations.
The summary of hypotheses is shown in Table 4, in Section 5.
Among the motifs (Figure 10, third row), the only significant motif is the negative triad (percentile 99.7). In other
words, the negative triad appears more frequently than would
be expected by chance, given the same degree sequence and
weight distribution. Path2 with positive ties (P0P), indicating
high presence of nodes which are bridges, is overrepresented,
and the imbalanced triad (PPN) is underrepresented, but
neither reaches the level of significance.
To identify only the strongest motifs, we looked at signed
motifs with an added threshold (see Figure 11). To end up
with a similar number of examples for each motif, we selected
a threshold of 0.15 (around 75 percentile of edge weights,
see Table 1) for closed triads and a threshold of 0.20 for
2path motifs. Among the motifs that meet this threshold,
one specific motif may be of relevance for psychological
networks. This is the last motif in Figure 11, which we called
imbalanced triplets II T., based on the work of Toivonen et al.
[61] (hence the T. in the name, for definition see Figure 11).
Toivonen and colleagues investigated a correlation network
of emotion concepts and argued that this motif describes
patterns that cannot be depicted in any dimensional space
without being distorted. This “imbalanced triplet” describes
a pattern which is contraintuitive, although not necessary
unreal, and it is similar in logic to NNP triad. If A, B, and
C represent three psychological dimensions (e.g., emotions
and traits), and positive correlations between A and B, and B
and C exist, depending on the strength of 𝑟𝐴𝐵 and 𝑟𝐵𝐶 , A and
C ought to correlate at least as the half of either of the two
(𝑟𝐴𝐵 or 𝑟𝐵𝐶 ) which is the weaker correlation. Otherwise the
ABC triad does not satisfy the triangle inequality principle;
that is, it cannot be described by dimensional techniques (in
Euclidean space), while a network representation can be used
for detecting their presence.
As mentioned for the NNN and PPN motifs, while we
can expect low occurrence of imbalanced triplets II T. in a
correlation network, in a partial correlation network this is
quite different. An imbalanced triad in a partial correlation
network implies that the partial correlation between A and
B is small given C, which means that A and B approach
16
Complexity
Positive
Negative
triads - PPP
triads - NNN
∀w > 0
∀w < 0
1 pos., 2 neg.
ties –NNP
wij < 0
wjk < 0
wik > 0
42
33
77
250
250
%ile 58.14
200
200
%ile 99.65
200
%ile 26.92
Imbalanced
triads - PPN
wij > 0
wjk > 0
wik < 0
2Path pos.
- P0P
wij > 0
wjk > 0
wik = 0
2Path neg.
- N0N
wij < 0
wjk < 0
wik = 0
2Path mix.
- P0N
wij < 0
wjk > 0
wik = 0
93
265
156
397
250
250
%ile 12.39
200
200
%ile 91.41
200
%ile 34.82
200 %ile 21.13
150
150
150
150
100
100
100
100
100
100
50
50
50
50
50
50
50
0
0
0
0
0
0
150
100
40
60
Positive triads
10
20
30
Negative triads
150
150
60
80
100
Triangles (2 neg)
80
100
120
Triangles (1 neg)
220 240 260 280
2Path pos.
0
140
160 180
2Path neg.
375 400 425
2Path mix.
Figure 10: Signed motifs, name used in this study, the definition, schematic figure, and the figure showing distribution of motif frequencies
in 1000 random networks with the same degree sequence and weight distribution and percentile value of the frequency of empirical network
in that distribution.
Imbalanced
Strong
Strong
Strong
triads II T.
wij ≥ 0.20
Strong
positive
triads
Strong
negative
triads
triads I
wij ≥ 0.15
2paths pos.
wij > 0.20
2paths neg.
wij < −0.20
2paths mix.
wij > 0.20
wjk ≥ 0.20
wjk ≥ 0.15
wjk > 0.20
wjk < −0.20
wjk < −0.20
wik ≤ GCH
∀w > 0.15
∀w < −0.15
wik ≤ −0.15
wik = 0
wik = 0
wik = 0
(wij , wjk )/2
2
6
1
3
5
4
8
500
400
Strong imb.
600
%ile 90.16
500
400
300
200
200
100
100
100
50
0
0
0
0
2
4
Strong pos. triads
%ile 20.68
150
300
200
250
250
%ile 100.0
0
2
4
6
Strong neg. triads
0.0 2.5 5.0 7.5 10.0
Strong imb. triads I
300
250
200
150
100
50
0
%ile 3.35
0
5 10 15
Strong 2path pos.
300
250
200
150
100
50
0
%ile 79.22
%ile 25.47
200
%ile 5.54
200
150
150
100
100
50
50
0
0
5
10
Strong 2path neg.
250
0
0
5
10
Strong 2path mix.
0
10
20
Imb. triads II T.
Figure 11: Weighted and signed motifs, name used in this study, the definition, schematic figure, and the figure showing distribution of
motif frequencies in 1000 random networks with the same degree sequence and weight distribution and percentile value of the frequency of
empirical network in that distribution.
conditional independence given C. This in turn is consistent
with a chain (A->B->C or A<-B<-C) or a fork (A<-B>C). Both of these may yield indirect, but important clues
to the causal structure within the triad. Those triads are
good candidates for more focused analytical approaches that
allow for causal inference (e.g., mediation or path analysis).
Thus, regardless of frequency, the imbalanced triplets II T.
represent a configuration that describes possibly interesting
phenomena which would go unnoticed with dimensional
methods [61].
Results show that, even when we “focus” just on motifs
of relatively strong ties (Figure 11 (third row), all of them
identified in Table 3) again only the NNN triad occurs significantly and more than expected by chance. The cardinality (a
term used in network analysis to address the significance of a
motif) of the motifs in this network is thus not dependent on
the strength of the weights. However, the strong imbalanced
triads, 2paths with positive weights, and imbalanced triplets
II T. have the tendency to be underrepresented. This pattern
is expected in social networks, where imbalanced triads and
“forbidden triads” (2paths) are generally less expressed, and
this network shows similar tendencies.
All motifs defined in Figure 11 are identified and
described in more detail in Table 3.
Complexity
17
Table 3: Weighted and signed motifs identified.
J
Motif
I
K i, j, k; (pr , pr , pr ), [rij , rjk , rik ]
ij
jk
ik
Private body, Public body, Body competence; (.30, .40, .25), [.52, .58, .49]
Conscientiousness, Fair-Mindedness, Self-Disclosure; (.16, .21, .16), [.34, .37, .37]
Tradition, Universalism, Power; (-.34, -.29, -.16), [-.20, -.46, -.11]
Tradition, Universalism, Hedonism; (-.34, -.21, -.36), [-.20, -.12, -.38]
Tradition, Universalism, Achievement; (-.34, -.30, -.28), [-.20, -.34, -.24]
Tradition, Self-direction, Power; (-.37, -.20, -.16), [-.47, -.12, -.11]
Tradition, Hedonism, Achievement; (-.36, -.17, -.28), [-.38, .01, -.24]
Universalism, Hedonism, Achievement; (-.21, -.17, -.30), [-.38, -.01, -.24]
Militaristic int.-, Universalism, Wholesome act. int.; (.22, .16, -.39), [.20, .19, -.37]
Agreeableness, Empathy, Extraversion; (.27, .32, .0), [.45, .39, .16]
Life satisfaction, Emotional stability, Agreeableness; (.25, .26, .0), [.48, .35, .25]
Wholesome act. int., Intellectual int., Openness; (31, .21, .0), [.41, .44, .17]
Self-direction, Tradition, Universalism; (-.37, -.34, .0), [-.47, -.20, .21]
Hedonism, Tradition, Self-direction; (-.36, -.37, .0), [-.38, -.47, .16]
Achievement, Tradition, Self-direction; (-.28, -.37, .0), [-.24, -47, .09]
Self-direction, Power, Universalism; (-.20, -.29, .0), [-.12, -.46, .21]
Hedonism, Universalism, Power; (-.21, -.29, .0), [-.12, -.46, .21]
Militaristic int.-, Universalism, Tradition; (.22, -.34, .0), [.20, -.20, -.10]
Intellectual int., Wholesome act. int., Militaristic int.-; (.31, -.39, .0), [.41, -.37, -.11]
Militaristic int.-, Universalism, Power; (.22, -.29, .0), [.20, -.46, -.10]
Militaristic int.-, Universalism, Achievement; (.22, -.30, .0), [.20, -.34, -.10]
Self-monitoring, Extraversion, Empathy; (.30, .32, .09), [.31, .39, .12]
Depression-, Emot. stability, Agreeableness; (.38, .26, -.06), [.55, .35, .17]
Emot. stability, Agreeableness, Empathy; (.26, .27, -.06), [.35, .45, .15]
Violent-occult int.-, Militaristic int.-, Universalism; (.53, .22, -.11), [.45, .20, -.07]
Life satisfaction, Emot. stability, Depression-; (.25, .38, .11), [.48, .55, .40]
Wholesome act. int., Intellectual int., Openness; (.31, .21, .0), [.41, .44, .17]∗
Agreeableness, Empathy, Extraversion; (.27, .32, .0), [.45, .39, .16]∗
Life satisfaction, Emot. stability, Agreeableness; (.25, .26, .0), [.48, .35, .25]∗
∗Identified also as a 2path pos. motif due to overlap in the motif definition with Imb. triad II T.
(-) after the name of a psychological attribute means that it has been reversed.
Strong PPP triads may indicate the presence of a common
cause, for instance, because the three variables measure the
same underlying psychological construct, which then acts
as a latent variable. Unsurprisingly, the relationships among
the three constructs measured by the Body Consciousness
questionnaire represent one such case. Another such motif is
made of Conscientiousness and two integrity measures, FairMindedness and Self-Disclosure, pointing out that they are
likely capturing similar psychological dimension. A second
possibility that may underlay PPP triads is a positive feedback
between the variables, as found in the mutualism model for
intelligence.
All six NNN triads involve Schwartz’s values, with Tradition being present in five of them. This configuration
cannot emerge from a common cause and may suggest a
negative feedback loop between the attributes. Still, such an
interpretation is formed on conclusions about intraindividual
differences that are based on interindividual data, which may
not necessarily hold. A second possible reason for observing
NNN triads is that the variables have been conditioned on a
common effect to which each of them positively contributes.
The logic here is the following. Suppose that three variables
A, B, and C increase the probability of common effect D. If
we condition on D, we only consider the values of A, B, and
C for a given value of D. Suppose we observe that the effect is
present (or D has a high value), but A is not present (or has a
low value). Then that information makes it more likely that B
or C are present (or have a high value). Thus, conditioning on
D, we expect A, B, and C to be negatively related so that they
form an NNN triangle in the partial correlation network.
18
Complexity
0.15
0.925
0.920
Coherence
Intensity
0.14
0.13
0.12
0.915
0.910
0.905
0.900
Motif
2Path mix. (P0N)
2Path neg. (N0N)
2Path pos. (P0P)
All 2Path
3 neg. (NNN)
2 neg. (NNP)
1 neg. (PPN)
All pos. (PPP)
All triangles
2Path mix. (P0N)
2Path neg. (N0N)
2Path pos. (P0P)
All 2Path
3 neg. (NNN)
2 neg. (NNP)
1 neg. (PPN)
All pos. (PPP)
All triangles
0.11
Motif
Figure 12: Means of intensity and coherence of all triads and signed motifs in the network.
One NNP triad consists of a negative association between
Low Militaristic values and Interests in wholesome activities,
while both variables are positively correlated with Universalism. This triad identifies a puzzling relationship that
might suggest multidimensionality of the Universalism value.
Positive 2paths show that Empathy, Emotional Stability, and
Intellectual Interests may play the role of mediators. Negative
and mixed 2paths similarly show the variable in central
position (position “J” in Table 3) as bridging the remaining
two attributes in the subgraph. Finally, eight configurations
present the strongest imbalanced triplets II T. in the network,
which are not possible to describe in the metric space. Three
of them also fall under 2paths, due to the overlap in the motif
definition. The variable in position “J” (see Table 3, first row)
in this motif is likely to be a broad concept with multiple
meanings.
4.3.2. Analysis of Motif Intensity. In previous research, the
Intensity measure has been applied for triadic motifs consisting of positive weights only. Therefore, we modified the
approach described by Onnela et al. [53] by calculating
I and Q separately for triads with a different configuration of positive and negative ties to allow comparing the
Intensities across different motifs. The average Intensity
and Coherence for all investigated motifs are shown in
Figure 12.
Visual inspection of Figure 12 reveals that the differences in Intensity and Coherence between the motifs are
very small (y axes show range of 0.05 for I, and 0.025
for Q). When looking at the structural motifs concerned
only about presence and absence of ties, and not their
weights, all triads have a higher Intensity than 2paths, but
the difference is very small. In psychological networks, it
would be expected that triangles have a higher Intensity than
2paths, as triangles represent mutual connections between
all three nodes, making it more likely that the nodes will
reinforce each other. Because of this reinforcement, it would
be expected that the weights are of higher absolute value than
in 2paths, where one edge is missing, making such effect less
plausible.
The most intensive motif, that is, the motif with the
highest average geometric mean of weights, is a triad made
of three negative ties NNN, followed by positive triad (PPP)
and 2path with two negative ties (N0N). The finding that a
NNN motif is the most intensive is somewhat surprising for
networks of this kind, but, before attempting interpretation,
we will proceed first with analysis of Coherence, followed by
significance testing.
Internal Coherence of 2paths (open triads) is somewhat
higher than for closed triads (Figure 12, right panel), which is
to be expected as 2paths consist of one weight less than triads.
PPN seems to have relatively higher, while PPP relatively
lower Q.
Having a high (low) average Intensity of a motif does
not imply that the motif is highly (lowly) expressed in the
network. Therefore, the next step is to check how significant
the Intensities are. The same applies to the Q, where a high
Q of a motif does not imply it is significantly more coherent.
To answer those questions, the Intensities and Coherences of
each motif are compared with the mean of I and Q of each
motif in an ensemble of 1000 random networks. The results
of the analysis are shown in Figure 13.
The only motif whose Intensity (percentile value > 97.5)
is significantly high is a triad with three negative ties (NNN),
which is in line with the results on the frequency and the
descriptive analysis presented in Figure 12. Although the
average Intensity is not high in absolute terms (slightly above
0.14), the frequency and Intensity analysis both suggest that
the NNN motif is an important characteristic of the network.
In Table 3, we saw that all NNNs involve only Schwartz’s
values. NNN motifs show a tendency to be “nested” around
few nodes; only the nodes that represent Schwartz’s values are
“responsible” for the high frequency (and Intensity) of that
Complexity
19
50
0
0
0
2Path pos. I
100
50
50
0
0
0
0.95
0.90
0.88
0.925
100
50
0.120
100
2 neg. Q
3 neg. Q
250
200
150
150
150
150
150
100
100
100
100
100
50
50
50
50
50
0
0
0
0
0
250
2Path mix. I
2Path pos. Q
2Path neg. Q
200
0.94
%ile 22.38
0.93
%ile 15.58
0.94
200
0.92
0.94
0.93
%ile 12.99
200
0.92
0.125
0.120
%ile 1.8
0.115
0.13
0.92
2Path Q
200 %ile 75.82
2Path neg. I
%ile 4.0
200
150
2Path I
0.90
0.88
0.900
%ile 82.02
150
250
0.12
0.13
0.12
0.11
%ile 4.0
200
0.92
3 neg. I
0.11
300
250
200
150
100
50
0
0.115
0.13
2 neg. I
250
%ile 92.41
0.935
100
50
0.14
100
50
0.12
100
0.10
150
0.12
150
0.11
150
150
1 neg. Q
250
0.930
200
%ile 0.3
%ile 97.5
All pos. Q
0.92
200
0.850
All triangles Q
0.925
0
0.875
0
0.91
50
0
250
%ile 99.9
200
%ile 11.89
200
50
1 neg. I
250
250
100
50
0.85
All pos. I
150
0.90
0.10
All triangles I
200
100
0.90
0
250
%ile 36.76
100
0.89
0
200
150
0.13
50
250
%ile 98.8
150
0.12
50
0.120
100
0.115
150
100
0.110
150
200
250
%ile 47.8
200
150
100
50
0
0.11
200 %ile 84.82
0.14
250
200 %ile 92.91
0.12
250
2Path mix. Q
Figure 13: Significance of motifs’ intensity and coherence: distributions of intensity (first three columns, colored blue) and coherence (last
three columns, colored light brown) of all closed and open triads, 2paths, and signed motifs in 1000 random networks with same structure
and weight distribution as empirical network, with its percentile values.
motif on a network level. Furthermore, from Figure 1 (and
the centrality analyses) we observed that not all Schwartz’s
values are central. From that we may generate a hypothesis
that the most prominent characteristic of the psychological
system of 26 attributes is described by a negative feedback
between values, although the cluster with such pattern is not
central in the system. A second possibility is that some of the
values are involved in a common effect with respect to one
of them, which might for instance arise when, say, Tradition
is caused by all other variables. Due to the conditioning on
the common effect, the NNN pattern may arise for the causal
variables in the partial correlation network. A final possibility
would be that the high occurrence of NNN may be the result
of estimating network on a sample which is self-selected (i.e.,
implicitly conditioned) on a variable that is a common effect
of Schwartz’s values.
Two motifs with significantly small Intensity (percentile
value < 2.5) are all 2paths motif (structural, disregarding
the signs of ties), and 2paths with one negative and one
positive tie (with mixed ties, P0N). The later finding is an
example of the importance of comparison with the reference
system. When we analyzed only the average Intensity, we have
found that 2paths have a higher Intensity than other motifs.
Comparing this to what may be expected given the network
structure and weight distribution, we can see that, in fact, the
Intensity of 2paths, although somewhat higher in absolute
value than Intensity of other motifs, is significantly smaller
than it would be expected by the null model. The “intuitive”
expectation about smaller Intensity of 2paths due to the lack
of third link is supported.
Closed triads (all triangles) display significantly high
internal Coherence. From the tie’s perspective, this may
suggest that weights of similar strengths show the tendency
to form triads, or, from a node perspective, that psychological
attributes that form a triad tend to be connected with ties
of similar strengths (in absolute values). Imbalanced triad
(PPN, called “1 neg.” in Figure 13) is also significantly more
coherent, meaning that the weights within this triangle tend
to be equally distributed (they do not show big variations).
Interestingly, so-called imbalanced triads in this network
consist of “balanced” edge weights. The overall pattern of
results show that a significant I does not imply significance in
Q, which highlights that they measure two different aspects
of this system.
5. General Discussion and Conclusions
This paper has demonstrated how the use of three metrics
taken from network science can enrich our understanding
about psychological networks. Given the effort invested in
estimating the network structure, it is a missed opportunity
not to use the information it entails more fully to gain deeper
understanding of estimated network. This “omission” may
be understood and partly explained by researchers in the
field being preoccupied primarily with network estimation
methods [11, 47, 62] and replicability issues [49, 63, 64]
that arise from the fact that network structures between
variables are considerably more difficult to determine, relative
to, for example, internet links or electricity nets; after all,
conditional association between variables is not observable,
20
but must be estimated from data. Appropriately dealing with
sampling error in estimating network structures, as well as
assessing their robustness, has therefore been the priority in
psychological network analysis.
The concise overview of the three methods in terms of
hypothesis and research questions and procedure is given in
Table 4.
We demonstrated on illustrative dataset how each of the
methods proposed here adds new information about the network structure. First, the MST can help us in shedding light
on the topological arrangement of psychological attributes in
the network. Specifically, in the current example, the MST
suggests that Empathy is the most similar to all other traits
and plays the role of a “network connector;” it is the most
central trait when centrality is based on the network filtered
down to its most essential ties. In the network which also
includes Big Five traits, it was somewhat surprising to see that
Empathy has such important standing. This could be due to
the questionnaire used for this trait (the Empathy Quotient)
which captures affective and cognitive aspects (see Table 1).
The authors [39] of the questionnaire state that the cognitive
component of Empathy is closely related with an individual’s
“Theory of Mind,” a cognitive process that allows people to
understand others and oneself. It might, thus, be plausible
that cognitive processes related to Theory of Mind serve as
a central hub in the system. In addition, it is tempting to see
the analogy and state that the trait which is seen by some to
hold society together may also hold this network of different
psychological attributes together. This finding is worth of
further attention due to an implicit and misguided notion
that Big Five traits are the best representative of psychological
differences between individuals. If true, in network terms
that would imply that they are expected to be in the top five
most central nodes, which is the case only for some of them.
In fact, Openness is among peripheral nodes. Nonetheless,
further theoretical consideration and research is needed. The
MST provided an additional insight into possible clusters of
attributes and showed that clusters, that is, different branches
of the tree, for the most part do not align with different kinds
of psychological variables. For example, Big five traits and
Schwartz’s values are placed on different branches, suggesting
that the grouping of variables is based on specific content
rather than “nature” of a psychological variable (e.g., whether
it is a trait, value, or interest). Furthermore, we used the fact
that MST preserves the information of edge signs to employ
it for robustness test of network estimation.
Second, by including information about the participation
coefficient based on predefined communities, which also
included “communities of one,” we highlighted the specific
role of some nodes based on their equal importance to
the structure of different parts of the network. We found
that Intelligence, although weakly connected to other traits,
and by all centrality measures quite peripheral, does seem
to have an interesting property of being relatively equally
associated with all different kinds of nodes in the network.
Based on this finding we can hypothesize that cognitive
ability relates to personality: not in terms of substantial effect
sizes but because it relates at a constant strength to most
“parts” of psychological system. In other words, the question
Complexity
about relation between cognitive ability and psychological
individual differences could be better answered if instead of
looking at the “size” of that influence (operationalized with
some statistical measure), researchers refocus their attention
on the “broadness” of that influence. This agrees with the
suggestion of Salovey and Mayer [25] that, instead of looking
at pairwise correlations, a more complex analysis that looks
at many connections at once should be preferred. Likewise,
network ties of Intelligence seem to imply a different relation
with Big Five model than reported in the recent review [65].
When 24 other relevant individual differences (26 minus
2 variables whose connection is under consideration) are
controlled for, the strongest tie is not with Openness, but with
Agreeableness and Extraversion (both negative and around
0.10).
We used PC together with the Participation Ratio to arrive
at more sensible centrality measure, which showed that different centrality indices converge to Extraversion, Emotional
Stability, and Empathy as the three most central nodes in this
network. Centrality of Extraversion and Emotional Stability
would be expected since they are one of the traits that have
been recognized as important psychological dimensions and
systematically studied from early on in psychological science.
Empathy taking the “third place” is somewhat surprising, but,
as discussed before, could be related with this trait capturing
cognitive processes that are essential and fundamental in
many social interactions [66].
Finally, we used motif analysis to research possibly interesting three-node configurations and investigate whether
this psychological network “behaves” as a social network
regarding its balance of negative and positive ties within a
triad, and the results showed this is not the case. We learned
that some configurations that are challenging to interpret
exist in the network at a higher frequency than would be
expected in the reference system; most notably, this was the
case for NNN triads. Identification of strong motifs revealed
that these triads originate mostly from one group of nodes,
Schwartz’s values, possibly revealing negative feedback or
(implicit) conditioning on a common effect of some or all of
the variables. NNN triads are also significantly stronger than
expected, but otherwise intensity and coherence do not seem
to be related with frequency of motifs.
Methodological Considerations Related with the Reverse Coding of Variables. An important issue related with network
modeling of relationships between continuous variables
which probably did not receive enough attention so far is
the effect of reverse coding of variables on the results of
network methods (we are grateful to a reviewer for pointing
out this issue). It becomes an even more pressing issue
when nodes are aggregations of more complex concepts,
not easily described as positive or negative (e.g., some values), or when variables present dimensions which are interpretable on both ends (e.g., emotional stability–neuroticism,
extraversion–introversion), and often coded arbitrary. This is
the case for many continuous variables in psychology, and
probably for all variables in our dataset to some extent. For
example, Emotional Stability (ES) is often coded negatively
as Neuroticism (N), begging the question what would happen
Complexity
Table 4: An overview of the three methods.
Recommended for network
Important procedure steps
Output
Methodological considerations
Other analytical possibilities
Effect of reverse-coding variables
Hypothesis/Research question
Recommended for network
Important procedure steps
Output
Methodological considerations
Other analytical possibilities
Effect of reverse-coding variables
Hypothesis/Research question
MINIMUM SPANNING TREE
Dense networks and/or networks with many small edges
(1) Selecting a distance measure:
(i) Gower’s distance
(ii) Distance inversely proportional to the shared variance
(2) Centrality analysis (by inspection or/and with standard centrality measures computed)
MST – the filtered network
Distance measures (i) and (ii) will produce different MSTs if a network has negative ties∗
Looking at MST branches as communities
Using MST to test the robustness of the network estimation of the most essential edges MST can include distance metric as weight for
further analysis
(i) affected∗ (ii) not affected
RQ: Which node is the most central?
Which (overlapping) communities exist in the network?
PARTICIPATION COEFFICIENT as a corrective
(1) Pre-existing differences in the kinds of nodes
(2) Networks with communities
If (1) is true:
(a) Defining the node groups
(b) Calculating PC
(c) Choosing the centrality measure to be corrected with PC (optional)
If (2) is true:
(a) Data-driven detection of communities
(b) Calculating PC
(c) Choosing the centrality measure to be corrected with PC
(d) Comparing the rank order of chosen centrality measure before and after the correction
PC values for each node (in (1)(b) and (2)(b); The corrected centrality measure (for (1)(c) and (2)(c))
Communities should not overlap ((1) and (2))
(1) a Group affiliations may be ambiguous
(2) a Decision about appropriate community detection algorithm
PC version that treats positive and negative edges separately
Not affected if signs are not taken in the account when calculating PC
𝐻0 = A centrality measure is not affected by (pre-existing or data-driven) communities
21
22
Table 4: Continued.
Recommended for network
Important procedure steps
Output
Methodological considerations
Other analytical possibilities
Effect of reverse-coding variables
Hypothesis/Research question
MOTIF ANALYSIS
(1) Not for networks with small number of nodes and/or very low density
(2) Has negative and positive ties
(3) If weighted, additional steps in the procedure
(a) Defining motifs of interest and the null model
(b) Motif identification and frequency
(c) Significance testing of motif frequency
If (3) is true:
(d) Motif intensity (and coherence)
(e) Significance testing of motif intensity (and coherence)
Identified motifs; Motif Frequency; P-values for frequencies; Motif intensity (and coherence); P-values for the intensities (and coherence)
The definition of the null (reference) model
Other motif structures, e.g. that include more nodes
Identified motifs and motif frequency will be different∗ , but onclusions about significance (of frequency, intensity, and coherence) will tend
to converge
Many research questions and hypotheses possible. In this study:
Signed edges will tend to cluster in the line with what is observed in social networks and correlational networks (balance theory, forbidden
triads, imbalanced triplets):𝐻1 = P0P, NNN, and PPN will be less frequent than expected by chance.
RQ1 = Do same pattern of results holds true when only relatively stronger motifs are considered?
RQ2= Are Intensity and Coherence measure following the same pattern of results as the frequency of motifs?
∗
This should be the case, but there is a possibility that distances related to negative edges are present in a network in such a way that will not affect the MST construction, e.g. a weak negative tie that exists among
two peripheral nodes that have ties to other more central node.
Complexity
Complexity
with the results of analyses if we used N instead of ES? To find
out we repeated most of the analyses reported in this paper
with the network that had N instead of ES, and several other
networks with some of the variables recoded. The results are
presented in detail in SM (Section 12), while here we will
highlight just the most important conclusions. The estimated
network will have the same structure and absolute values of
weights, but all the edges of reversed node will change their
sign. Weight distribution of network is affected too, due to the
changes in signs of some of the weights. The most affected
are the results of MST, but only if the preferred distance
measure is used. Otherwise, with the measure inversely
proportional to shared variance, MST results are unaffected.
This situation brings up the dilemma of which distance metric
to use: the more rigorous one that is affected by variable
coding, or the one which leads to a possibly substantial loss
of information but is immune to reverse coding? We do
not provide an answer, because, as usual, it will depend on
the specific network, variables included, and the research
question. Nevertheless, researchers need to be aware of this
issue. In contrast with MST, PC that takes only absolute value
of weights is not affected by reverse coding. Motif analysis will
produce different motif frequencies, intensity, and coherence
values, but the results of significance testing will not be
affected to a greater extent and will tend to converge for the
same network with differently coding some of the variables.
A logical conclusion following from previous section is
that the three methods discussed in this paper require an
effort to be applied to a psychological network, as some
additional decisions need to be reached such that they are in
accordance with research questions/goals (also explained in
Sections 4.1, 4.2, and 4.3). Each decision has its repercussions.
In case of MST, one needs to consider the presence of
negative ties and what is achieved by deciding to look at
two negatively associated nodes as more dissimilar than two
nodes that are not connected at all. For PC, the nature
of nodes included in the network needs to be carefully
looked at, while for motif analysis some notion about which
specific configurations may reveal interesting patterns in
the network should be formed. The common ground of all
three methods is that they look at direct, local ties, but
in the contrast to the degree centrality they provide more
fine-grained information. This presents a potential for a
deeper understanding of any network but is also a very
convenient feature for networks that do not have well-defined
boundaries. By boundaries, we refer to two issues. The first
issue is the possibility that some node(s) which are part of
the system are not included in the network analysis. This
is an issue for our network where selection of variables
was atheoretical, since a “global” theory that describes all
psychological attributes does not exist. The selection was
further constrained by data availability. For example, we can
think of some potentially important attributes that are not
in the network, for example, self-efficacy, need for cognition,
and narcissism. While acknowledging this, the limitation
had its advantage in indirectly preselecting some of the currently most studied/used (and therefore, it could be argued,
important) concepts. The second issue is related to the first
one and refers to the nature of the investigated network.
23
Some networks are more easily influenced by “externalities;”
for example, for a psychological network this may include
some important life events that can bring about the change
in the network by directly or indirectly influencing one or
more nodes. Hence, global properties of such network, and
measures relying on all ties in it, may be less useful. The
fact that whole system is not represented and that it is an
“open” system, as is the case in probably many psychological
networks studied so far, was the motivation for introducing
these three network methods that rely more heavily on local
than global network structure.
To conclude, the added value of more information provided by more complex network tools comes at the price of
less straightforward procedures, and making more decisions
(hopefully informed by theory and previous research). However, we believe that those elements are just more salient when
using these three methods, than when using typical centrality
analysis based on different centrality indices, where many
assumptions are implicit (e.g., that all nodes are equally likely
to be connected to any other node). Therefore, we look at
this requirement for higher deliberation as a good practice in
general when applying any network analysis to psychometric
data, as it challenges researchers to think more about nature
of nodes, ties, and smaller network configurations in the
network. Nevertheless, that is not an easy task. Understanding these “new” methods may be at first somewhat
less straightforward and difficult for researchers not heavily
involved in network analysis. This is especially true for motif
analysis, which is by far the most complex of the three. Given
that network approach is relatively new in psychology, it will
take some time for network ideas and methods to “sink in.”
Unfortunately, it also lacks strong theories. Be that as it may,
better understanding of its analytical tools and exploratory
(and that sometimes means undertheorized) potential will
greatly facilitate the development of such theories. William
James’s argument that “a degree of vagueness can be beneficial
to science when attempting new research directions” [67, p.2]
nicely captures the point we are trying to make. This holds
true not only for network theories, but also for any kind of
theories which aim to integrate many small (“local”) theories
in psychology.
The methodology presented offers interesting possibilities for applications to other areas. For example, it would
be informative to see how equally distributed ties are of
depression symptoms among different groups of symptoms
(e.g., thoughts, physical symptoms, behaviors, and feelings),
and which symptoms are most central when that information
is taken into account. We are not suggesting that all methods
should be used in every analysis. The most appropriate
methods and its specific procedure should be established
based on careful consideration of the data at hand, research
questions and theory behind it, and knowledge of existing
network science tools. Our goal was to expand the latter.
Network approach is often compared to other multivariate methods more commonly used in the field of psychology, for example, structural equation modelling (SEM),
confirmatory factor analysis (CFA), mediation analysis (MA),
hierarchical clustering (HC), and multidimensional scaling
(MDS). Although detailed comparison is out of scope of this
24
Complexity
paper, we will proceed with a general overview with highlight
on three most notable differences between network approach
and most of multivariate methods used in psychology that
are more closely related with three specific methods we
introduced in this paper. Firstly, the network approach is
less directly guided by researcher’s assumptions about the
connections between variables than most other methods
(e.g., CFA), that is, except for the decision about the variables
that will be included in the network. In reality, the decision
about which variables will be included in the network is
constrained with data availability. In this regard, using PC
can help in indirectly controlling for some aspects of that
constraint, acting as a corrective measure for possible bias
in the selection of nodes that have been included in the
network.
Secondly, in comparison with SEM, and MA, network
analysis usually deals with a greater number of variables at
once, implying that SEM and MA may be more appropriate
for smaller set of variables, especially if clear theoretical
expectations exist about relationships between the constructs.
Finally, other approaches are not trying to look at the set
of investigated variables as a system and reveal the properties
of that system; they rarely go beyond the microlevel of
examining specific connections. In that sense, MST and motif
analysis are valuable tools within network approach. MST can
be used, among other reasons (mentioned in this paper), to
filter the most important connections in the system and to
provide answer about the most central variables/nodes on a
more general level than specific centrality measures. One part
of the output of motif analysis, the identification of motifs,
can be viewed as a counterpart to MA (or SEM if configurations tested with motif analysis include more than three
variables/nodes) among network methods. However, other
outputs of motif analysis, significance of motif frequency,
intensity/coherence analysis, and its corresponding significance testing aim at insights that use aggregated information
about microlevel to inform about the properties of system as
a whole.
In conclusion, at this rather early stage of its application
in the field of psychology, network analysis is mostly an
exploratory approach, but that is likely to change with the
introduction of more sophisticated methods that may provide
additional insights. In turn, this will enhance the development of specific network theories that can be explicitly tested,
resulting in unique contributions to our knowledge about
psychological phenomena.
If we view network approach as a different way of thinking
about psychological constructs, then exploring networks
more “deeply” may lead us to interesting and important
findings that would otherwise be missed. Those findings can
lead to new questions, generate new specific hypotheses, and
help form truly progressive network theories of psychological
phenomena.
missing data, and how exactly to deal with this problem in
network modelling is still an open question [68]. Another
open issue in psychological networks is measurement error,
which is not accounted for. On an interpretative level, since
nodes in network are entities, it is not clear whether their
associations can be interpreted as conceptual overlap. To the
list of open questions that fall beyond the scope of this paper,
we may add the common method variance, which could be
responsible for observing some of the edges. However, given
that we used partial correlations in the network construction,
we believe that most of common method variance (except
those unique to a pair of variables) is in that way excluded.
Furthermore, one of the sources of common method variance, social desirability, is explicitly included in our network
because Self-Disclosure is used as indicator of proclivity
to give socially desirable answers (the higher the trait, the
smaller the proclivity). Finally, although we had a relatively
big sample (pairwise), we do not know how selection bias may
influence the results. The trade-offs of “big data” in general is
that, on the one hand, it provides more diverse and bigger
samples, but, on the other hand, self-selection bias can affect
results in many different and unexpected ways. This can play
out at multiple levels. For instance, FB users may be unrepresentative regarding some of the traits or due to demographics
[69], or FB users who used the myPersonality application
could be, on average, psychologically different. For example,
it could be argued that the sample consists of people who
are more interested in psychological aspects of reality and
in understanding themselves and others when compared to
the general population. In line with this possibility, general
self-selection may have influenced our findings about the
important role of Empathy in the network. Lastly, individuals
chose freely to fulfil certain questionnaire(s). Insomuch as
the choice was not random, there is always a possibility
that individual psychological attributes influenced that choice
(e.g., more depressed individuals could be less likely to fill in
an intelligence test).
In the context of those limitations, the findings we arrived
at while demonstrating three methods are presented as
tentative and their value is in generating new and interesting
hypotheses. Furthermore, in our tentative interpretations,
due to our network made of well-studied and diverse psychological attributes and due to the scope of this article, we
just scratched the surface of many more interesting “small”
findings (e.g., each identified triad in Table 3 would be a
good starting point for discussion and for generating further
hypotheses). That being said, harvesting an already existing
dataset, which contains information about many psychological attributes of big number of people, repurposing it to
demonstrate “new” methods, and, while doing so, addressing
some new and some old questions (network of psychological
attributes and cognition-personality relationship) present
potentially useful exploratory research.
Limitations of This Study. Our goal was to demonstrate three
methods by applying them to an illustrative dataset. The
dataset, however, has some limitations that are important
to note. Although we had an atypically large sample (for
psychological research), it featured considerable amount of
Future Research. Regarding specific questions related to our
dataset, future research would benefit from more theoretically guided inclusion of psychological attributes in the
network, including different types of intelligence measures
that capture more than g-factor. More objective (behavioral)
Complexity
measures of attributes would enhance the validity of findings. Longitudinal data (within-subjects networks) and data
on specific populations (e.g., regarding mental health, age,
gender, and culture) would in addition enable answering
questions about network dynamics and network structure.
Future research can use simulation studies to determine how
exactly each of the methods is affected by differences in
network density, size, number of groups, structure, weight
distribution, etc. This would be especially interesting for
MST, as we explicitly mentioned that it could be used to
check the robustness of network estimations. We used PC
on what we called “predefined communities,” but when there
are no differences between nature of psychological attributes
PC might be used in typical way as well, which starts
from empirically determined communities (such example
is given in SM, section 13). Likewise, the PC measure can
be extended in such a way that one could calculate it for
positive and negative links separately. In the motif analysis,
we looked only at triads; future work can include higherorder configurations, motifs that involve more than three
nodes (e.g., bow tie).
Finally, we selected three network metrics for this article,
but there are other measures and techniques that could be
fruitfully used in the analysis of psychological networks (e.g.,
coefficient of intramodule activity, missing link prediction).
The message is that network science methodology develops
rapidly, and psychologists using network analysis would do
well to embrace the possibilities these methods offer in both,
analysis and stating new research questions, hypotheses, and
even theories.
Data Availability
The data used to support the findings of this study were
supplied by David Stillwell and Michal Kosinski under
license and so cannot be made freely available. Requests
for access to these data should be made to David Stillwell,
contact@mypersonality.org.
Conflicts of Interest
The authors declare that there are no conflicts of interest
regarding the publication of this article.
Authors’ Contributions
Srebrenka Letina conceived the idea for the study, asked
for the data access, did the data processing, analyses and
visualizations, and wrote the paper. Tessa F. Blanken, Denny
Borsboom, and Marie K. Deserno edited the text and its
structure and provided feedback on the manuscript.
Acknowledgments
We thank David Stillwell and Michal Kosinski for allowing
the access to the myPersonality database (myPersonality.org).
The work on this paper was partially sponsored by Central
European University Foundation, Budapest (CEUBPF). The
25
theses explained herein are representing the author’s own
ideas but do not necessarily reflect the opinion of CEUBPF.
We acknowledge COSTNET (Cost Action CA15109) in
funding the short scientific mission which resulted in this
work. This project has received funding from the European Research Council (ERC) under the European Union’s
Horizon 2020 Research and Innovation Programme (Grant
Agreement no. 648693). Denny Borsboom is supported by
ERC Consolidator Grant no. 647209. We thank Donald
Williams for the help in the estimation of nonregularized
partial correlation network and Tamer Khraisha for advice on
coding and visualizations.
Supplementary Materials
More details about procedures and results of the analyses are
organized in 13 sections of the Supplementary Materials: data
processing, sample description, description of missing data,
descriptive statistics of 26 psychological attributes, the choice
of the estimation method, robustness analyses, network of 26
psychological attributes, analysis of network ties, centrality
analysis, correlations between four centrality measures in
full network and in MST, the MST with different distance
measures, the effect of reverse coding on the analyses, and
participation coefficient based on empirical (data-driven)
communities. (Supplementary Materials)
References
[1] H. L. J. Van Der Maas, C. V. Dolan, R. P. P. P. Grasman, J.
M. Wicherts, H. M. Huizenga, and M. E. J. Raijmakers, “A
dynamical model of general intelligence: The positive manifold
of intelligence by mutualism,” Psychological Review, vol. 113, no.
4, pp. 842–861, 2006.
[2] A. O. J. Cramer, C. D. Van Borkulo, E. J. Giltay et al., “Major
depression as a complex dynamic system,” PLoS ONE, vol. 11,
no. 12, article e0167490, 2016.
[3] G. Costantini, J. Richetin, E. Preti, E. Casini, S. Epskamp, and
M. Perugini, “Stability and variability of personality networks.
A tutorial on recent developments in network psychometrics,”
Personality and Individual Differences, vol. 136, pp. 68–78, 2017.
[4] J. Dalege, D. Borsboom, F. van Harreveld, and H. L. J. van der
Maas, “Network analysis on attitudes: a brief tutorial,” Social
Psychological and Personality Science, vol. 8, no. 5, pp. 528–537,
2017.
[5] V. D. Schmittmann, A. O. J. Cramer, L. J. Waldorp, S. Epskamp,
R. A. Kievit, and D. Borsboom, “Deconstructing the construct:
A network perspective on psychological phenomena,” New
Ideas in Psychology, vol. 31, no. 1, pp. 43–53, 2013.
[6] A. O. J. Cramer, S. van der Sluis, A. Noordhof et al., “Dimensions
of normal personality as networks in search of equilibrium: you
can’t like parties if you don’t like people: dimensions of normal
personality as networks,” European Journal of Personality, vol.
26, no. 4, pp. 414–431, 2012.
[7] D. Borsboom and A. O. J. Cramer, “Network analysis: An
integrative approach to the structure of psychopathology,”
Annual Review of Clinical Psychology, vol. 9, pp. 91–121, 2013.
[8] J. Kossakowski and A. O. J. Cramer, “Complex dynamical systems in psychology,” in Network Science in Cognitive Psychology,
26
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
Complexity
M. Vitevitch, Ed., chapter: Complex Dynamical Systems in
Psychology, Rutledge Taylor & Francis Group, 2017.
S. Epskamp, L. J. Waldorp, R. Mõttus, and D. Borsboom, “The
gaussian graphical model in cross-sectional and time-series
data,” Multivariate Behavioral Research, vol. 53, no. 4, pp. 453–
480, 2018.
C. D. van Borkulo, D. Borsboom, S. Epskamp et al., “A new
method for constructing networks from binary data,” Scientific
Reports, vol. 4, no. 5918, 2015.
S. Epskamp and E. I. Fried, “A tutorial on regularized partial
correlation networks,” Psychological Methods, vol. 23, no. 4, pp.
617–634, 2018.
C. D. van Borkulo, L. Boschloo, J. Kossakowski et al., “Comparing network structures on three aspects: A permutation test,” In
press.
R. N. Mantegna, “Hierarchical structure in financial markets,”
The European Physical Journal B, vol. 11, no. 1, pp. 193–197, 1999.
T. F. Blanken, M. K. Deserno, J. Dalege et al., “The role of
stabilizing and communicating symptoms given overlapping
communities in psychopathology networks,” Scientific Reports,
vol. 8, no. 1, 2018.
P. J. Jones, A. Heeren, and R. J. McNally, “Commentary: A
network theory of mental disorders,” Frontiers in Psychology,
vol. 8, Article ID 1305, 2017.
R. Guimerà and L. A. N. Amaral, “Functional cartography of
complex metabolic networks,” Nature, vol. 433, no. 7028, pp.
895–900, 2005.
P. W. Holland and S. Leinhardt, “Local structure in social
networks,” Sociological Methodology, vol. 7, pp. 1–45, 1976.
R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii, and
U. Alon, “Network motifs: simple building blocks of complex
networks,” Science, vol. 298, no. 5594, pp. 824–827, 2002.
P. L. Ackerman and E. D. Heggestad, “Intelligence, personality,
and interests: Evidence for overlapping traits,” Psychological
Bulletin, vol. 121, no. 2, pp. 219–245, 1997.
A. R. Jensen, The g factor: The Science of Mental Ability, Praeger
Publishers/Greenwood Publishing Group, Westport, CT, USA,
1998.
A. S. Griffin, L. M. Guillette, and S. D. Healy, “Cognition and
personality: an analysis of an emerging field,” Trends in Ecology
& Evolution, vol. 30, no. 4, pp. 207–214, 2015.
H. J. Eysenck, “Personality and intelligence: psychometric
and experimental approaches,” in Personality and Intelligence,
R. J. Sternberg and P. Ruzgis, Eds., pp. 221–247, Cambridge
University Press, Cambridge, Uk, 1994.
A. J. Berg, G. M. Ingersoll, and R. L. Terry, “Canonical analysis
of the MMPI and WAIS in a psychiatric sample.,” Psychological
Reports, vol. 56, no. 1, pp. 115–122, 1985.
R. Howard and M. McKillen, “Extraversion and performance in
the perceptual maze test,” Personality and Individual Differences,
vol. 11, no. 4, pp. 391–396, 1990.
P. Salovey and J. Mayer, “Some final thoughts about personality
and intelligence,” in Personality and Intelligence, R. J. Sternberg
and P. Ruzgis, Eds., pp. 221–247, Cambridge University Press,
Cambridge, UK, 1994.
J. F. Kihlstrom and N. Cantor, “Social Intelligence,” in Handbook
of intelligence, R. J. Sterrnberg and S. B. Kaufman, Eds., pp. 564–
581, Cambridge University Press, Cambridge, UK, 2011.
D. H. Ford, Humans as Self-Constructing Living Systems:
A Developmental Perspective on Behavior and Personality,
Lawrence Erlbaum Associates, Inc., Hillsdale, NJ, USA, 1987.
[28] S. D. Smirnov, “Intelligence and personality in the psychological
theory of activity,” in Personality and Intelligence, R. J. Sternberg
and P. Ruzgis, Eds., pp. 221–247, Cambridge University Press,
Cambridge, UK, 1994.
[29] M. Kosinski, S. C. Matz, S. D. Gosling, V. Popov, and D.
Stillwell, “Facebook as a research tool for the social sciences:
Opportunities, challenges, ethical considerations, and practical
guidelines,” American Psychologist, vol. 70, no. 6, pp. 543–556,
2015.
[30] D. J. Stillwell and M. Kosinski, “MyPersonality project: example of successful utilization of online social networks for
large-scale social research,” in Proceedings of the 1st ACM
Workshop on Mobile Systems for Computational Social Science
(MobiSys), 2012, https://www.gsb.stanford.edu/sites/gsb/files/
conf-presentations/stillwell and kosinski 2012.pdf.
[31] S. H. Schwartz, “An overview of the schwartz theory of basic
values,” Online Readings in Psychology and Culture, vol. 2, no. 2,
2012.
[32] L. R. Goldberg, J. A. Johnson, H. W. Eber et al., “The international personality item pool and the future of public-domain
personality measures,” Journal of Research in Personality, vol. 40,
no. 1, pp. 84–96, 2006.
[33] V. Egan and V. Campbell, “Sensational interests, sustaining fantasies and personality predict physical aggression,” Personality
and Individual Differences, vol. 47, no. 5, pp. 464–469, 2009.
[34] L. C. Miller, R. Murphy, and A. H. Buss, “Consciousness of
body: private and public,” Journal of Personality and Social
Psychology, vol. 41, no. 2, pp. 397–406, 1981.
[35] J. Rust, “The validation of the orpheus minor scales in a working
population,” Social Behavior and Personality, vol. 26, no. 4, pp.
399–406, 1998.
[36] D. Watson and J. W. Pennebaker, “Health complaints, stress,
and distress: exploring the central role of negative affectivity,”
Psychological Review, vol. 96, no. 2, pp. 234–254, 1989.
[37] M. Snyder, “Self-monitoring of expressive behavior,” Journal of
Personality and Social Psychology, vol. 30, no. 4, pp. 526–537,
1974.
[38] P. M. Lewinsohn, J. R. Seeley, R. E. Roberts, and N. B. Allen,
“Center for epidemiologic studies depression scale (CES-D)
as a screening instrument for depression among communityresiding older adults,” Psychology and Aging, vol. 12, no. 2, pp.
277–287, 1997.
[39] S. Baron-Cohen and S. Wheelwright, “The empathy quotient:
an investigation of adults with asperger syndrome or high
functioning autism, and normal sex differences,” Journal of
Autism and Developmental Disorders, vol. 34, no. 2, pp. 163–175,
2004.
[40] W. Pavot and E. Diener, “Review of the satisfaction with life
scale,” in Assessing Well-Being, E. Diener, Ed., vol. 39 of Social
Indicators Research Series, pp. 101–117, Springer, Dordrecht,
Netherlands, 2009.
[41] J. Raven, “The Raven’s progressive matrices: change and stability
over culture and time,” Cognitive Psychology, vol. 41, no. 1, pp. 1–
48, 2000.
[42] A. Mohammadi and E. C. Wit, “Bayesian Structure Learning in
Sparse Gaussian Graphical Models,” Bayesian Analysis, vol. 10,
no. 1, pp. 109–138, 2015.
[43] S. Epskamp, A. O. J. Cramer, L. J. Waldorp, V. D. Schmittmann,
and D. Borsboom, “Qgraph: Network visualizations of relationships in psychometric data,” Journal of Statistical Software, vol.
48, no. 4, pp. 1–18, 2012.
Complexity
[44] P. J. Jones, “Package “Networktools”,” 2018, https://cran.r-project
.org/web/packages/networktools/networktools.pdf.
[45] “NetworkX — NetworkX [Internet],” 2018, https://networkx
.github.io/.
[46] V. D. Schmittmann, S. Jahfari, D. Borsboom, A. O. Savi, and L. J.
Waldorp, “Making large-scale networks from fMRI data,” PLoS
ONE, vol. 10, no. 9, article e0129074, 2015.
[47] D. R. Williams, M. Rhemtulla, A. Wysocki, and P. Rast,
“On non-regularized estimation of psychological networks,”
PsyArXiv [Internet], 2018, https://psyarxiv.com/xr2vf/.
[48] D. R. Williams and P. Rast, “Back to the basics: rethinking
partial correlation network methodology,” Open Sci Framew
[Internet], 2018, https://osf.io/fndru/.
[49] M. K. Forbes, A. G. C. Wright, K. E. Markon, and R. F.
Krueger, “Evidence that psychopathology symptom networks
have limited replicability,” Journal of Abnormal Psychology, vol.
126, no. 7, pp. 969–988, 2017.
[50] D. J. Robinaugh, A. J. Millner, and R. J. McNally, “Identifying
highly influential nodes in the complicated grief network,”
Journal of Abnormal Psychology, vol. 125, no. 6, pp. 747–757,
2016.
[51] J. C. Gower, “Some distance properties of latent root and vector
methods used in multivariate analysis,” Biometrika, vol. 53, no.
3-4, pp. 325–338, 1966.
[52] T. Opsahl, F. Agneessens, and J. Skvoretz, “Node centrality in
weighted networks: Generalizing degree and shortest paths,”
Social Networks, vol. 32, no. 3, pp. 245–251, 2010.
[53] J.-P. Onnela, J. Saramäki, J. Kertész, and K. Kaski, “Intensity and
coherence of motifs in weighted complex networks,” Physical
Review E: Statistical, Nonlinear, and Soft Matter Physics, vol. 71,
no. 6, 2005.
[54] J. Siltaloppi and S. L. Vargo, “Triads: A review and analytical
framework,” Marketing Theory, vol. 17, no. 4, pp. 395–414, 2017.
[55] Y. Kalish and G. Robins, “Psychological predispositions and network structure: The relationship between individual predispositions, structural holes and network closure,” Social Networks,
vol. 28, no. 1, pp. 56–84, 2006.
[56] M. Newman, Networks: An Introduction, Oxford University
Press, New York, NY, USA, 2010.
[57] M. A. Serrano, M. Boguñá, and R. Pastor-Satorras, “Correlations in weighted networks,” Physical Review E, vol. 74, no. 5,
Article ID 055101, pp. 1–4, 2006.
[58] D. Cartwright and F. Harary, “Structural balance: a generalization of Heider’s theory,” Psychological Review, vol. 63, no. 5, pp.
277–293, 1956.
[59] J. A. Davis, “Structural balance, mechanical solidarity, and
interpersonal relations,” American Journal of Sociology, vol. 68,
no. 4, pp. 444–462, 1963.
[60] F. Heider, “Attitudes and cognitive organization,” The Journal of
Psychology: Interdisciplinary and Applied, vol. 21, no. 1, pp. 107–
112, 1946.
[61] R. Toivonen, M. Kivelä, J. Saramäki, M. Viinikainen, M. Vanhatalo, and M. Sams, “Networks of emotion concepts,” PLoS ONE,
vol. 7, no. 1, Article ID e28883, 2012.
[62] A. P. Christensen, Y. N. Kenett, T. Aste, P. J. Silvia, and T.
R. Kwapil, “Network structure of the wisconsin schizotypy
scalesshort forms: examining psychometric network filtering
approaches,” Behavior Research Methods, vol. 50, no. 6, pp. 2531–
2550, 2018.
[63] S. Epskamp, D. Borsboom, and E. I. Fried, “Estimating psychological networks and their accuracy: A tutorial paper,” Behavior
Research Methods, vol. 50, no. 1, pp. 195–212, 2018.
27
[64] D. Borsboom, D. J. Robinaugh, M. Rhemtulla, and A. O.
J. Cramer, “Robustness and replicability of psychopathology
networks,” World Psychiatry, vol. 17, no. 2, pp. 143-144, 2018.
[65] L. Stankov, “Low correlations between intelligence and big five
personality traits: need to broaden the domain of personality,”
Journal of Intelligence, vol. 6, no. 2, p. 26, 2018.
[66] F. S. Ahmed and L. Stephen Miller, “Executive function mechanisms of theory of mind,” Journal of Autism and Developmental
Disorders, vol. 41, no. 5, pp. 667–678, 2011.
[67] M. A. Brandimonte, N. Bruno, and S. Collina, “Cognition,” in
Psychological Concepts: An International Historical Perspective,
P. Pawlik and G. d’Ydewalle, Eds., Psychology Press, Howe, UK,
2006.
[68] S. Epskamp, Network Psychometrics [Ph.D. Dissertation], University of Amsterdam, Amsterdam, Netherlands, 2015.
[69] F. T. McAndrew and H. S. Jeong, “Who does what on Facebook?
Age, sex, and relationship status as predictors of Facebook use,”
Computers in Human Behavior, vol. 28, no. 6, pp. 2359–2365,
2012.
Advances in
Advances in
Operations Research
Hindawi
www.hindawi.com
Volume 2018
Decision Sciences
Hindawi
www.hindawi.com
Volume 2018
Journal of
Applied Mathematics
Hindawi
www.hindawi.com
Volume 2018
The Scientific
World Journal
Hindawi Publishing Corporation
http://www.hindawi.com
www.hindawi.com
2013
Volume 2018
Journal of
Probability and Statistics
Hindawi
www.hindawi.com
Volume 2018
International
Journal of
Mathematics and
Mathematical
Sciences
Journal of
Optimization
Hindawi
www.hindawi.com
Hindawi
www.hindawi.com
Volume 2018
Volume 2018
Submit your manuscripts at
www.hindawi.com
International Journal of
Engineering
Mathematics
Hindawi
www.hindawi.com
International Journal of
Analysis
Hindawi
www.hindawi.com
Volume 2018
Journal of
Complex Analysis
Hindawi
www.hindawi.com
Stochastic Analysis
Volume 2018
Mathematical Problems
in Engineering
Hindawi
www.hindawi.com
Volume 2018
International Journal of
Hindawi
www.hindawi.com
Numerical
erical Analysis
eric
nalys
Volume 2018
Journal of
Journal of
Mathematics
Function Spaces
Hindawi
www.hindawi.com
Volume 2018
Hindawi
www.hindawi.com
Volume 2018
International Journal of
Differential Equations
Hindawi
www.hindawi.com
Volume 2018
Abstract and
Applied Analysis
Hindawi
www.hindawi.com
Volume 2018
Discrete Dynamics in
Nature and Society
Hindawi
www.hindawi.com
Volume 2018
Advances in
Mathematical Physics
Volume 2018
Hindawi
www.hindawi.com
Volume 2018