Psychometric Evaluation of the NSD-Krapal Test Online: Structure Validity,

Reliability, and Invariance Across Educational Levels

Isman Rahmani Yusron1, Dirga Atmaka Nitiharja1, Muhammad
Ardi2, Nur Chairul Edwiansyah2
1
Program Studi Psikologi, Univeritas Muhammadiyah Bandung, Indonesia
2,
PT.Nirmala Satya Development, Jakarta, Indonesia
E-mail: isman.rahmani@umbandung.ac.id

Abstract
The Krapal Test, a digital adaptation of the traditional Kraepelin and Pauli Tests, was
evaluated for its construct validity, reliability, and measurement invariance across
different educational levels. This study assesses the psychometric properties of the
digital Krapal Test, a fusion of the Kraepelin and Pauli Tests, facilitated by Nirmala
Satya Development. Analyzing data from 33,341 participants (mean age = 26.56,
SD = 6.59), we utilized JASP and Jamovi for statistical analysis, including
confirmatory factor analysis (CFA) and multigroup confirmatory factor analysis
(MFCA) to test validity and measurement invariance across educational strata. The
Krapal Test exhibited excellent model fit indices (CFI = 0.963, RMSEA = 0.051,
SRMR = 0.018) and high reliability coefficients (ω and α > 0.990) across all levels of
invariance (configural, metric, scalar, strict). The findings highlight the Krapal Test
as a reliable and valid tool for psychological and human resource assessments,
measuring cognitive processing speed and accuracy effectively. Future research
should continue to validate its psychometric properties across various cultural and
organizational contexts and explore the potential impacts of the digital medium on
test outcomes. The study's implications underscore the importance of digital
transformation in psychological testing, promoting efficient and effective
assessment practices in modern settings.
Keywords: Krapal Test, construct validity, measurement invariance, digital
psychological assessment, multigroup confirmatory factor analysis.

Abstrak
Tes Krapal, adaptasi digital dari Tes Kraepelin dan Pauli tradisional, dievaluasi untuk
validitas konstruk, reliabilitas, dan invariansi pengukuran di berbagai tingkat
pendidikan. Studi ini menilai properti psikometrik dari Tes Krapal digital, fusi dari
Tes Kraepelin dan Pauli, yang dipermudah oleh Pengembangan Nirmala Satya.
Menganalisis data dari 33.341 partisipan (rerata usia = 26,56, SD = 6,59), kami
menggunakan JASP dan Jamovi untuk analisis statistik, termasuk analisis faktor
konfirmatori (CFA) dan analisis faktor konfirmatori multigroup (MFCA) untuk
menguji validitas dan invariansi pengukuran di seluruh strata pendidikan. Tes
Krapal menunjukkan indeks kecocokan model yang sangat baik (CFI = 0,963,
RMSEA = 0,051, SRMR = 0,018) dan koefisien reliabilitas tinggi (ω dan α > 0,990)
di semua tingkat invariansi (konfigurasi, metrik, skalar, ketat). Temuan ini
menyoroti Tes Krapal sebagai alat yang dapat diandalkan dan valid untuk penilaian
psikologis dan sumber daya manusia, mengukur kecepatan dan ketepatan
pemrosesan kognitif secara efektif. Penelitian masa depan seharusnya terus
memvalidasi properti psikometriknya di berbagai konteks budaya dan organisasi
serta menjelajahi dampak potensial media digital pada hasil tes. Implikasi studi ini
menekankan pentingnya transformasi digital dalam pengujian psikologis,
mempromosikan praktik penilaian yang efisien dan efektif di lingkungan modern.
Kata kunci: Tes Krapal, validitas konstruk, invariansi pengukuran, penilaian
psikologis digital, analisis faktor konfirmatori multigroup.

Introduction
Kraepelin and Pauli Tests are two psychological instruments widely used in
Indonesia. Although initially developed in the context of psychiatry and clinical
psychology, both have found broader applications in various fields. Emil Kraepelin,
the creator of the Kraepelin Test, began a series of experiments aiming to develop
the concept of "messende Individualpsychologie," or quantitative individual
psychology, to understand the basic mental characteristics of individuals. Kraepelin
hoped that by using several psychological tests, he could establish the "status
psychicus" of individuals, similar to how general medicine uses chemical and
physical tests to diagnose diseases (Engstrom, 2016).
As its use evolved, the Kraepelin Test has been modified and adopted in various
fields, particularly in industry and human resource management. The modification
of the Kraepelin Test by Prof. Dr. Richard Pauli in 1938 resulted in the Pauli Test,
which is now widely used. Arnold (1975) noted several successes and benefits of
using the Pauli Test, which have been widely recognized. One significant success is
the Pauli Test's ability to reliably identify individual differences caused by age
factors. Additionally, this test has proven useful in measuring constitutive
differences, such as those between genders and between urban and rural
populations. The Pauli Test has also been valuable in assessing job performance and
work attitudes, as well as in the context of personnel selection (Arnold, 1975).
Despite its wide use, the test presentation methods still largely rely on traditional
means such as paper and pencil. In today's digital era, the use of paper and pencil
tests has become inefficient for various reasons. Particularly in the context of the
Kraepelin and Pauli Tests, which require a significant amount of paper, often
referred to as the newspaper test. Efficiency in test presentation is a challenge for
these newspaper tests. Although the tests are still frequently used, their space-
consuming and inefficient presentation forms raise validity issues related to test
administration.
In today's era of digital penetration in society, there are solutions to the efficiency
problems in test presentation and administration. The use of computerized tests,
now widely available, offers various administrative conveniences, cost-
effectiveness, minimizes paper use, and reduces administrative costs and scoring
(Jeong, 2014; Threlfall et al., 2007). Although presented in different media—such as
computer and digital formats—statistically, the use of computer-based tests does
not show significant differences in performance outcomes (McClelland & Cuevas,
2020). Moreover, more and more people are accustomed to using digital media and
applications in their daily lives.
One provider of digital and online test services, PT.Nirmala Satya Development
(https://nsd.co.id/), offers tests with a framework similar to the Pauli and Kraepelin
tests. The test, named the Krapal Test, was developed by combining elements of
the Kraepelin and Pauli tests and is efficiently presented with a digital user interface
that facilitates user engagement. Despite being used and interpreted for various
purposes for a long time, issues regarding validity and reliability need to be further
explored. The transformation from paper and pencil tests to online digital tests
creates a different medium for test completion.
Essentially, the development of digital software applications for the Pauli Test has
provided significant benefits for psychologists in efficiently and quickly calculating
test results (Analya et al., 2023). However, exploratively examining the
psychometric properties with data obtained through online tests will provide the
necessary insights for developing other digital measurement tools that have the
potential for efficiency and effectiveness for users. Particularly in the field of
psychological assessment practice and human resource assessment in various
settings.
This digital transformation not only demands adjustments in test presentation but
also requires a reevaluation of the instrument's validity and reliability. Construct
validity and content validity testing are crucial to ensure that the digital test
measures the same psychological constructs as its traditional version. Additionally,
the instrument's reliability must be ensured through repeated trials to check for
consistency in results. Further research is needed to explore how differences in test
presentation can affect outcomes and data interpretation. Therefore, this study will
assess and reevaluate the psychometric properties of the Krapal Test in digital
format to ensure that this tool remains effective and reliable in various modern
assessment contexts.

Methods
Participants
A total of 33,341 sample data from the Krapal Test, provided by the digital and
online professional test service PT. Nirmala Satya Development, were analyzed in
this study. The mean age of the participants was M=26.56 with a standard deviation
of SD=6.59. The demographic details of the sample are shown in Table 1.

Educati Mea Media Varian Minimu Maximu

N SE SD
on n n ce m m

27. 0.128 6.7

Age D3 2761 25 45.2 19 62
7 0 2
25. 0.136 3.8
D4 792 24.0 14.9 19 59
1 9 5
1884 27. 0.042 5.8
S1 26 34.2 18 64
5 4 6 5
32. 0.261 6.9
S2 708 31.0 48.3 22 60
5 2 5
25. 0.142 8.4
SMA 3517 23 71.6 16 63
5 7 6
 Educati Mea Media Varian Minimu Maximu
N SE SD
on n n ce m m
23. 0.079 6.4
SMK 6718 22.0 42.0 16 66
8 0 8

Instruments
The instruments examined in this study include the Krapal Test, developed by
PT.Nirmala Satya Development, which combines elements of the Kraepelin and Pauli
Tests. The test is presented digitally and online, designed to facilitate user
engagement through a digital user interface. The Krapal Test aims to measure
individual differences in cognitive processing speed and accuracy, among other
psychological constructs.
Data Analysis
Data analysis was performed using several software tools, including JASP and
Jamovi. The analyses included descriptive statistics, reliability testing, and
confirmatory factor analysis (CFA). Additionally, a Multigroup Confirmatory Factor
Analysis (MFCA) was conducted to examine the invariance of the measurement tool
across different educational strata.
Confirmatory Factor Analysis (CFA)
CFA was employed to evaluate the model fit using various fit indices such as the
Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root Mean Square Error of
Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR).
These indices provide insights into how well the empirical data align with the
hypothesized theoretical model (Brown, 2006; Harrington, 2009; Thompson, 2004).
Invariance Testing (MFCA)
MFCA was used in this study to ensure the equivalence of the factor structure
across different educational strata. The invariance testing process involved stages
of model testing, including configural invariance, metric invariance, scalar
invariance, and residual invariance (Chen, 2007; Engelhard & Wind, 2018; Millsap,
2011; Wicherts & Dolan, 2010). Measurement invariance indicates that the
instrument functions consistently across various participant groups.
Reliability

The reliability of the instrument was evaluated using omega (ω) and alpha (α)
coefficients (Zhang & Yuan, 2016). These coefficients measure the internal
consistency of the test items. The analysis results showed that the ω and α
coefficients were close to 1, indicating excellent reliability of the test (Allen & Yen,
2001; Bentler & Woodward, 1980; Raykov & Marcoulides, 2011).

Result and Discussions

Factor Structure Analysis
Based on the results from various model fit indices in Table 2, the factor structure
model of the Krapal Test demonstrates a high degree of model goodness of fit, with
coefficients exceeding 0.95. All indices surpass the commonly accepted threshold of
0.95, indicating excellent model fit. The Comparative Fit Index (CFI), which
compares the hypothesized model to a baseline model, stands at 0.963, suggesting
a very good fit between the hypothesized model and the observed data.
A CFI value close to 1 indicates an excellent fit, with values above 0.95 considered
very good (Chen, 2007; Hu & Bentler, 1999). This suggests that the hypothesized
factor structure of the Krapal Test is valid and aligns well with empirical data,
making it a reliable tool in psychological and human resource assessments.

Table 2 Model Fit Indices

Fit Index
Va
Index lu
e
0.
Comparative Fit Index (CFI) 96
3
0.
Tucker-Lewis Index (TLI) 96
1
0.
Bentler-Bonett Non-normed Fit Index
96
(NNFI)
1
0.
Bentler-Bonett Normed Fit Index
96
(NFI)
2
0.
Parsimony Normed Fit Index (PNFI) 91
3
0.
Bollen's Relative Fit Index (RFI) 96
0
0.
Bollen's Incremental Fit Index (IFI) 96
3
0.
Relative Noncentrality Index (RNI) 96
3

Besides CFI, model fit indices are also assessed through the Root Mean Square Error
of Approximation (RMSEA) and Standardized Root Mean Square Residual (SRMR).
RMSEA evaluates how well a model with a certain number of parameters
approximates the actual data in the population. As shown in Table 3, the RMSEA
value for the Krapal Test is below 0.05, generally considered indicative of a very
good fit (Chen, 2007). In this study, an RMSEA value of 0.051 suggests that the
hypothesized model fits the observed data well. The 90% confidence interval (0.051
to 0.052) is very narrow, indicating the stability of this RMSEA estimate. The
significant RMSEA p-value (p = 0.000) further supports the model's significant fit
with the data.
Table 3 Additional Goodness of Fit Indices
Other fit measures
Valu
Metric
e
Root mean square error of approximation 0.05
(RMSEA) 1
0.05
RMSEA 90% CI lower bound
1
0.05
RMSEA 90% CI upper bound
2
0.00
RMSEA p-value
0
Standardized root mean square residual 0.01
(SRMR) 8
406.
Hoelter's critical N (α = .05)
428
420.
Hoelter's critical N (α = .01)
556
0.90
Goodness of fit index (GFI)
3
0.37
McDonald fit index (MFI)
5
1.99
Expected cross validation index (ECVI)
1

Additionally, the SRMR value shown in Table 3 is below 0.08, typically indicating
good model-data fit. SRMR measures the average discrepancy between observed
and predicted correlations by the model. An SRMR value of 0.018 in this study
indicates that the hypothesized model fits the observed data very well. The low
SRMR signifies minimal discrepancies between predicted and actual correlations,
suggesting excellent model fit.
Based on an RMSEA of 0.051 and an SRMR of 0.018, the hypothesized model in this
study shows a very good fit with the empirical data from the Krapal Test. These
indices confirm that the model accurately reflects the factor structure of the Krapal
Test. The RMSEA within a good range and the very low SRMR provide strong
evidence that this model is reliable for describing the relationships among the
measured variables. These results support the factor structure validity of the Krapal
Test, reinforcing its use in psychological assessments and human resource
management.
Table 4 Standardized Loadings of the Single-Factor Model
 Standardized
Loadings of the
Single-Factor
Model
Standardiz
Item
ed loading
L01 0.722
L02 0.829
L03 0.846
L04 0.863
L05 0.871
L06 0.862
L07 0.853
L08 0.858
L09 0.865
L10 0.875
L11 0.878
L12 0.882
L13 0.877
L14 0.879
L15 0.875
L16 0.872
L17 0.873
L18 0.870
L19 0.871
L20 0.870
L21 0.871
L22 0.870
L23 0.871
L24 0.869
L25 0.869
L26 0.870
L27 0.870
L28 0.873
L29 0.872
L30 0.871
L31 0.872
L32 0.872
L33 0.869
L34 0.870
L35 0.870
L36 0.871
L37 0.868
L38 0.867
L39 0.869
L40 0.865

Table 4 displays the factor loadings for each item of the Krapal Test. High
standardized loadings on each item indicate excellent measurement quality. The
high standardized loading factor for each item shows strong consistency in
measurement, with an average value of 0.871. This confirms that each item
effectively reflects the single variable measured by the model, demonstrating
measurement consistency and accuracy. Moreover, the consistency in standardized
loading factors indicates that the Krapal Test is reliable and provides consistent
results across different measurements. The construct validity of the Krapal Test is
also confirmed by the high standardized loading factors, demonstrating that the
constructs measured are effectively represented by the items in the test.

Measurement Invariance Analysis

The measurement invariance testing for the Krapal Test in this study includes
several stages: configural, metric, scalar, and residual. The results show that the
Krapal Test exhibits good invariance at various levels of testing. Overall, the CFI,
RMSEA, and SRMR values at each level of invariance testing show good values as
suggested in Chen's (2007) research on model fit indices, with no significant
changes. This indicates consistency in model fit at each level of testing.
Specifically, as shown in Table 5, at the configural invariance level, the Krapal Test
shows good model fit indices with a CFI of 0.961, above the minimum
recommended value of 0.95 (Chen, 2007; Hu & Bentler, 1999). The RMSEA and
SRMR values at this level also show figures below the recommended minimum
(RMSEA<0.06; SRMR<0.08), indicating that at the configural level, the factor
structure of the Krapal Test shows consistency across different educational levels.
Table 5: Measurement Invariance Testing Results

Invarian RMSE SRM

Level CFI TLI GFI Δ
ce A R Δ RMSEA Δ SRMR
CFI

Config
0.961 0.959 0.903 0.052 0.019
ural

Loading Metric 0.96 0.96 0.90 0.02 0.00 0.001 -0.006

0.051
1 1 2 5 0

Intersep Scalar 0.96 0.96 0.90 0.050 0.02 0.00 0.001 -0.001
 0 1 1 6 1

Residual Strict 0.95 0.96 0.90 0.02 0.00 0.000 0.000

0.050
9 2 1 6 1
CFI= Comparative Fit Index; TLI=Tucker Lewis Index; GFI=Goodness of Fit Index; RMSEA= Root Mean Square Error of Approximation; SRMR=

Standardized Root Mean Square Residual; Δ CFI, RMSEA, SRMR = Difference in CFI, RMSEA, & SRMR from previous level invariance testing.

At the metric invariance level, the Krapal Test shows a CFI value remaining at
0.961, with ΔCFI = 0.000, meaning no significant changes in model fit. The RMSEA
value slightly decreases to 0.051, and SRMR increases to 0.025, but these changes
remain within accepted limits. Metric invariance indicates that the regression
coefficients from indicators to factors are the same across groups, suggesting that
the relationship between indicators and the construct measured is consistent across
groups. This means that score differences between groups can be attributed to
differences in the construct measured, not the items used to measure the construct.

At the scalar invariance level, the CFI value slightly decreases to 0.960, with ΔCFI =
0.001, which is still within the acceptable threshold. The RMSEA value also remains
stable at 0.050, with ΔRMSEA = 0.001. SRMR value remains stable at 0.026, with
ΔSRMR = 0.001. Scalar invariance shows that the items' intercepts are equal across
groups, indicating that differences in item scores reflect differences in the construct
being measured, rather than differences in the measurement model across groups.
Finally, the strict invariance level shows that the CFI value remains stable at 0.959,
with ΔCFI = 0.001, and the RMSEA value remains at 0.050, with ΔRMSEA = 0.000.
The SRMR value also remains stable at 0.026, with ΔSRMR = 0.000. Strict
invariance suggests that the error variances of the items are equal across groups,
indicating that the measurement model shows very good invariance across
educational levels.
Overall, the measurement invariance testing results indicate that the Krapal Test
maintains consistency in measurement across different educational levels,
suggesting that the test is equally valid and reliable for measuring constructs across
diverse groups. This indicates that the Krapal Test can be used in various settings
and for different educational levels without concern for measurement bias.
Based on the analysis of the Krapal Test, including factor structure analysis, model
fit indices, and measurement invariance testing, it can be concluded that the Krapal
Test demonstrates good construct validity and reliability. The factor structure
analysis shows that the items effectively represent the constructs being measured,
and the model fit indices confirm a good fit between the hypothesized model and
the observed data. Measurement invariance testing further supports the test's
validity and reliability across different groups, indicating that the Krapal Test can be
used in various settings without concern for measurement bias. These findings
suggest that the Krapal Test is a valuable tool in psychological and human resource
assessments, providing consistent and accurate measurements of relevant
constructs.

Conclusion
This research aimed to evaluate the construct validity and reliability of the Krapal
Test, a digital psychological assessment tool that combines elements from the
traditional Kraepelin and Pauli Tests. The study also sought to examine the
measurement invariance of the Krapal Test across different educational levels,
ensuring its applicability in diverse settings. The results demonstrate that the Krapal
Test possesses strong construct validity, excellent reliability, and robust
measurement invariance across different educational groups.
The confirmatory factor analysis (CFA) and model fit indices, such as the CFI, TLI,
RMSEA, and SRMR, consistently showed values exceeding acceptable thresholds,
confirming that the Krapal Test reliably measures the intended psychological
constructs. High standardized loadings for each item further support the test's
validity and reliability, suggesting that the Krapal Test effectively captures
individual differences in cognitive processing speed and accuracy.
The shift from traditional paper-based tests to a digital format represents a
significant advancement in test administration efficiency. Digital tests offer several
advantages, including reduced paper usage, cost-effectiveness, and streamlined
data collection and scoring processes. This transformation aligns with the increasing
digital penetration in society and the growing familiarity of users with digital media.
The multigroup confirmatory factor analysis (MFCA) demonstrated that the Krapal
Test maintains consistency in measurement across different educational levels,
including high school, vocational school, diploma, and undergraduate and graduate
levels. This invariance indicates that the Krapal Test is equally valid and reliable for
measuring constructs across diverse groups, making it a versatile tool in various
contexts.
The robust psychometric properties of the Krapal Test justify its use in psychological
and human resource assessments. Its demonstrated validity and reliability support
its application in measuring cognitive processing speed, accuracy, job performance,
work attitudes, and personnel selection. Additionally, the digital format enhances its
practicality and efficiency in modern assessment environments.
While the current study provides strong evidence for the Krapal Test's validity and
reliability, ongoing research is necessary to continually validate its psychometric
properties. Future studies should explore the test's performance in different cultural
and organizational contexts to ensure its broader applicability. The continued
integration of digital technologies in psychological assessment presents
opportunities for further innovations.
Future research could explore the development of adaptive testing algorithms,
which tailor the difficulty of test items to the individual’s performance in real-time,
enhancing the precision and efficiency of assessments. Future studies should
investigate the potential impact of the digital medium on test outcomes.
Although initial findings suggest no significant differences between digital and
traditional formats, further exploration is needed to understand how the digital
interface influences test-taker engagement, response styles, and overall
performance. Longitudinal research is warranted to examine the stability and
predictive validity of the Krapal Test over time. Such studies could provide insights
into the test's ability to predict long-term outcomes, such as job performance,
career progression, and academic success.
In conclusion, this study advances the field by providing a comprehensive
evaluation of the Krapal Test's psychometric properties, demonstrating its validity,
reliability, and measurement invariance in a digital format. The findings support the
test's utility in various psychological and human resource assessment contexts,
highlighting the benefits of digital transformation in psychological testing. Future
research should continue to explore and enhance the test's applications and
psychometric robustness, ensuring its effectiveness in an evolving digital
landscape.

Acknowledgement
This research was supported by funding from Nirmala Satya Development. We
express our deepest gratitude to Nirmala Satya Development for their financial
assistance, which made this study possible. Additionally, we would like to thank
them for providing the data required for our analysis. Their support has been
invaluable in the successful completion of this research.

References

Allen, M. J., & Yen, W. M. (2001). Introduction to Measurement Theory (1 edition, 1

edition). Waveland Pr Inc.

Analya, P., Ratnadewi, R., Adelina, I., Pintu, M. C. P., & Hangkawidjaja, A. D. (2023).
Pauli testing tool application design. 7(3), 2257–2265.

Arnold, W. (1975). Der Pauli-Test: Anweisung zur sachgemäßen Durchführung,

Auswertung und Anwendung des Kraepelinschen Arbeitsversuches (5.,
korrigierte Aufl). Springer.

Bentler, P. M., & Woodward, J. A. (1980). Inequalities among lower bounds to

reliability: With applications to test construction and factor analysis.
Psychometrika, 45(2), 249–267. https://doi.org/10.1007/BF02294079

Brown, T. A. (2006). Confirmatory factor analysis for applied research. Guilford

Press.

Chen, F. F. (2007). Sensitivity of Goodness of Fit Indexes to Lack of Measurement

Invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3),
464–504. https://doi.org/10.1080/10705510701301834

Engelhard, G., & Wind, S. A. (2018). Invariant measurement with raters and rating
scales: Rasch models for rater-mediated assessments (First edition).
Routledge.

Engstrom, E. J. (2016). Tempering Madness: Emil Kraepelin’s Research on Affective

Disorders. Osiris, 31(1), 163–180. https://doi.org/10.1086/687834

Harrington, D. (2009). Confirmatory factor analysis. University Press.

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure
analysis: Conventional criteria versus new alternatives. Structural Equation
Modeling: A Multidisciplinary Journal, 6(1), 1–55.
https://doi.org/10.1080/10705519909540118

Jeong, H. (2014). A comparative study of scores on computer-based tests and

paper-based tests. Behaviour & Information Technology, 33(4), 410–422.
https://doi.org/10.1080/0144929X.2012.710647

McClelland, T., & Cuevas, J. (2020). A COMPARISON OF COMPUTER BASED TESTING

AND PAPER AND PENCIL TESTING IN MATHEMATICS ASSESSMENT. 10(2).

Millsap, R. E. (2011). Statistical approaches to measurement invariance. Routledge.

Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory.

Routledge.

Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding

concepts and applications (1st ed). American Psychological Association.

Threlfall, J., Pool, P., Homer, M., & Swinnerton, B. (2007). Implicit aspects of paper
and pencil mathematics assessment that come to light through the use of the
computer. Educational Studies in Mathematics, 66(3), 335–348.
https://doi.org/10.1007/s10649-006-9078-5

Wicherts, J. M., & Dolan, C. V. (2010). Measurement Invariance in Confirmatory

Factor Analysis: An Illustration Using IQ Test Performance of Minorities.
Educational Measurement: Issues and Practice, 29(3), 39–47.
https://doi.org/10.1111/j.1745-3992.2010.00182.x

Zhang, Z., & Yuan, K.-H. (2016). Robust Coefficients Alpha and Omega and
Confidence Intervals With Outlying Observations and Missing Data: Methods
and Software. Educational and Psychological Measurement, 76(3), 387–411.
https://doi.org/10.1177/0013164415594658