research-article

Stable Specification Search in Structural Equation Models with Latent Variables

Authors:

Tom HeskesAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 10, Issue 5

Article No.: 48, Pages 1 - 23

https://doi.org/10.1145/3341557

Published: 09 September 2019 Publication History

Abstract

In our previous study, we introduced stable specification search for cross-sectional data (S3C). It is an exploratory causal method that combines the concept of stability selection and multi-objective optimization to search for stable and parsimonious causal structures across the entire range of model complexities. S3C, however, is designed to model causal relations among observed variables. In this study, we extended S3C to S3C-Latent, to model linear causal relations between latent variables that are measured through observed proxies. We evaluated S3C-Latent on simulated data and compared the results to those of PC-MIMBuild, an extension of the PC algorithm, the state-of-the-art causal discovery method. The comparison shows that S3C-Latent achieved better performance. We also applied S3C-Latent to real-world data of children with attention deficit/hyperactivity disorder and data about measuring mental abilities among pupils. The results are consistent with those of previous studies.

References

[1]

Georgia Andreou, Paraskevi Agapitou, and Anargyros Karapetsas. 2005. Verbal skills in children with ADHD. European Journal of Special Needs Education 20, 2 (2005), 231--238.

[2]

Lawrence W. Barsalou. 2008. Grounded cognition. Annual Review of Psychology 59, 1 (2008), 617--645.

[3]

José J. Bauermeister, Patrick E. Shrout, Ligia Chávez, Maritza Rubio-Stipec, Rafael Ramírez, Lymaries Padilla, Adrianne Anderson, Pedro García, and Glorisa Canino. 2007. ADHD and gender: Are risks and sequela of ADHD the same for boys and girls? Journal of Child Psychology and Psychiatry 48, 8 (2007), 831--839.

[4]

Kenneth Bollen. 1989. Structural Equations With Latent Variables. WILEY.

[5]

Sarah A. Burnett, David M. Lane, and Lewis M. Dratt. 1979. Spatial visualization and sex differences in quantitative ability. Intelligence 3, 4 (1979), 345--354.

[6]

Judy Cantor, Randall W. Engle, and George Hamilton. 1991. Short-term memory, working memory, and verbal abilities: How do they relate? Intelligence 15, 2 (1991), 229--246.

[7]

M. Beth Casey, Ronald L. Nuttall, and Elizabeth Pezaris. 2001. Spatial-mechanical reasoning skills versus mathematics self-confidence as mediators of gender differences on mathematics subtests using cross-national gender-based items. Journal for Research in Mathematics Education 32, 1 (2001), 28--57.

[8]

Zaira Cattaneo, Albert Postma, and Tomaso Vecchi. 2006. Gender differences in memory for object and word locations. Quarterly Journal of Experimental Psychology 59, 5 (2006), 904--919.

[9]

David Maxwell Chickering. 2002. Learning equivalence classes of Bayesian-network structures. Journal of Machine Learning Research 2, Feb (2002), 445--498.

Digital Library

[10]

Ruifei Cui, Perry Groot, Moritz Schauer, and Tom Heskes. 2018. Learning the causal structure of copula models with latent variables. In Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence. 188--197.

[11]

Meredyth Daneman and Patricia A. Carpenter. 1980. Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior 19, 4 (1980), 450--466.

[12]

Bert De Smedt, Rianne Janssen, Kelly Bouwens, Lieven Verschaffel, Bart Boets, and Pol Ghesquière. 2009. Working memory and individual differences in mathematics achievement: A longitudinal study from first grade to second grade. Journal of Experimental Child Psychology 103, 2 (2009), 186--201.

[13]

Kalyanmoy Deb. 2001. Multi-Objective Optimization Using Evolutionary Algorithms. John Wiley 8 Sons, Inc., New York, NY.

Digital Library

[14]

Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (2002), 182--197.

Digital Library

[15]

Geoff Der and Ian J. Deary. 2006. Age and sex differences in reaction time in adulthood: Results from the united kingdom health and lifestyle survey. Psychology and Aging 21, 1 (2006), 62.

[16]

Fritz Drasgow. 2006. Polychoric and polyserial correlations. In Encyclopedia of Statistical Sciences, S. Kotz, C. B. Read, N. Balakrishnan, B. Vidakovic, and N. L. Johnson (Eds.).

[17]

Doris Entner and Patrik O. Hoyer. 2011. Discovering unconfounded causal relationships using linear non-Gaussian models. In New Frontiers in Artificial Intelligence, Takashi Onada, Daisuke Bekki, and Eric McCready (Eds.). Springer, Berlin, 181--195.

Digital Library

[18]

Tom Fawcett. 2004. ROC graphs: Notes and practical considerations for researchers. Machine Learning 31, 1 (2004), 1--38.

[19]

Thomas W. Frazier, Heath A. Demaree, and Eric A. Youngstrom. 2004. Meta-analysis of intellectual and neuropsychological test performance in attention-deficit/hyperactivity disorder. Neuropsychology 18, 3 (2004), 543--555.

[20]

Miranda Gaub and Caryn L. Carlson. 1997. Gender differences in ADHD: A meta-analysis and critical review. Journal of the American Academy of Child 8 Adolescent Psychiatry 36, 8 (1997), 1036--1045.

[21]

Jonathan Gershon. 2002. A meta-analytic review of gender differences in ADHD. Journal of Attention Disorders 5, 3 (2002), 143--154.

[22]

Zoubin Ghahramani and Geoffrey E. Hinton. 1996. The EM Algorithm for Mixtures of Factor Analyzers. Technical Report. CRG-TR-96-1, University of Toronto.

[23]

John J. Grefenstette. 1986. Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics 16, 1 (1986), 122--128.

Digital Library

[24]

Shane Harwood and Richard Scheines. 2002. Genetic Algorithm Search Over Causal Models (CMU-PHIL-131). Technical Report. Carnegie Mellon University.

[25]

Karl John Holzinger and Frances Swineford. 1939. A study in factor analysis: The stability of a Bi-factor solution. Supplementary Educational Monographs (1939). University of Chicago, Chicago, Series 48.

[26]

James Honaker, Gary King, and Matthew Blackwell. 2011. Amelia II: A program for missing data. Journal of Statistical Software 45, 7 (2011), 1--47.

[27]

Patrik O. Hoyer, Shohei Shimizu, Antti J. Kerminen, and Markus Palviainen. 2008. Estimation of causal effects using linear non-gaussian causal models with hidden variables. International Journal of Approximate Reasoning 49, 2 (2008), 362--378.

Digital Library

[28]

Paul B. Jacobsen, Lora M. Azzarello, and Danette M. Hann. 1999. Relation of catastrophizing to fatigue severity in women with breast cancer. Cancer Research, Therapy, and Control 8 (1999), 155--164.

[29]

Karl G. Jöreskog. 1967. Some contributions to maximum likelihood factor analysis. Psychometrika 32, 4 (1967), 443--482.

[30]

Karl G. Jöreskog. 1977. Structural equation models in the social sciences: Specification, estimation and testing. Applications of Statistics, P. R. Krishnaiah (Ed.).

[31]

Karl G. Jöreskog. 1978. Structural analysis of covariance and correlation matrices. Psychometrika 43, 4 (1978), 443--477.

[32]

Ken Kelley. 2007. Methods for the behavioral, educational, and social sciences: An R package. Behavior Research Methods 39, 4 (2007), 979--984.

[33]

David A. Kenny, Deborah A. Kashy, and Niall Bolger. 1998. Data analysis in social psychology. In The Handbook of Social Psychology, Daniel Gilbert, Susan Fiske, and Gardner Lindzey (Eds.). Vol. 1. Boston, MA: McGraw-Hill, 233--265.

[34]

George Lakoff and Rafael E. Núñez. 2000. Where mathematics comes from: How the embodied mind brings mathematics into being. AMC 10, 12 (2000), 720--733.

[35]

Marcia C. Linn and Anne C. Petersen. 1985. Emergence and characterization of sex differences in spatial ability: A meta-analysis. Child Development 56, 6 (1985), 1479--1498.

[36]

John Scott Long. 1983. Covariance Structure Models: An Introduction to LISREL. Sage Beverly Hills.

[37]

Richard S. Longman, Donald H. Saklofske, and Tak S. Fung. 2007. WAIS-III percentile scores by education and sex for US and canadian populations. Assessment 14, 4 (2007), 426--432.

[38]

Richard Lynn. 1987. The intelligence of the mongoloids: A psychometric, evolutionary and neurological theory. Personality and Individual Differences 8, 6 (1987), 813--844.

[39]

Marloes H. Maathuis, Markus Kalisch, and Peter Bühlmann. 2009. Estimating high-dimensional intervention effects from observational data. The Annals of Statistics 37, 6A (2009), 3133--3164.

[40]

Robert MacCallum. 1986. Specification searches in covariance structure modeling. Psychological Bulletin 100, 1 (1986), 107--120.

[41]

George A. Marcoulides and Zvi Drezner. 2001. Specification searches in structural equation modeling with a genetic algorithm. New Development and Techniques in Structural Equation Modeling, George A. Marcoulides and Randall E. Schumacker (Eds.). Lawrence Erlbum Associates, 247--268.

[42]

George A. Marcoulides and Zvi Drezner. 2003. Model specification searches using ant colony optimization algorithms. Structural Equation Modeling: A Multidisciplinary Journal 10, 1 (2003), 154--164.

[43]

George A. Marcoulides, Zvi Drezner, and Randall E. Schumacker. 1998. Model specification searches in structural equation modeling using Tabu search. Structural Equation Modeling: A Multidisciplinary Journal 5, 4 (1998), 365--376.

[44]

Nicolai Meinshausen and Peter Bühlmann. 2010. Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 72, 4 (2010), 417--473.

[45]

Micaela Mitolo, Simona Gardini, Paolo Caffarra, Lucia Ronconi, Annalena Venneri, and Francesca Pazzaglia. 2015. Relationship between spatial ability, visuospatial working memory and self-assessed spatial orientation ability: A study in older adults. Cognitive Processing 16, 2 (2015), 165--176.

[46]

Kelly S. Mix, Susan C. Levine, Yi-Ling Cheng, Chris Young, D. Zachary Hambrick, Raedy Ping, and Spyros Konstantopoulos. 2016. Separate but correlated: The latent structure of space and mathematics across development. Journal of Experimental Psychology: General 145, 9 (2016), 1206--1227.

[47]

Torunn Stene Nøvik, Amaia Hervas, Stephen J. Ralston, Søren Dalsgaard, Rob Rodrigues Pereira, Maria J. Lorenzo, and ADORE Study Group. 2006. Influence of gender on attention-deficit/hyperactivity disorder in Europe--ADORE. European Child 8 Adolescent Psychiatry 15, 1 (2006), i15--i24.

[48]

Juan Miguel Ogarrio, Peter Spirtes, and Joe Ramsey. 2016. A hybrid causal search algorithm for latent variable models. In Proceedings of the Conference on Probabilistic Graphical Models. 368--379.

[49]

Ulf Olsson, Fritz Drasgow, and Neil J. Dorans. 1982. The polyserial correlation coefficient. Psychometrika 47, 3 (1982), 337--347.

[50]

Judea Pearl. 2000. Causality: Models, Reasoning and Inference. Vol. 29. Cambridge University Press.

Digital Library

[51]

Ridho Rahmadi, Perry Groot, Marianne Heins, Hans Knoop, Tom Heskes, and the OPTIMISTIC consortium. 2017. Causality on cross-sectional data: Stable specification search in constrained structural equation modeling. Applied Soft Computing 52 (2017), 687--698.

Digital Library

[52]

Joseph Ramsey. 2010. Bootstrapping the PC and CPC Algorithms to Improve Search Accuracy (CMU-PHIL-187). Technical Report. Carnegie Mellon University.

[53]

Nancy M. Robinson, Robert D. Abbott, Virginia W. Berninger, and Julie Busse. 1996. Structure of abilities in math-precocious young children: Gender similarities and differences. Journal of Educational Psychology 88, 2 (1996), 341.

[54]

Douglas E. Scates. 1940. A study in factor analysis: The stability of a Bi-Factor solution. Karl J. Holzinger, Frances Swineford. The Elementary School Journal 40, 6 (1940), 468--469.

[55]

Marco Scutari, Catharina Elisabeth Graafland, and José Manuel Gutiérrez. 2018. Who learns better Bayesian network structures: Constraint-based, score-based or hybrid algorithms? In International Conference on Probabilistic Graphical Models. 416--427.

[56]

Shohei Shimizu and Kenneth Bollen. 2014. Bayesian estimation of causal direction in acyclic structural equation models with individual-specific confounder variables and non-gaussian distributions. Journal of Machine Learning Research 15, 1 (2014), 2629--2652.

Digital Library

[57]

Ricardo Silva, Richard Scheines, Clark Glymour, and Peter Spirtes. 2006. Learning the structure of linear latent variable models. Journal of Machine Learning Research 7, Feb (2006), 191--246.

Digital Library

[58]

Elena Sokolova, Perry Groot, Tom Claassen, Kimm J. van Hulzen, Jeffrey C. Glennon, Barbara Franke, Tom Heskes, and Jan Buitelaar. 2016. Statistical evidence suggests that inattention drives hyperactivity/impulsivity in attention deficit-hyperactivity disorder. PLoS One 11, 10 (2016), e0165120.

[59]

Elena Sokolova, Anoek M. Oerlemans, Nanda N. Rommelse, Perry Groot, Catharina A. Hartman, Jeffrey C. Glennon, Tom Claassen, Tom Heskes, and Jan K. Buitelaar. 2017. A causal and mediation analysis of the comorbidity between attention deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD). Journal of Autism and Developmental Disorders 47, 6 (2017), 1595--1604.

[60]

Peter Spirtes. 2010. Introduction to causal inference. Journal of Machine Learning Research 11, May (2010), 1643--1662.

Digital Library

[61]

Peter Spirtes, Clark N. Glymour, and Richard Scheines. 2000. Causation, Prediction, and Search. Vol. 81. MIT press.

[62]

Michael J. Sullivan and Joyce L. D’Eon. 1990. Relation between catastrophizing and depression in chronic pain patients. Journal of Abnormal Psychology 99, 3 (1990), 260--263.

[63]

Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological) 58, 1 (1996), 267--288.

[64]

Daphne J. van Steijn, Jennifer S. Richards, Anoek M. Oerlemans, Saskia W. de Ruiter, Marcel A.G. van Aken, Barbara Franke, Jan Buitelaar, and Nanda N. J. Rommelse. 2012. The co-occurrence of autism spectrum disorder and attention-deficit/hyperactivity disorder symptoms in parents of children with ASD or ASD with ADHD. Journal of Child Psychology and Psychiatry 53, 9 (2012), 954--963.

[65]

Rose K. Vukovic and Nonie K. Lesaux. 2013. The relationship between linguistic skills and arithmetic knowledge. Learning and Individual Differences 23 (2013), 87--91.

[66]

Erik G. Willcutt, Bruce F. Pennington, and John C. DeFries. 2000. Etiology of inattention and hyperactivity/impulsivity in a community sample of twins with learning difficulties. Journal of Abnormal Child Psychology 28, 2 (2000), 149--159.

Cited By

Brenneke IIsaak MLentz W(2022)Review – Measuring the Reputation of Companies and Industries Using the Example of HorticultureJournal of International Food & Agribusiness Marketing10.1080/08974438.2021.200392235:3(305-335)Online publication date: 7-Jan-2022
https://doi.org/10.1080/08974438.2021.2003922
Nur YRahmadi REffendy C(2021)Causal Modeling Between Factors on Quality of Life in Cancer Patients Using S3C-Latent AlgorithmJournal of Information Systems Engineering and Business Intelligence10.20473/jisebi.7.1.74-837:1(74)Online publication date: 27-Apr-2021
https://doi.org/10.20473/jisebi.7.1.74-83

Index Terms

Stable Specification Search in Structural Equation Models with Latent Variables
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Learning in probabilistic graphical models
        Latent variable models
2. Mathematics of computing
  1. Mathematical analysis
    1. Mathematical optimization
      1. Discrete optimization
        Optimization with randomized search heuristics
        Evolutionary algorithms
  2. Probability and statistics
    1. Probabilistic representations
      1. Causal networks

Recommendations

On the testability of causal models with latent and instrumental variables
UAI'95: Proceedings of the Eleventh conference on Uncertainty in artificial intelligence

Certain causal models involving unmeasured variables induce no independence constraints among the observed variables but imply, nevertheless, inequality constraints on the observed distribution. This paper derives a general formula for such inequality ...
Causality on cross-sectional data

Graphical abstractDisplay Omitted HighlightsA finite-sample robust, hypothesis-free, score-based causal discovery algorithm.Reliability scores for inferred causal relationships.An innovative application of multi-objective optimization. Causal modeling ...
Causal generative explainers using counterfactual inference: a case study on the Morpho-MNIST dataset
Abstract
In this paper, we propose leveraging causal generative learning as an interpretable tool for explaining image classifiers. Specifically, we present a generative counterfactual inference approach to study the influence of visual features (pixels) ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 10, Issue 5

Special Section on Advances in Causal Discovery and Inference and Regular Papers

September 2019

314 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/3360733

Editor:
Yu Zheng
JD Finance, China

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2019

Accepted: 01 June 2019

Revised: 01 May 2019

Received: 01 August 2018

Published in TIST Volume 10, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
193
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Brenneke IIsaak MLentz W(2022)Review – Measuring the Reputation of Companies and Industries Using the Example of HorticultureJournal of International Food & Agribusiness Marketing10.1080/08974438.2021.200392235:3(305-335)Online publication date: 7-Jan-2022
https://doi.org/10.1080/08974438.2021.2003922
Nur YRahmadi REffendy C(2021)Causal Modeling Between Factors on Quality of Life in Cancer Patients Using S3C-Latent AlgorithmJournal of Information Systems Engineering and Business Intelligence10.20473/jisebi.7.1.74-837:1(74)Online publication date: 27-Apr-2021
https://doi.org/10.20473/jisebi.7.1.74-83

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents