Keywords
d-separation
substantiation
overcontrol bias
endogenous selection bias
How to Cite
Number of views: 319
Number of downloads: 303
Abstract
Causal inference is a fundamental goal of many research endeavors, including scholarship in the field of language education and learning. Randomized controlled trials are considered an ideal design to test causal claims, but not all claims can be subjected to experimental treatment due to ethical and practical constraints. In this article, we provide an overview of the conditions under which causal inference may be made from observational data. This includes recognition of the role of confounders and colliders; the former are common causes of the independent and dependent variables and must be controlled, while the latter are common effects and must not be controlled. We illustrate these ideas with two examples involving ability beliefs and demonstrate them through directed acyclic graphs. We discuss the implications of this approach to causal inference from observational data, specifically in individual differences in language learning research, highlighting the need for explicit modeling of causal relationships and the risk of the atheoretical inclusion of variables, whether as controls, predictors, or covariates.
References
Acharya, A., Blackwell, M., & Sen, M. (2016). Explaining causal findings without bias: Detecting and assessing direct effects. American Political Science Review, 110(3), 512-529. DOI: https://doi.org/10.1017/S0003055416000216
Achen, C. H. (2005). Let’s put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science, 22(4), 327-339. DOI: https://doi.org/10.1080/07388940500339167
Al-Hoorie, A. H. (2017). Implicit attitudes in language learning [Doctoral dissertation, University of Nottingham].
Al-Hoorie, A. H. (2018). The L2 motivational self system: A meta-analysis. Studies in Second Language Learning and Teaching, 8(4), 721-754. DOI: https://doi.org/10.14746/ssllt.2018.8.4.2
Al-Hoorie, A. H., Hiver, P., & In’nami, Y. (2024). The validation crisis in the L2 motivational self system tradition. Studies in Second Language Acquisition, 46(2), 307-329. DOI: https://doi.org/10.1017/S0272263123000487
Al-Hoorie, A. H., Hiver, P., Kim, T.-Y., & De Costa, P. I. (2021). The identity crisis in language motivation research. Journal of Language and Social Psychology, 40(1), 136-153. DOI: https://doi.org/10.1177/0261927X20964507
Al-Hoorie, A. H., Hiver, P., Larsen-Freeman, D., & Lowie, W. (2023). From replication to substantiation: A complexity theory perspective. Language Teaching, 56(2), 276-291. DOI: https://doi.org/10.1017/S0261444821000409
Al-Hoorie, A. H., Oga-Baldwin, W. L. Q., Hiver, P., & Vitta, J. P. (2022). Self-determination mini-theories in second language learning: A systematic review of three decades of research. Language Teaching Re-search, 29(4), 1603-1638. DOI: https://doi.org/10.1177/13621688221102686
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444-455. DOI: https://doi.org/10.1080/01621459.1996.10476902
Bareinboim, E., & Pearl, J. (2016). Causal inference and the data-fusion problem. Proceedings of the National Academy of Sciences, 113(27), 7345-7352. DOI: https://doi.org/10.1073/pnas.1510507113
Cinelli, C., Forney, A., & Pearl, J. (2022). A crash course in good and bad controls. Sociological Methods & Research. DOI: https://doi.org/10.1177/00491241221099552
Diener, E., Northcott, R., Zyphur, M. J., & West, S. G. (2022). Beyond experiments. Perspectives on Psychological Science, 17(4), 1101-1119. DOI: https://doi.org/10.1177/17456916211037670
Ding, P., & Miratrix, L. W. (2015). To adjust or not to adjust? Sensitivity analysis of M-Bias and Butterfly-Bias. Journal of Causal Inference, 3(1), 41-57. DOI: https://doi.org/10.1515/jci-2013-0021
Dörnyei, Z. (2005). The psychology of the language learner: Individual dif-ferences in second language acquisition. Lawrence Erlbaum.
Dörnyei, Z. (2009). The L2 motivational self system. In Z. Dörnyei & E. Ush-ioda (Eds.), Motivation, language identity and the L2 self (pp. 9-42). Multilingual Matters. DOI: https://doi.org/10.21832/9781847691293-003
Dörnyei, Z., & Ryan, S. (2015). The psychology of the language learner revisited. Routledge. DOI: https://doi.org/10.4324/9781315779553
Elwert, F., & Winship, C. (2014). Endogenous selection bias: The problem of conditioning on a collider variable. Annual Review of Sociology, 40(1), 31-53. DOI: https://doi.org/10.1146/annurev-soc-071913-043455
Fiedler, K. (2011). Voodoo correlations are everywhere – not only in neuro-science. Perspectives on Psychological Science, 6(2), 163-171. DOI: https://doi.org/10.1177/1745691611400237
Gardner, R. C. (2000). Correlation, causation, motivation, and second language acquisition. Canadian Psychology/Psychologie canadienne, 41(1), 10-24. DOI: https://doi.org/10.1037/h0086854
Glynn, A. N., & Kashin, K. (2018). Front-door versus back-door adjustment with unmeasured confounding: Bias formulas for front-door and hybrid adjustments with application to a job training program. Journal of the American Statistical Association, 113(523), 1040-1049. DOI: https://doi.org/10.1080/01621459.2017.1398657
Grosz, M. P., Rohrer, J. M., & Thoemmes, F. (2020). The taboo against explicit causal inference in nonexperimental psychology. Perspectives on Psychological Science, 15(5), 1243-1255. DOI: https://doi.org/10.1177/1745691620921521
Hayduk, L., Cummings, G., Stratkotter, R., Nimmo, M., Grygoryev, K., Dosman, D., Gillespie, M., Pazderka-Robinson, H., & Boadu, K. (2003). Pearl’s d-separation: One more step Into causal thinking. Structural Equation Modeling: A Multidisciplinary Journal, 10(2), 289-311. DOI: https://doi.org/10.1207/S15328007SEM1002_8
Hernán, M. A. (2018). The C-word: Scientific euphemisms do not improve causal inference from observational data. American Journal of Public Health, 108(5), 616-619. DOI: https://doi.org/10.2105/AJPH.2018.304337
Hernán, M. A., Hsu, J., & Healy, B. (2019). A second chance to get causal inference right: A classification of data science tasks. CHANCE, 32(1), 42-49. DOI: https://doi.org/10.1080/09332480.2019.1579578
Hiver, P., & Al-Hoorie, A. H. (2020a). Reexamining the role of vision in second language motivation: A preregistered conceptual replication of You, Dörnyei, and Csizér (2016). Language Learning, 70(1), 48-102. DOI: https://doi.org/10.1111/lang.12371
Hiver, P., & Al-Hoorie, A. H. (2020b). Research methods for complexity theory in applied linguistics. Multilingual Matters. DOI: https://doi.org/10.21832/9781788925754
Hiver, P., Al-Hoorie, A. H., Vitta, J. P., & Wu, J. (2021). Engagement in language learning: A systematic review of 20 years of research methods and definitions. Language Teaching Research, 28(1), 201-230. DOI: https://doi.org/10.1177/13621688211001289
Killingsworth, M. A. (2021). Experienced well-being rises with income, even above $75,000 per year. Proceedings of the National Academy of Sciences, 118(4), e2016976118. DOI: https://doi.org/10.1073/pnas.2016976118
Lamb, M. (2017). The motivational dimension of language teaching. Language Teaching, 50(3), 301-346. DOI: https://doi.org/10.1017/S0261444817000088
MacKinnon, D. P., & Pirlott, A. G. (2015). Statistical approaches for enhancing causal interpretation of the M to Y relation in mediation analysis. Personality and Social Psychology Review, 19(1), 30-43. DOI: https://doi.org/10.1177/1088868314542878
McElreath, R. (2020). Statistical rethinking: A Bayesian course with examples in R and Stan (2nd ed.). CRC Press. DOI: https://doi.org/10.1201/9780429029608
Meyer, J., Lüdtke, O., Schmidt, F. T. C., Fleckenstein, J., Trautwein, U., & Köl-ler, O. (2022). Conscientiousness and cognitive ability as predictors of academic achievement: Evidence of synergistic effects from integrative data analysis. European Journal of Personality. DOI: https://doi.org/10.1177/08902070221127065
Montgomery, J. M., Nyhan, B., & Torres, M. (2018). How conditioning on posttreatment variables can ruin your experiment and what to do about it. American Journal of Political Science, 62(3), 760-775. DOI: https://doi.org/10.1111/ajps.12357
Orben, A., & Lakens, D. (2020). Crud (re)defined. Advances in Methods and Practices in Psychological Science, 3(2), 238-247. DOI: https://doi.org/10.1177/2515245920917961
Pearl, J. (1995). Causal diagrams for empirical research. Biometrika, 82(4), 669-688. DOI: https://doi.org/10.1093/biomet/82.4.669
Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd ed.). Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511803161
Pearl, J., Glymour, M., & Jewell, N. P. (2016). Causal inference in statistics: A primer. Wiley.
Rohrer, J. M. (2018). Thinking clearly about correlations and causation: Graphical causal models for observational data. Advances in Methods and Practices in Psychological Science, 1(1), 27-42. DOI: https://doi.org/10.1177/2515245917745629
Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2), 287-305. DOI: https://doi.org/10.1177/1094428110369842
Stroebe, W. (2020). Student evaluations of teaching encourages poor teaching and contributes to grade inflation: A theoretical and empirical analysis. Basic and Applied Social Psychology, 42(4), 276-294. DOI: https://doi.org/10.1080/01973533.2020.1756817
Tennant, P. W. G., Murray, E. J., Arnold, K. F., Berrie, L., Fox, M. P., Gadd, S. C., Harrison, W. J., Keeble, C., Ranker, L. R., Textor, J., Tomova, G. D., Gilthorpe, M. S., & Ellison, G. T. H. (2020). Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: Review and recommendations. International Journal of Epidemiology, 50(2), 620-632. DOI: https://doi.org/10.1093/ije/dyaa213
Textor, J., van der Zander, B., Gilthorpe, M. S., Liśkiewicz, M., & Ellison, G. T. H. (2016). Robust causal inference using directed acyclic graphs: The R package “dagitty.” International Journal of Epidemiology, 45(6), 1887-1894. DOI: https://doi.org/10.1093/ije/dyw341
Tosh, C., Greengard, P., Goodrich, B., Gelman, A., Vehtari, A., & Hsu, D. (2021). The piranha problem: Large effects swimming in a small pond. arXiv.
Westreich, D., & Greenland, S. (2013). The table 2 fallacy: Presenting and interpreting confounder and modifier coefficients. American Journal of Epidemiology, 177(4), 292-298. DOI: https://doi.org/10.1093/aje/kws412
Wysocki, A. C., Lawson, K. M., & Rhemtulla, M. (2022). Statistical control requires causal justification. Advances in Methods and Practices in Psychological Science, 5(2), 1-19. DOI: https://doi.org/10.1177/25152459221095823
You, C., Dörnyei, Z., & Csizér, K. (2016). Motivation, vision, and gender: A survey of learners of English in China. Language Learning, 66(1), 94-123. DOI: https://doi.org/10.1111/lang.12140
Young, S. S., & Karr, A. (2011). Deming, data and observational studies. Significance, 8(3), 116-120. DOI: https://doi.org/10.1111/j.1740-9713.2011.00506.x
Zhou, S., Hiver, P., & Al-Hoorie, A. H. (2021). Measuring L2 engagement: A review of issues and applications. In P. Hiver, A. H. Al-Hoorie, & S. Mercer (Eds.), Student engagement in the language classroom (pp. 75-98). Multilingual Matters. DOI: https://doi.org/10.2307/jj.22730722.10
License
Copyright (c) 2025 Ali H. Al-Hoorie, Phil Hiver
This work is licensed under a Creative Commons Attribution 4.0 International License.
1.1 The Author hereby warrants that he/she is the owner of all the copyright and other intellectual property rights in the Work and that, within the scope of the present Agreement, the paper does not infringe the legal rights of another person. The owner of the copyright work also warrants that he/she is the sole and original creator thereof and that is not bound by any legal constraints in regard to the use or sale of the work.
1.2. The Publisher warrants that is the owner of the PRESSto platform for open access journals, hereinafter referred to as the PRESSto Platform.
2. The Author grants the Publisher non-exclusive and free of charge license to unlimited use worldwide over an unspecified period of time in the following areas of exploitation:
2.1. production of multiple copies of the Work produced according to the specific application of a given technology, including printing, reproduction of graphics through mechanical or electrical means (reprography) and digital technology;
2.2. marketing authorisation, loan or lease of the original or copies thereof;
2.3. public performance, public performance in the broadcast, video screening, media enhancements as well as broadcasting and rebroadcasting, made available to the public in such a way that members of the public may access the Work from a place and at a time individually chosen by them;
2.4. inclusion of the Work into a collective work (i.e. with a number of contributions);
2.5. inclusion of the Work in the electronic version to be offered on an electronic platform, or any other conceivable introduction of the Work in its electronic version to the Internet;
2.6. dissemination of electronic versions of the Work in its electronic version online, in a collective work or independently;
2.7. making the Work in the electronic version available to the public in such a way that members of the public may access the Work from a place and at a time individually chosen by them, in particular by making it accessible via the Internet, Intranet, Extranet;
2.8. making the Work available according to appropriate license pattern Attribution 4.0 International (CC BY 4.0) as well as another language version of this license or any later version published by Creative Commons.
3. The Author grants the Publisher permission to reproduce a single copy (print or download) and royalty-free use and disposal of rights to compilations of the Work and these compilations.
4. The Author grants the Publisher permission to send metadata files related to the Work, including to commercial and non-commercial journal-indexing databases.
5. The Author represents that, on the basis of the license granted in the present Agreement, the Publisher is entitled and obliged to:
5.1. allow third parties to obtain further licenses (sublicenses) to the Work and to other materials, including derivatives thereof or compilations made, based on or including the Work, whereas the provisions of such sub-licenses will be the same as with the Attribution 4.0 International (CC BY 4.0) Creative Commons sub-license or another language version of this license, or any later version of this license published by Creative Commons;
5.2. make the Work available to the public in such a way that members of the public may access the Work from a place and at a time individually chosen by them, without any technological constraints;
5.3. appropriately inform members of the public to whom the Work is to be made available about sublicenses in such a way as to ensure that all parties are properly informed (appropriate informing messages).
6. Because of the royalty-free provision of services of the Author (resulting from the scope of obligations stipulated in the present Agreement), the Author shall not be entitled to any author’s fee due and payable on the part of the Publisher (no fee or royalty is payable by the Publisher to the Author).
7.1. In the case of third party claims or actions for indemnity against the Publisher owing to any infractions related to any form of infringement of intellectual property rights protection, including copyright infringements, the Author is obliged to take all possible measures necessary to protect against these claims and, when as a result of legal action, the Publisher, or any third party licensed by the Publisher to use the Work, will have to abandon using the Work in its entirety or in part or, following a court ruling in a legal challenge, to pay damages to a third party, whatever the legal basis
7.2. The Author will immediately inform the Publisher about any damage claims related to intellectual property infringements, including the author’s proprietary rights pertaining to a copyrighted work, filed against the Author. of liability, the Author is obliged to redress the damage resulting from claims made by third party, including costs and expenditures incurred in the process.
7.3. To all matters not settled herein provisions of the Polish Civil Code and the Polish Copyright and Related Rights Act shall apply.