Effects of EMI-CLIL on secondary-level students’ English learning: A multilevel meta-analysis
PDF

Keywords

English as a foreign language
English medium instruction
multilevel meta-analysis
secondary level
content and language integrated learning

How to Cite

Lee , J. H., Lee, H., & Lo, Y. Y. (2023). Effects of EMI-CLIL on secondary-level students’ English learning: A multilevel meta-analysis. Studies in Second Language Learning and Teaching, 13(2), 317–345. https://doi.org/10.14746/ssllt.38277

Number of views: 975


Number of downloads: 990

Abstract

This meta-analysis synthesized the effects of the English medium instruction and content and language integrated learning (EMI-CLIL) approach on secondary-level students’ English learning. The dataset included 44 samples (N = 7,434) from 38 primary studies. The results revealed EMI-CLIL’s overall effectiveness for the development of English competence compared to the mainstream condition in the short term (d = 0.73, SE = 0.06, 95% CI [0.61, 0.86]) and longer term (d = 1.01, SE = 0.06, 95% CI [0.88, 1.15]). Additionally, we found that EMI-CLIL’s overall effectiveness was influenced by several moderator variables. Its effectiveness was significantly: (1) higher for learners whose first language (L1) was linguistically related to English; (2) lower for primary studies which confirmed the homogeneity of the EMI-CLIL and comparison groups; (3) lower when studies targeted the productive (rather than receptive or overall) dimension of English learning; and (4) higher when outcome measures focused on vocabulary. Implications for pedagogy and future research are discussed.

https://doi.org/10.14746/ssllt.38277
PDF

References

An, J., Macaro, E., & Childs, A. (2019). Language focused episodes by monolingual teachers in English medium instruction science lessons. Journal of Immersion and Content-Based Language Education, 7(2), 166-191. DOI: https://doi.org/10.1075/jicb.18019.an

Assink, M., & Wibbelink, C. J. (2016). Fitting three-level meta-analytic models in R: A step-by-step tutorial. The Quantitative Methods for Psychology, 12(3), 154-174. DOI: https://doi.org/10.20982/tqmp.12.3.p154

Basterrechea, M., & García Mayo, M. del P. (2014). Dictogloss and the production of the English third person “-s” by CLIL and mainstream EFL learners: A comparative study. International Journal of English Studies, 14(2), 77-98. DOI: https://doi.org/10.6018/j.177321

Bayram, D., Öztürk, R. Ö., & Atay, D. (2019). Reading comprehension and vocabulary size of CLIL and non-CLIL students: A comparative study. Language Teaching and Educational Research, 2(2), 101-113. DOI: https://doi.org/10.35207/later.639337

Beaufils, V., & Tomin, J. (2020, October 30). Stochastic approach to worldwide language classification: The signals and the noise towards long-range exploration. SocArXiv Papers. DOI: https://doi.org/10.31235/osf.io/5swba

Broca, Á. (2016). CLIL and non-CLIL: Differences from the outset. ELT Journal, 70(3), 320-331. DOI: https://doi.org/10.1093/elt/ccw011

Bruton, A. (2013). CLIL: Some of the reasons why . . . and why not. System, 41(3), 587-597. DOI: https://doi.org/10.1016/j.system.2013.07.001

Cambridge ESOL. (2008). Official examination papers from University of Cambridge ESOL examinations. Cambridge University Press.

Canga Alonso, A., & Arribas García, M. (2015). The benefits of CLIL instruction in Spanish students’ productive vocabulary knowledge. Encuentro, 24, 15-31. Retrieved from http://www3.uah.es/encuentrojournal/index.php/encuen-tro/issue/viewIssue/12/24

Castellano-Risco, I., Alejo-González, R., & Piquer-Píriz, A. M. (2020). The development of receptive vocabulary in CLIL vs EFL: Is the learning context the main variable? System, 91, 102263. DOI: https://doi.org/10.1016/j.system.2020.102263

Coyle, D., Hood, P., & Marsh, D. (2010). CLIL: Content and language integrated learning. Cambridge University Press. DOI: https://doi.org/10.1017/9781009024549

Dallinger, S., Jonkmann, K., Hollm, J., & Fiege, C. (2016). The effect of content and language integrated learning on students’ English and history competences: Killing two birds with one stone? Learning and Instruction, 41, 23-31. DOI: https://doi.org/10.1016/j.learninstruc.2015.09.003

Dalton-Puffer, C. (2007). Discourse in content and language integrated learning (CLIL) classrooms. John Benjamins. DOI: https://doi.org/10.1075/lllt.20

Dalton-Puffer, C. (2008). Outcomes and processes in content and language integrated learning (CLIL): Current research from Europe. In W. Delanoy & L. Volkmann (Eds.), Future perspectives for English language teaching (pp. 139-157). Carl Winter.

Dalton-Puffer, C. (2011). Content-and-language integrated learning: From practice to principles? Annual Review of Applied Linguistics, 31, 182-204. DOI: https://doi.org/10.1017/S0267190511000092

Dalton-Puffer, C. (2013). A construct of cognitive discourse functions for conceptualising content-language integration in CLIL and multilingual education. European Journal of Applied Linguistics, 1(2), 216-253. DOI: https://doi.org/10.1515/eujal-2013-0011

Dalton-Puffer, C., Llinares, A., Lorenzo, F., & Nikula, T. (2014). “You can stand under my umbrella”: Immersion, CLIL and bilingual education. A response to Cenoz, Genesee & Gorter (2013). Applied Linguistics, 35(2), 213-218. DOI: https://doi.org/10.1093/applin/amu010

Evans, S. (2017). Language policy in Hong Kong education: A historical overview. European Journal of Language Policy, 9(1), 67-84. DOI: https://doi.org/10.3828/ejlp.2017.5

Feddermann, M., Möller, J., & Baumert, J. (2021). Effects of CLIL on second language learning: Disentangling selection, preparation, and CLIL-effects. Learning and Instruction, 74, 101459. DOI: https://doi.org/10.1016/j.learninstruc.2021.101459

Fernández-Castilla, B., Jamshidi, L., Declercq, L., Beretvas, S. N., Onghena, P., & Van den Noortgate, W. (2020). The application of meta-analytic (multi-level) models with multiple random effects: A systematic review. Behavior Research Methods, 52, 2031-2052. DOI: https://doi.org/10.3758/s13428-020-01373-9

Gallardo del Puerto, F., & Gómez-Lacabex, E. (2017). Oral production outcomes in CLIL: An attempt to manage amount of exposure. European Journal of Applied Linguistics, 5(1), 31-54. ectiveness of content and language integrated learning (CLIL) in foreign language contexts: The assessment of English pronunciation. In Y. Ruiz de Zarobe & R. M. Jiménez Catalán (Eds.), Content and language integrated learning: Evidence from research in Europe (pp. 63-80). Multilingual Matters. DOI: https://doi.org/10.21832/9781847691675-007

Genesee, F., & Lindholm-Leary, K. (2013). Two case studies of content-based language education. Journal of Immersion and Content-Based Language Education, 1(1), 3-33. DOI: https://doi.org/10.1075/jicb.1.1.02gen

Gierlinger, E., & Wagner, T. (2016). The more the merrier: Revisiting CLIL-based vocabulary growth in secondary education. LACLIL, 9(1), 37-63. DOI: https://doi.org/10.5294/laclil.2016.9.1.3

Goris, J., Denessen, E., & Verhoeven, L. (2013). Effects of the content and language integrated learning approach to EFL teaching: A comparative study. Written Language & Literacy, 16(2), 186-207. DOI: https://doi.org/10.1075/wll.16.2.03gor

Goris, J., Denessen, E., & Verhoeven, L. (2019). Effects of content and language integrated learning in Europe: A systematic review of longitudinal experimental studies. European Educational Research Journal, 18(6), 675-698. DOI: https://doi.org/10.1177/1474904119872426

Graham, K. M., Choi, Y., Davoodi, A., Razmeh, S., & Dixon, L. Q. (2018). Language and content outcomes of CLIL and EMI: A systematic review. LACLIL, 11(1), 19-37. DOI: https://doi.org/10.5294/laclil.2018.11.1.2

Gutiérrez-Mangado, M. J., & Martínez-Adrián, M. (2018). CLIL at the linguistic interfaces. Journal of Immersion and Content-Based Language Education, 6(1), 85-112. DOI: https://doi.org/10.1075/jicb.17002.gut

Gutiérrez Ortiz, M. (2020). Assessing the development of second language syntax in content and language integrated learning. Revista De Lenguas Para Fines Específicos, 26(2), 111-130. Retrieved from https://ojsspdc.ulpgc.es/ojs/index.php/LFE/article/view/1333 DOI: https://doi.org/10.20420/rlfe.2020.361

Hamidavi, N., Shekaramiz, M., & Gorjian, B. (2016). The effect of CLIL method on teaching reading comprehension to junior high school students. Modern Journal of Language Teaching Methods, 6(9), 64-73. Retrieved from http://mjltm.org/article-1-65-en.html DOI: https://doi.org/10.25518/0037-9565.6361

Hedges, L. V. (1981). Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 6(2), 107-128. DOI: https://doi.org/10.3102/10769986006002107

Hendrikx, I., & Van Goethem, K. (2020). Receptive knowledge of intensifying adjectival compounds: Belgian French-speaking CLIL and non-CLIL learners of Dutch and English. International Journal of Bilingual Education and Bilingualism. Advance Publication Online. DOI: https://doi.org/10.1080/13670050.2020.1720592

Hu, J., & Gao, X. (2021). Understanding subject teachers’ language-related pedagogical practices in content and language integrated learning classrooms. Language Awareness, 30(1), 42-61. DOI: https://doi.org/10.1080/09658416.2020.1768265

Jeon, E. H., & Yamashita, J. (2014). L2 reading comprehension and its correlates: A meta-analysis. Language Learning, 64(1), 160-212. DOI: https://doi.org/10.1111/lang.12034

Jexenflicker, S., & Dalton-Puffer, C. (2010). The CLIL differential: Comparing the writing of CLIL and non-CLIL students in higher colleges of technology. In C. Dalton-Puffer, T. Nikula, & U. Smit (Eds.), Language use and language learning in CLIL classrooms (pp. 169-189). John Benjamins. DOI: https://doi.org/10.1075/aals.7.09jex

Jiménez Catalán, R. M., & Agustín Llach, M. P. (2017). CLIL or time? Lexical profiles of CLIL and non-CLIL EFL learners. System, 66, 87-99. DOI: https://doi.org/10.1016/j.system.2017.03.016

Johnson, R. K., & Swain, M. (1994). From core to content: Bridging the L2 proficiency gap in late immersion. Language and Education, 8(4), 211-229. DOI: https://doi.org/10.1080/09500789409541392

Juan-Garau, M., Prieto-Arranz, J. I., & Salazar-Noguera, J. (2015). Lexico-grammatical development in secondary education CLIL learners. In M. Juan-Garau & J. Salazar-Noguera (Eds.), Content-based language learning in multilingual educational environments (pp. 179-195). Springer. DOI: https://doi.org/10.1007/978-3-319-11496-5_11

Krashen, S. (1982). Principles and practice in second language acquisition. Prentice Hall. DOI: https://doi.org/10.1111/j.1467-971X.1982.tb00476.x

Lado, R. (1957). Linguistics across cultures: Applied linguistics for language teachers. University of Michigan Press.

Lahuerta, A. (2017). Syntactic complexity in secondary-level English writing: Differences among writers enrolled on bilingual and non-bilingual pro-grammes. Porta Linguarum, 28, 67-80. Retrieved from https://www.ugr.es/~portalin/articulos/PLnumero28/5%20Lahuerta.pdf

Lahuerta, A. (2020). Analysis of accuracy in the writing of EFL students enrolled on CLIL and non-CLIL programmes: The impact of grade and gen-der. The Language Learning Journal, 48(2), 121-132. DOI: https://doi.org/10.1080/09571736.2017.1303745

Lasagabaster, D. (2008). Foreign language competence in content and language integrated courses. The Open Applied Linguistics Journal, 1, 30-41. DOI: https://doi.org/10.2174/1874913500801010030

Lee, H., Warschauer, M., & Lee, J. H. (2019). The effects of corpus use on second language vocabulary learning: A multilevel meta-analysis. Applied Linguistics, 40(5), 721-753. DOI: https://doi.org/10.1093/applin/amy012

Lee, J. (2020). Assessing the effects of CLIL on Korean high school students’ writing. Linguistic Research, 37, 89-112.

Lee, J. H., & Lee, H. (2022). Teachers’ verbal lexical explanation for L2 vocabulary learning: A meta-analysis. Language Learning, 72(2), 576-612. DOI: https://doi.org/10.1111/lang.12493

Lin, L. H. F., & Morrison, B. (2010). The impact of the medium of instruction in Hong Kong secondary schools on tertiary students’ vocabulary. Journal of English for Academic Purposes, 9(4), 255-266. DOI: https://doi.org/10.1016/j.jeap.2010.09.002

Lo, Y. Y., & Lo, E. S. C. (2014). A meta-analysis of the effectiveness of English-medium education in Hong Kong. Review of Educational Research, 84(1), 47-73. DOI: https://doi.org/10.3102/0034654313499615

Lo, Y. Y., & Macaro, E. (2012). The medium of instruction and classroom interaction: Evidence from Hong Kong secondary schools. International Journal of Bilingual Education and Bilingualism, 15(1), 29-52. DOI: https://doi.org/10.1080/13670050.2011.588307

Lo, Y. Y., & Murphy, V. A. (2010). Vocabulary knowledge and growth in immersion and regular language-learning programmes in Hong Kong. Language and Education, 24(3), 215-238. DOI: https://doi.org/10.1080/09500780903576125

Long, M. H. (1996). The role of the linguistic environment in second language acquisition. In W. Ritchie & T. Bhatia (Eds.), Handbook of second language acquisition (pp. 413-468). Academic Press. DOI: https://doi.org/10.1016/B978-012589042-7/50015-3

Lorenzo, F., Casal, S., & Moore, P. (2010). The effects of content and language integrated learning in European education: Key findings from the Andalusian bilingual sections evaluation project. Applied Linguistics, 31(3), 418-442. DOI: https://doi.org/10.1093/applin/amp041

Lyster, R., & Ruiz de Zarobe, Y. (2018). Introduction: Instructional practices and teacher development in CLIL and immersion school settings. International Journal of Bilingual Education and Bilingualism, 21(3), 273-274. DOI: https://doi.org/10.1080/13670050.2017.1383353

Macaro, E. (2018). English medium instruction. Oxford University Press. DOI: https://doi.org/10.30687/978-88-6969-227-7/001

Manzano Vázquez, B. (2014). Lexical transfer in the written production of a CLIL group and a non-CLIL group. International Journal of English Studies, 14(2), 57-76. DOI: https://doi.org/10.6018/j.166251

Marsh, D. (Ed.) (2002). CLIL/EMILE. The European dimension. Actions, trends, and foresight potential. University of Jyväskylä.

Martínez-Adrián, M., & Gutiérrez-Mangado, M. J. (2009). The acquisition of English syntax by CLIL learners in the Basque country. In Y. Ruiz de Za-robe & R. M. Jiménez Catalán (Eds.), Content and language integrated learning: Evidence from research in Europe (pp. 176-196). Multilingual Matters. DOI: https://doi.org/10.21832/9781847691675-013

Martínez-Adrián, M., & Gutiérrez-Mangado, M. J. (2015). L1 use, lexical richness, accuracy and complexity in CLIL and non-CLIL learners of English. Atlantis, Journal of the Spanish Association for Anglo-American Studies, 37(2), 175-197. https://www.atlantisjournal.org/index.php/atlantis/article/view/273

Martínez Agudo, J. (2019). Which instructional programme (EFL or CLIL) results in better oral communicative competence? Updated empirical evidence from a monolingual context. Linguistics and Education, 51, 69-78. DOI: https://doi.org/10.1016/j.linged.2019.04.008

Martínez Agudo, J. (2020). The impact of CLIL on English language competence in a monolingual context: A longitudinal perspective. The Language Learning Journal, 48(1), 36-47. DOI: https://doi.org/10.1080/09571736.2019.1610030

Merino, J. A., & Lasagabaster, D. (2018). The effect of content and language integrated learning programmes’ intensity on English proficiency: A longitudinal study. International Journal of Applied Linguistics, 28(1), 18-30. DOI: https://doi.org/10.1111/ijal.12177

Morton, T., & Llinares, A. (2017). Content and language integrated learning: Type of programme or pedagogical model? In A. Llinares & T. Morton (Eds.), Applied linguistics perspectives on CLIL (pp. 1-16). John Benja-mins. DOI: https://doi.org/10.1075/lllt.47.01mor

Murphy, V. A., Arndt, H., Briggs J. G., Chalmers, H., Macaro, E., Rose, H., Vanderplank, R., & Woore, R. (2020). Foreign language learning and its impact on wider academic outcomes: A rapid evidence assessment. Education Endowment Foundation. https://educationendowmentfoundation.org.uk/public/files/Publica-tions/Foreign_language_learning_and_its_impact_on_wider_academic_outcomes_-_A_rapid_evidence_assessment.pdf

Nunan, D. (2011). Teaching English to young learners. Anaheim University Press.

Olsson, E. (2015). Progress in English academic vocabulary use in writing among CLIL and non-CLIL students in Sweden. Moderna Språk, 109(2), 51-74. https://ojs.ub.gu.se/index.php/modernasprak/article/view/3261 DOI: https://doi.org/10.58221/mosp.v109i2.7927

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., . . . & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372(71), 1-9.

Pérez-Cañado, M. L. (2012). CLIL research in Europe: Past, present, and future. International Journal of Bilingual Education and Bilingualism, 15(3), 315-341. DOI: https://doi.org/10.1080/13670050.2011.630064

Pérez-Cañado, M. L. (2016). From the CLIL craze to the CLIL conundrum: Addressing the current CLIL controversy. Bellaterra Journal of Teaching & Learning Language & Literature, 9(1), 9-31. DOI: https://doi.org/10.5565/rev/jtl3.667

Pérez Cañado, M. L., & Lancaster, N. K. (2017). The effects of CLIL on oral comprehension and production: A longitudinal case study. Language, Culture and Curriculum, 30(3), 300-316. DOI: https://doi.org/10.1080/07908318.2017.1338717

Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878-912. DOI: https://doi.org/10.1111/lang.12079

Prieto-Arranz, J. I., Rallo Fabra, L., Calafat-Ripoll, C., & Catrain-González, M. (2015). Testing progress on receptive skills in CLIL and non-CLIL contexts. In M. Juan-Garau & J. Salazar-Noguera (Eds.), Content-based language learning in multilingual educational environments (pp. 123-137). Springer. DOI: https://doi.org/10.1007/978-3-319-11496-5_8

Rallo Fabra, L., & Jacob, K. (2015). Does CLIL enhance oral skills? Fluency and pronunciation errors by Spanish-Catalan learners of English. In M. Juan-Garau & J. Salazar-Noguera (Eds.), Content-based language learning in multilingual educational environments (pp. 163-177). Springer. DOI: https://doi.org/10.1007/978-3-319-11496-5_10

Rallo Fabra, L., & Juan-Garau, M. (2011). Assessing EFL pronunciation in a semi-immersion setting: The effects of CLIL instruction on Spanish-Catalan learners perceived comprehensibility and accentedness. Poznań Studies in Contempo-rary Linguistics, 47(1), 96-108. DOI: https://doi.org/10.2478/psicl-2011-0008

Rose, H., Macaro, E., Sahan, K., Aizawa, I., Zhou, S., & Wei, M. (2021). Defining English medium instruction: Striving for comparative equivalence. Language Teaching. DOI: https://doi.org/10.1017/S0261444821000483

Ruiz de Zarobe, Y. (2011). Which language competencies benefit from CLIL? An insight into applied linguistics research. In Y. Ruiz de Zarobe, J. M. Sierra, & F. Gallardo del Puerto (Eds.), Content and foreign language integrated learning: Contributions to multilingualism in European contexts (pp. 129-153). Peter Lang. DOI: https://doi.org/10.3726/978-3-0351-0171-3

Rumlich, D. (2017). CLIL theory and empirical reality: Two sides of the same coin? Journal of Immersion and Content-Based Language Education, 5(1), 110-134. DOI: https://doi.org/10.1075/jicb.5.1.05rum

Salili, F., & Lai, M. K. (2003). Learning and motivation of Chinese students in Hong Kong: A longitudinal study of contextual influences on students’ achievement orientation and performance. Psychology in the Schools, 40(1), 51-70. DOI: https://doi.org/10.1002/pits.10069

Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the Vocabulary Levels Test. Language Testing, 18(1), 55-88. DOI: https://doi.org/10.1177/026553220101800103

Stockwell, R., Bowen, J., & Martin, J. (1965). The grammatical structures of English and Spanish. University of Chicago Press.

Swain, M. (1995). Three functions of output in second language learning. In G. Cook & B. Seidlhofer (Eds.), Principle and practice in applied linguistics: Studies in honour of H. G. Widdowson (pp. 125-144). Oxford University Press.

Tabachnik, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Allyn and Bacon.

Van Mensel, L., Bulon, A., Hendrikx, I., Meunier, F., & Van Goethem, K. (2020). Effects of input on L2 writing in English and Dutch: CLIL and non-CLIL learners in French-speaking Belgium. Journal of Immersion and Content-Based Language Education, 8(2), 173-199. DOI: https://doi.org/10.1075/jicb.18034.van

Verspoor, M., de Bot, K., & Xu, X. (2015). The effects of English bilingual education in the Netherlands. Journal of Immersion and Content-Based Language Education, 3(1), 4-27. DOI: https://doi.org/10.1075/jicb.3.1.01ver

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1-48. DOI: https://doi.org/10.18637/jss.v036.i03

Viechtbauer, W., & Cheung, M. W. L. (2010). Outlier and influence diagnostics for meta‐analysis. Research Synthesis Methods, 1(2), 112-125. DOI: https://doi.org/10.1002/jrsm.11