The effects of CLIL on mathematical content learning: A longitudinal study

Previous research has shown that content and language integrated learning (CLIL), an educational approach that offers content courses through more than one educational language, increases metalinguistic awareness. This improved insight into language structures is supposed to extend beyond the linguistic domain. In the present study, the question whether pupils who learn in a CLIL environment outperform their traditionally schooled peers in mathematics is investigated. In total, 107 pupils entered the study. All participants were in the first year of secondary education at a school in Ostend, in Flanders, the Dutch-speaking part of Belgium. Thirty-five pupils followed CLIL education in a foreign language (French) and 72 followed traditional education that was given in the native language (Dutch). All participants were tested using a mathematical test at the beginning of the year, after three months, and Jill Surmont, Esli Struys, Maurits Van Den Noort, Piet Van De Craen 320 after ten months. The first measurement of the mathematical scores showed that the two groups did not differ. In accordance with our hypothesis, the CLIL group scored higher than the non-CLIL group after ten months. Surprisingly, an effect was also found after three months. To conclude, CLIL appears to have a positive impact on the mathematical performance of pupils even after a short period of time.


Introduction
Content and language integrated learning (CLIL) refers to a teaching approach where subjects, or parts of subjects, are taught in a foreign language with dualfocused aims, namely the learning of content and the simultaneous learning of a foreign language (Marsh, 1994). CLIL combines a number of pedagogies that have proven to work well and, as a result, it has become a well-established part of education systems across Europe (e.g., Banegas, 2012;Bonnet, 2012;Nikula, Dalton-Puffer, & García, 2013;Pérez-Cañado, 2012). There are several reasons why CLIL is different from traditional language learning approaches.

Differences between CLIL and traditional language education
First of all, CLIL is dual-focused and has both content and language goals, which creates a continuum with both goals at one end, without any specification of importance of one over the other (Coyle, 2006). The target language is used as a vehicle for communication and is therefore not the ultimate goal. In other words, CLIL is a task-based language learning approach by which language is learned through usage of the language while discussing content matter. This creates a meaningful learning environment in which pupils have an immediate need to use the target language. Feedback is given through scaffolding, ensuring the pupils are constantly in what Vygotsky (1978) called "the zone of proximal development." This means that the pupil is able to fulfill tasks because the teacher is helping, whereas alone the pupil would not be able to do so.
A second difference from traditional language learning approaches is the way CLIL is pupil-centered rather than teacher-centered (González & Barbero, 2013). In many Western societies, the dominant teaching model is based on what Freire (1972) calls the "banking model." This means that the teacher has the role of the expert and deposits information and skills into the memory bank of the pupil. This banking model creates a teacher-controlled and teacher-led learning process. CLIL, on the other hand, has a more social-constructivist approach, meaning that the emphasis is on the student experience and on the encouragement of active student learning (Cummins, 2005;Marsh, 2012). The focus of CLIL is therefore on interactive, mediated, and student-led learning, which requires social interaction between teachers and students.
The third difference focuses on the resemblance of the language learning process in CLIL to the natural cognitive skill acquisition. There are two types of processes that interact during a learning process, namely implicit learning and explicit learning (Sun, Mathews, & Lane, 2007). Surmont, Van de Craen, Struys, and Somers (2014) compared this interaction with the way a child learns how to tie his/her laces: He/she needs to practise on his/her own, but when a parent or sibling provides explicit information on what to do with the laces, the child will figure out much faster how it is done. Research by Bialystok and Barac (2012) showed that when second languages are learned implicitly, the brain responds the same way as it does when using the mother tongue. This does not imply that it is better, but it proves that the learning context has a major influence on how something is processed. This can, for instance, explain previous neurolinguistic results in which bilingual brains were found to be more efficient than monolingual brains (Mondt, Struys, Balériaux, & Van de Craen, 2008).
In traditional education, there are various reasons why implicit (language) learning is not used often enough. Sun, Mathews, and Lane (2007) described it as follows: "Most educational settings focus on teaching conceptual (explicit) knowledge rather than setting up an opportunity for gaining substantial experiential (mostly implicit) knowledge" (p. 1). In traditional language education, the emphasis is all too often on grammatical correctness and often teachers are unable to give their students sufficient time to use and to practise using the target language due to time constraints. In CLIL education, on the other hand, students learn the language in a much more implicit way, as they have to use it to understand and communicate about the content of the course. They basically learn the language "along the way" (Surmont et al., 2014). Due to a much higher implicit learning aspect in CLIL classes, children benefit more from the interaction between implicit and explicit education. It is of course impossible to solely offer implicit or explicit learning. Rather, learning should be seen as a continuum in which both sides can never be fully tuned out (Sun et al., 2007), thus implying that traditional language courses are not obsolete in CLIL programs, because they provide the necessary, more explicit focus on language. Francis (2004) even suggests that it is not bilingualism per se that creates increased metalinguistic awareness (see further) but that the learning context in which the pupils are situated has the largest influence. The combination of a CLIL class and a traditional language class creates a learning environment in which the brain thrives, an environment where explicit and implicit learning can take place (Ellis, 2008).
A final difference between CLIL and traditional foreign language classes is the possible increased development of metalinguistic awareness. Metalinguistic awareness can be described as the ability to "reflect on and manipulate the structural features of languages" (Nagy & Anderson, 1998, p. 155). It permits reasoning and application of logic with language (Ter Kuile, Veldhuis, Van Veen, & Wicherts, 2011). Usage of metalinguistic awareness usually requires a momentary shift of attention from content to form (Edwards & Kirkpatrick, 1999). In normal language use, metalinguistic awareness is hardly relevant as conveying the message is normally considered as more important than the linguistic structures used to convey it (Nagy & Anderson, 1995). Speakers do rely on it when they have to access deeper linguistic knowledge when, for example, correcting certain slips of the tongue (Fromkin, 1980) or deciphering an atypical form of speech or dialect (Kemper & Vernooy, 1993), when making puns or word jokes (Horgan, 1981), when linguistic ambiguities have to be resolved, when grammaticality or appropriateness has to be judged, or when decisions have to be made on deeper meanings or intentions based on word choices or paralinguistic cues (Edwards & Kirkpatrick, 1999).

Increased metalinguistic awareness
It is clear that the ability to reflect on language structures requires a higher level of abstract thinking and understanding. Research reports that metalinguistic skills (Bialystok, Peets, & Moreno, 2014;Hermanto, Moreno, & Bialystok, 2012) are an important factor that distinguishes bilingual from monolingual language users and learners (Bialystok, 2001;Jessner, 2006). It has been found that bilingual children have an increased metalinguistic awareness compared to monolingual children (Ransdell, Barbier, & Niit, 2006;Whitehurst & Lonigan, 1998).
The influence of an increased metalinguistic awareness may extend the linguistic domain. For instance, studies have shown that bilingual pupils have an advantage in mathematics when they are highly competent in both languages, compared to their monolingual peers (Clarkson, 1992(Clarkson, , 2007Dawe, 1983). A possible explanation for this is the increased metalinguistic awareness of high ability bilingual students (Clarkson, 2007). Those bilinguals can compare the language structures of the languages they speak and come to a higher understanding of how each language structure functions on its own, whereas monolinguals do not (or rarely) question the way their language system is built up. These metalinguistic abilities seem to be important factors in mathematical performance (e.g., the better metalinguistic skills of bilingual students allow them, for instance, to self-correct when solving problems; moreover, bilingual students are more confident in their approach to solving difficult problems compared to monolingual students) (Clarkson, 2007).
It can be argued that mathematics is a "language" (Arianrhod, 2005;Goldhaber, 2006), because it has its own set of rules, and with a finite number of symbols an infinite number of utterances can be created. Moreover, as in other languages, students need to master this language in order to read, understand, write down, and discuss (mathematical) ideas (Thompson & Rubenstein, 2000). Orr (1987) even claimed that most difficulties that students have with mathematics are rooted in language. Orr noticed that the structure of the so called Black English, that is, English spoken by many black students, causes their problems in mathematics. In her work, she illustrates the concepts which these students do not understand and which underlie mathematics, namely prepositions, conjunctions, and relative pronouns. If we accept that mathematics is a language (Usiskin, 1996), then it might be possible that bilinguals may acquire the language of mathematics faster than their monolingual peers and may have a better insight into it.
When it comes to CLIL, it is likely that pupils will benefit from an increased metalinguistic awareness, as the extra language provided in a meaningful context will create a deeper insight into language structures. Ter Kuile and colleagues (2011) have shown that pupils in CLIL programs have a better insight into language structures, even with respect to an unknown language. When looking at the maths performance of CLIL pupils compared to their traditionally schooled peers, research indicates that CLIL pupils have an advantage (e.g., Murray, 2010;Valladoloid, 1991;Van de Craen, Ceuleers, & Mondt, 2007). This not only confirms that CLIL has a positive influence on content knowledge but also that it stimulates cognitive development. The influence it seems to have on maths performance could be an indication that CLIL stimulates metalinguistic skills in such a way that the understanding of the language of maths (and/or science, as seen in Jäppinen, 2005) increases.

Research question and hypotheses
The question in the present study is if monolingual pupils whose mathematics is delivered via CLIL outperform their traditionally schooled peers whose mathematics is taught in their mother tongue. The first hypothesis of this study is that pupils who learn in a CLIL environment are likely to outperform their traditionally schooled peers on mathematical tests, because CLIL is assumed to create an increased metalinguistic awareness (Mehisto & Morsh, 2011, p. 36) and, as a result, their understanding of the "language of math" (e.g., Arianrhod, 2005;Goldhaber, 2006) increases (Jäppinen, 2005). However, as the learning process cannot be rushed, and a small change in the learning environment is unlikely to show its effect on a cognitive level immediately (Goswami, 2008), the second hypothesis is that differences between the CLIL group and the non-CLIL group will not be apparent after only a short period of time but will require a longer period of study, after which they will become visible.

Participants
The participants of this study were 107 pupils from the first year of secondary education in a school in Ostend, in Flanders, the Dutch-speaking part of Belgium. Thirty-five pupils had chosen to follow the CLIL course, while 72 had preferred the traditional curriculum. For ethical and legal reasons, it was not possible to randomly divide the 107 pupils in a CLIL group and a non-CLIL group; however, this selection procedure might have resulted in the so-called "creaming effect" (Bredenbröker, 2002, p. 146;Rumlich, 2013, p. 185) because it cannot be excluded that the two groups are different with respect to factors such as motivation, parental support, and so forth (for a detailed discussion, see Jäkel, 2015). The CLIL course lasted one hour per week and the pupils of the CLIL group followed the CLIL course throughout the whole school year (i.e., 10 months). The pupils were taught in French with dual-focused aims, namely the learning of the subject (mathematics; content) and the simultaneous learning of the language (French; foreign language). The CLIL teaching was pupil-centered and interactive, there was a significant role for implicit learning, and the teacher helped the pupils in order to succeed. Aside from the fact that the CLIL group received their maths instruction through French, there were no significant differences in the teaching strategies or the curriculum content between the two groups, and all other instruction was delivered in their native Dutch. Furthermore, there were no significant differences in the teaching strategies used for all other subjects of the curriculum between the CLIL and the non-CLIL pupils. All participants were being raised in Dutch only, and, although they all had had two years of introduction to French in primary education, their French proficiency level was so low that the secondary French teachers informed the researchers that they had to start from the very beginning. For this reason, the pupils in this research are considered to be at a very early stage of second language acquisition. Participants from both groups did not differ (p > .05) in age in years (the CLIL group: M = 12.23, SD = .39; the non-CLIL group: M = 12.36, SD = .45). Gender distributions were equal for both groups (Pearson Chi square = .47): In the CLIL group, there were 19 females and 16 males; in the non-CLIL group there were 34 females and 38 males. Lastly, pupils in both groups had the same socio-economic status (SES) as measured by the educational attainment of both parents (Pearson Chi square for mother = .31; Pearson Chi square for father = .13).

Materials
Three versions of a new mathematical test called the Mathematical Assessment Test-Help (MATH) have been developed by the researchers. At the time, there were no official standardized tests in Flanders (Belgium) to test mathematical knowledge in secondary education. Each of the three versions of the MATH consisted of the same kind of mathematical exercises and were presented in the native language of the pupils (Dutch). The test consisted of four different parts: I. Mental calculation, II. Numeracy, III. Mathematical applications and insight, IV. Geometry, measurement and arithmetic calculation; and it included both theoretical as well as practical exercises covering arithmetic and geometry (for the three versions of the mathematical test, see the appendices of the online supporting information at http://ssllt.amu.edu.pl/download/docs/SSLLT%206(2)%20Surmont, %20Struys,%20Van%20Den%20Noort,%20Van%20De%20Craen%20Appendices.pdf). The correction of all tests was conducted by one corrector who was not familiar with the aims of the study, and the maximum score on the MATH was 100.

Procedure
At the beginning of the school year, all pupils completed the first of the mathematical tests based on the official standards that have to be met in order to start secondary education. At that point, no CLIL education had been introduced to the pupils (hereafter we refer to this time point as T = 0 throughout the paper). After three months, all pupils completed a second mathematical test (T = 1), and after 10 months all pupils completed a final mathematical test (T = 2). The study was approved and conducted according to the ethical guidelines of the Vrije Universiteit Brussel, Belgium and the Declaration of Helsinki (World Medical Organization, 1996).

Statistics
SPSS 22.0 (IBM Corp., 2013) was used for all statistical analyses. A one-way repeated measures analysis of variance (ANOVA) was conducted in order to analyse the scores of the CLIL group versus the non-CLIL group on the mathematical test over the three different time points. A Greenhouse-Geisser correction and a Huynh-Feldt correction were used in order to elicit a more accurate and conservative significance value (Abdi, 2010). In addition, ANOVAs were conducted to analyse the between-group results of the three different time points (T0, T1, and T2) separately and paired samples t tests were conducted in order to analyse the withingroup results separately (T0 compared to T1 and T0 compared to T2 respectively). Finally, ANOVAs were conducted to analyse the between-group difference scores (T1-T0 and T2-T0) separately and paired samples t tests were conducted in order to analyse the within-group difference scores (T1-T0 compared to T2-T0) for the CLIL group and the non-CLIL group separately. Table 1 shows the mean scores and standard deviations of the CLIL group versus the non-CLIL group on the mathematical test at the beginning of the study (T0), after three months (T1), and after ten months (T2). At the beginning of the education year (T0), the ANOVA revealed that there was no significant difference between the CLIL group and the non-CLIL group on the mathematical test (F(2,105) = .28, p = .60, partial η 2 = .003), meaning that the pupils of the CLIL group and the non-CLIL group performed equally well on the mathematical test at the beginning of the study. As can be seen in Table 1 and Figure 1, the oneway repeated measures ANOVA results showed that after three months the CLIL group scored significantly better than the non-CLIL group (F(2,105) = 3.00, p ≤ .05, partial η 2 = .055). This difference remained significant from the mean score at the beginning (T0) on the delayed posttest (T3) (F(2,105) = 3.42, p < .05, partial η 2 = .032), meaning that the pupils of the CLIL group performed better than the pupils of the non-CLIL group after ten months of CLIL education in comparison with the traditional education (see Figure 1). Notes. a Mean score of the CLIL group is significantly different (p < .05) from the mean score of the non-CLIL group; b mean score after three months (T1) is significantly different (p < .05) from the mean score at the beginning (T0); c mean score after ten months (T2) is significantly different (p < .05) from the mean score at the beginning (T0).

Figure 1
Mean scores of the CLIL group (n = 35) versus the non-CLIL group (n = 72) on the mathematical test at the beginning of the study (T0), after three months (T1), and after ten months (T2) Examining the within-group results more closely, both groups showed a significant improvement of the mathematical test scores after three months (T1) in comparison with the scores at the beginning of the study (T0) (the CLIL group: t = 5.09, p < .001; the non-CLIL group: t = 3.17, p < .01). Moreover, both groups showed a significant improvement of the mathematical test scores after ten months (T2) in comparison with the scores at the beginning of the study (T0) (the CLIL group: t = 4.34, p < .001; the non-CLIL group: t = 3.42, p < .01). Overall, the within-group results showed that both groups scored higher on the mathematical test after following ten months of CLIL or non-CLIL education.

Between-group and within-group difference scores
As can be seen in Table 2, the analysis of the between-group difference scores gives a better insight into the learning curves. Between T0 and T1 (after three months) a significant difference in progression between the CLIL group and the non-CLIL group was found (F(2,105) = 5.48, p < .05, partial η 2 = .050), with more progression visible for the CLIL group within the first three months after starting CLIL education in comparison with the non-CLIL group. In contrast, between T1 and T2 (between three and ten months) there was no significant difference in progression between the CLIL group and the non-CLIL group (F(2,105) = 1.81, p = .18, partial η 2 = .017): Both groups showed the same learning curves and remained relatively stable (the CLIL group: t = -1.18, p = .24; the non-CLIL group: t = .66, p = .51). Finally, overall, CLIL pupils' progress was significantly better over the ten months when compared to their traditionally schooled peers (F(2,105) = 3.42, p < .05, partial η 2 = .003).

Discussion
In this research, a longitudinal study was performed to find out what the influence of CLIL would be on math performance. There is a growing body of research that suggests that CLIL influences cognitive development, which results in better outcomes in science and/or mathematics (Jäppinen, 2005;Murray, 2010;Van de Craen et al., 2007). The first hypothesis of this study was that pupils who learn in a CLIL environment are likely to outperform their traditionally schooled peers on mathematical tests, because CLIL creates an increased metalinguistic awareness. The second hypothesis was that it was unlikely that CLIL students would outperform their traditionally schooled peers after a short period of time (three months), as the researchers expected that the influence of CLIL would only be noticeable on cognitive tasks after a longer period of time (at least ten months). Here, it is important to note that before the commencement of any CLIL education (at T = 0), a pretest showed that the CLIL group and the non-CLIL group were at the same level. After three months (T = 1) and after ten months (T = 2), second and third tests were undertaken to see how the students' abilities had evolved. Repeated measures analysis showed that the CLIL group improved significantly better than the non-CLIL group over time. On the third test both groups remained at the approximately the same level they had reached on the second test.
Our between-group data is clear proof for the first hypothesis of our study. Pupils who learn in a CLIL environment indeed outperform their traditionally schooled peers on a mathematics test. However, the within-group results reveal that both groups showed an improvement in mathematical performance. The second hypothesis, that the CLIL influence on cognitive level would only be noticed after a longer period of time, was proven incorrect because, in contrast to our expectations, even after three months the positive effect of CLIL teaching was clearly noticeable.
It was argued that if bilinguals had an increased metalinguistic awareness, which gave them a better understanding and insight into (language) structures (Ter Kuile et al., 2011), the same could be said about pupils who are at a very early stage of second language acquisition and are learning a second language through CLIL. This would not only give them an advantage in understanding the language structures of the "new" language and their mother tongue (Lorenzo et al., 2010;Ter Kuile et al., 2011), but it would also help them to understand the language of mathematics (Arianrhod, 2005;Goldhaber, 2006). This claim is backed up with previous research into the influence of CLIL on mathematics (e.g., Murray, 2010;Van de Craen et al., 2007).

Implications
The present results confirm the above-mentioned studies on CLIL and mathematics and indicate a number of things. First of all, these results add to the research body that indicates that CLIL not only positively affects language learning but also content acquisition. CLIL stimulates pupils in such a way that more than one aspect of the learning process is influenced (Surmont et al., 2014).
Secondly, it is very surprising to find these positive results already after only three months of CLIL education. It was hypothesized that more time should be given to pupils, at least 10 months upwards, before effects would be found. It would appear that the combination of mathematics and CLIL is so effective that the cognitive stimulation is enormous; however, caution is needed because other possible explaining factors, such as increased motivation, higher self-confidence, and more parental support in the CLIL group were not taken into account (for a detailed discussion of these factors, see Jäkel, 2015). It is recommended that future research look into the impact of these factors on student outcomes.
Thirdly, these data indicate that CLIL possibly influences pupils' cognitive development, and more specifically, their metalinguistic awareness. This increased metalinguistic awareness could lead to a better understanding of and insight into the abstract language of math. Research by Bialystok and Barac (2012) indicates that even in moderate bilinguals an increased metalinguistic awareness can be found, meaning that pupils do not have to be highly proficient in both languages to profit from an increased metalinguistic awareness. In this study pupils were still at a very low level of French proficiency and the results seem to indicate that they indeed already profit from an increased metalinguistic awareness. However, this conclusion has to be approached with caution as an increased metalinguistic awareness has not been directly proven here. Future research should therefore look into the impact of CLIL on metalinguistic awareness and the question to what extent the advantage (if any) impacts other competences such as math achievement. A longitudinal setup in which language proficiency, math achievement and metalinguistic awareness are measured and where variables such as motivation are taken into account should give a clearer insight into this matter. It would also be very interesting to see how different age groups respond (e.g., Jäppinen, 2005) and how much the unique linguistic environment of a specific school (i.e., the native and foreign language background of the teachers, of the other pupils, the location of the school, etc.) affects the (possible) effect CLIL has on the cognitive development of the pupils.
A final consequence of the present results, and of other results found in the literature (e.g., Adanur, Yagiz, & Işik, 2004;Jäppinen, 2005), is the fact that policy makers should be aware of the link between languages and mathematics. If the European Union intends to improve math performance, it should create bridges between language learning and mathematics (and science for that matter). The present results indicate that a combination of language and content has a positive effect on cognitive development. This indicates that teachers should cooperate with each other and that the schism between language teachers and math/science teachers should be overcome. Cooperative learning, both by pupils and teachers, is the way forward. Knowledge is built on previous learning (OECD, 2007), and enabling peer groups to synthesize ideas can create learners with options for accessing learning (Gardner, 1983;Marsh, 2012). This coincides with Vygotsky's (1978) zone of proximal development, where pupils are provided with the necessary help and support for tasks they cannot yet perform on their own. Governments, both local and (trans)national, should increase help for schools which want to introduce CLIL, be it on legislative level or through the provision of enough support and guidance for schools who introduce CLIL. A European framework for CLIL implementation and follow-up could be a possible solution, although in order to succeed, it is important to stop using the current rigid national education systems. Finally, it is important to come to a European agreement on the exact definition of CLIL so that CLIL teachers are able to benefit from the experiences and knowledge acquired in various educational settings across Europe (Cenoz, Genesee, & Gorter, 2013).

Limitation of the study
Naturally, the present study has several limitations. One limitation is that there were no standardized math tests for secondary education available at the time of this study. Therefore, three versions of a new mathematical test called the Mathematical Assessment Test-Help had to be developed by the researchers.
Although the stimulus material was prepared as carefully as possible, in close collaboration with experts in the field of mathematical education, it would have been better if validated math tests could have been used.
A second limitation of the study is the time frame. The students in both the CLIL and non-CLIL group were observed and tested throughout only one academic year, or ten calendar months. A more interesting test would be to observe the groups over a longer time in order to ascertain whether or not the two groups continued to develop. A more detailed, longer study to compare the CLIL group's advantages in mathematics over the non-CLIL group and to measure any continued expansion would be hugely beneficial.
Thirdly, a specific test of metalinguistic awareness would have been able to test the increased metalinguistic awareness hypothesis. A test such as the Indonesian Language Test (Ter Kuile et al., 2011) or the Wug task (Berko, 1958;Bialystok & Barac, 2012) could have been implemented in order to strengthen the assertion that CLIL education develops such an awareness.
This increased metalinguistic awareness, along with increased motivation, higher self-confidence, and more parental support (e.g., Jäkel, 2015) seem to be the factors explaining the results. A specific test of metalinguistic awareness could have proven this more conclusively and could have shed light on the degree to which it is a factor. Moreover, in order to be able to draw firm conclusions, in future research moderator variables (including metalinguistic awareness) should be factored in and controlled for, and multivariate analyses (such as factor or discriminant analyses) should be performed in order to determine which variables (the independent variable, CLIL, or other moderator variables, such as increased metalinguistic awareness) are truly responsible for the better mathematical results of the CLIL group compared to the non-CLIL group.

Conclusions
The latest results of the Programme for International Student Assessment (PISA) by the Organisation for Economic Cooperation and Development (OECD) have shown that the need to increase mathematic abilities of pupils in Europe is high (European Commission, 2013). The 2020 goal to have a maximum of 15% being low achievers is far off, and the decline in the rate of low achievers is too slow. In this paper, a suggestion was made to take a different route towards better math achievement, namely the CLIL route. The present results confirmed previous research results (e.g., Dawe, 1983;Clarkson, 1992Clarkson, , 2007 showing that CLIL pupils outperformed their traditionally schooled peers in mathematics. The CLIL group's progress over ten months (one secondary education year) was significantly better than the progress of the traditionally schooled pupils and was already visible after a very short period of time (three months). To conclude, these remarkable results shed new light on the math issue and should be taken into account when new math policy is written; however, more research needs to be done in the future, especially over longer periods of time, but also in different contexts and with different age groups.